UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Validity of the Devereux Early Childhood Assessment instrument Van Leeuwen, Sarah 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2007-0265.pdf [ 4.31MB ]
Metadata
JSON: 831-1.0100659.json
JSON-LD: 831-1.0100659-ld.json
RDF/XML (Pretty): 831-1.0100659-rdf.xml
RDF/JSON: 831-1.0100659-rdf.json
Turtle: 831-1.0100659-turtle.txt
N-Triples: 831-1.0100659-rdf-ntriples.txt
Original Record: 831-1.0100659-source.json
Full Text
831-1.0100659-fulltext.txt
Citation
831-1.0100659.ris

Full Text

V A L I D I T Y OF T H E D E V E R E U X E A R L Y C H I L D H O O D A S S E S S M E N T I N S T R U M E N T ' by S A R A H V A N L E E U W E N B. Sc., University of Northern Brit ish Columbia, 2000 A THESIS S U B M I T T E D IN P A R T I A L F U L F I L M E N T OF T H E R E Q U I R E M E N T S F O R T H E D E G R E E OF M A S T E R OF A R T S in T H E F A C U L T Y OF G R A D U A T E S T U D I E S (School Psychology) T H E U N I V E R S I T Y OF B R I T I S H C O L U M B I A Apr i l 2007 © Sarah van Leeuwen, 2007 Abstract Parent ratings of social-emotional development on standardized assessment instruments for a sample of 69 kindergarten children in a mid-size Canadian city are util ized to examine the validity of the Devereux Early Childhood Assessment ( D E C A ; LeBuffe & Naglier i , 1999a). Results provide support for the D E C A ' s reliability and internal validity when used with a sample different from the standardization sample. In general, results illustrate an expected pattern of convergence and divergence between the D E C A scales and scales from two comparison instruments, the Behavior Assessment System for Children, Second Edition (Reynolds & Kamphaus, 2004) and the Preschool and Kindergarten Behavior Scales, Second Edit ion (Merrell, 2002). The D E C A ' s protective factor scales relate positively to other measures of social skills/adaptive behaviours, and negatively to other measures of problematic/clinical behaviours; these correlations were strongest for the D E C A ' s Self-Control scale, and weakest for the D E C A ' s Attachment scale. The D E C A ' s Behavioral Concerns screener scale related negatively to other measures of social skills/adaptive behaviours, and positively to other measures of problematic/clinical behaviours, particularly those reflecting externalizing behaviour problems. The D E C A is a psychometrically sound instrument that makes an important and unique contribution to the field of social-emotional assessment pf young children. Table of Contents Abstract ... • i i Table of Contents i i i List of Tables v Chapter I: Introduction 1 Purpose of Study and Research Questions. 2 Definit ion of Key Terms 3 Val id i ty . . . . 3 Reliabil ity ; , 4 Kindergarten 4 Chapter II: Literature Review 5 Assessment of Early Childhood Social and Emotional Development..... 5 Third party rating scales 5 Technical adequacy of existing preschool rating scales , 7 Reliabil ity and Validi ty 9 Reliabil i ty... 10 Internal validity 11 Convergent and divergent validity 13 The Devereux Early Childhood Assessment 13 Resilience and social-emotional development 15 Reviews of the D E C A 16 Reliabil ity evidence for the D E C A 17 Validity evidence for the D E C A . . 17 The Preschool and Kindergarten Behavior Scales-2 24 Val idi ty evidence. 25 Convergent validity with the D E C A 26 Divergent validity with the D E C A .... 28 The Behavior Assessment System for Children-2 29 Val idi ty evidence 30 Convergent validity with the D E C A 31 Divergent validity with the D E C A 33 Chapter III: Method 35 Purpose of Study and Research Questions .35 Data Collection Procedures. .• 38 Sample size determination 38 Recruitment process .40 Protocol scoring 42 Demographic Questionnaire 42 Description O f Sample .....43 Composition 43 i i i Student demographics 44 Rater demographics -44 Representativeness of sample 45 Chapter IV: Results 48 Research Question 1: Reliabil ity And Internal Val idi ty 48 Alphas on subscales and total 48 Item-scale correlations. 49 Scale correlations 51 Mean scores 52 Research Question 2: Convergent Val idi ty Wi th The P K B S - 2 . 61 Research Question 3: Divergent Val idi ty With The P K B S - 2 62 Research Question 4: Convergent Validity Wi th The B A S C - 2 63 Research Question 5: Divergent Val idi ty With The B A S C - 2 65 Chapter V : Discussion 68 Reliabil ity and Internal Val idi ty 68 Convergent and Divergent Val idi ty Evidence. 69 D E C A Initiative scale 69 D E C A Self-Control scale 70 D E C A Attachment scale 71 D E C A Behavioral Concerns scale... 71 Limitations of Study 72 Strengths of Study 72 Implications for Other School Psychologists 73 Conclusion ..: 74 * References 76 Appendix A : Item- and Scale-level comparisons of the D E C A , B A S C - 2 , & P K B S - 2 , and scale descriptions 85 Appendix B: Reliabil ity Statistics for the D E C A , B A S C - 2 , & P K B S - 2 . 95 Appendix C : Demographic Questionnaire .98 Appendix D: Letters to Participants : - - .101 Appendix E: U B C Behavioural Research Ethics Board Certificate of Approval 107 List of Tables Table 2.1 Pearson Product-Moment Correlations between the D E C A and the B A S C 20 Table 3.1 Demographic Characteristics of Parent Respondents (N = 69).. 45 Table 4.1 Internal Consistency Coefficients for the D E C A 48 Table 4.2 Item-Scale Correlations for the D E C A 50 Table 4.3 D E C A Scale Correlations. 51 Table 4.4 D E C A Mean Scale Scores 52 Table 4.5 P K B S - 2 Mean Scale Scores Compared to Expected Mean of 100 54 Table 4.6 B A S C - 2 Mean Scale Scores Compared to Expected Mean of 50 55 Table 4.7 Correlations between all Scales and Subscales: D E C A , P K B S - 2 , and B A S C - 2 58 Table A l . Item-Level Comparison of D E C A and B A S C - 2 86 Table A 2 . Item-Level Comparison of D E C A and P K B S - 2 89 Table A 3 . Number of Similar Items from B A S C - 2 and P K B S - 2 Loading onto D E C A scales 92 Table A4 . Descriptions of Scales on the D E C A , P K B S - 2 , and B A S C - 2 93 Table B l . Normative Sample Reliability Statistics for the D E C A , P K B S - 2 , and B A S C - 2 P R S - P 96 v Chapter I: Introduction The importance of early childhood social emotional development, and effective interventions to promote this development, is well established (e.g., Cicchetti & Toth, 1997; Kei th & Campbell, 2000; Meisels & Atkins-Burnett, 2000; Raver, 2002; Shonkoff & Phil l ips, 2000). However, there remains a need for reliable and valid assessment instruments to guide such interventions (Bracken, Keith, & Walker, 1998; Denham & Weissberg, 2004; Drotar, 2002; Merrel l , 2002. A review of the technical adequacy of existing third-party rating scales of preschool behaviour and social emotional functioning found that, although more recently published instruments met the majority of the recommended criteria, most of the rating scales were severely limited in critical areas (Bracken et al., 1998). Thus, the development and ongoing research of such instruments is an important area of investigation. The Devereux Early Childhood Assessment ( D E C A ; LeBuffe & Naglier i , 1999a) is an third-party rating scale that is part of a promising, strength-based primary prevention program designed to foster resilience and healthy social and emotional growth by promoting the development of within-child protective factors in children aged two to five years. The D E C A provides measures of within-child protective factors labelled Attachment, Initiative, and Self-Control, as well as a screener for Behavioral Concerns. From the observations as reported on the behaviour ratings, plans and goals to increase protective factors and decrease behavioural concerns in individual children and/or whole classrooms can be developed, implemented, and monitored v ia reassessment with the D E C A (LeBuffe & Naglier i , 1999b). Although at the time of this study the D E C A has not yet been employed or evaluated in published research studies, it has received positive 1 reviews (Buhs, 2003; Chittooran, 2003; Squires, Bricker, Heo, & Twombly, 2001), and is being used by numerous early childhood organizations (Devereux Early Childhood Institute, n.d..). Given the appeal of this program, the unique construct the D E C A is designed to measure, and broader issues regarding the adequacy of early childhood assessment instruments, it is important to gain more information about the D E C A , particularly in terms of its validity. Purpose of Study and Research Questions The purpose of this study is to contribute to the body of knowledge regarding the D E C A ' s psychometric properties, and in particular to evidence regarding the validity of the D E C A . The first set of analyses examine i f the D E C A demonstrates reliability and internal validity with this sample of kindergarten children from a mid-sized Canadian city. The second set of analyses investigate convergent and divergent validity evidence regarding the constructs measured on the D E C A in comparison to constructs measured on other established and psychometrically sound rating scales, the Preschool and . Kindergarten Behavior Scales, Second Edit ion ( P K B S - 2 ; Merrel l , 2002) and the Behavior Assessment System for Children, Second Edit ion ( B A S C - 2 ; Reynolds & Kamphaus, 2004). The P K B S - 2 (Merrell, 2002) is a 76 item parent rating scale used to evaluate social skills and problem behaviours in children aged 3 to 6 years for the purposes of screening, intervention planning, or as part of a multi-axial assessment battery for formal identification and classification. The B A S C - 2 (Reynolds. & Kamphaus, 2004) is a 134 item rating scale designed to facilitate diagnosis and classification of children's emotional and behavioural disorders and aid in treatment plan design by measuring both positive (adaptive) and negative (clinical) dimensions of personality and behaviour. 2 Research question 1. Do investigations of test content, internal structure, and mean scale scores using this study's Canadian sample provide support for the reliability and internal validity of the Devereux Early Childhood Assessment? Research question 2. What similarities are there in the information provided on the Devereux Early Childhood Assessment Devereux Early Childhood Assessment when compared to information provided on the Preschool and Kindergarten Behavior Scales, Second Edition (Merrell, 2002)? Research question 3. What differences are there in the information provided on the Devereux Early Childhood Assessment when compared to information provided on the Preschool and Kindergarten Behavior Scales, Second Edit ion (Merrell, 2002)? Research question 4. What similarities are there in the information provided on the Devereux Early Childhood Assessment when compared to information provided on the Behavior Assessment System for Children, Second Edit ion (Reynolds & Kamphaus, 2004)? Research question 5. What differences are there in the information provided on the Devereux Early Childhood Assessment when compared to information provided on the Behavior Assessment System for Children, Second Edit ion (Reynolds & Kamphaus, 2004)? Definition of Key Terms Reliability. The consistency or stability of a measure when used on different populations (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). 3 Val id i ty . A unitary concept describing the degree to which evidence and theory support the intended interpretations of test scores for the proposed purpose (American Educational Research Association et al., 1999). One of the most common methods of investigating an assessment instrument's validity is traditionally referred to as convergent and divergent construct validity, which involves considering the extent to which the instrument's scales correlate with other instruments that have been subjected to a . program of validation research. K indergar ten. The first year of children's formal schooling; in Brit ish Columbia, children must be 5 years old by December 31 of the current year in order to register in kindergarten. 4 Chapter II: Literature Review Assessment of Early Childhood Social and Emotional Development Investigations of early childhood development have yielded an extensive multidisciplinary knowledge base that, while complex and evolving, arrives at a common conclusion: children's early experiences, particularly in social and emotional domains, build the foundation for all subsequent learning and development. A growing body of evidence indicates that, once established, social/emotional difficulties tend to remain stable or intensify over time and be highly resistant to change (Raver, 2002; Fei l , Severson, & Walker, 1998). However, there is also considerable evidence that early, effective interventions can change the balance between risk and protective factors in favour of more adaptive short- and long-term outcomes (Cicchetti & Toth, 1997; Shonkoff & Phil l ips, 2000). Thus, there is a need for sound assessment procedures and instrumentation to facilitate early identification and effective intervention for young children who may be at risk for poor developmental outcomes (Keith & Campbell, 2000; Meisels & Atkins-Burnett, 2000; Raver, 2002). This need has facilitated the development of guidelines for accurate and appropriate assessment techniques, as well as minimum standards for the technical adequacy of assessment instruments (e.g., Bracken, 1987; Bracken, Kei th, & Walker, 1998; Goal 1 Early Childhood Assessments Resource Group, n.d.; Knoff, 1992; Meisels & Atkins-Burnett, 2000; Salvia & Ysseldyke, 2001, Shepard, Taylor, & Kagan, 1996; Shonkoff & Phil l ips, 2000). Third party rating scales. Primary methods for assessing young children's social and emotional functioning include direct child assessment methods (e.g., observations, interviews with children, play-based and projective techniques) and 5 gathering information from third-party informants (e.g., interviews with adults, parent and teacher rating scales). Each of these methods provides unique information and has distinct advantages and disadvantages (Keith & Campbell , 2000). Whether used in isolation as a screening tool or as part of a comprehensive assessment of children's social, emotional, and behavioural development, obtaining third party ratings using standardized instruments is well-supported and among the most commonly employed procedures (Diamond & Squires, 1993; Kei th & Campbell , 2000; Meisels & Atkins-Burnett, 2000; Merrel l , 2002; Squires et al., 2001). The literature contains clear guidelines regarding the technical adequacy of such instruments (e.g., Bracken, Kei th, & Walker, 1998), as wel l as their general design and use. The primary advantage of third party rating instruments over other procedures is time and cost efficiency; rating scales can yield a significant amount of information and are generally inexpensive and relatively simple to complete and score (Keith & Campbell, 2000; Meisels & Atkins-Burnett, 2000; Merrel l , 2002; Raver & Zigler, 1997; Squires et al., 2001). A second advantage is that parents, teachers, and other caregiving adults observe children over time and in different contexts, and thus can provide the most complete and accurate picture of a child's functioning, including information on low-frequency but important behaviours (Keith & Campbell , 2000; Merrel l , 2001; Squires et al., 2001). The use of parents as informants also serves to increase parental participation in the assessment and intervention processes (Diamond & Squires, 1993; Drotar, 2002; Meisels & Atkins-Burnett, 2000; Squires et al., 2001). There are also disadvantages to the use of third-party rating instruments. A problem common to all third-party rating instruments is the potential for construct-6 irrelevant variance associated with the persons providing the ratings; this is, variations in scores that are due to factors associated with the raters (e.g., positive or negative response bias, difficulty reading questions) rather than to true differences on the constructs being rated (Keith & Campbell, 2000; Squires et a l , 2001). A second major limitation of this assessment method is the questionable technical qualities of rating instruments, which appears to be a particular problem with preschool versions of social-emotional rating instruments (Keith & Campbell , 2000). However, particularly in the case of social emotional assessment in young children, the advantages, and oftentimes the necessity, of gathering information using third-party rating scales outweigh the disadvantages. Despite obvious need and clear guidelines, there remains a dearth of rating scales specifically developed to assess young children's behavioural, social and emotional development (Bracken et al., 1998; Denham & Weissberg, 2004; Merrel l , 2002). There is a critical need to validate measures and describe how they can be used to guide intervention for young children with behavioural problems (Drotar, 2002). Accurate, useful, and affordable screening measures have been particularly slow to emerge (Squires et al., 2001). O f the instruments that are available, many have weak psychometric properties or insufficient evidence supporting their use (Bracken et al ; 1998; Kei th & Campbell, 2000; Meisels, 1999). Technical adequacy of existing preschool rating scales. Bracken, Keith, and Walker (1998) have reviewed 13 third-party rating instruments of preschool behaviour and social-emotional functioning in terms of their technical adequacy using the same criteria initially proposed by Bracken (1987) for preschool instruments used for placement decisions or individual diagnostic assessment of skills and abilities. These 7 specific criteria include: (a) a median subtest/subscale internal consistency of .80 or greater, (b) total test internal consistency of .90 or greater, (c) total test stability coefficient of .90 or greater, (d) interrater reliability .90 or greater, (e) an average subtest/subscale floor or ceiling at least two standard deviations beyond the normative mean subtest/subscale score, (f) a total test floor or ceiling at least two standard deviations beyond the normative mean total test score, and (g) an item gradient no steeper than three items per standard score standard deviation. In addition, the instruments' standardization samples were evaluated according to their size, representativeness, and whether they were nationally or locally/regionally normed. The minimum internal consistency criteria were based on earlier recommendations made by Nunnally (1978, as cited in Bracken, 1987), and Salvia and Ysseldyke (1981, as cited in Bracken, 1987), and the criteria for subtest and total test floors and item gradients were based on Bracken's determination of the minimal levels that allow for meaningful differentiation among differing levels of abilities, as well as meaningful differentiation of low-functioning children from those functioning in the average or low average ranges. Bracken, Keith, and Walker's (1998) review concluded that overall, most of the rating scales were severely limited in critical areas of technical adequacy. However, more recently published instruments such as the Behavior Assessment System for Children (Reynolds & Kamphaus, 1992), Chi ld Behaviour Checklist (Achenbach, 1991), Conners' Rating Scales (Conners, 1990), Preschool and Kindergarten Behavior Scales (Merrell, 1994), Social Skil ls Rating System (Gresham & Ell iot, 1990), and the Vineland Adaptive Behavior Scales (Sparrow, Bal la, & Cicchetti, 1984) met or partially met the majority of these evaluation criteria, and thus had a greater degree of technical adequacy. 8 It is noted that there was considerable variability in the number of instruments that met the desired criteria in various areas, and that none of the instruments fully met the desired criteria for interrater reliability. Although the reliability of a rating instrument can be easily evaluated according to objective criteria, validity is a much broader concept that is harder to evaluate and very difficult to quantify. In fact, these researchers did not attempt to evaluate the quality of the validity evidence for the assessment instruments they reviewed, but simply noted whether various types of validity evidence were present or absent in the test manual. L ike their advice on evaluating the importance of the various psychometric qualities of assessment instruments, Bracken et al. (1998) state that it is up to examiners to judge for themselves whether the validity evidence presented for a particular instrument is sufficiently convincing to warrant its use for a particular purpose. Rel iabi l i ty and Val id i ty Among the multiple factors that should be considered when evaluating the quality and usefulness of an assessment instrument, reliability and validity are central considerations. In fact, most agree that validity is the most fundamental consideration in evaluating assessment instruments (e.g., American Educational Research Association et al., 1999; Salvia & Ysseldyke, 2001). The reliability of an assessment instrument is integral to its validity to the extent that reliability can be seen as an aspect of validity (Salvia & Ysseldyke, 2001). According to the Standards for Educational and Psychological Testing (American Educational Research Association et al., 1999), validity is a unitary concept describing the degree to which evidence and theory support the intended interpretations of test scores for the proposed purpose. These standards emphasize the importance of accumulating types of evidence that support the validity of 9 the test, rather than the distinct types of validity that haye traditionally been considered (content, criterion, and construct; American Educational Research Association et al., 1999). Accordingly, isolated pieces of evidence cannot answer whether or not an instrument "has" the three distinct types of validity (Hubley & Zumbo, 1996). Instead, validation is an evolving, integrative process in which various lines of evidence are amassed and considered in terms of the degree to which they support the adequacy and appropriateness of inferences and actions suggested by the assessment results (Hubley & Zumbo, 1996; Kazdin, 1995; Messick, 1989). Rel iabi l i ty . Reliabil ity refers to the consistency or stability of a measure when used on different populations (American Educational Research Association et al., 1999). The reliability of assessment scales should be evaluated across different samples in order to judge whether the instrument and the construct it measures are appropriate (i.e., valid) for the particular sample under investigation (Kazdin, 1995; Pedhazur & Pedhazur Schmelkin, 1991). One common type of reliability is internal consistency reliability, which involves considering intercorrelations between parts of a measure using Cronbach's (1990) coefficient alpha method. Min imum recommended criteria are a median subtest internal consistency of .80 or greater, and a total test internal consistency of .90 or greater (Bracken, 1987; Salvia & Ysseldyke, 2001); these recommendations reflect in part the fact that internal consistency reliability is positively related to the number of items on a scale. Changes in the scale's alpha when individual items are deleted is also an important consideration, as i f the alpha improves the item is l ikely not a valid inclusion on the scale. It is anticipated that the alphas obtained from this study's 10 sample wi l l meet recommended standards, and that the elimination of individual items wi l l not change the scale alphas. Another source of reliability evidence comes from evaluating the mean scale scores when an assessment instrument is used with a new but comparable population. Support for the test's stability is provided i f the mean scale scores are equivalent to those found during the norming process. A s the D E C A was standardized using T-scores, it is anticipated that the mean scale scores for the D E C A in the current study's sample wi l l not differ significantly from a mean of 50. Internal val idi ty. Analysis of the internal structure of a test indicates the degree to which relationships among test items or scales conform to the theoretical construct supporting test interpretation (American Educational Research Association et al., 1999). Internal validity involves evidence based on the content and internal structure of a measure, traditionally referred to as content and construct validity. One source of content validity evidence for a measure involves analyzing how wel l individual items fit within the domains they have been placed by considering item-scale correlations. For an item to be a valid inclusion on a scale, the correlation should be at least .25 or .30 (Salvia & Ysseldyke, 2001). Items that do not correlate to this degree with their scales weaken the internal validity of the test, as they are likely not homogenous (i.e., measuring the same ski l l or trait; Salvia & Ysseldyke, 2001). Items that correlate too strongly with their scales are also suspect, as this may suggest that the items were selected on the basis of high correlations, rather than because they are good representations of the construct the scale is intended to measure. 11 In addition to the exploratory and confirmatory factor analysis generally conducted during the development of a measure, evidence of construct validity can be obtained by examining the relationships between scales on a measure using bivariate Pearson product-moment correlations. The resultant interrelationships should demonstrate the distinctiveness of each scale, and should be consistent with theoretically expected relationships between the constructs each scale is intended to measure (American Educational Research Association et al., 1999). Although there is not yet sufficient theoretical information regarding the relationship between various within-child protective factors, and their relationship to problem behaviours, a pattern similar to that found in previous research with the D E C A (Devereux Early Childhood Institute, n.d.b; Jassal et al., 2005; Tran & Shapiro, n.d.) is expected. That is, the D E C A ' s individual protective factor scales wi l l correlate with each other to a moderate degree, and these scales wi l l correlate more strongly to the Total Protective Factors scale than to each other. Based on previous research identifying an inverse relationship between social skills and behaviour problems (e.g., Merrel l , 2002), as well as previous research with the D E C A , it is expected that the Behavioral Concerns scale and the protective factor scales wi l l have negative correlations. Although it is difficult to predict the strength of these relationships on a theoretical basis, it is expected that the strongest negative correlation wi l l be found with the Self-Control scale, as was the case in one previous investigation (Jassal et al., 2005). However, in another investigation, the strongest negative correlation was found between teacher ratings of Behavioral Concerns and Attachment (Devereux Early Childhood Initiative, n.d.b). 12 Convergent and divergent validity. One of the most common methods of gathering validity evidence for an assessment instrument involves considering the extent to which it is correlated with other instruments that have been subjected to a program of validation research, traditionally called convergent and divergent construct validity. However, interpreting correlations between instruments is complex and requires the consideration of multiple factors (Kazdin, 1995). For example, when comparing scales, the validity coefficients obtained are affected by the validity of both the target test and the comparison test (Bracken, 1987). Another challenge to using this method of gathering validity evidence occurs when the instrument being validated was designed to measure unique constructs not represented on other assessment instruments, as is the case with the D E C A (LeBuffe & Naglier i , 1999a; Naglieri & LeBuffe, 2005). It is a difficult endeavour to establish whether the observed relationships between test scores are consistent with theory and research evidence regarding the relationships between the constructs the test is intended to measure. This is a particular challenge for the current study, as the research on the many variables associated with resilience, and their relationships to each other and other constructs such as social skil ls, is still evolving (Naglieri & LeBuffe, 2005). In cases such as this where there are no acceptable benchmark instruments against which to compare a new instrument, K l ine (1993) recommends correlating the new test with other tests that imperfectly measure the variables of interest and viewing moderate correlations (around .4 to .5) as part of the evidence of the construct validity of the test. Despite these challenges, the Standards for Educational and Psychological Testing state that analyzing the relationship between test scores and external variables, 13 such as scores from other tests, provides an important source of validity evidence (American Educational Research Association et al., 1999). Convergent evidence is provided by moderate or strong relationships between test scores and other measures intended to assess the same or similar constructs, and divergent evidence is provided by weak or nonexistent relationship between test scores and other measures intended to assess dissimilar constructs, as well as negative relationships between theoretically opposing constructs (American Educational Research Association et al., 1999). Thus, convergent and divergent evidence operate together to contribute to construct validity by identifying what the construct is, and is not, like (Kazdin, 1995). The Devereux Ea r l y Ch i ldhood Assessment The focus of the current investigation is the D E C A , a rating scale containing 37 descriptors of child behaviours that adult respondents rate on a 5-point frequency scale. This rating scale is part of a strength-based primary prevention program focusing on within-child protective factors derived from research on resilience. The scales are intended to: (1) identify children who are have not yet sufficiently developed protective factors for targeted curricular interventions, (2) screen for children who may be exhibiting emotional and or behavioural problems for potential referral to a mental health professional, and (3) generate classroom profiles indicating the relative strengths of all children, so that classroom design and instructional strategies can build upon these strengths to facilitate healthy social and emotional growth in all children (LeBuffe & Naglier i , 1999b). The D E C A provides scores for an overall Total Protective Factors scale, empirically derived within-child protective factor scales labelled Initiative, Self-Control, and Attachment, and a screening scale measuring Behavioral Concerns. 14 (Appendix A , Table A 4 contains brief descriptions of what these scales are intended to measure.) A l l scores are presented as T-scores, with a mean of 50 and a standard deviation of 10. Scores one standard deviation beyond the mean in the desired direction indicate a 'strength,' and scores one standard deviation beyond the mean in the undesirable direction indicate a 'concern ' . Resilience and social-emotional development. Although the science of resilience is as yet relatively immature and poorly differentiated (Dol l & Lyon, 1998; Werner, 2000), most researchers agree that resilience refers to positive outcomes or adaptation in the face of adversity, risk, or stress. Research on individuals who demonstrate resilient outcomes has identified an incredibly wide range of risk and protective factors at three levels - the individual, the immediate family, and the larger social context (Masten, Best, & Garmezy, 1990; Rutter, 2000; Werner, 2000). Protective factors are defined as characteristics or processes that moderate or buffer the negative effects of stress or adversity, resulting in more positive behavioural or psychological outcomes than would be expected in their absence (Masten & Coatsworth, 1998). Many of the individual, or within-child protective factors identified in various resilience studies are similar, i f not identical, to factors studied under the broad construct of social and emotional development. The Consortium on the School-Based Promotion of Social Competence states "there is a clear link of skills related to social competence and the concept of risk and protective factors" (1996, p. 277). Naglieri and LeBuffe (2005) state that a wide variety of variables have been measured in an attempt to identify those variables that lead to, and therefore can be used to predict, resilience. However, they also caution that measurement of these variables is not simple, and it is even more difficult to 15 measure a child's potential to be resilient to future life events. Given the appeal of the concept of resilience and it's fit with the trend towards strength-based, early intervention for young children, it is clear that more work is needed to clarify how chi ld, family, and contextual risk and protective factors interact, as well as how these factors are related to other important constructs such as social and emotional development (Drotar, 2002; Naglieri & LeBuffe, 2005). Reviews of the DECA. Reviews of the D E C A are generally positive regarding its design and overall technical quality (Buhs, 2003; Chittooran, 2003; Squires et al., 2001). Buhs (2003) points out some potential weaknesses related to the D E C A ' s treatment of within-child protective factors, stating that the literature on children's resiliency is sufficiently well-developed that the D E C A authors could have listed, a priori, the specific target constructs their items were intended to tap, and described the role these constructs .play in children's resilient responses to stress. Buhs (2003) also cautions the authors' conceptualization of the causal relationship between protective factors and problem behaviours (i.e., that higher levels of protective factors lead to lower levels of behaviour problems), suggesting the possibility that children's existing behavioural orientations may foster or inhibit access to factors that support the development of protective factors. However, Chittooran (2003) points out that resilience is a relatively new area of study and there is little longitudinal research that clarifies the relationship between protective factors, problem behaviours, and positive outcomes. Both Buhs (2003) and Chittooran (2003) agree that the D E C A could be improved by the addition of more items addressing each of the protective factors, and both caution the use of the D E C A with parents because of its low level of interrater reliability. 16 Reliability evidence for the DECA. Although the D E C A has received favourable reviews regarding its psychometric qualities, an examination of the parent rating form finds that it falls short of the minimum standards provided by Bracken et al. (1998) in a number of areas. (Appendix B reproduces the reliability statistics obtained during the norming process.) The internal consistency of the Total Protective Factors scale (.91) and the median internal consistency of the protective factor scales (.82) meet recommended criteria. Test-retest and interrater reliabilities of all five scales are lower than the recommended minimum of .90, ranging from .55 (Attachment scale) to .80 (Initiative scale) for test-retest, and from .21 (Total Protective Factors scale) to .44 (Behavior Concerns scale) for interrater. It should be noted that the test-retest and • Y interrater reliablity statistics are more favourable for teacher ratings. Ceil ings, floors, and item gradients are not reported in the D E C A manual and so cannot be evaluated. Validity evidence for the DECA. Because the D E C A is the only published, norm-referenced rating scale of within-child protective factors, it was not possible during the development, standardization, and initial validation process to compare it to other established instruments that measure the same construct (LeBuffe & Nagl ier i , 1999a). However, several other lines of evidence supporting the validity of the D E C A are presented in the technical manual (LeBuffe & Naglier i , 1999a). The item selection procedures, including a review of the resilience literature, focus groups with parents and preschool teachers, and pilot testing, are cited as evidence of content-related validity. Evidence of criterion-related validity is provided by several investigations. Compared to a community sample, children with identified emotional and behavioural problems had significantly different scores on all five scales, with large effect sizes for the Self-Control, 17 Total Protective Factors, and Behavioral Concerns scales; cut-off scores were able to accurately predict group membership for 69 percent (based on the Total Protective Factors scale) and 71 percent (based on the Behavioral Concerns scale) of this sample (LeBuffe & Naglier i , 1999a). It is noteworthy that the strength-oriented scale is almost as effective as the problem-oriented scale in this regard. Another investigation found that effect size calculations of the mean scale score differences for Black and White children and Hispanic and non-Hispanic children were negligible or small, indicating that scale scores do not differ according to minority group status, a variable thought to be irrelevant to the purpose of the instrument (LeBuffe & Naglier i , 1999a). Perhaps the most compelling validity evidence reported in the technical manual comes from an investigation of the relationship between measures of stress and risk in children and families (measured by parent reports of major life events and daily hassles), Total Protective Factors scores, and Behavioral Concerns scores. Children with high risk and low Total Protective Factors scores had the highest mean score on the Behavioral Concerns scale, children with high risk and high Total Protective Factors scores had a mean Behavioral Concerns score that was 6 points lower, and children with low/average risk and high Total Protective Factors scores had the lowest mean Behavioral Concerns score (LeBuffe & Naglier i , 1999a). The authors conclude that this finding is consistent with resilience theory in that protective factors moderate the effects of risk on behavioural outcomes, suggesting that D E C A scale scores truly relate to resilience in young children. A number of research bulletins posted on the Devereux Foundation's website . (www.devereuxearlychildhood.org) present findings similar to those obtained during the 18 standardization process, and provide further validity-related information about the D E C A , particularly regarding its application in early childcare settings. Information specific to the current study includes D E C A scale intercorrelations for both typical and identified samples, and correlations between the D E C A and measures of school readiness. Within a typical sample, the three protective factor scales were correlated, though no correlation exceeded .66, and these scales were negatively correlated with the Behavioral Concerns scale (Devereux Early Childhood Initiative, n.d.a). Weaker correlations among the three protective factor scales, ranging from .24 to .44, were also found within a sample of children previously.diagnosed with internalizing and/or externalizing disorders. Negative correlations between the protective factor scales and the Behavioral Concerns scale were also found, and for groups of children this correlation was the strongest with the Self-Control scale (Pilecki & Schrage, n.d.). When the D E C A was compared to other measures of children's development, it was found that the protective factor scales correlate positively with measures of social/emotional health and negatively with concerns about social/emotional functioning (Tran & Shapiro, n.d.). Protective factor scores have also been shown to correlate positively with early cognitive and language abilities (Devereux Early Childhood Initiative, n.d.b; Tran & Shapiro, n.d.). Further evidence regarding the convergent and divergent validity of the D E C A is provided in an analysis of correlations between the D E C A and the Behavior Assessment System for Children, First Edit ion (Jassal, van Leeuwen, Ford, & Dahinten, 2005). As shown in Table 2.1, the convergent validity of the D E C A was supported by strong correlations between the D E C A Behavioral Concerns scale and the B A S C Behavior Symptoms Index and Externalizing Problems Composite, and the D E C A Self-Control 19 Table 2.1 Pearson Product-Moment Correlations between the D E C A and B A S C D E C A " : B A S C 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 in — .64 .63 .93 -.37 -.22 -.10 -.31 -.17 -.05 -.24 -.24 .67 -.45 -.33 .69 2 sc .51 — .56 .82 -.57 -.45 -.42 -.38 -.16 -.05 -.30 -.26 .52 -.28 -.39 .66 3 at .66 .40 — .77 -.26 -.15 -.13 -.16 -.11 -.06 -.20 -.31 .56 -.34 -.28 .62 4 tpf .91 .76 .78 — -.47 -.32 -.23 -.35 -.20 -.06 -.30 -.31 .68 -.42 -.39 .75 5 be -.08 -.38 -.12 -.24 — .53 .50 .43 .21 .07 .29 .31 -.40 .27 .43 -.54 6 epc .06 -.49 -.0.3 -.15 .50 . .91 .86 .56 .41 .58 .29 -.17 .36 .79 -.37 7 ag .14 -.44 -.02 -.10 .47 .88 • — .58 .46 .25 .53 .16 -.13 .16 .62 -.36 8 hy -.04 -.43 .05 -.17 .42 .90 .58 ~ .57 .49 .49 .37 -.18 .52 .80 -.31 9 ipc -.10 -.21 -.11 -.16 .29 .34 .30 .32 ~ .87 .85 .42 .02 .33 .83 -.22 10 an -.02 -.11 -.50 -.07 .23 .26 .25 •21 .82 — .56 .36 .17 .26 .70 -.03 Table continues O D E C A B A S C 1 2 3 4. 5 6 7 8 9 10 11 12 13 14 15 16 11 de -.09 -.28 -.14 -.19 .48 .42 .42 .33 .82 .57 — .42 -.13 .36 .80 -.39 12 ay -.08 -.20 -.10 -.15 .32 .40 .27 .44 .56 .60 .57 . — -.23 .47 .63 -.28 13 ss .38 .44 .48 .51 -.32 -.19 -.17 • -.18 -.01 -.03 -.22 -.11 — -.34 -.19 .90 H a t -.27 -.27 -.29 -.26 .24 .34 .20 .38 .12 • 14 .16 .38 -.27 — .67 -.40 15 bsi -.10 -.42 -.10 -.23 .51 .74 .62 .69 .70 .68 .75 .80 -.24 .56 — -.40 16 asc .42 .59 .43 .58 -.39 -.34 -.33 -.29 -.26 -.21 -.37 -.19 .81 -.26 -.40 — Note. Bold print shows significance at p <05. Correlations for females are printed on lower half of table. N = 149 for all correlations expect those with B A S C Internalizing and B A S C Adaptive Skills D E C A scales: 1 in = initiative; 2 sc = self-control; 3 at = attachment; 4 tpf = total protective factors; 5 be = behavioral concerns; B A S C scales: 6 epc = externalizing problems composite; 7 ag = aggression; 8 hy = hyperactivity; 9 ipc = Internalizing Problems Composite; 10 an = Anxiety; 11 de = Depression; 12 ay = Atypicality; 13 ss = Social Skills; 14 at = Attention Problems; 15 bsi = Behavior Symptoms Index; 16 asc = Adaptive Skills Composite. to scale and the B A S C Adaptive Skil ls Composite. The divergent validity of the D E C A was supported by strong negative correlations between the D E C A Self-Control scale and the B A S C Externalizing Problems Composite and Hyperactivity scale. While there were a number of similarities between the two instruments, these data also indicate there are a number of key differences, and demonstrate that the D E C A makes a unique contribution to the assessment of social-emotional-behavioural functioning in young children. A dissertation study using parent ratings on the D E C A as wel l as other measures with. 125 at risk preschool children also provides some validity information about the D E C A , although conclusions must be regarded cautiously because of some notable limitations of the study. When compared to the D E C A ' s normative sample, children in this study received lower scores on the Attachment and Total Protective Factors scales and higher scores on the Behavioral Concerns scale (Baldo, 2000), consistent with theoretical expectations that children deemed 'at risk' differ from a general sample. The girls in this study scored higher than the boys on the Initiative and. Self-Control scales (Baldo, 2000), consistent with the gender differences, deemed to reflect real disparities in how boys and girls behave, that were found in the normative sample (no specific details regarding these gender differences are provided; LeBuffe & Naglier i , 1999a). Children classified as normal on the Denver II Developmental Screening (Frankenburg et al, 1990; i.e., no delays or cautions regarding Personal-Social, Fine-Motor/Adaptive, Language, and Gross-Motor development) had higher scores on the D E C A ' s Initiative, Self-Control, and Total Protective Factors scales than did children rated as having one or more delays or cautions; no differences between these two groups were evident on the Attachment or Behavioral Concerns scales (Baldo, 2000). This finding suggests that children with 22 normal developmental profiles demonstrate higher levels of initiative and self-control, but may also suggest that a certain level of initiative and self-control is required to successfully complete the Denver II. Finally, the D E C A ' s three protective factor scales did not relate to anyo f the risk factors investigated in this study (birth weight, mother's age at child's birth, age of opposite sex parent at birth, number of children in the home, alternative caregivers available to the mother in the household, current employment status of the mother, presence of an older sibling, family income, maternal substance abuse during pregnancy, and mother's education level). The only significant finding here was that having alternative caregivers in the home was associated with lower (more positive) scores on the Behavioral Concerns scale (Baldo, 2000). However, it is not clear whether this is due to actual differences in the childrens' behaviour or to differences in parents' perceptions and reports of behaviour, which may vary based on the amount of support for child rearing that parents receive. There are numerous possible explanations for the lack of significant findings in this study, ranging from ambiguities in the literature on the relationship between contextual risk factors and within-child protective factors to weaknesses in the design and execution of this study. For example, the Denver II is not. generally regarded as a psychometrically strong assessment instrument (Hughes, 1995; Meisels & Atkins-Burnett, 2000). Thus, it is not possible to draw any definite conclusions from these results, except to say that further effort is required to elucidate the relationship between risk and protective factors and the role of the D E C A in measuring protective factors. 23 The Preschool and Kindergarten Behavior Scales, Second Edition The Preschool and Kindergarten Behavior Scales, Second Edit ion ( P K B S - 2 ; Merrel l , 2002) is a parent rating scale used to evaluate social skills and problem behaviours in children aged 3 to 6 years for the purposes of screening, intervention planning, or as part of a multi-axial assessment battery for formal identification and classification. The P K B S - 2 was chosen as a comparison instrument for this study because of its strong psychometric properties, large amount of validity evidence, and similarity to the D E C A . Both instruments focus on typical, general social competencies and problem behaviours as opposed to clinical measures of pathology, both are intended to be used as screening instruments, both were designed specifically for young children rather than being downward extensions of instruments designed for older children, and there is considerable overlap in the constructs each instrument is intended to measure and the items used to describe these constructs (Appendix A , Tables A l to A 3 provides an analysis of items from the P K B S - 2 and B A S C - 2 that are similar or the opposite of items from the D E C A ) . The P K B S - 2 consists of 76 items describing adaptive or positive behaviours and problem behaviours that parents rate on a four-point frequency scale. Ratings yield scores for two major scales, labelled Social Ski l ls Total and Problem Behaviors Total, and five empirically derived subscales labelled Social Cooperation, Social Interaction, Social Independence, Externalizing Problems, and Internalizing Problems. (Appendix A , Table A 4 contains brief descriptions of what these scales are intended to measure.) A l l scores are presented as standard scores, with a mean of 100 and a standard deviation of 15. 24 The P K B S - 2 has the same items and basic scale structure as the original version, of the P K B S , but includes several advantages over the original version, most notably a larger normative sample that conforms to 2000 U S Census demographic data and enhances the racial/ethnic diversity within the normative sample (Merrell, 2002). The P K B S - 2 also has strong psychometric properties when evaluated according to criteria proposed by Bracken et al. (1998). (Appendix B reproduces reliability statistics obtained during the norming process.) Buros reviews of the original version of the P K B S are generally favourable, although they stress the need for additional evidence regarding validity in terms of how this instrument contributes uniquely to screening, diagnosis, and intervention planning (MacPhee, 1998; Watson, 1998). Reviews of the P K B S - 2 are similarly positive (Fairbank, 2005; Madle, 2005), though it is stated that the primary weakness of the P K B S - 2 is its normative information, as it does not appear to be normed using a planned representative sample, and does not provide separate norm tables by age (Madle, 2005). Va l id i ty evidence. Validity evidence presented in the P K B S - 2 examiner's manual (Merrell, 2002) indicates that the test items and scales are consistent and appropriate representations of their behavioural domains; that the test was able to correctly predict the special education classification of 90% of the standardization sample; that the Social Ski l ls and Problem Behaviors scales have moderate negative correlations; and that the scale scores differ as expected on the basis of age, gender, disability status (Merrell, 1994), presence of internalizing disorders (Merrell , 1996a), teacher judgment regarding internalizing and externalizing behaviour problems (Jentzsch & Merrel l , 1996), and A D H D status (Merrell & Wolfe, 1998). The negative correlation 25 between Social Skil ls and Problem Behaviors and the ability of the P K B S - 2 to differentiate between groups are compelling evidence, as the patterns found in these studies are consistent with the literature on social skills and problem behaviour development in young children. Convergent and divergent validity evidence comes from studies comparing parent and teacher ratings on the P K B S - 2 for various samples of children to scores on the Social Ski l ls Rating System (Gresham & Ell iot, 1990), the Matson Evaluation of Social Skil ls with Youngsters (Matson, Rotari, & Helsen, 1985), the Conners Teacher Rating Scales (Conners, 1990), the School Social Behavior Scales (Merrell, 1993), the Walker-McConnel l Scale of Social Competence and School Adjustment (Walker & McConnel l , 1995), the Chi ld Behaviour Checklist Teacher's Form (Achenbach, 1991), and the Adjustment Scales for Children and Adolescents (McDermott, Marston, & Stott, 1993). These comparisons generally found moderate to very strong relationships between P K B S - 2 scales and scales from other instruments measuring similar constructs, weak or nonexistent relationships between P K B S - 2 scales and scales from other instruments measuring dissimilar constructs, and consistent inverse relationships between the P K B S - 2 Social Ski l ls scales and problem behaviour scales from other instruments and between P K B S - 2 Problem Behavior scales and.social skills/prosocial behaviour scales from other instruments (Canivez & Bordenkircher, 20.02; Canivez & Rains, 2002; Jentzsch & Merrel l , 1996; Merrel l , 1996b) Convergent val id i ty wi th the D E C A . Predictions about divergence and convergence between the D E C A and P K B S - 2 are based on theory and common sense, an informal analysis of item similarity, and similarity in the descriptions of each scale's construct. In general, it is expected that moderate to strong positive correlations wi l l be 26 found between the D E C A protective factor scales and the P K B S - 2 social skills scales, and between the D E C A Behavior Concerns scale and the P K B S - 2 problem behaviour scales. More specific predictions are described below. The initiative construct on the D E C A , described as the child's ability to use independent thought and action to meet his/her needs, seems more similar to the P K B S - 2 construct of social independence than to the constructs of social interaction or social cooperation. A n item-level analysis finds that the specific items on the D E C A Initiative scale have more overlap with items on the Social Independence scale than on the other two scales. Thus, the Initiative scale is predicted to relate more strongly to Social Independence than to Social Interaction or Social Cooperation. The Self-Control scale on the D E C A is described as measuring the ability to experience a range of feelings and express them using the words and actions that society considers appropriate. The P K B S - 2 describes the Social Interaction and Social Independence scales as being linked to peer-related forms of social adjustment, whereas the Social Cooperation scale is more reflective of adult-related forms of social adjustment. Conceptually, as well as based on item similarity, the Self-Control scale on the D E C A is expected to relate most strongly to the Social Cooperation scale on the P K B S - 2 . The D E C A Attachment scale is described as primarily measuring a mutual, strong, and long-lasting relationship between a child and significant adults, and the majority of items on this scale describe the child's orientation towards adults; the P K B S -2 Social Interaction and Social Independence scales are described as being more strongly linked to peer-related forms of social adjustment, whereas the Social Cooperation scale is 27 more reflective of adult-related forms of social adjustment. On this basis, the Attachment scale may be most strongly related to the Social Cooperation scale. Conceptually, however, attachment seems to be a unique construct. In addition, an analysis of item similarity finds that very few items on the Attachment scale are similar to items on the P K B S - 2 social skills scales. Thus it is predicted that correlations between the Attachment scale and P K B S - 2 scales wi l l be weak or nonexistent. The D E C A Behavioral Concerns scale is described as measuring a wide variety of problem or challenging behaviours such as aggression, withdrawal, attention, and extreme emotions. O f the ten items on the D E C A Behavioral Concerns scale, seven are similar to items on the P K B S - 2 Externalizing Problems scale, but only two are similar to items on the Internalizing Problems scale. Thus, it is expected that this D E C A scale wi l l correlate more highly with the P K B S - 2 ' s Externalizing Problems scale than with the Internalizing or Problem Behavior Total scales. Divergent val idi ty wi th the D E C A . In general, it is expected that moderate negative correlations wi l l be found between the D E C A protective factor scales and the P K B S - 2 problem behaviour scales, and between the D E C A Behavioral Concerns scale and P K B S - 2 social skills scales. For the D E C A Initiative and Attachment scales, it is difficult to predict the strength of these relationships. However, the D E C A Self-Control scale has many items that are the inverse of items describing externalizing problems on the P K B S - 2 . On this basis, as well as the negative relationship between the D E C A Self-Control and Behavioral Concerns scales (recall that the Behavioral Concerns scale is more strongly representative of externalizing than internalizing problems), Self-Control is 28 predicted to have a moderate to strong negative relationship with the P K B S - 2 Externalizing Problems scale, and to be less related to the Internalizing Problems scale. The D E C A Behavioral Concerns scale is described as measuring problems with aggression, withdrawal, attention, and extreme emotions. Logical ly, this seems most in contrast with the P K B S - 2 Social Cooperation scale; the strongest divergence is expected between these two scales. The Behavior Assessment System for Children, Second Edition The Behavior Assessment System for Children, Second Edit ion ( B A S C - 2 ; Reynolds & Kamphaus, 2004) is designed to facilitate diagnosis and classification of children's emotional and behavioural disorders and aid in treatment plan design by measuring both positive (adaptive) and negative (clinical) dimensions of personality and behaviour. Separate rating forms are provided for parents and teachers, and for different age ranges (preschool, child, and adolescent); the scales and composites for each age range contain essentially the same content, with only slight differences due to developmental changes in the manifestations of childhood problems (Reynolds & Kamphaus, 2004). The B A S C - 2 was chosen as a comparison instrument primarily because it is one of the most commonly-used and respected measures of children's social-emotional and behavioural functioning, and because of its strong psychometric properties. Although the B A S C - 2 ' s intended use is quite different than the D E C A ' s (diagnosis versus screening and program planning), there is overlap in the constructs each instrument is intended to measure and the items used to describe these constructs. Each form contains 134 descriptors of behaviour rated on a four-point frequency scale. The ratings yield scores for a Behavioral Symptoms Index, composites measuring 29 Externalizing Problems, Internalizing Problems, and Adaptive Ski l ls, as well as individual scales measuring Aggression, Hyperactivity, Anxiety, Depression, Somatization, Attention Problems, Atypicality, Withdrawal, Adaptability, and Social Ski l ls. (Appendix A , Table A 4 contains brief descriptions of what these scales are intended to measure.) L ike the D E C A , scale and composite scores on the B A S C - 2 are reported as T-scores, with scores one or more standard deviations from the mean indicating areas of concern. The B A S C - 2 is very similar to the original B A S C , with relatively minor improvements to the scale and item content, normative samples, and psychometric properties (Reynolds & Kamphaus, 2004). Although the B A S C - 2 has not at the time of this proposal in early 2007 received formal external published review, the original B A S C received positive reviews regarding its psychometric properties and clinical utility (Bracken et al., 1998; DiPerna, 2001; Flanagan, 1995; Spies & Jones, 2001). The psychometric properties of the B A S C - 2 parent rating form when used to rate children age 4 to 5 years meet Bracken et al.'s (1998) criteria for internal consistency, but fall short for test-retest and interrater reliability. This is similar to the reliability of the D E C A and P K B S - 2 (Appendix B reproduces reliability statistics obtained during the norming processes for all three instruments.) Va l id i ty evidence. In addition to the ample validity-related evidence for the original B A S C , the B A S C - 2 manual presents various types of validity evidence specific to the second edition parent rating scales for preschoolers (PRS-P). Content related validity evidence indicates that the B A S C - 2 P R S - P content relates wel l to standard diagnostic systems such as the D S M ; to expert opinion; and to the perceptions of 30 teachers, parents and children (Reynold's & Kamphaus, 2004). Internal validity evidence is demonstrated by empirical support for scale intercorrelations, and factor analysis supporting the grouping of scales into composites. Consistent with theoretical expectations, correlations within the B A S C - 2 ' s adaptive and clinical scales are positive, and correlations between clinical and adaptive scales are negative (Reynolds & Kamphaus, 2004). Convergent and divergent validity evidence presented in the B A S C - 2 manual for the P R S - P specifically comes from comparisons to only one other instrument, the Achenbach System of Empirical ly Based Assessment Chi ld Behaviour Checklist for Ages 1 - 5 (Achenbach, 1991). In general, correlations between similar composites were high, ranging from .83 for externalizing composites to .69 for internalizing composites, and correlations between similarly named scales were moderate to high, ranging from .35 for anxiety scales to .83 for aggression scales (Reynolds & Kamphaus, 2004). Final ly, the original B A S C parent rating form correlates strongly with the B A S C - 2 parent rating form, indicating that research done on the original version can be generalized to the B A S C - 2 (Reynolds & Kamphaus, 2004). A t the time of this study in early 2007, a search of the literature finds no studies that have utilized the B A S C - 2 P R S - P since its release in 2004. Convergent val idi ty wi th the D E C A . Predictions for expected convergence and divergence with the D E C A scales are based on theory and common sense, an informal analysis of item similarity, and previous research comparing the D E C A and the first edition of the B A S C (Jassal et al., 2005). The expected pattern of correlations is very similar to those expected between the D E C A and P K B S - 2 . In general, it is expected that the D E C A Behavioral Concerns scale and the B A S C - 2 problem behaviour scales wi l l 31 correlate to a moderate to strong degree. It is also expected that moderate to strong positive correlations wi l l be found between the D E C A protective factor scales and the B A S C - 2 adaptive scales, particularly Social Ski l ls and Adaptability. Weaker correlations are expected with the B A S C - 2 Functional Communication and Activities of Dai ly L iv ing scales, and any observed relationship wi l l be more l ikely attributable to the co-occurrence of particular behaviours/traits, rather than to construct overlap. More specific predictions are described below. Conceptually, the D E C A Initiative scale is most closely related to the B A S C - 2 Social Ski l ls and Adaptability scales. Previous research comparing the D E C A and the B A S C - 1 found that Initiative was moderately to strongly correlated with these two B A S C scales, and interestingly this relationship was stronger for boys than girls (Jassal et al., 2005). Thus, it is expected that Initiative wi l l have moderate to strong correlations with these two B A S C - 2 scales. The D E C A Self-Control scale should also correlate with the B A S C - 2 Social Ski l ls and Adaptability scales. Based on analysis of item similarity and previous research (Jassal et al., 2005), these correlations are expected to be stronger than those with the D E C A Initiative and Attachment scales. The D E C A Attachment scale should also correlate with the B A S C - 2 Social Ski l ls and Adaptability scales. Because the Attachment scale appears to be a unique.construct, these correlations are expected to be weaker than those with the Self-Control or Initiative scales. However, previous research with the B A S C - 1 found that Attachment had moderate (for girls) to strong (for boys) correlations with Social Ski l ls and Adaptability (Jassal et al., 2005). 32 As was the case in item analysis between the D E C A and P K B S - 2 , items on the D E C A Behavioral Concerns scale are similar to 13 items from the B A S C - 2 Externalizing Problems Composite, but only 4 from the Internalizing Problems Composite. Thus, the D E C A Behavioral Concerns scale is predicted to correlate most strongly with measures of externalizing behaviour problems on the B A S C - 2 , including the Externalizing Problems Composite, the Behaviour Symptoms Index, and the Aggression and Hyperactivity scales. These predictions are consistent with previous findings using the B A S C - 1 , although it is noted that in this study, for girls only, the Behavioral Concerns scale was also moderately to strongly correlated with the B A S C - 2 Depression scale (Jassal et al., 2005). Divergent val idi ty wi th the D E C A . In general, it is expected that moderate negative correlations wi l l be found between the D E C A protective factor scales and the B A S C - 2 problem behaviour scales, and between the D E C A Behavioral Concerns scale and B A S C - 2 adaptive scales. Conceptually, the D E C A Initiative scale seems most in contrast to the B A S C - 2 Depression, Withdrawal and Anxiety scales; analysis of item similarity confirms this and suggests that Initiative w i l l negatively correlate at least to a moderate degree with these three B A S C - 2 scales. However, previous research with the B A S C - 1 found that for girls, the Initiative scale had very little negative relationship to clinical scales on the B A S C , and for boys, the Initiative scale was most strongly negatively correlated with the Attention Problems and Hyperactivity scales (Jassal et al., 2005). O f the three D E C A protective factor scales, Self-Control is expected to have the strongest negative correlations with clinical scales on the B A S C - 2 , particularly those 33 measuring externalizing behaviour problems. This was the case in previous findings, where Self-Control had a moderate negative relationship with the B A S C - 1 Externalizing Problems Composite, Behavior Symptoms Index, and Aggression and Hyperactivity scales (Jassal et al., 2005), and a weaker relationship with measures of internalizing problems. The D E C A Attachment scale is again expected to have only weak negative relationships with any of B A S C - 2 scales. Item analysis finds that the D E C A Behavioral Concerns scale does not have any items that are the inverse of items on B A S C - 2 adaptive scales. Nevertheless, on a conceptual basis it is expected that this D E C A scale wi l l have moderate negative correlations with the B A S C - 2 Adaptability and Social Ski l ls scales, as was the case in previous analyses (Jassal et al., 2005). 34 Chapter III: Method In this chapter the details of this study's design, methodology, and sample are provided. The purpose of the study and specific questions under investigation related to the methodology of the current study are highlighted, along with hypotheses related to each research question. Participant selection and recruitment procedures, including sample size determination, are then described. The design and structure of the demographic questionnaire and a summary of the demographic information for the study sample is provided. Purpose of Study and Research Questions The purpose of this study is to examine the validity of the Devereux Early Childhood Assessment instrument ( D E C A ) using a Canadian sample of kindergarten-age children. The following are the specific research questions under investigation, with the anticipated outcomes. Research question 1. Do investigations of test content, internal structure, and mean scale scores using this study's Canadian sample provide support for the reliability and internal validity of the D E C A ? Anticipated outcome l a : The internal reliability coefficients derived from this study's sample wi l l not differ significantly from those derived from the D E C A ' s normative sample, and wi l l meet standards recommended in the literature. Anticipated outcome l b : Items wi l l correlate most strongly with their respective scales than with other scales or the Total Protective Factors scale. Alphas for each scale wi l l not improve notably when individual items are deleted. 35 Anticipated outcome l c : The D E C A ' s three protective factor scales wi l l have moderate positive correlations with each other, and moderate negative correlations with the Behavioral Concerns scale. These protective factor scales wi l l be more strongly correlated with the Total Protective Factors score than with each other. Anticipated outcome Id: The mean scale scores of this study's sample wi l l not differ significantly from those derived from the D E C A ' s normative sample. Research question 2. What similarities are there in the information provided on the D E C A when compared to information provided on the P K B S - 2 ? Anticipated outcome 2: Moderate to strong positive correlations are expected between D E C A scales and certain scales from the P K B S - 2 , listed from highest expected correlation to lowest, as follows: 5 2a. D E C A Initiative and P K B S - 2 Social Independence, Social Interaction, and Social Cooperation. 2b. D E C A Self-Control and P K B S - 2 Social Cooperation, Social Interaction, and Social Independence. 2c. D E C A Attachment should not relate meaningfully to any of the P K B S - 2 scales. 2d. D E C A Behavioral Concerns scale and P K B S - 2 Externalizing Problems and Internalizing Problems. Research question 3. What differences are there in the information provided on the D E C A when compared to information provided on the P K B S - 2 ? 36 Anticipated outcome 3: Moderate to strong negative correlations are expected between D E C A scales and certain scales from the P K B S - 2 , listed from highest expected correlation to lowest, as follows: 3a. D E C A Initiative scale and P K B S - 2 Externalizing Problems and Internalizing Problems. 3b. D E C A Self-Control scale and P K B S - 2 Externalizing Problems and Internalizing Problems. These negative correlations should be stronger than those found with the D E C A Initiative scale. 3c. D E C A Attachment should not relate meaningfully to any of the P K B S - 2 scales. 3d. D E C A Behavioral Concerns and P K B S - 2 Social Cooperation, Social Interaction, and Social Independence. Research question 4. What similarities are there in the information provided on the D E C A when compared to information provided on the Behavior Assessment System for Children, Second Edition? Anticipated outcome 4: Moderate to strong positive correlations are expected between the D E C A scales and certain scales from the B A S C - 2 , listed from highest expected correlation to lowest, as follows: 4a. D E C A Initiative and B A S C - 2 Adaptability and Social Ski l ls. 4b, D E C A Self-Control and B A S C - 2 Adaptability and Social Ski l ls. These correlations are expected to be stronger than those for the Initiative or Attachment scales. 37 4c. D E C A Attachment scale and B A S C - 2 Adaptability and Social Skil ls. These correlations are expected to be weaker than those for the Self-Control and Initiative scales. 4d. D E C A Behavioral Concerns scale and B A S C - 2 Externalizing Problems Composite, Behaviour Symptoms Index, and Aggression and Hyperactivity scales. Research question 5. What differences are there in the information provided on the D E C A when compared to information provided on the Behavior Assessment System for Children, Second Edition? Anticipated outcome 5: Moderate to strong' negative correlations are expected between the D E C A scales and certain scales from the B A S C - 2 , listed from highest expected correlation to lowest as follows: 5a. D E C A Initiative and B A S C - 2 Internalizing Problems Composite and scales (Anxiety, Withdrawal, and Depression). 5b. D E C A Self-Control and B A S C - 2 Externalizing Problems Composite and scales (Aggression and Hyperactivity). 5c. D E C A Attachment should not relate meaningfully to any of the B A S C - 2 scales. 5d. D E C A Behavioral Concerns and B A S C - 2 Adaptability and Social Ski l ls. Data Collection Procedures Sample size determination. The desired sample size for this study was determined using several methods: statistical power analysis, examination of available pilot data comparing the D E C A and B A S C , and review of other validation studies. The 38 power of a statistical test is related to three variables: sample size (TV), significance criterion (a), and population effect size (d). Specifying an exact effect size for the current study is difficult, as there are no published studies comparing the D E C A to other similar instruments, and it is expected that the magnitude of the correlations between the D E C A and the other two rating instruments utilized in this study wi l l differ depending upon the degree of similarity between the particular scales being compared. In cases such as this where there are no acceptable benchmark instruments against which to compare a new instrument, Kl ine (1993) recommends correlating the new test with other tests that imperfectly measure the variables of interest and viewing moderate correlations (around .4 to .5) as part of the evidence of the construct validity of the test. These correlations constitute medium/large effect sizes according to criteria provided by Cohen (1992). In a pilot study with 25 participants, significant correlations between the D E C A scales and selected scales from the B A S C - 1 ranged in magnitude from .44 to .65 (Dahinten & Ford, 2004). Thus, a medium effect size of .40 was used in the statistical power analysis. To balance the probability of Type I and Type II errors, Cohen (1992) recommends conventions for the significance criterion to be .05 for a two-tailed test, and power to be .80. Howel l (1997) provides a formula for calculating the sample size required to obtain a desired effect size in correlational analyses; 6 = d (n- l ) ' / 2 . Using Cohen's recommendations for the significance criterion and power, this formula indicates that a sample size of 50 is required to detect a medium (r = .40) effect. This roughly corresponds to the guidelines provided by Cohen for the sample size required to detect an effect size of .40 when using a Mest for the significance of a product-moment correlation coefficient r, with df= N - 2. (That is, a sample size of 85 is required for an effect size of 39 .30, and a sample size of 28 is required for an effect size of .50; by extrapolation, a sample size of 57 is required to detect an effect size of .40). Thus, this analysis indicates that a sample size of 55 to 60 participants wi l l be adequate to detect moderate correlations between scales on the D E C A , B A S C - 2 , and P K B S - 2 . Pilot data from 25 participants (Dahinten & Ford, 2004) indicate that this sample size w i l l be sufficient to detect significant correlations between similar scales from the D E C A and B A S C - 2 . Recrui tment process. Participant recruitment occurred in a series of steps. First, permission to conduct this study and access individual schools was obtained from the Director of School Services in School District 57, Prince George. The Director of School Services was also asked to providesome input as to the individual schools that were most appropriate for the study, based on a list of potential schools prepared by the researcher. Because it was close to the end of the school year, all 25 elementary schools in the district were invited to participate in an effort to obtain a sufficient sample size. Next, the principals of these schools were contacted. The researcher provided an overview of the study's purpose and methods, clearly outlined what would be asked of individual kindergarten teachers, and requested permission to contact kindergarten teachers within that school. Once permission was secured from the principals at 19 of the 25 schools, the third step was to recruit individual classrooms within that school. Kindergarten teachers were contacted v ia an email in which the study's purpose and methods were outlined, as well as what assistance was being requested from the teachers. Teachers were also given an opportunity to ask questions about the study, and were then asked i f they were wi l l ing to participate. A l l teachers who were approached expressed a willingness to participate 40 (although in one case the school's principal distributed the invitation letters and collected the reply forms). Next, all parents (or legal guardians) of 5-year-old students in each participating classroom were invited to participate via letters of invitation distributed by classroom teachers. These letters described the study using easily-understood language, and detailed the amount of time required to complete the study's questionnaires (based on estimates provided in each rating instrument's manual). A l l parents were asked to complete a reply form indicating whether or not they were wi l l ing to participate, and return it to the teacher with their children. Ask ing all parents to return the reply forms was done to provide the researcher with an estimate of the number of parents who actually received and attended to the invitation letters brought home by their children. Teachers were asked to collect and hold these forms for the researcher, who returned to gather them within one to two weeks of their distribution. Parents who indicated a willingness to participate were asked to provide their child's birthday and gender, as wel l as their phone number and mailing address, on the reply form. Those parents who were wi l l ing to participate were then mailed a package of research materials. The package included a brief set of instructions for completing the materials and contact numbers for • questions or concerns; D E C A , B A S C - 2 , and P K B S - 2 protocols; a demographic questionnaire; and a stamped and addressed return envelope. Parents were instructed to complete the entire package in one session, and asked to enter the date on each protocol to evaluate this. The order of the three protocols was counterbalanced, with equal numbers of packages representing the six possible orders. Each set of materials was clipped together, and parents were instructed to complete the protocols in the order in 41 which they were arranged. After completing all forms, parents were instructed to mail them back to the researcher in the stamped and addressed envelopes provided. Reminder phone calls were made to parents who did not return, the packages within four weeks of their mailout. Completion and return of these packages served as the parents' consent to participate. (Appendix C contains all forms and documents used in communication with study participants.) Protocol scoring. A l l 207 protocols were scored by the author as they were received in the mail. D E C A and P K B S - 2 protocols were hand-scored, and B A S C - 2 protocols were scored using computer software. Two practicing school psychologists trained in the administration and scoring of standardized assessment instruments conducted reliability checks. For the first thirty sets of protocols, every fifth set was re-scored by a reliability checker. No errors were found in the original scoring of any of these eighteen protocols. Thus, for the next set of thirty-nine protocols, every tenth one was re-scored by the reliability checker. Across these nine protocols, addition errors on two scales were found and corrected in the data set. Demographic questionnaire. In order to evaluate the composition and representativeness of this study's sample, respondents were asked to complete a brief demographic questionnaire. Questions included: respondent's relationship to the child being rated, the child and respondent's racial/ethnic background, family composition, respondent's age, level of education, and household income category. The questions about education and income were optional. (Appendix C contains a copy of this questionnaire.) 42 Descript ion of Sample Composi t ion. Primary caregivers of kindergarten children were recruited through public schools in a mid-sized community in central Brit ish Columbia, Canada. This age was selected because children's entry into kindergarten is a particularly important time to evaluate their social, emotional, and behavioural development. To ensure the study sample had sufficient variability in terms of socioeconomic factors, a school-level vulnerability index score was used to attempt to recruit equal numbers of participants from schools at high, medium, and low categories of vulnerability. This index score was provided by the Director of School Services based on 2004 statistics from the B C Ministry of Children and Family Development. The vulnerability index is a composite of the school-wide percentages of aboriginal students, special education students, students from families with incomes less than $30000, students from single parent families, and students whose parents have less than high school education. O f the 319 invitations sent home with kindergarten students from 19 schools, 135 primary caregivers returned the response cards, yielding a response rate of 42%. O f these, 115 expressed an interest in participation, yielding a consent rate of 36%. O f the positive respondents, 100 were eligible for the study (15 respondents had children who had already turned six and thus were too old for the study). O f the 100 sets of questionnaires mailed out, 73 were returned and 69 were included in the study (4 sets of questionnaires were eliminated from the study because of problems such as all three questionnaires not being completed or questionnaires being completed by different respondents or on different days). The final sample consisted of parent ratings for 69 kindergarten students, 32 (46%>) boys and 37 (54%) girls. 43 Student demographics. The mean age of students was five years and 8 months (68 .4 months; range 6 4 to 71 months). 3 8 % of these students attended schools in the high vulnerability category, 3 9 % in the medium vulnerability category, and 2 0 % in the low vulnerability category (3%> of the sample were from 'unknown' schools). In terms of race/ethnicity, 77%> of these students were White/Caucasian, 1 3 % were First Nations/Aboriginal, and 1 0 % were 'other' (mixed race, Metis, East Indian, Asian). In terms of family composition, 8 4 % of students were from two parent families, 10%> from single-parent families, and 6%> from blended families. 16%> of these students were only children, 54%> had one sibling, 22%> had two siblings, and 8% had three or more siblings. Rater demographics. O f the 6 9 sets of ratings collected, 87%> were completed by the student's biological mother, 4%> by the student's biological father, 4 % by the student's stepmother, 1%> by the student's stepfather, and 3% by another guardian (grandmother, foster mother). 20%> of the raters were 2 0 to 2 9 years old, 6 1 % were 3 0 to 3 9 years old, and 1 9 % were older than 4 0 . In terms of the race/ethnicity of raters, 81%> were White/Caucasian, 11%> were First Nations/Aboriginal, and 8%> were 'other' (mixed race, Metis, East Indian, Asian). 93%> of the raters responded to the two optional questions about education and income. O f these, 8%> had less than high school education, 28%o had graduated from high school, 3 3 % had taken some college, trades, or technical school, 14%o had taken some university, and 1 7 % had at least one university degree. 2%> of these raters had a total yearly household income of less than $ 1 4 , 9 9 9 , 6%> between $ 1 5 , 0 0 0 and $ 1 9 , 9 9 9 , 6% between $ 2 0 , 0 0 0 and $ 2 9 , 9 9 9 , 1 6 % between 3 0 , 0 0 0 and 3 9 , 9 9 9 , 2 4 % between $ 4 0 , 0 0 0 and $ 5 9 , 9 9 9 , and 4 6 % $ 6 0 , 0 0 0 or more. 4 4 Table 3.1 Demographic Characteristics of Parent Respondents (N = 69) Relationship to student Education Level Biological mother 87% Less than high school 8% . Biological Father 4% Graduated high school 28% Stepmother 4% Some college, trades ' 33% Stepfather 1% Some university 14% Other 3% University degree 17% Age Yearly Household Income 2 0 - 2 9 20% Less than $14,999 2% 3 0 - 3 9 61% $15,000-$19,999 6% 40 or older 19% $20,000 - $29,999 •6% Race/ethnicity $30,000 - $39,999 16% White/Caucasian 81% $40,000 - $59,999 24% First Nations/Aboriginal 11% $60,000 + 46% Other 8% Representativeness of sample. Although equal numbers of schools from the high, medium, and low vulnerability categories were invited to participate, factors such as the school principal's willingness to participate and the number of 5-year-old kindergarten students at each school lead to this sample being underrepresentative of students from schools in the low vulnerability (i.e., higher socioeconomic status) category. However, when the demographic information provided by participating individuals is considered in more detail and compared to information about the 25 45 schools that were invited to participate (school-level vulnerability index) and available 2001 census information for the city of Prince George (Statistics Canada, 2002), it is evident that the current sample is biased towards children and raters from higher socioeconomic levels. Thirteen percent of the students rated were First Nations/Aboriginal, and 11 % of raters were First Nations/Aboriginal. School-level statistics (derived from the vulnerability index) across the 25 schools invited indicate that 22% of students are aboriginal. However, 2001 census data indicates that within the community of Prince George, 10% of individuals are Aboriginal (Statistics Canada, 2002). Thus, this sample underrepresents the First Nations/Aboriginal student population, but is representative in terms of those providing the ratings. Children from single-parent families comprised 10% of this study's sample. In contrast, school-level statistics indicate that 20% of students in the 25 invited schools are from single-parent families. This estimate is consistent with data for the city of Prince George (Statistics Canada, n.d.). Thus, children from single-parent families, and single parents providing ratings, are underrepresented in the current sample. In terms of education level, 8%> of raters had less than a high school education. In contrast, school-level statistics indicate that 27 % of students from.the 25 invited schools-are from families in with the parent(s) have less than high school education. Further, across the community of Prince George, 19% of the population aged 20 to 44 haveless than high school education (Statistics Canada, n.d.). In terms of income level, 14%> of this study's raters have a yearly income lower than $30,000. In contrast, school-level statistics indicate that across the 25 schools, 20%> of students are from families with an 46 income lower than $30,000. This study's sample is underrepresentative of individuals from the lowest education and income categories. 47 Chapter IV: Results Research Question 1: Reliability and Internal Validity Alphas on subscales and total. Cronbach's A lpha was used to investigate the internal reliability of each scale (see Table 4.1). The Total Protective Factors scale alpha is comparable to that reported in the manual (.91). Alphas for the scale scores were also strong, with a median of .82. With the exception of a higher Behavioral Concerns scale alpha in the current sample (.71 in normative sample), these internal consistency coefficients are comparable to those derived from the normative sample. Thus, although the Alpha for the Attachment scale is somewhat weaker than desired, the internal consistency of the D E C A scales within a new sample continue to meet or exceed the minimum standards of a median of .80 for subtests and .90 for total test described by Bracken (1987). Table 4.1 Internal Consistency Coefficients for the D E C A Scales Scale Cronbach's alpha Initiative .83 Self-Control .88 Attachment .73 Total Protective Factors .92 Behavioral Concerns .82 The appropriateness of items included on each scale was also investigated by considering changes in the scales' alpha as individual items were deleted. On the three 48 protective factor scales, the deletion of each item did not change the scale's alpha. On the Behavioral Concerns scale, the deletion of item 8 (fail to show joy or gladness at happy occasion) improved the scale alpha, but only nominally (from .82 to .83). These two types of analysis provide further support for the internal structure of the D E C A scales by demonstrating that the scales continue to have strong internal consistency when used with a new population. Item-scale correlations. The internal validity of the D E C A was investigated by looking at interrelationships between items and their scales using bivariate Pearson product-moment correlations (see Table.4.2). For the Initiative scale, item-scale correlations ranged from .48 to .70. For the Self-Control scale, item-scale correlations ranged from .47 to .79. For the Attachment scale, item-scale correlations ranged from .44 to .67. For all three of these individual protective factor scales, the vast majority of items were more strongly correlated with their respective scales than with the Total Protective Factors scale, providing support for the validity of the structure of these three scales. For the two items on the Initiative scale ('participate actively in make-believe play with others' and 'try or ask to try new things or activities') that were more strongly correlated with the Total Protective Factors scale than with the Initiative scale, the magnitude of this difference was very small. For the Behavioral Concerns scale, item-scale correlations ranged from .36 to .70. This analysis finds that all of the items on the D E C A haye adequate correlations with their respective scales, in most cases far exceeding the minimum requirements of .25 to .30 proposed by Salvia and Ysseldyke (2001). 49 Table 4.2 Item-Scale Correlations for the D E C A Item 2 a 3 a 1 2 a 1 6 a 1 9 a 2 0 a 2 4 a 2 8 a 32 a 3 6 a 13 1 21 25 30 ' 33 ' 34 ' Item-scale Item-TPF correlation correlation .49 .74 .52 .48 .70 .69 .69 .56 .61 .53 .63 .68 .76 .67 .79 .66 .66 .71 .71 .41 .68 .54 .42 .60 .64 .66 .59 .64 .46 .60 .64 .66 .57 .66 .60 .63 .61 .69 Item l c 6 C 10 c 17° 2 2 c 2 9 c 31° 3 7 c 8 d 9 d l l d 14 d 15 d 18 d 23 d 26 d 27 d 35 d Item-scale Item-TPF correlation correlation .53 .59 .59 .46 .44 .58 .64 .67 .36 .37 .68 .42 .39 .65 .68 .59 .65 .70 .47 .54 .42 .22 .23 .60 .72 .57 Note:a Initiative scale; bSelf-Control scale; Attachment scale; dBehavioral Concerns scale 50 Scale correlations. Interrelationships between the five D E C A scales were examined using bivariate Pearson product-moment correlations (see Table 4.3). For the three protective factor scales, intercorrelations ranged between .49 (Attachment and Self-Control) and .71 (Attachment and Initiative). A l l three scales were more strongly correlated with the Total Protective Factors scale, ranging from .79 (Attachment) to .93 (Initiative). This indicates that although there is some overlap in the constructs reflected on the protective factor scales, each scale measures a distinct construct. The Behavioral Concerns scale had moderate negative relationships with the protective factor scales, ranging from -.44 (Attachment) to -.65 (Self-Control). The Behavioral Concerns scale was more strongly negatively related to the Self-Control scale than to the Initiative, Attachment, or Total Protective Factors scale. This pattern of scale intercorrelations is as predicted, and provides further support for the internal validity of the D E C A with a new population. Table 4.3 D E C A Scale Correlations Total Protective Initiative Self-Control Attachment Factors Initiative Self-Control Attachment Behavioral Concerns .67* .71* Total Protective Factors .93* -.51 = .49* .83* -.65* .79* -.44* -.62* Note. Cor re la t ion is significant at the .01 level (2-tailed) 51 M e a n scale scores. The mean scores for each of the five D E C A scales within the current sample (see Table 4.4) were compared to an expected mean of 50 using one-sample t-tests. This sample's mean scores did not differ significantly from the expected , mean on the Initiative, Attachment, and Total Protective Factors scales. However, this sample's mean score was significantly higher on the Self-Control scale t(68) = 2.0, p = .05 (2-tailed) and the Behavior Concerns scale t(68) = 3.2, p = .00 (2-tailed). These differences were gender-specific. On the Self-Control scale, the boys' mean score did hot differ significantly from the expected mean of 50, but the girls' mean score was significantly higher, t(68) = 3.93, p = .00. On the Behavioral Concerns scale, the girls' mean score did not differ significantly from expected, but the boys' mean score was significantly higher, t(68) = 3.90, p = .00. Gender differences in mean scale scores, deemed to reflect real differences in how boys and girls behave, were also found during the D E C A ' s norming process (LeBuffe & Nagl ier i , 1999a), however no specific details about these differences are provided in the technical manual. Table 4.4 D E C A Mean Scale Scores Compared to Expected Mean of 50 Mean SD t df ~p Total Sample Initiative 51.22 8.69 1.16 68 .25 Self-Control 52.70 10.98 2.04 68 .05 Attachment 48.61 9.92 -1.17 68 .25 Total Protective Factors 50.74 9.73 .63 68 .53 Behavioral Concerns 54.74 12.21 3.22 68 .00 Table continues 52 Mean SD t df p Boys Only r Initiative 49.44 8.46 -.38 31 .71 Self-Control 49.28 12.37 -.33 31 .75 Attachment 47.78 9.75 -1.30 31 .21 Total Protective Factors 48.47 9.95 -.87 31 . .39 Behavioral Concerns 58.28 12.02 3.90 31 .00 rls Only Initiative 52.76 8.71 1.93 36 .06 Self-Control 55.65 8.74 3.93 . 36 .00 Attachment 49.32 10.13 -.41 36 .69 Total Protective Factors 52.70 9.22 1.78 36 .08 Behavioral Concerns 51.68 11.67 :87 36 .39 Comparing boys' and girls' mean scales scores to the expected means on the P K B S - 2 and B A S C - 2 provides further information about these gender differences (see Tables 4.5 and 4.6). Both boys and girls scored significantly higher than expected on all of the P K B S - 2 social skil ls scales (with the exception of the boys' score on the Social Cooperation scale) and the B A S C - 2 adaptive scales. In addition, girls scored significantly lower than expected on P K B S - 2 Externalizing Problems, Internalizing Problems, and Problem Behavior Total, as wel l as on the B A S C - 2 Externalizing Problems Composite and Aggression scale. 53 Table 4.5 P K B S - 2 Mean Scale Scores Compared to Expected Mean of 100 Mean SD t df P Total Sample Social Cooperation 104.00 11.56 2.85 67 .01 Social Interaction 106.76 8.80 6.34 67 .00 Social Independence 105.16 10.01 4.25 67 .00 Social Ski l ls Total 106.18 9.72 5.24 67 .00 Externalizing Problems 99.53 13.91 -.28 67 .78 Internalizing Problems 98.59 12.66 -.92 67 .36 Problem Behavior Total 99.00 13.39 -.62 67 .54 Boys Only Social Cooperation 102.61 12.15 1.20 30 - .24 Social Interaction 106.19 8.41 4.10 30 .00 Social Independence 104.48 8.42 2.97 30 .01 Social Ski l ls Total 105.13 8.84 3.23 30 .00 Externalizing Problems 104.06 14.09 1.60 30 .12 Internalizing Problems 101.29 13.79 .52 30 .61 Problem Behavior Total 102.87 13.45 1.19 30 .24 Girls Only Social Cooperation 105.16 11.10 2.83 3 6 , .01 . Social Interaction 107.24 9.20 4.79 36 .00 Social Independence 105.73 , 11.27 3.09 36 .00 Table continues 54 Mean SD t df p Social Ski l ls Total 107,05 10.44 4.11 36 .00 Ext. Problems 95.73 12.74 -2.04 36 .05 Int. Problems 96.32 11.33 -1.97 36 .06 Problem Behavior Total 12.62 12.62 -2.05 36 .05 Table 4.6 B A S C - 2 Mean Scale Scores Compared to Expected Mean of 50 Mean SD t ' df p Total Sample Ext. Problems Comp. 49.42 11.56 -.42 68 .68 Int. Problems Comp. 51.90 10.54 1.50 68 •14 Behavior Symptoms Index 49.84 10.94 -.12 68 .90 Adaptive Skil ls Comp. 57.01 8.30 7.02 68 .00 Hyperactivity 50.00 11.45 .00 68 1,0 Aggression 48.23 10.63 -1.38 68 .17 Anxiety 53.59 11.22 2.66 68 •01 Depression 51.52 10.54 1.20 68 .24 Somatization 49.38 8.43 -.62 68 .54 Atypicality 50.81 11.84 .57 68 .57 Withdrawal 49.14 9.43 -.75 68 .45 Attention Problems 49.58 10.13 -.35 68 .73 Adaptability 53.87 10.08 3.19 68 .00 Table continues 55 Mean SD t df p Social Skil ls 54.93 8.17 5.01 68 .00 Activit ies of Daily L iv ing 57.01 7.87 7.40 68 .00 Functional Communication 56.12 8.40 6.05 68 .00 Boys Only Ext. Problems Comp. 52.78 • 12.65 1.24 31 . .22 Int. Problems Comp. 53.94 11.37 1.96 31 .06 Behavior Symptoms Index 53.03 11.50 1.49 31 .15 Adaptive Skil ls Comp. 55.72 7.43 4.36 31 ,00 Hyperactivity 53.19 . 12.58 1.43 31 .15 Aggression 52.06 . 11.56 1.01 31 .32 Anxiety 55.56 12.45 2.53 31 .02 Depression 53.78 11.12 1.92 31 .06 Somatization 49.84 8.87 -.10 31 .92 Atypicality 53.19 11.75 1.54 31 .14 Withdrawal 49.06 10.32 -.51 31 .61 Attention Problems 52.31 10.18 1.29 31 .21 Adaptability 52.81 10.81 1.47 31 .15 Social Skil ls 53.16 8.08 2.21 31 .04 Activit ies of Dai ly L iv ing 56.00 6.63 5.12 31 .00 Functional Communication 56.59 7.74 4.82 31 .00 Table continues 56 Mean SD t df p Girls Only Ext. Problems Comp. 46.51 9.78 -2.17 36 .04 Int. Problems Comp. 50.14 9.56 .09 36 .93 Behavior Symptoms Index 47.08 9.76 -1.82 36 .08 Adaptive Skil ls Comp. 58.14 8.94 5.54 36 .00 Hyperactivity 47.24 9.72 -1.73 36 .09 Aggression 44.92 8.60 -3.56 36 !00 Anxiety 51.89 9.90 1.16 36 .25 Depression " 49.57 9.74 -.27 36 .79 Somatization 48.97 8.12 -.77 36 .45 Atypicality , 48.76 11.69 -.65 36 .52 Withdrawal 49.22 8.7 -.55 36 .59 Attention Problems 47.22 9.60 -1.76 36 .09 Adaptability 54.78 9.45 3.08 36 .00 Social Ski l ls 56.46 8.04 4.89 36 .00 Activit ies of Dai ly L iv ing 57.89 8.81 5.45 36 .00 Functional Communication 55.70 9.02 3.85 36 .00 57 Table 4.7 Correlations 3 between all Scales and Subscales: D E C A , P K B S - 2 , and B A S C - 2 D E C A . PKBS-2 BASC-2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 in -2 sc 67 • - -3 at 71 49 -4 tpf 93 83 79 -5 be -51 -65 -44 -62 -6 sc 40 65 16 50 -55 -7 si 40 38 22 39 -36 55 -8 sn 52 40 16 46 -36 50 57 . -9 sst 54 59 22 56 -52 85 82 83 -10 ep -36 -64 -22 -49 71 -75 -37 -41 -64 - . 11 ip -43 -51 -27 -47 50 -50 -29 -59 -58 67 -12 pbt -44 -63 -28 -52 66 -69 -35 -54 -66 92 91 -Table continues oo D E C A PKBS-2 BASC-2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 " 22 23 24 25 26 27 13epc -30 -60 -17 -43 61 -74 -33 -35 -59 88 54 77 -14 ipc -39 -40 -19 -39 34 -32 -07 -37 -32 38 63 55 42 -15 bsi -54 -68 -27 -60 70 -76 -44 -55 -72 84 69 84 85 64 -16 asc 71 60 44 71 -62 58 58 68 74 -54 -53 -59 -46 -37 -67 --17 hy -31 -56 -15 -43 63 -76 -37 -40 -63 86 -54 76 91 40 86 -50 --18ag -29 -62 -20 -44 63 -71 -29 -26 -53 86 52 75 91 43 84 -47 82 -19an -20 -16 -01 -15 05 -09 04 -25 -13 14 48 33 19 84 37 -11 17 19 --20 de -48 -49 -28 -51 52 -46 -18 -39 -44 52 61 61 53 86 77 ":49 52 55 59 --21 so -25 -34 -18 -30 .25 -20 -02 -23 -20 26 42 . 38 27 68 39 -31 27 29 33 42 --22 ay -40 -52 -22 -47 50 -63 -27 -44 -56 62 56 64 64 62 83 -44 66 61 39 68 39 -23 wi -38 -29 -19 -34 20 -14 -35 -53 -40 23 53 42 18 41 47 -47 18 14 38 35 23 26 --24 at -56 -59 -22 -57 71 -74 -57 -51 -73 71 38 59 65 19 77 -75 69 65 -06 36 15 52 28 --Table continues D E C A ' PKBS-2 BASC-2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 25 ad 58 49 37 59 -45 41 29 53 50 -35 -41 -42 -30 -42 -53 73 -33 -25 -21 -49 -32 -36 -56 -48 ~ 26 ss 66 60 45 68 -56 55 66 50 69 -55 ' -37 -51 -44 -17 -57 81 -45 -49 05 -30 -17 -31 -35 -76 56 -27 al 35 27 13 33 -35 34 31 46 46 -30 -34 -35 -27 -35 -41 63 -38 -26 -11 -37. -40 -30 -16 -42 23 30 ~ 28 fc 45 36 38 47 -47 48 48 49 59 -43 -38 -46 -36 -09 -48 77 -38 -39 07 -22 -09 -36 -25 -58 36 54 42 Note. a Decimal points have been omitted in the correlations. Bold print shows significance atp <05. N = 69 for all correlations. 1 in = initiative; 2 sc = self-control; 3 at = attachment; 4 tpf = total protective factors; 5 be = behavioral concerns; 6 sc = social cooperation; 7 si = social interaction; 8 sn = social independence; 9 sst = social skills total; 10 ep = externalizing problems; 11 ip = internalizing problems; 12 pbt = problem behavior total; 13 epc = externalizing problems composite; 14 ipc = internalizing problems composite; 15 bsi = behavior symptoms index; 16 asc = adaptive skills composite; 17 hy = hyperactivity; 18 ag = aggression; 19 an = anxiety; 20 de = depression; 21 so = somatization; 22 ay = atypicality; 23 wi = withdrawal; 24 at = attention problems; 25 ad = adaptability; 26 ss = social skills; 27 al = activities of daily living; 28 fc = functional communication. o Research Question 2: Convergent Va l id i ty wi th the P K B S - 2 The D E C A Initiative scale had moderate positive correlations with social skil ls scale and subscales from the P K B S - 2 as follows: Social Ski l ls Total (.54), Social Independence subscale (.52), and Social Cooperation and Social Interaction subscales (.40 each). This pattern of correlations is as predicted on a conceptual basis as well as an analysis of item similarity. The D E C A Self-Control scale had a strong positive correlation with the P K B S - 2 Social Cooperation subscale (.65), and moderate positive correlations with the P K B S - 2 Social Ski l ls Total scale (.54), and Social Independence (.40), and Social Interaction (.38) subscales. This pattern of correlations is as predicted based on the fact that the P K B S - 2 describes the Social Interaction and Social Independence scales as being more strongly linked to peer-related forms of social adjustment, whereas the Social Cooperation scale is more reflective of adult-related forms of social adjustment. In contrast, the D E C A Attachment scale had only weak and nonsignificant correlations with all of the P K B S - 2 social skills scales (ranging from .22 to .16). This indicates, as was predicted, that the Attachment scale measures a construct that has little in common with social skills as measured on the P K B S - 2 . The D E C A Total Protective Factors scale had moderate positive correlations with the P K B S - 2 Social Skil ls Total scale (.56), Social Cooperation subscale (.50), and Social Independence subscale (.46), and a significant but weaker correlation with the Social Interaction subscale. These relationships indicate that the Total Protective Factors scale has approximately equal strength relationships to each of the social skills scales on the P K B S - 2 . 61 The D E C A Behavioral Concerns scale had moderate to strong positive correlations with the scales measuring problem behaviours on the P K B S - 2 ; the strongest relationship was with the Externalizing Problems scale (.71) and the weakest with the Internalizing Problems scale (.50). This finding is consistent with predictions. Research Question 3: Divergent Va l id i ty wi th the P K B S - 2 The D E C A Initiative scale had moderate negative correlations with the three scales measuring problem behaviours on the P K B S - 2 as follows: Problem Behavior Total (-.44), Internalizing Problems (-.43), and Externalizing Problems (-.36). A n interesting and unanticipated finding here is that the Initiative scale was slightly more strongly related to internalizing than externalizing problems as measured on the P K B S - 2 . The D E C A Self-Control scale was strongly to moderately related to the P K B S - 2 problem behaviour scales as follows: Externalizing Problems (-.64), Problem Behaviors Total (-.63), and Internalizing Problems (-.51). A s predicted, the Self-Control scale is more strongly related to externalizing than internalizing problems. A s was the case in examinations of convergent validity between the D E C A Attachment scale and P K B S - 2 social skills scales, weak negative correlations (ranging from -.28 to -.22) were observed between the D E C A ' s Attachment scale and the problem behaviour scales from the P K B S - 2 . Although these correlations are statistically significant, they are deemed to not be of practical significance. Attachment as measured on the D E C A does not appear to be have a meaningful inverse relationship to internalizing or externalizing problem behaviours as measured on the P K B S - 2 . Moderate and comparable negative correlations were also found between the D E C A ' s Total Protective Factors scale and the P K B S - 2 problem behaviour scales as 62-follows: Problem Behavior Total (-.52), Internalizing Problems (-.49), and Internalizing Problems (-.47). These correlations were stronger than those found with the D E C A ' s Initiative scale, but weaker than those found with the D E C A ' s Self-Control scale. These relationships suggest that problem behaviours are more strongly negatively related to lower Self-Control than to lower Total Protective Factors as measured on the D E C A . In contrast to the Self-Control scale, the relationship between the D E C A Total Protective Factors scale and the P K B S - 2 ' s Externalizing and Internalizing Problems scales was not significantly different. Moderate negative correlations were observed between the D E C A ' s Behavioral Concerns scale and the P K B S - 2 scale and subscales measuring social skil ls as follows: Social Cooperation subscale (-.55), Social Ski l ls Total scale (-.52), and Social Interaction and Social Independence subscales (-.36 each). This pattern mimics the general inverse relationship of social skills and problem behaviours reported in validation studies conducted for the P K B S - 2 and reported in the examiner's manual (Merrell , 2002). The finding that Behavioral Concerns are more strongly related to Social Cooperation than to Social Interaction or Social Independence is as predicted on a conceptual basis. Research Question 4: Convergent Va l id i ty wi th the B A S C - 2 The D E C A ' s Initiative scale had strong positive correlations with the B A S C - 2 ' s Adaptive Skil ls Composite (.71) and Social Skil ls scale (.66), moderate positive correlations with the Adaptability (.58) and Functional Communication (.45) scales, and a significant but weaker correlation with the Activities of Dai ly L iv ing scale (.35). These relationships are generally as predicted. 63 The D E C A ' s Self-Control scale had a similar but weaker pattern o f correlations with B A S C - 2 adaptive scales as follows: Adaptive Skil ls Composite, Social Ski l ls scale (.60 each), Adaptability scale (.49), and Functional Communication scale (.36). Although this finding is generally as predicted, an unexpected finding is that the Initiative scale is more strongly correlated with these B A S C - 2 adaptive scales than is the Self-Control scale. f The D E C A ' s Attachment scale, which had only weak correlations with P K B S - 2 social skills scales, did have moderate positive correlations with the B A S C - 2 ' s Social Ski l ls scale (.45) and Adaptive Skil ls Composite (.44), and significant but relatively weak correlations with Adaptability (.37) and Functional Communication (.38). A s was predicted and found in analyses with the P K B S - 2 , the Attachment scale measures a unique construct. Correlations between the D E C A Total Protective Factors scale and the B A S C - 2 adaptive scales were similar to those with the D E C A Initiative scale in strength and pattern. Specifically, the Total Protective Factors scale had strong positive correlations with the Adaptive Skil ls Composite (.71) and Social Ski l ls scale (.68), and moderate r positive correlations with the Adaptability (.59) and Functional Communication (.47) scales. A s was found in investigations of convergence with the P K B S - 2 problem behaviour scales, the D E C A ' s Behavioral Concerns scale has a much stronger relationship with the B A S C - 2 ' s Externalizing Problems Composite (.61) than with the Internalizing Problems Composite (.34). The Behavioral Concerns scale also correlated strongly with the B A S C - 2 ' s Behavioral Symptoms Index (.70), which itself is more 64 strongly related to measures of externalizing than internalizing problems on both the B A S C - 2 and P K B S - 2 . In terms of individual clinical scales on the B A S C - 2 , the D E C A Behavioral Concerns scale is strongly related to Attention Problems (.71), Hyperactivity and Aggression (.63 each), and moderately related to Depression (.52) and Atypicality (.50). Although these relationships are generally as predicted, the strength of the relationships between the Behavioral Concerns scale and the Depression and Atypicality scales is unexpected. This suggests that, although the Behavioral Concerns scale is better at identifying externalizing problems, it does also reflect some internalizing problems. Research Question 5: Divergent Va l id i ty wi th the B A S C - 2 The D E C A ' s Initiative scale had moderate negative correlations with several scales in the B A S C - 2 ' s clinical domain, including Attention Problems (-.56), Depression (-.48), and Atypicality (-.40). Statistically significant but weaker negative correlations were found with other B A S C - 2 clinical scales (Withdrawal, Hyperactivity, Aggression, and Somatization). As was the case in investigations of divergence with the P K B S - 2 , this is the only D E C A scale that had a slightly stronger relationship with the Internalizing Problems Composite than the Externalizing Problems Composite (-.39 versus -.30 respectively). The prediction that Initiative would have moderate negative correlations with the Depression, Withdrawal and Anxiety scales was only supported for the Depression scale. Overall, as was found in previous analyses (Jassal et al., 2005), the Initiative scale has the strongest inverse relationship to the B A S C - 2 Attention Problems scale. A s was predicted, the D E C A ' s Self-Control scale, in comparison to the Initiative and Attachment scales, had a similar but stronger pattern of negative correlations with the 65 B A S C - 2 clinical composites and scales. Self-Control was strongly negatively correlated with the Behavior Symptoms Index (-.68) and Externalizing Problems Composite (-.60), but only moderately negatively correlated with the Internalizing Problems Composite (-.40). On individual B A S C - 2 scales, Self-Control was strongly negatively correlated with Aggression (-.62); moderately negatively correlated with Attention Problems (-.59), Hyperactivity (-.56), Atypicality (-.52), and Depression (-.49); and significantly but weakly correlated with Somatization (-.34) and Withdrawal (-.29). A s has been the case in'comparisons with the P K B S - 2 , the D E C A Attachment scale has very little meaningful negative relationship to any of the B A S C - 2 scales measuring problem behaviour, with weak correlations ranging from -.01 for the Anxiety scale to -.28 for the Depression scale. This again supports that Attachment is a unique construct. The D E C A Total Protective Factors scale also has moderate negative correlations with B A S C - 2 clinical composites as follows: Behavior Symptoms Index (-.60), Externalizing Problems (-.43), and Internalizing Problems (-39). As was the case in comparisons with the P K B S - 2 , the Total Protective Factors scale is about equally related to both internalizing and externalizing problems. On individual scales, Total Protective Factors is moderately negatively correlated with Attention Problems (-.57), Depression (-.51), Atypicality (-.47), Aggression (-.44), and Hyperactivity (-.43); and significantly but more weakly correlated with Withdrawal (-.34) and Somatization (-.30). The D E C A ' s Behavioral Concerns scale had a strong negative correlation with the B A S C - 2 Adaptive Skil ls Composite (-.62), and moderate negative correlations with all four of the adaptive scales: Social Ski l ls (-.56), Functional Communication (-.47), 66 Adaptability (-.45), and Activit ies of Dai ly L iv ing (-.35). This pattern is again predicted. Chapter V: Discussion Reliability and Internal Validity Using this new sample of parent ratings for five-year-old kindergarten students, the five scales on the D E C A continue to meet recommended criteria for internal consistency. Item-scale correlations and examining changes in scale alphas as items were deleted indicate that the items included on each scale are reliable and valid representations of the construct each scale is intended to measure. Correlations between the five D E C A scales are generally consistent with theoretical predictions. Moderate to strong positive correlations between the protective factor scales indicate that there is considerable overlap or co-occurrence with these constructs, but also that the measurement of each construct adds a unique component to understanding children's functioning. Intercorrelations between the protective factor scales in the current study were somewhat stronger than those previously reported, but this may be due to the restricted age range employed in the current study. Moderate to strong negative correlations between the protective factor scales and the Behavioral Concerns scale indicate that although these factors have a negative relationship, the constructs are not polar opposites of each other and both need to be measured in order to obtain the fullest picture of children's functioning. Examinations of the mean scale scores obtained from this sample provide some support for the stability of the D E C A when used on a different population. However, when compared to an expected mean of 50, significant differences in mean scores on the Self-Control (for girls only) and Behavioral Concerns (for boys only) scales were found. Examinations of mean scale scores on the other two instruments employed in this study 68 find that the mean scale scores for this study's sample are not significantly different from expected on the clinical/problem behavior scales and composites overall, but for girls only, these scores are significantly lower than expected. On the adaptive/social skills scales and composites, this sample's mean scores are significantly higher for both boys and girls. This indicates that this study's sample is not representative in terms of ratings on scales measuring adaptive behaviour, social skil ls, and protective factors, but is representative in terms of ratings on scales measuring problem behaviours. Thus, these results suggest that some degree of caution is required when interpreting scores on the D E C A Behavioral Concerns scale. However, it is important to remember that the intended use of the Behavioral Concerns scale is as a screener to identify children who may require more comprehensive assessment of behavioural concerns; for this purpose, elevated scores (false positives) are more desirable than false negatives. Convergent and Divergent Validity Evidence DECA Initiative scale. Consistent with predictions, this scale was strongly to moderately correlated with scales measuring adaptive/social skills. In comparisons with the P K B S - 2 , Initiative was more strongly related to the Social Independence scale than to Social Cooperation or Social Initiative scales, indicating that the conceptualization of Initiative is somewhat more reflective of adult-related than peer-related forms of social adjustment, and emphasizes independence over cooperation or interaction. In comparisons with the B A S C - 2 , Initiative was strongly to moderately related to the Social Ski l ls, Adaptability, and Functional Communication scales. Overall , these correlations suggest that although Initiative has considerable overlap with social and adaptive skil ls, there are aspects of this construct that are unique. 69 The Initiative scale was moderately negatively related to scales measuring problem behaviours; these constructs are not opposites of each other, but do have some degree of negative relationship. A n unexpected finding was that this is the only D E C A scale that had a stronger relationship to internalizing problems than to externalizing problems, suggesting that children who are rated higher on the Initiative scale are less l ikely to have internalizing problems than externalizing problems. D E C A Sel f -Contro l scale. Consistent with predictions, the Self-Control scale was strongly to moderately correlated with scales measuring adaptive/social skills, and these correlations were stronger than those found with the D E C A Initiative and Attachment scales. Self-Control was most strongly related to Social Cooperation ( P K B S -2) and Social Ski l ls (BASC-2) , and had moderate but slightly weaker correlations with the P K B S - 2 Social Independence and Social Interaction scales, and the B A S C - 2 Adaptability scale. Thus, of the three D E C A protective factor scales, Self-Control has the most in common with measures of positive/adaptive behaviours on the two comparison instruments. This pattern of relationships indicates that, although there is considerable overlap, this scale still appears to provide unique information when compared to the other two instruments. Self-Control was also the D E C A scale that had the strongest negative correlation with scales measuring clinical/problem behaviours. These negative correlations were particularly strong with measures of externalizing behaviours (e.g., aggression, hyperactivity) on the other two instruments, but it is important to note that moderate negative correlations were also observed with some scales measuring internalizing behaviours (i.e., depression but not anxiety). 70 DECA Attachment scale. O f the three D E C A protective factor scales, Attachment had the weakest positive and negative correlations with scales from the other two instruments. This scale's strongest relationships were found in moderate positive correlations with the adaptive scales on the B A S C - 2 (Social Ski l ls, Adaptability, Functional Communication), indicating some degree of overlap. Overall , however, the Attachment scale appears to be measuring a unique construct that is not represented on the two comparison instruments. DECA Behavioral Concerns scale. A s expected, the Behavioral Concerns scale was positively related to scales measuring clinical/problem behaviours, and negatively related to scales measuring adaptive/social skills. The pattern of relationships between the D E C A Behavioral Concerns scale and measures of externalizing and internalizing problems from the P K B S - 2 and B A S C - 2 suggest that this D E C A scale is an effective screener for children with elevated levels of externalizing problems, but not as effective at identifying children with elevated levels of internalizing problems. This is consistent with predictions based on an item analysis that found that more items on the Behavioral Concerns scale were similar to items on the externalizing scales of the other two instruments than to items on the internalizing scales. This result may also reflect, to a degree, the general finding that externalizing problems are easier to observe than are internalizing problems. Overall, however, moderate to strong correlations between the Behavioral Concerns scale and composite scales measuring behaviour problems from the P K B S - 2 and B A S C - 2 indicate that this D E C A scale is a good overall screener for problem behaviours. Thus, when the Behavioral Concerns scale is used, as intended by 71 the authors, as a screener to identify children who require more in-depth assessment of problem behaviours, it is an effective measure. L imi tat ions of Study The main limitations of this study lie in the nature of the sample. Although this sample of sixty-nine was sufficiently large to detect significant correlations, it was somewhat smaller than samples used in other correlational studies. Another limitation of this sample is that, although attempts were made to obtain a sample that is representative of the population from which it was drawn, the final sample underrepresents individuals from lower socioeconomic levels and individuals who identify as First Nations/Aboriginal. This study's sample is also not comparable to the standardization sample. The age range of the children rated in this sample was also very restricted; the D E C A was designed for children aged two to five years, but the current sample includes only children aged five years one month to five years eleven months. Thus, the findings of this study can be confidently generalized only to children at the top end of the age range for which the D E C A was designed. A final limitation of this study lies in the general finding that ratings provided on measures of adaptive skil ls, social skil ls, and the D E C A Self-Control measure were generally higher than expected based on the standardization samples. This finding may have exaggerated the strength of the correlations that were found. Strengths of Study The popularity and wide spread use of the D E C A means that this study was timely and necessary, as validation evidence for the D E C A is still at an early stage. Although accumulating research and practical experience indicates the D E C A is a useful 72 instrument, this study provides further psychometric evidence supporting its use. It is hoped that the results of this study wi l l make a strong contribution to the ongoing program of validation research regarding the D E C A . Two factors mentioned as possible limitations of this study could also be viewed as strengths. Most importantly, the fact that this sample is not comparable to the standardization sample is important; it is evident that the reliability and internal validity of the D E C A holds when it is used with a different population. While this sample's age range may limit the generalizability of the results, this age range was selected purposefully because children's entry into formal schooling is a particularly important time to assess their social-emotional status. Final ly, although this sample is not completely representative of the population of the community from which it was drawn, it is a sample with considerable variability on important factors such as income and race/ethnicity. Implications for Other School Psychologists The results of this study indicate that the D E C A ' s internal consistency and scale structure remain strong when it is used on a different population. Patterns of correlations with scales from other well-established and psychometrically sound rating scales indicate that the constructs measured on the D E C A are consistent with theoretical propositions. Although there are a number of similarities in the information provided by the D E C A and . that provided by the comparison instruments, there are also a number of key differences, suggesting that the D E C A does provide unique information regarding children's social and emotional functioning. Thus, when ratings obtained with the D E C A are used for the purposes described by the authors, these ratings can be interpreted with confidence. 73 A number of practical implications regarding conducting research in schools were also evident. Most important is the need to fol low the appropriate procedures and obtain consent from all individuals involved, as well as to be very organized in order to garner the most effective help from school personnel. While it was found that school personnel were wi l l ing to help and interested in the research topic, difficulties arose because of the timing of data collection. Schools are busy places all through the year, but particularly so in the spring, closer to the end of the year. The timing of data collection also lead to a restricted age range within the sample, as the majority of the kindergarten students invited to participate were already well into their fifth year, and in fact many had already turned six and were too old for the study. Conclus ion The results of this study provide support for the reliability and internal validity of the D E C A when used with a sample different from the standardization sample, as well as validity evidence regarding the D E C A ' s convergence and divergence with scales from the P K B S - 2 and B A S C - 2 . Because the D E C A was designed to measure unique constructs not already represented on other rating instruments, and because of ambiguities in the research literature on the interaction of risk and protective factors and their fit with other constructs such as social skil ls, it is difficult to critically evaluate the pattern of correlations presented here. In general, results illustrate an expected pattern of convergence and divergence between the D E C A scales and scales from two comparison instruments, the Behavior Assessment System for Children, Second Edit ion (Reynolds & Kamphaus, 2004) and the Preschool and Kindergarten Behavior Scales, Second Edit ion (Merrell , 2002). The D E C A ' s protective factor scales relate positively to other measures 74 of social skills/adaptive behaviours, and negatively to other measures of problematic/clinical behaviours; these correlations were strongest for the D E C A ' s Self-Control scale, and weakest for the D E C A ' s Attachment scale. The D E C A ' s Behavioral Concerns screener scale related negatively to other measures of social skills/adaptive behaviours, and positively to other measures of problematic/clinical behaviours, particularly those reflecting externalizing behaviour problems. Although there is overlap among the constructs measured on these three instruments, it is also evident that the D E C A measures unique constructs that are not represented on existing assessment instruments. The D E C A is a psychometrically sound instrument that makes an important and unique contribution to the field of social-emotional assessment of young children. 75 References Achenbach, T. M . (1991). Child Behavior Checklist. Burlington, V T : University of Vermont Department of Psychiatry. American Educational Research Association, American Psychological Association, & National Counci l on Measurement in Education (1999). Standards for educational and psychological testing. Washington, D C : Author. Baldo, A . J . (2000). Risk factors, protective factors, and challenging behaviors in at risk preschool aged children: A n exploratory analysis. Dissertation Abstracts International, 61, 12A. (UMI No. 9999504). Bracken, B. A . (1987). Limitations of preschool instruments and standards for minimal levels of technical adequacy. Journal of Psychoeducational Assessment, 4, 313-326. Bracken, B. A . , Keith, L. K., & Walker, K. C. (1998). Assessment of preschool behavior and social-emotional functioning: A review of thirteen third-party instruments. Journal of Psychoeducational Assessment, 16, 153-169. Buhs, E.S. (2003). Test review of the Devereux Early Childhood Assessment. From B. S. Plake, J . C. Impara, & R. A . Spies (Eds.), The fifteenth mental measurements yearbook [Electronic version]. Retrieved February 18, 2004, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros Canivez, G. L., & Bordenkircher, S. E. (2002). Convergent and divergent validity of the Adjustment Scales for Children and Adolescents and the Preschool and Kindergarten Behavior Scales. Journal of Psychoeducational Assessment, 20, 30-45. 76 Canivez, G. L., & Rains, J . D. (2002). Construct validity of the Adjustment Scales for Children and Adolescents and the Preschool and Kindergarten Behaviour Scales: Convergent and divergent evidence. Psychology in the Schools, 39, 621-633. Chittooran, M . M . (2003). Test review of the Devereux Early Childhood Assessment. From B. S. Plake, J . C. Impara, & R. A.. Spies (Eds.), The fifteenth mental measurements yearbook [Electronic version].' Retrieved February 18, 2004, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros Cicchetti, D., & Toth, S. L. (1997). Transactional ecological systems in developmental psychopathology. - In S. S. Luthar, J . A . Burack, D. Cicchetti, & J . R. Weisz (Eds.), Developmental Psychopathology. New York: Cambridge University Press. Cohen, J . (1992). A power primer. Psychological Bulletin, 112, 155-159. Conners, C. K. (1990). Conners' Rating Scales. North Tonawanda, N Y : Mult i-Health Systems. Consortium on the School-based Promotion of Social Competence (1996). The school-based promotion of social competence: Theory, research, practice, and policy. In R. J . Haggert, L. R. Sherrod, N . Garmezy, & M . Rutter (Eds.), Stress, risk, and resilience in children and adolescents: Processes, mechanisms, and interventions (pp. 268-316). New York: Cambridge University Press. Cronbach, L. J . (1990). Essentials of psychological testing (5th ed.). New York: Harper Coll ins. Dahinten, V . S.,.& Ford, L. (2004). Healthy Childhood Project. Unpublished findings, University of Brit ish Columbia 77 Denham, S. A . , & Weissberg, R. (2004). Social-emotional learning in early childhood: What we know and where to go from here. In E. Chesebrough, P. K ing , T. P. Gullotta, & M . Bloom (Eds.), A blueprint for the promotion ofprosocial behaviors in early childhood (pp. 13-50). N e w York: Kluwer Academic/ Plenum Publishers. Devereux Early Childhood Initiative (n.d.a). Prevalence of preschool protective factors as perceived by parents and teachers. Research Bulletin #2. Retrieved March 2, 2003, from www.devereux.org Devereux Early Childhood Initiative (n.d.b). The Devereux Early Childhood Assessment and school readiness. Research Bulletin #3. Retrieved March 2, 2003, from www.devereux.org Diamond, K. E., & Squires, J . K. (1993). The role of parental report in the screening and assessment of young children. Journal of Early Intervention, 17, 107-115. DiPerna, J . C. (2001). Test review of the Behaviour Assessment System for Ch i ld ren-Revised. From B. S. Plake & J. C. Impara (Eds.), The fourteenth mental measurements yearbook [Electronic version]. Retrieved September 26, 2003, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros Do l l , B., & Lyon, M . A . (1998). Introduction to the mini-series: Resilience applied: The promise and pitfalls for school-based resilience programs. School Psychology Review, 27,346-347. Drotor, D. (2002). Behavioral and emotional problems in infants and young children: Challenges of clinical assessment and intervention. Infants and Young Children, 14, 1-5. Fairbank, D. W. (2005). Test review of the Preschool and Kindergarten Behavior Scales Second Edition. From R. A . Spies & B. S. Plake (Eds.), The sixteenth mental 78 measurements yearbook [Electronic version]. Retrieved October 13, 2005, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros Fei l , E., Severson, H. , & Walker, H. (1998). Screening for emotional and behavioral delays: The Early Screening Project. Journal of Early Intervention, 21, 252-265. Flanagan, R. (1995). A review of the Behavior Assessment System for Children ( B A S C ) : Assessment consistent with the requirements of the Individuals with Disabilities Education Act ( IDEA). Journal of School Psychology, 33, 177-186. Frankenburg, W. K., Dodds, J . , Archer, P., Bresnick, B., Maschka, P., Edelman, N. , & Shapiro, H. (1990). Denver II Developmental Screening Test. Denver, C O : Denver Developmental Materials Inc. Goal 1 Early Childhood Assessments Resource Group (n.d.). Principles and recommendations for early childhood assessments. Retrieved February 20, 2003, from http://www.negp.gov/Reports/prinrec.pdf Gresham, F. M . , & El l iot , S. N . (1990). Social Skills Rating System Manual. Circle Pines, M N : American Guidance Service. Howel l , D. C. (1997). Statistical methods for psychology (4 t h ed.). Belmont, C A : Wadsworth Publishing. Hubley, A . M . , & Zumbo, B. D. (1996). A dialect on validity: Where we have been and where we are going. The Journal of General Psychology, 725,207-215. Hughes, S. (1995). Test review of the Denver Developmental Screening.Test, Revised. From J. C. Conoley & J . C. Impara (Eds.), The twelfth mental measurements yearbook [Electronic version]. Retrieved August 12, 2004, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros 79 Jassal, S., van Leeuwen, S., Ford, L., & Dahinten, V . S . (2005, August). Contemporary assessment in early childhood: Validating the Devereux Early Childhood Assessment. Paper presented at the annual meeting of the American Psychological Association, Washington, D C . Jentzsch, C. E., & Merrel l , K. W. (1996). A n investigation of the construct validity of the Preschool and Kindergarten Behavior Scales. Diagnostique, 21, 1-15. Kazdin, A . E. (1995). Preparing and evaluating research reports. Psychological Assessment, 7,228-237. Keith, L. K., & Campbell , J . M . (2000). Assessment of social and emotional development in preschool children. In B. Bracken (Ed.), The psychoeducational assessment of preschool children (3rd ed.). Boston: A l l yn & Bacon. Kl ine, P. (1993). The handbook of psychological testing. New York: Routledge. Knoff, H. M . (1992). Assessment of social-emotional functioning and adaptive behavior. In E. Vasquez Nuttall, I. Romero, & J. Kalesnik (Eds.), Assessing and screening preschoolers. Needham Heights, M A : A l l yn and Bacon. LeBuffe, P. A . , & Naglier i , J . A . (1999a). Devereux Early Childhood Assessment: Technical manual. Lewisvi l le, N C : Kaplan Press. LeBuffe, P. A . , & Naglieri , J . A . (1999b). Devereux Early Childhood Assessment: User's Guide. Lewisvi l le, N C : Kaplan Press. MacPhee, D. (1998). Test review of the Preschool and Kindergarten Behavior Scales. From J. C. Impara & B. S. Plake (Eds.), The thirteenth mental measurements yearbook [Electronic version]. Retrieved September 26, 2003, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros 80 Madle, R. A . (2005). Test review of the Preschool and Kindergarten Behavior Scales Second Edition. From R. A . Spies & B. S. Plake (Eds.), The sixteenth mental measurements yearbook [Electronic version]. Retrieved October 13, 2005, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros Masten, A . S., & Coatsworth, J . D. (1998). The development of competence in favourable and unfavourable environments: Lessons from research on successful children. American Psychologist, 53, 205-220. Masten, A . S., Best, K. M . , & Garmezy, N . (1990). Resilience and development: Contributions from the study of children who overcome adversity. Development and Psychopathology, 2, 425-444. Matson, J . L., Rotari, A . F., & Helsel, W. J . (1985). Development of a rating scale to measure social skills in children: The Matson Evaluation of Social Ski l ls with Youngsters ( M E S S Y ) . Psychopharmacology Bulletin, 21, 855-867. McDermott, P. A . , Marston, N . C , & Stott, D. H. (1993). Adjustment Scales for Children and Adolescents. Philadelphia, P A : Edumetric and Cl in ical Services. Meisels, S. J . (1999). Assessing readiness. In R. C. Pianta & M . J . Cox (Eds.), The transition to kindergarten (pp. 39-66). Baltimore, M D : Paul H. Brooks Publishing. Meisels, S. J . , & Atkins-Burnett, S. (2000). The elements of early childhood assessment. In S. J . Meisels & J . P. Shonkoff (Eds.), Handbook of early childhood intervention (2nd ed., pp. 231-257). New York: Cambridge University Press. Merrel l , K. W. (1993). School Social Behavior Scales. Eugene, OR: Assessment-Intervention Resources. 81 Merrel l , K. W. . (1994). Preschool and Kindergarten Behaviour Scales. Brandon, V T : Cl in ical Psychology Publishing. Merrel l , K .W. (1996a). Social-emotional problems in early childhood: New directions in conceptualization, assessment, and treatment. Education & Treatment of Children, 19, 458-474. Merrel l , K. W. (1996b). Social-emotional assessment in early childhood: The Preschool and Kindergarten Behavior Scales. Journal of Early Intervention, 20, 132-145. Merrel l , K. W. (2001).- Assessment of children's social skil ls: Recent developments, best practices, and new directions. Exceptionality, 9, 3-18. Merrel l , K. W. (2002). Preschool and Kindergarten Behaviour Scales Second Edition. Brandon, V T : Cl inical Psychology Publishing. Merrel l , K. W., & Wolfe, T. M . (1998). The relationship of teacher-rated social skills deficits and A D H D characteristics among kindergarten-age children. Psychology in the Schools, 35, 101-109. Messick, S. (1989). Validity. In R. L. L inn (Ed.), Educational Measurement (3rd ed., pp. 13-103). New York: Macmil lan. Naglier i , J . A . , & LeBuffe, P. A . (2005). Measuring resilience in children: From theory to practice. In S. Goldstein & R. B. Brooks (Eds.), Handbook of resilience in children (pp. 107-121). New York: Kluwer Academic/Plenum Publishers. Pedhazur, E. J . , & Pedhazur Schmelkin, L. (1991). Measurement, design, and analysis: An integrated approach. Hil lsdale, N J : Lawrence Erlbaum. 82 Pi lecki , B. C , & Schrage, D. N . (n.d.). Protective factors in preschool children with internalizing and externalizing disorders. Research Bulletin #6. Retrieved August 7, 2005, from www.devereux.org Raver, C. C. (2002). Emotions matter: Making the case for the role of young children's emotional development for early school readiness. Social Policy Report, 16. Society for Research in Chi ld Development. Raver, C. C , & Zigler, E. F. (1997). Social Competence: A n untapped dimension in evaluating Head Start's success. Early Childhood Research Quarterly, 12, 363-385. Reynolds, C. R., & Kamphus, R. W. (2004). Behavior Assessment System for Children Second Edition. Circle Pines, M N : American Guidance Service. Rutter, M . (2000). Resilience reconsidered: Conceptual considerations, empirical findings, and policy implications. In S. J . Meisels & J. P. Shonkoff (Eds.), Handbook of early childhood intervention (2nd ed., pp. 651-682). New York: Cambridge University Press. Salvia, J . , & Ysseldyke, J . E. (2001). Assessment (8 t h ed.). Boston: Houghton Mi f f l in . Shepard, L. A . , Taylor, G: A . , & Kagan, S. L. (1996). Trends in early childhood assessment policies and practices. Retrieved February 20, 2003, from http://www.negp.gov/Repprts/ecms2.pdf Shonkoff, J . P., & Phil l ips, D. A . (2000). From neurons to neighbourhoods: The science of early childhood development. Retrieved March 22, 2004, from http://books.nap.edu/books/0309069882/html/ Sparrow, S. S., Bal la, D. A . , & Cicchetti, D. V . (1984). Vineland Adaptive Behavior Scales. Circle Pines, M N : American Guidance Service. 83 Spies, R., & Jones, C. F. (2001). Test review of the Behaviour Assessment System for Children Revised. In B. S. Plake & J . C. Impara (Eds.), The fourteenth mental measurements yearbook [Electronic version]. Retrieved September 26, 2003, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros Squires, J . , Bricker, D., Heo, K., & Twombly, E. (2001). Identification of social-emotional problems in young children using a parent-completed screening measure. Early Childhood Research Quarterly, 16, 405-419. Statistics Canada (2002). 2001 Community profiles: Community highlights for Prince George. Released June 27, 2002. Last modified November 30, 2005. Statistics Canada catalogue no. 93F0053XIE. Tran, A . & Shapiro, V . B. (n.d.). The Devereux Early Childhood Assessment: Promoting resilience and school readiness. Research Bulletin #8. Retrieved December 30, 2006, • from www.devereux.org Walker, H . M . , & McConnel l , S. R. (1995). The Walker-McConnell Scale of Social Competence and School Adjustment, Elementary Version. San Diego, C A : Singular. Watson, T. S. (1998). Test review of the Preschool and Kindergarten Behavior Scales. From J. C. Impara & B. S. Plake (Eds.), The thirteenth mental measurements yearbook [Electronic version]. Retrieved September 26, 2003, from the Buros Institute's Test Reviews Online website: http://www.unl.edu/buros Werner, E. E. (2000). Protective factors and individual resilience. In S. J . Meisels & J. P. Shonkoff (Eds.), Handbook of early childhood intervention (2nd ed., pp. 115-132). New York: Cambridge University Press. 84 Appendix A : Item- and scale-level comparisons of the D E C A , B A S C - 2 , & P K B S - 2 , and scale descriptions 85 Table A l • Item-Level Comparison of D E C A and P K B S - 2 Scale Item DECA Scale Item PKBS-2 A T 1. act in a way that made adults smile or show interest in him/her IN 2 do things for himself/herself A 3 A3 1 8 works or plays independently attempts new tasks before asking for help (somewhat) IN - 3 choose to do a task that was challenging for her/him A 3 8 attempts new tasks before asking for help (somewhat) SC 4 listen to or respect others A l 7 listens to and respects others SC 5 control his/her anger A l B l 10 41 . shows self-control is easily provoked; has a short fuse A T 6 respond positively to adult comforting when upset A 2 B2 B2 24 4 27 seeks comfort from an adult when hurt does not respond to affection from others is difficult to comfort when upset IN 7 participate actively in make-believe play with others (dress-up, etc.) B2 12 avoids playing with other children B C 8 fail to show joy or gladness at a happy occasion B2 30 seems unhappy or depressed B C 9 touch children/adults inappropriately A T 10 show affection for familiar adults A 2 34 shows affection for other children B C 11 have temper tantrums B l 7 has temper outbursts or tantrums IN 12 keep trying when unsuccessful (act persistent) A3 8 attempts new tasks before asking for help (somewhat) SC 13 handle frustration well A l 10 shows self-control B C 14 has no reaction to children/adults B2 4 does not respond to affection from others B C 15 use obscene gestures or offensive language B l 26 calls people names IN 16 try different ways to solve a problem A 3 8 attempts new tasks before asking for help (somewhat) 86 Scale Item DECA Scale Item PKBS-2 A T . 17 act happy or excited with parent/guardian returned B C 18 destroy or damage property B l 34 destroys things that belong to others IN 19 try or ask to try new things or activities A3 8 attempts new tasks before asking for help (somewhat) IN 20 start or organize play with other children B2 12 avoids playing with other children SC 21 show patience B l 41 is easily provoked, has a short fuse A T 22 ask adults to play with or read to him/her A l 16 sits and listens when stories are being read B C 23 have a short attention span (difficulty concentrating) B l 15 has difficulty concentrating or staying on task IN 24 focus his/her attention or concentrate on a task or activity B l 15 has difficulty concentrating or staying on task SC 25 share with other children A l B l 25 10 shares toys and other belongings w i l l not share ' B C 26 fight with other children B l 11 is physically aggressive (hits, kicks, punches) B C 27 become upset or cry easily? B l 33 is moody or temperamental IN 28 say positive things about the future (act optimistic) A T 29 trust familiar adults and believe what they say SC 30 accept another choice when his/her first choice was unavailable B l 19 must have his/her own way A T 31 seek help from children/adults when necessary A 2 15 asks for help from adults when needed IN 32 ask other children to play with him/her A 2 B2 21 12 Invites other children to play avoids playing with other children SC 33 cooperate with others A l 2 is cooperative SC 34 ' calm herself/himself down when upset B C • 35 get easily distracted B l 15 has difficulty concentrating or staying on task 87 Scale Item DECA Scale Item PKBS-2 IN 36 make decisions for himself/herself A T 37 shows an interest in what children/adults are doing B2 28 withdraws from the company of others DECA Scales A T attachment IN initiative SC self control B C behavioural concerns PKBS Scales A l Social B l Externalizing Behavior Cooperation Problems A 2 Social B2 Internalizing Behavior Interaction Problems A3 Social Independence 88 Table A 2 Item-Level Comparison of D E C A and B A S C - 2 Scale Item D E C A Scale Item BASC-2 A T 1 act in a way that made adults smile or show interest in him/her WI 82 is shy with adults IN 2 do things for himself/herself H Y 61 needs too much supervision IN .3 choose to do a task that was challenging for her/him A N A N 35 91 says " I 'm afraid I w i l l make a mistake" worries about making mistakes SC 4 listen to or respect others A G A P 21 34 . argues when denied own way listens carefully • SC 5 control his/her anger H Y H Y H Y A G A G H Y 5 17 33 68 77 117 acts without thinking acts out of control has poor self-control hits other children loses temper too easily throws tantrums A T 6 respond positively to adult comforting when upset A D 85 easily soothed when angry IN 7 participate actively in make-believe play with others (dress-up, etc.) WI WI WI WI 37 26 105 134 is shy with other children avoids other children refuses to jo in group activities quickly joins group activities B C 8 fail to show joy or gladness at a happy occasion D E A N 44 56 is sad is too serious B C 9 touch children/adults inappropriately A T 10 show affection for familiar adults B C 11 have temper tantrums H Y H Y 33 . 117 Has poor self-control throws tantrums IN 12 keep trying when unsuccessful (act persistent) A P D E A G A N 6 42 77 119 has a short attention span is easily frustrated loses temper too easily says " I 'm not very good at this" 89 Scale Item D E C A Scale Item BASC-2 SC 13 handle frustration well A G H Y D E D E A G D E H Y 21 33 42 52 77 100 117 argues when denied own way has poor self-control is easily frustrated' cries easily loses temper too easily is easily upset throws tantrums B C 14 has no reaction to children/adults A T A P A P 8 62 108 seems unaware of others pays attention pays attention when spoken to B C 15 use obscene gestures or offensive language IN 16 try different ways to solve a problem D E 42 is easily frustrated A T 17 act happy or excited with parent/guardian returned B C 18 destroy or damage property A G 12 breaks other children's things IN 19 try or ask to try new things or activities A D 69 tries new things IN 20 start or organize play with other children WI WI 26 37 avoids other children is shy with other children SC 21 show patience H Y D E D E H Y 33 42 100 117 118 poor self-control is easily frustrated is easily upset throws tantrums is easily annoyed by others A T 22 ask adults to play with or read to him/her B C 23 have a short attention span (difficulty concentrating) A P . A P A P 6 62 90 has a short attention span pays attention is easily distracted IN 24 focus his/her attention or concentrate on a task or activity A P A P A P 6 62 90 has a short attention span pays attention is easily distracted SC 25 • share with other children A D 1 shares toys or possessions with other children B C 26 fight with other children H Y A G A G 33 49 68 poor self-control threatens to hurt others hits other children B C 27 become upset or cry easily? H Y D E D E D E 33 42 52 100 poor self-control is easily frustrated cries easily is easily upset 90 Scale Item D E C A Scale Item B A S C - 2 IN 28 say positive things about A N 63 worries about things that the future (act optimistic) A N 76 cannot be changed worries A T 29 trust familiar adults and believe what they say SC 30 accept another choice when his/her first choice was unavailable D E 14 pouts A T 31 seek help from children/adults when necessary SS 88 politely asks for help IN 32 ask other children to play WI 9 has trouble making new with him/her WI 54 friends makes friends easily SC 33 cooperate with others A D A G H Y 1 21 55 shares toys or possessions with other children argues when denied his own way cannot wait to take turn SC 34 calm herself/himself down when upset H Y 33 has poor self-control B C 35 get easily distracted A P 90 is easily distracted IN 36 make decisions for himself/herself A T 37 shows an interest in what children/adults are doing WI 134 Quickly joins group activities D E C A Scales B A S C Scales A T attachment A D adaptability D E depression IN initiative A G aggression H Y hyperactivity SC self control A N anxiety SS social skills B C behavioural concerns A P attention WI withdrawal problems A T atypicality 91 Table A3 Number of Similar Items from B A S C - 2 and P K B S - 2 Loading onto D E C A scales D E C A B A S C - 2 P K B S - 2 Attachment Social Skil ls 1 Social Cooperation 1 Withdrawal 2 Social Interaction 3 Adaptability 1 Internalizing Beh. Prob. 3 total= 4 total= 7 Initiative Adaptability 1 Social Interaction 2 Anxiety 5 Social Independence 6 Attention prob. 4 Externalizing Beh. Prob. 1 Hyperactivity 1 Withdrawal 8 Depression 2 Aggression 1 total= 2 2 total= 9 Self Control Adaptability 3 Social Cooperation 5 Aggression 6 Externalizing Beh. Prob. 4 Depression . 6 Hyperactivity . 10 total = 2 5 total= 9 Behavior Concerns Aggression 3 Externalizing Beh. Prob. 7 Attention prob. 6 Internalizing Beh. Prob. 2 Depression 3 Hyperactivity 4 Atypicality 1 total = 17 total = 9 92 Table A 4 Descriptions of Scales on the D E C A , P K B S - 2 , and B A S C - 2 DECA Initiative scale Abi l i ty to use independent thought and action to meet his or her needs; somewhat independent, active learners, w i l l start or organize activities with other children, positive, optimistic, good problem solvers, enjoy challenges Self-Control scale Abi l i ty to experience a range of feelings and express them using the words and actions that society considers appropriate; able to handle frustration and negative emotions without acting out, cooperative, patient, respectful, share wel l , able to calm themselves down and recover quickly when upset Attachment scale Mutual, strong, and long-lasting relationship between a child and significant adults such as parents, family members, and teachers; actively seek out adults and other children, effective in gaining positive attention from others, affectionate, trusting of familiar adults, respond to adult comforting when upset Behavioral Concerns scale Wide variety of problem or challenging behaviours; aggression, withdrawal, attention, extreme emotions Total Protective Factors scale Includes Initiative, Self-Control, and Attachment scales PKBS-2 Social Cooperation subscale Behaviours and characteristics deemed important in following instructions from adults, cooperating and compromising with peers, and showing appropriate self-restraints; appropriate compliance with the types of structure and regulation typically imposed by parents, teachers, and other caregivers -Social Interaction subscale Behaviours and characteristics deemed important in gaining and maintaining acceptance and friendship from others, primarily peers Social Independence subscale Behaviours and characteristics deemed important in achieving social independence within the domain of the peer group; being able to separate appropriately from'caregivers, showing appropriate confidence and positive assertiveness in interactions with peers Externalizing Problems subscale Externalizing domain of child behavioural problems, including acting out and disruptive, undercontrolled, and overactive behaviour. Internalizing Problems subscale Internalizing domain of child behavioural problems, including anxiety, fearfulness, or emotional oversensitivity. Social Ski l ls scale Includes Social Interaction, Social Independence, and Social Cooperation subscales 93 Problem Behaviour scale Includes Externalizing Problems and Internalizing Problems subscales B A S C - 2 Aggression scale Tendency to do physical or emotional harm to others or their property, inc. both verbal and physical aggression • Withdrawal scale Tendency to evade others to avoid social contact and to lack interest in making contact in social settings Somatization scale Tendency to be overly sensitive and complain about relatively minor physical problems or ailments, and to overreport the occurrence of various physical complaints Depression scale Maladaptive cognitions about self, the world, and the future, inc. dysphoric mood, suicidal ideation, withdrawal from others, and self-reproach Atypicality scale Tendency to behave in ways that are considered odd or strange, esp. disconnection from or unawareness of normal surroundings and psychotic tendencies; may also point to immaturity or developmental delay . Anxiety scale Excessive worry, fears and phobias, self-depreciation, and nervousness Social Ski l ls scale Interpersonal aspects of social adaptation such as complimenting others, encouraging others, offering assistance, and using manners , Adaptability scale Abi l i ty to adjust to changes in routine, to shift from one task to another, and to share toys or possessions with other children; may also indicate tendency towards negative emotionality and poor emotional self-control Functional Communication scale Abi l i ty to express ideas and communicate in ways that others can easily understand Activities of Dai ly L iv ing scale Screen for adaptive behaviour deficits ' Behaviour Symptoms Index Includes Hyperactivity, Aggression, Depression, Attention Problems, Atypicality, and Withdrawal scales ' Externalizing Problems composite Includes Hyperactivity and Aggression scales Internalizing Problems composite Includes Anxiety, Depression, and Somatization scales ' Adaptive Skil ls composite Includes Adaptability, Social Ski l ls, Activities of Dai ly L iv ing, and Functional Communication scales 94 Appendix B: Reliabil ity Statistics for the D E C A , B A S C - 2 , & P K B S - 2 Table B l Normative Sample Reliabil ity Statistics for the D E C A , PKBS - 2 , and B A S C - 2 PRS-P Internal Consistency Test- Interrater Retest M A I N S C A L E / C O M P O S I T E Chronbach's teacher- parent- parent-Scale alpha SEm teacher teacher parent DECA T O T A L P R O T E C T I V E F A C T O R S .91/.94 a 2.97/2.39 a .74A94 b .69 d .29 f .21 h Initiative .84/.90 4.03/3.15 .80/.91 .59 .34 .32 Self-Control .86/.90 3.74/3.21 .64/.91 .77 .23 .33 Attachment .76A85 4.91/3.87 .S5/.87 .57 .19 .26 Behavioral Concerns .71/.80 5.40/4.46 .55A68 .62 .23 •44. PKBS-2 S O C I A L S K I L L S .93A96 a 4.37/2.25 a .58/.69 c .48 e .38 g — Social Cooperation .89A94 2.10/1.22 .65/.70 .61 .57 — Social Interaction .84A92 2.77/1.19 .62/.70 .47 .27 — Social Independence .81/.88 2.29/1.28 .66A66 .36 .20 — P R O B L E M B E H A V I O R S .96/.97 3.95/2.80 .86A78 .59 .16 — E X T E R N A L I Z I N G P R O B L E M S .95/.97 4.13/2.53 .87A78 .63 .42 — Self-Centered/Explosive .90/.94 2.33/1.73 .87A75 .62 .28 Attention Problems/Overactive .88/.93 2.32/1.27 .86A74 .51 .48 — Antisocial/Aggressive .87A92 1.95/1.25 .81/.75 .62 .47 . — I N T E R N A L I Z I N G P R O B L E M S .87A91 3.01/2.10 .80/.70 .46 .13 Social Withdrawal .80A86 2.01/1.31 .68A63 .46 .21 — Anxiety/Somatic Problems .78A84 2.12/1.64 .81/.36 .42 .17 — Table continues M A I N S C A L E / C O M P O S I T E Scale Internal Consistency Test-Retest Interrater Chronbach's alpha SEm teacher-teacher parent-teacher parent-parent B A S C - 2 P R S - P E X T E R N A L I Z I N G P R O B L E M S .90 1 3.2 1 .81 J — — .66 k I N T E R N A L I Z I N G P R O B L E M S .87 3.6 .86 — — .69 A D A P T I V E S K I L L S .91 3.0 .83 — . . . .84 B E H A V I O R A L S Y M P T O M S IND .93 2.6 .84 — — .72 Hyperactivity .85 3.9 .81 — — .74 Aggression .84 4.0 .75 — — .53 Anxiety .81 4.4 .78 — — .57 Depression .80 4.5 .73 — — .59 Somatization .79 4.6 .84 — • — .70 Atypicality .75. 5.0 .73 — — .73 Withdrawal .83 4.1 .83 — — .78 Attention Problems .87 3.6 .72 — — .77 Adaptability .80. 4.5 .74 — — .78 Social Ski l ls . .85 3.9 .72 — . . . .64 Activities of Dai ly L iv ing .70 5.5 .80 — — .78 Functional Communication .83 4.1 .85 . . . .88 Note. Significant statistics (p < .05) shown in bold. aparent/teacher. D parent(n=26)/teacher (n=82); time interval 24 to 72 hours. c three-week interval/three month interval; based on teacher ratings (n = 82). d (n = 80). e (n = 82). f (n = 98). 8 (n = 102). ( = 62). ' General norm group, combined gender, ages 4-5 only. j median test interval 40 days (n=87). k median test interval 35 days (n=40). • —1 Appendix C : Demographic Questionnaire The following questions are a bit more sensitive to some people. If you are comfortable answering the questions, it w i l l help us know more about the families who take part in our study. If you are not we understand. 9. What is your highest education level? (check only one) € Less than high school € College, trades, or technical school € High school graduate € Some University University Degree 10. What best describes your total yearly household income? (check only one) € Less than $14,999 € $30,000 to $39,999 € $15,000 to $19,999 € $40,000 to 59,999 € $20,000 to $29,999 € $60,000 or more Thank You Very Much for Your Help!! 100 Appendix D: Letters to Participants T H E U N I V E R S I T Y O F B R I T I S H C O L U M B I A Department of Educational & Counsell ing Psychology & Special Education 2125 Main Mall Vancouver, B.C. Canada V6K 1Z4 Tel: (604) 822-0091 Fax: (604) 822-3302 Dear Parent or Gua rd ian , What Are We Doing? We are writing to see i f you wi l l help us with our research project. We want to learn more about the development of kindergarten age children. Questionnaires are often used by teachers and other people working with young children to help them better help the children with whom they work. We want to test a new questionnaire, the Devereux Early Childhood Assessment ( D E C A ) . The D E C A helps us learn more about young children's social skil ls, behaviours, and emotional growth. We think this is a good questionnaire but we need to know more about it. That is why we are asking for your help. How Can You Help Us? If you agree to help us, we wi l l mail you a package with four different questionnaires. It w i l l take you about 45 to 60 minutes to complete them. When you are done, you wi l l mail them directly back to us in a stamped self-addressed envelope. Three of the questionnaires are rating scales or tests for parents to tell us about their children's behavior. The questions tell us what you've seen your child doing, and are easy to answer. The last questionnaire asks for you to tell us a little about you and your family. Some Things You Should Know. • If you take part in our project the care your child gets in his or her classroom wi l l not be impacted. • We do not think there are risks i f you take part in this project. • Y o u may find f i l l ing out the forms interesting. It may be a chance for you to think about your child's growth. • If for some reason we see any concerns with your chi ld's social, emotional, or behavior development, we wi l l let you know and wi l l give you some suggestions of where to call for help i f you wish. • The information you tell us wi l l be shared with no one else. N o individual information wi l l be reported, and no parent or child wi l l be identified by name. • This project is a part of the Masters degree research of the co-investigator, Sarah van Leeuwen. • If you have any questions or would like to talk about this project, please phone or email us at the number/email address listed below. 102 ALL PARENTS/GUARDIANS: Please complete this card and return it your child's teacher Chi ld 's name (Please Print): Parent/Guardian's Name (Please Print): , Check one: Y E S , I am interested in participating. Please mail the study materials to the address below. N O , I am not interested in participating. (Please return the card to us anyway so we know you received our packet but do not complete the rest of the information) Complete if you are interested in participating: Chi ld 's date of birth: Day Month Year Mai l ing Address: City: Postal Code: ; Phone number: Would you like a summary of the results of this research project mailed to the above address (available in Spring 2005)? Note you may request a summary even if you decide not to participate. Just fill in your address above but make sure you check "NO, I am not interested in participating" above. Y E S , I would like a summary of the results mailed to me. N O , I don't want a copy mailed to me. 104 2. The questions will take about 45 to 60 minutes. 3. This project is part of the Masters degree thesis for Ms van Leeuwen, the co-investigator. 4. Your taking part is voluntary and will not affect any services that you, your child or your family receive from School District No. 57. 5. You have the right to withdraw from the study at any time and you have the right to refuse to answer any of the questions. 6. If there are any major concerns about your child's development, we will contact you and let you know some places you can get some support in the community. In addition, we have enclosed a list of places to contact if you have concerns about your child's development. 7. This information is confidential. No individual information will be reported and no parent or child will be identified by name in any reports about the completed study. The only people who will have access to the information you give us are the researchers working with this project. 8. By taking part in this project, you will provide information that may help to improve services to young children and their families. 9. If at any time you have any concerns about your treatment or rights as a person taking part in a research study, you may contact the Research Subject Information Line in the UBC Office of Research Services at the University of British Columbia at (604) 822-8598 (in Vancouver). If you have any questions or concerns about this project you may contact either of the researchers at the addresses listed above. Laurie Ford, PhD S a r a n v a n Leeuwen Principal Investigator Co-Investigator 106 Appendix E: U B C Behavioural Research Ethics Board Certificate of Approval 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0100659/manifest

Comment

Related Items