Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Identifying children at risk : the predictive validity of kindergarten screening measures Jacobsen, S. Suzanne 1990

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1990_A2 J32.pdf [ 10.62MB ]
Metadata
JSON: 831-1.0100582.json
JSON-LD: 831-1.0100582-ld.json
RDF/XML (Pretty): 831-1.0100582-rdf.xml
RDF/JSON: 831-1.0100582-rdf.json
Turtle: 831-1.0100582-turtle.txt
N-Triples: 831-1.0100582-rdf-ntriples.txt
Original Record: 831-1.0100582-source.json
Full Text
831-1.0100582-fulltext.txt
Citation
831-1.0100582.ris

Full Text

IDENTIFYING CHILDREN AT RISK:  THE PREDICTIVE  VALIDITY OF KINDERGARTEN SCREENING MEASURES By S. Suzanne Jacobsen B.Sc,  C a l i f o r n i a State Polytechnic University, 1973  M.A., California State Polytechnic University, 1978  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS OF THE DEGREE OF DOCTOR OF EDUCATION in THE FACULTY OF EDUCATION Department of Educational Psychology and Special  Education  We accept this thesis as conforming to the required standard  THE UNIVERSITY OF BRITISH COLUMBIA August 1990 ® S. Suzanne Jacobsen  In  presenting this  degree at the  thesis  in  University of  partial  fulfilment  of  of  department  this or  thesis for by  his  or  requirements  British Columbia, I agree that the  freely available for reference and study. I further copying  the  representatives.  an advanced  Library shall make  it  agree that permission for extensive  scholarly purposes may be her  for  It  is  granted  by the  understood  that  head of copying  my or  publication of this thesis for financial gain shall not be allowed without my written permission.  The University of British Columbia Vancouver, Canada  DE-6 (2/88)  Abstract Identifying Children at Risk:  The Predictive Validity  of Kindergarten Screening Measures The early identification of children who are "at risk" of experiencing learning problems is of interest to educators and policymakers. Conflicting evidence exists regarding the efficacy of screening measures for identifying children "at risk".  The rationale for screening  programs is that early identification of problems allows for treatment which may eliminate more severe problems from developing.  If a student  is identified as "at risk", school personnel may intervene with remedial programs.  Subsequently, i f the student succeeds, the earlier prediction  is no longer valid.  The identification of "at risk" would appear  inaccurate because the intervention was successful in improving s k i l l s . Researchers often measure the prediction of "at risk" with a correlation coefficient.  To the extent that the intervention is successful, the  correlation of the identification of "at risk" with later measures of achievement is lowered. One of the problems with research on early prediction has been failure to control for the effects of the interventions which were implemented as a consequence of screening. An evaluation of "at risk" prediction is important because results of screening procedures are used to make decisions about retentions and the allocation of special services. The purpose of this study is to investigate the relationship between kindergarten screening measures and grade three achievement for two entire cohorts enrolled in 30 schools in one school d i s t r i c t . The ii  analysis employs a two-level  hierarchical linear regression model to  estimate the average within-school relationship between kindergarten screening measures and grade three achievement in basic s k i l l s , and determine whether this relationship varies significantly across  schools.  The model allows for the estimation of the relationship with control for individual pupil characteristics such as age, gender and physical problems.  The study examines the extent to which the relationship  between kindergarten screening and grade three achievement is mediated by children receiving learning assistance or attending extended primary schooling.  (4-year)  The study also examines differences among schools  in  the kindergarten screen/achievement relationships and the achievement of "at risk" pupils by including school characteristics in the analysis. The results of this study indicate positive relationships between kindergarten screening measures and achievement outcomes, even after controlling for age, gender and physical conditions. screen/achievement  The kindergarten  relationship did not vary among schools.  The study  f a i l e d to demonstrate that controlling for interventions would improve the kindergarten screen/achievement relationship; in fact the effects were in the opposite d i r e c t i o n . Levels of adjusted achievement of pupils who obtained scores at the cut-off point for risk status varied significantly among schools.  The  "at risk" pupils performed better on a l l four achievement measures in schools with high school mean-ability than similar pupils in schools with low school mean-ability. These results show that progress in the study of the predictive v a l i d i t y of screening measures can be made through the use of  iii  hierarchical regression techniques.  Researchers need to give  consideration to the effects of educational interventions and the contextual effects of schools.  iv  Table of Contents Page Abstract  ii  Table of Contents  v  L i s t of Tables  viii  L i s t of Figures  ix  Acknowledgements  x  Chapter 1:  Introduction  1  Background of the Problem Purpose of the Study Definition of Terms Research Questions Rationale Limitations of the Study J u s t i f i c a t i o n for the Study Organization of the Study  1 6 6 8 9 10 11 12  Chapter 2:  . .  Review of Literature  Part 1:  13  Early Identification of Risk  13  Underlying Assumptions of Risk At Risk On Screening Measures Advantages of Early Identification Disadvantages of Early Identification Status on Criterion Measures Problems in Identification Kindergarten Screening Screening Instruments Human Figure Drawing Test Copying of Geometric Shapes Tests of Language Development Knowledge of Letters and Numbers Summary of Screening Tests  14 15 17 18 20 23 24 28 29 31 33 35 37  Part 2:  Factors Which May Affect Prediction Research  Age at Entry Gender Differences Health Problems and Physical Problems Educational Interventions Remedial assistance and i t s effect on achievement. . . Retention and i t s effect on achievement Contextual Effects  v  38 38 39 41 43 44 47 50  Part 3:  Prediction Studies  55  R e l i a b i l i t y and V a l i d i t y 55 Methodological Paradox of Prediction-Performance Research . 58 Prediction-Performance Research 59 Correlation Analysis 59 T-tests and ANOVA 60 Discriminant Analysis 62 Prediction-Performance Matrices 63 Multiple Regression 68 Multilevel Modelling 69 Prediction Studies Summary 71 Summary Chapter 3:  73 Research Methodology  76  Introduction Subjects Procedures Instruments Draw-a-Person Test Mann-Suiter Visual Motor Screen Kindergarten Language Screening Test Deverell Test of Letter and Numbers Canadian Test of Basic S k i l l s Research Questions Analysis of the Data Data Analyses Preliminary Analyses HLM Analysis Threats to V a l i d i t y Summary Chapter 4:  Findings  76 78 79 82 83 83 84 84 85 85 89 89 91 96 96 97 99 100  Introduction Correlation Matrix Format of the Tables Model I: Kindergarten Screening Measure/Achievement Relationships Model II: Controlling for Pupil Characteristics Model I I I : Controlling for Educational Interventions Model IV: School-Level Variables Parameter Variance Explained Model V: Four Kindergarten Screening Measures i n the Model Model VI: Simplified Models Including Only Significant Variables Summary  vi  100 100 102 103 108 112 115 118 . . 120 124 124  Chapter 5:  Summary and Conclusions  Overview of the Study Principal Findings of the Study Limitations of the Study Implications of the Study Recommendations for Future Research Research References Descriptors of Physical Problems  Appendix B:  Number of Pupils Identified with Physical Problems  and Number Receiving Interventions Technical Information - Draw-A-Person  Appendix D: Technical Information - Mann-Suiter Visual Motor Screen Appendix E: Technical Information - Kindergarten Language Screening Test Appendix F: Technical Information - Deverell Test of Letters and Numbers Appendix G:  175  176 177  178 179  180  Technical Information - Canadian Tests of Basic S k i l l s  (CTBS) Appendix H:  125 126 140 141 144 148  Appendix A:  Appendix C:  125  182 Characteristics of Schools i n the Study  183  Appendix I: Use of Grade Equivalent Scores  184  Appendix J :  Data Plots Reflecting Interventions  187  Appendix K:  Graphic Representation of Predictive U t i l i t y  191  Appendix Table 1:  Summary of Selected Prediction Performance  Studies Appendix Tables 2:  195 HLM Results for A t t r i t i o n  201  Appendix Tables 3-20: HLM Results  202  Appendix Table 21: Means and Standard Deviations of Outcome Measures For Four Samples Appendix Table 22: Prediction-Performance Matrix Analysis  220 221  vii  List of Tables Page Table 1:  Table 2:  Table 3:  Table 4:  Table 5:  Table 6:  Table 7:  Means, Standard Deviations, and Correlations of Student-Level Variables  101  D i s t r i c t and Sample Means and Intercepts for Pupils at the Risk Cut-off Score  103  Estimates of the Effects on Grade Three Achievement of One Point (or one SD) Kindergarten Screening Measure Score  106  Estimated Residual Parameter Variance of Mean Achievement for Pupils at the Cut-Off Score for Risk Status  Ill  Estimated Coefficients for Kindergarten Screen/ Achievement Relationships  114  "Null-Model" Estimates of Variance in Grade Three Achievement  119  Average Achievement Scores for Pupils Who at the Cut-off Point for "Risk" Status  122  viii  Scored  List of Figures Page Figure 1: Effect of Changing the Cut-off Score of the Predictor or Criterion Measure  22  Figure 2: Prediction-Performance Comparison Matrix  64  Figure 3: Numerical Example of Prediction-Performance Matrix  . . .  66  Figure 4: Administration of Kindergarten Screening Measures  . . .  82  ix  Acknowledgements  I would like to thank the members of the committee for their support and p a r t i c i p a t i o n , Dr. D. Willms, Dr. P. L e s l i e , Dr. J . Conry and Dr. O.A.  Oldridge. I wish to extend my appreciation and gratitude to Doug Willms.  provided the opportunity to conduct the study.  He  He patiently guided me  through the hierarchical linear modelling analysis.  He contributed ideas  to the manuscript at every stage and extended support and encouragement throughout the study. I am indebted to him for his contributions to t h i s work and to my education.  I wish to thank him for being a better advisor  than I had hoped f o r . I wish to thank Steve Raudenbush for the long-distance support and HLM trouble-shooting.  He w i l l i n g l y took phone c a l l s at any hour and  patiently provided advice on the analyses. L a s t l y , I owe special appreciation to my husband John, for his support and encouragement; to our daughters, Sulynn and Teresa, f o r t h e i r untiring support, patience and help; and to our son K i r i n , whose surprise a r r i v a l made being a student d i f f i c u l t but whose presence brings us great joy every single day.  x  1  Chapter 1  Introduction  Background of the Problem Educators and policy makers are interested i n the early prediction of children's school achievement because i t i s important to intervene with appropriate educational strategies for children "at r i s k " of experiencing d i f f i c u l t y i n school achievement.  Researchers estimate that  between 15 and 25 percent of the children i n school have learning d i f f i c u l t i e s and thus, are "at risk" of school f a i l u r e (Satz & F r i e l , 1978;  Norton, 1979).  For the educational system to respond to the needs  of "at r i s k " children, their early i d e n t i f i c a t i o n i s a p r i o r i t y . In recent years, screening programs have been implemented throughout Canada and the United States for the purpose of early i d e n t i f i c a t i o n and treatment of children who display signs of possible learning problems (Norton, 1979).  Individual school d i s t r i c t s are responsible for the  selection and administration of particular screening instruments and provide the resources for subsequent intervention programs.  These  screening programs involve the evaluation of large groups of children with b r i e f , low-cost procedures.  T y p i c a l l y , a screening test or battery  w i l l include developmental measures and measures of s p e c i f i c s k i l l s related to academic performance. The findings are used to make some i n i t i a l d i s t i n c t i o n between children who are expected to progress successfully and those who may need special services. The rationale for screening programs i s that early i d e n t i f i c a t i o n of  2  problems allows f o r treatment which may prevent more severe problems from developing.  The value of identifying children  "at risk" of school  f a i l u r e i s determined by the intervention e f f o r t s which follow the i d e n t i f i c a t i o n (Mercer, Algozzine & T r i f i l e t t i , 1988).  Screening,  therefore, i s only the f i r s t step i n a process aimed at identifying s p e c i f i c s k i l l s prerequisite to successful academic performance f o r individual children.  The results of screening should a l e r t educators to  general areas of delayed development and lead to preventive action to improve academic performance. Some educators oppose the use of screening techniques because the outcome of the screening process involves labelling children "at r i s k " or "not at r i s k " .  The application of a label "at risk" may lead to further  assessment and intervention which may eventually result i n the c h i l d acquiring another label such as "learning disabled" or "mentally handicapped". include:  Researchers have stated the negative effects of l a b e l l i n g  the label has an adverse effect on teacher expectancy (Foster,  Schmidt & Sabatino, 1976; Foster, Ysseldyke & Reese, 1975); classmates, teachers and parents perceive the child more negatively than their normal peers ( P u l l i s & Smith, 1981) and the child's self-concept i s reduced (Guskin, Bartel & McMillan, 1975). Although educators are not in complete agreement regarding the value of  kindergarten screening, l e g i s l a t i o n in the United States (Public Law  94-142) and i n certain provinces in Canada (e.g., B i l l 82 i n Ontario), requires school d i s t r i c t s to identify "high risk" children in kindergarten.  T y p i c a l l y , the i d e n t i f i c a t i o n of these children involves  the use of t e s t s , but the v a l i d i t y of screening measures i s frequently  3  questioned.  For a screening program to be effective both content  v a l i d i t y and predictive v a l i d i t y must be given consideration. v a l i d i t y i s also important.  Construct  Construct v a l i d i t y i s not necessary f o r  prediction research in which the purpose of the study i s to find predictors of a particular c r i t e r i o n (Borg & G a l l , 1983).  However,  i t i s essential to prediction research which relates theory to research findings.  Screening measures with construct v a l i d i t y strengthen the  predictive v a l i d i t y of the measures in theory-based research. Some screening instruments f a i l to meet sufficient standards of technical adequacy for use in making decisions about children (Bracken, 1987). Shepard and Smith (1986) point out that the v a l i d i t y of a screening instrument depends on how the test is used and is entwined with the effectiveness of interventions that follow. Most predictive v a l i d i t y studies express v a l i d i t y in terms of correlation coefficients which give no indication of the accuracy or e f f i c i e n c y of the t e s t s .  The result i s that screening tests are commonly  selected on the basis of inadequate data, face v a l i d i t y , testimonial evidence, or frequency of use by other screening programs (Lichtenstein, 1981). One concern regarding the v a l i d i t y of screening programs i s the lack of theoretical framework for conceptualizing the nature of handicapping conditions and the precipitating factors which lead up to them.  Satz,  Taylor, F r i e l and Fletcher (1979) state, "without a testable theory one lacks guidelines f o r the selection of a test battery which purports to identify the potentially high risk child" (p. 318).  Jarman (1980) notes,  however, that the lack of theoretical rationale i s i m p l i c i t l y encouraged  4  by the exploratory nature of prediction research. Policy makers and legislators are asking for better evidence to document the immediate and long-term effects and cost-benefits of early intervention (White, 1986).  Judy (1986) suggests the long-term  consequences of screening include significant savings to society i n terms of services that w i l l not be required, increased educational productivity and enhanced self-concept of children who otherwise might experience academic f a i l u r e before assistance could be provided. Many researchers have developed screening and readiness tests to assess facets of children's development assumed to be related to later school achievement (de Hirsch, Jansky & Langford, 1966; Book, 1974; Satz & F r i e l , 1978; Beery & Buktenica, 1982; Goldman, F r i s t o e , & Woodcock, 1970; Gauthier & Madison, 1973; Deverell, 1974; Harris, 1963).  Despite  many years of research directed at early i d e n t i f i c a t i o n the results are inconclusive.  Researchers do not know which factors predict risk of  f a i l u r e , whether remedial interventions affect achievement or whether early screening tests r e l i a b l y predict achievement for children "at risk". Reviews of research provide conflicting evidence regarding the efficacy of screening measures for identifying children who risk learning failure.  Factors which contribute to c o n f l i c t i n g findings include:  short time-frame prediction (the usual study i s kindergarten to grade one); small samples (Glazzard, 1979; Meyers, Attwell, & Orpet, 1968); f a i l u r e to consider the effects of gender (Badian, 1986); and f a i l u r e to consider minor physical conditions which may affect academic performance.  Wendt (1978) reports that there i s considerable v a r i a b i l i t y  5 in the purposes for screening and types of measures, and thus, not a l l investigations are d i r e c t l y comparable. An important issue of prediction research concerns the confounding effects of interventions on the relationship between screening and subsequent achievement.  If students are suspected of potential learning  problems because of low kindergarten screening scores, school personnel may intervene with some remedial program.  In schools with good remedial  programs, children identified as "at risk" would, on average, a t t a i n higher achievement scores than children i n other schools with comparable screening scores who received l i t t l e or no remediation.  The e a r l i e r  i d e n t i f i c a t i o n of "at risk" would therefore appear inaccurate for some schools because the intervention was successful i n improving skills.  their  In e f f e c t , successful interventions would lower the c o r r e l a t i o n  of the i d e n t i f i c a t i o n of "at risk" with later measures of achievement. The decision for intervention may be dependent on factors other than awareness of screening information.  Also, individual schools may vary i n  teaching practices, allocation of resources, a v a i l a b i l i t y or intensity of interventions or class size and heterogeneity (Raudenbush & Bryk, 1986). Although there i s not consensus among researchers, there i s growing evidence that children perform better i n schools with a high SES enrollment, even after family background characteristics are considered (Willms & Raudenbush, 1989; Willms, 1986). effects.  This i s called contextual  These, and other factors, may also influence the relationships  between screening scores and subsequent achievement. Nearly a l l research on early i d e n t i f i c a t i o n has been concerned with the relationship between screening measures and subsequent achievement,  6 but few studies have given consideration to whether the relationship varies across schools, or to the effect that intervention e f f o r t s have in mediating the relationships.  may  This study examines the  relationships between kindergarten screening measures and grade three achievement in basic s k i l l s .  It investigates whether the relationships  vary s i g n i f i c a n t l y across schools, and examines the extent to which the relationships are mediated by children receiving learning assistance or attending extended primary schooling.  The study also examines whether  the inclusion of school level variables in the analysis, such as school s i z e , heterogeneity, rural or urban location or mean a b i l i t y , provides explanation for between school differences in the r e l a t i o n s h i p s . Purpose of the Study The purpose of the present study i s to examine the relationships between performance on four kindergarten screening instruments  and  achievement test scores obtained at grade three, for two entire age cohorts enrolled in 30 schools in one school d i s t r i c t .  The analysis  employs a two-level hierarchical linear regression model to estimate the average within-school relationship between kindergarten screening measures and grade three achievement in reading, mathematics, vocabulary and language and determines whether these relationships vary s i g n i f i c a n t l y across schools.  The analysis controls for the effects of  individual student characteristics of age, gender and physical problems. The study also examines the extent to which the relationship between kindergarten screening and grade three achievement is mediated by children receiving learning assistance or attending extended primary schooling.  The study also examines the extent to which the between  7  school differences can be explained by school level variables of school s i z e , heterogeneity, rural or urban location and mean school a b i l i t y . D e f i n i t i o n of Terms The definitions of educational terms used in this study are l i s t e d below: Screening refers to a systematic examination of supposed prerequisite s k i l l s and a b i l i t i e s necessary for learning in school. The absence of these i s expected to identify educationally "at r i s k " children at an early stage (Stevens, 1987). Screening tests are tests which provide a brief assessment of a child's developmental a b i l i t i e s , particularly those which are highly associated with future school success. Learning d i f f i c u l t i e s i s used to describe a l l conditions ( i . e . , learning disabled, developmental delay, experientially deprived, educationally handicapped, etc.) which result i n the i d e n t i f i c a t i o n of children who may have d i f f i c u l t y in achieving the goals of education. "At r i s k " i s defined as the designation acquired from performance on a kindergarten screening measure.  It i s used i n the l i t e r a t u r e to refer  to children who have s i g n i f i c a n t l y low test scores on screening measures and who are l i k e l y to be retained i n grade or referred for special educational services.  It also i s used to refer to children who f a i l to  make adequate progress in school and therefore, are achieving below the level expected for their age and grade (Karweit, 1988).  The l i t e r a t u r e  i s confusing because "at risk" refers both to the prediction of a child's subsequent status and to students who have f a i l e d to make adequate progress.  In this study, "at-risk" i s used i n the former sense; i t w i l l  8 be used to describe children who score at or below a particular score on a kindergarten screening measure. Learning assistance i s individual instruction provided by a teacher other than the classroom teacher to remediate d e f i c i t s k i l l s .  It i s  usually provided during regular classroom time in a room other than the classroom. Extended primary consists of attending school four years to complete the three-year primary curriculum.  It may represent a program of  continuous progress or may be an alternate form of retention. Research Questions The purpose of this study i s to examine the relationship between kindergarten screening measures and grade three achievement i n reading, mathematics, vocabulary and language.  The study also examines the extent  to which the relationship between kindergarten screening and grade three achievement i s mediated by the provision of learning assistance or extended primary. 1. a)  The study examines four research questions:  What i s the average within-school relationship between grade three  test scores i n academic achievement and scores on kindergarten screening measures of perceptual-motor, language and cognitive s k i l l s ? b)  To what extent do the relationships between achievement scores and  kindergarten screening scores vary across schools? 2.  a) What i s the relationship between grade three achievement and  kindergarten screening after controlling for the effects of gender, age at entry to kindergarten and whether the child has a physical problem? b) Do the relationships between grade three achievement and kindergarten screening vary across schools after taking account of pupil  9  characteristics? 3.  a) To what extent are the relationships between grade three  achievement and kindergarten screening mediated by the provision of learning assistance or extended (4 year) primary schooling? b) Does the extent to which the relationships are mediated vary across schools? 4.  a) If there i s significant variation between schools in their  relationships between screening and outcome measures, to what extent can i t be explained by school s i z e , rural versus urban location or the school mean and variance of pupils' a b i l i t y ? b) To what extent are the between-school differences in achievement explained by various school-level variables? Rationale The study addresses some of the shortcomings of previous research. Despite many years of research directed at early i d e n t i f i c a t i o n of children "at risk" for learning problems, the results are inconclusive. Prior research has been limited by factors that contribute to c o n f l i c t i n g findings:  short time-frame prediction, small samples, f a i l u r e to  consider the effects of gender,  and f a i l u r e to consider minor physical  conditions that might affect academic achievement.  The factors which  predict r i s k of f a i l u r e , whether early screening r e l i a b l y predicts achievement f o r children "at risk" and in what ways intervention affects achievement are not known with certainty.  Numerous researchers state  that screening for the i d e n t i f i c a t i o n of "at risk" learners i s of v i t a l importance to help prevent children from exposure to recurrent f a i l u r e and f r u s t r a t i o n (Norton, 1979; Keogh & Becker, 1973; Barnes, 1982).  This  10 study extends the examination of prediction research by controlling for the effects of remedial  interventions during the study and by employing  techniques of analysis that investigate variation across schools. This study i s a result of the need for further investigation into the area of early i d e n t i f i c a t i o n of "at risk" children using a s t a t i s t i c a l technique which considers the hierarchical nature of educational programs.  In a sense i t serves as a case study of one  d i s t r i c t ' s screening practices.  The analysis allows for the examination  of the relationships within schools, as well as the study of how between-school relationships may  the  vary. This study i s an attempt to gain  greater understanding of the relationships between early screening information and academic achievement, giving consideration to the effects of educational interventions and to school e f f e c t s . Limitations of the Study This study uses extant data available from a school d i s t r i c t screening program that was  implemented for a period of ten years.  The  particular screening measures are not necessarily the best predictors of r i s k status, and the outcome measures are limited to the standardized achievement tests scores.  Admittedly, achievement tests do not cover a l l  the important goals of schooling.  Nevertheless, the system was  operating  in this d i s t r i c t and i s better than most screening programs i n that there are four measures administered at different times of the year.  A l s o , the  d i s t r i c t established d i s t r i c t norms for the cut-off scores on the screening measures. This study i s limited also in the following ways: - the four screening instruments measure specific but limited  11 aspects of development; - i t i s concerned exclusively with academic outcomes as measured on standardized t e s t s ; - the physical problems are grouped as a single variable and  thus,  there is no discrimination between the effects of hearing, v i s i o n , speech or physical handicaps; - although, the analysis controls for whether pupils received some intervention, there is no data describing the type or quality of the specific treatments administered, their timing or t h e i r duration; - s i m i l a r l y , the school-level measures are proxies for processes and p o l i c i e s operating within the schools.  Although the inclusion of  these variables is useful for controlling for their e f f e c t s , the analysis does not provide the kind of detailed  information  necessary for informing school p o l i c y . J u s t i f i c a t i o n for the Study If screening procedures determine whether children receive special services or are retained a year or more, then an evaluation of the accuracy and predictive v a l i d i t y of screening, giving consideration the effects of interventions, i s necessary.  to  The primary emphasis of  screening i s to predict which children are l i k e l y to experience school problems and to provide intervention to children who progress successfully.  require i t to  This study addresses the need for further  investigation into the area of early i d e n t i f i c a t i o n of pupils designated "at r i s k " using hierarchical linear regression analysis.  Analysis which  allows for the investigation of the within-pupil and between-school  12 variation i s an attempt to gain greater understanding of the relationships between early screening information and grade three achievement, giving consideration to the effects of educational interventions and school e f f e c t s . Organization of the Study Chapter 2 is a Review of Literature which is divided into three major parts.  Part one reviews the literature on early i d e n t i f i c a t i o n and  screening practices.  Part two discusses variables which may  the predictive v a l i d i t y of screening programs. prediction-performance chapter.  influence  Part three describes  methodology and concludes with a summary of the  Chapter 3 describes the methodology for the study.  presents the findings of the study.  Chapter 5 summarizes the f i n d i n g s ,  discusses the implications for educational policy and recommendations for future research. follow Chapter 5.  Chapter 4  provides  The bibliography and appendices  13  Chapter 2 Review of Literature To investigate the relationship between kindergarten screening measures and academic achievement a review of several important areas i s necessary.  The area of screening and early i d e n t i f i c a t i o n of children at  risk for experiencing d i f f i c u l t y in learning is complex.  Researchers  apply the d e f i n i t i o n of "at risk" f i r s t to performance on screening measures and secondly, to subsequent performance on achievement outcome, measures.  The particular areas of development considered to underlie  achievement and the means to measure those areas must be considered. Prediction research requires a time lag between screening and outcome thus, intervening factors which occur during the time under study and which may  affect prediction findings merit discussion. The methods of  analyzing prediction research vary greatly, and the means chosen affect the interpretation and application of the findings. To address these areas the review of literature is divided into three principal parts.  The f i r s t part is a review of early  i d e n t i f i c a t i o n , screening practice and screening instruments. part discusses factors which may  affect prediction research.  part discusses prediction-performance  The second The t h i r d  studies and examines methods of  analysis for prediction-performance research.  The chapter concludes with  a summary of the important considerations from the review of l i t e r a t u r e . Part 1 Early Identification of Risk The early i d e n t i f i c a t i o n of children "at risk" of  experiencing  learning d i f f i c u l t i e s receives wide support from professionals of varied  14  d i s c i p l i n e s and from parents (Keogh & Becker, 1973). A variety of s o c i a l , emotional, i n t e l l e c t u a l , b i o l o g i c a l , physical, l i n g u i s t i c , environmental or any combination of such factors may interfere with a child's optimum growth and normal development and result in the child requiring special attention.  Before school age, the i d e n t i f i c a t i o n of children "at r i s k "  generally rests with professionals in the f i e l d s of health and social welfare.  When children reach school-age, the educational system assumes  primary r e s p o n s i b i l i t y for the i d e n t i f i c a t i o n of students believed to be "at risk" of experiencing d i f f i c u l t y in school achievement. Many educators believe the i d e n t i f i c a t i o n of the school-age child with learning d i f f i c u l t i e s should be made as early as possible i n the child's school career to prevent more serious learning problems from developing (Haring & Ridgway, 1967; Judy, 1986; Norton, 1979; Barnes, 1982).  Most  children enter the public school system in kindergarten, which provides the e a r l i e s t opportunity for screening for potential d i f f i c u l t i e s . Underlying Assumptions of Risk The implementation of screening practices for identifying children i s based on several common assumptions regarding models of r i s k . and Daley (1983) identify three important assumptions.  Keogh  F i r s t , the  problems in development have their primary locus in the c h i l d . Second, development i s assumed to be continuous such that early problems are precursors of subsequent problems.  T h i r d , differentiated treatments or  interventions are assumed to be d i r e c t l y linked to risk or d i s a b i l i t y conditions.  Two additional assumptions identified by Barnes (1982) are  that the c h i l d at risk can be detected early in a r e l i a b l e and predictable manner and that the e a r l i e r a child's potential d i s a b i l i t y i s  15  detected, the more effective and beneficial treatment w i l l be. At Risk On Screening Measures Describing educational risk and detecting children who f a l l such a c l a s s i f i c a t i o n are not simple tasks.  into  Numerous d e f i n i t i o n s of risk  are used which confound explanations and contribute to inconsistent research findings and make comparisons of research findings d i f f i c u l t . Researchers use poor performance i n areas of developmental s k i l l s , maturity, or preacademic s k i l l s as indicators of risk status for young children.  In c l a s s i f y i n g the "at risk" population some researchers  identify a s p e c i f i c c r i t e r i o n , or "cut-off" score, on a p a r t i c u l a r instrument or battery of tests to indicate risk status (Fletcher & Satz, 1982; Book, 1974), while others use a vague d e f i n i t i o n (Jansky, 1978; Stevens, 1987; McCann & Austin, 1988; Karweit, 1988).  Z e i t l i n (1976)  defined the "at risk" population as those children, who because of problems of development and experience, are least able to meet the expectations of the school.  Leigh (1983) stated that children who lack  essential preacademic knowledge and s k i l l s are at risk f o r academic failure.  Lerner, Mardell-Czudnowski and Goldenberg (1981) defined  preacademic d e f i c i t s as any d e f i c i t i n cognitive, affective or psychomotor domains that might hamper the child's progress in learning and i n school. Opinions d i f f e r regarding the number of children at risk f o r experiencing d i f f i c u l t y i n learning academic s k i l l s .  Norton (1979)  estimates that 25 percent of children entering school show some signs of developmental deviations in areas having a slower rate of development such as language, perceptual s k i l l s and attentional maturity.  Mardell  16  and Goldenberg (1975a) used the lowest 10 percent of screening scores to identify high-risk children, children who were seriously behind other children of the same age, gender and location. An important concept underlying screening of young children i s that of readiness.  The concept of readiness i s b u i l t on the b e l i e f  formulated  1  during the 1920 s and 1930's that a child developed in predetermined stages which could be measured by a readiness test (Durkin, 1974). Numerous tests were developed and used for early i d e n t i f i c a t i o n of risk status.  A high score on a readiness test indicated the child was  ready  to learn while a low score indicated the child was not ready and instruction should be delayed.  Children who encounter d i f f i c u l t y in  school often were described as immature or lagging in development. de Hirsch, Jansky and Langford (1966) found a close link between children's maturational status in kindergarten and reading and spelling achievement several years l a t e r .  In a landmark project on screening,  they developed a Predictive Index to be used as a screening device for identifying children who were l i k e l y to experience d i f f i c u l t y i n learning to read.  A later study by Jansky and de Hirsch (1972) concluded  that  behavior, maturation, language and reading readiness were the factors which best predicted academic performance. After many years of research numerous individual screening tests and screening batteries have been developed.  Most tests and batteries of  tests include tasks to measure both developmental a b i l i t i e s and readiness skills.  Meisels (1987) defined and analyzed the differences between  developmental screening tests and readiness.  He indicated that  developmental tests provide a brief assessment of a child's developmental  17  a b i l i t i e s which are highly associated with future school success. Readiness tests are concerned with the curriculum-related s k i l l s a c h i l d has acquired, s k i l l s that are t y p i c a l l y prerequisite for s p e c i f i c instructional programs.  The primary areas included in early  i d e n t i f i c a t i o n assessment are language, i n t e l l i g e n c e , motor s k i l l s , social-emotional development and preacademic s k i l l s .  (For reviews see:  Dykstra, 1967; Satz & Fletcher, 1979; Book, 1974; Paget & Bracken, 1983; Bracken, 1987; Mercer, Algozzine & T r i f i l e t t i , 1988.)  Children are  i d e n t i f i e d as potentially "at risk" of experiencing learning d i f f i c u l t i e s when their obtained scores are below a designated c r i t i c a l level of performance on a screening test or a battery of t e s t s . Advantages of Early Identification Proponents of early i d e n t i f i c a t i o n of children "at r i s k " of experiencing school d i f f i c u l t i e s suggest several advantages.  Keogh and  Becker (1973) suggest that the sooner treatment begins, the greater the likelihood the treatment will have a positive impact and the treatment may prevent the development of other deleterious conditions or compounding problems.  The early i d e n t i f i c a t i o n of handicapping  conditions allows for family adjustment and acceptance which may  result  in additional support for intervention efforts (Hayden, 1974; Keogh & Becker, 1973).  The implementation of well-designed early intervention  can y i e l d many positive benefits f o r children and their f a m i l i e s . may include:  These  enhancing developmental progress as compared with children  who are not provided with appropriate interventions (Casto & Mastropieri, 1986; Reynolds, Egan & Lerner, 1983); improved school performance as demonstrated by fewer grade retentions (Leinhardt, 1980), better academic  18 test scores and a reduction in numbers of school dropouts (Lazar & Darlington, 1978; Schweinhart & Weikart, 1986); and reducing the total number of years of special education required, resulting in a s i g n i f i c a n t cost savings (Keogh & Daley, 1983; Lazar & Darlington, 1978). The stated goals of screening programs include intention to provide appropriate interventions following screening, however the results of an extensive program of early childhood screening in f i v e Head Start centers do not support the stated goals.  Richardson-Koehler  (1988) found that  the screening data was used to identify special needs c h i l d r e n , but  was  not used to provide feedback about individual or group needs, nor for planning and implementing instruction geared to meeting the i d e n t i f i e d needs of c h i l d r e n .  Teachers and aides had l i t t l e understanding  purposes and appropriate uses of the test r e s u l t s .  of the  They viewed the  i d e n t i f i c a t i o n of special needs children as unrelated to other aspects of t h e i r work. Disadvantages of Early Identification The major concern expressed regarding screening programs i s that the outcome of screening may  lead to "labelling" children "at r i s k " .  The  l i t e r a t u r e describing early i d e n t i f i c a t i o n and categorizing children as "at r i s k " or "not at risk" f a i l s to support either negative or positive effects leading from the labels.  Keogh and Becker (1973) suggest that  when a c h i l d i s identified "at r i s k " , a set of expectancies, anxieties, and d i f f e r e n t i a l treatment patterns may develop both at home and at school which may  be detrimental to the development of the c h i l d .  Most  concerns regarding the labelling of children stem from the belief that teacher or parent expectancy results in a " s e l f - f u l f i l l i n g prophecy  19  e f f e c t " as suggested by Rosenthal and Jacobson (1968). Palady (1969) showed a strong relationship between teacher expectancy and student performance.  In a study involving two groups of  22 elementary teachers, Foster, Schmidt and Sabatino (1976) concluded  the  label "learning disabled" generated negative expectancies which affect the teacher's objective observation of behavior and may the child's academic progress.  be detrimental to  They suggested that labelling may  serve  as s e l f - f u l f i l l i n g prophecy because of lowered expectations among parents and teachers.  In reporting the findings of a rating task given to 38  teacher trainees, Foster, Ysseldyke and Reese (1975) reported that the teacher trainees held negative stereotypical expectations of children labelled "emotionally disturbed". Bak, Cooper, Dobroth and Siperstein (1987) administered a questionnaire to 77 fourth-through sixth-grade children.  Findings  indicate that the children enrolled in regular classrooms who did not have special needs held higher expectations of the c a p a b i l i t i e s for the special needs children enrolled in a resource room than for the special needs children enrolled in the special c l a s s .  The authors conclude that  special class placements can act as de facto labels. MacMillan, Jones and Aloia (1974) reviewed the l i t e r a t u r e and found few studies investigating the effects of labelling children. concluded that the available data was of  labelling.  They  inconclusive regarding the effects  They noted that children identified as handicapped  received special education assistance which confounded the effects of l a b e l l i n g with the effects of special class placement.  Rogers, Smith and  Coleman (1978) contend that special class placement had a favorable, not  20  negative, impact on children's self-concepts although, their investigation did not investigate the effects on achievement or sustained positive self-concept i f the children were returned to regular class placement. The outcome of screening does not necessarily lead to a c h i l d being labelled as learning disabled, emotionally disturbed or some other such categorical l a b e l .  Although categorical labels may  be avoided,  (1977) suggested that terms such as "high r i s k " or "at r i s k "  Keogh  may  themselves assume the characteristics and consequent detrimental effects of  labels. Mercer, Algozzine & T r i f i l e t t i  disadvantages  (1988) summarize the major  of early i d e n t i f i c a t i o n programs as follows:  "Since  measurement inadequacies and d i f f e r e n t i a l developmental problems make i t d i f f i c u l t to accurately diagnose children "at r i s k " , the major disadvantage  of early i d e n t i f i c a t i o n becomes evident.  Many children  who  are not disabled receive a d i s a b i l i t y label and the detrimental effects of that label present a problem to the child and his/her family." Status on Criterion Measures One cannot discuss prediction without an adequate d e f i n i t i o n of the criterion.  The problem i s the number of possible c r i t e r i o n measures i s  as varied as the number of schools using screening programs.  The  determination of risk status on a c r i t e r i o n measure i s equally v a r i e d . Some researchers use a specific c r i t e r i o n , or "cut-off" score to indicate risk status. Fletcher and Satz (1982) used a designated number of years below grade level to discriminate mild and severe d e f i c i t s from average performance.  Jansky (1978) used the c r i t e r i a of " f a i l e d in reading at  21 the end of second grade".  The bottom quarter of the achievement  d i s t r i b u t i o n has also been described as representing the population not meeting with success (Book, 1974; Karweit, 1988).  Other researchers use  vague definitions or broad descriptors for students who f a i l to perform at a predetermined level such as:  students f a i l i n g i n their f i r s t years  (Stevens, 1987); and students who have increased probability of learning problems, adjustment d i f f i c u l t i e s , or dropping out of school (Scott, 1981). Most researchers attempt to dichotomize the outcome variable by selecting a particular cut-off score.  Children who f a l l below the  cut-off score are considered to be most vulnerable for dropping out or for  longer term f a i l u r e .  Researchers often use the term "at r i s k " to  refer to children who score below the cut-off point on a c r i t e r i o n measure.  In f a c t , Richardson-Koehler  (1988) pointed out that i n many  cases the term "at risk" has taken over from such descriptors as disadvantaged, low SES, underachieving and problem c h i l d r e n . In many studies researchers use screening measures to categorize children as "at risk" or "not at r i s k " .  They also dichotomize the  outcome variable using a predetermined cut-off score.  When t h i s i s done,  the relationship between "at risk" status and c r i t e r i o n depends heavily on the selection of the cut-off scores.  The decisions regarding the  p a r t i c u l a r c r i t e r i o n measures and the cut-off scores selected f o r screening measures influence the v a l i d i t y of prediction of risk status. This problem makes i t d i f f i c u l t to compare results across studies even i f the same c r i t e r i o n measures are used because the cut-off scores may vary across studies.  22 Figure 1 i l l u s t r a t e s the effect of moving the cut-off score for either the screening measure or the c r i t e r i o n measure. Figure 1 Effect of Changing the Cut-off Score of the Predictor or Criterion Measure  The number of pupils identified as at risk can be increased or decreased by adjusting the cut-off score of either the predictor or c r i t e r i o n measures.  The adjustment of the cut-off score of the screening  measure has implications for the provision of services for pupils "at r i s k " and for subsequent costs to school d i s t r i c t s .  The adjustment of  the c r i t e r i o n measure has implications for validating the effectiveness of the screening program.  Thus, the absolute number of pupils i d e n t i f i e d  as "at r i s k " on screening measures or of f a i l i n g to achieve at a c r i t i c a l l e v e l , is an a r t i f a c t of the cut-off scores selected. Another important point which makes comparisons across studies d i f f i c u l t i s that local standards vary, across d i s t r i c t s and schools. McCann and Austin (1988) suggest "at risk" refers to students who, f o r whatever reason, are at risk of not achieving the goals of education, of not meeting local standards to complete their education.  Beyer and  Smey-Richman (1988) identified the "at risk" population as those who  are not meeting minimum standards of academic achievement as  students  23  determined by l o c a l l y imposed standards. standards may will  To the extent that local  vary, the i d e n t i f i c a t i o n of particular children "at r i s k "  vary.  Problems in Identification A number of d i f f i c u l t i e s present in attempting to identify kindergarten children who difficulties.  are l i k e l y to experience learning  One of the d i f f i c u l t i e s when assessing young children i s  that d i f f e r e n t i a l developmental patterns make i t d i f f i c u l t to determine i f a particular child i s truly at risk or simply needs more time to mature before becoming an e f f i c i e n t learner (Mercer, Algozzine & T r i f i l e t t i , 1988).  Young children tend to produce widely discrepant  results for the same test administered more than one time due to t h e i r changing physical, mental and emotional conditions. Judy (1986) explained that these changing conditions in young children result i n large standard errors of measurement in their test r e s u l t s . A further problem in attempting to identify kindergarten children who of  are l i k e l y to experience learning d i f f i c u l t i e s is that the conditions learning d i f f i c u l t y or f a i l u r e have not developed at the time of  i d e n t i f i c a t i o n (Keogh & Becker, 1973).  Thus i d e n t i f i c a t i o n i s an  hypothesis that a problem w i l l develop, not a confirmation that i t exists.  A related d i f f i c u l t y i s that educationally handicapping  conditions, l i k e reading d i s a b i l i t i e s or language disorders, have few, i f any, well-known etiological components.  Identification of "at r i s k " by  screening therefore refers to the presymptomatic detection of a disorder which could interfere with the child's progress i f l e f t (Barnes, 1982).  undetected  Children whose kindergarten performance deviates  24  s i g n i f i c a n t l y from "normal" ranges may be easily i d e n t i f i e d ; however, the deviation of performance for children with mild handicaps may not be great i n kindergarten but may result in learning d i f f i c u l t i e s later i n school (Paget & Nagel, 1986). Despite these d i f f i c u l t i e s , investigators in psychology and education continue working to develop methods of predicting children's achievement and identify those children "at risk" (Stevenson, Parker, Wilkinson, Hegion, & F i s h , 1976).  Screening for  children "at r i s k " i s  not a precise or exact science, p a r t i a l l y because many aspects of early development are subject to considerable variation i n time and across individuals and many intervening variables during the time under study cannot be t o t a l l y controlled. Kindergarten Screening Barnes (1982) defines screen  ing as a process of early detection to  identify those children i n the general population who may be at r i s k for a s p e c i f i c d i s a b i l i t y or who may otherwise need special services.  The  stated purposes for early screening and i d e n t i f i c a t i o n include: identifying those children who have special learning needs (Lerner, et a l . , 1981; Salvia & Ysseldyke, 1985); describing individual strengths and weaknesses, p a r t i c u l a r l y as these relate to programming (Meisels, 1985); developing appropriate recommendations for interventions t a i l o r e d to each child's individual needs (Bricker, 1986); attempting to provide preventive education, rather than to wait for problems to c r y s t a l l i z e i n later grades which would then require more costly and less e f f e c t i v e remediation strategies (Wendt, 1978); and f a c i l i t a t i n g delivery of early intervention services, thereby, enhancing eventual adjustment (White,  25  1986). Social and educational p o l i c i e s concerning screening children have been guided by the notion that prevention i s preferable to remediation (Evans, 1976; Glazzard, 1982).  The long-term consequences of screening  include s i g n i f i c a n t savings to society in terms of services that w i l l not be required, increased educational productivity and the enhanced self-concept of children who otherwise might have experienced academic f a i l u r e before assistance could be provided (Keogh & Becker, 1973; Hayden, 1974; Keogh & Daley, 1983; Lazar & Darlington, 1978). The importance of early i d e n t i f i c a t i o n of children "at risk" i s based on s o l i d evidence showing that early i d e n t i f i c a t i o n coupled with remedial assistance can reduce the risk of school f a i l u r e and subsequent grade retention (Simner, 1983; Becker & Gersten, 1982; Lazar & Darlington, 1978). Screening i s only the f i r s t step i n a process to provide appropriate services to children.  Meisels (1987) suggested that testing in  kindergarten should only be used to make better and more appropriate services available to the largest number of children.  According to Leigh  (1983), when screening i s not followed by provision of either a thorough diagnostic evaluation or some type of intervention within a reasonable period of time, early i d e n t i f i c a t i o n efforts serve no useful purpose. Keogh and Daley (1983) state that unless i d e n t i f i c a t i o n leads to d i f f e r e n t i a t e d services, the screening is wasteful. The main purpose of screening i s the examination of large age groups of children with b r i e f , low-cost procedures, to identify those children who appear to f a l l above or below certain c r i t i c a l  levels of performance  26  ( G u l l i f o r d , 1976).  Single variable and multiple-variable predictive  batteries are used to predict academic achievement.  Some of the single  variables studied as predictors of academic achievement include:  visual  acuity ( G r i f f i n & Eberly, 1971); hearing acuity (Goetz, 1971); chronological age (Dykstra, 1966); gender (Weintraub, 1966); and intelligence (Black, 1971). Numerous studies examine combinations of variables or screening batteries for predicting achievement or "at risk" status (Glazzard, 1979; Book, 1974; Satz & F r i e l , 1974; Adelman & Feshbach, 1971; de Hirsch, Jansky & Langford, 1966).  The major variables considered  to be among the  best predictors of first-grade performance when children were tested in kindergarten  include:  - performance on tasks of cognitive development (Kaufman & Kaufman, 1972); - f a c i l i t y at using a pencil (Eaves, Kendall, & Crichton, 1974); - pre-academic s k i l l s such as letter recognition (Colligan & O'Connell, 1974; Telegdy, 1975; Badian, 1986; Keogh & Becker, 1973; Stevenson, et a l . 1976; Mercer, Algozzine, & T r i f i l e t t i , 1988); - level of perceptual development (Morency & Wepman, 1973); - level of language development (Jansky & de Hirsch, 1972; Eaves, Kendall I Crichton, 1974; Stevenson et a l . , 1976). Some studies have investigated teacher observations  and behavioral  or academic ratings of children as a screening tool or as a component of a battery, but the findings are inconclusive. ratings of three teachers on 112 kindergarten  Kirk (1966) found the children favored  older  27  children as being bright and younger children as being slow.  She  concluded the ratings had reached an acceptable level i n identifying slow c h i l d r e n , but were only marginal at identifying bright c h i l d r e n .  Meyers,  Atwell and Orpet (1968) found that the behavior rating of attention i n kindergarten was as predictive of reading words, comprehension and s p e l l i n g in f i f t h grade as was a picture vocabulary t e s t . Feshbach, Adelman and Fuller (1974) concluded from a study of 888 kindergarten children that a kindergarten teacher's ratings could predict f i r s t grade reading achievement as e f f i c i e n t l y as a psychometric battery.  Wells and Peterson (1978) studied 111 kindergarten children i n  four classes.  They found the Kindergarten Teachers' Checklist (KTC)  was  a good predictor of scores in the Iowa Tests of Basic S k i l l s in f i r s t grade accounting for 30% of the variance.  Keogh's (1977) study found  that trained observers using a systematic observation of children's behavior were in strong agreement with kindergarten teachers' perceptions of risk or non-risk.  In a second related project involving 250  kindergarten children and 20 teachers, Keogh found teachers' ratings of children in kindergarten and f i r s t grade consistently favored g i r l s , although objective measures did not y i e l d significant gender d i f f e r e n c e s . In a review of research Simner (1983) found that many t r a d i t i o n a l warning signs of school f a i l u r e had l i t t l e actual bearing on later school achievement.  He identified f i v e signs more l i k e l y to be evident among  kindergarten children who are truly at risk for school f a i l u r e than among children who  are not at risk for f a i l u r e .  The five signs are:  in-class  attention span, d i s t r a c t i b i l i t y or memory span; in-class verbal fluency; in-class interest and participation; and letter or number i d e n t i f i c a t i o n  28  s k i l l s and printing errors.  He concluded that children at risk are  "...not necessarily lacking in many basic motor, language, drawing, and copying s k i l l s when compared to the average kindergarten c h i l d "  (p.24).  The results of t h i s , and other studies, suggest that, to the extent that the c r i t e r i o n measures are related to school performance or school success, predictors based on school-related tasks are s i g n i f i c a n t l y more r e l i a b l e than measures of other tasks (Meisels, 1984). Screening Instruments Measurement instruments, other than behavior scales, used for kindergarten screening may school-readiness t e s t s . identify children who may  be described as developmental tests and  Developmental screening tests are designed to have a learning problem or a handicapping  condition that could affect their overall potential for success i n school.  Such tests focus on performance in a wide range of areas  including speech, language, cognition, perception, affect and gross and fine motor s k i l l s .  Readiness tests focus on current s k i l l achievement  and performance rather than on a child's developmental p o t e n t i a l .  Thus,  readiness tests and developmental screening tests sample d i f f e r e n t , although potentially overlapping areas of measurable behavior (Meisels, 1984).  The primary areas included in early i d e n t i f i c a t i o n assessment are  language, i n t e l l i g e n c e , motor s k i l l s , social-emotional development and preacademic s k i l l s .  Individual screening tests and screening batteries  have been developed to assess these areas. (For reviews see:  Dykstra,  1967; Satz & Fletcher, 1979; Book, 1974; Paget & Bracken, 1983; 1987; Mercer, Algozzine & T r i f i l e t t i , 1988.)  Bracken,  29 Human Figure Drawing Test Children's drawings have been investigated as a measure of developmental status since the late 1800's.  Many researchers have  described the changes in children's drawings over the course of development. (For reviews see:  Goodenough, 1926; Harris, 1963.)  Goodenough (1926) developed the Draw-A-Man Test based on the assumption that the intelligence of children could be estimated from their drawings of the figure of a man.  Harris (1963) revised and refined the Goodenough  test and the Goodenough-Harris Drawing Test became the principal rating approach applied to children's drawings to estimate i n t e l l e c t u a l ( N a g l i e r i , 1988).  ability  The purpose of the test is to measure i n t e l l e c t u a l  maturity which was defined by Harris (1963) as the a b i l i t y to form concepts of an abstract character.  Evaluation of the child's drawing of  a human figure serves as a way of measuring the complexity of his or her concept formation a b i l i t y ( S a t t l e r , 1988). In a study i n which children were asked to draw a person, Ferinden, Jacobson and Linden (1970) found that a high-risk drawing correctly identifed 99% of the children who had d i f f i c u l t y with reading i n f i r s t grade.  Eaves, Kendall and Critchton (1974) found the draw-a-man subtest  of the Modified Predictive Index (de Hirsch et a l . , 1966) was strongly correlated with word analysis at the grade two l e v e l . The Draw-a-person test has been shown to correlate with intelligence tests such as the Wechsler (Dunn, 1967; P i k u l s k i , 1972; T r a m i l l , Edwards & T r a m i l l , 1980) and the Stanford Binet ( R i t t e r , Duffy, & Fischman, 1974). Correlations range from .24 to .88. Interrater r e l i a b i l i t i e s are satisfactory ranging from .80 to .90 for the point scale and .70 to .90  30 for the quality scale ( S a t t l e r , 1988; Naglieri & Maxwell, 1981).  Sattler  described the Draw-A-Man Test as an acceptable screening instrument f o r use as a nonverbal measure of cognitive  ability.  Dunleavy, Hansen, Szasz and Baade (1981) administered a human figure drawing test to 141 kindergartners.  In comparing a group of students who  passed the Metropolitan Readiness Test with students who f a i l e d the t e s t , the human figure test had i d e n t i f i e d 42% of the non-ready c h i l d r e n .  The  researchers concluded the test was useful for the early i d e n t i f i c a t i o n of the academically not-ready c h i l d . Goldman and Velasco (1980) investigated  the relationship between  human figure drawings and risk for experiencing emotional problems. Their results suggest that drawings which omit important body-parts are predictive that the child i s a high risk for developing emotional problems. Duffy, R i t t e r and Fedner (1976) conducted a study of 80 children who were administered a battery of tests including the Draw-A-Man test in kindergarten.  The test was a s t a t i s t i c a l l y significant predictor of  academic success as measured by the Total Stanford Achievement Battery. However, because the test accounted for only 9.3% of the variance, the researchers concluded the Draw-A-Man test had l i t t l e practical u t i l i t y as a predictor of school performance. In a recent study in which the scoring system was altered to u t i l i z e an empirically derived subset of items, Simner (1985) found the overall predictive v a l i d i t y of the Draw-A-Man test equalled or exceeded that achieved with many other school readiness t e s t s . scoring  He concluded that i f  was confined to certain key items, the drawings could i d e n t i f y  31 five-year old children who are at risk for school f a i l u r e . The Draw-a-person test has been found to be predictive of school performance. maturity.  The purpose of the test i s to measure i n t e l l e c t u a l  The areas of performance measured by the test include  nonverbal cognitive a b i l i t y and concept formation.  The test requires  fine motor a b i l i t y to draw with a pencil and attention to complete the task. Copying of Geometric Shapes Tests of copying of geometric shapes have long been used as developmental tests for children.  Developmental factors which may  be  involved in copying simple geometric designs include appropriate motor development, perceptual discrimination and the a b i l i t y to integrate perceptual and motor processes (Sattler, 1982). de Hirsch (1966) stated that pattern copying and human figure drawing tasks require a r e l a t i v e l y high degree of integrative competence.  Both tasks, according to de Hirsch, are like reading,  writing and s p e l l i n g , in that they require the a b i l i t y to organize parts into a meaningful whole. Two form copying tests have been widely researched as predictive instruments, the Bender-Gestalt Test (Bender, 1938) and the Developmental Test of Visual Motor Integration (Beery, 1989).  Research on these tests  indicates that the age at which figure-copying tests are administered  may  a f f e c t t h e i r predictive power. Duffy, Ritter and Fedner (1976) reported the Developmental Test of Visual-Motor Integration administered to 182 kindergartners was a s i g n i f i c a n t predictor of reading and mathematics subtests of the Stanford  32  Achievement Test in second grade.  They also indicated, however, that the  practical u t i l i t y of the test was limited. Ferinden, Jacobson and Linden (1970) reported the Bender-Gestalt administered in grade one was a better predictor of reading a b i l i t y i n grade one than when i t was administered i n kindergarten.  Duffy, Keogh  and Becker (1966) found negligible correlations between scores on the Bender and grade three reading a b i l i t y when the effect of intelligence was held constant.  In a longitudinal study conducted by Stevenson et  a l . , (1976), the Bender-Gestalt test proved to have l i t t l e predictive value for achievement measured in grade two. In a review of research on traditional warning signs of school f a i l u r e , Simner (1983) reported on fourteen studies involving copying tasks. The r e l a t i v e l y low correlations (.00 to .54) led Simner to advise caution when using the results of copying tasks for the purpose of individual prediction. Park (1978) suggested the power of drawing tests was i n identifying children at risk for learning problems.  Because these tasks require  attention, fine motor c o n t r o l , attention to detail and the a b i l i t y to follow instructions, poor performance might r e f l e c t disruption of the processes of focal attention which could lead to d i f f i c u l t y i n learning. Tests of copying geometric designs have been found to be predictive of school performance.  The predictive power may vary with the age at  which the test i s administered.  The s k i l l s required include visual  perception, motor production and visual motor integration.  The task also  requires attention and a b i l i t y to follow directions, s k i l l s which are necessary f o r success i n school.  33  Tests of Language Development Researchers have provided empirical support for the hypothesis that competence in oral language i s predictive of satisfactory performance i n academic s k i l l s ( I l g & Ames, 1964; Jansky & de Hirsch, 1972;  Zeitlin,  1976; Stevenson et a l . , 1976; Steinbauer & H e l l e r , 1978; Book 1980). General b e l i e f and acceptance of this relationship has influenced the design of curriculum and development of primary level materials (Gray, S a s k i , McEntire & Larsen, 1980). Although i t seems logical that various language processes and underlie subsequent academic achievement, a strong  skills  cause-and-effect  relationship has not been established. A review of research i l l u s t r a t e s that findings vary considerably depending on the particular oral language process being considered and the outcome measure analysed. Jansky and de Hirsch (1972) used a predictive battery developed from t h e i r previous research.  They reported that the Oral Language subtests  accounted for the greatest proportion of variance (14%).  The picture  naming subtest was reported to be highly predictive of reading status at the end of grade two  (r=.53).  Similar findings were reported by Satz, F r i e l and Rudegeair (1976) following a factor analysis of kindergarten a b i l i t i e s and t h e i r relationship to grade three achievement.  They found that the oral  language factor contributed most to reading while visual-motor a b i l i t y contributed most to s p e l l i n g . Stevenson et a l . (1976) found that attention span and verbal fluency were the best overall in-class indicators of future academic achievement.  Verbal fluency included the spontaneous use of precise  34  words and the capacity to convey abstractions when asked to describe events. Groff (1977) reviewed the research related to oral language and reading.  His analysis identified ten studies which concluded there was a  s i g n i f i c a n t degree of correlation between oral fluency and reading achievement.  At least eleven studies indicated a s i g n i f i c a n t  relationship between the complexity of oral language and reading achievement.  In contrast, six studies indicated no s i g n i f i c a n t  relationship between syntax complexity and reading achievement. (See Groff for details and references.) Simner (1983) reviewed research studies and reported that basic language s k i l l s such as defining common words, naming colors and body p a r t s , and identifying pictures of common objects, show only marginal relationship with subsequent school achievement. Basic assessment and child development texts often state that language and intelligence are closely related and i t i s impossible to indicate where one ends and the other begins (Salvia & Ysseldyke, 1985; Anastasia, 1976; Papalia & Olds, 1975).  Language cannot be measured  without measuring i n t e l l e c t u a l a b i l i t i e s to some degree. Gray, S a s k i , McEntire and Larsen (1980) i l l u s t r a t e d the close relationship between language and i n t e l l i g e n c e .  Their study of 74 f i v e - and six-year old  children indicated that a strong and s t a t i s t i c a l l y s i g n i f i c a n t correlation between oral language and readiness existed when age was controlled.  However, the language test did not discriminate readiness  groups when intelligence was entered as a covariate. The researchers concluded there was l i t t l e relationship between oral language and school  35  readiness.  They suggested that pervasive effects of IQ were a  s i g n i f i c a n t determinant of a child's performance on both measures of oral language and school readiness. Hammill and McNutt (1980) synthesized the results of 89 correlational studies of the relationship of various language constructs to measures of reading.  Their results indicated a low  (r=.39) between oral receptive  relationship  language and reading and p r a c t i c a l l y no  relationship between oral expressive language and reading. of l i t e r a t u r e focused on hypothetical  Their review  constructs of various language  processes and t h e i r relationship to reading, not on s p e c i f i c tests or subtests.  They suggest that individual subtests or tests vary greatly in  their predictive power. The and  strength of the relationship between oral language proficiency  school achievement is not c l e a r .  confound findings when not controlled. intelligence and difficult.  The effects of age and  intelligence  The close relationship between  language makes measurement of "pure" language  The demands of curriculum materials and expectations for oral  p a r t i c i p a t i o n within the classroom make a measure of the child's oral language desirable as a screening measure. Knowledge of Letters and Numbers Chall (1967) conducted a thorough review of the research on relationship between knowledge of letters and reading.  She  the  concluded  that a child's a b i l i t y to identify letters by name in kindergarten or beginning of grade one was in grade one references.)  and two  the  an important predictor of reading achievement  (r's from .3 to .9). (See Chall for d e t a i l s and  Wide support for the strength of this relationship led to  36  the inclusion of tasks of l e t t e r recognition in many readiness tests (Jansky & de Hirsch, 1972; Deverell, 1974; Adelman & Feshbach, 1971). In a study i n which Dykstra (1967) isolated the components of readiness t e s t s , he found that tests of letters and numbers were the best single predictor of reading success in grade one.  The addition of other  tasks on readiness tests added l i t t l e to the predictive value of the test. Numerous researchers also found tests of l e t t e r recognition to be important factors for predicting reading success ( D u r r e l l , 1958; Askov, Otto & Smith, 1972; Jansky & de Hirsch, 1972; K l e i n , 1977; B u t l e r , 1979; Badian, 1986).  Stevenson et a l . (1976) reported that the number and  l e t t e r s k i l l s which children knew before entering kindergarten were good predictors of their learning during their f i r s t three years of school. Busch (1980) conducted a study of 1000 grade-one students and  concluded  that the a b i l i t y to recognize upper- and lower-case letters and  beginning  sounds was the best single predictor of reading achievement. Lesiak's (1978) data provides guidelines for cut-off scores.  He  indicated the average kindergarten child could name 14 to 15 of the 26 upper-case l e t t e r s .  Children l i k e l y to experience later learning  problems could only name one to f i v e l e t t e r s . The primary expectation for schooling is l i t e r a c y .  Pupils are  expected to learn to read and write.  Letter recognition has been found  to be predictive of reading success.  The educational goals that children  learn to read and write makes a measure of letter recognition a l o g i c a l choice as a screening measure.  37 Summary of Screening Tests The  l i t e r a t u r e describing four types of screening measures has been  reviewed; draw-a-person t e s t s , copying geometric designs, expressive language and recognition of letters and numbers. The tests measure p a r t i c u l a r areas of development but there i s overlap in the s k i l l s they measure.  For example, the draw-a-person is administered as a measure of  cognitive a b i l i t y but i t requires visual-motor integration and fine-motor s k i l l to draw. The test of expressive language requires a level of receptive language for the child to perform.  A l l the tests require  receptive language, a b i l i t y to follow directions and attention. These four tests cover three areas of development generally assumed related to early school learning:  cognitive development; visual-motor  development; and language development (Lesiak, 1978).  The  of a screening program is intended to identify children who experience d i f f i c u l t y learning and who  implementation may  require intervention to a l l e v i a t e  or eliminate the source of the d i f f i c u l t y (Keogh & Becker, 1973).  These  particular screening tests have been found to be predictive of school achievement, and therefore they are consistent with the stated purpose of screening.  38  Part 2 Factors Which May Affect Prediction Research Many factors may have an effect on the outcomes investigated by prediction-performance research of young children.  Three areas which  deserve consideration are individual pupil c h a r a c t e r i s t i c s , the effects of interventions which occur during the time under study, and contextual effects of schools. Three characteristics of the child which may affect the relationship between prediction and performance are reviewed: entry to kindergarten; gender; and physical problems.  age at  Two educational  interventions, learning assistance and attending an extended primary program, are also discussed.  Contextual effects of schools i s discussed  briefly. Age at Entry Researchers and reviewers have addressed the question of within-grade age effects because chronological age i s the major c r i t e r i o n for admitting children to school.  When children who are the youngest i n  their grade are compared with their older classmates, they are usually less successful (Beattie, 1970; Davis, Trimble & Vincent, 1980; Ames, 1963; H a l l , 1963; M i l l e r & Norris, 1967; Gredler, 1978). Hedges (1977) reviewed i n detail the research l i t e r a t u r e related to screening and early i d e n t i f i c a t i o n . as i t relates to school success.  One area focussed on age at entrance  The conclusions he reached from the  l i t e r a t u r e on age at entry follow:  the older children are at entrance,  the greater their chances of academic success; when comparing the achievement of an older child of comparable i n t e l l i g e n c e , the younger child's scores remain i n f e r i o r ; younger children do not seem to have the  39  social maturity desirable for successful performance; and chronological age has s l i g h t l y more effect on boys in younger and normal age groups than on g i r l s of comparable intelligence (see Hedges for d e t a i l s and references).  Other studies noted that children who were youngest i n  their class had the highest incidence of f a i l u r e  (Walsh, 1988); were more  l i k e l y to repeat a grade (Lloyd, 1978); were more l i k e l y to be referred to special education  (Di Pasquale, Moule & Flewelling, 1980); and were  more l i k e l y to be labelled as learning disabled (Diamond, 1983). Two  important points regarding research on age at entry were made by  Shepard and Smith (1987).  The f i r s t point they make regarding  the  research on age at entry is that most researchers f a i l to control for the effects of intelligence or gender in analysing age e f f e c t s .  After  analysing the age trend by the a b i l i t y status in one study, Shepard and Smith (1987) suggested that the low achievement reported for some younger children was more accurately a combination of youngness and low than of age alone.  ability  The second major point was the consideration of  practical rather than s t a t i s t i c a l significance.  They found differences  in percentile between the oldest three month children and the youngest three month children was only nine points, a difference of l i t t l e practical s i g n i f i c a n c e . Gender Differences Research findings vary regarding the existence, cause and significance of gender differences in academic achievement.  In a  frequently cited review, Maccoby and Jacklin (1974) reported that differences existed between males and females i n measured verbal and quantitative a b i l i t y .  Females tended to score higher on verbal a b i l i t y  40  and males tended to score higher on quantitative a b i l i t y .  The  differences were apparent during elementary years and increased into adolescence and adulthood.  Numerous studies from many countries report  sex-related differences i n achievement (Walden & Walkerdin, 1985; Hanna & Kuendiger, 1986; Brandon, Newton & Hammond, 1987; Johnson, 1987; Shuard, 1986). Aiken (1972) reported sex-related differences as early as kindergarten, with males performing at higher levels than females. Fox and Cohn (1980) reported differences in performance between males and females i n early elementary school to be small, but increasing through grade seven. Researchers suggest a number of possible explanations f o r the observed differences in performance. Although i t i s generally acknowledged that gender differences in intelligence are non-existent ( S a t t l e r , 1988; Hyde, 1981;  Stockard, 1980), some researchers interpret  selected research findings to suggest there may be biological differences in i n t e l l e c t u a l aptitude (Fox & Cohn, 1980).  Other researchers seek to  explain the differences by social and psychological f a c t o r s .  Parental  views and behaviors towards children of different sex which are used as possible explanatory variables include:  providing gender-specific toys  and different opportunities for play (Fennema & Peterson, 1985); providing a c t i v e , vigorous play for boys which  encourages the  development of spatial and constructional s k i l l s for boys (Burton, Drake, Ekins, Graham, Topi in & Weiner, 1986); and holding lower educational expectations for g i r l s than for boys (Maccoby & J a c k l i n , 1974).  School  settings and culture may provide d i f f e r e n t i a l opportunities which lead to d i f f e r e n t i a l performance (Hieronymus, King, Bourdon, Gossling, Grywinski,  41  & Moss, 1976) as illustrated in the following ways: males are more likely to be assigned to high-ability groups for mathematics (Hallinan & Sorenson, 1987); males often receive more hours of formal instruction in primary years than do females (Eccles & Jacobs, 1986); teachers have lower expectations for girls and make less academic demands (Burton, et a l . , 1986); and teachers may promote confidence, f l e x i b i l i t y , risk-taking and rule breaking, behaviors found more often in males than females (Walden & Walkerdine, 1985). Student attitudes and motivations may explain differences in performance in various academic subjects (Good & Slavings, 1988; Burton et a l . , 1986; Pattison & Grieve, 1984). One important fact noted in a study conducted by Martin and Hoover (1987) was that there is greater variability in the skills of males than females across all subtests and all grades three to eight on the Iowa Tests of Basic S k i l l s . Sabers, Cushing and Sabers (1987) also noted that the size of the differences between the sexes was not very great when compared with the differences within the sexes. Males were found to be more variable than females in both reading and mathematics.  Willms and  Kerr (1987) found that social-class differences were far greater than gender differences.  They reported differences in mean levels of  performance between working class and middle class groups to be between 1.25 and 1.5 standard deviations, compared with sex differences of about .25 of a standard deviation. Health and Physical Problems Educators must deal with the effects of physical disease and impairment on a regular basis. Academic progress can be negatively affected by chronic illness (Stehbens, Kisker & Wilson, 1983), speech and  42  language d i f f i c u l t i e s and motor production d i f f i c u l t i e s (Gubbay, 1975), and the side effects of medications may influence attention and concentration (Rapoport & F l i n t , 1976).  The relationships between  various d i s a b i l i t i e s , allergies and chronic i l l n e s s and a variety of adverse academic, social and emotional problems have been demonstrated by Cowen, Weissberg and Gisare (1984), Larter (1982), Kornberg and Kaplan (1980), and Rawls, Rawls, and Harrison (1971). The implications for meeting the needs of students who are b l i n d , deaf or physically handicapped are apparent in school settings.  However,  mild to moderate physical conditions which are less obvious may also affect a student's academic performance.  The range of possible physical  handicapping conditions includes, but i s not limited t o , problems of v i s i o n or hearing, speech and language d i f f i c u l t i e s , motor production d i f f i c u l t i e s , a l l e r g i e s and physical i l l n e s s .  These d i f f i c u l t i e s may  appear insignificant when compared with the d i f f i c u l t i e s of children with obvious physical d i s a b i l i t i e s , but the impact on school performance may be great. Grimley and McKinlay (1977) stated that children with subtle d i f f i c u l t i e s of learning can be i n desperate need of help and i f t h e i r needs are not recognized, secondary emotional problems are bound to arise. The estimated prevalence of communication disorders i s f i v e percent of school age children (Frisch & Handler, 1974).  Research studies of  children with speech d i f f i c u l t i e s or motor performance  difficulties  frequently refer to subjects demonstrating d i f f i c u l t y i n both speech and motor areas (Yoss & Darley, 1974; Jenkins & Lohr, 1964; Gubbay, 1975; Gordon and McKinlay, 1980; Crary, 1984).  D i f f i c u l t i e s in speech and  43  motor production may negatively affect acquisition of s k i l l s in spoken and written language and the children may be perceived as lazy or unmotivated because they f a i l to complete daily assignments (Gubbay, 1975). One  in f i v e children has a major a l l e r g i c disease (Rapoport, 1976).  Reaction to food i s one of many variables which may combine and interact to give r i s e to learning and behavior problems (Hammond, 1980). Allergy related problems may complicate learning problems.  For example,  comprehension d e f i c i t s may be intensified by o t i t i s media resulting from a l l e r g y , and the side effects of allergy medication may increase attentional disorders and hyperactivity in some children (Mc Loughlin, H a l l , Isaacs, Petroski, Karibo & Lindsey, 1983). The incidence rate of medical problems which may influence performance i n school i s large enough to be given consideration. Gortmaker and Sappenfield (1984) estimated that 10 to 20 per cent of a l l children have a chronic medical disorder.  Perrin (1986) reported that  two percent of a l l children suffer from a severe chronic i l l n e s s that regularly interferes with daily a c t i v i t i e s including school attendance and performance. The effects of various medical problems are usually studied i n i s o l a t i o n .  Researchers investigating  prediction-performance  on educational outcome measures rarely control for the effects of physical problems on performance. Educational  Interventions  Educators continually question how best to help students experiencing academic d i f f i c u l t y .  Interventions are implemented in the  b e l i e f that they w i l l help improve the academic achievement of students.  44  Remedial assistance i s a widely accepted practice, but there i s l i t t l e controlled assessment of the effectiveness of remedial programs. Retention i s also a widely accepted practice.  Retention p o l i c i e s and  rates vary greatly between schools and across d i s t r i c t s (Holmes & Matthews, 1984; Jackson, 1975). Where educational interventions are provided during the time between the administration of a screening measure and an outcome measure, the intervention may affect the student's performance on the outcome measure. Two educational interventions which may affect student performance on outcome measures are remedial assistance and retention i n grade. Remedial assistance and i t s effect on achievement. When students have not progressed within the regular instructional program, educators may intervene with a remedial program for individual students.  Resources are allocated to provide instructional intervention  to increase student success (Deno, 1986).  Remedial programs are usually  intended to supplement the regular educational program.  Most o f t e n ,  students are taken out of their regular classrooms f o r remedial instruction i n specific academic areas, often reading or mathematics (Madden & S l a v i n , 1987; McNutt & Friend, 1985). The research l i t e r a t u r e i s ambiguous regarding the efficacy of educational interventions. The evidence i s not strong f o r p o s i t i v e , negative or neutral e f f e c t s .  Comparisons across studies are d i f f i c u l t  because intervention models vary in many ways such as, s e t t i n g , instructional strategies, types of pupils served and goals f o r instruction.  One d i f f i c u l t y in interpreting research findings i s that i t  45  i s not clear whether differences are related to materials studies, s e t t i n g , grouping or the effects of the remedial l a b e l . Few studies compare the progress of students participating part-time in resource room programming with students in regular class placement. Smith and Kennedy (1967) studied educable mentally handicapped students assigned randomly to either daily part-time resource room instruction or full-time regular class placement.  They found no s i g n i f i c a n t differences  in academic achievement between the groups.  In contrast, Glavin, Quay,  Annesley and Werry (1971) found behavior disordered students participating i n resource room programming gained s i g n i f i c a n t l y i n reading and mathematics achievement as compared with behavior disordered students in regular class placement. The important issue i s not the setting in which remedial instruction i s provided, but the effectiveness of the remediation on academic achievement.  Variables which have been identified as having strong  relationships to the acquisition of academic s k i l l s include:  time and  opportunity to learn (Gettinger, 1984); level of academic engaged time (Haynes & Jenkins, 1986); opportunities for a student to make correct responses (Greenwood, Dinwiddie, Terry, Wade, Stanley, Thibadeau & Delquadri, 1984); and implementation of a s p e c i f i c reinforcement contingency plan (Shapiro, 1987).  An individual or small group  intervention program may include some or a l l of these variables.  Shapiro  (1988) stated that interventions which incorporate these potent variables have been shown to be powerful and effective in remediating academic ski l i s . Numerous researchers have c r i t i c i z e d resource room programs f o r  46  reasons such as:  f a i l i n g to increase academic learning time (Haynes &  Jenkins, 1986); f a i l i n g to coordinate instruction with that of the classroom  (Johnstone, A l l i n g t o n , & Afflerbach, 1985); and f a i l i n g to  produce transfer to the regular program (Anderson-Inman, 1986). Gallagher (1984) considers resource rooms ineffective and A f f l e c k , Madge, Adams & Lowenbraun, (1988) found them to be more costly than other alternative  interventions.  Thurlow, Ysseldyke, Graden and Algozzine (1983) observed eight students receiving  instruction in the classroom and i n a resource room.  Although opportunities for differentiated instruction were available in the resource rooms, no practical differences were noted i n the amount of time the students were actively engaged in instruction in the  two  settings. Some researchers have reported positive findings for resource room programs.  Leinhardt (1980) reported the findings of a study of  low-achieving kindergarten students who were promoted to f i r s t grade, but were given a special remedial instructional program.  At the end of grade  one, the low-achievers performed at higher levels than promoted students given conventional instruction or students who were placed in a t r a n s i t i o n room with special instruction. Wolfenden (1980) conducted a longitudinal four years.  study of 108 students over  He reported that remedial intervention, started in  kindergarten, reduced the number of grade retentions and  individual  assistance programs that would have been required i f the intervention  had  not occurred. Other researchers report positive effects of intervention  programs  47  on reading (Boehnlein, 1987) and mathematics (Peterson, 1989).  Madden &  Slavin (1983) examined effective remedial programs and determined that the achievement of students identified as "at risk" can be s i g n i f i c a n t l y increased, by either extensive modifications in the regular program or by intensive remedial pull-out intervention. Retention and i t s effect on achievement. Retention is a common educational practice but research has f a i l e d to validate i t s effectiveness.  A c r i t i c a l review of research on  retention by Jackson (1975) concluded there was no r e l i a b l e evidence that grade retention resulted in higher achievement for pupils having d i f f i c u l t y learning than did grade promotion for similar p u p i l s .  Holmes  and Matthews (1984) analysed eight studies in which retained students were matched with promoted counterparts on the basis of achievement. They concluded that the research did not support that retention improves basic s k i l l s . In contrast, some researchers report positive achievement gains for retainees.  McAfee (1981) reported on three groups of students:  those  retained in grade one; those who were i n a compensatory education program during grade one; and those promoted to grade two.  McAfee's analysis of  the data revealed retention appeared to be beneficial i n early grades, one to f o u r , but had no effect in intermediate grades, five to seven. Sandoval and Hughes (1981) defined successful retention as one i n which the retained child completed the retained year ranking in the top t h i r d of the c l a s s .  They found that students who made academic and  social-emotional gains after repeating grade one lacked serious academic d e f i c i t s in the year prior to retention, had strong self-esteem and  48  social s k i l l s and showed signs of d i f f i c u l t y in school because of lack of exposure to the material. Peterson, DeGracie and Ayabee (1987) studied f i r s t - , second-, and third-grade retainees matched on several variables as same age students not retained.  Retained students improved their r e l a t i v e class standing  by the end of the retained year, but after three years there were no differences between retained and promoted students. Baenen (1988) conducted a five-year study of 243 students and a comparison group matched on several variables. She reported following:  the  retention did not meet i t s goals of helping students catch up  to grade level and stay there; there was no significant difference in growth trends in those retained in grade one versus a later grade; and those promoted showed better growth in both reading and math than those retained. The research on retention has been generally c r i t i z e d as being flawed and of poor quality (Jackson, 1975; Medway & Rose, 1986). Some problems with the research include:  more stringent retention p o l i c i e s  exist i n some schools than in other schools; control groups are sometimes age peers and sometimes grade peers. The major concern regarding on retention is the effect of selection bias. may  research  That i s , selection bias  favor promotion because, at the time of the decision to promote or  r e t a i n , the promoted students were performing better than retained students in ways not captured by the control variables. One concern regarding studies of interventions or retentions i s the threat to internal v a l i d i t y of selection bias which may findings.  affect the  However, the elimination of selection bias i n studies of the  49  effects of retention would be d i f f i c u l t to achieve.  Shepard and Smith  (1987) point out that random assignment of children who are candidates for retention into retained or not-retained groups i s unethical. The desire for random assignment also lacks f e a s i b i l i t y i n that parents, or teachers, rather than researchers, often control the decision of whether a c h i l d i s retained. Holmes and Matthews (1984) conducted a meta-analysis of 44 studies on retention in which they investigated the effects of selection b i a s . They calculated 575 effect sizes for variables within the studies. The mean effect size was -.37.  This indicated that on average, the retained  pupils scored .37 standard deviation units lower on various outcome measures than promoted p u p i l s .  Eighteen of the 44 studies had matched  subjects, that i s a retained group, and a promoted group matched on several variables.  A mean effect size was calculated for the matched  group studies to see i f i t differed from overall effect s i z e s .  The  effect size for the matched groups was -.38, similar to the e f f e c t size for a l l the studies of -.37.  The consistency between the two measures  supported their conclusion that differences i n designs of studies resulted in no significant amount of bias i n the r e s u l t s .  They concluded  that the cumulative research evidence shows the potential f o r negative effects consistently outweighs the positive outcomes.  Their findings  suggested that retention had a negative effect on pupil's personal adjustment, self-concept and attitude toward school.  They also found  that retained students performed 0.44 standard deviations below t h e i r promoted counterparts  on various measures of academic achievement.  50  Contextual  Effects  Contextual  effects i s the term used by researchers to describe the  effects of the collective properties of a school.  These c o l l e c t i v e  properties within a school have an effect on individual pupil achievement over and above the effects of the personal characteristics or attributes of the pupils (Willms, 1985).  Researchers have attributed contextual  effects to the teaching environment, the d i s c i p l i n a r y climate, curriculum patterns, course content and "peer group" influences (Willms, 1986; Summers & Wolfe, 1977; Winkler, 1975, C l i f f o r d & Heath, 1984). The l i t e r a t u r e describes two alternative points of view which attempt to explain school e f f e c t s .  The f i r s t position r e f l e c t s the  organizational view of school effectiveness. The effects of family influences and experiences  at school determine the learning outcomes.  The school experiences are shaped by the organizational structures and practices of the classroom, school and d i s t r i c t . (For reviews of the l i t e r a t u r e see Anderson, 1982; Murnane, 1981; Rutter, 1983). In general, the findings of these reviews are contradictory, and suggest weak organizational effects (Willms, 1987). The second viewpoint suggests that the most important determinants of school effects are institutional (Meyer, 1977, 1980). The i n s t i t u t i o n a l view suggests that schooling outcomes are determined by elements of the schooling system. r u l e s , roles and d e f i n i t i o n s .  These elements are defined by certain  These elements include educational  levels,  types of schools, curricular topics and the s p e c i f i c roles of instructors and students.  They derive their meaning from societal d e f i n i t i o n s rather  than organizational circumstance (Willms, 1987).  Meyer (1980) contends  51  that the structures and practices, the organizational aspects that affect student outcomes are r e l a t i v e l y homogeneous within a school, and t h e i r effects are small compared with effects of r u l e s , roles and d e f i n i t i o n s and the i n s t i t u t i o n a l e f f e c t s . Many researchers have abandoned the search for school effects explained soley by institutional or organizational elements and have turned to examining differences inside schools based on the research concluding that student achievement varies as much, or more, within schools as between schools (Bidwell & Kasarda, 1980). An explanation which allows for a relationship between organizational or i n s t i t u t i o n a l elements and within-school elements as explanation for school differences i s one which incorporates a hierarchical view of educational processes.  Two models which  conceptualize learning as a multi-level process have emerged in the l i t e r a t u r e , the additive model and the interactive model of schooling (Gamoran, in press). Barr and Dreeben (1977, 1983) describe an example of an hierarchical model.  They view schools as "nested layers" in which the outcomes of one  hierarchical level constitute the inputs at the next l e v e l .  They suggest  that d i s t r i c t and school administrators allocate resources to classrooms; key resources include time, curricular materials and the competencies of teachers and students. They emphasize the c o l l e c t i v e nature of schooling.  Students receive instruction in groups (such as classes or  within-class groups), so i t i s the characteristics of the group that must be most closely tied to the instruction that occurs in a given context. Thus, instruction i s a group-level outcome with consequences f o r the  52 individual-level process of learning.  This additive model (Barr &  Dreeben, 1983) views classroom instruction as the crucial force i n achievement.  Learning i s seen as a consequence of interaction between  individual characteristics and features of instructional opportunity. Sorensen and Hallinan  (1977) suggest that the opportunities for  learning apply to classes, not individual students in i s o l a t i o n . In their formulation, class-level variables  (opportunities  for learning)  affect the relation between individual level inputs ( a b i l i t y and e f f o r t ) and outputs (achievement).  Their interactive model of schooling views  learning as the result of student a b i l i t y and e f f o r t , but depends on the opportunity to learn (Sorensen & Hallinan, 1977).  The additive model and  the interactive model each have d i s t i n c t elements, however, they share a common view of education as a hierarchical model in which processes at one  level have an effect on outcomes at another level (Gamoran, i n  press). A number of class-level variables have been examined i n the research on contextual e f f e c t s .  Instructional time is one class-level  which has been found to contribute to achievement. in many different ways including:  variable  It has been measured  the length of the school year (Wiley &  Harnischfeger, 1974); daily time teachers devote to instruction (Gamoran & Dreeben, 1986); and time students spend engaged i n academic work (Denham & Lieberman, 1980). contribute to achievement. in the following ways:  Instructional practices have been found to Instructional practice has been demonstrated  the more words taught during f i r s t grade reading,  the more students learn (Barr & Dreeben, 1983; Dreeben & Gamoran, 1986); the more curriculum covered during the year, the higher the attainments  53  at the end of the year (Tizard, Blatchford, Burke, Farquhar & Plewis, 1988); the more content coverage in math, and the higher quality of instructional discourse i n English, the greater the achievement in the respective subjects (Gamoran, 1988). School mean SES or school mean a b i l i t y has been shown to have an effect on pupils' academic achievement, even after controlling for the individual effects of pupils' family background (Willms & Raudenbush, 1989; Willms, 1986; Summers & Wolfe, 1977; Henderson, Mieszkowski, & Sauvageau, 1978; Brookover, Sweitzer, Schneider, Beady, Flood & Wisenbaker, 1978; Rutter, Maughan, Mortimore, Ouston, & Smith, 1979). Attempts to measure d i r e c t l y some within-school processes have shown that a number of variables are associated with school mean SES (Brookover et a l . , 1978; Alexander, Fennessey, McDill & D'Amico, 1979).  Therefore, i n  the absence of a study that includes a wide range of variables describing administrative and teaching practices, c u r r i c u l a , and school climate, school-level aggregates of pupil-level characteristics such as school mean-ability, may act as proxies for variables describing certain school processes (Willms, 1986). Few studies have examined the impact of school composition on pupils with below-average a b i l i t y .  In a study of a sample of Scottish secondary  p u p i l s , Willms (1985) found that the average a b i l i t y level of a school was associated with higher exam performance at the secondary l e v e l , f o r pupils of d i f f e r i n g levels of a b i l i t y , even after controlling f o r individual pupil a b i l i t y and family background c h a r a c t e r i s t i c s .  Summers  and Wolfe (1977) found that elementary school p u p i l s , who tested at or below the average for their grade, scored higher i f they attended schools  54 with high achieving students, but students scoring above average for t h e i r grade were not particularly affected. Recent advances in s t a t i s t i c a l estimation have shown that single-level methods are not optimal for estimating multi-level models.  Problems of aggregation bias and mis-estimation of standard  errors have distorted single-level estimates of multi-level  processes  (see Bryk & Raudenbush, 1987).  available  S t a t i s t i c a l methods are now  that permit one to estimate data at more than one level so that each variable can be measured at i t s own  simultaneously,  level (Raudenbush &  Bryk, 1988; Aiken & Longford, 1986; Goldstein, 1986). In the examination of the relationship between kindergarten screening measures and achievement, the inclusion of contextual effects as a component of the analysis may provide explanatory power not available from a pupil-level analysis.  Hauser (1970) suggests that  contextual effects may only be a r t i f a c t s of an underspecified model. This study provides a good test for an elementary age sample because the model includes several measures of a b i l i t y in kindergarten and the outcomes are measured i n grade three.  Consideration of contextual  effects i s pertinent to this study because i f there are s i g n i f i c a n t contextual e f f e c t s , and i f they are stronger for low a b i l i t y p u p i l s , they would have the effect of lowering the kindergarten screen/achievement outcome relationship.  55 Part 3 Prediction Studies The implementation of early intervention should be based on a v a l i d and e f f i c i e n t detection program. The primary concern regarding screening of young children i s the potential for misdiagnosis.  The f a l s e l a b e l l i n g  of a child as being at risk may have negative effects on the c h i l d and family ( S a l v i a , Clark, & Ysseldyke, 1973; Foster, Schmidt, & Sabatino, 1976; Algozzine, Mercer, & Countermine, 1977) and result in wasted expenditures for unnecessary services (Gallagher & Bradley, 1972). A screening program may result i n two types of m i s c l a s s i f i c a t i o n s . One i s the i d e n t i f i c a t i o n of children "at risk" who are not actually "at r i s k " of school d i f f i c u l t i e s .  This can occur i f the screening measures  are not valid predictors of future academic success, or because children performed poorly on the screening measure due to extraneous f a c t o r s . Some children may have an accurate screening score indicating they are "at r i s k " , but they are only slow i n development. even after grade one, they might make rapid gains.  After kindergarten, or The second  m i s c l a s s i f i c a t i o n i s when children are not identified as being "at r i s k " when they actually could benefit from remedial services.  This kind of  error can also stem from invalid tests or from measurement error at the time of screening.  Children misclassified in this way require  intervention but do not receive i t .  Thus the r e l i a b i l i t y and v a l i d i t y of  screening measures are of primary concern i n prediction studies. R e l i a b i l i t y and V a l i d i t y Some screening instruments show satisfactory levels of r e l i a b i l i t y  56  and predictive v a l i d i t y but others do not (Lindsay & Wedell, 1982). Appendices A-E provide technical information regarding the s p e c i f i c screening measures included i n this study. related.  R e l i a b i l i t y and v a l i d i t y are  A test cannot be valid i f i t is not r e l i a b l e , but  reliability  i s not s u f f i c i e n t to make a test valid (Gronlund, 1975). R e l i a b i l i t y refers to the consistency and s t a b i l i t y of measurement, and r e f l e c t s the degree to which examiners can rely upon the score (Goodwin & D r i s c o l l , 1980).  A reliable test should y i e l d similar results  when administered two or more times during a short period to the same students.  In young children, development i s uneven and thus, measures of  t h e i r performance tend not to be as r e l i a b l e as those designed for older children and adults (NAEYC, 1988). There are several types of r e l i a b i l i t y and several ways of deriving estimates of r e l i a b i l i t y which are discussed in texts of educational and psychological measurement (e.g. Anastasi, 1976; Salvia & Ysseldyke,  1985;  Glass & Hopkins, 1984; Gronlund, 1985). Test r e l i a b i l i t y i s determined through s t a t i s t i c a l estimated using correlation methods.  procedures and i s  Essentially the various methods  determine how much error is present under different conditions.  In  general, the more consistent the test results are from one measurement to another, the less error there w i l l be and the greater the r e l i a b i l i t y . Different types of consistency are determined by different methods and thus, the r e l i a b i l i t y coefficient must be interpreted according to the type of consistency being investigated. The major methods of estimating r e l i a b i l i t y are t e s t - r e t e s t , which is an index of (Stability, equivalent forms, s p l i t - h a l f and the Kuder-Richardson method, which are indices of  57  the internal consistency of items on the t e s t .  Without evidence of  consistency i n the screening measures, the results may simply be products of chance. V a l i d i t y refers to the degree to which the instrument measures what i t i s purported to measure.  V a l i d i t y i s an indicator of the accuracy of  a test and of the inferences that may be drawn from i t ; the stronger the v a l i d i t y of a screening t e s t , the more credible i t s results (Meisels, 1984).  A screening measure i s valid to the extent that i t d i f f e r e n t i a t e s  between those students who are at risk for experiencing d i f f i c u l t y in school, and those who are not at r i s k . There are two ways of determining v a l i d i t y of screening measures: logical and empirical ( Z e i t l i n , 1976).  Logical v a l i d i t y refers to a  judgement about the adequacy and appropriateness of the content of a test.  The test instrument i s inspected to determine that the content and  format are consistent with the domain of s k i l l s , a b i l i t i e s or behaviors that the instrument purports to measure. determined through s t a t i s t i c a l procedures.  Empirical v a l i d i t y i s To determine i f the test  works as i t i s intended t o , the results are compared to a c r i t e r i o n measure that i s a meaningful  indicator of the target problem  (Lichstenstein & Ireton, 1984). V a l i d i t y may be concurrent or predictive.  Different ways to  determine a test's v a l i d i t y include comparing results with scores derived from other measures given at the same time (concurrent v a l i d i t y ) or at a later time (predictive v a l i d i t y ) .  V a l i d i t y , whether concurrent or  p r e d i c t i v e , i s measured by the strength of association and i s frequently expressed by Pearson correlation c o e f f i c i e n t s .  Correlations indicate the  58 strength of relationship between two instruments and r e f l e c t the accuracy with which one measure can be used to predict a second measure.  With  regard to kindergarten screening, v a l i d i t y is measured by the strength of association between findings identified in screening and the presence of d i f f i c u l t y in school performance as confirmed in subsequent assessment. The predictive v a l i d i t y of a screening process depends f i r s t on the r e l i a b i l i t y and construct v a l i d i t y of both the screening and outcome measures.  If either of the measures are unreliable, the predictive  v a l i d i t y of the screening process w i l l be jeopardized.  A l s o , i f either  the screening measure or the outcome measure does not adequately  reflect  the constructs they are meant to represent, the judgement about predictive v a l i d i t y w i l l be inaccurate. Methodological  Paradox of Prediction-Performance  Research  There are unique features of prediction research during early childhood and kindergarten which may paradox.  contribute to a methodological  The nature of the research is long-term but there can be  p o l i t i c a l and economic pressures to release findings of predictor measures before the outcome measures are collected. methodological  This may  lead to a  paradox which has implications for the predictive v a l i d i t y  of screening measures (Keogh & Becker, 1973; Z e i t l i n , 1976).  If early  i d e n t i f i c a t i o n and diagnosis is accurate and remedial interventions are successful, a child at risk of experiencing d i f f i c u l t y receives help which results in successful school performance.  Subsequently, the  child's score on a c r i t e r i o n measure is improved and the predictive v a l i d i t y of the screening instrument appears to be low.  Having  i d e n t i f i e d the child as at r i s k , the educator i s obligated to intervene  59  and the effects of the intervention limit the predictive v a l i d i t y of the instruments by raising the scores on the c r i t e r i o n measure so that the prediction that the child was at risk appears inaccurate.  When the  screening "at risk" prediction is accurate, f a i l u r e to provide remedial intervention would guarantee high predictive v a l i d i t y , however, the purpose of i d e n t i f i c a t i o n i s to provide intervention to children  who  require i t to be successful. Where a random sample i s selected, findings of predictor measures guarded and no intervention provided, there would be no paradox. Predict ion-Performance Research Several models for validating screening instruments have been discussed in the l i t e r a t u r e .  Detailed descriptions of the s t a t i s t i c a l  procedures and the ways various methods can be applied to research designs can be found in texts of s t a t i s t i c s and measurement (Pedhazur, 1982; Tabachnick, 1983; Glass & Hopkins, 1984).  The remainder of t h i s  chapter i s a review of techniques most frequently used in prediction-performance research. the following: ordinal:  interval:  Reference to measurement scales include  number represent rank order of observations;  numbers indicate rank order of observations; nominal:  represent categories.  numbers  For discussion of Scales of Measurement, see Glass  and Hopkins (1984) and G h i s e l l i , Campbell and Zedeck (1981).  Appendix  Table 1 presents a number of a studies which used multiple-instrument batteries as predictors and includes information regarding the subject sample, time-frame of study, analysis used and correlations obtained. Correlation Analysis The f i r s t approach to establishing predictive v a l i d i t y of a  60  screening measure or screening battery i s the v a l i d i t y c o e f f i c i e n t model.  In longitudinal  i s correlated  studies, interval data from a c r i t e r i o n measure  with screening test interval data.  The resulting  correlation coefficient i s used as an indicator of the effectiveness of the screening measure. When s t a t i s t i c a l l y significant correlations are obtained, an indication of the screening test's predictive v a l i d i t y i s inferred. The  results of a correlational analysis describe the degree of  overlap between two measures of the same phenomenom, but not the number of correct and incorrect decisions concerning children at r i s k (Wilson & Reichmuth, 1985).  The limitation of correlational analysis was  i l l u s t r a t e d by Lichtenstein instruments. two  (1981) in a comparison of two screening  The correlation coefficient between the total scores of the  tests was high (.82) which indicated a strong linear  but the c l a s s i f i c a t i o n analysis different children at r i s k .  relationship,  i l l u s t r a t e d the tests i d e n t i f i e d  He concluded that making assumptions about  the predictive v a l i d i t y of screening measures on the basis of correlational s t a t i s t i c s i s tenuous and i l l - a d v i s e d (p.68).  Although  many researchers report correlational s t a t i s t i c s as a component of t h e i r analysis, several studies have reported correlational c o e f f i c i e n t s as their primary means of analysis i n prediction research studies. (See: Ferinden, Jacobson & Linden, 1970; Book, 1974; Buttram, Covert & Hayes, 1976;  Duffy, Ritter & Fedner, 1976; Rubin, Balow, Dorle & Rosen, 1978;  Goldman & Velasco, 1980; Lindquist, 1982; Simner, 1985.) T-tests and ANOVA Another approach to establish the predictive v a l i d i t y of a screening  61  test i s to use t-tests (Barnes, 1982; Borg & G a l l , 1983; M i l l e r , 1988). Scores obtained on a screening test are used to c l a s s i f y children as "at r i s k " or "not at r i s k " , that i s , interval data i s collapsed into dichotomous data.  At a later date, the children are tested on a  c r i t e r i o n measure to ascertain the v a l i d i t y of the prediction. c r i t e r i o n data i s interval data.  The  T-tests show whether there are  s t a t i s t i c a l l y significant differences between the mean c r i t e r i o n score of the children identified as "at risk" and the mean score of those i d e n t i f i e d as "not at r i s k " .  Analysis using the means of each group may  i l l u s t r a t e that the mean performances are d i f f e r e n t , but gives no consideration to the distributions of the groups or of any overlap in the distributions.  This approach also f a i l s to indicate the number of  correctly or incorrectly identified students.  Many prediction-  performance studies report t-tests as one component of the analysis which also may include correlations and regression analysis.  (See: Eaves,  Kendall & Crichton, 1972; Hartlage & Lucas, 1973; Stevenson, et a l . , 1976; Wells & Peterson, 1978; Butler, 1979; M i l l e r , 1988.) In some prediction-performance research the interval data from the c r i t e r i o n measure i s c l a s s i f i e d into three or more groups such as high, middle or low scoring. Analysis of Variance (ANOVA) i s an i n f e r e n t i a l technique which can be used to determine whether the differences among three or more sample means are greater than would be expected from sampling error alone.  If multiple t-tests are used for three or more  means, the probability of error increases as the number of groups increases.  ANOVA i s appropriate because the chance of error i s reduced.  ANOVA also may be used to compare subgroups that vary on more than one  62  f a c t o r . For example, the differences between males and females within the high, middle and low groups can be investigated using ANOVA. The ANOVA analysis results in an omnibus F value which indicates i f means d i f f e r significantly.  If the F-ratio i s s i g n i f i c a n t , special t-tests are used  to specify which particular means d i f f e r s i g n i f i c a n t l y . special t-tests for multiple comparisons including:  There are  Duncan's,  Newman-Keuls, and Tukey and Scheffe. One-way ANOVA may be used when the subgroups d i f f e r on one f a c t o r , two-way ANOVA when the subgroups d i f f e r on two factors.  More complex  variations of ANOVA and Analysis of Co-variance (ANCOVA) are discussed i n most texts of s t a t i s t i c a l analysis, but are not used frequently i n prediction-performance research. (See Hartlage & Lucas, 1973; Badian & Serwer, 1975; Stevenson et a l . , 1976; Dunleavy, Hansen, Szasz & Baade, 1981.) Generally the T-test and ANOVA approaches have less power than a correlational approach; essentially the interval or ordinal data obtained by screening i s collapsed to a dichotomous, nominal measure (at r i s k vs not at r i s k ) .  If screening could only provide a dichotomous score, the  t-test and correlational approaches would be i d e n t i c a l . Discriminant Analysis Another approach to establishing v a l i d i t y of a screening test i s Discriminant Analysis.  It i s essentially an adaptation of the regression  analysis technique, designed s p e c i f i c a l l y for situations in which the c r i t e r i o n variable i s categorical rather than quantitative.  Discriminant  Analysis involves two or more predictor variables and a single c r i t e r i o n variable which r e f l e c t s an individual's group membership ( i . e . , "at  63 r i s k " , "not at r i s k " ) .  The analysis equation uses the individual's score  on the predictor variables in an attempt to predict the group of which the individual i s a member. (See Satz & F r i e l , 1974; La Torre, Hawkhead, Kawahua & Bilow, 1982; Fletcher & Satz, 1982.) Discriminant Analysis i s a useful technique when the c r i t e r i o n variable i s i n the form of categories reflecting discrete groups. The c r i t e r i o n i n prediction-performance research i s often based on a continuous variable (e.g. achievement) with a selected cut-off score which, in e f f e c t , creates a dichotomous variable.  This technique i s  valuable when the c r i t e r i o n variable i s a category such as "drop-out". However, when the c r i t e r i o n i s continuous such as academic achievement, one loses power by creating a dichotomous variable based on some arbitrary cut-off score. Prediction-Performance Matrices The major model for evaluating the u t i l i t y of educational screening instruments i s a prediction-performance matrix (Meehl & Rosen, 1955). The matrix i s composed of two levels of performance on the screening measure, that i s , interval data i s dichotomized to be "at r i s k " and "not at r i s k " .  The performance on the c r i t e r i o n measure i s dichotomized  two l e v e l s .  into  The levels are usually designated as poor or good  performance. Figure 2 presents an example of a prediction-performance matrix and l i s t s several formulas created to evaluate the effectiveness of a screening t e s t .  Effectiveness i s measured as attaining higher rates of  accurate i d e n t i f i c a t i o n and prediction than would be possible without the test (Satz & Fletcher, 1979).  64 Figure 2 Prediction-Performance Comparison Matrix Criterion Measure Performance Poor  Good K  Poor Screening Measure Performance Good  True Positives A  False Positives B  False Negatives C  True Negatives D  NOTE: Sensitivity=(A/A+C); specificity=(D/B+D); overreferral=(B/A+B); underreferral=(C/C+D); predictive u t i l i t y of screening positive=(A/A+B); predictive u t i l i t y of screening negative=(D/C+D).  Quadrant A shows the true positives.  These are students who were  predicted to be "at risk" by the screening instrument and who performed poorly on the c r i t e r i o n measure. Quadrant D shows the true negatives, students predicted to perform well who performed well on the c r i t e r i o n measure.  Quadrant B shows the false positives, students i d e n t i f i e d as  "at r i s k " by screening measure but "not at risk" on the c r i t e r i o n measures.  These students are seen to be misclassified "at r i s k " by the  screening measure because they obtain successful scores on the c r i t e r i o n measure.  Quadrant C shows the false negatives, students not i d e n t i f i e d  as "at risk" by screening but identified by later poor performance on the c r i t e r i o n measure. These students are seen to be misclassified "not at r i s k " by the kindergarten screening measure because they experience d i f f i c u l t y in learning as indicated by low scores on the c r i t e r i o n measures.  The two-by-two matrix displays two groups correctly i d e n t i f i e d  65 by the screening measure, the true positives and true negatives (Quadrant A, true "at risk", Quadrant 0, true "not at risk") and two groups incorrectly identified, the false positives and false negatives (Quadrant B, students identified "at risk" but who perform successfully on the criterion and Quadrant C, students identified "not at risk" but having difficulty). The outcomes of prediction-performance comparisons can be evaluated by different approaches. Classificational analysis, also called cross-tabulations, evaluates the accuracy of the screening instrument in terms of the correspondence between the screening outcome and the status of the child on the criterion measure. Mercer, Algozzine and T r i f i l e t t i (1988) illustrate the outcomes of vertical and horizontal analysis in a review of single-instrument and multiple-instrument prediction studies. Classificational analysis allows for the comparison of false inclusions and exclusions (B & C) to true positives and true negatives (A & D).  Figure 3 illustrates a numerical example of a prediction-  performance matrix.  By applying a horizontal analysis method,  percentages of correct and incorrect outcomes can be obtained. For example, A/A+B and D/C+D give the percent of correct outcomes, C/C+D and B/A+B give the percent of incorrect outcomes. A vertical analysis allows for the consideration of the relationship between prediction (i.e., within the cells) and actual performance. For example, A/A+C and D/B+D give the percent of students who performed as predicted, conversely, C/A+C and B/B+D give the percent of students for whom the prediction was inaccurate. The percent of correctly identifed students, also called the overall hit rate, is computed A+D/A+B+C+D.  66  The proportion of students with special needs who are i d e n t i f i e d accurately as "at risk" by the screening instrument i s reported as s e n s i t i v i t y and may be computed as A/A+C.  S p e c i f i c i t y indicates the  proportion of children not in need of special services whose scores on the screening measure were above the cut-off score and may be computed as D/D+B. Together, s e n s i t i v i t y and s p e c i f i c i t y permit comparisons to be made between the base rate, or prevalence, of a physical problem and c l a s s i f i c a t i o n a l decisions derived from a screening test (Harber, 1981).  Figure 3 Numerical  Example of Prediction-Performance Matrix Criterion Measure Performance Reading  Poor  Draw-A-Person Good  Poor  Good  30 19% (32%)  135 81% (14%)  65 8% (68%)  743 92% (86%)  165  -808 r-  95  878  973  Overall h i t rate=79%; Sensitivity=32%; Specificity=86% 0verreferraT=81%; Underreferral=8% Predictive u t i l i t y of screening positive=19% Predictive u t i l i t y of screening negative=92%  These types of analyses allow for the observation of numbers of correctly and incorrectly identified children u t i l i z i n g the performance score on the prediction and c r i t e r i o n measures.  They are called the  67 predictive u t i l i t y of screening positive or negative.  Policy makers l i k e  t h i s analysis because i t identifies proportions of the students requiring special services. However, a l l of these proportions depend on an arbitrary cut-off score for both the screening measure and the c r i t e r i o n measure.  By raising or lowering the cut-off point on either the  screening measure or the c r i t e r i o n measure, i t would be possible to reduce the number of false positives. false positives in this way increased.  The consequence of reducing the  i s that the number of false negatives i s  The ideal screening instrument would refer a l l children in  need of special services, but minimize the number of false r e f e r r a l s (Lichtenstein, 1981).  (For review and comparisons of horizontal and  v e r t i c a l analysis see Mercer, Algozzine & T r i f i l e t t i , 1988.) If one c o l l e c t s data on the screening measure and the outcome measure at the interval l e v e l , one can judge the v a l i d i t y of the screening process using a correlational method, which i s more powerful than c l a s s i f i c a t i o n a l analysis. The data could also be used to report s t a t i s t i c s derived from a number of prediction-performance  matrices on  various cut-off scores for both the c r i t e r i o n and the outcomes. An important consideration regarding c l a s s i f i c a t i o n a l analysis i s that the analysis f a i l s to consider effects of interventions which may have occurred during the time under study.  C l a s s i f i c a t i o n a l analysis  does not consider which students identified "at risk" perform well on the c r i t e r i o n due to the effects of the intervention. If interventions are e f f e c t i v e , f a i l u r e to consider the intervention effects may  i n f l a t e the  false positives because the prediction "at risk" was accurate and the intervention accomplished the desired goal of the students experiencing success.  68 Multiple Regression Multiple regression i s a s t a t i s t i c a l technique f o r determining the correlation between a c r i t e r i o n variable and some combination of two or more predictor variables.  The multiple correlation c o e f f i c i e n t (R) i s a  measure of the relationship between a c r i t e r i o n variable and a predictor variable or combination of predictor variables.  2  R  i s the c o e f f i c i e n t of  determination and expresses the amount of variance in the c r i t e r i o n variable that i s accounted for by a l l the predictor variables combined. There are several variations of multiple regression analysis: backward and stepwise.  forward,  Each variation uses a different procedure f o r  selecting predictor variables to obtain the best prediction of the c r i t e r i o n variable. In multiple regression, the beta coefficients are sometimes referred to as partial regression c o e f f i c i e n t s . They express the correlation between two variables under the condition that a l l other concommitantly measured variables are held constant. The raw score form of the regression equation i s useful f o r predicting the effects on the c r i t e r i o n variable of a unit increase i n each predictor variable.  The standardized form i s needed to interpret  the r e l a t i v e importance of various predictor variables. In kindergarten prediction, multiple regression may be used to identify the amount of variance i n the c r i t e r i o n variable accounted f o r by the prediction variables taken as a group.  A variation in the  analysis, step-wise regression, may be used to indicate the rank-ordering of the predictor variables in terms of their efficacy i n accounting f o r variance in the c r i t e r i o n variable.  This analysis allows the researcher  69  to identify the screening measures or subtests which account for the greatest variance and may be helpful in selecting batteries of tests for screening programs. (See:  Randel, Fry & R a l l s , 1977; Rourke & Orr,  1977;  Glazzard, 1982; Schmidt & Perino, 1985; Badian, 1986; Jacob, Snider & Wilson, 1988.) Multilevel Modelling Multilevel modelling i s an extension of simple linear regression. The interpretation of multilevel modelling is similar to the interpretation of ordinary regression with the extension of being able to say whether relationships vary between schools.  Multilevel models have  been developed which can simultaneously estimate the effects of variables at three or more l e v e l s , such as the p u p i l , school and d i s t r i c t levels (Aikin & Longford, 1986; Goldstein, 1986; Raudenbush & Bryk, 1986). The major reason for using multilevel techniques i n educational research is that multileveling techniques allow the researcher to take schools into account in the analysis.  Schools introduce an extra random  component - an extra degree of uncertainty. In using the multilevel technique the standard error of estimates i s a more precise  estimate  because i t includes the extra random component introduced by the school variable.  The (two-level) multilevel technique disaggregates the  relationship between variables into two components, within-school and between-school.  The within-school component describes the relationship  inside the school, thus, i t compares the Kindergarten screen/achievement relationship of individuals who attend the same school.  The  between-school component takes account of differences between schools. The multilevel model estimates an overall pupil-level relationship  70 between outcomes and background f a c t o r s , taking account of the nested structure of the data.  It provides estimates of the relationships within  each school that are d i f f e r e n t i a l l y "shrunk" towards the overall pupil-level relationship (Raudenbush & Bryk, 1986).  The multilevel  estimates are biased, but consistent, and less variable than OLS regressions which are unbiased, but may have large standard errors when the sample sizes within schools are f a i r l y small. The result of f i t t i n g the multilevel model i s an estimated regression line for each school, analogous to the single line for a whole sample.  Multilevel regression provides a test of whether observed  differences i n intercepts and slopes could have occurred by chance, or whether there r e a l l y are differences in the population of schools. It also provides a test of whether a general tendency in the slopes, i s l i k e l y to be present in the population, or whether i t i s a random a r t i f a c t of the sampling.  Third, i t provides a test of whether  differences i n the slopes between schools could have arisen by chance. In kindergarten prediction research multilevel modelling may be used to identify the relationships between kindergarten screening and achievement outcomes at the pupil-level and determine i f the relationships vary s i g n i f i c a n t l y among schools.  Where findings indicate  that schools d i f f e r s i g n i f i c a n t l y i n the kindergarten screen/achievement relationships or the achievement levels of pupil's "at r i s k " , one can attempt to explain differences between schools in terms of school characteristics.  In the present study i t i s possible to observe the  relationships between kindergarten screening measures and grade three achievement for the entire sample arid to observe between-school variation  71  in the relationships.  After controlling for the effects of student  characteristics and educational interventions, i t can be determined i f these variables have an effect of mediating the relationships between screening measures and achievement. The parameters ( i . e . , intercepts and slopes) that specify  the  relationships between kindergarten screening and achievement within each school can become the dependent variables in a school-level  regression  that attempts to determine the importance of certain school-level variables ( i . e . , school s i z e , school mean a b i l i t y ) in explaining between-school variation in the kindergarten screen/achievement relationship or achievement of pupils designated "at r i s k " . Hierarchical linear modelling allows one to examine what relationships exist within the entire pupil-level sample and between schools represented in the sample.  The simultaneous estimation of the  parameters at both levels results in more accurate estimates than can  be  obtained by single-level analysis techniques. Prediction Studies Summary Three important factors which influence the strength of prediction using kindergarten screening measures are:  r e l i a b i l i t y of both the  screening and c r i t e r i o n measures; v a l i d i t y of the screening and  criterion  measures; and the analyses employed. The predictive power is weakened i f any of the measures have poor r e l i a b i l i t y or v a l i d i t y . The technique of analysis has an effect on the power of p r e d i c t i o n . Most prediction and c r i t e r i o n measures result in interval data. Frequently the data i s collapsed dichotomous variable.  into two categories  and analysed as a  These analyses have less power than analyses which  72 u t i l i z e the f u l l range of the data.  Several different analyses were  described which use combinations of dichotomous and interval data. The most frequently used prediction-performance analysis i s c l a s s i f i c a t i o n a l analysis which u t i l i z e s two dichotomous variables.  The  advantage of c l a s s i f i c a t i o n a l analysis i s that the percentages of correct and  incorrect predictions are i d e n t i f i e d .  This information i s of  interest to policy-makers because i t has direct implications f o r services provided and expenditures. The application of a multilevel analysis has several advantages, over other analysis:  interval data i s u t i l i z e d for both the predictor and  c r i t e r i o n measures; student background characteristics can be controlled; the effects of interventions can be controlled to determine i f they mediate the kindergarten screen/achievement relationships; and the relationships can be examined at the pupil-level and the between-school level.  If schools d i f f e r i n the kindergarten screen/achievement  relationships or achievement levels of "at risk" p u p i l s , the differences can be investigated by including school characteristics i n the analysis. The application of a multilevel analysis allows f o r investigation of kindergarten prediction consistent with the hierarchical structure of education.  Children are nested within schools.  Schools may respond  d i f f e r e n t i a l l y to screening data and may allocate resources d i f f e r e n t l y . An investigation of the predictive v a l i d i t y of kindergarten screening measures i s strengthened by examining the relationships at both the pupil-level and the between-school l e v e l s .  73 Summary The review of literature in Chapter 2 has addressed three main areas:  kindergarten screening, variables which may affect prediction;  and the most common methodological  approaches used in prediction-  performance research. V i r t u a l l y a l l screening programs are implemented with the stated purpose to identify children who may experience d i f f i c u l t y learning and who  require educational interventions to assist them to learn.  Educators  and researchers are not of unanimous opinion regarding the value of screening.  The primary concern involves the possible negative  consequences of "labelling" the c h i l d .  Lack of screening programs may  result in f a i l u r e to recognize a child at risk of experiencing d i f f i c u l t y learning in school.  Failure to provide appropriate intervention may  be  costly to the child who does not experience success in school, to the school system in terms of retentions, and to society in terms of numbers of school dropouts and subsequent costs for the outcomes brought about by lack of education.  Despite the lack of consensus on the value of  screening, wide-spread l e g i s l a t i o n has required the development of screening programs in many school d i s t r i c t s in North America. The particular d i f f i c u l t i e s in prediction-performance include:  research  d i f f e r e n t i a l developmental patterns of young children making  r e l i a b l e measurement uncertain; the goal of screening i s to hypothesize the existence of a problem before the symptoms of the problem are present; there i s a p r o l i f e r a t i o n of screening instruments, some of which are not technically adequate; and remedial interventions are implemented to correct or alleviate the predicted problem before the measurement  74 occurs which is meant to confirm the existence of the predicted problem. These d i f f i c u l t i e s are confounded in the research by the fact there i s no singular definition of the term "at r i s k " .  "At risk" i s used to  describe pupils performance at screening level and performance at the time of subsequent outcome measure; i t is used to identify pupils who obtain a score below a particular cut-off point and i t is used generally to describe any pupils who  have d i f f i c u l t y for any of a number of s o c i a l ,  emotional or physical reasons. The variables included in a study affect the v a l i d i t y of the study as do the variables which are not included.  Many studies have f a i l e d to  include important pupil characteristics such as age at entry and gender. Failure to include these variables may  result in masking findings of a  developmental nature related to age, gender or a combination of both and also weaken the power of the analyses. Numerous methodological  approaches have been u t i l i z e d for  prediction-performance research.  Appendix Table 1 provides an overview  of a selection of prediction-performance research studies.  Eighteen  studies cover only a short-time frame, end of kindergarten or f i r s t grade; five used sample sizes of less than one hundred subjects; seventeen studies f a i l e d to consider gender, most of the studies f a i l e d to consider chronological age at entry which may important developmental levels and may i d e n t i f i e r of "risk" status.  be representative of  interact with gender as an  A l l the prediction-performance studies  reviewed are flawed in at least one major way.  Also, they f a i l to  consider the hierarchical nature of educational e f f e c t s . within-classes,  Children learn  classes function within schools, schools allocate  75 resources which affect learning including opportunities for educational interventions.  If educational interventions are allocated d i f f e r e n t i a l l y  or i f class s i z e , peer group, or instructional practices vary among schools, i t i s necessary to view the predictive v a l i d i t y of screening measures from a multilevel perspective and look to both within-school  and  between-school variables to explain the relationships between kindergarten screening measures and outcome measures. This study i s an attempt to f i l l  a gap in the  prediction-performance  research literature by applying a s t a t i s t i c a l l y appropriate hierarchical model and by controlling for pupil's background characteristics and f o r the effects of interventions which occur during the time under study.  76 Chapter 3 Research Methodology Introduction Kindergarten screening refers to the administration of screening measures, or t e s t s , to kindergarten pupils. to discover those pupils who learning academic s k i l l s .  The purpose of screening i s  are at risk of experiencing d i f f i c u l t y i n  The underlying assumption for i d e n t i f i c a t i o n  is that the early provision of educational interventions w i l l eliminate Chapter 2 is a review  or a l l e v i a t e the predicted learning d i f f i c u l t i e s .  of the advantages and disadvantages of screening practices.  The primary  concern regarding screening practices is the possible negative effects which may  result from labelling a child "at r i s k " .  The important variables which might have significant effects on the predicted outcomes merit consideration. The purpose and underlying assumptions for screening include the intention to provide educational intervention to pupils identified as "at r i s k " .  Interventions are made  available because educators believe they have a positive effect on academic outcomes. Therefore, a study of the predictive v a l i d i t y of screening measures should include controls for effects of the interventions.  Failure to control for the effects of interventions may  result in low predictive v a l i d i t y , because i f the intervention a l l e v i a t e d or eliminated the factors which were predicted to lead to poor performance, and the pupil performed well on the outcome measures, the i n i t i a l prediction of "at risk" would appear inaccurate. This study examines the relationship between four kindergarten screening measures and grade three achievement in reading, mathematics,  77 vocabulary and language.  I expect that the relationships between  individual screening measures and different outcome scores may screening measures may  vary; some  be better predictors of particular outcome scores  than other screening measures.  I also expect that the relationships  vary across schools as the allocation of resources may  may  be different for  individual schools. I expect pupil characteristics of age, gender and physical problems have an effect on the relationship between screening measures and grade three achievement. and gender may  The normal developmental differences related to age  result in pupils of similar age and gender attaining  scores on screening measures which are different from older or younger pupils. three.  Maturation may mediate these i n i t i a l differences by grade When age and gender are controlled, the kindergarten  screen/achievement relationships may physical problem may  be weaker.  The effects of a  depress the screening or outcome scores.  I expect the kindergarten screen/achievement relationships w i l l increase after controlling for the effects of educational interventions. If schools respond to the screening information and provide e f f e c t i v e instructional intervention to students identified as "at r i s k " , the outcome scores of those at risk would be higher than without intervention.  The effect would be to lower the relationship between the  screening scores and outcome scores. Therefore, by s t a t i s t i c a l l y controlling for this effect through the inclusion of interventions in the model, one would expect the kindergarten screen/achievement relationship to increase. This study extends the traditional predictive v a l i d i t y approach by  78  investigating the kindergarten screen/achievement relationships at two l e v e l s , the pupil-level and the school-level.  This allows f o r the  investigation of between-school differences which may exist i n the kindergarten screen/achievement relationships and i n the average levels of achievement of pupils who obtained scores at risk on screening measures.  The contextual effects of schools may contribute to school  differences.  I expect that school-level variables w i l l explain some of  the between-school differences in the kindergarten screen/achievement relationships and i n the average levels of achievement of pupils i d e n t i f i e d as "at r i s k " . This chapter describes the research methodology f o r the study.  The  chapter begins with a description of the subject population and the procedures for data c o l l e c t i o n , followed by a brief description of the four kindergarten measures and the outcome measures. The research questions and hypotheses for each question are presented. i s discussed in three sections:  The analysis  the variables used i n the study; a  description of the application of hierarchical linear modelling; and the preliminary analyses. threats to v a l i d i t y .  The last section of the chapter discusses the The chapter concludes with a brief summary.  Subjects The subjects for this investigation include a l l pupils enrolled in one Canadian school d i s t r i c t who were born in 1975 or 1976, and who were enrolled in the school d i s t r i c t in 1987-88. represent two age cohorts, not grade cohorts. enrolled approximately  Thus, the subjects The school d i s t r i c t  15,000 students with about 1000 students at each  grade l e v e l . The school d i s t r i c t included over 30 elementary schools  79  serving the municipality.  The smallest schools were two-room, rural  schools with enrollments of 40-60 students.  The largest schools were  within the c i t y core, with enrollments ranging from 300-500. The municipality includes two c i t i e s in which there is multi-family housing f o r low-income f a m i l i e s .  Large suburban areas accommodate  working class and middle-class f a m i l i e s .  Areas exist where upper-middle  and high-income professionals l i v e and many large r u r a l , agricultural farms are part of the community. The population i s of mixed SES  and  several r a c i a l and ethnic groups are represented in the community.  The  municipality i s a growing area with a l i g h t - i n d u s t r i a l and agricultural base.  It i s located within easy driving distance of a major c i t y .  The 1975 and 1976 cohorts were comprised of 1030 and 1035 respectively.  students  Subjects were selected who had been administered four  screening measures in kindergarten and the Canadian Test of Basic S k i l l s (CTBS) in grade three.  One hundred and twenty students who were  administered kindergarten measures were enrolled in special class placements and were not administered grade three CTBS i n their grade three year.  These students were excluded from the study.  A t t r i t i o n of  students resulted largely from movement out of the d i s t r i c t .  The  achieved sample included 957 students, 497 in the 1975 cohort and 460 i n the 1976 cohort.  A discussion of the analysis of a t t r i t i o n bias i s  presented in the Threats to Validity section of this chapter. Procedures The d i s t r i c t granted permission to review the pupil records f o r a l l children enrolled i n 1987-88 who were in grades four through eight, or in special class placement.  Each pupil record card (PR Card) was  reviewed  80 to identify a l l children who were born in 1975 or 1976. The majority of the pupils comprising the 1975 and 1976 cohorts were in grades five and s i x , respectively, at the time of data c o l l e c t i o n .  Therefore, I was able  to obtain records of grade three achievement f o r most p u p i l s , even those who had repeated one or two grades.  Each student was assigned an  i d e n t i f i c a t i o n number and pertinent information was recorded from the PR Card:  birthdate, retention i n grade, extended primary, learning  assistance and known medical conditions. The pupil cumulative record of each student was then reviewed f o r test scores, medical information and educational interventions.  A l l information was recorded on fortran data  record sheets. The following test scores were obtained, from the original protocols: Kindergarten Screening: Draw-a-person; Mann Suiter Visual Motor Test; Kindergarten Language Screening Test; Deverell Test of Letters and Numbers. The following test scores were obtained from computer generated reports of test scores administered in grade 3: Canadian Tests of Basic S k i l l s : Test scores (each of four years) - Vocabulary - Reading - Total Language - Total Mathematics.  81 Canadian Cognitive A b i l i t i e s Test: - Verbal Test - Non Verbal Test - Quantitative  Test.  School records were reviewed to obtain evidence of the following educational  interventions:  - extended primary (retention) - acceleration - learning assistance intervention - speech and language therapy - E.S.L. instruction - special class placement. A data sheet was  provided to each learning assistance teacher on  which the names of the subject pupils were l i s t e d .  Learning assistance  teachers placed a check in columns labelled with the following individual physical conditions which may  have a effect on academic achievement  during the primary years: - history of ear infections - conductive hearing losses - hearing impairment - vision problem (wears glasses) - allergies - physical handicap - chronic  illness  (See Appendices A and B for descriptors of physical problems and number of pupils reported to have physical problems.)  82 Upon completion of collection of pupil information, data were entered on the University Mainframe computing system.  Once the data were  entered, a comprehensive data cleaning procedure was employed to ensure the information entered was accurate.  The cleaning procedure entailed  the computing of frequencies and other basic descriptive s t a t i s t i c s f o r each of the student- and school-level variables.  The goal was to check  for inconsistencies which might indicate an entry e r r o r .  The  hierarchical linear model s t a t i s t i c a l program requires that the data f i l e s be set up i n a particular way.  Data c o l l e c t i o n , computer entry,  cleaning and set up of the raw data in preparation to begin analysis required approximately one year. Instruments The kindergarten screening measures were four individual tests administered at designated times throughout the school year to screen f o r exceptionalities in cognitive, language, visual-motor and pre-academic areas.  The d i s t r i c t provided inservice sessions and written instructions  to a l l kindergarten teachers to ensure standard administration and scoring of each of the instruments.  Figure 4 i l l u s t r a t e s the  administration of the various screening instruments across the school year. Figure 4 Administration of Kindergarten Screening Measures  DAP  MS KLST  W  1 Sept  Oct  Nov  DEVTOT  Dec  Jan  I Feb  Mar  Apr  May  June  83  Draw-a-Person Test The kindergarten teachers administered the Draw-a-Person Test (DAP) in November. Teachers administer the test by asking pupils to draw a person as well as they can.  The teachers make no suggestions regarding  how to complete the f i g u r e . The score of the DAP i s an indicator of non-verbal cognitive a b i l i t y .  The d i s t r i c t selected i t to identify the  maturation level at which the child was functioning (Harris, 1963). Researchers have demonstrated and discussed extensively the r e l i a b i l i t y and v a l i d i t y of the Draw-a-Person Test (Dunn, 1967; White, 1979; Naglieri & Maxwell, 1981).  Appendix C presents the technical characteristics of  the t e s t . The d i s t r i c t adapted the scoring from the Goodenough-Harris scale to include the items on the scale which primary and younger children were l i k e l y to draw. The d i s t r i c t developed local norms from the f i r s t administration of the instrument.  The test has a total score  of 31. A score of 7 or lower was the d i s t r i c t ' s cut-off score f o r indicating students were "at r i s k " . Mann-Suiter Visual Motor Screen The kindergarten teachers administered the Mann Suiter Visual Motor Screen (MS) i n January as an indicator of visual perception and f i n e motor s k i l l s .  The Mann Suiter i s a simple, normed, screening test which  consists of copying four geometric figures:  c i r c l e , square, t r i a n g l e ,  and diamond. Successful completion of the four figures represents minimal standards for success i n handwriting (Mann, Suiter & McClung, 1987).  Appendix D presents the norms for completion of each f i g u r e .  mark i s given for each correct figure. student was "at r i s k " .  One  Two or more errors indicated the  84  Kindergarten Language Screening Test The kindergarten teachers administered the Kindergarten Language Screening Test (KLST) individually to each pupil in January as an indicator of the child's language development. The KLST i s a normed screening test which investigates several aspects of language; previous studies have demonstrated i t s r e l i a b i l i t y and v a l i d i t y (Gauthier & Madison, 1973). test.  Appendix E presents technical information about the  It consists of seven parts:  a) f i r s t and last name, age; b)  i d e n t i f i c a t i o n of four colors - red, yellow, blue, green; c) counting with pointing 1-4, 5-10; d) i d e n t i f i c a t i o n of body parts - chin, knee, elbow, ankle; e) three part oral sequential command, knowledge of prepositions; f ) sentence repetition; g) spontaneous language sample from a three-picture representation. The total score i s 29.  A score below 21  indicated "at risk" status. Deverell Test of Letter and Numbers The kindergarten teachers administered the Deverell C l a s s i f i c a t i o n Test at the end of the school year when the children have had a common experience base.  The purpose of the test i s to measure the child's  a b i l i t y to recognize upper and lower case letters and numerals to 12. Appendix F presents technical information about the t e s t .  The d i s t r i c t  chose this task because researchers have shown consistently that knowledge of letters and numbers is one of the best predictors of academic success (Dykstra, 1967; Jansky & de Hirsch, 1972; Simner, 1982).  The total score possible i s 64 and a score of less than 56  indicated the student was "at r i s k " .  85 Canadian Tests of Basic S k i l l s The Canadian Tests of Basic S k i l l s (CTBS) is a  group-administered,  norm-referenced achievement t e s t , derived from the Iowa Tests of Basic Skills. G.  Technical characteristics of the tests are presented  in Appendix  The d i s t r i c t had administered the CTBS to a l l pupils in May of grade  three, and thereafter, annually through grade seven.  Classroom teachers  administered the CTBS to the majority of pupils after they had  attended  39 months of primary school (including 10 months of kindergarten). Approximately 10 per cent of the students remained in primary grades f o r four years, and therefore these pupils did not complete the CTBS u n t i l after 49 months of primary schooling. The outcome measures i n this study were grade three levels of reading, mathematics, vocabulary and language. Research Questions The purpose of this study is to examine the relationship between kindergarten screening measures and grade three achievement i n reading, mathematics, vocabulary and language.  The study also examines the extent  to which the relationship between kindergarten screening and grade three achievement i s mediated by the provision of learning assistance or extended primary.  1. a)  The study examines four research questions:  What i s the average within-school relationship between grade three  test scores i n academic achievement and scores on kindergarten screening measures of perceptual-motor, b)  language, and cognitive s k i l l s ?  To what extent do the relationships between achievement scores and  kindergarten screening scores vary across schools?  86 I hypothesize that for each measure of grade three achievement, the relationship between achievement scores and screening scores w i l l  vary  across various screening measures; some screening measures may be better predictors of particular outcomes scores than other screening measures. I expect also there w i l l be significant variation across schools in these relationships because schools vary in their allocation of resources to pupils with d i f f e r i n g levels of a b i l i t y . be more successful  For example, some schools may  in bolstering achievement of low a b i l i t y pupils than  of high a b i l i t y p u p i l s , or vice versa.  2.  a) What i s the relationship between grade three achievement and  kindergarten screening after controlling for the effects of gender, age at entry to kindergarten, and whether the child has a physical problem? b) Do the relationships between grade three achievement and kindergarten screening vary across schools after taking account of pupil characteristics?  I expect that the average within-school relationships w i l l be weaker after controlling for pupil's personal c h a r a c t e r i s t i c s , and that there w i l l be less variation across schools i n these relationships.  The  l i t e r a t u r e reviewed i n Chapter 2 suggested that younger, male children are l i k e l y to perform worse on the screening measures. may interfere also with the pupils performance.  Physical problems  Therefore, when the  effects of these characteristics are controlled, I expect the kindergarten screen/achievement relationships to be weaker and the between-school differences in the relationships to decrease.  The  87 decrease in variation between schools w i l l be minimal i f the pupil characteristics are distributed equally across schools because the effects of controlling the variable would be similar for a l l schools.  3.  a) To what extent are the relationships between grade three  achievement and kindergarten screening mediated by educational interventions of learning assistance or attending extended (4 year) primary schooling? b) Does the extent to which the relationships are mediated vary across schools?  My hypothesis is that remedial interventions depress the relationships between kindergarten screening and grade three achievement.  Therefore, I expect that the average within-school  relationship between screening and achievement w i l l be greater after taking account of the effects of learning assistance and extended primary schooling.  Because the effects of these interventions may  schools, there may  vary across  be greater variation in kindergarten  screen/achievement relationships after removing the effects of the intervention.  4.  a) If there i s significant variation between schools in their  relationships between screening and outcome measures, to what extent can i t be explained by school s i z e , rural versus urban location or the school mean and variance of pupils' a b i l i t y ? b) To what extent are the between-school differences in achievement  88  explained by various school-level variables?  I expect that i f there i s significant variation between-schools in their kindergarten screen/achievement relationships, some of the variation may be explained by school s i z e .  In smaller schools, low  a b i l i t y pupils may have greater opportunities to benefit from regular i n s t r u c t i o n , and may have a better chance of receiving remedial instruction.  If so, this would result in shallower kindergarten  screen/achievement slopes for smaller schools.  The same processes may  apply i n rural schools compared with urban schools.  Allocation of  resources to pupils of varying a b i l i t y may also be related to the distributions of a b i l i t y within and across schools.  Several studies have  shown that school mean-ability or SES can have an effect on students' outcomes over and above the effects associated with students' individual backgrounds (Willms & Chen, 1989; Brookover et a l . , 1978; Summers & Wolfe, 1977). I hypothesize that the school-level variables w i l l explain some achievement differences between-schools because some schools may be more successful in bolstering achievement of low a b i l i t y p u p i l s .  These  differences may result from various processes which contribute to p u p i l s ' achievement.  This study did not include variables for measuring  processes such as teaching practices, curriculum coverage or parent and teacher press for academic success (Anderson, 1982).  Therefore, the  school-level variables in this study may act as proxies for other variables of school processes and provide explanatory power i n the analysis (Willms, 1986).  89  Analysis of the Data Data The goal of the analysis i s to examine the relationships between kindergarten screening measures and grade three achievement i n reading, mathematics, vocabulary and language, and to determine the extent to which the relationships are mediated by educational interventions.  The  analysis includes examination of the relationships within schools and between schools, and investigates whether the relationships vary across schools. The within-school variables include kindergarten screening measures, student c h a r a c t e r i s t i c s , and interventions. They are: DAP  This is a continuous variable measuring the score obtained on  the Draw-A-Person Test.  The variable was centered on the cut-off score  of 7, which indicated "at risk" status. To center a variable on a particular value, one subtracts that value from each individual's score.  After centering, therefore, children with  a score of zero on DAP had scored at the "at risk" cut-off score.  Those  with negative DAP scores scored below the cut-off score, and those with positive scores scored above the cut-off score.  Centering f a c i l i t a t e s  interpretation of the intercepts in the within-school equations  (see  Willms, 1984). KLST  This i s a continuous variable measuring the score obtained on  the Kindergarten Language Screening Test.  The variable was centered on  the cut-off score of 20, which indicated "at risk" status. MS  This i s a continuous variable measuring the score obtained on  the Mann-Suiter Visual Motor Screening Test.  The variable was  centered  90 on the cut-off score of 2, which indicated "at risk" status. DEVTOT  This i s a continuous variable measuring the score obtained  on the combined subtests of the Deverell Test of Letters and Numbers. The variable was centered on 55 which indicated "at risk" status. AGE  Age at entry was based on the month born.  increments from -5.5 to 5.5:  It was coded i n unit  students who were r e l a t i v e l y young f o r  their cohort received negative values (e.g., birth dates in December and November were assigned values of -5.5 and -5.4 respectively); students who r e l a t i v e l y were old for their cohort received positive values. GENDER  A dummy variable representing pupil's sex was coded zero f o r  males and one f o r females. HANDICAP was a dummy variable representing whether or not the pupil was affected by one or more of the following physical conditions:  visual  impairment, chronic ear infections, hearing impairment, a l l e r g i e s , physical handicap or chronic i l l n e s s . A zero indicates no physical impairment present, a one represents the presence of a physical impairment. LEARNING ASSISTANCE i s a dummy variable representing whether or not the pupil received remedial instruction.  A zero indicates no assistance  was provided; a one indicates the pupil participated in remedial instructional a c t i v i t i e s . EXTENDED PRIMARY i s a dummy variable representing whether or not the pupil attended four years of primary schooling (after completing kindergarten) to complete grades one to three.  A zero indicates the  pupil completed the primary school in three years; a one indicates the pupil attended four years of primary schooling.  91  The between-school variables are: MEANCCAT  This i s a continuous variable representing the mean  a b i l i t y score within each school calculated from the grade three level Canadian Cognitive A b i l i t i e s Test scores of the combined cohorts of pupils.  Before aggregating to the school l e v e l , the variable was  centered on 100, which i s the national norm for the t e s t . SDCCAT  This i s a continuous variable, the standard deviation of the  CCAT scores for the combined cohorts within each school.  This i s used to  represent the heterogeneity of pupil a b i l i t y within each school. SCHOOLSIZE  This i s a continuous variable describing the size of the  grade three enrollment. RURAL  This i s a dummy variable representing geographic location of  the school; i t i s coded Rural = 0, Urban = 1. ATTRITION  This i s a continuous variable representing the effect of  a t t r i t i o n within each school.  The computation of the a t t r i t i o n variable  i s described under A t t r i t i o n Bias. (See Appendix H for Characteristics of Schools.) Analyses The analyses of the relationships between kindergarten screening measures and grade three achievement employ a two-level hierarchical linear regression model (Bryk & Raudenbush, 1987). The model estimates the average within-school relationship for each screening measure with reading, mathematics, vocabulary and language and the extent of variation in these relationships across schools (questions l a & l b ) . The model i s represented by the following equations:  92 (Achievement)^ = p  Pupil-level:  School-level: p  0 j  + p  x j  (K-Screen)^ + ei;j  (1)  = e 0 0 + Uoj  (2)  Pij = e 1 0 + Uxj  (3)  o j  where the subscripts ^ denote:  pupil i(i=1,2,...,n) in school j  (j=l,2,...30). The f i r s t level of the model comprises 30 separate within-school regressions, represented by equation 1. p  o j  and p  l j s  The parameters of interest are  the intercepts and slopes for the 30 schools.  K-Screen  refers to one of the kindergarten screening measures, or to a combination of screening measures.  Because the screening measures are centered  around the cut-off score for "at risk" status, the estimate of p  o j  for a  particular school i s an estimate of how well a pupil with a kindergarten screening score at the cut-off would score on the outcome variable. Estimates of p  specify the outcome/kindergarten  x j  screen relationship f o r  each school. The second level of the model, represented by equations 2 and expresses the p  o j  and p  x j  3,  as a grand mean ( e 0 0 and e 10 respectively) and  a school-level residual term (U oj and ULj respectively).  This i s the  simplest school-level model; i t contains no school-level variables.  An  estimate of e o j therefore represents the average achievement score for the entire d i s t r i c t for a pupil with a K-screen score at the c u t - o f f . estimate of e L j represents the average achievement/kindergarten  screen  relationship for the d i s t r i c t (Question l a ) . HLM combines the equations and estimates the parameters at both levels simultaneously (Raudenbush & Bryk, 1986).  HLM also provides  An  93  estimates of the variance of school-level residual terms, that i s , Var (U oj ) and Var (U 1 ; j ). = Var (Uj^).  In this model, Var (p o j ) = Var ( U 0 j ) , and Var  By examining whether Var p o j =0, one can determine  whether the observed differences in adjusted means across schools could have occurred by chance.  The examination of whether the Var (p 1:j )=0  determines whether the observed differences in the outcome/kindergarten screening relationship across schools could have occurred by chance. test of t h i s hypothesis addresses question l b .  A  I expect s i g n i f i c a n t  variation across schools in the outcome/kindergarten screening relationship because the schools probably vary in their a l l o c a t i o n of resources to pupils with d i f f e r i n g levels of a b i l i t y . Questions 2a and 2b are addressed by adding variables describing pupil-level characteristics to the f i r s t level of the model:  (Achievement)^ = p  o j  (K-Screen)i;j + p 2 j (Gender) i j  + e  P3j ( 9 ) i j A  +  P4j  The second level of the model now  (Handicap)i;j + ei;j  + (4)  includes f i v e equations, similar  to Equations 2 and 3, which model the between-school variation in the intercepts and in the four f i r s t - l e v e l parameters. In t h i s hierarchical model, the p  0 j  for a particular school i s an  estimate of how well a pupil with a screening score at the cut-off point would score on the outcome variable after controlling f o r gender, age on entry to kindergarten and handicapping  conditions.  I expect that the  average within-school relationships would be weaker after controlling f o r  94  pupil c h a r a c t e r i s t i c s , that there would be less variation across schools, and that variation between schools in the outcome/kindergarten screening relationships would be influenced minimally. To address questions 3a and 3b, two dummy variables are added to the pupil-level model;  (Achievement)^ = p oj + p xj P3j  (K-Screen)^ + p 2j (Gender)^ +  (Age)ij + P4j  Prim)^ + p 6j  The school-level model now  (Handicap)i;j + p 5 j  (Extended  (Lrn. A s s t ) ^ +  (5)  includes seven equations.  The main interest of this model is the size and direction of the average p 5 j and p 6 j across schools.  If the estimates of these parameters  were p o s i t i v e , the results would suggest that those who received remedial interventions scored higher than their peers with comparable scores on the screening measure.  If this were the case, I would expect the  estimate of p x j for this model to be larger than the estimate for the model given by Equation 4.  This would lend support to the hypothesis  that remedial interventions have the effect of lowering observed relationships between screening measures and subsequent academic achievement. To address questions 4a and 4b, school-level variables are added to the model.  The school level model regresses the parameters from the  student-level (within-school) model on particular school-level variables, such as school mean-ability or school s i z e .  The f u l l model includes the  95 pupil-level equation (Equation 5) and two school-level equations:  POJ = e 00 + e 0 iZij + U0j  (6)  Pij = e 10 + e^Zjj + Uxj  (7)  where Z x j i s a school-level variable. one school-level variable.) interest.  (The model can include more than  e 01 and e u  are the regression parameters of  They indicate the strength of the effects of the  school-level  variable on the average levels of achievement and on the outcome/ kindergarten screen relationship. school-level error terms.  The error terms U0j  and  are  They specify the unique contribution of each  school not explained by the school-level variables in the model.  I am  interested in whether the estimates of the adjusted slopes of outcome/kindergarten screen, p l j t and the estimates of adjusted school performance, p oj  , are a function of particular school-level variables.  I expect that the inclusion of one or more school-level variables would explain some of the between-school differences in the outcome/ kindergarten screen relationships. the variance of U Lj .  This would result in a decrease i n  S i m i l a r l y , I expect that one or more school-level  variables would explain the between-school differences in achievement f o r students who measures.  obtained scores "at risk" on kindergarten screening  If this were the case then the variance of the school-level  r e s i d u a l s , U oj ,  would decrease s i g n i f i c a n t l y with the addition of the  school-level variables.  96 Preliminary Analyses The SPSS-X program was used in the following ways: a)  to prepare the raw data for analysis;  b)  to calculate means and standard deviations of a l l variables f o r  each cohort and for the combined cohorts and achieved sample; c) to compute a Pearson-product-moment correlation matrix of a l l variables; d) to create new variables from raw data, including interaction terms for a l l kindergarten screening measures with age, gender and intervention v a r i a b l e s , to calculate the mean CCAT score for each student; to calculate the mean a b i l i t y score for each school; e) to plot the mean achievement score at each point of the kindergarten screening measures to ensure a linear relationship between the kindergarten screening and outcome measures; f ) to run ordinary least squares regression to investigate the effects of some variables for selection of variables to be included i n hierarchical model analysis, including a model with a quadratic term, to check for l i n e a r i t y ; subsequent to these exploratory regressions, physical conditions were combined because individual problems lacked explanatory power i n this analysis.  Educational interventions of ESL,  speech/language therapy and special class placement were dropped as they also lacked explanatory power. The Tell-A-Graf graphics program was used to generate plots and graphs. HLM Analysis Preliminary HLM analyses tested whether regression c o e f f i c i e n t s  97  varied s i g n i f i c a n t l y from school to school.  No significant variation i n  the effects of age, gender or handicapping conditions occurred  and  therefore regression coefficients were modelled as fixed e f f e c t s . the p 2 j , p 3j and p 4j  in Equations 4 and 5 become simply (j 2 , fj3 and  Now p4;  therefore there are no school-level equations for these parameters.  I  tested also whether there were significant differences between the cohorts in the adjusted levels of achievement.  Differences were small  and s t a t i s t i c a l l y i n s i g n i f i c a n t , so cohort was dropped from the model. Interaction terms between kindergarten screening measures and age or gender were tested in preliminary analyses. terms were not s i g n i f i c a n t .  Most of the interaction  When they were significant the effects were  small, and thus not included in further analyses.  The regression  c o e f f i c i e n t s which varied s i g n i f i c a n t l y (intercepts, slopes, and interventions) were modelled as random e f f e c t s . School-level variables were included in exploratory models to examine their effects on both kindergarten screen/achievement relationship and achievement.  Only school mean a b i l i t y was  contribute s i g n i f i c a n t l y so i t remained in the model. variable was  The  found to attrition  not significant in preliminary analysis when modelled on  both the intercept and the slope, but because i t had a consistent, small, negative effect on the slope, i t was retained in the equation which modelled variations in slopes. Threats to V a l i d i t y The question of v a l i d i t y i s always of concern in predictive research.  The quality of the screening and outcome measures, the  characteristics of the sample and selection bias may  influence the  98 v a l i d i t y of the findings.  Care was given to address some of these  concerns in the following ways: a)  Appendices C-G present the technical information f o r the  kindergarten screening measures and outcome measures; b)  Characteristics of the sample: i)  age cohorts were selected to prevent i n f l a t i o n of performance scores by "over-age" pupils enrolled in grade l e v e l ;  ii)  male-female representation i s similar for each total cohort and for the achieved sample;  i i i ) mean scores on c r i t e r i o n measures for the total data set and for each cohort are similar and although they are s l i g h t l y higher than national norms, the cohort and sample means are representative of the d i s t r i c t performance during the years the CTBS and CCAT were administered (Appendix Table 21). c)  A t t r i t i o n Bias One concern was that the achieved sample would be biased because i t  represented a less transient population than those who had not been i n the d i s t r i c t continuously.  A second concern was the loss of the  grade-three outcome scores f o r the 120 pupils i n special class placements.  Although the kindergarten screening scores f o r these pupils  were included i n the total data, there were no CTBS scores f o r them, and therefore they were not included in the hierarchical analysis. I was able to estimate the extent of a t t r i t i o n bias by comparing the kindergarten screening scores of pupils i n the achieved sample with scores of pupils who were tested in kindergarten, but were not tested i n grade three, either because they had l e f t the d i s t r i c t , were in special  99 programs or for some other reason.  The comparison was accomplished i n  the following way. Standard scores were computed for each of the kindergarten screening measures. A principal components analysis was computed on the kindergarten scores to determine the factor loading of the four kindergarten screening measures.  The f i r s t principal component  was used as a composite screening measure. A dummy variable i d e n t i f i e d pupils with both kindergarten screening scores and outcome scores (coded 1), and those with only kindergarten screening scores (coded 0 ) . An hierarchical linear regression was run on the entire sample using the following equation:  (Composite)^ = p  o j  + p  x j  (Study)i;j +  The results presented in Appendix Table 2 suggest that the mean scores of the achieved sample were upwardly biased, but the differences were small.  This does not mean that a t t r i t i o n would necessarily have  affected the observed Kindergarten-screen/achievement slopes.  However,  the finding did suggest that i t would be worthwhile to model e x p l i c i t l y the effects of a t t r i t i o n on slopes to test their s i g n i f i c a n c e .  Thus a  measure of the extent of a t t r i t i o n for each school was included i n the school-level equations of the HLM (see Equations 6 and 7).  The estimates  of the parameter p for each school are used as the A t t r i t i o n v a r i a b l e . x  Summary This chapter has described the subject sample, research methodology and threats to v a l i d i t y . the analyses.  Chapter 4 presents the findings of  Chapter 5 discusses the findings and includes  recommendations for future research.  procedures,  100  Chapter 4 Findings Introduction This chapter presents the findings of the analyses.  The  first  section presents a correlation matrix of the major dependent and independent variables.  The second section explains the format of the  tables which display the findings of the HLM  analyses.  The next four  sections present the results of the f i t t e d models, Models I to IV, which address the research questions and investigate the relationships between the outcome measures on each of the kindergarten screening measures.  The  seventh section discusses the estimated parameter variance explained within-schools and between schools.  The eighth section presents Model V  which extends the investigation by including a l l four screening measures in the same model.  The ninth section presents Model VI, which includes  only those variables found to be s i g n i f i c a n t in prior models. Some tables are presented in the text. particular HLM models may  Tables describing the findings for  be found in the Appendices.  The discussion of  the findings and recommendations for further research are presented i n Chapter 5. Correlation Matrix Table 1 shows the means and standard deviations of the pupil-level variables and their correlations.  Most of the correlations were  s t a t i s t i c a l l y s i g n i f i c a n t and r e l a t i v e l y low:  they range from -.002  .704**. (**Indicates correlations are s i g n i f i c a n t at the .01  level)  to The  highest correlations were between the grade three achievement measures, which range from .589** to .704**. Correlations between educational  Table 1 Means, Standard Deviations, and Correlations of Student-Level Variables DAP Means (Centered) 5.05 Standard Deviation (4.57) Total (N=957)  DAP  KLST 4.43 (3.12)  MS 6.40 (5.29)  DEVTOT READ3 MATH 3 V0CAB3 LANG3 AGE .29 41.45 41.62 41.52 43.89 .150 (.80) (8.61) (7.40) (8.17) (8.33) (3.43)  HANDIGENDER CAP .005 .086 (.509) (.409)  1.00  KLST  .252** 1.00  DEVTOT  .249** .374** 1.00  MS  .280** .231**  .216** 1.00  READ3  .292** .329**  .292** .200** 1.00  MATH3  .203** .296**  .287** .237** .651** 1.00  V0CAB3  .283** .357**  .297** .204** .685** .589** 1.00  LANG3  .309** .346**  .360** .269** .699** .704** .627** 1.00  AGE  .220** .148**  .090** .164** .087** .126**  .118** .124** 1.00  GENDER  .231** .299**  .133** .058*  .142** .047  .082** .196** -.027  .053  .010  .025  -.010  .014  .012  -.013  .016  1.00 -.002  1.00  HANDICAP  -.047  EX.PRIM  -.256** -.307** -.434** -.234** -.286** -.290** -.262** -.346** -.088** -.099** -.147**  LRN ASST  -.168** -.190** -.219** -.176** -.265** -.270** -.263** -.291** -.056  Note: * Significant at the .05 level. **Significant at the .01 level.  .028  .085**  EX LRN PRIM ASST .093 .095 (.291) (.293)  102  interventions and screening and achievement measures were negative and statistically significant.  On the kindergarten screening and grade three  achievement measures, g i r l s had an advantage over boys, and older students had an advantage over younger students.  The correlations  between educational interventions with sex and age favored boys and younger students. Format of the Tables Appendix Tables 3 to 20 present the results f o r the regressions of reading, mathematics, vocabulary and language achievement on each of the kindergarten screening measures.  The format i s essentially the same f o r  a l l the tables although not a l l models include a l l of the variables 1 isted. The table i s divided into three parts.  The f i r s t section of the top  part of the table shows the average within-school equation (see Chapter 3, Equation 1).  Parameters were fixed for variables which did not vary  across schools in the exploratory analysis.  The second section of the  top part of the tables shows the effects of the between-school variables.  These are estimates of parameters that were allowed to vary  across schools, which were included only in Models IV, V, and VI. The middle part of the table shows the estimates of the extent to which the parameters vary across schools.  The bottom section of the tables show  2  2  the maximum likelihood of 0 , and the two estimates of R and school levels:  at the pupil  the total variance explained and the residual  parameter variance explained on achievement.  103  Model I Kindergarten Screening Measure/Achievement Relationships The Model I regressions address the questions "What are the average within-school relationships between screening measures and achievement measures?"  and "Do the relationships vary across schools?"  Table 2 displays the d i s t r i c t and achieved sample means and standard deviations f o r the grade three test scores.  The d i s t r i c t s t a t i s t i c s were  computed on complete data for two cohorts combined, the study sample s t a t i s t i c s were computed f o r subjects having scores on a l l variables included in the analysis.  The means are expressed as grade equivalent  measures in months of schooling; f o r example, a score of 41.4 i s equivalent to a grade equivalent of grade 4.14.  The tests were taken at  the end of grade 3, that i s , after 39 months of schooling f o r most p u p i l s , and after 49 months for those who attended the extended primary program.  (See Appendix I for discussion on Grade Equivalent Scores.)  Table 2 D i s t r i c t and Sample Means and Intercepts f o r Pup i l s at the Risk Cut-off Score Reading Mathematics Vocabulary Language D i s t r i c t Means (2 Cohorts)  41.1  41.2  41.3  43.7  Study Sample Means 41.5  41.7  41.5  43.9  Intercepts "at risk" DAP 38.31 KLST 37.52 DEVTOT 38.23 MS 40.64  39.48 38.54 38.38 40.67  38.62 37.49 38.39 40.78  40.17 39.77 39.93 42.62  104  The achievement scores for reading, mathematics, vocabulary  and  language were regressed on each kindergarten screening measure.  The  figures were taken from Model I of Appendix Tables 3 to 18.  The  intercepts represent the average achievement score for pupils who obtained a score on a screening measure at the cut-off point for "risk" status.  The expected average score for a l l pupils is 39 based on the  norms of the CTBS. The intercept scores for pupil's obtaining a screening score at the cut-off point reported in Table 2 are near the expected average score.  The d i s t r i c t means are higher than the expected  score thus the average performance of the pupils at the cut-off point for r i s k status i s below the d i s t r i c t average. One reason for the r e l a t i v e l y high scores may pupils due to placement i n special c l a s s .  be the loss of the  120  These pupils required  intensive intervention in response to their learning d i f f i c u l t i e s .  The  scores on achievement measures for these pupils would probably be low and would have lowered the average score for the pupils "at r i s k " i f they had been administered the CTBS. Two approaches to data selection may also have contributed to the fact that the intercepts are r e l a t i v e l y high for pupils i d e n t i f i e d as "at risk".  The pupils lost through a t t r i t i o n had s l i g h t l y lower kindergarten  screening scores than the remaining pupils. achievement was  If their subsequent  low, the f a i l u r e to include the scores would have the  result of a higher mean score for achievement than i f their scores had been included. The selection of the study sample required listwise deletion of subjects.  To be included in this study, a pupil had to have  105  scores for eight test measures. testing may  Students who were absent during the  have had lower achievement scores than students for whom  there were eight scores. There are at least three other reasons why the scores may Some pupils may  be high.  have performed poorly on one test only and thus may  have truly been at risk of experiencing learning d i f f i c u l t i e s .  not  The  outcome scores for these pupils may have been r e l a t i v e l y high when compared to pupils who  had learning problems during their primary years.  Another p o s s i b i l i t y is that the cut-off score was not the most appropriate score for identifying the true "at risk" p u p i l . off score was set too high, normal measurement error may the inclusion of pupils who were not truly at r i s k .  If the cut-  have resulted in  The selection of a  lower cut-off score would lower the level of the average performance (intercept). A third p o s s i b i l i t y is that the achievement of pupils i d e n t i f i e d as "at risk" was bolstered by interventions and therefore, their grade three achievement i s similar to that of other pupils who  had performed  acceptably on the screening measures. Table 3 displays the estimates of the coefficients for a l l of the within-school relationships between the grade three achievement measures and the kindergarten screening measures. A l l relationships were s t a t i s t i c a l l y significant at the .01 level of significance. relationships were s t a t i s t i c a l l y significant at the .001  ( A l l 16  level of  significance; the probability of a Type I error i s .003 thus I am not concerned about the Type I error rate being inflated because of the large number of s t a t i s t i c a l tests.)  106  The metric across kindergarten screening measures i s d i f f e r e n t ; therefore, the within-school effects must be interpreted for each measure.  independently  For example, the coefficient for Reading on  Draw-a-person test i s .57; this means that each point earned on the Draw-a-person test represents .57 of a month growth in reading.  The  c o e f f i c i e n t f o r Language on Mann-Suiter test represents 2.57; t h i s means that each point earned on the Mann-Suiter test represents 2.57 months of growth in Language. The number of items on Draw-a-person i s 31 and the number of items on the Mann-Suiter is 4, therefore, the interpretation of the c o e f f i c i e n t s must be considered with regard to the individual measures.  The standardized coefficients are presented in parenthesis.  These c o e f f i c i e n t s may be compared across predictor and c r i t e r i o n measures.  Table 3 Estimates of the Effects on Grade Three Achievement of One Point (or one SD) Kindergarten Screening Measure Score DAP (31 items)  KLST (28 items)  DEVTOT (64 items)  MS (4 items)  Reading  .57** (2.60)  .86** (2.68)  .47** (2.49)  1.89** (1.51)  Mathematics  .35** (1.60)  .66** (2.06)  .40** (2.12)  2.29** (1.83)  Vocabulary  .53** (2.42)  .90** (2.81)  .46** (2.43)  1.92** (1.54)  Language  .61** (2.79)  .86** (2.68)  .55** (2.91)  2.57** (2.06)  ** Significant at the .01 l e v e l .  The relationships between the kindergarten screening measure of language a b i l i t y (KLST) are strong for a l l four outcome measures of  107  achievement.  The relationship between the kindergarten screening measure  of cognitive a b i l i t y , the DAP, was stronger for reading, vocabulary and language (.57, .53, .61 resp.) than for math (.35). The predictive relationships for the test of letters and numbers, the Deverell, are strongest for language, followed by reading and vocabulary and weakest f o r mathematics.  The relationship between the  visual motor t e s t , Mann-Suiter, i s stronger for language and mathematics than for reading and vocabulary.  Because the Mann-Suiter had only four  items, the coefficients appear large compared with the other screening measures.  One d i f f i c u l t y in interpreting the relationships of the DEVTOT  and MS with achievement outcome measures i s the c e i l i n g e f f e c t f o r both tests caused by a limited number of items.  These two tests did not  measure the entire range of performance possible on visual-motor performance or on recognition of letters and numbers.  The relationships  between these screening measures and achievement might be different i f more d i f f i c u l t items had been included which measured the f u l l range of a b i l i t y with respect to visual-motor performance or l e t t e r and number recognition. The standardized coefficients can be compared.  In general, the  strongest predictive relationships for a l l four outcome measures were found for the KLST and the Deverell Test of Letters and Numbers. weakest predictor i s the Mann-Suiter.  The  The low number of items in the  Mann-Suiter resulted in c e i l i n g effects which probably limited the predictive power of the t e s t . The r e l i a b i l i t y of the estimates of adjusted school meanachievement for the f i r s t model ranged from .4 to .6.  The Kindergarten-  108  screen/achievement relationships did not vary across schoolscould not be estimated r e l i a b l y ; the range was from .022 to  The slopes .252.  Reliable estimation of the slopes for individual schools would require larger numbers of subjects within schools.  In most cases t h i s i s not  possible because enrollment in elementary schools i s small.  An alternate  approach to increase the r e l i a b i l i t y would be to c o l l e c t data on the same schools over a number of years and to estimate the average within-school slope over time (Willms & Raudenbush, 1989). Model II Controlling for Pupil Characteristics The second model (see Appendix Tables 3-18)  examines the  kindergarten screen/achievement relationships after controlling for the effects of student-level c h a r a c t e r i s t i c s , and examines the extent to which the relationships vary across schools (Questions 2a and 2b).  The  second model includes age, gender and physical problems as control variables.  The within-school coefficients for age, gender and physical  problems were constrained to be identical across schools.  The intercepts  and the c o e f f i c i e n t s for kindergarten screening measures were allowed to vary across schools. The age-at-entry effect ranged from .07 to .25 across the 16 regressions (Model I I ) , with a median value of .18.  The effect  was  s t a t i s t i c a l l y significant for mathematics and language across a l l regressions, and significant for vocabulary in three of the four regressions.  Age-at-entry was not a s t a t i s t i c a l l y s i g n i f i c a n t predictor  of grade three reading scores.  An age-at-entry effect of .18 means that  for every month a child i s older than his or her peers, the average grade  109  three achievement score i s .18 months of schooling higher, after taking account of the child's kindergarten screening score, sex, and whether the c h i l d has a handicapping condition. Thus, by grade three there remains, on average, approximately, a two-month gap between those who  are  r e l a t i v e l y young for their grade ( i . e . , those born in November and December), and those who  are r e l a t i v e l y old for their grade ( i . e . , those  born in January or February). The estimated gender effect ranged from .06 to 2.94 (Model II) regressions, with a median value of 1.09.  across the 16  The estimates were  s i g n i f i c a n t for reading and language on a l l four kindergarten screening measures and significant for vocabulary in one regression (MS).  Gender  was not a s t a t i s t i c a l l y significant predictor of grade three mathematics, nor of vocabulary in three of the four regressions.  A gender effect of  1.09  means that for females, the average grade three achievement score i s  1.09  months of schooling higher, after taking account of the child's  kindergarten screening score, age, and whether the child has a physical problem.  Thus, by grade three, females achieve, on average, more than  one month of schooling higher than males with similar c h a r a c t e r i s t i c s . The gap between males and females i s larger, on average, for achievement in reading and language than for math and vocabulary (1-2 months in reading and 2-3 months in language). If the primary interest of research was s p e c i f i c a l l y age or gender, one would not control for kindergarten screening measures.  The  unadjusted gap for age and gender would be larger, suggesting that females make greater progress i n these areas between kindergarten and grade three.  110  The estimated effects of physical problems ranged from -.80 to with a median value of .04.  .82,  The effects were not s i g n i f i c a n t for any  measures; however, the estimates for math and language were small and negative, while the estimates for reading and vocabulary were small and positive. Question 2a asks what the relationships would be after controlling for pupil c h a r a c t e r i s t i c s .  The average within-school relationships of  outcomes on kindergarten screening scores ranged from .40 to 2.57 Model I and from .32 to 2.29  in  in Model I I . The estimated c o e f f i c i e n t s a l l  declined minimally, the difference between the Model I and Model II c o e f f i c i e n t s ranged from .02 to .28, with a median of .04.  Although the  c o e f f i c i e n t s declined, they remained s t a t i s t i c a l l y s i g n i f i c a n t and nearly as strong as in Model I. Question 2b asked whether the kindergarten screen/achievement relationships varied across schools after controlling for pupil characteristics. schools was  The range of the estimated parameter variance across  .01 to .16.  Most of the slopes were found not to vary  s i g n i f i c a n t l y across schools.  The f a i l u r e to reject the hypothesis of  s i g n i f i c a n t variation across schools may  be a Type II e r r o r .  Difference  between schools in their within-school slopes could not be r e l i a b l y estimated because the number of subjects within the school was r e l a t i v e l y small at the primary l e v e l . s i g n i f i c a n t for f i v e slopes:  The estimated parameter variance was reading, vocabulary and language on the  Draw-a-Person test (.04*,.04*,.05** respectively), mathematics on KLST (.08*), and language on the Deverell (.03*). As an example, the language with DAP  shrunken estimates of slopes were about -.28, -.25, -.25 for the  Ill bottom three schools and .23, .23, .21 for the top three schools. The significant differences in the relationships of three achievement measures on Draw-a-Person and of mathematics on KLST may indicate d i f f e r e n t i a l achievement outcomes across schools for pupils who obtained low scores on Draw-a-Person.  Table 4 Estimated Residual Parameter Variance of Mean Achievement for Pupils at the Cut-Off Score for Risk Status DAP  KLST  DEVTOT  MS  Reading  7.20**  6.84**  5.02**  4.06**  Mathematics  8.40**  8.35**  2.52  3.56**  Vocabulary  5.16**  4.43**  .51  2.26**  12.46**  13.86**  8.81**  6.52**  Language  ** Significant at the .01 l e v e l .  Table 4 presents the estimates of the residual parameter variance of mean achievement scores for pupils at the cut-off score that indicates r i s k status on kindergarten screening measures.  Fourteen of the sixteen  estimates of the parameter variance of achievement varied s i g n i f i c a n t l y across schools with a range from .51 to 13.86**.  In most cases the  variation among schools i s large in substantive terms.  For example, the  estimated parameter variance of achievement (intercepts) f o r language on DAP was 12.46. The standard deviation of the estimate i s therefore about 3.5 (Square Root of 12.46).  Therefore, the difference in performance  between an average school and a low (or high) scoring school could be as much as six months of schooling.  112  Model III Controlling for Educational Interventions The third model examines the mediating effects of educational interventions on the relationships (Questions 3a and 3b).  It allows for  the examination of whether the kindergarten screen/achievement relationships are mediated by educational interventions and investigates i f the mediating effects vary across schools. The estimated coefficients for the two interventions are negative, large and s t a t i s t i c a l l y significant across a l l models.  The range of the  effects of attending an extra year of primary schooling i s from -2.22** to -6.74**, with a median effect of -4.16**. The range of the effects of receiving learning assistance i s from -4.92** to -6.50**, with a median effect of -5.52**. An extended primary effect of -4.16 means that a pupil who obtained a score at the cut-off point on a kindergarten screening measure and who  attended an extra year (four years total) of  primary school, scored on average, 4.16 months of schooling below pupils with similar kindergarten screening scores who attended only three years of primary school. for risk and who  S i m i l a r l y , the pupils who  scored at the cut-off score  received learning assistance, achieved on average, 5.52  months of schooling below similar pupils who did not receive the assistance.  This suggests that on average, pupils who received special  education interventions had lower achievement scores, even after controlling for their individual pupil characteristics of age, gender and physical problems, than did their peers who obtained similar screening scores but who did not receive special educational interventions. There are at least three possible explanations for the large,  113  s i g n i f i c a n t , negative e f f e c t s . ineffective or i n s u f f i c i e n t .  One i s that the interventions were Second, placement i n intervention programs  and subsequent progress may have been influenced by factors not included in the model.  Third, opportunities to receive assistance may have been  determined by factors other than screening information and may have occurred later for some pupils than for others and thus, the effects may vary for different children.  These p o s s i b i l i t i e s are discussed i n  greater depth in Chapter 5. After controlling for the effects of educational interventions, the effects of age and gender declined minimally. effects for age was .06 to .21*.  The range of the estimated  Age effects remained s i g n i f i c a n t f o r  mathematics and language on a l l four kindergarten screening measures and for vocabulary on the Deverell and Mann-Suiter.  The range of the  estimated effects for gender was 1.29* to 2.41**.  The gender effects  remained significant for language on a l l four screening measures and f o r reading with the Deverell and Mann-Suiter.  This means that advantages in  achievement for older pupils and for females described under Model II findings remain even after controlling for the effects of special educational interventions. The average within-school estimated coefficients for the relationships between outcome measures and kindergarten screening measures decreased across a l l measures.  Table 5 presents the estimated  c o e f f i c i e n t s for the outcome measures on screening measures from Models II and I I I .  114  Table 5 Estimated Coefficients f o r Kindergarten Screen/Achievement Relationships DAP Models II  KLST III  DEVTOT  II  III  II  MS III  II  III  .98*  Reading  .51** .39**  .80**  .61** .44**  .33**  1.70**  Mathematics  .32**  .21**  .64**  .48** .38**  .26**  2.16** 1.56**  Vocabulary  .50**  .40**  .87**  .72** .44**  .34**  1.73** 1.16**  Language  .52**  .37**  .76**  .54** .51** .36**  2.29** 1.52**  * Significant at the .05 l e v e l . ** Significant at the .01 l e v e l .  Every estimated coefficient declined. The range of the estimated c o e f f i c i e n t s in Model II was .32** to 2.29**, with a median of .52**. The range of the estimated coefficients i n Model III was .21** to 1.56**, with a median of .48**.  This means that the estimated c o e f f i c i e n t s of  the relationships between achievement measures on screening measures were lower after controlling f o r the effects of educational  interventions.  I hypothesized that participation in special educational interventions  would result in improvement of achievement scores f o r  students identified as "at risk" on kindergarten screening measures during the time under study.  Higher achievement scores f o r pupils with  low scores on screening measures would have the effect of flattening the slope; that i s , lowering the estimated c o e f f i c i e n t .  If t h i s were the  case, when the effects of the educational interventions  were c o n t r o l l e d ,  the slope would become steeper; that i s , the estimated c o e f f i c i e n t would increase.  The results of Model III indicate that the slopes were  115  mediated by the educational interventions, but in the opposite direction of my hypothesis.  Controlling f o r interventions decreased the estimated  c o e f f i c i e n t s for the outcome on screening relationships. the predictive power of the screening measures decreased  This means that after  controlling for the effects of the interventions. Differences between schools in the estimated parameter variance f o r achievement for pupils with screening scores at the cut-off point f o r risk status decreased after controlling for educational interventions. The estimated variances between schools remained significant for reading on DAP (4.33*), and MS (2.73**), and for language on DAP (10.41**), KLST (9.98**) and MS (6.23**).  This means that the average achievement of  pupils who received scores at the cut-off score for risk status on kindergarten screening measures remained different across schools even after controlling for the effects of educational interventions. The slopes did not vary across schools. The estimated parameter variances for extended primary and f o r learning assistance were significant for language on KLST (13.86*) and for reading on Draw-a-person (13.49*).  This means that the effects of  these educational interventions differed s i g n i f i c a n t l y among schools f o r the pupils who obtained scores at the cut-off point for risk status on the KLST and Draw-a-person. Model IV School-Level Variables The fourth model allows for the examination of whether differences between schools i n the kindergarten screen/achievement relationships or between-school differences in achievement can be explained by  116  school-level variables (Questions 4a and 4b).  The primary interest of  t h i s study was the relationships between kindergarten screening measure and grade three achievement.  The research questions explored the extent  of the relationships, the effects of pupil characteristics and educational interventions on the relationships, and how the relationships varied across the schools.  The results of the analyses indicated the  kindergarten screen/achievement relationships did not vary, however, the average achievement of pupils who scored at the cut-off point f o r r i s k status varied s i g n i f i c a n t l y across schools, even after controlling f o r pupil characteristics and the effects of educational  interventions.  The c o l l e c t i v e properties of a school which have an effect on individual pupil achievement over and above characteristics or attributes the pupils bring to the learning situation are called contextual effects (Willms, 1986).  Researchers have shown that school mean-ability has an  effect on pupils' academic achievement, even after controlling f o r the individual effects of pupils' family background (Summers & Wolfe, 1977; Henderson, Mieszkowski & Sauvageau, 1978; Brookover et a l . , 1978; Rutter, et a l . , 1979; Willms, 1986).  Willms and Raudenbush (1989) noted that i n  studies which f a i l to include a wide range of variables that describe school p o l i c i e s and practices, an aggregate variable of pupil-level characteristics may act as a proxy f o r variables describing p o l i c i e s and practices.  In this study, school mean-ability may act as a proxy f o r  other variables not available for investigation. Model IV includes two school-level variables, school mean-ability modelled only on the intercepts and a t t r i t i o n modelled on the kindergarten screening/achievement relationship slope.  During  117  preliminary analysis school mean-ability was modelled on the slope but was found not to be s i g n i f i c a n t .  There were no between-school  differences i n the kindergarten screening/achievement r e l a t i o n s h i p s , and therefore, school mean-ability was not modelled on the slopes i n the f i n a l model.  The a t t r i t i o n variable remained i n the model because i t had  a small negative effect on the slope.  As the primary interest of t h i s  study i s the kindergarten screening/achievement relationship (slope), i t was appropriate to include this negative effect which represented the a t t r i t i o n b i a s , even though the effects were not s i g n i f i c a n t . The age and gender effects declined minimally, or remained the same as in Model I I I . The age effect remained small but s i g n i f i c a n t f o r Mathematics and Language on a l l four screening measures and f o r Vocabulary with DEVTOT and MS. .23**.  The range of the age effects was .08 to  The gender effect remained significant for Language on a l l four  screening measures and for Reading on DEVTOT and MS.  The range of the  gender effects was -.63 to 2.32**. These effects may be interpreted as described i n Model I I . The estimated coefficients for the effects of interventions remained s i g n i f i c a n t , large and negative across the 16 regressions. The range of the c o e f f i c i e n t s for extended primary was -2.25* to -6.65**. The range of the coefficients for learning assistance was -4.33** to -5.81**. These c o e f f i c i e n t s also did not change appreciably, and may be interpreted as described in Model I I I . The estimated coefficients for school mean-ability modelled on achievement are s t a t i s t i c a l l y significant across a l l models. of the coefficients i s .30** to .46**, with the median .34**.  The range This  118  suggests that pupils in schools with higher school mean-ability had higher achievement scores than similar pupils i n schools with low school mean-ability. After controlling for school mean-ability, the estimated parameters of variance for mean achievement for pupils at the cut-off point for risk status remained significant for half the models:  reading on DEVTOT  (3.48*); mathematics on DAP (3.76**), KLST (5.36*), and DEVTOT (3.58**); language on DAP (6.57**), KLST (6.40**), DEVTOT (3.48*), and MS (7.77**).  The estimated parameter variance for the remaining models  declined to a level which could have occurred by chance.  This means that  differences between schools i n the mean achievement of pupils at risk on certain kindergarten screening measures was lowered by controlling f o r school mean-ability for eight of the models.  This suggests that pupils  who scored at the cut-off score for risk status earned higher achievement scores on average i n schools with high school mean-ability than similar pupils in schools with low school mean-ability.  For example, the mean  language achievement of pupils "at risk" within the highest school was about f i v e months of schooling ahead of the mean achievement of similar pupils in the lowest school, even after controlling for school meanability. Parameter Variance Explained The estimated variance i n grade three achievement at the pupil-and school-levels, based on a " n u l l " model ( i . e . without any pupil or schoollevel variables i n the model), are as follows:  119  Table 6 "Null-Model" Estimates of Variance in Grade Three Achievement Pupil-Level  School-Level  Total  OLS  Reading  69.13  5.49  =  74.62  74.13  Mathematics  50.89  4.28  =  55.17  54.76  Vocabulary  63.69  3.58  =  67.27  66.75  Language  62.64  7.85  =  70.49  69.39  Thus about 3 to 8 percent of the variance in achievement i s between schools. R-squared values for the various models can be expressed of ways.  i n a number  One s t a t i s t i c of interest i s the proportion of the total  variance explained by the inclusion of a l l variables in the model; which 2  2  I w i l l denote R T:  RT =  Var ( Y ) - 0  2  Var (Y) where o  i s the maximum likelihood estimate of the residuals.  2  Estimates  of the proportion of (null model) pupil-level variance explained by pupil-level variables, and (null model) school-level variance explained by school-level variables, are also of interest. 2  I w i l l represent them  2  as R p and R S: r 2  = p  °a  ~ a  2  r  2 0  = s  °0  where o  2  To_Z_L 1  0  and T 0 are the estimates of pupil- and school-level variances  for the null model, and 0  2  and T are estimates of variance after  inclusion of relevant variables (e.g. Zuzovsky & A i t k i n , in press). In models where a l l pupil-level variables are f i x e d , these estimates are  120  straight forward.  However, when within-school regression slopes are  allowed to vary, the covariance between school-level variables and 2  2  within-school slopes complicates the estimates of both R p and R S. 2  some cases, T > T 0 , yielding a negative value of R g.  In  In this study the  2  principal measures of interest are R T f o r the models that include only 2  the kindergarten screening measures, and R S f o r the Model IV regressions, which include school-level variables.  The Appendix Tables  include the proportion of total variance explained f o r a l l models and the proportion of school-level variances explained for adjusted levels of achievement f o r Model IV. The range of pupil-level variance explained by Model I i s 7.47% to 20.47%.  2  2  These R are s l i g h t l y larger than the R observed using simple  OLS regression which would be equivalent to the squares of the intercorrelations between the screening measures and outcome measures shown in Table 1 (which range from 5.48% to 18.84%). Model V Four Kindergarten Screening Measures in the Model The f i f t h model allows f o r the examination of whether the relationships between achievement outcome measures and kindergarten screening measures change when a l l four kindergarten screening measures are included in the model. Appendix Table 19 presents the results of the regressions f o r each of the four outcome measures on a l l of the kindergarten screening measures, controlling f o r the other variables which have been introduced in prior models. The estimated coefficients for the average within-school relationships of outcomes on kindergarten screening scores are lower with a l l four  121  kindergarten measures as covariates i n the model than when only one kindergarten screening measure i s included.  The coefficients are lower  because of the inter-correlations amongst the screening measures; the predictive power of the screening measures, i n a sense, i s shared across the four measures. The average within-school relationships of outcomes on kindergarten screening scores ranged from .09 to .30** for DAP, .34** to  .55** for KLST, .18** to .23** for the Deverell and .26 to 1.10** f o r  the Mann-Suiter.  The estimated coefficients for a l l four outcome  measures on the KLST and DEVTOT remained s i g n i f i c a n t .  The estimated  c o e f f i c i e n t s for reading, vocabulary and language were s i g n i f i c a n t on DAP while only mathematics and language were significant on the Mann-Suiter. This means that thirteen of the sixteen relationships between achievement and each kindergarten screening measure remained positive and s t a t i s t i c a l l y s i g n i f i c a n t , even after controlling for a l l other variables and the other kindergarten screening measures. The average within-school achievement of pupils who obtained scores on a l l four kindergarten screening measures at the cut-off point for r i s k status are a l l lower than for models including only one kindergarten screening measure. Table 7 presents the mean achievement scores f o r the d i s t r i c t , for pupils who scored at the cut-off point for risk status on one kindergarten screening measure and for pupils who scored at the cutoff score on a l l four kindergarten screening measures.  122  Table 7 Average Achievement Scores f o r Pupils Who Scored at the Cut-off Point for "Risk" Status  Study Sample  Reading  Mathematics  Vocabulary  Language  41.45  41.67  41.52  43.89  K-Screen Cut-off Point Mean DAP  37.83  39.16  38.16  39.85  KLST  37.21  38.06  37.06  39.45  DEVTOT  37.72  38.17  37.98  39.36  MS  39.55  39.71  39.78  41.29  at cut-off point 34.56  36.58  34.81  36.98  Four K-Screen  An examination of Table 7 indicates that pupils who obtained scores at the cut-off point f o r risk status on a l l four kindergarten screening measures were from five to seven months of schooling behind the average pupil after controlling f o r pupil c h a r a c t e r i s t i c s , educational interventions and contextual effects of school mean a b i l i t y . With a l l four kindergarten screening measures in the model, no age effects reached a level of significance.  The estimated c o e f f i c i e n t s of  the gender effect f o r three outcomes were not s i g n i f i c a n t , but the gender effect for language remained significant at 1.27**. Thus, by grade three, females achieve on average, more than one month higher than males, even after controlling f o r the kindergarten screening scores, pupil c h a r a c t e r i s t i c s , educational interventions and school mean- a b i l i t y . The estimated effects of the educational interventions remained  123  negative; six of the eight coefficients are s t a t i s t i c a l l y s i g n i f i c a n t . The effect of extended primary for mathematics was -2.21** and f o r language -3.59**.  The range of the effects for receiving learning  assistance was from -4.04** to -4.86**. An extended primary effect of -3.59  means that a pupil who obtained four kindergarten screening scores  at the cut-off point for risk status and who attended 49 months of primary school, scored on average, 3.59 months of schooling below pupils with similar kindergarten screening scores but who had attended only 39 months of primary school.  Similarly the pupils at risk on four  kindergarten screening measures who received learning assistance achieved, on average, four and a half months of schooling below similar students who did not receive assistance. This finding again  suggests  that pupils who obtained scores at the cut-off for risk status, on average, who received special education interventions had lower achievement scores, even after controlling for their pupil c h a r a c t e r i s t i c s , gender and physical problems, and school mean-ability than did their peers who obtained similar screening scores but who did not receive special education interventions. The estimated coefficients f o r school mean-ability modelled on achievement were s t a t i s t i c a l l y significant across a l l models.  The range  of the estimated coefficients i s .30** to .42**. This suggests that pupils in schools with higher school mean-ability had higher achievement scores than similar pupils in schools with low school mean-ability. Pupils who scored at risk on the four measures and attended a low a b i l i t y school would be about six to nine months behind their peers who attend a high a b i l i t y school.  The estimated variance of mean achievement f o r  124  pupils at the cut-off score f o r risk status on a l l four kindergarten screening measures, remained significant for reading (4.86*), mathematics (9.52**), and language (10.17**). Model VI Simplified Models Including Only Significant Variables Appendix Table 20 presents the results of regressions of the four outcome measures modelled on the four kindergarten screening measures and other variables which had a significant effect in Model V.  The f i n a l  model estimates reported indicate there are only minor differences in the estimates when compared with Model V.  There are no substantive changes  which require presentation. Summary This chapter has presented the findings of six HLM regression models.  The discussion of the findings and recommendations f o r further  research are presented in Chapter 5.  125  Chapter 5 Summary and  Conclusions  The f i n a l chapter presents an overview of the study, conclusions and discussion of the findings for the research questions addressed i n the study.  It also presents implications for the study and suggestions for  future research.  This study f i l l s a gap in the l i t e r a t u r e on  prediction-performance research by examining the relationships between kindergarten screening measures and achievement outcome measures in a hierarchical model.  The analysis includes control for important pupil  characteristics and the effects of educational interventions during the time of the study. Overview of the Study This study examines the relationships between kindergarten screening measures and grade three achievement in reading, mathematics, vocabulary and language for two entire age cohorts enrolled in 30 schools in one school d i s t r i c t .  The analysis employs a two-level hierarchical  linear  model to estimate the average within-school relationship between kindergarten screening measures and achievement.  It also determines  whether s i g n i f i c a n t differences exist between the 30 elementary schools i n the relationships between the screening measures and the achievement measures, and examines the extent to which educational interventions mediate the relationships. The intention of the screening process i n this school d i s t r i c t was  to  identify children with handicaps or developmental delays. The presence of handicaps or developmental delays were considered to indicate the need f o r special education programming for the pupil's optimal educational  126  progress.  The d i s t r i c t implemented kindergarten screening in the b e l i e f  the findings would result i n : e a r l i e r i d e n t i f i c a t i o n of pupils "at r i s k " of experiencing d i f f i c u l t y learning; and provision of remedial programming which would a l l e v i a t e or eliminate the d i f f i c u l t i e s . I expected that educational interventions provided for pupils designated "at risk" would improve their achievement.  The effect of  improving achievement of pupils designated "at risk" would lower the correlation between the screening score and subsequent achievement. would "mask" the i n i t i a l prediction of r i s k .  This  I also expected that schools  might allocate resources d i f f e r e n t i a l l y , and thus, some schools might be more effective than others in bolstering the achievement of at risk students. These achievement gains would have the effect of lowering the relationship between the screening measure and achievement in some schools which would result in the relationships varying between schools. Principal Findings of the Study 1.  Positive relationships were found for a l l screening measures with a l l  outcome measures. The findings of this study support that positive relationships exist between these kindergarten screening measures and grade three achievement. The relationships between screening scores and achievement scores varied across screening measures.  In other words, these particular kindergarten  screening measures were moderate predictors of subsequent achievement i n particular s k i l l s during grade three.  The best predictors for reading,  vocabulary and language, language related achievement measures, were the KLST and DEVTOT, also language related measures.  The strongest predictors  for mathematics were visual-motor a b i l i t y and expressive language.  127  The strong relationship between language a b i l i t y (KLST) and language related achievement scores i s not surprising.  The assumption that  proficiency i n oral language underlies academic achievement i s l o g i c a l . Most primary curriculum materials rely heavily on s k i l l i n oral language. The recent interest i n "whole language" approach to reading attests to the fact that many educators believe oral language i s essential f o r academic success. Recent research has questioned the strength of the relationships between oral language and achievement (Hammill & McNutt, 1980; Gray, Saski, McEntire & Larson, 1980).  Hammill and McNutt (1980) conducted a review of  l i t e r a t u r e which synthesized the results of 89 correlational  studies.  Studies were selected which i l l u s t r a t e d the common belief that a child must have adequate oral language to learn to read. Hammill and McNutt concluded that oral expressive language i s not related to reading performance although they suggested that some aspects of oral receptive language are minimally related. The results of the present study controvert the findings of Hammill and McNutt (1980). There are several possible explanations for the contradictory findings including:  different language tests may measure  different aspects of language; tests administered to pupils at d i f f e r e n t ages may result i n contradictions  as acquisition of language s k i l l s i s  uneven in young children; language tests are correlated with measures of intelligence and there may be confounding effects when intelligence level i s not included as a covariate. The findings of this study suggest there i s a strong positive relationship between performance on a measure of oral language in  128  kindergarten and achievement in grade three.  This may i l l u s t r a t e that  proficiency i n oral language i s important for enabling children to interact with the school environment.  Oral language proficiency allows f o r  opportunities to interact with the teacher and classmates which may f a c i l i t a t e further language development.  Proficiency in oral language may  r e f l e c t underlying a b i l i t i e s to perform with curriculum materials which rely heavily on language s k i l l s i n the primary grades, and thus the language measure would be an appropriate measure f o r identifying  students  at risk of experiencing d i f f i c u l t y learning academic s k i l l s i n school. The strong relationship between achievement in mathematics and v i s u a l motor a b i l i t y i s consistent with prior research in which visual-spatial a b i l i t i e s have correlated highly with performance i n mathematics. The strong relationship between mathematics and oral language may be a r e f l e c t i o n of the heavy emphasis on language in the presentation of basic mathematical concepts i n primary grades. I hypothesized that the kindergarten screen/achievement relationships would vary across schools as I expected that schools would allocate resources d i f f e r e n t l y , and thus, some schools would be more successful in bolstering the achievement of low a b i l i t y pupils.  The findings for only  one of the 16 screening/outcome relationships lends support f o r t h i s hypothesis.  The relationships f o r language s k i l l s with the kindergarten  measure of cognitive a b i l i t y varied s i g n i f i c a n t l y across schools.  The  s k i l l s measured by the language test include s p e l l i n g , grammar and punctuation; s k i l l s which are highly dependent upon school learning. This finding indicates that some schools were better at developing  skills  measured by the language test for pupils who obtained similar scores on the  129  test of cognitive a b i l i t y (DAP) than other schools. the findings do not support the hypothesis.  In general, however,  The strength of the  relationships varied across the achievement measures with each of the kindergarten screening measures. When a l l four screening measures are included in the analysis as covariates, most of the estimated coefficients for a l l four achievement outcomes with the screening measures remain s i g n i f i c a n t .  The exceptions  are mathematics with the test of cognitive a b i l i t y and reading and vocabulary with the visual-motor t e s t . 2. The kindergarten screen/achievement relationships declined minimally, but remained s i g n i f i c a n t , after controlling for the effects of individual pupil characteristics of age, gender and physical problems. The addition of variables for student characteristics of age, gender and physical problems resulted in the kindergarten screen/achievement relationships declining minimally across a l l measures.  The kindergarten  screening relationships remained s t a t i s t i c a l l y s i g n i f i c a n t .  The hypothesis  that the average within-school relationships would be weaker after controlling for pupil characteristics was supported.  The second hypothesis  was supported only by the decline in the one relationship which was significant (DAP/Language). a) Effects of age-at-entry. Prior to controlling for the effects of educational interventions, the effects of age-at-entry were significant for mathematics and language with a l l four screening measures and for vocabulary with the tests of knowledge of l e t t e r s and numbers and visual-motor a b i l i t i e s .  After controlling f o r  the interventions, the effects of age-at-entry were not s i g n i f i c a n t f o r any  130  of the achievement outcome measures. There were significant age effects for mathematics and language with a l l four screening measures and for vocabulary with the test of l e t t e r s and numbers and the visual motor t e s t .  This finding is consistent with  research which consistently supports that older students perform better in early grades than younger children (Davis, Trumble & Vincent, 1980). finding may  This  have implications for making a decision regarding entry to  school or providing educational intervention. Gredler (1980) pointed out that older children arrive at school knowing more or having more than younger children. for learning and may  experienced  In some ways, therefore, they are more "ready"  appear to learn more i f achievement at a single point  in time i s the c r i t e r i o n .  The difference in achievement between younger  and older students apparent at the grade three level declines continuously until i t is no longer distinguishable by grade eight (Davis, Trimble & Vincent, 1980). b) Effects of gender. Gender may  be i n f l u e n t i a l in the prediction of risk status for  learning academic s k i l l s with boys appearing to be more vulnerable than girls.  Before controlling for educational interventions, gender effects  favoring g i r l s were significant for reading and language with a l l four screening measures.  The gender effects were particularly large for the  relationships between language achievement - s p e l l i n g , grammar and punctuation, and kindergarten screening measures.  This finding i s  consistent with prior research which suggests that, generally speaking, g i r l s perform better than boys of the same chronological age on readiness tests and on later achievement (Beattie, 1970).  131  When considering an individual pupil's score on a kindergarten screening measure, both the chronological age and gender of the c h i l d should be given consideration. an older child may  It is possible that an "at r i s k " score for  be more meaningful than for a younger c h i l d ,  c) Effects of physical problems. The effects of physical problems were not significant for any kindergarten screen/achievement models. 3.  The effects of the educational  interventions of extended primary and  learning assistance were large, significant and negative for a l l achievement outcomes with a l l kindergarten screening measures. One can imagine similar analyses for investigating the effects of educational  interventions.  In such an analysis, the kindergarten  screening  measures would serve as control variables to adjust for differences between pupils who  did or did not receive an intervention.  The  variables  describing interventions used in this study would be inadequate for that purpose.  If the primary interest of a study was  to examine the effects of  different intervention approaches or to examine the benefits for subjects within categorical groups, one would need to c o l l e c t information  such as  the location of the intervention, time actively participating or duration of remedial i n s t r u c t i o n . If the investigation of the effects of the interventions was  the purpose of a study, one might estimate the likelihood  of being placed in a remedial program, given the pupil's kindergarten screen score. In this study, the primary interest is the kindergarten screen/ achievement relationship and the extent to which educational mediate the relationship.  For this reason, the educational  interventions interventions  132  were used only as control variables and are adequate for that purpose. The hypothesis that educational interventions would mediate the kindergarten screen/achievement relationships was not supported; in f a c t , the effects were in the opposite d i r e c t i o n .  I hypothesized that the  relationship would be stronger after controlling for the effects of learning assistance and retention because I expected that the interventions would be provided to students obtaining "at risk" scores on screening measures and would improve their subsequent achievement. There are at least three possible explanations for the large, s i g n i f i c a n t , negative e f f e c t s .  One i s that the interventions were  i n e f f e c t i v e ; that i s , pupils who attended the special programs progressed at a slower rate than pupils with comparable a b i l i t y in the regular program or the interventions were ineffective in developing the s k i l l s measured by the CTBS. Additionally, one could interpret that the interventions were detrimental to pupil progress and the effects were to help " f u l f i l l prophecy" made by the screening measures.  the  Common educational practices  such as decreasing the pace of instruction, grouping by a b i l i t y , or lowered teacher expectations may  have contributed to lower achievement scores for  pupils participating in interventions (Slavin, 1987; Peterson, 1989). The remaining two explanations concern model s p e c i f i c a t i o n .  One  p o s s i b i l i t y i s that pupils were assigned to special programs on the basis of low screening scores, but progress thereafter depended mainly on other f a c t o r s , such as family socioeconomic status, that were not included in the model.  The other p o s s i b i l i t y i s that the assignment of pupils was  not  based solely on the screening information.  Other f a c t o r s , such as pupil  behavior, attention or work completion, may  have played a key role in these  133 decisions.  Also, opportunities to receive assistance may  have been  provided later in the primary grades for some pupils with the decision based on actual achievement rather than on a b i l i t y .  The model did not  include variables which represented these factors. Scatterplots of the achievement data against the screening data suggest that the latter explanation i s plausible; there were many pupils with low screening scores who did not receive intervention, and several with high screening scores who did (Appendices J & K). Based on this study i t would be inappropriate to suggest that the kindergarten screening was  ineffective.  The purpose of the screening  was  to identify students for whom educational interventions would be required for optimal progress.  One hundred and twenty pupils who  participated in  screening were in special classes and their achievement in grade three not measured by CTBS testing.  was  Inclusion of this group in the data might  have improved the correlations somewhat as these children were s i g n i f i c a n t l y at risk and required intensive interventions to make educational progress, their kindergarten screening and subsequent achievement scores probably would have been low. The hierarchical model used in this study provides a means to study variation between schools in the effects and application of educational interventions.  Most of the effects of interventions did not vary  s i g n i f i c a n t l y across schools.  The exceptions are for Learning Assistance  on the relationships between reading and measures of cognitive a b i l i t y , language and letters and numbers, and relationships between reading and the visual motor test; and Extended Primary Schooling on the relationship between language achievement with kindergarten oral language and reading  134  with the test of letters and numbers.  These findings may indicate that  interventions are requested d i f f e r e n t i a l l y in response to screening information or to other variables, the interventions are d i f f e r e n t i a l l y e f f e c t i v e , or a selection process contributed to the effects p a r t i c u l a r l y where the achievement measure i s reading.  A more in-depth study of the  individual interventions might c l a r i f y the between-school variation i n these e f f e c t s . 4.  The adjusted achievement levels of pupils "at risk" varied  s i g n i f i c a n t l y among schools. The adjusted achievement levels of pupils who obtained scores on one kindergarten screening measure at the cut-off point for risk status were lower than the d i s t r i c t mean performance but not lower than the average expected performance for the grade based on the test norms.  However, the  adjusted achievement levels for pupils who obtained scores on four kindergarten screening measures at the cut-off point were considerably below expectancy for the grade placement. The mean adjusted achievement levels of pupils who obtained scores on one kindergarten screening measure at the cut-off point for r i s k status varied s i g n i f i c a n t l y across schools for language with a l l four kindergarten screening measures, for mathematics with kindergarten tests of cognitive a b i l i t y , expressive language and letters and numbers and for reading with the test of letters and numbers. This study did not attempt to explain why the achievement of pupils designated "at risk" varied s i g n i f i c a n t l y across schools; however, the model could be extended to include variables describing school policy and practice that might explain these differences.  For example, the inclusion of variables such as the  135  performance on tests during the primary years, heterogeneity of c l a s s e s , or teacher observations of behaviors, might i l l u s t r a t e p o l i c i e s and practices which d i f f e r between schools.  Qualitative methods could then be employed  for intensive study of particularly effective or ineffective schools. 5.  Pupils who  scored "at risk" on kindergarten screening measures  performed better on a l l four achievement measures in schools with high school mean-ability than similar pupils in schools with low school meanability. Although i t was not the primary purpose of this study, school-level variables were added to the model in an attempt to explain the between-school differences in the average achievement of pupil's designated at r i s k .  Preliminary analysis determined that school s i z e , geographic  location and heterogeneity did not contribute s i g n i f i c a n t l y to either the achievement differences (intercept) or the kindergarten screen/achievement relationships (slopes). School mean-ability was found to have a positive effect on the within-school average achievement across a l l models. pupil's who  This indicated that  scored "at risk" on kindergarten screening measures performed  better on a l l four achievement measures in schools with high school mean-ability than similar pupils in schools with low school mean-ability. Because this study did not include variables for school or home processes which may contribute to differences in achievement, school mean-ability may  act as a proxy for many other variables.  This study did  not attempt to identify variables which may be represented by school mean-ability but the amount of variance explained by school mean-ability suggest a strong argument for contextual e f f e c t s .  136  6.  The kindergarten screen/achievement relationships were not improved by  controlling for the effects of interventions. A major interest of this study was the application of a s t a t i s t i c a l procedure which would simultaneously analyze the data at two l e v e l s .  I  hypothesized that a major contributor to low correlations between screening measures and achievement outcomes was f a i l u r e on the part of researchers to control for important student c h a r a c t e r i s t i c s , the effects of interventions and between school differences.  Although this study found s i g n i f i c a n t  positive effects of student characteristics and significant achievement differences between schools, i t f a i l e d to demonstrate that controlling f o r interventions would improve the kindergarten screen/achievement relationships. One possible explanation i s that the choice of kindergarten screening measures and their relationships with subsequent achievement was not based on theory.  In other words, the specific s k i l l s and a b i l i t i e s measured by  the kindergarten measures may  not be the most important prerequisites for  the development of the s k i l l s measured on the achievement measures. Another possible explanation i s that these particular measures were not intended to be interpreted i n d i v i d u a l l y .  The intent of administering  four measures at different time points across the school year was  to  identify and intervene for children scoring "at risk" after each measure. Referral for more intensive diagnostic assessment was deemed appropriate only after a child scored at risk on two or more screening measures. composite model demonstrated that pupils who  The  scored at risk on a l l four  measures and were administered the grade three CTBS, obtained, on average, much lower achievement scores than students designated at risk on  137  individual measures. This supports an argument for on-going assessment. Pupils who  scored low on measures of different a b i l i t i e s over time were  more l i k e l y to have d i f f i c u l t y or achieve at lower levels than pupils  who  were low on any one measure. A consideration in examining predictive v a l i d i t y i s that d i f f e r e n t analysis may  be appropriate  for different purposes.  Predictive v a l i d i t y  which examines the relationship of screening measures to subsequent achievement across the entire d i s t r i b u t i o n of a b i l i t i e s may  be p a r t i c u l a r l y  valuable for developing theories or for understanding differences in performances for pupils with various levels of a b i l i t y .  The addition of  controlling variables provides important information for differences in performance related to age and gender.  By d e f i n i t i o n , predictive v a l i d i t y  implies time between the predictive measure and the outcome measure; i t i s important to control for important variables which may  influence the  outcomes. The purpose for screening  always entails intervention, therefore,  controls for the effects of interventions are essential to understand the kindergarten screen/achievement relationships.  If the large, negative  effects of interventions found in this study are generalizable to other studies which did not control for interventions, the previously  reported  low correlations between kindergarten screening and achievement may  have  been inflated rather than deflated. A second consideration involves predictive u t i l i t y . which choose to implement a screening program may  School d i s t r i c t s  be more interested in  data analysis which i d e n t i f i e s the proportion of pupils correctly i d e n t i f i e d as "at risk" and the proportion of individuals "not at r i s k "  who  138  are correctly excluded from further assessment or intervention.  Test  s e n s i t i v i t y i s the proportion of pupils with special needs who are i d e n t i f i e d accurately, and s p e c i f i c i t y i s the proportion of pupils not in need of special services whose scores are above the cut-off score on the screening measure.  The proportions of the correct c l a s s i f i c a t i o n s are  inversely r e l a t e d . That i s , by adjusting cut-off scores, the proportion of pupils identified as "at risk" can be increased  although, some of the  pupils are not truly "at risk" of school d i f f i c u l t i e s .  By r a i s i n g the cut-  off score, more pupils w i l l require interventions, thus some pupils not i n need of special services w i l l receive them. results i n a decrease i n the other.  An increase i n one group  Prediction-performance matrices and  c l a s s i f i c a t i o n a l analysis are used to i l l u s t r a t e numerical data and calculate s e n s i t i v i t y and s p e c i f i c i t y . and  Provision of intensive assessment  intervention have cost implications, therefore, t h i s type of  information  could be valuable i n the decision f o r selecting cut-off scores  and planning budgets f o r services. Appendix Table 22 presents the data used i n this study reported in proportions such as those presented i n prediction-performance matrices. Extension of the analysis to identify the proportions of young children and representation  by sex i n the group c l a s s i f i e d as "at r i s k " , i l l u s t r a t e s how  these variables could be considered using this technique.  The most  interesting finding relates to the proportion of pupils who obtained scores within the "at risk" category who received learning assistance or attended extended primary. The overall "hit rate" of these measures was as high as expected from the research of kindergarten prediction, 57 to 75 percent.  The proportion  139  of pupils scoring at risk on screening measures who participated in an educational intervention ranged from 12 percent to 49 percent.  In other  words, half of the pupils who were identified "at risk" on a kindergarten screening measure were not provided with an educational intervention intended to have a remedial effect on their performance. Males made up the highest proportion of the "at risk" group (51-71 percent across models) and young pupils were over-represented cases (50-64 percent).  i n several  These figures lead to some interesting questions  regarding how, when and why teachers respond to kindergarten screening measure r e s u l t s :  What response was provided to a pupil achieving an "at  r i s k " score on a screening measure?  When did a response to an "at r i s k "  score result in referral for an intervention?  What factors determined that  a pupil would participate i n learning assistance or extended primary? If the large, negative effects of educational intervention found i n this study indicate that pupils identified at risk attain higher achievement scores i f they do not participate i n learning assistance or extended primary, the findings could have significant financial implications for school d i s t r i c t s . Appendix K presents sample graphic representations of the predictive u t i l i t y of a kindergarten screening measure with an outcome measure. The cut-off score used for this study i s indicated on the graph.  By adjusting  the cut-off score up or down, the proportion of correct and incorrect decisions can be i l l u s t r a t e d .  Or, in other words, the proportion of pupils  i d e n t i f i e d as requiring remedial intervention services can be increased or decreased by adjusting the cut-off score of the screening measure.  As a l l  measurement contains error, i t can be expected that some pupils w i l l always  140  be m i s c l a s s i f i e d . The decision faced by school d i s t r i c t s selecting cut-off scores i s , which "misclassified" group w i l l be the largest.  Will services  be provided to a larger group which includes some pupils not t r u l y in need, or w i l l services be provided to a smaller group, denying some pupils services who are truly in need? In light of the findings of this study, the more important decisions may be whether to provide interventions, and i f so, what kind of interventions and for whom. Limitations of the Study The major limitation of this study i s that the prediction measures and the outcome measures are limited to results of standardized instruments. The appropriate use of standardized tests with young children has recently been under close scrutiny by educators and other professionals.  The  findings of t h i s study indicate positive relationships exist between these kindergarten screening measures and the achievement outcomes. The NAEYC (1988) recommended that, "The purpose of testing must be to improve services f o r children and ensure that children benefit from t h e i r educational experiences" (p.44).  Emphasis i s placed on standardized tests  as only one of multiple sources of assessment information  that should be  used when decisions are made about what i s best for young c h i l d r e n . Perhaps, the major problem with predictive v a l i d i t y studies has been f a i l u r e to include, in addition to standardized test r e s u l t s , measures of the many possible sources which influence academic performance over time. There also are technical limitations of the study related to the use of extant data.  The kindergarten screening measures and achievement  measures were limited to those instruments previously selected by the  141  d i s t r i c t f o r administration. Academic outcomes were limited to the results of the standardized measures.  It was not possible to examine p o l i c i e s and  procedures which may have been influential in the outcomes of the study. Implications of the Study The findings of this study have implications f o r kindergarten screening programs, f o r instructional practices, for provision of special services and f o r predictive v a l i d i t y studies. 1.  Norm-referenced standardized tests are appropriate f o r inclusion in a  kindergarten screening program. The positive relationships between kindergarten screening measures and achievement measures demonstrates that the use of standardized test measures can be valuable in a screening program. Many teachers do not l i k e to use standardized test measures because the results may be misinterpreted or misused.  Following a review of the literature on teacher judgement,  Hoge and Coladarci (1989) reported that teacher judgement about student achievement i s generally accurate but levels of accuracy vary across teachers.  A combination of standardized test measures with reports of  teacher judgment would l i k e l y improve the assessment process. The results of the analysis suggest that the administration of a single measure or a one time assessment i s inadequate f o r making decisions about an individual p u p i l .  Students with comparable screening scores may  have different needs not identified by the screening measures. Placement and programming should be determined only after comprehensive data i s collected.  A more appropriate program would result i n a comprehensive  p r o f i l e of pupil a b i l i t i e s and s k i l l s would include: a) a battery of screening measures which have high predictive v a l i d i t y  142  for particular outcomes; b) knowledge of the SES area or mean a b i l i t y of the school attended; c) consideration for d i f f e r e n t i a l effects on performance of age, gender and physical problems; d) teacher observation and checklists; e) on-going assessment:  kindergarten measures administered across the  school year and possibly readministered to pupils of p a r t i c u l a r chronological age;  outcome measures administered on several occasions  such as, the middle of grade one, and the end of grade one, which would allow for interventions at the time the need presents. 2.  Educators should monitor the effects of remedial programs to determine  what i s appropriate, for whom, when and for how  long.  The large negative effects of the interventions found in t h i s study have implications for the provision of special services.  If receiving  remedial help or progressing more slowly through the curriculum have the result of lowering achievement below that of pupils who remain in the classroom without assistance, the continued provision of special services would be questionable.  The financial expenditures required to provide  services are defensible only i f the outcomes are p o s i t i v e . A l t e r n a t i v e l y , i f the provision of special services had a positive effect which was not measured by standardized test measures, i t would be desirable to have documentation that the benefits to the pupils are worth the costs. 3.  The examination of policies and practices in schools which are most  effective should guide the development of such practices in schools which are less e f f e c t i v e .  143  The significant differences in mean adjusted achievement between schools for pupils who obtained scores at the cut-off point for r i s k status indicates some schools are more effective for pupils at risk than other schools.  The determination of policies and practices which make some  schools more effective could guide the development of similar practices i n the less effective schools. 4.  Prediction-performance  research should incorporate analysis of student-  and school-level variables to accurately r e f l e c t the hierarchical of  nature  education. The findings of this study have implications for future predictive  v a l i d i t y studies.  Examination of the models i l l u s t r a t e s that strong  positive relationships were consistently found for achievement with kindergarten screening measures.  However, controlling f o r important  variables changed the significance of the effects of other variables. For example, age-at-entry and gender were found to be significant across several measures when only pupil characteristics were included in the analysis.  When the effects of educational interventions were c o n t r o l l e d ,  age and gender effects were significant i n fewer models.  Also, when  including four kindergarten screening measures in the analysis, only the gender effect on language was s i g n i f i c a n t .  This i l l u s t r a t e s how analyses  which f a i l to consider these variables, or perhaps other variables, may report age or gender effects which are inaccurate. The significant effects of school mean-ability i l l u s t r a t e the importance of considering both within-school relationships and betweenschool relationships.  Analyses which use small subject samples or which  f a i l to consider contextual effects of schools may f a i l to identify  144  measures which are good predictors or a l t e r n a t i v e l y , measures which appear to be predictive but are weak predictors when the effects of other variables are considered. Recommendations for Future Research Suggestions for research derived from the findings of this study include: 1.  Examination of the relationship between s p e c i f i c language s k i l l s and  achievement i s appropriate to identify specific language s k i l l s which may be predictive of achievement. The relationships between the kindergarten screening measure of expressive language with the outcome measures was r e l a t i v e l y strong i n the present study.  Researchers have reported contradictory findings.  Research  studies could be directed at identifying specific receptive and expressive language s k i l l s which have significant effects on different areas of achievement.  For example, tests of receptive and expressive language  s k i l l s for pupils of different chronological  ages could be compared with  growth rates in language-related areas of reading, vocabulary and written language s k i l l s .  Hierarchical linear modelling would allow for the  analysis of growth rates and the analysis of between pupil differences in the rates.  Multivariate analysis of individual and combinations of s k i l l s  with achievement measures could identify the best language predictors. Greater understanding of these relationships could have strong implications for trends such as teaching reading through the "whole language" approach which i s premised on a relationship between language and reading which has yet to be unequivocally proven. 2.  The qualitative effects of particular interventions should be examined  145  to ensure pupils identified as "at risk" are provided optimal opportunities to progress. The research designs for examining educational interventions are numerous; the access to data to complete such research i s more d i f f i c u l t . Experimental studies in which treatment and control subjects are matched for age, gender, handicaps and school enrollment would be valuable to gain greater understanding of the effects of interventions.  Caution would be  required as denial of a particular intervention for an identified "at r i s k " pupil would be unethical.  However, autonomy granted teachers, schools or  d i s t r i c t s in the selection of particular approaches to intervention should allow f o r investigation of various interventions and their e f f e c t s . Study of matched groups of subjects for whom the school recommends intervention and the parent refuses to accept the intervention would also be of i n t e r e s t .  However, i t would be important to investigate variables  r e f l e c t i n g family processes, ethnic background, SES, which might have an effect on the decision and which might vary between parents who accept the intervention and those who 3.  refuse.  An examination of child variables which result in retention and the  decision making processes of teachers and parents which lead to retention could be conducted through interviews and questionnaires. Ten percent of the pupils in the achieved sample attended extended primary, although half or more of those pupils who remained in the primary grades an extra year scored above the cut-off score for "at r i s k " status. It i s clear that the decision to extend the pupil's primary schooling was based on information other than screening information in many cases. Remaining an extra year in school i s costly both to the pupil i n earnings  146  at the completion of school and to d i s t r i c t s which fund the additional year of school.  Research on retention consistently reports negative effects on  the child's social-emotional  status and minimal benefits to achievement  status. Examination of social-emotional  or behaviorial factors of the pupils  which contribute to the decision to provide or withhold interventions be investigated through observational  study or  could  pupil-teacher-parent  questionnaires. 4.  The examination of home and school processes which lead to pupils  performing better in some schools than pupils of similar a b i l i t y in other schools could be undertaken. The  large adjusted average achievement differences between schools for  children who  obtain a score "at risk" i l l u s t r a t e the importance of  examining performance at both the within-pupil and the between-school levels.  While i t i s not possible to model a l l possible v a r i a b l e s , theory  driven investigation into these processes or experimental investigation might identify the important manipulable variables which improve performance.  The a v a i l a b i l i t y  of s t a t i s t i c a l programs which analyze the  data at two or more levels simultaneously allows for structuring research which includes particular school-level variables or variables which represent d i s t r i c t practices and p o l i c i e s .  A l t e r n a t i v e l y , the achievement  of pupils can be investigated over time by repeated measurement leading to a "growth rate" rather than a single point in time measure.  Differences in  growth rates between pupils can be investigated, and differences between schools in the average growth rates of pupils or for pupils of d i f f e r e n t a b i l i t y or SES  levels could be examined.  147  5.  As school mean-ability may  have acted as proxy for other important  v a r i a b l e s , i t would be desirable to conduct research to identify factors highly associated with school mean-ability which add predictive power to achievement performance. The contextual effects of school mean-ability identified in this study could be investigated in a variety of ways. One approach would be to identify i f procedures or practices which lead to improved academic performance are different in schools with different levels of school mean-ability.  Another approach would be to determine i f the improved  performance requires generally higher a b i l i t y for the majority of class members or i f some c r i t i c a l number of pupils of high a b i l i t y could have the same effect (Willms, 1986). pupils with low a b i l i t y .  The question could be reversed regarding  Hierarchical linear modelling i s an appropriate  method for this type of investigation.  This investigation could have  implications for grouping pupils in classrooms, for maintaining neighborhood boundaries for enrollment and for mainstreaming pupils with special needs.  148  REFERENCES A f f l e c k , J . O . , Madge, S. Adams, A . & L o w e n b r a u n , S . ( 1 9 8 8 ) . I n t e g r a t e d c l a s s r o o m s vs r e s o u r c e m o d e l : A c a d e m i c v i a b i l i t y and e f f e c t i v e n e s s . Exceptional Children, 54, 339-348. A d e l m a n , H . S . & F e s h b a c h , S. ( 1 9 7 1 ) . P r e d i c t i n g r e a d i n g f a i l u r e : Beyond the r e a d i n e s s m o d e l . Exceptional Children, 37, 349-354. A i k e n , L . R . ( 1 9 7 2 ) . Language f a c t o r s Review of Educational Research,  in learning mathematics. 42, 3 5 9 - 3 8 5 .  A i k e n , M . A . & L o n g f o r d , N . T . (1986). S t a t i s t i c a l modelling i s s u e s i n s c h o o l e f f e c t i v e n e s s s t u d i e s . Journal of the Royal Statistical Society, A, 149, 1-26. Alexander, K . L . , Fennessey, J . M c D i l l , E . L . & D'Amico, R . J . ( 1 9 7 9 ) . S c h o o l SES i n f l u e n c e s — c o m p o s i t i o n o r c o n t e x t ? Sociology of Education, 52, 222-237. A l g o z z i n e , B . , M e r c e r , C D . & C o u n t e r m i n e , T . ( 1 9 7 5 ) . The e f f e c t s o f l a b e l s and b e h a v i o r on t e a c h e r e x p e c t a t i o n s . Exceptional Children, 44, 1 3 1 - 1 3 2 . A l g o z z i n e , B. , Mercer, C D . & Countermine, T . (1977). L a b e l i n g e x c e p t i o n a l c h i l d r e n : An a n a l y s e s o f e x p e c t a t i o n s . Exceptional Children, 44, 1 3 1 - 1 3 2 . American P s y c h o l o g i c a l A s s o c i a t i o n Educational and Psychological Ames,  L . B . (1963). H a r p e r & Row.  Is Your  Child  ( 1 9 7 4 ) . Standards for Tests. W a s h i n g t o n , DC: A P A .  in the  A n a s t a s i , A . ( 1 9 7 6 ) . Psychological Testing MacMillan Publishing C o . , Inc.  Wrong Grade? (4th  New  York:  E d . ) . New Y o r k :  A n d e r s o n , C S . ( 1 9 8 2 ) . The s e a r c h f o r s c h o o l c l i m a t e : o f the r e s e a r c h . Review of Educational Research, 368-420.  A review 52(3),  Anderson-Inman, L . (1986). B r i d g i n g the gap: Student-centered s t r a t e g i e s for promoting t r a n s f e r of l e a r n i n g . Exceptional Children, 52, 5 6 2 - 5 7 2 . A n g o f f , W . H . ( 1 9 7 1 ) . S c a l e s , norms, and e q u i v a l e n t s c o r e s . In R . L . T h o r n d i k e ( E d . ) , Educational Measurement (2nd e d . ) . W a s h i n g t o n , DC: A m e r i c a n C o u n c i l on E d u c a t i o n .  149  A s k o v , W . , O t t o , W. & S m i t h , R . ( 1 9 7 2 ) . A s s e s s m e n t o f t h e de H i r s c h p r e d i c t i v e i n d e x t e s t s o f r e a d i n g f a i l u r e . In R . C . Auckerman ( E d . ) , Some Persistent Questions on Beginning Reading, (pp. 33-427). Newark, D E : I n t e r n a t i o n a l R e a d i n g Association. B a d i a n , N . (1986). Improving the p r e d i c t i o n o f r e a d i n g f o r t h e i n d i v i d u a l c h i l d : A f o u r - y e a r f o l l o w - u p . Journal of Learning Disabilities, 1 9 ( 5 ) , 262-269. B a d i a n , N . & S e r w e r , B . ( 1 9 7 5 ) . The i d e n t i f i c a t i o n o f h i g h c h i l d r e n : A r e t r o s p e c t i v e look a t s e l e c t i o n c r i t e r i a . Journal of Learning Disabilities, 8 ( 5 ) , 283-287.  risk  B a e n e n , N . R . ( 1 9 8 8 ) . Perspectives after five years — has retention passed or failed? Paper p r e s e n t e d t o t h e American E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n i n New Orleans. Bak,  J . J . , C o o p e r , E . M . , D o b r o t h , K . M . & S i p e r s t e i n , G.N. (1987). S p e c i a l c l a s s p l a c e m e n t s as l a b e l s : E f f e c t s on c h i l d r e n ' s a t t i t u d e s toward l e a r n i n g h a n d i c a p p e d p e e r s . Exceptional Children, 54, 1 5 1 - 1 5 5 .  Barnes, K . E . (1982). Preschool Screening: The Measurement and Prediction of Children At-risk. Springfield, IL: Charles C . Thomas P u b l i s h i n g . Barr,  R . & D r e e b e n , R. ( 1 9 7 7 ) . I n s t r u c t i o n L . S . Shulman ( E d . ) , Review of Research 5, I t a s c a , I L : P e a c o c k .  i n classrooms. In in Education, V o l  Barr,  R . & D r e e b e n , R. ( 1 9 8 3 ) . How Schools U n i v e r s i t y of Chicago Press.  Work.  Chicago, I L :  B e a t t i e , C . ( 1 9 7 0 ) . E n t r a n c e age t o k i n d e r g a r t e n and f i r s t g r a d e : I t s e f f e c t on c o g n i t i v e and a f f e c t i v e development of s t u d e n t s . (ERIC NO ED 133 050) B e c k e r , W . C . & G e r s t e n , R. ( 1 9 8 2 ) . A f o l l o w - u p o f f o l l o w t h r o u g h : The l a t e r e f f e c t s o f t h e d i r e c t i n s t r u c t i o n model on c h i l d r e n i n f i f t h and s i x t h g r a d e s . American Educational Research Journal, 19, 7 5 - 9 2 . B e e r s , C . S . & B e e r s , J . W . (1980). E a r l y i d e n t i f i c a t i o n o f l e a r n i n g d i s a b i l i t i e s : F a c t s and f a l l a c i e s . The Elementary School Journal, 81, 2. Beery, K. (1989). Integration.  The VMI Developmental Test of Visual T o r o n t o : Modern C u r r i c u l u m P r e s s .  Motor  150  B e e r y , K . & B u k t e n i c a , N . ( 1 9 8 2 ) . The Developmental Test of Visual Motor Integration. C l e v e l a n d , OH: Modern C u r r i c u l u m Press. B e n d e r , L . ( 1 9 3 8 ) . A V i s u a l Motor G e s t a l t T e s t and I t s C l i n i c a l U s e . R e s e a r c h Monograph Number 3 . New Y o r k , NY: A m e r i c a n Orthopsychiatric Association. B e r e i t e r , C . ( 1 9 6 3 ) . Some p e r s i s t i n g dilemmas i n t h e measurement o f c h a n g e . In C . W . H a r r i s ( E d . ) , Problems in Measuring Change. M a d i s o n , WI: U n i v e r s i t y o f W i s c o n s i n Press. B e y e r , F . S . & Smey-Richman, B . ( 1 9 8 8 ) . Addressing the "at-risk" challenge in the nonurban setting. Paper p r e s e n t e d a t t h e a n n u a l m e e t i n g o f the A m e r i c a n E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n , New O r l e a n s . B i d w e l l , C . E . & K a s a r d a , J . D . ( 1 9 8 0 ) . C o n c e p t u a l i z i n g and m e a s u r i n g the e f f e c t s o f s c h o o l and s c h o o l i n g . American Journal of Education, 88, 4 0 1 - 4 3 0 . B i l k a , L . ( 1 9 7 2 ) . An e v a l u a t i o n o f t h e p r e d i c t i v e v a l u e o f c e r t a i n readiness measures. In R . C . Auckerman ( E d . ) , Some Persistent Questions on Beginning Reading, (pp. 43-49). Newark, D E : I n t e r n a t i o n a l R e a d i n g A s s o c a t i o n . B l a c k , T . ( 1 9 7 1 ) . I n v e s t i g a t i o n o f i n t e l l i g e n c e as a f a c t o r i n r e a d i n g p r o b l e m s . Journal of Learning Disabilities, 4(3),22-25.  causal  B o e h n l e i n , M. (1987). Reading i n t e r v e n t i o n f o r h i g h - r i s k g r a d e r s . Educational Leadership, 44(6), 32-37.  first-  Book,  R . M . (1974). P r e d i c t i n g reading f a i l u r e : A s c r e e n i n g b a t t e r y f o r k i n d e r g a r t e n c h i l d r e n . Journal of Learning Disabilities, 7(1), 43-47.  Book.  R . M . (1980). I d e n t i f i c a t i o n of e d u c a t i o n a l l y a t - r i s k c h i l d r e n d u r i n g the k i n d e r g a r t e n y e a r : A f o u r - y e a r f o l l o w - u p s t u d y o f g r o u p s t e s t p e r f o r m a n c e . Psychology the Schools, 17, 1 5 3 - 1 5 8 .  Borg,  in  W . R . & G a l l , M . D . ( 1 9 8 3 ) . Fourth Edition Educational Research: An Introduction. New Y o r k : Longman.  Bracken, B . A . (1987). L i m i t a t i o n s of p r e s c h o o l instruments standards f o r minimal l e v e l s of t e c h n i c a l adequacy. Journal of Psychoeducational Assessment, 4, 3 1 3 - 3 2 6 .  and  151 B r a n d o n , P . R . , Newton, B . J . & Hammond, 0 . ( 1 9 8 7 ) . C h i l d r e n ' s m a t h e m a t i c s a c h i e v e m e n t i n H a w a i i : Sex d f f f e r e n c e s f a v o r i n g g i r l s . A m e r i c a n Educational Research Journal, 24(3), 437-461. B r e m e r , N . ( 1 9 8 0 ) . Do r e a d i n g t e s t s p r e d i c t s u c c e s s Elementary School Journal, 59, 2 2 2 - 2 2 9 . B r i c k e r , D . D . ( 1 9 8 6 ) . Early Education of At-risk Handicapped Infants, Toddlers, and Preschool G l e n v i e w , I L : S c o t t , Foresman, & C o .  in  reading?  and children.  Brookover, W . B . , Sweitzer, J . H . , Schneider, J . M . , Beady, C . H . , F l o o d , P . K . & Wisenbaker, J . M . (1978). Elementary s c h o o l s o c i a l c l i m a t e and s c h o o l a c h i e v e m e n t . American Educational Research Journal, 15, 3 0 1 - 3 1 8 . Bryk,  A . S . & Raudenbush, S.W. ( 1 9 8 7 ) . A p p l i c a t i o n o f h i e r a r c h i c a l l i n e a r models t o a s s e s s i n g c h a n g e . Psychological Bulletin, 101, 1 4 7 - 1 5 8 .  B u r t o n , L . , D r a k e , P . , E k i n s , J . , Graham, L . , T a p l i n , M . & Weiner, G . (1986). G i r l s i n t o mathematics. Centre f o r mathematics e d u c a t i o n . The Open U n i v e r s i t y i n a s s o c i a t i o n w i t h The London E d u c a t i o n A u t h o r i t y . C a m b r i d g e , MA: Cambridge U n i v e r s i t y P r e s s . Busch, R. (1980). P r e d i c t i n g f i r s t Learning Disability Quarterly,  grade reading 3, 3 8 - 4 7 .  achievement.  B u t l e r , S . R . (1979). P r e d i c t i v e antecedents of reading d i s a b i l i t y i n the e a r l y y e a r s o f s c h o o l i n g . Journal Special Education, 3, 263-274.  of  B u t t r a m , J . , C o v e r t , R. & H a y e s , M . ( 1 9 7 6 ) . P r e d i c t i o n o f s c h o o l r e a d i n e s s and e a r l y g r a d e a c h i e v e m e n t by c l a s s r o o m teachers. Educational and Psychological Measurement, 36, 543-546. C a s t o , G . & M a s t r o p i e r i , M . A . ( 1 9 8 6 ) . The e f f i c a c y o f e a r l y i n t e r v e n t i o n programs: A m e t a - a n a l y s i s . Exceptional Children, 52, 417-424. C h a l l , J . S . ( 1 9 6 7 ) . Learning to Read: The T o r o n t o : McGraw H i l l Book C o .  Great  Debate.  C l i f f o r d , P . & H e a t h , A . ( 1 9 8 4 ) . S e l e c t i o n does make a d i f f e r e n c e . Oxford Review of Education, 20(1) 85-97. C o l e m a n , J . M . ( 1 9 8 3 ) . H a n d i c a p p e d l a b e l s and i n s t r u c t i o n a l s e g r e g a t i o n : I n f l u e n c e s on c h i l d r e n ' s s e l f - c o n c e p t s v e r s u s  152 the p e r c e p t i o n s 6, 3 - 1 1 .  of others.  Learning  Disaility  Quarterly,  C o l l i g a n , R . C . & O ' C o n n e l l , E . J . (1974). Should psychometric s c r e e n i n g be made an a d j u n c t t o t h e p e d i a t r i c p r e s c h o o l examination? Clinical Pediatrics, 13, 2 9 - 3 4 . Cowen, E . L . , W e i s s b e r g , R . P . & G i s a r e , J . ( 1 9 8 4 ) . Differentiating attributes of c h i l d r e n referred to a s c h o o l m e n t a l h e a l t h p r o g r a m . Journal of Abnormal Child Psychology, 12, 3 9 7 - 4 0 9 . C r a r y , M . A . (1984). Phonological c h a r a c t e r i s t i c s of d e v e l o p m e n t a l v e r b a l a p r a x i a . Seminars in Speech Language, 5(2), 71-82.  and  D a v i s , B . G . , T r i m b l e , C . S . & V i n c e n t , D . R . ( 1 9 8 0 ) . Does age o f e n t r a n c e a f f e c t s c h o o l a c h i e v e m e n t ? The Elementary School Journal, 80, 1 3 4 - 1 4 3 . de H i r s c h , K . , J a n s k y , J . & L a n g f o r d , W. ( 1 9 6 6 ) . Predicting Reading Failure. New Y o r k , NY: Harper & Row. Denham, C . & L i e b e r m a n , A . ( E d s . ) ( 1 9 8 0 ) . rime to Learn. W a s h i n g t o n , DC: N a t i o n a l I n s t i t u t e o f E d u c a t i o n . Deno,  S . L . (1986). Formative e v a l u a t i o n of i n d i v i d u a l student p r o g r a m s : A new r o l e f o r s c h o o l p s y c h o l o g i s t s . School Psychology Review, 15(3), 358-374.  D e v e r e l l , A . ( 1 9 7 4 ) . The D e v e r e l l c l a s s i f i c a t i o n t e s t f o r u s e w i t h s c h o o l b e g i n n e r s ( l e t t e r s and n u m b e r s ) . In Teaching Children to Read and Write. Toronto O n t . : H o l t , Rinehart & Winston. Diamond, G . H . ( 1 9 8 3 ) . effect? Journal 164. Di  The b i r t h d a t e e f f e c t — A m a t u r a t i o n a l of Learning Disabilities, 16(3), 161-  P a s q u a l e , G . W . , M o u l e , A . D . & F l e w e l l i n g , R . W . ( 1 9 8 0 ) . The b i r t h d a t e e f f e c t . Journal of Learning Disabilities, 13(5), 4-8.  D o n o f r i o , A . F . ( 1 9 7 7 ) . Grade r e p e t i t i o n : T h e r a p y o f c h o i c e . Journal of Learning Disabilities, 10(6), 3 4 9 - 3 5 1 . D r e e b e n , R . & Gamoran, A . ( 1 9 8 6 ) . R a c e , i n s t r u c t i o n , a n d l e a r n i n g . American Sociological Review, 51, 6 6 0 - 6 6 9 . D u f f y , J . , R i t t e r , D . & Fedner, M. (1976). Developmental t e s t o f v i s u a l motor i n t e g r a t i o n and t h e Goodenough d r a w - a - m a n  153 t e s t as p r e d i c t o r s o f academic Motor Skills, 43, 543-546.  success.  Perceptual  and  Dunleavy, R . A . , Hansen, J . L . , S z a s z , C.W. & Baade, L . E . (1981). E a r l y kindergarten i d e n t i f i c a t i o n of academically n o t - r e a d y c h i l d r e n by use o f human f i g u r e d r a w i n g developmental score. Psychology in the Schools, 18(1), 35-38. Dunn,  J . A . ( 1 9 6 7 ) . Note on the r e l a t i o n o f H a r r i s ' t o WISC I . Q . ' S . Perceptual and Motor Skills,  D u r k i n , D . ( 1 9 7 4 ) . Teaching Young Children MA: A l l y n and B a c o n , I n c . D u r k i n , D. (1987). T e s t i n g Teacher, 40, 7 6 6 - 7 7 0 .  To Read.  i n the k i n d e r g a r t e n .  D u r r e l l , D . D . (1958). F i r s t - g r a d e reading summary. Journal of Education, 140,  draw a woman 24, 316.  The  Boston,  Reading  success study: A 2-6.  D y k s t r a , R . ( 1 9 6 7 ) . The use o f r e a d i n g r e a d i n e s s t e s t s f o r d i a g n o s i s and p r e d i c t i o n : A c r i t i q u e . In T . C . B a r r e t t (Ed.), The Evaluation of Children's Reading Achievement. Newark D E : I n t e r n a t i o n a l Reading A s s o c i a t i o n . E a v e s , L . C . , K e n d a l l , D . C . & C r i c h t o n , J . A . ( 1 9 7 4 ) . The e a r l y i d e n t i f i c a t i o n of l e a r n i n g d i s a b i l i t i e s : A f o l l o w - u p study. Journal of Learning Disability, 7(10), 42-48. E c c l e s , J . & J a c o b s , J . E . ( 1 9 8 6 ) . S o c i a l f o r c e s shape math a t t i t u d e s and p e r f o r m a n c e . S i g n s : Journal of Women in Culture and Society, 2 2 ( 2 ) , 367-380. Evans, R. (1976). longitudinal  The p r e d i c t i o n o f e d u c a t i o n a l h a n d i c a p - a s t u d y . Educational Research, 19, 5 7 - 6 8 .  Fennema, E . & P e t e r s o n , P . ( 1 9 8 5 ) . Autonomous l e a r n i n g behavior: explanation of gender-related d i f f e r e n c e s mathematics. In L . C . W i l k i n s o n & C . B . M a r e t t ( E d . ) , Influences In Classroon Interaction. (pp. 17-35). O r l a n d o , F L : Academic P r e s s .  in Gender  F e r i n d e n , W . , J a c o b s o n , S. & L i n d e n , N . ( 1 9 7 0 ) . E a r l y i d e n t i f i c a t i o n o f l e a r n i n g d i s a b i l i t i e s . Journal of Learning Disabilities, 3 ( 1 1 ) , 589-593. F e s h b a c h , S . , A d e l m a n , H . & F u l l e r , W. ( 1 9 7 4 ) . E a r l y i d e n t i f i c a t i o n of c h i l d r e n with high r i s k of reading failure. Journal of Learning Disabilities, 7(10), 49-54.  154 F l e t c h e r , J . M . & S a t z , P. (1982). K i n d e r g a r t e n p r e d i c t i o n o f reading achievement. A seven-year l o n g i t u d i n a l f o l l o w - u p . Educational and Psychological Measurement, 42, 6 8 1 - 6 8 5 . F o s t e r , G . , Schmidt, C . & Sabatino D. (1976). Teacher e x p e c t a n c i e s and t h e l a b e l " l e a r n i n g d i s a b i l i t e s " . of Learning Disabilites, 9 ( 2 ) , 58-61.  Journal  F o s t e r , G . , Y s s e l d y k e , J . E . & Reese, J . H . (1975). I w o u l d n ' t have s e e n i t i t i f I h a d n ' t b e l i e v e d i t . Exceptional Children, 41, 4 6 9 - 4 7 3 . Fox,  L . H . & C o h n , S . J . ( 1 9 8 0 ) . Sex d i f f e r e n c e s i n t h e development o f p r e c o c i o u s mathematical t a l e n t . In L . H . F o x , L . B r o d y , & D . T o b i n , ( E d s . ) , Women and the Mathematical Mystique. B a l t i m o r e , MD: The Johns H o p k i n s U n i v e r s i t y P r e s s . 94-112.  F r i s c h , G . R . & H a n d l e r , L . (1974). A N e u r o p s y c h o l o g i c a l i n v e s t i g a t i o n of f u n c t i o n a l d i s o r d e r s of speech articulation. Journal of Speech and Hearing Research, 432-445.  17,  G a l a n t e , M . B . , F l y e , M . E . , Stephens, R . N . & Stephens, L . S . ( 1 9 7 2 ) . C u m u l a t i v e minor d e f i c i t s : A l o n g i t u d i n a l s t u d y o f the r e l a t i o n of p h y s i c a l f a c t o r s to school achievement. Journal of Learning Disabilities, 5, 7 5 - 8 0 . G a l l a g h e r , J . J . (1984). Learning d i s a b i l i t i e s f u t u r e . Journal of Learning Disabilities,  and t h e n e a r 54(4), 339-348.  G a l l a g h e r , J . J . & B r a d l e y , R . H . (1972). E a r l y i d e n t i f i c a t i o n of d e v e l o p m e n t a l d i f f i c u l t i e s . In I . Gordon ( E d . ) , Early Childhood Education. Chicago, I L : U n i v e r s i t y of Chicago Press. Gamoran, A . ( i n p r e s s ) . S c h o o l i n g and a c h i e v e m e n t : versus i n t e r a c t i v e m u l t i l e v e l models.  Additive  G a r m o r a n , A . ( 1 9 8 4 ) . Teaching, grouping and learning: A study of the consequences of educational stratification. Unpublished P h . D . d i s s e r t a t i o n , Dept. of E d u c a t i o n . U n i v e r s i t y of Chicago. Gamoran, A . ( 1 9 8 8 ) . R e s o u r c e a l l o c a t i o n and t h e e f f e c t s o f s c h o o l i n g : A s o c i o l o g i c a l p e r s p e c t i v e , ( p p . 2 0 7 - 2 3 2 ) . In D . H . Monk & J . Underwood ( E d s . ) . Microlevel School Finance: Issues and Implications for Policy. Ninth Annual Yearbook o f t h e A m e r i c a n E d u c a t i o n a l F i n a n c e A s s o c i a t i o n , C a m b r i d g e , MA: B a l l i n g e r .  155 Gamoran, A . & D r e e b e n , R. ( 1 9 8 6 ) . C o u p l i n g and c o n t r o l i n educational organizations. Administrative Science Quarterly, 31, 6 1 2 - 6 3 2 . G a u t h i e r , S. & M a d i s o n , C . ( 1 9 7 3 ) . Kindergarten Screening Test. T i g a r d , OR: CC P u b l i c a t i o n s .  Language  G e r m a i n , R . b . & M e r l o , M. ( 1 9 8 5 ) . B e s t p r a c t i c e s i n a s s i s t i n g i n p r o m o t i o n and r e t e n t i o n d e c i s i o n s . In A . Thomas & J . Grimes ( E d s . ) . Best Practices In School Psychology. Kent, OH: N a t i o n a l A s s o c i a t i o n o f S c h o o l Psychologists. G e t t i n g e r , M . ( 1 9 8 4 ) . Achievement as a f u n c t i o n o f t i m e s p e n t i n l e a r n i n g and t i m e needed f o r l e a r n i n g . American Educational Research Journal, 21, 6 1 7 - 6 2 8 . G h i s e l l i , E . , C a m p b e l l , J . & Z e d e c k , S. Theory for the Behavioural Sciences. W . H . Freeman & C o . G l a s s , G . & Hopkins, K. (1984). Education and Psychology. H a l l , Inc.  (1981). Measurement San F r a n s c i s c o , C A :  Statistical Methods in Englewood C l i f f s , N J : P r e n t i c e  G l a v i n , J . , Quay, H . , A n n e s l e y , F . & W e r r y , J . ( 1 9 7 1 ) . An e x p e r i m e n t a l r e s o u r c e room f o r b e h a v i o r p r o b l e m c h i l d r e n . Exceptional Children, 38, 1 3 1 - 1 3 8 . G l a z z a r d , M. ( 1 9 7 9 ) . The l o n g - r a n g e e f f e c t i v n e s s o f t h r e e k i n d e r g a r t e n p r e d i c t o r s f o r f i r s t - , s e c o n d - , t h i r d - and f o u r t h - g r a d e achievement. Journal of Learning Disabilities, 12(10), 55-60. G l a z z a r d , P . H . (1982). Long-range k i n d e r g a r t e n p r e d i c t i o n of r e a d i n g achievement i n f i r s t through s i x t h g r a d e . Learning Disability Quaterly, 5, 8 5 - 8 8 . G o e t z , E . ( 1 9 7 1 ) . H e a r i n g growth and r e a d i n g . In M. D o u g l a s s ( E d . ) , Thirty-fifth Yearbook of the Claremont Reading Conference, 120-126. Goldman, R . , F r i s t o e , M. & Woodcock, R. ( 1 9 7 0 ) . Test of auditory discrimination. C i r c l e P i n e s , MN: A m e r i c a n Guidance S e r v i c e , Inc. Goldman, R . K . & V e l a s c o , J r . , M. ( 1 9 8 0 ) . Toward t h e development o f a r a t i o n a l s c a l e i n the use o f h u m a n - f i g u r e d r a w i n g s as a k i n d e r g a r t e n s c r e e n i n g m e a s u r e . Perceptual and Motor Skills, 50, 5 7 1 - 5 7 7 .  156 G o l d s t e i n , H . ( 1 9 8 6 ) . M u l t i l e v e l mixed l i n e a r model a n a l y s e s using i t e r a t i v e generalized least squares. Biometrika, 73(1), 43-56. Good,  T . L . & S l a v i n g s , R . L . ( 1 9 8 8 ) . Male and f e m a l e q u e s t i o n a s k i n g b e h a v i o r i n e l e m e n t a r y and s e c o n d a r y m a t h e m a t i c s and l a n g u a g e a r t s c l a s s e s . Journal of Research in Childhood Education, 3(1), 5-23.  Goodenough, T . Drawings.  (1926). Measurement of Intelligence Y o n k e r s - o n - H u d s o n , NY: World B o o k .  Goodwin, W . L . & D r i s c o l l , L . A . (1980). Handbook Measurement and Evaluation in Early Childhood San F r a n c i s c o , CA: J o s e y - B a s s P u b l i s h e r s . Gordon, N. & McKinlay, I . Children. Churchill York.  by for Education.  ( E d s . ) . (1980). Helping Clumsy L i v i n g s t o n e , E d i n b u r g h London and New  G o r t m a k e r , S . L . & S a p p e n f i e l d , W. ( 1 9 8 4 ) . C h r o n i c c h i l d h o o d d i s o r d e r s : P r e v a l e n c e and i m p a c t . Pediatric Clinics of North America, 31, 3-18. G o t t f r e d s o n , G . D . (1988). You get what you measure - you get what you don't: Higher standards, higher test scores, more retention in grade. Paper p r e s e n t e d a t the A n n u a l M e e t i n g o f t h e A m e r i c a n E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n , New Orleans. Gray,  R . A . , S a s k i , J . , McEntire, M . E . & Larsen, S . C . (1980). p r o f i c i e n c y i n o r a l language a p r e d i c t o r o f a c a d e m i c success? The Elementary School Journal, 80, 261-268.  Is  G r e d l e r , G . R . ( 1 9 7 8 ) . A l o o k a t some i m p o r t a n t f a c t o r s i n assessing readiness for school. Journal of Learning Disabilities, 11(5), 25-31. G r e d l e r , G . R . ( 1 9 8 0 ) . C u m u l a t i v e r e t e n t i o n r a t e as an i n d e x academic progess: A third look. Journal of Learning Disabilities, 13, 1 5 - 1 8 . G r e d l e r , G . R . ( 1 9 8 0 ) . The b i r t h d a t e e f f e c t : F a c t o r Journal of Learning Disabilities, 13, 9-12.  of  artifact?  G r e d l e r , G . R . (1984). T r a n s i t i o n c l a s s e s : A v i a b l e a l t e r n a t i v e f o r the a t - r i s k c h i l d ? Psychology in the Schools, 21, 463-470. Greenwood, C . R . , D i n w i d d i e , G . , T e r r y , B . , Wade, L . , S t a n l e y , S . , T h i b a d e a u , S. & D e l q u a d r i , J . ( 1 9 8 4 ) . T e a c h e r - v e r s u s  157 p e e r - m e d i a t e d i n s t r u c t i o n : An e c o b e h a v i o r a l a n a l y s i s achievement outcomes. Journal of Applied Behavior Analysis, 17, 521-538.  of  G r i f f i n , M . & E b e r l y , D.W. ( 1 9 7 1 ) . V i s u a l problems and reading. W a s h i n g t o n , DC: N a t i o n a l R e a d i n g C e n t e r Foundation. G r i m l e y , A . M . & McKinlay, I . A . (1977). The Clumsy Child. A s s o c i a t i o n of P e d i a t r i c Chartered P s y c h o t h e r a p i s t s , Sussex. G r o f f , P. (1977). 17, 71-78. G r o f f , P. (1978). composition.  Oral  Language and r e a d i n g .  Children's oral 272e Elementary  Reading  World,  l a n g u a g e and t h e i r w r i t t e n School Journal, 78, 181-191.  G r o n l u n d , N . E . (1975). Measurement and Evaluation New Y o r k : M a c M i l l a n P u b l i s h i n g C o . Gubbay, S . S . (1975). Saunders.  The  The Clumsy C h i l d .  in  Teaching.  P h i l a d e l p h i a , PA:  G u l l i f o r d , R. ( 1 9 7 6 ) . The e a r l y i d e n t i f i c a t i o n o f e d u c a t i o n a l l y a t r i s k c h i l d r e n . In K . W e d e l l & B . C . R a y b o u l d ( E d s . ) , The early identification of 'at risk' children. Educational R e v i e w , O c c a s i o n a l P u b l i c a t i o n #6, U n i v e r s i t y o f Birmingham. G u r a l n i c k , M . J . & B r i c k e y D . D . (1987). The e f f e c t i v n e s s o f e a r l y i n t e r v e n t i o n f o r c h i l d r e n w i t h c o g n i t i v e and g e n e r a l developmental d e l a y s . In M . J . G u r a l n i c k & F . C . B e n n e t t ( E d s . ) , The Effectivness of Early Intervention for At-Risk and Handicapped Children, (pp. 115-173). O r l a n d o , F L : Academic P r e s s . G u s k i n , S . L . , B a r t e , N . R . , & Macmillan, D . L . (1975). P e r s p e c t i v e o f t h e l a b e l e d c h i l d . In N . Hobbs ( E d . ) , Issues in the Classification of Children, 2. San F r a n c i s c o , CA: J o s s e y - B a s s . Hall,  R . V . ( 1 9 6 3 ) . Does e n t r a n c e age a f f e c t a c h i e v e m e n t : Elementary School Journal, 64, 3 9 1 - 3 9 6 .  H a l l i n a n , M . T . , & Sorenson, A . B . (1987). A b i l i t y sex d i f f e r e n c e s i n m a t h e m a t i c s a c h i e v e m e n t . Education, 60, 6 3 - 7 2 . Hammill,  D . D . & McNutt,  G . (1980).  g r o u p i n g and Sociology of  Language a b i l i t i e s  and  158 r e a d i n g : A r e v i e w of the l i t e r a t u r e on t h e i r The Elementary School Journal, 80, 2 6 9 - 2 7 7 . Hammond, J . M . ( 1 9 8 0 ) . N u t r i t i o n and h y p e r a c t i v i t y . Education, 4, 34.  relationship.  American  Hanna, G . & K u e n d i g e r , E . (1986). Differences in mathematical achievement levels in attitude for girls and boys in twenty countries. Paper p r e s e n t e d a t the a n n u a l m e e t i n g o f t h e A m e r i c a n E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n , San F r a n c i s c o , CA. H a r b e r , J . R . ( 1 9 8 1 ) . A s s e s s i n g the q u a l i t y o f in special education. Journal of Special 77-90. H a r i n g , N . G . & Ridgway, R.W. ( 1 9 6 7 ) . E a r l y c h i l d r e n with l e a r n i n g d i s a b i l i t i e s . Children, 33, 3 8 7 - 3 9 5 . H a r r i s , D. (1963). Children's Intellectual Maturity. World, Inc.  d e c i s i o n making Education, 15,  identification Exceptional  of  Drawings as Measures of New Y o r k , NY: H a r c o u r t , B r a c e &  H a r t l a g e , L . C . & L u c a s , D . G . ( 1 9 7 3 ) . Group s c r e e n i n g f o r r e a d i n g d i s a b i l i t y i n f i r s t grade c h i l d r e n . Journal Learning Disabilities, 6(5), 48-52. H a u s e r , R. ( 1 9 7 0 ) . C o n t e x t and C o n s e x : Americal Journal of Sociology, 75,  of  A Cautionary Tale. 645-663.  Hayden, A . H . (1974). P e r s p e c t i v e s of e a r l y c h i l d h o o d e d u c a t i o n in special education. In N . G . H a r i n g ( E d . ) , Behavior of Exceptional Children: An Introduction To Special Education. Columbus, OH: C h a r l e s E . M e r r i l l . Haynes, M . C . & J e n k i n s , J . R . (1986). Reading i n s t r u c t i o n i n s p e c i a l e d u c a t i o n r e s o u r c e rooms. American Educational Research Journal, 23, 1 6 1 - 1 9 0 . Hedges, W.D. (1977). At What Age Should Children Enter First Grade. A Comprehensive Review of the Research. Published for F l o r i d a E d u c a t i o n a l R e s e a r c h & Development C o u n c i l . Ann A r b o r , M I : U n i v e r s i t y M i c r o f i l m s I n t e r n a t i o n a l . Henderson, V . , M i e s z k o w s k i , P. & Sauvageau, Y . (1978). Peer g r o u p e f f e c t s and e d u c a t i o n a l p r o d u c t i o n f u n c t i o n s . Journal of Public Economics, 10, 97-106. Hieronymus, A . N . , K i n g , E . M . , Bourdon, J . W . , G o s s l i n g , D . , G r y w i n s k i , N . J . , & Moss, G . L . (1976). Canadian Tests  of  159 Basic Skills: and Counsellors.  Manuals for Canada:  Administrators, Supervisors, Thomas N e l s o n & S o n s .  H i n t o n , G . G . & K n i g h t s , R . M . (1971). C h i l d r e n with l e a r n i n g p r o b l e m s : Academic h i s t o r y , a c a d e m i c p r e d i c t i o n and adjustment three years a f t e r assessment. Exceptional Children, 37, 5 1 3 - 5 1 9 . Hoge,  R. & C o l a d a r c i , J . (1989). T e a c h e r - b a s e d judgements o f academic achievement: A research of l i t e r a t u r e . Review of Educational Research, 5 9 ( 3 ) , 297-313.  H o l m e s , C . T . & M a t t h e w s , K . M . ( 1 9 8 4 ) . The e f f e c t s o f n o n p r o m o t i o n on e l e m e n t a r y and j u n i o r h i g h p u p i l s : A meta-analysis. Review of Educational Research, 54, 225-236. Hoover, H . D . (1984). The most a p p r o p r i a t e s c o r e s f o r m e a s u r i n g e d u c a t i o n a l development i n the e l e m e n t a r y s c h o o l s : GE's. Educational Measurement: Issues and Practice, 3(4), 8-14. Hoover, H . D . (1984). Rejoinder to B u r k e t . Educational Measurement: Issues and Practices, 3(4), 16-18. H o p k i n s , K . D . & G l a s s , G . V . (1978). Basic Statistics for the Behavioral Sciences. Englewood C l i f f s , N J : P r e n t i c e Hall. H o r s t , D . P . (1975). A practical guide to measuring project impact on student achievement. Number 1 i n a S e r i e s o f Monographs on E v a l u a t i o n i n E d u c a t i o n . U . S . D e p t . o f H e a l t h , E d u c a t i o n and W e l f a r e . W a s h i n g t o n , DC: U . S . G o v ' t Printing Office. H o r s t , D . P . (9186). What's bad about grade-equivalent scores. ESEA T i t l e 1 E v a l u a t i o n and R e p o r t i n g S y s t e m , T e c h n i c a l Paper No. 1. M o u n t a i n V i e w , C A : RMC R e s e a r c h Corporation. H u n t e r , E . J . & J o h n s o n , L . C . ( 1 9 7 1 ) . D e v e l o p m e n t a l and p s y c h o l o g i c a l d i f f e r e n c e s between r e a d e r s and n o n r e a d e r s . Journal of Learning Disabilities, 4 ( 1 0 ) , 572-577. Hyde,  J . ( 1 9 8 1 ) . How l a r g e a r e c o g n i t i v e gender American Psychologist, 36, 2 9 2 - 3 0 1 .  Ilg,  F . L . & Ames, T . B . ( 1 9 6 4 ) . Tests used at the Gessell & Row.  differences?  School Readiness. Behavior Institute. New Y o r k , NY: H a r p e r  160  J a c k s o n , G . B . (1975). grade r e t e n t i o n . 613-635.  The r e s e a r c h e v i d e n c e Review of Educational  on t h e e f f e c t s Research, 45,  of  J a c o b , S . , S n i d e r , K . P . & W i l s o n , J . F . (1988). V a l i d i t y of the DIAL-R f o r i d e n t i f y i n g c h i l d r e n w i t h s p e c i a l e d u c a t i o n needs and p r e d i c t i n g e a r l y r e a d i n g a c h i e v e m e n t . Journal of Psychoeducational Assessment, 6, 2 8 9 - 2 9 7 . J a n s k y , J . & de H i r s c h , K . ( 1 9 7 2 ) . New Y o r k , NY: H a r p e r & Row.  Preventing  Jansky, J . J . (1973). Early prediction of Orton Society Reprint S e r i e s , XXIII, Towson, MD: The O r t o n S o c i e t y , I n c .  Reading  reading problems. R e p r i n t N o . 55,  Failure. The  Jansky, J . J . (1978). A c r i t i c a l r e v i e w o f some d e v e l o p m e n t a l and p r e d i c t i v e p r e c u r s o r s o f r e a d i n g d i s a b i l i e s . I n A . L . B e n t o n & D. P e a r l ( E d s . ) , Dyslexia an Appraisal of Current Knowledge. New Y o r k , NY: O x f o r d U n i v e r s i t y Press. Jarman, R . R . (1980). T h e o r e t i c a l and m e t h o d o l o g i c a l i s s u e s i n the e a r l y i d e n t i f i c a t i o n of cognitive-developmental d i s a b i l i t i e s : D e s c r i p t i o n s o f a l o n g i t u d i n a l s t u d y . In R . F . Jarman & J . P . Das ( E d s . ) . Issues in Developmental Disabilities. Ann A r b o r , M I : U n i v e r s i t y M i c r o f i l m s International. J e n k i n s , E . & L o h r , F . E . (1964). and motor a b i l i t y . Journal Disorders, 2 9 ( 3 ) , 286-292.  Severe a r t i c u l a t i o n d i s o r d e r s of Speech and Hearing  J o h n s o n , S. ( 1 9 8 7 ) . E a r l y - d e v e l o p e d sex d i f f e r e n c e s and m a t h e m a t i c s i n the U n i t e d Kingdom. Journal Adolescence, 7(1), 1-3.  in science of Early  J o h n s t o n e , P . , A l l i n g t o n , R. & A f f l e r b a c h , P . ( 1 9 8 5 ) . The c o n f l u e n c e o f c l a s s r o o m and r e m e d i a l i n s t r u c t i o n . Elementary School Journal, 85, 4 6 5 - 4 7 7 . Judy,  J . (1986). Early screening is essential for educational a c c o u n t a b i l i t y : Response t o S a l z e r and t o S h e p a r d and Smith. Educational Leadership, 11, 87-88.  K a r w e i t , N . (1988). Effective elementary programs and practices for at risk students. Paper p r e p a r e d f o r a n n u a l m e e t i n g o f A m e r i c a n E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n , New Orleans.  161 Kaufman, A . S . & Kaufman, N . L . ( 1 9 7 2 ) . T e s t s b u i l t from and G e s e l l ' s t a s k s as p r e d i c t o r s o f f i r t s - g r a d e achievement. Child Development, 43, 521-535.  Piaget's  Keogh, B . (1977). E a r l y I . D . : s e l e c t i v e p e r c e p t i o n or perceptive selection? Academic Therapy, 12, 267-274. Keogh, B . & B e c k e r , L . D . (1973). E a r l y d e t e c t i o n of p r o b l e m s : Q u e s t i o n s , c a u t i o n s and g u i d e l i n e s . Children, 40, 5-11.  learning Exceptional  Keogh, B . K . & D a l e y , J . E . (1983). E a r l y i d e n t i f i c a t i o n : One component o f c o m p r e h e n s i v e s e r v i c e s f o r a t - r i s k c h i l d r e n . Topics in Early Childhood Special Education, 3(3), 7-16. K e o g h , B . K . , T c h i r , M . A . & Wendeguth-Behn, A . ( 1 9 7 4 ) . T e a c h e r s ' p e r c e p t i o n s of e d u c a t i o n a l l y high r i s k c h i l d r e n . Journal of Learning Disabilities, 7(6), 43-50. K i n g , E . M . ( 1 9 8 2 ) . Canadian Test of Basic Skills (Teacher's Guide). C a n a d a : N e l s o n Canada L i m i t e d . Kirk,  W.D. (1966). A t e n t a t i v e s c r e e n i n g p r o c e d u r e f o r s e l e c t i n g b r i g h t and slow c h i l d r e n i n k i n d e r g a r t e n . Exceptional Children, 33, 2 3 5 - 2 4 1 .  K l e i n , A . E . ( 1 9 7 7 ) . The v a l i d i t y o f the S c r e e n i n g T e s t o f A c a d e m i c R e a d i n e s s i n p r e d i c t i n g a c h i e v e m e n t i n f i r s t and second g r a d e s . Educational and Psychological Measurement, 37, 493-499. K o p p i t z , E . M . (1975). The Bender Gestalt Test for Children. New Y o r k , NY: Greene & S t r a t t o n .  Young  R o m b e r g , M . S . & K a p l a n , G . ( 1 9 8 0 ) . R i s k f a c t o r s and i n t e r v e n t i o n i n c h i l d t h e r a p y , A r e v i e w . Journal Prevention, 1, 7 1 - 1 3 3 .  preventive of  L a r t e r , S. ( 1 9 8 2 ) . The physically handicapped and health impaired children: Do they prosper in regular Toronto elementary school? T o r o n t o , O n t a r i o : Toronto Board of E d u c a t i o n Research Department. L a T o r r e , R . A . , Hawkhead, F . , K a w a h i r a , R. & B i l o w , L . ( 1 9 8 2 ) . K i n d e r g a r t e n s c r e e n i n g p i l o t p r o j e c t i n Vancouver s c h o o l s 1 9 7 9 - 8 0 : A t w o - y e a r f o l l o w - u p o f the M c C a r t h y S c r e e n i n g Test. B . C . Journal of Special Education, 6, 2 3 - 4 1 . L a z a r , I . & D a r l i n g t o n , R. ( 1 9 7 8 ) . Lasting Effects After Preschool, W a s h i n g t o n , DC: O f f i c e o f Human Development S e r v i c e s , U . S . Department o f H e a l t h , E d u c a t i o n and Welfare.  162  L a z a r , I . , H u b b e l l , R . , Murray, H . , Rosche, M. & Royce, J . (1978). P r e l i m i n a r y f i n d i n g s o f the d e v e l o p m e n t a l c o n t i n u i t y l o n g i t u d i n a l s t u d y . In K . A l l e n , V . Holm & R . Schiefelbusch (Eds.), Early Intervention — A Team Approach. B a l t i m o r e , MD: U n i v e r s i t y Park P r e s s . L e i g h , J . E . (1983). E a r l y a l t e r n a t i v e s . Topics 3(3) 1-6.  l a b e l i n g o f c h i l d r e n : C o n c e r n s and In Early Childhood Special Education,  L e i n h a r d t , G . ( 1 9 8 0 ) . T r a n s i t i o n rooms: P r o m o t i n g m a t u r a t i o n o r r e d u c i n g e d u c a t i o n ? Journal of Educational Psychology, 72, 55-61. L e r n e r , J . , Mardell-Czudnowski, C . & Goldenberg, D. (1981). Special Education for the Early Childhood Years. Englewood C l i f f s , N J : P r e n t i c e - H a l l . L e s i a k , W . J . (1978). Developmental tasks for k i n d e r g a r t e n r e a d i n e s s . Archives of Behavioral Science Monograph (No.52). Levi,  G . , M u s a t t i , L . , L e t i z i a P r i e d d a , M. S e c h i , E . ( 1 9 8 4 ) . C o g n i t i v e and l i n g u i s t i c s t r a t e g i e s i n c h i l d r e n w i t h r e a d i n g d i s a b i l i t i e s i n an o r a l s t o r y t e l l i n g t e s t . Journal of Learning Disabilities, 17, 4 0 6 - 4 1 0 .  L e w i s , A . ( 1 9 8 0 ) . The e a r l y i d e n t i f i c a t i o n o f c h i l d r e n w i t h l e a r n i n g d i f f i c u l t i e s . Journal of Learning Disabilities, 13(2), 51-57. L i c h t e n s t e i n , R. ( 1 9 8 1 ) . C o m p a r a t i v e v a l i d i t y o f two p r e s c h o o l s c r e e n i n g t e s t s : C o r r e l a t i o n a l and c l a s s i f i c a t i o n a l a p p r o a c h e s . Journal of Learning Disabilities, 14, 68-72. L i c h t e n s t e i n , R. & I r e t o n , N . ( 1 9 8 4 ) . P r e s c h o o l S c r e e n i n g : Identifying Young Children with Developmental and Educational Problems. O r l a n d o , F l : Grune & S t r a t t o n . Lidz,  C . S . (1983). Issues i n Assessing K . D . Paget & B . A . B r a c k e n ( E d s . ) . Assessment of Preschool Children, NY: Grune & S t r a t t o n .  Preschool C h i l d r e n . In The Psychoeducational (pp. 17-27). New Y o r k ,  L i n d q u i s t , G . T . ( 1 9 8 2 ) . P r e s c h o o l s c r e e n i n g as a means p r e d i c t i n g l a t e r r e a d i n g a c h i e v e m e n t . Journal of Disabilities, 2 5 ( 6 ) , 331-332. Lindsay,  G . A . & W e d e l l , K . ( 1 9 8 2 ) . The e a r l y  of Learning  identification  of  163 e d u c a t i o n a l l y at r i s k Learning Disabilities, Linn,  children revisited. 15, 212-216.  Journal  of  R . J . & S l i n d e , J . A . (1977). The d e t e r m i n a t i o n o f t h e s i g n i f i c a n c e o f change between p r e - and p o s t t e s t i n g periods. Review of Educational Research, 47, 1 2 1 - 5 0 .  L l o y d , D . N . (1978). P r e d i c t i o n of s c h o o l t h i r d - g r a d e d a t a . Educational and Measurement, 38, 1193-1200.  f a i l u r e from Psychological  M a c c o b y , E . E . & J a c k l i n , C . N . ( 1 9 7 4 ) . The Psychology of Differences. S t a n f o r d , CA: The S t a n f o r d U n i v e r s i t y  Sex Press.  M a c M i l l a n , L . , J o n e s , R. & A l o i a , G . ( 1 9 7 4 ) . The m e n t a l l y r e t a r d e d l a b e l : a t h e o r e t i c a l a n a l y s i s and r e v i e w o f research. American Journal of Mental Deficiency, 79, 261.  241-  Madden, N . & S l a v i n , R. ( 1 9 8 3 ) . M a i n s t r e a m i n g s t u d e n t s w i t h m i l d a c a d e m i c h a n d i c a p s : Academic and s o c i a l o u t c o m e s . Review of Educational Research, 43, 519-569. Madden, N . & S l a v i n , R. ( 1 9 8 7 ) . Effective Pull-out Programs for Students At Risk. B a l t i m o r e , MD: C e n t r e f o r R e s e a r c h on E l e m e n t a r y and M i d d l e S c h o o l s , The Johns H o p k i n s University. M a i t l a n d , S . , Nadeau, J . B . E . & Nadeau, G . ( 1 9 7 4 ) . s c r e e n i n g p r a c t i c e s . Journal of Learning 7(10), 55-59. Mann,  P., Suiter, of Diagnostic  Early school Disabilities,  P . & M c C l u n g , R. ( 1 9 8 7 ) . Teacher's Handbook Screening. B o s t o n , MA: A l l y n and B a c o n , I n c .  M a r d e l l , C . & G o l d e n b e r g , D. ( 1 9 7 5 a ) . F o r p r e k i n d e r g a r t e n s c r e e n i n g d i s a b i l i t e s : DIAL. J o u r n a l of L e a r n i n g D i s a b i l i t e s , 8 ( 3 ) , 140-47. M a r d e l l , C . & G o l d e n b e r g , D . ( 1 9 7 5 b ) . Developmental for the Assessment of Learning. Highland Park, Inc.  Indicators I L : DIAL,  M a r t i n , D . J . & H o o v e r , H . D . ( 1 9 8 7 ) . Sex d i f f e r e n c e s e d u c a t i o n a l achievement: A l o n g i t u d i n a l study. Early Adolescence, 7(1), 65-83.  in Journal  Mayron, L . W . (1978). E c o l o g i c a l f a c t o r s i n l e a r n i n g disabilities. Journal of Learning Disabilities, 40.  22(8),  of  164  M c A f e e , J . K . ( 1 9 8 1 ) . Toward a theory of promotion: Does retaining students really work? Paper presented at the meeting of the American Educational Research Association, Los Angeles, CA. McCann, R. & Austin, S. (1988). At-risk youth: Definitions, dimensions, and relationships. Paper p r e s e n t e d f o r t h e A n n u a l Meeting of the American E d u c a t i o n a l R e s e a r c h Association. McLoughlin, J . A . , H a l l , M . , Isaacs, B . , P e t r o s k i , J . , Karibo, J . & L i n d s e y , B . ( 1 9 8 3 ) . The r e l a t i o n s h i p o f a l l e r g i e s and a l l e r g y t r e a t m e n t t o s c h o o l p e r f o r m a n c e . A n n a l s of Allergy, 51, 506. M c N u t t , G . & F r i e n d , M . ( 1 9 8 5 ) . S t a t u s o f the r e s o u r c e room model i n l o c a l e d u c a t i o n a g e n c i e s : A d e s c r i p t i v e s t u d y . Learning Disability Quarterly, 8, 1 0 1 - 1 0 8 . Medway, F . J . & R o s e , J . S . ( 1 9 8 6 ) . Grade r e t e n t i o n . In T . K r a t o c h w i l l ( E d . ) , Advances in School Psychology, 1, 141-175. M e e h l , P . E . & R o s e n , A . ( 1 9 5 5 ) . A n t e c e d e n t p r o b a b i l i t y and e f f i c a c y of p s y c h o m e t r i c s i g n s , p a t t e r n s or c u t t i n g scores. Psychological Bulletin, 52, 1 9 4 - 2 1 6 .  the  M e i s e l s , S . J . ( 1 9 8 4 ) . Prediction, prevention and developmental screening in the EPSDT program. In H.W. S t e v e n s o n & A . E . S i e g e l ( E d s . ) . C h i l d Development R e s e a r c h and S o c i a l P o l i c y . C h i c a g o , I L : U n i v e r s i t y of Chicago P r e s s . M e i s e l s , S . J . ( 1 9 8 5 ) . Developmental Screening in Early Childhood: A Guide (Rev.Ed.). W a s h i n g t o n DC: N a t i o n a l A s s o c i a t i o n f o r the E d u c a t i o n o f Young C h i l d r e n . M e i s e l s , S . J . ( 1 9 8 6 ) . T e s t i n g f o u r - and f i v e - y e a r - o l d s . Response t o S a l z e r and t o S h e p a r d and S m i t h . Educational Leadership, 44(3) 90-92. M e i s e l s , S . J . ( 1 9 8 7 ) . Uses and a b u s e s o f d e v e l o p m e n t a l s c r e e n i n g and s c h o o l r e a d i n e s s t e s t i n g . Young Children, 42(2), 68-73. M e i s e l s , S . J . (1989). High-stakes Leadership, 46(7), 16-22.  testing.  M e r c e r , C D . ( 1 9 7 9 ) . Children and Adolescents Disabilities. C o l u m b u s , OH: C h a r l e s E .  Educational with Learning Merrill.  165 Mercer, C D . , A l g o z z i n e , B. & T r i f i l e t t i , i d e n t i f i c a t i o n — an a n a l y s i s o f the Disability Quarterly, 11, 176-188. M e y e r , J . W . ( 1 9 7 7 ) . The e f f e c t s o f A m e r i c a n Journal of Sociology,  J . (1988). research.  e d u c a t i o n as an 83(1),55-77.  Early Learning  institution.  M e y e r , J . W . ( 1 9 8 0 ) . L e v e l s o f the e d u c a t i o n a l s y s t e m and s c h o o l i n g e f f e c t s . In C . B i d w e l l & D . Windham, ( E d s . ) . The Analyses of Educational Productivity, Vol II. C a m b r i d g e , MA: B a l l i n g e r . Meyers, C . E . , A t t w e l l , A . A . & Orpet, R . E . (1968). P r e d i c t i o n of f i f t h g r a d e achievement from k i n d e r g a r t e n t e s t and r a t i n g d a t a . Educational and Psychological Measurement, 28, 457-463. M i l l e r , L . J . (1988). D i f f e r e n t i a t i n g c h i l d r e n with s c h o o l r e l a t e d p r o b l e m s a f t e r f o u r y e a r s u s i n g the M i l l e r a s s e s s m e n t f o r p r e s c h o o l e r s . Psychology in the Schools, 25, 10-15. M i l l e r , L . & S p r o n g , T . ( 1 9 8 6 ) . P s y c h o m e t r i c and q u a l i t a t i v e comparison of four preschool screening i n s t r u m e n t s . Journal of Learning Disabilities, 19, 480-484. M i l l e r , L . J . & S c h o u t e n , P . G . ( 1 9 8 8 ) . A g e - r e l a t e d e f f e c t s on t h e p r e d i c t i v e v a l i d i t y o f the M i l l e r a s s e s s m e n t f o r p r e s c h o o l e r s . Journal of Psychoeducational Assessment, 6, 99-106. M i l l e r , W . D . & N o r r i s , R . C . ( 1 9 6 7 ) . E n t r a n c e age and s c h o o l s u c c e s s . Journal of School Psychology, 6, 4 7 - 5 9 . M o r e n c y , A . & Wepman, J . ( 1 9 7 3 ) . E a r l y p e r c e p t u a l a b i l i t y l a t e r s c h o o l a c h i e v e m e n t . Elementary School Journal, 323-327.  and 73,  Murnane, R . J . ( 1 9 8 1 ) . I n t e r p r e t i n g the e v i d e n c e on s c h o o l e f f e c t i v e n e s s . Teachers College Record, 83, 19-35. N a g l i e r i , J . A . ( 1 9 8 8 ) . Draw-a-Person: A Quantitative Scoring System. Toronto: Harcourt-Brace, Jovanovich, Inc. N a g l i e r i , J . A . & M a x w e l l , S. ( 1 9 8 1 ) . I n t e r - r a t e r r e l i a b i l i t y and c o n c u r r e n t v a l i d i t y o f the G o o d e n o u g h - H a r r i s d r a w - a p e r s o n and M c C a r t h y . Perceptual and Motor Skills, 53, 343348. N a t i o n a l A s s o c i a t i o n f o r the E d u c a t i o n o f Young C h i l d r e n (NAEYC). P o s i t i o n Statement on S t a n d a r d i z e d T e s t i n g  of  166 Young C h i l d r e n 3 Through 8 y e a r s o f A g e . Children, 88, 4 2 - 4 7 .  (1988).  N o r t o n , L . ( 1 9 7 9 ) . The i d e n t i f i c a t i o n o f "at r i s k " c h i l d r e n . Special Education in Canada, 53(4),  Young  kindergarten 20-22.  O ' C o n n o r , A . ( 1 9 8 0 ) . E a r l y s c r e e n i n g - some p r o b l e m s and p r a c t i c e s . British Columbia Journal of Special Education, 4, 2 7 1 - 2 8 3 . P a g e t , K . D . & B r a c k e n , B . A . ( E d s . ) . ( 1 9 8 3 ) . The educational Assessment of Preschool Children. Grune & S t r a t t o n .  PsychoNew Y o r k :  P a g e t , K . D . & N a g l e , R . J . ( 1 9 8 6 ) . A c o n c e p t u a l model o f p r e s c h o o l a s s e s s m e n t . School Psychology Review, 15, 154-165. P a l a r d y , J . ( 1 9 6 9 ) . What t e a c h e r s b e l i e v e , what c h i l d r e n achieve. Elementary School Journal, 69, 3 7 0 - 3 7 4 . P a p a l i a , D . E . & O l d s , S.W. ( 1 9 7 5 ) . A Child's World: Through Adolescence. New York^ NY: M c G r a w - H i l l P a r k R . ( 1 9 7 8 ) . P e r f o r m a n c e on g e o m e t r i c f i g u r e p r e d i c t o r s of types of e r r o r s i n decoding. Research Quarterly, 14, 1 0 0 - 1 1 8 .  Infancy Book C o .  copying test Reading  as  P a t t i s o n , P . & G r i e v e , N . ( 1 9 8 4 ) . Do s p a t i a l s k i l l s c o n t r i b u t e t o sex d i f f e r e n c e s i n d i f f e r e n t t y p e s o f m a t h e m a t i c a l problems? Journal of Educational Psychology, 76, 678689. P e a r s o n , L . & L i n d s a y , G . ( 1 9 8 6 ) . Special Needs in the School: Identification and Intervention. Windsor, B e r k s h i r e , England: NFER-Nelson.  Primary  P e d h a z u r , E . ( 1 9 8 2 ) . Multiple Regression in Behavioral Research. New Y o r k , NY: H o l t , R i n e h a r t & W i n s t o n . P e r r i n , J . M . (1986). Topics in Early  C h r o n i c a l l y i l l c h i l d r e n : An o v e r v i e w . Childhood Special Education, 5, 1-11.  P e t e r s o n , J . M . (1989). Remediation i s Leadership, 4 6 ( 6 ) , 24-25.  no remedy.  Educational  P e t e r s o n , S . E . , De G r a c i e , J . S . & A y a b e , C R . ( 1 9 8 7 ) . A l o n g i t u d i n a l s t u d y o f the e f f e c t s o f r e t e n t i o n / p r o m o t i o n on a c a d e m i c a c h i e v e m e n t . American Educational Research Journal, 24, 1 0 7 - 1 1 8 .  167 Phye,  G . D . & Halderman, M . A . (1980). Cumulative r e t e n t i o n r a t e as an i n d e x o f academic p r o g r e s s : A s e c o n d l o o k . Journal of Learning Disabilities, 23(5) 13-14.  P i k u l s k i , J . J . (1972). A comparison of f i g u r e I . Q ' s among d i s a b l e d r e a d e r s . Journal of Disabilties, 5(3), 41-44. Pomplun, M. (1988). of Educational Pope,  R e t e n t i o n : The e a r l i e r the Research, 81, 2 8 1 - 2 8 7 .  d r a w i n g s and WISC Learning better?  Journal  J . , L e h r e r , B. & Stevens, J . (1980). A m u l t i p h a s i c r e a d i n g s c r e e n i n g p r o c e d u r e . Journal of Learning Disabilites, 23(2), 47-51.  Posno, R. (1982). E a r l y i d e n t i f i c a t i o n / i n t e r v e n t i o n : e d u c a t i o n a l f o p p e r y o r p r o m i s e . Special Education Canada, 56, 12-17.  in  P o t t e r , M . ( 1 9 8 7 ) . C h i l d r e n and c h r o n i c i l l n e s s . In A . Thomas & J . G r i m e s ( E d s . ) . Children's Needs: Psychological Perspectives. W a s h i n g t o n , DC: N a t i o n a l A s s o c i a t i o n o f School Psychologists. P u l l i s , M. & S m i t h , D . C . ( 1 9 8 1 ) . S o c i a l c o g n i t i v e d e v e l o p m e n t of l e a r n i n g d i s a b l e d c h i l d r e n : I m p l i c a t i o n s of P i a g e t ' s t h e o r y f o r r e s e a r c h and i n t e r v e n t i o n . Topics in Learning and Learning Disabilities, 1, 4 3 - 5 5 . R a n d e l , M . A . , F r y , M . A . & R a l l s , E . M . ( 1 9 7 7 ) . Two r e a d i n e s s m e a s u r e s as p r e d i c t o r s o f f i r s t - and t h i r d - g r a d e r e a d i n g a c h i e v e m e n t . Psychology in the Schools, 14(1), 37-40. R a p o p o r t , H . G . & F l i n t , S . H . ( 1 9 7 6 ) . Is t h e r e a r e l a t i o n s h i p between a l l e r g y and l e a r n i n g d i s a b i l i t i e s ? The Journal of School Health, XLVI, 139. R a u d e n b u s h , S.W. ( 1 9 8 8 ) . E d u c a t i o n a l a p p l i c a t i o n s h i e r a r c h i c a l l i n e a r m o d e l s : A r e v i e w . Journal Educational Statistics, 13, 8 5 - 1 1 6 .  of of  R a u d e n b u s h , S.W. ( 1 9 8 9 ) . E d u c a t i o n a l a p p l i c a t i o n s h i e r a r c h i c a l l i n e a r m o d e l s : A r e v i e w . Journal Educational Statistics, 2 3 ( 2 ) , 85-116.  of of  R a u d e n b u s h , S.W. & B r y k , A . S . ( 1 9 8 6 ) . A h i e r a r c h i c a l model f o r s t u d y i n g s c h o o l e f f e c t s . Sociology of Education, 59, 1-17. R a u d e n b u s h , S.W. & B r y k , A . S . ( 1 9 8 8 ) . M e t h o d o l o g i c a l a d v a n c e s i n a n a l y s i n g the e f f e c t s o f s c h o o l s and c l a s s r o o m s on s t u d e n t l e a r n i n g . In E . Z . R o t h k o p f ( E d . ) . Research in  168 Education, (pp.423-475). Washington, E d u c a t i o n a l Research A s s o c i a t i o n .  DC: A m e r i c a n  R a w l s , D . J . , R a w l s , J . R . & H a r r i s o n , C . W . ( 1 9 7 1 ) . An i n v e s t i g a t i o n of s i x - t o e l e v e n - y e a r - o l d c h i l d r e n with a l l e r g i c d i s o r d e r s . Journal of Consulting and Clinical Psychology, 36, 2 6 0 - 2 6 4 . R e y n o l d s , L . , E g a n , R. & L e r n e r , i n t e r v e n t i o n on p r e a c a d e m i c l i t e r a t u r e . Topics In Early 55.  J . (1983). E f f i c a c y of e a r l y d e f i c i t s : A review of the Childhood Education, 10, 47-  R i c h a r d s o n - K o e h l e r , V . ( 1 9 8 8 ) . Teachers' beliefs about at-risk students. Paper p r e s e n t e d a t the A n n u a l M e e t i n g o f t h e A m e r i c a n E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n , New O r l e a n s . R i t t e r , D . , D u f f e y , J . & F i s c h m a n , R. ( 1 9 7 4 ) . C o m p a r i s o n s o f t h e i n t e l l e c t u a l e s t i m a t e s o f the draw-a-man t e s t , Peabody p i c t u r e v o c a b u l a r y t e s t and S t a n f o r d - B i n e t (L-M) f o r k i n d e r g a r t e n c h i l d r e n . Psychology in the Schools, 11, 412-415. R o g e r , C , S m i t h , M . D . & Coleman, J . M . ( 1 9 7 8 ) . S o c i a l c o m p a r i s o n i n the c l a s s r o o m : The r e l a t i o n s h i p between a c a d e m i c a c h i e v e m e n t and s e l f - c o n c e p t . Journal of Educational Psychology, 70, 5 0 - 5 7 . R o s e n b e r g , J . B . & W e l l e r , G . M . ( 1 9 7 3 ) . Minor p h y s i c a l a n o m a l i e s and a c a d e m i c p e r f o r m a n c e i n young s c h o o l - c h i l d r e n . Developmental Medicine and Child Neurology, 15, 131-135. Rosenshine, time.  B . V . & B e r l i n e r , D . C . ( 1 9 7 8 ) . Academic engaged British Journal of Teacher Education, 4, 3-16.  R o s e n t h a l , R. & J a c o b s o n , L . (1968). Classroom. New Y o r k , NY: H a l t ,  Pygmalion in the R i n e h a r t and W i n s t o n .  R o u r k e , B . & O r r , R . R . ( 1 9 7 7 ) . P r e d i c t i o n o f t h e r e a d i n g and s p e l l i n g p e r f o r m a n c e s o f normal and r e t a r d e d r e a d e r s : f o u r - y e a r f o l l o w - u p . Journal of Abnormal Child Psychology, 5, 9-20. R u b i n , R . , Balow, B . , D o r l e , J . & Rosen, M. (1978). P r e s c h o o l p r e d i c t i o n o f low a c h i e v e m e n t i n b a s i c s c h o o l s k i l l s . Journal of Learning Disabilities, 12(10), 62-78. R u t t e r , M . ( 1 9 8 3 ) . S c h o o l e f f e c t s on p u p i l p r o g r e s s : R e s e a r c h f i n d i n g s and p o l i c y i m p l i c a t i o n s . Child Development, 54(1), 1-29.  169 R u t t e r , M . , Maughan, B . , M o r t i m o r e , P . , O u s t o n , J . & S m i t h , A . ( 1 9 7 9 ) . Fifteen Thousand Hours. L o n d o n , E n g l a n d : Open Books. S a b e r s , D . , C u s h i n g , K . , & S a b e r s , D . ( 1 9 8 7 ) . Sex differences i n r e a d i n g and mathematics a c h i e v e m e n t f o r m i d d l e s c h o o l s t u d e n t s . Journal of Early Adolescence, 7(1), 117-128. S a l v i a , J . , C l a r k , G . , & Yssledyke, J . (1973). Teacher r e t e n t i o n of s t e r e o t y p e s of e x c e p t i o n a l i t y . Exceptional Children, 39, 651-652. S a l v i a , J . & Yssledyke, Remedial Education.  J . ( 1 9 8 5 ) . Assessment in Special B o s t o n , MA: Houghton M i f f l i n .  and  S a m u e l s , S . J . ( 1 9 7 3 ) . S u c c e s s and f a i l u r e i n l e a r n i n g t o r e a d : A c r i t i q u e o f the r e s e a r c h . Reading Research Quarterly, 8, 200-239. S a n d o v a l , J . & Hughes, P . G . ( 1 9 8 1 ) . Success in nonpromoted first-grade children. F i n a l R e p o r t . D a v i e s , CA: U n i v e r s i t y o f C a l i f o r n i a . (ERIC Document R e p r o d u c t i o n S e r v i c e No. ED 212-371.) Sattler, J . Special Inc.  ( 1 9 8 2 ) . Assessment of Children's Abilities. (2nd E d . ) . B o s t o n ,  S a t t l e r , J . ( 1 9 8 8 ) . Assessment of Children. Jerome M . S a t t l e r , P u b l i s h e r .  Intelligence and MA: A l l y n & B a c o n , San D i e g o , C A :  Satz,  P. & F l e t c h e r , J . M . (1979). E a r l y s c r e e n i n g t e s t s : u s e s and a b u s e s . Journal of Learning Disabilities, 65-69.  some 12(1),  Satz,  P . & F r i e l , J . ( 1 9 7 4 ) . Some p r e d i c t i v e a n t e c e d e n t s o f s p e c i f i c reading d i s a b i l i t y : A p r e l i m i n a r y two-year follow-up. Journal of Learning Disabilities, 7(1), 48-55.  Satz,  P . & F r i e l , J . ( 1 9 7 8 ) . P r e d i c t i v e v a l i d i t y o f an a b b r e v i a t e d s c r e e n i n g b a t t e r y . Journal of Learning Disabilities, 1 1 ( 6 ) , 20-24.  Satz,  P . , F r i e l , J . & R u d e g e a i r , F . ( 1 9 7 6 ) . Some p r e d i c t i v e antecedents of s p e c i f i c reading d i s a b i l i t y : A two-, threeand f o u r y e a r f o l l o w - u p . In J . t . G u t h r i e ( E d . ) . Aspects of Reading Acquisition. B a l t i m o r e , MD: J o h n s H o p k i n s Press.  170  Satz,  P . , T a y l o r , H . G . , F r i e l , J . & F l e t c h e r , J . M . ( 1 9 7 9 ) . Some d e v e l o p m e n t a l and p r e d i c t i v e p r e c u r s o r s o f r e a d i n g disabilities: A s i x year f o l l o w - u p . In A . B e n t o n & D . P e a r l ( E d s . ) , Dyslexia: A Critical Appraisal of Current Theory. Oxford, England: Oxford U n i v e r s i t y P r e s s , (pp. 313-318).  S c h m i d t , S. & P e r i n o , J . ( 1 9 8 5 ) . K i n d e r g a r t e n s c r e e n i n g r e s u l t s as p r e d i c t o r s o f a c a d e m i c a c h i e v e m e n t , p o t e n t i a l , and p l a c e m e n t i n s e c o n d g r a d e . Psychology in the Schools, 22, 146-151. S c h w e i n h a r t , L . J . & W e i k a r t , D . P . (1986) E a r l y development p r o g r a m s : A p u b l i c i n v e s t m e n t Educational Leadership, 4 4 ( 3 ) , 4-12. S c o t t , L . H . (1981). Measuring i n t e l l i g e n c e with G o o d e n o u g h - H a r r i s d r a w i n g t e s t . Psychological 89, 483-505.  childhood opportunity. the Bulletin,  S h a p i r o , E . S . ( 1 9 8 7 ) . Behavioral Assessment in School Psychology. H i l l s d a l e , N J : Lawrence E r l b a u m A s s o c . S h a p i r o , E . S . (1988). P r e v e n t i n g academic f a i l u r e . Psychology Review, 2 7 ( 4 ) , 6 0 1 - 6 1 3 .  School  S h e p a r d , L . A . & S m i t h , M . L . ( 1 9 8 6 ) . S y n t h e s i s o f r e s e a r c h on s c h o o l r e a d i n e s s and k i n d e r g a r t e n r e t e n t i o n . Educational Leadership, 44(3), 78-86. Shepard, L . A . & Smith, M . L . (1987). E f f e c t s of k i n d e r g a r t e n r e t e n t i o n a t t h e end o f f i r s t g r a d e . Psychology In the Schools, 24, 3 4 6 - 3 5 7 . S h u a r d , H . ( 1 9 8 6 ) . The r e l a t i v e a t t a i n m e n t o f g i r l s and boys i n m a t h e m a t i c s i n t h e p r i m a r y y e a r s . In L . B u r t o n ( E d . ) , Girls Into Mathematics Can Go. L o n d o n : H o l t , R i n e h a r t , and Winston. S i l v e r , A . A . ( 1 9 7 8 ) . P r e v e n t i o n . In A . L . B e n t o n & D . P e a r l ( E d s . ) , Dyslexia An Appraisal of Current Knowledge. New Y o r k , NY: O x f o r d U n i v e r s i t y P r e s s . S i m n e r , M . L . ( 1 9 8 2 ) . P r i n t i n g e r r o r s i n k i n d e r g a r t e n and t h e p r e d i c t i o n o f a c a d e m i c p e r f o r m a n c e . Journal of Learning Disabilities, 2 5 ( 3 ) , 155-159. S i m n e r , M . L . ( 1 9 8 3 ) . The w a r n i n g s i g n s o f s c h o o l f a i l u r e : An updated p r o f i l e of the a t - r i s k k i n d e r g a r t e n c h i l d . Topics in Early Childhood Special Education, 3(3), 17-27.  171 S i m n e r , M . L . ( 1 9 8 5 ) . S c h o o l r e a d i n e s s and t h e d r a w - a - m a n t e s t : An e m p i r i c a l l y d e r i v e d a l t e r n a t i v e t o H a r r i s s c o r i n g system. Journal of Learning Disabilities, 18(2), 72-81. S l a v i n , R . E . ( 1 9 8 8 ) . S y n t h e s i s o f r e s e a r c h on g r o u p i n g i n e l e m e n t a r y and s e c o n d a r y s c h o o l s . Education Leadership, 46(1), 67-77. S m i t h , H.W. & Kennedy, W. ( 1 9 6 7 ) . E f f e c t s o f t h r e e e d u c a t i o n a l programs on m e n t a l l y r e t a r d e d c h i l d r e n . Perceptual and Motor Skills, 24, 174. Sorensen, A . B . & H a l l i n a n , M . T . (1977). A r e c o n c e p t u a l i z a t i o n o f s c h o o l e f f e c t s . Sociology of Education, 50, 273-289. Stehbens, J . , K i s k e r , C . T . & Wilson, and a t t e n d a n c e d u r i n g the f i r s t c h i l d h o o d c a n c e r . Psychology in  B. (1983). School behavior year of treatment of the Schools, 20, 223-228.  S t e i n b a u e r , E . & H e l l e r , M . S . ( 1 9 7 8 ) . The Boehm t e s t c o n c e p t s as a p r e d i c t o r o f a c a d e m i c a c h i e v e m e n t 2. Psychology in the Schools, 15, 357-360.  of in  basic grade  S t e v e n s , L . M . ( 1 9 8 7 ) . Assessment and i n t e r v e n t i o n w i t h e a r l y i d e n t i f i e d , e d u c a t i o n a l l y a t - r i s k c h i l d r e n : Some m e t h o d o l o g i c a l c o n s i d e r a t i o n s . European Journal of Special Needs Education, 2(1), 1-12. Stevenson, H.W., Parker, T . , Wilkinson, A . , Hegion, A . & F i s h , E . (1976). L o n g i t u d i n a l study of i n d i v i d u a l d i f f e r e n c e s i n c o g n i t i v e development and s c h o l a s t i c a c h i e v e m e n t . Journal of Educational Psychology, 68, 3 7 7 - 4 0 0 . S t o c k a r d , J . ( 1 9 8 0 ) . Sex i n e q u i t i e s i n t h e e x p e r i e n c e o f s t u d e n t s . In J . S t o c k a r d e t a l . ( E d s . ) , Sex Equity in Education. New Y o r k , NY: Academic P r e s s . S t o n e , B . , Swanson, P . & C u n d i c k , B . S p e c i a l e d u c a t i o n s c r e e n i n g s y s t e m : Group a c h i e v e m e n t t e s t . Exceptional Children, 55, 7 1 - 7 5 . Summers, A . A . & W o l f e , B . L . ( 1 9 7 7 ) . Do s c h o o l s d i f f e r e n c e ? The American Economic Review,  make a 67, 6 3 9 - 6 5 2 .  T a b a c h n i c k , B . & F i d e l l , L . ( 1 9 8 3 ) . Using Multivariate Statistics. New Y o r k , NY: H a r p e r & Row P u b l i s h e r s . T e l e g d y , G . ( 1 9 7 5 ) . The e f f e c t i v e m e n s s o f f o u r r e a d i n e s s as p r e d i c t o r s o f f i r s t g r a d e a c a d e m i c a c h i e v e m e n t . Psychology in the Schools, 12, 4-11.  tests  172 T h o r n d i k e , R . L . & Hagen, E . ( 1 9 7 1 ) . Measurement in Psychology and Education (4th Edition). Wiley.  and Evaluation New Y o r k :  Thurlow, M . L . , O ' S u l l i v a n , P . J . & Ysseldyke, J . E . (1986). E a r l y s c r e e n i n g f o r s p e c i a l e d u c a t i o n : How a c c u r a t e ? Educational Leadership, 44(3), 93-95. Thurlow, M . L . Ysseldyke, J . E . , Graden, J . L . & A l g o z z i n e , B. ( 1 9 8 3 ) . W h a t ' s ' s p e c i a l ' about the s p e c i a l e d u c a t i o n r e s o u r c e room f o r l e a r n i n g d i s a b l e d s t u d e n t s ? Learning Disability Quarterly, 6(3), 283-288. T i z a r d , B . , B l a t c h f o r d , P . , Burke, J . , Farquhar, G . , & P l e w i s , I . ( 1 9 8 8 ) . Young Children At School in the Inner City. H o v e , S u s s e x : Lawrence E r l b a u m . T j o s s e m , T . D . ( E d . ) ( 1 9 7 6 ) . Intervention Strategies for High Risk Infants and Young Children. B a l t i m o r e , MD: U n i v e r s i t y Park P r e s s . T r a m i l l , J . L . , Edwards, R . P . & T r a m i l l , J . K . (1980). Comparison o f t h e G o o d e n o u g h - H a r r i s d r a w i n g t e s t and the WISC-R f o r c h i l d r e n e x p e r i e n c i n g academic d i f f i c u l t i e s . Perceputal and Motor Skills, 50, 543-546. W a l d e n , R . & W a l k e r d e n e , V . ( 1 9 8 5 ) . Girls and mathematics: From primary to secondary schooling. B e d f o r d Way P a p e r s , 24. I n s t i t u t e of E d u c a t i o n , U n i v e r s i t y of London: Turnaround Distribution Ltd. W a l s h , D . J . ( 1 9 8 8 ) . The two-year route to first grade: The use of testing to validate decisions based on class and age. Paper p r e s e n t e d a t the A n n u a l M e e t i n g o f the A m e r i c a n E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n i n New O r l e a n s . W a n c z y c k i , W. ( 1 9 8 3 ) . The k i n d e r g a r t e n e a r l y i d e n t i f i c a t i o n component o f B i l l 82. Special Education in Canada, 5 7(1), 25-32. W e d e l l , K . & R a y b o u l d ( E d s . ) ( 1 9 7 6 ) . The early identification of educationally 'at risk' children. E d u c a t i o n a l Review O c c a s i o n a l P u b l i c a t i o n s Number s i x . E d g b a s t o n , B i r m i n g h a m : U n i v e r s i t y o f Burmingham. W e i n t r a u b , S. ( 1 9 6 6 ) . What r e s e a r c h s a y s t o the r e a d i n g teacher. Sex d i f f e r e n c e s i n r e a d i n g a c h i e v e m e n t . The Reading Teacher, 20, 1 5 5 - 1 6 5 . Weintraub,  S.  ( 1 9 6 7 ) . What r e s e a r c h s a y s t o  the  reading  173 teacher. R e a d i n e s s measures f o r p r e d i c t i n g r e a d i n g a c h i e v e m e n t . The Reading Teacher, 20, 5 5 1 - 5 5 8 . W e l l s , M.G & P e t e r s o n , G . V . (1978). K i n d e r g a r t e n b e h a v i o r r a t i n g as a p r e d i c t o r o f f i r s t - g r a d e a c h i e v e m e n t . Journal of Learning Disabilities, 22(6), 17-20. Wendt, R . N . ( 1 9 7 8 ) . K i n d e r g a r t e n e n t r a n c e a s s e s s m e n t : w o r t h t h e e f f o r t ? Psychology in the Schools, 15,  Is i t 56-62.  White, K . R . (1986). E f f i c a c y of e a r l y i d e n t i f i c a t i o n . Journal of Special Education, 19, 401-415.  The  W h i t e , M. (1979). A f i r s t - g r a d e i n t e r v e n t i o n program f o r c h i l d r e n a t r i s k f o r r e a d i n g f a i l u r e . Journal of Learning Disabilities, 2 2 ( 4 ) , 27-32. Wick,  J . W . ( 1 9 7 3 ) . Educational Measurement. and How Will We Know When We Get There? Chas. E . M e r r i l l Pub. Co.  Where Are We Going C o l u m b u s , OH:  W i l e y , D . E . & H a r n i s c h f e g e r , A . (1974). E x p l o s i o n of a myth: Q u a n t i t y o f s c h o o l i n g and e x p o s u r e t o i n s t r u c t i o n , major e d u c a t i o n a l v e h i c l e s . Educational Researcher, 3, 7-12. W i l l m s , J . D . ( 1 9 8 4 ) . S c h o o l e f f e c t i v e n e s s w i t h i n t h e p u b l i c and p r i v a t e s e c t o r s : An e v a l u a t i o n . Evaluation Review, 8, 113135. W i l l m s , J . D . ( 1 9 8 5 ) . The b a l a n c e t h e s i s : C o n t e x t u a l e f f e c t s o f a b i l i t y on p u p i l s ' O - g r a d e e x a m i n a t i o n r e s u l t s . Oxford Review of Education, 22(1), 33-41. W i l l m s , J . D . ( 1 9 8 6 ) . S o c i a l c l a s s s e g r e g a t i o n and i t s r e l a t i o n s h i p to p u p i l s ' examination r e s u l t s i n S c o t l a n d . American Sociological Review, 51, 2 2 4 - 2 4 1 . W i l l m s , J . D . ( 1 9 8 7 ) . D i f f e r e n c e s between S c o t t i s h a u t h o r i t i e s i n t h e i r examination attainment. of Education, 2 3 ( 2 ) , 211-232.  education Oxford Review  W i l l m s , J . D . & C h e n , M. ( 1 9 8 9 ) . The e f f e c t s o f a b i l i t y g r o u p i n g on t h e e t h n i c a c h i e v e m e n t gap i n I s r a e l i e l e m e n t a r y s c h o o l s . American Journal of Education, ???, ???-???. W i l l m s , J . D . & J a c o b s e n , S. ( 1 9 9 0 ) . Growth i n m a t h e m a t i c s s k i l l s d u r i n g t h e i n t e r m e d i a t e y e a r s : Sex d i f f e r e n c e s and s c h o o l e f f e c t s . Perspectives on Research in Mathematics Education, 157-174.  174 W i l l m s , J . D . & K e r r , P . D . ( 1 9 8 7 ) . Changes i n sex i n S c o t t i s h e x a m i n a t i o n r e s u l t s s i n c e 1976. Early Adolescence, 7(1), 85-105.  differenaces Journal of  W i l l m s , J . D . & Raudenbush, S.W. ( 1 9 8 9 ) . A l o n g i t u d i n a l h i e r a r c h i c a l l i n e a r model f o r e s t i m a t i n g s c h o o l e f f e c t s and t h e i r s t a b i l i t y . Journal of Educational Measurement, 2 6 ( 3 ) , 209-232. W i l s o n , B . J . & R e i c h m u t h , M. ( 1 9 8 5 ) . E a r l y s c r e e n i n g p r o g r a m s : When i s p r e d i c t i v e a c c u r a c y s u f f i c i e n t ? Learning Disabilities Quarterly, 8, 1 8 2 - 1 8 8 . W i n k l e r , D . R . ( 1 9 7 5 ) . E d u c a t i o n a l a c h i e v e m e n t and s c h o o l p e e r g r o u p c o m p o s i t i o n . J o u r n a l of Human Resources, 10, 189204. W o l f e n d e n , G . A . ( 1 9 8 0 ) . The e f f e c t s o f k i n d e r g a r t e n i n t e r v e n t i o n on the need f o r c o n t i n u i n g i n d i v i d u a l a s s i s t a n c e . British Columubia Journal of Special Education, 4, 3 5 5 - 3 6 3 . Wood,  C , P o w e l l , S. & K n i g h t , R . C . ( 1 9 8 4 ) . P r e d i c t i n g s c h o o l readiness: The v a l i d i t y o f d e v e l o p m e n t a l a g e . Journal of Learning Disabilites, 27(1), 8-11.  Yoss,  K . A . & D a r l e y , F . L . (1974). Developmental a p r a x i a of s p e e c h i n c h i l d r e n w i t h d e f e c t i v e a r t i c u l a t i o n . Journal Speech and Hearing Research, 17, 399-416.  of  Y s s e l d y k e , J . E . , Thurlow, M . L . , O ' S u l l i v a n , P . O . & Bursaw, R . A . ( 1 9 8 6 ) . C u r r e n t s c r e e n i n g and d i a g n o s t i c p r a c t i c e s i n a s t a t e o f f e r i n g f r e e p r e s c h o o l s c r e e n i n g s i n c e 1977: I m p l i c a t i o n s f o r the f i e l d . Journal of Psychoeducational Assessment, 4, 1 9 1 - 2 0 1 . Z e i t l i n , S. ( 1 9 7 6 ) . Kindergarten Screening: Identification of Potential High Risk S p r i n g f i e l d , I L : C h a r l e s C . Thomas.  Early Learners.  Z u z o v s k y , R. & A i t k i n , M . ( i n p r e s s ) A T w o - l e v e l model o f a c h i e v e m e n t i n the s e c o n d IEA s c i e n c e s t u d y i n  analyses Israel.  175  Appendix A Descriptors of Physical Problems  Visual problems: - Wears glasses - Identified as visually impaired Hearing problems: - History of ear infections - Conductive hearing loss - Identified hearing impaired Allergies Chronic  illness  Physically handicapped Other  Three sources were used to obtain indicators of physical problems for the subject sample.  The school-based learning assistance teachers  completed a data checklist with the names of a l l subjects and the above categories of physical problems.  The d i s t r i c t - l e v e l resource teachers  responsible f o r categorical services completed a similar c h e c k l i s t . L a s t l y , the school records were reviewed to locate references to the above l i s t e d problems. Although approximately twenty percent of the sample were reported to have physical problems, the individual variables lacked explanatory power in exploratory analyses and thus, were combined to act as a control variable.  Appendix B Number of Pupils Identified with Physical Problems and Number Receiving Interventions (Based on Achieved Sample - 459 Females, 498 Males)  Physical Problems  Male  Total  1  0  1  Wears Glasses  46  32  78  History of Ear Infections  17  16  33  Conductive Hearing Loss  6  11  17  Hearing Impaired  8  5  13  Allergies  41  61  102  Chronic Illness  10  7  17  3  1  4  97  103  200  Learning Assistance  23  68  91  Extended Primary  29  60  89  One or Both Interventions  42  99  141  Visually Impaired  Physical Handicap One or more physical problems  Female  Interventions  177  Appendix C Technical Information - Draw-A-Person  Authors:  Goodenough's Draw-A-Man Test  Scoring system -Harris, D.B. Date:  (A modified scoring system was  applied.  1963  Purpose:  The Draw-a-person test is a screening instrument f o r use as a  nonverbal measure of cognitive a b i l i t y . Age Range:  5 years through 15 years.  Administration: Description: requested  Can be administered to groups by classroom teacher. The  child  i s provided  a pencil  to draw a complete person.  No  and  blank  additional  provided although encouragement to complete the task may The  score  i s calculated  and  points are of  given  body p a r t s , not  paper and  instructions  is are  be given. for  inclusion,  e l a b o r a t i o n , and  proportionality  f o r realism or  esthetic q u a l i t y .  The total raw score is compared to norms for the point  scale or converted to standard scores (M=100, SD=15). Reliability:  S p l i t - h a l f r e l i a b i l i t y r=.89 Test-retest r e l i a b i l i t y r=.74 (Scott,  1981)  Interrater r e l i a b i l i t y .70 to .90 ( S a t t l e r , Validity:  1982)  With Stanford-Binet IQ Scores .36 to .55 With Stanford-Binet MA  Scores  Duffy & Fischman, 1974).  .26 to .92  (Ritter,  178  Appendix D Technical Information - Mann-Suiter Visual Motor Screen Authors: Date:  P. Mann, P. Suiter and R. McClung  (based on norms from Ilg & Ames, 1964)  Purpose:  To measure the degree to which  visual  perception  and motor  behavior are integrated. Age Range:  Three years to nine years.  Description:  Four designs are presented to the student.  given three chances  The student i s  to copy each design, but only the best e f f o r t i s  counted. (Reliability Suiter  and v a l i d i t y  information was not available f o r the Mann-  but form-copying  validity.  tests  have  satisfactory  r e l i a b i l i t y and  S t a t i s t i c s are reported for similar tests.)  Reliability:  Test-retest of a similar form-copying test  (Beery, 1982)  range from .63-.92. Validity:  Correlations  between some form-copying tests  and readiness  tests i n kindergarten range from .50 to .70 (Berry, 1989; S a t t l e r , 1988).  Normative Data (Ilg and Ames, 1964) Design 1.  A child  3 years, of age should be able to make a s i n g l e  circle.  2.  A rectangle shape i s normative for children after age 4.  3.  A triangle i s normative for g i r l s after age 5 1/2 and f o r boys after age 6.  4.  The diamond i s normative for children after age 6.  179  Appendix E Technical Information - Kindergarten Language Screening Test  Test:  Kindergarten Language Screening Test (KLST)  Authors: Date:  Gauthier, S. and Madison, C.L  1978  Purpose:  The KLST i s designed to compare kindergaraten pupil's language  a b i l i t i e s to a level appropriate for their age and grade. Age Range:, 48 to 83 months. Administration: Description:  Classroom teacher administers to individual p u p i l s . The KLST  expressive language.  i s designed  to measure  both  The items on the test include:  r e c e p t i v e and  giving f u l l  name  and age; name the primary colors; count to thirteen; identify major body parts;  follow  a sequential command and demonstrate understanding of  prepositional concepts; repeat sentences of varying length and complexity and  a spontaneous language sample e l i c i t e d  throught  the use of s e r i a l  photographs. A cut-off point for the total test i s used to indicate likelihood of later school problems and need for further diagnostic t e s t i n g . Reliability:  Test-retest correlation .87 Kuder-Richardson  Validity:  R e l i a b i l i t y Coefficient .86  Construct v a l i d i t y - correlation  .70 between KLST  and Boehm Test of Basic Concepts. Correlation with subtests of the ITPA = .36, .37, .40. Predictive administered  Validity:  the Northwestern Syntax Screening  Basic Concepts. functioning  30 children who obtained  low KLST scores were  Test and Boehm Test of  Twenty-three (82%) of the 30 low scoring students were  below grade l e v e l , had repeated  Special Education.  a grade or been placed in  180  Appendix F Technical Information - Deverell Test of Letters and Numbers  Test:  Deverell Test of Letters and Numbers  Date:  Derived from a study i n Saskatoon in the late 1960's. Book published 1974.  Purpose:  The purpose of this  test  i s to evaluate  the child's  visual  perception of symbolic material. The child must have learned to perceive visually  the d i s t i n c t i v e  another.  features which differentiate  Uppercase and lower case  one symbol from  letters and the numerals 1-12 are  included in t h i s t e s t . Age Range:  This i s a readiness test appropriate f o r assessing a child's  knowledge of letters regardless of age. from a population kindergarten,  of "school  scored  The tables below were derived  beginners".  significantly  Children who  than those who  had attended  had not attended  kindergarten. Administration: Description: capital  Classroom teacher administers.  A child  i s presented with the visual  l e t t e r s then small l e t t e r s .  symbols f o r numbers,  The examiner points to a l e t t e r and  the c h i l d i s asked to name the number or l e t t e r . R e l i a b i l i t y and V a l i d i t y information was not available f o r this test but, the l i t e r a t u r e review identified several studies which reported a task of identifying  letters  and numbers to be a good predictor of achievement  ( C h a l l , 1967; Jansky & de Hirsch, 1972; Adelman & Feshbach, 1971).  Percent of Correct Response in Descending Order: A b i l i t y to Name Numbers Shown  Test item Percent  1  10  3  95  93  92  6 92  2 92  4  12 92  92  7 91  91  5  0 91  8 87  9 83  Average number of numbers known, 11; average percent 91.  181  Percent of Correct Response in Descending Order: A b i l i t y to Name Capital Letters Shown  Test item Percent  Test item Percent  X  0  A  6  C  Z  S  H  K  J  P  D  F  93  92  85  83  80  80  78  76  74  73  72  72  71  M  W  Q  E  Y  T  V  G  N  U  68  67  65  65  63  62  60  60  59  56  Average number of capital  letters known, 19; average percent, 72.  Percent of Correct Response in Descending Order: A b i l i t y to Name Small Letters Shown Test item Percent  Test item Percent  0  X  s  z  m  c  j  P  k  y  w  r  n  91  89  78  76  73  72  68  68  66  65  65  63  62  e  i  V  b  a  f  h  d  1  u  t  g  q  61  60  60  57  56  54  50  49  45  45  45  38  26  Average number of small letters known, 16; average percent, 61 l e t t e r s .  182  Appendix G Technical Information - Canadian Tests of Basic S k i l l s (CTBS)  Test:  Canadian Test of Basic S k i l l s (CTBS)  Editors: Date:  Hieronymus, A.N. and King,  E.M.  1976  Purpose:  The CTBS was  determine  the  designed to serve several purposes including:  developmental  level  of  each pupil  i n order  to  to adapt  materials and instructional procedures to individual needs and a b i l i t i e s ; to  diagnose  strengths and  weaknesses of  individual  students  and  of  groups; to provide information useful i n making administrative decisions, to  assess the effects  of alternate methods of  instruction conditions,  experimentation and innovation. Age Range:  Grade 3 to Grade 8 (Age 9 to 14).  Administration:  Administered  by  classroom  teachers  to  groups  of  students. Description:  The CTBS are concerned  only with generalized i n t e l l e c t u a l  s k i l l s and a b i l i t i e s and do not provide separate measures of achievement in content subjects. The s k i l l s measured by the CTBS are c l a s s i f i e d into five  major  areas:  v o c a b u l a r y , r e a d i n g , language, work-study  and  mathematics. Reliability:  Split-Halves R e l i a b i l i t y Analysis (Tables provided i n the  Technical Manual) Pearson product-moment correlations .72 to .98. Validity:  The CTBS were not designed as aptitude tests nor as predictors  of future academic success. validation  of test  content  individual test items" (p.7).  " A l l the commonly used p r i n c i p l e s have been applied i n the  in the  preparation of  The test was constructed to "correspond to  the widely accepted goals of instruction in schools across the nation" (p.41).  The v a l i d i t y of the CTBS i s dependent on how  closely the  on the test match the objectives of instruction within the d i s t r i c t .  items  183  Appendix H Characteristics of Schools i n the Study  School  School Size  Rural=0 Urban=l  School Mean-Ability  Standard Deviation Mean-Abili1  1  7  1  96.57  20.00  2  10  0  97.19  11.06  3  10  0  99.00  12.27  4  22  0  99.03  14.77  5  17  0  100.44  14.49  6  20  0  101.53  10.49  7  39  1  101.79  11.31  8  45  0  103.53  15.90  9  11  1  103.91  7.46  10  8  1  104.29  6.79  11  26  1  104.59  8.42  12  23  0  104.64  11.66  13  65  0  105.01  12.04  14  20  1  105.32  11.43  15  47  1  105.35  12.92  16  33  0  106.10  16.73  17  38  0  106.53  12.90  18  23  1  106.94  13.94  19  55  0  107.18  11.91  20  53  0  107.38  13.95  21  45  1  107.42  11.01  22  8  1  107.57  9.55  23  52  0  108.10  13.19  24  48  0  108.33  12.83  25  12  0  108.83  11.82  26  68  1  108.93  10.55  27  39  0  110.10  13.66  28  38  0  110.27  12.39  29  55  0  116.67  9.37  30  15  1  117.27  10.38  184  Appendix I  Use of Grade Equivalent Scores  The  choice of the metric to be used in analysis  i s important  ensure i n t e r p r e t a b i l i t y of findings and to ensure necessary assumptions are met. t h i s study was use of GE's  The  selection of grade-equivalent  to  statistical  (GE) scores f o r  determined after reviewing the literature regarding  the  and after exploratory analyses to ensure the CTBS GE's  met  the assumptions necessary for s t a t i s t i c a l analysis. The  CTBS test  scores for each student  can  be  represented  as  raw  scores, percentiles, or grade-equivalent scores. (Other types of scores, such as stanines, standard obtained  scores, or normal curve  equivalents can  be  from simple one-to-one transformations of p e r c e n t i l e s , and  so  w i l l not be considered separately.) The appropriate metric depends on the use of the reported score. Percentile scores are useful for representing relative  standing  amongst peers, or  f o r comparing  an  individual's  standing across subject areas. Percentiles are not useful f o r comparing school performance, or for examining the relationship between predictor variables  and  performance, because they  have unequal  intervals:  they  spread out raw scores in the middle of the d i s t r i b u t i o n and squeeze them together at the extremes. Raw  scores  (1984) r e f e r s  and  grade-equivalent  to as developmental  scores  (GE  scores), which Hoover  scores, are appropriate f o r making  comparisons between schools, or for examining relationships with measures. Scale values are closer to being equal  i n t e r v a l . The  other  problem  185  with raw scores, however, i s that they are less easily interpreted; t h e i r interpretation requires translation to a metric that shows the level which various items are mastered. Grade equivalent scores do  not  at  have  t h i s problem. Some researchers have eschewed GE  scores because they do not have  equal intervals at the extreme values on a test (e.g., Angoff, 1971; & S l i n d e , 1977;  Linn  Horst, 1986). This i s a problem p a r t i c u l a r l y when making  diagnostic assessments based on an individual's scores. However, unless a school  had  a large percentage of pupils who  scored  at the  extremes,  average scores for a school would be about the same i f based on either raw  scores  or GE's.  The  same would be  true of c o r r e l a t i o n s  between  outcomes and predictor variables. For example, Hoover (1984) showed that the correlation between school mean GE derived from forcing raw grade, were above  .995  scores and  developmental  scores  scores onto a normal d i s t r i b u t i o n within each for three separate measures at three  separate  grade l e v e l s . In t h i s  study, t h e r e f o r e , I am  confident that the  relationships  between kindergarten screening scores and outcome scores would not d i f f e r substantially i f raw scores had been used. S i m i l a r l y , i f I had  estimated  the predicted raw score for an "at risk" c h i l d , and then transformed i t to a GE  score, I expect the results would be v i r t u a l l y  problem  with  using  GE  scores  (or  f o r that  matter  percentiles) for this purpose is that there are few s k i l l s mastered by these pupils. Estimates than f o r estimates  of pupils nearer  the same. The raw  scores  items which  or  cover  therefore are more unstable  the middle of the  distribution.  Although less stable, they are not necessarily biased. If the estimation  186  of  the e f f e c t s  of intervention  are the main purpose of a study, a  researcher would achieve more accurate estimates by using the level of test appropriate for this subpopulation.  Appendix J :  Data Plots Reflecting  Interventions  /  I!  i  7°  !I  .©:  ! • • .• .9  •9  .9  Its? o c e  I© 8*.  0  ryi  CU  57 ©J©  CU  © 9»  ^ 7  ©  ©  &  a  CO  ©  03  /  ©  o  c  h  /  / *° <o «o ^ •?  /  /  0  /  «o  "°  0  «*  *  £  <o  Draw-a-Person Scores vs. Grade 3 CTBS Mathematics Scores Grade Equivalent Score  70  •  All Slud«nl«  O Learning Assistance  •  fxlsndsd Primary Schooling  65 60 55 50  •  45  .  *  B  t.. '  0  « • • • • .  • .J3  ^P>-  '- *  '  • "  •  •  . -  H  0  40 35  •  13.  30-  ID  25 0  20 -  0  •  ••  •  *  B  15 0  T  5  ~i  10  1  r  15 20 Draw-a-Person Score  25  30  KLST vs. Grade 3 CTBS Reading Scores 70  Grade Equivalent Score  •  O L.ornlng Aislslanc.  • All Slud.nti  Extended Primary Schooling  -i  65 60 55 *  50 -  - . . . . . . . ..... . ,  45 40 13 0  0  35-  «  &  Tar-. . •© - v ft «  0  30 -  0  •  • B G*3  0  15 0  —  a. 4>  25 20 -  •• *" • . . . . . a . .  ra  0  r  0 *  D 0  B0  . • •  • 0  •••  •• • **  * •*0  •  a o  10  15 KLST  20  25  30  191  Appendix K: Graphic Representation of Predictive Utility  Percentages of Students Correctly Identified As At-Risk in Reading at Grade 3 Level Fbr Varying Cut-off Scores on the Draw-a-Person Percentage of Students  Draw-a-Person Score  Percentages of Students Correctly Identified As At-Risk in Mathematics at Grade 3 Level For Varying Cut-off Scores on the KLSTT  100  Percentage of Students  90 80 70  •A  60  Unn«c*saary To I  50 40 ' / / / ' / / A ' / / ' '  30 20 10  •  /  * / • t  /  '  * >''/t'(-'\'i-<  0  ** y ' ' - • ' ** j *  *  A  s * *  .  10  20  KLST Score  u>  Percentages of Students Correctly Identified As At Risk in Mathematics at Grade 3 Level For Varying Cut-off Scores on the Draw-a-Person Percentage of Students  100 90 80 70 60 50 40 30 20  Should Hovo Inlorvonoc  9m, 1 1 1 wmmmm. •mmmmmmmms Unnecessary To Inlcrvono  10 0  10  20 Draw-a-Person Score  30 <^3  Appendix Table 1  Summary of Selected Prediction Performance Studies  Study  Sample  Time  Screening Measures  Outcome Measures  Analysis  Badian (1986)  208 males, white lower mid SES  K - Gr.3  Holbrook Screening Battery WPPSI, Language Sample Naming, Visual Motor, DAP  Stanford Ach.Test  Regression ANOVA  Badian & Serwer 300 K children (1975) 60 at risk 37 nt 25 f  K - Gr.l  DAP, Primary Mental Geometric form copy, A test of letter naming MRT  WISC or WPPSI Metropolitan Ach. Test  ANOVA N/S  Book (1974)  725 K All SES, races & religions  K - Gr.2  SIT Bender-Gestalt  Scott Foresman Reading  Correlation  gr.l r=.99 gr.2 r=.99  Book (1980)  193 K children 105 m 88 f  K - Gr.3  Kind. Evaluation of Learning Potential(KELP), BenderSIT  Stanford Ach. Tests Gestalt,  Correlation 2-way ANOVA  gr.l gr.2 gr.3 gr.4  Book (1980b)  472 K, 193 gr.4 105 m, 88 f  K - Gr.4  KELP,Bender-Gestalt,SIT  Stanford Ach. Tests  Correlation 2-way ANOVA  Busch (1980)  1052  B 1- End 1 Cognitive Abilities Test Dev. Test of Visual Motor Integration (VMI),Pre-Reading Procedures, Stanford Early School Achievement,MET Readiness, Boehm Concepts Behavior Rating  Gates-MacGinitie Reading  Correlation Regression  gr.l r=.76 gr.2 r=.71 gr.3 r=.72 gr.4 r=.62 r's=.01-.68 R=.76  Correlatioi R=.65  r=.76 r=.71 r=.72 r=.62  Screening Measures  Outcome Measures  Analysis  Ayres-Sensory Motor Schonell Integration, Finger ID.Doren Fine Motor Coord. Frostig-Visual PerceptionStanford Auditory Tests, PPVT  Reading Test t- tests Diag. Reading Test of Word Recognition, Diagnostic Test  Correlation Regression  r=.24 - .57 R=.81  Hayes Early ID Listening Response  MRT  Correlation  gr.l r=.62 gr.2 r=.81  Duffy, Ritter, 82 Caucasian K- 2 & Fedner (1976) 35 nt 47 f SES lower-up.mid.  Visual Motor Integration Draw-a-Man (Goodenough)  Stanford Ach. Test  Correlation  r=.20  Dunleavy, 141 kindergartner K Hansen, Szasz & 62 m Baade (1981) 79 f SES low, mid, high  Human Figure Drawing  MRT, Met. Ach. Test  ANOVA  Ferinden, 67 K children Jacobsen & Linden  WRAT, Evanston Early ID Scale, Bender Gestalt, MRT  Pre- post same  Correlation  Eaves, Kendall, 228 children & Crichton 25 (random sel) (1972) 25 (matched)  DAP, Bender, Name printing, pencil use, Wepman, words in a story, word recognition  Beery VMI, WISC, ITPA, Kephart Motor Survey MRT  t- tests Correlation Regression reported)  Feshbach, Adelman, & Fuller (1974)  WPPSI, Otis Lennon Group IQ, deHirsch Predictive Index, Student Rating Scale, Prim.A,  Cooperative Primary Correlation Gates Mac Ginitie Reading Regression Comprehension  Study  Sample  Butler (1979)  392 K children K- 2 204 m 188 f Mean age 5yr.8mo. Representative SES  Buttram, Covert, 121 K children & Hayes (1976) private school  Time  K- K  Correlation  .46  Pre. r=.28 Post r=.76  (1970)  888 K K (children IQ <90 eliminated)  r= .40 - .66 (means  R= .57 w/Gates R= .63 w/ de H i r s c h Prediction 74% M  Study Fletcher & Satz (1982)  Sample 195 white males  Time K- 6  Analysis  Screening Measures  Outcome Measures  PPVT, VMI, Recognition Alphabet Recitation  MRT (Four groups Discriminant Severely disabled = 53 Function Mildly disabled = 28 Analysis Average = 79 Superior = 35  Correlation 77% accuracy  Galante, Flye & Stephens (1972)  114 K K - 6 mid to upper class  Birth & developmental history, Medical examination  Dominance test, speech Three groups evaluation, group IQ, Calif.Ach. Tests, Stanford Ach. Tests,WISC  Glazzard (1982)  107 K 50 m 57 f .  Teacher rating, Gates MacGinitie, Readiness Skills each grade level  Gates- MacGinitie Reading Regression Pairwise ANC0VA  Goldman & Velasco (1980)  123 K 69 nt 51 f  Draw-a-person  Koppitz Scale of Emotional Indicators  Haring &  1200 K  Stanford Binet, Teacher Observation  ITPA, Detroit TLA, WRAT, Correlation r=.49 -.71 Aud. Discrim., Beery VMI, Prin. Componentslst.factor Perc.Motor Survey 20%  Test-visual seq., auditory seq., visual w/WRAT & auditory space w/rating  Wrat, Bender- Gestalt, Draw-a-person  Correlation Regression  Ridgway (1967)  K- 6  gr.l R= .77 R= .75 R= .57  Inter-rater Rel. Correlation r=.35 -.64  Hartlage & Lucas (1972)  44 K  K- 1  Hartlage & Lucas (1973)  1132 K  Bl - El  MRT  WRAT Teacher ranks  t-tests ANOVA  Jacob, Snider & Wilson (1988)  463 K 51* m 94% caucasion  K- 1  Clymer-Barrett Readiness Test  Stanford Ach. Test Reading Matrix  Regression specificity sensitivity  R=.78 R=.77  R=.65 .97 .43  Study  Sample  Time  LaTorre, Hawkhead, Kawahua & Bilow (1982)  796 K K- 1 Teacher's predictions  Screening Measures  Outcome Measures  Analysis  Correlation  McCarthy Screening Test Ginn 720 series level  Evaluative survey  ANOVA Disc. Function average Matrix  77.1% poor 67.9% Overall hit 77%  Lewis (1980)  86 75 44 m 37 m  K- 2  English Picture Vocab. Test Test, Croydon checklist  Young's Group Reading Test  Reported as % Use of screening did not improve hits beyond chance.  Lindquist (1982)  351  K -1,2,3  Denver Developmental Screening Test  Gates-MacGinitie Reading Test  Correlation  r=.009 - .32 gr.l r=.46 gr.2 r=.29 gr.3 r=.27  Meyers, Attwell & Orpet (1968)  57 K 25 m 32 f  K- 5  13 Individually admin, tests (motor, perception, Ravens matrices & Binet digits)  Calif. Ach. Test Calif. Mental Maturity  Correlation Regression  all sig R=.63 -.76  Miller (1988)  338  Pre.K - 3 Miller Assessment for Preschoolers  Retention Correlation Teacher observation-poor t-tests Received special services  Miller & Schouten (1988)  338 184 m 154 f  Pre.K - 3 Miller Assessment for 309 normal Preschoolers 29 at-risk  Woodcock- Johnson Ach. Reading, Math & Lang.  Correlation Regression  R=.18  Pope, Lehrer & Stevens (1980)  46 105 35 m 42 m 11 f 63 f  K- 5 Kind. Reading Screening high & low Battery, WRAT, Teacher checklist, SIT  Wookcock Reading Mastery  ANOVA Correlation  Differed Sig r=.50  Randel, Fry & Ralls (1977)  153 K  K - 1 & 3 DAP, counting, Gen. Info., mid class MRT, Prim. Mental Abilities  Stanford Ach. Test  Regressions  R=.34 -.57  -.59  vo oo  Time  Screening Measures  Study  Sample  Rourke & Orr (1977)  23 normal readers 1&2 -5&6 MAT, WISC, PPVT, WRAT, 19 retarded " underlining test  Analysis  Met. Ach Test, WRAT, PPVT, underlining test  Regression Disc.  R=.82 73.7%  Stanford Ach. Test  Correlation  r=.50 -.70  Correlation  Rubin, Balow, 732 Dorle, & Rosen (1978)  K -1  Satz & Friel (1974)  497  K- 1 22 Variables -PPVT, VMI, white,male alphabet recitatation,  10 item scale of reading no readiness-advanced reader  Disc. Regression  84.4% R=.81  Satz & Friel (1978)  28 104 13 m 54 m 15 f 50 f  K- 2 Black/ White  Modified Screening, finger localization. VMI, PPVT alphabet recitation, etc.  IOTA, classroom reading level materials  Matrix  Accurate 90% severe 69% mild  Schmidt & Perino (1985)  378 K 201 m 177 f  Vane-Test of Language Vane Kindergarten Test Draw-a-man, perc.motor  Met. Ach Test Otis- Lennon Ability Test  Regressions  R=.48 -.50  Simner (1982)  166 K 79 m 87 f  K- K  41 reversible letters and numbers  Teacher's rating rank ordering  Correlation Matrix  r=.53 -.63  Silver, Hagin Beecher (1978)  2319 K 51.4% m 48.6% f  K -1,2,3,4 SEARCH -ten component test  WRAT oral reading  Matrix  10% false & neg.  Simner (1985)  118 61 in 57 f  Draw-a-man  Criterion. Ref. Read & & Math, WRAT, Simner Printing, Developmental Tasks for K Readiness  Correlation  rs=.52 -.57  Stevenson, et al. (1976)  255 K 133 m 122 f SES Ave-Hi Mid  11 Cognitive measures 14 Psychometric tasks Teacher ratings -13 vars Ach Test (Gr.3)  WRAT (Gr.2), MET Readiness (Gr.l), Gray Oral Reading, Stanford  t-tests Correlation ANC0VA Regression  R=.56 -.77  K -1,2,3  MRT  Outcome Measures  VO VO  Study  Sample  Time  Screening Measures  Outcome Measures  Wells & Peterson (1978)  111 K  K-1  KELP -Kindergarten Teacher Checklist for Potential Learning Problems  Devereux Behavior Rating Correlation Iowa Test of Basic Skills t-tests Regression  Note:  Analysis  Correlation r=-.33-.43 27% variance  WPPSI - Wechsler Preschool and Primary Scale of Intelligence WISC (R) - Wechsler Intelligence Scale for Children (Revised) DAP - Draw-a-Person KELP - Kindergarten Evaluation of Learning Problems MRT - Metropolitan Readiness Test SIT - Slosson Intelligence Test VMI - Beery-Developmental Test of Visual Motor Integration WRAT - Wide Range Achievement Test  NJ O O  201  Appendix Table 2 HLM Results f o r A t t r i t i o n  Fixed Effects  Effects (SE)  Average within school equation: Intercept Slope (Study)  Estimates of Parameter Variance  -.08 (0.08) .09 (0.07)  2  Estimate  (x )  Intercept  .09**  (68.41)  Slope  .07**  (56.56)  (Study)  df 29  Appendix Table 3 HLM Models Explaining Variation in Draw-A-Person/Grade 3 Reading Relationships Model I Model II Model III Effect (SE) Effect (SE) Effect (SE) Fixed Effects Average within-school equation: Intercept 38.31** (0.66) 38.46** (0.66) 40.04** (0.59) .51** (0.07) .39** (0.07) K-Screen Slope .57** (0.07) .07 (0.08) .06 (0.08) Age on Entry Gender 1.26* (0.54) .87 (0.51) (0.65) .70 (0.68) 1.00 Physical Problems Extended Primary Schooling -3.90** (1.23) Learning Assistance -5.53** (1.29) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Model IV Effect (SE) 37.83** .38** .08 .88 1.04 -4.21** -4.84**  (0.71) (0.08) (0.08 (0.52) (0.64) (1.11) (1.30)  .36** -.16  (0.08) (0.25)  Estimate  Estimate  Estimate  Estimate  7.35** .04**  7.20** .04*  4.33** .05 12.92 13.49*  2.04 .05 5.97 15.19*  62.65  56.43  56.76  15.40  23.79  23.35  Model Statistics: Maximum likelihood 62.83 estimate of J~ R : Percent of Total Pupil Level 15.15 Variance Explained: R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  2  52.88  Note: * Significant at the .05 level. **Significant at the .01 level. M  o to  Appendix Table 4 HLM Models Explaining Variation in Draw-A-Person/Grade 3 Mathematic Relationships Model 1 Fixed Effects Effect (SE) Average within-school equation: 39.48** (0.65) Intercept K-Screen Slope .35** (0.06) Age on Entry Gender Physical Problems Extended Primary Schooling Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Model II Effect (SE)  Model III Effect (SE)  Model IV Effect (SE)  39.58** .32** .18* -.06 -.09  41.12** .21** .16* -.42 .20 -4.52** -5.13**  39.16** .19** .18* -.44 .12 -4.42** -4.80**  (0.68) (0.06) (0.07) (0.45) (0.59) (0.90) (0.86)  .37** -.06  (0.07) (0.17)  (0.66) (0.06) (0.07) (0.47) (0.59)  (0.65) (0.06) (0.07) (0.45) (0.56) (0.93) (0.86)  Estimate  Estimate  Estimate  Estimate  8.22** .03  8.40** .03  7.58** .03 3.20 .66  3.76** .03 1.93 .72  Model Statistics: Maximum likelihood 48.38 estimate of R : Percent of Total Variance Explained: 11.67 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  48.28  43.58  43.56  11.85  20.43  41.18  2  2  Note: * Significant at the .05 level. **Significant at the .01 level.  50.40  to o  Appendix Table 5 HLM Models Explaining Variation in Draw-A-Person/Grade 3 Vocabulary Relationships Model I Fixed Effects Effect (SE) Average within-school equation: Intercept 38.62+* (0.59) K-Screen Slope .53** (0.07) Age on Entry Gender Physical Problems Extended Primary Schooling Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition  Model II Effect (SE)  Model III Effect (SE)  Model IV Effect (SE)  38.72** .50** .14 .15 .06  40.15** .40** .13* -.22 .42 -3.38** -5.22**  38.16** .38** .15 -.16 .32 -3.47** -4.33**  (0.67) (0.07) (0.07) (0.50) (0.66) (0.97) (1.08)  .33** -.16  (0.06) (0.21)  Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  Estimate  5.13** .05*  5.16** .04*  3.67 .04* 5.33 5.49  Model Statistics: Maximum likelihood 57.84 estimate of R : Percent of Total Variance Explained: 13.40 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  (0.60) (0.07) (0.08) (0.52) (0.64)  (0.56) (0.07) (0.07) (0.50) (0.63) (1.07) (1.07)  Estimate 2.63 .05 1.40 7.27  57.84  53.76  53.55  13.39  19.51  19.82  2  2  Note: * Significant at the .05 level. **Significant at the .01 level.  28.33  NJ O  Appendix Table 6 HLM Models Explaining Variation in Draw-A-Person/Grade 3 Language Relationships Model II Model III Model I Effect (SE) (SE) Effect (SE) Effect Fixed Effects Average within-school equation: 40.65** (0.76) 42.55** (0.73) Intercept 40.17** (0.79) .52** (0.07) .37** (0.07) K-Screen Slope .61** (0.08) .17* (0.07) .15* (0.07) Age on Entry 1.74** (0.47) Gender 2.05** (0.50) (0.64) -.34 (0.60) -.65 Physical Problems -6.26** (1.10) Extended Primary Schooling -5.77** (1.16) Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Model IV Effect (SE) 39.85** .37** .16* 1.68** -.46 -6.18** -5.35**  (0.85) (0.07) (0.07) (0.47) (0.66) (1.06) (1.15)  .46** .01  (0.10) (0.23)  Estimate  Estimate  Estimate  Estimate  13.36** .07**  12.46** .05**  10.41** .04 8.47 10.55  6.57** .04 7.04 10.40  54.21  46.77  46.55  21.80  32.54  32.86  Model Statistics: Maximum likelihood 2  55.14 estimate of R : Percent of Total 20.47 Variance Explained: R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 0  Z  2  36.88  Note: * Significant at the .05 level. ••Significant at the .01 level. O  Appendix Table 7 HLM Models Explaining Variation in KLST/Grade 3 Reading Relationships Model III Model I Model 11 Effect (SE) Effect (SE) Effect (SE) Fixed Effects Average within-school equation: Intercept 37.52** (0.67) 37.68** (0.67) 39.33** (0.61) K-Screen Slope .86** (0.08) .61** (0.09) .80** (0.10) Age on Entry .11 (0.08) .08 (0.07) Gender 1.35* (0.53) 1.02 (0.51) .66 (0.67) .97 (0.65) Physical Problems Extended Primary Schooling -3.46** (1.18) -5.41** (1.19) Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  6.85** .01  6.84** .02  Model Statistics: Maximum likelihood estimate of 62.06 R^: Percent of Total Variance Explained: 16.18 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  Estimate 3.76 .01 9.15 9.26  Model IV Effect (SE) 37.21** .60** .10 1.01 1.00 -3.70** -4.78**  (0.77) (0.09) (0.07) (0.51) (0.66) (1.11) (1.19)  .34** -.10  (0.08) (0.27)  Estimate 2.87 .02 5.70 9.83  61.71  56.84  56.94  16.67  23.24  23.10  2  23.67  Note: * Significant at the .05 level. **Significant at the .01 level. o  CTl  Appendix Table 8 HLM Models Explaining Variation in KLST/Grade 3 Mathematics Relationships Model I Model 11 Model III Effect (SE) Effect (SE) Effect (SE) Fixed Effects Average within-school equation: Intercept 38.54+* (0.68) 38.61** (0.68) 40.14** (0.69) K-Screen Slope .66** (0.09) .64** (0.09) .48** (0.09) Age on Entry .17* (0.07) .14* (0.07) Gender -.26 (0.45) -.60 (0.44) -.22 (0.58) (0.56) Physical Problems .09 Extended Primary Schooling -3.59** (0.94) Learning Assistance -4.92** (0.85) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  8.7?** .09*  8.35** .08*  Model Statistics: Maximum likelihood 46.38 estimate of R : Percent of Total Variance Explained: 15.30 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  Estimate 8.18* .06 3.34 .87  Model IV Effect (SE) 38.06** .46** .15* -.63 .08 -3.70** -4.63**  (0.74) (0.09) (0.07) (0.43) (0.56) (0.90) (0.86)  .34** -.10  (0.07) (0.23)  Estimate 5.36* .08 2.31 .98  46.24  42.54  42.35  15.56  22.33  22.65  2  2  Note: * Significant at the .05 level. +*Significant at the .01 level.  34.47  O  —1  Appendix Table 9 HLM Models Explaining Variation in KLST/Grade 3 Vocabulary Relationships Model I Model II Model III Effect (SE) Effect (SE) Effect (SE) Fixed Effects Average within-school equation: 37.49** (0.59) 37.58** (0.59) 38.95** (0.58) Intercept .90** (0.08) .87** (0.09) .72** (0.08) K-Screen Slope .16* (0.07) .14 (0.07) Age on Entry Gender .15 (0.50) -.19 (0.49) .04 (0.63) .31 (0.62) Physical Problems Extended Primary Schooling -2.61* (0.97) -5.08** (0.98) Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  4.71** .02  4.43** .02  Model Statistics: Maximum likelihood 56.02 estimate of o R : Percent of Total 16.11 Variance Explained: R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  Estimate  Model IV Effect (SE) 37.06** .70** .16 -.16 .24 -2.66* -4.45**  (0.69) (0.09) (0.07) (0.49) (0.64) (0.95) (1.00)  .31** .10  (0.07) (0.25)  Estimate  3.06 .01 1.33 2.62  1.94 .02 1.04 4.35  55.95  52.88  52.64  16.22  20.81  21.17  2  2  Note: * Significant at the .05 level. ••Significant at the .01 level.  36.60  O CO  Appendix Table 10 HLM Models Explaining Variation in KLST/Grade 3 Language Relationships Model III Model II Model I Effect (SE) Effect (SE) Effect (SE) Fixed Effects Average within-school equation: 39.77** (0.82) 40.22** (0.82) 42.12** (0.74) Intercept .76** (0.09) .54** (0.08) .86** (0.10) K-Screen Slope .21** (0.07) .18** (0.07) Age on Entry 2.23** (0.49) 1.95** (0.46) Gender -.80 (0.63) -.48 (0.60) Physical Problems -5.99** (1.21) Extended Primary Schooling -5.92** (1.08) Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Model IV Effect (SE) 39.45** .54** .18* 1.84** -.68 -5.84** -5.32**  (0.86) (0.09) (0.07) (0.46) (0.69) (1.21) (1.00)  .46** .27  (0.09) (0.27)  Estimate  Estimate  Estimate  Estimate  14.16** .08  13.86** .07  9.98** .02 13.86* 6.75  6.40** .06 14.04* 3.85  53.97  47.04  46.62  22.14  32.15  32.75  Model Statistics: Maximum likelihood 55.46 estimate of o R : Percent of Total Variance Explained: 19.99 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  2  2  Note: * Significant at the .05 level. **Significant at the .01 level.  35.87  o vo  Appendix Table 11 HLM Models Explaining Variation in Deverell/Grade 3 Reading Relationships Model I Model II Model III Effect (SE) Effect (SE) Fixed Effects Effect (SE) Average within-school equation: 38.23+* (0.62) Intercept 38.30** (0.61) 39.89** (0.67) K-Screen Slope .47** (0.06) .44** (0.06) .33** (0.07) Age on Entry .16 (0.08) .13 (0.08) Gender 1.79** (0.52) 1.29* (0.51) Physical Problems .82 (0.67) 1.17 (0.65) Extended Primary Schooling -2.89** (1.29) Learning Assistance -6.50** (1.23) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  5.48** .02  5.02** .02  Model Statistics: Maximum likelihood estimate of 63.47 R : Percent of Total Variance Explained: 14.27 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  Model IV Effect (SE) 37.72** .33* .15 1.27* 1.17 -3.11* -5.80**  (0.78) (0.06) (0.07) (0.50) (0.68) (1.21) (1.25)  .33** -.17  (0.08) (0.17)  Estimate  Estimate  6.05 .04 13.08 10.74  3.48* .03 7.98* 11.84*  62.69  57.02  56.97  15.33  22.99  23.05  £  Note: * Significant at the .05 level. **Significant at the .01 level.  42.48  to o  Appendix Table 12 HLM Models Explaining Variation in Deverell/Grade 3 Mathematics Relationships Model III Model II Model I Effect (SE) (SE) Effect (SE) Effect Fixed Effects Average within-school equation: Intercept 38.85*+ (0.49) 38.92** (0.48) 40.46** (0.46) K-Screen Slope .40** (0.05) .38** (0.05) .26** (0.05) Age on Entry .21** (0.07) .19* (0.07) Gender .09 (0.45) -.28 (0.44) -.08 (0.58) .20 (0.56) Physical Problems Extended Primary Schooling -3.30** (0.98) Learning Assistance -5.22** (1.01) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance Model Statistics: Maximum likelihood estimate of R : Percent of Total Variance Explained: 2  0  Estimate  Estimate  Estimate  Model IV Effect (SE) 38.17** .27** .19** -.37 .26 -3.37** -5.02**  (0.69) (0.06) (0.07) (0.44) (0.57) (0.92) (0.90)  .37** .00  (0.06) (0.15)  Estimate  2.79 .02  2.52 .02  1.06 .01 3.60 5.67  3.58* .03 1.78 2.22  47.11  46.85  43.42  42.87  13.97  14.45  20.71  21.71  7  R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement Note: * Significant at the .05 level. **Significant at the .01 level.  -237.73  Appendix Table 13 HLM Models Explaining Variation in Deverell/Grade 3 Vocabulary Relationships Model I Model II Model III Effect (SE) Effect (SE) Effect (SE) Fixed Effects Average within-school equation: Intercept 38.39+* (0.43) 38.46** (0.43) 39.80** (0.50) K-Screen Slope .46** (0.06) .44** (0.06) .34** (0.06) Age on Entry .23** (0.08) .21* (0.07) Gender .67 (0.05) .24 (0.49) Physical Problems .24 (0.63) .57 (0.62) Extended Primary Schooling -2.22 (1.08) Learning Assistance -5.52** (1.07) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  37.98** .33** .23** .26 .44 -2.25* -4.69**  (0.64) (0.05) (0.07) (0.49) (0.67) (1.00) (1.09)  .30** -.30  (0.06) (0.15)  Estimate  Estimate  .71 .03  .51 .03  .94 .02 3.55 4.58  .80 .00 .80 6.78  57.92  54.51  54.31  13.28  18.38  18.67  Model Statistics: Maximum likelihood estimate of 58.32 R : Percent of Total Variance Explained: 12.67 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  Estimate  Model IV Effect (SE)  Estimate  Z  Note: * Significant at the .05 level. **Significant at the .01 level.  14.89  to K)  Appendix Table 14 HLM Models Explaining Variation in Deverell/Grade 3 Language Relationships Model I Model II Model III (SE) (SE) iSEJ_ Effect Effect Effect Fixed Effects Average within-school equation: 39.93++ (0.67) 40.22** (0.69) 42.16+* (0.81) Intercept .55** (0.06) .51** (0.06) .36** (0.07) K-Screen Slope (0.46) .25** (0.07) .21** Age on Entry 2.56** (0.48) 2.18** (0.46) Gender (0.59) (0.63) -.56 -.24 Physical Problems -5.17** (1.06) Extended Primary Schooling -6.42** (1.07) Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  8.13** .03*  8.81** .03*  Model Statistics: Maximum likelihood estimate of o 54.63 R : Percent of Total Variance Explained: 21.18 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  Estimate 13.12 .04 5.54 7.25  Model IV Effect 39.36** .36** .21** 2.09** -.41 -4.92** -5.81**  (0.88) (0.06) (0.06) (0.46) (0.65) (1.04) (1.04)  .45** .17  (0.09) (0.18)  Estimate 7.77** .05* 4.42 5.96  52.54  46.36  46.15  24.21  33.12  33.42  2  2  40.78  Note: * Significant at the .05 level. **Significant at the .01 level. u>  Appendix Table 15 HLM Models Explaining Variation in MS/Grade 3 Reading Relationships Model I Model II Model III Fixed Effects Effect (SE) Effect (SE) Effect (SE) Average within-school equation: Intercept 40.64** (0.50) 40.57** (0.49) 41.80** (0.44) K-Screen Slope 1.89** (0.35) 1.70** (0.35) .98* (0.35) Age on Entry .16 (0.08) .13 (0.08) Gender 2.19+* (0.53) 1.61** (0.51) Physical Problems .76 (0.69) 1.09 (0.66) Extended Primary Schooling -4.66** (1.20) Learning Assistance -5.78** (1.24) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Model IV Effect (SE) 39.55** .95* .15 1.61** 1.17 -4.95** -5.02**  (0.60) (0.34) (0.08) (0.51) (0.67) (1.09) (1-24)  .35** -2.47  (0.08) (1.59)  Estimate  Estimate  Estimate  Estimate  4.55** .16  4.06** .15  2.73** .21 10.41 10.46  .64 .14 4.50 11.03*  66.14  59.36  59.45  10.67  19.83  19.71  Model Statistics: Maximum likelihood estimate of c 67.32 R : Percent of Total Variance Explained: 9.07 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  2  2  76.56  Note: * Significant at the .05 level. **Significant at the .01 level. 4^  Appendix Table 16 HLM Models Explaining Variation in MS/Grade 3 Mathematics Relationships Model I Model II Model III Effect (SE) Effect (SE) Effect (SE) Fixed Effects Average within-school equation: 40.67*+ (0.44) 40.68** (0.44) 41.78** (0.41) Intercept K-Screen Slope 2.29** (0.30) 2.16** (0.30) 1.56** (0.31) Age on Entry .19* (0.07) .16* (0.07) Gender .35 (0.45) -.13 (0.44) -.17 (0.59) .09 (0.57) Physical Problems Extended Primary Schooling -4.41** (0.98) Learning Assistance -5.12** (0.89) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  Estimate  3.61** .13  3.56** .14  2.62** .38 5.36 1.49  Model Statistics: Maximum likelihood estimate of 47.98 R : Percent of Total Variance Explained: 12.38 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  Note: * Significant at the .05 level. **Significant at the .01 level.  Model IV Effect (SE) 39.71** 1.53** .16* -.11 .07 -4.40** -4.57**  (0.53) (0.31) (0.07) (0.44) (0.60) (0.90) (0.86)  .33** -1.50  (0.07) (1-42)  Estimate .69 .32 2.17 1.64  47.74  43.18  43.24  12.81  21.15  21.03 73.66  ro HLn  1  Appendix Table 17 HLM Models Explaining Variation in MS/Grade 3 Vocabulary Relationships Model III Model I Model II (SE) Effect (SE) Effect Effect (SE) Fixed Effects Average within-school equation: Intercept 40.78+* (0.41) 40.78** (0.41) 41.85** (0.39) K-Screen Slope 1.92** (0.33) 1.73** (0.34) 1.16** (0.35) Age on Entry .22** (0.08) .20* (0.07) Gender 1.09* (0.51) .52 (0.50) (0.66) .47 (0.64) Physical Problems .12 Extended Primary Schooling -4.16** (1.04) Learning Assistance -5.64 (1.05) Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Estimate  Estimate  2.41** .16  2.26** .16  Model Statistics: Maximum likelihood estimate of 61.79 R : Percent of Total Variance Explained: 7.47 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  Note: * Significant at the .05 level. **Significant at the .01 level.  61.24 8.31  Estimate 1.62 .45 3.52 4.39  56.24 15.80  Model IV Effect (SE) 39.78** 1.04** .22** .58 .45 -4.16** -4.67**  (0.53) (0.33) (0.07) (0.49) (0.67) (1.05) (1.05)  .32** 3.65*  (0.06) (1.50)  Estimate .25 .10 .50 4.69  55.98 16.17 84.57  to CTN  Appendix Table 18 HLM Models Explaining Variation in MS/Grade 3 Language Relationships Model III Model II Model I (SE) Effect (SE) Effect Effect (SE) Fixed Effects Average within-school equation: 42.62** (0.56) 42.71** (0.56) 44.07** (0.55) Intercept 2.57** (0.33) 2.29** (0.32) 1.52** (0.31) K-Screen Slope .24** (0.08) .20* (0.07) Age on Entry 2.94** (0.49) 2.41** (0.46) Gender -.67 (0.65) -.38 (0.61) Physical Problems -6.74** (1.13) Extended Primary Schooling -6.01** (1.10) Learning Assistance Effects of Between-School Variables: On Achievement: School Mean Ability On K-Screen/Ach Relationship: Attrition Estimates of Parameter Variance Residual Parameter Variance: Achievement K-Screen/Ach Relationship Extended Primary Schooling Learning Assistance  Model IV Effect (SE) 41.29** 1.56** .20* 2.32** -.53 -6.65** -5.48**  (0.70) (0.31) (0.07) (0.46) (0.69) (1.13) (1.05)  .46** 1.05  (0.09) (1.47)  Estimate  Estimate  Estimate  Estimate  6.63** .13  6.52** .10  6.23** .08 9.96 7.14  2.61** .10 10.32* 5.93  Model Statistics: Maximum likelihood estimate of 58.93 R : Percent of Total Variance Explained: 14.99 R : Percent of Residual Parameter Variance Explained On Adjusted Levels of Achievement 2  0  Note: * Significant at the .05 level. ••Significant at the .01 level.  56.41  48.43  48.06  18.62  30.13  32.75 58.11  ro  Appendix Table 19 HLM Results for Grade Three Achievement on Kindergarten Screening Measures Model V Reading Mathematics Vocabulary Fixed Effects Effect (SE) Effect (SE) Effect (SE) Average within-school equation: Intercept 34.56** (0.89) 36.58** (0.88) 34.81** (0.81) DAP . 30** (0.07) .09 (0.05) .28** (0.06) KLST .44** (0.09) .34** (0.09) .55** (0.09) DEVTOT .23** (0.06) .18** (0.05) .19** (0.05) MS .26 (0.34) 1.10** (0.32) .35 (0.33) Effects of Between Students Covar iates: Age on Entry (0.07) .10 (0.07) .08 .01 (0.07) Gender .46 (0.51) -.86 (0.43) -.71 (0.49) Physical Problems 1.05 (0.62) .20 (0.53) .25 (0.58) Extended Primary Schooling -1.59 (1.29) -2.21* (0.99) -.79 (1.05) Learning Assistance -4.51** (1.24) -4.40** (0.90) -4.04** (1.06) Effects of Between-School Variabl es: On Achievement: .34** (0.80) School Mean Ability .33** (0.07) .30** (0.07) On K-Screen: Attrition -.07 DAP (0.31) -.02 (0.23) -.03 (0.26) KLST .36 (0.38) .11 (0.32) .52 (0.35) DEVTOT -.25 (0.22) -.10 (0.19) -.39 (0.20) MS -2.41 (1.69) -1.47 (1.57) -3.15 (1.65) Estimates of Parameter Variance Residual Parameter Variance: Achievement DAP KLST DEVTOT MS Extended Primary Schooling Learning Assistance  Estimate 4.86* .04 .04 .01 .21 13.99* 13.15*  Language Effect (SE) 36.98** .24** .35** .27** .98**  (0.96) (0.06) (0.09) (0.06) (0.31)  .09 1.27* -.46 -3.59** -4.86**  (0.07) (0.46) (0.57) (1.14) (1.02)  .42**  (0.08)  -.18 .11 .13 .96  Estimate  Estimate  Estimate  9.52** .01 .07 .01 .47 5.39 2.57  4.37 .02 .03 .00 .25 3.79 7.65*  10.17** .02 .05 .04 .16 9.59 5.66  Model Statistics: Maximum likelihood 9  estimate of R : Percent of Total Pupil Level Variance Explained:  52.88  40.00  49.24  42.94  28.59  26.95  26.26  38.06  0  2  Note: * Significant at the .05 level. Significant at the .01 level.  (0.26) (0.38) (0.22) (1.55)  NJ M CO  Appendix Table 20 HLM Models Explaining Variation in Draw-A-Person/Grade 3 Reading Relationships Model VI Reading Mathematics Vocabulary (SE) Effect (SE) Effect (SE) Effect Fixed Effects Average within-school equation: 34.,71** (0.72) 34.58*+ (0.88) 36.91*+ (0.82) Intercept .32** (0.06) .29** (0.06) ,25* (0.06) DAP KLST .46** (0.09) .34** (0.09) ,57** (0.08) ,20** (0.05) DEVTOT .23** (0.06) .18** (0.05) 1.28** (0.30) .97** (0.30) MS Effects of Between Students Covari ates: -1.67 -2.42* (0.97) -3.,61** (1.14) Extended Primary Schooling (1.28) -4.95** (1.20) -4.35*+ (0.87) -4.,21** (0.99) Learning Assistance Effects of Between-School Variables: On Achievement: .35** (0.08) .34** (0.07) ,30** (0.06) School Mean Ability Random Effects: Residual Parameter Variance: Achievement DAP KLST DEVTOT MS Extended Primary Schooling Learning Assistance Model Statistics: Maximum likelihood estimate of R : Percent of Total Pupil Level Variance Explained: 0  Estimate 5.13 .03 .03 .02  Estimate 8.28*+  Est :imate 2.,91 02 01 01  Language Effect (SE) 36.97**  (0.96)  .36** .27**  (0.09) (0.06)  -4.87**  (1.01)  .39**  (0.08)  Estimate 10.41** .02 .05 .04 .19 10.12 5.75  13.11* 10.91*  .06 .01 .32 4.34 2.02  52.92  40.37  50. 03  42.82  28.53  26.28  25. 84  38.23  5. 80  2  Note: * Significant at the .05 level. **Significant at the .01 level.  ro i—•  220  Appendix Table 21 Means and Standard Deviations of Outcome Measures For Four Samples Outcome Measure  Mean  Std.Dev  Number  Read3  40.39  8.981  2193  Math3  40.55  7.692  2175  Vocab3  40.58  8.355  2180  Lang3  42.69  8.717  2188  Read3  41.44  8.650  708  Math3  41.29  7.575  705  Read3  40.84  8.856  745  Math3  41.19  7.485  740  Read3  41.45  8.605  957  Math3  41.67  7.400  957  Vocab3  41.52  8.172  957  Lang3  43.89  8.326  957  District  Cohortl  Cohort2  Achieved Sample  221  Appendix Table 22 Prediction-Performance Matrix Analysis  Valid False False Valid Overall Positives Positives Negatives Negatives Hit H V H V H V H V Sen. Spec. Rate DAP/ Read Math Vocab Lang  55 52 58 47  24 24 23 28  45 48 42 53  12 13 12 12  at risk = 161 not at risk = 796  LAC = 17% (27) LAC = 8% (64)  KLST/Read Math Vocab Lang  24 24 23 28  55 52 58 47  45 48 42 53  12 13 12 12  at risk = 102 not at risk = 855  LAC = 18% (18) LAC = 9% (77)  MS/  63 66 63 68  Read Math Vocab Lang  45 46 49 36  55 54 51 64  47 46 46 48  at risk = 510 not at risk = 447  LAC = 12% (61) LAC = 6% (27)  Devtot/Read Math Vocab Lang  18 16 16 21  79 69 77 65  at risk = 84 not at risk = 873 Note:  21 31 23 35  3 4 3 4  35 34 42 24  76 76 77 72  65 66 62 76  88 87 88 88  24 24 23 28  88 87 88 88  64 63 61 71  Ext. Prim. = 25% (40) Ext. Prim. = 6% (48) 35 34 38 24  76 76 77 72  65 66 62 76  88 87 88 88  24 24 23 28  88 87 88 88  64 63 61 71  Ext. Prim. = 29% (30) Ext. Prim. = 7% (60) 30 28 32 19  37 34 37 32  70 72 68 81  53 54 54 52  63 66 63 68  53 54 54 52  57 58 58 57  Ext. Prim. = 49% (25) Ext. Prim. = 4% (18) 34 34 38 24  LAC = 27% (23) LAC = 8% (70)  Horizontal (H) Percentages Vertical (V) Percentages Achievement <3.9  82 84 84 79  66 66 62 76  97 96 97 96  18 16 16 21  97 96 97 96  Ext. Prim. = 49% (41) Ext. Prim. = 5% (44)  67 66 63 75  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0100582/manifest

Comment

Related Items