UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A comparison of curriculum-based and norm-referenced measures in the identification of reading difficulty Dunn, Rita L. 1992

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1992_spring_dunn_rita_l.pdf [ 3.43MB ]
Metadata
JSON: 831-1.0054600.json
JSON-LD: 831-1.0054600-ld.json
RDF/XML (Pretty): 831-1.0054600-rdf.xml
RDF/JSON: 831-1.0054600-rdf.json
Turtle: 831-1.0054600-turtle.txt
N-Triples: 831-1.0054600-rdf-ntriples.txt
Original Record: 831-1.0054600-source.json
Full Text
831-1.0054600-fulltext.txt
Citation
831-1.0054600.ris

Full Text

A COMPARISON OF CURRICULUM-BASED ANDNORM-REFERENCED MEASURES IN THEIDENTIFICATION OF READING DIFFICULTYbyRita L. DunnB.A., University of British Columbia, 1974M.Ed., University of British Columbia, 1984A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS OF THE DEGREE OFDOCTOR OF EDUCATIONinTHE FACULTY OF EDUCATIONDepartment of Educational Psychology and Special EducationWe accept this thesis as conformingTHE UNIVERSITY OF BRITISH COLUMBIADecember 1991c Rita L. Dunn, 1991to the required standardSignature(s) removed to protect privacyNational Libaryof CanadaCanadian Theses ServiceO(tawa. CanadaKIAON4Bibliothêque nationaledu CanadaService des (hêês canadiennesThe author has granted an irrevocable non-exclusive licence allowing the National Libcaiyof Canada to reproduce, loan, distribute or sellcopies of his/her thesis by any means and inany fotm or format, making this thesis availableto interested persons.The author retains ownership of the copyrightin his/her thesis. Neither the thesis norsubstantial extracts from it may be printed orotherwise reproduced without his/her permission.L’auteur a accordé une licence irrevocable etnon exdusive permettant a Ia Bibtiothéquenatiönale du Canada do reproduire, préter,dist,ibuer on vendre des copies de sa thesede quelque manière et sous quelque tormeque ce wit pour mettre des exemplaires decette these a Ia disposition des personnesintéressées.L’auteur conserve Ia propriété du droitd’auteurqui protege sa these. Ni Ia these iii des extraitssubstantiels de celle-ci ne doivent êtreimprimés ou autrement reproduits sans sonautorisation.I—.’ 1IIanaaaISBN ø-315-75401-XIn presenting this thesis in partial fulfilment of the requirements for an advanceddegree at the University of British Columbia, I agree that the Library shall make itfreely available for reference and study. I further agree that permission for extensivecopying of this thesis for scholarly purposes may be granted by the head of mydepartment or by his or her representatives. It is understood that copying orpublication of this thesis for financial gain shall not be allowed without my writtenpermission.(Signature)Department of Li IThe University of British ColumbiaVancouver, CanadaDateDE-6 (2/88)Signature(s) removed to protect privacyiiABSTRACTThe purpose of this study is to investigate thetechnical adequacy of two reading Curriculum—Based Measures(CBM5), to examine the relationships of the CBM5 to norm-referenced tests, and to compare the strength ofrelationship of both kinds of measures to school—basedindices of reading performance. The two CBMs (a word listsampled from several reading series and a passage readingtest composed of ten Ginn 720 passages) were taken from theliterature; comprehensive information about their technicaladequacy had not been previously available.A review of the literature indicates that CBM,particularly reading CBM, is gaining increased attention ineducation because of claims regarding its utility inmonitoring pupil progress, its ease of administration, andits relationship to local curricula as well as to learninggains. This study examined how reading CBMs and twosubtests from the Kaufman Test of Educational Achievement(Kaufman & Kaufman, 1985) relate to each other and to threeschool—based indices of reading performance: a schooldistrict reading test, program placement status (learningdisabled or regular education), and a teacher rating scaleof reading skill.iiiGrade four students from one metropolitan Vancouverschool district served as subjects (n=105). Of these, 35were classified as learning disabled and 70 were classifiedas regular education. Learning disabled status wasdetermined by judgments of a school district screeningcommittee and by examining previous psychoeducationalassessments.Reliability indices calculated on the CBMs supportedclaims for technical adequacy. These estimates were asfollows: internal consistency of the word list was .97,internal consistency of the reading passages was .98 and.94 for reading speed and accuracy, test—retestreliability of the reading passages was .89 and .79 forreading speed and accuracy, and inter—rater reliability ofthe reading passages was .99 and .96 for reading speed andaccuracy. Results indicated that the CBMs used in thisstudy have high reliabilities.CBM5, especially the speed score from the- readingpassages, demonstrated strong relationships to the twonorm—referenced subtests. The pattern of correlationsbetween the measures differed between the learning disabledand normal sample; analyses of variance demonstrated thatall measures used in the study discriminated between thelearning disabled and the regular education groups.ivStepwise multiple regression and canonical analysisindicated that the two norm—referenced subtests, the speedscore from the Curriculum—Based Reading Passages, and theaccuracy score from the Curriculum—Based Word List weremost efficient in “predicting” the three school-basedindices of reading performance. Evidence for concurrentvalidity of curriculum—based and norm—referenced measureswas found in this study. When administration time,instructional utility, and technical properties areconsidered, results indicated that the Kaufman Test ofEducational Achievement Reading Decoding subtest and theCurriculum—Based Reading Passages speed score are the mostefficient of the predictor measures investigated inidentifying and programming for Year Four children withsignificant reading difficulty. Implications for furtherresearch and the potential of CBM to accommodateinstructional and measurement needs is discussed.VTABLE OF CONTENTSChapter PageAbstract iiList of Tables ixAcknowledgements xiIntroduction and Delineation of the Problem.. 1Measurement of Academic Achievement 1Limitations of Norm—Referenced AchievementTests 2Curriculum—Based Measurement— A ProposedAlternative to Standardized Tests 4Description of the Study 5Purpose of the Study 6CBM Technical Adequacy 7Prediction -of School-Based ReadingPerformance 7Correlations Between the Curriculum—BasedMeasures and the Norm—ReferencedMeasures 8Comparison of the Two Curriculum—BasedMeasures 8Nature of the Criteria 9Research Questions 10Justification for the Study 11Definition of Terms 122. Review of the Literature 16Chapter Overview 16Definitional Issues — Curriculum—BasedMeasurement and Curriculum—BasedAssessment 17Kinds of Informal Measures in Reading 22Question Answering 22Recall Procedures 24Cloze Techniques 24Oral Passage Reading Measures 26Words in Isolation 27viTABLE OF CONTENTS (continued)Chapter 2. (cont.) PageThe Content Validity Problem . 27Aptitude—Treatment Interaction Research 31Difference Scores 37Technical Adequacy and Norm—ReferencedTests 42Curriculum—Based Measurement Reliability 43Curriculum—Based Measurement Validity 48Curricular Validity 49Concurrent Validity - Curriculum-BasedMeasurement and Norm—Referenced Tests 52Concurrent Validity - Curriculum-BasedMeasurement and Teacher Rating 54Construct Validity 55Construct Validity - DifferentiationBetween Groups 56Construct Validity - Sensitivity toGrowth 57Curriculum—Based Measurement andDevelopment-al Reading Models.. . ..... .. 5-8Summary of Reliability and ValidityResearch 59Learning Disability Diagnosis 60Norm—Referenced Measures and LearningDisability Identification 61Curriculum—Based Measurement andLearning Disability Identification 64Cognitive Psychology and LearningDisability Identification... .... . 66Eligibility for Special Services 69Curriculum—Based Measurement and LearningGains. . . . . . . . . 72Varying Curriculum—Based MeasurementDifficulty Levels 76Curriculum—Based Measurement and GoalStructures 76Graphing and Computers in Curriculum—BasedMeasurement 78SuiInnarlr 793 . Methodology. . . . . . . . . . . . . . . . 82Design 82PopulationandSampling 83Identification of Participants 83Recruitment of Schools and Students 88viiTABLE OF CONTENTS (continued)Chapter 3. (cont.) PageTesting Procedures 89Test Selection Criteria . 90Kaufman Test of Educational Achievement 90Curriculum—Based Reading Passages 91Curriculum—Based Word List 96Selection Criteria: Decision orCriterion Measures 100TeacherRatingScale 100School District Core Mastery Test:Grade4Reading 104Placement Status 104Data Collection and Data Preparation 106Examiner Training 107Test Administration 108Data Preparation 1094. Re-sults. . . . • 111Means and Standard Deviations of thePredictor and Criterion Measures......... 111Reliability of the Curriculum-BasedMeasures . . . . . . . 116Correlations Within the Measures... ... ..... 122TeacherRatingScale 123Curriculum—Based Reading PassagesAverageReadingSpeed 123Curriculum—Based Reading PassagesAverage Percentage Correct Score 123Correlations Among the Measures 124AnalysesofVariance..Predictions . . . . . . . . . . . . 130Summary of Results . 1565. Summary, Conclusions, and Implications . 162Summary. . . . . . . 162Purpose 162Results 163viiiTABLE OF CONTENTS (continued)Chapter 5. (cont.) PageResponse to Research Questions 168Conclusions and Implications 171Recommendations for Practice 176LiiuitationsoftheStudy 178Directions for Future Research 182References 186AppendicesA. Table A.1- Squared multiple correlationsof each independent variable with allother independent variablesTable A.2- Squared multiple correlationsof each dependent variable with allother dependent variables 207B. Table B- Inter-rater reliabilitydescriptive statistics and T—tests, 209C. Principal Permission Letter 210D. Parent Permission Letter 213E. Parent Permission Letter— Attachment forthe Learning Disabled Sample 217F. Testing Procedure Instructions 219G. Request for Subject Participation 221H. Teacher Rating Scale of Reading Skill 223I. Curriculum—Based Reading Passages TestAdministration and Scoring Instructions 225J. Curriculum—Based Reading Passages 228K. Curriculum—Based Reading Passages ScoringForm 239L. Curriculum-Based Word List Administrationand Scoring Instructions 250M. Curriculum—Based Word List 252N. Curriculum-Based Word List Scoring Form 2540. District Test Reading Passages 256P. District Test Student Form 259Q. District Test Scoring Instructions 265ixLIST OF TABLESPageTable 1. Characteristics of the Sample 85Table 2. Means and Standard Deviations of thePredictor and Criterion Measures 112Table 3. Internal Consistency of the Word List 118Table 4. Internal Consistency of the ReadingPassages 118Table 5. Test-Retest Reliability of the ReadingPassages 119Table 6. Inter—Rater Reliability of the ReadingPassages 121Table 7. Correlations Between the Measures for theTotal Sample 125Table 8. Correlations Between the Measures for theLearning Disabled Sample 127Table 9. Correlations Between the Measures for theRegular Sample 128Table 10. Summary of Analyses of Variance:Differences Between Learning Disabledand Regular Program Means 131Table 11. Predicting School-Based Criteria fromCurriculum—Based and Norm—ReferencedMeasures: Stepwise Multiple Regressionfor the Total Sample 133Table 12. Predicting School-Based Criteria fromCurriculum—Based and Norm—ReferencedMeasures: Stepwise Multiple Regressionfor the Learning Disabled Sample 135Table 13. Predicting School-Based Criteria fromCurriculum—Based and Norm—ReferencedMeasures: Stepwise Multiple Regressionfor the Regular Sample 136Table 14. Multiple Regression Beta Weights forthe Total Sample 141xTable 15. Multiple Regression Beta Weights forthe Learning Disabled and RegularEducation Samples 143Table 16. Canonical Analysis: KTEA Subtests andCurriculum—Based Measures withDependent Variables 145Table 17. Squared Multiple Correlations andCanonical Variable Loadings for AllIndependent Variables 148Table 18. Squared Multiple Correlations andCanonical Variable Loadings forDependent Variables 149Table 19. Canonical Analysis: KTEA Subtests withDependent Variables 151Table 20. Squared Multiple Correlations and- Canonical Variable Loadings:KTEA Subtests with Dependent Variables... 153Table 21. Canonical Analysis: Curriculum—BasedMeasures with Dependent Variables 154Table 22. Squared Multiple Correlations and CanonicalVariable Loadings: Curriculum—BasedMeasures with Dependent Variables 157xiACKNOWLEDGEMENTSSeveral hundred persons— parents, children, schooland district staff- participated in this research.Without the cooperation of all these people and thewillingness of senior officials in the school district,this dissertation would not have been possible.My thanks also go the district Area Counsellors fortheir considerable time and support during the gathering ofdata for this study. Their efforts made a difficult taskmuch simpler and more enjoyable. I would also like tothank Mrs. Helen Dawe, a school district Area Counsellor,for her constant support and enthusiasm for the study. Shefacilitated the organization of the data gathering withinthe district.I wish to especially thank Dr. Robert Conry who hasprovided consistent guidance and support from the beginningof this study. It is very unlikely that I would have hadthe courage to embark upon such a project without hisadvice and encouragement. I also wish to thank Dr. Conryfor the considerable time he spent with me and his patienceand sense of humour during the process of data analysis.I also owe a debt of gratitude to my family, friends,and colleagues who helped and also listened to my whiningfor the last two years.1Chapter 1Introduction and Delineation of the ProblemMeasurement of Academic AchievementAccurate information about student academicachievement is necessary for both educational practice andresearch. Decisions about eligibility for special services,statements about pupil progress and the success ofeducational interventions, and research conclusions areoften based upon the results of norm—referenced achievementtests. The sole use of a norm—referenced achievement test,however, to evaluate academic achievement and propose aremediation plan, if necessary, has been questioned(Gerken, 1985; Gresham, 1983; Ysseldyke, 1979; Ysseldyke &Marston, 1982; Ysseldyke & Mirkin, 1982).Gerken (1985) has stated that some psychologists havenever understood that assessment goes beyond testing andthat school psychologists need to cease their search forthe “ideal” test instrument that is technically adequate,comprehensive, precise, economical and useful. She furtherstates that there are few if any instruments that fit thatdescription, especially in the area of academic skills.Gresham (1983) states that the major problems withpsychoeducational assessment are: (a) obtaining2insufficient assessment information, (b) using technicallyinadequate tests, and (c) using measures that yieldinappropriate or educationally irrelevant information. Hesuggests using more than one method to assess the samestudent trait to obtain more valid and relevantinformation. The use of a single norm—referenced test tomake a meaningful statement about student academicachievement, even when the norm—referenced instrumentpossesses good technical adequacy, may be an insufficientmethod to assess academic achievement. Many practicingschool psychologists, however, continue this practice andmany researchers continue to make claims based upon asingle method of assessment— that is, the norm—referencedtest. Information about how different kinds of academicmeasures relate to school based assessment of academicperformance may be helpful in supporting better practicesin academic assessment.Limitations of Norm-Referenced Achievement TestsCritics of norm—referenced reading achievement testingstate that standardized reading tests lack content validity(Eaton & Lovitt, 1972; Floden, Porter, Schmidt, & Freeman,1980; Good & Salvia, 1988; Jenkins & Pany, 1978; Webster,Mclnnis, & Craver, 1986). Part of this concern arises fromthe varying degrees of reading vocabulary overlap between3different basal reading series and standardized readingachievement tests. Critics of norm—referenced tests assertthat a greater overlap in vocabulary between reading seriesand standardized test causes .a higher student test scoreand that standardized tests do not present equal chancesfor success for students using different basal readingseries (Good & Salvia, 1988; Jenkins & Pany, 1978; Shapiro& Derr, 1987).Other limitations of norm—referenced tests include theconcern that technically inadequate norm—referencedinstruments are often used to make statements about studentlearning ability or achievement (Fuchs, Fuchs, Benowitz, &Barringer, 1987; Ysseldyke et al., 1983). Further to this,researchers have stated that reading fluency is notconsidered in published tests of reading achievement(Ysseldyke & Marston, 1982). Some critics point to theinability of standardized tests to measure learning gainsor treatment effects in pre/post—test format (Carver, 1974;Shinn, Good, & Stein, 1989) and the unreliability oftraditional pre/post—test change scores in educationalmeasurement is well documented (Crocker & Algina, 1986;Ghiselli, Campbell, & Zedeck, 1981; Salvia & Ysseldyke,1985). These opinions of standardized tests have led tothe development of an alternate approach called Curriculum—Based Measurement. As will be further elaborated upon inthe following pages, CBM is a set of standardized4measurements that teachers can use within any curriculum toprovide assessment information that helps teachers planbetter instructional programs and answer questions aboutthe effectiveness of programs (Fuchs & Deno, in press).Proponents of CBM assert that these measures overcomethe aforementioned difficulties of norm—referenced testsand that CBM has demonstrated utility in instructionalplanning and the documentation of student learning gainsthat advocates of norm—referenced achievement testing havebeen unable to document (Fuchs & Fuchs, 1986a, 1986b;Fuchs, Deno, & Mirkin, 1984; Fuchs, Fuchs, & Stecker,1989)Curriculum-Based Measurement - A Proposed Alternative toStandardized TestsCurriculum—Based Measurement (CBM) has been proposedas an alternative to the use of standardized tests (Deno,1985; Galagan, 1985; Howell, 1986; Marston & Magnusson,1985; Shinn, 1988; Ysseldyke et al., 1983). Curriculum—Based Measurement should not be confused with Curriculum—Based Assessment (CBA) which has been defined as “anyprocedure that directly assesses student performance withinthe course content for the purpose of determining thatstudent’s instructional needs” (Tucker, 1985, p.200). WhileCBA is a generic term CBM is one particular form of CBA and5“has been developed empirically during the past decade andprovides clear guidelines for many dimensions of theassessment process including measurement methods, graphingprocedures, and aspects of data use” (Fuchs, Fuchs, &Hamlett, 1989a, p.430). The term Curriculum—BasedMeasurement “refers to a specific set of procedures createdthrough a research and development program supported by theInstitute for Research on Learning Disabilities at theUniversity of Minnesota” (Deno, 1987, p.41). Although CBMsvary in the literature, reading measures typically includeword lists and reading passages that are read orally by thestudent.Description of the StudyThere were three sets of variables or indicators usedin this study. These were: the set of norm—referencedmeasures, the set of Curriculum—Based Measures, and the setof school—based indices of reading skill. The set of norm—referenced measures was comprised of two subtests from theKaufman Test of Educational Achievement (Kaufman & Kaufman,1985). The set of Curriculum—Based Measures (CBMs) wascomprised of a reading speed arid a reading accuracy scorefrom a Word List and Reading Passages. For purposes ofdiscussion, these two norm—referenced and four Curriculum—Based Measures will be referred to as the set of6“independent variables”. It is understood, however, thatthe main focus of the study was to examine the relationshipbetween the set of the norm—referenced and the CBMs withthe set of “criterion measures” or “school—based indices”of reading skill.These “school—based indices” of reading skillreflected actual school district practice in allocatingteaching resources and evaluating students. In this studythese “decision measures” are often referred to ascriterion measures or dependent variables, even thoughthere is no intent to imply causation between the two setsof “independent” and the single set of “dependent”variables. The study instead focussed on the relationshipbetween sets of variables.The “school—based indices” included: a schooldistrict group-administered reading test (Core MasteryTest), a teacher rating scale, and learningdisability/regular education program placement status.Purpose of the StudyThe purpose of the study was to examine therelationships among the three sets of indicators (norm—referenced, curriculum—based, and school—based measures)and to ascertain whether or not the norm—referenced and the7Curriculum—Based Measures differ with regard to theirrelationship to the school—based “decision measures”.CBM Technical AdequacyCBM has been proposed as a viable alternative to theuse of norm—referenced achievement tests for the screeningand identification of students experiencing academicdifficulty (Germann & Tindal, 1985; Marston & Magnusson,1985; Shinn, 1988; Ysseldyke et al., 1983). Salvia andYsseldyke (1985) state that reliabilities of .90 or higherare necessary when a test score is to be used for importanteducational decisions, such as tracking and placement in aspecial class (p.127). These authors further state thatreliabilities of at least .80 are necessary if “thedecision being made is a screening decision, such as arecommendation that a child receive further assessment”(p.127). The study investigated whether or not the readingCBMs chosen have adequate reliabilities to meet acceptablestandards for screening and placement decisions.Prediction of School-Based Reading PerformanceIf the claims that CBMs have better curricular orcontent validity than norm—referenced tests are accurate,reading CBMs should show stronger relationships with8school—based reading performance indices than will norm—referenced measures. This study examined the technicalcharacteristics of CBM5, and whether they relate to school—based indices more validly than a norm—referenced test indistinguishing between normal and learning disabledreaders. Whether or not there is any advantage in“prediction” of the school—based indices to the combineduse of a CBM and a norm—referenced test was also examined.Correlations Between the Curriculum-Based Measures and the Norm-Referenced MeasuresThere is evidence that reading CBM5 have highcorrelations with norm—referenced reading tests (Deno,Mirkin, & Chiang, 1982; Derio, Mirkin, Chiang, & Lowry,1980; Fuchs & Deno, 198la; Fuchs, Fuchs, & Deno, 1982;Marston & Magnusson, 1985). This study investigated howstrongly reading CBMs correlate with the norm—referencedmeasures as well as the pattern of relationships betweenthe two forms of assessment.Comparison of the Two Curriculum-Based MeasuresOrally read word lists and reading passages have beenshown to have better validity than some other reading CBM5(Deno, Mirkin, & Chiang, 1982). Evidence also exists that9CBMs of longer duration are more reliable than shortermeasures (Fuchs, Tindal, & Deno, 1981). This studyinvestigated whether one CBM (Word List or ReadingPassages) is superior to the other when reliability andvalidity are considered and whether both measures combinedresult in a superior (compared to either one taken singly)Curriculum—Based Measurement of reading skill.Nature of the CriteriaSchool—based measures of reading performance have beenchosen as criterion variables because reading measures mustperform in manners acceptable to school systems beforetheir use can be advocated. One of the main criticisms ofnorm—referenced achievement tests is that they lack contentvalidity and are not acceptable to teachers (Eaton & Lovitt1972; Floden et al., 1980; Good & Salvia, 1988; Jenkins &Pany, 1978; Webster et al., 1986). A teacher rating scaleof student reading achievement was therefore chosen toinvestigate the relationship between teacher rating andnorm—referenced measures of reading. The relationshipbetween teacher rating of reading and Curriculum—BasedMeasures of reading was also investigated.An important determinant of an achievement test’svalidity in school settings is whether it can distinguishbetween regular program students and those placed in10special programs. For this reason thirty—five studentsplaced in special programs because of significant readingdifficulty were included in the study so that therelationship between norm—referenced reading performanceand placement status as well as curriculum—based readingperformance and placement status could be examined.A school district Reading Core Mastery Test used atYear Four was chosen as a third criterion variable becauseit is another school—based measure of reading skill. TheCore Mastery Test was drafted by a committee of teachersand contains content that teachers believe reflect studentschool performance.Research Questions1. Do Curriculum—Based Measures (orally read wordlist and reading passages) demonstrate sufficientreliability to meet conventional standards for screeningand placement decisions?2. What is the pattern of correlations, and what are theimplications of this pattern, between the curriculum—based and norm—referenced measures of reading ability?113. Is a Curriculum—Based Measurement battery or part of aCurriculum—Based Measurement battery comparable to,or superior to, traditional norm—referenced subtestsin identifying students with reading difficulties?4. Does combining the Curriculum—Based Measures withnorm—referenced measures result in increased precisionin prediction of the school-based indices of readingachievement?Justification for the StudyThere is evidence to support the notion that the solereliance on norm—referenced test information to diagnoselearning difficulty, make placement recommendations,develop instructional programs, and evaluate learning gainsmay be inappropriate (Fuchs, Deno, & Mir]cin, 1984; Tucker,1985; Ysseldyke et al., 1983). Curriculum-BasedMeasurement has been proposed as a reconunended alternativeto the use of norm—referenced achievement tests. Themotivation for designing this study is the author’soriginal concern based on field experience, that theinformation obtained from technically adequate norm—referenced reading tests is often incomplete. Further tothis, norm—referenced tests may not always reflect teacher12evaluations of achievement. As well, the advantage inusing CBMs for instructional planning has been argued(Fuchs, Deno, & Mirkin, 1984; Fuchs & Fuchs, 1986a, 1986b).More information about the technical adequacy of CBM5 andtheir relationship to school-based indicators ofachievement was sought. This information is necessarybefore CBM procedures might properly be advocated.Definition of TermsThe following definitions will be used throughout thisresearch:Learning Disabilities —“Learning disabilities is a general termthat refers to a heterogeneous group of disordersmanifested by significant difficulties in the acquisitionand use of listening, speaking, reading, writing, reasoningor mathematical abilities. These disorders are intrinsicto the individual, presumed to be due to central nervoussystem dysfunction, and may occur across the life span.Problems in self—regulatory behaviors, social perception,and social interaction may exist with learning disabilitiesbut do not by themselves constitute a learning disability.Although learning disabilities may occur concomitantly withother handicapping conditions (for example, sensoryimpairment, mental retardation, serious emotional13disturbance) or with extrinsic influences (such as culturaldifferences, insufficient or inappropriate instruction),they are not the result of those conditions or influences”(NJCLD, 1988, p.1)Norm-referenced test — These tests are used to differentiateindividuals by comparing them to a norm or referencesample. Items for these tests are selected to ensure an“expected degree of variation in performance” (Crocker &Algina, 1986, p.69).Criterion-referenced test — These kinds of test scores “derive theirmeaning from the examinees’ absolute levels of performanceon a series of test items in which each set of itemscorresponds to a known level of proficiency on a criterionof importance” (Crocker & Algina, 1986, p.69).Curriculum-Based Assessment (CBA) — This has been defined as any setof measurement procedures that use “direct observation andrecording of a student’s performance in the localcurriculum as a basis for gathering information to makeinstructional decisions” (Deno, 1987, p.41). CurriculumBased Assessment has three critical features: (1) CBAs“assess progress on short—term objectives; (2) once masteryof one objective is achieved, the measurement focus shifts;14and (3) the specific measurement tasks are designed bypractioners” (Fuchs & Fuchs, 1990, p.437).Curriculum-Based Measurement(CBM) - This has been defined as oneparticular variant of Curriculum—Based Assessment and “hasbeen developed empirically during the past decade andprovides clear guidelines for many dimensions of theassessment process including measurement methods, graphingprocedures, and aspects of data use” (Fuchs, et al., 1989a,p.430). While “most models of CBA rely on informal, nonstandardized procedures ... CBM is an empirically derived,standardized form of CBA” (Potter & Wamre, 1990, p. 16).In contrast to CBA5, CBMs emphasize assessment of long—terminstructional goals (most often the annual goal), aredesigned to assess students on alternate forms of the sametest, and are constructed with specific measurement andevaluation procedures (Fuchs & Fuchs, 1990).Curriculum-Based Measure— In this study a Curriculum—BasedMeasure is defined as : (a) a word list comprising 64 wordsrandomly sampled from Basic Elementary Reading Vocabulary(Harris & Jacobson, 1982); and (b) approximately 100 wordpassages selected from ten reading levels in Reading 720(Ginn and Company, 1979).15General Cognitive Skill - This is general intellectual functioningor general intelligence as defined by the whole or majorscales of an individually administered and technicallyadequate intelligence test.16Chapter 2Review of the LiteratureChapter OverviewCurriculum—Based Measurement (CBM) has been proposedas an alternative to norm—referenced tests (Deno, 1985;Galagan, 1985; Howell, 1986, Marston & Magnusson, 1985;Shinn, 1988). In this chapter CBM will be defined,information about CBM reliability, validity, andinstructional utility will be provided, and difficultieswith other identification and measurement approaches willbe discussed. These include a discussion of aptitude—treatment interaction research, characteristics of norm—referenced tests, as well as learning disability diagnosisand cognitive psychology. A case will be made that CBMprocedures show potential to address some of the currentlimitations of traditional approaches, although CBMpractices should not preclude the use of traditionalpractices, and a combined approach may be warranted.17Definitional Issues - Curriculum-Based Measurement andCurriculum-Based AssessmentIn the past decade, both Curriculum—Based Assessment(CBA) and Curriculum—Based Measurement (CBM) have becomecommonly used educational terms. Despite attempts atdefinition, the terms may sometimes be used interchangeablyand confusion may exist. Curriculum-Based Measurement isthe result of a research program at the University ofMinnesota Learning Disabilities Institute. They originallyused Curriculum—Based Assessment procedures derived fromthe principles of criterion—referenced testing and masterymeasurement (Fuchs & Deno, in press). Curriculum—BasedMeasurement was designed “to develop measurement andevaluation procedures that teachers could use routinely tomake decisions about whether and when to modify a student’sinstructional program” (Deno, 1985, p. 221).Curriculum—Based Measurement was also developed sothat reliable and valid procedures could be used byteachers to evaluate instructional programs. Whereas CBAprocedures vary and are related to short—term teachingobjectives, CBMs are usually more standardized in designand administration, and as well reflect long—term goalmeasurement. In other words, CBM procedures remain thesame over long periods (often one year) and measures are18commonly taken from the yearly achievement goal. Forexample, in reading CBM, the end of the school year readinglevel goal can be established and a number of readingpassages can be randomly selected as the set of measurementitems. This establishes the “measurement pool” from whichmonitoring probes can be drawn randomly (Fuchs, Deno, &Mirkin, 1984). These probes are administered in aprescribed manner and the difficulty level of these probesremains fairly constant. A long—term monitoring probemight involve oral reading speed, accuracy, and/orcomprehension of passages from a basal reader (Fuchs,1986).In contrast to this, Curriculum—Based Assessmentprocedures are concerned with short—term objectives andmonitoring probes are drawn from current instructionalmaterial. Because of this short—term objective strategy,CBA has a closer relationship to instruction and is moresensitive to immediate learning gains than is CBM (Fuchs &Deno, in press; Fuchs & Fuchs, 1990). If the short—terminstructional goal is mastery of r—controlled words, theCBA is reading a list of r—controlled words and scoringthe number of words read correctly. One method ofconstructing CBAs is to first examine the year longcurriculum, then establish an instructional hierarchy orinstructional sequence, and finally to design a CBA19procedure to match each step in the instructional hierarchy(Fuchs & Deno, in press). In this way, the instructionalhierarchy determines measurement.Since CBA is concerned with immediate curriculumcontent and single instructional units, it does notautomatically assess retention and generalization ofrelated skills (Fuchs & Deno, in press; Fuchs & Fuchs,1986c, 1990). A further limitation of CBA is the lack ofcomparability of CBAs from different instructionalsequences. There is no prescription in CBA that measuresare equal in difficulty or represent equal curriculumunits. The construction of progress graphs betweendifferent CBAs is not possible because of this lack ofequality; whereas when CBM procedures are used theestablishment of equal units is attempted and a progressgraph can be constructed. CBA procedures only allow forvery limited representations of learning rate; whereas theuse of fairly equal curriculum units in CBM allow for moremeaningful summaries of learning rate (Fuchs & Deno, inpress).Because CBAs vary and are constructed by teachers theypossess unknown technical characteristics. Sincereliability and validity studies are costly and timeconsuming, it is not possible to gather technicalinformation about teacher made tests. CBMs, on the other20hand, are often developed in similar manners that aredescribed in the CBM literature and some reliability andvalidity information is now available. Much of this workhas been done in the area of reading CBMs (Deno, Mirkin, &Chiang, 1982; Fuchs, Deno, & Marston, 1983; Fuchs, et al.,1982; Fuchs, Fuchs, & Maxwell, 1988) although technicalinformation about spelling, mathematics, and writtenexpression is also available (Deno, Marston, & Mirkin,1982; Deno, Mirkin, Lowry, & Kuehnle, 1980; Fuchs, et al.,1983; Tindal, Marston, & Deno, 1983).The emphasis on long—term goal measurement of a CBMhas several advantages when compared to CBA procedures. Itallows teachers to try different instructional materials,styles, and sequences and to use the CBM as a method forevaluating the effectiveness of different instructionalapproaches. It also enables the assessment of retentionand generalization of learning because it samples skillsacross the annual curriculum. Because of this, however,CBM5 are less sensitive than CBAs to the acquisition ofspecific skills. This may appear to be a limitation ofCBM, although evidence exists that CBMs are sensitive toinstructional changes (Fuchs, in press; Fuchs & Fuchs,l986a; Fuchs & Deno, in press) even if the degree ofsensitivity does not match CBA approaches. If the educatoris interested in the measurement of smaller skill blocks or21sequences then CBA approaches seem indicated. On the otherhand, if the concern lies in monitoring progress over ayear and obtaining measures that have good criterionvalidity or good relationships to other important indicesof achievement (Deno, 1985; Fuchs & Deno, in press), goodcontent validity, or representation of the true desiredoutcome (Fuchs & Deno, in press; Fuchs & Fuchs, 1986a), andgood concurrent validity (Deno, Mirkin, & Chiang, 1982)then CBM procedures are indicated.Another advantage of CBM procedures is that they allowthe comparison of individual differences among students.While CBA is concerned solely with intraindividualimprovement, CBM is capable of generating information onboth intraindividual improvement and interindividualcomparisons. In fact, CBM procedures have been proposed asan alternative to the traditional use of norm—referencedtests for the screening, referral, and identification ofacademically handicapped students (Marston, Mirkin, & Deno,1984; Shinn, 1988, 1989). Although more work needs to bedone in this area, CBM procedures are advocated foridentification because they unite the traditionalpsychometric concepts of standardized measurement,reliability, validity, and normative sampling with theconcepts of repeated behavior sampling, direct observationrecording, local norms, and graphic data display (Fuchs &22Deno, in press). The real potential of CBM procedures isthe ability to provide a meaningful assessment tointervention link by incorporating the best features oftraditional and direct observation forms of measurement.CBA procedures, while useful for ongoing classroommeasurement, fail to demonstrate this broader potential.Kinds of Informal Measures in ReadingInformal methods of measuring reading skill vary inthe literature but can be categorized into the followingkinds of measures: question answering, recall measures,doze responses, oral passage reading, and oral word listreading. When systematic procedures for developing readingmeasures are used, any of these kinds of measures couldbecome CBMs (Fuchs & Deno, in press). The most prevalentforms of CBM, however, are passage reading and word listreading. A discussion of five possible categories of CBMsfollows.Question AnsweringAlthough question answering is the most commoninformal reading comprehension strategy (Fuchs, Fuchs, &Maxwell, 1988), it has some inherent limitations. Question23answering may only tap comprehension of selected parts oftext that others have judged important, may be related tohow well answers can be inferred directly from thequestions without referring directly to the reading text(Hansen, 1979) and may yield different reading placementsfor a given student (Peterson, Greenlaw, & Tierney, 1978).To address the problem of generating questions that areconsistent across testers or passages (Johnson, 1982), aprocedure for formulating reading comprehension questionshas been developed (Jenkins, Heliotis, Haynes, & Beck,1986). Although question answering measures have beenfound to have adequate criterion, construct, and concurrentvalidity when this procedure is used (Fuchs, Fuchs, &Maxwell, 1988), it is a time—consuming process not easilyavailable to educational practitioners. Fuchs, Fuchs, &Maxwell (1988) state that “although question answeringappears to be an easy assessment device to create,developing questions that are a) representative of entirestories, (b) passage independent, and (c) useful foraccurately judging comprehension competency can be adifficult task to achieve” (p.21).24Recall ProceduresWhen reading comprehension is assessed through recallprocedures students are required to read passages and thenretell in their own words remembered content of thepassages while not referring back to the text. Althoughrecall procedures are popular as a dependent measure inreading comprehension research (Fuchs, Fuchs, & Maxwell,1988) methods for scoring can be difficult and time—consuming, the practitioner can infer little aboutinformation omitted from recalls (Johnson, 1982) and scantinformation exists concerning psychometric properties ofrecalls (Fuchs & Maxwell, 1987). Nevertheless, writtenrecall using the total number of words retold score hasbeen found to have acceptable criterion and constructvalidity (Fuchs, Fuchs, & Maxwell, 1988).Cloze TechniquesIn doze procedures, every nth word is omitted from apassage, a blank is inserted in its place and the pupil isrequired to provide the missing or appropriate word.Cautions pertaining to the use of doze procedures includethe concerns that the doze technique may measure textualredundancy rather than comprehension (Tuinman, Blanton, &25Gray, 1975), that the constraint operating on a doze itemmay reside within the sentence in which the item occursrather than in the actual textual context (Suhorsky, 1975),and that the technique does not tap inferentialcomprehension (Alderson, 1978). It has been found thatdoze procedures relate closely (correlations between .60to .83) to performance on standardized reading tests (Deno,Mirkin, & Chiang, 1982). The doze measure, which is oftenused to assess reading comprehension, has been found to bemore closely related to word recognition performance thanto comprehension performance on standardized tests. Deno,Mirkin, & Chiang, (1982) reported oral reading correlationsof .78 and .80 with two comprehension subtests from theStanford Diagnostic Reading Test (Karisen, Madden, &Gardner, 1975) but reported lower correlations of .67 and.71 between the doze comprehension measure and the twoStanford subtests. No test of the significance betweenthese differences is reported. From this they concludethat the reading comprehension validity coefficients fordoze measures at best equal those for measures based onreading words aloud from isolated lists and in context(Deno, Mirkin, & Chiang, 1982).26Oral Passage Reading MeasuresIn oral passage reading tests, students read aloudwhile examiners time them and score words correct anderrors, often in words correct and errors per minutescores. Words correct scores have been found to be morevalid than error scores (Deno, Mirkin, & Chiang, 1982;Fuchs, 1981; Fuchs, Fuchs, & Maxwell, 1988). Although oralpassage reading is not usually perceived as an index ofreading comprehension, this measure has demonstrated highcorrelations (ranging from .78 to .92) with standardizedreading comprehension criterion measures (Deno, Mirkin, &Chiang, 1982; Fuchs, 1981; Fuchs, Fuchs, & Maxwell, 1988).Further to this, growth over time on oral passage readinghas been shown to relate to growth on global tests ofreading comprehension (Fuchs, Deno, & Mirkin, 1984) and ithas been demonstrated that passages are sensitive toimproved reading skill (Fuchs, 1986; Fuchs, Fuchs, & Deno,1984). Because of this research, in most CBM systems oralpassage reading is the utilized reading measure (Fuchs &Fuchs, 1990).27Words in IsolationLists of words (often 60 words in length) have beenconstructed by randomly selecting from each grade level ofwords listed in Basic Elementary Reading Vocabulary (Harris& Jacobson, 1972). These word recognition procedures havebeen used widely in CBM research, and there is evidence tosupport their reliability and validity (Deno, Mirkin, &Chiang, 1982; Fuchs et al., 1983). Because the Harris andJacobson book was updated in 1982, current CBM word listsare developed from the more recent edition which containsreading vocabulary from grades one to eight and addsvocabulary from newly developed basal reading series.The Content Validity ProblemOne of the main criticisms of standardized achievementtests is that they lack content validity (Floden et al.,1980; Good & Salvia, 1988; Jenkins & Pany, 1978, Schmidt,1983; Webster et al., 1986). Content validity deficiencieshave been inferred from a study conducted by Jenkins andPany (1978) in which reading vocabulary in standardizedachievement tests is compared with reading vocabulary indifferent basal reading series. Reading vocabulary overlapis reported in terms of standardized achievement test grade28equivalent scores, that would be expected, given the wordsthat appear both as items on an achievement test and asinstructional content in a reading series. Jenkins andPany (1978) found clear discrepancies between the gradeequivalents obtained both between tests for a singlecurriculum and on a single test for different readingcurricula. Similar findings by other researchers (Flodenet al., 1980; Webster et al., 1986) have supported theclaim that norm—referenced achievement tests aredifferentially biased toward different reading curricula.These claims are based upon the assumption that actualstudent reading skill is limited only to vocabularyformally taught in basal reading series. While the kind ofreading vocabulary taught should logically bear somerelationship to performance on a standardized achievementtest, these researchers may have overemphasized theimportance of learned vocabulary in reading performance andunderestimated the ability of children to acquire newvocabulary once the reading process has begun. The failureto test real samples of children in these studies cannot beoverlooked since actual subjects may not perform in themanner predicted by test and curriculum vocabulary overlap.Standardized tests are designed to measure broad,generally accepted goals (Mehrens, 1984). Test publishersgo to great lengths to define their test’s content so that29they have as much in common with as large a set of schooldistricts’ ob:jectives as possible (Mehrens & Phillips,1986). In doing this, the liklihood that one specificcurriculum, that is, one basal reading series, will have asignificant vocabulary overlap with a standardized testdiminishes. Mehrens and Phillips (1986) point out thatsince all specifics cannot be taught one would hope thatgeneralized skills and understandings develop from any goodcurriculum.Some research indicates that the degree of matchbetween test and curriculum does not impact on norm—referenced achievement test scores (Gramenz, Johnson, &Jones, 1982). Mehrens and Phillips (1986) examined theproblem by obtaining expert ratings of test/curriculummatch on the mathematics and reading sections of theStanford Achievement Test (Gardner et al., 1982) and theCalifornia Achievement Test (Tiegs & Clark, 1970) and bydocumenting information on textbook series used. Dependentvariables were the raw scores on the standardizedachievement tests. MANCOVA results indicated that none ofthe overall multivariate F values were significant at the.05 level and the research conclusion was that neithertextbook series used nor ratings regarding the match of thecurriculum to the tests affect test results. The advantageof this study is that actual students were tested and they30did not achieve scores that supported previous researchsuggesting that students perform differently onstandardized tests according to the match between basalreader and standardized test content (Floden et al., 1980;Good & Salvia, 1988; Jenkins & Pany, 1978; Webster et al.,1986). To discontinue the use of norm—referenced readingtests as advocated by some researchers (Galagan, 1985;Jenkins & Pany, 1978) is premature and perhaps unwarranted.The important issue has been raised by Gerken (1985) andGresham (1983). Their assertion that norm—referencedachievement assessment on its own may not be sufficient tofully represent a student’s skill, even when a technicallyadequate instrument is used, seems reasonable. Further tothis, the difficulty in obtaining a technically adequatemeasure that contains enough normative data to makestatements about specialized populations must be considered(Fuchs, Fuchs, Benowitz & Barringer, 1987; Ysseldyke et.al., 1983). A sensible approach may be to consider bothnorm—referenced and curriculum—based assessment when makingstatements about achievement.The primary concern with reliance on norm—referencedachievement assessment should not be in the area ofcurriculum/test overlap but in the area of instructionalrelevance. Current research suggests that CBM proceduresare associated with achievement improvement (Fuchs & Fuchs,311986a). Similar research with norm—referenced tests hasnot been conducted.Aptitude-Treatment Interaction ResearchAptitude—treatment interaction research has centeredaround the identification of significant interactionsbetween student characteristics (cognitive or learningvariables) and appropriate interventions (Cronbach & Snow,1977). Some of the criticism of norm-referenced testinghas been related to the failure to find significanttreatment differences in aptitude—treatment interaction(ATI) research (Arter & Jenkins, 1979; Kavale & Forness,1987, 1990). In a meta-analysis of thirty-nine ATIstudies, Kavale and Forness (1987) found that modalitypreference groups were not clearly differentiated and thatthere was no benefit to subjects taught by methods matchedto their modality preferences. Student test scores areoften used to make statements about learning styles oraptitudes, that are then used to choose educationaltreatments (Howell, 1986). When this is done, certainassumptions are made. These assumptions include the beliefthat stable aptitude—treatment interactions exist and canbe reliably assessed before they are employed in32educational decision making. This may not currently bepossible (Ysseldyke, Algozzine, Regan, & Potter, 1980).Lakin (1983) asserts that “not only has thediagnostic-prescriptive approach lacked substantiatedeffectiveness in teaching children but, also, its generalacceptance in special education circles has encouraged thecreation of many essentially worthless, though profitable,enterprises of psychometry and treatment” (p. 236).Seemingly, much of the concern about educational planningfrom identified aptitude-treatment interactions has beendirected at the use of norm—referenced tests.A more useful endeavour may be to separate the use ofnorm—referenced tests from the identification and use ofaptitude—treatment interactions, because norm—referencedtests are useful even when the diagnosis of generalaptitudes and the prescription of treatments is avoided,and then examine the aptitude—treatment interactionproblem.The combining of time—series and conventionalmultisubject research has been advocated (Kratochwill,Brody, & Piersel, 1979). Kratochwill et al. state that a“marriage of time—series research with other researchstrategies may provide more refined answers to certainresearch questions” (p. 56). Time—series researchendeavours can provide the exploratory type of data needed33to pursue more large—scale research programs and theopposition between the two types of methodologies may beunnecessary.The potential in combining types of researchmethodologies should be kept in mind as one considers thecriticism directed at ATI approaches. Deno (1990) statesthat “our knowledge of how to design instruction toaccommodate individual differences has not progressed muchin twenty-two years”(p.161). He further states that thetheoretical and methodological problems associated with theuse of aptitude—treatment interactions “are so great thatno particular ATI relationship can be applied ininstructional design” (p. 162). He suggests that whileeducators and researchers believe that individualdifferences of students are very important in planningtreatments or educational programs, we are unable to applyATI methodology at present because the modest relationshipsidentified in well—controlled research settings may not beapparent in less controlled classroom environments; currenttests may be insufficiently reliable and valid measures oftraits that may be dynamic and interactive in nature; andbecause groups of students may not be homogeneous withrespect to aptitudes and, even if these generalizationscould be made, they may not be equally applicable to allmembers of the group. Deno presents formative evaluation34and the use of Curriculum—Based Measurement as analternative to the use of traditional psychometric tests.He asserts that individualized instructional programs notbe associated with ATI research. In other words, he arguesthat at present not enough is known about identifyingaptitudes and planning educational interventions to warrantthe use of such procedures in educational practice. Hetherefore advocates the use of CBM procedures to developand monitor individualized programs.CBM procedures may be used productively in time—seriesresearch, since an essential feature of time—seriesresearch is the emphasis on measures repeated frequentlyover time (Hersen & Barlow, 1976). The difficulty with CBMprogress monitoring, however, is that the emphasis has beenon measurement of achievement and not on the interventionsthat may be related to improved achievement. Anapplication of time-series methodology to CBM procedures inorder to address the problem of finding appropriateinterventions may be useful. This could set the stage formore refined experimental research.Reynolds (1988) agrees that few reliable ATIs havebeen found, and Phillips (1986) argues that thedifficulties go beyond methodological inadequacy of theresearch and that the problem is a conceptual one. Speece(1990) supports this contention suggesting that more35attention be paid to psychological rather than statisticalinteractions and that traditional ATI research has ignoredthe complexity of real life ATI5 which are transactional,multivariate, and developmental. Reynolds (1988) assertsthat a move from group ATI research to the individual ATIis needed and that good school psychologists have beenattempting to do this for many years. He refers toKaufman’s (1979) model of “intelligent testing” as a methodfor focussing on the individual with use of an ATIframework and likens the psychologist’s role in“intelligent testing” to “that of a detective attempting toferret out the relevant strengths and weaknesses of a childso that the best possible hypotheses about how to remediatethe child’s academic or other problems are derived”(p.326). Reynolds insists that well-developed andtechnically adequate tests of intelligence, achievement,and special aptitudes may be useful in this process butthat any one test or approach is most likely to beincomplete. He advocates a process of organizational andindividual consultation to facilitate meaningful academicinterventions and achieve individualized education,maintaining that only when true individualized education isoccurring can meaningful research begin.Whereas direct assessment of academic performance(Deno, 1990; Howell, 1986) is advocated as a better36practice for remediation than the search for meaningfulATIs, Reynolds (1988) indicates that the use of publishedstandardized tests should accompany formative evaluationtechniques. He states that an analysis of academic skills:should include at least a good norm—referencedevaluation of achievement (to determine actualdeficiencies relative to age and to intellectuallevel as well as to pinpoint well—developed areasof function) and more detailed diagnostic forms ofachievement testing. The latter might includecriterion—referenced tests, task analysis, informalassessment, or diagnostic achievement tests. (p.327)Reynolds (1988) and Deno (1990) appear to agree thatcurrent ATI research is not useful for educationalplanning, but differ on recommended approaches. Reynoldsadvocates the use of published standardized tests alongwith more informal techniques (including formativeevaluation) whereas Deno asserts that norm—referencedinformation is not a necessary adjunct to formativeevaluation or Curriculum—Based Measurement, even though thedevelopment of norms for CBM is often advocated.Although published norm—referenced tests have beenlinked to the difficulties associated with ATI research(Galagan, 1985; Howell, 1986), their usefulness is notnecessarily tied to the capabilities of these tests in37identifying cognitive processes. Reynolds (1988) iscorrect when he points out that norm—referenced assessmentis both useful and necessary “to determine actualdeficiencies relative to age and to intellectual level aswell as to pinpoint well—developed areas of function”(p.327). Also Deno (1990) is correct when he points outthe instructional benefits of formative evaluation. Theissue is not the superiority of one assessment method overthe other but rather in what circumstances or for whatpurpose is each method indicated.Difference ScoresNorm—referenced achievement tests have been criticizedbecause they do not adequately measure change or learning(Carver, 1974; Deno, 1985). They are sometimes usedinappropriately in pre-post treatment situations toestimate learning gains. The use of a CBM in the same pre—post treatment situation would also be inappropriatebecause the same problem with reliability of the differencescore would also exist (Crocker & Algina, 1986). Theadvantage of CBMs, however, is that they are not usuallyused in this way; rather they are used in manners thatrequire many test administrations and data are examinedgraphically rather than statistically. This procedure38avoids the problems associated with the reliability ofdifference scores.Advocates of CBM may be correct when they point outthe limitations of norm—referenced tests in measuringlearning gains or treatment effects in pre/post—testformat. Shinn et al. (1989) have summarized commonproblems with pre—post testing with norm—referenced testsas follows; “(a) testing-teaching mismatch (Good & Salvia,1988; Shapiro & Derr, 1987), (b) infrequent administrationmaking program modification difficult, and (c)insensitivity to individual student improvement (Marston &Magnusson, 1985)” (p. 356). Tindal (1989) states thatnorm—referenced assessment for group evaluation may beproblematic because the tests are often constructed withitem difficulty as the main selection criterion rather thanmatching tests to what is taught in the classroom. Headvocates the use of CBM procedures to include the benefitsof traditional norm—referenced assessment, criterion—referenced assessment, and individually referencedassessment. Further to this, systematic formativeevaluation or CBM procedures have been associated withgains in student achievement and these measures have beenshown to be sensitive to learning gains (Fuchs & Fuchs,1986a)39Tindal (1989) maintains that norm-referencedprocedures can be applied to CBM and that local norms canbe developed as was done in Pine County, Minnesota (Tindal,Germann, & Deno, 1983). If CBM measures are administeredat the same times for both the original norm group andspecial education students, indices of learning gains withreference to the local student population can be obtained.Although Tindal does not address this issue, it isnecessary that many testing occasions are available.Without many testing occasions, the same problem with thereliability of difference scores that is often associatedwith standardized achievement tests, applies to CBMs.Multiple occasion data gathering reduces the problemsassociated with the reliability of difference scores.Tindal (1989) states that data can be reported withconversion of raw scores to percentile ranks or with use ofa graphic plot or frequency polygon. Tindal argues thatusing traditional norm—referenced tests that may notreflect local curricula or student populations, does notyield the evaluation information that can be obtained fromwell developed CBM procedures.Curriculum—Based Measurement is also suggested as asuperior evaluation procedure to criterion—referencedassessment (Tindal, 1989). Concern is raised about thelimited empirical basis for establishing criterion—40referenced assessment, the need to exercise caution insampling from domains to ensure instructional sensitivity,and the difficulty in interpreting criterion-referencedassessment data. Further to this, Tindal (1989) maintainsthat individual—referenced approaches to assessment can beincorporated within CBM procedures that gather informationfor groups. Deno (1986) also states that individualstudent data can be aggregated across students to providesummaries of the average effectiveness of programinteractions (p. 372).Proponents of CBM suggest that published norm-referenced achievement tests be substituted with the use ofCBM5 administered over multiple occasions (Marston, Tindal,& Deno, 1982). Published norm—referenced achievementtests, unless they have alternate forms, should not beadministered repeatedly. The CBM alternative is attractivebecause repeated samplings increase reliability (Epstein,1980; Fuchs, Deno, & Marston, 1983; Nunnally, 1978) andprovide new indices of progress such as slope ofimprovement or learning rate (Marston & Magnusson,1985).Curriculum—Based Measures in reading that are administeredon many occasions may provide an alternative to theincorrect pre/post—treatment use of norm—referenced teststo measure learning gains. Although published norm—referenced achievement tests have been used in empirical41research to measure treatment effects (Handley, 1986,Rathbone & Graham, 1981, Trembley, Capanigo, & Gaffney,1980) and have been successful in documenting largedifferences in achievement, the difficulty with thereliability of difference scores is not addressed in thisresearch. While there is evidence that published norm—referenced achievement tests are sensitive to largeachievement differences, CBM5 have been found to be moresensitive to short—term student progress (Friedman, 1990).More information about the reliability and validity ofCBM is necessary, however, before conclusive statements canbe made about its utility in measuring change or learning.Additional research about the gathering of norms and thereporting of data is also necessary. It could be arguedthat the gathering of local norms is not essential to theuse of CBM to monitor progress or measure change inperformance, although the availability of local normativeinformation for purposes of comparison appears to beadvantageous (Tindal, 1989). It is apparent, though, thatmuch time and research is necessary before this can beaccomplished and that the inappropriate use of norm—referenced tests to measure difference scores orachievement gains may continue simply because norm—referenced tests are readily available and possess clear42scoring criteria and CBM procedures require a substantialknowledge base before data can be gathered and interpreted.Technical Adequacy and Norm-Referenced TestsAlthough technically adequate norm—referencedachievement tests are available, most do not include largenumbers of special education students in theirstandardization sample nor do they provide separate normsfor special education students. Although the Kaufman Testof Educational Achievement (Kaufman & Kaufman, 1985) hasbeen reviewed as a psychometrically sound achievementbattery, no systematic attempt to include special educationstudents was made when the test norms were being developed(Witt, Elliot, Greshaiu, & Kramer, 1988). While permissionslips for participating in the standardization were givento special education students, no record was kept of thenumber returning permission slips.This lack of inclusion of special education studentsin test norm development is a common situation in normreferenced testing. Fuchs, Fuchs, Benowitz, and Barringer(1987) reviewed user manuals and technical supplements to27 aptitude and achievement tests. They found thatalthough no test developer explicitly excluded handicappedstudents from the norms, only 5 of 27 tests provided43specific percentages of handicapped students in the norms.This is problematic because if tests are to be consideredunbiased they must demonstrate that they measure the sameskills among handicapped and nonhandicapped groups.Other difficulties associated with the use of norm-referenced tests include the finding that educators oftenchoose to use technically inadequate instruments in thedecision making process (Ysseldyke et al., 1980) and thatstandards of technical adequacy are not agreed upon byeducators (Thurlow & Ysseldyke, 1983). Although it ispossible to find technically adequate tests and clearcriteria for judging technical adequacy (Salvia &Ysseldyke, 1985) the common educational practice of failingto adhere to standards causes a sometimes unjustifiedabandonment of all norm—referenced test practices. Thereal concerns with the use of norm—referenced tests are thefailure to include or specify percentages of handicappedstudents in norms and the improper use of pre/post—testdifference scores.Curriculum-Based Measurement ReliabilityReliability can be defined as the degree to whichindividual’s deviation scores remain relatively consistentover repeated administrations of the same test or alternate44test forms (Crocker & Algina, 1986). In order to makemeaningful decisions about students, we must be confidentthat test results reflect an individual’s true score withina reasonable band of error, and that significantlydifferent results will not be obtained on another testingoccasion. Although some initial information about thereliability of CBM looks promising (Deno, 1985; Marston,1982; Marston & Magnusson, 1985; Shinn, 1981; Tindal &Deno, 1981; Tindal, Germann, & Deno, 1983; Tindal, Marston,& Deno, 1983) the reliability of CBM has not beenconclusively established. While some CBM5 enjoy highcurricular validity, the reliability of such assessment maybe unknown (Fuchs et al., 1983; Fuchs & Fuchs, 1986b;Tindal et al., 1985). Fuchs and Fuchs (1986b) furtherstate that part of the difficulty in establishing CBMreliability is that it is logistically difficult to studythe accuracy of each curriculum—related test because thesevary considerably.Aggregating observations over occasions is advocatedto improve the reliability of CBMs (Epstein, 1980; Fuchs etal., 1983; Fuchs et al., 1982; Mirkin, Deno, Tindal, &Kuehnle, 1982). Some CBMs have obtained high stabilitycoefficients over alternate occasions and do not requireaggregation (Fuchs et al., 1983). In one study, wordscorrect per minute scores from Ginn 720 third grade 20045word reading passages were found to have two—day stabilitycoefficients of .96; p<.001 (Fuchs et al., 1983). Thisstudy does not examine more lengthy test—retest intervals;the longest interval in this study is 4 days and thiscoefficient (.96) was calculated between the average ofdays one and three and the average of days two and four.Although the words correct per minute score is taken from arelatively short measure (a 200 word reading passage) thelack of improvement of reliability with aggregationindicates that this is initially a very reliable measure.Other studies have yielded some encouragingreliability results. Marston (1982) gathered data onreading word lists for 83 grades three to six students whoscored below the 15th percentile in written expression. Heobtained ten week test-retest coefficients of .82 and .90and mean parallel forms coefficients of .90 and .91. Shinn(1981) also obtained high reliability coefficients onreading word lists with 71 low-achieving and learningdisabled grade five students. His five week test-retestcoefficients were .90 and his parallel form coefficientswith a one week test re—test interval had a median of .91.Initial reliability results look equally promisingwhen reading passages are examined. Tindal, Germann & Deno(1983) examined 30 grade five regular education students onCBM reading passages and obtained a two week test-retest46coefficient of .97. They further examined 110 grade fourregular education students and obtained a parallel formscoefficient of .94 when forms were administered at the sametime. Tindal, Marston, and Deno (1983) used a large sampleof 566 randomly selected grades one to six students andobtained a ten week test—retest coefficient of .92, a oneweek alternate form coefficient of .89, and an interratercoefficient of .99 on CBM reading passages.Although more evidence is required, initial findingsindicate that passage reading CBMs may have acceptablereliability. It is difficult, however, to generalizereading CBM reliability findings because content andprocedures differ. This is illustrated when one considersthat Fuchs et al. (1983) obtained a high stabilitycoefficient when reading passage correct scores were usedbut obtained much lower stability coefficients when passagereading errors per minute were scored (r=.78, p<.OOl).Fuchs et al.’s (1982) claim that CBMs are strong in contentvalidity but haven’t yet demonstrated conclusivereliability may still hold.Most passage reading CBMs assume that assessment willoccur over more than one occasion. This enhances theopportunity for increased reliability (Epstein, 1980). Ifa CBM is to be used on one occasion, findings from Fuchs etal.’s (1981) study should be considered. The use of a more47lengthy reading CBM is indicated by their finding that thereliability of reading CBMs is related to the duration ofthe measure. They found that increasing sample durationfrom thirty seconds to a three minute sample reduced day—to—day variability in performance and resulted in a morerapid increase in student performance. They furtherquestioned the reliability and validity of traditionalinformal reading inventory passage sampling procedures andthe use of one—level floors and ceilings to estimatereading skill. Their findings indicate that if readingCBMs are not to be administered repeatedly and scores are- not to be aggregated, students should be required to readrepresentative passages from each level of a text ratherthan using a floor/ceiling approach. These findings arenot surprising since test theory indicates that the morelengthy the measure the more reliable it is (Crocicer &Algina, 1986).It is because of the above findings that a morelengthy reading CBM procedure was chosen for use in thisstudy where single occasion data was examined. Thisreading passage measure has been used in studies at theUniversity of Minnesota Institute for Research on LearningDisabilities (Fuchs & Deno, 1981a, 1981b; Fuchs et al.,1982). While these studies gathered validity informationon this passage reading CBM, no real reliability evidence48was obtained. This information was gathered in the presentstudy.Deno (1985) states that “a simple datum like thenumber of words read aloud correctly and incorrectly from abasal text reliably and validly discriminates growth inreading proficiency throughout the elementary school years”(p.224). Although some initial reading CBM reliabilityinformation looks promising, error scores have been foundto be less reliable than correct scores (Fuchs et al.,1983) and more reliability studies are necessary beforeconclusions can be made. The concern that issues ofreliability have been only minimally addressed (Fuchs,Deno, & Marston, 1983) still exists. There is presentlymore evidence to support reading CBM validity claims.Curriculum-Based Measurement ValidityIn the following section, three kinds of validity willbe discussed with reference to CBM procedures. These are:curricular, concurrent, and construct validity. Whether ornot CBM procedures have a closer relationship to schoolcurricula than norm—referenced measures will be examined inthe section on curricular validity. The relationship ofCBMs to norm—referenced achievement tests and to teacherrating of achievement will be discussed in the concurrent49validity section. CBM ability to differentiate learningdisabled from regular student groups, sensitivity toachievement growth, and relationship to developmentalreading models will be examined in the section pertainingto construct validity.Curricular ValiditySchool psychologists typically employ global tests ofacademic skills which are useful in summarizing studentperformance relative to other pupils (Sattler, 1988). Suchtests have been criticized because they fail to identifyspecific curriculum-related skill deficiencies (Jenkins &Pany, 1978), guide future instructional intervention (Fuchs& Fuchs, 1986b), and provide multiple alternate forms withwhich instructional effectiveness can be monitored (Fuchs,Deno, & Mirkin, 1984). Advocates of CBM, on the otherhand, state that because curriculum—based tests aretypically derived from curriculum materials, they can beemployed to identify specific curriculum-related skilldeficiencies, to guide additional instruction, and tomonitor instructional effectiveness with multiple alternateforms (Fuchs & Fuchs, 1986b).The claim for good curricular validity in CBM5 appearsto be a result of expert judgment (Fuchs et al., 1982;50Fuchs & Fuchs, 1986b) and an inference drawn from theapparent lack of curricular validity in standardizedachievement measures (Floden et al., 1980; Good & Salvia,1988; Jenkins & Pany, 1978; Webster et al., 1986). One ofthe problems with the tenets of reading CBM curricularvalidity is that the relationship between the chosen CBMand the actual student curriculum often remains unclear.While descriptions of Curriculum—Based Assessmentprocedures do make clear that the reading CBA is drawn fromactual local curricula, (Fuchs & Deno, in press; Fuchs &Fuchs, 1990), that is, from the prescribed district orclassroom basal reader, most reading CBM studies fail toestablish a link between the local reading curriculum andthe reading CBM employed in the research. Many studies userandomly sampled words from Basic Elementary ReadingVocabularies (Harris & Jacobson, 1972) which includesreading vocabulary from many basal reading series and istherefore not curriculum specific. The fact thatdescriptions of local curricula and resulting CBMs are notoften clarified in the literature leads to the possibilitythat reading CBM5 used in research may not reflect or maynot always reflect local curricula. For example, Fuchs etal. (1983) sampled a passage from the Ginn 720 readingseries to develop a CBM but did not provide evidence thatthe children in the study were using Ginn 720 as their51basal reader. An informal survey conducted in one schooldistrict by the author revealed that more than one basalreader is frequently used in one classroom and schoolsoften use several different basal readers at each gradelevel.In many cases, expecially with the advent of whole—language instruction, it may be impossible to define onelocal curriculum for use in developing a CBM. This issuemay be an unimportant point because it is possible that thebenefit of CBM approaches is in the fact that the materialsare more “curriculum—like” and tasks tend to resembleactual classroom practice more than those found in norm—referenced tests. If this is the case, this point shouldbe clarified in the literature and the issue of curricularvalidity defined as “curriculum—like” or “curriculum—possible” tasks. While it is possible to use the Ginn 720reading series as a teaching resource, it is not possibleto use a norm—referenced reading test for this purpose.A further point should also be raised in the area ofcurricular validity. Most passage reading CBMs employ areading speed measure that many traditional norm—referencedtests fail to include. This speed factor may bear a closerrelationship to actual classroom performance where timeconstraints are usually apparent.52Concurrent Validity- Curriculum-Based Measurement and Norm-ReferencedTestsThe largest body of validity research for reading CBMis the relationship of reading CBMs with standardizedachievement measures (Deno, Mirkin, & Chiang, 1982; Deno,Mirkin, Chiang, & Lowry, 1980; Fuchs, 1981; Fuchs & Deno,1981a, l98lb; Fuchs et al., 1982; Fuchs et al., 1983;Fuchs, Fuchs, & Maxwell, 1988; Marston, 1982; Marston &Deno, 1982; Marston & Magnusson, 1985; Skiba, Wesson, &Deno, 1982; Tindal et al., 1985). Deno, Mirkin, and Chiang(1982) obtained high correlations between five reading CBM5(words in isolation, words in context, oral reading, dozecomprehension, and word meaning) and the ReadingComprehension subtest of the Stanford Diagnostic ReadingTest (Karlsen et al., 1975) and the Word Identification andWord Comprehension subtests of the Woodcock Reading MasteryTest (Woodcock, 1973). Unfortunately they fail to providea correlation matrix between the curriculum—based and thestandardized measures, but they state that the correlationsbetween the three reading aloud measures (words inisolation, words in context, and oral reading) and thestandardized reading measures were highest and ranged from.73 to .91 with most coefficients in the .80 range (p.38).They also found that reading aloud measures correlated53higher than expected with standardized tests ofcomprehension.Fuchs, Fuchs, and Maxwell (1988) assessed therelationship between four informal reading comprehensionmeasures (question answering tests, recall measures, oralpassage reading tests, and doze techniques) and theReading Comprehension and Word Study Skills subtests of theStanford Achievement Test (Gardner et al., 1982). Theyfound that the average number of words read correctly perminute correlated with the Stanford Achievement Testsubtests with a mean of .89. They further found that theoral passage reading correct rate score correlated morestrongly with the Stanford Reading Comprehension subtestthan did each of the other three reading comprehemsionmeasures. This provides concurrent validity evidence forthe oral passsage reading measure and supports Deno,Mirkin, and Chiang’s (1982) finding that oral passagereading has high correlations with standardized measures ofreading comprehension.Deno, Mirkin, Chiang, & Lowry (1980) obtained moderateto high correlations (.57 to .95) between reading aloudperformance on a passage reading CBM and the WordIdentification and Passage Comprehension subtests of theWoodcock Reading Mastery Test (Woodcock, 1973).54Correlations varied according to which scoring criteriawere applied to the CBM.Marston and Magnusson (1985) obtained highcorrelations (ranging between .80 and .90) between thenumber of words read correctly in one minute from Ginnreading series readers (Clymer & Fenn, 1979) and subtestsfrom the Stanford Achievement Test (Madden, Gardner,Rudman, Karlsen, & Merwin, 1973).Fuchs and Deno (1981a) found high correlations betweenreading placements based on standardized tests (WordIdentification and Passage Comprehension subtests from theWoodcock Reading Mastery Test) and a reading CBM composedof reading passages from the Ginn 720 reading series(Clymer & Fenn, 1979). Despite the high correlations theyobtained, achievement test scores and curriculum—basedreading placement scores agreed for only fifty-eightpercent of the students sampled. These results may suggestthat reading CBM5 cannot be used interchangeably withstandardized tests.Concurrent Validity - Curriculum-Based Measurement and Teacher RatingSeveral studies have demonstrated a strongrelationship between teacher rating of reading skill andreading CBMs. Fuchs and Deno (l981a) found congruence55between teacher judgment of reading level and reading CBMs.Teacher congruence with CBMs was higher than withstandardized tests (average of 64.5 percent congruenceversus 48 percent).Fuchs and Deno (1981b) also found that for a group of91 first through sixth graders sampled from both regularand special education settings, reading fluency measureswere highly related to teachers’ judgment of studentreading proficiency (median r=.86). Marston and Deno(1982) also obtained evidence for the validity of CBM oralreading fluency when they found that the relationshipbetween oral reading fluency and teacher ratings of readingskill was significantly greater than teacher ratings withnorm—referenced achievement tests. Further to this, acorrelation of .77 between teacher judgment of readingachievement and a words read correctly CBM was obtained byMarston and Magnusson (1985). Teacher judgment wasmeasured with a rating scale developed for the study.Construct ValidityA psychological construct can be defined as “a productof informed scientific imagination, an idea developed topermit categorization and description of some directlyobservable behavior” (Crocker & Algina, p. 230). Construct56validation is necessary in situations where “no criterionor universe of content is accepted as entirely adequate todefine the quality to be measured .. .“ (Cronbach & Meehi,1955). This is true in reading measurement where no onecriterion can be found to be entirely adequate insupporting the conclusion that the measure in questionactually assesses the construct of reading. Further tothis, construct validation requires “compilation ofmultiple types of evidence” that support interpretation oftest scores (Crocker & Algina, 1986, p.231). It is alsoimportant to note that validity, in general, is a matter ofdegree, is an evolving property, and validation is acontinuingprocess (Messick, 1989). Evidence to supportthe construct validity of CBMs is reviewed in the followingsections pertaining to reading CBM ability to differentiategood and poor readers, reflect student growth acrossgrades, and relate to theoretical models of reading.Construct Validity - Differentiation Between GroupsDeno, Marston, Shinn, and Tindal (1983) found evidencefor the discriminant validity of one minute oral readingsamples when they differentiated learning disabled fromChapter 1 and regular education first—, second—, and thirdgrade students. Shinn and Marston (1985) also found that57words read aloud differentiated regular education students,pupils served in Chapter 1, and mildly handicapped studentswith learning difficulties. In addition, Marston, Tindal,and Deno (1983) demonstrated that reading CBM procedurespredicted learning disability classification as well astraditional measures of aptitude—achievement discrepancy.Construct Validity - Sensitivity to GrowthIf reading CBMs are valid, they should indicate growthas students’ skills improve. Some evidence exists thatreading CBMs are sensitive to growth. With a sample of 550grades one through six students, reliable gains in readingfluency were demonstrated across the grades (Deno, Marston,Mirkin, et al., 1982). Marston, Fuchs, and Deno (1986)examined reading progress of students across 10—week and16—week intervals with both standardized reading tests andCBM procedures. They found that reading CBMs delineatedgreater growth in the reading performance of students andcorrelated much more closely with teacher perceptions ofindividual student improvement.While these studies provide support for reading CBMsensitivity to growth, more research in this area isnecessary before conclusions can be made. Reading CBM doesshow promise in this area, however, when one considers the58inappropriateness of norm—referenced tests to measuregrowth in learning.Curriculum-Based Measurement and Developmental Reading ModelsCurriculum—Based Measurement research has failed toprovide a theoretical rationale for why CBMs are valid.Potter and Wamre (1990) argue that two reading modelsprovide a theoretical explanation for the validity of CBM’sreading rate measures.Potter and Wamre (1990) state that Chall’s (1983)stages of reading development and LaBerge and Samuels’(1974) model of automaticity in information processing bothview learning to read as a developmental process consistingof component skills that build on each other. These modelsemphasize decoding skills at the beginning levels ofreading, in contrast to holistic models that emphasizewhole-word meanings. They postulate that CBM reading ratemeasures are valid because they measure reading speed whichis a necessary precursor to comprehension skill. In bothreading models, the reader begins with letter recognition,proceeds to decoding, gains fluency, and developscomprehension skills. LaBerge and Samuels’ informationprocessing model indicates that the goal of fluent readingis automaticity of decoding skills so that the reader can59focus more attention on the meaning of what is being read.Potter and Wamre argue that CBM reading speed measuresreflect the automaticity of decoding skills necessary formature reading. They also assert that this provides anexplanation for the positive correlation betweencomprehension skills and decoding fluency found by Deno,Mirkin, and Chiang (1982) and Fuchs, Fuchs, and Maxwell(1988). Reading speed, then, may be a precursor skill tocomprehension. Further to this, reading aloud unfamiliarmaterials with acceptable fluency may be a more valid meansof assessing reading skills than are standardized tests(Anderson, Heibert, Scott, & Wilkinson, 1988).Although more explanatory information is needed aboutwhy CBM rate measures are valid indices of general readingskill, the linking of CBM rate measures to theaforementioned two developmental reading models provides abeginning point for understanding why CBM rate measuresassess reading ability.Summary of Reliability and Validity ResearchAt present, more evidence exists for the validity ofreading CBMs than for the reliability of these procedures.Validity research has been reviewed that demonstrates highcorrelations with norm—referenced reading measures, high60correlations with teacher rating of reading achievement,the use of these measures to distinguish learning disabledand non—learning disabled students, the existence ofdevelopmental growth patterns in reading CBM5, and therelationship of CBMs to developmental reading models. Themost convincing validity evidence is in the relationship ofreading CBMs to norm—referenced reading measures. Furthervalidity information in the area of differentiation ofgroups, sensitivity to growth, and relationship to teacherrating is needed. More information with goodrepresentation of statistical information is also necessaryin all areas of reliability. Initial reliability data,however, looks promising.Learning Disability DiagnosisIn the following section three kinds of research onlearning disability identification will be reviewed. Theseare norm—referenced measures, curriculum—based measures,and cognitive psychology and learning disabilityidentification.61Norm-Referenced Measures and Learning Disability IdentificationGalagan (1985) suggests that the practice of assessingstudents with psychoeducational or norm—referencedinstruments should be discontinued because the mostcommonly employed psychometric instruments are technicallyinadequate and produce considerable misclassification ofnon—handicapped students. Other researchers have alsoexpressed concerns about the technical adequacy of manywell—used norm—referenced tests (Ysseldyke et al., 1980;Ysseldyke, Regan, Thurlow, & Schwartz, 1981; Ysseldyke etal., 1983). - Ysseldyke, Algozzine, Shinn, & McGue (1982)also question the use of psychometric tests for learningdisability identification. In their study psychometricevaluation of learning disabled students and low—achievingstudents were compared and no reliable differences betweengroups in either individual test scores or in profiles ofscores were found. They further found an average ofninety—six percent overlap between scores for the twogroups. Ysseldyke, Algozzine, and Epps (1983) alsoemphasize that few psychometric differences between thelearning disabled and slow learning students exist whenstandardized tests are used. Algozzine and Ysseldyke(1987) found that the traditionally used method ofassessing ability—achievement discrepancies for children62suspected of having a learning disability was not useful inidentifying children with learning disabilities.Researchers also claim that current learning disabilityidentification practices are inadequate (Berk, 1982; Salvia& Ysseldyke, 1985; Sattler, 1988; Telzrow, 1985; Ysseldyke& Thurlow, 1983; Ysseldyke et al., 1983) and that continuedpsychometric refinements in diagnosis and classification oflearning disabilities are unproductive (Algozzine &Ysseldyke, 1986).The failure of norm—referenced test profile analysisto increase accurate learning disability identification(Epps, Ysseldyke & McGue, 1984) has added to a movementaway from the use of norm—referenced instruments by someeducators (Ysseldyke et al., 1983). Epps et al. (1984)found that undergraduates with no training in education orpsychology were more accurate than school psychologistsusing profile analysis in differentiating between learningdisabled and non-learning disabled students. Thurlow andYsseldyke (1983) point out that commonly used psychometrictests are used improperly because they are not validated onspecific populations. Ysseldyke et al. (1983) do state,however, that there are technically adequate norm—referenced tests that can be used to make decisions aboutstudents and that for the most part, these are restrictedto the domains of intelligence and academic achievement.63This statement introduces an element of confusion becausethe premise of much of the CBM research has been that norm—referenced tests are not useful in the identification of(Algozzine & Ysseldyke, 1983; Algozzine & Ysseldyke, 1986;Thurlow & Ysseldyke 1983) and program planning for learningdisabled students (Carver, 1974; Galagan, 1985). Do thesecriticisms of norm—referenced tests apply only totechnically inadequate tests or do they include technicallyadequate norm—referenced tests as well? Because the use oftechnically adequate norm—referenced tests is advocated(Salvia & Ysseldyke, 1985; Ysseldyke et al., 1980), can itbe assumed that at least some of the criticism of norm—referenced testing is directed at the use of technicallyinadequate instruments? Further to this, good schoolpsychologists may be able to use clinical skills along withtechnically adequate norm—referenced test information togenerate meaningful hypotheses about students with learningdifficulty when dealing with individual students (Reynolds,1988). This kind of clinical approach may not be evidentin studies of groups of students where generalized decisionrules are employed.64Curriculum-Based Measurement and Learning Disability IdentificationThere is some evidence that CBM procedures candifferentiate learning disabled from low-achievingstudents. Deno et al. (1983) found that learning disabledand low-achieving students identified by the schooldistrict performed differently on curriculum—based wordlists and reading passages. Using five weekly measurementsfor the word lists and sixteen weekly measurements for thereading passages they found that students in learningdisability programs read aloud more slowly and lessaccurately than do other low—achieving students. Shinn,Tindal, Spira, and Marston (1987) obtained similar resultswhen they used 250 word reading passages from Ginn 720(Clymer & Fenn, 1979) to distinguish between learningdisabled, low achieving, and regular education students.Level of CBM reading achievement emerged as the best singlepredictor of group membership with little contribution fromgrade level, ethnic background, or sex.Other researchers have also documented CBM’s utilityin differentiating students placed into differentinstructional groupings and handicapped classifications(Joyce & Wolking, 1987; Shinn & Marston, 1985; Shinn,Ysseldyke, Deno, & Tindal, 1986). While reading CBMprocedures appear to be useful in diagnosing students with65reading disability, it should be remembered that CBMinformation is gathered over weeks and sometimes months.The use of increased data gathering points increasesreliability (Epstein, 1980) and may result in improvedvalidity. Most norm—referenced test procedures only allowone data gathering instance and these measures are notconducive to several measurement occasions. The comparisonof multi—occasion data gathering with single occasion datagathering confounds the issue of norm—referenced versus CBMidentification practices. In many present educationalsituations, however, the gathering of multi—occasion CBMinformation is not practical. One of the purposes of thisstudy is to investig.ate the validity of single occasion CBMprocedures and norm—referenced procedures in theidentification of learning disabled students.Shinn (1988) makes reference to the feasibility of CBMprocedures when he states that “there are no data tosuggest that CBM screening and eligibility procedures aresuperior to current practice with respect to validity” (p.76) and that the justification for the expenditure of timeand resources in CBM data gathering or the development oflocal norms can be found in CBM utility for progressmonitoring and information for instructional planning. CBMprocedures may or may not be more efficient than norm—referenced tests in the identification of handicapped66students. The pertinent issue is what is the benefit tothe student from educational resources used? Although CBMprocedures for identification purposes are often more timeconsuming than norm—referenced procedures (due to multi—occasion data gathering), if they are used to monitorperformance and identify when instructionally relevantchanges are needed they are valid and cost/time efficient(Algozzine & Ysseldyke, 1986; Deno, 1985; Marston &Magnusson, 1985). Algozzine and Ysseldyke (1986) suggestthat it is time to abandon concern with identification andredirect resources toward corrective and preventiveeducational interventions. CBM procedures appear to bemore useful than norm—referenced assessment in thisendeavour.Cognitive Psychology and Learning Disability IdentificationThe concept of learning disabilities as achievementnot commensurate with ability due to neurologicalimpairment has been criticized (Shinn et al., 1987) and theneed for a consensus on defining learning disabilities hasbeen emphasized (Haimuill, 1990). While the formerresearchers criticize references to central nervous systemor neurological impairment because these theories have notbeen fruitful in benefitting students, Hammill asserts that67the National Joint Committee on Learning Disabilitiesdefinition which includes reference to neurologicalimpairment should be adopted. CBM researchers concludethat the efforts of the psychometric and the cognitivepsychologists have not been useful (Algozzine & Ysseldyke,1986; Shinn, 1989) and that educators should abandon thesekinds of endeavours and substitute more pragmatic CBMsolutions in identification and programming for studentswith learning difficulty. Although the efforts ofcognitive psychology may not be immediately transferableinto educational settings, the sound practitioner shouldpay attention to recent information in this field. It ispossible that this kind of research can immediately yielduseful information for hypothesis generation in individualclinical practice and may in future uncover some of themystery of learning difficulty.Hynd and Semrud-Clikeman (1989) hypothesize thatpoorly automatized semantic-linguistic and deficientmetacognitive memory processes distinguish disabled fromnondisabled readers. They cite supporting research thatstrongly implicates deficient semantic-linguistic memoryprocesses and inadequate phonological coding among disabledreaders (Barron, 1980; Gough & Hillinger, 1980; GrossGlenn, Lewis, Smith, & Lubs, 1985; Liberman, 1973). UsingBaddeley’s (1979) working memory model, Swanson, Cochran,68and Ewers (1990) obtained results that support thehypothesis that working memory processes underlieindividual differences in learning ability. Working memoryis conceptualized as a system for temporary storage andinformation manipulation. Swanson et al. (1990)administered a sentence span, a digit and word list, and aconcurrent memory task involving digit strings and cardsorting and found support for the ability of their memoryperformance battery to discriminate learning disabled fromslow learning and regular learners. Further research isindicated from these results and although no conclusion canbe drawn at present, it may be reasonable to investigatethe performance of children referred to schoolpsychologists for academic difficulty on more involvedmemory tasks so that hypotheses can be made about whetheror not severe learning difficulty exists. The notion thatpoor readers have basic visual—perceptual processingdeficits has not been supported (Stanovich, 1982;Vellutino, 1979) while the position that deficientsemantic—linguistic memory processes separates disabledfrom nondisabled readers is receiving increased support.This information should be available to clinicians as theymake identification and programming decisions aboutstudents.69A review of norm—referenced, curriculum—based, andcognitive psychological methods for identifying childrenwith learning difficulty reveals limitations in theadherence to any one approach. CBM procedures appear to bemost efficient in making identification and instructionaldecisions about large groups of students. More refined andoften hypothetical information may be available on anindividual basis from technically adequate norm—referencedmeasures and recent information from cognitive psychology.Eligibility for Special ServicesCBM procedures have been advocated as an alternativeto traditional or psychometric screening, referral, andidentification practices (Marston, Deno, and Tindal, 1984;Marston, Mirkin, & Deno, 1984; Shinn, 1988, 1989). Theseprocedures are deemed to be reliable, valid, and conduciveto the establishment of local norms (Deno, 1985).Shinn (1989) describes procedures for the developmentof CBM local norms and provides information on classroom,school, and district norms options. He advocates the useof reading passages 250 words in length randomly sampledfrom the grade level reader as an appropriate reading CBM.He presents a model in which normative data is gatheredthree times a year on three reading passages to reflect the70growth in the regular education classrooms. Shinnrecommends choosing the “most typical” grade level readerin situations in which more than one reader is used, butfails to address the problems in obtaining the “typical”reader in whole language situations. He provides examplesof CBM forming experiences, describing the efforts of theMinneapolis Public Schools (Marston & Magnusson, 1985,1988) and the Pine County Special Education Cooperative(Germann, 1985).Two types of CBM cutting scores have been used indetermining a student’s eligibility for special education.The most frequently used score is a discrepancy ratio thatis calculated by dividing the peer median score by thereferred student score. The cutting score for determiningeligibility for special education is typically -2.0 orgreater. Shinn (1989) states that research on the specificdiscrepancy score to use in CBM decision making has beenlimited. Marston, Tindal, and Deno (1983) found thatdifferent percentages of students may be identified byusing one cutting score for all the elementary grades. Forthis reason the second type of cutting score or percentilerank score is advocated by Shinn (1989). The percentilerank score is easier to understand and identifies the samenumber of students as eligible regardless of the referredstudent’s grade.71Shinn (1989) discusses single—step and multiple-stepeligibility models. In the former, the quality of norms isvery important because the student’s performance on CBMmaterials is compared to local CBM normative informationand eligibility is determined. In multiple—step modelsstudents are first assessed with a CBM and if their scoreexceeds the cutting score, the student may be furtherassessed. Marston and Magnusson (1988) examined theirsystematic screening process and reported thatapproximately 50 percent of the students who were referredprogressed to the second step of the eligibility process,resulting in considerable savings in assessment time.Although studies of the effects of multiple—step models arelimited (Shinn, 1989) evidence exists that they cause lowerreferral—to—placement rates. In the Minneapolis model, theprocedures resulted in referral—to—placement rates ofapproximately 25 percent to 45 percent (Marston &Magnusson, 1985), which are below accepted national ratesof 75 percent to 90 percent (Algozzine, Christenson, &Ysseldyke, 1982).Although more research on the use of locally developedCBM norms in special education eligibility decision makingis necessary, the available information suggests that thiscan be a valuable procedure. The expenditure of time andresources, however, can only be justified if CBM procedures72can be shown to be valuable in instructional decisionmaking as well as in placement considerations. As themovement toward integration of all students into regularclassrooms continues, placement considerations become lessimportant. While there is still a need, however, tojustify the expenditure of extra teaching resources forhandicapped students, eligibility will continue to be animportant concern and CBM procedures may be valuable inassisting these eligibility decisions.Curriculum-Based Measurement and Learning GainsConsiderable evidence exists that the frequent use ofreading CBM by teachers results in improved studentachievement. Ysseldyke et al. (1983) state that studentperformance can be improved by applying curriculum—baseddata utilization strategies and that students make moreprogress when their performance data are usedsystematically and teachers are satisfied with theprocedures. Mirkin, Deno, Tindal, and Kuehnle (1982) foundthat students make better progress when teachers applyspecific data—utilization strategies than when they usetheir judgment alone. Further to this, Fuchs and Fuchs(1984) found that when teachers rely on unsystematicobservation to evaluate student performance on objectives,73they are often inaccurate and optimistic about studentperformance. In their study teachers tended to beconfident in their judgments based upon unsystematicobservation and were highly inaccurate in their estimatesof the student’s actual level of performance when theydetermined that objectives were not met.Tindal, Fuchs, Christenson, Mirkin, and Deno (1981)determined that, although daily measurement is optimal,teachers can make satisfactory decisions on the basis ofdata collected three times a week. Marston et al. (1986)compared traditional standardized achievement tests andCBMs and found that CBMs were more sensitive in assessingstudent progress in reading and writing and related moreconsistently to a criterion measure of student growth (ateacher rating scale). In an earlier study (Marston, Deno,& Tindal, 1983) similar results were found when the ReadingComprehension and Language subtests from the StanfordAchievement Tests (Madden et al., 1973) and direct measuresof reading and written language were compared. Greaterstudent gains were evident on the direct measures than onthe standardized achievement test.Further evidence that CBM results in improved studentperformance was found. by Wesson, Skiba, Sevcik, King, andDeno (1984). In their study, one hundred and seventeenlearning disabled students were ranked by degree of74implementation of a curriculum—based measurement andevaluation system. Comparison of the top and bottomtwenty—seven percent of students on three reading passagesrevealed score differences on all passages, suggesting thatthe use of reading CBM5 can result in greater studentachievement.Fuchs, Deno, and Mirkin (1984) compared specialeducation students assigned to a curriclum—basedmeasurement/evaluation treatment to regular specialeducation students and found that CBM approaches resultedin greater student achievement as measured by a CBM PassageReading Test (comprising three passages from a third gradebook of the Ginn 720 series) and two subtests from theStanford Diagnostic Reading Test (Karlsen et al., 1975),than conventional special education. Results alsosuggested that CBM procedures affect pedagogy. Whereascontrast teachers’ structure decreased, CBM teachersincreased their structure. Contrast teachers were alsomore uncertain and more optimistic in judging studentgrowth. Further to this, they found that students weremore knowledgeable about their own learning as a result ofthe systematic measurement and evaluation treatment.Fuchs, Fuchs, and Stecker (1989) obtained similarfindings when they compared CBM teachers who used a oneminute passage reading measure to contrast teachers who75used their standard monitoring procedures. They found thatCBM teachers used more specific goals, were less optimisticabout goal attainment, cited more objective and frequentdata sources for monitoring student progress, and modifiedstudent programs more frequently. The improvement instudent achievement associated with CBM procedures may berelated to the change in teachers’ instructional planningthat occurs when CBM methods are utilized.Using twenty—one controlled studies, Fuchs and Fuchs(1986a) conducted a meta-analysis investigating the effectsof formative evaluation procedures on student achievementand obtained an average weighted effect size of .70. Fromtheir results they concluded that one “can expecthandicapped students whose individualized educationalprograms are monitored systematically and developedformatively over time to achieve, on average, .7 standarddeviation units higher than students whose programs are notsystematically monitored and developed formatively” (p.205). Their findings also suggested that achievementeffects may be enhanced when teachers also employ behaviormodification, data—evaluation rules, and graphed datadisplays. The authors concluded that systematic formativeevaluation or CBM procedures may be worth additionalteacher time.76Varying Curriculum-Based Measurement Difficulty LevelsThere is some evidence that varying the difficultylevel of CBM materials does not result in different slopesof student improvement. In contrast to the assumption thatincreasing difficulty of material results in differentslopes of student performance over time (Fuchs, Tindal, &Deno, 1984; Fuchs & Shinn, 1989), Shinn, Gleason, & Tindal(1989) found that, for most students, performance wassimilar across levels of curriculum. When they testedstudents in reading materials (200 word passages fromlevels 5 through 12 of Ginn 360) sampled one level belowand one level above their instructional placement level,there were no differences in rate of improvement. Theauthors encourage further similar research with largersample sizes before firm conclusions about varyingdifficulty levels can be drawn.Curriculum-Based Measurement and Goal StructuresWhen CBMs are selected it is important to addressissues related to short—term versus long—term goals. Infact, one of the definitional features of a CBM is that itreflects the yearly or more long—term goal, rather than theimmediate instructional or more short—term goal that is77usually associated with CBA methodology (Fuchs & Deno, inpress). Some information about goal structures is nowavailable.Fuchs, Fuchs, and Deno (1985) found that ambitiousnessof goals was associated positively with achievement butthat goal mastery was not. In a meta—analysis thatexplored how measuring student progress toward long—termversus short—term goals affected achievement outcomes,Fuchs and Fuchs (1986c) found that achievement effect sizeson global measures of achievement (Stanford DiagnosticReading Test) were higher when progress was measured towardlong—term goals than when progress was measured towardshort—term goals. They concluded that short—term goalmeasurement may be misleading because when this is usedprogress may be limited on more global indices ofachievement which better represent the true desiredoutcome. The authors recommend the use of long—term goalassessment (CBM) to validly assess pupil progress.In a study of CBM alternate goal structures inmathematics, teachers using dynamic rather than static goalCBMs obtained better student achievement results (Fuchs,Fuchs et al., 1989a). Dynamic goals were defined asteacher goals which can be adjusted upward when studentprogress suggests that a more difficult goal isappropriate. Static goals, on the other hand, were defined78as fixed annual goals that do not take into account therate of student progress. The effect magnitude associatedwith the dynamic goal CBM procedures was .52 orapproximately one—half standard deviation. Furtherresearch pertaining to dynamic versus static goals isnecessary before conclusions can be drawn and studiesemploying reading CBMs are necessary.Present research on CBM goal structures indicates thatthe use of ambitious, long—term, and dynamic goalstructures results in improved student achievement.Graphing and Computers in Curriculum-Based MeasurementSome research has been done in the the use of graphingand computers in CBM. Initial findings suggest thatachievement is associated with graphed displays (Fuchs &Fuchs, 1986a) but that method of graphing does not affectachievement (Fuchs & Fuchs, 1987). Fuchs and Fuchs (1986a)found that when data are charted rather than simplyrecorded, achievement improves approximately .5 of astandard deviation unit.One of the difficulties with the adoption of CBMprocedures by teachers has been the perception that CBM istime consuming (Wesson, King, & Deno, 1984). Currentresearch pertaining to the use of computers and CBM79indicates an increase in teacher satisfaction (Fuchs,Fuchs, Hamlett, & Hasselbring, 1987) and economy in time togather data (Fuchs, Hamlett, Fuchs, Stecker, & Ferguson,1988) with the use of computers over conventional recordingtechniques. Further to this there is evidence that greaterstudent achievement results when teachers inspect graphedassessment information from computers and use the databaseto determine when to introduce program adjustments thanwhen teachers use CEM procedures but fail to use thedatabase to determine when to make program adjustments(Fuchs et al., l989b). Computer supported CBM groups havedemonstrated greater achievement than control groups,although computerized expert system advice has not resultedin superior student achievemnent over teacher directedinstructional changes (Fuchs, Fuchs, Hamlett, & Allinder,1991). While more research on computers and CBM isnecessary, initial results indicate that the use ofcomputerized data collection can increase efficiency,teacher satisfaction, and student achievement.SummaryIn this chapter essential differences betweenCurriculum-Based Assessment (CBA) and Curriculum—BasedMeasurement (CBM) were discussed. While CBA is concerned80with short—term instructional goals, CBM adheres to longer—term goals (often yearly instructional goals), isstandardized in procedure, and is more researched in theareas of reliability and validity.The chapter also described the advantages anddisadvantages of different kinds of reading CBMs andaddressed the issue of whether or not CBMs have bettercontent validity than norm—referenced measures. It wasfurther argued that although current aptitude-treatmentinteraction research may have no practical application forlarge groups of students, individual ATI approaches may beuseful. At present, CBM approaches may have moreinstructional utility than ATI methods or diagnosticmethods arising out of psychometrics or cognitivepsychology. Information from these disciplines, however,should be surveyed to contribute to clinical work.The limitations of norm—referenced tests in themeasurement of learning gains and in representation ofspecial populations was discussed and information about thetechnical characteristics of CBMs was provided. There isnow reasonable evidence to support the validity of CBM5 andreliability information looks promising.CBM was examined as an alternative to norm—referencedtests for the screening, referral, and identification of81handicapped students and the possibility of local norms wasdiscussed.Evidence that the use of CBM procedures may be relatedto improved student achievement was presented and finallyinformation about CBM and goal structures, graphed datadisplay, and computer assisted data collection waspresented.This chapter provided a rationale for the use of CBM,an indication of its strengths and weaknesses, and ajustification for its use as a means of providing anassessment to instruction link. A review of the researchon CBM indicates that it is a promising tool in thedevelopment of meaningful instructional and assessmentprograms.82Chapter 3MethodologyThis chapter describes procedures for collecting andanalyzing the data. This description is divided into fivemajor sections: Design, Population and Sampling, TestingProcedures, Data Collection and Data Preparation, and DataAnalysis.DesignThe study was designed to allow the investigation ofthe properties of reading Curriculum—Based Measurement, inparticular how reading CBM data gathered on a singleoccasion, compares to norm—referenced information and howreading CBMs relate to performance on “school—based”indicators of reading skill. Three measures were chosen toreflect school—based estimations of reading performance.These were: a) a reading score from a school districtLanguage Arts test, b) a teacher rating scale, and c)placement status (whether or not the student was enrolledin a regular program or a program for severely learningdisabled students). These were determined to be the bestavailable estimates of how teachers view student83performance. Taken singly, these three indicators wouldnot have formed an adequate criterion for comparison, buttaken together as a set, these indicators reflect schooldistrict assessment of reading skill.Because the study’s design necessitated the inclusionof a learning disabled sample and because testing was heldto only one grade to facilitate interpretation of results,a completely random sample was not possible. Selection ofthe Year Four learning disabled students was determined byschool district elegibility procedures and these studentswere randomly matched with regular class students.Population and SamplingThe following discussion will be divided into asection on Identification of the Participants, and asection pertaining to Recruitment of Schools and Students.Identification of ParticipantsThe population selected for this study was determinedto be Year Four students in one suburban school district.Generalizations to other grades and to other districtscannot strictly be made from this study’s results.84Within the district 35 learning disabled students fromYear Four were available. Year Four was chosen because theschool district had a newly developed Language Arts testfor this grade that could be used as a criterion variableand because at this level the students would demonstratesufficient reading proficiency so that reading could beaccomplished at several levels and a passage readingmeasure could be used.These 35 students were matched with regular programstudents resulting in a total sample size of 105 students.For more information about the characteristics of thesample, refer to Table a, Characteristics of the Sample.After obtaining appropriate permission from theUniversity of British Columbia and the Director ofInstruction, Curriculum/Assessment as well as the Directorof Student Services from the suburban school districtchosen, subject selection procedures were undertaken.School district screening committee review forms wereobtained so that the 35 learning disabled subjects could bechosen. Each learning disabled subject in the study wasregistered in Year Four for the 1989 to 1990 school year,had an average range or higher Wechsler Intelligence Scale£Qr Children—Revised score (Wechsler, 1974) or an averagerange or higher Stanford Binet Intelligence Scale: FourthEdition score (Thorndike, Hagen, & Sattler, 1986), and a85TABLE 1Characteristics of the SampleCharacteristicsAge Mean = 9.81 StandardDeviation = .49Gender 65 males 40 femalesProgram 35 Learning 70 RegularDisabledNumber ofSchools 23Number ofTeacher Raters 24Average Number ofratings per teacher 4.3786significant lag in reading achievement as measured by theWoodcock—Johnson Psychoeducational Battery ReadingAchievement score (Woodcock, 1978). Further to this, eachlearning disabled subject had been referred by his or herteacher and area counsellor to the screening committee forplacement in a district Skill Development Program or aschool Learning Resource Program and had been determinedeligible and placed in a program for severely learningdisabled students.The author reviewed the screening committee summarysheets listing Wechsler and Woodcock—Johnson scores andwhen necessary, reviewed complete psychoeducationalassessments on the students. Intelligence scale percentileranks ranged from 21 to 88, including the top of the LowAverage, the Average, and the High Average range. In twocases, Full Scale I.Q. scores were not reported because ofthe large discrepancy between Verbal and Performance Scalescores. In these cases, average cognitive potential wasdemonstrated by an average range performance score whilethe Verbal Scale score was somewhat lower. In addition,lower Full Scale scores were examined by referring to theoriginal psychoeducational report. In these cases, averagerange cognitive skill was demonstrated with either higherPerformance or Verbal Scale scores.87Woodcock—Johnson Reading percentile rank scores rangedfrom 4 to 22. Twenty-eight out of the thirty-five scoresfell at or below the 16th percentile while seven of thescores ranged from the 18th to the 22nd percentile. Whenconsideration of prior learning assistance supplementaryassessment, teacher referral, and screening committeeplacement was made, all students appeared to demonstrate asignificant lag in reading when compared to generalcognitive skill.Forty-seven learning disabled subjects were identifiedas being eligible for participation in the study. Whenconsideration was given to principal and parental consent,mistakes in recording of grade placement, students leavingthe district, and availability of examiners, 35 learningdisabled students were finally selected.Examiners for the study were district areacounsellors, the author, and five Masters students inEducational Psychology at The University of BritishColumbia. Area counsellors reviewed the class lists of thelearning—disabled pupils in their schools, assigningnumbers to the sets of girls and boys in each class whowere not receiving learning assistance and were notenrolled in a Resource Room or a Skill Development Program.With use of a random numbers table each learning disabledstudent was randomly matched with two regular program88students in the same class and of the same gender.Parental consent was then obtained for these regularprogram students. When parental consent was declinedanother regular program student was selected with use ofthe random numbers table and parental consent was obtained.Recruitment of Schools and StudentsAs mentioned, after the identification of potentialsubjects for inclusion in the study, area counsellors werecontacted to determine if they would be willing tosupervise or gather data for students in their schools.Then a list of involved schools was made and a letter wassent to the principal of each school (see Appendix B). Allbut one principal agreed to participate in the study. Inmany cases personal contact by the author was necessary toexplain the importance of the study and the lack ofanticipated disruption to the students and teachers. Areacounsellor support at this time was invaluable in gainingprincipal and teacher support.After approval by the school principals, areacounsellors forwarded letters to the parents/guardians ofeach subject. Letters contained a description of the studyand sought permission from the parents for inclusion oftheir child in the study (see Appendix C). Different89letters were sent to the parents of learning disabledchildren to emphasize the importance of obtaining theirpermission (see Appendix D). Only one out of forty—sevenlearning disabled students was unable to participatebecause of parental refusal. Area counsellors did not keeprecords of regular program student parental refusal,although these were estimated to be less than five out ofseventy students.Testing ProceduresIn the study a group of predictor measures was used todemonstrate a relationship to a group of criterionmeasures. The predictor measures fell into two batteriesor kinds of assessment. The first was the two norm—referenced reading subtests from the Kaufman Test ofEducational Achievement (Kaufman & Kaufman, 1985). Thesecond was the curriculum—based reading battery composed ofa set of ten reading passages and a word list.The criterion measures were chosen to reflect schooldistrict estimation of reading. These were a teacherrating scale of reading proficiency, a school district coremastery test from which the reading section score was used,and learning disabled or regular program placement status.90Test Selection CriteriaTwo kinds of tests were chosen as predictor measuresin the study — norm—referenced and curriculum—based. TheKaufman Test of Educational Achievement (KTEA) (Kaufman &Kaufman, 1985) norm—referenced subtests were selectedbecause they possess good reliability and validity, arecommonly used, and are recently published. The curriculum—based reading passages and word list were selected becausethey have performed well in the Curriculum—BasedMeasurement literature. The following is a list of tests- used as predictor measures in the study:Kaufman Test of Educational Achievement ReadingDecoding SubtestKaufman Test of Educational Achievement ReadingComprehension SubtestCurriculum—Based Reading PassagesCurriculum-Based Word ListKaufman Test of Educational Achievement (1985; KTEA)The Kaufman Test of Educational Achievement hasexcellent psychometric properties. Mean split-halfreliability coefficients for the Reading Decoding, ReadingComprehension subtests and the Reading Composite were .95,.92, and .96 respectively by grade in the standardizationsample and .95, .93, and .97 respectively by age in the91standardization sample. The manual reports that test—retest reliability coefficients for grades one to six were.95, .92, and .96 for Reading Decoding, ReadingComprehension, and the Reading Composite. Standard Errorsof Measurement for the Reading Composite are small andaverage about three points both by grade and age.The manual also provides good evidence for thecontent, construct, and concurrent validity of the KTEA.An example of concurrent validity evidence presented isthat KTEA Reading Decoding correlated .84 with PeabodyIndividual Achievement Test (PIAT: Dunn, & Markwardt,1970) Reading Recognition and KTEA Reading Comprehensioncorrelated .74 with PIAT Reading Comprehension. KTEAReading Decoding and Reading Comprehension respectivelycorrelated .71 and .78 with the Stanford Achievement Test(Gardner et. al., 1982) Reading subtest.Good evidence for the technical adequacy of the KTEAcan also be found in Kamphaus, Schmitt, & Mings (1986),Sattler (1988), and Witt et al. (1988).Curriculum-Based Reading PassagesThe Curriculum—Based Reading Passages used in thisstudy were obtained from Dr. L.S. Fuchs, VanderbiltUniversity, and have been used in previous research in92Curriculum—Based Measurement (Fuchs & Deno, 1981a, 1981b;Fuchs et al., 1982; Fuchs et al., 1983).The one—hundred word passages were selected asmeasures from each of ten reading levels in Reading 720(Ginn and Company, 1979). An elaborate passage samplingprocedure adapted from Fuchs and Balow (1974) was employedto ensure that these passages were representative of thereading diffficulty of the levels from which they werechosen. Fuchs et al. (1982) found that for over one—halfof the nineteen books used in their study, adequatereadability representation was not achieved until ten ormore passages were sampled. They further found that:despite the use of representative passages, that infact did increase in difficulty within each readingseries, students’ performances did not necessarilyweaken as a function of this increasing difficulty.An average of only one—half to three—quarters of meanperformance scores decreased on adjacent passages.(p.19)For this reason these authors state that examiners shouldrequire students to read representative passages from eachlevel of a text rather than using a floor/ceiling approach.They also insist that passage selection procedures bethorough and that the arbitrary selection of readingpassages be avoided. This difficulty with passage sampling93can be overcome and single passages from one grade levelcan be used if many testing occasions are available. Thedesign of this study, however, did not permit multipleoccasion CBM data gathering and the research questionspertained to comparing a CBM administered on a singleoccasion to a norm—referenced test administered on a singleoccasion.These passages were also chosen because theyconstitute a longer measure and therefore have betterpotential reliability (Crocker & Algina, 1986; Epstein,1980) and because they appear to be the most valid ofsingle occasion curriculum—based reading passage measuresavailable (Fuchs & Deno, 1981a).The design of this study made necessary the use ofone Curriculum—Based Measure rather than several CBMs.Multiple occasion CBM data gathering was too disruptiveto the schools and was not supported by the examinersor principals. Further to this, the concern with accuracyof passage sampling and varying readability levels atsingle grade levels found in previous research (Fuchs,Fuchs, & Deno, 1982) made Dr. Fuch’s carefully selectedpassages attractive.Some of the study’s examiners expressed concern overthe use of the Reading 720 (Clymer & Fenn, 1979) readingseries because it may be no longer the most commonly used94basal reader. An informal assessment done by the schooldistrict area counsellors in 1989 revealed that readingseries varied within and between classrooms and that no oneseries was more prevalent than another. Reading 720 was ascommon as any series and the lack of a prevalence of anyone series made the use of Dr. Fuch’s passages feasible.The reading passages and the word list were scoredseparately for reading speed and reading accuracy toevaluate the independent contributions of speed andaccuracy. Reading accuracy has been used as an indicationof student achievement for several years (Betts, 1946;Gickling & Thompson, 1985; Hargis, 1987; Kender, 1969;Powell, 1971; Smith, 1959. Oral fluency and speed measuresalso seem to be receiving more attention in theprofessional reading literature (Bowers, Steffy, & Tate,1988; Walsh, Price, & Gillingham, 1988). Also, if thetheoretical models of reading are accurate and automaticityof decoding is a direct correlate of comprehension, oralreading rates may, in fact, be a more theoretically soundmethod of assessing overall reading skill, at least at somestages of reading development, than are many traditionalmethods (Potter & Waiure, 1990). These authors also statethat “reading rate measures would provide a measure, withproven technical adequacy, more like the subjects’ everydayreading experiences” (p. 22). Although some researchers95advocate the use of a combined metric or correct words perminute measure (Deno, 1986) this applies to passagessomewhat longer in length than the ones used in this studyand to passages sampled from the child’s basal reader. Italso applies to situations in which multiple occasion datais gathered and graphed, and measures need to be gatheredquickly. The correct words per minute measure is a shortmeasure designed to be given on several occasions. Theintent of this combined metric is to avoid the ceilingproblems associated with reading accuracy measures drawnfrom the classroom reader. Although this wasn’t the casein the present study, it was hypothesized that the sameceiling problem would not exist for grade four readers whowere required to read beginning grade seven material. Anopportunity was seen in this study to gather informationabout the unique contributions of the speed and accuracymeasures.The correct words per minute scoring method wasavoided in the passages because of the difficulty withexaminer error in the correct words per minute scoring inthe CBM word list. It is difficult for examiners to attendto accuracy of words while the student reads aloud as wellas to be aware of the one minute demarcation. To gaininformation about the relationship between the passagespeed measure and the correct words per minute measure,96correlations were calculated between the two measures withcomputer support, thus avoiding the problem of examinersattending to one minute time samples as well as accuracy.Correlations were calculated with the SPSS-X Release3.0 program (SPSS Inc., 1988). In the total sample thecorrelation between the CBM reading passages averagereading speed measure and the CBM reading passages averagecorrect words per minute measure was .998; p<.O0l. For theregular education sample the correlation also was .998;p.c.OOl. When the learning disabled sample was analyzed,the correlation was .98; p.<.OOl. These very highcorrelations indicate that the two measures are virtuallythe same, and that the correct words per minute measure isin reality a speed measure. The extra examiner demand ofscoring correct words per minute rather than simple readingspeed may not be warranted. In any case, the highrelationship between the two measures providesjustification for scoring speed separately from accuracy.Curriculum-Based Word ListA word recognition measure similar to the onedeveloped in Deno, Mirkin, and Chiang’s (1982) study waschosen as another Curriculum—Based Measure of reading97skill. Deno, Mirkin, and Chiang (1982) developed a Words inIsolation measure consisting of:three alternative forms of 60 words each that wererandomly selected by grade level from the Core Listof 5,167 words listed in Basic Elementary ReadingVocabulary (Harris & Jacobson, 1972). Each 60-wordlist consisted of 10 words from each of the sixgrade levels. Words were included on the word listsonly if they had a frequency index of more than 10per million words in the Teacher’s Wordbook of 30,000Words (Thorndike & Lorge, 1944). After a pool of 60words was obtained for each list, the words were typedin 12 rows with five words in each row. (S.L. Deno,personal communication, February 6, 1989).Because Basic Elementary Reading Vocabulary (Harris &Jacobson, 1972) has been updated to Basic ReadingVocabularies (Harris & Jacobson, 1982) and because thelatter book accomodates the major changes that have takenplace in basal—reader vocabulary since the 1972 lists werecompiled, Basic Reading Vocabularies was substituted forthe earlier edition. The 1982 edition shows a largeincrease in the number of word entries when compared to the1972 book. Further to this, the newer book containsreading vocabulary from Year Seven and Year Eight basalreaders, while the 1972 edition contains only reading98vocabulary up to Year Six. Reading series surveyed werealso changed from 1972 to 1982, reflecting the revision andaddition of new basal reading series.Three 64—word lists were developed using a similarprocedure to that cited in Deno, Mirkin, and Chiang’s(1982) study. The 1982 edition of the Harris and Jacobsonbook was substituted for Basic Elementary ReadingVocabulary (Harris & Jacobson, 1972). With use of computergenerated random numbers, eight words from each of theeight grade levels were chosen to comprise each 64—wordlist.A pilot study of the Curriculum—Based Measures wasconducted in June of 1989. The Curriculum—Based. Passagesand all three Word Lists were administered to 20 Year Threechildren by this researcher and another rater. Lists Twoand Three seemed somewhat easier for the children and ListOne was found to have a slightly lower mean score when thepilot data were analyzed. Administration time constraintsnecessitated the omission of two lists. List One wasretained because it appeared easier to administer (childrenread this list somewhat slower, meaning that examinerscoring was less rushed — mean seconds at 59.84 versus46.58 and 52.32 for the other two lists) and yielded alower mean score, resulting in increased discriminationpotential. A similar word recognition procedure has been99used in many Curriculum—Based Measurement studies (Deno,Mirkin, & Chiang, 1982; Deno, Mirkin, Chiang, & Lowry,1980; Fuchs et al., 1983; Fuchs, Tindal, & Deno, 1984:Marston, Lowry, Deno, & Mirkin, 1981; Tindal & Deno, 1981).Deno, Mirkin, and Chiang (1982) found correlations betweentheir word recognition test (Words in Isolation) andstandardized reading measures ranging from .73 to .91,providing evidence for the validity of the Words inIsolation measure. Using a similar word recognition test,Fuchs et al., (1983) found that the measure had highreliability initially (r=.94, p.<.OOl) and aggregation hadlittle impact on improving reliability. Fuchs, Tindal, andDeno (1984) found concurrent validity between a wordrecognition measure adapted from Deno, Mirkin, & Chiang(1982) and informal comprehension tests. Word recognitiontests sampled from Basic Elementary Reading Vocabulary(Harris & Jacobson 1972) have consistently performed wellthroughout the Curriculum—Based Measurement literature.Graded word lists have also demonstrated high correlationswith functional grade placements derived from morecomprehensive reading measures (Froese, 1971, 1975, 1976).100Selection Criteria: Decision or Criterion MeasuresCriterion measures in the study were the bestavailable estimates that could be found of school-basedindices of reading performance. One intent of the studywas to gather information about how curriculum—based andnorm—referenced tests perform in relation to districtinformation about students’ reading capabilities. Thefollowing three school—based measures were used in thestudy:Teacher Rating ScaleSchool District Core Mastery Test: Grade 4 ReadingPlacement Status — Learning Disabled or RegularProgramTeacher Rating ScaleTeacher judgment of student academic achievement hasbeen found to be accurate. Gresham, Reschly, and Carey(1987) found that teacher judgments were as accurate inseparating learning disabled and non—handicapped groups asstandardized tests of intelligence and achievement.Oakland, Shermis, & Coleman (1990) also found that teacherrating accurately discriminated students later classifiedas learning disabled from students not subsequentlyclassified as learning disabled. Oliver and Arnold (1978)101found that, when third—grade teachers were asked toestimate instructional reading levels, the teacherjudgments and the standardized test scores were notsignificantly different. Further to this, they found acorrelation of .81 between teacher judgments and InformalReading Inventory scores, and a correlation of .74 betweenteacher judgments and a standardized test. Morine—Dershimer(1978) also found that teachers were able to predictreading achievement placements on national test norms.Teacher rating has been shown to demonstrate significantconcurrent validity with Comprehensive Tests of BasicSkills (CTBS) scores and the concurrent validitycoefficients for language arts, reading, and math weresignificantly higher than those for science and socialstudies (Hopkins, George, & Williams, 1985). Teacherratings have also demonstrated concurrent validity with thePeabody Individual Achievement Test (PIAT), the CaliforniaAchievement Test (CAT), and the Wide Range Achievement Test(WRAT) (Bray & Estes, 1975). In another study, teacherswere asked to predict whether their third, fourth, andfifth grade students had responded correctly or incorrectlyto selected items on the Science Research Associates (SRA)Achievement Series. Aggregate measures of teachers’judgments of their students’ responses correlatedpositively and substantially with aggregate measures of102students’ actual responses (Coladarci, 1986). It has alsobeen found that teachers are able to differentiate effortand achievement when assigning ratings (Wright & Wiese,1988)There is evidence that teacher rating is not affectedby information about students from other sources. Cahill(1979) investigated whether giving teachers informationabout student performance on standardized tests influencedtheir academic rating of students. She concluded thatgiving teachers varying amounts of information aboutstudents’ test performance did not have a major impact onthe way in which teachers viewed their students and thatthe addition of a piece of information that was extrinsicto the classroom setting (standardized test scores) was notpowerful enough to influence teachers’ perceptions.Teacher rating has been found to be an accurateindicator of student achievement and has even been used asthe principal criterion measure in some research (Webster,Hewett, & Crumbacker (1989). In other research, teacherjudgment of reading level has served as one of a number ofcriteria (Fuchs & Deno, 1981b; Fuchs, Fuchs, & Deno,(1982)The Teacher Rating Scale used in this study wasdevised by modifying a scale developed for earlier schoolbased educational research conducted in British Columbia103(Robinson, Conry and Conry, 1985). It was originallydesigned as a seven—point, equal—appearing—interval scalewhich was presented to teachers in pictoral form andreferenced to the normal distribution; a number of ratingdimensions were presented and teachers were asked to usethe demonstrated scale to supply ratings for each of theirchildren on each of the given dimensions. The authorsreported that the internal consistency reliability of thisteacher rating scale (with four academic dimensions beingrated) was .88 when used with teachers in schoolscontaining a large proportion of Native Indian Students;the stability coefficient, over a four-week period in theFall of the year, was .83. They also reported that when adistribution was formed by combining the ratings of 12teachers and 345 students in two intermediate grades fromone school district, that distribution did not departsignificantly from the normal curve when tested with a Chisquare test of normality.For use in this study, three “dimensions”, or ratingconstructs, were attached to the pictoral rating scale (seeAppendix G). Thus teachers were asked to rate students ona seven point scale in three areas: overall reading skill,oral reading fluency, and reading comprehension. On thesame form, an additional question asked whether a studenthad been considered for psychoeducational assessment. A104third question asked how many grades the child hadrepeated, but was not used in analyses because a number ofteachers said that they did not have that information.A Hoyt estimate of internal consistency was calculatedfor the three questions pertaining to reading skill in thisstudy’s teacher rating scale. Using the Lertap 2.0 program(Nelson, 1974) a reliability coefficient of .97 wasobtained, providing evidence for the reliability of theaverage of the three reading ratings used in much of thedata analyses.School District Core Mastery Test: Grade 4 ReadingThis group administered test had recently beendeveloped by a district conuuittee of teachers to measurestudent performance in Reading, Listening, Writing, andSpeaking. For this study only the Reading section wasadministered (see Appendices N, 0, P).Placement StatusStudents were assigned to Group 1 —— also referred tohereafter as Learning Disabled or “LD” —— if they wereenrolled in a Skill Development Program or a LearningResource Room during the 1989—1990 school year. These105children had been assigned to these placements by thedistrict screening committee. Placement in such programswas determined, in part, by performance on a norm—referenced reading test, usually the one from the Woodcock—Johnson Psychoeducational Battery (Woodcock, 1978).However, placement status also depended upon teacherreferral, learning assistants’ interventions and reports,and counsellor referral. Hence the “placement status”indicator was composed of professional experience andjudgement—— often from several professionals in differingroles, and information from standardized tests usedroutinely in the district for such decisions; thus it isnot possible to determine the influence, or “weight”, ofany contributing element in the determination of groupstatus for any child. Students were assigned to Group 2—— also referred to hereafter as “Regular Education” ifthey were enrolled in the regular class program with nosupport from a Learning Assistance Center, a LearningResource Program, or a Skill Development Program.In choosing predictor scales for this study, care wastaken to select standardized measures (the KTEA subtests)which were different from the tests used in placementdecisions, to minimize the overlap between this criterionindex (placement status) and the predictor variables to aslow a level as possible. The placement committee did not106know the results of KTEA testing when their decisions weremade, and the KTEA was not given, for purposes of thisresearch, until one to two years after placement status hadbeen determined.The use of extant groupings as variables is notuncommon in applied educational research; the opacity ofthe actual dimensions, and the resistance of such variablesto componential analysis, is a limitation frequentlyimposed on the findings of studies which utilize school-based indicators of instructional decisions. One majoraspect of the study was to ascertain the relationshipsbetween school—based indeces and other measures: theplacement status index was used despite the above—notedlimitations because it is the final, cumulative index ofschool—based decisions regarding the instructionaltreatment of children in reading.Data Collection and Data PreparationData was collected by 22 examiners, all with level “C”training in test administration (Croribach, 1970, p.18).Sixteen of the examiners were school district areacounsellors, while five were Masters students inEducational Psychology at U.B.C. The author alsoparticipated in the data gathering.107Examiner TrainingAll examiners received two two—hour training workshopson the administration and scoring of the KTEA and theCurriculum—Based Measures. Background information onrationale for the study and literature pertaining toCurriculum—Based Measurement were also provided.It is important to note that all participation in datagathering by the district area counsellors was voluntaryand that demands upon their time prevented further trainingworkshops.It is the opinion of the author that more time intraining would have been beneficial to the accuracy of theobtained data. Although these examiners were experiencedand formally trained in the administration and scoring ofnorm—referenced achievement and intelligence tests, furthertraining time might have emphasized the importance ofcareful attention to detail when administering Curriculum—Based Measures. The author checked the accuracy of scoringof all Curriculum—Based Measures and found that 38 percentof the Word Lists and 2 percent of the Reading Passageswere scored incorrectly.These findings were surprising because good inter—rater reliability was obtained in the pilot study. Amedian correlation coefficient of .97 (p <001) was108obtained between the two pilot study raters for the WordList and the Reading Passages scores. It should be noted,however, that the two raters in the pilot worked togetherin developing the scoring criteria and that much discussionabout administration and scoring criteria occurred. Theexperience of the pilot study demonstrates that accuracy inthe administration and scoring of Curriculum—Based Measurescan be obtained, although generous training time may benecessary.Test AdministrationTesting of the 105 students was done during a six weekperiod from January 12th to February 23rd, 1990. Theindividual testing was done over a one to one and one—halfhour period and the group testing was often done in aseparate session. Examiners did not always complete theentire individual session in one sitting because of jobconsiderations and school activities. The order of testadministration was the same for all children (see AppendixE). Request for subject participation was obtained fromthe students (see Appendix F).The district Core Mastery Test was given by someteachers in November or December, 1989. If the test hadalready been administered, these scores were used.109Examiners reported, however, that most students in thestudy had not been given the test and found it necessary toadminister it in January, 1990.Teacher Rating Scales were completed by the classroomteacher even if the student received service from a SkillDevelopment Program ora Learning Resource Room. In thisway, all students in the study were rated by regular classteachers who may have a better conceptualization of normalvariability than special education teachers who work withspecial populations.Data PreparationIn 28 percent of the Word Lists, testing had stoppedat one minute whereas instructions clearly stated that theone—minute interval was to be noted and children were tocontinue reading (see Appendix K). Scores for theseincorrectly administered tests were extrapolated. Toestimate the amount of time that each of these studentswould have taken to read all 64 words, the following ratioprocedure was applied: 60/no, of words read in 60 seconds x64. The author corrected all recoverable errors andentered corrected scores into the computer. Three WordLists were omitted because the source of the error was notrecoverable.110Curriculum—Based Reading Passages were scored withbetter accuracy; 45 out of 2040 passage scores were foundto be incorrect resulting in an error rate of only 2percent. These were corrected and entered into thecomputer.The source of most errors were simple addition andsubtraction or counting errors. Examiners often circlederrors but failed to count them correctly and then failedto accurately subtract errors from total words. Ratersalso made errors in converting minutes and seconds toseconds only scores. A failure to follow administrationinstructions was apparent in the Word Lists but lessnoticeable in the Reading Passages.KTEA scores were checked in the following manner. Oneprotocol from each examiner was checked for administration,addition and subtraction, and conversion to table standardscore errors. If no error was found, the remainder of theexaminer’s tests were not checked. If any error was found,all test protocols from the examiner were checked andcorrected. Errors in the KTEA were found for 4 out of 22examiners. A total of 9 protocols were corrected. Onlycorrected scores for both the curriculum—based and norm—referenced tests were used in data analysis.111Chapter 4ResultsThis chapter is divided into five sections. First,information about the means and standard deviations of theCurriculum—Based Measures, norm—referenced measures,teacher rating scale, and district test is provided.Second, indices of reliability for the Curriculum—BasedMeasures are given. Third, correlations between themeasures used in the study are examined. Fourth, analysesof variance are presented which demonstrate the efficiencyof measures in discriminating between regular and learningdisabled samples. Finally, predictions of the dependentvariables from the independent variables are investigatedwith stepwise multiple regression and canonical analysis.Means and Standard Deviations of the Predictor and CriterionMeasuresMeans and standard deviations for all measures can befound in Table 2.TABLE2MeansandStandardDeviationsofthePredictorandCriterionMeasuresMeansStandardDeviationsTestLrng.Ohs.RegularCombinedLrng.Dis.RegularCombined(n=35)(n=70)(n=105)(n=35)(n70)(n105)1.KTEADSS82.83112.33102.505.1412.6317.602.KTEACSS84.26112.01102.766.0313.4317.453.RATPASAV85.3397.7693.6214.621.5710.304.SPDPASAV62.51132.00108.8316.8829.2741.775.WORDDSC28.5751.9044.1210.816.9713.886.WORDPM29.1449.6442.8110.329.8913.937.TRAT1A2.345.214.261.061.171.778.TRATlB2.235.134.161.001.021.709.TRAT1C2.805.234.421.231.161.6510.TRATAV2.455.194.28.981.071.6611.DISTEST21.2932.5428.797.923.517.56LEGENDT1.*KTEADSSKaufmanTestofEducationalAchievementReadingDecoding:StandardScore2.*KTEACSSKaufmanTestofEducationalAchievemntReadingComprehension:StandardScore3.*RATPASAVCurriculum—BasedReadingPassagesAveragePercentageCorrectScore4.*SPDPASAVCurriculum—BasedReadingPassagesAverageReadingSpeed(CorrectandIncorrectWordsPerMinute)Score.*WORDSCCurriculum—BasedWordListCorrectScore6.*WORDPMCurriculum—BasedWordListWordsPerMinuteScore7.*TRAT1ATeacherRatingScaleQuestion1(a):OverallReadingSkill8.*TRATlBTeacherRatingScaleQuestion1(b):OralReadingFluency9.*TRAT1CTeacherRatingScaleQuestiQn1(c):ReadingComprehension10.*TRATAVAverageofTeacherRatingScaleQuestion1(a):OverallReadingSkill;Question1(b):OralReadingFluency;andQuestion1(c):ReadingComprehension11.*DISTESTSchoolDistrictCoreMasteryTest:Grade4Reading*Thesecodifiedvariablenameswillbeusedinthebalanceofthischapter.*TeacherRatingScaleQuestion2hasbeenomittedbecauseofitsverycloserelationshiptoplacementstatus.113An examination of the two Kaufman Test of EducationalAchievement (Kaufman & Kaufman, 1985) standard scoresindicates that the mean scores for the entire study sample(102.50 and 102.76) are close to the mean KTEA normalizedstandard score of 100. This provides some evidence thatthe study sample has a similar central tendency to the KTEAforming sample. Standard deviation indices (17.60 and17.45) suggest that the entire study sample had slightlymore variability in reading scores than the KTEA formingsample (S.D.=15). Indices generally indicate that thestudy sample was similar to the KTEA forming sample.Mean KTEA standard scores for the study’s learningdisabled sample (82.83 and 84.26) are much lower than forthe regular sample (112.33 and 112.01) indicating theKTEA’S ability to discriminate between the two groups.Scores were also less variable for the learning disabledsample (S.D.=5.l4 and 6.03) than for the regular sample(S.D.=12.63 and 13.43).Curriculum—Based Reading Passages Average PercentageCorrect mean scores indicate that regular sample subjectswere able to read the passages with a high degree ofaccuracy (97 percent correct) and were close to thecriterion for independent level reading of 98 percentaccuracy (Betts, 1946; Hargis, 1987) while the learning114disabled sample generally read the passages at frustrationlevel (Betts, 1946; Hargis, 1987) with only 85 percent ofwords read correctly. sixty—six percent of the entiresample was able to read the passages with 97 percentaccuracy. An examination of the Reading Passages correctscores histogram indicates a strong ceiling effect for thepassages but it must be remembered that oral readingaccuracy scores are not normally distributed and thatscores below ninety—five percent accuracy indicate somereading frustration (Fuchs et al., 1982). When this isconsidered, it appears that the Reading Passages AveragePercentage Correct mean scores discriminate between thelearning disabled and the regular samples.Inspection of the sample standard deviations indicatethat the learning disabled sample was much more variable inreading accuracy (5.D.=14.62) than the regular sample(S.D.=1.57). As stated earlier, the regular sample tendedto read the passages with a uniformly high degree ofaccuracy.Curriculum—Based Reading Passages Average ReadingSpeed (Words Per Minute) mean scores indicate a largedifference between the regular sample (mean=132.OO) and thelearning disabled sample (mean=62.51). In this test theregular sample was more variable in reading speed115(S.D.=29.27) than the learning disabled group (S.D.=l6.88).This pattern of variability is opposite to that obtained inthe Percentage Correct reading passage scores. It shouldalso be noted that this measure has a large standarddeviation (41.77) for the combined groups, reflectingsubstantial variability in this test and a large differencebetween the groups.An examination of the Curriculum—Based Word ListCorrect and Word List Words Per Minute mean scores showsthat the learning disabled group performed much lower inboth wor list reading accuracy and speed. Standarddeviation differences between the regular and learningdisabled groups were not as large as in previous measuresdiscussed.Means and standard deviations for the reading sectionof the Teacher Rating Scale (questions la, ib, and ic)showed that the learning disabled group was rated muchlower in reading proficiency by their teachers but that thetwo groups demonstrated similar variability in scores.The district reading test yielded a lower mean scorefor the learning disabled group (21.28) than for theregular sample (32.54), and scores for the learningdisabled group (S.D.=7.92) were more variable than those ofthe regular sample (S.D.=3.51).116Reliability of the Curriculum-Based MeasuresThe following kinds of reliability were calculated forthe CBMs: internal consistency reliability of the wordlist, internal consistency reliability of the readingpassages, test-retest reliability of the reading passages,and inter-rater reliability of the reading passages. Twointernal consistency coefficients were used. They arealgebraically equivalent; either coefficient reflects thedegree of inter—item correlation within a single instrumentadministered on a single occasion (Crocker & Algina, 1986).Internal consistency reliability coefficients for theCurriculum-Based Word List can be found in Table 3.Hoyt estimates of internal consistency reliabilitywere calculated using the Lertap 2.0 (Nelson, 1974) programfor both the corrected or extrapolated versions and the 73word lists where administration was done correctly. Bothresults yielded very high estimates of reliability,indicating homogeneity within the providingsupport for the content sampling method used in theconstruction of this test.The Hoyt estimate is the only available reliabilitycoefficient for the Word List. Constraints on examinertime prevented the gathering of data to calculate testretest and inter—rater estimates.117Internal consistency reliability coefficients for theCurriculum—Based Reading Passages can be found in Table 4.Coefficient Alpha estimates of internalconsistency were calculated on the average reading passagespeed and the average reading passage accuracy using theSPSS-X Release 3.0 program (SPSS Inc., 1988). Bothcoefficients were high, indicating homogeneity within themeasure and providing support for the content samplingmethod used in the construction of this test.Pearson correlation coefficients for test—retestreliability of the Curriculum-Based Reading Passages can befound in Table 5.Ten examiners were available, each to re—testthree students on the CBM reading passages after a sevenweek interval. Because one student moved out of thedistrict, only 29 re—test passage scores were available.The median passage reading speed test—retest coefficient of.89 (p<.001) is somewhat higher than the passage readingaccuracy coefficient of .79 (p<.001), suggesting thatreading speed may be a more reliable measure than readingaccuracy.The lower reliability coefficient for reading accuracymay be due to difficulties with examiner scoring, althoughhigher coefficients were obtained in the more difficult to118TABLE 3Internal Consistency of the Word ListReading AccuracyHoyt EstimateExtrapolated (n=102) .97No Extrapolation (n=73) .96TABLE 4Internal Consistency of the Reading PassagesScale Coefficient AlphaReading Speed in Seconds .98Reading Accuracy (Words Correct) .94n= 105119TABLE 5Test-Retest Reliability of the Reading PassagesTest Reading Speed Reading AccuracyPassage 1 .87 .65Passage 2 .86 .66Passage 3 .93 *54Passage 4 .91 .62Passage 5 .93 .66Passage 6 .88 .80Passage 7 .89 .88Passage 8 .89 .79Passage 9 .91 .89Passage 10 .87 .82Median .89 .79*p<.OO1 for every entry except Passage 3 Accuracy wherep<.002n=29120score later passages indicating that there was morevariation between test and retest in the easy to scoreearlier passages. The lower coefficient may also be due toa change in true score in that examinees benefitted from theexperience of the first test and were able to demonstratethat most readily in the reading accuracy of the first fewpassages. It should also be noted that the obtained readingaccuracy coefficient of .79 indicates a significantrelationship between the test and retest occasions andsupports an assumption of reasonable stability of themeasure.Pearson correlation coefficients for inter—raterreliability of the curriculum—based reading passages can befound in Table 6.Data for inter-rater reliability coefficients wasobtained by the author independently scoring 30 of thestudy’s reading passage tests with the use of audiotaperecordings. Only passage scores originally obtained byexaminers other than the author were used. Coefficientswere computed between the 30 original scores for passagereading speed and accuracy and the author’s reading speedand accuracy scores from audiotape recordings of theoriginal testing sessions.121TABLE 6Inter-Rater Reliability of the Reading PassagesTime(speed) Score (accuracy)Passage 1 .99 .85Passage 2 1.00 .95Passage 3 .99 .96Passage 4 1.00 .96Passage 5 .98 .97Passage 6 .87 .97Passage 7 .98 .97Passage 8 1.00 .92Passage 9 1.00 .98Passage 10 .98 .92Median .99 .96*p<.OOl for every entryn=3 0122Inter-rater reliability coefficients for both passagereading speed and accuracy were high. The median readingspeed coefficient of .99 (p<.OO1) and the median readingaccuracy coefficient of .96 (p<.OO1) both meet reliabilitystandards for the making of important educational decisions(Salvia & Ysseldyke, 1991).Descriptive statistics and t—tests for inter—raterreliability can be found in Appendix B. Although theintercorrelations were high, significant differences in themeans were found between the two raters. These, however,usually constitute trivial differences when the actualmeans are examined. Apparently trivial differences weresignificant in part due to the high correlation; thestandard error of the difference between means becomesvanishingly small as the correlation between the twomeasures approaches unity, resulting in the identificationof very small mean differences as significant.Correlations Within the MeasuresThe following section explains the subsequent use ofan averaged score for the three teacher rating scalequestions, the ten passage reading speed scores, and theten passage accuracy scores.123Teacher Rating ScaleThe three reading ratings on the Teacher Rating Scalehave a median Pearson inter—correlation coefficient of .93(p<.Ol); this substantial relationship among the threeratings suggests that the use of separate ratings would bepointless; the average teacher rating for reading score(TRATAV) was therefore calculated and used as a dependentvariable.Curriculum-Based Reading Passages Average Reading Speed (Words PerMinute)The median Pearson correlation coefficient of .90(p<.001) between the ten passage reading speed scoresindicates strong relationships among the passage speedscores; therefore, the average passage reading speed score(SPDPASAV) was used to represent student performance onthe aggregate of ten passages.Curriculum-Based Reading Passages Average Percentage Correct ScoreThe median Pearson correlation coefficient of .77(p<.OOl) between the ten passage reading correct scores islower than the median passage speed score; nonetheless,124relationships among the passage reading scores are strong,and the use of the average passage reading correct score(RATPASAV) is used in following analyses for the purpose ofrepresenting aggregate student performance, in view of thehomogeneity of the passage intercorrelations and compositeinternal consistency reliability.Correlations Among the MeasuresPearson correlations among the eleven principalmeasures to this study were calculated for three samplesets: the total sample, the learning disabled sample, andthe regular sample.Coefficients for the total sample can be found inTable 7, which indicates that all of the measures correlatesignificantly with each other. Median correlations for theindividual variables range from .57 to .78. TheCurriculum—Based Reading Passages Average Reading Speed(Words Per Minute) Score correlates .74 and .75 (p<.OO1)with the two Kaufman reading achievement subtests and theCurriculum—Based Word List Correct Score correlates .74 and.80 (p<.OO1) with the Kaufman subtests, indicating thatsome of the Curriculum—Based Measures have strongrelationships to the norm—referenced measures. The highcorrelation between the Curriculum—Based Word List CorrectTABLE7Correlations*BetweentheMeasuresfortheTotalSampleSPDPASAVRATPASAVWORDSCWORDPMKTEADSS(TEACSSTRATAVDISTESTPROGRAMTRATQ2SPDPASAV100RATPASAV60100WORDSC7770100WORDPM855162100KTEADSS75578067100KTEACSS7455746586100TRATAV785476637874100DISTEST64617657656967100PROGRAM7957807079757871100TRATQ2715272637873756288100MEDIAN75577663787475657872*Pearsoncoefficientsareroundedtotwosignificantfigures,anddecimalsareomitted.Allsubsequentcorrelationtablesinthisreportfollowthesameformat.n=105;everycoefficientissignificantat=.001I..’r%JUi126Score and the Kaufman Test of Educational AchievementReading Decoding Standard Score (r=.80, p<.OO1) may be duein part to the similarity of item type: both are oralreading word lists.Pearson correlation coefficients for the learningdisabled sample can be found in Table 8 and for the regularsample in Table 9.Tables 8 and 9 indicate that when the learningdisabled and regular samples are considered separately,correlation coefficients between the measures used in thestudy become smaller. This would be expected, in somedegree, as a function of the greater within—grouphomogeneity resulting from this classification. However,the median correlation coefficient is not lower for thelearning disabled sample; indeed, the median for thelearning disabled sample (.42) is slightly higher than thatfor the regular education sample (.38). Within thelearning disabled sample, markedly lower variability wasnoted for three measures: Curriculum—Based Reading PassagesAverage Reading Speed, KTEA Reading Decoding, and KTEAReading Comprehension (see Table 2). Despite this apparentrestriction of range, these subtests do not exhibitsystematically lower correlation coefficients with othermeasures, when compared to the regular sample’s values.TABLE8CorrelationsBetweentheMeasuresfortheLearningDisabledSampleSPDPASAVRATPASAVWORDSCWORDPMKTEADSSKTEACSSTRATAVDISTESTSPDPASAV100RATPASAV65a100WORDSC100WORDPM52a24—01100KTEADSS63a56a22100KTEACSS47b55a47b2763a100TRATAV45b29d36c101819100DISTEST2761a30d54&23100MEDIAN4455442242494730n=35;a=significantat=.001b=significantat=.005c=significantat=.01d=significantat=.05TABLE9CorrelationsBetweentheMeasuresfortheRegularSampleSPDPASAVRATPASAVWORDSCWORDPMKTEADSSKTEACSSTRATAVDISTESTSPDPASAV100RATPASAV44a100WORDSC41a57a100WORDPM75a38a100KTEADSS31b54a56a31b100KTEACSS34b36a37a29c65a100TRATAV43a27d40a49a100DISTEST21d1331b28c38a37a100MEDIAN4338403149374031n=70;a=significantat=.001b=significantat=.005c=significantat=.01d=signicicantat=.05129To determine if the pattern of correlations for thelearning disabled sample is significantly different fromthat of the regular sample, PROGRAM MULTICORR- VERSION 2.4(Steiger, 1987) was used. This program allows small-sampletesting of correlational pattern hypotheses and computes achi—square statistic for testing pattern hypotheses oncorrelation matrices. The statistic is based on amultivariate generalization of the Fisher r—to—ztransformation. Monte Carlo studies have demonstrated thatthis statistic performs well with small samples (Steiger,1979; Steiger & Browne, 1984).Using this program, a chi—square of 98.12(p<.000l;df=28) was obtained, indicating that the patternof correlations in the learning disabled sample issignificantly different from the pattern of correlations inthe regular sample, which was presumed to represent thepopulation value in this analysis. Thus the form ofrelationships, as well as the mean level of scores in thevariable measured differs between the two groups in thisstudy.The results of the MULTICORR program raise manyquestions about the nature of performance among learningdisabled students. They support the hypothesis thatlearning disabled students differ qualitatively from othersin their performance (Swanson et al., 1990) and that muchmore research with this population is necessary.130Analyses of VarianceAnalyses of variance were calculated on all measuresused in the study to determine if significant differencesexist between means of students in learning disabled andregular programs.Suimnaries of these analyses of variance can be foundin Table 10, which shows that all independent and dependentmeasures used in the study discriminate between thelearning disabled and the regular groups. All F ratios arehighly significant, providing some evidence for thevalidity (sensitivity) of the dependent measures listedabove (TRAT1A, TRAT1B, TRAT1C, DISTEST). As well, the Fratios indicate that all six ability measures (KTEADSS,KTEACSS, RATPASAV, SPDPASAV, WORDSC, and WORDPM) can beused to identify regular or learning disabled programstatus.PredictionsTo determine the effect of the six predictor measureson the four criterion measures several analyses wereperformed. These were: a stepwise multiple regression forthe total sample, a stepwise multiple regression for the131TABLE 10Summary of Analyses of Variance: Differences BetweenLearning Disabled and Regular Program MeansTESTKTEADSSKTEACSSRATPASAVSPDPASAVWORDSCWORDPMTRAT 1ATRAT lBTRAT 1CDISTESTF RATIO*175.60135.3849.83168. 69178 .5697.41150.49190.6598.47102 . 11F PROB.<.001<.001<.001<.001<.001<.001<.001<.001<.001<.001* all F—tests were conducted with df=1;l03132learning disabled sample, a stepwise multiple regressionfor the regular sample, an analysis of the beta weights,and a canonical analysis between the set of independent andthe set of dependent variables.Stepwise multiple regressions were run using SPSS—XRelease 3.0 (SPSS Inc., 1988) with each of the fourcriterion variables used singly in each analysis. Theresults are summarized in Table 11. For each dependentvariable, the six steps of the stepwise analysis issummarized in Table 11 by giving the change in thecumulative R2 when the indicated predictor variable isentered into the analysis. Predictors are listed (Steps 1through 6) in their order of entry into the eventualstepwise equations.The six independent variables perform in similarfashion against all dependent variables except for theSchool District Core Mastery Test: Grade 4 Reading. Thistest also performs somewhat differently than the othercriterion variables in a correlation matrix involving allmeasures used in the study. The district test had a medianPearson correlation coefficient of .65 with the othermeasures while the other three dependent variables hadmedian correlations ranging from .72 to .75 (see Table 7).Further to this, the district test had the lowest squaredmultiple correlation with all other dependent variables,TABLE11PredictingSchool—BasedCriteriafromCurriculum—BasedandNorm—ReferencedMeasures:StepwiseMultipleRegressionfortheTotalSampleDependentVariablesTRATQ2VARIABLERSQCHKTEADSS.61SPDPASAV.04WORDSC.01KTEACSS.01WORDPM.00RATPASAV.00RSquare.66MultipleR.81PROGRAMVARIABLERSQCHWORDSC.63SPDPASAV.08KTEADSS.03KTEACSS.00WORDPM.00RATPASAV.00RSquare.75MultipleR.86DISTESTVARIABLERSQCHWORDSC.58KTEACSS.04RATPASAV.01KTEADSS.00WORDPM.00SPDPASAV.00RSquare.63MultipleR.80n=105df=6;98STEP 1 2 3 4 5 6TRATAVVARIABLEKTEADSSSPDPASAVWORDSCWORDPMKTEACSSRATPASAVRSquareMultipleRRSOCH.61 08.01.01 00 00.72.85134indicating a weaker relationship to the set of dependentvariables than the other dependent variables demonstratedwhen taken singly (see Appendix, Table A.2). The differentperformance of the six independent variables against thedistrict test may therefore be due to its lowerrelationship with other measures in the study and mayreflect the comparative reliability and validity of thattest, rather than information about the performance of thesix predicted variables. Although the district test wasdeveloped by a committee of teachers and was designed toreflect school district evaluation of reading achievement,no information about its reliability is available.In the other stepwise analyses the Kaufman Test ofEducational Achievement Reading Decoding standard score(KTEADSS), the Curriculum-Based Reading Passages AverageReading Speed (Words Per Minute) Score (SPDPASAV), and theCurriculum-Based Word List Correct Score (WORDSC) wereentered into the equation in the first three steps.Stepwise regression analyses were also performed forthe two groups separately; summaries of analyses of the twodependent variables for the learning disabled sample can befound in Table 12. (This should be interpreted with somecaution due to the small size of the learning disabledsample). A stepwise multiple regression with two dependentvariables for the regular sample can be found in Table 13.135TABLE 12Predicting School-Based Criteria from Curriculum-Basedand Norm—Referenced Measures:Stepwise Multiple Regression for the Learning DisabledSampleDependent RSQCH Dependent RSQCHVariable VariableTeacher Rating District TestScaleSTEP1 SPDPASAV .200 WORDSC .3702 WORDSC .033 KTEACSS .0853 KTEADSS .030 KTEADSS .0164 WORDPM .015 SPDPASAV .0025 RATPASAV .003 WORDPM .0006 KTEACSS .001 RATPASAV .000R Square .28 R Square .47Multiple R .53 Multiple R .69n=35df=6, 28136TABLE 13Predicting School-Based Criteria from Curriculum-Basedand Norm—Referenced Measures:Stepwise Multiple Regression for the Regular SampleDependent RSOCH Dependent RSQCHVariable VariableTeacher Rating District TestScaleSTEP1 KTEADSS .241 KTEACSS .1462 SPDPASAV .085 WORDPM .0423 WORDPM .038 SPDPASAV .0074 RATPASAV .007 WORDSC .0085 WORDSC .006 RATPASAV .0126 KTEACSS .006 KTEADSS .000R Square .38 R Square .21Multiple R .62 Multiple R .46n= 70df=6;63137When the learning disabled and regular samples wereanalyzed separately, some similar patterns emerged. In astepwise multiple regression with the teacher rating as thedependent variable (TRATAV), The Curriculum-Based ReadingPassages Average Reading Speed Score (SPDPASAV) and theKaufman Test of Educational Achievement Reading DecodingStandard Score (KTEADSS) were entered into the equation inthe first three steps when both samples were considered.The order of entry into the equation of theindependent variables changed with the district test(DISTEST) as a dependent variable in similar fashion to thestepwise analyses on the total sample. The learningdisabled sample resembled the total sample in that theCurriculum-Based Word List Correct Score (WORDSC) and theKaufman Test of Educational Achievement ReadingComprehension Standard Score (KTEACSS) were entered intothe equation in the first two steps. Patterns of entrydiffered, however, when the regular and the learningdisabled samples were compared.Although there are no clear conclusions about singlevariables from the stepwise analyses, the CBM5 performed aswell as the norm—referenced subtests in the prediction ofthe school—based indices. When the total sample isexamined, the Curriculum—Based Reading Passages AverageReading Speed (Words Per Minute) Score (SPDPASAV), the KTEA138Decoding Standard SCore, and the Curriculum-Based Word ListCorrect Score (WORDSC) were entered in the first threesteps for all dependent measures except the District Test.The passages speed score and the KTEA Decoding score werealso entered first for the learning disabled sample and theregular sample with Teacher Rating as a single dependentvariable. This pattern changed for both samples when theDistrict Test was considered. When the District Test isconsidered along with the other dependent variables, theonly conclusion that can be made is that the CBMs were aspredictive of the school—based indices as the norm—referenced tests. If the somewhat confusing results of theDistrict Test analyses are excluded, the KTEA Decoding,passage speed, and word list correct scores emerge at thetops of lists of predictor variables.Because the word list correct score is similar informat to the KTEA Reading Decoding subtest, there may beno practical value in administering both measures. The CBMword list also requires more time for administration andscoring than does the Kaufman Decoding subtest.In summary, no simple conclusions can be drawn fromthe stepwise analyses, except that the CBMs and the normreferenced tests showed similar predictive characteristics.When teacher rating and program status are chosen asindependent variables, three measures are at the tops of139lists of predictor variables: the KTEA Decoding subtest,the CBM passage reading speed test, and the CBM word listcorrect test.The stepwise regression technique, results of whichhave been summarized in Tables 11—13, was chosen as theanalytical method most responsive to research questionsposed in Chapter 1. However, the multicolinearity of thevariables -- especially the KTEA and CBM variables --raised the issue of whether that analysis might serve tomask the contribution of any variable which happened not tobe entered as the first predictor in a regression equation.For example, in the Total Sample equation for predictingthe ‘Program’ dependent variable (see Table 11), SPDPASAVadded only .08 to the R2 as the second variable to enterthe equation, though its zero-order correlation with thedependent variable was essentially the same as that forWORDSC, which entered the equation first by virtue of itsslightly greater sensitivity. SPDPASAV contributes arelatively modest increment to the R2, at itspoint-of-entry into the equation, because it is stronglycorrelated with WORDSC (r=77); variance shared among thesetwo predictors and the dependent variable was “accountedfor” when WORDSC entered the equation, causing an initialchange of R2=.63.140To assess the possibility that the relative order ofpredictive power among the independent variables might bean artifact of multicolinearity, “simultaneous” regressionanalyses were also applied to the data. The statistic ofinterest in this analysis is the standardized betacoefficient, which reflects the unique contribution of apredictor in an equation. Beta weights can be compared andused to rank the independent variables in order of theirpredictive value, and each can be independently tested forsignificance. The R2 for the full model (with all sixpredictors) is equal to that obtained in the stepwiseanalysis.-Table 14 provides a sunimary of the total sample betaweights yielded by the individual regression equations.When predictors are ordered by their beta weights for thetotal sample, the conclusion is similar to that reachedwith reference to the stepwise analysis. When teacherrating and program status are considered, the largest betaweights are associated with three measures: the CBM passagereading speed test, the KTEA Decoding subtest, and the CBMword list correct test. Similar to the stepwise analysis,the highest beta weights for the district test wereassociated with the CBM word list correct test and the KTEAComprehension subtest. As with the total sample stepwise141TABLE 14Multiple Regression Beta Weights for the Total SampleTeacher Rating Scale AverageVariable Beta SE Beta Prob.WORDSC .21 .11 .06WORDPM—.14 .10 .17RATPASAV-.03 .08 .70KTEACSS .12 .11 .28KTEADSS .28 .12 O2SPDPASAV .46 .13 <.01District TestVariable Beta SE Beta Prob.WORDSC .54 .13 <.01WORDPM .14 .12 .23RATPASAV .12 .09 .16KTEACSS .34 .12 .01KTEADSS—.17 .14 .22SPDPASAV—.10 .14 .51ProgramVariable Beta SE Beta Prob.WORDSC .33 .10 <.01WORDPM .10 .10 .30RATPASAV-.03 .07 .70KTEACSS .11 .10 .27KTEADSS .22 .12 .06SPDPASAV .22 .12 .07142analysis, when the beta weights are examined the CBM5demonstrated comparable predictability to the norm-referenced measures.Simultaneous regression analyses were also applied tothe two groups separately. Table 15 provides informationabout the learning disabled sample and regular educationsample beta weights in the individual regression equations.Again, results are similar to the stepwise analyses forthese samples, especially for the learning disabled sample.For both samples, when compared to teacher rating, the CBMpassage speed score has the highest beta weights, followedby the KTEA Decoding subtest and the CBM word list correcttest. As with the stepwise analyses, this pattern changeswith the district test.An analysis of the beta weights for all three samplesleads to the same conclusions as the stepwise analyses.These are: the CBM5 were as predictive of the dependentvariables as the norm—referenced tests, and when teacherrating and program status are chosen as independentvariables, three measures are at the tops of lists ofpredictor variables. These are the KTEA Decoding subtest,the CBM passage reading speed test, and the CBM word listcorrect test. Results change when the district test is143TABLE 15Multiple Regression Beta Weights for the Learning Disabledand Regular Education SamplesLearning Disabled SampleTeacher Rating Scale Average District TestRegular Education SampleVariableWORDS CWORDPMKTEADSSKTEACSSRATPASAVSPDPASAVDistrict TestBeta SE Beta.17 .15.33 .17.34 .15—.14 .15—.02 .17—.15 .18Prob..26• 06.03.36.90.40Variable Beta SE Beta Prob. Variable Beta SE Beta Prob.WORDSC .22 .22 .31 WORDSC .49 .18 .01WORDPM -.16 .20 .44 WORDPM -.02 .17 .93KTEADSS —.25 .24 .31 KTEADSS —.14 .21 .50KTEACSS .04 .23 .86 KTEACSS .43 .19 .04RATPASAV -.09 .25 .72 RATPASAV -.00 .21 .98SPDPASAV .63 .27 .03 SPDPASAV —.04 .23 .86Teacher Rating Scale AveracteVariable Beta SE Beta Prob.WORDSC .11 .13 .43WORDPM -.27 .15 .08KTEADSS .11 .13 .43KTEACSS —.13 .13 .31RATPASAV -.36 .15 .02SPDPASAV .50 .16 .00144chosen as a single dependent variable. Limitations of thismeasure’s (unknown) reliability and validity may accountfor these different results.Although the findings of the regression analyses areinteresting, the ultimate purpose of this study was toconsider the dependent variables as a set rather thansimply to inspect them singly. For this reason, canonicalanalysis was used. A canonical analysis of therelationship between the entire set of independentvariables (the four CBM measures and the two KTEA subtests)and the set of dependent variables (two questions from theteacher’s rating scale, program status, and the districttest) can be found in Table 16. Learning disabled andregular groups were combined for this analysis.Results of the MULTICORR analysis, indicating inter-correlation pattern differences between the two groups,suggest that separate canonical analyses would be aninformative adjunct to the combined groups’ analysis.However, this analysis was not possible because of thesmall learning disabled sample size (n=35), and becausesuch analysis would necessitate the elimination of onedependent variable, namely, program status. Data for thisstudy was gathered in the third largest school district inBritish Columbia and the size of the learning disabledsample was determined by the availability of identifiedTABLE16CanonicalAnalysis:KTEASubtestsandCurriculum-BasedMeasureswithDependentVariablesEigenvalueCanonicalNumberofBartlett’sTestRedundancyRedundancySquaredCorrelationEigenvaluesforRemainingIndexIndexCanonicalEigenvalues(1stSet)(2ndSet)CorrelationChi-D.F.Prob.Square.85.921205.2224.00.58.67.85.12.35220.5215.15.01.01.12.06.2437.688.47.00.00.06.02.141.883.60.00.00.02*OnlythefirstcanonicalvariateissignificantUi146learning disabled students at one grade (Year Four).Forty—seven possible Year Four students were identified butby the time principal, area counsellor, and parent consentwas obtained, only 35 learning disabled students wereavailable for the study. In studies designed to behomogeneous with respect to grade level, it is often notfeasible, or economically practical to assess a largeenough learning disabled sample to allow the use of somemultivariate analyses. Results of the present canonicalanalysis are to be considered in light of the likelihoodthat diffferent results may well be obtained if thelearning disabled sample could be analyzed separately. TheBMDP6M (Dixon, 1985) program was used for canonicalanalyses.Results indicate that the first canonical variate issignificant (p<.OOl) and that the set of independentvariables has a strong relationship to the set of dependentvariables (Canonical correlation = .92). Results indicatethat the canonical variate for the independent variables(first set) and the canonical variate for the dependentvariables (second set) share 85 percent common variance;these two sets of variables demonstrate a very strongrelationship.Squared multiple correlations of each of theindependent variables with the set of dependent variables,147along with canonical variable loadings, can be found inTable 17, which indicates that all the independentvariables have significant loadings on the first linearcombination of all dependent variables. Results alsosuggest that, for predictive purposes, the two norm—referenced reading tests (KTEADSS and KTEACSS) can be usedinterchangeably with the reading speed score from thecurriculum-based passages (SPDPASAV). These results, alongwith considerations of CBM test development requirements,test administration time and instructional utility, supportthe use of the Kaufman Reading Decoding subtest and thereading speed score from the curriculum—based passages toidentify most efficiently, and to plan for, children withsignificant reading difficulty. The Kaufman ReadingDecoding subtest is quickly administered and can providenormative information not now available with administrationof typical curriculum—based passages. Similar to informalreading inventories, the curriculum—based reading passagesyield a wealth of informal information about how the childreads that can be used as an initial step in programplanning (Oliver & Arnold, 1978).Squared multiple correlations of each of the dependentvariables with all of the independent variables can befound in Table 18, which indicates that all the dependent148TABLE 17Squared Multiple Correlations and Canonical VariableLoadings for All Independent VariablesVariableKTEADSSSPDPASAVKTEACSSWORD S CWORDPMRATPASAVCanonicalVariableLoading.91.90.88.83.76• 67R-Sguared.71• 69.66.59.49.40F Statistic*62.7156.3249.3536.5024.2216.87P-Value<.001<.001<.001<.001<.001<.001*all F—tests were conducted with df=4;lOO149TABLE 18Squared Multiple Correlations and Canonical VariableLoadings for Dependent VariablesVariable Canonical R-Sguared F Statistic* P—ValueVariableLoadingPROGRAM .94 .75 48.42 <.001TRATAV .93 .74 45.37 <.001TRATQ2 .87 .66 31.53 <.001DISTEST .82 .61 25.43 <.001* all F-tests were conducted with df=6;98150variables have significant loadings on the first linearcombination of all independent variables. All canonicalvariable loadings are high suggesting strong relationshipsbetween all dependent variables and the set of independentvariables.To determine whether any advantage in prediction isgained by combining the norm—referenced and the curriculum—based measures into one set of independent variables, ascompared to the level of prediction by either the norm-referenced or the curriculum—based measures usedindividually, separate canonical analyses were run for eachof the norm—referenced and the curriculum—based measures.This analysis also was used to test whether standardizedtests and CBM’s are differentially predictive of school-based measures; two additional canonical analyses wereeffected: one related the KTEA measures to the school—baseddependent variables, and another did the same for the CBM’s.Results of the canonical analysis relating the two KTEAsubtests (Reading Decoding and Reading Comprehension) to theset of dependent variables can be found in Table 19. Theyindicate that the first canonical variate is significant(p<.OO1) and that the norm—referenced subtests have a strongrelationship to the school-based indicators of readingskill. The canonical correlation of .86 with the KTEAsubtests used as the set of independent variables, isTABLE19CanonicalAnalysis:KTEASubtestswithDependentVariablesEicienvalueCanonicalNumberofBartlett’sTestRedundancyRedundancySquaredCorrelationEigenvaluesforRemainingIndexIndexCanonicalEigenvalues(1stSet)(2ndSet)CorrelationChi-D.F.prob.Square.74.861140.128.00.69.59.74.04.193.783.29.00.00.04*OnlythefirstcanonicalvariateissignificantUi152only slightly lower than the canonical correlation of .92obtained when both norm—referenced and curriculum—basedmeasures are used as the set of independent variables.When the KTEA subtests are used on their own in a canonicalanalysis, shared variance between the two canonicalvariates is 74 percent, as compared to shared variance of85 percent when the entire set of independent variables(both curriculum—based and norm—referenced) is used in acanonical analysis. This difference is not meaningful.Squared multiple correlations of each of the KTEAmeasures with all of the dependent variables, along withcanonical variable loadings, can be found in Table 20,which indicates that both KTEA measures have significantloadings on the first linear combination of all dependentvariables. A canonical correlation analysis between theset of four curriculum—based independent variables and theset of four dependent variables was also performed.Results given in Table 21, indicate that the firstcanonical variate is significant (p<.OOl) and that the CBM5have a strong relationship to the school-based indicatorsof reading skill (dependent variables). The canonicalcorrelation of .89 with the CBMs used as the set ofindependent variables,, is also similar to the canonicalcorrelation of .92 when both norm—referenced andTABLE20SquaredMultipleCorrelationsandCanonicalVariableLoadings:KTEASubtestswithDependentVariablesVariableCanonicalR-SguaredFStatistic*P-ValueVariableLoadingKTEADSS98.7162.71<.001KTEACSS.94.6649.35<.001*Al1F—testswereconductedwithdf=4;100VI wflnrni‘—.1TABLE21Analysis:Curriculum—BasedMeasureswithDeoendentVariablesRedundancyIndex(1stSet)RedundancyIndex(2ndSet)SquaredCanonicalCorrelationEigenvalueCanonicalNumberofBartlett’sTestCorrelationEigenvaluesforRemainingEigenvaluesChi—D.F.rob.Square.79.891165.4216.00.54.62.79.08.29210.719.30.01.01.08.02.1432.104.72.00.00.02.00.030.121.73.00.00.00*OnlythefirstcanonicalvariateissignificantU’155curriculum—based measures are used as the set ofindependent variables. The CBM canonical correlation of.89 may be slightly higher than the norm-referencedcanonical correlation of .86 because there is a larger setof CBM variables; the canonical R of .92, obtained when thenorm—referenced and the curriculum—based predictors arecombined, might also be explained by the even larger numberof variables in the equation. When the CBMs are used ontheir own in a canonical analysis, shared variance betweenthe two canonical variates is 79 percent, as compared toshared variance of 85 percent when the entire set ofindependent variables (both curriculum—based and norm—referenced) is used in a canonical analysis. This is not ameaningful difference.To test the significance of the difference between theKTEA and CBM canonical correlations (R=.89 for the CBMswith the dependent variables; R=.86 for the KTEA subtestswith the dependent variables)9 it was necessary todetermine the canonical correlation between the set of KTEAvariables and the set of CBM variables. Using the BMDP6Mprogram (Dixon, 1985) a canonical correlation of R=.81(p<.OO1) was obtained. This, along with the two previouslyobtained canonical coefficients, was used to test thesignificance of the difference between two correlated156correlation coefficients, procedures described in Ferguson(1971). The value of t was 1.03 (df was set conservativelyat 25; p=.32), indicating that there is no significantdifference between the KTEA and CBM canonical correlations.Squared multiple correlations of each of the CBMs withall of the dependent variables, along with canonicalvariable loadings, are given in Table 22, which indicatesthat the reading passages speed score (SPDPASAV) has thehighest loading on the first linear combination of alldependent variables. This is consistent with otheranalyses which suggest that the reading passages speedscore is one of the best CBM predictors.Separate canonical analyses for the KTEA subtests andthe CBM5 indicate that both groups of measures are verystrongly associated with the school-based dependentmeasures, and that no practically useful increase inprediction occurs when the two groups of measures arecombined compared to either group taken singly.Summary of ResultsReliability coefficients for the CBMs were generallyhigh and for the most part met Salvia and Ysseldyke’s(1985) standards for important educational decisionsTABLE22SquaredMultipleCorrelationsandCanonicalVariableLoadings:Curriculum-BasedMeasureswithDependentVariablesVariableCanonicalR-SguaredFStatistic*P—ValueVariableLoadingRATPASAV.70.4016.87<.001SPDPASAV.93.6956.32<.001WORDPM.78.4924.22<.001WORDSC.86.5936.50<.001*AllF-testswereconductedwithdf=4;l00U’158(reliabilities of .90 or higher). Test—retest coefficientsfor the CBM Reading Passages fell below the .90 standard(.89 and .79 for reading speed and reading accuracy), butinternal consistency and inter—rater reliabilitycoefficients exceeded .90. The lower test—retestcoefficients may have been related to a change in truescores over the seven week test—retest interval. Ingeneral, the CBM’s used in this study were found to possesshigh reliabilities and data indicates that these CBM’scould be used in making tracking and placement decisions.The analyses summarized in this chapter also indicatethat some of the Curriculum—Based Measures are highlycorrelated with the norm—referenced measures, providingsome evidence for the validity of some of the CBMs used inthis study. When the total sample was considered, the twoCBMs with the strongest relationship to the KTEA subtestswere the Curriculum—Based Reading Passages Average ReadingSpeed (Words Per Minute) Score (correlations of .74 and .75with the two KTEA subtests) and the Curriculum-Based WordList Correct Score (correlations of .74 and .80 with thetwo KTEA subtests). These two CBMs also had the highestbeta weights when program status and teacher rating wereconsidered singly as dependent variables. The passagespeed score also had a substantial loading on the firstlinear combination of dependent variables (Table 16). When159the learning disabled sample was analyzed separately, allrelationships between the CBMs and the norm—referencedtests were not significant. The passage speed score,however, demonstrated a significant relationship with theKTEA Decoding subtest (r=.63; p<.OOl). For purposes ofpredicting school—based reading indicators, there is noadvantage in administering the Curriculum-Based Word ListCorrect test along with the Kaufman Reading Decodingsubtest: both have similar items (word lists).Some caution in interpreting the performance of thelearning disabled sample from the analyses involving theentire sample is indicated when the patterns ofcorrelations between the regular and the learning disabledsamples are compared. Because these patterns aresignificantly different, it is possible that if thelearning disabled sample was analyzed separately, using acanonical analysis, different conclusions might be reached.Stepwise multiple regression results for the learningdisabled and the regular sample, however, yielded similarresults when the teacher rating scale was used as thedependent variable. When the beta weights for TeacherRating were examined for the separate samples, the highestbeta—weight was associated with the passage speed measure.This was not the case when the district test was used asthe dependent variable, but it should be remembered that160the district test performed differently from the otherdependent variables, in stepwise multiple regressionsusing the entire sample (Table 10). The differentperformance of the district test leaves some questionsunanswered. This cannot be resolved at present becausereliability and validity data are not available for thismeasure.The issue of whether or not the learning disabledsample would perform differently in a canonical analysisremains unresolved. Some stepwise multiple regressionresults suggest that the learning disabled and the regularsample perform similarly, while a comparison of the patternof correlations between the two samples indicates that thetwo samples are very different. It is not possible at thistime to resolve this apparently contradictory information,but it is prudent to consider the possibility that all theresults obtained in this study may not apply to theseparate sample of learning disabled students.An examination of the analyses of variancedemonstrates that all measures used in the study can beused to discriminate between the learning disabled and theregular groups. Results indicate that both the curriculumbased and the norm—referenced measures can be used toindentify regular or learning disabled program status.161Regression analyses indicate that the set ofindependent variables have strong relationships to the setof dependent variables. In practical terms, both thecurriculum—based and norm—referenced measures demonstrategood ability to predict the school—based indices of readingachievement and there is no substantial increase inprecision of prediction when both groups of measures arecombined compared to either group taken singly. Evidencefor the validity of both kinds of assessment was found inthis study. When teacher rating and program status areconsidered, the Kaufman Test of Educational AchievementReading Decoding subtest and the Curriculum—Based ReadingPassages Average Reading Speed score are at the tops oflists of predictor variables. This is also true when theset of dependent variables is considered (Table 16).These results may or may not be applicable to learningdisabled students when considered separately.162Chapter 5Summary, Conclusions, and ImplicationsSummaryThis final chapter includes a summary of the purposeand the results of the study, as well as conclusions andimplications drawn from the study. Recommendations forpractice and for future research are discussed andlimitations of the study are developed.PurposeThe purpose of this study was to examine theproperties of single—occasion Curriculum—Based Measures inreading, and to relate these measures to norm—referencedreading tests and to school—based indicators of readingability. Also, information was sought about thereliability and validity of CBM5. To accomplish this, tworeading Curriculum—Based Measures were administered, alongwith two reading subtests from the Kaufman Test ofEducational Achievement (Kaufman & Kaufman, 1985). Bothtypes of measures were related to three school—basedvariables: a teacher rating scale of reading skill,163student placement status, and a school district test ofreading skill. Because no one of these measures wouldsuffice as a criterion measure in a “prediction” study, thethree variables were composited, resulting in a more stableand valid criterion than any single indicator.ResultsThe analyses conducted in this study indicate that theCurriculum—Based Measures possess high reliabilities andthat some of the CBMs are highly correlated with the norm-referenced measures. Correlations were highest when thetotal sample was analyzed due in part to the greatervariability obtained when subsamples are combined. Theseresults are similar to previous research pertaining to therelationship between CBM5 and norm—referenced reading tests(Deno, Mirkin, & Chiang, 1982; Deno, Mirkin, Chiang, &Lowry, 1980; Fuchs & Deno, l981a, Fuchs, Fuchs, & Deno,1982; Marston & Magnusson, 1985). A significant differencebetween the pattern of correlations among the learningdisabled and the regular education sample was found,suggesting that the learning disabled sample differs inpattern of reading relationships as well as in level ofreading ability from the regular sample. When the learningdisabled and regular education samples were analyzed164separately, less substantial relationships between CBM5 andnorm—referenced tests exist. For the learning disabledsample, the highest correlation between a CBM and a norm—referenced measure (r=.63; p<.OO1) was found between theCBM passage speed measure and KTEA Reading Decodingsubtest.Analyses of variance indicate that all measures usedin the study, including all norm—referenced and thecurriculum—based measures, can be used to discriminatebetween the learning disabled and the regular sample.Regression analyses indicate that the sets ofindependent variables have strong relationships to theschool based dependent variables. Both the curriculum—based and the norm—referenced measures are predictive ofthe school—based indices of reading achievement. Someanalyses place the Kaufman Test of Educational AchievementReading Decoding subtest and the Curriculum-Based ReadingPassages Average Reading Speed score at the top of the listof predictor variables. This pattern holds for bothstepwise regression (with analyses of beta weights) whichpredict regression analyses and simultaneous teacher ratingand program placement status, but it does not hold for thedistrict test. The aforementioned pattern is alsoevidenced when the canonical variable loadings are examinedin the canonical analyses.165Reading speed and passage reading may be more validthan other CBM5. Deno (1986) states that timed tasks areless prone to ceiling effects and more likely to allownormal distributions to be established, and the use ofrate—based rather than accuracy measures has been advocated(Deno, Mirkin, Chiang, & Lowry, 1980; Deno, Mirkin, Lowry,& Kuehnle, 1980; Deno, Mirkin, & Marston, 1980; Potter &Wamre, 1990). Further, Marston (1989) found that whenbasal mastery tests were used as a criterion, reading frompassages was more highly related to test performance thanwas reading from word lists. Rate or reading speed may bemore valid than accuracy, arid passage reading may be morevalid than word list reading. Passage reading also appearsto possess more clinical utility since it is easier todiagnose difficulty and plan remediation strategies afterlistening to students read from passages rather than fromword lists (Oliver & Arnold, 1978). Passage reading ratesamples may also be more like the subjects’ everydayreading experience (Potter & Wamre, 1990).The single occasion Curriculum—Based Measures examinedin this research perform well when conventionalpsychometric criteria are applied. Somewhat higherreliability coefficients were found for the speed than forthe accuracy score in the Curriculum—Based ReadingPassages. This study has demonstrated that the CBMs in166general were as predictive of the school—based indices ofreading as the norm—referenced measures, both whendependent variables were considered singly or as a set.Although different results were seen when the district testwas considered singly as a dependent variable, in all otherregression analyses one norm—referenced test and two CBM5consistently provided the most powerful prediction. Thesewere: the KTEA Reading Decoding subtest, the CBM ReadingPassages Average Reading Speed test, and the CBM Word ListAccuracy Score. The Word List Accuracy Score alsoaccounted for the most variance in the district test forthe learning disabled sample.The finding that reading word lists (both the KTEA andthe CBM word list) are predictive of the school-basedindices of reading reflects other research. Deno, Mirkin,& Chiang (1982) found correlations between their wordrecognition test and standardized reading measures rangingfrom .73 to .91. Fuchs, Tindal, and Deno (1984) foundconcurrent validity between a word recognition measure andinformal comprehension tests. Graded word lists have alsodemonstrated high correlations with functional gradeplacements derived from more comprehensive reading measures(Froese, 1971, 1975, 1976).The value of the KTEA Reading Decoding subtest overthe CBM word list (both are similar in item type and both167are generally predictive of the school—based indices) isthat the KTEA Decoding subtest is somewhat quicker toadminister and offers normative comparison. It also isaccompanied with more research on its technical adequacy.The CBM reading passages speed measure demonstrated apattern of predictive utility in all but one analysis wherethe district test was considered as a single dependentvariable. The high internal consistency and stabilitycoefficients yielded by this measure along with thisinformation provide some evidence for its reliability andvalidity. Further to this, the reading passages resemblean informal reading inventory, which yields more usefulinformation in terms of reading instruction (Oliver &Arnold, 1978). As well, a passage reading rate measure maybe more like subjects’ everyday reading experiences (Potter& Wamre, 1990).The above provides some justification for consideringthe KTEA Reading Decoding subtest and the CBM passage speedmeasure over the other predictor measures used in thestudy. For clinical use, however, it is recommended thatthe passages be scored for both speed and accuracy toresemble procedures associated with informal readinginventories. Clinical use might also be served better withthe use of multi—occasion CBM passage data gatheringadvocated by Deno, Mirkin, & Chiang (1982).168Response to Research Questions1. Do the Curriculum—Based Measures (orally read word listand reading passages) demonstrate sufficient reliabilityto meet acceptable standards for screening and placementdecisions?Salvia and Ysseldyke (1985) state that if a test scoreis used to make decisions about individual students, a highstandard of reliability is demanded. They specify thatwhen important educational decisions are being made, suchas tracking and placement in a special class, the minimumreliability standard should be .90; as well, when thedecision being made is a screening decision, such as arecommendation that a child receive further assessment, aminimum level of .80 is required (p.127).The Curriculum—Based Measures in this study consistentlydemonstrated high reliabilities, ranging from .79 to .99.The lowest coefficients were obtained in the test—retestanalysis and the possibility that some change in truescore could have occurred in the seven—week interval shouldbe considered. Inter—rater and internal consistencycoefficients were in the .90 range, indicating that thisstudy’s reading Curriculum—Based Measures meet conventional169reliability standards for screening and placementdecisions.2. What is the pattern of relationships between theCurriculum—Based Measures and the norm—referencedmeasures?This study demonstrated that when the total sample wasexamined, the curriculum—based and the norm—referencedmeasures were highly correlated with correlations betweenthe two kinds of measures ranging from .55 to .80(see Table 7). Lower correlations were observed when thelearning disabled and the regular education samples wereconsidered separately, indicating that, in general, theserelationships do not hold for the separate samples. Asubstantial relationship between the KTEA Decoding subtestand the CBM passage speed test (r=.63) was found for thelearning disabled sample, providing some validity evidencefor this measure. Separate sample correlations do notreflect the substantial relationships found in theliterature.3. Is a Curriculum—Based Measurement battery, or part of aCurriculum—Based Measurement battery, interchangeablewith or superior to the traditional norm—referencedsubtests in identifying reading difficulty?170Canonical analyses (see Table 17, Chapter Four) indicatethat the Curriculum-Based Reading Passages Average ReadingSpeed Score is as efficient as the two Kaufman subtests inidentifying school-based indices of reading difficulty.Results from this table also indicate that some of thecurriculum—based and the two norm—referenced measures arevery efficient in predicting the dependent variables. Inother words, when two CBMs are considered (the passagespeed measure and the word list accuracy measure) CBMsappear to be interchangeable with the norm—referencedsubtests in identifying reading difficulty. Further tothis, there is no advantage in prediction in combining theCBM Word List with the CBM Reading Passages.4. Does combining the Curriculum—Based Measures with thenorm—referenced tests result in increased precision inprediction compared to either one taken singly?Separate canonical analyses were run for each of thenorm—referenced and the Curriculum—Based Measures. Thecanonical correlation of .86 with the KTEA subtests used asthe set of predictor variables is similar to the canonicalcorrelation of .92 when both norm—referenced andCurriculum—Based Measures are used as the set of predictorvariables. As well, the canonical correlation of .89 with171the CBMs used as the set of predictor variables, is alsosimilar to the canonical correlation of .92 when both norm—referenced and Curriculum—Based Measures are used as theset of predictor variables.The CBM canonical correlation of .89 may be slightlyhigher than the norm—referenced canonical correlation of.86 because there is a larger set of CBM variables; themarginally larger canonical R of .92, obtained when thenorm—referenced and curriculum—based predictors arecombined, might also be explained by the even greaternumber of variables in the equation.Because the differences in canonical correlations aresmall and possibly due to differing numbers of predictorvariables in the equations, it is apparent that, forpredictive purposes, no useful increase in precisionresults when the norm—referenced and the Curriculum—BasedMeasures are combined. The lack of increase may well bedue to the already high level of prediction for each set ofindependent variables (norm—referenced and curriculum—based) and the high correlations between the two sets.Conclusions and ImplicationsInformation from this study indicates that someCurriculum—Based Measures have potential in the172identification of students with significant readingdifficulty. The CBM passage speed and the CBM word listaccuracy scores were as predictive of the school—basedindices as were the two norm—referenced subtests.The potential of a passage reading measure to aid ininstructional planning is linked to its resemblance to aninformal reading inventory. A criticism of standardizedachievement tests is that the results may not be useful forinstructional planning, whereas the informal readinginventory yields more useful information for readinginstruction (Oliver & Arnold, 1978). Further to this,Mccracken (1962) found that standardized reading testsplace 63 percent of the students at their informal readinginventory frustration level. Similar results were reportedby Sipay (1964). The finding in this study that one scorefrom the CBM passage reading measure is as predictive ofthe school—based indices as the norm—referenced subtests isimportant because the passage reading measure may havegreater instructional utility than the traditional norm-referenced measures.Although the accuracy score from the CBM word listalso was predictive of the school—based indices, it issimilar in item type to the KTEA Reading Decoding subtestwhich offers the advantage of representative norms. Thereis no evidence that the CBM word list offers any173instructional or predictive benefit over the KTEA ReadingDecoding subtest. Word lists have been reconuuended forgeneral reading placement (Froese, 1976) but not forinstructional planning (Oliver & Arnold, 1978).In the present study, total sample correlationsindicated a strong relationship between the norm—referencedand Curriculum—Based Measures (r’s of .55 to .75) ——resembling other findings reported in the literature (Deno,Mirkin, & Chiang, 1982; Deno, Mirkin, Chiang, & Lowry,1980; Fuchs & Deno, 1981a, 1981b) —— but lower correlationswere obtained when the regular education and learningdisabled samples were analyzed separately. While the CBMpassage reading correlations ranged from .47 to .63 withthe KTEA subtests (passage speed having the highestcorrelation) in the learning disabled sample, CBM word listcorrelations were much lower (.22 to .47). When samplesare considered separately, CBMs do not have strongrelationships to norm—referenced measures. In a learningdisabled sample, moderate correlations were evidenced forthe passage speed measure, providing some evidence for theconcurrent validity of this CBM with the KTEA Decodingsubtest. The strong relationship between CBMs and norm—referenced tests may not apply to the learning disabled.However, the somewhat lower variance (for most measures) in174the learning disabled sample may have been responsible fora certain degree of the attenuation of these correlations.This study’s results provides some evidence for thereliability and validity of one score from a passagereading measure. The superior performance of the readingspeed over the accuracy score echoes findings in previousresearch that rate—based measures, rather than accuracy,are to be preferred (Deno, Mirkin, Chiang, & Lowry, 1980;Deno, Mirkin, Lowry, & Kuehnle, 1980; Deno, Mirkin, &Marston, 1980; Potter & Wamre, 1990).Most of the CBM literature investigates theproperties of multiple-occasion CBMs. This study providesinformation about a passage reading CBM that can beadministered on a single occasion, in similar fashion to anorm—referenced test. There may be potential for such apassage reading measure in district screening of studentsfor remedial services. Replication of the present studywith another passage measure with more ceiling isrecommended before this is considered.This study also confirms the validity of the two KTEAsubtests, especially the KTEA Reading Decoding subtest.The KTEA Reading Decoding subtest was usually the mostpredictive variable in the regression analyses.The high degree of intercorrelation among all thereading measures and the school—based indices in this study175most likely reflects some degree of individual differencesin general intellectual ability. In some researchtraditions, the investigators might obtain intelligencemeasures for all of the students in the study, and usethese to “correct” all other ability measures beforeconducting subsequent analyses. This approach, however,would have been contrary to the assumptions of Curriculum—Based Measurement and to the research questions in thisstudy. In CBM approaches, the focus is on skills andacademic performance, not on general intelligence or on“skill residuals” which may remain after intelligence ispartialled out. This is not intended to deny that moreintelligent students are often better readers; this can beassumed but is of no particular import when readingperformance is being examined. Consideration should alsobe given to the fact that the learning disabled sample waswithin the average range in intelligence scores butdemonstrated a significant lag in reading achievement.The results from this study do not indicate thesuperiority of one assessment method over the other, ratherthey illustrate that some CBMs are as reliable and as validas well recognized norm—referenced subtests. Theinteresting part of this finding is that some CBMs showpotential for instructional programme planning.176Some of the CBM literature stresses that traditionalnorm—referenced testing be abandoned (Galagan, 1985).This study did not gather information on the benefits ofother kinds of norm—referenced assessment, nor does thisstudy’s finding of CBM utility preclude the use of norm-referenced assessment. This research concentrated on theproblem of identifying students experiencing readingdifficulty and no conclusions about the utility ofintelligence testing should be drawn from these results.It is the author’s opinion, however, that clinical benefitscan be obtained from the administration of technicallyadequate and individually administered intelligence testswhen accompanied by curriculum—based information, providedthat this work is conducted by a well trained psychologist(see Reynolds, 1988).Recommendations for PracticeResults from this study indicate that no benefit maybe gained from the lengthy administration of complete norm-referenced reading batteries. In most analyses, theKaufman Test of Educational Achievement ReadingComprehension subtest accounted for less variance in theset of dependent variables than the much shorter KTEAReading Decoding subtest. Because no further instructional177information appears to be gained from the lengthyadministration of the KTEA Reading Comprehension subtest,it is recommended that school psychologists conductingreading assessment concentrate on gathering quick andreliable norm—referenced information with tests like theKTEA Reading Decoding subtest, and then move on to moreinstructionally relevant curriculum—based data gathering.This study provides some evidence that a singleoccasion passage reading CBM has good reliability andvalidity. Further evidence of this is required and theextension of the passages into higher grade levels to avoidceiling problems is necessary. If this is accomplished,consideration could be given to the development of localnorms. Ideally, however, data on reading performanceshould be gathered throughout the school year so thatcomparisons of progress performance can also be made. Thiswould involve developing a bank of representative readingpassages at each grade from an accepted reading series(concentrating on the elementary school at present) andadministering them three or four times during the schoolyear to representative samples of students so that localnorms for both speed and accuracy could be developed.Alternatively, a correct words per minute score could beused. This kind of CBM normative information would offerthe advantage of traditional psychometric comparison along178with data for instructional planning. It could also resultin considerable time saving for school psychologists sincethe need for formal psychoeducational assessment wouldlikely decline if CBM procedures preceded referral forformal assessment (Shinn, 1989).Although this study concentrated on single occasionCBM data gathering, the practical application of CBMappears to lie in its sensitivity to learning gains(Friedman, 1990) and its relationship to achievement gains(Fuchs & Fuchs, l986a). For this reason, the multiple-occasion use of CBMs, in particular passage reading CBM5,is recommended in educational practice.Limitations of the StudyBecause this study was conducted with Year Fourregular class and severely learning disabled children inone Metropolitan Vancouver school district, generalizationsto other grades and to other school districts cannot bemade from this study. Further to this, the sample size of105 students is not large enough or representative enoughto be considered as norms for Year Four.Availability of examiner testing time prevented thegathering of test-retest and inter-rater reliability dataon the Curriculum-Based Word List. Thus the only179reliability coefficient that could be calculated for theWord List was an internal consistency coefficient on theaccuracy score.A further limitation of this study was the restrictionon the amount of examiner training time. Because theexaminers required that they be trained in two sessions,sufficient practice time was not available. This resultedin some examiner error in the administration and scoring ofthe Curriculum—Based Measures and some error in the scoringof the Kaufman subtests. This has negligible impact on thefindings of the study, since the author personally examinedthe data and made the appropriate corrections. Then,separate analyses were run on some of the uncorrected dataand compared to the corrected data: no significantdifferences in results were obtained. However, there areimplications for the training that should occur beforeschool—based professionals use such measures in classroomsettings on a regular basis.Some results indicate that the learning disabledsample performs differently in pattern of relationshipsamong the measures. The relatively small learning disabledsample in this study (n=35) and the use of program statusas one dependent measure prevent an analysis of how thelearning disabled sample considered separately relates tothe set of dependent measures.180It was not possible to analyze the relationshipbetween the school-based indicators and sets of CBM or KTEApredictors for the learning disabled sample takenseparately. There were two reasons for this: the samplewas too small to permit this kind of analysis (n=35), andone of the criterion measures (placement status — learningdisabled or regular) would have become a constant and onlytwo criterion variables would have remained to form thecomposite. Whether the observed difference in pattern ofcorrelations between the learning disabled and regularsample is indicative of different predictions for thelearning disabled sample is unknown at present.Stepwise multiple regression results for the learningdisabled sample should also be interpreted with cautionbecause of the small sample size. A further limitation ofthis study is the lack of reliability and validityinformation for the district test. Although this test wasdeveloped by a school district committee and was designedto reflect school district evaluations of readingachievement, in stepwise regressions and in an analysis ofthe beta weights, different patterns were observed for thedistrict test. Questions about the test’s reliability andvalidity arise when this is considered.The use of a single—occasion rather than a multi—occasion CBM can also be seen as a limitation of this181study. Because the bulk of CBM research pertains to multi-occasion CBNs, the use of a longer and single—occasion CBMin this study does not reflect established CBM practice.Serious ceiling difficulties were observed for the CBMpassage reading percent correct measure. This may havecontributed to the somewhat lower reliability of thismeasure and to its lower relationship to the school—basedindices. Passages higher than early grade seven arenecessary in this measure to avoid ceiling difficulty.Another limitation of this study is the possibility ofcontamination of program placement status with norm—referenced testing. Part of the decision to place studentsin special programs was based upon scores from theWoodcock—Johnson Psychoeducational Battery: AchievementTests. Although other factors, including teacher, learningassistance centre, counsellor, and screening committeeassessment were involved in placement decisions, Woodcock—Johnson scores were also considered. The problem ofcontamination of the criterion may not be serious, however,because there is evidence that different results areobtained and different decisions about students are madewhen different standardized achievement tests are used(Cordini & Snyder, 1981; Lindsey & Armstrong, 1984).This study provides information about two single-occasion reading CBMs. Generalizations to other reading182CBMs or to multi-occasion reading CBMs cannot strictly bemade from this research. The study, however, supportsprevious research attesting to the reliability and validityof some CBMS (see Chapter Two).Directions for Future ResearchThis study investigated two kinds of single occasionreading Curriculum—Based Measures. Results generallyindicated that the reading passages performed well whencompared to school—based indices of reading achievement.Further research is needed to investigate the properties ofa similar passage reading CBM (with higher grade levelpassages) derived from other reading series. Similarreliability and validity investigations should beconducted.Results from this research indicate that some readingCBMs discriminate learning disabled and regular educationstudents as well as norm—referenced reading tests. Alimitation of this research is the lack of informationabout how low—achieving students perform on norm—referencedand Curriculum—Based Measures in comparison to the twoother groups. Although there is some evidence that readingCBMs can discriminate learning disabled and low-achievingstudents (Deno et al., 1983; Shinn et al., 1987),183information in this regard about the performance ofspecific CBMs is necessary. Examination of a low—achievingstudent group, along with regular education and learningdisabled groups as used in this study, would provide usefuldirection for future research.It would also be useful to replicate this study inanother district with a representative sample and othergrade levels. This would also allow the gathering of test-retest and inter-rater reliability data on the Curriculum-Based Word List, information that could not be supplied bythe present study. If a succession of elementary gradelevels was sampled, it would be possible to examine thepossible existence of developmental growth patterns inscores from the reading passages, and whether such patternsin speed and accuracy relate well to other findings fromthe reading literature. The establishment of developmentalgrowth patterns in the reading passages would also providefurther evidence for their validity. If the sample sizewas large and representative, district norms could bedeveloped.If the aforementioned study was conducted, anotherstudy could be done, wherein reading passage norms would beused for district screening and eligibility purposes, and acomparison between the identification rates for CBM andtraditional procedures would be made. Further to this,184teacher satisfaction with the traditional versus the CBMidentification practice could be rated. Included in thiscould be information about teacher rating of instructionalutility of the two practices. Information about the timenecessary for each identification procedure should also begathered in such a study.Another possibility for research would be to compare asingle—occasion passage reading CBM with a multi—occasionpassage reading CBM from the same reading series anddetermine if there is a benefit, in instructional planningor sensitivity to student learning gain, from the multi-occasion data, and if this benefit justifies the increasedassessment time.The issue of examiner accuracy arose in this study.Research is necessary to determine what level of examinertraining is necessary to ensure the gathering of accurateCBM data. The amount of training time necessary foraccurate CBM and norm—referenced reading assessment couldalso be compared.The learning disabled sample in the present study wastoo small to permit a separate regression analysis.Because the learning disabled sample demonstrated asignificantly different pattern of correlations from theregular sample, a separate canonical analysis for thelearning disabled sample would have been desirable.185Replication, with a larger learning disabled sample, wouldpermit such a canonical analysis.Another possibility for future research is to use asimilar passage reading CBM derived from another readingseries, with adequate range of difficulty, that would bescored for correct words per minute as well as for speedand accuracy separately. Since much of the CBM literaturerefers to correct words per minute scores, it would beuseful to know how the three scores are related to eachother and how they relate to school—based indices ofreading. Information from the present study indicates thatwithin the reading passages, correct words per minutescores are very highly related to reading speed scores.Quality research is necessary in the area of readingCBM. More methodologically stringent and well describedstudies in reading CBM are necessary. Further researchshould concentrate on generating substantial evidence forthe reliability, validity, and utility of reading CBMprocedures.186ReferencesAlderson, J. C. (1978). doze procedures. In 0. K. Buros(Ed.), The eighth mental measurements yearbook (pp.1171-1174). Highland Park, NJ: Gryphon Press.Algozzine, B., Christenson, S., & Ysseldyke, J. E. (1982).Probabilities associated with the referral toplacement process. Teacher Education and SpecialEducation, , 19—23.Algozzine, B. & Ysseldyke, J. E. (1983). Learningdisabilities as a subset of school failure: Theoversophistication of a concept. ExceptionalChildren, Q, 242-246.Algozzine, B. & Ysseldyke, J. E. (1986). The future of theLD field: Screening and diagnosis. Journal ofLearning Disabilities, 19, 394—398.Algozzine, B., & Ysseldyke, J. -E. (l9&7) Questioningdiscrepancies: Retaking the first step 20 yearslater. Learning Disability Quarterly, ,301—312.Anderson, R. C., Heibert, E. H., Scott, J. A., &Wilkinson, I. A. G. (1985). Becoming anation of readers: The report of theCommission on Reading. Washington, DC:The National Institute of Education.Arter, J., & Jenkins, J. (1979). Differentialdiagnosis- prescriptive teaching: A criticalappraisal. Review of Educational Research, 4.,517—555.Baddeley, A. D. (1979). Working memory and reading.In P. A. Kolers, M. E. Wrolstad, & H. Bouma(Eds.), Processing of visible language. New York:Plenum Press.Barron, R. (1980). Visual-orthographic and phonologicalstrategies in reading and spelling. In U. Frith(Ed.), Cognitive processes in spelling. London: H. K.Lewis.Berk, R. A., (1982). Effectiveness of discrepancy scoremethods for screening children with learningdisabilities. Learning Disabilities, 1, 11—24.187Betts, E. A. (1946). Foundations of reading instruction.New York: American Book.Bowers, R. G., Steffy, R., & Tate, E. (1988). Comparison ofthe effects of IQ control methods on memory and namingspeed predictors of reading disability. ReadingResearch quarterly, , 304—319.Bray, N. M., & Estes, R. E. (1975). A comparison of thePIAT, CAT, and WRAT scores and teacher ratings forlearning disabled children. Journal of LearningDisabilities, a, 519—523.Cahill, V. M. (1979). The effects of standardized testinformation on teachers’ perceptions of their studentsand teachers’ perceptions on student achievement: Alongitudinal study. Dissertation AbstractsInternational, j, 637.Carver, R. P. (1974). Two dimensions of tests:Psychometric and edumetric. AmericanPsychologist, , 512—518.Chall, J. S. (1983). Stages of reading development.New York: McGraw-Hill.Clymer, T., & Fenn, T. (1979). Reading 720 RainbowEdition. Lexington, MA: Ginn.Coladarci, T. (1986). Accuracy of teacher judgments ofstudent responses to standardized test items. Journalof Educational Psychology, j, 141-146.Cordoni, B. K., & Snyder, M. K. (1981). A comparison oflearning disabled college students’ achievement fromWRAT and PIAT grade, standard, and subtest scores.Psychology in the Schools, , 28—34.Crocker, L. & Algina, J. (1986). Introduction toclassical and modern test theory. New York: Holt,Rinehart and Winston.Cronbach, L. J. (1970). Essentials of PsychologicalTesting.Third Edition. New York: Harper and Row.Cronbach, L. J., & Meehl, P. E. (1955). Construct validityin psychological tests. Psychological Bulletin, ,281—302.188Cronbach, L. J., & Snow, R. (1977). Aptitudes andinstructional methods. New York: Irvington.Deno, S. L. (1985). Curriculum—based measurement: Theemerging alternative. Exceptional Children, 52,219—232.Deno, S. L. (1986). Formative evaluation of individualstudent programs: A new role for schoolpsychologists. School Psychology Review, 15,358—374.Deno, S. L. (1987). Curriculum—based measurement.Teaching Exceptional Children, Fall, 41—47.Deno, S. L. (1990). Individual differences andindividual difference: The essential differenceof special education. The Journal of SpecialEducation, j, 160—173.Deno, S., Marston, D., & Mirkin, P. (1982). Validmeasurement procedures for continuous evaluationof written expressioni Exceptional Children, 48,368—371.Deno, S. L., Marston, D., Mirkin, P. K., Lowry, L.,Sindelar, P., & Jenkins, J. (1982). The use ofstandard tasks to measure achievement in reading,spelling, and written expression: A normative anddevelopmental study (Research Report No. 87).Minneapolis: University of Minnesota Institute forResearch on Learning Disabilities (ERIC DocumentReproduction Service No. ED 227 129).Deno, S. L., Marston, D., Shinn, M., & Tindal, G. (1983).Oral reading fluency: A simple datum for scalingreading disability. Tonics in Learning and LearningDisabilities, 2., 53—59.Deno, S. L., Mirkin, P. K., & Chiang, 8. (1982).Identifying valid measures of reading. ExceptionalChildren, , 36-45.Deno, S. L., Mirkin, P. K., Chiang, B., & Lowry, L.(1980). Relationships among simple measures ofreading and performance on standardized achievementtests. (Report No. 20). Minneapolis MN: University ofMinnesota, Institute for Research on LearningDisabilities. (ERIC Document Reproduction Service No.ED 197 507).189Deno, S, Mirkin, P., Lowry, L., & Kuehnle, K. (1980).Relationships among simple measures of spelling andperformance on standardized achievement tests.(Research Report No. 21). Minneapolis, MN:University of Minnesota, Institute for Research onLearning Disabilities. (ERIC Document ReproductionService No. ED 197 508).Deno, S. L., Mirkin, P. K., & Marston, D. (1980).Relationships among simple measures of writtenexpression and performance on standardizedachievement tests (Research Report No. 22).Minneapolis: University of Minnesota Institutefor Research on Learning Disabilities.Dixon, W. J. (1985). BMDP statistical software. Berkeley:University of California Press.Dunn, L. M., & Markwardt, F. C. (1970). Peabody IndividualAchievement Test. Circle Pines, MN: American GuidanceService, Inc.Eaton, 14, & Lovitt, R.- (1972>. Achievement tests vs.direct and daily measurement. In G. Semb (Ed.),Behavior analysis and education. Lawrence KS:University of Kansas.Epps, S., Ysseldyke, J. E., & Mc Gue, M. (1984). “I knowone when I see one” — Differentiating LD and non—LDstudents. Learning Disability Quarterly, j, 89-101.Epstein, S. (1980). The stability of behavior:Implications for Psychological Research. AmericanPsychologist, 35, 790-806.Ferguson, G. A. (1971). Statistical Analysis in Psychologyand Education, Third Edition. New York: McGraw-Hill.Floden, R., Porter, A., Schmidt, W., & Freeman, D.(1980). Don’t they all measure the same thing?In E. Barker & E. Quellnalz (Eds.), Educationaltesting and evaluation. Beverly Hills, CA: Sage.Friedman, J. (1990). An evaluation of the relativesensitivity to student growth in reading and spellingof standardized achievement tests and curriculum—basedmeasures. Dissertation Abstracts International, ,1551.190Froese, V. (1971). Word recognition tests: Are theyuseful beyond grade three?. The Reading Teacher, 24,432—438.Froese, V. (1975). Does the dolch do?. The Alberta Journalof Educational Research, 2J, 23—27.Froese, V. (1976). Functional reading levels: From gradedword lists?. The Alberta Journal of EducationalResearch, , 325-329.Fuchs, L. S. (1981). The concurrent validity of progressmeasures of basal reading material. Unpublisheddoctoral dissertation, University of Minnesota.Fuchs, L. S. (1986). Monitoring progress among mildlyhandicapped pupils: A review of current practiceand research. Remedial and Special Education, 2,5—12.Fuchs, L. S. (in press). Enhancing instructionalprogramming and student achievement withcurriculum—based measurement. In J. Kramer (Ed.>,Curriculum—based assessment: Examining oldproblems. evaluating new solutions. Hillsdale, NJ:Erlbaum.Fuchs, D., & Balow, B. (1974). Formulating an informalreading inventory, unpublished manuscript.(Available from B. Balow, University of Minnesota,Pattee Hall, Minneapolis, Minnesota, 55455).Fuchs, D., & Deno, S. (1981a). A comparison of readingplacements based on teacher judgment, standardizedtesting, and curriculum—based assessment. (Report No.56). Minneapolis, MN: University of Minnesota,Institute for Research on Learning Disabilities.(ERIC Document Reproduction Service No. ED 211 603).Fuchs, L., & Deno, S. (1981b). The relationship betweencurriculum—based mastery measures and standardizedachievement tests in reading. (Report No. 57).Minneapolis, MN: University of Minnesota, Institutefor Research on Learning Disabilities. (ERICDocument Reproduction Service No. ED 212 662).Fuchs L.S., & Deno, S. L. (in press). Curriculum—basedmeasurement and curriculum—based assessment:Paradigmatic distinctions. Exceptional Children.191Fuchs, L. S., Deno, S. L., & Marston, D. (1983).Improving the reliability of curriculum—basedmeasures of academic skills forpsychoeducational decision making. Diagnostigue,a, 135—149.Fuchs, L., Deno, S.L., & Mirkin, P.K. (1984). Theeffects of frequent curriculum—based measurementand evaluation on pedagogy, student achievement,and student awareness of learning. AmericanEducational Research Journal, 449—460.Fuchs, L. S., & Fuchs, D. (1984). Criterion referencedassessment without measurement: How accurate forspecial education? Remedial and Special Education,, 29—32.Fuchs, L. S., & Fuchs, D. (l986a). Effects of systematicformative evaluation: A meta—analysis. ExceptionalChildren, •3, 199—208.Fuchs, L. S., & Fuchs, D. (1986b). Linking assessment toinstruc-tion-al intervention: An overview. Sc-hoolPsychology Review, 15, 318—323.Fuchs, L. S., & Fuchs, D. (1986c). Curriculum—basedassessment of progress toward long—term and short—term goals. The Journal of Special Education, Q, 69—82.Fuchs, L. S., & Fuchs, D. (1987). The relation betweenmethods of graphing student performance data andachievement: A meta—analysis. Journal of SpecialEducation Technology, , 5-13.Fuchs, L. S., & Fuchs D. (1990). Curriculum-basedassessment. In C. R. Reynolds & R. W. Kamphaus(Eds.), Handbook of psychological and educationalassessment of children. New York: The GuilfordPress.Fuchs, D., Fuchs, L. S., Benowitz, S.., & Barringer, K.(1987). Norm-referenced tests: Are they valid foruse with handicapped students? Exceptional Children,.4, 263—271.Fuchs, L., Fuchs, D., & Deno, S. (1982). Reliability andvalidity of curriculum—based informal readinginventories. Reading Research quarterly, j, 6—26.192Fuchs, L. S., Fuchs, D., & Deno, S. L. (1984). Samplingpassages and studentbehavior for IRIS: A reply toDuff lemeyer. Reading Research Quarterly, , 248-249.Fuchs, L. S., Fuchs, D., & Deno, S. L. (1985).Importance of goal ambitiousness and goal masteryto student achievement. Exceptional Children, ,63—71.Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (l989a).Effects of alternative goal structures withincurriculum—based measurement. Exceptional Children,, 429—438.Fuchs, L. S., Fuchs. D., & Hamlett, C. L. (1989b).Computers and curriculum—based measurement: Effectsof teacher feedback systems. School PsychologyReview, , 112—125.Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Allinder,R. M. (1991). Effects of expert system advicewithin curr-iculum—basedmeasuremert on teacherplanning and student achievement in spelling,School Psychology Review, Q, 49-66.Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Hasseibring, T.S. (1987). Using computers with curriculum-basedmonitoring: Effects on teacher efficiency andsatisfaction. Journal of Special EducationTechnology, 8, 14-27.Fuchs, L. S., Fuchs, D.., & Maxwell, L. (1988). Thevalidity of informal reading comprehension measures.Remedial and Special Education, 9, 20—28.Fuchs, L. S., Fuchs, D., & Stecker, P. M. (1989). Effectsof curriculum—based measurement on teachers’instructional planning. Journal of LearningDisabilities, 22, 51—58.Fuchs, L. S., Haiulett, C. L., Fuchs, D., Stecker, P. M.,& Ferguson, C. (1988). Conducting curriculum-basedmeasurement with coinpterized data collection:Effects on efficiency and teacher satisfaction.Journal of Special Education Technology, 9, 73—86.193Fuchs, L. S., & Maxwell. L. (1987). Interactive effectsof reading mode, production format, and structuralimportance of text among reading disabled pupils.Manuscript submitted for publication. (Availablefrom L. S. Fuchs, Peabody College, VanderbiltUniversity, Nashville, TN 37203).Fuchs, L. S., & Shinn, M. R. (1989). writing curriculum-based IP objectives. In N. R. Shinn (Ed.),Curriculum—based measurement: Assessing specialchildren (pp. 130-153). New York: Guilford Press.Fuchs, L., Tindal, G., & Deno, S. (1981). Effects ofvarying item domain and sample duration ontechnical characteristics of daily measures inreading. (Report No. 48). Minneapolis, MN:University of Minnesota, Institute for Research onLearning Disabilities. (ERIC Document ReproductionService No. ED 211 606).Fuchs, L. S., Tindal, G., & Deno, S. L. (1984).Methodological issues in curriculum—based readingassessment. Diagriostigue, 9, 191=207.Galagan, J. E. (1985). Psychoeducational testing: Turnout the lights, the party’s over. ExceptionalChildren, 52, 288—299.Gardner, E. F., Rudman, H. C., Karisen, B., & Merwin,J. C. (1982). Stanford Achievement Test (7th ed.).Cleveland: The Psychological Corporation.Gerken, K. (1985). Best practices in academic assessment.In A. Thomas & J. Grimes (Eds.), Best practices inschool psychology (pp. 157—170). Kent, Ohio: TheNational Association of School Psychologists.Germann, G. (1985). Pine county special educationcooperative total special education (TSES: Asummary. Sandstone MN: Pine County SpecialEducation Cooperative.Germann, G., & Tindal, G. (1985). An application ofcurriculum—based assessment: The use of directand repeated measurement. Exceptional Children,52, 244—265.194Ghiselli, E. E., Campbell, J. P., & Zedeck, S. (1981).Measurement theory for the behavioral sciences.San Francisco: W. H. Freeman and Company.Gickling, E. E., & Thompson, V. P. (1985). A personal viewof curriculum—based assessment. Exceptional Children,..a, 205—218.Gjnn and Company. (1979). Reading 720. Xerox of Canada:Ginn and Company.Good, R. H., & Salvia, J. (1988). Curriculum bias inpublished, norm—referenced reading tests:Demonstrable effects. School Psychology Review, jJ,51—60.Gough, P., & Hillinger, N. (1980). Learning to read: Anunnatural act. Bulletin of the Orton Society, Q,171—196.Gramenz, G. W., Johnson, R. C., & Jones, B. G.(1982 March). An exploratory study of the concept of-- curriculum—referenced norms using the Stanford-Achievement Test sixth edition. Paper presented atthe annual meeting of the National Council onMeasurement in Education, New York.Gresham, F. N. (1983). Multitrait - multimethod approachto multifactored assessment: Theoretical rationaleand practical application. School Psychology Review,j, 26—34.Gresham, F. M., Reschly, D. J., & Carey, M. P. (1987).Teachers as “tests”: Classification accuracy andconcurrent validation in the identification oflearning disabled children. School Psychology Review,543—553.Gross-Glenn, K., Lewis, D. C., Smith, S. D., & Lubs,H. A. (1985). Phenotype of adult familialdyslexia: Reading of visually transformed textsand nonsense passages. International Journal ofNeuroscience, , 49—59.Hammill, D. D. (1990). On defining learning disabilities:An emerging consensus, Journal of LearningDisabilities, 23, 74—83.195Handley, H. M. (1986). Third phase: achievement gainsassociated with implementation of the assistantteacher program in the first three grades.Mississippi: Mississippi State University, Bureau ofEducational Research.Hansen, C. L. M. (1979). Chicken soup and other forms ofcomprehension. In J. E. Button, T. Lovitt, & T. D.Rowland (Eds.), Communications research in learningdisabilities and mental retardation. Baltimore:University Park Press.Hargis, C. H. (1987). Curriculum based assessment: Aprimer. Springfield, Illinois: Charles C. ThomasPublisher.Harris, A. J., & Jacobson, M. D. (1972). Basicelementary reading vocabularies — The first Rseries. New York: MacMillan.Harris, A. J., & Jacobson, H. D. (1982). Basic readingvocabularies. New York: Macmillan Publishing Co.,Inc.Hersen, H., & Barlow, D. H. Single—case experimentaldesigns: strategies for studying behavior change.New York: Pergamon Press, 1976.Hopkins, K. D., George, C. A., & Williams, D. D. (1985).The concurrent validity of standardized achievementtests by content area using teachers’ ratings ascriteria. Journal of Educational Measurement, , 177—182.Howell, K. W. (1986). Direct assessment of academicperformance. School Psychology Review, , 324—335.Hynd, G. W., & Semrud-Clikemen, M. (1989). Dyslexiaand neurodevelopmental pathology: Relationshipsto cognition, intelligence, and reading skillacquisition. Journal of Learning Disabilities,22, 204—216.Jenkins, J. R., Heliotis, J., Haynes, M., & Beck, K.(1986). Does passive learning account fordisabled readers’ comprehension deficits inordinary reading situations? Learning Disabilityquarterly, , 69-75.196Jenkins, J. R., & Pany, D. (1978). Standardizedachievement tests: How useful for specialeducation? Exceptional Children, j4., 448-453.Johnston, P. B. (1982). Implications of basic researchfor the assessment of reading comprehension(Technical Report No. 206). Urbana: University ofIllinois, Center for the Study of Reading. (ERICDocument Reproduction Service No. ED 201-987)Joyce, B. G., & Wolking, W. D. (1987). Standardizedtests and timed curriculum—based assessments: Acomparison of two methods for screening high—risk students. Journal of PsychoeducationalAssessment, 3, 185—193.Kamphaus, R. W., Schmitt, C. S., & Mings, D. R. (1986).Three studies of the validity of the Kaufman Testof Educational Achievement. Journal ofPsychoeducational Assessment, 4., 299—305.Karlsen, B., Madden, R., & Gardner, E. F. (1975).Stanford Diagnostic Reading Test. -New York:Harcourt, Brace, Jovanovich, Inc.Kaufman, A. S. (1979). Intelligent testing with theWISC-R. New York: John Wiley & Sons.Kaufman, A. S., & Kaufman, N. L. (1985). Kaufman Testof Educational Achievement. Circle Pines, MN:American Guidance Service, Inc.Kavale, K. A., & Forness, S. R. (1987). Substanceover style: Assessing the efficacy of modalitytesting and teaching. Exceptional Children, 4.,228—239.Kavale, K. A., & Forness, S. R. (1990). Substanceover style: A rejoinder to Dunn’s animadversions.Exceptional Children, , 357—361.Kender, J. P. (1969). How useful are informal readingtests? In A. Beery, T. C. Barrett, & W. R. Powell(Eds.), Elementary reading instruction. Boston: Allyn& Bacon.Kratochwill, T. R., Brody, G. H., & Piersel, W. C.(1979). Time—series research: Some comments ondesign methodology for research in learningdisabilities. Journal of Learning Disabilities,.i.a, 52—58.197LaBerge, D., & Samuels, S. J. (1974). Toward a theoryof automatic information processing in reading.Cognitive Psychology, 6, 293—323.Lakin, K. C. (1983). A response to Gene V. Glass.Policy Studies Review, a, 233—239.Liberman, I. (1973). Segmentation of the spoken wordand reading acquisition. Bulletin of the OrtonSociety, 23, 65—77.Lindsey, J. D., & Armstrong, S. W. (1984). Performance ofEMR and learning—disabled students on the Brigance,Peabody, and Wide Range Achievement tests. AmericanJournal of Mental Deficiency, , 197—201.Madden, R., Gardner, E. F., Rudman, H. C., Karisen, B.,& Merwin, J. C. (1973). Stanford achievement tests.New York: Harcourt, Brace, Jovanovich, Inc.Marston, D. (1982). The technical adequacy of direct,repeated measurement of academic skills in low—achieving elementary students. Unpublisheddoctoral dissertation, Minneapolis: University ofMinnesota.Marston, D. B. (1989). A curriculum—based measurementapproach to assessing academic performance: Whatit is and why do it. In M. R. Shinn (Ed.),Curriculum—based measurement: Assessing specialchildren (pp 18-78). New York: The Guilford Press.Marston, D., & Deno, S. L. (1982). Implementation ofdirect and repeated measurement in the schoolsetting (Research Report No. 106). Minneapolis:University of Minnesota Institute for Researchon Learning Disabilities.Marston, D., Deno, S., & Tindal, G. (1983). A comparisonof standardized achievement tests and directmeasurement techniques in measuring pupil progress.(Report No. 126). Minneapolis, MN: University ofMinnesota, Institute for Research on LearningDisabilities. (ERIC Document Reproduction ServiceNo. ED 236 198).Marston, 0., Deno, S. L., & Tindal, G. (1984). Eligibilityfor learning disabilities services: A direct andrepeated measurement approach. Exceptional Children,._Q, 554—555.198Marston, D., Fuchs, L. S., & Deno, S. L. (1986). Measuringpupil progress: Comparison of standardizedachievement tests and curriculum—related measures.Diagnostigue,,77—90.Marston, D., Lowry, L., Deno, S., & Mirkin, p. (1981). Ananalysis of learning trends in simple measures ofreading, spelling, and written expression: Alongitudinal study. (Report No. 49). Minneapolis,MN: University of Minnesota, Institute for Researchon Learning Disabilities. (ERIC DocumentReproduction Service No. ED 211 602).Marston, D., & Magnusson, D. (1985). Implementingcurriculum—based measurement in special and regulareducation settings. Exceptional Children, 52,266—276.Marston, D., & Magnusson, D. (1988). Curriculum—basedassessment: District level implementation. In J.Graden, J. Zins, & M. Curtis (Eds.), Alternativeeducational delivery systems: Enhancinginstructional options for allstudents (pp. 137-172). Washington, DC: National Association ofSchool Psychologists.Marston, D., Mirkin, P., & Deno, S. (1984). Curriculum—based measurement: An alternative to traditionalscreening, referral, and identification. Journalof Special Education, j, 109-117.Marston, D., Tindal, G., & Deno, S. L. (1982).Predictive efficiency of direct, repeatedmeasurement: An analysis of cost and accuracy inclassification. (Report No. 104). Minneapolis, MN:University of Minnesota, Institute for Researchon Learning Disabilities. (ERIC DocumentReproduction Service No. ED 266 052).Marston, D., Tindal, G., & Deno, S. L. (1983). Acomparison of standardized achievement tests anddirect measurement techniques in measuring studentprogress (Research Report No. 126). Minneapolis:University of Minnesota Institute for Research onLearning Disabilities.McCracken, R. A. (1962). Standardized reading tests andinformal reading inventories. Education, , 366-369.199Mehrens, W. A. (1984). National tests and localcurriculum: Match or mismatch? EducationalMeasurement: Issues and Practice, 3, 9—15.Mehrens, W. A., & Phillips, S. E. (1986). Detectingimpacts of curricular differences in achievementtest data. Journal of Educational Measurement, 23,185—196.Messick, S. (1989). Validity. In R. L. Linn (Ed.),Educational Measurement Third Edition. New York,NY: Macmillan.Mirkin, P., Deno, S., Tindal, G., & Kuehnle, K. (1982).Frequency of measurement and data utilization asfactors in standardized behavioral assessment ofacademic skill. Journal of Behavioral Assessment,4, 361—370.Mirkin, P., Marston, D., & Deno, S. L. (1982). Directand repeated measurement of academic skills: Analternative to traditional screening, referral,and identification of 1earning disabled studeiits.(Report No. 75). Minneapolis, MN: University ofMinnesota, Institute for Research on LearningDisabilities. (ERIC Document Reproduction ServiceNo. ED 224 191)Morine-Dershimer, G. (1978). The anatomy of teacherprediction. Educational Research Quarterly, 2, 59-65.National Joint Committee on Learning Disabilities.(1988). Letter to NJCLD member organizations.(Available from The Orton Dyslexia Society, 724York Road, Baltimore, MD 21204).Nelson, L. R. (1974). Guide to LERTAP use andinterpretation. Dunedin, N.Z.: University ofOtago.Nunnally, J. C. (1978). Psychometric theory. New York:McGraw-Hill.Oakland, T., Shermis, M. D., & Coleman, M. (1990). Teacherperceptions of differences among elementary studentswith and without learning disabilities in referredsamples. Journal of Learning Disabilities, 23, 499-504.200Oliver, J. 0., & Arnold, R. (1978). Comparing astandardized test, an informal inventory, and teacherjudgment on third grade reading. Reading Improvement,, 56—59.Peterson, J., Greenlaw, M. J., & Tierney, R. J. (1978).Assessing instructional placement with the IRI: Theeffectiveness of comprehension questions. Journal ofEducational Research, 7j, 247—250.Phillips, B. N. (1986). New directions in aptitude-treatment interaction research: Concepts and methods.In C. R. Reynolds & U. L. Willson (Eds.),Methodological and statistical advances in the studyof individual differences. New York: Plenum Press.Potter, N. L., & Wamre, H. M. (1990). Curriculum-basedmeasurement and developmental reading models:Opportunities for cross—validation. ExceptionalChildren, , 16-25.Powell, W. K. (1971). Validity of the IRI reading levels..Elementary English, 4, 637-642.Rathbone, N., & Graham, N. C. (1981). Parent participationin the primary school. Educational Studies, 2, 145—150.Reynolds, C. R. (1988). Putting the individual intoaptitude—treatment interaction. ExceptionalChildren, 54, 324-331.Robinson, G. C., Conry R. F., & Conry, J. L. (1985). TheCanim Lake survey of special needs children(Unpublished Confidential Report). Vancouver:University of British Columbia, Department ofPaediatrics.Salvia, J., & Ysseldyke, J.E. (1991). Assessment inspecial and remedial education (5th ed.). Boston:Houghton Miffun.Sattler, J. M.(1988). Assessment of children (3rd ed.).San Diego: Author.Schmidt, W. H. (1983). Content biases in achievementtests. Journal of Educational Measurement, ,165—178.201Shapiro, E. S., & Derr, T. F. (1987). An examinationof the overlap between reading curricula andstandardized achievement tests. The Journal ofSpecial Education, , 59—67.Shinn, M. R. (1981). A comparison of psychometric andfunctional differences between students labeledlearning disabled and low achieving. Unpublisheddoctoral dissertation, Minneapolis: University ofMinnesota.Shinn, M.R. (1988). Development of curriculum-basedlocal norms for use in special educationdecision-making. School Psychology Review, j,61—80.Shinn, N. R. (1989). Identifying and defining academicproblems: CBM screening and eligibility procedures.In M. R. Shinn (Ed.), Curriculum—based measurement:Assessing special children. New York: The GuilfordPress.Shinn, N. R., Gleason, H. N., & Tindal, G. (1989). Varyingthe difficulty of testing materials: Implications forcurriàulum—based measurement. The Journal of SpecialEducation, 23, 223-233.Shinn, M. R., Good, R. H., & Stein, S. (1989). Summarizingtrend in student achievement: A comparison ofmethods. School Psychology Review, , 356—370.Shinn, M., & Marston, D. (1985). Differentiating mildlyhandicapped, low—achieving, and regular educationstudents: A curriculum—based approach. Remedial andSpecial Education, 6, 31-38.Shinn, N. R., Tindal, G. A., Spira, D., & Marston, D.(1987). Practice of learning disabilities as socialpolicy. Learning Disability quarterly, Q, 17-28.Shinn, N. R., Ysseldyke, J. E., Deno, S. L., & Tindal,G. (1986). A comparison of differences betweenstudents labeled learning disabled and lowachieving on measures of classroom performance.Journal of Learning Disabilities, , 545—552.Sipay, E. R. (1964). A comparison of standardized readingtests and informal reading inventories. The ReadingTeacher, j2, 265-268.202Skiba, R., Wesson, C., & Deno, S. L. (1982). Theeffects of training teachers in the use offormative evaluation in reading: An experimental—control comparison. (Report No. 88). Minneapolis,MN: University of Minnesota, Institute forResearch on Learning Disabilities. (ERICDocument Reproduction Service No. ED 228 821).Smith, N. B. (1959). Graded selections for informal readingdiagnosis. New York: New York University Press.Speece, D. L. (1990). Aptitude-treatment interactions:Bad rap or bad idea? The Journal of SpecialEducation, j, 139-149.SPSS Inc. (1988). SPSS—X User’s Guide: 3rd Edition.SPSS Inc.: Chicago, Ill.Stanovich, K., (1982). Individual differences in thecognitive processes of reading: I. Word decoding.Journal of Learning Disabilities, , 485—493.Steiger, J. H. (1q79). Multicorr: A computer program forfast, accurate, small—sample testing ofcorrelational pattern hypotheses. Educational andPsychological Measurement, 39, 677-680.Steiger, J. H. (1987). Program multicorr — version 2.4 —May 1987. University of British Columbia:Department of Psychology.Steiger, J. H., & Browne, M. W. (1984). The comparisonof interdependent correlations between optimallinear composites. Psychometrika, , 11—24.Suhorsky, J. (1975). An investigation of the relationshipbetween undeleted text preceding a doze test anddoze test results. Unpublished doctoraldissertation, University of Maryland, College Park.Swanson, H. L., Cochran, K. F., & Ewers, C. A. (1990).Can learning disabilities be determined from workingmemory performance? Journal of Learning Disabilities,21, 59—67.Telzrow, C. (1985). Best Practices in reducing error inlearning disability qualification. In A. Thomas &J. Grimes (Eds.), Best Practices in SchoolPsychology (pp. 431—446). Kent, Ohio: NationalAssociation of School Psychologists.203Thorndike, E. L., & Lorge, I. (1944). The teacher’s wordbook of 30,000 words. New York: Teachers College,Columbia University.Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986).The Stanford—Binet Intelligence Scale: Fourth Edition.Chicago: The Riverside Publishing Company.Thurlow, M., & Ysseldyke, J. (1983). We can agree ontechnical adequacy if we can agree on our standards:A response to Mardell—Czudnowski and Lesson.Diagnostigue, 8, 98—100.Tiegs, E. W., & Clark, W. W. (1970). CaliforniaAchievement Test. CA: CTB/McGraw-Hill.Tindal, G. (1989). Evaluating the effectiveness ofeducational programs at the systems level usingcurriculum—based measurement. In M. R. Shinn(Ed.), Curriculum—based measurement: Assessingspecial children. New York: The Guilford Press.Tindal, G., & Deno, S. L. (1981). Daily measurement ofreading: Effects of varying the size of the itempool. (Report No. 55). Minneapolis, MN: Universityof Minnesota, Institute for Research on LearningDisabilities. (ERIC Document Reproduction ServiceNo. ED 211 605).Tindal, G., Fuchs, L., Christenson, S., Mirkin, P., &Deno, S. (1981). The relationship between studentachievement and teacher assessment of short orlong—term goals. (Report No. 61). Minneapolis,MN: University of Minnesota, Institute forResearch on Learning Disabilities. (ERICDocument Reproduction Service No. ED 218 846).Tindal, G., Fuchs, L. S., Fuchs, D., Shinn,. M. R.,Deno, S. L., & Germann, G. (1985). Empiricalvalidation of criterion—referenced tests.Journal of Educational Research, j, 203-209.Tindal, G., Germann, G., & Deno, S. L. (1983).Descriptive research on the Pine County norms:A compilation of findings. Minneapolis:University of Minnesota Institute for Researchon Learning Disabilities.204Tindal, G., Marston, D., & Deno, S. L. (1983). Thereliability of direct and repeated measurement(Research Report No. 109). Minneapolis:University of Minnesota Institute for Researchon Learning Disabilities.Trembley, P. W., Caponigro, J. D., & Gaffney, V. (1980).Effects of programming from the WRAT and the PIAT forstudents determined to have learning disabilities inarithmetic. Journal of Learning Disabilities, 13, 297-93.Tucker, J. A. (1985). Curriculum—based assessment: Anintroduction. Exceptional Children, , 199—204.Tuinman, J. J., Blanton, W. E., & Gray, G. (1975). A noteon doze as a measure of comprehension. Journal ofPsychology, Q, 159-162.Vellutino, F. (1979). Dyslexia: Theory and research.Cambridge, MA: MIT Press.Webster, R. E., Hewett, J. B., & Crumbacker, H. M. (1989).Criterian-related validity of the WRAT-R and K-TEAwith teacher estimates of actual classroom academicperformance. Psychology in the Schools, , 243—248.Webster, R. E., Mclnnis, E. D., & Craver, L. (1986).Curriculum biasing effects in standardized andcriterion—referenced reading achievement tests.Psychology in the Schools, , 205-213.Wechsler, D. (1974). Wechsler Intelligence Scale forChildren- Revised. New York: PsychologicalCorporation.Wesson, C., King, R., & Deno, S. L. (1984). Direct andfrequent measurement: If it’s so good for us, whydon’t we use it? Learning Disability Quarterly, Z,45—48.Wesson, C., Skiba, R., Sevcik, B. King, R. P., & Deno,S. L. (1984). The effects of technically adequateinstructional data on achievement. Remedial andSpecial Education, 5, 17—22.Witt, J. C., Elliott, S. N., Gresham, F. M., & Kramer,J. J. (1988). Assessment of special children: Testsand the problem—solving process. Glenview, Illinois:Scott, Foresman and Company.205Woodcock, R. W. (1973). Woodcock Reading Mastery Tests:Form A. Circle Pines, MN: American Guidance Service,Inc.Woodcock, R. W. (1978). Woodcock-Johnson PsychoeducationalBattery. Allen, TX: DLM Teaching Resources.Wright, D., & Wiese, M. J. (1988). Teacher judgment instudent evaluation: A comparison of grading methods.Journal of Educational Research, 82, 10-14.Ysseldyke, J. E. (1979). Issues in psychoeducationalassessment. In G. D. Phye & D. J. Reschly (Eds.),School psychology perspectives and issues (pp.87—121). New York: Academic Press.Ysseldyke, J. E., Algozzine, B., & Epps, 5. (1983). Alogical and empirical analysis of current practicein classifying students as handicapped.Exceptional Children, , 160—166.Ysseldyke, J. E., Algozzine, B., Regan, R., & Potter,M. (1980). Technical adequacy of tests used byprofessionals in simulated decision making.Psychology in the Schools, 17, 202-209.Ysseldyke, J. E., Algozzine, B., Shinn, N., & McGue,N. (1982). Similarities and differences betweenunderachievers and students classified learningdisabled. Journal of Special Education, ,73—85.Ysseldyke, J. E., & Marston, D. (1982). A criticalanalysis of standardized reading tests. SchoolPsychology Review,,257-266.Ysseldyke, J. E., & Mirkin, P. K. (1982). The useof assessment information to plan instructionalinterventions: A review of the research. InC. R. Reynolds & T. B. Gutkin (Eds.), ghandbook of school psychology (pp.395—409). NewYork: John Wiley and Sons.Ysseldyke, J. E., Regan, R. R., Thurlow, N. L., &Schwartz, S. Z. (1981). Current assessmentpractices: The “cattle dip” approach.Diagnostigue, , 16-27.Ysseldyke, J. E., & Thurlow, M. (1983). We can agreeon technical adequacy if we can agree on ourstandards: A response to Mardell—Czudnowski andLessen. Diacnostigue, 8, 98—100.Ysseldyke, J. E., Thurlow, M., Graden, J., Wesson, C.,Algozzine, B., & Deno, S. (1983). Generalizationsfrom five years of research on assessment anddecision making: The University of MinnesotaInstitute. Exceptional Education quarterly, ,75—93.206APPENDIX ATABLE A.l - Squared multiple correlations of eachindependent variable with all otherindependent variablesTABLE A.2 - Squared multiple correlations of eachdependent variable with all otherdependent variables207208APPENDIX ASquared multiple correlations of each independent variablewith all other independent variables can be found in Table1.TABLE A.lCode R-SpuaredRATPASAV .40SPDPASAV .82WORDPM .73WORDSC 49KTEADSS .78KTEACSS .76Squared multiple correlations of each dependent variablewith all other dependent variables can be found in Table 2.TABLE A.2Code R-SguaredTRATAV .65TRATQ2 .79PROGRAM .83DISTEST .54II**<•t’tjiJ01HHI-HIi00001111111111Wtnt’)01CnWWHCDrtrtOUICDCD1(D(flHI-H--‘iaeft11CD01MHODCD—0O0OWWI.3WWOI-’Hw••••••••••••••••••••••••••01WWOO0W0WW00WOW0000WW-I-.I0WW%)00-J-I0ft CDbCDWL.3WI.)IJ1U1WI.)WWHHMI’.)HHU)(DC)OO)JO)O)O)MMGOO)0OU1WMWO)MM00MM0)0)rtPlO••••••••••••••••••••••••CD01I-I-<(DOU1WM-.)0)P)•HwU)oCDft01rtODl•••••••••••••••IMUIHMI.)HCDII.CDHUi0.0)G14-I.)14l.D0)HHP1-3IIO’131140P414H14HH14IHZ01CD(D•PlUI10Ct-C)PUIwoUI1IIIIIIIIIHHHP4HIH141’.)HIftftCD•••••••••.ai—aO.Ui%JH—10.0iI.)14H014.POUirtf%JI.3a0)Ui‘O0i00)0)OUiPH—.10)CDHHU)CDft 0114141414I’.)14r)I’)I’.)I’.)14Is)ftD.DW‘.0‘.0‘.0‘.0‘.0‘.0‘.0‘.0UI I-a.a UI14•a••••I•••a••I•••aHH1Is)0HU)HI.)H000)00HUiUI-3I’)-J14C7UIH0iUi0)00)3Ui0(.4I.)a-1010I-i.‘.0HAPPENDIX CPrincipal Permission Letter210211The University of British ColumbiaThe Education ClinicFaculty of Education2125 Main MailVancouver, British ColumbiaV6T 1Z5October 16, 1989Dear____________________Re: Study of Curriculum - Based and Standardized Tests of Reading AchievementThis is to request your permission to allow a few Year Four students in yourschool to particionte in a research project which is planned for January, 1990 in SchoolDistrict No. This project has been approved by the school district.The purpose of the project is to compare different kinds of reading tests with•school estimates of reading skill. In this way we may determine the best way to measurereading skill so that we can make correct decisions about programs for our students.Each child will individually be given a norm—referenced end a curriculum —based measure of reading skill. Tes-ting w-iIl take approximately one hour. Ift addition tothis, teachers of selected students will be asked to complete a simple rating scale ofreading achievement. It is estimated that this would take five minutes of their time. Inorder to compare reading measures with school — based indices of reading achievement,we will need the first — term scores from the District Grade Four CoreMastery Test (Reading subtests) which we understand from Ms. - IntermediateCo—ordinator, are routinely given in November and June.Testing will be done by Counsellors and by Masters and Doctoralstudents from U.B.C. Results will be confidential end parental and student permissionwill be obtained before anj testing is done.Resource Room and Skill Development Program students who are registered inYear Four and have reading difficulty have been selected as subjects for the study.These students will be matched by two other students in the same class andof the samegender who do not have reading difficulty. The Resource Room or Skill DevelopmentProgram students in your school who have been selected are listed on the attachedpage. Matching of the “regular” students will be done once principal end parentalpermission for the learning disabled students is obtained.We would be pleased to answer any question you may have regarding theproject. Please contact us at either of the telephone numbers or addresses below.2212Please return the attached Permission Form toArea Counsellor, at the Student Services Department,Sincerely,Thank you for your consideration of this project.School Board Office.Dr. Julianne Conry, Ph.D.Dept. of Educational Psychologyand Special EducationUniversity of British ColumbiaVancouver, British ColumbiaTelephone: 228—5384Rita Dunn, M.Ed.former Area Counsellor2676 West 45th Avenue,Vancouver, British ColumbiaV6N 3L3Telephone:Message:266—7323228—5384Signature(s) removed to protect privacyAPPENDIX DParent Permission Letter213214The University of British ColumbiaThe Education ClinicFaculty of Education1.25 Main MaUVancouver, British ColumbiaV6T 1Z5November 1989Dear Parent/Guardian:Re: Study of Curriculum — Based and Standardized Tests of Reading AchievementThis is to request your permission to aflow your child,_____________________to participate in a research project which is planned for January, 1990 in School DistrictNo. This project has been approved by the school district.The purpose of the project is to compare different kinds of reading tests withschool estimates of reading skill. In this way we may determine the best way to measurereading skill so that we can make correct decisions about programs for our students.The tests we will be using involve oral reading skill and readingcomprehension. Based on past experience, it has been found that- children enjoy workingwith the test materials.Approximately one hundred and ten Year Four students in School District No.will be selected for testing which requires approximately one hour intotal.The results of the project will be used for research purposes. Your child’sscores on the tests, therefore, will not become part of your child’s record. In the eventthat the tests do indicate sgme specific difficulties, we would contact you to ask forpermission to consult with your child’s school and discuss the possibilities of additionaltesting.We would be pleased to answer any questions you may have regarding theproject. Please contact us at either of the telephone numbers or the address be’ow.It is important to note that your child’s participation in this project is completelyvoluntary. If you decide that your child should not participate in the project, or wish towithdraw at any time, this decision will not affect your child’s progress or status inschool in any way.4.2215Please see the Parent Permission Form on the following page.Thank you for your consideration,Sincerely,Dr. Julianne Conry, Ph.D. Rita Dunn, M.Ed.Dept. of Educational Psychology former Area Counsellorand Special Education School DistrictUniversity of British ColumbiaVancouver, British ColumbiaTelephone: 228—5384 Telephone: 266—7323Message: 228—5384Signature(s) removed to protect privacya.216Parent Permission FormI do or do not (circle one) grant permission for my child to participate in thisproject, and I acknowledge receipt of a copy of the attached letter. I understand that mychild will be tested by a qualified examiner in my child’s school, and that my child’steacher will be asked to complete a brief rating form about his/her reading skilL I alsounderstand that my child’s individual results will be kept strictly confidential.I am this child’s parent or legal guardian and I am completing this form on mychild’s behalf.rr:(please print)_Signature:____________Relationship to child:Mdress:TelephonerAPPENDIX EParent Permission Letter- Attachment for the Learning Disabled Sample217218The University of British ColumbiaThe Education ClinicFaculty of Education2125 Main HailVancouver, British ColumbiaV6T 1Z5November , 1989.Dear________________Your child has been very carefully selected for this study because_____________is experiencing some reading difficulty at school and is getting extra helpwith reading. We are very concerned about finding better ways to measure reading skillwith children who have reading difficulty. We have only a small sample of children like____in Year Four in so it is very important to us that________be included in our study.The reading tests have been designed to avoid frustrating your child and webelieve that_________________will find the individual testing to be an enjoyableexperience. We have found that children with reading difficulty enjoy the individualadult &ttentiori a.n encoiragement that occurs in an assessment session.Please consider the attached request carefully. We would be very pleased toanswer any questions you may have about_________________‘sinvolvement in our study.Sincerely,Dr. Julienne Conry, Ph.D. Rita Dunn, M.Ed.Dept. of Educational Psychology former Area Counsellorand Special Education 2676 West 45th Avenue,University of British Columbia Vancouver, British ColumbiaVancouver, British Columbia- V6N 3L3228—5384 Telephone; 266—7323Message: 228—5384Teiephone:Signature(s) removed to protect privacyAPPENDIX FTesting Procedure Instructions219220Testing Procedure* all testing must be done within 6 weeks (January 15 -February 23, 1990)* please try to do all testing in January1. Read request for subject participation to child - getagreement to participate.2. Administer Words In Isolation Test (curriculum-basedmeasure) — you will need the score sheet, the word list,and the instructions — score the test3. Administer Reading Passages Test (curriculum—basedmeasure) — you will need the score sheets, the 10 readingpassages, and the instructions— score the test4. Administer Reading Decoding then Reading Comprehensionfrom the Kaufman Test of Educational Achievement—please complete standard scores using the Age norms-use Table 1 (Fall testing) if testing in January or Table3 (Spring testing) if testing in February5. Ask the student’s teacher to complete the Teacher RatingScale6. Ask the student’s teacher if the student has completedthe Core Mastery Test Grade 4 (Reading Section, Level 4A).Obtain the subtest scores and the total score (total=40marks). Record the scores on the student answer pages andplace the student’s name on the test. You have a sample ofthe test with instructions in your examiner package. Ifthe student has not completed the Reading section,administer this to any students in the school who have NOTcompleted the test and are in the study. Please coordinate this with the Area Counsellor in the school. Thisis a group administered test so it does NOT need to begiven individually. The resource Room or Skill DevelopmentProgram students, however, will find this difficult andwill need extra encouragement.7. Please return the student packages to the School BoardOffice. Please return them personally. A box will beavailable for the packages.You are finished! Thank—you so much for your support.Research like this would not be possible without you.APPENDIX GRequest for Subject Participation221222REQUEST FOR SUBJECT PARTICIPATION(to be read to each subject individuallyprior to testing)_____________________________,as you may know by now,you have been selected to take part in a researchproject to find out more about different ways oftesting student’s reading. When we finish I willsend these papers with your work to UBC. Yourname won’t be on them so nobody will know it wasyou — we only want to see how children answer thequestions, okay?I want you to remember that these tests have nothingto do with your schoolwork and will not count foryour grades on your report card. Most children enjoydoing the tests and I’m sure you will too. Before westart, I want you to know that you don’t have to do-this, but that your help is important-tous. I wouldappreciate it if you would agree to work on thesetests with me. Okay?APPENDIX HTeacher Rating Scale of Reading Skill(Photo—reduced for Presentation)223224TEACHER RATING SCALEREADING SKILLStudent’s be_______________________Grade TeacherRegular Class Resource RoaTi_________Skill Devetoent Progrwt_________________IN RATING YOUR STUDENTS, YOU ARE ASKED TO USE THE SEVEN-POINT SCALESHOWN BELOW WHICH HAS BEEN DESIGNED TO REFLECT NORMALLYDISTRIBUTED DIFFERENCES AMONG CHILDREN IN THIS SCHOOL DISTIUCT.Selow Average Above AverageLowest 4‘ Mle Highest20 3CZ 1ZOI 3 4 5 6 7PLEAS! CIRCLE ONE NUMBER BETWEEN I AND 7.lit ompaeIscm to other children you have observed in this ,chool district, and jainghe “normal curve” rating scale given b.Iow. pieaic rate this child’!:(a) Overall Reading Skill 1 2 3 4 S 5 7(bi Oral Reading fluency 1 2 3 4 5 5 7(.ed ai accuracy oforal reading skill)Cc) Reading Comprehension 1 2 3 4 5 6 1(reading understandingdemonstrated orally orIn writing)2. May. you ever considerei referring this child (or a psychoeducational assem.ntthat may lead to a specIal class placement to remediate reading deficiency’YesNo— I sometimes feel that this child tould be referred toe assessment ofreadi& difficultyAny comment?3. Hew many school grades has this child repea ted’____________(specify which ones If known:__________________________APPENDIX ICurriculum—Based Reading Passages TestAdministration and Scoring Instructions225226READING PASSAGES TEST - ADMINISTRATION AND SCORINGThese Reading Passages Test instructions will be read verbatim to the subject.When I say begin, you may begin reading out loud at the top ofthis page. Read across the page. (Demonstrate by pointing).Try every word. If you wait too long on a word, I will tell youthe word. Read the whole page. Do your best reading. Do youhave any questions? Ready? Begin.Start the stopwatch after you say “begin”. Supply a word only after thestudent has waited 5 seconds. Do not say the correct word after the student has said anincorrect word.For subsequent passages, the examiner says “Ready begin” and immediatelystarts timing.-If the test administrator is unclear about the correctness of a child’s response,it is acceptable to ask the child to read the word again after the entire passage has beenread. Obvious errors, however, are not to be reread. Mispronunciations are errors.Record errors as follows:Mark words read incorrectly (with or slash (I) or a circle). The following arerecorded as errors: Teacher supplied words, rnispronunciations, omissions, words read outof sequence (transpositions — score as one error), and substitutions.Repetitions are not errors.Self — corrections are not errors.Dialect characteristics are not errors.Insertions are not errors.Special considerations:1. If a child continues to make the same error, score all incidences as errors (Eg.“turtle” read incorrectly three times = 3 errors).22272. Proper names are not errors (Eg. Aslak, Polyphernus, Merj.a, Papa—san,Scituate, Penelope, Beckie etc.)However:Mr. = errorMrs. = errorDipper = error3. Spoiled responses are scored as errors.4. Score hyphenated words as one error (Eg. “able-bodied” is one error if readincorrectly).5. If one whole line is omitted, score each omitted word as an error.Each passage wiil yield two scores: (1) time needed to read the entire passage(in seconds); and (2) number of words read correctly in the entire passage.Discontinue Rule1. If the student requires 4 or more minutes (240 seconds) to read one passage,administer one more passage. If the student requires the same or an increasedamount of time to complete the second passage, discontinue testing. If thestudent requires less time to read the second passage, continue testing untilthe student requires 4 or more minutes to read one passage.APPENDIX JCurriculum—Based Reading Passages228229Passage #1 - Level 3-4Jill said, “Here, little turtle.I’ll help you.”Nan said, “Swim, little turtle.We want to see you swim.”“The turtle can swim!” said Ted.“It likes to swim here.”Jill said, “Don’t hide, turtle.I want to see you swim.I don’t want you to hide.”Nan said, “Will the turtle eat?It can swim here.But what will it eat?”“It wants to eat.Don’t you see, Nan?Don’t you see what it wants?” said Bill.“I can see,” said Nan.“I can see what the turtle wants.It can eat at the park.”Jill said, “Guess what! This is a park turtle.”Ted said. “It likes the park.And we can come here to see it.”230Passage #2 - Level 5They went on.And on the way, they met a rabbit.“Where are you going?”asked the rabbit.“We are going to find the king,”said Henny Penny.“We are going to tell himthat the sky is falling.A bit of it fell on my tail.”“May I go with you?”asked the rabbit.“Certainly,” said Henny Penny.On they went-down a hill—and under a bridge.And under the bridge they met a fox.The fox said to Henny Penny,“Where are you going?”“We are going to tell the kingthe sky is falling.A bit of it fell on my tail,”said Henny Penny.231Passage #3 - Level 6In the morning the shoemakercame down into his store. Therebefore him were seven new pairs of shoes.“How can this be?” the shoemakerasked the little old woman. “Last nightI had no leather to work with.Today I find seven new pairs of shoes!And they are well—made shoes too.How can this be?”That day people cameinto the shoemaker’s store. They liked thenew shoes. They gave the shoemaker money.Late that day the old man went outto get more leather to make more shoes.When he got back, he didn’t have timeto make the shoes.So the shoemaker went to bed.Passage #4 - Level 7“I wish I could help,” Mr. Sing said. “ButI can’t stop to catch a monkey. I have tocatch a bus. Why don’t you call the zoo? Ifhe belonged to me, I’d take him to the zooand leave him there.”Jill Brown came running down the street.“Did I hear that there’s a monkey loose inthat tree?” she said. “Let ME get himdown.”Up she went. But the monkey went higher.When she could almost reach him, themonkey swung up to the branch above.“This could go on all day,” Mrs. Bell said.Jill came back down the tree.232233Passa2e #5 - Level 8“I don’t think Charlie really wants to bea tramp,” said Mother.“Yes, I do,” said Charlie. “Tramps don’thave to learn how to chop down trees andhow to roll logs and how to build dams.“Tramps don’t have to practice swimmingand diving and holding their breath underwater.“Nobody looks to see if their teeth aresharp. Nobody looks to see if their fur isoiled.“Tramps carry sticks with little bundlestied to them. They sleep in a field when theweather is nice, and when it rains they sleepin a barn.”“Tramps just tramp around and have agood time. And when they want somethingto eat, they do little jobsfor anybody thatwants little jobs-I’Passage #6 - Level 9Soon the lasso was circling Bill’s head. It madea singing sound through the bright air. Bill keptadding more rope to its length. At last, with onegreat toss, Bill let it go. Up, up it went toward thestars of the Little Dipper. Sue and Bill waited. Itseemed a long time before the line suddenlytightened.“I’ve got it,” shouted Bill, “Now pull.”Pull they did, as hard as they could. Slowly thehandle of the Little Dipper began to turn. Bill andSue pulled even harder and the handle moved alittle more. All night they tugged and tugged at thelong rope.234235Passage #7 - Level 10And, although Penelope still snoozed beneath thebleeding hearts in the garden, she did not do so to getaway from Beckie and Abbie’s fifing and drumming.War was too close now. The girls knew that their fatherwould go to fight the redcoats as soon as war wasdeclared. They had no heart for music. The fife anddrum hung silently on the wall.And then one lovely summer day the long—expectednews arrived. War had been declared. Every able—bodiedman in Scituate responded to the call to the colors.When Mr. Bates broke the news to his family he toldthem they had to be brave soldiers.236Passage #8 - Level 11Grandfather looked up. His dark eyes were glowing.“I must go up the mountain. How many more yearswill I be able to climb? Would you have me the onlyaged one in all Japan who hasn’t seen the sunrise fromthe top of our beloved Fuji—san?”Fujio watched his mother and father look at eachother. Papa—san shrugged his shoulders. “HonoredFather, if you feel that strongly about it, you mustcertainly try to climb the mountain. With a strong stick,and Fujio to help you, you might reach the summit.”Fujio felt his heart speed up. “Thank you, Papa—san.”237Passage #9 - Level 12Aslak did not stir until his uncle had been out ofsight a long time. Then as his strength returned, he foundhe could move. Painfully he crawled to the mossy patch,found his knife and cut the lasso from his neck.Standing, he discovered none of his bones broken, buthe was bruised from the fight and aching from hisclimb. He knew that sleep was the only healer closeat hand, so he wrapped himself in Merja’s dress,which his uncle had left behind, put his own valuablehat on his head, rolled under the shelter of the rock,and slept.Passage #10 - Level 13“Polyphemus then, groaning with pain, rolled awaythe stone and sat before the mouth of the cave withhis hands outstretched, thinking that he would catchus as we dashed out. I showed my companions how wemight pass by him. I laid hands on certain rams ofthe flock, and I lashed three of them together withsupple rods. Then on the middle ram I put a man ofmy company. Thus every three rams carried a man.As soon as the dawn had come, the rams hastened outto the pasture, and as they passed, Polyphemus laidhands on the first and the third of each three thatwent by. They passed out and Polyphemus did notguess that a ram that he did not touch carriedout a man.238APPENDIX KCurriculum—Based Reading PassagesScoring Form239240Passage #1 - Level 3-4Jill said, “Here, little turtle.I’ll help you.”Nan said, “Swim, little turtle.We want to see you swim.”“The turtle can swim!” said Ted.“It likes to swim here.”Jill said, “Don’t hide, turtle.I want to see you swim.I don’t want you to hide.”Nan said, “Will the turtle eat?It can swim here.But what will it eat?”“It wants to eat.Don’t you see, Nan?Don’t you see what it wants?” said Bill.“I can see,” said Nan.“I can see what the turtle wants.It can eat at the park.”Jill said, “Guess what! This is a park turtle.”Ted said. “It likes the park.And we can come here to see it.”time =__________secondsscore = 119— (errors)241Passage #2 - Level 5They went on.And on the way, they met a rabbit.“Where are you going?”asked the rabbit.“We are going to find the king,”said Henny Penny.“We are going to tell himthat the sky is falling.A bit of it fell on my tail.”“May I go with you?”asked the rabbit.“Certainly,” said Henny Penny.On they went-down a hill-and under a bridge.And under the bridge they met a fox.The fox said to Henny Penny,“Where are you going?”“We are going to tell the kingthe sky is falling.A bit of it fell on my tail,”said Henny Penny.time =__________secondsscore = 109— (errors)242Passaae #3 - Level 6In the morning the shoemakercame down into his store. Therebefore him were seven new pairs of shoes.“How can this be?” the shoemakerasked the little old woman. “Last nightI had no leather to work with.Today I find seven new pairs of shoes!And they are well—made shoes too.How can this be?”That day people cameinto the shoemaker’s store. They liked thenew shoes. They gave the shoemaker money.Late that day the old man went outto get more leather to make more shoes.When he got back, he didn’t have timeto make the shoes.So the shoemaker went to bed.time = secondsscore = 110— (errors)243Passage #4- Level 7“I wish I could help,” Mr. Sing said. “ButI can’t stop to catch a monkey. I have tocatch a bus. Why don’t you call the zoo? Ifhe belonged to me, I’d take him to the zooand leave him there.”Jill Brown came running down the street.“Did I hear that there’s a monkey loose inthat tree?” she said. “Let ME get himdown.”Up she went. But the monkey went higher.When she could almost reach him, themonkey swung up to the branch above.“This could go on all day,” Mrs. Bell said.Jill came back down the tree.time = secondsscore = 105— (errors)244Passage #5 - Level 8“I don’t think Charlie really wants to bea tramp,” said Mother.“Yes, I do,” said Charlie. “Tramps don’thave to learn how to chop down trees andhow to roll logs and how to build dams.“Tramps don’t have to practice swimmingand diving and holding their breath underwater.“Nobody looks to see if their teeth aresharp. Nobody looks to see if their fur isoiled.“Tramps carry sticks with little bundlestied to them. They sleep in a field when theweather is nice, and when it rains they sleepin a barn.”“Tramps just tramp around and have agood time. And when they want somethingto eat, they do little jobsfor anybody thatwants little jobsdone.”time =__________secondsscore = 124— (errors)245Passage #6 - Level 9Soon the lasso was circling Bill’s head. It madea singing sound through the bright air. Bill keptadding more rope to its length. At last, with onegreat toss, Bill let it go. Up, up it went toward thestars of the Little Dipper. Sue and Bill waited. Itseemed a long time before the line suddenlytightened.“I’ve got it,” shouted Bill, “Now pull.”Pull they did, as hard as they could. Slowly thehandle of the Little Dipper began to turn. Bill andSue pulled even harder and the handle moved alittle more. All night they tugged and tugged at thelong rope.time =__________secondsscore = 107— (errors)246Passage #7 - Level 10And, although Penelope still snoozed beneath thebleeding hearts in the garden, she did not do so to getaway from Beckie and Abbie’s fifing and drumming.War was too close now. The girls knew that their fatherwould go to fight the redcoats as soon as war wasdeclared. They had no heart for music. The fife anddrum hung silently on the wall.And then one lovely summer day the long—expectednews arrived. War had been declared. Every able—bodiedman in Scituate responded to the call to the colors.When Mr. Bates broke the news to his family he toldthem they had to be brave soldiers.time =__________secondsscore =- ill— (errors)247Passage #8 - Level 11Grandfather looked up. His dark eyes were glowing.“I must go up the mountain. How many more yearswill I be able to climb? Would you have me the onlyaged one in all Japan who hasn’t seen the sunrise fromthe top of our beloved Fuji—san?”Fujio watched his mother and father look at eachother. Papa—san shrugged his shoulders,. “HonoredFather, if you feel that strongly about it, you mustcertainly try to climb the mountain. With a strong stick,and Fujio to help you, you might reach the summit.”Fujio felt his heart speed up. “Thank you, Papa—san.”time =__________secondsscore = 101(errors)248Passage #9 - Level 12Aslak did not stir until his uncle had been out of sighta long time. Then as his strength returned, he foundhe could move. Painfully he crawled to the mossy patch,found his knife and cut the lasso from his neck.Standing, he discovered none of his bones broken, buthe was bruised from the fight and aching from hisclimb. He knew that sleep was the only healer closeat hand, so he wrapped himself in Merja’s dress,which his uncle had left behind, put his own valuablehat on his head, rolled under the shelter of the rock,and slept.time =___________secondsscore = 103— (errors)249Passage #10 - Level 13“Polyphemus then, groaning with pain, rolled away thestone and sat before the mouth of the cave with his handsoutstretched, thinking that he would catch us as we dashedout. I showed my companions how we might pass by him.I laid hands on certain rams of the flock, and I lashedthree of them together with supple rods. Then on themiddle ram I put a man of my company. Thus every threerams carried a man. As soon as the dawn had come, the ramshastened out to the pasture, and as they passed, Polyphemuslaid hands on the first and the third of each three thatwent by. They passed out and Polyphemus did not guess thata ram that he did not touch carried out a man.time =__________secondsscore = 132— (errors)APPENDIX LCurriculum-Based Word List Administrationand Scoring Instructions250251WORDS IN ISOLATION TEST - ADMINISTRATION AND SCORINGThese Words in Isolation test instructions will be read verbatim to the subject.Here is a word list that I want you to read. When I tell you tostart, you can read down the page. Please read as fast and ascarefully as you can. If you get stuck on any of the words, moveon to the next one. Do you have questions? Ready? Begin.Then the word list will be given to the child and a stopwatch will be triggeredfor a sixty second timing. The test administrator will mark whether each word Iscorrectly read on follow—along sheets that are identical to the word list itself. If thechild falls to respond after an interval of approximately 5 seconds, the test administratorwill urge the child to move on to the next word. The test administrator will place a markafter the last word read in the sixty second interval and allow the child to continuereading. Responses must be completely accurate to be scored as correct.If the test administrator is unclear about the correctness of a child’s response,it is acceptable to ask the child to read the word again after the entire word list hasbeen read. Obvious errors, however, are not to be reread. Mispronunciations are errors.For the sixty second timing, if the child has started but not completed a wordin the sixty second interval, include the word in the score.Each word lIst will yield two scores: (1) number of words read in one minute(correct arid incorrect words); and (2) number of words read correctly in the entire wordlist.APPENDIX MCurriculum-Based Word List252WORD LIST 253country floppy din civichead headquarters envy geometryhungry merchant midst immensitysang owe receiver portholesea rip unaware unprotectedsoon war wary yiptire yum amid adjacentwind blush anticipation ascertainanyway comfortably basketful bartercrash extend circuit bredexcept habit generosity brusqueheld inventor hibernate horsemenplain protection political minutesrode squat surprisingly prospectorsomehow trouser appliance spentbadger attraction fake brightnessAPPENDIX NCurriculum-Based Word ListScoring Form254255WORD LISTcountry floppy din civichead headquarters envy geometryhungry merchant midst immensitysang owe receiver portholesea rip unaware unprotectedsoon war wary yiptire yuin amid adjacentwind blush anticipation ascertainanyway comfortably basketful bartercrash extend circuit bredexcept habit generosity brusqueheld inventor hibernate horsemenplain protection political minutesrode squat surprisingly prospectorsomehow trouser appliance spentbadger attraction fake brightnesswords per minute(correct and incorrect)score = 64(errors)APPENDIX 0District Test Reading Passages(Photo—reduced for presentation)256LeveL 4A 257Reading Section I— FictionSuddenly a loud noise awakened Bob.. He lay stilland listened. Except for the quiet breathing of theother boys as they slept, the cabin was quiet. Outsidehe could hear nothing except the hum of the wind throughthe pine trees.Earlier that evening Bob, and his friends had sataround the fireplace listening to Mr. Fletcher tellstories about Big Bear. According to legend, Big Bearwas huge. Usually he. stayed high up in the mountains,but sothetimes he wandered dcwn into the foothills. Somepeople said that Big Bear came down into the foothillsto look for food. Other people said that the bear thoughtthe foothills belonged to him and that he didn’t likecampers using them.There.! The noise came again. This time Bob wassure it came from outside the cabin. Bob quietly climbeddown from his bunk and tiptoed to the door. Maybe hewould be one of the few people to catch a glimpse of BigBear.Cautiously Bob opened the cabin door. He steppedout into the cold night air and stood on the Porch. Ther.c•ise was coming from the north side of the cabin, wherethe garbage cans were. As Bob started across the porch,a board creaked under his weight. He froze. The noiseat the side of the cabin stopped, and the night air wassilent for a moment. Then Bob thought he heard somethingmove off into the bushes. A little frightened, Bobslipped back into the cabir.The next morning when Bob told the other boys whathe had heard, they teased him. They said he had dreamedit. After all, they hadn’t heard anything.Later, they all went outside to check. Somethinghad scattered the garbage, all right. The garbage cansc.ere tipped over, and one of them had a big dent in it.“Could a raccoon do that?” Bob wondered. Had the dentbeen in the garbage can before? Or did he just missseeing BigBear?Go to Student Pages 1(a) & 1(b)258Level-4AReading Section IL — Non—FictionWould you like to know how to send secret messagesto your friends? There is an easy way to do this. Allyou need is ink that cannot be seen after it dries.Only your friends will know how to make your invisibleink appear.You will need the following things to write asecret message: a juicy lemon, a pen with a clean pointor a fine paint brush, and a sheet of plain white paper.The first thing to do is cut the lemon in half aridsqueeze the juice into a saucer. Now dip the penpointinto the lemon juice, which serves as the invisible ink.Write your secret message. When the lemon juice dries,the words will be invisible. No one will be able to see- what you have written until -the scet is used tQmake the ink visible.After. a friend receives your secret message, thepaper must be held over a lighted electric bulb for afew minutes. Then the heat from the bulb will make thewords begin to appear on the paper. The message willlook as if i iere written in brown ink-. This is thesecret method of making invisible ink become visible.When you use invisible ink, you and your friendscan have fun sending hidden messages to one another.If others find your notes, they will not be able to seeany writing on the paper. They will wonder how you areable to read a message fom a blank sheet.Go to student Pages 2(a) & 2(b)APPENDIX PDistrict Test Student Form(photo—reduced for presentation)259260ealrzg Selection I Answer SheetLevel 4ACH?REHNStON 1 aarkeachne:1ine te coet aflsde.‘. What awakened SoS?a) a Loid noiseh) the hua of the wind throuçh te treesc) Hr. Fletcher $ veic.2. Where was ie ca5iit7a) hiçh in th. aeuntain.e5) in he foothiLlsc) in the city3. What aad. Bob think of Siq Bear when he heard the noise?a) The boys had teased Sob aSout being scared of Big Bear.5) The other Soya heard the noise.c3 4:. Fe:cher had to.4 stories about Big Sear ea:Li.ri the evenLn.ve one reason that the boys though: a anisa had Seensearching for food.y did: te soya hear what tob had heard?i. What is the best title for this story?a) A Caa;nq Trip5) BLg 3cc:c) CanpfLre Steies7. 4mS.r :e even:, of the story in the order they happened._____The boys went to bed.Kr. tLetche cold a story of Big Sear. :::—Bob went out onto the porch, eachtn th. •ornioq there was a dent in a garbage can.Name261Student Page #1(b)Reading Selection IL Answer SheetLevel 4AVOCA3Ut..AY!ob qui. yc.i=..d down froo his 3. •arkn this a.ntec. nk .ans: eacha) to hold underwaterb) a kind of .dci a lot of neoa.na.z. Cauttcust 3e opened tha cabin deer.Zn this sentence. cautiously a.ans:a) ca:efuUybi paInfullyci .atly3. iad ve cI&It b.en i th. qarhac. can b.for.?n t’js s.nt.nce d.’tt scans:a)j hole-b) rotten feedci a hallow cad. yhittlnç4. 4ayb. Sob would eatc a gtiose of Siq Sear.:n this 2.nt.nC. chaps. .ans:a faLt hlçh:1 a quick leek:1 a shiny object5. .a:er t!ey a... w.tt oi.ta..d. to criek.n this sentence • eS.k scans:a) a bank papera sack to show se.e:hjnq is correctC) prow, it to be tru..URAZ.S‘:.ce th. plural for. of each word below.ezaepl.: !X!. ½ •ackeachI. story2. leaf4. key3. chiLd6. checkName_____262Student Page #2(a)ead1ng Selection II Answer SheetLeveI4Ai. Underline the best titl, for this article:a) )Iew to Tool Othersb) S.cre:.wassaqes tn Xgtvisible Znk 1 earkci How e Coeunjcate with Triends2. Urd.rline th. best answer:Th read he .ssaq. hold te piper over—a) an unliqht,d electric bulbb) a liqht.d electric bulb 1 aarkc) a le.en3 After you have written you: s.siaq. and the juice hasd:led, what will the paper look like?a) blank paperb3 a paper wIth yellow writinq 1 aarkc) a paper with brown writing4ber t. activities below in the order in which theyapp.ar.d in the article.— On pap.: writ, a e.ssaq. to your friend.____________Sque.z.the 3u1c. free a l.aon.2 marksCiv. the pap.: to your friend.C½eark____ ipyour paIn:Srush in the juice, each). Aswe: th. foowLnq in a good sentence;tr;.ain what sakes he words appear on th. paper.learkVCCA3U.ARYAnnv.s and Synonv.s.. Choos, the cor:.ct word and writ, it in the blank.ordinary ink,____________________butleson juic. ink is(vLsOle LnvLsLbte)when lisen ju_c. d:i.s it____________________butwhen it isheated it will_.(dlsaøpears reaoear)2. Cheese a word wIth the saMe •.aninq to replac. the underlinedword and circl, it.Write your aessaq. en plain paper. 1 sark(blank rlnted)This is th. secret •arhed to cak, invisibl, ink.(way essage)Name -Level 4AStudent Page #2(b)Reading SelectIon 11 Answer Sheet263WORD ATTACX — STht.A3XCAIXONFill in the blanks below usinç these words:visible iovisibl.I. Print aene—syllaLs word:_2. Print a three—syllable word:paperWORD A?ACX - ROOT WORDSWrit. the bass (or root) word on the linewritinq_______________________________inv1iible___________driesfor the foUoviaq½ mark.ahZANG SWOY- NO1t-TAWGi* a paaqraph from the story. Read it and fLnd somekey wocds. they have been written as notes for you.You will need the followinç thinqs to writ, aseOret essaqe: a juLcy lemon, a pen with aclean point or a fine paint brush, and a sheetof plaIn whit. paper.Sow find the key words for this paragraph and write them in Oeteform.Th, first thing to do is cut the lemon in half andsqueez. the juice into a saucer. Now dip the pen—point into the lemon juice, which serves as theinvisIble ink. Writ, your secret message. Whenthe leson juice dries, the words will b. invisible.No one will he able to see what you hav, writtenuntil the Ser.t method is used to alce the inkvisible.massage electric electricalsarksashJuicy leDorlpen or fine oaint brushoaer2. markeach264Student Page 2Cc)Reading Selection U—Study SkillsLevel 4Atiis?*ucTzoNsthi word.i inA.el .a.cf L gftdauLs.cawaywo:wo:dj. :qS.1.c: :.e words :ha: would b. found on .si dL::ionay paq.steinon. 1iç: .43I Load2. 3auce s.nds:and :ocq1mah*..dLnqrea1aLphab.ti:a1 order:8..esaaq.______uic.Le2oncithod3 arksqu..:.Na meAPPENDIX QDistrict Test Scoring Instructions265266TEACHR INSTRUCTIONS I Answer SheetPage 1(a) — the sequence question is 2 marks.It is either all right or wrong.Remind the student f the meanings of antonym0synonym and plural.______________Page 1(a)Reading Selection IComprehension1. a2. b3. c4. Garbage was scattered5 They were asleep6. Big Bear7. 21 (2 marks)34Page 1 (b)Vocabulary1. b2.. a3. C-4. b5. cPlurals1. stories 4. keys2. leaves 5. children (½ mark each)3. bushes 6. checksIDJ’a.JIi‘M(4fl‘-3i1%)r’aLfltWN-ain,‘aimig•m’-m••.•lomDIa0aOu,t,IHDI‘Ifta(iifthiftNPom-’miaNrI-wD,bbI’1INaNaJI-’1NrjeimHsamOrt‘oi-’aiLle00‘1flrt-.EUI”IflJLJt(i)rt4ibii:JUDI•1°’1)—ft1fl.-‘itla_I’aIu(IifttIaiIo’i”0II._).d.aIonxo.n’iiiO£C‘‘<I‘Iaa0rtzoWOftU0Ia<(4IiHrtJoB‘II’DD‘O(n‘I.DII-’i.’.HII?cI-’Ii-’oII0ImoacnnftHm’•DI.‘l?cImo(41-’1’•bIHitt(I)I-’i‘Or.aHI•(4iI(40(41.flIDIHiDirtii0’ait,rIZI-’ftILi.Iftn‘<aai’•i-’imIa•ctC)Ii’IIIflLlbr.1mitiLaC)ftIZP’ILflrt :3.IIiLai(41•N1JIc0rt(100’le100I••boeiImoI‘‘1‘1InftImir0ftQ<(ftIIDI16aftInai’a’<iniIDII-’miIiIQ.Om1(4F1I-’1E0•eifti-’.<.LT•1<DII-’Irtrt‘<DIDI(a(aII-’IHfl’ioe-’bmrtIVDIi1iP’0i-aDI,ro’u16b’I’II‘ift‘1<a,C)Im’n,’i-’I’<IO,aiuab’aj>IIO’l:3B1II‘Ia0Ija(4.0I(6‘I(6(40iILa)‘DftImI0CI),r.I-ICtmi’‘IUftftDI(4‘I0rima DI(‘3H DI II—3?c.

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0054600/manifest

Comment

Related Items