UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Utilizing decision matrices to validate kindergarten screening measures Peach, Connie J. 1992

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1992_fall_peach_connie_j.pdf [ 4.44MB ]
Metadata
JSON: 831-1.0098885.json
JSON-LD: 831-1.0098885-ld.json
RDF/XML (Pretty): 831-1.0098885-rdf.xml
RDF/JSON: 831-1.0098885-rdf.json
Turtle: 831-1.0098885-turtle.txt
N-Triples: 831-1.0098885-rdf-ntriples.txt
Original Record: 831-1.0098885-source.json
Full Text
831-1.0098885-fulltext.txt
Citation
831-1.0098885.ris

Full Text

UTILIZING D E C I S I O N M A T R I C E S T O V A L I D A T E K I N D E R G A R T E N S C R E E N I N G M E A S U R E S by C O N N I E J . P E A C H B . A . (Psych . ) , The Un ivers i ty of Br i t ish C o l u m b i a , 1 9 7 3 A T H E S I S S U B M I T T E D IN P A R T I A L F U L F I L M E N T O F T H E R E Q U I R E M E N T S FOR T H E D E G R E E O F M A S T E R O F A R T S in T H E F A C U L T Y O F G R A D U A T E S T U D I E S Depar tment of Educat iona l P s y c h o l o g y and Spec ia l Educa t ion W e accep t this thes is as con fo rm ing to the required s tandard T H E U N I V E R S I T Y O F BRITISH C O L U M B I A A u g u s t , 1 9 9 2 ® C O N N I E J . P E A C H , 1 9 9 2 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that pemriission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of - ^ 9 The University of British Columbia Vancouver, Canada Date / . - / ^ ^ ^ / ^ DE-6 (2/88) A B S T R A C T The early ident i f icat ion of s tudents at - r isk for future learning p rob lems typ ica l ly fo rms the bas is for the Implementat ion of ear ly in tervent ion p rograms des igned to prevent , d im in ish , and /or co r rec t learning d i f f icu l t ies. K indergar ten sc reen ing resul ts in f luence the a l locat ion of spec ia l se rv i ces and are l inked w i th the expendi ture of moneta ry and personne l resou rces . The ex ten t to w h i c h sc reen ing cont r ibu tes to accura te and usefu l educa t iona l dec i s ion -mak ing requires eva lua t ion . The purpose of the present s tudy w a s to invest igate the val id i ty and uti l i ty of four k indergar ten sc reen ing measures and their compos i t e sc reen ing c lass i f i ca t ion as pred ic tors of third grade ach ievemen t . H is to ry of s c h o o l - b a s e d in tervent ion and retent ion s ta tus we re cons ide red to be addi t iona l ind ices of s c h o o l per fo rmance and their re lat ionship w i th k indergar ten sc reen ing resul ts w a s a lso inves t iga ted . T h e sc reen ing measures inc luded the D r a w - A - P e r s o n , the K indergar ten Language Sc reen ing Tes t , the Mann-Su i t e r V i s u a l M o t o r S c r e e n , and the Devere l l Tes t of Let ters and Numbers . The ach ievemen t measure e m p l o y e d w a s the C a n a d i a n Tes t s of Bas ic Sk i l l s . Va l id i t y da ta ind icat ing the degree of a c c u r a c y of sc reen ing c lass i f i ca t ion dec i s ions (r isk/no-r isk) w a s poss ib le th rough the ut i l izat ion of dec is ion matr ix ana lys is . Interpretat ions in this s tudy inc luded percentage ca lcu la t ions of the p rob lem base rate, referral rate, and overal l hit rate. Ver t i ca l eva lua t ion presents pred ic t ion a c c u r a c y in relat ion to cr i ter ion (actual) pe r fo rmance ve rsus hor izonta l eva lua t ion w h i c h is ca lcu la ted in relat ion to sc reen ing (predicted) pe r fo rmance . Pred ic t ion -per fo rmance matr ices p resented in this s tudy represent da ta avai lable for one age co l io r t of 6 8 4 sub jec ts enrol led s ince k indergar ten in one s c h o o l d is t r ic t located near V a n c o u v e r , Br i t ish C o l u m b i a . S e v e n ach ieved s a m p l e s were genera ted , the number of sub jec ts rang ing f rom 5 7 6 - 6 6 3 . T h e resul ts of this s tudy demons t ra te that sc reen ing referral rates w e r e less than their respec t i ve prob lem base ra tes, ind icat ing general under-referral of at-risk s tuden ts . For all ana l yses , ver t ica l eva lua t ion more appropr ia te ly demons t ra ted greater under-referra l rates than did hor izonta l eva lua t ion . Ver t i ca l eva lua t ion a lso con t r ibu ted to greater a c c u r a c y of interpretat ion than did hor izonta l eva lua t ion w h i c h p roved to be mis lead ing . Spec i f i c i t y rates (vert ical ly ca lcu la ted true negat ives) w e r e m u c h larger than sens i t i v i ty rates (vert ical ly ca lcu la ted t rue pos i t i ves) , ind icat ing far greater a c c u r a c y for the ident i f icat ion of non-r isk than at-r isk s tuden ts . Overa l l hit rates w e r e h igh but mis lead ing as the propor t ions of cor rec t ly ident i f ied at-r isk and non-r isk s tuden ts we re not ind ica ted . TABLE OF CONTENTS A B S T R A C T ii T A B L E O F C O N T E N T S iv LIST O F T A B L E S v i LIST O F F I G U R E S vi l A C K N O W L E D G E M E N T S vi i i I. I N T R O D U C T I O N 1 A . B A C K G R O U N D O F T H E P R O B L E M 1 B. R E S E A R C H C O N T E X T 3 C . S T A T E M E N T O F T H E P R O B L E M 4 D. DEFINITION O F T E R M S 5 E. L I M I T A T I O N S O F T H E S T U D Y 6 II. R E V I E W O F T H E L I T E R A T U R E 7 A . I N T R O D U C T I O N 7 B. E A R L Y IDENTIF ICATION O F RISK 7 1. Def in i t ion of Risk 8 2 . P reva lence of Excep t iona l Ch i ld ren 11 3 . C o n c e p t and Purpose of Ear ly Ident i f icat ion 1 4 4 . Ear ly Ident i f icat ion Prac t ice 15 5 . Ear ly Intervent ion Link 1 7 6. Issues in Ear ly Ident i f icat ion 2 0 7. S u m m a r y 21 C . T H E S C R E E N I N G C O M P O N E N T 2 2 1. Def in i t ion and Purpose of Sc reen ing 2 4 2 . The Sc reen ing P rocess 2 5 a . Procedura l Cons ide ra t ions 2 5 b. Sc reen ing A p p r o a c h e s 2 7 c . Sc reen ing Con ten t 3 0 d . Read iness Tes ts ve rsus Deve lopmen ta l Sc reen ing T e s t s 3 3 3 . Sc reen ing Benef i ts 3 4 4 . Sc reen ing C o n c e r n s 3 6 a . Genera l C o n c e r n s 3 6 b. Va l ida t ion M o d e l s 3 7 D. D E C I S I O N M A T R I C E S 3 8 1. Descr ip t ion and Interpretive Formulas 3 8 2 . S u m m a r y 4 2 E. S U M M A R Y 4 3 III. M E T H O D O L O G Y 4 4 A . I N T R O D U C T I O N 4 4 B. T H E E A R L Y IDENTIF ICATION P R O G R A M 4 4 C . S A M P L E 4 6 D. I N S T R U M E N T A T I O N 4 7 1. Pred ic t ive M e a s u r e s 4 8 a . D r a w - A - P e r s o n 4 8 b. K indergar ten Language Sc reen ing Tes t 5 0 c . Mann -Su i t e r V i s u a l M o t o r S c r e e n 5 3 d . The Devere l l Tes t of Let ters and Numbers 5 5 2 . Cr i ter ion M e a s u r e s 5 6 E. P R O C E D U R E F O R D A T A C O L L E C T I O N A N D C O D I N G 5 8 F. D A T A A N A L Y S I S 6 0 IV. R E S U L T S 6 2 A . I N T R O D U C T I O N 6 2 B. A C H I E V E D S A M P L E S 6 2 C . R E S U L T S 6 3 1. Pred ic t ive A c c u r a c y of Individual K indergar ten Sc reen ing M e a s u r e s 6 4 a . D r a w - A - P e r s o n 6 4 b. K indergar ten Language Sc reen ing Tes t 6 6 c . Mann -Su i t e r V i s u a l M o t o r S c r e e n 6 8 d . Devere l l Tes t of Let ters and N u m b e r s 7 0 2 . Pred ic t ive A c c u r a c y of the C o m p o s i t e Sc reen ing C lass i f i ca t ion 7 2 a . Sc reen ing C o m p o s i t e per C T B S 7 2 b. Sc reen ing C o m p o s i t e per H is to ry of Intervent ion . . . 7 4 c . Sc reen ing C o m p o s i t e per Retent ion S ta tus 7 6 3 . S u m m a r y of Resu l ts 7 8 D. S U M M A R Y 8 2 V . S U M M A R Y A N D C O N C L U S I O N S 8 3 A . S U M M A R Y 8 3 B. D I S C U S S I O N 8 4 a . Pred ic t i ve A c c u r a c y of Indiv idual Sc reen ing M e a s u r e s 8 7 b. Pred ic t ive A c c u r a c y of the C o m p o s i t e Sc reen ing C lass i f i ca t ion 9 0 C . L I M I T A T I O N S O F T H E S T U D Y 9 5 D. I M P L I C A T I O N S A N D C O N C L U S I O N S 9 6 E. R E C O M M E N D A T I O N S F O R F U T U R E R E S E A R C H 9 8 R E F E R E N C E S 101 LIST OF TABLES Table 1. Number of Sub jec ts in S e v e n A c h i e v e d Samp les 6 3 Tab le 2 . K indergar ten S c r e e n i n g : A c c u r a c y of Risk Ident i f icat ion for S c h o o l A c h i e v e m e n t , Intervent ion, and Retent ion Ut i l iz ing Dec is ion Mat r i x A n a l y s i s (Presented as Percentages) 7 9 Tab le 3 . K indergar ten S c r e e n i n g : A c c u r a c y of Risk Ident i f icat ion for S c h o o l A c h i e v e m e n t , Intervent ion, and Retent ion Ut i l iz ing Dec is ion Mat r i x A n a l y s i s (Presented in A b s o l u t e Numbers) 81 LIST OF FIGURES Figure 1. Dec is ion Mat r i x 3 9 Figure 2 . Nunnerical I l lustration of Dec is ion Mat r i x 41 Figure 3 . D r a w - A - P e r s o n : A c c u r a c y of Risk Ident i f icat ion per C T B S Per fo rmance Ut i l iz ing Dec is ion Mat r i x A n a l y s i s {n = 576) . . . . 6 5 Figure 4 . K indergar ten Language Sc reen ing Tes t : A c c u r a c y of Risk Ident i f icat ion per C T B S Per fo rmance Ut i l iz ing Dec is ion Mat r i x A n a l y s i s (n = 601) 6 7 Figure 5 . Mann -Su i t e r V i s u a l M o t o r S c r e e n : A c c u r a c y of Risk Ident i f icat ion per C T B S Per fo rmance Ut i l iz ing Dec is ion Mat r i x A n a l y s i s (n = 596) 6 9 Figure 6. Devere l l Tes t of Let ters and N u m b e r s : A c c u r a c y of Risk Ident i f icat ion per C T B S Per fo rmance Ut i l iz ing Dec is ion Mat r i x A n a l y s i s {n = 606) 71 Figure 7. C o m p o s i t e Sc reen ing C lass i f i ca t i on : A c c u r a c y of Risk Ident i f icat ion per C T B S Per fo rmance Ut i l iz ing Dec is ion Mat r i x A n a l y s i s {n = 592) 7 3 Figure 8 . C o m p o s i t e Sc reen ing C lass i f i ca t i on : A c c u r a c y of R isk Ident i f icat ion per H is to ry of Intervent ion Ut i l iz ing Dec is ion Mat r i x A n a l y s i s (n = 663 ) 7 5 Figure 9 . C o m p o s i t e Sc reen ing C lass i f i ca t i on : A c c u r a c y of R isk Ident i f icat ion per Retent ion S ta tus Ut i l iz ing Dec is ion Mat r i x A n a l y s i s (n = 663) 7 7 ACKNOWLEDGEMENTS I w i s h to exp ress my apprec ia t ion to the many people w h o have suppo r ted my ef for ts to comp le te th is documen t . I a m thank fu l for the encou ragemen t g i ven to me by m y fami ly members . In part icular , I a m thankfu l for the gif ts of pa t ience and coopera t i on of fered by my daughter B r o n w e n and my son T imo thy . I a m gratefu l for the gu idance of my commi t tee members . Dr. J a c q u e l y n Baker -Senne t t (chair). Dr. Don A l l i s o n , and Dr. S u z a n n e J a c o b s e n . M y acqua in tance w i th S u z a n n e J a c o b s e n is of longer durat ion and I great ly admire her p ro fess iona l and persona l a c c o m p l i s h m e n t s . I a l so ex tend my grat i tude to Dr. Ju l i e C o n r y and Dr. Dav id Ba teson for their con t r ibu t ions and to Dr . Laurie R icou of the Facu l ty of Gradua te S tud ies for his def in i t ive suppor t and recogni t ion of m y pers is tence and desire to s u c c e e d . F inanc ia l ass i s tance for a c a d e m i c s tudy w a s avai lable in part th rough the receipt o f the R ick H a n s e n " M a n - l n - M o t i o n " Fe l lowsh ip (Univers i ty o f Br i t ish Co lumb ia ) and the A l i c e E. W i l s o n Gran t (Canadian Federat ion of Un ivers i t y W o m e n ) . I. INTRODUCTION A . BACKGROUND OF THE PROBLEM The c o n c e p t of ear ly ident i f icat ion of s tuden ts at r isk for poss ib le learn ing prob lems represents the logic of prevent ion as o p p o s e d to the batt le for cu re (Hobbs, 1 9 7 5 ) . Ear ly r isk de tec t ion fo rms the bas is for the imp lement ion of in tervent ion p rograms des igned to prevent , d im in ish , and /or co r rec t learn ing di f f icul t ies (McLough l i n & L e w i s , 1 9 8 6 ) . Rela ted benef i ts are reported to inc lude reduct ion in spec ia l educa t i on p lacement , grade re tent ion, and s c h o o l d ropout rates (National A s s o c i a t i o n of S c h o o l Psycho log i s t s [ N A S P ] , 1 9 8 9 ) . Desp i te interest in the c o n c e p t of ear ly de tec t i on , Sa tz & F le tcher (1982) report that mos t ch i ld ren w i th reading and learning di f f icul t ies are not ident i f ied unti l the age of t en , o f ten fo l l ow ing years of a c a d e m i c fa i lure. Debate regard ing educa t iona l re form and spec ia l educa t ion se rv i ce de l ivery w o u l d benef i t f rom the cons idera t ion of prevent ion as a v iab le a l ternat ive to cur rent post- fa i lure de l ivery mode ls (P ianta , 1 9 9 0 ) . Resou rce l imi tat ions are current ly hard p ressed to meet the increas ing ly d iverse needs of the es t imated 1 0 - 1 5 % of Nor th A m e r i c a n s tuden ts present ly se rved by spec ia l educa t ion (Counc i l for Excep t iona l Ch i l d ren , 1 9 9 0 ; Neufe ld & S t e v e n s , 1 9 9 2 ; Pa l f rey , 1 9 8 1 ) . A s w e l l , 2 5 % of s tuden ts enter ing s c h o o l may demons t ra te s igns of deve lopmen ta l dev iance f rom w h i c h learning d i f f icu l t ies might result (Nor ton , 1979 ) . To address the p reva lence of s c h o o l d y s f u n c t i o n and fai lure th rough the use of prevent ive prac t ice may prov ide relief for the a c a d e m i c . emot iona l , s o c i a l , and monetary cos t s invo lved (Mercer , A lgozz ine & Tr i f i let t i , 1 9 8 8 ; J u d y , 1 9 8 6 ) . Pr imary prevent ion targets those g roups and indiv iduals not ye t ind icat ing the ex i s tence of any p rob lem. The p rocess of ear ly r isk ident i f icat ion w o u l d appear to be one of pr imary prevent ion . The c o n c e p t of " r i sk " requires c la r i f i ca t ion . Ea ton (cited in P lan ta , 1990) referred to fac tors of r isk as t hose charac te r i s t i cs assoc ia ted w i th a high potent ia l for the deve lopment of a target p rob lem, s u c h as s c h o o l fai lure. Cur ren t descr ip t ion of educat iona l r isk refers , in general t e rms , to those s tudents hav ing or l ikely to have di f f icul ty in s c h o o l . R isk is probabi l is t ic in nature and does not insure p rob lem o u t c o m e or imply causa t i on (Keogh & Becke r , 1 9 7 3 ) . It d o e s , h o w e v e r , require the se lec t ion of a target p rob lem to be ident i f ied and the a t tendant cr i ter ia w h i c h may indicate r isk. In the pub l ic s c h o o l set t ing the k indergar ten popula t ion has been mos t ex tens ive ly target ted for ear ly ident i f icat ion. A comprehens i ve detec t ion p rogram is c o m p r i s e d of var ious s tages , one of w h i c h invo lves the admin is t ra t ion of sc reen ing measu res (Cross , 1 9 7 7 ) . The sc reen ing c o m p o n e n t e n c o m p a s s e s a dec i s ion p r o c e s s w h i c h c lass i f ies s tuden ts as ei ther at-r isk or non-r isk (L ichtenste in & Ireton, 1 9 8 4 ) . It separa tes the popula t ion g lobal ly for poss ib le referral to the subsequen t s tage of d iagnos t i c a s s e s s m e n t , thereby prov id ing a necessa ry link to ear ly in te rvent ion . B. RESEARCH CONTEXT Resea rch ef for ts have a t tempted to deve lop e f fec t ive sc reen ing me thods and to determine val id pred ic tors of later s c h o o l pe r fo rmance . M a n y measures have been deve loped but f e w have been eva lua ted for sa t i s fac to ry levels o f rel iabi l i ty and pred ic t ive va l id i ty (B racken , 1 9 8 7 ; G r e e n w o o d , 1 9 8 1 ; J a c o b s e n , 1 9 9 0 ; L indsay & W e d e l l , 1 9 8 2 ) . R e v i e w s of research ind icate lack of cons is ten t ev idence w i th regard to the e f f i cacy of sc reen ing measures (Leach , 1 9 8 0 ; Horn & P a c k a r d , 1 9 8 5 ) . Suppor t for ear ly sc reen ing con t inues but it has been argued that sys tema t i c p lanning mus t a c c o m p a n y p rogram imp lementa t ion and ins t rument se lec t ion mus t be gu ided by accep tab le s tandards (Me ise ls , 1 9 8 4 ; F rankenbu rg , 1977 ) . B e c a u s e the a im of k indergar ten sc reen ing is to se lec t t hose s tuden ts w h o are potent ia l ly at-r isk for later learning p rob lems , va l id i ty data ind icat ing the predic t ive e f f i c iency of sc reen ing c lass i f i ca t ion dec i s ions are desi rable (L ich tens te in , 1 9 8 1 ) . The ex tent to w h i c h sc reen ing cont r ibu tes to accura te and usefu l dec i s ion -mak ing requires fur ther eva lua t ion . In par t icu lar , sc reen ing o u t c o m e may result in t w o t ypes of c lass i f i ca t ion errors: ident i f icat ion of r isk for adequate ly func t ion ing s tuden ts and non-ident i f icat ion of r isk for s tudents in need (C la rkson , 1 9 8 9 ) . T h e s e errors represent i naccura te , or fa lse , predic t ions as o p p o s e d to accu ra te pred ic t ions of t rue risk and true non-r isk c lass i f i ca t ions . T h e t radi t ional exp ress ion of predic t ive val id i ty in te rms of cor re la t ion coe f f i c ien ts has been cr i t ic ized w h e n appl ied to the va l ida t ion of sc reen ing dec i s ions (L ich tenste in , 1 9 8 1 ) . Because corre la t ional ana lyses ind icate the s t rength o f l inear re la t ionsh ips be tween ear ly predic t ion and later pe r fo rmance m e a s u r e s , the coef f i c ien t represents cong ruence of s co res w i thou t c lar i f icat ion of the degree to w h i c h c lass i f i ca t ion dec is ions (r isk/no-r isk) are val id (Schu l te & M o o r e , 1 9 8 8 ; W i l s o n & R e i c h m u t h , 1 9 8 5 ) . Est imat ing the predic t ive e f fec t i veness and dec i s i on val id i ty o f sc reen ing dev ices so le ly on the bas is of t radi t ional cor re la t iona l s ta t is t ics is thus cons ide red tenuous . M a n y invest igators have sugges ted that test va l id i ty inc reases in uti l i ty or pract ica l i ty w h e n presented in te rms of c lass i f i ca t iona l hit rates. T h e s e ind icate the f r equency of accu ra te and inaccura te pred ic t ions fo l l ow ing compar i son w i t h later s c h o o l pe r fo rmance (Barnes, 1 9 8 2 ; Harber , 1 9 8 1 a , 1 9 8 1 b ; L i ch tens te in , 1 9 8 4 ; M e e h l & R o s e n , 1 9 5 5 ; M e i s e l s , 1986 ) . Pred ic t ion-per fo rmance matr ices are c o m m o n representa t ions of th is mode l of eva lua t ion , a lso k n o w n as dec i s i on mat r i ces . T h e latter te rm is ind icat ive of the under ly ing intent ion of sc reen ing measures for mak ing ef f ic ient and usefu l educa t iona l dec i s i ons . M e s s i c k (1990) sugges t s that pred ic t ive uti l i ty refers to the relat ive benef i ts and c o s t s der ived f rom ut i l iz ing part icular tes ts in dec i s i on mak ing . Errors may be di f ferent ia l ly impor tant to dec i s ion makers in di f ferent se t t ings . C. STATEMENT OF THE PROBLEM The purpose of the present s t u d y is to invest igate the predic t ive val id i ty and uti l i ty of four ind iv idual k indergar ten sc reen ing measu res and their compos i t e sc reen ing sco re by determin ing the a c c u r a c y of the c lass i f i ca t ion dec i s i ons (r isk/no-r isk) th rough ut i l izat ion of dec i s ion mat r ices . The fo l l ow ing ques t ions are a d d r e s s e d : 1. H o w accura te l y does each k indergar ten sc reen ing measure ind iv idual ly predict s tuden ts at-r isk or not at-r isk for future s c h o o l d i f f icul t ies as measu red by th i rd-grade per fo rmance on a s tandard ized ach ievemen t tes t? 2 . H o w accura te l y does the compos i t e sc reen ing c lass i f i ca t ion predict s tuden ts at-r isk or not at-r isk for future s c h o o l d i f f icul t ies as measured by th i rd-grade per fo rmance o n a s tandard ized ach ievemen t tes t? 3 . H o w accura te ly does the compos i t e sc reen ing c lass i f i ca t ion predict s tuden ts at-r isk or not at-r isk for future s c h o o l d i f f icul t ies as ind icated by s c h o o l -based remedia l in tervent ion dur ing pr imary g rades? 4 . H o w accura te l y does the compos i t e sc reen ing c lass i f i ca t ion predict s tuden ts at-r isk or not at-r isk for future s c h o o l d i f f icul t ies as ind icated by retent ion of one or more years in a pr imary grade? D. DEFINITION OF TERMS The def in i t ion of key te rms emp loyed in this s t udy are as f o l l ows . Early identification refers to the ear ly se lec t ion of h igh-r isk s tuden ts w h o have , or are mos t l ikely to deve lop , p rob lems in learn ing. It p rov ides an impor tant link to ear ly in tervent ion se rv i ce . A comprehens i ve ident i f icat ion p rogram typ ica l l y inc ludes the s tages of case f i nd ing , sc reen ing , d iagnos is , and eva lua t ion . Screening is a brief, l ow -cos t , s ys tema t i c eva luat ion of a c a d e m i c ski l ls and abi l i t ies assoc i a t ed w i th s u c c e s s f u l learning in s c h o o l , i ts pu rpose is to ident i fy s tuden ts w h o are at-r isk for learning d i f f icu l t ies. At-risf( refers to the c lass i f i ca t ion (r isk/no-r isk) resul t ing f r om unsa t i s fac to ry pe r fo rmance on a k indergar ten sc reen ing measure or c o m b i n e d measu res . A s tudent w h o dif fers suf f ic ient ly f rom their peers is cons ide red to be at-r isk for learning di f f icul t ies and may require indiv idual remedia l in tervent ion. Intervention refers to indiv idual t reatment g iven outs ide the c l a s s r o o m for all cond i t i ons (eg. learning d isabi l i ty , s p e e c h and language dif f iculty) though t t o be interfer ing w i th a s tuden t ' s adequate progress in s c h o o l . In this s tudy , re tent ion in one or more of the pr imary grades is inc luded as one in tervent ion. E. LIMITATIONS OF THE STUDY T h e four sc reen ing measures emp loyed in th is s t udy were p re-se lec ted by the publ ic s c h o o l d istr ic t i nvo l ved . The cut-of f s c o r e s used to def ine sa t i s fac to ry and unsa t i s fac to ry per fo rmance w e r e a lso p rev ious ly des igna ted . The measu res may not be cons ide red to be the best pred ic tors o f r isk s ta tus . Add i t i ona l l y , a c a d e m i c o u t c o m e is l imited to th i rd-grade s tandard ized ach ievemen t tes t i ng , h is tory o f in tervent ion, and retent ion s ta tus . E f fec ts of in tervent ion, gender , age and the like are not cont ro l led for. T h e f ind ings of this s t udy are spec i f i c to the populat ion i nvo l ved . Genera l i za t ions o f in terpretat ions are cau t i oned . II. REVIEW OF THE LITERATURE A . INTRODUCTION The fo l l ow ing rev iew of the l i terature desc r ibes i ssues pert inent to the prac t ice o f ear ly ident i f icat ion of ch i ld ren cons ide red to be at-r isk for exper ienc ing learning p rob lems . Add i t iona l l y presented are def in i t iona l , p rocedura l , and techn ica l cons idera t ions of sc reen ing . The ut i l izat ion of dec i s ion matr ices to va l idate the predic t ive a c c u r a c y of sc reen ing measures is a lso d i s c u s s e d . B. EARLY IDENTIFICATION OF RISK Paren ts , educa to rs , psycho log i s t s , and other p ro fess iona ls all share c o n c e r n regard ing the large numbers of s tudents not progress ing we l l or fai l ing in s c h o o l . T o meet the needs of these ch i ld ren , ef for ts to deve lop methods of r isk ident i f icat ion in ear ly ch i l dhood have been w ide ly adop ted . S o m e measures are cons ide red to be more sys tema t i c , comprehens i ve and/or pred ic t ive than o thers . Wo l f enda le (1981) notes the occu r rence of this m o v e m e n t in the Uni ted K i n g d o m , Aus t ra l i a , Europe , C a n a d a , and the Uni ted S ta tes . T h e prac t ice of r isk ident i f icat ion a s s u m e s the subsequen t implementat ion of ear ly t reatment in tervent ion to al leviate present or future di f f icul t ies (Morgan , 1981 ). The var ious fac to rs and assump t i ons w h i c h underl ie s u c h prevent ive ef for ts in f luence the degree to w h i c h risk ident i f icat ion is appropr ia te and ef f ic ient for the popula t ion i nvo l ved . 1 . Definition of Risk T l ie def in i t ion of r isk l ias p roven to be one of c o n f u s i o n and c o n t r o v e r s y . T l ie roots of ident i f icat ion or ig inated in the f ield of med ic ine w h e r e c lear def in i t ion of o rgan ic e t io log ies is more poss ib le (Keogh & Becke r , 1 9 7 3 ) . Risk is ind ica ted by p resymp toma t i c s tages k n o w n to o c c u r before p rob lem matura t ion . Th is t ime interval , k n o w n as lead t ime, is cons ide red to increase the p rospec ts for the benef ic ia l e f fec ts of ear ly in tervent ion. In med ica l c a s e s , r isk general ly o c c u r s fo l l ow ing the exper ience of adve rse phys io log ica l or env i ronmenta l cond i t ions w h i c h are k n o w n to cause or be h igh ly corre lated w i th the appearance of future abnormal i t ies (Pe te rson , 1987 ) . K e o g h & Da ley (1983) v i e w the probabi l i ty of r isk related to phys io log ica l cond i t i ons to be far more cer ta in than that w h i c h is assoc ia ted w i th p s y c h o l o g i c a l , s o c i a l , or educat iona l p rob lems . The p resence of risk fac tors se rves as an ear ly wa rn ing s ignal and not as proof of inevi table d isorder . T j o s s e m (1976) dep ic ted three ca tegor ies of r isk. " E s t a b l i s h e d " risk refers to d iagnosed cond i t i ons , mos t of ten med i ca l , w i th k n o w n et io log ies and o u t c o m e s . " E n v i r o n m e n t a l " r isk deno tes those b io log ica l ly s o u n d indiv iduals w h o s e life s i tuat ions may threaten their we l l -be ing . " B i o l o g i c a l " r isk des igna tes those ch i ldren w h o s e ear ly deve lopmen ta l h istor ies s u g g e s t poss ib le p rob lem o u t c o m e s . These risk cond i t i ons f requent ly over lap , inc reas ing the probabi l i ty or intensi ty of impai rment . Ch i ld ren are cons ide red to be at-r isk w h e n their s ta tus dev ia tes suf f ic ient ly f rom s o m e s tandard of normal i ty (Peterson, 1 9 8 7 ) . Soc ia l va lues and ind iv idual pe rspec t i ve are highly in f luencia l in judging w h a t is cons ide red to be normal or dev ian t , thus comp l i ca t ing the issue of r isk def in i t ion. It is a rgued that mode ls of r isk ident i f icat ion deve loped for use w i th in one f ield of s t u d y are not necessar i l y appropr ia te for another (Keogh & Da ley , 1 9 8 3 ) . The e x p a n s i o n f rom heal th-related to psycho log i ca l and educat iona l c o n c e r n s revealed m a n y assump t i ons of the med ica l mode l to be incompat ib le w i th the nature of ch i ld deve lopmen t (L ichtenste in & Ireton, 1 9 8 4 ) . For examp le , the def ic i t or ientat ion a s s u m e d in med ic ine does not a l l ow for the de tec t ion of both s t rengths and w e a k n e s s e s of the deve lop ing ch i l d . Wh i l e the et io log ies of educa t iona l p rob lems canno t be as c lear ly es tab l i shed as phys io log ica l cond i t i ons , theor ies of human deve lopmen t and learning desc r ibe charac te r i s t i cs assoc ia ted w i th ch i ld ren w h o have the greatest d i f f icul ty learning in s c h o o l (Zeit l in, 1 9 7 6 ) . Descr ip t ion of the potent ia l ly u n s u c c e s s f u l learner o f ten fo l l ows f rom k n o w n charac te r is t i cs of the s u c c e s s f u l learner. The deve lopmen ta l mode l has domina ted ear ly ch i l dhood l i terature for m a n y years (Wendt , 1 9 7 8 ) . It e n c o m p a s s e s a sequent ia l p rogress ion of behav io rs , f rom s imple to c o m p l e x , d isp layed by mos t ch i ld ren over a per iod of t ime. A c c o r d i n g to s o m e theor ies , mas te ry of p roceed ing levels prec ludes a d v a n c e m e n t a long the con t i nuum (Gesel l Inst i tute, 1 9 8 3 ) . Initially, maturat ion w a s cons ide red to be the agent of c h a n g e for the emerg ing s tages . A more modern v i e w a l lowed for c h a n g e as a resul t o f soc ia l and s i tuat ional in f luences in the env i ronment . T h e nature-nur ture debate w a s modi f ied by interact ional theor is ts w h o a c k n o w l e d g e d a syn thes i s of both internal and external in f luences (Hobbs , 1975 ) . Samero f f (1975) fur ther pos i ted a t ransac t iona l v i e w w h e r e neither the chi ld nor the env i ronment remain a s ta t ic cont r ibu tor to deve lopmen t . Rather , e a c h c h a n g e s and is changed by the other in a con t inu ing ser ies w h i c h se rves to e m p h a s i z e the long- term, inferent ial cou rse of " r i sk " deve lopmen t and its chang ing def in i t ion ac ross t ime (Keogh & Da ley , 1 9 8 3 ) . Leve ls of deve lopmenta l func t ion ing are o f ten the f o c u s of ear ly r isk ident i f icat ion. B e c a u s e ch i ldren func t ion d i f ferent ly at va r ious points in t ime and in di f ferent c i r c u m s t a n c e s , def in i t ions of ear ly r isk shou ld ref lect t h e s e incons is tenc ies (L ichtenste in & Ireton, 1 9 8 4 ) . In p rac t i ce , r isk o f ten re f lec ts spec i f i c educa t iona l needs rather than deve lopmen ta l levels (Barnes, 1 9 8 2 ) , Add i t iona l l y , r isk may ref lect the s tandards of educa t i on and behav ior cons ide red sa t i s fac to ry in ind iv idual s choo l se t t ings . P ioneers in the f ield of a c a d e m i c risk ident i f icat ion inc lude J a n s k y and de H i rsch ( 1972 ) , T jossen (1976) , and Merce r et a l . ( 1988 ) . A t - r i sk s tuden ts are usual ly t h o s e cons ide red to dev ia te s ign i f icant ly f r om " n o r m a l " ranges (Paget & Nag le , 1 9 8 6 ) . Researchers present a w i d e var ie ty of descr ip t ions of r isk, for examp le : ch i ld ren w h o may require spec ia l educa t i on and related se rv i ces if t hey are to real ize their full human potent ia l (Hal lahan & K a u f f m a n , 1 9 8 2 ) ; ch i ld ren w h o demons t ra te cogn i t i ve delay ( A a r o n s o n , Phi l l ips, Ba r to lucc i , & A a r o n s o n (1978) ; ch i ld ren at risk for reading di f f icul t ies (Gray, 1988) ch i ld ren unready for s c h o o l ent ry (Bi l lman, 1 9 8 8 ) ; and ch i ldren though t to be learn ing d isab led (Badian & S e r w e r , 1 9 7 5 ) . A n even broader descr ip t ion of risl< can be found in educat iona l research l i terature. Th is e n c o m p a s s e s those s tudents at risk for demonst ra t ing an inabi l i ty to meet a c a d e m i c expec ta t i ons (Hil ls, 1987) and for exper ienc ing mild to modera te learning d i f f icu l t ies in their later s c h o o l career (Leach , 1 9 8 0 ) . Add i t i ona l l y , an externa l ly der ived def in i t ion of r isk results f rom s c o r e s obta ined th rough tes t ing dur ing ear ly ident i f icat ion p rocedures . A des igna ted cut-of f point b e c o m e s the determin ing fac tor in r isk c lass i f i ca t ion , a t echn i ca l def in i t ion al terable by s imp ly rais ing or lower ing the cut -o f f sco re (Badian, 1 9 8 6 ; E a v e s , Kenda l l & C r i c h t o n , 1 9 7 4 ; S t o n e , C u n d i c k , & S w a n s o n , 1 9 8 8 ) . Ra is ing the cut-of f s c o r e of a test used to predict r isk w o u l d increase the number of s tuden ts cons ide red to be at-r isk (Mess i ck , 1 9 9 0 ) . 2 . Prevalence of Exceptional Children Preva lence es t imates va ry acco rd ing to the spec i f i c cr i ter ia used to def ine excep t iona l i t y . C lass i f i ca t iona l fea tu res , spec i f i c i t y of f o c u s , and degree of impai rment m a y be inf luent ial in the determinat ion of numbers . Es t imates may a lso be a f fec ted by the popu la t ion and age- leve l to be ta rge t ted . Repor t ing for the C a n a d i a n C o u n c i l for Excep t iona l Ch i l d ren , Neufe ld & S t e v e n s (1992) s tate that 1 0 - 1 5 % of the total s c h o o l popu la t ion are es t imated to fall under the spec ia l educa t ion umbre l la . Sa tz and F le tcher (1982) indicate that approx ima te ly 1 0 - 1 5 % of Nor th A m e r i c a n s c h o o l ch i ld ren of at least average inte l l igence suf fer a c a d e m i c hand i caps . Nor ton (1979) and Zeit l in (1976) report that an es t imated 2 5 % of ch i ld ren at s c h o o l ent ry may d isp lay s igns of deve lopmen ta l dev ia t ions w h i c h cou ld lead to learning p rob lems . Rogo l sky , in 1 9 6 8 , sugges ted that 4 0 % w o u l d ev idence prob lems w i th learning or ad jus tment in the pr imary yea rs . The Uni ted S ta tes Depar tment of Educa t ion c i tes 1 9 8 5 f igures desc r ib ing percen tages of s tuden ts aged 3 th rough 21 ac tua l ly rece iv ing spec ia l educa t i on se rv i ces for d i f ferent hand icapp ing cond i t ions (Lerner, M a r d e l l - C z u d n o w s k i & G o l d e n b u r g , 1 9 8 7 ) . The mos t f requent ly repor ted ( 4 . 5 7 % of 1 0 . 8 9 % total) represents the ca tego ry of learning d isabi l i t ies. A n es t imate of 1 2 % of A m e r i c a n schoo l ch i ld ren exper ience menta l , emot iona l or phys ica l impa i rments w h i c h prove prob lemat ic to s u c c e s s f u l s c h o o l func t ion ing (Brewer & Kaka l i k , 1 9 7 9 ; P ian ta , 1 9 9 0 ) . G r a y (1988) refers to the Early Ch i l dhood A s s e s s m e n t Project at the Un ivers i ty of M i n n e s o t a w h i c h revealed t remendous var iabi l i ty b e t w e e n s c h o o l d is t r ic ts in the pe rcen tages of ch i ldren ident i f ied as hav ing a c a d e m i c p rob lems. Wi th in spec i f i c s c h o o l popu la t ions , W i l s o n and Re ichmuth ( 1985) c o n c l u d e that the propor t ion of p rob lem learners, or p rob lem base rate, m a y f luc tuate dramat ica l ly depend ing upon distr ic t charac ter is t i cs and the cr i ter ia e m p l o y e d for ident i f ica t ion. A base rate o f 3 to 5 % , for examp le , might appropr ia te ly def ine learning d isab led th i rd-graders . T e n to 2 0 % might def ine th i rd-graders w i th reading p rob lems. In c o m p a r i s o n , 2 5 % of pupi ls in the Uni ted S ta tes in 1 9 6 9 were reported to have s ign i f icant reading def ic ienc ies (Zeit l in, 1 9 7 6 ) . A pub l ica t ion o f the M in i s t r y of Educa t ion for the p rov ince of Bri t ish C o l u m b i a ind icates that up to 2 0 % of s tuden ts in four th , s e v e n t h , and tenth grades are reading at levels w h i c h sugges t that cu r r i cu lum goals are not being met (Educat iona l Innovat ion D iv i s ion , B. C . M in is t ry of E d u c a t i o n , 1 9 8 9 ) . A nat ional su rvey on adult l i teracy c o n d u c t e d in 1 9 8 9 by Sta t is t ics C a n d a d a (1991) revealed that on ly 6 9 % of B . C . adu l ts and 6 2 % of Canad ian adu l ts aged 1 6 - 6 9 have suf f ic ient reading ski l ls to deal w i t h mos t eve ryday requ i rements . A s accura te base rate f igures are typ ica l l y unavai lab le , Barnes (1982) has sugges ted the implementa t ion of regional and local p rograms that w o u l d f o l l o w ch i ld ren 's progress over severa l years f rom the point of ear ly ident i f icat ion sc reen ing . S u c h p rograms w o u l d prov ide normat ive in format ion on the number and t ypes of d i f f icul t ies that ch i ldren expe r i ence . The programs cou ld a lso p rove usefu l for eva luat ing the long- term a c c u r a c y of part icular sc reen ing measu res or p rograms. T h e en fo rcemen t of Bill 8 2 in the p rov ince of Ontar io required that s c h o o l d is t r ic ts ident i fy high-r isk s tuden ts in k indergar ten ( W a n c z y c k i , 1 9 8 3 ) . Wh i le there is no s imi lar mandate w i th in the p rov ince of Br i t ish C o l u m b i a , interest in ear ly ident i f icat ion ex i s t s . Cur ren t l y , bo th p rog ram des ign and expense are the responsib i l i ty of the indiv idual d is t r ic ts i nvo l ved . Ear ly ident i f icat ion of except iona l i t ies is a lso in a c c o r d a n c e w i th U . S . Pub l ic L a w 9 9 - 4 5 7 (1986) w h i c h makes p rov is ion for ear ly in tervent ion se rv i ces for hand i capped infants and todd lers as we l l as more c o m p r e h e n s i v e se rv i ces for t hose aged 3-5 (Lerner et a l . , 1 9 8 7 ) . O f immense impac t in the Un i ted S t a t e s , the enac tmen t of Pub l ic L a w 9 4 - 1 4 2 (1975) required that all hand i capped persons aged 3-21 (wi th cer ta in state restr ic t ions) rece ive free publ ic educa t ion at a level c o m m e n s u r a t e w i th their ind iv idual needs and in the least restr ic t ive set t ing ava i lab le . It spec i f ied that e a c h state nnay not coun t more than 1 2 % of ch i ldren aged 5 th rough 17 as h a n d i c a p p e d . Th is coun t is , there fore , inf luent ial in the deve lopment o f ident i f icat ion cr i ter ia and cut-of f s co res w h i c h may be arbitrar i ly increased or dec reased in order to attain required numbers rather than to detec t the ac tua l numbers of hand icapped learners (Salv ia & Y s s e l d y k e , 1 9 7 8 ) . 3 . C o n c e p t and Purpose of Early Ident i f icat ion T h e major i ty of educat iona l s y s t e m s have been react ive in nature, typ ica l ly imp lement ing eva lua t i ve , remedial or c o m p e n s a t o r y p rac t i ces after p rob lems have su r faced (Reyno lds , G u t k i n , El l iott , & Wi t t , 1 9 8 4 ) . Rather than seek ing proac t ive or p revent ive in tervent ion at the p rob lem source and before the point of c o n c e r n , s c h o o l personne l f requent ly react w h e n s tudent fai lure has b e c o m e ev ident or even c r i t i ca l . Desp i te the real i ty of this p rac t i ce , it is general ly accep ted that ear ly p rob lem de tec t ion inc reases the l ik l ihood of p reven t ion , co r rec t ion , or d im in ishment of p rob lem impac t w h e n appropr ia te ear ly t reatment is imp lemented (Mc lough l i n , 1 9 8 8 ) . C o m m e n c i n g in the Uni ted S ta tes in the 1 9 6 0 s and expand ing in ternat ional ly , cons iderab le effort has been expended on the deve lopmen t and app l i ca t ion of ear ly ident i f icat ion p rocedures (L indsay & W e d e l l , 1 9 8 2 ) . A l t h o u g h va ry ing in con ten t and spec i f i c intent, their c o m m o n purpose has been to de tec t ind iv idual ch i ld ren w h o are l ikely to exper ience learning d i f f icu l ty and to a t tempt p revent ion of s c h o o l fai lure and any at tendant negat ive e f fec ts (Leach , 1 9 8 0 ) . L inked w i t h the c o n c e p t of p reven t ion , ear ly se lec t ion of high-r isk s tuden ts is essent ia l ly a pred ic t ive ac t iv i ty , mak ing in ference f r om the present to the fu tu re . Keogh & Becke r (1973) descr ibe this ac t iv i t y as hypo thes iz ing rather t han con f i rm ing . The goa l is not to ascr ibe c a u s a t i o n , but to at tempt a c c u r a c y in ident i f icat ion. A s s i g n i n g risk c lass i f i ca t ion is t yp ica l l y a d i cho tomous p r o c e s s indicat ing ei ther the p resence or absence of r isk. Berk (1984) , Harber ( 1 9 8 1 a , 1981 b) and L ich tens te in (1981 ) s t ress the c o n c e p t of dec is ion -mak ing inherent in the c lass i f i ca t ion me thod and the util ity of se rv i ce dec i s ions resul t ing f rom the ach ieved level of c lass i f i ca t ion a c c u r a c y . S c h o o l s y s t e m s are faced w i th the ex t raord inary task of d is t ingu ish ing ear ly on in t ime those ch i ld ren w h o have , or are mos t l ikely to deve lop , p rob lems in learn ing. Behav io r and /or other p rob lems w h i c h might hinder cogn i t i ve , soc i a l , or emot iona l deve lopmen t are a lso of c o n c e r n . T h e prac t ice of ear ly ident i f icat ion p lays an impor tant role in improv ing serv ice to ch i ld ren in need . Ef fec t ive ef for ts e n c o m p a s s a ser ies of c o m p o n e n t s w h i c h compr i se a sound p rogram of ident i f ica t ion. 4. Early Identification Practice L a c k of cons i s ten t te rmino logy is a sou rce of c o n f u s i o n in the p rocess of ear ly ident i f i ca t ion. Pe te rson (1987) no tes the non-s tandard ized use and interpretat ion o f w o r d s s u c h as a s s e s s m e n t , s c r e e n i n g , eva lua t ion , ident i f ica t ion, and d iagnos i s . S u c h te rmino logy is o f ten ind iscr iminant ly app l i ed , w i thou t c la r i f i ca t ion , and w i th re ference to di f ferent c o n c e p t s . The va lue of d i s cuss i on and compar ison of results may also be threatened and possib ly mis leading or detr imental . T o address th is i ssue . C r o s s (1977) presented an o v e r v i e w of p rocedures cons ide red integral to a comprehens i ve and sequent ia l s y s t e m of ear ly ident i f i ca t ion. T h e y inc lude the s tages of case f i nd ing , sc reen ing , d iagnos is , and p rogram eva lua t ion . App l i cab le to educa t iona l and other goa ls , these p rocedures genera l ly c o n c u r w i th those of other exper ts in the field (Paget & Nag le , 1 9 8 6 ; B o e h m & S a n d b e r g , 1 9 8 2 ; Barnes , 1 9 8 2 ) . Casefinding is descr ibed as the p rocess of a c c e s s i n g cand ida tes for par t ic ipat ion in the subsequen t s tages of ident i f icat ion. Spec i f i c popu la t ions and ob jec t ives in f luence the act iv i t ies i nvo l ved . Infant and p reschoo l age-g roups require ou t reach ef for ts , wh i le k indergar ten ch i ld ren , for examp le , are readi ly loca ted in the s c h o o l se t t ing . Screening refers to the brief s t ra teg ies or measurement act iv i t ies used to ident i fy ch i ld ren w h o fall ou ts ide the normal range of expec ted deve lopmen t and are thus cons ide red at r isk for a spec i f ied p rob lem. C o n f u s i o n may result w h e n the te rms " s c r e e n i n g " and " iden t i f i ca t ion" are used in te rchangeab ly . The latter techn ica l l y refers to the entire p rocess of ear ly ident i f icat ion. Sc reen ing is one c o m p o n e n t of the ident i f icat ion p rocess and must be v i e w e d separate ly w i th in this larger f r a m e w o r k , part icular ly for interpret ive and eva luat ive pu rposes . Sc reen ing y ie lds on ly genera l in format ion in tended to ver i fy whe the r a chi ld shou ld p roceed for fur ther d iagnos t i c eva lua t ion . Diagnosis de te rmines the ex i s tence , nature and ex tent of a prob lem th rough in-depth ana lys is and syn thes i s of co l l ec ted data and in format ion . It may a lso be referred to as d iagnos t i c eva lua t ion , d iagnos t i c a s s e s s m e n t , or s imp ly a s s e s s m e n t ( N A S P , 1 9 8 9 ) . Included in Pe te r s on ' s (1987) p rocess mode l is an "educa t iona l a s s e s s m e n t " s tage w h i c h fo l l ows d iagnos i s . The purpose of educa t iona l a s s e s s e m e n t is to determine w h i c h spec i f i c ins t ruct ional p rograms shou ld be o f fe red . Th is co i nc i des w i th the general expec ta t i on that ident i f icat ion is in te rven t ion- focused and imp lemented on ly if coup led w i th se rv i ces for ch i ldren in need (Paget & Nag le , 1 9 8 6 ) . Program evaluation is the ongo ing appra isa l of a ch i l d ' s p rogress w i th in the in tervent ion p rog ram. In Pe te r son ' s mode l , th is ac t iv i ty is equa ted to the "pe r fo rmance mon i to r ing " s tage , f o l l owed by a p rogram eva lua t ion phase w h i c h refers to the a s s e s s m e n t of the in tervent ion p rogram itself. P rog ram eva lua t ion c a n a lso refer to a s s e s s m e n t o f the entire ear ly ident i f icat ion p r o c e s s . C lear l y , def in i t ion of te rms mus t be es tab l i shed for mean ingfu l d i s c u s s i o n . Ear ly ident i f icat ion is a comprehens i ve p rocess . S tages are not iso lated even t s , nor do t hey t hemse l ves cons i s t of s ing le even t s . A s w e l l , a p rogram of ear ly ident i f icat ion is cons ide red w i thou t pu rpose if not c o m p l e m e n t e d by the pract ice of ear ly and ef f ic ient in tervent ion (Salv ia & Y s s e l d y k e , 1 9 7 8 ) . 5 . Early Intervention Link Cons ide rab le debate and emot iona l inves tment has su r rounded the top ic of ear ly in te rvent ion , as ev idenced by the publ icat ion of t h o u s a n d s of ar t ic les and hundreds of empir ica l s tud ies (Whi te , 1 9 8 5 - 8 6 ) . The rat ionale for the ear ly educa t i on of hand i capped , at-r isk, or d i sadvan taged ch i ld ren has der ived f rom sou rces w h i c h inc lude theory , research , exper t op in ion , and soc ie ta l va lues (Mc lough l in , 1 9 8 8 ) . Seve ra l prominent developnnental psycho log i s t s are c i ted as inf luent ial in th is m o v e m e n t (B loom, 1 9 6 4 ; Ki rk, 1 9 5 8 ; S k e e l s , 1966 ) . A landmark s u m m a r y of s tud ies by Hunt (1961 ) cha l lenged the c o n c e p t of predetermined deve lopmen t and f ixed in te l l igence. Hunt fur ther in t roduced A m e r i c a to the interact ionist theor ies of P iaget and H e b b , a t tes t ing to the ef fect of env i ronmenta l in f luences and prov id ing a theoret ica l rat ionale for ear ly in tervent ion. He c o n c l u d e s that the " w a i t and s e e " app roach popular in earl ier years w a s highly unfor tunate . T h e re lat ionship b e t w e e n the ear ly years of life and later g row th and learning is w ide ly recogn i zed . Ear ly prevent ive in tervent ion a s s u m e s that the deve lopmen t and behav io r o f y o u n g ch i ldren is more suscep t ib le to change than the behav ior of older ch i ld ren (Bush & W a u g h , 1 9 8 2 ; Gura ln ick & Bennet t , 1 9 8 7 ; H a y d e n , 1 9 7 4 ) . Wh i le deve lopmen ta l mi les tones normal ly o c c u r , the p resence of cr i t ica l per iods for op t ima l response to learning exper iences has been sugges ted (B loom, 1 9 6 4 ; E r i c k s o n , 1 9 6 3 ; W h i t e , 1 9 7 5 ) . Enr ichment and spec ia l educa t ion in ear ly years is cons ide red power fu l and appropr ia te (Hayden & M c G i n n e s s , 1 9 7 7 ) . Fur ther , p roponen ts of the life s p a n pos i t ion a c k n o w d e d g e the impor tance of ear ly learn ing, but main ta in that adequa te s t imula t ion and educa t iona l oppor tun i t ies remain cont inua l ly impor tant in later ch i l dhood years (Clarke & C la r ke , 1 9 7 7 ; Ho row i t z , 1 9 8 0 ) . A l t h o u g h W h i t e , B u s h , & C a s t o (1985 -86 ) ind icate a lmos t unan imous suppor t for i nc reased ear ly in tervent ion se rv i ces and s t rong a d v o c a c y f rom exper ts and pract i t ioners , cr i t ics l iave posed conce rns regarding the es tab l ished va lue of ear ly in tervent ion p rograms. It is argued that other educa t iona l se rv i ces are o f higher pr ior i ty; that da ta regarding e f fec t i veness are i nconc lus i ve ; that there is insuf f ic ient ev idence that ear ly rather than later in tervent ion is more e f fec t ive ; and that it represents a "baby -s i t t i ng " se rv ice for ch i ld ren w i th a quest ionab le c a p a c i t y to learn (Mc lough l i n , 1 9 8 8 ) . The ques t ion of in tervent ion e f f i cacy , init ial ly one of a c a d e m i c and research or ienta t ion, n o w entai ls f inancia l cons idera t ion (Reyno lds , E g a n , & Lerner , 1 9 8 3 ) . Compe t i t i on for moneta ry resources and the d e m a n d for accountab i l i t y require ev idence of s u c c e s s in te rms of ch i ld progress and overa l l p rogram e f fec t i veness . Wh i te (1985 -86 ) sugges t s that research c o n d u c t e d to ach ieve s u c h pol i t ical rather than sc ient i f i c goa ls a f fec ts the es t imat ion of in tervent ion e f f i cacy , a s ign i f i cant issue inf luent ia l in po l icy dec i s ion -mak ing . Reyno lds et a l . (1983) report that s tud ies of spec i f i c in tervent ion var iab les are f e w in c o m p a r i s o n to long- term s tud ies of p rogram e f fec t i veness . W h i t e , B u s h and C a s t o (1985) ident i fy a number of mediat ing var iab les assoc ia ted w i th ear ly in tervent ion e f fec t i veness w h i c h require more spec i f i c s tudy to ascer ta in their t rue va lue for d i f fer ing popu la t ions . Examp les of media t ing var iab les inc lude age at start of in te rvent ion , durat ion and intensi ty , degree of s t ruc ture , and parental i nvo lvement . C a s t o and Mast rop ier i (1986) exp ress c o n c e r n that many rev iews have d r a w n c o n c l u s i o n s f rom a l imited number of s tud ies and have genera l ized e f f i cacy da ta f rom d isadvan taged and at-r isk popu la t ions to make s ta temen ts abou t all hand i capped ch i ld ren. Add i t iona l l y , Barnet t & Paget (1988) note that evaluat ion s tud ies o f ten present descr ip t ions of in tervent ions w h i c h lack the deta i l necessa ry for research rep l icat ion. Desp i te these d i f f icu l t ies, it is general ly c o n c l u d e d that qual i ty ear ly in tervent ion p rograms are e f fec t i ve . N A S P (1989) ind ica tes that poss ib le benef i ts inc lude improvemen t in s c h o o l pe r fo rmance , dec reased grade retent ion and dropout , less need for spec ia l educa t ion c l a s s e s , and dec reased emot iona l c o s t for the chi ld and fami ly . In add i t ion , long- term and cos t -e f fec t i ve benef i ts to soc i e t y inc lude rate reduc t ions in unemp loymen t , we l fa re , and c r ime . In s u m m a r y , s u c c e s s f u l p rogramming for y o u n g except iona l ch i ld ren necess i ta tes that appropr ia te ear ly in tervent ion be p receeded by accura te ear ly ident i f icat ion p rac t i ce . A number of fac tors ex is t w h i c h may in f luence the level of s u c c e s s at ta ined in the ident i f icat ion p r o c e s s . 6. Issues in Early Identification M a n y p rograms and techn iques in tended to ident i fy s tuden ts before rather than at the t ime of fai lure a s s u m e k n o w l e d g e of the p rob lem cond i t ion and its under ly ing e t io logy (Me ise ls , 1 9 8 4 ) . Th is is not as real ist ic for educa t iona l except iona l i t ies w h e r e e x a c t k n o w l e d g e of p recursors is uncer ta in . Individual cont inu i ty and level of ear ly deve lopmen t is di f f icul t to asce r ta i n , part icular ly in l ight of deve lopmen ta l unpredic tab i l i ty and shor t - te rm var iabi l i ty (L ichtenste in & Ireton, 1 9 8 4 ) . In add i t ion , deve lopmen ta l d i f f icul t ies in cer ta in areas are more re levant to s c h o o l pe r fo rmance than o thers . The p resence of fac to rs ind icat ive of r isk se rves on ly to increase the probabi l i ty of , but does not con f i rm , the ex i s tence of a future p rob lem. Pred ic t ion is not an e x a c t s c i e n c e and furtf ier l imi tat ions are imposed w h e n a t tempts are made to ident i fy y o u n g ch i ldren w h o may be at-r isk for later s c h o o l d i f f icu l ty even before t hose ch i ld ren have ac tua l ly been e x p o s e d to fo rmal schoo l i ng (Keogh , 1 9 7 7 ) . The dec i s i on con tex t w i th in w h i c h ear ly de tec t ion o f risk opera tes is general ly d i c h o t o m o u s . Errors of inc lus ion or exc lus ion are more l ikely to o c c u r w h e n us ing me thods w h i c h rely upon in ference (Wi lson & R e i c h m u t h , 1985) and negat ive e f fec ts c a n result f rom misc lass i f i ca t ion . Ch i ld ren w h o require in tervent ion m a y be exc luded wh i le others w h o have no need for t reatment may be i nc l uded . App rop r ia te measurement se lec t ion ref lect ing the purpose of the ident i f icat ion p rogram a lso in f luences the level of a c c u r a c y at ta ined in the eva lua t ion p r o c e s s (Cross , 1 9 7 7 ) . Techn i ca l a d e q u a c y is an i ssue pert inent to any app l ica t ion of measuremen t , ye t empir ica l research s e l d o m a c c o m p a n i e s the app l ica t ion o f ear ly ident i f icat ion ins t ruments (L indsay & W e d e l l , 1 9 8 2 ) . Mis in te rpre ta t ion c a n further hinder the deve lopmen t of appropr ia te early in tervent ion and thereby af fect the we l l -be ing of the ch i l d . 7 . Summary Suppo r t for the c o n c e p t and prac t ice of ear ly ident i f icat ion remains v igo rous desp i te theoret ica l and procedura l c o n c e r n s . Guide l ines for gather ing in format ion regard ing excep t i ona l s tudents encourages pert inent research , respons ib le po l icy dec i s i ons , se rv i ce improvement , and protect ion of the due p rocess r ights of s tuden ts and parents (McLough l in & L e w i s , 1986 ) . S c h o o l s y s t e m s prov ide conven ien t a c c e s s to the the vas t major i ty of ch i ld ren , thus enabl ing the implementa t ion of ear ly r isk ident i f icat ion p rog rams . The sc reen ing p rocedu re a l l ows in format ion to be gathered for e a c h s tudent . C. THE SCREENING COMPONENT Prov is iona l leg is la t ion, l i t igat ion, and c o n c e r n s regard ing the need for upda ted s tandards in p ro fess iona l pract ice have increased d e m a n d s regarding a s s e s s m e n t , and thereby ear ly ident i f icat ion d e m a n d s , made by educa t iona l po l i cy -makers and admin is t ra t ive personne l (Str ichart & Lazarus , 1 9 8 6 ) . M a n y s c h o o l d is t r ic ts have imp lemented ear ly de tec t ion p rograms w h i c h target the k indergar ten popu la t i on , thus fulf i l l ing case f ind ing requ i rements w i th in the ident i f icat ion p rocess . The subsequen t sc reen ing s tage a t tempts to se lec t educat iona l l y at- r isk s tuden ts . Nat iona l , reg iona l , and local i ncons i s tency in sc reen ing p rograms has p rovoked m u c h con t rove rsy , par t icu lar ly w i th regard to i ssues of ins t rumentat ion and usefu l interpretat ion (B racken , 1 9 8 2 ; G r a c e y et a l . , 1 9 8 4 ) . Sc reen ing endeavou rs have p rompted a prol i ferat ion of ear ly de tec t ion dev i ces (Reynolds et a l . , 1 9 8 4 ) . Zeit l in (1976) reported s ix teen years ago the ex i s tence of over one t housand sc reen ing and d iagnos t i c ins t ruments . S i nce that t ime, n e w measures have appeared on a regular bas is . A n e f fec t i ve sc reen ing p rogram requires sys tema t i c p lann ing to inc rease the a c c u r a c y o f resul ts and to a l l ow for qual i ty interpretat ion and dec i s ion -mak ing (L ichtenste in & Ireton, 1 9 8 4 ) . Sc reen ing is essent ia l l y a dec i s i on p rocess s ign i fy ing w l i i c l i s tuden ts do or do not require furt l ier a s s e s s m e n t . D e c i s i o n o u t c o m e s may truly ident i fy at-r isk (test posi t ive) and non-r isk (test negat ive) s ta tus , but may a lso prove to be fa lse for e a c h of these c lass i f i ca t i ons . C o n s e q u e n t l y , there are t w o t ypes of errors poss ib le (Mil ler and S p r o n g , 1 9 8 6 ) . Under- ident i f ica t ion ref lects the fai lure to de tec t s tuden ts w h o may indeed require in tervent ion se rv i ces (a fa lse negat ive c lass i f i ca t ion) . Over- ident i f ica t ion represents the fai lure to exc lude adequate ly func t ion ing s tuden ts w h o may unnecessa r i l y receive fur ther d iagnos is and in tervent ion (a fa lse posi t ive c lass i f i ca t ion) . Bo th s i tuat ions may require was te fu l mone ta ry expend i tu re and cause need less anx ie ty for the ch i l d , the fami ly , and the educa to r . Best pro fess iona l p rac t ice w o u l d a t tempt to reduce the propor t ion of error invo lved in these fa lse pred ic t ions . Pred ic t ion is rarely per fect for a n y s i tuat ion and is ideal ly eva lua ted aga ins t ac tua l fu ture per fo rmance to help ascer ta in its level of va l id i ty . M e t h o d s of va l idat ion c a n el ici t numer ica l ly impress ive ind icat ions of se lec t ion a c c u r a c y , but more per t inent d i s c u s s i o n might add ress the propor t ions of i naccu racy . T h e uti l i ty of a sc reen ing ins t rument rests upon its abi l i ty to predict not on ly m a x i m u m levels of co r rec t c lass i f i ca t i ons , but a lso m in imum levels of incorrect c lass i f i ca t ions . Sa tz and F le tcher (1982) sugges t that it is u l t imately a ques t ion of w h i c h error t ype cons t i t u tes the more ser ious r isk. W h i l e the sc reen ing p rocess invo lves many ac t iv i t ies , the spec i f i c sou rces of in fo rmat ion regard ing s tudent pe r fo rmance fo rm the bas is f rom w h i c h initial ind iv idual p lann ing dec i s ions are made (Wi lson & R e i c h m u t h , 1 9 8 5 ) . It is essent ia l that the a c c u r a c y of se lec ted measu res be cr i t ica l ly exam ined w i th in the con tex t of c lass i f i ca t ion and in tervent ion dec i s ions . In l ine w i t h the present inves t iga t ion , Satz & F le tcher (1982) argue that the sc reen ing po l i cy dec is ions of educa t iona l admin is t ra tors are best se rved by demons t ra t ion o f the uti l i ty of a pred ic t ive measure . 1 . Definition and Purpose of Screening Kindergar ten sc reen ing is a brief, l ow -cos t eva lua t i ve procedure used w i t h large g roups of ch i ld ren . Sc reen ing measures are t yp ica l l y admin is tered to g roups rather than to ind iv idua ls . The purpose is to ident i fy t h o s e s tuden ts cons ide red to be at-r isk for learning di f f icul t ies w h o may require spec ia l se rv i ces to aid their deve lopmen t (Barnes, 1 9 8 2 ; Pal f rey , 1 9 8 1 ) . W i t h i n the ear ly ident i f icat ion p rocess , sc reen ing se rves as a referral agent pre l iminary to d iagnos t ic a s s e s s m e n t (Bagnato & N e i s w o r t h , 1 9 8 7 ) . It is not just i f iable as a s ing le event , but rather as a c o m p o n e n t o f a comprehens i ve s y s t e m w h i c h u l t imately prov ides ear ly in tervent ion for t hose in need (Sh ipman, 1981 ) . Sc reen ing con ten t is , by def in i t ion, brief and resul ts are inferent ial in nature. Potent ia l m isuse as a d iagnost ic ind icator is therefore cau t i oned . G r a c e y et a l . (1984) report resul ts o f a nat ional su rvey of educa t iona l sc reen ing prac t ice w h i c h ind icate the m isuse of sc reen ing in format ion in the d iagnos i s of learning disabi l i t ies and in the d i rect ass ignmen t of remedia l p rog ramming . P lacemen t dec i s i ons , exc lus ion f rom s c h o o l ent ry and/or p rograms, re tent ion, or t rans i t ional k indergar ten recommenda t i ons are not to be made f rom sc reen ing resul ts or " r e a d i n e s s " tes t ing (Gredler, 1 9 9 0 ; S c h u l t z , 1 9 8 9 ; Tucke r , 1 9 8 9 ) . S u c h t rends are cons ide red unaccep tab le p rac t i ce . 2 . The Screening Process A v iab le sc reen ing program compr i ses ob jec t ives and methods that add ress local c o n c e r n s and are deve loped w i t h re ference to general ly accep ted p lann ing and implementa t ion p rocedures (Cross , 1 9 7 7 ) . L ich tenste in & Ireton (1984) sugges t that c lear concep tua l i za t ion of the essent ia l charac ter is t i cs (who and w h y ) a l l ow for c lar i f icat ion of operat iona l detai ls (what , w h e n , w h e r e , and h o w ) . a. Procedural Considerations It is cr i t ica l to se lec t a sc reen ing app roach and spec i f i c ins t ruments that wi l l prov ide the greatest amoun t of re levant in fo rmat ion . W h e n cons ider ing both app roach and ins t rumenta t ion . C r o s s (1977) and Frankenburg (1977) s u g g e s t that three impor tant cr i ter ia are a c c u r a c y , brev i ty , and s impl ic i ty . Add i t i ona l l y , F rankenburg (1977) sugges t s that the cr i ter ia of in tervent ion and the avai labi l i ty o f d iagnos t i c se rv i ces are a lso important cons ide ra t i ons . He treats c o s t benef i ts separa te ly , r e c o m m e n d i n g that f inanc ia l , fac i l i ty , and m a n p o w e r resources be d ist r ibuted in a manner that y ie lds the greatest benef i t for the c o s t i nvo l ved . IVIonetary c o s t s for sc reen ing m a y inc lude those assoc ia ted w i th tes t mater ia ls , ma in tenance , pe rsonne l , t ra in ing, test admin is t ra t ion , re lease t ime, c o r r e s p o n d e n c e , and record-keep ing (Lerner et a l . , 1 9 8 7 ) . In comb ina t ion w i th fur ther d iagnos is and poss ib le in tervent ion c o s t s , mone ta ry expend i tu re must p rove to be reasonab le w h e n c o m p a r e d w i th the benef i ts der ived for the chi ld and for the s c h o o l d is t r ic t . S c r e e n i n g a p p r o a c h e s may inc lude parent and teacher repor ts as we l l as fo rmal tes t admin is t ra t ion . The N A S P manua l (1989) separa tes the cri teria invo lved in the se lec t ion of an app roach f rom those invo lved in the se lec t ion o f spec i f i c tes t ing ins t ruments . A p p r o a c h cons ide ra t i ons inc lude spec i f i c i t y o f popu la t ion , pu rpose , pe rsonne l , f i nances , r esou rces , fac i l i ty , and parenta l invo lvement . Wh i l e s t r ingent s tandards for tes ts do ex is t , o ther fac tors in f luence inst rument se lec t ion for sc reen ing purposes and al ternate app roaches . A re levant connec t i on b e t w e e n dist r ic t ob jec t ives and the charac te r i s t i cs of the sc reen ing dev ice is requ i red. Pu rpose is regarded as a cr i t ica l fac tor . For examp le , w h e n sc reen ing for later reading p rob lems, the ins t rument shou ld s t ress related behav io r . S tandard iza t ion shou ld inc lude a normat ive popu la t ion that ma tches the target popu la t ion , w i th part icular at tent ion g iven to e thn ic i ty , s o c i o e c o n o m i c s ta tus , and age. Rel iabi l i ty and va l id i ty levels mus t be adequa te . W i t h regard to brev i ty , B o e h m & S a n d b u r g (1982) sugges t a limit of 15 to 2 0 minutes for admin is t ra t ion and Lerner et a l . (1987) emphas ize that ins t ruments be se lec ted re levant to the shor ter a t tent ion span of y o u n g ch i ld ren . Personne l requ i rements va ry regard ing t ra in ing leve l , t hough k n o w l e d g e of y o u n g ch i ld ren and the abil i ty to faci l i tate op t imum per fo rmance is essen t ia l . Other fac tors to be cons ide red are s impl ic i ty , cos t , appea l for ch i ld ren , and credibi l i ty for parents . In add i t ion to the se lec t ion of a prob lem cond i t ion and sc reen ing a p p r o a c h , p rocedura l cons ide ra t i ons inc lude recommenda t ions for a mode l that ref lects ongo ing sc reen ing in ear ly ch i l dhood (Barnes, 1 9 8 2 ; G r a c e y et a l . , 1 9 8 4 ; S h i p m a n , 1 9 8 1 ) . G i v e n the range of indiv idual d i f fe rences in deve lopmen t represented w i th in any one age group at any one t ime, repeated sc reen ing a l l ows for ind icat ions of deve lopmenta l shi f ts ac ross t ime and permits notat ion o f c h a n g e in risk s ta tus . M o r g a n (1981) a lso suppor ts sens i t i v i ty to change over t ime and notes the necess i t y for t ime passage w h e n teache r observa t ions are e m p l o y e d . Sc reen ing dur ing the f irst t w o or three w e e k s of s c h o o l is thus cau t i oned . K e o g h (1977) fur ther mainta ins that in format ion shou ld be gathered on at least t w o separa te o c c a s i o n s in order to reduce inaccura te ident i f icat ions. M e r c e r et a l . (1988) note that k indergar ten sc reen ing in the spr ing m a y s e e m more advan tageous than in the fall due to the s c h o o l exper ience ga ined and the inc reased possib i l i ty for teacher obse rva t i on . Admin is t ra t ion in D e c e m b e r , h o w e v e r , improves ear ly in tervent ion poss ib i l i t ies . T im ing of sc reen ing appears to ref lect d i f ferent theoret ica l bel iefs that v i e w sc reen ing con ten t as ind ica t ive o f deve lopmen ta l abi l i t ies or of ski l ls co r respond ing to a c a d e m i c cu r r i cu lum and ob jec t i ves . Ef f ic ient documen ta t i on is a lso cons ide red essent ia l to proper sc reen ing pract ice (L ichtenste in & Ireton, 1 9 8 4 ) . C o m p r e h e n s i v e record ing of resu l ts , s u b s e q u e n t referra ls, and fo l low-up serv ice is cr i t ica l to the eva lua t ion of the sc reen ing p rog ram. Eva lua t ion of the overa l l ear ly ident i f icat ion p rog ram, s tudent progress dur ing or fo l l ow ing in tervent ion, or the spec i f i c con ten t w i th in e a c h p rogram c o m p o n e n t (eg. the se lec ted sc reen ing approach) b e c o m e s poss ib le (Bagnato & N e i s w o r t h , 1 9 8 7 ) . b. Screening Approaclies S c h o o l d is t r ic ts m a y c h o o s e f rom a var ie ty of sc reen ing in format ion s o u r c e s . The mos t t yp ica l are parent repor ts , t eacher obse rva t i ons , and da ta -based s tandard ized tes ts (Barnett & Page t , 1 9 8 8 ) . N A S P (1989) reports that a n y p rocedure w h i c h ident i f ies a ch i ld as poss ib ly hav ing spec ia l learning needs is a fo rm of sc reen ing . Each app roach has advan tages and d i sadvan tages w h i c h mus t address the cr i ter ia presented above and the goa ls of the d ist r ic t . Parent report ing a s s u m e s an int imate k n o w l e d g e of the chi ld and p rov ides a d i rect , suppor t i ve link to in tervent ion ( N A S P , 1 9 8 9 ) . It requires a c c u r a c y of obse rva t i on , recal l and c o m m u n i c a t i o n w h i c h is though t by s o m e to be a f fec ted by emot iona l bias (Barnes, 1 9 8 2 ; Zei t l in , 1 9 7 6 ) . Paget & Nagle (1986) v i e w parental i nvo lvement as an essent ia l sou rce of k n o w l e d g e represent ing a broad range of t ime and c i r c u m s t a n c e s . T h e y report a genera l t rend in research resul ts w h i c h demons t ra tes parent -pro fess iona l agreement . Parent in format ion may be at ta ined th rough in te rv iews , ques t ionna i res , check l i s t s , or s tandard ized inventor ies (L ichtenste in & Ireton, 1 9 8 4 ) . The use of sys tema t i c observa t ion by teachers (rat ings, check l i s t s , or quest ionnai res) as a means of k indergar ten sc reen ing has been repor ted to be more e c o n o m i c a l and as e f fec t ive as formal tes t ing (Dyks t ra , 1 9 6 7 ; F e s h b a c h , A d e l m a n , & Ful ler, 1 9 7 4 ; Har ing & R i d g w a y , 1 9 6 7 ; N o v a c k , Bonaven tu ra & M e r e n d a , 1 9 7 3 ) . T e a c h e r observa t ions have the advan tage of occur r ing w i th in the s a m e env i ronment in w h i c h learning takes p lace and recogn izes the s t rength of k n o w l e d g e ga ined th rough famil iar i ty w i th indiv idual s tuden ts . A l l o w i n g suf f ic ient t ime to insure fami l iar i ty , h o w e v e r , has rece ived c r i t i c i sm regarding the subsequen t de lay in t rea tment in tervent ion. Sa tz and F le tcher (1982) note that Insuff icient research has o c c u r r e d to suppor t the c la im that pred ic t ions der ived f rom teacher observa t ions are equa l l y or more val id than psychome t r i c tes t ing . It is sugges ted that wh i le t eache rs appear to accura te ly predict h igh-r isk s tuden ts , they may do s o by inc lud ing on l y the more obv ious c a s e s , thereby exc lud ing other s tuden ts in need . Th is t e n d e n c y may be related to a fear of mis labe l ing (Satz & Fr ie l , 1 9 7 8 ) . A l s o , wh i le the bes t and mos t exper ienced teachers may be able to se lec t the at-r isk s tudent , n e w and inexper ienced teachers might require more ob jec t ive measu res (Butler, 1 9 7 9 ) . Despi te the apparent advan tages of teacher obse rva t i ons , L ich tens te in and Ireton (1984) c i te the a rgument by M e e h l (1954) that cons is ten t interpretat ion of va l id s tandard ized tes t ing ins t ruments may prove more pred ic t ive than the sub jec t i ve judgement o f teachers . Parent and teacher observa t ions are indirect , sub jec t i ve measures of a ch i l d ' s pe r fo rmance . F o r m a l , s tandard ized tes ts prov ide an ob ject ive bas is for compar ing indiv idual s tudent pe r fo rmance aga ins t that o f a norm group (norm-referenced) or aga ins t persona l mas te ry of de f ined ski l ls or behav iors (cr i ter ion-referenced) (Peterson, 1 9 8 7 ) . P rocedura l cons idera t ions for the se lec t ion of sc reen ing ins t ruments have been add ressed earl ier in th is chapter . Regard less of the s i tuat ion in w h i c h it is u s e d , a sc reen ing ins t rument must exhib i t t echn ica l a d e q u a c y th rough accep tab le levels o f rel iabi l i ty and val id i ty . A s sc reen ing measures a t tempt to predict w h i c h s tuden ts wi l l exper ience future learning d i f f icu l t ies , eva lua t ion of their predic t ive val id i ty is mos t pert inent to es tab l ish ing their degree of e f f i c i ency and usefu lness (Lewis , 1 9 8 0 ) . Laclc o f eva luat ion or con f l i c t ing ev idence in th is regard is no tab le (L indsay & W e d e l l , 1 9 8 2 ) . P resen tday c o n c e r n s address the a d e q u a c y of va r ious va l idat ion mode ls used in the eva luat ion of sc reen ing dec is ions (L ichtenste in & Ireton, 1 9 8 4 ; Me rce r et a l . , 1 9 8 8 ; Schu l t e , 1 9 8 8 ; W i l s o n & R e i c h m u t h , 1 9 8 5 ) . R e s e a r c h has a t tempted to d i scove r t hose var iab les w h i c h best predic t fu ture di f f icul t ies in s tudent pe r fo rmance . Initially, f ocus ing upon s ingle var iab les as predic tors w a s preva lent , but mult iple var iab le app roaches have b e c o m e more w idesp read as the var ie ty of fac tors w h i c h in f luence chi ld deve lopmen t and s c h o o l per fo rmance have been recogn ized (Keogh & Da ley , 1 9 8 3 ) . c. Screening Content Ident i f icat ion of educat iona l ly at-r isk s tuden ts is a c o m p l e x task both concep tua l l y and at the measurement level due to the mul t ip l ic i ty of separa te or comb ined externa l and wi th in-ch i ld in f luences on human deve lopmen t (Keogh & Da ley , 1 9 8 3 ) . Add i t i ona l l y , educat iona l o u t c o m e goals represent abi l i t ies, sk i l l s , and k n o w l e d g e w i th in mul t i fac i ted d o m a i n s . G r a c e y et a l . (1984) report resul ts f rom their nat ional su r vey of sc reen ing prac t ices w h i c h ind icate that con ten t areas are genera l ly representa t ive of four d o m a i n s : Physical: senso ry , motor (often referred to as perceptua l -motor ) , heal th h is tory , cur rent hea l th , immun iza t ions , denta l Language: s p e e c h , l anguage , b i l ingual ism Cognitive: cogn i t i ve sk i l l s , learn ing, apt i tude Behavioral: behav ior , soc ia l and emot iona l f unc t i on ing . G r a c e y ' s su r vey revealed that all four d o m a i n s w e r e se ldom inc luded in indiv idual sc reen ing p rograms a l though e a c h is c o m p r i s e d of con ten t a reas general ly found in the sc reen ing l i terature. S u c h var ie ty ind icates recogni t ion of the chi ld as a w h o l e person (Sh ipman, 1981 ). Un fo r tuna te ly , in their brevi ty and nature of con ten t , k indergar ten sc reen ing measu res are mos t of ten unable to sample s u c h a broad s p e c t r u m (Keogh & Becke r , 1 9 7 3 ) . M e r c e r et a l . (1988) sugges t that the inc lus ion of p reacademic con ten t in k indergar ten sc reen ing c a n lead to s u c c e s s f u l ident i f ica t ion of high-r isk learners . Spec i f i c ski l ls relat ing to p remathemat ics and preread ing are f o c u s e d u p o n , s u c h as recogni t ion of numbers , let ters, and shapes (Reyno lds , G u t k i n , El l iott , & W i t t (1984) . Pred ic t ion at an earl ier age w o u l d rely less upon s u c h academica l l y acqu i red sk i l ls than those of language and phys i ca l f ac to rs . M a n y s tud ies have f o c u s e d on the pred ic t ion of o u t c o m e s in spec i f i c a c a d e m i c areas cons ide red to be of pr ime impor tance to schoo l -age func t ion ing (Ade lman & F e s h b a c h , 1 9 7 1 ) . The language arts compr i se a substant ia l por t ion of pr imary grade cur r icu la and a good deal of research in th is area has invest iga ted the pred ic t ion of later ach ievemen t in reading (Bad ian, 1 9 8 6 ; F e s h b a c h , A d e l m a n & Ful ler, 1 9 7 4 ; G r a y , 1 9 8 8 ; J a n s k y & de H i r s c h , 1 9 7 2 ) . Horn & Packa rd (1985) summar i zed da ta on the pred ic t ion of later reading ach ievemen t by c o n d u c t i n g a meta-ana lys is on the resul ts of 5 8 k indergar ten or f i rst grade ear ly ident i f icat ion measures . Resu l ts sugges ted that the best ind icators inc lude language var iab les , at tent ion-dist ract ib i l i ty , general cogn i t i ve f u n c t i o n i n g , and the behav io r of internal iz ing problenns s u c h as anx ie ty , d e p r e s s i o n , or personal i ty p rob lems . Even these ind icators w e r e cons idered to be on l y moderate ly usefu l in group pred ic t ion of subsequen t schoo l per fo rmance a n d of l imited uti l i ty in indiv idual c a s e s . The l o w predic t ive magni tude of sof t neuro log ica l var iab les and s e n s o r y integrat ion w a s u n e x p e c t e d , part icular ly for measu res of g ross sk i l l s , cerebra l d o m i n a n c e , and h a n d e d n e s s . These var iab les are o f ten a s s u m e d to more d i rect ly infer spec i f i c h igher cor t i ca l dys func t i on w h i c h a c c o u n t s for poor s c h o o l pe r fo rmance than do g lobal cogn i t i ve measures s u c h as in te l l igence tes ts . In c o m p a r i s o n , therefore, spec i f i c var iab les have not p roven to be more ef fect ive ear ly pred ic tors of s c h o o l s u c c e s s than the more g lobal measu res as ind icated by the resu l ts . Internal iz ing behav io r p rob lems s u c h as anx ie ty and depress ion w e r e unexpec ted l y good pred ic tors of ach ievemen t . It has been a s s u m e d that behav io r and emot iona l p rob lems are a result of learn ing d i f f icu l t ies, rather than p recurso rs . It is n o t e d , h o w e v e r , that ex te rna l , ac t ing-out behav io r m a y be the resul t of long-te rm fai lure and f rus t ra t ion . G r a c e y et a l . (1984) a lso note the re lat ionship be tween ear ly behav iora l p rob lems and later a c a d e m i c func t i on ing , but ind icate that f e w requ i rements ex is t for behav iora l s c r e e n i n g . D isagreement con t inues as to w h i c h var iab les prov ide the best early pred ic t ion o f s c h o o l learning prob lems (S imner , 1 9 8 3 ) . The rev iew by Horn & Packa rd (1985) s e r v e s , in part , to emphas i ze the lack of cons i s ten t predic t ive ev idence of the many and var ious measu res e m p l o y e d in sc reen ing p rograms. Leach (1980) notes that too f e w a t tempts have been m a d e to eva luate the genera l e f fec t i veness of sc reen ing measures and their spec i f i c uti l i ty for the s c h o o l d ist r ic t i nvo l ved . O f no te , con fus i on be tween deve lopmenta l sc reen ing tes ts and read iness tests has comp l i ca ted con ten t se lec t ion issues (Me ise ls , 1 9 8 7 ) . M i s tak i ng one for the other mis represents the s c o p e and purpose of both tes ts . Th is c o n c e r n mus t be add ressed more thorough ly . d. Readiness Tests versus Developmental Screening Tests T w o bas ic tes ts are assoc ia ted w i th sc reen ing and a s s e s s m e n t o f ch i ldren at k indergar ten entry (Hil ls, 1 9 8 7 ) . S c h o o l read iness tes ts ind icate the acqu i red level of spec i f i c k n o w l e d g e and ski l ls that are cons ide red to be impor tant ent ry cr i ter ia for part icular s c h o o l p rog rams . T h e y descr ibe ch i ld ent ry charac te r is t i cs rather than predict ch i ld o u t c o m e s . Deve lopmen ta l sc reen ing tes ts y ie ld pe r fo rmance in format ion for broad areas of normal deve lopmen t and abi l i t ies that are assoc i a t ed w i th the potent ia l to acqui re further k n o w l e d g e and sk i l l s . M e i s e l s (1987) con t ras ts the purpose , con ten t , and p s y c h o m e t r i c propert ies of t hese t w o t ypes of tes ts . He s ta tes that the purpose of read iness tes ts shou ld be to ident i fy a ch i l d ' s p reparedness for spec i f i c a c a d e m i c p rograms and to faci l i tate cu r r i cu lum p lann ing . T h e purpose of deve lopmenta l sc reen ing is to ident i fy s tuden ts w h o may require and profit f rom modi f ied or ind iv idual ized p rog rams . T h e i tem con ten t of read iness tes ts ideal ly f o c u s e s o n cur rent ski l l a c h i e v e m e n t , pe r fo rmance , and genera l know ledge as o p p o s e d to deve lopmenta l screen ing i tems w h i c h d isp lay abi l i ty or potent ia l to acqu i re academ ic sk i l l s . Read iness tes ts are o f ten mis taken ly subst i tu ted for deve lopmen ta l sc reen ing t es t s w h i c h Me i se l s (1987) sugges t s have a s t ronger re lat ionship to future s c h o o l pe r fo rmance . In p rac t i ce , puri ty of con ten t is not preva lent . Read iness tes ts are mos t of ten cr i te r ion- re ferenced and are a better sub jec t for cons t ruc t val id i ty wh i le deve lopmenta l sc reen ing is norm-re fe renced and are a better sub jec t for pred ic t ive va l id i ty . Me ise l s (1987) cons ide rs that the re lat ionship be tween read iness tes ts and o u t c o m e measures s u c h as s c h o o l pe r fo rmance or comprehens i ve deve lopmenta l a s s e s s m e n t s is w e a k e r than that b e t w e e n deve lop -mental sc reen ing and o u t c o m e measu res . 3 . S c r e e n i n g Benef i ts By far the greatest advan tage of a c o m p r e h e n s i v e sc reen ing p rogram lies in the link b e t w e e n early ident i f icat ion of learning needs and the posi t ive e f fec ts on schoo l pe r fo rmance der ived f rom subsequen t and s u c c e s s f u l ear ly in tervent ion ( N A S P , 1 9 8 9 ) . Va r ious benef i ts ind icated in the r e v i e w of l i terature inc lude reduct ions in grade re tent ion, spec ia l educa t ion p lacemen t , unemp loymen t and s c h o o l d rop out rates. The s u m m a r y report of The Roya l C o m m i s s i o n of Educa t i on for Br i t ish Co lumb ia (Su l l i van, 1988) es t imates that 3 5 - 4 0 % of s tuden ts enter ing ninth grade fail to meet graduat ion requ i rements . Th is s i tuat ion is cons ide red to be intolerable for soc ie t y as a w h o l e , for the educa t iona l s y s t e m , and for the s tudent . It is noted that if cu r r i cu lum and inst ruct ion are to respond to the deve lopmen ta l needs of s tuden ts , a s s e s s m e n t mus t be inst igated at s c h o o l ent ry . Ear ly ident i f icat ion represents inc reased educa t iona l serv ice prov is ion for the y o u n g chi ld and ref lects due p rocess r ights of the parent and chi ld (Lerner et a l . , 1 9 8 7 ) . Emot iona l c o s t s to the chi ld and the fami ly are hopefu l ly reduced or p revented as a result of ear ly ident i f icat ion- intervent ion p rograms. A s all s tuden ts are inc luded in mass k indergar ten sc reen ing , initial admin is t ra t ion does not imply ind iv idual s t i gma (L ichtenste in & Ireton, 1 9 8 4 ) . Emphas is on the d i s cove ry of s t rengths as we l l as w e a k n e s s c a n help mit igate any label ing ef fects wh i le a id ing in tervent ion p lann ing . Ear ly de tec t ion faci l i tates fami ly ad jus tment and a c c e p t a n c e to more qu ick l y enable the implementat ion of ear ly in tervent ion prac t ices (Hayden , 1974 ) . Resu l ts of k indergar ten sc reen ing and va l idat ion tes t ing result in more accura te local or regional base rates of part icular cond i t ions (Barnes, 1 9 8 2 ) . Ev idence of p reva lence a l l ows educat iona l s y s t e m s to address present ing p rob lems. G r a c e y et a l . (1984) sugges t that sc reen ing resul ts might be used to deve lop a prof i le that desc r ibes those ch i ld ren mos t l ikely to be misc lass i f led (over ident i f ied or under ident i f ied) by a part icular measure . Sc reen ing dec is ions for these ch i ldren can be more carefu l ly mon i to red . T h e s e poss ib le benef i ts are encourag ing and prov ide incent ive to mainta in ear ly ident i f ica t ion and sc reen ing p rograms. A w a r e n e s s of c o n c e r n s regarding the sc reen ing p rocess prov ides a bas is for potent ia l improvement . 4. Screening Concerns a. General Concerns Screen ing as a means of risl< ident i f icat ion c a n n o t be just i f ied un less it results in more e f fec t ive educat iona l p rogramming for the s tudent (Book, 1 9 8 0 ) . Its va lue is cons ide red insuf f ic ient w i thou t better educa t iona l oppor tun i ty . T h e desired response of t reatment in tervent ion may in t roduce , h o w e v e r , a c o n f o u n d i n g ef fect on the pred ic t ive re lat ionship b e t w e e n sc reen ing and o u t c o m e measu res s u c h as ach ievemen t . Remed ia l p rogramming may pos i t ive ly in f luence subsequen t a c h i e v e m e n t levels s u c h that initial at-r isk ident i f icat ion w o u l d appear inaccura te . J a c o b s e n (1990) no tes that fai lure to cont ro l for the e f fec ts of in tervent ion has been prob lemat ic for researchers interested in ear ly p red ic t ion . J a c o b s e n inves t iga ted the re lat ionship b e t w e e n k indergar ten sc reen ing data for language, pe rcep tua l -motor , cogn i t i ve , and academ ic var iab les and four -year longi tudinal data represent ing ach ievemen t in vocabu la r y , l anguage , read ing , and ma th . Resu l ts of h ierarch ica l l inear regress ion ana lys is fai led to demons t ra te , h o w e v e r , that cont ro l l ing for in tervent ion improved the p red ic t ion-per fo rmance re la t ionsh ip . In fac t , the e f fec ts sugges t that in tervent ion did not improve s u b s e q u e n t ach ievemen t . Cont ro l l ing for the e f fec ts of in tervent ion is methodo log ica l l y cha l leng ing and beyond the s c o p e of the present s tudy a l though rates o f m isd iagnos is wi l l be a d d r e s s e d . A g e , gender , s o c i o - e c o n o m i c s ta tus , e thn ic cons ide ra t ions , s ize of samp le , genera l izabi l i ty of f ind ings ac ross popu la t ions , ref lect ion of cu r r i cu lum con ten t . soc ia l /emot iona l /behav io ra l fac to rs , and label l ing are examples of i s s u e s cont r ibut ing to the debate sur round ing the use of sc reen ing measures in ear ly ident i f icat ion p rog rams . In his o v e r v i e w of sc reen ing p rac t i ce . L e a c h (1980) relates the c o n c e r n for pract ica l e f fec t i veness and e f f i c iency of sc reen ing to the cr i t ical issue of p red ic t i ve val id i ty and h o w it is measu red . b. Validation Models T h e pr imary emphas i s of ear ly ident i f icat ion pract ice w i th in educa t ion is to predict w h i c h ch i ldren are l ikely at-r isk for exper ienc ing learning d i f f icu l t ies. T h e va lue of the sc reen ing c o m p o n e n t is represented in general by its inc lus ion as one s tage in a mul t i -s tage p r o c e s s , and in part icular by the level of predic t ive a c c u r a c y , or va l id i ty , of the part icular sc reen ing measure or measures e m p l o y e d . A s sc reen ing measu res are des igned to ass is t dec is ion c lass i f i ca t ions (r isk/no-r isk) , va l id i ty data ind icat ing the pred ic t ive e f f i c iency and ine f f i c iency (errors in predict ion) of t hese dec is ions are required (L ichtenste in , 1 9 8 1 ) . T w o main va l ida t ion a p p r o a c h e s have been repor ted . The first exp resses va l id i ty in te rms of cor re la t ion coe f f i c i en ts , the s e c o n d in te rms of pe rcen tages of accura te and inaccura te p red ic t ions . T h e use of the cor re la t ion coef f i c ien t in iso lat ion has been cr i t ic ized as inadequate because it p rov ides in format ion abou t the c o n g r u e n c e b e t w e e n the sco res on sc reen ing and o u t c o m e measures but not about the number of cor rec t and incor rec t dec i s ions conce rn i ng ch i ld ren at-r isk (Schul te & M o o r e , 1988 ) . C o n s e q u e n t l y , the essent ia l issue sur round ing the dec i s ion p rocess is not d i rect ly addressed th rough corre lat ional ana lys is . Cur ren t l y o f more prevalent use in the evaluat ion o f predic t ive e f f i c iency is the p red ic t ion -per fo rmance matr ix (Mercer et a l . , 1 9 8 8 ; W i l s o n & Re i chmu th , 1985 ) . Th is m o d e l , or ig inal ly p roposed by M e e h l & R o s e n (1955) , has many t i t les: c lass i f i ca t ion matr ix , hit rate mode l , c ross - tabu la t i on , e x p e c t a n c y tab le , and dec is ion mat r ix . A s educa t iona l dec i s ion -mak ing is inherent in k indergar ten sc reen ing , the latter tit le has been c h o s e n for use in th is s tudy . D. D E C I S I O N M A T R I C E S 1. Descr ip t ion and Interpretive Formu las The dec i s i on matr ix mode l typ ica l ly emp loys a 2 X 2 matr ix w h i c h can be interpreted in var ious w a y s . For sc reen ing dec i s ions (predict ive measure) , d i c h o t o m o u s r isk /no-r isk c lass i f i ca t ion resul ts are usua l ly recorded ver t ica l ly d o w n the left-side quadrants of the matr ix . Resu l ts of the cr i ter ion (per formance measure) , o f ten des igna ted as sa t i s fac to ry /unsa t i s fac to ry or p o o r / g o o d , are usual ly recorded hor izonta l ly a long the top-s ide quadran ts . F igure 1 i l lustrates the four quadrants w h i c h are fo rmed and their ca tegor ica l desc r ip t ions . E a c h quadrant ( A , B , C , D ) represents a dec is ion c lass i f i ca t ion ind icat ing the re lat ionship b e t w e e n s tudent per fo rmance on both the sc reen ing and cr i ter ion measu res . Fo l l ow ing med ica l c lass i f i ca t ion p rocedure , unsa t i s fac to ry results are cons ide red " p o s i t i v e " (risk) and sa t i s fac to ry resul ts are cons ide red "nega t i ve " (no-r isk). Figure 1. Decision IVIatrix Screen ing Pe r fo rmance Cr i ter ion P e r f o r m a n c e Unsa t i s fac to ry Sa t i s fac to ry Unsa t i s fac to ry (Risic) True Pos i t i ves A Fa lse Pos i t i ves (over-referral) B Sa t i s fac to ry (No-risl<) False Negat ives (under-referral) C T rue Negat i ves D 1 Quadran t A - unsa t i s fac to rysc reen ing /unsa t i s fac to ryc r i t e r i on (true pos i t i ves o r T - l - ) Quadran t B - unsa t i s fac to ry sc reen ing /sa t i s fac to ry cr i ter ion (false pos i t i ves or F + , a lso cons ide red an over-referral) Quadran t C - sa t i s fac to ry sc reen ing /unsa t i s fac to ry cr i ter ion (false negat ives or F-, a lso cons ide red an under-referral) Quadran t D - sa t i s fac to ry sc reen ing /sa t i s fac to ry cr i ter ion (true negat ives or T-) . The mode l s u m m a r i z e s the re lat ionship b e t w e e n sc reen ing o u t c o m e s (predict ion) and " a c t u a l " s ta tus as def ined by the cr i ter ion o u t c o m e . The predic t ive a c c u r a c y of c lass i f i ca t ion dec is ions c a n be interpreted in a number of w a y s us ing di f ferent fo rmu las ut i l iz ing percen tage ca l cu la t i ons . Individual c a s e s ass igned to Quadran ts A , B, C , and D fo rm the bas is for these fo rmu las . Hor izonta l and ver t ica l eva lua t ion prov ide compara t i ve o u t c o m e s (Mercer et a l . , 1 9 8 8 ) . Hor izonta l ana lys is ca lcu la tes percen tages of " p r o b a b l e " sc reen ing a c c u r a c y in relat ion to co r rec t and incorrect sc reen ing o u t c o m e s of r isk /no- r isk c lass i f i ca t ions : Ver t i ca l ana lys is ca lcu la tes percen tages of " a c t u a l " sc reen ing a c c u r a c y in relat ion to unsa t i s fac to ry and sa t is fac to ry cr i ter ion resu l ts . T w o fo rmulas are part icular ly impor tant to ver t ica l ly ca lcu la ted predic t ive a c c u r a c y : - sens i t i v i ty is the propor t ion of s tuden ts cor rec t l y ident i f ied as "a t - r i sk " (T-h), A/A + C - spec i f i c i t y is the propor t ion of s tuden ts cor rec t ly ident i f ied as "no - r i sk " (T-), D/B + D. T h e p rob lem base rate (or ac tua l p rob lem inc idence w i th in a g iven populat ion) is o f va lue w h e n mak ing c o m p a r i s o n s regarding probable and ach ieved a c c u r a c y rates of sc reen ing measures and is ca lcu la ted by A + C / A - I - B - H C + D . T h e referral rate represents the propor t ion o f s tuden ts w i th in the popula t ion w h o s e sc reen ing resul ts ind icated " r i sk . " It mus t be greater than the base rate as a cer ta in propor t ion o f referrals wi l l p rove to be in error. Sc reen ing is des igned to opt imize the poss ib i l i ty that all target group (at-risk) ch i ldren may be p icked up by the measu re or measu res emp loyed (L ichtenste in & Ireton, 1 9 8 4 ) . The referral rate is ca l cu la ted by A -i- B / A + B + C + D . - e f f i c iency of sc reen ing " r i sk " , (T + ) - ine f f i c iency of sc reen ing " r i sk " , (F+ or over-referral) A/A + B B/A + B - e f f i c iency of sc reen ing " n o r isk" , (T-) - ine f f i c iency of sc reen ing "no r isk" , (F- or under-referrral) . D/C + D C / C + D The overa l l hit rate represents the overa l l pe rcen tage of s tudents w h o have been cor rec t l y Identif ied as both " r i sk " and "no - r i s k " . It is ca lcu la ted by A + D /A+B + C + D. Figure 2 presents a numer ica l i l lustrat ion o f a dec i s ion matr ix as c i ted in L ich tenste in & Ireton (1984) . Figure 2 . Numerical Illustration of Decision Matrix Cr i ter ion S c r e e n Unsa t i s fac to ry Sa t i s fac to ry 17 (T + ) 2 7 (F-I-) Unsa t i s fac to ry (at-risk) * H (39%) * * V (77%) A H (61%) V (10%) B (over-referral) 4 4 5 (F-) 2 4 7 (T-) Sa t i s fac to ry (no-risk) H (2%) V (23%) C (under-referral) H (98%) V (90%) D 2 5 2 2 2 2 7 4 N = 2 9 6 * H = Hor izonta l eva luat ion * * V = Ver t i ca l eva luat ion P rob lem Base Rate Referra l Rate Sens i t i v i t y (T + ) (actual a c c u r a c y of p red ic t ion , V) Spec i f i c i t y (T-) (actual a c c u r a c y of p red ic t ion , V) = 2 2 / 2 9 6 = 7 % = 4 4 / 2 9 6 = 1 5 % = 1 7 / 2 2 = 7 7 % = 2 4 7 / 2 7 4 = 9 0 % Ef f i c iency of sc reen ing "a t r isk (probable a c c u r a c y of p red ic i ton , H) = 1 7 / 4 4 = 3 9 % Inef f ic iency of sc reen ing "a t r isk" (ie. over- refer ra l , H) = 2 7 / 4 4 = 6 1 % Ef f i c iency of sc reen ing "no r isk" (probable a c c u r a c y of p red ic t ion , H) = 2 4 7 / 2 5 2 = 9 8 % Inef f ic iency of sc reen ing " n o r isk" (ie. under-referra l , H) = 5 / 2 5 2 = 2 % Overa l l Hit Rate = 2 6 4 / 2 9 6 = 8 9 % Va r i ous interpretat ions are poss ib le . For examp le , hor izonta l eva lua t ion ind ica tes that 3 9 % of t h o s e pred ic ted to do unsat is fac tor i ly actua l ly per fo rmed s o , yet the pred ic t ion tes t w a s cor rec t for 7 7 % of t hose w h o actua l ly per fo rmed poor ly w h e n ver t ica l eva lua t ion w a s app l ied . M a n y authors indicate a pre ference for ver t ica l eva lua t ion and cons ide r sens i t iv i ty and spec i f i c i t y to con ta in the mos t pert inent in format ion ( G r e e n w o o d , 1 9 8 1 ) . It is noted that by mere ly increas ing or dec reas ing the cut-of f s co re on either pred ic t ive or cr i ter ion m e a s u r e s , the der ived in format ion is al tered acco rd ing l y . 2 . Summary Resu l ts of dec i s i on matr ix ana lys is are cons ide red to prov ide more va luab le in format ion than the cor re la t ion coef f i c ien t w h e n invest igat ing the predic t ive va l id i ty and uti l i ty of sc reen ing c lass i f i ca t ion dec i s i ons . Hor izonta l and ver t ica l eva lua t ion o f the matr ix c o m p a r e probable ve rsus ac tua l a c c u r a c y rates of risk ident i f i ca t ion . P rob lem base rate, referral rate, and overa l l hit rate may a lso be c a l c u l a t e d . E. S U M M A R Y The present s tudy wi l l p rov ide addi t ional k n o w l e d g e regarding the va l ida t ion of k indergar ten sc reen ing measu res . W i th in the f r amework of empi r ica l eva lua t i on , dec i s i on matr ices wi l l be uti l ized to demons t ra te the predic t ive val id i ty and uti l i ty o f spec i f i c k indergar ten sc reen ing m e a s u r e s . Th is ana lys is represents ef for ts to generate in format ion of a prac t ica l rather than s tat is t ica l nature. Hor i zon ta l , ver t i ca l , and overa l l eva lua t ions of c lass i f i ca t ion dec is ions wi l l be p resen ted . For the s c h o o l d is t r ic t , in format ion wi l l be p rov ided regarding the pred ic t ive a c c u r a c y o f sc reen ing at-r isk or not at-r isk us ing the se lec ted sc reen ing m e a s u r e s . Th is in format ion may prove inf luent ial in mak ing po l i cy dec i s ions perta in ing to the con t inua t ion and /o r modi f i ca t ion of the k indergar ten sc reen ing p rog ram. Rela ted fac tors to be cons ide red m a y inc lude the a l locat ion of mone ta ry and personne l resources . III. METHODOLOGY A . INTRODUCTION The purpose of th is s tudy is to invest igate the val id i ty and uti l i ty of four k indergar ten sc reen ing nneasures and the c o m p o s i t e sc reen ing c lass i f i ca t ion as pred ic tors of th i rd-grade ach ievemen t , h is tory o f in tervent ion, and retent ion s ta tus . Th is chapter desc r ibes the target p rogram and sub jec t samp le , the pred ic t ive and cr i ter ion ins t ruments , and the me thod of data co l lec t ion and ana lys i s . B. THE EARLY IDENTIFICATION PROGRAM T h e program under s tudy w a s init iated in 1981 by one s c h o o l d ist r ic t to aid in the ear ly ident i f icat ion of k indergar ten ch i ld ren cons ide red to be at-r isk for except iona l i t ies in learn ing. A l t h o u g h th is descr ip t ion might inc lude potent ia l ly gi f ted s tuden ts , these ch i ld ren we re not the f o c u s of the present inves t iga t ion . Ex tens i ve documen ta t i on and its access ib i l i t y w e r e addi t ional advan tages of fered by the se lec ted s c h o o l d ist r ic t . The ear ly ident i f icat ion p rocess emp loyed a th ree-s tage mode l w h i c h inc luded the tr i - level se rv i ces of c l a s s r o o m , bu i ld ing, and d is t r ic t resource staf f . A spec i f i c k indergar ten sc reen ing p rogram w a s des igned for the initial s tage in the ear ly ident i f icat ion p r o c e s s , f o l l owed by further d iagnos t i c a s s e s s m e n t as requi red. The k indergar ten teacher , learning ass i s tance teacher , and spec ia l se rv i ces staff w e r e the agents respons ib le for gather ing in format ion at p rogress ive s tages . The sc reen ing s tage itself w a s carr ied out th roughout the k indergar ten year in order to mainta in ear ly de tec t ion of r isk a n d , t hus , to enable more e f fec t i ve dec is ion -mak ing and indiv idual p rogramming to take p lace . Pub l ic heal th s c r e e n i n g , parent i n te rv iews , s c h o o l ent ry in fo rmat ion , and ongo ing teacher obse rva t i ons prov ided the bas is for qual i tat ive da ta . Th is s tudy w a s in tended to invest igate the val id i ty o f t hose quant i ta t ive sc reen ing measures se lec ted by the dist r ic t as predic t ive ind icators of r isk w h i c h we re admin is tered at di f ferent t imes th roughou t the k indergar ten year . T h e deve lopmen ta l areas noted by the distr ic t as the mos t f requent ly invest igated in the ident i f icat ion of at-r isk ch i ldren inc luded those of l anguage , v i sua l -motor , f ine /g ross motor , cogn i t i ve , a c a d e m i c / p r e - a c a d e m i c , and s o c i a l / emot iona l . The fo l low ing measures w e r e se lec ted by the d ist r ic t : 1. Draw-A-Person ( D A P ) ; Th is test , admin is te red in N o v e m b e r , is cons ide red to be indicat ive of non-verba l cogn i t i ve abi l i ty. 2 . Kindergarten Language Screening Test (KLST) ; Th i s tes t , admin is te red in J a n u a r y , is cons ide red to be ind icat ive of verba l language abi l i ty. 3 . Mann-Suiter Visual Motor Screen (MS) ; This tes t , admin is te red in J a n u a r y , is cons ide red to be ind icat ive of a ch i l d ' s deve lopmen ta l level of v isua l -motor integrat ion ski l ls for recogn iz ing and reproduc ing geomet r i c s h a p e s . 4 . Deverell Test of Letters and Numbers (DLN) ; Th is tes t , admin is te red in M a y , is cons ide red to be indicat ive of the abi l i ty to ident i fy the numbers 1-12 and the upper and lower c a s e letters of the a lphabet . For e a c h sc reen ing measure , fai lure to meet the pass ing cr i ter ia w a s ind icated by a " f a i l ed " ( "Risk") s tand ing rather than by the ac tua l s c o r e . E a c h tes t . al though di f fer ing in con ten t and length , w a s thus cons ide red of equal w e i g h t as a sc reen ing measure . A compos i t e sc reen ing c lass i f i ca t ion of " R i s k " w a s ob ta ined w h e n t w o or more " fa i lu res" we re a c c u m u l a t e d . A compos i t e c lass i f i ca t i on o f "No - r i sk " w a s ob ta ined w h e n three or more passes we re a c c u m u l a t e d . " R i s k " s tuden ts we re subsequen t l y referred to the learning a s s i s t a n c e teacher for further sc reen ing and d iagnos is . Shou ld intervent ion p rog ramming prove u n s u c c e s s f u l , third s tage referral to distr ic t spec ia l se rv i ces staf f for psychoeduca t i ona l a s s e s s m e n t f o l l owed . Parents we re kept in formed at e a c h s tage. A n or ientat ion sess ion for k indergar ten and learning ass i s tance teache rs w a s held at the beg inn ing of the s c h o o l year to faci l i tate the e f fec t ive imp lementa t ion of s c r e e n i n g , d o c u m e n t a t i o n , and s u m m a r y record-keep ing p rocedures and to prov ide the required t ra in ing in admin is t ra t ion and sco r ing of the measu res invo lved . O n g o i n g in-serv ice sess i ons w e r e then held dur ing the regular ly schedu led meet ings of these teachers . For the d is t r ic t , the major f inanc ia l expend i tu res inc luded the pu rchase of test mater ia ls , dup l ica t ion c o s t s , and the e m p l o y m e n t of subst i tu te staf f required to rel ieve k indergar ten teachers dur ing the indiv idual admin is t ra t ion of the K indergar ten Language Sc reen ing Tes t . C. SAMPLE The par t ic ipat ing d is t r ic t is s i tuated w i th in one hou r ' s dr ive of V a n c o u v e r , B . C . and p rov ides both rural and urban hous ing and e m p l o y m e n t for l ow , middle. and upper i ncome fami l ies and ind iv iduals . M a n y e thn ic groups are rep resen ted . Distr ic t pupi l enro l lment approx imated 1 5 , 0 0 0 , roughly 1 ,000 s tuden ts at e a c h grade leve l . Da ta f rom 31 e lementary schoo l s is avai lable for this s tudy . T h e 6 8 4 sub jec ts in this s t udy are d r a w n f rom one age cohor t to be admin is tered both the k indergar ten sc reen ing measures and the cr i ter ion measure four yea rs later. Included were 1 9 8 6 - 8 7 s tudents w h o remained in a t tendance in the d is t r ic t fo l l ow ing their enro lment in k indergar ten in 1 9 8 3 . Sub jec ts w e r e se lec ted w h o s e records ind icated comp le te or part ial k indergar ten sc reen ing resul ts and th i rd-grade s tandard ized ach ievemen t test resu l ts . Reten t ion w a s cons ide red to be indicat ive of learning di f f icu l ty and thus ac ted as one cr i ter ion measure . T h e samp le therefore inc luded those s tuden ts retained one or more years in a pr imary grade. S imi la r ly , a h is tory of s choo l - based in tervent ion w a s cons ide red another cr i ter ion measu re and s tudents k n o w n to have rece ived in tervent ion w e r e i nc luded . D. INSTRUMENTATION T h e se lec ted k indergar ten sc reen ing measures w e r e thought to ref lect the deve lopmen ta l areas cons ide red by the distr ic t to be inf luential in the ear ly ident i f icat ion o f educat iona l ly at-r isk ch i ld ren . Th i rd-grade ach ievemen t sco res we re ob ta ined f rom a s tandard ized ach ievemen t test admin is tered rout inely ac ross the d is t r ic t at the end of the s c h o o l year . 1 . Predictive IVIeasures T h e k indergar ten sc reen ing bat tery inc luded four measures as p red ic to rs . The fo l l ow ing ins t ruments w e r e c h o s e n for their hypo thes ized predic t ive con ten t . a. Draw-A-Person (DAP), 1963 Th is brief test is cons ide red to be an ind icator of non-verba l cogn i t i ve abi l i ty . It w a s group admin is te red and sco red by the teacher in N o v e m b e r of the s c h o o l year . S tuden ts w e r e ins t ruc ted to d r a w a person and po in ts w e r e cred i ted for accep tab le execu t i on o f var ious body par ts . A trial admin is t ra t ion of the tes t prov ided local no rms . "Fa i l u re " s ta tus w a s ind icated by a sco re of 6 or less ou t of a poss ib le sco re o f 3 1 . T h e con ten t of the d is t r ic t ' s D r a w - A - P e r s o n tes t w a s adapted f rom the Goodenough-Har r i s D r a w i n g Tes t (Harr is, 1963 ) and inc luded those i tems cons ide red l ikely to be d r a w n by pr imary and younge r ch i ld ren . The G o o d e n o u g h -Harr is test (GHDT) is i tself a rev is ion of the 1 9 2 6 D r a w - A - M a n Tes t ( G o o d e n o u g h , 1926 ) . T h e G H D T addi t ional ly required the d r a w i n g of a w o m a n and of the sel f and prov ided newer no rms for a w ide r age range and in t roduced a dev ia t ion IQ s c o r e . The purpose o f the G H D T is to measure inte l lectual matur i ty and the deve lopmen t of c o n c e p t u a l th ink ing w h i c h g r o w s w i th menta l leve l , expe r ience , and k n o w l e d g e (Harr is, 1 9 6 3 ) . It is cons ide red accep tab le as a sc reen ing ins t rument o f cogn i t i ve abi l i ty , most e f fec t i ve in the lower ranges of in te l l igence, but it d o e s not ind icate an actua l measu re of in te l l igence nor ref lect the dec i s ion -mak ing p o w e r o f more accura te tes ts of in te l l igence (Satt ler, 1 9 8 8 ) . A s s u c h , its use as a supp lementa ry inst rument of cogn i t i ve abi l i ty or its in tegrat ion into a bat tery o f tes ts c a n be r e c o m m e n d e d . Fur ther , it has been cons ide red less cul tural ly loaded for use w i th e thn ic minor i t ies, t hough the e m p h a s e s w h i c h di f ferent cu l tu res p lace upon body parts and c lo th ing is of note (Oak land & D o w l i n g , 1 9 8 3 ) . The preferred age range for use of th is inst rument is 3 to 10 yea rs , a l though the tes t spans ages 3 -0 to 1 5 - 1 1 . T h e s tandard iza t ion samp le inc luded 2 , 9 7 5 A m e r i c a n ch i ld ren representat ive of the popula t ion as ref lectd by the 1 9 5 0 c e n s u s and dist r ibuted a m o n g four major geograph ic areas ( A n a s t a s i , 1 9 7 2 ) . D u n n (1972) repor ts test - retest rel iabi l i t ies for the D r a w - A - M a n as ranging f rom . 6 0 to . 7 0 . S tud ies rev iewed by Sco t t (1981) ind icate a median test - re test rel iabil i ty o f . 7 4 . A med ian interrater rel iabil i ty of . 9 0 for D r a w - A - M a n and . 9 4 for D r a w - A - W o m a n are fur ther reported by Sco t t . D u n n (1972) remarks that m u c h of Har r i s ' 1 9 6 3 report ing on va l id i ty and rel iabil i ty refers to 1 9 2 6 D r a w - A - M a n data and that too little in format ion is avai lable regard ing the newer d raw ing t a s k s . Nagl ier i (1988) and others c i te research w h i c h demons t ra tes corre la t ion of the G H D T w i th inte l l igence tes ts s u c h as the W e c h s l e r a n d Binet sca les (Dunn, 1 9 6 7 ; Nagl ier i & M a x w e l l , 1 9 8 1 ; Trami l l , E d w a r d s , & Trami l l , 1 9 8 0 ) . Sat t ler (1988) refers to S c o t t ' s (1981) rev iew to ind icate tha t the D r a w - A - M a n Tes t c a n e f fec t ive ly d iscr iminate age level pe r fo rmance f rom ages 5 to 1 2 , but is relat ively poor at predic t ing sco res on other in te l l igence tes ts (median corre la t ion of .49 ) . Cr i t i c i sm of the test typ ica l ly inc ludes requests for more cur ren t s tandard iza t ion , modern iza t ion of sco r i ng gu ide l ines, and a sugges t ion by Sat t le r (1988) and Sco t t (1981) that the the norms be ref ined to inc lude half- and quar ter-year ly in tervals rather than year ly in tervals . b. Kindergarten Language Screening Test (KLST), 1978 T h e K indergar ten Language Sc reen ing T e s t is a normed test des igned to compa re the verba l language abi l i t ies of k indergar ten ch i ldren w i th levels cons ide red appropr ia te for their age and grade. Th is ins t rument w a s indiv idual ly admin is te red by the k indergar ten teacher in J a n u a r y of the s c h o o l year . T o ensure a c c u r a c y , t eachers w e r e e n c o u r a g e d , but not requi red, to tape- record responses . The manua l (Gauthier & M a d i s o n , 1983) s ta tes that tasks ref lect both recept ive and exp ress i ve language c o m p e t e n c e . The c o m p e t e n c i e s inc lude : s ta t ing name and age ; naming four spec i f i c co lo r s ; c o u n t i n g , w i th po in t ing , to t e n ; ident i fy ing four named b o d y par ts ; fo l low ing three-par t oral sequent ia l c o m m a n d s w i th k n o w l e d g e of prepos i t ions no ted ; repeat ing s e n t e n c e s ; and of fer ing a spon taneous s p e e c h samp le fo l l ow ing presentat ion o f a p ic ture ser ies . T h e s e s e v e n i tems represented a poss ib le sco re of 2 9 , of w h i c h 19 or less s ign i f ied " fa i lu re " s ta tus . The intent o f the K L S T is to ind icate the probabi l i ty of a language def ic i t and the need for more in tensive tes t ing before de fec t i ve ski l ls might poss ib l y acce lera te a c a d e m i c fai lure (Gauthier & M a d i s o n , 1 9 8 3 ) . The authors suppor t the concep t that the deve lopmen t of oral language ski l ls is prerequis i te to the use and unders tand ing of wr i t ten language and enab les ch i ldren to a c h i e v e s u c c e s s in s c h o o l . The tes t manua l reports cons t ruc t va l id i ty cor re la t ions ranging f rom . 3 6 to . 60 (each s ign i f i cant at the .05 level or better) b e t w e e n the K L S T and the U tah Test of L a n g u a g e Deve lopmen t ( M e c h a m , J e x , & J o n e s , 1967) as we l l as three sub tes ts (Aud i to ry C l o s u r e , Grammat i c C l o s u r e , and Verba l Express ion) o f the Illinois Tes t o f Psycho l i ngu is t i c Abi l i t ies (Kirk, M c C a r t h y , & Kirk, 1 9 6 8 ) . T h e s e sub tes ts w e r e c h o s e n for their s imi lar i ty to sk i l ls re f lected in the K L S T t a s k s . A corre la t ion of . 7 0 is repor ted be tween the B o e h m Tes t of Bas ic C o n c e p t s (Boehm, 1971) and the K L S T . In research ing the predic t ive val id i ty of the K L S T , the authors s tud ied the progress o f 3 0 of 2 3 3 s tuden ts w h o had t w o and a half years prev ious ly rece ived test s c o r e s of 9 - 1 9 . Resu l ts ind icated that 9 (36%) s tuden ts had been re ta ined, 7 (25%) w e r e p laced in or el igible for spec ia l educa t i on , and 6 (21 %) had rece ived remedia l read ing ins t ruc t ion . A l t h o u g h ass igned to on ly one of these s u b g r o u p s , severa l s tuden ts had rece ived serv ice w i th in more than one ca tego ry . S u m m a r y data ind ica ted that 2 3 (82%) of the 3 0 or ig inal at-r isk s tuden ts had encoun te red educa t iona l d i f f icu l t ies , se rv ing to va l idate the use of the K L S T as a pred ic tor of l ower language and a c a d e m i c func t ion ing , l l le rbraun, Ha ines , & Greenough (1985) note that the K L S T y ie lded a cor re la t ion o f , 68 w i th a c o m p o s i t e cr i ter ion sco re der ived f rom the c o m b i n e d use of the Tes t of Language Deve lopomen t ( N e w c o m e r & H a m m e l l , 1 9 7 7 ) , the Tes t for A u d i t o r y C o m p r e h e n s i o n of Language (Car row , 1 9 7 3 ) , and the C a r r o w El ic i ted Language Inventory (Ca r row , 1 9 7 4 ) . The K L S T , h o w e v e r , w a s the poorest predictor a m o n g tine 5 sc reen ing measures tes ted (the o thers ranging f rom . 7 5 to .85) . Test - re tes t rel iabil i ty data y ie lded a s ign i f i cant Pea rson coef f i c ien t of . 8 7 in a s tudy of 2 2 Headstar t s tuden ts randomly se lec ted f rom an or iginal popu la t ion of 8 8 s tuden ts . Th is test is not to be c o n f u s e d w i th another sc reen ing measure of the s a m e name dev ised for loca l use w i th in the L o n d o n (Ontario) Board of E d u c a t i o n . A l t hough simi lar in c o n c e p t and w i th s o m e s imi lar i tems, the Ontar io tes t w a s normed for loca l cond i t i ons . Its use , h o w e v e r , has been reported in a s tudy uti l iz ing a bat tery of 4 sc reen ing measures for pred ic t ion of at-r isk s ta tus . T h e Ontar io K L S T w a s found to best se lec t s tuden ts requir ing spec ia l educa t i on p lacement (Stone & Tre loar , 1990) by locat ing 2 5 of 9 8 subsequen t l y ident i f ied excep t iona l s tuden ts , on ly 3 8 of w h i c h had ac tua l ly been se lec ted by the tota l bat tery. T h e s e authors sugges t that the level of verba l c o m p r e h e n s i o n and exp ress i ve abi l i ty apparent ly con ta ined in the Ontar io K L S T i tems represents a broad c o n c e p t u a l doma in and serves to ind icate the ef fect w h i c h language faci l i ty can have o n educa t iona l p lacement . The K L S T under d i s cuss i on in this s t udy is becom ing more w ide l y adop ted as inc reas ing numbers of sc reen ing tes ts are emerg ing for use w i t h schoo l -aged ch i ld ren . T e x t b o o k s c o n c e r n e d w i th spec ia l educa t ion and learning disabi l i t ies make cur rent re ference to the K L S T , a l though little in format ion appears regard ing empi r ica l eva lua t ion (Berdine & M e y e r , 1 9 8 7 ) . c. Mann-Suiter Visual Motor Screen (MS), 1974 Th is s imple sc reen ing d e v i c e , ind icat ive of the abi l i ty to recogn ize and p roduce geomet r i c s h a p e s , w a s admin is tered in groups of f ive in J a n u a r y o f the s c h o o l year . Integrat ion of v isua l and motor ski l ls is demons t ra ted by c o p y i n g four geomet r i c f igures ( M a n n , Sui ter & M c C l u n g , 1 9 7 9 ) . Sub jec ts are a l l owed three a t tempts to c o p y e a c h f igure. T h e y are not permi t ted to erase their m i s takes . No rmed w i t h re ference to ch rono log ica l age as sugges ted by llg & A m e s ( 1 9 6 4 ) , e a c h shape represents per fo rmance appropr ia te for p rogress ive ages : c i r c le , 3 yea rs ; rec tang le , 4 yea rs ; t r iangle, girls 5 1/2 years and boys 6 y e a r s ; and d i a m o n d , 6 yea rs . Teache rs eva lua te responses acco rd ing to sco r ing gu ide l ines for the execu t i on of bas ic s h a p e s , l ines, and ang les . M ino r in f ract ions are o v e r l o o k e d . In the present s tudy a sco re of 2 or less o f a poss ib le 4 ident i f ied " fa i lu re . " C o p y i n g of geomet r i c shapes is repor ted by Sat t ler (1988) to be ind icat ive of appropr ia te perceptua l d iscr iminat ion abi l i ty ( input), f ine motor deve lopmen t (output) , and the abi l i ty to integrate both these p rocesses wh i le shi f t ing at tent ion b e t w e e n the or ig inal and the c o p y (storage and retr ieval) . Va r iab les a f fec t ing poor func t ion ing might inc lude b io log ica l , env i ronmen ta l , in te l lec tua l , or exper ient ia l s o u r c e s . The Mann -Su i t e r V i s u a l M o t o r S c r e e n (MS) is one of a number of deve lopmen ta l sc reen ing dev i ces f irst pub l ished by the au thors in 1 9 7 4 for the pu rpose of f ocus ing on par t icu lar deve lopmen ta l a reas . T h e s e areas are inves t iga ted to ascer ta in m i n i m u m levels of read iness rather t han as measures of l imi ted abi l i ty ( M a n n , Sui ter & M c C l u n g , 1 9 8 7 ) . W i th in th is d iagnos t ic -p rescr ip t i ve f r amework , eva lua t ion of indiv idual learn ing charac te r is t i cs w a s cons ide red essent ia l to any subsequen t d iagnos is or des ign of indiv idual educat iona l p rograms. Ap t i tude for acqu i r ing cer ta in k n o w l e d g e and ski l ls w a s noted as be ing as important as the ac tua l per fo rmance capac i t y . S u c c e s s f u l comp le t ion of the v isua l motor s c r e e n a lso ind icated min imal s tanda rds for s u c c e s s in handwr i t i ng . Di f f icul ty af ter the age of 7 sugges ted p rog ramming requi rements for prac t ice in eye-hand read iness sk i l l s . Cr i ter ion for min imal func t ion ing in e igh teen deve lopmenta l sc reen ing areas were ident i f ied by the au thors after an ex tens i ve rev iew of s tandard ized t es t s . T h e normat ive popu la t ion cons i s ted of 4 3 6 s tuden ts aged 4 th rough 12 hav ing borderl ine or above inte l l igence and represent ing a range of s o c i o e c o n o m i c levels and e thn ic i t ies . T e a c h e r s reported 291 s tuden ts to be fai l ing in s c h o o l . Resul ts were ana lyzed by relat ing sc reen ing pe r fo rmance to spec i f i c ski l ls required for s u c c e s s in read ing , w r i t i ng , spe l l ing , and ar i thmet ic . T h e major i ty of s tuden ts w h o fai led the v i sua l motor s c r e e n were found to a lso have d i f f icu l ty w i t h handwr i t i ng . T h u s , t hose s tuden ts w h o had no s u c h di f f icu l ty p rov ided the lower limit norms for s u c c e s s o n this sc reen ing dev i ce . Overa l l , the major i ty of the academica l l y s u c c e s s f u l s tuden ts met the m in imum s tandard for all of the a reas , wh i le the major i ty o f fai l ing s tuden ts fai led to meet the s tanda rd . Initial con ten t va l id i ty of the sc reen ing dev i ces w a s es tab l i shed over a t w o - y e a r per iod by more than 2 0 0 teache rs , counse l l o r s , and psycho log i s t s th rough the use of the measures in addi t ion to other measu res a l ready emp loyed in the ident i f icat ion of learning p rob lems (Mann , Su i te r & M c C l u n g , 1987 ) . Further, a su r vey of many of the t housands of educa to rs w h o have used the dev ices s i nce 1 9 7 4 has ind icated their va lue in sc reen ing for learning d i f f i cu l t ies . d. The Deverell Test of Letters and Numbers (DLN), formally The Deverell Beginners' Classification Tests, 1974: Form 1G, Knowledge of Numbers Form 2G, Knowledge of Capital Letters Form 3G, Knowledge of Lower Case Letters. Th is bat tery of t es t s , ind icat ive of the abi l i ty to ident i fy the numbers 1-12 and e a c h letter of the a lphabet (upper and lower case ) , w a s group admin is te red in M a y of the s c h o o l year . In response to oral ly p resented numbers (or let ters) , s tudents w e r e ins t ruc ted to c i rc le the cor rec t number f rom a m o n g a samp le of four non-sequent ia l c h o i c e s (or f ive letter cho ices) per i tem (Deverel l , 1 9 7 4 ) . G r o u p admin is t ra t ion w a s ut i l ized. A sco re of 5 4 or less of a poss ib le sco re of 6 4 indicated " fa i lu re " s ta tus . R e s e a r c h has long reported that k n o w l e d g e of letter names and fo rmat ion is pred ic t ive of a c a d e m i c a n d , in part icular , read ing s u c c e s s (Busch , 1 9 8 0 ; Durre l l , 1 9 5 8 ; D y k s t r a , 1 9 6 7 ; J a n s k y & de H i r s c h , 1 9 7 2 ; S a t z , Tay lo r , Friel & F le tcher , 1979 ) . Devere l l , h o w e v e r , cons ide red the c a p a c i t y of an ear ly s c h o o l test bat tery to ident i fy the current ly required ins t ruct iona l level of the s tudent to be more important than its va lue as a pred ic tor of later ach ievemen t . Cons t r uc t i on of the D L N ind icated d i f ferent iat ion a m o n g s tuden ts by ex tend ing their pe r fo rmance a long a sca le f rom little or no unders tand ing of s y m b o l s to a we l l deve loped abi l i ty. Number recogn i t ion w a s in t roduced to reduce the lack of d i f ferent iat ion a m o n g very l o w let ter- ident i f icat ion s c o r e s . It w a s s h o w n that p re -schoo le rs learn number digi ts more easi ly than let ters, thus their inc lus ion he lped to di f ferent iate m a n y of the l owes t sco r ing ch i ld ren . A supp lementa ry w o r d recogni t ion test is s imi lar ly avai lable for use w i t h ch i ld ren at ta in ing h igh sco res in both letter and number k n o w l e d g e . The bat tery a lso inc ludes a four th f o rm , letter wr i t ing , w h i c h w a s not emp loyed by the target d ist r ic t . T h e bat tery is charac ter is t i c of a p o w e r test in that i tems of e a c h fo rm are ar ranged in a gradual ly ascend ing order of d i f f icu l ty . Fur ther , the fo rms t hemse l ves are presented in s imi lar order w i th number ident i f icat ion be ing cons ide red eas ier than upper case let ters, and the latter cons ide red eas ier than lower c a s e let ters. T h e author sugges t s opt imal t iming for test admin is t ra t ion to be in the spr ing of the k indergar ten year . The dist r ic t admin is tered th is tes t in M a y . 2 . Criterion Measures Cr i ter ion data w e r e prov ided by the Canad ian Tes ts of Bas i c Ski l ls (CTBS) w h i c h w a s rout inely admin is tered at year end by c l a s s r o o m teachers to all th i rd-grade s tuden ts . Retent ion of one or more years in a pr imary grade and a h is tory o f s c h o o l - b a s e d remedia l in tervent ion w e r e a lso cons ide red to be ind icat ive o f s c h o o l d i f f icu l ty and thus we re emp loyed as cr i ter ion ind ices . Canadian Tests of Basic Sl<ills (CTBS), 1976, (Level 9, Form 51. T h e C a n a d i a n T e s t s of Bas ic Ski l ls have been used w ide l y th roughout s c h o o l s y s t e m s in C a n a d a for s tandard ized tes t ing of genera l learning ach ievemen t . Admin is te red in M a y wi t l i in t l ie par t ic ipat ing dist r ic t , the mul t i level edi t ion for grades 3 -8 compr i ses a comprehens i ve bat tery us ing a mul t ip le -cho ice fo rmat . It measures ski l ls in f ive major areas cons ide red to over lap th roughout the g rades . These are vocabu la r y , language, reading c o m p r e h e n s i o n , w o r k - s t u d y and mathemat i cs sk i l ls . A compos i t e sco re is ca lcu la ted and w a s e m p l o y e d in th is s tudy . R a w sco res are conve r ted to deve lopmen ta l s c o r e s (grade equ iva lents) or to s ta tus s c o r e s (stanines or percent i le ranks) . It w a s usefu l to th is s t u d y to se lec t percent i le ranks for compara t i ve pu rposes . Pe r fo rmance w a s cons ide red to be sa t i s fac to ry above the 2 5 t h percent i le , and therefore unsa t i s fac to ry at or b e l o w the 25 th percent i le . The C T B S are an adapta t ion of the Iowa Tes t s of Bas i c Ski l ls (L indquist , H i e ronymus , & Hoove r , 1 9 5 5 ) . The des ign and cons t ruc t i on o f the C a n a d i a n tes ts we re deve loped by the staf f at the Un ivers i t y o f I owa 's Co l lege of Educa t ion under the d i rec t ion of Dr . Ethel K ing of the Un ivers i ty o f Ca lga ry . A number o f rev is ions we re n e c e s s a r y to create tes ts su i tab le for use in Canad ian s c h o o l s . T e a c h e r ' s gu ides , manua ls for admin is t ra tors , supe rv i so rs , and c o u n s e l o r s , t echn ica l manua ls , and other mater ia ls have been pub l i shed . Me t r i c ed i t ions are ava i lab le . Forms 1 and 2 of the C T B S w e r e pub l ished in 1 9 6 6 , Fo rms 3 and 4 in 1 9 7 4 , and Fo rm 5 in 1 9 8 3 . The 1 9 7 3 s tandard iza t ion w a s based on a strat i f ied random samp le of more than 3 0 , 0 0 0 s tuden ts f rom 2 2 5 s c h o o l s se lec ted to represent C a n a d a ' s Eng l ish-dominan t s c h o o l popu la t ion (B i rch, 1 9 7 2 ) . Rel iabi l i ty coe f f i c ien ts for the sub tes ts w e r e found to fall be tween .88 and . 9 8 . Va l id i t y da ta are not repor ted . It is sugges ted that va l id i ty be judged in relat ion to des i red purpose and rests upon the inst ruct ional ob jec t ives o f indiv idual s c h o o l s or reg ions. The s tandard iza t ion w a s c o n d u c t e d joint ly w i th that of the C a n a d i a n Cogn i t i ve Ab i l i tes Tes t (Thorndike and H a g e n , 1984) to a t tempt interpretat ion o f ach ievemen t test s c o r e s in relat ion to pe r fo rmance at the s a m e grade and s a m e cogn i t i ve apt i tude leve l . A simi lar nat ional s tandard iza t ion program w a s carr ied out in the fall o f 1 9 8 0 invo lv ing 3 1 , 1 3 7 s tuden ts f rom 1 0 2 Canad ian s c h o o l s under the d i rec t ion of Edgar Wr igh t of the To ron to , Ontar io Board of Educa t ion (Thorndike and H a g e n , 1 9 8 4 ) . B i rch (1972) cons ide rs the techn ica l s ta tus of the C T B S to be undoubtab le due to its respec ted an teceden t . Sa lv ia and Y s s e l d y k e (1981) report the deve lopmen t of the ITBS to be sa t i s fac to ry and its rel iabil i ty to be adequate w h e n based on internal c o n s i s t e n c y . In con t ras t , But ler (1981) argues that the C T B S does not sa t is fac tor i l y measure Canad ian con ten t and that a t eache r ' s persona l k n o w l e d g e of the comprehens i ve range of a s tuden t ' s ski l ls and interests far o u t w e i g h s w h a t he cons ide rs to be cu l tura l ly -b iased resul ts of a test of ques t ionab le rel iabil i ty and val id i ty . E. PROCEDURE FOR D A T A COLLECTION AND CODING Data w a s ob ta ined for 1 9 8 3 - 8 4 k indergar ten entry pupi ls v ia perusal of ind iv idual pe rmanen t record cards and s tudent f i les at e a c h of 31 s c h o o l s . Sub jec ts w e r e ass iged a numer ica l ident i f icat ion rather than l isted by name in order t o main ta in conf ident ia l i ty and to ass is t in da ta ana lys is . Permanen t record cards prov ided in fo rmat ion regarding e a c h s u b j e c t ' s history of remedia l in tervent ion or re tent ion, as we l l as the compos i t e percent i le rank sco re at ta ined on the th i rd-grade cr i ter ion measure of ach ievemen t . T h e Canad ian T e s t s of Bas i c Ski l ls (CTBS) . Pe r fo rmance on the C T B S w a s c o n s i d e r e d to be sa t i s fac to ry above the 25 th percent i le , the reby unsat is fac to ry at the 2 5 t h percent i le or b e l o w . "A t - r i s k " s tudents w e r e more l ikely to be ident i f ied w i th in th is range. S tuden t f i les w e r e a lso rev iewed for ind icat ions of educa t iona l in te rvent ion . A s c h o o l s u r v e y l ist ing e a c h sub ject w a s comp le ted by current learning ass i s t ance teachers to cor robora te or add to a k n o w n h is tory of remedia l in tervent ion exper ienced by the sub jec ts i nvo l ved . Remed ia l in tervent ion w a s , for the pu rposes of this s t u d y , inc lus ive of learning a s s i s t a n c e , s p e e c h and language the rapy , d iagnos t i c t e a c h i n g , and training in Engl ish as a S e c o n d Language . S tuden t f i les a lso supp l ied s u m m a r y shee ts ind icat ing s c o r e s ob ta ined by each sub jec t for the fo l low ing k indergar ten sc reen ing measu res : D r a w - A - P e r s o n ( D A P ) , K indergar ten Language Sc reen ing Tes t (KLST) , Mann -Su i t e r V i s u a l M o t o r S c r e e n ( M S ) , and the Devere l l Tes t o f Let ters and Numbers (DLN) . The above in format ion w a s entered on the Un ivers i ty Ma in f rame compu t i ng s y s t e m and the Sta t is t ica l Package for S o c i a l S c i e n c e - E x t e n d e d V e r s i o n ( S P S S X ) , Re lease 3 . 0 w a s emp loyed to ass is t data ana lys i s . Sa t i s fac to ry sco res as p rev ious ly desc r ibed for e a c h sc reen ing measure were then c o d e d as 0 , unsa t i s fac to ry sco res as 1. C o m p o s i t e sc reen ing c lass i f i ca t ions w e r e c o d e d as 0 to represent "non- r i sk " or as 1 to represent " r i s k " w h e n t w o or more ind iv idual measures w e r e fa i led . C o d i n g in th is manner ref lects the or ig in of risk ident i f icat ion w i th in the med ica l f ield w h e r e posi t ive resul ts infer p rob lem ex i s tence . F. D A T A ANALYSIS T h e Un ivers i ty C o m p u t i n g Cen t re ' s S S P S X program w a s emp loyed to ob ta in 2 X 2 p red ic t ion-per fo rmance mat r ices used to invest igate the re lat ionship b e t w e e n k indergar ten sc reen ing and th i rd-grade ach ievemen t measu res . For e a c h k indergar ten sc reen ing measure as we l l as the c o m p o s i t e sc reen ing c lass i f i ca t ion (predict ive measures ) , c ross tabu la t ions w i th the cr i ter ion ach ievemen t test (per formance measure) p rov ided dec is ion matr ices ind icat ing the number of sub jec ts w h o a t ta ined : Quad ran t A : unsa t i s fac to ry resul ts o n the predic t ive measu re ; unsa t i s fac to ry resul ts on the cr i ter ion measure (true pos i t i ves , T + ) Quad ran t B: unsa t i s fac to ry resul ts on the pred ic t ive measu re ; sa t i s fac to ry resul ts on the cr i ter ion measure (false pos i t i ves , F-I-) Quadran t C : sa t i s fac to ry resul ts on the pred ic t ive measu re ; unsa t i s fac to ry resul ts o n the cr i ter ion measure (false nega t i ves , F-). Quad ran t D: sa t i s fac to ry resul ts on the pred ic t ive measu re ; sa t i s fac to ry resul ts on the cr i ter ion measure (true nega t i ves , T-) T w o fur ther mat r ices w e r e deve loped to invest igate the re lat ionship b e t w e e n the c o m p o s i t e sc reen ing c lass i f i ca t ion and s tudent h is tory o f s c h o o l - b a s e d remedia l in tervent ion as we l l as s tuden t h is tory of retent ion of one or more yea rs in a pr imary g rade . Both hor izonta l and ver t ica l eva lua t ion of e a c h nnatrix a l l owed manua l tabulat ion of pe rcen tages indicat ive of the pred ic t ive a c c u r a c y (utility) of the sc reen ing c lass i f i ca t ion dec is ions (r isk/no-r isk) inc lud ing sens i t iv i ty and spec i f i c i t y ind ices . A l s o ca lcu la ted for compar i son purposes w e r e the base rates (the p reva lence of a part icular p rob lem cond i t ion w i th in a g iven popula t ion) , referral ra tes (percentage of s tuden ts c lass i f ied by sc reen ing as at- r isk) , and overal l hit ra tes (percentage of sub jec ts accura te ly ident i f ied both at-r isk and non-r isk) . IV. RESULTS A . INTRODUCTION Th is chapter p resents the resul ts of dec is ion matr ix ana lys is ut i l ized to invest igate the predic t ive val id i ty and uti l i ty of four k indergar ten sc reen ing measures and compos i t e sc reen ing c lass i f i ca t ion dec i s i ons . The chapter a lso desc r ibes s e v e n ach ieved samp les generated by the avai lable da ta . D i s c u s s i o n of the f ind ings and recommenda t i ons for future research are presented in Chap te r 5 . B. ACHIEVED SAMPLES Data w a s co l lec ted for 1 9 8 3 - 8 4 k indergar ten entry s tuden ts w h o remained enrol led in the target d ist r ic t in 1 9 8 6 - 8 7 . The 6 8 4 sub jec ts of the s tudy inc luded those ch i ld ren w h o s e records prov ided k indergar ten sc reen ing data for one or more of the four measures admin is te red and th i rd-grade s tandard ized ach ievemen t test resu l ts . Add i t i ona l data inc luded h is tory of s c h o o l - b a s e d in tervent ion or re tent ion. A l l sub jec ts did not have a sco re for e a c h sc reen ing measure , therefore the number o f sub jec ts per ach ieved samp le var ied (Table 1). Table 1. Number of Subjects in Seven Achieved Samples F Predictor Criterion 576 D A P C T B S 601 K L S T C T B S 1 596 M S C T B S || 606 D L N C T B S 592 Screen ing C o m p o s i t e C T B S 663 Screen ing C o m p o s i t e Retent ion S ta tus 1 663 Screen ing C o m p o s i t e H is tory of Intervent ion C. RESULTS Resu l ts of dec i s ion matr ix ana lyses are presented for four research ques t ions . Referral ra tes , base rates, overa l l hit ra tes, and hor izonta l and ver t ica l eva lua t ions are reported in pe rcen tages . T h e referral rate represents the percen tage of s tuden ts predic i ted by sc reen ing to be at-r isk. The base rate represents the ac tua l percen tage of s tuden ts w h o s e cr i ter ion per fo rmance ind icated learning p rob lems. Four poss ib le re la t ionships ex is t b e t w e e n sc reen ing and cr i ter ion pe r fo rmance : True Pos i t i ve (T-I-) Fa lse Pos i t i ve (F-i-) True Negat i ve (T-) Fa lse Negat i ve (F-) poor sc reen ing /poor cr i ter ion poor sc reen ing /good cr i ter ion (over-referral) good sc reen ing /good cr i ter ion good sc reen ing /poor cr i ter ion (under-referral) . Cut -o f f or dec is ion points b e l o w w h i c h per fo rmance w a s cons ide red to be unsa t i s fac to ry we re pre-set by the dist r ic t for the indiv idual k indergar ten sc reen ing measu res . The D r a w - A - P e r s o n emp loyed distr ic t norms deve loped f rom prev ious admin is t ra t ion of the measure . T h e cut -o f f for th i rd-grade a c h i e v e m e n t e n c o m p a s s e d those ch i ldren per forming w i th in the bo t tom quart i le. A s tudent w a s cons ide red to be at-r isk w h e n per forming unsat is fac tor i l y o n t w o or more sc reen ing measures . 1. Predictive Accuracy of Individual Kindergarten Screening Measures Ques t i on 1 : H o w accura te ly does e a c h of the four k indergar ten sc reen ing measures indiv idual ly predict s tuden ts at-r isk or not at-r isk for future s c h o o l d i f f icul t ies as measured by th i rd-grade per fo rmance on a s tandard ized ach ievemen t test (Canad ian Tes t s of Bas ic Sk i l l s , or C T B S ) ? a. Dra w-A -Person Dec is ion matr ix ana lys is of the resul ts of the D r a w - A - P e r s o n and C T B S admin is t ra t ion are presented in Figure 3 . Figure 3 . Draw-A-Person: Accuracy of RisIt Identification per C T B S Performance Utilizing Decision Matrix Analysis (n = 576) D r a w - A - P e r s o n S c r e e n i n g C T B S Pe r fo rmance Unsa t i s fac to ry Sa t i s fac to ry 6 (T + ) 19 {F + ) Risl< * H (24%) H (76%) 2 5 * * V (8%) V (4%) (Sensi t iv i ty) (over-referral) 6 9 (F-) 4 8 2 (T-) No-Risic H (13%) H (87%) 551 V (92%) V (96%) (under-referral) (Speci f ic i ty) 7 5 501 n = 5 7 6 Referral Rate = 4 % Base Rate = 1 3 % Overa l l Hit Rate = 8 5 % * H = Hor izon ta l eva lua t ion * * V = Ver t i ca l eva lua t ion True Pos i t i ves : F igure 3 ind icates that 4 % of the ach ieved k indergar ten samp le (25 of n) w e r e pred ic ted to be at-r isk. Of t hose , 2 4 % (6 of 25) w e r e accu ra te l y ident i f ied. T h e s e sub jec ts represent 8 % (6 of 75) of t hose s tuden ts w h o ac tua l l y per fo rmed unsat is fac tor i l y o n the cr i ter ion measu re . Th is is the rate o f sens i t i v i t y of the sc reen ing measure . Fa lse Pos i t i ves : C o n v e r s e l y , of t hose pred ic ted to be at-r isk, 7 6 % (19 of 25 ) w e r e non-r isk and therefore inaccura te ly ident i f ied or over - re fer red. T h e s e sub jec ts represent 4 % (19 of 501) over-referra l of those s tuden ts w h o ac tua l ly pe r fo rmed sa t is fac tor i l y on the cr i ter ion measure . True Nega t i ves : O f those pred ic ted to be non-r isk, 8 7 % (482 of 551 ) w e r e accura te l y ident i f ied. T h e s e sub jec ts represent 9 6 % (482 of 501) of t hose s tuden ts w h o actua l ly per fo rmed sat is fac tor i l y on the cr i ter ion measure . Th is is the rate o f spec i f i c i t y of the sc reen ing measure . Fa lse Nega t i ves : C o n v e r s e l y , o f t hose pred ic ted to be non-r isk, 1 3 % (69 of 551) w e r e at-r isk and therefore inaccura te ly ident i f ied or under- re fer red. T h e s e sub jec ts represent 9 2 % (69 of 75) under-referral of t hose s tuden ts w h o ac tua l ly per formed unsat is fac tor i ly on the cr i ter ion measure . A p rob lem base rate o f 1 3 % w a s ca l cu la ted . The overa l l hit rate equal led 8 5 % . b. Kindergarten Language Screening Test Dec i s i on matr ix ana lys is of the resul ts of the K indergar ten Language Sc reen ing Tes t and C T B S admin is t ra t ion are presented in Figure 4 . Figure 4. Kindergarten Language Screening Test: Accuracy of Risic Identification per CTBS Performance Utilizing Decision IVIatrix Analysis (n = 601) Kindergar ten Language Sc reen ing T e s t C T B S Pe r fo rmance Unsa t i s fac to ry Sa t i s fac to ry Risk 15 (T + ) * H (36%) * * V (20%) (Sensi t iv i ty) 2 7 (F + ) H (64%) V (5%) (over-referral) 4 2 61 (F-) 4 9 8 (T-) No -R isk H (11%) H (89%) 5 5 9 V (80%) V (95%) (under-referral) (Speci f ic i ty) 7 6 5 2 5 n = 6 Referral Rate = 7 % Base Rate = 1 3 % Overa l l Hit Rate = 8 5 % * H = Hor izonta l eva lua t ion * * V = Ver t i ca l eva lua t ion T rue Pos i t i ves : F igure 4 ind icates that 7 % of the ach ieved k indergar ten samp le (42 of n) w e r e pred ic ted to be at-r isk. Of t h o s e , 3 6 % (15 Of 42) w e r e accu ra te l y ident i f ied. T h e s e sub jec ts represent 2 0 % (15 of 76) of t hose s tuden ts w h o ac tua l l y per fo rmed unsat is fac tor i l y on the cr i ter ion measu re . Th is is the rate of sens i t i v i t y of the sc reen ing measu re . Fa lse Pos i t i ves : C o n v e r s e l y , of t hose pred ic ted to be at-r isk, 6 4 % (27 of 42 ) w e r e non-r isk and therefore inaccura te ly ident i f ied or over - re fer red. These sub jec ts represent 5 % (27 of 525) over-referra l of t hose s tuden ts w h o actua l ly per fo rmed sa t is fac tor i l y on the cr i ter ion measure . True Nega t i ves : O f those predic ted to be non-risl<, 8 9 % (498 of 559) w e r e accura te ly ident i f ied. T h e s e sub jects represent 9 5 % (498 of 525) of t hose w h o s tudents ac tua l l y per fo rmed sat is factor i ly on the cr i ter ion measure . Th is is the rate of spec i f i c i t y o f the sc reen ing measure . Fa lse Nega t i ves : C o n v e r s e l y , of t hose pred ic ted to be non-r isk, 11 % (61 of 559) w e r e at-r isk and therefore inaccura te ly ident i f ied or under- re fer red. T h e s e sub jects represent 8 0 % (61 of 76) under-referral o f t hose s tudents w h o ac tua l ly per formed unsat is fac tor i l y on the cr i ter ion m e a s u r e . A p rob lem base rate of 1 3 % w a s ca l cu l a ted . T h e overal l hit rate equa l led 8 5 % . c. Mann-Suiter Visual Motor Screen Dec i s i on matr ix ana lys is of the resul ts of the Mann -Su i t e r V i sua l M o t o r S c r e e n and C T B S admin is t ra t ion are presented in F igure 5 . Figure 5. Mann-Suiter Visual Motor Screen : Accuracy of Risk Identification per CTBS Performance Utilizing Decision Matrix Analysis (n = 596) M a n n - S u i t e r V i s u a l M o t o r S c r e e n C 1 B S Pe r fo rmance Unsa t i s fac to ry Sa t i s fac to ry 18 (T-^) 5 6 {F + ) Risk * H (24%) * * V (24%) (Sensi t iv i ty) H (76%) V (11%) (over-referral) 7 4 5 8 (F-) 4 6 4 (T-) No-R isk H (11%) V (76%) H (89%) V (89%) 5 2 2 (under-referral) (Speci f ic i ty) 7 6 5 2 0 n = 5 9 6 Referral Rate = 1 2 % Base Rate = 1 3 % Overa l l Hit Rate = 8 1 % * H = Hor izonta l eva lua t ion * * V = Ver t i ca l eva lua t ion True Pos i t i ves : F igure 5 ind icates that 1 2 % of the ach ieved k indergar ten samp le (74 of n) w e r e pred ic ted to be at-r isk. Of t h o s e , 2 4 % (18 of 74) w e r e accura te l y ident i f ied. T h e s e sub jec ts represent 2 4 % (18 of 76) of t hose s tuden ts w h o ac tua l l y per fo rmed unsat is fac tor i l y on the cr i ter ion measure . Th is is the rate of sens i t iv i ty of the sc reen ing measu re . Fa lse Pos i t i ves : C o n v e r s e l y , of t hose pred ic ted to be at-r isk, 7 6 % (56 of 74) w e r e non-r isk and therefore inaccura te ly ident i f ied or over - re fe r red . These sub jec ts represent 11 % (56 of 520) over- referra l of t h o s e s tuden ts w h o actual ly per fo rmed sa t is fac tor i l y on the cr i ter ion measure . True Nega t i ves : Of those pred ic ted to be non-r isk , 8 9 % (464 of 522) w e r e accura te ly ident i f ied. These sub jec ts represent 8 9 % (464 of 520) of t hose s tuden ts w h o actua l ly per formed sat is fac tor i l y o n the cr i ter ion measure . Th is is the rate o f spec i f i c i t y o f the sc reen ing measu re . Fa lse Nega t i ves : C o n v e r s e l y , o f those p red ic ted to be non-r isk, 11 % (58 of 522) w e r e at-r isk and therefore inaccura te ly ident i f ied or under- re fer red. T h e s e sub jec ts represent 7 6 % (58 of 76) under-referra l of those s tudents w h o ac tua l ly per fo rmed unsat is fac tor i ly on the cr i ter ion m e a s u r e . A p rob lem base rate of 1 3 % w a s c a l c u l a t e d . The overal l hit rate equal led 8 1 % . d. Deverell Test of Letters and Numbers (Deverell) Dec is i on matr ix ana lys is of the resul ts of the Devere l l Tes t of Let ters and Numbers and C T B S admin is t ra t ion are presented in Figure 6. Figure 6. Deverell Test of Letters and Numbers: Accuracy of Risk Identification per CTBS Performance Utilizing Decision Matrix Analysis (n = 606) II C T B S Pe r fo rmance Devere l l S c r e e n i n g Unsa t i s fac to ry Sa t i s fac to ry 11 (T-^) 9 (F-I-) Risk * H (55%) * * V (14%) (Sensit iv i ty) H (45%) V (2%) (over-referral) 2 0 6 9 (F-) 5 1 9 (T-) No-R isk H (11%) V (86%) H (89%) V (98%) 5 8 6 (under-referral) (Speci f ic i ty) 7 8 5 2 8 n = 6 Referral Rate = 3 % Base Rate = 1 3 % Overa l l Hit Rate = 8 7 % * H = Hor izonta l eva lua t ion * * V = Ver t i ca l eva lua t ion True Pos i t i ves : F igure 6 ind icates that 3 % of the ach ieved k indergar ten samp le (20 of n) w e r e pred ic ted to be at-r isk. O f t h o s e , 5 5 % (11 of 20) w e r e accura te ly ident i f ied. T h e s e sub jec ts represent 1 4 % (11 of 78) o f t hose s tuden ts w h o actua l ly per fo rmed unsat is fac tor i ly on the cr i ter ion measu re . Th is is the rate of sens i t iv i ty of the sc reen ing measure . Fa lse Pos i t i ves : C o n v e r s e l y , of t hose pred ic ted to be at-r isk, 4 5 % (9 of 20) w e r e non-r isk and therefore inaccura te ly ident i f ied or over - re fe r red . T h e s e sub jects represent 2 % (9 of 528 ) over-referra l of t hose s tuden ts w h o ac tua l l y per formed sat is fac tor i l y on the cr i ter ion measure . True Nega t i ves : Of t l i ose pred ic ted to be non-risic, 8 9 % (519 of 586 ) w e r e accura te ly ident i f ied. T h e s e sub jects represent 9 8 % (519 of 528) of t h o s e w h o actual ly per fo rmed sat is fac tor i ly on the cr i ter ion measure . Th is is the rate o f spec i f i c i t y of the sc reen ing measure . Fa lse Nega t i ves : C o n v e r s e l y , of t hose pred ic ted to be non-r isk, 11 % (67 of 586) w e r e at-r isk and therefore inaccura te ly ident i f ied or under- re fer red. T h e s e sub jec ts represent 8 6 % (67 of 78) under-referral of those s tudents w h o ac tua l ly per formed unsat is fac tor i l y on the cr i ter ion measu re . A p rob lem base rate o f 1 3 % w a s ca l cu la ted . The overal l hit rate equa l led 8 7 % . 2. Predictive Accuracy of the Composite Screening Classification (Screening Composite) "Fa i l u re " of t w o or more of the indiv idual sc reen ing measures resul ted in the c o m p o s i t e sc reen ing c lass i f i ca t ion "a t - r i sk " . Referral for further d iagnos t i c a s s e s s m e n t w a s in tended . a. Screening Composite per CTBS Q u e s t i o n 2 : H o w accura te ly does the compos i t e sc reen ing c lass i f i ca t ion predict s tuden ts at-r isk or not at-r isk for future s c h o o l di f f icul t ies as measured by th i rd-grade pe r fo rmance o n a s tandard ized ach ievemen t test (CTBS)? Dec i s i on matr ix ana lys is of the c o m p o s i t e sc reen ing c lass i f i ca t ion and C T B S admin is t ra t ion are presented in Figure 7. Figure 7. Composite Screening Classification: Accuracy of RisIc Identification per CTBS Performance Utilizing Decision Matrix Analysis (n = 592) C T B S Pe r f o rmance Sc reen ing C o m p o s i t e Unsa t i s fac to ry Sa t i s fac to ry 11 (T + ) 16 (F + ) Risl< * H (41%) H (59%) 2 7 * * V (14%) V (3%) (Sensi t iv i ty) (over-referral) 6 5 (F-) 5 0 0 (T-) No-Risic H (12%) H (88%) 5 6 5 V (86%) V (97%) (under-referral) (Speci f ic i ty) 7 6 5 1 6 n = 5 Referral Rate = 5 % Base Rate = 1 3 % Overa l l Hit Rate = 8 6 % * H = Hor izon ta l eva luat ion * * V = Ver t i ca l eva lua t ion True Pos i t i ves : Figure 7 ind icates that 5 % of the a c h i e v e d k indergar ten samp le (27 of n) w e r e pred ic ted to be at-r isk. Of t hose , 4 1 % (11 of 27) we re accura te ly ident i f ied. T h e s e sub jec ts represent 1 4 % (11 of 76) o f t hose s tuden ts w h o ac tua l l y per formed unsat is fac tor i l y on the cr i ter ion measu re . Th is is the rate of sens i t i v i ty of the sc reen ing measure . Fa lse Pos i t i ves : C o n v e r s e l y , of t hose pred ic ted to be at- r isk, 5 9 % (16 of 27) w e r e non-r isk and therefore inaccura te ly ident i f ied or over - re fe r red . These sub jec ts represent 3 % (16 of 516) over-referra l of those s tuden ts w h o actua l ly per fo rmed sat is fac tor i l y on the cr i ter ion measu re . True Nega t i ves : Of those pred ic ted to be non-risl<, 8 8 % (500 of 565 ) w e r e accura te l y ident i f ied. These sub jec ts represent 9 7 % (500 of 516) of t h o s e s tuden ts w h o actua l ly per formed sat is fac tor i l y on the cr i ter ion measure . Th is is the rate of spec i f i c i t y of the sc reen ing measure . Fa lse Nega t i ves : C o n v e r s e l y , of those pred ic ted to be non-r isk, 1 2 % (65 of 565) w e r e at-r isk and therefore inaccura te ly ident i f ied or under- re fer red. T h e s e sub jec ts represent 8 6 % (65 of 76) under-referral of those s tuden ts w h o ac tua l ly per formed unsat is fac tor i ly on the cr i ter ion measu re . A p rob lem base rate of 1 3 % w a s ca l cu la ted . The overal l hit rate equa l led 8 6 % . b. Screening Composite per History of Intervention Ques t i on 3 : H o w accura te l y does the compos i t e sc reen ing c lass i f i ca t ion predict s tuden ts at-r isk or not at-r isk for future s c h o o l d i f f icul t ies as ind icated by s c h o o l - b a s e d in tervent ion dur ing pr imary g rades? Dec i s i on matr ix ana lys is of c o m p o s i t e sc reen ing c lass i f i ca t ion and h is tory of s c h o o l - b a s e d in tervent ion are presented in Figure 8 . Retent ion in a pr imary grade is cons ide red an in tervent ion. Figure 8. Composite Screening Classification: Accuracy of Risk Identification per History of Intervention Utilizing Decision Matrix Analysis (n = 663) His tory o f Intervent ion Sc reen ing C o m p o s i t e Intervent ion No Intervent ion 2 8 (T-I-) 10 (F + ) Risk * H (74%) H (26%) 3 8 * * V (15%) V (2%) (Sensit iv i ty) (over-referral) 1 5 9 (F-) 4 6 6 (T-) No-R isk H (25%) H (75%) 6 2 5 V (85%) V (98%) (under-referral) (Speci f ic i ty) 1 8 7 4 7 6 n = 6 Referral Rate = 6 % Base Rate = 2 8 % Overa l l Hit Rate = 7 5 % * H = Hor izon ta l eva lua t ion * * V = Ver t i ca l eva luat ion True Pos i t i ves : Figure 8 ind icates that 6 % of the ach ieved k indergar ten samp le (38 of n) w e r e pred ic ted to be at-r isk. Of t h o s e , 7 4 % (28 of 38) w e r e accura te ly ident i f ied. T h e s e sub jec ts represent 1 5 % (28 of 187) of t hose s tudents w h o ac tua l ly per fo rmed unsat is fac tor i ly on the cr i ter ion measu re . Th is is the rate of sens i t i v i ty of the sc reen ing measure . Fa lse Pos i t i ves : C o n v e r s e l y , of those pred ic ted to be at-r isk, 2 6 % (10 of 38) we re non-r isk and therefore inaccura te ly ident i f ied or over - re fer red. These sub jec ts represent 2 % (10 of 476 ) over-referral of t hose s tuden ts w h o actual ly per fo rmed sa t is fac tor i l y on the cr i ter ion measure . True Nega t i ves : Of those pred ic ted to be non-risl< 7 5 % (466 of 625) w e r e accura te ly ident i f ied. T h e s e sub jec ts represent 9 8 % (466 of 476 ) of t h o s e s tudents w h o actua l ly per formed sat is fac tor i l y on the cr i ter ion measure . T h i s is the rate of spec i f i c i t y o f the sc reen ing measure . Fa lse Nega t i ves : C o n v e r s e l y , of t hose pred ic ted to be non-r isk, 2 5 % ( 1 5 9 of 625 ) w e r e at-r isk and therefore inaccura te ly ident i f ied or under- re fer red. T h e s e sub jec ts represent 8 5 % ( 1 5 9 of 187) under-referral of t hose s tudents w h o ac tua l l y per formed unsat is fac tor i l y on the cr i ter ion measure . A p rob lem base rate of 2 8 % w a s ca l cu la ted . The overal l hit rate equa l led 7 5 % . c. Screening Composite per Retention Status Ques t i on 4 : H o w accura te ly does the c o m p o s i t e sc reen ing c lass i f i ca t ion predict s tuden ts at-r isk or not at-r isk for future s c h o o l d i f f icul t ies as ind icated by retent ion of one or more years in a pr imary grade? Dec is ion matr ix ana lys is of compos i t e sc reen ing c lass i f i ca t ion and retent ion s ta tus are presented in Figure 9 . Figure 9. Composite Screening Classification: Accuracy of Risk Identification per Retention Status Utilizing Decision Matrix Analysis (n = 663) Reten t ion S ta tus || S c r e e n i n g C o m p o s i t e Reta in P romote 1 6 {T + ) 2 2 (F-J-) Risk * H (42%) H (58%) 3 8 * * V (23%) V (4%) (Sensi t iv i ty) (over-referral) 5 5 (F-) 5 7 0 (T-) No -R i sk H (9%) H (91%) 6 2 5 V (77%) V (96%) 1 (under-referral) (Speci f ic i ty) 71 5 9 2 n = 6 6 3 Referral Rate = 6 % Base Rate = 1 1 % Overa l l Hit Rate = 8 8 % * H = Hor izon ta l eva lua t ion * * V = Ver t i ca l eva lua t ion T rue Pos i t i ves : F igure 9 ind icates that 6 % of the ach ieved k indergar ten samp le (38 of n) we re pred ic ted to be at-r isk. Of t h o s e , 4 2 % (16 of 38) w e r e accura te l y ident i f ied. T h e s e sub jec ts represent 2 3 % (16 of 71 ) of t hose s tuden ts w h o ac tua l l y per fo rmed unsat is fac tor i l y o n the cr i ter ion measu re . Th is is the rate of sens i t i v i t y of the sc reen ing measure . Fa lse Pos i t i ves : C o n v e r s e l y , of t hose pred ic ted to be at-r isk, 5 8 % (22 of 38) w e r e non-r isk and therefore inaccura te ly ident i f ied or over - re fer red. T h e s e sub jec ts represent 4 % (22 of 592) over-referra l of t hose s tuden ts w h o actua l ly per fo rmed sa t is fac tor i l y on the cr i ter ion measure . True Nega t i ves : Of t l i ose pred ic ted to be non-risIc, 91 % (570 of 625 ) w e r e accura te ly ident i f ied. T h e s e sub jec ts represent 9 6 % (570 of 592) of t hose w h o s tuden ts ac tua l ly per formed sat is fac tor i l y on the cr i ter ion measure . Th is is the rate of spec i f i c i t y o f the sc reen ing measure . Fa lse Nega t i ves : C o n v e r s e l y , of t hose pred ic ted to be non-r isk , 9 % (55 of 625) w e r e at-r isk and therefore inaccura te ly ident i f ied or under- re fer red. T h e s e sub jec ts represent 7 7 % (55 of 71) under-referral of those s tuden ts w h o actua l ly per fo rmed unsat is fac tor i ly on the cr i ter ion measu re . A p rob lem base rate of 11 % w a s ca l cu la ted . The overal l hit rate equal led 8 8 % . 3 . S u m m a r y of Resu l ts The a c c u r a c y of k indergar ten sc reen ing for predic t ing s tuden ts at-r isk or not not at-r isk for future s c h o o l d i f f icul t ies is summar i zed in Tab les 2 and 3 . Tab le 2 presents the s u m m a r y of resul ts as pe rcen tages . Table 2. Kindergarten Screening: Accuracy of Risk Identification for School Achievement, Intervention, and Retention Utilizing Decision Matrix Analysis (Presented as Percentages) True Pos i t i ves (Sensit iv i ty)^ Fa l se Pos i t i ves (over-referral) T rue Nega t i ves (Speci f ic i ty)^ Fa lse Nega t i ves (under-referraî) Referral Rate Prob lem Base Rate Overa l l Hit Rate Pred ic to r / Cr i ter ion H * v * H V H V H V D A P / C T B S 2 4 8 7 6 4 8 7 9 6 13 9 2 4 13 8 5 K L S T / C T B S 3 6 2 0 6 4 5 8 9 9 5 11 8 0 7 1 3 8 5 M S / C T B S 2 4 2 4 7 6 11 8 9 8 9 11 7 6 12 13 81 D L N / C T B S 5 5 14 4 5 2 8 9 9 8 11 8 6 3 13 8 7 C o m p o s i t e / C T B S 41 14 5 9 3 8 8 9 7 12 8 6 5 13 8 6 C o m p o s i t e / Intervent ion 7 4 15 2 6 2 7 5 9 8 2 5 8 5 6 2 8 75 C o m p o s i t e / Retent ion 4 2 2 3 5 8 4 91 9 6 9 7 7 6 11 8 8 * H = Hor izonta l Eva lua t ion * * V = Ver t i ca l Eva luat ion 1 = Sens i t i v i ty (True Pos i t i ve , V) 2 = Spec i f i c i t y (True Nega t i ve , V) Prob lem base rates of 1 3 % , 2 8 % , and 11 % w e r e ca lcu la ted w i th re ference to C T B S , in tervent ion, and retent ion cr i ter ion respec t i ve ly . S imi lar ly , respec t i ve referral rates of 5 % , 6 % , and 6 % were ca lcu la ted f rom sc reen ing c o m p o s i t e c lass i f i ca t ions . A l l referral rates w e r e less than their respect ive base ra tes, ind icat ing genera l under-referra l . Ver t i ca l eva luat ion (actual ou tcome) revealed greater under-referral rates than did hor izonta l eva luat ion (predicted ou tcome) w h i c h ind icated greater over- referra l rates. Over- referra l o c c u r s w h e n pred ic t ion of r isk is incor rect . Under- referra l o c c u r s w h e n pred ic t ion of non-r isk is incor rec t . Wh i le both hor izonta l and ver t ica l eva lua t ion appeared to ind icate greater sc reen ing a c c u r a c y for ident i f icat ion of non-r isk s tuden ts than at-r isk s tuden ts , ver t ica l eva lua t ion demons t ra ted the far greater rates of spec i f i c i t y (true non-r isk) than sens i t i v i ty (true risk) w h i c h ac tua l ly ex i s t ed . Overa l l hit rates appeared to be h igh , ranging f rom 7 5 % - 8 8 % . Tab le 3 presents the s u m m a r y of resul ts in abso lu te numbers represent ing indiv idual c a s e s . Table 3. Kindergarten Screening: Accuracy of RisIc Identification for School Achievement, Intervention, and Retention Utilizing Decision IVIatrix Analysis (Presented in Absolute Numbers) Pred ic to r / Cr i ter ion T + P + T- F- Refer N o n -Refer Unsa t i s fac to ry Cr i ter ion Sa t i s fac to ry Cr i ter ion n D A P / C T B S 6 19 4 8 2 6 9 2 5 551 7 5 501 5 7 6 K L S T / C T B S 15 2 7 4 9 8 61 4 2 5 5 9 7 6 5 2 5 601 M S / C T B S 18 5 6 4 6 4 5 8 7 4 5 2 2 7 6 5 2 0 5 9 6 D L N / C T B S 11 9 5 1 9 6 7 2 0 5 8 6 7 8 5 2 8 6 0 6 C o m p o s i t e / C T B S 11 1 6 5 0 0 6 5 2 7 5 6 5 7 6 5 1 6 5 9 2 C o m p o s i t e / Intervent ion 2 8 10 4 6 6 1 5 9 3 8 6 2 5 1 8 7 4 7 6 6 6 3 C o m p o s i t e / 1 Retent ion 16 2 2 5 7 0 5 5 3 8 6 2 5 71 5 9 2 6 6 3 Presenta t ion o f abso lu te numbers a l l ows c o m p a r i s o n of resul ts and c lar i f ies interpretat ion of pe rcen tages . For examp le , wh i le 2 8 of 3 8 s tudents (74%) predic ted by the c o m p o s i t e sc reen ing c lass i f i ca t ion to be at-r isk rece ived in tervent ion se rv i ce , 1 5 9 other s tuden ts w h o required intervent ion w e r e not ident i f ied. Th is represents 8 5 % under-referra l . in real i l ty, 1 5 9 ch i ldren did not benefi t f r om ear ly ident i f icat ion by this measure . If the pred ic t ion had been co r rec t for 3 8 of 3 8 (100%) referred s tuden ts , under-referral w o u l d sti l l be h igh (159 of 1 9 7 , or 8 1 % ) . D. SUIVIMARY The a c c u r a c y of k indergar ten sc reen ing for p red ic t ing s tuden ts at-r isk or not at-r isk for fu ture s c h o o l d i f f icul t ies w a s inves t iga ted ut i l iz ing dec is ion matr ix ana lys is . Resu l ts of hor izonta l and ver t ica l eva lua t ion w e r e p resen ted , as we l l as referral ra tes , p rob lem base rates, and overa l l hit ra tes. D i s c u s s i o n of the f ind ings and recommenda t i ons for future research are p resented in Chap te r 5 . V . SUMMARY AND CONCLUSIONS A . S U M M A R Y The present s tudy invest igated the va l id i ty and uti l i ty o f four k indergar ten sc reen ing measures and their c o m p o s i t e sc reen ing c lass i f i ca t ion as p red ic to rs of th i rd-grade ach ievemen t . H is tory of in tervent ion and retent ion s ta tus w e r e cons ide red to be addi t ional ind ices of s c h o o l per fo rmance and their re la t ionsh ip w i th k indergar ten sc reen ing resul ts w a s a lso inves t iga ted . Dec is ion matr ix ana lys i s w a s ut i l ized to examine the pred ic t ive a c c u r a c y of r isk and non-r isk sc reen ing c lass i f i ca t ions . Resu l ts inc luded hor izonta l and ver t ica l eva lua t ions , p rob lem base rates, referral ra tes , and overal l hit ra tes. The k indergar ten sc reen ing p rogram w a s imp lemented by one s c h o o l d is t r ic t to a l l o w for the ear ly ident i f icat ion of s tuden ts at-r isk for future learning p rob lems . S u b s e q u e n t in tervent ion w a s in tended to prevent , reduce , and /or e l iminate learning d i f f icu l t ies. Sc reen ing resul ts in f luenced the a l locat ion of spec ia l se rv i ces and are therefore l inked w i t h expend i tu res of moneta ry and personne l resou rces . The samp le o f 6 8 4 sub jec ts compr i sed 1 9 8 6 - 8 7 s tuden ts remain ing in the d is t r ic t s i nce k indergar ten entry in 1 9 8 3 - 8 4 . S e v e n ach ieved samp les a l l owed examina t ion of s e v e n p red ic t ion-per fo rmance ana l yses . The number of sub jec ts per a c h i e v e d samp le va r i ed , ranging f rom 5 7 6 to 6 6 3 . The present invest iga t ion exam ined the re lat ionship b e t w e e n four indiv idual k indergar ten sc reen ing measu res and th i rd-grade ach ievemen t as measured by the C a n a d i a n T e s t s of Bas i c Sk i l ls ( C T B S ) . The four sc reen ing measures inc luded the D r a w - A - P e r s o n , the K indergar ten Language S c r e e n i n g Tes t , the Mann-Su i t e r V i s u a l M o t o r S c r e e n (Mann-Su i te r ) , and the Devere l l T e s t of Letters and Numbers (Deverel l ) . Th is invest igat ion a lso examined the re lat ionship b e t w e e n the c o m p o s i t e sc reen ing c lass i f i ca t ion (Screen ing Compos i t e ) and ach ievemen t , h is tory of s c h o o l - b a s e d in tervent ion, and retent ion s ta tus . B. DISCUSSION The major f ind ings of the s tudy are s u m m a r i z e d by the f o l l ow ing . 1. For all ana l yses , sc reen ing referral rates w e r e less than their respec t i ve p rob lem base ra tes, ind icat ing genera l under-referral of at-r isk s tuden ts . Resu l ts for all p red ic t ion-per fo rmance ana l yses in this s tudy ind icated that p rob lem base rates (derived f rom cr i ter ion per formance) w e r e greater than a c c o m p a n y i n g referral rates (derived f rom sc reen ing per fo rmance) . Nei ther the indiv idual k indergar ten sc reen ing measures nor the compos i t e sc reen ing c lass i f i ca t ion we re able to se lec t for referral an equ iva lent or greater number of s tuden ts than w a s ac tua l ly found to be at-r isk. B e c a u s e per fect pred ic t ion is not expec ted of sc reen ing measu res , a propor t ion o f referrals shou ld prove to be in error. T o a l l ow for the possib l i l i ty that all at-r isk s tuden ts be ident i f ied, it is typ ica l ly argued that the referral rate shou ld be greater than the base rate (L ichtenste in & Ireton, 1 9 8 4 ) . Th is presents an inherent d i f f icu l ty regard ing the a c c u r a c y of r isk ident i f icat ion poss ib le by the sc reen ing measures e m p l o y e d in this s tudy . Under-referra l of t ru ly at-r isk s tuden ts w o u l d be expec ted a n d , t hus , in f luence the resul ts of the inves t iga t ion . 2. For all a n a l y s e s , ver t ica l eva luat ion (actual ou tcome) of dec is ion mat r i ces demons t ra ted greater rates of under-referral (F-) than did hor izontal eva lua t ion (predicted ou tcome) w h i c h ind icated greater rates of over-referral (F-t-). Ver t i ca l eva lua t ion ca lcu la tes the rates of pred ic t ive a c c u r a c y or i n a c c u r a c y in relat ion to the number of s tuden ts ac tua l ly at-r isk or not at-r isk. Hor izon ta l eva luat ion ca lcu la tes the rates of predic t ive a c c u r a c y or i naccu racy in relat ion to the number of s tuden ts predic ted to be at-r isk or not at-r isk. In re ference to the expec ta t ion of general under-referral sugges ted by the first f ind ing , ver t ica l eva lua t ion generated greater under-referral rates than did hor izontal eva lua t ion . Under-referra l rates (false nega t i ves , or F-) represent the percen tage of at-r isk s tuden ts w h o w e r e not ident i f ied for referral . For all ana lyses in th is s t u d y , results of hor izonta l eva luat ion of under-referral demons t ra ted relat ively l o w rates ranging f rom 9 - 2 5 % . Ver t i ca l eva lua t ion of under-referral demons t ra ted higher rates ranging f rom 7 6 - 9 2 % . C o n v e r s e l y , relat ively h igh over-referra l rates w e r e ca lcu la ted us ing hor izonta l eva lua t ion . Over- referra l rates (false pos i t i ves , or F-i-) represent the percentage of non-r isk s tuden ts w h o w e r e ident i f ied for referral . For all inves t iga t ions , resul ts of hor izonta l eva lua t ion demons t ra ted higher rates of over-referra l ranging f rom 2 6 - 7 6 % than those of ver t ica l eva lua t ion ranging f rom 2-11 %. It is no ted that over-referra l m a y ref lect the ex ten t to w h i c h in tervent ion is s u c c e s s f u l for t hose s tuden ts pred ic ted by sc reen ing to be at-r isk w h o subsequen t l y rece ive remedia l t rea tment . Related improvement in schoo l per fo rmance w o u l d make the initial pred ic t ion of r isk appear to be inaccura te , thereby inc reas ing the rate of over- re fer ra ls . Shor t of in tervent ion be ing unavai lab le or del iberate ly w i thhe ld , pure inves t iga t ions of va l id i ty are h indered in this regard . Ma tu ra t ion and deve lopmen t m a y prov ide simi lar threats to pred ic t ive a c c u r a c y of r isk ident i f icat ion. T h e c o s t of under-referral is related to the loss of educa t iona l in tervent ion for the t ru ly at-r isk s tudent . A d v a n t a g e s inc lude dec reased d e m a n d on both moneta ry and personne l resources . T h e oppos i te is true for over-referra l w h e n non-r isk s tuden ts rece ive cos t l y and u n n e c e s s a r y t reatment and may expe r ience the poss ib le c o n s e q u e n c e s of label l ing. The advan tage lies in the improved c h a n c e s for the ident i f icat ion of more at-r isk s tuden ts . The impac t o f m isc lass i f i ca t ion is def ined by the requi rements of the spec i f i c d ist r ic t and popu la t ion i nvo l ved . Hor izon ta l eva lua t ion ind icated that cor rec t referral rates w e r e greater than over-referra l rates for on ly t w o ana lyses in th is s tudy : Devere l l Tes t of Let ters and Numbers per C T B S ( 5 5 % true pos i t i ves , or T-h) and Sc reen ing C o m p o s i t e per H is to ry o f Intervent ion ( 7 4 % T-I-). Wh i l e this sugges t s that these measures m a y be s t ronger pred ic tors of bas ic ski l ls and in tervent ion serv ice respec t i ve ly , it is impor tant to c o m p a r e these resul ts w i th co r respond ing ver t ica l eva lua t ions . T h e ac tua l a c c u r a c y of referral ca lcu la ted ver t ica l ly is k n o w n as the rate of sens i t i v i ty o f the measu re . The ac tua l a c c u r a c y of non-referra ls , k n o w n as the rate o f spec i f i c i t y , a lso prov ides va luab le in format ion regarding the e f f i c iency of s c reen ing . 3 . Ver t i ca l eva luat ion of sc reen ing pred ic t ions cont r ibu ted to greater a c c u r a c y of interpretat ion in th is s tudy than did hor izonta l eva luat ion w h i c h inv i ted mis in terpreta t ion. Ver t i ca l eva luat ion of sc reen ing c lass i f i ca t ions ref lects the a c c u r a c y of predic t ion in relat ion to the cr i ter ion pe r fo rmance for the popula t ion i nvo l ved . Th is cont r ibu tes to accura te interpretat ion. Initial interpretat ion of hor izonta l eva lua t i on , w h i c h ref lects pred ic t ive a c c u r a c y in relat ion to sc reen ing pe r fo rmance , m a y require a l terat ion w h e n c o m p a r e d w i th co r respond ing ver t ica l ca l cu la t i ons . a. Predictive Accuracy of Individual Screening Measures Hor izonta l eva luat ion of the four ind iv idual sc reen ing measures as pred ic tors of th i rd-grade ach ievemen t (per C T B S ) revealed the Devere l l Tes t of Let ters and Numbers (Deverel l) to have the h ighest rate of a c c u r a c y for ident i fy ing at-r isk s tudents ( 5 5 % T + ). Ver t i ca l eva lua t ion ind ica ted , h o w e v e r , that these sub jec ts represented on ly 1 4 % of t hose s tuden ts w h o actua l ly per fo rmed unsat is fac tor i l y on the C T B S . Th is percen tage represents the rate of sens i t iv i ty of the measure . Add i t i ona l l y , hor izonta l eva lua t ion ind icated that on ly 11 % of t hose pred ic ted by th is measure to be non-r isk w e r e ac tua l ly at-r isk or under-referred (F-). W h e n ca lcu la ted ver t ica l ly , th is represented 8 6 % ac tua l under-referral for the samp le . Bo th hor izonta l and ver t ica l eva lua t ion demons t ra ted high rates of a c c u r a c y for predic t ing t ruly non-r isk s tuden ts ( 8 9 % and 9 8 % T-, respect ive ly ) . Of the indiv idual measu res , the Devere l l ' s greater probabi l i ty o f pred ic t ing at-r isk s tuden ts may have been in f luenced by the t iming of admin is t ra t ion and the re lat ionship of con ten t to s c h o o l t a s k s . M a y is late in the s c h o o l year and exposure to and pract ice of letter and number k n o w l e d g e has been a f o c u s o f the cur r i cu lum. The measure may a lso be lack ing in range of task d i f ferent iat ion or d i f f icul ty , resul t ing in an "al l or no th ing " pe r fo rmance t e n d e n c y ref lected in the referral rate of 3 % , the l owes t in the s tudy . In s u m m a r y , wh i le the Deverel l p roved cor rec t for 5 5 % of those pred ic ted to be at-r isk, they represented on ly 1 4 % of t hose ac tua l l y at-r isk in the s a m p l e . Ident i f icat ion of truly non-r isk s tudents w a s more accu ra te for both t ypes of eva lua t ion . A s imi lar pat tern of pred ic t ive a c c u r a c y w a s noted for the D r a w - A - P e r s o n and K indergar ten Language Sc reen ing Tes t w h e n c o m p a r i n g hor izontal ca lcu la t ions ( 2 4 % and 3 6 % T-I-, respect ive ly) w i th ver t ica l ca lcu la t ions ( 8 % and 2 0 % T-I-, respect ive ly ) . H igh rates of true negat ives we re s imi lar ly noted for both measures us ing ei ther t ype of eva lua t ion , as w a s true of all invest iga t ions in the s tudy ( 7 5 % - 9 1 % hor izon ta l , 8 9 % - 9 8 % vert ica l ) . Pred ic t i ve a c c u r a c y rates for the Mann -Su i t e r fur ther demons t ra ted the possib i l i ty of mis in terpretat ion w h e n c o m p a r i s o n of ca lcu la t ions does not o c c u r . Individual c a s e s (absolute numbers) resul t ing f rom the invest igat ion of the Mann -Su i t e r as a pred ic tor of th i rd-grade ach ievemen t w e r e as f o l l ows : {T + ) 18 {F + ) 5 6 = 7 4 (F-) 5 â (T-) 4 6 4 = 5 2 2 7 6 5 2 0 n = 5 9 6 A referral rate of 1 2 % (74 of 596) w a s the largest in the s tudy and near ly m a t c h e d the p rob lem base rate of 1 3 % as measured by the C T B S . Of those pred ic ted by the Mann -Su i t e r to be at-risl<, 2 4 % were accu ra te l y ident i f ied. Th is w a s one of the l owes t hor izonta l ly ca lcu la ted pred ic t ion rates for true pos i t i ves by an indiv idual measu re . It represented 2 4 % of t hose w h o ac tua l l y per formed unsat is fac tor i l y . Th is w a s the h ighest ver t ica l ly ca lcu la ted pred ic t ion rate for t rue pos i t i ves in the entire s t udy . T h u s , wh i le both t ypes of eva lua t ions resul ted in ident ical pe rcen tages , one represented the l owes t and the other the h ighest rate of t rue pos i t i ves . Add i t i ona l l y , of t hose pred ic ted to be at-r isk by the Mann -Su i t e r , 7 6 % w e r e over- referred (F + ), represent ing on ly 1 1 % of t hose w h o ac tua l ly per fo rmed sat is fac tor i l y . Of those predic ted to be non-r isk , 11 % were under-referred (F-), represent ing 7 6 % of those w h o ac tua l ly per fo rmed unsat is fac tor i l y . T h e s e oppos ing f igures we re related to the number of indiv idual c a s e s w i th in e a c h quadrant of the dec i s ion matr ix . Fa lse pos i t i ves and negat ives w e r e near ly ident ical in number , as w e r e the pred ic ted number of at-r isk and the ac tua l number of at-r isk. Wh i l e the numbers we re s imi lar , they represented di f ferent ind iv idua ls . The 5 6 over- referred s tuden ts (F + ) and 5 8 under-referred s tuden ts (F-) are separa te ind iv idua ls , a l t hough , numer ica l l y , these f igures resul ted in ident ical hor izontal(H) and ver t i caKV) ca lcu la t ions of t rue pos i t i ves . Under- refer ra l rates { 1 1 % H , ac tua l ly 7 6 % V ) we re oppos i te to those of over-referra l rates ( 7 6 % H , ac tua l ly 11 % V ) . Hor izonta l eva lua t ion , par t icu lar ly w h e n cons ide red in iso la t ion, thus s e e m s more vu lnerable to mis interpretat ion than does ver t ica l eva lua t ion . T h o u g h relat ively l o w , the Mann -Su i t e r reported the h ighest sens i t i v i t y rate (24%) of all ana lyses in th is s tudy . A cont r ibu t ing fac tor may relate to the smal l number of i tems w h i c l i compr i se th is measure and do not ful ly represent the range of v isua l -motor pe r fo rmance poss ib le . If a s tudent fai ls t w o or more of the four i tems, the test is " f a i l e d " , thus creat ing a larger body of referrals and inc reas ing the probabi l i ty of co r rec t l y ident i fy ing at-r isk s tuden ts . The relat ively s t ronger rate of sens i t iv i ty of the Mann -Su i t e r may have been in f luenced by the inc reased sc reen ing referral rate rather than by the d iscr iminatory p o w e r of the four i tems. Sens i t iv i ty of 2 4 % sti l l leaves 7 6 % of truly at-r isk s tuden ts un ident i f ied. b. Predictive Accuracy of the Composite Screening Classification (Screening Composite) Hor izonta l eva lua t ion sugges t s that the Sc reen ing C o m p o s i t e is more accura te for pred ic t ing s tuden ts at-r isk for future in tervent ion (74%) . O f the 3 8 s tuden ts referred by s c r e e n i n g , records indicate that 2 8 rece ived in tervent ion t reatment . T h e a c c u r a c y of r isk pred ic t ion might be expec ted to be h igh as the dist r ic t sc reen ing p rocess required that ident i f ied s tuden ts w e r e to rece ive further d iagnos is and in te rvent ion , thus in f luenc ing (or " con tamina t i ng " ) the pred ic tor /cr i ter ion re la t ionsh ip . Ver t i ca l eva lua t ion of the Sc reen ing C o m p o s i t e for pred ic t ing in tervent ion, h o w e v e r , ind ica tes 1 5 % sens i t iv i ty and 8 5 % under-referral of s tuden ts actua l ly at-r isk. S h o u l d in tervent ion o c c u r for reasons other than a c a d e m i c need , s u c h as behav io ra l c o n c e r n s or s t udy sk i l l s , or for mild d i f f icul t ies not fo reseeab le , pred ic t ion f rom sc reen ing w o u l d be unl ikely and therefore inc rease the rate of under- re fer ra l . Add i t i ona l l y , if in tervent ion w a s s u c c e s s f u l , fa lse pos i t ive rates might be expec ted to increase as true risl< s tuden ts b e c o m e ach ievers . T h e s e rates, h o w e v e r , remained relat ively l ow . U n d o c u m e n t e d remedia l se rv ice cou ld a lso reduce a c c u r a c y rates. Th i s inves t iga t ion , h o w e v e r , reveals in tervent ion w a s d o c u m e n t e d for 2 8 % of the samp le , represent ing the h ighest p rob lem base rate in the s tudy . If un recorded t reatment e x i s t e d , th is w o u l d l ikely serve to fur ther dec rease the rate of r isk ident i f icat ion and inc rease the t endancy for under- refer ra l . Because h is tory of in tervent ion for th is s t udy referred to all remedia l p rog ramming over four y e a r s , inc lud ing re tent ion, a higher rate of true r isk ident i f icat ion might have been e x p e c t e d . The Sc reen ing C o m p o s i t e as a predic tor of s tuden ts at-r isk for d i f f icu l ty w i th ach ievemen t (CTBS) and retent ion s ta tus w a s accura te for 41 % and 4 2 % , respec t i ve ly . T h e s e percen tages represent sens i t i vy rates of 1 4 % and 2 3 % acco rd ing l y . Of the retained s tuden ts , on ly 16 of 71 had been accura te l y ident i f ied by sc reen ing (16 of on ly 2 2 pred ic ted) . If it is a s s u m e d that s tuden ts w h o are eventua l ly reta ined might l ikely demons t ra te ear ly or more obv ious educa t iona l d i f f icu l t ies, inc reased ident i f icat ion w o u l d be e x p e c t e d . Wh i le the sens i t iv i ty of the Sc reen ing C o m p o s i t e for predic t ing retent ion w a s amongs t the s t ronges t in the s tudy , a rate of 2 3 % is not h igh . Under- referra l ind icates that 5 5 s tuden ts w h o w e r e not pred ic ted to be at-r isk w e r e ac tua l ly reta ined (77%) . If fac to rs o ther than a c a d e m i c di f f icul ty in f luenced re tent ion, sc reen ing w o u l d refer p ropor t ionate ly f e w e r s tuden ts . T h e s e results sugges t that hor izonta l eva lua t ion may invite mis in terpretat ion of the pred ic t ive a c c u r a c y of a sc reen ing measure or measures , par t icu lar ly if p resented in iso la t ion. It generates percen tages in relat ion to predic ted n u m b e r s , not ac tua l numbers . Ver t i ca l eva luat ion appears more l ikely to encourage c lar i ty of in terpretat ion. Resea rch increas ing ly reports sc reen ing a c c u r a c y in te rms of sens i t iv i ty and spec i f i c i t y rates on ly . P rev ious research and f indings w h i c h ut i l ize hor izonta l ca lcu la t ions may require réévaluat ion and interpret ive cau t ion . 4 . For all ana l vses . soec i f i c i t v rates (actual t rue necat ives) we re much larcer than sens i t iv i ty rates (actual true pos i t i ves) , ind icat ing far greater a c c u r a c v for ident i f icat ion of non-r isk than at-r isk s tuden ts . Spec i f i c i t y rates for all ana lyses in th is s t udy ( 8 9 % - 9 8 % ) w e r e h igh . Sens i t iv i ty rates ( 8 % - 2 3 % ) w e r e l ow . T h e s e resul ts ind icate greater sc reen ing a c c u r a c y for pred ic t ion of non-r isk s tuden ts than at-r isk s tuden ts . The h ighest rates of sens i t iv i ty in th is s t udy w e r e reported for the pred ic t ion of th i rd-grade ach ievemen t by the Mann -Su i t e r (24%) and retent ion s ta tus by the Sc reen ing C o m p o s i t e (23%) . D i s c u s s i o n in F ind ing 3 sugges ted that the higher rate of referral by the Mann -Su i t e r rather than the d iscr iminatory powe r of the four tes t i tems m a y have been inf luent ial in the apparent sens i t iv i ty of the measu re . D i s c u s s i o n in F ind ing 3 a lso ind icated that on ly 16 of 71 retained s tuden ts w e r e cons ide red by sc reen ing to be at-r isk. Wh i l e retent ion t radi t ional ly infers more s ign i f i cant d i f f icu l ty or de lay in s c h o o l pe r fo rmance , these prob lems may be at t r ibuted to reasons other than a c a d e m i c sk i l l , for e x a m p l e , a t t endance , behav ior , a t ten t ion , a g e , gender , s o c i o - e c o n o m i c s ta tus , or fami ly s y s t e m . T h e s e fac to rs may in f luence all ana lyses in this s tudy , but retent ion is amongs t the s t ronges t fo rms of in tervent ion and ear ly ident i f icat ion of r isk might be e x p e c t e d . A base rate of 11 % retent ion might a lso be cons ide red high in relat ion to the current p rac t ice o f non-retent ion. It is no tewo r t hy that C T B S resul ts w e r e avai lable for approx imate ly 1/3 of retained s tuden ts in the or iginal s a m p l e , ind icat ing that retent ion occu r red at the end of third grade. Distr ict cr i ter ia for retent ion m a y reveal reasons unrelated to k indergar ten sc reen ing con ten t . S imi la r ly , wh i le the Sc reen ing C o m p o s i t e pred ic ted in tervent ion for 2 8 of 3 8 referred s tuden ts w i th a spec i f icy rate of 1 5 % (28 of 187 ) , 1 5 9 pupi ls w h o rece ived in tervent ion we re not ident i f ied. Ques t i on ing w h y or w h e n a s tudent rece ives in tervent ion may reveal reasons other than those measured or pred ic tab le by the k indergar ten sc reen ing measures . Read ing d i f f icu l ty , for examp le , w o u l d l ikely be no ted later in the cu r r i cu lum. Behav io r and w o r k habi ts may a lso init iate a referral . The s u c c e s s of in tervent ion might be ind icated by higher over-referra l ra tes, but are not apparent in this inves t iga t ion . Under- re fer ra l , h o w e v e r , w a s a pervas ive d i f f icu l ty in all invest igat ions and cont r ibu ted to th is rather l o w ind icat ion of sc reen ing e f f i c iency . T h e K indergar ten Language Sc reen ing Tes t at ta ined a sens i t i v i ty rate of 2 0 % . Th is s tandard ized test of fered a poss ib le sco re of 2 9 resul t ing f rom 7 i tems. Fai lure w a s ind icated b e l o w 2 0 and no di f ferent iat ion w a s cons ide red for the range of s c o r e s b e t w e e n 0 and 19 . If the distr ic t c h o s e to prov ide loca l norms genera t ing a di f ferent cut -o f f s c o r e , inc reased a c c u r a c y of pred ic t ion may have been poss ib le . The language arts compr i se a s ign i f icant por t ion of the pr imary schoo l cu r r i cu l um. A test representat ive of th is age , con ten t and length might be expec ted to cont r ibu te more s t rong ly to ear ly r isk ident i f icat ion. It might a lso be cons ide red to cont r ibu te more than equa l we igh t as one of four sc reen ing measu res , espec ia l l y w h e n t w o of t hose may p roduce cei l ing e f fec ts c a u s e d by l imited number o f i tems or range in t asks as noted w i th the Mann-Su i te r the and Devere l l . Bo th the Devere l l and the Sc reen ing C o m p o s i t e reported sens i t iv i ty rates of 1 4 % for pred ic t ion of ach ievemen t as measured by the C T B S . The Devere l l cor rec t ly ident i f ied 5 5 % (11 of 20) of t hose refer red, but 8 6 % (67 of 78) of t h o s e w h o per fo rmed unsat is fac tor i l y on the C T B S w e r e not c lass i f ied as at-r isk w h e n s c r e e n e d . The Sc reen ing C o m p o s i t e a lso reported 8 6 % (65 of 76) under-referra l . On ly 2 7 of 5 9 2 s tuden ts in th is ana lyses w e r e c lass i f ied as at-r isk, 11 of w h i c h were pred ic ted accu ra te l y . The fac to rs leading to under-referral for the indiv idual measures cont r ibu te to the compos i t e abi l i ty to refer. W h e n t w o or more of the four measures w e r e fa i led , a s tudent w a s cons ide red by sc reen ing to be at-r isk. Shou ld e v e n one fai lure have deno ted r isk, the referral rate may have inc reased . Th i s , h o w e v e r , emphas i zes the assump t i on that e a c h measure c a n be ass igned equal we igh t in pred ic t ive abi l i ty w h i c h does not s e e m p laus ib le . T h e D r a w - A - P e r s o n demons t ra ted the least sens i t iv i ty (8%) for predic t ing ach ievemen t on the C T B S . Interrater rel iabil i t ies ranging f rom . 6 0 to . 9 4 (Scot t , 1981) have been repor ted . Th is test may be sub jec t to s o m e incons i s tency in sco r ing by the teachers i nvo l ved . A referral rate of 4 % is ve ry l o w and may ref lect the t e n d e n c y to a l l ow the benef i t of doub t regard ing s tuden t pe r fo rmance . The cut-of f point dev i sed th rough distr ic t norming m a y a lso not be appropr ia te for th is or other c o h o r t s . In s u m m a r y , resul ts of invest igat ions w e r e not able to demons t ra te s t reng th in sens i t iv i ty ra tes. Spec i f i c i t y ra tes, h o w e v e r , we re h igh . 5 . For all a n a l v s e s . overa l l hit rates w e r e h igh but are mis leadino as thev do not indicate the cont r ibu t ing propor t ions of cor rec t l v ident i f ied at-r isk ( T - H and n o n -risk (T-) s tuden ts . The overa l l hit rate represents the percentage of s tuden ts in a samp le w h o have been cor rec t ly ident i f ied as at-r isk (T + ) or non-r isk (T-). Rates for invest igat ions in th is s tudy w e r e h igh , ranging f rom 7 5 % - 8 8 % . A s the propor t ions of t rue pos i t i ves and negat ives w h i c h cont r ibu ted to the overa l l hit rates are not d i f ferent ia ted, the poss ib i l i ty for mis interpretat ion i nc reases . The overa l l hit rates in this s tudy do not ind icate the high rates of a c c u r a c y for se lec t ing non-r isk s tuden ts and the relat ively l o w rates for se lec t ing s tuden ts at-r isk. T h e overa l l hit rates are high largely due to the e f f i c iency of sc reen ing non-r isk . C. LIMITATIONS OF THE STUDY The ex ten t to w h i c h these resul ts c a n be genera l ized to other popu la t ions is s u s p e c t . Resu l ts are spec i f i c to the distr ic t and cohor t . Instrument se lec t ion a n d dec i s ion or cu t -o f f rules ind icat ing sa t i s fac to ry or unsa t i s fac to ry per fo rmance w e r e d ic ta ted by the dist r ic t and may not be app l icab le to th is or other g roups . Fur ther inves t iga t ion cont ro l l ing for the e f fec ts of , for e x a m p l e , in tervent ion. gender , and age m a y be war ran ted . D i f fe rences in s u c l i p rac t ices as educa t i ona l in tervent ion de l ivery se rv ice or retent ion se lec t i on w o u l d a lso negate genera l c o m p a r i s o n . Pred ic t ion -per fo rmance results are l imited to the pre-se lected s tandard ized measures c h o s e n for use in th is d is t r ic t . U s e of s u c h tes t s , part iculary for th is age group, is cur rent ly a top ic of regional and nat ional deba te . Th is m a y have in f luenced the d is t r i c t ' s even tua l dec i s ion to s u s p e n d year ly sc reen ing . The da ta for this s t u d y w a s co l l ec ted for the 1 9 8 3 - 8 4 age cohor t w h o were admin is te red both pred ic t ive and cr i ter ion measures . The da ta w a s , there fore , ex tant and m a y not ref lect the cur rent level of s tudent pe r fo rmance . The sc reen ing measures we re t reated as equa l con t r ibu tors to the c o m p o s i t e sc reen ing c lass i f i ca t ion a l though the numbers of i tems and range of t asks var ied cons ide rab ly . T h e four measures are ind iv idual ins t ruments w h i c h the d is t r ic t se lec ted for comb ina t i on . It is a lso not k n o w n h o w cons is ten t l y they w e r e admin is te red and sco red by the staff . D. IMPLICATIOIMS AND CONCLUSIONS The f ind ings of th is s tudy ind icated that the ind iv idual k indergar ten sc reen ing measu res and the compos i t e sc reen ing c lass i f i ca t ion demons t ra ted greater sc reen ing a c c u r a c y for pred ic t ion o f non-r isk s tuden ts than at-r isk s tuden ts . M isc lass i f i ca t i ons a lso demons t ra ted the uti l i ty of the sc reen ing measure or measu res for a id ing dec i s ions regarding the e f fec t i veness of the ear ly ident i f icat ion p rog ram. Ident i f icat ion of more obv ious learning di f f icul t ies is t radi t ional ly more ef f ic ient than that of mild or modera te d i f f icu l t ies. For distr ict pu rposes it m a y be suf f ic ient to con f i rm the ident i f icat ion of non-r isk s tuden ts . Ident i f icat ion of at-r isk s tudents o f ten invo lves many p rocedures of w h i c h k indergarten sc reen ing is on ly one . Dist r ic t dec is ions regard ing the con t i nued ass ignmen t of mone ta ry and personne l resources for sc reen ing are of ten gu ided by the results of over-referra l or under-referral ana l yses . The va lue of re instat ing the sc reen ing p rogram w o u l d need to be cons is ten t w i th d ist r ic t pu rposes for sc reen ing and the avai labi l i ty of resou rces . A r e a s requir ing greater personne l and monetary expend i tu re might be inves t iga ted . In th is s tudy , for e x a m p l e , the pu rchase and admin is t ra t ion of the K indergar ten Language Sc reen ing Tes t might be eva lua ted in te rms of c o s t and e f f i c iency w h e n uti l izing loca l ly deve loped norms ve rsus tes t -deve lopment no rms . Genera l under-referral w a s noted for all invest iga t ions in this s tudy . Desp i te the unders tand ing that the probabi l i ty for r isk ident i f icat ion improves w i th over-referral , it has been argued that greater under-referral t endenc ies of sc reen ing measures are appropr ia te and accep tab le as they ref lect the purpose for w h i c h the tes ts are ut i l ized (Laroche, 1 9 8 9 ) . Sc reen ing measures are des igned to se lec t on ly t hose ch i ld ren w h o dif fer s ign i f i cant ly f rom their peers . W h e n tes ts are expec ted to predic t mild r isks , dec i s i on errors increase great ly . Resu l ts are in f luenced and c a n be al tered by the cut -o f f po in ts se lec ted b e l o w w h i c h unsa t i s fac to ry pe r fo rmance on the sc reen ing or cr i ter ion measure is i nd ica ted . T o raise the cut -o f f of a sc reen ing test w o u l d generate more referrals and increase the possib i l i ty of cor rec t ly ident i fy ing at-r isk s tudents at the e x p e n s e of over- re fer ra l . T o lower the cr i ter ion cut -o f f w o u l d l ikely dec rease under-referral and improve sens i t i v i ty . Th is cou ld be inves t iga ted in a c c o r d a n c e w i t h spec i f i c d istr ic t requ i rements . W i th in the l imitat ions of this s t u d y , the f ind ings prov ide in format ion for eva lua t ion of the sc reen ing p rogram. Its cont r ibu t ion to the goal of prov id ing ear ly in tervent ion for educat iona l ly at-r isk s tuden ts as a result of ear ly ident i f icat ion c a n be cons ide red a long w i th other ind icators s u c h as parent , teacher , and med ica l report . Eva lua t ion of the relat ive benef i ts der ived f rom uti l izing the sc reen ing measures in dec i s i on -mak ing , or test ut i l i ty, w a s made poss ib le by ut i l iz ing dec is ion matr ix ana l ys i s . The pract ica l i ty of resul ts of th is fo rm of ana lys is demons t ra te its d i rect appl icabi l i ty to the real i ty of educa t iona l p rac t i ce . W h i l e cur rent educa t iona l ph i losophy in the p rov ince of Br i t ish C o l u m b i a a l l ows s tuden ts to p rogress at their indiv idual rate, the realit ies of p rac t ice regard ing in tervent ion and/or fund ing for s tuden ts w i th learning prob lems sugges t the con t i nued implementa t ion of ear ly ident i f icat ion p rocedu res . Ut i l iz ing dec i s ion matr ix ana lys is to eva lua te the predic t ive val id i ty and uti l i ty of se lec ted sc reen ing measures p resents resul ts in te rms of levels of e f f i c iency and pract ica l i ty . E. RECOMMENDATIONS FOR FUTURE RESEARCH Resu l ts of this s tudy generate recommenda t i ons for future research . P red ic t i on -per fo rmance invest igat ions w h i c h cont ro l for the e f fec ts of in tervent ion are relat ively f e w ( J a c o b s e n , 1 9 9 0 ) . B e c a u s e one purpose of ear ly ident i f icat ion of educa t iona l r isk is to implement subsequen t in tervent ion, the e f fec ts o f s u c c e s s f u l t rea tment might negate earlier r isk pred ic t ion for indiv idual s t uden ts . Research that con t ro ls for the e f fec ts of in tervent ion w o u l d cont r ibu te to know ledge regard ing ear ly pred ic tors of educa t iona l r isk. S imi lar ly , cont ro l l ing for the e f fec ts of o ther fac tors s u c h as age , gender , or t ype of risk shou ld be inves t iga ted . T im ing of and cr i ter ia for in tervent ion se rv i ce or retent ion pract ice va ry ac ross s i tua t ions . Thei r e f fec t on the resul ts of p red ic t ion-per fo rmance research in w h i c h they ac t as cr i ter ion ind ices of educa t iona l d i f f icu l ty requires fur ther inves t iga t ion . For s tuden ts w h o rece ive in tervent ion or w h o are retained for reasons not s t r ic t ly related to learning p rob lems , s u c h as behav ioura l d i f f icu l ty or s tudy hab i ts , p red ic t ion of r isk is less l ikely. Add i t i ona l invest igat ion regard ing w h e n or w h y s tuden ts are ass igned these t rea tments might cont r ibute to k n o w l e d g e of their p rac t ice and level o f s u c c e s s . A l te ra t ion of cut -of f points for the ins t ruments used in th is s tudy w o u l d result in d i f ferent rates of pred ic t ive a c c u r a c y . L o w e r C T B S cut -o f fs (perhaps at the 10th or 15 th percent i le) w o u l d l ikely increase T + ,F-f- and dec rease F-,T-. W e r e it poss ib le to raise the sc reen ing cut -o f fs for ind iv idual or compos i t e measu res , s imi lar resul ts might o c c u r a l though the con t r ibu t ions of s ingle or c o m b i n e d tes ts m a y alter this expec ta t i on . R e s e a r c h into the e f f i c iency of ear ly ident i f icat ion sc reen ing p rograms w o u l d benef i t f r om best p rac t i ces noted in the l i terature (C ross , 1 9 7 7 ; J a c o b s e n , 1 9 9 0 ; K e o g h , 1 9 7 7 ; L ich tenste in & Ireton, 1 9 8 4 ; Mc lough l i n & R a u s c h , 1 9 9 0 ; M e r c e r et a l . , 1 9 8 8 ; N A S P , 1 9 8 9 ) . T h e s e w o u l d inc lude, for e x a m p l e , ef f ic ient documen ta t i on and retr ieval p rocedures , large samp les , sk i l led and cons i s ten t admin is t ra t ion and sco r i ng , mult iple p red ic to rs , acceptab i l i t y of ins t rumenta t ion , mult iple eva lua t ion over t ime, cont ro l for e f fec ts of var iab les d i s c u s s e d in the prev ious recommenda t i ons , and appropr ia te me thods of ana lys i s . De te rmin ing and c o m m u n i c a t i n g sc reen ing resul ts have more recent ly emp loyed me thods s u c h as improvemen t -ove r -chance or IOC (Wi lson & R e i c h m u t h , 1985 ) . In th is m e t h o d , in format ion in addi t ion to that resul t ing f rom ut i l izat ion o f p red ic t ion-per fo rmance matr ices relates h o w m u c h improvement in a c c u r a c y of referral is ob ta ined over that ach ieved by random referral . IOC g ives cons idera t ion to the d i s c r e p a n c y b e t w e e n the base rate and the referral rate because of its s ign i f i cant e f fec t on the number of co r rec t dec is ions that b e c o m e poss ib le . IOC ana lys is might add to the interpretat ion and compar i son of s tud ies found in ear ly ident i f icat ion l i terature. R E F E R E N C E S A a r o n s o n , M . , Phi l l ips, J . , Be r to lucc i , D., & A a r o n s o n , D. ( 1 9 7 8 , M a r c h ) . P reschoo l Prepos i t ion Tes t : Deve lopmen ta l sc reen ing for Head Star t . Paper p resented at the 6 2 n d A n n u a l M e e t i n g of the A m e r i c a n Educa t ion R e s e a r c h A s s o c i a t i o n , To ron to , C a n a d a . A d e l m a n , H . S . , & F e s h b a c h , S . (1971) . Pred ic t ing reading fai lure: B e y o n d the read iness mode l . Excep t iona l Ch i l d ren . 37(5 ) . 3 4 9 - 3 5 . A n a s t a s i , A . (1972) . A R e v i e w of the Goodenough -Har r i s D r a w - A - M a n Tes t . In 0 . K. Buros (Ed.) , The seven th menta l measu remen ts yea rbook (pp. 6 6 9 - 6 7 1 ) . H igh land Park, N J : G r y p h o n P r e s s . Bad ian , N . A . (1986) . Improving the pred ic t ion of reading for the indiv idual ch i l d : A four -year fo l l ow-up . Jou rna l of Learn ing Disab i l i t ies . 1 9 . 2 6 2 - 2 6 9 . Bad ian , N . A . , & S e r w e r , B. L. (1975) . The ident i f icat ion o f high-r isk ch i ld ren : A re t rospect ive look at se lec t ion cr i ter ia . Jou rna l of Learn ing Disabi l i t ies. 8 (5) , 2 8 3 - 2 8 7 . Bagna to , S . J . , & N e i s w o r t h , J . T . ( 1987 ) . The y o u n g excep t i ona l ch i l d : Ear ly deve lopmen t and educa t i on . N e w Y o r k : M a c M i l l a n . Barnes , K . E . (1982) . P reschoo l s c reen ing : The measu remen t and pred ic t ion of ch i ld ren at-r isk. Spr ing f ie ld , IL: Cha r les C . T h o m a s . Barnet t , D . W . , & Page t , K . D . (1988) . A l te rna t ive se rv i ce de l ivery in p reschoo l se t t ings : Prac t i ca l and concep tua l founda t ions . In J . L . G r a d e n , J . E . Z i ns , & M . J . Cur t is (Eds. ) , A l te rna t ive educa t iona l de l ivery s v s t e m s : Enhanc ing inst ruct ional op t ions for all s tuden ts (pp. 2 9 1 - 3 0 8 ) . W a s h i n g t o n , D C : Nat iona l A s s o c i a t i o n of S c h o o l P s y c h o l o g i s t s . Berd ine, W . H . , & M e y e r , S . A . (1987) . A s s e s s m e n t in spec ia l educa t i on . B o s t o n , M A : Li t t le, B r o w n , & C o . Berk, R . A . (1984) . Sc reen ing and d iagnos i s of ch i ldren w i t h learning d isabi l i t ies. Spr ing f ie ld , IL: Char les T h o m a s . B i l lman, J . (1988) . T w o years o f k indergar ten : E th ica l and curr icu lar cons ide ra t i ons . (ERIC D o c u m e n t Reproduc t ion S e r v i c e N o . ED 2 9 7 8 8 4 ) . Bi rch , L. B. ( 1972 ) . R e v i e w of the Canad ian Tes t s of Bas i c SIcills. In O . K. Bu ros (Ed.). T h e seven th menta l nneasurements vea rbook (pp. 15 -16 ) . H igh land Park , N J : G r y p h o n P ress . B l oom, B. S . ( 1964 ) . Stabi l i tv and change in human charac te r i s t i cs . N e w Y o r k , N Y : W i l e y . B o e h m , A . ( 1971 ) . B o e h m Tes t of Bas ic C o n c e p t s . N e w Y o r k , N Y : P s y c h o l o g i c a l Co rpo ra t i on . B o e h m , A . , & S a n d b e r g , B. (1982) . A s s e s s m e n t of the p reschoo l ch i l d . In C . Reyno lds & T . Gu tk in (Eds.) , The handbook of s c h o o l p s y c h o l o g y (pp. 8 2 - 1 2 0 ) . N e w Y o r k , N Y : W i l ey . Book , R . M . (1980 ) . Ident i f icat ion of educat iona l ly at-r isk ch i ldren dur ing the k indergar ten year : A four -year fo l l ow-up s tudy of group test pe r fo rmance . P s v c h o l o o v in the S c h o o l s . 17(2) . 1 5 3 - 1 5 8 . B r a c k e n , B. A . ( 1982 ) . Obse rv ing the a s s e s s m e n t behav ior of p reschoo l ch i ld ren . In K . D . Page t & B . A . B racken (Eds.) , The psvchoeduca t i ona l a s s e s s m e n t o f p reschoo l ch i ld ren (pp. 6 3 - 7 9 ) . Or lando , FL : Grune & S t ra t ton . B r a c k e n , B. A . ( 1987 ) . L imi tat ions o f p reschoo l ins t ruments and s tandards for min imal levels o f techn ica l a d e q u a c y . Jou rna l of P y c h o e d u c a t i o n a l A s s e s s m e n t . 4 , 3 1 3 - 3 2 6 . B rewer , G . D . , & Kaka l i k , J . S . (1979) . Hand i capped ch i ldren - St ra teg ies for improv ing s e r v i c e s . N e w Y o r k , N Y : M c G r a w - H i l l . B u s c h , R. ( 1980 ) . Pred ic t ing f irst grade reading ach ievemen t . Learn ing Disabi l i tv Quar te r l v . 3 , 3 8 - 4 7 . B u s h , W . , & W a u g h , K. (1982) . D iagnos ing learning p rob lems (3rd ed. ) . C o l u m b u s , O H : Char les E. Merr i l l But ler , S . ( 1979 ) . Pred ic t i ve an teceden ts of reading d isabi l i ty in the ear ly years of s c h o o l i n g . B . C . Jou rna l of Spec ia l Educa t i on . 3 , 2 6 3 - 2 7 4 . But ler , S . ( 1981 ) . H o w not to test for bas ic sk i l ls . The B . C . Teache r . 60(3) . 9 7 - 9 8 , 1 1 0 . C a r r o w , E. (1973) . Tes t of Aud i t o r v C o m p r e h e n s i o n of Language . To ron to , On ta r io : G i n n . C a r r o w , E. (1974) . C a r r o w El ic i ted Langage Inventorv. To ron to , Ontar io : G i n n . C a s t o , G . , & Mas t rop ie r i , M . A . (1986) . T i i e e f f i cacy of early in tervent ion p rog rams : A meta-ana lys is . Excep t iona l Ch i l d ren . 5 2 . 4 1 7 - 4 2 4 . C la rke , A . M . , & C l a r k e , A . D . B . (1977) . Ear ly expe r i ence : M y t h and e v i d e n c e . N e w Y o r k , N Y : Free Press . C l a r k s o n , D.R. ( 1989 ) . T h e use fu lness of the D IAL -R as a sc reen ing too l to ident i fy Nat ive A m e r i c a n p resechoo les in need of spec ia l se rv i ces (Doctora l d isser ta t ion , U tah Sta te Un ivers i t y , 1 9 8 1 ) . D isser ta t ion A b s t r a c t s Internat ional . 51 /01 A , p .75 . Counc i l for Excep t iona l Ch i ld ren (1990) . C E C of fers input to Pres ident B u s h ' s educa t i on summi t . T e a c h i n g Excep t iona l Ch i l d ren . 22(2) , 8 9 . C r o s s , L. ( 1977 ) . Ident i f icat ion of y o u n g ch i ld ren w i th hand i caps : A n o v e r v i e w . In N . Ell is & L. C r o s s (Eds.) , P lann ing p rog rams for ear ly educa t ion of the hand i capped (pp. 9 6 - 8 2 ) . N e w Y o r k : W a l k e r . Devere l l , A . ( 1974 ) . T h e Deverel l c lass i f i ca t ion tes t for use w i th s c h o o l beg inners (letters and numbers ) . In Teach ing ch i ld ren to read and wr i te . T o r o n t o , Ontar io : Ho l t , R inehar t , & W i n s t o n . D u n n , J . A . (1967) . No te on the relat ion of Har r i s ' D r a w - A - W o m a n to W I S C I .Q. 's . Percep tua l and M o t o r Sk i l l s . 2 4 , 3 1 6 . D u n n , J . A . (1972) . A R e v i e w of the Goodenough -Ha r r i s D r a w - A - M a n Tes t . In O . K . Buros (Ed.), The seven th menta l measu remen ts vea rbook (pp. 6 7 1 - 6 7 2 ) . H igh land Park , N J : G r y p h o n P r e s s . Durrel l , D. ( 1958 ) . F i rs t -grade reading s u c c e s s s tudy : A s u m m a r y . Jou rna l o f E d u c a t i o n . 1 4 0 . 2 -6 . D y k s t r a , R. ( 1967 ) . The use of reading read iness tes ts for d iagnos is and p red ic t ion : A cr i t ique. In T . Barrett (Ed.), T h e eva lua t ion of ch i l ren 's reading ach ievemen t . N e w a r k , D E : Internat ional Read ing A s s o c i a t i o n . E a v e s , L . C . , Kenda l l , D . C . , & C r i c h t o n , J . U . ( 1974 ) . T h e ear ly ident i f icat ion of learning d isabi l i t ies : A fo l low-up s tudy . J o u r n a l of Learn ing Disabi l i t ies . 7 (10 ) , 6 3 2 - 6 3 8 . E r i c k s o n , E. (1963) . Ch i l dhood and soc ie tv . N e w Y o r k : W . W . No r t on . F e s h b a c h , S . , A d e l m a n , H . , & Ful ler, W . (1974 ) . Ear ly ident i f icat ion of ch i ldren w i th h igh risk of reading fai lure. Jou rna l of Learn ing Disabi l i t ies . 7 (10) , 6 3 9 - 6 4 4 . F e s h b a c h , S . , A d e l m a n , H . , & W i l l i a m s o n , W . F . (1974) . Ear ly ident i f ica t ion of ch i ld ren w i th h igh risk of reading fa i lure. Jou rna l of Learn ing Disab i l i t ies . 7 (10 ) , 6 3 9 - 6 4 4 . F rankenburg , W . K. ( 1977 ) . Cons ide ra t i ons for sc reen ing . In N . Ell is & L. C r o s s (Eds. ) , P lann ing programs for ear ly educa t i on of the hand icapped (pp. 8 3 - 9 6 ) . N e w Y o r k : Wa lke r & C o . Gauth ier , S . , & M a d i s o n , C . (1978) . K indergar ten Language Sc reen ing T e s t . T i g a r d , O R : C C Pub l i ca t ions . Gauth ier , S . , & M a d i s o n , C . (1983) . K indergar ten Language Sc reen ing T e s t : M a n u a l . T i ga rd , O R : C C Pub l i ca t ions . Gese l l Insti tute of H u m a n Deve lopmen t (1983) . A gift of t i m e . . . A deve lopmen ta l point of v i e w . N e w H a v e n , C T : A u t h o r . G o o d e n o u g h , F. ( 1926 ) . M e a s u r e m e n t of inte l l igence bv d r a w i n g s . Y o n k e r s - o n - H u d s o n , N Y : W o r l d B o o k . G r a c e y , C . A . , A z z a r a , C . V . , & Re inherz , H. (1984) . Sc reen ing rev is i ted: A s u r v e y of U . S . requ i rements . The Jou rna l of S p e c i a l Educa t i on . 1 8 . 1 0 1 - 1 0 7 . G ray , E. (1988) . Ident i f icat ion and in tervent ion s t ra teg ies for p reschoo l k indergar ten , f i rst and s e c o n d grade ch i ld ren at risk for reading d i f f icu l t ies. (ERIC D o c u m e n t Reproduc t ion Se rv i ce N o . E D 2 9 7 5 1 2 ) . Gred ler , G . ( 1 9 9 0 , M a r c h ) . A d v o c a t e s for the y o u n g ch i l d : Cur ren t a s s e s s m e n t . C o m m u n i q u e . 18(6) . 10 . G r e e n w o o d , B. (1981) . N o r m re fe renced ve rsus prec is ion a s s e s s m e n t s for sc reen ing high r isk and high ach iev ing y o u n g ch i ldren (Doctora l d isser ta t ion . Un ive rs i t y of F lo r ida , 1 9 8 1 ) . D isser ta t ion A b s t r a c t s Internat ional . 4 2 / 0 5 A . 2 0 9 2 . Gura ln ick , M . J . , & Br icker , D .D . (1987) . T h e e f fec t i veness of ear ly in tervent ion for ch i ld ren w i th cogn i t i ve and genera l deve lopmen ta l de lays . In M . J . Gura ln i ck & F . C . Bennet t (Eds. ) , The e f fec t i veness of ear lv in tervent ion for at- r isk and hand icapped ch i ld ren . O r l ando , F L : A c a d e m i c P r e s s . Ha l lahan , D . , & K a u f f m a n , J . (1978) . Excep t iona l ch i ld ren . E n g l e w o o d C l i f f s , N J : P ren t i ce -Ha l l . Harber , J . ( 1981a ) . A s s e s s i n g the qual i ty of dec i s ion mak ing in spec ia l educa t i on . J o u r n a l of Spec ia l E d u c a t i o n . 7 7 - 9 0 . Harber , J . ( 1981b ) . Eva luat ing uti l i ty in d iagnos t i c dec i s i on mak ing . Jou rna l o f Spec ia l Educa t i on . 1 5 . 4 1 3 - 4 2 8 . Har ing , N . , & R i d g w a y , R. (1967) . Ear ly ident i f icat ion o f ch i ldren w i th learning d isabi l i t ies. Excep t iona l Ch i l d ren . 3 3 . 3 8 7 - 3 9 5 . Harr is , D. (1963) . Ch i l d ren ' s d raw ings as measu res of intel lectual matur i ty : A rev is ion and ex tens ion of the G o o d e n o u g h D r a w - A - M a n Tes t . N e w Y o r k , N Y : Harcour t , B race , & W o r l d . H a y d e n , A . (1974) . Pe rspec t i ves of ear ly ch i l dhood educa t ion in spec ia l e d u c a t i o n . In N . G . Har ing (Ed.), Behav io r of Excep t iona l Ch i l d ren : A n in t roduct ion to spec ia l educa t i on . C o l u m b u s , O H : Char les E. Merr i l l . H a y d e n , A . H . , & M c G i n n e s s , G . D . (1977) . Bases for ear ly in tervent ion. In E. S o n t a g (Ed.), Educat iona l p rogramming for the severe ly and p ro found ly h a n d i c a p p e d . R e s t o n , V A : Counc i l for Excep t iona l Ch i l d ren . H ie ronymus , A . N . , K i n g , E . M . , B o u r d o n , J . W . , G o s s l i n g , D., G r y w i n s k i , N . J . , & M o s s . G . L. (1976) . Canad ian Tes t s of Bas i c Sk i l l s : M a n u a l for admin is t ra to rs , supe rv i so rs , and counse l l o rs . C a n a d a : T h o m a s Ne lson & S o n s . Hi l ls , T . W . (1987) . Sc reen ing for s c h o o l ent ry . In ERIC Dioes t . Urbana, IL : ERIC C lea r i nghouse on E lementary and Early Ch i l dhood Educa t i on . H o b b s , N . , (1975) . The fu tures of ch i ld ren . S a n F r a n c i s c o , C A : J o s s e y - B a s s . H o r n , W . F . , & P a c k a r d , T . (1985) . Ear ly ident i f icat ion of learning p rob lems : A meta -ana lys i s . Jou rna l of Educat iona l P s y c h o l o g y . 7 7 . 5 9 7 - 6 0 7 . H o r o w i t z , F .D . (1980) . Intervent ion and its e f fec ts on ear ly deve lopmen t : W h a t mode l of deve lopmen t is appropr ia te? In R. Turner & H. Reese (Eds.) , L i fe-s p a n deve lopmenta l p s y c h o l o g y : Intervent ion. N e w Y o r k , N Y : A c a d e m i c P r e s s . H o u s e , E .R . , M a t h i s o n , S . , & M c T a g g a r t , R. (1989) . Va l id i ty and teacher in fe rence . Educat iona l Researcher . 18 (7 ) , 1 1 - 1 5 , 2 6 . Hunt , J . ( 1961 ) . Intel l igence and exper ience . N e w Y o r k , N Y : Rona ld P ress . l lg , F. , & A m e s , T . (1964) . S c h o o l read iness : Behav io r tes ts used at the Gesse l l Inst i tute. N e w Y o r k , N Y : Harper & R o w . l l lerbrun, D. , Ha ines , L., & G r e e n o u g h , P. (1985) . Language ident i f ica t ion Sc reen ing Tes t for K indergar ten : A connpar ison w i th four sc reen ing and three d iagnos t i c language tes ts . Language . S p e e c h , and Hear ino Se rv i ces in S c h o o l s . 1 6 . 2 8 0 - 2 9 2 . J a c o b s e n , S . (1990) . Identi fying ch i ld ren at r isk: The predic t ive val id i ty o f k inderoar ten sc reen ing measu res . Unpub l i shed doc to ra l d isser ta t ion . Un ive rs i t y o f Br i t ish C o l u m b i a , V a n c o u v e r . J a n s k y , J . , & de H i r s c h , K. (1972) . Prevent ing reading fa i lure. N e w Y o r k , N Y : Harper & R o w . J u d y , J . (1986) . Ear ly sc reen ing is essent ia l for educa t iona l accoun tab i l i t y : R e s p o n s e to Sa lze r and to Shepa rd and S m i t h . Educa t iona l Leadersh ip . 4 4 ( 3 ) , 8 7 - 8 8 . K e o g h , B .K . (1977) . Ear ly ID: Se lec t i ve percept ion or percept ive se lec t ion? A c a d e m i c T h e r a p y , 1 2 , 2 6 7 - 2 7 4 . K e o g h , B .K . , & Becke r , L .D . (1973) . Ear ly de tec t ion of learning p rob lems: Q u e s t i o n s , cau t i ons , and gu ide l ines . Excep t iona l Ch i l d ren . 4 0 . 5 - 1 1 . K e o g h , B .K . , & Da ley , S . E . (1983) . Ear ly ident i f icat ion: One c o m p o n e n t of c o m p r e h e n s i v e se rv i ces for at-r isk ch i ld ren . T o p i c s in Earlv Ch i l dhood Spec ia l E d u c a t i o n . 3 , 7 -16 . Kirk, S . ( 1958 ) . Ear ly educa t ion of the menta l ly re tarded. U r b a n a , IL: Un ivers i ty o f Illinois P r e s s . Kirk, S . , M c C a r t h y , J , & Kirk, S . ( 1968 ) . The Illinois Tes t o f Psvcho l i ngu is t i c Ab i l i t i es . U r b a n a , IL: Un ivers i t y of Illinois P ress . La roche , M . (1989) . A predic t ive va l id i ty s tudy of the D A B E R O N : A sc reen ing tes t used for ident i fy ing k indergar ten ch i ldren w h o m a y be at-r isk for a c a d e m i c fai lure (Doctora l d isser ta t ion . Un ivers i ty of M a s s a c h u s e t t s , 1 9 8 9 ) . D isser ta t ion A b s t r a c t s Internat ional . 5 0 / 0 8 A . p. 2 4 6 5 . L e a c h , D . J . (1980) . Cons ide ra t i ons in sc reen ing for s c h o o l learning d i f f icu l t ies: A n o v e r v i e w of p rac t i ce . Educa t iona l S tud ies . 6(3) , 1 8 1 - 1 9 7 . Lerner, J . , M a r d e l l - C z u d n o w s k i , C , & G o l d e n b e r g , D. ( 1987 ) . Soec ia l educa t ion for the ear lv ch i l dhood vears (2nd ed. ) . E n g l e w o o d C l i f f s , N J : Prent ice-Ha l l . L e w i s , A . ( 1980 ) . The ear ly ident i f icat ion of ch i ldren w i t h learning di f f icul t ies. J o u r n a l of Learn ing Disabi l i t ies. 13(2) . 1 0 2 - 1 0 8 . L ich tens te in , R. ( 1981 ) . Compara t i ve val id i ty of t w o p reschoo l sc reen ing t e s t s : cor re la t iona l and c lass i f i ca t iona l a p p r o a c h e s . Jou rna l of Learn ing Disabi l i t ies . 1 4 , 6 8 - 7 2 . L ich tens te in , R., & Ireton, H. (1984) . P reschoo l sc reen ing : Identi fvino v o u n o ch i ld ren w i th deve lopmenta l and educa t iona l p rob lems. Or lando , FL : G r u n e & S t ra t ton . L indqu is t , E . , H i e r o n y m o u s , A . , & H oove r , D. (1955) . Iowa Tes t s of Bas i c Sk i l l s . C h i c a g o , IL: R ivers ide . L indsay , G . A . , & W e d e l l , K. (1982) . The ear ly ident i f icat ion of educat iona l l y 'a t r isk ' ch i ld ren rev is i ted . Jou rna l of Learn ing Disabi l i t ies. 15(4) . 2 1 2 - 2 1 7 . M a n n , P . , Su i te r , P . , & M c C l u n g , R. (1979) . H a n d b o o k of d iagnos t ic -p rescr ip t i ve teach ing (2nd ed. ) . B o s t o n , M A : A l l y n & B a c o n . M a n n , P . , Su i te r , P . , & M c C l u n g , R. (1987) . T e a c h e r ' s handbook of d iagnos t i c s c r e e n i n g . B o s t o n , M A : A l l y n & B a c o n . M c l o u g h l i n , C . (1988) . P rov is ion of psycho log i ca l se rv i ces to ve ry y o u n g ch i ld ren and their fami l ies . In J . G r a d e n , J . Z i n s , & M , Cur t i s (Eds.) , A l te rna t ive educa t iona l de l iverv s v s t e m s : Enhanc ing inst ruct ional op t ions for all s tuden ts (pp. 2 6 9 - 2 9 0 ) . W a s h i n g t o n , D C : Nat iona l A s s o c i a t i o n of S c h o o l P s y c h o l o g i s t s . M c l o u g h l i n , C , & R a u s c h , E. (1990) . Best p rac t ices in k indergar ten sc reen ing . In A . T h o m a s and J . Gr imes (Eds.) , Best p rac t ices in s c h o o l p s y c h o l o g y - II (pp. 4 5 5 - 4 6 7 ) . W a s h i n g t o n , D C : Nat iona l A s s o c i a l t i o n of S c h o o l P s y c h o l o g i s t s . M c L o u g h l i n , J . , & L e w i s , R. ( 1986 ) . A s s e s s i n g spec ia l s tuden ts (2nd ed. ) . C o l u m b u s , O H : Merr i l l . M e e c h u m , M . , J e x , J . , & J o n e s , J . ( 1967 ) . U tah Tes t of Language Deve lopmen t . Sa l t Lake C i t y , U T : C o m m u n i c a t i o n Resea rch A s s o c i a t e s . M e e h l , P. ( 1954 ) . C l in ica l ve rsus s tat is t ica l p red ic t ion . M inneapo l i s , M N : Un ivers i t y of M i n n e s o t a P r e s s . M e e h l , P . , & R o s e n , A . (1955) . A n t e c e d e n t probabi l i ty and the e f f i cacy o f p s y c h o m e t r i c s i g n s , pat terns or cu t t ing s c o r e s . P s y c h o l o g i c a l Bul le t in . 5 2 . 1 9 4 - 2 1 6 . Meise l s , S . J . (1984) . P red ic t ion , p revent ion and deve lopmen ta l sc reen ing in the E P S D T p rog ram. In H. S t e v e n s o n & A . S iege l (Eds. ) , Chi ld d e v e l o p m e n t research and soc ia l oo l i cv . C h i c a g o , IL: Un ive rs i t y o f C h i c a g o P ress . Me i se l s , S . (1987) . U s e s and a b u s e s of deve lopmen ta l sc reen ing and s c h o o l read iness tes t ing . Y o u n o Ch i l d ren . 4 2 ( 2 ) , 6 8 - 7 3 . M e s s i c k , S . (1989) . Va l id i ty . In R. L inn (Ed.) , Educa t iona l measurement (3rd ed . ) . N e w Y o r k , N Y : M a c m i l l a n . Merce r , C D . , A l g o z z i n e , B., & Tr i f i let t i , J . (1988) . Ear ly ident i f icat ion - an ana lys is o f the research . Learn ing Disabi l i ty Quar te r l y . 1 1 . 1 7 6 - 1 8 8 . Mi l ler , L., & S p r o n g , T . (1986) . P s y c h o m e t r i c and qual i ta t ive compar i son of four p reschoo l sc reen ing ins t ruments . J o u r n a l of Learn ing Disabi l i t ies. 1 9 . 4 8 0 -4 8 4 . M o r g a n , G . (1981) . P rob lem and promise in ear ly ident i f ica t ion: Ch i ld ren w i t h spec ia l needs . In H. S i l ve rman , I. D a v i d s o n , & L. We in t raub (Eds.) , Ear lv ident i f icat ion & in tervent ion: Se lec ted p roceed ings f rom the four th internat ional s y m p o s i u m on learning p rob lems . To ron to 1 9 7 9 (pp. 4 8 - 5 9 ) . T o r o n t o , Ontar io : Ontar io M in is t ry of E d u c a t i o n . Nagl ier i , J . (1988) . D r a w - A - P e r s o n : A quant i ta t ive sco r i ng s v s t e m . To ron to , On ta r io : Harcour t -B race , J o v a n o v i c h . Nagl ier i , J . , & M a x w e l l , S . (1981 ). Inter-rater rel iabil i ty and concur ren t va l id i ty o f the Goodenough -Ha r r i s D r a w - A - P e r s o n and M c C a r t h y . Percep tua l and M o t o r Sk i l l s . 5 3 . 3 4 3 - 3 4 8 . Nat ional A s s o c i a t i o n of S c h o o l P s y c h o l o g i s t s (1989) . T h e ear ly ch i l dhood ident i-f i ca t ion p r o c e s s : A manua l for sc reen ing and a s s e s s m e n t . W a s h i n g t o n , D C : Nat iona l A s s o c i a t i o n of S c h o o l P s y c h o l o g i s t s . Neu fe ld , G . , & S t e v e n s , A . (1992) . S tav in s c h o o l in i t ia t ives: Book 1. A s u m m a r v of research on s c h o o l d ropou ts and impl ica t ions for spec ia l educa t i on . K i n g s t o n , Ontar io : The Canad ian C o u n c i l for Excep t iona l Ch i l d ren . N e w c o m e r , P . , & Hammi l l , D. (1977) . Tes t of Language Deve lopmen t . Los A n g e l e s , C A : W e s t e r n Psycho log i s t s P r e s s . No r ton , L. ( 1979 ) . T h e ident i f icat ion of "at r isk" k indergar ten ch i ld ren . Spec ia l Educa t i on in C a n a d a . 53(4) , 2 0 - 2 2 . N o v a c k , H . , Bonaven tu ra , E. , & M e r e n d a , P. (1973) . A sca le for ear ly de tec t ion o f ch i ld ren w i th learning problenns. Excep t iona l Ch i ld ren . 4 0 . 9 8 - 1 0 5 . O a k l a n d , T . , & D o w l i n g , L. (1983) . The D r a w - A - P e r s o n Tes t : Va l id i ty proper t ies for nonb iased a s s e s s m e n t . Learn ing Disabi l i t ies Quar ter ly . 6 , 5 2 6 - 5 3 4 . Paget , K . D . , & Nag le , R . J . (1986) . A c o n c e p t u a l mode l of p reschoo l a s s e s s m e n t . S c h o o l P s v c h o l o o v R e v i e w . 1 5 . 1 5 4 - 1 6 5 . Pal f rey, J . (1981) . P reschoo l sc reen ing : Ready? se t? go? In H. S i l v e r m a n , I. D a v i d s o n , & L. We in t raub (Eds. ) , Ear lv ident i f icat ion & in tervent ion : Se lec ted p roceed inos f rom the four th internat ional s y m p o s i u m o n learning p rob lems . To ron to 1 9 7 9 (pp. 3 5 - 4 5 ) . To ron to , Ontar io : Ontar io M in is t r y of E d u c a t i o n . P lan ta , R . C . , (1990) . W iden ing the debate on educat iona l re form: Preven t ion as a v iab le a l ternat ive. Excep t iona l Ch i l d ren . 5 6 . 3 0 6 - 3 1 3 . Pe te rson , N .L . (1987) . Earlv in tervent ion for hand icapped and at-r isk ch i ld ren : A n in t roduct ion to ear lv ch i l dhood-spec ia l educa t i on . Denver , C O : L o v e . Publ ic L a w 9 4 - 1 4 2 (1975) . The Educa t ion for A l l Hand i capped Ch i ld ren A c t of 1 9 7 5 . W a s h i n g t o n , D C : Federa l Reg is ter . Publ ic L a w 9 9 - 4 5 7 (1986) . The Educa t ion for A l l Hand i capped Ch i ld ren A c t of 1 9 8 6 . W a s h i n g t o n , D C : Federa l Regis ter . Reyno lds , L., E g a n , R., & Lerner , J . ( 1983 ) . E f f i cacy of ear ly in tervent ion on p reacadem ic def ic i ts : A rev iew of the l i terature. T o p i c s in Earlv Ch i l dhood S p e c i a l Educa t i on . 3(3) , 4 7 - 5 6 . Reyno lds , C , G u t k i n , T . , El l iott , S . , & Wi t t , J . (1984) . S c h o o l p s v c h o l o o v : Essen t ia ls of theory and prac t i ce . N e w Y o r k , N Y : W i l e y . R o g o l s k y , M . (1968) . Sc reen ing k indergar ten ch i ld ren : A r e v i e w and recommenda t i ons . Jou rna l of S c h o o l P s v c h o l o q v . 7 . 1 8 - 2 5 . Sa l v i a , J . , & Y s s e l d y k e , J . (1978) . A s s e s s m e n t in spec ia l educa t i on . B o s t o n , M A : Hough ton -M i f f l i n . Sa l v i a , J . , & Y s s e l d y k e , J . (1981) . A s s e s s m e n t in spec ia l educa t ion (2nd ed.) . B o s t o n , M A : Hough ton Mi f f l in . Samero f f , A . (1975) . T ransac t iona l mode ls in ear ly soc ia l re la t ions. Human D e v e l o p m e n t . 18 . 6 5 - 7 9 . Sat t ler , J . ( 1988 ) . A s s e s s m e n t of ch i ld ren (3rd ed. ) . S a n D iego , C A : J e r o m e M . Sat t le r , Pub l isher . S a t z , P . , & F le tcher , J . (1982) . M a n u a l for the Flor ida K indergar ten Sc reen ing Bat te ry . O d e s s a , FL : P s y c h o l o g i c a l A s s e s s m e n t R e s o u r c e s . S a t z , P . , & Fr ie l , J . F . (1978) . Pred ic t ive val id i ty of an abbrev ia ted sc reen ing bat tery . Jou rna l of Learn ing Disabi l i t ies. 11(6) . 3 4 7 - 3 5 1 . S a t z , P . , Tay lo r , H . , Fr ie l , J . , & F le tcher , J . (1979) . S o m e deve lopmenta l and pred ic t ive precursors of reading d isabi l i t ies: A s ix year fo l l ow-up . In J . Guthr ie (Ed.). A s p e c t s of reading acqu is i t i on . Ba l t imore, M D : J o h n Hopk ins P r e s s . S c h u l t e , A . , & M o o r e , S . ( 1 9 8 8 , Apr i l ) . M e t h o d s of va l idat ing p reschoo l sc reen ing d e c i s i o n s . Paper presented at the 20 th A n n u a l C o n v e n t i o n of the Nat ional A s s o c i a t i o n of S c h o o l P s y c h o l o g i s t s , C h i c a g o , IL. S c h u l t z , T . ( 1989 ) . Tes t i ng and retent ion of y o u n g ch i ld ren : M o v i n g f rom c o n t r o v e r s y to re form. Phi Del ta K a p p a n . 7 1 . 1 2 5 - 1 2 9 . S c o t t , L. (1981) . Measu r i ng inte l l igence w i th the Goodenough -Har r i s D raw ing Tes t . P s y c h o l o g i c a l Bul le t in . 8 9 . 4 8 3 - 5 0 5 . S h i p m a n , V . (1981) . Pe rspec t i ves in ear ly ident i f icat ion and in tervent ion . In H. S i l v e r m a n , I. D a v i d s o n , & L. We in t raub (Eds.) , Ear lv ident i f icat ion & in tervent ion : Se lec ted p roceed ings f rom the four th internat ional s y m p o s i u m on learning p rob lems. To ron to 1 9 7 9 . To ron to , Ontar io : M in is t ry of E d u c a t i o n . S imner , M . (1983) The warn ing s igns of s c h o o l fai lure: A n updated prof i le of the at-r isk k indergar ten ch i ld . T o p i c s in Early Ch i l dhood Spec ia l Educa t i on . 3 (3) , 1 7 - 2 7 . S k e e l s , H . ( 1966 ) . Adu l t s ta tus of ch i ldren w i th con t ras t ing ear ly life expe r iences . M o n o g r a p h s of the Soc ie t v for Research in Chi ld Deve lopmen t . 31 (3). 1-65. C h i c a g o : Un ivers i ty of C h i c a g o P ress . S ta t is t i cs C a n a d a , 1 9 9 1 . Adu l t l i teracy in C a n a d a : Resu l ts of a nat ional s tudy (Cata logue N o . 8 9 - 5 2 5 E ) . O t t a w a , Ontar io : S ta t is t ics C a n a d a . S t o n e , B. , C u n d i c k , B . P . , & S w a n s o n , D. (1988) . Spec ia l educa t i on sc reen ing s y s t e m : G roup ach ievemen t test . Excep t iona l Ch i l d ren . 55(1 ) . 7 1 - 7 5 . S t o n e , B. , & Tre loar , J . ( 1 9 9 0 , Apr i l ) . A c c u r a c y and bias of p reschoo l sc reen ing t es t s for pred ic t ing at-r isk s ta tus . Paper p resented at the 2 2 n d A n n u a l C o n v e n t i o n of the Nat iona l A s s o c i a t i o n of S c h o o l Psycho log i s t s , S a n F r a n c i s c o , C A . St r ichar t , S . , & Laza rus , P. (1986) . Low- i nc i dence a s s e s s m e n t : Inf luences and i s s u e s . In P. Lazarus & S . St r ichar t (Eds.) , P s y c h o e d u c a t i o n a l eva luat ion of ch i l ren and ado lescen ts w i t h l ow- inc idence hand i caps . Or lando , F L : Grune & S t ra t ton . Su l l i van , B. (1988) . A l eoacv for learners - The report of the Roya l c o m m i s s i o n o n Educa t i on . V i c t o r i a , B C : Q u e e n ' s Pr inters for Br i t ish C o l u m b i a . Tho rnd i ke , R., & H a g e n , E. (1984) . Canad ian Cogn i t i ve Ab i l i t ies Tes t : Techn i ca l N o t e s . C a n a d a : Ne l son C a n a d a . T j o s s e m , T . (1976) . Ear ly in tervent ion: Issues and a p p r o a c h e s . In T . T j ossem (Ed.) , Intervent ion s t ra teg ies for high risk in fants and y o u n g ch i ld ren. Ba l t imore , M D : Un ivers i ty Park P ress . Trami l l , J . , E d w a r d s , R., & Trami l l , J . (1980) . C o m p a r i s o n of the G o o d e n o u g h -Harr is d raw ing test and the W I S C - R for ch i ld ren exper ienc ing a c a d e m i c d i f f icu l t ies . Percep tua l and M o t o r Sk i l l s . ^ , 5 4 3 - 5 4 6 . W a n c z y c k i , W . (1983) . T h e k indergar ten ear ly ident i f icat ion c o m p o n e n t of Bill 8 2 . S p e c i a l Educat ion in C a n a d a . 57(1) , 2 5 - 3 2 . W e n d t , R . N . (1978) . K indergar ten en t rance a s s e s s m e n t : Is it wo r t h the ef fort? P s v c h o l o o y in the S c h o o l s . 1 5 . 5 6 - 6 2 . Wh i t e , B. (1975) . The f irst three years of l i fe. E n g l e w o o d C l i f f s , N J : Prent ice-Ha l l . Wh i t e , K .R . ( 1985 -86 ) . E f f i cacy of ear ly in tervent ion. T h e Jou rna l of Spec ia l E d u c a t i o n . 19(4) , 4 0 1 - 4 1 6 . Wh i t e , K . R . , Bush D . W . , & C a s t o , G . C . ( 1 9 8 5 - 8 6 ) . Learn ing f rom rev iews of early in tervent ion . The Jou rna l of Spec ia l Educa t i on . 19(4) . 4 1 7 - 4 2 8 . W i l s o n , B . J . , & Re i chmu th , M . (1985) . Ear ly sc reen ing p rog rams : W h e n is pred ic t ive a c c u r a c y suf f ic ien t? Learn ing Disabi l i ty Quar te r l y . 8 , 1 8 2 - 1 8 8 . W o l f e n d a l e , S . (1981) . Ear ly ident i f icat ion and in tervent ion. In H . S i l ve rman , I, D a v i d s o n , & L. We in t raub (Eds.) , Ear ly ident i f icat ion & in tervent ion: Se lec ted o roceed ina f rom the four th internat ional s v m o o s i u m on learning p rob lems . To ron to 1 9 7 9 (pp. 7 0 - 9 4 ) . To ron to , Ontar io : Ontar io Min is t ry of E d u c a t i o n . Ze i t l in , S . ( 1976 ) . K indergar ten sc reen ing : Ear lv ident i f icat ion o f potent ia l h igh-risk learners. Spr ing f ie ld , IL: Char les C . T h o m a s . 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0098885/manifest

Comment

Related Items