Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The development and norming of the Delta Tests of Mathematics Skills Cummins, John Martin 1988

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1989_A8 C85.pdf [ 7.61MB ]
Metadata
JSON: 831-1.0054372.json
JSON-LD: 831-1.0054372-ld.json
RDF/XML (Pretty): 831-1.0054372-rdf.xml
RDF/JSON: 831-1.0054372-rdf.json
Turtle: 831-1.0054372-turtle.txt
N-Triples: 831-1.0054372-rdf-ntriples.txt
Original Record: 831-1.0054372-source.json
Full Text
831-1.0054372-fulltext.txt
Citation
831-1.0054372.ris

Full Text

THE DEVELOPMENT AND NORMING OF THE DELTA TESTS OF MATHEMATICS SKILLS By JOHN MARTIN CUMMINS B.A., U n i v e r s i t y of Western Ontario, 1966 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS i n v THE FACULTY OF GRADUATE STUDIES The Measurement, Evaluation and Research Methodology Programme Department of Educational Psychology and Special Education We accept t h i s t h e s i s as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA September 1988 cj John Martin Cummins, 1988 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of cA t^c^oi^-t r^L cj c A c I o j «* al The University of British Columbia P Vancouver, Canada Date ^&p-\~<t.,—- U * r - t DE-6 (2/88) A b s t r a c t L i t e r a t u r e r e g a r d i n g t h e u s e o f n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s s u g g e s t s t h a t t h e y a r e u n s a t i s f a c t o r y f o r s y s t e m a t i c a l l y m o n i t o r i n g p u p i l p r o g r e s s . The s h o r t c o m i n g s o f t h e s e m e a s u r e s i n c l u d e : a ) t h e q u e s t i o n a b l e c o n g r u e n c e b e t w e e n t h e g r o u p b e i n g t e s t e d and t h e n o r m i n g g r o u p , b) t h e l a c k o f c o n t e n t v a l i d i t y w i t h r e s p e c t t o a s t u d e n t ' s c u r r i c u l u m , c ) t h e r e d u c e d d i a g n o s t i c u t i l i t y o f t h e s e t e s t s r e s u l t i n g f r o m t h e i r l i m i t e d i t e m c o n t e n t , d) t h e p e r c e i v e d i n a d e q u a c y o f s c o r e s d e r i v e d f r o m n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s and e ) t h e i n a b i l i t y t o a d m i n i s t e r s u c h t e s t s w i t h any r e g u l a r i t y w i t h o u t i n v a l i d a t i n g t h e m . C u r r i c u l u m - b a s e d a s s e s s m e n t ( C . B . A . ) s t r a t e g i e s h a v e been d e v e l o p e d i n r e s p o n s e t o t h e s e l i m i t a t i o n s . S u c h m e a s u r e s may a l l o w r e p e a t e d a d m i n i s t r a t i o n s , t h u s p e r m i t t i n g t h e measurement o f b o t h p r o f i c i e n c y and r a t e o f l e a r n i n g . I f normed on a d e f i n e d p o p u l a t i o n o f s t u d e n t s a t r e g u l a r i n t e r v a l s d u r i n g t h e s c h o o l y e a r , t h e d i s c r e p a n c y be tween a p a r t i c u l a r s t u d e n t ' s p e r f o r m a n c e and t h a t o f h i s p e e r s c o u l d e a s i l y be d e t e r m i n e d . The D e l t a T e s t s o f M a t h e m a t i c s S k i l l s (DTMS) a r e c u r r i c u l u m -b a s e d m e a s u r e s o f m a t h e m a t i c s f a c t s and m a t h e m a t i c s c o m p u t a t i o n s s k i l l s f o r G r a d e s 2 , 3 and 4. They a r e b a s e d on t h e B . C . M a t h e m a t i c s C u r r i c u l u m , normed i n t h e s c h o o l s o f D e l t a , B . C . and p e r m i t r e p e a t e d a d m i n i s t r a t i o n . T h i s s t u d y d e m o n s t r a t e s t h a t u n l i k e s t a n d a r d i z e d a c h i e v e m e n t t e s t s , t h e DTMS c a n measu re c h a n g e s i n a s t u d e n t ' s o r g r o u p ' s p e r f o r m a n c e on s p e c i f i c s k i l l s o v e r b r i e f (one month) p e r i o d s o f t i m e . i i i T a b l e o f C o n t e n t s A b s t r a c t i i L i s t o f T a b l e s v i i L i s t o f F i g u r e s x i v A c k n o w l e d g e m e n t x v i D e d i c a t i o n x v i i C H A P T E R 1 - I N T R O D U C T I O N 1 G e n e r a l S t a t e m e n t o f t h e P r o b l e m 1 The P u r p o s e o f t h e S t u d y 5 S t a t e m e n t o f H y p o t h e s i s 6 O r g a n i z a t i o n o f t h e R e m a i n i n g C h a p t e r s 7 C H A P T E R 2 - R E V I E W O F R E L A T E D L I T E R A T U R E 8 N o r m - R e f e r e n c e d A c h i e v e m e n t T e s t s 8 G r o u p C o m p a r a b i l i t y 9 A c h i e v e m e n t T e s t / C u r r i c u l u m C o n g r u e n c e 10 The L i m i t e d I t e m C o n t e n t o f A c h i e v e m e n t T e s t s . . 13 The I n a d e q u a c y o f D e r i v e d S c o r e s 13 C u r r i c u l u m - B a s e d A s s e s s m e n t (CBA) 19 CBA: The Mount P l e a s a n t M o d e l 20 CBA: The P i n e C o u n t y M o d e l 21 The Two M o d e l s : A Summary 22 CBA - G o a l and CBA - O b j e c t i v e 23 Summary 25 C H A P T E R 3 - C U R R I C U L U M - B A S E D A S S E S S M E N T V S . S T A N D A R D I Z E D A C H I E V E M E N T T E S T S 26 The D e l t a T e s t s o f M a t h e m a t i c s S k i l l s (DTMS) : An O v e r v i e w 26 i v The Math Facts Tests of the DTMS 27 The Computations Tests of the DTMS 28 The Canadian Tests of Basic S k i l l s (CTBS) 29 The B.C. Quick Ind i v i d u a l Educational Test <The B.C.Quiet) 33 A Comparison of the DTMS and the CTBS 34 A Comparison of Student Performance on the DTMS and the CTBS 37 A Comparison of the DTMS and The B.C. Quiet . . . . 41 Summary 41 CHAPTER 4 - METHOD 43 The Study: Purpose and Hypotheses 43 The Community 43 An Overview 44 The Development of the Math Facts Tests of the DTMS 47 The Development of the Computation Tests of the DTMS 47 Norming the DTMS 48 The Subjects . 48 Testing Procedures 50 Summary 52 CHAPTER 5 - DATA ANALYSIS 53 R e l i a b i l i t y and V a l i d i t y 53 R e l i a b i l i t y and V a l i d i t y : Discussion 64 Mean Scores 67 Mean Scores: Discussion 73 Trend A n a l y s i s 75 Trend A n a l y s i s : Discussion 78 V Cumulative Percentage Polygons (Ogives) 80 Cumulative Percentage Polygons: Discussion . . . . 82 Summary 82 CHAPTER 6 - INTERPRETATION AND USE OF THE DTMS . . . 84 Overview 84 Conclusions 88 I n t e r p r e t a t i o n and Use of the DTMS 90 Summary 95 References . 96 Appendices A - Tables Comparing the DTMS and the CTBS . . . 100 B - Section I: L i s t of Questions which f o r the Purposes of This Study Are Considered "Math Facts" 125 Section I I : The Mathematics Facts Tests of the Delta Tests of Mathematics S k i l l s 129 Section I I I : L i s t of Computation S k i l l s Taught at Each Grade Level 134 Section IV: The Computations Subtests of the DTMS 141 C - The Ins t r u c t i o n s Given to Pupil s Being Administered the Delta Tests of Mathematics S k i l l s Section I: D i r e c t i o n s - Math Facts (Complete) 152 - Math Facts (Short Form) 154 Section I I : D i r e c t i o n s - Computation S k i l l s (Complete) 155 D i r e c t i o n s - Computation S k i l l s (Short Forms) 157 D - D e s c r i p t i v e S t a t i s t i c s of Indi v i d u a l Functions Section I: Test-Retest R e l i a b i l i t y C o e f f i c i e n t s Between Sessions 159 Section I I : The Sessional Means, Standard Deviations and Number of V a l i d Cases of the Indi v i d u a l Functions Tested with the DTMS at Grades 2, 3 and 4 169 v i S e c t i o n I I I : T h e A n a l y s i s o f V a r i a n c e S u m m a r i e s a n d T u k e y ' s H S D ( H o n e s t l y - S i g n i f i c a n t -D i f f e r e n c e ) f o r t h e DTMS T e s t S c o r e s , S e s s i o n s 1 t o 4. 1 7 5 S e c t i o n I V : T r e n d A n a l y s i s S u m m a r i e s a n d r *•, t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r b y S i g n i f i c a n t T r e n d s f o r t h e DTMS S e s s i o n a l M e a n s 1 8 9 E - S e c t i o n I : T h e B e s t F i t t i n g C u r v e s I n d i c a t e d b y T r e n d A n a l y s i s f o r T o t a l M a t h e m a t i c s 2 0 4 S c o r e s o n t h e DTMS S e c t i o n I I : C u m u l a t i v e P e r c e n t a g e P o l y g o n s f o r T o t a l M a t h e m a t i c s S c o r e s o n t h e DTMS 2 1 0 S e c t i o n I I I : R e a l a n d P r o j e c t e d M e a n s a n d S t a n d a r d D e v i a t i o n s f o r T o t a l M a t h e m a t i c s S c o r e s o n t h e DTMS 2 1 7 L i s t of Tables v i i TABLE 1 The Kuder-Richardson Formula 20 Internal Consistency R e l i a b i l i t y C o e f f i c i e n t s f o r the CTBS Mathematics S k i l l s Tests . . . . . . . 32 TABLE 2 A Numerical Summary of the Categorization of the CTBS Test Items, E f f e c t e d i n Tables A - l to A-9. 35 TABLE 3 The Squared Pearson C o r r e l a t i o n C o e f f i c i e n t s between Student Scores on the CTBS Computations Test and t h e i r Mean Total Mathematics Fact Scores (TOTFAC) and t h e i r Mean Total Computations S k i l l s Scores (TOTCOMP) on the DTMS 39 TABLE 4 A Table Showing the Frequency with which S k i l l s are Tested i n the B.C. Quiet Mathematics Subtest ' 40 TABLE 5 Functions Tested at each Grade Level by the DTMS 46 TABLE 6 Test Schedule f o r the Norming of the DTMS . . 46 TABLE 7 Table Showing the Number of Students by School P a r t i c i p a t i n g i n the Norming of the DTMS at 1 each Grade as of the F i r s t Testing Session . . 49 TABLE 8 Table Showing the Order of Presentation of Tests during Norming Sessions 51 TABLE 9 R e l i a b i l i t y C o e f f i c i e n t s f o r Each Test of the DTMS Obtained by Averaging the Co r r e l a t i o n s Calculated between Sessions 1 and 2, 2 and 3 and 3 and 4 as Reported i n Tables D-l to D-6. 59 TABLE 10 R e l i a b i l i t y C o e f f i c i e n t s f o r Total Mathematics Facts Correct and Total Mathematics Computations S k i l l s Correct Obtained by Averaging the Cor r e l a t i o n s Calculated between Sessions 1 and 2, 2 and 3 and 3 and 4 as Reported i n Tables D-7 to D-9 62 TABLE 11 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s between Sessions 1 and 2, 1 and 3, and 1 and 4 Scores f o r T o t a l Mathematics Facts Correct and Total Mathematics Computations S k i l l s Correct as Reported i n Tables D-7 to D-9. 63 V 1 1 X TABLE 12 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s between Sessions 1 and 2, 1 and 3, and 1 and 4 Mathematics Facts and Mathematics Computations S k i l l s Scores f o r each Function Tested as Reported i n Tables D-l to D-6 65 TABLE 13 The Mean, the Standard Deviation and the Number of V a l i d Cases of the Total Mathematics Facts Correct Scores and the Total Mathematics Computations S k i l l s Correct Scores at Grade 2 i n Testing Sessions 1 through 4. 68 TABLE 14 The Mean, the Standard Deviation and the Number of V a l i d Cases of the Total Mathematics Facts Correct Scores and the Total Mathematics Computation S k i l l s Scores Correct at Grade 3 i n Testing Sessions 1 through 4. 69 TABLE 15 The Mean, the Standard Deviation and the Number of V a l i d Cases of the Total Mathematics Facts Correct Scores and the Total Mathematics Computation S k i l l s Correct Scores at Grade 4 i n Testing Sessions 1 through 4. 70 TABLE 16 S i g n i f i c a n t l y D i f f e r e n t Means as Determined Using Tukey's HSD ( H o n e s t l y - S i g n i f i c a n t -Difference) Test on the Sessional Mean Total Mathematics Fact Scores (TOTFAC) and the Sessional Mean To t a l Computation S k i l l s Scores (TOTCOMP). 72 TABLE 17 S i g n i f i c a n t l y D i f f e r e n t Means as Determined Using Tukey's HSD ( H o n e s t l y - S i g n i f i c a n t -Difference) Test on the Sessional Mean Scores of the Functions and S k i l l s Tested at each Grade. , 74 TABLE 18 S i g n i f i c a n t Trends Found f o r the Sessional Mean Total Mathematics Fact Scores (TOTFAC) and the Sessional Mean Total Computation S k i l l s Scores (TOTCOMP) by Trend Anal y s i s Using Orthogonal Polynomials. 77 TABLE 19 S i g n i f i c a n t Trends Found f o r the Sessional Mean Scores of the Functions and S k i l l s Tested at each Grade by Trend Analysis using Orthogonal Polynomials. 79 TABLE A - l TABLE A-2 TABLE A - 3 TABLE A-4 TABLE A - 5 TABLE A-6 TABLE A-7 TABLE A - 8 TABLE A - 9 TABLE D - l TABLE D-2 TABLE D-3 i x A C o m p a r i s o n o f t h e G r a d e 2 A d d i t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS 1 0 1 A C o m p a r i s o n o f t h e G r a d e 2 S u b t r a c t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS 103 A C o m p a r i s o n o f t h e G r a d e 3 A d d i t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS 105 A C o m p a r i s o n o f t h e G r a d e 3 S u b t r a c t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS 108 A C o m p a r i s o n o f t h e G r a d e 3 M u l t i p l i c a t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS I l l A C o m p a r i s o n o f t h e G r a d e 4 A d d i t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS 113 A C o m p a r i s o n o f t h e G r a d e 4 S u b t r a c t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS 117 A C o m p a r i s o n o f t h e G r a d e 4 M u l t i p l i c a t i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS and t h e CTBS 120 A C o m p a r i s o n o f t h e G r a d e 4 D i v i s i o n C o m p u t a t i o n S k i l l s T e s t e d by t h e DTMS a n d t h e CTBS 1 2 3 G r a d e 2 T e s t - R e t e s t R e l i a b i l i t y C o e f f i c i e n t s f o r S e s s i o n s 1 t o 4 S c o r e s , A d d i t i o n and S u b t r a c t i o n F a c t s T e s t s o f t h e DTMS . . . . 160 G r a d e 2 T e s t - R e t e s t R e l i a b i l i t y C o e f f i c i e n t s f o r S e s s i o n s 1 t o 4 S c o r e s , A d d i t i o n and S u b t r a c t i o n C o m p u t a t i o n S k i l l s T e s t s o f t h e DTMS 161 G r a d e 3 T e s t - R e t e s t R e l i a b i l i t y C o e f f i c i e n t s f o r S e s s i o n s 1 t o 4 S c o r e s , A d d i t i o n , S u b t r a c t i o n and M u l t i p l i c a t i o n F a c t s T e s t s o f t h e DTMS 162 X TABLE D-4 Grade 3 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s f o r Sessions 1 to 4 Scores, Addition, Subtraction and M u l t i p l i c a t i o n Computation S k i l l s Tests of the DTMS 163 TABLE D-5 Grade 4 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s f o r Sessions 1 to 4 Scores, Addition, Subtraction, M u l t i p l i c a t i o n and D i v i s i o n Facts Tests of the DTMS 164 TABLE D-6 TABLE D-7 Grade 4 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s f o r Sessions 1 to 4 Scores, A d d i t i o n , Subtraction, M u l t i p l i c a t i o n and D i v i s i o n Computation S k i l l s Tests of the DTMS . . . 165 Grade 2 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s f o r Sessions 1 to 4 Scores f o r T o t a l Mathematics Facts and Total Mathematics Computation S k i l l s Scores on the DTMS 166 TABLE D-8 TABLE D-9 TABLE D-10 TABLE D - l l TABLE D-12 TABLE D-13 Grade 3 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s f o r Sessions 1 to 4 Scores f o r Total Mathematics Facts and To t a l Mathematics Computation S k i l l s Scores on the DTMS 167 Grade 4 Test-Retest R e l i a b i l i t y C o e f f i c i e n t s f o r Sessions 1 to 4 Scores f o r Total Mathematics Facts and To t a l Mathematics Computation S k i l l s Scores on the DTMS 168 Des c r i p t i v e S t a t i s t i c s of the Addi t i o n and Subtraction Facts and Computations S k i l l s Scores at Grade 2 i n Testing Sessions 1 Through 4. . 170 De s c r i p t i v e S t a t i s t i c s of the Addi t i o n and Subtraction Facts and Computations S k i l l s Scores at Grade 3 i n Testing Sessions 1 Through 4. 171 Des c r i p t i v e S t a t i s t i c s of the M u l t i p l i c a t i o n Facts and Computations S k i l l s Scores at Grade 3 i n Testing Sessions 1 Through 4. . . 172 Des c r i p t i v e S t a t i s t i c s of the Addition and Subtraction Facts and Computations S k i l l s Scores at Grade 4 i n Testing Sessions 1 Through 4. 173 x i TABLE D-14 TABLE D-15 TABLE D-16 TABLE D-17 De s c r i p t i v e S t a t i s t i c s of the M u l t i p l i c a t i o n and D i v i s i o n Facts and Computations S k i l l s Scores at Grade 4 i n Testing Sessions 1 Through 4. 174 Ana l y s i s of Variance Summary, and Tukey's HSD (Hone s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 2 Total Mathematics Facts Correct and Total Mathematics Computations S k i l l s . . . . . 177 Ana l y s i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 3 Tot a l Mathematics Facts Correct and To t a l Mathematics Computations S k i l l s . . . . . 178 Ana l y s i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 4 Tot a l Mathematics Facts Correct and To t a l Mathematics Computations S k i l l s . . . . . 179 TABLE D-18 Analys i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 2 Addi t i o n Facts Correct and Add i t i o n Computation S k i l l s Correct. 180 TABLE D-19 TABLE D-20 Analys i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 2 Subtraction Facts Correct and Subtraction Computation S k i l l s Correct. 181 Ana l y s i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 3 Addition Facts Correct and Add i t i o n Computation S k i l l s Correct. 182 TABLE D-21 TABLE D-22 TABLE D-23 Ana l y s i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 3 Subtraction Facts Correct and Subtraction Computation S k i l l s Correct. 183 Ana l y s i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 3 M u l t i p l i c a t i o n Facts Correct and M u l t i p l i c a t i o n Computation S k i l l s Correct. 184 Analys i s of Variance Summary, and Tukey's HSD (Hon e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r Grade 4 Addi t i o n Facts Correct and Add i t i o n Computation S k i l l s Correct. . . . . . . . . 185 A n a l y s i s o f V a r i a n c e Summary, and T u k e y ' s HSD ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r G r a d e 4 S u b t r a c t i o n F a c t s C o r r e c t and S u b t r a c t i o n C o m p u t a t i o n S k i l l s C o r r e c t . 186 A n a l y s i s o f V a r i a n c e Summary, and T u k e y ' s HSD ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r G r a d e 4 M u l t i p l i c a t i o n F a c t s C o r r e c t and M u l t i p l i c a t i o n C o m p u t a t i o n S k i l l s C o r r e c t . 187 A n a l y s i s o f V a r i a n c e Summary, and T u k e y ' s HSD ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r G r a d e 4 D i v i s i o n F a c t s C o r r e c t and D i v i s i o n C o m p u t a t i o n S k i l l s C o r r e c t . 188 T r e n d A n a l y s i s Summary and r , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 2 T o t a l M a t h e m a t i c s F a c t s C o r r e c t 190 T r e n d A n a l y s i s Summary and r a , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T remds f o r G r a d e 2 T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s C o r r e c t . 191 T r e n d A n a l y s i s Summary and r 3*, t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 3 T o t a l M a t h e m a t i c s F a c t s C o r r e c t and T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s C o r r e c t . 192 T r e n d A n a l y s i s Summary and r a , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 4 T o t a l M a t h e m a t i c s F a c t s C o r r e c t and T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s C o r r e c t . 193 T r e n d A n a l y s i s Summary and r l , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f a r G r a d e 2 A d d i t i o n F a c t s C o r r e c t a n d A d d i t i o n C o m p u t a t i o n s S k i l l s C o r r e c t . . . 194 T r e n d A n a l y s i s Summary and r * , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 2 S u b t r a c t i o n F a c t s C o r r e c t and S u b t r a c t i o n C o m p u t a t i o n S k i l l s C o r r e c t . . . 195 T r e n d A n a l y s i s Summary and r a , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 3 A d d i t i o n F a c t s C o r r e c t and A d d i t i o n C o m p u t a t i o n s S k i l l s C o r r e c t . . . 196 x i i i TABLE D-34 T r e n d A n a l y s i s Summary and r l , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 3 S u b t r a c t i o n F a c t s C o r r e c t and S u b t r a c t i o n C o m p u t a t i o n s S k i l l s C o r r e c t . 197 TABLE D-35 T r e n d A n a l y s i s Summary and r S t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 3 M u l t i p l i c a t i o n F a c t s C o r r e c t and M u l t i p l i c a t i o n C o m p u t a t i o n s S k i l l s C o r r e c t . 198 TABLE D-36 T r e n d A n a l y s i s Summary and r a , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 4 A d d i t i o n F a c t s C o r r e c t and A d d i t i o n C o m p u t a t i o n s S k i l l s C o r r e c t . . . . 199 TABLE D-37 T r e n d A n a l y s i s Summary and r a , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 4 S u b t r a c t i o n F a c t s C o r r e c t and S u b t r a c t i o n C o m p u t a t i o n s S k i l l s C o r r e c t . 200 TABLE D-38 T r e n d A n a l y s i s Summary and r i , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 4 M u l t i p l i c a t i o n F a c t s C o r r e c t and M u l t i p l i c a t i o n C o m p u t a t i o n s S k i l l s C o r r e c t . 201 TABLE D-39 T r e n d A n a l y s i s Summary a n d xz, t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r by S i g n i f i c a n t T r e n d s f o r G r a d e 4 D i v i s i o n F a c t s C o r r e c t and D i v i s i o n C o m p u t a t i o n s S k i l l s C o r r e c t . . . 202 xiv L i s t o f F i g u r e s FIGURE 1 Cognitive Growth by the Test Publisher's 50th P e r c e n t i l e C h i l d versus a More R e a l i s t i c Expectation 15 FIGURE 2 Comparison of the Median (50th P e r c e n t i l e ) Score and the Grade Norm Line. 18 FIGURE 3 D i s t r i b u t i o n of Test-Retest R e l i a b i l i t y C o e f f i c i e n t s Calculated Between Sessions 1 and 2, 2 and 3 and 3 and 4 f o r Individual Functions Reported i n Tables D-l Through D-6 58 FIGURE 4 D i s t r i b u t i o n of the Test-Retest R e l i a b i l i t y C o e f f i c i e n t s Calculated Between Sessions 1 and 2, 2 and 3 and 3 and 4 f o r Total Scores Correct as Reported i n Tables D-7, D-8 and D-9. . . 61 FIGURE 5 Hypothetical Grade 2 Total Mathematics Facts Correct Scores P l o t t e d on Grid Containing Actual Means and Standard Deviations. . . . 91 FIGURE 6 Grade 2 Total Mathematics Facts Sessional Means and Standard Deviations. 93 FIGURE E - l The Best F i t t i n g Curves Indicated by Trend Analysis f o r Grade 2 Total Mathematics Facts Correct Scores. 204 FIGURE E-2 The Best F i t t i n g Curves Indicated by Trend Anal y s i s f o r Grade 2 Total Computations S k i l l s Correct Scores. 205 FIGURE E-3 The Best F i t t i n g Curves Indicated by Trend Analysis f o r Grade 3 Total Mathematics Facts Correct Scores. 206 FIGURE E-4 The Best F i t t i n g Curves Indicated by Trend Analysis f o r Grade 3 Total Computations S k i l l s Correct Scores. 207 FIGURE E-5 The Best F i t t i n g Curve Indicated by Trend Analysis f o r Grade 4 Total Mathematics Facts Correct Scores. 208 FIGURE E-6 The Best F i t t i n g Curves Indicated by Trend Analysis f o r Grade 4 Total Computations S k i l l s Correct Scores. . . . . 209 XV FIGURE E-7 Cumulative Percentage Polygon f o r Total Mathematics Facts Correct, Grade 2. . . . . 211 FIGURE E-8 Cumulative Percentage Polygon f o r Total Mathematics Computations Correct, Grade 2 . . 212 FIGURE E-9 Cumulative Percentage Polygon f o r Total Mathematics Facts Correct, Grade 3. . . . . 213 FIGURE E-10 Cumulative Percentage Polygon f o r Total Mathematics Computations Correct, Grade 3. . 214 FIGURE E - l l Cumulative Percentage Polygon f o r Total Mathematics Facts Correct, Grade 4. . . . . 215 FIGURE E-12 Cumulative Percentage Polygon f o r Total Mathematics Computations Correct, Grade 4. . 216 FIGURE E-13 Grade 2 Total Mathematics Computations S k i l l s Sessional Means and Standard Deviations. . . 218 FIGURE E-14 Grade 3 Total Mathematics Facts Sessional Means and Standard Deviations. . . . \ . . 219 FIGURE E-15 Grade 3 T o t a l Mathematics Computations S k i l l s Sessional Means and Standard Deviations. . . 220 FIGURE E-16 Grade 4 Total Mathematics Facts Sessional Means and Standard Deviations. 221 FIGURE E-17 Grade 4 Total Mathematics Computations S k i l l s Sessional Means and Standard Deviations. . . 222 x v i Acknowledgement I would l i k e to thank ray advisor, Dr. Donald F. A l l i s o n , f o r h i s wit, wisdom, and a b i l i t y to keep things i n perspective during the course of t h i s study. In a d d i t i o n , I would l i k e to express my a p p r e c i a t i o n to the members of my committee, Dr. David A. Bain and Dr. Harold R a t z l a f f , f o r t h e i r e f f o r t s on my behalf and to Dr. Walter Boldt f o r h i s advice and a c c e s s i b i l i t y . L a s t l y , I would l i k e to thank the teachers of Delta who p a r t i c i p a t e d i n the norming and most e s p e c i a l l y Darlene Nordheimer, without whose timely o f f e r of assistance t h i s study might have floundered before i t s t a r t e d and without whose continued e f f o r t s i t might never have been completed. D e d i c a t i o n T h i s T h e s i s i s d e d i c a t e d i n l o v i n g m e m o r y o f my w i f e P h y l l i s K a t h r i n e ' T r a c e y ' Cummins ( n e e K u r t z ) B . A . , U n i v e r s i t y o f B r i t i s h C o l u m b i a , 1 9 6 3 Who p a s s e d a w a y i n D e l t a , B . C . O n J u l y 3 1 , 1 9 8 8 F o r t o t h o s e w h o h a v e f a i t h i n Y o u , L o r d , l i f e d o e s n o t c e a s e - -i t m e r e l y c h a n g e s ! A n d w h e n o u r e a r t h l y d w e l l i n g f a l l s t o r u i n s we f i n d a n e t e r n a l h o m e i n h e a v e n . 1 CHAPTER 1 - INTRODUCTION General Statement of the Problem T r a d i t i o n a l l y educators have looked to the r e s u l t s of commercial norm-referenced achievement t e s t s f o r guidance i n generating i n d i v i d u a l i z e d educational programs and f o r assessing the e f f e c t s of those programs (Ti n d a l , Fuchs, Fuchs, Shinn, Deno and Germann, 1985). This r e l i a n c e on standardized t e s t s has not gone without c r i t i c i s m . Caltagirone and Glover (1985) note that "increased p r o f e s s i o n a l s c r u t i n y has led to a r i s i n g t i d e of c r i t i c i s m s which question the evaluative usefulness of both the pre-post technique and the achievement instruments themselves." (p.355) They i d e n t i f y the more pertinent c r i t i c i s m s as: "(a) the t y p i c a l l y poor congruence between t e s t items and curriculum content . . . (b) the reduced diag n o s t i c u t i l i t y of these t e s t s associated with t h e i r l i m i t e d item content . . . and (c) the f a c t that evaluations performed on an annual basis represent both a summative and re t r o s p e c t i v e a n a l y s i s of le a r n i n g . " (p.356) Mirkin , Fuchs and Deno (1982) note that "norm-referenced achievement t e s t s are unsuitable f o r ongoing monitoring of the appropriateness of educational programs and frequently lack content v a l i d i t y with respect to a student's curriculum." (p.11) This f i n a l c r i t i c i s m , the f a i l u r e of these measures to r e l a t e to c u r r i c u l a r requirements, i s supported by an a n a l y s i s , d e t a i l e d i n Chapter 3, which demonstrates the lack of concurrence between the Mathematics Computation Subtest of the Canadian Tests of Basic 2 S k i l l s ( C T B S ) , a n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t , a n d t h e M a t h e m a t i c s C o m p u t a t i o n s T e s t s o f t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ( D T M S ) , t e s t s d e s i g n e d t o r e f l e c t t h e c u r r i c u l a r r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m f o r G r a d e s 2 , 3 , a n d 4 a s i n t e r p r e t e d b y D e l t a t e a c h e r s . T h e l a c k o f c o n c u r r e n c e b e t w e e n t h e M a t h e m a t i c s S u b t e s t o f t h e B . C . Q u i e t a n d t h e DTMS i s a l s o d e m o n s t r a t e d . S a l m o n - C o x ( 1 9 8 1 ) r e p o r t s t h a t s t a n d a r d i z e d t e s t s a r e r e l a t i v e l y u n i m p o r t a n t i n t h e d a i l y d e c i s i o n - m a k i n g o f t h e c l a s s r o o m t e a c h e r a n d t h a t i n f o r m a l o b s e r v a t i o n s o f p u p i l p e r f o r m a n c e o n c l a s s r o o m a s s i g n m e n t s w a s t h e m o s t c o m m o n l y i d e n t i f i e d m e t h o d u s e d b y t e a c h e r s t o a s s e s s t h e i r p u p i l s . S a l m o n - C o x a l s o n o t e s t h a t t e a c h e r s e x p r e s s e d a p r e f e r e n c e f o r c o n t i n u o u s p i c t u r e s o f p u p i l p r o g r e s s r a t h e r t h a n s i n g l e s n a p s h o t s . F u c h s , F u c h s a n d W a r r e n ( 1 9 8 2 ) e x a m i n e d t h e a b i l i t y o f t e a c h e r s t o m a k e j u d g m e n t s a b o u t w h e t h e r o r n o t s t u d e n t s a c h i e v e d l e s s o n o b j e c t i v e s . T h e y d i s c o v e r e d t h a t t e a c h e r s w e r e g e n e r a l l y o v e r l y o p t i m i s t i c w h e n j u d g i n g w h e t h e r o r n o t t h e i r p u p i l s h a d s u c c e s s f u l l y m a s t e r e d o b j e c t i v e s a n d t h a t t h e d i s c r e p a n c y b e t w e e n a c t u a l p u p i l p e r f o r m a n c e a n d t e a c h e r j u d g m e n t s o f t h a t p e r f o r m a n c e w e r e s t a t i s t i c a l l y s i g n i f i c a n t . D e n o , J e n k i n s a n d M i r k i n ( 1 9 7 9 ) n o t e t h a t a c h i e v e m e n t t e s t s a r e u n s a t i s f a c t o r y f o r s y s t e m a t i c a l l y m o n i t o r i n g p u p i l p r o g r e s s b e c a u s e t h e y c a n n o t b e a d m i n i s t e r e d w i t h a n y r e g u l a r i t y w i t h o u t i n v a l i d a t i n g t h e t e s t . M i r k i n , F u c h s a n d D e n o ( 1 9 8 2 ) s u g g e s t t h a t 3 achievement t e s t s are too long to be administered r e g u l a r l y . These t e s t s seem to provide l i t t l e more than an overview of a pu p i l ' s general a b i l i t i e s where comparisons are made to a norming population which may have l i t t l e i n common with the l o c a l sample (Ca1tagirone, 1982). In a d d i t i o n , no information i s provided as to the rate at which s k i l l s were acquired or the rate at which the learning of s i m i l a r s k i l l s i s l i k e l y to occur. Several problems can a r i s e i n t e r p r e t i n g scores generated from achievement t e s t s . For example, student scores are often reported as grade equivalent scores (GES). GES may be based on a grading system or curriculum which d i f f e r s s u b s t a n t i a l l y from the system i n which the student tested operates. Most importantly GES are of l i t t l e use i n program evaluation because they are not equal i n t e r v a l u n i t s and cannot, therefore, be aggregated and averaged. Furthermore, GES are generally estimates. They do not represent the actual marks of t h i r d grade students on a f i f t h grade t e s t . Rather, they represent the t e s t constructor's best guess at how such students would perform. F i n a l l y , GES do not normally provide information regarding a student's r e l a t i v e standing as p r e c i s e l y as p e r c e n t i l e scores. However, while p e r c e n t i l e s can provide accurate information on r e l a t i v e standing they too cannot be aggregated and averaged because they do not represent equal i n t e r v a l u n i t s (Deno, Jenkins and Mirkin, 1979). As d e t a i l e d i n Chapter 2, Tallmadge (1977) notes that grade equivalent norms are produced i n such a way that low performing students can be shown 4 to make more than one month's growth per month i n school when i n f a c t they are f a l l i n g f u r t her and further behind. Concerns, such as those noted above, about the adequacy of standardized achievement t e s t s f o r assessing and monitoring p u p i l progress through the mathematics curriculum prompted the in v e s t i g a t i o n s which r e s u l t e d i n t h i s study. I t was determined that an acceptable a l t e r n a t i v e to standardized achievement t e s t s must overcome these d e f i c i t s . Such measures must be curriculum-based and must permit repeated administrations. This would allow f o r the measurement of both p r o f i c i e n c y and rate of lea r n i n g . I f normed on a defined population of students at regular i n t e r v a l s during a school year the discrepancy between a p a r t i c u l a r student's t e s t score(s) and the mean performance of h i s peers could be c a l c u l a t e d as well as the discrepancy between h i s rate of learning and that of h i s peers. I t should be noted that f o r the purposes of t h i s study the terms "math f a c t " and "computation s k i l l " have d i s t i n c t and pr e c i s e meanings and should not be used interchangeably. Math Fact: "Math f a c t s " are simple one step operations to which the answers are normally memorized. Thus, a student would be expected to supply from memory the answers to questions such as: 2 + 2 = , 3 - 2 = , 2 x 3 = , and 9 - 3 = . He would not be expected to have to compute the answer. (The universe of questions which f o r the purposes of t h i s study are considered "math f a c t s " i s contained i n Appendix B). 5 C o m p u t a t i o n S k i l l : T h e t e r m " c o m p u t a t i o n s k i l l " r e f e r s t o t h e a b i l i t y t o p e r f o r m t h e m a t h e m a t i c a l o p e r a t i o n s n e c e s s a r y t o s o l v e q u e s t i o n s s u c h a s 56 + 75 = , 72 - 34 = , 53 x 129 = , a n d 298 - 24 = . T h e m a t h e m a t i c a l c a p a b i l i t i e s r e q u i r e d t o s o l v e s u c h q u e s t i o n s i n c l u d e s t h e a b i l i t y t o r e c a l l , o r a t l e a s t t h e a b i l i t y t o c a l c u l a t e , b a s i c m a t h f a c t s . O b v i o u s l y t h e a n s w e r s t o q u e s t i o n s r e q u i r i n g t h e a p p l i c a t i o n o f c o m p u t a t i o n s k i l l s w o u l d n o t n o r m a l l y b e c o m m i t t e d t o m e m o r y . The P u r p o s e o f t h e S t u d y I n r e s p o n s e t o t h e a f o r e m e n t i o n e d s h o r t c o m i n g s o f s t a n d a r d i z e d a c h i e v e m e n t t e s t s a n d i n r e c o g n i t i o n o f t h e f a c t t h a t , a s d e t a i l e d i n C h a p t e r 3, t h e r e i s a c o n s i d e r a b l e d i s c r e p a n c y b e t w e e n t h e c o m p u t a t i o n s k i l l s r e q u i r e d b y t h e B . C . M a t h e m a t i c s C u r r i c u l u m a n d t h o s e r e q u i r e d b y t h e m a t h e m a t i c s t e s t s o f t w o a c h i e v e m e n t t e s t s i n c o m m o n u s e i n D e l t a ( T h e C a n a d i a n T e s t s o f B a s i c S k i l l s ( C T B S ) a n d t h e B . C . Q u i e t ) , m e a s u r e s o f m a t h e m a t i c s c o m p u t a t i o n s k i l l s a n d m e a s u r e s o f f a c i l i t y w i t h m a t h e m a t i c s f a c t s w e r e d e v e l o p e d f o r e a c h o f G r a d e s 2, 3 a n d 4 ( T h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ) . E a c h o f t h e s e t e s t s w a s d e s i g n e d t o m e a s u r e s t u d e n t a c h i e v e m e n t i n o n e o f t h e f o u r b a s i c f u n c t i o n s : a d d i t i o n , s u b t r a c t i o n , m u l t i p l i c a t i o n a n d d i v i s i o n . T e s t i t e m s w e r e s e l e c t e d t o r e f l e c t t h e r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m f o r e a c h g r a d e l e v e l a s i n t e r p r e t e d b y t e a c h e r s i n t h e D e l t a S c h o o l D i s t r i c t . T h e r e l i a b i l i t y o f t h e DTMS w a s e s t a b l i s h e d d u r i n g t h e i r n o r m i n g o n D e l t a ' s s t u d e n t s . 6 Statement of Hypotheses The f o l l o w i n g h y p o t h e s e s we re s t a t e d t o f o c u s t h e d e v e l o p m e n t o f t h i s s t u d y and t o o f f e r d i r e c t i o n i n t h e a n a l y s i s o f t h e d a t a r e s u l t i n g f r o m i t s i m p l e m e n t a t i o n : 1. C u r r i c u l u m - b a s e d a c h i e v e m e n t t e s t s s u c h a s t h e D e l t a - T e s t s o f M a t h e m a t i c s S k i l l s w h i c h a s s e s s a c h i e v e m e n t i n m a t h e m a t i c s f a c t s and m a t h e m a t i c s c o m p u t a t i o n s k i l l s c a n r e v e a l s i g n i f i c a n t c h a n g e s i n s t u d e n t p e r f o r m a n c e o v e r b r i e f (one month) p e r i o d s o f t i m e . T h e s e c h a n g e s c o u l d n o t be m e a s u r e d on s t a n d a r d i z e d a c h i e v e m e n t t e s t s s u c h a s t h e C T B S . 2. C u r r i c u l u m - b a s e d a c h i e v e m e n t t e s t s s u c h a s t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s w h i c h a s s e s s a c h i e v e m e n t i n m a t h e m a t i c s f a c t s and m a t h e m a t i c s c o m p u t a t i o n s k i l l s c a n r e v e a l s i g n i f i c a n t g r o w t h i n g r o u p p e r f o r m a n c e o v e r b r i e f (one month) p e r i o d s o f t i m e . T h i s g r o w t h c o u l d n o t be m e a s u r e d on s t a n d a r d i z e d a c h i e v e m e n t t e s t s s u c h a s t h e C T B S . S u p p o r t f o r t h e s e h y p o t h e s e s was g e n e r a t e d by n o r m i n g t h e DTMS i n t h e e l e m e n t a r y s c h o o l s o f D e l t a . I t s h o u l d be n o t e d t h a t i n h y p o t h e s i z i n g s i g n i f i c a n t g r o w t h i n g r o u p p e r f o r m a n c e o v e r b r i e f (one month) p e r i o d s o f t i m e i t was a n t i c i p a t e d t h a t a t e a c h g r a d e l e v e l , i n e a c h o f t h e s k i l l a r e a s t e s t e d , t h e m o n t h l y mean t o t a l c o r r e c t s c o r e s w o u l d be s e e n t o i n c r e a s e s i g n i f i c a n t l y o v e r t h e p r e v i o u s m o n t h ' s s c o r e s . T h u s , f o r e x a m p l e , i n t h i s s t u d y G r a d e 2 s t u d e n t s we re t e s t e d on a d d i t i o n and s u b t r a c t i o n f a c t s o n c e a month f o r f o u r m o n t h s . F o r e a c h t e s t i n g s e s s i o n , s t u d e n t s c o r e s on t h e s e t e s t s we re t o t a l l e d and t h e m o n t h l y mean t o t a l m a t h e m a t i c s f a c t s s c o r e f o r t h e g r o u p t e s t e d was c a l c u l a t e d . A c c o r d i n g t o t h e h y p o t h e s i s t h e s e m o n t h l y mean t o t a l m a t h e m a t i c s f a c t s s c o r e s w i l l i n c r e a s e s i g n i f i c a n t l y b e t w e e n S e s s i o n s 1 and 2, 2 and 3, and 3 and 4. 7 S i m i l a r l y , t h e m o n t h l y m e a n s f o r t h e s u m o f t h e G r a d e 2 a d d i t i o n a n d s u b t r a c t i o n c o m p u t a t i o n s k i l l s t e s t s c o u l d b e o b t a i n e d a n d t h e i n c r e a s e i n m e a n t o t a l s c o r e s b e t w e e n s e s s i o n s w o u l d a l s o b e h y p o t h e s i z e d t o i n c r e a s e s i g n i f i c a n t l y . I t s h o u l d b e n o t e d t h a t i t i s h y p o t h e s i z e d t h a t t h e m o n t h l y m e a n t o t a l m a t h e m a t i c s f a c t s s c o r e s a n d t h e m o n t h l y m e a n t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s s c o r e s w i l l i n c r e a s e s i g n i f i c a n t l y . T h i s i s n o t t o s u g g e s t t h a t t h e r e w i l l n e c e s s a r i l y b e a s i g n i f i c a n t i n c r e a s e i n t h e m o n t h l y m e a n o f e a c h f u n c t i o n t e s t e d ( i . e . a d d i t i o n f a c t s a n d s u b t r a c t i o n f a c t s i n G r a d e 2 ) . R a t h e r , i t i s a n t i c i p a t e d t h a t t h e c h a n g e i n t h e m e a n o f e a c h f u n c t i o n m a y v e r y w e l l n o t b e s i g n i f i c a n t a s t h e r e m a y b e n o i n s t r u c t i o n a n d p r a c t i c e i n t h a t f u n c t i o n t h a t m o n t h w h i c h w o u l d g e n e r a t e s i g n i f i c a n t c h a n g e . O r g a n i z a t i o n o f t h e R e m a i n i n g C h a p t e r s C h a p t e r 2 o f t h i s s t u d y i s a r e v i e w o f r e l a t e d l i t e r a t u r e . C h a p t e r 3 p r o v i d e s a c o m p a r i s o n o f t h e m a t h e m a t i c s c o m p u t a t i o n s k i l l s t e s t e d b y t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s , t h e C a n a d i a n T e s t s o f B a s i c S k i l l s , a n d t h e B . C . Q u i e t . C h a p t e r 4 d e s c r i b e s t h e d e v e l o p m e n t o f t h e DTMS a n d t h e p r o c e d u r e s f o l l o w e d i n i t s n o r m i n g . C h a p t e r 5 p r e s e n t s a n a n a l y s i s a n d d i s c u s s i o n o f t h e d a t a g e n e r a t e d b y t h e n o r m i n g o f t h e D T M S . C h a p t e r 6 c o n t a i n s a s u m m a r y o f t h e f i r s t f i v e c h a p t e r s a n d c o n c l u s i o n s r e a c h e d a s a r e s u l t o f t h e f i n d i n g s d e t a i l e d i n C h a p t e r 5. I t c o n c l u d e s w i t h s o m e p r a c t i c a l s u g g e s t i o n s f o r t h e u s e o f t h e m a t e r i a l s d e v e l o p e d a n d r e c o m m e n d a t i o n s f o r a d d i t i o n a l r e s e a r c h . 8 CHAPTER 2 - REVIEW OF RELATED LITERATURE T y p i c a l l y , educators use commercial norm-referenced achievement t e s t s i n the development and monitoring of i n d i v i d u a l i z e d educational programs (Tindal et a l . , 1985). C r i t i c s contend that such measures do not adequately sample s p e c i f i c s k i l l s (Zigmond & Silverman, 1984) and that they lack content v a l i d i t y with respect to a student's curriculum (Jenkins & Pany, 1978). Scores generated from achievement t e s t s are frequently inadequate f o r monitoring p u p i l progress through the curriculum (Tallmadge, 1977; Deno, Jenkins & Mirk i n , 1979). In a d d i t i o n , Tallmadge points out that care must a l s o be taken to ensure that the treatment group of c h i l d r e n i n any norm-referenced evaluation, i s not d i s s i m i l a r from the c h i l d r e n i n the norming group. Curriculum-based assessment (CBA) s t r a t e g i e s have been developed i n response to the shortcomings i d e n t i f i e d i n commercial norm-referenced achievement t e s t s (Caltagirone & Glover, 1985; Germann & T i n d a l , 1985). Norm-Referenced Achievement Testa Educators' r e l i a n c e on the r e s u l t s of commercial norm-referenced achievement t e s t s to monitor student progress through the curriculum has not gone without c r i t i c i s m . The more pertinent shortcomings which have been discussed i n the f o l l o w i n g sections include: a) the questionable congruence between the group being tested and the norming group (Tallmadge, 1977); b) the t y p i c a l l y poor congruence between achievement t e s t items and c u r r i c u l a r 9 r e q u i r e m e n t s ( J e n k i n s & P a n y , 1978: A i r a s i a n & M a d a u s , 1983; H a e r t e l & C a l f e e , 1983; L i n n , 1983; S c h m i d t , 1983); c ) t h e r e d u c e d d i a g n o s t i c u t i l i t y o f t h e s e t e s t s r e s u l t i n g f r o m t h e i r l i m i t e d i t e m c o n t e n t ( S a l v i a & Y s s e l d y k e , 1981; S n a r t , D e n n i s & B r a i l s . f o r d , 1983); a n d d ) t h e p e r c e i v e d i n a d e q u a c y o f s c o r e s d e r i v e d f r o m n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s ( T a l l m a d g e , 1977; D e n o , J e n k i n s & M i r k i n , 1979). Group - Comparability. T a l l m a d g e (1977) n o t e d t h a t , " T h e r e a r e m a n y w a y s a p a r t i c u l a r g r o u p o f c h i l d r e n c o u l d d i f f e r f r o m t h e n o r m i n g s a m p l e w h o o b t a i n e q u i v a l e n t t e s t s c o r e s " (p.353). H e s u g g e s t e d t h e s e d i f f e r e n c e s m a y a f f e c t f u t u r e p e r f o r m a n c e o n t h e t e s t a n d t h a t we m a y b e u n a b l e t o p r e d i c t e i t h e r t h e d i r e c t i o n o r m a g n i t u d e o f t h e r e s u l t i n g d i s c r e p a n c i e s . T o i l l u s t r a t e t h i s p o i n t , T a l l m a d g e u s e d t h e e x a m p l e o f t w o f o u r t h - g r a d e c l a s s e s w i t h e q u a l t e s t s c o r e s . O n e c l a s s c o n t a i n e d s e v e r a l c h i l d r e n w h o h a d b e e n h e l d b a c k f o r o n e o r m o r e y e a r s w h i l e t h e o t h e r c l a s s h a d n o s u c h c h i l d r e n . H e n o t e d t h a t we w o u l d n o t e x p e c t t h e s e t w o c l a s s e s t o o b t a i n e q u a l s c o r e s o n t h e t e s t a t s o m e f u t u r e d a t e b u t we w o u l d e x p e c t t h e c l a s s w i t h t h e o l d e r f o u r t h - g r a d e s t u d e n t s t o l e a r n a t a s l o w e r r a t e t h a n t h e c l a s s w i t h t h e y o u n g e r f o u r t h - g r a d e s t u d e n t s . T a l l m a d g e c o n t e n d e d t h a t t h e r e a r e n o s t a t i s t i c a l t e c h n i q u e s w h i c h c a n b e u s e d t o c o m p e n s a t e f o r s u c h i n c o n g r u i t i e s a n d t h a t " s u c h n o n c o m p a r a b i l i t y c a n l e a d t o e i t h e r u n d e r e s t i m a t i o n o r o v e r - e s t i m a t i o n o f t r e a t m e n t e f f e c t s " (p.352) w h e n n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s a r e u s e d t o m e a s u r e p u p i l p r o g r e s s t h r o u g h t h e c u r r i c u l u m . 10 Achievement Test/Curriculum Congruence. I n d i s c u s s i n g t h e d e v e l o p m e n t and u s e o f a c h i e v e m e n t t e s t s , G r o n l u n d (1982) p r o p o s e d s i x " p r i n c i p l e s o f a c h i e v e m e n t t e s t i n g , " t h e f i r s t o f w h i c h w a s , " A c h i e v e m e n t t e s t s s h o u l d measu re c l e a r l y d e f i n e d l e a r n i n g o u t c o m e s t h a t a r e i n harmony w i t h t h e i n s t r u c t i o n a l o b j e c t i v e s " ( p . 8 ) . A i r a s i a n & Madaus (1983) n o t e d t h a t s k e p t i c s m i g h t r e a s o n a b l y a r g u e , " I t i s a t r u i s m t h a t t h e c o n t e n t o f s t a n d a r d i z e d t e s t s . . . mus t be l i n k e d t o i n s t r u c t i o n i f m e a n i n g f u l i n f e r e n c e s a b o u t p e r f o r m a n c e a r e t o be made. F u r t h e r , t h e y m i g h t c o n t e n d t h a t a c e n t r a l t e n e t o f t h e e v a l u a t i o n movement i s t h e n e e d t o g a t h e r i n f o r m a t i o n a b o u t s t u d e n t a c h i e v e m e n t u s i n g t e s t s d i r e c t l y l i n k e d t o e d u c a t i o n a l o b j e c t i v e s and i n s t r u c t i o n s " ( p . 1 0 3 ) . S c h m i d t (1983) e x a m i n e d t h e r e l a t i o n s h i p b e t w e e n t h e c o n t e n t o f a c h i e v e m e n t t e s t s a n d t h e c o n t e n t o f i n s t r u c t i o n u n d e r t h e g e n e r a l r u b r i c o f c o n t e n t v a l i d i t y . He i d e n t i f i e d " t h r e e d o m a i n s o f c o n t e n t t h a t c o u l d s e r v e a s d e f i n i n g d o m a i n s f o r a t e s t . . . 1) a n a p r i o r i s p e c i f i e d d o m a i n , 2) a c u r r i c u l a r d o m a i n , and 3) an i n s t r u c t i o n a l d o m a i n " ( p . 1 6 7 ) . T h e s e d o m a i n s we re d e f i n e d a s f o l l o w s : A P r i o r i Domains: T h i s c a t e g o r y r e f e r r e d t o a doma in d e f i n e d i n d e p e n d e n t l y o f t h e c u r r i c u l a r o r i n s t r u c t i o n a l d o m a i n s t o r e p r e s e n t what s h o u l d be l e a r n e d by a l l t h e s t u d e n t s . A s t a n d a r d i z e d a c h i e v e m e n t t e s t s u c h a s t h e C a n a d i a n T e s t s o f B a s i c S k i l l s (CTBS) c o u l d be s a i d t o t e s t an v a p r i o r i d o m a i n ' i n s o f a r a s i t h a s b e e n c o n s t r u c t e d " t o c o r r e s p o n d t o t h e w i d e l y a c c e p t e d 1 1 g o a l s o f i n s t r u c t i o n i n s c h o o l s a c r o s s t h e n a t i o n " ( C T B S , M a n u a l , p . 5 1 ) n o t t o s c h o o l o r e v e n p r o v i n c i a l s p e c i f i c c u r r i c u l u m o r i n s t r u c t i o n a l p r o c e d u r e s . S u p p o r t e r s o f t e s t s d e v e l o p e d o n t h e b a s i s o f a n a p r i o r i domain a r g u e d t h a t , " I t p e r m i t s t h e a s s e s s m e n t o f k n o w l e d g e a n d c o m p e t e n c i e s t h a t t h e p u b l i c b e l i e v e s s h o u l d b e k n o w n b y a l l s t u d e n t s i n a s c h o o l s y s t e m , s t a t e o r n a t i o n , w h e t h e r o r n o t t h e s t u d e n t s h a v e b e e n a d e q u a t e l y e x p o s e d t o i n s t r u c t i o n d e a l i n g w i t h t h i s d o m a i n " ( S c h m i d t , p . 1 6 7 ) . D e t r a c t o r s o f a p r i o r i d o m a i n t e s t i n g c o n t e n d e d t h a t , " T h e c o n t e n t t o p i c s a n d s k i l l a r e a s i n c l u d e d i n t h e t e s t s a r e s e l e c t e d t o a s s e s s a " l o w e s t c o m m o n d e n o m i n a t o r ' o f a v a i l a b l e c u r r i c u l a r m a t e r i a l s a n d c o n t e n t i n o r d e r t o b e " f a i r * a c r o s s a l l s c h o o l s a n d p u p i l s " ( A i r a s i a n & M a d a u s , 1 9 8 3 , p . 1 0 5 ) . T h e y a r g u e d t h a t a p r i o r i d o m a i n t e s t i n g i n a d e q u a t e l y r e f l e c t s t h e c u r r i c u l u m a n d i n s t r u c t i o n a l p r a c t i c e s o f p a r t i c u l a r s c h o o l s a n d p r o g r a m s a n d t h e r e f o r e c a n n o t a d e q u a t e l y a s s e s s t h e l e a r n i n g w h i c h t a k e s p l a c e . F u r t h e r , c r i t i c s o f a p r i o r i d o m a i n t e s t i n g e x p r e s s e d t h e c o n c e r n t h a t s u c h t e s t s a r e i n t e n d e d t o d i f f e r e n t i a t e a m o n g c h i l d r e n a n d t h a t t e s t i t e m s a r e s e l e c t e d t o f o s t e r t h i s d i f f e r e n t i a t i o n . I n o t h e r w o r d s , t e s t i t e m s w e r e s e l e c t e d b e c a u s e t h e y a r e g o o d d i s c r i m i n a t o r s , n o t n e c e s s a r i l y b e c a u s e t h e y r e p r e s e n t e d a c r i t i c a l c o n t e n t a r e a ( A i r a s i a n & M a d a u s , p . 1 0 5 ) . Curricular Domains: A c u r r i c u l a r d o m a i n w a s d e f i n e d b y m a t e r i a l s s u c h a s e d u c a t i o n a l p r o g r a m s a n d t e x t b o o k s . S c h m i d t n o t e d t h a t , " T h e o b j e c t i v e s o f t h e c u r r i c u l u m , i f s t a t e d e x p l i c i t l y , c a n b e u s e d t o o p e r a t i o n a l l y d e f i n e t h e c o n t e n t d o m a i n . W h e n o b j e c t i v e s a r e n o t e x p l i c i t l y s t a t e d t h e y a r e i m p l i e d b y t h e c o n t e n t o f t h e m a t e r i a l s " ( p . 1 6 8 ) . C u r r i c u l u r o - b a s e d a s s e s s m e n t ( C B A ) s t r a t e g i e s h a v e b e e n d e v e l o p e d t o a s s e s s s t u d e n t p e r f o r m a n c e i n c u r r i c u l a r o b j e c t i v e s . Instructional Domains: " T h e i n s t r u c t i o n a l d o m a i n r e f l e c t s t h e c o n t e n t a c t u a l l y t a u g h t i n t h e c l a s s r o o m " ( S c h m i d t , p . 1 6 8 ) . I t i s t h e t e a c h e r u s i n g , a l t e r i n g , m o d i f y i n g o r i g n o r i n g c u r r i c u l a r m a t e r i a l s . T h e i n s t r u c t i o n a l d o m a i n i s , t h e r e f o r e , n o t n e c e s s a r i l y s y n o n y m o u s w i t h t h e c u r r i c u l a r d o m a i n . T e a c h e r m a d e t e s t s m a y b e u s e d t o a s s e s s s t u d e n t p e r f o r m a n c e i n t h e i n s t r u c t i o n a l d o m a i n . A s h a s b e e n i n d i c a t e d , t e s t s m a y b e d e v e l o p e d t o a s s e s s e a c h o f t h e s e d o m a i n s . O f t e n t e s t s g e n e r a t e d t o a s s e s s o n e d o m a i n a r e u s e d t o e s t i m a t e p e r f o r m a n c e a s s o c i a t e d w i t h a n o t h e r . F o r e x a m p l e , t h e C T B S ( a n a p r i o r i d o m a i n t e s t ) m a y b e u s e d t o a s s e s s s t u d e n t p r o g r e s s i n e i t h e r t h e c u r r i c u l a r o r i n s t r u c t i o n a l d o m a i n s ( C T B S , M a n u a l , p . 4 2 ) . H o w e v e r , a s S c h m i d t n o t e d , " u n l e s s t h e t h r e e d o m a i n s a r e c o n g r u e n t , c o n t e n t b i a s e s a r i s e a n d s h o u l d b e a d d r e s s e d e x p l i c i t l y " . H e m a i n t a i n s t h a t c u r r e n t l y "we l a c k a d e q u a t e m o d e l s a n d m e t h o d s f o r d o i n g s o " ( p . 1 6 8 ) . I n t e r e s t i n g l y , t h i s v i e w a p p e a r s t o b e s h a r e d b y t h e d e v e l o p e r s o f t h e C T B S w h o n o t e d t h a t " s t a n d a r d i z e d t e s t r e s u l t s m a y b e u s e d a s a p a r t i a l b a s i s f o r e v a l u a t i n g t h e e f f e c t i v e n e s s o f i n s t r u c t i o n " b u t c a u t i o n e d t h a t " t h e w o r d ^ p a r t i a l ' i n t h i s s t a t e m e n t d e s e r v e s h e a v y e m p h a s i s " ( p . 4 2 ) . The Limited Item Content of Achievement Tests. The l i m i t e d number o f b e h a v i o r a l s a m p l e s o f s k i l l s t a k e n by many a c h i e v e m e n t t e s t s r e d u c e s t h e i r d i a g n o s t i c u t i l i t y . To assume t h a t one o r two b e h a v i o r a l s a m p l e s o f a s p e c i f i c s k i l l a r e s u f f i c i e n t t o a d e q u a t e l y d i a g n o s e a s t u d e n t ' s p e r f o r m a n c e on t h a t s k i l l i s p r e s u m p t u o u s t o s a y t h e l e a s t ( S a l i v a & Y s s e l d y k e , 1981; S n a r t e t a l . , 1 9 8 3 ) . I n a d d i t i o n , o u r a b i l i t y t o a d e q u a t e l y a s s e s s p u p i l p r o g r e s s t h r o u g h t h e c u r r i c u l u m u s i n g s t a n d a r d i z e d a c h i e v e m e n t t e s t s i s c o m p r o m i s e d s i m p l y b e c a u s e t h e y do n o t a d e q u a t e l y t e s t t h e r a n g e o f s k i l l s r e q u i r e d by t h e c u r r i c u l u m ( M i r k i n , F u c h s & D e n o , 1982; C a l t a g i r o n e and G l o v e r , 1 9 8 5 ) . The l i m i t e d d i a g n o s t i c e f f e c t i v e n e s s o f t h e C o m p u t a t i o n s T e s t o f t h e C a n a d i a n T e s t s o f B a s i c S k i l l s i s a d d r e s s e d i n C h a p t e r 3. The Inadequacy of Derived Scores. N o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s t y p i c a l l y c o n v e r t raw t e s t s c o r e s i n t o v a r i o u s t y p e s o f d e r i v e d s c o r e s s u c h a s p e r c e n t i l e s and g r a d e e q u i v a l e n t s c o r e s w h i c h c a n p r o v i d e u s e f u l f r a m e s o f r e f e r e n c e f o r t h e i n t e r p r e t a t i o n o f t e s t r e s u l t s . U n f o r t u n a t e l y , a s T a l l m a d g e (1977) n o t e d , "The c o n v e n t i o n s a d a p t e d by t e s t p u b l i s h e r s i n m a n i p u l a t i n g and r e p o r t i n g t h e i r n o r m a t i v e d a t a seem l i k e l y t o e n h a n c e t h e p r o b a b i l i t y o f m a k i n g v a r i o u s t y p e s o f e r r o r s " ( p . 3 5 6 ) . T a l l m a d g e g r a p h i c a l l y d e m o n s t r a t e d ( F i g u r e 1) t h a t t h e r e p o r t i n g o f d e r i v e d s c o r e s ( i n t h i s c a s e , p e r c e n t i l e s ) c a n be s e r i o u s l y f l a w e d . He n o t e d t h a t p u b l i s h e r s o f t e n p r o v i d e f a l l , m i d - y e a r and s p r i n g norms f o r t h e i r t e s t s . F a l l norms a r e t y p i c a l l y " g o o d " f o r S e p t e m b e r , O c t o b e r and November . M i d - y e a r 14 n o r m s a r e g o o d f o r D e c e m b e r , J a n u a r y a n d F e b r u a r y w h i l e s p r i n g n o r m s a r e g o o d f o r M a r c h , A p r i l , M a y a n d p o s s i b l y J u n e . T h e s o l i d l i n e i n F i g u r e 1 s h o w s t h e n u m b e r o f i t e m s t h e m e d i a n o r 5 0 t h p e r c e n t i l e 6 t h - g r a d e c h i l d w o u l d a n s w e r c o r r e c t l y a t v a r i o u s t i m e s d u r i n g t h e s c h o o l y e a r . T a l l m a d g e s u g g e s t s t h i s i m p l i e s t h a t a l l t h e c o g n i t i v e g r o w t h t h a t t a k e s p l a c e i n G r a d e 6 t a k e s p l a c e o v e r n i g h t o n N o v e m b e r 30 a n d F e b r u a r y 2 8 . T h e b r o k e n l i n e w h i c h c r o s s e s t h e l i n e r e p r e s e n t i n g t h e p u b l i s h e r ' s " 5 0 t h p e r c e n t i l e c h i l d " a t m i d - O c t o b e r , m i d - J a n u a r y a n d m i d - A p r i l i s a m o r e b e l i e v a b l e , a l t h o u g h a s n o t e d b e l o w a s t i l l q u e s t i o n a b l e , e x p e c t a t i o n o f c o g n i t i v e g r o w t h f o r t h i s c h i l d . H o w e v e r , t h i s ' m o r e b e l i e v a b l e ' m e d i a n c h i l d i s s e e n t o b e b e l o w a v e r a g e a t t h e b e g i n n i n g o f e a c h n o r m i n g p e r i o d d e f i n e d b y t h e t e s t p u b l i s h e r a n d a b o v e a v e r a g e a t t h e e n d o f t h e p e r i o d ( p . 3 5 8 ) . A s i m i l a r p r o b l e m c a n b e s h o w n t o e x i s t w i t h g r a d e e q u i v a l e n t s c o r e s . A c h i l d ' s a c t u a l p r o g r e s s c o u l d b e i n f l a t e d u s i n g g r a d e e q u i v a l e n t s c o r e s i n a p r e - t e s t , p o s t - t e s t s i t u a t i o n i f p r e -t e s t i n g o c c u r r e d e a r l y i n a n o r m i n g p e r i o d a n d p o s t - t e s t i n g o c c u r r e d l a t e i n a n o r m i n g p e r i o d . O b v i o u s l y a n i n a p p r o p r i a t e s e l e c t i o n o f p r e a n d p o s t t e s t i n g t i m e s c o u l d s e r i o u s l y d i s t o r t t h e f i n d i n g s o f a n e v a l u a t i o n s t u d y . N o r m a t i v e d a t a f o r m o s t c o m m e r c i a l n o r m - r e f e r e n c e d t e s t s w e r e c o l l e c t e d d u r i n g o n e s h o r t i n t e r v a l d u r i n g t h e s c h o o l y e a r . F o r t i m e s w h e n n o e m p i r i c a l d a t a i s a v a i l a b l e d e r i v e d s c o r e s h a v e b e e n g e n e r a t e d t h r o u g h i n t e r p o l a t i o n o r e x t r a p o l a t i o n . T h e s e p r o c e d u r e s a s s u m e d t h a t t h e n u m b e r o f q u e s t i o n s a n s w e r e d c o r r e c t l y 15 FIGURE 1: COGNITIVE GROWTH BY THE TEST PUBLISHER'S 50TH PERCENTILE CHILD VERSUS A MORE REALISTIC EXPECTATION. 40 -| 35 -30 -25 - Publisher's Median 'Real' Median School Year Months (Tallmadge, 1977, p.358) 16 w e r e a l i n e a r f u n c t i o n o f t i m e o v e r t h e s c h o o l y e a r . T a l l m a d g e m a i n t a i n e d t h a t s u c h a n a s s u m p t i o n " i s t e n u o u s a t b e s t a n d m a y b e s i g n i f i c a n t l y i n e r r o r " ( p . 3 5 7 ) . F u r t h e r m o r e , T a l l m a d g e n o t e d t h a t e x t e n d i n g t h e l i n e a r g r o w t h f u n c t i o n a c r o s s t h e s u m m e r m o n t h s i s m o s t q u e s t i o n a b l e . A s t u d y b y B e g g s a n d H i e r o n y m u s ( 1 9 6 8 ) s u p p o r t e d t h i s c o n t e n t i o n . T h e y f o u n d t h a t c h i l d r e n t e s t e d o n d i f f e r e n t s u b t e s t s o f t h e I o w a T e s t s o f B a s i c S k i l l s ( t h e t e s t o n w h i c h t h e C T B S w a s p a t t e r n e d ) s h o w e d c o n s i s t e n t a n d s u b s t a n t i a l l o s s e s i n l a n g u a g e a n d a r i t h m e t i c o v e r t h e s u m m e r m o n t h s , b u t n o t i n r e a d i n g . F u r t h e r m o r e , t h e y r e p o r t e d s o m e e v i d e n c e o f a c c e l e r a t e d g r o w t h f r o m m i d - J a n u a r y t o m i d - A p r i l i n t h e l a n g u a g e , w o r k - s t u d y a n d a r i t h m e t i c a r e a s . T e s t i n g i n J u n e a n d S e p t e m b e r u s i n g t h e S t a n f o r d A c h i e v e m e n t T e s t s , M o u s l e y ( 1 9 7 3 ) f o u n d n o c h a n g e i n e i t h e r v o c a b u l a r y o r r e a d i n g c o m p r e h e n s i o n g r a d e e q u i v a l e n t s c o r e s f o r c h i l d r e n m o v i n g f r o m G r a d e 3 t o G r a d e 4 . T h e f a c t t h a t c o g n i t i v e g r o w t h i s n o t a l i n e a r f u n c t i o n c a n p r o d u c e s o m e u n u s u a l c o n s e q u e n c e s - p a r t i c u l a r l y w h e r e t e s t s h a v e b e e n n o r m e d t w i c e d u r i n g t h e s c h o o l y e a r . R e c a l l t h a t g r a d e e q u i v a l e n t s c o r e s a r e g e n e r a l l y e q u a t e d t o m e d i a n r a w s c o r e v a l u e s f o r e a c h g r a d e l e v e l o f t h e m o n t h t h e t e s t w a s n o r m e d . T h e i n t e r v a l b e t w e e n m e d i a n s i s d i v i d e d i n t o t e n e q u a l p a r t s a n d t h e i n t e r m e d i a t e g r a d e e q u i v a l e n t s c o r e s a r e e q u a t e d w i t h t h e n e a r e s t i n t e g r a l r a w s c o r e v a l u e . A s T a l l m a d g e ( 1 9 7 7 ) n o t e d , t e s t d e v e l o p e r s g e n e r a l l y p r o v i d e a s i n g l e t a b l e c o n v e r t i n g r a w s c o r e s t o g r a d e e q u i v a l e n t s f o r e a c h l e v e l o f a t e s t . " T h i s r e q u i r e s 17 t h a t t h e m e d i a n c h i l d a c h i e v e a h i g h e r r a w s c o r e a t e a c h s u c c e s s i v e p o i n t i n t i m e " (p.361). F i g u r e 2 i l l u s t r a t e s a h y p o t h e t i c a l s i t u a t i o n w h e r e t h i s d o e s n o t o c c u r . T h e d a t a p o i n t s c o n n e c t e d b y t h e s o l i d l i n e r e p r e s e n t r a w s c o r e s o f t h e m e d i a n c h i l d a t g r a d e l e v e l s 3.1, 3.8, 4.8, 5.1 a n d 5.8. O b v i o u s l y a l o s s o f r a w s c o r e p o i n t s o c c u r r e d o v e r t h e s u m m e r . C o n v e r t i n g t h e m e d i a n c h i l d ' s r a w s c o r e s t o g r a d e e q u i v a l e n t s w o u l d r e s u l t i n a r a w s c o r e o f 146 c o r r e s p o n d i n g t o g r a d e e q u i v a l e n t s o f 3.7 a n d 4.1 w h i l e a s c a l e d s c o r e o f 147 w o u l d c o n v e r t t o t h r e e d i f f e r e n t g r a d e e q u i v a l e n t s . T o a v o i d t h i s c o n f u s i o n , " t e s t p u b l i s h e r s h a v e a d o p t e d t h e c o n v e n t i o n o f c o n s t r u c t i n g a s m o o t h e d l i n e ( s u c h a s t h e d o t t e d l i n e i n F i g u r e 2) t o c o n v e r t r a w o r s c a l e d s c o r e s t o g r a d e e q u i v a l e n t s " e v e n t h o u g h i t " g i v e s t h e m i s t a k e n i m p r e s s i o n t h a t l e a r n i n g i s a m o r e o r d e r l y p h e n o m e n a t h a n i t r e a l l y i s " ( T a l l m a d g e , 1977, p.361). O n e r e s u l t o f t h i s " l i n e s t r a i g h t e n i n g " i s t h a t t h e f i f t i e t h p e r c e n t i l e c h i l d i s o f t e n n o t a t g r a d e l e v e l . A s e x a m p l e s o f t h i s a b e r r a t i o n , T a l l m a d g e r e p o r t e d t h a t t h e m e d i a n t h i r d - g r a d e s t u d e n t o n t h e M e t r o p o l i t a n A c h i e v e m e n t T e s t s (1970 e d . ) i s t w o m o n t h s b e l o w g r a d e l e v e l i n r e a d i n g a t t h e e n d o f t h e s c h o o l y e a r w h i l e t h e f i f t i e t h p e r c e n t i l e c h i l d i s f i v e m o n t h s a b o v e g r a d e l e v e l i n m a t h e m a t i c s c o m p u t a t i o n a t t h e e n d o f t h i r d g r a d e o n F o r m B o f t h e S t a n f o r d A c h i e v e m e n t T e s t s (1973 e d . ) . 18 FIGURE 2: COMPARISON OF THE MEDIAN (50TH PERCENTILE) SCORE AND THE GRADE NORM LINE. (Tallmadge, 1977, p.362) 1 9 C u r r i c u l u m - B a s e d A s s e s s m e n t ( C B A ) B l a n k e n s h i p a n d L i l l y ( 1 9 8 1 ) d e f i n e d c u r r i c u l u m - b a s e d a s s e s s m e n t a s , " t h e p r a c t i c e o f o b t a i n i n g d i r e c t a n d f r e q u e n t m e a s u r e s o f a s t u d e n t ' s p e r f o r m a n c e o n a s e r i e s o f s e q u e n t i a l l y a r r a n g e d o b j e c t i v e s d e r i v e d f r o m t h e c u r r i c u l u m u s e d i n t h e c l a s s r o o m " ( p . 8 1 ) . I n t h e a b o v e d e f i n i t i o n " d i r e c t " m e a n s t h a t " t h e b e h a v i o r t a u g h t i s t h e o n e m e a s u r e d " ( p . 8 4 ) a n d " f r e q u e n t " m e a n s t h a t t h e d i r e c t m e a s u r e s m a y b e c o l l e c t e d d a i l y o r a t l e a s t p e r i o d i c a l l y t h r o u g h o u t t h e s c h o o l y e a r . I n a d d i t i o n , B l a n k e n s h i p a n d L i l l y n o t e t h a t c u r r i c u l u m - b a s e d a s s e s s m e n t i s b a s e d u p o n s e v e r a l a s s u m p t i o n s , f o u r o f w h i c h a r e o f p a r t i c u l a r r e l e v a n c e t o t h e s t u d y u n d e r d i s c u s s i o n : 1 ) T h e b e s t m e a s u r e o f a s t u d e n t ' s m a t h e m a t i c s a b i l i t y i s o b t a i n e d b y d i r e c t l y m e a s u r i n g t h e s t u d e n t ' s p e r f o r m a n c e i n t h e c u r r i c u l u m u s e d f o r i n s t r u c t i o n . 2 ) T h e m o s t a c c u r a t e i n d i c a t i o n o f a s t u d e n t ' s a b i l i t y i s o b t a i n e d b y m e a s u r i n g a s t u d e n t ' s p e r f o r m a n c e o n t h e s a m e s k i l l s o n t w o o r m o r e o c c a s i o n s . 3) B y a n a l y z i n g a s t u d e n t ' s p e r f o r m a n c e o n a C u r r i c u l u m - B a s e d A s s e s s m e n t M e a s u r e ( C B A M ) , a t e a c h e r c a n : ( 1 ) p i n p o i n t d e f i c i t s k i l l s , ( 2 ) i d e n t i f y e r r o r p a t t e r n s , a n d (3) p l a c e s t u d e n t s a t a n a p p r o p r i a t e l e v e l i n t h e c u r r i c u l u m . 4 ) F o r t h e m a j o r i t y o f s t u d e n t s , p r o g r e s s c a n b e e v a l u a t e d b y p e r i o d i c a l l y r e a d m i n i s t e r i n g a C u r r i c u l u m - B a s e d A s s e s s m e n t M e a s u r e t h r o u g h o u t t h e s c h o o l y e a r . D i r e c t a n d f r e q u e n t m e a s u r e m e n t i s n e e d e d , h o w e v e r , t o g u i d e t h e i n s t r u c t i o n o f s t u d e n t s w h o e v i d e n c e l e a r n i n g p r o b l e m s ( p . 8 5 ) . C a l t a g i r o n e a n d G l o v e r ( 1 9 8 5 ) a n d F u c h s a n d D e n o ( 1 9 8 2 ) c o n c u r r e d w i t h t h e f i r s t o f t h e s e a s s u m p t i o n s b y B l a n k e n s h i p a n d L i l l y . T h e y e m p h a s i z e d t h a t s t a n d a r d i z e d a c h i e v e m e n t t e s t s a r e g e n e r a l l y t o o l a c k i n g i n c o n t e n t v a l i d i t y t o b e o f m u c h u s e i n t h e o n g o i n g m o n i t o r i n g o f p u p i l p r o g r e s s t h r o u g h t h e c u r r i c u l u m . T h e 20 v a l u e o f t h e o t h e r a s s u m p t i o n s i s e v i d e n c e d i n a s t u d y b y C a l t a g i r o n e a n d G l o v e r (1985) c o n d u c t e d i n t h e M o u n t P l e a s a n t S c h o o l D i s t r i c t i n S o u t h e r n C a l i f o r n i a . CBA: The Mount Pleasant Model I n t h i s s t u d y a s i n g l e c u r r i c u l u m - b a s e d i n s t r u m e n t ( p r o b e ) w a s d e v e l o p e d t o a s s e s s t h e a c q u i s i t i o n o f t h i r d - g r a d e m a t h e m a t i c s c o m p u t a t i o n s k i l l s . T h e p r o b e w a s a d m i n i s t e r e d m o n t h l y f r o m O c t o b e r , 1983 t o J u n e , 1984 t o 214 t h i r d - g r a d e s t u d e n t s i n s e v e n c l a s s r o o m s . E a c h s t u d e n t ' s p r o b e w a s c o r r e c t e d a l o n g t w o s k i l l m e a s u r e s : ( a ) p e r c e n t c o r r e c t ( s k i l l a c c u r a c y ) , a n d c o r r e c t p e r m i n u t e ( s k i l l f l u e n c y ) . M o n t h l y m e a n s a n d s t a n d a r d d e v i a t i o n s f o r e a c h o f t h e m e a s u r e s ( p e r c e n t c o r r e c t a n d c o r r e c t p e r m i n u t e ) w e r e c a l c u l a t e d . T h e r e s e a r c h e r s r e p o r t t h a t d i s t r i c t - w i d e c h a n g e s i n c o m p u t a t i o n s k i l l s o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e c o u l d b e m e a s u r e d . T h e p e r c e n t c o r r e c t s c o r e o f 25 r a n d o m l y s e l e c t e d s t u d e n t s w a s c o m p a r e d w i t h t h e i r r e s p e c t i v e s c o r e s o n t h e c a l c u l a t i o n s u b t e s t o f t h e W o o d c o c k - J o h n s o n P s y c h o e d u c a t i o n a l B a t t e r y . C a l t a g i r o n e a n d G l o v e r r e p o r t e d t h a t " t h e r e s u l t i n g P e a r s o n P r o d u c t M o m e n t C o r r e l a t i o n ( r =.68, p < .001, d f = 23) i n d i c a t e s a s i g n i f i c a n t r e l a t i o n s h i p b e t w e e n t h e t w o m e a s u r e s " ( p . 9 ) . T h e r e s e a r c h e r s n o t e d t h a t , " T h e r e s u l t i n g d i s t r i c t - w i d e l e a r n i n g c u r v e s d e f i n i n g a v e r a g e a n n u a l p r o g r e s s t h r o u g h t h e c o m p u t a t i o n c u r r i c u l u m p r o v i d e d n o r m a t i v e s t a n d a r d s a g a i n s t w h i c h 21 f r e q u e n t c o m p a r i s o n s o f a s t u d e n t ' s p r o g r e s s c o u l d be made . I n f o r m a t i o n p r o v i d e d t h r o u g h t h e s e c o m p a r i s o n s a p p e a r s t o f a c i l i t a t e t i m e l y and a c c u r a t e d e c i s i o n s r e g a r d i n g i n s t r u c t i o n a l and p r o g r a m e f f e c t i v e n e s s " ( p . 2 ) . CBA: The Pine County Model A v a r i a t i o n o f c u r r i c u l u m - b a s e d a s s e s s m e n t h a s been u s e d i n t h e P i n e C o u n t y S p e c i a l E d u c a t i o n C o - o p e r a t i v e i n S a n d s t o n e , M i n n e s o t a s i n c e 1 9 8 0 . H e r e t h e p r i n c i p l e s o f c u r r i c u l u m - b a s e d a s s e s s m e n t have been e x t e n d e d i n t o a l l a r e a s o f s p e c i a l e d u c a t i o n , b o t h a c a d e m i c and s o c i a l . A c a d e m i c m e a s u r e s we re d e v e l o p e d f o r t h e a r e a s o f r e a d i n g , s p e l l i n g , w r i t t e n e x p r e s s i o n and m a t h e m a t i c s . As a n e x a m p l e , i n m a t h e m a t i c s : "2 m i n u t e s a m p l e s o f c o m p u t a t i o n p r o b l e m s a p p e a r i n g i n t h e b a s a l t e x t a r e o b t a i n e d f o r e a c h f u n c t i o n ( a d d i t i o n , s u b t r a c t i o n , m u l t i p l i c a t i o n and d i v i s i o n ) and t h e number o f d i g i t s compu ted c o r r e c t l y and i n c o r r e c t l y a r e c a l c u l a t e d " (Germann & T i n d a l , 1 9 8 5 , p . 2 4 7 ) . S t u d e n t s were t e s t e d u n t i l " a s t a b l e b a s e l i n e l e v e l o f p e r f o r m a n c e i s e s t a b l i s h e d , c o r r e s p o n d i n g t o a p p r o x i m a t e l y 5% t o 10% v a r i a t i o n a c r o s s t h e m e a s u r e s . . . The d a t a a r e s u m m a r i z e d by c a l c u l a t i n g t h e m e d i a n " ( p . 2 4 7 ) . T h i s s c o r e was compa red t o " t h e n o r m a t i v e p e r f o r m a n c e l e v e l s o f p e e r s - t h e m e d i a n p e r f o r m a n c e o f d i s t r i c t s t u d e n t s i n t h e same g r a d e a s t h e t a r g e t s t u d e n t . T h e s e d i s t r i c t norms r e p r e s e n t t h e d e s i r e d r a t e s f o r s t u d e n t s , by g r a d e l e v e l , f o r e a c h a c a d e m i c a r e a " ( p . 2 4 7 ) . D i s t r i c t norms were e s t a b l i s h e d by t e s t i n g a g r o u p o f r a n d o m l y s a m p l e d s t u d e n t s d u r i n g t h e f i r s t w e e k s o f s c h o o l ( F a l l ) , t h e f i r s t w e e k s o f t h e s e c o n d s e m e s t e r ( W i n t e r ) , a n d t h e l a s t w e e k s o f s c h o o l ( S p r i n g ) ( p . 2 4 8 ) . A d i s c r e p a n c y r a t i o w a s c a l c u l a t e d t o p e r m i t t h e c o m p a r i s o n o f a s s e s s e d s t u d e n t s t o t h e i r r e g u l a r c l a s s p e e r s . T h e d i s c r e p a n c y r a t i o w a s o b t a i n e d b y d i v i d i n g t h e l o w e r l e v e l o f p e r f o r m a n c e i n t o t h e h i g h e r l e v e l p e r f o r m a n c e . W h e n t h e a s s e s s e d s t u d e n t ' s s c o r e i s l o w e r t h a n t h e n o r m s f o r h i s g r a d e a p p r o p r i a t e p e e r s t h e d i s c r e p a n c y r a t i o i s g i v e n a n e g a t i v e v a l u e . I f t h e s t u d e n t ' s s c o r e i s h i g h e r , i t r e c e i v e s a p o s i t i v e s i g n . A s t u d e n t ' s m e d i a n p e r f o r m a n c e m u s t b e m o r e t h a n t w o t i m e s d i s c r e p a n t , a n d n e g a t i v e f o r h i m o r h e r t o b e r e c o m m e n d e d f o r s p e c i a l a s s i s t a n c e . I n d i v i d u a l i z e d E d u c a t i o n a l P r o g r a m s ( I E P ) a r e d e v e l o p e d f o r e a c h c h i l d a c c e p t e d f o r s p e c i a l c l a s s p l a c e m e n t . P r o g r e s s i s m o n i t o r e d u s i n g c u r r i c u l u m - b a s e d a s s e s s m e n t p r o c e d u r e s . T h e u s e o f t h i s m e t h o d o l o g y e n a b l e d P i n e C o u n t y s p e c i a l e d u c a t i o n t e a c h e r s t o d e m o n s t r a t e t h a t i n r e a d i n g , s p e l l i n g a n d w r i t i n g , " s t u d e n t s s e r v e d i n s p e c i a l e d u c a t i o n f o r o n e f u l l a c a d e m i c y e a r p e r f o r m e d , o n t h e a v e r a g e a t a h i g h e r l e v e l i n t h e S p r i n g t h a n i n t h e F a l l . . . a n d w i t h t h e e x c e p t i o n o f t h e w r i t t e n e x p r e s s i o n m e a s u r e s a l l d i s c r e p a n c i e s i m p r o v e d f r o m o n e t e s t i n g p e r i o d t o t h e n e x t " ( p . 2 5 5 ) . The Two Models: A Summary I n b o t h s t u d i e s r e s e a r c h e r s w e r e a b l e t o d e m o n s t r a t e t h a t c u r r i c u l u m - b a s e d a s s e s s m e n t w a s a b l e t o d e t e c t c h a n g e s i n s t u d e n t p e r f o r m a n c e o v e r t i m e . H o w e v e r , i t s h o u l d b e n o t e d t h a t 2 3 t e s t i n g t h e f o u r b a s i c f u n c t i o n s s e p a r a t e l y a s w a s d o n e i n t h e G e r m a n n a n d T i n d a l (1985) s t u d y , w o u l d s e e m t o m a k e i m p o r t a n t d i a g n o s t i c i n f o r m a t i o n m o r e r e a d i l y a v a i l a b l e t o t e a c h e r s t h a n i t i s f r o m t h e a l l i n c l u s i v e C a l t a g i r o n e a n d G l o v e r (1985) p r o b e . CBA - Goal and CBA - Objective F u c h s a n d F u c h s (1986) i d e n t i f i e d t w o t y p e s o f c u r r i c u l u m -b a s e d a s s e s s m e n t ( C B A ) , C B A - g o a l a n d C B A - o b j e c t i v e . C B A - g o a l f o c u s e s o n t h e a t t a i n m e n t o f l o n g - t e r m g o a l s . A s i n t h e C a l t a g i r o n e a n d G l o v e r , a n d t h e G e r m a n n a n d T i n d a l s t u d i e s , a p o o l o f i t e m s w a s c r e a t e d w h i c h r e f l e c t s t h e o b j e c t i v e s o f t h e p e r t i n e n t c u r r i c u l u m . F r o m t h i s p o o l o f i t e m s , a m o n i t o r i n g p r o b e , o r s e r i e s o f p r o b e s o f a c o n s t a n t l e v e l o f d i f f i c u l t y w a s c r e a t e d . T h e s e p r o b e s w e r e i n t e n d e d f o r r e p e a t e d a d m i n i s t r a t i o n o v e r a l o n g p e r i o d o f t i m e . I n t h e C B A - o b j e c t i v e a p p r o a c h , a s e r i e s o f s e q u e n t i a l o b j e c t i v e s w i t h i n t h e c u r r i c u l u m i s i d e n t i f i e d a n d p o o l s o f t e s t i t e m s f o r e a c h o b j e c t i v e a r e c r e a t e d . T h e l e v e l o f d i f f i c u l t y o f t h e r e s u l t i n g p r o b e s i n c r e a s e s a s s t u d e n t s m a s t e r t h e s e q u e n t i a l o b j e c t i v e s . F o r e x a m p l e i n a d d i t i o n c o m p u t a t i o n s , t h e s e q u e n t i a l o b j e c t i v e s c o u l d b e : N o r e g r o u p i n g - 2 d i g i t + 1 d i g i t N o r e g r o u p i n g - 2 d i g i t + 2 d i g i t R e g r o u p i n g l ' s - 2 d i g i t + 1 d i g i t R e g r o u p i n g l ' s - 2 d i g i t + 2 d i g i t B o t h t y p e s o f C B A a r e o n g o i n g a n d e n j o y s t r o n g c u r r i c u l a r v a l i d i t y . H o w e v e r , C B A - o b j e c t i v e a p p e a r s t o h a v e s t r o n g e r 24 i n s t r u c t i o n a l v a l i d i t y o r c o r r e s p o n d e n c e b e t w e e n t e s t s a n d i n s t r u c t i o n t h a n C B A - g o a l w h e r e a s C B A - g o a l p o s s e s s e s b e t t e r c o n t e n t v a l i d i t y , t h a t i s , i t b e t t e r r e f l e c t s t h e d e s i r e d c u r r i c u l a r o b j e c t i v e t h a n d o e s C B A - o b j e c t i v e ( Y a l o w & P o p h a m , 1983). I n a d d i t i o n , t h e c o n c u r r e n t v a l i d i t y o r c o r r e l a t i o n o f C B A - g o a l w i t h o t h e r m e a s u r e s o f a c h i e v e m e n t a p p e a r s s t r o n g e r t h a n t h a t o f C B A - o b j e c t i v e ( F u c h s c i t e d i n F u c h s & F u c h s , 1986). F u c h s & F u c h s (1986) e x a m i n e d 18 s t u d i e s t o d e t e r m i n e h o w m e a s u r i n g s t u d e n t p r o g r e s s t o w a r d l o n g - t e r m g o a l s ( C B A - g o a l ) v s . s h o r t - t e r m o b j e c t i v e s ( C B A - o b j e c t i v e ) a f f e c t e d a c h i e v e m e n t o u t c o m e s . T h e y n o t e d t h a t a l t h o u g h C B A - o b j e c t i v e m a y b e t h e f a v o u r e d m e t h o d o f m e a s u r e m e n t b e c a u s e i t i s e a s i e r t o u n d e r s t a n d a n d b e c a u s e i t s e r v e s a s a g u i d e t o i n s t r u c t i o n s u g g e s t i n g w h e n t h e c u r r e n t o b j e c t i v e h a s b e e n m a s t e r e d a n d s t u d e n t s a r e r e a d y f o r i n s t r u c t i o n i n t h e s u c c e e d i n g s k i l l , s h o r t - t e r m g o a l m e a s u r e m e n t m a y b e m i s l e a d i n g . F u c h s a n d F u c h s c o n t e n d e d t h a t s t u d e n t s m a y roaster a s e r i e s o f s e q u e n t i a l o b j e c t i v e s b u t t h e i r p e r f o r m a n c e o n a m o r e g l o b a l t e s t o f a c h i e v e m e n t ( C B A - g o a l ) , w h i c h i s a b e t t e r m e a s u r e o f d e s i r e d o u t c o m e p e r f o r m a n c e , m a y b e l i m i t e d . C o n s e q u e n t l y , F u c h s & F u c h s r e c o m m e n d c a u t i o n w h e n i n t e r p r e t i n g m a s t e r y s c o r e s o n s h o r t - t e r m o b j e c t i v e s a n d s u g g e s t t h a t C B A - g o a l a s s e s s m e n t m a y b e a n e c e s s a r y s u p p l e m e n t a r y s t r a t e g y f o r v a l i d l y a s s e s s i n g s t u d e n t p r o g r e s s . 25 Summary A l t h o u g h w i d e s p r e a d u s e i s m a d e b y e d u c a t o r s o f c o m m e r c i a l n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s i n t h e d e v e l o p m e n t a n d m o n i t o r i n g o f e d u c a t i o n a l p r o g r a m s , s u c h m e a s u r e s e x h i b i t m a n y s h o r t c o m i n g s . T h e m o s t s i g n i f i c a n t o f t h e s e s h o r t c o m i n g s w e r e d i s c u s s e d i n t h i s c h a p t e r . T h e y i n c l u d e : a ) t h e q u e s t i o n a b l e c o n g r u e n c e b e t w e e n t h e g r o u p b e i n g t e s t e d a n d t h e n o r m i n g g r o u p ( T a l l m a d g e , 1977); b ) t h e t y p i c a l l y p o o r c o n g r u e n c e b e t w e e n a c h i e v e m e n t t e s t i t e m s a n d c u r r i c u l a r r e q u i r e m e n t s ( J e n k i n s & P a n y , 1978: A i r a s i a n & M a d a u s , 1983; H a e r t e l & C a l f e e , 1983; L i n n , 1983; S c h m i d t , 1983); c ) t h e r e d u c e d d i a g n o s t i c u t i l i t y o f t h e s e t e s t s r e s u l t i n g f r o m t h e i r l i m i t e d i t e m c o n t e n t ( S a l v i a & Y s s e l d y k e , 1981; S n a r t , D e n n i s & B r a i l s f o r d , 1983); a n d d ) t h e p e r c e i v e d i n a d e q u a c y o f s c o r e s d e r i v e d f r o m n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s ( T a l l m a d g e , 1977; D e n o , J e n k i n s & M i r k i n , 1979). C u r r i c u l u m - b a s e d a s s e s s m e n t ( C B A ) s t r a t e g i e s h a v e b e e n d e v e l o p e d i n r e s p o n s e t o t h e s e s h o r t c o m i n g s . T h e s t u d i e s b y C a l t a g i r o n e a n d G l o v e r (1985) a n d G e r m a n n a n d T i n d a l (1985) d e m o n s t r a t e d t h e s e n s i t i v i t y o f c u r r i c u l u m - b a s e d m e a s u r e s a n d t h e e a s e w i t h w h i c h s u c h m e a s u r e s c o u l d p r o v i d e v a l u a b l e d a t a a s t o a s t u d e n t ' s p r o g r e s s t h r o u g h t h e c u r r i c u l u m . F u c h s & F u c h s (1986) i d e n t i f i e d t w o t y p e s o f c u r r i c u l u m - b a s e d a s s e s s m e n t : C B A - g o a l a n d C B A - o b j e c t i v e . C B A - o b j e c t i v e i s i d e n t i f i e d a s a n e c e s s a r y s u p p l e m e n t a r y s t r a t e g y w h e n i n t e r p r e t i n g m a s t e r y s c o r e s o n s h o r t - t e r m o b j e c t i v e s . 26 CHAPTER 3 : CURRICULUM-BASED ASSESSMENT VS. STANDARDIZED ACHIEVEMENT TESTS The pre-post administration of standardized achievement t e s t s to measure student progress through the curriculum and the achievement instruments themselves have come under increasing c r i t i c i s m (Eaton and L o v i t t , 1972; S a l v i a and Ysseldike, 1981; A i r a s i a n and Madaus, 1983; Linn, 1 9 8 3 ) . Most pertinent i s the poor congruence between curriculum content and t e s t items (Jenkins & Pany, 1978; A i r a s i a n & Madaus, 1983; Haertel & Calfee, 1983, Linne, 1983; Schmidt, 1983) and the f a i l u r e of achievement measures to sample domains of i n t e r e s t comprehensively (Zigmond & Silverman, 1984). The Delta Tests of Mathematics S k i l l s were designed to overcome these shortcomings. The Delta Test of Mathematics S k i l l s (DTMS): An Overview Based on the B.C. Mathematics Curriculum, the DTMS t e s t s student a b i l i t i e s along two s k i l l measures "math f a c t s " and "computation s k i l l s " . For the purposes of t h i s study the terms "math f a c t " and "computation s k i l l " have d i s t i n c t and p r e c i s e meanings and should not be used interchangeably. Math Fact: "Math f a c t s " are simple one step operations to which the answers are normally memorized. Thus, a student would be expected to supply, from memory, the answers to questions such as: 2 + 2 = , 3 - 2 = , 2 x 3 = , and 9 - 3 = . He would not be expected to have to compute the answer. (The universe of questions which f o r the purposes of t h i s study are considered "math f a c t s " i s contained i n Appendix B). 27 Computation S k i l l : The term "computation s k i l l " r e f e r s t o the a b i l i t y t o perform the mathematical o p e r a t i o n s necessary t o so l v e q u e s t i o n s such as: 56 + 75 = , 72 - 34 = , 53 x 129 = , and 298 - 24 = . The mathematical c a p a b i l i t i e s r e q u i r e d t o s o l v e such q u e s t i o n s i n c l u d e s the a b i l i t y t o r e c a l l , or at l e a s t the a b i l i t y t o compute b a s i c math f a c t s . The answers t o q u e s t i o n s r e q u i r i n g the a p p l i c a t i o n of computation s k i l l s are not normally committed t o memory. Separate forms f o r each of Grades 2, 3 and 4 were designed f o r the DTMS. A d d i t i o n and s u b t r a c t i o n were t e s t e d at Grade 2. M u l t i p l i c a t i o n , as w e l l as a d d i t i o n and s u b t r a c t i o n , were t e s t e d a t the Grade 3 l e v e l w h i l e d i v i s i o n was added at Grade 4. The sequence of c u r r i c u l a r items t e s t e d a t each grade l e v e l were i d e n t i f i e d and a t t e s t e d t o by teachers c u r r e n t l y t e a c h i n g the grade covered by the t e s t . T h i s v e r i f i c a t i o n helped ensure t h a t the DTMS would be a v a l i d t e s t of the D e l t a c u r r i c u l u m and the manner i n which teachers i n t e r p r e t e d t h a t c u r r i c u l u m . The Math Facts T e s t s of the DTMS The math f a c t s t e s t s f o r each f u n c t i o n were c o n s t r u c t e d by t a k i n g a s t r a t i f i e d random sample of the f a c t s as l i s t e d i n Appendix B. Such an approach t o item o r d e r i n g was taken i n an attempt t o ensure a uniform l e v e l of d i f f i c u l t y throughout the t e s t and t o av o i d a c l u s t e r i n g of q u e s t i o n t y p e s . The t e s t s are a l s o found i n Appendix B. 28 The Computations Tests of the DTMS T h e t e s t s w e r e d e v e l o p e d f r o m a p o o l o f i t e m s a s s e m b l e d i n M a y o f 1 9 8 6 a t H e a t h E l e m e n t a r y i n D e l t a . I n a s s e m b l i n g t h e p o o l , t e a c h e r s w h o w e r e c u r r e n t l y t e a c h i n g a t e a c h o f t h e g r a d e s i n q u e s t i o n w e r e a s k e d t o i d e n t i f y " q u e s t i o n t y p e s " w h i c h w o u l d b e r e p r e s e n t a t i v e o f t h e c o m p u t a t i o n s c u r r i c u l u m a s t h e y t a u g h t i t . " T e s t p a p e r s " r e p r e s e n t i n g e a c h o f t h e s e q u e s t i o n t y p e s w e r e t h e n a d m i n i s t e r e d t o c l a s s e s a t t h e a p p r o p r i a t e g r a d e l e v e l . Q u e s t i o n s w h i c h w e r e i d e n t i f i e d a s g o o d d i s c r i m i n a t o r s w e r e t h e n a c c e p t e d f o r t h e p o o l o f t e s t i t e m s f r o m w h i c h t h e t e s t s w e r e d e v e l o p e d . ( I t e m s w e r e c o n s i d e r e d g o o d d i s c r i m i n a t o r s i f 2 5 t o 7 5 % o f t h e s t u d e n t s a n s w e r e d t h e m c o r r e c t l y . ) S e p a r a t e t e s t s w e r e c r e a t e d f o r e a c h f u n c t i o n a t e a c h g r a d e l e v e l . E a c h r o w o f q u e s t i o n s o n e a c h t e s t c o n t a i n e d o n e q u e s t i o n o f e a c h t y p e i d e n t i f i e d b y t e a c h e r s a s r e p r e s e n t a t i v e o f t h e c o m p u t a t i o n c u r r i c u l u m . T h e p r e s e n t a t i o n o r d e r o f i t e m s w i t h i n e a c h 1 r o w w a s r a n d o m l y a s s i g n e d t o a v o i d e s t a b l i s h i n g a n y p a t t e r n o f i t e m d i f f i c u l t y . N o t e , t h a t i n c o n s t r u c t i n g t h e D T M S , t h e q u e s t i o n t y p e s i n c l u d e d i n t h e t e s t w e r e s e l e c t e d t o r e p r e s e n t t h e c u r r i c u l u m a s t a u g h t i n D e l t a . G o o d d i s c r i m i n a t o r s o f t h e q u e s t i o n t y p e w e r e t h e n c h o s e n f o r t h e t e s t . T h i s i s i n c o n t r a s t t o t h e p r o c e d u r e s u s e d i n t h e c o n s t r u c t i o n o f m a n y s t a n d a r d i z e d a c h i e v e m e n t t e s t s w h e r e q u e s t i o n t y p e s , w h i c h a r e g o o d d i s c r i m i n a t o r s , a r e s e l e c t e d f o r i n c l u s i o n i n t h e t e s t . A p p e n d i x B c o n t a i n s c o p i e s o f t h e c o m p u t a t i o n t e s t s o f t h e D T M S . Q u e s t i o n t y p e s a r e i d e n t i f i e d o n t h e s e c o p i e s o f t h e t e s t s . 29 The Canadian Tests of Basic S k i l l s (CTBS) The Primary and Elementary Multilevel Batteries of the Canadian Tests of Basic S k i l l s were designed and constructed by the professional staff of the -College of Education at the University of Iowa assisted by Dr. Ethel Kins of the Faculty of Education at the University of Calgary. The developers maintain that the CTBS "provide for the comprehensive measurement of growth in the fundamental s k i l l s : listening, vocabulary, reading, word analysis, language, word study, and mathematics." They further contend that, "The s k i l l s represented in the tests are crucial in educational development for they determine, for the most part, the extent to which pupils can p r o f i t from later instruction." More s p e c i f i c a l l y they suggest the Canadian Tests of Basic S k i l l s were designed to serve the following purposes: 1. To determine the developmental level of the pupil in order to better adapt materials and instructional procedures to individual needs and a b i l i t i e s ; 2. To diagnose specific qualitative strengths and weaknesses in a pupil's educational development; 3. To indicate the extent to which individual pupils have the specific readiness s k i l l s and a b i l i t i e s needed to begin instruction or to proceed to the next step in a planned instructional sequence; 4. To provide information that i s useful in making administrative decisions in grouping or programming to better provide for individual differences; 5. To diagnose strengths or weaknesses in group performance (class, building, or system) which have implications for changes in curriculum, instructional procedures, or emphasis; 6. To determine the relative effectiveness of alternate methods of instruction and the conditions which determine the effectiveness of the various procedures; 3 0 7 . T o a s s e s s t h e e f f e c t s o f e x p e r i m e n t a t i o n a n d i n n o v a t i o n ; 8. T o p r o v i d e a b e h a v i o u r a l m o d e l t o s h o w w h a t i s e x p e c t e d o f e a c h p u p i l a n d t o p r o v i d e f e e d b a c k w h i c h w i l l i n d i c a t e p r o g r e s s t o w a r d s u i t a b l e i n d i v i d u a l g o a l s ; 9. T o p r o v i d e i n f o r m a t i o n u s e f u l i n p l a n n i n g i n d i v i d u a l p r o g r a m s o f s t u d y a n d f o r m a k i n g f u t u r e e d u c a t i o n a l a n d v o c a t i o n a l p l a n s ; 1 0 . T o r e p o r t p e r f o r m a n c e i n b a s i c s k i l l s t o parentB a n d p a t r o n s i n o b j e c t i v e , m e a n i n g f u l t e r m s , ( p g 1 ) I n s e l e c t i n g b e h a v i o r a l o b j e c t i v e s t o b e r e p r e s e n t e d i n t h e t e s t s , t h e d e v e l o p e r s c o n d u c t e d a s y s t e m a t i c c o n s i d e r a t i o n o f c o u r s e s o f s t u d y , a n d r e l i e d o n t h e s t a t e m e n t s o f a u t h o r i t i e s i n m e t h o d a n d r e c o m m e n d a t i o n s o f n a t i o n a l c u r r i c u l u m g r o u p s . T h e i t e m s i n c l u d e d i n t h e t e s t w e r e s e l e c t e d f o r t h e i r " c r u c i a l i t y a n d d i s c r i m i n a t i n g p o w e r " ( p . 6 ) f r o m a l a r g e p o p u l a t i o n o f i t e m s t r i e d o u t i n c o n n e c t i o n w i t h t h e a n n u a l t e s t i n g p r o g r a m i n t h e s t a t e o f I o w a . " E a c h t e s t w a s m a d e l o n g e n o u g h t o p r o v i d e a s o u n d b a s i s f o r d i a g n o s i n g r e l a t i v e s t r e n g t h s a n d w e a k n e s s e s o f i n d i v i d u a l p u p i l s a n d a s s e s s i n g c h a n g e s i n p e r f o r m a n c e f r o m y e a r t o y e a r " ( p . 7 ) . T h e C T B S w a s s t a n d a r d i z e d i n C a n a d a i n t h e f a l l o f 1 9 8 0 . D r . E d g a r N . W r i g h t ( D i r e c t o r o f R e s e a r c h , t h e B o a r d o f E d u c a t i o n f o r t h e C i t y o f T o r o n t o ) w a s r e s p o n s i b l e f o r s e l e c t i n g a s t r a t i f i e d r a n d o m s a m p l e o f s c h o o l s , s e l e c t e d o n t h e b a s i s o f p r o v i n c e a n d s i z e o f s c h o o l . ( A l l t e n C a n a d i a n p r o v i n c e s a n d t w o t e r r i t o r i e s w e r e i n c l u d e d i n t h e s a m p l i n g . ) E n g l i s h w a s t h e m a j o r l a n g u a g e o f i n s t r u c t i o n i n p a r t i c i p a t i n g s c h o o l s a n d p u p i l s w h o w e r e l e a r n i n g E n g l i s h a s a s e c o n d l a n g u a g e w e r e n o t i n c l u d e d i n t h e s t a n d a r d i z a t i o n p r o c e s s . T h e r e l i a b i l i t y o f t e s t s c o r e s g e n e r a t e d b y t h e C T B S w a s e s t i m a t e d u s i n g K u d e r - R i c h a r d s o n F o r m u l a 20 ( K - R 20). T h e s e i n t e r n a l c o n s i s t e n c y r e l i a b i l i t y c o e f f i c i e n t s f o r t h e i n d i v i d u a l M a t h e m a t i c s S k i l l s T e s t s ( T a b l e 1) r a n g e f r o m .69 t o .88 w h i l e t h e c o m p o s i t e r e l i a b i l i t y f o r M a t h e m a t i c s S k i l l s T e s t s r a n g e s f r o m .89 t o .92. A s t h e d e v e l o p e r s o f t h e C T B S n o t e , e q u i v a l e n t f o r m r e l i a b i l i t y c o e f f i c i e n t s a r e g e n e r a l l y c o n s i d e r e d s u p e r i o r t o i n t e r n a l c o n s i s t e n c y m e a s u r e s i n t h a t a l l f o u r m a j o r s o u r c e s o f e r r o r a r e c o n s i d e r e d : " v a r i a t i o n s a r i s i n g w i t h i n t h e m e a s u r e m e n t p r o c e d u r e , c h a n g e s i n t h e s p e c i f i c s a m p l e o f t a s k s , c h a n g e s i n t h e i n d i v i d u a l f r o m d a y t o d a y a n d c h a n g e s i n t h e i n d i v i d u a l ' s s p e e d o f w o r k " (p.66). I n t e r n a l c o n s i s t e n c y m e a s u r e s s u c h a s K-R 20 t a k e i n t o a c c o u n t o n l y t h e f i r s t t w o s o u r c e s o f e r r o r . T h u s , K-R 20 e s t i m a t e s t e n d t o b e h i g h e r t h a n t h o s e o b t a i n e d b y e q u i v a l e n t f o r m p r o c e d u r e s . U n d o u b t e d l y , t h e C T B S h a s b e e n d e v e l o p e d t o t h e h i g h e s t p r o f e s s i o n a l s t a n d a r d s . H o w e v e r , t h a t d o e s n o t p e r m i t i t t o e s c a p e t h e o b j e c t i o n s r a i s e d i n C h a p t e r 2 c o n c e r n i n g s t a n d a r d i z e d a c h i e v e m e n t t e s t s . S t a n d a r d i z e d a c h i e v e m e n t t e s t s d o n o t a d e q u a t e l y s a m p l e s p e c i f i c s k i l l s ( Z i g m o n d & S i l v e r m a n , 1984). T h e y l a c k c o n t e n t v a l i d i t y w i t h r e s p e c t t o a s t u d e n t ' s c u r r i c u l u m ( J e n k i n s & P a n y , 1978) a n d d i f f i c u l t i e s c a n a r i s e i n t e r p r e t i n g d e r i v e d s c o r e s g e n e r a t e d b y s u c h t e s t s ( T a l l m a d g e , 1977; D e n o , J e n k i n s a n d M i r k e n , 1979). 32 TABLE 1 THE KODER-RICHARDSON FORMULA 20 INTERNAL CONSISTENCY RELIABILITY COEFFICIENTS FOR THE CTBS MATHEMATICS SKILLS TESTS. C o n c e p t s P r o b l e m s C o m p u t a t i o n s T o t a l G r a d e 2 .78 .69 .80 .89 G r a d e 3 .84 .79 .87 .92 G r a d e 4 .79 .81 .88 .92 F r o m C T B S M a n u a l f o r A d m i n i s t r a t o r s , S u p e r v i s o r s a n d C o u n s e l l o r s (p.68). 33 The B.C. Quick Individual  Educational Test (The B.C. Quiet) T h e B . C . Q u i e t w a s d e s i g n e d t o m e a s u r e i n s t r u c t i o n i n G r a d e s 1 t o 7 i n t h r e e g e n e r a l a r e a s : r e a d i n g , s p e l l i n g a n d a r i t h m e t i c . T h e d e v e l o p e r o f t h e B . C . Q u i e t c o n t e n d s t h a t t h e r e i s a c o n s i d e r a b l e d e g r e e o f u n i f o r m i t y i n t h e e d u c a t i o n o f p u p i l s i n B r i t i s h C o l u m b i a b e c a u s e o f t h e i n f l u e n c e t h a t t h e M i n i s t r y o f E d u c a t i o n e x e r t s t h r o u g h t h e d i s t r i b u t i o n o f p r o v i n c i a l c u r r i c u l u m g u i d e s a n d t h e s e l e c t i o n o f p r e s c r i b e d a n d a u t h o r i z e d m a t e r i a l s . T h e s e s a m e m a t e r i a l s a n d c u r r i c u l u m g u i d e s w e r e u s e d i n t h e d e v e l o p m e n t o f t h e B . C . Q u i e t i n a n e f f o r t t o c o n s t r u c t a t e s t c l o s e l y r e l a t e d t o i n s t r u c t i o n i n t h e s c h o o l s . I t e m s i n t h e m a t h e m a t i c s s u b t e s t w e r e s e l e c t e d " t o r e p r e s e n t t h e " c o r e c u r r i c u l u m " l e a r n i n g o u t c o m e s f o r G r a d e s 1 t h r o u g h 7 d e s c r i b e d i n t h e B . C . M a t h e m a t i c s C u r r i c u l u m G u i d e " ( W o r m e l i , 1983, p.5). T h a t i s , t h e B . C . Q u i e t w a s n o t d e s i g n e d t o t e s t o n l y c o m p u t a t i o n s k i l l s ; r a t h e r , i t w a s , d e s i g n e d t o a s s e s s p u p i l p r o g r e s s t h r o u g h t h e e s e e n t i a l s o f t h e e n t i r e m a t h e m a t i c s c u r r i c u l u m . I n n o r m i n g t h e B . C . Q u i e t t h e s a m p l e w a s s e l e c t e d t o b e r e p r e s e n t a t i v e o f t h e E n g l i s h - s p e a k i n g e l e m e n t a r y s c h o o l p o p u l a t i o n o f B r i t i s h C o l u m b i a . T h i s p o p u l a t i o n e x c l u d e d c h i l d r e n w h o w e r e f o r m a l l y d i a g n o s e d a s r e t a r d e d , e m o t i o n a l l y d i s t u r b e d , p h y s i c a l l y h a n d i c a p p e d t o t h e e x t e n t t h a t i t w o u l d b e d i f f i c u l t f o r t h e m t o t a k e a p a p e r a n d p e n c i l t e s t o r w h o h a d n o t m a s t e r e d E n g l i s h s u f f i c i e n t l y t o u n d e r s t a n d t e s t d i r e c t i o n s . A t w o - s t a g e s t r a t i f i e d d e s i g n w a s e m p l o y e d t o s e l e c t t h e n o r m i n g s a m p l e . I n t h e f i r s t s t a g e s c h o o l s w e r e s t r a t i f i e d b y 34 g e o g r a p h i c r e g i o n , r u r a l - u r b a n d i s t r i b u t i o n a n d s i z e . I n t h e s e c o n d s t a g e p u p i l s w e r e s t r a t i f i e d b y g r a d e a n d s e x . T w o h u n d r e d a n d f i f t y p u p i l s p e r g r a d e w e r e s e l e c t e d t o p a r t i c i p a t e i n t h e n o r m i n g . O f t h e s e i t w a s h o p e d t o o b t a i n 200 u s a b l e p r o t o c o l s . I n f a c t , o n l y a n a v e r a g e o f r o u g h l y 150 u s a b l e p r o t o c o l s p e r g r a d e w e r e r e t u r n e d . I n t e r n a l c o n s i s t e n c y r e l i a b i l i t i e s c a l c u l a t e d f o r t h e B . C . Q u i e t m a t h e m a t i c s s u b t e s t s w e r e r e p o r t e d a s : .87 f o r G r a d e 2, .82 f o r G r a d e 3 a n d .83 f o r G r a d e 4 ( W o r m e l i , 1983, p.38). A Comparison of the DTMS and the CTBS S i n c e t h e D e l t a T e s t o f M a t h e m a t i c s S k i l l s w a s j u d g e d t o b e a n a c c u r a t e a n d c o m p r e h e n s i v e t e s t o f t h e c o m p u t a t i o n r e q u i r e m e n t s o f t h e D e l t a M a t h e m a t i c s C u r r i c u l u m b y t e a c h e r s c u r r e n t l y t e a c h i n g t h e c u r r i c u l u m , a c o m p a r i s o n o f t h e DTMS t e s t i t e m s a n d t h o s e o f t h e c o m p u t a t i o n s t e s t o f t h e C a n a d i a n T e s t o f B a s i c S k i l l s s h o u l d p r o v i d e a n a c c u r a t e p i c t u r e o f how w e l l t h e C T B S t e s t s t h e c o m p u t a t i o n r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m . T a b l e s A - l t o A-9 c o m p a r e t h e DTMS a n d t h e C T B S . T h e i t e m s t e s t e d b y t h e DTMS h a v e b e e n c a t e g o r i z e d a s " A t G r a d e L e v e l ' i n t h e s e T a b l e s o n t h e s t r e n g t h o f t h e a s s e s s m e n t n o t e d a b o v e . T h e C T B S t e s t i t e m s h a v e b e e n c a t e g o r i z e d a s e i t h e r ""At G r a d e L e v e l ' , " A b o v e G r a d e L e v e l ' , " B e l o w G r a d e L e v e l ' o r a " V a r i a t i o n a t G r a d e L e v e l ' . I t e m s i n t h i s l a t t e r c a t e g o r y c o r r e s p o n d i n s o m e r e s p e c t s t o c u r r i c u l a r r e q u i r e m e n t s b u t c o n t a i n v a r i a t i o n s w h i c h m a y p r e s e n t u n n e c e s s a r y o b s t a c l e s t o t h e s t u d e n t b e i n g t e s t e d . T A B L E 2 A N U M E R I C A L SUMMARY T H E C T B S T E S T I T E M S TO A-9. B e l o w G r a d e L e v e l G r a d e 2 A d d i t i o n 1* S u b t r a c t i o n TOTALS 1 G r a d e 3 A d d i t i o n 1 S u b t r a c t i o n 2 M u l t i p l i c a t i o n TOTALS 3 G r a d e 4 A d d i t i o n 10 S u b t r a c t i o n 7 M u l t i p l i c a t i o n 6 D i v i s i o n 3 TOTALS 26 35 O P T H E C A T E G O R I Z A T I O N O F E F F E C T E D I N T A B L E S A - l V a r i a t i o n A t A t Above G r a d e G r a d e G r a d e L e v e l L e v e l L e v e l 3 2 2 3 2 3 6 4 5 3 11 10 3 1 2 5 15 14 6 2 1 6 3 2 1 12 1 2 * The numbers r e p r e s e n t t h e number o f CTBS q u e s t i o n s i n e a c h c a t e g o r y d e f i n e d . « 36 The placement of the CTBS t e s t items w i t h r e s p e c t t o the f o u r c a t e g o r i e s has been summarized n u m e r i c a l l y i n Table 2. With one apparent e x c e p t i o n (Grade 3 s u b t r a c t i o n ) , i t would appear t h a t the CTBS does not t e s t the computations' requirements of the B.C. Mathematics C u r r i c u l u m as d e f i n e d by teachers who s e l e c t e d the behaviors t e s t e d by the DTMS. The "apparent e x c e p t i o n ' does not, however, stand up under c l o s e r s c r u t i n y . An examination of Table A-4 r e v e a l s t h a t the CTBS t e s t s o n l y f i v e of the nine c u r r i c u l a r items the D e l t a t e s t i d e n t i f i e s under the heading of Grade 3 s u b t r a c t i o n . With eleven c u r r i c u l a r v a r i a t i o n s , the Grade 3 a d d i t i o n q u e s t i o n s on the CTBS appear more a t e s t of a student's a b i l i t y t o d e a l w i t h c u r r i c u l a r v a r i a t i o n s than w i t h standard c u r r i c u l a r items. S i m i l a r l y , w i t h f i v e items above grade l e v e l and o n l y two a t grade l e v e l , the Grade 3 m u l t i p l i c a t i o n q u e s t i o n s would appear t o pr o v i d e a t best an i n t e r e s t i n g f i r s t l ook a t a student's a b i l i t y t o m u l t i p l y . The Grade 2 and 4 s u b t e s t s p r o v i d e a p u z z l i n g c o n t r a s t w i t h the m a j o r i t y of qu e s t i o n s a t the Grade 2 l e v e l being designated " c u r r i c u l a r v a r i a t i o n s " and "above grade l e v e l " w h i l e the que s t i o n s on the Grade 4 t e s t f a l l i n t o the "below grade l e v e l " c a t egory. Furthermore, those c u r r i c u l a r items which are t e s t e d on the CTBS are not t e s t e d s u f f i c i e n t l y t o a l l o w comment as t o whether, i n f a c t , the s k i l l was mastered. C l e a r l y the CTBS computations s u b t e s t i s not a t e s t of the computations' requirements of the B.C. Mathematics C u r r i c u l u m . 37 A C o m p a r i s o n o f S t u d e n t P e r f o r m a n c e  on t h e DTMS a n d t h e CTBS In developing the DTMS Delta teachers who were c u r r e n t l y teaching at each of the grades i n question were asked to i d e n t i f y "question types" which would be representative of the computations curriculum as they taught i t . These "question types" were then included i n the DTMS, thus ensuring i t s c u r r i c u l a r v a l i d i t y with regard to the B.C. Mathematics Curriculum as taught i n Delta. The CTBS does not enjoy t h i s c u r r i c u l a r v a l i d i t y with respect to the B.C. Curriculum as taught i n Delta because, as previously noted, i t was designed to represent general c u r r i c u l a r trends across Canada. Subsequently normed on students from schools representing a l l the provinces and t e r r i t o r i e s i n the country, one cannot automatically assume the norming population would accurately r e f l e c t the student population of Delta. Assessment of the value of the CTBS Computations Test as a measure of success with the computation requirements of the B.C. Mathematics Curriculum was obtained by administering the CTBS Computations Test to Grade 2, 3, and 4 students at one of the schools p a r t i c i p a t i n g i n the study and comparing the scores they obtained on t h i s t e s t with t h e i r Session 3 and 4 scores on the DTMS. (Due to timetabling considerations the CTBS was administered three-four weeks a f t e r the Session 4 t e s t was administered.) Hopkins and Stanley (1981) state that standardized t e s t s should have r e l i a b i l i t y c o e f f i c i e n t s of at l e a s t .90 because they must be able to accurately assess a student's achievement without 38 r e f e r e n c e t o o t h e r t e s t s . F e r g u s s o n (1981) n o t e s t h a t s q u a r i n g t h e c o r r e l a t i o n c o e f f i c i e n t o b t a i n e d f o r t w o t e s t s p e r m i t s a v a r i a n c e i n t e r p r e t a t i o n o f t h e r e l a t i o n s h i p b e t w e e n s t u d e n t s c o r e s o n t h e m . T h u s , i f t h e c o r r e l a t i o n c o e f f i c i e n t b e t w e e n t w o t e s t s i s .90 ( H o p k i n s a n d S t a n l e y ' s r e c o m m e n d e d r e l i a b i l i t y c o e f f i c i e n t ) , we c a n s q u a r e t h a t a n d c l a i m t h a t i f we h a v e o n e t e s t s c o r e , we k n o w 81% o f w h a t we n e e d t o k n o w t o m a k e a p e r f e c t p r e d i c t i o n o f a s t u d e n t ' s s c o r e o n t h e o t h e r t e s t . A p p l y i n g t h i s s t a n d a r d t o t h e r e l a t i o n s h i p b e t w e e n t h e G r a d e 4 C T B S C o m p u t a t i o n s T e s t a n d t h e S e s s i o n 3 a n d 4 s c o r e s o n t h e D TMS we c a n s e e t h a t t h e r e l a t i o n s h i p f a l l s s o m e w h a t s h o r t . A v a r i a n c e i n t e r p r e t a t i o n o f t h e c o r r e l a t i o n c o e f f i c i e n t ( T a b l e 3) r e v e a l s t h a t t h e C T B S a c c o u n t s f o r b e t w e e n 29 a n d 53 p e r c e n t o f t h e v a r i a n c e o f a s t u d e n t ' s D TMS s c o r e . T h a t i s , i f we k n o w a s t u d e n t ' s C T B S s c o r e , we k n o w b e t w e e n 29 a n d 53 p e r c e n t o f w h a t we n e e d t o k n o w i f we w e r e t o m a k e a p e r f e c t p r e d i c t i o n o f t h e s t u d e n t ' s D TMS s c o r e i f t e s t e d w i t h i n t w o m o n t h s o f t a k i n g t h e C T B S C o m p u t a t i o n s T e s t . T h a t p a r t o f t h e v a r i a n c e o f t h e DTMS w h i c h c a n b e a c c o u n t e d f o r b y t h e C T B S w o u l d a p p e a r t o d i m i n i s h s l i g h t l y ( f r o m S e s s i o n 4 t o 3) w i t h t i m e . I t i s a l s o i n t e r e s t i n g t o n o t e t h a t t h e C T B S a c c o u n t s f o r s u b s t a n t i a l l y m o r e v a r i a t i o n i n t h e G r a d e 2 DTMS t h a n a t t h e G r a d e 3 o r 4 l e v e l w h e r e t h e v a r i e t y o f c o m p u t a t i o n s k i l l s w h i c h c a n b e t a u g h t a n d t e s t e d i s m u c h l a r g e r t h a n a t t h e G r a d e 2 l e v e l . 39 TABLE 3 THE SQUARED PEARSON CORRELATION COEFFICIENTS BETWEEN STUDENT SCORES ON THE CTBS COMPUTATIONS TEST AND THEIR MEAN TOTAL MATHEMATICS FACT SCORES (TOTFAC) AND THEIR MEAN TOTAL COMPUTATIONS SKILLS SCORES (TOTCOMP) ON THE DTMS. C T B S Grade 2 Grade 3 Grade 4 Session 3 TOTFAC TOTCOMP .30 .49 .29 .29 .29 .30 Session 4 TOTFAC .52 .36 .29 TOTCOMP .53 .42 .39 40 TABLE 4 A TABLE SHOWING THE FREQUENCY WITH WHICH SKILLS ARE TESTED IN THE B.C. QUIET MATHEMATICS SUBTEST ( 6 5 I T E M S ) . W h o l e N u m b e r s D e c i m a l F r a c t i o n s C ommon F r a c t i o n s M i x e d N u m b e r s A d d i t i o n S u b t r a c t i o n M u l t i p l i c a t i o n D i v i s i o n 3 5 9 5 3 1 1 G e o m e t r y 6 A l g e b r a 3 W r i t i n g N u m b e r s 3 G r o u p i n g 2 T e l l i n g T i m e 2 M e a s u r i n g L e n g t h 2 M o n e y 1 T h e C a l e n d a r 1 F a c t o r i n g S q u a r e R o o t 1 I n t e r e s t 1 C h a n g i n g D e c i m a l s t o F r a c t i o n s 1 C h a n g i n g C ommon F r a c t i o n s t o P e r c e n t 1 C h a n g i n g D e c i m a l F r a c t i o n s t o P e r c e n t 1 * P l u s 1 1 o t h e r s k i l l s 41 I t w o u l d a p p e a r t h a t t h e C T B S i s n o t a g o o d p r e d i c t o r o f s u c c e s s o n t h e D T M S . T h e r e f o r e , b y e x t e n s i o n i t m a y a l s o b e a r g u e d t h a t t h e C T B S i s n o t a g o o d m e a s u r e o f s u c c e s s o n t h e B . C . M a t h e m a t i c s C o m p u t a t i o n s C u r r i c u l u m . A Comparison of the DTMS and the B.C. Quiet T h e B . C . Q u i e t w a s n o r m e d i n B r i t i s h C o l u m b i a i n 1982 u s i n g t h e n c u r r e n t c u r r i c u l a r m a t e r i a l s a n d i t e m s . W i t h m i n o r v a r i a t i o n s i n i n s t r u c t i o n s , a s i n g l e t e s t h a s b e e n d e s i g n e d f o r u s e b e t w e e n G r a d e 1 a n d G r a d e 7. T a b l e 4 s h o w s t h e f r e q u e n c y w i t h w h i c h s k i l l s a r e t e s t e d i n t h e M a t h e m a t i c s S u b t e s t o f t h e B . C . Q u i e t . O b v i o u s l y t h e B . C . Q u i e t h a s s e v e r e l i m i t a t i o n s a s a m e a s u r e o f p r o g r e s s t h r o u g h t h e c o m p u t a t i o n r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m . Summary T h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s w e r e d e v e l o p e d i n c o n s u l t a t i o n w i t h t e a c h e r s c u r r e n t l y t e a c h i n g t h e D e l t a M a t h e m a t i c s C u r r i c u l u m a t e a c h g r a d e l e v e l t e s t e d . I t m a y b e j u d g e d t o b e a c o m p r e h e n s i v e a n d a c c u r a t e m e a s u r e o f t h e B . C . M a t h e m a t i c s C o m p u t a t i o n C u r r i c u l u m a s t a u g h t a t e a c h o f G r a d e s 2, 3, a n d 4 i n D e l t a . T h e C a n a d i a n T e s t s o f B a s i c S k i l l s h a s , i n t h e v i e w o f i t s d e s i g n e r s , b e e n d e v e l o p e d i n r e s p o n s e t o n a t i o n w i d e t r e n d s i n c u r r i c u l u m a n d m e t h o d o l o g y . A n a n a l y s i s o f t h e c o n t e n t d e m o n s t r a t e s t h a t i t m a y n o t a d e q u a t e l y r e f l e c t t h e s k i l l s t e s t e d i n t h e DTMS a l l o w i n g u s t o c o n c l u d e t h a t i t m a y n o t 42 b e a n a d e q u a t e m e a s u r e o f t h e c o m p u t a t i o n r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m . S i m i l a r l y , t h e B . C . Q u i e t m a y n o t b e s e e n t o m e a s u r e t h o s e r e q u i r e m e n t s . C l e a r l y t h e DTMS w o u l d a p p e a r t o f i l l a v o i d l e f t b y t h e s e t w o c o m m o n l y u s e d a c h i e v e m e n t t e s t s . 43 CHAPTER 4: METHOD The Study; Purpose and Hypotheses The purpose of t h i s study was to develop measures of mathematics computation s k i l l s and measures of f a c i l i t y with mathematics f a c t s f o r each of Grades 2, 3 and 4 (The Delta Tests of Mathematics S k i l l s ) . Each of these t e s t s was designed to measure student achievement i n one of the four basic functions: a d d i t i o n , subtraction, m u l t i p l i c a t i o n and d i v i s i o n . Test items were sel e c t e d to r e f l e c t the requirements f o r each grade l e v e l as prescribed by the B r i t i s h Columbia Mathematics Curriculum. The following hypotheses were stated to focus the development of t h i s study and to o f f e r d i r e c t i o n i n the a n a l y s i s of the data r e s u l t i n g from i t s implementation: 1. Curriculum-based achievement t e s t s such as the Delta Tests of Mathematics S k i l l s which assess achievement i n mathematics f a c t s and mathematics computation s k i l l s can reveal s i g n i f i c a n t changes i n student performance over b r i e f (one month) periods of time. These changes could not be measured on standardized achievement t e s t s such as the CTBS. 2. Curriculum-based achievement t e s t s such as the Delta Tests of Mathematics S k i l l s which assess achievement i n mathematics f a c t s and mathematics computation s k i l l s can reveal s i g n i f i c a n t growth i n group performance over b r i e f (one month) periods of time. This growth could not be measured on standardized achievement t e s t s such as the CTBS. The Community Delta i s a rural/suburban community bordering on the S t r a i t s of Georgia and the south arm of the Fraser River about 20 miles south of the c i t y of Vancouver, B r i t i s h Columbia's l a r g e s t 44 p o p u l a t i o n c e n t r e . T h e 1 7 , 3 3 9 s t u d e n t s i n D e l t a a t t e n d 2 5 e l e m e n t a r y s c h o o l s a n d e i g h t s e c o n d a r y s c h o o l s . S i x o f t h e e l e m e n t a r y s c h o o l s a r e " d u a l - t r a c k " s c h o o l s ( D u a l - t r a c k s c h o o l s a r e s c h o o l s w i t h i n s t r u c t i o n i n b o t h E n g l i s h a n d F r e n c h . ) A l t h o u g h E n g l i s h i s t h e m o t h e r t o n g u e o f t h e m a j o r i t y o f D e l t a ' s s t u d e n t s , i t i s a s e c o n d l a n g u a g e f o r o n e o r t w o p e r c e n t o f t h e c h i l d r e n i n s o m e s c h o o l s . M o s t o f t h e s e c h i l d r e n a r e o f E a s t I n d i a n o r i g i n . An Overview T h e t e s t s c o m p r i s i n g T h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ( D T M S ) w e r e b a s e d o n t h e B . C . M a t h e m a t i c s C u r r i c u l u m a n d d e v e l o p e d t o a s s e s s m a t h e m a t i c s c o m p u t a t i o n s k i l l s a n d f l u e n c y w i t h m a t h e m a t i c s f a c t s a t g r a d e s 2, 3, a n d 4. T h e t e r m s " m a t h e m a t i c s c o m p u t a t i o n s k i l l " a n d " m a t h e m a t i c s f a c t " h a v e p a r t i c u l a r m e a n i n g s f o r t h i s s t u d y a n d h a v e b e e n d e f i n e d i n C h a p t e r 3. D i f f e r e n t t e s t p a c k a g e s w e r e c r e a t e d f o r e a c h g r a d e . T h e s e q u e n c e o f c u r r i c u l a r i t e m s t e s t e d a t e a c h g r a d e l e v e l a r e d e s c r i b e d i n A p p e n d i x B . T h e y w e r e i d e n t i f i e d b y t e a c h e r s f a m i l i a r w i t h t h e B . C . M a t h e m a t i c s C u r r i c u l u m w h o w e r e t h e n t e a c h i n g t h e g r a d e o n w h i c h t h e i r a d v i c e w a s s o u g h t . T h e p r a c t i c e o f s e e k i n g t h e a d v i c e o f t e a c h e r s c u r r e n t l y t e a c h i n g t h e g r a d e i n q u e s t i o n w a s u n d e r t a k e n t o e n s u r e t h a t t h e t e s t s w o u l d h a v e c u r r i c u l a r a n d i n s t r u c t i o n a l v a l i d i t y . O n c e q u e s t i o n t y p e s w e r e i d e n t i f i e d , " t e s t p a p e r s " o f e a c h q u e s t i o n t y p e w e r e p r e p a r e d a n d a d m i n i s t e r e d t o c l a s s e s o f s t u d e n t s a t t h e g r a d e l e v e l i n q u e s t i o n . A p o o l o f t e s t i t e m s w a s s e l e c t e d f r o m t h e s e q u e s t i o n s . T h e s e p o o l i t e m s w e r e t h e n u s e d i n t h e c o n s t r u c t i o n o f t h e t e s t s . E a c h o f t h e f o u r f u n c t i o n s ( a d d i t i o n , s u b t r a c t i o n , m u l t i p l i c a t i o n a n d d i v i s i o n ) w a s t e s t e d s e p a r a t e l y b u t n o t n e c e s s a r i l y a t e a c h g r a d e l e v e l . T a b l e 5 d i s p l a y s t h e f u n c t i o n s t e s t e d a t e a c h g r a d e l e v e l . O n e M a t h e m a t i c s F a c t s t e s t f o r e a c h f u n c t i o n w a s c r e a t e d f o r u s e a c r o s s a p p r o p r i a t e g r a d e s b u t s e p a r a t e C o m p u t a t i o n s S k i l l s t e s t s w e r e d e v e l o p e d f o r e a c h g r a d e . C o p i e s o f t h e t e s t s c a n b e f o u n d i n A p p e n d i x B . T h e f i r s t o f t h e f o u r n o r m i n g s e s s i o n s b e g a n i n t h e l a s t w e e k o f J a n u a r y , 1 9 8 7 . ( T h e t e s t s c h e d u l e i s c o n t a i n e d i n T a b l e 6.) D u r i n g e a c h s e s s i o n , e a c h p a r t i c i p a t i n g c l a s s w a s a d m i n i s t e r e d o n e c o m p l e t e p a c k a g e o f g r a d e a p p r o p r i a t e t e s t s . U p o n c o m p l e t i o n , t h e t e s t b o o k l e t s w e r e c o l l e c t e d b y t h e r e s e a r c h a s s i s t a n t . C l a s s r o o m t e a c h e r s w e r e n o t r e q u i r e d t o p a r t i c i p a t e i n e i t h e r t h e a d m i n i s t r a t i o n o r m a r k i n g o f t h e t e s t s . I t w a s h y p o t h e s i z e d t h a t p u p i l p e r f o r m a n c e a s m e a s u r e d b y t h e D T M S , w o u l d i m p r o v e f r o m m o n t h t o m o n t h . T h i s g r o w t h c o u l d b e a t t r i b u t e d t o o n - g o i n g i n s t r u c t i o n a n d p r a c t i c e , m a t u r a t i o n a n d o t h e r f a c t o r s t h a t m i g h t n o r m a l l y b e a s s o c i a t e d w i t h i m p r o v e d p u p i l p e r f o r m a n c e i n t h e a r e a s t e s t e d . T o e n s u r e t h a t n o r m a l m o n t h l y g r o w t h w a s m e a s u r e d , t e a c h e r s w e r e a s k e d t o c o n t i n u e t e a c h i n g t h e m a t h e m a t i c s c u r r i c u l u m a s t h e y n o r m a l l y w o u l d . I n o r d e r t o m i n i m i z e t h e t e m p t a t i o n t o a d j u s t t h e i r p r o g r a m s t o e f f e c t i m p r o v e m e n t i n p u p i l p e r f o r m a n c e o n t h e D T M S , t e a c h e r s w e r e n o t i n f o r m e d o f t h e s c o r e s o b t a i n e d b y t h e i r s t u d e n t s . E v a l u a t i o n o f t h e r e s u l t s o f t h e s t u d y ( d e t a i l e d i n C h a p t e r 5 ) u t i l i z e d c o n v e n t i o n a l , g e n e r a l l y a c c e p t e d s t a t i s t i c a l p r o c e d u r e s . TABLE 5 FUNCTIONS TESTED AT EACH GRADE LEVEL BY THE DTMS. G r a d e 2 G r a d e 3 G r a d e 4 A d d i t i o n S u b t r a c t i o n M u l t i p l i c a t i o n D i v i s i o n TABLE 6 TEST SCHEDULE FOR THE NORMING OF THE DTMS. S c h o o l 2 J a n .28 F e b .25 Mar .25 A p r .22 5 J a n .29 Feb .26 Mar .26 A p r .23 6 J a n .29 Feb .26 Mar .26 A p r .23 7 J a n .30 Feb .27 Mar .27 A p r .24 3 F e b . 2 M a r . 2 Mar .30 A p r .27 4 F e b . 3 M a r . 3 Ma r .31 A p r .30 1 F e b . 4 M a r . 4 A p r . 1 A p r .29 *30 d a y s b e t w e e n t e s t i n g due t o B . C . T . F . s t r i k e . The Development of the Math Facts Tests of the DTMS T h e m a t h f a c t s t e s t s f o r e a c h f u n c t i o n w e r e c o n s t r u c t e d b y t a k i n g a s t r a t i f i e d r a n d o m s a m p l e o f t h e f a c t s a s l i s t e d i n A p p e n d i x B . T h i s r a n d o m l y o r d e r e d l i s t o f q u e s t i o n s w a s p r e s e n t e d i n t h r e e d o u b l e - s p a c e d c o l u m n s o n t h e t e s t p a p e r s ( A p p e n d i x B ) . T h e s e t e s t s w e r e c o m m o n a c r o s s g r a d e s . The Development of the Computations Tests of the DTMS T h e C o m p u t a t i o n S k i l l s t e s t s o f t h e DTMS a r e g r a d e s p e c i f i c T h e y w e r e d e v e l o p e d f r o m a p o o l o f t e s t i t e m s s e l e c t e d f r o m t h e B . C . M a t h e m a t i c s C u r r i c u l u m . I n a s s e m b l i n g t h e p o o l o f t e s t i t e m s t e a c h e r s w h o w e r e c u r r e n t l y t e a c h i n g a t e a c h o f t h e g r a d e s i n q u e s t i o n w e r e a s k e d t o i d e n t i f y " q u e s t i o n t y p e s " w h i c h w o u l d b e r e p r e s e n t a t i v e o f t h e c o m p u t a t i o n s c u r r i c u l u m a s t h e y t a u g h t i t . I n A p r i l , M a y a n d J u n e o f 1 9 8 6 a t H e a t h E l e m e n t a r y i n D e l t a , " t e s t p a p e r s " r e p r e s e n t i n g e a c h o f t h e s e q u e s t i o n t y p e s w e r e a d m i n i s t e r e d t o c l a s s e s a t t h e a p p r o p r i a t e g r a d e l e v e l . Q u e s t i o n s w h i c h w e r e i d e n t i f i e d a s g o o d d i s c r i m i n a t o r s w e r e a c c e p t e d f o r t h e p o o l o f t e s t i t e m s . ( I t e m s w e r e c o n s i d e r e d g o o d d i s c r i m i n a t o r s i f 2 5 t o 7 5 % o f t h e s t u d e n t s a n s w e r e d t h e m c o r r e c t l y . ) N o t e t h a t g o o d d i s c r i m i n a t o r s o f q u e s t i o n t y p e s r e p r e s e n t a t i v e o f t h e B . C . C u r r i c u l u m w e r e s e l e c t e d f o r i n c l u s i o n i n t h e D T M S . M a n y s t a n d a r d i z e d a c h i e v e m e n t t e s t s , o n t h e o t h e r h a n d , s e l e c t q u e s t i o n t y p e s w h i c h a r e g o o d d i s c r i m i n a t o r s . S e l e c t i n g f r o m t h e s e i t e m p o o l s , t e s t s w e r e d e v e l o p e d t o a s s e s s t h e a c q u i s i t i o n o f c o m p u t a t i o n s k i l l s a t e a c h g r a d e l e v e l . S e p a r a t e t e s t s w e r e c r e a t e d f o r e a c h f u n c t i o n a t e a c h g r a d e l e v e l . 48 E a c h r o w o f q u e s t i o n s o n e a c h t e s t c o n t a i n e d o n e q u e s t i o n o f e a c h t y p e i d e n t i f i e d b y t e a c h e r s a s r e p r e s e n t a t i v e o f t h e c o m p u t a t i o n c u r r i c u l u m . T h e p r e s e n t a t i o n o r d e r o f i t e m s w i t h i n e a c h r o w w a s r a n d o m l y a s s i g n e d t o a v o i d c l u s t e r i n g o f q u e s t i o n t y p e s . A p p e n d i x B c o n t a i n s c o p i e s o f t h e c o m p u t a t i o n t e s t s o f t h e D T M S . Q u e s t i o n t y p e s a r e i d e n t i f i e d o n t h e s e c o p i e s o f t h e t e s t s . Norming the DTMS The Subjects. A t t h e i r r e g u l a r m e e t i n g i n O c t o b e r o f 1986 L e a r n i n g A s s i s t a n c e t e a c h e r s i n D e l t a w e r e i n f o r m e d o f t h e d e v e l o p m e n t a n d n o r m i n g o f t h e D T M S . T h e y w e r e a s k e d t o a d v i s e t h e p r i n c i p a l a n d t e a c h e r s a t t h e i r s c h o o l o f t h e p r o j e c t a n d t o r e q u e s t t h e i r s c h o o l ' s p a r t i c i p a t i o n i n t h e n o r m i n g . T e a c h e r s a t s e v e n o f D e l t a ' s 2 5 e l e m e n t a r y s c h o o l s a g r e e d t o p a r t i c i p a t e . T h e s e s c h o o l s r e f l e c t e d t h e d i s t r i c t ' s g e o g r a p h i c a l v a r i a t i o n s . T o e n s u r e t h a t a s c h o o l ' s s c o r e s w o u l d n o t b e s k e w e d b y s a m p l e s e l e c t i o n , a l l t h e s t u d e n t s i n a g r a d e w e r e r e q u i r e d t o p a r t i c i p a t e i n t h e n o r m i n g b e f o r e t h a t g r a d e a t t h a t s c h o o l , w a s a c c e p t e d f o r t h e p r o j e c t . I n " d u a l - t r a c k " s c h o o l s ( s c h o o l s w i t h i n s t r u c t i o n i n b o t h E n g l i s h a n d F r e n c h ) a l l t h e s t u d e n t s a t e a c h g r a d e l e v e l i n a p r o g r a m ( e i t h e r E n g l i s h o r F r e n c h ) w e r e r e q u i r e d t o p a r t i c i p a t e i n t h e n o r m i n g b e f o r e t h a t g r a d e i n t h a t p r o g r a m , i n t h a t s c h o o l w a s p e r m i t t e d t o p a r t i c i p a t e . A s c a n b e s e e n f r o m T a b l e 7, f u l l p a r t i c i p a t i o n w a s n o t a c h i e v e d o r m a i n t a i n e d a t t h e g r a d e 4 l e v e l i n t h r e e s c h o o l s . 49 TABLE 7 TABLE SHOWING THE NUMBER OF STUDENTS BY SCHOOL PARTICIPATING IN THE NORMING OF THE DTMS AT EACH GRADE AS OF THE FIRST TESTING SESSION. S c h o o l T o t a l s G r a d e 2 G r a d e 3 G r a d e 4 1 58 52 57 2 60 64 84 3 43 44 43 4 47 52 51 (25)** 5 28 40 * 6 22 24 41 7 28E 29E * 55F 47F 53F 341 352 329 E - S t u d e n t s i n a d u a l t r a c k s c h o o l w h o r e c e i v e i n s t r u c t i o n i n E n g l i s h . F - F r e n c h I m m e r s i o n a n d P r o g r a m m e C a d r e s t u d e n t s i n a d u a l - t r a c k s c h o o l . * - T h e G r a d e 4's w e r e n o t a c c e p t e d f o r t h e p r o j e c t b e c a u s e 100% p a r t i c i p a t i o n c o u l d n o t b e a c h i e v e d . * * - A G r a d e 4 c l a s s w i t h d r e w f r o m t h e p r o j e c t a f t e r t h e f i r s t r o u n d o f t e s t i n g . T h e s e c o n d c l a s s w a s r e t a i n e d b e c a u s e l i t t l e d i f f e r e n c e w a s p e r c e i v e d b e t w e e n t h e t w o c l a s s e s a n d i t s r e t e n t i o n w a s n o t c o n s i d e r e d a t h r e a t t o t h e a c c u r a c y o f t h e r e s u l t s . Testing Procedures. B e g i n n i n g w i t h S c h o o l #2 o n J a n u a r y 2 8 , 1 9 8 7 a n d e x t e n d i n g t h r o u g h t o A p r i l 3 0 , t h e m a t h f a c t a n d c o m p u t a t i o n p r o b e s w e r e a d m i n i s t e r e d t o e a c h c l a s s o n c e e v e r y 2 8 d a y s b y a r e s e a r c h a s s i s t a n t ( T a b l e 6 ) . T h e o r d e r o f p r e s e n t a t i o n o f t h e t e s t s i s d i s p l a y e d i n T a b l e 8. T h e f i r s t c o m p u t a t i o n s a d m i n i s t r a t i o n o f e a c h d a y ( a d d i t i o n ) w a s p r e c e d e d b y o r a l i n s t r u c t i o n s a s d e t a i l e d i n A p p e n d i x C . I n a n e f f o r t t o e n s u r e t h a t a l l c h i l d r e n p a i d c l o s e a t t e n t i o n t o t h e i n s t r u c t i o n s , a q u e s t i o n a n d c l a s s r e s p o n s e t e c h n i q u e w a s i n c o r p o r a t e d i n t o t h e i n s t r u c t i o n s . T h e i n s t r u c t i o n s e n c o u r a g e d c h i l d r e n t o l o o k c a r e f u l l y a t e a c h q u e s t i o n o n t h e t e s t p a p e r . I f a f t e r a c a r e f u l e x a m i n a t i o n o f a q u e s t i o n t h e y w e r e s u r e t h e y d i d n o t k n o w h o w t o d o i t , t h e y w e r e i n s t r u c t e d t o p u t a n "X" o n i t . T h i s t e c h n i q u e o f r e q u i r i n g a c h i l d t o p u t a n "X" o n a q u e s t i o n h e c o u l d n o t d o w a s u t i l i z e d b y C a l t a g i r o n e a n d G l o v e r ( 1 9 8 5 ) t o e n s u r e t h a t p u p i l s l o o k e d a t e a c h i t e m o n t h e t e s t p a p e r . T h e c h i l d r e n w e r e e n c o u r a g e d t o w o r k q u i c k l y a n d a c c u r a t e l y a n d c h e c k t h e i r w o r k o n t h e c u r r e n t t e s t i f t h e y f i n i s h e d b e f o r e t i m e . S u b s e q u e n t c o m p u t a t i o n p r o b e s r e c e i v e d s h o r t e n e d i n s t r u c t i o n s a s d e t a i l e d i n A p p e n d i x C , P a r t B . T h e s e i n s t r u c t i o n s c o n t a i n e d t h e s a m e i n f o r m a t i o n c o n t a i n e d i n t h e l o n g e r v e r s i o n b u t w e r e m o r e c o n c i s e a n d t o t h e p o i n t , a s s u m i n g s t u d e n t s h a d l e a r n e d a b o u t t h e t e s t p r o c e d u r e s f r o m t h e f i r s t s e t o f i n s t r u c t i o n s . 51 TABLE 8 TABLE SHOWING THE ORDER OF PRESENTATION OF TESTS DURING NORMING SESSIONS. G r a d e 2 A d d i t i o n F a c t s A d d i t i o n C o m p u t a t i o n s S u b t r a c t i o n F a c t s S u b t r a c t i o n C o m p u t a t i o n s G r a d e 3 A d d i t i o n F a c t s A d d i t i o n C o m p u t a t i o n s S u b t r a c t i o n F a c t s S u b t r a c t i o n C o m p u t a t i o n s M u l t i p l i c a t i o n F a c t s M u l t i p l i c a t i o n C o m p u t a t i o n s G r a d e 4 A d d i t i o n F a c t s A d d i t i o n C o m p u t a t i o n s S u b t r a c t i o n F a c t s S u b t r a c t i o n C o m p u t a t i o n s M u l t i p l i c a t i o n F a c t s M u l t i p l i c a t i o n C o m p u t a t i o n s D i v i s i o n F a c t s D i v i s i o n C o m p u t a t i o n s 52 T h e f i r s t m a t h f a c t s a d m i n i s t r a t i o n o f t h e d a y a l s o r e c e i v e d d e t a i l e d i n s t r u c t i o n s s i m i l a r i n c o n t e n t t o t h e l o n g v e r s i o n o f t h e m a t h e m a t i c s c o m p u t a t i o n i n s t r u c t i o n s ( A p p e n d i x C , P a r t C ) . S u b s e q u e n t " m a t h f a c t s " p r o b e s r e c e i v e d a n a b b r e v i a t e d f o r m o f i n s t r u c t i o n s ( A p p e n d i x C , P a r t D ) . O n c e t h e s t u d e n t s b e g a n w o r k o n a t e s t t h e a d m i n i s t r a t o r w a l k e d a m o n g s t t h e m t o e n s u r e t h a t t h e d i r e c t i o n s w e r e b e i n g a d h e r e d t o a n d t o e n c o u r a g e t h e s t u d e n t s t o d o t h e i r b e s t w o r k . A t t h e e n d o f t h e f i v e m i n u t e t i m e p e r i o d t h e e x a m i n e r s t o p p e d t h e s t u d e n t s . I f t h e r e w e r e o t h e r p r o b e s t o a d m i n i s t e r , t h e s t u d e n t s w e r e g i v e n a s h o r t b r e a k ( o n e o r t w o m i n u t e s ) b e f o r e b e g i n n i n g t h e n e x t o n e . A t t h e c o n c l u s i o n o f a d a y ' s t e s t i n g t h e b o o k l e t s w e r e c o l l e c t e d b y t h e r e s e a r c h a s s i s t a n t . T h e y w e r e c o r r e c t e d l a t e r b y r e s e a r c h a s s i s t a n t s . Summary I n t h i s c h a p t e r t h e d e v e l o p m e n t a n d n o r m i n g o f t h e DTMS w e r e d i s c u s s e d . D e v e l o p m e n t a l i s s u e s a d d r e s s e d i n c l u d e d a n o u t l i n i n g o f t h e p r o c e d u r e s f o l l o w e d i n s e l e c t i n g q u e s t i o n t y p e s f o r e a c h o f t h e t e s t s t o e n s u r e c u r r i c u l a r v a l i d i t y , a d e t a i l i n g o f t h e m e t h o d u s e d t o c r e a t e i t e m p o o l s f o r e a c h o f t h e t e s t s a n d a d e s c r i p t i o n o f t h e d e s i g n o f t h e t e s t s . S u b j e c t s e l e c t i o n , t i m e t a b l i n g a n d t e s t a d m i n i s t r a t i o n w e r e d i s c u s s e d i n t h e s e c t i o n o n n o r m i n g t h e D T M S . CHAPTER 5: DATA ANALYSIS 5 3 T h i s c h a p t e r b e g i n s w i t h a n a n a l y s i s o f t h e r e l i a b i l i t y a n d v a l i d i t y o f t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ( D T M S ) . T h i s a n a l y s i s i s f o l l o w e d b y a n e x a m i n a t i o n o f t h e s e s s i o n a l m e a n s c o r e s o b t a i n e d d u r i n g n o r m i n g . T h e s e s c o r e s w e r e e x a m i n e d u s i n g a r e p e a t e d m e a s u r e s d e s i g n o f a n a l y s i s o f v a r i a n c e , T u k e y ' s H S D ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) t e s t a n d t r e n d a n a l y s i s . A n e x a m i n a t i o n o f t h e c u m u l a t i v e p e r c e n t a g e p o l y g o n s ( O g i v e s ) r e s u l t i n g f r o m t h e a c h i e v e d s c o r e s c o m p l e t e s t h e d a t a a n a l y s i s . R e l i a b i l i t y and Validity " T h e c o n c e p t s o f r e l i a b i l i t y a n d v a l i d i t y a r e c e n t r a l t o t h e t h e o r y a n d p r a c t i c e o f e d u c a t i o n a l a n d p s y c h o l o g i c a l t e s t i n g " ( H o p k i n s & S t a n l e y , 1 9 8 1 ; p . 1 1 3 ) . T o e n s u r e t h e c u r r i c u l a r a n d i n s t r u c t i o n a l v a l i d i t y o f t h e D T M S i t w a s c a r e f u l l y c o n s t r u c t e d t o r e f l e c t b o t h t h e B . C . M a t h e m a t i c s C u r r i c u l u m a n d t h e m a n n e r i n w h i c h t h i s c u r r i c u l u m i s e v i d e n c e d i n t h e i n s t r u c t i o n a l p r a c t i c e s o f D e l t a ' s t e a c h e r s ( s e e C h a p t e r s 3 a n d 4). T h e r e l i a b i l i t y o f t h e D TMS h a s b e e n d i s c u s s e d b e l o w . T o d o i t s j o b w e l l t h e DTMS o r a n y o t h e r t e s t m u s t y i e l d a c c u r a t e r e s u l t s . T h a t i s t o s a y , t h e s c o r e i t y i e l d s f o r a s t u d e n t t o d a y m u s t b e q u i t e s i m i l a r t o t h e s c o r e i t y i e l d s f o r h i m / h e r u n d e r s i m i l a r c o n d i t i o n s t o m o r r o w . R e l i a b i l i t y i s t h e g e n e r a l t e r m u s e d t o d e s c r i b e t h e e x t e n t t o w h i c h a t e s t m e a s u r e s c o n s i s t e n t l y ( H o p k i n s & S t a n l e y , 1 9 8 1 ) . 54 O n c r i t e r i o n - r e f e r e n c e d t e s t s o r o t h e r m i n i m u m c o m p e t e n c y m e a s u r e s H o p k i n s a n d S t a n l e y ( 1 9 8 1 ) n o t e d t h a t t h e s t a n d a r d e r r o r o f m e a s u r e m e n t i s a m o r e m e a n i n g f u l i n d i c a t i o n o f m e a s u r e m e n t p r e c i s i o n t h a n r e l i a b i l i t y c o e f f i c i e n t s . T h e y m a i n t a i n e d t h a t i f t h e s t a n d a r d e r r o r o f m e a s u r e m e n t i s f i v e p e r c e n t o r l e s s o f t h e n u m b e r o f t e s t i t e m s t h e t e s t w i l l o r d i n a r i l y h a v e v e r y h i g h m e a s u r e m e n t p r e c i s i o n r e g a r d l e s s o f t h e m a g n i t u d e o f t h e r e l i a b i l i t y c o e f f i c i e n t ( p . 1 3 4 ) . H o w e v e r , i n " s p e e d e d t e s t s , " t h a t i s , t e s t s i n w h i c h n o t a l l s t u d e n t s h a d s u f f i c i e n t t i m e t o d o a l l t h e i t e m s , c o m p a r i n g t h e s i z e o f t h e s t a n d a r d e r r o r o f m e a s u r e m e n t t o t h e n u m b e r o f i t e m s i n t h e t e s t w o u l d b e i n a p p r o p r i a t e . T h e DTMS i s a s p e e d e d t e s t . T o a s s e s s t h e r e l i a b i l i t y o f s p e e d e d t e s t s , H o p k i n s a n d S t a n l e y ( 1 9 8 1 ) r e c o m m e n d e d t h e u s e o f t e s t - r e t e s t p r o c e d u r e s . T h e y n o t e d t h a t " s p l i t - h a l f a n d i n t e r n a l c o n s i s t e n c y m e t h o d s a r e a p p r o p r i a t e o n l y f o r p o w e r t e s t s , t e s t s i n w h i c h e v e r y s t u d e n t h a s a d e q u a t e t i m e t o c o m p l e t e e a c h i t e m , " ( p . 1 3 3 ) a n d t h a t i f s p l i t -h a l f a n d i n t e r n a l c o n s i s t e n c y p r o c e d u r e s a r e u s e d w i t h s p e e d e d t e s t s t h e y w i l l p r o d u c e s p u r i o u s l y h i g h r e l i a b i l i t y c o e f f i c i e n t s . T e s t - r e t e s t p r o c e d u r e s r e q u i r e t h e a d m i n i s t r a t i o n o f a g i v e n m e a s u r e t w o o r m o r e t i m e s t o t h e s a m e g r o u p o f i n d i v i d u a l s . T h e i n t e r c o r r e l a t i o n s a m o n g t h e s c o r e s o n t h e v a r i o u s a d m i n i s t r a t i o n s a r e t a k e n a s t h e r e l i a b i l i t y c o e f f i c i e n t . " I f t h e t e s t i s a d m i n i s t e r e d s e v e r a l t i m e s , t h e u s u a l p r a c t i c e i s t o t a k e t h e a v e r a g e o f t h e i n t e r c o r r e l a t i o n s a m o n g t h e s c o r e s o b t a i n e d o n t h e 55 v a r i o u s o c c a s i o n s a s t h e e s t i m a t e o f t h e r e l i a b i l i t y c o e f f i c i e n t " ( G h i s e l l i , C a m p b e l l a n d Z e d e c k , 1 9 8 1 ; p . 2 4 7 ) . T h e m o s t o b v i o u s a d v a n t a g e o f t h e t e s t - r e t e s t m e t h o d o f e s t a b l i s h i n g r e l i a b i l i t y i s t h a t o n l y o n e f o r m o f t h e t e s t i s r e q u i r e d . T h i s m o s t o b v i o u s a d v a n t a g e i s a l s o s e e n a s a m o s t o b v i o u s l i a b i l i t y a s i t p e r m i t s a v a r i e t y o f c a r r y - o v e r e f f e c t s t o c o m e i n t o p l a y w h i c h c o u l d i m p a c t o n a s t u d e n t ' s s c o r e . G h i s e l l i , C a m p b e l l a n d Z e d e c k ( 1 9 8 1 ) i d e n t i f i e d t h r e e c a r r y -o v e r e f f e c t s w h i c h c o u l d b e s e e n t o a p p l y w h e n i n t e r p r e t i n g t h e r e s u l t s o f t h i s s t u d y . T h e f i r s t o f t h e s e i s t h a t t h e a d m i n i s t r a t i o n o f t h e t e s t p r o v i d e s t h e s t u d e n t w i t h a n o p p o r t u n i t y t o p r a c t i s e t h e p a r t i c u l a r s k i l l s a d d r e s s e d b y t h e t e s t . T h e e f f e c t o f t h i s p r a c t i c e c o u l d b e a n i n c r e a s e i n a s t u d e n t ' s s c o r e s o n a s u b s e q u e n t a d m i n i s t r a t i o n o f t h e t e s t . " P r a c t i c e e f f e c t s ' w e r e n o t v i e w e d a s a p r o b l e m w h e n i n t e r p r e t i n g t h e t e s t - r e t e s t c o r r e l a t i o n s o b t a i n e d d u r i n g t h e n o r m i n g o f t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ( D T M S ) b e c a u s e o f t h e l e n g t h o f t h e t i m e i n t e r v a l b e t w e e n a d m i n i s t r a t i o n s o f t h e t e s t ( o n e m o n t h ) a n d t h e l i m i t e d p r a c t i c e o f e a c h s k i l l p r o v i d e d b y t h e t e s t s . R e m e m b e r i n g o n o n e t e s t i n g s e s s i o n t h e r e s p o n s e s g i v e n o n a n o t h e r s e s s i o n a n d m e r e l y r e p e a t i n g t h e s e r e s p o n s e s i s t h e s e c o n d s p e c i f i c c a r r y - o v e r e f f e c t i d e n t i f i e d b y G h i s e l l i , C a m p b e l l a n d Z e d e c k ( 1 9 8 1 ) . A g a i n , i n t e r m s o f t h i s s t u d y , t h e f a c t t h a t t h e c h i l d r e n w e r e a d m i n i s t e r e d a s e r i e s o f t e s t s i n a r e l a t i v e l y b r i e f t i m e f r a m e a n d t h e n n o t r e t e s t e d f o r a m o n t h s h o u l d e f f e c t i v e l y 56 p r e c l u d e a n y s i g n i f i c a n t c a r r y - o v e r e f f e c t r e s u l t i n g f r o m t h e m e m o r i z a t i o n o f r e s p o n s e s . T h e t h i r d c a r r y - o v e r e f f e c t , t h e t i m e i n t e r v a l b e t w e e n t e s t a d m i n i s t r a t i o n s c a n b e a t r o u b l e s o m e p r o b l e m i n s o m e i n s t a n c e s . G h i s e l l i , C a m p b e l l a n d Z e d e c k (1981) m a i n t a i n e d t h a t w h i l e " i t i s d e s i r a b l e t o m a x i m i z e t h e i n t e r v a l b e t w e e n t e s t i n g o c c a s i o n s i n o r d e r t o m i n i m i z e t h e e f f e c t s o f m e m o r y . . . t h e l o n g e r t h e t i m e i n t e r v a l b e t w e e n t h e t w o t e s t i n g o c c a s i o n s , t h e g r e a t e r t h e l i k e l i h o o d t h a t t h e t r u e s c o r e o f t h e i n d i v i d u a l w i l l c h a n g e . . . C o n s e q u e n t l y , we e x p e c t l o w e r a n d l o w e r e s t i m a t e s o f r e l i a b i l i t y a s t h e t i m e i n t e r v a l b e t w e e n t h e t e s t i n g o c c a s i o n s i n c r e a s e s " (p.249). U n d o u b t a b l y t h i s t h i r d c a r r y - o v e r e f f e c t s h o u l d b e c o n s i d e r e d w h e n i n t e r p r e t i n g t e s t - r e t e s t c o e f f i c i e n t s a s s o c i a t e d w i t h t h e D T M S . R e c a l l t h e f o l l o w i n g h y p o t h e s e s w h i c h w e r e p r o p o s e d t o f o c u s t h e d e v e l o p m e n t a n d n o r m i n g o f t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ( D T M S ) . 1. C u r r i c u l u m - b a s e d a c h i e v e m e n t t e s t s s u c h a s t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s w h i c h a s s e s s a c h i e v e m e n t i n m a t h e m a t i c s f a c t s a n d m a t h e m a t i c s c o m p u t a t i o n s k i l l s c a n r e v e a l s i g n i f i c a n t c h a n g e s i n s t u d e n t p e r f o r m a n c e o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e . T h e s e c h a n g e s c o u l d n o t b e m e a s u r e d o n s t a n d a r d i z e d a c h i e v e m e n t t e s t s s u c h a s t h e C T B S . 2. C u r r i c u l u m - b a s e d a c h i e v e m e n t t e s t s s u c h a s t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s w h i c h a s s e s s a c h i e v e m e n t i n m a t h e m a t i c s f a c t s a n d m a t h e m a t i c s c o m p u t a t i o n s k i l l s c a n r e v e a l s i g n i f i c a n t g r o w t h i n g r o u p p e r f o r m a n c e o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e . T h i s g r o w t h c o u l d n o t b e m e a s u r e d o n s t a n d a r d i z e d a c h i e v e m e n t t e s t s s u c h a s t h e C T B S . T h e s e h y p o t h e s e s s t a t e t h a t we s h o u l d e x p e c t c h a n g e i n t h e p e r f o r m a n c e o f b o t h i n d i v i d u a l s a n d g r o u p s o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e a n d t h a t f o r m a t h e m a t i c s f a c t s , a n d m a t h e m a t i c s c o m p u t a t i o n s k i l l s , t h e DTMS s h o u l d p e r m i t u s t o m e a s u r e t h o s e c h a n g e s . T h a t i s t o s a y , i f we e x p e c t c h a n g e s i n t h e t r u e s c o r e o f i n d i v i d u a l s t a k i n g t h e D T M S , we m u s t , a s i n d i c a t e d a b o v e , e x p e c t l o w e r e s t i m a t e s o f r e l i a b i l i t y . I t w o u l d n o t b e u n r e a s o n a b l e t o e x p e c t t h a t we c a n p r o b a b l y a c c e p t a s s a t i s f a c t o r y r e l i a b i l i t y c o e f f i c i e n t s l o w e r t h a n , b u t a p p r o a c h i n g t h e .90 r e c o m m e n d e d b y H o p k i n s a n d S t a n l e y (1981) f o r s t a n d a r d i z e d a c h i e v e m e n t t e s t s s u c h a s t h o s e u s e d f o r s p e c i a l c l a s s p l a c e m e n t b e c a u s e , a s t h e y n o t e d , c l a s s r o o m t e s t s ( s u c h a s t h e D T M S ) c a n b e l e s s r e l i a b l e b e c a u s e t h e y a r e g e n e r a l l y o n l y p a r t o f a t o t a l " s c o r e ' w h i c h w i l l l i k e l y h a v e h i g h e r r e l i a b i l i t y t h a n i t s c o m p o n e n t s . T e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s w e r e c a l c u l a t e d b e t w e e n S e s s i o n s 1 a n d 2, 2 a n d 3, a n d 3 a n d 4 f o r t h e i n d i v i d u a l f u n c t i o n s t e s t e d b y t h e DTMS a t e a c h g r a d e l e v e l ( T a b l e s D - l t o D - 6 ) . R e l i a b i l i t y c o e f f i c i e n t s r a n g i n g f r o m .50 t o .89 w e r e o b t a i n e d . T h e d i s t r i b u t i o n o f t h e s e c o r r e l a t i o n c o e f f i c i e n t s i s s h o w n i n F i g u r e 3. A s n o t e d a b o v e , i f a t e s t i s a d m i n i s t e r e d s e v e r a l t i m e s , t h e r e l i a b i l i t y c o e f f i c i e n t i s o b t a i n e d b y a v e r a g i n g t h e i n t e r -c o r r e l a t i o n s a m o n g t h e s c o r e s o b t a i n e d o n t h e v a r i o u s o c c a s i o n s ( G h i s e l l i e t a l ; 1 9 8 1 ) . T a b l e 9 g i v e s t h e r e l i a b i l i t y c o e f f i c i e n t s o b t a i n e d b y s u c h a p r o c e d u r e f o r t h e v a r i o u s t e s t s o f t h e D T M S . T h e y a r e s e e n t o r a n g e f r o m .59 f o r G r a d e 4 D i v i s i o n C o m p u t a t i o n s t o .86 f o r G r a d e 2 S u b t r a c t i o n F a c t s . 58 FIGURE 3 DISTRIBUTION OF TEST-RETEST RELIABILITY COEFFICIENTS CALCULATED BETWEEN SESSIONS 1 AND 2, 2 AND 3 AND 3 AND 4 FOR INDIVIDUAL FUNCTIONS AS REPORTED IN TABLES D-l THROUGH D-6. Frequency 16 15 14 13 12 11 10 9 0 -8 - 0 0 7 - 0 0 -6 - - 0 0 -5 - - 0 0 -4 0 X 0 0 -3 0 X X 0 0 2 0 0 X X 0 X 1 X X X X X X .46-50 .51-55 .56-60 .61-65 .66-70 .71-75 . 76-80 .81-85 .86-C o r r e l a t i o n C o e f f i c i e n t s Grade 2 - 'X' Grade 3 - '0' Grade 4 - '-' 59 TABLE 9 RELIABILITY COEFFICIENTS FOR EACH TEST OF THE DTMS OBTAINED BY AVERAGING THE CORRELATIONS CALCULATED BETWEEN SESSIONS 1 AND 2, 2 AND 3 AND 3 AND 4 AS REPORTED ON TABLES T>-1 TO D-6. G r a d e 2 G r a d e 3 G r a d e 4 A d d i t i o n F a c t s .77 .74 .70 A d d i t i o n C o m p u t a t i o n s .72 . 8 1 .66 S u b t r a c t i o n F a c t s .86 .80 .80 S u b t r a c t i o n C o m p u t a t i o n s .73 .80 .80 M u l t i p l i c a t i o n F a c t s .77 .84 M u l t i p l i c a t i o n C o m p u t a t i o n s .76 .80 D i v i s i o n F a c t s .85 D i v i s i o n C o m p u t a t i o n s .59 6 0 I n a d d i t i o n t o c a l c u l a t i n g r e l i a b i l i t y c o e f f i c i e n t s f o r i n d i v i d u a l t e s t s a t e a c h g r a d e l e v e l , e a c h s t u d e n t ' s m a t h e m a t i c s f a c t s s c o r e s a n d m a t h e m a t i c s c o m p u t a t i o n s k i l l s s c o r e s w e r e t o t a l e d f o r e a c h s e s s i o n a n d t e s t - r e t e s t r e l i a b i l i t i e s w e r e c a l c u l a t e d f o r t h e s e t o t a l s c o r e s ( T a b l e s D-7 t o D-9). R e l i a b i l i t y c o e f f i c i e n t s r a n g i n g b e t w e e n .72 a n d .94 w e r e o b t a i n e d . T h e d i s t r i b u t i o n o f t h e c o r r e l a t i o n c o e f f i c i e n t s i s s h o w n i n F i g u r e 4. T a b l e 1 0 g i v e s t h e r e l i a b i l i t y c o e f f i c i e n t s f o r e a c h g r o u p i n g w h i c h w e r e o b t a i n e d b y a v e r a g i n g t h e c o r r e l a t i o n s a m o n g t h e s c o r e s o b t a i n e d o n t h e f o u r a d m i n i s t r a t i o n s o f t h e t e s t s . T h e e f f e c t o f t i m e o n t h e m a g n i t u d e o f t h e c o r r e l a t i o n s f o r t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s c o r r e c t i s m a d e e v i d e n t i n T a b l e 1 1 . I n a l l c a s e s t h e c o r r e l a t i o n c o e f f i c i e n t s o f t h e t o t a l s t a t i s t i c s a r e s e e n t o d e c r e a s e i n s i z e o v e r t i m e . W h i l e t h e d e c r e a s e f o r a n y o n e g r o u p i n g o f t e s t s a t a n y p a r t i c u l a r g r a d e l e v e l ( e . g . T o t a l C o m p u t a t i o n s S k i l l s , G r a d e 4) m a y n o t b e l a r g e e n o u g h t o b e s i g n i f i c a n t b y t h e m s e l v e s , t h e t r e n d f o r a l l g r o u p i n g s f o r a l l g r a d e s w o u l d a p p e a r t o b e t h a t c o r r e l a t i o n c o e f f i c i e n t s d e c r e a s e o v e r t i m e . T h i s p e r m i t s u s t o s a y , f o r e x a m p l e , t h a t t o t a l t e s t r e s u l t s f r o m S e s s i o n 1 a r e b e t t e r a b l e t o p r e d i c t t o t a l t e s t s c o r e s i n S e s s i o n 2 t h a n t h e y a r e i n S e s s i o n 3 a n d b e t t e r a b l e t o p r e d i c t t o t a l t e s t s c o r e s i n S e s s i o n 3 t h a n i n S e s s i o n 4. A n e x a m i n a t i o n o f t h e t e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s c a l c u l a t e d b e t w e e n S e s s i o n s 1 a n d 2, 1 a n d 3, a n d 1 a n d 4 f o r 61 FIGURE 4 DISTRIBUTION OF THE TEST-RETEST RELIABILITY COEFFICIENTS CALCULATED BETWEEN SESSIONS 1 AND 2, 2 AND 3 AND 3 AND 4 FOR TOTAL SCORES CORRECT AS REPORTED IN TABLES D-7, D-8 AND D-9. Frequency 8 7 6 - -5 0 0 4 0 0 3 X 0 -2 X X -I X X X 0 .76- .80 .81- .85 .86- .90 .91- .95 C o r r e l a t i o n C o e f f i c i e n t s Grade 2 - 'X* Grade 3 - '0 ' Grade 4 - '-' 62 TABLE 10 RELIABILITY COEFFICIENTS FOR TOTAL MATHEMATICS FACTS CORRECT AND TOTAL MATHEMATICS COMPUTATIONS SKILLS CORRECT OBTAINED BY AVERAGING THE CORRELATIONS CALCULATED BETWEEN SESSIONS 1 AND 2, 2 AND 3 AND 3 AND 4 AS REPORTED IN TABLES D-7 TO D-9. G r a d e 2 G r a d e 3 G r a d e 4 T o t a l M a t h e m a t i c s F a c t s C o r r e c t .82 .87 .92 T o t a l C o m p u t a t i o n s S k i l l s C o r r e c t .88 .88 .86 63 TABLE 11 TEST-RETEST RELIABILITY COEFFICIENTS BETWEEN SESSIONS 1 AND 2, 1 AND 3, AND 1 AND 4 SCORES FOR TOTAL MATHEMATICS FACTS CORRECT AND TOTAL MATHEMATICS COMPUTATIONS SKILLS CORRECT AS REPORTED IN TABLES D-7 TO D-9. T e s t - R e t e s t R e l i a b i l i t y Between S e s s i o n s Total Mathematics Facts Correct Grade 1 and 2 1 and 3 1 and 4 2 3 4 .79 .85 .88 .74 .84 .84 .72 .80 .81 Total Mathematics Computations S k i l l s Correct Grade 1 and 2 1 and 3 1 and 4 2 3 4 .84 .83 .83 .77 .80 .82 .74 .76 .80 6 4 i n d i v i d u a l f u n c t i o n s ( T a b l e 1 2 ) r e v e a l s t h a t w i t h t h e e x c e p t i o n o f G r a d e 2 A d d i t i o n F a c t s a n d C o m p u t a t i o n S k i l l s , t h e a b i l i t y t o p r e d i c t p e r f o r m a n c e o n t h e i n d i v i d u a l t e s t s o f t h e DTMS f o r G r a d e s 2 a n d 3, d e c r e a s e s o v e r t i m e . T h i s t r e n d i s n o t e v i d e n t f o r t h e i n d i v i d u a l t e s t s a t t h e G r a d e 4 l e v e l . R e l i a b i l i t y and Validity; Discussion A s p r e v i o u s l y n o t e d , " t h e c o n c e p t s o f r e l i a b i l i t y a n d v a l i d i t y a r e c e n t r a l t o t h e t h e o r y a n d p r a c t i c e o f e d u c a t i o n a l a n d p s y c h o l o g i c a l t e s t i n g " ( H o p k i n s a n d S t a n l e y , 1 9 8 1 ; p . 1 1 3 ) . A s d e t a i l e d i n C h a p t e r 3, t h e DTMS w a s c a r e f u l l y c o n s t r u c t e d t o r e f l e c t b o t h t h e B . C . M a t h e m a t i c s C u r r i c u l u m a n d t h e m a n n e r i n w h i c h t h i s c u r r i c u l u m i s e v i d e n c e d i n t h e i n s t r u c t i o n a l p r a c t i c e s o f D e l t a ' s t e a c h e r s . I t s c u r r i c u l a r a n d i n s t r u c t i o n a l v a l i d i t y w a s e n s u r e d b y i n c l u d i n g i n i t o n l y i t e m t y p e s w h i c h h a d b e e n i d e n t i f i e d a n d a p p r o v e d b y e x p e r i e n c e d t e a c h e r s w h o w e r e t h e n e n r o l l i n g c l a s s e s a t t h e g r a d e s i n q u e s t i o n . T h e r e l i a b i l i t y o f t h e DTMS h a s b e e n d e m o n s t r a t e d b y u s e o f t h e t e s t - r e t e s t p r o c e d u r e s a n d r e s u l t s r e p o r t e d i n t h e p r e v i o u s s e c t i o n . A s e v i d e n c e d i n T a b l e 1 0 , t h e t e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s f o r t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s e i t h e r e x c e e d o r a p p r o a c h t h e .90 r e c o m m e n d e d b y H o p k i n s a n d S t a n l e y ( 1 9 8 1 ) f o r s t a n d a r d i z e d a c h i e v e m e n t t e s t s . C o n s i d e r e d i n l i g h t o f t h e a s s e r t i o n b y G h i s e l l i , C a m p b e l l a n d Z e d e c k ( 1 9 8 1 ) t h a t "we e x p e c t l o w e r a n d l o w e r e s t i m a t e s o f r e l i a b i l i t y a s t h e t i m e i n t e r v a l b e t w e e n t h e t e s t i n g o c c a s i o n s i n c r e a s e s " ( p . 2 4 9 ) t h e a c h i e v e m e n t o f t e s t -TABLE 12 TEST-RETEST RELIABILITY COEFFICIENTS BETWEEN SESSIONS 1 AND 2, 1 AND 3, AND 1 AND 4 MATHEMATICS FACTS AND MATHEMATICS COMPUTATIONS SKILLS SCORES FOR EACH FUNCTION TESTED AS REPORTED IN TABLES D-l TO D-6. Mathematics Facts T e s t - R e t e s t R e l i a b i l i t y B e t w e e n S e s s i o n s • a d e F u n c t i o n 1 a n d 2 1 a n d 3 1 a n d 2 A d d i t i o n . 7 5 .70 .70 2 S u b t r a c t i o n . 8 3 .76 .74 3 A d d i t i o n .64 .62 . 5 7 3 S u b t r a c t i o n . 7 9 .76 .75 3 M u l t i p l i c a t i o n . 7 0 . 6 5 . 5 9 4 A d d i t i o n .50 .59 .54 4 S u b t r a c t i o n . 71 . 7 3 . 7 1 4 M u l t i p l i c a t i o n .82 .80 .80 4 D i v i s i o n . 80 .77 . 7 3 Mathematics Computations S k i l l s a d e F u n c t i o n 1 a n d 2 1 a n d 3 1 a n d 2 A d d i t i o n . 6 9 .62 . 6 3 2 S u b t r a c t i o n . 6 5 . 5 9 .54 3 A d d i t i o n . 7 9 .70 . 6 5 3 S u b t r a c t i o n .70 .68 .63 3 M u l t i p l i c a t i o n . 67 . 5 5 .46 4 A d d i t i o n .60 . 5 9 . 5 9 4 S u b t r a c t i o n .74 . 7 3 .67 4 M u l t i p l i c a t i o n .72 .65 .68 4 D i v i s i o n . 50 .38 . 2 5 66 r e - t e s t r e l i a b i l i t y c o e f f i c i e n t s o f t h e m a g n i t u d e o f t h o s e r e p o r t e d a b o v e s u g g e s t s t h a t a s t u d e n t ' s t o t a l m a t h e m a t i c s f a c t s s c o r e a n d h i s / h e r t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s s c o r e m a y b e q u i t e r e l i a b l e . T h e t e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s f o r e a c h t e s t o f t h e DTMS a r e r e p o r t e d i n T a b l e 9. F o r t h e m o s t p a r t , t h e s e c o r r e l a t i o n s a r e s e e n t o b e s u b s t a n t i a l , t h o u g h s m a l l e r t h a n t h o s e o b t a i n e d f o r t h e " t o t a l ' s c o r e s . T h i s d e c r e a s e i n r e l i a b i l i t y i s m o s t l i k e l y a t t r i b u t a b l e t o t h e r e d u c e d s i z e o f t h e b e h a v i o r a l s a m p l e a v a i l a b l e w h e n c o n s i d e r i n g t h e r e s u l t s o f o n e t e s t . F o r e x a m p l e , a s t u d e n t ' s s c o r e o n t h e G r a d e 4 d i v i s i o n f a c t s t e s t r e p r e s e n t s o n l y o n e - q u a r t e r o f h i s / h e r t o t a l m a t h e m a t i c s f a c t s s c o r e . T h e e f f e c t o f l e n g t h e n i n g t h e d i v i s i o n t e s t c a n b e e s t i m a t e d u s i n g t h e S p e a r m a n - B r o w n f o r m u l a : r\*K = k r > x / <1 + <k - 1 ) r y K ) w h e r e r y y = a n e s t i m a t e o f r e l i a b i l i t y o f a t e s t o f u n i t l e n g t h r f e k = r e l i a b i l i t y o f a t e s t m a d e k t i m e s a s l o n g ( F e r g u s o n , 1 9 8 1 , p . 4 4 0 ) T h u s , i f r x y = . 5 9 ( a s i t d o e s f o r G r a d e 4 D i v i s i o n ) a n d t h e t e s t i s m a d e f o u r t i m e s a s l o n g , t h e r e l i a b i l i t y c o e f f i c i e n t r ^ ^ f o r t h e l e n g t h e n e d t e s t i s e s t i m a t e d a s . 8 5 . W h i l e t h e o r e t i c a l l y we c o u l d m a k e t h e i n d i v i d u a l t e s t s a s r e l i a b l e a s we l i k e b y i n c r e a s i n g t h e i r l e n g t h , t h e i r r e d u c e d r e l i a b i l i t y w a s c o n s i d e r e d a c c e p t a b l e b e c a u s e o f t h e m o r e 6 7 s u b s t a n t i a l r e l i a b i l i t y o f t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s c o r r e c t s c o r e s . C e r t a i n l y t h e r e l i a b i l i t y o f t h e s e t o t a l s c o r e s o f t h e DTMS m a y b e s e e n a s m o r e t h a n a c c e p t a b l e f o r a c l a s s r o o m t e s t i n t e n d e d f o r u s e w i t h o t h e r m e a s u r e s t o d e t e r m i n e s t u d e n t p e r f o r m a n c e . Mean Scores T a b l e 5 ( p a g e 4 6 ) d i s p l a y s t h e f u n c t i o n s t a u g h t a t e a c h g r a d e . E a c h f u n c t i o n w a s t e s t e d a l o n g t w o d i s t i n c t s k i l l a r e a s : m a t h e m a t i c s f a c t s a n d m a t h e m a t i c s c o m p u t a t i o n s k i l l s . I t w a s h y p o t h e s i z e d t h a t t h e m o n t h l y m e a n t o t a l c o r r e c t s c o r e s i n e a c h o f t h e s k i l l a r e a s a t e a c h g r a d e l e v e l w o u l d i n c r e a s e o v e r t h e f o u r t e s t i n g s e s s i o n s . T h u s a t t h e G r a d e 2 l e v e l t h e m e a n o f t h e s u m s o f t h e a d d i t i o n f a c t s c o r r e c t a n d t h e s u b t r a c t i o n f a c t s c o r r e c t f o r e a c h t e s t i n g s e s s i o n w o u l d b e s h o w n t o i n c r e a s e o v e r t h e p r e v i o u s s e s s i o n ' s s c o r e . T h e s a m e c o u l d b e s a i d f o r t h e m e a n o f t h e s u m s o f t h e G r a d e 2 a d d i t i o n c o m p u t a t i o n a n d s u b t r a c t i o n c o m p u t a t i o n s k i l l s s c o r e s a n d f o r t h e m e a n s o f t h e s u m s o f t h e f u n c t i o n s t e s t e d a l o n g e a c h s k i l l a r e a a t e a c h g r a d e l e v e l . T h e s e i n c r e a s e d s c o r e s w o u l d r e p r e s e n t a n i n c r e a s e i n t h e s u m o f t h e s c o r e s o f t h e f u n c t i o n s t e s t e d i n e a c h s k i l l a r e a a t e a c h g r a d e l e v e l b u t w o u l d n o t n e c e s s a r i l y i n d i c a t e a s i g n i f i c a n t i n c r e a s e i n e a c h f u n c t i o n ' s s c o r e s . T a b l e s 1 3 , 1 4 a n d 1 5 c o n t a i n t h e m o n t h l y m e a n s c o r e s a t e a c h g r a d e l e v e l f o r ' t o t a l m a t h e m a t i c s f a c t s c o r r e c t 1 a n d " t o t a l m a t h e m a t i c s c o m p u t a t i o n s c o r r e c t ' . T h e y r e v e a l t h a t i n a l l i n s t a n c e s , a t e a c h g r a d e l e v e l m e a n s c o r e s i n c r e a s e d s e s s i o n b y 68 TABLE 13 THE MEAN, THE STANDARD DEVIATION AND THE NUMBER OF VALID CASES OF THE TOTAL MATHEMATICS FACTS CORRECT SCORES AND THE TOTAL MATHEMATICS COMPUTATION SKILLS CORRECT SCORES AT GRADE 2 IN TESTING SESSIONS 1 THROUGH 4. T o t a l M a t h e m a t i c s F a c t s C o r r e c t ( N . B . : T o t a l T e s t i n g T i m e P e r S e s s i o n = 4 m i n . ) T e s t i n g S e s s i o n 1 2 3 4 M e a n 1 6 . 2 1 9 . 2 2 0 . 3 2 1 . 5 S t d . D e v . 8 . 3 9 . 0 9 . 8 1 0 . 5 T o t a l M a t h e m a t i c s C o m p u t a t i o n S k i l l s C o r r e c t ( N . B . : T o t a l T e s t i n g T i m e P e r S e s s i o n = 20 m i n . ) T e s t i n g S e s s i o n 1 2 3 4 M e a n 1 3 . 9 1 9 . 2 2 1 . 8 2 3 . 7 S t d . D e v . 1 0 . 4 1 1 . 8 1 3 . 0 1 3 . 6 V a l i d C a s e s * 2 8 6 3 0 5 3 0 8 3 0 4 * N u m b e r o f v a l i d c a s e s e q u a l s e a c h s e s s i o n . n u m b e r o f s t u d e n t s w r i t i n g t e s t a t 69 TABLE 14 THE MEAN, THE STANDARD DEVIATION AND THE NUMBER OF VALID CASES OF THE TOTAL MATHEMATICS FACTS CORRECT SCORES AND THE TOTAL MATHEMATICS COMPUTATION SKILLS CORRECT SCORES AT GRADE 3 IN TESTING SESSIONS 1 THROUGH 4. T o t a l M a t h e m a t i c s F a c t s C o r r e c t ( N . B . : T o t a l T e s t i n g T i m e P e r S e s s i o n = 4 m i n . ) T e s t i n g S e s s i o n 1 2 3 4 M e a n 36.7 40.2 42.2 43.0 S t d . D e v . 13.6 15.5 16.4 16.5 T o t a l M a t h e m a t i c s C o m p u t a t i o n S k i l l s C o r r e c t ( N . B . : T o t a l T e s t i n g T i m e P e r S e s s i o n = 20 m i n . ) T e s t i n g S e s s i o n 1 2 3 4 M e a n 30.1 43.1 47.8 50.4 S t d . D e v . 13.8 18.6 20.0 21.8 V a l i d C a s e s * 322 319 327 327 * N u m b e r o f v a l i d c a s e s e q u a l s n u m b e r o f s t u d e n t s w r i t i n g t e s t a t e a c h s e s s i o n . 7 0 TABLE 15 THE MEAN, THE STANDARD DEVIATION AND THE NUMBER OF VALID CASES OF THE TOTAL MATHEMATICS FACTS CORRECT SCORES AND THE TOTAL MATHEMATICS COMPUTATIONS SKILLS CORRECT SCORES AT GRADE 4 IN TESTING SESSIONS 1 THROUGH 4. T o t a l M a t h e m a t i c s F a c t s C o r r e c t ( N . B . : T o t a l T e s t i n g T i m e = 4 m i n . ) T e s t i n g S e s s i o n 1 2 3 4 M e a n 7 4 . 5 7 9 . 1 8 4 . 6 8 7 . 7 S t d . D e v . 2 7 . 3 2 6 . 2 2 9 . 1 3 1 . 6 T o t a l M a t h e m a t i c s C o m p u t a t i o n S k i l l s C o r r e c t ( N . B . : T o t a l T e s t i n g T i m e P e r S e s s i o n = 2 0 m i n . ) T e s t i n g S e s s i o n 1 2 3 4 M e a n 3 0 . 2 3 7 . 0 4 0 . 4 4 4 . 4 S t d . D e v . 1 4 . 1 1 5 . 7 1 7 . 9 1 8 . 7 V a l i d C a s e s * 2 8 3 2 8 1 2 8 4 2 7 9 * N u m b e r o f v a l i d c a s e s e q u a l s n u m b e r o f s t u d e n t s w r i t i n g t e s t a t e a c h s e s s i o n . 7 1 s e s s i o n . A r e p e a t e d m e a s u r e s d e s i g n , a n a l y s i s o f v a r i a n c e w a s p e r f o r m e d o n t h e s e m e a n s t o d e t e r m i n e i f t h e o b s e r v e d d i f f e r e n c e s b e t w e e n t h e s e m e a n s r e p r e s e n t e d a c h a n c e o c c u r r e n c e o r a t r e a t m e n t e f f e c t . I n a l l c a s e s t h e d i f f e r e n c e s i n t h e m e a n s a r e s e e n t o b e s i g n i f i c a n t a t . 0 0 1 ( T a b l e s D - 1 5 t o D - 1 7 ) . T u k e y ' s H S D ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) t e s t w a s u s e d t o d e t e r m i n e i f t h e d i f f e r e n c e b e t w e e n t h e m e a n s o f s u c c e s s i v e t e s t s w a s s i g n i f i c a n t . T h e r e s u l t s o f t h i s a n a l y s i s ( r e p o r t e d i n T a b l e s D - 1 5 t o D - 1 7 a n d s u m m a r i z e d i n T a b l e 1 6 f o r t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s c o r r e c t ) i n d i c a t e t h a t w i t h t w o e x c e p t i o n s ( G r a d e 2, S e s s i o n s 2 a n d 3 m e a n s a n d G r a d e 3, S e s s i o n s 3 a n d 4 m e a n s ) t h e d i f f e r e n c e b e t w e e n s e s s i o n a l m e a n s f o r t o t a l m a t h e m a t i c s f a c t s c o r r e c t w e r e s i g n i f i c a n t l y d i f f e r e n t w h i l e a l l t h e b e t w e e n s e s s i o n m e a n s f o r t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s c o r r e c t w e r e s i g n i f i c a n t l y d i f f e r e n t . I t s h o u l d b e n o t e d t h a t w h i l e a l l t h e o b s e r v e d b e t w e e n s e s s i o n i n c r e a s e s w e r e n o t l a r g e e n o u g h t o b e s i g n i f i c a n t b y t h e m s e l v e s , t h e m e a n s c o r e s d i d , a s p r e d i c t e d , i n c r e a s e s e s s i o n b y s e s s i o n . A l t h o u g h s u c h a t r e n d w a s n o t a n t i c i p a t e d f o r t h e i n d i v i d u a l t e s t s , b e c a u s e i t w a s . n o t e x p e c t e d t h a t i n s t r u c t i o n w o u l d b e g i v e n i n e a c h f u n c t i o n b e t w e e n e a c h s e s s i o n a c o m p a r i s o n o f t h e a v e r a g e s c o r e s o b t a i n e d o n e a c h o f t h e f u n c t i o n s t e s t e d a t e a c h g r a d e l e v e l ( T a b l e s D - 1 0 t o D - 1 4 ) , i n d i c a t e s t h a t w i t h o n e e x c e p t i o n ( G r a d e 3 a d d i t i o n f a c t s i n t h e f o u r t h t e s t i n g s e s s i o n ) m e a n s c o r e s i n c r e a s e d o v e r t h e p r e v i o u s s e s s i o n ' s s c o r e s . TABLE 16 SIGNIFICANTLY DIFFERENT MEANS* AS DETERMINED USING TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) TEST ON THE SESSIONAL MEAN TOTAL MATHEMATICS FACT SCORES (TOTFAC) AND THE SESSIONAL MEAN TOTAL COMPUTATION SKILLS SCORES (TOTCOMP). * ' X ' i n d i c a t e s m e a n s w h i c h a r e s i g n i f i c a n t l y d i f f e r e n t . G r a d e 2 T o t f a c T o t c o m p P a i r e d S e s s i o n a l M e a n s S e s s i o n s 1 & 2 2 & 3 3 & 4 X X X X G r a d e 3 T o t f a c T o t c o m p X X X X G r a d e 4 T o t f a c T o t c o m p X X X X X X A r e p e a t e d m e a s u r e s d e s i g n , a n a l y s i s o f v a r i a n c e w a s p e r f o r m e d o n t h e s e m e a n s . I n a l l c a s e s t h e d i f f e r e n c e s i n t h e m e a n s a r e s e e n t o b e s i g n i f i c a n t a t . 0 0 1 ( T a b l e s D - 1 8 t o D - 2 6 ) . T u k e y ' s H S D t e s t ( d e t a i l e d i n J T a b l e s D - 1 8 t o D - 2 6 a n d s u m m a r i z e d i n T a b l e 1 7 ) r e v e a l s t h a t i n m o s t c a s e s ( 3 4 o u t o f 5 3 ) t h e d i f f e r e n c e b e t w e e n s u c c e s s i v e m e a n s w a s s i g n i f i c a n t . I t s h o u l d b e n o t e d t h a t i n t h e i n s t a n c e w h e r e t h e m e a n d i d n o t i n c r e a s e , ( G r a d e 3 a d d i t i o n f a c t s i n t h e f o u r t h t e s t i n g s e s s i o n ) i t i s n o t s i g n i f i c a n t l y d i f f e r e n t f r o m t h e m e a n o f t h e p r e v i o u s t e s t i n g s e s s i o n . Mean Scores; Discussion T h e s t a t e d p u r p o s e o f t h i s s t u d y w a s t o d e v e l o p m e a s u r e s o f m a t h e m a t i c s c o m p u t a t i o n s k i l l s a n d m e a s u r e s o f f a c i l i t y w i t h m a t h e m a t i c s f a c t s f o r e a c h o f G r a d e s 2, 3 a n d 4. E a c h o f t h e s e t e s t s ( T h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ) w a s d e s i g n e d t o r e f l e c t t h e r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m f o r o n e o f t h e f o u r b a s i c f u n c t i o n s : a d d i t i o n , s u b t r a c t i o n , m u l t i p l i c a t i o n a n d d i v i s i o n . I t w a s h y p o t h e s i z e d t h a t t h e s e t e s t s w o u l d m e a s u r e c h a n g e s i n t h e p e r f o r m a n c e o f i n d i v i d u a l s t u d e n t s a n d i n t h e p e r f o r m a n c e o f g r o u p s o f s t u d e n t s o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e . F u r t h e r m o r e , i t w a s a n t i c i p a t e d t h a t t h e n e t e f f e c t o f c h a n g e s i n t h e p e r f o r m a n c e o f i n d i v i d u a l s t u d e n t s w o u l d r e s u l t i n a n i n c r e a s e i n t h e g r o u p m e a n s f o r e a c h o f t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t h e t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s c o r r e c t . 74 TABLE 17 SIGNIFICANTLY DIFFERENT MEANS* AS DETERMINED USING TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) TEST ON THE SESSIONAL MEAN SCORES OF THE FUNCTIONS AND SKILLS TESTED AT EACH GRADE. * ' X ' i n d i c a t e s m e a n s w h i c h a r e s i g n i f i c a n t l y d i f f e r e n t . P a i r e d S e s s i o n a l M e a n s S e s s i o n s 1 & 2 2 & 3 3 & 4 G r a d e 2 A d d . F a c t s X X A d d . C o m p . X X X S u b t . F a c t s X X S u b t . C o m p . X X G r a d e 3 A d d . F a c t s X X A d d . C o m p . X X S u b t . F a c t s X S u b t . C o m p . X M u l t . F a c t s X X M u l t . C o m p . X X X G r a d e 4 A d d . F a c t s X A d d . C o m p . X X S u b t . F a c t s X S u b t . C o m p . X X X M u l t . F a c t s X M u l t . C o m p . X D i v . F a c t s X X X D i v . C o m p . X X 75 T h a t , i n f a c t , i s w h a t h a p p e n e d . I n a l l i n s t a n c e s , a t e a c h g r a d e l e v e l t o t a l m e a n s c o r e s i n c r e a s e d s e s s i o n b y s e s s i o n . F u r t h e r m o r e , a s r e p o r t e d i n T a b l e 1 6 , p r o c e d u r e s u t i l i z i n g a n a n a l y s i s o f v a r i a n c e f o r r e p e a t e d m e a s u r e s a n d T u k e y ' s HSD ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) t e s t d e t e r m i n e d t h a t w i t h t w o e x c e p t i o n s t h e d i f f e r e n c e s b e t w e e n s e s s i o n a l m e a n s f o r t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s c o r r e c t w e r e s i g n i f i c a n t . C l e a r l y t h e DTMS m a y , a s h y p o t h e s i z e d , b e s e e n t o m e a s u r e c h a n g e s i n t h e p e r f o r m a n c e o f i n d i v i d u a l s t u d e n t s a n d i n t h e p e r f o r m a n c e o f g r o u p s o f s t u d e n t s o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e . F u r t h e r m o r e , a s d i s c u s s e d e a r l i e r , s u c h c h a n g e s m a y n o t b e m e a s u r e d b y a s t a n d a r d i z e d a c h i e v e m e n t t e s t s u c h a s t h e C a n a d i a n T e s t s o f B a s i c S k i l l s ( C T B S ) . T r e n d A n a l y s i s T r e n d o r r e g r e s s i o n a n a l y s i s , a n a p p l i c a t i o n o f a n a l y s i s o f v a r i a n c e , w a s p e r f o r m e d t o p r o v i d e i n f o r m a t i o n a b o u t t h e n a t u r e o f t h e i n c r e a s e i n g r o u p m e a n s . E s s e n t i a l l y t r e n d a n a l y s i s c o r r e l a t e s t h e g r o u p m e a n s w i t h s e t s o f c o e f f i c i e n t s w h i c h d e s c r i b e v a r i o u s c u r v e s a n d t e s t s t h e s i g n i f i c a n c e o f t h e o b t a i n e d c o r r e l a t i o n s . W i t h f o u r g r o u p m e a n s , o r l e v e l s o f t h e i n d e p e n d e n t v a r i a b l e , i t w a s a p p r o p r i a t e t o t e s t t h e s i g n i f i c a n c e o f t h e f i t o f t h e l i n e a r , q u a d r a t i c a n d c u b i c c u r v e s . W h e n a s i g n i f i c a n t t r e n d h a d b e e n i d e n t i f i e d , t h e c o r r e l a t i o n ' r ' b e t w e e n t h e s u m o f s q u a r e s f o r t h e s i g n i f i c a n t t r e n d a n d t h e t o t a l s u m o f s q u a r e s w a s c a l c u l a t e d . T h i s c o r r e l a t i o n c o e f f i c i e n t 76 s q u a r e d , r l , i s t h e p r o p o r t i o n o f t h e t o t a l v a r i a t i o n a c c o u n t e d f o r b y t h a t s i g n i f i c a n t t r e n d . A s a n i l l u s t r a t i v e e x a m p l e T a b l e D-27 p r o v i d e s a d e t a i l e d s u m m a r y o f t h e c o m p u t a t i o n r e q u i r e d f o r a t r e n d a n a l y s i s u s i n g o r t h o g o n a l p o l y n o m i a l s w i t h t h e d a t a f o r G r a d e 2 t o t a l m a t h e m a t i c s f a c t s c o r r e c t . I n a d d i t i o n , i t i l l u s t r a t e s t h e c a l c u l a t i o n o f r x . L e s s d e t a i l e d s u m m a r i e s o f t h e t e s t s f o r t r e n d a n d t h e c a l c u l a t i o n o f r x a r e p r o v i d e d f o r t h e b a l a n c e o f t h e s e s s i o n a l m e a n s i n T a b l e s D-28 t o D-39. C u r v e s o f t h e s i g n i f i c a n t t r e n d s f o r t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s c o r r e c t w e r e p l o t t e d i n F i g u r e s E - l t o E-6. T h e c o o r d i n a t e s u s e d w e r e g e n e r a t e d f r o m t h e a p p r o p r i a t e r e g r e s s i o n e q u a t i o n s a s o u t l i n e d i n A p p e n d i x D, S e c t i o n V . T a b l e 18 s u m m a r i z e s t h e r e s u l t s o f t h e t r e n d a n a l y s i s f o r t h e s e s s i o n a l m e a n t o t a l m a t h e m a t i c s f a c t s c o r r e c t s c o r e s a n d t h e s e s s i o n a l m e a n t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s c o r r e c t s c o r e s a t e a c h g r a d e . I n a l l c a s e s t h e l i n e a r t r e n d i s s e e n t o b e s i g n i f i c a n t a n d t o a c c o u n t f o r a l a r g e p e r c e n t a g e o f t h e t o t a l v a r i a t i o n . I t i s a l s o r e a d i l y a p p a r e n t t h a t a l t h o u g h t h e q u a d r a t i c t r e n d i s s i g n i f i c a n t f o r f i v e o f t h e s i x g r o u p s o f m e a n s , i t d o e s n o t a c c o u n t f o r a v e r y l a r g e p e r c e n t a g e o f t h e t o t a l v a r i a t i o n a n d a s c a n b e s e e n i n F i g u r e s E - l , E-2, E-3, E-4 a n d E-5 r e s u l t s i n a v e r y m o d e s t q u a d r a t i c c u r v e . 7 7 TABLE 18 SIGNIFICANT TRENDS* FOUND FOR THE SESSIONAL MEAN TOTAL MATHEMATICS FACT SCORES (TOTFAC) AND THE SESSIONAL MEAN TOTAL COMPUTATION SKILLS SCORES (TOTCOMP) BY TREND ANALYSIS USING ORTHOGONAL POLYNOMIALS. S i g n i f i c a n t T r e n d s ( r * ) L i n e a r Q u a d r a t i c C u b i c G r a d e 2 T o t f a c . 9 3 .05 T o t c o m p .94 .05 G r a d e 3 T o t f a c .92 .08 T o t c o m p .87 .12 . 0 1 G r a d e 4 T o t f a c . 9 9 T o t c o m p .97 .02 .01 S i g n i f i c a n t t r e n d s a r e i n d i c a t e d b y t h e c o r r e l a t i o n c o e f f i c i e n t s q u a r e d ( r x ) w h i c h i s t h e p r o p o r t i o n o f t h e t o t a l v a r i a t i o n a c c o u n t e d f o r b y t h a t s i g n i f i c a n t t r e n d . 78 S i m i l a r l y , t h e c u b i c t r e n d i s s e e n t o a c c o u n t f o r o n l y o n e p e r c e n t o f t h e v a r i a t i o n i n t w o o f t h e s i x g r o u p m e a n s . A s c a n b e s e e n i n F i g u r e s E-4 a n d E-5 t h i s s l i g h t c u b i c t r e n d p r o d u c e s a c u r v e n o t t o o d i s s i m i l a r f r o m t h e a s s o c i a t e d q u a d r a t i c c u r v e . I n f a c t , t h e c u b i c t r e n d p r o d u c e s a c u r v e w h i c h , i f v i e w e d a l o n e , w o u l d b e d e s c r i b e d a s q u a d r a t i c . T a b l e 19 s h o w s t h e l i n e a r t r e n d m o s t d o m i n a n t f o r t h e s e s s i o n a l m e a n s c o r e s o f t h e f u n c t i o n s a n d s k i l l s t e s t e d a t e a c h g r a d e . H o w e v e r , 25% o f t h e v a r i a t i o n i n t h e G r a d e 3 a d d i t i o n f a c t s m e a n s , 21% o f t h e v a r i a t i o n i n t h e G r a d e 3 s u b t r a c t i o n c o m p u t a t i o n s m e a n s , a n d 18% o f t h e v a r i a t i o n i n t h e G r a d e 4 s u b t r a c t i o n c o m p u t a t i o n s m e a n s i s s e e n t o b e a c c o u n t e d f o r b y t h e q u a d r a t i c t r e n d . ( R e c a l l t h a t t h e G r a d e 3 a d d i t i o n f a c t s m e a n f o r t h e f o u r t h t e s t i n g s e s s i o n d r o p p e d b e l o w t h e m e a n o f t h e t h i r d s e s s i o n ; i . e . 19.9 t o 19.3. O b v i o u s l y t h i s w o u l d c o n t r i b u t e t o t h e s i g n i f i c a n t q u a d r a t i c t r e n d . ) S i x p e r c e n t o f t h e v a r i a t i o n i n t h e G r a d e 3 a d d i t i o n f a c t s s c o r e s a n d 6% o f t h e v a r i a t i o n i n t h e G r a d e 4 a d d i t i o n f a c t s s c o r e s c a n b e a c c o u n t e d f o r b y t h e c u b i c t r e n d . I n t e r e s t i n g l y , t h e G r a d e 4 a d d i t i o n f a c t s m e a n s a r e n o t s i g n i f i c a n t l y a f f e c t e d b y t h e q u a d r a t i c t r e n d . Trend A n a l y s i s ; Discussion T r e n d a n a l y s i s w a s p e r f o r m e d o n t h e n o r m i n g d a t a t o p r o v i d e i n f o r m a t i o n a b o u t t h e n a t u r e o f t h e i n c r e a s e i n g r o u p m e a n s . T h a t i s , we w i s h e d t o d e t e r m i n e w h e t h e r t h e g r o u p m e a n s i n c r e a s e d s i g n i f i c a n t l y i n a l i n e a r f a s h i o n o r i f s i g n i f i c a n t d e v i a t i o n s f r o m l i n e a r i t y e x i s t e d . 79 TABLE 19 SIGNIFICANT TRENDS* FOUND FOR THE SESSIONAL MEAN SCORES OF THE FUNCTIONS AND SKILLS TESTED AT EACH GRADE BY TREND ANALYSIS USING ORTHOGONAL POLYNOMIALS. S i g n i f i c a n t T r e n d s ( r * " ) L i n e a r Q u a d r a t i c C u b i c G r a d e 2 A d d . F a c t s .93 .06 A d d . C o m p u t a t i o n s .97 .03 S u b t . F a c t s .95 S u b t . C o m p u t a t i o n s .88 .12 G r a d e 3 A d d . F a c t s A d d . C o m p u t a t i o n s S u b t . F a c t s S u b t . C o m p u t a t i o n s M u l t . F a c t s M u l t . C o m p u t a t i o n s .69 .25 .06 .88 .11 .95 .77 .21 .96 .03 1.00 G r a d e 4 A d d . F a c t s .94 .06 A d d . C o m p u t a t i o n s .82 .18 S u b t . F a c t s .99 S u b t . C o m p u t a t i o n s .98 M u l t . F a c t s .98 M u l t . C o m p u t a t i o n s .91 .08 D i v . F a c t s .96 .04 D i v . C o m p u t a t i o n s .96 .04 • S i g n i f i c a n t t r e n d s a r e i n d i c a t e d b y t h e c o r r e l a t i o n c o e f f i c i e n t s q u a r e d C r * ) w h i c h i s t h e p r o p o r t i o n o f t h e t o t a l v a r i a t i o n a c c o u n t e d f o r b y t h a t s i g n i f i c a n t t r e n d . 8 0 R e c a l l t h a t i n C h a p t e r 2 i t w a s n o t e d t h a t n o r m a t i v e d a t a f o r m o s t n o r m - r e f e r e n c e d t e s t s w e r e c o l l e c t e d d u r i n g o n e s h o r t i n t e r v a l d u r i n g t h e s c h o o l y e a r a n d t h a t f o r t i m e s w h e n n o e m p i r i c a l d a t a i s a v a i l a b l e d e r i v e d s c o r e s h a v e b e e n g e n e r a t e d t h r o u g h i n t e r p o l a t i o n a n d e x t r a p o l a t i o n . T h e s e p r o c e d u r e s a s s u m e d t h a t t h e n u m b e r o f q u e s t i o n s a n s w e r e d c o r r e c t l y w e r e a l i n e a r f u n c t i o n o f t i m e o v e r t h e s c h o o l y e a r , a n a s s u m p t i o n T a l l m a d g e m a i n t a i n s " i s t e n u o u s a t b e s t a n d m a y b e s i g n i f i c a n t l y i n e r r o r " ( p . 3 5 7 ) . F u r t h e r m o r e , T a l l m a d g e n o t e d t h a t e x t e n d i n g t h e l i n e a r g r o w t h f u n c t i o n s c r o s s t h e s u m m e r m o n t h s i s m o s t q u e s t i o n a b l e . T a l l m a d g e ' s r e s e r v a t i o n s a b o u t t h e l i n e a r n a t u r e o f l e a r n i n g a p p e a r t o b e s u p p o r t e d b y t h e d a t a r e p o r t e d i n T a b l e 1 8 . H e r e i t c a n b e s e e n t h a t a l t h o u g h t h e l i n e a r t r e n d a c c o u n t s f o r t h e m o s t v a r i a t i o n i n t h e s e s s i o n a l m e a n t o t a l m a t h e m a t i c s f a c t s c o r r e c t s c o r e s a n d t h e s e s s i o n a l m e a n t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s s c o r e s a t e a c h g r a d e l e v e l , t h e f i t o f t h e l i n e a r r e l a t i o n s h i p w a s s h o w n t o b e s a t i s f a c t o r y f o r o n l y o n e s e t o f m e a n s ( G r a d e 4, T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s ) . F o r t h r e e o f t h e s e t s o f m e a n s a q u a d r a t i c c u r v e w a s s e e n t o p r o v i d e t h e b e s t f i t w h i l e t h e c u b i c t r e n d w a s f o u n d m o s t a p p r o p r i a t e f o r t w o . Cumulative Percentage Polygons (Ogives) A n e x a m i n a t i o n o f t h e C u m u l a t i v e P e r c e n t a g e P o l y g o n ( O g i v e s ) o f t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l c o m p u t a t i o n s s k i l l s c o r r e c t o f e a c h g r a d e l e v e l ( F i g u r e s E - 7 t o E - 1 2 i n c l u s i v e ) p r o v i d e s a n i n t e r e s t i n g i n s i g h t i n t o t h e n a t u r e o f t h e 8 1 r e l a t i o n s h i p b e t w e e n t o t a l s c o r e a n d t h e n u m b e r o f s t u d e n t s a c h i e v i n g t h a t s c o r e . T y p i c a l l y , t h e g r e a t e s t i m p r o v e m e n t i n a v e r a g e s c o r e s f r o m s e s s i o n t o s e s s i o n w a s a c h i e v e d b y t h o s e s t u d e n t s w h o s e s c o r e s f e l l i n t h e a v e r a g e r a n g e . F o r e x a m p l e , a s c a n b e s e e n f r o m F i g u r e E - 7 , a p p r o x i m a t e l y 8 7 . 5 % o f t h e s t u d e n t s s c o r e d 2 5 o r l e s s i n S e s s i o n 1 . I n S e s s i o n 2 a p p r o x i m a t e l y 8 0 % s c o r e d 2 5 o r l e s s . T h i s f i g u r e d r o p p e d t o 7 1 % f o r S e s s i o n 3 a n d a p p r o x i m a t e l y 6 8 % f o r S e s s i o n 4. S t u d e n t s w h o s c o r e d a t e i t h e r e n d o f t h e s p e c t r u m , t h a t i s , s t u d e n t s w h o w e r e e i t h e r h i g h a c h i e v e r s o r l o w a c h i e v e r s d i d n o t s h o w a s m u c h c h a n g e f r o m t e s t i n g s e s s i o n t o t e s t i n g s e s s i o n . I n d e e d , i n s o m e i n s t a n c e s t h e c u m u l a t i v e p e r c e n t a g e i s s e e n t o i n c r e a s e a t l o w l e v e l s o f a c h i e v e m e n t f r o m a t l e a s t o n e s e s s i o n t o t h e n e x t ( i . e . F i g u r e s E - 9 a n d E - l l , G r a d e s 3 a n d 4 T o t a l M a t h e m a t i c s F a c t s C o r r e c t a n d T a b l e s E - 1 0 a n d E - 1 2 , G r a d e s 3 a n d 4 T o t a l M a t h e m a t i c s C o m p u t a t i o n s C o r r e c t ) i n d i c a t i n g a n i n c r e a s e i n t h e n u m b e r o f l o w a c h i e v e r s f r o m o n e s e s s i o n t o t h e n e x t . I n a d d i t i o n , o n F i g u r e E - l l G r a d e 4 T o t a l M a t h e m a t i c s F a c t s C o r r e c t , t h e c u m u l a t i v e p e r c e n t a g e f o r S e s s i o n 2 i s s e e n t o i n c r e a s e a b o v e t h e S e s s i o n 1 t o t a l s a t t h e h i g h e r a c h i e v e m e n t l e v e l s . O g i v e s o f t h e i n d i v i d u a l t e s t r e s u l t s c o u l d b e s h o w n t o b e s i m i l a r t o t h o s e o f t h e t o t a l m a t h e m a t i c s s c o r e s i n t h a t t h e g r e a t e s t i m p r o v e m e n t i n a v e r a g e s c o r e s w a s a c h i e v e d b y t h o s e s t u d e n t s w h o f e l l i n t h e a v e r a g e r a n g e w h i l e t h o s e w h o s c o r e d a t e i t h e r e x t r e m e s h o w e d l e s s c h a n g e f r o m s e s s i o n t o s e s s i o n . A s 82 w o u l d b e e x p e c t e d i n s o m e i n s t a n c e s t h e c u m u l a t i v e p e r c e n t a g e c o u l d b e s e e n t o i n c r e a s e a t l o w l e v e l s o f a c h i e v e m e n t f r o m o n e s e s s i o n t o t h e n e x t . C u m u l a t i v e P e r c e n t a g e P o l y g o n s ; D i s c u s s i o n T h e c u m u l a t i v e p e r c e n t a g e p o l y g o n s f o r t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t s c o r e s a n d t h e t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s c o r r e c t s c o r e s f o r a l l g r a d e s i n d i c a t e t h a t t h e t e s t s m e a s u r e l i t t l e c h a n g e i n t h e p e r f o r m a n c e o f v e r y h i g h o r v e r y l o w s c o r i n g s t u d e n t s . T h a t m i n i m a l c h a n g e w a s e v i d e n c e d b y h i g h p e r f o r m i n g s t u d e n t s m a y v e r y w e l l b e a t t r i b u t e d t o t h e f a c t t h a t t h e y h a v e m a s t e r e d t h e g r a d e l e v e l r e q u i r e m e n t s a n d a n y v a r i a t i o n r e c o r d e d , i s t h e r e s u l t o f v a r i a b l e s o t h e r t h a n t h o s e b e i n g t e s t e d ( e . g . w r i t i n g s p e e d , a t t i t u d e , e t c . ) . L o w p e r f o r m i n g s t u d e n t s , o n t h e o t h e r h a n d , m a y b e s o l a c k i n g i n t h e s k i l l s n o r m a l l y m a s t e r e d a t g r a d e l e v e l s b e l o w t h e i r p l a c e m e n t t h a t t h e i r a b i l i t y t o r e g i s t e r s u b s t a n t i a l i m p r o v e m e n t i s i m p a i r e d . I n a n y c a s e , t h e s u g g e s t i o n i s , t h a t f o r t h o s e s t u d e n t s , g r a d e l e v e l t e s t i n g i s n o t a p p r o p r i a t e t o m e a s u r e p r o g r e s s a n d t h a t i t m a y b e m o r e i n f o r m a t i v e t o t e s t h i g h s c o r i n g s t u d e n t s a t a g r a d e a b o v e t h e i r c u r r e n t p l a c e m e n t a n d l o w s c o r i n g s t u d e n t s a t a g r a d e b e l o w . S u m m a r y T h i s c h a p t e r b e g a n w i t h a d i s c u s s i o n o f r e l i a b i l i t y . F o r s p e e d e d t e s t s , s u c h a s t h e D T M S , t e s t - r e t e s t p r o c e d u r e s w e r e f o u n d m o s t a p p r o p r i a t e f o r t h i s p u r p o s e . T h e s e w e r e c a l c u l a t e d f o r t h e DTMS a n d t h e i r s i g n i f i c a n c e w a s d i s c u s s e d w i t h i n t h e d e s i g n 83 l i m i t a t i o n s o f t h e t e s t . T h e m e a n m o n t h l y s c o r e s o f t h e DTMS w e r e e x a m i n e d a n d f o u n d , w i t h o n e e x c e p t i o n , t o i n c r e a s e a s p r e d i c t e d . T h e s i g n i f i c a n c e o f t h e s e i n c r e a s e s w a s d e t e r m i n e d u s i n g a n a n a l y s i s o f v a r i a n c e p r o c e d u r e f o r r e p e a t e d m e a s u r e s a n d b y c o m p a r i n g d i f f e r e n c e s b e t w e e n m e a n s w i t h a c a l c u l a t i o n o f T u k e y ' s H S D ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) . T r e n d a n a l y s i s w a s u s e d t o d e t e r m i n e a b e s t f i t t i n g c u r v e f o r t h e m e a n s . F i n a l l y , a n e x a m i n a t i o n o f t h e c u m u l a t i v e p e r c e n t a g e p o l y g o n s f o r t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l c o m p u t a t i o n s s k i l l s c o r r e c t p r o v i d e d a n i n t e r e s t i n g i n s i g h t i n t o t h e r e l a t i o n s h i p b e t w e e n t o t a l s c o r e a n d t h e n u m b e r o f s t u d e n t s a c h i e v i n g t h a t s c o r e . 84 CHAPTER 6: SUMMARY, CONCLUSIONS, RECOMMENDATIONS T h i s c h a p t e r b e g i n s w i t h a s u m m a r y o f t h e r e s e a r c h a n d e v e n t s w h i c h l e d t o t h e d e v e l o p m e n t a n d n o r m i n g o f t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ( D T M S ) . I t i n c l u d e s a b r i e f o v e r v i e w o f t h e d a t a a n a l y s i s r e p o r t e d i n C h a p t e r 5. T h i s s u m m a r y i s f o l l o w e d b y a b r i e f s t a t e m e n t o f t h e c o n c l u s i o n s a r r i v e d a t a s a r e s u l t o f t h e d a t a a n a l y s i s . I t c o n c l u d e s w i t h r e c o m m e n d a t i o n s a n d c a u t i o n s c o n c e r n i n g u s e o f t h e D T M S . Overview T h e r e s u l t s o f c o m m e r c i a l n o r m - r e f e r e n c e d a c h i e v e m e n t t e s t s h a v e l o n g b e e n u s e d i n t h e s e l e c t i o n a n d a s s e s s m e n t o f c l a s s r o o m m a t e r i a l s a n d t h e g e n e r a t i o n a n d a s s e s s m e n t o f i n d i v i d u a l i z e d e d u c a t i o n a l p r o g r a m s ( T i n d a l , F u c h s , F u c h s , S h i n n , D e n o a n d G e r m a n n , 1985) . C a l t a g i r o n e a n d G l o v e r (1985) i d e n t i f y s e v e r a l s h o r t c o m i n g s i n h e r e n t i n t h i s r e l i a n c e . M o s t i m p o r t a n t l y , t h e y n o t e " t h e t y p i c a l l y p o o r c o n g r u e n c e b e t w e e n t e s t i t e m s a n d c u r r i c u l u m c o n t e n t . . . a n d t h e r e d u c e d d i a g n o s t i c u t i l i t y o f t h e s e t e s t s a s s o c i a t e d w i t h t h e i r l i m i t e d i t e m c o n t e n t . " (p.356) I n C h a p t e r 3 a n i t e m a n a l y s i s o f t h e M a t h e m a t i c s C o m p u t a t i o n s T e s t s o f t h e C a n a d i a n T e s t s o f B a s i c S k i l l s a n d t h e B . C . Q u i e t , t w o c o m m o n l y u s e d t e s t s o f s t u d e n t a c h i e v e m e n t i n D e l t a , d e m o n s t r a t e t h a t t h e s e t e s t s d o n o t a d e q u a t e l y r e f l e c t t h e m a t h e m a t i c s c o m p u t a t i o n s r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m f o r G r a d e s 2, 3 a n d 4. I n c o n t r a s t , a s d i s c u s s e d i n 8 5 C h a p t e r 4, t h e D e l t a T e s t s o f B a s i c S k i l l s ( D T M S ) w e r e c a r e f u l l y d e s i g n e d t o a c c u r a t e l y r e f l e c t t h e c o m p u t a t i o n s r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m a s i n t e r p r e t e d i n D e l t a . F u r t h e r m o r e , t h e e f f e c t o f t h e l a c k o f c o n g r u e n c e b e t w e e n t h e M a t h e m a t i c s C o m p u t a t i o n s T e s t o f t h e C a n a d i a n T e s t s o f B a s i c S k i l l s ( C T B S ) a n d t h e c o m p u t a t i o n s r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m w a s d e m o n s t r a t e d b y a d m i n i s t e r i n g b o t h t h e C o m p u t a t i o n s T e s t o f t h e C T B S a n d t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s ( D T M S ) t o t w o c l a s s e s o f G r a d e 4 s t u d e n t s a n d c o m p a r i n g t h e r e s u l t s . A n o t h e r s i g n i f i c a n t s h o r t c o m i n g i n h e r e n t i n t h e u s e o f s t a n d a r d i z e d a c h i e v e m e n t t e s t s f o r t h e a s s e s s m e n t o f p u p i l p r o g r e s s n o t e d b y C a l t a g i r o n e a n d G l o v e r ( 1 9 8 5 ) i s t h a t s t a n d a r d i z e d a c h i e v e m e n t t e s t s s e e m t o p r o v i d e l i t t l e m o r e t h a n a n o v e r v i e w o f a p u p i l ' s g e n e r a l a b i l i t i e s w h e r e c o m p a r i s o n s a r e m a d e t o a n o r m i n g p o p u l a t i o n w h i c h m a y h a v e l i t t l e i n c o m m o n w i t h t h e l o c a l s a m p l e . F u r t h e r m o r e , t h e y p o i n t o u t t h a t n o i n f o r m a t i o n i s p r o v i d e d w h i c h i n d i c a t e s t h e r a t e a t w h i c h s k i l l s a r e a c q u i r e d o r t h e r a t e a t w h i c h t h e l e a r n i n g o f s i m i l a r s k i l l s i s l i k e l y t o o c c u r . T h e D TMS a d d r e s s e d t h e s e l i m i t a t i o n s o f s t a n d a r d i z e d a c h i e v e m e n t s t e s t s . T h e D T M S t e s t s t w o f e a t u r e s o f t h e m a t h e m a t i c s c u r r i c u l u m : " m a t h f a c t s " a n d " c o m p u t a t i o n s k i l l s " . F o r t h e p u r p o s e o f t h i s s t u d y " m a t h f a c t s " w e r e d e f i n e d a s s i m p l e o n e s t e p o p e r a t i o n s t o w h i c h a n s w e r s a r e n o r m a l l y m e m o r i z e d . T h u s t h e a n s w e r t o 3 + 4, w o u l d b e a n a d d i t i o n f a c t a n d t h e a n s w e r t o 4 - 2 w o u l d b e a 8 6 s u b t r a c t i o n f a c t . T o s o l v e q u e s t i o n s s u c h a s 53 + 7 5 a n d 7 4 - 35 r e q u i r e s c o m p u t a t i o n s k i l l s . C o m p u t a t i o n s k i l l s c a n b e s e e n t o b e a c o m p o s i t e o f k n o w l e d g e o f " m a t h f a c t s " a n d k n o w l e d g e o f c e r t a i n m a t h e m a t i c s o p e r a t i o n s . T h e DTMS w e r e a d m i n i s t e r e d m o n t h l y f o r f o u r m o n t h s b e g i n n i n g i n J a n u a r y , 1 9 8 7 t o a p p r o x i m a t e l y 3 0 0 p u p i l s i n e a c h o f G r a d e s 2, 3 a n d 4 i n s e v e n s c h o o l s i n D e l t a . T h e d a t a r e s u l t i n g f r o m t h i s t e s t i n g w a s u s e d t o e s t a b l i s h m o n t h l y n o r m s f o r e a c h o f t h e t e s t s . T h e s i g n i f i c a n c e o f t h e d i f f e r e n c e s b e t w e e n t h e s e m o n t h l y m e a n s w a s e v a l u a t e d u s i n g a n a n a l y s i s o f v a r i a n c e , r e p e a t e d m e a s u r e s d e s i g n a n d T u k e y ' s H S D ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) T e s t . T h e r e s u l t s o f t h i s a n a l y s i s ( r e p o r t e d i n T a b l e s D - 1 5 t o D - 1 7 a n d s u m m a r i z e d i n T a b l e 1 6 f o r t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s c o r r e c t ) i n d i c a t e t h a t w i t h t w o e x c e p t i o n s , t h e d i f f e r e n c e s b e t w e e n s e s s i o n a l m e a n s w e r e s i g n i f i c a n t . E v e n w h e n t h e b e t w e e n s e s s i o n i n c r e a s e s w e r e n o t l a r g e e n o u g h t o b e s i g n i f i c a n t b y t h e m s e l v e s , t h e m e a n s c o r e s d i d , a s h y p o t h e s i z e d , i n c r e a s e s e s s i o n b y s e s s i o n . I n C h a p t e r 2 r e s e r v a t i o n s a b o u t t h e a s s u m p t i o n o f t h e l i n e a r n a t u r e o f l e a r n i n g , w h i c h u n d e r l i e t h e g e n e r a t i o n o f t h e d e r i v e d s c o r e s o f m o s t c o m m e r c i a l n o r m r e f e r e n c e d a c h i e v e m e n t t e s t s w e r e d i s c u s s e d . T h e s e r e s e r v a t i o n s r e c e i v e d l i m i t e d s u p p o r t f r o m t h e t r e n d a n a l y s i s c o n d u c t e d o n t h e s e s s i o n a l m e a n s o b t a i n e d d u r i n g t h e n o r m i n g o f t h e D T M S . H e r e i t w a s d e m o n s t r a t e d ( T a b l e 1 8 ) t h a t a l t h o u g h t h e l i n e a r t r e n d a c c o u n t s f o r t h e m o s t v a r i a t i o n i n t h e s e s s i o n a l m e a n s t o t a l 8 7 m a t h e m a t i c s f a c t s c o r r e c t s c o r e s a n d t h e s e s s i o n a l m e a n t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s s c o r e s a t e a c h g r a d e l e v e l , t h e f i t o f t h e l i n e a r r e l a t i o n s h i p i s s h o w n t o b e s a t i s f a c t o r y f o r o n l y o n e s e t o f m e a n s . F o r t h r e e o f t h e s e t s o f m e a n s a q u a d r a t i c c u r v e i s s e e n t o p r o v i d e t h e b e s t f i t w h i l e t h e c u b i c t r e n d w a s f o u n d m o s t a p p r o p r i a t e f o r t w o . A n a n a l y s i s o f t h e c u m u l a t i v e p e r c e n t a g e p o l y g o n s o f t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t s c o r e s , a n d t h e t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s c o r r e c t s c o r e s i n d i c a t e s t h a t t h e t e s t s a p p e a r t o m e a s u r e l i t t l e c h a n g e i n t h e p e r f o r m a n c e o f v e r y h i g h a n d v e r y l o w p e r f o r m i n g s t u d e n t s . T h e s u g g e s t i o n i s t h a t f o r t h e s e s t u d e n t s , g r a d e l e v e l t e s t i n g i s n o t a p p r o p r i a t e t o m e a s u r e p r o g r e s s a n d t h a t i t m a y b e m o r e i n f o r m a t i v e t o t e s t l o w p e r f o r m i n g s t u d e n t s a t a l o w e r g r a d e l e v e l a n d h i g h p e r f o r m i n g s t u d e n t s a t a h i g h e r g r a d e . T e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s w e r e c a l c u l a t e d t o d e t e r m i n e t h e r e l i a b i l i t y o f t h e D T M S . A s s h o w n i n T a b l e 1 0 , t h e t e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s f o r t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s k i l l s c o r r e c t r a n g e f r o m .82 t o .92 a n d t h u s c a n b e s e e n t o e i t h e r e x c e e d o r a p p r o a c h t h e .90 r e c o m m e n d e d b y H o p k i n s a n d S t a n l e y ( 1 9 8 1 ) f o r s t a n d a r d i z e d a c h i e v e m e n t t e s t s . C o n s i d e r e d i n l i g h t o f t h e a s s e r t i o n b y G h i s e l l i , C a m p b e l l a n d Z e d e c k ( 1 9 8 1 ) , t h a t "we e x p e c t l o w e r a n d l o w e r e s t i m a t e s o f r e l i a b i l i t y a s t h e t i m e i n t e r v a l b e t w e e n t h e t e s t i n g o c c a s i o n s i n c r e a s e s ( p . 2 4 9 ) " , t h e a c h i e v e m e n t o f t e s t -r e t e s t r e l i a b i l i t y c o e f f i c i e n t s o f t h e m a g n i t u d e o f t h o s e r e p o r t e d 88 a b o v e , w i t h 2 8 d a y s b e t w e e n t e s t i n g s e s s i o n s , s u g g e s t s t h a t a s t u d e n t ' s t o t a l m a t h e m a t i c s f a c t s s c o r e a n d h i s / h e r t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s s c o r e a r e q u i t e r e l i a b l e . Conclusions A s p r e v i o u s l y n o t e d , H o p k i n s a n d S t a n l e y ( 1 9 8 1 ) o b s e r v e d t h a t " t h e c o n c e p t s o f r e l i a b i l i t y a n d v a l i d i t y a r e c e n t r a l t o t h e t h e o r y a n d p r a c t i c e o f e d u c a t i o n a l a n d p s y c h o l o g i c a l t e s t i n g " ( p . 1 1 3 ) . W i t h r e g a r d t o v a l i d i t y , t h e C T B S M a n u a l f o r A d m i n i s t r a t o r s , S u p e r v i s o r s a n d C o u n s e l l o r s ( 1 9 8 4 ) n o t e d t h a t v a r i o u s t y p e s o f s t a t i s t i c a l d a t a i n c l u d i n g " c o r r e l a t i o n s w i t h o t h e r m e a s u r e s s u c h a s c o u r s e g r a d e s a n d s c o r e s o n o t h e r t e s t s o f t h e s a m e t y p e . . . d o n o t g u a r a n t e e i t s ( a t e s t ' s ) v a l i d i t y . T h e y d o n o t p r o v e t h a t t h e t e s t m e a s u r e s w h a t i t p u r p o r t s t o m e a s u r e . T h e y c e r t a i n l y c a n n o t r e v e a l w h e t h e r t h e t h i n g s b e i n g m e a s u r e d a r e t h o s e t h a t o u g h t t o b e m e a s u r e d . " T h e m a n u a l c a u t i o n s t h a t " t h i s i s n o t m e a n t t o i m p l y t h a t w e l l - d e s i g n e d v a l i d a t i o n s t u d i e s a r e o f n o v a l u e " , b u t r a t h e r t h a t " i t s h o u l d b e r e c o g n i z e d t h a t r a t i o n a l j u d g m e n t p l a y s a n i m p o r t a n t p a r t i n e v a l u a t i n g t h e v a l i d i t y o f a c h i e v e m e n t t e s t s w i t h r e s p e c t t o b o t h c o n t e n t a n d p r o c e s s o b j e c t i v e s " ( p . 5 0 ) . I n d e v e l o p i n g t h e DTMS i t w a s r e c o g n i z e d t h a t i f t e a c h e r s d i d n o t v i e w t h e DTMS a s a v a l i d t e s t o f t h e B . C . M a t h e m a t i c s C o m p u t a t i o n s C u r r i c u l u m a s t h e y t a u g h t i t , t h e y w o u l d n ' t u s e i t . T h e r e f o r e , w h e n s e l e c t i n g i t e m t y p e s f o r i n c l u s i o n i n t h e D T M S , e x p e r i e n c e d t e a c h e r s , w h o w e r e t h e n t e a c h i n g t h e g r a d e s i n q u e s t i o n , w e r e a s k e d , " D o e s t h i s i t e m r e f l e c t t h e w a y i n w h i c h y o u 89 t e a c h t h e B . C . M a t h e m a t i c s C u r r i c u l u m i n y o u r c l a s s r o o m ? " T h u s , b y s e l e c t i n g i t e m t y p e s w h i c h w e r e g e n e r a l l y a c c e p t a b l e t h e c u r r i c u l a r a n d i n s t r u c t i o n a l v a l i d i t y o f t h e DTMS w a s a s s u r e d . I n a s s e s s i n g t h e r e l i a b i l i t y o f t h e DTMS t h e f a c t t h a t i t i s a " s p e e d e d " t e s t ; t h a t i s , a t e s t i n w h i c h n o t a l l s t u d e n t s h a d s u f f i c i e n t t i m e t o d o a l l t h e i t e m s , i m p o s e d l i m i t a t i o n s o n t h e t y p e o f a n a l y s i s p e r f o r m e d . F o r e x a m p l e , H o p k i n s a n d S t a n l e y (1981) n o t e d t h a t " s p l i t h a l f a n d i n t e r n a l c o n s i s t e n c y m e t h o d s a r e a p p r o p r i a t e o n l y f o r p o w e r t e s t s , t e s t s i n w h i c h e v e r y s t u d e n t h a s a d e q u a t e t i m e t o c o m p l e t e e a c h i t e m " (p.133). F o r s p e e d e d t e s t s s u c h a s t h e D T M S , t h e y r e c o m m e n d e d t h e u s e o f t e s t - r e t e s t p r o c e d u r e s . T e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s f o r t h e t o t a l m a t h e m a t i c s f a c t s c o r r e c t a n d t o t a l m a t h e m a t i c s c o m p u t a t i o n s s k i l l s s c o r e s o f t h e DTMS w e r e s e e n t o e i t h e r e x c e e d o r c l o s e l y a p p r o x i m a t e t h e .90 r e c o m m e n d e d f o r s t a n d a r d i z e d a c h i e v e m e n t t e s t s b y H o p k i n s a n d S t a n l e y (1981) i n s p i t e o f t h e c a u t i o n b y G h i s e l l i , C a m p b e l l a n d Z e d e c k (1981) t h a t "we e x p e c t l o w e r a n d l o w e r e s t i m a t e s o f r e l i a b i l i t y a s t h e t i m e i n t e r v a l b e t w e e n t h e t e s t i n g o c c a s i o n s i n c r e a s e s " (p.249). ( R e c a l l t h e r e w e r e 28 d a y s b e t w e e n t h e t e s t i n g s e s s i o n s o.f t h e D T M S . ) C l e a r l y i t h a s b e e n d e m o n s t r a t e d t h a t t h e DTMS i s a v a l i d a n d r e l i a b l e m e a s u r e o f t h e B . C . M a t h e m a t i c s C o m p u t a t i o n s C u r r i c u l u m f o r G r a d e s 2, 3, a n d 4. T h e h y p o t h e s e s w h i c h f o c u s e d t h e d e v e l o p m e n t o f t h i s s t u d y a n d o f f e r e d d i r e c t i o n i n t h e a n a l y s i s o f t h e d a t a s t a t e d t h a t c u r r i c u l u m - b a s e d a s s e s s m e n t c o u l d m e a s u r e c h a n g e s i n a 90 s t u d e n t ' s / g r o u p ' s p e r f o r m a n c e o n s p e c i f i c s k i l l s o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e a n d t h a t t h i s c h a n g e c o u l d n o t b e m e a s u r e d o n c o n v e n t i o n a l s t a n d a r d i z e d a c h i e v e m e n t t e s t s s u c h a s t h e C a n a d i a n T e s t s o f B a s i c S k i l l s . I n h y p o t h e s i z i n g c h a n g e s i n p e r f o r m a n c e o v e r b r i e f p e r i o d s o f t i m e i t w a s a n t i c i p a t e d t h a t t h e m o n t h l y m e a n t o t a l c o r r e c t s c o r e s i n e a c h o f t h e s k i l l a r e a s ( m a t h e m a t i c s f a c t s a n d m a t h e m a t i c s c o m p u t a t i o n s ) a t e a c h g r a d e l e v e l w o u l d i n c r e a s e o v e r t h e f o u r t e s t i n g s e s s i o n s . A s p r e v i o u s l y n o t e d , i n a l l i n s t a n c e s , a t e a c h g r a d e l e v e l t o t a l m e a n s c o r e s i n c r e a s e d s e s s i o n b y s e s s i o n . W i t h t w o e x c e p t i o n s , a l l o f t h e m o n t h l y m e a n t o t a l c o r r e c t s c o r e s , i n e a c h o f t h e s k i l l a r e a s , a t e a c h g r a d e l e v e l w e r e f o u n d s i g n i f i c a n t b y p r o c e d u r e s u t i l i z i n g a n a n a l y s i s o f v a r i a n c e f o r r e p e a t e d m e a s u r e s d e s i g n a n d T u k e y ' s H S D t e s t . C l e a r l y , t h e r e s u l t s o f t h e n o r m i n g s u p p o r t t h e a s s e r t i o n s o f t h e h y p o t h e s e s t h a t c u r r i c u l u m - b a s e d a s s e s s m e n t c a n m e a s u r e c h a n g e s i n a s t u d e n t ' s o r g r o u p ' s p e r f o r m a n c e o n s p e c i f i c m a t h e m a t i c s s k i l l s o v e r b r i e f ( o n e m o n t h ) p e r i o d s o f t i m e . I n t e r p r e t a t i o n and Dse o f t h e D T M S T h e DTMS h a s b e e n s h o w n t o b e a v a l i d a n d r e l i a b l e m e a s u r e o f s t u d e n t p r o g r e s s t h r o u g h t h e c o m p u t a t i o n s r e q u i r e m e n t s o f t h e B . C . M a t h e m a t i c s C u r r i c u l u m f o r G r a d e s 2, 3 a n d 4 a s t a u g h t i n D e l t a . S u i t a b l e f o r r e p e a t e d a d m i n i s t r a t i o n , t h e DTMS p e r m i t s t h e m e a s u r e m e n t o f b o t h p r o f i c i e n c y a n d r a t e o f l e a r n i n g . H o w e v e r , a s w i t h a n y t e s t , c a r e m u s t b e t a k e n w h e n i n t e r p r e t i n g s t u d e n t 91 FIGURE 5: HYPOTHETICAL GRADE 2 TOTAL MATHEMATICS FACTS CORRECT SCORES PLOTTED ON GRID CONAINING ACTUAL MEANS AND STANDARD DEVIATIONS. 92 r e s u l t s . P r o p e r p r o c e d u r e s f o r i n t e r p r e t i n g D TMS r e s u l t s a r e o u t l i n e d i n t h e f o l l o w i n g s e c t i o n . F i g u r e 5 d i s p l a y s t h e m e a n s a n d s t a n d a r d d e v i a t i o n s o f t h e m o n t h l y t o t a l m a t h e m a t i c s f a c t s s c o r e s f o r G r a d e 2. T h e l e a r n i n g c u r v e g e n e r a t e d b y t h e s e m o n t h l y m e a n s p r o v i d e s a n i n d e x o f a v e r a g e m o n t h l y g r o w t h i n s t u d e n t f a c i l i t y w i t h m a t h e m a t i c s f a c t s . S t u d e n t s w h o s e s c o r e s f a l l w i t h i n p l u s o r m i n u s o n e S t a n d a r d D e v i a t i o n c a n b e s a i d t o b e p e r f o r m i n g i n t h e a v e r a g e r a n g e o f t h e n o r m i n g g r o u p . R e p e a t e d m o n t h l y t e s t i n g w o u l d p e r m i t u s t o c o m p a r e a s t u d e n t ' s r a t e o f 1 e a r n i n g w i t h t h a t o f t h e n o r m i n g g r o u p . T h u s , a s i n d i c a t e d i n F i g u r e 5, S t u d e n t 'A' c a n b e e n s e e n t o b e p e r f o r m i n g w i t h i n t h e a v e r a g e r a n g e a n d h i s r a t e o f l e a r n i n g w o u l d a l s o a p p e a r a v e r a g e i n t h a t t h e s l o p e o f h i s l e a r n i n g c u r v e c a n b e s e e n t o a p p r o x i m a t e t h a t o f t h e n o r m i n g g r o u p . I n c o n t r a s t t o t h i s , S t u d e n t 'B' i s s e e n t o b e f u n c t i o n i n g b e l o w t h e a v e r a g e g r o u p . F u r t h e r m o r e , a s h i s s c o r e s a r e b e c o m i n g i n c r e a s i n g l y d i s c r e p a n t we c a n c o n c l u d e t h a t h i s l e a r n i n g r a t e i s b e l o w t h a t o f t h e n o r m i n g g r o u p . W i t h o u t i m m e d i a t e i n t e r v e n t i o n t h i s s t u d e n t w i l l o n l y f a l l f u r t h e r b e h i n d . I n F i g u r e 6 a n t i c i p a t e d m e a n s c o r e s a r e p r o j e c t e d f o r o n e m o n t h p r i o r t o t h e n o r m i n g p e r i o d a n d f o r o n e m o n t h f o l l o w i n g t h e n o r m i n g p e r i o d f o r t h e G r a d e 2 T o t a l M a t h e m a t i c s F a c t s C o r r e c t S c o r e s . U s i n g t h e s e p r o j e c t e d s c o r e s we a r e a b l e t o e x p a n d t h e u s e f u l n e s s o f t h e t e s t t o c o v e r s i x m o n t h s o f t h e s c h o o l y e a r ( D e c e m b e r t o M a y ) . T h i s e x p a n s i o n i s r e a s o n a b l e 93 FIGURE 6: GRADE 2 TOTAL MATHEMATICS FACTS SESSIONAL MEANS AND STANDARD DEVIATIONS. Months 94 c o n s i d e r i n g t h e s t a b i l i t y o f t h e d a t a g a t h e r e d d u r i n g n o r m i n g a n d l i m i t e d n a t u r e o f t h e p r o j e c t i o n . F i g u r e E-13 t o E-17 c o n t a i n g r a p h i c r e p r e s e n t a t i o n s o f t h e m e a n s , a n t i c i p a t e d m e a n s , t h e s t a n d a r d d e v i a t i o n s a n d a n t i c i p a t e d s t a n d a r d d e v i a t i o n s o f t h e b a l a n c e o f t h e t o t a l m a t h e m a t i c s s c o r e s . B o t h t h e M o u n t P l e a s a n t M o d e l ( C a l t a g i r o n e & G l o v e r , 1985) a n d t h e P i n e C o u n t y M o d e l ( T i n d a l e t a l . , 1982) u s e d t h e s a m e t e s t t o a s s e s s a g r a d e f r o m S e p t e m b e r t h r o u g h J u n e . S u c h a p r o c e d u r e i s o f q u e s t i o n a b l e v a l u e e a r l y i n t h e s c h o o l y e a r w h e n t e a c h e r s a r e m o s t o f t e n r e v i e w i n g t h e p r e v i o u s y e a r ' s w o r k o r a t l e a s t h a v e o n l y b e g u n t o t e a c h t h e c u r r e n t y e a r ' s c u r r i c u l u m c o n t e n t . A t t h i s t i m e i t w o u l d b e m u c h m o r e a p p r o p r i a t e t o t e s t s t u d e n t s u s i n g t h e p r e v i o u s g r a d e ' s t e s t a n d t h e M a y n o r m s . S t u d e n t s w o u l d c e r t a i n l y a p p r o a c h t h e t e s t m o r e p o s i t i v e l y a n d t e a c h e r s w o u l d h a v e a n e x c e l l e n t i n d i c a t i o n o f w h o w a s r e a d y t o p r o c e e d w i t h t h e new y e a r ' s w o r k . R e c a l l t h a t c a u t i o n i s r e q u i r e d w h e n i n t e r p r e t i n g t h e s c o r e s o f v e r y h i g h o r v e r y l o w p e r f o r m e r s . T h e p r o g r e s s o f t h e s e p u p i l s w o u l d b e s t b e m e a s u r e d b y r e s p e c t i v e l y t h e n e x t h i g h e r o r t h e n e x t l o w e r t e s t . I t s h o u l d n o t , h o w e v e r , b e f o r g o t t e n t h a t a s i n d i c a t e d e l s e w h e r e t h e b e s t u s e o f t h e t e s t i s i n c o n j u n c t i o n w i t h o t h e r m e a s u r e s o f c l a s s r o o m p e r f o r m a n c e . 95 Summary I n t h i s c h a p t e r we h a v e r e v i e w e d t h e d e v e l o p m e n t o f t h e DTMS a n d t h e r e l a t e d r e s e a r c h . A r e v i e w o f t h e d a t a e m a n a t i n g f r o m t h e n o r m i n g o f t h e DTMS c o n f i r m e d i t a s a r e l i a b l e m e a s u r e o f a G r a d e 2, 3 o r 4 s t u d e n t ' s f a c i l i t y w i t h m a t h e m a t i c s f a c t s a n d m a t h e m a t i c s c o m p u t a t i o n s a s r e q u i r e d b y t h e B . C . M a t h e m a t i c s C u r r i c u l u m . G u i d e l i n e s w e r e p r e s e n t e d f o r i n t e r p r e t i n g t h e DTMS. 9 6 REFERENCES A i r a s i a n , P.W., & M a d a u s , G . F . ( 1 9 8 3 ) . L i n k i n g i n s t r u c t i o n : P o l i c y i s s u e s . J o u r n a l o f E d u c a t i o n a l M e a s u r e m e n t , 2 0 , 1 0 3 - 1 1 8 . B e g g s , D . L . & H i e r o n y m u s , A . N . ( 1 9 6 8 ) . U n i f o r m i t y o f g r o w t h i n t h e b a s i c s k i l l s t h r o u g h o u t t h e s c h o o l y e a r a n d d u r i n g t h e s u m m e r . J o u r n a l o f E d u c a t i o n a l M e a s u r e m e n t , 5, 9 1 - 9 7 . B l a n k e n s h i p , C , & L i l l y , M . S . ( 1 9 8 1 ) . M a i n s t r e a m i n g s t u d e n t s  w i t h l e a r n i n g a n d b e h a v i o r p r o b l e m s : T e c h n i q u e s f o r t h e  c l a s s r o o m t e a c h e r . New Y o r k : H o l t , R h i n e h a r t & W i n s t o n . C a l t a g i r o n e , P . J . , & G l o v e r , C E . ( 1 9 8 5 , M a y ) . P r e c i s i o n l e a r n i n g  a s s e s s m e n t : A r e a l i s t i c a l t e r n a t i v e t o t r a d i t i o n a l  a s s e s s m e n t t e c h n i q u e s . P a p e r p r e s e n t e d a t t h e 5 t h I n t e r n a t i o n a l P r e c i s i o n T e a c h i n g / L e a r n i n g C o n f e r e n c e , S e a t t l e , WA. C a l t a g i r o n e , P . J . , & G l o v e r , C E . ( 1 9 8 5 ) . P r e c i s i o n l e a r n i n g a s s e s s m e n t : A n a l t e r n a t i v e t o t r a d i t i o n a l l e a r n i n g a s s e s s m e n t t e c h n i q u e s . B . C . J o u r n a l o f S p e c i a l E d u c a t i o n , 9, 3 5 5 - 3 6 4 . C a n a d i a n T e s t s o f B a s i c S k i l l s ( 1 9 8 4 ) . L e v e l s 5 - 1 8 , F o r m s 5 & 6. M a n u a l f o r A d m i n i s t r a t o r s , S u p e r v i s o r s a n d C o u n s e l l o r s . N e l s o n C a n a d a L t d . D e n o , S . L . ( 1 9 8 5 ) . C u r r i c u l u m - b a s e d m e a s u r e m e n t : T h e e m e r g i n g a l t e r n a t i v e . E x c e p t i o n a l C h i l d r e n , 5 2 ( 3 ) , 2 1 9 - 2 3 2 . D e n o , S . L . , & F u c h s , L . S . ( 1 9 8 7 ) . D e v e l o p i n g c u r r i c u l u m - b a s e d m e a s u r e m e n t s y s t e m s f o r d a t a - b a s e d s p e c i a l e d u c a t i o n p r o b l e m -s o l v i n g . F o c u s o n E x c e p t i o n a l C h i l d r e n , 1 9 ( 8 ) , 1 - 1 6 . D e n o , S . L . , J e n k i n s , J . , & M i r k i n , P. ( 1 9 7 9 ) . M e a s u r i n g p u p i l p r o g r e s s t o w a r d t h e l e a s t r e s t r i c t i v e e n v i r o n m e n t . L e a r n i n g  D i s a b i l i t i e s Q u a r t e r l y , 2 ( 4 ) , 8 1 - 9 2 . D e n o , S . L . , M i r k i n , P . K . , C h i a n g , B . , & L o w r y , L . ( 1 9 8 0 ) . R e l a t i o n s h i p s a m o n g s i m p l e m e a s u r e s o f r e a d i n g a n d  p e r f o r m a n c e o n s t a n d a r d i z e d a c h i e v e m e n t t e s t s ( R e s e a r c h R e p o r t N o . 2 0 ) . M i n n e a p o l i s : U n i v e r s i t y o f M i n n e s o t a , I n s t i t u t e f o r R e s e a r c h o n L e a r n i n g D i s a b i l i t i e s ( E R I C D o c u m e n t R e p r o d u c t i o n S e r v i c e N o . E D 1 9 7 5 0 7 ) . 97 D e n o , S . L . , M i r k i n , P . K , . , L o w r y , L . , & K u e h n l e , K. ( 1 9 8 0 ) . R e l a t i o n s h i p s among s i m p l e m e a s u r e s o f s p e l l i n g and  p e r f o r m a n c e on s t a n d a r d i z e d a c h i e v e m e n t t e s t s ( R e s e a r c h R e p o r t N o . 2 1 ) . M i n n e a p o l i s : U n i v e r s i t y o f M i n n e s o t a , I n s t i t u t e f o r R e s e a r c h on L e a r n i n g D i s a b i l i t i e s (ERIC Document R e p r o d u c t i o n S e r v i c e N o . ED 197 5 0 8 ) . D e n o , S . L . , M i r k i n , P . K . , & M a r s t o n , D. ( 1 9 8 0 ) . R e l a t i o n s h i p s among s i m p l e m e a s u r e s o f w r i t t e n e x p r e s s i o n and p e r f o r m a n c e  on s t a n d a r d i z e d a c h i e v e m e n t t e s t s ( R e s e a r c h R e p o r t N o . 2 2 ) . M i n n e a p o l i s : U n i v e r s i t y o f M i n n e s o t a , I n s t i t u t e f o r R e s e a r c h on L e a r n i n g D i s a b i l i t i e s (ERIC Document R e p r o d u c t i o n S e r v i c e N o . ED 107 509) . F e r g u s o n , G . A . ( 1 9 8 1 ) . S t a t i s t i c a l a n a l y s i s i n p s y c h o l o g y and  e d u c a t i o n ( 5 t h e d . ) . M c G r a w - H i l l Book Company. New Y o r k : New Y o r k . F u c h s , L . , & D e n o , S . L . ( 1 9 8 1 ) . The R e l a t i o n s h i p b e t w e e n  c u r r i c u l u m b a s e d m a s t e r y m e a s u r e s and s t a n d a r d i z e d  a c h i e v e m e n t t e s t s i n r e a d i n g ( R e s e a r c h R e p o r t N o . 5 7 ) . M i n n e a p o l i s : U n i v e r s i t y o f M i n n e s o t a , I n s t i t u t e f o r R e s e a r c h on L e a r n i n g D i s a b i l i t i e s (ERIC Document R e p r o d u c t i o n S e r v i c e N o . ED 212 6 6 2 ) . F u c h s , L . , F u c h s , D . , & W a r r e n , L . ( 1 9 8 2 ) . S p e c i a l e d u c a t i o n  p r a c t i c e i n e v a l u a t i n g s t u d e n t p r o g r e s s t o w a r d s g o a l s ( R e s e a r c h R e p o r t N o . 2 1 ) . M i n n e a p o l i s : I n s t i t u t e f o r R e s e a r c h on L e a r n i n g D i s a b i l i t i e s . F u c h s , L . S . , & F u c h s , D. ( 1 9 8 6 ) . C u r r i c u l u m - b a s e d a s s e s s m e n t o f p r o g r e s s t o w a r d l o n g - t e r m and s h o r t - t e r m g o a l s . The J o u r n a l  o f S p e c i a l E d u c a t i o n , 2 0 , 6 9 - 8 2 . G e r m a n n , G . , & T i n d a l , G . ( 1 9 8 5 ) . An a p p l i c a t i o n o f c u r r i c u l u m -b a s e d a s s e s s m e n t : The u s e o f d i r e c t and r e p e a t e d m e a s u r e m e n t . E x c e p t i o n a l C h i l d r e n , 52 ( 3 ) , 2 4 4 - 2 6 5 . G h i s e l l i , E . E . , C a m p b e l l , J . P . , & Z e d e c k , S . ( 1 9 8 1 ) . Measuremen t  T h e o r y f o r t h e B e h a v i o r a l S c i e n c e s . W . H . F reeman and Company. New Y o r k : New Y o r k . G i c k i n g , E.E., & Thompson , V.P. ( 1 9 8 5 ) . A p e r s o n a l v i e w o f c u r r i c u l u m b a s e d a s s e s s m e n t . E x c e p t i o n a l C h i l d r e n , 52 ( 3 ) , 2 0 5 - 2 1 8 . G r o n l u n d , N . E . ( 1 9 8 2 ) . C o n s t r u c t i n g A c h i e v e m e n t T e s t s ( 3 r d e d . ) P r e n t i c e - H a l l I n c . E n g l e w o o d C l i f f s : New J e r s e y . H a e r t e l , E . , & C a l f e e , R. ( 1 9 8 3 ) . S c h o o l a c h i e v e m e n t : T h i n k i n g a b o u t what t o t e s t . J o u r n a l o f E d u c a t i o n a l M e a s u r e m e n t , 2 0 , 1 1 9 - 1 3 2 . 98 H o p k i n s , K.D., & S t a n l e y , J . C . ( 1 9 8 1 ) . E d u c a t i o n a l a n d p s y c h o l o g i c a l m e a s u r e m e n t a n d e v a l u a t i o n ( 6 t h e d . ) P r e n t i c e -H a l l I n c . E n g l e w o o d C l i f f s : New J e r s e y . J e n k i n s , J . R . , D e n o , S . L . , & M i r k i n , P . K . ( 1 9 7 9 ) . M e a s u r i n g p u p i l  p r o g r e s s t o w a r d t h e l e a s t r e s t r i c t i v e e n v i r o n m e n t ( M o n o g r a p h N o . 1 0 ) . M i n n e a p o l i s : U n i v e r s i t y o f M i n n e s o t a , I n s t i t u t e f o r R e s e a r c h o n L e a r n i n g D i s a b i l i t i e s . J e n k i n s , J . R . , & P a n y , D. ( 1 9 7 8 ) . S t a n d a r d i z e d a c h i e v e m e n t t e s t s How u s e f u l f o r s p e c i a l e d u c a t i o n ? E x c e p t i o n a l C h i l d r e n , 4 4 , 4 4 8 - 4 5 3 . K i n g , E . M . ( C a n a d i a n e d . ) ( 1 9 8 2 ) . C a n a d i a n T e s t s o f B a s i c S k i l I s , P r i m a r y B a t t e r y , L e v e l 8, F o r m 5 . N e l s o n C a n a d a L t d . K i n g , E . M . ( C a n a d i a n e d . ) ( 1 9 8 2 ) . C a n a d i a n T e s t s o f B a s i c S k i l l s , P r i m a r y B a t t e r y , L e v e l s 7 & 8, F o r m 5, T e a c h e r ' s G u i d e . N e l s o n C a n a d a L t d . K i n g , E . M . ( C a n a d i a n e d . ) ( 1 9 8 2 ) . C a n a d i a n T e s t s o f B a s i c S k i l l s , M u l t i l e v e l E d i t i o n , L e v e l s 9 - 1 4 . N e l s o n C a n a d a L t d . L i n n , R . L . ( 1 9 8 3 ) . T e s t i n g a n d i n s t r u c t i o n : L i n k s a n d d i s t i n c t i o n s . J o u r n a l o f E d u c a t i o n a l M e a s u r e m e n t , 2 0 , 1 7 9 - 1 8 9 . M i r k i n , P . K . , F u c h s , L . S . , & D e n o , S . L . ( 1 9 8 2 ) . A p p r o a c h e s t o  m e a s u r e m e n t . U n p u b l i s h e d m a n u s c r i p t , U n i v e r s i t y o f M i n n e s o t a , I n s t i t u t e f o r R e s e a r c h o n L e a r n i n g D i s a b i l i t i e s . M o u s l e y , W. ( 1 9 7 3 ) . T e s t i n g t h e " s u m m e r l e a r n i n g l o s s " a r g u m e n t . P h i D e l t a K a p p a n , 5 4 , 7 0 5 . S a l v i a , J . & Y s s e l d y k e , J . E . ( 1 9 8 1 ) . A s s e s s m e n t i n S p e c i a l R e m e d i a l E d u c a t i o n ( 2 n d e d . ) H o u g h t o n - M i f f l i n : B o s t o n . S a l m o n - C o x , L . ( 1 9 8 1 ) . T e a c h e r s a n d s t a n d a r d i z e d a c h i e v e m e n t t e s t s : W h a t ' s r e a l l y h a p p e n i n g ? P h i D e l t a K a p p a n , 6 2 ( 9 ) , 6 3 1 - 6 3 4 . S c h m i d t , W.H. ( 1 9 8 3 ) . C o n t e n t b i a s e s i n a c h i e v e m e n t t e s t s . J o u r n a l o f E d u c a t i o n a l M e a s u r e m e n t , 2 0 , 1 6 5 - 1 7 7 . S h a v e l s o n , R . J . ( 1 9 8 1 ) . S t a t i s t i c a l R e a s o n i n g f o r t h e B e h a v i o r a l  S c i e n c e s . A l l y n a n d B a c o n I n c : B o s t o n . S n a r t , F . , D e n n i s , S . , & B r a i l s f o r d , A . ( 1 9 8 3 ) . C o n c e r n s r e g a r d i n g t h e w i d e r a n g e a c h i e v e m e n t t e s t . C a n a d i a n  P s y c h o l o g y , 2 4 , 9 9 - 1 0 3 . 99 Tallmadge, G.K. (1977). Cautions to evaluators. In M. Wargo and D. Green (ed.) Achievement t e s t i n g of disadvantaged and minority students f o r educational program evaluation, 331-384. CTB/McGraw-Hil1. Thurlow, M.L. , & Ysseldyke, J..E. (1982). I n s t r u c t i o n a l planning: Information c o l l e c t e d by school psychologists vs. information considered u s e f u l by teachers. Journal of School Psychology, 20, 3-10. T i n d a l , G., Fuchs, L.S., Fuchs, D., Shinn, M.R., Deno, S.L., & Germann, G. (1985). Empirical v a l i d a t i o n of c r i t e r i o n -referenced t e s t . Journal of Educational Research, 78, 203-209. T i n d a l , G. & Germann, G. (1985). Models of d i r e c t measurement i n the determination of e l i g i b i l i t y , monitoring of student programs, and evaluation of program e f f e c t s . B.C. Journal of  Special Education, 9 (4), 365-382. T i n d a l , G. , Wesson, C , Germann, G., Deno, S.L., & Mirkin, P.K. (1982). The Pine County model f o r s p e c i a l education  d e l i v e r y : A data based model (Monograph No. 19). Minneapolis: U n i v e r s i t y of Minnesota, I n s t i t u t e f o r Research on Learning D i s a b i l i t i e s , 1982. Tucker, J.A. (1985). Curriculum based assessment: An i n t r o d u c t i o n . Exceptional C h i l d r e n , 52 (3), 199-204. Wormeli, C.T. (1983). B.C. Quick In d i v i d u a l Educational Test. The Education C l i n i c , The U n i v e r s i t y of B r i t i s h Columbia, Vancouver, B.C. Wormeli, C.T. (1983). B.C. Quick Individual Educational Test, Technical Manual. The Education C l i n i c , The U n i v e r s i t y of B r i t i s h Columbia, Vancouver, B.C. Yalow, E.S., & Popham, W.J. (1983). Content v a l i d i t y at the crossroads. Educational Researcher, 12 (8), 10-14, 21. Zigmond, N., & Silverman, R. (1984). Informal assessment for program planning and evaluation i n s p e c i a l education. Educational Psychologist, 19, 163-171. 100 A P P E N D I X A At each grade l e v e l items which appear i n the Canadian Tests of B a s i c S k i l l s (CTBS) column on the same l i n e as items i n the D e l t a Tests of Mathematics S k i l l s (DTMS) column are considered s i m i l a r . Items i n the CTBS column which precede item number 1 i n the DTMS column are co n s i d e r e d t o represent s k i l l s mastered below the grade l e v e l t e s t e d or are math f a c t items i n the DTMS. Items i n the CTBS column which f a l l below the l a s t item i n the DTMS are con s i d e r e d t o represent s k i l l s which students are not y et r e q u i r e d t o have mastered a t the l e v e l being t e s t e d but may be r e q u i r e d t o master i n higher grades. Items i n the CTBS column between the f i r s t and l a s t q u e s t i o n i n the DTMS but which are not on the same l i n e as the items i n the DTMS column are items which correspond t o some c u r r i c u l a r requirements but c o n t a i n v a r i a t i o n s which may present unnecessary o b s t a c l e s t o the student being t e s t e d . For example, r e q u i r i n g a Grade 2 student t o regroup when adding a two-d i g i t number and a o n e - d i g i t number pl a c e d so t h a t one i s above the o ther may s u f f i c i e n t l y t e s t the c u r r i c u l a r requirement f o r adding such q u e s t i o n s without a s k i n g him/her t o perform a s i m i l a r o p e r a t i o n when the numbers are pl a c e d s i d e by s i d e . 101 TABLE A-l A COMPARISON OF THE GRADE 2 ADDITION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f i v e t i m e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS M a t h f a c t t e s t e d (1) 1 - C o m p u t e s u m s 2 + 4 + 2 = 1 - T e s t e d (1) 2 - C o m p u t e s u m s 2 4 + 2 2 - N o t t e s t e d 3 - M i s s i n g a d d e n d 4 + = 9 3 - N o t t e s t e d 4 - M i s s i n g a d d e n d 4 4 - N o t t e s t e d 5 - N o r e g r o u p i n g -2 - d i g i t + 1 - d i g i t 5 - N o t t e s t e d 1 0 2 TABLE A-l - Cont'd DTMS CTBS 6 - N o r e g r o u p i n g - 6 - T e s t e d ( 1 ) 2 - d i g i t + 2 - d i g i t . 1 4 + 24 V a r i a t i o n t e s t e d ( 1 ) e . g . 1 1 + 1 4 = 7 - R e g r o u p i n g 1 ' s 7 -2 - d i g i t + 1 - d i g i t 2 4 + 6 V a r i a t i o n t e s t e d ( 1 ) e . g . 3 5 + 9 = 8 - R e g r o u p i n g l ' s 8 - T e s t e d ( 1 ) 2 - d i g i t + 2 - d i g i t 2 7 + 24 R e g r o u p i n g l ' s t e s t e d ( 1 ) 2 - d i g i t s + 1 - d i g i t + 1 - d i g i t e . g . 1 3 + 5 + 6 = R e g r o u p i n g l ' s t e s t e d ( 1 ) 3 - d i g i t s + 2 - d i g i t s e . g . 1 3 4 + 5 7 = 1 0 3 TABLE A-2 A COMPARISON OF THE GRADE 2 SUBTRACTION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f i v e t i r o e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS 1 - M i s s i n g n u m e r a l 9 - = 4 -4 = 5 1 - N o t t e s t e d 2 - M i s s i n g n u m e r a l 9 - 5 4 4 2 - N o t t e s t e d 3 - N o r e g r o u p i n g -2 - d i g i t - 1 - d i g i t 2 7 - 6 3 - N o t t e s t e d 4 - N o r e g r o u p i n g -2 - d i g i t - 2 - d i g i t 48 - 1 5 4 - T e s t e d ( 1 ) V a r i a t i o n t e s t e d ( 1 ) e . g . 3 7 - 1 3 = 1 0 4 TABLE A-2 - Cont'd DTMS CTBS 5 - R e g r o u p i n g . 1 0 - s - 5 - T e s t e d ( 1 ) 2 - d i g i t - 1 - d i g i t 2 1 - 5 V a r i a t i o n t e s t e d ( 1 ) e . g . 2 3 - 5 = 6 - R e g r o u p i n g 1 0 ' s 6 - V a r i a t i o n t e s t e d ( 1 ) 45 - 3 6 N o r e g r o u p i n g t e s t e d ( 2 ) 3 - d i g i t s - 2 - d i g i t s e . g . 2 3 0 - 2 0 R e g r o u p i n g t e s t e d ( 1 ) 3 - d i g i t - 1 - d i g i t e . g . 1 2 4 - 8 105 TABLE A-3 A COMPARISON OF THE GRADE 3 ADDITION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f i v e t i r o e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS M a t h f a c t s t e s t e d (1) 1 - C o m p u t e s u m s t o 18 1 - T e s t e d (1) 2 4 + 3 2 - C o m p u t e s u m s t o 18 2 - N o t t e s t e d 4 + 6 + 8 = 3 - C o m p u t e s u m s 18+ 3 - N o t t e s t e d 5 9 + 9 4 - M i s s i n g a d d e n d 4 - N o t t e s t e d 1 + 3 + =6 TABLE A-3 - Cont'd DTMS CTBS 5 - W i t h o u t r e g r o u p i n g 2 - d i g i t + 2 - d i g i t 80 - 17 5 - N o t t e s t e d V a r i a t i o n t e s t e d (1) e . g 12 + 15 = V a r i a t i o n t e s t e d (2) e . g . 35 + 4 V a r i a t i o n t e s t e d (1) e . g . 12 + 6 + 31 = V a r i a t i o n t e s t e d (1) e . g . 200 + 46 = 6 - W i t h r e g r o u p i n g l ' s o n l y 6 - T e s t e d (1) 2 - d i g i t + 2 - d i g i t 17 + 26 V a r i a t i o n t e s t e d (1) e . g . 57 + 34 = V a r i a t i o n t e s t e d (1) e . g . 23 + 7 = V a r i a t i o n t e s t e d (1) e . g . 64 + 8 7 - W i t h r e g r o u p i n g - 10's o n l y 7 - N o t t e s t e d 2 - d i g i t + 2 - d i g i t 97 + 32 1 0 7 TABLE A-3 - Cont'd DTMS CTBS 8 - R e g r o u p i n g l ' s o r 1 0 ' 3 - d i g i t + 2 / 3 - d i g i t s 3 4 7 + 561 8 - T e s t e d ( 2 ) V a r i a t i o n t e s t e d ( 1 ) e . g . 8 6 + 7 7 V a r i a t i o n t e s t e d e . g . 1 8 1 7 1 6 + 1 5 ( 1 ) 9 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' s 9 - N o t t e s t e d 3 - d i g i t + 2 / 3 - d i g i t s i n c o l u m n o f 3 n u m b e r s 7 7 2 4 5 7 + 1 9 8 V a r i a t i o n t e s t e d ( 1 ) e . g . 6 7 5 + 6 2 6 = _ 1 0 8 TABLE A-4 A COMPARISON OF THE GRADE 3 SUBTRACTION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f i v e t i m e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS 32 - 2 2 0 - 1 0 4 2 6 - 3 1 3 1 5 8 - 1 7 2 7 9 - 8 5 3 4 M a t h f a c t s t e s t e d ( 2 ) 1 - N o r e g r o u p i n g 2 - d i g i t - 1/2 d i g i t s 1 - T e s t e d ( 2 ) V a r i a t i o n t e s t e d ( 1 ) e . g . 6 7 - 7 = 2 - N o r e g r o u p i n g 3 - d i g i t s - 1, 2, o r 3 2 - T e s t e d ( 3 ) 3 - R e g r o u p i n g 1 0 ' s 2 - d i g i t - 1 - d i g i t 3 - T e s t e d ( 1 ) V a r i a t i o n t e s t e d ( 1 ) e . g . 3 2 - 3 = 1 0 9 TABLE A-4 - Cont'd DTMS CTBS 4 - R e g r o u p i n g 1 0 ' s 4 - N o t t e s t e d 2 - d i g i t - 2 - d i g i t 3 6 - 1 9 V a r i a t i o n t e s t e d ( 1 ) e . g . 5 8 - 2 9 = 5 - R e g r o u p i n g 1 0 ' s 5 - T e s t e d ( 2 ) 3 - d i g i t - 2 / 3 d i g i t 1 8 2 5 8 2 - 6 9 - 1 6 6 6 - R e g r o u p i n g 1 0 0 ' s 6 - N o t t e s t e d 3 - d i g i t - 2 - d i g i t 6 2 8 - 8 4 7 - R e g r o u p i n g 1 0 0 ' s 7 - N o t t e s t e d 3 - d i g i t - 3 - d i g i t 4 3 9 - 2 8 4 8 - R e g r o u p i n g 1 0 ' s a n d / o r 8 - T e s t e d ( 2 ) 1 0 0 ' s 3 - d i g i t - 2 - d i g i t 7 3 5 - 8 6 110 TABLE A-4 - Cont'd DTMS CTBS 9 - R e g r o u p i n g 10's and/ o r 9 - Not t e s t e d 100's 3 - d i g i t - 3 - d i g i t 653 - 284 No r e g r o u p i n g t e s t e d (1) 4 - d i g i t - 4 - d i g i t e.g. 7,824 - 6,302 1 1 1 TABLE A-5 A COMPARISON OF THE GRADE 3 MULTIPLICATION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f o u r t i m e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS 1 - B a s i c f a c t s t o 5 0 1 - T e s t e d ( 1 ) 9 x 5 2 - B a s i c f a c t s ( m i s s i n g f a c t o r ) 2 - N o t t e s t e d 8 3 - N o r e g r o u p i n g 3 - T e s t e d ( 1 ) 1 - d i g i t m u l t i p l i e r x 2 - d i g i t s 2 4 x 2 4 - B a s i c f a c t s t o 5 0 4 - N o t t e s t e d 5 x 9 = 1 1 2 TABLE A-5 - Cont'd DTMS CTBS 5 - B a s i c f a c t s ( m i s s i n g f a c t o r ) 5 - N o t t e s t e d 8 x = 1 6 N o r e g r o u p i n g t e s t e d ( 2 ) 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s e . g . 3 1 2 x 3 No r e g r o u p i n g t e s t e d ( 1 ) 3 - d i g i t m u l t i p l i e r x 1 - d i g i t e . g . 1 0 0 x 4 = N o r e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t m u l t i p l i e r x 1 - d i g i t e . g . 8 x 1 0 R e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t m u l t i p l i e r x 1 - d i g i t e . g . 3 8 x 6 = A COMPARISON OF THE GRADE 4 ADDITION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f o u r t i r o e s i n t h e D T M S . * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n a n i t e m w a s p r e s e n t e d i n t h e t e s t . CTBS N o r e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t + 1 - d i g i t e . g . 1 2 + 3 N o r e g r o u p i n g t e s t e d ( 1 ) 3 o r m o r e 1 & 2 - d i g i t n u m b e r s e . g . 1 2 + 6 + 3 1 = R e g r o u p i n g 1 0 ' s t e s t e d ( 1 ) 2 - d i g i t + 1 - d i g i t e . g . 6 4 + 8 R e g r o u p i n g 1 0 ' s t e s t e d ( 1 ) 2 - d i g i t + 2 - d i g i t e . g . 5 7 + 34 = R e g r o u p i n g l ' s , 1 0 ' s t e s t e d t e s t e d ( 2 ) 2 - d i g i t + 2 - d i g i t e . g . 8 6 + 7 7 1 1 4 TABLE A-6 - Cont'd DTMS CTBS R e g r o u p i n g l ' s , 1 0 ' s t e s t e d ( 2 ) C o l u m n o f 3 o r m o r e 2 - d i g i t n u m b e r s e . g . 1 8 1 7 1 6 + 1 5 N o r e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t + 3 - d i g i t e . g . 4 6 + 2 0 0 N o r e g r o u p i n g t e s t e d ( 1 ) 3 - d i g i t + 3 - d i g i t e . g . 1 2 1 + 3 4 6 1 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' s 1 - T e s t e d ( 1 ) 3 - d i g i t + 3 - d i g i t 8 6 6 + 2 4 3 V a r i a t i o n t e s t e d ( 1 ) e . g . 6 7 5 + 6 2 6 = _ 2 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' s 2 - N o t t e s t e d 3 - d i g i t + 1, 2, & 3 - d i g i t s 49 9 6 7 + 8 3 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' s 3 - N o t t e s t e d 3 - d i g i t + 3 - d i g i t 1 2 8 2 6 4 4 3 2 + 3 5 4 115 TABLE A-6 - Cont'd DTMS CTBS 4 - R e g r o u p i n g l ' s , 10*s, 100's 4. T e s t e d (1) 3 - d i g i t + 1, 2, o r 3 - d i g i t 29 356 218 + 97 5 - A m i x t u r e o f 1 t o 4 d i g i t s 5 - N o t t e s t e d R e g r o u p i n g a m i x t u r e l ' s , 10's, 100's 2349 + 368 6 - A m i x t u r e o f 2 t o 5 d i g i t s 6 - N o t t e s t e d R e g r o u p i n g a m i x t u r e l ' s , 10's, 100's, 1000's 73 + 54328 7 - A m i x t u r e o f 2 t o 5 d i g i t s 7 - N o t t e s t e d R e g r o u p i n g a m i x t u r e o f l ' s , 10's, 100's, 1000's 29644 1738 29 + 4667 8 - A d d i t i o n o f l i k e f r a c t i o n s 8 - N o t t e s t e d N o r e g r o u p i n g , n o r e d u c i n g ± + 2- r 4 ^ 4 TABLE A - 6 - Cont'd DTMS CTBS 9 - A d d i t i o n o f l i k e f r a c t i o n s 9 - Not t e s t e d No r e g r o u p i n g , no r e d u c i n g 117 TABLE A-7 A COMPARISON OF THE GRADE 4 SUBTRACTION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f i v e t i m e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS N o r e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t - 2 - d i g i t e . g . 3 8 - 1 4 N o r e g r o u p i n g t e s t e d ( 1 ) 3 - d i g i t - 2 - d i g i t e . g . 1 6 8 - 4 3 N o r e g r o u p i n g t e s t e d ( 1 ) 3 - d i g i t - 2 o r 3 - d i g i t e . g . 7 3 8 - 4 3 7 No r e g r o u p i n g t e s t e d ( 1 ) 4 - d i g i t - 4 - d i g i t e . g . 7 8 2 4 - 6 3 0 2 R e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t - 1 - d i g i t e . g . 5 3 - 5 R e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t - 1 - d i g i t e . g . 8 1 - 4 1 1 8 TABLE A-7 - Cont'd DTMS CTBS R e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t - 2 - d i g i t e . g . 5 8 - 2 9 = 1 - R e g r o u p i n g 1 0 ' s a n d / o r 1 0 0 ' s 1 - T e s t e d ( 4 ) 3 - d i g i t - 2 - d i g i t 8 0 8 - 6 9 2 - R e g r o u p i n g 1 0 ' s a n d / o r 1 0 0 ' s 2 - T e s t e d ( 1 ) 3 - d i g i t - 3 - d i g i t 6 6 9 - 3 7 8 3 - R e g r o u p i n g a m i x t u r e o f 3 - T e s t e d ( 1 ) 1 0 ' s , 1 0 0 ' s 4 - d i g i t - 3 o r 4 - d i g i t 9 7 3 1 5 6 9 0 - 6 2 9 7 - 1 2 8 4 - R e g r o u p i n g a m i x t u r e o f 4 - N o t t e s t e d 1 0 ' s , 1 0 0 ' s 4 - d i g i t - 3 o r 4 - d i g i t 7 5 7 4 9 6 9 8 - 2 6 5 1 - 9 8 0 5 - R e g r o u p i n g a m i x t u r e o f 5 - N o t t e s t e d 1 0 ' s , 1 0 0 ' s , 1 0 0 0 ' s , 1 0 0 0 0 ' s 7 0 2 3 0 8 0 5 4 5 - 4 2 2 1 - 3 9 6 4 4 119 TABLE A-7 - Cont'd DTMS CTBS 6 - No r e d u c i n g , s u b t r a c t i o n 6 - Not t e s t e d o f l i k e f r a c t i o n s 6_ _ 2 -8 8 7 - No r e d u c i n g , s u b t r a c t i o n 7 - Not t e s t e d o f l i k e f r a c t i o n s i 6 1 6 1 2 0 TABLE A-8 A COMPARISON OF THE GRADE 4 MULTIPLICATION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f i v e t i m e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS 1 0 8 x 8 M a t h f a c t s t e s t e d ( 2 ) N o r e g r o u p i n g t e s t e d ( 1 ) 1 - d i g i t x 2-digitB e . g . 4 2 x 4 N o r e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t s x 1 - d i g i t e . g . 8 x 1 0 1 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s N o r e g r o u p i n g 1 3 2 x 3 1 - T e s t e d ( 2 ) 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g l ' s o n l y 2 - T e s t e d <1) TABLE A -8 - Cont'd DTMS CTBS 3 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g 10's o n l y 141 x 6 3 - N o t t e s t e d 4 - 1 - d i g i t m u l t i p l i e r x 2 - d i g i t s R e g r o u p i n g l ' s a n d 10's 15 x 7 4 - N o t t e s t e d 5 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g l ' s , 10's 154 x 5 5 - N o t t e s t e d 6 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g a m i x t u r e o f l ' s , 10's, 100's 6 - N o t t e s t e d 293 x 6 7 - 1 - d i g i t m u l t i p l i e r x 4 - d i g i t s R e g r o u p i n g a m i x t u r e o f l ' s , 10's, 100's ' 7 - N o t t e s t e d 1062 x 8 1 2 2 TABLE A-8 - Cont'd DTMS CTBS 8 - 2 - d i g i t m u l t i p l i e r x 8 - T e s t e d ( 1 ) 2 - d i g i t s 30 x 52 N o r e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t s x 4 - d i g i t s e . g . 2 5 x 1 0 0 0 = N o r e g r o u p i n g t e s t e d ( 1 ) 3 - d i g i t s x 1 - d i g i t e . g 1 0 0 x 4 = R e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t s x 1 - d i g i t e . g . 3 8 x 6 = R e g r o u p i n g t e s t e d ( 1 ) 2 - d i g i t s x 3 - d i g i t s e . g . 5 2 3 x 5 6 1 2 3 TABLE A-9 A COMPARISON OF THE GRADE 4 DIVISION COMPUTATION SKILLS TESTED BY THE DTMS AND THE CTBS. * E a c h q u e s t i o n t y p e w a s p r e s e n t e d f i v e t i m e s i n t h e D T M S . * * T h e n u m b e r i n b r a c k e t s a f t e r t h e w o r d t e s t e d i n t h e C T B S c o l u m n i n d i c a t e s t h e n u m b e r o f t i m e s a n i t e m w a s p r e s e n t e d i n t h e t e s t . DTMS CTBS M a t h f a c t s t e s t e d ( 3 ) 1 - d i g i t d i v i s o r , n o r e m a i n d e r 1 - 1 - N o t t e s t e d 2 ) 2 8 2 - 2 - T e s t e d ( 1 ) 4 ) 4 4 8 3 - 3 - T e s t e d ( 1 ) 2 ) 6 4 4 8 1 - d i g i t d i v i s o r , w i t h r e m a i n d e r 4 - 4 - N o t t e s t e d 5 ) 1 4 5 - 5 - N o t t e s t e d 3 ) 5 5 3 6 - 6 - N o t t e s t e d 2 ) 8 0 4 1 124 APPENDIX B S e c t i o n I L i s t o f q u e s t i o n s w h i c h f o r t h e p u r p o s e s o f t h i s s t u d y a r e c o n s i d e r e d " M a t h F a c t s " . S e c t i o n I I T h e M a t h F a c t s T e s t s o f t h e D e l t a T e s t s o f M a t h e m a t i c s S k i l l s . S e c t i o n I I I L i s t o f C o m p u t a t i o n S k i l l s T a u g h t a t G r a d e s 2, 3 a n d 4 . S e c t i o n I V T h e C o m p u t a t i o n s T e s t s o f t h e D T M S . 125 Section I L i s t o f q u e s t i o n s w h i c h f o r t h e p u r p o s e s of t h i s s t u d y a r e c o n s i d e r e d " M a t h F a c t s " . A d d i t i o n F a c t s 0 + 1 = = 1 0 + 8 = = 8 3 + 9 = = 12 1 + 0 = 1 + 7 = 4 + 8 = 2 + 6 = 5 + 7 = 0 + 2 = = 2 3 + 5 = 6 + 6 = 1 + 1 = 4 + 4 = 7 + 5 = 2 + 0 = 5 + 3 = 8 + 4 = 6 + 2 = 9 + 3 = 0 + 3 = = 3 7 + 1 = 1 + 2 = 4 + 9 = = 13 2 + 0 = 0 + 9 = = 9 5 + 8 = 1 + 8 = 6 + 7 = 0 + 4 = = 4 2 + 7 = 7 + 6 = 1 + 3 = 3 + 6 = 8 + 5 = 2 + 2 = 4 + 5 = 9 + 4 = 3 + 1 = 5 + 4 = 4 + 0 = 6 + 3 = 5 + 9 = = 14 7 + 2 = 6 + 8 = 0 + 5 = = 5 8 + 1 = 7 + 7 = 1 + 4 = 9 + 0 = 8 + 6 = 2 + 3 = 9 + 5 = 3 + 2 = 1 + 9 = = 10 4 + 1 = 2 + 8 = 6 + 9 = = 15 5 + 0 = 3 + 7 = 7 + 8 = 4 6 = 8 + 7 = 0 + 6 = = 6 5 + 5 = 9 + 6 = 1 + 5 = 6 + 4 = 2 + 4 = 7 + 3 = 7 + 9 = = 16 3 + 3 = 8 2 = 8 + 8 = 4 + 2 = 9 + 1 = 9 + 7 = 5 + 1 = 6 + 0 = 2 + 9 = = 11 8 + 9 = = 17 3 + 8 = 9 + 8 = 0 + 7 = = 7 4 + 7 = 1 + 6 = 5 + 6 = 9 + 9 = = 18 1 + 6 = 6 + 5 = 2 + 5 = 7 + 4 = 3 + 4 = 8 + 3 = 4 + 3 = 9 + 2 = 5 + 2 = 6 + 1 = 7 + 0 = S u b t r a c t i o n F a c t s 1 - 0 = 1 - 1 = 2 - 0 = 2 - 1 = 2 - 2 = 3 - 0 = 3 - 1 = 3 - 2 = 3 - 3 = 4 - 0 = 4 - 1 = 4 - 2 = 4 - 3 = 4 - 4 = 5 - 0 = 5 - 1 = 5 - 2 = 5 - 3 = 5 - 4 = 5 - 5 = 6 - 0 = 6 - 1 = 6 - 2 = 6 - 3 = 6 - 4 = 6 - 5 = 6 - 6 = 7 - 0 = 7 - 1 = 7 - 2 = 7 - 3 = 7 - 4 = 7 - 5 = 7 - 6 = 7 - 7 = 8 - 0 = 8 - 1 = 8 - 2 = 8 - 3 = 8 - 4 = 8 - 5 = 8 - 6 = 8 - 7 = 8 - 8 = 9 - 0 = 9 - 1 = 9 - 2 = 9 - 3 = 9 - 4 = 9 - 5 = 9 - 6 = 9 - 7 = 9 - 8 = 9 - 9 = 10 - 1 = 10 - 2 = 10 - 3 = 10 - 4 = 10 - 5 = 10 - 6 = 10 - 7 = 10 - 8 = 10 - 9 = 11 - 2 = 11 - 3 = 11 - 4 = 11 - 5 = 11 - 6 = 11 - 7 = 11 - 8 = 11 - 9 = 12 - 3 12 - 4 12 - 5 12 - 6 12 - 7 12 - 8 12 — 9 13 _ 4 13 - 5 13 - 6 13 - 7 13 - 8 13 - 9 14 — 5 14 - 6 14 - 7 14 - 8 14 - 9 15 _ 6 15 - 7 15 - 8 15 - 9 16 _ 7 16 - 8 16 - 9 17 _ 8 17 - 9 18 _ 9 127 M u l t i p l i c a t i o n F a c t s 0 x 0 = 4 x 0 = 8 x 0 = 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9 1 x 0 = 5 x 0 = 9 x 0 = 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9 2 x 0 = 6 x 0 = 1 0 x 1 0 = 100 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 3 x 0 = 7 x 0 = 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 128 81 72 64 63 56 54 49 48 - 9 = - 9 = - 8 = - 8 = - 9' = - 7 = - 8 = - 8 = - 9 = - 6 = - 7 = - 8 = - 6 = 45 -42 -40 -36 -35 -32 -30 -28 -27 -9 5 7 6 8 5 9 4 6 7 5 8 4 6 5 7 4 D i v i s i o n F a c t s 25 - 5 = 24 - 8 = - 3 = - 6 = - 4 = 21 -20 -18 -16 -15 -14 -12 -10 -9 -8 -7 3 5 4 9 2 6 3 8 2 4 5 3 7 2 6 2 4 3 5 2 3 1  9 - 3  4 - 2 - 1 7 - 1 6 - 3 - 2 - 1 5 4 3 2 1 - 1 2 = 1 = 1 = 1 = 1 = 1 2 9 Section II The Math Facts Tests of the Delta TestB of Mathematics S k i l l s . O n l y o n e t e s t f o r e a c h f u n c t i o n w a s p r e p a r e d . T h i s t e s t w a s u s e d a t e a c h g r a d e w h e r e t h e f u n c t i o n w a s t e s t e d . G r a d e 2 G r a d e JJ G r a d e 4 A d d i t i o n X X X S u b t r a c t i o n X X X M u l t i p l i c a t i o n X X D i v i s i o n X 130 SUfHRALTIf)N FACTS 131 9 - 5 - 1 1 - 6 - 9 - 6 « 1 0 - 3 - 8 - 3 • 1 0 - 2 -7 - 6 - 5 - 2 - lk - 5 -1 5 - 8 " 1 5 - 9 - 6 - 2 -1 3 - 4 - 1 2 - 7 - 6 - 6 -1 1 - 7 - 9 - H - 1 2 - 8 . 5 - n . 1 1 - 3 - 5 - 1 . 1 2 - 5 - 7 - 1 - 1 7 - q . 4 - 2 - 1 2 - 9 - n • R . 8 - 7 • 9 - 0 - 1 1 - ii . 7 - 3 - 3 - 2 - q - ? • 1 0 - 8 - 1 «» - 8 • 1 D - 7 . 1 H - C - 1 1 - 9 - 1 u - q . 1 6 - 9 - 8 - 5 • 1 0 - 1 1 5 - 6 - 1 0 - 9 - H - X 8 - 1 - 1 6 - 7 - I T - ft 9 - 8 - 1 0 - U - 1 9 - K 10 - 5 - 1 fl - 9 - fi - •> 6 - 3 - 1 2 - U - 1 1 - ? 1 3 - 7 - 7 - 5 • 1 JI - 7 1 2 - 3 - 1 - 1 - 8 - fi 1 0 - C - 9 - 7 - 1 5 - 7 n - 9 • 1 3 - 5 - K . Ii . q - i - 1 6 - 8 - n . H , 1 7 - 9 - 7 - 1 - ' R . Ii 6 - 5 - . 11 - 8 -132 1 x S - . 5 x 9 -3 x 3 -8 x 9 -6 x 3 -9 x 5 -2 x 3 -Hill Tl PI I fA™" FAfTf\ 9 x 8 - | 2 x 1 -4 x 7 -5 x 2 -7 x. 6 -7 x 7 -3 x 9 -8 x 2 -7 x 9 -6 x 1 -i\ x 1 -5 x 8 -2 x 2 - . 9 x 6 -3 x 6 -7 x 2 -8 x 1 -1 x C -5 x 5 -4 x 3 -2 x 7 -8 x 7 - . 2 x 9 -9 x 3 -3 x 5 " 6 x 9 ' 6 x 7 - . 9 x 2 - . 7 x 1 m q x 5 -5 x 6 " 8 x 3 " 3 x 7 - . 2 x 5 - . 6 x S -H x 1 -5 x 3 -3 x 8 -8 x 6 " 7 x 5 ' 9 x 7 - . 5 x 3 -7 x 8 -6 x 6 -3 x 4 ' 2 x 6 ' 9 x 9 8 x 5 H x t ° -1 x 9 " . 7 x 8 -3 x 2 -5 x 1 -2 x 8 -0 x 7 ' 6 x 5 9 x 1 " 1 x 2 - . 7 x 3 -6 x 2 -5 x 7 -133 1 0 6 3 1 G t 2 2 1 3 2 1 8 1 1 7 2 8 H 8 2 7 5 k 3 0 2 1 1 2 3 C C 3 5 1 2 H 2 5 8 5 «4 D1V1S1M FACTS TO 31 2 1 1 8 2 0 3 0 C 1 0 1 0 1 6 1 2 2 8 6 1 2 1 2 8 1 1 5 6 5 6 4 7 3 6 i 6 2 1 f 7 2 5 ^ 5 2 8 r H 3 5 f 7 1 5 •J 1 8 6 3 2 0 1 0 2 8 1 8 5 M 4 0 2 H 2 7 8 7 2 3 6 1 4 H 5 3 2 4 9 1 6 8 1 8 f 9 r H • 7 8 f 2 i 9 1 3 4 Section III L i s t of Computation S k i l l s Taught at Each Grade Level. G r a d e "2_ A d d i t i o n 1 - C o m p u t e s u m s 2 + 4 + 2 2 - C o m p u t e s u m s 2 4 + 2 3 - M i s s i n g a d d e n d 4 + 4 - M i s s i n g a d d e n d 4 + 5 - N o r e g r o u p i n g - 1 1 2 - d i g i t + 1 - d i g i t + 2 6 - N o r e g r o u p i n g - 1 4 2 - d i g i t + 2 - d i g i t + 24 7 - R e g r o u p i n g l ' s 2 4 2 - d i g i t + 1 - d i g i t + 6 8 - R e g r o u p i n g l ' s 2 7 2 - d i g i t + 2 - d i g i t + 24 G r a d e 2 S u b t r a c t i o n 1 - M i s s i n g n u m e r a l 9 - = 4 - 4 = 5 2 - M i s s i n g n u m e r a l 9 - 5 4 4 3 - N o r e g r o u p i n g -2 - d i g i t - 1 - d i g i t 2 7 - 6 G r a d e 2 S u b t r a c t i o n ( C o n t ' d ) 4 - No r e g r o u p i n g -2 - d i g i t - 2 - d i g i t 5 - R e g r o u p i n g 1 0 * s -2 - d i g i t - 1 d i g i t 6 - R e g r o u p i n g 1 0 ' s G r a d e 3_ A d d i t i o n 1 - C o m p u t e s u m s t o 1 8 2 - C o m p u t e s u m s t o 1 8 3 - C o m p u t e s u m s 18+ 4 - M i s s i n g a d d e n d 5 - W i t h o u t r e g r o u p i n g -2 - d i g i t + 2 - d i g i t 6 - W i t h r e g r o u p i n g l ' s o n l y 2 - d i g i t + 2 - d i g i t 7 - W i t h r e g r o u p i n g - 1 0 ' s o n 2- d i g i t + 2 - d i g i t 8 - R e g r o u p i n g l ' s o r 1 0 ' s 3- d i g i t + 2 / 3 - d i g i t s 9 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' 3 - d i g i t + 2 / 3 - d i g i t s i n c o l u m n o f 3 n u m b e r s G r a d e 3 S u b t r a c t i o n 136 1 - N o r e g r o u p i n g 2 - d i g i t - 1/2 d i g i t s 32 - 2 20 - 10 2 - No r e g r o u p i n g 3 - d i g i t s - 1, 2, o r 3 426 313 158 - 17 279 - 8 3 - R e g r o u p i n g 10's 2 - d i g i t - 1 - d i g i t 53 4 4 - R e g r o u p i n g 10's 2 - d i g i t - 2 - d i g i t 36 19 5 - R e g r o u p i n g 10's 3 - d i g i t - 2/3 d i g i t 182 - 69 582 - 166 6 - R e g r o u p i n g 100's 3 - d i g i t - 2 - d i g i t 628 - 84 7 - R e g r o u p i n g 100's 3 - d i g i t - 3 - d i g i t 439 284 8 - R e g r o u p i n g 10's a n d / o r 100's 3 - d i g i t - 2 - d i g i t 735 86 9 - R e g r o u p i n g 10's a n d / o r 100's 3 - d i g i t - 3 - d i g i t 653 284 G r a d e 3_ M u l t i p l i c a t i o n 1 - B a s i c f a c t s t o 50 9 x 5 2 - B a s i c f a c t s ( m i s s i n g f a c t o r ) 8 x G r a d e 3 M u l t i p i i c a t i o n ( C o n t ' d ) 3 - No r e g r o u p i n g 24 1 - d i g i t m u l t i p l i e r x 2 x 2 - d i g i t s 4 - B a s i c f a c t s t o 5 0 5 x 9 = 5 - B a s i c f a c t s ( m i s s i n g f a c t o r ) 8 x G r a d e j4 A d d i t i o n 1 - R e g r o u p i n g l ' s , 10's, 100's 8 6 6 3 - d i g i t + 3 - d i g i t + 2 4 3 2 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' s 4 9 3 - d i g i t + 1 , 2, & 3 - d i g i t s 9 6 7 + 8 3 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' s 1 2 8 3 - d i g i t + 3 - d i g i t 2 6 4 4 3 2 + 3 5 4 4 - R e g r o u p i n g l ' s , 1 0 ' s , 1 0 0 ' s 2 9 3 - d i g i t + 1, 2, o r 3 - d i g i t 3 5 6 2 1 8 + 9 7 5 - A m i x t u r e o f 1 t o 4 d i g i t s 2 3 4 9 R e g r o u p i n g a m i x t u r e l ' s , + 3 6 8 1 0 ' s , 1 0 0 ' s 6 - A m i x t u r e o f 2 t o 5 d i g i t s 7 3 R e g r o u p i n g a m i x t u r e l ' s , + 5 4 3 2 8 10's, 100's, 1000's G r a d e 4 A d d i t i o n ( C o n t ' d ) 7 - A m i x t u r e o f 2 t o 5 d i g i t s 29644 R e g r o u p i n g a m i x t u r e o f l ' s , 1738 10's, 100's, 1000's 29 + 4667 138 8 - A d d i t i o n o f l i k e f r a c t i o n s N o r e g r o u p i n g , n o r e d u c i n g 1 4 2 + -4 9 - A d d i t i o n o f l i k e f r a c t i o n s _2_ N o r e g r o u p i n g , n o r e d u c i n g 5 JL_ + 5 G r a d e 4 S u b t r a c t i o n 1 - R e g r o u p i n g 10's a n d / o r 100's 808 3 - d i g i t - 2 - d i g i t - 69 2 - R e g r o u p i n g 10's a n d / o r 100's 669 3 - d i g i t - 3 - d i g i t - 378 3 - R e g r o u p i n g a m i x t u r e o f 10*s, 100's 4 - d i g i t - 3 or 4 - d i g i t 9731 6297 5690 - 128 4 - R e g r o u p i n g a m i x t u r e o f 10's, 100's 4 - d i g i t - 3 o r 4 - d i g i t 7574 2651 9698 • 980 5 - R e g r o u p i n g a m i x t u r e o f 10's, 100's, 1000's, 10000's 70230 - 4221 80545 - 39644 G r a d e 4 S u b t r a c t i o n ( C o n t ' d ) 6 - N o r e d u c i n g , s u b t r a c t i o n o f l i k e f r a c t i o n s 7 - N o r e d u c i n g , s u b t r a c t i o n o f l i k e f r a c t i o n s G r a d e 4^  M u l t i p l i c a t i o n 1 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s N o r e g r o u p i n g 2 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g l ' s o n l y 3 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g 1 0 ' s o n l y 4 - 1 - d i g i t m u l t i p l i e r x 2 - d i g i t s R e g r o u p i n g l ' s a n d 1 0 ' s 5 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g l ' s , 1 0 ' s 6 - 1 - d i g i t m u l t i p l i e r x 3 - d i g i t s R e g r o u p i n g a m i x t u r e o f l ' s , 1 0 ' s , 1 0 0 ' s 7 - 1 - d i g i t m u l t i p l i e r x 4 - d i g i t s R e g r o u p i n g a m i x t u r e o f l ' s , 1 0 ' s , 1 0 0 ' s 140 Grade 4 M u l t i p l i c a t i o n (Cont'd) 8 - 2 - d i g i t m u l t i p l i e r x 30 2 - d i g i t s x 52 Grade 4 D i v i s i o n 1 - d i g i t d i v i s o r , no r e m a i n d e r 1 -2) 28 2 -4) 448 3 -2) 6448 1 - d i g i t d i v i s o r , w i t h r e m a i n d e r 4 -5) 14 5 -3) 553 6 -2) 8041 141 Section IV The Computations Subtests of the DTMS. *The number i n b r a c k e t s b e s i d e e a c h q u e s t i o n c o r r e s p o n d s t o t h e number o f t h e q u e s t i o n t y p e as i d e n t i f i e d i n S e c t i o n I I I o f t h i s A p p e n d i x . 00 cn 1 0 a- K-\ L O ST C M K > ^ C M CM cn • cn | oo i—i • u L A + C O • L A 00 oo cn LO CSI O l I I A • LO 0 0 • r o L A CN —^' N K \ N • C M O C M cn L A •—I oo CA L A •» • • OO CO •f " 3 = • 1 cn • cn •» o I f l J l \ 4 o o • O0 L A C M cn 143 to 3_ 5." O CD /5 K> OO cn LA i 10 • • L A OO i n cn rvi L A g 3D CM c r c n CNI LO *—' L A • CA CD LD LTV • oo —I i c n L A v -<M L A L A .-VI I f A CO —^' K> cn en a-i m cn i • LA CO OO o o I r v . CO CO L A CM OO f A I L A CO co a-• cn »A = • 3 GRADE 3 ADDITION COMPUTATIONS (6) 3 8 43 4 8 7 • 5 (3) (2) 8 -(1) (4) 9 • D 1 8 7 7 2 4 5 7 + 19 8 (9) (8) 3 17 3 2 5 6 1 +24 (5) (7) 3 9 • 7 7 (6)9 5 3 9 5 9 • 2 2 + 8 7 6 (9) (1) • (4) (7) 17 6 5 • 7 9 (3) (8) (2) (5) 5 9 9 + 321 • 7 0 + 15 (2) (A) 6 • 5 • 5 6 *\^\* 1 - 1 3 1 2 7 •12 9 (8) 7 5 • 8 (3) 2 8 + 2 (1) (5) (7) 7 6 • 2 1 8 3 + 67 2 8 6 3 2 5 f g 9 (9) (6) 2 8 • 3 5 (5) 5 3 • 2 5 9 1 + 6 (1) (4) + 2 1 5 5 1 1 3 2 1 • 667 (9) 7 6 + 9 (3) 3 6 + 6 5 2 1 5 + 666 (8) 5 7 • 2 8 (6) • (2) 8 8 8 1 6 y 9 7 (9) (5) 1 2 + 3 7 a- (4) (7) (8) (6) 1 3 7 6 • 5 4 6 0 8 + 96 1 7 + 2 6 (2) 8 9 • 6 (3) 1 9 • 5 (1) GRADE 3 SUBTRACTION COMPUTATIONS ( n (7) (3) (2) '(9) (4) (6) (5) (8) 1 9 u ; 9 1 6 u ; 8 1 v ; 815 923 91 202 223 988 -15 -161 -6 -13 -172 -83 -12 -219 - 8 3 7 0 (3) 9 5 2 (2) 1 0 7 (8) , 5 (i) 6 7 (A> 2 7 2 ( 9 ) 5 7 5 ( 7 ) 6 2 8 ( 6 ) 5 5 7 ( 5 ) -6 -121 -18 -3 -18 -133 -281 - 8 1 - 2 8 1 1 0 (9) 7 3 1 (5) 5 0 7 (7) 9 7 O) 6 7 g (8) , 2 6 (2) 7 2 (4) ? 3 (1) , „ 9 (6) -395 -303 -152 - 9 - 9 9 - 5 -56 -32 -72 5 9 3 (5) 1 0 7 (8) 9 0(3) 7 2 3 (6) 8 1 8 <2) 1 1 8 <7) 1 5 2 (9> 5 1 <*> 9 1 (1> -27 -67 -3 -31 -105 -383 -327 -16 -1 8 3 (4) 5 1 6 (8) 8 0 6 (6) 5 0 0 (7) 1 5 8 (2) 3 19 <9> 5 1 1 <5) 5 3 (3) 8 8 (1> -21 -31 -73 -280 -17 -126 -538 -1 -21 GRADE 3 MULTIPLICATION' COMPUTATIONS (1) Q (2) (4) (5) ' (3) (2) 7 -I I 3 X - 2 1 x I x Q 6 x k x k k x T e 3 6 , „ (3) (4) (1) (5) (1) (3) 3 2 . 1 8 8 3 0 x 3 6 x 7 - x 6 x 6 - 2 «f x 5 x 3 2 x [ ] . ] 8 ( 5 ) e X 6 . [ ] w jff (2) r ] x 8 " 3 2 ( 5 ) £1 (2) i j (1) 2«t 28 S x 5 W 2 3 x 3 (3) 2 4 x 2 (3) 3 JO 2 7 (2) (5) H 5 k x 8 • (4) x 6 (1) 147 c o u r. o V =3 OJ m 6 *—^  in + vo a. ><r o c « IN ONMfl r» co «r r~ m w-i en r» CN 00 O OA v o n r - l m ^ •+ v—<-«f Ninon cn co in co co m vo r» a> vo ^ «• v D m r - l oo fN r-<-t in in f—t oo GO vo o r» c o H H o v n (N i-H fN r-l vo r» rH fN m + 00 r-t IN I*. 00 VO 00 + —^\ <T V ' an m on oo 00 m r» r-t P-an r-l an m .—^ + fN r-t 00 fN O fN fN ft U> fN r» •v fN m n IN + r-l vo m an in + cn s—' CN >N n | r -— + co cn oo fN 10 m r-t + r-t o 00 in vo an r-t + o in "» O in m r » rH + on n IN r -m v O in vo 00 IN an r-r-t 00 r-l + m cn vo an m r-co f r» o m o vo m on on oo m CN an CN vo oo m r m r » » o cn r- vo r~ m o on en «N r-t fN m r-t 00 + o on m m m vo CN i n m OS vo + 00 «* cn rH CN fN rH «—« v O •—-+ r » CO vo cn an CN m IN 00 rH + rH —' rH on rn r~ VO m + «»• m ao vo cn an fN vo r » O rH fN o CN m a + vo m on oo CM on IN m + CO r - m on 00 n an o on r « m on /•—V + 00 Cn s m f c o ,—* U + rH |m * |C0 rH f-m so SO rH o m ao m co in + 10 r -OB c n m IN r - CO 1 OQ on O co to l r—\ CM CO 00 n m lA • n ut M* r~ «*1 CM CO cn (N 1 I O <H r - CA CO CN oa r-l r» so cn m i o o o on o n IN h** plcn <N1CT\ I CO M" lA 00 CI CO on r-r-t I r-i r~ O fN o CO o r> m V ca rH CO so CN cn 1 r-t cn r-l fN CN n m oa on co r» cn r-l l co r~ oo o\ i k v o | c n i co r» cn r> n i I in. co i n r -o cn O an O o oa in I o r-t o r» *r aa co cn l Imkn i-tUn t o cn CO CN in I co r» M * m r» on cn CN I cD I co an o oa co l co en m r-t\ i 149 «-< x m CM m m CN VO >—-CN m 0\ 00 -3-00 CO *r oo CO X 00 M ' m CO VO CO CO • o CN . rH CN CN CO CN H VO o CN r- vo m CN 00 —^' en r-VO N—' n vo cn CN VO VO o CN vO >—' r» m CN m CN n on X oo 00 •J § U 6 a 5 ro r-Cf» CN X f> CN vo VO CN vo r~ on X cn • CN co vo m cn vo m fN m CN CO r- oo in vo •8 ^ in cn in CN CO CN co VO o Grade 4 Division Computations 3 T e o i r ( 3 ) 6 T 9 3 — W 9jn^~(6) 4TTor- ( 5 ) e T W ^ 4 T 6 8 — ( 1 ) iVn—t" ijen 9imrW 7)1848 ( 3 ) 3 T F 5 ( A ) 77ToT- ( 2 ) SjTtW*^ 4 ) 3 9 1 ( 5 ) &JWfZ~*6^ 7JW (1) (2) • _ (4) (2) (3) (1) • (6) 2751 7)602 8;TS <)348 9)7083 3751 7")5Z28" 9T&6T (6) (A) (1) . (2) (5) i (3) 6)4583 9733 6T5H 9 ) 7 2 9 8 ) 6 3 7 9 ) 6 5 7 0 151 APPENDIX C The instructions given to pupils being administered the Delta Tests of Mathematics S k i l l s . S e c t i o n I D i r e c t i o n s : M a t h F a c t s ( C o m p l e t e a n d S h o r t F o r m ) S e c t i o n I I : D i r e c t i o n s : C o m p u t a t i o n s S k i l l s ( C o m p l e t e a n d S h o r t F o r m ) 152 Section I: Directions: Math Facts (Complete) E x a m i n e r : P l e a s e l e a v e y o u r b o o k l e t s c l o s e d a n d l o o k a t m i n e . W h e n I s a y b e g i n , I w a n t y o u t o s t a r t o n t h e f i r s t q u e s t i o n o n t h e t o p l e f t h a n d c o r n e r o f t h e p a g e . [ E x a m i n e r i n d i c a t e s f i r s t q u e s t i o n o n s a m p l e p r o b e ] O n t h i s p a g e d o t h e q u e s t i o n s d o w n t h e p a g e . I f y o u g e t t o t h e b o t t o m o f t h e c o l u m n s t a r t o n t h e n e x t c o l u m n . [ E x a m i n e r d e m o n s t r a t e s ] D o n o t s k i p q u e s t i o n s . L o o k c a r e f u l l y a t e a c h q u e s t i o n . I f y o u c o m e t o a q u e s t i o n y o u a r e s u r e y o u d o n o t k n o w h o w t o d o y e t , p u t a . b i g "X" o n i t a n d g o o n t o t h e n e x t q u e s t i o n . [ E x a m i n e r d e m o n s t r a t e s ] E x a m i n e r : O n t h i s p a g e y o u d o t h e q u e s t i o n s d o w n t h e p a g e . W h a t d o y o u d o i f y o u c o m e t o a q u e s t i o n y o u d o n o t k n o w h o w t o d o y e t ? S t u d e n t s : P u t a b i g "X" o n i t a n d g o o n t o t h e n e x t q u e s t i o n . E x a m i n e r : A l l t h e q u e s t i o n s o n t h i s p a g e a r e ( a d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) q u e s t i o n s . W h a t k i n d o f q u e s t i o n s a r e y o u d o i n g o n t h i s p a g e ? S t u d e n t s : ( A d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) q u e s t i o n s . E x a m i n e r : Y o u w i l l h a v e o n e m i n u t e t o w o r k o n t h e s e q u e s t i o n s . Y o u a r e n o t e x p e c t e d t o b e a b l e t o c o m p l e t e t h e p a g e i n t h e t i m e y o u h a v e B U T y o u a r e e x p e c t e d t o w o r k a s f a s t a s y o u c a n . I f , h o w e v e r , y o u d o a l l t h e q u e s t i o n s y o u c a n b e f o r e t i m e i s u p , c h e c k y o u r w o r k o n t h i s p a g e B U T d o n o t t u r n t o a n o t h e r p a g e . A r e y o u e x p e c t e d t o f i n i s h t h e p a g e ? 1 5 3 S t u d e n t s : N o . E x a m i n e r : M a y y o u t u r n t o a n o t h e r p a g e ? S t u d e n t s : N o . E x a m i n e r : A r e y o u e x p e c t e d t o w o r k a s f a s t a s y o u c a n a n d d o y o u r b e s t w o r k ? S t u d e n t s : Y e s . E x a m i n e r : R e m e m b e r , w o r k q u i c k l y a n d d o y o u r b e s t w o r k . D o n o t s k i p q u e s t i o n s . D o e a c h q u e s t i o n o n e a f t e r t h e o t h e r d o i n g d o w n t h e p a g e a n d i f y o u c o m e t o a q u e s t i o n y o u c a n n o t d o y e t p u t a b i g "X" o n i t a n d g o o n t o t h e n e x t q u e s t i o n . A s w e l l , r e m e m b e r t h i s i s a ( a d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) p a g e . W h a t k i n d o f p a g e i s t h i s ? S t u d e n t s : ( A d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) p a g e . E x a m i n e r : P l e a s e t u r n t o p a g e 1 . [ W h e n t h e c h i l d r e n h a v e t u r n e d t o t h e c o r r e c t p a g e ] E x a m i n e r : Y o u m a y b e g i n . 154 Directions: Math Facts (Short Form) E x a m i n e r : S t u d e n t s : E x a m i n e r : S t u d e n t s : E x a m i n e r : S t u d e n t s : E x a m i n e r : S t u d e n t s : E x a m i n e r : [ W h e n t h e E x a m i n e r : T h e n e x t p a g e i s a ( a d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) p a g e . O n t h i s p a g e d o t h e q u e s t i o n s d o w n t h e p a g e . Y o u w i l l h a v e o n e m i n u t e t o w o r k o n i t . A r e y o u e x p e c t e d t o f i n i s h t h e p a g e ? N o . M a y y o u t u r n t o a n o t h e r p a g e ? N o . A r e y o u e x p e c t e d t o w o r k a s f a s t a s y o u c a n a n d d o y o u r b e s t w o r k ? S t u d e n t s : Y e s . E x a m i n e r W h a t d o y o u d o w h e n y o u c o m e t o a q u e s t i o n y o u c a n ' t d o y e t ? P u t a b i g "X" o n i t a n d g o o n t o t h e n e x t q u e s t i o n . W h a t k i n d o f t e s t i s t h i s ? ( A d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n d i v i s i o n ) p a g e . P l e a s e t u r n t o p a g e . c h i l d r e n h a v e t u r n e d t o t h e c o r r e c t p a g e ] Y o u m a y b e g i n . 155 Section II; Directions; Computation S k i l l s (Complete) E x a m i n e r : P l e a s e l e a v e y o u r b o o k l e t s c l o s e d and l o o k a t m i n e . When I s a y b e g i n , I want y o u t o s t a r t on t h e f i r s t q u e s t i o n on t h e t o p l e f t hand c o r n e r o f t h e p a g e s . [ E x a m i n e r i n d i c a t e s f i r s t q u e s t i o n on s a m p l e p r o b e ] On t h i s page do t h e q u e s t i o n s a c r o s s t h e p a g e . I f y o u g e t t o t h e end o f t h e row s t a r t on t h e n e x t r o w . [ E x a m i n e r d e m o n s t r a t e s ] Do n o t s k i p q u e s t i o n s . Look c a r e f u l l y a t e a c h q u e s t i o n . I f y o u come t o a q u e s t i o n y o u a r e s u r e y o u do n o t know how t o do y e t , p u t a b i g " X " on i t and go on t o t h e n e x t q u e s t i o n . [ E x a m i n e r d e m o n s t r a t e s ] E x a m i n e r : On t h i s page y o u do t h e q u e s t i o n s a c r o s s t h e p a g e . What do y o u do i f y o u come t o a q u e s t i o n y o u do n o t know how t o do y e t ? S t u d e n t s : P u t a b i g " X " on i t and go on t o t h e n e x t q u e s t i o n . E x a m i n e r : A l l t h e q u e s t i o n s on t h i s page a r e ( a d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) q u e s t i o n s . What k i n d o f q u e s t i o n s a r e y o u d o i n g on t h i s p a g e ? S t u d e n t s : ( A d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) q u e s t i o n s . E x a m i n e r : You w i l l h a v e f i v e m i n u t e s t o work on t h e s e q u e s t i o n s . You a r e n o t e x p e c t e d t o be a b l e t o c o m p l e t e i n t h e t i m e y o u h a v e BUT y o u a r e e x p e c t e d t o w o r k a s f a s t a s y o u c a n . I f , h o w e v e r , y o u do a l l t h e q u e s t i o n s y o u c a n b e f o r e t ime, i s u p , c h e c k y o u r wo rk on t h i s page BUT do n o t t u r n t o a n o t h e r p a g e . A r e y o u e x p e c t e d t o f i n i s h t h e p a g e ? S t u d e n t s : N o . E x a m i n e r : May y o u t u r n t o a n o t h e r p a g e ? S t u d e n t s : N o . 156 E x a m i n e r : A r e y o u e x p e c t e d t o w o r k a s f a s t a s y o u c a n a n d d o y o u r b e s t w o r k ? S t u d e n t s : Y e s . E x a m i n e r : R e m e m b e r , w o r k q u i c k l y a n d d o y o u r b e s t w o r k . D o n o t s k i p q u e s t i o n s . D o e a c h q u e s t i o n o n e a f t e r t h e o t h e r g o i n g a c r o s s t h e p a g e a n d i f y o u c o m e t o a q u e s t i o n y o u c a n n o t d o y e t p u t a b i g "X" o n i t a n d g o o n t o t h e n e x t q u e s t i o n . A s w e l l , r e m e m b e r t h i s i s a a d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) p a g e . W h a t k i n d o f p a g e i s t h i s ? S t u d e n t s : ( A d d i t i o n / s u b t r a c t i o n / r o u l t i p l i c a t i o n / d i v i s i o n ) p a g e . E x a m i n e r : P l e a s e t u r n t o P a g e 2. [ W h e n t h e c h i l d r e n h a v e t u r n e d t o t h e c o r r e c t p a g e ] E x a m i n e r : Y o u m a y b e g i n . 157 D i r e c t i o n s : C o m p u t a t i o n S k i l l s ( S h o r t F o r m ) E x a m i n e r : S t u d e n t s : E x a m i n e r : S t u d e n t s : E x a m i n e r : S t u d e n t s : E x a m i n e r : S t u d e n t s : E x a m i n e r : S t u d e n t s : E x a m i n e r : [ W h e n t h e E x a m i n e r : T h e n e x t p a g e i s a J a d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n / d i v i s i o n ) p a g e . O n t h i s p a g e d o t h e q u e s t i o n s a c r o s s t h e p a g e . Y o u w i l l h a v e f i v e m i n u t e s t o w o r k o n i t . A r e y o u e x p e c t e d t o f i n i s h t h e p a g e ? N o . M a y y o u t u r n t o a n o t h e r p a g e ? N o . A r e y o u e x p e c t e d t o w o r k a s f a s t a s y o u c a n a n d d o y o u r b e s t w o r k ? Y e s . W h a t d o y o u d o w h e n y o u c o m e t o a q u e s t i o n y o u c a n ' t d o y e t ? P u t a b i g "X" o n i t a n d g o o n t o t h e n e x t q u e s t i o n . W h a t k i n d o f t e s t i s t h i s ? ( A d d i t i o n / s u b t r a c t i o n / m u l t i p l i c a t i o n d i v i s i o n ) q u e s t i o n s . P l e a s e t u r n t o p a g e . c h i l d r e n h a v e t u r n e d t o t h e c o r r e c t p a g e ] Y o u m a y b e g i n . 158 APPENDIX D Descriptive S t a t i s t i c s of Individual Functions. S e c t i o n I : T e s t - R e t e s t R e l i a b i l i t y C o e f f i c i e n t s B e t w e e n S e s s i o n s . S e c t i o n I I ; T h e S e s s i o n a l M e a n s , S t a n d a r d D e v i a t i o n s a n d N u m b e r o f V a l i d C a s e s o f t h e I n d i v i d u a l F u n c t i o n s T e s t e d w i t h t h e DTMS a t G r a d e s 2, 3 a n d 4. S e c t i o n I I I : T h e A n a l y s i s o f V a r i a n c e S u m m a r i e s a n d T u k e y ' s H S D ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r t h e DTMS T e s t S c o r e s , S e s s i o n s 1 t o 4. S e c t i o n I V : T r e n d A n a l y s i s S u m m a r i e s a n d r , t h e P r o p o r t i o n o f T o t a l V a r i a t i o n A c c o u n t e d f o r b y S i g n i f i c a n t T r e n d s f o r t h e DTMS S e s s i o n a l M e a n s . Section I: Test-ReteBt R e l i a b i l i t y Coefficients Between Sessions. The t e s t - r e t e s t r e l i a b i l i t y c o e f f i c i e n t s b e t w e e n s e s s i o n s we re c a l c u l a t e d u s i n g t h e " P e a r s o n C o r r ' command o f t h e U . B . C . S t a t i s t i c a l P a c k a g e f o r t h e S o c i a l S c i e n c e s e x t e n d e d v e r s i o n ( S P S S x ) . TABLE D-l GRADE 2 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES, ADDITION AND SDBTRACTION FACTS TESTS OF THE DTMS. 160 Addition Facts SESSION 1 2 3 4 1 1.00 .75 .70 .70 2 1.00 .78 .75 3 1.00 .77 4 1.00 Subtraction Facts SESSION 1 2 3 4 1 1.00 .83 .76 .74 2 1.00 .89 .82 3 1.00 .86 4 1.00 TABLE D-2 GRADE 2 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES, ADDITION AND SUBTRACTION COMPUTATION SKILLS TESTS OF THE DTMS. 161  A d d i t i o n Computation S k i l l s SESSION 1 2 3 4 1 1.00 .69 .62 .63 2 1.00 .73 .74 3 1.00 .73 4 1.00 S u b t r a c t i o n Computation S k i l l s SESSION 1 2 3 4 1 1.00 .65 .59 .54 2 1.00 .74 .66 3 1.00 .80 4 1.00 162 TABLE D-3 GRADE 3 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES, ADDITION, SDBTRACTION AND MULTIPLICATION FACTS TESTS OF THE DTMS. A d d i t i o n F a c t s SESSION 1 2 3 4 1 1 .00 .64 .62 . 57 2 1 .00 . 7 9 .74 3 1 .00 . 78 4 1 .00 S u b t r a c t i o n F a c t s SESSION 1 2 3 4 1 1 .00 . 79 .76 . 7 5 2 1 .00 .81 . 77 3 1 .00 . 7 9 4 1 .00 M u l t i p l i c a t i o n F a c t s SESSION 1 2 3 4 1 1 .00 .70 . 65 . 5 9 2 1 .00 . 79 . 75 3 1 .00 . 82 4 1 .00 163 TABLE D-4 GRADE 3 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES, ADDITION, SUBTRACTION AND MULTIPLICATION COMPUTATION SKILLS TESTS OF THE DTMS. A d d i t i o n C o m p u t a t i o n S k i l l s S E S S I O N 1 2 3 4 1 1.00 .79 .70 .65 2 1.00 .82 .79 3 1.00 .82 4 1.00 S u b t r a c t i o n C o m p u t a t i o n S k i l l s S E S S I O N 1 2 3 4 1 1.00 .70 .68 .63 2 1.00 .83 .80 3 1.00 .86 4 1.00 M u l t i p l i c a t i o n C o m p u t a t i o n S k i l l s S E S S I O N 1 2 3 4 1 1.00 .67 , .55 .46 2 1.00 .81 .72 3 1.00 .81 4 1.00 TABLE D-5 GRADE 4 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES, ADDITION, SDBTRACTION, MULTIPLICATION AND DIVISION FACTS TESTS OF THE DTMS. A d d i t i o n F a c t s S E S S I O N 1 2 3 4 1 1.00 .50 .59 .54 2 1.00 .79 .75 3 1.00 .81 4 1.00 S u b t r a c t i o n F a c t s S E S S I O N 1 2 . 3 4 1 1.00 .71 .73 .71 2 1.00 .86 .81 3 1.00 .84 4 1.00 M u l t i p l i c a t i o n F a c t s S E S S I O N 1 2 3 4 1 1.00 .82 .80 .80 2 1.00 .84 .82 3 1.00 .87 4 1.00 D i v i s i o n F a c t s S E S S I O N 1 2 3 4 1 1.00 .80 .77 .73 2 1.00 .88 .85 3 1.00 .86 4 1.00 TABLE D-6 GRADE 4 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES, ADDITION, SUBTRACTION, MULTIPLICATION AND DIVISION COMPUTATION SKILLS TESTS OF THE DTMS. A d d i t i o n C o m p u t a t i o n S k i l l s S E S S I O N 1 2 3 4 1 1 . 0 0 .60 . 5 9 . 5 9 2 1 . 0 0 .68 .62 3 1 . 0 0 .71 4 . 1 . 0 0 S u b t r a c t i o n C o m p u t a t i o n S k i l l s S E S S I O N 1 2 3 4 1 1 . 0 0 .74 . 7 3 . 6 7 2 1 . 0 0 .84 . 7 9 3 1 . 0 0 .82 4 1 . 0 0 M u l t i p l i c a t i o n C o m p u t a t i o n S k i l l s S E S S I O N 1 2 3 4 1 1 . 0 0 .72 .65 .68 2 1 . 0 0 .84 .82 3 1 . 0 0 . 8 3 4 1 . 0 0 D i v i s i o n C o m p u t a t i o n S k i l l s S E S S I O N 1 2 3 4 1 1 . 0 0 .50 .38 . 2 5 2 1 . 0 0 . 6 9 .48 3 1 . 0 0 .57 4 1 . 0 0 166 TABLE D-7 GRADE 2 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES FOR TOTAL MATHEMATICS FACTS AND TOTAL MATHEMATICS COMPUTATION SKILLS SCORES ON THE DTMS. Total Mathematics Facts SESSION 1 2 3 4 1 1.00 .79 .74 .72 2 1.00 .83 .80 3 1.00 .85 4 1.00 Total Mathematics Computations S k i l l s SESSION 1 2 3 4 1 1.00 .84 .77 .74 2 1.00 .90 .84 3 1.00 .89 4 1.00 1 6 7 TABLE D-8 GRADE 3 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES FOR TOTAL MATHEMATICS FACTS AND TOTAL MATHEMATICS COMPUTATION SKILLS SCORES ON THE DTMS. T o t a l M a t h e m a t i c s F a c t s S E S S I O N 1 2 3 4 1 1 . 0 0 .85 .84 .80 2 1 . 0 0 .88 .84 3 1 . 0 0 .87 4 1 . 0 0 T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s S E S S I O N 1 2 3 4 1 1 . 0 0 . 8 3 .80 .76 2 1 . 0 0 . 8 9 .86 3 1 . 0 0 .91 4 1 . 0 0 168 TABLE D-9 GRADE 4 TEST-RETEST RELIABILITY COEFFICIENTS FOR SESSIONS 1 TO 4 SCORES FOR TOTAL MATHEMATICS FACTS AND TOTAL MATHEMATICS COMPUTATION SKILLS SCORES ON THE DTMS. T o t a l M a t h e m a t i c s F a c t s SESSION 1 2 3 4 1 1 .00 . 88 .84 .81 2 1 .00 . 94 .91 3 1 .00 . 9 3 4 1 .00 T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s SESSION 1 2 3 4 1 1 .00 . 83 . 82 . 80 2 1 .00 .90 . 8 5 3 1 .00 .86 4 1 .00 1 6 9 Section I I : T h e S e s s i o n a l M e a n s , S t a n d a r d D e v i a t i o n s a n d N u m b e r o f V a l i d C a s e s o f t h e I n d i v i d u a l F u n c t i o n s T e s t e d w i t h t h e DTMS a t G r a d e s 2, 3 a n d 4. T h e m e a n s a n d s t a n d a r d d e v i a t i o n s w e r e c a l c u l a t e d u s i n g t h e " F r e q u e n c i e s ' c o m m a n d o f t h e U.B.C. S t a t i s t i c a l P a c k a g e f o r t h e S o c i a l S c i e n c e s e x t e n d e d v e r s i o n ( S P S S x ) . T o f a c i l i t a t e c o m p u t a t i o n s r e q u i r i n g e q u a l n ' s t h e h a r m o n i c m e a n o f t h e n u m b e r o f v a l i d c a s e s w a s c a l c u l a t e d u s i n g t h e f o r m u l a : = 4 / 1 / n , + l / n a + l / n 3 + l / n ^ w h e r e 4 = n u m b e r o f s e s s i o n s n = n u m b e r o f v a l i d c a s e s i n e a c h s e s s i o n ( F e r g u s o n , 1 9 8 1 ) 170 TABLE D-10 DESCRIPTIVE STATISTICS OF THE ADDITION AND SDBTRACTION FACTS AND COMPUTATIONS SKILLS SCORES AT GRADE 2 IN TESTING SESSIONS 1 THROUGH 4. T e s t i n g S e s s i o n A d d i t i o n F a c t s Mean 8.8 11.0 11.3 12.3 S t d . D e v . 4.7 5.0 5.5 5.9 V a l i d C a s e s * 285 304 308 304 A d d i t i o n C o m p u t a t i o n s Mean 9.4 12.7 14.3 16.0 S t d . D e v . 7.0 8.1 8.7 9.0 V a l i d C a s e s * 286 304 308 303 S u b t r a c t i o n F a c t s Mean 7.4 8.3 9.0 9.3 S t d . D e v . 4.5 4.6 5.1 5.3 V a l i d C a s e s * 286 305 308 304 S u b t r a c t i o n C o m p u t a t i o n s Mean 4.5 6.5 7.4 7.7 S t d . D e v . 4.0 4.6 5.0 5.4 V a l i d C a s e s * 286 305 308 304 *Number o f v a l i d c a s e s e q u a l s number o f s t u d e n t s w r i t i n g t e s t a t e a c h s e s s i o n . TABLE D - l l DESCRIPTIVE STATISTICS OF THE ADDITION AND SUBTRACTION FACTS AND COMPUTATIONS SKILLS SCORES AT GRADE 3 IN TESTING SESSIONS 1 THROUGH 4. 1 A d d i t i o n F a c t s M e a n 17.6 S t d . D e v . 6.8 V a l i d C a s e s * 322 A d d i t i o n C o m p u t a t i o n s M e a n 18.3 S t d . D e v . 7.4 V a l i d C a s e s * 322 S u b t r a c t i o n F a c t s M e a n 14.6 S t d . D e v . 6.6 V a l i d C a s e s * 322 S u b t r a c t i o n C o m p u t a t i o n s M e a n 10.8 S t d . D e v . 6.7 V a l i d C a s e s * 322 T e s t i n g S e s s i o n 2 3 4 18.7 19.9 19.3 7.0 7.3 7.4 319 327 327 21.6 23.1 23.6 7.8 8.1 8.9 319 326 327 15.5 15.9 16.3 6.8 7.0 7.2 319 327 327 13.9 14.7 14.8 9.1 9.3 9.8 319 327 327 * N u m b e r o f v a l i d c a s e s e q u a l s n u m b e r o f s t u d e n t s w r i t i n g t e s t e a c h s e s s i o n . 172 TABLE D-12 DESCRIPTIVE STATISTICS OF THE MULTIPLICATION FACTS AND COMPUTATIONS SKILLS SCORES AT GRADE 3 IN TESTING SESSIONS 1 THROUGH 4. 1 M u l t i p l i c a t i o n F a c t s Mean 4.5 S t d . D e v . 3.8 V a l i d C a s e s * 322 T e s t i n g S e s s i o n 2 3 4 6.0 6.7 7.4 4.8 5.1 5.0 319 313 327 M u l t i p l i c a t i o n C o m p u t a t i o n s Mean 5.9 S t d . D e v . 5.1 V a l i d C a s e s * 320 7.7 10.0 11.9 6.0 7.0 7.6 319 326 327 *Number o f v a l i d c a s e s e q u a l s number o f s t u d e n t s w r i t i n g t e s t a t e a c h s e s s i o n . TABLE D-l3 DESCRIPTIVE STATISTICS OF THE ADDITION AND SDBTRACTION FACTS AND COMPUTATIONS SKILLS SCORES AT GRADE 4 IN TESTING SESSIONS 1 THROUGH 4. 1 A d d i t i o n F a c t s Mean 22.4 S t d . D e v . 7.6 V a l i d C a s e s * 282 A d d i t i o n C o m p u t a t i o n s Mean 8.6 S t d . D e v . 3.0 V a l i d C a s e s * 283 S u b t r a c t i o n F a c t s Mean 20.8 S t d . D e v . 7.7 V a l i d C a s e s * 282 S u b t r a c t i o n C o m p u t a t i o n s Mean 7.9 S t d . D e v . . 5.0 V a l i d C a s e s * 283 T e s t i n g S e s s i o n 2 3 4 23.1 25.3 25.9 7.3 8.2 9.0 281 283 279 10.6 11.3 11.4 3.6 3.8 4.0 280 284 279 21.3 22.3 23.1 7.0 8.0 8.7 281 284 279 9.8 10.8 12.1 6.1 6.9 6.8 281 284 279 *Number o f v a l i d c a s e s e q u a l s number o f s t u d e n t s w r i t i n g t e s t a e a c h s e s s i o n . TABLE D-14 DESCRIPTIVE STATISTICS OF THE MULTIPLICATION AND DIVISION FACTS AND COMPUTATIONS SKILLS SCORES AT GRADE 4 IN TESTING SESSIONS 1 THROUGH 4. 1 M u l t i p l i c a t i o n F a c t s M e a n 19.0 S t d . D e v . 10.5 V a l i d C a s e s * 283 M u l t i p l i c a t i o n C o m p u t a t i o n s M e a n 12.8 S t d . D e v . 8.2 V a l i d C a s e s * 283 D i v i s i o n F a c t s M e a n 12.4 S t d . D e v . 8.8 V a l i d C a s e s * 283 D i v i s i o n C o m p u t a t i o n s M e a n .9 S t d . D e v . 2.2 V a l i d C a s e s * 283 T e s t i n g S e s s i o n 2 3 4 19.7 26.9 21.5 9.4 9.3 10.0 281 284 279 15.1 15.9 16.6 8.0 8.3 8.4 281 284 279 14.9 16.2 17.3 8.4 8.8 9.0 280 284 278 1.5 2.7 4.4 2.4 3.2 4.3 281 284 277 * N u m b e r o f v a l i d c a s e s e q u a l s n u m b e r o f s t u d e n t s w r i t i n g t e s t e a c h s e s s i o n . 175 S e c t i o n I I I : The A n a l y s i s o f V a r i a n c e Summar ies and T u k e y ' s HSD ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) f o r t h e DTMS T e s t S c o r e s , S e s s i o n s 1 t o 4. 1 7 6 T h e A n a l y s i s o f V a r i a n c e w e r e c a l c u l a t e d u s i n g t h e ' U s e r p r o c A n o v a r ' c o m m a n d o f t h e U.B.C. s t a t i s t i c a l p a c k a g e f o r t h e S o c i a l S c i e n c e s e x t e n d e d v e r s i o n ( S P S S ) . D i f f e r e n c e s b e t w e e n a d j a c e n t m e a n s ( X ^ - X , , X 3 - X x , X + - X 3 ) w e r e c o m p a r e d w i t h t u k e y ' s H S D ( H o n e s t l y - S i g n i f i c a n t - D i f f e r e n c e ) a t t h e .05 l e v e l t o i d e n t i f y m e a n s w h i c h w e r e ( h o n e s t l y ) s i g n i f i c a n t l y d i f f e r e n t . w h e r e q = v a l u e o f s t u d e n t i z e d r a n g e s t a t i s t i c «C = p r o b a b i l i t y o f a T y p e I e r r o r dfvt/ = d e g r e e s o f f r e e d o m f o r M S W k = n u m b e r o f g r o u p s n = n u m b e r o f s u b j e c t s w i t h i n g r o u p s T u k e y ' s H S D = q <«c, d f k ) ( S h a v e l s o n , 1 9 8 1 ) 1 7 7 T A B L E D - 1 5 A N A L Y S I S O F V A R I A N C E S U M M A R Y , A N D T U K E Y ' S HSD ( H O N E S T L Y - S I G N I F I C A N T - D I F F E R E N C E ) FOR G R A D E 2 T O T A L M A T H E M A T I C S F A C T S C O R R E C T A N D T O T A L M A T H E M A T I C S C O M P U T A T I O N S S K I L L S . T o t a l M a t h e m a t i c s F a c t s A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S um o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 3 5 9 1 . 0 1 1 9 7 . 2 5 9 . 0 4 0 . 0 0 1 R e s i d u a l 1 4 5 3 9 . 1 7 1 7 2 0 . 3 D i f f e r e n c e s b e t w e e n M e a n s Xx - X,= 3 . 1 5 X j - Xx= 0 . 9 3 X ^ - X,= 1 . 1 3 T u k e y ' s H S D = 1 . 0 6 T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f S u m o f D e g r e e s o f V a r i a t i o n S q u a r e s F r e e d o m M e a n S q u a r e F P r o b a b i l i t y T r e a t m e n t 1 3 4 3 7 . 5 4 4 7 9 . 2 1 6 1 . 6 0 0 . 0 0 1 R e s i d u a l 1 9 8 7 3 . 8 7 1 7 2 7 . 7 D i f f e r e n c e s b e t w e e n M e a n s Xx - X,= 5 . 5 6 X 3 - X z = 2 . 5 1 X H - X 3 = 1 . 8 7 T u k e y ' s H S D = 1 . 2 3 1 7 8 TABLE D-l6 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 3 TOTAL MATHEMATICS FACTS CORRECT AND TOTAL MATHEMATICS COMPUTATIONS SKILLS. Total Mathematics Facts A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S um o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 6 5 7 7 . 3 2 1 9 2 . 4 6 0 . 5 0 0 . 0 0 1 R e s i d u a l 2 8 8 0 9 . 0 7 9 5 3 6 . 2 D i f f e r e n c e s b e t w e e n M e a n s X x - X,= 4 . 0 4 X 3 - X a = 1 . 7 1 XH - Xy 0 . 6 3 T u k e y ' s H S D = 1 . 3 4 Total Mathematics Computations S k i l i s A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S um o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 3 7 6 1 5 . 1 1 2 5 3 8 . 4 2 1 2 . 6 0 0 . 0 0 1 R e s i d u a l 4 6 8 8 7 . 0 7 9 5 5 9 . 0 D i f f e r e n c e s b e t w e e n M e a n s X z - X x= 8 . 9 6 X 3 - X v = 4 . 1 1 X H - X 3 = 2 . 5 3 T u k e y ' s H S D = 1 . 7 1 179 TABLE D-17 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 4 TOTAL MATHEMATICS FACTS CORRECT AND TOTAL MATHEMATICS COMPUTATIONS SKILLS. Total Mathematics Facts A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F P r o b a b i l i t y T r e a t m e n t 25851.8 3 8617.3 R e s i d u a l 71370.0 711 100.4 85.85 0.001 D i f f e r e n c e s b e t w e e n M e a n s X a - X,= 4.77 X , - Xx= 5.73 X 4 - X 3 = 3.03 T u k e y ' s H S D = 2.36 Total Mathematics Computations S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F P r o b a b i l i t y T r e a t m e n t 2 6 4 5 2 . 4 3 8 8 1 7 . 5 R e s i d u a l 3 3 4 1 5 . 0 7 1 1 4 7 . 0 1 8 7 . 6 0 0 . 0 0 1 D i f f e r e n c e s b e t w e e n M e a n s X z - X ( = 6 . 5 9 X 3 - X a = 3 . 8 9 X 1 * - X 3 = 3 . 7 8 T u k e y ' s H S D = 1 . 6 1 180 TABLE D-18 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 2 ADDITION FACTS CORRECT AND ADDITION COMPUTATION SKILLS CORRECT. Addition Facts A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S um o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 1343.4 447.8 57.14 0.001 R e s i d u a l 5595.4 714 7.8 D i f f e r e n c e s b e t w e e n M e a n s X a - X,= 2.13 X , - X a = 0.14 X H - X 3 = 0.98 T u k e y ' s H S D = 0.66 Addition Computation S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F P r o b a b i l i t y T r e a t m e n t 5576.9 1859.0 143.00 0.001 R e s i d u a l 9279.6 714 13.0 D i f f e r e n c e s b e t w e e n M e a n s X ,= 3.51 X 3 - X*= 1.54 X H - X 3= 1.43 T u k e y ' s H S D = 0.85 181 T A B L E D - 1 9 A N A L Y S I S O F V A R I A N C E S U M M A R Y , A N D T U K E Y ' S HSD ( H O N E S T L Y - S I G N I F I C A N T - D I F F E R E N C E ) FOR G R A D E 2 S U B T R A C T I O N F A C T S C O R R E C T A N D S U B T R A C T I O N C O M P U T A T I O N S K I L L S C O R R E C T . S u b t r a c t i o n F a c t s A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S um o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 533.1 177.7 22.67 0.001 R e s i d u a l 5621.4 717 7.8 D i f f e r e n c e s b e t w e e n M e a n s X j. - X, = 0.95 X 3 - Xx= 0.78 X 4 - X3= 0.15 T u k e y ' s H S D = 0.66 S u b t r a c t i o n C o m p u t a t i o n S k i l l s A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S u m o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 1738.6 579.5 74.10 0.001 R e s i d u a l 5606.4 717 7.8 D i f f e r e n c e s b e t w e e n M e a n s X 3 L-X X=2.06 X,-Xj=0.97 X H - Xj= 0.47 T u k e y ' s H S D = 0.65 1 8 2 TABLE D-20 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 3 ADDITION FACTS CORRECT AND ADDITION COMPUTATION SKILLS CORRECT. Addition Facts S o u r c e o f V a r i a t i o n T r e a t m e n t Sum o f S q u a r e s 8 2 5 . 5 A n o v a S u m m a r y D e g r e e s o f F r e e d o m M e a n S q u a r e 2 7 5 . 2 P r o b a b i l i t y R e s i d u a l 1 1 6 3 4 . 7 7 9 5 1 4 . 6 1 8 . 8 0 0 . 0 0 1 D i f f e r e n c e s b e t w e e n M e a n s X, - X^ = 0 . 6 2 X x - X,= 1 . 4 3 X 7 - Xx= 0 . 9 7 T u k e y ' s H S D = 0 . 8 5 Addition Computation S k i l l s S o u r c e o f V a r i a t i o n T r e a t m e n t S u m o f S q u a r e s 4 6 2 9 . 9 A n o v a S u m m a r y D e g r e e s o f F r e e d o m M e a n S q u a r e 1 5 4 3 . 3 1 0 5 . 2 6 P r o b a b i l i t y 0 . 0 0 1 R e s i d u a l 1 1 6 5 6 . 3 7 9 5 1 4 . 7 D i f f e r e n c e s b e t w e e n M e a n s X JL - X t= 3 . 5 9 X 7 - Xj= 1 . 1 2 XH - Xj= 0 . 6 3 T u k e y ' s H S D = 0 . 8 5 183 TABLE D-21 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 3 SUBTRACTION FACTS CORRECT AND SUBTRACTION COMPUTATION SKILLS CORRECT. Subtraction Facts Anova Summary S o u r c e o f V a r i a t i o n Sum o f S q u a r e s D e g r e e s o f F reedom Mean S q u a r e P r o b a b i l i t y T r e a t m e n t 4 0 9 . 5 136.5 1 2 . 8 6 0 . 0 0 1 R e s i d u a l 8 4 3 6 . 8 795 10.6 D i f f e r e n c e s be tween Means X x - X,= 0.94 X 3 " X a = 0 . 3 4 X u - X ,= 0 . 4 0 / T u k e y ' s HSD = 0 . 7 3 Subtraction Computation S k i l l s Anova Summary S o u r c e o f V a r i a t i o n Sum o f S q u a r e s D e g r e e s o f F reedom Mean S q u a r e P r o b a b i l i t y T r e a t m e n t 3 1 9 5 . 1 1 0 6 5 . 0 5 5 . 7 4 0 .001 R e s i d u a l 1 5 1 9 0 . 9 795 1 9 . 1 D i f f e r e n c e s be tween Means X j - X|— 3 . 3 8 X 3 - X t = 0 . 8 0 X ^ - X,= 0 . 0 2 T u k e y ' s HSD = 0.97 184 TABLE D-22 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 3 MULTIPLICATION FACTS CORRECT AND MULTIPLICATION COMPUTATION SKILLS CORRECT. Multiplication Facts S o u r c e o f V a r i a t i o n T r e a t m e n t R e s i d u a l Sum o f S q u a r e s 1166.0 5046.7 Anova Summary D e g r e e s o f F reedom 759 Mean S q u a r e 388.7 6.6 58.45 P r o b a b i l i t y 0.001 D i f f e r e n c e s b e t w e e n Means X 4 - X3= 0.54 X a - X,= 1.64 X 3 - X;= 0.67 T u k e y ' s HSD = 0.59 Multiplication Computation S k i l l s Anova Summary S o u r c e o f V a r i a t i o n T r e a t m e n t R e s i d u a l Sum o f S q u a r e s 5407.4 11260.7 D e g r e e s o f F reedom 786 Mean S q u a r e 1802.5 14.3 125.81 P r o b a b i l i t y 0.001 D i f f e r e n c e s be tween Means X j, - X ,= 1.83 X 3 - Xx= 2.29 X ^  — Xj= 1.86 T u k e y ' s HSD = 0.85 185 TABLE D-23 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 4 ADDITION FACTS CORRECT AND ADDITION COMPUTATION SKILLS CORRECT. Addition Facts S o u r c e o f V a r i a t i o n T r e a t m e n t Sum o f S q u a r e s 2 0 1 1 . 9 Anova Summary D e g r e e s o f F reedom Mean S q u a r e 6 7 0 . 6 P r o b a b i l i t y 3 0 . 1 6 0 . 0 0 1 R e s i d u a l 1 5 7 4 2 . 4 708 2 2 . 2 D i f f e r e n c e s be tween Means X* - X,= 0 . 6 7 X a - X a= 2 . 2 2 X* - X 3 = 0 . 5 8 T u k e y ' s HSD = 1 . 1 1 Addition Computation S k i l l s A n o v a Summary S o u r c e o f Sum o f D e g r e e s o f Mean V a r i a t i o n S q u a r e s F reedom S q u a r e F P r o b a b i l i t y T r e a t m e n t 1 1 8 5 . 2 3 3 9 5 . 1 R e s i d u a l 3 4 7 4 . 8 708 4 . 9 8 0 . 4 9 0 . 0 0 1 D i f f e r e n c e s b e t w e e n Means X x - X ,= 1 .99 Xj-Xa.= 0 . 7 0 X * - X3= 0 . 0 9 T u k e y ' s HSD = 0 . 5 2 186 TABLE D-24 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 4 SUBTRACTION FACTS CORRECT AND SUBTRACTION COMPUTATION SKILLS CORRECT. Subtraction Facts A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S u m o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 772.3 257.4 18.10 0.001 R e s i d u a l 10068.5 708 14.2 D i f f e r e n c e s b e t w e e n M e a n s X a - X , = 0 . 6 9 X 8 - X x = 1 . 0 9 X „ - X 3 = 0 . 5 2 T u k e y ' s H S D = 0.89 Subtraction Computation S k i l l s A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S u m o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e P r o b a b i l i t y T r e a t m e n t 2405.9 802.0 83.26 0.001 R e s i d u a l 6847.9 711 9.6 D i f f e r e n c e s b e t w e e n M e a n s X a - X v= 2.03 X 3 - X*= 1.16 X * - X,= 1.11 T u k e y ' s H S D = 0.73 187 TABLE D-25 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 4 MULTIPLICATION FACTS CORRECT AND MULTIPLICATION COMPUTATION SKILLS CORRECT. Multiplication Facts A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F P r o b a b i l i t y T r e a t m e n t 995.2 3 331.7 R e s i d u a l 12698.4 711 17.9 1 8 . 5 8 0 .001 D i f f e r e n c e s b e t w e e n M e a n s X A - X x= 0 . 7 1 X 3 - X x = 1 .10 X * - X 3 = 0 . 8 6 T u k e y ' s H S D = 1 .00 M u l t i p l i c a t i o n Computation S k i l l s A n o v a S u m m a r y S o u r c e o f S u m o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F P r o b a b i l i t y T r e a t m e n t 1 8 1 0 . 6 3 6 0 3 . 5 R e s i d u a l 1 2 1 4 0 . 2 711 1 7 . 1 3 5 . 3 5 0 .001 D i f f e r e n c e s b e t w e e n M e a n s X a - X i= 2 . 0 9 X 3 - Xj,= 0 . 7 2 X a - X 3= 0 . 9 3 T u k e y ' s H S D = 0 . 9 7 188 TABLE D-26 ANALYSIS OF VARIANCE SUMMARY, AND TUKEY'S HSD (HONESTLY-SIGNIFICANT-DIFFERENCE) FOR GRADE 4 DIVISION FACTS CORRECT AND DIVISION COMPUTATION SKILLS CORRECT. D i v i s i o n F a c t s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F P r o b a b i l i t y T r e a t m e n t 3477.4 3 1159.1 R e s i d u a l 10395.1 705 14.7 78.61 0.001 D i f f e r e n c e s b e t w e e n M e a n s X l - Xx = 2.73 X 3 - Xa= 1.31 X * - Xj = 1.10 T u k e y ' s H S D = 0.91 D i v i s i o n C o m p u t a t i o n S k i l l s A n o v a S u m m a r y S o u r c e o f S u m o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F P r o b a b i l i t y T r e a t m e n t 1686.3 3 562.1 R e s i d u a l 4011.7 705 5.7 98.78 0.001 D i f f e r e n c e s b e t w e e n M e a n s X 2 - X x= 0.54 X 3 - Xa= 1.26 X 4. - X,= 1.68 T u k e y ' s H S D = 0.56 189 Section IV: Trend Analysis Summaries and r , the Proportion of Total V a r i a t i o n Accounted f o r by S i g n i f i c a n t Trends f o r the DTMS Sessional Means. 190 TABLE D-27 TREND ANALYSIS SUMMARY AND r x , THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY SIGNIFICANT TRENDS FOR GRADE 2 TOTAL MATHEMATICS FACTS CORRECT. S e s s i o n a l D a t a 2 3 300 1 6 . 2 4860 300 1 9 . 2 5760 300 2 0 . 3 6090 300 2 1 . 5 6450 O r t h o g o n a l P o l y n o m i a l s C j c 3 ; - 3 1 -1 -1 -1 3 1 -1 - 3 3 1 1 20 4 20 A n o v a S u m m a r y S o u r c e o f V a r i a t i o n L i n e a r Q u a d r a t i c C u b i c Sum o f S q u a r e s T l * / n £ c (5100 ) / 3 0 0 ( 2 0 ) ( -540 ) / 3 0 0 ( 4 ) (600 ) / 3 0 0 ( 2 0 ) D e g r e e s o f F r e e d o m 1 1 1 M e a n S q u a r e 4335 243 60 [MS/MSvJ 2 1 3 . 7 8 1 1 . 7 0 2 . 9 6 T e s t S t a t i s t i c F ( . 0 5 , 1 , 4 0 0 ) = 3 . 8 6 T h e C a l c u l a t i o n o f r = A/SS / S S T 0 . 9 7 * - 0 . 9 3 r = JSS a) / S S T = 0 . 2 3 r * = 0 . 0 5 1 9 1 TABLE D-28 TREND ANALYSIS SUMMARY AND r  x, THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY SIGNIFICANT TRENDS FOR GRADE 2 TOTAL MATHEMATICS COMPUTATIONS SKILLS CORRECT. A n o v a S u m m a r y S o u r c e o f V a r i a t i o n L i n e a r Q u a d r a t i c C u b i c Sum o f S q u a r e s 1 5 3 6 0 . 0 8 6 7 . 0 6 0 . 0 D e g r e e s o f M e a n F r e e d o m S q u a r e 1 1 1 1 5 3 6 0 . 0 8 6 7 . 0 6 0 . 0 F 5 5 4 . 1 5 3 1 . 2 8 2 . 1 7 T e s t S t a t i s t i c F = 3 . 8 6 r X L i n e a r Q u a d r a t i c C u b i c r l = 0 . 9 4 r 1 = 0 . 0 5 1 9 2 TABLE D-29 TREND ANALYSIS SUMMARY AND r l , THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY SIGNIFICANT TRENDS FOR GRADE 3 TOTAL MATHEMATICS FACTS CORRECT AND TOTAL MATHEMATICS COMPUTATIONS SKILLS CORRECT. Total Mathematics Facts A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 7 0 7 6 . 3 1 7 0 7 6 . 3 1 9 5 . 2 7 Q u a d r a t i c 5 9 0 . 5 1 5 9 0 . 5 1 6 . 3 0 C u b i c 1.5 1 1 . 5 0 . 0 4 T e s t S t a t i s t i c F = 3 . 8 6 r x L i n e a r Q u a d r a t i c C u b i c r * = 0 . 9 2 r x = 0 . 0 8 Total Mathematics Computations S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 6 7 1 8 7 . 2 1 6 7 1 8 7 . 2 1 1 3 9 . 2 1 Q u a d r a t i c 9 4 4 7 . 8 1 9 4 4 7 . 8 1 6 0 . 2 0 C u b i c 5 4 5 . 0 1 5 4 5 . 0 9 . 2 4 T e s t S t a t i s t i c F = 3 . 8 6 r L i n e a r Q u a d r a t i c C u b i c r x = 0 . 8 7 r x = 0 . 1 2 r * = 0 . 0 1 193 TABLE D-30 TREND ANALYSIS SUMMARY AND r x , THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 4 TOTAL MATHEMATICS FACTS CORRECT AND TOTAL MATHEMATICS COMPUTATIONS S K I L L S CORRECT. T o t a l M a t h e m a t i c s F a c t s A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S um o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e F L i n e a r 28679.5 1 28679.5 285.71 Q u a d r a t i c 158.6 1 158.6 1.58 C u b i c 153.5 T e s t F 1 S t a t i s t i c = 3.86 153.5 1.53 L i n e a r Q u a d r a t i c C u b i c r a = 0.99 T o t a l M a t h e m a t i c s C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f V a r i a t i o n S um o f S q u a r e s D e g r e e s o f F r e e d o m M e a n S q u a r e F L i n e a r 29835.6 1 29835.6 634.84 Q u a d r a t i c 552.7 1 552.7 11.76 C u b i c 225.6 T e s t F 1 S t a t i s t i c = 3.86 225.6 4.80 L i n e a r Q u a d r a t i c C u b i c r *• = 0.97 r x = = 0.02 r x = 0.01 194 TABLE D-31 TREND ANALYSIS SUMMARY AND r  x, THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 2 ADDITION FACTS CORRECT AND ADDITION COMPUTATIONS S K I L L S CORRECT. A d d i t i o n F a c t s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 1749.6 1 1749.6 223.25 Q u a d r a t i c 108.0 1 108.0 13.78 C u b i c 14.1 1 14.1 1.80 T e s t S t a t i s t i c F = 3.86 r 1 L i n e a r Q u a d r a t i c C u b i c r l = 0 . 9 3 r z = 0.06 A d d i t i o n C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 6869.4 1 6869.4 528.54 Q u a d r a t i c 192.0 1 192.0 14.77 C u b i c 48.6 1 48.6 3.74 T e s t S t a t i s t i c F = 3.86 L i n e a r Q u a d r a t i c C u b i c r x = 0.97 r x = 0.03 195 TABLE D-32 TREND ANALYSIS SUMMARY AND r \ THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 2 SUBTRACTION FACTS CORRECT AND SUBTRACTION COMPUTATIONS S K I L L S CORRECT. S u b t r a c t i o n F a c t s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 614.4 1 614.4 78.37 Q u a d r a t i c 27.0 1 27.0 3.44 C u b i c 0.6 1 0.6 0.08 T e s t S t a t i s t i c F = 3.86 l r L i n e a r Q u a d r a t i c C u b i c r 1 = 0.95 S u b t r a c t i o n C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 1653.7 1 1653.7 211.50 Q u a d r a t i c 216.7 1 216.7 27.72 C u b i c 3.7 1 3.7 0.48 T e s t S t a t i s t i c F = 3.86 r_ L i n e a r Q u a d r a t i c C u b i c r *•= 0 . 8 8 r X = 0.12 196 TABLE D-33 TREND ANALYSIS SUMMARY AND r a , THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 3 ADDITION FACTS CORRECT AND ADDITION COMPUTATIONS S K I L L S CORRECT. A d d i t i o n F a c t s Anova Summary Source of Sum of Degrees of Mean V a r i a t i o n Squares Freedom Square F Linear 643.5 1 643.5 43.97 Quadratic 235.1 1 235.1 16.06 Cubic 58.6 1 58.6 4.00 Test S t a t i s t i c F = 3.86 Linear Quadratic Cubic r * = 0.69 r 1 = 0.25 r * = 0.06 A d d i t i o n C o m p u t a t i o n s S k i l l s Anova Summary Source of Sum of Degrees of Mean V a r i a t i o n Squares Freedom Square F Linear 4889.2 1 4889.2 333.46 Quadratic 632.5 1 632.5 43.14 Cubic 10.5 1 10.5 0.71 Test S t a t i s t i c F = 3.86 „ x Linear r 1 = 0.88 Quadratic r x = 0.11 Cubic 1 9 7 TABLE D-34 TREND ANALYSIS SUMMARY AND r S THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 3 SUBTRACTION FACTS CORRECT AND SUBTRACTION COMPUTATIONS S K I L L S CORRECT. S u b t r a c t i o n F a c t s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 4 9 0 . 6 1 4 9 0 . 6 4 6 . 2 3 Q u a d r a t i c 2 0 . 5 1 2 0 . 5 1 . 9 3 C u b i c 4 . 0 1 4 . 0 0 . 3 8 T e s t S t a t i s t i c F = 3 . 8 6 r 1 L i n e a r Q u a d r a t i c C u b i c r *• = 0 . 9 5 S u b t r a c t i o n C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 2 6 5 4 . 0 1 2 6 5 4 . 0 1 3 8 . 8 9 Q u a d r a t i c 7 3 0 . 5 1 7 3 0 . 5 3 8 . 2 3 C u b i c 4 1 . 6 1 4 1 . 6 2 . 1 8 T e s t S t a t i s t i c F = 3 . 8 6 2-L i n e a r r x = 0 . 7 7 Q u a d r a t i c r x = 0 . 2 1 C u b i c 198 TABLE D-35 TREND ANALYSIS SUMMARY AND r z , THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 3 MULTIPL ICATION FACTS CORRECT AND MULTIPL ICATION COMPUTATIONS S K I L L S CORRECT. M u l t i p l i c a t i o n F a c t s A n o v a S u m m a r y S o u r c e o f S u m o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 1413.8 1 1413.8 212.63 Q u a d r a t i c 51.2 1 51.2 7.70 C u b i c 10.2 1 10.2 1.54 T e s t S t a t i s t i c F = 3.86 r x L i n e a r Q u a d r a t i c C u b i c r r = 0.96 r x = 0.03 M u l t i p l i c a t i o n C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 6655.5 1 6655.5 464.54 Q u a d r a t i c 0.8 1 0.8 0.06 C u b i c 13.1 1 13.1 0.91 T e s t S t a t i s t i c F = 3.86 L i n e a r Q u a d r a t i c C u b i c TABLE D-36 TREND ANALYSIS SUMMARY AND r \ THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY SIGNIFICANT TRENDS FOR GRADE 4 ADDITION FACTS CORRECT AND ADDITION COMPUTATIONS SKILLS CORRECT. Addition Facts A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 2267 .8 1 2267 .8 101 .99 Q u a d r a t i c 0 .7 1 0 .7 0 .03 C u b i c 134 .7 1 134 .7 6 .06 T e s t S t a t i s t i c F = 3 .86 r 1 -L i n e a r Q u a d r a t i c C u b i c r * = 0 .94 r i = 0 . 0 6 A d d i t i o n C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 1160 .7 1 1160 .7 236.48 Q u a d r a t i c 253 .7 1 253 .7 51 .69 C u b i c 7 .0 1 7 .0 1.42 T e s t S t a t i s t i c F = 3 .86 r l L i n e a r Q u a d r a t i c C u b i c r = 0 .82 r *" = 0.18 2 0 0 TABLE D-37 TREND A N A L Y S I S SUMMARY AND r*V THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 4 SUBTRACTION FACTS CORRECT AND SUBTRACTION COMPUTATIONS S K I L L S CORRECT. S u b t r a c t i o n F a c t s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 8 7 6 . 1 1 8 7 6 . 2 6 1 . 6 1 Q u a d r a t i c 6 . 4 1 6 . 4 0 . 4 5 C u b i c 6 . 9 1 6 . 9 0 . 4 9 T e s t S t a t i s t i c F = 3 . 8 6 L i n e a r Q u a d r a t i c C u b i c r 1 = 0 . 9 9 S u b t r a c t i o n C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 2 6 0 6 . 3 1 2 6 0 6 . 3 2 7 0 . 6 2 Q u a d r a t i c 2 5 . 6 1 2 5 . 6 2 . 6 6 C u b i c 2 0 . 3 1 2 0 . 3 0 . 1 0 T e s t S t a t i s t i c F = 3 . 8 6 r * L i n e a r Q u a d r a t i c C u b i c 201 TABLE D-38 TREND ANALYSIS SUMMARY AND r \ THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY SIGNIFICANT TRENDS FOR GRADE 4 MULTIPLICATION FACTS CORRECT AND MULTIPLICATION COMPUTATIONS SKILLS CORRECT. M u l t i p l i c a t i o n Facts A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 1 0 6 7 . 7 1 1 0 6 7 . 8 5 9 . 7 8 Q u a d r a t i c 0 . 7 1 0 . 7 0 .04 C u b i c 1 7 . 3 1 1 7 . 3 0 . 9 7 T e s t S t a t i s t i c F = 3 . 86 L i L i n e a r Q u a d r a t i c C u b i c r 1 = 0 . 9 8 M u l t i p l i c a t i o n Computations S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 2 0 9 6 . 9 1 2 0 9 6 . 9 1 2 2 . 8 1 Q u a d r a t i c 1 8 0 . 3 1 1 8 0 . 3 1 0 . 5 6 C u b i c 2 7 . 4 1 2 7 . 4 1 .60 T e s t S t a t i s t i c F = 3 . 8 6 r 1 L i n e a r Q u a d r a t i c C u b i c r 1 = 0 .91 r x = 0 . 0 8 202 TABLE D-39 TREND A N A L Y S I S SUMMARY AND r \ THE PROPORTION OF TOTAL VARIATION ACCOUNTED FOR BY S IGNIF ICANT TRENDS FOR GRADE 4 D I V I S I O N FACTS CORRECT AND D I V I S I O N COMPUTATIONS S K I L L S CORRECT. D i v i s i o n F a c t s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 3596.8 1 3596.8 2 4 3 . 9 3 Q u a d r a t i c 138.1 1 138.1 9.37 C u b i c 14.1 1 14.2 0.96 T e s t S t a t i s t i c F = 3.86 r*-L i n e a r Q u a d r a t i c C u b i c r 1 = 0.96 r3- = 0.04 D i v i s i o n C o m p u t a t i o n s S k i l l s A n o v a S u m m a r y S o u r c e o f Sum o f D e g r e e s o f M e a n V a r i a t i o n S q u a r e s F r e e d o m S q u a r e F L i n e a r 1923.3 1 1923.3 338.01 Q u a d r a t i c 85.0 1 85.0 14.94 C u b i c 0.1 1 0.1 0.02 T e s t S t a t i s t i c F = 3.86 L i n e a r Q u a d r a t i c C u b i c 203 APPENDIX E S e c t i o n I_: T h e B e s t F i t t i n g C u r v e s I n d i c a t e d b y T r e n d A n a l y s i s f o r T o t a l M a t h e m a t i c s S c o r e s o n t h e D T M S . * S e c t i o n I I : C u m u l a t i v e P e r c e n t a g e P o l y g o n s f o r T o t a l M a t h e m a t i c s S c o r e s o n t h e D T M S . S e c t i o n I I I : R e a l a n d P r o j e c t e d M e a n s a n d S t a n d a r d D e v i a t i o n s f o r T o t a l M a t h e m a t i c s S c o r e s o n t h e D T M S . * T h e s e b e s t f i t t i n g c u r v e s w e r e c a l c u l a t e d a s f o l l o w s : L i n e a r y , = x . . + £ c , • T j / n £ c , j * ( x - 2 . 5 ) Q u a d r a t i c y a = y , + £ c 4 j T j / n £ c ^ j 1 ( x x - 5 x + 5 ) C u b i c y 3 = yx + $ c3j T j / n £c3j X ( x 3 - 7 . 5 x x + 1 6 . 7 x - 1 0 . 204 TABLE E - l : THE BEST FITTING CURVES INDICATED BY TREND ANALYSIS FOR GRADE 2 TOTAL MATHEMATICS FACTS CORRECT SCORES. 25 • 20 -15 -Sessional Means Linear Curve Quadratic Curve 2 Sessions 205 TABLE E-2: THE BEST FITTING CURVES INDICATED BY TREND ANALYSIS FOR GRADE 2 TOTAL COMPUTATIONS SKILLS CORRECT SCORES. Sessions 206 TABLE E-3: THE BEST FITTING CURVES INDICATED BY TREND ANALYSIS FOR GRADE 3 TOTAL MATHEMATICS FACTS CORRECT SCORES. 45 40 U o u to 35 Sessional Means Linear Curve Quadratic Curve —i— 1 Sessions 207 TABLE E-4: THE BEST FITTING CURVES INDICATED BY TREND ANALYSIS FOR GRADE 3 TOTAL COMPUTATIONS SKILLS CORRECT SCORES. 50 -45 -<: • • ^ / 40 -(LI U O U CO 35 7 30 -Sessional Means Linear Curve Quadratic Curve Cubic Curve "T" 4 Sessions 208 Sessions 209 TABLE E-6: THE BEST FITTING CURVES INDICATED BY TREND ANALYSIS FOR GRADE 4 TOTAL COMPUTATIONS SKILLS CORRECT SCORES. 45 -40 -35 -30 Sessional Means Linear Curve Quadratic Curve Cubic Curve —r-2 Sessions 210 Section I I : Cumulative Percentage Polygons f o r Total Mathematics Scores on the DTMS. Total Score FIGURE E-8: CUMULATIVE PERCENTAGE POLYGON FOR TOTAL MATHEMATICS COMPUTATIONS, CORRECT, GRADE 2. Total Score FIGURE: E - 9: CUMULATIVE PERCENTAGE POLYGON FOR TOTAL MATHEMATICS FACTS CORRECT, GRADE 3 . Total Score FIGURE E - l l : CUMULATIVE PERCENTAGE POLYGON FOR TOTAL MATHEMATICS FACTS CORRECT, GRADE 4. FIGURE E-12: CUMULATIVE PERCENTAGE POLYGON FOR TOTAL MATHEMATICS COMPUTATIONS CORRECT, GRADE 4. 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Total Score 217 Section I I I : Real and Projected Means and Standard Deviations f o r Total Mathematics Scores on the DTMS. FIGURE E-13: GRADE 2 TOTAL MATHEMATICS COMPUTATIONS SKILLS SESSIONAL MEANS AND STANDARD DEVIATIONS. Months 219 FIGURE E-14: GRADE 3 TOTAL MATHEMATICS FACTS SESSIONAL MEANS AND STANDARD DEVIATIONS. 80 H 75 4 70 -65 . Months FIGURE E-15: GRADE 3 TOTAL MATHEMATICS COMPUTATIONS SKILLS SESSIONAL MEANS AND STANDARD DEVIATIONS. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0054372/manifest

Comment

Related Items