UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Assessment of teachers' grading practises 1986

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
UBC_1986_A8 D38.pdf
UBC_1986_A8 D38.pdf [ 4.58MB ]
Metadata
JSON: 1.0054517.json
JSON-LD: 1.0054517+ld.json
RDF/XML (Pretty): 1.0054517.xml
RDF/JSON: 1.0054517+rdf.json
Turtle: 1.0054517+rdf-turtle.txt
N-Triples: 1.0054517+rdf-ntriples.txt
Citation
1.0054517.ris

Full Text

ASSESSMENT OF TEACHERS ' GRADING PRACTISES by WILLIAM MONTE DAUNCEY B.Ed., U n i v e r s i t y of V i c t o r i a , 1976 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS i n THE FACULTY OF GRADUATE STUDIES (Department of E d u c a t i o n a l Psychology) We accept t h i s t h e s i s as conforming to the r e q u i r e d standard THE UNIVERSITY OF BRITISH COLUMBIA October, 1986 %pj W i l l i a m Monte Dauncey, 1986 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t o f the requirements f o r an advanced degree a t the U n i v e r s i t y o f B r i t i s h Columbia, I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and study. I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the head o f my department o r by h i s o r her r e p r e s e n t a t i v e s . I t i s understood t h a t copying or p u b l i c a t i o n o f t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be allowed without my w r i t t e n p e r m i s s i o n . Department o f EDUCATIONAL PSYCHOLOGY The U n i v e r s i t y of B r i t i s h Columbia 1956 Main Mall Vancouver, Canada V6T 1Y3 D a t e OCTOBER 1 1 , 1986 / a n A b s t r a c t E d u c a t o r s have been u s i n g t h e l e t t e r grade system o f g r a d i n g and r e p o r t i n g s t u d e n t achievement f o r s e v e r a l decades. S i n c e i t s i n c e p t i o n , t e a c h e r s have d e r i v e d l e t t e r grades from a v a r i e t y of g r a d i n g t e c h n i q u e s . As a r e s u l t , t h i s approach t o g r a d i n g has o f t e n r e c e i v e d c r i t i c i s m from t h o s e who • q u e s t i o n i t s r e l i a b i l i t y and u s e f u l n e s s . The purpose of t h i s s t u d y was t o de t e r m i n e i f l e t t e r grades c o u l d be made more r e l i a b l e by s t a t i s t i c a l l y b a l a n c i n g raw achievement s c o r e s p r i o r t o a g g r e g a t i o n f o r r e p o r t i n g p u r p o s e s . Many a u t h o r s have w r i t t e n t o a t t a c k l e t t e r grades w h i l e o t h e r s have w r i t t e n t o defend t h e i r use. Some have w r i t t e n t o suggest a l t e r n a t i v e s t o l e t t e r grades w h i l e s t i l l o t h e r w r i t e r s have s u g g e s t e d methods o f i m p r o v i n g g r a d i n g t e c h n i q u e s . However, l i t e r a t u r e s e a r c h e s have shown t h a t v e r y l i t t l e r e s e a r c h has been done t o a s s e s s t e a c h e r s ' g r a d i n g p r a c t i s e s and t h e grades t h e y award s t u d e n t s . T h i s i n v e s t i g a t i v e s t u d y was d e s i g n e d t o e v a l u a t e the g r a d i n g methods used by 37 randomly s e l e c t e d e l e m e n t a r y s c h o o l t e a c h e r s . I n f o r m a t i o n on t h e i r methods o f g r a d i n g was c o l l e c t e d i n t h r e e ways: (a) by way o f a q u e s t i o n n a i r e , (b) by h a v i n g t h e s u b j e c t s w e i g h t , t o t a l , and rank a h y p o t h e t i c a l s e t of raw achievement s c o r e s , and (c) by h a v i n g t h e s u b j e c t s submit c l a s s r e c o r d s f o r one r e p o r t i n g p e r i o d from two s u b j e c t i i i a reas, mathematics and s o c i a l s t u d i e s . The raw scores f o r each c l a s s were s t a t i s t i c a l l y balanced and the teach e r s regraded t h e i r students based on the r e v i s e d aggregate t o t a l s . In o r d e r t o c o n t r o l f o r extraneous and s u b j e c t i v e f a c t o r s , the same g r a d i n g c r i t e r i o n was used f o r both the o r i g i n a l and the r e v i s e d aggregate s c o r e s . The rankings of the o r i g i n a l aggregate scores were compared t o those d e r i v e d from the balanced aggregate scores u s i n g the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t . The c o r r e l a t i o n s were found t o be s i g n i f i c a n t f o r each r e c o r d sheet, i n d i c a t i n g t h a t the n u l l h ypothesis should be r e j e c t e d f o r each of the 56 c l a s s e s s t u d i e d . A n a l y s i s of the c l a s s r e c o r d sheets and q u e s t i o n n a i r e responses a l s o r e v e a l e d : (a) t h a t 46% of the 1,314 students i n v o l v e d i n the study r e c e i v e d a change i n l e t t e r grade i n s p i t e of the s i g n i f i c a n t rank c o r r e l a t i o n c o e f f i c i e n t s . T h i s suggests t h a t f o r many students, the assignment of l e t t e r grades was u n r e l i a b l e and based on f a c t o r s other than t o t a l score rankings. (b) t h a t o n l y 5% of the respondents used methods t h a t would apply the d e s i r e d weighting f a c t o r s t o the raw s c o r e s . T h i s suggests t h a t many teach e r s used u n r e l i a b l e methods t o weight assignment s c o r e s . (c) t h a t none of the s u b j e c t s i n t h i s study used r e l i a b l e methods t o compensate a student who has missed one or more assignments or t e s t s . T h i s suggests t h a t students who were i v a b sent may have been u n j u s t l y rewarded o r p e n a l i z e d when t h e i r a g g r e g a t e s c o r e s were c a l c u l a t e d . (d) t h a t 76% o f the r e s p o n d e n t s showed a d e s i r e t o l e a r n more about c o l l a t i n g raw s c o r e s and a s s i g n i n g l e t t e r grades t o agg r e g a t e s c o r e s . , , These r e s u l t s s u g g e sted t h a t i n - s e r v i c e i n s t r u c t i o n and p r e - s e r v i c e t r a i n i n g i n p a r t i c u l a r a s p e c t s o f g r a d i n g and r e p o r t i n g would be j u s t i f i e d f o r many members of t h e r e s e a r c h sample. Areas o f g r e a t e s t need a r e tho s e c o n c e r n i n g t h e w e i g h t i n g o f raw s c o r e s , t h e a l l o c a t i o n o f l e t t e r g r a d e s , and the c a l c u l a t i o n of compensation s c o r e s f o r s t u d e n t s who have mi s s e d a s s i g n m e n t s . R e s e a r c h S u p e r v i s o r : Dr. H. R a t z l a f f V TABLE OF CONTENTS A b s t r a c t i i L i s t o f T a b l e s v i Acknowledgements v i i i C h a p ter 1 - Scope o f the Study 1 - I d e n t i f i c a t i o n o f the Problem 4 - D e f i n i t i o n o f Terms 12 - R e s e a r c h Q u e s t i o n s 16 - R a t i o n a l e 17 - D e l i m i t a t i o n s 19 - J u s t i f i c a t i o n o f t h e Study 20 Chapter 2 - L i t e r a t u r e Review 2 4 - G r a d i n g Systems 2 9 - C r i t e r i o n - r e f e r e n c e d G r a d i n g 33 - Norm-referenced G r a d i n g 3 6 Chapter 3 - Methodology 4 5 - Sample and P o p u l a t i o n 45 - Q u e s t i o n n a i r e 46 - Record Sheets 4 9 - G r a d i n g P o l i c y 52 - Data C o l l e c t i o n and A n a l y s i s 54 Chapter 4 - R e s u l t s 56 - Sample 56 - G r a d i n g Techniques 64 - Record Sheets 70 Chapter 5 - C o n c l u s i o n s and F u r t h e r R e s e a r c h 77 - Summary 7 7 - R e s e a r c h Q u e s t i o n 1 79 - R e s e a r c h Q u e s t i o n 2 86 - R e s e a r c h Q u e s t i o n 3 89 - R e s e a r c h Q u e s t i o n 4 91 - Weaknesses To Be C o n s i d e r e d 92 - O p p o r t u n i t i e s f o r F u t u r e R e s e a r c h 94 B i b l i o g r a p h y 98 Appendix A - Q u e s t i o n n a i r e 101 LIST OF TABLES v i T a b l e I - F i c t i t i o u s d a t a t o be c o l l a t e d and ranked 5 0 T a b l e I I - F i c t i t i o u s d a t a a f t e r b e i n g s t a t i s t i c a l l y b a l a n c e d 50 T a b l e I I I - L e t t e r grade d i s t r i b u t i o n g u i d e l i n e s 53 Ta b l e IV - D i s t r i b u t i o n by Gender 57 T a b l e V - D i s t r i b u t i o n by Age 57 T a b l e VI - D i s t r i b u t i o n by u n i v e r s i t y t r a i n i n g ( y e a r s ) 57 Ta b l e V I I - D i s t r i b u t i o n by Measurement Courses Taken 5 8 Ta b l e V I I I - D i s t r i b u t i o n by T e a c h i n g Assignment 58 Ta b l e IX - D i s t r i b u t i o n by T e a c h i n g E x p e r i e n c e 58 Ta b l e X - D i s t r i b u t i o n o f C o l l e a g u e s ' A t t i t u d e s Toward Re p o r t Card Marks 6 0 Ta b l e XI - D i s t r i b u t i o n o f A t t i t u d e Toward L e t t e r Grade E f f e c t i v e n e s s 60 Ta b l e X I I - D i s t r i b u t i o n o f Agreement R e g a r d i n g P a r e n t s ' S a t i s f a c t i o n Toward L e t t e r Grade R e p o r t i n g Systems 61 Ta b l e X I I I - D i s t r i b u t i o n o f Agreement R e g a r d i n g S u b j e c t s ' Own G r a d i n g A b i l i t y 61 Ta b l e XIV - D i s t r i b u t i o n o f Agreement R e g a r d i n g S u b j e c t s ' D e s i r e To L e a r n More About M a r k i n g and G r a d i n g 6 2 Ta b l e XV - D i s t r i b u t i o n o f Agreement on E v a l u a t i o n I n s e r v i c e S e s s i o n s . 62 Ta b l e XVI - D i s t r i b u t i o n o f Agreement R e g a r d i n g S u b j e c t s ' O p i n i o n on P r e - s e r v i c e T r a i n i n g 62 v i i T a b l e X V I I - D i s t r i b u t i o n o f Agreement R e g a r d i n g E f f e c t i v e n e s s o f the R e p o r t Card Format As a Communication D e v i c e 63 Ta b l e X V I I I - D i s t r i b u t i o n o f G r a d i n g Methods 66 T a b l e XIX - D i s t r i b u t i o n o f A p p l i c a t i o n o f G r a d i n g Technique 66 Ta b l e XX - D i s t r i b u t i o n o f Compensation Method 68 Tab l e XXI - D i s t r i b u t i o n o f Ranks o f F i c t i t i o u s Data 6 9 T a b l e X X I I - Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s R e l a t i n g Raw Score and B a l a n c e d Score T o t a l s 72 Ta b l e X X I I I - D i s t r i b u t i o n of Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s 73 Ta b l e XXIV - D i s t r i b u t i o n o f L e t t e r Grade Changes 74 Tab l e XXV - Frequency o f C a t e g o r y S h i f t 74 Tab l e XXVI - Frequency of L e t t e r Grade Changes R e s u l t i n g From A r i t h m e t i c E r r o r s 75 Tab l e XXVII - Frequency of C a t e g o r y S h i f t R e s u l t i n g From L e t t e r Grade R e c a l c u l a t i o n 76 Tab l e X X V I I I - E f f e c t s o f r For Between S c o r e s on T o t a l Score r When r (between) =+1 8 4 s T a b l e XXIX - E f f e c t on r For Between Scores On T o t a l Score r When r (betwlen) =. 65 8 5 s T a b l e XXX - Compensation F o r M i s s e d Assignment 90 v i i i Acknowledgement I would l i k e t o e x p r e s s my g r a t i t u d e t o my r e s e a r c h s u p e r v i s o r , Dr. H. R a t z l a f f , and t o committee members Dr. W. B o l d t and Dr. W. S z e t e l a f o r t h e i r h e l p and c o u n s e l . I w i s h t o extend my s i n c e r e thanks and a p p r e c i a t i o n t o my w i f e , Roxanne, f o r her c o n t i n u o u s s u p p o r t and encouragement th r o u g h o u t t h e c o u r s e o f my s t u d i e s . F i n a l l y , t o my c h i l d r e n , D e n i s e and Douglas, a s p e c i a l note o f g r a t i t u d e f o r t h e i r p a t i e n c e and u n d e r s t a n d i n g . 1 CHAPTER 1 Scope of the Study- Background I n f o r m a t i o n F o r more th a n f i f t y y e a r s , the l e t t e r grade system has been used i n e d u c a t i o n t o r e p o r t and document the l e v e l of s t u d e n t achievement. However, even a f t e r decades of use, g r a d i n g p r a c t i s e s w i t h i n t h e l e t t e r grade system s t i l l t e n d t o v a r y among t e a c h e r s , departments, s c h o o l s , and s c h o o l d i s t r i c t s . Many t e a c h e r s a c q u i r e g r a d i n g p r a c t i s e s i n f o r m a l l y ; from a c o l l e a g u e , department head, o r by u s i n g methods t h a t seem i n t u i t i v e l y r e a s o n a b l e . Others may have had a f l e e t i n g exposure t o an " a c c e p t a b l e " g r a d i n g p r o c e d u r e d u r i n g t e a c h e r t r a i n i n g . I t i s s u r p r i s i n g t h a t many t e a c h e r s embark on t h e i r c a r e e r s i n the s c h o o l s w i t h o u t e v e r h a v i n g been t a u g h t a n y t h i n g about g r a d i n g . T h i s v e r y i m p o r t a n t p a r t o f the t e a c h e r ' s t a s k i s o f t e n i g n o r e d i n t e a c h e r t r a i n i n g . Perhaps i t i s j u s t t a k e n f o r g r a n t e d , o r perhaps i t seems t o o i n t r a c t a b l e t o be t a u g h t . ( H i l l s , 1981, p. 316) C o n s e q u e n t l y , when i n c o n s i s t e n c i e s i n g r a d i n g p r a c t i s e s e x i s t , e q u a l l e v e l s o f achievement are not n e c e s s a r i l y g i v e n e q u a l l e t t e r g r a d e s . F o r example, a grade of "C" i n one c l a s s r o o m may be a "C-" o r a "C+" i n a n o t h e r . As a r e s u l t , grades and g r a d i n g p r a c t i s e s become somewhat s u s p e c t and o f t e n l o s e c r e d i b i l i t y w i t h t h e p o s t - s e c o n d a r y i n s t i t u t i o n s , p r o s p e c t i v e employers, s t u d e n t s , and o t h e r i n d i v i d u a l s o r a g e n c i e s t h a t 2 depend upon r e l i a b l e academic assessments as a b a s i s f o r d e c i s i o n making. T h i s n o t i o n i s s u p p o r t e d by H i l l s (1981) who s t a t e s , F a u l t y g r a d i n g can be s e r i o u s l y d e t r i m e n t a l t o s t u d e n t s . I t can g i v e them m i s l e a d i n g i n f o r m a t i o n and i n t e r f e r e w i t h the l e a r n i n g p r o c e s s . I t can g i v e m i s l e a d i n g i n f o r m a t i o n f o r o t h e r s who l e g i t i m a t e l y use s c h o o l grades as p a r t o f t h e i r p r o c e d u r e s i n e v a l u a t i n g p e o p l e . And i t can m i s l e a d t h o s e who would s t u d y the i n s t r u c t i o n a l p r o c e s s t o improve i t o r t o h o l d i t a c c o u n t a b l e t o the s o c i e t y t h a t s u p p o r t s i t . (p. 317) However, f o r most s t u d e n t s u n r e l i a b l e c a l c u l a t i o n s o f a term's grades w i l l n o t cause much harm. For example, i f a "C" i s awarded i n s t e a d o f a C+ o r a C- , t h e d i f f e r e n t i a l e f f e c t s . upon t h e s t u d e n t w i l l not v a r y s i g n i f i c a n t l y . D i s c r e p a n c i e s such as t h e s e w i l l seldom l e a d t o i n c o r r e c t d e c i s i o n s . a s C+, C, and C- a r e a l l symbols o f " s a t i s f a c t o r y " achievement. On the o t h e r hand, when a s t u d e n t has been g i v e n an F when he m e r i t e d a p a s s i n g grade, some f a r r e a c h i n g consequences c o u l d r e s u l t . Such an e r r o r c o u l d c o s t the s t u d e n t c r e d i t f o r t h e c o u r s e , o r even an e x t r a y e a r i n s c h o o l i f a co u r s e f a i l u r e p r e v e n t s p r o m o t i o n . At t h e o t h e r end o f the s c a l e , a C+ i n s t e a d o f a B, o r a B i n s t e a d o f an A c o u l d r e s u l t i n t h e l o s s o f a s c h o l a r s h i p , a b u r s a r y , o r an o p p o r t u n i t y t o a t t e n d a p a r t i c u l a r l y c o m p e t i t i v e p o s t - s e c o n d a r y i n s t i t u t i o n . Of c o u r s e , t h e i n v e r s e i s e q u a l l y s i g n i f i c a n t . F o r example, i f a 3 s t u d e n t i s e r r o n e o u s l y awarded a p a s s i n g g r a d e , he w i l l be g i v e n c r e d i t f o r a c o u r s e when i t i s not w a r r a n t e d and when he i s n o t s u f f i c i e n t l y p r e p a r e d t o go on t o the nex t l e v e l . In e l e m e n t a r y s c h o o l s , marks and o t h e r measures o f s t u d e n t achievement form t h e b a s i s f o r d e t e r m i n i n g whether a s t u d e n t w i l l be promoted i n t o the n e x t grade, whether he w i l l r e c e i v e an academic award o r whether he w i l l be a c a n d i d a t e f o r l e a r n i n g a s s i s t a n c e o r a s p e c i a l r e m e d i a l c l a s s . I n j u n i o r h i g h s c h o o l , term grades a r e used as i n the e l e m e n t a r y s c h o o l s e t t i n g but w i t h the a d d i t i o n a l use of i d e n t i f y i n g honour r o l l s t u d e n t s . S e n i o r h i g h s c h o o l s use term grades not o n l y f o r p r o m o t i o n and r e t e n t i o n but a l s o t o det e r m i n e which s t u d e n t s w i l l be r e q u i r e d t o w r i t e f i n a l exams, which s t u d e n t s w i l l be e l i g i b l e f o r s c h o l a r s h i p s and b u r s a r i e s , and which s t u d e n t s s h o u l d be encouraged t o a t t e n d p o s t - s e c o n d a r y i n s t i t u t i o n s . A l t h o u g h t h e r e a re t h o s e who d i s a g r e e w i t h the l e t t e r system o f g r a d i n g and r e p o r t i n g (see Curwin, 1978), t h o s e who do agree f e e l i t may be one of t h e t e a c h e r ' s most i m p o r t a n t r o l e s . "One o f the most i m p o r t a n t p r o f e s s i o n a l r e s p o n s i b i l i t i e s o f academic s t a f f i s t h a t of a l l o c a t i n g and r e p o r t i n g marks (or grades) as measures of s t u d e n t performance" ( I s a a c s and I m r i e , 1981, p. 3 ) . Curwin s u g g e s t s t h a t t e a c h e r s s h o u l d q u e s t i o n themselves as t o the r e l i a b i l i t y of grades i n f u l f i l l i n g t he r e p o r t i n g and l e a r n i n g f u n c t i o n s . He s t a t e s , "The one b a s i c r e q u i r e m e n t of the r e p o r t i s t h a t a l l i n f o r m a t i o n r e c e i v e d must have t h e same meaning t h a t was 4 s e n t by the g r a d e r . Grades must be v a l i d , r e l i a b l e and have a h i g h degree of p r e d i c t a b i l i t y . But t h e y don't." ( C u r w i n , 1978 , p. 6 0 ) . I d e n t i f i c a t i o n of t h e Problem I t seems c l e a r t h a t t h e r e i s a need f o r t e a c h e r s ' g r a d i n g p r a c t i s e s t o be as r e l i a b l e and a c c u r a t e as p o s s i b l e . Most s c h o o l boards a l s o r e c o g n i z e t h e need f o r r e l i a b i l i t y i n e v a l u a t i o n and r e p o r t i n g . As s t a t e d i n t h e A d m i n i s t r a t i v e Handbook of one l a r g e d i s t r i c t , The Board of T r u s t e e s b e l i e v e s t h a t e v a l u a t i o n and r e p o r t i n g o f p u p i l achievement a r e i n t e g r a l and v i t a l p a r t s o f t e a c h i n g and l e a r n i n g . The Board f u r t h e r b e l i e v e s t h a t e v a l u a t i o n and r e p o r t i n g s h o u l d be based on r e l i a b l e i n f o r m a t i o n about p u p i l performance i n r e l a t i o n t o t h e s p e c i f i c r e q u i r e m e n t s of a program, c o u r s e , or grade. ( S c h o o l D i s t r i c t A d m i n i s t r a t i v e Handbook, 1978) W i th r e l i a b l e and a c c u r a t e r e p o r t i n g , c o n s i s t e n c y among s c h o o l s and c l a s s r o o m s i s m a i n t a i n e d , c r e d i b i l i t y i n t h e eyes of t h e community i s p r e s e r v e d , and e d u c a t i o n a l d e c i s i o n s can be made more c o n f i d e n t l y and c o r r e c t l y . However, the q u e s t i o n remains: Are the t e a c h e r s ' c u r r e n t g r a d i n g p r a c t i s e s as r e l i a b l e and a c c u r a t e as p o s s i b l e ? C urwin (1978), f o r one, c l a i m s t h e y a r e n o t . 5 With the i n t r o d u c t i o n o f t h e l e t t e r grade r e p o r t c a r d s i n B r i t i s h Columbia's e l e m e n t a r y s c h o o l s , t h e r e i s the assumption t h a t t e a c h e r s w i l l have the background knowledge and t h e r e s o u r c e s n e c e s s a r y t o c o l l a t e s c o r e s , grade, and r e p o r t s t u d e n t achievement r e l i a b l y . As i n d i c a t e d p r e v i o u s l y , many t e a c h e r s may not have t h e n e c e s s a r y u n d e r s t a n d i n g or t r a i n i n g t o p e r f o r m such t a s k s w e l l . Many t e a c h e r s s i m p l y do not know how t o combine s t u d e n t achievement s c o r e s i n such a way as t o y i e l d r e l i a b l e l e t t e r grades f o r r e p o r t i n g p u r p o s e s . Teachers agree t h a t g i v i n g and r e p o r t i n g marks (or grades) i s one of t h e i r most u n c o m f o r t a b l e r e s p o n s i b i l i t i e s . Some l a c k c o n f i d e n c e i n the marks t h e y a s s i g n ; o t h e r s b e l i e v e t h e i r marks are f a i r , but f i n d them d i f f i c u l t t o d e f e n d . B e h i n d t h e s e n e g a t i v e a t t i t u d e s l i e s the f a c t t h a t the b a s i s f o r • a s s i g n i n g marks i s o f t e n u n c l e a r . (Ahmann and G l o c k , 1981, p. 417) As a r e s u l t , many t e a c h e r s combine and c a l c u l a t e s c o r e s i n a manner t h a t seems i n t u i t i v e l y r e a s o n a b l e , and by v i r t u e o f t h e i r l a c k o f knowledge, b l i s s f u l l y a c c e p t t h e i r l e t t e r g rades as b e i n g r e l i a b l e assessments o f t h e i r s t u d e n t s ' work. Many assume t h a t by u s i n g raw s c o r e s t h e i r term assignments w i l l be a p p r o p r i a t e l y w e i g h t e d when combining marks f o r t h e i r term t o t a l s . Many t e a c h e r s seem t o r e g a r d the mark v a l u e o f each assignment as b e i n g an i n d i c a t o r o f t h e w e i g h t c o n t r i b u t i n g t o the t o t a l term s c o r e ( i e . an assignment worth 100 marks c o n t r i b u t e s t w i c e as much toward term t o t a l s as does an assignment w o r t h 50 marks). I t would seem t h a t one of t h e 6 more common methods of c o l l a t i n g achievement r e s u l t s f o r g r a d i n g purposes i n v o l v e s s i m p l e t o t a l l i n g of raw s c o r e s o r a v e r a g i n g ( h i g h e s t term t o t a l o r average r e c e i v e s an A e t c . ) . I n extreme c a s e s , where s t a n d a r d d e v i a t i o n s among term assignments a r e v e r y d i f f e r e n t , s i m p l e summations and a v e r a g i n g can r e s u l t i n h i g h l y e r r o n e o u s a g g r e g a t e s c o r e s and u n r e l i a b l e r e p o r t i n g , as w i l l be demonstrated i n a subsequent example. A l s o , some t e a c h e r s s t i l l a b i d e by t h e p e r c e n t a g e ' r u l e - o f - t h u m b ' i n terms of t e s t , a s s i g nment, o r term r e s u l t s . They may, f o r example, c o n s i d e r any s c o r e above 80 per c e n t t o be e q u i v a l e n t t o an 'A', even i n extreme c a s e s where an 80 per c e n t i s the l o w e s t s c o r e o r where the h i g h e s t s c o r e i s l e s s t h a n 8 0 per c e n t . Grades o b t a i n e d from t h e s e and o t h e r s i m i l a r l y q u e s t i o n a b l e methods a r e a c c e p t e d as b e i n g c r e d i b l e and r e l i a b l e by i n t e r e s t e d p a r t i e s (such as p a r e n t s , p r i n c i p a l s , p r o s p e c t i v e e m p l o y e r s , and h i g h e r e d u c a t i o n a l i n s t i t u t i o n s ) and a r e used as a b a s i s f o r making i m p o r t a n t d e c i s i o n s . In s h o r t , t e a c h e r s o f t e n have c o n s i d e r e d n e i t h e r mean s c o r e s nor t h e v a r i a b i l i t y o f s c o r e s when e v a l u a t i n g p u p i l achievement. As w i l l be shown, c a l c u l a t i n g term t o t a l s from s e v e r a l a ssignments may y i e l d unexpected and u n r e l i a b l e r e s u l t s i f t h e c l a s s averages and t h e amount of v a r i a t i o n i n each assignment a r e not c o n s i d e r e d . " I f we hope t o m a i n t a i n the w e i g h t i n g scheme o r i g i n a l l y chosen, we must t a k e i n t o c o n s i d e r a t i o n t h e d i f f e r e n c e s i n v a r i a b i l i t y . A f a i l u r e t o do t h i s w i l l r e s u l t i n i n e q u i t i e s " (Ahmann and G l o c k , 1981, p. 426 ) . 7 B e f o r e examining why mean s c o r e s and s t a n d a r d d e v i a t i o n s s h o u l d be employed, a r e v i e w of how f i n a l o r term t o t a l s a r e t y p i c a l l y o b t a i n e d i s i n o r d e r . F i n a l c o u r s e grades a r e t y p i c a l l y based on v a r i o u s t y p e s o f s t u d e n t achievement, such as s h o r t q u i z z e s , term p r o j e c t s , l a b o r a t o r y work, and e x a m i n a t i o n s . Thus, a s s i g n i n g c o u r s e grades i n v o l v e s combining t h e s e v a r i o u s measures t o o b t a i n a composite o r term s c o r e i n such a way t h a t each assignment r e c e i v e s i t s i n t e n d e d w e i g h t . For example, i f i t has been d e c i d e d t h a t t h e f i n a l e x a m i n a t i o n s h o u l d count t w i c e as much as the s h o r t q u i z z e s , t h e n the grades s h o u l d r e f l e c t t h i s emphasis. S i m i l a r l y , i f the l a b o r a t o r y work i s t o make up 25 per c e n t o f the f i n a l c o u r s e grade, t h e n i t i s i m p o r t a n t t h a t i t w i l l be r e p r e s e n t e d t o t h a t e x t e n t . I t i s not uncommon f o r t e a c h e r s t o communicate t h i s t y p e o f i n f o r m a t i o n t o t h e i r s t u d e n t s a t the b e g i n n i n g of the c o u r s e . S t u d e n t s a r e p a r t i c u l a r l y i n t e r e s t e d i n knowing which elements w i l l be i n c l u d e d i n t h e f i n a l grade and t h e r e l a t i v e w e i g h t each w i l l be g i v e n . D e t e r m i n i n g how much w e i g h t s h o u l d be g i v e n t o each of the v a r i o u s assignments i s a m a t t e r o f judgment. T h i s judgment i s g u i d e d by the importance o f the v a r i o u s i n s t r u c t i o n a l o b j e c t i v e s , the t e a c h i n g emphasis g i v e n t o each t y p e o f c o u r s e a c t i v i t y , and o t h e r s i m i l a r c o n s i d e r a t i o n s . The t e a c h e r must d e c i d e on t h e c o m p o s i t i o n o f the grades g i v e n i n each c o u r s e and t h e r e l a t i v e emphasis t o be a l l o c a t e d t o each component. W i t h o u t a s s i g n i n g d e s i r e d w e i g h t i n g s t o assignments and by u s i n g o n l y the n a t u r a l w e i g h t i n g s , i t i s 8 p o s s i b l e f o r a r e l a t i v e l y minor assignment t o have a v e r y s t r o n g , i f not dominant, i n f l u e n c e on the c o u r s e grade. A l s o , the n a t u r a l w e i g h t i n g s may r e f l e c t a t o t a l l y d i f f e r e n t emphasis on the v a r i o u s i n s t r u c t i o n a l o b j e c t i v e s than was a c t u a l l y used by t h e t e a c h e r d u r i n g t h e c o u r s e . When t h e d e c i s i o n has been made c o n c e r n i n g what p r o p o r t i o n o f the f i n a l a g g r e g a t e s c o r e i s t o be a l l o c a t e d t o each measure o f s t u d e n t performance, t h e measures must be combined i n such a way t h a t t h e d e s i r e d w e i g h t i n g i s o b t a i n e d . T h i s b r i n g s us back t o t h e imp o r t a n c e o f c a l c u l a t i n g c l a s s means and s t a n d a r d d e v i a t i o n s . To i l l u s t r a t e t he problems o f the v a r i o u s g r a d i n g p r a c t i s e s i n t h e w e i g h t i n g of achievement d a t a , which i s t h e problem o f i n t e r e s t i n t h i s s t u d y , c o n s i d e r a s i m p l e example ( t a k e n from G r o n l u n d , 1974). L e t us assume t h a t we want t o combine s c o r e s on an e x a m i n a t i o n and s c o r e s on a term p r o j e c t , and t h a t we want them t o c o n t r i b u t e e q u a l l y t o the f i n a l ( o r term) grade. Suppose t h e range o f s c o r e s on the two measures a r e : Range o f s c o r e s on e x a m i n a t i o n 20 t o 120 Range o f s c o r e s on term p r o j e c t 40 t o 60 I f we s i m p l y add t o g e t h e r the e x a m i n a t i o n s c o r e and t h e p r o j e c t s c o r e f o r each i n d i v i d u a l , t h e f i n a l grade w i l l be de t e r m i n e d l a r g e l y by how w e l l t h e s t u d e n t s performed on t h e e x a m i n a t i o n . To demonstrate t h i s , compare a s t u d e n t who i s h i g h e s t on t h e e x a m i n a t i o n and l o w e s t on t h e term p r o j e c t 9 (Student A) with a student who i s h i g h e s t on the term p r o j e c t and lowest on the examination (Student B). Student A Student B Examination score 120 20 Term p r o j e c t s c o r e 40 60 Composite score 160 80 I t i s obvious t h a t simply adding the two scores w i l l not gi v e them equal r e p r e s e n t a t i o n i n the composite s c o r e . Nor w i l l c a l c u l a t i n g an average score p r o v i d e equal weighting. The examination score has much g r e a t e r i n f l u e n c e than the term p r o j e c t s c o r e . In a s i t u a t i o n such as t h i s , t eachers f r e q u e n t l y attempt t o equate the i n f l u e n c e of the two measures by making the maximum p o s s i b l e score f o r both assignments e q u i v a l e n t . In the above example, t h i s would mean m u l t i p l y i n g the s t u d e n t s ' term p r o j e c t score by two t o make the t o t a l p o s s i b l e score 120 f o r both measures of achievement. The f o l l o w i n g shows what happens when t h i s i s done: Student A Student B Examination score 120 20 Term p r o j e c t score (x2) 80 120 Composite score 200 140 10 As t h e example d e m o n s t r a t e s , e q u a t i n g on t h e b a s i s of maximum p o s s i b l e s c o r e does not p r o v i d e f o r e q u a l emphasis i n the c o m p o s i t e s c o r e e i t h e r . . The e x a m i n a t i o n s c o r e s t i l l has t h e g r e a t e r i n f l u e n c e when t h e two a r e combined. T h i s i s because th e c o n t r i b u t i o n a g i v e n measure makes t o a composite s c o r e depends l a r g e l y on the v a r i a b i l i t y , o r s p r e a d , o f the s c o r e s i n t h e s e t . To equate the e x a m i n a t i o n s c o r e and t h e term p r o j e c t s c o r e , t h e v a r i a t i o n o f the s c o r e s i n each s e t must be made e q u a l . T h i s i s done by a d j u s t i n g the s t a n d a r d d e v i a t i o n o f one o f t h e s e t s o f s c o r e s so t h a t i t i s e q u a l t o t h e o t h e r s e t . The method most o f t e n used i s the s t a n d a r d T-score t r a n s f o r m a t i o n . T h i s changes each s e t so .that t h e c l a s s average (mean) i s 50 and t h e s t a n d a r d d e v i a t i o n i s 10. T h i s i s n ot an easy t a s k w i t h o u t t h e use of a computer o r a s o p h i s t i c a t e d c a l c u l a t o r , a g r e a t d e a l o f t i m e , and some s t a t i s t i c a l knowledge. However, once the s c o r e s a r e ' b a l a n c e d ' , w e i g h t i n g s can be a p p l i e d t o each t r a n s f o r m e d s c o r e w i t h t h e c o n f i d e n c e t h a t t h e d e s i r e d w e i g h t i n g s w i l l be o b t a i n e d . I n t h i s s i m p l e example, th e two a ssignments a r e w e i g h t e d e q u a l l y . Our example would t h e n l o o k l i k e t h i s : S tudent A Student B E x a m i n a t i o n T-score (weighted) Term p r o j e c t T - s c o r e (weighted) 100 80 80 100 Composite T-score (weighted) 180 180 11 C l e a r l y , the e x a m i n a t i o n and the term p r o j e c t s c o r e s a re e q u a l ; b o t h s t u d e n t s r e c e i v e t h e same composite s c o r e . T h i s i s p o s s i b l e o n l y by e q u a t i n g the two assignments i n terms o f means and s t a n d a r d d e v i a t i o n s . I n t h i s c a s e , t h e s t a n d a r d T-score was used f o r bo t h e x e r c i s e s . The main q u e s t i o n s t o be ad d r e s s e d t h e n a r e t h e f o l l o w i n g : 1. I s t h e r e a s i g n i f i c a n t d i f f e r e n c e between t h e grades r e p o r t e d by t e a c h e r s u s i n g i n t u i t i v e methods and t h o s e r e p o r t e d when means and s t a n d a r d d e v i a t i o n s a r e u t i l i z e d and a p p l i e d ? I n o t h e r words, i f t e a c h e r s were t o f o l l o w t h e a d v i c e o f many a u t h o r s and employ b a s i c s t a t i s t i c s when a g g r e g a t i n g marks, would t h e r e s u l t s be s i g n i f i c a n t l y d i f f e r e n t from t h o s e o b t a i n e d from e x i s t i n g g r a d i n g p r a c t i s e s ? 2. Are the raw achievement s c o r e s combined and w e i g h t e d r e l i a b l y ? 3. Are the methods used t o compensate s t u d e n t s who have mis s e d a ssignments o r t e s t s f a i r and r e l i a b l e ? Are s t u e n t s g i v e n grades t h a t p e n a l i z e o r reward them f o r b e i n g absent f o r a t e s t o r assignment? 4. Would any improvement i n r e s u l t s be o f such a magnitude t o j u s t i f y t he time and expense r e q u i r e d t o r e t r a i n t e a c h e r s ? I t was t h e purpose o f t h i s s t u d y t o i n v e s t i g a t e t h e s e q u e s t i o n s t o de t e r m i n e i f t e a c h e r s ' a g g r e g a t i n g and g r a d i n g methods c o u l d be improved. 12 DEFINITION OF TERMS The f o l l o w i n g are terms used i n t h i s study: 1. ASSIGNMENT SCORES: "Assignments" r e f e r t o the v a r i o u s e x e r c i s e s , t e s t s , l a b o r a t o r y work, term p r o j e c t s , e t c e t e r a , t h a t a teacher might a s s i g n h i s students d u r i n g the course of i n s t r u c t i o n i n a p a r t i c u l a r s u b j e c t . An "assignment score" i s the r e s u l t t h a t each student would r e c e i v e a f t e r completing a p a r t i c u l a r assignment. At the end of each term, a t o t a l or "term s c o r e " i s obtained f o r each student by summing t h e i r r e s p e c t i v e r e s u l t s on each assignment. For r e p o r t i n g purposes each student's term score i s then ranked l a r g e s t t o s m a l l e s t and l e t t e r grades are assig n e d based on the rankings of the t o t a l s c o r e s . 2. BALANCED SCORES: "Balanced Scores" or " s t a t i s t i c a l l y balanced s c o r e s " r e f e r to assignment scores t h a t have undergone a l i n e a r t r a n s f o r m a t i o n t o a l t e r the mean and standard d e v i a t i o n . When s e v e r a l c l a s s assignments have been "balanced", they have been g i v e n equal means and equal standard d e v i a t i o n s . Balanced assignments contribute- e q u a l l y towards the f i n a l , course mark. 3. FAIR: For the purpose of t h i s study, " f a i r " i s used to mean " j u s t " or "not f a v o u r i n g one more than t h e ' o t h e r " . F a i r grades are those t h a t do not p e n a l i z e or reward i n d i v i d u a l students more than they do other students. F a i r grades are a l s o c o n s i s t e n t . For example, two students having 13 the same raw scores and subjected t o i d e n t i c a l grading c r i t e r i a would be expected to r e c e i v e i d e n t i c a l grades. However, i f one of the students were t o r e c e i v e a higher grade than the other, then the grades would not be c o n s i s t e n t and would be considered " u n f a i r " . 4. LETTER GRADE: For the purpose of t h i s study, a l e t t e r grade i s defined as: "1) a summary symbol, 2) e v a l u a t i n g a s u b s t a n t i a l segment of achievement which i s 3) a t t a i n e d by a p u p i l i n a course and 4) assigned by a teacher 5) f o r the purpose of record and r e p o r t " ( M i n i s t r y of Education, Science and Technology, 1979, p. 9). A l e t t e r grade should be reserved f o r judgements t h a t i n c l u d e evidence a v a i l a b l e on student achievement i n a complete course or a segment of a course such as a r e p o r t i n g p e r i o d . L e t t e r grades are u s u a l l y derived from the evidence a v a i l a b l e i n a set of scores f o r each p u p i l . I t i s a l s o assumed that a teacher has a systematic procedure f o r a s s i g n i n g grades. Because a l e t t e r grade i s "used t o record and r e p o r t " , i t i s assumed th a t a l e t t e r grade i s a "permanent record u s u a l l y r e t a i n e d f o r a long p e r i o d of time i n the school record system and used t o communicate with i n d i v i d u a l students, parents, teachers, and other educational a u t h o r i t i e s and p o t e n t i a l employers" ( M i n i s t r y of Education, Science and Technology, 1979, p. 10). 5. MEAN: "Mean" i s synonymous with " a r i t h m e t i c mean" or "average" and i s a measure of c e n t r a l tendency. I t i s c a l c u l a t e d by summing a s e r i e s of numbers and d i v i d i n g the sum by the number of numbers summed. The value of the mean i s 14 a f f e c t e d by the i n d i v i d u a l v a l u e s o f a l l o f the s c o r e s i n the se t o f d a t a . I n t h i s s t u d y , t h e mean i s denoted as X and 2 m i n d i v i d u a l raw s c o r e s as X.. 1 6. RELIABLE: R e l i a b l e grades a r e t h o s e t h a t a r e "dependable and f i t t o be r e l i e d upon". Grades t h a t a r e r e l i a b l e p r o v i d e an a c c u r a t e i n d i c a t i o n of a s t u d e n t ' s performance and a r e worthy o f b e i n g used as a b a s i s f o r d e c i s i o n making. 7. T-SCORES: "T-Scores" a r e a l i n e a r t r a n s f o r m a t i o n of s t a n d a r d s c o r e s . S t a n d a r d i z e d T - s c o r e s a re t r a n s f o r m e d t o a s e t of d a t a h a v i n g a mean o f 50 and s t a n d a r d d e v i a t i o n of 10. C a l c u l a t i o n o f T-sc o r e s i s a c h i e v e d by m u l t i p l y i n g a s t a n d a r d s c o r e (z s c o r e ) by 10 and a d d i n g 50 t o t h e p r o d u c t . The shape of t h e d i s t r i b u t i o n i s not changed w i t h the t r a n s f o r m e d s c o r e s . 8. STANDARD DEVIATION: "Sta n d a r d D e v i a t i o n " i s a measure of d i s p e r s i o n , s c a t t e r , h e t e r o g e n e i t y , o r v a r i a t i o n i n a s e t of d a t a . A s e t o f s c o r e s w i t h g r e a t h e t e r o g e n e i t y w i l l have some l a r g e d e v i a t i o n s c o r e s (X.-X ). St a n d a r d d e v i a t i o n i s c a l c u l a t e d by f i r s t summing the squared d e v i a t i o n s c o r e s and then d i v i d i n g the sum by the number o f s c o r e s . The square r o o t o f the q u o t i e n t i s the s t a n d a r d d e v i a t i o n . I n t h i s s t u d y , t h e s t a n d a r d d e v i a t i o n i s denoted as SD. 9. STANDARD SCORES: "Sta n d a r d S c o r e s " can be used t o d e s c r i b e t h e p o s i t i o n o f a s c o r e i n a s e t o f s c o r e s by measuring i t s d e v i a t i o n from t h e mean o f a l l s c o r e s i n s t a n d a r d d e v i a t i o n u n i t s . A s t a n d a r d s c o r e ( c a l l e d a z - s c o r e ) 15 i s o b t a i n e d by s u b t r a c t i n g the mean from an ob s e r v e d s c o r e and d i v i d i n g t h e d i f f e r e n c e by t h e s t a n d a r d d e v i a t i o n . The mean and s t a n d a r d d e v i a t i o n o f the t r a n s f o r m e d raw s c o r e s ( z - s c o r e s ) a r e 0 and 1 r e s p e c t i v e l y . I n t h i s way, t h e n , the d i f f e r e n c e between an i n d i v i d u a l s c o r e and t h e mean s c o r e of the group, i s s c a l e d i n terms of s t a n d a r d d e v i a t i o n u n i t s . T h i s has t h e advantage i n t h a t a l l d e v i a t i o n s c o r e s a r e measured i n the same u n i t s . 10. TERM: A "term" i s an i n s t r u c t i o n a l p e r i o d where c o u r s e s can be o f f e r e d i n whole o r i n p a r t . The end o f each term i s n o r m a l l y marked by t h e e v a l u a t i o n and r e p o r t i n g of s t u d e n t achievement i n each o f t h e c o u r s e s o f f e r e d d u r i n g t h a t p e r i o d . I t i s s t i p u l a t e d by t h e B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n t h a t , f o r s c h o o l s i n B r i t i s h C o lumbia, t h e r e w i l l be a t l e a s t 3 r e p o r t i n g p e r i o d s ( o r terms) d u r i n g each s c h o o l y e a r . 11. WEIGHTING: "W e i g h t i n g " r e f e r s t o t h e amount of i n f l u e n c e a p a r t i c u l a r e x e r c i s e o r assignment has i n d e t e r m i n i n g the t o t a l s c o r e f o r t h a t s u b j e c t . F o r example, an e x e r c i s e might c o n t r i b u t e 20 per c e n t toward t h e t o t a l mark f o r t h e c o u r s e , meaning t h a t 20 per c e n t o f t h e t o t a l c o u r s e mark i s d e r i v e d from t h a t p a r t i c u l a r a ssignment. W e i g h t i n g s h o u l d n o r m a l l y r e f l e c t t h e im p o r t a n c e o r the emphasis p l a c e d on the o b j e c t i v e s c o v e r e d i n t h e assignment. "NATURAL WEIGHTING" r e f e r s t o the c o n t r i b u t i o n an assignment g i v e s t o the c o u r s e mark u s i n g o n l y raw s c o r e d a t a . I n o t h e r words, f o r a g i v e n s e t o f assignments i n a p a r t i c u l a r c o u r s e , each 16 assignment w i l l c o n t r i b u t e a c e r t a i n p e r c e n t a g e o f the t o t a l s c o r e . The c o n t r i b u t i o n w i l l be d e t e r m i n e d by the v a r i a b i l i t y of t h e raw s c o r e s . T h i s raw s c o r e i n f l u e n c e i s c a l l e d " n a t u r a l w e i g h t i n g " . R e s e a r c h Q u e s t i o n s S p e c i f i c a l l y , t h e q u e s t i o n s o f i n t e r e s t a r e : 1. I s t h e r e a s i g n i f i c a n t d i f f e r e n c e between the r a n k i n g o b t a i n e d from a t e a c h e r ' s c o u r s e r e c o r d and t h e r a n k i n g o b t a i n e d from s t a t i s t i c a l l y b a l a n c i n g t h i s same r e c o r d ? 2. Do a t e a c h e r ' s c o u r s e assignments c o n t r i b u t e t h e i n t e n d e d p o r t i o n o f marks towards the c o u r s e t o t a l ? (Are the term t o t a l s , from which t h e l e t t e r grades a r e d e r i v e d , c o l l a t e d and w e i g h t e d r e l i a b l y ? ) 3. Are t e a c h e r s ' grades r e l i a b l e and f a i r f o r s t u d e n t s m i s s i n g one o r more assignments o r t e s t s ? Are s t u d e n t s compensated f a i r l y and r e l i a b l y when t h e y have been absent and have n o t r e c e i v e d a s c o r e f o r one o r more assignments o r t e s t s ? 4. Would the r e v i s e d r e s u l t s be o f such a magnitude t o j u s t i f y t h e t i m e and expense r e q u i r e d t o r e t r a i n t e a c h e r s and r e v i s e u n i v e r s i t y c o u r s e s and/or programs? In t h i s s t u d y , t h e q u e s t i o n s of i n t e r e s t r e l a t e t o the 17 methods used by t e a c h e r s t o c o l l a t e achievement d a t a f o r r e p o r t i n g p u r p o s e s . The h y p o t h e s i s t o be t e s t e d i s t h a t t e a c h e r s , f o r one r e a s o n o r a n o t h e r , do not c o l l a t e achievement s c o r e s r e l i a b l y . I t i s assumed t h a t t e a c h e r s do not c o n s i d e r t h e c a l c u l a t i o n o f c l a s s means o r s t a n d a r d d e v i a t i o n s when combining achievement s c o r e s . As a r e s u l t , the assignment s c o r e s a r e c o n t r i b u t i n g d i s p r o p o r t i o n a t e w e i g h t s toward t h e c o u r s e t o t a l s . I t i s h y p o t h e s i z e d t h a t t h e method most commonly used by t e a c h e r s t o combine c o u r s e marks i s s i m p l e summation o f t h e raw s c o r e s f o l l o w e d by r a n k i n g each s t u d e n t ' s t o t a l . I t i s a l s o h y p o t h e s i z e d t h a t the summation method of r a n k i n g w i l l be s i g n i f i c a n t l y d i f f e r e n t from r a n k i n g s o b t a i n e d from s t a t i s t i c a l l y b a l a n c e d assignment s c o r e s . R a t i o n a l e P r i o r t o the i n t r o d u c t i o n o f the l e t t e r grade r e p o r t c a r d f o r e l e m e n t a r y s c h o o l s i n 1979, el e m e n t a r y s c h o o l r e p o r t c a r d s were p r i m a r i l y a n e c d o t a l , and achievement symbols (Needs Improvement, S a t i s f a t o r y , Good), were based l a r g e l y upon s u b j e c t i v e e v a l u a t i o n . When t h e format o f t h e r e p o r t was changed t o c o m p a r a t i v e achievement and l e t t e r grades were used t o r e f l e c t t h i s achievement, t e a c h e r s were g i v e n v e r y l i t t l e i n - s e r v i c e t r a i n i n g on t h e new r e p o r t i n g p r o c e d u r e . The B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n d i d p u b l i s h b o o k l e t s 18 ( " C o n s t r u c t i o n and Uses of C l a s s r o o m T e s t s : A Resource Book For T e a c h e r s " and " G r a d i n g P r a c t i c e s : I s s u e s and A l t e r n a t i v e s " ) t o a s s i s t t e a c h e r s i n t h e i r e v a l u a t i o n o f s t u d e n t achievement. A l t h o u g h t h e s e b o o k l e t s were made a v a i l a b l e , i t appears t h a t few t e a c h e r s s t u d i e d them i n d e t a i l p r i o r t o u s i n g t h e new r e p o r t c a r d s . C o n s e q u e n t l y , i t would seem t h a t few t e a c h e r s a r e aware of t h e i m p o r t a n c e o f the mean and s t a n d a r d d e v i a t i o n when a g g r e g a t i n g assignment s c o r e s . Most t e a c h e r s use o n l y t h e raw s c o r e s t o t a l f o r d e t e r m i n i n g t h e r a n k i n g o f t h e i r s t u d e n t s . Many e d u c a t o r s seem t o f o l l o w the erroneous b e l i e f t h a t the assignment w i t h the l a r g e s t p o s s i b l e s c o r e s h o u l d c o n t r i b u t e the g r e a t e s t p e r c e n t a g e toward t h e f i n a l c o u r s e mark. The absence o f s t a t i s t i c a l a n a l y s i s and ' b a l a n c i n g ' i n the p a s t may have been p a r t i a l l y due t o a l a c k o f t r a i n i n g and p a r t i a l l y due t o a l a c k o f time and r e s o u r c e s . A s t a t i s t i c a l c a l c u l a t o r i s a l m o s t a n e c e s s i t y b u t a g r e a t d e a l o f t e a c h e r time i s s t i l l r e q u i r e d t o c a l c u l a t e the s t a n d a r d i z e d s c o r e s - time t h a t some might f i n d d i f f i c u l t t o j u s t i f y i n l i g h t of t h e many o t h e r p r o f e s s i o n a l commitments of t e a c h e r s . However, w i t h t h e advent of t h e micro-computer, "mark-keeping" programs can be d e v i s e d t o t a k e the time consumption and d r u d g e r y out of t h e s t a t i s t i c a l b a l a n c i n g o f s t u d e n t assignment s c o r e s . Even so , many t e a c h e r s seem t o a b i d e by t h e i r p r e v i o u s l y e s t a b l i s h e d g r a d i n g p r a c t i s e s . They a r e n o t aware of the need t o c o n s i d e r the e f f e c t s o f d i f f e r e n t v a r i a n c e s w i t h i n t h e i r assignment s c o r e s , and t h e r e f o r e a r e o b l i v i o u s t o the need f o r 19 improved g r a d i n g p r a c t i s e s . H i l l s (1981) s u p p o r t s t h i s n o t i o n and s t a t e s , "The s t a n d a r d d e v i a t i o n . . . i s what most t e a c h e r s would not t h i n k about w i t h o u t some t r a i n i n g " (p. 319). Perhaps i f t h e r e i s a p e r c e i v e d need f o r a change i n the way s c o r e s a r e c o l l a t e d , t e a c h e r s w i l l be r e c e p t i v e t o an a l t e r n a t e and improved method o f combining s t u d e n t s ' s c o r e s f o r r e p o r t i n g p u r p o s e s . I t i s hoped t h a t t h e outcome of t h i s s t u d y w i l l h e l p t o d e t e r m i n e i f a more e f f e c t i v e method of combining achievement s c o r e s i s needed and whether more i n - s e r v i c e and p r e - s e r v i c e t r a i n i n g i s r e q u i r e d t o improve the g r a d i n g p r a c t i s e s of t e a c h e r s . D e l i m i t a t i o n s F or t h e purposes of t h i s s t u d y , t h e sample was s e l e c t e d a t random from t e a c h e r s o f grades f o u r t o seven l e v e l who t e a c h i n a l a r g e m e t r o p o l i t a n a r e a of c e n t r a l B r i t i s h Columbia. Examples of t e a c h e r s ' r e c o r d s were s e l e c t e d from two s u b j e c t a r e a s : mathematics and s o c i a l s t u d i e s . The i n s t r u m e n t s used by the sample s u b j e c t s t o c o l l e c t s t u d e n t achievement d a t a were not e v a l u a t e d . I t was assumed t h a t a l l i n s t r u m e n t s were r e l i a b l e and v a l i d . We were i n t e r e s t e d o n l y i n t h e combined t o t a l s c o r e f o r t h e term and t h e a s s i g n e d l e t t e r grade f o r each s t u d e n t as d e t e r m i n e d from t h e c o l l a t i o n of term o r c o u r s e assignment s c o r e s . T h i s s t u d y was not concerned w i t h t h e v a l i d i t y of the 20 e d u c a t i o n a l o b j e c t i v e s f o r each o f t h e two s u b j e c t s sampled. I t s h o u l d be r e c o g n i z e d t h a t w i t h i n t h e c u r r i c u l u m s t i p u l a t e d by t h e P r o v i n c i a l M i n i s t r y o f E d u c a t i o n , c o n s i d e r a b l e f l e x i b i l i t y i s p r o v i d e d t o e l e m e n t a r y t e a c h e r s . E d u c a t o r s may s e l e c t from a v a r i e t y o f c o n t e n t , media, p e d a g o g i c a l t e c h n i q u e s , and whatever e l s e i s n e c e s s a r y , t o f u l l f i l t h e demands o f t h e c u r r i c u l u m and the needs o f t h e s t u d e n t s i n t h e most e f f e c t i v e manner. C o n s e q u e n t l y , the c o u r s e c o n t e n t may v a r y among t e a c h e r s of t h e same s u b j e c t . U n i t s and u n i t o b j e c t i v e s may a l s o be t a u g h t w i t h d i f f e r e n t emphases. T h i s f l e x i b i l i t y n u l l i f i e s any o p p o r t u n i t y t o " s t a n d a r d i z e " e l e m e n t a r y s u b j e c t s i n terms o f u n i t s , u n i t o b j e c t i v e s , a s s i g n m e n t s , and assignment emphasis on c o u r s e marks. As a r e s u l t , the v a l i d i t y , emphasis, and a p p r o p r i a t e w e i g h t i n g s o f the o b j e c t i v e s were o u t s i d e the scope of t h i s s t u d y . They were assumed t o be a p p r o p r i a t e f o r t h e grade l e v e l , t he s t u d e n t s , and t h e c u r r i c u l u m . J u s t i f i c a t i o n of the Study Why i s t h i s i s s u e i m p o r t a n t i n e d u c a t i o n ? F i r s t l y , s t u d e n t achievement s h o u l d be r e p o r t e d as r e l i a b l y and a c c u r a t e l y as p o s s i b l e . " F a u l t y g r a d i n g can be s e r i o u s l y d e t r i m e n t a l t o s t u d e n t s . I t can g i v e them m i s l e a d i n g i n f o r m a t i o n and i n t e r f e r e w i t h t h e l e a r n i n g p r o c e s s " ( H i l l s , 1981, p. 281). S e c o n d l y , p a r e n t s , p r o s p e c t i v e e mployers, and 21 the s t u d e n t s themselves v i e w r e p o r t c a r d grades as a c c u r a t e i n d i c a t o r s o f s t u d e n t achievement and use the i n f o r m a t i o n as a b a s i s f o r d e c i s i o n - m a k i n g and f o r e v a l u a t i n g p e o p l e . T h i r d l y , u n r e l i a b l e o r i n a c c u r a t e grades may l e a d t o erroneous d e c i s i o n s r e g a r d i n g b u r s a r i e s , s c h o l a r s h i p s , d i p l o m a s , o r o t h e r such awards based on g r a d e s . More i m p o r t a n t l y , improper assessment o f s t u d e n t performance may r e s u l t i n i n c o r r e c t d e c i s i o n s r e g a r d i n g a s t u d e n t ' s s u c c e s s i n a p a r t i c u l a r c o u r s e . I n a d d i t i o n , i n a c c u r a t e g r a d i n g "can m i s l e a d t h o s e who would s t u d y the i n s t r u c t i o n a l p r o c e s s t o improve i t o r t o h o l d i t a c c o u n t a b l e t o t h e s o c i e t y t h a t s u p p o r t s i t " ( H i l l s , p. 281). A f i f t h r e a s o n why t h i s i s s u e , i s i m p o r t a n t t o e d u c a t i o n i s due t o the f a c t t h a t t h e r e a r e many m i s c o n c e p t i o n s and c o n f u s i o n s among e d u c a t o r s as t o what c o n s t i t u t e s an e f f e c t i v e g r a d i n g p r o c e d u r e . H i l l s a l s o s u p p o r t s t h i s p e r c e p t i o n and s t a t e s , " t h e r e a r e many i r r a t i o n a l i d e a s about g r a d i n g t h a t seem sound u n t i l t h e y a r e c o n s i d e r e d i n d e t a i l . To a v o i d some o f t h e s e i r r a t i o n a l n o t i o n s , a t e a c h e r must l e a r n t o r e c o g n i z e what i s wrong w i t h them" (p. 280). The i m p l i c a t i o n s o f t h i s s t u d y p e r t a i n t o t e a c h e r p r e - s e r v i c e and i n - s e r v i c e t r a i n i n g . I f t h e samples o f t e a c h e r g r a d i n g p r a c t i s e s a r e s i g n i f i c a n t l y i n f e r i o r when compared t o the s t a t i s t i c a l l y b a l a n c e d and t r a n s f o r m e d s c o r e s ( T - s c o r e s ) , t h e n i t may be pr u d e n t t o examine t h e p r e - s e r v i c e t r a i n i n g t e a c h e r s r e c e i v e i n e v a l u a t i n g s t u d e n t performance 22 and achievement. I s t h i s an a r e a t h a t needs more a t t e n t i o n i n t h e p r e - s e r v i c e t r a i n i n g of t e a c h e r s ? S h o u l d u n i v e r s i t i e s make c o u r s e s i n e v a l u a t i o n mandatory f o r s t u d e n t - t e a c h e r s ? A l s o , i f i t i s apparent t h a t g r a d i n g p r a c t i s e s can be improved, s h o u l d s c h o o l d i s t r i c t s d e v e l o p i n - s e r v i c e programs t o enhance t e a c h e r g r a d i n g p r a c t i s e s ? These a r e i m p o r t a n t q u e s t i o n s i f the r e s u l t s show t h a t g r a d i n g p r a c t i s e s can be improved. Many a u t h o r s r e f l e c t t h i s i m p o r t a n c e by s u g g e s t i n g t h a t t e a c h e r s s h o u l d s t r i v e t o improve t h e i r g r a d i n g p r o c e d u r e s . H i l l s ( 1981), f o r example, s t a t e s , G r a d i n g i s a c o m p l i c a t e d problem. There a r e many p i t f a l l s ; e r r o n e o u s p r a c t i s e s abound; and t h e t e a c h e r cannot s a f e l y p r o c e e d on the assumption t h a t what was done when she was a p u p i l i s s a t i s f a c t o r y f o r t h e s t u d e n t s now i n her c a r e . Good t e a c h e r s w i l l t r y t o do a b e t t e r job t h a n was done f o r them as p u p i l s , r a t h e r t h a n s i m p l y c a r r y i n g f o r w a r d t h e m i s t a k e s o f the p a s t . (p. 280) On the o t h e r hand, i f t h e p r e s e n t g r a d i n g p r a c t i s e s a r e not s i g n i f i c a n t l y i n f e r i o r and a r e i n f a c t g e n e r a l l y as good as t h e s t a t i s t i c a l l y b a l a n c e d g r a d e s , t h e n we can c o n c l u d e t h a t t e a c h e r s , s i m i l a r t o t h o s e i n t h e sample, a r e e v a l u a t i n g and g r a d i n g s t u d e n t achievement e f f e c t i v e l y . I f t h i s i s t h e c a s e , t h e n u n i v e r s i t i e s and s c h o o l d i s t r i c t s need not c o n c e r n themselves w i t h r e v i s i n g t h e i r p r e s e n t measurement and s t u d e n t e v a l u a t i o n components o r programs. The i m p l i c a t i o n would be 2.3 t h a t t h e t r a i n i n g t e a c h e r s r e c e i v e and t h e e v a l u a t i o n t e c h n i q u e s t e a c h e r s employ a r e b o t h adequate and r e l i a b l e . 24 CHAPTER 2 Survey of t h e L i t e r a t u r e From t h e r e v i e w o f the l i t e r a t u r e , t h e r e appear t o be v e r y few e m p i r i c a l o r i n v e s t i g a t i v e s t u d i e s done t o a s s e s s the g r a d i n g p r a c t i s e s o f t e a c h e r s . A l t h o u g h t h e r e i s an abundance of t h e o r e t i c a l i n f o r m a t i o n on how t e a c h e r s s h o u l d e v a l u a t e and r e p o r t t h e achievement o f t h e i r s t u d e n t s , i t seems t h a t no one has a c t u a l l y t r i e d t o f i n d o u t i f t e a c h e r s a r e p u t t i n g t h e o r y i n t o p r a c t i s e . H i l l s (1981) r e i n f o r c e s t h i s f a c t when he s t a t e s , There i s l i t t l e e m p i r i c a l r e s e a r c h on g r a d i n g . A l t h o u g h some f a c t s a r e w e l l - e s t a b l i s h e d , t h e r e has been f a r t o o l i t t l e s t u d y of such an i m p o r t a n t t o p i c . T h e r e f o r e , everyone seems t o f e e l e n t i t l e d t o an o p i n i o n , and no da t a e x i s t t o r e f u t e most o f them. (p. 317) S e v e r a l a u t h o r s , however, have su g g e s t e d i n n o v a t i v e methods of g r a d i n g f o r s p e c i a l e d u c a t i o n a l s i t u a t i o n s o r f o r p a r t i c u l a r c o u r s e s . F o r example, D a v i d Cohen (1973) d i s c u s s e s a means o f i m p r o v i n g the e v a l u a t i o n o f the A u s t r a l i a n s c i e n c e c u r r i c u l u m . G e n s l e y (1969) p r o v i d e s an i n s i g h t i n t o a l t e r n a t e methods of e v a l u a t i n g " g i f t e d c h i l d r e n " . Many o t h e r w r i t e r s advocate a l t e r n a t e forms o f e v a l u a t i o n f o r t h e i r p a r t i c u l a r s u b j e c t a r e a o r f o r t h e i r p a r t i c u l a r s p e c i a l t y s i t u a t i o n . However, none o f the a u t h o r s o f f e r any i n s i g h t i n t o the adequacy o f t h e g r a d i n g p r a c t i s e s o f " r e g u l a r " c l a s s r o o m t e a c h e r s . Other w r i t e r s e i t h e r a t t a c k o r defend t h e use o f l e t t e r 25 grades as a symbol o f s t u d e n t achievement. For example, W.J. S t e w a r t (1975) condemns l e t t e r grades and p r o f e s s e s a new system f o r e v a l u a t i n g t h e " m u l t i - f a c e t e d a c a d e m i c - p e r s o n a l - s o c i a l growth of e l e m e n t a r y s c h o o l s t u d e n t s " (p. 174). He s t a t e s t h a t one grade does not r e p r e s e n t a d e q u a t e l y t h e c o m p l e x i t y i n a c h i l d ' s academic, s o c i a l , and e m o t i o n a l growth. I n s t e a d , he recommends a m u l t i - d i m e n s i o n a l system t h a t would accommodate (a) a wide range of g o a l s , (b) uniqueness of each s t u d e n t ' s a b i l i t i e s and needs, (c) an account f o r each c h i l d ' s need t o d e v e l o p a p o s i t i v e s e l f - c o n c e p t , and (d) a need t o p r o v i d e p a r e n t s w i t h more u s e f u l and more m e a n i n g f u l i n f o r m a t i o n r e g a r d i n g a c h i l d ' s p r o g r e s s . S t e w a r t c l a i m s t h a t h i s e v a l u a t i o n - r e p o r t i n g system would de-emphasize s k i l l and s u b j e c t m a t t e r mastery but would s t r e s s a c h i l d ' s a b i l i t y t o use s k i l l s and s u b j e c t m a t t e r f o r d e a l i n g w i t h l i f e ' s d a i l y problems. H i s m u l t i - d i m e n s i o n a l system would f o c u s o n l y upon i n d i v i d u a l s and would not s t r e s s comparisons w i t h o t h e r c l a s s members as do l e t t e r g r a d e s . W h i l e S t e w a r t p o i n t s out what may be c o n s i d e r e d by some t o be weaknesses o f t h e l e t t e r grade system, he does not address what t e a c h e r s a r e p r e s e n t l y d o i n g t o cope w i t h t h e problems he i d e n t i f i e s . He c i t e s no r e s e a r c h t o j u s t i f y h i s c l a i m t h a t a new system o f " e v a l u a t i o n - r e p o r t i n g " i s r e q u i r e d . M a r s h a l l (1971) a l s o c r i t i c i z e s t h e l e t t e r grade system. He c l a i m s t h a t grades ( o r any "codes w h i c h r e f e r t o r a n k , grade, o r p o s i t i o n on a s c a l e " ) t e n d t o be v e r y s u b j e c t i v e . He s t a t e s , "Grades s u r v i v e p r i m a r i l y because t h e y p r o v i d e 26 d i s g u i s e . Almost a n y t h i n g can be, and i s , r e a d i n t o them. L e t t e r grades . . . are g e n e r a l terms, i n t h e m s e l v e s m e a n i n g l e s s " (p. 350). M a r s h a l l i n d i c a t e s t h e b e n e f i t s o f an " i n d i v i d u a l d e s c r i p t i v e o r n o n g r a d i n g " system of e v a l u a t i o n and r e p o r t i n g t h a t would i n c l u d e p r o v i d i n g t e a c h e r s w i t h "more t i m e , g r e a t e r freedom, and new i n t e r e s t s i n t e a c h i n g " (p. 352). He a l s o s t a t e s t h a t " t e a c h e r s a r e employed t o t e a c h , not t o grade o r p s y c h o a n a l y z e " (p. 352). He c o n c l u d e s by s t a t i n g , "A t e a c h e r s h o u l d want, be a l l o w e d , and be asked t o say what he o r she means, i n words s p e c i f i c a l l y p e r t i n e n t t o a g i v e n i n q u i r y o r s i t u a t i o n " (p. 353). As w i t h S t e w a r t , M a r s h a l l makes no mention of t e a c h e r s ' c u r r e n t g r a d i n g p r a c t i s e s . A t h i r d c r i t i c i s m o f the l e t t e r grade system i s s t a t e d by R i c h a r d Curwin (1978). He c l a i m s t h a t "grades don't work" because t h e y l a c k a c c u r a c y , c o n s i s t e n c y , v a l i d i t y , r e l i a b i l i t y , and p r e d i c t a b i l i t y . He o f f e r s f o u r a l t e r n a t i v e s ( S e l f - G r a d i n g , C o n t r a c t G r a d i n g , Peer G r a d i n g , and B l a n k e t G r a ding) t o "reduce the dangers of the g r a d i n g system" (p. 6 1 ) . A l t h o u g h he s t a t e s t h a t " r e s e a r c h s t u d i e s c l e a r l y i n d i c a t e t h a t t e a c h e r s grade th e same s t u d e n t s ' work d i f f e r e n t l y " (p. 6 0 ) , he does not e l a b o r a t e on the f i n d i n g s o r the n a t u r e o f the r e s e a r c h . He ends by s t a t i n g t h a t t h e a l t e r n a t i v e s "undoubtedly w i l l be an improvement over t r a d i t i o n a l g r a d i n g systems" (p. 6 4 ) . Defenders of l e t t e r grades as symbols of s t u d e n t achievement t a k e a s t a n c e o p p o s i t e t o the a u t h o r s mentioned 27 above. Rob e r t L. E b e l (1974), f o r example, defends grades a g a i n s t the c r i t i c i s m s o f t e n aimed a t g r a d i n g . He c l a i m s t h a t g r a d i n g can be improved. "The remedy i s not t o get r i d o f grades bu t s i m p l y t o do a b e t t e r j o b o f g r a d i n g " (p. 3 ) . He i n d i c a t e s t h a t a l l a l t e r n a t i v e s t o the grade a r e l e s s c o s t e f f i c i e n t and l e s s i n f o r m a t i v e t h a n the " f a m i l i a r summary s t a t i s t i c , t h e grade." A l t h o u g h E b e l c i t e s no r e s e a r c h t o demonstrate how c l a s s r o o m t e a c h e r s c o l l e c t achievement i n f o r m a t i o n upon which t o base the assignment of g r a d e s , he does e l a b o r a t e on how grades can be improved. He, l i k e many o t h e r grade s u p p o r t e r s , a d v o c a t e s c a l c u l a t i n g the s t a n d a r d d e v i a t i o n s when a g g r e g a t i n g and w e i g h t i n g s t u d e n t assignment s c o r e s . By u s i n g a sample of s t u d e n t marks, he shows how e q u a t i n g the v a r i a t i o n o f the v a r i o u s assignments can l e a d t o improved g r a d i n g . "Thus the i n f l u e n c e of one component on a composite depends not on t o t a l p o i n t s o r mean s c o r e but on s c o r e v a r i a b i l i t y " ( E b e l , 1972, p. 350). Many o t h e r grade d e f e n d e r s a l s o s u p p o r t t h i s n o t i o n . H i l l s (1981), f o r example, s t a t e s , "The key element i n w e i g h t i n g v a r i a b l e s t h a t a r e t o be combined i s the s t a n d a r d d e v i a t i o n of t h e i r v a l u e s " (p. 320). L i k e E b e l , H i l l s a l s o uses examples t o show how c a l c u l a t i n g the s t a n d a r d d e v i a t i o n i s e s s e n t i a l f o r a c c u r a t e g r a d i n g . With c o n v e r s i o n c h a r t s , he d emonstrates how t h e s t a n d a r d d e v i a t i o n can be e s t i m a t e d w i t h o u t t h e use of a hand c a l c u l a t o r . H i l l s summarizes by s t a t i n g , " U s i n g t h e s e p r o c e d u r e s w i l l h e l p t o e x t i n g u i s h t h e c r i t i c i s m t h a t grades a r e m e a n i n g l e s s " (p. 328). 28 G r o n l u n d (1981) a l s o s u p p o r t s t h e use o f l e t t e r grades i n c l a s s r o o m s . He s t a t e s , "When l e t t e r grades a r e supplemented by o t h e r methods of r e p o r t i n g , t h e s e grades themselves become more m e a n i n g f u l " (p. 520). He t o o u t i l i z e s an example t o demonstrate how g r a d i n g can be improved by making mean s c o r e s and s t a n d a r d d e v i a t i o n s e q u a l when combining and w e i g h t i n g raw s c o r e s . He a l s o e x p r e s s e s t h e i m p o r t a n c e o f c o n s i d e r i n g the d i s p e r s i o n o f s c o r e s . "Thus, t o p r o p e r l y w e i g h t t h e components i n a composite s c o r e , t h e v a r i a b i l i t y o f the s c o r e s must be t a k e n i n t o a c c o u n t " (p. 522). However, i n h i s example, G r o n l u n d shows how t h e v a r i a b i l i t y i n each s e t of assignment s c o r e s can be equated u s i n g t h r e e d i f f e r e n t methods: (a) by u s i n g the range of s c o r e s , (b) by c o n v e r t i n g a l l s e t s o f s c o r e s t o s t a n i n e s , and ( c ) , t h e most r e f i n e d system, by u s i n g s t a n d a r d d e v i a t i o n s . Many o t h e r a u t h o r s , (such as Ahmann & G l o c k , 1981; Townsend & Burke, 1975; G u i l f o r d & F r u c h t e r , 1973) a l s o s u p p o r t the use of l e t t e r grades as a method of g r a d i n g and r e p o r t i n g s t u d e n t achievement. B u t , w h i l e a l l p r o f e s s t h e i m p o r t a n c e o f e q u a l i z i n g the v a r i a b i l i t y i n a s e t of s c o r e s , none c i t e any r e s e a r c h t h a t would i n d i c a t e whether o r not t e a c h e r s c o l l a t e and w e i g h t s c o r e s p r o p e r l y . None i n d i c a t e whether o r not p r a c t i s i n g t e a c h e r s a c t u a l l y c o n s i d e r the v a r i a t i o n i n t h e i r s c o r e s . Perhaps by i m p r o v i n g t h e methods of g r a d i n g as suggested by the d e f e n d i n g a u t h o r s t h e c o n t r o v e r s y s u r r o u n d i n g the u s e f u l n e s s o f the l e t t e r grade as a symbol of s t u d e n t achievement w i l l d i m i n i s h . 29 G r a d i n g Systems Over the l a s t f i f t y y e a r s , s e v e r a l a t t e m p t s have been made t o r e p l a c e the l e t t e r - g r a d e e v a l u a t i o n system w i t h o t h e r m a r k i n g systems o r t o a b o l i s h s c h o o l marks a l t o g e t h e r . Curwin (1978) has o u t l i n e d some of the p o s s i b l e a l t e r n a t i v e s t o e v a l u a t i o n systems i n s c h o o l s . A l t h o u g h t h e s e a l t e r n a t i v e s have not been p a r t i c u l a r l y p o p u l a r i n the p a s t , t h e y a r e mentioned here t o r e v e a l t h e o p t i o n s a v a i l a b l e t o t e a c h e r s and t o i n d i c a t e the v a r i o u s s t r e n g t h s and weaknesses of each e v a l u a t i o n system. The f i r s t a l t e r n a t i v e Curwin o f f e r s i s c a l l e d " S e l f - G r a d i n g " . I n t h i s system, th e t e a c h e r s u p p l i e s the c r i t e r i a and t h e d a t a f o r the s t u d e n t s t o use i n a s s i g n i n g t h e i r own g r a d e s . As a v a r i a n t t o s e l f g r a d i n g , t h e s t u d e n t s can a l s o g e n e r a t e t h e i r own e v a l u a t i o n c r i t e r i a , as l o n g as the c r i t e r i a remain w i t h i n t h e g u i d e l i n e s o f t h e c o u r s e . Curwin s t a t e s t h a t t h e main b e n e f i t o f t h i s approach i s t h a t s t u d e n t s a r e encouraged t o e v a l u a t e t h e i r own work and t h e r e b y l e a r n about t h e m s e l v e s and t h e i r a b i l i t i e s . However, c r i t i c s may q u e s t i o n t h e r e l i a b i l i t y o f the grades r e s u l t i n g from such a system. I t would seem t h a t the r e l i a b i l i t y i s dependent on t h e s u i t a b i l i t y , a p p r o p r i a t e n e s s , and v a l i d i t y of t h e e v a l u a t i o n c r i t e r i a as w e l l as on the r e s p o n s i b i l i t y and m a t u r i t y o f t h e s t u d e n t s . A n other o p t i o n a v a i l a b l e t o t e a c h e r s i s c a l l e d " C o n t r a c t G r a d i n g " . A l t h o u g h s i m i l a r t o s e l f - g r a d i n g , t h i s system i s 30 more s t r u c t u r e d . I t c e n t e r s around a c o n t r a c t t h a t i s a g r e e a b l e t o both s t u d e n t and t e a c h e r . The c o n t r a c t s t i p u l a t e s t h e work t o be a c c o m p l i s h e d and t h e time frame w i t h i n which the work must be completed. The f o u r elements of the c o n t r a c t a r e : (1) c r i t e r i a f o r s u c c e s s f u l c o m p l e t i o n o f the c o n t r a c t , (2) d a t a o r r e s u l t s o f work t h a t i n d i c a t e how w e l l the c r i t e r i a have been met, (3) an a n a l y s i s o f o r comparison between the d a t a and c r i t e r i a , and (4) grade (or p o i n t s ) e a r n e d . A d i s a d v a n t a g e o f c o n t r a c t g r a d i n g i s t h a t "the c o n t r a c t s sometimes become i n f l e x i b l e . S t u d e n t s might s e t u n r e a l i s t i c g o a l s - - t o o h i g h o r t o o low" ( C u r w i n , 1978, p. 62). A n o t h e r d i f f i c u l t y i s t h a t s t u d e n t s i n such a system t e n d t o do o n l y what i s s t i p u l a t e d i n the c o n t r a c t and t o a v o i d the spontaneous e x t e n s i o n o f t h e i r l e a r n i n g beyond what i s w r i t t e n . Curwin d e s c r i b e s "Peer G r a d i n g " as a n o t h e r a l t e r n a t i v e t o the l e t t e r grade system of e v a l u a t i o n . T h i s method u t i l i z e s a c o n t r a c t f o r m u l a t e d by a group of s t u d e n t s . Upon c o m p l e t i o n of t h e c o n t r a c t , the group e v a l u a t e s i t s e l f and each member r e c e i v e s t h e same group grade. The groups can be formed on the b a s i s of a s u b j e c t a r e a , a homogeneous a b i l i t y g r o u p i n g , or around a mutual i n t e r e s t . Curwin c l a i m s , S t u d e n t s o f t e n a r e i n s p i r e d t o l e a r n as energy i s g e n e r a t e d t h r o u g h w o r k i n g w i t h f r i e n d s and p e e r s on a common p r o j e c t , but a n e g a t i v e a t t i t u d e by some s t u d e n t s can d r a g the r e s t o f the group down. I f some s t u d e n t s work h a r d e r than o t h e r s , t h e y may p e r c e i v e i t as u n f a i r 3 1 t o r e c e i v e the same grade as tho s e who worked l e s s h a r d , (p. 62) A f o u r t h a l t e r n a t i v e t o g r a d i n g i s r e f e r r e d t o as " B l a n k e t G r a d i n g " . With t h i s system, "each s t u d e n t i n a c l a s s r e c e i v e s t h e same grade ( u s u a l l y an A or B) p r o v i d i n g t h a t some minimum s t a n d a r d i s met. Those s t u d e n t s who do not r e a c h the s p e c i f i e d s t a n d a r d must n e g o t i a t e a grade i n d i v i d u a l l y w i t h t h e t e a c h e r " (p. 6 4 ) . U n l i k e Curwin's o t h e r a l t e r n a t i v e s , B l a n k e t G r a d i n g r e q u i r e s a c o n t i n u o u s e v a l u a t i o n p r o c e s s as p a r t of the r e g u l a r c l a s s a c t i v i t i e s . T h i s can become a n e g a t i v e f e a t u r e of t h i s a l t e r n a t i v e s i n c e a v e r y s y s t e m a t i c and w e l l o r g a n i z e d e v a l u a t i o n system i s r e q u i r e d . S i m p l y d e c l a r i n g t h a t everyone w i l l r e c e i v e an A o r B w i t h o u t any o t h e r form o f e v a l u a t i o n would prove t o be c o u n t e r - p r o d u c t i v e ; s t u d e n t s would be l e s s m o t i v a t e d t o a c h i e v e t o t h e i r p o t e n t i a l . Another n e g a t i v e a s p e c t o f B l a n k e t G r a d i n g i s t h a t a d m i n i s t r a t o r s o f t e n have a d i f f i c u l t t i me a c c e p t i n g t h i s system o f g r a d i n g , e s p e c i a l l y when a d i s p r o p o r t i o n a t e number of A's o r B's a r e g i v e n . T h i s , a c c o r d i n g t o Curwin, can cause t h e t e a c h e r " p e r s o n a l o r p r o f e s s i o n a l harm" (p. 64). Curwin a l s o emphasizes t h a t t h i s approach t o g r a d i n g needs t o be v e r y t h o r o u g h l y e x p l a i n e d t o the a d m i n i s t r a t i o n and t o the s t u d e n t s b e f o r e b e i n g used. In r e f e r e n c e t o t h e s e f o u r a l t e r n a t e g r a d i n g schemes, Curwin s t a t e s , " I f you use any o f the a l t e r n a t i v e s p r e s e n t e d h e r e , you w i l l a v o i d many ( a l t h o u g h not a l l ) o f the problems i n t r i n s i c t o g r a d i n g , but you s t i l l must meet some of the o r i g i n a l purposes of grades -- e s p e c i a l l y the f u n c t i o n o f s u p p l y i n g . i n f o r m a t i o n t o p a r e n t s " (p. 6 4 ) . Of the more p o p u l a r g r a d i n g systems, most a r e v a r i a t i o n s on one o f t h r e e approaches: (a) p e r c e n t g r a d i n g , (b) c r i t e r i o n - r e f e r e n c e d g r a d i n g , and (c) n o r m - r e f e r e n c e d g r a d i n g . Each has been p o p u l a r i n the p a s t as a method of d e t e r m i n i n g grades i n the e d u c a t i o n a l s e t t i n g . P e r c e n t G r a d i n g , which i s c u r r e n t l y the l e a s t p o p u l a r o f the t h r e e b a s i c approaches, i n v o l v e s s c o r e s t h a t a r e averaged and c o n v e r t e d t o a p e r c e n t . W i t h t h i s method, the p e r c e n t i t s e l f may r e p r e s e n t t h e grade. For example, mathematics might be r e p o r t e d on a r e p o r t c a r d as s i m p l y "Mathematics 73%". A n o t h e r method of r e p o r t i n g p e r c e n t g r a d i n g i s t o c o n v e r t a p e r c e n t a g e t o a l e t t e r grade. An A, f o r example, may r e p r e s e n t s c o r e s r a n g i n g from 80% t o 100%. P e r c e n t G r a d i n g has not g a i n e d p o p u l a r i t y m a i n l y because of t h e o b s c u r i t y o f the g r a d e s . A s c o r e o f 80%, f o r example, t e l l s l i t t l e more than 80 p e r c e n t o f t h e q u e s t i o n s were answered c o r r e c t l y . Such a grade does not r e v e a l how much a s t u d e n t l e a r n e d i n r e l a t i o n t o i n t e n d e d outcomes nor does i t r e v e a l how he compared t o the achievement of o t h e r s t u d e n t s . With such grades i t i s d i f f i c u l t t o determine i f the 80 p e r c e n t r e p r e s e n t s e x c e p t i o n a l , a v e r a g e , o r poor achievement. I f i t i s t h e h i g h e s t s c o r e , t h e n i t i s a v e r y good s c o r e . I f i t i s near t h e mean o r median s c o r e , t h e n i t i s o n l y average, 33 and i f i t i s one o f the l o w e s t s c o r e s , t h e n i t r e p r e s e n t s poor achievement. I t i s i m p o s s i b l e t o t e l l what l e v e l o f achievement 80 p e r c e n t a c t u a l l y r e p r e s e n t s . A l t h o u g h p e r c e n t g r a d i n g was once a p o p u l a r method o f g r a d i n g , i t has l o s t i t s a p p e a l i n f a v o r of the o t h e r two major systems, namely t h e c r i t e r i o n - r e f e r e n c e d and no r m - r e f e r e n c e d approaches t o g r a d i n g . C r i t e r i o n - r e f e r e n c e d g r a d i n g C r i t e r i o n - r e f e r e n c e d g r a d i n g i n v o l v e s e x p r e s s i n g a s t u d e n t ' s achievement i n r e l a t i o n t o some p r e s p e c i f i e d s t a n d a r d s o r c r i t e r i a . These s t a n d a r d s a re u s u a l l y concerned w i t h t h e l e v e l o f mastery t o be a c h i e v e d by the s t u d e n t s and are o f t e n s t a t e d i n terms o f : (a) the s p e c i f i c t a s k s t o be performed, and (b) t h e p e r c e n t a g e o f c o r r e c t answers t o be o b t a i n e d on a t e s t t h a t measures a c l e a r l y d e f i n e d s e t o f l e a r n i n g o b j e c t i v e s . As a r e s u l t , t h e s e s t a n d a r d s a r e a b s o l u t e and a r e not r e l a t i v e t o o t h e r s t u d e n t s o r t o p r e v i o u s i n d i v i d u a l achievement. A l l s t u d e n t s who a c h i e v e a t the same l e v e l o r master the same o b j e c t i v e s a c h i e v e the same grade. "No s t u d e n t i s f a i l e d s i m p l y because he o r she a c h i e v e d lower t h a n o t h e r s t u d e n t s p r o v i d e d the s t u d e n t a c h i e v e s the c r i t e r i o n " (Gray, 1980, p. 490). G r a d i n g i n the c r i t e r i o n - r e f e r e n c e d system i s s i m p l e r t h a n many o t h e r g r a d i n g methods. Those who r e a c h the s p e c i f i e d c r i t e r i o n on each l e a r n i n g o b j e c t i v e r e c e i v e c r e d i t , and t h o s e who do n o t , f a i l . 34 F a i l u r e t o s a t i s f y the c r i t e r i o n may a l s o n e c e s s i t a t e t h a t the s t u d e n t r e p e a t the r e q u i r e d t a s k s u n t i l the o b j e c t i v e s have been a c h i e v e d . C o n s e q u e n t l y , n e a r l y a l l s t u d e n t s i n the c r i t e r i o n - r e f e r e n c e d system pas s . The c r i t e r i o n - r e f e r e n c e d g r a d i n g t e c h n i q u e w i t h t h e mastery l e a r n i n g approach, makes i t r e l a t i v e l y easy f o r t e a c h e r s t o i d e n t i f y problem a r e a s i n t h e i r c u r r i c u l u m and t o i d e n t i f y weaknesses w i t h i n i n d i v i d u a l s t u d e n t s . C r i t e r i o n - r e f e r e n c e d t e s t s , f o r example, w i l l r e v e a l which s t u d e n t s had d i f f i c u l t y ( o r f a i l e d t o meet c r i t e r i o n ) w i t h which o b j e c t i v e s . Those o b j e c t i v e s t h a t were p o o r l y g r a s p e d by a m a j o r i t y of s t u d e n t s and t h e r e f o r e need t o be r e t a u g h t w i l l a l s o be r e v e a l e d . "Thus, d e f i c i e n c i e s a r e q u i c k l y c o r r e c t e d and s t u d e n t s do not get f a r t h e r and f a r t h e r b e h i n d " (Gray, p. 490). S t u d e n t s i n t h i s system a r e p e r m i t t e d t h e o p p o r t u n i t y t o p r o g r e s s a t t h e i r own l e v e l s - some w i l l meet the c r i t e r i o n sooner than o t h e r s . However, by t h e end o f the c o u r s e o r term, most s t u d e n t s w i l l have demonstrated an a c c e p t a b l e l e v e l of performance on t h e r e q u i r e d l e a r n i n g o b j e c t i v e s . G r o n lund (1981) s t a t e s , "To e f f e c t i v e l y use a b s o l u t e l e v e l o f achievement as a b a s i s f o r g r a d i n g r e q u i r e s t h a t (1) the domain of l e a r n i n g t a s k s be c l e a r l y d e f i n e d , (2) the s t a n d a r d s of performance be c l e a r l y s p e c i f i e d and j u s t i f i e d , and (3) t h e measures of p u p i l achievement be c r i t e r i o n r e f e r e n c e d " (p. 524). W h i l e t h e s e c o n d i t i o n s a r e r e l a t i v e l y easy t o s a t i s f y i n t h e m astery approach, t h e y become v e r y d i f f i c u l t t o a p p l y t o nonmastery l e a r n i n g o b j e c t i v e s . T h i s prompts Gronlund t o s t a t e , "The c r i t e r i o n - r e f e r e n c e d system g r a d i n g i s much more complex t h a n i t f i r s t a ppears" (p. 524) Because most e d u c a t i o n a l programs i n the e l e m e n t a r y s c h o o l s e t t i n g w i l l i n c l u d e b o t h mastery and nonmastery o b j e c t i v e s , the e l e m e n t a r y t e a c h e r w i l l have c o n s i d e r a b l e d i f f i c u l t y s e t t i n g the c r i t e r i a f o r a c c e p t a b l e performance w i t h a l l o b j e c t i v e s . Even when t h e c r i t e r i a a r e e s t a b l i s h e d , Gray (1980) c l a i m s t h a t " i t i s d i f f i c u l t t o defend such s t a n d a r d s t o p a r e n t s , even i f t h e y have been s e t by c u r r i c u l u m e x p e r t s i n a c u r r i c u l u m g u i d e , because t h e y a r e based p r i m a r i l y on judgment" (p. 491). Gronlund (1981) c l a i m s t h a t t h e s e judgments a r e " l i k e l y t o be co n t a m i n a t e d by achievements t o some unknown degree. Thus, the l a c k o f r e l i a b l i l i t y i n j u d g i n g achievement i n r e l a t i o n t o p o t e n t i a l , and i n j u d g i n g degree.of improvement, would r e s u l t i n grades o f low d e p e n d a b i l i t y " (p. 524). T h i s i s v e r y s i g n i f i c a n t s i n c e demonstrated improvement and l e a r n i n g p o t e n t i a l have been w i d e l y used as a b a s i s f o r g r a d i n g t h e nonmastery components of t h e c r i t e r i o n - r e f e r e n c e d g r a d i n g system i n e l e m e n t a r y s c h o o l s . I t i s a l s o i n t e r e s t i n g t o note t h a t when r e p o r t i n g s t u d e n t achievement on nonmastery o b j e c t i v e s , rank o r d e r bas on s c o r e s o r on number o f o b j e c t i v e s completed i s o f t e n used t o p r o v i d e some measure o f r e l a t i v e p o s i t i o n w i t h i n the c l a s Rank and r e l a t i v e p o s i t i o n a r e terms t h a t connote t h e norm - r e f e r e n c e d approach t o g r a d i n g s t u d e n t achievement. 36 Norm-referenced grading T r a d i t i o n a l l y norm-referenced gr a d i n g has been the most common approach to gr a d i n g i n ed u c a t i o n . I t i n v o l v e s rank o r d e r i n g students and e x p r e s s i n g each student's achievement i n r e l a t i o n t o the achievement of h i s class-mates. U n l i k e c r i t e r i o n - r e f e r e n c e d g r a d i n g , a norm-referenced grade does not i n d i c a t e what a student has a c t u a l l y achieved; i t on l y d e s c r i b e s the r e l a t i v e p o s i t i o n w i t h i n the c l a s s . In the extreme form of norm-referenced g r a d i n g , the c l a s s serves as the normative group. Assignments and norm-referenced t e s t s are designed to produce a wide range of scores and i t i s t h i s v a r i a b l i l i t y t h a t i s used t o determine the d i s t r i b u t i o n of f i n a l grades. For example, i f a student ranks high i n the group, he w i l l r e c e i v e a high grade. I f h i s r e l a t i v e achievement i s low, he w i l l r e c e i v e a low grade. Since the gr a d i n g i s based on r e l a t i v e achievement, the s t u d e n t s ' grades are i n f l u e n c e d by both h i s performance and by the performance of the group. Consequently, a p a r t i c u l a r student w i l l - do b e t t e r i n a low a c h i e v i n g c l a s s than i n a h i g h a c h i e v i n g c l a s s . Because the norm-referenced approach assumes a normal d i s t r i b u t i o n i n the p o p u l a t i o n of students, most w i l l r e c e i v e a grade i n d i c a t i n g average achievement. A few students w i l l r e c e i v e A's and a few w i l l f a i l . However, t h i s i s more c h a r a c t e r i s t i c of normal curve gradi n g , which i s one of s e v e r a l v a r i a t i o n s of the norm-referenced technique a v a i l a b l e t o t e a c h e r s . Normal curve grading i s the extreme form of 37 n o r m - r e f e r e n c e d g r a d i n g . I n t h i s system, a f i x e d p e r c e n t a g e of s t u d e n t s r e c e i v e each grade. A commonly used grade d i s t r i b u t i o n i n t h i s system a s s i g n s the t o p 5% an A, the next 24% a B, the m i d d l e 40% a C, t h e nex t 24% a D, and t h e l o w e s t 7% a f a i l i n g grade (Gray, 1980, p. 488). A weakness w i t h t h i s a pproach i s t h a t some s t u d e n t s must r e c e i v e f a i l i n g grades r e g a r d l e s s of t h e i r l e v e l s o f achievement. F o r example, a s t u d e n t who a c h i e v e s a s c o r e of 70% may r e c e i v e a f a i l i n g grade i f h i s s c o r e i s one o f t h e l o w e s t i n t h e group. On the o t h e r hand, i f 70% r e p r e s e n t e d a h i g h s c o r e , then the s t u d e n t would r e c e i v e a h i g h grade. Another weakness i s t h a t the same grade does not n e c e s s a r i l y r e p r e s e n t t h e same l e v e l of achievement. I n one c l a s s , a s t u d e n t ' s achievement may y i e l d a B; i n a n o t h e r c l a s s , t h e same achievement might be e q u i v a l e n t t o a C o r a D. Many s t o r i e s have been t o l d how non-academics, d e r e l i c t s , and i n one American a i r f o r c e s t o r y , the s t u d e n t s ' wives were persuaded t o e n r o l l i n c o u r s e s where normal c u r v e g r a d i n g was used. The t h e o r y was t h a t t h e s e " c l a s s f i l l e r s " would absorb t h e f a i l i n g grades and t h e r e b y a l l o w the r e s t o f the s t u d e n t s t o pass t h e c o u r s e . These s t o r i e s h i g h l i g h t t h e weaknesses of normal c u r v e g r a d i n g . A nother v a r i a t i o n t o no r m - r e f e r e n c e d g r a d i n g i s " P a s s - F a i l g r a d i n g " (Gray, 1980, p. 488). T h i s system has o n l y two p o s s i b l e g r a d e s ; e i t h e r Pass o r F a i l . Any s t u d e n t who a c h i e v e s a t a l e v e l t h a t i s a t or above the minimum a c c e p t a b l e s t a n d a r d , r e c e i v e s the "Pass" grade. The r a t i o n a l e b e h i n d t h i s t y p e o f g r a d i n g i s t h a t such a system t a k e s t h e 38 p r e s s u r e o f f students and encourages them t o take courses they might normally a v o i d due t o f e a r of r e c e i v i n g a lower grade. However, s t u d i e s have shown t h a t students tend to be l e s s motivated to do t h e i r best i n courses u s i n g the p a s s - f a i l approach. "Thus, p a s s - f a i l g r a d i n g does not f u l f i l l any of the purposes of grades—communication, m o t i v a t i o n , and p r e d i c t i o n . . . . In g e n e r a l i t s use should be discouraged" (Gray, 1980, p. 489). A more s a t i s f a c t o r y v a r i a t i o n t o norm-referenced grading i s "standard score g r a d i n g " . T h i s technique i n v o l v e s c o n v e r t i n g raw scores from each t e s t or assignment t o s t a n d a r d i z e d z s c o r e s , or T s c o r e s , averaging the T scores and a s s i g n i n g grades based on the average T score v a l u e . For example, average T scores of 65 and above might r e c e i v e an A and scores between 5 5 and 6 4 might r e c e i v e a B and so on. The main advantage of t h i s g r a d i n g system i s t h a t a r i t h m e t i c o p e r a t i o n s can be performed on the .T s c o r e s . T h i s i s an important f e a t u r e because "we can m e a n i n g f u l l y combine scores i n a way t h a t a d j u s t s a p p r o p r i a t e l y f o r the f a c t t h a t the v a r i o u s c o n t r i b u t i n g t e s t s had d i f f e r e n t degrees of v a r i a b i l i t y " (Gray, 1980, p. 489). Another advantage to standard score g r a d i n g i s t h a t teachers can weight assignments d i f f e r e n t i a l l y . Seldom are a l l t e s t s and assignments co n s i d e r e d to be of equal importance when combining marks. Standard score g r a d i n g allows teachers to a s s i g n a d e s i r e d weight to each of the c o n t r i b u t i n g assignments before combining the s c o r e s . However, "Good judgment has to enter 39 i n t o t h e g r a d i n g p r o c e s s . . . . The p o i n t t o keep i n mind i s t h a t n o r m - r e f e r e n c e d g r a d i n g s i m p l y rank o r d e r s the s t u d e n t s ; i t i n d i c a t e s n o t h i n g about a c t u a l l e v e l o f achievement" (Gray, p. 490). One of the major c r i t i c i s m s o f the extreme form o f n o r m - r e f e r e n c e d g r a d i n g i s t h e f a c t t h a t a f i x e d p e r c e n t a g e o f s t u d e n t s r e c e i v e each grade. As s t a t e d p r e v i o u s l y , t h i s i m p l i e s t h a t a s m a l l p e r c e n t a g e of s t u d e n t s w i l l always r e c e i v e f a i l i n g g r a d e s , r e g a r d l e s s o f the l e v e l s o f achievement. However, t o overcome the n e g a t i v e a s p e c t s , t h e B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n (1979) o f f e r s a n o t h e r v a r i a t i o n t o n o r m - r e f e r e n c e d g r a d i n g . I t s u g g e s t s t h e use o f s t a n d a r d s c o r e g r a d i n g but w i t h a grade d i s t r i b u t i o n based on s t u d e n t a b i l i t y r a t h e r t h a n on a normal c u r v e . The methods of e s t a b l i s h i n g t h e grade d i s t r i b u t i o n makes t h i s approach d i f f e r e n t from t h e o t h e r methods of n o r m - r e f e r e n c e d g r a d i n g a l r e a d y mentioned. E s s e n t i a l l y , an a b i l i t y - b a s e d g r a d i n g d i s t r i b u t i o n means t h a t the grade d i s t r i b u t i o n f o r a g i v e n c l a s s r e f l e c t s t h e a b i l i t y l e v e l s o f the s t u d e n t s i n t h a t p a r t i c u l a r c l a s s . "The p r o p o r t i o n o f d i f f e r e n t grades a s s i g n e d i s p r e d e t e r m i n e d by the a b i l i t y l e v e l s w i t h i n the group" ( B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n , p. 2 6 ) . T h e r e f o r e , i t i s p o s s i b l e t h a t , f o r a p a r t i c u l a r c l a s s , a l a r g e r number of grades may be g i v e n a t one l e v e l than i n any o t h e r . Some d i s t r i b u t i o n s may have l e v e l s where no grades a r e awarded. To c r e a t e such an a b i l i t y - b a s e d d i s t r i b u t i o n r e q u i r e s t h e e s t i m a t i o n of t h e 4 0 g e n e r a l a b i l i t y l e v e l of s t u d e n t s w i t h i n t h e c l a s s . N o r m a l l y , the n e c e s s a r y i n f o r m a t i o n can be o b t a i n e d from s t a n d a r d i z e d achievement t e s t s c o r e s o r from g e n e r a l a b i l i t y t e s t s , o r b o t h . I n f o r m a t i o n on s t u d e n t a b i l i t y can a l s o be o b t a i n e d from p a s t performance r e c o r d s , such as p r e v i o u s r e p o r t c a r d s o r s c h o o l r e c o r d s . U s i n g t h i s i n f o r m a t i o n , a g r a d i n g d i s t r i b u t i o n can be d e t e r m i n e d f o r each c l a s s . I t i s i m p o r t a n t t o p o i n t out t h a t the grades or p e r c e n t i l e r a n k i n g s a s s i g n e d t o i n d i v i d u a l s t u d e n t s d u r i n g t h e e s t i m a t i o n p r o c e s s are not t o be c o n s i d e r e d as e x p e c t a t i o n l e v e l s f o r t h e s t u d e n t s so c a t e g o r i z e d . For example, s t u d e n t s who r e c e i v e A's may not be t h e same s t u d e n t s who s c o r e d above the 96 p e r c e n t i l e on the s t a n d a r d i z e d t e s t . W h i l e t h e use of T s c o r e s m a i n t a i n s t h e advantages of s t a n d a r d s c o r e g r a d i n g , a grade d i s t r i b u t i o n based on s t u d e n t a b i l i t y overcomes the problems of p r e d e t e r m i n e d f a i l u r e s . These methods, combined w i t h t h e use of " n a t u r a l b r e a k s " i n the d i s t r i b u t i o n of c o l l a t e d s c o r e s t o determine l e t t e r grade c u t - o f f p o i n t s , " o f f e r s a v e r y s a t i s f a c t o r y approach t o t h e assignment of l e t t e r g r a d e s " ( B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n , p. 2 7 ) . T h i s v a r i a n t t o the n o r m - r e f e r e n c e d g r a d i n g p r o c e d u r e has been adopted by the s c h o o l d i s t r i c t i n v o l v e d i n t h i s s t u d y . There a r e s e v e r a l r e a s o n s why t h e n o r m - r e f e r e n c e d approach t o g r a d i n g has remained one o f the most p o p u l a r methods of e v a l u a t i n g and g r a d i n g s t u d e n t achievement. F i r s t l y , grades based on r e l a t i v e achievement appear t o be 41 more r e a d i l y u n d e r s t o o d by p a r e n t s , a d m i n i s t r a t o r s , p r o s p e c t i v e e mployers, and o t h e r s who have an i n t e r e s t i n s t u d e n t achievement. R e l a t i v e p o s i t i o n i n a group o r c l a s s , as i n d i c a t e d by the l e t t e r grade, can be comprehended q u i c k l y and e a s i l y . Very l i t t l e i n f o r m a t i o n needs t o accompany the l e t t e r grade t o e x p l a i n i t s meaning; due t o i t s f r e q u e n t use and f a m i l i a r i t y i t i s al m o s t common knowledge t h a t an A r e p r e s e n t s o u t s t a n d i n g work w h i l e a D o r E i n d i c a t e s t h a t the s t u d e n t has e x p e r i e n c e d c o n s i d e r a b l e d i f f i c u l t y . Other g r a d i n g systems r e q u i r e c h e c k l i s t s o r g r a d i n g c r i t e r i a t o p r o v i d e meaning t o the l e t t e r g r a d e s . These l i s t s need t o be i n t e r p r e t e d and u n d e r s t o o d b e f o r e f u l l v a l u e can be d e r i v e d from t h e a s s o c i a t e d l e t t e r grade. S e c o n d l y , p a r e n t s g e n e r a l l y a r e i n t e r e s t e d i n knowing how w e l l t h e i r c h i l d performed r e l a t i v e t o the o t h e r s i n the c l a s s . The c o m p e t i t i v e n a t u r e of p e o p l e , the same s o r t t h a t d r i v e s them t o "keep up w i t h t h e Jo n e s " , emerges when d i s c u s s i n g t h e i r c h i l d ' s academic achievement. F or example, th e y might want t o know i f t h e i r c h i l d ' s A was a h i g h A o r a low A o r how many o t h e r s i n the c l a s s r e c e i v e d an A. Pe o p l e o f t e n appear t o use r e l a t i v e p o s i t i o n as an i n d i c a t o r o f s u c c e s s , whether i t be i n b u s i n e s s , i n d u s t r y , s o c i e t y , o r e d u c a t i o n . The f u r t h e r up t h e r a n k i n g s c a l e a p e r s o n i s t h e more s u c c e s s f u l he i s p e r c e i v e d t o be. A t h i r d r e a s o n why n o r m - r e f e r e n c e d g r a d i n g methods have been more common i s due t o the p r e d i c t i v e q u a l i t i e s o f t h i s g r a d i n g system. Many of t h e uses o f grades a r e p r e d i c t i v e i n 42 n a t u r e . F o r example, employers want t o be a b l e t o p r e d i c t t h e a b i l i t i e s and s u c c e s s of t h e s t u d e n t s t h e y h i r e . C o l l e g e s and u n i v e r s i t i e s use grades t o p r e d i c t the s u c c e s s of t h e i r a p p l i c a n t s . " I f a l l s t u d e n t s r e c e i v e d n o t h i n g but p a s s - f a i l g r a d e s , i t would be v e r y d i f f i c u l t f o r c o l l e g e a d m i s s i o n s p e r s o n n e l , f o r example, t o s e l e c t from among a p p l i c a n t s s i n c e a l l a p p l i c a n t s would have the same g r a d e s " (Gray, 1980, p. 489). Because the n o r m - r e f e r e n c e d system attemps t o produce maximum v a r i a b i l i t y of s c o r e s , t h e c o r r e l a t i o n c o e f f i c i e n t i s enhanced and becomes more u s e f u l f o r p r e d i c t i o n purposes t h a n c o e f f i c i e n t s d e r i v e d from o t h e r g r a d i n g systems. A f o u r t h r e a s o n why n o r m - r e f e r e n c e d g r a d i n g has m a i n t a i n e d i t s p o p u l a r i t y i s due t o the r e l a t i v e ease o f s e t t i n g s t a n d a r d s and grade d i s t r i b u t i o n s . F o r e d u c a t i o n a l i n s t i t u t i o n s u s i n g t h e extreme form o f n o r m - r e f e r e n c e d g r a d i n g , t h e normal c u r v e i s used t o e s t a b l i s h t h e grade d i s t r i b u t i o n . I n t h i s c a s e , t h e s t a n d a r d s and grade d i s t r i b u t i o n s between v a r i o u s c l a s s e s o r c o u r s e s remain s i m i l a r r e g a r d l e s s o f c o u r s e c o n t e n t o r s t u d e n t a b i l i t y l e v e l s . C o n s e q u e n t l y , t h e n o r m - r e f e r e n c e d g r a d i n g system i s e a s i e r t o a d m i n i s t e r and t o a p p l y . Once t h e f i n a l s c o r e r a n k i n g has been a c h i e v e d , t h e number of grades t o be a l l o t t e d a t each l e v e l can be e a s i l y d e t e r m i n e d . Even t h e v a r i a t i o n s of n o r m - r e f e r e n c e d g r a d i n g , t h o s e t h a t do not s t r i c t l y adhere t o the normal c u r v e d i s t r i b u t i o n , a r e more e a s i l y a d m i n i s t e r e d t h a n most of t h e o t h e r g r a d i n g t e c h n i q u e s . One of t h e main d i s a d v a n t a g e s of t h e c r i t e r i o n - r e f e r e n c e d systems, f o r 43 example, i s the d i f f i c u l t y o f f o r m u l a t i n g t h e c r i t e r i o n and s t a n d a r d s f o r a c c e p t a b l e performance. With a n o r m - r e f e r e n c e d approach, t h e s e t a s k s a r e much s i m p l e r ; the a d m i n i s t r a t i o n o f grades i s more m e c h a n i c a l and m a t h e m a t i c a l , r e q u i r i n g l e s s t i me and judgment from t h e t e a c h e r . A n o t h e r r e a s o n f o r t h e p o p u l a r i t y o f n o r m - r e f e r e n c e d approaches i s due t o t h e m o t i v a t i o n a l a s p e c t s . One of the main purposes of grades i s t o m o t i v a t e s t u d e n t s t o p e r f o r m as c l o s e t o t h e i r p o t e n t i a l as p o s s i b l e . Norm-referenced g r a d i n g has been shown t o m o t i v a t e s t u d e n t s more e f f e c t i v e l y t han do most o t h e r methods o f g r a d i n g . F o r example, c r i t e r i o n - r e f e r e n c e d approaches t o g r a d i n g c r e a t e l e s s m o t i v a t i o n i n s t u d e n t s . Gray (1980) s t a t e s , " i f a l l c o u r s e s were p a s s - f a i l , s t u d e n t m o t i v a t i o n (and subsequent achievement) would d e c r e a s e a c r o s s t h e board" (p. 492). Norm-referenced systems can c a p i t a l i z e on the c o m p e t i t i v e n a t u r e o f s t u d e n t s and m o t i v a t e them t o do t h e i r b e s t . I n B r i t i s h C olumbia, the M i n i s t r y of E d u c a t i o n s t i p u l a t e d t h a t a l l p u b l i c s c h o o l s would r e p o r t c o m p a r a t i v e achievement w i t h a seven l e v e l l e t t e r grade system i n grades f o u r t o t w e l v e . T h i s d e c i s i o n was based on the r e s u l t s of a survey conducted by t h e M i n i s t r y i n the l a t e 1970s. T h i s method of r e p o r t i n g n e c e s s i t a t e d t h e use o f some form of n o r m - r e f e r e n c e d g r a d i n g p r o c e d u r e w i t h i n t h e s c h o o l system. B o o k l e t s were p u b l i s h e d by t h e M i n i s t r y t o p r o v i d e g u i d e l i n e s and t o a s s i s t t e a c h e r s i n u t i l i z i n g n o r m - r e f e r e n c e d t e c h n i q u e s i n t h e i r c l a s s r o o m s . The r e p o r t i n g and g r a d i n g methods used i n the e l e m e n t a r y s c h o o l s o f the s c h o o l d i s t r i c t i n v o l v e d i n t h i s s t u d y f o l l o w t h e s e g u i d e l i n e s and a r e o u t l i n e d i n c h a p t e r 3 t h i s s t u d y . 45 CHAPTER 3 Methodology I t was the purpose of t h i s s t u d y t o e v a l u a t e the g r a d i n g methods used by 3 7 randomly s e l e c t e d t e a c h e r s . I n f o r m a t i o n on t h e i r methods of g r a d i n g was c o l l e c t e d i n t h r e e ways: (a) by way o f a q u e s t i o n n a i r e , (b) by h a v i n g the s u b j e c t s w e i g h t , t o t a l , and rank a h y p o t h e t i c a l s e t o f raw achievement s c o r e s , and (c) by h a v i n g the s u b j e c t s submit t h e i r mathematics and s o c i a l s t u d i e s c l a s s r e c o r d s h e e t s f o r one r e p o r t i n g p e r i o d . Sample and P o p u l a t i o n The sample was s e l e c t e d a t random from a l i s t o f male and female t e a c h e r s who t e a c h a t t h e grades f o u r t o seven l e v e l i n a l a r g e m e t r o p o l i t a n a r e a o f c e n t r a l B r i t i s h Columbia. The a c c e s s i b l e p o p u l a t i o n from which the sample was s e l e c t e d c o n s i s t e d o f t e a c h e r s who t e a c h a t the i n t e r m e d i a t e l e v e l i n e l e m e n t a r y s c h o o l s . At t h i s l e v e l , t e a c h e r s c u s t o m a r i l y t e a c h a l l of t h e M i n i s t r y p r e s c r i b e d s u b j e c t s t o one c l a s s . The s i z e s of most e l e m e n t a r y s c h o o l c l a s s e s n o r m a l l y range from twenty f i v e t o t h i r t y s t u d e n t s . Teachers are r e s p o n s i b l e f o r a d m i n i s t e r i n g the p r e s c r i b e d c u r r i c u l a , and f o r m e a s u r i n g , e v a l u a t i n g , g r a d i n g , and r e p o r t i n g s t u d e n t achievement. I n t h e e l e m e n t a r y s c h o o l s e t t i n g , measurement i n s t r u m e n t s c o n s i s t p r i m a r i l y of t e a c h e r made t e s t s and a s s i g n m e n t s . 46 Q u e s t i o n n a i r e A q u e s t i o n n a i r e was g i v e n t o each member of the sample of t e a c h e r s i n o r d e r t o g a i n some i n s i g h t i n t o t h e i r t e a c h i n g e x p e r i e n c e , t h e i r knowledge of c o l l a t i n g achievement s c o r e s f o r g r a d i n g p u r p o s e s , and t h e i r knowledge o f b a s i c s t a t i s t i c s ( p a r t i c u l a r l y s t a n d a r d d e v i a t i o n ) . A L i k e r t - t y p e s c a l e was used f o r some q u e s t i o n n a i r e items and the c l o s e d form format was used t o o b t a i n t h e demographic and g r a d i n g p r o c e d u r e i n f o r m a t i o n (Appendix A ) . Such q u e s t i o n s as t h e f o l l o w i n g were asked: 1. My c o l l e a g u e s a t t h i s s c h o o l t a k e r e p o r t c a r d marks s e r i o u s l y . 2. L e t t e r grades a r e an e f f e c t i v e method o f i n f o r m i n g p a r e n t s of t h e i r c h i l d ' s p r o g r e s s and achievement. 3. To the b e s t o f my knowledge, p a r e n t s a r e g e n e r a l l y s a t i s f i e d w i t h t h e l e t t e r grade system of r e p o r t i n g . 4. I f e e l v e r y c o m f o r t a b l e w i t h the l e t t e r grades I award s t u d e n t s . 5. I am c o n f i d e n t t h a t t h e l e t t e r grades I a s s i g n a r e a c c u r a t e and r e l i a b l e . 6. I would l i k e t o l e a r n more about c o l l a t i n g and a s s i g n i n g l e t t e r grades t o raw s c o r e s . 7. More i n - s e r v i c e s e s s i o n s on g r a d i n g and r e p o r t i n g s h o u l d be o f f e r e d . 8. U n i v e r s i t i e s s h o u l d o f f e r more p r e - s e r v i c e i n s t r u c t i o n i n e f f e c t i v e g r a d i n g and r e p o r t i n g . 9. The r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o be communicated t o p a r e n t s , s t u d e n t s , e t c . 10. I n . t h e space p r o v i d e d , b r i e f l y d e s c r i b e how you combine assignment and t e s t s c o r e s t o determine l e t t e r g r a d e s . 11. I n r e f e r e n c e t o the method d e s c r i b e d i n #10, which o t h e r s u b j e c t s a r e a l s o graded t h i s way? ( a l l s u b j e c t s , mathematics, lanauaae a r t s , s c i e n c e , s o c i a l s t u d i e s , a r t , P.E., m u s i c , French) 12. I n the space p r o v i d e d , b r i e f l y d e s c r i b e how you a r r i v e a t a f i n a l term s c o r e f o r a s t u d e n t who has been absent f o r one o r more t e s t s o r a s s i g n m e n t s . In o t h e r words, how do you compensate f o r a l e g i t i m a t e l y m i s s e d assignment? The q u e s t i o n n a i r e a l s o c o l l e c t e d d a t a u s i n g t h e c o m p l e t i o n f o r m a t . Such q u e s t i o n s as the f o l l o w i n g were asked: 1. Which i s your age c a t e g o r y ? ( <25,25-30 , 31-40 ,41-50 , 51-60 ,>60) ( y e a r s ) 2. How many y e a r s t e a c h i n g e x p e r i e n c e do you have? (combine b o t h p u b l i c and p r i v a t e ) ( 0-2,3-5,6-10,11-15,16-20,>20) 3. How many c o u r s e s i n s t a t i s t i c s o r measurement and e v a l u a t i o n do you have? ( d i s r e g a r d u n i t v a l u e ) (0,1,2,3,>3) 4. What i s your t e a c h i n g assignment? ( f u l l t i m e , p a r t t i m e ) 48 5. I f your answer t o #4 was " p a r t t i m e " , e x p l a i n your t e a c h i n g assignment ( p e r c e n t a g e o f t i m e , s u b j e c t s , e t c . ) I n c l u d e d w i t h t h e q u e s t i o n n a i r e was a sample of f i v e f i c t i t i o u s achievement s c o r e s f o r f i v e f i c t i t i o u s s t u d e n t s and the w e i g h t t h a t each assignment c o n t r i b u t e d t o the f i n a l c o u r s e s c o r e . The r e s p o n d e n t s were asked t o c o l l a t e and rank the s c o r e s u s i n g the method most f a m i l i a r t o them. T h i s s e r v e d as a g r a p h i c d e m o n s t r a t i o n of the c o l l a t i n g methods commonly used by t e a c h e r s . F i c t i t i o u s Data The h y p o t h e t i c a l d a t a c o n s i s t e d o f s t u d e n t names, t h e i r raw s c o r e s on each of the f i c t i t i o u s a s s i g n m e n t s , and the w e i g h t each assignment was t o c o n t r i b u t e t o the t o t a l term s c o r e . The t e a c h e r s were asked t o w e i g h t , c o l l a t e , and rank the assignment s c o r e s u s i n g t h e i r accustomed methods. S u b s e q u e n t l y , t h e s e r a n k i n g s were a n a l y z e d and compared t o the r a n k i n g s o f the same d a t a a f t e r the raw assignment s c o r e s had been s t a t i s t i c a l l y b a l a n c e d . For comparison p u r p o s e s , t h e raw s c o r e s on each of t h e f i c t i t i o u s a s s ignments were t r a n s f o r m e d t o T-scores and b a l a n c e d p r i o r t o b e i n g w e i g h t e d and c o l l a t e d . Each s t u d e n t ' s r e v i s e d t o t a l was t h e n ranked and compared t o the t e a c h e r s ' r a n k i n g of t h e raw s c o r e d a t a . By a n a l y z i n g the r e s u l t s of 4 9 the f i c t i t i o u s d a t a , i t was p o s s i b l e t o determine the p o t e n t i a l f o r e r r o r and l a c k o f r e l i a b i l i t y when t e a c h e r s c o l l a t e s t u d e n t achievement s c o r e s . An example of t h e h y p o t h e t i c a l d a t a s u p p l i e d t o the s u b j e c t s i s l i s t e d i n Ta b l e I. The b r a c k e t t e d i n f o r m a t i o n (weighted t o t a l s c o r e ) was not s u p p l i e d t o the res p o n d e n t s and i s l i s t e d here o n l y t o i n d i c a t e the d i s t r i b u t i o n o f t h e raw s c o r e s . T a b l e I I d i s p l a y s the same d a t a a f t e r i t has been b a l a n c e d so t h a t each assignment now has a mean of 50 and a s t a n d a r d d e v i a t i o n of 10. S t a t i s t i c a l b a l a n c i n g was a p p l i e d b e f o r e the assignment s c o r e s were w e i g h t e d and c o l l a t e d . I n t h i s s t u d y , t h e r a n k i n g of the "Weighted T o t a l S c o r e s " . was -the main p o i n t of i n t e r e s t . Of p a r t i c u l a r i n t e r e s t i n t h i s example i s how t h e s t u d e n t r a n k i n g s i n T a b l e I I d i f f e r from t h o s e i n T a b l e I . T h i s demonstrates how, even w i t h o n l y f i v e s t u d e n t s , s t a t i s t i c a l b a l a n c i n g can have a n o t i c e a b l e e f f e c t on s t u d e n t r a n k i n g s . With a l a r g e r number o f s t u d e n t s , the e f f e c t may be even more pronounced. S i n c e t h e r a n k i n g s a r e used t o dete r m i n e l e t t e r g r a d e s , i t i s i m p e r a t i v e t h a t the c o l l a t e d s c o r e s be as r e l i a b l e as p o s s i b l e . Record Sheets Examples o f t e a c h e r s ' r e c o r d s were s e l e c t e d from two s u b j e c t a r e a s : mathematics and s o c i a l s t u d i e s . Each o f the two r e c o r d s h e e t s o b t a i n e d from the re s p o n d e n t s r e v e a l e d t h e s t u d e n t s ' names, t h e raw s c o r e s f o r the v a r i o u s assignments and t e s t s , t h e s t u d e n t s ' c o l l a t e d s c o r e s and l e t t e r g r a d e s , as 50 ASSIGNMENTS raw s c o r e s WEIGHTED NAME 1 2 3 4 5 TOTAL SCORE RANK TOTAL SCORE RANK 1 . S t u d e n t A 50 55 65 57 95 [64.4] [3] [72.7] [1] 2 . S t u d e n t B 69 60 58 80 83 [70.0] [1] [71.7] [2] 3 . St u d e n t C 45 72 52 71 75 [63.0] [4.5] [66.4] [3] 4 . St u d e n t D 75 95 45 83 25 [64.6] [2] [53.8] [4] 5 . S t u d e n t E 73 83 60 63 36 [63.0] [4.5] [45.8] [5] WEIGHTS ( %) 10 20 20 10 40 [ X m ] [ 6 2 ] [ 7 3 ] [ 5 6 ] [ 7 1 ] [ 6 3 ] [SD] [ 1 4 ] [ 1 6 ] [ 8 ] [ 1 1 ] [ 3 1 ] T a b l e I - F i c t i t i o u s d a t a t o be c o l l a t e d and ranked ASSIGNMENTS T-scores WEIGHTED NAME 1 2 3 4 5 TOTAL SCORE RANK TOTAL SCORE RANK 1 . Student A 41 39 61 37 60 47.6 4 51. 8 2 2 . Student B 55 42 53 58 56 52.8 1 52.7 1 3 . Student C 38 49 45 50 54 47.2 5 49 . 2 3 4 . Student D 59 64 36 61 38 51.6 2 47.2 5 5 . Student E 58 56 55 43 41 50.6 3 48.7 4 WEIGHTS (%) 10 20 20 10 40 Xm 50 50 50 50 50 SD 10 10 10 10 10 Table I I - F i c t i t i o u s d a t a a f t e r b e i n g s t a t i s t i c a l l y b a l a n c e d 51 w e l l as the w e i g h t i n g t h a t each assignment was t o c o n t r i b u t e t o t h e c o l l a t e d s c o r e s . At the end of a term ( j u s t p r i o r t o the d i s t r i b u t i o n of r e p o r t c a r d s ) , c o p i e s of t h e r e c o r d s h e e t s f o r bo t h s u b j e c t a r e a s were c o l l e c t e d from the r e s p o n d e n t s . Each c l a s s r e c o r d sheet was s u b j e c t e d t o the f o l l o w i n g p r o c e d u r e . With the a i d of a computer, t h e assignment raw s c o r e s were s t a t i s t i c a l l y b a l a n c e d and c o n v e r t e d t o T - s c o r e s . They were then w e i g h t e d , c o l l a t e d and ranked u t i l i z i n g t he same assignment w e i g h t i n g s as used by the t e a c h e r . The r e v i s e d t o t a l s and r a n k i n g s were r e t u r n e d t o the o r i g i n a l respondent t o be r e g r a d e d . I t was hoped t h a t , by h a v i n g the respondent r e g r a d e t h e b a l a n c e d r e s u l t s , a l l p r o c e d u r a l and s u b j e c t i v e f a c t o r s a f f e c t i n g t h e s t u d e n t s ' grades would be h e l d c o n s t a n t f o r b o t h the o r i g i n a l and the b a l a n c e d s e t s of d a t a . The d i f f e r e n c e s i n the r a n k i n g s o f the raw s c o r e and the t r a n s f o r m e d s c o r e s were a n a l y z e d i n two ways. F i r s t l y , t he r a n k i n g s were compared t o determine i f t h e r e were s t a t i s t i c a l l y s i g n i f i c a n t d i f f e r e n c e s i n the c o r r e l a t i o n between the two r a n k i n g s . S e c o n d l y , l e t t e r grades r e s u l t i n g from b o t h the raw s c o r e and the T-score r a n k i n g s were compared t o d e t e r m i n e how many s t u d e n t s r e c e i v e d a change i n l e t t e r grade. The r e s u l t s from t h e s e a n a l y s e s were c o m p i l e d and t a b u l a t e d . A complete documentation of the r e s u l t s can be found i n c h a p t e r s f o u r and f i v e o f t h i s s t u d y . 52 G r a d i n g P o l i c y I n o r d e r t o promote c o n s i s t e n c y i n the a l l o c a t i o n of l e t t e r grades i n the e l e m e n t a r y s c h o o l s e t t i n g , the s c h o o l d i s t r i c t used i n t h i s s tudy has i n e f f e c t a p o l i c y o u t l i n i n g how t h e l e t t e r grade f r e q u e n c y d i s t r i b u t i o n s h o u l d be d e t e r m i n e d f o r each c l a s s . E s s e n t i a l l y , the p o l i c y s t a t e s t h a t t h e l e t t e r grade d i s t r i b u t i o n s h o u l d be based o n . s t u d e n t a b i l i t y e s t i m a t e s d e r i v e d from t h e n a t i o n a l p e r c e n t i l e s o f s t a n d a r d i z e d achievement t e s t r e s u l t s . T h i s i s a c h i e v e d by a d m i n i s t e r i n g the s t a n d a r d i z e d Canadian Achievement T e s t (CAT) t o the i n t e r m e d i a t e grades once each s c h o o l y e a r . The r e s u l t i n g n a t i o n a l p e r c e n t i l e s f o r each c l a s s a r e used t o a s s i s t i n the e s t a b l i s h m e n t of an approximate grade d i s t r i b u t i o n f o r t h a t c l a s s . T a b l e I I I i l l u s t r a t e s the g u i d e l i n e s t h a t a r e s u p p l i e d t o t e a c h e r s i n o r d e r t o a s s i s t them i n r e l a t i n g n a t i o n a l p e r c e n t i l e s t o l e t t e r grade d i s t r i b u t i o n s . . I f , f o r example, i n a g i v e n c l a s s 3 s t u d e n t s s c o r e d between 96 and 100 on t h e CAT n a t i o n a l p e r c e n t i l e , 7 between 7 6 and 95, 4 between 61 and 75, 9 between 41 and 60, 4 between 2 6 and 40, 3 between 6 and 25, and none between 0 and 5, then a p p r o x i m a t e l y t h r e e s t u d e n t s s h o u l d r e c e i v e an A, a p p r o x i m a t e l y 7 s h o u l d r e c e i v e a B, and so on. The d i s t r i c t g u i d e l i n e s s u ggest o n l y an approximate d i s t r i b u t i o n as t e a c h e r s a r e urged t o c o n s i d e r a l s o the s t a n d a r d s of a c c e p t a b l e achievement f o r t h e i r p a r t i c u l a r grade when a l l o c a t i n g l e t t e r g r a d e s . However, i f the g u i d e l i n e s were t o be s t r i c t l y f o l l o w e d , t h e n t h e t h r e e s t u d e n t s w i t h t h e h i g h e s t NATIONAL PERCENTILE LETTER GRADE 96 - 100 A 76 - 95 B 61 - 75 C+ 41 - 60 C 26 - 40 C- 6 - 25 D 0 - 5 E Ta b l e I I I - L e t t e r grade d i s t r i b u t i o n g u i d e l i n e s c u m u l a t i v e s c o r e s from t h e v a r i o u s term assignments would r e c e i v e the A's, w h i l e the n e x t seven h i g h e s t would r e c e i v e the B's, e t c e t e r a . T h i s p r o c e d u r e " t a k e s i n t o c o n s i d e r a t i o n the a b i l i t y l e v e l s o f the p u p i l s i n a p a r t i c u l a r grade o r c l a s s " ( B r i t i s h Columbia M i n i s t r y of E d u c a t i o n , 1979, p. 26) However, the s c h o o l d i s t r i c t p o l i c y a l s o s t a t e s , " I n c o n s i d e r i n g the c u m u l a t i v e marks f o r each s u b j e c t , " n a t u r a l b r e a k s " were used f u r t h e r t o e s t a b l i s h l e t t e r grade d i s t r i b u t i o n s " ( S c h o o l D i s t r i c t P o l i c i e s and R e g u l a t i o n s ) . T h i s approach i s a l s o advocated by the B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n i n t h e i r b o o k l e t on g r a d i n g p r a c t i s e s . " N a t u r a l b r e a k s " r e f e r t o the gaps t h a t appear i n the ranked d i s t r i b u t i o n o f s t u d e n t s ' c u m u l a t i v e s c o r e s . The gaps can b used as grade c u t - o f f p o i n t s t o s e p a r a t e one grade c a t e g o r y 54 from a n o t h e r . Combining the n a t u r a l b reaks and the group a b i l i t y methods t o determine the l e t t e r grade d i s t r i b u t i o n a l l o w s " t e a c h e r s t o make j u d i c i o u s d e c i s i o n s as t o what c o n s t i t u t e s h i g h and low achievement" ( B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n , 1979, p. 2 6 ) . U s i n g t h e s e p r o c e d u r e s t o e s t a b l i s h t h e l e t t e r grade d i s t r i b u t i o n f o r each c l a s s h i g h l i g h t s the importance of combining assignment s c o r e s r e l i a b l y . Data C o l l e c t i o n and A n a l y s i s The t e a c h e r s were g i v e n r e c o r d s h e e t s a t the b e g i n n i n g of the r e p o r t i n g p e r i o d on which t o r e c o r d the achievement s c o r e s f o r b o t h academic s u b j e c t s . At the end of the term the r e c o r d s h e e t s and the q u e s t i o n n a i r e s were c o l l e c t e d and a n a l y z e d . I n f o r m a t i o n from t h e q u e s t i o n n a i r e s was c o l l a t e d and t a b u l a t e d . The achievement s c o r e s f o r each o f the two s u b j e c t s were s t a t i s t i c a l l y b a l a n c e d and a g g r e g a t e d , and t h e s t u d e n t s were ranked based on the a g g r e g a t e d s c o r e . T h i s " b a l a n c e d " r a n k i n g was c o r r e l a t e d w i t h t h e t e a c h e r s ' r a n k i n g u s i n g the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t r , . ranKs A d d i t i o n a l l y , the r e s u l t s from the r e g r a d i n g of t h e b a l a n c e d d a t a were compared t o t h e grades a s s i g n e d u s i n g t h e t e a c h e r s ' o r i g i n a l d a t a . The number of s t u d e n t s who r e c e i v e d a change i n l e t t e r grade was r e c o r d e d and documented. The d i r e c t i o n and magnitude of the change was a l s o t a b u l a t e d . From t h e t a b u l a t e d r e s u l t s , i t was p o s s i b l e t o 55 g e n e r a l i z e as t o whether t e a c h e r s , s i m i l a r t o tho s e i n t h e sample, c o u l d b e n e f i t from i n - s e r v i c e and p r e - s e r v i c e t r a i n i n g on r e v i s e d g r a d i n g t e c h n i q u e s . I t a l s o became a p p a r e n t , from the r e s u l t s , whether o r not g r a d i n g p r a c t i s e s can be and s h o u l d be made more r e l i a b l e . 56 CHAPTER 4 R e s u l t s The data c o l l e c t e d i n the study were d i v i d e d i n t o f o u r c a t e g o r i e s : (a) demographic i n f o r m a t i o n concerning the sample, (b) data concerning the s u b j e c t s ' a t t i t u d e s and p e r c e p t i o n s toward g r a d i n g and r e p o r t i n g , (c) a n a l y s i s of the sample's grading techniques, and (d) a n a l y s i s of the s u b j e c t s ' course r e c o r d sheets. Sample A sample of 37 s u b j e c t s was used i n the study. While a l l 37 responded to the f i r s t data request (copies of t h e i r r e c o r d s h e e t s ) , 3 4 responded to the second set of data (survey and regraded r e c o r d s h e e t s ) . Three of the 34 s u b j e c t s e l e c t e d t o w i t h h o l d t h e i r regraded r e c o r d sheets and submitted o n l y the completed q u e s t i o n n a i r e . A t o t a l of 31 s u b j e c t s submitted both regraded r e c o r d sheets and completed q u e s t i o n n a i r e s . Of the 31 respondents, 25 p r o v i d e d regraded data f o r both mathematics and s o c i a l s t u d i e s . The sample p r o v i d e d raw scores and l e t t e r grades f o r 1,314 elementary school students from 56 c l a s s e s . Tables IV through IX p r o v i d e the frequency d i s t r i b u t i o n s of the sample's demographic v a r i a b l e s . Tables X through XVII d i s p l a y the frequency d i s t r i b u t i o n s of the s u b j e c t s ' responses to the a t t i t u d e and p e r c e p t i o n q u e s t i o n s p e r t a i n i n g t o grading and r e p o r t i n g . 57 Demographic V a r i a b l e s a) Gender Male 24 Feraale 8 T o t a l 34 T a b l e IV - D i s t r i b u t i o n by Gender b) Age ( y e a r s ) Less than 25 0 25-30 0 31-40 25 41-50 8 51-60 1 G r e a t e r t h a n 60 ... 0 T o t a l 34 T a b l e V - D i s t r i b u t i o n by Age ( y e a r s ) c) Y e a r s of u n i v e r s i t y two 0 t h r e e 1 f o u r 10 B.Ed 15 MA o r MEd 7 o t h e r ( i e d i p l o m a ) . 1 T o t a l 34 T a b l e VI - D i s t r i b u t i o n by U n i v e r s i t y T r a i n i n g ( y e a r s ) 58 d) Number of measurement and/or e v a l u a t i o n courses taken none 11 one 21 two 1 three 1 More than t h r e e . . . 0 T o t a l . 34 Table VII - D i s t r i b u t i o n by Measurement Courses Taken e) Teaching assignment p a r t time 3 f u l l time 31 T o t a l 34 Table V I I I - D i s t r i b u t i o n by Teaching Assignment f) Teaching Experience (years) 0-5 0 6-10 7 11-15 15 16-20 9 Greater than 20 ... 3 T o t a l 34 Table IX - D i s t r i b u t i o n by Teaching Experience (years) 59 Of the 34 s u b j e c t s c o m p l e t i n g the second s e t of d a t a , most ( 7 4 % ) were male t e a c h e r s and most ( 7 4 % ) were i n the 31 t o 40 y e a r age c a t e g o r y . Twenty f o u r per c e n t o f the r e s p o n d e n t s were i n the 41 t o 50 y e a r age c a t e g o r y . Teachers who had completed a B a c h e l o r ' s degree made up 44% of t h e sample w h i l e 29% had f o u r y e a r s u n i v e r s i t y e x p e r i e n c e and 2 1 % had completed a M a s t e r ' s degree. N e a r l y a l l of the r e s p o n d e n t s ( 9 1 % ) were f u l l time t e a c h e r s and 4 4 per c e n t had between 11 and 15 y e a r s t e a c h i n g e x p e r i e n c e . Twenty s i x per c e n t of the t e a c h e r s had 16 .to 20 y e a r s t e a c h i n g e x p e r i e n c e . Many of the s u b j e c t s (62%) had completed one measurement and/or e v a l u a t i o n c o u r s e w h i l e 32% had not t a k e n any measurement o r e v a l u a t i o n c o u r s e s . In g e n e r a l , t h e t y p i c a l respondent was a 31 t o 40 year o l d male t e a c h e r who was employed f u l l t i me and who had between 11 and 15 y e a r s of t e a c h i n g e x p e r i e n c e . He t y p i c a l l y h e l d a B a c h e l o r ' s degree and had completed one measurement and/or e v a l u a t i o n c o u r s e . A t t i t u d e s and P e r c e p t i o n s Toward G r a d i n g and R e p o r t i n g T a b l e s X t h r o u g h XVII r e f l e c t t h e sample's a t t i t u d e and p e r c e p t i o n towards g r a d i n g and r e p o r t i n g s t u d e n t achievement. The s u b j e c t s were t o respond on a f i v e - p o i n t s c a l e t h e e x t e n t of agreement between th e f e e l i n g e x p r e s s e d i n each statement and t h e i r own p e r s o n a l f e e l i n g . The f i v e o p t i o n s were: S t r o n g l y D i s a g r e e (SD), D i s a g r e e (D), Undecided (U), Agree (A ) , and S t r o n g l y Agree (SA). 60 ) My c o l l e a g u e s a t t h i s s c h o o l t a k e r e p o r t c a r d marks v e r y s e r i o u s l y . SD 0 D 0 U 1 A 15 SA 18 T o t a l 34 T a b l e X - D i s t r i b u t i o n o f c o l l e a g u e s ' a t t i t u d e s toward r e p o r t c a r d marks h) L e t t e r grades are an e f f e c t i v e method of i n f o r m i n g p a r e n t s of t h e i r c h i l d ' s p r o g r e s s and achievement. SD 1 D 5 U 2 A 22 SA 4 T o t a l 34 T a b l e XI - D i s t r i b u t i o n of a t t i t u d e toward l e t t e r grade e f f e c t i v e n e s s To t h e b e s t o f my knowledge, p a r e n t s a r e g e n e r a l l y s a t i s f i e d w i t h t h e l e t t e r grade system o f r e p o r t i n g . SD 0 D 2 U 1 A 28 SA 3 T o t a l 34 T a b l e X I I - D i s t r i b u t i o n o f agreement r e g a r d i n g p a r e n t s s a t i s f a c t i o n toward l e t t e r grade r e p o r t i n g systems. I am v e r y c o n f i d e n t t h a t t h e l e t t e r grades I a s s i g n a r e a c c u r a t e and r e l i a b l e . SD 0 D 0 U 4 A 23 SA. . 7 T o t a l 34 T a b l e X I I I - D i s t r i b u t i o n o f agreement r e g a r d i n g s u b j e c own g r a d i n g a b i l i t y 62 (k) I would l i k e t o l e a r n more about c o l l a t i n g and a s s i g n i n g l e t t e r grades t o raw s c o r e s . SD 0 D 5 U 3 A 16 SA 10 T o t a l 3~4 T a b l e XIV - D i s t r i b u t i o n of agreement r e g a r d i n g s u b j e c t s ' d e s i r e t o l e a r n more about m a r k i n g and g r a d i n g (1) More i n - s e r i v c e s e s s i o n s on g r a d i n g and r e p o r t i n g s h o u l d be o f f e r e d . SD 0 D 5 U 4 A 11 SA 14 T o t a l 34 Ta b l e XV - D i s t r i b u t i o n o f agreement on e v a l u a t i o n o f i n - s e r v i c e s e s s i o n s (m) U n i v e r s i t i e s s h o u l d o f f e r more p r e - s e r v i c e i n s t r u c t i o n i n e f f e c t i v e g r a d i n g and r e p o r t i n g . SD 0 D 3 U 4 A . 14 SA 13 T o t a l 34" Tab l e XVI - D i s t r i b u t i o n of agreement r e g a r d i n g s u b j e c t s ' o p i n i o n on p r e - s e r v i c e t r a i n i n g (n) The r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o communicated t o p a r e n t s , s t u d e n t s , e t c . SD 0 D 8 U 6 A 16 SA 4 T o t a l 34 T a b l e XVII - D i s t r i b u t i o n o f agreement r e g a r d i n g e f f e c t i v e n e s s of t h e r e p o r t c a r d format as communication d e v i c e N e a r l y a l l (97%) of t h e t e a c h e r s s u r v e y e d agreed t h a t t h e i r c o l l e a g u e s t r e a t m arking and g r a d i n g s e r i o u s l y (44% Agree;53% S t r o n g l y A g r e e ) . A m a j o r i t y of the r e s p o n d e n t s (71%) agreed t h a t l e t t e r grades a r e an e f f e c t i v e method of i n f o r m i n g p a r e n t s of t h e i r c h i l d ' s p r o g r e s s and achievement. F i f t y n i n e per c e n t o f the sample a l s o agreed w i t h t h e f e e l i t h a t t h e r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o be communicated t o p a r e n t s , s t u d e n t s , and o t h e r i n t e r e s t e d groups. However, 18% were u n d e c i d e d and 24% d i s a g r e e d w i t h t h i s s t a t e m e n t . I n r e sponse t o the statement c o n c e r n i n g p a r e n t s a t i s f a c t i o n , most respon d e n t s (91%) agreed t h a t p a r e n t s were s a t i s f i e d w i t h the l e t t e r grade system of r e p o r t i n g . In r e sponse t o the q u e s t i o n s p e r t a i n i n g t o t e a c h e r s ' g r a d i n g and r e p o r t i n g t e c h n i q u e s , a m a j o r i t y o f the s u b j e c t s 64 (88%) agreed (68% Agree; 20% S t r o n g l y Agree) t h a t the l e t t e r grades they a s s i g n are r e l i a b l e and a c c u r a t e . However, most respondents i n d i c a t e d a d e s i r e to l e a r n more about c o l l a t i n g and g r a d i n g raw achievement scores (76% Agree or S t r o n g l y Agree; 9% Undecided; 15% D i s a g r e e ) . The percentages of respondents agr e e i n g to the need f o r more i n - s e r v i c e and p r e - s e r v i c e i n s t r u c t i o n i n t h i s area were 74% and 79% r e s p e c t i v e l y . On the same q u e s t i o n s , 12% were undecided f o r both i n - s e r v i c e and p r e - s e r v i c e s e s s i o n s while the percentages of respondents who d i s a g r e e d with the need f o r i n - s e r v i c e and p r e - s e r v i c e i n s t r u c t i o n were 15% and 9% r e s p e c t i v e l y . In g e n e r a l , the t y p i c a l respondent was one who (a) thought h i s c o l l e a g u e s took r e p o r t c a r d marks s e r i o u s l y , (b) thought l e t t e r grades are an e f f e c t i v e method of i n f o r m i n g parents of t h e i r students p r ogress, (c) thought t h a t parents were s a t i s f i e d with the l e t t e r grade system of marking, (d) thought the B r i t i s h Columbia M i n i s t r y of Education r e p o r t c a r d format allows s u f f i c i e n t i n f o r m a t i o n t o be communicated to parents and students, (e) thought t h a t more i n - s e r v i c e and p r e - s e r v i c e s e s s i o n s on e v a l u a t i o n and g r a d i n g should be o f f e r e d by school d i s t r i c t s and u n i v e r s i t i e s , and (f) d i s p l a y e d an i n t e r e s t i n l e a r n i n g more about aggregating and grading raw s c o r e s . Grading Techniques The r e c o r d sheets and the responses to the gra d i n g and r e p o r t i n g s e c t i o n of the survey were analyzed t o determine the 65 g r a d i n g p r o c e d u r e s used by the s u b j e c t s . A l t h o u g h a l l o f the r e s p o n d e n t s employed a v a r i a t i o n o f the n o r m - r e f e r e n c e d approach t o g r a d i n g , the sample's g r a d i n g t e c h n i q u e s were c l a s s i f i e d f u r t h e r a c c o r d i n g t o the f o l l o w i n g g r a d i n g methods: 1. Method One - Summing raw s c o r e s and c o n v e r t i n g each s t u d e n t ' s t o t a l t o a p e r c e n t a g e p r i o r t o g r a d i n g by n a t u r a l b r e a k s (see page 53) . 2. Method Two - Summing each s t u d e n t ' s raw s c o r e s and c o n v e r t i n g the t o t a l t o a p e r c e n t a g e . L e t t e r grades a r e a s s i g n e d based on the n u m e r i c a l v a l u e o f t h e p e r c e n t a g e as w e l l as a s u b j e c t i v e f a c t o r . Examples of s u b j e c t i v e f a c t o r s were: (a) a d j u s t i n g l e t t e r grades t o r e f l e c t i n d i v i d u a l e f f o r t ; (b) a d j u s t i n g l e t t e r grades t o r e f l e c t a change i n a s t u d e n t ' s performance from one term t o a n o t h e r ; o r (c) a d j u s t i n g l e t t e r grades t o compensate f o r m i s s e d a s s i g n m e n t s . 3. Method Three - Summing w e i g h t e d raw s c o r e s f o r each s t u d e n t and c o n v e r t i n g t h e sum t o a p e r c e n t a g e p r i o r t o g r a d i n g a c c o r d i n g t o n a t u r a l b r e a k s . 4. Method Four - Summing each s t u d e n t ' s raw s c o r e s and c o n v e r t i n g the t o t a l t o a p e r c e n t a g e . L e t t e r grades a r e a s s i g n e d based on the n u m e r i c a l v a l u e of t h e p e r c e n t a g e . T h i s method i s commonly c a l l e d " p e r c e n t g r a d i n g " (see page 32). 5. Method F i v e - Term a s s i g n m e n t s , p r o j e c t s and t e s t s a r e g i v e n l e t t e r grades r a t h e r than n u m e r i c a l s c o r e s . Each s t u d e n t ' s f i n a l grade i s t h e average of a l l the l e t t e r grades awarded t h a t s t u d e n t d u r i n g the c o u r s e . For a v e r a g i n g p u r p o s e s , each l e t t e r grade i s g i v e n an o r d i n a l v a l u e . 66 6. Method S i x - Raw s c o r e s a r e b a l a n c e d t o have e q u a l means and e q u a l s t a n d a r d d e v i a t i o n s b e f o r e b e i n g w e i g h t e d and c o l l a t e d . The c o l l a t e d s c o r e s a r e ranked and graded a c c o r d i n g t o n a t u r a l b r e a k s . T a b l e X V I I I g i v e s the f r e q u e n c y breakdown o f each g r a d i n g method. T a b l e XIX d i s p l a y s how e x t e n s i v e l y each respondent a p p l i e d t h e i r g r a d i n g t e c h n i q u e . Method One 2 4 Method Two 5 Method Three 2 Method Four 2 Method F i v e 2 Method S i x 2 T o t a l 37 Ta b l e X V I I I - D i s t r i b u t i o n o f g r a d i n g method Every s c h o o l s u b j e c t 9 Core s u b j e c t s (Math, S c i e n c e S o c i a l S t u d i e s , E n g l i s h ) . . . 24 I n d i s c r i m i n a t e use 1 T o t a l 34 Ta b l e XIX - D i s t r i b u t i o n of a p p l i c a t i o n of g r a d i n g t e c h n i q u e 67 The r e s p o n d e n t s were a l s o asked t o d e s c r i b e how t h e y d e t e r m i n e d a f i n a l s c o r e f o r t h o s e s t u d e n t s who have m i s s e d one o r more t e s t s o r a s s i g n m e n t s . T h e i r r e s p o n s e s were c l a s s i f i e d under the f o l l o w i n g methods: 1. Method One - C a l c u l a t e the average p e r c e n t a g e on o n l y the a ssignments completed and i g n o r e the m i s s e d a s s i g n m e n t s . 2. Method Two - E s t i m a t e a s c o r e f o r the m i s s e d assignment by l o o k i n g a t : (a) the absent s t u d e n t ' s p r e v i o u s work, (b) the marks o t h e r s t u d e n t s of s i m i l a r a b i l i t y o b t a i n e d on t h e m i s s e d assignment. 3. Method Three - A s s i g n a mark of z e r o f o r t h e missed assignment. 4. Method Four - C a l c u l a t e a l e t t e r grade based on p r e v i o u s work and reduce by one l e t t e r grade f o r m i s s e d a s s i g n m e n t ( s ) . 5. Method F i v e - C a l c u l a t e an average T s c o r e based on a l l completed work and a s s i g n t h a t v a l u e t o the missed a s s i g n m e n t ( s ) . When a l l assignment s c o r e s have been s t a t i s t i c a l l y b a l a n c e d t h i s method e s s e n t i a l l y m a i n t a i n s rank p o s i t i o n and does not p e n a l i z e nor reward t h e s t u d e n t f o r b e i n g a b s e n t . T a b l e XX shows the f r e q u e n c y o f the methods used by the sample t o a r r i v e a t a mark f o r an absent s t u d e n t . 68 Method One 2 5 Method Two 5 Method Three 2 Method Four 1 Method F i v e 0 Other 1 T o t a l 34 Table XX - D i s t r i b u t i o n of compensation method used The r e s u l t s o f the p r o c e d u r e s used t o compensate s t u d e n t s f o r m i s s e d assignments ( T a b l e XX) i n d i c a t e d t h a t most responde n t s (74%) c a l c u l a t e d an average s c o r e based s o l e l y on the work the absent s t u d e n t had completed. The d a t a a l s o r e v e a l s t h a t 15% o f the r e s p o n d e n t s use s u b j e c t i v e means t o e s t i m a t e a s c o r e f o r a m i s s e d assignment. The p a r t i c i p a n t s i n t h e s t u d y were asked t o a p p l y t h e i r g r a d i n g t e c h n i q u e t o a s e t of f i c t i t i o u s d a t a (see page 48) and rank t h e f i v e s t u d e n t s from h i g h e s t t o l o w e s t . T a b l e XXI shows the f r e q u e n c y o f t h e r a n k s as d e t e r m i n e d by the v a r i o u s g r a d i n g t e c h n i q u e s . In r e s p o n s e t o the second r e s e a r c h q u e s t i o n (page 16) r e g a r d i n g t h e r e l i a b i l i t y of the a g g r e g a t i o n and w e i g h t i n g o f raw s c o r e s f o r g r a d i n g p u r p o s e s , t h e sample's r e c o r d s h e e t s and the q u e s t i o n n a i r e r e s p o n s e s p e r t a i n i n g t o g r a d i n g t e c h n i q u e s were a n a l y z e d . The a n a l y s i s r e v e a l e d a h i g h degree of s i m i l a r i t y between th e a g g r e g a t i o n methods e v i d e n t on the 69 Student A B C D E 1 , 2 , 3 , 5 , 4 16 3 , 1 , 4 . 5 , 2 , 4 . 5 6 2 , 1 , 3 , 5 , 4 3 1 . 2 . 4 . 5 . 3 2 4 . 5 , 1 , 3 , 2 , 4 . 5 1 2 . 1 . 5 . 3 . 4 1 2 , 3 , 5 , 1 , 4 1 1 . 2 . 5 . 2 . 5 . 5 . 4 1 2 . 1 . 4 . 5 . 3 . 4 . 5 1 Incomplete 2 T o t a l 34 Table XXI - D i s t r i b u t i o n of ranks of f i c t i t i o u s data r e c o r d sheets and the aggregation techniques d e s c r i b e d by the respondents on the q u e s t i o n n a i r e . However, the grading methods used i n response to the f i c t i t i o u s data s e c t i o n of the survey were l e s s c o n s i s t e n t with the methods e v i d e n t on the grade sheets and i n the w r i t t e n d e s c r i p t i o n s . The most frequent d i s c r e p a n c y concerned the weighting f a c t o r s t o be a p p l i e d t o the f i c t i t i o u s data. Although most respondents d i d not normally weight raw sc o r e s , many attempted t o apply weights t o the f i c t i t i o u s data. x As shown i n t a b l e XXI, 18% of the sample obtained a rank order f o r the f i c t i t i o u s t o t a l scores t h a t was c o n s i s t e n t with the most common grading method i d e n t i f i e d i n t h i s study (see 70 T a b l e X V I I I ) . Nine per c e n t o f the re s p o n d e n t s a p p l i e d a g g r e g a t i o n and w e i g h t i n g t e c h n i q u e s t h a t a r e advocated by many e x p e r t s (see page 2 8 ) . The su g g e s t e d methods a r e t a b u l a t e d as Method 6 i n T a b l e X V I I I . Many s u b j e c t s (47%) used a g g r e g a t i o n and w e i g h t i n g t e c h n i q u e s t h a t were i n c o n s i s t e n t w i t h any of t h e o t h e r g r a d i n g methods p r e v i o u s l y i d e n t i f i e d i n t h i s s t u d y . T h e i r methods e s s e n t i a l l y i n v o l v e d combining w e i g h t e d raw s c o r e s . However, t h e y o b t a i n e d r e s u l t s t h a t were v e r y s i m i l a r t o b o t h t h e r a n k s of the ag g r e g a t e d w e i g h t e d raw s c o r e s ( T a b l e I ) and t o t h e r a n k s of the agg r e g a t e d w e i g h t e d b a l a n c e d s c o r e s ( T a b l e I I ) . The r e s u l t s o f the c l a s s i f i c a t i o n and t a b u l a t i o n of the v a r i o u s g r a d i n g methods used by the r e s p o n d e n t s , as t a b l e d i n Tab l e X V I I I , r e v e a l e d t h a t a 65% o f the sample employed an a g g r e g a t i o n t e c h n i q u e t h a t i n v o l v e d summing the raw s c o r e s and c o n v e r t i n g t h e c o l l a t e d s c o r e s t o a p e r c e n t a g e p r i o r t o g r a d i n g by n a t u r a l b r e a k s . The r e s u l t s a l s o showed t h a t 14% of t h e sample used an a g g r e g a t i o n t e c h n i q u e t h a t i n v o l v e d summing the raw s c o r e s , c o n v e r t i n g t h e combined s c o r e s t o a p e r c e n t a g e , and u s i n g the n u m e r i c a l v a l u e o f t h e p e r c e n t a g e as w e l l as s u b j e c t i v e f a c t o r s t o determine t h e l e t t e r grade. Only two o f the 37 re s p o n d e n t s (5%) used " r e l i a b l e " methods t o a p p l y t h e d e s i r e d w e i g h t i n g f a c t o r s t o the raw s c o r e s . Record S h e e t s . The r a n k i n g s o f t h e s t a t i s t i c a l l y b a l a n c e d t o t a l s were compared t o the r a n k i n g s of the raw s c o r e t o t a l s u s i n g 71 Spearman's R a n k - C o r r e l a t i o n C o e f f i c i e n t . T a b l e X X I I l i s t s the s u b j e c t number, the number of s t u d e n t s i n each of the s u b j e c t ' s two c l a s s e s ( n ) , t h e rank c o r r e l a t i o n c o e f f i c i e n t ( r g ) , and t h e c r i t i c a l v a l u e s o f Spearman's rank C o r r e l a t i o n C o e f f i c i e n t f o r t e s t i n g the n u l l h y p o t h e s i s o f no c o r r e l a t i o n w i t h a t w o - t a i l e d t e s t a t the alpha=.01 l e v e l o f s i g n i f i c a n c e ( G l a s s , G., S t a n l e y , J . , 1970, p. 539). T a b l e X X I I I d i s p l a y s the d i s t r i b u t i o n of the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s . Of t h e 56 c o r r e l a t i o n s computed, a l l had r v a l u e s t h a t s were g r e a t e r t h a n the c o r r e s p o n d i n g c r i t i c a l v a l u e . These r e s u l t s i n d i c a t e t h a t the c o r r e l a t i o n s between th e r a n k i n g s of the raw s c o r e t o t a l s and t h e r a n k i n g s of the b a l a n c e d s c o r e t o t a l s were p o s i t i v e and s i g n i f i c a n t . T h e r e f o r e , the n u l l h y p o t h e s i s (H Q:rho=0) s h o u l d be r e j e c t e d a t the .01 l e v e l o f s i g n i f i c a n c e i n f a v o u r o f t h e a l t e r n a t e h y p o t h e s i s f o r each of the 56 r e c o r d s h e e t s a n a l y z e d . In t h e second s e t o f d a t a , the s u b j e c t s were asked t o r e g r a d e t h e i r s t u d e n t s based on the s t a t i s t i c a l l y b a l a n c e d t o t a l s p r o v i d e d . The l e t t e r grades based on b a l a n c e d s c o r e s were compared t o the l e t t e r grades a s s i g n e d i n i t i a l l y by t h e r e s p o n d e n t . The d i f f e r e n c e s between the two l e t t e r grades were c l a s s i f i e d a c c o r d i n g t o magnitude and d i r e c t i o n o f "change". For example, a change from an i n i t i a l l e t t e r grade of B t o a b a l a n c e d s c o r e l e t t e r grade of C+ was r e c o r d e d as a o n e - l e t t e r grade d e c r e a s e . A change from a C+ t o a b a l a n c e d l e t t e r grade of A was r e c o r d e d as a t w o - l e t t e r grade i n c r e a s e . T a b l e XXIV 72 MATHEMATICS SOCIAL STUDIES SUBJECT CRITICAL R Q CRITICAL # s n VALUE s n VALUE 001 .941 24 .537 . 976 24 . 537 002 .873 34 <. 478 . 957 34 <. 478 003 . 955 18 . 625 . 985 18 . 625 004 .983 24 .537 N/A N/A N/A 005 .919 33 < . 478 . 984 33 <. 478 006 .954 14 .716 N/A N/A N/A 007 .953 27 . 505 .975 27 . 505 008 .975 14 .716 .970 14 . 716 009 N/A N/A N/A . 968 21 . 576 010 . 785 25 . 526 . 903 25 . 526 O i l .992 24 .537 . 943 33 <. 478 012 .746 20 .591 . 857 20 . 591 013 .987 16 . 6 66 . 981 36 <. 478 014 N/A N/A N/A . 980 24 . 537 015 .984 17 .645 .939 17 . 654 016 N/A N/A N/A . 949 18 . 625 017 .937 16 . 666 . 797 16 . 666 018 .992 28 . 496 . 962 28 . 496 019 .956 13 . 745 .995 13 . 745 020 . 955 30 .478 . 960 32 <. 478 021 .977 24 . 537 N/A N/A N/A 022 .979 33 < . 478 . 876 32 <. 478 023 . 988 32 < . 478 . 957 31 <. 478 024 .821 16 .666 . 900 16 . 666 025 .931 23 . 549 . 947 23 . 549 026 .984 29 . 487 .976 27 . 505 027 .929 16 .666 . 918 16 . 666 028 . 986 24 .537 . 902 24 . 537 029 .986 28 . 496 . 869 28 . 496 030 .972 25 . 526 . 960 25 . 526 031 .999 16 . 666 . 875 16 . 666 T o t a l 643 671 Ta b l e X X I I - Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s ( R Q ) R e l a t i n g Raw Score and B a l a n c e d Score T o t a l s F o r (a) Mathematics and (b) S o c i a l S t u d i e s 73 Frequency Score I n t e r v a l Math S.S. 0.970 - 1.000 14 9 0. 940 - 0.969 6 9 0.910 - 0.939 4 2 0.880 - 0.909 0 3 0.850 - 0.879 1 4 0. 820 - 0.849 1 0 0. 790 - 0.819 0 1 0. 760 - 0.789 1 0 0.730 - 0.759 1 0 0 . 700 - 0.729 0 0 T o t a l 28 28 T a b l e X X I I I - D i s t r i b u t i o n of Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s shows the f r e q u e n c y , the magnitude, and the d i r e c t i o n o f the l e t t e r grade changes. For a n a l y s i s purposes the range of l e t t e r grades (A t o E) was d i v i d e d i n t o t h r e e c a t e g o r i e s i n accordance w i t h t h e M i n i s t r y o f E d u c a t i o n l e t t e r grade i n t e r p r e t a t i o n s : Above Average, Average, and Below Average. L e t t e r grades A and B were grouped i n t o the Above Average c a t e g o r y ; l e t t e r grades C+, C, and C- were grouped i n t o the Average c a t e g o r y ; and l e t t e r grades of D and E were grouped i n t o the Below Average c a t e g o r y . 7 4 One l e t t e r grade i n c r e a s e 238 (18. 1%) One l e t t e r grade d e c r e a s e 313 (23.8%) Two l e t t e r grade i n c r e a s e 28 (2.1%) Two l e t t e r grade d e c r e a s e 23 (1.8%) Three l e t t e r grade i n c r e a s e . . . . 4 (0.3%) Three l e t t e r grade d e c r e a s e . . . . 1 (0.1%) No change i n l e t t e r grade 707 (53. 8%) T o t a l 1314 (100 . 0% ) T a b l e XXIV - D i s t r i b u t i o n o f l e t t e r grade changes The d i f f e r e n c e s i n l e t t e r grade were a l s o a n a l y z e d t o deter m i n e how many "changes" would r e s u l t i n a s h i f t from one c a t e g o r y t o a n o t h e r . T a b l e XXV shows the f r e q u e n c y of c a t e g o r y s h i f t s t h a t r e s u l t e d when t h e o r i g i n a l l e t t e r grades were r e p l a c e d by l e t t e r grades based on s t a t i s t i c a l l y b a l a n c e d s c o r e s . Above Average t o Average 112 (8.5%) Average t o Above Average 5 4 (4.1%) Below Average t o Average 4 4 (3.3%) Average t o Below Average 29 (2.2%) L e t t e r grade change b u t no change i n c a t e g o r y 368 (28 . 1%) No l e t t e r g r a d e / c a t e g o r y change . . 707 (53.8%) T o t a l 1314 (100. 0% ) T a b l e XXV - Frequency of c a t e g o r y s h i f t 75 The r e s u l t s o f the l e t t e r grade comparison, l i s t e d i n T a b l e s X X I I I , XXIV, and XXV, i n d i c a t e d t h a t 46% of t h e 1,314 grades i n i t i a l l y awarded t o s t u d e n t s were changed when t h e s t u d e n t s were r e g r a d e d on the b a s i s of t h e b a l a n c e d s c o r e t o t a l s . T h i r t y n i n e per c e n t o f t h e r e v i s e d grades (or 18% of a l l grades) a l s o i n v o l v e d c a t e g o r y changes. F i n a l l y , the sample's r e c o r d s h e e t s were a n a l y z e d t o determine how many l e t t e r grades were based on m i s c a l c u l a t e d raw s c o r e t o t a l s . T a b l e XXVI d i s p l a y s t h e f r e q u e n c y o f e r r o r s , as w e l l as the d i r e c t i o n and magnitude o f the l e t t e r grade c o r r e c t i o n . T a b l e XXVII shows the f r e q u e n c y o f c a t e g o r y changes t h a t r e s u l t e d when i n c o r r e c t raw s c o r e t o t a l s were r e c a l c u l a t e d and t h e c o r r e s p o n d i n g l e t t e r grades r e v i s e d . One l e t t e r grade i n c r e a s e 2 (0.2%) One l e t t e r grade d e c r e a s e 6 (0.4%) Two l e t t e r grade i n c r e a s e 0 (0.0%) Two l e t t e r grade d e c r e a s e 2 (0.2%) C o r r e c t i o n s t h a t d i d not a f f e c t l e t t e r grade 53 (4.0%) No e r r o r s 1251 ( 95. 2%) T o t a l 1314 (100 . 0% ) Table XXVI - Frequency of l e t t e r grade changes r e s u l t i n g from a r i t h m e t i c e r r o r s 76 Above Average to Average 4 (0.3%) Average to Above Average 1 (0.1%) Below Average t o Average 0 (0.0%) Average to Below Average 2 (0.2%) L e t t e r grade change but no change i n category 3 (0.2%) No l e t t e r grade change 1304 (99 . 2%) T o t a l 1314 (100.0% ) Table XXVII - Frequency of category s h i f t r e s u l t i n g from l e t t e r grade r e c a l c u l a t i o n The a n a l y s i s of the r e c o r d sheets r e v e a l e d t h a t about 5% of the 1,314 student grades assi g n e d by the sample were based on i n c o r r e c t l y computed raw score, t o t a l s . However, o n l y about 1% of the students r e c e i v e d i n c o r r e c t l e t t e r grades as a r e s u l t of the c a l c u l a t i o n e r r o r s . Of the ten improperly assigned grades, seven r e s u l t e d i n an i n c o r r e c t category placement as w e l l . T h i s concludes the r e s u l t s s e c t i o n of t h i s study. Chapter f i v e w i l l c o n t a i n a s h o r t summary, an a n a l y s i s of the data i n response to the q u e s t i o n s of i n t e r e s t , an o u t l i n e of the l i m i t a t i o n s , and f i n a l l y , the p o s s i b i l i t i e s f o r f u r t h e r r e s e a r c h . 77 CHAPTER 5 C o n c l u s i o n s and Recommendations For F u r t h e r Study Summary The g e n e r a l problem t h a t t h i s i n v e s t i g a t i v e s t u d y c o n s i d e r e d was t o determine i f t e a c h e r s ' g r a d i n g methods c o u l d be r e v i s e d so as t o i n c r e a s e the r e l i a b i l i t y and f a i r n e s s of the l e t t e r grades t h e y award t o s t u d e n t s . I t was d i f f i c u l t t o a d e q u a t e l y d e a l w i t h t h i s problem i n one s t u d y . I t would be al m o s t i m p o s s i b l e t o a n a l y z e each o f t h e many components comprised i n the mark i n g and g r a d i n g p r o c e s s e s . C o n s e q u e n t l y , the scope o f t h i s s t u d y was r e s t r i c t e d t o f o c u s o n l y on the e f f e c t s of a p p l y i n g s t a t i s t i c a l b a l a n c i n g t o raw achievement s c o r e s p r i o r t o the c a l c u l a t i o n of s t u d e n t term t o t a l s . From t h i s p e r s p e c t i v e , t h e f o l l o w i n g q u e s t i o n s o f i n t e r e s t e v o l v e d : 1. I s t h e r e a s i g n i f i c a n t d i f f e r e n c e between the r a n k i n g s of the ag g r e g a t e d s c o r e s c a l c u l a t e d by t e a c h e r s u s i n g i n t u i t i v e methods and t h e r a n k i n g s o f ag g r e g a t e d b a l a n c e d s c o r e s ? 2. Are the term t o t a l s , from which the l e t t e r grades a r e d e r i v e d , a g g r e g a t e d and w e i g h t e d r e l i a b l y ? 3. Are t e a c h e r s ' grades r e l i a b l e and f a i r ? 4. Would the r e v i s e d r e s u l t s be of such a magnitude as t o 78 j u s t i f y t he time and expense r e q u i r e d t o r e - e d u c a t e t e a c h e r s ? The raw s c o r e s s u p p l i e d by the sample i n the f i r s t s e t of d a t a were c o n v e r t e d t o T - s c o r e s , w e i g h t e d , and c o l l a t e d t o o b t a i n a " s t a t i s t i c a l l y b a l a n c e d " t o t a l s c o r e f o r each s t u d e n t l i s t e d on the r e c o r d s h e e t s . The w e i g h t i n g s f o r t h e b a l a n c e d d a t a were made i d e n t i c a l t o tho s e used i n i t i a l l y by the r e s p o n d e n t s . In t h e second s e t o f d a t a , the s u b j e c t s were asked t o use t h e b a l a n c e d s c o r e s t o r e g r a d e t h e i r s t u d e n t s , and a l s o t o complete the q u e s t i o n n a i r e l i s t e d i n Appendix A. Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s were c a l c u l a t e d t o compare the r a n k i n g s o f the raw s c o r e t o t a l s w i t h the r a n k i n g s of t he s t a t i s t i c a l l y b a l a n c e d t o t a l s . The c r i t i c a l v a l u e s of Spearman's Rank C o r r e l a t i o n C o e f f i c i e n t f o r t e s t i n g the n u l l h y p o t h e s i s o f no c o r r e l a t i o n w i t h a t w o - t a i l e d t e s t ( a l p h a = .01) were o b t a i n e d from G l a s s and S t a n l e y (1970, p. 539) w i t h the a p p r o p r i a t e n v a l u e s c o r r e s p o n d i n g t o the c l a s s s i z e s . The l e t t e r grades based on raw s c o r e s were compared t o t h o s e based on b a l a n c e d s c o r e s t o dete r m i n e i f s i g n i f i c a n t l y d i f f e r e n t l e t t e r grades would r e s u l t when raw s c o r e s were s t a t i s t i c a l l y b a l a n c e d p r i o r t o b e i n g a g g r e g a t e d f o r g r a d i n g purposes. The r e s u l t s o f the comparison were r e c o r d e d i n terms of t h e change i n l e t t e r grade t h a t o c c u r r e d when the b a l a n c e d s c o r e t o t a l s r e p l a c e d raw s c o r e t o t a l s . The grade changes were a n a l y z e d t o dete r m i n e how many s t u d e n t s would have r e c e i v e d a c a t e g o r y change (see page 73) i f t h e i r o r i g i n a l grades were r e p l a c e d w i t h b a l a n c e d s c o r e g r a d e s . 79 T a b l e s XXIV and XXV d i s p l a y t h e r e s u l t s of t h e s e comparisons and a n a l y s e s . The raw s c o r e grades were a l s o a n a l y z e d t o det e r m i n e how many s t u d e n t s r e c e i v e d raw s c o r e grades based on m i s c a l c u l a t e d term t o t a l s . The numbers of c a t e g o r y s h i f t s t h a t r e s u l t e d from m i s c a l c u l a t e d grades were a l s o r e c o r d e d . The r e s u l t s of t h e s e a n a l y s e s a r e t a b u l a t e d i n T a b l e s XXVI and XXVII. The q u e s t i o n n a i r e was used t o c o l l e c t d a t a p e r t a i n i n g t o the sample's demographic v a r i a b l e s , the sample's a t t i t u d e and p e r c e p t i o n o f g r a d i n g and r e p o r t i n g , and t h e sample's g r a d i n g t e c h n i q u e s . The r e s u l t s a r e t a b u l a t e d i n T a b l e s IV t h r o u g h XXI i n c h a p t e r f o u r . R e s e a r c h Q u e s t i o n 1 In response t o the f i r s t r e s e a r c h q u e s t i o n r e g a r d i n g the d i f f e r e n c e between t h e r a n k i n g s o f t h e raw s c o r e t o t a l s and the r a n k i n g s o f b a l a n c e d s c o r e t o t a l s , Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s were c a l c u l a t e d f o r each o f t h e s u b j e c t ' s two r e c o r d s h e e t s . The r e s u l t s i n d i c a t e d t h a t t h e r a n k i n g s of the raw s c o r e t o t a l s were s i g n i f i c a n t l y and p o s i t i v e l y c o r r e l a t e d w i t h t h e r a n k i n g s o f the b a l a n c e d s c o r e t o t a l s (see page 7 1 ) . The v a l u e s o f the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s ranged from +.746 t o +.999 (see T a b l e X X I I I ) . From t h e s e r e s u l t s i t can be c o n c l u d e d t h a t t h e r e i s a s i g n i f i c a n t r e l a t i o n s h i p between t h e r a n k i n g s o f t h e raw s c o r e t o t a l s c a l c u l a t e d by the s u b j e c t s and the r a n k i n g s o f 8 0 the balanced score t o t a l s . I t can a l s o be concluded t h a t the methods used by the s u b j e c t s t o aggregate raw scores f o r r e p o r t i n g purposes are reasonably r e l i a b l e when compared with the s t a t i s t i c a l b a l a n c i n g methods advocated by many authors. Since norm-referenced g r a d i n g , the method used i n B r i t i s h Columbia s c h o o l s , bases l e t t e r grades on the rank order of the aggregate t o t a l s , s i g n i f i c a n t p o s i t i v e c o r r e l a t i o n s between the raw score t o t a l s and the balanced t o t a l s suggest t h e r e w i l l be a s i g n i f i c a n t and p o s i t i v e c o r r e l a t i o n between the l e t t e r grades r e s u l t i n g from each method of combining s c o r e s . In other words, the l e t t e r grade a student r e c e i v e s under the raw score method of combining scores should, i n most cases, be s i m i l a r to the l e t t e r grade he would r e c e i v e when the balanced score technique i s a p p l i e d . Both methods should y i e l d s i m i l a r l y d i s t r i b u t e d l e t t e r grades f o r a gi v e n c l a s s . I t can be concluded, then, t h a t on the whole, the students should r e c e i v e reasonably r e l i a b l e grades (see page 2). However, of the 1,314 students i n v o l v e d i n t h i s , study, 46% r e c e i v e d a change i n l e t t e r grade when t h e i r grades were based on balanced score t o t a l s r a t h e r than raw score t o t a l s (Table XXIV). T h i s would suggest t h a t , f o r many students, the assignment of grades was u n r e l i a b l e and was based on, or i n f l u e n c e d by, f a c t o r s other than the t o t a l score r a n k i n g s . Since s e v e r a l respondents i n q u i r e d about c o n v e r t i n g the T-scores on the r e c o r d sheets t o per cent s c o r e s , i t i s p o s s i b l e t h a t the va l u e s of the converted per cent scores were 81 i n f l u e n c i n g f a c t o r s f o r some s u b j e c t s d u r i n g t h e r e g r a d i n g p r o c e s s . The lower per c e n t s c o r e s (based on an a r b i t r a r y maximum of 275) r e s u l t i n g from th e T - s c o r e s would a l s o e x p l a i n why more l e t t e r grades d e c r e a s e d t h a n i n c r e a s e d (see T a b l e X X I V ) . I n r e f e r e n c e t o the consequences of u n r e l i a b l e g r a d i n g p r a c t i s e s (see page 2 ) , the l e t t e r grade changes suggest t h a t 46% of the s t u d e n t s i n t h i s s t u d y r e c e i v e d i n c o n s i s t e n t and m i s l e a d i n g i n f o r m a t i o n t h a t c o u l d p o t e n t i a l l y l e a d o t h e r s t o f o r m u l a t e erroneous d e c i s i o n s and c o n c l u s i o n s . I n a d d i t i o n , 18% of the 1,314 s t u d e n t s r e c e i v e d l e t t e r grade changes t h a t r e s u l t e d i n c a t e g o r y s h i f t s ( T a b l e XXV). These s t u d e n t s a r e most l i k e l y t o s u f f e r t h e consequences of e rroneous d e c i s i o n s as academic programs te n d g e n e r a l l y t o c a t e r t o the below average, average, and above average a b i l i t y groups. For example, s t u d e n t s who have average a b i l i t y c o u l d be e r r o n e o u s l y p l a c e d i n a s p e c i a l program d e s i g n e d f o r "below average a c h i e v e r s " o r t h e y c o u l d be e r r o n e o u s l y e n r o l l e d i n an e n r i c h e d program f o r s t u d e n t s w i t h above average a b i l i t y . In e i t h e r c a s e , t h e average s t u d e n t ' s academic needs may not be s a t i s f a c t o r i l y met. In l i g h t of t h e c u r r e n t l i t e r a t u r e , t h e r e s u l t s of Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s were somewhat s u r p r i s i n g . Why a r e the r a n k i n g s of the raw s c o r e t o t a l s (as c a l c u l a t e d by the s u b j e c t s ) so s t r o n g l y s i g n i f i c a n t and p o s i t i v e l y c o r r e l a t e d w i t h t h e r a n k i n g s o f t h e b a l a n c e d s c o r e 82 t o t a l s i n t h i s s t u d y ? Many a u t h o r s have s u p p o r t e d t h e n o t i o n t h a t raw s c o r e s can o n l y be c o l l a t e d and we i g h t e d r e l i a b l y when t h e y have been b a l a n c e d t o have e q u a l means and e q u a l s t a n d a r d d e v i a t i o n s (see page 27). For example, Ahmann and Gl o c k (1981) s t a t e , " I f we hope t o m a i n t a i n t h e w e i g h t i n g scheme o r i g i n a l l y chosen, we must t a k e i n t o c o n s i d e r a t i o n the d i f f e r e n c e s i n v a r i a b i l i t y . A f a i l u r e t o do t h i s w i l l r e s u l t i n i n e q u i t i e s " (p. 426). But t h e s e and many o t h e r a u t h o r s have f a i l e d t o mention t h e imp o r t a n c e o f t h e rank c o r r e l a t i o n f o r "between" s c o r e s . From the r e s u l t s o f t h i s s t u d y , i t appears t h a t t h e rank c o r r e l a t i o n c o e f f i c i e n t f o r between s c o r e s can i n f l u e n c e the rank o r d e r o f the ag g r e g a t e d s c o r e s . Indeed, the " i n e q u i t i e s " t h a t r e s u l t when a g g r e g a t i n g raw s c o r e s w i t h l a r g e d i f f e r e n c e s i n s t a n d a r d d e v i a t i o n s can be reduced o r e l i m i n a t e d w i t h a l a r g e p o s i t i v e rank c o r r e l a t i o n f o r between s c o r e s . However, t h e s e i n e q u i t i e s can a l s o be a m p l i f i e d by a l a r g e n e g a t i v e rank c o r r e l a t i o n . Such i s t h e case w i t h t h e example t a k e n from G r o n l u n d (1974) on page 8 of t h i s s t u d y . Gronlund has used t h e extreme s i t u a t i o n (as have o t h e r a u t h o r s ) o f l a r g e v a r i a b i l i t y and a v e r y l a r g e n e g a t i v e between s c o r e s rank c o r r e l a t i o n (r - = -1) t o emphasize the e f f e c t s o f combining s c o r e s w i t h d i f f e r e n t s t a n d a r d d e v i a t i o n s . I n t h e i r examples, s t u d e n t A s c o r e s v e r y h i g h on the f i r s t assignment and v e r y low on t h e second w h i l e s t u d e n t B s c o r e s low on the f i r s t and h i g h on the second. B u t, t h i s i s seldom the case w i t h s c o r e s t a k e n from t e a c h e r s ' r e c o r d s h e e t s . More commonly, s t u d e n t s t e n d t o m a i n t a i n a r e l a t i v e l y 83 s t a b l e rank p o s i t i o n w i t h i n the c l a s s . Above average s t u d e n t s t e n d t o o b t a i n the above average s c o r e s on most assignments w h i l e t h e below average s t u d e n t s t e n d t o a c h i e v e the lo w e r s c o r e s . T h e r e f o r e , the r a n k i n g s f o r between raw s c o r e s would more l i k e l y be p o s i t i v e . I t i s i n t e r e s t i n g t o note t h a t e l e m e n t a r y t e a c h e r s q u e s t i o n assignments t h a t r e s u l t i n "u n u s u a l " c l a s s r a n k i n g s . F o r example, some s u b j e c t s q u e s t i o n e d t h e v a l i d i t y of t h e f i f t h assignment i n t h e f i c t i t i o u s d a t a (see Appendix A) because the r a n k i n g s on t h a t assignment f a i l e d t o f i t t h e r a n k i n g p a t t e r n e s t a b l i s h e d by the o t h e r f o u r a s s i g n m e n t s . S t u d e n t s who p l a c e d h i g h i n t h e r a n k i n g on t h e f i r s t f o u r assignments s h o u l d p l a c e r e l a t i v e l y h i g h i n the r a n k i n g o f t h e f i f t h assignment a l s o . A l t h o u g h a rank c o r r e l a t i o n o f r =+1 would be e x t r e m e l y d i f f i c u l t t o s 1 o b t a i n i n a c l a s s o f twenty o r t h i r t y s t u d e n t s , the s t a b i l i t y of rank p o s i t i o n s u g g e s t s t h e r e might be a r e a s o n a b l y s t r o n g p o s i t i v e rank c o r r e l a t i o n f o r between s c o r e s i n many c l a s s r e c o r d s h e e t s . C o n s i d e r i n g the r e s u l t s o f t h i s s t u d y and the d i f f e r e n t s t a n d a r d d e v i a t i o n v a l u e s i n the raw s c o r e d a t a , the l i k e l y p r e s e nce o f p o s i t i v e rank c o r r e l a t i o n s f o r between s c o r e s may have reduced the e f f e c t of v a r i a n c e on the sample's aggregate s c o r e s . To demonstrate the e f f e c t t h a t the between s c o r e rank c o r r e l a t i o n has on aggre g a t e t o t a l s , c o n s i d e r the f o l l o w i n g examples. I n each c a s e , t h e assignments a r e t o be w e i g h t e d e q u a l l y . The raw s c o r e t o t a l s and t h e b a l a n c e d s c o r e t o t a l s a re c a l c u l a t e d by summing the a p p r o p r i a t e s c o r e s . F o r each 84 example, t h e c r i t i c a l v a l u e (alpha=.10; n=6) o f the ranked c o r r e l a t i o n c o e f f i c i e n t f o r raw s c o r e t o t a l s w i t h b a l a n c e d s c o r e t o t a l s i s r = .829. The b r a c k e t t e d d a t a a r e t h e s b a l a n c e d s c o r e s . Assignment Student 1 2 T o t a l A 100 [64. 3] 40 [64 .8] 140 [129 .1] B 83 [59. 1] 31 [57 .3] 114 [116.4] C 58 [51. 5] 26 [53 .1] 84 [104.6] D 31 [43. 3] 15 [43 .9] 46 [ 87.2] E 25 [41. 5] 12 [41 .4] 37 [ 82.9] F X m S.D 21 53 . 32. [40. 9 3] 10 22. 11. [39 3 9 .7] 31 [ 80.0] r s (between) = +1 r s ( t o t a l ) T a b l e X X V I I I - E f f e c t of r f o r between s c o r e s on t o t a l s c o r e r when r fbetween) =+1 Tabl e X X V I I I d i s p l a y s t h e r e s u l t i n g rank c o r r e l a t i o n c o e f f i c i e n t ( r =+1) f o r raw s c o r e t o t a l s w i t h b a l a n c e d s c o r e s t o t a l s when the between s c o r e s rank c o r r e l a t i o n c o e f f i c i e n t i s v e r y l a r g e ( r =+1). When t h e s c o r e s o f the second assignment are r e a r r a n g e d t o become rank o r d e r e d as 2,3,1,5,6,4 (as i n Tabl e X X I X ) , the between s c o r e rank c o r r e l a t i o n c o e f f i c i e n t becomes +.657 w h i l e the rank c o r r e l a t i o n c o e f f i c i e n t f o r aggregate raw s c o r e s w i t h a g g r e g a t e b a l a n c e d s c o r e s becomes + . 942 . 85 Assignment ;udent 1 2 T o t a l A 100 [64. 3] 31 [57. 3] 131 [121.6] B 83 [59. 1] 26 [53. 1] 109 [112.2] C 58 [51. 5] 40 [64. 8] 98 [116.3] D 31 [43. 3] 12 [41. 4] 43 [ 84.7] E 25 [41. 5] 10 [39. 7] 35 [ 81.2] F 21 [40. 3] 15 [43. 9] 36 [ 84.2] X •m 53 22 . 3 S.D. 32. 9 11. 9 r s . 657 r = s . 942 Tab l e XXIX - E f f e c t of r f o r between s c o r e s on t o t a l s c o r e r when r (between) =.657 I f t h e o r d e r o f the second assignment i s a l t e r e d t o 3,1,5,6,4,2 the r f o r between s c o r e s and the r f o r t o t a l s s s c o r e s become +.14 and +.83 r e p e c t i v e l y . F i n a l l y , i f the ranks a re changed t o 6,5,4,3,2,1 the n the r g f o r between s c o r e s and r f o r t o t a l s c o r e s become -1 and +.31. s I t i s i n t e r e s t i n g t o note t h a t , w i t h t h e s e examples, o n l y the l a s t has a rank c o r r e l a t i o n c o e f f i c i e n t f o r raw s c o r e t o t a l s and b a l a n c e d s c o r e t o t a l s t h a t i s not s i g n i f i c a n t ( i e . < .829) a t the alpha=.10 l e v e l o f s i g n i f i c a n c e (n=6). These examples show how changes i n t h e rank c o r r e l a t i o n c o e f f i c i e n t f o r between s c o r e s a f f e c t the rank c o r r e l a t i o n c o e f f i c i e n t f o r raw s c o r e t o t a l s and b a l a n c e d s c o r e t o t a l s . C o n s i d e r i n g a h i g h m a j o r i t y o f t h e sample i n t h i s s t u d y 86 compute t o t a l s by summing raw s c o r e s (see Ta b l e X V I I I page 66) and c o n s i d e r i n g c l a s s assignments a r e e x p e c t e d t o c o r r e l a t e p o s i t i v e l y w i t h each o t h e r , i t becomes more apparent why t h e r a n k i n g o f t h e raw s c o r e t o t a l s c o r r e l a t e d h i g h l y w i t h the r a n k i n g of t h e b a l a n c e d s c o r e t o t a l s i n t h i s s t u d y , i n s p i t e of t h e many l a r g e d i f f e r e n c e s i n raw s c o r e v a r i a b i l i t y . R e s e a r c h Q u e s t i o n 2 In r e s ponse t o the second r e s e a r c h q u e s t i o n r e g a r d i n g t h e r e l i a b i l i t y o f t h e a g g r e g a t i o n and w e i g h t i n g of raw s c o r e s f o r g r a d i n g p u r p o s e s , the sample's r e c o r d s h e e t s and t h e q u e s t i o n n a i r e responses p e r t a i n i n g t o g r a d i n g t e c h n i q u e s were a n a l y z e d . The a n a l y s i s r e v e a l e d a h i g h degree o f s i m i l a r i t y between the a g g r e g a t i o n methods e v i d e n t on t h e r e c o r d s h e e t s and the a g g r e g a t i o n t e c h n i q u e s d e s c r i b e d on the q u e s t i o n n a i r e . Based on t h e s e r e s u l t s and on t h e rank c o r r e l a t i o n c o e f f i c i e n t s o b t a i n e d (as d i s c u s s e d i n Res e a r c h Q u e s t i o n 1 ) , i t can be c o n c l u d e d t h a t t h i s sample o f s u b j e c t s c o l l a t e d raw s c o r e s i n a r e a s o n a b l y r e l i a b l e manner. The a n a l y s i s o f t h e d a t a a l s o i n d i c a t e d t h a t most res p o n d e n t s d i d not attempt t o a p p l y w e i g h t i n g f a c t o r s t o raw s c o r e s . As i n d i c a t e d p r e v i o u s l y i n t h i s s t u d y (page 7 ) , w e i g h t i n g raw s c o r e s and how much t o weight i s a m a t t e r f o r p r o f e s s i o n a l judgement. However, many s u b j e c t s seemed o b l i v i o u s t o t h e f a c t t h a t by not w e i g h t i n g , s m a l l and r e l a t i v e l y i n s i g n i f i c a n t a s s ignments may e a s i l y c o n t r i b u t e as 87 much o r more t o an aggr e g a t e d s c o r e as does a major t e s t o r p r o j e c t . Most o f t h e re s p o n d e n t s i n t h i s s t u d y d i d not a p p l y w e i g h t s and t h e r e b y a l l o w e d each assignment t o c o n t r i b u t e i t s " n a t u r a l w e i g h t " toward t h e a g g r e g a t e d s c o r e . Many of t h o s e who d i d attempt t o we i g h t the raw s c o r e s c o n s i d e r e d an assignment's t o t a l p o s s i b l e mark t o be t h e w e i g h t i n g f a c t o r . F o r example, an assignment t h a t has a t o t a l p o s s i b l e s c o r e of 100 was thought by many t o c o n t r i b u t e t w i c e as much t o an aggreg a t e s c o r e as would an assignment t h a t has a t o t a l p o s s i b l e s c o r e o f 50. The example on page 9 o f t h i s s t u d y demonstrates t h e i n a p p r o p r i a t e n e s s o f t h i s method as a means t o w e i g h t raw s c o r e s . Assignments w i t h a l a r g e r t o t a l p o s s i b l e v a l u e may p o s s i b l y c o n t r i b u t e l e s s toward an aggregate s c o r e t h a n does an assignment w i t h a s m a l l e r t o t a l p o s s i b l e v a l u e . The key f a c t o r s i n r e l i a b l e w e i g h t i n g a r e t h e s t a n d a r d d e v i a t i o n v a l u e s r a t h e r t h a n the t o t a l p o s s i b l e s c o r e s (see page 10 ) . T h i s n o t i o n was s u p p o r t e d by such a u t h o r s as Ahmann and G l o c k (see page 6 ) , E b e l (see page 2 7 ) , and Gronlund (see page 2 8 ) . I n l i g h t o f t h e s e f a c t s , t h e r e s u l t s o f t h i s s t u d y would suggest t h a t the s u b j e c t s ' w e i g h t i n g t e c h n i q u e s a r e l e s s t h a n r e l i a b l e . U n r e l i a b l e w e i g h t i n g methods may have had a p r o f o u n d e f f e c t on the l e t t e r grades r e p o r t e d i n t h i s s t u d y . S i n c e 95% of the s u b j e c t s e i t h e r r e l i e d on n a t u r a l w e i g h t i n g s o r a p p l i e d u n r e l i a b l e w e i g h t i n g methods (see Ta b l e XV), many s t u d e n t s may have r e c e i v e d an ag g r e g a t e s c o r e t h a t was not i n d i c a t i v e o f the emphasis p l a c e d on t h e assignments and c o u r s e o b j e c t i v e s . 88 F o r example, a minor assignment d e a l i n g w i t h o n l y a few c o u r s e c o n c e p t s c o u l d have had more i n f l u e n c e on the aggr e g a t e s c o r e t h a n d i d a comprehensive p r o j e c t t h a t a d d r e s s e d s e v e r a l c o u r s e o b j e c t i v e s . However, i f the raw s c o r e s had been p r o p e r l y b a l a n c e d and we i g h t e d t o r e f l e c t t h e emphasis o f the c o u r s e , t h e n many s t u d e n t s i n t h i s s t u d y would have l i k e l y r e c e i v e d a d i f f e r e n t and more r e p r e s e n t a t i v e a g g r e g a t e s c o r e . F u r t h e r m o r e , some s t u d e n t s may have a l s o r e c e i v e d d i f f e r e n t l e t t e r grades based on t h e r a n k i n g o f the new ag g r e g a t e s c o r e s . C o n s e q u e n t l y , l e t t e r grades based on b a l a n c e d and a p p r o p r i a t e l y w e i g h t e d raw s c o r e s have g r e a t e r v a l i d i t y and r e l i a b i l i t y as i n d i c a t o r s of t h e s t u d e n t s ' r e l a t i v e achievement. C o n s i d e r i n g t h e p o s s i b l e consequences o f u n r e l i a b l e g r a d i n g p r a c t i s e s (see page 2 ) , the r e s u l t s of t h i s s t u d y suggest t h a t many s t u d e n t s may have r e c e i v e d m i s l e a d i n g and i n c o n s i s t e n t i n f o r m a t i o n i n the form o f u n r e l i a b l e l e t t e r g r a d e s . D e c i s i o n makers, u s i n g t h e s e grades as a s o u r c e o f i n f o r m a t i o n , c o u l d be prone t o making erroneous d e c i s i o n s . For example, t h e l e t t e r grades awarded some s t u d e n t s may i n d i c a t e t h a t the c o u r s e o b j e c t i v e s have been s u c c e s s f u l l y completed when i n f a c t , the g o a l s have not been met. In such i n s t a n c e s , s t u d e n t s c o u l d be promoted t o the nex t c o u r s e o r l e v e l when such a pr o m o t i o n i s not j u s t i f i e d . The o p p o s i t e c o u l d a l s o o c c u r . S t u d e n t s may be r e t a i n e d t o r e p e a t o r r e v i e w t h e c o u r s e o b j e c t i v e s when i n f a c t the m a t e r i a l has been w e l l mastered and such a r e t e n t i o n i s n o t w a r r a n t e d . 8 9 O t h e r s may a l s o be m i s g u i d e d by u n r e l i a b l e g r a d e s . For example, t h o s e who n o r m a l l y use s t u d e n t grades as a means of e v a l u a t i n g c o u r s e s , programs, o r p e d a g o g i c a l t e c h n i q u e s , may implement u n n e c e s s a r y o r i n c o r r e c t m o d i f i c a t i o n s based on the u n r e l i a b l e i n f o r m a t i o n . In s h o r t , l e t t e r grades d e r i v e d from u n r e l i a b l e w e i g h t i n g p r a c t i s e s may be l a c k i n g i n v a l i d i t y and r e l i a b i l i t y and may m i s l e a d o t h e r s t o f o r m u l a t e i n c o r r e c t c o n c l u s i o n s and d e c i s i o n s . Raw s c o r e s s h o u l d be b a l a n c e d and w e i g h t e d i n such a manner as t o make the a g g r e g a t e s c o r e r e p r e s e n t a t i v e o f the emphasis p l a c e d on the c o u r s e c o n t e n t and the v a r i o u s c o u r s e o b j e c t i v e s . Based on the r e s u l t s o f t h i s s t u d y , i t can be c o n c l u d e d t h a t r e l a t i v e l y few t e a c h e r s w e i g h t raw s c o r e s r e l i a b l y . R e s earch Q u e s t i o n 3 In r e s p o n s e t o the t h i r d r e s e a r c h q u e s t i o n p e r t a i n i n g t o the r e l i a b i l i t y and f a i r n e s s o f the sample's g r a d e s , the r e s p o n d e n t s ' raw s c o r e grades and b a l a n c e d s c o r e grades were compared. The r e s u l t s were r e c o r d e d and t a b u l a t e d i n terms of the change t h a t o c c u r r e d t o both l e t t e r grade and c a t e g o r y when b a l a n c e d grades r e p l a c e d raw s c o r e g r a d e s . To a s s e s s f a i r n e s s o f the g r a d e s , t h e methods used t o c a l c u l a t e aggregate s c o r e s f o r s t u d e n t s who m i s s e d r e c e i v i n g a s c o r e on one o r more assignments were a n a l y z e d . These r e s u l t s would i n d i c a t e t h a t , i n some 90 c i r c u m s t a n c e s , s t u d e n t s c o u l d be u n f a i r l y p e n a l i z e d o r rewarded f o r m i s s i n g an assignment. The f o l l o w i n g example demonstrates how a s t u d e n t might be p e n a l i z e d by m i s s i n g an assignment when means and s t a n d a r d d e v i a t i o n s a r e not c o n s i d e r e d . ASSIGNMENT SCORES (%) Student 1 2 3 T o t a l A 50 30 100 60 B 40 28 ab 34 C 30 26 74 43 D 10 20 60 30 Mean 33 26 78 S.D. 17 4 20 T a b l e XXX - Compensation f o r mi s s e d assignment T a b l e XXX shows how s t u d e n t B has dropped t o the t h i r d rank p o s i t i o n on t h e b a s i s of t h e f i r s t two assignment s c o r e s , even though he has c o n s i s t e n t l y ranked second. By m i s s i n g t h e t h i r d assignment w i t h i t s h i g h e r mean s c o r e , s t u d e n t B has been p e n a l i z e d one rank p o s i t i o n . On the o t h e r hand, i f the missed assignment were c a l c u l a t e d by a v e r a g i n g b a l a n c e d s c o r e s ( i e . T - s c o r e s ) , t h e n h i s second p l a c e r a n k i n g would be p r e s e r v e d . The b a l a n c e d s c o r e averages f o r s t u d e n t s A, B, C, and D would be 60, 55, 49, and 37 r e s p e c t i v e l y . C l e a r l y , s t u d e n t B has not been p e n a l i z e d nor rewarded under the 91 b a l a n c e d s c o r e method of compensation. The r e s u l t s o f t h i s s t u d y would suggest t h a t t e a c h e r s " grades may l a c k r e l i a b i l i t y and, when s t u d e n t s have mi s s e d a s s i g n m e n t s , t h e t e c h n i q u e f o r compensation may not always be f a i r . R e s e a r c h Q u e s t i o n 4 In c o n s i d e r i n g the f o u r t h r e s e a r c h q u e s t i o n r e g a r d i n g t h e j u s t i f i c a t i o n of t h e ti m e and expense r e q u i r e d t o r e t r a i n t e a c h e r s , t h e res p o n s e s t o the a t t i t u d i n a l s e c t i o n of the q u e s t i o n n a i r e were a n a l y z e d and the r e s u l t s t a b u l a t e d i n T a b l e s X I I I t h r o u g h X V I I . These r e s u l t s , i n c o n j u n c t i o n w i t h the p r e v i o u s l y d i s c u s s e d r e s u l t s , would i n d i c a t e t h a t i n - s e r v i c e i n s t r u c t i o n and p r e - s e r v i c e t r a i n i n g i n p a r t i c u l a r a s p e c t s o f g r a d i n g and r e p o r t i n g would be j u s t i f i e d f o r many members o f t h e r e s e a r c h sample. Areas of g r e a t e s t need appear t o be tho s e c o n c e r n i n g t h e r e l i a b l e w e i g h t i n g o f raw s c o r e s , the r e l i a b l e a l l o c a t i o n o f l e t t e r g r a d e s , and the r e l i a b l e c a l c u l a t i o n o f compensation s c o r e s f o r s t u d e n t s who have missed a s s i g n m e n t s . C u r r e n t l i t e r a t u r e i n d i c a t e s t h a t many t e a c h e r s a r e uneasy o r d e f e n s i v e c o n c e r n i n g t h e i r grades and g r a d i n g p r o c e d u r e s (see page 5 ) . D u r i n g t h e c o u r s e of t h i s s t u d y , t h i s n o t i o n was v e r i f i e d i n f o r m a l l y on s e v e r a l o c c a s i o n s . Comments, such as " . . . s u r e hope t h e s e marks a r e a l r i g h t " , o r 92 " I ' v e never been t r a i n e d i n t h i s a r e a " , and o t h e r s i m i l a r remarks were f r e q u e n t l y made by the r e s p o n d e n t s . These comments and t h e u n c e r t a i n t y accompanying them would s u g g e s t t h a t t i m e devoted t o p r e - s e r v i c e and i n - s e r v i c e would be w e l l s p e n t . Weaknesses To Be C o n s i d e r e d The framework of the proposed s t u d y was c o n s i d e r e d t o be sound. Many of t h e e x t r a n e o u s f a c t o r s t h a t c o u l d a f f e c t t h e r e s u l t s were c o n t r o l l e d by r a n d o m i z a t i o n wherever p o s s i b l e . A l t h o u g h t h i s p r o j e c t e n c o u n t e r e d few s h o r t c o m i n g s , t h o s e t h a t d i d e x i s t were c o n s i d e r e d t o be minor and not s e r i o u s enough t o a d v e r s e l y a f f e c t the r e s u l t s . One of the most s e r i o u s o b s t a c l e s concerned th e sample. A l t h o u g h of s u f f i c i e n t s i z e f o r an adequate s t u d y , the sample was c o m p r i s e d e n t i r e l y o f v o l u n t e e r s . As a r e s u l t , a b i a s may have been i n t r o d u c e d i n t o the s t u d y . The p o s s i b l e b i a s would r e s u l t from the sample n o t b e i n g t o t a l l y r e p r e s e n t a t i v e of t h e t a r g e t p o p u l a t i o n . Those who p a r t i c i p a t e d may have c o n t r i b u t e d a p a r t i c u l a r b i a s w h i l e t h o s e who d e c l i n e d t o p a r t i c i p a t e may have d e p r i v e d the s t u d y of d a t a unique t o n o n - p a r t i c i p a n t s . For example, t h o s e who a r e p a r t i c u l a r l y d e f e n s i v e about t h e i r g r a d i n g p r o c e d u r e s o r about the r e l i a b i l i t y o f t h e i r grades may not have v o l u n t e e r e d . On the o t h e r hand, perhaps o n l y t h o s e who a r e c o n f i d e n t t h e i r g r a d i n g 93 t e c h n i q u e s a r e d e f e n s i b l e o r t h o s e who t a k e a p a r t i c u l a r i n t e r e s t i n marking and g r a d i n g v o l u n t e e r e d t o become members of t h e sample. Teachers b e l o n g i n g t o t h e s e and o t h e r such groups s h o u l d be i n c l u d e d i n t h e sample s i n c e g e n e r a l i z a t i o n s w i l l be made about the p o p u l a t i o n t o which t h e y b e l o n g . However, w i t h v o l u n t e e r samples, t h e s e v a r i a b l e s a r e d i f f i c u l t t o c o n t r o l . Other weaknesses i n t h i s s t u d y concerned the q u e s t i o n n a i r e t h a t was used t o c o l l e c t d a t a on g r a d i n g t e c h n i q u e s . The q u e s t i o n o f f e r i n g f i c t i t i o u s d a t a t o be c o l l a t e d and ranked posed p a r t i c u l a r problems f o r many o f the r e s p o n d e n t s . Most r e c o g n i z e d , from the i n f o r m a t i o n g i v e n i n the d i r e c t i o n s , t h a t some s o r t o f w e i g h t i n g was r e q u i r e d , even when th e y d i d not c u s t o m a r i l y w e i g h t raw s c o r e s . Some i g n o r e d the d i r e c t i o n s on w e i g h t i n g and a p p l i e d t h e i r r e g u l a r g r a d i n g methods. Others a t t e m p t e d t o accommodate the d i r e c t i o n s by d e v e l o p i n g new and u n f a m i l i a r s t r a t e g i e s t o we i g h t t h e raw s c o r e s i n t h e q u e s t i o n . The f i c t i t i o u s d a t a s e c t i o n posed a n o t h e r weakness, a l t h o u g h l e s s s e r i o u s and u n r e l a t e d t o the one p r e v i o u s l y d i s c u s s e d . An o p p o r t u n i t y f o r t h e re s p o n d e n t s t o c a l c u l a t e an aggreg a t e s c o r e f o r a s t u d e n t who had been absent s h o u l d have been i n c l u d e d i n the f i c t i t i o u s d a t a . T h i s would have p r o v i d e d a t h i r d s o u r c e of i n f o r m a t i o n f o r t h a t p a r t i c u l a r a s p e c t o f g r a d i n g . A l t h o u g h adequate, t h e r e c o r d s h e e t s and the w r i t t e n r e s p o n s e s were t h e o n l y s o u r c e s of d a t a f o r t h i s t o p i c . As a r e s u l t of t h e s e weaknesses, t h e f i c t i t i o u s d a t a 94 s e c t i o n i n t h i s s t u d y s h o u l d be r e v i e w e d and r e v i s e d b e f o r e b e i n g used on o t h e r s i m i l a r r e s e a r c h p r o j e c t s . A n o t h e r minor weakness concerned the b a l a n c e d raw s c o r e t o t a l s t h a t were g i v e n t o t h e res p o n d e n t s t o r e g r a d e . U n l i k e raw s c o r e s , T - s c o r e s do not have a maximum p o s s i b l e s c o r e ; r a t h e r , t h e y a r e c o n t i n u o u s . When t h e b a l a n c e d T - s c o r e s were s u p p l i e d t o the res p o n d e n t s t o be r e g r a d e d , a maximum s c o r e s h o u l d have been i n c l u d e d f o r t h o s e who u t i l i z e per c e n t g r a d i n g t e c h n i q u e s . F o r t h o s e r e s p o n d e n t s who i n q u i r e d , a s c o r e of 275 was p r o v i d e d as t h i s would exceed a l l b a l a n c e d s c o r e t o t a l s and t h e r e f o r e would y i e l d p e r c e n t a g e s l e s s t h a n 100 per c e n t . A l t h o u g h t h i s weakness caused some c o n f u s i o n among the r e s p o n d e n t s , i t was not c o n s i d e r e d s e r i o u s enough t o f l a w the r e s u l t s . R a t h e r , i t may have c o n t r i b u t e d toward a b e t t e r u n d e r s t a n d i n g o f the d i f f i c u l t y t e a c h e r s e x p e r i e n c e when th e y a s s i g n l e t t e r grades t o aggr e g a t e d s c o r e s (see page 79) . F u t u r e R e s e a r c h With t h e i n t r o d u c t i o n o f computers i n t o most s c h o o l s , the o p p o r t u n i t i e s f o r improvement i n t h e measurement, e v a l u a t i o n , g r a d i n g , and r e p o r t i n g of s t u d e n t achievement have expanded d r a m a t i c a l l y and o f f e r many new ar e a s f o r f u t u r e r e s e a r c h . These a r e a s have been d i v i d e d i n t o two main s e c t i o n s t o f a c i l i t a t e d i s c u s s i o n . The f i r s t a r e a o f r e s e a r c h p o s s i b i l i t i e s d e a l s s p e c i f i c a l l y w i t h t h o s e t o p i c s r e l a t e d t o 95 t e a c h e r s ' g r a d i n g p r a c t i s e s . The second a r e a d e a l s w i t h measurement and e v a l u a t i o n o f s t u d e n t achievement on a more g l o b a l s c a l e . W h i l e t h i s s t u d y touched o n l y a few a s p e c t s of t e a c h e r g r a d i n g p r a c t i s e s , i t l e f t many q u e s t i o n s unanswered. A t the same t i m e , i t r a i s e d more q u e s t i o n s t o be r e s o l v e d . Among the q u e s t i o n s t h a t s t i l l need t o be ad d r e s s e d i s the c o n c e r n r e g a r d i n g the a p p r o p r i a t e n e s s o f u s i n g T - s c o r e s i n the c l a s s r o o m s i t u a t i o n . The l i n e a r t r a n s f o r m a t i o n o f raw s c o r e s t o T - s c o r e s f i r s t r e q u i r e s t h a t the raw s c o r e s be c o n v e r t e d t o z s c o r e s . F u r t h e r s t u d i e s s h o u l d be c a r r i e d o u t t o dete r m i n e i f t h e use o f T-sco r e s i s a p p r o p r i a t e f o r w e i g h t i n g and combining raw achievement s c o r e s . Research o p p o r t u n i t i e s e x i s t i n the r e p l i c a t i o n o f t h i s s t u d y . One p o s s i b i l i t y would be t o determine i f l o c a t i o n has an e f f e c t on t e a c h e r s ' g r a d i n g p r a c t i s e s . F o r example, a stud y c o u l d i n v o l v e s a m p l i n g t e a c h e r s from d i f f e r e n t r e g i o n s of the p r o v i n c e t o de t e r m i n e i f d i f f e r e n c e s i n g r a d i n g p r a c t i s e s can be a t t r i b u t e d t o g e o g r a p h i c l o c a t i o n . I t might be argued t h a t t e a c h e r s w i t h s u p e r i o r g r a d i n g p r a c t i s e s a r e those who l i v e i n c l o s e p r o x i m i t y t o u n i v e r s i t i e s . C o m puterized r e c o r d k e e p i n g and mark management i s anot h e r a r e a t h a t s h o u l d a l s o be s t u d i e d . I f t e a c h e r s a r e g i v e n computer programs t h a t w i l l r e l i a b l y b a l a n c e , a g g r e g a t e , and weight raw achievement s c o r e s , would t h e y be w i l l i n g t o a l t e r t h e i r p r e s e n t g r a d i n g p r a c t i s e s f o r the newer computer approach? I f i t can be shown t h a t t e a c h e r s would r e a d i l y 96 a c c e p t t h e new computer g r a d i n g method, th e n s t u d i e s would a l s o have t o be done t o a s c e r t a i n the most e f f e c t i v e and u s e f u l computer program c u r r e n t l y a v a i l a b l e . T h i s s t u d y has p o s s i b l y i d e n t i f i e d a r e l a t i o n s h i p between (a) t h e e f f e c t s d i f f e r e n t s t a n d a r d d e v i a t i o n s have on raw s c o r e t o t a l s and b) the rank c o r r e l a t i o n c o e f f i c i e n t s f o r between s c o r e s . The e f f e c t s o f l a r g e d i f f e r e n c e s i n v a r i a n c e appear t o be d i m i n i s h e d by l a r g e p o s i t i v e rank c o r r e l a t i o n s f o r between s c o r e s . S t u d i e s s h o u l d be done t o d e t e r m i n e i f t h i s r e l a t i o n s h i p a p p l i e s t o a l l s e t s o f raw s c o r e s and, i f so, how the r e l a t i o n s h i p c o u l d b e s t be a p p l i e d t o c l a s s r o o m g r a d i n g p r a c t i s e s . A n other f u t u r e r e s e a r c h p o s s i b i l i t y r e l a t e s t o the rank c o r r e l a t i o n c o e f f i c i e n t s f o r between s c o r e s . S t u d i e s s h o u l d be done t o d e t e r m i n e how w e l l s e t s o f c l a s s r o o m achievement s c o r e s a c t u a l l y c o r r e l a t e . As i n d i c a t e d i n t h i s s t u d y , i t would be e x p e c t e d t h a t t h e more c a p a b l e s t u d e n t would n o r m a l l y a t t a i n the h i g h e r marks w h i l e the l e s s a b l e s t u d e n t would a c h i e v e the lower s c o r e s . The rank p o s i t i o n of each s t u d e n t would t h e r e f o r e be r e l a t i v e l y s t a b l e . R esearch s h o u l d be conducted t o d e t e r m i n e i f t h i s i s a c t u a l l y t r u e f o r most c l a s s e s . The second a r e a f o r f u t u r e r e s e a r c h , t h e g e n e r a l t o p i c of measurement and e v a l u a t i o n o f s t u d e n t achievement, a l s o needs t o be a d d r e s s e d . More s t u d i e s need t o be done t o e x p l o r e the r e l i a b i l i t y of t e a c h e r c o n s t r u c t e d e v a l u a t i o n i n s t r u m e n t s . G r a d i n g p r o c e d u r e s can be " s t a t e o f the a r t " , but u n l e s s t h e 97 i n s t r u m e n t s used t o measure s t u d e n t achievement are r e l i a b l e and v a l i d , s t u d e n t grades w i l l s t i l l l a c k r e l i a b i l i t y , v a l i d i t y and f a i r n e s s . S t u d i e s need t o be u n d e r t a k e n t o answer such q u e s t i o n s as the f o l l o w i n g : 1. Are t e a c h e r c o n s t r u c t e d t e s t s r e l i a b l e and v a l i d ? 2. Are t h e r e ways t o improve t h e s e i n s t r u m e n t s ? 3. Are t h e r e b e t t e r ways f o r t e a c h e r s t o c o n s t r u c t measurement i n s t r u m e n t s ? These a r e q u e s t i o n s t h a t s h o u l d be i n v e s t i g a t e d . Another p o s s i b i l i t y f o r f u t u r e r e s e a r c h i n v o l v e s the medium used t o a c q u i r e measurement d a t a on s t u d e n t achievement. With computers becoming more p o p u l a r i n s c h o o l s , s t u d i e s s h o u l d be performed t o d e t e r m i n e i f the r e l i a b i l i t y of c l a s s r o o m t e s t s can be improved t h r o u g h the use of the computer. With the a v a i l a b i l i t y o f computers t o a s s i s t t e a c h e r s i n e v a l u a t i n g s t u d e n t achievement, i t i s hoped t h a t f u t u r e r e s e a r c h w i l l o f f e r new and more r e l i a b l e methods of m e a s u r i n g , e v a l u a t i n g , g r a d i n g , and r e p o r t i n g s t u d e n t achievement. 98 B i b l i o g r a p h y Ahmann, J . S., & G l o c k , M. D. (1981). E v a l u a t i n g s t u d e n t p r o g r e s s - p r i n c i p l e s of t e s t s and measurements ( 6 t h ed . ) . B o s t o n : A l l y n & Bacon. Cohen, D. (1973). E v a l u a t i o n r e i n t e r p r e t e d . A u s t r a l i a n S c i e n c e Teachers J o u r n a l , 1 9 ( 2 ) , 29-34. Curwin, R. (1978). The grades of w r a t h : Some a l t e r n a t i v e s . L e a r n i n g , 6̂ , 60-64. E b e l , R. L. (1972). E s s e n t i a l s of e d u c a t i o n a l measurement. Englewood C l i f f s : P r e n t i c e - H a l l . E b e l , R. L. (1974). S h a l l we g e t r i d of g r a d e s ? . NCME Measurement i n E d u c a t i o n , 5 ( 4 ) , 1-5. E r l i n g , A. S. (1979). Combining s t u d e n t t e s t s c o r e s r e l i a b l y . E d u c a t i o n a l R e s e a r c h and Methods, 11 (2 ),42-50 . G e n s l e y , J . T. (1969). A new method o f e v a l u a t i o n f o r g i f t e d s t u d e n t s : A d i a r y of l e a r n i n g . G i f t e d C h i l d r e n Q u a r t e r l y , 13, 119-25. G l a s s , G. V., & S t a n l e y , J . C. (1970). S t a t i s t i c a l methods i n e d u c a t i o n and p s y c h o l o g y . Englewood C l i f f s , New J e r s e y : P r e n t i c e - H a l l . Gray, L. R. (1980). E d u c a t i o n a l e v a l u a t i o n and measurement - Competencies f o r a n a l y s i s and a p p l i c a t i o n . Columbus: M e r r i l l . Green, H. A., J o r g e n s e n , A. N., & G e r b e r i c h , J . R. (1962). Measurement and e v a l u a t i o n i n the e l e m e n t a r y s c h o o l . New York: McKay. Gronlund N.E. (1974). Improving M a r k i n g and R e p o r t i n g i n Cl a s s r o o m I n s t r u c t i o n . New York: M a c M i l l a n . 99 G r o n l u n d , N. E. (1981). Measurement and e v a l u a t i o n i n t e a c h i n g (4t h e d . ) . New York: M a c M i l l a n . G u i l f o r d , J . P., & F r u c h t e r , B. (1973). Fundamental s t a t i s t i c s i n p s y c h o l o g y and e d u c a t i o n ( 5 t h e d . ) . New York: M c G r a w - H i l l . H i l l s , J . R. (1981). Measurement and e v a l u a t i o n i n t h e c l a s s r o o m . Columbus: M e r r i l l . H o p kins, K. D., & S t a n l e y , J . C. (1981). E d u c a t i o n a l and p s y c h o l o g i c a l measurement and e v a l u a t i o n ( 6 t h ed.). Englewood C l i f f s , New J e r s e y : P r e n t i c e - H a l l . H o pkins, C. D., & A n t e s , R. L. (1979). C l a s s r o o m t e s t i n g , A t a s c a , I l l i n o i s : Peacock. H o r r o c k s , J . E., & Schoonover, T. I . (1968). Measurement f o r t e a c h e r s . C o l u m b u s : M e r r i l 1 . I s a a c s , G. , & I m r i e , B. (1981). A case f o r p r o f e s s i o n a l judgment when combining marks. Assessment and E v a l u a t i o n i n H i g h e r E d u c a t i o n , j5, 3-25. Johnson, D. M., & Van O s d o l , B. M. (1974). A computer program f o r r e l i a b l e grade assignment. J o u r n a l of E d u c a t i o n a l Data P r o c e s s i n g , 1~L(2) , 11-19. L i e n , A. J . (1976). Measurement and e v a l u a t i o n of l e a r n i n g . Dubuque, Iowa: Brown. L i n d v a l l , C. M. (1967). M e a s u r i n g p u p i l achievement and a p t i t u d e . San F r a n c i s c o : H a r c o u r t , Brace & World Lyman, H. B. (1978). T e s t s c o r e s and what t h e y mean. Englewood C l i f f s , New J e r s e y : P r e n t i c e - H a l l . M a r s h a l l , M. S. (1971). Why grades a r e argued. S c h o o l and S o c i e t y , 99, 350-353. 1 Mehrens, W. A., & Lehmann, I . J . (1984). Measurement and e v a l u a t i o n i n e d u c a t i o n and p s y c h o l o g y ( 3 r d e d . ) . New York: CBS C o l l e g e . M i n i s t r y o f E d u c a t i o n , P r o v i n c e o f B r i t i s h Columbia. (1979). C o n s t r u c t i o n and use of c l a s s r o o m t e s t s . V i c t o r i a , BC: Aut h o r . M i n i s t r y o f E d u c a t i o n , P r o v i n c e o f B r i t i s h Columbia. (1979). G r a d i n g p r a c t i s e s : I s s u e s and a l t e r n a t i v e s . V i c t o r i a , BC: A u t h o r . N e l s o n , C. H. (1970). Measurement and e v a l u a t i o n i n the c l a s s r o o m . New York: M a c M i l l a n . R a t z l a f f , H. C. (1979). A t o E approaches t o g r a d i n g s t u d e n t achievement. V e c t o r , 2 1 ( 1 ) , 22-28. S t e w a r t , W. J . (1975). A m u l t i - d i m e n s i o n a l e v a l u a t i n g - r e p o r t i n g system i n t h e e l e m e n t a r y s c h o o l . Reading Improvement, 12, 174-76. T e n b r i n k , T. D. (1974). E v a l u a t i o n - A p r a c t i c a l g uide f o r t e a c h e r s . New York: M c G r a w - H i l l . Townsend, E. A., & Burke, P. J . (1975). U s i n g s t a t i s t i c s i n c l a s s r o o m i n s t r u c t i o n . New York: M a c M i l l a n . Wick, J . W. (1973). E d u c a t i o n a l measurement - Where a r e we g o i n g and how w i l l we know when we g e t t h e r e . Columbus M e r r i 1 1 . 101 Appendix A The f o l l o w i n g q u e s t i o n n a i r e was used t o c o l l e c t i n f o r m a t i o n on the s u b j e c t s ' demographic v a r i a b l e s , on t h e i r a t t i t u d e s and p e r s p e c t i v e s toward g r a d i n g , and on t h e i r g r a d i n g p r a c t i s e s and t e c h n i q u e s . I t was d i s t r i b u t e d t o each s u b j e c t a t t h e end o f t h e d a t a c o l l e c t i o n p e r i o d . 102 Respondent # GRADING TECHNIQUES SURVEY S e c t i o n I - Demographic I n f o r m a t i o n DIRECTIONS: C i r c l e t he l e t t e r t h a t c o r r e s p o n d s t o the c o r r e c t o r most a p p r o p r i a t e r e s p o n s e . 1. How many y e a r s t e a c h i n g e x p e r i e n c e do you have? (combine b o t h p u b l i c and p r i v a t e s c h o o l e x p e r i e n c e ) a. 0-5 b. 6-10 c. 11-15 d. 16-20 e. more than 2 0 2. What i s your t e a c h i n g assignment? a. f u l l t i me b. p a r t time I f " p a r t t i m e " , d e s c r i b e your t e a c h i n g assignment ( p e r c e n t a g e , s u b j e c t s t a u g h t , e t c . ) 3. How many S t a t i s t i c s c o u r s e s o r Measurement and E v a l u a t i o n c o u r s e s have you completed? ( d i s r e g a r d u n i t v a l u e ) a. none b. one c. two d. t h r e e e. more th a n t h r e e 4. How many y e a r s of u n i v e r s i t y have you completed? a. two b. t h r e e c. f o u r d. completed B a c h e l o r o f E d u c a t i o n degree e. completed MA o r MEd degree f . o t h e r ( e x p l a i n : ) 5. Which i s your age c a t e g o r y ? a. l e s s t h a n 25 y e a r s b. 25-30 y e a r s c. 31-40 y e a r s d. 41-50 y e a r s e. 51-60 y e a r s f. g r e a t e r t h a n 60 y e a r s S e c t i o n I I - G r a d i n g and R e p o r t i n g DIRECTIONS: You a r e t o e x p r e s s , on a f i v e - p o i n t s c a l e , t h e e x t e n t o f agreement between t h e f e e l i n g e x p r e s s e d i n each s t a t e m e n t and your own p e r s o n a l f e e l i n g . The f i v e p o i n t s a r e : S t r o n g l y D i s a g r e e (SD), D i s a g r e e (D), Undecided (U), Agree ( A ) , S t r o n g l y Agree (SA). You a r e t o e n c i r c l e t h e l e t t e r ( s ) which b e s t i n d i c a t e s how c l o s e l y you agree o r d i s a g r e e w i t h the f e e l i n g e x p r e s s e d i n each s t a t e m e n t . 1. My c o l l e a g u e s a t t h i s s c h o o l t a k e r e p o r t c a r d marks v e r y s e r i o u s l y . SD D U A SA 103 2. L e t t e r grades a r e an e f f e c t i v e method of i n f o r m i n g p a r e n t s o f t h e i r c h i l d ' s p r o g r e s s and achievement. SD D U A SA 3. To the b e s t o f my knowledge, p a r e n t s a r e g e n e r a l l y s a t i s f i e d w i t h the l e t t e r grade system o f r e p o r t i n g . SD D U A SA 4. I am v e r y c o n f i d e n t t h a t the l e t t e r grades I a s s i g n a r e a c c u r a t e and r e l i a b l e . SD D U A SA 5. I would l i k e t o l e a r n more about c o l l a t i n g and a s s i g n i n g l e t t e r grades t o raw s c o r e s . SD D U A SA 6. More i n - s e r v i c e s e s s i o n s on g r a d i n g and r e p o r t i n g s h o u l d be o f f e r e d . SD D U A SA 7. U n i v e r s i t i e s s h o u l d o f f e r more p r e - s e r v i c e i n s t r u c t i o n i n e f f e c t i v e g r a d i n g and r e p o r t i n g . SD D U A SA 8. The r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o be communicated t o p a r e n t s , s t u d e n t s , e t c . SD D U A SA DIRECTIONS: Answer the f o l l o w i n g i n the space p r o v i d e d . 10. B r i e f l y d e s c r i b e how you combine assignment and t e s t s c o r e s t o d e t e r m i n e l e t t e r g r a d e s . (Use back o f page i f n e c e s s a r y ) I n r e f e r e n c e t o the method d e s c r i b e d i n #10, which s u b j e c t s are' graded t h i s way? ( a l l s u b j e c t s , mathematics, language a r t s , s c i e n c e , s o c i a l s t u d i e s , a r t , P.E., music, French) B r i e f l y d e s c r i b e how you a r r i v e a t a f i n a l term s c o r e f o r a s t u d e n t who has been absent f o r one o r more t e s t s o r a s s i g n m e n t s . In o t h e r words, how do you compensate f o r a l e g i t i m a t e l y m i ssed assignment? (Use back of page i f n e c e s s a r y ) 104 DIRECTIONS: s c o r e s ( p e r c a s s i g n m e n t s . 100. The f i c o u r s e ; t h e 10%; and t h e methods you t o c a l c u l a t e I n d i c a t e t h e h i g h e s t ; e t c The f o l l o w i n g e nt) t h a t f i v e Each assignme r s t assignment second and t h i r f i f t h , t he f i n n o r m a l l y use (a and r e c o r d the rank of each s ) i n the space r e c o r d sheet r e p r e s e n t s t h e raw s t u d e n t s o b t a i n e d on f i v e n t had a t o t a l p o s s i b l e s c o r e o f r e p r e s e n t e d about 10% o f the d about 20% each: t h e f o u r t h about a l t e s t , about 40%. Use the s i n d i c a t e d i n PART I I #10 above) T o t a l Score f o r each s t u d e n t , t u d e n t (1 = h i g h e s t ; 2 = n e x t p r o v i d e d . ASSIGNMENTS raw s c o r e s D NAME 1 2 3 4 5 1. Stud e n t A 50 55 65 57 95 2 . Student B 69 60 58 80 83 3 . Stud e n t C 45 72 52 71 75 4 . Stu d e n t D 75 95 45 83 25 5 . Student E 73 83 60 63 36 SCORE RANK Thank you f o r t a k i n g t h e t i m e t o complete t h i s s u r v e y .

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
United States 4 0
China 4 0
France 3 0
Japan 1 0
Germany 1 1
City Views Downloads
Unknown 4 1
Beijing 4 0
Tokyo 1 0
Mountain View 1 0
Sunnyvale 1 0
Redmond 1 0
Ashburn 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}

Share

Share to:

Comment

Related Items