UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Assessment of teachers' grading practises Dauncey, William Monte 1986

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
UBC_1986_A8 D38.pdf [ 4.58MB ]
Metadata
JSON: 1.0054517.json
JSON-LD: 1.0054517+ld.json
RDF/XML (Pretty): 1.0054517.xml
RDF/JSON: 1.0054517+rdf.json
Turtle: 1.0054517+rdf-turtle.txt
N-Triples: 1.0054517+rdf-ntriples.txt
Original Record: 1.0054517 +original-record.json
Full Text
1.0054517.txt
Citation
1.0054517.ris

Full Text

ASSESSMENT OF TEACHERS ' GRADING PRACTISES by WILLIAM MONTE DAUNCEY B.Ed., U n i v e r s i t y of V i c t o r i a , 1976 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS i n THE FACULTY OF GRADUATE STUDIES (Department of E d u c a t i o n a l Psychology) We accept t h i s t h e s i s as conforming to the r e q u i r e d standard THE UNIVERSITY OF BRITISH COLUMBIA October, 1986 %pj W i l l i a m Monte Dauncey, 1986 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t o f the requirements f o r an advanced degree a t the U n i v e r s i t y o f B r i t i s h Columbia, I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and study. I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the head o f my department o r by h i s o r her r e p r e s e n t a t i v e s . I t i s understood t h a t copying or p u b l i c a t i o n o f t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be allowed without my w r i t t e n p e r m i s s i o n . Department o f EDUCATIONAL PSYCHOLOGY The U n i v e r s i t y of B r i t i s h Columbia 1956 Main Mall Vancouver, Canada V6T 1Y3 D a t e OCTOBER 1 1 , 1986 / a n A b s t r a c t E d u c a t o r s have been u s i n g t h e l e t t e r grade system o f g r a d i n g and r e p o r t i n g s t u d e n t achievement f o r s e v e r a l decades. S i n c e i t s i n c e p t i o n , t e a c h e r s have d e r i v e d l e t t e r grades from a v a r i e t y of g r a d i n g t e c h n i q u e s . As a r e s u l t , t h i s approach t o g r a d i n g has o f t e n r e c e i v e d c r i t i c i s m from t h o s e who • q u e s t i o n i t s r e l i a b i l i t y and u s e f u l n e s s . The purpose of t h i s s t u d y was t o de t e r m i n e i f l e t t e r grades c o u l d be made more r e l i a b l e by s t a t i s t i c a l l y b a l a n c i n g raw achievement s c o r e s p r i o r t o a g g r e g a t i o n f o r r e p o r t i n g p u r p o s e s . Many a u t h o r s have w r i t t e n t o a t t a c k l e t t e r grades w h i l e o t h e r s have w r i t t e n t o defend t h e i r use. Some have w r i t t e n t o suggest a l t e r n a t i v e s t o l e t t e r grades w h i l e s t i l l o t h e r w r i t e r s have s u g g e s t e d methods o f i m p r o v i n g g r a d i n g t e c h n i q u e s . However, l i t e r a t u r e s e a r c h e s have shown t h a t v e r y l i t t l e r e s e a r c h has been done t o a s s e s s t e a c h e r s ' g r a d i n g p r a c t i s e s and t h e grades t h e y award s t u d e n t s . T h i s i n v e s t i g a t i v e s t u d y was d e s i g n e d t o e v a l u a t e the g r a d i n g methods used by 37 randomly s e l e c t e d e l e m e n t a r y s c h o o l t e a c h e r s . I n f o r m a t i o n on t h e i r methods o f g r a d i n g was c o l l e c t e d i n t h r e e ways: (a) by way o f a q u e s t i o n n a i r e , (b) by h a v i n g t h e s u b j e c t s w e i g h t , t o t a l , and rank a h y p o t h e t i c a l s e t of raw achievement s c o r e s , and (c) by h a v i n g t h e s u b j e c t s submit c l a s s r e c o r d s f o r one r e p o r t i n g p e r i o d from two s u b j e c t i i i a reas, mathematics and s o c i a l s t u d i e s . The raw scores f o r each c l a s s were s t a t i s t i c a l l y balanced and the teach e r s regraded t h e i r students based on the r e v i s e d aggregate t o t a l s . In o r d e r t o c o n t r o l f o r extraneous and s u b j e c t i v e f a c t o r s , the same g r a d i n g c r i t e r i o n was used f o r both the o r i g i n a l and the r e v i s e d aggregate s c o r e s . The rankings of the o r i g i n a l aggregate scores were compared t o those d e r i v e d from the balanced aggregate scores u s i n g the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t . The c o r r e l a t i o n s were found t o be s i g n i f i c a n t f o r each r e c o r d sheet, i n d i c a t i n g t h a t the n u l l h ypothesis should be r e j e c t e d f o r each of the 56 c l a s s e s s t u d i e d . A n a l y s i s of the c l a s s r e c o r d sheets and q u e s t i o n n a i r e responses a l s o r e v e a l e d : (a) t h a t 46% of the 1,314 students i n v o l v e d i n the study r e c e i v e d a change i n l e t t e r grade i n s p i t e of the s i g n i f i c a n t rank c o r r e l a t i o n c o e f f i c i e n t s . T h i s suggests t h a t f o r many students, the assignment of l e t t e r grades was u n r e l i a b l e and based on f a c t o r s other than t o t a l score rankings. (b) t h a t o n l y 5% of the respondents used methods t h a t would apply the d e s i r e d weighting f a c t o r s t o the raw s c o r e s . T h i s suggests t h a t many teach e r s used u n r e l i a b l e methods t o weight assignment s c o r e s . (c) t h a t none of the s u b j e c t s i n t h i s study used r e l i a b l e methods t o compensate a student who has missed one or more assignments or t e s t s . T h i s suggests t h a t students who were i v a b sent may have been u n j u s t l y rewarded o r p e n a l i z e d when t h e i r a g g r e g a t e s c o r e s were c a l c u l a t e d . (d) t h a t 76% o f the r e s p o n d e n t s showed a d e s i r e t o l e a r n more about c o l l a t i n g raw s c o r e s and a s s i g n i n g l e t t e r grades t o agg r e g a t e s c o r e s . , , These r e s u l t s s u g g e sted t h a t i n - s e r v i c e i n s t r u c t i o n and p r e - s e r v i c e t r a i n i n g i n p a r t i c u l a r a s p e c t s o f g r a d i n g and r e p o r t i n g would be j u s t i f i e d f o r many members of t h e r e s e a r c h sample. Areas o f g r e a t e s t need a r e tho s e c o n c e r n i n g t h e w e i g h t i n g o f raw s c o r e s , t h e a l l o c a t i o n o f l e t t e r g r a d e s , and the c a l c u l a t i o n of compensation s c o r e s f o r s t u d e n t s who have mi s s e d a s s i g n m e n t s . R e s e a r c h S u p e r v i s o r : Dr. H. R a t z l a f f V TABLE OF CONTENTS A b s t r a c t i i L i s t o f T a b l e s v i Acknowledgements v i i i C h a p ter 1 - Scope o f the Study 1 - I d e n t i f i c a t i o n o f the Problem 4 - D e f i n i t i o n o f Terms 12 - R e s e a r c h Q u e s t i o n s 16 - R a t i o n a l e 17 - D e l i m i t a t i o n s 19 - J u s t i f i c a t i o n o f t h e Study 20 Chapter 2 - L i t e r a t u r e Review 2 4 - G r a d i n g Systems 2 9 - C r i t e r i o n - r e f e r e n c e d G r a d i n g 33 - Norm-referenced G r a d i n g 3 6 Chapter 3 - Methodology 4 5 - Sample and P o p u l a t i o n 45 - Q u e s t i o n n a i r e 46 - Record Sheets 4 9 - G r a d i n g P o l i c y 52 - Data C o l l e c t i o n and A n a l y s i s 54 Chapter 4 - R e s u l t s 56 - Sample 56 - G r a d i n g Techniques 64 - Record Sheets 70 Chapter 5 - C o n c l u s i o n s and F u r t h e r R e s e a r c h 77 - Summary 7 7 - R e s e a r c h Q u e s t i o n 1 79 - R e s e a r c h Q u e s t i o n 2 86 - R e s e a r c h Q u e s t i o n 3 89 - R e s e a r c h Q u e s t i o n 4 91 - Weaknesses To Be C o n s i d e r e d 92 - O p p o r t u n i t i e s f o r F u t u r e R e s e a r c h 94 B i b l i o g r a p h y 98 Appendix A - Q u e s t i o n n a i r e 101 LIST OF TABLES v i T a b l e I - F i c t i t i o u s d a t a t o be c o l l a t e d and ranked 5 0 T a b l e I I - F i c t i t i o u s d a t a a f t e r b e i n g s t a t i s t i c a l l y b a l a n c e d 50 T a b l e I I I - L e t t e r grade d i s t r i b u t i o n g u i d e l i n e s 53 Ta b l e IV - D i s t r i b u t i o n by Gender 57 T a b l e V - D i s t r i b u t i o n by Age 57 T a b l e VI - D i s t r i b u t i o n by u n i v e r s i t y t r a i n i n g ( y e a r s ) 57 Ta b l e V I I - D i s t r i b u t i o n by Measurement Courses Taken 5 8 Ta b l e V I I I - D i s t r i b u t i o n by T e a c h i n g Assignment 58 Ta b l e IX - D i s t r i b u t i o n by T e a c h i n g E x p e r i e n c e 58 Ta b l e X - D i s t r i b u t i o n o f C o l l e a g u e s ' A t t i t u d e s Toward Re p o r t Card Marks 6 0 Ta b l e XI - D i s t r i b u t i o n o f A t t i t u d e Toward L e t t e r Grade E f f e c t i v e n e s s 60 Ta b l e X I I - D i s t r i b u t i o n o f Agreement R e g a r d i n g P a r e n t s ' S a t i s f a c t i o n Toward L e t t e r Grade R e p o r t i n g Systems 61 Ta b l e X I I I - D i s t r i b u t i o n o f Agreement R e g a r d i n g S u b j e c t s ' Own G r a d i n g A b i l i t y 61 Ta b l e XIV - D i s t r i b u t i o n o f Agreement R e g a r d i n g S u b j e c t s ' D e s i r e To L e a r n More About M a r k i n g and G r a d i n g 6 2 Ta b l e XV - D i s t r i b u t i o n o f Agreement on E v a l u a t i o n I n s e r v i c e S e s s i o n s . 62 Ta b l e XVI - D i s t r i b u t i o n o f Agreement R e g a r d i n g S u b j e c t s ' O p i n i o n on P r e - s e r v i c e T r a i n i n g 62 v i i T a b l e X V I I - D i s t r i b u t i o n o f Agreement R e g a r d i n g E f f e c t i v e n e s s o f the R e p o r t Card Format As a Communication D e v i c e 63 Ta b l e X V I I I - D i s t r i b u t i o n o f G r a d i n g Methods 66 T a b l e XIX - D i s t r i b u t i o n o f A p p l i c a t i o n o f G r a d i n g Technique 66 Ta b l e XX - D i s t r i b u t i o n o f Compensation Method 68 Tab l e XXI - D i s t r i b u t i o n o f Ranks o f F i c t i t i o u s Data 6 9 T a b l e X X I I - Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s R e l a t i n g Raw Score and B a l a n c e d Score T o t a l s 72 Ta b l e X X I I I - D i s t r i b u t i o n of Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s 73 Ta b l e XXIV - D i s t r i b u t i o n o f L e t t e r Grade Changes 74 Tab l e XXV - Frequency o f C a t e g o r y S h i f t 74 Tab l e XXVI - Frequency of L e t t e r Grade Changes R e s u l t i n g From A r i t h m e t i c E r r o r s 75 Tab l e XXVII - Frequency of C a t e g o r y S h i f t R e s u l t i n g From L e t t e r Grade R e c a l c u l a t i o n 76 Tab l e X X V I I I - E f f e c t s o f r For Between S c o r e s on T o t a l Score r When r (between) =+1 8 4 s T a b l e XXIX - E f f e c t on r For Between Scores On T o t a l Score r When r (betwlen) =. 65 8 5 s T a b l e XXX - Compensation F o r M i s s e d Assignment 90 v i i i Acknowledgement I would l i k e t o e x p r e s s my g r a t i t u d e t o my r e s e a r c h s u p e r v i s o r , Dr. H. R a t z l a f f , and t o committee members Dr. W. B o l d t and Dr. W. S z e t e l a f o r t h e i r h e l p and c o u n s e l . I w i s h t o extend my s i n c e r e thanks and a p p r e c i a t i o n t o my w i f e , Roxanne, f o r her c o n t i n u o u s s u p p o r t and encouragement th r o u g h o u t t h e c o u r s e o f my s t u d i e s . F i n a l l y , t o my c h i l d r e n , D e n i s e and Douglas, a s p e c i a l note o f g r a t i t u d e f o r t h e i r p a t i e n c e and u n d e r s t a n d i n g . 1 CHAPTER 1 Scope of the Study- Background I n f o r m a t i o n F o r more th a n f i f t y y e a r s , the l e t t e r grade system has been used i n e d u c a t i o n t o r e p o r t and document the l e v e l of s t u d e n t achievement. However, even a f t e r decades of use, g r a d i n g p r a c t i s e s w i t h i n t h e l e t t e r grade system s t i l l t e n d t o v a r y among t e a c h e r s , departments, s c h o o l s , and s c h o o l d i s t r i c t s . Many t e a c h e r s a c q u i r e g r a d i n g p r a c t i s e s i n f o r m a l l y ; from a c o l l e a g u e , department head, o r by u s i n g methods t h a t seem i n t u i t i v e l y r e a s o n a b l e . Others may have had a f l e e t i n g exposure t o an " a c c e p t a b l e " g r a d i n g p r o c e d u r e d u r i n g t e a c h e r t r a i n i n g . I t i s s u r p r i s i n g t h a t many t e a c h e r s embark on t h e i r c a r e e r s i n the s c h o o l s w i t h o u t e v e r h a v i n g been t a u g h t a n y t h i n g about g r a d i n g . T h i s v e r y i m p o r t a n t p a r t o f the t e a c h e r ' s t a s k i s o f t e n i g n o r e d i n t e a c h e r t r a i n i n g . Perhaps i t i s j u s t t a k e n f o r g r a n t e d , o r perhaps i t seems t o o i n t r a c t a b l e t o be t a u g h t . ( H i l l s , 1981, p. 316) C o n s e q u e n t l y , when i n c o n s i s t e n c i e s i n g r a d i n g p r a c t i s e s e x i s t , e q u a l l e v e l s o f achievement are not n e c e s s a r i l y g i v e n e q u a l l e t t e r g r a d e s . F o r example, a grade of "C" i n one c l a s s r o o m may be a "C-" o r a "C+" i n a n o t h e r . As a r e s u l t , grades and g r a d i n g p r a c t i s e s become somewhat s u s p e c t and o f t e n l o s e c r e d i b i l i t y w i t h t h e p o s t - s e c o n d a r y i n s t i t u t i o n s , p r o s p e c t i v e employers, s t u d e n t s , and o t h e r i n d i v i d u a l s o r a g e n c i e s t h a t 2 depend upon r e l i a b l e academic assessments as a b a s i s f o r d e c i s i o n making. T h i s n o t i o n i s s u p p o r t e d by H i l l s (1981) who s t a t e s , F a u l t y g r a d i n g can be s e r i o u s l y d e t r i m e n t a l t o s t u d e n t s . I t can g i v e them m i s l e a d i n g i n f o r m a t i o n and i n t e r f e r e w i t h the l e a r n i n g p r o c e s s . I t can g i v e m i s l e a d i n g i n f o r m a t i o n f o r o t h e r s who l e g i t i m a t e l y use s c h o o l grades as p a r t o f t h e i r p r o c e d u r e s i n e v a l u a t i n g p e o p l e . And i t can m i s l e a d t h o s e who would s t u d y the i n s t r u c t i o n a l p r o c e s s t o improve i t o r t o h o l d i t a c c o u n t a b l e t o the s o c i e t y t h a t s u p p o r t s i t . (p. 317) However, f o r most s t u d e n t s u n r e l i a b l e c a l c u l a t i o n s o f a term's grades w i l l n o t cause much harm. For example, i f a "C" i s awarded i n s t e a d o f a C+ o r a C- , t h e d i f f e r e n t i a l e f f e c t s . upon t h e s t u d e n t w i l l not v a r y s i g n i f i c a n t l y . D i s c r e p a n c i e s such as t h e s e w i l l seldom l e a d t o i n c o r r e c t d e c i s i o n s . a s C+, C, and C- a r e a l l symbols o f " s a t i s f a c t o r y " achievement. On the o t h e r hand, when a s t u d e n t has been g i v e n an F when he m e r i t e d a p a s s i n g grade, some f a r r e a c h i n g consequences c o u l d r e s u l t . Such an e r r o r c o u l d c o s t the s t u d e n t c r e d i t f o r t h e c o u r s e , o r even an e x t r a y e a r i n s c h o o l i f a co u r s e f a i l u r e p r e v e n t s p r o m o t i o n . At t h e o t h e r end o f the s c a l e , a C+ i n s t e a d o f a B, o r a B i n s t e a d o f an A c o u l d r e s u l t i n t h e l o s s o f a s c h o l a r s h i p , a b u r s a r y , o r an o p p o r t u n i t y t o a t t e n d a p a r t i c u l a r l y c o m p e t i t i v e p o s t - s e c o n d a r y i n s t i t u t i o n . Of c o u r s e , t h e i n v e r s e i s e q u a l l y s i g n i f i c a n t . F o r example, i f a 3 s t u d e n t i s e r r o n e o u s l y awarded a p a s s i n g g r a d e , he w i l l be g i v e n c r e d i t f o r a c o u r s e when i t i s not w a r r a n t e d and when he i s n o t s u f f i c i e n t l y p r e p a r e d t o go on t o the nex t l e v e l . In e l e m e n t a r y s c h o o l s , marks and o t h e r measures o f s t u d e n t achievement form t h e b a s i s f o r d e t e r m i n i n g whether a s t u d e n t w i l l be promoted i n t o the n e x t grade, whether he w i l l r e c e i v e an academic award o r whether he w i l l be a c a n d i d a t e f o r l e a r n i n g a s s i s t a n c e o r a s p e c i a l r e m e d i a l c l a s s . I n j u n i o r h i g h s c h o o l , term grades a r e used as i n the e l e m e n t a r y s c h o o l s e t t i n g but w i t h the a d d i t i o n a l use of i d e n t i f y i n g honour r o l l s t u d e n t s . S e n i o r h i g h s c h o o l s use term grades not o n l y f o r p r o m o t i o n and r e t e n t i o n but a l s o t o det e r m i n e which s t u d e n t s w i l l be r e q u i r e d t o w r i t e f i n a l exams, which s t u d e n t s w i l l be e l i g i b l e f o r s c h o l a r s h i p s and b u r s a r i e s , and which s t u d e n t s s h o u l d be encouraged t o a t t e n d p o s t - s e c o n d a r y i n s t i t u t i o n s . A l t h o u g h t h e r e a re t h o s e who d i s a g r e e w i t h the l e t t e r system o f g r a d i n g and r e p o r t i n g (see Curwin, 1978), t h o s e who do agree f e e l i t may be one of t h e t e a c h e r ' s most i m p o r t a n t r o l e s . "One o f the most i m p o r t a n t p r o f e s s i o n a l r e s p o n s i b i l i t i e s o f academic s t a f f i s t h a t of a l l o c a t i n g and r e p o r t i n g marks (or grades) as measures of s t u d e n t performance" ( I s a a c s and I m r i e , 1981, p. 3 ) . Curwin s u g g e s t s t h a t t e a c h e r s s h o u l d q u e s t i o n themselves as t o the r e l i a b i l i t y of grades i n f u l f i l l i n g t he r e p o r t i n g and l e a r n i n g f u n c t i o n s . He s t a t e s , "The one b a s i c r e q u i r e m e n t of the r e p o r t i s t h a t a l l i n f o r m a t i o n r e c e i v e d must have t h e same meaning t h a t was 4 s e n t by the g r a d e r . Grades must be v a l i d , r e l i a b l e and have a h i g h degree of p r e d i c t a b i l i t y . But t h e y don't." ( C u r w i n , 1978 , p. 6 0 ) . I d e n t i f i c a t i o n of t h e Problem I t seems c l e a r t h a t t h e r e i s a need f o r t e a c h e r s ' g r a d i n g p r a c t i s e s t o be as r e l i a b l e and a c c u r a t e as p o s s i b l e . Most s c h o o l boards a l s o r e c o g n i z e t h e need f o r r e l i a b i l i t y i n e v a l u a t i o n and r e p o r t i n g . As s t a t e d i n t h e A d m i n i s t r a t i v e Handbook of one l a r g e d i s t r i c t , The Board of T r u s t e e s b e l i e v e s t h a t e v a l u a t i o n and r e p o r t i n g o f p u p i l achievement a r e i n t e g r a l and v i t a l p a r t s o f t e a c h i n g and l e a r n i n g . The Board f u r t h e r b e l i e v e s t h a t e v a l u a t i o n and r e p o r t i n g s h o u l d be based on r e l i a b l e i n f o r m a t i o n about p u p i l performance i n r e l a t i o n t o t h e s p e c i f i c r e q u i r e m e n t s of a program, c o u r s e , or grade. ( S c h o o l D i s t r i c t A d m i n i s t r a t i v e Handbook, 1978) W i th r e l i a b l e and a c c u r a t e r e p o r t i n g , c o n s i s t e n c y among s c h o o l s and c l a s s r o o m s i s m a i n t a i n e d , c r e d i b i l i t y i n t h e eyes of t h e community i s p r e s e r v e d , and e d u c a t i o n a l d e c i s i o n s can be made more c o n f i d e n t l y and c o r r e c t l y . However, the q u e s t i o n remains: Are the t e a c h e r s ' c u r r e n t g r a d i n g p r a c t i s e s as r e l i a b l e and a c c u r a t e as p o s s i b l e ? C urwin (1978), f o r one, c l a i m s t h e y a r e n o t . 5 With the i n t r o d u c t i o n o f t h e l e t t e r grade r e p o r t c a r d s i n B r i t i s h Columbia's e l e m e n t a r y s c h o o l s , t h e r e i s the assumption t h a t t e a c h e r s w i l l have the background knowledge and t h e r e s o u r c e s n e c e s s a r y t o c o l l a t e s c o r e s , grade, and r e p o r t s t u d e n t achievement r e l i a b l y . As i n d i c a t e d p r e v i o u s l y , many t e a c h e r s may not have t h e n e c e s s a r y u n d e r s t a n d i n g or t r a i n i n g t o p e r f o r m such t a s k s w e l l . Many t e a c h e r s s i m p l y do not know how t o combine s t u d e n t achievement s c o r e s i n such a way as t o y i e l d r e l i a b l e l e t t e r grades f o r r e p o r t i n g p u r p o s e s . Teachers agree t h a t g i v i n g and r e p o r t i n g marks (or grades) i s one of t h e i r most u n c o m f o r t a b l e r e s p o n s i b i l i t i e s . Some l a c k c o n f i d e n c e i n the marks t h e y a s s i g n ; o t h e r s b e l i e v e t h e i r marks are f a i r , but f i n d them d i f f i c u l t t o d e f e n d . B e h i n d t h e s e n e g a t i v e a t t i t u d e s l i e s the f a c t t h a t the b a s i s f o r • a s s i g n i n g marks i s o f t e n u n c l e a r . (Ahmann and G l o c k , 1981, p. 417) As a r e s u l t , many t e a c h e r s combine and c a l c u l a t e s c o r e s i n a manner t h a t seems i n t u i t i v e l y r e a s o n a b l e , and by v i r t u e o f t h e i r l a c k o f knowledge, b l i s s f u l l y a c c e p t t h e i r l e t t e r g rades as b e i n g r e l i a b l e assessments o f t h e i r s t u d e n t s ' work. Many assume t h a t by u s i n g raw s c o r e s t h e i r term assignments w i l l be a p p r o p r i a t e l y w e i g h t e d when combining marks f o r t h e i r term t o t a l s . Many t e a c h e r s seem t o r e g a r d the mark v a l u e o f each assignment as b e i n g an i n d i c a t o r o f t h e w e i g h t c o n t r i b u t i n g t o the t o t a l term s c o r e ( i e . an assignment worth 100 marks c o n t r i b u t e s t w i c e as much toward term t o t a l s as does an assignment w o r t h 50 marks). I t would seem t h a t one of t h e 6 more common methods of c o l l a t i n g achievement r e s u l t s f o r g r a d i n g purposes i n v o l v e s s i m p l e t o t a l l i n g of raw s c o r e s o r a v e r a g i n g ( h i g h e s t term t o t a l o r average r e c e i v e s an A e t c . ) . I n extreme c a s e s , where s t a n d a r d d e v i a t i o n s among term assignments a r e v e r y d i f f e r e n t , s i m p l e summations and a v e r a g i n g can r e s u l t i n h i g h l y e r r o n e o u s a g g r e g a t e s c o r e s and u n r e l i a b l e r e p o r t i n g , as w i l l be demonstrated i n a subsequent example. A l s o , some t e a c h e r s s t i l l a b i d e by t h e p e r c e n t a g e ' r u l e - o f - t h u m b ' i n terms of t e s t , a s s i g nment, o r term r e s u l t s . They may, f o r example, c o n s i d e r any s c o r e above 80 per c e n t t o be e q u i v a l e n t t o an 'A', even i n extreme c a s e s where an 80 per c e n t i s the l o w e s t s c o r e o r where the h i g h e s t s c o r e i s l e s s t h a n 8 0 per c e n t . Grades o b t a i n e d from t h e s e and o t h e r s i m i l a r l y q u e s t i o n a b l e methods a r e a c c e p t e d as b e i n g c r e d i b l e and r e l i a b l e by i n t e r e s t e d p a r t i e s (such as p a r e n t s , p r i n c i p a l s , p r o s p e c t i v e e m p l o y e r s , and h i g h e r e d u c a t i o n a l i n s t i t u t i o n s ) and a r e used as a b a s i s f o r making i m p o r t a n t d e c i s i o n s . In s h o r t , t e a c h e r s o f t e n have c o n s i d e r e d n e i t h e r mean s c o r e s nor t h e v a r i a b i l i t y o f s c o r e s when e v a l u a t i n g p u p i l achievement. As w i l l be shown, c a l c u l a t i n g term t o t a l s from s e v e r a l a ssignments may y i e l d unexpected and u n r e l i a b l e r e s u l t s i f t h e c l a s s averages and t h e amount of v a r i a t i o n i n each assignment a r e not c o n s i d e r e d . " I f we hope t o m a i n t a i n the w e i g h t i n g scheme o r i g i n a l l y chosen, we must t a k e i n t o c o n s i d e r a t i o n t h e d i f f e r e n c e s i n v a r i a b i l i t y . A f a i l u r e t o do t h i s w i l l r e s u l t i n i n e q u i t i e s " (Ahmann and G l o c k , 1981, p. 426 ) . 7 B e f o r e examining why mean s c o r e s and s t a n d a r d d e v i a t i o n s s h o u l d be employed, a r e v i e w of how f i n a l o r term t o t a l s a r e t y p i c a l l y o b t a i n e d i s i n o r d e r . F i n a l c o u r s e grades a r e t y p i c a l l y based on v a r i o u s t y p e s o f s t u d e n t achievement, such as s h o r t q u i z z e s , term p r o j e c t s , l a b o r a t o r y work, and e x a m i n a t i o n s . Thus, a s s i g n i n g c o u r s e grades i n v o l v e s combining t h e s e v a r i o u s measures t o o b t a i n a composite o r term s c o r e i n such a way t h a t each assignment r e c e i v e s i t s i n t e n d e d w e i g h t . For example, i f i t has been d e c i d e d t h a t t h e f i n a l e x a m i n a t i o n s h o u l d count t w i c e as much as the s h o r t q u i z z e s , t h e n the grades s h o u l d r e f l e c t t h i s emphasis. S i m i l a r l y , i f the l a b o r a t o r y work i s t o make up 25 per c e n t o f the f i n a l c o u r s e grade, t h e n i t i s i m p o r t a n t t h a t i t w i l l be r e p r e s e n t e d t o t h a t e x t e n t . I t i s not uncommon f o r t e a c h e r s t o communicate t h i s t y p e o f i n f o r m a t i o n t o t h e i r s t u d e n t s a t the b e g i n n i n g of the c o u r s e . S t u d e n t s a r e p a r t i c u l a r l y i n t e r e s t e d i n knowing which elements w i l l be i n c l u d e d i n t h e f i n a l grade and t h e r e l a t i v e w e i g h t each w i l l be g i v e n . D e t e r m i n i n g how much w e i g h t s h o u l d be g i v e n t o each of the v a r i o u s assignments i s a m a t t e r o f judgment. T h i s judgment i s g u i d e d by the importance o f the v a r i o u s i n s t r u c t i o n a l o b j e c t i v e s , the t e a c h i n g emphasis g i v e n t o each t y p e o f c o u r s e a c t i v i t y , and o t h e r s i m i l a r c o n s i d e r a t i o n s . The t e a c h e r must d e c i d e on t h e c o m p o s i t i o n o f the grades g i v e n i n each c o u r s e and t h e r e l a t i v e emphasis t o be a l l o c a t e d t o each component. W i t h o u t a s s i g n i n g d e s i r e d w e i g h t i n g s t o assignments and by u s i n g o n l y the n a t u r a l w e i g h t i n g s , i t i s 8 p o s s i b l e f o r a r e l a t i v e l y minor assignment t o have a v e r y s t r o n g , i f not dominant, i n f l u e n c e on the c o u r s e grade. A l s o , the n a t u r a l w e i g h t i n g s may r e f l e c t a t o t a l l y d i f f e r e n t emphasis on the v a r i o u s i n s t r u c t i o n a l o b j e c t i v e s than was a c t u a l l y used by t h e t e a c h e r d u r i n g t h e c o u r s e . When t h e d e c i s i o n has been made c o n c e r n i n g what p r o p o r t i o n o f the f i n a l a g g r e g a t e s c o r e i s t o be a l l o c a t e d t o each measure o f s t u d e n t performance, t h e measures must be combined i n such a way t h a t t h e d e s i r e d w e i g h t i n g i s o b t a i n e d . T h i s b r i n g s us back t o t h e imp o r t a n c e o f c a l c u l a t i n g c l a s s means and s t a n d a r d d e v i a t i o n s . To i l l u s t r a t e t he problems o f the v a r i o u s g r a d i n g p r a c t i s e s i n t h e w e i g h t i n g of achievement d a t a , which i s t h e problem o f i n t e r e s t i n t h i s s t u d y , c o n s i d e r a s i m p l e example ( t a k e n from G r o n l u n d , 1974). L e t us assume t h a t we want t o combine s c o r e s on an e x a m i n a t i o n and s c o r e s on a term p r o j e c t , and t h a t we want them t o c o n t r i b u t e e q u a l l y t o the f i n a l ( o r term) grade. Suppose t h e range o f s c o r e s on the two measures a r e : Range o f s c o r e s on e x a m i n a t i o n 20 t o 120 Range o f s c o r e s on term p r o j e c t 40 t o 60 I f we s i m p l y add t o g e t h e r the e x a m i n a t i o n s c o r e and t h e p r o j e c t s c o r e f o r each i n d i v i d u a l , t h e f i n a l grade w i l l be de t e r m i n e d l a r g e l y by how w e l l t h e s t u d e n t s performed on t h e e x a m i n a t i o n . To demonstrate t h i s , compare a s t u d e n t who i s h i g h e s t on t h e e x a m i n a t i o n and l o w e s t on t h e term p r o j e c t 9 (Student A) with a student who i s h i g h e s t on the term p r o j e c t and lowest on the examination (Student B). Student A Student B Examination score 120 20 Term p r o j e c t s c o r e 40 60 Composite score 160 80 I t i s obvious t h a t simply adding the two scores w i l l not gi v e them equal r e p r e s e n t a t i o n i n the composite s c o r e . Nor w i l l c a l c u l a t i n g an average score p r o v i d e equal weighting. The examination score has much g r e a t e r i n f l u e n c e than the term p r o j e c t s c o r e . In a s i t u a t i o n such as t h i s , t eachers f r e q u e n t l y attempt t o equate the i n f l u e n c e of the two measures by making the maximum p o s s i b l e score f o r both assignments e q u i v a l e n t . In the above example, t h i s would mean m u l t i p l y i n g the s t u d e n t s ' term p r o j e c t score by two t o make the t o t a l p o s s i b l e score 120 f o r both measures of achievement. The f o l l o w i n g shows what happens when t h i s i s done: Student A Student B Examination score 120 20 Term p r o j e c t score (x2) 80 120 Composite score 200 140 10 As t h e example d e m o n s t r a t e s , e q u a t i n g on t h e b a s i s of maximum p o s s i b l e s c o r e does not p r o v i d e f o r e q u a l emphasis i n the c o m p o s i t e s c o r e e i t h e r . . The e x a m i n a t i o n s c o r e s t i l l has t h e g r e a t e r i n f l u e n c e when t h e two a r e combined. T h i s i s because th e c o n t r i b u t i o n a g i v e n measure makes t o a composite s c o r e depends l a r g e l y on the v a r i a b i l i t y , o r s p r e a d , o f the s c o r e s i n t h e s e t . To equate the e x a m i n a t i o n s c o r e and t h e term p r o j e c t s c o r e , t h e v a r i a t i o n o f the s c o r e s i n each s e t must be made e q u a l . T h i s i s done by a d j u s t i n g the s t a n d a r d d e v i a t i o n o f one o f t h e s e t s o f s c o r e s so t h a t i t i s e q u a l t o t h e o t h e r s e t . The method most o f t e n used i s the s t a n d a r d T-score t r a n s f o r m a t i o n . T h i s changes each s e t so .that t h e c l a s s average (mean) i s 50 and t h e s t a n d a r d d e v i a t i o n i s 10. T h i s i s n ot an easy t a s k w i t h o u t t h e use of a computer o r a s o p h i s t i c a t e d c a l c u l a t o r , a g r e a t d e a l o f t i m e , and some s t a t i s t i c a l knowledge. However, once the s c o r e s a r e ' b a l a n c e d ' , w e i g h t i n g s can be a p p l i e d t o each t r a n s f o r m e d s c o r e w i t h t h e c o n f i d e n c e t h a t t h e d e s i r e d w e i g h t i n g s w i l l be o b t a i n e d . I n t h i s s i m p l e example, th e two a ssignments a r e w e i g h t e d e q u a l l y . Our example would t h e n l o o k l i k e t h i s : S tudent A Student B E x a m i n a t i o n T-score (weighted) Term p r o j e c t T - s c o r e (weighted) 100 80 80 100 Composite T-score (weighted) 180 180 11 C l e a r l y , the e x a m i n a t i o n and the term p r o j e c t s c o r e s a re e q u a l ; b o t h s t u d e n t s r e c e i v e t h e same composite s c o r e . T h i s i s p o s s i b l e o n l y by e q u a t i n g the two assignments i n terms o f means and s t a n d a r d d e v i a t i o n s . I n t h i s c a s e , t h e s t a n d a r d T-score was used f o r bo t h e x e r c i s e s . The main q u e s t i o n s t o be ad d r e s s e d t h e n a r e t h e f o l l o w i n g : 1. I s t h e r e a s i g n i f i c a n t d i f f e r e n c e between t h e grades r e p o r t e d by t e a c h e r s u s i n g i n t u i t i v e methods and t h o s e r e p o r t e d when means and s t a n d a r d d e v i a t i o n s a r e u t i l i z e d and a p p l i e d ? I n o t h e r words, i f t e a c h e r s were t o f o l l o w t h e a d v i c e o f many a u t h o r s and employ b a s i c s t a t i s t i c s when a g g r e g a t i n g marks, would t h e r e s u l t s be s i g n i f i c a n t l y d i f f e r e n t from t h o s e o b t a i n e d from e x i s t i n g g r a d i n g p r a c t i s e s ? 2. Are the raw achievement s c o r e s combined and w e i g h t e d r e l i a b l y ? 3. Are the methods used t o compensate s t u d e n t s who have mis s e d a ssignments o r t e s t s f a i r and r e l i a b l e ? Are s t u e n t s g i v e n grades t h a t p e n a l i z e o r reward them f o r b e i n g absent f o r a t e s t o r assignment? 4. Would any improvement i n r e s u l t s be o f such a magnitude t o j u s t i f y t he time and expense r e q u i r e d t o r e t r a i n t e a c h e r s ? I t was t h e purpose o f t h i s s t u d y t o i n v e s t i g a t e t h e s e q u e s t i o n s t o de t e r m i n e i f t e a c h e r s ' a g g r e g a t i n g and g r a d i n g methods c o u l d be improved. 12 DEFINITION OF TERMS The f o l l o w i n g are terms used i n t h i s study: 1. ASSIGNMENT SCORES: "Assignments" r e f e r t o the v a r i o u s e x e r c i s e s , t e s t s , l a b o r a t o r y work, term p r o j e c t s , e t c e t e r a , t h a t a teacher might a s s i g n h i s students d u r i n g the course of i n s t r u c t i o n i n a p a r t i c u l a r s u b j e c t . An "assignment score" i s the r e s u l t t h a t each student would r e c e i v e a f t e r completing a p a r t i c u l a r assignment. At the end of each term, a t o t a l or "term s c o r e " i s obtained f o r each student by summing t h e i r r e s p e c t i v e r e s u l t s on each assignment. For r e p o r t i n g purposes each student's term score i s then ranked l a r g e s t t o s m a l l e s t and l e t t e r grades are assig n e d based on the rankings of the t o t a l s c o r e s . 2. BALANCED SCORES: "Balanced Scores" or " s t a t i s t i c a l l y balanced s c o r e s " r e f e r to assignment scores t h a t have undergone a l i n e a r t r a n s f o r m a t i o n t o a l t e r the mean and standard d e v i a t i o n . When s e v e r a l c l a s s assignments have been "balanced", they have been g i v e n equal means and equal standard d e v i a t i o n s . Balanced assignments contribute- e q u a l l y towards the f i n a l , course mark. 3. FAIR: For the purpose of t h i s study, " f a i r " i s used to mean " j u s t " or "not f a v o u r i n g one more than t h e ' o t h e r " . F a i r grades are those t h a t do not p e n a l i z e or reward i n d i v i d u a l students more than they do other students. F a i r grades are a l s o c o n s i s t e n t . For example, two students having 13 the same raw scores and subjected t o i d e n t i c a l grading c r i t e r i a would be expected to r e c e i v e i d e n t i c a l grades. However, i f one of the students were t o r e c e i v e a higher grade than the other, then the grades would not be c o n s i s t e n t and would be considered " u n f a i r " . 4. LETTER GRADE: For the purpose of t h i s study, a l e t t e r grade i s defined as: "1) a summary symbol, 2) e v a l u a t i n g a s u b s t a n t i a l segment of achievement which i s 3) a t t a i n e d by a p u p i l i n a course and 4) assigned by a teacher 5) f o r the purpose of record and r e p o r t " ( M i n i s t r y of Education, Science and Technology, 1979, p. 9). A l e t t e r grade should be reserved f o r judgements t h a t i n c l u d e evidence a v a i l a b l e on student achievement i n a complete course or a segment of a course such as a r e p o r t i n g p e r i o d . L e t t e r grades are u s u a l l y derived from the evidence a v a i l a b l e i n a set of scores f o r each p u p i l . I t i s a l s o assumed that a teacher has a systematic procedure f o r a s s i g n i n g grades. Because a l e t t e r grade i s "used t o record and r e p o r t " , i t i s assumed th a t a l e t t e r grade i s a "permanent record u s u a l l y r e t a i n e d f o r a long p e r i o d of time i n the school record system and used t o communicate with i n d i v i d u a l students, parents, teachers, and other educational a u t h o r i t i e s and p o t e n t i a l employers" ( M i n i s t r y of Education, Science and Technology, 1979, p. 10). 5. MEAN: "Mean" i s synonymous with " a r i t h m e t i c mean" or "average" and i s a measure of c e n t r a l tendency. I t i s c a l c u l a t e d by summing a s e r i e s of numbers and d i v i d i n g the sum by the number of numbers summed. The value of the mean i s 14 a f f e c t e d by the i n d i v i d u a l v a l u e s o f a l l o f the s c o r e s i n the se t o f d a t a . I n t h i s s t u d y , t h e mean i s denoted as X and 2 m i n d i v i d u a l raw s c o r e s as X.. 1 6. RELIABLE: R e l i a b l e grades a r e t h o s e t h a t a r e "dependable and f i t t o be r e l i e d upon". Grades t h a t a r e r e l i a b l e p r o v i d e an a c c u r a t e i n d i c a t i o n of a s t u d e n t ' s performance and a r e worthy o f b e i n g used as a b a s i s f o r d e c i s i o n making. 7. T-SCORES: "T-Scores" a r e a l i n e a r t r a n s f o r m a t i o n of s t a n d a r d s c o r e s . S t a n d a r d i z e d T - s c o r e s a re t r a n s f o r m e d t o a s e t of d a t a h a v i n g a mean o f 50 and s t a n d a r d d e v i a t i o n of 10. C a l c u l a t i o n o f T-sc o r e s i s a c h i e v e d by m u l t i p l y i n g a s t a n d a r d s c o r e (z s c o r e ) by 10 and a d d i n g 50 t o t h e p r o d u c t . The shape of t h e d i s t r i b u t i o n i s not changed w i t h the t r a n s f o r m e d s c o r e s . 8. STANDARD DEVIATION: "Sta n d a r d D e v i a t i o n " i s a measure of d i s p e r s i o n , s c a t t e r , h e t e r o g e n e i t y , o r v a r i a t i o n i n a s e t of d a t a . A s e t o f s c o r e s w i t h g r e a t h e t e r o g e n e i t y w i l l have some l a r g e d e v i a t i o n s c o r e s (X.-X ). St a n d a r d d e v i a t i o n i s c a l c u l a t e d by f i r s t summing the squared d e v i a t i o n s c o r e s and then d i v i d i n g the sum by the number o f s c o r e s . The square r o o t o f the q u o t i e n t i s the s t a n d a r d d e v i a t i o n . I n t h i s s t u d y , t h e s t a n d a r d d e v i a t i o n i s denoted as SD. 9. STANDARD SCORES: "Sta n d a r d S c o r e s " can be used t o d e s c r i b e t h e p o s i t i o n o f a s c o r e i n a s e t o f s c o r e s by measuring i t s d e v i a t i o n from t h e mean o f a l l s c o r e s i n s t a n d a r d d e v i a t i o n u n i t s . A s t a n d a r d s c o r e ( c a l l e d a z - s c o r e ) 15 i s o b t a i n e d by s u b t r a c t i n g the mean from an ob s e r v e d s c o r e and d i v i d i n g t h e d i f f e r e n c e by t h e s t a n d a r d d e v i a t i o n . The mean and s t a n d a r d d e v i a t i o n o f the t r a n s f o r m e d raw s c o r e s ( z - s c o r e s ) a r e 0 and 1 r e s p e c t i v e l y . I n t h i s way, t h e n , the d i f f e r e n c e between an i n d i v i d u a l s c o r e and t h e mean s c o r e of the group, i s s c a l e d i n terms of s t a n d a r d d e v i a t i o n u n i t s . T h i s has t h e advantage i n t h a t a l l d e v i a t i o n s c o r e s a r e measured i n the same u n i t s . 10. TERM: A "term" i s an i n s t r u c t i o n a l p e r i o d where c o u r s e s can be o f f e r e d i n whole o r i n p a r t . The end o f each term i s n o r m a l l y marked by t h e e v a l u a t i o n and r e p o r t i n g of s t u d e n t achievement i n each o f t h e c o u r s e s o f f e r e d d u r i n g t h a t p e r i o d . I t i s s t i p u l a t e d by t h e B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n t h a t , f o r s c h o o l s i n B r i t i s h C o lumbia, t h e r e w i l l be a t l e a s t 3 r e p o r t i n g p e r i o d s ( o r terms) d u r i n g each s c h o o l y e a r . 11. WEIGHTING: "W e i g h t i n g " r e f e r s t o t h e amount of i n f l u e n c e a p a r t i c u l a r e x e r c i s e o r assignment has i n d e t e r m i n i n g the t o t a l s c o r e f o r t h a t s u b j e c t . F o r example, an e x e r c i s e might c o n t r i b u t e 20 per c e n t toward t h e t o t a l mark f o r t h e c o u r s e , meaning t h a t 20 per c e n t o f t h e t o t a l c o u r s e mark i s d e r i v e d from t h a t p a r t i c u l a r a ssignment. W e i g h t i n g s h o u l d n o r m a l l y r e f l e c t t h e im p o r t a n c e o r the emphasis p l a c e d on the o b j e c t i v e s c o v e r e d i n t h e assignment. "NATURAL WEIGHTING" r e f e r s t o the c o n t r i b u t i o n an assignment g i v e s t o the c o u r s e mark u s i n g o n l y raw s c o r e d a t a . I n o t h e r words, f o r a g i v e n s e t o f assignments i n a p a r t i c u l a r c o u r s e , each 16 assignment w i l l c o n t r i b u t e a c e r t a i n p e r c e n t a g e o f the t o t a l s c o r e . The c o n t r i b u t i o n w i l l be d e t e r m i n e d by the v a r i a b i l i t y of t h e raw s c o r e s . T h i s raw s c o r e i n f l u e n c e i s c a l l e d " n a t u r a l w e i g h t i n g " . R e s e a r c h Q u e s t i o n s S p e c i f i c a l l y , t h e q u e s t i o n s o f i n t e r e s t a r e : 1. I s t h e r e a s i g n i f i c a n t d i f f e r e n c e between the r a n k i n g o b t a i n e d from a t e a c h e r ' s c o u r s e r e c o r d and t h e r a n k i n g o b t a i n e d from s t a t i s t i c a l l y b a l a n c i n g t h i s same r e c o r d ? 2. Do a t e a c h e r ' s c o u r s e assignments c o n t r i b u t e t h e i n t e n d e d p o r t i o n o f marks towards the c o u r s e t o t a l ? (Are the term t o t a l s , from which t h e l e t t e r grades a r e d e r i v e d , c o l l a t e d and w e i g h t e d r e l i a b l y ? ) 3. Are t e a c h e r s ' grades r e l i a b l e and f a i r f o r s t u d e n t s m i s s i n g one o r more assignments o r t e s t s ? Are s t u d e n t s compensated f a i r l y and r e l i a b l y when t h e y have been absent and have n o t r e c e i v e d a s c o r e f o r one o r more assignments o r t e s t s ? 4. Would the r e v i s e d r e s u l t s be o f such a magnitude t o j u s t i f y t h e t i m e and expense r e q u i r e d t o r e t r a i n t e a c h e r s and r e v i s e u n i v e r s i t y c o u r s e s and/or programs? In t h i s s t u d y , t h e q u e s t i o n s of i n t e r e s t r e l a t e t o the 17 methods used by t e a c h e r s t o c o l l a t e achievement d a t a f o r r e p o r t i n g p u r p o s e s . The h y p o t h e s i s t o be t e s t e d i s t h a t t e a c h e r s , f o r one r e a s o n o r a n o t h e r , do not c o l l a t e achievement s c o r e s r e l i a b l y . I t i s assumed t h a t t e a c h e r s do not c o n s i d e r t h e c a l c u l a t i o n o f c l a s s means o r s t a n d a r d d e v i a t i o n s when combining achievement s c o r e s . As a r e s u l t , the assignment s c o r e s a r e c o n t r i b u t i n g d i s p r o p o r t i o n a t e w e i g h t s toward t h e c o u r s e t o t a l s . I t i s h y p o t h e s i z e d t h a t t h e method most commonly used by t e a c h e r s t o combine c o u r s e marks i s s i m p l e summation o f t h e raw s c o r e s f o l l o w e d by r a n k i n g each s t u d e n t ' s t o t a l . I t i s a l s o h y p o t h e s i z e d t h a t the summation method of r a n k i n g w i l l be s i g n i f i c a n t l y d i f f e r e n t from r a n k i n g s o b t a i n e d from s t a t i s t i c a l l y b a l a n c e d assignment s c o r e s . R a t i o n a l e P r i o r t o the i n t r o d u c t i o n o f the l e t t e r grade r e p o r t c a r d f o r e l e m e n t a r y s c h o o l s i n 1979, el e m e n t a r y s c h o o l r e p o r t c a r d s were p r i m a r i l y a n e c d o t a l , and achievement symbols (Needs Improvement, S a t i s f a t o r y , Good), were based l a r g e l y upon s u b j e c t i v e e v a l u a t i o n . When t h e format o f t h e r e p o r t was changed t o c o m p a r a t i v e achievement and l e t t e r grades were used t o r e f l e c t t h i s achievement, t e a c h e r s were g i v e n v e r y l i t t l e i n - s e r v i c e t r a i n i n g on t h e new r e p o r t i n g p r o c e d u r e . The B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n d i d p u b l i s h b o o k l e t s 18 ( " C o n s t r u c t i o n and Uses of C l a s s r o o m T e s t s : A Resource Book For T e a c h e r s " and " G r a d i n g P r a c t i c e s : I s s u e s and A l t e r n a t i v e s " ) t o a s s i s t t e a c h e r s i n t h e i r e v a l u a t i o n o f s t u d e n t achievement. A l t h o u g h t h e s e b o o k l e t s were made a v a i l a b l e , i t appears t h a t few t e a c h e r s s t u d i e d them i n d e t a i l p r i o r t o u s i n g t h e new r e p o r t c a r d s . C o n s e q u e n t l y , i t would seem t h a t few t e a c h e r s a r e aware of t h e i m p o r t a n c e o f the mean and s t a n d a r d d e v i a t i o n when a g g r e g a t i n g assignment s c o r e s . Most t e a c h e r s use o n l y t h e raw s c o r e s t o t a l f o r d e t e r m i n i n g t h e r a n k i n g o f t h e i r s t u d e n t s . Many e d u c a t o r s seem t o f o l l o w the erroneous b e l i e f t h a t the assignment w i t h the l a r g e s t p o s s i b l e s c o r e s h o u l d c o n t r i b u t e the g r e a t e s t p e r c e n t a g e toward t h e f i n a l c o u r s e mark. The absence o f s t a t i s t i c a l a n a l y s i s and ' b a l a n c i n g ' i n the p a s t may have been p a r t i a l l y due t o a l a c k o f t r a i n i n g and p a r t i a l l y due t o a l a c k o f time and r e s o u r c e s . A s t a t i s t i c a l c a l c u l a t o r i s a l m o s t a n e c e s s i t y b u t a g r e a t d e a l o f t e a c h e r time i s s t i l l r e q u i r e d t o c a l c u l a t e the s t a n d a r d i z e d s c o r e s - time t h a t some might f i n d d i f f i c u l t t o j u s t i f y i n l i g h t of t h e many o t h e r p r o f e s s i o n a l commitments of t e a c h e r s . However, w i t h t h e advent of t h e micro-computer, "mark-keeping" programs can be d e v i s e d t o t a k e the time consumption and d r u d g e r y out of t h e s t a t i s t i c a l b a l a n c i n g o f s t u d e n t assignment s c o r e s . Even so , many t e a c h e r s seem t o a b i d e by t h e i r p r e v i o u s l y e s t a b l i s h e d g r a d i n g p r a c t i s e s . They a r e n o t aware of the need t o c o n s i d e r the e f f e c t s o f d i f f e r e n t v a r i a n c e s w i t h i n t h e i r assignment s c o r e s , and t h e r e f o r e a r e o b l i v i o u s t o the need f o r 19 improved g r a d i n g p r a c t i s e s . H i l l s (1981) s u p p o r t s t h i s n o t i o n and s t a t e s , "The s t a n d a r d d e v i a t i o n . . . i s what most t e a c h e r s would not t h i n k about w i t h o u t some t r a i n i n g " (p. 319). Perhaps i f t h e r e i s a p e r c e i v e d need f o r a change i n the way s c o r e s a r e c o l l a t e d , t e a c h e r s w i l l be r e c e p t i v e t o an a l t e r n a t e and improved method o f combining s t u d e n t s ' s c o r e s f o r r e p o r t i n g p u r p o s e s . I t i s hoped t h a t t h e outcome of t h i s s t u d y w i l l h e l p t o d e t e r m i n e i f a more e f f e c t i v e method of combining achievement s c o r e s i s needed and whether more i n - s e r v i c e and p r e - s e r v i c e t r a i n i n g i s r e q u i r e d t o improve the g r a d i n g p r a c t i s e s of t e a c h e r s . D e l i m i t a t i o n s F or t h e purposes of t h i s s t u d y , t h e sample was s e l e c t e d a t random from t e a c h e r s o f grades f o u r t o seven l e v e l who t e a c h i n a l a r g e m e t r o p o l i t a n a r e a of c e n t r a l B r i t i s h Columbia. Examples of t e a c h e r s ' r e c o r d s were s e l e c t e d from two s u b j e c t a r e a s : mathematics and s o c i a l s t u d i e s . The i n s t r u m e n t s used by the sample s u b j e c t s t o c o l l e c t s t u d e n t achievement d a t a were not e v a l u a t e d . I t was assumed t h a t a l l i n s t r u m e n t s were r e l i a b l e and v a l i d . We were i n t e r e s t e d o n l y i n t h e combined t o t a l s c o r e f o r t h e term and t h e a s s i g n e d l e t t e r grade f o r each s t u d e n t as d e t e r m i n e d from t h e c o l l a t i o n of term o r c o u r s e assignment s c o r e s . T h i s s t u d y was not concerned w i t h t h e v a l i d i t y of the 20 e d u c a t i o n a l o b j e c t i v e s f o r each o f t h e two s u b j e c t s sampled. I t s h o u l d be r e c o g n i z e d t h a t w i t h i n t h e c u r r i c u l u m s t i p u l a t e d by t h e P r o v i n c i a l M i n i s t r y o f E d u c a t i o n , c o n s i d e r a b l e f l e x i b i l i t y i s p r o v i d e d t o e l e m e n t a r y t e a c h e r s . E d u c a t o r s may s e l e c t from a v a r i e t y o f c o n t e n t , media, p e d a g o g i c a l t e c h n i q u e s , and whatever e l s e i s n e c e s s a r y , t o f u l l f i l t h e demands o f t h e c u r r i c u l u m and the needs o f t h e s t u d e n t s i n t h e most e f f e c t i v e manner. C o n s e q u e n t l y , the c o u r s e c o n t e n t may v a r y among t e a c h e r s of t h e same s u b j e c t . U n i t s and u n i t o b j e c t i v e s may a l s o be t a u g h t w i t h d i f f e r e n t emphases. T h i s f l e x i b i l i t y n u l l i f i e s any o p p o r t u n i t y t o " s t a n d a r d i z e " e l e m e n t a r y s u b j e c t s i n terms o f u n i t s , u n i t o b j e c t i v e s , a s s i g n m e n t s , and assignment emphasis on c o u r s e marks. As a r e s u l t , the v a l i d i t y , emphasis, and a p p r o p r i a t e w e i g h t i n g s o f the o b j e c t i v e s were o u t s i d e the scope of t h i s s t u d y . They were assumed t o be a p p r o p r i a t e f o r t h e grade l e v e l , t he s t u d e n t s , and t h e c u r r i c u l u m . J u s t i f i c a t i o n of the Study Why i s t h i s i s s u e i m p o r t a n t i n e d u c a t i o n ? F i r s t l y , s t u d e n t achievement s h o u l d be r e p o r t e d as r e l i a b l y and a c c u r a t e l y as p o s s i b l e . " F a u l t y g r a d i n g can be s e r i o u s l y d e t r i m e n t a l t o s t u d e n t s . I t can g i v e them m i s l e a d i n g i n f o r m a t i o n and i n t e r f e r e w i t h t h e l e a r n i n g p r o c e s s " ( H i l l s , 1981, p. 281). S e c o n d l y , p a r e n t s , p r o s p e c t i v e e mployers, and 21 the s t u d e n t s themselves v i e w r e p o r t c a r d grades as a c c u r a t e i n d i c a t o r s o f s t u d e n t achievement and use the i n f o r m a t i o n as a b a s i s f o r d e c i s i o n - m a k i n g and f o r e v a l u a t i n g p e o p l e . T h i r d l y , u n r e l i a b l e o r i n a c c u r a t e grades may l e a d t o erroneous d e c i s i o n s r e g a r d i n g b u r s a r i e s , s c h o l a r s h i p s , d i p l o m a s , o r o t h e r such awards based on g r a d e s . More i m p o r t a n t l y , improper assessment o f s t u d e n t performance may r e s u l t i n i n c o r r e c t d e c i s i o n s r e g a r d i n g a s t u d e n t ' s s u c c e s s i n a p a r t i c u l a r c o u r s e . I n a d d i t i o n , i n a c c u r a t e g r a d i n g "can m i s l e a d t h o s e who would s t u d y the i n s t r u c t i o n a l p r o c e s s t o improve i t o r t o h o l d i t a c c o u n t a b l e t o t h e s o c i e t y t h a t s u p p o r t s i t " ( H i l l s , p. 281). A f i f t h r e a s o n why t h i s i s s u e , i s i m p o r t a n t t o e d u c a t i o n i s due t o the f a c t t h a t t h e r e a r e many m i s c o n c e p t i o n s and c o n f u s i o n s among e d u c a t o r s as t o what c o n s t i t u t e s an e f f e c t i v e g r a d i n g p r o c e d u r e . H i l l s a l s o s u p p o r t s t h i s p e r c e p t i o n and s t a t e s , " t h e r e a r e many i r r a t i o n a l i d e a s about g r a d i n g t h a t seem sound u n t i l t h e y a r e c o n s i d e r e d i n d e t a i l . To a v o i d some o f t h e s e i r r a t i o n a l n o t i o n s , a t e a c h e r must l e a r n t o r e c o g n i z e what i s wrong w i t h them" (p. 280). The i m p l i c a t i o n s o f t h i s s t u d y p e r t a i n t o t e a c h e r p r e - s e r v i c e and i n - s e r v i c e t r a i n i n g . I f t h e samples o f t e a c h e r g r a d i n g p r a c t i s e s a r e s i g n i f i c a n t l y i n f e r i o r when compared t o the s t a t i s t i c a l l y b a l a n c e d and t r a n s f o r m e d s c o r e s ( T - s c o r e s ) , t h e n i t may be pr u d e n t t o examine t h e p r e - s e r v i c e t r a i n i n g t e a c h e r s r e c e i v e i n e v a l u a t i n g s t u d e n t performance 22 and achievement. I s t h i s an a r e a t h a t needs more a t t e n t i o n i n t h e p r e - s e r v i c e t r a i n i n g of t e a c h e r s ? S h o u l d u n i v e r s i t i e s make c o u r s e s i n e v a l u a t i o n mandatory f o r s t u d e n t - t e a c h e r s ? A l s o , i f i t i s apparent t h a t g r a d i n g p r a c t i s e s can be improved, s h o u l d s c h o o l d i s t r i c t s d e v e l o p i n - s e r v i c e programs t o enhance t e a c h e r g r a d i n g p r a c t i s e s ? These a r e i m p o r t a n t q u e s t i o n s i f the r e s u l t s show t h a t g r a d i n g p r a c t i s e s can be improved. Many a u t h o r s r e f l e c t t h i s i m p o r t a n c e by s u g g e s t i n g t h a t t e a c h e r s s h o u l d s t r i v e t o improve t h e i r g r a d i n g p r o c e d u r e s . H i l l s ( 1981), f o r example, s t a t e s , G r a d i n g i s a c o m p l i c a t e d problem. There a r e many p i t f a l l s ; e r r o n e o u s p r a c t i s e s abound; and t h e t e a c h e r cannot s a f e l y p r o c e e d on the assumption t h a t what was done when she was a p u p i l i s s a t i s f a c t o r y f o r t h e s t u d e n t s now i n her c a r e . Good t e a c h e r s w i l l t r y t o do a b e t t e r job t h a n was done f o r them as p u p i l s , r a t h e r t h a n s i m p l y c a r r y i n g f o r w a r d t h e m i s t a k e s o f the p a s t . (p. 280) On the o t h e r hand, i f t h e p r e s e n t g r a d i n g p r a c t i s e s a r e not s i g n i f i c a n t l y i n f e r i o r and a r e i n f a c t g e n e r a l l y as good as t h e s t a t i s t i c a l l y b a l a n c e d g r a d e s , t h e n we can c o n c l u d e t h a t t e a c h e r s , s i m i l a r t o t h o s e i n t h e sample, a r e e v a l u a t i n g and g r a d i n g s t u d e n t achievement e f f e c t i v e l y . I f t h i s i s t h e c a s e , t h e n u n i v e r s i t i e s and s c h o o l d i s t r i c t s need not c o n c e r n themselves w i t h r e v i s i n g t h e i r p r e s e n t measurement and s t u d e n t e v a l u a t i o n components o r programs. The i m p l i c a t i o n would be 2.3 t h a t t h e t r a i n i n g t e a c h e r s r e c e i v e and t h e e v a l u a t i o n t e c h n i q u e s t e a c h e r s employ a r e b o t h adequate and r e l i a b l e . 24 CHAPTER 2 Survey of t h e L i t e r a t u r e From t h e r e v i e w o f the l i t e r a t u r e , t h e r e appear t o be v e r y few e m p i r i c a l o r i n v e s t i g a t i v e s t u d i e s done t o a s s e s s the g r a d i n g p r a c t i s e s o f t e a c h e r s . A l t h o u g h t h e r e i s an abundance of t h e o r e t i c a l i n f o r m a t i o n on how t e a c h e r s s h o u l d e v a l u a t e and r e p o r t t h e achievement o f t h e i r s t u d e n t s , i t seems t h a t no one has a c t u a l l y t r i e d t o f i n d o u t i f t e a c h e r s a r e p u t t i n g t h e o r y i n t o p r a c t i s e . H i l l s (1981) r e i n f o r c e s t h i s f a c t when he s t a t e s , There i s l i t t l e e m p i r i c a l r e s e a r c h on g r a d i n g . A l t h o u g h some f a c t s a r e w e l l - e s t a b l i s h e d , t h e r e has been f a r t o o l i t t l e s t u d y of such an i m p o r t a n t t o p i c . T h e r e f o r e , everyone seems t o f e e l e n t i t l e d t o an o p i n i o n , and no da t a e x i s t t o r e f u t e most o f them. (p. 317) S e v e r a l a u t h o r s , however, have su g g e s t e d i n n o v a t i v e methods of g r a d i n g f o r s p e c i a l e d u c a t i o n a l s i t u a t i o n s o r f o r p a r t i c u l a r c o u r s e s . F o r example, D a v i d Cohen (1973) d i s c u s s e s a means o f i m p r o v i n g the e v a l u a t i o n o f the A u s t r a l i a n s c i e n c e c u r r i c u l u m . G e n s l e y (1969) p r o v i d e s an i n s i g h t i n t o a l t e r n a t e methods of e v a l u a t i n g " g i f t e d c h i l d r e n " . Many o t h e r w r i t e r s advocate a l t e r n a t e forms o f e v a l u a t i o n f o r t h e i r p a r t i c u l a r s u b j e c t a r e a o r f o r t h e i r p a r t i c u l a r s p e c i a l t y s i t u a t i o n . However, none o f the a u t h o r s o f f e r any i n s i g h t i n t o the adequacy o f t h e g r a d i n g p r a c t i s e s o f " r e g u l a r " c l a s s r o o m t e a c h e r s . Other w r i t e r s e i t h e r a t t a c k o r defend t h e use o f l e t t e r 25 grades as a symbol o f s t u d e n t achievement. For example, W.J. S t e w a r t (1975) condemns l e t t e r grades and p r o f e s s e s a new system f o r e v a l u a t i n g t h e " m u l t i - f a c e t e d a c a d e m i c - p e r s o n a l - s o c i a l growth of e l e m e n t a r y s c h o o l s t u d e n t s " (p. 174). He s t a t e s t h a t one grade does not r e p r e s e n t a d e q u a t e l y t h e c o m p l e x i t y i n a c h i l d ' s academic, s o c i a l , and e m o t i o n a l growth. I n s t e a d , he recommends a m u l t i - d i m e n s i o n a l system t h a t would accommodate (a) a wide range of g o a l s , (b) uniqueness of each s t u d e n t ' s a b i l i t i e s and needs, (c) an account f o r each c h i l d ' s need t o d e v e l o p a p o s i t i v e s e l f - c o n c e p t , and (d) a need t o p r o v i d e p a r e n t s w i t h more u s e f u l and more m e a n i n g f u l i n f o r m a t i o n r e g a r d i n g a c h i l d ' s p r o g r e s s . S t e w a r t c l a i m s t h a t h i s e v a l u a t i o n - r e p o r t i n g system would de-emphasize s k i l l and s u b j e c t m a t t e r mastery but would s t r e s s a c h i l d ' s a b i l i t y t o use s k i l l s and s u b j e c t m a t t e r f o r d e a l i n g w i t h l i f e ' s d a i l y problems. H i s m u l t i - d i m e n s i o n a l system would f o c u s o n l y upon i n d i v i d u a l s and would not s t r e s s comparisons w i t h o t h e r c l a s s members as do l e t t e r g r a d e s . W h i l e S t e w a r t p o i n t s out what may be c o n s i d e r e d by some t o be weaknesses o f t h e l e t t e r grade system, he does not address what t e a c h e r s a r e p r e s e n t l y d o i n g t o cope w i t h t h e problems he i d e n t i f i e s . He c i t e s no r e s e a r c h t o j u s t i f y h i s c l a i m t h a t a new system o f " e v a l u a t i o n - r e p o r t i n g " i s r e q u i r e d . M a r s h a l l (1971) a l s o c r i t i c i z e s t h e l e t t e r grade system. He c l a i m s t h a t grades ( o r any "codes w h i c h r e f e r t o r a n k , grade, o r p o s i t i o n on a s c a l e " ) t e n d t o be v e r y s u b j e c t i v e . He s t a t e s , "Grades s u r v i v e p r i m a r i l y because t h e y p r o v i d e 26 d i s g u i s e . Almost a n y t h i n g can be, and i s , r e a d i n t o them. L e t t e r grades . . . are g e n e r a l terms, i n t h e m s e l v e s m e a n i n g l e s s " (p. 350). M a r s h a l l i n d i c a t e s t h e b e n e f i t s o f an " i n d i v i d u a l d e s c r i p t i v e o r n o n g r a d i n g " system of e v a l u a t i o n and r e p o r t i n g t h a t would i n c l u d e p r o v i d i n g t e a c h e r s w i t h "more t i m e , g r e a t e r freedom, and new i n t e r e s t s i n t e a c h i n g " (p. 352). He a l s o s t a t e s t h a t " t e a c h e r s a r e employed t o t e a c h , not t o grade o r p s y c h o a n a l y z e " (p. 352). He c o n c l u d e s by s t a t i n g , "A t e a c h e r s h o u l d want, be a l l o w e d , and be asked t o say what he o r she means, i n words s p e c i f i c a l l y p e r t i n e n t t o a g i v e n i n q u i r y o r s i t u a t i o n " (p. 353). As w i t h S t e w a r t , M a r s h a l l makes no mention of t e a c h e r s ' c u r r e n t g r a d i n g p r a c t i s e s . A t h i r d c r i t i c i s m o f the l e t t e r grade system i s s t a t e d by R i c h a r d Curwin (1978). He c l a i m s t h a t "grades don't work" because t h e y l a c k a c c u r a c y , c o n s i s t e n c y , v a l i d i t y , r e l i a b i l i t y , and p r e d i c t a b i l i t y . He o f f e r s f o u r a l t e r n a t i v e s ( S e l f - G r a d i n g , C o n t r a c t G r a d i n g , Peer G r a d i n g , and B l a n k e t G r a ding) t o "reduce the dangers of the g r a d i n g system" (p. 6 1 ) . A l t h o u g h he s t a t e s t h a t " r e s e a r c h s t u d i e s c l e a r l y i n d i c a t e t h a t t e a c h e r s grade th e same s t u d e n t s ' work d i f f e r e n t l y " (p. 6 0 ) , he does not e l a b o r a t e on the f i n d i n g s o r the n a t u r e o f the r e s e a r c h . He ends by s t a t i n g t h a t t h e a l t e r n a t i v e s "undoubtedly w i l l be an improvement over t r a d i t i o n a l g r a d i n g systems" (p. 6 4 ) . Defenders of l e t t e r grades as symbols of s t u d e n t achievement t a k e a s t a n c e o p p o s i t e t o the a u t h o r s mentioned 27 above. Rob e r t L. E b e l (1974), f o r example, defends grades a g a i n s t the c r i t i c i s m s o f t e n aimed a t g r a d i n g . He c l a i m s t h a t g r a d i n g can be improved. "The remedy i s not t o get r i d o f grades bu t s i m p l y t o do a b e t t e r j o b o f g r a d i n g " (p. 3 ) . He i n d i c a t e s t h a t a l l a l t e r n a t i v e s t o the grade a r e l e s s c o s t e f f i c i e n t and l e s s i n f o r m a t i v e t h a n the " f a m i l i a r summary s t a t i s t i c , t h e grade." A l t h o u g h E b e l c i t e s no r e s e a r c h t o demonstrate how c l a s s r o o m t e a c h e r s c o l l e c t achievement i n f o r m a t i o n upon which t o base the assignment of g r a d e s , he does e l a b o r a t e on how grades can be improved. He, l i k e many o t h e r grade s u p p o r t e r s , a d v o c a t e s c a l c u l a t i n g the s t a n d a r d d e v i a t i o n s when a g g r e g a t i n g and w e i g h t i n g s t u d e n t assignment s c o r e s . By u s i n g a sample of s t u d e n t marks, he shows how e q u a t i n g the v a r i a t i o n o f the v a r i o u s assignments can l e a d t o improved g r a d i n g . "Thus the i n f l u e n c e of one component on a composite depends not on t o t a l p o i n t s o r mean s c o r e but on s c o r e v a r i a b i l i t y " ( E b e l , 1972, p. 350). Many o t h e r grade d e f e n d e r s a l s o s u p p o r t t h i s n o t i o n . H i l l s (1981), f o r example, s t a t e s , "The key element i n w e i g h t i n g v a r i a b l e s t h a t a r e t o be combined i s the s t a n d a r d d e v i a t i o n of t h e i r v a l u e s " (p. 320). L i k e E b e l , H i l l s a l s o uses examples t o show how c a l c u l a t i n g the s t a n d a r d d e v i a t i o n i s e s s e n t i a l f o r a c c u r a t e g r a d i n g . With c o n v e r s i o n c h a r t s , he d emonstrates how t h e s t a n d a r d d e v i a t i o n can be e s t i m a t e d w i t h o u t t h e use of a hand c a l c u l a t o r . H i l l s summarizes by s t a t i n g , " U s i n g t h e s e p r o c e d u r e s w i l l h e l p t o e x t i n g u i s h t h e c r i t i c i s m t h a t grades a r e m e a n i n g l e s s " (p. 328). 28 G r o n l u n d (1981) a l s o s u p p o r t s t h e use o f l e t t e r grades i n c l a s s r o o m s . He s t a t e s , "When l e t t e r grades a r e supplemented by o t h e r methods of r e p o r t i n g , t h e s e grades themselves become more m e a n i n g f u l " (p. 520). He t o o u t i l i z e s an example t o demonstrate how g r a d i n g can be improved by making mean s c o r e s and s t a n d a r d d e v i a t i o n s e q u a l when combining and w e i g h t i n g raw s c o r e s . He a l s o e x p r e s s e s t h e i m p o r t a n c e o f c o n s i d e r i n g the d i s p e r s i o n o f s c o r e s . "Thus, t o p r o p e r l y w e i g h t t h e components i n a composite s c o r e , t h e v a r i a b i l i t y o f the s c o r e s must be t a k e n i n t o a c c o u n t " (p. 522). However, i n h i s example, G r o n l u n d shows how t h e v a r i a b i l i t y i n each s e t of assignment s c o r e s can be equated u s i n g t h r e e d i f f e r e n t methods: (a) by u s i n g the range of s c o r e s , (b) by c o n v e r t i n g a l l s e t s o f s c o r e s t o s t a n i n e s , and ( c ) , t h e most r e f i n e d system, by u s i n g s t a n d a r d d e v i a t i o n s . Many o t h e r a u t h o r s , (such as Ahmann & G l o c k , 1981; Townsend & Burke, 1975; G u i l f o r d & F r u c h t e r , 1973) a l s o s u p p o r t the use of l e t t e r grades as a method of g r a d i n g and r e p o r t i n g s t u d e n t achievement. B u t , w h i l e a l l p r o f e s s t h e i m p o r t a n c e o f e q u a l i z i n g the v a r i a b i l i t y i n a s e t of s c o r e s , none c i t e any r e s e a r c h t h a t would i n d i c a t e whether o r not t e a c h e r s c o l l a t e and w e i g h t s c o r e s p r o p e r l y . None i n d i c a t e whether o r not p r a c t i s i n g t e a c h e r s a c t u a l l y c o n s i d e r the v a r i a t i o n i n t h e i r s c o r e s . Perhaps by i m p r o v i n g t h e methods of g r a d i n g as suggested by the d e f e n d i n g a u t h o r s t h e c o n t r o v e r s y s u r r o u n d i n g the u s e f u l n e s s o f the l e t t e r grade as a symbol of s t u d e n t achievement w i l l d i m i n i s h . 29 G r a d i n g Systems Over the l a s t f i f t y y e a r s , s e v e r a l a t t e m p t s have been made t o r e p l a c e the l e t t e r - g r a d e e v a l u a t i o n system w i t h o t h e r m a r k i n g systems o r t o a b o l i s h s c h o o l marks a l t o g e t h e r . Curwin (1978) has o u t l i n e d some of the p o s s i b l e a l t e r n a t i v e s t o e v a l u a t i o n systems i n s c h o o l s . A l t h o u g h t h e s e a l t e r n a t i v e s have not been p a r t i c u l a r l y p o p u l a r i n the p a s t , t h e y a r e mentioned here t o r e v e a l t h e o p t i o n s a v a i l a b l e t o t e a c h e r s and t o i n d i c a t e the v a r i o u s s t r e n g t h s and weaknesses of each e v a l u a t i o n system. The f i r s t a l t e r n a t i v e Curwin o f f e r s i s c a l l e d " S e l f - G r a d i n g " . I n t h i s system, th e t e a c h e r s u p p l i e s the c r i t e r i a and t h e d a t a f o r the s t u d e n t s t o use i n a s s i g n i n g t h e i r own g r a d e s . As a v a r i a n t t o s e l f g r a d i n g , t h e s t u d e n t s can a l s o g e n e r a t e t h e i r own e v a l u a t i o n c r i t e r i a , as l o n g as the c r i t e r i a remain w i t h i n t h e g u i d e l i n e s o f t h e c o u r s e . Curwin s t a t e s t h a t t h e main b e n e f i t o f t h i s approach i s t h a t s t u d e n t s a r e encouraged t o e v a l u a t e t h e i r own work and t h e r e b y l e a r n about t h e m s e l v e s and t h e i r a b i l i t i e s . However, c r i t i c s may q u e s t i o n t h e r e l i a b i l i t y o f the grades r e s u l t i n g from such a system. I t would seem t h a t the r e l i a b i l i t y i s dependent on t h e s u i t a b i l i t y , a p p r o p r i a t e n e s s , and v a l i d i t y of t h e e v a l u a t i o n c r i t e r i a as w e l l as on the r e s p o n s i b i l i t y and m a t u r i t y o f t h e s t u d e n t s . A n other o p t i o n a v a i l a b l e t o t e a c h e r s i s c a l l e d " C o n t r a c t G r a d i n g " . A l t h o u g h s i m i l a r t o s e l f - g r a d i n g , t h i s system i s 30 more s t r u c t u r e d . I t c e n t e r s around a c o n t r a c t t h a t i s a g r e e a b l e t o both s t u d e n t and t e a c h e r . The c o n t r a c t s t i p u l a t e s t h e work t o be a c c o m p l i s h e d and t h e time frame w i t h i n which the work must be completed. The f o u r elements of the c o n t r a c t a r e : (1) c r i t e r i a f o r s u c c e s s f u l c o m p l e t i o n o f the c o n t r a c t , (2) d a t a o r r e s u l t s o f work t h a t i n d i c a t e how w e l l the c r i t e r i a have been met, (3) an a n a l y s i s o f o r comparison between the d a t a and c r i t e r i a , and (4) grade (or p o i n t s ) e a r n e d . A d i s a d v a n t a g e o f c o n t r a c t g r a d i n g i s t h a t "the c o n t r a c t s sometimes become i n f l e x i b l e . S t u d e n t s might s e t u n r e a l i s t i c g o a l s - - t o o h i g h o r t o o low" ( C u r w i n , 1978, p. 62). A n o t h e r d i f f i c u l t y i s t h a t s t u d e n t s i n such a system t e n d t o do o n l y what i s s t i p u l a t e d i n the c o n t r a c t and t o a v o i d the spontaneous e x t e n s i o n o f t h e i r l e a r n i n g beyond what i s w r i t t e n . Curwin d e s c r i b e s "Peer G r a d i n g " as a n o t h e r a l t e r n a t i v e t o the l e t t e r grade system of e v a l u a t i o n . T h i s method u t i l i z e s a c o n t r a c t f o r m u l a t e d by a group of s t u d e n t s . Upon c o m p l e t i o n of t h e c o n t r a c t , the group e v a l u a t e s i t s e l f and each member r e c e i v e s t h e same group grade. The groups can be formed on the b a s i s of a s u b j e c t a r e a , a homogeneous a b i l i t y g r o u p i n g , or around a mutual i n t e r e s t . Curwin c l a i m s , S t u d e n t s o f t e n a r e i n s p i r e d t o l e a r n as energy i s g e n e r a t e d t h r o u g h w o r k i n g w i t h f r i e n d s and p e e r s on a common p r o j e c t , but a n e g a t i v e a t t i t u d e by some s t u d e n t s can d r a g the r e s t o f the group down. I f some s t u d e n t s work h a r d e r than o t h e r s , t h e y may p e r c e i v e i t as u n f a i r 3 1 t o r e c e i v e the same grade as tho s e who worked l e s s h a r d , (p. 62) A f o u r t h a l t e r n a t i v e t o g r a d i n g i s r e f e r r e d t o as " B l a n k e t G r a d i n g " . With t h i s system, "each s t u d e n t i n a c l a s s r e c e i v e s t h e same grade ( u s u a l l y an A or B) p r o v i d i n g t h a t some minimum s t a n d a r d i s met. Those s t u d e n t s who do not r e a c h the s p e c i f i e d s t a n d a r d must n e g o t i a t e a grade i n d i v i d u a l l y w i t h t h e t e a c h e r " (p. 6 4 ) . U n l i k e Curwin's o t h e r a l t e r n a t i v e s , B l a n k e t G r a d i n g r e q u i r e s a c o n t i n u o u s e v a l u a t i o n p r o c e s s as p a r t of the r e g u l a r c l a s s a c t i v i t i e s . T h i s can become a n e g a t i v e f e a t u r e of t h i s a l t e r n a t i v e s i n c e a v e r y s y s t e m a t i c and w e l l o r g a n i z e d e v a l u a t i o n system i s r e q u i r e d . S i m p l y d e c l a r i n g t h a t everyone w i l l r e c e i v e an A o r B w i t h o u t any o t h e r form o f e v a l u a t i o n would prove t o be c o u n t e r - p r o d u c t i v e ; s t u d e n t s would be l e s s m o t i v a t e d t o a c h i e v e t o t h e i r p o t e n t i a l . Another n e g a t i v e a s p e c t o f B l a n k e t G r a d i n g i s t h a t a d m i n i s t r a t o r s o f t e n have a d i f f i c u l t t i me a c c e p t i n g t h i s system o f g r a d i n g , e s p e c i a l l y when a d i s p r o p o r t i o n a t e number of A's o r B's a r e g i v e n . T h i s , a c c o r d i n g t o Curwin, can cause t h e t e a c h e r " p e r s o n a l o r p r o f e s s i o n a l harm" (p. 64). Curwin a l s o emphasizes t h a t t h i s approach t o g r a d i n g needs t o be v e r y t h o r o u g h l y e x p l a i n e d t o the a d m i n i s t r a t i o n and t o the s t u d e n t s b e f o r e b e i n g used. In r e f e r e n c e t o t h e s e f o u r a l t e r n a t e g r a d i n g schemes, Curwin s t a t e s , " I f you use any o f the a l t e r n a t i v e s p r e s e n t e d h e r e , you w i l l a v o i d many ( a l t h o u g h not a l l ) o f the problems i n t r i n s i c t o g r a d i n g , but you s t i l l must meet some of the o r i g i n a l purposes of grades -- e s p e c i a l l y the f u n c t i o n o f s u p p l y i n g . i n f o r m a t i o n t o p a r e n t s " (p. 6 4 ) . Of the more p o p u l a r g r a d i n g systems, most a r e v a r i a t i o n s on one o f t h r e e approaches: (a) p e r c e n t g r a d i n g , (b) c r i t e r i o n - r e f e r e n c e d g r a d i n g , and (c) n o r m - r e f e r e n c e d g r a d i n g . Each has been p o p u l a r i n the p a s t as a method of d e t e r m i n i n g grades i n the e d u c a t i o n a l s e t t i n g . P e r c e n t G r a d i n g , which i s c u r r e n t l y the l e a s t p o p u l a r o f the t h r e e b a s i c approaches, i n v o l v e s s c o r e s t h a t a r e averaged and c o n v e r t e d t o a p e r c e n t . W i t h t h i s method, the p e r c e n t i t s e l f may r e p r e s e n t t h e grade. For example, mathematics might be r e p o r t e d on a r e p o r t c a r d as s i m p l y "Mathematics 73%". A n o t h e r method of r e p o r t i n g p e r c e n t g r a d i n g i s t o c o n v e r t a p e r c e n t a g e t o a l e t t e r grade. An A, f o r example, may r e p r e s e n t s c o r e s r a n g i n g from 80% t o 100%. P e r c e n t G r a d i n g has not g a i n e d p o p u l a r i t y m a i n l y because of t h e o b s c u r i t y o f the g r a d e s . A s c o r e o f 80%, f o r example, t e l l s l i t t l e more than 80 p e r c e n t o f t h e q u e s t i o n s were answered c o r r e c t l y . Such a grade does not r e v e a l how much a s t u d e n t l e a r n e d i n r e l a t i o n t o i n t e n d e d outcomes nor does i t r e v e a l how he compared t o the achievement of o t h e r s t u d e n t s . With such grades i t i s d i f f i c u l t t o determine i f the 80 p e r c e n t r e p r e s e n t s e x c e p t i o n a l , a v e r a g e , o r poor achievement. I f i t i s t h e h i g h e s t s c o r e , t h e n i t i s a v e r y good s c o r e . I f i t i s near t h e mean o r median s c o r e , t h e n i t i s o n l y average, 33 and i f i t i s one o f the l o w e s t s c o r e s , t h e n i t r e p r e s e n t s poor achievement. I t i s i m p o s s i b l e t o t e l l what l e v e l o f achievement 80 p e r c e n t a c t u a l l y r e p r e s e n t s . A l t h o u g h p e r c e n t g r a d i n g was once a p o p u l a r method o f g r a d i n g , i t has l o s t i t s a p p e a l i n f a v o r of the o t h e r two major systems, namely t h e c r i t e r i o n - r e f e r e n c e d and no r m - r e f e r e n c e d approaches t o g r a d i n g . C r i t e r i o n - r e f e r e n c e d g r a d i n g C r i t e r i o n - r e f e r e n c e d g r a d i n g i n v o l v e s e x p r e s s i n g a s t u d e n t ' s achievement i n r e l a t i o n t o some p r e s p e c i f i e d s t a n d a r d s o r c r i t e r i a . These s t a n d a r d s a re u s u a l l y concerned w i t h t h e l e v e l o f mastery t o be a c h i e v e d by the s t u d e n t s and are o f t e n s t a t e d i n terms o f : (a) the s p e c i f i c t a s k s t o be performed, and (b) t h e p e r c e n t a g e o f c o r r e c t answers t o be o b t a i n e d on a t e s t t h a t measures a c l e a r l y d e f i n e d s e t o f l e a r n i n g o b j e c t i v e s . As a r e s u l t , t h e s e s t a n d a r d s a r e a b s o l u t e and a r e not r e l a t i v e t o o t h e r s t u d e n t s o r t o p r e v i o u s i n d i v i d u a l achievement. A l l s t u d e n t s who a c h i e v e a t the same l e v e l o r master the same o b j e c t i v e s a c h i e v e the same grade. "No s t u d e n t i s f a i l e d s i m p l y because he o r she a c h i e v e d lower t h a n o t h e r s t u d e n t s p r o v i d e d the s t u d e n t a c h i e v e s the c r i t e r i o n " (Gray, 1980, p. 490). G r a d i n g i n the c r i t e r i o n - r e f e r e n c e d system i s s i m p l e r t h a n many o t h e r g r a d i n g methods. Those who r e a c h the s p e c i f i e d c r i t e r i o n on each l e a r n i n g o b j e c t i v e r e c e i v e c r e d i t , and t h o s e who do n o t , f a i l . 34 F a i l u r e t o s a t i s f y the c r i t e r i o n may a l s o n e c e s s i t a t e t h a t the s t u d e n t r e p e a t the r e q u i r e d t a s k s u n t i l the o b j e c t i v e s have been a c h i e v e d . C o n s e q u e n t l y , n e a r l y a l l s t u d e n t s i n the c r i t e r i o n - r e f e r e n c e d system pas s . The c r i t e r i o n - r e f e r e n c e d g r a d i n g t e c h n i q u e w i t h t h e mastery l e a r n i n g approach, makes i t r e l a t i v e l y easy f o r t e a c h e r s t o i d e n t i f y problem a r e a s i n t h e i r c u r r i c u l u m and t o i d e n t i f y weaknesses w i t h i n i n d i v i d u a l s t u d e n t s . C r i t e r i o n - r e f e r e n c e d t e s t s , f o r example, w i l l r e v e a l which s t u d e n t s had d i f f i c u l t y ( o r f a i l e d t o meet c r i t e r i o n ) w i t h which o b j e c t i v e s . Those o b j e c t i v e s t h a t were p o o r l y g r a s p e d by a m a j o r i t y of s t u d e n t s and t h e r e f o r e need t o be r e t a u g h t w i l l a l s o be r e v e a l e d . "Thus, d e f i c i e n c i e s a r e q u i c k l y c o r r e c t e d and s t u d e n t s do not get f a r t h e r and f a r t h e r b e h i n d " (Gray, p. 490). S t u d e n t s i n t h i s system a r e p e r m i t t e d t h e o p p o r t u n i t y t o p r o g r e s s a t t h e i r own l e v e l s - some w i l l meet the c r i t e r i o n sooner than o t h e r s . However, by t h e end o f the c o u r s e o r term, most s t u d e n t s w i l l have demonstrated an a c c e p t a b l e l e v e l of performance on t h e r e q u i r e d l e a r n i n g o b j e c t i v e s . G r o n lund (1981) s t a t e s , "To e f f e c t i v e l y use a b s o l u t e l e v e l o f achievement as a b a s i s f o r g r a d i n g r e q u i r e s t h a t (1) the domain of l e a r n i n g t a s k s be c l e a r l y d e f i n e d , (2) the s t a n d a r d s of performance be c l e a r l y s p e c i f i e d and j u s t i f i e d , and (3) t h e measures of p u p i l achievement be c r i t e r i o n r e f e r e n c e d " (p. 524). W h i l e t h e s e c o n d i t i o n s a r e r e l a t i v e l y easy t o s a t i s f y i n t h e m astery approach, t h e y become v e r y d i f f i c u l t t o a p p l y t o nonmastery l e a r n i n g o b j e c t i v e s . T h i s prompts Gronlund t o s t a t e , "The c r i t e r i o n - r e f e r e n c e d system g r a d i n g i s much more complex t h a n i t f i r s t a ppears" (p. 524) Because most e d u c a t i o n a l programs i n the e l e m e n t a r y s c h o o l s e t t i n g w i l l i n c l u d e b o t h mastery and nonmastery o b j e c t i v e s , the e l e m e n t a r y t e a c h e r w i l l have c o n s i d e r a b l e d i f f i c u l t y s e t t i n g the c r i t e r i a f o r a c c e p t a b l e performance w i t h a l l o b j e c t i v e s . Even when t h e c r i t e r i a a r e e s t a b l i s h e d , Gray (1980) c l a i m s t h a t " i t i s d i f f i c u l t t o defend such s t a n d a r d s t o p a r e n t s , even i f t h e y have been s e t by c u r r i c u l u m e x p e r t s i n a c u r r i c u l u m g u i d e , because t h e y a r e based p r i m a r i l y on judgment" (p. 491). Gronlund (1981) c l a i m s t h a t t h e s e judgments a r e " l i k e l y t o be co n t a m i n a t e d by achievements t o some unknown degree. Thus, the l a c k o f r e l i a b l i l i t y i n j u d g i n g achievement i n r e l a t i o n t o p o t e n t i a l , and i n j u d g i n g degree.of improvement, would r e s u l t i n grades o f low d e p e n d a b i l i t y " (p. 524). T h i s i s v e r y s i g n i f i c a n t s i n c e demonstrated improvement and l e a r n i n g p o t e n t i a l have been w i d e l y used as a b a s i s f o r g r a d i n g t h e nonmastery components of t h e c r i t e r i o n - r e f e r e n c e d g r a d i n g system i n e l e m e n t a r y s c h o o l s . I t i s a l s o i n t e r e s t i n g t o note t h a t when r e p o r t i n g s t u d e n t achievement on nonmastery o b j e c t i v e s , rank o r d e r bas on s c o r e s o r on number o f o b j e c t i v e s completed i s o f t e n used t o p r o v i d e some measure o f r e l a t i v e p o s i t i o n w i t h i n the c l a s Rank and r e l a t i v e p o s i t i o n a r e terms t h a t connote t h e norm - r e f e r e n c e d approach t o g r a d i n g s t u d e n t achievement. 36 Norm-referenced grading T r a d i t i o n a l l y norm-referenced gr a d i n g has been the most common approach to gr a d i n g i n ed u c a t i o n . I t i n v o l v e s rank o r d e r i n g students and e x p r e s s i n g each student's achievement i n r e l a t i o n t o the achievement of h i s class-mates. U n l i k e c r i t e r i o n - r e f e r e n c e d g r a d i n g , a norm-referenced grade does not i n d i c a t e what a student has a c t u a l l y achieved; i t on l y d e s c r i b e s the r e l a t i v e p o s i t i o n w i t h i n the c l a s s . In the extreme form of norm-referenced g r a d i n g , the c l a s s serves as the normative group. Assignments and norm-referenced t e s t s are designed to produce a wide range of scores and i t i s t h i s v a r i a b l i l i t y t h a t i s used t o determine the d i s t r i b u t i o n of f i n a l grades. For example, i f a student ranks high i n the group, he w i l l r e c e i v e a high grade. I f h i s r e l a t i v e achievement i s low, he w i l l r e c e i v e a low grade. Since the gr a d i n g i s based on r e l a t i v e achievement, the s t u d e n t s ' grades are i n f l u e n c e d by both h i s performance and by the performance of the group. Consequently, a p a r t i c u l a r student w i l l - do b e t t e r i n a low a c h i e v i n g c l a s s than i n a h i g h a c h i e v i n g c l a s s . Because the norm-referenced approach assumes a normal d i s t r i b u t i o n i n the p o p u l a t i o n of students, most w i l l r e c e i v e a grade i n d i c a t i n g average achievement. A few students w i l l r e c e i v e A's and a few w i l l f a i l . However, t h i s i s more c h a r a c t e r i s t i c of normal curve gradi n g , which i s one of s e v e r a l v a r i a t i o n s of the norm-referenced technique a v a i l a b l e t o t e a c h e r s . Normal curve grading i s the extreme form of 37 n o r m - r e f e r e n c e d g r a d i n g . I n t h i s system, a f i x e d p e r c e n t a g e of s t u d e n t s r e c e i v e each grade. A commonly used grade d i s t r i b u t i o n i n t h i s system a s s i g n s the t o p 5% an A, the next 24% a B, the m i d d l e 40% a C, t h e nex t 24% a D, and t h e l o w e s t 7% a f a i l i n g grade (Gray, 1980, p. 488). A weakness w i t h t h i s a pproach i s t h a t some s t u d e n t s must r e c e i v e f a i l i n g grades r e g a r d l e s s of t h e i r l e v e l s o f achievement. F o r example, a s t u d e n t who a c h i e v e s a s c o r e of 70% may r e c e i v e a f a i l i n g grade i f h i s s c o r e i s one o f t h e l o w e s t i n t h e group. On the o t h e r hand, i f 70% r e p r e s e n t e d a h i g h s c o r e , then the s t u d e n t would r e c e i v e a h i g h grade. Another weakness i s t h a t the same grade does not n e c e s s a r i l y r e p r e s e n t t h e same l e v e l of achievement. I n one c l a s s , a s t u d e n t ' s achievement may y i e l d a B; i n a n o t h e r c l a s s , t h e same achievement might be e q u i v a l e n t t o a C o r a D. Many s t o r i e s have been t o l d how non-academics, d e r e l i c t s , and i n one American a i r f o r c e s t o r y , the s t u d e n t s ' wives were persuaded t o e n r o l l i n c o u r s e s where normal c u r v e g r a d i n g was used. The t h e o r y was t h a t t h e s e " c l a s s f i l l e r s " would absorb t h e f a i l i n g grades and t h e r e b y a l l o w the r e s t o f the s t u d e n t s t o pass t h e c o u r s e . These s t o r i e s h i g h l i g h t t h e weaknesses of normal c u r v e g r a d i n g . A nother v a r i a t i o n t o no r m - r e f e r e n c e d g r a d i n g i s " P a s s - F a i l g r a d i n g " (Gray, 1980, p. 488). T h i s system has o n l y two p o s s i b l e g r a d e s ; e i t h e r Pass o r F a i l . Any s t u d e n t who a c h i e v e s a t a l e v e l t h a t i s a t or above the minimum a c c e p t a b l e s t a n d a r d , r e c e i v e s the "Pass" grade. The r a t i o n a l e b e h i n d t h i s t y p e o f g r a d i n g i s t h a t such a system t a k e s t h e 38 p r e s s u r e o f f students and encourages them t o take courses they might normally a v o i d due t o f e a r of r e c e i v i n g a lower grade. However, s t u d i e s have shown t h a t students tend to be l e s s motivated to do t h e i r best i n courses u s i n g the p a s s - f a i l approach. "Thus, p a s s - f a i l g r a d i n g does not f u l f i l l any of the purposes of grades—communication, m o t i v a t i o n , and p r e d i c t i o n . . . . In g e n e r a l i t s use should be discouraged" (Gray, 1980, p. 489). A more s a t i s f a c t o r y v a r i a t i o n t o norm-referenced grading i s "standard score g r a d i n g " . T h i s technique i n v o l v e s c o n v e r t i n g raw scores from each t e s t or assignment t o s t a n d a r d i z e d z s c o r e s , or T s c o r e s , averaging the T scores and a s s i g n i n g grades based on the average T score v a l u e . For example, average T scores of 65 and above might r e c e i v e an A and scores between 5 5 and 6 4 might r e c e i v e a B and so on. The main advantage of t h i s g r a d i n g system i s t h a t a r i t h m e t i c o p e r a t i o n s can be performed on the .T s c o r e s . T h i s i s an important f e a t u r e because "we can m e a n i n g f u l l y combine scores i n a way t h a t a d j u s t s a p p r o p r i a t e l y f o r the f a c t t h a t the v a r i o u s c o n t r i b u t i n g t e s t s had d i f f e r e n t degrees of v a r i a b i l i t y " (Gray, 1980, p. 489). Another advantage to standard score g r a d i n g i s t h a t teachers can weight assignments d i f f e r e n t i a l l y . Seldom are a l l t e s t s and assignments co n s i d e r e d to be of equal importance when combining marks. Standard score g r a d i n g allows teachers to a s s i g n a d e s i r e d weight to each of the c o n t r i b u t i n g assignments before combining the s c o r e s . However, "Good judgment has to enter 39 i n t o t h e g r a d i n g p r o c e s s . . . . The p o i n t t o keep i n mind i s t h a t n o r m - r e f e r e n c e d g r a d i n g s i m p l y rank o r d e r s the s t u d e n t s ; i t i n d i c a t e s n o t h i n g about a c t u a l l e v e l o f achievement" (Gray, p. 490). One of the major c r i t i c i s m s o f the extreme form o f n o r m - r e f e r e n c e d g r a d i n g i s t h e f a c t t h a t a f i x e d p e r c e n t a g e o f s t u d e n t s r e c e i v e each grade. As s t a t e d p r e v i o u s l y , t h i s i m p l i e s t h a t a s m a l l p e r c e n t a g e of s t u d e n t s w i l l always r e c e i v e f a i l i n g g r a d e s , r e g a r d l e s s o f the l e v e l s o f achievement. However, t o overcome the n e g a t i v e a s p e c t s , t h e B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n (1979) o f f e r s a n o t h e r v a r i a t i o n t o n o r m - r e f e r e n c e d g r a d i n g . I t s u g g e s t s t h e use o f s t a n d a r d s c o r e g r a d i n g but w i t h a grade d i s t r i b u t i o n based on s t u d e n t a b i l i t y r a t h e r t h a n on a normal c u r v e . The methods of e s t a b l i s h i n g t h e grade d i s t r i b u t i o n makes t h i s approach d i f f e r e n t from t h e o t h e r methods of n o r m - r e f e r e n c e d g r a d i n g a l r e a d y mentioned. E s s e n t i a l l y , an a b i l i t y - b a s e d g r a d i n g d i s t r i b u t i o n means t h a t the grade d i s t r i b u t i o n f o r a g i v e n c l a s s r e f l e c t s t h e a b i l i t y l e v e l s o f the s t u d e n t s i n t h a t p a r t i c u l a r c l a s s . "The p r o p o r t i o n o f d i f f e r e n t grades a s s i g n e d i s p r e d e t e r m i n e d by the a b i l i t y l e v e l s w i t h i n the group" ( B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n , p. 2 6 ) . T h e r e f o r e , i t i s p o s s i b l e t h a t , f o r a p a r t i c u l a r c l a s s , a l a r g e r number of grades may be g i v e n a t one l e v e l than i n any o t h e r . Some d i s t r i b u t i o n s may have l e v e l s where no grades a r e awarded. To c r e a t e such an a b i l i t y - b a s e d d i s t r i b u t i o n r e q u i r e s t h e e s t i m a t i o n of t h e 4 0 g e n e r a l a b i l i t y l e v e l of s t u d e n t s w i t h i n t h e c l a s s . N o r m a l l y , the n e c e s s a r y i n f o r m a t i o n can be o b t a i n e d from s t a n d a r d i z e d achievement t e s t s c o r e s o r from g e n e r a l a b i l i t y t e s t s , o r b o t h . I n f o r m a t i o n on s t u d e n t a b i l i t y can a l s o be o b t a i n e d from p a s t performance r e c o r d s , such as p r e v i o u s r e p o r t c a r d s o r s c h o o l r e c o r d s . U s i n g t h i s i n f o r m a t i o n , a g r a d i n g d i s t r i b u t i o n can be d e t e r m i n e d f o r each c l a s s . I t i s i m p o r t a n t t o p o i n t out t h a t the grades or p e r c e n t i l e r a n k i n g s a s s i g n e d t o i n d i v i d u a l s t u d e n t s d u r i n g t h e e s t i m a t i o n p r o c e s s are not t o be c o n s i d e r e d as e x p e c t a t i o n l e v e l s f o r t h e s t u d e n t s so c a t e g o r i z e d . For example, s t u d e n t s who r e c e i v e A's may not be t h e same s t u d e n t s who s c o r e d above the 96 p e r c e n t i l e on the s t a n d a r d i z e d t e s t . W h i l e t h e use of T s c o r e s m a i n t a i n s t h e advantages of s t a n d a r d s c o r e g r a d i n g , a grade d i s t r i b u t i o n based on s t u d e n t a b i l i t y overcomes the problems of p r e d e t e r m i n e d f a i l u r e s . These methods, combined w i t h t h e use of " n a t u r a l b r e a k s " i n the d i s t r i b u t i o n of c o l l a t e d s c o r e s t o determine l e t t e r grade c u t - o f f p o i n t s , " o f f e r s a v e r y s a t i s f a c t o r y approach t o t h e assignment of l e t t e r g r a d e s " ( B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n , p. 2 7 ) . T h i s v a r i a n t t o the n o r m - r e f e r e n c e d g r a d i n g p r o c e d u r e has been adopted by the s c h o o l d i s t r i c t i n v o l v e d i n t h i s s t u d y . There a r e s e v e r a l r e a s o n s why t h e n o r m - r e f e r e n c e d approach t o g r a d i n g has remained one o f the most p o p u l a r methods of e v a l u a t i n g and g r a d i n g s t u d e n t achievement. F i r s t l y , grades based on r e l a t i v e achievement appear t o be 41 more r e a d i l y u n d e r s t o o d by p a r e n t s , a d m i n i s t r a t o r s , p r o s p e c t i v e e mployers, and o t h e r s who have an i n t e r e s t i n s t u d e n t achievement. R e l a t i v e p o s i t i o n i n a group o r c l a s s , as i n d i c a t e d by the l e t t e r grade, can be comprehended q u i c k l y and e a s i l y . Very l i t t l e i n f o r m a t i o n needs t o accompany the l e t t e r grade t o e x p l a i n i t s meaning; due t o i t s f r e q u e n t use and f a m i l i a r i t y i t i s al m o s t common knowledge t h a t an A r e p r e s e n t s o u t s t a n d i n g work w h i l e a D o r E i n d i c a t e s t h a t the s t u d e n t has e x p e r i e n c e d c o n s i d e r a b l e d i f f i c u l t y . Other g r a d i n g systems r e q u i r e c h e c k l i s t s o r g r a d i n g c r i t e r i a t o p r o v i d e meaning t o the l e t t e r g r a d e s . These l i s t s need t o be i n t e r p r e t e d and u n d e r s t o o d b e f o r e f u l l v a l u e can be d e r i v e d from t h e a s s o c i a t e d l e t t e r grade. S e c o n d l y , p a r e n t s g e n e r a l l y a r e i n t e r e s t e d i n knowing how w e l l t h e i r c h i l d performed r e l a t i v e t o the o t h e r s i n the c l a s s . The c o m p e t i t i v e n a t u r e of p e o p l e , the same s o r t t h a t d r i v e s them t o "keep up w i t h t h e Jo n e s " , emerges when d i s c u s s i n g t h e i r c h i l d ' s academic achievement. F or example, th e y might want t o know i f t h e i r c h i l d ' s A was a h i g h A o r a low A o r how many o t h e r s i n the c l a s s r e c e i v e d an A. Pe o p l e o f t e n appear t o use r e l a t i v e p o s i t i o n as an i n d i c a t o r o f s u c c e s s , whether i t be i n b u s i n e s s , i n d u s t r y , s o c i e t y , o r e d u c a t i o n . The f u r t h e r up t h e r a n k i n g s c a l e a p e r s o n i s t h e more s u c c e s s f u l he i s p e r c e i v e d t o be. A t h i r d r e a s o n why n o r m - r e f e r e n c e d g r a d i n g methods have been more common i s due t o the p r e d i c t i v e q u a l i t i e s o f t h i s g r a d i n g system. Many of t h e uses o f grades a r e p r e d i c t i v e i n 42 n a t u r e . F o r example, employers want t o be a b l e t o p r e d i c t t h e a b i l i t i e s and s u c c e s s of t h e s t u d e n t s t h e y h i r e . C o l l e g e s and u n i v e r s i t i e s use grades t o p r e d i c t the s u c c e s s of t h e i r a p p l i c a n t s . " I f a l l s t u d e n t s r e c e i v e d n o t h i n g but p a s s - f a i l g r a d e s , i t would be v e r y d i f f i c u l t f o r c o l l e g e a d m i s s i o n s p e r s o n n e l , f o r example, t o s e l e c t from among a p p l i c a n t s s i n c e a l l a p p l i c a n t s would have the same g r a d e s " (Gray, 1980, p. 489). Because the n o r m - r e f e r e n c e d system attemps t o produce maximum v a r i a b i l i t y of s c o r e s , t h e c o r r e l a t i o n c o e f f i c i e n t i s enhanced and becomes more u s e f u l f o r p r e d i c t i o n purposes t h a n c o e f f i c i e n t s d e r i v e d from o t h e r g r a d i n g systems. A f o u r t h r e a s o n why n o r m - r e f e r e n c e d g r a d i n g has m a i n t a i n e d i t s p o p u l a r i t y i s due t o the r e l a t i v e ease o f s e t t i n g s t a n d a r d s and grade d i s t r i b u t i o n s . F o r e d u c a t i o n a l i n s t i t u t i o n s u s i n g t h e extreme form o f n o r m - r e f e r e n c e d g r a d i n g , t h e normal c u r v e i s used t o e s t a b l i s h t h e grade d i s t r i b u t i o n . I n t h i s c a s e , t h e s t a n d a r d s and grade d i s t r i b u t i o n s between v a r i o u s c l a s s e s o r c o u r s e s remain s i m i l a r r e g a r d l e s s o f c o u r s e c o n t e n t o r s t u d e n t a b i l i t y l e v e l s . C o n s e q u e n t l y , t h e n o r m - r e f e r e n c e d g r a d i n g system i s e a s i e r t o a d m i n i s t e r and t o a p p l y . Once t h e f i n a l s c o r e r a n k i n g has been a c h i e v e d , t h e number of grades t o be a l l o t t e d a t each l e v e l can be e a s i l y d e t e r m i n e d . Even t h e v a r i a t i o n s of n o r m - r e f e r e n c e d g r a d i n g , t h o s e t h a t do not s t r i c t l y adhere t o the normal c u r v e d i s t r i b u t i o n , a r e more e a s i l y a d m i n i s t e r e d t h a n most of t h e o t h e r g r a d i n g t e c h n i q u e s . One of t h e main d i s a d v a n t a g e s of t h e c r i t e r i o n - r e f e r e n c e d systems, f o r 43 example, i s the d i f f i c u l t y o f f o r m u l a t i n g t h e c r i t e r i o n and s t a n d a r d s f o r a c c e p t a b l e performance. With a n o r m - r e f e r e n c e d approach, t h e s e t a s k s a r e much s i m p l e r ; the a d m i n i s t r a t i o n o f grades i s more m e c h a n i c a l and m a t h e m a t i c a l , r e q u i r i n g l e s s t i me and judgment from t h e t e a c h e r . A n o t h e r r e a s o n f o r t h e p o p u l a r i t y o f n o r m - r e f e r e n c e d approaches i s due t o t h e m o t i v a t i o n a l a s p e c t s . One of the main purposes of grades i s t o m o t i v a t e s t u d e n t s t o p e r f o r m as c l o s e t o t h e i r p o t e n t i a l as p o s s i b l e . Norm-referenced g r a d i n g has been shown t o m o t i v a t e s t u d e n t s more e f f e c t i v e l y t han do most o t h e r methods o f g r a d i n g . F o r example, c r i t e r i o n - r e f e r e n c e d approaches t o g r a d i n g c r e a t e l e s s m o t i v a t i o n i n s t u d e n t s . Gray (1980) s t a t e s , " i f a l l c o u r s e s were p a s s - f a i l , s t u d e n t m o t i v a t i o n (and subsequent achievement) would d e c r e a s e a c r o s s t h e board" (p. 492). Norm-referenced systems can c a p i t a l i z e on the c o m p e t i t i v e n a t u r e o f s t u d e n t s and m o t i v a t e them t o do t h e i r b e s t . I n B r i t i s h C olumbia, the M i n i s t r y of E d u c a t i o n s t i p u l a t e d t h a t a l l p u b l i c s c h o o l s would r e p o r t c o m p a r a t i v e achievement w i t h a seven l e v e l l e t t e r grade system i n grades f o u r t o t w e l v e . T h i s d e c i s i o n was based on the r e s u l t s of a survey conducted by t h e M i n i s t r y i n the l a t e 1970s. T h i s method of r e p o r t i n g n e c e s s i t a t e d t h e use o f some form of n o r m - r e f e r e n c e d g r a d i n g p r o c e d u r e w i t h i n t h e s c h o o l system. B o o k l e t s were p u b l i s h e d by t h e M i n i s t r y t o p r o v i d e g u i d e l i n e s and t o a s s i s t t e a c h e r s i n u t i l i z i n g n o r m - r e f e r e n c e d t e c h n i q u e s i n t h e i r c l a s s r o o m s . The r e p o r t i n g and g r a d i n g methods used i n the e l e m e n t a r y s c h o o l s o f the s c h o o l d i s t r i c t i n v o l v e d i n t h i s s t u d y f o l l o w t h e s e g u i d e l i n e s and a r e o u t l i n e d i n c h a p t e r 3 t h i s s t u d y . 45 CHAPTER 3 Methodology I t was the purpose of t h i s s t u d y t o e v a l u a t e the g r a d i n g methods used by 3 7 randomly s e l e c t e d t e a c h e r s . I n f o r m a t i o n on t h e i r methods of g r a d i n g was c o l l e c t e d i n t h r e e ways: (a) by way o f a q u e s t i o n n a i r e , (b) by h a v i n g the s u b j e c t s w e i g h t , t o t a l , and rank a h y p o t h e t i c a l s e t o f raw achievement s c o r e s , and (c) by h a v i n g the s u b j e c t s submit t h e i r mathematics and s o c i a l s t u d i e s c l a s s r e c o r d s h e e t s f o r one r e p o r t i n g p e r i o d . Sample and P o p u l a t i o n The sample was s e l e c t e d a t random from a l i s t o f male and female t e a c h e r s who t e a c h a t t h e grades f o u r t o seven l e v e l i n a l a r g e m e t r o p o l i t a n a r e a o f c e n t r a l B r i t i s h Columbia. The a c c e s s i b l e p o p u l a t i o n from which the sample was s e l e c t e d c o n s i s t e d o f t e a c h e r s who t e a c h a t the i n t e r m e d i a t e l e v e l i n e l e m e n t a r y s c h o o l s . At t h i s l e v e l , t e a c h e r s c u s t o m a r i l y t e a c h a l l of t h e M i n i s t r y p r e s c r i b e d s u b j e c t s t o one c l a s s . The s i z e s of most e l e m e n t a r y s c h o o l c l a s s e s n o r m a l l y range from twenty f i v e t o t h i r t y s t u d e n t s . Teachers are r e s p o n s i b l e f o r a d m i n i s t e r i n g the p r e s c r i b e d c u r r i c u l a , and f o r m e a s u r i n g , e v a l u a t i n g , g r a d i n g , and r e p o r t i n g s t u d e n t achievement. I n t h e e l e m e n t a r y s c h o o l s e t t i n g , measurement i n s t r u m e n t s c o n s i s t p r i m a r i l y of t e a c h e r made t e s t s and a s s i g n m e n t s . 46 Q u e s t i o n n a i r e A q u e s t i o n n a i r e was g i v e n t o each member of the sample of t e a c h e r s i n o r d e r t o g a i n some i n s i g h t i n t o t h e i r t e a c h i n g e x p e r i e n c e , t h e i r knowledge of c o l l a t i n g achievement s c o r e s f o r g r a d i n g p u r p o s e s , and t h e i r knowledge o f b a s i c s t a t i s t i c s ( p a r t i c u l a r l y s t a n d a r d d e v i a t i o n ) . A L i k e r t - t y p e s c a l e was used f o r some q u e s t i o n n a i r e items and the c l o s e d form format was used t o o b t a i n t h e demographic and g r a d i n g p r o c e d u r e i n f o r m a t i o n (Appendix A ) . Such q u e s t i o n s as t h e f o l l o w i n g were asked: 1. My c o l l e a g u e s a t t h i s s c h o o l t a k e r e p o r t c a r d marks s e r i o u s l y . 2. L e t t e r grades a r e an e f f e c t i v e method o f i n f o r m i n g p a r e n t s of t h e i r c h i l d ' s p r o g r e s s and achievement. 3. To the b e s t o f my knowledge, p a r e n t s a r e g e n e r a l l y s a t i s f i e d w i t h t h e l e t t e r grade system of r e p o r t i n g . 4. I f e e l v e r y c o m f o r t a b l e w i t h the l e t t e r grades I award s t u d e n t s . 5. I am c o n f i d e n t t h a t t h e l e t t e r grades I a s s i g n a r e a c c u r a t e and r e l i a b l e . 6. I would l i k e t o l e a r n more about c o l l a t i n g and a s s i g n i n g l e t t e r grades t o raw s c o r e s . 7. More i n - s e r v i c e s e s s i o n s on g r a d i n g and r e p o r t i n g s h o u l d be o f f e r e d . 8. U n i v e r s i t i e s s h o u l d o f f e r more p r e - s e r v i c e i n s t r u c t i o n i n e f f e c t i v e g r a d i n g and r e p o r t i n g . 9. The r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o be communicated t o p a r e n t s , s t u d e n t s , e t c . 10. I n . t h e space p r o v i d e d , b r i e f l y d e s c r i b e how you combine assignment and t e s t s c o r e s t o determine l e t t e r g r a d e s . 11. I n r e f e r e n c e t o the method d e s c r i b e d i n #10, which o t h e r s u b j e c t s a r e a l s o graded t h i s way? ( a l l s u b j e c t s , mathematics, lanauaae a r t s , s c i e n c e , s o c i a l s t u d i e s , a r t , P.E., m u s i c , French) 12. I n the space p r o v i d e d , b r i e f l y d e s c r i b e how you a r r i v e a t a f i n a l term s c o r e f o r a s t u d e n t who has been absent f o r one o r more t e s t s o r a s s i g n m e n t s . In o t h e r words, how do you compensate f o r a l e g i t i m a t e l y m i s s e d assignment? The q u e s t i o n n a i r e a l s o c o l l e c t e d d a t a u s i n g t h e c o m p l e t i o n f o r m a t . Such q u e s t i o n s as the f o l l o w i n g were asked: 1. Which i s your age c a t e g o r y ? ( <25,25-30 , 31-40 ,41-50 , 51-60 ,>60) ( y e a r s ) 2. How many y e a r s t e a c h i n g e x p e r i e n c e do you have? (combine b o t h p u b l i c and p r i v a t e ) ( 0-2,3-5,6-10,11-15,16-20,>20) 3. How many c o u r s e s i n s t a t i s t i c s o r measurement and e v a l u a t i o n do you have? ( d i s r e g a r d u n i t v a l u e ) (0,1,2,3,>3) 4. What i s your t e a c h i n g assignment? ( f u l l t i m e , p a r t t i m e ) 48 5. I f your answer t o #4 was " p a r t t i m e " , e x p l a i n your t e a c h i n g assignment ( p e r c e n t a g e o f t i m e , s u b j e c t s , e t c . ) I n c l u d e d w i t h t h e q u e s t i o n n a i r e was a sample of f i v e f i c t i t i o u s achievement s c o r e s f o r f i v e f i c t i t i o u s s t u d e n t s and the w e i g h t t h a t each assignment c o n t r i b u t e d t o the f i n a l c o u r s e s c o r e . The r e s p o n d e n t s were asked t o c o l l a t e and rank the s c o r e s u s i n g the method most f a m i l i a r t o them. T h i s s e r v e d as a g r a p h i c d e m o n s t r a t i o n of the c o l l a t i n g methods commonly used by t e a c h e r s . F i c t i t i o u s Data The h y p o t h e t i c a l d a t a c o n s i s t e d o f s t u d e n t names, t h e i r raw s c o r e s on each of the f i c t i t i o u s a s s i g n m e n t s , and the w e i g h t each assignment was t o c o n t r i b u t e t o the t o t a l term s c o r e . The t e a c h e r s were asked t o w e i g h t , c o l l a t e , and rank the assignment s c o r e s u s i n g t h e i r accustomed methods. S u b s e q u e n t l y , t h e s e r a n k i n g s were a n a l y z e d and compared t o the r a n k i n g s o f the same d a t a a f t e r the raw assignment s c o r e s had been s t a t i s t i c a l l y b a l a n c e d . For comparison p u r p o s e s , t h e raw s c o r e s on each of t h e f i c t i t i o u s a s s ignments were t r a n s f o r m e d t o T-scores and b a l a n c e d p r i o r t o b e i n g w e i g h t e d and c o l l a t e d . Each s t u d e n t ' s r e v i s e d t o t a l was t h e n ranked and compared t o the t e a c h e r s ' r a n k i n g of t h e raw s c o r e d a t a . By a n a l y z i n g the r e s u l t s of 4 9 the f i c t i t i o u s d a t a , i t was p o s s i b l e t o determine the p o t e n t i a l f o r e r r o r and l a c k o f r e l i a b i l i t y when t e a c h e r s c o l l a t e s t u d e n t achievement s c o r e s . An example of t h e h y p o t h e t i c a l d a t a s u p p l i e d t o the s u b j e c t s i s l i s t e d i n Ta b l e I. The b r a c k e t t e d i n f o r m a t i o n (weighted t o t a l s c o r e ) was not s u p p l i e d t o the res p o n d e n t s and i s l i s t e d here o n l y t o i n d i c a t e the d i s t r i b u t i o n o f t h e raw s c o r e s . T a b l e I I d i s p l a y s the same d a t a a f t e r i t has been b a l a n c e d so t h a t each assignment now has a mean of 50 and a s t a n d a r d d e v i a t i o n of 10. S t a t i s t i c a l b a l a n c i n g was a p p l i e d b e f o r e the assignment s c o r e s were w e i g h t e d and c o l l a t e d . I n t h i s s t u d y , t h e r a n k i n g of the "Weighted T o t a l S c o r e s " . was -the main p o i n t of i n t e r e s t . Of p a r t i c u l a r i n t e r e s t i n t h i s example i s how t h e s t u d e n t r a n k i n g s i n T a b l e I I d i f f e r from t h o s e i n T a b l e I . T h i s demonstrates how, even w i t h o n l y f i v e s t u d e n t s , s t a t i s t i c a l b a l a n c i n g can have a n o t i c e a b l e e f f e c t on s t u d e n t r a n k i n g s . With a l a r g e r number o f s t u d e n t s , the e f f e c t may be even more pronounced. S i n c e t h e r a n k i n g s a r e used t o dete r m i n e l e t t e r g r a d e s , i t i s i m p e r a t i v e t h a t the c o l l a t e d s c o r e s be as r e l i a b l e as p o s s i b l e . Record Sheets Examples o f t e a c h e r s ' r e c o r d s were s e l e c t e d from two s u b j e c t a r e a s : mathematics and s o c i a l s t u d i e s . Each o f the two r e c o r d s h e e t s o b t a i n e d from the re s p o n d e n t s r e v e a l e d t h e s t u d e n t s ' names, t h e raw s c o r e s f o r the v a r i o u s assignments and t e s t s , t h e s t u d e n t s ' c o l l a t e d s c o r e s and l e t t e r g r a d e s , as 50 ASSIGNMENTS raw s c o r e s WEIGHTED NAME 1 2 3 4 5 TOTAL SCORE RANK TOTAL SCORE RANK 1 . S t u d e n t A 50 55 65 57 95 [64.4] [3] [72.7] [1] 2 . S t u d e n t B 69 60 58 80 83 [70.0] [1] [71.7] [2] 3 . St u d e n t C 45 72 52 71 75 [63.0] [4.5] [66.4] [3] 4 . St u d e n t D 75 95 45 83 25 [64.6] [2] [53.8] [4] 5 . S t u d e n t E 73 83 60 63 36 [63.0] [4.5] [45.8] [5] WEIGHTS ( %) 10 20 20 10 40 [ X m ] [ 6 2 ] [ 7 3 ] [ 5 6 ] [ 7 1 ] [ 6 3 ] [SD] [ 1 4 ] [ 1 6 ] [ 8 ] [ 1 1 ] [ 3 1 ] T a b l e I - F i c t i t i o u s d a t a t o be c o l l a t e d and ranked ASSIGNMENTS T-scores WEIGHTED NAME 1 2 3 4 5 TOTAL SCORE RANK TOTAL SCORE RANK 1 . Student A 41 39 61 37 60 47.6 4 51. 8 2 2 . Student B 55 42 53 58 56 52.8 1 52.7 1 3 . Student C 38 49 45 50 54 47.2 5 49 . 2 3 4 . Student D 59 64 36 61 38 51.6 2 47.2 5 5 . Student E 58 56 55 43 41 50.6 3 48.7 4 WEIGHTS (%) 10 20 20 10 40 Xm 50 50 50 50 50 SD 10 10 10 10 10 Table I I - F i c t i t i o u s d a t a a f t e r b e i n g s t a t i s t i c a l l y b a l a n c e d 51 w e l l as the w e i g h t i n g t h a t each assignment was t o c o n t r i b u t e t o t h e c o l l a t e d s c o r e s . At the end of a term ( j u s t p r i o r t o the d i s t r i b u t i o n of r e p o r t c a r d s ) , c o p i e s of t h e r e c o r d s h e e t s f o r bo t h s u b j e c t a r e a s were c o l l e c t e d from the r e s p o n d e n t s . Each c l a s s r e c o r d sheet was s u b j e c t e d t o the f o l l o w i n g p r o c e d u r e . With the a i d of a computer, t h e assignment raw s c o r e s were s t a t i s t i c a l l y b a l a n c e d and c o n v e r t e d t o T - s c o r e s . They were then w e i g h t e d , c o l l a t e d and ranked u t i l i z i n g t he same assignment w e i g h t i n g s as used by the t e a c h e r . The r e v i s e d t o t a l s and r a n k i n g s were r e t u r n e d t o the o r i g i n a l respondent t o be r e g r a d e d . I t was hoped t h a t , by h a v i n g the respondent r e g r a d e t h e b a l a n c e d r e s u l t s , a l l p r o c e d u r a l and s u b j e c t i v e f a c t o r s a f f e c t i n g t h e s t u d e n t s ' grades would be h e l d c o n s t a n t f o r b o t h the o r i g i n a l and the b a l a n c e d s e t s of d a t a . The d i f f e r e n c e s i n the r a n k i n g s o f the raw s c o r e and the t r a n s f o r m e d s c o r e s were a n a l y z e d i n two ways. F i r s t l y , t he r a n k i n g s were compared t o determine i f t h e r e were s t a t i s t i c a l l y s i g n i f i c a n t d i f f e r e n c e s i n the c o r r e l a t i o n between the two r a n k i n g s . S e c o n d l y , l e t t e r grades r e s u l t i n g from b o t h the raw s c o r e and the T-score r a n k i n g s were compared t o d e t e r m i n e how many s t u d e n t s r e c e i v e d a change i n l e t t e r grade. The r e s u l t s from t h e s e a n a l y s e s were c o m p i l e d and t a b u l a t e d . A complete documentation of the r e s u l t s can be found i n c h a p t e r s f o u r and f i v e o f t h i s s t u d y . 52 G r a d i n g P o l i c y I n o r d e r t o promote c o n s i s t e n c y i n the a l l o c a t i o n of l e t t e r grades i n the e l e m e n t a r y s c h o o l s e t t i n g , the s c h o o l d i s t r i c t used i n t h i s s tudy has i n e f f e c t a p o l i c y o u t l i n i n g how t h e l e t t e r grade f r e q u e n c y d i s t r i b u t i o n s h o u l d be d e t e r m i n e d f o r each c l a s s . E s s e n t i a l l y , the p o l i c y s t a t e s t h a t t h e l e t t e r grade d i s t r i b u t i o n s h o u l d be based o n . s t u d e n t a b i l i t y e s t i m a t e s d e r i v e d from t h e n a t i o n a l p e r c e n t i l e s o f s t a n d a r d i z e d achievement t e s t r e s u l t s . T h i s i s a c h i e v e d by a d m i n i s t e r i n g the s t a n d a r d i z e d Canadian Achievement T e s t (CAT) t o the i n t e r m e d i a t e grades once each s c h o o l y e a r . The r e s u l t i n g n a t i o n a l p e r c e n t i l e s f o r each c l a s s a r e used t o a s s i s t i n the e s t a b l i s h m e n t of an approximate grade d i s t r i b u t i o n f o r t h a t c l a s s . T a b l e I I I i l l u s t r a t e s the g u i d e l i n e s t h a t a r e s u p p l i e d t o t e a c h e r s i n o r d e r t o a s s i s t them i n r e l a t i n g n a t i o n a l p e r c e n t i l e s t o l e t t e r grade d i s t r i b u t i o n s . . I f , f o r example, i n a g i v e n c l a s s 3 s t u d e n t s s c o r e d between 96 and 100 on t h e CAT n a t i o n a l p e r c e n t i l e , 7 between 7 6 and 95, 4 between 61 and 75, 9 between 41 and 60, 4 between 2 6 and 40, 3 between 6 and 25, and none between 0 and 5, then a p p r o x i m a t e l y t h r e e s t u d e n t s s h o u l d r e c e i v e an A, a p p r o x i m a t e l y 7 s h o u l d r e c e i v e a B, and so on. The d i s t r i c t g u i d e l i n e s s u ggest o n l y an approximate d i s t r i b u t i o n as t e a c h e r s a r e urged t o c o n s i d e r a l s o the s t a n d a r d s of a c c e p t a b l e achievement f o r t h e i r p a r t i c u l a r grade when a l l o c a t i n g l e t t e r g r a d e s . However, i f the g u i d e l i n e s were t o be s t r i c t l y f o l l o w e d , t h e n t h e t h r e e s t u d e n t s w i t h t h e h i g h e s t NATIONAL PERCENTILE LETTER GRADE 96 - 100 A 76 - 95 B 61 - 75 C+ 41 - 60 C 26 - 40 C- 6 - 25 D 0 - 5 E Ta b l e I I I - L e t t e r grade d i s t r i b u t i o n g u i d e l i n e s c u m u l a t i v e s c o r e s from t h e v a r i o u s term assignments would r e c e i v e the A's, w h i l e the n e x t seven h i g h e s t would r e c e i v e the B's, e t c e t e r a . T h i s p r o c e d u r e " t a k e s i n t o c o n s i d e r a t i o n the a b i l i t y l e v e l s o f the p u p i l s i n a p a r t i c u l a r grade o r c l a s s " ( B r i t i s h Columbia M i n i s t r y of E d u c a t i o n , 1979, p. 26) However, the s c h o o l d i s t r i c t p o l i c y a l s o s t a t e s , " I n c o n s i d e r i n g the c u m u l a t i v e marks f o r each s u b j e c t , " n a t u r a l b r e a k s " were used f u r t h e r t o e s t a b l i s h l e t t e r grade d i s t r i b u t i o n s " ( S c h o o l D i s t r i c t P o l i c i e s and R e g u l a t i o n s ) . T h i s approach i s a l s o advocated by the B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n i n t h e i r b o o k l e t on g r a d i n g p r a c t i s e s . " N a t u r a l b r e a k s " r e f e r t o the gaps t h a t appear i n the ranked d i s t r i b u t i o n o f s t u d e n t s ' c u m u l a t i v e s c o r e s . The gaps can b used as grade c u t - o f f p o i n t s t o s e p a r a t e one grade c a t e g o r y 54 from a n o t h e r . Combining the n a t u r a l b reaks and the group a b i l i t y methods t o determine the l e t t e r grade d i s t r i b u t i o n a l l o w s " t e a c h e r s t o make j u d i c i o u s d e c i s i o n s as t o what c o n s t i t u t e s h i g h and low achievement" ( B r i t i s h Columbia M i n i s t r y o f E d u c a t i o n , 1979, p. 2 6 ) . U s i n g t h e s e p r o c e d u r e s t o e s t a b l i s h t h e l e t t e r grade d i s t r i b u t i o n f o r each c l a s s h i g h l i g h t s the importance of combining assignment s c o r e s r e l i a b l y . Data C o l l e c t i o n and A n a l y s i s The t e a c h e r s were g i v e n r e c o r d s h e e t s a t the b e g i n n i n g of the r e p o r t i n g p e r i o d on which t o r e c o r d the achievement s c o r e s f o r b o t h academic s u b j e c t s . At the end of the term the r e c o r d s h e e t s and the q u e s t i o n n a i r e s were c o l l e c t e d and a n a l y z e d . I n f o r m a t i o n from t h e q u e s t i o n n a i r e s was c o l l a t e d and t a b u l a t e d . The achievement s c o r e s f o r each o f the two s u b j e c t s were s t a t i s t i c a l l y b a l a n c e d and a g g r e g a t e d , and t h e s t u d e n t s were ranked based on the a g g r e g a t e d s c o r e . T h i s " b a l a n c e d " r a n k i n g was c o r r e l a t e d w i t h t h e t e a c h e r s ' r a n k i n g u s i n g the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t r , . ranKs A d d i t i o n a l l y , the r e s u l t s from the r e g r a d i n g of t h e b a l a n c e d d a t a were compared t o t h e grades a s s i g n e d u s i n g t h e t e a c h e r s ' o r i g i n a l d a t a . The number of s t u d e n t s who r e c e i v e d a change i n l e t t e r grade was r e c o r d e d and documented. The d i r e c t i o n and magnitude of the change was a l s o t a b u l a t e d . From t h e t a b u l a t e d r e s u l t s , i t was p o s s i b l e t o 55 g e n e r a l i z e as t o whether t e a c h e r s , s i m i l a r t o tho s e i n t h e sample, c o u l d b e n e f i t from i n - s e r v i c e and p r e - s e r v i c e t r a i n i n g on r e v i s e d g r a d i n g t e c h n i q u e s . I t a l s o became a p p a r e n t , from the r e s u l t s , whether o r not g r a d i n g p r a c t i s e s can be and s h o u l d be made more r e l i a b l e . 56 CHAPTER 4 R e s u l t s The data c o l l e c t e d i n the study were d i v i d e d i n t o f o u r c a t e g o r i e s : (a) demographic i n f o r m a t i o n concerning the sample, (b) data concerning the s u b j e c t s ' a t t i t u d e s and p e r c e p t i o n s toward g r a d i n g and r e p o r t i n g , (c) a n a l y s i s of the sample's grading techniques, and (d) a n a l y s i s of the s u b j e c t s ' course r e c o r d sheets. Sample A sample of 37 s u b j e c t s was used i n the study. While a l l 37 responded to the f i r s t data request (copies of t h e i r r e c o r d s h e e t s ) , 3 4 responded to the second set of data (survey and regraded r e c o r d s h e e t s ) . Three of the 34 s u b j e c t s e l e c t e d t o w i t h h o l d t h e i r regraded r e c o r d sheets and submitted o n l y the completed q u e s t i o n n a i r e . A t o t a l of 31 s u b j e c t s submitted both regraded r e c o r d sheets and completed q u e s t i o n n a i r e s . Of the 31 respondents, 25 p r o v i d e d regraded data f o r both mathematics and s o c i a l s t u d i e s . The sample p r o v i d e d raw scores and l e t t e r grades f o r 1,314 elementary school students from 56 c l a s s e s . Tables IV through IX p r o v i d e the frequency d i s t r i b u t i o n s of the sample's demographic v a r i a b l e s . Tables X through XVII d i s p l a y the frequency d i s t r i b u t i o n s of the s u b j e c t s ' responses to the a t t i t u d e and p e r c e p t i o n q u e s t i o n s p e r t a i n i n g t o grading and r e p o r t i n g . 57 Demographic V a r i a b l e s a) Gender Male 24 Feraale 8 T o t a l 34 T a b l e IV - D i s t r i b u t i o n by Gender b) Age ( y e a r s ) Less than 25 0 25-30 0 31-40 25 41-50 8 51-60 1 G r e a t e r t h a n 60 ... 0 T o t a l 34 T a b l e V - D i s t r i b u t i o n by Age ( y e a r s ) c) Y e a r s of u n i v e r s i t y two 0 t h r e e 1 f o u r 10 B.Ed 15 MA o r MEd 7 o t h e r ( i e d i p l o m a ) . 1 T o t a l 34 T a b l e VI - D i s t r i b u t i o n by U n i v e r s i t y T r a i n i n g ( y e a r s ) 58 d) Number of measurement and/or e v a l u a t i o n courses taken none 11 one 21 two 1 three 1 More than t h r e e . . . 0 T o t a l . 34 Table VII - D i s t r i b u t i o n by Measurement Courses Taken e) Teaching assignment p a r t time 3 f u l l time 31 T o t a l 34 Table V I I I - D i s t r i b u t i o n by Teaching Assignment f) Teaching Experience (years) 0-5 0 6-10 7 11-15 15 16-20 9 Greater than 20 ... 3 T o t a l 34 Table IX - D i s t r i b u t i o n by Teaching Experience (years) 59 Of the 34 s u b j e c t s c o m p l e t i n g the second s e t of d a t a , most ( 7 4 % ) were male t e a c h e r s and most ( 7 4 % ) were i n the 31 t o 40 y e a r age c a t e g o r y . Twenty f o u r per c e n t o f the r e s p o n d e n t s were i n the 41 t o 50 y e a r age c a t e g o r y . Teachers who had completed a B a c h e l o r ' s degree made up 44% of t h e sample w h i l e 29% had f o u r y e a r s u n i v e r s i t y e x p e r i e n c e and 2 1 % had completed a M a s t e r ' s degree. N e a r l y a l l of the r e s p o n d e n t s ( 9 1 % ) were f u l l time t e a c h e r s and 4 4 per c e n t had between 11 and 15 y e a r s t e a c h i n g e x p e r i e n c e . Twenty s i x per c e n t of the t e a c h e r s had 16 .to 20 y e a r s t e a c h i n g e x p e r i e n c e . Many of the s u b j e c t s (62%) had completed one measurement and/or e v a l u a t i o n c o u r s e w h i l e 32% had not t a k e n any measurement o r e v a l u a t i o n c o u r s e s . In g e n e r a l , t h e t y p i c a l respondent was a 31 t o 40 year o l d male t e a c h e r who was employed f u l l t i me and who had between 11 and 15 y e a r s of t e a c h i n g e x p e r i e n c e . He t y p i c a l l y h e l d a B a c h e l o r ' s degree and had completed one measurement and/or e v a l u a t i o n c o u r s e . A t t i t u d e s and P e r c e p t i o n s Toward G r a d i n g and R e p o r t i n g T a b l e s X t h r o u g h XVII r e f l e c t t h e sample's a t t i t u d e and p e r c e p t i o n towards g r a d i n g and r e p o r t i n g s t u d e n t achievement. The s u b j e c t s were t o respond on a f i v e - p o i n t s c a l e t h e e x t e n t of agreement between th e f e e l i n g e x p r e s s e d i n each statement and t h e i r own p e r s o n a l f e e l i n g . The f i v e o p t i o n s were: S t r o n g l y D i s a g r e e (SD), D i s a g r e e (D), Undecided (U), Agree (A ) , and S t r o n g l y Agree (SA). 60 ) My c o l l e a g u e s a t t h i s s c h o o l t a k e r e p o r t c a r d marks v e r y s e r i o u s l y . SD 0 D 0 U 1 A 15 SA 18 T o t a l 34 T a b l e X - D i s t r i b u t i o n o f c o l l e a g u e s ' a t t i t u d e s toward r e p o r t c a r d marks h) L e t t e r grades are an e f f e c t i v e method of i n f o r m i n g p a r e n t s of t h e i r c h i l d ' s p r o g r e s s and achievement. SD 1 D 5 U 2 A 22 SA 4 T o t a l 34 T a b l e XI - D i s t r i b u t i o n of a t t i t u d e toward l e t t e r grade e f f e c t i v e n e s s To t h e b e s t o f my knowledge, p a r e n t s a r e g e n e r a l l y s a t i s f i e d w i t h t h e l e t t e r grade system o f r e p o r t i n g . SD 0 D 2 U 1 A 28 SA 3 T o t a l 34 T a b l e X I I - D i s t r i b u t i o n o f agreement r e g a r d i n g p a r e n t s s a t i s f a c t i o n toward l e t t e r grade r e p o r t i n g systems. I am v e r y c o n f i d e n t t h a t t h e l e t t e r grades I a s s i g n a r e a c c u r a t e and r e l i a b l e . SD 0 D 0 U 4 A 23 SA. . 7 T o t a l 34 T a b l e X I I I - D i s t r i b u t i o n o f agreement r e g a r d i n g s u b j e c own g r a d i n g a b i l i t y 62 (k) I would l i k e t o l e a r n more about c o l l a t i n g and a s s i g n i n g l e t t e r grades t o raw s c o r e s . SD 0 D 5 U 3 A 16 SA 10 T o t a l 3~4 T a b l e XIV - D i s t r i b u t i o n of agreement r e g a r d i n g s u b j e c t s ' d e s i r e t o l e a r n more about m a r k i n g and g r a d i n g (1) More i n - s e r i v c e s e s s i o n s on g r a d i n g and r e p o r t i n g s h o u l d be o f f e r e d . SD 0 D 5 U 4 A 11 SA 14 T o t a l 34 Ta b l e XV - D i s t r i b u t i o n o f agreement on e v a l u a t i o n o f i n - s e r v i c e s e s s i o n s (m) U n i v e r s i t i e s s h o u l d o f f e r more p r e - s e r v i c e i n s t r u c t i o n i n e f f e c t i v e g r a d i n g and r e p o r t i n g . SD 0 D 3 U 4 A . 14 SA 13 T o t a l 34" Tab l e XVI - D i s t r i b u t i o n of agreement r e g a r d i n g s u b j e c t s ' o p i n i o n on p r e - s e r v i c e t r a i n i n g (n) The r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o communicated t o p a r e n t s , s t u d e n t s , e t c . SD 0 D 8 U 6 A 16 SA 4 T o t a l 34 T a b l e XVII - D i s t r i b u t i o n o f agreement r e g a r d i n g e f f e c t i v e n e s s of t h e r e p o r t c a r d format as communication d e v i c e N e a r l y a l l (97%) of t h e t e a c h e r s s u r v e y e d agreed t h a t t h e i r c o l l e a g u e s t r e a t m arking and g r a d i n g s e r i o u s l y (44% Agree;53% S t r o n g l y A g r e e ) . A m a j o r i t y of the r e s p o n d e n t s (71%) agreed t h a t l e t t e r grades a r e an e f f e c t i v e method of i n f o r m i n g p a r e n t s of t h e i r c h i l d ' s p r o g r e s s and achievement. F i f t y n i n e per c e n t o f the sample a l s o agreed w i t h t h e f e e l i t h a t t h e r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o be communicated t o p a r e n t s , s t u d e n t s , and o t h e r i n t e r e s t e d groups. However, 18% were u n d e c i d e d and 24% d i s a g r e e d w i t h t h i s s t a t e m e n t . I n r e sponse t o the statement c o n c e r n i n g p a r e n t s a t i s f a c t i o n , most respon d e n t s (91%) agreed t h a t p a r e n t s were s a t i s f i e d w i t h the l e t t e r grade system of r e p o r t i n g . In r e sponse t o the q u e s t i o n s p e r t a i n i n g t o t e a c h e r s ' g r a d i n g and r e p o r t i n g t e c h n i q u e s , a m a j o r i t y o f the s u b j e c t s 64 (88%) agreed (68% Agree; 20% S t r o n g l y Agree) t h a t the l e t t e r grades they a s s i g n are r e l i a b l e and a c c u r a t e . However, most respondents i n d i c a t e d a d e s i r e to l e a r n more about c o l l a t i n g and g r a d i n g raw achievement scores (76% Agree or S t r o n g l y Agree; 9% Undecided; 15% D i s a g r e e ) . The percentages of respondents agr e e i n g to the need f o r more i n - s e r v i c e and p r e - s e r v i c e i n s t r u c t i o n i n t h i s area were 74% and 79% r e s p e c t i v e l y . On the same q u e s t i o n s , 12% were undecided f o r both i n - s e r v i c e and p r e - s e r v i c e s e s s i o n s while the percentages of respondents who d i s a g r e e d with the need f o r i n - s e r v i c e and p r e - s e r v i c e i n s t r u c t i o n were 15% and 9% r e s p e c t i v e l y . In g e n e r a l , the t y p i c a l respondent was one who (a) thought h i s c o l l e a g u e s took r e p o r t c a r d marks s e r i o u s l y , (b) thought l e t t e r grades are an e f f e c t i v e method of i n f o r m i n g parents of t h e i r students p r ogress, (c) thought t h a t parents were s a t i s f i e d with the l e t t e r grade system of marking, (d) thought the B r i t i s h Columbia M i n i s t r y of Education r e p o r t c a r d format allows s u f f i c i e n t i n f o r m a t i o n t o be communicated to parents and students, (e) thought t h a t more i n - s e r v i c e and p r e - s e r v i c e s e s s i o n s on e v a l u a t i o n and g r a d i n g should be o f f e r e d by school d i s t r i c t s and u n i v e r s i t i e s , and (f) d i s p l a y e d an i n t e r e s t i n l e a r n i n g more about aggregating and grading raw s c o r e s . Grading Techniques The r e c o r d sheets and the responses to the gra d i n g and r e p o r t i n g s e c t i o n of the survey were analyzed t o determine the 65 g r a d i n g p r o c e d u r e s used by the s u b j e c t s . A l t h o u g h a l l o f the r e s p o n d e n t s employed a v a r i a t i o n o f the n o r m - r e f e r e n c e d approach t o g r a d i n g , the sample's g r a d i n g t e c h n i q u e s were c l a s s i f i e d f u r t h e r a c c o r d i n g t o the f o l l o w i n g g r a d i n g methods: 1. Method One - Summing raw s c o r e s and c o n v e r t i n g each s t u d e n t ' s t o t a l t o a p e r c e n t a g e p r i o r t o g r a d i n g by n a t u r a l b r e a k s (see page 53) . 2. Method Two - Summing each s t u d e n t ' s raw s c o r e s and c o n v e r t i n g the t o t a l t o a p e r c e n t a g e . L e t t e r grades a r e a s s i g n e d based on the n u m e r i c a l v a l u e o f t h e p e r c e n t a g e as w e l l as a s u b j e c t i v e f a c t o r . Examples of s u b j e c t i v e f a c t o r s were: (a) a d j u s t i n g l e t t e r grades t o r e f l e c t i n d i v i d u a l e f f o r t ; (b) a d j u s t i n g l e t t e r grades t o r e f l e c t a change i n a s t u d e n t ' s performance from one term t o a n o t h e r ; o r (c) a d j u s t i n g l e t t e r grades t o compensate f o r m i s s e d a s s i g n m e n t s . 3. Method Three - Summing w e i g h t e d raw s c o r e s f o r each s t u d e n t and c o n v e r t i n g t h e sum t o a p e r c e n t a g e p r i o r t o g r a d i n g a c c o r d i n g t o n a t u r a l b r e a k s . 4. Method Four - Summing each s t u d e n t ' s raw s c o r e s and c o n v e r t i n g the t o t a l t o a p e r c e n t a g e . L e t t e r grades a r e a s s i g n e d based on the n u m e r i c a l v a l u e of t h e p e r c e n t a g e . T h i s method i s commonly c a l l e d " p e r c e n t g r a d i n g " (see page 32). 5. Method F i v e - Term a s s i g n m e n t s , p r o j e c t s and t e s t s a r e g i v e n l e t t e r grades r a t h e r than n u m e r i c a l s c o r e s . Each s t u d e n t ' s f i n a l grade i s t h e average of a l l the l e t t e r grades awarded t h a t s t u d e n t d u r i n g the c o u r s e . For a v e r a g i n g p u r p o s e s , each l e t t e r grade i s g i v e n an o r d i n a l v a l u e . 66 6. Method S i x - Raw s c o r e s a r e b a l a n c e d t o have e q u a l means and e q u a l s t a n d a r d d e v i a t i o n s b e f o r e b e i n g w e i g h t e d and c o l l a t e d . The c o l l a t e d s c o r e s a r e ranked and graded a c c o r d i n g t o n a t u r a l b r e a k s . T a b l e X V I I I g i v e s the f r e q u e n c y breakdown o f each g r a d i n g method. T a b l e XIX d i s p l a y s how e x t e n s i v e l y each respondent a p p l i e d t h e i r g r a d i n g t e c h n i q u e . Method One 2 4 Method Two 5 Method Three 2 Method Four 2 Method F i v e 2 Method S i x 2 T o t a l 37 Ta b l e X V I I I - D i s t r i b u t i o n o f g r a d i n g method Every s c h o o l s u b j e c t 9 Core s u b j e c t s (Math, S c i e n c e S o c i a l S t u d i e s , E n g l i s h ) . . . 24 I n d i s c r i m i n a t e use 1 T o t a l 34 Ta b l e XIX - D i s t r i b u t i o n of a p p l i c a t i o n of g r a d i n g t e c h n i q u e 67 The r e s p o n d e n t s were a l s o asked t o d e s c r i b e how t h e y d e t e r m i n e d a f i n a l s c o r e f o r t h o s e s t u d e n t s who have m i s s e d one o r more t e s t s o r a s s i g n m e n t s . T h e i r r e s p o n s e s were c l a s s i f i e d under the f o l l o w i n g methods: 1. Method One - C a l c u l a t e the average p e r c e n t a g e on o n l y the a ssignments completed and i g n o r e the m i s s e d a s s i g n m e n t s . 2. Method Two - E s t i m a t e a s c o r e f o r the m i s s e d assignment by l o o k i n g a t : (a) the absent s t u d e n t ' s p r e v i o u s work, (b) the marks o t h e r s t u d e n t s of s i m i l a r a b i l i t y o b t a i n e d on t h e m i s s e d assignment. 3. Method Three - A s s i g n a mark of z e r o f o r t h e missed assignment. 4. Method Four - C a l c u l a t e a l e t t e r grade based on p r e v i o u s work and reduce by one l e t t e r grade f o r m i s s e d a s s i g n m e n t ( s ) . 5. Method F i v e - C a l c u l a t e an average T s c o r e based on a l l completed work and a s s i g n t h a t v a l u e t o the missed a s s i g n m e n t ( s ) . When a l l assignment s c o r e s have been s t a t i s t i c a l l y b a l a n c e d t h i s method e s s e n t i a l l y m a i n t a i n s rank p o s i t i o n and does not p e n a l i z e nor reward t h e s t u d e n t f o r b e i n g a b s e n t . T a b l e XX shows the f r e q u e n c y o f the methods used by the sample t o a r r i v e a t a mark f o r an absent s t u d e n t . 68 Method One 2 5 Method Two 5 Method Three 2 Method Four 1 Method F i v e 0 Other 1 T o t a l 34 Table XX - D i s t r i b u t i o n of compensation method used The r e s u l t s o f the p r o c e d u r e s used t o compensate s t u d e n t s f o r m i s s e d assignments ( T a b l e XX) i n d i c a t e d t h a t most responde n t s (74%) c a l c u l a t e d an average s c o r e based s o l e l y on the work the absent s t u d e n t had completed. The d a t a a l s o r e v e a l s t h a t 15% o f the r e s p o n d e n t s use s u b j e c t i v e means t o e s t i m a t e a s c o r e f o r a m i s s e d assignment. The p a r t i c i p a n t s i n t h e s t u d y were asked t o a p p l y t h e i r g r a d i n g t e c h n i q u e t o a s e t of f i c t i t i o u s d a t a (see page 48) and rank t h e f i v e s t u d e n t s from h i g h e s t t o l o w e s t . T a b l e XXI shows the f r e q u e n c y o f t h e r a n k s as d e t e r m i n e d by the v a r i o u s g r a d i n g t e c h n i q u e s . In r e s p o n s e t o the second r e s e a r c h q u e s t i o n (page 16) r e g a r d i n g t h e r e l i a b i l i t y of the a g g r e g a t i o n and w e i g h t i n g o f raw s c o r e s f o r g r a d i n g p u r p o s e s , t h e sample's r e c o r d s h e e t s and the q u e s t i o n n a i r e r e s p o n s e s p e r t a i n i n g t o g r a d i n g t e c h n i q u e s were a n a l y z e d . The a n a l y s i s r e v e a l e d a h i g h degree of s i m i l a r i t y between th e a g g r e g a t i o n methods e v i d e n t on the 69 Student A B C D E 1 , 2 , 3 , 5 , 4 16 3 , 1 , 4 . 5 , 2 , 4 . 5 6 2 , 1 , 3 , 5 , 4 3 1 . 2 . 4 . 5 . 3 2 4 . 5 , 1 , 3 , 2 , 4 . 5 1 2 . 1 . 5 . 3 . 4 1 2 , 3 , 5 , 1 , 4 1 1 . 2 . 5 . 2 . 5 . 5 . 4 1 2 . 1 . 4 . 5 . 3 . 4 . 5 1 Incomplete 2 T o t a l 34 Table XXI - D i s t r i b u t i o n of ranks of f i c t i t i o u s data r e c o r d sheets and the aggregation techniques d e s c r i b e d by the respondents on the q u e s t i o n n a i r e . However, the grading methods used i n response to the f i c t i t i o u s data s e c t i o n of the survey were l e s s c o n s i s t e n t with the methods e v i d e n t on the grade sheets and i n the w r i t t e n d e s c r i p t i o n s . The most frequent d i s c r e p a n c y concerned the weighting f a c t o r s t o be a p p l i e d t o the f i c t i t i o u s data. Although most respondents d i d not normally weight raw sc o r e s , many attempted t o apply weights t o the f i c t i t i o u s data. x As shown i n t a b l e XXI, 18% of the sample obtained a rank order f o r the f i c t i t i o u s t o t a l scores t h a t was c o n s i s t e n t with the most common grading method i d e n t i f i e d i n t h i s study (see 70 T a b l e X V I I I ) . Nine per c e n t o f the re s p o n d e n t s a p p l i e d a g g r e g a t i o n and w e i g h t i n g t e c h n i q u e s t h a t a r e advocated by many e x p e r t s (see page 2 8 ) . The su g g e s t e d methods a r e t a b u l a t e d as Method 6 i n T a b l e X V I I I . Many s u b j e c t s (47%) used a g g r e g a t i o n and w e i g h t i n g t e c h n i q u e s t h a t were i n c o n s i s t e n t w i t h any of t h e o t h e r g r a d i n g methods p r e v i o u s l y i d e n t i f i e d i n t h i s s t u d y . T h e i r methods e s s e n t i a l l y i n v o l v e d combining w e i g h t e d raw s c o r e s . However, t h e y o b t a i n e d r e s u l t s t h a t were v e r y s i m i l a r t o b o t h t h e r a n k s of the ag g r e g a t e d w e i g h t e d raw s c o r e s ( T a b l e I ) and t o t h e r a n k s of the agg r e g a t e d w e i g h t e d b a l a n c e d s c o r e s ( T a b l e I I ) . The r e s u l t s o f the c l a s s i f i c a t i o n and t a b u l a t i o n of the v a r i o u s g r a d i n g methods used by the r e s p o n d e n t s , as t a b l e d i n Tab l e X V I I I , r e v e a l e d t h a t a 65% o f the sample employed an a g g r e g a t i o n t e c h n i q u e t h a t i n v o l v e d summing the raw s c o r e s and c o n v e r t i n g t h e c o l l a t e d s c o r e s t o a p e r c e n t a g e p r i o r t o g r a d i n g by n a t u r a l b r e a k s . The r e s u l t s a l s o showed t h a t 14% of t h e sample used an a g g r e g a t i o n t e c h n i q u e t h a t i n v o l v e d summing the raw s c o r e s , c o n v e r t i n g t h e combined s c o r e s t o a p e r c e n t a g e , and u s i n g the n u m e r i c a l v a l u e o f t h e p e r c e n t a g e as w e l l as s u b j e c t i v e f a c t o r s t o determine t h e l e t t e r grade. Only two o f the 37 re s p o n d e n t s (5%) used " r e l i a b l e " methods t o a p p l y t h e d e s i r e d w e i g h t i n g f a c t o r s t o the raw s c o r e s . Record S h e e t s . The r a n k i n g s o f t h e s t a t i s t i c a l l y b a l a n c e d t o t a l s were compared t o the r a n k i n g s of the raw s c o r e t o t a l s u s i n g 71 Spearman's R a n k - C o r r e l a t i o n C o e f f i c i e n t . T a b l e X X I I l i s t s the s u b j e c t number, the number of s t u d e n t s i n each of the s u b j e c t ' s two c l a s s e s ( n ) , t h e rank c o r r e l a t i o n c o e f f i c i e n t ( r g ) , and t h e c r i t i c a l v a l u e s o f Spearman's rank C o r r e l a t i o n C o e f f i c i e n t f o r t e s t i n g the n u l l h y p o t h e s i s o f no c o r r e l a t i o n w i t h a t w o - t a i l e d t e s t a t the alpha=.01 l e v e l o f s i g n i f i c a n c e ( G l a s s , G., S t a n l e y , J . , 1970, p. 539). T a b l e X X I I I d i s p l a y s the d i s t r i b u t i o n of the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s . Of t h e 56 c o r r e l a t i o n s computed, a l l had r v a l u e s t h a t s were g r e a t e r t h a n the c o r r e s p o n d i n g c r i t i c a l v a l u e . These r e s u l t s i n d i c a t e t h a t the c o r r e l a t i o n s between th e r a n k i n g s of the raw s c o r e t o t a l s and t h e r a n k i n g s of the b a l a n c e d s c o r e t o t a l s were p o s i t i v e and s i g n i f i c a n t . T h e r e f o r e , the n u l l h y p o t h e s i s (H Q:rho=0) s h o u l d be r e j e c t e d a t the .01 l e v e l o f s i g n i f i c a n c e i n f a v o u r o f t h e a l t e r n a t e h y p o t h e s i s f o r each of the 56 r e c o r d s h e e t s a n a l y z e d . In t h e second s e t o f d a t a , the s u b j e c t s were asked t o r e g r a d e t h e i r s t u d e n t s based on the s t a t i s t i c a l l y b a l a n c e d t o t a l s p r o v i d e d . The l e t t e r grades based on b a l a n c e d s c o r e s were compared t o the l e t t e r grades a s s i g n e d i n i t i a l l y by t h e r e s p o n d e n t . The d i f f e r e n c e s between the two l e t t e r grades were c l a s s i f i e d a c c o r d i n g t o magnitude and d i r e c t i o n o f "change". For example, a change from an i n i t i a l l e t t e r grade of B t o a b a l a n c e d s c o r e l e t t e r grade of C+ was r e c o r d e d as a o n e - l e t t e r grade d e c r e a s e . A change from a C+ t o a b a l a n c e d l e t t e r grade of A was r e c o r d e d as a t w o - l e t t e r grade i n c r e a s e . T a b l e XXIV 72 MATHEMATICS SOCIAL STUDIES SUBJECT CRITICAL R Q CRITICAL # s n VALUE s n VALUE 001 .941 24 .537 . 976 24 . 537 002 .873 34 <. 478 . 957 34 <. 478 003 . 955 18 . 625 . 985 18 . 625 004 .983 24 .537 N/A N/A N/A 005 .919 33 < . 478 . 984 33 <. 478 006 .954 14 .716 N/A N/A N/A 007 .953 27 . 505 .975 27 . 505 008 .975 14 .716 .970 14 . 716 009 N/A N/A N/A . 968 21 . 576 010 . 785 25 . 526 . 903 25 . 526 O i l .992 24 .537 . 943 33 <. 478 012 .746 20 .591 . 857 20 . 591 013 .987 16 . 6 66 . 981 36 <. 478 014 N/A N/A N/A . 980 24 . 537 015 .984 17 .645 .939 17 . 654 016 N/A N/A N/A . 949 18 . 625 017 .937 16 . 666 . 797 16 . 666 018 .992 28 . 496 . 962 28 . 496 019 .956 13 . 745 .995 13 . 745 020 . 955 30 .478 . 960 32 <. 478 021 .977 24 . 537 N/A N/A N/A 022 .979 33 < . 478 . 876 32 <. 478 023 . 988 32 < . 478 . 957 31 <. 478 024 .821 16 .666 . 900 16 . 666 025 .931 23 . 549 . 947 23 . 549 026 .984 29 . 487 .976 27 . 505 027 .929 16 .666 . 918 16 . 666 028 . 986 24 .537 . 902 24 . 537 029 .986 28 . 496 . 869 28 . 496 030 .972 25 . 526 . 960 25 . 526 031 .999 16 . 666 . 875 16 . 666 T o t a l 643 671 Ta b l e X X I I - Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s ( R Q ) R e l a t i n g Raw Score and B a l a n c e d Score T o t a l s F o r (a) Mathematics and (b) S o c i a l S t u d i e s 73 Frequency Score I n t e r v a l Math S.S. 0.970 - 1.000 14 9 0. 940 - 0.969 6 9 0.910 - 0.939 4 2 0.880 - 0.909 0 3 0.850 - 0.879 1 4 0. 820 - 0.849 1 0 0. 790 - 0.819 0 1 0. 760 - 0.789 1 0 0.730 - 0.759 1 0 0 . 700 - 0.729 0 0 T o t a l 28 28 T a b l e X X I I I - D i s t r i b u t i o n of Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s shows the f r e q u e n c y , the magnitude, and the d i r e c t i o n o f the l e t t e r grade changes. For a n a l y s i s purposes the range of l e t t e r grades (A t o E) was d i v i d e d i n t o t h r e e c a t e g o r i e s i n accordance w i t h t h e M i n i s t r y o f E d u c a t i o n l e t t e r grade i n t e r p r e t a t i o n s : Above Average, Average, and Below Average. L e t t e r grades A and B were grouped i n t o the Above Average c a t e g o r y ; l e t t e r grades C+, C, and C- were grouped i n t o the Average c a t e g o r y ; and l e t t e r grades of D and E were grouped i n t o the Below Average c a t e g o r y . 7 4 One l e t t e r grade i n c r e a s e 238 (18. 1%) One l e t t e r grade d e c r e a s e 313 (23.8%) Two l e t t e r grade i n c r e a s e 28 (2.1%) Two l e t t e r grade d e c r e a s e 23 (1.8%) Three l e t t e r grade i n c r e a s e . . . . 4 (0.3%) Three l e t t e r grade d e c r e a s e . . . . 1 (0.1%) No change i n l e t t e r grade 707 (53. 8%) T o t a l 1314 (100 . 0% ) T a b l e XXIV - D i s t r i b u t i o n o f l e t t e r grade changes The d i f f e r e n c e s i n l e t t e r grade were a l s o a n a l y z e d t o deter m i n e how many "changes" would r e s u l t i n a s h i f t from one c a t e g o r y t o a n o t h e r . T a b l e XXV shows the f r e q u e n c y of c a t e g o r y s h i f t s t h a t r e s u l t e d when t h e o r i g i n a l l e t t e r grades were r e p l a c e d by l e t t e r grades based on s t a t i s t i c a l l y b a l a n c e d s c o r e s . Above Average t o Average 112 (8.5%) Average t o Above Average 5 4 (4.1%) Below Average t o Average 4 4 (3.3%) Average t o Below Average 29 (2.2%) L e t t e r grade change b u t no change i n c a t e g o r y 368 (28 . 1%) No l e t t e r g r a d e / c a t e g o r y change . . 707 (53.8%) T o t a l 1314 (100. 0% ) T a b l e XXV - Frequency of c a t e g o r y s h i f t 75 The r e s u l t s o f the l e t t e r grade comparison, l i s t e d i n T a b l e s X X I I I , XXIV, and XXV, i n d i c a t e d t h a t 46% of t h e 1,314 grades i n i t i a l l y awarded t o s t u d e n t s were changed when t h e s t u d e n t s were r e g r a d e d on the b a s i s of t h e b a l a n c e d s c o r e t o t a l s . T h i r t y n i n e per c e n t o f t h e r e v i s e d grades (or 18% of a l l grades) a l s o i n v o l v e d c a t e g o r y changes. F i n a l l y , the sample's r e c o r d s h e e t s were a n a l y z e d t o determine how many l e t t e r grades were based on m i s c a l c u l a t e d raw s c o r e t o t a l s . T a b l e XXVI d i s p l a y s t h e f r e q u e n c y o f e r r o r s , as w e l l as the d i r e c t i o n and magnitude o f the l e t t e r grade c o r r e c t i o n . T a b l e XXVII shows the f r e q u e n c y o f c a t e g o r y changes t h a t r e s u l t e d when i n c o r r e c t raw s c o r e t o t a l s were r e c a l c u l a t e d and t h e c o r r e s p o n d i n g l e t t e r grades r e v i s e d . One l e t t e r grade i n c r e a s e 2 (0.2%) One l e t t e r grade d e c r e a s e 6 (0.4%) Two l e t t e r grade i n c r e a s e 0 (0.0%) Two l e t t e r grade d e c r e a s e 2 (0.2%) C o r r e c t i o n s t h a t d i d not a f f e c t l e t t e r grade 53 (4.0%) No e r r o r s 1251 ( 95. 2%) T o t a l 1314 (100 . 0% ) Table XXVI - Frequency of l e t t e r grade changes r e s u l t i n g from a r i t h m e t i c e r r o r s 76 Above Average to Average 4 (0.3%) Average to Above Average 1 (0.1%) Below Average t o Average 0 (0.0%) Average to Below Average 2 (0.2%) L e t t e r grade change but no change i n category 3 (0.2%) No l e t t e r grade change 1304 (99 . 2%) T o t a l 1314 (100.0% ) Table XXVII - Frequency of category s h i f t r e s u l t i n g from l e t t e r grade r e c a l c u l a t i o n The a n a l y s i s of the r e c o r d sheets r e v e a l e d t h a t about 5% of the 1,314 student grades assi g n e d by the sample were based on i n c o r r e c t l y computed raw score, t o t a l s . However, o n l y about 1% of the students r e c e i v e d i n c o r r e c t l e t t e r grades as a r e s u l t of the c a l c u l a t i o n e r r o r s . Of the ten improperly assigned grades, seven r e s u l t e d i n an i n c o r r e c t category placement as w e l l . T h i s concludes the r e s u l t s s e c t i o n of t h i s study. Chapter f i v e w i l l c o n t a i n a s h o r t summary, an a n a l y s i s of the data i n response to the q u e s t i o n s of i n t e r e s t , an o u t l i n e of the l i m i t a t i o n s , and f i n a l l y , the p o s s i b i l i t i e s f o r f u r t h e r r e s e a r c h . 77 CHAPTER 5 C o n c l u s i o n s and Recommendations For F u r t h e r Study Summary The g e n e r a l problem t h a t t h i s i n v e s t i g a t i v e s t u d y c o n s i d e r e d was t o determine i f t e a c h e r s ' g r a d i n g methods c o u l d be r e v i s e d so as t o i n c r e a s e the r e l i a b i l i t y and f a i r n e s s of the l e t t e r grades t h e y award t o s t u d e n t s . I t was d i f f i c u l t t o a d e q u a t e l y d e a l w i t h t h i s problem i n one s t u d y . I t would be al m o s t i m p o s s i b l e t o a n a l y z e each o f t h e many components comprised i n the mark i n g and g r a d i n g p r o c e s s e s . C o n s e q u e n t l y , the scope o f t h i s s t u d y was r e s t r i c t e d t o f o c u s o n l y on the e f f e c t s of a p p l y i n g s t a t i s t i c a l b a l a n c i n g t o raw achievement s c o r e s p r i o r t o the c a l c u l a t i o n of s t u d e n t term t o t a l s . From t h i s p e r s p e c t i v e , t h e f o l l o w i n g q u e s t i o n s o f i n t e r e s t e v o l v e d : 1. I s t h e r e a s i g n i f i c a n t d i f f e r e n c e between the r a n k i n g s of the ag g r e g a t e d s c o r e s c a l c u l a t e d by t e a c h e r s u s i n g i n t u i t i v e methods and t h e r a n k i n g s o f ag g r e g a t e d b a l a n c e d s c o r e s ? 2. Are the term t o t a l s , from which the l e t t e r grades a r e d e r i v e d , a g g r e g a t e d and w e i g h t e d r e l i a b l y ? 3. Are t e a c h e r s ' grades r e l i a b l e and f a i r ? 4. Would the r e v i s e d r e s u l t s be of such a magnitude as t o 78 j u s t i f y t he time and expense r e q u i r e d t o r e - e d u c a t e t e a c h e r s ? The raw s c o r e s s u p p l i e d by the sample i n the f i r s t s e t of d a t a were c o n v e r t e d t o T - s c o r e s , w e i g h t e d , and c o l l a t e d t o o b t a i n a " s t a t i s t i c a l l y b a l a n c e d " t o t a l s c o r e f o r each s t u d e n t l i s t e d on the r e c o r d s h e e t s . The w e i g h t i n g s f o r t h e b a l a n c e d d a t a were made i d e n t i c a l t o tho s e used i n i t i a l l y by the r e s p o n d e n t s . In t h e second s e t o f d a t a , the s u b j e c t s were asked t o use t h e b a l a n c e d s c o r e s t o r e g r a d e t h e i r s t u d e n t s , and a l s o t o complete the q u e s t i o n n a i r e l i s t e d i n Appendix A. Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s were c a l c u l a t e d t o compare the r a n k i n g s o f the raw s c o r e t o t a l s w i t h the r a n k i n g s of t he s t a t i s t i c a l l y b a l a n c e d t o t a l s . The c r i t i c a l v a l u e s of Spearman's Rank C o r r e l a t i o n C o e f f i c i e n t f o r t e s t i n g the n u l l h y p o t h e s i s o f no c o r r e l a t i o n w i t h a t w o - t a i l e d t e s t ( a l p h a = .01) were o b t a i n e d from G l a s s and S t a n l e y (1970, p. 539) w i t h the a p p r o p r i a t e n v a l u e s c o r r e s p o n d i n g t o the c l a s s s i z e s . The l e t t e r grades based on raw s c o r e s were compared t o t h o s e based on b a l a n c e d s c o r e s t o dete r m i n e i f s i g n i f i c a n t l y d i f f e r e n t l e t t e r grades would r e s u l t when raw s c o r e s were s t a t i s t i c a l l y b a l a n c e d p r i o r t o b e i n g a g g r e g a t e d f o r g r a d i n g purposes. The r e s u l t s o f the comparison were r e c o r d e d i n terms of t h e change i n l e t t e r grade t h a t o c c u r r e d when the b a l a n c e d s c o r e t o t a l s r e p l a c e d raw s c o r e t o t a l s . The grade changes were a n a l y z e d t o dete r m i n e how many s t u d e n t s would have r e c e i v e d a c a t e g o r y change (see page 73) i f t h e i r o r i g i n a l grades were r e p l a c e d w i t h b a l a n c e d s c o r e g r a d e s . 79 T a b l e s XXIV and XXV d i s p l a y t h e r e s u l t s of t h e s e comparisons and a n a l y s e s . The raw s c o r e grades were a l s o a n a l y z e d t o det e r m i n e how many s t u d e n t s r e c e i v e d raw s c o r e grades based on m i s c a l c u l a t e d term t o t a l s . The numbers of c a t e g o r y s h i f t s t h a t r e s u l t e d from m i s c a l c u l a t e d grades were a l s o r e c o r d e d . The r e s u l t s of t h e s e a n a l y s e s a r e t a b u l a t e d i n T a b l e s XXVI and XXVII. The q u e s t i o n n a i r e was used t o c o l l e c t d a t a p e r t a i n i n g t o the sample's demographic v a r i a b l e s , the sample's a t t i t u d e and p e r c e p t i o n o f g r a d i n g and r e p o r t i n g , and t h e sample's g r a d i n g t e c h n i q u e s . The r e s u l t s a r e t a b u l a t e d i n T a b l e s IV t h r o u g h XXI i n c h a p t e r f o u r . R e s e a r c h Q u e s t i o n 1 In response t o the f i r s t r e s e a r c h q u e s t i o n r e g a r d i n g the d i f f e r e n c e between t h e r a n k i n g s o f t h e raw s c o r e t o t a l s and the r a n k i n g s o f b a l a n c e d s c o r e t o t a l s , Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s were c a l c u l a t e d f o r each o f t h e s u b j e c t ' s two r e c o r d s h e e t s . The r e s u l t s i n d i c a t e d t h a t t h e r a n k i n g s of the raw s c o r e t o t a l s were s i g n i f i c a n t l y and p o s i t i v e l y c o r r e l a t e d w i t h t h e r a n k i n g s o f the b a l a n c e d s c o r e t o t a l s (see page 7 1 ) . The v a l u e s o f the Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s ranged from +.746 t o +.999 (see T a b l e X X I I I ) . From t h e s e r e s u l t s i t can be c o n c l u d e d t h a t t h e r e i s a s i g n i f i c a n t r e l a t i o n s h i p between t h e r a n k i n g s o f t h e raw s c o r e t o t a l s c a l c u l a t e d by the s u b j e c t s and the r a n k i n g s o f 8 0 the balanced score t o t a l s . I t can a l s o be concluded t h a t the methods used by the s u b j e c t s t o aggregate raw scores f o r r e p o r t i n g purposes are reasonably r e l i a b l e when compared with the s t a t i s t i c a l b a l a n c i n g methods advocated by many authors. Since norm-referenced g r a d i n g , the method used i n B r i t i s h Columbia s c h o o l s , bases l e t t e r grades on the rank order of the aggregate t o t a l s , s i g n i f i c a n t p o s i t i v e c o r r e l a t i o n s between the raw score t o t a l s and the balanced t o t a l s suggest t h e r e w i l l be a s i g n i f i c a n t and p o s i t i v e c o r r e l a t i o n between the l e t t e r grades r e s u l t i n g from each method of combining s c o r e s . In other words, the l e t t e r grade a student r e c e i v e s under the raw score method of combining scores should, i n most cases, be s i m i l a r to the l e t t e r grade he would r e c e i v e when the balanced score technique i s a p p l i e d . Both methods should y i e l d s i m i l a r l y d i s t r i b u t e d l e t t e r grades f o r a gi v e n c l a s s . I t can be concluded, then, t h a t on the whole, the students should r e c e i v e reasonably r e l i a b l e grades (see page 2). However, of the 1,314 students i n v o l v e d i n t h i s , study, 46% r e c e i v e d a change i n l e t t e r grade when t h e i r grades were based on balanced score t o t a l s r a t h e r than raw score t o t a l s (Table XXIV). T h i s would suggest t h a t , f o r many students, the assignment of grades was u n r e l i a b l e and was based on, or i n f l u e n c e d by, f a c t o r s other than the t o t a l score r a n k i n g s . Since s e v e r a l respondents i n q u i r e d about c o n v e r t i n g the T-scores on the r e c o r d sheets t o per cent s c o r e s , i t i s p o s s i b l e t h a t the va l u e s of the converted per cent scores were 81 i n f l u e n c i n g f a c t o r s f o r some s u b j e c t s d u r i n g t h e r e g r a d i n g p r o c e s s . The lower per c e n t s c o r e s (based on an a r b i t r a r y maximum of 275) r e s u l t i n g from th e T - s c o r e s would a l s o e x p l a i n why more l e t t e r grades d e c r e a s e d t h a n i n c r e a s e d (see T a b l e X X I V ) . I n r e f e r e n c e t o the consequences of u n r e l i a b l e g r a d i n g p r a c t i s e s (see page 2 ) , the l e t t e r grade changes suggest t h a t 46% of the s t u d e n t s i n t h i s s t u d y r e c e i v e d i n c o n s i s t e n t and m i s l e a d i n g i n f o r m a t i o n t h a t c o u l d p o t e n t i a l l y l e a d o t h e r s t o f o r m u l a t e erroneous d e c i s i o n s and c o n c l u s i o n s . I n a d d i t i o n , 18% of the 1,314 s t u d e n t s r e c e i v e d l e t t e r grade changes t h a t r e s u l t e d i n c a t e g o r y s h i f t s ( T a b l e XXV). These s t u d e n t s a r e most l i k e l y t o s u f f e r t h e consequences of e rroneous d e c i s i o n s as academic programs te n d g e n e r a l l y t o c a t e r t o the below average, average, and above average a b i l i t y groups. For example, s t u d e n t s who have average a b i l i t y c o u l d be e r r o n e o u s l y p l a c e d i n a s p e c i a l program d e s i g n e d f o r "below average a c h i e v e r s " o r t h e y c o u l d be e r r o n e o u s l y e n r o l l e d i n an e n r i c h e d program f o r s t u d e n t s w i t h above average a b i l i t y . In e i t h e r c a s e , t h e average s t u d e n t ' s academic needs may not be s a t i s f a c t o r i l y met. In l i g h t of t h e c u r r e n t l i t e r a t u r e , t h e r e s u l t s of Spearman Rank C o r r e l a t i o n C o e f f i c i e n t s were somewhat s u r p r i s i n g . Why a r e the r a n k i n g s of the raw s c o r e t o t a l s (as c a l c u l a t e d by the s u b j e c t s ) so s t r o n g l y s i g n i f i c a n t and p o s i t i v e l y c o r r e l a t e d w i t h t h e r a n k i n g s o f t h e b a l a n c e d s c o r e 82 t o t a l s i n t h i s s t u d y ? Many a u t h o r s have s u p p o r t e d t h e n o t i o n t h a t raw s c o r e s can o n l y be c o l l a t e d and we i g h t e d r e l i a b l y when t h e y have been b a l a n c e d t o have e q u a l means and e q u a l s t a n d a r d d e v i a t i o n s (see page 27). For example, Ahmann and Gl o c k (1981) s t a t e , " I f we hope t o m a i n t a i n t h e w e i g h t i n g scheme o r i g i n a l l y chosen, we must t a k e i n t o c o n s i d e r a t i o n the d i f f e r e n c e s i n v a r i a b i l i t y . A f a i l u r e t o do t h i s w i l l r e s u l t i n i n e q u i t i e s " (p. 426). But t h e s e and many o t h e r a u t h o r s have f a i l e d t o mention t h e imp o r t a n c e o f t h e rank c o r r e l a t i o n f o r "between" s c o r e s . From the r e s u l t s o f t h i s s t u d y , i t appears t h a t t h e rank c o r r e l a t i o n c o e f f i c i e n t f o r between s c o r e s can i n f l u e n c e the rank o r d e r o f the ag g r e g a t e d s c o r e s . Indeed, the " i n e q u i t i e s " t h a t r e s u l t when a g g r e g a t i n g raw s c o r e s w i t h l a r g e d i f f e r e n c e s i n s t a n d a r d d e v i a t i o n s can be reduced o r e l i m i n a t e d w i t h a l a r g e p o s i t i v e rank c o r r e l a t i o n f o r between s c o r e s . However, t h e s e i n e q u i t i e s can a l s o be a m p l i f i e d by a l a r g e n e g a t i v e rank c o r r e l a t i o n . Such i s t h e case w i t h t h e example t a k e n from G r o n l u n d (1974) on page 8 of t h i s s t u d y . Gronlund has used t h e extreme s i t u a t i o n (as have o t h e r a u t h o r s ) o f l a r g e v a r i a b i l i t y and a v e r y l a r g e n e g a t i v e between s c o r e s rank c o r r e l a t i o n (r - = -1) t o emphasize the e f f e c t s o f combining s c o r e s w i t h d i f f e r e n t s t a n d a r d d e v i a t i o n s . I n t h e i r examples, s t u d e n t A s c o r e s v e r y h i g h on the f i r s t assignment and v e r y low on t h e second w h i l e s t u d e n t B s c o r e s low on the f i r s t and h i g h on the second. B u t, t h i s i s seldom the case w i t h s c o r e s t a k e n from t e a c h e r s ' r e c o r d s h e e t s . More commonly, s t u d e n t s t e n d t o m a i n t a i n a r e l a t i v e l y 83 s t a b l e rank p o s i t i o n w i t h i n the c l a s s . Above average s t u d e n t s t e n d t o o b t a i n the above average s c o r e s on most assignments w h i l e t h e below average s t u d e n t s t e n d t o a c h i e v e the lo w e r s c o r e s . T h e r e f o r e , the r a n k i n g s f o r between raw s c o r e s would more l i k e l y be p o s i t i v e . I t i s i n t e r e s t i n g t o note t h a t e l e m e n t a r y t e a c h e r s q u e s t i o n assignments t h a t r e s u l t i n "u n u s u a l " c l a s s r a n k i n g s . F o r example, some s u b j e c t s q u e s t i o n e d t h e v a l i d i t y of t h e f i f t h assignment i n t h e f i c t i t i o u s d a t a (see Appendix A) because the r a n k i n g s on t h a t assignment f a i l e d t o f i t t h e r a n k i n g p a t t e r n e s t a b l i s h e d by the o t h e r f o u r a s s i g n m e n t s . S t u d e n t s who p l a c e d h i g h i n t h e r a n k i n g on t h e f i r s t f o u r assignments s h o u l d p l a c e r e l a t i v e l y h i g h i n the r a n k i n g o f t h e f i f t h assignment a l s o . A l t h o u g h a rank c o r r e l a t i o n o f r =+1 would be e x t r e m e l y d i f f i c u l t t o s 1 o b t a i n i n a c l a s s o f twenty o r t h i r t y s t u d e n t s , the s t a b i l i t y of rank p o s i t i o n s u g g e s t s t h e r e might be a r e a s o n a b l y s t r o n g p o s i t i v e rank c o r r e l a t i o n f o r between s c o r e s i n many c l a s s r e c o r d s h e e t s . C o n s i d e r i n g the r e s u l t s o f t h i s s t u d y and the d i f f e r e n t s t a n d a r d d e v i a t i o n v a l u e s i n the raw s c o r e d a t a , the l i k e l y p r e s e nce o f p o s i t i v e rank c o r r e l a t i o n s f o r between s c o r e s may have reduced the e f f e c t of v a r i a n c e on the sample's aggregate s c o r e s . To demonstrate the e f f e c t t h a t the between s c o r e rank c o r r e l a t i o n has on aggre g a t e t o t a l s , c o n s i d e r the f o l l o w i n g examples. I n each c a s e , t h e assignments a r e t o be w e i g h t e d e q u a l l y . The raw s c o r e t o t a l s and t h e b a l a n c e d s c o r e t o t a l s a re c a l c u l a t e d by summing the a p p r o p r i a t e s c o r e s . F o r each 84 example, t h e c r i t i c a l v a l u e (alpha=.10; n=6) o f the ranked c o r r e l a t i o n c o e f f i c i e n t f o r raw s c o r e t o t a l s w i t h b a l a n c e d s c o r e t o t a l s i s r = .829. The b r a c k e t t e d d a t a a r e t h e s b a l a n c e d s c o r e s . Assignment Student 1 2 T o t a l A 100 [64. 3] 40 [64 .8] 140 [129 .1] B 83 [59. 1] 31 [57 .3] 114 [116.4] C 58 [51. 5] 26 [53 .1] 84 [104.6] D 31 [43. 3] 15 [43 .9] 46 [ 87.2] E 25 [41. 5] 12 [41 .4] 37 [ 82.9] F X m S.D 21 53 . 32. [40. 9 3] 10 22. 11. [39 3 9 .7] 31 [ 80.0] r s (between) = +1 r s ( t o t a l ) T a b l e X X V I I I - E f f e c t of r f o r between s c o r e s on t o t a l s c o r e r when r fbetween) =+1 Tabl e X X V I I I d i s p l a y s t h e r e s u l t i n g rank c o r r e l a t i o n c o e f f i c i e n t ( r =+1) f o r raw s c o r e t o t a l s w i t h b a l a n c e d s c o r e s t o t a l s when the between s c o r e s rank c o r r e l a t i o n c o e f f i c i e n t i s v e r y l a r g e ( r =+1). When t h e s c o r e s o f the second assignment are r e a r r a n g e d t o become rank o r d e r e d as 2,3,1,5,6,4 (as i n Tabl e X X I X ) , the between s c o r e rank c o r r e l a t i o n c o e f f i c i e n t becomes +.657 w h i l e the rank c o r r e l a t i o n c o e f f i c i e n t f o r aggregate raw s c o r e s w i t h a g g r e g a t e b a l a n c e d s c o r e s becomes + . 942 . 85 Assignment ;udent 1 2 T o t a l A 100 [64. 3] 31 [57. 3] 131 [121.6] B 83 [59. 1] 26 [53. 1] 109 [112.2] C 58 [51. 5] 40 [64. 8] 98 [116.3] D 31 [43. 3] 12 [41. 4] 43 [ 84.7] E 25 [41. 5] 10 [39. 7] 35 [ 81.2] F 21 [40. 3] 15 [43. 9] 36 [ 84.2] X •m 53 22 . 3 S.D. 32. 9 11. 9 r s . 657 r = s . 942 Tab l e XXIX - E f f e c t of r f o r between s c o r e s on t o t a l s c o r e r when r (between) =.657 I f t h e o r d e r o f the second assignment i s a l t e r e d t o 3,1,5,6,4,2 the r f o r between s c o r e s and the r f o r t o t a l s s s c o r e s become +.14 and +.83 r e p e c t i v e l y . F i n a l l y , i f the ranks a re changed t o 6,5,4,3,2,1 the n the r g f o r between s c o r e s and r f o r t o t a l s c o r e s become -1 and +.31. s I t i s i n t e r e s t i n g t o note t h a t , w i t h t h e s e examples, o n l y the l a s t has a rank c o r r e l a t i o n c o e f f i c i e n t f o r raw s c o r e t o t a l s and b a l a n c e d s c o r e t o t a l s t h a t i s not s i g n i f i c a n t ( i e . < .829) a t the alpha=.10 l e v e l o f s i g n i f i c a n c e (n=6). These examples show how changes i n t h e rank c o r r e l a t i o n c o e f f i c i e n t f o r between s c o r e s a f f e c t the rank c o r r e l a t i o n c o e f f i c i e n t f o r raw s c o r e t o t a l s and b a l a n c e d s c o r e t o t a l s . C o n s i d e r i n g a h i g h m a j o r i t y o f t h e sample i n t h i s s t u d y 86 compute t o t a l s by summing raw s c o r e s (see Ta b l e X V I I I page 66) and c o n s i d e r i n g c l a s s assignments a r e e x p e c t e d t o c o r r e l a t e p o s i t i v e l y w i t h each o t h e r , i t becomes more apparent why t h e r a n k i n g o f t h e raw s c o r e t o t a l s c o r r e l a t e d h i g h l y w i t h the r a n k i n g of t h e b a l a n c e d s c o r e t o t a l s i n t h i s s t u d y , i n s p i t e of t h e many l a r g e d i f f e r e n c e s i n raw s c o r e v a r i a b i l i t y . R e s e a r c h Q u e s t i o n 2 In r e s ponse t o the second r e s e a r c h q u e s t i o n r e g a r d i n g t h e r e l i a b i l i t y o f t h e a g g r e g a t i o n and w e i g h t i n g of raw s c o r e s f o r g r a d i n g p u r p o s e s , the sample's r e c o r d s h e e t s and t h e q u e s t i o n n a i r e responses p e r t a i n i n g t o g r a d i n g t e c h n i q u e s were a n a l y z e d . The a n a l y s i s r e v e a l e d a h i g h degree o f s i m i l a r i t y between the a g g r e g a t i o n methods e v i d e n t on t h e r e c o r d s h e e t s and the a g g r e g a t i o n t e c h n i q u e s d e s c r i b e d on the q u e s t i o n n a i r e . Based on t h e s e r e s u l t s and on t h e rank c o r r e l a t i o n c o e f f i c i e n t s o b t a i n e d (as d i s c u s s e d i n Res e a r c h Q u e s t i o n 1 ) , i t can be c o n c l u d e d t h a t t h i s sample o f s u b j e c t s c o l l a t e d raw s c o r e s i n a r e a s o n a b l y r e l i a b l e manner. The a n a l y s i s o f t h e d a t a a l s o i n d i c a t e d t h a t most res p o n d e n t s d i d not attempt t o a p p l y w e i g h t i n g f a c t o r s t o raw s c o r e s . As i n d i c a t e d p r e v i o u s l y i n t h i s s t u d y (page 7 ) , w e i g h t i n g raw s c o r e s and how much t o weight i s a m a t t e r f o r p r o f e s s i o n a l judgement. However, many s u b j e c t s seemed o b l i v i o u s t o t h e f a c t t h a t by not w e i g h t i n g , s m a l l and r e l a t i v e l y i n s i g n i f i c a n t a s s ignments may e a s i l y c o n t r i b u t e as 87 much o r more t o an aggr e g a t e d s c o r e as does a major t e s t o r p r o j e c t . Most o f t h e re s p o n d e n t s i n t h i s s t u d y d i d not a p p l y w e i g h t s and t h e r e b y a l l o w e d each assignment t o c o n t r i b u t e i t s " n a t u r a l w e i g h t " toward t h e a g g r e g a t e d s c o r e . Many of t h o s e who d i d attempt t o we i g h t the raw s c o r e s c o n s i d e r e d an assignment's t o t a l p o s s i b l e mark t o be t h e w e i g h t i n g f a c t o r . F o r example, an assignment t h a t has a t o t a l p o s s i b l e s c o r e of 100 was thought by many t o c o n t r i b u t e t w i c e as much t o an aggreg a t e s c o r e as would an assignment t h a t has a t o t a l p o s s i b l e s c o r e o f 50. The example on page 9 o f t h i s s t u d y demonstrates t h e i n a p p r o p r i a t e n e s s o f t h i s method as a means t o w e i g h t raw s c o r e s . Assignments w i t h a l a r g e r t o t a l p o s s i b l e v a l u e may p o s s i b l y c o n t r i b u t e l e s s toward an aggregate s c o r e t h a n does an assignment w i t h a s m a l l e r t o t a l p o s s i b l e v a l u e . The key f a c t o r s i n r e l i a b l e w e i g h t i n g a r e t h e s t a n d a r d d e v i a t i o n v a l u e s r a t h e r t h a n the t o t a l p o s s i b l e s c o r e s (see page 10 ) . T h i s n o t i o n was s u p p o r t e d by such a u t h o r s as Ahmann and G l o c k (see page 6 ) , E b e l (see page 2 7 ) , and Gronlund (see page 2 8 ) . I n l i g h t o f t h e s e f a c t s , t h e r e s u l t s o f t h i s s t u d y would suggest t h a t the s u b j e c t s ' w e i g h t i n g t e c h n i q u e s a r e l e s s t h a n r e l i a b l e . U n r e l i a b l e w e i g h t i n g methods may have had a p r o f o u n d e f f e c t on the l e t t e r grades r e p o r t e d i n t h i s s t u d y . S i n c e 95% of the s u b j e c t s e i t h e r r e l i e d on n a t u r a l w e i g h t i n g s o r a p p l i e d u n r e l i a b l e w e i g h t i n g methods (see Ta b l e XV), many s t u d e n t s may have r e c e i v e d an ag g r e g a t e s c o r e t h a t was not i n d i c a t i v e o f the emphasis p l a c e d on t h e assignments and c o u r s e o b j e c t i v e s . 88 F o r example, a minor assignment d e a l i n g w i t h o n l y a few c o u r s e c o n c e p t s c o u l d have had more i n f l u e n c e on the aggr e g a t e s c o r e t h a n d i d a comprehensive p r o j e c t t h a t a d d r e s s e d s e v e r a l c o u r s e o b j e c t i v e s . However, i f the raw s c o r e s had been p r o p e r l y b a l a n c e d and we i g h t e d t o r e f l e c t t h e emphasis o f the c o u r s e , t h e n many s t u d e n t s i n t h i s s t u d y would have l i k e l y r e c e i v e d a d i f f e r e n t and more r e p r e s e n t a t i v e a g g r e g a t e s c o r e . F u r t h e r m o r e , some s t u d e n t s may have a l s o r e c e i v e d d i f f e r e n t l e t t e r grades based on t h e r a n k i n g o f the new ag g r e g a t e s c o r e s . C o n s e q u e n t l y , l e t t e r grades based on b a l a n c e d and a p p r o p r i a t e l y w e i g h t e d raw s c o r e s have g r e a t e r v a l i d i t y and r e l i a b i l i t y as i n d i c a t o r s of t h e s t u d e n t s ' r e l a t i v e achievement. C o n s i d e r i n g t h e p o s s i b l e consequences o f u n r e l i a b l e g r a d i n g p r a c t i s e s (see page 2 ) , the r e s u l t s of t h i s s t u d y suggest t h a t many s t u d e n t s may have r e c e i v e d m i s l e a d i n g and i n c o n s i s t e n t i n f o r m a t i o n i n the form o f u n r e l i a b l e l e t t e r g r a d e s . D e c i s i o n makers, u s i n g t h e s e grades as a s o u r c e o f i n f o r m a t i o n , c o u l d be prone t o making erroneous d e c i s i o n s . For example, t h e l e t t e r grades awarded some s t u d e n t s may i n d i c a t e t h a t the c o u r s e o b j e c t i v e s have been s u c c e s s f u l l y completed when i n f a c t , the g o a l s have not been met. In such i n s t a n c e s , s t u d e n t s c o u l d be promoted t o the nex t c o u r s e o r l e v e l when such a pr o m o t i o n i s not j u s t i f i e d . The o p p o s i t e c o u l d a l s o o c c u r . S t u d e n t s may be r e t a i n e d t o r e p e a t o r r e v i e w t h e c o u r s e o b j e c t i v e s when i n f a c t the m a t e r i a l has been w e l l mastered and such a r e t e n t i o n i s n o t w a r r a n t e d . 8 9 O t h e r s may a l s o be m i s g u i d e d by u n r e l i a b l e g r a d e s . For example, t h o s e who n o r m a l l y use s t u d e n t grades as a means of e v a l u a t i n g c o u r s e s , programs, o r p e d a g o g i c a l t e c h n i q u e s , may implement u n n e c e s s a r y o r i n c o r r e c t m o d i f i c a t i o n s based on the u n r e l i a b l e i n f o r m a t i o n . In s h o r t , l e t t e r grades d e r i v e d from u n r e l i a b l e w e i g h t i n g p r a c t i s e s may be l a c k i n g i n v a l i d i t y and r e l i a b i l i t y and may m i s l e a d o t h e r s t o f o r m u l a t e i n c o r r e c t c o n c l u s i o n s and d e c i s i o n s . Raw s c o r e s s h o u l d be b a l a n c e d and w e i g h t e d i n such a manner as t o make the a g g r e g a t e s c o r e r e p r e s e n t a t i v e o f the emphasis p l a c e d on the c o u r s e c o n t e n t and the v a r i o u s c o u r s e o b j e c t i v e s . Based on the r e s u l t s o f t h i s s t u d y , i t can be c o n c l u d e d t h a t r e l a t i v e l y few t e a c h e r s w e i g h t raw s c o r e s r e l i a b l y . R e s earch Q u e s t i o n 3 In r e s p o n s e t o the t h i r d r e s e a r c h q u e s t i o n p e r t a i n i n g t o the r e l i a b i l i t y and f a i r n e s s o f the sample's g r a d e s , the r e s p o n d e n t s ' raw s c o r e grades and b a l a n c e d s c o r e grades were compared. The r e s u l t s were r e c o r d e d and t a b u l a t e d i n terms of the change t h a t o c c u r r e d t o both l e t t e r grade and c a t e g o r y when b a l a n c e d grades r e p l a c e d raw s c o r e g r a d e s . To a s s e s s f a i r n e s s o f the g r a d e s , t h e methods used t o c a l c u l a t e aggregate s c o r e s f o r s t u d e n t s who m i s s e d r e c e i v i n g a s c o r e on one o r more assignments were a n a l y z e d . These r e s u l t s would i n d i c a t e t h a t , i n some 90 c i r c u m s t a n c e s , s t u d e n t s c o u l d be u n f a i r l y p e n a l i z e d o r rewarded f o r m i s s i n g an assignment. The f o l l o w i n g example demonstrates how a s t u d e n t might be p e n a l i z e d by m i s s i n g an assignment when means and s t a n d a r d d e v i a t i o n s a r e not c o n s i d e r e d . ASSIGNMENT SCORES (%) Student 1 2 3 T o t a l A 50 30 100 60 B 40 28 ab 34 C 30 26 74 43 D 10 20 60 30 Mean 33 26 78 S.D. 17 4 20 T a b l e XXX - Compensation f o r mi s s e d assignment T a b l e XXX shows how s t u d e n t B has dropped t o the t h i r d rank p o s i t i o n on t h e b a s i s of t h e f i r s t two assignment s c o r e s , even though he has c o n s i s t e n t l y ranked second. By m i s s i n g t h e t h i r d assignment w i t h i t s h i g h e r mean s c o r e , s t u d e n t B has been p e n a l i z e d one rank p o s i t i o n . On the o t h e r hand, i f the missed assignment were c a l c u l a t e d by a v e r a g i n g b a l a n c e d s c o r e s ( i e . T - s c o r e s ) , t h e n h i s second p l a c e r a n k i n g would be p r e s e r v e d . The b a l a n c e d s c o r e averages f o r s t u d e n t s A, B, C, and D would be 60, 55, 49, and 37 r e s p e c t i v e l y . C l e a r l y , s t u d e n t B has not been p e n a l i z e d nor rewarded under the 91 b a l a n c e d s c o r e method of compensation. The r e s u l t s o f t h i s s t u d y would suggest t h a t t e a c h e r s " grades may l a c k r e l i a b i l i t y and, when s t u d e n t s have mi s s e d a s s i g n m e n t s , t h e t e c h n i q u e f o r compensation may not always be f a i r . R e s e a r c h Q u e s t i o n 4 In c o n s i d e r i n g the f o u r t h r e s e a r c h q u e s t i o n r e g a r d i n g t h e j u s t i f i c a t i o n of t h e ti m e and expense r e q u i r e d t o r e t r a i n t e a c h e r s , t h e res p o n s e s t o the a t t i t u d i n a l s e c t i o n of the q u e s t i o n n a i r e were a n a l y z e d and the r e s u l t s t a b u l a t e d i n T a b l e s X I I I t h r o u g h X V I I . These r e s u l t s , i n c o n j u n c t i o n w i t h the p r e v i o u s l y d i s c u s s e d r e s u l t s , would i n d i c a t e t h a t i n - s e r v i c e i n s t r u c t i o n and p r e - s e r v i c e t r a i n i n g i n p a r t i c u l a r a s p e c t s o f g r a d i n g and r e p o r t i n g would be j u s t i f i e d f o r many members o f t h e r e s e a r c h sample. Areas of g r e a t e s t need appear t o be tho s e c o n c e r n i n g t h e r e l i a b l e w e i g h t i n g o f raw s c o r e s , the r e l i a b l e a l l o c a t i o n o f l e t t e r g r a d e s , and the r e l i a b l e c a l c u l a t i o n o f compensation s c o r e s f o r s t u d e n t s who have missed a s s i g n m e n t s . C u r r e n t l i t e r a t u r e i n d i c a t e s t h a t many t e a c h e r s a r e uneasy o r d e f e n s i v e c o n c e r n i n g t h e i r grades and g r a d i n g p r o c e d u r e s (see page 5 ) . D u r i n g t h e c o u r s e of t h i s s t u d y , t h i s n o t i o n was v e r i f i e d i n f o r m a l l y on s e v e r a l o c c a s i o n s . Comments, such as " . . . s u r e hope t h e s e marks a r e a l r i g h t " , o r 92 " I ' v e never been t r a i n e d i n t h i s a r e a " , and o t h e r s i m i l a r remarks were f r e q u e n t l y made by the r e s p o n d e n t s . These comments and t h e u n c e r t a i n t y accompanying them would s u g g e s t t h a t t i m e devoted t o p r e - s e r v i c e and i n - s e r v i c e would be w e l l s p e n t . Weaknesses To Be C o n s i d e r e d The framework of the proposed s t u d y was c o n s i d e r e d t o be sound. Many of t h e e x t r a n e o u s f a c t o r s t h a t c o u l d a f f e c t t h e r e s u l t s were c o n t r o l l e d by r a n d o m i z a t i o n wherever p o s s i b l e . A l t h o u g h t h i s p r o j e c t e n c o u n t e r e d few s h o r t c o m i n g s , t h o s e t h a t d i d e x i s t were c o n s i d e r e d t o be minor and not s e r i o u s enough t o a d v e r s e l y a f f e c t the r e s u l t s . One of the most s e r i o u s o b s t a c l e s concerned th e sample. A l t h o u g h of s u f f i c i e n t s i z e f o r an adequate s t u d y , the sample was c o m p r i s e d e n t i r e l y o f v o l u n t e e r s . As a r e s u l t , a b i a s may have been i n t r o d u c e d i n t o the s t u d y . The p o s s i b l e b i a s would r e s u l t from the sample n o t b e i n g t o t a l l y r e p r e s e n t a t i v e of t h e t a r g e t p o p u l a t i o n . Those who p a r t i c i p a t e d may have c o n t r i b u t e d a p a r t i c u l a r b i a s w h i l e t h o s e who d e c l i n e d t o p a r t i c i p a t e may have d e p r i v e d the s t u d y of d a t a unique t o n o n - p a r t i c i p a n t s . For example, t h o s e who a r e p a r t i c u l a r l y d e f e n s i v e about t h e i r g r a d i n g p r o c e d u r e s o r about the r e l i a b i l i t y o f t h e i r grades may not have v o l u n t e e r e d . On the o t h e r hand, perhaps o n l y t h o s e who a r e c o n f i d e n t t h e i r g r a d i n g 93 t e c h n i q u e s a r e d e f e n s i b l e o r t h o s e who t a k e a p a r t i c u l a r i n t e r e s t i n marking and g r a d i n g v o l u n t e e r e d t o become members of t h e sample. Teachers b e l o n g i n g t o t h e s e and o t h e r such groups s h o u l d be i n c l u d e d i n t h e sample s i n c e g e n e r a l i z a t i o n s w i l l be made about the p o p u l a t i o n t o which t h e y b e l o n g . However, w i t h v o l u n t e e r samples, t h e s e v a r i a b l e s a r e d i f f i c u l t t o c o n t r o l . Other weaknesses i n t h i s s t u d y concerned the q u e s t i o n n a i r e t h a t was used t o c o l l e c t d a t a on g r a d i n g t e c h n i q u e s . The q u e s t i o n o f f e r i n g f i c t i t i o u s d a t a t o be c o l l a t e d and ranked posed p a r t i c u l a r problems f o r many o f the r e s p o n d e n t s . Most r e c o g n i z e d , from the i n f o r m a t i o n g i v e n i n the d i r e c t i o n s , t h a t some s o r t o f w e i g h t i n g was r e q u i r e d , even when th e y d i d not c u s t o m a r i l y w e i g h t raw s c o r e s . Some i g n o r e d the d i r e c t i o n s on w e i g h t i n g and a p p l i e d t h e i r r e g u l a r g r a d i n g methods. Others a t t e m p t e d t o accommodate the d i r e c t i o n s by d e v e l o p i n g new and u n f a m i l i a r s t r a t e g i e s t o we i g h t t h e raw s c o r e s i n t h e q u e s t i o n . The f i c t i t i o u s d a t a s e c t i o n posed a n o t h e r weakness, a l t h o u g h l e s s s e r i o u s and u n r e l a t e d t o the one p r e v i o u s l y d i s c u s s e d . An o p p o r t u n i t y f o r t h e re s p o n d e n t s t o c a l c u l a t e an aggreg a t e s c o r e f o r a s t u d e n t who had been absent s h o u l d have been i n c l u d e d i n the f i c t i t i o u s d a t a . T h i s would have p r o v i d e d a t h i r d s o u r c e of i n f o r m a t i o n f o r t h a t p a r t i c u l a r a s p e c t o f g r a d i n g . A l t h o u g h adequate, t h e r e c o r d s h e e t s and the w r i t t e n r e s p o n s e s were t h e o n l y s o u r c e s of d a t a f o r t h i s t o p i c . As a r e s u l t of t h e s e weaknesses, t h e f i c t i t i o u s d a t a 94 s e c t i o n i n t h i s s t u d y s h o u l d be r e v i e w e d and r e v i s e d b e f o r e b e i n g used on o t h e r s i m i l a r r e s e a r c h p r o j e c t s . A n o t h e r minor weakness concerned the b a l a n c e d raw s c o r e t o t a l s t h a t were g i v e n t o t h e res p o n d e n t s t o r e g r a d e . U n l i k e raw s c o r e s , T - s c o r e s do not have a maximum p o s s i b l e s c o r e ; r a t h e r , t h e y a r e c o n t i n u o u s . When t h e b a l a n c e d T - s c o r e s were s u p p l i e d t o the res p o n d e n t s t o be r e g r a d e d , a maximum s c o r e s h o u l d have been i n c l u d e d f o r t h o s e who u t i l i z e per c e n t g r a d i n g t e c h n i q u e s . F o r t h o s e r e s p o n d e n t s who i n q u i r e d , a s c o r e of 275 was p r o v i d e d as t h i s would exceed a l l b a l a n c e d s c o r e t o t a l s and t h e r e f o r e would y i e l d p e r c e n t a g e s l e s s t h a n 100 per c e n t . A l t h o u g h t h i s weakness caused some c o n f u s i o n among the r e s p o n d e n t s , i t was not c o n s i d e r e d s e r i o u s enough t o f l a w the r e s u l t s . R a t h e r , i t may have c o n t r i b u t e d toward a b e t t e r u n d e r s t a n d i n g o f the d i f f i c u l t y t e a c h e r s e x p e r i e n c e when th e y a s s i g n l e t t e r grades t o aggr e g a t e d s c o r e s (see page 79) . F u t u r e R e s e a r c h With t h e i n t r o d u c t i o n o f computers i n t o most s c h o o l s , the o p p o r t u n i t i e s f o r improvement i n t h e measurement, e v a l u a t i o n , g r a d i n g , and r e p o r t i n g of s t u d e n t achievement have expanded d r a m a t i c a l l y and o f f e r many new ar e a s f o r f u t u r e r e s e a r c h . These a r e a s have been d i v i d e d i n t o two main s e c t i o n s t o f a c i l i t a t e d i s c u s s i o n . The f i r s t a r e a o f r e s e a r c h p o s s i b i l i t i e s d e a l s s p e c i f i c a l l y w i t h t h o s e t o p i c s r e l a t e d t o 95 t e a c h e r s ' g r a d i n g p r a c t i s e s . The second a r e a d e a l s w i t h measurement and e v a l u a t i o n o f s t u d e n t achievement on a more g l o b a l s c a l e . W h i l e t h i s s t u d y touched o n l y a few a s p e c t s of t e a c h e r g r a d i n g p r a c t i s e s , i t l e f t many q u e s t i o n s unanswered. A t the same t i m e , i t r a i s e d more q u e s t i o n s t o be r e s o l v e d . Among the q u e s t i o n s t h a t s t i l l need t o be ad d r e s s e d i s the c o n c e r n r e g a r d i n g the a p p r o p r i a t e n e s s o f u s i n g T - s c o r e s i n the c l a s s r o o m s i t u a t i o n . The l i n e a r t r a n s f o r m a t i o n o f raw s c o r e s t o T - s c o r e s f i r s t r e q u i r e s t h a t the raw s c o r e s be c o n v e r t e d t o z s c o r e s . F u r t h e r s t u d i e s s h o u l d be c a r r i e d o u t t o dete r m i n e i f t h e use o f T-sco r e s i s a p p r o p r i a t e f o r w e i g h t i n g and combining raw achievement s c o r e s . Research o p p o r t u n i t i e s e x i s t i n the r e p l i c a t i o n o f t h i s s t u d y . One p o s s i b i l i t y would be t o determine i f l o c a t i o n has an e f f e c t on t e a c h e r s ' g r a d i n g p r a c t i s e s . F o r example, a stud y c o u l d i n v o l v e s a m p l i n g t e a c h e r s from d i f f e r e n t r e g i o n s of the p r o v i n c e t o de t e r m i n e i f d i f f e r e n c e s i n g r a d i n g p r a c t i s e s can be a t t r i b u t e d t o g e o g r a p h i c l o c a t i o n . I t might be argued t h a t t e a c h e r s w i t h s u p e r i o r g r a d i n g p r a c t i s e s a r e those who l i v e i n c l o s e p r o x i m i t y t o u n i v e r s i t i e s . C o m puterized r e c o r d k e e p i n g and mark management i s anot h e r a r e a t h a t s h o u l d a l s o be s t u d i e d . I f t e a c h e r s a r e g i v e n computer programs t h a t w i l l r e l i a b l y b a l a n c e , a g g r e g a t e , and weight raw achievement s c o r e s , would t h e y be w i l l i n g t o a l t e r t h e i r p r e s e n t g r a d i n g p r a c t i s e s f o r the newer computer approach? I f i t can be shown t h a t t e a c h e r s would r e a d i l y 96 a c c e p t t h e new computer g r a d i n g method, th e n s t u d i e s would a l s o have t o be done t o a s c e r t a i n the most e f f e c t i v e and u s e f u l computer program c u r r e n t l y a v a i l a b l e . T h i s s t u d y has p o s s i b l y i d e n t i f i e d a r e l a t i o n s h i p between (a) t h e e f f e c t s d i f f e r e n t s t a n d a r d d e v i a t i o n s have on raw s c o r e t o t a l s and b) the rank c o r r e l a t i o n c o e f f i c i e n t s f o r between s c o r e s . The e f f e c t s o f l a r g e d i f f e r e n c e s i n v a r i a n c e appear t o be d i m i n i s h e d by l a r g e p o s i t i v e rank c o r r e l a t i o n s f o r between s c o r e s . S t u d i e s s h o u l d be done t o d e t e r m i n e i f t h i s r e l a t i o n s h i p a p p l i e s t o a l l s e t s o f raw s c o r e s and, i f so, how the r e l a t i o n s h i p c o u l d b e s t be a p p l i e d t o c l a s s r o o m g r a d i n g p r a c t i s e s . A n other f u t u r e r e s e a r c h p o s s i b i l i t y r e l a t e s t o the rank c o r r e l a t i o n c o e f f i c i e n t s f o r between s c o r e s . S t u d i e s s h o u l d be done t o d e t e r m i n e how w e l l s e t s o f c l a s s r o o m achievement s c o r e s a c t u a l l y c o r r e l a t e . As i n d i c a t e d i n t h i s s t u d y , i t would be e x p e c t e d t h a t t h e more c a p a b l e s t u d e n t would n o r m a l l y a t t a i n the h i g h e r marks w h i l e the l e s s a b l e s t u d e n t would a c h i e v e the lower s c o r e s . The rank p o s i t i o n of each s t u d e n t would t h e r e f o r e be r e l a t i v e l y s t a b l e . R esearch s h o u l d be conducted t o d e t e r m i n e i f t h i s i s a c t u a l l y t r u e f o r most c l a s s e s . The second a r e a f o r f u t u r e r e s e a r c h , t h e g e n e r a l t o p i c of measurement and e v a l u a t i o n o f s t u d e n t achievement, a l s o needs t o be a d d r e s s e d . More s t u d i e s need t o be done t o e x p l o r e the r e l i a b i l i t y of t e a c h e r c o n s t r u c t e d e v a l u a t i o n i n s t r u m e n t s . G r a d i n g p r o c e d u r e s can be " s t a t e o f the a r t " , but u n l e s s t h e 97 i n s t r u m e n t s used t o measure s t u d e n t achievement are r e l i a b l e and v a l i d , s t u d e n t grades w i l l s t i l l l a c k r e l i a b i l i t y , v a l i d i t y and f a i r n e s s . S t u d i e s need t o be u n d e r t a k e n t o answer such q u e s t i o n s as the f o l l o w i n g : 1. Are t e a c h e r c o n s t r u c t e d t e s t s r e l i a b l e and v a l i d ? 2. Are t h e r e ways t o improve t h e s e i n s t r u m e n t s ? 3. Are t h e r e b e t t e r ways f o r t e a c h e r s t o c o n s t r u c t measurement i n s t r u m e n t s ? These a r e q u e s t i o n s t h a t s h o u l d be i n v e s t i g a t e d . Another p o s s i b i l i t y f o r f u t u r e r e s e a r c h i n v o l v e s the medium used t o a c q u i r e measurement d a t a on s t u d e n t achievement. With computers becoming more p o p u l a r i n s c h o o l s , s t u d i e s s h o u l d be performed t o d e t e r m i n e i f the r e l i a b i l i t y of c l a s s r o o m t e s t s can be improved t h r o u g h the use of the computer. With the a v a i l a b i l i t y o f computers t o a s s i s t t e a c h e r s i n e v a l u a t i n g s t u d e n t achievement, i t i s hoped t h a t f u t u r e r e s e a r c h w i l l o f f e r new and more r e l i a b l e methods of m e a s u r i n g , e v a l u a t i n g , g r a d i n g , and r e p o r t i n g s t u d e n t achievement. 98 B i b l i o g r a p h y Ahmann, J . S., & G l o c k , M. D. (1981). E v a l u a t i n g s t u d e n t p r o g r e s s - p r i n c i p l e s of t e s t s and measurements ( 6 t h ed . ) . B o s t o n : A l l y n & Bacon. Cohen, D. (1973). E v a l u a t i o n r e i n t e r p r e t e d . A u s t r a l i a n S c i e n c e Teachers J o u r n a l , 1 9 ( 2 ) , 29-34. Curwin, R. (1978). The grades of w r a t h : Some a l t e r n a t i v e s . L e a r n i n g , 6̂ , 60-64. E b e l , R. L. (1972). E s s e n t i a l s of e d u c a t i o n a l measurement. Englewood C l i f f s : P r e n t i c e - H a l l . E b e l , R. L. (1974). S h a l l we g e t r i d of g r a d e s ? . NCME Measurement i n E d u c a t i o n , 5 ( 4 ) , 1-5. E r l i n g , A. S. (1979). Combining s t u d e n t t e s t s c o r e s r e l i a b l y . E d u c a t i o n a l R e s e a r c h and Methods, 11 (2 ),42-50 . G e n s l e y , J . T. (1969). A new method o f e v a l u a t i o n f o r g i f t e d s t u d e n t s : A d i a r y of l e a r n i n g . G i f t e d C h i l d r e n Q u a r t e r l y , 13, 119-25. G l a s s , G. V., & S t a n l e y , J . C. (1970). S t a t i s t i c a l methods i n e d u c a t i o n and p s y c h o l o g y . Englewood C l i f f s , New J e r s e y : P r e n t i c e - H a l l . Gray, L. R. (1980). E d u c a t i o n a l e v a l u a t i o n and measurement - Competencies f o r a n a l y s i s and a p p l i c a t i o n . Columbus: M e r r i l l . Green, H. A., J o r g e n s e n , A. N., & G e r b e r i c h , J . R. (1962). Measurement and e v a l u a t i o n i n the e l e m e n t a r y s c h o o l . New York: McKay. Gronlund N.E. (1974). Improving M a r k i n g and R e p o r t i n g i n Cl a s s r o o m I n s t r u c t i o n . New York: M a c M i l l a n . 99 G r o n l u n d , N. E. (1981). Measurement and e v a l u a t i o n i n t e a c h i n g (4t h e d . ) . New York: M a c M i l l a n . G u i l f o r d , J . P., & F r u c h t e r , B. (1973). Fundamental s t a t i s t i c s i n p s y c h o l o g y and e d u c a t i o n ( 5 t h e d . ) . New York: M c G r a w - H i l l . H i l l s , J . R. (1981). Measurement and e v a l u a t i o n i n t h e c l a s s r o o m . Columbus: M e r r i l l . H o p kins, K. D., & S t a n l e y , J . C. (1981). E d u c a t i o n a l and p s y c h o l o g i c a l measurement and e v a l u a t i o n ( 6 t h ed.). Englewood C l i f f s , New J e r s e y : P r e n t i c e - H a l l . H o pkins, C. D., & A n t e s , R. L. (1979). C l a s s r o o m t e s t i n g , A t a s c a , I l l i n o i s : Peacock. H o r r o c k s , J . E., & Schoonover, T. I . (1968). Measurement f o r t e a c h e r s . C o l u m b u s : M e r r i l 1 . I s a a c s , G. , & I m r i e , B. (1981). A case f o r p r o f e s s i o n a l judgment when combining marks. Assessment and E v a l u a t i o n i n H i g h e r E d u c a t i o n , j5, 3-25. Johnson, D. M., & Van O s d o l , B. M. (1974). A computer program f o r r e l i a b l e grade assignment. J o u r n a l of E d u c a t i o n a l Data P r o c e s s i n g , 1~L(2) , 11-19. L i e n , A. J . (1976). Measurement and e v a l u a t i o n of l e a r n i n g . Dubuque, Iowa: Brown. L i n d v a l l , C. M. (1967). M e a s u r i n g p u p i l achievement and a p t i t u d e . San F r a n c i s c o : H a r c o u r t , Brace & World Lyman, H. B. (1978). T e s t s c o r e s and what t h e y mean. Englewood C l i f f s , New J e r s e y : P r e n t i c e - H a l l . M a r s h a l l , M. S. (1971). Why grades a r e argued. S c h o o l and S o c i e t y , 99, 350-353. 1 Mehrens, W. A., & Lehmann, I . J . (1984). Measurement and e v a l u a t i o n i n e d u c a t i o n and p s y c h o l o g y ( 3 r d e d . ) . New York: CBS C o l l e g e . M i n i s t r y o f E d u c a t i o n , P r o v i n c e o f B r i t i s h Columbia. (1979). C o n s t r u c t i o n and use of c l a s s r o o m t e s t s . V i c t o r i a , BC: Aut h o r . M i n i s t r y o f E d u c a t i o n , P r o v i n c e o f B r i t i s h Columbia. (1979). G r a d i n g p r a c t i s e s : I s s u e s and a l t e r n a t i v e s . V i c t o r i a , BC: A u t h o r . N e l s o n , C. H. (1970). Measurement and e v a l u a t i o n i n the c l a s s r o o m . New York: M a c M i l l a n . R a t z l a f f , H. C. (1979). A t o E approaches t o g r a d i n g s t u d e n t achievement. V e c t o r , 2 1 ( 1 ) , 22-28. S t e w a r t , W. J . (1975). A m u l t i - d i m e n s i o n a l e v a l u a t i n g - r e p o r t i n g system i n t h e e l e m e n t a r y s c h o o l . Reading Improvement, 12, 174-76. T e n b r i n k , T. D. (1974). E v a l u a t i o n - A p r a c t i c a l g uide f o r t e a c h e r s . New York: M c G r a w - H i l l . Townsend, E. A., & Burke, P. J . (1975). U s i n g s t a t i s t i c s i n c l a s s r o o m i n s t r u c t i o n . New York: M a c M i l l a n . Wick, J . W. (1973). E d u c a t i o n a l measurement - Where a r e we g o i n g and how w i l l we know when we g e t t h e r e . Columbus M e r r i 1 1 . 101 Appendix A The f o l l o w i n g q u e s t i o n n a i r e was used t o c o l l e c t i n f o r m a t i o n on the s u b j e c t s ' demographic v a r i a b l e s , on t h e i r a t t i t u d e s and p e r s p e c t i v e s toward g r a d i n g , and on t h e i r g r a d i n g p r a c t i s e s and t e c h n i q u e s . I t was d i s t r i b u t e d t o each s u b j e c t a t t h e end o f t h e d a t a c o l l e c t i o n p e r i o d . 102 Respondent # GRADING TECHNIQUES SURVEY S e c t i o n I - Demographic I n f o r m a t i o n DIRECTIONS: C i r c l e t he l e t t e r t h a t c o r r e s p o n d s t o the c o r r e c t o r most a p p r o p r i a t e r e s p o n s e . 1. How many y e a r s t e a c h i n g e x p e r i e n c e do you have? (combine b o t h p u b l i c and p r i v a t e s c h o o l e x p e r i e n c e ) a. 0-5 b. 6-10 c. 11-15 d. 16-20 e. more than 2 0 2. What i s your t e a c h i n g assignment? a. f u l l t i me b. p a r t time I f " p a r t t i m e " , d e s c r i b e your t e a c h i n g assignment ( p e r c e n t a g e , s u b j e c t s t a u g h t , e t c . ) 3. How many S t a t i s t i c s c o u r s e s o r Measurement and E v a l u a t i o n c o u r s e s have you completed? ( d i s r e g a r d u n i t v a l u e ) a. none b. one c. two d. t h r e e e. more th a n t h r e e 4. How many y e a r s of u n i v e r s i t y have you completed? a. two b. t h r e e c. f o u r d. completed B a c h e l o r o f E d u c a t i o n degree e. completed MA o r MEd degree f . o t h e r ( e x p l a i n : ) 5. Which i s your age c a t e g o r y ? a. l e s s t h a n 25 y e a r s b. 25-30 y e a r s c. 31-40 y e a r s d. 41-50 y e a r s e. 51-60 y e a r s f. g r e a t e r t h a n 60 y e a r s S e c t i o n I I - G r a d i n g and R e p o r t i n g DIRECTIONS: You a r e t o e x p r e s s , on a f i v e - p o i n t s c a l e , t h e e x t e n t o f agreement between t h e f e e l i n g e x p r e s s e d i n each s t a t e m e n t and your own p e r s o n a l f e e l i n g . The f i v e p o i n t s a r e : S t r o n g l y D i s a g r e e (SD), D i s a g r e e (D), Undecided (U), Agree ( A ) , S t r o n g l y Agree (SA). You a r e t o e n c i r c l e t h e l e t t e r ( s ) which b e s t i n d i c a t e s how c l o s e l y you agree o r d i s a g r e e w i t h the f e e l i n g e x p r e s s e d i n each s t a t e m e n t . 1. My c o l l e a g u e s a t t h i s s c h o o l t a k e r e p o r t c a r d marks v e r y s e r i o u s l y . SD D U A SA 103 2. L e t t e r grades a r e an e f f e c t i v e method of i n f o r m i n g p a r e n t s o f t h e i r c h i l d ' s p r o g r e s s and achievement. SD D U A SA 3. To the b e s t o f my knowledge, p a r e n t s a r e g e n e r a l l y s a t i s f i e d w i t h the l e t t e r grade system o f r e p o r t i n g . SD D U A SA 4. I am v e r y c o n f i d e n t t h a t the l e t t e r grades I a s s i g n a r e a c c u r a t e and r e l i a b l e . SD D U A SA 5. I would l i k e t o l e a r n more about c o l l a t i n g and a s s i g n i n g l e t t e r grades t o raw s c o r e s . SD D U A SA 6. More i n - s e r v i c e s e s s i o n s on g r a d i n g and r e p o r t i n g s h o u l d be o f f e r e d . SD D U A SA 7. U n i v e r s i t i e s s h o u l d o f f e r more p r e - s e r v i c e i n s t r u c t i o n i n e f f e c t i v e g r a d i n g and r e p o r t i n g . SD D U A SA 8. The r e p o r t c a r d format a l l o w s s u f f i c i e n t i n f o r m a t i o n t o be communicated t o p a r e n t s , s t u d e n t s , e t c . SD D U A SA DIRECTIONS: Answer the f o l l o w i n g i n the space p r o v i d e d . 10. B r i e f l y d e s c r i b e how you combine assignment and t e s t s c o r e s t o d e t e r m i n e l e t t e r g r a d e s . (Use back o f page i f n e c e s s a r y ) I n r e f e r e n c e t o the method d e s c r i b e d i n #10, which s u b j e c t s are' graded t h i s way? ( a l l s u b j e c t s , mathematics, language a r t s , s c i e n c e , s o c i a l s t u d i e s , a r t , P.E., music, French) B r i e f l y d e s c r i b e how you a r r i v e a t a f i n a l term s c o r e f o r a s t u d e n t who has been absent f o r one o r more t e s t s o r a s s i g n m e n t s . In o t h e r words, how do you compensate f o r a l e g i t i m a t e l y m i ssed assignment? (Use back of page i f n e c e s s a r y ) 104 DIRECTIONS: s c o r e s ( p e r c a s s i g n m e n t s . 100. The f i c o u r s e ; t h e 10%; and t h e methods you t o c a l c u l a t e I n d i c a t e t h e h i g h e s t ; e t c The f o l l o w i n g e nt) t h a t f i v e Each assignme r s t assignment second and t h i r f i f t h , t he f i n n o r m a l l y use (a and r e c o r d the rank of each s ) i n the space r e c o r d sheet r e p r e s e n t s t h e raw s t u d e n t s o b t a i n e d on f i v e n t had a t o t a l p o s s i b l e s c o r e o f r e p r e s e n t e d about 10% o f the d about 20% each: t h e f o u r t h about a l t e s t , about 40%. Use the s i n d i c a t e d i n PART I I #10 above) T o t a l Score f o r each s t u d e n t , t u d e n t (1 = h i g h e s t ; 2 = n e x t p r o v i d e d . ASSIGNMENTS raw s c o r e s D NAME 1 2 3 4 5 1. Stud e n t A 50 55 65 57 95 2 . Student B 69 60 58 80 83 3 . Stud e n t C 45 72 52 71 75 4 . Stu d e n t D 75 95 45 83 25 5 . Student E 73 83 60 63 36 SCORE RANK Thank you f o r t a k i n g t h e t i m e t o complete t h i s s u r v e y .

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
United States 16 0
Canada 4 0
China 4 0
France 3 0
Pakistan 1 0
Japan 1 0
Germany 1 1
City Views Downloads
Unknown 7 1
Ashburn 7 0
Columbus 4 0
Beijing 4 0
San Jose 2 0
Sunnyvale 1 0
Kelowna 1 0
Mountain View 1 0
Redmond 1 0
Islamabad 1 0
Tokyo 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0054517/manifest

Comment

Related Items