Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Towards identifying critical factors in teaching behavioural sciences Haridas, T. P. 1976

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1976_A4_6 H37.pdf [ 8.53MB ]
Metadata
JSON: 831-1.0093508.json
JSON-LD: 831-1.0093508-ld.json
RDF/XML (Pretty): 831-1.0093508-rdf.xml
RDF/JSON: 831-1.0093508-rdf.json
Turtle: 831-1.0093508-turtle.txt
N-Triples: 831-1.0093508-rdf-ntriples.txt
Original Record: 831-1.0093508-source.json
Full Text
831-1.0093508-fulltext.txt
Citation
831-1.0093508.ris

Full Text

TOWARDS IDENTIFYING CRITICAL FACTORS IN TEACHING BEHAVIOURAL SCIENCES by T . P . HARIDAS B-.Coram., University of Madras, 1967 A THESIS SUBMITTED IN PARTIAL FULFILMENT THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE i n Business Administration i n the F a c u l t y o f Commerce and B u s i n e s s A d m i n i s t r a t i o n We a c c e p t t h i s t h e s i s as^  c o n f o r m i n g to the r e q u i r e d s t a n d a r d THE UNIVERSITY OF BRITISH COLUMBIA A p r i l , 1976 © T.P.Haridas, 1976 In p r e s e n t i n g t h i s t h e s i s in p a r t i a l f u l f i l m e n t o f the r e q u i r e m e n t s f o r an advanced degree at the U n i v e r s i t y o f B r i t i s h Co lumb ia , I a g ree that the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and s tudy . I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e c o p y i n g o f t h i s t h e s i s f o r s c h o l a r l y purposes may be g r a n t e d by the Head o f my Department o r by h i s r e p r e s e n t a t i v e s . It i s u n d e r s t o o d tha t c o p y i n g o r p u b l i c a t i o n o f t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be a l l o w e d w i thou t my w r i t t e n p e r m i s s i o n . Department o f ^  The U n i v e r s i t y o f B r i t i s h Co lumbia 20 75 Wesbrook Place Vancouver, Canada V6T 1W5 11 Abstract The major objective of this study is to i d e n t i f y the c r i t i c a l factors in e f f e c t i v e teaching in the IR&M Division of the Faculty of Commerce, at Univer-s i t y of B.C. It is hoped that many of the findings of this study w i l l also be relevant to teaching of other behavioural sciences l i k e anthropology, psychology and sociology. The findings of this study are also hoped to throw some light- on the content v a l i d i t y of the S.I.R Questionnaire currently used in the faculty for teacher evaluation. Three samples, 381 students, 9 professors and 7 administrative personnel were used for the present study. Three sets of behavioural scales for measuring Faculty performance (in the IR&M Division) are constructed using the Behavi oural ly (Anchored Rating Scale technique. In a p i l o t study using 62 students i t was found that the Behavioural Scales has high r e l i a b i l i t y . The i n i t i a l examination of the i n s t r u -ment showed i t s high content v a l i d i t y and freedom from central tendency and leniency problems. The implications and uses of the current study with some suggestions for improving the Behaviourally Anchored scales are also discussed. i i i TABLE OF CONTENTS CHAPTER I : INTRODUCTION Pa g e 1 CHAPTER I I G o a l s o f e d u c a t i o n M o d e l s o f t e a c h i n g TOWARDS I D E N T I F Y I N G THE E F F E C T I V E TEACHER 1 4 8 Some v i e w s on what makes t h e e f f e c t i v e t e a c h e r T o w a r d s i d e n t i f y i n g a c r i t e r i o n f o r e f f e c t i v e t e a c h i n g * A b r i e f l o o k a t some s u b s t i t u t e c r i t e r i a t h e c r i t e r i o n e f f e c t i v e n e s s S e l e c t i o n o f f o r t e a c h i n g T o w a r d s a model o f t e a c h e r e v a l u a t i o n u s i n g ' s u b s t i t u t e ' c r i t e r i a 8 11 13 23 31 CHAPTER I I I : BEHAVIOURALLY ANCHORED RATING INSTRUMENTS 37 I n t r o d u c t i on 37 A c o m p a r i s o n o f B e h a v i o u r a l l y A n c h o r e d R a t i n g I n s t r u m e n t s ( B A R I ) w i t h g r a p h i c s c a l e s 39 D e v e l o p m e n t o f B e h a v i o u r a l l y R a t e d I n s t r u m e n t s 42 P a s t r e s e a r c h on B e h a v i o u r a l l y A n c h o r e d I n s t r u m e n t s 46 Use o f BARI i n t e a c h i n g -e f f e c t i v e n e s s r e s e a r c h 52 C o n c l u s i o n 53 CHAPTER IV: DESIGNING BEHAVIOURAL SCALES FOR TEACHER EVALUATION E x i s t i n g m e a s u r e s o f t e a c h e r e f f e c t i v e n e s s i n F a c u l t y 54 i v of Commerce Ob jec t i ve s of the study Steps i n the study Psychometr ic p r ope r t i e s of the sea le CHAPTER V: IMPLICATIONS OF THE STUDY Other f i n d i n g s of the study Some uses of the* s tudy Some l i m i t a t i o n s of BARI research Towards some s o l u t i o n s 54 58 59 72 C 83 83 87 90 94 CHAPTER V I : CONCLUSION 97 BIBLIOGRAPHY 99 APPENDIX: A C r i t e r i a of teacFser e f f e c t i v e n e s s , g iven by Domas and Tiedeman B Genera l i zed schema f o r research in teach ing e f f e c t i v e n e s s g iven by M i t z e l C Specimen S.I.R. Ques t ionna i re D Specimen S.I.R. Summary score sheet E Table showing course-wise d i s t r i b u t i o n of student sample used f o r data c o l l e c t i o n , r e - t r a n s l a t i o n and s c a l i n g F Dimensions ( c a tego r i e s ) of teacher behaviour i d e n t i f i e d from i n c i d e n t s g iven by students G Dimensions ( c a tego r i e s ) of teacher behaviour i d e n t i f i e d from i n c i d e n t s g iven by pro fes so r s 111 112 113 115 119 120 124 V H D i m e n s i o n s ( c a t e g o r i e s ) o f t e a c h e r 126 b e h a v i o u r i d e n t i f i e d f r o m i n c i d e n t s g i v e n by t h e a d m i n i s t r a t i v e s t a f f I T a b l e s h o w i n g r e s u l t s o f t h e t e s t 128 f o r c o m p r e h e n s i v e n e s s o f i n c i d e n t s c o l 1 e c t e d J B e h a v i o u r a l s c a l e s I 129 ( s c a l e t o be f i l l e d i n by s t u d e n t s ) K B e h a v i o u r a l s c a l e s I I 1 3 8 ( s c a l e t o b e . f i l l e d i n by p r o f e s s o r s ) L B e h a v i o u r a l s c a l e s I I I 143 ( s c a l e t o be f i l l e d i n by a d m i n i s t r a t i v e s t a f f ) M S a m p l e q u e s t i o n n a i r e u s e d f o r 1 5 0 p i l o t s t u d y . • • N A n a l y s i s o f c e n t r a l t e n d e n c y and 1 6 5 l e n i e n c y e r r o r s o f t h e two i n s t r u m e n t s . -1-CHAPTER I INTRODUCTION I GOALS OF EDUCATION E d u c a t i o n i s one o f t h e m o s t c r u c i a l a c t i v i t i e s o f a n y modern s o c i e t y . Y e t , no s i n g l e d e f i n i t i o n o f e d u c a t i o n seems t o h a v e u n i v e r s a l a c c e p t a n c e . V a r i o u s a u t h o r s and e d u c a t i o n a l i s t s h a v e d e f i n e d ' e d u c a t i o n ' i n w i d e l y d i f f e r e n t ways d e p e n d i n g upon t h e i r s p e c i f i c p h i l o s o p h i e s and o r i e n t a t i o n s . T h u s , w h i l e some h a v e l o o k e d a t e d u c a t i o n as a t o o l t o i n c r e a s e t h e i n d i v i -d u a l ' s s p e c i f i c s k i l l s ( D o d g e , 1973; F r e e m a n , 1975) o t h e r s h a v e d e f i n e d i t t o mean an a c t i v i t y c o m m i t t i n g a s t u d e n t t o a way o f l i f e ( F r a n z , 1 9 7 2 ) , o r an a c t i v i t y i m p a r t i n g k n o w l e d g e t o an i n d i v i d u a l ( A c q u i n a s , q u o t e d by P e g i s , 1 9 4 8 ) , o r s a v i n g t h e s o u l o f t h e i n d i v i d u a l ( K i e r k e g a a r d , 1 9 4 6 ) , o r i n v o l v i n g t h e t r a i n i n g o f t h e mind and c h a r a c t e r o f t h e s t u d e n t c o n c e r n e d ( R u s s e l l , 1 9 2 6 ) . V a r i a t i o n s and e x t e n s i o n s o f t h e s e a r e a l s o g i v e n by o t h e r w r i t e r s and e d u c a t i o n a -l i s t s ( B r u b a c h e r , 1939; Dewey, 1916; H o m e , 1927 ; T h e l e n , 1 9 6 0 ) . T h e s e v a r i e d d e f i n i t i o n s o f e d u c a t i o n s h o u l d -2-s u g g e s t t o us t h e p r o b l e m s i n v o l v e d i n d e f i n i n g t h e p u r p o s e o f e d u c a t i o n and i n i d e n t i f y i n g t h e s e t o f c h a r a c t e r i s t i c s w h i c h make t h e ' i d e a l ' . t e a c h e r a n d good t e a c h i n g . I n d e e d , t h e g o a l s o f e d u c a t i o n h a v e v a r i e d s o much o v e r t i m e t h a t t e a c h i n g t o d a y i s q u i t e d i f f e r e n t i n p u r p o s e and f o r m f r o m what G r e e k s c a l l e d ' t e c h n e ' d e n o t i n g good t r a i n i n g . T h u s , f o r S o c r a t e s and I s o c r a t e s t h e aim o f e d u c a t i o n was t o " p u r g a t e t h e c h a r a c t e r by d i a l e c t i c " and t o p r o d u c e t h e w e l l - i n f o r m e d and e l o q u e n t A t h e n i a n c i t i z e n . F o r P l a t o and D e s c a r t e s t h e p u r p o s e o f e d u c a t i o n was t o e n l i g h t e n t h e i n d i v i d u a l and i n many c a s e s r e - d i s c o v e r t h e k n o w l e d g e t h e p e r s o n a l r e a d y p o s s e s s e d . To t h e " e s s e n t i a l i s t s " ( B a g l e y ,".1938 ; Demi a s h k e v i c h , 1935) and t h e " c o n s e r v a t i v e s " (Adams, 1915; B u r k e , 1 9 5 5 ) , t h e p u r p o s e o f e d u c a t i o n was t h e i n t e l l e c t u a l and m o r a l e n l i g h t e n m e n t a n d d i s c i p l i n e o f t h e i n d i v i d u a l . W i t h i n t h e s e b r o a d o b j e c t i v e s , t h e " e s s e n t i a l i s t s " e m p h a s i z e d t h e t r a n s f e r o f some p r e d e t e r m i n e d e s s e n t i a l k n o w l e d g e and s k i l l s , w h i l e t h e l a t t e r s c h o o l a d v o c a -t e d t h e t e a c h i n g o f u n i v e r s a l and u n c h a n g i n g s t a n d a r d s o f v a l u e . To " i d e i l l i s t s " l i k e I F r i e d r i c h H e g e l , F r i e d r i c h W i l h e l m yon S c h e l l i n g , Immanuel K a n t , J o h a n n G o t t l i e b F i c h t e ; t h e a c t o f k n o w i n g i s e s s e n t i a l l y - 3 -an a c t o f r e c o n s t r u c t i n g t h e d a t a o f a w a r e n e s s i n t o i n t e l l i g i b l e s y s t e m s o f i d e a s . ( W i n g o , 1 9 7 4 ) . S i n c e t h e u n i v e r s e i s r a t i o n a l and o r d e r l y a n d h e n c e i n t e l l i g i b l e , t h e o b j e c t i v e o f t h e i n d i v i d u a l s s h o u l d be t o c o m p r e h e n d t h e t r u t h . " P r o g r e s s i v i s t s " l i k e J o h n Dewey , J o s e p h Mayer R i c e , F r a n c i s P a r k e r , a n d o t h e r s h o w e v e r f e l t t h a t t h e p u r p o s e o f e d u c a t i o n was t o d e v e l o p t h e n a t i v e p o w e r s and t e n d e n c i e s o f t h e c h i l d and h e l p t h e s t u d e n t t o f i n d a n s w e r s by h i m s e l f . The a c t i v i t i e s t h a t go on i n s i d e t h e c l a s s v room and o u t s i d e must s t i m u l a t e t h o u g h t , r e s p e c t c o n t i n u i t y o f e x p e r i e n c e , and mesh p r e s e n t l e a r n i n g w i t h p a s t k n o w l e d g e and p r a c t i c e s o u t s i d e . I n t h e words o f Dewey ( 1 9 1 6 ) : " S i n c e g r o w t h i s t h e c h a r a c t e r i s t i c o f l i f e , e d u c a t i o n i s a l l one w i t h g r o w i n g ; i'c i t has no end b e y o n d i t s e l f . T h e c r i t e r -i o n o f t h e v a l u e o f s c h o o l e d u c a t i o n i s t h e e x t e n t i n w h i c h i t c r e a t e s a d e s i r e f o r c o n t i n u e d g r o w t h and s u p p l i e s means f o r m a k i n g t h e d e s i r e e f f e c t i v e i n f a c t . " ( p a g e 6 2 ) O t h e r e d u c a t i o n a l i s t s who h o l d " p r o g r e s s i v i s t i c " v i e w s c o n s i d e r e d u c a t i o n as a p r o c e s s o f d e v e l o p i n g -4-d e m o c r a t i c s k i l l s (Dewey 1910; H u l 1 f i s h and S m i t h , 1961; J u d d , 1 9 3 4 ) , and o f t e a c h i n g o f a j u r i s p r u d e n -t i a l f r a m e o f r e f e r e n c e f o r p r o c e s s i n g i n f o r m a t i o n ( O l i v e r and S h a v e r , 1 9 6 6 ) . W i t h s l i g h t v a r i a t i o n s , some o t h e r w r i t e r s e m p h a s i z e d t h e need f o r t e a c h i n g s o c i a l p r o b l e m s o l v i n g p r i m a r i l y t h r o u g h a c a d e m i c i n q u i r y and l o g i c a l r e a s o n i n g ( M a s s i a l a s and C o x , 1 9 6 6 ) , d e v e l o p m e n t o f i n t e r p e r s o n a l and g r o u p s k i l l s ( B r a d f o r d , 1961; S h e p a r d , 1964) and d e v e l o p m e n t o f s t u d e n t ' s i n d u c t i v e r e a s o n i n g , c o n c e p t b u i l d i n g and c a t e g o r i s i n g a b i l i t i e s ( A u s u b e l , 1967 ; B r u n e r , Goodnow and A u s t i n , 1967; Suchman, 19621 T a b a , 1 9 6 6 ) . V a r i o u s o t h e r m o d e l s o f e d u c a t i o n e m p h a s i z i n g t h e n e e d f o r i n d i v i d u a l i z e d and c r e a t i v e i n s t r u c t i o n h a v e b e e n s u g g e s t e d by many p r a c t i t i o n e r s ( G l a s s e r , 1 9 6 5 ; G o r d o n , 1970; H u n t , 1970; R o g e r s , 1 9 5 1 ; R o g e r s , 1 9 6 9 ) . A l m o s t i n f u l l c o n t r a s t t o t h i s , Gagne ( 1 9 6 5 ) , S k i n n e r ( 1 9 6 8 ) , S u l l i v a n ( 1 9 6 3 ) , T a b e r , G l a s e r a n d H a l m u t h ( 1 9 6 5 ) and o t h e r s h a v e a d v o c a t e d t h e u s e o f e x t e r n a l r e i n f o r c e m e n t s t o s h a p e t h e i n d i v i d u a l ' s b e h a v i o u r i n p r e - d e t e r m i n e d d i r e c t i o n s . I I MODELS OF TEACHING T h e d i f f e r e n c e s i n e m p h a s i s on t h e g o a l s o f e d u c a t i o n h a v e r e s u l t e d i n t h e e m e r g e n c e o f v a r i e d -5-m o d e l s o f t e a c h i n g w i t h i n t h e c l a s s room. M o d e l s s e r v e as i n t e r p r e t i v e f r a m e w o r k s w h i c h r e p r e s e n t r e a l i t y t o d i f f e r i n g d e g r e e s ( P o l a n y i , 1 9 5 8 ) . E d u c a t o r s , p s y c h o l o g i s t s , s o c i o l o g i s t s , and p h i l o s o -p h e r s , h a v e p r o d u c e d t h e o r e t i c a l m o d e l s a b o u t l e a r n i n g and t e a c h i n g and a b o u t t h e r o l e s o f t h e t e a c h e r and t h e s t u d e n t . T h e " c o n s e r v a t i v e s " a ssumed t h a t t h e u n e d u c a t e d man i s p r i m i t i v e , i m p u l s i v e , and i g n o r a n t and h e n c e t e a c h i n g was t h e p r o c e s s w h e r e b y t h e t e a c h e r (who was t h e e mbodiment o f a l l k n o w l e d g e ) t r a n s f e r r e d p a r t o f h i s k n o w l e d g e t o t h e i g n o r a n t s t u d e n t s . To t h e " p r o g r e s s i v i s t s " h o w e v e r , t h e t e a c h e r ' s r o l e was more a k i n t o t h a t o f a c a t a l y s t ; he was t o c r e a t e and m a i n t a i n an e n v i r o n m e n t w h e r e t h e e x p e r i e n c e o f a s t u d e n t c o n t i n u o u s l y d e v e l o p e d and n a t i v e p o w e r s were n u r t u r e d and f o r m e d i n t o f u n c t i o n a l h a b i t s . The e m p h a s i s h e r e was on t h e d i s c o v e r y and m a i n t e n a n c e o f c o n d i t i o n s t h a t s t i m u l a t e t h e s t u d e n t s t o t h i n k and f i n d a n s w e r s a l l by t h e m s e l v e s . F o r t h i s i t was i m p e r a t i v e t h a t t h e t e a c h e r knew and u n d e r s t o o d t h e p s y c h o l o g y o f t h o u g h t . ( W i n g o , 1974) The b e h a v i o u r c o n t r o l o r b e h a v i o u r m o d i f i c a t i o n - 6 -model i s t h e s t i m u l u s - r e s p o n s e v i e w o f t e a c h i n g . The e s s e n t i a l t a s k o f t e a c h i n g a c c o r d i n g t o t h i s a p p r o a c h , i s t o make s t u d e n t s p e r f o r m p r e - d e t e r m i n e d b e h a v i o u r a l r e s p o n s e s t o s p e c i f i c s t i m u l i ( T a b e r , G l a s e r and H a l m u t h , 1 9 6 5 ) . The " d i s c o v e r y l e a r n i n g 1 ' m o d e l " o f t e a c h i n g , on t h e o t h e r h a n d , v i e w s t h e s t u d e n t as t h e m a i n a c t o r i n t h e e d u c a t i o n a l p r o c e s s ( B r o w n , 1 9 7 1 ; S c h u t z , 1 9 6 7 ) . I t e m p h a s i s e s s e l f d i r e c t e d a c t i v i t y a nd d e v e l o p m e n t o f . c r e a t i v i t y i n t h e l e a r n e r . F i n a l l y , t h e r a t i o n a l model o f t e a c h i n g s t r e s s e s t h e a p p l i c a t i o n o f a n a l y t i c p h i l o s o p h y and e m p i r i c i s m t o e d u c a t i o n a l i s s u e s . I t s h o u l d be e m p h a s i z e d t h a t t h e s e m o d e l s u n l i k e t h o s e i n p h y s i c a l a n d e x a c t s c i e n c e s a r e n o t m u t u a l l y e x c l u s i v e . T h e y o n l y p o r t r a y i n a l t e r n a t e w a y s , t h e b a s i c a c t i v i t y o f t r a n s f e r o f k n o w l e d g e and s k i l l s t o t h e l e a r n e r . T h e r e i s l i t t l e , i f a n y , e m p i r i c a l d a t a s u p p o r t i n g one model o f t e a c h i n g a s s u p e r i o r t o o t h e r s o r e v e n a s a p p l i c a b l e i n a l l s i t u a t i o n s . T h e v a r i e d m o d e l s o f t e a c h i n g h a v e g i v e n r i s e t o d i f f e r e n t , o f t e n c o n f l i c t i n g v i e w s on what c o n s t i t u t e s e f f e c t i v e t e a c h i n g . D e p e n d i n g upon t h e p a r t i c u l a r o r i e n t a t i o n o f t h e e d u c a t i o n a l i s t s , and t h e i r a s s u m p -- 7 -t i o n s a b o u t t h e models o f t e a c h i n g , w i d e l y d i f f e r i n g c r i t e r i a have been o f f e r e d as i n d i c a t o r s o f good t e a c h i n g . D i v e r s e v i e w s a l s o e x i s t on what c o n s t i -t u t e s e f f e c t i v e l e a r n i n g e n v i r o n m e n t s , good t e a c h i n g methods and i n s t r u c t i o n a l s t y l e s . I t i s o u r c o n t e n t i o n t h a t no s i n g l e model c a n a d e q u a t e l y d e s c r i b e a l l t h e complex p r o c e s s e s i n v o l v e d i n t e a c h i n g . A p o s s i b l e s o l u t i o n . t o t h e p r o b l e m i s t h e i n t e g r a t i o n o f v a r i o u s models o f t e a c h i n g b a s e d on e m p i r i c a l r e s e a r c h . Such r e s e a r c h s h o u l d i d e n t i f y t h e key v a r i a b l e s i n v o l v e d i n t e a c h i n g and t h e d i m e n s i o n s o f e f f e c t i v e t e a c h e r b e h a v i o u r . T h i s would p r o v i d e a b a s i s on w h i c h t h e i n t e g r a t i o n o f v a r i o u s m o d e ls o f t e a c h i n g c o u l d be a t t e m p t e d . T h i s t h e s i s hopes t o t h r o w some l i g h t on what c o n s t i t u t e s e f f e c t i v e t e a c h i n g and t e a c h e r b e h a v i o u r . Such an i d e n t i f i c a t i o n o f t h e b e h a v i o u r a l d i m e n s i o n s o f e f f e c t i v e t e a c h i n g s h o u l d a l s o be u s e f u l i n t h e g e n e r a t i o n o f i n f o r m a t i o n r e l e v a n t f o r t h e p o s s i b l e i n t e g r a t i o n o f e x i s t i n g m o d e ls o f t e a c h i n g . CHAPTER II TOWARDS IDENTIFYING THE EFFECTIVE TEACHER I SOME VIEWS ON WHAT MAKES THE EFFECTIVE TEACHER A key p r o b l e m t h a t emerges as a d i r e c t c o n s e q u e -nce o f l a c k o f agreement on p u r p o s e s and methods o f e d u c a t i o n i s t h e a b s e n c e o f an u n i v e r s a l s e t o f c h a r a c t e r i s t i c s t h a t d e f i n e t h e e f f e c t i v e t e a c h e r . W h i l e many s e t s o f c r i t e r i a and b e h a v i o u r s have been o f f e r e d i n t h e p a s t as ' t h e ' c h a r a c t e r i s t i c s o f 'good' t e a c h e r s , t h e s e i d e a l t r a i t s and b e h a v i o u r s have been s i g n i f i c a n t l y i n f l u e n c e d by t h e i n d i v i d u a l ' s a s s u m p t i o n s a b o u t t h e g o a l s o f e d u c a t i o n , i n s t r u c t i o n a l m e thods, and t h e t e a c h e r - s t u d e n t r e l a t i o n s h i p . F o r exa m p l e , t o S o c r a t e s , t h e good t e a c h e r s h o u l d a c t as a good m i d w i f e , t h a t i s , a good t e a c h e r was one who c o u l d h e l p t h e p u p i l V t o b r i n g t o b i r t h what he had a l r e a d y c o n c e i v e d ; t o S o p h i s t s , t h e good t e a c h e r was a good o r g a n i z e r and m o r a l i s t ; t o A b e l a r d and t h e s c h o l a s t i c s , t h e good t e a c h e r was a good l o g i c i a n ; t o Dewey and o t h e r s he was a good c a t a l y s t and k n o w l e d g a b l e i n d i v i d u a l . ( O t h e r - v i e w s on t h e e f f e c t i v e t e a c h e r a r e g i v e n by A u s u b e l , 1967 ; B r a d f o r d , 1961; G o r d o n , -9-1970; Kaufman, 1967; M a s s i a l a s and Cox, 1966; R o g e r s , 1951; Schwab, 1965; S k i n n e r , 1968; and T a b a , 1966). The m a g n i t u d e o f t h e p r o b l e m i n v o l v e d i n i d e n t i -f y i n g good t e a c h i n g and good t e a c h e r s has prompted some c r i t i c s t o a r g u e a g a i n s t any s o r t o f r e s e a r c h on t e a c h i n g . T h u s , t o q u o t e H l g h e t ( 1 9 5 4 ) : " T e a c h i n g i n v o l v e s e m o t i o n s w h i c h c a n n o t be : v.: s y s t e m a t i c a l l y a p p r a i s e d and employed and human v a l u e s which a r e q u i t e o u t s i d e t h e g r a s p o f s c i e n c e ' s c i e n t i f i c ' t e a c h i n g e v e n o f s c i e n t i f i c s u b j e c t s w i l l be i n a d e -q u a t e as l o n g as b o t h t h e t e a c h e r s and p u p i l a r e human b e i n g s . T e a c h i n g i s n o t l i k e i n d u c i n g a c h e m i c a l r e a c t i o n . I t i s much more l i k e p a i n t i n g a p i c t u r e o r making a p i e c e o f m u s i c " (page 7-8) T h i s however, i s an e x t r e m e r e a c t i o n . W h i l e we a g r e e t h a t t h e p r o c e s s o f e s t a b l i s h i n g c o n s i s t e n t and s t r o n g r e l a t i o n s h i p s between t e a c h e r c h a r a c t e r i s t i c s and a c c o m p l i s h m e n t o f e d u c a t i o n a l o b j e c t i v e s ( w h a t e v e r t h e y may be) i s c h a r a c t e r i s e d by many d i f f i c u l t t h e o r e t i c a l and p r a c t i c a l p r o b l e m s , t h i s g o a l i s s t i l l -10-w o r t h y o f o u r e f f o r t s . A l s o , i n t h e l o n g r u n , some t a n g i b l e b e n e f i t s can be a t t a i n e d by s y s t e m a t i c r e s e a r c h on t e a c h i n g and t e a c h e r s . We t e n d t o a g r e e more w i t h Gage (1964) who s a i d : " . . . . a l t h o u g h ( t e a c h i n g ) r e q u i r e s a r t i s t r y i t can be s u b j e c t e d t o s c i e n t i f i c s c r u t i n y . The power t o e x p l a i n , p r e d i c t , and c o n t r o l t h a t may r e s u l t from such s c r u t i n y w i l l n o t d ehumanize t e a c h i n g . . . and f o r t h e work o f t h o s e who t r a i n , h i r e and s u p e r v i s e t e a c h e r s , t h e o r y and e m p i r i c a l knowledge o f t e a c h i n g w i l l p r o v i d e s c i e n t i f i c g r o u n d i n g . " (page 270-71) I t i s o u r v i e w t h a t a compromise between t h e s e two o p p o s i n g v i e w s can be a r r i v e d a t by f o c u s i n g o u r r e s e a r c h and h y p o t h e s e s on s p e c i f i c s i t u a t i o n s . T h u s , we f e e l t h a t any a t t e m p t t o i d e n t i f y t h e good t e a c h e r , i n terms o f d e f i n i t e a t t r i b u t e s he p o s s e s s e s o r c e r t a i n t e c h n i q u e s he e m p l o y s , w i l l be q u i t e d i f f i c u l t i f n o t f u t i l e . However, r e s e a r c h o r i e n t e d t o w a r d s f i n d i n g t h e r e l a t i o n s h i p between p a r t i c u l a r s e t ( s ) o f i n s t r u c t o r s and l e a r n e r s i n s p e c i f i c s i t u a t i o n s , w o r k i n g t o w a r d s p a r t i c u l a r g o a l s , w i l l be r e w a r d i n g and i l l u m i n a t i n g . T h i s s i t u a t i o n a l a p p r o a c h t o m e a s u r i n g what c o n s t i t u t e s e f f e c t i v e p e r f o r m a n c e has a l r e a d y been a p p l i e d i n t h e f i e l d s o f l e a d e r s h i p r e s e a r c h ( F i e d l e r , 1965} , p e r s o n a l i t y and b e h a v i o u r a l r e s e a r c h ( S t a a t s , 1 9 7 5 ) , and O r g a n i z a t i o n a l s t r u c t u r e r e s e a r c h ( L a w r e n c e and L o r s c h , 1 9 6 7 ) . I t i s f e l t t h a t t h e use o f such a c o n t i n g e n c y t h e o r y t o a t e a c h i n g s i t u a t i o n w i l l a l s o be u s e f u l . . H i TOWARDS IDENTIFYING A CRITERION FOR E F F E C T I V E  TEACHING A c r i t e r i o n i s a s t a n d a r d , measure o r t e s t w h i c h i s u sed f o r j u d g i n g t h e p e r f o r m a n c e o f an i n d i v i d u a l on a j o b . I t may be a p h y s i c a l , e c o n o m i c o r p s y c h o l o -g i c a l m e a s u r e , b u t i n a l l c a s e s i t has an u n d e r l y i n g d i m e n s i o n a l i t y . Even where t h e c r i t e r i o n measurement i s by a d i c h o t o m o u s s t a n d a r d ( e g . , 'gcfod' and ' b a d ' ) , t h e r e i s an u n d e r l y i n g d i m e n s i o n a l i t y f o r t h e w h o l e c o n c e p t . In t h e c a s e o f a s i m p l e manual o r f a c t o r y j o b , e s t a b l i s h m e n t and measurement o f c r i t e r i a f o r p e r f o r -mance i s q u i t e e a s i l y a c c o m p l i s h e d . The volume^ .of p r o d u c t i o n , t h e q u a l i t y o f t h e p r o d u c t o r s e r v i c e , t h e -12-c o s t o f p r o d u c t i o n , t h e e x t e n t o f w a s t a g e and r e j e c -t i o n , e t c . , c a n a l l s e r v e as v a l i d c r i t e r i a d e p e n d i n g on t h e t y p e o f t a s k . The measurement o f t h e s e c r i t e r i o n v a r i a b l e s can a l s o be done w i t h o u t much ado i n s u c h s i m p l e c a s e s . However, i n t h e c a s e o f a more complex j o b s u c h as t e a c h i n g , t h e s i t u a t i o n i s q u i t e d i f f e r e n t . P e r h a p s t h e b e s t c r i t e r i o n by w h i c h t o j u d g e t e a c h e r competence i s t h e l e a r n i n g and b e h a v i o u r a l change i n d u c e d i n t h e l e a r n e r . S i n c e t h e t e a c h e r i s an i n d i v i d u a l engaged i n i n t e r a c t i v e b e h a v i o u r w i t h one o r more s t u d e n t s f o r p u r p o s e o f e f f e c t i n g a change i n them ( M c N e i l and Popham, 1 9 7 3 ) , t h e r e i s p e r h a p s no b e t t e r measure o f h i s e f f e c t i v e n e s s t h a n t h e e f f e c t he had on t h e s t u d e n t s . T h i s was a l s o p o i n t e d o u t by Gage (1963) when he w r o t e : "By t e a c h e r e f f e c t i v e n e s s i s u s u a l l y meant t h e t e a c h e r ' s e f f e c t on t h e r e a l i z a t i o n o f some e d u c a t i o n a l o b j e c t i v e , d e f i n e d i n a i r terms o f d e s i r e d p u p i l b e h a v i o u r s , a t t r i -b u t e s , h a b i t s , o r c h a r a c t e r i s t i c s . Hence t h e u l t i m a t e c r i t e r i o n o f a t e a c h e r ' s e f f e c t i v e n e s s i s u s u a l l y c o n s i d e r e d t o be h i s e f f e c t on h i s pupil's a c h i e v e m e n t o f s u c h o b j e c t i v e s . " (page 116) -13-The'measurement o f t h e p u p i l ' s g a i n o r a c h i e v e -ment a r e however d i f f i c u l t b e c a u s e many c o n c e p t u a l and p r a c t i c a l d i f f i c u l t i e s b e s e t any s u c h measurement ( M i t z e l , 1960; Ryans, 1 9 6 0 ) . The l e a r n i n g and b e h a v i o u r m o d i f i c a t i o n t h a t o c c u r i n t h e s t u d e n t s as a r e s u l t o f t h e t e a c h i n g c a n n o t be s e p a r a t e d f r o m v a r i o u s o t h e r c a u s e s l i k e s t u d e n t p e r s o n a l i t y , e n v i r o n -m e n t a l and o t h e r s o c i o - e c o n o m i c and c u l t u r a l v a r i a b l e s . A l s o , many p r o b l e m s a r i s e f r o m l a c k o f a s t a n d a r d d e f i n i t i o n o f ' l e a r n i n g ' and t h e c h a n g e s i n t h e meaning o f t h e c o n s t r u c t d e p e n d i n g on t i m e and s i t u a t i o n a l d i m e n s i o n s . P r o b l e m s a r i s e i n t h e d e s i g n o f r e l i a b l e and v a l i d m e a s u r e s / t e s t s o f l e a r n i n g . R e a l i z a t i o n o f t h e s e d i f f i c u l t i e s by t h e r e s e a r c h e r s has l e d t o t h e use o f some s u b s t i t u t e c r i t e r i a i n t h e measurement o f t e a c h e r e f f e c t i v e n e s s i n t h e p l a c e o f ' u l t i m a t e c r i t e r i o n '. I l l A BRIEF LOOK AT SOME SUBSTITUTE CRITERIA A v a r i e t y o f models have been d e s i g n e d i n t h e p a s t f o r t h e p u r p o s e o f e v a l u a t i n g t h e t e a c h e r compe-t e n c e t h r o u g h t h e use o f ' s u b s t i t u t e ' c r i t e r i a . F o r example, Domas and Tiedeman (1950) u s e d j u d g e m e n t s o f A d m i n i s t r a t o r s , p e e r s and s t u d e n t s and o b j e c t i v e t e s t s -14-o f p e r f o r m a n c e by s t u d e n t s as t h e c r i t e r i a o f e f f e c t i v e t e a c h e r p e r f o r m a n c e . ( F o r a b r i e f schemat o f t h i s model , see A p p e n d i x A ) . M i t z e l (1957) on t h e o t h e r hand r e l i e d h e a v i l y o n . p u p i l - g r o w t h measures such as changes i n s o c i a l i n t e r a c t i o n s k i 1 1 s , a t t i t u d e s , knowledge o f s p e c i f i c t h e o r i e s o f s t u d e n t s ( A p p e n d i x B ) . S p r i n t h a l 1 , W h i t e ! ey. and Mosher ( 1966) c o n s i d e r e d c o g n i t i v e f l e x i b i l i t y o f t h e t e a c h e r as en i n d i c a t o r o f h i s e f f e c t i v e n e s s as a t e a c h e r , w h i l e C l a r k ( 1 9 5 0 ) , R yans, (1960) and o t h e r s i d e n t i f i e d v e r y many p e r s o n a -l i t y c o r r e l a t e s ( s u c h as o b j e c t i v i t y , a g r e e a b l e n e s s , d e p r e s s i o n , c y c l o i d d i s p o s i t i o n , e m o t i o n a l s t a b i l i t y , f r i e n d l i n e s s , e t c ) o f e f f e c t i v e t e a c h i n g . O t h e r models o f t e a c h i n g have been g i v e n by S m i t h ( 1 9 6 0 ) , S e a r s ( 1 9 5 1 ) , S t o l u r o w (1961) and Runke! ( 1 9 5 8 ) . The most p o p u l a r o f t h e ' s u b s t i t u t e ' c r i t e r i a used f o r e v a l u a t i n g t h e t e a c h e r s so f a r have been t e a c h e r r a t i n g s , measures o f t e a c h e r ' s b e h a v i o u r w i t h i n t h e c l a s s room and t h e c h a r a c t e r i s t i c s o f c l a s s room e n v i r o n m e n t s c r e a t e d by t h e t e a c h e r . L e a r n i n g g a i n o f s t u d e n t s measured by p e r f o r m a n c e t e s t s and b e h a v i o u r a l o b s e r v a t i o n s has a l s o been a w i d e l y used c r i t e r i o n o f t e a c h e r e f f e c t i v e n e s s . -15-F i n a l l y , c e r t a i n o b j e c t i v e c r i t e r i a l i k e t h e number o f p u b l i c a t i o n s o f t h e t e a c h e r and t h e number o f c o n s u l t a n c y a s s i g n m e n t s c a r r i e d o u t by him have a l s o been t h o u g h t o f as some i n d i c a t o r s o f h i s a b i l i t y . , We i n t e n d t o d i s c u s s below t h e r e l a t i v e u s e f u l n e s s and d r a w b acks o f t h e c u r r e n t l y p o p u l a r m e a s u r e s o f t e a c h i n g e f f e c t i v e n e s s . R a t i n g s : R a t i n g s o f t e a c h e r p e r f o r m a n c e may be done 1) by t h e t e a c h e r h i m s e l f , 2) by h i s p e e r s and s u p e r i o r s and 3) by h i s s t u d e n t s . S e l f r a t i n g s have been i n s t a l l e d on t h e h y p o t h e s e s t h a t t h e o r e t i c a l l y e v e r y p a r s o n wants t o e v a l u a t e h i m s e l f i n r o r d e r t o o b t a i n an a c c u r a t e p i c t u r e o f h i s a b i l i t i e s ( F e s t i n g e r , 1954). Many s t u d i e s i n t h e p a s t have a s s e s s e d t h e r e l a t i v e b e n e f i t s and drawbacks o f t h i s method. (Saloman and M c D o n a l d , 1970; W e i n e r and K u k l a , 1970). A m a j o r drawback a r i s e s f r o m t h e f a c t t h a t few p e o p l e a r e t r a i n e d i n o b j e c t i v e l y j u d g i n g t h e i r own p e r f o r m a n c e and hence may t e n d t o o v e r r a t e t h e m s e l v e s o r use i n c o r r e c t c r i t e r i a . A l s o , t e a c h e r s a r e not l i k e l y to u n d ergo a t t i t u d i n a l o r b e h a v i o u r a l c h a n g e s u n l e s s t h e r e v i e w p r o v i d e s i n f o r m a t i o n a b o u t t h e e x t e n t o f d e p a r t u r e from a d e s i r e d s t a n d a r d w h i c h has been a c c e p t e d as t h e s t a n d a r d by t h e v i e w e r . -16-Many a t i m e , s u p e r f i c i a l c r i t e r i a s u c h as p e r s o n a l m a n n e r i s m s , a p p e a r a n c e and v o i c e t e n d t o p r e d o m i n a t e t h e s e e v a l u a t i o n s . ( M c N e i l and Popham, 1973). R a t i n g s by p e e r s , s u p e r v i s o r s and p r i n c i p a l s have a l s o been used as measures o f t e a c h i n g e f f e c t i v e -n e s s . S i n c e t h e s e . i n d i v i d u a l s a r e l i k e l y t o know most a b o u t t h e t e a c h e r ' s t e a c h i n g t e c h n i q u e s , and h i s s c h o l a s t i c and o r g a n i z a t i o n a l s k i l l s , t h e i r r a t i n g s have been c o n s i d e r e d " v a l i d measures o f h i s c o m p e t e n c e and e f f e c t i v e n e s s . The use o f s t u d e n t r a t i n g s o f i n s t r u c t o r s i s g r o w i n g i n p o p u l a r i t y p a r t i c u l a r l y a t t h e c o l l e g e l e v e l . C o l l e g e s and u n i v e r s i t i e s a r e i n c r e a s i n g l y i n t r o d u c i n g p o l i c i e s r e q u i r i n g s t u d e n t e v a l u a t i o n o f p r o f e s s o r s f o r t h e p u r p o s e o f c o n t r a c t r e n e w a l , t e n u r e and p r o m o t i o n . In t h e words o f Fox; and B r o o k s h i r e ( 1 9 7 1 ) : " . . . C o l l e g e s t u d e n t s a r e a s k i n g f o r an o p p o r t u n i t y t o e v a l u a t e t h e i r p r o f e s s o r s . Many f a c u l t y members a r e becoming unhappy w i t h a l o c k - s t e p s a l a r y s c a l e , a u t o m a t i c p r o m o t i o n s and t e n u r e . I n s t e a d , t h e f a c u l t y members d e s i r e a c h a n c e to p r o v e -17-t h e m s e l v e s t h r o u g h some system;:of e v a l u a -t i o n w i t h t h e hope t h a t an o u t s t a n d i n g j o b on t h e i r p a r t w i l l be r e w a r d e d . . . " ( p 37-40) S e v e r a l s t u d i e s have emerged r e c e n t l y s h o w i n g t h e a p p l i c a t i o n o f s t u d e n t e v a l u a t i o n s i n d e f i n i n g e f f e c t i v e c o l l e g e t e a c h i n g ( C o s t i n * Greenough and .. Menges, 1971; C o a t s and S w i e r e n g a , 1972; Fox and B r o o k s h i r e , 1971; Tuckman and O l i v e r , 1 9 6 8 ) . The most common method employed i n s t u d e n t r a t i n g s i s t h e o p i n i o n a i r e ( F i t c h , 1965; O v e r t u r f , 1966). Most p r o c e d u r e s ask t h e s t u d e n t t o r a t e an i n s t r u c t o r on v a r i o u s a t t r i b u t e s . Open ended q u e s t i o n s , r a n k i n g and c o m p a r i s o n s y s t e m s and c h e c k l i s t s a r e u s e d on o t h e r o c c a s i o n s . S t u d e n t r a t i n g s o f i n s t r u c t o r s on p e r s o n a l i t y a t t r i b u t e s and s u b j e c t i v e c r i t e r i a have t h e p r o b l e m o f h a l o , l e n i e n c y and c e n t r a l t e n d e n c y e f f e c t s . A l s o , v e r y o f t e n , t h e s t u d e n t r a t i n g s a r e t a i n t e d by t h e s t u d e n t ' s own p e r s o n a l i t y c h a r a c t e r i s t i c s , a t t i t u d e s and a b i l i t i e s . Thus Morsh and W i l d e r (1954) f o u n d t h a t t h e i n s t r u c t o r whose t e a c h i n g i s more o r i e n t e d t o w a r d s t h e b r i g h t s t u d e n t s w i l l be a p p r o v e d by them and hence t h e r e w i l l be a p o s i t i v e c o r r e l a t i o n between r a t i n g s and g r a d e s . However, i f t h e t e a c h e r o r i e n t s -18-himself to the weaker students, the superior students w i l l not approve of him and hence the c o r r e l a t i o n between ratings and grades w i l l be negative. The student, ratings, as expressions of feelings and teacher behaviour, can, however be extremely useful. They serve as a tool in teacher assessment, i n s t r u c t i o n a l improvement, teacher selection and t r a i n i n g . The argument that student ratings are just a product of popularity, charisma, halo e f f e c t , or the student's sex, age or grades, has been challenged in many other studies (Costin, Greenough and'Menges, 1971; Kohlan, 1973). It is also pointed out that r e l i a b i l i t y of student ratings can increase by having students focus on observable behaviours of the teacher (McNeil and Popham, 1973). Personal A t t r i b u t e s : A variety of personality factors l i k e i n t e l l i g e n c e , emotional s t a b i l i t y , health, humour, poise, f r i e n d l i n e s s , v i t a l i t y , extroversion and accept-ance of c r i t i c i s m have been used in the past as indicators of teaching effectiveness. A few recent studies have given evidence of the increasing use of these c r i t e r i a (Durflinger, 1963; Gough, Durflinger and H i l l , 1968; Gough and Pemberton, 1952; Moore and Cole, 1957; Peck, I960; Ryans, 1960; Veldman and peck, -19-1963). The use o f p e r s o n a l a t t r i b u t e s and p e r s o n a l i t y c r i t e r i a i n a s s e s s i n g t e a c h i n g e f f e c t i v e n e s s s u f f e r f r o m : -1) P r o b l e m o f d e f i n i t i o n : - T h e r e e x i s t p r o f o u n d d i f f e r e n c e s as t o what i s meant by v a r i o u s p e r s o n a l i t y a t t r i b u t e s . The b e h a v i o u r a l d e f i n i t i o n s v a r y q u i t e l a r g e l y f r o m d e p t h d e f i n i t i o n s w h i c h a g a i n a r e q u i t e d i f f e r e n t f r o m s o c i a l s t i m u l u s d e f i n i t i o n s ( G e t z e l s and J a c k s o n , 1963). The p r o b l e m s a r i s e when d a t a o b t a i n e d i n terms o f one d e f i n i t i o n a r e n o t c l e a r l y d i f f e r e n t i a t e d f r o m d a t a o b t a i n e d i n terms o f a n o t h e r ( G e t z e l s , 1953). 2) P r o b l e m s o f c h o i c e o f i n s t r u m e n t t o measure v a r i o u s p e r s o n a l i t y a t t r i b u t e s : - D a t a f r o m s e l f r e p o r t " m e a s u r e s , r a t i n g s c a l e s , p r o j e c t i v e t e c h n i q u e s and t e s t s v a r y q u i t e l a r g e l y f r o m each o t h e r . 3) P r o b l e m o f c r i t e r i o n : - T h e r e i s as y e t no r e a s o n t o assume t h a t a_ s e t o f a t t r i b u t e s i s c h a r a c -t e r i s t i c o f t h e ' e f f e c t i v e t e a c h e r 1 . Even c o n s i s t e n t c o r r e l a t i o n does n o t e n s u r e a c a u s a l r e l a t i o n s h i p . Many o f t h e so c a l l e d ' t r a i t s ' may e x i s t l a r g e l y i n t h e eyes o f t h e b e h o l d e r . As M c N e i l and Popham (197:3) warned, t h e r a t i n g o f t e a c h e r s on a s e t o f c r i t e r i a may t e l l us more a b o u t t h e a s s e s s o r t h a n a b o u t t h e t e a c h e r . T h e r e i s a l s o no g r e a t r e l i a b i l i t y f o r most such r a t i n g s . D i r e c t o b s e r v a t i o n s : - W h i l e d i r e c t o b s e r v a t i o n i n t h e c l a s s room i s one o f t h e most o b v i o u s and i m p o r t a n t methods o f e d u c a t i o n a l r e s e a r c h , i t s use i n e v a l u a t i n g t e a c h e r e f f e c t i v e n e s s has so f a r been i n d i r e c t and m i n i m a l . T h i s i s m a i n l y b e c a u s e t e a c h e r e f f e c t i v e n e s s must u l t i m a t e l y be d e f i n e d i n terms o f e f f e c t s o f t e a c h e r ' s s t r a t e g i e s on t h e b e h a v i o u r o f t h e s t u d e n t s i n s i d e and o u t s i d e t h e c l a s s room w h i c h a r e n o t a l w a y s a p p a r e n t t o a t y p i c a l o b s e r v e r . As M o r s c h and W i l d e r (1954) c o n c l u d e d , a t t e m p t s to v a l i d a t e p r o c e s s c r i t e r i a by c o r r e l a t i n g them w i t h measured p u p i l g r o w t h have been on t h e whole somewhat u n s u c e s s f u l i n t h e p a s t . As s u c h e f f e c t i v e t e a c h i n g c a n n o t be p r o v e n by t h e p r e s e n c e o r a b s e n c e o f any s i t u a t i o n a l v a r i a b l e s . The i d e n t i f i c a t i o n o f p a t t e r n s o f b e h a v i o u r w h i c h d i f f e r e n t i a t e e f f e c t i v e t e a c h e r s f r o m t h e i n e f f e c t i v e ones i s s t i l l a w o r t h w h i l e g o a l and k i n e s c o p i c t e c h n i -ques o f c l a s s room o b s e r v a t i o n have d i s t i n c t a d v a n t a g e s -21-o f u n o b t f c u s i v e n e s s and f l e x i b i l i t y i n u s e . However, h a l o i n d a t a c o l l e c t i o n , a n a l y s i s , and i n t e r p r e t a t i o n c a u s e m a j o r p r o b l e m s . When u n o b t r u s i v e t e c h n i q u e s a r e n o t u s e d , t h e v a l i d i t y o f most d a t a i s q u e s t i o n a b l e b e c a u s e o f t h e e f f e c t s o f an o u t s i d e o b s e r v e r on t h e b e h a v i o u r o f t h e t e a c h e r and h i s s t u d e n t s . O b s e r v a -t i o n a l d a t a a r e a l s o p e r h a p s t h e most e x p e n s i v e t o o b t a i n s i n c e t h i s t e c h n i q u e ^ n e c e s s i t a t e s o b s e r v a t i o n by s k i l l e d r e s e a r c h e r s o v e r a r e l a t i v e l y l o n g p e r i o d o f t i m e . P r o b l e m s a s s o c i a t e d w i t h sample s e l e c t i o n may a l s o be v e r y c r u c i a l h e r e . U n l e s s t h e randomness and a p p r o p r i a t e n e s s o f t h e sample used a r e e n s u r e d , t h e whole e x e r c i s e may l a c k e x t e r n a l v a l i d i t y . A s s e s s i n g c l a s s room e n v i r o n m e n t s : - In t h e s e a r c h f o r c o r r e l a t e s o f e f f e c t i v e t e a c h i n g , a s p e c t s o f c l a s s room e n v i r o n m e n t s t h e m s e l v e s a r e o f t e n used as p r e d i c t o r s by many i n v e s t i g a t o r s ( A n t h o n y , 1968;: S o b o l ; 1971; W a l b e r g and A n d e r s o n , 1 9 6 8 ) , The assump-t i o n h e r e i s t h a t e n v i r o n m e n t i s l i n k e d t o p u p i l a c h e i v e -ment s i n c e p u p i l i n t e r a c t w i t h e l e m e n t s o t h e r t h a n t h e t e a c h e r i n t h e c l a s s room, ^yet t h e t e a c h e r may mani p u -l a t e many o f t h e s e e l e m e n t s i n s u b t l e ways. E x t e n d i n g t h i s c o n c e p t , e n v i r o n m e n t a l c h a r a c t e r i s t i c s ( o r t e a c h i n g c l i m a t e ) o f t h e e n t i r e i n s t i t u t i o n has been -22-a s c e r t a i n e d by A s t i n and H o l l a n d ( 1 9 6 1 ) , Pace ( 1 9 6 3 ) , S t e r n (1963) and o t h e r s . Some o f t h e . e n v i r o n m e n t a l v a r i a b l es /icommonly used i n t e a c h i n g e v a l u a t i o n s t u d i e s have been l e a d e r s h i p s t y l e o f t h e t e a c h e r , h i s i n t e r - p e r s o n a l o r i e n t a t i o n s , i n t e l 1 e c t u a l i s m , c l a r i t y and s t y l e o f r o l e r e l a t i o n s h i p s , s t a t u s , r e i n f o r c e m e n t and r e w a r d s t r u c t u r e . The m a j o r p r o b l e m i n a n a l y s e s such as t h e s e i s to remove t h e e f f e c t s o f s t u d e n t s and p a r t i c u l a r outcomes on t h e e n v i r o n m e n t s . C o n s i -d e r i n g t h e e n v i r o n m e n t s as t h e s o l e ends i n t h e m s e l v e s r a t h e r t h a n as i n t e r v e n i n g v a r i a b l e s c a n a l s o l e a d t o t h e d i s c o u n t i n g o f t h e i m p o r t a n c e o f l e a r n i n g , t e a c h e r a t t i t u d e s , knowledge and c h a r a c t e r i s t i c s . P e r f o r m a n c e t e s t s and measures o f l e a r n i n g g a i n : -P e r f o r m a n c e t e s t s and t e s t s o f l e a r n i n g g a i n have been u s e d i n some i n s t a n c e s t o measure t h e o v e r a l l e f f e c t i v e n e s s o f t h e t e a c h e r ( J u s t i z , 1969; Popham, 1967; Taneman, 1970).. Here a number o f t e a c h e r s a r e g i v e n one o r more i d e n t i c a l t e a c h i n g o b j e c t i v e s and p e r f o r m a n c e e v a l u a t i o n measures and t h e i r r e l a t i v e e f f e c t i v e n e s s r a n k e d a c c o r d i n g to t h e i r p u p i l ' s p e r f o r m a n c e i n t h e g i v e n t e s t s . The o b j e c t i v e s t h e m s e l v e s may be t o i m p r o v e t h e c o g n i t i v e , a f f e c t i v e -23-or p s y c h o - m o t o r a b i l i t i e s o f t h e s t u d e n t s . W h i l e t h e r e s o u r c e m a t e r i a l s may be i d e n t i c a l , t h e methods o f i n s t r u c t i o n t o be used w i l l be l e f t e n t i r e l y t o t h e d i s c r e t i o n o f t h e t e a c h e r . The mean p o s t - t e s t s c o r e s becomes t h e c r i t e r i o n o f e f f e c t i v e n e s s . The r e l i a b i l i t y o f t h e s e measures can be i n c r e a s e d by u s i n g a number o f l e s s o n s and d i f f e r e n t k i n d s o f t e a c h i n g o b j e c t i v e s . C o n t r o l l i n g o f t h e c h a r a c t e r i s -t i c s o f t h e p u p i l s , w h i l e e x t r e m e l y d i f f i c u l t , c a n make t h e whole p r o c e s s more v a l i d . The u t i l i t y o f s u c h p e r f o r m a n c e t e s t s and measures o f l e a r n i n g g a i n becomes h i g h when t h e r e i s good c o n t r o l o v e r a l l . r e l e v a n t e x t e r n a l f a c t o r s . IV SELECTION OF THE CRITERION FOR TEACHING EFFECTIVENESS I t w i l l be r e l e v a n t h e r e to d i s c u s s some key c r i t e r i a f o r t h e s e l e c t i o n o f an a p p r o p r i a t e c r i t e r i o n f o r t e a c h i n g e f f e c t i v e n e s s . F o l l o w i n g G h i s e l l i ( 1 9 5 6 ) , G u i o n ( 1 9 6 5 ) , M c N e i l and Popham ( 1 9 7 3 ) , N a g l e (1953) and T h o r n d i k e ( 1 9 4 9 ) , we c o n s i d e r t h e f o l l o w i n g as some key prop a ^ t i e s o f a c r i t e r i o n f o r t e a c h i n g e f f e c t i v e n e s s . 1) R e l e v a n c e : - A c r i t e r i o n s h o u l d be a r e l e v a n t measure o f t h e c o n s t r u c t o f t e a c h i n g e f f e c t i v e n e s s - 2 4 -w h i c h we want t o e v a l u a t e . T h e r e i s no p o i n t i n d e v e l o p i n g a t r i v i a l measure o f t h e c o n s t r u c t ; n o r i s t h e r e any use o f a measure w h i c h i s i n d i r e c t l y and a m b i g u o u s l y r e l a t e d t o t h e main c o n s t r u c t . A l s o , t h e c r i t e r i a s h o u l d be c o m p l e t e i n t h e s e n s e t h a t t h e y s h o u l d c o v e r a l l t h e m a j o r d i m e n s i o n s o f t h e p e r f o r -mance ( N a g l e , 1953). The r e l e v a n c e d e t e r m i n e s t h e m easure's c o n s t r u c t v a l i d i t y and hence i s an i m p o r t a n t c h a r a c t e r i s t i c . 2) R e l i a b i l i t y : - The c o n s i s t e n c y i n measurement s h o u l d be h i g h . The measurement s h o u l d be a f f e c t e d by as ; few i n t e r n a l and e x t e r n a l f a c t o r s as p o s s i b l e , T h u s , t h e s o c i o - e c o n o m i c and p s y c h o l o g i c a l v a r i a b l e s , t h e t i m e f a c t o r , e t c . , s h o u l d n o t i n f l u e n c e t h e e f f e c t i v e -n e ss s c o r e s t o any g r e a t d e g r e e . The d e f i n i t i o n s s h o u l d be p r e c i s e , t h e manner o f d a t a c o l l e c t i o n s t a n d a r d i z e d , and t h e p e r f o r m a n c e ( e f f e c t i v e n e s s ) measure s h o u l d not be t a k e n o v e r a v e r y s h o r t time;;, s p a n . 3) D i s c r i m i n a t i o n : - The c r i t e r i a s h o u l d be a b l e t o d i s c r i m i n a t e between e f f e c t i v e t e a c h e r s and i n e f f e c t -i v e t e a c h e r s . Even where t h e g r o u p i s homogenous and t h e o v e r a l l e f f e c t i v e n e s s i s n o t h i g h some method - 2 5 -t o d i s c r i m i n a t e and r a n k t h e t e a c h e r s s h o u l d be e v o l v e d . A l s o , i t s h o u l d be a b l e t o i d e n t i f y t h e c h a n g e s i n a t e a c h e r ' s p e r f o r m a n c e o v e r t i m e . 4) P r a c t i c a l i t y : - [The e a s e o f measurement o f p e r s o n s on t h e c r i t e r i o n ( j u d g e d i n terms o f c o s t and man h o u r s ) and t h e ea s e o f c o n t r o l o f e x t r a n e o u s v a r i a b l e s , go to d e f i n e t h e e x t e n t o f p r a c t i c a l i t y o f a m e a s u r e . 5) F I e x i b i 1 i t y : - The c r i t e r i o n s h o u l d be a good measure o f teach.i n g : e f f e c t i v e n e s s even when t h e . t e a c h e r s t h e m s e l v e s have d i f f e r e n t g o a l p e r c e p t i o n s . The g o a l d i f f e r e n c e s and t e a c h i n g method d i f f e r e n c e s s h o u l d n o t r e s t r i c t t h e use o f t h e c r i t e r i o n m e a s u r e . 6) F e e d b a c k : - The s c o r e s on t h e c r i t e r i o n m e asure s h o u l d p r o v i d e some c o n c r e t e f e e d b a c k t o t h e t e a c h e r i n t h e a r e a s o f a) h i s s t r e n g t h s and we a k n e s s e s i n t e a c h i n g and b) r e q u i r e d changes i n h i s b e h a v i o u r / a t t i t u d e s t o i m p r o v e h i s e f f e c t i v e n e s s . T h i s n e c e s s i -t a t e s t h e c r i t e r i o n s c o r e s t o have i ) e a s y i n t e r p r e -t a b i l i t y i i ) c o n t r o l l a b i l i t y and i i i ) s p e c i f i c i t y . I t w i l l be a u s e f u l e x e r c i s e t o compare t h e (vari o u s s u b s t i t u t e c r i t e r i a o f t e a c h i n g e f f e c t i v e n e s s -26-on t he above r e q u i r e m e n t s as a f i r s t s t e p i n ou r p r e s e n t s t u d y . Two c a u t i o n s a re n e c e s s a r y b e f o r e we beg i n such an a n a l y s i s . F i r s t , the e v a l u a t i o n o f t h e v a r i o u s c r i t e r i a on t he above d i m e n s i o n s has been done here on a s u b j e c t i v e b a s i s . Wh i l e a r e v i e w o f p a s t r e s e a r c h on t e a c h i n g e f f e c t i v e n e s s has been comp le ted as a p r e l u d e to the . ana l y s i s c a r r i e d o u t h e r e . r t he c o n c l u s i o n s t hemse l ve s a re a r r i v e d a t by t he w r i t e r . S e c o n d l y , t he a n a l y s i s i s not c o m p l e t e . There a re some c r i t e r i a ( o f t e a c h i n g e f f e c t i v e n e s s ) f o r wh ich adequate e m p i r i c a l da ta a r e not a v a i l a b l e even to form a s u b j e c t i v e o p i n i o n . The r e l e v a n c e o f s e l f r a t i n g s , p e r s o n a l a t t r i -bu te s and c l a s s room e n v i r o n m e n t s t o the u l t i m a t e e f f e c t i v e n e s s o f t e a c h e r i s a t b e s t d o u b t f u l a t t h i s s t a g e o f r e s e a r c h on t e a c h i n g . Q u i t e o f t e n t h e measurements on t he se c r i t e r i a may be i n d i c a t i v e o f some o t h e r s o c i o - p s y c h o l o g i c a l or e n v i r o n m e n t a l v a r i a b l e s . R a t i n g s done by peer s and s t u d e n t s howeve r , a re r e l e v a n t as i n d i c a t e d by pa s t w r i t i n g s . D i r e c t o b s e r v a t i o n s and pe r f o rmance t e s t s a l s o a r e r e l e v a n t measures o f t e a c h i n g e f f e c t i v e n e s s s i n c e t hey t end to measure v a r i a b l e s wh ich a re s i g n i f i c a n t l y and c o n s i s -t e n t l y r e l a t e d to t he t e a c h e r ' s b e h a v i o u r . - 2 7 -.The r e l i a b i l i t y o f s e l f r a t i n g s , a t t r i b u t e r a t i n g s , e n v i r o n m e n t a l a s ses sments and c l a s s . r o o m o b s e r v a t i o n s a re some what q u e s t i o n a b l e . R a t i n g s seem to w i d e l y d i f f e r between d i f f e r e n t r a t e r s and f o r t he same r a t e r o ve r t i m e . The h a l o , c e n t r a l t e n -d e n c y '^''V' and l e n i e n c y e r r o r s p r e s e n t i n t he se r a t i n g s a r e of c o n s i d e r a b l e magn i t ude . The r e l i a b i l i t y o f pee r and s t u d e n t r a t i n g s a r e a l s o , on many o c c a s i o n s , l ow. Many s t u d i e s r e p o r t t h a t r a t i n g s done by s t u -d e n t s " - 1 and peer s on g r a p h i c s c a l e s a r e some what u n s t a b l e even ove r s h o r t p e r i o d s o f t i m e . ( P a i n t e r and G r a n z i n , 1972; Que re sh i and W i d l e k , 1973 ) . The pe r f o rmance o f s t u d e n t s i n t e s t s o f knowledge a l s o seem to be c o n s i d e r a b l y i n f l u e n c e d by v e r y many p e r s o n a l i t y and e n v i r o n m e n t a l v a r i a b l e s thus r e d u c i n g t h e i r r e l i a b i l i t y c o n s i d e r a b l y . B a r r i n g p e r s o n a l a t t r i b u t e s and o b s e r v a t i o n s o f c l a s s room e n v i r o n m e n t s , the r e m a i n i n g c r i t e r i a seem to be c a p a b l e o f d i s c r i m i n a t i n g between p e r f o r m a n c e o f any two t e a c h e r s . However, s e l f r a t i n g s do no t seem to change c o n s i d e r a b l y o ve r t ime thus l o s i n g t h e i r d i s c r i m i n a t i o n power. V/e.feel t h a t the c h a n g e s : in , - the r e l a t i v e >e:ff e c t i v e n e s s . o f - t he t e a c h e r o ve r t ime -28-i s n o t f u l l y e x p l a i n a b l e by s u c h measures as p e r s o -n a l a t t r i b u t e s , o b s e r v a t i o n s o f c l a s s room b e h a v i o u r , e n v i r o n m e n t a l a s s e s s m e n t s and p e r f o r m a n c e t e s t s . The c o s t and t i m e i n v o l v e d i n o b t a i n i n g r a t i n g s o f t e a c h e r s by s e l f , p e e r and s t u d e n t s a r e q u i t e low. D i r e c t o b s e r v a t i o n s a r e c o s t l y ; and c o s t s a s s o c i a t e d w i t h a t t r i b u t e m e a s u r e m e n t s , e n v i r o n m e n t a l a s s e s s m e n t and p e r f o r m a n c e t e s t s v a r y i n r d i r e c t p r o p o r t i o n t o t h e d e g r e e o f s o p h i s t i c a t i o n o f t h e methods o f o b s e r v a -t i o n and a n a l y s i s e m p l o y e d . The a b i l i t y o f t h e r e s e a r c h e r t o c o n t r o l i n f l u e n c e s Of e x t e r n a l v a r i a b l e s on t h e s c o r e s o f s t u d e n t s i n p e r f o r m a n c e t e s t s and t h e c h a n g es i n c l a s s room e n v i r o n m e n t s a l s o seem t o be c o n s i d e r a b l y l o w e r t h a n i n o t h e r c a s e s (Ashmus and H a i g h , 1952; G u t h r i e , 1971; W a t e r h o u s e and C h i l d , 1953). G e n e r a l l y s p e a k i n g , r a t i n g s a r e much more f l e x i b l e t h a n o b s e r v a t i o n a l c r i t e r i a w h i c h a g a i n a r e s u p e r i o r t o p e r f o r m a n c e t e s t s on t h i s d i m e n s i o n . T h i s i s b e c a u s e t h e g o a l d i f f e r e n c e s and t e a c h i n g method d i f f e r e n c e s c o n s i d e r a b l y r e s t r i c t t h e use o f t h e c r i t e -r i o n measure i n t h e c a s e o f p e r f o r m a n c e t e s t s . A t t h i s s t a g e o f r e s e a r c h , i t i s not p o s s i b l e t o come t o any c o n c l u s i o n r e g a r d i n g t h e e x t e n t o f f l e x i b i l i t y - 2 9 -o f the c r i t e r i o n o f p e r s o n a l a t t r i b u t e s . In most c a s e s , t he r a t i n g s p r o v i d e m e a n i n g f u l f eedback to the a s s e s s ee by t e l l i n g him about s p e c i f i c , r o b s e r v a b l e and c o n t r o l l a b l e f a c t o r s . As the r a t i n g s f o c u s more and more on o b j e c t i v e , o b s e r v a b l e f a c t o r s , t he s p e c i f i c i t y o f f eedback a l s o i n c r e a s e s c o n s i d e r a ' - : , b l y . A t t r i b u t e measures and pe r fo rmance t e s t s a r e d i f f i c u l t t o i n t e r p r e t and c o n t r o l . D i r e c t o b s e r v a -t i o n s hav e the advantage o f i n t e r p r e t a b i 1 i t y and s p e c i f i c i t y , but q u i t e o f t e n the t e a c h e r h i m s e l f may have no s i g n i f i c a n t c o n t r o l o v e r the s p e c i f i c b e h a v i o u r s and even t s wh ich happen i n h i s c l a s s room. D e s c r i p t i o n s o f h i s c l a s s room env i r onment does no t p r o v i d e the t e a c h e r w i t h any s p e c i f i c f eedback nor a re they under h i s c o n t r o l i n most c a s e s . A t a b l e showing t he e x t e n t t o wh ich v a r i o u s c r i t e r i a f o r t e a c h i n g e f f e c t i v e n e s s s a t i s f y the r e q u i r e m e n t s d i s c u s s e d above i s g i v e n on t he f o l l o w i n g page. An e x a m i n a t i o n o f the t a b l e would i n d i c a t e t h a t s t u d e n t and peer r a t i n g s have major advan tage s o ve r o t h e r methods on most d i m e n s i o n s . TABLE SHOWING THE EXTENT TO WHICH MEASURES OP TEACHING EFFFCTIVENESS SATISFY THE REQUIREMENTS FOR A GOOD CRITERION A t t r i b u t e s Mea s u r e s t , o f t e a c h i n g " ^ e f f e c t i v e n e s s ^ ^ Relevance >> D i s c r i m i n a t i o n P r a c t i c a l i t y Flexibility F eedba a—i 1-ck Relevance +J -•f— r — •r— _D (C •r-cu ar. Between teachers Teacher over time Cost & Time Control over vari ables Flexibility Interpre-tabi1i ty fO O >, S- 4-> +-> T-C i — O T-f 1 o Feedback • Specifici-ty S e l f r a t i n g s ? + - + - + + + R a t i n g s by p e e r s & s u p e r i o r s + 1 + + + 1 + + • + ? S t u d e n t r a t i n g s + 1 + + + ? + + + 7 P e r s o n a l a t t r i b u t e s ? - ? 7 ? - ? ? - ' D i r e c t o b s e r v a t i o n s + - + 1 - - ? + ? + A s s e s s m e n t o f c l a s s -room e n v i r o n m e n t s ? - ? ? ? - ? ? ? -P e r f o r m a n c e t e s t s + 1 + ? ? i + + I co o + I n d i c a t e s t n a t t n e a t i n u u i e \ui v , n a i « v . u _ i . l r i H i r a t p < ; tha t the measure does not s a t i s f y the a t t r i b u t e . ? Shows that at best The measure s a t i s f i e s the a t t r i b u t e only to a ques t ionab le B l a n i T i n d i c a t e s that no conc lus ions are po s s i b l e at t h i s time based on ava iTab le data ; V TOWARDS A MODEL OF TEACHER EVALUATION USING 'SUBSTITUTE' CRITERIA I t i s our c o n t e n t i o n t h a t t e a c h e r e f f e c t i v e n e s s i s a s i t u a t i o n a l c o n c e p t . The t e a c h e r ' s e f f e c t i v e n e s s i s s i g n i f i c a n t l y i n f l u e n c e d by a v a r i e t y o f e n v i r o n -m e n t a l , o r g a n i z a t i o n a l and p e r s o n a l i t y f a c t o r s and d e p e n d i n g upon t h e n a t u r e o f t h e above v a r i a b l e s t h e same i n s t r u c t o r may be e f f e c t i v e i n c e r t a i n s i t u a -t i o n s w h i l e b e i n g i n e f f e c t i v e on many o t h e r o c c a s i o n s . In e v a l u a t i n g a t e a c h e r , we have t o c o n s i d e r t h e i n f l u e n c e o f a number o f f a c t o r s such a s : -T e a c h e r ' s p e r s o n a l i t y c h a r a c t e r i s t i c s , T e a c h e r ' s t r a i n i n g and knowledge o f t h e s u b j e c t m a t t e r , T e a c h e r ' s v a l u e s and a t t i t u d e s , T e a c h e r ' s i n t e r e s t i n t h e s u b j e c t m a t t e r , T e a c h e r ' s s o c i o - e c o n o m i c and c u l t u r a l back g r o u n d , S t u d e n t s ' p e r s o n a l i t y c h a r a c t e r i s t i c s , • S t u d e n t s ' p a s t t r a i n i n g and knowledge o f t h e s u b j e c t , S t u d e n t s ' i n t e r e s t i n t h e s u b j e c t m a t t e r , S t u d e n t s ' v a l u e s and a t t i t u d e s , Students.'i ' s o c i o-economi CL and c u l t u r a l b a c k g r o u n d , T e a c h i n g methods u s e d by t h e i n s t r u c t o r , T e a c h i n g s t y l e o f t h e i n s t r u c t o r , C l i m a t e i n the c l a s s room and t h e i n s t i t u t i o n , S o c i o - e c o n o m i c , c u l t u r a l and p o l i t i c a l e n v i r o n -ments s u r r o u n d i n g t h e e d u c a t i o n a l i n s t i t u t i o n . • C o n s e q u e n t l y i t makes l i t t l e meaning t o s u g g e s t a s e t o f p e r s o n a l i t y f a c t o r s , t r a i n i n g m ethods, t e a c h i n g s t y l e s , e t c . , o f t h e i n s t r u c t o r as s o l e d e t e r m i n a n t , o f t e a c h e r e f f e c t i v e n e s s . In o u r v i e w , t h e d e t e r m i -n a n t s o f t e a c h e r e f f e c t i v e n e s s a r e q u i t e d i f f e r e n t i n v a r i o u s s i t u a t i o n s and t h e c o n c l u s i o n s i n t h i s r e g a r d have l i m i t e d g e n e r a l i z a b i 1 i t y . The p r e s e n c e o f so many v a r i a b l e s , e a c h o f w h i c h s i g n i g i c a n t l y i n f l u e n c e t h e t e a c h e r ' s e f f e c t i v e n e s s makes t h e p r e c i s e measurement o f t e a c h i n g e f f e c t i v e n e s s an e x t r e m e l y d i f f i c u l t , i f n o t i m p o s s i b l e p r o c e d u r e . In many c a s e s t h e use o f ' u l t i m a t e ' c r i t e r i o n ( o f l e a r n i n g and b e h a v i o u r a l change i n d u c e d i n t h e l e a r n e r ) as a m e a s u r e o f t e a c h e r e f f e c t i v e n e s s i s i m p o s s i b l e . Even where i t i s t h e o r e t i c a l l y p o s s i b l e , t h e p r a c t i c a l d i f f i c u l t i e s and c o s t s o f s u c h a s o p h i s t i c a t e d m e a s u r e -ment may q u i t e o u t w e i g h t h e b e n e f i t s o f s u c h an a n a l y s i s t o t h e t e a c h e r and h i s i n s t i t u t i o n . H ence, i t i s f e l t t h a t the use o f some ' s u b s t i t u t e ' c r i t e r i o n o f t e a c h e r e f f e c t i v e n e s s may be j u s t i f i e d by t h e s a v i n g s i n money, t ime and e f f o r t s i n d i c a t e d by such a d e c i s i o n . Among t he v a r i o u s ' s u b s t i t u t e ' c r i t e r i a o f t e a c h e r e f f e c t i v e n e s s , s t u d e n t and peer r a t i n g s seem to be s u p e r i o r on many p o i n t s t h a n , the r e s t o f t he c r i t e r i a . (See t a b l e 2V;1 on page 30) • A l s o , t h e use o f b e h a v i o u r a l c r i t e r i a f o r such an e v a l u a t i o n o f t e a c h e r e f f e c t i v e n e s s appear s to be s u p e r i o r i n deg ree o f r e l i a b i l i t y , o b j e c t i v i t y and j o b o r i e n t a t i o n than g l o b a l o r g r a p h i c s c a l e s (Campbe l l e t . a l , 1973; McNe i l and Popham, 1973) . I t s h o u l d be emphas i zed he re t h a t we a re not by any means d i s c o u n t i n g the i m p o r t a n c e o f o t h e r c r i t e r i a l i k e the p e r s o n a l i t y t r a i t s o f t he t e a c h e r , the o b j e c t i v e measures o f c l a s s room e n v i r o n m e n t s , e t c . as d e t e r m i n a n t s o f t he s u c ce s s o f a pe r son as a t t e a c h e r . We a re o n l y s u g g e s t i n g t h a t the s t u d e n t and peer p e r c e p t i o n s o f t he se t r a i t s a re more i m p o r t a n t than t he t r a i t s per s e . Thus the h i gh l e v e l o f i n te l . 1 i gence o f the t e a c h e r (as i n d i c a t e d by s t a n d a r d p s y c h o -l o g i c a l t e s t s ) may not c o n t r i b u t e t o the p e r c e i v e d e f f e c t i v e n e s s o f t h e t e a c h e r r i f t h e s t u d e n t s and p e e r s do not p e r c e i v e him as i n t e l l i g e n t . S i m i l a r l y , t h o u g h t h e c l a s s room o b s e r v a t i o n t e c h n i q u e s may p r o v i d e t h e r e s e a r c h e r w i t h d e t a i l s o f c l a s s room b e h a v i o u r s o f t h e t e a c h e r and s t u d e n t s , t h e y may n o t g i v e him a c o m p l e t e p i c t u r e o f t h e impact o f t h e s e b e h a v i o u r s on t h e s t u d e n t s and on t h e t e a c h e r h i m s e l f . As such t h e p e r c e p t i o n s o f t h e t e a c h e r and t h e s t u d e n t s a b o u t e a c h o t h e r ' s b e h a v i o u r may be more r e l e v a n t i n t h e c o n t e x t o f m e a s u r i n g t e a c h e r e f f e c t i v e n e s s t h a n t h e r e a l t r a i t s and a t t i t u d e s o f t h e t e a c h e r and h i s s t u d e n t s . In our model o f t e a c h e r e f f e c t i v e n e s s t h e n , t h e r e a r e a t l e a s t t h r e e m a j o r d i m e n s i o n s f o r an e f f e c t i v e t e a c h e r : -1 ) H i s e f f e c t i v e n e s s as a t e a c h e r 2) H i s e f f e c t i v e n e s s as a r e s e a r c h e r and 3) H i s e f f e c t i v e n e s s as a c o l l e a g u e and an •.; admi n i s t r a t o r . The e x t e n t o f h i s e f f e c t i v e n e s s as a t e a c h e r w i l l be e v a l u a t e d p r i m a r i l y by h i s s t u d e n t s , h i s e f f e c t i v e n e s s as a r e s e a r c h e r and c o l l e a g u e w i l l be i n d i c a t e d m a i n l y by t h e r a t i n g s o f h i s c o l l e a g u e s and f i n a l l y h i s e f f e c t i v e n e s s as an a d m i n i s t r a t o r w i l l be j udged by t he a d m i n i s t r a t i v e and s u p p o r t i n g s t a f f and by h i s s t u d e n t s . However, the r a t i n g s o f each d i m e n s i o n need not be r e s t r i c t e d to any p a r t i c u l a r group o f i n d i v i d u a l s . Thus , t he s t u d e n t s may, i n some cases r a t e the i n s t r u c t o r on h i s r e s e a r c h p e r f o r m a n c e , w h i l e on o t h e r o c c a s i o n s the peers may know s u f f i c i e n t l y about h i s t e a c h i n g methods and s t y l e t o be a b l e to r a t e t h e i n s t r u c t o r on t h i s d i m e n s i o n . A BROAD CONCEPTUALISATION OF THE TEACHER"S FUNCTIONS I t i s the aim o f t h i s t h e s i s t o i d e n t i f y t h e v a r i o u s b e h a v i o u r s wh ich r e s u l t i n e f f e c t i v e and i n e f f e c t i v e pe r fo rmance by the i n s t r u c t o r , on each o f t he se d imens ions , . The d e t a i l s o f t he r e s e a r c h methodo logy a re g i v e n i n the nex t c h a p t e r . - 3 7 -CHAPTER I I I BEHAVIOURALLY ANCHORED RATING INSTRUMENTS I INTRODUCTION. The d e c i s i o n to use s t u d e n t and p e e r r a t i n g s as i n d i c a t o r s o f t e a c h i n g e f f e c t i v e n e s s r a i s e s t h e most i m p o r t a n t q u e s t i o n : what t y p e o f s c a l e do we use to e v a l u a t e t h e t e a c h e r s ? The answer i s n o t a s i m p l e one. The m a j o r s u b j e c t m a t t e r o f most t h e o r e t i c a l c o n t r o v e r s y i n p e r s o n n e l t e s t i n g and measurement has been as t o what c o n s t i t u t e s an i d e a l s c a l e . Two a s s o c i a t e d p r o b l e m s he r e a r e : -1) Whether p e r f o r m a n c e s h o u l d be v i e w e d and measured as a s i n g l e c o m p o s i t e o r as a m u l t i d i m e n s i o n a l c o n s t r u c t . 2 ) Whether a. ' g r a p h i c r a t i n g - . s c a l e ' ... s h o u l d be used or w h e t h e r a b e h a v i o u r a l s c a l e s h o u l d b e a p p l i e d . The view t h a t p e r f o r m a n c e s h o u l d be c o n s i d e r e d as a s i n g l e c o m p o s i t e and c a p a b l e o f b e i n g m easured by a ' g l o b a l ' measure has been a d v o c a t e d among o t h e r s -38-by Brogden and T a y l o r : , ( 1 9 5 0 ) , Nagel ( 1953) and V Toops ( 1 9 4 4 ) . The m u l t i d i m e n s i o n a l a p p r o a c h assumes t h a t a l l j o b s ( e v e n mundane on e s ) a r e p s y c h o l o g i c a l l y complex and t h a t each j o b has many i n d e p e n d e n t p e r f o r -mance d i m e n s i o n s . T h u s , a p e r s o n ' s p e r f o r m a n c e on any one d i m e n s i o n need n o t be h i g h l y c o r r e l a t e d w i t h h i s p e r f o r m a n c e on o t h e r d i m e n s i o n s . ( D u n n e t t e , 1963; S c h m i d t and K a p l a n , 1971). The g r a p h i c r a t i n g s c a l e s show r e l a t i v e s t r e n g t h s and w eaknesses i n d i f f e r e n t ' t r a i t s ' o r a s p e c t s o f j o b p e r f o r m a n c e ( G u i o n , 1965). F o r e x a m p l e , a p e r s o n may be r a t e d on an e f f e c t i v e t o i n e f f e c t i v e c o n t i n u u m on v a r i o u s d i m e n s i o n s o f t h e j o b l i k e s p e e d , a c c u r a c y , r e g u l a r i t y o f p e r f o r m a n c e , e t c . An o v e r a l l r a t i n g may a l s o be made i n d e p e n d e n t l y o r c o n s i d e r e d e q u i v a -l e n t t o t h e sum o r a v e r a g e o f t h e s e t r a i t r a t i n g s . In some s i t u a t i o n s w e i g h t e d a v e r a g e s a r e a l s o u s e d . W h i l e t h e g r a p h i c r a t i n g s c a l e s a l s o view p e r f o r -mance as m u l t i d i m e n s i o n a l , t h e ' t r a i t s ' t h e m s e l v e s a r e o f t e n ambiguous and n e c e s s i t a t e c o m b i n i n g o f s c o r e s (Schwab, e t . a l . , 1975). The b e h a v i o u r a l l y r a t e d s c a l e s , on t h e o t h e r hand, a t t e m p t to d e f i n e and measure p e r f o r m a n c e d i m e n s i o n s i n b e h a v i o u r a l t e r m s . -39-The use o f b e h a v i o u r a l s c a l e s has been a d v o c a t e d among ( o t h e r s by ' • B a r r e t t ( 1 9 6 6 ) , B l o o d ( 1 9 7 4 ) , Borman and D u n n e t t e ( 1 9 7 5 ) , Campbel1 e t . a l . , ( 1970) and Keavney and McGann ( 1 9 7 5 ) , on grounds o f h i g h e r r e l i a -b i l i t y , v a l i d i t y and u s e f u l n e s s . C a m p b e l l , D u n n e t t e , L a w l e r and Weick (1970) have a l s o a r g u e d t h a t s i n c e ' e f f e c t i v e n e s s ' i s a f f e c t e d by a v a r i e t y o f e x t r a n e o u s f a c t o r s w hich a r e not u s u a l l y under t h e c o n t r o l o f t h e i n d i v i d u a l and hence n o t d e p e n d e n t on h i s p e r f o r -mance. Hence a measurement o f ' e f f e c t i v e n e s s ' a c c o r d i n g t o them w i l l be q u i t e m e a n i n g l e s s . To q u o t e them: "... I t i s o u r c o n t e n t i o n t h a t p s y c h o l o g i s t s s h o u l d be t r y i n g t o measure and p r e d i c t t h e m a j o r d i m e n s i o n s o f p e r f o r m a n c e r a t h e r t h a n e f f e c t i v e n e s s , s i n c e a measure o f e f f e c t i v e -ness i s one o r more s t e p s removed f r o m what t h e i n d i v i d u a l a c t u a l l y d o e s . . . " (page 1 5 ) . I I A COMPARISON OF BEHAVIOURALLY ANCHORED RATING  INSTRUMENTS (BARI) NlIfcUGRAPHIC SCALES A number o f a d v a n t a g e s o f BARI o v e r t r a d i t i o n a l r a t i n g methods and g r a p h i c s c a l e s have been c l a i m e d . B r o a d l y , t h e a d v a n t a g e s c l a i m e d f o r BARI can be -40-g r o u p e d i n t o two c a t e g o r i e s 1) P r a c t i c a l a d v a n t a g e s and 2) P s y c h o m e t r i c p r o p e r t i e s . 1) P r a c t i c a l a d v a n t a g e s : - Many o f t h e a d v a n t a g e s o f BARI stem from t h e h i g h l e v e l s o f p a r t i c i p a t i o n by p e r s o n s who w i l l u l t i m a t e l y be u s i n g t h e BARI i n t h e i r d a y - t o - d a y work l i f e . As Borman and D u n n e t t e (1975) n o t e d : -"By i n v o l v i n g s u p e r i o r s a n d / o r j o b i n c u m b e n t s a t a l l s t a g e s d u r i n g t h e d e v e l o p m e n t o f j o b b e h a v i o u r o b s e r v a t i o n s c a l e s , t h e method s h o u l d b e t t e r f a c i l i t a t e t h e c h o o s i n g o f j o b d i m e n s i o n s d e f i n i t i o n s and b e h a v i o u r e x amples t h a t a r e r e a d i l y u n d e r s t o o d and a c c e p t e d by p e r s o n s a s k e d t o make p e r f o r m a n c e r a t i n g s . By c o l l e c t i n g c r i t i c a l i n c i d e n t s a b o u t j o b p e r f o r m a n c e and u s i n g them t o d e f i n e dimen-s i o n s and t o a n c h o r d i f f e r e n t l e v e l s o f p e r f o r m a n c e on each d i m e n s i o n , t h e method s h o u l d a l s o h e l p t o d e c r e a s e t h e s e m a n t i c a m b i g u i t i e s t h a t a r e so p r e v a l e n t i n most p e r f o r m a n c e r a t i n g s y s t e m s . " (page 561) The emphasis on j o b o r i e n t e d c r i t e r i a ( as E d i f f e r e n t from t r a i t o r i e n t e d c r i t e r i a ) a n d : t h e -41-r e t r a n s l a t i o n p r o c e d u r e a l s o " . . . a s s u r e s t h a t t h e meaning o f b o t h t h e j o b d i m e n s i o n s and t h e b e h a v i o u r a l i n c i d e n t s c h o s e n t o i l l u s t r a t e them i s h i g h l y s p e c i -f i c and non-ambiguous." ( C a m p b e l l , D u n n e t t e , L a w l e r and W e i c k , 1970; p 1 2 0 ) . In a d d i t i o n , t h e p a r t i c i p a -t i o n by e v a l u a t o r s and a s s e s s e e s i n t h e d e v e l o p m e n t o f t h e s c a l e s may i n c r e a s e t h e e v a l u a t o r ' s m o t i v a t i o n to do an e f f e c t i v e e v a l u a t i o n j o b and e l i m i n a t e t h e a s s e s s e e ' s a n x i e t y and h o s t i l i t y t o w a r d s t h e s c a l e . The use o f c r i t i c a l i n c i d e n t s may a l s o p r o v e t o be u s e f u l i n p r o v i d i n g c o n c r e t e and s p e c i f i c f e e d b a c k t o t h e j o b h o l d e r s and s u g g e s t i o n s f o r i m p r o v i n g t h e i r p e r f o r m a n c e (Cummings and Schwab, 1973). B l o o d ( 1974) s u g g e s t s t h a t t h e c r i t i c a l . i n c i d e n t s can s e r v e as b a s e s f o r t r a i n i n g programs and as m e a s u r e s o f g e n e r a l agreement on and c o m m u n i c a t i o n o f o r g a n i z a t i o n a l p o i i c y . 2) P s y c h o m e t r i c p r o p e r t i e s : - I t has been c l a i m e d t h a t BARI have d e c r e a s e d l e n i e n c y e r r o r ( b e c a u s e l e v e l s o f p e r f o r m a n c e a r e s p e c i f i c ) and h i g h i n t e r -r a t e r a greement (Bo'rman and D u n n e t t e , 1 9 7 5 ) . S i n c e t h e i n p u t s ( i n c i d e n t s ) a r e g i v e n by i n d i v i d u a l s who p r o b a b l y know t h e j o b r e q u i r e m e n t s b e s t , t h e i n s t r u m e n t w i l l a d q u a t e l y c o v e r t h e p e r f o r m a n c e domai - 4 2 -and hence be c o n t e n t v a l i d . R e t r a n s l a t i o n i s a l s o c l a i m e d t o g e n e r a t e i n d e p e n d e n t p e r f o r m a n c e d i m e n s i o n s . The r e l i a b i l i t y o f r a t i n g i s l i k e l y t o be h i g h e r t h a n g r a p h i c s c a l e s b e c a u s e t h e t e r m i n o l o g y used i s m e a n i n g f u l and unambiguous t o t h e r a t e r s . A l s o , c e n t r a l t e n d e n c y e r r o r s o f t h e i n s t r u m e n t a r e c l a i m e d to be much l o w e r t h a n u s u a l g r a p h i c s c a l e s . ( F o r exa m p l e , see S m i t h and K e n d a l l , 1 963). I l l DEVELOPMENT OF BEHAVIOURALLY RATED INSTRUMENTS The p r o c e d u r e f o r d e v e l o p i n g B e h a v i o u r a l l y A n c h o r e d R a t i n g I n s t r u m e n t s b a s e d on t h e p r o c e d u r e o f s c a l e d e x p e c t a t i o n s was g i v e n by S m i t h and K e n d a l l , (1963) and s i n c e used and m o d i f i e d by Borman and D u n n e t t e ( 1 9 7 5 ) , C a m p b e l l , D u n n e t t e , A r v e y and H e l l e r v i k ( 1 9 7 3 ) , F o g l i , H u l i n and B l o o d ( 1 9 7 1 ) , Landy and G u i o n (1970) and Zede c k and B a k e r ( 1 9 7 1 ) . W h i l e t h e r e a r e m i n o r v a r i a t i o n s i n t h e p r o c e d u r e e m p l o y e d , t h e c o n s t r u c t i o n o f BARI has t y p i c a l l y t h e f o l l o w i n g f i v e s t e p s : - -X V ^ o l l e c t i o n o f c r i t i c a l i n c i d e n t s : - I n d i v i d u a l s who p o s s e s s knowledge a b o u t t h e j o b t o be i n v e s t i g a t e d ( f o r e x a m p l e , j o b h o l d e r s , p e e r s and s u p e r v i s o r s ) a r e r e q u e s t e d t o p r o v i d e t h e i n v e s t i g a t o r w i t h s p e c i f i c i l l u s t r a t i o n s o f e f f e c t i v e and i n e f f e c t i v e p e r f o r -mance b e h a v i o u r s . The d a t a c o l l e c t e d a r e i n t h e form o f ' c r i t i c a l i n c i d e n t s ' f i r s t d i s c u s s e d by F l a n a g a n ( 1 9 5 4 ) . 2) I d e n t i f y i n g j o b p e r f o r m a n c e d i m e n s i o n s : - The i n c i d e n t s a r e c l u s t e r e d i n t o a s m a l l e r s e t o f p e r f o r m a n c e d i m e n s i o n s by t h e i n v e s t i g a t o r s . The i d e n t i f i c a t i o n o f t h e d i m e n s i o n s may be done by e i t h e r o f t h e f o l l o w i n g methods: a) C l a s s i f i c a t i o n o f t h e i n c i d e n t s i n t o d i f f e r e n t g r o u p s o f b e h a v i o u r b a s e d on s i m i l a r i t y or d i s s i m i l a r i t y o f t h e i n c i d e n t s . Each g r o u p o f b e h a v i o u r , a f t e r n e c e s s a r y m o d i f i c a t i o n s , i s named by t h e r e s e a r c h e r s t o d e n o t e a d i m e n s i o n o f b e h a v i o u r . b) T h r o u g h a l i t e r a t u r e s u r v e y o r i n i t i a l b r a i n - s t o r m i n g s e s s i o n t h e d i m e n s i o n s o f b e h a v i o u r o f t h e j o b u n d e r c o n s i d e r a t i o n a r e i d e n t i f i e d and named. M o d i f i c a t i o n s i n t h e number and n a t u r e o f t h e d i m e n s i o n s a r e done where e v e r n e c e s s a r y . 3) R e t r a n s l a t i o n o f i n c i d e n t s : - A s e c o n d g r o u p o f p a r t i c i p a n t s , who a r e a l s o k n o w l e d g a b l e a b o u t t h e c h a r a c t e r i s t i c s and r e q u i r e m e n t s o f t h e j o b u n d e r i n v e s t i g a t i o n i s us e d t o r e t r a n s l a t e ( o r r e - a l l o c a t e ) t h e v a r i o u s i n c i d e n t s i n t o t h e d i m e n s i o n s i d e n t i f i e d i n s t e p 2. The p u r p o s e o f r e t r a n s l a t i o n i s t o c h e c k t h e r e l e v a n c y and n o n - a m b i g u i t y o f t h e d i m e n s i o n s and i n c i d e n t s . F o r t h e p u r p o s e o f r e t r a n s l a t i o n , t h e r e t r a n s l a t i n g group i s p r o v i d e d w i t h t h e d e f i n i t i o n s o f v a r i o u s d i m e n s i o n s and asked t o a s s i g n e a c h i n c i d e n t t o t h e d i m e n s i o n i t b e s t d e s c r i b e s . The d e c i s i o n as t o wh i c h i n c i d e n t s a r e t o be k e p t f o r p o t e n t i a l i n c l u s i o n i n t h e f i n a l i n s t r u m e n t i s made b a s e d on what p e r c e n t a g e o f t h e group members a s s i g n them i n t o t h e same d i m e n s i o n as d i d t h e gro u p i n s t e p 2. 4) S c a l i n g o f t h e i n c i d e n t s : - The gro u p o f s u b j e c t s used f o r r e t r a n s l a t i o n i s a l s o a s k e d t o r a t e t h e b e h a v i o u r s ( d e s c r i b e d i n t h e i n c i d e n t s ) on an e f f e c t i v e - i n e f f e c t i v e c o n t i n u u m . ( U s u a l l y a 7;;point o r 9 p o i n t s c a l e i s used. ) Items w i t h r a t i n g s h a v i n g a s t a n d a r d d e v i a t i o n o f 1.50 o r l e s s (on a 7 p o i n t s c a l e ) a r e r e t a i n e d f o r p o s s i b l e i n c l u s i o n i n t h e f i n a l i n s t r u m e n t . 5) D e s i g n i n g t h e f i n a l i n s t r u m e n t : - A s e t o f i n c i d e n t s ( u s u a l l y 7 o r 8 per d i m e n s i o n ) w h i c h meet -45-b o t h t h e r e t r a n s l a t i o n and s t a n d a r d d e v i a t i o n c r i t e r i a a r e used as b e h a v i o u r a l a n c h o r s f o r t h e p e r f o r m a n c e d i m e n s i o n s . I f t h e r e a r e more t h a n 8 i t e m s w h i c h meet b o t h c r i t e r i a , t h e n i t e m s which i n d i c a t e a w i d e r r a n g e o f p e r f o r m a n c e a r e t a k e n . The f i n a l i n s t r u m e n t g e n e r a l l y c o n s i s t s o f a s e t o f v e r t i c a l s c a l e s (one f o r e a c h d i m e n s i o n ) a n c h o r e d by t h e i n c i d e n t s f i n a l l y s e l e c t e d . For e x a m p l e , t h e s c a l e used by H a r a r i and Z e d e c k , 1973, f o r m e a s u r i n g t h e p r o f e s s o r ' s r a p p o r t w i t h and s e n s i t i v i t y t o s t u d e n t s , was b a s i c a l l y o f t h i s form, 7 6-4 when a c l a s s does not u n d e r s t a n d a c e r t a i n c o n c e p t o r f e e l s ' l o s t ' t h i s p r o f e s s o r c o u l d be e x p e c t e d t o s e n s e i t and a c t t o c o r r e c t t h e s i t u a t i o n . When c o n f r o n t e d w i t h q u e s t i o n s a f t e r c l a s s t h i s s t a t i s t i c s p r o f e s s o r c o u l d be e x p e c t e d t o s t a y and t a l k t o t h e s t u d e n t s u n t i l t h e n e x t c l a s s must b e g i n . T h i s p r o f e s s o r , when a s t u d e n t comes t o h i s o f f i c e . , f o r h e l p c o u l d be e x p e c t e d t o go t h r o u g h one e x p l a n a t i o n o f t h e m a t e r i a l and t e l l t h e s t u d e n t t o r e a d c e r t a i n c h a p t e r s o f t h e t e x t and come back i f he s t i l l has p r o b l e m s D u r i n g l e c t u r e s , t h i s p r o f e s s o r c o u l d o f t e n be e x p e c t e d t o t e l l s t u d e n t s w i t h q u e s t i o n s t o see him d u r i n g h i s o f f i c e h o u r s . u T h i s p r o f e s s o r c o u l d be e x p e c t e d n o t s e e s t u d e n t s i n d i v i d u a l l y e x c e p t d u r i n g h i s r e g u l a r l y s c h e d u l e d o f f i c e h o u r s . T h i s p r o f e s s o r c o u l d be e x p e c t e d t o t r y t o hum-i l i a t e o r embarass s t u d e n t s who d i s a g r e e w i t h him, -46-IV PAST RESEARCH ON BEHAVIOURALLY ANCHORED INSTRUMENTS A number o f r e s e a r c h e r s have i n v e s t i g a t e d t h e v a r i o u s p s y c h o m e t r i c and o t h e r p r o p e r t i e s o f BARI. W h i l e no d e t a i l e d d i s c u s s i o n o f t h e s e v a r i o u s e x p e r i -ments i s i n t e n d e d h e r e , i t i s c o n s i d e r e d u s e f u l t o summarize t h e i m p o r t a n t f i n d i n g s r e l a t e d t o r e l i a b i l i t y , h a l o , l e n i e n c y and i n d e p e n d e n c e o f p e r f o r m a n c e d i m e n s i o n s i n BARI. 1) R e l i a b i l i t y : - R e l i a b i l i t y d e n o t e s t h e c o n s i s t e n c y o f measurement o f an i n s t r u m e n t . I t a l s o shows t h e e x t e n t t o w h i c h a s e t o f measurements i s f r e e f r o m random e r r o r v a r i a n c e . ( G u i o n , 1965). I h e e a r l y s t u d i e s u s i n g BARI ( F o g l i , H u l i n and B l o o d , 1971; Landy and G u i o n , 1970; S m i t h and K e n d a l l , 1963) have p r o d u c e d h i g h r e l i a b i l i t i e s . However, C a m p b e l l , D u n n e t t e , A r v e y and H e l l e r v i k ( 1973) f o u n d r e l i a b i l i t i e s t o be low. In 2 s u b s e q u e n t s t u d i e s Borman and V a l l o n (1974) and W i l l i a m s and S e i l e r (1973) f o u n d t h e r e l i a b i l i t y o f t h e s c a l e t o be h i g h e r t h a n o t h e r n o n - b e h a v i o u r a l s c a l e s . F i n a l l y , Keaveny and McGann (1975) i n a v e r y r e c e n t s t u d y f o u n d t h e v a r i a b i l i t y i n r a t i n g s t o be q u i t e low f o r BARI. R e l i a b i l i t y o f BARI, on t h e w h o l e , seems t o be - 4 7 -q u i t e h i g h i n most c a s e s . / 2) H a l o : - H a l o r e f e r s t o t h e use o f one p a r t i c u l a r l y f a v o u r a b l e o r u n f a v o u r a b l e r a t i n g t o c o l o u r e v e r y o t h e r r a t i n g g i v e n t o an i n d i v i d u a l . Thus i n f i l l i n g o u t t h e r a t i n g f o r m , t h e s t u d e n t may make an o v e r a l l e v a l u a t i v e j u d g e m e n t o f t h e i ns t r u c t o r w h i c h may i n f l u e n c e a l l h i s l a t e r r a t i n g s . C a m p b e l l , D u n n e t t e , A r v e y and H e l l e r v i k (1973) f o u n d t h a t s c a l e s d e r i v e d f r o m b e h a v i o u r s c a l i n g m e t h o d o l o g y showed l e s s h a l o t h a n summated r a t i n g s c a l e s d e s i g n e d t o measure t h e same p e r f o r m a n c e d i m e n s i o n s . G r o n e r (1974) a l s o f o u n d s u p p o r t i n g e v i d e n c e f o r t h e above h y p o t h e s i s . However, Borman and V a l l o n (1974) f o u n d BARI t o be no b e t t e r t h a n o t h e r n o n - b e h a v i o u r a l s c a l e s as f a r as h a l o e r r o r s were c o n c e r n e d . Borman and D a h h e t t e (1975) and Keaveny and McGann (1975) f o u n d t h a t BARI do have l e s s h a l o e r r o r s . B u r n a s k a and H o l l m a n (1974) f o u n d t h a t BARI r e s u l t e d i n l e s s h a l o t h a n a n u m e r i -c a l l y a n c h o r e d and / t r a d i t i o n a l s c a l e . However, t h e y f e l t t h a t a l l t h e t h r e e p r o c e d u r e s have e x c e s s i v e l e v e l s o f h a l o . Many o t h e r s t u d i e s ( H a r a r i and Z e d e c k , 1973 ; Zedeck and B a k e r , 1972) d i d not s p e c i f i -c a l l y answer t o t h i s q u e s t i o n . I t i s o b v i o u s f r o m t h e above d i s c u s s i o n t h a t r e s u l t s o f p a s t r e s e a r c h -48-on t h i s q u e s t i o n a r e somewhat c o n f l i c t i n g . However, t h e y a l s o i n d i c a t e p o s s i b i l i t y o f t h e s u p e r i o r i t y o f BARI o v e r o t h e r g r a p h i c s c a l e s on t h i s p r o p e r t y . 3) L e n i e n c y : - L e n i e n c y d e n o t e s t h e t e n d e n c y on t h e p a r t o f t h e r a t e r t o g i v e m a r k e d l y skewed d i s t r i b u t i o n o f r a t i n g s , a l t h o u g h n o t n e c e s s a r i l y i n t h e f a v o u r a b l e d i r e c t i o n . T h a t BARI s h o u l d be l e s s s u s c e p t i b l e t o l e n i e n c y e f f e c t s b e c a u s e o f unambiguous d i m e n s i o n s and a n c h o r s was h y p o t h e s i s e d by S m i t h and K e n d a l l i n 1963. C a m p b e l l , D u n n e t t e , A r v e y and H e l l e r v i k (1973) f o u n d t h a t BARI had much l o w e r l e n i e n c y e r r o r s t h a n n u m e r i c a l l y a n c h o r e d s c a l e s s i n c e a v e r a g e r a t i n g s on t h e i r b e h a v i o u r a l i n s t r u m e n t were g e n e r a l l y c l o s e r t o t h e m i d - p o i n t s o f the s c a l e s t h an were a v e r a g e r a t i n g s o f n u m e r i c a l l y a n c h o r e d r a t i n g s c a l e s . However, Borman and V a l l o n (1974) f o u n d BARI t o p o s s e s s more l e n i e n c y and s m a l l e r s t a n d a r d d e v i a t i o n s o f r a t i n g s t h a n g r a p h i c s c a l e s . F i n a l l y , Keaveny and McGann (1973) d i d n o t f i n d any s i g n i f i c a n t d i f f e r e n c e s i n l e n i e n c y e r r o r s between BARI and o t h e r s c a l e s . The c u r r e n t e v i d e n c e , h e n c e , i s n o t a d e q u a t e t o form any c o n c l u s i v e o p i n i o n s on t h i s q u e s t i o n . 4) D i m e n s i o n i n d e p e n d e n c e : - D i m e n s i o n i n d e p e n d e n c e p r e s e n t when r a t i n g s r e c e i v e d by t h e a s s e s s e e on any s i n g l e d i m e n s i o n o f p e r f o r m a n c e a r e n o t s i g n i f i c a n t l y c o r r e l a t e d w i t h r a t i n g s on o t h e r d i m e n s i o n s . Many o f t h e s t u d i e s t h a t have e x a m i n e d d i m e n s i o n i n t e r c o r r e l a t i o n s o f BARI have n o t compared t h e s e f i g u r e s w i t h s i m i l a r c o r r e l a t i o n f i g u r e s o f n u m e r i c a l l y a n c h o r e d s c a l e s , (Campion , 1972 ; Landy and G u i o n , 1970; Zedeck and B a k e r , 1972). The s t u d i e s w h i c h have f o c u s e d t h e m s e l v e s on t h i s i s s u e have o b t a i n e d mixed r e s u l t s . F o r example, C a m p b e l l e t . a l (1973) f o u n d t h a t t h e d i m e n s i o n s o f t h e i r b e h a v i o u r a l l y a n c h o r e d s c a l e had somewhat l o w e r i n t e r - c o r r e l a t i o n t h a n a n u m e r i c a l l y a n c h o r e d i n s t r u -ment. (0..28 t o 0.63 f o r BARI as a g a i n s t 0.35 t o 0.76 f o r n u m e r i c a l l y r a t e d s c a l e s . ) A r v e y and H o y l e (1974) and Borman and V a l l o n (1974) on t h e o t h e r h a n d , f o u n d no d i f f e r e n c e s i n t h e d i m e n s i o n i n t e r - c o r r e l a t i o n s o f BARI and o t h e r t r a d i t i o n a l s c a l e s . G e n e r a l l y s p e a k i n g t h e d i m e n s i o n i n t e r - c o r r e l a t i o n s o f BARI seem t o /be q u i t e low. The r a n g e o f i n t e r - c o r r e l a t i o n s o f r a t i n g s on p e r f o r m a n c e d i m e n s i o n s o f BARI i n v a r i o u s s t u d i e s a r e g i v e n .in t h e n e x t page. However, i t s h o u l d be n o t e d t h a t c o m p a r i t i v e f i g u r e s f o r n u m e r i c a l l y a n c h o r e d s c a l e s a r e n o t p r o v i d e d h e r e . -50-Range o f c o r r e l a t i o n s o f r a t i n g s on v a r i o u s Name o f r e s e a r c h e r ( s ) p e r f o r m a n c e d i m e n s i o n s o f BARI used by t h e r e s e a r c h e r ( s ) Landy and G u i o n , (1970) D i c k i n s o n and T i c e (1973) W i l l i a m s , e t . a l (19 73) Borman (1974) 0.11 to 0.70 0.24 t o 0.54 0.08 t o 0.86 0.19 t o 0.78 In t h i s i n s t a n c e a l s o , no c o n c l u s i o n s a r e p o s s i b l e i n t h e a b s e n c e o f a d d i t i o n a l e m p i r i c a l e v i d e n c e . 5) O t h e r p r o p e r t i e s : - C a m p b e l l e t . a l (1973) f o u n d t h a t BARI p o s s e s s e d g r e a t e r d i s c r i m i n a n t v a l i d i t y ; t h a n n u m e r i c a l l y a n c h o r e d s c a l e s . Keaveny and McGann (1975) a l s o p r o v i d e d some r e c e n t e v i d e n c e s u p p o r t i n g t h i s c l a i m . The i n t e r - r a t e r r e l i a b i l i t y was a l s o h i g h e r f o r BARI t h a n n u m e r i c a l s c a l e s . i n many s t u d i e s (Borman and V a l l o n , 1974; Ma.as , 1 965). The c o n t e n t v a l i d i t y o f t h e BARI i s a l s o c l a i m e d t o be h i g h e r t h a n o t h e r c a s e s ( C a m p b e l l e t . a l , 1 9 7 3). T h e s e f i n d i n g s , though r e q u i r i n g f u r t h e r c o n f i r m a t i o n , -51-p r o v i d e some g r o u n d f o r t h e i n i t i a l a s s u m p t i o n t h a t BARI i s s u p e r i o r t o o t h e r n u m e r i c a l s c a l e s i n c e r t a i n r e s p e c t s . Many o f t h e p r o b l e m s r e l a t e d t o BARI seem t o a r i s e f r o m 1) n o t d e f i n i n g t h e r e - t r a n s l a t i o n c r i t e r i o n s t r i n g e n t l y enough t o a c h i e v e a d e q u a t e c l a r i t y o f d i m e n s i o n s ; 2) n o t u s i n g s m a l l s t a n d a r d d e v i a t i o n v a l u e s f o r r a t i n g s a t t h e s c a l i n g s t a g e ; 3) s h r i n k a g e i n t h e number o f i n c i d e n t s by t h e t i m e t h e f i n a l i n s t r u m e n t i s d e s i g n e d ; 4) i n e x p e r i e n c e and l a c k o f a d e q u a t e s k i l l on t h e p a r t o f r a t e r s . However, t h e s e p r o b l e m s a r e n o t i n s u r m o u n t a b l e and as Schwab e t . a l (1975) o b s e r v e d , " I t i s t o o e a r l y to c o n c l u d e t h a t BARI c a n n o t e v e n t u a l l y d e m o n s t r a t e t h e i r t h e o r e t i c a l a d v a n t a g e s . " Many o f t h e a d v a n t a g e s o f BARI may be i n t h e d i m e n s i o n s g e n e r a t e d and i n t h e p a r t i c i p a t i v e p r o c e s s o f s c a l e c o n s t r u c t i o n r a t h e r t h a n t h r o u g h i n h e r e n t p s y c h o m e t r i c p r o p e r t i e s o f t h e s c a l e . As Borman and D u n n e t t e (1975) o b s e r v e d , t h e s t r e n g t h o f BARI p e r h a p s l i e i n t h e w e a l t h o f d a t a i t p r o v i d e s a b o u t t h e j o b and t h e j o b o c c u p a n t r a t h e r t h a n on any o t h e r s i n g l e p r o p e r t y . In t h e i r w o r d s : -"Time and e f f o r t r e q u i r e d f o r d e v e l o p i n g ;• -52-b e h a v i o u r - b a s e d j o b p e r f o r m a n c e s c a l e s a r e v e r y g r e a t . R e s u l t s o f t h e p r e s e n t s t u d y c a n n o t be used t o a r g u e c o n v i n c i n g l y t h a t s u c h t i m e and e f f o r t i s w a r r a n t e d i f t h e s c a l e s a r e t o be used s t r i c t l y f o r p e r f o r -mance r a t i n g s . One would be b e t t e r o f f , p e r h a p s , : t o a r g u e t h e u s e f u l n e s s o f t h e b e h a v i o u r s c a l i n g m e t h o d o l o g y ( a s B l o o d , 1974 has) on t h e b a s i s o f t h e w e a l t h o f i n f o r m a t i o n a b o u t j o b r e q u i r e m e n t s and j o b p e r f o r m a n c e y i e l d e d by t h e t e c h n i q u e - a d d i n g i n a r a s p y s t a g e w h i s p e r , t h a t t h e y do a l s o show a modest s u p e r i o r i t y o v e r o t h e r methods i n r e d u c i n g t h e u s u a l r a t i n g e r r o r s . " (page 565) V USE OF BARI IN TEACHING EFFECTIVENESS RESEARCH W h i l e many b e h a v i o u r a l l y r a t e d s c a l e s have been d e v e l o p e d t o e v a l u a t e t h e p e r f o r m a n c e o f n u r s e s , s a l e s m e n , m a n a gers, g r o c e r y c l e r k s , s y s t e m a n a l y s t s , s e c r e t a r i e s , f i r e f i g h t e r s and e n g i n e e r s , t h e a p p l i c a -t i o n o f BARI t o t e a c h i n g e f f e c t i v e n e s s r e s e a r c h has n o t been done on a c o n t i n u o u s b a s i s and t o any g r e a t -5.3-e x t e n t . In f a c t , t h e o n l y m a j o r work done so f a r have been by H a r a r i and Zedeck ( 1 9 7 3 ) , J e n s e n , ( 1 9 5 1 ) , Keaveny and McGann (1975) and K o n i s b u r g ( 1 9 5 4 ) . T h i s l a c k o f i n t e r e s t seems t o be m a i n l y due t o t h e e m p h a s i s o f e d u c a t i o n a l r e s e a r c h e r s on o t h e r c r i t e r i a o f t e a c h i n g e f f e c t i v e n e s s such as p e r s o n a l a t t r i b u t e s , c l a s s r o o m o b s e r v a t i o n s and l e a r n i n g g a i n . A l s o , t h e o r g a n i z a t i o n a l r e s e a r c h e r s who a r e c o n c e r n e d w i t h such p s y c h o m e t r i c •;. r e s e a r c h seem t o p r e f e r c o m m e r c i a l o r i n d u s t r i a l s e t t i n g s f o r t h e i r r e s e a r c h as a g a i n s t e d u c a t i o n a l i n s t i t u t i o n s . VI CONCLUSION One o f t h e m a j o r o b j e c t i v e s o f t h e p r e s e n t s t u d y i s t o i d e n t i f y t h e b e h a v i o u r a l d i m e n s i o n s o f t e a c h i n g e f f e c t i v e n e s s , t h r o u g h t h e use o f B e h a v i o u r a l l y A n c h o r e d R a t i n g I n s t r u m e n t . In d o i n g s o , t h i s s t u d y a l s o hopes t o f e l p f i l l t h e p r e s e n t gap i n t h e a p p l i c a t i o n o f BARI i n t e a c h i n g r e s e a r c h and t o g e n e r a t e d a t a v a l i d a t i n g t h e h y p o t h e s i z e d a d v a n t a g e s o f BARI. The s i t e c h o s e n i s IRM D i v i s i o n o f t h e F a c u l t y o f Commerce a t U.B.C. The s t e p s i n t h i s s t u d y a r e d e t a i l e d i n t h e n e x t c h a p t e r . -54-CHAPTER IV DESIGNING BEHAVIOURAL SCALES FOR TEACHER EVALUATION I EXISTING MEASURES OF TEACHER EFFECTIVENESS IN FACULTY OF COMMERCE A m a j o r measure o f t e a c h i n g e f f e c t i v e n e s s c u r r e n t l y used i n t h e F a c u l t y o f Commerce f o r t h e p u r p o s e s o f i n s t r u c t o r and c o u r s e e v a l u a t i o n i s t h e S t u d e n t I n s t r u c t i o n a l R e p o r t (SIR) q u e s t i o n n a i r e d e s i g n e d by t h e E d u c a t i o n a l T e s t i n g S e r v i c e (ETS) P r i n c e t o n , New J e r s e y , U.S.A. ( C e n t r a , 1 9 7 2 ) . The SIR q u e s t i o n n a i r e i s d e s i g n e d f o r e v a l u a t i o n o f i n s t r u c t o r s and c o u r s e s i n N o r t h A m e r i c a n u n i v e r s i -t i e s and c o m p r i s e s o f 39 s t a t e m e n t s r e l a t e d t o i n s t r u c t o r p e r f o r m a n c e and c o u r s e c h a r a c t e r i s t i c s . S t u d e n t s a r e a s k e d t o r a t e t h e i r i n d i v i d u a l i n s t r u c t o r on a 1-4 o r 1-5 s c a l e f o r e a c h s t a t e m e n t . An e a r l i e r s t u d y u s i n g SIR i t e m s have shown t h a t t h e i t e m s have r e l i a b i l i t y e x c e e d i n g 0.70 f o r c l a s s e s e n r o l l i n g 20 o r more s t u d e n t s and s l i g h t l y l e s s f o r c l a s s e s h a v i n g 15 - 2Q s t u d e n t s . ( C e n t r a , 1 9 7 3 ) . From t h e q u e s t i o n n a i r e d a t a , t h e ETS produces;.' summary s c o r e s f o r i n d i v i d u a l i n s t r u c t o r s . The -55-s p e c i f i c d e t a i l s g i v e n i n t h e summary s c o r e s a r e : -1) Mean and s t a n d a r d d e v i a t i o n ( f o r e a c h o f t h e 39 i t e m s i n t h e q u e s t i o n n a i r e ) f o r e a c h c o u r s e i n t h e f a c u l t y and i; 2) Summary o f t h e r e s p o n s e s t o v a r i o u s q u e s t i o n s i n t h e q u e s t i o n n a i r e . A}? s p e c i m e n o f SIR q u e s t i o n n a i r e and summary s c o r e s h e e t a r e g i v e n i n A p p e n d i x C and D r e s p e c t i v e l y . A f a c t o r i a l s t u d y o f t h e SIR s c o r e s has shown t h a t t h e y r e p r e s e n t f o u r m a j o r d i m e n s i o n s : S t r u c t u r e C o n s i d e r a -t i on Degree t o w h i c h t h e i n s t r u c t o r c l a r i f i e d t h e m a t e r i a l , met o b j e c t i v e s , s e t g o a l s and was o r g a n i s e d or p r e p a r e d . Degree to w h i c h the i n s t r u c t o r showed c o n c e r n t o s t u d e n t s , was a v a i l a b l e and h e l p f u l and e m p h a s i z e d on f r e e d o m and o p e n n e s s . E f f o r t : Degree t o w h i c h t h e i n s t r u c t o r showed c o n c e r n f o r t h e w o r k l o a d o f t h e s t u d e n t and m a t c h e d ] t h e c o u r s e w i t h t h e p a s t p r e p a r a t i o n o f t h e s t u d e n t . E v a l u a t i o n C o n c e r n shown by t h e i n s t r u c t o r f o r ; f a i r mode o f e v a l u a t i o n . F o r d e t a i l s o f t h i s s t u d y see Baba and A c e (1975)-. - 5 6 -From t h e summary s c o r e s r e c e i v e d f r o m ETS, t h e s c o r e s o f each i n s t r u c t o r ( f o r each c o u r s e ) on 3 d i m e n s i o n s o f s t r u c t u r e , c o n s i d e r a t i o n and w o r k l o a d ( e f f o r t ) a r e c a l c u l a t e d . A l l t h e i n s t r u c t o r s i n t h e f a c u l t y a r e t h e n r a n k e d a c c o r d i n g t o t h e i r s c o r e s on t h e above d i m e n s i o n s . The p r e s e n t SIR e v a l u a t i o n s y s t e m p r e s e n t s some t h e o r e t i c a l and p r a c t i c a l p r o b l e m s . A t a t h e o r e t i c a l l e v e l , t h e v a l i d i t y o f t h i s s c a l e f o r e v a l u a t i n g p e r f o r m a n c e o f U.B.C F a c u l t y has n o t y e t been e s t a b l i -s h e d . S i n c e t h e o b j e c t i v e s , m e t h o d o l o g y , and c l i m a t e o f t e a c h i n g and l e a r n i n g d i f f e r c o n s i d e r a b l y f r o m one u n i v e r s i t y t o a n o t h e r ; and f r o m one f a c u l t y t o a n o t h e r , no s i n g l e q u e s t i o n n a i r e o r e v a l u a t i o n s c a l e i s l i k e l y t o meet t h e u n i q u e r e q u i r e m e n t s o f d i f f e r e n t c o u r s e s , f a c u l t i e s and u n i v e r s i t i e s . As H a r a r i and Zedeck (1973) r e c e n t l y p o i n t e d o u t : " . . . I t i s u n l i k e l y t h a t , a l 1 , o r e v e n most d i s c i p l i n e s r e q u i r e i d e n t i c a l p a t t e r n s o f t e a c h e r b e h a v i o u r . By i t s v e r y n a t u r e , an e v a l u a t i o n f o r m f o r a l l d i s c i p l i n e s i s n o t c o n d u c i v e t o s p e c i f i c b e h a v i o u r and t r a i t d e s c r i p t i o n s . S p e c i f i c i t e m s , and -57-even d i m e n s i o n s o f t e a c h i n g a b i l i t y , a p p r o p r i a t e f o r t e a c h i n g o f p s y c h o -be q u i t e i n a p p r o p r i a t e f o r t h e o f a r t , p h i l o s o p h y , p h y s i c s , e t c . . " ( p a g e 261) L i k e many o t h e r g r a p h i c r a t i n g s c a l e s , SIR a l s o s u f f e r s f r o m p r o b l e m s o f c e n t r a l t e n d e n c y , h a l o and r e l i a b i l i t y . R n e c d o t a l l y , many s t u d e n t s and f a c u l t y members have e x p r e s s e d c o n c e r n o v e r t h e ambiguous and g l o b a l n a t u r e o f t h e d i m e n s i o n s used i n SIR q u e s t i o n n a i r e s . Some members o f t h i s f a c u l t y have o b s e r v e d c o n s i d e r a b l e c e n t r a l t e n d e n c y i n t h e i r r a t i n g s by u n d e r g r a d u a t e s t u d e n t s . T h a t r a t i n g s may be h i g h l y c o r r e l a t e d w i t h ' l i k i n g ' o f a t e a c h e r by h i s s t u d e n t s and t h e t e a c h e r ' s ' p o p u l a r i t y ' may make e f f e c t i v e n e s s synonymous w i t h p o p u l a r i t y ( e v e n t h o u g h t h e o r e t i c a l l y t h e y s h o u l d be i n d e p e n d e n t o f e a c h o t h e r ) . I f t h i s i n d e e d i s t r u e , t h e f e e d b a c k coming t o a f a c u l t y member may a t b e s t t e l l him o n l y w h e t h e r he i s ' p o p u l a r ' o r i s g e n e r a l l y ' l i k e d ' r a t h e r t h a n how he can im p r o v e h i s e f f e c t i v e -n e s s as a t e a c h e r . From t h e s t u d e n t s ' p o i n t o f v i e w , many o f the- s t a t e m e n t s on w h i c h t h e y have t o g r a d e t h e i r i n s t r u c t o r u s i n g t h e SIR a r e c o n c e p t u a l l y p o s s l b l y t h a t a r e l o g y may t e a c h i ng -58-complex and a s k i n g f o r g l o b a l j u d g e m e n t s on t h e i r p a r t . The n o n - b e h a v i o u r a l t o n e o f many q u e s t i o n s and t h e use o f a L i k e r t - t y p e s c a l e may r e s u l t i n c o n s i d e r a b l e s t u d e n t s u b j e c t i v i t y c r e e p i n g i n t o t h e e v a l u a t i o n s . In t h i s s e n s e , t h e q u e s t i o n n a i r e may be m e a s u r i n g more t h e p a r t i c u l ar. p r e d i spos i.tons and p s y c h o l o g i c a l a t t r i b u t e s o f t h e s t u d e n t r a t h e r t h a n t h a t o f h i s i n s t r u c t o r whom he i s e v a l u a t i n g . The g l o b a l n a t u r e o f many o f t h e q u e s t i o n s i n SIR and t h e n o n - s p e c i f i c i t y o f t h e f e e d b a c k coming t o t h e t e a c h e r l i m i t s t h e u s e f u l n e s s o f t h e i n s t r u m e n t as a measure o f t e a c h i n g e f f e c t i v e n e s s . I U OBJECTIVES OF THE STUDY The p r e s e n t s t u d y hopes t o i d e n t i f y t h e c r i t i c a l f a c t o r s o f e f f e c t i v e t e a c h i n g i n t h e IRM D i v i s i o n o f t h e F a c u l t y o f Commerce a t U.B.C. I t i s f e l t t h a t t h e we a k n e s s e s i n h e r e n t i n SIR q u e s t i o n n a i r e can be r e d u c e d by use o f b e h a v i o u r a l a n c h o r i n g p r o c e d u r e s u g g e s t e d by S m i t h and K e n d a l l ( 1 9 6 3 ) . U s i n g t h i s p r o c e d u r e , t h i s s t u d y hopes t o a c h i e v e t h e f o l l o w i n g o b j e c t i v e s : -1) To c o n s t r u c t a t e a c h i n g e f f e c t i v e n e s s i n s t r u m e n t f o r e v a l u a t i o n o f f a c u l t y members -59-i n t h e IRM D i v i s i o n o f t h e F a c u l t y o f Commerce. T h i s i n s t r u m e n t , h o p e f u l l y w i l l i d e n t i f y t h e i m p o r t a n t p e r f o r m a n c e d i m e n s i o n s o f t h e i n s t r u c -t o r t e a c h i n g b e h a v i o u r a l s c i e n c e s . Some o f t h e s e d i m e n s i o n s may be g e n r a l - i z a b l e t o o t h e r d i v i s i o n s and t o o t h e r u n i v e r s i t i e s . 2) To c o n t e n t v a l i d a t e t h e e x i s t i n g SIR c o u r s e e v a l u a t i o n i t e m s by c o m p a r i n g t h e SIR i n s t r u -ments w i t h t h e p e r f o r m a n c e d i m e n s i o n s a r r i v e d a t d u r i n g t h i s s t u d y . I l l STEPS IN THE STUDY A Sample D e t e r m i n a t i o n : T h r e e s a m p l e s , ; 274 s t u d e n t s , 5 f a c u l t y members and 5 a d m i n i s t r a t i v e p e r s o n n e l were used f o r i n i t i a l d a t a c o l l e c t i o n ( i n t h e f o r m o f c r i t i c a l i n c i d e n t s ) . T h i s sample c o n s t i t u t e d a p p r o x i m a t e l y one h a l f o f t h e t o t a l number o f p e r s o n s a v a i l a b l e i n IRM d i v i s i o n do t h e c a s e o f s t u d e n t s and f a c u l t y members. O n l y t h o s e a d m i n i s t r a t o r s who had d i r e c t c o n t a c t w i t h p r o f e s s o r s i n IRM d i v i s i o n were c o n s i d e r e d . The s t u d e n t sample s i z e f a r e s f a v o u r a b l y w i t h -60-sample s i z e s i n e a r l i e r s t u d i e s aimed a t d e s i g n o f BARI. T h u s , f o r d a t a c o l l e c t i o n p u r p o s e s K i r c h n e r and D u n n e t t e (1957) u s e d 85 s u b j e c t s , J e n s e n (1960) 144 s u b j e c t s and H a r a r i and Zedeck (1973) 66 s u b j e c t s . A l s o , u s i n g c e n t r a l l i m i t t h e o r e m i t can be shown t h a t we need o n l y 96 s u b j e c t s t o a r r i v e a t w i t h i n 0.2 s t a n d a r d d e v i a t i o n v a l u e o f t h e p o p u l a t i o n mean w i t h a p r o b a b i l i t y o f 0.95 ( H a y s , 1966). The p r o b l e m w i t h such m a t h e m a t i c a l e s t i m a t i o n s i s , however, t h a t t h e y i n d i c a t e o n l y t h e number o f s u b j e c t s t o be used w i t h o u t any c o n s i d e r a t i o n f o r t h e q u a l i t y o f i n c i d e n t s , ; g e n e e a t e d by t h e s e p e r s o n s . F o r r e - t r a n s l a t i o n and s c a l i n g p u r p o s e s a s e c o n d g r o u p o f 103 s t u d e n t s , 4 f a c u l t y members and 2 a d m i n i -s t r a t i v e s t a f f were u s e d . No member o f t h i s s e c o n d g r o u p had p a r t i c i p a t e d a t t h e d a t a c o l l e c t i o n s t a g e . The c o u r s e - w i s e d i s t r i b u t i o n o f t h e sample u s e d f o r d a t a c o l l e c t i o n , r e - t r a n s ! a t i o n and s c a l i n g i s g i v e n i n A p p e n d i x E. B Data C o l 1 e c t i o n : C r i t i c a l i n c i d e n t s were c o l l e c t e d f r o m g r o u p s o f s t u d e n t s and i n d i v i d u a l f a c u l t y members and a d m i n i s t r a t i v e p e r s o n n e l i n September and O c t o b e r -61-1975. The g r o u p s o f s t u d e n t s were s e l e c t e d i n s u c h a way as t o be r e p r e s e n t a t i v e o f t h e v a r i o u s c o u r s e l e v e l s i n t h e f a c u l t y . The number o f s t u d e n t s who p a r t i c i p a t e d i n t h e d a t a c o l l e c t i o n and r e - t r a n s l a t i o n and s c a l i n g a r e shown i n A p p e n d i x E. The methods o f d a t a c o l l e c t i o n f r o m t h e s t u d e n t s was as f o l l o w s : The r e s e a r c h e r spoke t o each c l a s s o f s t u d e n t s f o r t e n m i n u t e s t e l l i n g them a b o u t t h e o b j e c t i v e s o f t h e s t u d y and a b o u t t h e t y p e o f i n f o r m a -t i o n needed t o c o n s t r u c t a b e h a v i o u r a l l y a n c h o r e d r a t i n g s c a l e . Two c r i t i c a l i n c i d e n t s s h o w i n g . ' e f f e c t i v e and i n e f f e c t i v e p e r f o r m a n c e i n an u n r e l a t e d o c c u p a t i o n were g i v e n t o t h e s t u d e n t s as examples t o f a m i l i a r i z e them w i t h t h e t e c h n i q u e o f w r i t i n g c r i t i c a l i n c i d e n t s . In t h e n e x t t w e n t y m i n u t e s s t u d e n t s and t h e f a c u l t y member p r e s e n t were a s k e d t o w r i t e e f f e c t i v e and i n e f f e c t i v e c r i t i c a l i n c i d e n t s , r e p r e s e n t i n g d a y - t o - d a y o r t y p i c a l p e r f o r m a n c e o f f a c u l t y e n g a g e d i n t e a c h i n g b e h a v i o u r a l s c i e n c e m a t e r i a l . The t e r m ' B e h a v i o u r a l S c i e n c e ' was d e f i n e d t o i n c l u d e O r g a n i z a t i o n Beha-v i o u r , I n d u s t r i a l R e l a t i o n s , P s y c h o l o g y , S o c i o l o g y and A n t h r o p o l o g y . T h i s method 1 o f da'ta c o l l e c t i o n ^ Was hoped t o e l i m i n a t e t h e b i a s and p e r c e p t u a l d i f f e r e n c e s c h a r a c t e r i s t i c o f t h e i n t e r v i e w method o f d a t a c o l l e c -t i o n . A l s o , t h e same r e s e a r c h e r went t o a l l c l a s s e s and g r o u p s o f s t u d e n t s , t h u s e l i m i n a t i n g p r o b l e m s o f i n t e r - i n t e r v i e w e r d i f f e r e n c e s . In t o t a l 1311 i n c i d e n t s were c o l l e c t e d f r o m 271 s t u d e n t s 5% o f t h e i n c i d e n t s ( s e l e c t e d a t random) were k e p t s e p a r a t e f o r l a t e r c o m p a r i s o n p u r p o s e s . The r e m a i n i n g 95% was u s e d f o r i d e n t i f y i n g t h e d i m e n s i o n s o f b e h a v i o u r . As f a r as p o s s i b l e t h e i n c i d e n t s were n o t e d i t e d . However, o b v i o u s g r a m m a t i c a l e r r o r s and n o n - b e h a v i o u r a l i t e m s o f d a t a were removed. The f i n a l number o f i n c i d e n t s u s e d f o r d i m e n s i o n i d e n t i f i c a t i o n was 1171. 33 i n c i d e n t s were c o l l e c t e d f r o m f a c u l t y members and 60 f r o m t h e a d m i n i s t r a t i v e s t a f f . In t h e s e c a s e s a l l t h e i n c i d e n t s were used f o r t h e p u r p o s e o f i d e n t i f i c a t i o n o f p e r f o r m a n c e d i m e n s i o n s . The i n c i d e n t s g i v e n by s t u d e n t s , p r o f e s s o r s and a d m i n i s t r a t o r s were t r e a t e d as s e p a r a t e g r o u p s o f i t e m s f o r l a t e r a n a l y s i s . C D i m e n s i o n I d e n t i f i c a t i o n : A g r o u p c o n s i s t i n g o f one f a c u l t y member, one -63-Ph.D s t u d e n t , one g r a d u a t e s t u d e n t and one u n d e r -g r a d u a t e s t u d e n t (3 males and 1 f e m a l e ) i d e n t i f i e d a s e t o f p e r f o r m a n c e d i m e n s i o n s b a s e d on t h e i n c i -d e n t s . The g r o u p was c h o s e n i n s u c h a manner as t o r e p r e s e n t t h e v a r i o u s l e v e l s o f a c a d e m i c community i n t h e IRM D i v i s i o n . Each member o f t h e g r o u p r e a d t h e 1171 i n c i d e n t s g i v e n by t h e s t u d e n t s a m p l e . B a s e d on t h e s e i n c i d e n t s v a r i o u s d i m e n s i o n s o f p e r f -mance were i d e n t i f i e d by e a c h g r o u p member. A f t e r > d i s c u s s i o n s , a s e t o f p e r f o r m a n c e d i m e n s i o n s a g r e e d upon by a l l members o f t h e g r o u p were i d e n t i f i e d . The v a r i o u s b e h a v i o u r a l i t e m s ( o r i n c i d e n t s ) f o r m i n g p a r t o f each d i m e n s i o n were a l s o i d e n t i f i e d by t h e g r o u p members. 8 major p e r f o r m a n c e d i m e n s i o n s ( o r c a t e g o r i e s o f b e h a v i o u r ) were i d e n t i f i e d f r o m t h e i n c i d e n t s g i v e n by s t u d e n t s . From t h e i n c i d e n t s g i v e n by .; F a c u l t y members and a d m i n i s t r a t i v e s t a f f 5 p e r f o r -mance d i m e n s i o n s each were i d e n t i f i e d . O n l y 2 members o f t h e group ( i n v o l v e d i n i d e n t i f i c a t i o n o f d i m e n s i o n s o f p e r f o r m a n c e ) p a r t i c i p a t e d i n t h i s c a s e . / - 6 4 -E a c h p e r f o r m a n c e d i m e n s i o n was d e f i n e d i n t e r m s o f one o r more b e h a v i o u r a l d e s c r i p t i o n s o r sub c a t e -g o r i e s o f b e h a v i o u r . T h e r e were 39 s u c h b e h a v i o u r a l d e s c r i p t i o n s ( o r s u b - c a t e g o r i e s o f b e a h v i o u r ) , f o r t h e 8 d i m e n s i o n s e m e r g i n g f r o m i n c i d e n t s g i v e n by s t u d e n t s a m p l e . F o r t h e d i m e n s i o n s i d e n t i f i e d f r o m i n c i d e n t s g i v e n by f a c u l t y members and a d m i n i s t r a t o r s , t h e r e were 19 and 17 s u b - c a t e g o r i e s o f b e h a v i o u r s r e s p e c t i v e l y . The v a r i o u s d i m e n s i o n s and s u b - c a t e g o r i e s o f b e h a v i o u r i d e n t i f i e d a t t h i s s t a g e a r e shown i n A p p e n d i x F, G and H t o t h i s s t u d y , D T e s t i n g C o m p r e h e n s i v e n e s s o f d a t a c o l l e c t e d : T e s t i n g c o m p r e h e n s i v e n e s s o f t h e i n c i d e n t s c c o l l e c t e d from s t u d e n t s ( i . e . , w h e t h e r t h e i n c i d e n t s c o v e r a l l majo r c a t e g o r i e s o f b e h a v i o u r ) was done i n two ways. A f i r s t c h e c k was made by c l a s s i f y i n g ' s e p a r a t e l y t h e 5% o f t o t a l i n c i d e n t s ( f r o m s t u d e n t s ) i n i t i a l l y k e p t a p a r t f o r t h i s p u r p o s e . I t was f o u n d t h a t t h e i n c i d e n t s c o u l d be f i t t e d i n t o t h e a l r e a d y e x i s t i n g d i m e n s i o n s ( f o r m e d on t h e b a s i s o f 95% o f t o t a l i n c i d e n t s ) . T h i s was a f i r s t i n d i c a t i o n t h a t \ - 6 5 -t h e d i m e n s i o n s i d e n t i f i e d were a d e q u a t e . A more d e t a i l e d c h e c k was done i n t h e ; f o i l o w i n g manner: The 1 2 6 1 i n c i d e n t s c o l l e c t e d f r o m t h e . s t u d e n t s were d i v i d e d i n t o 20 g r o u p s (on a random b a s i s ) o f a p p r o x i m a t e l y 65 i n c i d e n t s e a c h . Each o f t h e s e g r o u p s was examined i n d i v i d u a l l y t o see how many o f t h e 39 s u b - c a t e g o r i e s o f b e h a v i o u r t h e y a c c o u n t e d f o r . The c u m u l a t i v e number o f s u b - c a t e g o r i e s o f b e h a v i o u r a c c o u n t e d f o r by t h e 20 g r o u p s ( o f i n c i -d e n t s ) i n c r e a s e d v e r y r a p i d l y i n t h e b e g i n n i n g and t h e n r o s e v e r y s i o w l y . A f t e r t h e f i r s t few g r o u p s , t h e c u m u l a t i v e number o f s u b - c a t e g o r i e s a c c o u n t e d f o r became q u i t e s t a b l e . I t i s p o s s i b l e t o assume t h e r e f o r e , t h a t t h e c o l l e c t i o n o f d a t a has n o t been s t o p p e d t o o a b r u p t l y . T h i s method has been used e a r l i e r by A n d e r s o n and N i l s s o n ( 1 9 6 4 ) . The d e t a i l s o f t h i s a n a l y s i s i s g i v e n i n A p p e n d i x I o f t h i s t h e s i s . E R e - t r a n s l a t i o n o f I n c i d e n t s : Of t h e i n c i d e n t s c o l l e c t e d f r o m s t u d e n t s , -p r o f e s s o r s and a d m i n i s t r a t i v e s t a f f , t h e r e were 430, 26 and 54 n o n - r e p e t i t i v e i t e m s r e s p e c t i v e l y . -66-O n l y t h e s e i n c i d e n t s were us e d f o r r e - t r a n s l a t i o n p u r p o s e s . A new g r o u p o f 110 s t u d e n t s was used f o r r e - t r a n s l a t i o n o f t h e i n c i d e n t s o r i g i n a l l y c o l l e c t e d f r o m s t u d e n t s . In t h e c a s e o f r e - t r a n s l a t i o n o f i n c i d e n t s c o l l e c t e d from p r o f e s s o r s and a d m i n i s t r a -t i v e s t a f f , 4 f a c u l t y members and 2 a d m i n i s t r a t o r s r e s p e c t i v e l y j were used f o r t h i s p u r p o s e . The r e - t r a n s l a t i o n p r o c e s s was as f o l l o w s : a) Each member who p a r t i c i p a t e d i n t h e s t u d y was g i v e n : i ) a l i s t o f c r i t i c a l i n c i d e n t s ( I n t h e c a s e o f s t u d e n t s , t h e number o f i n c i d e n t s g i v e n t o ea c h p e r s o n was a p p r o x i m a t e l y 30; each p r o f e s s o r r e - t r a n s l a t e d 26 i n c i d e n t s and t h e a d m i n i s t r a t i v e s t a f f d i d 54 i n c i d e n t s ) i i ) a l i s t o f c a t e g o r i e s ( d i m e n s i o n s ) o f t e a c h e r b e h a v i o u r which c o n t a i n e d t h e name o f t h e c a t e g o r y and t h e v a r i o u s b e h a v i o u r a l d e s c r i p t i o n ( o r sub-,; c a t e g o r i e s ) a s s o c i a t e d w i t h i t . i i i ) a b l a n k s h e e t o f p a p e r . -67-b) Each p a r t i c i p a n t r e a d a l l t h e c a t e g o r i e s o f b e h a v i o u r g i v e n to him ( h e r ) and g a i n e d an u n d e r s t a n d -i n g o f t h e b e h a v i o u r a l d e s c r i p t i o n s a s s o c i a t e d w i t h i t . Then he r e a d e a c h c r i t i c a l i n c i d e n t i n d i v i -d u a l l y and i n d i c a t e d t o w h i c h c a t e g o r y t h e c r i t i c a l i n c i d e n t b e s t b e l o n g e d . v c) In t h e c a s e o f s t u d e n t and f a c u l t y g r o u p s , e a c h member, a f t e r he had i d e n t i f i e d t h e r e l a t e d c a t e g o r i e s ( d i m e n s i o n s ) o f b e h a v i o u r f o r each i n c i d e n t a s s i g n e d t o him, a l s o e v a l u a t e d t h e r e l e v a n c e o f t h e v a r i o u s i n c i d e n t s as f o l l o w s : 1) He r e a d as a g r o u p , a l l t h e i n c i d e n t s w h i c h b e l o n g e d t o t h e same c a t e g o r y o f b e h a v i o u r . 2) He t h e n r a t e d t h e r e l a t i v e r e l e v a n c e o f e a c h i n c i d e n t t o i t s a s o c i a t e d c a t e g o r y on a 9 p o i n t s c a l e . A s c o r e o f 9 on t h e s c a l e i n d i c a t e d h i g h r e l e v a n c e . 3) He r e p e a t e d t h i s p r o c e d u r e f o r e a c h c a t e g o r y u n t i l a l l t h e i n c i d e n t s were r a t e d f o r t h e i r r e l e v a n c e w i t h i n t h e a s s i g n e d c a t e g o r y . 4) In t h e c a s e o f s t u d e n t g r o u p s o n l y , t h e p a r t i c i -p a n t s r a n k e d t h e 8 c a t e g o r i e s o f b e h a v i o u r -68-c o m p r i s i n g t h e i n s t r u m e n t . The p u r p o s e o f t h i s s t e p was t o u n d e r s t a n d t h e r e l a t i v e i m p o r t a n c e o f v a r i o u s c a t e g o r i e s o f t e a c h e r b e h a v i o u r as p e r c e i v e d by t h e s t u d e n t s . The i n c i d e n t s t h a t were c a t e g o r i s e d by a t l e a s t 75% o f t h e s t u d e n t s c o r r e c t l y were c o n s i d e r e d non-ambiguous and c l e a r i n n a t u r e . 116 i n c i d e n t s g i v e n by s t u d e n t s , a c c o r d i n g to t h i s c r i t e r i o n , were non-ambiguous i n n a t u r e . In t h e c a s e o f p r o f e s s o r s and a d m i n i s t r a t i v e s t a f f t h e number o f non-ambiguous i n c i d e n t s were 17 and 28 r e s p e c t i v e l y . F S c a l i n g t h e I t e m s : The p u r p o s e o f t h e s c a l i n g e x e r c i s e was t o i d e n t i f y t h e c r i t i c a l i n c i d e n t s w h i c h would form p a r t o f t h e f i n a l i n s t r u m e n t s . F o r t h i s p u r p o s e , t h e s t u d e n t s , f a c u l t y members and a d m i n i s t r a t i v e s t a f f i n t h e r e t r a n s l a t i n g g r o u p were a s k e d t o r a t e e a c h i n c i d e n t i n d i c a t i n g t h e r e l a t i v e e f f e c t i v e n e s s o f t h e s p e c i f i c b e h a v i o u r . Each s t u d e n t was g i v e n a p p r o x i m a t e l y 30 i n c i d e n t s w h i c h he was t o s c a l e -69-on a 9 p o i n t s c a l e . (A r a t i n g o f '1' i n d i c a t e d ' e x t r e m e l y i n e f f e c t i v e b e h a v i o u r ' where as a r a t i n g o f '9' i n d i c a t e d ' e x t r e m e l y e f f e c t i v e b e h a v i o u r ' ) . In t h e c a s e o f p r o f e s s o r s and a d m i n i s t r a t i v e s t a f f , a 7 p o i n t s c a l e was u s e d s i n c e i t was f e l t t h a t r a t i n g on a 7 p o i n t s c a l e was e a s i e r f o r t h e r a t e r t h a n r a t i n g on a 9 p o i n t s c a l e . A f t e r t h e r a t i n g s f o r a l l i n c i d e n t s were o b t a i n e d , i t e m s w i t h s t a n d a r d d e v i a t i o n s g r e a t e r t h a n 1.50 were e l i m i n a t e d . In t h e c a s e o f i t e m s s c a l e d by s t u d e n t s , t h i s c r i t e r i o n was more s t r i n g e n t t h a n t h o s e u s e d by F o g l i , H u l i n and B l o o d ( 1 9 7 1 ) , H a r a r i and Z e d e c k (1973) and S m i t h and K e n d a l l (1963) who u s e d s t a n d a r d d e v i a t i o n c r i t e r i o n o f 1.5 on a s h o r t e r , 7 p o i n t s c a l e . The use o f t h i s c u t - o f f c r i t e r i a made t h e r e t a i n e d i t e m s show c o n s i d e r a b l e a g reement as t o t h e d e g r e e t o w h i c h e a c h b e h a v i o u r r e p r e s e n t e d ' i n e f f e c t i v e ' or ' e f f e c t i v e ' p e r f o r m a n c e by t h e i i n s t r u c t o r . G C o n s t r u c t i o n o f t h e F i n a l S c a l e : T h o s e i t e m s ( o r i n e f f e c t i v e ) w h i c h showed h i g h b e h a v i o u r and had r a n g e o f e f f e c t i v e h i g h r e l e v a n c e -70-were s e l e c t e d f o r t h e p u r p o s e o f i n c l u s i o n i n "the f i n a l s c a l e . Where 2 i t e m s showed t h e same d e g r e e o f e f f e c t i v e / i n e f f e c t i v e b e h a v i o u r t h e i t e m w i t h h i g h e r r e l e v a n c e was s e l e c t e d f o r i n c l u s i o n . A n o t h e r g u i d e l i n e u s e d i n t h e s e l e c t i o n o f an i t e m f r o m a g r o u p o f 2 o r more i t e m s h a v i n g same mean and s t a n d a r d d e v i a t i o n v a l u e s was t h e e x t e n t t o w h i c h t h e b e h a v i o u r was c o n s i d e r e d l i k e l y to happen i n d a y - t o - d a y l i f e by t h e r e s e a r c h e r . The b e h a v i o u r a l i t e m s t h a t were used as a n c h o r s i n t h e f i n a l e v a l u a t i o n f o r m were r e w o r d e d from a c t u a l b e h a v i o u r s t o e x p e c t e d b e h a v i o u r s . T h a t i s , a s t a t e m e n t s u c h as "The i n s t r u c t o r h o l d s an i n t r o -d u c t o r y s e s s i o n i n w h i c h th e s t u d e n t s ' e x p e c t a t i o n s a r e a s c e r t a i n e d and i n s t r u c t o r ' s c o u r s e o b j e c t i v e s a r e made c l e a r " was r e - w o r d e d t o "The i n s t r u c t o r c o u l d be e x p e c t e d to h o l d an i n t r o d u c t o r y s e s s i o n i n w h i c h t h e s t u d e n t s ' e x p e c t a t i o n s a r e a s c e r t a i n e d and i n s t r u c t o r ' s c o u r s e o b j e c t i v e s a r e made c l e a r . " A c c o r d i n g t o S m i t h and K e n d a l l (1963) t h e r a t i o n a l e f o r t h i s s t e p i s two f o l d : -1) The a n c h o r s become "so c o n c r e t e t h a t i n v i e w o f p r e v i o u s agreement by t h e p e e r g r o u p -71-c e n t r a l t e n d e n c y o r h e d g i n g ' / e f f e c t s wi 11 be m i n i m i zed 2) The a n c h o r s become so v e r i f i a b l e t h a t t h e i i n s i g h t , j u d g e m e n t and v a l u e s o f t h e r a t e r become open t o c h a l l e n g e l a t e r i f s u b s e q u e n t b e h a v i o u r o f t h e r a t e e s h o u l d f a i l t o c o n f o r m t o p r e d i c t i o n . /y A t h i r d a d v a n t a g e has been n o t e d by H a r a r i and Z e d e c k (1973) who f e e l t h a t such a r e - w o r d i n g w i l l make i t e a s i e r f o r t h e r a t e r t o compare t h e b e h a v i o u r a l a n c h o r s t o s i m i l a r b e h a v i o u r s t h a t t h e r a t e e has e i t h e r d e m o n s t r a t e d i n t h e p a s t o r c o u l d be e x p e c t e d t o show i n f u t u r e . The f i n a l s c a l e s t o be used f o r e v a l u a t i n g t e a c h e r e f f e c t i v e n e s s by h i s s t u d e n t s , p e e r s and t h e a d m i n i s t r a t i v e s t a f f a r e g i v e n i n A p p e n d i x J , K and L r e s p e c t i v e l y . The s c a l e s t o be used by s t u d e n t s f o r r a t i n g t h e i r p r o f e s s o r s c o n t a i n 7 d i m e n s i o n s ( c a t e g o r i e s ) o f b e h a v i o u r . O n l y one i t e m r e p r e s e n t i n g t h e d i m e n s i o n ' A d m i n i s t r a t i v e H a n d l i n g " came t h r o u g h t h e s c a l i n g p r o c e d u r e w i t h a s t a n d a r d d e v i a t i o n o f -72-l e s s t h a n 1.50. Hence t h i s d i m e n s i o n was d r o p p e d f r o m t h e f i n a l s c a l e . I n t h e c a s e o f s c a l e s t o be us e d by p r o f e s s o r s f o r r a t i n g t h e i r c o l l e a g u e s t h e r e a r e o n l y t h r e e d i m e n s i o n s o f p e r f o r m a n c e . No i t e m r e p r e s e n t i n g t h e d i m e n s i o n s " B e h a v i o u r i n t h e c l a s s room" a n d " H a n d l i n g A d m i n i s t r a t i v e m a t t e r s ' came t h r o u g h t h e s c a l i n g p r o c e d u r e w i t h s t a n d a r d d e v i a t i o n o f l e s s t h a n 1.50. The s c a l e t o be us e d by a d m i n i s t r a t i v e s t a f f f o r r a t i n g t h e p r o f e s s o r s c o n t a i n s 5 d i m e n s i o n s o f b e h a v i o u r . A s | an I e x a m p l e , d i m e n s i o n s ( c a t e g o r i e s ) | o f ! " '—.' 1 • •• i. The s c a l e s t o be u s e d by s t u d e n t s f o r r a t i n g t h e i r p r o f e s s o r s j a r e g i v e n b e l o w : - j 1. COURSE OUTLINING AND STRUCTURING Sub- # D e f i n i t i o n o f c o u r s e o b j e c t i v e s , w o r k l o a d c a t e g o r i e s : and s t u d e n t e v a l u a t i o n p r o c e d u r e s ; e x t e n t t o w h i c h s t u d e n t s ' c o u r s e e x p e c t a t i o n s and d i f f i c u l t i e s a r e t a k e n i n t o a c c o u n t i n t h i s d e f i n i t i o n ; p r o v i s i o n o f a d e t a i l e d s c h e d u l e f o r t h e l e c t u r e s , d i s c u s s i o n s a n d a s s i g n m e n t -s u b m i s s i o n s ; - 7 2 A -e x t e n t t o w h i c h s t r u c t u r e and c o n c e p t u a l f ramework p r o v i d e d i n l e c t u r e s and d i s c u s s i o n s e x t e n t o f r e v i e w and summary o f c o u r s e r n a t e r i a l ; and p r o v i s i o n o f l e c t u r e n o t e s o r o t h e r m a t e r i a l . 1. / 2\ COVERAGE OF MATERIAL Sub- The n a t u r e and r e l e v a n c e o f s u b j e c t m a t t e r c a t e g o r i e s : t a u g h t i n t h e c o u r s e ; t h e s c o p e and d e p t h o f c o v e r a g e o f t h e s u b j e c t m a t t e r ; t h e e x t e n t t o w h i c h c o u r s e c o n t e n t i s l i n k e d t o o t h e r r e s e a r c h e s and r e a l - l i f e p r a c t i c e s ; v a r i e t y and r e l e v a n c e o f r e a d i n g m a t e r i a l s u g g e s t e d ; d e g r e e t o w h i c h c o u r s e i s r e l a t e d t o o t h e r d i s c i p l i n e s and e x t e n t t o w h i c h i n s t r u c t o r a c c e p t s new i d e a s s u g g e s t e d by s t u d e n t s . c a t e g o r i e s : l e c t u r e s and d i s c u s s i o n s ; a c t u a l l a n g u a g e and j a r g o n s u s e d by t h e i n s t r u c t o r ; e n e r g y and e n t h u s i a s m d i s p l a y e d by h i m ; s t y l e and pace o f t a l k i n g i n c l u d i n g use o f j o k e s , a n a l o g i e s and p e r s o n a l s t o r i e s ; Sub- E x t e n t o f l o g i c a l f r a m e w o r k p r o v i d e d i n -72B-i n f o r m a l i t y and n a t u r a l n e s s o f t h e i n s t r u c t o r ' s b e h a v i o u r ; e m p h a s i s shown by him on two-way communi-c a t i o n s i n s i d e , t h e c l a s s room; and p e r s o n a l h y g i e n e , o u t f i t and a p p e a r a n c e o f t h e i n s t r u c t o r . 4 .-TEACHING METHODS Sub-. r > V a r i o u s t e c h n i q u e s and m e d i a u s e d by t h e c a t e g o r i e s ' : i n s t r u c t o r t o c o n v e y t h e c o u r s e c o n t e n t t o t h e s t u d e n t s ; methods u s e d may be l e c t u r e s , c a s e and g r o u p d i s c u s s i o n s , p r o j e c t g r o u p s , r o l e p l a y i n g , games, g u e s t s p e a k e r s , f i e l d t r i p s and u s e o f b l a c k b o a r d and o t h e r v i s u a l a i d s . 5I EVALUATION Sub- E x t e n t t o w h i c h e v a l u a t i o n c r i t e r i a i s c a t e g o r i e s : o b j e c t i v e l y d e f i n e d ; e x t e n t t o w h i c h p a p e r s and o t h e r m a t e r i a l a r e g r a d e d p r o m p t l y , o b j e c t i v e l y and e q u i t a b l y ; d e g r e e t o w h i c h t h e i n s t r u c t o r g i v e s f e e d b a c k t o t h e s t u d e n t s a b o u t t h e i r p e r f o r m a n c e ; d e g r e e o f f l e x i b i l i t y s e e n i n h i s e v a l u -a t i o n p r o c e s s ; and e x t e n t t o w h i c h he g i v e s p o s i t i v e c r i t i c i s m s •''j • ' a n c * e n c ° u r a g e m e n t t o h i s s t u d e n t s . 6/. INTERACTION OUTSIDE CLASS Sub- E x t e n t t o w h i c h t h e i n s t r u c t o r i s a v a i l a b l e c a t e g o r i e s : f o r ' c o n s u l t a t i o n o u t s i d e c l a s s h o u r s ; d e g r e e o f f r i e n d l i n e s s and i n t e r e s t shown by him t o s t u d e n t s who go t o him w i t h t h e i r p r o b l e m s and i d e a s ; -72C-e x t e n t o f h e l p p r o v i d e d by him i n s o l v i n g s t u d e n t s ' c o u r s e / r e s e a r c h / c a r e e r r e l a t e d o r o t h e r p e r s o n a l p r o b l e m s ; and d e g r e e o f i n f o r m a l i t y o f h i s b e h a v i o u r i n i n t e r a c t i n g w i t h s t u d e n t s o u t s i d e c l a s s . F L E X I B I L I T Y AND RESPONSIVENESS Sub- E x t e n t t o w h i c h t h e i n s t r u c t o r i s f l e x i b l e c a t e g o r i e s : i n h i s t e a c h i n g s c h e d u l e s , methods an d s t y l e ; d e g r e e o f r e s p o n s i v e n e s s shown by him t o s t u d e n t s ' needs and r e q u e s t s ; e x t e n t t o w h i c h he i s aware o f s t u d e n t s ' p r o b l e m s and c a p a b i l i t i e s and a d j u s t s h i s s t y l e , p a c e and d e p t h o f c o v e r a g e t o s o l v e t h e i r p r o b l e m s and s u i t t h e i r c a p a b i l i t i e s ; • e x t e n t t o w h i c h he c h a n g e s h i s g o a l s and methods t o s u i t • s t u d e n t s * a s p i r a t i o n s and n e e d s ; and e x t e n t t o w h i c h he c o n s c i o u s l y r e c e i v e s f e e d b a c k a b o u t h i s p e r f o r m a n c e and c o n s t a n t l y s t r i v e s t o i m p r o v e h i s p e r f o r m a n c e . IV PSYCHOMETRIC PROPERTIES OF THE S C A L E : A T e s t i n g t h e p r o p e r t i e s o f t h e s c a l e : A m o d i f i e d v e r s i o n o f t h e s c a l e t o be u s e d by s t u d e n t s f o r e v a l u a t i n g t h e i r p r o f e s s o r s was a d m i n i s t e r e d t o a g r o u p o f 98 s t u d e n t s w i t h a v i e w t o a s s e s s some o f i t s p s y c h o m e t r i c p r o p e r t i e s . On t h e w h o l e 62 u s a b l e and c o m p l e t e q u e s t i o n n a i r e s were r e t u r n e d . A sample o f t h e s c a l e u s e d f o r t h i s s t u d y i s shown i n A p p e n d i x M. -73-The t i t l e page o f t h e q u e s t i o n n a i r e d e s c r i b e d t h e o b j e c t i v e s o f t h e s c a l e . The s t u d e n t s were a l s o a s k e d t o r a t e t h e i r i n s t r u c t o r on t h e 7 p o i n t g l o b a l s c a l e o f e f f e c t i v e n e s s g i v e n on t h e f i r s t page. The c a t e g o r y t i t l e and s t a t e m e n t s d e f i n i n g each c a t e g o r y were p r o v i d e d on t h e f i r s t o f two.pages a l l o c a t e d f o r e a c h c a t e g o r y . The s t u d e n t s were a s k e d t o r e a d t h e d e f i n i n g s t a t e m e n t s f o r a c a t e g o r y and rate, t h e i n s t r u c t o r on a 4 p o i n t s c a l e i n d i c a t i n g how o f t e n t h e i n s t r u c t o r behaved i n t h e p a r t i c u l a r manner. The s e c o n d page c o n s i s t e d o f t h e b e h a v i o u r a l a n c h o r s a r r a n g e d on a d e s c e n d i n g o r d e r ( b a s e d on t h e mean v a l u e o f each i t e m ) . The r a t e r was a s k e d to c h o o s e any one b e h a v i o u r w h i c h was t y p i c a l o f t h e i n s t r u c t o r . The 4 p o i n t s c a l e used i n t h e d e f i n i t i o n o f v a r i o u s c a t e g o r i e s o f b e h a v i o u r had 2 m a j o r f u n c t i o n s 1) t o e n s u r e t h a t each s t u d e n t r e a d t h e d e f i n i -t i o n a l s t a t e m e n t s c a r e f u l l y and u n d e r s t o o d them. 2) To a c t as a c h e c k on t h e r e l i a b i 1 i t y o f t h e i n s t r u m e n t . S i n c e b o t h t h e d e f i n i t i o n a l ( w h i c h were q u a s i - b e h a v i o u r a l i n n a t u r e ) and t h e b e ha-v i o u r a l i t e m s b e l o n g t o one c a t e g o r y and hence -74-a r e p a r t s o f t h e same s c a l e , t h e y a r e i n f a c t t h e s p l i t - h a l f s o f one and t h e same s c a l e . A h i g h c o r r e l a t i o n o f a l l p a i r s o f s c a l e s ( i . e . , a l l p a i r s o f q u a s i - b e h a v i o u r a l and b e h a v i o u r a l s c a l e s ) i n d i c a t e h i g h ' r e l i a b i l i t y o f t h e i n s t r u m e n t . B. F i n d i ngs :•*:• ! I The c o r r e l a t i o n between t h e s c o r e s g i v e n t o t h e i n s t r u c t o r on t h e d e f i n i t i o n a l i t e m s (summated mean s c o r e s ) and s c o r e s on t h e b e h a v i o u r a l : s c a l e s were as f o l l o w s : D i m e n s i o n No. C o r r e l a t i o n between s c o r e s on d e f i n i t i o n a l i t e m s and s c o r e s on t h e b e h a v i o u r a l s c a l e s . 1 0..625 2 0.853 3 0.851 4 0.772 5 0.668 6 0.80 3 7 0.859 As can be s e e n , t h e c o r r e l a t i o n s a r e q u i t e h i g h i n d i c a t i n g h i g h r e l i a b i l i t y o f t h e s c a l e s . -75-2. :. C o n t e n t v a l i d i t y o f t h e i n s t r u m e n t : A c h e c k on t h V'content v a l i d i t y o f t h e i n s t r u m e n t was done i n t h i s s t u d y w i t h d i m e n s i o n s o f c o l l e g e t e a c h i n g as d e s c r i b e d i n some e a r l i e r s t u d i e s . The r e s u l t s o f t h i s a n a l y s i s a r e g i v e n below:-A c o m p a r i s o n o f t h e d i m e n s i o n s o f c o l l e g e t e a c h i n g d e s c r i b e d i n p a s t r e s e a r c h w i t h t h e d i m e n s i o n s > . o f t h e t h r e e b e h a v i o u r a l s c a l e s Name o f t h e D i m e n s i o n s o f c o l l e g e t e a c h i n g d i m e n s i o n ( s ) r e s e a r c h e r ( s ) i d e n t i f i e d by t h e r e s e a r c h e d s ) i n t h e B e h a -by c o m p a r i n g t h e d i m e n s i o n s o f t e a c h i n g a r r i v e d a t E q u i v a l e n t v i o u r a l s c a l e Brau n s t e i n & B e n s t o n Use o f c l a s s t i m e S t i m u l a t i o n o f i n t e r e s t s T e a c h i ng S t y l e (1973) * C u r r e n t r e s e a r c h p e r f o r m a n c e P r o f e s s i o n a l v i s i b i l i t y R e s e a r c h A c t i v i t i e s C l a r i t y o f a s s i g n m e n t s E v a l u a t i on C o n t r i b u t i o n t o t h e d e v e l o p -ment Co m m u n i c a t i o n w i t h i n d e p a r t m e n t I n t e r a c t i on wi t h c o l l e a g u e s O v e r a l l t e a c h i n g p e r f o r m a n c e O v e r a l l e v a l -u a t i o n on 7 p o i n t g l o -b a l s c a l e A v a i l a b i l i t y t o s t u d e n t s I n t e r a c t i o n o u t s i d e * T h i s p u b l i c a t i o n does not g i v e a f u l l l i s t i n g o f a l l t e a c h e r b e h a v i o u r d i m e n s i o n s used by t h e r e s e a r c h e r s . ^76-H a r a r i and Z edeck (1973) O r g a n i z a t i on C o u r s e o u t l i n -i n g & s t r u c t u -r i n g R e l e v a n c e ( o f m a t e r i a l ) A s s i g n m e n t and w o r k l o a d D e l i v e r y ( o f s p e e c h ) Depth o f knowledge A b i l i t y t o m o t i v a t e T e s t i n g G r a d i ng I n t e r - p e r s o n a l r e l a t i o n s w i t h s t u d e n t s C o v e r a g e o f m a t e r i a l T e a c h i n g s t y l e E v a l u a t i o n I n t e r a c t i o n o u t s i d e c l a s s H a r v e y and B a r k e r f ; (1970) O b j e c t i v e s c l a r i f i e d by t h e i n s t r u c t o r O r g a n i z a t i o n o f c o u r s e S k i l l s as a t e a c h e r S k i l l as a d i s c u s s i o n l e a d e r P r e p a r a t i o n f o r c l a s s Range o f i n t e r e s t s Knowledge o f s u b j e c t A b i l i t y t o a r o u s e i n t e r e s t Mannerisms A t t e n t i o n t o s t u d e n t p r o d u c t S ense o f humor Speech and e n u n c i a t i o n s A s s i g n m e n t s F a i r n e s s i n g r a d i n g W i l l i n g n e s s t o h e l p s t u d e n t s R e c o g n i t i o n o f own l i m i t a -t i o n s c o u r s e o u t ! i n-i n g and s t r u c t u r i ng T e a c h i n g s t y l e E v a l u a t i o n C o v e r a g e o f m a t e r i a l E v a l u a t i o n I n t e r a c t i on o u t s i d e F l e x i b i l i t y and R e s p o n s i v e n e s s A b i l i t y t o a r o u s e i n t e r e s t E x t e n t o f i n v o l v e m e n t i n t e a c h i n g and r e s e a r c h S k i l l i n g u i d a n c e o f l e a r n i n g None -77-I s a a c s o n , M c K e a c h i e , M i l h o l l a n d L i n , H o f e l 1 e r , B a e r w a l d t and Z i n n (1964) O r g a n i z a t i o n and p l a n n i n g o f t h e c o u r s e Group i n t e r a c t i o n ( i . e . , c l a s s room b e h a v i o u r G e n e r a l t e a c h i n g s k i l l s S t u d e n t - t e a c h e r r a p p o r t ( i . e . , g e n e r a l f r i e n d l i n e s and p e r m i s s i v e n e s s o f t e a c h e r ) Amount and d i f f i c u l t y o f work a s s i g n e d ; r e l e v a n c e o f r e a d i ngs c o u r s e o u t ! i n -i n g and s t r u c t u r i n g T e a c h i ng s t y 1 e c o v e r a g e o f m a t e r i a l C o n c e r n o v e r q u a l i t y o f work E v a l u a t i o n M c K e a c h i e A b i l i t y t o make s t u d e n t s (1969) + f e e l f r e e t o ask q u e s t i o n s Volume, t o n e , e n u n c i a t i o n , r a t e , v o c a b u l a r y e t . , o f - . s p e e c h E x t e n t o f r e s p e c t shown t o s t u d e n t s E x t e n t t o w h i c h p r o f e s s o r i n t r o d u c e s i n t e r e s t i n g i d e a s T e a c h i n g I n t e r e s t i n t h e s u b j e c t t a u g h t s t y l e Use o f examples o r i l l u s t r a -t i o n s i n e x p l a i n i n g t h e o r y E x t e n t t o w h i c h o r a t e r i a l i s p r e s e n t e d i n an o r g a n i z e d f a s h i o n E x t e n t t o which t h e p r o f e s s o r s t i m u l a t e s t h i n k i n g E x t e n t t o w h i c h m a t e r i a l i s p u t a c r o s s i n an i n t e r e s t i n g way . E x t e n t t o w h i c h h e l p s s t u d e n t s who have d i f f i c u l t y S e n s i t i v i t y t o s t u d e n t ' s f e e l i n g s and p r o b l e m s F'l e x i b i 1 i t y F l e x i b i l i t y & r e s p o n s i v e n e s s Based on an e v a l u a t i o n f o r m used by t h e D e p a r t m e n t o f P s y c h o l o g y a t t h e U n i v e r s i t y o f M i c h i g a n . -78-F a i r n e s s and i m p a r t i a l t y i n d e a l i n g s w i t h s t u d e n t s Feedback t o s t u d e n t s on t h e i r p e r f o r m a n c e F a i r n e s s o f t e s t s and g r a d e s F o l l o w s an O b j e c t i ves c l e a r o u t l i ne o f c o u r s e made E v a l u a t i o n C o u r s e o u t l i n i ng and s t r u c t u r i ng D i f f i c u l t y o f a s s i g n e d r e a d i ng Volume o f c o u r s e work Q u a l i t y o f t e x t books , e t c C o v e r a g e o f m a t e r i a l W a l l e r (1971) E x p l a i n i n g c o n c e p t s and -t h e o r i e s D e m o n s t r a t i n g ' t h e o r i e s Q u e s t i o n i n g u s e d s o f t h e o r i e s G u i d i n g s t u d e n t s i n t h e c l a s s O r d e r m a i n t a i n i n g i n c l a s s R e w a r d i n g t h e s t u d e n t and l e a r n i n g and c o m p l i a n c e T e a c h i ng s t y l e C u r r i c u l u m p l a n n i n g H o u s e k e e p i n g a c t i v i t i e s C o u r s e o u t l i n -i n g and s t r u c t u r i ng A s s i g n m e n t making T e s t i n g and e v a l u a t i o n E v a l u a t i o n -79-As can be s e e n , t h e B e h a v i o u r a l l y r a t e d i n s t r u m e n t a p p e a r s t o p o s s e s s c o n s i d e r a b l e c o n t e n t v a l i d i t y . The one m a j o r a s p e c t o f c o l l e g e i n s t r u c t i on* not i n c l u d e d i n i t was t h e i n s t r u c t o r ' s p e r s o n a l i t y c o r r e l a t e s , an a s p e c t i t was not meant t o m e a s u r e . A c o m p a r i s o n o f t h e c o n t e n t o f c u r r e n t l y u s e d SIR q u e s t i o n n a i r e s w i t h t h e B e h a v i o u r a l : i n s t r u m e n t i n d i c a t e s t h e a d v a n t a g e s o f t h e l a t t e r . The SIR q u e s t i o n n a i r e does n o t measure d i m e n s i o n s s u c h as " t e a c h i n g methods" and " f l e x i b i l i t y and r e s p o n s i v e -n e s s " t o any e x t e n t and i t s measurement o f o t h e r d i m e n s i o n s l i k e " t e a c h i n g s t y l e " , " e v a l u a t i o n " and " i n t e r a c t i o n o u t s i d e c l a s s " a p p e a r t o be much l e s s c o m p r e h e n s i v e t h a n t h e B e h a v i o u r a l s c a l e s . F o r e x a m p l e , i n t h e e n t i r e SIR q u e s t i o n n a i r e , t h e r e ace o n l y 2 q u e s t i o n s r e l a t e d t o " t e a c h i n g s t y l e " ( v i z . , " t o what e x t e n t d i d t h e i n s t r u c t o r use e x a m p l e s o r i l l u s t r a t i o n s t o h e l p c l a r i f y t h e m a t e r i a l ? " and "Was t h e i n s t r u c t o r open to o t h e r v i e w p o i n t s ? " ) and " e v a l u a t i o n " i s r e p r e s e n t e d by a s i n g l e q u e s t i o n ( " E x a m i n a t i o n s r e f l e c t e d t h e i m p o r t a n t a s p e c t s o f t h e c o u r s e " ) , j A l s o , o u t o f t h e 39 q u e s t i o n s i n SIR q u e s t i o n n a i r e , 15 q u e s t i o n s a r e n o t d i r e c t l y -80-r e l a t e d t o t h e i n s t r u c t o r ' s b e h a v i o u r . To c o n c l u d e , t h e c o n t e n t v a l i d i t y o f SIR q u e s t i o n n a i r e a p p e a r s t o be c o n s i d e r a b l y l o w e r t h a n d e s i r e d l e v e l s f o r t h e p u r p o s e o f t e a c h e r e v a l u a t i o n i n t h e F a c u l t y o f Commerce. 3. L e n i e n c y and C e n t r a l t e n d e n c y o f t h e i n s t r u m e n t : L e n i e n c y p r o b l e m s a p p e a r t o be m i n i m i z e d on t h i s i n s t r u m e n t . S t u d e n t r a t i n g s on t h e b e h a v i o u r a l s c a l e s were l e s s skewed t h a n t h e i r r a t i n g s on SIR q u e s t i o n n a i r e . D e t a i l s o f r a t i n g s by 10 s t u d e n t s o f t h e i r p r o f e s s o r s on t h e two i n s t r u m e n t s a r e g i v e n i n A p p e n d i x N. I t can be seen t h a t r a t i n g s by e a c h s t u d e n t on v a r i o u s d i m e n s i o n s o f t h e b e h a v i o u r a l s c a l e s had more v a r i a t i o n t h a n i n d i v i d u a l r a t i n g s on s e l e c t e d i t e m s o f SIR q u e s t i o n n a i r e ^ . t h u s i n d i c a t i n g l e s s l e n i e n c y f o r t h e B e h a v i o u r a l s c a l e s . Mean r a t i n g s by t h e e n t i r e sample o f s t u d e n t s (n=62) on v a r i o u s d i m e n s i o n s o f t h e b e h a v i o u r a l s c a l e s showed more s t a n d a r d d e v i a t i o n s t h a n mean r a t i n g s on s e l e c t e d i t e m s i n SIR q u e s t i o n n a i r e . T h i s s u g g e s t s t h a t t h e B e h a v i o u r a l s c a l e s a r e r e l a t i v e l y more f r e e f r o m c e n t r a l t e n d e n c y p r o b l e m s t h a n t h e SIR q u e s t i o n n a i r e . D e t a i l s o f t h i s a n a l y s i s a r e g i v e n i n A p p e n d i x N. -81-'4. D i m e n s i o n I n d e p e n d e n c e : - The i n t e r - d i m e n s i o n c o r r e l a t i o n o f t h e B e h a v i o u r a l s c a l e s does not seem t o be s i g n i f i c a n t l y d i f f e r e n t f r o m t h e i n t e r -i t e m c o r r e l a t i o n o f t h e SIR q u e s t i o n n a i r e . The d e t a i l s o f i n t e r d i m e n s i o n c o r r e l a t i o n o f t h e B e h a v i o u r a l i n s t r u m e n t a r e g i v e n below a l o n g w i t h t h e i n t e r - i t e m c o r r e l a t i o n s o f SIR q u e s t i o n n a i r e . TABLE 4.1 CORRELATION BETWEEN SCORES ON VARIOUS DIMENSIONS OF THE BEHAVIOURAL SCALE - Dim e n s i o n 1 2 3 .4 5 6 7 1 1. 0* D i 2 0. 171 1. 0> m e 3 0. 197 0. 176 1. 0 n s 4 0. 362 0. 448 0. 175 1.0 i i 5 0. 264 0. 375 0. 162 0.145 1. 0 0 n 6 0. 377 0. 019 0. 059 0.024 0. 033 1. 0 7 0. 347 0. 326 0. 050 0.563 0. 219 0. 140 1.0 -82-TABLE 4.2 CORRELATION BETWEEN SCORES ON SELECTED ITEMS IN S.I.R. QUESTIONNAIRE • • Item Mo, i n S. 1 [ /R ' Q u e s t i o n n a i re 1 2 4 13 r 19 35 I 1 l . d t 9 e 2 0.512 1.0 m.. 4 0.113 0.113 1.0 M o 12 0.264 0.378 -0.366 1.0 i 13 0.343 0.097 -0.014 0.332 1.0 n 19 -0.052 -0.031 -0.035 0.122 0.003 1.0 S I 35 0.317 0.470 -0.368 Ov-6'18 0.054 0.102 1.0 R W h i l e no f i n a l c o n c l u s i o n s a r e p o s s i b l e f r o m . such a l i m i t e d c o m p a r i s o n i t may be n o t e d t h a t t h e h i g h e s t c o r r e l a t i o n i n t h e c a s e o f B e h a v i o u r a l s c a l e s i s 0.563 whereas i n t h e c a s e o f SIR i t e m s , i t i s 0.618. The o t h e r c o r r e l a t i o n f i g u r e s do n o t i n d i c a t e any m a j o r d i f f e r e n c e s i n t h e e x t e n t o f d i m e n s i o n - i n d e p e n d e n c e o f t h e two i n s t r u m e n t s . However, f u r t h e r r e s e a r c h i s n e c e s s a r y b e f o r e coming t o a c o n c l u s i o n . The o t h e r i m p l i c a t i o n s o f t h e use o f t h e b e h a v i o u r a l s c a l e s a r e g i v e n i n t h e n e x t c h a p t e r . •83-CHAPTER -'V IMPLICATIONS OF THE STUDY I OTHER FINDINGS OF THE STUDY The p e r f o r m a n c e d i m e n s i o n s e m e r g i n g from t h e s t u d y measure f i f t e e n d i f f e r e n t f a c e t s o f a p r o f e s s o r ' s p e r f o r m a n c e : -E x t e n t o f c o u r s e o u t l i n i n g and s t r u c t u r i n g done by t h e p r o f e s s o r ; E x t e n t o f c o v e r a g e o f t h e s u b j e c t m a t t e r by him; A p p r o p r i a t e n e s s o f h i s t e a c h i n g s t y l e ; V a r i e t y and u s e f u l n e s s o f t e a c h i n g methods employed by him; S t r a t e g i e s o f s t u d e n t e v a l u a t i o n e m p l o y e d by t h e p r o f e s s o r ; N a t u r e o f h i s i n t e r a c t i o n s w i t h s t u d e n t s o u t s i d e t h e c l a s s ; D egree o f f l e x i b i l i t y and r e s p o n s i v e n e s s shown by him i n h i s i n t e r a c t i o n s w i t h s t u d e n t s ; as p e r c e i ved by h i s s t u d e n t s N a t u r e o f h i s i n t e r a c t i o n s w i t h c o l l e a g u e s ; H i s r e s e a r c h a c t i v i t i e s ; H i s i n t e r a c t i o n s w i t h s t u d e n t s o u t s i d e c l a s s . as p e r c e i ved by h i s c o l l eagiies -84-E x t e n t t o w h i c h he f a c i l i t a t e s a d m i n i s t r a -t i v e work f l ow ; H i s c o n c e r n f o r c o n t r o l l i n g e x p e n d i t u r e s ; as p e r c e i ved H i s t e n d e n c y to a d h e r e to p o l i c i e s o f by t h e t h e d i v i s i o n and t h e e n t i r e f a c u l t y ; a d m i n i s t r a -t i v e s t a f f E x t e n t to which he f a c i l i t a t e s s t u d e n t c o u n s e l l i n g and p r o v i d i n g f e e d b a c k t o s t a f f . I t i s hoped t h a t t h e above d i m e n s i o n s r e p r e s e n t t h e m a j o r a c t i v i t i e s o f a p r o f e s s o r o f B e h a v i o u r a l s c i e n c e s . The r a n k i n g o f t h e d i m e n s i o n s o f p e r f o r m a n c e by s t u d e n t s o f t h e r e t r a n s l a t i n g group r e f l e c t e d t h e r e l a t i v e i m p o r t a n c e o f t h e s e v e n c a t e g o r i e s as p e r c e i v e d by them a s : -C a t e g o r y o f B e h a v i o u r C o v e r a g e o f m a t e r i a l T e a c h i n g s t y l e F l e x i b i l i t y C o u r s e o u t l i n i n g and s t r u c t u r i n g E v a l u a t i o n T e a c h i ng methods I n t e r a c t i o n o u t s i d e Rank 1 2 3 4 5 6 7 -85-The emphasis shown by s t u d e n t s on c o v e r a g e o f m a t e r i a l by t h e t e a c h e r and h i s t e a c h i n g s t y l e ( r a n k 1 and 2 i n t h e above l i s t ) was somewhat e x p e c t e d c o n s i d e r i n g the f a c t t h a t a l a r g e p r o p o r t i o n ( a p p r o x i m a t e l y 55%) o f t h e t o t a l i n c i d e n t s c o l l e c t e d were c o n c e r n e d w i t h t h e s e d i m e n s i o n s . The i n c i d e n t s g i v e n by s t u d e n t s were a n a l y s e d t o see w h e t h e r any d i f f e r e n c e s e x i s t between males and f e m a l e s and g r a d u a t e s and u n d e r g r a d u a t e s i n t h e i r r e l a t i v e e mphasis on v a r i o u s d i m e n s i o n s o f t e a c h e r p e r f o r m a n c e . The r e s u l t s o f t h e a n a l y s i s a r e g i v e n i n T a b l e 5.1 i n t h e n e x t page. As can be s e e n , t h e emphasis o f u n d e r g r a d u a t e s t u d e n t s was on ' T e a c h i n g s t y l e 1 o f t h e p r o f e s s o r , f o l l o w e d by o t h e r d i m e n s i o n s l i k e ' T e a c h i n g methods' and 'Coverage o f m a t e r i a l ' . The g r a d u a t e s t u d e n t s a l s o c o n s i d e r e d ' T e a c h i n g s t y l e o f t h e p r o f e s s o r as t h e most i m p o r t a n t p e r f o r m a n c e d i m e n s i o n . However, the n e x t i m p o r t a n t d i m e n s i o n , i n t h e i r o p i n i o n was ' I n t e r a c t i o n o u t s i d e ' ; b o t h 'Coverage o f m a t e r i a l ' a n d ' e v a l u a t i o n ' r a n k e d t h i r d i n o r d e r o f i m p o r t a n c e . T e a c h i n g s t y l e o f t h e p r o f e s s o r was c o n s i d e r e d as t h e most i m p o r t a n t d i m e n s i o n o f p e r f o r m a n c e by -86-TABLE 5.1 ANALYSIS OF CRITICAL INCIDENTS SEX-WISE AND CLASS LEVEL WISE D i m e n s i o n t o which t h e i n c i d e n t s b e l o n g Number o f c r i t i c a l i n c i d e n t s U n d e r g r a d u a t e G r a d u a t e T o t a l Male 'Female Mai e Female C o u r s e o u t l i n i n g & s t r u c t u r i n g 39 14 5 14 72 C o v e r a g e o f M a t e r i a l 73 11 15 7 106 T e a c h i n g s t y l e 316 87 48 25 476 T e a c h i n g methods 106 13 13 6 138 I n t e r a c t i o n o u t s i d e 28 19 20 10 77 E v a l u a t i on 38 23 10 . 12 83 F l e x i b i l i t y & r e s p o n s i v e n e s s 52 8 5 4 69 1021 b o t h f e m a l e and male s t u d e n t s ( g r a d u a t e s and u n d e r g r a d -u a t e s t a k e n t o g e t h e r ) . F o r male s t u d e n t s , ' T e a c h i n g m ethods', 'Coverage o f m a t e r i a l ' and ' F l e x i b i l i t y and r e s p o n s i v e n e s s ' came n e x t i n o r d e r o f i m p o r t a n c e . Female s t u d e n t s however, e m p h a s i z e d on ' E v a l u a t i o n ' , ' I n t e r a c t i o n o u t s i d e ' and 'Course o u t l i n i n g and s t r u c t u r i n g ' i n t h a t o r d e r . , •.•••0'--87-The above f i n d i n g s s h o u l d p r o v i d e some u s e f u l g u i d e l i n e s t o f a c u l t y members i n a d a p t i n g t h e i r b e h a v i o u r t o s u i t t h e r e q u i r e m e n t s / p r i o r i t i e s o f t h e i r s t u d e n t s . II SOME USES OF THE STUDY The b e h a v i o u r a l l y a n c h o r e d r a t i n g s c a l e s d e v e l o p e d i n t h i s s t u d y and d i s c u s s e d i n t h e p r e v i o u s c h a p t e r i s e x p e c t e d t o be a r e l i a b l e and v a l i d measure o f t e a c h i n g e f f e c t i v e n e s s i n t h e IRM D i v i s i o n o f t h e F a c u l t y o f Commerce. I t i s f o u n d t h a t t h e i n s t r u m e n t does p o s s e s s c o n s i d e r a b l e f a c e and c o n t e n t v a l i d i t y . The c o n s t r u c t v a l i d i t y o f t h e i n s t r u m e n t i s y e t t o be e s t a b l i s h e d . Two methods s u g g e s t e d h e r e f o r t h i s p u r p o s e a r e : -1) The s c o r e s on t h i s i n s t r u m e n t can be compared w i t h s c o r e s on a s i m i 1 a r s e a l e -measuri ng t e a c h i n g e f f e c t i v e n e s s i n B e h a v i o u r a l s c i e n c e s u b j e c t s . The s c a l e used f o r c o m p a r i s o n p u r p o s e s s h o u l d , to b e g i n w i t h , have h i g h c o n s t r u c t v a l i d i t y . 2) The s c o r e s on t h i s i n s t r u m e n t c a n be compared w i t h s c o r e s on o t h e r c r i t e r i a o f t e a c h i n g -88-e f f e c t i v e n e s s l i k e p e r f o r m a n c e t e s t s , c l a s s room o b s e r v a t i o n s and e n v i r o n m e n t a l a s s e s s m e n t s . A t e a c h e r who i s r a t e d as h i g h l y e.f f e c t i ve.v.on t h i s i n s t r u m e n t s h o u l d a l s o r e c e i v e h i g h s c o r e s on o t h e r c r i t e r i a o f t e a c h e r e f f e c t i v e n e s s . The use o f s t u d e n t s , p e e r s , and a d m i n i s t r a t i v e s r t a f f o f t h e F a c u l t y o f Commerce ( i n U.B.C) t h r o u g h o u t t h e d e v e l o p m e n t o f t h e p r e s e n t i n s t r u m e n t r e n d e r s i t e x t r e m e l y r e l e v a n t and " u s e f u l f o r t h i s d i v i s i o n . However, some o f t h e f i n d i n g s o f t h i s s t u d y may, a f t e r n e c e s s a r y m o d i f i c a t i o n s , be g e n e r a l i z e d to o t h e r d i v i s i o n s o f t h e f a c u l t y and to o t h e r u n i v e r s i t i e s . The knowledge o f t h e needs and r e q u i r e m e n t s o f h i s s t u d e n t s and p e e r s s h o u l d h e l p each f a c u l t y member t o improve h i s e f f e c t i v e n e s s b o t h i n s i d e and o u t s i d e t h e c l a s s room. The b e h a v i o u r a l r a t i n g s o f i n s t r u c t o r s by s t u d e n t s and p e e r s can i n d i c a t e t h e / t r a i n i n g needs o f i n d i v i d u a l i n s t r u c t o r s . A c a r e f u l r e v i e w o f t h e s t u d e n t and p e e r needs can a l s o f a c i l i t a t e improvement i n t h e c r i t e r i a f o r s e l e c t i o n and p r o m o t i o n o f f a c u l t y members i n t h e d i v i s i o n . -89-The f a c t t h a t t h e i t e m s i n t h e b e h a v i o u r a l i n s t r u m e n t a r e m a n i f e s t and s p e c i f i c s h o u l d e n a b l e t h e s t u d e n t s t o p r o v i d e an o b j e c t i v e e v a l u a t i o n o f t h e i r t e a c h e r s . The i n s t r u m e n t seems t o be r e l a t i v e l y f r e e from p r o b l e m s o f h a l o , c e n t r a l t e n d e n c y and l e n i e n c y w h i c h makes t h e e n t i r e ! e v a l u a t i o n p r o c e s s more r e l i a b l e and v a l i d t h a n t h e p r e s e n t method u s i n g t h e SIR i n s t r u m e n t . The b e h a v i o u r a l i n s t r u m e n t may a l s o p r o v i d e t h e F a c u l t y w i t h i n d i r e c t measures o f t h e a t t i t u d e s ^ o f t h e p r o f e s s o r s . W h i l e t h e r e i s much c o n t r o v e r s y s u r r o u n d i n g t h e s p e c i f i c r e l a t i o n between a t t i t u d e s and b e h a v i o u r , i t has been d e m o n s t r a t e d i n many s t u d i e s t h a t t h e p r e d i c t i o n o f a s e t o f a t t i t u d e s o f an i n d i v i d u a l i s p o s s i b l e by o b s e r v a t i o n o f d i f f e r e n t p a t t e r n s o f h i s b e h a v i o u r w i t h r e s p e c t t o o t h e r i n d i v i d u a l s and o b j e c t s ( F i s h b e i n , 1971; T i t t l e and H i l l , 1 967). As s u c h , t h e p a t t e r n s o f b e h a v i o u r o f t h e p r o f e s s o r s may a c t as i n d i c a t o r s o t t h e p r e v a i l i n g f a c u l t y a t t i t u d e s . F i n a l l y , t h e p r e s e n t i n s t r u m e n t and t h e r e l a t e d d a t a can s e r v e as b e g i n n i ng: p o i n t s from w h i c h f u r t h e r r e s e a r c h may be c o n d u c t e d i n t o t e a c h i n g . : -90-e f f e c t i v e n e s s and o t h e r a s s o c i a t e d a r e a s . A u s e f u l f i r s t e x e r c i s e w i l l be to e x t e n d t h e p r e s e n t s t u d y to t h e e n t i r e F a c u l t y o f Commerce. E f f o r t s t o c o r r e l a t e s p e c i f i c p e r s o n a l i t y t r a i t s and a t t i t u d e s o f t h e p r o f e s s o r s w i t h v a r i o u s b e h a v i o u r s s h o u l d a l s o be;;, o f some v a l u e . Once v a l i d r e l a t i o n s h i p s a r e e s t a b l i s h e d t h e s e t r a i t s and a t t i t u d e s can s e r v e as good p r e d i c t o r s o f t e a c h e r e f f e c t i v e n e s s i n s e l e c -t i o n and c o u n s e l l i n g o f p r o f e s s o r s and i n c u r r i c u l u m p l a n n i n g . F u r t h e r r e s e a r c h i n t o c l a s s room c l i m a t e , t e a c h e r - s t u d e n t r e l a t i o n s h i p s and s o c i a l p s y c h o l o g y o f t e a c h i n g may a l s o be c o n s i d e r a b l y a i d e d by t h e d a t a g e n e r a t e d i n t h i s s t u d y . I l l SOME LIMITATIONS OF 'BARI' RESEARCH As i n t h e c a s e o f any i n s t r u m e n t d e s i g n e d to e v a l u a t e t h e p e r f o r m a n c e o f i n d i v i d u a l s , t h e p r o c e d u r e o f d e v e l o p i n g BARI i s b e s e t w i t h some p r a c t i c a l p r o b l e m s . T h e s e p r o b l e m s a r e d i s c u s s e d below:-i ) Lack o f c o m p r e h e n s i v e n e s s o f i n c i d e n t s c o l l e c t e d : The v a l i d i t y and p r a c t i c a l u s e f u l n e s s o f t h e i n s t r u m e n t i s c o n s i d e r a b l y i n f l u e n c e d by t h e domain o f b e h a v i o u r s i n d i c a t e d by t h e ' c r i t i c a l i n c i d e n t s ' . -91-T h i s makes t h e c o l l e c t i o n o f c r i t i c a l i n c i d e n t s a c r u c i a l l y i m p o r t a n t s t a g e i n t h e d e v e l o p m e n t o f BABI . O f t e n , c o l l e c t i o n o f c r i t i c a l i n c i d e n t s i s done t h r o u g h i n t e r v i e w s . However, t h i s method i s somewhat s u s c e p t i b l e t o p r o b l e m s a r i s i n g f r o m p e r c e p t u a l s e t , b i a s , and s o c i a l d e s i r a b i l i t y n eeds o f t h e i n t e r v i e w e r and t h e i n t e r v i e w e e . Some o f t h e s e p r o b l e m s can be a v o i d e d by t r a i n i n g t h e i n t e r v i e w e r and making t h e i n t e r v i e w s more s t r u c t u r e d A n o t h e r method t o overcome t h e b i a s and p e r c e p t u a l s e t o f t h e i n t e r v i e w e r may be to ask t h e s u b j e c t s t o w r i t e down t h e i n c i d e n t s . T h i s method, i f i t f ;; g u a r a n t e e s a n o n y m i t y o f t h e r e s p o n d e n t s , can a l s o s o l v e many o f t h e p r o b l e m s e m e r g i n g f r o m s o c i a l d e s i r a b i l i t y needs of. t h e s u b j e c t s . F o r e x a m p l e , a s t u d e n t may not d i s c l o s e a p a r t i c u l a r i n c i d e n t t o t h e i n t e r v i e w e r i f s u c h d i s c l o s i n g e m b a r a s s e s him i n any manner. The same s t u d e n t may be p r e p a r e d t o w r i t e down t h e i n c i d e n t i f h i s a n o n y m i t y i s e n s u r e d by t h e r e s e a r c h e r . However, i n w r i t i n g t h e i n c i d e n t s t h e ' r e c e n c y e f f e c t ' o f t e n o p e r a t e s ( t h a t i s , t h e r e s p o n d e n t w r i t e s down o n l y t h e r e c e n t e v e n t s i n h i s memory). U n l e s s n e c e s s a r y a c t i o n s a r e t a k e n , t h i s may l a t e r p r o v e t o be t h e s o u r c e o f m a j o r p r o b l e m s a s s o c i a t e d w i t h t h e c o n t e n t v a l i d i t y o f the i n s t r u m e n t . 2) L o s s o f i n c i d e n t s : In d e v e l o p i n g B e h a v i o u r a l l y A n c h o r e d R a t i n g I n s t r u m e n t s , many i n c i d e n t s a r e l o s t i n two s t a g e s :-i ) i n c i d e n t s w h i c h a r e not r e t r a n s l a t e d t o t h e same c a t e g o r i e s o f b e h a v i o u r by a p r e d e t e r m i n e d p r o p o r t i o n o f t h e sample s i z e a r e d e l e t e d as ambi-guous. i i ) A f t e r s c a l i n g o f t h e i n c i d e n t s f o r t h e i r / r e l a t i v e e f f e c t i v e n e s s , i t e m s w i t h r a t i n g s h a v i n g s t a n d a r d d e v i a t i o n v a l u e s e x c e e d i n g a p r e - d e t e r m i n e d f i g u r e ( f o r example, 1.50) a r e e l i m i n a t e d as ambi-guous. P r o b l e m s a r i s e when some o f t h o s e i n c i d e n t s w h i c h a r e e l i m i n a t e d as ambiguous s t i l l r e p r e s e n t c r u c i a l b e h a v i o u r s . The l i k e l i h o o d o f i m p o r t a n t i n c i d e n t s b e i n g e l i m i n a t e d as ambiguous i n c r e a s e s as t h e p e r f o r m a n c e d i m e n s i o n s become l e s s and l e s s i n d e p e n d e n t o f each o t h e r . In su c h c a s e s , t h e same i n c i d e n t ( s ) may be a l l o c a t e d t o two o r more d i m e n s i o n s ( c a t e g o r i e s ) o f p e r f o r m a n c e by members o f t h e -93-r e - t r a n s l a t i n g g r o u p , t h u s n e c e s s i t a t i n g e l i m i n a t i o n o f t h e i n c i d e n t ( s ) . Some p r o b l e m s a l s o a r i s e when an i t e m e l i m i n a t e d on a c c o u n t o f i t s h i g h s t a n d a r d d e v i a t i o n v a l u e s t i l l a p p e a r s t o be q u i t e r e l e v a n t to t h e j o b . To q u o t e one example from o u r s t u d y , t h e ; i n c i d e n t " P r o f e s s o r p l a y e d t h r e e d i f f e r e n t r o l e s i n a s k i t w h i c h was aimed t o e x p l a i n group c o n f l i c t . " had t h e f o l l o w i n g e f f e c t i v e n e s s r a t i n g s on t h e 9 p o i n t s c a l e . 5 , 6 , 9 and 9. The s t a n d a r d d e v i a t i o n v a l u e o f t h i s r a t i n g was a p p r o x i m a t e l y 1.54 wh i c h was g r e a t e r t h a n o u r c u t - o f f p o i n t o f 1.50 and hence t h i s i t e m was d e l e t e d . However, t h e behaviour..shown i n t h e i n c i d e n t i t s e l f seems to be i m p o r t a n t as i n d i c a t e d by t h e r a t i n g s g i v e n t o i t . The e x c l u s i o n o f s u c h an e f f e c t i v e i t e m may d i m i n i s h t h e c o n t e n t v a l i d i t y o f t h e i n s t r u m e n t . T h a t t h e s t a n d a r d d e v i a t i o n c r i t e r i a may d i s p r o p o r -t i o n a t e l y e x c l u d e n e u t r a l ( as a g a i n s t e x t r e m e ) i n c i d e n t s has been p o i n t e d o u t by D e c o t i i s ( 1974) . 3) A m b i g u i t y o f i n c l u d e d i n c i d e n t s : - A p r o b l e m w h i c h i s p e r h a p s a l m o s t c o n t r a r y to t h e p r o b l e m d i s c u s s e d above i s t h e i n c l u s i o n o f ambiguous i n c i d e n t In many p a s t r e s e a r c h e s on BARI t h e c u t - o f f !; c r i t e r i o n used a t t h e r e - t r a n s l a t i o n s t a g e has been q u i t e n o n - s t r i n g e n t . Thus H a r a r i and Zedeck (1973) us e d a mere 60% agreement ( o f r e - t r a n s l a t o r s ) as th e c r i t e r i o n , w h i l e F o g l i e t . a l (1971) needed o n l y s i m p l e m a j o r i t y a greement f o r a c c e p t i n g i t e m ? . C l e a r l y , s u c h r e - t r a n s l a t i o n c r i t e r i a c a n c a u s e p r o b l e m s o f a m b i g u i t y i n t h e f i n a l s c a l e . The s t a n d a r d d e v i a t i o n c r i t e r i a a l s o have i n t h e p a s t , been t y p i c a l l y s e t a t s u c h a l e v e l t h a t i n c i d e n t s r e p r e s e n t i n g a s u b s t a n t i a l amount o f d i s a g r e e m e n t (as t o t h e l e v e l o f e f f e c t i v e n e s s t h e y r e p r e s e n t ) a r e r e t a i n e d i n t h e f i n a l i n s t r u m e n t (Schwab, Heneman and D e c o t i i s , 1975). IV TOWARDS SOME SOLUTIONS Many o f t h e p r o b l e m s c o n c e r n e d w i t h t h e c o l 1 e c t i o f c r i t i c a l i n c i d e n t s may be overcome by c a r e f u l p l a n n i n g . The use o f a s i n g l e i n t e r v i e w e r o r p e r s o n s w i t h s i m i l a r a t t r i b u t e s and a t t i t u d e s c a n m i n i m i s e many o f t h e i n t e r - i n t e r v i e w e r v a r i a n c e s i n d a t a c o l l e c t i o n . G i v i n g t h e s u b j e c t s s u f f i c i e n t t i m e to r e s p o n d , o f t e n f a c i l i t a t e s c o l l e c t i o n o f many -95-good c r i t i c a l i n c i d e n t s . The p r o b l e m s c a u s e d by ' r e c e n c y ' e f f e c t s can be e l i m i n a t e d to a c o n s i d e r a b l e d e g r e e i f t h e r e s e a r c h e r c l a r i f i e s t h e p u r p o s e o f t h e s t u d y t o t h e s u b j e c t s and g i v e s them many examples o f - ' c r i t i c a l ' i n c i d e n t s i n s i m i l a r o c c u p a t i o n s . A s s u r i n g t h e s u b j e c t s o f a n o n y m i t y and p r o m i s i n g f e e d b a c k t o them about t h e f i n a l r e s u l t s o f t h e s t u d y may g e n e r a t e good r e s p o n s e f r o m t h e r e s p o n d e n t s . T h e r e i s a p p a r a n t l y no s i n g l e e f f e c t i v e way t o e n s u r e t h a t 'key' i n c i d e n t s a r e not l o s t i n t h e r e - t r a n s l a t i o n and s c a l i n g s t a g e s o f t h e s t u d y . One p o s s i b l e s o l u t i o n i s t o i d e n t i f y i n c i d e n t s w h i c h have been f o u n d t o be s i g n i f i c a n t i n p a s t r e s e a r c h and have them r e - c l a s s i f i e d and s c a l e d by a new g r o u p o f r e s p o n d e n t s . T h i s p r o c e d u r e g i v e t h e 'key' i n c i d e n t s a s e c o n d c h a n c e f o r p o s s i b l e i n c l u s i o n i n t h e f i n a l s c a l e . The s u b j e c t s who a r e d o i n g t h e r e t r a n s l a t i o n and s c a l i n g may a l s o be a s k e d t o g i v e t h e i r r e a s o n s f o r a) c a t e g o r i s i n g i n c i d e n t s i n t o p a r t i c u l a r d i m e n s i o n s and b) g i v i n g t h e p a r t i c u l a r s c a l e v a l u e s ( d e n o t i n g d e g r e e o f e f f e c t i v e b e h a v i o u r ) to t h e v a r i o u s i n c i d e n t s . Such an e x e r c i s e w i l l be u s e f u l i n two ways:-1) I t w i l l a c t as a c h e c k on t h e r e l i a b i l i t y o f e a r l i e r c l a s s i f i c a t i o n s and r a t i n g s ; and -96-2) I t w i l l p r o v i d e some f r e s h g u i d e l i n e s t o tihe r e s e a r c h e r f o r t h e i n c l u s i o n / e x c l u s i o n o f t h e i t e m s and the need f o r r e - d e f i n i n g t h e c a t e g o r i e s . The l o s s o f many i n c i d e n t s a t t h e r e - t r a n s l a t i o n s t a g e may be a v o i d e d by c l e a r and unambiguous d e f i n i t i o n o f d i m e n s i o n s . As f a r as p o s s i b l e , t h e v a r i o u s d i m e n s i o n s o f p e r f o r m a n c e s h o u l d be i n d e p e n -d e n t o f each o t h e r . In c o n c l u s i o n , we f e e l t h a t many o f t h e p r o b l e m s w i t h BARI a r e n e i t h e r i n h e r e n t i n t h e i n s t r u m e n t n o r i n s u r m o u n t a b l e . On most o c c a s i o n s t h e s e p r o b l e m s can be a v o i d e d by d i l i g e n t p l a n n i n g . -97-CHAPTER VI CONCLUSION The g u i d i n g f o r c e f o r t h i s s t u d y was an : i n t e r e s t i n t e a c h i n g and s p e c i f i c a l l y i n t h e d e s i g n ' o f an i n s t r u m e n t w h i c h i s d e m o n s t r a b l y v a l i d and u s e f u l i n m e a s u r i n g t e a c h i n g e f f e c t i v e n e s s . The e v i d e n c e so f a r has been g e n e r a l l y s u p p o r t i v e o f t h e a c c o m p l i s h m e n t o f t h e o r i g i n a l i n t e n t - 1 The i n s t r u m e n t d e s i g n e d does seem, on an i n i t i a l e x a m i n a -t i o n , t o be q u i t e r e l i a b l e and c a p a b l e o f d i s c r i m i -n a t i n g between e f f e c t i v e and i n e f f e c t i v e t e a c h e r s . A n o t h e r o b j e c t i v e o f t h i s s t u d y was t o p r o v i d e t h e f a c u l t y members and t h e i r s u p e r i o r s w i t h an i n s t r u m e n t w h i c h t e l l s them a b o u t r e l e v a n t a s p e c t s o f a c o u r s e and t h e b e h a v i o u r o f t h e i n s t r u c t o r i n «rn. c h a r g e o f t h e c o u r s e . The emphasis i n t h e e n t i r e s t u d y was on t h e g e n e r a t i o n o f d e s c r i p t i v e f e e d b a c k i n f o r m a t i o n to t h e f a c u l t y member w h i c h would a i d him i n t h e i d e n t i f i c a t i o n and r e m e d i a l o f h i s w e a k n e s s e s . T h i s was due to t h e f i r m c o n v i c t i o n o f t h e a u t h o r t h a t a p e r s o n ' s knowledge o f what... c o n s t i t u t e s ' e f f e c t i v e p e r f o r m a n c e ' i s even more c r u c i a l t h a n t h e r e c e i p t o f g l o b a l r a t i n g s o f h i s -98-p e r f o r m a n c e . A m a j o r p r o b l e m i n any s t u d y r e l a t e d t o t e a c h i n g e f f e c t i v e n e s s i s t h e i d e n t i f i c a t i o n o f a v a l i d c r i t e r i o n o f e f f e c t i v e n e s s . We hope t h a t i n t h e p r e s e n t s t u d y s u f f i c i e n t e v i d e n c e has been p r o v i d e d i n s u p p o r t o f t h e use o f s t u d e n t and p e e r r a t i n g s as good i n d i c a t o r s o f t e a c h i n g e f f e c t i v e n e s s . The p r o c e s s o f e d u c a t i o n not o n l y i m p r o v e s t h e knowledge o f t h e s t u d e n t but a l s o c h a n g e s h i s c h a r a c t e r and way o f l i f e . In t h i s s e n s e t h e s t u d e n t u n d e r g o e s a f u n d a m e n t a l and t h o r o u g h change p r o c e s s w h i l e a t t h e u n i v e r s i t y . In many c a s e s t h e c h a n g e s i n t h e s e l f c o n c e p t o f t h e s t u d e n t i s l i n k e d t o t h e b e h a v i o u r and a t t i t u d e s o f t h e c: t e a c h e r s u n d e r whom he s t u d i e s . Hence, t h e n a t u r e o f r e l a t i o n s h i p between t e a c h e r . a n d s t u d e n t and t h e e v e n t s t h a t happen d u r i n g t h e s t u d e n t ' s s t a y i n t h e campus a r e key f a c t o r s t h a t shape th e s t u d e n t ' s l a t e r b e h a v i o u r . Thus an exami n a t i on: of; i ..• t h e : ; b e h a v i c u r o f t h e t e a c h e r r s i s r e l e v a n t f o r t h e p u r p o s e o f a s s e s s i n g t h e i m p a c t o f e d u c a t i o n on t h e s t u d e n t s . I t i s t h e s e s t u d e n t s who l a t e r become the f u l c r u m o f t h e e n t i r e s o c i a l change p r o c e s s . A s t u d y such as t h i s , h e n c e , may be u u s e f u l f o r s o c i e t y as a whole. -99-BIBLIOGRAPHY Adams, J . The E v o l u t i o n o f E d u c a t i o n a l T h e o r y . M a c w f i l l a n and Co.. L t d . , L ondon, 1915. A n d e r s s o n , B. and N i l s s o n , S. S t u d i e s i n t h e r e l i a b i l i t y and v a l i d i t y o f t h e c r i t i c a l i n c i d e n t t e c h n i q u e . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1964, 48, 6, 398-403. A n t h o n y , B.M. A new a p p r o a c h t o m e r i t r a t i n g o f t e a c h e r s . A d m i n i s t r a t o r ' s Note book. 1968, 17, 1, 1-4. A r v e y , R.D. and H o y l e J.C. A Guttman a p p r o a c h t o t h e d e v e l o p m e n t o f b e h a v i o u r a l ' ! y b a s e d r a t i n g s c a l e s f o r s y s t e m s a n a l y s t s and p r o g r a m m e r s / a n a l y s t s . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1974, 59, 61-68. Ashmus, M. and H a i g h , G. Some f a c t o r s w h i c h may be a s s o c i a t e d w i t h s t u d e n t c h o i c e between d i r e c t i v e and n o n - d i r e c t i v e c l a s s e s . A m e r i c a n P s y c h o l o g i s t 1952, 7,P247. As t i n , A.W. and H o l l a n d , J . L . The e n v i r o n m e n t a l a s s e s s m e n t t e c h n i q u e : A way t o measure c o l l e g e e n v i r o n m e n t s . J o u r n a l o f E d u c a t i o n a l P s y c h o l o g y . 1961, 52, 308-316. A u s u b e l , D.P. L e a r n i n g t h e o r y and c l a s s room  p r a c t i c e . The O n t a r i o I n s t i t u t e f o r s t u d i e s i n E d u c a t i o n . B u l l e t i n No. 1, T o r o n t o , 1967. Baba, V.V. and A c e , M.E. L e a d e r B e h a v i o u r s i n t h e U n i v e r s i t y c l a s s room. P r o c e e d i n g s o f A.I.D.S. c o n f e r e n c e , Las V e g a s , March 1975. B a g l e y , W.C. An e s s e n t i a l i s t ' s p l a t f o r m f o r t h e advancement o f A m e r i c a n e d u c a t i o n . E d u c a t i o n a l  A d m i n i s t r a t i o n and S u p e r v i s i o n . 1938, 24, A p r i 1 , 241-256. B a r r e t t , R.S. P e r f o r m a n c e R a t i n g . S c i e n c e R e s e a r c h A s s o c i a t e s , C h i c a g o , 1966. -100-B l o o d , M.R. S p i n - o f f s f r o m B e h a v i o u r a l E x p e c t a t i o n S c a l e p r o c e d u r e s . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1974, 59, 4, 513-515. Borman, W.C. The r a t i n g o f i n d i v i d u a l s i n o r g a n i z a -t i o n s : An a l t e r n a t e a p p r o a c h . O r g a n i z a t i o n a l  B e h a v i o u r and Hyman P e r f o r m a n c e , 1974,12,1057124. Borman, W.C. and D u n n e t t e , M.D. B e h a v i o u r b a s e d V e r s u s T r a i t o r i e n t e d p e r f o r m a n c e r a t i n g s : An e m p i r i c a l s t u d y . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1975 , 60, 5 ,,561-565. Borman, W.C. and V a l l o n , W.R. A view o f what can happen when b e h a v i o u r a l e x p e c t a t i o n s c a l e s a r e d e v e l o p e d i n one s e t t i n g and used i n a n o t h e r . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1974, 59 ,2, 197-201. B r a d f o r d , L.P. D e v e l o p i n g p o t e n t i a l i t i e s t h r o u g h c l a s s g r o u p s . Human F o r c e s i n T e a c h i n g and L e a r n i n g . L e l a n d P. B r a d f o r d (Ed) W a s h i n g t o n , D.C. : N a t i o n a l T r a i n i n g L a b o r a t o r y , N a t i o n a l E d u c a t i o n A s s o c i a t i o n . 1961. B r a u n s t e i n , D.N. and B e n s t o n , G.J. S t u d e n t and Depa r t m e n t c h a i r m a n v i e w s o f t h e p e r f o r m a n c e o f U n i v e r s i t y p r o f e s s o r s . J o u r n a l o f A p p l i e d P s y c h o l o g y . -1973, 58, 2, 244-249. B r o g d e n , J . E . and T a y l o r , E.D. The d o l l a r c r i t e r i o n : A p p l y i n g t h e C o s t a c c o u n t i n g c o n c e p t t o c r i t e r i o n c o n s t r u c t i o n . P e r s o n n e l P s y c h o l o g y . 1950, 3, 133-154. Brown, G. Human t e a c h i n g f o r Human l e a r n i n g . V i k i n g P r e s s , 1971. B r u b a c h e r , J . S . Modern p h i l o s o p h i e s o f e d u c a t i o n . F i r s t e d i t i o n , McGraw H i l l book Co. I n c . , New Y o r k , 1939, B r u n e r , J . , Goodnow, J . J . and A u s t i n , G.A. A s t u d y o f t h i n k i n g . S c i e n c e E d i t i o n s I n c . , New Y o r k , 1967. B u r k e , E. R e f l e c t i o n s on t h e r e v o l u t i o n i n F r a n c e . L i b e r a l A r t s P r e s s , Bobbs - M e r r i l l Co., I n d i a n a p o l i s , 1955. B u r n a s k a , R.F. and H o l l m a n n , T.D. An e m p i r i c a l c o m p a r i s o n o f t h e r e l a t i v e e f f e c t s o f r a t e r r e s p o n s e b i a s e s on t h r e e r a t i n g s c a l e f o r m a t s . J o u r n a l o f  A p p l i e d P s y c h o l o g y . • 1974, 59 , 307-312. -101-C a m p b e l l , J . P . , D u n n e t t e , M.D., A r v e y , R.D. and H e l l e r v i k , L.V. The d e v e l o p m e n t and e v a l u a t i o n o f B e h a v i o u r a l l y Based R a t i n g S c a l e s . J o u r n a l o f  A p p l i e d P s y c h o l o g y , 1973, 57, 1, 15-22. C a m p b e l l , J.P. D u n n e t t e , M.D. , - L a w l e r I I I E.E. and Weick J r . , K.E. M a n a g e r i a l b e h a v i o u r , p e r f o r m a n c e  and e f f e c t i v e n e s s . McGraw H i l l Book Co., New Y o r k 1970. Campion, J . E . Work s a m p l i n g f o r p e r s o n n e l s e l e c t i o n . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1972, 56, 40-44. C e n t r a , J.A. The S t u d e n t I n s t r u c t i o n a l R e p o r t :  i t s d e v e l o p m e n t and u s e s . S.I.R. R e p o r t # 1, E.T.S. C o l l e g e and U n i v e r s i t y p r o g r a m s , P r i n c e t o n , New J e r s e y 1972. C e n t r a , J . A : The S t u d e n t I n s t r u c t i o n a l R e p o r t . S.I.R. R e p o r t # 3, E.T.S C o l l e g e and u n i v e r s i t y p r o g r a m s , P r i n c e t o n , New J e r s e y , 1973. C l a r k , E . J . The mental h e a l t h o f e l e m e n t a r y s c h o o l  t e a c h e r s as measured by G u i l d f o r d - M a r t i n P e r s o n a l i t y  B a t t e r y . P a p e r r e a d a t N a t i o n a l C o u n c i l on Measurements u s e d i n e d u c a t i o n , A t l a n t i c C i t y , March 1950. C o a t s , W.D. and S w i e r e n g a , L. S t u d e n t p e r c e p t i o n s o f t e a c h e r s : a f a c t o r a n a l y t i c s t u d y . The J o u r n a l  o f E d u c a t i o n a l R e s e a r c h . 1972, Volume 65, No. 8, A p r i l , 357-360. C o s t i n , F. , G r e e n o u g h , W.T. and Menges, R.J. S t u d e n t r a t i n g s o f c o l l e g e t e a c h i n g : R e l i a b i l i t y , v a l i d i t y and u s e f u l n e s s . Review o f E d u c a t i o n a l  R e s e a r c h . 1971, Volume 41, 511-535. Cummings, L.L. and Schwab, D.P. P e r f o r m a n c e i n  O r g a n i z a t i o n s : D e t e r m i n a n t s and a p p r a i s a l . S c o t t , Foresman and Co., G l e n v i e w , I l l i n o i s , 1973. D e c o t i i s , T.A. The d e v e l o p m e n t and e v a l u a t i o n o f  j j e h a v i o u r a l l y a n c h o r e d r a t i n g s c a l e s f o r t h e i n b n f  r a d i o p a t r o l m a n . U n p u b l i s h e d Ph.D T h e s i s , U n i v e r s i t y o f W i s c o n s i n - M a d i s o n , W i s c o n s i n , 1974. Demi a s h k e v i c h , M. An i n t r o d u c t i o n to t h e p h i l o s o p h y  e d u c a t i on. A m e r i c a n Book Co., New Y o r k , 1935. -102-Dewey, J . How we t h i nk. B o s t o n ; H e a t h , 1910. Dewey, J . Democracy and E d u c a t i o n . The M a c M i l l a n Company, New Y o r k , 1916. D i c k i n s o n , T .L. and T i c e , T.E. A m u l t i t r a i t - -m u l t i m e t h o d a n a l y s i s o f s c a l e s d e v e l o p e d by r e - t r a n s l a t i o n . O r g a n i z a t i o n a l B e h a v i o u r and  Human P e r f o r m a n c e . 1973, 9, 421-438. Dodge, D.A. R e t u r n s t o i n v e s t m e n t i n u n i v e r s i t y  t r a i n i n g . Queen's U n i v e r s i t y I n d u s t r i a l Re 1 a t i o n s C e n t r e , K i n g s t o n , O n t a r i o , 1973. Domas, S . J . and T i e d e m a n , D. T e a c h e r c o m p e t e n c e : An a n n o t a t e d b i b l i o g r a p h y . J o u r n a l o f e x p e r i m e n t a l  e d u c a t i o n . 1950, 19, 101-218. D u n n e t t e , M.D. A note on t h e c r i t e r i o n . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1963, 47, 251-254. D u r f l i n g e r , G.W. P e r s o n a l i t y c o r r e l a t e s o f s u c c e s s i n s t u d e n t t e a c h i n g . E d u c a t i o n a l and P s y c h o l o g i c a l  measurement. 1963, 23, 383-390. F e s t i n g e r , L.A. A t h e o r y o f s o c i a l c o m p a r i s o n p r o c e s s , Human R e l a t i o n s . 1954, 7, 117-140." F i e d l e r , F. E n g i n e e r t h e j o b t o f i t t h e manager. H a r v a r d B u s i n e s s Review. 1965 , 43, 5 , S e p t - O c t . , 115-122. F i s h b e i n , M. The s e a r c h f o r a t t i t u d i n a l - b e h a v i o u r a l c o n s i s t e n c y . i n ' J . Cohen (ed) B e h a v i o u r a l S c i e n c e  F o u n d a t i o n s o f consumer b e h a v i o u r . F r e e P r e s s , G l e n c o e , I l l i n o i s , 1971. F i t c h , N. E v a l u a t i o n o f i n s t r u c t o r s i n C a l i f o r n i a  J u n i o r c o l l e g e s . U n i v e r s i t y o f C a l i f o r n i a , B e r k e l e y , 1965. F l a n a g a n , J . C . The c r i t i c a l i n c i d e n t t e c h n i q u e . P s y c h o l o g i c a l B u l l e t i n . 1954, 51, 327-358. Fog 1 i , L., H u l i n , C.L. and B l o o d , M.R. D e v e l o p m e n t o f f i r s t l e v e l , b e h a v i o u r a l j o b c r i t e r i a . J o u r n a l o f  A p p l i e d P s y c h o l o g y . 1971, 55, 3-8. Fox, A.M. and B r o o k s h i r e , W.K. D e f i n i n g e f f e c t i v e c o l l e g e t e a c h i n g . The J o u r n a l o f E x p e r i m e n t a l  E d u c a t i o n . 1971, 40, No. 2, W i n t e r , 37-40. -103-F r a n z , H.J. The c r i t e r i a o f b e i n g e d u c a t e d . E d u c a t i o n T h e o r y . 1972, Volume 22, F a l l , No. 4, 395-402. Freeman, R.B. O v e r i n v e s t m e n t i n c o l l e g e t r a i n i n g ? J o u r n a l o f Human R e s o u r c e s . 1975, Summer, Volume 10, No. 3, 287-311. Gage, N.L. P a r a d i g m s f o r r e s e a r c h on t e a c h i n g . In N.L. Gage ( e d ) Hand book o f R e s e a r c h on T e a c h i n g Rand M c N a l l y and Co., C h i c a g o , 1963. Gage, N.L. T h e o r i e s o f t e a c h i n g In E.R. H i l g a r d v ( e d ) T h e o r i e s o f L e a r n i n g and I n s t r u c t i o n NSSE y e a r book, No. 63, P a r t 1, U n i v e r s i t y o f C h i c a g o P r e s s , C h i c a g o , 1964. Gagne, R.M. The c o n d i t i o n s o f l e a r n i n g . H o 4 t , R i n e h a r t and W i n s t o n , New Y o r k , 1965. G e t z e l s , J.W. Methods u s e d t o s t u d y p e r s o n a l i t y . J . N a t. A s s . Deans Women, 1953, 16, 154-158. A l s o q u o t e d i n N.L. Gage (ed) Hand book o f r e s e a r c h  on t e a c h i n g Rand M c N a l l y and Co., C h i c a g o , 1963. G e t z e l s , J.W. and J a c k s o n , P.W. The t e a c h e r ' s p e r s o n a l i t y and c h a r a c t e r i s t i c s . In N.L. Gage ( e d ) Hanbook o f R e s e a r c h on t e a c h i n g Rand M c N a l l y and Co. , C h i c a g o , 1963. G h i s e l l i , E.E. D i m e n s i o n a l p r o b l e m s o f c r i t e r i a . J o u r n a l o f A p p l i e d P s y c h o l o g y . 1956, 40, 1-4. G l a s s e r , W. R e a l i t y therapy-^ H a r p e r and Row, New Y o r k , 1965. G o r d o n , W.J.J. The m e t a p h o r i c a l way o f l e a r n i n g  and knowing. Synet'i c s - E ' d u c a t i on P r e s s , Cambridge Mass., 1970. Gough, H.G., D u r f l i n g e r , G.W. and H i l l J r . R.E P r e d i c t i n g p e r f o r m a n c e i n s t u d e n t t e a c h i n g f r o m t h e C a l i f o r n i a P s y c h o l o g i c a l i n v e n t o r y . J o u r n a l o f  E d u c a t i o n a l P s y c h o l o g y 1968, 59, 2, 119-127. Gough, H.G. and Pemberton, W.H. P e r s o n a l i t y c h a r a c t e r i s t i c s r e l a t e d t o s u c c e s s i n p r a c t i c e t e a c h i n g . J o u r n a l o f A p p l i e d P s y c h o l o g y 1952, 36, 307-309. -104-G r o n e r , D.M. R e l i a b i l i t y and s u s c e p t i b i l i t y t o  b i a s o f b e h a v i o u r a l and g r a p h i c r a t i n g s c a l e s . U n p u b l i s h e d d o c t o r a l d i s s e r t a t i o n , U n i v e r i s t y o f M i n n e s o t a , 1974. G u i o n , R.M. P e r s o n n e l t e s t i n g . McGraw H i l l book Co., New Y o r k , 1965. G u t h r i e , J . T . R e l a t i o n s h i p s o f t e a c h i n g method, S o c i o e c o n o m i c s t a t u s and i n t e l l i g e n c e i n c o n c e p t f o r m a t i o n . J o u r n a l o f E d u c a t i o n a l P s y c h o l o g y 1971, 62, 4, 345-351. H a r a r i , 0. and Z e d e c k , S. Development o f b e h a v i o u r a l l y a n c h o r e d s c a l e s f o r t h e e v a l u a t i o n o f f a c u l t y t e a c h i n g . J o u r n a l o f A p p l i e d P s y c h o l o g y 1973, 38, 2, 261-265 H a r v e y , J.N. and B a r k e r , D.G. S t u d e n t e v a l u a t i o n o f t e a c h i n g e f f e c t i v e n e s s . I m p r o v i n g c o l l e g e and  u n i v e r s i t y t e a c h i n g 1970, Volume X V I I I , No. 4 Autumn, 275-278. Hays, W.L.. S t a t i s t i c s f o r p s y c h o l o g i s t s . H o l t , R i n e h a r t and W i n s t o n , New Y o r k , 1966. H i g h e t , G. The a r t o f t e a c h i n g . V i n t a g e B o o k s , New Y o r k , 195?. H o m e , H.H. The p h i l o s o p h y o f e d u c a t i o n . R e v i s e d e d i t i o n , The M a c m i l l a n Company/New Y o r k , 1927. H u l l f i s h , G.H. and S m i t h , P.G. R e f l e c t i v e t h i n k i n g : The method o f e d u c a t i o n . Dodd, Mead and Co., 1961. Hunt, D.E. A c o n c e p t u a l l e v e l m a t c h i n g model f o r c o o r d i n a t i n g l e a r n e r c h a r a c t e r i s t i c s w i t h e d u c a t i o n a l a p p r o a c h e s . I n t e r c h a n g e : A J o u r n a l o f E d u c a t i o n a l  S t u d i e s Volume 1, No. 2, 1970 I s a a c s o n , R . I . , M c K e a c h i e , W.J., M i l h o l l a n d , J . E . , Lin;' Y.G., H o f e l l e r , M. , B a e r w a l d t , J.W. and Z i n n , K.L. The d i m e n s i o n s o f s t u d e n t e v a l u a t i o n s o f t e a c h i n g . In M c K e a c h i e , W.J., I s a a c s o n , R .I. and M i l h o l l a n d , J . E . R e s e a r c h on t h e ' ' ; c h a r a c t e r i s t i c s o f e f f e c t i v e c o l l e g e  t e a c h i n g . Coop. Res. P r o j e c t No, E850, U n i v e r s i t y o f M i c h i g a n , Ann A r b o r , 1964. J e n s e n , A.C. D e t e r m i n i n g c r i t i c a l r e q u i r e m e n t s f o r t e a c h e r s . J o u r n a l o f E x p e r i m e n t a l E d u c a t i o n . 1951, 20, 79-86. J u d d , C H . E d u c a t i o n and s o c i a l p r o g r e s s . H a r t c o u r t , B r a c e , J o v a n o v i c h , 1934. -105-J u s t i z , T.B. A r e l i a b l e measure o f t e a c h e r e f f e c t i v e n e s s . E d u c a t i o n a l L e a d e r s h i p R e s e a r c h  S u p p l e m e n t 1969, 3, 1, O c t o b e r , 49-55. Kaufman, A. On some p h i l o s o p h i c g r o u n d s f o r u n d e r s t a n d i n g t e a c h i n g . S t u d i e s i n P h i l o s o p h y and  E d u c a t i o n , 1967, 5, 347-357. K e a v e n y , T . J . and McGann, A.F. A c o m p a r i s o n o f b e h a v i o u r a l e x p e c t a t i o n s c a l e s and g r a p h i c r a t i n g s c a l e s . J o u r n a l o f A p p l i e d P s y c h o l o g y , 1975, 60, 6 , 695-703. K i e r k e g a a r d , S. P h i l o s o p h i c a l F r a g m e n t s . P r i n c e t o n . U n i v e r s i t y P r e s s , P r i n c e t o n , New J e r s e y , 1946. K i r c h n e r , U.K. and D u n n e t t e , M.D. I d e n t i f y i n g t h e c r i t i c a l f a c t o r s i n s u c c e s s f u l s a l e s m a n s h i p . P e r s o n n e l 1957, 34, 54-59. K o h l a n , R.G. A c o m p a r i s o n o f f a c u l t y e v a l u a t i o n s e a r l y and l a t e i n t h e c o u r s e . J o u r n a l o f H i g h e r  E d u c a t i o n 1973, Volume VL I V , November, No,8, 587-595. K o n i g s b u r g , D. The d e v e l o p m e n t and p r e l i m i n a r y  e v a l u a t i o n o f an i n s t r u c t o r c h e c k l i s t b a s e d on t h e  c r i t i c a l i n c i d e n t t e c h n i q u e . U n p u b l i s h e d d o c t o r a l d i s s e r t a t i o n , U n i v e r s i t y o f P i t t s b u r g h , 1954. L a n d y , F . J . and G u i o n , R.M. D e v e l o p m e n t o f s c a l e s f o r t h e measurement o f work m o t i v a t i o n . O r g a n i z a t i o n a l B e h a v i o u r and Human p e r f o r m a n c e 1970, 5, 93-103. L a w r e n c e , P. and L o r s c h , J.W. O r g a n i z a t i on and  E n v i r o n m e n t - Managing d i f f e r e n t i a t i o n and i n t e g r a t i o n . H a r v a r d U n i v e r s i t y , B o s t o n , 1967. Maas, J . B . P a t t e r n e d s c a l e e x p e c t a t i o n i n t e r v i e w : R e l i a b i l i t y s t u d i e s on a new t e c h n i q u e . J o u r n a l o f  A p p l i e d P s y c h o l o g y , 1965, 49, 431-433. M a s s i a l a s , B. and Cox, B. I n q u i r y i n s o c i a l s t u d i e s McGraw H i l l , New Y o r k , 1966. M c K e a c h i e , W.J. T e a c h i n g T i p s : A g u i d e book f o r  t h e b e g i n n i n g c o l l e g e t e a c h e r . D.C. Heath and Co., L e x i n g t o n , Mass., 1969. -106-M c N e i l , J.D. and Popham, W.J. The a s s e s s m e n t o f t e a c h e r competence i n R.M.W T r a v e r s ( e d ) . S e c o n d Handbook o f R e s e a r c h on T e a c h i n g R a n d . M c N a l l y and Co., C h i c a g o , 1973. M i t z e l , H.E. A b e h a v i o u r a l a p p r o a c h t o t h e a s s e s s m e n t  o f t e a c h e r e f f e c t i v e n e s s . C o l l e g e o f t h e c i t y o f New Y o r k , D i v i s i o n o f t e a c h e r e d u c a t i o n , New Y o r k , 1957. M i t z e l , H.E. T e a c h e r e f f e c t i v e n e s s i n C.W. H a r r i s l_ed) E n c y c l o p e d i a o f E d u c a t i o n a l R e s e a r c h 3 r d e d i -t i o n , The M a c m i l l a n Company, New Y o r k , 1960. Moore, C H . and C o l e , D. The r e l a t i o n s h i p o f MMPI s c o r e s t o p r a c t i c e t e a c h i n g s c o r e s . J o u r n a l o f E d u c a t i o n a l R e s e a r c h 1957, 50, 711-716. M o r s h , J . E . and W i l d e r , E.W. I d e n t i f y i n g t h e .: e f f e c t i v e i n s t r u c t o r : A r e v i e w o f the, q u a n t i t a t i v e  s t u d i e s ; 1900-1952. R e s e a r c h B u l l e t i n No. AFP TRC _ TR _ 54-^ 44 San A n t o n i o , T e x a s ; USAF P e r s o n n e l i r a i n i n g R e s e a r c h C e n t y e . 1954. N a g l e , B.F. C r i t e r i o n D e v e l o p m e n t . P e r s o n n e l  P s y c h o l o g y 1953, 6, 271-289. O l i v e r , D.W. and S h a v e r , J.P. T e a c h i n g p u b l i c  i s s u e s i n t h e h i g h s c h o o l , Houghton M i f f l i n , B o s t o n , 1966. Q y e r t u r f , C.L. S t u d e n t r a t i n g o f f a c u l t y a t  S t . J o h n s R i v e r J u n i o r C o l l e g e , w i t h addendum  f o r A l b e n y J u n i o r C o l l e g e . S t . J o h n s R i v e r J u n i o r C o l l e g e , P a l a t k a , F l o r i d a , 1966. P a c e , C.R. C o l l e g e and u n i v e r s i t y e n v i r o n m e n t  s c a l e s ( C U E S ) : P r e l i m i n a r y t e c h n i c a l m a n u e l . E d u c a t i o n a l T e s t i n g S e r v i c e , P r i n c e t o n , New J e r s e y , 1963. P a i n t e r , J . J . and G r a n z i n , K.L. C o n s i s t e n c y t h e o r y as an e x p l a n a t i o n o f s t u d e n t s ' c o u r s e e v a l u a -t i o n t e n d e n c i e s . The J o u r n a l o f E x p e r i m e n t a l  E d u c a t i o n 1972, 41, 1, F a l l . P e c k , R.F. P e r s o n a l i t y p a t t e r n s o f p r o s p e c t i v e t e a c h e r s . J o u r n a l o f E x p e r i m e n t a l E d u c a t i o n 1960, 29. 169-175. -107-P e g i s , R.(ed) I n t r o d u c t i o n to S t . Thomas A c q u i n a s . Modern L i b r a r y , New Y o r k , 1948. P o l a n y i , M. P e r s o n a l Knowledge. H a r p e r and Row, New Y o r k , 1958. Popham, W.J. D e v e l o p m e n t o f a p e r f o r m a n c e t e s t  o f t e a c h i n g p r o f i c i e n c y . USOE, F i n a l r e p o r t , C o n t r a c t No., OE - 6 - 10 -254, U n i v e r s i t y o f C a l i f o r n i a , L o s A n g e l e s , 1 9 6 7 . Q u e r e s h i , M.Y. and W i d l a k , F.W. S t u d e n t s ' p e r c e p t i o n o f a c o l l e g e t e a c h e r as a f u n c t i o n o f t h e i r sex and a c h i e v e m e n t l e v e l . J o u r n a l o f  E x p e r i m e n t a l E d u c a t i o n , 1973, 41, 3, S p r i n g , 53-57. R o g e r s , C R . C l i e n t c e n t e r e d t h e r a p y . Houghton M i f f i n , B o s t o n , 1951. R o g e r s , C R . Freedom t o l e a r n C h a r l e s E. M e r r i l l and Co., Columbus, O h i o , 1969. Runkel , P . J . 1958. Quoted by Gage, N.L. P a r a d i g m s f o r r e s e a r c h on t e a c h i n g i n N.L. Gage", (ed) Handbook o f R e s e a r c h on T e a c h i n g , Rand M c N a l l y and Co., C h i c a g o , 1963, 126. R u s s e l l , B. E d u c a t i o n and good l i f e , 3 r d p r i n t i n g B o n i and L i v e r i g h t , New Y o r k , 1926. Ryans, D.G. C h a r a c t e r i s t i c s o f t e a c h e r s . A m e r i c a n C o u n c i l on E d u c a t i o n , W a s h i n g t o n , D.C, 1960. S a l o m o n , G. amd M c D o n a l d , F . J . P r e t e s t and p o s t t e s t r e a c t i o n s t o s e l f r e v i e w i n g one's t e a c h i n g p e r f o r m a n c e on v i d e o t a p e . J o u r n a l o f E d u c a t i o n a l P s y c h o l o g y 1970, 61, 280-286. S c h m i d t , F.L. and K a p l a n , L.B. C o m p o s i t e Vs. M u l t i p l e c r i t e r i a : A r e v i e w and r e s o l u t i o n o f t h e c o n t r o v e r s y . P e r s o n n e l P s y c h o l o g y 1971, 24, 419-434. S c h u t z , W. J o y : E x p a n d i n g human a w a r e n e s s . Grove p r e s s , New York , 1967. Schwab, J . J . B i o l o g y t e a c h e r s ' Handbook. W i l e y , New Y o r k , 1965. -108-Schwab, D.P., Heneman 111, H.G. , and D e c o t i i s , Y.A. B e h a v i o u r a l l y A n c h o r e d R a t i n g s S c a l e s : A r e v i e w o f  t h e 1 i t e r a t u r e . G r a d u a t e s c h o o l o f B u s i n e s s , U n i v e r s i t y o f W i s c o n s i n - M a d i s o n , W i s c o n s i n 53706, An e a r l i e r d r a f t o f t h e p a p e r r e a d a t t h e 35th N a t i o n a l Academy o f Management M e e t i n g s , New O r l e a n s , A u g u s t 1975. S e a r s , R.R. A t h e o r e t i c a l framework f o r p e r s o n a l i t y and s o c i a l b e h a v i o u r . A m e r i c a n P s y c h o l o g i s t , 1951, 6, 476-484. S h e p a r d , H.A. The T group as t r a i n i n g i n o b s e r v a n t p a r t i c i p a t i o n . The p l a n n i n g o f change: R e a d i n g s i n  t h e A p p l i e d B e h a v i o u r a l , S c i e n c e W.G. B e n n i s , K.D. Benna, and R . C h r i s ( e d s ) H o l t , R i n e h a r t and W i n s t o n , New Y o r k , 1964. S k i n n e r , B.F. The t e c h n o l o g y o f t e a c h i n g , A p p l e t o n C e n t u r y C r o f t s , New Y o r k , 1968. S m i t h , B.O. A c o n c e p t o f t e a c h i n g . T e a c h e r s C o l 1. Rec. , 1960, 61 , 229-241. A l s o i n N.L7~Gage ( e d ) Handbook o f r e s e a r c h on t e a c h i n g , Rand M c N a l l y and Co. , C h i c a g o , 1963, P.122. . S m i t h , P.C. and K e n d a l l , L.M. R e t r a n s l a t i o n o f e x p e c t a t i o n s : An a p p r o a c h to t h e c o n s t r u c t i o n o f unambiguous a n c h o r s f o r r a t i n g s c a l e s . J o u r n a l o f  A p p l i e d P s y c h o l o g y 1963, 47, 149-155. S o b o l , F.T. A model t e a c h e r o b s e r v a t i o n s y s t e m -U n p u b l i s h e d d o c t o r a l d i s s e r t a t i o n , S t a n f o r d U n i v e r s i t y 1971: S p r i n t h a l l , N.A., W h i t e l e y , J.M. and Mosher, R.L. A s t u d y o f t e a c h e r e f f e c t i v e n e s s . The J o u r n a l o f  T e a c h e r E d u c a t i o n , 1966, 17, 93-106. S t a a t s , A.W. S o c i a l b e h a v i o u r i s m . The D o r s e y P r e s s , Homewood, I l l i n o i s , 1975. S t e r n , G.G. C h a r a c t e r i s t i c s o f t h e i n t e l l e c t u a l c l i m a t e i n c o l l e g e e n v i r o n m e n t s . H a r v a r d E d u c a t i o n a l  Review 1963, 33, 5-41. S t o l u r o w , L.M. T e a c h i n g by m a c h i n e . U.S. o f f i c e o f E d u c a t i o n , Coop. Res. Monogr. No. 6, OE -34010, U.S. G o v t . P r i n t i n g O f f i c e , W a s h i n g t o n , D.C., 1961. -109-Suchman, J.R. The e l e m e n t a r y s c h o o l t r a i n i n g  p r ogram i n s c i e n t i f i c e n q u i r y . R e p o r t o f t h e U.S. o f f i c e o f E d u c a t i o n , p r o j e c t t i t l e . V I I I , P r o j e c t 216, U n i v e r s i t y o f I l l i n o i s , U r b a n a , 1962. S u l l i v a n , M.W. Programmed E n g l i s h : A modern grammer  f o r H i g h s c h o o l and c o l l e g e s t u d e n t s . The M a c m i l l a n Company, New Y o r k , 1963. T a b a , H. T e a c h i n g s t r a t e g i e s arid c o g n i t i v e f u n c t i o n - i n g i n e l e m e n t a r y s c h o o l c h i l d r e n . S a n f r a n c i s c o S t a t e c o l l e g e , Coop. R e s e a r c h p r o j e c t No. 2404, 1966. T a b e r , J . , G l a s e r , R.. H a l m u t h , H.S. L e a r n i n g and  programmed i n s t r u c t i o n . A d d i s o n - W e s l e y , R e a d i n g , Mass. , 1965. Taneman, I. A t e a c h i n g power t e s t : A method o f  u s i n g p u p i l g a i n as a c r i t e r i o n o f t e a c h e r e d u c a t i o n . U n p u b l i s h e d d o c t o r a l d i s s e r t a t i o n , U n i v e r s i t y o f C a l i f o r n i a , Los A n g e l e s , 1970. T h e l e n , H.A. E d u c a t i o n and t h e human q u e s t . H a r p e r and Row, New Y o r k , 1 9 6 0 . T h o r n d i k e , R.L. P e r s o n n e l s e l e c t i o n : T e s t and  measurement t e c h n i q u e . W i l e y , New Y o r k , 1949 T i t t l e , C R . and H i l l , R .J. A t t i t u d e measurement and p r e d i c t i o n o f b e h a v i o u r : An e v a l u a t i o n o f c o n d i t i o n s and measurement t e c h n i q u e s . S o c i o m e t r y 1967, 30, 199-213. T o o p s , H.A. The c r i t e r i o n . E d u c a t i o n a l and P s y c h o -l o g i c a l Measurement, 1944, 4 , 2 7 1 - 2 9 7 . ; Tuckman, B.W. and O l i v e r , W.F. E f f e c t i v e n e s s o f f e e d b a c k t o t e a c h e r s as a f u n c t i o n o f s o u r c e . J o u r n a l o f E d u c a t i o n a l p s y c h o l o g y , 1968, Volume 59, No. 4, 297-301. Veldman, D.J. and P e c k , R.F. S t a d e n t t e a c h e r c h a r a c t e r i s t i c s f r o m t h e p u p i l ' s view p o i n t . J o u r n a l - o f E d u c a t i o n a l P s y c h o l o g y 1963, 54, 346-355. W a l b e r g , H.J. and A n d e r s o n , G.J. C l a s s room c l i m a t e and i n d i v i d u a l l e a r n i n g . J o u r n a l o f E d u c a t i o n a l  P s y c h o l o g y 1968, 59, 414-419. W a l l e r , R.A. I's on t h e i m p o r t a n c e o f t e a c h i n g . I m p r o v i n g c o l l e g e and u n i v e r s i t y t e a c h i n g 1971, Volume XIX, No. 3, Summer, 196-197. -110-W a t e r h o u s e , I.K. and C h i l d , I . L . F r u s t r a t i o n and t h e q u a l i t y o f p e r f o r m a n c e . J o u r n a l o f P e r s o n n e l  P s y c h o l o g y 1953, 21, 298-311. W e i n e r , B. and K u k l a , A. An a t t r i b u t i o n a l a n a l y s i s o f a c h i e v e m e n t m o t i v a t i o n . J o u r n a l o f P e r s o n a l i t y  and S o c i a l P s y c h o l o g y , 1970, 15, 1-20. W i l l i a m s , W.E. and S e i l e r , D.A. R e l a t i o n s h i p between measures o f e f f o r t and j o b p e r f o r m a n c e . J o u r n a l o f A p p l i e d P s y c h o l o g y 1973, 57, 49-54. Wingo, G.M. P h i l o s o p h i e s o f e d u c a t i o n : An i n t r o d u c -t i o n . D.C. Heath and Co., L e x i n g t o n , Mass., 1974. Z e d e c k , S. and B a k e r , H.T. N u r s i n g p e r f o r m a n c e as measured by b e h a v i o u r a l e x p e c t a t i o n s c a l e s : A m u l t i t r a i t - m u l t i r a t e r a n a l y s i s . O r g a n i z a t i o n a l  B e h a v i o u r and Human P e r f o r m a n c e , 1972, 7, 457-466. -111-APPENDIX A CRITERIA OF TEACHER EFFECTIVENESS GIVEN BY DOMAS AND TIEDEMAN (1950, pp 214-215) Pupil Achievement! r?fl-5. Judgments (by Administrators Selves, Teachers, Pupils A l . "Objectively' Observed A2. Subjectively Evaluated C7. Performance on Tests Pupil Achievement Bl -2 . Subjectively Evaluated D l - 4 . Judgments oy Administtatots Teachers, Pupils, etc. D5. [Performance on Tests -112-APPEMDIX B GENERALIZED SCHEMA FOR RESEARCH IN TEACHING EFFECTIVENESS GIVEN BY MITZEL (1957, p 5) T y p e I Var iables Prediction Sources Teacher V a r i a b l e s T y p e II Var iables Contingency Factors Teache* Personality A t t .udes Interests A b i l i t i e s Ttxcher Training Factors A c h i e v e m e n t i n t e a c h i n g m e t h o d s courses P c r f o r m a 1 - :e i n s tudent teaching. Specific k n o w l e d g e s a n d s k i l l s , etc. T y P : - III Var iables Classroom Behaviors T y p e I V V a r i a b l e s Crite ria of Effect it eness (Intermediate E d u c a t i o n a l G o a l s ) Teacher B e h a v i o r s In the c o m m u n i t y I n extracurricular s c h o o l activit ies Jin p r o m o t i n g mental heal th , etc. In c lassroom E n v i r o n m e n t a l / V a r i a b l e s / r DI < i w i C| w M l M | I I I S c h o o l l o c a t i o n S c h o o l size. S c h o o l o r g a n i s a t i o n S c h o o l plant a n d e q u i p m e n t [ C o m m u n i t y e c o n o m i c factors, etc. / I / I L Changes i n P u p i l B e h a v i o r ' ( P u p i l G r o w t h ) P u p i l - T e a c h e r Interact ion I n subject matter k n o w l e d g e In soc ia l s k i l l s I n a p p r e c i a t i o n o f . d e m o c r a t i c values I n att i tudes, apprec ia t i o n s , etc. I I n c l a s s r o o m O u t o f c l a s s r o o m P u p i l B e h a v i o r s P u p i l Var iab les I A t t i t u d e s Interests A b i l i t i e s etc. <-JTCIip_PACK. J o z • s cc • O FOR CANADIAN COLLEGES AND UNIVERSITIES SI andZZl \ h T " V O U u n ° P P ° r t U n i t y t 0 « P « anonymously your views of this course and the way ,t has been taught. Indicate the response closest to your view by blackening the approbate ova!. Use a soft lead pencil (preferably No. 2) for all responses to the q e tio naire Do not use an ink or ball point pen. questionnaire. R Report Number SECTION I Items 1-20. Blacken one response number for each question. NA (0) = Not Applicable or don't know. The statement does not apply to this course or instructor, or you simply are not able to give a knowledgeable response. = Strongly Agree. You strongly* agree with the statement as it applies to this course or instructor. = Agree. You agree more than you disagree with the state-ment as it applies to this course or instructor. = Disagree. You disagree more than you agree with the statement as it applies to this course or instructor. = Strongly Disagree. You strongly disagree with the statement as it applies to this course or instructor. SA (4) A (3) D (2) SD (1) 1. 2. 3. 4. 5. 6. 7. 8. 9. 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 20. NA The instructor's objectives for the course have been made clear ^ There was considerable agreement between the announced objectives of the course and what was actually taught The instructor used class time well The instructor was readily available for consultation with students The instructor seemed to know when students didn't understand the material Lectures were too repetitive of what was in the textbook(s) r... The instructor encouraged students to think for themselves The instructor seemed genuinely concerned with students' progress and was actively helpful The instructor made helpful comments on papers or exams The instructor raised challenging questions or problems for discussion In this class I felt free to ask questions or express my opinions The instructor was well-prepared for each class The instructor told students how they would be evaluated in the course The instructor summarized or emphasized major points in lectures or discussions ® My interest in the subject area has been stimulated by this course The scope of the course has been too limited; not enough material has been covered Examinations reflected the important aspects of the course I have been putting a good deal of effort into this course The instructor was open to other viewpoints In my opinion, the instructor has accomplished (is accomplishing) his or her objectives for the course SECTION II Items 21-31. Blacken one response number for each question 21 SA A D SD CE) © © © © © © © © © © CD a> © © © © © © © © © CD © © © © © © © © © © © © © © © © CE) © © © © CE) © © © © © © © © © © © © © © © © © © ® © © © © © ® © © © © © © © © © © © © © © © © © © © © © © © © © © © For my preparation and ability, the level of difficulty of this course was: CD Very elementary © Somewhat difficult © Somewhat elementary © Very difficult © About right 23. For me, the pace at which the instructor covered the material during the term was: Very slow © Somewhat fast Somewhat slow © Very fast Just about right • 22. The work load for this course in relation to other courses of equal credit was: 24. Much lighter Lighter About the same Heavier Much heavier To what extent did the instructor use examples or illustrations to help clarify the material? © Frequently © Seldom © Occasionally © Never Questionnaire continued on the other side. Copyright © 1971 by Educational Testing Service. All Rights Reserved. No part of the Student Instructional Report may be adapted or reproduced 572MRC93P100 in any form without permission in writing from the publisher. 275603 IS M S 3 M X o 25. Was class size satisfactory for the method of conducting the class? 28. What grade do you expect to receive in this course? ® Yes, most of the time ® 1 3 3 No, class was too large ® No, class was too small It didn't make any differ-ence one way or the other 26. CD CO 27. CD ca CD © CO ® CD Which one of the following best describes this course for you? CD 75-100% ® 66-74% CD 60-65% GD 50-59% ® Fail ® Pass ® No credit © Other Major requirement or elective within major field Minor requirement or required elective out-side major field CD © CD College requirement but not part of my major or minor field Elective not required in any way Other ® 50-54% ® Less than 50% © None yet-first year or transfer Which one of the following was your most important reason for selecting this course? Friend(s) recommended it Faculty advisor's recommendation Teacher's excellent reputation Thought I could make a good grade Could use pass/no credit option It was required Subject was of interest Other 29. What is your approximate cumulative average? ® 75-100% <2> 71-74% a> 66-70% ® 60-65% ® 55-59% 30. What is your class level? © 1st year © 4th year ® 2nd year ® Graduate ® 3rd year ® Other 31. Sex: CD Female c» Male SECTION III Items 32-39. Blacken one response number for each question. 32. 33. 34. 35. 36. 37. 38. 39. Overall, I would rate the textbook(s) © Overall, I would rate the supplementary readings © Overall, I would rate the quality of the exams.... ® I would rate the general quality of the lectures ® I would rate the overall value of class discussions.... c© Overall, I would rate the laboratories ® I would rate the overall value of this course to me as ® * of <<7 CD CD CO CD CD CD CO ® CO © © © © © cf CD CO © CD CO © CD © © CD © © CD CD © CD CO © CD © © Compared to other instructors you have had (secondary school and college), how effective has the instructor been in this course? (Blacken one response number.) One of the most More effective Not as effective effective than most About as most (among the top 10%) (among the top 30%) average (in the lowest 30%) CD © © © One of the least effective (in the lowest 10%) © SECTION IV Items 40-49. If the instructor provided supplementary questions and response options, use this section for responding. Blacken only one response number for each question. N A N A 40. CD © © © © © © © © © 45. © © © ® © © ® © ® © 41. © © © © © © ® ® © © 46. © © © © ® © © © 42. © © © © © © © © © © 47. © © ® © © © © ® © © 43. © © © © © © ® ® © © 48. © © ® ® © ® ® ® © ® 44. © © © © © ® CD ® ® © 49. © © © © © ® ® © ® ® If you would like to make additional comments about the course or instruction, use a separate sheet of paper. You might elaborate on the particular aspects you liked most as well as those you liked least. Also, how can the course or the way it was taught be improved? PLEASE GIVE THESE COMMENTS TO THE INSTRUCTOR. If you have any comments or suggestions about this questionnaire (for example, the content or responses available), please send them to: Student Instructional Report, Educational Testing Service, Princeton, New Jersey 08540. -115-JTUDEIMT INSTRUCTIONAL REPORT INSTITUTIONAL RESEARCH PROGRAM FOR HIGHER EDUCATION V j L / EDUCATIONAL TESTING SERVICE, PRINCETON, N.J. 08540 APPENDIX D ' TWO-YEAR COLLEGE COMPARATIVE DATA n . / £ y-»3 R E P O R T D A T E C O L L E G E C O D E S I R R F P O R T N U M B E R C L A S S I E N R O L L 1 M M E N T J 12203 ; S E C T I O N I A l l r » s p o n s c s u m m a r i e s s i r e > i P E R C E N T A G E o f t i n ; t n l d l n u m b e r r e s p o n d i n g , w h i c h i s 772 1. The instructor's objectives for the course have been made clear. 2. There was considerable agreement between the announced objectives of the course and what was actually taught. 3. The instructor used class time well. 4. The instructor was readily available for consultation with students. 5. The instructor seemed to know when students didn't understand the material. 6. Lectures were too repetitive of what was in the textbook(s). 7. The instructor encouraged students to think for themselves. 8. The instructor seemed genuinely concerned with students' progress and was actively helpful. 9. The instructor made helpful comments on papers or exams. 10. The instructor raised challenging questions or problems for discussion. 11. In this class I felt free to ask questions or express my opinions. 12. The instructor was well-prepared for each class. 13. The instructor told students how they would be evaluated in the course. 14. The instructor summarized or emphasized major points in lectures or discussions. O m i t s 15. My interest in the subject area has been stimulated by this course. 16. The scope of the course has been too limited; not enough material has been covered. 17. Examinations reflected the important aspects of the course. 18. I have been putting a good deal of effort into this course. 19. The instructor was open to other viewpoints. 20. In my opinion, the instructor has accomplished (is accomplishing) his objectives for the course. • . , "4 T " " 3 2 I 1 . . . . . i Stronqlv A q r e o ' Disagree Stongly Applicable I A g r e e I " n . « n % F 47 1 4 10 12 42 50 50 36 43 4 7 33 37 55 56 41 46 40 30 37 40 48 43 43 38 37 43 13 41 37 37 39 36 43 40 35 11 42 45 42 40 12 43 14 11 15 47 10 13 D i s a g r e e 2 21 30 M e a n * 3.39 3.30 3.38 3.44 3.19 2.10 3 .34 3.34 3 . 12 3.23 3.47 3.49 3.30 3.41 3 .13 1.94 3.11 3.20 3.31 3.39 S t a n d a r d D e v i a t i o n * .36 .36 .43 .33 .3t> .41 .33 .%7 .40 . 32 . 36 .34 i3-3: |J49: .37 .40 .34 .38 .39 • O M I T S a n d N O T A P P L I C A B L E r e s p o n s e s a r e e x c l u d e d i n c o m p u t i n g t h e m e a n a n d s t a n d a r d d e v i a t i o n . i -116-JTUDEIMT IIMSTRUCTIOIMAL REPORT . INSTITUTIONAL RESEARCH PROGRAM FOR HIGHER EDUCATION EDUCATIONAL TESTING SERVICE, PRINCETON, N.J. 08540 TWO-YEAR COLLEGE A D M I N COMPARATIVE DATA D A T E R E P O R T C O L L E G E SIR R E P O R T E N R O L L D A T E C O D E N U M B L R I * 7.773 M E N T i z •:• u i Very Elementary Somewhat Elementary 3 * About Right * Somewhat Difficult Very Difficult S E C T I O N n Al l response summaries are a P E R C E N T A G E of the total number responding, which is: Omits Mean* Standard Deviation* 21. For my preparation and ability, the level of difficulty of this course was: L 9 26 3 . 1 9 Omits Much Lighter Lighter About the same Heavier Much Heavier Mean* Standard Deviation* 22. The work load for this course in relation to other courses of equal credit was: 2> I*" b 3.09 .54 Omits Very Slow Somewhat Just about slow right Somewhat fast Very fast Mean" Standard Deviation* 23. For me, the pace at which the instructor covered the matei lal clui my the temi was: I 2 ". j j 3 . I t .31 24. To what extent did the instructor use examples or illustrations to help clarify the material? Omits Frequently Occasionally Seldom I " -Never Mean* Standard Deviation* 2 6 3 28 6 2 0 3.5»4 .3<> Omits 25. Was class size satisfactory for the method of conducting the class? Yes, most of ! No, class was too large the time t i 6 No, class was too small It didn't make any difference Major requirement Omits oi uli'ctive within m.ijor field • • • • • 1 Omits Friend (s) ri'com Minor requirement or required elective outsidP m.ijor field 13 College requirement but not part of my niiijor or minor field Elective not required in any wny Other I 27. Which one of the following was your most important reason for selecting this course? Faculty I Teacher's Thought I . . ddvisors encclleiil could make mended it recommend reputation a good grade : ; Y Could use . Subject pass/no credit " w . a s , f—*" . ; required option 1 H was of ' Other interest ; 28. What grade do you expect to receive in this course? 29. What is your approximate cumulative grade-point average? Omits i - J 3D Omit 3.504.00 Fail ~ l _L "1 T -P a s s ' CrNeS,t ' °ther 3.00-3.49 2 2 ? Sit) ?.99 ?.00 2 49 T Omits ! Freshman ; Sophomore 1 Junior 1.50-1.99 1.00-1.49 I Less than None yet-freshman or transfer 1.00 0 30. What is your class level? -r„ 3<L 31. Sex: Omits • . . . Female ililltsiiliUp Male X <i i1 J Senior Other 10 ' O M I T S responses are excluded in computing the mean and standard deviation -117-JTUDENT INSTRUCTIONAL REPORT (VW) TWO-YEAR COLLEGE INSTITUTIONAL R E S E A R C H PROGRAM FOR HIGHER EDUCATION "^t^ COMPARATIVE DATA EDUCATIONAL TESTING SERVICE, PRINCETON, N.J. 08540 R E P O R T D A T E C O L L F G E C O D E i37_/73 C L A S S S I R H F P O R T E N R O L L N U M B E R M E N T 122u o S E C T I O N m A l l r e s p o n s e s u m m a r i e s a t e B P E R C E N T A G E . > o f t h e t o t a l n u m b a r r e s p o n d i n g , w h i c h i s : ; " O m i t s N o t a p p l i c a b l e , d o n ' t k n o w , o r t h e r e w e r e n o n e 5 E x c e l l e n t 4 G o o d S a t i s f a c t o r y X - 2 F a i r i P o o r „ i Standard n"'!an Deviation 1 ( . . ._ 32. Overall, 1 would rate the textbook(s): j 0 1 9 }J> lo l u 6 3.5 9 .67 33. Overall, 1 would rate the supplementary readings: 4 31 14 2 8 1-r 7 3 3.63 . 3 b 34. Overall, 1 would rate the quality of the exams: 10 1 9 3 9 i 3 7 3 3.74 . 2 4 35. 1 would rate the general quality of the lectures: 6 34 34 12 7 3 3.99 . o 3 36. 1 would rate the overall value of class discussions: 8 29 '5, •™-» 14 8 4 3.86 .62 37. Overall,' 1 would rate the laboratories: 7 44 12 17 -9 5 5 3.55 .74 38. 1 would rate the overall value of this course to me as: 3 1 -34 35 15 3 £.* 3.91 .60 O m i t s A m o n g t h e t o p 1 0 % A m o n g t h e t o p 3 0 % A b o u l A v t w q e I n t h e l o w e s t 3 0 % I n t h e l o w e s t 1 0 % M e a n * S t a n d a r d D e v i a t i o n * 39. Compared to other instructors you have had (secondary school and college), how effective has the instructor been in this course? 3 5 29 23 5 3 3.93 • 63 7 CD I 57207 • T103P35 • 283564 • P R I N T E D IN U.S.A. September 1973 APPENDIX > TABLE SHOWING COURSE-WISE DISTRI"BUTION"OF STUDENT SAMPLE USED FOR DATA COLLECTION,RE-TRANSLATION AND SCALING COURSE LEVEL DATA COLLECTION ; RE-TRANSLATION AND . . SCALING . TOT AL Male Female Male Female Male Female 200 92 18 48 12 140 30 Under- 300 81 22 18 6 99 28 g r a d u a t e 16 400 16 6 6 500 17 13 15 5 32 18 . G r a d u a t e 600 3 1 2 1 5 2 O t h e r G r a d s . 2 r 1 2 3 2 1—1 : T o t a l 211 60 84 26 295 86 DIMENSIONS (CATEGORIES) OF TEACHER BEHAVIOUR IDENTIFIED FROM INCIDENTS GIVEN BY STUDENTS 1. COURSE OUTLINING AND STRUCTURING Sub- D e f i n i t i o n o f c o u r s e o b j e c t i v e s , w o r k l o a d c a t e g o r i e s : and s t u d e n t e v a l u a t i o n p r o c e d u r e s ; e x t e n t t o w h i c h s t u d e n t s ' c o u r s e e x p e c t a t i o n s and d i f f i c u l t i e s a r e t a k e n i n t o a c c o u n t i n t h i s d e f i n i t i o n ; p r o v i s i o n o f a d e t a i l e d s c h e d u l e f o r t h e l e c t u r e s , d i s c u s s i o n s and a s s i g n m e n t -s u b m i s s i o n s ; e x t e n t t o which s t r u c t u r e and c o n c e p t u a l framework p r o v i d e d i n l e c t u r e s and d i s c u s s i o n s e x t e n t o f r e v i e w and summary o f c o u r s e m a t e r i a l ; and p r o v i s i o n o f l e c t u r e n o t e s o r o t h e r m a t e r i a l . 2. ADMINISTRATIVE HANDLING Sub- Managing and d i r e c t i n g t h e c l a s s i n t h e c a t e g o r i e s : u s u a l c o u r s e and i n e m e r g e n c i e s ; p l a n n i n g and making a v a i l a b l e t e x t books and o t h e r m a t e r i a l ; c o m p l e t i ng p r o m i s e d ' admi i i i s t r a t a Ve work p r o m p t l y and e f f i c i e n t l y ; c o - o r d i n a t i n g t h e c o u r s e w i t h o t h e r s e c t i o n s ; a s s i g n i n g s t u d e n t s t o p r o j e c t teams; making n o n - r o u t i n e d e c i s i o n s l i k e c h a n g i n g c l a s s rooms t o s u i t c l a s s s i z e , c h a n g i n g l e c t u r e h o u r s t o a v o i d c o n f l i c t s o f c o u r s e h o u r s ; e t c . - 1 2 1 -3. COVERAGE OF MATERIAL Sub- The n a t u r e and r e l e v a n c e o f s u b j e c t m a t t e r c a t e g o r i e s : t a u g h t i n t h e c o u r s e ; t h e s c o p e and d e p t h o f c o v e r a g e o f t h e s u b j e c t m a t t e r ; t h e e x t e n t t o w h i c h c o u r s e c o n t e n t i s l i n k e d t o o t h e r r e s e a r c h e s and r e a l - l i f e p r a c t i c e s ; v a r i e t y and r e l e v a n c e o f r e a d i n g m a t e r i a l s u g g e s t e d ; d e g r e e t o w h i c h c o u r s e i s r e l a t e d t o o t h e r d i s c i p l i n e s and e x t e n t t o w h i c h i n s t r u c t o r a c c e p t s new i d e a s s u g g e s t e d by s t u d e n t s . 4. TEACHING STYLE S u D _ E x t e n t o f l o g i c a l framework p r o v i d e d i n c a t e g o r i e s : l e c t u r e s and d i s c u s s i o n s ; a c t u a l l a n g u a g e and j a r g o n s used by t h e i n s t r u c t o r ; e n e r g y and e n t h u s i a s m d i s p l a y e d by him; s t y l e and pace o f t a l k i n g i n c l u d i n g use o f j o k e s , a n a l o g i e s and p e r s o n a l s t o r i e s ; i n f o r m a l i t y and n a t u r a l n e s s o f t h e : r-i n s t r u c t o r ' s b e h a v i o u r ; e m p h a s i s shown by him on. two-way communi-c a t i o n s i n s i d e t h e c l a s s room; and p e r s o n a l h y g i e n e , o u t f i t and a p p e a r a n c e o f t h e i n s t r u c t o r . -122-• 5* TEACHING METHODS Sub- V a r i o u s t e c h n i q u e s and m e d i a used by t h e c a t e g o r i e s : i n s t r u c t o r t o c o n v e y t h e c o u r s e c o n t e n t to t h e s t u d e n t s ; methods u s e d may be l e c t u r e s , c a s e and g r o u p d i s c u s s i o n s , p r o j e c t g r o u p s , r o l e p l a y i n g , games, g u e s t s p e a k e r s , f i e l d t r i p s and use o f b l a c k b o a r d and o t h e r v i s u a l a i d s . 6 . EVALUATION Sub- E x t e n t t o w h i c h e v a l u a t i o n c r i t e r i a i s c a t e g o r i e s : o b j e c t i v e l y d e f i n e d ; e x t e n t t o w h i c h p a p e r s and o t h e r m a t e r i a l a r e g r a d e d p r o m p t l y , o b j e c t i v e l y and e q u i t a b l y ; d e g r e e t o w h i c h t h e i n s t r u c t o r g i v e s f e e d b a c k t o t h e s t u d e n t s a b o u t t h e i r p e r f o r m a n c e ; d e g r e e o f f l e x i b i l i t y s een i n h i s e v a l u -a t i o n p r o c e s s ; and e x t e n t t o w h i c h he g i v e s p o s i t i v e c r i t i c i s m s and e n c o u r a g e m e n t t o h i s s t u d e n t s . 7. INTERACTION OUTSIDE CLASS Sub- E x t e n t t o w h i c h t h e i n s t r u c t o r i s a v a i l a b l e c a t e g o r i e s : f o r - / c o n s u l t a t i o n o u t s i d e c l a s s h o u r s ; d e g r e e o f f r i e n d l i n e s s and i n t e r e s t shown by him t o s t u d e n t s who go t o him w i t h t h e i r p r o b l e m s and i d e a s ; e x t e n t o f h e l p p r o v i d e d by him i n s o l v i n g s t u d e n t s ' c o u r s e / r e s e a r c h / c a r e e r r e l a t e d or o t h e r p e r s o n a l p r o b l e m s ; and -123-d e g r e e o f i n f o r m a l i t y o f h i s b e h a v i o u r i n i n t e r a c t i n g w i t h s t u d e n t s o u t s i d e c l a s s . 8. F L E X I B I L I T Y AND RESPONSIVENESS Sub- E x t e n t t o w h i c h t h e i n s t r u c t o r i s f l e x i b l e c a t e g o r i e s : i n h i s t e a c h i n g s c h e d u l e s , methods and s t y l e ; d e g r e e o f r e s p o n s i v e n e s s shown by him t o s t u d e n t s ' needs and r e q u e s t s ; e x t e n t t o w h i c h he i s aware o f s t u d e n t s ' p r o b l e m s .-and c a p a b i l i t i e s and a d j u s t s h i s s t y l e , pace and d e p t h o f c o v e r a g e t o s o l v e t h e i r p r o b l e m s and s u i t t h e i r c a p a b i l i t i e s ; extent t o w h i c h he changes h i s g o a l s and methods t o s u i t s t u d e n t s ' a s p i r a t i o n s and needs ; and e x t e n t t o w h i c h he c o n s c i o u s l y r e c e i v e s f e e d b a c k a b o u t h i s p e r f o r m a n c e and c o n s t a n t l y s t r i v e s t o i m p r o v e h i s p e r f o r m a n c e . -124-APPENDJX G IV DIMENSIONS' (CATEGORIES) OF)TEACHER BEHAVIOUR IDENTIFIED FROM INCIDENTS GIVEN: BY PROFESSORS INTERACTION WITH COLLEAGUES Sub- B e i n g a v a i l a b l e t o c o l l e a g u e s f o r m i n o r c a t e g o r i e s : c o n s u l t a t i o n s ; e x c h a n g i n g i d e a s and o p i n i o n s w i t h c o l l e a g u e s d u r i n g f o r m a l and i n f o r m a l m e e t i n g s ; m a i n t a i n i n g r a p p o r t w i t h c o l l e a g u e s ; p r o v i d i n g h e l p t o and w o r k i n g a l o n g w i t h o t h e r i n s p e c i f i c p r o j e c t s ; and a c t i n g as a s o u r c e o f i n s p i r a t i o n t o c o l l e a g u e s . 2, INTERACTLON WITH STUDENTS OUTSIDE CLASS Sub- B e i n g a v a i l a b l e t o s t u d e n t s d u r i n g / o u t s i d e c a t e g o r i e s : o f f i c e h o u r s ; h e l p i n g s t u d e n t s i n c o u r s e - r e l a t e d / o t h e r p r o b l e m s ; c o u n s e l l i n g and c a r e e r g u i d a n c e g i v e n t o s t u d e n t s ; and i n t e r a c t i n g i n f o r m a l l y and n a t u r a l l y w i t h s t u d e n t s . 3. BEHAVIOUR IN THE CLASS ROOM Sub- D e f i n i n g t h e c o u r s e o b j e c t i v e s and c a t e g o r i e s : p l a n s f o r each s e s s i o n ; u s i n g a p p r o p r i a t e t e a c h i n g s t y l e and t e a c h i n g methods t o s u i t t h e c l a s s s i z e and r e q u i r e m e n t s . r e s p o n d i n g t o s t u d e n t ' s needs and i n p u t s w i t h i n t h e c l a s s ; -125-p r o v i d i n g c o n s t r u c t i v e f e e d b a c k a b o u t s t u d e n t ' s p e r f o r m a n c e ; and b u i l d i n g and m a i n t a i n i n g r a p p o r t w i t h s t u d e n t s . 4, RESEARCH A C T I V I T I E S Sub- B e i n g i n v o l v e d i n v a r i o u s r e s e a r c h c a t e g o r i e s : a c t i v i t i e s ; and p e r f o r m i n g h i g h q u a l i t y r e s e a r c h . 5. HANDLING ADMINISTRATIVE MATTERS Sub- O r g a n i z i n g and e f f e c t i v e l y u t i l i s i n g c a t e g o r i e s : t o t a l t i m e a v a i l a b l e ; m e e t i n g j o b d e a d l i n e s ; and r e s p o n d i n g t o demands from t h e Dean and t h e a d m i n i s t r a t i v e s t a f f on t i m e and e f f e c t i v e l y . ; - 1 2 6 -APPENDIX H DIMENSIONS (CATEGORIES) OF TEACHER BEHAVIOUR IDENTIFIED FROM INCIDENTS GIVEN BY ADMINISTRATIVE STAFF 1. FACILITATION OF ADMINISTRATIVE WORK FLOW Sub- I n f o r m i n g t h e a d m i n i s t r a t i v e s t a f f a b o u t c a t e g o r i e s : h i s r e q u i r e m e n t s / d e m a n d s w e l l i n a d v a n c e ; c o m m u n i c a t i n g t o t h e a d m i n i s t r a t i v e s t a f f a b o u t c h a nges i n h i s t i m e s c h e d u l e s p r o m p t l y ; and b e i n g f l e x i b l e i n h i s demands f o r e q u i p m e n t / s e r v i c e s . 2. CONTROLLING EXPENDITURES Sub- Not s p e n d i n g o u t s i d e t h e b u d g e t c o n s t r a i n t s c a t e g o r i e s : imposed by t h e f a c u l t y ; n o t p u r c h a s i n g low p r i o r i t y i t e m s o f e q u i p m e n t ( o r s e r v i c e ) ; ,•. n ot a u s i n g f a c i l i t i e s f o r p r i v a t e e n d s ; not w a s t i n g m a t e r i a l s ; and not h o a r d i n g e q u i p e m n t s . 3. ADHERENCE TO POLICIES Sub- C o n f o r m i n g t o e x i s t i n g a d m i n i s t r a t i v e c a t e g o r i e s : p o l i c i e s and Dean's d i r e c t i v e s ; a d h e r i n g t o a l l r e s t r a i n t s ( e x c e p t f i n a n c i a l r e s t r a i n t s , coming u n d e r 2 above) imposed by t h e F a c u l t y o f Commerce i n d a y - t o - d a y o p e r a t i o n s . c o n f o r m i n g t o a l l p r o c e d u r a l g u i d e l i n e s g i v e n by t h e F a c u l t y o f Commerce; and r e s p o n d i n g p r o m p t l y to t h e demands and d i r e c t i v e s o f t h e f a c u l t y . -127-4. PROVIDING FEEDBACK TO THE STAFF Sub P r o v i d i n g f e e d b a c k t o t h e a d m i n i s t r a t i v e c a t e g o r i e s : s t a f f a b o u t t h e i r p e r f o r m a n c e and s e r v i c e s i n a c o n s t r u c t i v e and p l e a s a n t manner. 5. COUNSELLING A C T I V I T I E S Sub P r o v i d i n g helpful a d v i c e t o s t u d e n t s on c a t e g o r i e s : t h e c h o i c e o f c o u r s e s and a c a d e m i c programs ; k e e p i n g r e c o r d s o f s t u d e n t s ' a c a d e m i c p e r f o r m a n c e and o t h e r r e l e v a n t d e t a i l s t o f a c i l i t a t e c o u n s e l l i n g by o t h e r s ; and b e i n g f l e x i b l e i n own s t a n d a r d s t o s u i t t h e d i f f e r i n g needs o f s t u d e n t s . APPENDIX I RESULTS OF THE TEST FOR COMPREHENSIVENESS OF THE INCIDENTS COLLECTED ,••';:"[. C u m u l a t i v e # o f Number of group s u b - c a t e g o r i e s a c c o u n t e d f o r by each g r o u p 1 16 2 7 3 5 4 2 5 2 6 2 7 0 8 1 9 1 10 0 11 1 12 1 13 o 14 0 15 1 16 -•.-.20 0 T o t a l 39 -129-APPENDIX J BEHAVIOUR SCALES I TO BE FILLED 'IN BY STUDENTS Name o f t h e i n s t r u c t o r b e i n g e v a l u a t e d T i t l e o f t h e c o u r s e Date o f e v a l u a t i o n Note F i g u r e s ' i n p a r a n t h e s i s a g a i n s t each i t e m i n t h e s c a l e d e n o t e t h e mean s c o r e o f t h e p a r t i c u l a r b e h a v i o u r . T h e s e f i g u r e s w i l l n o t be shown when u s i n g t h e " s c a l e s - for.: e v a l u a t i n g "prof e s s o r s . -130-T h i s s e t o f s c a l e s i s d e s i g n e d t o g i v e you an o p p o r t u n i t y t o express- anonymously y o u r v i e w s on t h e b e h a v i o u r of; y o u r i n s t r u c t o r d u r i n g t h e c o n d u c t o f t h i s c o u r s e . T h e r e a r e 7 s c a l e s i n a l l , e a ch m e a s u r i n g one d i m e n s i o n o f t h e i n s t r u c t o r ' s b e h a v i o u r . A p p r o x i -m a t e l y t e n b e h a v i o u r s a r e g i v e n i n e a c h s c a l e , o u t o f w h i c h you have t o c h o o s e one which i s most t y p i c a l o f y o u r i n s t r u c t o r . P l e a s e n o t e t h a t t h e i t e m s a r e n o t l i s t e d i n any p a r t i c u l a r o r d e r d e n o t i n g e f f e c t i v e p r ,i n e f f e c t i v e b e h a v i o u r . B e f o r e you t u r n ; t h i s page, p l e a s e c o m p l e t e t h e q u e s t i o n g i v e n b e l o w . You may mark any where on t h e s c a l e g i v e n . OVERALL, I WOULD RATE THE INSTRUCTOR AS 1 2 3 4 5 6 7 E x t r e m e l y i n e f f e c t i v e n e i t h e r e f f e c t i v e n o r i n e f f e c t i ve e x t r e m e l y e f f e c t i ve -131-SCALE A: COURSE OUTLINING AND STRUCTURING In h i s j o b as i n s t r u c t o r , how e f f e c t i v e i s t h i s p e r s o n i n o u t l i n i n g and s t r u c t u r i n g t h e c o u r s e ? C o n s i d e r h i s t y p i c a l j o b b e h a v i o u r and t i c k a g a i n s t any one b e h a v i o u r w h i c h f i t s him most c l o s e l y i n h i s d a y - t o - d a y b e h a v i o u r . C o u l d be e x p e c t e d t o g i v e c o u r s e o u t l i n e s and Q s c h e d u l e s t o s t u d e n t s . (6.09) C o u l d be e x p e c t e d t o t a k e q u i t e a few days t o r^, t e l l t h e s t u d e n t s what i s t h e c o u r s e c o n t e n t . ( 2 . 8 3 ) L-T C o u l d be e x p e c t e d t o s e t s p e c i f i c t a r g e t s f o r each s e s s i o n . (5.18) C o u l d be e x p e c t e d t o announce mid-way t h r o u g h term r — i t h a t t h e r e would be a f i n a l exam, i n c o n t r a d i c - j - — ' t i o n t o h i s e a r l i e r s t a t e m e n t t h a t t h e r e would be no f i n a l . (1.26) C o u l d be e x p e c t e d t o h o l d an i n t r o d u c t o r y s e s s i o n i n w h i c h t h e ^ s t u d e n t s ' e x p e c t a t i o n s i I a r e a s c e r t a i n e d and i n s t r u c t o r ' s c o u r s e '—-1 o b j e c t i v e s a r e made c l e a r . (5.95) C o u l d be e x p e c t e d t o s e t c l a s s p a r t i c i p a t i o n as an e v a l a a t i o n - c r i t e r i o n w i t h o u t c l e a r l y s t a t i n g h i s e x p e c t a t i o n s . (2.17) * • C o u l d be e x p e c t e d t o have no d i r e c t i o n i n , J~j c o v e r a g e o f m a t e r i a l and cl.ass room d i s c u s s i o n s . (1.59) C o u l d be e x p e c t e d t o e x p l a i n i n d e t a i l t h e j j c o u r s e work l o a d and h i s e x p e c t a t i o n s . (6.01) — 1 CouTd be e x p e c t e d t o s p e c i f i c a l l y s t a t e .—. r e q u i r e m e n t s f o r t h e c o u r s e and use d e f i n i t e | | and s t a t e d c r i t e r i a f o r e v a l u a t i o n . (6.23) -132-SCALE B: COVERAGE-OF METERIAL In h i s j o b as i n s t r u c t o r , how e f f e c t i v e i s t h i s p e r s o n i n t h e c o v e r a g e o f r e l e v a n t m a t e r i a l i n t h e c o u r s e ? C o n s i d e r h i s t y p i c a l j o b b e h a v i o u r and t i c k a g a i n s t any one b e h a v i o u r w h i c h f i t s him most c l o s e l y i n h i s d a y - t o - d a y b e h a v i o u r . C o u l d be e x p e c t e d t o a p p l y t h e o r i e s and p r i n c i -p l e s t o c u r r e n t e v e n t s . (6.09) C o u l d be e x p e c t e d t o e x p l a i n a majo r p a p e r v e r y p o o r l y ; t . h u s Jco.nfu;si.ntj::miQSjt o f :the stu.dents.:(1:.^1.: C o u l d be e x p e c t e d n o t t o c o - o r d i n a t e v a r i o u s t h e o r i e s i n t o a t o t a l f r a m e work. (2.10) C o u l d be e x p e c t e d t o c o v e r t e x t book m a t e r i a l t o a v e r y g r e a t d e g r e e . (4.40) C o u l d be e x p e c t e d to hand o u t o u t l i n e s o f c u r r e n t c l a s s d i s c u s s i o n s . (5.21) C o u l d be e x p e c t e d t o p r o v i d e e m p i r i c a l d a t a and r e l a t e d a r t i c l e s s u p p o r t i n g t h e v a r i o u s t h e o r i e s d i s c u s s e d by him. (5.88) C o u l d be e x p e c t e d t o a s s i g n volumes o f r e a d i n g m a t e r i a l w i t h o u t d i s c u s s i n g them i n c l a s s . ( 1 . 6 3 ) C o u l d be e x p e c t e d t o h o l d h i s s t u d e n t s r e s p o n -s i b l e f o r m a t e r i a l o t h e r t h a n i n t h e t e x t book. O f t e n t h e s e m a t e r i a l a r e n o t a v a i l a b l e . ( 1 . 1 9 ) C o u l d be e x p e c t e d t o skim t h r o u g h t h e e n t i r e c o u r s e c o n t e n t w i t h o u t g o i n g i n t o any d e t a i l on any s p e c i f i c p o i n t . (1.86) C o u l d be e x p e c t e d t o r e l a t e t h e c o u r s e t o o t h e r d i s c i p i i n e s . (5.60) -133-SCALE C: TEACHING STYLE How e f f e c t i v e i s t h i s i n s t r u c t o r ' s t e a c h i n g s t y l e ? C o n s i d e r h i s t y p i c a l j o b b e h a v i o u r and t i c k a g a i n s t any one ' b e h a v i our w h i c h i s t y p i c a l o f him. C o u l d be e x p e c t e d t o p r o v i d e r e a l - l i f e and i — i p e r s o n a l e x a m p l e s , t o e x p l a i n a concept.(6.23) I—I C o u l d be e x p e c t e d to speak w i t h c o n s i d e r a b l e e n e r g y and e n t h u s i a s m . (5.88) C o u l d be e x p e c t e d to e m p h a s i z e on as many s t u d e n t s p a r t i c i p a t i n g i n c l a s s d i s c u s s i o n s r a t h e r t h a n on g e t t i n g r i g h t answers.(5.39) • C o u l d be e x p e c t e d t o g i v e d u l l l e c t u r e s w h i c h r~j a r e r e p e t i t i v e o f t h e t e x t book. (2.10) C o u l d be e x p e c t e d t o use a b s t r a c t and d i f f i c u l t l a n g u a g e i n h i s 1ectures.(1•75) • C o u l d be e x p e c t e d t o spend c o n s i d e r a b l e c l a s s I I t i m e i n s t a t i n g h i s p e r s o n a l v i e w s on 1 — 1 d i f f e r e n t m a t t e r s . (3.71) C o u l d be e x p e c t e d t o e n c o u r a g e s t u d e n t s t o a s k F~) q u e s t i o n s and answer t h e i r q u e s t i o n s w i t h o u t p u t t i n g them down. (6.44) C o u l d be e x p e c t e d t o t a k e c o n s i d e r a b l e e f f o r t P~l to l e a r n names o f s t u d e n t s and engage i n ' — c o n v e r s a t i o n s . (5.81) Coiuld be e x p e c t e d t o t r e a t h i s s t u d e n t s as e q u a l s w i t h i n c l a s s . (5.46) C o u l d be e x p e c t e d t o s e c u r e p a r t i c i p a t i o n o f O i n d i v i d u a l s t u d e n t s by a s k i n g them s p e c i f i c q u e s t i o n s . (4.69) -134-SCALE D: TEACHING METHODS How e f f e c t i v e a r e t h e t e a c h i n g methods us e d by t h i s i n s t r u c t o r ? C o n s i d e r h i s t y p i c a l j o b b e h a v i o u r and t i c k a g a i n s t any one b e h a v i o u r w h i c h f i t s him most c l o s e l y i n h i s d a y - t o - d a y b e h a v i o u r . C o u l d be e x p e c t e d t o use a v a r i e t y o f t e a c h i n g ,—. a i d s ( f i l m s , c a s e s , d i s c u s s i o n s , r o l e p l a y s , < _ j e t c . ) t o c o n v e y t h e s u b j e c t m a t t e r . ( 5 . 6 0 ) • C o u l d be e x p e c t e d to b r i n g g u e s t s p e a k e r s who a r e d o i n g e i t h e r r e s e a r c h o r p r a c t i c a l work. (5.6) C o u l d be e x p e c t e d t o put s t u d e n t s on t h e ' s p o t ' ; — . by a s k i n g q u e s t i o n s t o i n d i v i d u a l members.(4.20) i _ J C o u l d be e x p e c t e d t o use o u t d a t e d f i l m s . ( 2 . 4 5 ) j ~ j C o u l d be e x p e c t e d t o use d i f f e r e n t t e a c h i n g r—-. methods t o c l a s s e s o f d i f f e r e n t s i z e . ( 6 . 2 3 ) L J C o u l d be e x p e c t e d t o spend whole c l a s s hour j—i r e a d i n g o u t an a r t i c l e f r o m a m a g a z i n e . (1.41) I—I C o u l d be e x p e c t e d t o a s k i n d i v i d u a l s t u d e n t i — i to p r e p a r e l e c t u r e l a s t i n g a p p r o x i m a t e l y t e n .'—' m i n u t e s and d e f e n d h i s v i e w s . (3.36) C o u l d be e x p e c t e d t o show good and r e l e v a n t Q f i l m s . (5.6) C o u l d be e x p e c t e d t o s p l i t t h e c l a s s i n t o I—• s m a l l g r o u p s when d i s c u s s i n g c a s e s . (5.6) L J -135-SCALE E: EVALUATION In h i s j o b as i n s t r u c t o r , how e f f e c t i v e i s t h i s p e r s o n i n e v a l u a t i n g h i s s t u d e n t s ? C o n s i d e r h i s t y p i c a l j o b b e h a v i o u r and t i c k a g a i n s t any one b e h a v i o u r w h i c h f i t s him most c l o s e l y i n h i s d a y - t o - d a y b e h a v i o u r . C o u l d be e x p e c t e d t o p r o v i d e f e e d b a c k a b o u t j — , p oor p e r f o r m a n c e t o s t u d e n t s i n a s u p p o r t i v e L_J manner w i t h o u t c r i t i c i z i n g . (5.91) C o u l d be e x p e c t e d n o t to p r o v i d e mark d i s t r i - Q b u t i o n o f t h e c o u r s e . (2.21) C o u l d be e x p e c t e d t o m o t i v a t e s t u d e n t s by a l l o t i n g j — i e x t r a ' m a r k s f o r : d o i n g e x t r a r e a d i n g s . (4.27) , I—I C o u l d be e x p e c t e d to e x p r e s s i n t e r e s t i n p e r s o n a l o p i n i o n s and o r i g i n a l i t y o f t h o u g h t , Q b u t g i v e h i g h g r a d e s o n l y when answers c o i n c i d e w i t h h i s own v i e w s . (1.68) C o u l d be e x p e c t e d to p r o v i d e c o n s t r u c t i v e comments r—y and c r i t i c i s m s . o n ,exams, -papers , e t c . ( 6 . 22) . i—I C o u l d be e x p e c t e d t o s t a t e ( i n t h e b e g i n n i n g ) t h a t a s s i g n m e n t o f g r a d e s w i l l be d e c i d e d i n c o n s u l t a t i o n w i t h t h i s s t u d e n t s and l a t e r d e c i d e t h e m a t t e r a l l by h i m s e l f . (2.70) C o u l d be e x p e c t e d t o p r o v i d e t o o much bonus J I m arks. ( ( 3 . 3 6 ) U C o u l d be e x p e c t e d t o e v a l u a t e p a p e r s u s i n g o b j e c t i v e c r i t e r i a and f a i r s t a n d a r d s . ( 6 . 4 1 ) • - 1 3 6 -SCALE F: INTERACTION OUTSIDE In h i s j o b as i n s t r u c t o r , how e f f e c t i v e i s t h i s p e r s o n i n h i s i n t e r a c t i o n s w i t h s t u d e n t s o u t i s d e c l a s s ? C o n s i d e r a l l t h e f o l l o w i n g i t e m s and t i c k a g a i n s t any one i t e m w h i c h shows h i s t y p i c a l b e h a v i o u r . C o u l d be e x p e c t e d t o p r o v i d e c a r e e r c o n s u l t a -t i o n s to s t u d e n t s who ask f o r h i s h e l p i n t h i s a r e a . (5.04) C o u l d be e x p e c t e d n o t to d i s c l o s e h i s o f f i c e h o u r s . (1.33) C o u l d be e x p e c t e d t o p r o v i d e o f f i c e s p a c e f o r s t u d e n t s who work under him. (3.71) • • C o u l d be e x p e c t e d to h e l p s t u d e n t s who a r e .—> hawing d i f f i c u l t i e s w i t h t h e i r a s s i g n m e n t s . ( 5 . 9 8 ) I—5 C o u l d . b e e x p e c t e d t o t a k e an a c t i v e i n t e r e s t i n t h e a c a d e m i c f u t u r e o f h i s s t u d e n t s and L J p r o v i d e c o n c r e t e h e l p t o t h e s t u d e n t s who go and meet him f o r g u i d a n c e . (6.30) C o u l d be e x p e c t e d t o t a k e an a c t i v e i n t e r e s t i n s t u d e n t s ' p r o b l e m s and h e l p them i n s o l v i n g Q t h e s e p r o b l e m s o u t s i d e c l a s s h o u r s . ( 5 . 9 5 ) • C o u l d be e x p e c t e d n o t t o keep up h i s o f f i c e a p p o i n t m e n t s ; i s h a b i t u a l l y l a t e o r does n o t show up a t a l 1 . ( 1 . 1 2 ) C o u l d be e x p e c t e d t o i n t e r a c t s o c i a l l y w i t h f"~J s t u d e n t s o u t s i d e c l a s s . (4.20) C o u l d be e x p e c t e d to i n t e r v i e w s t u d e n t s b e f o r e ,—, t h e y r e g i s t e r f o r h i s c o u r s e and g i v e them an I I i d e a o f t h e c o n t e n t and work l o a d o f t h e c o u r s e . ( 4 . 9 0 ) C o u l d be e x p e c t e d t o be i n c o n s i d e r a t e and . i n s e n s i t i v e t o s t u d e n t ' s p r o b l e m s and r e q u e s t s L J and r u s h s t u d e n t o u t o f h i s o f f ic e . . (1 • 19) ' -137- . . . / : • • . SCALE G: F L E X I B I L I T Y AND RESPONSIVENESS In h i s j o b as i n s t r u c t o r / how f l e x i b l e and r e s p o n s i v e i s t h i s p e r s o n ? C o n s i d e r h i s t y p i c a l j o b b e h a v i o u r and t i c k a g a i n s t any one o f t h e f o l l o w i n g i t e m s . C o u l d be e x p e c t e d t o g r a n t s t u d e n t s ' r e q u e s t s f o r a d d i t i o n a l t i m e f o r c o m p l e t i n g c e r t a i n a s s i g n m e n t s . (4.69) • C o u l d be e x p e c t e d t o ask f o r s t u d e n t i n p u t [ | r e g a r d i n g t h e i r c o u r s e e x p e c t a t i o n s and a r e a s o f i n t e r e s t . (5.60) C o u l d be e x p e c t e d to t e l l t h e c l a s s how t h e ,—, p a r t i c u l a r f i l m ends when a f i l m b e i n g shown I I i n t h e c l a s s b r e a k s i n b e t w e e n . (4.06) C o u l d be e x p e c t e d n o t t o p r o v i d e any e x p l a n a -t i o n s a b o u t an u n f a i r exam w h i c h he gave and j | ask t h e s t u d e n t s t o meet t h e Dean i f t h e y have any c o m p l a i n t s . (1.30) C o u l d be e x p e c t e d to pause d u r i n g h i s l e c t u r e s t o make s u r e a l l t h e s t u d e n t s i n t h e c l a s s Q a r e f o l l o w i n g him; r e p e a t s m a j o r p o i n t s . ( 6 . 0 9 ) C o u l d be e x p e c t e d t o be open to s u g g e s t i o n s . — r e g a r d i ng c o u r s e c o n t e n t and t e a c h i n g m e t h o d s ; I .1 he i s n o t annoyed when he r e c e i v e s a d v e r s e comments on h i s t e a c h i n g methods. (6.02) C o u l d be e x p e c t e d t o t e l l t h e s t u d e n t s t h a t r__ he does not l i k e t e a c h i n g them and t h a t he ]__! d i d n o t v a l u e t h e i r o p i n i o n s a b o u t him o r h i s t e a c h i n g methods. (1.05) • C o u l d be e x p e c t e d t o hand o u t s u g g e s t i o n s h e e t s a t t h e end o f each c l a s s t o s t u d e n t s f o r w r i t i n g t h e i r comments on t h e day's l e c t u r e . A l s o t o d i s c u s s t h e s e comments a t t h e b e g i n n i n g o f n e x t c l a s s - (3.78) C o u l d , be e x p e c t e d t o s t o p h i s l e c t u r e s t o answer •,: q u e s t i o n s by s t u d e n t s even when i t j j i n t e r f e r e - s ? w i t h h i s p l a n s f o r t h e day. ( 5 . 1 1 ) — - 1 3 8 -APPENDIX K BEHAVIOUR SCALES I I TO BE FILLED IN BY FACULTY MEMBERS Name o f t h e i n s t r u c t o r b e i n g e v a l u a t e d D e s i g n a t i o n Date o f e v a l u a t i o n Note F i g u r e s i n p a r a n t h e s i s a g a i n s t each i t e m i n t h e s c a l e d e n o t e t h e mean s c o r e o f t h e p a r t i c u l a r b e h a v i o u r . T h e s e f i g u r e s w i l l n o t be shown when u s i n g t h e s c a l e s f o r e v a l u a t i n g p r o f e s s o r s -139-Th i s s e t o f s c a l e s i s d e s i g n e d t o e x p r e s s a n o n y m o u s l y y o u r views on t h e b e h a v i o u r o f y o u r c o l l e a g u e d u r i n g t h e l a s t s e m e s t e r . T h e r e a r e t h r e e s c a l e s i n a l l , e ach m e a s u r i n g one d i m e n s i o n o f a p r o f e s s o r ' s b e h a v i o u r . From t h e l i s t o f b e h a v i o u r s d e s c r i b e d i n e a c h s c a l e , you have t o c h o o s e one i t e m w h i c h i s most t y p i c a l o f y o u r c o l l e a g u e . P l e a s e n o t e t h a t t h e i t e m s a r e not l i s t e d i n any p a r t i c u l a r o r d e r d e n o t i n g e f f e c t i v e or: i n e f f c c t i ve . b e h a v i o u r . . B e f o r e you t u r n t h i s p age, p l e a s e c o m p l e t e t h e q u e s t i o n g i v e n below. You: may mark any where on t h e s c a l e g i v e n . OVERALL, I WOULD RATE MY COLLEAGUE AS . 7 . e x t r e m e l y e f f e c t i ve E x t r e m e l y n e i t h e r i n e f f e c t i v e e f f e c t i v e .. nor i n e f f e c t i ve -140-SCALE A: INTERACTION WITH COLLEAGUES In h i s d a y - t o - d a y l i f e , how does t h i s p e r s o n i n t e r a c t w i t h h i s c o l l e a g u e s ? C o n s i d e r h i s t y p i c a l b e h a v i o u r and t i c k a g a i n s t any one b e h a v i o u r w h i c h f i t s . h i m most c l o s e l y i n h i s d a y - t o - d a y b e h a v i o u r . ; C o u l d be e x p e c t e d t o p r o v i d e h i s c o l l e a g u e s .—v w i t h h i s t r u e and f r a n k o p i n i o n s on o t h e r ' s 1 ! p e r f o r m a n c e and b e h a v i o u r . ( 6 . 0 ) C o u l d be e x p e c t e d t o keep h i m s e l f away f r o m Q a l l h i s c o l l e a g u e s . ( 1 . 5 ) C o u l d be e x p e c t e d to h e l p c o l l e a g u e s by ,—, a s k i n g p e r t i n e n t q u e s t i o n s and p r o v i d i n g L J s u g g e s t i o n s f o r i m p r o v i n g t h e i r p e r f o r m a n c e s . ( 5 . 5 ) C o u l d be e x p e c t e d t o go t h r o u g h i n d e t a i l a i — i m a n u s c r i p t o r w o r k i n g p a p e r g i v e n by c o l l e a g u e '—' and o f f e r a p p r o p r i a t e comments. ( 7 . 0 ) C o u l d be e x p e c t e d t o . b e r e a d i l y a v a i l a b l e j — i f o r c o n s u l t a t i o n by h i s c o l l e a g u e s a t a l l '—* t i m e s . ( 6 . 2 5 ) -141-SCALE B: RESEARCH A C T I V I T I E S In h i s d a y - t o - d a y l i f e how does t h i s p e r s o n c a r r y o u t r e s e a r c h . C o n s i d e r h i s t y p i c a l b e h a v i o u r and t i c k a g a i n s t any one b e h a v i o u r w h i c h f i t s him most c l o s e l y , C o u l d be e x p e c t e d t o p u b l i s h a l a r g e number o f r e s e a r c h p a p e r s i n an a c a d e m i c y e a r . ( 6 . 0 ) C o u l d be e x p e c t e d t o be m e t h o d o l o g i c a l l y r i g o r o u s i n h i s r e s e a r c h . (6.25) C o u l d be e x p e c t e d n o t to use sound r e s e a r c h methods and a n a l y t i c a l t e c h n i q u e s i n h i s r e s e a r c h e f f o r t s . (1.0) • • C o u l d be e x p e c t e d t o e m p h a s i s e on q u a n t i t y • — i o f r e s e a r c h p u b l i c a t i o n s t o h i s c r e d i t , I | o f t e n o v e r l o o k i n g t h e i r q u a l i t y . (.1.50) • -142-SCALE C: INTERACTION WITH STUDENTS OUTSIDE CLASS (AS SEEN BY HIS COLLEAGUES) In h i s d a y - t o - d a y l i f e , how does t h i s i n d i v i d u a l i n t e r a c t w i t h h i s s t u d e n t s o u t s i d e t h e c l a s s ? C o n s i d e r t h e t y p i c a l b e h a v i o u r o f t h i s p e r s o n and t i c k a g a i n s t any one b e h a v i o u r w h i c h f i t s him most c l o s e l y . C o u l d be e x p e c t e d t o t r e a t h i s s t u d e n t s as Q e q u a l s o u t s i d e t h e c l a s s . (5.5) C o u l d be e x p e c t e d t o be not a v a i l a b l e f o r j [ c o n s u l t a t i o n by h i s s t u d e n t s . (1.0) C o u l d be e x p e c t e d t o g i v e h i s t r u e and f r a n k o p i n i o n s a b o u t t h e s t u d e n t s ' p e r f o r m a n c e s t o them. (6.0) • C o u l d be e x p e c t e d t o l i s t e n t o a l l p r o b l e m s posed by h i s s t u d e n t s ( o u t s i d e c l a s s ) e v e n [ j when some o f t h e p r o b l e m s a r e not r e l a t e d t o h i s c o u r s e . (6.0) - 1 4 3 -APPENDIX L BEHAVIOURAL SCALES I I I TO BE FILLED IN BY THE ADMINISTRATIVE STAFF Name o f t h e i n s t r u c t o r b e i n g e v a l u a t e d D e s i g n a t i on Date o f e v a l u a t i o n Note F i g u r e s i n p a r a n t h e s i s a g a i n s t e a c h i t e m i n t h e s c a l e d e n o t e t h e mean s c o r e o f t h e p a r t i c u l a r b e h a v i o u r . T h e s e f i g u r e s w i l l not. be shown when u s i n g t h e s c a l e s f o r e v a l u a t i n g p r o f e s s o r s . -144-Th i s s e t o f s c a l e s i s d e s i g n e d t o e n a b l e you t o e x p r e s s anonymously y o u r v i e w s on t h e b e h a v i o u r o f p a r t i c u l a r i n s t r u c t o r s . T h e r e a r e 5 s c a l e s i n a l l , e a c h m e a s u r i n g one d i m e n s i o n o f t h e i n s t r u c t o r ' s b e h a v i o u r s . A p p r o x i m a t e l y 6 i n s t r u c t o r b e h a v i o u r s a r e l i s t e d under each d i m e n s i o n , o u t o f w h i c h you a r e t o c h o o s e any one w h i c h i s most t y p i c a l o f t h e i n s t r u c t o r whom you a r e e v a l u a t i n g . P l e a s e n o t e t h a t t h e i t e m s a r e n o t l i s t e d i n any p a r t i c u l a r o r d e r d e n o t i n g e , f f e c t i v e o r i n e f f e c t i v e b e h a v i o u r . B e f o r e you t u r n t h i s page, p l e a s e c o m p l e t e t h e q u e s t i o n g i v e n below. You may mark any where on t h e g i v e n s c a l e . OVERALL, I WOULD RATE THE INSTRUCTOR AS 1 2 3 4 5 6 7 E x t r e m e l y i n e f f e c t i v e n e i t h e r e f f e c t i v e n o r i n e f f e c t i ve e x t r e m e l y e f f e c t i v e -145-SCALE A: FACILITATION OF ADMINISTRATIVE WORK FLOW In h i s d a y - t o - d a y l i f e , how much does t h i s p e r s o n f a c i l i t a t e t h e smooth f l o w o f a d m i n i s t r a t i v e p r o c e d u r e s ? C o n s i d e r h i s t y p i c a l b e h a v i o u r and t i c k a g a i n s t any one i t e m below w h i c h f i t s him most c l o s e l y . C o u l d be e x p e c t e d t o i n f o r m t h e a d m i n i s t r a - .—t t i v e s t a f f p r o m p t l y a b o u t changes i n h i s I—I l e c t u r e t i m i n g s , e x a m i n a t i o n s c h e d u l e s and o t h e r p r o c e d u r a l , m a t t e r s . (5.0) C o u l d be e x p e c t e d t o change h i s c l a s s : j— j t i m i n g s f r e q u e n t l y w i t h o u t g i v i n g s u f f i - 1—I c i e n t advance n o t i c e t o t h e a d m i n i s t r a t i v e s e c t i o n . (2.5) C o u l d be e x p e c t e d t o t u r n i n ' r u s h ' j o b s to t h e a d m i n i s t r a t i v e s e c t i o n f r e q u e n t l y Q w i t h o u t any p r i o r a s s e s s m e n t o f t h e s e c t i o n ' s p r e s e n t w o r k l o a d and commitments. ( 3 . 0 ) C o u l d be e x p e c t e d ,not to g i v e a d vance n o t i c e o f h i s r e q u i r e m e n t s f o r a p a r t i c u l a r e q u i p m e n t o r s e r v i c e . (3.5) C o u l d be e x p e c t e d t o hang on t o e q u i p m e n t s (many o f w h i c h a r e i n s h o r t s u p p l y ) e v e n a f t e r h i s u s e . (2.0) • C o u l d be e x p e c t e d t o g i v e a t l e a s t 3 d a y s r—, n o t i c e o f h i s r e q u i r e m e n t s f o r a p a r t i c u - I—I l a r e q u i p m e n t o r s e r v i c e . (5.5) • C o u l d be e x p e c t e d to a p p r e c i a t e t h e f a c t t h a t c e r t a i n e q u i p m e n t s and s e r v i c e s a r e F~j i n s h o r t s u p p l y and be f l e x i b l e i n h i s demands f o r them, (5.0) -146-SCALE B: CONTROLLING EXPENDITURES In h i s d a y - t o - d a y l i f e how w e l l does t h i s p e r s o n c o n t r o l h i s own o f f i c i a l e x p e n d i t u r e s and f a c i l i t a t e t h e a d m i n i s t r a t o r i n d o i n g t h e same? C o n s i d e r t h e i n s t r u c t o r ' s t y p i c a l b e h a v i o u r and t i c k a g a i n s t any one i t e m below w h i c h f i t s him most c l o s e l y . C o u l d be e x p e c t e d t o go o u t and buy an __ e x p e n s i v e p i e c e o f e q u i p m e n t and l a t e r I | ask t h e f a c u l t y t o pay f o r i t w i t h o u t a d e q u a t e l y j u s t i f y i n g h i s a c t i o n . ( 2.0) C o u l d be e x p e c t e d t o p u r c h a s e l a r g e i— j volume o f low p r i o r i t y i t e m s . (3.0) I—I C o u l d be e x p e c t e d t o i n s i s t on h a v i n g h i s ,—\ own t y p e w r i t e r even when t h e f a c u l t y I I c a n n o t a f f o r d i t . (2.5) C o u l d be e x p e c t e d to make e f f o r t s i n r e c o v -e r i n g money owed by o u t s i d e i n d i v i d u a l s P I and a g e n c i e s to t h e f a c u l t y . (5.0) I -147-SCALE C: ADHERENCE TO POLICIES In h i s d a y - t o - d a y b e h a v i o u r , t o what d e g r e e does t h i s p e r s o n a d h e r e t o p l a n s and p o l i c i e s o f t h e f a c u l t y ? C o n s i d e r t h e i n s t r u c t o r ' s t y p i c a l b e h a v i o u r and t i c k a g a i n s t any one o f t h e f o l l o w i n g i t e m s w h i c h t i t s him most c l o s e l y . C o u l d be e x p e c t e d t o r e q u e s t f r e q u e n t l y .<;—i f o r e x e m p t i o n f r o m some r u l e s w h i c h a r e }—I r e a l l y p o l i c i e s o f t h e f a c u l t y . (3.5) C o u l d be e x p e c t e d t o i n t e r a c t w i t h P-l e x t e r n a l a g e n c i e s d i r e c t l y i n c o n t r a d i c t i o n I—J t o f a c u l t y p o l i c y . (2-0) C o u l d be e x p e c t e d t o a d h e r e t o t h e p o l i c i e s — . o f t h e f a c u l t y i n a l l h i s d a y - t o - d a y i J b e h a v i o u r . (6.0) C o u l d be e x p e c t e d not t o comply w i t h Dean's ,—. i n s t r u c t i o n s r e l a t e d t o c e r t a i n p r o c e d u r a l - j I m a t t e r s . (3.0) C o u l d be e x p e c t e d t o r e s p o n d p r o m p t l y t o t h e r e q u e s t s and c h a n g e s i n p o l i c i e s o f [ ] G r a d u a t e and U n d e r g r a d u a t e o f f i c e s . ( 6 - ° ) -148-SCALE D: PROVIDING FEEDBACK TO STAFF In h i s d a y - t o - d a y l i f e , how good i s t h i s p e r s o n i n p r o v i d i n g c o n s t r u c t i v e f e e d b a c k to t h e a d m i n i s t r a t i v e s t a f f . C o n s i d e r t h e i n s t r u c t o r ' s t y p i c a l b e h a v i o u r and t i c k a g a i n s t any one o f t h e f o l l o w i n g i t e m s w h i c h f i t s him most c l o s e l y . C o u l d be e x p e c t e d t o a p p r e c i a t e good work .—. done by a d m i n i s t r a t i v e s t a f f by g i v i n g ! _ J /Verbal o r w r i t t e n f e e d b a c k . ( 6 . 0 ) C o u l d be e x p e c t e d not t o g i v e any s p e c i f i c f e e d b a c k to t h e s t a f f a b o u t t h e i r o r r — j t h e i r s u b o r d i n a t e s ' p e r f o r m a n c e . He l a t e r '—' makes b l a n k e t a c c u s a t i o n s a b o u t t h e i n e f f i -c i e n c y o f t h e s y s t e m . (1.5) C o u l d be e x p e c t e d t o m e r e l y c r i t i c i z e some work done w i t h o u t p r o v i d i n g any s p e c i f i c f~I e x p l a n a t i o n s o r c o n s t r u c t i v e s u g g e s t i o n s . ( 2 . 5 ) C o u l d be e x p e c t e d t o a p p r e c i a t e : good work r—, done by s e n d i n g f l o w e r s and c h o c o l a t e s . ( 4 . 5 ) L J -149-SCALE E: MAINTAINING STUDENT RECORDS AND PROVIDING COUNSELLING * In h i s r o l e as i n s t r u c t o r how w e l l does t h i s p e r s o n m a i n t a i n r e c o r d s o f h i s s t u d e n t s and p r o v i d e c o u n s e l l i n g t o them? C o n s i d e r h i s t y p i c a l b e h a v i o u r and t i c k a g a i n s t any one o f t h e f o l l o w i n g i t e m s w h i c h f i t s him most c l o s e l y . C o u l d be e x p e c t e d t o c i r c u l a t e h i s s t u d e n t - j j f i l e s w i t h o u t any d e l a y . ( 5 . 5 ) '— C o u l d be e x p e c t e d t o a r b i t r a r l y d e c i d e t h e p r e - r e q u i s i t e s f o r h i s s t u d e n t s o f t e n t o Q t h e u l t i m a t e d i s a d v a n t a g e o f t h e s t u d e n t s . ( 2 . 0 ) C o u l d be e x p e c t e d t o m a i n t a i n m e t i c u l o u s j — i r e c o r d s o f h i s s t u d e n t s . ( 5 . 5 ) ' — ' C o u l d be e x p e c t e d not t o t u r n i n s t u d e n t r~i g r a d e s by dead l i n e d a t e s . ( 2 . 5 ) ' — ' C o u l d be e x p e c t e d t o be v e r y r i g i d i n h i s i — i s t a n d a b o u t p r e - r e q u i s i t e s ; does not change L J r u l e s t o s u i t d i f f e r e n t s t u d e n t s . ( 2 . 0 ) C o u l d be e x p e c t e d t o e n c o u r a g e s t u d e n t s t o do g r a d u a t e work when t h e y a r e not r e a l l y Q ] p r e p a r e d f o r i t o r do not have r e q u i s i t e q u a l i f i c a t i o n . (3.0) * To be f i l l e d i n o n l y by G r a d u a t e and U n d e r g r a d u a t e o f f i c e s t a f f . - 1 5 0 -APPEN H)I X M BEHAVIOURAL SCALES  Instructions This i s the second part of the Instructor Evaluation process. Please complete these Evaluation sheets along with the SIR Course Evaluation Forms. This part contains description of seven major categories of Teacher Behaviour. Each category of hehaviour i s accompanied by a scale on which you may evaluate your instructor. The sp e c i f i c instructions are given i n each page. Before you turn t h i s page, please complete the questio Agiven below. OVERALL, I TTOULD RATE THE INSTRUCTOR A3: ; ' i 2 3 4 5 *> • l Extremely Neither < Extremely ineffective effective e f f e c t i v e nor i n e f f -ective. -151-I COURSE OUTLINING AND STRUCTURING CONCERN 13 WITH DEFINITION OF COURSE OBJECTIVES AND EVALUATION CRITERIA AND STRUCTURING OF COURSE, DAY TO DAY LECTURES AND DISCUSSIONS. Please read the item3 l i s t e d under t h i s category and rate your instructor. Defines course objectives, workload, and student evaluation procedures. Takes Students' course expectations and d i f f i c u l -t i e s into consideration when defining the above. Provides a detailed schedule for the lectures, discussions and assignment submissions. Provides structural and conceptual framework i n lectures and discussions. 1 ! 1 I Almost always usually I some times 1 very r a r e l y 1 Almost always 1 usually i some times 1 very r a r e l y 1 Almost alwa ys usually some times very r a r e l y 1 1 J 1 Almost usually some very always i ' times rare : «~ n-•->.,> v. mr, + „^ ,*Q1 1 I-Provides lecture ; or other material Almost usually some very always times rare - 1 5 2 -SCALE A In his job as instructor, how ef f e c t i v e i s this person i n out l i n i n g and structuring the course? Consider not only his day-to-day or typic a l job hehaviour but also how he does when he's at his best. Write BEST on the scale opposite the action that seems to f i t him most c l o s e l y when he's doing his best in course outlining and structuring. Write TYPICAL on the scale opposite the action that seems to f i t him most clesely when he's doing his usual or typical job i n course outlining and structuring. Could be expected to s p e c i f i c a l l y state require-ments for the course and use d e f i n i t e and stated c r i t e r i a for evaluation. Could be expected to give course o u t l i -nes and schedules to students. Could be expected to set spe c i f i c targ-ets for each session. Could be expected to set class p a r t i c i -pation as an evaluation c r i t e r i o n with-out c l e a r l y stating his expectations. Could be expected to hold an introductory session i n which the students' expectations are ascer-tained and instructor's course objectives are made clear. Could be expected to take quite a few days to t e l l the students what i s the course content of the course. Could be expected to announce mid-way through that there would be a f i n a l exam, i n contradiction to h i s e a r l i e r statement that there would be no f i n a l . - 1 5 3 -II COVERAGE OP MATERIAL CONCERN IS WITH THE NATURE, SCOPE AND RELEVANCE OP SUBJECT MATTER TAUGHT IN THE COURSE Al® HOW THE COURSE IS RELATED TO TOTHER DISCIPLINES AND REAL WORLD SITUATIONS. Please read the items l i s t e d under t h i s category and rate your instructor. Covers the subject matter i n adequate depth and scope. Links the course contant to other researches and r e a l - l i f e practices. Suggests variety of reading material Suggests relevant readinr; material. Relates course content to other d i s c i p l i n e s . Accepts new ideas suggested by students. I 1 < t Almost usually some verv, always times r a r e l y 1 1 1 t Almost usually some very always times r a r e l y 1 1 \ 1 Almost usually some very always 1 1 times 1 r a r e l y 1 Almost usually some xery always times r a r e l y 1 1 / 1 Almost " usually some very always times r a r e l y 1 1 1 1 Almost usually some very always times r a r e l y - 1 5 4 -SCALE B In his job as instructor, how effective i s this person i n the coverage of material i n the course? Consider not only his day-to-day or typical job behaviour but also how he does when he's at his best. ; Write BEST on the scale opposite . ^  the action that seems to f i t him most closely when he's doing his best i n th§ coverage of material. • I Write TYPICAL on the scale opposite the action that seems to f i t him most closely when he's doing his usual or typica l job in'the coverage of material. Could be expected to provide empirical data and related a r t i c l e s supporting the various theories discussed by him. could be expected to cover text book mater-i a l to a very great degree. Could be expected to assign volumes of reading without discussing them i n the class. Could be expected to apply theories and principles to current events. Could be expected to hand out outlines of current class discussions. [Could be expected not to coordinate various theories into a to t a l framework. Could be expected to explain a major paper very poorly, thus confusing most of the students. - 1 5 5 -III TEACHING STYLE CONCERN IS ¥ITH THE ACTUAL BEHAVIOUR OF THE INSTRUCTOR WITHIN THE CLASS. Please read the items l i s t e d under t h i s category and rate your instructor. I 1 / ! Provides l o g i c a l framework in lectures and discussions Almost usually some very always times r a r e l y Use3 appropriate language and technical terms. -r-r-^—x T I — - ' * v' • * & fa Almost usually some very always times r a r e l y Displays energy and enthusiasm. - J : U _ L_ I ,. Almost usually some very always times r a r e l y Uses jokes, analogies and personal stories. — ^ Lrr ' I — — Almost usually some very always times r a r e l y Speaks a t a comfortable pace i n a w e l l modulated v o i c e Almost usually some very always times r a r e l y oehaves i n f o r m a l l y and n a t u r a l l y . -) - ^1 ' J J Almost u s u a l l y some very always times ra r e l y Emphasizes two-way communications inside class room. r * * J Almost usually some very always times ra r e l y Jias b e f i t t i n g o u t f i t and appearance. —T 1 ^T~- ^ ^ Almost usually some very always times r a r e l y -156-SCALE C In his job as instructor, how effective i s this person i n teaching style? Consider not only his day—to-day or typical job behaviour but also how he does when he's at his best. ; Write BEST on the scale opposite the action that seems to f i t him most closely when he's at i his best st y l e . | Write TYPICAL on the scale opposite the action that seems to f i t him most closely when he's at his usual or typical s t y l e . Could be expected to provide r e a l - l i f e and personal examples to explain a concept. Could be expected to emphasize on as many students participating i n dis cussions as possible rather than on right answers. Could be expected to force two-way communication by asking too many quest-ions. Could be expected to inter.f era with class discussions emphasizing his ideas Could be expected to speak i n a mono-tonous—low voice, without emitting any enthusiasm. Could be expected to encourage students to ask questions and answer t h e i r questions without putting them down. Could be expected to speak with considerable energy and enthusiasm. Could be expected to secure p a r t i c i p a t i o n of individual students by asking them spec i f i c Questions. Could be expected to spend considera ble cftass time i n stating his personal views on diffeV-cnt^ matters. Could be expected to give d u l l lectures which are repetitive of the text book. Could be expected to use abstract and d i f f i -c u l t language i n his lectures. - 1 5 7 -IV TEACHING METHODS CONCERN IS WITH THE VARIOUS TECHNIQUES AND MEDIA USED 3Y THE INSTRUCTOR TO CONVEY THE COURSE CONTENT TO THE STUDENTS. Please read the items l i s t e d under t h i s category and rate your instructor. Uses a variety of ^©.chniques to convey/ the subject matter. Uses black board, films and other v i s u a l aida to supplement lectures. Uses group and case discussions as some of the teaching methods. Encourages role-playing, games and other beha-vioural techniques to convey a major concept. Arranges for guest speakers, f i e l d t r i p s and project groups as part of teaching the course. Almost usually some very always times ra r e l y / l 1 i Almost usually some very always times ra r e l y 1 t 1 i Almost usually some very always times ra r e l y 1 1 I i Almost usually some very always times ra r e l y 1 I 1 t Almost usually some very always times r a r e l y i - 1 5 8 -SCALE | D i In his job as instructor, how effective i s this person i n L h i s teaching methods? Consider not only his day-to-day or ty p i c a l job behaviour but also how he does when he's at his best. j i Write BEST on the scale opposite the action that seems to f i t him most closely when he's doing his best i n 'applying these teaching methods. Write TYPICAL on the scale opposite the action that seems to f i t him most clos e l y when he's doing his usual or t y p i c a l job i n applying these teaching methods. Could be expected to provide many real-l i f e examples to convey,major points. Could be expected to ask individual student to prepare lecture l a s t i n g approximately ten minutes and defend his views. Could be expected to use outdated films. Could be expected to use di f f e r e n t teaching methods to classes of d i f f e r e n t size. Could be expected to bring guest speakers who are doing either research or p r a c t i c a l work. Could be expected to put the students on the'spot' by asking questions to i n d i v i - r\ dual members. . Could be expected to spend whole class hour reading out an a r t i c l e from a magazine. - 1 5 9 -V • EVALUATION CONCERN IS WITH THE WAY IN WHICH THE INSTRUCTOR EVALUATES THE STUDENTS' COURSE PERFORMANCE. Please read the items l i s t e d under t h i s category and rate your instructor. J I I Objectively defines the evaluation c r i t e r i a . and equitably. Gives feedback to the students about their performance. Is flexible: in his evaluation process. Almost always usually some times very r a r e l y / 1 » 1 Almost always usually some times very ra r e l y / l 1 J Almost always i usually 1 ;some times / very r a r e l y l Almost always usually some times very r a r e l y Gives positive c r i t i c i s m and encouragement to students. J —I L i Almost usually some very always times r a r e l y - 1 6 0 -5CALS E In his job as instructor, how effective i s t h i s person i n evaluating the students? Consider not only h i s day-to-day or typica l job behaviour but also how he does when he's at his best. ¥rite BEST on the scale opposite the action that seems to f i t him most closely when he', i s doing his best i n evaluating the students. Write TYPICAL when he i s .at on the scale opposite the action that . seems to f i t him most closely his usual or typica l i n evaluating the students. Could be expected to provide constructive comments and crit i c i s m s on exams, papers etc. Could be expected to evaluate students on whole term performance rather than just exams. Could be expected to motivate students by a l l o t i n g extra marks for doing extra readings. Could be expected to state ( i n the begin-ning) that assignment of grades w i l l be decided i n consultation with students and la t e r decide the matter a l l by himself. Could be expected to express interest i n personal opinions and o r i g i n a l i t y of thought, but give high grades only when answers coincide with his own views. Could be expected to provide feedback about poor farformance to students i n a supportive manner without c r i t i c i z i n g . Could be expected to provide too much bonus marks. Could be expected not to provide mark d i s t r i bution of the course. -161-I I INTERACTION OUTSIDE CONCERN IS WITH THE GENERAL BEHAVIOUR OP THE INSTRUCTOR OUTSIDE THE CLASS. Please read the items l i s t e d under this category and rate your instructor. Is available for consultation outside the class hours. Evinces interest i n students who go to him with their problems and ideas. Provides help to students to solwe course/research/ career/peraOnal problems. Behaves informally with students outside the class. _ J I ! I Almost usually- some very always times r a r e l y Almost usHally some very always times r a r e l y Almost usually some very always times rarely Almost usually some very always times rarer-SCALE F In*his job as instructor, .how: effectiveris^this-nersonvin-hi's-iinteractions' with the students outside class?-,-Consider not only-has day-to-day or typical fob-behaviour but'also how he does-when .he's at his -best. • \-^::y-':. -\ . •:x^<-::^£>,i?- • 7frite :'BEST on the scale opposite the action that seems to f i t him mo3t c l o s e l y -whenshe doing his best i n in'-eracting with ..students outside-class. .Write - TYPICAL on thescale.opposite the action that seems to f i t him most'-closely j^eo-Jie i s doing his ;,usual or t y p i c a l ^ ^ i n interacting with-students outside clas3. " .Could be "expected..to take • any active^, interest •in:the academic future of his students and iprovide concrete help to the students who go and meet him f o r guidance. Could be expected to taker-an active mterj r e s t i n students'- problems and help them i n solving these problems outside class-hours. Could bp expected to &elp students who are havin" d i f f i c u l t i e s with t h e i r assignments. Could be exuected to•:provide career consuj -{tations to students.who,ask for his help i n this area. Could be exoected -to .interact s o c i a l l y with students outside class hours. Could b<=> expected .mot to disclose h i s --office hours. •'.••:..-,r^.v: Could be -x^ected to interview students before they regis t e r : f o r h i s course'and give them : an idea of Jthe-content and: workload,., of. .the •. course. Could b» expected:to provide office- space-to students: who work under him. Could be. expected not to.ks.epN.-;;,up h i s - o f f i c e appointments j' i s habitually late«.or .does not show un at a l l . s ^ v -163-VII FLEXIBILITY AND RESPONSIVENESS CONCERN IS WITH THE EXTENT TO WHICH THE INSTRUCTOR IS FLEXIBLE IN HIS TEACHim SCHEDULES,. METHODS AND STYLE. Please read the items l i s t e d under this category and rate your instruc 1 1 1 1 Is responsive to students' needs and requests. Almost always usually some times very ra r e l y 1 J I 1 Changes his goals and methods to suit students' aspirations and needs. Almost always I usually / some times 1 very r a r e l y 1 Adjusts his style of teaching to suit students' needs. Almost always / usually I some times 1 very r a r e l y f Consciously receives feedback about his performance Almost usually some very and constantly strives to improve his performance. always times r a r e l y 'v-^ -7 : -[nj h i 3 job as instructor, how-flexible and resnon3ive i s - t h i s person? Consider not only bis day-to-day or typical'. j ob.behavi our but, a l so bow he does when he i s at • ,;>:4 / r i t e - , BEST onthe; scale opposite vfche ??action that seems- to ' f i t : him moit closely when hp i s doing his best i n being f l e x i b l e and responsive. :v; . . , 1 ,'' r i t e .TYPICAL .on the scale opposite ^ the action that seems to f i t him most ' closely when he-is at--his usual or,typical Sin being f l e x i b l e and r e s p o n s i v e -jj Could be expected; to pause during his | lectures to make sure that a l l - t h e students are following him; repeats major points.. Could, be expected.,to be open to' suggest*^: ions' regarding course.content and. teaching sr methods; he i s not annoyed when he r e c e i -ves adverse comments on his teaching methods. Could be expected to-stop, his lectiires to answer.questions by students even when i t interferes with his plans for the day. f Could be exnectedutos ftell.the class how >the. f l l m ends when .the film, being shown ~-break n betveen. • Could be expected to t e l l the students -.: •that he does not l i k e teaching them-and that he did not value th e i r opinions.-: about him or his .teaching metaods.•-. {vCould.be exDected to- ask f or^ student inDut, J. regarding their course expectations and areas \ of interest. SCould be expected to grant students'.requests' for more time for completing certain assign-,^ ments. Could be, expected to* Sw.nd out suggestion • sheets s t 1 t h e end of each class to students f o r writing t h e i r comments on ;the:day's- lecture Also.', to discuss these- comments.at the beginning -erf -fche next class. Could be.expected not to provide any expla-nations about an unfair exam which he gave and ask the students-to meet the Dean i f they have any complaints. - t. > s,-. - .. - 1 6 5 -APPENDIX N ANALYSIS OF CENTRAL TENDENCY AND LENIENCY ERRORS OF THE TWO INSTRUMENTS A. R a t i n g s by t e n s t u d e n t s o f t h e i r p r o f e s s o r s on t h e two i n s t r u m e n t s . S t u d e n t R a t i ngs on v a r i o u s d i m e n s i o n s o f B e h a v i o u r a l s c a l e # 1 2 3 4 5 6 7 1 2.17 1.13 3.71 4.20 4.27 1.33 1.05 2 2.17 2.10 5.88 3.36 3.36 3.71 3.78 3 2.83 5.21 3.71 4.20 1.68 4. 20 1.05 4 2.83 1.13 2.10 4. 20 4.27 3.71 4.06 5 2.17 5.21 2.10 3.36 3.36 1.33 3.78 6..;? 2.17 2.10 3.71 5.60 , 1.68 4.20 3.78 7 1.26 5.21 2.10 2.45 2.70 5.98 4.06 8 2.83 1.13 1.75 4.20 4.27 3.71 1.05 9 2.17 4.40 3.71 2.45 3. 36 1. 33 5.60 10 5. 18 2. 10 3.71 5.60 1.68 4.20 4.06 I s t u d e n t R a t i ngs on s e l e c t e d i t e m s i n SIR q u e s t i o n n a i r e * 1 # 1 2 4 12 13 20 35 ( i ! . l 3 3 3 4 3 3 • 4 ! 2 4 4 4 3 3 4 4 i 3 3 3 3 4 3 3 3 ! 4 4 4 3 4 4 4 4 i 5 3 3 3 3 3 3 2 6 4 1 3 4 4 4 4 7 3 3 3 3 2 . 2 3 8 2 4 3 2 4 3 3 9 3 3 3 3 4 3 3 10 3 3 4 3 3 1 1 . * Items were s e l e c t e d f r o m SIR q u e s t i o n n a i r e u s i n g t h e f o l l o w i n g c r i t e r i a : -a) Items were c o m p a r a b l e w i t h d i m e n s i o n s and i n c i d e n t s u sed i n t h e b e h a v i o u r a l s c a l e i n t h e i r i m p l i c a t i o n s . b) The r a t i n g s on t h e i t e m s e x h i b i t e d a wide r a n g e o f v a l u e s . -166-B. S t a n d a r d d e v i a t i o n s o f r a t i n g s by 62 s t u d e n t s on t h e two i n s t r u m e n t s . S t d . D e v i a t i o n s o f r a t i ngs on B e h a v i o u r a l I n s t r u m e n t 1 2 Di 3 m e n s i o 4 n No. 5 6 7 .0.672 1.365 1.199 .1.376 1.489 0.916 0.641 S t d . D e v i a t i o n s o f r a t i n g s on SIR Q u e s t i o n n a -i r e ( S e l e c t e d i terns) 1 I tern 2 Number 4 on S ] T2 R Ques 13 t i onnc 19 n're 35 0.474 0.676 0.504 0.668 0.691 0.506 0.981 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0093508/manifest

Comment

Related Items