Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Television content analysis : agreement between expert and naive coders Wotherspoon, David Kenneth 1988

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1988_A8 W67.pdf [ 5.69MB ]
Metadata
JSON: 831-1.0097819.json
JSON-LD: 831-1.0097819-ld.json
RDF/XML (Pretty): 831-1.0097819-rdf.xml
RDF/JSON: 831-1.0097819-rdf.json
Turtle: 831-1.0097819-turtle.txt
N-Triples: 831-1.0097819-rdf-ntriples.txt
Original Record: 831-1.0097819-source.json
Full Text
831-1.0097819-fulltext.txt
Citation
831-1.0097819.ris

Full Text

TELEVISION CONTENT ANALYSIS: AGREEMENT BETWEEN EXPERT AND NAIVE CODERS By DAVID WOTHERSPOON B.A. Simon F r a s e r U n i v e r s i t y , 1986 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS i n THE FACULTY OF GRADUATE STUDIES (Ps y c h o l o g y ) We a c c e p t t h i s t h e s i s as c o n f o r m i n g t o t h e r e q u i r e d s t a n d a r d s THE UNIVERSITY OF BRITISH COLUMBIA October 1988 ( c ) D a v i d Wotherspoon, 1988 In p r e s e n t i n g this thesis in part ia l f u l f i lmen t o f t he requ i remen ts fo r an a d v a n c e d d e g r e e at t h e Univers i ty o f Bri t ish C o l u m b i a , I agree that t h e Library shall m a k e it f ree ly avai lable fo r re ference a n d s tudy . I f u r the r agree that permiss ion fo r ex tens ive c o p y i n g o f th is thesis fo r scholar ly p u r p o s e s may b e g r a n t e d by the h e a d o f m y d e p a r t m e n t o r by his o r her representa t ives. It is u n d e r s t o o d that c o p y i n g o r pub l i ca t i on o f th is thesis fo r f inancia l gain shall n o t b e a l l o w e d w i t h o u t m y w r i t t e n pe rm iss ion . D e p a r t m e n t o f P s y c h o l o g y T h e Univers i ty o f Brit ish C o l u m b i a 1956 M a i n Ma l l Vancouve r , Canada V 6 T 1Y3 D a t e 14 O c t o b e r 1988  DE-6(3/81) i i A b s t r a c t Agreement between t r a i n e d and u n t r a i n e d c o d e r s i n a s s e s s i n g t e l e v i s i o n c o n t e n t was i n v e s t i g a t e d . A model i n t e g r a t i n g t h e d i f f e r e n t approaches t o c o n t e n t a n a l y s i s was pr o p o s e d . The model c o n t a i n s t h r e e d i m e n s i o n s : a u d i e n c e c o d e r s v e r s u s e x p e r t c o d e r s , m i c r o a n a l y s i s v e r s u s m a c r o a n a l y s i s , and q u a n t i t a t i v e v e r s u s q u a l i t a t i v e a n a l y s i s . The a u d i e n c e v e r s u s e x p e r t c o d e r s f a c e t o f t h a t model was e v a l u a t e d by h a v i n g u n i v e r s i t y s t u d e n t s watch and a s s e s s t h e c o n t e n t o f 24 t e l e v i s i o n programs chosen from p r i m e - t i m e on t h e b a s i s o f t h e i r p o p u l a r i t y . They were not t r a i n e d i n c o n t e n t a n a l y s i s and d i d no t know t h e q u e s t i o n s about which t h e y were asked u n t i l a f t e r v i e w i n g t h e i r program. T h e i r e v a l u a t i o n s were compared w i t h s i m i l a r e v a l u a t i o n s g i v e n p r e v i o u s l y by t r a i n e d ( e x p e r t ) c o d e r s . Each o f t h e 24 programs was watched by 5 male and 5 female n a i v e c o d e r s ( t o t a l N=240). The groups were b a l a n c e d f o r e t h n i c i t y and s o c i o e c o n o m i c s t a t u s . A s t a t i s t i c d e v e l o p e d e s p e c i a l l y f o r t h i s r e s e a r c h was used t o compare t h e n a i v e and e x p e r t r a t i n g s on 22 s e l e c t e d v a r i a b l e s . The r e s u l t s i n d i c a t e d t h a t u n t r a i n e d and t r a i n e d c o d e r s i n g e n e r a l e v a l u a t e d t h e programs s i m i l a r l y . Moreover, t h e q u e s t i o n s on which t h e e x p e r t s tended n o t t o agr e e ( t h a t i s , were u n r e l i a b l e ) were g e n e r a l l y t h e same ones on wh i c h t h e u n t r a i n e d c o d e r s d i d not agre e , b o t h amongst t h e m s e l v e s and w i t h t h e e x p e r t s . i i i T a b l e o f c o n t e n t s S e c t i o n Page A b s t r a c t i i L i s t o f T a b l e s i v L i s t o f F i g u r e s v I n t r o d u c t i o n 1 Genres o f TV r e s e a r c h 1 From C o n t e n t t o E f f e c t s 3 Schema Theory 3 S o c i a l L e a r n i n g Theory 4 C o n s t r u c t i v i s t T h e o r i e s 5 C o n t e n t A n a l y s i s 6 Background 6 M e t h o d o l o g i c a l and T h e o r e t i c a l Dimensions 10 UBC C o n t e n t A n a l y s i s P r o j e c t 19 Hypotheses 2 3 Method 24 S u b j e c t s 24 C o d i n g system 25 Programs 29 P r o c e d u r e 31 A n a l y s e s 32 R e s u l t s 34 Agreement Among N a i v e Coders 34 Agreement Between E x p e r t and N a i v e Coders 35 R e l a t i o n s h i p Between T s c o r e s and R e l i a b i l i t y 36 G e n e r a l i z a t i o n o f t h e R e s u l t s A c r o s s N a i v e Coders 37 D i s c u s s i o n 39 R e f e r e n c e s 47 Appendix A 67 Appendix B 91 Appendix C 118 Appendix D 120 i v L i s t o f T a b l e s T a b l e T i t l e Page 1 V a r i a b l e names and t h e i r l o c a t i o n i n t h e ACS 51 2 Frequency o f f a i l u r e t o answer q u e s t i o n s 52 3 S u b j e c t demographic v a r i a b l e s used i n t h e r e g r e s s i o n a n a l y s e s , t h e i r l o c a t i o n i n t h e ACS, and how t h e y were r e c o d e d 53 4 Program sample 54 5 Agreement among n a i v e c o d e r s u s i n g K e n d a l l ' s W 55 6 D i s t r i b u t i o n o f T s c o r e s 56 7 Comparison o f T s c o r e s w i t h r e l i a b i l i t y s c o r e s f o r e x p e r t s 57 8 Forward s t e p w i s e r e g r e s s i o n a n a l y s e s 59 L i s t o f F i g u r e s F i g u r e T i t l e Page F i g u r e c a p t i o n s 60 1 Three d i m e n s i o n a l box d e p i c t i n g d i f f e r i n g approaches t o t h e a n a l y s i s o f c o n t e n t 61 2 D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r a l l 129 v a r i a b l e s ( a f t e r r e c o d i n g ) i n t h e ACS and a l l 24 shows coded by t h e n a i v e c o d e r s 62 3 D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper and a l l 24 shows coded by t h e n a i v e c o d e r s 63 4 D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper f o r t h e 2 h a r d news shows o n l y 64 5 D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper f o r t h e 10 o t h e r n o n - f i c t i o n shows o n l y 65 6 D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s p aper f o r t h e 12 f i c t i o n shows o n l y 66 1 I n t r o d u c t i o n I n o n l y a few decades t e l e v i s i o n has become u b i q u i t o u s . I n most p l a c e s i n N o r t h A m e r i c a , and many o t h e r s around t h e w o r l d , i t i s d i f f i c u l t t o f i n d someone who i s n ' t f a m i l i a r w i t h A r n i e B e c k e r ' s l a t e s t a f f a i r on L.A. Law, o r Mrs. H u x t a b l e ' s problems d i s c i p l i n i n g Dr. H u x t a b l e on t h e Cosby show. I n i t s u b i q u i t y t e l e v i s i o n has become a major p u r v e y o r o f e n t e r t a i n m e n t , i n f o r m a t i o n . , p o l i t i c s , a d v e r t i s i n g , and c u l t u r e . Dorr (1986, p. 8) t e l l s us t h a t : T e l e v i s i o n i s i n more t h a n 95% o f a l l American households., more common t h a n t e l e p h o n e s and i n d o o r t o i l e t s . Most homes have more t h a n one o p e r a t i n g s e t . I n an average r e s i d e n c e , a s e t i s t u r n e d on about seven hours a day. The average f a m i l y member d e v o t e s two and one h a l f t o f i v e h o u r s a day t o v i e w i n g . A t h i g h s c h o o l g r a d u a t i o n , A m e r i c a n c h i l d r e n w i l l have s p e n t more t i m e i n f r o n t o f t h e t e l e v i s i o n s e t t h a n i n a c l a s s r o o m . By t h e t i m e t h e y a r e 65, more t h a n n i n e f u l l y e a r s o f t h e i r l i v e s w i l l have been dev o t e d t o w a t c h i n g t e l e v i s i o n . These s t a t i s t i c s f o r t h e U.S.A. a r e v e r y s i m i l a r t o t h o s e found i n Canada ( e . g . , W i l l i a m s & Boyes, 1986). I f t e l e v i s i o n i s h e r e t o s t a y , and p l a y s s u c h a prominent r o l e i n so many l i v e s , i t i s o f i n t e r e s t t o know how t h i s medium i n f l u e n c e s i t s v i e w e r s . When p e o p l e watch TV, what do t h e y t a k e away from t h e e x p e r i e n c e ? A l t h o u g h t h i s q u e s t i o n may be s t r a i g h t f o r w a r d , a n s w e r i n g i t i s n o t . Genres o f TV R e s e a r c h Whatever t h e i r t h e o r e t i c a l and m e t h o d o l o g i c a l approaches, a l l TV r e s e a r c h e r s have e s s e n t i a l l y t h e same g o a l : u n d e r s t a n d i n g t h e r o l e o f t e l e v i s i o n i n s o c i e t y . I n s p i t e o f t h i s common b a s i c g o a l t h e r e i s d i s a g r e e m e n t about how t o a s s e s s t h e impact o f t e l e v i s i o n . 2 There a r e t h r e e major approaches t o t e l e v i s i o n r e s e a r c h : e f f e c t s , uses and g r a t i f i c a t i o n s , and t h e a n a l y s i s o f c o n t e n t . E f f e c t s r e s e a r c h (e.g., W i l l i a m s , 1986) d e a l s w i t h t e l e v i s i o n ' s d i r e c t and i n d i r e c t i n f l u e n c e s on b e h a v i o r , a t t i t u d e s , and e x p e c t a t i o n s v i a s e v e r a l p r o c e s s e s . Some i n d i r e c t e f f e c t s o c c u r because t e l e v i s i o n d i s p l a c e s a c t i v i t i e s w h i c h o t h e r w i s e might have had c e r t a i n e f f e c t s on a t l e a s t some v i e w e r s . Of g r e a t e r r e l e v a n c e t o t h i s r e s e a r c h a r e e f f e c t s due t o t e l e v i s i o n ' s c o n t e n t . Most r e s e a r c h o f t h i s t y p e has been co n c e r n e d w i t h p a r t i c u l a r t o p i c c a t e g o r i e s , such as v i o l e n c e , and how exposure a f f e c t s a t t i t u d e s and b e h a v i o r s . Uses and g r a t i f i c a t i o n s r e s e a r c h ( e . g . , Rosengren, Wenner, & Palmgreen, 1985) f o c u s e s on how p e o p l e use media. B l u m l e r and K a t z (1974, c i t e d i n Palmgreen, Wenner, & Rosengren, 1985, p. 11) d e s c r i b e d t h e r o l e o f t h e uses and g r a t i f i c a t i o n s r e s e a r c h e r as b e i n g t o "ask not what media do t o p e o p l e , b u t ask what p e o p l e do w i t h media". T h i s approach a d d r e s s e s q u e s t i o n s such as what t y p e s o f p e o p l e watch which shows; how o f t e n t h e y watch TV; why t h e y watch; when they watch, and so on. The a n a l y s i s o f t e l e v i s i o n c o n t e n t can be c a t e g o r i z e d as e i t h e r c o n t e n t a n a l y s i s ( e . g . , Gerbner, H o l s t i , K r i p p e n d o r f , P a i s l e y , & Stone, 1969), o r a u d i e n c e r e s e a r c h ( e . g . , Gunter, 1983). I n b o t h approaches t h e c o n t e n t o f t e l e v i s i o n , f o r example, t h e number o f a c t s o f v i o l e n c e o r t h e most s a l i e n t message, i s a s s e s s e d . The main d i f f e r e n c e between t h e s e two approaches i s t h a t i n c o n t e n t a n a l y s i s t h e v i e w e r s a r e a s m a l l 3 number o f t r a i n e d c o d e r s , whereas i n a u d i e n c e r e s e a r c h t h e y a r e a l a r g e number o f u n t r a i n e d c o d e r s . The c u r r e n t r e s e a r c h a d d r e s s e d t h e r e l a t i o n s h i p between c o n t e n t a n a l y s i s and a u d i e n c e r e s e a r c h , and i n p a r t i c u l a r , e v a l u a t e d a methodology h y p o t h e s i z e d t o form a b r i d g e between t h e s e two approaches t o u n d e r s t a n d i n g t e l e v i s i o n . From c o n t e n t t o e f f e c t s B o t h c o n t e n t a n a l y s t s and a u d i e n c e r e s e a r c h e r s r e l y on e v i d e n c e from o t h e r s t u d i e s t h a t t e l e v i s e d c o n t e n t does i n f l u e n c e v i e w e r s . S e v e r a l d i f f e r e n t t h e o r i e s have been p r o p o s e d t o e x p l a i n t h i s p r o c e s s . Three seem p a r t i c u l a r l y i m p o r t a n t : schema t h e o r y (Schank & A b e l s o n , 1977; T a y l o r & C r o c k e r , 1981), s o c i a l l e a r n i n g t h e o r y (Bandura, 1977), and i n t e r a c t i v e c o n s t r u c t i v i s t t h e o r i e s ( e . g . , D o r r , 1986; Salomon, 1979). Schema Theory. Schema t h e o r y (Schank & A b e l s o n , 1977) i s a p s y c h o l o g i c a l t h e o r y d e v e l o p e d t o e x p l a i n how p e o p l e p r o c e s s i n f o r m a t i o n . Schemata a r e s e l f - r e l e v a n t a t t i t u d e s , b e l i e f s , and e x p e c t a t i o n s r e g a r d i n g t h e c h a r a c t e r i s t i c s and outcomes o f e v e n t s . I n e f f e c t , t h e y a r e f i l t e r s o r s t e r e o t y p e s t h a t d i r e c t a t t e n t i o n , p e r c e p t i o n s , and memory, and t h u s p r o v i d e an e f f i c i e n t way o f p r o c e s s i n g i n f o r m a t i o n . F o r example, Cordua, McGraw, and Drabman (1979) showed two n e a r l y i d e n t i c a l f i l m s t o two d i f f e r e n t groups o f c h i l d r e n . The f i r s t group saw a f i l m w i t h a male p h y s i c i a n and a female n u r s e ; t h e second group saw a f i l m w i t h a f e m a l e p h y s i c i a n and a male n u r s e . L a t e r , a l l c h i l d r e n i n t h e f i r s t group r e c a l l e d b o t h 4 c h a r a c t e r s c o r r e c t l y , whereas o n l y 22% i n t h e second group d i d s o , and more t h a n h a l f i n c o r r e c t l y i d e n t i f i e d b o t h t h e d o c t o r as male and t h e nurse as female. A c c o r d i n g t o schema t h e o r y , t h e s e c h i l d r e n p r o c e s s e d i n f o r m a t i o n from t h e f i l m by way o f t h e i r gender schemata, and e i t h e r d i d not n o t i c e t h e d i s c r e p a n c y o r a l t e r e d t h e i r memory t o be c o n s i s t e n t w i t h t h e i r gender schemata. Schemata a r e b u i l t up o r c o n s t r u c t e d i n i t i a l l y t h r o u g h b o t h d i r e c t e x p e r i e n c e ( e . g . , d o c t o r s and n u r s e s e n c o u n t e r e d i n r e a l l i f e ) and i n d i r e c t e x p e r i e n c e ( e . g . , media p o r t r a y a l s ) . Subsequent e x p e r i e n c e s a r e p r o c e s s e d t h r o u g h a c o g n i t i v e m a t c h i n g p r o c e d u r e a c c o r d i n g t o t h e i r s i m i l a r i t y t o t h e p r e e x i s t i n g schemata. I f t h e match between an event and a p r e e x i s t i n g schema i s good, t h e schema i s u p h e l d and remains r e l a t i v e l y unchanged. I f , however, an event p r e s e n t s some un i q u e o r l e s s f a m i l i a r c h a r a c t e r i s t i c , o r c o n t r a d i c t s t h e schema, v a r i o u s outcomes a r e p o s s i b l e . Most l i k e l y t h e d i s c r e p a n c y w i l l not be n o t i c e d ; t h e match w i l l be good enough. Even i f i t i s n o t i c e d , , t h e d i s c r e p a n c y i s l i k e l y t o be p r o c e s s e d as an e x c e p t i o n . S o c i a l L e a r n i n g Theory.. A second i m p o r t a n t avenue o f t h e i n f l u e n c e o f t e l e v i s i o n c o n t e n t on v i e w e r s i s o b s e r v a t i o n a l l e a r n i n g o r m o d e l i n g . Bandura (1977) p o i n t s o u t t h a t l e a r n i n g i n v o l v e s two s t e p s , a c q u i s i t i o n and performance. I n s u m m a r i z i n g t h e e v i d e n c e i n d i c a t i n g when b e h a v i o r o b s e r v e d on TV i s l i k e l y t o be performed Comstock (1980) d e s c r i b e d f o u r f a c t o r s : s o c i a l a p p r o v a l f o r t h e model and/or f o r b e h a v i o r i n t h e f i l m e d m a t e r i a l ; t h e s u c c e s s f u l n e s s o r e f f i c a c y o f t h e b e h a v i o r ; t h e p e r c e i v e d r e l e v a n c e o f t h e b e h a v i o r and t h e model's c h a r a c t e r i s t i c s t o t h e v i e w e r ; and whether t h e p o r t r a y a l o p t i m i z e s a r o u s a l f o r t h e v i e w e r . C o n s t r u c t i v i s t T h e o r i e s . A n a l y s t s o f t e l e v i s i o n c o n t e n t do not assume t h a t t e l e v i s i o n v i e w e r s a r e p a s s i v e l y i n f l u e n c e d by TV. J u s t as f o r any o t h e r communication t h e outcome i s a r e s u l t o f t h e i n t e r a c t i o n o f t h e c h a r a c t e r i s t i c s o f t h e messages and t h e c h a r a c t e r i s t i c s o f v i e w e r s , i n c l u d i n g b o t h t r a n s i t o r y ( e . g . , a r o u s a l ) and s t a b l e ( e . g . , s o c i o e c o n o m i c s t a t u s , p e r s o n a l i t y ) v i e w e r c h a r a c t e r i s t i c s . Salomon (1979) emphasizes t h e i n t e r a c t i o n o f t h e symbol systems o f t e l e v i s i o n and t h e c o g n i t i v e a s p e c t s o f i n d i v i d u a l v i e w e r s . I n h i s v i e w , TV can have a p p a r e n t o r s u r f a c e - l e v e l meanings, but symbols a l s o may i n t e r a c t t o c r e a t e a more e l a b o r a t e and l e s s o b v i o u s p s y c h o l o g i c a l e f f e c t . D o r r (1986) s t r e s s e s t h e a c t i v e c o n s t r u c t i o n o f meaning i n t h e v i e w i n g p r o c e s s . She b e l i e v e s c h i l d r e n and a d u l t s a c t i v e l y i n t e r p r e t c o n t e n t and s y n t h e s i z e messages w i t h i n t h e i r own framework. Two p e o p l e c o u l d s i t s i d e by s i d e w a t c h i n g t h e same show and come away w i t h c o m p l e t e l y d i f f e r e n t meanings. To t a l k s i m p l y about " t h e " messages on t e l e v i s i o n i s m i s l e a d i n g . M o r l e y (1980) argues t h a t t h e TV message i s a complex s i g n , i n w h i c h a p r e f e r r e d meaning has been i n s c r i b e d , b u t w h i c h r e t a i n s t h e p o t e n t i a l t o be decoded i n a d i f f e r e n t manner and t h u s t o communicate a d i f f e r e n t meaning; i t i s a s t r u c t u r e d polysemy. S i n c e communication i s a lways an 6 i n t e r a c t i o n of the c h a r a c t e r i s t i c s of the message and the c h a r a c t e r i s t i c s of the r e c e i v e r , i t has been argued t h a t i t i s unique t o i n d i v i d u a l s and cannot be p r e d i c t e d (Gunter, 1988). Perhaps concern with i n t e r - c o d e r r e l i a b i l i t y i s m i s p l a c e d and a more phenomenological i n d i v i d u a l a n a l y s i s i s more a p p r o p r i a t e . Morley responds t h a t " a l l meanings do not e x i s t ' e q u a l l y ' i n the message: i t has been s t r u c t u r e d i n dominance, alth o u g h i t s meaning can never be t o t a l l y f i x e d or ' c l o s e d ' . F u r t h e r , the ' p r e f e r r e d r e a d i n g ' i s i t s e l f p a r t of the message..." (p. 10). Because message p e r c e p t i o n v a r i e s as a f u n c t i o n of viewer c h a r a c t e r i s t i c s , some r e s e a r c h e r s have taken a p e s s i m i s t i c view of t h e u s e f u l n e s s of content a n a l y s i s (e.g., Gunter, 1988). I t i s our c o n t e n t i o n (e.g., W i l l i a m s , P h i l l i p s , T r a v i s , & Wotherspoon, 1988), however, t h a t both content a n a l y s i s and audience r e s e a r c h are worthwhile, the former t o e s t a b l i s h the dominant meanings, t o use Morley's term, and the l a t t e r t o determine who p e r c e i v e s them, how, and under what c o n d i t i o n s . Content A n a l y s i s Background. In October 1985, W i l l i a m s and her students a t the U n i v e r s i t y of B r i t i s h Columbia began a p r o j e c t designed t o a s s e s s the content of the major t e l e v i s i o n networks r e c e i v e d a c r o s s Canada (Williams, P h i l l i p s , & T r a v i s , 1985). The program sample c o n s i s t e d of a l l programs 15 minutes or longer (1089 programs) from seven networks over one f u l l week, 7:00 AM t o 1:00 AM. The networks were the two Canadian government 7 funded c h a n n e l s , CBC E n g l i s h and CBC F r e n c h (Radio Canada), t h e p r i v a t e C a n a d i a n network, CTV; t h e U.S. p u b l i c l y funded network, PBS; and t h e t h r e e p r i v a t e U.S. netwo r k s , ABC, CBS, and NBC. T r a i n e d c o d e r s watched t h e show and t h e n i m m e d i a t e l y answered 25 pages o f p r e d e t e r m i n e d q u e s t i o n s . The q u e s t i o n s c o v e r e d a v a r i e t y o f t o p i c s i n c l u d i n g e t h n i c m i n o r i t i e s ; c o u n t r i e s ; s e x , romance, and r e l a t i o n s h i p s ; gender r o l e p o r t r a y a l s ; a g g r e s s i o n ; i s s u e s , c o n t r o v e r s i e s , and dilemmas; p o r t r a y a l o f p e o p l e ; g l o b a l i m p r e s s i o n s ; and programming c h a r a c t e r i s t i c s . The U n i v e r s i t y o f B r i t i s h Columbia c o n t e n t c o d i n g system (UBCCS, W i l l i a m s , P h i l l i p s , & T r a v i s , 1985) was d e s i g n e d t o a s s e s s , i n a s y s t e m a t i c and r e l i a b l e way, t h e take-home message o f t h e ( m y t h i c a l ) average v i e w e r . That i s , t h e g o a l was t o c r e a t e a method o f c o n t e n t a n a l y s i s i n which t h e messages r e c o r d e d by t r a i n e d c o d e r s r e s e m b l e d as c l o s e l y as p o s s i b l e t h o s e t h a t v i e w e r s a t home would p e r c e i v e . The purpose o f t h e c u r r e n t r e s e a r c h was t o a s s e s s t h e e x t e n t t o whic h t h i s g o a l has been a c h i e v e d . Two d i f f e r e n t groups w i l l be u s i n g t h e same i n s t r u m e n t t o measure t h e same d a t a . The e x t e n t t o w h i c h t h e i r answers agre e , w h i c h i s t h e f o c u s o f t h i s s t u d y , c o u l d be c o n s t r u e d as an assessment o f r e l i a b i l i t y . The q u e s t i o n o f whether t h e e x t e n t o f agreement between t r a i n e d and n a i v e c o d e r s o b t a i n e d i n t h i s s t u d y would a l s o o c c u r f o r o t h e r groups o f n a i v e c o d e r s , a q u e s t i o n o f g e n e r a l i z a b i l i t y , w i l l be addr e s s e d i n 8 f u t u r e r e s e a r c h . The q u e s t i o n o f t h e v a l i d i t y o f t h e c o d i n g system d e v e l o p e d by W i l l i a m s e t a l . (1985), t h a t i s , whether i t a c c u r a t e l y a s s e s s e s c e r t a i n messages i n TV c o n t e n t , a l s o was n ot a d d r e s s e d i n t h i s s t u d y . I t would r e q u i r e c o n v e r g i n g e v i d e n c e t h a t a v a r i e t y o f d i f f e r e n t measurement approaches y i e l d e d t h e same messages f o r t h e same c o n t e n t . A t h o r o u g h l i t e r a t u r e s e a r c h r e v e a l e d o n l y t h r e e o t h e r a t t e m p t s t o compare c o d i n g by e x p e r t r a t e r s and u n t r a i n e d o r n a i v e v i e w e r s ( N i e l s o n , undated; L u l l , Hanson, & Marx, 1977; T a t e , 1977). L u l l , Hanson, and Marx (1977) showed t e l e v i s i o n c o m m e r c i a l s p r e v i o u s l y i d e n t i f i e d as c o n t a i n i n g s e x i s t p o r t r a y a l s t o male and female c o l l e g e s t u d e n t s . T h e i r sympathy w i t h t h e women's movement and t h e i r open-ended r e s p o n s e s t o t h e commercials were measured. They were g i v e n 4 min u t e s t o w r i t e t h e i r r e a c t i o n t o each c o m m e r c i a l , b u t were g i v e n no i n s t r u c t i o n s r e g a r d i n g t h e t y p e o f r e a c t i o n e x p e c t e d . The r e s u l t s i n d i c a t e t h a t women were more s e n s i t i v e t o s e x i s t p o r t r a y a l s t h a n were men. There a l s o was a s i g n i f i c a n t , a l t h o u g h n o t e s p e c i a l l y l a r g e (r=.35 f o r males, and r=.46 f o r females) p o s i t i v e c o r r e l a t i o n between f e m i n i s m s c o r e s and r e c o g n i t i o n o f sex r o l e s t e r e o t y p i n g . A l l o f t h e commercials had been i d e n t i f i e d by c o n t e n t a n a l y s t s as c o n t a i n i n g t r a d i t i o n a l sex r o l e p o r t r a y a l s , b u t few e r t h a n h a l f t h e s u b j e c t s commented on t h e s e . I n a s t u d y o f t e l e v i s i o n f o r Danmarks R a d i o , N i e l s o n ( u n d a t e d ) , f i r s t d i d a c o n t e n t a n a l y s i s o f t h e s y m b o l i c w o r l d o f t h e f a m i l y as p o r t r a y e d on t e l e v i s i o n . T h i s was not based on v a l u e s p o r t r a y e d i n i n d i v i d u a l e p i s o d e s , b u t i n s t e a d was based on t h e " t o t a l o u t p u t " (p. 4) . The sample audi e n c e r e p r e s e n t e d d i f f e r e n t f a m i l y t y p e s , and t h e i r r e a c t i o n s t o programs were compared w i t h t h e r e s u l t s o f t h e c o n t e n t a n a l y s i s . A u dience p e r c e p t i o n s were a s s e s s e d i n two ways. F i r s t , a s u r v e y o f about 1000 h ouseholds r e p r e s e n t i n g d i f f e r e n t t y p e s o f f a m i l i e s was conducted. Second, i n - d e p t h i n t e r v i e w s were done w i t h about 30 f a m i l i e s . The c o n t e n t a n a l y s e s and t h e a u d i e n c e a n a l y s i s d i d not use t h e same m e t r i c . The c o n t e n t work was done w i t h a f i x e d s e t o f q u e s t i o n s whereas t h e a u d i e n c e was i n t e r v i e w e d a t a more ph e n o m e n o l o g i c a l l e v e l , so d i r e c t s t a t i s t i c a l c o m p a r i s o n s were n o t p o s s i b l e . O v e r a l l , however, t h e a u d i e n c e r e a c t i o n s t o t h e p o r t r a y a l s o f f a m i l i e s on t e l e v i s i o n were c o n s i s t e n t w i t h t h o s e o f t h e c o n t e n t a n a l y s i s , t h a t i s , t h a t t e l e v i s i o n i s a d i s t o r t i o n o f r e a l i t y . T a t e ' s (1977) s t u d y was p a r t o f a l a r g e r body of work commissioned by t h e O n t a r i o R o y a l Commission on V i o l e n c e i n t h e Communications I n d u s t r y . I n t h i s work, w h i c h f o c u s e d on a g g r e s s i o n , T a t e compared t h e r e s u l t s o f a s y s t e m a t i c , d e t a i l e d , and r e l i a b l e c o n t e n t a n a l y s i s ( W i l l i a m s , Zabrack, & J o y , 1977) w i t h t h o s e o f an average a u d i e n c e . T a t e used a q u e s t i o n n a i r e t h a t was v e r y s i m i l a r t o t h e one used by W i l l i a m s , Zabrack, and J o y , and some o f t h e same programs. Respondents watched t h e programs i n t h e i r own homes, and c a r e was t a k e n t o make t h e v i e w i n g s i t u a t i o n as c l o s e t o normal as p o s s i b l e . Data were c o l l e c t e d d u r i n g and i m m e d i a t e l y a f t e r w a t c h i n g a show, so t h e v i e w e r s ' immediate p e r c e p t i o n s were a s s e s s e d . T h i s r e s e a r c h was done i n S a s k a t o o n where, a t t h e t i m e , no U.S. t e l e v i s i o n was a v a i l a b l e . I t i s l i k e l y , t h e r e f o r e , t h a t t h e (U.S.) TV programs were b e i n g seen f o r t h e f i r s t t i m e . A t o t a l o f 315 p e o p l e were s e l e c t e d a t random from t h e 1974 p r o v i n c i a l v o t e r s l i s t , and o f t h o s e , 124 completed t h e e n t i r e i n t e r v i e w . They f i l l e d o u t s e m a n t i c d i f f e r e n t i a l s c a l e s , w h i c h a s s e s s e d v a r i o u s a s p e c t s o f t h e program c o n t e n t , i m m e d i a t e l y a t t h e c o n c l u s i o n o f t h e program. T a t e compared t h e s c o r e s a s s i g n e d by t h e t r a i n e d c o d e r s ( W i l l i a m s e t a l . , 1977) w i t h t h e mean s c o r e g i v e n by t h e sample a u d i e n c e . He found c o n s i d e r a b l e d i s p a r i t y between t h e r e s u l t s o f t h e c o n t e n t a n a l y s i s and t h e p e r c e p t i o n s o f t h e a u d i e n c e . I n g e n e r a l , t h e sample audi e n c e was l e s s l i k e l y t o n o t i c e s p e c i f i c c o n t e n t when compared t o t h e t r a i n e d c o d e r s . He suggested t h a t t h e messages r e c o r d e d i n c o n t e n t a n a l y s i s be c o n s i d e r e d "as t h e most l i b e r a l e s t i m a t e o f t h e amount o f v i o l e n c e i n media c o n t e n t , w h i l e a c c e p t i n g t h e a u d i e n c e d a t a as t h e most c o n s e r v a t i v e " (p. 373) . U n f o r t u n a t e l y , a f l a w i n T a t e ' s s t a t i s t i c a l approach compromised h i s c o m p a r i s o n s i n ways d i s c u s s e d i n d e t a i l i n t h e Method s e c t i o n o f t h i s t h e s i s . Having e s t a b l i s h e d t h e need f o r r e s e a r c h t h a t compares messages coded by c o n t e n t a n a l y s t s w i t h t h o s e r e c o r d e d by average a u d i e n c e s , l e t us now t u r n t o t h e d e bate o v e r how b e s t t o a n a l y z e c o n t e n t . M e t h o d o l o g i c a l and t h e o r e t i c a l d i m e n s i o n s . As s t a t e d e a r l i e r , v a r i o u s t h e o r e t i c a l and m e t h o d o l o g i c a l approaches have been used t o a s s e s s t h e impact o f TV on s o c i e t y . The a n a l y s i s o f TV c o n t e n t i s , i n i t s e l f , i n s u f f i c i e n t t o e x p l a i n t h e e n t i r e p r o c e s s , b u t i t i s a n e c e s s a r y p a r t o f t h a t p r o c e s s ( S e p s t r u p , 1981). I n s p i t e o f , o r perhaps because o f , t h e n e c e s s i t y o f c o n t e n t a n a l y s i s t h e r e i s disagreement about how b e s t t o proce e d . The g o a l o f t h e f o l l o w i n g model i s t o e x p l a i n t h e d i s t i n c t i o n s among t h e m a n i f o l d t h e o r e t i c a l and m e t h o d o l o g i c a l approaches t o t h e a n a l y s i s o f TV c o n t e n t . T h i s s h o u l d h e l p o t h e r r e s e a r c h e r s more e a s i l y u n d e r s t a n d t h e s e v a r i o u s approaches and t h u s make more i n f o r m e d r e s e a r c h d e c i s i o n s . A l t h o u g h t h i s d i s c u s s i o n i s based on t h e a n a l y s i s o f t e l e v i s i o n c o n t e n t , i t i s i n t e n d e d t o be a p p l i c a b l e t o a l l approaches t o t h e a n a l y s i s o f a l l t y p e s o f c o n t e n t , e.g., r a d i o , t e x t . The a n a l y s i s o f c o n t e n t can be u n d e r s t o o d as v a r y i n g a l o n g s e v e r a l axes w i t h i n a t h r e e d i m e n s i o n a l box (see F i g u r e 1) . Any g i v e n p i e c e o f r e s e a r c h can be p l a c e d a t some p o i n t w i t h i n t h i s m e t h o d o l o g i c a l box. I n s e r t F i g u r e 1 about here The f i r s t , and s i m p l e s t , d i m e n s i o n i s c o n t e n t a n a l y s i s v e r s u s a u d i e n c e r e s e a r c h . These two approaches t o t h e a n a l y s i s o f c o n t e n t a r e d e f i n e d by who does t h e c o d i n g . I n c o n t e n t a n a l y s i s a s m a l l number o f c o d e r s a r e t r a i n e d t o use a s e t o f r u l e s and d e f i n i t i o n s . These e x p e r t s u s u a l l y a t t e n t i v e l y v i e w and i n t e n s i v e l y e v a l u a t e s p e c i f i c m a t e r i a l . 12 A u d i e n c e r e s e a r c h e r s , on t h e o t h e r hand, a l s o use s p e c i f i c m a t e r i a l , f o r example, a p a r t i c u l a r s e r i e s o r e p i s o d e o f a show, b u t i n t h i s case i t i s shown t o a l a r g e number o f v i e w e r s . These v i e w e r s a r e not t r a i n e d i n t h e use o f any c o d i n g system and t h e y do not use a p a r t i c u l a r s e t o f d e f i n i t i o n s and r u l e s t o e v a l u a t e t h e c o n t e n t ; i n s t e a d , t h e i r answers a r e based more on t h e i r own i n d i v i d u a l i n t e r p r e t a t i o n . T h i s approach i s not n e c e s s a r i l y l e s s s y s t e m a t i c t h a n c o n t e n t a n a l y s i s , but i t seems on t h e s u r f a c e t o be more s u b j e c t i v e ( e . g . , L u l l e t a l . 1977). The g o a l o f c o n t e n t a n a l y s i s i s t o d e t e r m i n e what o f t h e p o s s i b l e w o r l d i s p o r t r a y e d . The g o a l o f a u d i e n c e r e s e a r c h i s t o d e t e r m i n e what o f t h i s p o r t r a y e d w o r l d i s p e r c e i v e d . An example o f a u d i e n c e r e s e a r c h i s found i n t h e work o f J e n s e n (1987), who a s s e s s e d p e o p l e ' s memory f o r news programming. H i s s u b j e c t s were a non-random sample s e l e c t e d by a p o l l i n g f i r m t o " p r o c u r e a range o f r e s p o n d e n t s " (p. 8 ) . He c o n t a c t e d them i n i t i a l l y and made arrangements f o r them t o watch p a r t i c u l a r shows i n t h e i r own homes a t t h e i r r e g u l a r l y s c h e d u l e d t i m e s . He was i n t e r e s t e d i n how, and how much, p e o p l e remembered about TV news programming. The s u b j e c t s were i n t e r v i e w e d on t h e t e l e p h o n e t h e f o l l o w i n g day. They knew t h e y . were g o i n g t o be asked q u e s t i o n s about t h e shows, b u t t h e y d i d not know what t h e s e q u e s t i o n s would be. A s t u d y by Condry, Bence, and S c h e i b e (1987) p r o v i d e s an example o f c o n t e n t a n a l y s i s . They e v a l u a t e d t h e non-program c o n t e n t (e.g., commercials) o f S a t u r d a y morning programming. 13 I n t h i s r e s e a r c h , two t r a i n e d c o d e r s , w i t h r e l i a b i l i t y e s t a b l i s h e d a t 89%, e v a l u a t e d a l l programs i n t h e sample, w h i c h was t a k e n from t h e t h r e e U.S. p r i v a t e n etworks, ABC, CBS, and NBC. The d u r a t i o n o f non-program c o n t e n t was t i m e d , and c a t e g o r i z e d i n t o c o m m e r c i a l s , p u b l i c s e r v i c e announcements, program promotions and s t a t i o n i d e n t i f i c a t i o n s , and i n f o r m a t i o n a l d r o p - i n s . D i f f e r e n c e s were examined by s t a t i o n , t i m e o f day, and month. I n a second s t u d y t h e y l o o k e d s p e c i f i c a l l y a t t h e non-program c o n t e n t o f c h i l d r e n ' s programming a t t i m e s o t h e r t h a n S a t u r d a y morning. I t seems c l e a r from t h e s e two examples t h a t b o t h c o n t e n t a n a l y s t s and a u d i e n c e r e s e a r c h e r s a r e concerned w i t h t h e messages on t e l e v i s i o n , b u t t h e i r methods and g o a l s a r e somewhat d i f f e r e n t . These d i f f e r e n c e s become c o m p l i c a t e d because w i t h i n each approach t h e r e i s not unanimous agreement on t h e a p p r o p r i a t e methodology. The f o l l o w i n g two d i m e n s i o n s h e l p t o e x p l a i n some o f t h e d i f f e r e n c e s . The second d i m e n s i o n i n t h e proposed model i s q u a n t i t a t i v e v e r s u s q u a l i t a t i v e a n a l y s i s . T h i s d i m e n s i o n has been a s o u r c e o f d i s s e n s i o n among r e s e a r c h e r s , and t h e r e i s no s h o r t a g e o f o p i n i o n about which r o u t e i s more a p p r o p r i a t e ( S e p s t r u p , 1981). I t i s not always easy t o d i s t i n g u i s h between t h e two terms, but a c l e a r u n d e r s t a n d i n g o f t h e i r meaning i s e s s e n t i a l t o u n d e r s t a n d t h e proposed model. U n d e r s t a n d i n g t h e terms q u a n t i t a t i v e and q u a l i t a t i v e i s h i n d e r e d i n p a r t because t h e i r c o l l o q u i a l and s t a t i s t i c a l meanings a r e d i f f e r e n t . I n g e n e r a l p a r l a n c e , q u a n t i t a t i v e 14 r e f e r s t o c o u n t i n g and q u a l i t a t i v e r e f e r s t o t h e n a t u r e o f some phenomenon. For example, a q u a n t i t a t i v e a n a l y s i s might d e t e r m i n e how many a c t s o f v i o l e n c e were p o r t r a y e d , whereas a q u a l i t a t i v e approach might ask whether t h e v i o l e n c e was p o r t r a y e d as a c c e p t a b l e , o r what i t s y m b o l i z e d . To a s t a t i s t i c i a n , however, b o t h words r e f e r t o c o u n t i n g ; t h e y d i f f e r i n t h e n a t u r e o f t h e phenomena c o u n t e d . Q u a l i t a t i v e r e f e r s t o whether o r not some c a t e g o r y o f " t h i n g " i s p r e s e n t . Q u a n t i t a t i v e r e f e r s t o t h e degree o f t h e p r e s e n c e o f t h e " t h i n g " ( K i r k & M i l l e r , 1986). The q u a n t i t a t i v e a p p r o a c h i n t h e p r e v i o u s example would now be c o n s i d e r e d q u a l i t a t i v e , t h a t i s , whether o r not v i o l e n c e i s p r e s e n t . A q u a n t i t a t i v e ( s t a t i s t i c a l ) approach might measure t h e l e v e l o f v i o l e n c e o f each a c t . Q u a l i t a t i v e d a t a a r e n o m i n a l and q u a n t i t a t i v e d a t a c o u l d be o r d i n a l , i n t e r v a l , o r r a t i o (Kennedy, 1983). A number o f t h e o r i s t s ( e . g . , H o l s t i , 1969) have s a t i s f i e d t h e m s e l v e s t h a t t h e s o l u t i o n t o t h i s c o n t r o v e r s y o v e r q u a n t i t a t i v e v e r s u s q u a l i t a t i v e a n a l y s i s i s f o r r e s e a r c h e r s t o choose whatever method b e s t s u i t s t h e i r purpose on a p a r t i c u l a r o c c a s i o n . T h i s c o n c l u s i o n i s based, however, o n l y on s t a t i s t i c a l c o n s i d e r a t i o n o f t h e s e terms. P r o p o n e n t s o f t h i s v i e w do not address t h e more c o n t e n t i o u s i s s u e o f whether t o count o r t a l k about t h e n a t u r e o f some phenomenon. I n t h e i r c o n t e n t a n a l y s i s o f music t e l e v i s i o n , Sherman and Dominick (1984) used a c o m b i n a t i o n o f s t a t i s t i c a l l y q u a n t i t a t i v e and q u a l i t a t i v e a pproaches. Coders f i r s t e v a l u a t e d whether each v i d e o was a c o n c e p t ( s t o r y , d r a m a t i z a t i o n , o r n a r r a t i v e ) o r performance ( s t u d i o o r c o n c e r t ) p i e c e . I n d i v i d u a l c h a r a c t e r s were c a t e g o r i z e d a c c o r d i n g t o sex, age, economic s t a t u s , e t h n i c i d e n t i t y , and whether o r not p r o v o c a t i v e costumes were worn. Q u a n t i t a t i v e d a t a about s e x u a l i n t i m a c y f e l l on an o r d i n a l s c a l e from f l i r t i n g t o i n t i m a t e t o u c h i n g . S t e e v e s and Smith (1987) d i d an a n a l y s i s o f t h e c l a s s and gender c o n t e n t o f prime-time TV from a s o c i a l i s t f e m i n i s t p e r s p e c t i v e . T h e i r approach was q u a l i t a t i v e i n t h e p h e n o m e n o l o g i c a l sense. Both a u t h o r s watched shows t o g e t h e r a t l e a s t t w i c e and d i s c u s s e d e v i d e n c e about c l a s s and gender. They a l s o watched l a t e r e p i s o d e s and r e s e a r c h e d p r e v i o u s e p i s o d e s . T h i s approach y i e l d e d no n u m e r i c a l d a t a ; r a t h e r , i t y i e l d e d i m p r e s s i o n s about t h e c o n t e n t i n r e l a t i o n t o t h e c a t e g o r i e s o f i n t e r e s t . The f i n a l d i m e n s i o n o f t h e proposed model i s m i c r o a n a l y t i c v e r s u s m a c r o a n a l y t i c . These approaches a r e d i f f e r e n t i a t e d by b o t h t h e t y p e o f phenomena w i t h which t h e y d e a l , and t h e l e v e l a t which t h e y do so. M i c r o s c o p i c a n a l y s i s , as t h e p h rase i m p l i e s , i s v e r y d e t a i l e d , f o c u s i n g on i n c i d e n t s and i n d i v i d u a l c h a r a c t e r s . F o r example, t h e number o f d e a t h s i n a program o r t h e number o f t i m e s a p a r t i c u l a r e t h n i c group i s r e f e r r e d t o might be c o u n t e d . A m a c r o s c o p i c approach i s more g l o b a l . F or example, i n s t e a d o f c o u n t i n g how many p e o p l e d i e i n a program a m a c r o s c o p i c a n a l y s t might ask about t h e g e n e r a l i m p r e s s i o n g i v e n as a r e s u l t o f v i e w i n g t h o s e d e a t h s . The r e s e a r c h by S t e e v e s and S m i t h (1987) was m a c r o a n a l y t i c a t t h e same t i m e t h a t i t was q u a l i t a t i v e . The c o d e r s watched an e p i s o d e more t h a n once, watched l a t e r e p i s o d e s , and r e s e a r c h e d p a s t ones. T h i s gave them a g l o b a l u n d e r s t a n d i n g o f t h e c o n t e n t o f t h e s e r i e s . T h e i r a n a l y s i s was not o f i n d i v i d u a l c h a r a c t e r s and segments, b u t o f t h e show and s e r i e s . They were c l e a r l y i n f l u e n c e d by i n d i v i d u a l b i t s , b u t t h e i r a n a l y s i s o f t h e show was a t a more g l o b a l o r macro l e v e l . S u r l i n , Romanow, and S o d e r l u n d ' s (1984) c o n t e n t a n a l y s i s o f TV i n w h i c h t h e y compared C a n a d i a n and U.S. news programming was done a t a m i c r o l e v e l . Each news s t o r y and f e a t u r e was coded f o r d u r a t i o n , g e o g r a p h i c o r i g i n , t y p e o f c o n t e n t , and p r e s e n t a t i o n . Three d i f f e r e n t c o d e r s d i d t h e r a t i n g , and r e l i a b i l i t i e s were e s t a b l i s h e d . The emphasis on minute d e t a i l i n t h i s s t u d y and i n t h e work o f W i l l i a m s e t a l . (1977, 1982) s t a n d s i n c l e a r c o n t r a s t t o S t e e v e s and Smith (1987). N e i t h e r approach i s n e c e s s a r i l y b e t t e r i n g e n e r a l . R a t h e r , each has unique a t t r i b u t e s w h i c h need t o be u n d e r s t o o d i n d e s i g n i n g r e s e a r c h . These two axes, q u a n t i t y / q u a l i t y and micro/macro, and t h e debates around them r e v o l v e , t o some de g r e e , on t h e i s s u e s o f r e l i a b i l i t y and v a l i d i t y . F o r example, a m i c r o a n a l y t i c q u a n t i t a t i v e c o n t e n t r e s e a r c h e r can r e l i a b l y r e c o r d how many de a t h s o c c u r i n any g i v e n TV show o r s e r i e s . The number o f d e a t h s an average 1 4 - y e a r - o l d has seen on t e l e v i s i o n can t h e n be r e l i a b l y c a l c u l a t e d . Armed w i t h t h i s i n f o r m a t i o n , and t h a t 17 d e r i v e d from e f f e c t s r e s e a r c h , t h e c o n t e n t a n a l y s t can make s t a t e m e n t s about how t h e average 1 4 - y e a r - o l d might be changed by w a t c h i n g TV. A more m a c r o a n a l y t i c q u a l i t a t i v e c o n t e n t r e s e a r c h e r , however, can r i g h t l y f a u l t t h i s e n t e r p r i s e f o r i t s f a i l u r e t o c o n s i d e r what meaning t h e v i e w e r c o n s t r u c t s o u t o f t h i s e x p e r i e n c e (Newcomb, 1978). How many de a t h s a r e p e r c e i v e d t o have t a k e n p l a c e ? W i t h o u t knowing more about t h e v i e w e r , t h e m i c r o a n a l y t i c q u a n t i t a t i v e r e s e a r c h e r can a s s e s s t h e messages a v a i l a b l e , b u t cannot a s s e s s whether t h e p o t e n t i a l messages a r e p e r c e i v e d . The e m p i r i c a l approach may y i e l d a v e r y r e l i a b l e s t a t e m e n t , b u t t h e v a l i d i t y o f i t would be an open q u e s t i o n . The o p p o s i t e extreme t o t h e above example would be e q u a l l y f l a w e d , b u t i n t h i s case f o r emphasis on v a l i d i t y o v e r r e l i a b i l i t y . As e x p l a i n e d e a r l i e r , an extreme m a c r o a n a l y t i c q u a l i t a t i v e o r i e n t a t i o n would c o n s i d e r t h e a c t u a l c o n t e n t o f t e l e v i s i o n as i r r e l e v a n t because t h e c o n s t r u c t e d meanings a r e p o t e n t i a l l y i n f i n i t e . There i s no doubt t h a t t h e r e w i l l be much disag r e e m e n t on t h e c o n t e n t , b u t as M o r l e y (1980) p o i n t s o u t , t h e r e a l s o p r o b a b l y w i l l be much agreement, much more t h a n would be e x p e c t e d w i t h random outcomes. To some degree t h e show c o n t e n t g u i d e s t h e p e r c e i v e d message. To i g n o r e t h i s i s t o i g n o r e t h e a b i l i t y t o r e l i a b l y know some o f t h e i n f o r m a t i o n e s s e n t i a l t o u n d e r s t a n d i n g how t e l e v i s i o n i n f l u e n c e s s o c i e t y . The extreme q u a l i t a t i v e m a c r o a n a l y t i c r e s e a r c h e r i s r i g h t i n c o n s i d e r i n g t h e v i e w e r ' s p e r s p e c t i v e , 18 b u t s h a r e d e x p e r i e n c e t h a t can h e l p t o d e f i n e t h a t p e r s p e c t i v e s h o u l d n o t be i g n o r e d . The s e a r c h f o r a more v a l i d measure o f t e l e v i s i o n c o n t e n t may l e a d t h e q u a l i t a t i v e m a c r o a n a l y t i c r e s e a r c h e r t o s a c r i f i c e r e l i a b i l i t y and l o s e v a l i d i t y i n t h e p r o c e s s . T h i s may be t h e case i n t h e work o f S t e e v e s and Smith (1987). They i n t e r p r e t e d t h e shows from a M a r x i s t f e m i n i s t p e r s p e c t i v e , and went t o c o n s i d e r a b l e l e n g t h s t o u n d e r s t a n d t h e s e messages i n t h e m a t e r i a l t h e y coded. Because t h e y were concerned w i t h deep s t r u c t u r e , however, t h e i r c o n c l u s i o n s may have l i t t l e i n common w i t h t h e average N o r t h American v i e w e r ' s c o n s c i o u s p e r c e p t i o n o f t h e same m a t e r i a l . O b v i o u s l y , t h e s e t h r e e axes a r e n o t o r t h o g o n a l . Most r e s e a r c h e r s ' work can be c h a r a c t e r i z e d a t one c o r n e r o f t h e model. They t e n d n o t t o use d i f f e r e n t o r i e n t a t i o n s t o s o l v e d i f f e r e n t problems. Most r e s e a r c h i n t h e U.S. has been m i c r o a n a l y t i c and q u a n t i t a t i v e , and based on a p o s i t i v i s t s c i e n t i f i c o r i e n t a t i o n . Most E n g l i s h l a nguage books p u b l i s h e d i n N o r t h America on c o n t e n t a n a l y s i s d e f i n e i t s o l e l y i n t h o s e terms ( e . g . , Stone, Dunphy, S m i t h , & O g i l v i e , . 1966; Budd, Thorp, & Donohew, 1967; Gerbner e t a l . , 1969; H o l s t i , 1969; K r i p p e n d o r f , 1980), and t h u s i m p l y t h e r e i s no o t h e r way t o approach TV c o n t e n t b u t t o -count i n c i d e n t s and c h a r a c t e r s . European r e s e a r c h e r s , on t h e o t h e r hand, t e n d t o be more m a c r o a n a l y t i c and q u a l i t a t i v e i n t h e i r a p p r o a c h , and o f t e n work from. M a r x i s t , C r i t i c a l Theory b a c k g r o u n d s . Canadian r e s e a r c h e r s s t r a d d l e b o t h groups. One p o l e o f t h e model t h u s can be c h a r a c t e r i z e d as q u a n t i t a t i v e , m i c r o a n a l y t i c , 19 a p o l i t i c a l , U.S., c o n s e r v a t i v e , and e m p i r i c a l ; t h e o t h e r , as q u a l i t a t i v e , m a c r o a n a l y t i c , European, and M a r x i s t (Rosengren, 1981; S e p s t r u p , 1981). UBC Content A n a l y s i s P r o j e c t The g o a l o f t e l e v i s i o n c o n t e n t a n a l y s i s i s t o d e s c r i b e t h e messages p o r t r a y e d on TV and use t h e f i n d i n g s and t h e o r y t o make an inf o r m e d s t a t e m e n t about p o t e n t i a l v i e w e r e f f e c t s . I n d e s i g n i n g t h e UBC c o d i n g system, t h e g o a l o f W i l l i a m s e t a l . (1985) was t o s t r a d d l e a l l t h r e e o f t h e d i m e n s i o n s j u s t d e s c r i b e d . That i s , t h e y attempted t o c a p t u r e i n a r e l i a b l e c o d i n g system t h e take-home message o f t h e ( m y t h i c a l ) average v i e w e r , and a t t h e same t i m e t o a v o i d t h e p i t f a l l s o f t h e extremes o f both t h e q u a n t i t a t i v e / q u a l i t a t i v e d i m e n s i o n and t h e m i c r o a n a l y t i c / m a c r o a n a l y t i c d i m e n s i o n . I n e f f e c t , a l t h o u g h i t was d e v e l o p e d e a r l i e r (1985), t h e c o d i n g system was d e s i g n e d t o a d d r e s s many o f t h e c o n c e r n s about c o n t e n t a n a l y s i s r a i s e d by Cook, C u r t i n , Ettema, M i l l e r , and Van Camp (1986). A t y p i c a l c o n t e n t a n a l y s t would r e v i e w t h e c o n t e n t many t i m e s t o ensure e i t h e r t h a t e v e r y c h a r a c t e r and i n c i d e n t had been r e c o r d e d ( m i c r o a n a l y t i c approach) o r t h a t e v e r y s u b t l e message had been p i c k e d up ( m a c r o a n a l y t i c a p p r o a c h ) . By c o n t r a s t , t h e UBCCS s p e c i f i e d t h a t c o d e r s watch a TV program i n i t s e n t i r e t y b e f o r e a n s w e r i n g any q u e s t i o n s , and watch i t o n l y once. They were not a l l o w e d t o s t o p t h e v i d e o t a p e a t any t i m e d u r i n g v i e w i n g , n o r t o r e v i e w any s e c t i o n s . (They d i d , however, f a s t f o r w a r d t h r o u g h t h e c o m m e r c i a l s , as t h e s e 20 were not b e i n g coded.) These r e s t r i c t i o n s make t h e show t h e u n i t o f a n a l y s i s ; i n d i v i d u a l c h a r a c t e r s were n o t coded. What t h e co d e r has, somewhat l i k e a normal v i e w e r , i s a g l o b a l i m p r e s s i o n o f t h e show. One major d e p a r t u r e from t h e average v i e w i n g e x p e r i e n c e was t h a t c o d e r s t o o k n o t e s d u r i n g c o d i n g . The n o t e s were few, however, and were r e s t r i c t e d t o a l i s t o f' t h e c o u n t r i e s mentioned, t h e main c h a r a c t e r s and t h e i r e t h n i c i t y , and t h e c o n t e n t o f i n d i v i d u a l program segments. These n o t e s s e r v e d as memory cues f o r f i l l i n g o u t t h e c o d i n g s h e e t . T h i s c l e a r l y d i f f e r s from t h e average v i e w i n g e x p e r i e n c e , b u t i t was b e l i e v e d t o be n e c e s s a r y t o e s t a b l i s h r e l i a b i l i t y among t h e c o d e r s . W i l l i a m s ' team o f 20 t r a i n e d c o d e r s were m o s t l y t h i r d -and f o u r t h - y e a r u n i v e r s i t y s t u d e n t s . They were o t h e r w i s e q u i t e heterogeneous, however, coming from v a r i o u s e t h n i c backgrounds, d i f f e r e n t s o c i o e c o n o m i c s t a t u s (SES) l e v e l s , and so on. I n s p i t e o f t h e i r d i f f e r e n c e s , and perh a p s i n p a r t because o f t h e i r s i m i l a r i t i e s , t h e s e c o d e r s were a b l e t o e s t a b l i s h good r e l i a b i l i t y on most o f t h e q u e s t i o n s t h a t were coded. There were some i t e m s f o r wh i c h r e l i a b i l i t y was poor d e s p i t e a t t e m p t s to. t r a i n p e o p l e t o code shows t h e same way. T h i s might a l s o be t h e c a s e f o r n a i v e v i e w e r s . The q u e s t i o n s i n t h e c o d i n g system were many and v a r i e d . Some were s t a t i s t i c a l l y q u a l i t a t i v e : "Was t h e r e any r e f e r e n c e t o o r p o r t r a y a l o f E n g l i s h Canada o r C a n a d i a n s ? " , whereas o t h e r s were s t a t i s t i c a l l y q u a n t i t a t i v e : " I f y e s ( t o t h e p r e v i o u s q u e s t i o n ) how much o f t h e program's f o c u s d i d i t have?" The coder t h e n had t h e o p t i o n o f r a t i n g i t as e i t h e r -p a s s i n g r e f e r e n c e , minor f o c u s , o r major f o c u s [see A p p e n d i x A, p. 1, numbers 1 (a) and ( b ) ] . Coders were asked t o keep t r a c k o f a l l o f t h e major c h a r a c t e r s w h i l e w a t c h i n g t h e show. I f t h e s e c h a r a c t e r s were N o r t h Americans who were members o f an e t h n i c m i n o r i t y group t h e y were coded i n a more d e t a i l e d way. Prominent i n d i v i d u a l c h a r a c t e r s from a l l e t h n i c groups p o r t r a y e d o r r e f e r r e d t o were coded. Data about each i n c l u d e d : e t h n i c i t y ; s e x ; whether or. not t h e person was p o r t r a y e d as f u n c t i o n i n g w e l l i n mainstream N o r t h American s o c i e t y and i f not whether t h i s was a r e s u l t o f t h e i r e t h n i c i t y ; how s t r o n g l y t h e y were i d e n t i f i e d w i t h t h e i r e t h n i c background; and t h e f i n a l o v e r a l l i m p r e s s i o n ( p o s i t i v e , m i x e d / n e u t r a l , o r n e g a t i v e ) . A l t h o u g h t h e s e i s s u e s were approached i n a n u m e r i c a l way t h e answers s t i l l y i e l d a g e n e r a l i m p r e s s i o n o f t h e n a t u r e o f t h e e t h n i c m i n o r i t i e s mentioned, t h a t i s , a ( n o n s t a t i s t i c a l ) q u a l i t a t i v e e v a l u a t i o n . T h i s s e r i e s o f q u e s t i o n s a l s o f a l l s somewhere between t h e extremes o f both m i c r o and m a c r o s c o p i c e x a m i n a t i o n s . I t i s hoped t h a t t h e UBCCS ( W i l l i a m s e t a l . , 1985) a c h i e v e s . a l l t h e o b j e c t i v e s o u t l i n e d : t h a t i t i s on o c c a s i o n m i c r o a n a l y t i c and more o f t e n m a c r o a n a l y t i c , t h a t i t i s b o t h q u a n t i t a t i v e and q u a l i t a t i v e , and t h a t i t forms a b r i d g e between c o n t e n t a n a l y s i s and a u d i e n c e r e s e a r c h . The c u r r e n t r e s e a r c h was d e s i g n e d s p e c i f i c a l l y t o e v a l u a t e t h e l a t t e r c l a i m , t h a t i s , t o what e x t e n t have W i l l i a m s and h e r s t u d e n t s 22 been s u c c e s s f u l i n c a p t u r i n g i n a r e l i a b l e manner t h e same messages t h a t u n t r a i n e d v i e w e r s p e r c e i v e ? The h y p o t h e s e s d e s c r i b e d below a r e based on b o t h common sense and on a u d i e n c e r e s e a r c h by Himmelweit, S w i f t , and B i b e r i a n (1978) . They c a s t t h e a u d i e n c e as c r i t i c and had a group o f about 1,000 v o l u n t e e r s from t h e B r i t i s h B r o a d c a s t i n g C o r p o r a t i o n ' s v i e w i n g p a n e l e v a l u a t e programs on a number o f d i m e n s i o n s . Twenty shows were s e l e c t e d from peak v i e w i n g t i m e ; 18 were f i c t i o n e n t e r t a i n m e n t , one was a news show, and one was a c u r r e n t a f f a i r s program. They found t h a t t i m e spent w a t c h i n g TV, e d u c a t i o n a l l e v e l , and SES a l l were r e l a t e d t o p e r c e p t i o n s o f t h e programs seen. One o f Himmelweit e t a l . ' s (1978) p r e d i c t i o n s was t h a t t h e c o n s i s t e n c y o f t h e a u d i e n c e ' s r a t i n g s would v a r y as a f u n c t i o n o f program c a t e g o r y o r genre. T h e i r r e s u l t s d i d not s u p p o r t t h i s h y p o t h e s i s . I n s t e a d , a u d i e n c e s c o r e s were more c o n s i s t e n t f o r more p o p u l a r shows t h a n f o r l e s s p o p u l a r ones. Program p o p u l a r i t y was i d e n t i f i e d as a s t i m u l u s v a r i a b l e by Himmelweit e t a l . b u t i t seems more c o r r e c t l y t o be a measure o f v i e w e r f a m i l i a r i t y w i t h a program. That i s , t h e b e t t e r t h e group o f c o d e r s knew t h e show, t h e more c o n s i s t e n t were t h e i r e v a l u a t i o n s . T h i s may a l s o g e n e r a l i z e t o genre, t h a t i s , t h e more o f t e n a group watches s i t u a t i o n comedies, t h e , more s i m i l a r l y t h e y may r a t e a l l s i t u a t i o n comedies. I n some ways, t h e r e s u l t s o f Himmelweit e t a l . (1978) seem c o u n t e r i n t u i t i v e . News shows a r e v e r y d i f f e r e n t from s i t u a t i o n comedies. F o r example, t h e y have more i n d i v i d u a l segments, c o n t a i n much more i n f o r m a t i o n , do n o t have a p l o t , and p r i m a r i l y c o n t a i n d y s p h o r i c r a t h e r t h a n e u p h o r i c i n f o r m a t i o n . I t seems l i k e l y t h a t v i e w e r s would be more c o n s i s t e n t i n t h e i r i n t e r p r e t a t i o n o f s i t u a t i o n comedies t h a n news programs. A n o t h e r r e a s o n t h a t v i e w e r s might d i f f e r e n t i a l l y r espond t o some program c a t e g o r i e s a r e f l o o r and c e i l i n g e f f e c t s . F o r example, t h e UBCCS ( W i l l i a m s e t a l . , 1985) has a l a r g e s e c t i o n on a g g r e s s i o n . S i t u a t i o n comedies do not c o n t a i n much v i o l e n c e and p h y s i c a l a g g r e s s i o n , b u t c o n t a i n a g r e a t d e a l o f v e r b a l a g g r e s s i o n . L e v e l o f agreement among b o t h t r a i n e d and u n t r a i n e d c o d e r s m i g h t t h e r e f o r e be n e a r l y p e r f e c t . T h i s i s an a r t i f a c t o f t h e r e s e a r c h d e s i g n , not s u p p o r t f o r t h e a b i l i t y t o r e l i a b l y code t h e s e d a t a . L i k e w i s e , c r i m e d e t e c t i v e shows o f t e n c o n t a i n a g r e a t d e a l o f v i o l e n c e . I n t h i s case a l l o f t h e s c o r e s g i v e n i n t h e a g g r e s s i o n s e c t i o n might be a t t h e c e i l i n g and s i m i l a r l y a r t i f i c i a l l y i n f l a t e t h e l e v e l o f agreement. C o n t r a r y t o t h e f i n d i n g s o f Himmelweit e t a l . , program c a t e g o r y might be e x p e c t e d t o a f f e c t t h e c o n s i s t e n c y o f t h e messages p e r c e i v e d t o be on TV. Hypotheses I n t h e c u r r e n t r e s e a r c h , a s u b s e t o f t h e programs p r e v i o u s l y e x p e r t - c o d e d u s i n g t h e UBCCS was shown t o n a i v e v i e w e r s who d i d n o t see t h e q u e s t i o n s u n t i l a f t e r t h e y had watched t h e program. The major p r e d i c t i o n was t h a t f o r most q u e s t i o n s t h e n a i v e c o d e r s would agree amongst t h e m s e l v e s i n t h e i r e v a l u a t i o n s o f t h e c o n t e n t , and t h a t t h e i r e v a l u a t i o n s would agree w i t h t h e e x p e r t s ' . That i s , f o r any g i v e n q u e s t i o n i t was e x p e c t e d t h a t t h e s u b j e c t s and t h e e x p e r t s would t e n d t o answer i n t h e same manner. A l t h o u g h agreement was e x p e c t e d t o be good on average, i t a l s o was e x p e c t e d t h a t agreement among n a i v e v i e w e r s , between them and t h e e x p e r t c o d e r s , and among t h e e x p e r t c o d e r s ( r e l i a b i l i t y ) , would v a r y , w i t h agreement f o r some i n d i v i d u a l i t e m s b e i n g o n l y moderate and f o r o t h e r s , low. I t was f u r t h e r p r e d i c t e d t h a t t h e q u e s t i o n s on w h i c h low agreement would be fo u n d among n a i v e c o d e r s would be t h e same as t h o s e on wh i c h t h e e x p e r t s tended t o d i s a g r e e . Two s e p a r a t e b u t dependent p r e d i c t i o n s were made r e g a r d i n g t h e r o l e o f v i e w e r c h a r a c t e r i s t i c s . I f t h e g e n e r a l l e v e l o f agreement between e x p e r t s and n a i v e c o d e r s t u r n e d out t o be low, i t was e x p e c t e d t h a t s u b j e c t c h a r a c t e r i s t i c s (demographic i n f o r m a t i o n e t c . ) would p r e d i c t l e v e l o f agreement and be h e l p f u l i n u n d e r s t a n d i n g s o u r c e s o f agreement. That i s , some subgroups o f n a i v e c o d e r s might d i s a g r e e s t r o n g l y w i t h t h e e x p e r t c o d e r s , whereas o t h e r s might be more s i m i l a r . On t h e o t h e r hand, i f t h e g e n e r a l l e v e l o f agreement between n a i v e c o d e r s and e x p e r t s was h i g h , i t was not e x p e c t e d t h a t s u b j e c t c h a r a c t e r i s t i c s would r e l i a b l y p r e d i c t l e v e l o f agreement. Method S u b j e c t s V o l u n t e e r p a r t i c i p a n t s were 240 t h i r d - and f o u r t h - y e a r u n d e r g r a d u a t e s t u d e n t s (120 male, 120 female) a t t h e 25 U n i v e r s i t y o f B r i t i s h Columbia. As an i n c e n t i v e t o encourage p a r t i c i p a t i o n , a l l 240 names were e n t e r e d i n t o a l o t t e r y , w i t h t h e f i r s t name drawn r e c e i v i n g $100, t h e second $75, and t h e t h i r d $50. U n i v e r s i t y s t u d e n t s were chosen because t h e e x p e r t s were s t u d e n t s , so i t was i m p o r t a n t t o keep t h i s d i m e n s i o n c o n s t a n t . J u s t as t h e e x p e r t c o d e r s v a r i e d i n SES and e t h n i c background, n a i v e c o d e r s from a wide range o f backgrounds were s o l i c i t e d . The e m p i r i c a l d i s t r i b u t i o n s o f SES and e t h n i c i t y among t h o s e who v o l u n t e e r e d f o r t h e s t u d y were used t o s t r a t i f y t h e groups. V o l u n t e e r s were randomly a s s i g n e d t o groups o f 10 per program w i t h t h e f o l l o w i n g q u a l i f i c a t i o n s : t h e r e were f i v e males and f i v e f e m a l e s i n each group, and w i t h i n each sex t h e groups were s t r a t i f i e d a c c o r d i n g t o b o t h e t h n i c i t y and SES. There were f o u r c a t e g o r i e s o f e t h n i c i t y : w h i t e N o r t h A m e r i c a n (60% o f t h e s u b j e c t s ) ; v i s i b l e e t h n i c m i n o r i t y b o r n and r a i s e d i n N o r t h A m e r i c a , f o r example, Canadian b o r n o f Japanese d e s c e n t ( 1 5 % ) ; n ot v i s i b l e e t h n i c m i n o r i t y and not r a i s e d i n N o r t h A m e r i c a , f o r example, F i n n i s h (10%) ; v i s i b l e e t h n i c m i n o r i t y and not r a i s e d i n N o r t h A m e r i c a ( 1 5 % ) . The s u b j e c t s ' SES s c o r e s ( B l i s h e n , C a r r o l , & Moore, 1987) were based on t h e h i g h e r p a r e n t ' s o c c u p a t i o n and ranged from a low o f 2 3.31 ( f o o d and beverage s e r v i n g o c c u p a t i o n s ) t o a h i g h o f 101.74 ( d e n t i s t s ) , w i t h a mean s c o r e o f 57.26 and s t a n d a r d d e v i a t i o n o f 19.00. Coding System The Audience Coding Sheet (ACS, Appendix A) c o n t a i n e d a s u b s e t o f t h e q u e s t i o n s from t h e o r i g i n a l UBCCS ( W i l l i a m s e t a l . , 1985, Appendix B) . Q u e s t i o n s were i n c l u d e d from e v e r y s e c t i o n o f t h e UBCCS. I n c l u s i o n i n t h e ACS depended on s e v e r a l f a c t o r s . F i r s t , i f items r a r e l y o r ne v e r o c c u r r e d d u r i n g t h e e n t i r e week o f programming t h e y were n o t i n c l u d e d . F o r example, a l a r g e s e c t i o n c o n c e r n i n g on homosexual r e l a t i o n s h i p s , was n o t i n c l u d e d because o f t h e v e r y low f r e q u e n c y o f t h e i r p o r t r a y a l . Analogous q u e s t i o n s c o n c e r n i n g h e t e r o s e x u a l r e l a t i o n s h i p s were i n c l u d e d . Second, i f t h e format o f a q u e s t i o n would make i t v e r y d i f f i c u l t f o r an u n t r a i n e d c o d e r t o use t h e n t h e q u e s t i o n e i t h e r was not i n c l u d e d o r was reworded. F o r example, t h e r e were s e v e r a l t a b l e s i n t h e o r i g i n a l c o d i n g system w h i c h f a c i l i t a t e d c o d i n g by t r a i n e d c o d e r s b u t c o u l d not be used w i t h o u t e x t e n s i v e i n s t r u c t i o n . The m a j o r i t y o f t h e s e were not i n c l u d e d . Some q u e s t i o n s a d d r e s s e d i n t h e t a b l e s were reworded, however, i n t o a format t h a t c o u l d be u n d e r s t o o d e a s i l y by u n t r a i n e d c o d e r s . T h i r d , p i l o t t e s t i n g o f t h e ACS i n d i c a t e d t h a t some q u e s t i o n s i n t h e UBCCS were ambiguous. These q u e s t i o n s were r e p h r a s e d i n t o a more u n d e r s t a n d a b l e format. Care was t a k e n t o change o n l y t h e f o r m a t , n ot t h e meaning o f any q u e s t i o n . Data were c o l l e c t e d f o r a l l o f t h e 129 i t e m s i n t h e ACS and a s u b s e t o f v a r i a b l e s was chosen f o r a n a l y s e s d e s i g n e d t o answer t h e b a s i c q u e s t i o n posed i n . t h i s t h e s i s : do n a i v e c o d e r s a g r e e w i t h t h e e x p e r t s ? Subsequent a n a l y s e s and a r t i c l e s w i l l f o c u s on t h e e x t e n t t o w h i c h t h e f i n d i n g s f o r t h i s s u b s e t can be g e n e r a l i z e d t o o t h e r t y p e s o f i t e m s . The 27 v a r i a b l e s s e l e c t e d f o r a n a l y s i s a r e l i s t e d i n T a b l e 1 w i t h a s h o r t h a n d i d e n t i f y i n g name and t h e i r l o c a t i o n i n t h e ACS. I n s e r t T a b l e 1 about h e r e S e v e r a l c r i t e r i a were used t o s e l e c t t h i s s u b s e t o f v a r i a b l e s . F i r s t , b o t h c a t e g o r i c a l and e v a l u a t i v e q u e s t i o n s were i n c l u d e d . Answers were e x p e c t e d f o r a l l o f t h e i t e m s ; l e a v i n g t h e q u e s t i o n b l a n k was not a v a l i d o p t i o n . N e v e r t h e l e s s , a l l o f t h e n a i v e c o d e r s d i d not answer a l l o f t h e q u e s t i o n s . T a b l e 2 l i s t s t h e f r e q u e n c y and p e r c e n t a g e o f m i s s i n g d a t a f o r each v a r i a b l e . The a s t u t e r e a d e r w i l l n o t i c e t h a t f o r some o f t h e r e s u l t s more d a t a were m i s s i n g t h a n i s i n d i c a t e d i n T a b l e 2. T h i s o c c u r r e d because some o f t h e q u e s t i o n s were r e c o d e d b e f o r e t h e y were a n a l y s e d . PAUTETH, PAUTSX, PFOCMX, and PMOCMX, have a "Not A p p l i c a b l e " o p t i o n . F o r example, i f t h e r e were no p e o p l e i n t h e program, t h e c o d e r would check NA. Because t h i s o p t i o n does not f i t on t h e continuum o f t h e q u e s t i o n i t was e l i m i n a t e d from t h e a n a l y s e s . O t h e r q u e s t i o n s formed a q u a s i - c o n t i n u u m , e.g., PAG. These q u e s t i o n s have a 5 - p o i n t s c a l e r a n g i n g from d e f i n i t e l y n o t p r e s e n t (1) t h r o u g h unsure (3) t o d e f i n i t e l y p r e s e n t (5) . Coders ( b o t h t r a i n e d and n a i v e ) were i n s t r u c t e d t o t r y t o check o n l y (1) o r ( 5 ) ; (2) was t o be used i f t h e y t h o u g h t i t was (1) b u t were not a b s o l u t e l y s u r e , w i t h t h e same i n s t r u c t i o n s f o r (4) and (5) ; (3) was t o be checked o n l y i f t h e y were g u e s s i n g . These items were d e s i g n e d on t h e a d v i c e o f Huesmann (1985), who found t h a t 28 t h i s f o r m a t y i e l d e d agreement w i t h d e t a i l e d m i c r o s c o p i c a n a l y s e s done on t h e same program sample. The d a t a from t h e UBCCS w i l l be a n a l y z e d and p u b l i s h e d i n t h e manner i n which t h e y were i n t e n d e d t o be used, on a t w o - p o i n t s c a l e o f p r e s e n c e v e r s u s absence, w i t h "not s u r e " recoded t o m i s s i n g . I n s e r t T a b l e 2 about here The f i n a l c r i t e r i o n f o r s e l e c t i o n o f i t e m s f o r a n a l y s e s was t o a t t e m p t t o i n c l u d e q u e s t i o n s from a l l s e c t i o n s o f t h e ACS. There a r e no q u e s t i o n s from P a r t 1 on c o u n t r i e s ; 1 q u e s t i o n from P a r t 2 on e t h n i c s ; 8 q u e s t i o n s from P a r t 3 on t h e s e x e s ; 4 q u e s t i o n s from P a r t 4 on a g g r e s s i o n ; 4 q u e s t i o n s from P a r t 5 on I s s u e s , c o n t r o v e r s i e s and dilemmas; and 5 q u e s t i o n s from P a r t 6 on g l o b a l i s s u e s . A t t h e end o f t h e (ACS) c o d i n g s heet t h e n a i v e c o d e r s p r o v i d e d demographic i n f o r m a t i o n about TV use, h o u r s o f s t u d y i n g , f a m i l i a r i t y w i t h t h e program seen, e t h n i c i t y , and SES v a r i a b l e s . These i t e m s a r e l i s t e d a t t h e end o f t h e ACS, s e c t i o n 7, pp. 20-21. Ag a i n , , not a l l t h e d a t a c o l l e c t e d were used i n t h e c u r r e n t a n a l y s e s . T a b l e 3 l i s t s t h e v a r i a b l e s used, and any r e c o d i n g t h a t was done. I n s e r t T a b l e 3 about h e r e 29 Programs The sample o f programs coded by W i l l i a m s ( e . g . , W i l l i a m s e t a l . , 1988) i s an e n t i r e week o f t e l e v i s i o n v i d e o t a p e d from each o f t h e seven major networks r e c e i v e d a c r o s s Canada d u r i n g e a r l y O c t o b e r , 1985. Data on t h e p o p u l a r i t y o f t h e s e programs i n t h e G r e a t e r Vancouver a r e a were o b t a i n e d from t h e Bureau o f B r o a d c a s t Measurement (BBM). The BBM sampled t h r e e weeks from mid-October t o t h e end o f November, 1985. T h i s p a r t i c u l a r s i x - w e e k p e r i o d was s p e c i f i e d t o t h e networks i n advance, but t h e y d i d n o t know which t h r e e weeks i n t h a t p e r i o d would be sampled. The week v i d e o t a p e d by W i l l i a m s was i n t e n t i o n a l l y not i n t h e BBM sample. I t would not have been p o s s i b l e t o have a t l e a s t 10 s u b j e c t s v i e w each o f t h e 1089 programs i n t h e v i d e o t a p e d sample, so a s u b s e t o f 24 programs was s e l e c t e d f o r t h i s s t u d y . They were chosen from t h e s i x E n g l i s h c h a n n e l s on t h e b a s i s o f p o p u l a r i t y . I n c l u s i o n o f t h e F r e n c h programs would add an i m p o r t a n t d i m e n s i o n t o t h i s r e s e a r c h , b u t t h i s was not done f o r two r e a s o n s . F i r s t , because t h e F r e n c h v i e w i n g a u d i e n c e i n Vancouver i s so s m a l l none o f t h e programs was p o p u l a r enough t o meet t h e c r i t e r i a . Second, s i n c e t h e r e a r e so few F r e n c h s p e a k i n g s t u d e n t s a t UBC r e c r u i t m e n t would have been a p r o b l e m . W i t h t h e e x c e p t i o n o f news, a l l programs were s e l e c t e d from p r i m e - t i m e , d e f i n e d as 7:00 pm t o 11:00 pm, Monday t o Sunday, when t h e a u d i e n c e i s on a v e r a g e l a r g e s t . The p r i m e -t i m e programs i n W i l l i a m s ' sample have been c a t e g o r i z e d i n t o 30 one o f s i x program s u b - t y p e s , u s i n g a s e t o f d e f i n i t i o n s d e v e l o p e d by A l e t h a Huston and John W r i g h t and t h e i r c o l l e a g u e s a t t h e U n i v e r s i t y o f Kansas C e n t e r f o r R e s e a r c h on t h e I n f l u e n c e o f T e l e v i s i o n on C h i l d r e n (CRITC, 1983). The s i x program c a t e g o r i e s a r e : news, documentary, n o n - f i c t i o n e n t e r t a i n m e n t , s i t u a t i o n comedy, c r i m e - d e t e c t i v e , and o t h e r drama. The f o u r most p o p u l a r p r i m e - t i m e programs i n each o f t h e l a s t f i v e c a t e g o r i e s were used i n t h i s s t u d y , w i t h t h e c a v e a t t h a t o n l y one e p i s o d e o f any program s e r i e s was i n c l u d e d , even i f a n o t h e r e p i s o d e shown i n t h e same week was among t h e f o u r most p o p u l a r shows i n t h a t c a t e g o r y . I n t h e news c a t e g o r y , CBC i s t h e o n l y E n g l i s h network t h a t shows news i n p r i m e - t i m e . F o r t h e news c a t e g o r y o n l y , t h e f o u r most p o p u l a r n a t i o n a l network programs were i n c l u d e d i n t h e sample, r e g a r d l e s s o f t h e t i m e t h e y were a i r e d . The f o u r most p o p u l a r news shows were s e l e c t e d w i t h t h e c a v e a t t h a t o n l y one show p e r network was chosen. The sample o f 24 programs used i n t h i s s t u d y i s shown i n T a b l e 4. I n s e r t T a b l e 4 about h e r e F o r t h e p u r p o s e s o f some a n a l y s e s t h e shows were n o t broken down by c a t e g o r y , and f o r o t h e r s t h e y were grouped i n t o t h r e e c a t e g o r i e s . The primary- c o n c e p t u a l d i v i s i o n f o r t h e s e c a t e g o r i e s i s f i c t i o n v e r s u s n o n - f i c t i o n . N o n - f i c t i o n programs were f u r t h e r d i v i d e d i n t o " h a r d " news and o t h e r non-f i c t i o n . News shows t e n d t o have much s h o r t e r segments o r b i t l e n g t h s and many more o f them, as w e l l as more prominent c h a r a c t e r s , so t h e memory demands on t h e c o d e r s a r e l i k e l y t o be g r e a t e r t h a n i n o t h e r programs. The two h a r d news programs t h e r e f o r e were s e p a r a t e d from o t h e r n o n - f i c t i o n programs f o r some a n a l y s e s . The f i r s t o f t h e t h r e e c a t e g o r i e s t h u s was t h e two news programs (shows 1 and 2) , t h e second was t h e o t h e r n o n - f i c t i o n programs (shows 3 t o 1 2 ) , and t h e t h i r d c a t e g o r y was t h e 12 f i c t i o n programs (shows 13 t o 2 4 ) . P r o c e d u r e The a u t h o r o r a r e s e a r c h a s s i s t a n t went t o t h i r d - and f o u r t h - y e a r u n d e r g r a d u a t e c l a s s e s , p r i m a r i l y i n P s y c h o l o g y , t o r e c r u i t s u b j e c t s . The s t u d y was d e s c r i b e d b r i e f l y , w i t h an emphasis on what would be r e q u i r e d o f p a r t i c i p a n t s . They were t o l d d u r i n g t h i s i n i t i a l c o n t a c t t h a t t h e y would be w a t c h i n g a TV show and a n s w e r i n g some q u e s t i o n s about i t a f t e r w a r d . No mention o f t h e e x p e r t r a t i n g s was made, t h e n o r l a t e r . S t u d e n t s who a g r e e d t h e n , i n p r i n c i p l e , t o v o l u n t e e r completed a one-page i n f o r m a t i o n s h e e t , (see A ppendix C) . These v o l u n t e e r s were l a t e r c o n t a c t e d by phone t o a r r a n g e a s p e c i f i c d a t e and t i m e f o r t h e i r v i e w i n g . P a r t i c i p a n t s watched t h e i r program i n t h e l a b i n s m a l l groups o f 1 t o 5 p a r t i c i p a n t s . A t t h e s t a r t o f t h e c o d i n g s e s s i o n t h e y were g i v e n a copy o f t h e A u d i e n c e Coding Sheet (ACS) and a s k ed t o r e a d t h e i n s t r u c t i o n s . These i n s t r u c t i o n s (Appendix A) e x p l a i n e d t h a t p a r t i c i p a n t s c o u l d t a k e a few n o t e s d u r i n g t h e show, i n p a r t i c u l a r , i n f o r m a t i o n about i n d i v i d u a l segments o f t h e program, t h e names o f t h e major 32 c h a r a c t e r s , and whether o r n o t each c h a r a c t e r came from a N o r t h American e t h n i c m i n o r i t y group. The i n s t r u c t i o n s e x p l a i n e d t h a t t h e n o t e s were f o r t h e p a r t i c i p a n t s 1 own use as r e m i n d e r s when t h e y were l a t e r f i l l i n g o u t t h e c o d i n g s h e e t . P a r t i c i p a n t s a l s o were t o l d by t h e e x p e r i m e n t e r t h a t t h e y would be asked o n l y g l o b a l q u e s t i o n s i n t e n d e d t o c a p t u r e t h e i r g e n e r a l i m p r e s s i o n o f t h e show. They were i n s t r u c t e d n o t t o l o o k a t any o f t h e q u e s t i o n s b e f o r e o r w h i l e w a t c h i n g t h e program. Once t h e y were f a m i l i a r w i t h t h e i n s t r u c t i o n s t h e y watched t h e e n t i r e show. They f a s t - f o r w a r d e d t h r o u g h t h e c o m m e r c i a l s , and d i d not s t o p o r r e v i e w t h e t a p e . A t t h e c o n c l u s i o n o f t h e program t h e y f i l l e d out t h e r e m a i n d e r o f t h e c o d i n g s h e e t , which t o o k a p p r o x i m a t e l y 3 0 m i n u t e s . A n a l y s e s The a n a l y s e s were d e s i g n e d t o a d d r e s s s e v e r a l q u e s t i o n s . The f i r s t s e t f o c u s e d on agreement among t h e n a i v e c o d e r s . The second s e t o f a n a l y s e s f o c u s e d on agreement between t h e e x p e r t and t h e n a i v e c o d e r s . The e x p e r t s ' r a t i n g , w h i c h can be t h o u g h t o f s t a t i s t i c a l l y as a p o p u l a t i o n p a r a m e t e r , was compared w i t h t h e s c o r e s g i v e n by t h e n a i v e c o d e r s , f o r each q u e s t i o n w i t h i n each show. There i s no i n f e r e n t i a l s t a t i s t i c t o do t h i s . T a t e ' s (1977) g o a l was t o make t h i s same com p a r i s o n but t h e s t a t i s t i c a l approach he chose was not a p p r o p r i a t e t o answer t h e q u e s t i o n o f e x t e n t o f agreement. He compared t h e s c o r e g i v e n by an e x p e r t c o d e r f o r each i t e m w i t h t h e mean o f t h e s c o r e s a s s i g n e d by h i s sample a u d i e n c e . T h i s approach does n o t d i f f e r e n t i a t e between an i n s t a n c e o f p e r f e c t agreement, t h a t i s , a l l members o f t h e sample a u d i e n c e g i v i n g e x a c t l y t h e same s c o r e as t h e e x p e r t c o d e r , and one o f no agreement i n which t h e sample a u d i e n c e ' s mean s c o r e c o i n c i d e s w i t h t h a t o f t h e e x p e r t b u t t h e a u d i e n c e s c o r e s a r e sp r e a d a c r o s s a l l p o i n t s on t h e s c a l e . A d e s c r i p t i v e s t a t i s t i c , c a l l e d a T s c o r e , t h a t a d e q u a t e l y answers t h e main q u e s t i o n b e h i n d t h i s r e s e a r c h was e s p e c i a l l y d e v e l o p e d f o r t h i s s t u d y by P e t e r Schumacher o f t h e U n i v e r s i t y o f B r i t i s h Columbia S t a t i s t i c a l C o n s u l t i n g and Res e a r c h L a b o r a t o r y . F o r any g i v e n i t e m t h e T s c o r e y i e l d s a v a l u e between -1 and +1 t h a t i s a measure o f d i s a g r e e m e n t o r agreement between t h e n a i v e c o d e r s and t h e e x p e r t s . I t i s d e f i n e d i n such a way t h a t -1 i n d i c a t e s no n a i v e c o d e r s agree w i t h t h e e x p e r t ; 1 i n d i c a t e s t h a t a l l o f t h e n a i v e c o d e r s agree w i t h t h e e x p e r t ; and 0 i n d i c a t e s chance agreement, t h a t i s , t h e n a i v e c o d e r s ' s c o r e s a r e e q u a l l y d i s t r i b u t e d a c r o s s a l l p o i n t s on t h e s c a l e . A t t h e same t i m e t h i s s t a t i s t i c s t a n d a r d i z e s t h e s c a l e s so t h a t t h e same s c o r e i n d i c a t e s t h e same l e v e l o f agreement r e g a r d l e s s o f t h e l e n g t h o f t h e s c a l e from w h i c h i t was d e r i v e d . T h i s i s an i m p o r t a n t f e a t u r e because t h e ite m s i n t h e ACS and t h e UBCCS have s c a l e s r a n g i n g from 2 t o 5 p o i n t s . The T s c o r e i s d e s c r i b e d i n d e t a i l i n Appendix D. The t h i r d s e t o f a n a l y s e s f o c u s e d on t h e r e l a t i o n s h i p between e x p e r t - n a i v e c o d e r agreement ( i . e . , T s c o r e s ) and agreement among e x p e r t c o d e r s ( i . e . , r e l i a b i l i t y ) . The f i n a l s e t o f a n a l y s e s a d d r e s s e d t h e g e n e r a l i z a b i l i t y o f e x p e r t - n a i v e 34 c o d e r agreement a c r o s s n a i v e c o d e r s , t h a t i s , t h e e x t e n t t o wh i c h s u b j e c t v a r i a b l e s ( e . g . , SES) p r e d i c t e d agreement. R e s u l t s Agreement among n a i v e c o d e r s The f i r s t s t e p i n a n a l y z i n g t h e d a t a was t o d e t e r m i n e t h e e x t e n t t o wh i c h t h e n a i v e c o d e r s tended t o answer t h e q u e s t i o n s i n a s i m i l a r way. K e n d a l l ' s c o e f f i c i e n t o f concordance, K e n d a l l ' s W, (Ferguson, 1976; Hays, 1981) was c a l c u l a t e d f o r t h e 22 v a r i a b l e s w i t h i n each show. When any d a t a were m i s s i n g f o r any v a r i a b l e , however, t h e e n t i r e c a s e had t o be thrown out f o r t h a t a n a l y s i s . To s a f e g u a r d a g a i n s t t h i s p r o v i d i n g s p u r i o u s l y h i g h o r low r e s u l t s , K e n d a l l ' s W was c a l c u l a t e d i n two ways. F i r s t , t h e v a l u e f o r each show was c a l c u l a t e d u s i n g a l l o f t h e v a r i a b l e s . I n t h i s approach t h e number o f s u b j e c t s e n t e r i n g i n t o t h e a n a l y s i s f l u c t u a t e s . Second, t h e a n a l y s e s were redone u s i n g o n l y v a r i a b l e s f o r which a l l d a t a were p r e s e n t . I n t h i s approach, t h e number o f v a r i a b l e s f l u c t u a t e s . The average v a l u e f o r each o f t h e t h r e e program c a t e g o r i e s was c a l c u l a t e d . The r e s u l t s o f b o t h approaches a r e summarized i n T a b l e s 5. The average K e n d a l l ' s W and X2 v a l u e s f o r t h e t h r e e program c a t e g o r i e s u s i n g o n l y t h e v a r i a b l e s f o r which a l l s u b j e c t s had responded were: News, W=.63945, X 2=61.5166; o t h e r n o n - f i c t i o n , W=.72653, X 2=40.26538; f i c t i o n , W=.69946, X 2=61.1744. The c o r r e s p o n d i n g v a l u e s u s i n g a l l s u b j e c t s were: W=.60800, X 2=97.15520; W=.62122, X 2=77.98239; W=.64978, X 2=102.04043. N e i t h e r a pproach p r o v i d e s a p e r f e c t answer t o t h e q u e s t i o n o f agreement among n a i v e c o d e r s , b u t t h e two approaches do converge on an answer. The r e s u l t s i n d i c a t e s i g n i f i c a n t c o n c o r d a n c e amongst t h e s u b j e c t s i n t h e way t h e y answered t h e q u e s t i o n s f o r each o f t h e t h r e e program c a t e g o r i e s . I n s e r t T a b l e 5 about h e r e Agreement between e x p e r t and n a i v e c o d e r s The T s c o r e s on q u e s t i o n s t h a t had been reworded when t a k e n from t h e UBCCS d i d not d i f f e r from t h o s e t h a t had not been reworded when t a k e n from t h e UBCCS f o r use i n t h e ACS, t ( 2 0 ) = - . 9 5 , p.= .354. A l l 129 i t e m s i n t h e ACS were o r d e r e d from low t o h i g h T s c o r e s t o a s s e s s t h e e m p i r i c a l d i s t r i b u t i o n . The 22 q u e s t i o n s a n a l y z e d i n t h i s s t u d y were t h e n compared t o t h e t o t a l d i s t r i b u t i o n . F i v e d i s t r i b u t i o n s need t o be d i s c u s s e d . The f i r s t i s f o r t h e 24 shows used i n t h i s s t u d y and a l l 129 v a r i a b l e s i n t h e ACS. The second i s f o r t h e 24 shows and t h e s u b s e t o f 22 v a r i a b l e s s e l e c t e d f o r o t h e r a n a l y s e s . The n e x t t h r e e i n v o l v e t h i s same s u b s e t o f 22 v a r i a b l e s , but f o r t h e t h r e e s e p a r a t e program c a t e g o r i e s d e s c r i b e d e a r l i e r , news, o t h e r n o n - f i c t i o n , and f i c t i o n . The d e s c r i p t i v e s t a t i s t i c s f o r t h e s e f i v e d i s t r i b u t i o n s a r e shown i n T a b l e 6 and i n F i g u r e s 2 t h r o u g h 6. The means f o r a l l o f t h e d i s t r i b u t i o n s i n T a b l e 6 a r e v e r y s i m i l a r , v a r y i n g a maximum o f .097. The h i g h e s t mean i s f o r t h e d i s t r i b u t i o n o f a l l v a r i a b l e s . The medians t e n d t o be h i g h e r t h a n t h e means and range from .785 36 f o r t h e h a r d news, t o .530 f o r t h e 22 v a r i a b l e s and a l l shows. The s t a n d a r d d e v i a t i o n s a r e g e n e r a l l y low, w i t h two e x c e p t i o n s , t h e h a r d news shows, and t h e f u l l program sample f o r a l l 129 q u e s t i o n s . I t i s not s u r p r i s i n g t h a t t h e a d d i t i o n o f 107 v a r i a b l e s i n t h e l a t t e r c a s e y i e l d e d a h i g h e r . The d i s t r i b u t i o n o f t h e T s c o r e s f o r t h e h a r d news shows, on t h e o t h e r hand, s u g g e s t s t h a t t h e y d i f f e r from t h e o t h e r program t y p e s . The median l e v e l o f agreement was h i g h e s t f o r news (.785), but t h e range a l s o , was g r e a t e s t (1.5; s t a n d a r d d e v i a t i o n .438). T h i s i s c o n s i s t e n t w i t h t h e r a t i o n a l e f o r p l a c i n g them i n a s e p a r a t e c a t e g o r y . News programs d i f f e r from most o t h e r TV programs, and a l l used i n t h i s s t u d y , i n h a v i n g many more s h o r t b i t s o r segments and a g r e a t e r number o f prominent c h a r a c t e r s . T h i s c r e a t e s g r e a t e r memory demands f o r t h e c o d e r s . I n s e r t T a b l e 6 and F i g u r e s 2 t o 6 about h e r e Taken t o g e t h e r , t h e d a t a from T a b l e 6 and F i g u r e s 2 t h r o u g h 6 d e m o n s t r a t e t h a t t h e s u b s e t o f 22 v a r i a b l e s chosen f o r a n a l y s i s i n t h i s s t u d y i s s i m i l a r t o t h e t o t a l s e t o f v a r i a b l e s and t h e r e f o r e r e p r e s e n t a t i v e . Moreover, t h e s u b s e t f a l l s p r i m a r i l y w i t h i n t h e h i g h end o f t h e l a r g e r d i s t r i b u t i o n . R e l a t i o n s h i p between T s c o r e s and r e l i a b i l i t y T s c o r e s d e s c r i b i n g t h e n a i v e c o d e r s ' agreement w i t h t h e e x p e r t s were found t o be s i g n i f i c a n t l y c o r r e l a t e d w i t h t h e r e l i a b i l i t y s c o r e s f o r t h e e x p e r t c o d e r s , r=.65, t(20)=3.82, p_<.001 (see T a b l e 7 ) . T h i s i n d i c a t e s t h a t t h e q u e s t i o n s on wh i c h t h e n a i v e c o d e r s t e n d not t o agree w i t h t h e e x p e r t s t e n d t o be t h e same ones on which t h e e x p e r t s t e n d n ot t o agree amongst t h e m s e l v e s . I n s e r t T a b l e 7 about h e r e G e n e r a l i z a t i o n o f t h e r e s u l t s a c r o s s n a i v e c o d e r s R e g r e s s i o n a n a l y s e s were used t o a s s e s s whether e x p e r t -n a i v e c o d e r agreement a p p l i e d e q u a l l y a c r o s s subgroups o f n a i v e c o d e r s . The p r e d i c t o r v a r i a b l e s f o r t h e s u b j e c t s a r e l i s t e d i n T a b l e 3 a l o n g w i t h any r e c o d i n g done. F u l l s t e p - w i s e m u l t i p l e r e g r e s s i o n s were done, u s i n g e x t e n t o f agreement (Tscores) as t h e p r e d i c t e d v a r i a b l e s and t h e s u b j e c t d a t a as t h e p r e d i c t o r v a r i a b l e s . T h i s approach i n d i c a t e s t h e r e l a t i v e importance o f each s u b j e c t v a r i a b l e found t o s i g n i f i c a n t l y p r e d i c t agreement. I t i s i m p o r t a n t t o keep i n mind t h a t what i s b e i n g p r e d i c t e d i s t h e s i m i l a r i t y between t h e u n t r a i n e d and t r a i n e d c o d e r s ' s c o r e s f o r each o f t h e 22 i t e m s . Of t h e 10 p r e d i c t o r ( s u b j e c t ) v a r i a b l e s , 7 p r e d i c t e d a t l e a s t 1 o f t h e 22 dependent (item) v a r i a b l e s . C o n v e r s e l y , 7 o f t h e 22 i t e m T s c o r e s were p r e d i c t e d by one o r more o f t h e s u b j e c t v a r i a b l e s . T a b l e 8 l i s t s t h e s i g n i f i c a n c e o f t h e p r e d i c t o r s and t h e p e r c e n t a g e o f t h e p r e d i c t e d v a r i a n c e . 38 The number o f s i g n i f i c a n t p r e d i c t i o n s (11) i s what would be p r e d i c t e d p u r e l y by chance. They a r e d e s c r i b e d b r i e f l y f o r t h e i n f o r m a t i o n o f o t h e r r e s e a r c h e r s who o b t a i n s i m i l a r f i n d i n g s . I n s e r t T a b l e 8 about h e r e *** T a n n i s : Because I'm t a k i n g t h e s t a n c e t h a t t h e r e g r e s s i o n a n a l y s e s don't t e l l us a n y t h i n g I'm o f t h e o p i n i o n t h a t t h e b a l a n c e o f t h i s s h o u l d be d e l e t e d . Do you agree? C o d i n g t h e p r e s e n c e v e r s u s absence o f t e e n a g e r s i n a show (TEENREP) was n e g a t i v e l y r e l a t e d t o how much t h e v i e w e r watched t e l e v i s i o n (TVTOT). That i s , t h e more t e l e v i s i o n t h e n a i v e c o d e r s r e p o r t e d w a t c h i n g , t h e l e s s l i k e l y t h e y were t o code t h e p r e s e n c e o r absence o f t e e n a g e r s i n t h e same way as e x p e r t c o d e r s . L i k i n g t h e show was found t o p r e d i c t c o d i n g t h e presence v e r s u s absence o f n o n - t r a d i t i o n a l female b e h a v i o r (NTRFEM). The more t h e n a i v e c o d e r l i k e d t h e show t h e y watched, t h e more t h e y coded l i k e an e x p e r t c o d e r . The l e s s f a m i l i a r n a i v e c o d e r s were w i t h N o r t h American c u l t u r e , t h e l e s s l i k e l y t h e y were t o code t h e degree o f problems (PROB) l i k e an e x p e r t . Female n a i v e c o d e r s tended t o code PROBlems more l i k e e x p e r t c o d e r s t h a n d i d male n a i v e c o d e r s . The f r e q u e n c y o f w a t c h i n g a show (SHFRQ) was p o s i t i v e l y r e l a t e d t o one v a r i a b l e and n e g a t i v e l y r e l a t e d t o 2 o t h e r s . 39 The more o f t e n t h e n a i v e c o d e r r e p o r t e d s e e i n g t h e s e r i e s t h e y coded t h e more l i k e an e x p e r t coder were t h e i r e v a l u a t i o n s o f t h e degree o f r o m a n t i c c o n t e n t (ROMNCE) i n t h e show. The more f r e q u e n t l y n a i v e c o d e r s r e p o r t e d h a v i n g seen t h e s e r i e s , however, t h e l e s s l i k e l y t h e y were t o code l i k e an e x p e r t t h e p r e s e n c e o f p h y s i c a l a g g r e s s i o n (PAG) and t h e p o l i t i c a l l e a n i n g s o f t h e show (LFTRT). Two d i f f e r e n t v a r i a b l e s p r e d i c t e d e v a l u a t i n g t h e preponderance o f males v e r s u s females among t h e prominent c h a r a c t e r s (SXPROM). F i r s t , t h e h i g h e r t h e n a i v e c o d e r ' s o c c u p a t i o n a l g o a l (OCGOAL) on t h e SES s c a l e ( B l i s h e n , C a r r o l l , & Moore, 1987) , t h e more l i k e l y t h e n a i v e c o d e r was t o code l i k e an e x p e r t t h e gender mix o f t h e p r o m i n e n t c h a r a c t e r s . Second, t h e more t h e s u b j e c t s r e p o r t e d t h e y t y p i c a l l y s t u d i e d , t h e l e s s l i k e l y t h e y were t o code SXPROM l i k e an e x p e r t . D i s c u s s i o n The r e s u l t s i n d i c a t e t h a t d e s p i t e i t s emphasis on s y s t e m a t i c and r e l i a b l e assessment o f t e l e v i s i o n c o n t e n t , t h e c o d i n g system d e v e l o p e d by W i l l i a m s , and h e r s t u d e n t s f o r use by t r a i n e d c o d e r s i n c o n t e n t a n a l y s i s does, f o r t h e most p a r t , y i e l d t h e same r e s u l t s as does a u d i e n c e r e s e a r c h w i t h u n t r a i n e d v i e w e r s . The n a i v e c o d e r s i n t h i s r e s e a r c h c l e a r l y d i d n o t answer a l l o f t h e q u e s t i o n s i n an i d e n t i c a l manner, b u t t h e r e was a good d e a l o f concordance i n t h e i r answers. Moreover, t h e y t e n d e d t o agree w i t h t h e e x p e r t s . O v e r a l l , t h e h i g h l e v e l o f agreement between t h e n a i v e and e x p e r t coders 40 s u p p o r t s t h e e n t e r p r i s e o f c o n t e n t a n a l y s i s i n g e n e r a l , and t h i s s ystem o f c o n t e n t a n a l y s i s i n p a r t i c u l a r . As s t a t e d e a r l i e r , communication i s t h e r e s u l t o f t h e i n t e r a c t i o n o f v i e w e r and message c h a r a c t e r i s t i c s . I n r e a l i t y i t i s p r o b a b l y n o t t h i s s i m p l e , b u t f o r i l l u s t r a t i v e p u rposes t h i s can be seen as a continuum, r a n g i n g from i n s t a n c e s i n wh i c h message > c h a r a c t e r i s t i c s predominate i n d e t e r m i n i n g t h e communication outcome t o i n s t a n c e s i n w h i c h v i e w e r c h a r a c t e r i s t i c s p r e d o m i n a t e . I n comparing e x p e r t and n a i v e c o d e r s , when v i e w e r c h a r a c t e r i s t i c s predominate i t would be e x p e c t e d t h a t t h e r e would be low agreement among u n t r a i n e d c o d e r s , t h a t t h e r e l i a b i l i t y o f t r a i n e d c o d e r s would be low, and t h a t t h e r e would be l i t t l e o r no agreement between u n t r a i n e d and t r a i n e d c o d e r s . When t h e c h a r a c t e r i s t i c s o f t h e message pred o m i n a t e i t would be e x p e c t e d t h a t t h e r e would be h i g h agreement among u n t r a i n e d c o d e r s , h i g h r e l i a b i l i t y f o r t h e t r a i n e d c o d e r s , and h i g h agreement between t r a i n e d and u n t r a i n e d c o d e r s . The d a t a a r e c o n s i s t e n t w i t h t h i s h y p o t h e s i s . On some i t e m s i t was i m p o s s i b l e t o t r a i n c o d e r s t o r e l i a b l y s c o r e t h e i t e m s i n t h e same way. These tended t o be t h e same ite m s on whi c h n a i v e c o d e r s d i d n o t agree ( e . g . , LFTRT). S i m i l a r l y , on some i t e m s t h e n a i v e c o d e r s agreed w i t h t h e e x p e r t s t o a h i g h d e gree, and t h e e x p e r t s a g r e e d amongst t h e m s e l v e s . I n f u t u r e r e s e a r c h t h e d i s t r i b u t i o n o f agreement w i l l be examined i n g r e a t e r d e t a i l a c r o s s a l l 129 v a r i a b l e s . U n f o r t u n a t e l y , however, t h e r e i s no way t o measure d i r e c t l y t h e i n t e r a c t i o n 41 o f viewer and message c h a r a c t e r i s t i c s and thus address t h i s i s s u e e m p i r i c a l l y . I d e a l l y , a content a n a l y s i s coding system s h o u l d have g l o b a l a p p l i c a t i o n . That i s , no matter who uses i t s i m i l a r r e s u l t s s h o u l d be obtained. T h i s was the case i n t h i s study. The n a i v e coders v a r i e d c o n s i d e r a b l y i n t h e i r e t h n i c and SES backgrounds, and on other demographic and t e l e v i s i o n use v a r i a b l e s . These v a r i a t i o n s were not, however, s y s t e m a t i c a l l y r e l a t e d t o the l e v e l of t h e i r agreement with the expert coders i n the r e g r e s s i o n a n l a y s e s . T h i s i n d i c a t e s t h a t the coding system a s s e s s e s the same messages when used by a heterogeneous group o f u n i v e r s i t y students. U n f o r t u n a t e l y , we cannot know the g e n e r a l i z a b i l i t y of the r e s u l t s beyond the sample of naive coders i n t h i s study. They were a f a i r l y s e l e c t group, t h i s i s , • u n i v e r s i t y s t u d e n t s . They were chosen because the expert coders were u n i v e r s i t y s t u d e n t s and i t seemed wise t o vary only one dimension a t a time, t h a t i s , t r a i n i n g w i t h the coding system. W i t h i n t h i s group, however, both the expert and n a i v e . coders were q u i t e d i v e r s e . T h e i r p a r e n t s ' occupations ranged from m i l l worker and chambermaid t o p h y s c i a n and lawyer. They ranged from Caucasians and o r i e n t a l s whose f a m i l i e s had been i n Canada f o r g e n e r a t i o n s t o people who had themselves grown up i n China and I n d i a and come here r e c e n t l y t o study. I t i s i m p o s s i b l e to know whether s i m i l a r c o n s i s t e n c i e s i n the r e s u l t s would be found i f a more r e p r e s e n t a t i v e sample of the a d u l t p o p u l a t i o n s e r v e d as n a i v e viewers. T h i s i s s u e w i l l be addressed i n 42 f u t u r e r e s e a r c h . I t a l s o would be p a r t i c u l a r l y i m p o r t a n t t o u n d e r s t a n d t h e d e v e l o p m e n t a l i m p l i c a t i o n s o f t h i s r e s e a r c h by e x t e n d i n g i t t o younger groups and o l d e r groups. Perhaps o f g r e a t e s t c o n c e r n i s t o d e t e r m i n e how s i m i l a r t h e r e s u l t s would be i f c h i l d r e n and t e e n s s e r v e d as n a i v e c o d e r s , as t h e y a r e t h e ones most l i k e l y t o be a f f e c t e d by t e l e v i s i o n v i e w i n g . I f t h e c o n t e n t a s s e s s e d by a d u l t c o d e r s i s t o be o f v a l u e , we must know how i t i s p e r c e i v e d by c h i l d r e n and t e e n s . I n c o n s i d e r i n g a l l o f t h e p r e d i c t i o n a n a l y s e s , i t i s i m p o r t a n t t o c o n s i d e r t h e p r o b a b i l i t y o f Type I e r r o r , t h a t i s , f i n d i n g s i g n i f i c a n t p r e d i c t i o n s by chance. The 10 p r e d i c t o r v a r i a b l e s were used t o p r e d i c t 22 i t e m s . T h i s means t h a t o v e r a l l t h e r e g r e s s i o n a n a l y s e s about 11 Type I e r r o r s would be e x p e c t e d t o o c c u r p u r e l y by chance, and 11 s i g n i f i c a n t p r e d i c t i o n s o c c u r r e d . Thus, a l l must be c o n s i d e r e d w i t h a s k e p t i c a l eye, and i t c o u l d w e l l be argued t h a t a l l s h o u l d be i g n o r e d . Some o f t h e r e s u l t s do have some b a s i s i n p r e v i o u s l i t e r a t u r e , however, so w i l l be mentioned b r i e f l y . As a measure o f t h e n a i v e c o d e r s ' f a m i l i a r i t y w i t h t h e show t h e y watched, t h e y were asked how o f t e n t h e y watched t h e program s e r i e s (SHFRQ). T h i s c o r r e l a t e d p o s i t i v e l y w i t h c o d i n g romance as t h e e x p e r t c o d e r s had done, but n e g a t i v e l y ( t h a t i s , d i f f e r e n t l y from t h e e x p e r t s ) w i t h t h e p r e s e n c e o f p h y s i c a l a g g r e s s i o n and t h e p o l i t i c a l p h i l o s o p h y o f t h e program. I t may be t h a t i n d i v i d u a l s more f a m i l i a r w i t h a show were more s e n s i t i z e d t o t h e r o m a n t i c c o n t e n t , b u t d e s e n s i t i z e d t o t h e p h y s i c a l a g g r e s s i o n . T h i s l a t t e r p o s s i b i l i t y i s s u p p o r t e d by o t h e r r e s e a r c h i n which e x p o s u r e t o t e l e v i s e d v i o l e n c e i n c r e a s e d t o l e r a n c e f o r f u r t h e r t e l e v i s e d v i o l e n c e ; t h e q u e s t i o n o f d e s e n s i t i z a t i o n t o r e a l l i f e v i o l e n c e remains open (see Comstock, 1980, f o r a r e v i e w ) . The f i n d i n g t h a t t h e f r e q u e n c y o f w a t c h i n g a show a l s o p r e d i c t e d whether o r n o t a n a i v e v i e w e r p e r c e i v e d some c o n t e n t c h a r a c t e r i s t i c s i n t h e same way as a t r a i n e d coder i s c o n s i s t e n t w i t h Himmelweit e t a l . 's (1978) f i n d i n g t h a t t h e more p o p u l a r a program was t h e more s i m i l a r l y p e o p l e r a t e d i t . The f o r m a t o f some q u e s t i o n s was n o t t h e same i n t h e UBCCS and i n t h e ACS. The changes were made because i n t h e UBCCS t h e s e q u e s t i o n s were i n a p o t e n t i a l l y d i f f i c u l t f ormat and t h e g o a l o f t h i s s t u d y was t o a s s e s s message p e r c e p t i o n s o f t r a i n e d v e r s u s u n t r a i n e d c o d e r s , n o t t o e v a l u a t e q u e s t i o n f o r m a t . Had agreement on reworded q u e s t i o n s been d i f f e r e n t from t h a t on q u e s t i o n s t h a t were n o t reworded, i n t e r p r e t a t i o n o f t h e r e s u l t s would have been more d i f f i c u l t . The f i n d i n g t h a t agreement was s i m i l a r and h i g h , d e s p i t e r e w o r d i n g , l e n d s s t r e n g t h t o t h e c o n c l u s i o n t h a t n a i v e c o d e r s agreed w i t h t h e e x p e r t s i n t h e i r p e r c e p t i o n s o f t h e TV programs i n t h i s s t u d y . A number o f i m p o r t a n t i s s u e s d i s c u s s e d i n t h e i n t r o d u c t i o n c o u l d not be a d d r e s s e d e m p i r i c a l l y i n t h i s s t u d y . The r e s u l t s were s u f f i c i e n t l y p r o m i s i n g , however, t h a t t h e s e i s s u e s s h o u l d be addr e s s e d i n f u t u r e r e s e a r c h . The model p r o p o s e d t o e x p l a i n t h a t c o n t e n t a n a l y s i s has t h r e e f a c e t s : m i c r o v e r s u s m a c r o a n a l y t i c , q u a n t i t a t i v e v e r s u s q u a l i t a t i v e , and a u d i e n c e v e r s u s e x p e r t c o d e r s . The UBCCS c o d i n g system was d e s i g n e d t o b r i d g e a l l o f t h e s e i s s u e s , but o n l y t h e l a s t one was a d d r e s s e d i n t h i s r e s e a r c h . The f i r s t two f a c e t s may be an i s s u e o f t h e o r e t i c a l o r o t h e r p r e f e r e n c e , but t h e i s s u e o f whether an a u d i e n c e p e r c e i v e s t h e same messages as e x p e r t c o d e r s i s c r i t i c a l t o t h e a n a l y s i s o f c o n t e n t . I n d i v i d u a l r e s e a r c h e r s may have r e a s o n s t o choose a m i c r o a n a l y t i c approach o v e r a m a c r o a n a l y t i c approach, b u t t h e whole e n t e r p r i s e would be s u s p e c t i f t h e i r r e s u l t s had n o t h i n g t o say t o an average v i e w i n g a u d i e n c e . A n o t h e r i s s u e t h a t needs t o be a d d r e s s e d i s t h e impact o f not e t a k i n g on t h e c o n s i s t e n c y o f t h e r e s u l t s . I f t h e g o a l i s t o a s s e s s t h e messages r e c e i v e d by average v i e w e r s , how b e s t can c o n t e n t a n a l y s i s c a p t u r e t h o s e messages? Note t a k i n g was o r i g i n a l l y i n c l u d e d i n t h e e x p e r t methodology because i t was b e l i e v e d n e c e s s a r y t o e s t a b l i s h good r e l i a b i l i t y . The n a i v e c o d e r s a l s o were i n s t r u c t e d t o t a k e a few n o t e s . As s t a t e d e a r l i e r , t h i s made t h e e x p e r i e n c e d i f f e r e n t from t h a t o f t h e average v i e w e r . I t i s not known how i m p o r t a n t a f a c t o r t h i s t u r n e d o u t t o be, b u t i t i s i n t e r e s t i n g t h a t whereas many o f t h e n a i v e c o d e r s t o o k v e r y d e t a i l e d n o t e s , many chose n ot t o t a k e n o t e s a t a l l . The concordance among n a i v e v i e w e r s and agreement w i t h t h e e x p e r t s was o b t a i n e d i n s p i t e o f t h i s d i s c r e p a n t a p p r o a c h so i t seems l i k e l y t h a t note t a k i n g p l a y e d o n l y a minor r o l e i n e s t a b l i s h i n g agreement between t h e e x p e r t s and t h e n a i v e c o d e r s . 45 Gunter (1988) argued s t r o n g l y a g a i n s t d o i n g t r a d i t i o n a l t e l e v i s i o n c o n t e n t a n a l y s i s . The problem, he c l a i m e d , i s t h a t v i e w e r s a c t i v e l y watch and i n t e r p r e t t h e c o n t e n t w i t h r e f e r e n c e t o t h e i r own unique h i s t o r i e s . D e s p i t e t h i s r e a l i t y , t h e r e s u l t s o f t h i s s t u d y a r e c o n s i s t e n t w i t h M o r l e y ' s (1980) c o n t e n t i o n t h a t t h e r e i s a good d e a l o f consensus about what i s p e r c e i v e d . I n t h i s r e s e a r c h , as i n any r e s e a r c h , what we can know and what we a r e r e a l l y i n t e r e s t e d i n a r e two d i f f e r e n t t h i n g s . What we want t o know i s how t e l e v i s i o n a f f e c t s t h e p o p u l a t i o n o f average v i e w e r s , who watch TV a t home w i t h many d i s t r a c t i o n s and, t y p i c a l l y , w i t h low "amount o f i n v e s t e d m e n t a l e f f o r t " (AIME; Salomon, 1983). What we have found out about i s how u n i v e r s i t y s t u d e n t s responded t o q u e s t i o n s when t h e y knew ahead o f t i m e t h a t t h e y would be w a t c h i n g a TV show and l a t e r asked q u e s t i o n s about i t , and watched i n a room t h a t resembled a s m a l l c o n f e r e n c e room more t h a n a l i v i n g room. O t h e r s ( e . g . , L u l l . , Hanson, & Marx, 1977) might argue t h a t i t i s b e t t e r t o ask p e o p l e o n l y t o g i v e t h e i r open-ended i m p r e s s i o n s o f t b e show, not t o prompt them w i t h s p e c i f i c q u e s t i o n s . Even t h i s approach, however, w i l l n o t r e v e a l what t h e average v i e w e r t a k e s aWay ov e r t h e l o n g e r term from t h e v i e w i n g e x p e r i e n c e . Because o f l i m i t a t i o n s o f r e s e a r c h methodology t h e r e a l q u e s t i o n may n e v e r be answered. I n s t e a d , we have t o be c o n t e n t w i t h s m a l l i n c r e m e n t a l s t e p s t h a t t a k e us c l o s e r t o our g o a l . The s m a l l s t e p t a k e n i n t h i s r e s e a r c h i s t h a t , g i v e n t h e same shows and q u e s t i o n s , t h e messages 4 6 taken away from p o p u l a r TV programs by expert and na i v e coders ar e more s i m i l a r than d i f f e r e n t . i 47 R e f e r e n c e s Bandura, A. (1977). S o c i a l l e a r n i n g t h e o r y . Englewood C l i f f s , NJ: P r e n t i c e - H a l l . B l i s h e n , B.R., C a r r o l l , W.K., & Moore, C. (1987). The 1981 s o c i o e c o n o m i c i n d e x f o r o c c u p a t i o n s i n Canada. Canadian  Review o f S o c i o l o g y and A n t h r o p o l o g y . 24., 465-488. Budd, R.W., Thorp, R.K., & Donohew, L. (1967). C o n t e n t  a n a l y s i s o f communication. Macmilan: New York. Comstock, G.A. (1980) . New emphases i n r e s e a r c h on t h e e f f e c t s o f t e l e v i s i o n and f i l m v i o l e n c e . I n E.L. Palmer & A. D o r r ( E d s . ) , C h i l d r e n and t h e f a c e s o f t e l e v i s i o n . New Y o r k : Academic P r e s s . Condry, J . , Bence, P., & S c h e i b e , C. (1987, A p r i l ) . The non- program c o n t e n t o f c h i l d r e n ' s t e l e v i s i o n . Paper p r e s e n t e d a t The S o c i e t y f o r R e s e a r c h i n C h i l d Development, B i e n n i a l meeting, B a l t i m o r e , MD. Cook, T.D., C u r t i n , T.R., Ettema, J.S., M i l l e r , P.V., & Van Camp, K. (1986, November) . T e l e v i s i o n i n t h e l i f e o f  s c h o o l s . Report p r e p a r e d f o r t h e U.S. O f f i c e o f E d u c a t i o n a l R e s e a r c h and Improvement C o n f e r e n c e on A s s e s s i n g T e l e v i s i o n ' s Impact on C h i l d r e n ' s E d u c a t i o n , Washington, DC. Cordua, G., McGraw, K. , & Drabman, R. (1979). D o c t o r o r n u r s e : C h i l d r e n ' s p e r c e p t i o n s o f s e x - t y p e d o c c u p a t i o n s . C h i l d Development. 50, 590-593. C e n t e r f o r R e s e a r c h on t h e I n f l u e n c e s o f TV on C h i l d r e n . (CRITC, 1983). C r i t i c program c a t e g o r i z a t i o n system  c o d i n g manual. A v a i l a b l e from CRITC, Department o f Human Development, U n i v e r s i t y o f Kansas, Lawrence, Kansas, U.S.A. 66045. D o r r , A. (1986). T e l e v i s i o n and c h i l d r e n : A s p e c i a l medium  f o r a s p e c i a l a u d i e n c e . The Sage CommText s e r i e s ( V o l . 14), B e v e r l y H i l l s , CA: B e v e r l y H i l l s . F e r g u s o n , G.A. (1976). S t a t i s t i c a l a n a l y s i s i n p s y c h o l o g y &  e d u c a t i o n ( 4 t h e d . ) . New York: M c G r a w - H i l l . F l e i s s , J . L . (1981). S t a t i s t i c a l methods f o r r a t e s and  p r o p o r t i o n s (2nd e d . ) . T o r o n t o : W i l e y , C h a p t e r 13. Gerbner, G. , H o l s t i , O.E., K r i p p e n d o r f , K. , P a i s l e y , W.J., & S t o n e , P . J . (1969). The a n a l y s i s o f communicaiton  c o n t e n t : Developments i n s c i e n t i f i c t h e o r i e s and computer  t e c h n i q u e s . New York: W i l e y & Sons. 48 Gunter, B. (1983). P e r s o n a l i t y and p e r c e p t i o n s o f h a r m f u l and h a r m l e s s TV v i o l e n c e . P e r s o n a l i t y and I n d i v i d u a l D i f f e r e n c e s , 4, 665-670. Gunter, B. (1988). The p e r c e p t i v e a u d i e n c e . I n J.A. Anderson ( E d . ) , Communication Yearbook V o l . 11. B e v e r l e y H i l l s , CA: Sage. Hays, W.L. (1981). S t a t i s t i c s ( 3 r d e d . ) . New York: McGraw-H i l l . Himmelweit, H.T., S w i f t , B. , & B i b e r i a n , M.J. (1978). The a u d i e n c e as c r i t i c : An approach t o . t h e s t u d y o f e n t e r t a i n m e n t . In. P. Tannenbaum ( E d . ) , E n t e r t a i n m e n t  f u n c t i o n s o f t e l e v i s i o n . New York: Erlbaum. H o i s t i , O.R. (1969). Content a n a l y s i s f o r t h e s o c i a l s c i e n c e s  and h u m a n i t i e s . Reading, MA: Addison-Wesley. Huesmann, R. (1985). P e r s o n a l communication t o T.M. W i l l i a m s . James, C.L. (1979). Agreement measurement and t h e judgement p r o c e s s . J o u r n a l o f Nervous and M e n t a l D i s e a s e , 167(6), 343-347. Je n s e n , K.B. (1987, May). The s o c i a l u s e s o f t e l e v i s i o n news: A q u a l i t a t i v e e m p i r i c a l s t u d y o f t h e r e c e p t i o n o f Danish  t e l e v i s i o n news. Paper p r e s e n t e d t o t h e 37th Annual C o n f e r e n c e o f t h e I n t e r n a t i o n a l Communication A s s o c i a t i o n , M o n t r e a l . K i r k , J . , & M i l l e r , M.L. (1986). R e l i a b i l i t y and v a l i d i t y i n  q u a l i t a t i v e r e s e a r c h . Sage U n i v e r s i t y Paper S e r i e s on Q u a l i t a t i v e R e s e a r c h Methods, ( V o l . 1 ) . B e v e r l y H i l l s , CA: Sage. K r i p p e n d o r f , K. (1980). Content a n a l y s i s : An i n t r o d u c t i o n t o  i t s methodology. The Sage CommText S e r i e s ( V o l . 5 ) , B e v e r l y H i l l s , CA: B e v e r l y H i l l s . L u l l , J.T., Hanson, C.A., & Marx, M.J. (1977). R e c o g n i t i o n o f female s t e r e o t y p e s i n TV c o m m e r c i a l s . J o u r n a l i s m  Q u a r t e r l y . 153-157. M o r l e y , D. (1980). The " N a t i o n w i d e " a u d i e n c e : S t r u c t u r e and  d e c o d i n g . B r i t i s h F i l m I n s t i t u t e (BFI) Monograph 11. London: B F I . Newcomb, H. (1978). A s s e s s i n g t h e v i o l e n c e p r o f i l e s o f Gerbner and G r o s s : A h u m a n i s t i c c r i t i q u e and s u g g e s t i o n . Communication R e s e a r c h . 5, 264-282. 49 N i e l s o n , N.A. (undated). Content a n a l y s i s and r e c e p t i o n a n a l y s i s : The case o f t e l e v i s i o n and t h e f a m i l y . Media R e s e a r c h Department, Denmark's R a d i a . Rosengren, K.E. (1981). Advances i n S c a n d i n a v i a n c o n t e n t a n a l y s i s : An i n t r o d u c t i o n . I n K.E. Rosengren ( E d . ) , Advances i n c o n t e n t a n a l y s i s . B e v e r l y H i l l s : Sage Annual Reviews o f Communication Research, V o l . 9, 9-19. Rosengren, K.E., Wenner, L.A., & Palmgren, P. (1985). Media  g r a t i f i c a t i o n s r e s e a r c h : C u r r e n t p e r s p e c t i v e s . B e v e r l y H i l l s , CA: Sage. Salomon, G. (1979). i n t e r a c t i o n o f media, c o g n i t i o n , and  l e a r n i n g . New York: J o s s e y - B a s s . Salomon, G. (1983). T e l e v i s i o n w a t c h i n g and m e n t a l e f f o r t : A s o c i a l p s y c h o l o g i c a l v i e w . I n J . B r y a n t & D.R. Anderson ( E d s . ) , C h i l d r e n ' s u n d e r s t a n d i n g o f t e l e v i s i o n . New York: Academic P r e s s . Schank, R . C , & A b e l s o n , R.P. (1977). S c r i p t s , p l a n s , g o a l s .  and u n d e r s t a n d i n g . H i l l s d a l e , NJ: Erlbaum. S e p s t r u p , P. (1981). M e t h o d o l o g i c a l developments i n c o n t e n t a n a l y s i s . I n K.E. Rosengren ( E d . ) , Advances i n c o n t e n t  a n a l y s i s . B e v e r l y H i l l s , CA: Sage Annual Reviews o f Communication R e s e a r c h , V o l . 9, 13 3-158. Sherman, B.L., & Dominick, J.R. (1984, November). Guns, sex, & r o c k & r o l l : A c o n t e n t a n a l y s i s o f music t e l e v i s i o n . Paper p r e s e n t e d a t t h e Speech Communicaiton A s s o c i a t i o n A n n u a l G e n e r a l M e e t i n g , C h i c a g o . S t e e v e s , H.L., & Smith, M.C. (1987). C l a s s and gender i n p r i m e - t i m e t e l e v i s i o n e n t e r t a i n m e n t : O b s e r v a t i o n s from a s o c i a l i s t f e m i n i s t p e r s p e c t i v e . J o u r n a l o f Communication  I n q u i r y . 11. S t o n e , P . J . , Dunphy, D.C, Smith, M.S., & O g i l v i e , S.M. (1966). The g e n e r a l i n q u i r e r : A computer approach t o  c o n t e n t a n a l y s i s . Cambridge, MA: M.I.T. P r e s s . S u r l i n , S.H., Romanow, W.C. , & S o d e r l u n d , W.C (1987, May). TV network news: A Canadian-American comparison. Paper p r e s e n t e d a t t h e m e e t i n g o f t h e Canadian Communication A s s o c i a t i o n , M o n t r e a l . T a t e , E.D. (1977). V i e w e r s ' p e r c e p t i o n s o f s e l e c t e d t e l e v i s i o n programs. I n R e p o r t o f The R o y a l Commission  on V i o l e n c e i n t h e Communications I n d u s t r y , (RCVCI) V o l . 6, 283-401, T o r o n t o : RCVCI. 50 T a y l o r , S.E., & C r o c k e r , J . (1981). Schematic bases o f s o c i a l i n f o r m a t i o n p r o c e s s i n g . I n E.T. H i g g i n s , C P . Herman, & M.P. Zanna ( E d s . ) , S o c i a l c o g n i t i o n : The O n t a r i o  symposium i n p e r s o n a l i t y and s o c i a l p s y c h o l o g y . ( V o l . 1, pp. 89-135). H i l l s d a l e , NJ: Erlbaum. W i l l i a m s , T.M. ( E d . ) . (1986). The impact o f t e l e v i s i o n : A  n a t u r a l experiment i n t h r e e communities. New York: Academic P r e s s . W i l l i a m s , T.M., & Boyes, M.C (1986). T e l e v i s i o n v i e w i n g p a t t e r n s and use o f o t h e r media. I n T.M. W i l l i a m s ( E d . ) , The impact o f t e l e v i s i o n : A n a t u r a l e x p e r i m e n t i n t h r e e  communities. New York: Academic P r e s s . W i l l i a m s , T.M., P h i l l i p s , S., & T r a v i s , L. (1985). The U n i v e r s i t y o f B r i t i s h Columbia TV c o n t e n t c o d i n g system (UBCCS) and manual. U n p u b l i s h e d m a n u s c r i p t s a v a i l a b l e from T.M. W i l l i a m s , Department o f P s y c h o l o g y , U n i v e r s i t y o f B r i t i s h Columbia, 2136 West M a l l , V ancouver, B.C., Canada, V6T 1Y7. W i l l i a m s , T.M., P h i l l i p s , S., T r a v i s , L. , & Wotherspoon, D. (1988, s u b m i t t e d f o r p u b l i c a t i o n ) P o r t r a y a l o f Canada, t h e U.S.A., and o t h e r c o u n t r i e s on Cana d i a n v e r s u s U.S. t e l e v i s i o n . Canadian J o u r n a l o f Communication. W i l l i a m s , T.M., Young, R.A., P a r k e r , S., Wotherspoon, D., C u r r o r , S., & W i n t e r , A. (1987, O c t o b e r ) . Messages about e d u c a t i o n i n TV programs p o p u l a r w i t h c h i l d r e n and t e e n a g e r s . F i n a l r e p o r t s u b m i t t e d t o U.S. O f f i c e o f E d u c a t i o n . W i l l i a m s , T.M., Zabrack, M.L., & J o y , L.A. (1977). A c o n t e n t a n a l y s i s o f e n t e r t a i n m e n t t e l e v i s i o n programming. I n . R e p o r t o f The R o y a l Commision on V i o l e n c e i n t h e  Communications I n d u s t r y , (RCVCI) V o l . 3, 1-155, T o r o n t o : RCVCI. W i l l i a m s , T.M., Zabrack, ''M.L., & J o y , L.A. (1982). The p o r t r a y a l o f a g g r e s s i o n on N o r t h A m e r i c a n t e l e v i s i o n . J o u r n a l o f A p p l i e d S o c i a l P s y c h o l o g y . 12, 360-380. T a b l e 1 V a r i a b l e names and t h e i r l o c a t i o n i n t h e ACS V a r i a b l e Name S e c t i o n Page Q u e s t i o n NAETH 2 2 6 a SXPROM 3 5 12 PFOCMX 3 5 14 a NTRFEM 3 6 14 d PMOCMX 3 5 15 a NTRMAL 3 7 15 d SXOB 3 9 20 HUMSX 3 9 22 ROMNCE 3 9 24 PAG 4 12 30 a VAG 4 14 31 a DNGWRLD 4 15 32 GUNS 4 15 33 a PROB 5 .15 3 6 a POLITIC 5 16 37 a RICHFAM 5 16 38 TEENREP 5 16 39 LFTRT 6 18 42 NAT 6 18 43 LAWAUT 6 18 44 PAUTSX 6 18 45 a PAUTETH 6 18 45 b 52 T a b l e 2 Frequency o f f a i l u r e t o answer q u e s t i o n s V a r i a b l e Name Frequency % o f 240 of m i s s i n g NAETH 9 3.8 SXPROM 4 1.7 PFOCMX 2 .8 NTRFEM 6 2.5 PMOCMX 3 1.3 NTRMAL 5 2.1 SXOB 2 .8 HUMSX 1 .4 ROMNCE 1 .4 PAG 2 .8 VAG 1 .4 DNGWRLD 1 .4 GUNS 1 .4 PROB 1 .4 POLITIC 5 2.1 RICHFAM 4 1.7 TEENREP 0 0.0 LFTRT 7 2.9 NAT 0 0.0 LAWAUT 5 2.1 PAUTSX 1 .4 PAUTETH 5 2.1 53 T a b l e 3 S u b j e c t demographic v a r i a b l e s used i n r e g r e s s i o n a n a l y s e s ,  t h e i r l o c a t i o n i n t h e ACS, and how t h e v were r e c o d e d V a r i a b l e Name S e c t i o n Page Q u e s t i o n R e c o d i n g TVMEAN 7 20 50 Mean f o r a l l t i m e s SHFRQ 7 20 54 S c a l e r e v e r s e d SHKLIKE 7 21 55 AGE 7 21 56 SX 7 21 57 1=M 2=F YEAR 7 21 59 STUDY 7 21 60 T o t a l f o r a l l t i m e s ETHNIC 7 21 61-65 * OCGOAL 7 21 67 ** PAROCC 7 21 68,70 *** * l=White N o r t h A m e r i c a n ; 2 = V i s i b l e e t h n i c m i n o r i t y , b u t r a i s e d i n N o r t h A m e r i c a , e.g., Canadian b o r n o f Japanese d e s c e n t ; 3=Not v i s i b l e e t h n i c m i n o r i t y , and n o t r a i s e d i n N o r t h A m e r i c a , e.g., F i n n i s h ; 4 = V i s i b l e e t h n i c m i n o r i t y and not r a i s e d i n N o r t h A m e r i c a . ** S o c i o e c o n o m i c s t a t u s s c o r e ( B l i s h e n , C a r r o l l , & Moore, 1987) . *** Same as **, b u t h i g h e r o f e i t h e r mother o f f a t h e r used. 54 T a b l e 4. Program sample C a t e g o r y o f Program News 1. CTV: N a t i o n a l News 3. F i f t h E s t a t e N o n - F i c t i o n E n t e r t a i n m e n t 5. E n t e r t a i n m e n t T o n i g h t 7. Wheel o f F o r t u n e Documentary 9. W o r l d o f S u r v i v a l 11. N a t u r e o f T h i n g s S i t u a t i o n Comedy 13. Cosby Show 15. N i g h t C o u r t C r i m e - D e t e c t i v e 17. M i a m i - V i c e 19. H i t c h c o c k P r e s e n t s O t h e r Drama 21. D a l l a s 23. Highway t o Heaven Program 2. CBC: The N a t i o n a l 4. 20/20 6. Newlywed Game 8. L i f e s t y l e s o f R i c h & Fam. 10. Gzowzki & Co. 12. F r o n t Page C h a l l e n g e 14. F a m i l y T i e s 16. Golden G i r l s 18. H i l l S t r e e t B l u e s 20. Murder She Wrote 22. Dynasty 24. Love Boat 55 T a b l e 5 Agreement among n a i v e c o d e r s u s i n g K e n d a l l ' s W Mean W * Mean X 2 * d f E< n o f s u b j e c t s A l l v a r i a b l e s : f l u c t u a t i n g number o f s u b j e c t s News .63945 66.5166 21 .0001 10/20 Other non- .72653 40.26538 21 .005 16/100 f i c t i o n F i c t i o n .69946 67-29178 21 .0001 36/120 A l l s u b j e c t s : f l u c t u a t i n g number o f v a r i a b l e s n o f v a r i a b l e s News .60800 97.15520 16 .0001 17 Ot h e r non- .62122 77.98239 12.6 .0001 13.6 . f i c t i o n F i c t i o n .64798 102.04043 15.7 .0001 16.6 * V a l u e s a r e c a l c u l a t e d p e r show w i t h i n a c a t e g o r y and t h e n averaged. T a b l e 6 D i s t r i b u t i o n o f T s c o r e s 1 2 3 4 5 M i n -1.0 . 14 -.50 .01 .27 Max 1.0 .89 1. 00 .80 .95 Range 2 . 0 .75 1. 50 .79 . 68 Mean .594 .531 .556 .497 .546 S t d Dev .307 .172 .438 . 225 .187 Median .660 .530 .785 .545 . .570 Mode .670 .520 .800 .540 .580 1. A l l 129 v a r i a b l e s i n t h e ACS, and a l l 24 shows. 2. 22 s e l e c t e d v a r i a b l e s , and a l l 24 shows. 3. 22 s e l e c t e d v a r i a b l e s , 2 "hard" news shows. 4. 22 s e l e c t e d v a r i a b l e s , 10 o t h e r n o n - f i c t i o n shows. 5. 22 s e l e c t e d v a r i a b l e s , 12 f i c t i o n shows T a b l e 7 Comparison o f T s c o r e s w i t h r e l i a b i l i t y s c o r e s f o r e x p e r t s * V a r i a b l e Name %N ** T s c o r e R e l i a b i l i t y *** PMOCMX 85 .89 K=1.0 PAUTETH 79 .74 %=.90 GUNS 99 .71 RE=.727 PROB 100 -69 %=.904 SXPROM 98 .69 RE=1.0 ROMNCE 100 .64 K=.712 NAETH 96 .59 RE=.786 PAG 93 .59 K=. 810 TEENREP 100 .58 RE=.673 HUMSX 100 .57 K=.705 SXOB 99 .54 RE=.770 VAG 93 .52 RE=.614 DNGWRLD 100 .52 K=.650 NTRMAL 85 .51 RE=.684 RICHFAM 98 .50 K=.487 LAWAUT 98 -49 K=.441 NTRFEM 92 .48 RE=.697 POLITIC 83 .42 K=.598 PAUTSX 88 .35 %=.77 **** LFTRT 97 .27 K=.435 PFOCMX 66 .26 K=.618 NAT 76 .14 %=.684 58 ( T a b l e 7 c o n t i n u e d ) * The c o r r e l a t i o n between T s c o r e s and r e l i a b i l i t y i s .65, p<.001. ** %n r e f e r s t o t h e p e r c e n t a g e o f s u b j e c t s , a f t e r r e c o d e s , whose d a t a were used t o c a l c u l a t e t h e T s c o r e . *** I n r e p o r t i n g r e l i a b i l i t y 1 o f 3 s c o r e s has been r e p o r t e d , Kappa ( F l e i s s , 1981), Maxwell's RE (James, 1979), o r p e r c e n t agreement. F o r a complete e x p l a n a t i o n o f c h o o s i n g one over a n o t h e r see W i l l i a m s , Young, P a r k e r , Wotherspoon, C u r r o r , and W i n t e r , 1987. **** Only t h e c e n t r e t h r e e p o i n t s on LFTRT form an a c t u a l continuum. U s i n g t h e s e t h r e e p o i n t s o n l y , t h e T s c o r e i n c r e a s e s t o .53, b u t t h e %n drops t o 25. T a b l e 8 Forward s t e p w i s e r e g r e s s i o n a n a l y s e s Independent Dependent F d f E< R Be t a SHLIKE TEENREP 7.11 1, 76 .05 .0856 .2925 SHLIKE NTRFEM 3.93 1, 188 .05 ' .0205 . 1431 SX PROB 5.77 1, 188 . 01 .0298 . 1725 ETHNIC PROB 5.52 2, 187 .005 .0557 -.1612 SHFRQ ROMNCE 6.77 1, 196 .01 .0333 . 1827 SHFRQ PAG 6.35 1, 196 . 01 .0314 -.1772 SHFRQ LFTRT 6.25 1, 196 . 01 . 0309 -.1861 YEAR LFTRT 5. 18 2, 195 .01 .0505 -.1403 OCGOAL SXPROM 5.32 1. 196 . 05 . 0264 .2104 STUDY SXPROM 6.78 2, 195 . 001 .0650 -.2021 60 F i g u r e C a p t i o n s F i g u r e 1. Three d i m e n s i o n a l box d e p i c t i n g d i f f e r i n g approaches t o t h e a n a l y s i s o f c o n t e n t . F i g u r e 2. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r a l l 129 v a r i a b l e s ( a f t e r r e c o d i n g ) i n t h e ACS and a l l 24 shows coded by t h e n a i v e c o d e r s . F i g u r e 3. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper and a l l 24 shows coded by t h e n a i v e c o d e r s . F i g u r e 4. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper f o r t h e 2 h a r d news shows o n l y . F i g u r e 5. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper f o r t h e 10 o t h e r n o n - f i c t i o n shows o n l y . F i g u r e 6. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper f o r t h e 12 f i c t i o n shows o n l y . 61 Figure 1. Audience Research F R E Q U E N C Y 30-20-10. •1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -02-0.1 0 0.1 02 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 T S C O R E F i g u r e 2. F R E Q U E N C Y 10 8 -6 -2 . •1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -0.2-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 T S C O R E Figure 3. F R E Q U E N C Y 8 1 6 i 4H -1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -02-0.1 0 0.1 02 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 T S C O R E F i g u r e 4, 65 F R E Q U E N C Y 6 i 4H -1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -0.2-0.1 0 0.1 02 0.3.0.4 05 0.6 0.7 0.8 0.9 1.0 T S C O R E Figure 5. F R E Q U E N C Y 8 n 61 4H 21 -1.0 -0.9 -0.8 -0.7 -0.6 -05 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 05 0.6 0.7 0.8 0.9 1.0 . T S C O R E Figure 6. 67 Appendix A 69 Overview of your involvement i n t h i s research • j .. 1. Familiarize yourself with the research by reading the instructions and asking any questions. 2. Watch the show. 3. Answer the questionnaire. Instructions: ••.=:!.•;•••• The next page provides space for you to make some notes while you watch the show. These notes are intended to serve as memory cues while you are f i l l i n g out the questions. In the section labelled " B i t or Segment topic" you should make brief notes about the main events of the show as they occur. These notes are for your own use only. i n the next two sections-you should keep track of the prominent or main female and male characters i n the story as they appear.;.. The prominent characters are defined as the characters that are necessary lbo t e l l the story. When they f i r s t appear you may not know their name, so use some other cue (e.g., green dress) and add the name la t e r . If -you aren't sure i f they are prominent, make a note and i f they aren't, then cross them out l a t e r . I f any of the prominent characters are Horth Americans and members of ethnic minorities put a * beside their names. Again, these notes are for you to use later as memory cues. Do not read any of the rest of the questions u n t i l you have watched the complete show. You w i l l watch the show once only. We w i l l fast forward through the commercials. You w i l l not be allowed to review the show after you have finished watching i t . We want you to answer the questions on the basis of the information i n the show you watch, MOT on what you know about the show i n general. Remember, i t i s your general impression of the show and i t s content that we are • interested i n . Please try to watch much as you would in your own home. We hope you enjoy the show. AUDIENCE CODING SHEET General Information Program ID ' Program T i t l e Hotes taken during program: B i t or Segment topic  1. _ 2. _ 3-1 4. _ 5. _ C._ 7._ 8_ 9._ 10. 11. 12. 13. IA. 15. 10. 17. 18. 19. 20. Female prominent characters i n order of appearance (*ethnics!) 1. 5. : 9. 13. 2. 6. • 10 14. 3. 7.__ 11. 15. 4. 8._ . 12. 1G. Hale prominent characters. . 1. " 5._ 9. 13. 2. • 0. \ 10 _14. 3. . 7. 11, 15. 4. 8. '• • . ' 12. 16. Part 2: Horth American Ethnic Minorities Mote: The following questions refer only to North Americans who are members of an ethnc minority group. 0. (a) Were any North Americans portrayed who were ethnic minorities? 1. no 2. yes (b) Tho prominent North American characters i n the program were: 1 2 3 A 5 6 a l l mainstream mostly mainstream mixed mostly ethnics a l l not non--ethnics non-ethnics but but some ethnics a p p l i -some important important cable ethnics mainstream (NA) non-ethnics (c) The backfiround Horth American characters i n the program were: 1 2 3 •'i 5 0 a l l mainstream mostly mainstream mixed mostly ethnic9 a l l N/A non--ethnics non-ethnics but but some ethnics some important important . ethnics mainstream non-ethnics. (d) Wore there any verbal or vicuai jokes about North American ethnic minorities? 1. no 2 . yes (e) Was there any clear evidence (portrayal or reference) of racism or prejudice? 1. no 2. yes • If yes, specify your reasons . (f) If yes, what's the bottom l i n e message of this program about racism or prejudice? 1. acceptable (e.g., l e t pass, not contradicted) . 2. unacceptable (e.g., some contradictory comeback) 72 - 3 In the following questions, please evaluate the members of ethnic minorities that you *'d on the f i r s t page. For each minority group three w i l l be 3 different evaluations. 7. Name the ethnic group Male Female Doth (a) How well do they function i n mainstream North American society? 1 2 3 function problems functioning problems functioning adequately duo to ethnicity HOT due to ethni c i t y (b) How strongly did they identify with their own ethnicity? . 1 2 3 no evidence of some ethnic strongly identify ethnic identity traces with their ethnicity (c) What i s your.final overall impression of the people represented i n the show from this ethnic group? 1 2 - 3 positive overall neutral negative overall - impression impression 8.. Name the ethnic group Male Female Both (a) How well do they function i n mainstream North American society? 1 2 . 3 function problems functioning problems functioning adequately • due to ethnicity HOT due to ethnicity (b) How strongly did they identify with their own ethnicity? 1 2 3 no evidence of some ethnic strongly identify ethnic identity traces with their ethnicity (c) What is"your f i n a l overall impression of the people represented i n the show from this ethnic group? 1 2 J positive overall neutral negative overall impression impression ., 9. . Name the ethnic group • _. Male Female Both (a) How well do they function i n mainstream North American society? 1 2 ; 3 function problems functioning problems functioning adequately. due to ethnicity NOT due to ethnicity 73 - 4 -(b) How strongly did they identify with their own ethnicity? 1 2 3 no evidence of some ethnic strongly identify ethnic identity traces with their ethnicity (c) What i s your f i n a l overall impression of the people represented i n the show from this ethnic group? 1 2 3 positive overall neutral negative overall impression impression 10. Hame the ethnic group Male Female Doth_ (a) How well do they function i n r.ainstrcam North American society? 1 2 : 3 function problems functioning problems functioning adequately due to ethnicity NOT due to ethnicity (b) How strongly did they identify with their own ethnicity? 1 2 3 no evidence of some ethnic strongly identify ethnic identity traces with their ethnicity what i s your f i n a l overall impression of the people represented i n the show from this ethnic, group? (c) 1 2 3 positive overall neutral negative overall impression impression 11. Same the ethnic group Male Female ' Doth_ (a) How well do they function in mainstream North American society? 1 . 2 : 3_ function problems functioning problems functioning adequately due to ethnicity NCT due to ethnicity (b) How strongly did they identify with thoir own ethnicity? 1 2 3 no evidence of some ethnic strongly identify ethnic identity traces with their ethnicity (c) What is your f i n a l overall impressipn of the people represented i n the show from this ethnic group? positive overall impression neutral negative overall impression 74 - 5 Part 3: The Sexes 12. Were the prominent characters i n the program (those you would need to t e l l the story) a l l mostly male but male some important females even mix of females and males mostly female but a l l some important females r.ales 13. Were the background characters in the program? a l l mostly male but male some important females even mix of females and males mostly female but a l l some important females males 14. (a) The prominent adult females (relative to other females in this: program) were: predominantly i n tr a d i t i o n a l l y female occupations (include homemaker) equal mix of tr a d i t i o n a l l y and nontradi -t i o n a l l y female occupations predominantly i n nontraditionally female occupations N/A or not information re: occupation to decide (b) For the prominent adult females, which types of a c t i v i t i e s were emphasized (airtime focus) i n the program? (Check more than'one i f necessary for different individuals; use your l i s t on front page to remember.) major focus on occupation major focus on home/family role major focus on social l i f e equal focus on occupation and home/family role equal focus.on home/family role and s o c i a l l i f e equal focus on occupation and social l i f e equal focus on occupation, home/family role, and so c i a l role (c) The background adujt females ( r e l a t i v e to other more prominent females i n this program) were: predominantly i n tr a d i t i o n a l l y , female occupations (include homemaker) equal mix of tr a d i t i o n a l l y and nontradi-tio n a l l y female occupations predominantly i n nontraditionally female occupations N/A or not information re: occupation to decide 75 6 -(d) Were there any adult females who behaved i n a nontraditional way (e.g., strong i n an emergency, not dependent on males tor guidance, assertively saying what's on her nind, etc.)? Instructions: When answering questions of this sort attempt always to use the extremes of the scaie, i.e.. 1 or 5. Avoid ucing 2 and 4 i f at a l l possible. These questions do not refer to amount or frequency of behavior, only whether or not i t was portrayed. d e f i n i t e l y no nontraditional female behavior not sure can't decide nontraditional female behavior de f i n i t e l y present Specify your. r£asons_ (e) If you answered (45 or (5.) to the above question ' (non-traditional female behavior i s present) was i t portrayed ac: 1. a) Serious b) jol:t c) both 2. a) positive (e.g., succeccsful/good/rewarded/comp<stent) b) negative (e.g., uncuccescful/bad/punished/incompetent) c) both (f) Were there any adult females who act?d i n a t r a d i t i o n a l way (e.g., dependent on a male for guidance, f a l l s apart in c r i s i s , non-assertive, etc.)? d e f i n i t e l y no tr a d i t i o n a l female behavior Specify your reasons^ not sure can't decide tradi t i o n a l female behavior definitely present (g) If you answered 4 or 5 to tho above question (traditional female behavior i s present) was i t portrayed as: 1. a) Serious b) joke c) both 2. a) positive (e.g., successsful/good/rowarded/competent) b) negative (e.g., unsuccessful/bad/punished/incompetent) c) both (a) The prominent adult males (relative-, to other Mies i n this program) were: predominantly i n tr a d i t i o n a l l y male occupations equal mix of tr a d i t i o n a l l y and nontradi -t i o n a l l y male occupations predominantly i n nontraditionally male occupations (include hoii.emaker) H/A or not enough information re: occupation to decide (b) For the prominent adult males, which types of a c t i v i t i e s were emphasized (airtime focus) in the program? (Check more than one i f necessary for different individuals) major focus on occupation major focus on so c i a l l i f e major focus on home/family role equal focus on occupation and home/family role equal focus on home/family role and social l i f e equal focus on occupation and social l i f e equal focus on occupation, home/family role, and so c i a l role (c) The background adult males (relative to other more prominent males i n this program) were: 1  predominantly i n t r a d i t i o n a l l y male occupations 2 equal mix of t r a d i t i o n a l l y and nontradi-t i o n a l l y male occupations J predominantly i n nontraditionally male occupations (include homemaker) A H/A or not enough information re: occupation to decide (d) Were there any adult males who behaved i n a nontraditional way (e.g., shown crying, caring for children, doing domestic chores, sensitive to the needs of others)? 1 2__ 3 A 5 de f i n i t e l y no not sure nontraditional male nontraditional can't decide behavior d e f i n i t e l y male behavior present Specify your reasons (e) If you answered (A) or (5) to the above question (non-traditional male behavior i s present) was i t portrayed as: 1. a) Serious b) joke c) both 2. a) positive (e.g., successsful/good/rewarded/competent) b) negative (e.g., unsuccessful/bad/punished/incompetent) c) both (f) Were there any adult males who acted i n a t r a d i t i o n a l way (e.g., s k i r t chaser, tough/macho, c a l l s the shots, l i f e revolves around job and s e l f ) ? . d e f i n i t e l y no not sure . . t r a d i t i o n a l male t r a d i t i o n a l can't decide behavior d e f i n i t e l y male behavior present - 8 (g) If you answered (4) or (5) tc the above question (tr a d i t i o n a l male behavior i s present) was i t portrayed as: 1. a) Serious . b) joke c; both 2. a) positive (e.g., successsful/good/rewardcd/competent) b) negative (e.g., unsuceessful/bad/punished/incompetent) c) both 16. Were there any messages indicating females are. subordinate to or worth less than males; e.g., parents disappointed that i t was a g i r l baby; any mention of men trading or owning women; women taking a back seat to and/or being obedient to men; parents preferring a sen over a daughter (because he w i l l carry on the family name, business, etc.)? 1. no 2. yes It yes, specify what gave you this impression Was this portrayed as: • 1'. Acceptable 2. Unacceptable 17. Were there any messages indicating that .-.ales are subordinate to or worth less than females, e.g., parents disappointed that i t was a boy and they wanted a g i r l baby; parents favoring a daughter over a son? 1. no 2. yes If yes, specify what gave you this impression Was this portrayed as: 1. Acceptable 2. Unacceptable 18. (a) ' Did you notice any sexist comments/jokes/putdowns about females i n this program? (e.g., "'.soman driver" jokes, using the words "broad" or "dame", etc.) 1. no 2. yes If yes, specify your reasons (b) If yes, what's the bottom line message of this program re sexist messages about females? i . acceptable (e.g., comment 2. unacceptable (e.g., comeback l e t pass) to comment) 19. (a) Did you notice any sexist comments/jokes/putdowns about males in this program? (e.g., comments about men only having one thing on their mind, "di r t y old man",: "male chauvinist pig" comments, etc.) 1. no 2. yes If yes, specify your reasons • -(b) If yes, what's the bottom line message of this program re sexist messages about males? 1. acceptable (e.g., comment le t pass) 2. unacceptable (e.g., comeback to comment) 9 -78 20. Were-there any people shown or referred to as being sex objects (producer portrays someone solely or primarily i n a sexual manner)? 1. Ho 2. Yes 21. Was there evidence that a double standard exists for females and males about sexual behaviour? ( C i r c l e as many as apply) 1. no v 2. yes, evidence 3. evidence or A. joke about >• evidence consistent with arguments double re: double the double against the standard standard standard double standard Specify your reasons_ _^ _ 22. Was sex (portrayal or reference) a part of the program? 1 2 • 3 some sex sex i s a major focus 23. I f there were portrayals or references to any of the following, how would you rate them? 1. Contraception a) Serious b) joke c) both 2. Pregnancy a) Serious b) joke c) both 3. Sexually transmitted diseases (other than AIDS) a) Serious b) joke c) both A. AIDS a) Serious b) joke c) both 2A. Was romance a part of the program? 1 2 3 . not at a l l some romance romance i s a major focus - 10 -Please describe the following heterosexual relationships i f found in the show. 25. The couples married and l i v i n g together were (check a l l that apply): a) loving, caring hostile (verbal/psychological) cool, casual physically abusive exploitive b) child (prepuberty) middle aged (35-55) _. teenager (13 19) older (55 & over) young adult (20-35) c) no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual interest kiss/touch with clear sexual intentions . e x p l i c i t reference to sex between people in relationships 26. The couples in a committed leve relationship not l i v i n g together were (check a l l that apply): a) loving, caring hostile (verbal/psychological) cool, casual physically abusive exploitive b) child (prepuberty) middle aged (35-55) teenager (13-19) older (55 & over) young adult (20-35) c) no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual interest kiss/touch with clear sexual•intentions e x p l i c i t reference to sex between people in relationships 27. The couples who were friends, but not i n a love relationship were (check a l l that apply): . a) loving, caring hostile (verbal/psychological) cool, casual physically abusive exploitive b) child (prepuberty) middle aged (35-55) ; teenager (13-19) older (55 & over). young adult (20-35) c) . no sexual behavior inuendo kiss/touch with no sexual .intentions f l i r t i n g or showing sexual interest kiss/touch with clear sexual intentions e x p l i c i t reference to sex between people in relationships 80 - l l -28. The couples who were acquaintances were (check a l l that apply): a) , loving, caring cool, casual exploitive b) c h i l d (prepuberty) teenager (13-19) young adult (20-35) hos t i l e (verbal/psychological) physically abusive middle aged (35-55) older (55 & over) _ c) no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual interest kiss/touch with clear sexual intentions e x p l i c i t reference to sex between people i n relationships 29. The couples who were strangers were (check a l l that apply): a) loving, caring cool, casual exploitive b) c h i l d (prepuberty) teenager (13-19) young adult (20-35) h o s t i l e (verbal/psychological) physically abusive middle aged (35-55) older (55 & over) c) no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual interest . kiss/touch with clear sexual intentions e x p l i c i t reference to sex between people i n relationships 81 - 12 Part A: Aggression In the following questions, an aggressive act i s one which i s intentional. interpersonal, and (unless otherwise stated) v i s u a l l y portrayed. This includes a n t i s o c i a l acts with the potential to do harm but from which the victim escapes uninjured. I t does not include instances cf accidental injury. On the scales for aggressive a c t i v i t y , avoid using numbers 2 and 4 i f at a l l possible; use only i f you cannot possibly assign a 1 er 5. These questions do not relate to amount of aggression, only whether i t occurred. 30. (a) Was there any physical aggression (by anyone) shown in this program? 1 2 3_ _4 S d e f i n i t e l y can't decide physical no aggression unsure aggression shown definitely present (b) Was there any physical aggression by females shown i n this program? 1 2_ 3 5 d e f i n i t e l y can't decide » physical no aggression unsure • aggression by females by females definitely present (c) Was there any physical aggression against females shown i n this program. d e f i n i t e l y can't decide physical no aggression unsure aggression against against females females definitely present (d) Was there any physical aggression by males shown in this program? 1 2 _3 4 5 d e f i n i t e l y can't decide physical no aggression unsure aggression by males by males definitely present 82 13 -(o) Was there any physical aggression against males shown i n this program? 1 2 __3 J \ 5 de f i n i t e l y can't, decide physical no aggression unsure aggression against males against males de f i n i t e l y present (f) Was there any reference made to physical aggression i n this program? 1 2 : 3 4 5 d e f i n i t e l y no can't decide d e f i n i t e l y reference to unsure had reference physical to physical aggression aggression (g) Was there any violence shown in the program ( i . e . , extreme physical aggression that i s potentially f a t a l ) ? d e f i n i t e l y can't decide d e f i n i t e l y nonviolent unjure violent (h) Was there any reference to violence i n this program?' 1 2 3 4 5 de f i n i t e l y no can't decide d e f i n i t e l y references unsure had reference to violence to violence ( i ) What i s the bottom li n e message of t h i s program re the acceptability of physical aggression as a method'of c o n f l i c t resolution? (Check more than 1 i f more than 1 message was perceived.) physical aggression i s clearly acceptable . can't decide, unsure physical aggression i s clear l y unacceptable N/A or not enough information, to. code ^ Specify : ; 83 - 14 -(j) What i s the bottom li n e message of this program re the successfulness of physical aggression as a method of c o n f l i c t resolution in the short term rather than long term sense? (Check more than 1 i f more than 1 message was perceived.) physical aggression i s clearly successful (e.g., short term goals were achieved through physical aggression can't decide, unsure physical aggression i s clearly not successful (e.g., short term goals were not achieved) H/A or not enough information to code 31. (a) Is there any verbal/psychological aggression i n the program (e.g., . y e l l i n g , verbal' insults)? 1 2 3 4 _ d e f i n i t e l y not at can't decide d e f i n i t e l y a l l verbally/ verbally/ psychologically psychologically aggressive aggressive Specify (b) What i s the bottom line message of this program re the acceptabilty of verbal or psychological aggression as a method of c o n f l i c t resolution? (Check more than one i f more than one message was perceived) verbal/psychological aggression i s clearly acceptable can't decide verbal/psychological aggression i s clearly not acceptable H/A or not enough information to code Specify . (c) What i s the bottom li n e message of this program re the successfulness of verbal/psychological aggression as a method of c o n f l i c t resolution? (check more than I i f more than 1 message was perceived). verbal/psychological aggression i s clearly successful (e.g., short term goals were achieved through this behavior can't decide, unsure verbal/psychological aggression i s clearly successful (e.g., short term goals were.not achieved through this behavior H/A or not enough information to code \ Specify - 15 -32. Does the program give the impression that the world i s a dangerous place (e.g., because of war, physical aggression of others, natural disasters, f a t a l t r a f f i c accidents etc*)? program gives no some people might impression i s c l e a r l y impression that the got the impression given that the world world i s a dangerous that the world i s i s a dangerous place place a dangerous place Specify_ 33. (a) Were any real guns shown in'the program? 1. No, ._ 2. Yes (b) Who had them? (Check a l l that apply) M i l i t a r y Police Dad c i t i z e n Good c i t i z e n Other authority If other authority specify who (c) How were the guns used? (Check a l l that apply) some or a l l shown but not used aggression_ defense of self/others intimidation/coercion_ defense of property recreation 34. Was there any portrayal of or reference to death of humans? 1. no 2. yes Part 5. Issues. Controversies, and Dilemmas 30. Does this program present evidence that problems exist? For example, minor problems would be those that are r e l a t i v e l y short term such as family disagreements. Major problems are those that have long tens impact, such as alcoholism or divorce, or those that have an impact on a large number of people, such as p o l l u t i o n or corruption i n p o l i t i c s . C i r c l e a l l that apply. (a) 1 2 3 Ho problems Minor problems Major problems Specify . ; \ •__ ; • If minor (2), were the problems: (b) __1 central to the plot 2 incidental to the plot (c) 1 portrayed as serious 2 portrayed as funny 85 16 If major (3), were the problems: (d) 1 2 (e) central to incidental portrayed portrayed the plot to the plot as serious as funny (f) The take-home message from this program regarding issues and controversies i s that they are usually: (check one) 1. clear-cut ( r e l a - 2. not clear-cut 3. not clear-cut 4. N/A t i v e l y black and (shades of grey, (shades of grey, white) and the and some answers and there are no right answers are better than right or better are clear others) answers) 37. (a) Were there e x p l i c i t p o l i t i c a l comments or references i n the program? ( c i r c l e highest applicable number) 1 2 3 4 5  d e f i n i t e l y can't decide definitely not unsure yes (b) I f 4 or 5, was the content r.ore balanced or more biased? 1. balanced 2. biased Specify b r i e f l y 38. (a) Was the show concerned with the r i c h and/or famous? 1 2 31 not at a l l somewhat predominantly, or completely 39. Were any of the following groups represented by at least one individual i n the program? (Check as appropriate) Children (under 12) Teenagers (13-10) Adults (19-50) Mature Adults (51-05) Seniors (over 65) ._ (e.g., r e t i r e d , grandparents) Male Female Handicapped/chronically i l l Very poor people Very r i c h people Communists Homosexuals-86 40. If thero were individuals belonging to any of the following groups, f i l l oi-t the following table indicating whether they were shown i n ways consistent with the positive or negative stereotypes described, or neither. Check the last box i f there was not enough information to code. Check as many as apply. Please note that the stereotypes described are merely examples of positive and negative stereotypes for each category--others would also be relevant and these need not be there. p o s i t i v e , e.g., Seniors wise, kind, grand parently, active negative, e.g., dependent, senile, burden to others, inactive, crabby not stereotyped not enough information to code or HA 1 2 3 4 pos i t i v e , e.g., negative, e.g., not not enough Children sweet, innocent, noisy, bratty, stereotyped information chanting messy . to code or HA posi t i v e , e.g., Teenagers reasonable, sensible, studying hard, involved i n many a c t i v i t i e s negative, e.g., rebellious, obsessed with sexuality/rock music/being " i n " , inconsiderate not stereotyped not enough information to code or HA Handicapped/ Chronically-i l l p o s i t i v e , e.g., showing exceptional bravery, strength, and perserverance i n the face of d i f f i c u l t y negative.e.g., helpless, passive, dependent burden not not enough stereotyped information to code or HA pos i t i v e , e.g., negative, e.g.. Very poor warm, caring, lazy, ignorant, generous, struggling untrustworthy . i n a d i f f i c u l t s i t u a t i o n not not enough stereotyped information to code or HA Very r i c h p o s i t i v e , e.g., philanthropists, i n t e l l i g e n t , using money uns e l f i s h l y negative, e.g., mate r i a l i s t i c power-hungry, ruthless not stereotyped not enough information to code or HA pos i t i v e , e.g., negative, e.g.. Communists well-intentioned, secretive, v i l l a i n s , c o l l e c t i v i s m sharing threat to free world social responsi-b i l i t y not stereotyped not enough Information to code or' HA 1 2 ; 3 4 pos i t i v e , e.g., negative, e.g., not . not enough Homosexuals sensitive, caring, limp wrist, l i s p , stereotyped information into the fine arts butch or other to code or stereotypes HA - 18 -Part 6. Clobal Impressions 41. How involving was the program? 1 2 : 3 not a l somewhat very a l l involving involving 42. How would you best describe the p o l i t i c a l philosophy of this program? 1 2 3 4 5 a p o l i t i c a l left-wing ( l i b e r a l , s o c i a l i s t ) centre right-wing (conservative, c a p i t a l i s t ) d e f i n i t e l y p o l i t i c a l but not i d e n t i f i a b l y l e f t or right 43. Considering the country of origi n of this program, how n a t i o n a l i s t i c was i t ? (e.g., "rah rah for our country"; our country or i t s citizens are especially wonderful in some way) de f i n i t e l y not n a t i o n a l i s t i c can't decide unsure de f i n i t e l y n a t i o n a l i s t i c For each of the following questions (44-49) check the statement that best describes the take-away message. 44. Laws/authority/the state: no inform. are always right may be wrong, but s t i l l must be obeyed (The Law is the Law") may be wrong but can be worked around or bent as needed may be wrong and the best avenue for change i s . working outside the system (revolution or vigilantes taking law into their own hands) 45. The powerful/authoritative/knowledgeable are: a l l mostly male but mixed mostly female but a l l N/A males some important some important females females Bales (b) .1 2 3 4 5 6 a l l main mostly main- mixed mostly ethnics a l l N/A stream stream non- but some ethnics non- ethnics but important ethnics some important mainstream ethnics non-ethnics 88 - 19 -46. How complex was the plot ( i n f i c t i o n ) or how complex were the issues/topics as presented ( i n non-fiction)? 1. r e l a t i v e l y simple 2. r e l a t i v e l y complex 47. How would you rate the quality of the acting/announcing/interviewing overall, r e l a t i v e to other programs of this type? 1 2 If poor/specify why r e l a t i v e l y poor r e l a t i v e l y good • '" 48. How would you rate the quality of the program in a technical sense (sound, pictures, etc.)? 1 _2_ 3 poor quality professional s l i c k and (with or ( s l i c k ) but special effects without special no special effects) effects 49. a) Was there any portrayal of r e l i g i o n or comments concerning r e l i g i o n i n the program? 1 2_ 3 4 5 d e f i n i t e l y can't decide d e f i n i t e l y not unsure yes If yes, specify b) If r e l i g i o n or r e l i g i o u s people were discussed or portrayed how would you evaluate this discussion or portrayal? 1. a) joke/humorously, with defi n i t e negative impression b) joke/humorously, with defi n i t e positive impression c) both are present 2. a) seriously, with defi n i t e negative impression , b) seriously, with d e f i n i t e positive impression c) both are present - 20 -Part 7: Subject information 50. How many hours do you typically watch TV i n each time period? Please think carefully about each time and enter your most accurate estimate of your TV viewing for that period. Before Noon Noon to 7:00 pm After 7:00 pm Monday Tuesday Wednesday Thursday Friday Saturday Sunday 51. Please l i s t i n order your favorite shows. 1. Cmost favorite)' 2. ; 3. . 4. (Use the back of the page to continue i f necessary) 52. If different from above l i s t the shows that you most often watch. 1.(watch most often) £.. _ _ _ ^ 3. ; 4. _ 5 _ (Use the space at the end to continue i f necessary) 53. Most people have different reasons for watching television. Such as relaxation, information, entertainment. Please l i s t the various reasons you watch TV in order of importance to you. 1. (Most important) . . 2. __ ; ; 3. . 4. 5. (Use the space at the end to continue i f necessary) 54. On the following scale indicate how often you watch the show you've just seen. every about about about about a never week twice once every couple of seen i t each each few times a before month month months year 90 - 21 -55. How much did you l i k e this show? Hot at a l l . Liked i t Disliked i t a lot 56. Age 57. Sex 58. Faculty 59. Year 60. How many-hours do you t y p i c a l l y spend studying i n each time period?- Please think carefully about each time and enter your most accurate estimate of your studying for that period. This does not include time spent i n lectures. Defore Hoon Hoon to 7:00 pm After 7:00 pm Monday ' ' . Tuesday . Wednesday _ Thursday Friday . Saturday • ' ' : l;'r Sunday :  61. Ethnic background 62. Were you born i n Canada? Yes Ho If no, how many years have you been here? 63. Were your parents born i n Canada? Yes Ho If no, how many years have they been here? 64. Were your grandparents born i n Canada? Yes Ho __ If no, how many years have they been here? 65. What i s the primary language spoken by you, -your parents, , at home? 66. Educational goal (highest degree you hope to get) 67. Occupational goal 68. Father's current occupation ;  I f unemployed or r e t i r e d , last occupation :  69. Father's highest education 70. Mother's current occupation [ ; If unemployed or r e t i r e d , l a s t occupation 71. Mother's highest education _ 91 Appendix B 92 CODING SHliBT A. General Information 1. Program Tit le 2. Program ID 3 . Length of program in minutes 4. Channel _ _ _ _ 5. Date of program 6. Start time of program A.M./P.M. (Circle one) 7. Coder 8. Data of coding 9. Program production source (country) 1<K .Canadian content: Yes_ No 11. Interruption Count: tally it __. :  (number of times program was interrupted for ads, newsflashes, etc.) 12. Notes taken during program: Bit or Segment topic Length Countries mentioned Ethnic Groups Mentioned 1. . 3. 4. 5. 6. 8. 9 • 1 0 . 12. 13. 14. 15. Female prominent characters in order of appearance (*ethnics!) 1. 5. • 9. ; 13... 2. . 6. 10 14.. 3. 7. 11. 15.. 4. 8. 12. 16.. Male prominent characters.. 1. _ 5.. ' 9 . _ 13.. 2... 6. ; 10 14., 3. 7._ 11. 15.. 4. 8. 12. ; 16.. 93 B. Commercials • t 1. Were there any commercials? No. 2. Mention of program sponsorship? No, Yes. Yes C. CRITC Coding 1) Audience: 1. Children 2. Other 2) Informative: 1. No 2. Yes 3) Religious: 1. No 2. Yes 4) Animated: 1. Live 2. Both 5) Program Type: 3. Animated Non-fiction. Informative, Instructive 1. Direct instruction 2. How to and informational Non-fiction. Informative. Real World 4. News/weather 7. Hews and current events magazine • t 10. Documentary (historical, visual arts, people biography) Non-fiction Entertainment 13. Reality programs 16. Game show 31. Music videos Fiction - Comedy 19. Situational Comedy Fiction/Action/Adventure TXj.. Western Fiction, Other Drama 24. Medical 27. Other , 30. Modern Classics S. News/special coverage 3. Sports/covera,ge of event 11. Talk show/ interview 14. Talk/show/variety 17: Variety (dance, music, comedy, vignettes) 20. Other comedy story 22. PoliceAdetective/ crime 25. Horror/scary 28. Historical Drama 3. Religious Ser^ic* 6. News analysis commentary ' 9. Sports/magazine 12. Documentary science & nature 15. People and places magazine on location 18. Cultural events or performances 23. Other -26. soap opera's-29. Classical Drama 94 3 6) Kxpectations/Faaillarity: 1. Series with mostly the same major characters or presenting people and mostly the same format and settings from one program to next (very few new major characters occur in each episode), (e.g., Captain Kangaroo, Dallas, Cosby Show, Sesame Street), ( i . e . , high expectation of familiarity froa one episode to another). 2. Series With generally the same format/narrator/announcer/continuing •' characters but at least half of the people in the program are different from one episode to the next. The setting may be the same froa one episode to another (e.g.; Johnny Carson, game shows or there may be major changes in the settings from one episode to another (e.g., Fantasy Island, Quincy, Wide World of Animals, Nature of Things). 3. Series with different people and content in each episode but continuing format (e.g., NOVA), OR series with continuing characters over a few (2-10) episodes, i . e . , mini-series (e.g., Roots, Masterpiece theatre series), OR series with a completely unrelated story each program but the same person who introduces i t (e.g., Hitchcock, Twilight Zone). 4. No continuing people/characters, content or settings (e.g., movies, special broadcasts, etc.) . 7) Is the.program in 1. story format 3. not sure, can't decide 2. non-story format 8} Content Time Demands: (You need to use a watch for this question) Time usually devoted to story or bit: Single episode (If story does not begin and end in one episode then code as multiple episode) 1. Less than 5 minutes If you believe this episode is not typical 2. 5 to 15 minutes of other episodes of this program, check , 3 . 16 to 30 minutes here 4. 31 minutes to 1 hour 5. 1.C1 hours to 1 1/2 hours , . 6 . 1 1/7 hours or more Multiple episodes ( i . e . , single episode is not complete on its own; story continues) 7. Finite number of episodes (e.g., mini series with a continuing story but with a definite ending) 8. Indefinite number of episodes (e.g., soap opera where the story never really ends) 95 0. UBC CODING Part 1 : Catud*. the U . 8 . . irod other' Countries (Note: A* you weten the program, l i s t countries shown/mentioned in sequence .a,s they occur beside the numbers 1-15 on the front page of the coding sheet, but do not f i l l in the table below unti l the show ends.) 1 . (a) If there was anything about Canada, the U.S. , or other countries in the program, check the following table as appropriate. Indicate whether th* country Was a major focu» , minor focus, or passing reference, as.w*J.l as whether it was the subject of joke. Then indicate how It was portrayed, oc» balance, or whether there was not enough Information to code. J major focus " 1 minor focus i • passing reference 41 5 more negative emphasIs T3 _. O • • e <i —j n J3 more positive emphasis not enough -Information to code Canada U.S.A. Other countries (specify) 1 . • 2\ 3. A. 5 . 6 . J. . • 8. • 9. 1 l o . Q 1 1 . 3 - 2 . 1 1 3 . 1 IA. • •• rr > 1 1 5 . 1 1 , . . . J . 96 page 4(a) (b) Was there any reference to or portrayal of English Canada or English-speaking Canadians? no unsure yes If yes, how much of the program's focus did i t have? major focus passing reference minor focus joke context (c) Was there any reference to or portrayal of Prench Canada or French-speaking Canadians? no unsure yes If yes, how much of the program's focus did it have? major focus passing reference : minor focus joke context (d) Would a person from another country get any indication that Canada is a bilingual country? (e.g., portrayal of Francophones in an Anglophone context or vice versa, etc.) no unsure yes 5 97 finrt ?; North .ftwlw K ^ n v r H ^ f 2. (•) were any North American ethnic minorities portrayed? 1. no 2. yes (b) The prominent characters In the program were: 5 v5 al l mainstream non-ethnics mostly mainstream 'non-ethnics but : some important ethnics mixed mostly ethnics but some important mainstream non-ethnics al l ethnics : not appliceblrt (c> Were there any verbal or visual jok.es about ethnic minorities? 1. no 2. yes (d) Complete this table for the ethnic individuals identified on the flrat page. If not enough information Was given in the program to code them here, do not put the* in this table, just note them on the front page. Individual's Iethnic group sex LEVEL OF FUNCTIONING IIN MAINSTREAM NORTH I AMERICAN socimr Ti iH e w o C v> O -H *H O 4J W U C C X 3 VI <M 0) a c a 3 s O XT a • JO 9 O W D . ce a a o B Ti o a T> x v v o ai § o Xi U 2 S o. STRENGTH OF ETHNIC IDENTITY FINAL IMPRESSION V tl O TJ a -n o> •o o > c VI O V o v> u 1 VI 1 —* 1 ° i o 1 «-< 4) I *-< •H TJ 1 «• B 1 •*» •a 1 • C C 41 1 •* o 1 *> 1 o x -n 1 v •H 1 * o 1 11 ^1 1 > CD 1 TJ 41 1 > a ai 1 ° V 1 " T3 I ° n vi 4> >. C 1 4) VI 1 " VI I 41 u .H 41 1 > o. 1 r-i 1 > cv OfiTJ 1 •»* s \ * c I s C - i 1 f —1 1 U K | w —< o 1 1 vi o 1 " Vl 1 • 1 P 1 0 0 VI 1 o 1 % J 4) a • a. 1 c 1 C 98 6 (e) Was there any clear evidence (portrayal or reference) of racism or prejudice? no yes If yes, was it portrayed as: acceptable (e.g., let pass; not contradicted) unacceptable (e.g., comeback) 3. (a) Does the program take place in a current North American setting? No Yes '"'(b) If yes, did the program portray the following? 1. Everyday home l i f e of a North American (rich or poor) no yes 2. Typical North American workplace (eg., factory, labour •job, office lobs, with focus on the workers) Specifv briefly no yes 3. Typical public l i f e (eg-, shopping, public transport, places providing service such as post offices, laundromats, banks, restaurants, with focus on people receiving service) Specify briefly nO yes 4. Health care or legal (non-police) settingsin N.America Specify briefly no yes 5. Police or military settings in North America SDecifv briefly no yes (c) Were people shown doing spare-time activities? no yes Specify briefly (d) (i> Was there any reference to or portrayal of television within the program? (e.g., show on TV personalities or programs, fictional program about a character who is a TV producer, people shown watching TV) definitely can't definitely not decide yes Specify briefly. ( i i ) Was there any reference to or portrayal of reading within the program? (e.g., show on literary personalities, f ictional program about a writer, people shown reading) definitely can't definitely not decide yes Specify briefly 99 7 Part 3 : The Saxes 4. Were the prominent characters in the program (those you listed on the front page and did not later cross off) 1 2 ; 3. ii 5 mostly mostly male but even mix of females mostly female but mostly male some Important and males some important female females males Did you notice any sexist comments/jokes/putdowns about females in this program? (e.g., "Woman driver" jokes, using the words "broad",or "dame", etc.) 1 . no 2 . yes If yes, specify ;  What's tfce bottom line message of this program re sexist messages about females? 1 . acceptable (e.g., comment let pass). 2 . unacceptable (e.g., comeback to comment) Did you notice any sexist comments/jokes/putdowns about males in this program? (e.g., comments about men only having one thing on their mind, "dirty old man", "male chauvinist pig" coaments, etc.) 1 . no 2 . yes If yes, specify •. •  What's the bottom line message of this program re sexist messages about males? 1 . acceptable (e.g., comment let pass) ?. unacceptable (e.g., comeback to comment) 7. Were there any direct or indirect references to a relationship between mood or emotion/pain/dlscomfort and the menstrual cycle? 1 . No 2 . Tea 8. Was romance a part of the program? 1 2 - ". : " 3 . ; f ' - ' not at a l l 6ome romance romance is a major focus 9. Was sex (portrayal or reference) a part of the program? , in animals-1 . none 2 . some - c , 3 . a lot In humans 1 . none 2 . some 3 . a lot 100 i HOW WAS THE RELATIONSHIP DEPICTED? loving, caring o s •o 3" cool, casual e x p l o i t i v e h o s t i l i t y included (psych-=f - ^ o l o e i c a l / v e r b a l aggression 1) physical aggression included m 3= AGES OF PEOPLE IN RELATIONSHIP* child (pre-puberty) •o n m • » teenaaer (13 to 19) adult (20 to 35) middle aged (35 to 55) RELATIONSHIP IS CLEARLY ROMANTIC SEXUAL BEHAVIOR f l i r t ing verbally or shoving sexual interest kiss/touch but no clear sexua|. intentions . Kiss / touch with clear sexual Intentions .  clear implication of sexual but no outright portrayal clear portrayal of sexual act explicit reference to sex between people in relationship (not hint or innuendo) relationship .portrayed without any sexual behavior • tr 0 o V - ^ to C 00 rr • a • n ' -.jh • n •a c » D • v n • a 1 < a It m I T (6 " •0 5. in a cumtitted love B relattonahip, not 0 living together | 0 9 3 C» >-" <t m %> rr rr ZT ~~ n m o o "* 9 B •c <-• 9 ' a -»-* —- t-t' c o >- < a A ? 3 ' e» t 9 n rr — 7 A it a i . 3 0 r * < 3 * n «T a < «• c O • • (— . 9 1. Relationship Is | not clear 1 to • o 9 << l » 1 T ) m PJ 0 rr 1 — O X = 1 n o •• • 3 9-j a MOW WAS THE RELATIONSHIP J DEPICTED^,. „. EEafg^ HaaSHLHsuRimu i~* 5. in a cumtitted love B relattonahip, not 0 living together | loving, caring' ^ 3. cool, casual m exploitive "' .• £ hostility included (psych- g ological/verbal aggression) T> physlca'l aggression .'included » ACES OF PEOPLE ^RELATIONSHIP child (pre-puberty) teenager (13 to 19) • young adult (20 co 35) middle aged (35 to 55) older (55 or over) 1 RELATIONSHIP IS CLEARLY ROMANTIC • SEXUAL BEHAVIOR f l irt ing verbally or showing sexual interest kiss/touch but no clear sexual intentions kiss/touch with clear sexual Intentions • . clear implication of jsexual act but no outrlcht oortrtival i clear portrayal of eejxual act(s) i explicit reference tcj sex betveei people in indicated Relationship (not hint or innuendo) relationship portrayed without any sexual behavior i 102 vvfu • V.I 11. Was any of the behavior shown/references concerned with any of the following! 1. homosexual females homosexual males u a O 9 — o S ° M • 2 a B O •H O 8 <w oo u 5 § a u a « B. « JM 4) •O M t) O _> — S a « a, •o o f-l u « A O s IT; • o. a 41 a 41 ja W « 0 4) a. ti .o 41 •W >H V4 41 A O _; _ O <H • * s- a a 4i £ 4i a o • fi S o ^ 2. group sex (3 people or more) 3. masturbation 4. sexual sadicm/masochism 5. exhibitionists/voyturs 6. tellshes 7. tgsniesuals 8. tfansvostUes 1 her (specify) 1. sexual assault I?, incett 12. Was there evidence regarding a double standard for females and males regarding sexual bebavlourT (Circle as many as apply) 1. no 2. yes, consistent with it 3. evidence or arguments against it *«. joke Specify : ; •• - . . - v • 13. Was there any portrayal of or reference to (circle as many as apply): 1. Contraception 2. Pregnancy a) serious b) joke S. Spontaneous Abortion (miscarriage) a) serious b) joke a) serious b) joke 3. Sexually transmitted .4. AIDs diseases (other than AIDs) a) serious a) serious b) joke b) joke 6. Induced Abortion a) serious b) joke 7. Prostitution a) serious b) joke 14. Was there any nudity shown in the program? 1. no 2. yes, female 3. yes, male 4. yes, child (age 4 or under) 15. Were there any people ahown or referred to as being sex objects (.e.g., physically exposed or acting in a manner vhich excites interest In the opposite . sex). • ' • 1. No 2. Yes Complete the following table (whether or not sex objects were shown) •  Categories: c V a x a tJ tr 4) o ce x u ° s * o u to o 60 V V> • o o ce c 8^ a. •o «l Vl ll 13 "— a «i B . i H O VI O X 3 41 CO X >, VI —< o X 4) •H_T"> tg J i gg o z» 41 w a x oo si 1 o <W i-l v> >, C eg 41 Vi a «-> tl Vi Vi o a. a VI VI S3 eg O iH 41 a a ti S 41 1-1 >VI XI O o VI K v 4i a « O. 4) Vi o 60 n 5 8 4) X . si » 41 ,-1 VI O o u «l 1-1 > « X o o VI x vi 41 4 o a-« v> o •H 4) •-I a 41 60 OS Q ° s TJ O 4i a 41 O 4i e M 41 O Vi 1-1,41 <V| VI 41 U V t l i - l i - l H £ 4) > a u <H 41 O X VI VI O Vl o While answering the following questions (16 regarding females and 17 regarding males), be sure to look at the characters you have listed on the front page. 16. ^ _ Check here if there were no females in the program and go to #17. (a) The prominent adult females (relative to other females in this program) were predomlnantly in traditionally female occupations (include homemaker) equal mis o f traditionally and nontradi-ttonally female occupations predominantly in nontradltionally female occupations N/A (not enough information re: occupation to decide) (b) For the Prominent adult females, which types of activities were emphasized (airtime focus) in the program? (Check more than one if necessary for different individuals) major focus on occupation major focus on home/family role .major focus on soc i al l ife equal focus on occupation and home/family role JL equal focus on home/family role and social l i fe equal focus on occupation and social l i fe equal focus on occupation, home/ family role, and social Hi> N/A 104 12 (c) The background adult females (relative to other more prominent females in this program) were: predomih^nUy in .traditionally female occupations (include homemaker) equal mix of traditionally and nontradi-t^pa^lly female occupations predominantly in nontraditionally female occupations H/A or not enough information re: occupation to decide (d) Were there any adult females who behaved in a nontraditional way (e.g., strong in an emergency, not dependent on males for guidance, assertively saying What's on her mind, etc.)? definitely no nontraditional female behavior not sure can't decide nontraditional female behavior definitely present Specify, If (4) or (5) non-traditional female behavior present, was i t portrayed as (cirele a l l that apply): a) serious,,. O positive (e.g., successful/good/rewarded/competent) b) joke,.. . d) negative (e.g., unsuccessful/bad/punished/incompetent) (e) Were there any adult females who acted in a traditional way (e.g., a male for guidance, falls apart in crisis, non-assertive, etc.)? dependent on definitely no traditional female behavior np'jfc sure can't decide traditional female behavior definitely present Specify, If (4) or (5) traditional female behavior present, was i t portrayed as (circle a l l that apply): . *) serious c) positive (e.g., successful/good/rewarded/comp'etent) b) joke ' d) negative (e.g., unsuccessful/bad/punished/ihcompetent) (f) Were any females portrayed in a clearly positive way in the program? J _ l definitely no positive females not sure can't decide positivla females definitely present 105 13 (g) Were any females portrayed In a clearly negative way in the program? -1 2 3 A 5 definitely no negative females not sure can't decide ' negative females definitely present 17. Check here i f there were no males lei the program and go to #18. "'" (a) The prominept adult males (relatlye to other males in this, program) were: predominantly in traditionally male occupations equal mix of traditionally and nontradi-tionally male occupations predominantly in nontradltionally male occupations (include homemaker) N/A or not enough information re: occupation to decide (b) For the prominent adult males, which types of activities were emphasized (alrtime focus) in the program? (Check more than one i f necessary for different individuals) major focus on occupation' major focus pri home/family role major focus oh social " l i f e equal focus on occupation and home/family role . 5 7 8 equal focus on home/family role and social l i fe equal focus on occupation and social l i fe equal focus on occupation, home/ family role, and social l i fe N/A (c) The background adult males (relative to other more prominent males in this program) were: 'predominantly in traditionally male occupations equal mix of traditionally and nontradl-tionally male occupations predominantly in nontradltionally male occupations (include homemaker) H/A or not enough information re: occupation to, decide (d) Were there any adult males who behaved in a nontraditional way (e.g., shown crying, caring for children, doing domestic chores,., sensitive to the needs of others)? : definitely no nontraditional male behavior. not sure can't decide nontraditional male behavior definitely present Specify. 106 14 If (4) or (S) non-traditional male behavior present, was i t portrayed as (circle a l l that apply): a) serious c) positive (e.g., successful/good/rewarded/competent) b) joke d) negative (e.g., unsucceseful/bad/punished/incompetent) / (e) Were there any adult males who acted in a traditional way (e.g., skirt chaser, tough/macho, calls the shots, life revolves around job and self)? 1 2 3 4 5 definitely no not sure traditional male traditional can't decide behavior definitely male behavior present Spec i f y __ If (4) or (5) traditional male behavior present, was i t portrayed as (circle a l l that apply): a) serious c) positive (e.g., successful/good/rewarded/competent) b) joke d) negative (e.g., unsuccesBful/bad/punished/incompetent) (f) Were any males portrayed in a clearly positive way in the program? 1 2 3 4 5 definitely no not sure positive males positive males can't decide definitely present (g) Were any males portrayed in a clearly negative way in the program? 1 2 3 4 5 definitely no not sure negative males negative males can't decide definitely present 18. Were there any messages indicating females are subordinate to or worth less than males; e.g., parents disappointed that it was a gir l baby; any mention of men trading or owning women; women taking a back seat to and/or being obedient to men; parents preferring a son over a daughter (because he will carry on the family name, business, etc.)? 1. no 2. yes If yes, specify what gave you this impression Was thie portrayed as: J 1. acceptable : \ - :  2. unacceptable • '. '• 19. Were there any messages Indicating that males are subordinate to or worth less than females, e.g., parents disappointed that i t was a boy and they wanted a g ir l baby; parents favoring a daughter over a son? 1. no 2. yes If yes, specify what gave you this impression Was this portrayed as: ___ • . 1. acceptable : ;  2. unacceptable ; : ! 15 Part 4: Autression In the following question, an aggressive act it ona which la Intentional, interpersonal, and (unless otherwise stated) visually portrayed and 1% against , humans (alive or dead), humaneid ccjsc -^s (e.g., robots), animals, or any representations"of humans or animals (e.g., cartoons). This includes, antisocial. acts with the potential to do harm but from which the victim escapes uninjured. For example, intentionally trying to run someone down with a vehicle, whether successful or not, is aggression. But if someone accidentally crashes into another vehicle in which a person is Injured, this is not aggression (unless the person causing the accident was engaged deliberately in an antisocial act, e.g., speeding without regard for pedestrians). Anothor exsmple of aggression would be intentionally setting fire to a building, whether or not it resulted in injury. On the scales for aggressive activity, avoid using numbers 2 and 4 if at al l possible*, they mean toward aggression or no aggressIOBI.UBS only if you cannot possibly assign a 1 or 5. These questions do not relets to quantity, only whether aggression occurred. 20. (a) Was there any physical aggression shown in this program? definitely no aggression shown can't decide unsure physical aggression definitely present (b) Was there any physical aggression by females shown in this program? definitely no aggression by f^-ales can't decide unsure physical aggression by females definitely present (c) Vas there any physical axKressloa- axainst females shown in this program. ••• • i • 2 • • "3 • •  , 4 ' ' . ' ... 5 definitely no aggression against females can't decide unsure physical . ' aggression against females . definitely present (d) Was there any physical aggression by males shown in thia,jprogram? •i •> .<.-. definitely no aggression by males can't decide unsure • • physics-aggression by males definitely present 108 16 (e) Was there any p h y s i c a l aggression against males shown i n t h i s program?. 1 2 . 3 4 ' ' "-5 •-' ' d e f i n i t e l y no aggression against male6 can't decide unsure physical aggression against sales definitely present ( f ) Was there any reference made to p h y s i c a l aggression i n t h i s program: 1 2 3 4 5_ d e f i n i t e l y no reference to p h y s i c a l aggression . can't decide unsure definitely had reference to physical aggression (g) Was there any v i o l e n c e shown i n the program ( i . e . , extreme p h y s i c a l aggre.slon. that i s p o t e n t i a l l y f a t a l ) ? d e f i n i t e l y n o nviolent can't decide unsure d e f i n i t e l y v i o l e n t (h) Was there any reference to Violence i n t h i s program? 1 2 3 4 d e f i n i t e l y no references to v i o l e n c e can't decide unsure d e f i n i t e l y bad reference to v i o l e n c e (1) What i s the bottom l i n e message of t h i s program re the a c c e p t a b i l i t y of p h y s i c a l aggression as^ a method of c o n f l i c t r e s o l u t i o n ? (Check more than 1 i f more than 1 message was perceived.) p h y s i c a l aggression i s c l e a r l y acceptable can't decide .unsure p h y s i c a l aggression i s c l e a r l y unacceptable N/A or not enough information to code S p e c i f y , (j) What Is the bottom l i n e message of t h i s program re the successfulness of p h y s i c a l aggression as a method of c o n f l i c t r e s o l u t i o n i n the short term rather than long term sense? (Check more than 1 i f more than 1 message was perceived.) p h y s i c a l aggression i s c l e a r l y s u c c e s s f u l (e.g., short term goals were achieved through p h y s i c a l aggression) can't decide 'unsure p h y s i c a l aggression i s c l e a r l y not su c c e s s f u l (eg. short term goals were not achieved) N/A or not enough information to code S p e c i f y 109 17 21. (a) I I there any verbal/psychological aggression in the program? definitely not at a l l verbally/ psychologically aggressive can't docide definitely verbally/ psychologically aggressive Specify. (b) What is the bottom line message of this program re the ecceptabilty of verbal or psychological aggression as a method of conflict resolution!?, ' (Check more than one i f more than one message Was perceived) 1 2 3 4 5 ; " - ! verbal/psychological aggression is clearly acceptable can't decide verbal/psychological aggression is clearly unacceptable N/A or not enough information to code Specify. (c) What is the bottom line message of this program re the successfulness of verbal/psychological aggression as a method of conflict resolution? (check more then 1 i f more than 1 message was perceived). verbal/psychological is clearly successful (e.g., short term goals were achieved •. • through this behavior) can't decide unsure verbal/psychological is clearly not successful (eg. short term goala were not achieved ; through this oehavior) N/A or not enough Information to code Specify. 22. Does the program give the impression that the world is a dangerous place Oe.-g'., because of war, physical aggression of others, natural disasters, fatal traffic accidents etc.)? program gives no impression that the world is-a dangerous place-some people mlght get the impression that the world is a dangerous place impression lr clearly given that the weir Id is a dangerous place Specify. 110 18 23. (•) Were any reel guns shown in the program? No. <b) Who had them? (Check al l that apply) -Yes Military Good citizen Police Bad citizen^ Other authority. <c> How were the guns used?; (Check all that apply) some or al l not used defense of self/others, defense of property aggression intimidation/coercion, recreation, 24. Was there any portrayal of.or reference to death of humans? I. no 2. yes If yes. put checks below as appropriate for individual deaths; fW the deaths of several.people (2 or more) use "s" (for several) instead of a check as appropriate'. ' :' •••!..-• Mov . ••• u o 0 £ •w 0 > -o 0 * 2 0 0 u A i i O "O •S3 « v -o <* § in 0 « O; 9 i • r Death Portrayal « 30 •J V If <M •k-' « ' 3 o H 3 J3 >• • U 0. •a « o 41 a -- I u 4 0 JC 0 «s w oo 0 0 te a « « 0 o 0 "8 0 o —> e o 0 W .4) a. 0 c o ,0> . w r -0 0 o 0 A "3 s X to. 0 0 —' 19 Part 5. Issues. Controvertias. and Dilemmas 25. (a) Does this program: 1. Present no evidence that problems (either serious or minor) exist. 2. Deal with less serious problems such as family disagreements, boyfriend-girlfriend, wife-husband problems, or other problems that are relatively unimportant in the long run. Specify " 3. Deal with serious problems and issues (either having considerable Impact on the l ife of one individual—e.g. abortion, divorce, alcoholism, or having impact on the lives of many—e.g. pollution, corruption in politics). Specify •  (b) Were the problems or issues (draw lines from above if you've checked both (a) 2 and 3): 1. central to the plot 2. Incidental 3. portrayed seriously 4. portrayed as funny (c) The take-home message from this program regarding issues and controversies is that they are usually: (check one) 1. clear-cut (relatively 2. not clear-cut ' 3. not clear-cut . 4. N/A black and white) and (shades of gray, (shadea of grey, the right answers and some anstfmrs and there are ho , • • are clear - are batter then^ right or better others) answers) 26. (a) Were there explicit (surface structure) political comments or connotation in the program? (circle highest applicable number) - i : . 1 2 2 4 < 5 definitely can't decide definitely <in<nr« yes j (b) Was the content more balanced or more partisan? 1. balanced 2. partisan • Specify briefly • •- •• ... •• ••»-• • • , 27. (a) Was there any portrayal of religion or comments concerning religion in the program? 1 2 3 4 S definitely can't decide definitely not unsure y«* 112 20 ('$) I f r e l i g i o n or r e l i g i o u s people were portrayed or disc u s s e d , i n d i c a t e how ( c i r c l e as many as apply) f ( i ) - : joke/humorously, w i t h d e f i n i t e negative impression' j ( i i ) joke/humorously, w i t h n e u t r a l or p o s i t i v e impression : ( i i i ) s e r i o u s l y , w i t h d e f i n i t e negative impression ( l y ) s e r i o u s l y , with n e u t r a l or p o s i t i v e impression Part 6. P o r t r a y a l of People 28. O v e r a l l ! ( g l o b a l impression) i s t h i s program p r i m a r i l y about n i c e , decent, • w e l l - i n t e n t i o n e d people? mostly nasty even mix of nice and nasty mostly n i c e 29. (a) Was the show concerned w i t h the r i c h and/or famous? not at a l l somewhat predominantly, or completely T F 2 or 3, were they (b) \. f i c t i o n a l Did they l i v e i n the (c> l i j present .2. r e a l l i f e 2. past (d) i j j r e a l l i f e c h a r a c t e r s , were they 1 2 3 popular itertainment high brow ente r t a i nm.ent science or p o l i t i c s r o y a l t y , establishment Other Specify:. 30. (a) V**re any of the f o l l o w i n g groups represented by at l e a s t one i n d i v i d u a l i n the I program? (Check as appropriate) C h i l d r e n (under 12) TeenaBers (13-18) Adults (19-50) Maturf Adults. (51-65) . : Senloi s (over 65) (e.g. r e t i r e d , grandparents) Sex Unclear Male Female Handicapped/chronically i l l Very?feogypeefrl-e • - : , , , . . . Very r i c h people Communists Homosexuals 21 (b) If there were individuals belonging to any of the following groups, f i l l out the following table indicating whether they were shown in ways consistent with the positive or negative stereotypes described , there was not enough Information to code. Check as many as apply. Please note that the stereotypes described are merely examples of positive and negative stereotypes for each category—others would also be relevant and these need not be there. or neither. Check the lai GROUP Seniors Children Teenagers Handicapped/ Chronically-i l l Very poor Very rich Communists Homosexuals POSITIVE STEREOTYPE wise, kind, grand parently, active sweet, innocent, charming reasonable, sensible, studying hard, involved in many activities showing exceptional bravery, strength, and perserverance in the face of difficulty warm, 'carinsJY generous, struggling In a difficult situation philanthropists, intelligent, using money unselfishly well-intentioned, collectivist, sharing social responsi-bil ity sensitive, caring, Into the fine arts _ L VEGATIVE STEREOTYPE dependent, senile, burden to others, inactive, crabby _ noisy, messy bratty, rebellious, obsessed with sexuality/rock music/being "In", Inconsiderate helpless, passive, dependent burden laty. Ignorant, untrustworthy _ materialistic power-hungry, 'ruthless secretive, vil lains, threat to free world limp wrist, l isp, butch or other stereotypes < I OS S O Cu * £ H O W oa OS u H H Z V) al W H sa O t£ Z t box If 114 22 Part 7. Global Impressions 31. How involving was the program? 1 _2 3 not at somewhat very all involving involving 32. (a) Being humourous was: not at al l intended a minor goal of the show a major goal of the show (b) How humorous were the parts that were intended to be humourous? 1 2 3 ' not at all humourous/none some/mildly humourous most/very humourous The final questions deal with the overall message you are left with. These messages are not necessarily specifically stated or even intentionally implied by the producers. 33. How would you best describe the political philosophy of this program? 1 _ 2 3 4 5  apolitical left-wing (liberal, socialist) centre right-wing (conservative, capitalist) defini tely political but not identifiably left or right 34. Considering the 'country of origin of this program, how nationalistic was it? (e.g., "rah rah for our country"; our country or its citizens are especially wonderful in some way) definitely not nationalistic can't decide unsure definitely nationalistic For each of the following questions (36-39) check the statement that best describes the take-away message. 35. Laws/authority/the state: I 2 ' 3 4 5 no inform. are always right may be wrong, . but s t i l l must be obeyed ("The Law is the Law") may be wrong but can be worked around or bent as needed may be wrong and the best avenue for change is working outside the system (revolution or vigilantes taking law into their own hands) 115 23 3 6 . The m i l i t a r y and/or p o l i c e : (a) 1 2 need more au t h o r i t y have tbe r i g h t amount of a u t h o r i t y have too much a u t h o r i t y N/A (b) need to be stronger ( i n terms of numbers and/or money, equipment) are a p p r o p r i a t e l y strong are too strong N/A (c) are inept some are inept and some are competent are competent N/A 37. The powerful/authoritative/knowledgeable' are: (a) 1 7_ 3 4 mostly mostly male but males some Important females mixed mostly female but mostly N/A some important females, males (b) a l l mainstream non-ethnics mostly mainstream non-ethnics but some important eth n i c s mixed mostly e t h n i c s but some important mainstream., non-ethnica 38. Current p r o t e c t i o n of tbe environment al l e t h n i c s N/A i s not s u f f i c i e n t is s u f f i c i e n t i s top strong for economic good h e a l t h N/A Part 8: S t r u c t u r e of Program 39. When does tbe program take place 1 ' 2 current ( w i t h i n a decade) past i . e . h i s t o r i c a l future combination s p e c i f y : 1 & 3 1 & 2 . 2 A 3. 1, 2 . 3 40. How complex was the p l o t ( i n f i c t i o n ) or how complex were the IssueB/topics as presented ( i n n o n - f i c t i o n ) ? 1. r e l a t i v e l y simple 2 . r e l a t i v e l y complex 116 24 41. How hard/easy was the program to f o l l o w ? (a) Could someone leave the room f r e q u e n t l y or tune In during the middle of the program and s t i l l f o l l o w the s t o r y or comprehend the parts of the program (e.g., news) they d i d see? no yes Now, c i r c l e (b) or (c) below. (b) When i n the room w i t h the TV, could someone d i v i d e a t t e n t i o n f a i r l y r e a d i l y between TV and other a c t i v i t i e s (eg. k n i t , look at a magazine), by l i s t e n i n g and o c c a s i o n a l l y l o o k i n g at the screen? or (c) would i t be d i f f i c u l t to d i v i d e a t t e n t i o n w i t h other a c t i v i t i e s because they would be l i k e l y to miss something e s s e n t i a l ( i . e . , i n order t o understand the program i t i s necessary to both watch the screen and l i s t e n ) ? 42. How would you r a t e the q u a l i t y of the acting/announcing/lntervlewing o v e r a l l , r e l a t i v e to other programs of t h i s type? 1 2 I f poor, s p e c i f y why r e l a t i v e l y poor r e l a t i v e l y good 43. How would you rate the production values of the program i n a t e c h n i c a l sense? 1 2 3 poor q u a l i t y p r o f e s s i o n a l s l i c k and (with or ( s l i c k ) but s p e c i a l e f f e c t s without s p e c i a l no s p e c i a l e f f e c t s ) e f f e c t s 44. What was p a r t i c u l a r l y notable about t h i s program? nothing P o s i t i v e : • • Negative: Neutral: 117 25 45. Need bo consult re: the specific questions listed for each of the following pages: I. ' r 14. 2- ' ' '"' : " ' '' :' " ' '• "• 15. 3. 16. 4. | 17. 5. ' 18. 6. 19. 7. 20. 8. ' 21. 9. ' • • 22. 10. - . 23. II. 24. 12. 25. 13. : 26. After viewing the program, fast forward through commercials to the start of the following program. Is the program the same one as specified in the TV Guide Listings? ; No Yes Name of program following program viewed • •  If there are discrepancies, note them here. 118 Appendix C Audience Research Project Volunteer Information 119 If you are willing to participate in this research, please answer the following questions. All of the information will remain confidential. When you participate this sheet will be given back to you. You will not be asked to put your name on the questionnaire you complete then. This will ensure that the data you provide will be confidential. If you decide not to participate this sheet will be destroyed. We are not able to pay all participants, but at the end of the project we will randomly select the names of three participants. The first name selected will win $100, the second $75, and the third $50. Name Phone number Best time to call Age Sex -Faculty Year Ethnic background Were you born in Canada? Yes No If no, how many years have you been here? Were your parents born in Canada? Yes No If no, how many years have they been here? Were your grandparents born in Canada? Yes No If no, how many years have they been here? •  What is the primary language spoken by you, your parents, \ at home? Father's current occupation If unemployed or retired, last occupation Father's highest education ; "  Mother's current occupation If unemployed or retired, last occupation Mother's highest education 120 Appendix D 121 STATISTICAL CONSULTING AND RESEARCH LABORATORY DEPARTMENT OF STATISTICS THE UNIVERSITY OF BRITISH COLUMBIA TO: David Wotherspoon REF: File 87-10-099 Psychology, UBC FROM: Peter Schumacher DATE: Nov. 10, 1987 Managing Consultant Suppose you have a fc-point ordinal scale, which N subjects use to answer a question. A single expert also gives a score, which we will denote X*. The proposed statistic will summarize the degree to which the subjects agree with the expert. Moreover, it will also be reasonable to use the statistic to compare across different scales and X* 's. Definition of the Statistic. The proposed statistic differs algebraically from what we discussed at our meeting, because further examination revealed that there was no simple and elegant way of standardizing the scale of that statistic. Thus, for example, one could not have used it to compare two questions with different scales, or even two questions with different expert ratings but on the same scale. However, the spirit of the original statistic is retained here, in that it incorporates the total 'distance' of subjects from the expert. Essentially, one assigns a positive score to each subject who 1 122 agrees with the expert, and also to each subject who disagrees. Then one takes the sums of scores over the two respective groups; call these sums A and D respectively. The new statistic is A-D A + D, which we shall call T. (The notion of taking this kind of ratio is shared by the Goodman-Kruskal Gamma coefficient for agreement in ordered contingency tables. [1]) It follows that if no-one agrees with the expert, then A — 0, and T — — 1, regardless of the degree of disagreement of any individual. If everyone agrees with the expert, then D = 0 and T = +1. These facts hold for any scale, any value of the expert's rating, and any number of subjects; these are necessary conditions for inter-question comparisons. Indeed, these facts hold true for any scoring scheme as well, which shall be examined below. Recall that we discussed the concept of chance agreement. Suppose subjects scored randomly, paying no attention to the question's meaning. One would expect an approximately even (or uniform) distribution of scores. Thus there would be some subjects who agreed with the expert by chance alone. Intuitively, the statistic T should have an identifiable point corresponding to expected chance agreement. A natural choice is zero, for then positive values of T indicate a level of agreement in excess of, and negative ones a level of agreement below, this benchmark. The trick to accomplish this is to define the scores assigned to subjects in a certain way. Divide the group into those who agree with the expert, and those who don't. Denote.by Oj the score given to the i,h agreeing subject; and let dj be the score for the jth disagreeing subject. Now define dj to be the number of points on the scale by which subject j's score differs from the expert's, ignoring the direction. And D is the sum of these scores. Note that we originally discussed using the square of dj at our meeting; however, this will tend to inflate the effect of outliers on the statistic, and has no other apparent advantage. Subjects who agree with the expert will all be assigned the same score: a, = a, for all i. We want to define 'a' so that under uniform (random) scoring, T = 0. The choice of 'a' to do this will 2 123 depend on the number of points k on the scale, on the expert's rating X', and on the number of subjects N. So for a given question, a will have to be worked out using these known quantities. In what follows, a subscript '0 ' will denote values under uniform scoring. Using the definition of T as (A — D)/(A + D), setting Ao — D0 will make T0 = 0. Under uniform scoring, we expect N/k subjects to fall on each point. Since the ay's are just the number of points by which the subjects disagree with the expert, D0 can be calculated. And using Ac — D0, plus the fact that AQ = (N/k) x a, one gets a = (k/N) x Do-A n example may help to make this more clear. Suppose one had a five-point scale, with 100 subjects, and that the expert circled the second point from the left on the scale. Then the possible values of the dj are 1, 1, 2, and 3. Under random uniform response behavior, 20 subjects fall on each point. Then D0 = 20 x 1 + 20 x 1 + 20 x 2 + 20 x 3 = 140. And a = (5/100) x 140 = 7. Thus each agreeing subject will receive a score of 7. Now suppose the actual observed totals of respondents falling at each point were (15,30,25,15,15). Then D = 1 5 x 1 + 2 5 x 1 + 1 5 x 2 + 1 5 x 3 = 115, A = 30 x 7 = 210, and T = ( 2 1 0 - 115)/(210 +115) = 0.292. The fact that T is positive is consistent with the observed number of subjects in agreement with the expert, 30, being in excess of 20, the number expected under randomness. Further Comments. T is essentially a scaled version of the total agreement less the total disagreement, A — D, where the scores a, and bj define numerically what is meant by '(dis)agreement'. Dividing by A + D then standardizes the range of T, so that inter-question comparisons are possible. One cannot use A — D alone, for the possible values of D (and ultimately of A through a), depend on it and X*, and so differences or similarities in A — D for two questions represent not only subject's feelings, but also scale properties and the expert response. There is one drawback to the definition of T. When no-one agrees with the expert, T = — 1, regardless of the magnitude of disagreement. This can be seen from the definition: when no-one 3 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0097819/manifest

Comment

Related Items