Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Television content analysis : agreement between expert and naive coders Wotherspoon, David Kenneth 1988

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1988_A8 W67.pdf [ 5.69MB ]
Metadata
JSON: 831-1.0097819.json
JSON-LD: 831-1.0097819-ld.json
RDF/XML (Pretty): 831-1.0097819-rdf.xml
RDF/JSON: 831-1.0097819-rdf.json
Turtle: 831-1.0097819-turtle.txt
N-Triples: 831-1.0097819-rdf-ntriples.txt
Original Record: 831-1.0097819-source.json
Full Text
831-1.0097819-fulltext.txt
Citation
831-1.0097819.ris

Full Text

T E L E V I S I O N CONTENT A N A L Y S I S :  AGREEMENT BETWEEN EXPERT AND NAIVE CODERS  By DAVID WOTHERSPOON B.A. S i m o n F r a s e r U n i v e r s i t y ,  1986  THESIS SUBMITTED I N P A R T I A L FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES (Psychology) We a c c e p t t h i s t h e s i s a s c o n f o r m i n g to the required  standards  THE UNIVERSITY OF B R I T I S H COLUMBIA O c t o b e r 1988 (c)David  W o t h e r s p o o n , 1988  In  presenting  degree freely  at  this  the  available  copying  of  department publication  of  in  partial  fulfilment  of  the  University  of  British  Columbia,  I  agree  for  this or  thesis  reference  thesis by  this  for  his thesis  and  scholarly  or  her  for  of  Psychology  T h e U n i v e r s i t y o f British 1956 Main Mall Vancouver, Canada V 6 T 1Y3 Date  DE-6(3/81)  14  October  1988  Columbia  I  further  purposes  gain  that  agree  may  be  It  is  representatives.  financial  permission.  Department  study.  requirements  shall  not  that  the  Library  permission  granted  by  understood be  for  allowed  an  advanced  shall for  the that  without  make  it  extensive  head  of  my  copying  or  my  written  ii Abstract Agreement  between  television different  trained  content  and u n t r a i n e d  was i n v e s t i g a t e d .  approaches  t o content  microanalysis versus  versus coders  qualitative  A model  audience  The  watch  programs  chosen  popularity.  and  assess  from  the  on  They were n o t t r a i n e d  viewing  their  program.  Their  24  e v a l u a t i o n s were  of  their  until  after  compared  coders.  E a c h o f t h e 24 p r o g r a m s was w a t c h e d b y 5 m a l e a n d 5 coders  for ethnicity especially expert  that  The g r o u p s w e r e  status.  on  22  selected  untrained  and  t h e programs s i m i l a r l y .  the  unreliable) untrained  (expert)  balanced  A statistic  developed  f o r t h i s r e s e a r c h was u s e d t o c o m p a r e t h e n a i v e a n d  indicated  which  N=240).  and socioeconomic  ratings  evaluated  (total  trained  with  evaluations  naive  by  television  a n a l y s i s and d i d  were asked  previously  university  basis  i n content  expert  similar  female  given  of  the  n o t know t h e q u e s t i o n s a b o u t w h i c h t h e y  expert  versus  by h a v i n g  content  prime-time  versus  The  and q u a n t i t a t i v e  audience  f a c e t o f t h a t m o d e l was e v a l u a t e d  students  proposed.  coders  macroanalysis,  analysis.  i n assessing  integrating the  a n a l y s i s was  model c o n t a i n s t h r e e d i m e n s i o n s : coders,  coders  experts were  coders  with the experts.  tended generally  not the  d i d not agree,  variables. trained  coders  Moreover, to  results  in  general  the questions  agree  same both  The  ones amongst  (that on  on  i s , were which  the  themselves  and  iii Table of contents Section  Page  Abstract  i i  L i s t of Tables  iv  L i s t of Figures  v  Introduction  1  Genres  o f TV  research  1  From C o n t e n t t o E f f e c t s  3  Schema T h e o r y  3  Social  4  L e a r n i n g Theory  Constructivist  Theories  5  Content A n a l y s i s  6  Background  6  M e t h o d o l o g i c a l and T h e o r e t i c a l D i m e n s i o n s UBC  Content Analysis Project  10 19  Hypotheses  23  Method  24  Subjects  24  Coding system  25  Programs  29  Procedure  31  Analyses  32  Results  34  A g r e e m e n t Among N a i v e C o d e r s  34  A g r e e m e n t B e t w e e n E x p e r t and N a i v e C o d e r s  35  R e l a t i o n s h i p Between T s c o r e s and R e l i a b i l i t y  36  G e n e r a l i z a t i o n of the Results  37  A c r o s s Naive Coders  Discussion  39  References  47  Appendix A  67  Appendix  B  91  Appendix C  118  Appendix  120  D  iv L i s t of Tables Table  Title  Page  1  V a r i a b l e names a n d t h e i r l o c a t i o n  i n t h e ACS  2  Frequency o f f a i l u r e t o answer q u e s t i o n s  3  S u b j e c t demographic v a r i a b l e s used i n t h e r e g r e s s i o n a n a l y s e s , t h e i r l o c a t i o n i n t h e ACS, a n d how were r e c o d e d  51 52  they 53  4  Program sample  54  5  A g r e e m e n t among n a i v e c o d e r s u s i n g K e n d a l l ' s W  55  6  D i s t r i b u t i o n of Tscores  56  7  Comparison o f Tscores w i t h r e l i a b i l i t y experts  8  Forward stepwise r e g r e s s i o n analyses  scores f o r 57 59  L i s t of Figures Figure  Title  Page  Figure captions 1  Three  60  d i m e n s i o n a l box d e p i c t i n g the analysis  2  of content  61  ( a f t e r r e c o d i n g ) i n t h e ACS a n d a l l 24  shows c o d e d  by t h e n a i v e c o d e r s  62  D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 variables all  4  to  D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r a l l 129 variables  3  d i f f e r i n g approaches  selected  24 shows c o d e d  for analysis  i n t h i s p a p e r and  by t h e n a i v e c o d e r s  D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 variables  selected  for analysis  i n t h i s paper f o r  t h e 2 h a r d news shows o n l y 5  64  D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 variables  selected  for analysis  i n t h i s paper f o r  t h e 10 o t h e r n o n - f i c t i o n shows o n l y 6  63  65  D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 variables  selected  t h e 12 f i c t i o n  for analysis  shows o n l y  i n t h i s paper f o r 66  1 Introduction In In  o n l y a few d e c a d e s  most p l a c e s i n N o r t h  television  America,  h a s become  a n d many  o t h e r s around  world, i t i s d i f f i c u l t  t o f i n d someone who i s n ' t  Arnie  affair  Becker's  problems ubiquity  latest  television  Dorr  has  become  information.,  the  familiar  with  Law, o r M r s . H u x t a b l e ' s  d i s c i p l i n i n g D r . H u x t a b l e on t h e C o s b y show.  entertainment, culture.  on L.A.  ubiquitous.  a  major  politics,  In i t s  purveyor  of  advertising,  and  ( 1 9 8 6 , p . 8) t e l l s u s t h a t :  Television i s i n more than 95% of a l l American households., more common t h a n telephones and indoor toilets. M o s t homes h a v e more t h a n o n e o p e r a t i n g s e t . I n an a v e r a g e r e s i d e n c e , a s e t i s t u r n e d on about seven hours a day. The a v e r a g e f a m i l y member d e v o t e s two a n d one h a l f t o f i v e h o u r s a d a y t o v i e w i n g . A t h i g h s c h o o l g r a d u a t i o n , A m e r i c a n c h i l d r e n w i l l h a v e s p e n t more t i m e i n front of the t e l e v i s i o n s e t than i n a classroom. By t h e t i m e t h e y a r e 6 5 , more t h a n n i n e f u l l y e a r s o f t h e i r l i v e s w i l l have been devoted t o w a t c h i n g t e l e v i s i o n . These  statistics  f o u n d i n Canada  f o r t h e U.S.A.  ( e . g . , W i l l i a m s & Boyes,  If television role  i n s o many  take  away  similar  to  i t i s of interest  i t s viewers.  from  t o know  When p e o p l e w a t c h  the experience?  those  1986).  i s h e r e t o s t a y , and p l a y s s u c h a lives,  medium i n f l u e n c e s they  are very  Although  prominent how  this  TV, w h a t do  this  question  may be s t r a i g h t f o r w a r d , a n s w e r i n g i t i s n o t . G e n r e s o f TV  Research  Whatever t h e i r all  TV  theoretical  researchers  understanding the r o l e this  common  basic  have  and m e t h o d o l o g i c a l essentially  of television  goal  there  assess t h e impact o f t e l e v i s i o n .  the  i n society.  i s disagreement  approaches, same  goal:  In spite about  how  of to  2  There are  t h r e e major approaches t o t e l e v i s i o n  e f f e c t s , u s e s and Effects  gratifications,  research  television's attitudes,  (e.g.,  direct and  and  and  the a n a l y s i s of  Williams, indirect  expectations  1986)  several  content.  deals  influences  via  research:  on  with  behavior,  processes.  Some  indirect effects  occur because t e l e v i s i o n d i s p l a c e s a c t i v i t i e s  which  might  some  otherwise viewers.  effects  due  t y p e has  Of  to  have  greater  television's  been concerned  a s v i o l e n c e , and Uses Wenner, Blumler  how  and  &  Rosengren,  1985,  1985)  The either  often so  focuses  on  described as  being  what p e o p l e  questions  this  research  11)  p.  ask  w a t c h , and  Most  cited  to people,  how  content.  (1974,  researcher  shows;  to  on  at  least  research  research  are  of  this  w i t h p a r t i c u l a r t o p i c c a t e g o r i e s , such  gratifications  addresses  effects  relevance  gratifications  Katz  but  certain  e x p o s u r e a f f e c t s a t t i t u d e s and  Palmgreen, and  had  such  they  as  people  the to  role "ask  of  use  not  of  why  uses  This  do  approach  watch  watch;  & and  what media  people  they  media.  Wenner, the  do w i t h m e d i a " .  TV;  Rosengren,  Palmgreen,  what t y p e s  watch  (e.g.,  how  in  behaviors.  which  when  they  on.  a n a l y s i s of t e l e v i s i o n content  analysis  Paisley,  &  Stone,  1983).  In  both  example,  the  message,  i s assessed.  (e.g.,  1969),  or  the  acts The  of  can  Gerbner,  audience  approaches  number o f  content  be  categorized  Holsti,  research  content  of  v i o l e n c e or  Krippendorf,  (e.g.,  Gunter,  television, the  most  for  salient  main d i f f e r e n c e between t h e s e  approaches i s t h a t i n content  a n a l y s i s the viewers  as  are a  two  small  3  number are a  of  trained  coders,  whereas  l a r g e number o f u n t r a i n e d c o d e r s .  addressed  the  audience  relationship  research,  hypothesized  to  understanding  Both  to  important:  other  Salomon,  this  audience  a  two  social  (Schank  Three &  seem  Abelson,  (e.g.,  been  Taylor 1977),  Dorr,  & and  1986;  1979).  events.  Schema t h e o r y  Schemata  In  effect,  direct attention, e f f i c i e n t way McGraw, and  are  (Schank & A b e l s o n ,  t o e x p l a i n how  self-relevant  are  filters  p e r c e p t i o n s , and  Drabman  different a  male  saw  a  film  a l l  they  (1979)  or  people  groups  of  physician with  children  a in  two  children. and  female the  a  The  female  outcomes  that  thus provide  identical  of  an  Cordua, films  first  group  nurse;  the  second  male  nurse.  p h y s i c i a n and first  beliefs,  For example,  nearly  is  process  stereotypes  memory, and  showed  1977)  attitudes,  of processing information.  with  Later,  on does  have  1977;  e x p e c t a t i o n s r e g a r d i n g t h e c h a r a c t e r i s t i c s and  group  to  particularly  (Bandura,  theories  rely  content  theories  l e a r n i n g theory  constructivist  information.  film  methodology  researchers  different  a p s y c h o l o g i c a l theory developed  two  and  approaches  televised  process.  theory  Schema T h e o r y .  to  research  analysis  evaluated  that  Several  schema 1981),  and  studies  explain  interactive  in particular,  analysts  viewers.  proposed  content  they  effects  from  influence  current  form a b r i d g e between t h e s e  content  evidence  Crocker,  and  between  The  research  television.  From c o n t e n t t o  and  i n audience  group  a  recalled  saw  a  both  4 c h a r a c t e r s c o r r e c t l y , w h e r e a s o n l y 22% so,  and  more t h a n h a l f  a s m a l e and  the nurse  incorrectly as  these  children  processed  their  gender  schemata,  discrepancy  or  female.  i n the second  identified  the  doctor  A c c o r d i n g t o schema t h e o r y ,  i n f o r m a t i o n from  altered  both  group d i d  and  either  their  memory  the  did to  film  not  be  by  way  of  notice  the  consistent with  t h e i r gender schemata. Schemata both in  direct  are  built  experience  real  life)  portrayals).  preexisting  schemata.  unchanged.  unique  less  schema,  discrepancy enough.  Even  not  experience are  the  schema  I f , however,  be  encountered  (e.g.,  processed  media  through  a  similarity  i s upheld  an  event  characteristic,  outcomes  will  nurses  through  to  I f t h e m a t c h b e t w e e n an e v e n t and  familiar  various  and  initially  according to t h e i r  i s good,  relatively  doctors  experiences  procedure  schema  constructed  indirect  Subsequent  the p r e e x i s t i n g  or  (e.g.,  and  c o g n i t i v e matching  or  up  are  or  possible.  noticed;  the  i f i t i s noticed,, the  and  remains  presents  some  contradicts Most  match  the  likely  will  a  be  the good  discrepancy i s l i k e l y  to  be p r o c e s s e d a s an e x c e p t i o n . S o c i a l L e a r n i n g Theory.. influence  of  television  l e a r n i n g or modeling. involves  two  summarizing TV  the  is likely  factors:  to  social  evidence be  content  Bandura  steps,  A second on  viewers  indicating  and  the  i s observational  (1977) p o i n t s o u t t h a t  acquisition  performed  i m p o r t a n t avenue o f  learning  performance.  In  when b e h a v i o r o b s e r v e d  Comstock  (1980)  described  on  four  a p p r o v a l f o r t h e model and/or f o r b e h a v i o r  in  the  filmed  material;  behavior; model's  the  perceived  by  TV.  Just of  as  and  transitory  (e.g.,  also  have  may  within  side  side  (1980)  which  retains  structured  the  same  meanings.  (1979)  television view,  but  symbols  and  less  obvious  stresses  the  active  process. content Two  show To  both  In h i s  meanings,  framework.  the  and  come  simply  believes synthesize  people  and  talk  She  could away  about  sit with "the"  television i s misleading.  in  and  own  watching  different  sign,  manner  their  a  socioeconomic Salomon  c r e a t e a more e l a b o r a t e  interpret  of  including  (e.g.,  surface-level  content  influenced  symbol systems of  viewing  messages  the  outcome i s  i n d i v i d u a l viewers.  actively  whether  television  characteristics.  meaning i n the  adults  Morley  or  the  passively  stable  (1986)  and  m e s s a g e s on  of  viewers,  Dorr  children  and  the  and  characteristics  of  of the  effect. of  completely  viewer  to  construction  by  the  and  aspects of  interact  behavior  of  viewer.  Analysts  of  arousal)  apparent  efficacy  other communication the  interaction  psychological  the  viewer;  characteristics  cognitive  can  the  interaction  the  emphasizes the  TV  any  personality)  the  of  or  t e l e v i s i o n viewers are  for  the  messages  status,  to  Theories.  assume t h a t  result  and  relevance  o p t i m i z e s arousal f o r the  Constructivist not  successfulness  characteristics  portrayal  do  the  which  thus  a the to  argues  that  preferred  meaning  potential  to  communicate  polysemy.  the  Since  be a  TV has  message been  decoded  different  is  a  inscribed, in  a  is  but  different  meaning;  communication  complex  i t is  always  a an  6 interaction  of  the  characteristics unique  to  Perhaps a  of the receiver,  individuals  concern  more  'equally'  with  cannot  the  message..."  reliability  the  Travis,  &  analysis  and  some  is  our  totally  reading'  audience  t h e dominant determine  of  is  content  contention  Wotherspoon,  to  been s t r u c t u r e d be  the i t is  1988).  i s misplaced  analysis  is  and more  not  exist  i n dominance,  fixed  itself  or  'closed'.  part  as a f u n c t i o n  r e s e a r c h e r s have  usefulness  It  establish  and  (Gunter,  " a l l m e a n i n g s do  message p e r c e p t i o n v a r i e s  characteristics,  1988).  predicted  that  never  'preferred  message  of  the  (p. 1 0 ) .  Because  of  can  the  individual  responds  i t s meaning  Further,  be  inter-coder  Morley  of  i t has been argued t h a t  i n t h e message: i t h a s  although  latter  and  phenomenological  appropriate.  view  characteristics  meanings,  who  (e.g.,  that  Phillips,  the  content former  t o use M o r l e y ' s term, them,  Gunter,  both  worthwhile,  perceives  viewer  pessimistic  Williams,  however, are  a  analysis  (e.g.,  1988),  research  taken  of  how,  and  and  under  to the  what  conditions. Content  Analysis  Background. the  University  assess  the  In October  of B r i t i s h  content  across  Canada  program  sample  of  (Williams,  (1089  programs)  from  AM  1:00  The  to  AM.  Columbia  the  consisted  1985,  major  W i l l i a m s and  began a p r o j e c t television  Phillips,  &  networks  networks  over  were t h e  two  15  designed to  networks  Travis,  o f a l l programs  seven  her students at  1985).  minutes  one  received  full  Canadian  The  or longer week,  7:00  government  7  funded the  channels,  private  network, and  CBC  E n g l i s h a n d CBC  Canadian  network,  CTV;  French  (Radio  t h e U.S.  PBS; a n d t h e t h r e e p r i v a t e  U.S.  Canada),  publicly  networks,  funded  ABC, CBS,  NBC. Trained  answered  coders  25 p a g e s  covered  a  watched  o f predetermined  variety  of  countries;  sex,  portrayals;  aggression;  portrayal  of  t h e show  topics  romance,  people;  and  questions.  including  and  immediately  The  ethnic  relationships;  issues, global  then  minorities; gender  c o n t r o v e r s i e s , and impressions;  questions  and  role  dilemmas;  programming  characteristics. The (UBCCS, assess,  University Williams, in  a  of British Phillips,  systematic  Columbia  content  & Travis, and  1985) was  reliable  way,  message o f t h e ( m y t h i c a l ) a v e r a g e v i e w e r . was  to  create  a  messages  recorded  possible  those  purpose which  method by  that  of  trained  coders  viewers  at  analysis  home  would  take-home  i s , the goal i n which  the  as c l o s e l y perceive.  t o assess  to  as The  the extent t o  t h i s g o a l h a s been a c h i e v e d .  measure  t h e same  agree, which  data.  The  be u s i n g t h e same i n s t r u m e n t t o extent  i s t h e focus of t h i s  assessment  of reliability.  this  coders,  study  would  also  occur  t o which  their  answers  s t u d y , c o u l d be c o n s t r u e d a s The  e x t e n t o f agreement between t r a i n e d in  That  system  designed  the  resembled  o f t h e c u r r e n t r e s e a r c h was  Two d i f f e r e n t g r o u p s w i l l  an  content  coding  question and n a i v e  f o r other  a question of generalizability,  will  o f whether t h e coders groups  obtained of  naive  be addressed  in  8 future  research.  system  developed  it  The q u e s t i o n o f t h e v a l i d i t y by W i l l i a m s e t a l . (1985),  accurately assesses  was n o t a d d r e s s e d evidence  that  certain  i n this  a variety  messages  study.  of the coding  that  i s , whether  i n TV c o n t e n t ,  I t would r e q u i r e  of different  also  converging  measurement  approaches  y i e l d e d t h e same m e s s a g e s f o r t h e same c o n t e n t . A  thorough  attempts  literature  t o compare  coding  naive viewers  (Nielson,  Tate,  Lull,  1977).  commercials portrayals  revealed  by e x p e r t  undated;  male  Lull,  sympathy  with  responses  t o t h e commercials  female  t h e women's  minutes t o w r i t e t h e i r  as  college and  were m e a s u r e d .  & Marx, 1977; television  containing  sexist  students.  Their  their  open-ended  They were g i v e n 4  commercial,  b u t were  g i v e n no i n s t r u c t i o n s r e g a r d i n g t h e t y p e o f r e a c t i o n  expected.  The r e s u l t s portrayals although  indicate than  t h a t women w e r e more s e n s i t i v e  were  men.  not especially  females)  positive  of sex r o l e  had  identified  traditional  There  sex r o l e  also  was  a  l a r g e (r=.35 f o r m a l e s ,  correlation  recognition been  t o each  other  and u n t r a i n e d o r  Hanson,  movement  reaction  three  (1977) showed  identified  and  only  raters  Hanson, and Marx  previously to  search  between  stereotyping. by  content  portrayals,  to sexist  significant, a n d r=.46 f o r  feminism  scores  and  A l l o f t h e commercials  analysts b u t fewer  as  containing  than  half  the  s u b j e c t s commented o n t h e s e . In  a  (undated), of  study  of television  f i r s t d i d a content  f o r Danmarks analysis  Radio,  Nielson  of t h e symbolic  t h e f a m i l y a s p o r t r a y e d on t e l e v i s i o n .  world  T h i s was n o t b a s e d  on  values  based  portrayed  on  the  represented programs  in  "total  Audience  First,  survey  a  types  interviews analyses  were and  the  metric.  The  questions  whereas  of  of  the  d i s t o r t i o n of Tate's  1977)  Tate  and  Williams,  taken  possible.  about  was  done  that  use  a  statistical  ways.  in-depth  The  not  with  content the  fixed  interviewed  television  two  representing  families. did  to  content  Second,  so d i r e c t  on  in  households  analysis  audience  the  assessed  30  was  reactions  of  was  study  was  Ontario  compared  Zabrack,  of  was and  an  same  set  at  a  of more  c o m p a r i s o n s were reactions to  were  the  consistent  with  i s , that television  is a  Joy,  were  of  a  the  larger  Commission  In t h i s  body  on  of  a  work  Violence  in  focused  on  work, w h i c h  results  of  systematic,  a n a l y s i s ( W i l l i a m s , Zabrack, average  very  audience.  similar and  some  to  the own  s i t u a t i o n as  collected  during  Tate  the  of  programs i n t h e i r  t o make t h e v i e w i n g Data  Royal  content  those  that  part  Industry.  Respondents watched the was  results  however, t h e audience  reliable  with  questionnaire  their  conducted.  analysis,  the  Communications  detailed,  and  instead  reality.  by  aggression,  Joy,  content  The  audience  families  (1977)  commissioned the  Overall,  sample  were  was  work  the  4) .  1000  audience  level,  not p o s s i b l e .  those  with  content  phenomenological  portrayals  about  done  but  the  perceptions  families  episodes,  types,  with  of  of  (p.  family  compared  analysis.  different  output"  different  were  individual  and  one same  used used  & a by  programs.  homes, and  care  c l o s e t o normal immediately  as  after  watching  a  show,  assessed. time,  so  U.S.  time. the  A  1974  completed  entire scales,  program content, Tate  compared  (Williams  et  the  a l . , 1977)  of  the  audience. notice He  suggested  considered violence  content  content  were  the  i n media  most  They  assessed  assigned  Having messages  by  the  by  liberal while  average audiences, to  analyze  likely,  and  to  the  estimate  given  l e t u s now  by  the the  of  the  likely  trained  of  the  to  coders.  amount  audience  Unfortunately, a  analysts  coders  between  less  accepting the  for  the  i n content analysis  compromised  need  of  program.  perceptions was  124  semantic  trained  score  the  random  his  with  that  those  of data  flaw i n  comparisons  research  be  in  thesis. compares  recorded  t u r n t o t h e d e b a t e o v e r how  by best  content.  Methodological earlier,  out  i n t h e Method s e c t i o n o f t h i s  content  the  those,  aspects  the  mean  when c o m p a r e d  e s t a b l i s h e d the  coded  filled  sample audience  approach  ways d i s c u s s e d i n d e t a i l  of  at the c o n c l u s i o n of the  analysis  content,  statistical  and  various  a s t h e m o s t c o n s e r v a t i v e " ( p . 373) . Tate's  is  were  at  selected at  list,  t h a t t h e messages r e c o r d e d "as  It  considerable disparity  In general, the  specific  people  with  found  where,  available.  interview.  scores  He  i n Saskatoon  voters  which  perceptions  programs were b e i n g seen f o r t h e  315  immediately  sample audience. results  of  provincial  the  differential  was  (U.S.) TV  total  immediate  done  television  t h e r e f o r e , t h a t the  from  viewers'  T h i s r e s e a r c h was  no  first  the  various  and  theoretical  theoretical  and  dimensions. methodological  As  stated  approaches  have  been  analysis  used  t o assess  o f TV c o n t e n t  t h e impact  o f TV  i s , in itself,  insufficient  the e n t i r e process, but i t i s a necessary (Sepstrup,  1981).  necessity  of content  best t o proceed. the  should  help  various  i s disagreement about  how  the  and  a p p r o a c h e s t o t h e a n a l y s i s o f TV c o n t e n t .  This  researchers and this  content,  thus  manifold  more  easily  make  more  understand informed  these  research  d i s c u s s i o n i s based on t h e a n a l y s i s  i t i s intended  to the analysis of  t o be a p p l i c a b l e t o a l l  a l l types  of content,  e.g.,  text.  The along  of, the  theoretical  Although  approaches radio,  among  approaches  of t e l e v i s i o n  because  process  The g o a l o f t h e f o l l o w i n g m o d e l i s t o e x p l a i n  other  decisions.  The  to explain  part of that  o f , o r perhaps  analysis there  distinctions  methodological  1) .  In spite  on s o c i e t y .  analysis  of  content  c a n be  understood  s e v e r a l axes w i t h i n a t h r e e dimensional Any g i v e n  piece  of research  box  c a n be p l a c e d  as  varying  (see Figure  a t some  point  w i t h i n t h i s m e t h o d o l o g i c a l box.  I n s e r t F i g u r e 1 about here  The versus analysis content set  of  first, audience  and  simplest,  research.  of content  dimension These  are defined  two  by who  a n a l y s i s a s m a l l number o f c o d e r s rules  attentively  view  and  definitions.  and i n t e n s i v e l y  i s content approaches  does  analysis to  the  the coding.  In  a r e t r a i n e d t o use a  These evaluate  experts specific  usually material.  12 Audience  researchers,  material, show,  for  but  coding  example,  in  viewers.  this  These system  definitions  on  the  a  i t is  viewers  and  they  rules  hand,  particular  case  and  other  not  do  not  approach i s not  analysis,  but  (e.g., L u l l determine of  use  world  i t seems on  what o f t h e research  the  use  systematic  surface  The  goal of content  be  example o f a u d i e n c e  research  what  any  set  of  than  their  content  more s u b j e c t i v e analysis i s to  i s portrayed.  determine  of  interpretation.  the  i s to  a  of  instead,  individual  p o s s i b l e world  number  particular  to  of  of  The  this  goal  portrayed  i s perceived. An  Jensen  (1987),  programming.  who  His  assessed  a polling  firm to "procure  He  contacted  them i n i t i a l l y  watch p a r t i c u l a r scheduled people  shows  times.  He  remembered interviewed  about on  the  knew t h e y . were g o i n g  to  and  own  homes a t  news  in  asked  how,  and  study  example of content  by  C o n d r y , B e n c e , and  content  (e.g.,  analysis.  commercials)  They of  Scheibe  8).  The  much,  subjects  day.  about  the  They shows,  be.  the  morning  to  regularly  (1987) p r o v i d e s  evaluated  Saturday  (p.  how  following  questions  news  f o r them  their  programming. the  of  sample s e l e c t e d  b u t t h e y d i d n o t know w h a t t h e s e q u e s t i o n s w o u l d A  work  for  made a r r a n g e m e n t s  telephone be  memory  non-random  interested TV  i n the  a range of respondents"  in their  was  i s found  people's  s u b j e c t s were a  by  were  in  specific  episode  large  content;  necessarily less  use  or  a  a  to evaluate the  e t a l . 1977).  audience  to  trained  a n s w e r s a r e b a s e d more on t h e i r own This  series  shown  are  also  an  non-program programming.  13 In  this  research,  established which CBS,  was and  and  at  89%,  taken NBC.  two  evaluated  from  The  the  categorized  looked  time  of  with  a l l programs  t h r e e U.S.  into  informational  station,  coders,  in  private  reliability the  sample,  networks,  d u r a t i o n o f n o n - p r o g r a m c o n t e n t was  announcements, program and  trained  commercials,  p r o m o t i o n s and  drop-ins. day,  specifically  station  Differences  and  month.  at the  public  In  non-program  a  ABC,  timed, service  identifications,  were  examined  second  study  content of  by they  children's  programming a t t i m e s o t h e r than Saturday morning. It  seems c l e a r  f r o m t h e s e two  analysts  and  audience  messages  on  television,  somewhat  different.  because on  within  researchers but  These  each  examples are  their  concerned  methods  differences  approach  that both content  and  become  The  following  the  goals  are  complicated  t h e r e i s not unanimous  t h e a p p r o p r i a t e methodology.  with  two  agreement dimensions  h e l p t o e x p l a i n some o f t h e d i f f e r e n c e s . The  second  dimension  in  quantitative versus q u a l i t a t i v e  the  proposed  analysis.  model  is  This dimension  has  b e e n a s o u r c e o f d i s s e n s i o n among r e s e a r c h e r s , and t h e r e i s no shortage  of  (Sepstrup, between  opinion 1981).  the  two  about It  terms,  which  is  not  but  a  route always  clear  i s more  appropriate  easy  distinguish  to  understanding  meaning i s e s s e n t i a l t o u n d e r s t a n d t h e proposed Understanding hindered  in  meanings  are  part  the  terms  because  different.  quantitative  their In  and  colloquial  general  of  their  model. qualitative and  parlance,  is  statistical quantitative  14 refers  to  counting  some phenomenon. d e t e r m i n e how qualitative portrayed  and  qualitative  F o r example,  as  might  acceptable,  or  however,  differ  nature of the  i n the  r e f e r s t o whether  refers  (Kirk  whether  both  to  whether  what  phenomena  degree  1986).  w o u l d now  not  the  is  quantitative  themselves  that  the  or r a t i o  whatever  (e.g., H o l s t i ,  solution  method  occasion.  statistical  best  to  of  the  approach  suits  of  these  in  that  quantitative  of violence  of  data  1983).  1969)  this  This conclusion  consideration  they  i s present.  quantitative  (Kennedy,  quantitative versus q u a l i t a t i v e analysis  on  A  a  Qualitative  presence  Q u a l i t a t i v e d a t a a r e n o m i n a l and interval,  To  counting;  present.  a was  be c o n s i d e r e d q u a l i t a t i v e ,  violence  A number o f t h e o r i s t s  particular  to  counted.  of  The  whereas  symbolized.  refer  of  might  violence  approach might measure t h e l e v e l  c o u l d be o r d i n a l ,  choose  the  i t  words  the  & Miller,  or  (statistical) each a c t .  nature  analysis  o r n o t some c a t e g o r y o f " t h i n g "  t h e p r e v i o u s example is,  the  quantitative  ask  statistician,  "thing"  to  many a c t s o f v i o l e n c e w e r e p o r t r a y e d , approach  Quantitative  a  refers  have  satisfied  controversy  over  i s for researchers their  purpose  i s based, terms.  on  however,  a  only  Proponents  t h i s v i e w do n o t a d d r e s s t h e more c o n t e n t i o u s i s s u e o f  to  of  whether  t o c o u n t o r t a l k a b o u t t h e n a t u r e o f some phenomenon. In and  their  Dominick  quantitative evaluated  content analysis (1984) and  whether  used  a  of  television,  combination  qualitative each  music  video  approaches. was  a  of  Sherman  statistically Coders  concept  first (story,  dramatization,  or  concert)  piece.  according  to  whether data  or  sex,  not  about  narrative)  or  Individual  characters  age,  economic  provocative  sexual  performance  (studio  were  categorized  status, ethnic  identity,  costumes were worn.  intimacy  fell  on  an  or  and  Quantitative  ordinal  scale  from  f l i r t i n g to intimate touching. S t e e v e s and gender  Smith  content  of  prime-time  perspective.  Their  phenomenological  sense.  at  least  They  twice  also  yielded  and  TV  approach Both  discussed  watched  episodes.  (1987) d i d an  later  from  a  was  qualitative  authors  episodes  c a t e g o r i e s of The  about  the  content  dimension  of  macroanalytic.  differentiated  by  the  type  at  which  the  both level  of  These  they  do  incidents  and  individual  in a  group  program  characters. or  the  die  about  the  those  deaths.  the  model  is  approaches  are  which  focusing  example, the  of  times  counted.  impression  a macroscopic given  as  a  they  Microscopic  a A  particular macroscopic  a n a l y s t might result  on  number  For example, i n s t e a d of c o u n t i n g  i n a program  general  so.  detailed,  For  number  i s r e f e r r e d t o m i g h t be  a p p r o a c h i s more g l o b a l . many p e o p l e  to  phenomena w i t h  as t h e p h r a s e i m p l i e s , i s v e r y  ethnic  gender. previous  relation  proposed  analysis,  deaths  together  data; rather, i t  in  the  versus  of  shows  the  interest.  final  and  in  researched  numerical  and  feminist  a b o u t c l a s s and and  microanalytic  deal,  socialist  watched  evidence  T h i s a p p r o a c h y i e l d e d no impressions  a n a l y s i s of the c l a s s  of  how ask  viewing  The  research  macroanalytic coders  and  understanding  an  episode  and  more  Smith  i t was  than  (1987)  once,  T h i s gave  of  series.  the  not of i n d i v i d u a l  and  series.  content of  later  them a  global  analysis  b u t o f t h e show  i n f l u e n c e d by  o f t h e show was  The  watched  Their  c h a r a c t e r s and s e g m e n t s ,  They were c l e a r l y  analysis  the  was  qualitative.  r e s e a r c h e d p a s t ones.  was  but t h e i r  Steeves  a t t h e same t i m e t h a t  watched  episodes,  by  individual  a t a more g l o b a l  or  bits, macro  level. Surlin, of  TV  in  Romanow, and  Soderlund's  which  compared  p r o g r a m m i n g was feature  was  content, rating,  they  done a t a m i c r o  coded  and and  reliabilities  minute d e t a i l  in this  (1977,  stands  1982)  (1987).  Neither  s t u d y and in clear  approach  and  Each  geographic  Three were  content  Canadian  level.  for duration,  presentation.  (1984)  contrast  to  i s necessarily  them r e v o l v e ,  did  the  emphasis  Steeves better  quantitative  content r e s e a r c h e r can  deaths  i n any  average  validity.  t o some d e g r e e ,  and  reliably  coders  of  on  i n t h e work o f W i l l i a m s e t a l .  reliability  be  and  type  and  in  t o be  Smith  general.  understood  axes, q u a n t i t y / q u a l i t y and m i c r o / m a c r o ,  debates around  d e a t h s an  news  research.  T h e s e two  occur  origin,  The  R a t h e r , each has unique a t t r i b u t e s w h i c h need in designing  U.S.  news s t o r y  different  established.  analysis  g i v e n TV  For  reliably  show o r s e r i e s .  1 4 - y e a r - o l d has  calculated.  example,  Armed w i t h  seen this  on  on a  the  and  the  issues  of  microanalytic  record The  how  many  number  of  television  can  then  information,  and  that  17 derived  from  statements  research,  more  consider  macroanalytic  what  qualitative  fault  meaning  this  the  experience  (Newcomb, 1 9 7 8 ) .  have t a k e n  place?  microanalytic available,  c a n make  m i g h t be  changed  content  researcher,  enterprise f o r i t s failure to  viewer  constructs  out  of  this  How many d e a t h s a r e p e r c e i v e d t o  W i t h o u t k n o w i n g more a b o u t t h e v i e w e r , t h e  q u a n t i t a t i v e researcher  but cannot  perceived.  reliable  analyst  TV.  however, can r i g h t l y  are  the content  a b o u t how t h e a v e r a g e 1 4 - y e a r - o l d  by w a t c h i n g A  effects  The  statement,  assess  can assess  whether  empirical  the potential  approach  but the v a l i d i t y  t h e messages  may  messages  yield  o f i t would  a  very  be a n  open  question. The  opposite  extreme  to  the  e q u a l l y flawed, b u t i n t h i s case reliability. qualitative television potentially  As e x p l a i n e d orientation  as i r r e l e v a n t infinite.  there  than  would  also  probably  t h e show c o n t e n t g u i d e s is  to  ignore  information  the  influences  society.  researcher  i s right  be over  macroanalytic  the a c t u a l content  i s no d o u b t  will  that  be  much  of  outcomes.  to The  reliably  understanding extreme  will  be  (1980) p o i n t s  agreement,  much  To some  t h e p e r c e i v e d message. to  there  b u t as Morley  w i t h random  ability  essential  would  because t h e c o n s t r u c t e d meanings a r e  There  be e x p e c t e d  an extreme  consider  much d i s a g r e e m e n t on t h e c o n t e n t , out,  example  f o r emphasis on v a l i d i t y  earlier,  would  above  degree  To i g n o r e  know how  qualitative  i n considering the viewer's  some  more  of  this the  television macroanalytic perspective,  18 but  shared  experience  t h a t can help t o d e f i n e t h a t  s h o u l d n o t be i g n o r e d . television  content  researcher  This  may  (1987).  feminist  may  have  American viewer's Obviously, researchers'  t h e case  and  with  c a n be  Thorp,  &  and  TV  content  European  researchers,  can  and  from. M a r x i s t ,  be  and  are  not  research  from  a  thus  on  and  Smith,  lengths  coded.  to  Because  however,  their  t h e average  a t one  North  imply  the  Critical  in  the  based  U.S.  on  has  a  hand,  i n their  s t r a d d l e both groups. as  incidents  One  i n those  other  tend  approach,  backgrounds. pole  Budd,  Holsti,  and  1969; way  to  characters. to  be  and  more often  Canadian  o f t h e model  quantitative,  been  published  i t solely  i s no  solve  positivist  O g i l v i e , . 1966;  there  Theory  of the  orientations to  &  other  Most  corner  e t a l . , 1969;  t o -count  qualitative  characterized  and  Marxist  orthogonal.  analysis define  Gerbner  but  i n the  Steeves  Most E n g l i s h language books  1967;  approach  researchers  of  structure,  axes  Dunphy,  1980),  work  validity  p e r c e p t i o n o f t h e same m a t e r i a l .  quantitative,  Krippendorf,  macroanalytic  considerable  i n common w i t h  Most  Stone,  Donohew,  macroanalytic  to  characterized  A m e r i c a on c o n t e n t  (e.g.,  lose  shows  not t o use d i f f e r e n t  problems.  terms  and  the  deep  three  scientific orientation. in North  went  little  these  work  microanalytic  qualitative  i n t h e work  interpreted  conscious  They t e n d  different  the  messages i n t h e m a t e r i a l t h e y  concerned  conclusions  f o r a more v a l i d m e a s u r e o f  reliability  perspective,  were  model.  lead  be  They  understand these they  may  to sacrifice  process. Smith  The s e a r c h  perspective  thus  microanalytic,  19 apolitical,  U.S., c o n s e r v a t i v e , a n d e m p i r i c a l ;  qualitative, 1981;  m a c r o a n a l y t i c , European,  Sepstrup,  the other,  and M a r x i s t  as  (Rosengren,  1981).  UBC C o n t e n t A n a l y s i s P r o j e c t The  goal  of television  content  analysis  i s t o describe  t h e m e s s a g e s p o r t r a y e d o n TV a n d u s e t h e f i n d i n g s t o make a n i n f o r m e d In  statement  d e s i g n i n g t h e UBC  al.  coding  system,  (1985) was t o s t r a d d l e  described. coding  system  viewer, extremes the  That  i s , they  about p o t e n t i a l  attempted  was  a n d a t t h e same of both  time  of Williams et  of the dimensions  t o capture  to avoid  in a  t o address  raised  dimension.  earlier many  (1985),  Ettema,  average of the  dimension In  Miller,  and  effect,  the coding  of the concerns  b y Cook, C u r t i n ,  just  reliable  the p i t f a l l s  the quantitative/qualitative  i t was d e v e l o p e d  designed  analysis  effects.  t h e t a k e - h o m e message o f t h e ( m y t h i c a l )  microanalytic/macroanalytic  although  viewer  the goal  a l l three  and t h e o r y  about  system content  a n d V a n Camp  (1986). A typical times been  t o ensure recorded  message  had  contrast, in  content  any  been  did,  picked  up  t h e UBCCS s p e c i f i e d  once. time  t h a t every  (microanalytic  i t s entirety  only  either  a n a l y s t would  before  fast  t h a t coders  many  every  subtle  approach).  watch  a TV  By  program  any q u e s t i o n s , a n d w a t c h i t t o stop  nor t o review  forward  or that  (macroanalytic  They w e r e n o t a l l o w e d  however,  the content  c h a r a c t e r and i n c i d e n t had  approach)  answering  during viewing,  review  through  the video  tape a t  any s e c t i o n s .  t h e commercials,  as  (They these  20  were n o t b e i n g unit the  coded.)  of analysis; coder  These  individual  h a s , somewhat  restrictions  make  t h e show t h e  c h a r a c t e r s were n o t coded.  like  a  normal  viewer,  from  t h e average  is a  What global  i m p r e s s i o n o f t h e show. One m a j o r  departure  was t h a t c o d e r s however,  and  mentioned,  took notes were  during coding.  restricted  t h e main  to  characters  a  viewing  experience  The n o t e s  were few,  of' the  countries  list  and t h e i r  ethnicity,  c o n t e n t o f i n d i v i d u a l program segments.  These notes  memory  sheet.  cues  for filling  differs  from  the  believed  t o be  out the coding  average  necessary  viewing  experience,  to establish  and t h e served  as  This  clearly  but  i t was  reliability  among t h e  coders. Williams' and  team  fourth-year  quite  o f 20 t r a i n e d  university  heterogeneous,  so  on.  In spite  because  of  establish  their  good r e l i a b i l i t y  despite  attempts  T h i s m i g h t a l s o be t h e c a s e questions  or portrayal of  others  were  were  from  various  people  otherwise ethnic  were  reliability  and  i n part able  to  t h a t were was  poor  t o c o d e shows t h e same way.  f o rnaive  English  third-  (SES) l e v e l s ,  coders  f o r which  viewers.  s y s t e m w e r e many a n d v a r i e d .  qualitative:  statistically  They  status  these  i n the coding  Some w e r e s t a t i s t i c a l l y  mostly  on most o f t h e q u e s t i o n s  some i t e m s  to. t r a i n  were  d i f f e r e n c e s , and perhaps  similarities,  There were  to  coming  socioeconomic  of their  coded.  The  students.  however,  backgrounds, d i f f e r e n t  coders  Canada  "Was or  quantitative:  there  any r e f e r e n c e  Canadians?",  whereas  " I f yes  (to the  previous have?"  The  passing A,  p.  question)  how  much  coder then  reference,  had  minor  the  characters  were  asked  while  program's  option or  of  from  were coded. or.  not  to  keep  track  show.  w e r e members o f  a l l ethnic  person  was  of  an  groups  ethnic  a r e s u l t o f t h e i r e t h n i c i t y ; how  general  impression  mentioned, that This  series  of  is, a  of  the  as  is  achieves.all  the  microanalytic quantitative  also  between content research claim,  was  that  that  more  the  the  the  a n a l y s i s and  to  whether in  whether t h i s  was  identified impression  still  somewhere  issues yield  a  minorities  qualitative  evaluation. between  the  examinations.  (Williams  that  audience  specifically  i s , t o what e x t e n t  sex;  ethnic  et  i t  that  evaluate  have W i l l i a m s  and  1985)  occasion  i t is  forms  research. to  al. ,  i t i s on  macroanalytic, and  group  well  answers  of  were  referred  overall  outlined: that  often  or  Although these  falls  UBCCS  qualitative,  designed  final  macroscopic  objectives  and and  nature  questions  hoped  the  way  major  individual  functioning  i f not  (nonstatistical)  e x t r e m e s o f b o t h m i c r o and It  Prominent  or negative).  numerical  the  s t r o n g l y t h e y were  w i t h t h e i r e t h n i c b a c k g r o u n d ; and  in a  Appendix  minority  ethnicity;  m a i n s t r e a m N o r t h A m e r i c a n s o c i e t y and  were approached  [see  a l l of  portrayed  portrayed  ( p o s i t i v e , mixed/neutral,  focus  i t  either-  I f these characters  Data about each i n c l u d e d :  the  did  r a t i n g i t as  major  t h e y w e r e c o d e d i n a more d e t a i l e d way. characters  focus  (b)].  watching the  N o r t h A m e r i c a n s who  the  focus,  1, numbers 1 (a) and Coders  of  a  The the her  both bridge  current latter students  22  been  successful  in  capturing  in  messages t h a t u n t r a i n e d v i e w e r s The sense  hypotheses  and  Biberian  on  described  audience  (1978) .  group of about  panel  dimensions.  Twenty  shows  one  was  were f i c t i o n  TV,  One  of  Himmelweit  consistency  were  level,  of  et  the  this  consistent  Himmelweit e t a l . but of viewer  coders  evaluations. more  often  a  group  seem  some  (1978)  comedies.  Instead,  had  a  Broadcasting a  number  t h a t time  a l l were  of  viewing  a news show,  and  spent  related  r a t i n g s would  audience  as  vary  Their results  shows t h a n  to  scores  were  f o r less popular a  stimulus  of  shows  example,  as  not more  ones.  variable  t o be  a  by  a measure  That i s , the b e t t e r the  situation  results  that  did  show, t h e more c o n s i s t e n t w e r e  News For  and  p r e d i c t i o n s was  or genre.  watches  the  common  and  peak  a l s o g e n e r a l i z e t o genre,  counterintuitive.  situation  from  was  SES  rate a l l situation  ways,  same  Swift,  British  i t seems more c o r r e c t l y  T h i s may  both  critic  They found  identified  knew t h e  s i m i l a r l y t h e y may In  one  f a m i l i a r i t y w i t h a program.  group of  on  p r o g r a m s on  and  al.'s  f o r more p o p u l a r  p o p u l a r i t y was  as  selected  audience's  hypothesis.  the  seen.  f u n c t i o n of program category support  based  audience  entertainment,  educational  manner  Himmelweit,  evaluate  p e r c e p t i o n s o f the programs  Program  by  a c u r r e n t a f f a i r s program.  watching  the  are  v o l u n t e e r s from the  viewing  18  below  research  Corporation's  time;  reliable  perceive?  They c a s t t h e  1,000  a  their  that i s , the  comedies,  t h e , more  comedies. Himmelweit are  they  very have  et  al.  different more  (1978) from  individual  segments, and  contain  primarily  information. consistent news  contain It  1985)  has  not  respond  large  contain  n o t have a  to  than  euphoric  viewers  would  be  reason  some  that  program  both  nearly not  s e c t i o n on  trained  support  Likewise,  aggression.  violence  and  This  and  untrained  i s an  f o r the  In  aggression  this  section  physical  case  to  findings  Himmelweit  to affect  to  TV.  floor  aggression,  might  be  at  the l e v e l et  the the  a l . , program  these  data. deal  given  ceiling  and  be  design,  a great  o f agreement.  the consistency  therefore  code  scores  but  agreement  of the research  reliably  a l l of  might  inflate  expected  are  Level of  coders  artifact  ability  artificially of  might  S i t u a t i o n comedies  c r i m e d e t e c t i v e shows o f t e n c o n t a i n  violence.  be on  viewers  than  F o r e x a m p l e , t h e UBCCS ( W i l l i a m s e t a l . ,  much  perfect.  more  comedies  categories  contain a great deal of verbal aggression. among  plot,  rather  that  Another  effects. a  likely  do  i n t e r p r e t a t i o n of s i t u a t i o n  programs.  and c e i l i n g  dysphoric  seems  in their  differentially  do  much more i n f o r m a t i o n ,  in  of the  similarly  Contrary  category  to the  might  o f t h e messages  be  perceived  Hypotheses In  the  previously  current  research,  expert-coded  using  d i d not  the  viewers  who  watched  t h e program.  questions their  the naive  evaluations  see The  coders  a the  subset UBCCS  questions  major  of  the  was  shown  to  until  after  they  p r e d i c t i o n was  would  of the content,  agree amongst and  programs  that  that  naive  f o r most  themselves  their  had  in  evaluations  would  agree  question  with  i t was  would tend  experts'.  expected  t o answer  was e x p e c t e d agreement  the  that  That  the subjects  i n t h e same manner.  t o be good on a v e r a g e ,  among  i s , f o r any  naive  viewers,  and t h e e x p e r t s  Although  agreement  i t a l s o was e x p e c t e d  between  them  and t h e  c o d e r s , a n d among t h e e x p e r t c o d e r s  ( r e l i a b i l i t y ) , would  with  items  and  agreement  f o r others,  questions coders to  f o r some  individual  low.  on w h i c h  I t was  being  further  low agreement would  w o u l d b e t h e same a s t h o s e  only  predicted  be  found  separate  regarding  but  the role  be  low,  vary,  moderate that  on w h i c h t h e e x p e r t s  dependent  of viewer  i t was  (demographic agreement  predictions  characteristics.  expected  information  and  agreement.  be  That  that  the naive  tended  were  etc.)  helpful i s , some  in  would  more s i m i l a r .  agreement between expected  predict level  predict  of  naive  hand,  naive  and e x p e r t s  that  subject  level  of  sources  of  coders  might  whereas others  On t h e o t h e r coders  turned out  characteristics  understanding  subgroups  made  I f the general  subject  disagree s t r o n g l y with the expert coders,  not  expert  among  l e v e l o f agreement between e x p e r t s and n a i v e c o d e r s  be  that  disagree. Two  to  given  i f the general  characteristics  level of  was h i g h , would  might  i t was reliably  o f agreement. Method  Subjects Volunteer undergraduate  participants students  were  (120  240  male,  third120  and  fourth-year  female)  at  the  25 University  of B r i t i s h  participation, the  first  third  name d r a w n  receiving  University  were  ethnic  were  assigned  to  groups  qualifications: each  group,  according  naive  10  there  were  and w i t h i n  each  to  both  of ethnicity:  white  subjects);  visible  minority  not v i s i b l e for  ethnic  example,  raised  i n North  (Blishen,  &  parent's  occupation  beverage  serving  of  f o r t h e study were  randomly  the  following females i n  were  stratified  There  American  were  (15%).  Moore,  ethnic  The  1987) w e r e  and ranged  i n North (15%);  i n North America, minority  subjects' based  on  o f 101.74  and n o t  SES  scores  the higher  f r o m a l o w o f 2 3.31  occupations) t o a high  four  (60% o f the  and r a i s e d  and n o t r a i s e d  (10%) ; v i s i b l e  America  Carrol,  distributions  and f i v e  born  i n SES  Canadian born o f Japanese descent  minority  Finnish  North  this  of  with  SES.  the  range  Volunteers  and  keep  wide  volunteered  males  categories  America, f o r example,  a  sex t h e groups  ethnicity  ethnic  from  p e r program five  to  coders v a r i e d  The e m p i r i c a l  t h e groups.  of  because  important  coders  with  $75, and t h e  chosen  as t h e expert  among t h o s e who  to stratify  into a lottery,  were  i t was  solicited.  and e t h n i c i t y  were used  Just  background,  backgrounds SES  so  t o encourage  $100, t h e s e c o n d  students  students,  dimension constant. and  As an i n c e n t i v e  a l l 240 names w e r e e n t e r e d  $50.  experts  Columbia.  ( f o o d and (dentists),  w i t h a mean s c o r e o f 57.26 a n d s t a n d a r d d e v i a t i o n o f 19.00. Coding  System  The subset  Audience  Coding  of the questions  Sheet  (ACS, A p p e n d i x  from t h e o r i g i n a l  A) c o n t a i n e d  UBCCS  a  (Williams et  al.,  1985, A p p e n d i x  section  of  several  factors.  B) .  t h e UBCCS.  Questions  were  Inclusion  i n t h e ACS  First,  i f items  i n c l u d e d from  rarely  every  depended  o r never  on  occurred  d u r i n g t h e e n t i r e week o f p r o g r a m m i n g t h e y w e r e n o t i n c l u d e d . For  example,  relationships, frequency  a  large  section  was  not  included  of their  portrayal.  heterosexual  relationships  format  question  of  a  untrained  coder  included  o r was  tables by  to  trained  coders  instruction.  format  Third, in  into  then  f o r an  the question  c o d i n g system  which  n o t be  either  there  were  used  without  easily  Care  not  several coding  extensive  were n o t i n c l u d e d . however,  Some into  by u n t r a i n e d c o d e r s . t h a t some q u e s t i o n s  These q u e s t i o n s were  format.  was  facilitated  o f t h e ACS i n d i c a t e d  a more u n d e r s t a n d a b l e  concerning  difficult  F o r example,  were ambiguous.  low  i t very  i n t h e t a b l e s were reworded,  testing  very  i f the  t h a t c o u l d be u n d e r s t o o d  t h e UBCCS  the  Second,  The m a j o r i t y o f t h e s e  pilot  of  homosexual  included.  make  but could  questions addressed a  were  reworded.  i n the original  because  on  Analogous q u e s t i o n s  would  use  concerning  was t a k e n  rephrased t o change  o n l y t h e f o r m a t , n o t t h e meaning o f any q u e s t i o n . Data and  were c o l l e c t e d  a subset  the basic  coders  agree  this  will  subset  i n t h e ACS  o f v a r i a b l e s was c h o s e n f o r a n a l y s e s d e s i g n e d t o  answer  articles  f o r a l l o f t h e 129 i t e m s  question  posed  with  the  experts?  focus  on t h e e x t e n t  i n.t h i s  thesis:  Subsequent t o which  do  analyses  naive and  the findings f o r  c a n be g e n e r a l i z e d t o o t h e r t y p e s  of items.  The  27 variables  selected  f o r analysis  shorthand  i d e n t i f y i n g name a n d t h e i r l o c a t i o n  Insert Table  Several c r i t e r i a First,  both  included.  were used  are l i s t e d  1 about  and  of  the naive  Table 2 l i s t s  coders  option.  d i d n o t answer  The a s t u t e r e a d e r w i l l  before  they  were  analysed.  no  Because  people this  option  does  not  of missing data f o r  PAUTSX,  5-point unsure  a quasi-continuum, scale  r a n g i n g from  (3) t o d e f i n i t e l y  and  naive)  (2)  was  were  t h e coder  f i t on  t o be used  absolutely  i f they  (3) was t o be c h e c k e d were d e s i g n e d  check  Other  Coders  t o t r y t o check  s u r e , w i t h t h e same  would  not present  (5) .  thought  and  NA.  of the questions  These q u e s t i o n s have a  definitely present  recoded  PFOCMX,  t h e continuum  e . g . , PAG.  instructed  i n Table  F o r example, i f t h e r e  q u e s t i o n i t was e l i m i n a t e d f r o m t h e a n a l y s e s . formed  a l l  the questions.  i s indicated  PAUTETH,  i n t h e program,  leaving  Nevertheless,  o f t h e q u e s t i o n s were  PMOCMX, h a v e a "Not A p p l i c a b l e " o p t i o n . were  were  n o t i c e t h a t f o r some o f  t h e r e s u l t s more d a t a w e r e m i s s i n g t h a n T h i s o c c u r r e d b e c a u s e some  questions  a l lof  t h e frequency and percentage  each v a r i a b l e .  2.  subset o f v a r i a b l e s .  f o r a l l of the items;  t h e q u e s t i o n b l a n k was n o t a v a l i d  a  i n t h e ACS.  evaluative  Answers were e x p e c t e d  1 with  here  to select this  categorical  i n Table  i t was  instructions  only  (both  trained  (1) o r ( 5 ) ;  (1) b u t w e r e n o t f o r (4) a n d  o n l y i f t h e y were g u e s s i n g .  on t h e a d v i c e o f Huesmann  (1) t h r o u g h  These  ( 1 9 8 5 ) , who f o u n d  (5) ; items that  28 this  format  analyses  yielded  d o n e on  the  UBCCS w i l l  be  they  intended  were  agreement same p r o g r a m  analyzed  presence versus  with  and  to  be  sample.  published used,  absence, w i t h "not  detailed The  i n the  on  a  microscopic  data  from  manner  in  two-point  the  which  scale  sure" recoded to  of  missing.  I n s e r t Table 2 about here  The was  to  ACS.  final  attempt to There  question the from  sexes;  At  5  the  2  on  on  end  of  the  familiarity  variables. 7,  in  u s e d , and  ethnics;  for  analyses  from a l l s e c t i o n s of  from  Part  8  from P a r t  Issues,  demographic  studying,  used  Part  questions  items  1  on  questions  4 on  countries;  from  aggression;  controversies  and  the  Part 4  3  1 on  questions  dilemmas;  and  5  the  coders  f r o m P a r t 6 on g l o b a l i s s u e s .  provided  section  no  4 questions  Part  f o r s e l e c t i o n of  include questions  are  from  questions  SES  criterion  pp.  the any  (ACS)  coding  information  sheet  about  with  the  program  These items  are  listed  20-21.  current recoding  Again,, not analyses. t h a t was  TV seen,  at the  a l l the Table  3  data lists  done.  I n s e r t Table 3 about here  naive  use,  hours  ethnicity, end  of the  of and ACS,  c o l l e c t e d were the  variables  29 Programs The et  al.,  sample 1988)  o f programs  coded  by W i l l i a m s  (e.g.,  i s an e n t i r e week o f t e l e v i s i o n  Williams  v i d e o t a p e d from  each o f t h e s e v e n m a j o r n e t w o r k s r e c e i v e d a c r o s s Canada e a r l y O c t o b e r , 1985.  D a t a on t h e p o p u l a r i t y o f t h e s e  i n t h e G r e a t e r Vancouver a r e a were o b t a i n e d B r o a d c a s t Measurement mid-October  (BBM).  to  the  end  six-week p e r i o d  was  specified  they  d i d not  sampled. not  know w h i c h  The  view  have  each  of  study.  They were c h o s e n  subset  of  of popularity.  add an  weeks  audience  This  of  particular  i n advance,  i n that  Williams  possible  1089  period  was  24  two  but  would  be  intentionally  have  programs  Inclusion  least  was  selected  research,  because  small  none  p o p u l a r e n o u g h t o meet t h e c r i t e r i a .  but t h i s  the of  this  c h a n n e l s on  French  the  Second,  s o few F r e n c h s p e a k i n g s t u d e n t s a t UBC  10  for  o f t h e F r e n c h programs  First, i s so  at  i n the videotaped  from t h e s i x E n g l i s h  reasons.  i n Vancouver  to  programs  important dimension t o t h i s  for  1985.  t o the networks  been the  so  done  a  not  sample,  basis  from t h e Bureau  sampled t h r e e weeks from  November,  three  programs  sample.  would  subjects  BBM  week v i d e o t a p e d b y  i n t h e BBM It  of  The  during  the  would  was  not  viewing  programs  was  since there  r e c r u i t m e n t would  are have  been a p r o b l e m . With from  exception  prime-time, defined  Sunday, time  the  of as  news, 7:00  when t h e a u d i e n c e i s on  programs  i n Williams'  a l l programs  were  pm  pm,  to  average  sample  have  11:00  largest. been  selected Monday  The  to  prime-  categorized  into  30 one  of  six  program  sub-types,  developed  by  Aletha  colleagues  at the University  the  Influence of  six  program  drama. the  The  last  caveat  only  even  one  used  the  news  per  chosen.  The  t h i s s t u d y i s shown i n T a b l e  three  purposes  of  shown  i n the  categories. is  programs were fiction.  on The  non-fiction other  i n each  study,  program  with  of the  series  was  same week  i n that category.  category  the  only,  The  of  In  was the  the  four  sample,  f o u r most p o p u l a r  that only 24  one  programs  show  used  in  here  analyses  the  shows  were  f o r o t h e r s t h e y were grouped  versus  divided  most  4.  primary- c o n c e p t u a l  fiction further  The  caveat  sample  some  down b y c a t e g o r y , and  categories  programs  any  I n s e r t Table 4 about  broken  1983).  network programs were i n c l u d e d i n t h e  with  the  f o r Research  (CRITC,  in this  of  shows  selected  For  their  c r i m e - d e t e c t i v e , and  episode  news shows w e r e was  and  documentary,  r e g a r d l e s s o f t h e t i m e t h e y were a i r e d .  network  definitions  i s t h e o n l y E n g l i s h n e t w o r k t h a t shows news For  popular national  of  Wright  prime-time  episode  f o u r most p o p u l a r  prime-time.  John  news,  comedy,  i f another  set  Children  c a t e g o r i e s were  news c a t e g o r y , CBC in  on  f o u r most p o p u l a r  that  among t h e  and  a  of Kansas Center  Television  situation  five  included,  Huston  categories are:  entertainment,  using  into  division  non-fiction. "hard"  news and  for  not into these  Non-fiction other  non-  News shows t e n d t o h a v e much s h o r t e r s e g m e n t s o r b i t  lengths  and  many  characters,  so  more  the  of  them,  as  memory demands on  be g r e a t e r t h a n i n o t h e r p r o g r a m s . therefore  were  some a n a l y s e s . two  separated  from  The  news p r o g r a m s  non-fiction was  the  first (shows  programs  12  well the  The  other  as  more  prominent  coders are  two  hard  likely  to  news p r o g r a m s  n o n - f i c t i o n programs  for  of the  t h r e e c a t e g o r i e s t h u s was  the  1  2) , t h e  and  (shows  3 to  12),  second  and  f i c t i o n p r o g r a m s (shows 13 t o  the  was  the  third  other  category  24).  Procedure The  author  or  a  research  a s s i s t a n t went  to  third-  and  fourth-year undergraduate c l a s s e s , p r i m a r i l y i n Psychology,  to  recruit  an  subjects.  The  study  e m p h a s i s on w h a t w o u l d be told during TV  this  show a n d  mention  of  the  described  briefly,  r e q u i r e d of p a r t i c i p a n t s .  initial  answering  was  contact that they  some q u e s t i o n s  expert  ratings  i t afterward.  made,  then  or  S t u d e n t s who  agreed then,  in principle,  to volunteer  a  information  sheet,  Appendix  one-page  v o l u n t e e r s were l a t e r c o n t a c t e d d a t e and  time f o r t h e i r  Participants groups  of  session (ACS)  to  they  and  (Appendix notes  1  asked A)  during  individual  were  given  the  segments  copy  the  completed  by p h o n e t o a r r a n g e a  specific  the  program At of  in  the the  the  start  lab of  Audience  instructions.  that  show, of  later.  These  their  a  read  explained  No  C) .  participants.  to  a  viewing.  watched  5  (see  They w e r e  w o u l d be w a t c h i n g  about  was  with  particular,  program,  the  small  the  coding  Coding  Sheet  These i n s t r u c t i o n s  participants could  in  in  take  information names  of  the  a  few about major  32 characters, North  and  whether  American  or  ethnic  not  each  minority  character  group.  came  The  when t h e y  Participants w o u l d be  asked  general look  also  were were  any  program. watched  of  of the  the  the  and  show.  were  entire  commercials,  by  questions  Once t h e y the  told  filling  out  the  own  1  use  coding  experimenter  that  not  stop  or  instructed  while  w i t h the  They  c o n c l u s i o n of the program they coding sheet, which  before  familiar  show.  did  They were  or  review  filled  their not  instructions  the  through  tape.  At  out the remainder  took approximately  30  they  watching  fast-forwarded  as  sheet.  only global questions intended to capture  impression  at  later  a  instructions  e x p l a i n e d t h a t t h e notes were f o r t h e p a r t i c i p a n t s reminders  from  of  to the they the the the  minutes.  Analyses The  a n a l y s e s were d e s i g n e d  The  first  The  second  expert be  set  and  thought  set  focused of  agreement  analyses  the naive of  on  coders.  scores  e a c h show.  to  Tate's  this.  comparison  but  The  the  as  the  the  naive  agreement  naive  i s no  which  parameter,  coders,  for  inferential  was  approach  to he  coders.  between  rating,  population  goal  statistical  among on  a  There  (1977)  several questions.  experts'  g i v e n by  question within do  focused  statistically  compared w i t h t h e  t o address  make  expert coder  t h e mean o f t h e s c o r e s a s s i g n e d by approach  this  chose  was  was each  same not He  f o r each item w i t h  h i s sample audience.  does n o t d i f f e r e n t i a t e between an  can  statistic  a p p r o p r i a t e t o answer t h e q u e s t i o n o f e x t e n t o f agreement. c o m p a r e d t h e s c o r e g i v e n by an  the  This  instance of p e r f e c t  a g r e e m e n t , t h a t i s , a l l members o f t h e s a m p l e a u d i e n c e exactly  t h e same  agreement with  score  as t h e expert  coder,  i n which t h e sample audience's  that  of the expert  a n d o n e o f no  mean s c o r e  but t h e audience  giving  scores  coincides are  spread  a c r o s s a l l p o i n t s on t h e s c a l e . A descriptive  statistic,  called  answers t h e main q u e s t i o n b e h i n d developed of  f o rthis  British  between agreement defined with  Statistical  F o r any g i v e n  -1  and  +1  between  i n such  this  that  adequately  r e s e a r c h was  especially  s t u d y by P e t e r Schumacher o f t h e U n i v e r s i t y  Columbia  Laboratory.  a Tscore,  that  item  i s a  the naive  Consulting t h e Tscore  measure  coders  and  Research  yields  of  a  value  disagreement  and t h e e x p e r t s .  It is  a way t h a t -1 i n d i c a t e s no n a i v e c o d e r s  the expert;  1  indicates that  a l l of the naive  agree coders  agree w i t h t h e expert;  and 0 i n d i c a t e s chance agreement,  is,  the naive  coders'  scores  all  points  the scale.  on  standardizes same l e v e l from  i t was  derived.  across  statistic  indicates the  i n detail third  This  i s an  important  feature  i n t h e ACS a n d t h e  UBCCS h a v e s c a l e s r a n g i n g  between  this  t h e s c a l e s s o t h a t t h e same s c o r e  because t h e items  The  time  that  o f agreement r e g a r d l e s s o f t h e l e n g t h o f t h e s c a l e  which  described  are equally distributed  A t t h e same  or  from  2 to 5 points.  The T s c o r e i s  i n A p p e n d i x D.  s e t of analyses  expert-naive  a g r e e m e n t among e x p e r t  coder coders  set o f analyses addressed  focused  agreement (i.e.,  on  the  (i.e.,  relationship Tscores)  reliability).  the generalizability  The  and final  of expert-naive  34 coder  agreement  across  naive  coders,  that  i s , the extent  to  w h i c h s u b j e c t v a r i a b l e s ( e . g . , SES) p r e d i c t e d a g r e e m e n t . Results A g r e e m e n t among n a i v e The extent  first to  questions  in  calculated data had  s t e p i n a n a l y z i n g t h e d a t a was t o d e t e r m i n e t h e  which  concordance,  coders  the  a  naive  similar  Kendall's  coders  way.  W,  tended  to  Kendall's  (Ferguson,  coefficient  1976; Hays,  f o r t h e 22 v a r i a b l e s w i t h i n e a c h  were m i s s i n g  f o r any v a r i a b l e ,  i n t w o ways.  calculated  using  number  subjects  Second,  of  To s a f e g u a r d  entering were  into  redone  were p r e s e n t .  variables fluctuates. program  categories  was  In this  f o r the three  against  only  show was  approach t h e fluctuates.  variables f o r  a p p r o a c h , t h e number o f  calculated. i n Tables  f o r each  the analysis using  case  K e n d a l l ' s W was  In this  The a v e r a g e v a l u e  approaches a r e summarized W a n d X2 v a l u e s  the value  a l l of the variables.  the analyses  which a l l data  First,  of  When a n y  the entire  t h i s p r o v i d i n g s p u r i o u s l y high o r low r e s u l t s ,  the  1981) was  show.  however,  t o be t h r o w n o u t f o r t h a t a n a l y s i s .  calculated  answer  f o r each o f t h e t h r e e The  5.  results  The a v e r a g e  program  categories  of  both  Kendall's using  only  t h e v a r i a b l e s f o r w h i c h a l l s u b j e c t s h a d r e s p o n d e d w e r e : News, W=.63945,  X =61.5166;  X =40.26538; f i c t i o n , 2  values  using  W=.62122, approach  other  2  W=.69946, X = 6 1 . 1 7 4 4 .  X =77.98239; provides  a  were:  W=.64978, perfect  W=.72653,  The  corresponding  W=.60800,  X =97.15520;  2  a l l subjects  2  non-fiction,  2  X =102.04043.  answer  2  to  the  Neither  question  of  agreement  among  converge  on  naive  an  concordance  coders,  answer.  but  The  the  results  Agreement between e x p e r t and n a i v e  been  Tscores  from  t(20)=-.95,  The  variables  that  had  d i d not d i f f e r  from  assess  analyzed  the  Five  i n t h e ACS.  involve  three  separate  other  non-fiction,  Figures in  Table  highest  t h e UBCCS  been  reworded  those  that  when  had n o t  f o r u s e i n t h e ACS,  this  study  five  selected  program  medians t e n d  then  The  compared  22  to the  need t o be d i s c u s s e d .  i n this  s t u d y a n d a l l 129  i s f o r t h e 24 shows a n d t h e  categories described The  but f o r the  earlier,  descriptive  a r e shown  The n e x t  i n Table  news,  statistics 6  and i n  The means f o r a l l o f t h e d i s t r i b u t i o n s  6 are very s i m i l a r , mean  were  o f 22 v a r i a b l e s ,  and f i c t i o n .  6.  low t o h i g h  f o r other analyses.  subset  distributions  2 through  from  distribution.  distributions  The s e c o n d  same  ordered  empirical  i s f o r t h e 24 shows u s e d  three  these  from  i n this  s u b s e t o f 22 v a r i a b l e s  for  here  coders  i n t h e ACS w e r e  distribution.  first  significant  p.= .354.  to  questions  questions  when t a k e n  129 i t e m s  Tscores  total  on  t h e UBCCS  reworded  All  indicate  do  categories.  I n s e r t Table 5 about  taken  approaches  a m o n g s t t h e s u b j e c t s i n t h e way t h e y a n s w e r e d t h e  q u e s t i o n s f o r each o f t h e t h r e e program  The  two  v a r y i n g a maximum  i s f o r the distribution t o be h i g h e r t h a n  o f .097.  of a l l variables.  t h e means a n d r a n g e  from  The The .785  36 f o r the hard The  news, t o  standard  .530  deviations  exceptions,  the  f o r a l l 129  questions.  of  107  other  hard  of  hand,  types.  The  (.785),  but  deviation placing  from most having  in  other  level range  of  many more s h o r t  bits  of prominent c h a r a c t e r s . for the  Taken  for  yielded a hard  news  from  the  was  sample  and  shows, other  the  a l l used  on  for  the  news  standard  rationale  programs  in this  segments and  The  program  (1.5;  News  creates  higher.  highest  greatest  category.  This  6 and  together,  the  6 demonstrate  analysis  variables falls  case  was  or  program  two  differ  study,  a greater  for  in  number  g r e a t e r memory demands  coders.  Insert Table  through  full  with  s u r p r i s i n g t h a t the a d d i t i o n  agreement  programs,  a l l shows.  low,  i s consistent with  separate  TV  the  differ  also,  This a  and  f o r the  t h a t they  median  them  latter  Tscores  suggests  .438).  generally  I t i s not the  the  the  are  news shows,  variables in  distribution  f o r t h e 22 v a r i a b l e s and  and  in  this  data  that study  therefore  primarily  Figures 2 t o 6 about here  the is  from  Table  subset  of  similar  6 22  to  and  the  high  the  end  2  v a r i a b l e s chosen total  r e p r e s e n t a t i v e . Moreover,  within  Figures  of  the the  set  of  subset larger  distribution. R e l a t i o n s h i p between Tscores Tscores experts  were  d e s c r i b i n g the found  to  be  and naive  reliability coders'  significantly  agreement  with  the  correlated with  the  reliability p_<.001 which to  scores  f o r the expert  (see Table  7).  the naive coders  be t h e same  amongst  This  coders,  indicates  that  tend not t o agree  ones  on w h i c h  Regression naive  coder  naive  coders.  keep  applied  tend  not t o  agree  here  data  multiple (Tscores)  to significantly  22  items.  predicted  predict  Of  of  each  agreement.  i s being  t h e 10  coders'  predictor  1 o f t h e 22  of  dependent  and  approach  subject  variable  I t i s important t o  scores  similarity  f o r each o f variables,  (item)  7  variables.  w e r e p r e d i c t e d b y one o r  percentage  the  using  This  (subject)  of  and  done,  predicted i s the  7 o f t h e 22 i t e m T s c o r e s  predictors  were  variables.  Table  variance.  subgroups  as t h e p r e d i c t e d v a r i a b l e s  more o f t h e s u b j e c t v a r i a b l e s . the  expert-  f o r the subjects are  regressions  importance  t h a t what  at least  Conversely,  whether  across  variables  as t h e p r e d i c t o r  the relative  i n mind  coders  t o assess  equally  between t h e u n t r a i n e d and t r a i n e d the  tend  on  3 a l o n g w i t h any r e c o d i n g d o n e .  o f agreement  indicates  were used  The p r e d i c t o r  step-wise  subject  found  analyses  agreement  i n Table  Full  the  the questions  themselves.  Generalization of the r e s u l t s across naive  extent  t(20)=3.82,  with the experts  the experts  I n s e r t Table 7 about  listed  r=.65,  8 l i s t s the significance of  the  predicted  38 The  number o f s i g n i f i c a n t  predictions  be p r e d i c t e d p u r e l y b y c h a n c e . the  information  of  other  (11) i s w h a t w o u l d  They a r e d e s c r i b e d b r i e f l y f o r  researchers  who  obtain  similar  findings.  I n s e r t Table 8 about  *** T a n n i s : analyses  B e c a u s e I'm t a k i n g t h e s t a n c e t h a t t h e r e g r e s s i o n  don't  tell  us a n y t h i n g  I'm  b a l a n c e o f t h i s s h o u l d be d e l e t e d . Coding (TEENREP)  the presence was  coders  expert  (TVTOT).  That  versus  how  much  the  viewer  i s , t h e more t e l e v i s i o n t h e  the less  likely  o r absence o f teenagers  t h e show was f o u n d  absence  to predict  of non-traditional  more t h e n a i v e c o d e r  liked  t h e y coded l i k e an e x p e r t  they  were t o  i n t h e same way a s  familiar  culture,  the  less  problems  (PROB) l i k e a n e x p e r t .  PROBlems  more  naive  like  female  behavior  (NTRFEM).  coder.  less  likely  coding t h e presence  t h e show t h e y w a t c h e d , t h e more  The  code  to  i n a show  coders.  Liking  The  Do y o u a g r e e ?  related  reported watching,  code t h e presence  of the opinion that the  v e r s u s absence o f teenagers  negatively  watched t e l e v i s i o n naive  here  coders  they  expert  were w i t h N o r t h  were  to  code  American  t h e degree  of  Female n a i v e c o d e r s tended t o coders  than  d i d male  naive  coders. The related  frequency  of watching  t o one v a r i a b l e  a show  (SHFRQ) was  and n e g a t i v e l y r e l a t e d  positively  t o 2 others.  39 The  more o f t e n t h e n a i v e c o d e r  c o d e d t h e more l i k e  an  expert  the degree of romantic frequently  presence  coders  less  likely  of  physical  l e a n i n g s o f t h e show Two  characters  of  &  Moore,  like  an  aggression  (PAG)  First,  females the  The  the an  and  more  series,  expert  the  the  political  more  likely  gender  mix  the of  evaluating  among  higher  (OCGOAL) on t h e SES  the  seen  predicted  versus  (SXPROM).  expert  having  t h e y were t o code l i k e  variables  1987) , t h e  evaluations of  (ROMNCE) i n t h e show.  reported  males  occupational goal  were t h e i r  they  (LFTRT).  different  preponderance  coder  content  naive  however, the  reported seeing the s e r i e s  the  prominent  naive  coder's  scale (Blishen,  Carroll,  naive  to  the  the  the  coder  was  prominent  S e c o n d , t h e more t h e s u b j e c t s r e p o r t e d t h e y  characters.  typically  t h e l e s s l i k e l y t h e y w e r e t o c o d e SXPROM l i k e  an  code  studied,  expert.  Discussion The  results  s y s t e m a t i c and coding  system  by t r a i n e d yield  the  indicate  reliable developed  coders same  not  answer  but  there  was  The  content,  the  good  deal  high  agreement  despite  W i l l i a m s , and  her  students  for  use  a n a l y s i s does, f o r t h e most p a r t , as  the  tended  of  assessment of t e l e v i s i o n  naive  Moreover, they level  on  results  a l l of a  emphasis  i n content  untrained viewers. did  its  by  that  does  audience  coders  in this  questions of  i n an  concordance  research research  clearly  identical  manner,  in their  t o agree w i t h the e x p e r t s . between  the  naive  with  and  answers.  Overall, expert  the  coders  40  supports  the e n t e r p r i s e of content  analysis i n general,  and  t h i s system o f content a n a l y s i s i n p a r t i c u l a r . As  stated  earlier,  i n t e r a c t i o n of viewer it  i s probably  this  c a n be  which  as a  simple,  communication  outcome  characteristics  predominate.  when  expected coders, and  viewer  that  would  there  would  high the  agreement trained  untrained The  among  no  be e x p e c t e d  untrained and h i g h  coders, agreement  and  naive  i t would  be  untrained  would  be l o w ,  agreement  between  that there high  would  reliability  between  be  for  trained  and  coders. data  are consistent with  i t was i m p o s s i b l e t o t r a i n  items  i n t h e same way.  which naive coders some i t e m s  research  or  viewer  among  coders  the  When t h e c h a r a c t e r i s t i c s o f t h e  items  degree,  expert  low agreement  little  which  predominate  of trained  i t would  coders,  in  be  u n t r a i n e d and t r a i n e d c o d e r s . message p r e d o m i n a t e  instances  I n comparing  be  instances i n  i n determining  characteristics  there  from  purposes  predominate  to  that the r e l i a b i l i t y  that  ranging  of the  In reality  but f o r i l l u s t r a t i v e  continuum,  message > c h a r a c t e r i s t i c s  coders,  i s the result  and message c h a r a c t e r i s t i c s .  not t h i s  seen  communication  t h e naive coders  detail  however,  there  across  coders  to reliably  some  score the  Similarly,  on on  with the experts t o a high  amongst t h e m s e l v e s .  o f agreement  a l l 129  On  t o b e t h e same i t e m s  (e.g., LFTRT).  agreed  agreed  the distribution  greater  hypothesis.  These tended  d i d not agree  and t h e e x p e r t s  this  will  variables.  i s no way t o m e a s u r e d i r e c t l y  be  In future examined i n  Unfortunately, the interaction  41 of  viewer  issue  and  message  global  a  content  application.  results  should  naive  be  variables.  analysis  That  varied  and  on  other  address  this  system  should  have  level  regression anlayses.  system assesses  the  They  were  students time,  that  group,  because  both  to  come h e r e  whether  found served  similar  naive  expert  that  by  a  the  use  coders coding  heterogeneous  the  vary  the  expert  and  coders only  coding and  occupations  physcian  in this  who  had  to  study.  They  consistencies in  viewers.  This  of  issue  been  grown up It the the will  students.  is  at  Within  from  lawyer.  They  dimension  system.  ranged  the  university  naive. coders  themselves  recently  were  one  of  study.  is, • university  expert  to  with  coders  i f a more r e p r e s e n t a t i v e sample as  SES  however, s y s t e m a t i c a l l y  o r i e n t a l s whose f a m i l i e s had  generations t o people and  and  television  indicates  this  the  wise  Their parents'  C a u c a s i a n s and  India  group,  i s , training  chambermaid  ethnic  study.  know t h e g e n e r a l i z a b i l i t y  sample o f n a i v e  i t seemed  however,  diverse.  cannot  select  chosen  and  we  the  fairly  similar  in this  agreement w i t h t h e This  i t  students.  Unfortunately,  a  and  same m e s s a g e s when u s e d  group o f u n i v e r s i t y  r e s u l t s beyond  case  demographic  of t h e i r  uses  in their  T h e s e v a r i a t i o n s were n o t ,  in  know  thus  who  the  considerably  to the  were  matter  T h i s was  related the  coding  i s , no  obtained.  coders  backgrounds,  and  and  empirically. Ideally,  The  characteristics  were mill  a  this quite  worker  ranged  from  i n Canada  for  i n China  and  impossible  results  would  to be  adult population be  addressed  in  42 future  research.  understand extending  the  I t a l s o would  developmental  i t t o younger  g r e a t e s t concern be  i f children  and  content  In  is,  to  finding  significant  as  naive  over  would  a l l the  be  significant considered  coders  i s to  the  of  predictions  by  Type  with  in  a skeptical be  previous  eye,  ignored.  literature,  11  however,  Type I  chance, a l l  will  10  T h i s means errors and  11  must  be  be  r e s u l t s do so  is  that  The  i t could well  Some o f t h e  we  i t  error,  items.  Thus,  and  I  chance.  by  occurred.  value,  analyses,  about  purely  of  teens.  prediction  probability  be  as  would  If  occur  predictions  that a l l should basis  to  of  a f f e c t e d by t e l e v i s i o n v i e w i n g .  regression analyses  expected  Perhaps  coders,  p r e d i c t o r v a r i a b l e s w e r e u s e d t o p r e d i c t 22 that  by  are  adult  the  research  they  a l l of  consider  to  s i m i l a r the r e s u l t s  i t i s p e r c e i v e d by c h i l d r e n and  considering  important  by  important  o l d e r groups.  how  served  t o be  assessed  must know how  g r o u p s and  teens  particularly  i m p l i c a t i o n s of t h i s  i s to determine  t h e ones most l i k e l y the  be  argued  h a v e some  be  mentioned  briefly. As  a  show t h e y program coding (that  measure  of  the  watched, they series  romance as  i s , differently  physical  aggression  program.  I t may  coders'  were asked  (SHFRQ). the  naive  This  expert from and  often they  correlated  coders  the the  how  familiarity  had  experts) political  the  watched  the  positively  done, b u t with  with  the  with  negatively presence  philosophy  of  of the  be t h a t i n d i v i d u a l s more f a m i l i a r w i t h a show  w e r e more s e n s i t i z e d  t o the romantic  content, but d e s e n s i t i z e d  to  the  physical  supported violence the  by  aggression.  other  research  increased  This  latter  i n which  tolerance  exposure  to real  life  o p e n ( s e e C o m s t o c k , 1980, f o r a r e v i e w ) .  naive  o f watching  viewer  same way  as a t r a i n e d finding  format  and of  trained  o f some  format.  study  agreement  the results  that  would  i n the  i s c o n s i s t e n t w i t h Himmelweit e t  was  a p r o g r a m was t h e  n o t t h e same  was t o a s s e s s coders,  i n the  message  questions  and h i g h ,  format  perceptions question  been  t h a t were n o t reworded, h a v e b e e n more  i n the  difficult  not t o evaluate  on r e w o r d e d  a g r e e m e n t was s i m i l a r  strength  remains  characteristics  in a potentially  untrained  from t h a t on q u e s t i o n s of  violence  The c h a n g e s w e r e made b e c a u s e  of this  Had  content  questions  were  versus  violence;  rated i t .  questions  the goal  televised  The f i n d i n g t h a t t h e  t h a t t h e more p o p u l a r  UBCCS a n d i n t h e ACS. UBCCS t h e s e  some coder  more s i m i l a r l y p e o p l e The  different  interpretation  difficult.  The  finding  d e s p i t e rewording,  t o the conclusion that naive  coders  agreed  number  of  important  issues  i n t r o d u c t i o n c o u l d n o t be a d d r e s s e d The  results  issues  should  proposed micro  were be  sufficiently addressed  t o explain that  versus  promising, i n future  content  macroanalytic,  discussed  empirically  study. in  i n this  however,  research.  the  study.  that  these  The  model  a n a l y s i s has three  q u a n t i t a t i v e versus  lends  with the  e x p e r t s i n t h e i r p e r c e p t i o n s o f t h e TV p r o g r a m s i n t h i s A  is  a show a l s o p r e d i c t e d w h e t h e r o r n o t a  perceived  a l . 's (1978)  to  for further televised  question of desensitization  frequency  possibility  facets:  qualitative,  and  audience  versus  expert  was  designed  t o bridge a l l o f these  one  was a d d r e s s e d  i n this  be  an i s s u e o f t h e o r e t i c a l  of  whether  coders  an audience  i s critical  researchers approach  may  enterprise  a  would  The UBCCS  coding  The f i r s t  two f a c e t s may  or other preference,  but the issue  t h e same m e s s a g e s a s  t o the a n a l y s i s of content. reasons  to  macroanalytic be s u s p e c t  say t o an a v e r a g e v i e w i n g  choose  a  approach,  i f their  results  microanalytic  can  originally  coders  a n a l y s i s capture included  necessary  a l s o were  earlier,  this  average viewer. turned the take  those  i n the expert to establish  instructed  made  notes  agreement  I f the goal i s  methodology  good  the experience  took  at a l l . with  because  reliability.  t o take  very  detailed  the experts  was  obtained  e x p e r t s and t h e n a i v e  coders.  viewers  i n spite  agreement  i n establishing  a factor  among n a i v e  only  role  of the this  many c h o s e n o t t o  t h a t note  minor  As s t a t e d  from t h a t  d i s c r e p a n t a p p r o a c h s o i t seems l i k e l y a  naive  t h a t w h e r e a s many o f  notes,  The c o n c o r d a n c e  best  i t was  The  a few n o t e s .  different  how  N o t e t a k i n g was  I t i s n o t known how i m p o r t a n t  coders  to  i s t h e impact o f  messages?  o u t t o be, b u t i t i s i n t e r e s t i n g  naive  whole  audience.  t h e messages r e c e i v e d by average v i e w e r s ,  content  believed  the  had n o t h i n g  note t a k i n g on t h e c o n s i s t e n c y o f t h e r e s u l t s . assess  expert  Individual  but  A n o t h e r i s s u e t h a t needs t o be a d d r e s s e d  to  system  issues, but only the l a s t  research.  perceives  have  over  coders.  of  and this  taking played between  the  45 Gunter television viewers  (1988) content  argued  analysis.  actively  reference  to  watch  their  reality,  the  Morley's  (1980)  strongly  and  own  results  The  interpret  study  that  the  are  there  traditional  claimed,  is  i s that  content  with  Despite  this  consistent  with  histories.  this  contention  doing  p r o b l e m , he  unique  of  against  a  good  deal  of  c o n s e n s u s a b o u t what i s p e r c e i v e d . In and  this  w h a t we  What we of  research, are r e a l l y  viewers,  distractions  and,  mental e f f o r t "  they and  i s how  later  interested  i n a r e two  small  (e.g.,  better  watch  Lull.,  a b o u t i t , and  of  tbe  show,  questions.  Even t h i s  the  viewer  takes  experience.  with  us  c l o s e r t o our g o a l .  The  same  small  of  give  with of  many  invested  questions  a  out when  a TV  the  living  their them  show  room.  open-ended  with not  longer  specific  r e v e a l what  term  limitations  n e v e r be  from  the  of  research  answered.  Instead,  incremental  and  population  watching  small step taken shows  things.  might argue t h a t i t  prompt  aWay o v e r  q u e s t i o n may  content  the  to  Because  h a v e t o be  given  not  to  than  1977)  to  know  w a t c h e d i n a room t h a t  more  only  can  have found  approach, however, w i l l  we  that,  room  Hanson, & M a r x ,  impressions  is  What we  w o u l d be  people  methodology the r e a l  "amount  t h a t they  conference  the  home  responded  ask  viewing  low  we  different  at  students  to  average  TV  with  what  affects  (AIME; S a l o m o n , 1 9 8 3 ) . university  a  television  typically,  asked questions  resembled  is  research,  who  knew a h e a d o f t i m e  Others  i n any  w a n t t o know i s how  average  about  as  steps in this  questions,  the  that  take  research messages  46  t a k e n away f r o m are  more s i m i l a r  i  p o p u l a r TV p r o g r a m s by e x p e r t a n d n a i v e than  different.  coders  47 References B a n d u r a , A. Cliffs,  (1977). Social NJ: P r e n t i c e - H a l l .  learning  theory.  Englewood  B l i s h e n , B.R., C a r r o l l , W.K., & M o o r e , C. ( 1 9 8 7 ) . The 1981 s o c i o e c o n o m i c i n d e x f o r o c c u p a t i o n s i n Canada. Canadian R e v i e w o f S o c i o l o g y and A n t h r o p o l o g y . 24., 4 6 5 - 4 8 8 . Budd,  R.W., T h o r p , R.K., & Donohew, L. (1967). a n a l y s i s o f c o m m u n i c a t i o n . M a c m i l a n : New Y o r k .  Content  Comstock, G.A. (1980) . New emphases i n r e s e a r c h on t h e e f f e c t s o f t e l e v i s i o n and f i l m v i o l e n c e . I n E.L. P a l m e r & A. D o r r ( E d s . ) , C h i l d r e n and t h e f a c e s o f t e l e v i s i o n . New Y o r k : A c a d e m i c P r e s s . C o n d r y , J . , B e n c e , P., & S c h e i b e , C. (1987, A p r i l ) . The n o n program content of children's television. Paper presented at The Society for Research in Child D e v e l o p m e n t , B i e n n i a l m e e t i n g , B a l t i m o r e , MD. Cook,  T.D., C u r t i n , T.R., E t t e m a , J . S . , M i l l e r , P.V., & Van Camp, K. (1986, November) . T e l e v i s i o n i n the l i f e of schools. Report prepared f o r the U.S. Office of Educational Research and Improvement Conference on A s s e s s i n g T e l e v i s i o n ' s I m p a c t on C h i l d r e n ' s Education, W a s h i n g t o n , DC.  C o r d u a , G., McGraw, K. , & Drabman, R. (1979). Doctor or nurse: Children's perceptions of sex-typed occupations. C h i l d D e v e l o p m e n t . 50, 590-593. C e n t e r f o r R e s e a r c h on t h e I n f l u e n c e s o f TV on Children. (CRITC, 1983). Critic program categorization system c o d i n g manual. A v a i l a b l e f r o m CRITC, D e p a r t m e n t o f Human Development, University o f Kansas, Lawrence, Kansas, U.S.A. 6 6 0 4 5 . Dorr,  A. ( 1 9 8 6 ) . T e l e v i s i o n and c h i l d r e n : A s p e c i a l medium f o r a s p e c i a l audience. The S a g e CommText s e r i e s ( V o l . 1 4 ) , B e v e r l y H i l l s , CA: B e v e r l y H i l l s .  F e r g u s o n , G.A. education  (1976). Statistical ( 4 t h e d . ) . New Y o r k :  analysis i n psychology & McGraw-Hill.  Fleiss, J.L. (1981). Statistical methods f o r r a t e s p r o p o r t i o n s (2nd e d . ) . T o r o n t o : W i l e y , C h a p t e r 13.  and  G e r b n e r , G. , H o l s t i , O.E., K r i p p e n d o r f , K. , P a i s l e y , W.J., & Stone, P.J. (1969). The analysis of communicaiton c o n t e n t : D e v e l o p m e n t s i n s c i e n t i f i c t h e o r i e s and c o m p u t e r t e c h n i q u e s . New Y o r k : W i l e y & S o n s .  48  G u n t e r , B. ( 1 9 8 3 ) . P e r s o n a l i t y and p e r c e p t i o n s o f h a r m f u l and harmless TV violence. Personality and Individual D i f f e r e n c e s , 4, 6 6 5 - 6 7 0 . G u n t e r , B. ( 1 9 8 8 ) . The p e r c e p t i v e a u d i e n c e . I n J.A. A n d e r s o n ( E d . ) , C o m m u n i c a t i o n Y e a r b o o k V o l . 11. B e v e r l e y Hills, CA: S a g e . H a y s , W.L. Hill.  (1981).  Statistics  (3rd ed.).  New  York:  McGraw-  H i m m e l w e i t , H.T., S w i f t , B. , & B i b e r i a n , M.J. (1978). The audience as critic: An approach to. the study of entertainment. I n . P. Tannenbaum ( E d . ) , Entertainment f u n c t i o n s o f t e l e v i s i o n . New Y o r k : E r l b a u m . H o i s t i , O.R. (1969). and h u m a n i t i e s .  Content a n a l y s i s f o r the s o c i a l R e a d i n g , MA: Addison-Wesley.  Huesmann, R.  P e r s o n a l communication  (1985).  t o T.M.  sciences  Williams.  J a m e s , C.L. (1979). A g r e e m e n t m e a s u r e m e n t and t h e j u d g e m e n t process. J o u r n a l o f N e r v o u s and M e n t a l D i s e a s e , 1 6 7 ( 6 ) , 343-347. J e n s e n , K.B. (1987, May). The s o c i a l u s e s o f t e l e v i s i o n news: A q u a l i t a t i v e e m p i r i c a l study of the r e c e p t i o n of Danish t e l e v i s i o n news. Paper p r e s e n t e d t o t h e 37th Annual Conference of the International Communication Association, Montreal. Kirk,  J . , & M i l l e r , M.L. (1986). R e l i a b i l i t y and v a l i d i t y i n q u a l i t a t i v e research. Sage U n i v e r s i t y P a p e r S e r i e s on Q u a l i t a t i v e Research Methods, ( V o l . 1 ) . Beverly H i l l s , CA: S a g e .  K r i p p e n d o r f , K. ( 1 9 8 0 ) . C o n t e n t a n a l y s i s : An i n t r o d u c t i o n t o i t s methodology. The Sage CommText S e r i e s ( V o l . 5 ) , B e v e r l y H i l l s , CA: B e v e r l y H i l l s . Lull,  J . T . , H a n s o n , C.A., female stereotypes Q u a r t e r l y . 153-157.  M o r l e y , D. ( 1 9 8 0 ) . The decoding. British London: B F I .  & M a r x , M.J. ( 1 9 7 7 ) . R e c o g n i t i o n of in TV commercials. Journalism " N a t i o n w i d e " a u d i e n c e : S t r u c t u r e and Film Institute ( B F I ) M o n o g r a p h 11.  Newcomb, H. (1978). Assessing the v i o l e n c e p r o f i l e s of G e r b n e r a n d G r o s s : A h u m a n i s t i c c r i t i q u e and s u g g e s t i o n . C o m m u n i c a t i o n R e s e a r c h . 5, 2 6 4 - 2 8 2 .  49 Nielson, N.A. (undated). Content analysis a n a l y s i s : The c a s e o f t e l e v i s i o n and t h e R e s e a r c h D e p a r t m e n t , Denmark's R a d i a .  and reception family. Media  R o s e n g r e n , K.E. (1981). Advances i n S c a n d i n a v i a n content a n a l y s i s : An introduction. I n K.E. Rosengren (Ed.), Advances i n content a n a l y s i s . B e v e r l y H i l l s : Sage Annual R e v i e w s o f C o m m u n i c a t i o n R e s e a r c h , V o l . 9, 9-19. R o s e n g r e n , K.E., Wenner, L.A., & P a l m g r e n , P. ( 1 9 8 5 ) . Media g r a t i f i c a t i o n s research: Current perspectives. Beverly H i l l s , CA: Sage. S a l o m o n , G. (1979). i n t e r a c t i o n of learning. New Y o r k : J o s s e y - B a s s .  media,  cognition,  and  S a l o m o n , G. ( 1 9 8 3 ) . T e l e v i s i o n w a t c h i n g and m e n t a l e f f o r t : A s o c i a l p s y c h o l o g i c a l view. I n J . B r y a n t & D.R. Anderson (Eds.), Children's understanding of t e l e v i s i o n . New York: Academic P r e s s . Schank, R.C, & A b e l s o n , R.P. (1977). and u n d e r s t a n d i n g . H i l l s d a l e , NJ:  Scripts, plans, goals. Erlbaum.  S e p s t r u p , P. ( 1 9 8 1 ) . M e t h o d o l o g i c a l developments i n content analysis. I n K.E. R o s e n g r e n ( E d . ) , A d v a n c e s i n c o n t e n t analysis. Beverly H i l l s , CA: Sage A n n u a l Reviews of C o m m u n i c a t i o n R e s e a r c h , V o l . 9, 13 3-158. S h e r m a n , B.L., & D o m i n i c k , J.R. (1984, N o v e m b e r ) . & rock & r o l l : A content a n a l y s i s of music Paper p r e s e n t e d a t t h e Speech Communicaiton Annual General Meeting, Chicago.  Guns, s e x , television. Association  S t e e v e s , H.L., & S m i t h , M.C. (1987). C l a s s and g e n d e r i n prime-time t e l e v i s i o n e n t e r t a i n m e n t : O b s e r v a t i o n s from a s o c i a l i s t f e m i n i s t p e r s p e c t i v e . J o u r n a l of Communication I n q u i r y . 11. Stone, P.J., Dunphy, D . C , Smith, M.S., & Ogilvie, S.M. (1966). The g e n e r a l i n q u i r e r : A c o m p u t e r a p p r o a c h to content analysis. C a m b r i d g e , MA: M.I.T. P r e s s . S u r l i n , S.H., Romanow, W.C. , & S o d e r l u n d , W.C (1987, May). TV n e t w o r k news: A C a n a d i a n - A m e r i c a n comparison. Paper p r e s e n t e d a t the meeting of the Canadian Communication Association, Montreal. Tate,  E.D. (1977). Viewers' perceptions of selected t e l e v i s i o n programs. I n R e p o r t o f The R o y a l C o m m i s s i o n on V i o l e n c e i n t h e C o m m u n i c a t i o n s I n d u s t r y , (RCVCI) V o l . 6, 2 8 3 - 4 0 1 , T o r o n t o : R C V C I .  50 T a y l o r , S.E., & C r o c k e r , J . ( 1 9 8 1 ) . Schematic bases o f s o c i a l information processing. I n E.T. H i g g i n s , C P . Herman, & M.P. Zanna (Eds.), Social cognition: The Ontario s y m p o s i u m i n p e r s o n a l i t y a n d s o c i a l p s y c h o l o g y . ( V o l . 1, pp. 8 9 - 1 3 5 ) . H i l l s d a l e , NJ: Erlbaum. W i l l i a m s , T.M. ( E d . ) . (1986). The i m p a c t o f t e l e v i s i o n : A natural experiment i n three communities. New York: Academic P r e s s . W i l l i a m s , T.M., & B o y e s , M.C (1986). Television viewing p a t t e r n s and u s e o f o t h e r media. I n T.M. W i l l i a m s ( E d . ) , The i m p a c t o f t e l e v i s i o n : A n a t u r a l e x p e r i m e n t i n t h r e e c o m m u n i t i e s . New Y o r k : A c a d e m i c P r e s s . W i l l i a m s , T.M., Phillips, S., & T r a v i s , L. ( 1 9 8 5 ) . The U n i v e r s i t y o f B r i t i s h C o l u m b i a TV c o n t e n t c o d i n g system (UBCCS) a n d m a n u a l . Unpublished manuscripts available f r o m T.M. W i l l i a m s , D e p a r t m e n t o f P s y c h o l o g y , U n i v e r s i t y of B r i t i s h Columbia, 2136 West M a l l , V a n c o u v e r , B.C., C a n a d a , V6T 1Y7. W i l l i a m s , T.M., P h i l l i p s , S., T r a v i s , L. , & W o t h e r s p o o n , D. (1988, s u b m i t t e d f o r p u b l i c a t i o n ) P o r t r a y a l o f Canada, t h e U.S.A., a n d o t h e r c o u n t r i e s o n C a n a d i a n v e r s u s U.S. television. Canadian J o u r n a l o f Communication. W i l l i a m s , T.M., Young, R.A., P a r k e r , S., W o t h e r s p o o n , D., C u r r o r , S., & W i n t e r , A. ( 1 9 8 7 , O c t o b e r ) . Messages about education i n TV p r o g r a m s p o p u l a r with c h i l d r e n and teenagers. F i n a l r e p o r t s u b m i t t e d t o U.S. O f f i c e o f Education. W i l l i a m s , T.M., Z a b r a c k , M.L., & J o y , L.A. ( 1 9 7 7 ) . A content analysis of entertainment t e l e v i s i o n programming. In . Report o f The R o y a l Commision on V i o l e n c e i n the C o m m u n i c a t i o n s I n d u s t r y , (RCVCI) V o l . 3, 1-155, T o r o n t o : RCVCI. W i l l i a m s , T.M., Z a b r a c k , ''M.L., & J o y , L.A. ( 1 9 8 2 ) . The p o r t r a y a l o f a g g r e s s i o n on N o r t h A m e r i c a n television. J o u r n a l o f A p p l i e d S o c i a l P s y c h o l o g y . 12, 360-380.  Table 1 V a r i a b l e names a n d t h e i r l o c a t i o n i n t h e ACS V a r i a b l e Name  Section  Page  Question  NAETH  2  2  6 a  SXPROM  3  5  12  PFOCMX  3  5  14 a  NTRFEM  3  6  14 d  PMOCMX  3  5  15 a  NTRMAL  3  7  15 d  SXOB  3  9  20  HUMSX  3  9  22  ROMNCE  3  9  24  PAG  4  12  30 a  VAG  4  14  31 a  DNGWRLD  4  15  32  GUNS  4  15  33 a  PROB  5  .15  36a  POLITIC  5  16  37 a  RICHFAM  5  16  38  TEENREP  5  16  39  LFTRT  6  18  42  NAT  6  18  43  LAWAUT  6  18  44  PAUTSX  6  18  45 a  PAUTETH  6  18  45 b  52 Table 2 Frequency o f f a i l u r e t o answer V a r i a b l e Name  questions  Frequency of  % o f 240  missing  NAETH  9  3.8  SXPROM  4  1.7  PFOCMX  2  .8  NTRFEM  6  2.5  PMOCMX  3  1.3  NTRMAL  5  2.1  SXOB  2  .8  HUMSX  1  .4  ROMNCE  1  .4  PAG  2  .8  VAG  1  .4  DNGWRLD  1  .4  GUNS  1  .4  PROB  1  .4  POLITIC  5  2.1  RICHFAM  4  1.7  TEENREP  0  0.0  LFTRT  7  2.9  NAT  0  0.0  LAWAUT  5  2.1  PAUTSX  1  .4  PAUTETH  5  2.1  53 Table 3 S u b j e c t demographic v a r i a b l e s used i n r e g r e s s i o n a n a l y s e s , their location  i n t h e ACS, a n d how t h e v w e r e r e c o d e d  V a r i a b l e Name  Section  Page  Question  TVMEAN  7  20  50  Mean f o r a l l times  SHFRQ  7  20  54  Scale reversed  SHKLIKE  7  21  55  AGE  7  21  56  SX  7  21  57  YEAR  7  21  59  STUDY  7  21  60  ETHNIC  7  21  61-65  *  OCGOAL  7  21  67  **  PAROCC  7  21  68,70  Recoding  1=M 2=F  Total f o r a l l times  ***  * l=White N o r t h American; 2 = V i s i b l e e t h n i c m i n o r i t y , b u t r a i s e d i n N o r t h America, e.g., Canadian b o r n o f Japanese d e s c e n t ; 3=Not v i s i b l e e t h n i c m i n o r i t y , a n d n o t r a i s e d i n N o r t h A m e r i c a , e.g., F i n n i s h ; 4 = V i s i b l e e t h n i c m i n o r i t y and not r a i s e d i n North America. ** S o c i o e c o n o m i c s t a t u s s c o r e ( B l i s h e n , C a r r o l l , 1987) .  & Moore,  *** Same a s **, b u t h i g h e r o f e i t h e r m o t h e r o f f a t h e r  used.  54 T a b l e 4. Program sample Category o f Program  Program  News 1. CTV: N a t i o n a l  News  3. F i f t h E s t a t e Non-Fiction  2. CBC: The N a t i o n a l 4. 20/20  Entertainment  5. E n t e r t a i n m e n t  Tonight  7. W h e e l o f F o r t u n e  6. Newlywed Game 8. L i f e s t y l e s o f R i c h & Fam.  Documentary 9. W o r l d  of Survival  11. N a t u r e o f T h i n g s  10. G z o w z k i  & Co.  12. F r o n t P a g e C h a l l e n g e  S i t u a t i o n Comedy 13. C o s b y Show  14. F a m i l y T i e s  15. N i g h t  16. G o l d e n  Court  Girls  Crime-Detective 17.  Miami-Vice  19. H i t c h c o c k P r e s e n t s  18. H i l l  Street  20. M u r d e r S h e W r o t e  O t h e r Drama 21.  Dallas  23. Highway t o Heaven  Blues  22.  Dynasty  24. L o v e  Boat  55 Table 5 A g r e e m e n t among n a i v e c o d e r s u s i n g K e n d a l l ' s W Mean W *  Mean X  2  *  df  E<  n of subjects  A l l variables:  f l u c t u a t i n g number o f s u b j e c t s  News  .63945  66.5166  21  .0001  10/20  Other nonfiction  .72653  40.26538  21  .005  16/100  Fiction  .69946  67-29178  21  .0001  36/120  All  subjects:  f l u c t u a t i n g number o f v a r i a b l e s n of variables  News  .60800  97.15520  16  .0001  17  Other nonfiction  .62122  77.98239  12.6  .0001  13.6 .  Fiction  .64798  102.04043  15.7  .0001  16.6  * V a l u e s a r e c a l c u l a t e d p e r show w i t h i n a c a t e g o r y a n d t h e n averaged.  Table 6 D i s t r i b u t i o n of Tscores 1  2  3  4  5  Min  -1.0  . 14  -.50  .01  .27  Max  1.0  .89  1. 00  .80  .95  Range  2.0  .75  1. 50  .79  . 68  .594  .531  .556  .497  .546  .307  .172  .438  . 225  .187  Median  .660  .530  .785  .545 .  .570  Mode  .670  .520  .800  .540  .580  Mean Std  Dev  1. A l l 129 v a r i a b l e s i n t h e ACS, and a l l 24 s h o w s . 2. 22 s e l e c t e d v a r i a b l e s , and a l l 24 shows. 3. 22 s e l e c t e d v a r i a b l e s , 2 " h a r d " news s h o w s . 4. 22 s e l e c t e d v a r i a b l e s , 10 o t h e r n o n - f i c t i o n shows. 5. 22 s e l e c t e d v a r i a b l e s , 12 f i c t i o n shows  Table 7 Comparison o f Tscores w i t h r e l i a b i l i t y  scores f o r experts *  V a r i a b l e Name  %N ** T s c o r e  PMOCMX  85  .89  K=1.0  PAUTETH  79  .74  %=.90  GUNS  99  .71  RE=.727  PROB  100  -69  %=.904  SXPROM  98  .69  ROMNCE  100  .64  K=.712  NAETH  96  .59  RE=.786  PAG  93  .59  K=. 810  TEENREP  100  .58  RE=.673  HUMSX  100  .57  K=.705  SXOB  99  .54  RE=.770  VAG  93  .52  RE=.614  100  .52  K=.650  NTRMAL  85  .51  RE=.684  RICHFAM  98  .50  K=.487  LAWAUT  98  -49  K=.441  NTRFEM  92  .48  RE=.697  POLITIC  83  .42  K=.598  PAUTSX  88  .35  %=.77  97  .27  K=.435  PFOCMX  66  .26  K=.618  NAT  76  .14  %=.684  DNGWRLD  **** LFTRT  R e l i a b i l i t y ***  RE=1.0  58 (Table 7 continued) * The c o r r e l a t i o n b e t w e e n T s c o r e s and r e l i a b i l i t y p<.001.  i s .65,  ** %n r e f e r s t o t h e p e r c e n t a g e o f s u b j e c t s , a f t e r whose d a t a w e r e u s e d t o c a l c u l a t e t h e T s c o r e .  recodes,  *** I n r e p o r t i n g r e l i a b i l i t y 1 o f 3 s c o r e s h a s b e e n r e p o r t e d , Kappa ( F l e i s s , 1 9 8 1 ) , M a x w e l l ' s RE (James, 1 9 7 9 ) , o r p e r c e n t agreement. F o r a c o m p l e t e e x p l a n a t i o n o f c h o o s i n g one o v e r a n o t h e r s e e W i l l i a m s , Young, P a r k e r , W o t h e r s p o o n , C u r r o r , and W i n t e r , 1987. **** O n l y t h e c e n t r e t h r e e p o i n t s on LFTRT f o r m an a c t u a l continuum. Using these three points only, the Tscore i n c r e a s e s t o .53, b u t t h e %n d r o p s t o 25.  Table 8 Forward stepwise r e g r e s s i o n Independent Dependent  F  analyses df  E<  R  Beta  SHLIKE  TEENREP  7.11  1, 76  .05  .0856  .2925  SHLIKE  NTRFEM  3.93  1, 188  .05 '  .0205  . 1431  SX  PROB  5.77  1, 188  . 01  .0298  . 1725  ETHNIC  PROB  5.52  2, 187  .005  .0557  SHFRQ  ROMNCE  6.77  1, 196  .01  .0333  SHFRQ  PAG  6.35  1, 196  . 01  .0314  -.1772  SHFRQ  LFTRT  6.25  1, 196  . 01  . 0309  -.1861  YEAR  LFTRT  5. 18  2, 195  .01  .0505  -.1403  OCGOAL  SXPROM  5.32  1. 196  . 05  . 0264  .2104  STUDY  SXPROM  6.78  2, 195  . 001  .0650  -.2021  -.1612 . 1827  60 Figure F i g u r e 1. approaches  Captions  Three d i m e n s i o n a l box d e p i c t i n g d i f f e r i n g t o the analysis of content.  F i g u r e 2. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r a l l 129 v a r i a b l e s ( a f t e r r e c o d i n g ) i n t h e ACS a n d a l l 24 shows c o d e d by t h e n a i v e c o d e r s . F i g u r e 3. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s p a p e r a n d a l l 24 shows coded by t h e n a i v e c o d e r s . F i g u r e 4. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s paper f o r t h e 2 hard news shows o n l y . F i g u r e 5. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s p a p e r f o r t h e 10 o t h e r n o n - f i c t i o n shows o n l y . F i g u r e 6. D i s t r i b u t i o n o f f r e q u e n c y o f T s c o r e s f o r t h e 22 v a r i a b l e s s e l e c t e d f o r a n a l y s i s i n t h i s p a p e r f o r t h e 12 f i c t i o n shows o n l y .  61 Figure  1.  Audience Research  FREQUENCY  30-  20-  10.  •1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -02-0.1 0 0.1 02 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 TSCORE Figure  2.  FREQUENCY  10  86-  2.  •1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -0.2-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 TSCORE  Figure 3.  FREQUENCY 81  6i  4H  -1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -02-0.1 0 0.1 02 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 TSCORE F i g u r e 4,  65  FREQUENCY  6i  4H  -1.0-0.9 -0.8-0.7-0.6-0.5-0.4 -0.3 -0.2-0.1 0 0.1 02 0.3.0.4 05 0.6 0.7 0.8 0.9 1.0 TSCORE Figure 5.  FREQUENCY  8n  61  4H  21  -1.0 -0.9 -0.8 -0.7 -0.6 -05 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 05 0.6 0.7 0.8 0.9 1.0 . TSCORE Figure 6.  67  Appendix A  69  Overview of your involvement i n t h i s research •  j ..  1.  F a m i l i a r i z e yourself with the research by reading the instructions and asking any questions.  2.  Watch the show.  3.  Answer the questionnaire.  Instructions:  ••.=:!.•;••••  The next page provides space f o r you to make some notes while you watch the show. These notes are intended to serve as memory cues while you are f i l l i n g out the questions. In the section l a b e l l e d " B i t or Segment topic" you should make b r i e f notes about the main events of the show as they occur. These notes are f o r your own use only. i n the next two sections-you should keep track of the prominent or main female and male characters i n the story as they appear.;.. The prominent characters are defined as the characters that are necessary bo t e l l the story. When they f i r s t appear you may not know t h e i r name, so use some other cue (e.g., green dress) and add the name l a t e r . I f -you aren't sure i f they are prominent, make a note and i f they aren't, then cross them out l a t e r . I f any of the prominent characters are Horth Americans and members of ethnic m i n o r i t i e s put a * beside t h e i r names. Again, these notes are f o r you to use l a t e r as memory cues. l  Do not read any of the rest of the questions u n t i l you have watched the complete show. You w i l l watch the show once only. We w i l l f a s t forward through the commercials. You w i l l not be allowed to review the show a f t e r you have f i n i s h e d watching i t . We want you to answer the questions on the basis of the information i n the show you watch, MOT on what you know about the show i n general. Remember, i t i s your general impression of the show and i t s content that we are • interested i n . Please t r y to watch much as you would i n your own home. We hope you enjoy the show.  AUDIENCE CODING SHEET General  Information  Program ID  '  Program T i t l e Hotes taken during program: B i t or Segment topic 1. _ 2. _ 3-1 4. _ 5. _ C._ 7._ 8_ 9._ 10. 11. 12. 13. IA. 15. 10. 17. 18. 19. 20. Female prominent c h a r a c t e r s i n order of appearance (*ethnics!) 1. 5. 9. 13. 2. 6. • 10 14. 3. 7.__ 11. 15. 4. 8._ . 12. 1G. :  Hale prominent c h a r a c t e r s . 1. " 5._ 2. • 0. 3. . 7. 4. 8.  . \  9. 10 11, '• • . ' 12.  13. _14. 15. 16.  Part 2: Horth American Ethnic M i n o r i t i e s Mote: The following questions r e f e r only to North Americans who are members of an ethnc minority group. 0.  (a) Were any North Americans portrayed who were ethnic minorities? 1. no (b)  Tho prominent North American characters i n the program were:  1 a l l mainstream non--ethnics  (c)  A  mostly ethnics but some important mainstream non-ethnics  5 all ethnics  6 not applicable (NA)  2 mostly mainstream non-ethnics but some important ethnics  3 mixed  •'i mostly ethnic9 but some important . mainstream non-ethnics.  5 all ethnics  0 N/A  2 . yes  Was there any clear evidence (portrayal or reference) of racism or prejudice? 1. no  (f)  3 mixed  Wore there any verbal or v i c u a i jokes about North American ethnic minorities? 1. no  (e)  2  mostly mainstream non-ethnics but some important ethnics  The backfiround Horth American characters i n the program were:  1 a l l mainstream non- -ethnics  (d)  2. yes  2. yes  •  I f yes, specify your reasons  .  I f yes, what's the bottom l i n e message of this program about racism or prejudice? 1. acceptable (e.g., l e t pass, not contradicted) . 2. unacceptable (e.g., some contradictory comeback)  72  -3 In the following questions, please evaluate the members of ethnic m i n o r i t i e s that you *'d on the f i r s t page. For each minority group three w i l l be 3 d i f f e r e n t evaluations. 7. Name the ethnic group (a)  2 problems functioning duo to e t h n i c i t y  2 some ethnic traces  3 strongly i d e n t i f y with t h e i r e t h n i c i t y  2 neutral  -  8.. Name the ethnic group  Male  2 problems functioning • due to e t h n i c i t y  2 some ethnic traces  Both  .  3 problems functioning HOT due to e t h n i c i t y  3 strongly i d e n t i f y with t h e i r e t h n i c i t y  What is"your f i n a l o v e r a l l impression of the people represented i n the show from t h i s ethnic group? 1 positive overall impression  9. . Name the ethnic group (a)  Female  How strongly did they i d e n t i f y with t h e i r own e t h n i c i t y ? 1 no evidence of ethnic i d e n t i t y  (c)  3 negative o v e r a l l impression  How w e l l do they function i n mainstream North American society? 1 function adequately  (b)  3 problems functioning HOT due to e t h n i c i t y  What i s y o u r . f i n a l o v e r a l l impression of the people represented i n the show from this ethnic group? 1 positive overall - impression  (a)  Doth  How strongly d i d they i d e n t i f y with t h e i r own e t h n i c i t y ? . 1 no evidence of ethnic i d e n t i t y  (c)  Female  How w e l l do they function i n mainstream North American society? 1 function adequately  (b)  Male  2 neutral •  J negative o v e r a l l impression ., _.  Male  Female  How w e l l do they function i n mainstream North American society? 1 function adequately.  2 problems functioning due to e t h n i c i t y  ;  3 problems functioning NOT due to e t h n i c i t y  Both  73  - 4(b)  How strongly did they i d e n t i f y with t h e i r own ethnicity? 1 no evidence of ethnic i d e n t i t y  (c)  2 some ethnic traces  3 strongly i d e n t i f y with their e t h n i c i t y  What i s your f i n a l o v e r a l l impression of the people represented i n the show from t h i s ethnic group? 1 positive overall impression  2 neutral  3 negative o v e r a l l impression  10. Hame the ethnic group (a)  Female  Doth_  How well do they function i n r.ainstrcam North American society? 1 function adequately  (b)  Male  2 problems functioning due to e t h n i c i t y  :  3 problems functioning NOT due to e t h n i c i t y  How strongly did they i d e n t i f y with t h e i r own ethnicity? 1 no evidence of ethnic i d e n t i t y  2 some ethnic traces  3 strongly i d e n t i f y with their e t h n i c i t y  what i s your f i n a l o v e r a l l impression of the people represented i n the show from t h i s ethnic, group? (c)  1 positive overall impression  3 negative overall impression  2  neutral  11. Same the ethnic group (a)  .  2  :  problems functioning due to e t h n i c i t y  '  Doth_  3_ problems functioning NCT due to e t h n i c i t y  How strongly did they i d e n t i f y with t h o i r own ethnicity? 1 no evidence of ethnic i d e n t i t y  (c)  Female  How w e l l do they function i n mainstream North American society? 1 function adequately  (b)  Male  2 some ethnic traces  3 strongly i d e n t i f y with their e t h n i c i t y  What i s your f i n a l o v e r a l l impressipn of the people represented i n the show from t h i s ethnic group? positive overall impression  neutral  negative overall impression  74  - 5 Part 3:  The Sexes  12. Were the prominent characters i n the program (those you would need to t e l l the story) all male  mostly male but some important females  even mix of females and males  mostly female but some important r.ales  all females  mostly female but some important males  all females  13. Were the background characters i n the program? all male  mostly male but some important females  even mix of females and males  14. (a) The prominent adult females ( r e l a t i v e to other females i n this: program) were: predominantly i n traditionally female occupations (include homemaker) (b)  equal mix of traditionally and nontradi t i o n a l l y female occupations  predominantly i n nontraditionally female occupations  N/A or not information re: occupation to decide  For the prominent adult females, which types of a c t i v i t i e s were emphasized (airtime focus) i n the program? (Check more than'one i f necessary f o r d i f f e r e n t i n d i v i d u a l s ; use your l i s t on front page to remember.)  major focus on occupation major focus on home/family role  major focus on s o c i a l l i f e equal focus on occupation and home/family role  equal focus.on home/family r o l e and s o c i a l l i f e equal focus on occupation and s o c i a l l i f e equal focus on occupation, home/family r o l e , and s o c i a l r o l e (c)  The background adujt females ( r e l a t i v e to other more prominent females i n t h i s program) were:  predominantly i n traditionally, female occupations (include homemaker)  equal mix of traditionally and nontradit i o n a l l y female occupations  predominantly i n nontraditionally female occupations  N/A or not information re: occupation to decide  75  6 (d)  Were there any adult females who behaved i n a nontraditional way (e.g., strong i n an emergency, not dependent on males tor guidance, a s s e r t i v e l y saying what's on her nind, etc.)?  Instructions: When answering questions of t h i s sort attempt always to use the extremes of the scaie, i . e . . 1 or 5. Avoid ucing 2 and 4 i f at a l l possible. These questions do not refer to amount or frequency of behavior, only whether or not i t was portrayed. d e f i n i t e l y no nontraditional female behavior  not sure can't decide  nontraditional female behavior d e f i n i t e l y present  Specify your. r£asons_ (e)  I f you answered (45 or (5.) to the above question ' (non-traditional female behavior i s present) was i t portrayed ac:  1. a) Serious b) jol:t c) both 2. a) p o s i t i v e (e.g., succeccsful/good/rewarded/comp<stent) b) negative (e.g., uncuccescful/bad/punished/incompetent) c) both (f)  Were there any adult females who act?d i n a t r a d i t i o n a l way (e.g., dependent on a male f o r guidance, f a l l s apart i n c r i s i s , non-assertive, etc.)?  d e f i n i t e l y no traditional female behavior  not sure can't decide  t r a d i t i o n a l female behavior d e f i n i t e l y present  Specify your reasons^ (g)  I f you answered 4 or 5 to tho above question ( t r a d i t i o n a l female behavior i s present) was i t portrayed as:  1. a) Serious b) joke c) both 2. a) p o s i t i v e (e.g., successsful/good/rowarded/competent) b) negative (e.g., unsuccessful/bad/punished/incompetent) c) both (a)  The prominent adult males (relative-, to other were:  predominantly i n traditionally male occupations  equal mix of traditionally and nontradi t i o n a l l y male occupations  Mies i n this program)  predominantly i n nontraditionally male occupations (include hoii.emaker)  H/A or not enough information re: occupation to decide  (b)  For the prominent adult males, which types of a c t i v i t i e s were emphasized (airtime focus) i n the program? (Check more than one i f necessary f o r d i f f e r e n t i n d i v i d u a l s )  major focus on occupation major focus on home/family r o l e  major focus on s o c i a l l i f e equal focus on occupation and home/family r o l e  equal focus on home/family r o l e and s o c i a l l i f e equal focus on occupation and s o c i a l l i f e equal focus on occupation, home/family r o l e , and s o c i a l role (c)  The background adult males ( r e l a t i v e to other more prominent males i n t h i s program) were:  1 predominantly i n traditionally male occupations  (d)  2 equal mix of traditionally and nontradit i o n a l l y male occupations  J  A  predominantly i n nontraditionally male occupations (include homemaker)  H/A or not enough information re: occupation to decide  Were there any adult males who behaved i n a n o n t r a d i t i o n a l way (e.g., shown c r y i n g , caring f o r c h i l d r e n , doing domestic chores, s e n s i t i v e to the needs of others)?  1 d e f i n i t e l y no nontraditional male behavior  2__  3 not sure can't decide  A  5 n o n t r a d i t i o n a l male behavior d e f i n i t e l y present  Specify your reasons (e)  I f you answered (A) or (5) to the above question ( n o n - t r a d i t i o n a l male behavior i s present) was i t portrayed as:  1. a) Serious b) joke c) both 2. a) p o s i t i v e (e.g., successsful/good/rewarded/competent) b) negative (e.g., unsuccessful/bad/punished/incompetent) c) both (f)  Were there any adult males who acted i n a t r a d i t i o n a l way (e.g., s k i r t chaser, tough/macho, c a l l s the shots, l i f e revolves around job and s e l f ) ? .  d e f i n i t e l y no traditional male behavior  not sure . . can't decide  t r a d i t i o n a l male behavior d e f i n i t e l y present  - 8 (g)  I f you answered (4) or (5) tc the above question ( t r a d i t i o n a l male behavior i s present) was i t portrayed as: 1. a) Serious . b) joke c; both 2. a) p o s i t i v e (e.g., successsful/good/rewardcd/competent) b) negative (e.g., unsuceessful/bad/punished/incompetent) c) both  16. Were there any messages i n d i c a t i n g females are. subordinate to or worth less than males; e.g., parents disappointed that i t was a g i r l baby; any mention of men trading or owning women; women taking a back seat to and/or being obedient to men; parents preferring a sen over a daughter (because he w i l l carry on the family name, business, etc.)? 1. no  2. yes  It yes, specify what gave you this impression  Was t h i s portrayed as:  • 1'. Acceptable  2.  Unacceptable  17. Were there any messages i n d i c a t i n g that .-.ales are subordinate to or worth less than females, e.g., parents disappointed that i t was a boy and they wanted a g i r l baby; parents favoring a daughter over a son? 1. no Was  2. yes  If yes, specify what gave you t h i s impression  this portrayed as:  1. Acceptable  2.  Unacceptable  18. (a) ' Did you notice any sexist comments/jokes/putdowns about females i n this program? (e.g., "'.soman d r i v e r " jokes, using the words "broad" or "dame", etc.) 1. no  (b)  2. yes  I f yes, specify your reasons  I f yes, what's the bottom l i n e message of this program re s e x i s t messages about females? i . acceptable (e.g., comment l e t pass)  19. (a)  Did you notice any s e x i s t comments/jokes/putdowns about males i n this program? (e.g., comments about men only having one thing on t h e i r mind, " d i r t y o l d man",: "male chauvinist p i g " comments, etc.) 1. no  (b)  2. unacceptable (e.g., comeback to comment)  2. yes  If yes, specify your reasons  • -  I f yes, what's the bottom l i n e message of this program re sexist messages about males? 1. acceptable (e.g., comment l e t pass)  2. unacceptable (e.g., comeback to comment)  78  9 20. Were-there any people shown or referred to as being sex objects (producer portrays someone s o l e l y or p r i m a r i l y i n a sexual manner)? 1. Ho  2. Yes  21. Was there evidence that a double standard e x i s t s f o r females and males about sexual behaviour? ( C i r c l e as many as apply) 1. no v > • evidence re: double standard  2. yes, evidence consistent with the double standard  Specify your reasons_  _^  3. evidence or arguments against the double standard  A. joke about double standard  _  22. Was sex (portrayal or reference) a part of the program? 1  2 some sex  •  3 sex i s a major focus  23. I f there were p o r t r a y a l s or references to any of the f o l l o w i n g , how would you rate them? 1. Contraception a) 2. Pregnancy a) 3. Sexually transmitted a) A. AIDS a)  Serious b) joke c) both Serious b) joke c) both diseases (other than AIDS) Serious b) joke c) both Serious b) joke c) both  2A. Was romance a part of the program? 1 not at a l l  2  some romance  3 romance i s a major focus  .  - 10 Please describe the following heterosexual  relationships i f found i n the show.  25. The couples married and l i v i n g together were (check a l l that apply): a)  l o v i n g , caring cool, casual exploitive  h o s t i l e (verbal/psychological) p h y s i c a l l y abusive  b)  c h i l d (prepuberty) teenager (13 19) young adult (20-35)  middle aged (35-55) _. older (55 & over)  c)  no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual i n t e r e s t kiss/touch with c l e a r sexual intentions . e x p l i c i t reference to sex between people i n relationships  26. The couples i n a committed leve r e l a t i o n s h i p not l i v i n g together were (check a l l that apply): a)  loving, caring cool, casual exploitive  h o s t i l e (verbal/psychological) p h y s i c a l l y abusive  b)  c h i l d (prepuberty) teenager (13-19) young adult (20-35)  middle aged (35-55) older (55 & over)  c)  no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual i n t e r e s t kiss/touch with c l e a r sexual•intentions e x p l i c i t reference to sex between people i n relationships  27. The couples who were f r i e n d s , but not i n a love relationship were (check a l l that apply): . a)  b)  loving, caring cool, casual exploitive  h o s t i l e (verbal/psychological) p h y s i c a l l y abusive  c h i l d (prepuberty) teenager (13-19) young adult (20-35)  middle aged (35-55) older (55 & over).  ;  c) . no sexual behavior inuendo kiss/touch with no sexual .intentions f l i r t i n g or showing sexual i n t e r e s t kiss/touch with c l e a r sexual intentions e x p l i c i t reference to sex between people i n relationships  80  - l l28. The couples who were acquaintances were (check a l l that apply): a) , l o v i n g , caring c o o l , casual exploitive  h o s t i l e (verbal/psychological) p h y s i c a l l y abusive  b)  c h i l d (prepuberty) teenager (13-19) young adult (20-35)  middle aged (35-55) older (55 & over) _  c)  no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual i n t e r e s t kiss/touch with c l e a r sexual i n t e n t i o n s e x p l i c i t reference to sex between people i n relationships  29. The couples who were strangers were (check a l l that apply): a)  l o v i n g , caring c o o l , casual exploitive  h o s t i l e (verbal/psychological) p h y s i c a l l y abusive  b)  c h i l d (prepuberty) teenager (13-19) young adult (20-35)  middle aged (35-55) older (55 & over)  c)  no sexual behavior inuendo kiss/touch with no sexual intentions f l i r t i n g or showing sexual i n t e r e s t . kiss/touch with c l e a r sexual intentions e x p l i c i t reference to sex between people i n r e l a t i o n s h i p s  81  - 12 Part A:  Aggression  In the following questions, an aggressive act i s one which i s intentional. i n t e r p e r s o n a l , and (unless otherwise stated) v i s u a l l y portrayed. This includes a n t i s o c i a l acts with the p o t e n t i a l to do harm but from which the v i c t i m escapes uninjured. I t does not include instances cf accidental injury. On the scales f o r aggressive a c t i v i t y , avoid using numbers 2 and 4 i f at a l l p o s s i b l e ; use only i f you cannot possibly assign a 1 er 5. These questions do not r e l a t e to amount of aggression, only whether i t occurred. 30. (a) Was there any p h y s i c a l aggression (by anyone) shown i n this program? 1 definitely no aggression shown (b)  2_  _4  S physical aggression definitely present  3 can't decide unsure •  5 physical aggression by females definitely present  »  Was there any p h y s i c a l aggression against females shown i n t h i s program. definitely no aggression against females  (d)  3_ can't decide unsure  Was there any p h y s i c a l aggression by females shown i n this program? 1 definitely no aggression by females  (c)  2  can't decide unsure  physical aggression against females definitely present  Was there any p h y s i c a l aggression by males shown i n this program? 1 definitely no aggression by males  2  _3 can't decide unsure  4  5 physical aggression by males definitely present  82  13 (o)  Was there any physical aggression against males shown i n t h i s program? 1 definitely no aggression against males  (f)  __3 can't, decide unsure  2  :  4  5 definitely had reference to p h y s i c a l aggression  can't decide unjure  definitely violent  Was there any reference to violence i n t h i s program?' 1 d e f i n i t e l y no references to violence  (i)  3 can't decide unsure  Was there any violence shown i n the program ( i . e . , extreme p h y s i c a l aggression that i s p o t e n t i a l l y f a t a l ) ? definitely nonviolent  (h)  5 physical aggression against males definitely present  J\  Was there any reference made to p h y s i c a l aggression i n t h i s program? 1 d e f i n i t e l y no reference to physical aggression  (g)  2  2  3 can't decide unsure  4  5 definitely had reference to violence  What i s the bottom l i n e message of t h i s program re the a c c e p t a b i l i t y of p h y s i c a l aggression as a method'of c o n f l i c t resolution? (Check more than 1 i f more than 1 message was perceived.) p h y s i c a l aggression i s c l e a r l y acceptable . can't decide, unsure p h y s i c a l aggression i s c l e a r l y unacceptable N/A or not enough information, to. code ^  Specify  :  ;  83  - 14 (j)  What i s the bottom l i n e message of this program re the successfulness of p h y s i c a l aggression as a method of c o n f l i c t r e s o l u t i o n i n the short term rather than long term sense? (Check more than 1 i f more than 1 message was perceived.)  p h y s i c a l aggression i s c l e a r l y successful (e.g., short term goals were achieved through p h y s i c a l aggression can't decide, unsure p h y s i c a l aggression i s c l e a r l y not successful (e.g., short term goals were not achieved) H/A or not enough information to code 31. (a)  Is there any verbal/psychological aggression i n the program (e.g., . y e l l i n g , verbal' i n s u l t s ) ?  1 d e f i n i t e l y not at a l l verbally/ psychologically aggressive  2  3 can't decide  4  _ definitely verbally/ psychologically aggressive  Specify (b)  What i s the bottom l i n e message of t h i s program re the acceptabilty of verbal or psychological aggression as a method of c o n f l i c t resolution? (Check more than one i f more than one message was perceived)  verbal/psychological aggression i s c l e a r l y acceptable can't decide verbal/psychological aggression i s c l e a r l y not acceptable H/A or not enough information to code Specify (c)  .  What i s the bottom l i n e message of t h i s program re the successfulness of verbal/psychological aggression as a method of c o n f l i c t resolution? (check more than I i f more than 1 message was perceived).  verbal/psychological aggression i s c l e a r l y successful (e.g., short term goals were achieved through this behavior can't decide, unsure verbal/psychological aggression i s c l e a r l y successful (e.g., short term goals were.not achieved through this behavior H/A or not enough information to code \ Specify  - 15 32. Does the program give the impression that the world i s a dangerous place (e.g., because of war, p h y s i c a l aggression of others, n a t u r a l d i s a s t e r s , f a t a l t r a f f i c accidents etc*)? program gives no impression that the world i s a dangerous place  some people might got the impression that the world i s a dangerous place  impression i s c l e a r l y given that the world i s a dangerous place  Specify_ 33. (a)  Were any r e a l guns shown in'the program?  (b)  Who  had them?  1. No,  ._  2.  Yes  (Check a l l that apply)  Military Police Dad c i t i z e n Good c i t i z e n Other authority I f other authority specify who (c)  How were the guns used?  (Check a l l that apply)  some or a l l shown but not used defense of s e l f / o t h e r s defense of property 34. Was  there any p o r t r a y a l of or reference to death of humans?  1. no Part 5.  aggression_ intimidation/coercion_ recreation  2. yes Issues. Controversies, and Dilemmas  30. Does t h i s program present evidence that problems e x i s t ? For example, minor problems would be those that are r e l a t i v e l y short term such as family disagreements. Major problems are those that have long tens impact, such as alcoholism or divorce, or those that have an impact on a large number of people, such as p o l l u t i o n or corruption i n p o l i t i c s . C i r c l e a l l that apply. (a)  1  2  Ho problems Specify  3  Minor problems .  Major problems \  ;  I f minor (2), were the problems: (b) __1 2 c e n t r a l to incidental the p l o t to the p l o t  (c)  •__  1 portrayed as serious  ;  • 2 portrayed as funny  85  16 I f major (3), were the problems: 2 incidental to the plot  (d)  1 c e n t r a l to the p l o t  (f)  The take-home message from this program regarding issues and controversies i s that they are usually: (check one)  portrayed as funny  4.  Were there e x p l i c i t p o l i t i c a l comments or references i n the program? ( c i r c l e highest applicable number) 2  1 definitely not (b)  portrayed as serious  2. not clear-cut 3. not c l e a r - c u t (shades of grey, (shades of grey, and some answers and there are no are better than r i g h t or better others) answers)  1. clear-cut ( r e l a t i v e l y black and white) and the r i g h t answers are clear 37. (a)  (e)  3 can't decide unsure  4  5 definitely yes  I f 4 or 5, was the content r.ore balanced or more biased? 2. biased  1. balanced Specify b r i e f l y 38. (a)  Was the show concerned with the r i c h and/or famous?  1 not at a l l  2 somewhat  31 predominantly, or completely  39. Were any of the following groups represented by at least one i n d i v i d u a l i n the program? (Check as appropriate) Children (under 12) Teenagers (13-10) Adults (19-50) Mature Adults (51-05) Seniors (over 65) (e.g., r e t i r e d , grandparents) Handicapped/chronically Very poor people Very r i c h people Communists Homosexuals-  ill  ._ Male  Female  N/A  86  40. I f thero were i n d i v i d u a l s belonging to any of the f o l l o w i n g groups, f i l l oi-t the f o l l o w i n g table i n d i c a t i n g whether they were shown i n ways consistent with the p o s i t i v e or negative stereotypes described, or n e i t h e r . Check the l a s t box i f there was not enough information to code. Check as many as apply. Please note that the stereotypes described are merely examples of p o s i t i v e and negative stereotypes f o r each category--others would also be relevant and these need not be there.  Seniors  Children  Teenagers  Handicapped/ Chronicallyill  Very poor  Very r i c h  Communists  Homosexuals  p o s i t i v e , e.g., wise, k i n d , grand parently, active  negative, e.g., dependent, s e n i l e , burden to others, i n a c t i v e , crabby  1 p o s i t i v e , e.g., sweet, innocent, chanting  2 negative, e.g., noisy, b r a t t y , messy .  p o s i t i v e , e.g., reasonable, s e n s i b l e , studying hard, involved i n many activities  negative, e.g., r e b e l l i o u s , obsessed with sexuality/rock music/being " i n " , inconsiderate  not stereotyped  not enough information to code or HA  p o s i t i v e , e.g., showing exceptional bravery, strength, and perserverance i n the face of difficulty  negative.e.g., h e l p l e s s , passive, dependent burden  not stereotyped  not enough information to code or HA  p o s i t i v e , e.g., negative, e.g.. warm, c a r i n g , lazy, ignorant, generous, s t r u g g l i n g untrustworthy . in a difficult situation  not stereotyped  not enough information to code or HA  p o s i t i v e , e.g., philanthropists, i n t e l l i g e n t , using money u n s e l f i s h l y  negative, e.g., m a t e r i a l i s t i c powerhungry, ruthless  not stereotyped  not enough information to code or HA  p o s i t i v e , e.g., negative, e.g.. well-intentioned, secretive, v i l l a i n s , c o l l e c t i v i s m sharing threat to free world s o c i a l responsibility  not stereotyped  not enough Information to code or' HA  1 p o s i t i v e , e.g., sensitive, caring, i n t o the f i n e a r t s  2 negative, e.g., limp w r i s t , l i s p , butch or other stereotypes ;  not stereotyped  3  3  not stereotyped  not . stereotyped  not enough information to code or HA 4 not enough information to code or HA  4 not enough information to code or HA  - 18 Part 6.  Clobal Impressions  41. How involving was the program? 1 not a l  2 somewhat  all  involving  3 very  :  involving  42. How would you best describe the p o l i t i c a l philosophy of this program? a p o l i1t i c a l  2 left-wing (liberal, socialist)  3 centre  4 right-wing (conservative, capitalist)  d e f i n i5t e l y p o l i t i c a l but not i d e n t i f i a b l y l e f t or right  43. Considering the country of o r i g i n of this program, how n a t i o n a l i s t i c was i t ? (e.g., "rah rah f o r our country"; our country or i t s c i t i z e n s are e s p e c i a l l y wonderful i n some way) definitely not n a t i o n a l i s t i c  can't decide unsure  definitely nationalistic  For each of the following questions (44-49) check the statement that best describes the take-away message. 44. Laws/authority/the state: no inform.  are always right  may be wrong, but s t i l l must be obeyed (The Law i s the Law")  may be wrong but can be worked around or bent as needed  may be wrong and the best avenue for change i s . working outside the system (revolution or vigilantes taking law into t h e i r own hands)  45. The powerful/authoritative/knowledgeable are: all males (b)  .1 a l l main stream nonethnics  mostly male but some important females 2 mostly mainstream nonethnics but some important ethnics  mixed  3 mixed  mostly female but some important Bales 4 mostly ethnics but some important mainstream non-ethnics  all females 5  all ethnics  N/A  6 N/A  88  - 19 46. How complex was the p l o t ( i n f i c t i o n ) or how complex were the issues/topics as presented ( i n non-fiction)? 1. r e l a t i v e l y simple  2. r e l a t i v e l y complex  47. How would you rate the q u a l i t y of the acting/announcing/interviewing o v e r a l l , r e l a t i v e to other programs of t h i s type? 1 r e l a t i v e l y poor  2 r e l a t i v e l y good  I f p o o r / s p e c i f y why • '"  48. How would you rate the q u a l i t y of the program i n a technical sense (sound, pictures, etc.)? 1 poor q u a l i t y (with or without s p e c i a l effects) 49. a)  _2_ professional ( s l i c k ) but no s p e c i a l effects  3 s l i c k and special e f f e c t s  Was there any p o r t r a y a l of r e l i g i o n or comments concerning r e l i g i o n i n the program? 1 definitely not  2_  3 can't decide unsure  4  5 definitely yes  I f yes, specify b)  I f r e l i g i o n or r e l i g i o u s people were discussed or portrayed how would you evaluate t h i s discussion or portrayal? 1. a) joke/humorously, w i t h d e f i n i t e negative impression b) joke/humorously, with d e f i n i t e p o s i t i v e impression c) both are present 2. a) s e r i o u s l y , with d e f i n i t e negative impression , b) s e r i o u s l y , with d e f i n i t e p o s i t i v e impression c) both are present  - 20 Part 7:  Subject information  50. How many hours do you t y p i c a l l y watch TV i n each time period? Please think c a r e f u l l y about each time and enter your most accurate estimate of your TV viewing f o r that period. Before Noon  Noon to 7:00 pm  After 7:00  pm  Monday Tuesday Wednesday Thursday Friday Saturday Sunday 51. Please l i s t i n order your f a v o r i t e shows. 1. Cmost favorite)' 2. ; 3. . 4. (Use the back of the page to continue i f necessary) 52. I f d i f f e r e n t from above l i s t the shows that you most often watch. 1.(watch most often) £.. _ _ _ ^  3.  ;  4. _ 5_ (Use the space at the end to continue i f necessary) 53. Most people have d i f f e r e n t reasons f o r watching t e l e v i s i o n . Such as r e l a x a t i o n , information, entertainment. Please l i s t the various reasons you watch TV i n order of importance to you. 1. (Most important) 2. __ ;  3.  .  ;  .  .  4. 5. (Use the space at the end to continue i f necessary) 54. On the following scale indicate how often you watch the show you've j u s t seen. every week  about twice each month  about once each month  about every few months  about a couple of times a year  never seen i t before  90  - 21 55. How much d i d you l i k e t h i s show? Hot at a l l . Disliked i t  Liked i t a lot  56. Age 57. Sex 58. Faculty 59. Year 60. How many-hours do you t y p i c a l l y spend studying i n each time period?- Please think c a r e f u l l y about each time and enter your most accurate estimate of your studying for that period. This does not include time spent i n lectures. Defore Hoon Monday Tuesday Wednesday Thursday Friday Saturday Sunday  '  Hoon to 7:00 pm '  A f t e r 7:00 .  . _  •  '  :  61. Ethnic background 62. Were you born i n Canada? Yes Ho I f no, how many years have you been here? 63. Were your parents born i n Canada? Yes I f no, how many years have they been here?  Ho  64. Were your grandparents born i n Canada? Yes I f no, how many years have they been here?  Ho  65. What i s the primary language spoken by you, your parents,  66. Educational goal (highest degree you hope to get) 67. Occupational goal 68. Father's current occupation  ;  I f unemployed or r e t i r e d , l a s t occupation  :  69. Father's highest education 70. Mother's current occupation [ I f unemployed or r e t i r e d , l a s t occupation 71. Mother's highest education _  __ -  , at home?  ;  pm  '  '  : l;  r  .  91  Appendix B  92  CODING SHliBT A.  General Information  1.  Program T i t l e  2.  Program ID  3.  Length of program in minutes  4.  Channel _ _ _ _  5.  Date of program  6.  Start time of program  7.  Coder  8.  Data of coding  9.  Program production source (country)  1<K .Canadian content:  A.M./P.M.  Yes_  (Circle one)  No  11. Interruption Count: tally it  __.  :  (number of times program was interrupted for ads, newsflashes,  etc.)  12. Notes taken during program: Bit or Segment topic 1.  Length  Countries mentioned .  Ethnic Groups Mentioned  3. 4. 5. 6.  8. 9 •  10.  12. 13. 14. 15. Female prominent characters in order of appearance (*ethnics!) 1. 5. • 9. ; 2. . 6. 10 3. 7. 11. 4. 8. 12.  13... 14.. 15.. 16..  Male prominent 1. _ 2... 3. 4.  13.. 14., 15.. 16..  characters.. 5.. 6. 7._ 8.  ;  ' 9._ 10 11. 12.  ;  93  B.  Commercials  1.  Were there any commercials?  No.  Yes.  2.  Mention of program sponsorship?  No,  Yes  C.  CRITC Coding  1)  Audience:  2)  Informative:  3)  Religious:  1. No  4)  Animated:  1. Live  5)  Program Type:  • t  1. Children  2. Other  1. No  2. Yes 2. Yes 2. Both  Non-fiction. Informative, 1. Direct instruction  3. Animated  Instructive 2. How to and informational  Non-fiction. Informative. Real World S. News/special 4. News/weather coverage 7. Hews and current events magazine • t  10. Documentary ( h i s t o r i c a l , visual arts, people biography) Non-fiction Entertainment 13. Reality programs 16. Game show  3.  Sports/covera,ge of event  3. Religious Ser^ic*  6. News analysis commentary ' 9.  Sports/magazine  11. Talk show/ interview  12. Documentary science & nature  14. Talk/show/variety  15. People and places magazine on location  17: Variety (dance, music, comedy, vignettes)  18. Cultural events or performances  31. Music videos Fiction - Comedy 19. Situational Comedy  20. Other comedy story  Fiction/Action/Adventure TXj.. Western  22.  Fiction, Other Drama 24. Medical  25. Horror/scary  26. soap opera's-  27. Other ,  28. Historical Drama  29. Classical Drama  30. Modern Classics  PoliceAdetective/ crime  23. Other  -  94  3 6) Kxpectations/Faaillarity:  7)  1.  Series with mostly the same major characters or presenting people and mostly the same format and settings from one program to next (very few new major characters occur in each episode), ( e . g . , Captain Kangaroo, Dallas, Cosby Show, Sesame Street), ( i . e . , high expectation of familiarity froa one episode to another).  2.  Series With generally the same format/narrator/announcer/continuing •' characters but at least half of the people in the program are different from one episode to the next. The setting may be the same froa one episode to another (e.g.; Johnny Carson, game shows or there may be major changes in the settings from one episode to another ( e . g . , Fantasy Island, Quincy, Wide World of Animals, Nature of Things).  3.  Series with different people and content in each episode but continuing format (e.g., NOVA), OR series with continuing characters over a few (2-10) episodes, i . e . , mini-series (e.g., Roots, Masterpiece theatre series), OR series with a completely unrelated story each program but the same person who introduces i t (e.g., Hitchcock, Twilight Zone).  4.  No continuing people/characters, content or settings ( e . g . , movies, special broadcasts, e t c . ) .  Is the.program in  1. story format  3. not sure, can't decide  2. non-story format 8}  Content Time Demands: (You need to use a watch for this question) Time usually devoted to story or b i t : Single episode (If story does not begin and end in one episode then code as multiple episode) 1. 2. ,3. 4. 5. .6.  Less than 5 minutes 5 to 15 minutes 16 to 30 minutes 31 minutes to 1 hour 1.C1 hours to 1 1/2 hours 1 1/7 hours or more  If you believe this episode is not typical of other episodes of this program, check here ,  Multiple episodes ( i . e . , single episode is not complete on its own; story continues) 7.  Finite number of episodes ( e . g . , mini series with a continuing story but with a definite ending)  8.  Indefinite number of episodes ( e . g . , soap opera where the story never really ends)  95  0. UBC CODING Part 1 : Catud*. the U . 8 . . irod other' Countries (Note: A* you weten the program, l i s t countries shown/mentioned in sequence .a,s they occur beside the numbers 1-15 on the front page of the coding sheet, but do not f i l l in the table below u n t i l the show ends.) (a)  If there was anything about Canada, the U . S . , or other countries in the program, check the following table as appropriate. Indicate whether th* country Was a major f o c u » , minor focus, or passing reference, as.w*J.l as whether i t was the subject of joke. Then indicate how It was portrayed, oc» balance, or whether there was not enough Information to code.  5  J3  not enough Information to code  _.  more positive emphasis  more negative emphasIs  passing reference  41  T3 O • • e <i — j n  i  Canada  1 minor focus  J major focus "  •  1.  U.S.A. Other countries (specify)  •  1. 2\  3.  A. 5. 6.  J.  .  • •  8. 9.  1 Q  lo. 11. -2. 13.  31  IA.  • •• rr >  15.  1  1 1 1  ,...J.  96  page 4(a) (b) Was there any reference to or portrayal of English Canada or English-speaking Canadians? no  unsure  yes  If yes, how much of the program's focus did i t have? major focus minor focus  passing reference joke context  (c) Was there any reference to or portrayal of Prench Canada or French-speaking Canadians? no  unsure  yes  If yes, how much of the program's focus did i t have? major focus minor focus  passing reference joke context  :  (d) Would a person from another country get any indication that Canada is a bilingual country? (e.g., portrayal of Francophones in an Anglophone context or vice versa, etc.) no unsure yes  97 5  finrt 2.  ?; (•)  North .ftwlw  were any North American ethnic minorities portrayed? 1.  (b)  K^nvrH^f  no  2.  The prominent characters In the program were:  :  mostly ethnics but some important mainstream non-ethnics  v5  5  a l l ethnics :  not  appliceblrt  Were there any verbal or visual jok.es about ethnic minorities? 1.  (d)  mixed  mostly mainstream 'non-ethnics but some important ethnics  a l l mainstream non-ethnics  (c>  yes  no  2.  yes  Complete this table for the ethnic individuals identified on the flrat page. If not enough information Was given in the program to code them here, do not put the* in this table, just note them on the front page. STRENGTH OF ETHNIC IDENTITY  LEVEL OF FUNCTIONING IIN MAINSTREAM NORTH  I AMERICAN socimr Individual's Iethnic group  sex  C v> a o -H Ti *4O H O B o a J W x U C C X v v 3 0V )I o§ aoi ac T>  Ti  iH  e w  <M  aoO 3 sO 9 XT  a •  JO  W D . ce a  Xi  U  2 S o.  V tl O TJ a o>-n •o o > c  VI O V o  v> u  1 «— * o TJ 1 -< •H 1 «*•B C 4 1 1 • o x -n 1 v •H CD 1 > ai vi 1 °V >. C I 1> 4)V .H o. OfiT-J4i1 1 11 •»f*— s1 C ol 1 V 1• V aI •1 a.o  FINAL IMPRESSION  1 VI 1 °4) iI *-< 1 *•a » 11 • C 1 *•> o 1 *o 1 TJ41 1 "T3 1 VI 1 r-i" \ * Kc 1 1 vi o 1 P 1 % 1 c U  1 ^ 1 >11 a1  I ° n I1 >414 u> cv — < I s | w" 1 1 J 4) 0 0  1  C  98  6 (e)  Was there any clear evidence (portrayal or reference) of racism or prejudice? no  3.  (a)  If yes, was it portrayed as: acceptable (e.g., let pass; not contradicted) unacceptable (e.g., comeback)  Does the program take place in a current North American setting? No  '"'(b)  yes  Yes  If yes, did the program portray the following? 1. Everyday home l i f e of a North American (rich or poor)  no  yes  2. Typical North American workplace (eg., factory, labour •job, office lobs, with focus on the workers) Specifv b r i e f l y  no  yes  nO  yes  3. Typical public l i f e (eg-, shopping, public transport, places providing service such as post offices, laundromats, banks, restaurants, with focus on people receiving service) Specify b r i e f l y  (c)  4. Health care or legal (non-police) settingsin N.America no Specify b r i e f l y  yes  5. Police or military settings in North America SDecifv b r i e f l y  no  yes  Were people shown doing spare-time activities?  no  yes  Specify b r i e f l y (d)  (i>  Was there any reference to or portrayal of television within the program? ( e . g . , show on TV personalities or programs, f i c t i o n a l program about a character who is a TV producer, people shown watching TV)  definitely not  can't decide  definitely yes  Specify b r i e f l y . ( i i ) Was there any reference to or portrayal of reading within the program? ( e . g . , show on l i t e r a r y personalities, f i c t i o n a l program about a writer, people shown reading) definitely not Specify briefly  can't decide  definitely yes  99  7 Part 3 : 4.  The Saxes  Were the prominent characters in the program (those you listed on the front page and did not later cross off) 1  2  mostly male  ;  mostly male but some Important females  3.  even mix of females and males  ii mostly female but some important males  5  mostly female  Did you notice any sexist comments/jokes/putdowns about females in this program? ( e . g . , "Woman driver" jokes, using the words "broad",or "dame", etc.) 1 . no  2 . yes  If yes, specify  ;  What's tfce bottom line message of this program re sexist messages about females? 1 . acceptable ( e . g . , comment l e t pass).  2 . unacceptable ( e . g . , comeback to comment)  Did you notice any sexist comments/jokes/putdowns about males in this program? ( e . g . , comments about men only having one thing on their mind, "dirty old man", "male chauvinist pig" coaments, etc.) 1 . no  2 . yes  If yes, specify  •. •  What's the bottom l i n e message of this program re sexist messages about males? 1.  acceptable ( e . g . , comment let pass)  ?. unacceptable ( e . g . , comeback to comment)  7. Were there any direct or indirect references to a relationship between mood or emotion/pain/dlscomfort and the menstrual cycle? 1.  No  2.  Tea  8. Was romance a part of the program? 1  not at a l l 9.  2  -  6ome romance  ".  :  "  3  .  romance is a major focus  Was sex (portrayal or reference) a part of the program? , In humans in animals1 . none 2 . some 1 . none 3 . a lot 2 . some - , 3 . a lot c  ; f ' - '  100 i  o  HOW WAS THE RELATIONSHIP DEPICTED?  s  •o  loving, caring 3"  cool, casual exploitive h o s t i l i t y i n c l u d e d (psych-=f o l o e i c a l / v e r b a l aggression )  ^  1  physical aggression included 3= AGES OF PEOPLE IN RELATIONSHIP* m  •o  n  child (pre-puberty) teenaaer  (13 to 19)  m  •»  adult (20 to 35) middle aged (35 to 55) RELATIONSHIP IS CLEARLY ROMANTIC SEXUAL BEHAVIOR f l i r t i n g verbally or shoving sexual interest kiss/touch but no clear sexua|. intentions . Kiss/touch w i t h clear sexual Intentions . clear implication of sexual but no outright portrayal clear portrayal of sexual act explicit reference to sex between people in relationship (not hint or innuendo) relationship .portrayed without any sexual behavior  • tr 0  o V-  1  rr  a • n  • n •a c »  < a It m IT  D  • v  ^  n  to C  • a  (6 "  0 9 3 C» >-" <t m %> rr rr ZT ~~ n  moo "* B 9  •c <• 9 ' a»-*  —-  t-t'  c o >- < a A  ' -.jh  ?  3  e» t 9 n  rr — 7 A  it a i . 3 0  '  *  n «T a  r*  < 3  <  1 . Relationship Is not clear  •  5. in a cumtitted love relattonahip, not living together  00  o 9 << T)  l»  1 m  PJ  0 rr 1 — O  X  n o 3  •• •  =  9-  j  «• c  •  | 1  B 0 |  (—  .9  a J „.  MOW WAS THE RELATIONSHIP DEPICTED^,.  O •  •0  1  to •  EEafg^HaaSHLHsuRimu  i~*  loving, caring'  ^ 3.  cool, casual exploitive  m  "'  .•  hostility included (psychological/verbal aggression)  £ g T>  physlca'l aggression .'included  »  ACES OF PEOPLE ^RELATIONSHIP child (pre-puberty) teenager (13 to 19) •  young adult (20 co 35) middle aged (35 to 55) older (55 or over)  1  RELATIONSHIP IS CLEARLY ROMANTIC •  SEXUAL BEHAVIOR f l i r t i n g verbally or showing sexual interest kiss/touch but no clear sexual intentions kiss/touch with clear sexual Intentions • . clear implication of jsexual act but no outrlcht oortrtival  i  i  clear portrayal of eejxual act(s) explicit reference tcj sex betveei people in indicated Relationship (not hint or innuendo) relationship portrayed without any sexual behavior i  102  11.  • vvfu V.I  Was any of the behavior shown/references concerned with any of the following! 4)  u a  M  — o S °  B O •H O  O 9  •  2a  8 < w  oo u  5 § au a « B. « JM  1. homosexual females  •O M t) O  _ >—  •o  Sa «  s  a,  •  «  o  41 •W >H V4 41 A O _; _ O <H  IT;  • * s- a  O f-lA u  o. a  41 a 41 ja W « 0 4)  a 4i £ 4i a o • fi S o ^  a. ti .o  homosexual males 2. group sex (3 people or more) 3. masturbation 4. sexual sadicm/masochism 5. exhibitionists/voyturs 6. tellshes 7.  tgsniesuals  1  8. tfansvostUes her (specify)  1. sexual assault I?, incett 12. Was there evidence regarding a double standard for females and males regarding sexual bebavlourT (Circle as many as apply) 1. no  2. yes, consistent with it  Specify  3. evidence or arguments against it •• -  ;  :  *«. joke . .-v •  13. Was there any portrayal of or reference to (circle as many as apply): 1. Contraception  2. Pregnancy  a) serious b) joke  a) serious b) joke  S. Spontaneous Abortion (miscarriage) a) serious b) joke  3. Sexually transmitted diseases (other than AIDs) a) serious b) joke  6. Induced Abortion a) serious b) joke  .4. AIDs a) serious b) joke  7. Prostitution a) serious b) joke  14. Was there any nudity shown in the program? 1. no  2. yes, female  3. yes, male  4. yes, child (age 4 or under)  15.  Were there any people ahown or referred to as being sex objects (.e.g., physically exposed or acting in a manner vhich excites interest In the opposite . sex). • ' • 1. No 2. Yes Complete the following table (whether or not sex objects were shown) • •o  41 O  «l Vl  ll  13  c V a x  o  60 V V>  •o o  a  tJ tr  Categories:  4) o ce x u ° s * o u  to  "— a «i B.iH O VI O X 3 41 CO X >, VI  —< o  X 4) •H_T"> tg J i  gg o  z»  ce c  8^ a.  41 w a x  oo si  1  4i  a  o <W  i-l  v> >, C eg  41 Vi a «-> tl Vi Vi o  a.  a  VI VI  S3  eg  O iH  ti S 41 XI O  o  VI K v 4i a « O. 4) Vi 60  M 41  si  1-1 >VI  o  n  » 41 ,-1 VI O  o u  «l  •H 4)  1-1 > «  X  o  o  •-I  VI  x vi 41 4  o a« v>  o  a  41 60 OS Q  ° s  TJ O 4i a  54) 8 X  e  O Vi 1-1,41 <V| VI 41 U V tl i-li-l  H£ 4) > a u  <H 41 O X VI VI O Vl  o  .  41 a  While answering the following questions (16 regarding females and 17 regarding males), be sure to look at the characters you have listed on the front page. 16. ^ _ Check here i f there were no females in the program and go to #17. (a)  The prominent adult females (relative to other females in this program) were  predomlnantly in traditionally female occupations (include homemaker) (b)  equal mis o f traditionally and nontradittonally female occupations  predominantly in nontradltionally female occupations  N/A (not enough information re: occupation to decide)  For the Prominent adult females, which types of activities were emphasized (airtime focus) in the program? (Check more than one i f necessary for different individuals)  major focus on occupation  major focus on home/family role  equal focus on home/family role and social l i f e  equal focus on occupation and social l i f e  .major focus on soc i al life  equal focus on occupation and home/family role  equal focus on occupation, home/ family role, and social Hi>  JL  N/A  104  12 (c)  The background adult females (relative to other more prominent females in this program) were:  predomih^nUy in .traditionally female occupations (include homemaker) (d)  equal mix of traditionally and nontradit^pa^lly female occupations  predominantly in nontraditionally female occupations  H/A or not enough information re: occupation to decide  Were there any adult females who behaved in a nontraditional way (e.g., strong in an emergency, not dependent on males for guidance, assertively saying What's on her mind, etc.)? definitely no nontraditional female behavior  not sure can't decide  nontraditional female behavior definitely present  Specify, If (4) or (5) non-traditional female behavior present, was i t portrayed as (cirele a l l that apply): a) serious,,. O positive (e.g., successful/good/rewarded/competent) b) joke,.. . d) negative (e.g., unsuccessful/bad/punished/incompetent) (e)  Were there any adult females who acted in a traditional way (e.g., dependent on a male for guidance, falls apart in crisis, non-assertive, etc.)? definitely no traditional female behavior  np'jfc sure can't decide  traditional female behavior definitely present  Specify, If (4) or (5) traditional female behavior present, was i t portrayed as (circle a l l that apply): . *) serious c) positive (e.g., successful/good/rewarded/comp'etent) b) joke ' d) negative (e.g., unsuccessful/bad/punished/ihcompetent) (f)  Were any females portrayed in a clearly positive way in the program?  J_l  definitely no positive females  not sure can't decide  positivla females definitely present  105  13 (g)  Were any females portrayed In a clearly negative way in the program? -1 definitely no negative females  17.  2  3 not sure can't decide '  A  5 negative females definitely present  Check here i f there were no males lei the program and go to #18. "'" (a)  The prominept adult males (relatlye to other males in this, program) were:  predominantly in traditionally male occupations  (b)  5 equal focus on home/family role and social l i f e  major focus pri home/family role  major focus oh social "life  N/A or not enough information re: occupation to decide  equal focus on occupation and home/family role .  7 equal focus on occupation, home/ family role, and social l i f e  equal focus on occupation and social l i f e  8 N/A  The background adult males (relative to other more prominent males in this program) were:  'predominantly in traditionally male occupations  (d)  predominantly in nontradltionally male occupations (include homemaker)  For the prominent adult males, which types of activities were emphasized (alrtime focus) in the program? (Check more than one i f necessary for different individuals)  major focus on occupation'  (c)  equal mix of traditionally and nontraditionally male occupations  equal mix of traditionally and nontradltionally male occupations  predominantly in nontradltionally male occupations (include homemaker)  H/A or not enough information re: occupation to, decide  Were there any adult males who behaved in a nontraditional way (e.g., shown crying, caring for children, doing domestic chores,., sensitive to the needs of others)? :  definitely no nontraditional male behavior. Specify.  not sure can't decide  nontraditional male behavior definitely present  106  14 If (4) or (S) non-traditional male behavior present, was i t portrayed as (circle a l l that apply): a) serious b) joke (e)  c) positive (e.g., successful/good/rewarded/competent) d) negative (e.g., unsucceseful/bad/punished/incompetent)  /  Were there any adult males who acted in a traditional way (e.g., skirt chaser, tough/macho, calls the shots, l i f e revolves around job and self)? 1 definitely no traditional male behavior  3 not sure can't decide  2  4  5 traditional male behavior definitely present  Spec i f y  __  If (4) or (5) traditional male behavior present, was i t portrayed as (circle a l l that apply): a) serious b) joke (f)  c) positive (e.g., successful/good/rewarded/competent) d) negative (e.g., unsuccesBful/bad/punished/incompetent)  Were any males portrayed in a clearly positive way in the program? 1 definitely no positive males  (g)  2  3 not sure can't decide  4  5 positive males definitely present  Were any males portrayed in a clearly negative way in the program? 1 definitely no negative males  2  3 not sure can't decide  4  5 negative males definitely present  18. Were there any messages indicating females are subordinate to or worth less than males; e.g., parents disappointed that i t was a g i r l baby; any mention of men trading or owning women; women taking a back seat to and/or being obedient to men; parents preferring a son over a daughter (because he will carry on the family name, business, etc.)? 1. no 2. yes If yes, specify what gave you this impression Was thie portrayed as: J 1. acceptable \ 2. unacceptable • '. :  :  '•  19. Were there any messages Indicating that males are subordinate to or worth less than females, e.g., parents disappointed that i t was a boy and they wanted a g i r l baby; parents favoring a daughter over a son? 1. no 2. yes If yes, specify what gave you this impression Was this portrayed as: ___ • 1. acceptable : 2. unacceptable ; :  . ;  !  15 Part 4:  Autression  In the following question, an aggressive act it ona which la Intentional, interpersonal, and (unless otherwise stated) visually portrayed and 1% against , humans (alive or dead), humaneid ccjsc^-s (e.g., robots), animals, or any representations"of humans or animals (e.g., cartoons). This includes, antisocial. acts with the potential to do harm but from which the victim escapes uninjured. For example, intentionally trying to run someone down with a vehicle, whether successful or not, is aggression. But if someone accidentally crashes into another vehicle in which a person is Injured, this is not aggression (unless the person causing the accident was engaged deliberately in an antisocial act, e.g., speeding without regard for pedestrians). Anothor exsmple of aggression would be intentionally setting fire to a building, whether or not i t resulted in injury. On the scales for aggressive activity, avoid using numbers 2 and 4 i f at a l l possible*, they mean toward aggression or no aggressIOBI.UBS only i f you cannot possibly assign a 1 or 5. These questions do not relets to quantity, only whether aggression occurred. 20. (a)  Was there any physical aggression shown in this program? definitely no aggression shown  (b)  Was there any physical aggression by females shown in this program? definitely no aggression by f^-ales  (c)  physical aggression by females definitely present  can't decide unsure  Vas there any physical axKressloa- axainst females shown in this program. ••• • i • definitely no aggression against females  (d)  physical aggression definitely present  can't decide unsure  2  • • "3 can't decide unsure  • •• ,  4' '  .  ' ... 5 physical . ' aggression against females . definitely present  Was there any physical aggression by males shown in thia,jprogram? •i •> .<.-.  definitely no aggression by males  can't decide unsure • •  physicsaggression by males definitely present  108  16 (e)  Was t h e r e any p h y s i c a l a g g r e s s i o n a g a i n s t males shown i n t h i s program?. 1 definitely no a g g r e s s i o n a g a i n s t male6  (f)  . 3 can't d e c i d e unsure  2  3 can't d e c i d e unsure  '  "-5 •-' '  physical aggression against sales definitely present  4  5_  definitely had reference to physical aggression  definitely violent  can't decide unsure  Was t h e r e any r e f e r e n c e to V i o l e n c e i n t h i s program? 1 d e f i n i t e l y no references to v i o l e n c e  (1)  '  Was t h e r e any v i o l e n c e shown i n the program ( i . e . , extreme p h y s i c a l a g g r e . s l o n . that i s potentially fatal)? definitely nonviolent  (h)  4  Was t h e r e any r e f e r e n c e made to p h y s i c a l a g g r e s s i o n i n t h i s program: 1 d e f i n i t e l y no reference t o physical aggression .  (g)  2  2  3 can't d e c i d e unsure  4 definitely bad r e f e r e n c e to v i o l e n c e  What i s the bottom l i n e message o f t h i s program re t h e a c c e p t a b i l i t y o f p h y s i c a l a g g r e s s i o n as^ a method of c o n f l i c t r e s o l u t i o n ? (Check more than 1 i f more t h a n 1 message was p e r c e i v e d . )  physical aggression is clearly acceptable  can't decide .unsure  physical aggression is clearly unacceptable  N/A o r not enough information t o code  Specify, (j)  What Is the bottom l i n e message o f t h i s program re the s u c c e s s f u l n e s s o f p h y s i c a l a g g r e s s i o n as a method o f c o n f l i c t r e s o l u t i o n i n the s h o r t term r a t h e r than l o n g term sense? (Check more than 1 i f more than 1 message was perceived.)  physical aggression is clearly successful ( e . g . , s h o r t term g o a l s were a c h i e v e d through p h y s i c a l aggression) Specify  can't decide 'unsure  p h y s i c a l aggression is c l e a r l y not s u c c e s s f u l (eg. s h o r t term g o a l s were not a c h i e v e d )  N/A or not enough information to code  109  17 21. (a)  I I there any verbal/psychological aggression in the program?  definitely not at a l l verbally/ psychologically aggressive  can't docide  definitely verbally/ psychologically aggressive  Specify. (b)  What is the bottom line message of this program re the ecceptabilty of verbal or psychological aggression as a method of conflict resolution!?, ' (Check more than one i f more than one message Was perceived) 1 2 3 4 5 " verbal/psychological N/A or verbal/psychological can't decide aggression is clearly not aggression is clearly unacceptable enough acceptable information to code ;  !  Specify. (c)  What is the bottom line message of this program re the successfulness of verbal/psychological aggression as a method of conflict resolution? (check more then 1 i f more than 1 message was perceived).  verbal/psychological is clearly successful (e.g., short term goals were achieved •. • through this behavior)  can't decide unsure  verbal/psychological is clearly not successful (eg. short term goala were not achieved through this oehavior)  N/A or not enough Information to code  ;  Specify. 22. Does the program give the impression that the world is a dangerous place Oe.-g'., because of war, physical aggression of others, natural disasters, fatal traffic accidents etc.)? program gives no impression that the world is-a dangerous placeSpecify.  some people mlght get the impression that the world is a dangerous place  impression l r clearly given that the weir Id is a dangerous place  110  18  23. (•) <b)  Were any reel guns shown in the program? Who had them?  (Check a l l that apply)  Military Good citizen <c>  No.  Yes  -  Police Other authority. Bad citizen^  How were the guns used? (Check a l l that apply) ;  some or a l l not used defense of self/others, defense of property  aggression intimidation/coercion, recreation,  24. Was there any portrayal of.or reference to death of humans? I. no  2. yes  If yes. put checks below as appropriate for individual deaths; f W the deaths of several.people (2 or more) use "s" (for several) instead of a check as appropriate'. '' •••!..-• Mov . ••• i • r Death Portrayal « 30 § •J :  3  in  J3  0  >•  0  u o 0 £ •w 0  > -o  *  V  «  2  •S3  0 0  <M  O;  u A  ii O "O  «  v  -o <*  If  9  U  w oo 0 0  •a «  « «  0. o 41  3  •k- a -  o  '« '  H  •  -I u  4 0 JC 0 «s  0  te a e o  0  o  0  c o  ,0> . w r-  0 W .4)  "80  o — >  a. 0  0 o 0 A  "3  s  X to. 0 0 —'  19 Part 5.  Issues. Controvertias. and Dilemmas  25. (a)  Does this program:  1.  Present no evidence that problems (either serious or minor) exist.  2.  Deal with less serious problems such as family disagreements, boyfriendgirlfriend, wife-husband problems, or other problems that are relatively unimportant in the long run. Specify "  3.  Deal with serious problems and issues (either having considerable Impact on the l i f e of one individual—e.g. abortion, divorce, alcoholism, or having impact on the lives of many—e.g. pollution, corruption in politics). Specify •  (b)  Were the problems or issues (draw lines from above if you've checked both (a) 2 and 3): 1. central to the plot 3. portrayed seriously  (c)  The take-home message from this program regarding issues and controversies is that they are usually: (check one) 1. clear-cut black and the right are clear  26. (a)  2. Incidental 4. portrayed as funny  (relatively white) and answers -  2. not clear-cut ' 3. not clear-cut . (shades of gray, (shadea of grey, and some anstfmrs and there are ho are batter then^ right or better others) answers)  4. N/A ,  • •  Were there explicit (surface structure) political comments or connotation in the program? (circle highest applicable number) - . 1 2 2 4 <5 definitely can't decide definitely i:  <in<nr«  (b)  j  Was the content more balanced or more partisan? 1. balanced  2. partisan  • Specify briefly 27. (a)  yes  •  •-  ••  ...  ••  ••»-• • •  ,  Was there any portrayal of religion or comments concerning religion in the program? 1 definitely not  2  3 can't decide unsure  4  S definitely y«*  112  20 ('$)  I f r e l i g i o n o r r e l i g i o u s p e o p l e were p o r t r a y e d o r d i s c u s s e d , i n d i c a t e how ( c i r c l e as many as a p p l y ) f (i)- : j (ii) (iii) (ly) :  P a r t 6.  joke/humorously, w i t h d e f i n i t e n e g a t i v e i m p r e s s i o n ' joke/humorously, w i t h n e u t r a l o r p o s i t i v e i m p r e s s i o n s e r i o u s l y , w i t h d e f i n i t e negative impression s e r i o u s l y , w i t h n e u t r a l o r p o s i t i v e impression  P o r t r a y a l of People  28. O v e r a l l ! ( g l o b a l i m p r e s s i o n ) i s t h i s program p r i m a r i l y about n i c e , d e c e n t , • w e l l - i n t e n t i o n e d people?  mostly 29. (a)  Was the show concerned  not a t a l l TF (b)  mostly n i c e  even mix o f n i c e and n a s t y  nasty  w i t h the r i c h and/or famous?  somewhat  predominantly, or c o m p l e t e l y  2 o r 3, were they .2. r e a l  \. f i c t i o n a l  life  D i d they l i v e i n t h e 2.  (c>  l i j present  (d)  i j j r e a l l i f e c h a r a c t e r s , were they 1 popular itertainment  2  h i g h brow e n t e r t a i nm.ent  past  3 science or politics  royalty, establishment  Other  Specify:. 30.  (a)  V**re any o f the f o l l o w i n g groups r e p r e s e n t e d by a t l e a s t one i n d i v i d u a l i n t h e I program? (Check as a p p r o p r i a t e )  C h i l d r e n (under 12) TeenaBers (13-18) Adults (19-50) Maturf A d u l t s . (51-65) Senloi s (over 65) (e.g. r e t i r e d , grandparents)  .  :  Sex U n c l e a r Handicapped/chronically i l l Very?feogypeefrl-e • - : , , , ... Very r i c h people Communists Homosexuals  Male  Female  21 (b)  If there were individuals belonging to any of the following groups, f i l l out the following table indicating whether they were shown in ways consistent with the positive or negative stereotypes described , or neither. Check the lai t box If there was not enough Information to code. < Check as many as apply. Please note that the stereotypes described are merely examples of positive and negative stereotypes for each category—others would also be relevant and these need not be there.  IS OS O  W  oa OS H Z  al W sa  POSITIVE STEREOTYPE GROUP  VEGATIVE STEREOTYPE  Seniors  wise, kind, grand parently, active  dependent, senile, burden to others, inactive, crabby _  Children  sweet, innocent, charming  noisy, bratty, messy  Teenagers  reasonable, sensible, studying hard, involved in many activities  rebellious, obsessed with sexuality/rock music/being "In", Inconsiderate  Handicapped/ Chronicallyill  showing exceptional bravery, strength, and perserverance in the face of difficulty  helpless, passive, dependent burden  Very poor  warm, 'carinsJY generous, struggling In a difficult situation  laty. Ignorant, untrustworthy _  Very rich  philanthropists, intelligent, using money unselfishly  materialistic powerhungry, 'ruthless  Communists  well-intentioned, collectivist, sharing social responsibility  secretive, v i l l a i n s , threat to free world  Homosexuals  sensitive, caring, Into the fine arts  limp wrist, l i s p , butch or other stereotypes  _  L  Cu  *H £O  t£  u  H  V) H O Z  114  22 Part 7.  Global Impressions  31. How involving was the program? 1 not at all 32. (a)  _2 somewhat involving  Being humourous was:  not at a l l intended (b)  3 very involving  a minor goal of the show  a major goal of the show  How humorous were the parts that were intended to be humourous?  1 not at all humourous/none  2 some/mildly humourous  3 ' most/very humourous  The final questions deal with the overall message you are left with. These messages are not necessarily specifically stated or even intentionally implied by the producers. 33. How would you best describe the political philosophy of this program? 1 apolitical  _2 left-wing (liberal, socialist)  3 centre  4 right-wing (conservative, capitalist)  5 defini tely political but not identifiably left or right  34. Considering the 'country of origin of this program, how nationalistic was it? (e.g., "rah rah for our country"; our country or its citizens are especially wonderful in some way) definitely not nationalistic  can't decide unsure  definitely nationalistic  For each of the following questions (36-39) check the statement that best describes the take-away message. 35. Laws/authority/the state: I no inform.  2 are always right  ' 3 may be wrong, . but s t i l l must be obeyed ("The Law is the Law")  4 may be wrong but can be worked around or bent as needed  5 may be wrong and the best avenue for change is working outside the system (revolution or vigilantes taking law into their own hands)  115  23  3 6 . The m i l i t a r y and/or p o l i c e : (a)  1 need more authority  2  have tbe r i g h t amount of a u t h o r i t y  have t o o much authority  are a p p r o p r i a t e l y strong  are t o o s t r o n g  N/A  (b) need t o be stronger ( i n terms of numbers and/or money, equipment)  N/A  (c) some are i n e p t and some are competent  are i n e p t  are competent  N/A  37. The p o w e r f u l / a u t h o r i t a t i v e / k n o w l e d g e a b l e ' a r e : (a)  1 mostly males  7_ m o s t l y male but some Important females  3  mixed  4 m o s t l y female b u t some i m p o r t a n t males  mostly females,  (b) a l l mainstream non-ethnics  m o s t l y mainstream n o n - e t h n i c s but some important ethnics  mixed  mostly ethnics but some important mainstream., non-ethnica  all ethnics  38. C u r r e n t p r o t e c t i o n o f t b e environment i s not sufficient  P a r t 8:  is  i s top strong f o r economic good h e a l t h  sufficient  N/A  S t r u c t u r e o f Program  39. When does t b e program t a k e p l a c e 1 current (within a decade)  '  2  past i . e . historical  future  combination specify: 1 & 3 1 & 2 . 2 A 3. 1, 2 . 3  40. How complex was t h e p l o t ( i n f i c t i o n ) o r how complex were t h e I s s u e B / t o p i c s as p r e s e n t e d ( i n n o n - f i c t i o n ) ? 1. r e l a t i v e l y s i m p l e  2.  r e l a t i v e l y complex  N/A  N/A  116  24  41. How (a)  hard/easy was  the program t o  C o u l d someone l e a v e the room f r e q u e n t l y o r tune In d u r i n g the m i d d l e of the program and s t i l l f o l l o w the s t o r y or comprehend the p a r t s of the program ( e . g . , news) they d i d see? no  (b)  follow?  yes  Now,  c i r c l e (b) or ( c ) below.  When i n the room w i t h the TV, c o u l d someone d i v i d e a t t e n t i o n f a i r l y r e a d i l y between TV and o t h e r a c t i v i t i e s (eg. k n i t , l o o k a t a m a g a z i n e ) , by l i s t e n i n g and o c c a s i o n a l l y l o o k i n g a t the screen? or  (c)  would i t be d i f f i c u l t t o d i v i d e a t t e n t i o n w i t h o t h e r a c t i v i t i e s because they would be l i k e l y t o miss something e s s e n t i a l ( i . e . , i n o r d e r t o u n d e r s t a n d the program i t i s n e c e s s a r y t o both watch the s c r e e n and listen)?  42. How would you r a t e the q u a l i t y of the a c t i n g / a n n o u n c i n g / l n t e r v l e w i n g o v e r a l l , r e l a t i v e t o o t h e r programs o f t h i s type? 1 r e l a t i v e l y poor 43. How  2 r e l a t i v e l y good  I f poor, s p e c i f y  why  would you r a t e the p r o d u c t i o n v a l u e s o f the program i n a t e c h n i c a l sense? 1 poor q u a l i t y (with or without s p e c i a l effects)  2 professional ( s l i c k ) but no s p e c i a l effects  3 s l i c k and special effects  44. What was p a r t i c u l a r l y n o t a b l e about t h i s program? nothing Positive:  Negative:  Neutral:  •  •  117  25 45. Need bo consult re: the specific questions listed for each of the following pages: I. ' 2' ' '"' 3. 4. 5. 6. 7. 8. ' 9. ' • 10. II. 12. 13.  r :  "  ' '' ' " :  |  '  -  •  . :  ' '• "•  14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26.  After viewing the program, fast forward through commercials to the start of the following program. Is the program the same one as specified in the TV Guide Listings? ; No  Yes  Name of program following program viewed If there are discrepancies, note them here.  • •  118  Appendix C  Audience Research Project Volunteer Information  119  If you are willing to participate in this research, please answer the following questions. A l l of the information will remain confidential. When you participate this sheet will be given back to you. You will not be asked to put your name on the questionnaire you complete then. This will ensure that the data you provide w i l l be confidential. If you decide not to participate this sheet w i l l be destroyed. We are not able to pay a l l participants, but at the end of the project we w i l l randomly select the names of three participants. The f i r s t name selected w i l l win $100, the second $75, and the third $50.  Name  Phone number  Best time to call Age Sex  -  Faculty Year Ethnic background Were you born in Canada? Yes No If no, how many years have you been here? Were your parents born in Canada? Yes If no, how many years have they been here?  No  Were your grandparents born in Canada? Yes If no, how many years have they been here? • What i s the primary language spoken by you, your parents, \ at home? Father's current occupation If unemployed or retired, last occupation Father's highest education  ;  "  Mother's current occupation If unemployed or retired, last occupation Mother's highest education  No  120  Appendix D  121 STATISTICAL CONSULTING AND  RESEARCH L A B O R A T O R Y  D E P A R T M E N T OF STATISTICS T H E UNIVERSITY OF BRITISH COLUMBIA  TO:  David Wotherspoon Psychology,  REF:  File 87-10-099  UBC  FROM: Peter Schumacher  DATE: Nov. 10, 1987  Managing Consultant  Suppose you have a fc-point ordinal scale, which N subjects use to answer a question. A single expert also gives a score, which we will denote X*.  The proposed statistic will summarize the degree  to which the subjects agree with the expert. Moreover, it will also be reasonable to use the statistic to compare across different scales and X* 's.  Definition of the Statistic.  The proposed statistic differs algebraically from what we discussed at our meeting, because further examination revealed that there was no simple and elegant way of standardizing the scale of that statistic. Thus, for example, one could not have used it to compare two questions with different scales, or even two questions with different expert ratings but on the same scale. However, the spirit of the original statistic is retained here, in that it incorporates the total 'distance' of subjects from the expert. Essentially, one assigns a positive score to each subject who 1  122  agrees with the expert, and also to each subject who disagrees. Then one takes the sums of scores over the two respective groups; call these sums A and D respectively. The new statistic is  A-D A + D,  which we shall call T. (The notion of taking this kind of ratio is shared by the Goodman-Kruskal Gamma coefficient for agreement in ordered contingency tables. [1]) It follows that if no-one agrees with the expert, then A — 0, and T — — 1, regardless of the degree of disagreement of any individual. If everyone agrees with the expert, then D = 0 and T = +1. These facts hold for any scale, any value of the expert's rating, and any number of subjects; these are necessary conditions for inter-question comparisons. Indeed, these facts hold true for any scoring scheme as well, which shall be examined below. Recall that we discussed the concept of chance agreement. Suppose subjects scored randomly, paying no attention to the question's meaning. One would expect an approximately even (or uniform) distribution of scores. Thus there would be some subjects who agreed with the expert by chance alone. Intuitively, the statistic T should have an identifiable point corresponding to expected chance agreement. A natural choice is zero, for then positive values of T indicate a level of agreement in excess of, and negative ones a level of agreement below, this benchmark. The trick to accomplish this is to define the scores assigned to subjects in a certain way. Divide the group into those who agree with the expert, and those who don't. Denote.by Oj the score given to the i  ,h  agreeing subject; and let dj be the score for the j  th  disagreeing subject. Now define dj to  be the number of points on the scale by which subject j's score differs from the expert's, ignoring the direction. And D is the sum of these scores. Note that we originally discussed using the square of dj at our meeting; however, this will tend to inflate the effect of outliers on the statistic, and has no other apparent advantage. Subjects who agree with the expert will all be assigned the same score: a, = a, for all i. We want to define 'a' so that under uniform (random) scoring, T = 0. The choice of 'a' to do this will  2  123  depend on the number of points k on the scale, on the expert's rating X',  and on the number of  subjects N. So for a given question, a will have to be worked out using these known quantities. In what follows, a subscript ' 0 ' will denote values under uniform scoring. Using the definition of T as (A — D)/(A N/k  + D), setting Ao — D  0  subjects to fall on each point. 0  x a, one gets a = (k/N)  0  Since the ay's are just the number of points by which the  subjects disagree with the expert, D AQ = (N/k)  will make T = 0. Under uniform scoring, we expect  can be calculated. A n d using Ac — D , 0  plus the fact that  x Do-  A n example may help to make this more clear. Suppose one had a five-point scale, with 100 subjects, and that the expert circled the second point from the left on the scale. Then the possible values of the dj are 1, 1, 2, and 3. Under random uniform response behavior, 20 subjects fall on each point. Then D  0  = 20 x 1 + 20 x 1 + 20 x 2 + 20 x 3 = 140. A n d a = (5/100) x 140 = 7. Thus each  agreeing subject will receive a score of 7. Now suppose the actual observed totals of respondents falling at each point were (15,30,25,15,15). Then D = 1 5 x 1 + 2 5 x 1 + 1 5 x 2 + 1 5 x 3 = 115, A = 30 x 7 = 210, and T = ( 2 1 0 - 115)/(210 +115) = 0.292. The fact that T is positive is consistent with the observed number of subjects in agreement with the expert, 30, being in excess of 20, the number expected under randomness.  Further Comments. T is essentially a scaled version of the total agreement less the total disagreement, A — D, where the scores a, and bj define numerically what is meant by '(dis)agreement'. Dividing by A + D then standardizes the range of T , so that inter-question comparisons are possible. One cannot use A — D alone, for the possible values of D (and ultimately of A through a), depend on it and X*, and so differences or similarities in A — D for two questions represent not only subject's feelings, but also scale properties and the expert response. There is one drawback to the definition of T. When no-one agrees with the expert, T = — 1, regardless of the magnitude of disagreement. This can be seen from the definition: when no-one 3  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0097819/manifest

Comment

Related Items