UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Interactive evaluation : a user-oriented process to assist housing programme reformulation McAfee, Rosemary Ann Pickard 1975

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1975_A1 M32_6.pdf [ 18.78MB ]
Metadata
JSON: 831-1.0093605.json
JSON-LD: 831-1.0093605-ld.json
RDF/XML (Pretty): 831-1.0093605-rdf.xml
RDF/JSON: 831-1.0093605-rdf.json
Turtle: 831-1.0093605-turtle.txt
N-Triples: 831-1.0093605-rdf-ntriples.txt
Original Record: 831-1.0093605-source.json
Full Text
831-1.0093605-fulltext.txt
Citation
831-1.0093605.ris

Full Text

(  INTERACTIVE EVALUATION: A USER-ORIENTED PROCESS TO ASSIST HOUSING PROGRAMME REFORMULATION by  ROSEMARY ANN  (PICKARD) MCAFEE  B.A. U n i v e r s i t y o f B r i t i s h Columbia, 1962 M.A. U n i v e r s i t y o f B r i t i s h Columbia, 1967  A THESIS SUBMITTED IN PARTIAL FULFILLMENT • OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY  i n the School o f Community and R e g i o n a l P l a n n i n g (Interdisciplinary)  We accept t h i s t h e s i s as conforming i  THE  required  t o the  standard  UNIVERSITY OF BRITISH COLUMBIA 1975  In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the  require-  ments f o r an advanced degree a t the U n i v e r s i t y of B r i t i s h  Col-  umbia, I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and  study.  I f u r t h e r agree  that permission f o r  e x t e n s i v e copying of t h i s t h e s i s f o r s c h o l a r l y purposes be granted by the Head o f my ives.  I t i s understood  Department or by h i s r e p r e s e n t a t -  t h a t copying or p u b l i c a t i o n of t h i s  t h e s i s f o r f i n a n c i a l g a i n s h a l l not be a l l o w e d without written  permission.  School of Community and R e g i o n a l P l a n n i n g . The U n i v e r s i t y of B r i t i s h Vancouver, B.C.,  may  Canada.  Columbia  my  ii  ABSTRACT  T h i s d i s s e r t a t i o n evolved of  three trends.  The f i r s t  an i n c r e a s i n g concern uents.  T h i s concern  and r e s o u r c e s  from the i d e n t i f i c a t i o n  i s t h a t governments are e x h i b i t i n g  f o r the s o c i a l w e l f a r e of t h e i r  constit-  i s r e f l e c t e d i n the expanded number o f ,  committed t o , s o c i a l a c t i o n programmes.  The  second i s t h a t c i t i z e n s are becoming i n c r e a s i n g l y d e s i r o u s o f p a r t i c i p a t i o n i n the f o r m u l a t i o n of s o c i a l p o l i c i e s designed t o provide f o r t h e i r welfare.  And, f i n a l l y ,  as s o c i e t a l  change p o l i c i e s must be a l t e r e d t o r e f l e c t emerging  goals  directions.  E v a l u a t i o n o f e x i s t i n g s o c i a l a c t i o n programmes p o t e n t i a l l y p r o v i d e s d e c i s i o n makers with an i n d i c a t i o n o f the e x t e n t t o which government a c t i o n s a s s i s t i n d i v i d u a l s t o share of  life  enjoyed  the q u a l i t y  by s o c i e t y - a t - l a r g e .  U n f o r t u n a t e l y , an assessment o f e v a l u a t i o n attempts i n d i c a t e s f i n d i n g s have seldom made a s i g n i f i c a n t c o n t r i b u t i o n to  policy reformulation.  The q u e s t i o n a r i s e s as t o whether  e v a l u a t i o n o f government programmes i s , i n i t s e l f , or whether e x i s t i n g e v a l u a t i o n processes  unattainable  merely apply  inapprop-  r i a t e methodologies t o s o c i a l a c t i o n programme s i t u a t i o n s .  Con-  s i d e r a t i o n of t r a d i t i o n a l e v a l u a t i o n methods i n l i g h t of e v o l v ing  p l a n n i n g thought suggests  the l a t t e r c o n d i t i o n may w e l l  Programme e v a l u a t i o n must possess in  apply.  the c a p a c i t y t o a s s i s t  the c o n c e p t u a l i z a t i o n of s e n s i t i v e i s s u e s t o be e v a l u a t e d ,  p r o v i d e a framework f o r e x e c u t i n g the study  i n the context of  an e v o l v i n g programme, and make p r o v i s i o n f o r the d i s s e m i n a t i o n  and use of f i n d i n g s i n a manner amenable to p o l i c y  implement-  ation . In  a d d i t i o n , d e c i s i o n making through  the  political  process i n v o l v e s b a r g a i n i n g between d i v e r s e i n t e r e s t s . undertake  To  t h i s task r e q u i r e s an a p p r e c i a t i o n of the range of  a l t e r n a t e o p t i o n s f o r those a f f e c t e d whether i t i s the p u b l i c a t - l a r g e or a s p e c i f i c c l i e n t group.  Realistic tradeoffs  o n l y be made i n l i g h t of a knowledge of p e r s o n a l and  can  societal  impacts. T r a d i t i o n a l programme e v a l u a t i o n models assume s t a t i c programmes f o r which i n i t i a l g o a l s are a v a i l a b l e , c l e a r remain v a l i d over time.  and  T r a d i t i o n a l models seldom d i r e c t l y  a s s e s s user a t t i t u d e s t o programme dimensions  and make no  p r o v i s i o n f o r c i t i z e n p a r t i c i p a t i o n i n the e v a l u a t i o n p r o c e s s . Such p a t t e r n s appear i n c o n s i s t e n t w i t h emerging p l a n n i n g t h e o r y . Planners and p o l i t i c i a n s are c a l l i n g f o r new  processes to pro-  v i d e p o l i c i e s which are r e s p o n s i v e to v a r i e d user needs and anticipate social costs. procedures  An a n a l y s i s of e x i s t i n g e v a l u a t i o n  suggests t r a d i t i o n a l methods f a i l  to f u l f i l l  these  criteria. The  search f o r a more r e s p o n s i v e e v a l u a t i o n process  led  the r e s e a r c h e r t o s p e c u l a t e on the i m p l i c a t i o n s of  ing  members of the concerned  uation process.  I t was  includ-  r e c i p i e n t p o p u l a t i o n i n the  hypothesized  eval-  t h a t i f r e c i p i e n t s of  gover  nment programmes were d i r e c t l y i n v o l v e d i n the e v a l u a t i o n proces t h e i r a n a l y s i s of programme g u i d e l i n e s would a s s i s t r e d i r e c t i o n of government a c t i o n s .  sensitive  The  r e s u l t a n t u s e r - o r i e n t e d e v a l u a t i o n process  differs  from t r a d i t i o n a l methods by i n v o l v i n g programme r e c i p i e n t s a t all  stages  i n the e v a l u a t i o n p r o c e s s .  Programme users  d e s i r e d g o a l s and a s s i s t i n determining  the extent t o which  programme output meets user e x p e c t a t i o n s . nate e v a l u a t i o n are procedures  identify  Included  i n the a l t e r -  t o l o c a t e s e n s i t i v e programme  impact p o i n t s , t o handle the e v e n t u a l i t y o f changing user  concerns,  and t o i d e n t i f y a p p r o p r i a t e l e v e l s o f c a u s a t i o n . Recommendations f o r government agencies  to incorporate  user i n p u t i n f u t u r e programme e v a l u a t i o n s r e s t on a comparative t e s t o f t r a d i t i o n a l and user o r i e n t e d e v a l u a t i o n s undertaken on a s o c i a l housing  programme i n the Greater Vancouver a r e a .  t e s t s i t u a t i o n was s e l e c t e d t o r e p r e s e n t a t y p i c a l  The  planning  problem r e q u i r i n g i n t e r a g e n c y c o o p e r a t i o n on a m u l t i d i m e n s i o n a l social issue. Comparative a n a l y s i s of the two  evaluation-processes  i n d i c a t e s t h a t programme r e c i p i e n t s can p r o v i d e a d i s t i n c t i v e i n p u t i n t o the e v a l u a t i o n stage o f the p l a n n i n g p r o c e s s .  Prev-  i o u s l y h e l d p e r c e p t i o n s of programme r e c i p i e n t s as r e s e n t f u l toward p r i v a c y i n v a s i o n and l a c k i n g the knowledge base t o a s s i s t programme e v a l u a t i o n proved i n v a l i d . the sponsoring  Given  a s i n c e r e d e s i r e by  agency t o i n c l u d e user p e r s p e c t i v e s , the r e c i p i e n t  i s both w i l l i n g and a b l e t o p r o v i d e an e v a l u a t i o n o f the e f f e c t s of programme  participation.  T h i s i s not t o say t h a t f o l l o w i n g e v a l u a t i o n of a programme from the r e c i p i e n t ' s p e r s p e c t i v e the programme must  V.  n e c e s s a r i l y be r e w r i t t e n to h i s s p e c i f i c a t i o n s . User s p e c i f i c a t i o n s may accept.  i n v o l v e t r a d e o f f s s o c i e t y - a t - l a r g e i s u n w i l l i n g to  U l t i m a t e l y d e c i s i o n s r e s t on e l e c t e d r e p r e s e n t a t i v e s .  The  argument emerging from t h i s r e s e a r c h  ion  of users  at  i n the e v a l u a t i o n process  l e a s t be aware of c l i e n t r e a c t i o n b e f o r e  reaching  i t i s not  his decision.  intended  as  product. Findings  for  inclus-  the d e c i s i o n maker w i l l  User p a r t i c i p a t i o n i s p a r t of a p r o c e s s ; the  i s t h a t through  suggest i t i s n e i t h e r p r a c t i c a l nor  programme r e c i p i e n t s to r e p l a c e agency personnel  the e v a l u a t i o n p r o c e s s .  A more r e a l i s t i c  an i n t e r a c t i v e e v a l u a t i o n process  throughout  procedure emerges through  which combines i n p u t from  programme a d m i n i s t r a t o r s , p o l i t i c a l  decision-makers, and  knowledge s e c t o r w i t h t h a t from programme r e c i p i e n t s . i d e n t i f i c a t i o n of the r o l e and i n the e v a l u a t i o n process  desirable  An  r e s p o n s i b i l i t i e s of each a c t o r  emerges from the case study.  ance of study recommendations by a d m i n i s t r a t o r s ,  Accept-  decision-makers  and  r e c i p i e n t s p o i n t s toward i n t e r a c t i v e e v a l u a t i o n as a  ial  source of i n f o r m a t i o n  ramme r e f o r m u l a t i o n .  the  for p u b l i c p o l i c y generation  and  potentprog-  vi TABLE OF CONTENTS Page' CHAPTER I;  TOWARD CITIZEN INVOLVEMENT IN PLANNING PROCESS  THE  S o c i e t a l Guidance From the Top Down R e a c t i o n from the Bottom Up A s e a r c h f o r meaningful involvement R e j e c t i o n of a u n i f i e d public interest Concern f o r the p e r v a s i v e n e s s of government actions Q u e s t i o n i n g the e x p e r t s b i a s  2 4 5 5 6 6  Citizen Participation: Where we are now. Models of c i t i z e n involvement Examples of c i t i z e n involvement i n the p l a n n i n g process Goal f o r m u l a t i o n stage P o l i c y implementation stage E v a l u a t i o n stage E v a l u a t i o n of S o c i a l A c t i o n Programmes L i m i t a t i o n s o f e x i s t i n g e v a l u a t i o n methodologies The case f o r a user o r i e n t e d e v a l u a t i o n methodology  8 11 12 15 16 . 17  D i s s e r t a t i o n Proposal T e s t i n g a user o r i e n t e d e v a l u a t i o n methodology ... The t e s t s i t u a t i o n Toward an a p p r e c i a t i o n of l i m i t a t i o n s CHAPTER I I :  19 20 21 22 24  SOCIAL CONSCIOUSNESS IN A GOVERNMENT POLICY SETTING  Programme P r o l i f e r a t i o n Under an I n c r e a s i n g S o c i a l Consciousness An example i n the Context of Canadian Housing Policy Housing as an Economic T o o l The Emergence of Housing as a S o c i a l Concern ...  28 30 31  Programme E v a l u a t i o n as an I n d i c a t o r of Directions The Nature of E v a l u a t i v e Research D i v e r s e Purposes Diverse D e f i n i t i o n s  34 35 38  Societal  vii Page The E v o l u t i o n o f E v a l u a t i v e Research As a P o l i c y T o o l As a Concern o f the S o c i a l S c i e n t i s t Housing as an E v a l u a t i v e Research Focus L i m i t a t i o n s o f Academic E v a l u a t i o n s E f f o r t s o f F e d e r a l Housing Agencies  41 43 48 53  A p p l i c a t i o n o f E x i s t i n g E v a l u a t i o n Methodologies T h e o r e t i c a l Models Wholey's E x p e r i m e n t a l Model 56 C a r t e r & Wharf's Typology 56 Caputo's I n t e g r a t i o n Process 57 O p e r a t i o n a l Models CMHC's $200 M i l l i o n Programme 62 Onibokun's R e l a t i v e H a b i t a b i l i t y 64 Gruen & Gruen's P r i o r P o l i c y A n a l y s i s 66 Feagin's Programme R e f o r m u l a t i o n 68 The T r a d i t i o n a l E v a l u a t i o n Model The Process Stage One: D e f i n i t i o n o f Programme Goals .. 71 Stage Two: E v a l u a t i o n Design 71 Stage Three: Data C o l l e c t i o n 75 Stage Four: Data A n a l y s i s 75 Stage F i v e : Programme R e f o r m u l a t i o n 75 The Assumptions 75 Limitations of Existing Evaluation Methodologies M e t h o d o l o g i c a l Weaknesses 76 Administrative Resistance 79 R e s t r i c t e d Subject Matter 79 Currency o f Research 81 Emerging Trends i n E v a l u a t i o n Research An I d e n t i f i e d Need 82 User Involvement 83 The Task CHAPTER I I I ; PART I:  85  TOWARD A USER ORIENTED PROCESS FOR THE EVALUATION OF SOCIAL ACTION PROGRAMMES THE RESEARCH SETTING - A CONSIDERATION AND JUSTIFICATION  The R e g i o n a l S e t t i n g Sponsorship - The G r e a t e r Vancouver R e g i o n a l Housing Deapartment The Programme - The "300 U n i t A s s i s t e d Home Ownership Programme" T h e o r e t i c a l B a s i s o f t h e Programme The Ownership Assumption The Income Mix Assumption  88 91 92 94 95 100  viii Page The L e g i s l a t i v e B a s i s o f the Programme Pressures f o r Government I n t e r v e n t i o n P r o v i n c i a l Actions Federal Actions Origins: H e l l y e r Task Force Experience i n the United States. Implementation o f Canadian L e g i s l a t i o n $200 M i l l i o n Innovative Programme S e c t i o n 58 A s s i s t e d Home Ownership Programme  107 10 8 109 109 110  112 113  PART I I ; CONCEPTUALIZATION AND TESTING OF A USER ORIENTED EVALUATION METHODOLOGY E v a l u a t i o n One: The T r a d i t i o n a l E v a l u a t i o n Research Design E v a l u a t i o n Two: A User O r i e n t e d E v a l u a t i o n Process F o c u s s i n g on the Programme R e c i p i e n t A s s e s s i n g R e c i p i e n t Behavior Assessing Recipient Attitudes A s s e s s i n g R e c i p i e n t Opinions Data C o l l e c t i o n - The I n t e r v i e w Overcoming L i m i t a t i o n s o f T r a d i t i o n a l E v a l u a t i o n Research A Proposed Methodology t o I d e n t i f y User Concerns and S a t i s f a c t i o n L e v e l s I d e n t i f i c a t i o n o f User Concerns Determining S a t i s f a c t i o n L e v e l s A Proposed Methodology t o Handle a Multi-Demensional Programme Dimension T e s t i n g I n d i c a t o r s o f Economic and S o c i a l Consequences o f Ownership I n d i c a t o r s o f the S o c i a l Consequences o f Income Mix I n d i c a t o r s o f the S e r v i c e R e d i s t r i b u t i v e Consequences o f Income Mix I n d i c a t o r s o f R e c i p i e n t Involvement i n the P r o v i s i o n o f Housing A Proposed Methodology t o I d e n t i f y Household-Programme-System Intervention Directions Conclusion  117 118 120 120 121 123 124 130 132 135 138 141 142 143 144 146 149 153 155  ix  CHAPTER  IV: THE USER ORIENTED EVALUATION PROCESS A TEST CASE STUDY  —  A Comparative A n a l y s i s o f Two E v a l u a t i o n Processes I d e n t i f i c a t i o n o f Programme E v a l u a t i o n Goals T r a d i t i o n a l E v a l u a t i o n — Expert Enunciated e v a l u a t i o n dimensions Program Demand Agency P r o t e c t i o n P r i o r i t i e s for Analysis Research Design Success I n d i c a t o r s User O r i e n t e d E v a l u a t i o n -- User Enunciated e v a l u a t i o n dimensions R e d i s t r i b u t i v e E f f e c t s of P a r t i c i p a t i o n Continuing Obligations Research Design A Comparison o f Programme E v a l u a t i o n s Based upon Expert and User Enunciated Dimensions Content Comparison Research Design Comparison  159 160 160 162 163 165 165 166 166 167 168 169 171  D i s c u s s i o n o f Programme E v a l u a t i o n F i n d i n g s 171 The Impact o f Assuming Home Ownership Economic Impact 17 2 S t a b i l i z i n g S h e l t e r Costs 172 Economic R e d i s t r i b u t i o n 178 S o c i a l Impact Redistributive Effects 180 Social Stress 181 Comparison o f T r a d i t i o n a l and User F i n d i n g s . . . . 182 The S o c i a l and S e r v i c e R e d i s t r i b u t i v e E f f e c t s o f Income Mix Social Effects Stigma Reduction 184 Neighbouring P a t t e r n s 186 Service F f f e c t s Home Space 187 Neighbourhood Space 188 Community Space 190 Comparison o f T r a d i t i o n a l and User F i n d i n g s . . . . 190 The Impact o f R e c i p i e n t p a r t i c i p a t i o n i n the P r o v i s i o n o f Housing Contacting Applicants 191 Continuing Agency-client Relations 192 Comparison o f T r a d i t i o n a l & User F i n d i n g s 194  X  I d e n t i f i c a t i o n and D i s c u s s i o n o f Program Reformulation P r o p o s a l s 196 Reformulation o f Ownership G u i d e l i n e s 199 C o n t i n u i n g Agency R e s p o n s i b i l i t y 201 Maximizing R e d i s t r i b u t i v e E f f e c t s 203 Loan Terms 203 Reduction o f S p e c u l a t i v e Gains 206 Mobility F l e x i b i l i t y 206 Reformulation o f Income Mix G u i d e l i n e s 207 Quantitative Guidelines 207 Unit C r i t e r i a 208 Stigma Reduction 209 Operationalizing Integration 210 Reformulation o f C i t i z e n P a r t i c i p a t i o n G u i d e l i n e s 2 1 3 Contacting Applicants 213 Selection of Applicants 213 L e g a l Aspects 214 Continuing Agency-client Relations .215 A Comparison o f Program Reformulation Proposals Based Upon T r a d i t i o n a l and User E v a l u a t i o n s 215 I m p l i c a t i o n s o f Case Study  Findings  I m p l i c a t i o n s f o r E x i s t i n g Housing Theory Ownership i n Theory and P r a c t i c e Income Mix i n Theory and P r a c t i c e User P a r t i c i p a t i o n i n Theory and P r a c t i c e  216 217 220 223  I m p l i c a t i o n s f o r Canadian Housing P o l i c y Uncoordinated A c t i o n s Innapropriate Interventions Unrealized Expectations  223 224 228 229  Implications t o the Evaluation of R e s i d e n t i a l Environments The Use o f A t t i t u d e and B e h a v i o u r a l Data Residential Indicators Performance G u i d e l i n e s S a t i s f a c t i o n Ratios  231 234 ...235 236  I m p l i c a t i o n s t o Program E v a l u a t i o n Methodologies... 237 CHAPTER  V:  GUIDELINES TOWARD A PROCESS TO INVOLVE PROGRAMME RECIPIENTS IN THE EVALUATION OF SOCIAL ACTION PROGRAMMES.  Overview o f the Proposed Process  240  Role and R e s p o n s i b i l i t y o f the User i n Programme Evaluation User C o n t r i b u t i o n s t o the I d e n t i f i c a t i o n o f E v a l u a t i o n Goals 241 I d e n t i f y i n g Dimensions ....243 Complementary Input 244 User C o n t r i b u t i o n s Design  to Preparation  o f an E v a l u a t i o n  245  xi 246 246 247 247 248  L e t t e r s of I n t r o d u c t i o n Survey Format A u d i o - v i s u a l Aids Method vs Content Remuneration User C o n t r i b u t i o n s t o Data C o l l e c t i o n I n v a s i o n of P r i v a c y Cost Grateful Testimonial E v a l u a t i o n as a Change Agent  248 249 254 255 255  User C o n t r i b u t i o n s t o Data A n a l y s i s Analysis Rationale Program C o m p a t a b i l i t y Q u a n t i f i a b l e and N o n - q u a n t i f i a b l e Data Aggregation  256 257 257 258 259  User C o n t r i b u t i o n s t o Program Knowledge Base  260  The CHAPTER  Reformulation  Role of the User i n the E v a l u a t i o n Process VI:  POLICY EVALUATION AS  AN  262  INTERACTIVE PROCESS  L i m i t a t i o n s of Research F i n d i n g s  267  I m p l i c a t i o n s of F i n d i n g s f o r E x i s t i n g Housing Theory and P r a c t i c e  269  I m p l i c a t i o n of F i n d i n g s f o r the E v o l u t i o n of an I n t e r a c t i v e E v a l u a t i o n Process C o n c e p t u a l i z a t i o n of Issues f o r Programme Evaluation Executing Evaluative Studies  271 274  C o n d i t i o n s necessary f o r Implementing I n t e r a c t i v e Policy Evaluation Role and R e s p o n s i b i l i t y of C i t i z e n R e c i p i e n t s R e a l i z i n g Myths Realizing Realities Role and R e s p o n s i b i l i t y of Decision-Makers Role and R e s p o n s i b i l i t y of Government Agencies Role and R e s p o n s i b i l i t y of the Knowledge S e c t o r . . . . Planner as E v a l u a t o r Action Orientation Implementation O r i e n t a t i o n Broadening P e r s p e c t i v e s A l t e r n a t e Methodologies Rethinking Planning Education  276 277 278 279 280 282 284 285 287 288 288 289 292  Planning  293  and  the E v a l u a t i v e Process  270  FOOTNOTES  296  BIBLIOGRAPHY  337  APPENDICES Sample q u e s t i o n n a i r e s  360  xii LIST OF  ILLUSTRATIONS  1.  Diagram  Goal F o r m u l a t i o n D i r e c t i o n s  2.  Diagram  Models of C i t i z e n P a r t i c i p a t i o n  9  3.  Diagram  C a p u t o s Model f o r Urban P u b l i c Policy Evaluation  58  4.  Diagram  . .  1  1  A Model of the T r a d i t i o n a l  Social  A c t i o n Programme E v a l u a t i o n Process  71  5.  Map  The G r e a t e r Vancouver R e g i o n a l D i s t r i c t  90  6.  Table  NHA  108  7.  Diagram  G u i d e l i n e f o r t e s t i n g an A l t e r n a t e  A s s i s t a n c e to Lower Income Households ...  E v a l u a t i o n Process  116 127  8.  Table  D u r a t i o n of Programme Exposure  9.  Table  Components of a Household  Profile  and  Dimension I n d i c a t o r s  133  10. Diagram  Dimensions f o r e v a l u a t i o n  152  11. Table  Evaluation P r i o r i t i e s  161  12. Table  Incomes of Programme F a m i l i e s  173  13. Table  Percentage  14. Table  Resident Occupations  178  15. Table  Resale A c t i v i t y  179  16. Diagram  U n i t , Neighbourhood and Community Concerns ..  188  17. Diagram  Programme Dimensions —  of Income Expended on Housing  ....  174  E v a l u a t i o n and  R e f o r m u l a t i o n Process  198  18. Table  How  200  19. Diagram  Household  20. Diagram  The Canadian  21. Diagram  P o l i c y , Programme and  I n f l a t i o n H i t Vancouver, 1972-1973 Value C l u s t e r s Housing  Bureaucracy  219 225  Participant  Compatability  232  22. Table  P r i o r i t y Concerns  235  23. Diagram  The Role of the R e c i p i e n t i n Programme Evaluation Recommendations f o r a Revised E v a l u a t i o n  263  Process  272  25. Diagram  A c t o r s i n the E v a l u a t i o n Process  277  26. Diagram 27. Table  F u s i n g Process and P r a c t i c a l Knowledge Bases. Components of a Methodology to R e f l e c t the C o n t i n g e n c i e s experienced i n a Real World Setting Evaluation i n Perspective  286  24. Table  28. Diagram  291 294  xiii  ACKNOWLEDGEMENT The reader w i l l f i n d t h a t t h i s d i s s e r t a t i o n i s p r e d i c a t e d on the assumption  t h a t b e n e f i t s a r e t o be  d e r i v e d from i n t e r r e l a t i n g experiences gained through the processed knowledge of the u n i v e r s i t y w i t h knowledge obtained through  agency involvement  the housing p r o c e s s . D i s t r i c t Housing Housing  and user experiences i n  Both the G r e a t e r Vancouver R e g i o n a l  Department and C e n t r a l Mortgage and  C o r p o r a t i o n were s u p p o r t i v e o f t h i s r e s e a r c h . B i l l  Casson, GVRD Housing  Department D i r e c t o r , J i m Moodie,GVRD>  and Gary Young, CMHC, were p a r t i c u l a r i t y h e l p f u l i n arrangi n g i n t e r v i e w s and d i s c u s s i n g program a c t i o n s . the r e c i p i e n t s o f the "300 U n i t Housing  They, and  Programme" who so  w i l l i n g l y shared t h e i r housing e x p e r i e n c e s , p r o v i d e d a t r i a l t e s t o f the proposed  I n t e r a c t i v e E v a l u a t i o n Process*  However, without f r i e n d s and c o l l e a g u e s w r i t i n g a d i s s e r t a t i o n c o u l d have become a very l o n e l y and f r u s t r a t ing experience. committee — R.F.  I t was my good f o r t u n e t o r e c e i v e from my.  Dr. R.W. C o l l i e r , Dr. R.B. H o r s f a l l , and Dr.  K e l l y ; from my f r i e n d s and c o l l e a g u e s , most e s p e c i a l l y  Dr. Kent Gerecke, Rick E l l i g o t t , E m i l Gutman and Roger McAfee; and from my p a r e n t s , c o n t i n u i n g encouragement and support f o r my endeavours.  To those who shared  e x p e r i e n c e , my h e a r t f e l t thanks  this  f o r c a r i n g enough t o see  me through i t . The a s s i s t a n c e o f the Canada C o u n c i l , o f Miss E v e l y n Popoff who undertook t o spend her Christmas t y p i n g , and o f a women whose theme song p r o v i d e d background music if with  I have t o I can do anything  thanks.  ...."is  acknowledged  "the adequate assessment o f e x i s t i n g and i n n o v a t i v e programs can be a v i t a l f o r c e d i r e c t i n g s o c i a l change and improving the l i v e s and environments o f community members. " (Caro 71, p. 1 ) .  1 CHAPTER I TOWARD CITIZEN INVOLVEMENT IN THE PLANNING PROCESS The  "planning p r o c e s s " has emerged as a framework  through which t o guide s o c i e t y toward an improved q u a l i t y o f life.  Broadly speaking the p l a n n i n g p r o c e s s i n c l u d e s c l a r i f y -  i n g one's o b j e c t i v e s and then d e t e r m i n i n g what a c t i o n  shall  be taken, by whom, when, by what methods, and a t what c o s t s , i n o r d e r t o achieve d e s i r e d g o a l s . ^ J u s t who should d e s i g n a t e s o c i e t y ' s g o a l s remains a q u e s t i o n open t o debate. exist.  Two sources o f g o a l g e n e r a t i o n  I d e n t i f i c a t i o n o f s o c i e t a l g o a l s can be undertaken by  s e l e c t e d r e p r e s e n t a t i v e s o f the s o c i e t y  a s s i s t e d by persons  t r a i n e d i n the process o f s o c i e t a l guidance.  Alternatively,  s o c i e t a l v a l u e s a r e i d e n t i f i e d d i r e c t l y by s o c i e t y a t l a r g e . Recommendations a r e then passed  t o those w i t h s c i e n t i f i c and  t e c h n i c a l knowledge f o r t r a n s l a t i o n i n t o s p e c i f i c a c t i o n s . D i r e c t involvement  by c i t i z e n s i n the p l a n n i n g process has been  termed " c i t i z e n p a r t i c i p a t i o n " .  The two approaches can be de-  s c r i b e d as "top down" and "bottom up" g o a l f o r m u l a t i o n . ILLUSTRATION I GOAL FORMULATION Top Down Planning  Planner Generates Societal Directions  S o c i e t y Responds  Bottom Up Planning  Planners Respond 4>  S o c i e t y Generates Directions  2 The s p e c i f i c process o f g o a l g e n e r a t i o n i s a moot p o i n t when g o a l s , generated  by whatever p r o c e s s , are a c c e p t -  able to a l l sectors of s o c i e t y .  P l a n n i n g can proceed  through  the subsequent stages o f f o r m u l a t i n g o b j e c t i v e s , t a r g e t s , and actions.  Problems a r i s e when s o c i e t a l o b j e c t i v e s , as i d e n t i -  f i e d by the two p r o c e s s e s , d i f f e r . census,  In the absence o f con-  s e l e c t i o n o f one g o a l s e t may w e l l l e a d to a c t i o n s  c o n s i d e r e d d e t r i m e n t a l by opposing  interests.  Planners  have been p l a c e d i n the c e n t e r o f the c o n t r o v e r s y over  who  should determine s o c i e t y ' s d i r e c t i o n s . S o c i e t a l Guidance from the "Top Down" P l a c i n g the r e s p o n s i b i l i t y f o r s o c i e t a l on the d e c i s i o n s o f government b u r e a u c r a t s  guidance  assumes a b e l i e f  i n an i d e n t i f i a b l e p u b l i c i n t e r e s t , and a f u t u r e capable o f being designed by a c e n t r a l agency.  I n t e r p r e t a t i o n o f the  p u b l i c i n t e r e s t has been assumed to be o f the p l a n n i n g p r o f e s s i o n .  a  special  Wheaton (70) suggests  concern this res-  p o n s i b i l i t y was based upon some i m p l i e d knowledge o f the i n t e r d e p e n d e n c i e s o f l a n d uses.  T h i s knowledge p r o v i d e d p l a n -  ners w i t h the framework to c o o r d i n a t e s p e c i a l i s t p l a n s f o r the b e n e f i t o f s o c i e t y . L a c k i n g processes  to i n c l u d e p u b l i c p a r t i c i p a t i o n ,  p l a n n e r s have f r e q u e n t l y a c t e d on b e h a l f o f s o c i e t y .  Several  s t u d i e s have i d e n t i f i e d problems o f p u b l i c c o n s u l t a t i o n t o i n c l u d e the l i m i t a t i o n s laymen face i n a r t i c u l a t i n g needs i n a context amenable t o p o l i c y  f o r m u l a t i o n , and the d i f f i c u l t i e s  3  of s e n s i n g what they may not have e x p e r i e n c e d o r o f imagining what does n o t e x i s t . Shore  (67) and Loeks  nature o f c i t i z e n i n p u t .  (70) noted the u n r e l i a b l e  Feagin  (72), i n r e p o r t i n g a study  of persons r e c e i v i n g r e n t supplements i n the Boston  area,  found t h a t most r e c i p i e n t s had o n l y a vague i d e a o f t h e p r o gramme.  Few r e c o g n i z e d  subsidization".  such b u r e a u c r a t i c  Most were u n c l e a r  l e v e l o f government was r e s p o n s i b l e  l a b e l s as " r e n t  as t o which agency o r f o r the a i d and, i n t u r n ,  what t h e i r o b l i g a t i o n s were f o r t h i s a s s i s t a n c e .  Feagin*s  f i n d i n g s would n o t appear s u r p r i s i n g t o most p u b l i c A study conducted by Sewell servants  officials.  (71) on the a t t i t u d e s o f c i v i l  toward c o n s u l t i n g p u b l i c o p i n i o n concluded t h a t most  officials  believed  "the p u b l i c n o t w e l l informed and t h e r e f o r e  cannot make r a t i o n a l judgments". In a d d i t i o n , p o l i c y makers c o n t i n u a l l y f e a r e d t h a t . c i t i z e n s who appeared a b l e t o a r t i c u l a t e a d e s i r e d  future  p a r t i c i p a t e d on the b a s i s o f an e s s e n t i a l l y p e r s o n a l t i o n o f c o s t s and g a i n s . personal  vested  Whether such m o t i v a t i o n s  i n t e r e s t s as opposed t o any broader  evalua-  reflected societal  concerns i s a judgment p l a n n e r s have no methodology t o assess seen  ( K a f o g l i s 70). When p a r t i c i p a t i o n d i d o c c u r , i t was o f t e n  by p l a n n e r s as d i f f i c u l t , d u l l , and time consuming.  deadlines, to p r e c l u d e  Time  p r i o r i t y demands, and complex procedures were thought the p o s s i b i l i t y o f meaningful c i t i z e n p a r t i c i p a t i o n .  4 R e a c t i o n from the "Bottom  Up"  C i t i z e n p a r t i c i p a t i o n i n the decision-making r e s t s on the assumption t h a t the o r d i n a r y c i t i z e n  process  possesses  the r i g h t to p a r t i c i p a t e i n the d e c i s i o n s which a f f e c t h i s life.  Evidence  t h a t Canadian c i t i z e n s  increasingly desire  to assume a r o l e i n the p l a n n i n g process i s mounting. C i t i z e n p r o t e s t s i n e a s t e r n Canada have s u c c e s s f u l l y b l o c k e d s e v e r a l developments not c o n s i d e r e d to be i n the b e s t i n t e r e s t s of the community.  Marathon R e a l t y ' s  redevelop-  ment o f Marlborough S t r e e t , urban renewal o f T r e f a n n Court the Spadina  expressway are cases i n p o i n t .  and  On the West c o a s t ,  s i m i l a r examples e x i s t of community groups who  o r g a n i z e d i n an  attempt to d i r e c t l y i n f l u e n c e the decision-making  process.  C i t i z e n a c t i o n s f o r c e d Marathon R e a l t y t o " c o n v e r t a proposed shopping  c e n t e r i n t o r e s i d e n t i a l uses.  freeway d i s r u p t i n g the Vancouver Chinese  Citizens forstalled a community, and f o r c e d  reassessment o f l a n d uses a d j a c e n t to S t a n l e y Park.  The  S t r a t h c o n a P r o p e r t y Owners and Tenants A s s o c i a t i o n were a b l e to pursuade governments t o r e v i s e urban renewal p l a n s f o r the area.  D e m o l i t i o n and r e b u i l d i n g p r o p o s a l s were a l t e r e d i n  favour of neighbourhood p r e s e r v a t i o n , r e n o v a t i o n , and tation policies The  (Pendakur  rehabili-  72).  examples c i t e d are but a few s e l e c t e d from a  broad range of c i t i z e n a c t i o n s . c i t i z e n involvement  The p r e s s u r e s  i n d i v e r s e circumstances  c e r t a i n t r a i t s i n common.  generating  appear to have  5  1. standards  A s e a r c h f o r meaningful involvement:  Improved  of e d u c a t i o n have been accompanied by r i s i n g  expec-  t a t i o n s and by a more c r i t i c a l q u e s t i o n i n g o f s o c i e t y ' s actions.  S o c i a l unrest,  (while more o v e r t i n the  S t a t e s d u r i n g the l a t e 1960's), m a n i f e s t e d  itself  United i n Canada  as a movement t o improve the p o s i t i o n of the average c i t i z e n who  f e l t he had v i r t u a l l y no i n f l u e n c e on n a t i o n a l p o l i c y  possessed Redressing  no i n f o r m a t i o n t o enable  him  to make judgments.  i n e q u a l i t i e s became a concern  ments of s o c i e t y .  P a r t i c i p a t i o n was  o f the younger seg-  seen not o n l y as a r i g h t  but a l s o as a r e s p o n s i b i l i t y of the concerned c i t i z e n . r e s p o n s i b i l i t y became i n c r e a s i n g l y e v i d e n t w i t h the of  2.  Real Poverty Report  The  pot" theory e v o l v e d i n the U n i t e d S t a t e s to f a c i l i t a t e  American people. of  p o l i c y was due  and E n g l i s h o r i g i n s ,  not a n a t i o n a l p r i o r i t y .  to the shortage  d e s i g n concepts  "melting the  r a c i a l groups i n t o a u n i f i e d  In Canada, presumably due  b i - c u l t u r a l French  and pub-  (Adams 71).  Rejection of a U n i f i e d P u b l i c I n t e r e s t :  melding of d i v e r s e e t h n i c and  The  "rediscovery  p o v e r t y " by the S p e c i a l Senate Committee on Poverty  l i c a t i o n of The  and  to the r e c o g n i t i o n the m e l t i n g  Nevertheless  pot  (possibly  of Canadian p l a n n i n g t h e o r i s t s ) American  prevailed.  I m p l i c i t i n c i t i z e n p r o t e s t s i s a c h a l l e n g e to the u n i f i e d p u b l i c i n t e r e s t assumption. t h a t t h e r e i s no g e n e r a l enduring concensual  Wheaton  (70)  suggests  p u b l i c i n t e r e s t about which  goals can be a r t i c u l a t e d i n any meaningful  way.  6 Rather,  the p u b l i c i n t e r e s t should be regarded as a h i g h l y  dynamic complex o f v a r i o u s i n t e r e s t s .  Competition  between  i n t e r e s t s r e q u i r e s some form o f b a r g a i n i n g as a means of reconciling c o n f l i c t i n g goals. market process t o tion.  One  control  Our  s o c i e t y has r e l i e d on  b a r g a i n i n g and r e s o u r c e  the  alloca-  must assume t h a t under the market system the  i n t e r e s t s o f persons who  l a c k e f f e c t i v e demand i n the market  p l a c e w i l l remain under-represented.  R e a l i z a t i o n o f these  concerns p r o v i d e d currency and a c c e p t a b i l i t y t o the  inclusion  of n o n - p r o f e s s i o n a l c i t i z e n s i n the p l a n n i n g p r o c e s s . 3.  Concern f o r the p e r v a s i v e n e s s o f government a c t i o n s ;  Requirements o f c o n s t i t u e n t s are e n u n c i a t e d as s o c i a l  action  3 programmes.  These programmes e v o l v e from p o l i c i e s  and  Lead  t o p r a c t i c a l a c t i o n s designed t o meet s t a t e d requirements. Between 1960-1968 domestic  programmes i n the U n i t e d S t a t e s  i n c r e a s e d from 45 t o 435.  A t the i n c e p t i o n o f the Canadian  M i n i s t r y o f S t a t e f o r Urban A f f a i r s , noted t h a t , i n 1970,  S e c r e t a r y H. P.  t h e r e e x i s t e d 119  Oberlander  f e d e r a l programmes i n  27 departments d e a l i n g w i t h some a s p e c t o f urban concern. bureaucracy has  necessary t o cope w i t h the v a s t a r r a y o f programmes  i n c r e a s i n g l y s e p a r a t e d c l i e n t from decision-maker.  t i o n has l e d t o a q u e s t i o n i n g o f the decision-maker's of people's problems and a s e a r c h f o r new cation. 4.  The  Q u e s t i o n i n g the " e x p e r t s ' " b i a s ;  Separaawareness  avenues o f communiThe  assumption  that  the expert can a c t as a s u r r o g a t e f o r the user i n d e f i n i n g needs 4 i s q u e s t i o n e d by Turner  (72), Sewell  (71) and o t h e r s  who  7 found d i v e r g e n c i e s between s o l u t i o n s recommended by prof e s s i o n a l s and those i d e n t i f i e d by the p u b l i c . reasons  f o r d i f f e r i n g viewpoints  c i t i z e n s are:  Among the  between p r o f e s s i o n a l s and  the i n t e l l e c t u a l e l i t i s m o f p r o f e s s i o n a l  groups; i d e a l i s m o f the p r o f e s s i o n a l ; t h e s h o r t range p e r s p e c t i v e o f most c i t i z e n groups; d i f f i c u l t i e s o f communicat i o n a c r o s s s o c i a l c l a s s and e d u c a t i o n boundaries; o f understanding t i v e behaviour suggest  o f g o a l s and r e s p o n s i b i l i t i e s .  s t u d i e s by Wheaton  and a l a c k  Administra-  (70), Dye (72) and Gans (70)  t h a t p l a n n e r s undertake t h e i r t a s k s n o t from t h e p e r -  s p e c t i v e o f some a l l - p e r v a s i v e s o c i e t a l i n t e r e s t but from t h e viewpoint o f c e r t a i n e l i t e s e c t o r s o f t h e community. Too o f t e n p l a n n e r s have merely r e f l e c t e d the narrow c l a s s b i a s and v a l u e s c h a r a c t e r i s t i c of t h e i r s o c i e t a l o r i g i n s . They have p l a c e d g r e a t emphasis on l o n g range v a l u e s . . . t h e p r e s e r v a t i o n o f nature over the needs o f man.... p r o p e r t y v a l u e s r a t h e r than human v a l u e s . W i t h i n p r o p e r t y v a l u e s t h e i r c h i e f concern has been w i t h p r o p e r t y o f those who a r e b e t t e r o f f i n s o c i e t y r a t h e r than the p r o p e r t y - o r r a t h e r , l a c k o f i t - o f those who were l e s s w e l l - o f f . (Wheaton, 70, p. 154). The  r e c o g n i t i o n the.:, p l a n n i n g which i s r e s p o n s i v e 1  to e s t a b l i s h e d b u r e a u c r a c i e s and t o the i n t e r e s t s o f the developer might i n the process  f o r s a k e t h e p r e f e r e n c e s , needs,  and d e s i r e s o f the consuming p o p u l a t i o n i s e v i d e n t i n s t u d i e s o f housing,  r e c r e a t i o n , e d u c a t i o n , and h e a l t h care  undertaken by Gans, Dyckman and Meyerson. of p u b l i c p o l i c y ,  5  Other  facilities .. criticisms  from the p e r s p e c t i v e o f the low income  p o p u l a t i o n , emerged i n urban renewal studies.**  8  Citizen Participation; Draper  Where We  Are  now  (71) has d e f i n e d the term  " c i t i z e n " to r e f e r  to the d e s i r e one has t o belong t o and be a p a r t of  something.  In e f f e c t c i t i z e n p a r t i c i p a t i o n can extend t o " c i t i z e n power" ( A r n s t e i n 69).  I t i s the s t r a t e g y by which the  j o i n i n d e t e r m i n i n g how  i n f o r m a t i o n i s shared, g o a l s and  p o l i c i e s are s e t , and programmes are o p e r a t e d . t i f i a b l e citizen interests exist. user"  ( a l s o termed  "have-nots"  Various iden-  F o r example, the  "citizen  u l t i m a t e c l i e n t or programme r e c i p i e n t )  r e f e r s s p e c i f i c a l l y t o persons r e c e i v i n g a s s i s t a n c e under a government i n i t i a t e d s o c i a l a c t i o n programme.  "User-need"  has come t o mean the data and i n f o r m a t i o n t h a t r e p l a c e o r supplement the t r a d i t i o n a l owner's requirements Models of C i t i z e n  (Ferguson 72).  Involvement:  The degree of c i t i z e n p a r t i c i p a t i o n i n the d e c i s i o n making p r o c e s s has ranged from merely r e c e i v i n g i n f o r m a t i o n t o f u l l p a r t i c i p a t i o n i n decision-making.  Diagram 2 on the  lowing page i l l u s t r a t e s the range of c i t i z e n  involvement.  S e v e r a l models have been c r e a t e d t o suggest c i t i z e n involvement i n the p l a n n i n g p r o c e s s may f u l l y be accomplished.  fol-  how  most s u c c e s s -  In the l a t e 1960's a body o f  literature  and e x p e r t i s e developed which sought t o d e r i v e the l e g i t i m a c y of the p l a n n e r from the p r e f e r e n c e s of u l t i m a t e consumers. Davidoff  (65) e n u n c i a t e d p r o p o s a l s f o r advocacy p l a n n i n g  through which p l a n n e r s c o u l d a s s i s t those whose i n t e r e s t otherwise be u n d e r r e p r e s e n t e d  i n the p l a n n i n g p r o c e s s .  might  9  DIAGRAM MODELS OF CITIZEN PARTICIPATION Source:  C a n a d i a n C o u n c i l o f R e s o u r c e s and Environment M i n i s t e r s . P r o c e e d i n g s o f t h e Man and Resources C o n f e r e n c e 1973. p.15. INCREASING DEGREE OF CITIZEN PARTICIPATION  NON-PARTICIPATION  THE AREA OF AUTHORITY TO DECIDE VESTED IN POLITICAL, BURF*""»  INFORMATION MODEL  PERSUATION MODEL  CONSULTATION MOOEL  PARTERNERSHIP MODEL  CITIZEN CONTRHL MODEL  AN AUTHORITY MAKES A DECISION AND INFORMS THE PUBLIC ABOUT IT.  AN AUTHORITY MAKES A DECISION THEN PERSUADES THE PUBLIC TO ACCEPT IT.  AN AUTHORITY DEFINES THE PROBLEM, PRESETS IT TO THE PUBLIC INVITES COMMENTARY & SUGGESTIONS AND THEN MAKES A DECISION.  AN AUTHORITY PRESCIDES THE LIMITS AND WITHIN THESE LIMIT CITIZENS SHARE & MAY EVEN ASSUME THE DECISION-MAKING RESPONSIBILITY.  CITIZENS HAVE FULL RIGHTS TO PARTICIPATE IN AND ASSUME THE RESPONSIBILITY FOR DECISIONS.  A l t e r n a t e d i r e c t i o n s a r e suggested by Friedmann's (73) t h e o r y o f t r a n s a c t i v e p l a n n i n g and Schon's Beyond t h e Stable State  ( 7 1 ) . Both a u t h o r s s t r e s s t h e need f o r c o n t i n u a l  l e a r n i n g as t h e b a s i s f o r p o l i c y d i r e c t i v e s .  To Friedmann t h e  f u t u r e i s a dimension of u n r e a l i z e d p o s s i b i l i t i e s , homogeneous d i m e n s i o n open t o change and c h o i c e . w i t h t h e p r e s e n t w i l l shape t h i s f u t u r e .  a nonHow we cope  The p r e s e n t p r e s e n t s  10  a c r i s i s of v a l u e s and  knowledge.  The  c r i s i s of v a l u e s i s  r e f l e c t e d i n the s e a r c h f o r p a r t i c i p a n t forms of o r g a n i z a t i o n and  social  i n the attempts to accept a d i v e r s i t y of  views as a v a l i d f o u n d a t i o n f o r s o c i a l p l a n n i n g . accomplish  To  these ends Friedmann f e e l s t h a t the scope f o r  autonomous group a c t i o n must be e n l a r g e d by r e d u c i n g i n f l u e n c e of b u r e a u c r a t i c o r g a n i z a t i o n s through  the  a decentra-  l i z a t i o n of power. There i s a l s o an a s s o c i a t e d c r i s i s a r i s i n g from knowledge gaps.  In most i n s t a n c e s , n e i t h e r d e c i s i o n makers nor  c i t i z e n s are aware of c o n f l i c t i n g f u t u r e a s p i r a t i o n s and the p o s s i b l e consequences o f a l t e r n a t e a c t i o n s . to e f f e c t i v e communication between those who p r i m a r i l y to processed  The  of  barriers  have access",  knowledge and those whose knowledge  r e s t s on p r a c t i c a l experience are becoming more e v i d e n t . Messages may  be exchanged but meanings are o f t e n not  communicated.  Goodman i n A f t e r the P l a n n e r s  s i m i l i a r concern and c l i e n t .  effectively  identifies a  f o r communication d i f f i c u l t i e s between planner  He makes a case f o r d e m y s t i f y i n g the p l a n n i n g p r o f -  e s s i o n through  a t r a n s f e r of s k i l l s to the people.  Thus the  task  of p l a n n i n g i s a l l e v i a t e d as some of the r e s p o n s i b i l i t y i s shouldered  by the people The  f o r whom the p l a n n i n g o c c u r s .  e s t a b l i s h m e n t of a more s a t i s f a c t o r y form of  communication i s not simply a matter of t r a n s l a t i n g the a b s t r a c t and h i g h l y symbolic  language o f the p l a n n e r i n t o the s i m p l e r  and more e x p e r i e n c e - r e l a t e d v o c a b u l a r y of the c l i e n t .  The  real  11 s o l u t i o n i n v o l v e s a r e s t r u c t u r i n g o f the b a s i c r e l a t i o n s h i p between p l a n n e r and c l i e n t  (Friedmann 73 p. 172).  Friedmann b u i l d s h i s Theory o f T r a n s a c t i v e P l a n n i n g on the b a s i s o f a p a r t i c i p a n t s o c i e t y i n which c o n t i n u a l l e a r n i n g and feedback o f ideas o c c u r s .  His strategy includes  r e d u c i n g the s e p a r a t i o n between d e c i s i o n makers, p l a n n i n g u n i t s , and people  through a s o c i a l process b u i l t on d i a l o g u e ,  i n n o v a t i o n , and j o i n t problem s o l v i n g . remarkably s i m i l a r t o E t z i o n i ' s and T r i s t ' s  The s t r a t e g y i s  (6 8) "interwoven  planning"  (70) d i r e c t i v e s f o r t a c k l i n g the c h a l l e n g e s o f  the next t h i r t y  years.  Examples o f C i t i z e n Involvement i n the P l a n n i n g There i s mounting evidence  Process;  from a v a r i e t y o f  d i s c i p l i n e s concerned w i t h s o c i a l p o l i c i e s , o f a t r e n d toward i n c r e a s e d c i t i z e n p a r t i c i p a t i o n i n the p l a n n i n g C i t i z e n involvement  process.  i s i n c r e a s i n g l y seen as a p r e r e q u i s i t e t o  r e l e v a n t p o l i c y making in Architecture: C o n s i d e r i n g the c h a l l e n g e s o f the f u t u r e , d e s i g n can no longer remain the e x c l u s i v e mandate o f a s i n g l e p r o f e s s i o n o r group. The d e s i g n o f manmade e n v i r o n m e n t s . . . w i l l r e q u i r e the involvement not only o f the d e s i g n p r o f e s s i o n s b u t a l s o management, p l a n n e r s , s o c i a l and p h y s i c a l s c i e n t i s t s , economists, and, i n c r e a s i n g l y consumer groups....(Weiss 73 p. 19). i n S o c i a l Work: Casework i s most e f f e c t i v e when the focus i s kept on the problem as the c l i e n t understands i t (Mullen 72 p. 65).  12 i n Policy Sciences: One o f the most important i n f l u e n c e s on s e r v i c e s may be the nature o f the c l i e n t e l e , t h e i r numbers, the s e v e r i t y o f t h e i r needs and t h e i r motivations. P e r s o n a l w e l f a r e and the i n d i v i d u a l needs and r e s o u r c e s o f taxpayers are becoming an i n c r e a s i n g l y more important i n f l u e n c e upon government d e c i s i o n makers than sheer aggregate o f economic r e s o u r c e s . (Sharkansky 70 p.66). i n S o c i a l Policy Planning: The r e c o n s i d e r a t i o n o f environmental programs depends upon i n c r e a s i n g our understanding o f the needs and p r i o r i t i e s o f c l i e n t groups and c l e a r l y must i n v o l v e the p a r t i c i p a t i o n o f these groups. ( F r i e d e n 70 p. 2 86) . and i n R e s i d e n t i a l P l a n n i n g : In the f i n a l a n a l y s i s i t i s o n l y the people who e x p e r i e n c e the a c t i v i t y and i t s products who can e v a l u a t e them. (Turner 73 p. 153). Goal F o r m u l a t i o n  Stage:  During r e c e n t y e a r s t h e r e have been an i n c r e a s i n g number o f o p e r a t i o n a l examples o f c i t i z e n involvement a t v a r i o u s stages i n the d e c i s i o n making p r o c e s s .  P o s s i b l y the  most f r e q u e n t stage o f c i t i z e n involvement has been i n g o a l formulation.  E a r l y examples o c c u r r e d i n the 19 30's  "U.S.  A c t i o n f o r C i t i e s " p l a n s , and through the use o f simple p o l l i n g techniques d u r i n g the 19 40's and 1950's.  More r e c e n t  attempts t o i n v o l v e c i t i z e n s i n the i d e n t i f i c a t i o n o f p u b l i c i n t e r e s t d i r e c t i o n s i n c l u d e the 1964 C i t y o f Chicago r e p o r t i n which a s e r i e s o f r e g i o n a l meetings were h e l d f o r p l a n n e r s to e x p l a i n p r o p o s a l s and t o r e c e i v e comments f o r subsequent modification of basic p o l i c i e s .  Between 1966-67 the C i t y o f  13 Los Angeles sought c i t i z e n a s s i s t a n c e i n the d e f i n i t i o n o f i t s planning  goals.  In t h i s example a s e r i e s o f meetings was  held  i n v o l v i n g r e p r e s e n t a t i v e s of v a r i o u s p r o f e s s i o n a l s o c i e t i e s whose r e s p o n s i b i l i t i e s i n c l u d e d the  "environment".  These  pro-  f e s s i o n a l l y o r i e n t e d r e p r e s e n t a t i v e s were presumed to possess a s p e c i a l knowledge o f the p u b l i c i n t e r e s t . cess was  A s i m i l a r pro-  i n s t i g a t e d by the C i t y of Vancouver d u r i n g the summer  of 19 73 i n an attempt t o formulate  p o l i c y d i r e c t i o n s f o r the  redevelopment o f the I n n e r - c i t y F a l s e Creek Various munity P l a n n i n g  c i t i z e n planning  Basin.  groups, such as the Com-  A s s o c i a t i o n of Canada, have performed a r o l e  i n p r o v i d i n g governments w i t h  c r i t i q u e s o f proposed  l a t i o n from a v a r i e t y o f c i t i z e n p e r s p e c t i v e s .  legis-  Recent  examples i n c l u d e b r i e f s by the Canadian C o u n c i l on S o c i a l Development on the 1973 A c t and  a proposal  Amendments t o the N a t i o n a l Housing  on guaranteed annual income.  Non-government agencies were r e s p o n s i b l e f o r i n i t i a t i n g the 1966-67 Goals f o r D a l l a s programme. techniques  f o r evoking e x p r e s s i o n s  Various  of p u b l i c p r e f e r e n c e s  were  attempted i n c l u d i n g s o l i c i t e d essays on the problems f a c i n g D a l l a s , c o n f e r e n c e s o f r e p r e s e n t a t i v e c i t i z e n s to g o a l s , and hoods.  formulate  d i s t r i b u t i o n o f p r i n t e d l i t e r a t u r e to neighbour-  In Canada, the  " H a l i f a x Encounter" was  to i d e n t i f y c i t i z e n concerns and  a s i m i l a r attempt  t o evoke d i s c u s s i o n on  future  urban d i r e c t i o n s . In the G r e a t e r are attempting  Vancouver Regional  to formulate  District  planners  urban g o a l s based upon d i s c u s s i o n s  14 w i t h a v a r i e t y o f community i n t e r e s t groups concerned i n d i v i d u a l s outside  the p o l i t i c a l arena.  and  During 1972-73  meetings w i t h some f o r t y community groups i n i t i a t e d the cess o f d e v i s i n g a " L i v a b l e Region Programme". on  "What makes f o r l i v a b i l i t y ? " ,  lity?",  Discussions  "What works a g a i n s t L i v a b i -  "What are the important t h i n g s GVRD should  cerned w i t h i f i t i s to be r e s p o n s i b l e growth o f the r e g i o n ? "  pro-  be  f o r managing  con-  the  i n d i c a t e d c i t i z e n concerns f o r  r e s t r i c t i n g f u r t h e r r a p i d growth i n the r e g i o n , a d e s i r e to p a r t i c i p a t e i n community d e c i s i o n s , and Proposals  concerning  a wish t o see  transportation, industry,  housing, community l i f e and  land  s e r v i c e s , p o l l u t i o n and  p r e s e r v a t i o n o f n a t u r a l areas were f u r t h e r d i s c u s s e d the summer o f 1973  action.  costs, the during  by p o l i c y committees comprised o f i n t e r e s t e d  citizens. The  use  of c o n s u l t a n t  groups to i d e n t i f y p u b l i c  i n t e r e s t and p r e f e r r e d d i r e c t i o n s was  attempted i n the Miami  V a l l e y , Ohio.  f i r m o f Gruen and  was  In t h i s experiment the  r e t a i n e d to determine how  b e s t to i n t e g r a t e low  moderate income housing i n t o o t h e r w i s e middle-income bourhoods.  In a n a l y z i n g  the needs and  Gruen  and neigh-  d e s i r e s o f both lower  and middle income households the Gruens found t h a t d i r e c t i v e s were s u c c e s s f u l l y o b t a i n e d The  a n a l y s i s and  from a v a r i e t y of c i t i z e n  d i s c u s s i o n o f t r a d e o f f s between the  groups i n d i c a t e d t h a t there were indeed common points.  groups. two  bargaining  T h e i r study appears to j u s t i f y the involvement o f  15 c i t i z e n s i n p o l i c y formulation the long run m i n i m i z a t i o n  as a method of  of s o c i a l  facilitating  costs.  P o l i c y Implementation Stage: Examples of c i t i z e n involvement i n the implementation o f d e c i s i o n s are  fewer.  actual  S e v e r a l c i t i e s have  moved to d e c e n t r a l i z e d government i n an attempt to the c e n t e r of a u t h o r i t y , d e c i s i o n making, and as c l o s e to the u l t i m a t e Winnipeg and  user as p o s s i b l e .  place  implementation  In Vancouver,  Toronto, f o r example, i n n e r - c i t y r e s i d e n t s have  been a c t i v e i n upgrading t h e i r environments through i n i t i a t e d programmes of r e n o v a t i o n (72)  research  and  infilling.  Axworthy's  on c i t i z e n groups i n Winnipeg i n d i c a t e s t h a t  c i t i z e n s are able t o a r t i c u l a t e t h e i r concerns and priorities. i n 1973  citizen  C i t i z e n p a r t i c i p a t i o n trends  to  list  were strengthened  w i t h the i n c l u s i o n o f amendments to the  National  Housing A c t which r e q u i r e c i t i z e n involvement i n Neighbourhood Improvement Programmes. Among the problems emerging from c i t i z e n p a r t i c i p a t i o n i n goal formulation  and  resolved question  represents  emerge.  o f who  Elected o f f i c i a l s  implementation i s the y e t t o the p e o p l e .  (Arnstein  69).  law.  Citizens claim  A p r o l o n g e d debate  a u t h o r i t y i s beyond the scope o f t h i s d i s s e r t a t i o n . author accepts the  claims  l a y c l a i m t o decision-making  a u t h o r i t y on the b a s i s o f t r a d i t i o n and a u t h o r i t y as consumers  Two  be  stance t h a t programme u s e r s ,  a f f e c t e d by programme a c t i o n s , and  on  The  citizens  bureaucrats entrusted  with  16 programme implementation, are among the i n t e r e s t e d  parties  whose v a l u e s and v i e w p o i n t s should be c o n s i d e r e d by e l e c t e d politicians.  The e l e c t e d o f f i c i a l s are the u l t i m a t e  decision-makers. Evaluation  Stage: As  larger proportions  o f p u b l i c funds a r e committed  to s o c i a l programmes, agencies engaged i n s u p p l y i n g community s e r v i c e s are i n c r e a s i n g l y r e q u i r e d evidence o f programme e f f e c t i v e n e s s p u b l i c support.  Too f r e q u e n t l y  these  to provide  as a b a s i s  f o r continued  the s o c i a l and economic impact  of programmes which c o s t the p u b l i c m i l l i o n s o f d o l l a r s were not subjected  to c r i t i c a l  analysis.  When e v a l u a t i o n  o f programmes d e a l i n g w i t h the p r o -  v i s i o n o f human r e s o u r c e development d i d o c c u r findings often  suggested t h a t programmes were having minimum  impact on problems. of the i n c r e a s e d distance  tentative  The q u e s t i o n o f "why", the r e a l i z a t i o n  complexity o f problems, the expanded  social  between p o l i c y maker and c i t i z e n , and the need f o r  a c c o u n t a b i l i t y encouraged government agencies t o i n i t i a t e evaluative  research  programmes.  In t h i s sense e v a l u a t i o n as r e s e a r c h  research  i s considered  i n which the s c i e n t i f i c method i s a p p l i e d t o  a n a l y s i s o f p u b l i c programmes t o l e a r n what happens as a r e s u l t o f programme a c t i v i t i e s .  Evaluation  i d e n t i f i c a t i o n of programme o b j e c t i v e s ; measures toward these o b j e c t i v e s ;  includes:  the  the development o f  the assessment o f what  17  d i f f e r e n c e s p u b l i c programmes a c t u a l l y make; and  the  pro-  j e c t i o n o f what d i f f e r e n c e s c o u l d be expected i f the  pro-  gramme was  continued  o r expanded  (Wholey 7 0 ) .  To date e v a l u a t i o n o f programme impact has undertaken by p r o f e s s i o n a l p l a n n e r s .  Neither  been  programme  r e c i p i e n t s nor c i t i z e n s p o t e n t i a l l y a f f e c t e d by programme a c t i o n s are r e g u l a r l y i n c l u d e d i n the e v a l u a t i o n the p l a n n i n g  process.  stage o f  Mullen, i n d i s c u s s i n g t h i r t e e n  e v a l u a t i v e s t u d i e s by s o c i a l workers, concluded: Although the c l i e n t ' s views were p a r t o f the e v a l u a t i o n , d i r e c t measurement o f c l i e n t a t t i t u d e s was the e x c e p t i o n r a t h e r than the rule. T h i s stance i s r a t h e r s u r p r i s i n g s i n c e p r a c t i s e has become i n c r e a s i n g l y c l i e n t oriented. (Mullen 72, p. 3 4 ) . The  reluctance  of some p r o j e c t s to base  evaluation  on the c l i e n t ' s a t t i t u d e s about changes r e s u l t i n g from programme does not n e c e s s a r i l y p o i n t to c o n s e r v a t i v e methods.  There i s a g e n e r a l  the  research  l a c k o f agreement between c l i e n t s  and p r a c t i t i o n e r s i n a s s e s s i n g  treatment outcome.  Thus  i i n v e s t i g a t o r s f a c i n g v a r i o u s m e t h o d o l o g i c a l problems M u l l e n suggests, have d e c i d e d t i n g the c l i e n t as a key  to r e s o l v e t h i s one  i n d i c a n t of  Evaluation of S o c i a l A c t i o n  by  may, elimina-  evaluation.  Programmes  L i m i t a t i o n s o f E x i s t i n g E v a l u a t i o n Methodologies: E v a l u a t i o n processes, years,  evolved  during  the p a s t  ten  have attempted to apply methodology developed i n the  c o n t r o l l e d experimental s i t u a t i o n of the  laboratory  to  the  18  assessment of s o c i a l a c t i o n programmes.  R e s u l t s have not  7 been  impressive. The q u e s t i o n a r i s e s as to whether e v a l u a t i o n of  government programmes i s , i n i t s e l f , e x i s t i n g e v a l u a t i o n processes  u n a t t a i n a b l e o r whether  merely apply i n a p p r o p r i a t e  methodologies t o s o c i a l s i t u a t i o n s .  Consideration of  tradi-  t i o n a l e v a l u a t i o n methods i n l i g h t of e v o l v i n g p l a n n i n g suggests  the l a t t e r c o n d i t i o n may  thought  w e l l apply.  T r a d i t i o n a l programme e v a l u a t i o n models assume s t a t i c programmes i n which i n i t i a l programme g o a l s are a b l e , c l e a r and remain v a l i d over time.  avail-  T r a d i t i o n a l models  seldom d i r e c t l y assess user a t t i t u d e s t o programme dimensions and make no p r o v i s i o n f o r c i t i z e n p a r t i c i p a t i o n i n the evaluative process.  In most i n s t a n c e s , programme a d m i n i s t r a -  t o r s p r o v i d e i n f o r m a t i o n on user need and programme impact. U n f o r t u n a t e l y , r e p r e s e n t a t i v e s of the s p o n s o r i n g t l y understate  agency  frequen-  s o c i a l c o s t s i n c u r r e d by the programme r e c i p i e n t .  E x i s t i n g e v a l u a t i o n models appear i n c o n s i s t e n t w i t h emerging p l a n n i n g theory. Planners provide f l e x i b l e ,  are c a l l i n g  f o r new  processes  s h o r t range p l a n n i n g which i s r e s p o n s i v e  to  to  v a r i e d user needs and a n t i c i p a t e s s o c i a l c o s t s .  However, when  e v a l u a t i o n i s requested,  an  the r e s e a r c h e r p r e s e n t s  expertg  centered methodology r e q u i r i n g s t a t i c programmes and a n a l y s i s of q u a n t i f i a b l e programme p a t t e r n s . response t o a h i g h l y complex process ticularly  This  traditional  seems inadequate  from the s t a n d p o i n t of user needs and  summative  par-  involvement  19 of the c i t i z e n i n the e v a l u a t i o n  stage of the  planning  process. The  Case f o r a User O r i e n t e d  Evaluation  Methodology:  T h i s d i s s e r t a t i o n a c c e p t s the assumption  that  the adequate assessment of e x i s t i n g and i n n o v a t i v e programmes can be a v i t a l f o r c e i n d i r e c t i n g s o c i a l change and improving the l i v e s and environments of community members. (Caro 71 p . 1 ) . While i t i s c l e a r t h a t e a r l y o p t i m i s t i c p r e d i c t i o n s of e x t e n s i v e  scientific  s o c i a l programme e v a l u a t i o n  have  f a i l e d to m a t e r i a l i z e , the concern f o r the development o f a methodology to e v a l u a t e the remains.  Emerging government p o l i c i e s suggest no c u t back  i n s o c i a l programmes. the  impact of s o c i a l a c t i o n programmes  spectre  of c o n t i n u i n g  The evaluating  Acceptance o f the  s t a t u s quo  i n e f f e c t i v e programme a c t i o n s .  c r e a t i o n of a more e f f e c t i v e mechanism f o r  s o c i a l a c t i o n programmes i s the  dissertation.  The  task posed by  search f o r a l t e r n a t e evaluation  l e d the author to s p e c u l a t e  on the i m p l i c a t i o n s of  members of the concerned p o p u l a t i o n  c o n s i s t e n t w i t h emerging p l a n n i n g r e c i p i e n t s should i n c r e a s e  the  theory.  this  methodologies including  i n the e v a l u a t i o n  T h e o r e t i c a l l y , user involvement i n the e v a l u a t i o n  mme  raises  process.  process i s  I n c l u s i o n of p r o g r a -  l i k l i h o o d of  evaluation  r e f l e c t i n g dimensions s e n s i t i v e to programme o p e r a t i o n of f u t u r e p l a n s emerging i n response to s o c i a l needs.  and  20  D i s s e r t a t i o n Proposal Programme e v a l u a t i o n attempts  t o determine the  e x t e n t t o which o b j e c t i v e s are b e i n g f u l f i l l e d ,  the p r i n c i p l e s  or  t h e o r i e s u n d e r l y i n g a programme which account  or  f a i l u r e , whether i d e n t i f i e d changes are due t o programme  measures o r extraneous t i o n s o f changes.  f a c t o r s and t o determine  f o r success  the i m p l i c a -  In e f f e c t , e v a l u a t i o n i s a l e a r n i n g p r o -  cess - l e a r n i n g why programmes succeed o r f a i l and which a t t r i b u t e s o f the programme are most s e n s i t i v e t o change. T r a d i t i o n a l e v a l u a t i o n seeks  t o p r o v i d e t h i s knowledge  through  an i d e n t i f i c a t i o n by programme a d m i n i s t r a t o r s o f g o a l s and subsequent a n a l y s i s o f the e x t e n t t o which g o a l s are r e a l i z e d . The b a s i s upon which change i s measured i s e s t a b l i s h e d by p o l i c y p l a n n e r s not programme u s e r s . T h i s d i s s e r t a t i o n proposes  an a l t e r n a t e e v a l u a t i v e  system i n which programme users i d e n t i f y d e s i r e d g o a l s and a s s i s t i n determining the e x t e n t t o which programme meets user e x p e c t a t i o n s . for  output  The i d e n t i f i c a t i o n o f p o s s i b i l i t i e s  user o r i e n t e d e v a l u a t i o n i s not unique  to t h i s  dissertation.  S e v e r a l authors have noted the p o t e n t i a l advantage o f e v a l u a t i n g programmes from the p e r s p e c t i v e of the u s e r . f e e l s f u l l a p p r e c i a t i o n o f programme impact accomplished  Caro (71)  can o n l y be  through c l i e n t i n t e r p r e t a t i o n s .  Caputo (73),  C a r t e r and Wharf (73) see user p a r t i c i p a t i o n as l e g i t i m i z i n g evaluative findings.  U n f o r t u n a t e l y , these authors f a i l t o  o p e r a t i o n a l i z e t h e i r concern by i n c l u d i n g user at  a l l stages i n the e v a l u a t i v e p r o c e s s .  participation  21 T e s t i n g a User O r i e n t e d  Evaluation  Methodology:  To a s s i s t i n e s t a b l i s h i n g g u i d e l i n e s f o r a u s e r o r i e n t e d e v a l u a t i o n p r o c e s s i t appears necessary t o t e s t the p r o p o s i t i o n t h a t t h e demands and e x p e c t a t i o n s c e i v e d by t h e u l t i m a t e  o f need p e r -  c l i e n t can form an a l t e r n a t e and/or  complementary p e r s p e c t i v e  on s o c i a l p o l i c y d i r e c t i o n s .  To t e s t t h i s p r o p o s i t i o n two e v a l u a t i o n s w i l l be conducted on the same programme.  One e v a l u a t i o n w i l l be  undertaken f o l l o w i n g p r o c e s s e s c u r r e n t l y employed by government e v a l u a t i v e p e r s o n n e l .  The second e v a l u a t i o n w i l l  programme r e c i p i e n t s a t f i v e stages i n the e v a l u a t i v e  include process.  S p e c i f i c a l l y , the user w i l l be i n v o l v e d i n i d e n t i f i c a t i o n o f evaluation goals, design,  c o n s t r u c t i o n o f an e v a l u a t i v e  research  data c o l l e c t i o n and a n a l y s i s and in-programme r e -  formulation. The experts  assumptions, p r i o r i t i e s and s o l u t i o n s o f h o u s i n g  (persons r e s p o n s i b l e  for setting policy,  administer-  i n g and e v a l u a t i n g programmes a t f e d e r a l and l o c a l l e v e l s ) obtained  through the t r a d i t i o n a l e v a l u a t i o n p r o c e s s w i l l be  compared with those o f c i t i z e n s r e c e i v i n g housing  assistance.  I f , on a n a l y s i s , i t appears p o l i c y g u i d e l i n e s o b t a i n e d experts  from  c l o s e l y r e f l e c t the c l i e n t ' s demand s e t t h e need f o r  a user o r i e n t e d e v a l u a t i o n w i l l be r e j e c t e d .  Rejection  rests  on the r e a l i z a t i o n t h a t , w h i l e the v i e w p o i n t o f the user may be o f i n t e r e s t , i f the expert surrogate  can adequately f u n c t i o n as a  t o i d e n t i f y r e l e v a n t v a r i a b l e s the added expense  i n time and money o f i n c l u d i n g c i t i z e n s i n the e v a l u a t i o n  22  process i s not warranted.  I f , however, t r a d i t i o n a l programme  e v a l u a t i o n assumptions and output do not r e f l e c t c l i e n t  con-  cerns the d i s s e r t a t i o n p r o p o s i t i o n w i l l appear s u b s t a n t i a t e d . The recommendation w i l l be made t h a t the user o r r e c i p i e n t of a government programme should be an i n t e g r a l component i n any p l a n n i n g p r o c e s s which seeks t o t a c k l e programme e v a l u a tion. The t e s t  situation: S e l e c t i o n o f a t e s t s i t u a t i o n f o r the proposed  o r i e n t e d e v a l u a t i o n process was based ments.  user  upon s e v e r a l r e q u i r e -  F i r s t l y , a s i t u a t i o n r e f l e c t i n g a t y p i c a l planning  problem was needed.  S e l e c t i o n o f an e x i s t i n g s o c i a l  action  programme p r o v i d e d a s e t t i n g upon which both p l a n n e r s and c i t i z e n s could focus.  Secondly,  the programme had t o be one  which i n v o l v e d a v a l u e laden c o n t e x t .  In such  circumstances  i t would be reasonable t o assume t h a t an e v a l u a t i o n from the p e r s p e c t i v e o f agency g o a l s alone might y i e l d c o n c l u s i o n s t o t a l l y i n a p p r o p r i a t e from t h e v i e w p o i n t o f the u s e r . T h i r d l y , the programme must r e f l e c t emerging p o l i c y E v a l u a t i o n o f programme a c t i o n s would  directions.  then appear as a r e a l i s t i c  e x e r c i s e t o programme a d m i n i s t r a t o r s and u s e r s  alike.  F o u r t h l y , the programme should meet the p r a c t i c a l  requirements  o f b e i n g a c c e s s i b l e t o the r e s e a r c h e r . An A s s i s t e d Home Ownership programme b e i n g mented by the G r e a t e r Vancouver R e g i o n a l D i s t r i c t Department appeared  t o f i t a l l requirements.  imple-  Housing  An a n a l y s i s i n  23  the c o n t e x t o f housing  leads one i n t o an e v o l v i n g , m u l t i -  dimensional i s s u e which e x p l o r e s a c r o s s s e c t i o n o f s o c i e t y ' s s o c i a l , economic, and p o l i t i c a l a f f a i r s .  I t i s a central  p o i n t i n the l i v e s o f people r e p r e s e n t i n g s h e l t e r , a focus of s o c i a l l i f e , Housing  and the p r i v a c y o f the i n d i v i d u a l  household.  i s the l a r g e s t s i n g l e space user i n urban areas,  e f f e c t i n g impacts  on t h e o v e r a l l urban system.  thus  Housing i s  c o n s i s t e n t l y i d e n t i f i e d as a major urban problem l i n k e d w i t h poverty, urban redevelopment, suburban sprawl, s o c i a l u n r e s t , and t r a n s p o r t a t i o n .  H i s t o r i c a l l y , housing has been a focus  f o r s o c i a l p l a n n e r s and reformers s e e k i n g a p o l i c y  instrument  open t o governments w i s h i n g t o promote s o c i a l and economic programmes. Contained w i t h i n the A s s i s t e d Home Ownership Programme a r e s e v e r a l emerging Canadian housing p o l i c y  directions  - s p e c i f i c a l l y the move toward home ownership f o r low-income households  and the attempt  t o d i s p e r s e low income housing  through-  out the urban a r e a . Data on s i m i l i a r programmes i n the U n i t e d S t a t e s and elsewhere purposes.  i n Canada a r e a v a i l a b l e f o r comparative  In a d d i t i o n , home ownership programmes have  a t t r a c t e d the i n t e r e s t o f housing t h e o r i s t s .  Existing  theory i s , however, f o r m u l a t e d on the b a s i s o f r e s e a r c h i n the U n i t e d S t a t e s and needs r e - d e f i n i t i o n i n the Canadian context. The G r e a t e r Vancouver R e g i o n a l D i s t r i c t study a r e a i s a t y p i c a l Canadian urban s e t t i n g . p o p u l a t i o n o f one m i l l i o n p e o p l e .  The r e g i o n supports a  C u r r e n t l y , Vancouver i s  24 confronted  w i t h problems of r i s i n g housing c o s t s .  The  causes o f problems are s i m i l a r to those e x p e r i e n c e d i n o t h e r large centers.  S c a r c i t y of s e r v i c e d l a n d , r e s i s t a n c e to  development by  l o c a l governments, and  continued  are among the p r e s s u r e s f a c i n g p o l i c y makers. r e a c t i o n s to p l a n n i n g  Toward an A p p r e c i a t i o n The  of  focus o f the Evaluation  citizen residents  actions.  several l i m i t a t i o n s .  Re-  a c o n s t r a i n t on the s e l e c t i o n o f l o c a l e  number o f programmes c o n s i d e r e d . depth, o f one  Recent  Limitations:  proposed study has  sources p l a c e d  in-migration  p r o p o s a l s suggest t h a t Vancouver  are aware o f c i t i z e n p a r t i c i p a t i o n aims and  However, the  and  analysis, i n  programme i s j u s t i f i e d on the ground t h a t study i s to d e v i s e  o f one  an e v a l u a t i o n  user o r i e n t e d e v a l u a t i o n  I t i s r e a l i z e d t h a t as one t h e o r e t i c a l study to an e v a l u a t i v e over which the r e s e a r c h e r  has  environment and  to  processes.  moves from a  purely  study the number o f  c o n t r o l decreases.  o f the author to h o l d c o n s t a n t a c o n s i d e r a b l e to manipulate only a few  the  framework.  programme p r o v i d e s the o p p o r t u n i t y  compare t r a d i t i o n a l and  new  The  variables inability  p o r t i o n of  v a r i a b l e s may  as a r e j e c t i o n of t r a d i t i o n a l academic r e s e a r c h  appear  methods.  However, as Schon s t a t e s : In s i t u a t i o n s o f p u b l i c l e a r n i n g i t i s almost never p o s s i b l e t o h o l d some v a r i a b l e s c o n s t a n t while manipulating others. Any i n t e r v e n t i o n a f f e c t s more than one v a r i a b l e . (Schon 71 p. 214).  the  25  the r e l a t i v e funding c o n t r i b u t i o n s necessary  from v a r i o u s  l e v e l s of government i s not c o n s i d e r e d i n d e t a i l . concern i i t was  Since  the  to p r o v i d e an a n a l y s i s from the u s e r ' s p e r s p e c t i v e  s  c o n s i d e r e d t h a t a n a l y s i s of i n t e r g o v e r n m e n t a l  ions added another  i n a p p r o p r i a t e dimension  appropriat-  to an a l r e a d y complex  evaluation. The  absence of a c l e a r l y d e f i n e d h y p o t h e s i s might be  c o n s i d e r e d as a l i m i t a t i o n .  T r a d i t i o n a l l y , research involves  the d i s a g g r e g a t i o n of a problem to i d e n t i f y one which to formulate a s p e c i f i c h y p o t h e s i s .  dimension  upon  Controlled experi-  ments and the a p p l i c a t i o n of s c i e n t i f i c a l l y c o l l e c t e d  'tough  minded' data l e a d l o g i c a l l y to h y p o t h e s i s  theor-  e t i c a l answers to p a r t i c u l a r problems.  t e s t i n g and  In c o n t r a s t , t h i s  d i s s e r t a t i o n i d e n t i f i e s a multi-dimensional issue, i n t e r a c t i n g i n a r e a l world  situation.  The  s e a r c h i s f o r an o p e r a t i o n a l  framework f o r an a n a l y s i s based upon both minded d a t a .  'tough' and  P r o o f s , such as they a r e , e v o l v e through  i n the complexity  of the i s s u e and  and p o l i c y a n a l y s i s .  immersion  i n the r e a l i z a t i o n of the  r e s p o n s i b i l i t y i m p l i e d by recommendations f o r s o c i a l  alternate  'tender'  planning  C o n c l u s i o n s a t b e s t i d e n t i f y emerging  directions. T h i s chapter has o u t l i n e d the author's  concern f o r  the p r o l i f e r a t i o n of s o c i a l a c t i o n programmes which l a c k a process f o r i n c l u d i n g c i t i z e n p a r t i c i p a t i o n i n p l a n n i n g evaluation.  R e a l i z i n g t h i s concern,  the d i s s e r t a t i o n  attempt to evolve v e r y t e n t a t i v e g u i d e l i n e s i n d i c a t i n g  and  will the  v a l u e of i n c o r p o r a t i n g user i n p u t i n t o programme e v a l u a t i o n .  26  Chapter I I o u t l i n e s i n more d e t a i l the expanding s o c i a l consciousness  o f governments as i n d i c a t e d by the  of s o c i a l a c t i o n programmes. provide  E x i s t i n g e v a l u a t i o n methodologies  the b a s i s f o r the i d e n t i f i c a t i o n of a t r a d i t i o n a l  gramme e v a l u a t i o n model.  t r a d i t i o n a l and Concluding  A case study comparison o f  user o r i e n t e d models f o l l o w s i n Chapter  chapters  pro-  Chapter I I I suggests an a l t e r n a t e  u s e r - o r i e n t e d e v a l u a t i o n model.  present  IV.  g u i d e l i n e s f o r the i n c l u s i o n o f  user p a r t i c i p a t i o n i n the e v a l u a t i o n stage of the process.  growth  planning  CHAPTER I I SOCIAL CONSCIOUSNESS IN A GOVERNMENT POLICY SETTING T h i s d i s s e r t a t i o n e v o l v e s from t h r e e The f i r s t concern  assumptions.  i s t h a t governments are e x h i b i t i n g an  increasing  f o r the s o c i a l w e l f a r e of t h e i r c o n s t i t u e n t s .  This  concern i s r e f l e c t e d i n the expanded number o f , and  resources  committed t o , s o c i a l a c t i o n programmes.  i s that  The  second  c i t i z e n s are becoming i n c r e a s i n g l y d e s i r o u s o f p a r t i c i p a t i o n in  the f o r m u l a t i o n of s o c i a l p o l i c i e s designed t o p r o v i d e f o r  their welfare.  And,  finally,  as s o c i e t a l g o a l s change,  p o l i c i e s must be a l t e r e d and programmes r e f o r m u l a t e d t o r e f l e c t emerging d i r e c t i o n s . a c t i o n programmes may  Evaluation of e x i s t i n g  p r o v i d e decision-makers  social  with  an i n d i c a t i o n o f the e x t e n t to which government a c t i o n s a s s i s t i n d i v i d u a l s to share the q u a l i t y o f l i f e  enjoyed  by  society at large. Chapter  I documented the involvement  the p l a n n i n g p r o c e s s . in  Chapter  of c i t i z e n s i n  I I w i l l d e a l w i t h the i n c r e a s e  s o c i a l a c t i o n programmes - u s i n g as a s p e c i f i c example the  e v o l u t i o n o f Canadian F e d e r a l Housing  policies.  Literature  reviews of these, and o t h e r programmes, i n d i c a t e the p r e v a i l i n g s t a t e - o f - t h e - a r t of p o l i c y a n a l y s i s .  This chapter  concludes w i t h the a s s e r t i o n t h a t t r a d i t i o n a l p r o c e s s e s inadequate  f o r the e v a l u a t i o n of programmes both from  are  the  p e r s p e c t i v e o f the user's needs and from the s t a n d p o i n t of user involvement  i n t h i s aspect of the p l a n n i n g p r o c c e s s .  An  28 a l t e r n a t e p r o c e s s i s proposed and t e s t e d  in  subsequent  chapters. Programme P r o l i f e r a t i o n Under an I n c r e a s i n g The n e c e s s i t y  Social  Consciousness  to develop a methodology f o r e v a l u a t i o n  s o c i a l a c t i o n programmes r e s t s on the assumption  that  s t a t e i s i n c r e a s i n g l y assuming the r e s p o n s i b i l i t y o f for  the s o c i a l w e l f a r e o f i t ' s c i t i z e n s .  accepts the w i d e l y h e l d assumption  This  the providing  dissertation  t h a t one of the f o r c e s which  has most changed our l i v e s s i n c e the 19 30's has been the expansion of the f e d e r a l government and f e d e r a l power. The e x t e n t t o which the s t a t e now concerns i t s e l f w i t h and i s an agency f o r change would have been i n c o n c e i v a b l e i n 1930. (Graubard 67 p. 712). Programme P r o l i f e r a t i o n i n the Context of Canadian Policy '•  Housing  ;  The i n c r e a s i n g involvement o f the f e d e r a l government can be seen i n the f i e l d o f h o u s i n g .  P o t e n t i a l l y , housing  a c t i o n s p r o v i d e an a t t r a c t i v e p o l i c y instrument. sector presents a v a r i e t y of levers For example:  employment; r e n t  building subsidies increases  t i o n s or i n c r e a s e s  f o r c o n t r o l of the economy  encourage  investment  r e s t r a i n consumer demand and  i n subsidies  p r i v a t e and p u b l i c s e c t o r s .  of  housing  r e n t c o n t r o l measures dampen down demands f o r  wage i n c r e a s e s ;  vides  The  and reduc-  s h i f t investment between the  In a s o c i a l c o n t e x t , housing p r o -  a means o f r e d i s t r i b u t i n g funds to disadvantaged  society.  i  sectors  of  29 The Canadian government's concern f o r the p r o v i s i o n of housing  i s r e f l e c t e d i n p o l i c y statements c o n t a i n e d i n 9 House o f Commons debates. I f t h e Dominion, the p r o v i n c e s and the m u n i c i p a l i t i e s can do anything i n these t r o u b l e d times t o put people i n b e t t e r l i v i n g c o n d i t i o n s , they w i l l be doing a wonderful work t o make them happy and prosperous. (Carvell, Minister of P u b l i c Works, Feb. 25, 1919 p. 45). I t i s the aim o f t h i s government t o p r o v i d e good housing a t reasonable c o s t as a s o c i a l g o a l and government o b l i g a t i o n . (Basford, M i n i s t e r o f S t a t e f o r Urban A f f a i r s , March 15, 1973 p. 2257). The  r o l e o f the F e d e r a l government i n t h e housing  s e c t o r has expanded p r o g r e s s i v e l y s i n c e the i n t r o d u c t i o n o f the f i r s t  s t a t u t e i n 1919.  the f i e l d o f housing  The government o r i g i n a l l y  i n 1919 when shortages  entered  o c c a s i o n e d by  World War I encouraged f e d e r a l a u t h o r i t i e s t o make money a v a i l a b l e t o the p r o v i n c e s  f o r house c o n s t r u c t i o n . ^  f i r s t g e n e r a l p i e c e o f f e d e r a l housing  l e g i s l a t i o n was the  Dominion Housing A c t passed i n 1935 t o a l l e v i a t e conditions.  The  unemployment  T h i s was f o l l o w e d by the N a t i o n a l Housing A c t s  of 1938 and 1944, c u l m i n a t i n g ,  i n 1954, w i t h  the N a t i o n a l  Housing A c t (NHA), d e f i n e d a s : An a c t t o promote t h e c o n s t r u c t i o n o f new houses, the r e p a i r and m o d e r n i z a t i o n o f e x i s t i n g houses and t h e improvement o f housing and l i v i n g conditions. C e n t r a l Mortgage and Housing C o r p o r a t i o n Crown agency i n c o r p o r a t e d by A c t o f Parliament administers  (CMHC), a  i n 1945,  the N a t i o n a l Housing A c t and c o o r d i n a t e s the  a c t i v i t i e s o f t h e F e d e r a l Government i n Housing.  The  30 C o r p o r a t i o n has the r e s p o n s i b i l i t y f o r a v a r i e t y o f f u n c t i o n s a f f e c t i n g housing  i n i t s long-range outlook as w e l l as i n i t s  immediate requirements.  CMHC i s empowered t o a c t as an  i n s u r e r o f mortgage l o a n s , as a l e n d e r o r i n v e s t o r o f p u b l i c funds, as a guarantor, assets.  and as an owner o f p r o p e r t y and o t h e r  Programme o p t i o n s a t p r e s e n t i n c l u d e i n s u r e d mortgage  loans on new and e x i s t i n g housing,  d i r e c t loans t o r u r a l  areas,  loans f o r home improvement, s u b s i d i z e d r e n t a l accommodation, s u b s i d i z e d home ownership, l a n d assembly, sewerage  treatment,  new communities, renewal, neighbourhood improvement and a s s i s t a n c e t o s p e c i a l i n t e r e s t groups such as n a t i v e students, e l d e r l y and c o o p e r a t i v e o r g a n i z a t i o n s .  peoples,  The  C o r p o r a t i o n a c t s as a r e s e a r c h agency i n f i e l d s a s s o c i a t e d w i t h housing  and a s s i s t s p r o v i n c e s and m u n i c i p a l i t i e s i n many  aspects o f urban growth. Housing as an Economic T o o l D e s p i t e the v a r i e t y o f i n t e r v e n t i o n o p t i o n s a b l e , and the p o t e n t i a l a t t r a c t i v e n e s s o f housing t o o l , implementation was  avail-  as a p o l i c y  o f l e g i s l a t i o n p r i o r t o the mid 1960's  c o n s t r a i n e d by p r e v a i l i n g a t t i t u d e s .  r e s t r i c t e d t o the p r o v i s i o n o f housing  F e d e r a l a c t i o n s were  programmes designed t o  r e g u l a t e the economy. I n i t i a l l y the F e d e r a l Government showed a r e l u c t a n c e to assume r e s p o n s i b i l i t i e s c o n s t i t u t i o n a l l y a l l o c a t e d t o the 12 provinces  o r which, i n a market economy, c o u l d be borne by  31 private enterprise. programmes p l a c e d  13  These c o n s t r a i n t s meant t h a t f e d e r a l  the onus on p r i v a t e i n d u s t r y t o c o n s t r u c t  u n i t s and r e q u i r e d p r o v i n c i a l c o s t s h a r i n g and l o c a l t i v e p r i o r t o implementation. successive  initia-  The government, through  Housing A c t s , attempted t o s t i m u l a t e  and supplement  the market f o r housing by i n c r e a s i n g the flow o f mortgage money and encouraging l e n d e r s  t o make loans on more  favour-  14 able terms t o p r o s p e c t i v e  home-owners.  P r o v i s i o n o f housing was seen t o be the p r i v a t e r e s p o n s i b i l i t y o f the i n d i v i d u a l household.  Where housing  a s s i s t a n c e was made a v a i l a b l e t o lower-income f a m i l i e s , was  i t  viewed as a temporary haven w h i l e t h e i r f i n a n c e s were reorgan-  i z e d f o r another a s s u l t on the market. T h i s concept assumes t h a t t h e poor have i t w i t h i n t h e i r power t o c o r r e c t whatever  conditions  l e d them i n t o p o v e r t y ; a view c u r r e n t l y shared by few (Adams 71) . The  Emergence o f Housing as a S o c i a l Concern By  necessary.  1968 a r e d i r e c t i o n of f e d e r a l a c t i o n s I t had become e v i d e n t  that e x i s t i n g federal  p o l i c i e s had f a i l e d t o a l l e v i a t e housing a s s o c i a t e d During the 1960's the p r i c e o f s h e l t e r rose other  appeared  problems.  f a s t e r than any  component o f the Consumer P r i c e Index.  The N a t i o n a l  Housing A c t was s e r v i n g upper-middle-income people; g i v i n g them the p r o t e c t i o n o f pegged i n t e r e s t r a t e s and p r o v i d i n g s e r v i c e d land assembly p r o j e c t s .  L i t t l e was being  the growing numbers o f people e x p e r i e n c i n g  done f o r  difficulty  finding  32 accommodation. expectations fill  E x i s t i n g assumptions o f s t e r e o t y p e d  and  of the a b i l i t y of the p r i v a t e market t o  a d i v e r s i t y o f needs were q u e s t i o n e d .  u n i t s and  consumer  Availability  p a r t i c i p a t i o n i n the p r o v i s i o n o f housing by  fulof  the  15 consumer became, a t l e a s t i n r h e t o r i c , a RIGHT. Coupled w i t h housing c o s t s was ness o f the  interrelatedness  reports published  o f urban problems.  between 1967  o f urban housing problems.  an i n c r e a s i n g aware-  and  1973  i n d i c a t e d the  For example, the  1967  C o u n c i l o f Canada Review i d e n t i f i e d some o f the issues.  In 1968  A v a r i e t y of  Economic  emerging  a F e d e r a l - P r o v i n c i a l c o n f e r e n c e on Task Force on Housing and  extent  Housing  and  the F e d e r a l  led  by Paul H e l l y e r , formulated recommendations c o n c e r n i n g  (1) d u a l r o l e s o f p r i v a t e and  Urban Development,  public sectors  i n housing;  (2)  f i n a n c i n g changes to make mortgages a v a i l a b l e to more consumers; (3) l a n d c o s t r e g u l a t i o n s ; management programmes.  (4) s o c i a l housing; and  In 197o,  Lithwick,  s t u d i e s , i d e n t i f i e d Urban Canada: urban p r o c e s s e s a t work and  (5)  urban  i n a s e r i e s of  Problems and  Policies.  the i n t e r r e l a t e d n e s s  The  o f urban  i s s u e s l e d t o recommendations c o n c e r n i n g a l t e r n a t e urban futures. was  Probably the most c r i t i c a l  i s s u e d i n 19 72 by Dennis and  c r i t i c i z e d the F e d e r a l households.  The  the p r o d u c t i o n  income f a m i l i e s .  Fish.  federal  They a n a l y z e d  actions and  Housing programmes f o r low-income  Charney r e p o r t  and  r e p o r t on  followed  w i t h an a n a l y s i s  adequacy o f housing produced f o r  low-  of  33 F e d e r a l l e g i s l a t i o n responded  with p r o v i s i o n s to  low-income groups which r e f l e c t e d a more p o s i t i v e a p p r e c i a t i o n of demand. of  P u b l i c a i d was  t r a n s f e r r e d from the  the p r i v a t e middle-income housing market to the  a s s i s t a n c e o f low-income groups.  direct  Ottawa thereby embarked on  a s o c i a l housing p o l i c y whereby the S t a t e was  prepared t o  i n t e r v e n e on a s e l e c t i v e and l i m i t e d s c a l e to a s s i s t income households.  support  For the purposes  lower-  of housing a s s i s t a n c e ,  lower-income f a m i l i e s were d e f i n e d as a f a m i l y t h a t r e c e i v e s a t o t a l f a m i l y income t h a t , i n the o p i n i o n of the C o r p o r a t i o n , i s i n s u f f i c i e n t t o permit i t t o r e n t housing accommodation adequate f o r i t s need a t the c u r r e n t r e n t a l market i n the area i n which the f a m i l y l i v e s . (N.H.A. 73, p. 3). U t i l i z i n g the v a r i e t y of programmes and  funds  a v a i l a b l e as i n d i c a t o r s o f the i n t e n s i t y w i t h which p r o grammes are b e i n g pursued, d i r e c t i o n s are emerging.  i t i s e v i d e n t t h a t new In 1968,  a p p r o v a l was  d i r e c t loans i n v o l v i n g 15,221 d w e l l i n g u n i t s . 1970,  The t o t a l d i r e c t f e d e r a l  these two years were $257 m i l l i o n and  respectively. of  given to For the year  these f i g u r e s were 43,494 s e l f c o n t a i n e d u n i t s  20,263 h o s t e l u n i t s . for  policy  F u r t h e r , out o f the i n i t i a l  $854 m i l l i o n ,  m i l l i o n , two groups."^  and  investments  $596 m i l l i o n housing budget  l a t e r r a i s e d t o a r e c o r d high o f $1,094  t h i r d s was  By 1973  d i r e c t e d towards a i d to low-income  c l o s e to h a l f of the country's housing  stock o f approximately  6,300,000 housing u n i t s had been  c o n s t r u c t e d s i n c e the f i r s t  c o v e r i n g l e g i s l a t i o n was  enacted.  34  About o n e - t h i r d o f these were f i n a n c e d under the N a t i o n a l Housing A c t s . NHA  i n one  way  or another  Recent amendments to  suggest the F e d e r a l Government i s moving from an  on s t a r t s toward a mix  the  emphasis  o f s o l u t i o n s to improve the q u a l i t y o f 17  the t o t a l r e s i d e n t i a l environment. i n c r e a s i n g l y s e n s i t i v e to l o c a l and to the need f o r c h o i c e  New  policies  are  regional differences  i n tenure, l o c a t i o n and  and  financing  arrangements.^ The  assumption concerning  the i n c r e a s i n g  consciousness o f governments, as i n d i c a t e d by and  social  funds committed  programmes enacted i n the housing s e c t o r , appears  fied.  As  justi-  l a r g e r p o r t i o n s o f p u b l i c funds have been committed  to s o c i a l programmes, governments engaged i n s u p p l y i n g  these  community s e r v i c e s are i n c r e a s i n g l y r e q u i r e d to p r o v i d e dence o f programme e f f e c t i v e n e s s as a b a s i s f o r p u b l i c support.  continued  However, when e v a l u a t i o n o f programmes d e a l i n g  w i t h the p r o v i s i o n of a s s i s t a n c e ment has  evi-  occurred,  f o r human resource  develop-  t e n t a t i v e f i n d i n g s o f t e n suggest t h a t p r o -  grammes are having minimum impact on problems. In response to questions  of  'why', the r e a l i z a t i o n of the  of problems, and  increased  complexity  requirements f o r a c c o u n t a b i l i t y , government  agencies were encouraged to i n i t i a t e e v a l u a t i v e r e s e a r c h Programme E v a l u a t i o n The  as an I n d i c a t o r o f S o c i e t a l D i r e c t i o n s  Nature o f E v a l u a t i v e The  programmes.  Research  range o f f o r c e s encouraging the development o f  evaluative research  - changes i n the nature o f s o c i a l problems;  35  changes i n the s t r u c t u r e and changes i n the needs and  f u n c t i o n of p u b l i c agencies;  and  e x p e c t a t i o n s of the p u b l i c - have  r e s u l t e d i n e v a l u a t i v e s t u d i e s b e i n g undertaken f o r a v a r i e t y 19 of reasons, of programmes.  u t i l i z i n g v a r i o u s methodologies, on a wide range The very nature o f e v a l u a t i v e r e s e a r c h , i t s  f u n c t i o n and d e f i n i t i o n s , r e f l e c t t h i s Diverse  complexity.  Purposes: E x i s t i n g e v a l u a t i v e r e s e a r c h has been c l a s s i f i e d i n  numerous ways.  One  c l a s s i f i c a t i o n i d e n t i f i e s the stage i n  the p l a n n i n g process a t which e v a l u a t i o n i s undertaken. P r i o r p o l i c y e v a l u a t i o n i s concerned w i t h mining the magnitude of need and  deter-  the most s u i t a b l e d i s t r i b u -  t i o n o f r e s o u r c e s w i t h i n a g i v e n programme, or between a l t e r n a t e programmes, to reach c e r t a i n s t a t e d o b j e c t i v e s . F r e q u e n t l y i n p r i o r p o l i c y e v a l u a t i o n , experimental demonstration  p r o j e c t s are i n i t i a t e d and e v a l u a t e d  a s c e r t a i n whether proposed programmes adequately  or to  fulfill  s t a t e d needs. E v a l u a t i o n s of e x i s t i n g p o l i c y c o n s i d e r e f f o r t and/or e f f i c i e n c y of programmes. undertaken d u r i n g programme o p e r a t i o n  effect,  A n a l y s i s has  (formative  been  evaluation)  and a t the c o n c l u s i o n or d i s c o n t i n u a t i o n of programme a c t i o n s (summative e v a l u a t i o n ) . E f f e c t s t u d i e s focus on g o a l f u l f i l l m e n t . grammes are e v a l u a t e d w i t h a view to determining  the  Prooverall  e f f e c t i v e n e s s of a p a r t i c u l a r programme i n r e a c h i n g  stated  o b j e c t i v e s or o f the r e l a t i v e e f f e c t i v e n e s s o f a l t e r n a t e p r o grammes w i t h a view to recommending f u t u r e s t r a t e g y d i r e c t i o n s . Riecken m a i n t a i n s t h a t s t u d i e s o f e f f e c t s r e p r e s e n t the maximum cont r i b u t i o n t h a t s o c i a l s c i e n c e can make to s o c i a l p r a c t i c e , s i n c e they are u s u a l l y i n tended t o feedback r e s u l t s i n t o program p l a n n i n g or p o l i c y making. (Riecken 72, p. 87). Programme e f f i c i e n c y s t u d i e s attempt t o assess r e l a t i v e c o s t s of. a c h i e v i n g gramme.  Unfortunately,  the  the o b j e c t i v e s s e t f o r a p r o -  there are  few  successful applications  to s o c i a l a c t i o n programmes. A n a l y s i s o f e f f o r t or programme a c c o u n t a b i l i t y i s concerned w i t h the k i n d and  q u a n t i t y o f a c t i v i t i e s engaged i n  to s a t i s f y programme o b j e c t i v e s .  T y p i c a l l y , programme i n p u t  (funds) forms the b a s i s o f e v a l u a t i o n a g a i n s t output ( i . e . number o f c l i e n t s s e r v e d ) .  Rossi's  s t u d i e s undertaken f o r the U.S.  In a sample o f 200  jects contracted  by OEO,  Rossi  reached.  evaluative  the most customary form o f  out of 1,12 3 r e s e a r c h found t h a t 170  were mainly d e s c r i p t i v e accounts o f how being  review of  O f f i c e o f Economic Opportu-  n i t y shows " e f f o r t s t u d i e s " t o be evaluation.  (69)  pro-  (85 percent)  many people were  These r e p o r t s were p r e s e n t e d p r i m a r i l y i n  n a r r a t i v e form w i t h " v i r t u a l l y no the e f f e c t i v e n e s s o f programs"  systematic  (Rossi 69 p.  observations 218).  of assessment appears to be extremely l i m i t e d and lacking in u t i l i t y e f f e c t i v e n e s s and  unless  accompanied by  efficiency.  This  on kind  generally  information  on  37 O p e r a t i o n s a n a l y s i s attempts  t o c o n s i d e r the means  or o p e r a t i o n of programme p r o c e s s e s w i t h o u t s p e c i f i c t o ends.  Weiss and Rein  (72), f o r example, suggest  attention that  s t u d y i n g the implementation p r o c e s s f o r l a r g e s c a l e , d i m e n s i o n a l programmes may to i s o l a t e s p e c i f i c  multi-  be o f more v a l u e than attempting  variables.  The a n a l y t i c a l t e c h n i q u e s a p p l i e d t o e v a l u a t i v e r e s e a r c h show no l e s s d i v e r s i t y . F e l l i n and Meyer  (71) groups  A schema d e v i s e d by  the v a r i e t y o f  Tripodi,  methodologies  which have proven u s e f u l f o r d i f f e r e n t types of e v a l u a t i o n  into:  1.  M o n i t o r i n g t e c h n i q u e s , i n c l u d i n g procedures used f o r the d i r e c t review o f programme o p e r a t i o n s ; accountab i l i t y a u d i t , a d m i n i s t r a t i v e a u d i t , and time-motion studies.  2.  S o c i a l Research Techniques, r e f e r r i n g t o procedures which exclude c o s t c o n s i d e r a t i o n s and which are used f o r d e v e l o p i n g , m o d i f y i n g and expanding knowledge about the programme which can be communicated and v e r i f i e d by independent i n v e s t i g a t o r s : experiment, survey and case study.  3.  Cost a n a l y t i c t e c h n i q u e s , r e f e r r i n g t o procedures used t o a p p r a i s e the r e l a t i v e v a l u e of a programme i n r e l a t i o n t o programme c o s t s : cost accounting, c o s t - b e n e f i t a n a l y s i s , cost-outcome a n a l y s i s , and o p e r a t i o n a l r e s e a r c h which b i n d s e x p e r i m e n t a l and c o s t - a n a l y t i c methods. The d i v e r s i t y o f e v a l u a t i v e o b j e c t i v e s and method-  o l o g i e s i s i n d i c a t e d by t h i s overview.  G e n e r a l l y , the  r a t i o n a l e behind such r e s e a r c h has been t o p r o v i d e a base f o r improved  decision-making.  However, as Weiss  (72) p o i n t s out,  i n p r a c t i c e e v a l u a t i o n i s sometimes undertaken motives:  for less  noble  38  Program decision-makers may t u r n to e v a l u a t i o n to d e l a y a d e c i s i o n ; to j u s t i f y and l e g i t i m a t e a d e c i s i o n a l r e a d y made; t o e x t r i c a t e themselves from c o n t r o v e r s y about f u t u r e d i r e c t i o n s by p a s s i n g the buck; to v i n d i c a t e the program i n the eyes o f i t s c o n s t i t u e n t s , i t s funders o r the p u b l i c ; t o s a t i s f y c o n d i t i o n s o f a government o r foundat i o n grant through the r i t u a l o f e v a l u a t i o n . (Weiss 72, p. 14). Diverse D e f i n i t i o n s : Evaluative research vities  covers  a wide range of  i n v o l v i n g the assessment o f s o c i a l a c t i o n s .  e x i s t s no s i n g l e accepted surprising.  That  Rather, a v a r i e t y o f meanings have been  and o p e r a t i o n a l d e f i n i t i o n s w i t h  conceptual the  o f the many d e f i n i t i o n s  they i n c l u d e some s i m i l a r i t i e s ,  d i v e r g e n c i e s both on p o i n t s o f scope and Wholey  assigned  the g r e a t e r emphasis on  Examination o f j u s t a few  shows t h a t w h i l e  there  d e f i n i t i o n o f the concept i s not  t o the term r e v e a l i n g an i n e x t r i c a b l e mixture o f  latter.  acti-  (70), T r i p o d i (71), and  there  are  methodology. Freeman and  Sherwood  (70) view e v a l u a t i o n i n a somewhat narrow c o n t e x t .  They  e v a l u a t i o n as f o c u s s i n g on s p e c i f i c programmes, the  identifi-  c a t i o n o f programme g o a l s , and Freeman and  see  subsequent magnitude o f change.  Sherwood, f o r example, see e v a l u a t i o n  as:  e s s e n t i a l l y a procedure f o r c o n t r o l i n q u a l i t y and q u a n t i t y . A mechanism f o r determining whether the a c t u a l conduct of programs i s cons i s t e n t with the i n t e r v e n t i o n s t r a t e g y outl i n e d i n the development of the program. (Freeman and Sherwood 70, p. 70). A broader d e f i n i t i o n of e v a l u a t i o n i s seen i n the works o f , f o r example, J e n k i n s  (61), A l k i n  (72)  and Weiss  (72).  39  To these authors, programme e v a l u a t i o n the e x t e n t to which o b j e c t i v e s p r i n c i p l e s or t h e o r i e s  attempts to determine  are b e i n g f u l f i l l e d ,  underlying  the  the programme which account  f o r success o r f a i l u r e , whether i d e n t i f i e d changes are programme measures or extraneous f a c t o r s and implications  o f such changes.  Jenkins defines  evaluation  t o determine  In e f f e c t , e v a l u a t i o n  l e a r n i n g p r o c e s s - l e a r n i n g why  to the  is a  programmes succeed o r  operationally  due  fail.  as:  the p r o c e s s o f a c q u i r i n g , a n a l y z i n g and u s i n g information for decisions associated with p l a n n i n g , programming, implementing and r e c y c l i n g program components and a c t i v i t i e s . (Jenkins 61, p. 20 ) . As  the  scope o f e v a l u a t i v e  merges w i t h the g e n e r a l l y policy analysis. t i v e research  research  broadens, i t  more a l l - e n c o m p a s s i n g concept of  Where p o l i c y a n a l y s i s d i f f e r s  i s i n evaluation's  preoccupation with e x i s t i n g  programmes.  Policy analysis  hypothetical  a l t e r n a t i v e s o l u t i o n s to the  o b j e c t i v e of s p e c i f y i n g c h o i c e s as a b a s i s  u s u a l l y compares e x i s t i n g  w i l l be  example, A l k i n  and  same problem, w i t h  a l t e r n a t i v e p o l i c y and  programme  f o r decision-making.  For the purposes of t h i s d i s s e r t a t i o n , research  from e v a l u a -  interpreted  evaluative  i n the broader c o n t e x t o f ,  (72):  E v a l u a t i o n i s the p r o c e s s o f a s c e r t a i n i n g the d e c i s i o n areas o f concern, s e l e c t i n g a p p r o p r i a t e i n f o r m a t i o n and c o l l e c t i n g and a n a l y z i n g i n f o r m a t i o n i n order t o r e p o r t summary data u s e f u l to d e c i s i o n makers i n s e l e c t i n g among a l t e r n a t i v e s . ( A l k i n 72, p. 107).  for  the  40  T h i s broader d e f i n i t i o n appears perspectives.  Firstly,  as d i s c u s s e d i n Chapter  system o f p l a n n e r and c i t i z e n may .  .  justified  from  I, the value  indeed d i f f e r on  .  the  20  i d e n t i f i c a t i o n of programme g o a l s and s o l u t i o n s . the proposed  two  To base  u s e r - o r i e n t e d e v a l u a t i o n p r o c e s s on the narrower  f o u n d a t i o n of e x p e r t - e n u n c i a t e d g o a l s and t o then proceed compare the subsequent impact r e c i p i e n t s may  to  of these programme g o a l s on  w e l l l e a d t o an i n a p p r o p r i a t e a n a l y s i s o f  programme impact  from the v i e w p o i n t o f the u s e r . .  Secondly, most c o n c e p t u a l i z a t i o n s o f e v a l u a t i o n r e s e a r c h are based  upon an a n a l y s i s o f s i n g l e purpose p r o -  grammes i n , f o r example, the f i e l d s o f h e a l t h o r e d u c a t i o n . In such c o n t e x t s , the t r a d i t i o n a l e x p e r i m e n t a l d e s i g n method of a n a l y s i s appears  appropriate.  the housing e v a l u a t i o n proposed  However, i n the c o n t e x t o f as the focus o f t h i s  the more a l l encompassing m u l t i - d i m e n s i o n a l concerns  study, of  p o l i c y a n a l y s i s appear more r e a l i s t i c . The p r e c e e d i n g d i s c u s s i o n has i n d i c a t e d t h a t e v a l u a t i o n i s i n no way  a u n i d i m e n s i o n a l concept.  o f e v a l u a t i o n r e s e a r c h shows the o r i g i n s of these d i r e c t i o n s and i n d i c a t e s problems a t t e n d a n t t o the  A  review  varied tradi-  t i o n a l evaluation process. The E v o l u t i o n of E v a l u a t i v e Research The growth and development of e v a l u a t i v e r e s e a r c h c l o s e l y p a r a l l e l s the e v o l u t i o n of the p u b l i c s e r v i c e movement and the changing  acceptance  of the r e s p o n s i b i l i t y of  41 "relevance"  i n various s o c i a l science  disciplines.  As a P o l i c y T o o l : A t t i t u d e s presented  i n Chapter I j u s t i f y i n g  initial  e x c l u s i o n o f c i t i z e n involvement i n the  process  are e q u a l l y a p p l i c a b l e i n e x p l a i n i n g the  planning reluctance  o f many agencies to undertake e v a l u a t i v e r e s e a r c h . the mid o f new  twentieth  century,  the accent was  on the  Member of Parliament House as to why question  P r i o r to  initiation  s e r v i c e s to meet demand r a t h e r than on the  o f e x i s t i n g programmes.  evaluation  This a t t i t u d e i s exemplified  Dunning, who,  home b u i l d i n g was  the  when q u e s t i o n e d  by  i n the  i n a slump, r e j e c t e d the  as i r r e l e v a n t . I would not undertake t o send out an i n v e s t i g a t o r to f i n d out why people d i d not b u i l d houses. The reasons why they have not done so d u r i n g the l a s t number o f years would range over a very wide v a r i e t y o f o p i n i o n s and would d i f f e r i n v a r i o u s p a r t s o f Canada. So I doubt any p r a c t i c a l good would come o f a p p o i n t i n g people t o f i n d out why houses have not been b u i l t i n s u f f i c i e n t numbers. I would p r e f e r to proceed on the c o n s t r u c t i v e l i n e s which the b i l l contemplates by making i t e a s i e r f o r houses to be b u i l t i n the future. (Dunning, Hansard, June 13, 19 38, p. 3780). Such e v a l u a t i o n as d i d occur  time p e r i o d took the e v a l u a t i o n schedules, s e l f assessment.  and  grading  standards f o r community  S t a t e s , C. V. Chapin  a s e r i e s o f r a t i n g sheets  problem areas i n each s t a t e . t h i s r e s e a r c h was  earlier  form of community a p p r a i s a l forms,  In the U n i t e d  f o r example, d e v i s e d  during t h i s  The  to  (21),  identify  main c h a r a c t e r i s t i c  i t s b a s i s on the judgment of a few  of pro-  42 f e s s i o n a l a d m i n i s t r a t o r s . That these e v a l u a t i v e were inadequate becomes i n c r e a s i n g l y e v i d e n t .  procedures  In  1944,  Member of P a r l i a m e n t S t a n l e y Knowles e n u n c i a t e d t h i s when he c r i t i c i z e d the government's slowness  to  r e s e a r c h deemed necessary by the C u r t i s Report  concern  initiate (44) on the  s i t u a t i o n o f housing i n Canada. Requests  f o r e v a l u a t i v e r e s e a r c h have been slow  i n r e a c h i n g implementation. ment  Only a few s e c t o r s o f the govern-  ( f o r example, the Treasury Board) and s p e c i f i c programmes  (such as the L o c a l I n i t i a t i v e s Programme) i n c l u d e e v a l u a t i o n as p a r t o f the on-going p l a n n i n g p r o c e s s . Hartle's  (72) work p r o v i d e s an example o f the  e v a l u a t i v e schemes b e i n g developed i n the T r e a s u r y Board.  He  f e e l s the i d e a l i n f o r m a t i o n i n p u t i n t o a c t u a l p o l i c y making i s a sort of i n f i n i t e matrix.  The rows r e p r e s e n t i n g the  range  o f . a c t i o n s , programmes o r i n t e r v e n t i o n s open t o the a r b i t e r . The column headings  s h o u l d be thought o f i n two  tiers,  the  uppermost b e i n g statements o f the o b j e c t i v e s or g o a l s t h a t concern the a r b i t e r .  H a r t l e , b e i n g an economist,  sees the  maximization o f the aggregate net worth of a l l the of the n a t i o n as the source o f these g o a l s . not as n a i v e as i t sounds, net worth i s comprehensive.  citizens  This i s a c t u a l l y  as h i s f o r m u l a t i o n of i n d i v i d u a l H a r t l e ' s p r o c e s s when a p p l i e d t o 22  the r e a l w o r l d l e a v e s unanswered s e v e r a l (a)  concerns:  A c t u a l behaviour o f government d i f f e r s  u n i t a r y p e r f e c t i o n of the i d e a l model.  from the  Major Canadian  cities  u s u a l l y have f o u r l e v e l s o f government p l u s o t h e r autonomous  43  agencies s u p p l y i n g p u b l i c goods. (b)  Problems o f u n c e r t a i n t y w i t h  o t h e r governments o r (c)  agencies.  P o l i t i c a l and b u r e a u c r a t i c d e s i r e s to  or enhance (d)  r e s p e c t to a c t s o f  s t a t u s , f u n c t i o n s and  preserve  permanence.  Severe r e s t r i c t i o n s on the range o f a c t i o n s open  to each l e v e l , which i s s u f f i c i e n t to guarantee sub-optimal performance even i n the absence of i n s t i t u t i o n a l c o n s t r a i n t s . P a r t o f the o v e r a l l r e t i c e n c e to a l l o c a t e funds f o r e v a l u a t i o n i s the r e l u c t a n c e of a d m i n i s t r a t o r s programmes to i n q u i r y .  to submit t h e i r  A second, e q u a l l y r e a l i s t i c  t i o n , l i e s i n a perceived  explana-  l a c k of a s u i t a b l e e v a l u a t i o n methodology.  The E v o l u t i o n of E v a l u a t i v e Research as a Concern o f So~cial S c i e n t i s t : S e v e r a l academic d i s c i p l i n e s have methodologies t o e v a l u a t e  the  devised  social actions.  P o l i t i c a l Science:  The  emerging f i e l d o f p o l i c y  s c i e n c e s attempts t o account f o r the development o f p u b l i c p o l i c y by a n a l y z i n g c h o i c e s and  a n t i c i p a t e d programmes.  a n a l y s i s and  and  assumptions u n d e r l y i n g  Methods employed are network  case s t u d i e s o f p o l i c y making b e h a v i o u r .  s t u d i e s which c o n s i d e r u s i n g v o t i n g records Economics:  Many  the behaviour of the c i t i z e n do so  as a proxy f o r c i t i z e n has  by  viewpoints.  u n t i l recently considered p o l i c y  from the t r a d i t i o n a l market viewpoints labour economics.  present  of p u b l i c f i n a n c e  and  E v a l u a t i v e s t u d i e s i n economics have t h e i r  44  antecedents i n p r i v a t e market r e s e a r c h appraisals  and  in project  i n c o n n e c t i o n w i t h i n t e r n a t i o n a l loans through  World Bank, whose a r t i c l e s o f agreement r e q u i r e such tion.  This evaluation  various  has  contributed  cost-effectiveness  evalua-  to the development o f  project a p p r a i s a l techniques.  parisons,  the  Cost-benefit  a n a l y s i s , and  com-  programme  budgeting are based on the assumption t h a t the amount of money spent i n a j u r i s d i c t i o n i n d i c a t e s the nature and  quality  23  of s e r v i c e s provided. Cost-benefit  a n a l y s i s attempts to p r o v i d e a method-  ology to e v a l u a t e programmes, to serve as a p a r t i a l device  to e l i m i n a t e  obviously  uneconomical p r o j e c t s , and  base, however inadequate, f o r r a n k i n g and  screening  choosing among a l t e r n a t i v e s .  and  Teitz  comparing  (6 8) suggests  solutions defined Davis, S c o t t and  benefits of  (65)  costurban  alternate  by government performance measures. Ross  a  projects  e f f e c t i v e n e s s a n a l y s i s as an approach to q u a n t i f y i n g s e r v i c e o b j e c t i v e s w i t h the c o s t s and  as  Sewell,  p r o v i d e an example framework f o r  c o s t - b e n e f i t a n a l y s i s as a s y s t e m a t i c  approach to  evaluation  i n the s e l e c t i o n o f Canadian r e s o u r c e development p r o j e c t s . The  examples i n the guide r e l a t e t o water r e s o u r c e s .  The  p r i n c i p l e s have wider a p p l i c a t i o n . While the  f i e l d of c o s t - b e n e f i t a n a l y s i s  includes  a wide range of s t u d i e s , t e c h n i q u e s f o r assembling q u a n t i f i a b l e c o s t s and not e v i d e n t . there  benefits  for rigorous  non-  comparison  D e s p i t e the growing l i t e r a t u r e i n t h i s  i s an a t t e n d a n t concern t h a t ,  i n view of the  are  field  difficulty  45  of q u a n t i f y i n g "human b e n e f i t s " , the methodology may  have  l i m i t e d a p p l i c a b i l i t y to s o c i a l planners. Faced w i t h the need t o examine programmes o b j e c t i v e s , economists,  and  working f o r f e d e r a l governments both 24  i n Canada and the U n i t e d S t a t e s ,  have moved  s i n c e the  1960's, toward the use of P l a n n i n g , Programming and Systems  (PPBS) as t h e i r main a n a l y t i c a l t o o l .  mid  Budgeting  To Wright  (69),  PPBS i s not so much a body o f techniques as i t i s a v i e w p o i n t . I t i s the view t h a t the m a j o r i t y o f government a c t i v i t y to do w i t h the a l l o c a t i o n o f r e s o u r c e s .  has  As such, government  a c t i v i t y s h o u l d be the s u b j e c t o f economic a n a l y s i s . The crux of PPBS i s a s y s t e m a t i c a n a l y s i s o f a l t e r n a t e ways o f meeting government o b j e c t i v e s .  PPBS i s a  25  t o t a l programming systems approach  a s k i n g not A, B, o r C but  r a t h e r "Given our o b j e c t i v e s and r e s o u r c e s what i s the optimum mix of A, B, and C?", t i c u l a r combination  the optimum mix b e i n g the p a r -  o f o b j e c t i v e s , r e s o u r c e s and s c h e d u l i n g  which comes c l o s e s t t o the g r e a t e s t e f f i c i e n c y and ness.  PPBS does not make c h o i c e s but i t does h e l p t o make  c l e a r the i n g r e d i e n t s o f c h o i c e and the probable of  effective-  consequences  alternatives. There are s e v e r a l c r i t i c i s m s of PPBS as a t o o l t o  analyze s o c i a l programmes.  Herbert B a l l s  PPBS i n a Canadian c o n t e x t suggests  (70) i n d i s c u s s i n g  t h a t more emphasis  been p l a c e d on programming and budgeting  and  and systems a n a l y s i s than perhaps should be.  has  l e s s on p l a n n i n g The method i s  46 c o n s i d e r e d of most use on programmes w i t h c l e a r ,  non-contro-  v e r s i a l g o a l s , where t h e r e i s c e n t r a l i z e d d e c i s i o n making and where c o s t s and b e n e f i t s can be measured - c o n d i t i o n s seldom found i n m u l t i - d i m e n s i o n a l s o c i a l a c t i o n programmes. S e v e r a l urban economists a p p l i c a t i o n of mathematical  have e x p l o r e d the p o t e n t i a l  programming models t o the e v a l u a -  t i o n of a l t e r n a t e urban p l a n s and to the improvement of p r o grammes.  S t u a r t (70), u s i n g the Model C i t i e s programme as an  example, develops a simple l i n e a r programming model b u i l t around a m a t r i x o f r e l a t i v e e f f e c t i v e n e s s c o e f f i c i e n t s , a s e t of performance standards, and a p p r o p r i a t e programme  budgets.  The problem of t h i s , and s i m i l a r models, i s t h a t the most important  u n r e s o l v e d i s s u e s i n m o d e l l i n g s t r a t e g y are  still 2 6  the i n c o r p o r a t i o n o f b e h a v i o u r a l concepts  i n t o the model.  Amenity, a e s t h e t i c s , s o c i a l g o a l s and e f f e c t s are among the c l a s s o f v a r i a b l e s which have been o m i t t e d or convenience)  from most computer models.  (through o v e r s i g h t There i s then  some doubt c o n c e r n i n g the v a l i d i t y o f u s i n g l i n e a r programming models as a b a s i s f o r s o c i a l p l a n n i n g . Some evidence e x i s t s o f attempts impact  of the investment  t o c o n s i d e r the  of human c a p i t a l i n the e v a l u a t i o n  o f p o v e r t y and e d u c a t i o n programmes.  In macro economics,  n a t i o n a l economic a c c o u n t i n g has expanded to i n c l u d e the ment i n human r e s o u r c e s under the r u b r i c of s o c i a l In w e l f a r e economics, i n d i f f e r e n c e curve a n a l y s i s a p p l i e d by L i t t l e  (65) t o behaviour  invest-  accounts. concepts,  l i n e s , t e s t the  success  47 of a programme by whether the c l i e n t advanced to a behaviour  "higher"  line. However, the o v e r a l l c r i t i c i s m o f economic programme  e v a l u a t i o n remains.  Measures o f success  those which are e a s i l y q u a n t i f i a b l e and  are i n v a r i a b l y  only  characteristically  ignore n o n - q u a n t i f i a b l e d a t a . T y p i c a l o f such s t u d i e s i s t h a t by Bateman (67)  i n which a  programme f o r work r e - t r a i n i n g i s e v a l u a t e d s o l e l y from the c r i t e r i a of income b e n e f i t s d e r i v e d over programme c o s t s . P o t e n t i a l b e n e f i t s i n terms of user s a t i s f a c t i o n or f a m i l y s t a b i l i t y are  ignored.  S o c i o l o g i s t s , p s y c h o l o g i s t s and  behavioural  s c i e n t i s t s have c o n s i d e r e d the d i v e r s e needs and  expectations  o f the c i t i z e n t o a g r e a t e r e x t e n t than have economists.  In  c o n t r a s t t o economists, b e h a v i o u r a l s c i e n t i s t s have seldom approached need from a p u b l i c p o l i c y , decision-making work.  frame-  Many s o c i a l s c i e n t i s t s have p r e f e r r e d to remain i n the  l a b o r a t o r y w i t h the " p u r i t y " o f i t s c o n t r o l l e d environment. T h i s i s p a r t l y e x p l a i n e d by the reward s t r u c t u r e o f the  social  sciences. Many academic s c i e n t i s t s v a l u e the p r e s t i g e t h a t t h e i r c o n t r i b u t i o n s to b a s i c r e s e a r c h and theory g i v e them i n the eyes of t h e i r peers more than whatever reward might be o b t a i n e d from c l i e n t s who would f i n d t h e i r work u s e f u l . (Williams 71, p. 62). In the f i e l d of housing,  s e v e r a l examples are a v a i l a b l e o f  attempts to e v a l u a t e programmes u t i l i z i n g  methodologies  d e v i s e d by s o c i o l o g i s t s , p s y c h o l o g i s t s and o t h e r scientists.  behavioural  48 Housing  as an E v a l u a t i v e Research  Focus  E v a l u a t i v e s t u d i e s of programmes designed t o improve the q u a l i t y of the r e s i d e n t i a l environment  have been under-  taken from two p e r s p e c t i v e s - t h a t o f the s o c i a l  scientist  whose concern i s t o develop t h e o r i e s e x p l a i n i n g man-environment i n t e r a c t i o n and,  secondly, of government agencies  i n v e s t i g a t i n g funding a l l o c a t i o n s . L i m i t a t i o n s o f Academic Housing  Research:  s t u d i e s undertaken  by academic s o c i a l  s c i e n t i s t s have l i m i t e d a p p l i c a b i l i t y i n the c o n t e x t o f u s e r 27 o r i e n t e d e v a l u a t i o n methodology. B e h a v i o u r a l s c i e n t i s t s f r e q u e n t l y f a i l t o focus on v a r i a b l e s amenable t o p o l i c y i n t e r v e n t i o n .  Alternatively,  the r e s e a r c h e r w i l l n e g l e c t t o t r a n s l a t e h i s f i n d i n g s i n t o the terminology of the p o l i c y p l a n n e r .  Examples o f such s t u d i e s  i n c l u d e the man-environment i n v e s t i g a t i o n s o f d w e l l i n g l a y o u t and l o c a t i o n undertaken Buttimer  (72); Gans  1  by Chapin  Housing  (71)  (61) a n a l y s i s o f s o c i a l mix;  choice studies of Michelson case-in-point.  (47), L a n s i n g  (72).  Michelson's  and  and housing  (72) work i s a  P o l i c y p l a n n e r s a t C e n t r a l Mortgage and  expressed disappointment  i n the r e s u l t s o f r e s e a r c h  designed to i d e n t i f y the s o c i a l b a s i s o f f a m i l y d e c i s i o n s on housing type and l o c a t i o n .  During i n f o r m a l d i s c u s s i o n s w i t h  v a r i o u s CMHC employees concern was  expressed t h a t M i c h e l s o n ' s  r e s e a r c h , w h i l e t h e o r e t i c a l l y i n t e r e s t i n g as p r e s e n t e d , l a c k e d  49  applicability  i n a p o l i c y context.  able, unassisted,  to bridge  The p o l i c y p l a n n e r was not  the t h e o r e t i c a l - p r a c t i c a l gap.  Where attempts have been made t o i n t e r p r e t s o c i a l v a r i a b l e s i n a p o l i c y context, appointing.  Schorr  t h e r e s u l t s have been d i s -  (64) t a c k l e d the task o f d e f i n i n g and  measuring the s o c i a l c o s t s o f slum l i v i n g .  His indicators  to compare 'slum' and 'non-slum' areas were r e a d i l y a c c e s s i b l e and  admittedly  s u p e r f i c i a l , i n d i c e s of disease,  fire loss,  crime and d e l i n q u e n c y . A second l i m i t a t i o n o f s o c i a l e v a l u a t i o n the focus on unique p r o j e c t s . Chapin's  studies i s  Examples o f such r e s e a r c h  include  ( 4 7 ) e f f e c t s o f p u b l i c housing on p r o j e c t r e s i d e n t s ;  Festinger's  (50) study o f community o r g a n i z a t i o n and morale i n a  housing p r o j e c t ; Deutsch and C o l l i n s ' r a c i a l housing p r o j e c t ; and W i l n e r ' s  (51) work on an i n t e r (62) i m p l i c a t i o n s o f p u b l i c  housing f o r h e a l t h and p s y c h o l o g i c a l adjustment. i n d i v i d u a l i n s i g h t s but f a i l s  Each  provides  to a s s e s s comparative advantages o f  a l t e r n a t e environments. One  o f t h e few attempts t o p r o v i d e  a comprehensive  methodology f o r measuring the performance of s o c i a l programmes was d e v i s e d operations  research.  formance should of s o c i e t y .  ( 7 0 ) u s i n g concepts  by A l b e r t s  A l b e r t s s t r e s s e s t h a t measures o f p e r -  r e f l e c t the goals  He a c c e p t s ,  c u l t y o f seeking  developed i n  o f t h e i n d i v i d u a l and  but does not cope w i t h ,  the d i f f i -  q u a n t i t a t i v e measures of s u b j e c t i v e v a r i a b l e s .  A l b e r t s determines s a t i s f a c t i o n l e v e l s based on the degree to which a person i s ' s a t u r a t e d '  with respect  i n d i v i d u a l or s o c i e t a l goal: that  i s when  to a p a r t i c u l a r he i s  50  i n d i f f e r e n t as to whether o r not to s e l e c t a course of a c t i o n . Unfortunately,  Alberts neglects  to t a c k l e the q u e s t i o n  to determine these i n d i v i d u a l and p o i n t o f the c i t i z e n - r a t h e r he opinion, his  s o c i e t a l goals  of  how  from the  i n f e r s them through  view-  expert  own!  Housing Economists: f a i l e d to provide  Behavioural  s c i e n t i s t s have  p o l i c y o r i e n t e d housing e v a l u a t i o n .  Housing  economists, i n c o n t r a s t , o f f e r s e v e r a l o p e r a t i o n a l methodolog i e s , but u n d e r s t a t e the s o c i a l c o s t s and  b e n e f i t s o f programme  actions. The  majority  o f economic-based housing r e s e a r c h  been undertaken i n the context programmes.  o f U.S.  r  has  Inner C i t y urban renewal  C o s t - b e n e f i t methodologies are used t o measure 28  the e f f i c i e n c y o f p u b l i c e x p e n d i t u r e s . studying  Rothenberg  (67)  in  a s p e c i f i c redevelopment p r o j e c t , m a i n t a i n s t h a t  consequences o f government a c t i o n s should  be t r a c e d back to  the f u l f i l l m e n t of the wants o f i n d i v i d u a l s .  The  limitation  o f h i s work i s t h a t h i s d e f i n i t i o n o f "wants" uses as i n d i c a t o r the i n c r e a s e i n land values  the  an  i n the p r o j e c t area;  a  29  c r i t e r i o n h a r d l y r e l e v a n t to r e n t i n g r e s i d e n t s . may  Rothenberg  have sought to f a s h i o n a technique to f a c i l i t a t e  e m p i r i c a l measurement of s o c i a l good. presents  at best a w e l l defined  d e t e r m i n i n g c o s t s and  Indianapolis  (67)  research  statement of the problems o f  b e n e f i t s , of a g g r e g a t i n g b e n e f i t s ,  of d e f i n i n g the r e l e v a n t Messner  In r e a l i t y h i s  the  and  population.  a p p l i e d Rothenberg's'framework to  redevelopment p r o j e c t .  an  His economic a n a l y s i s  51 was  c o n s i d e r e d u s e f u l by p l a n n e r s i n the  and  l o c a t i o n of the p r o j e c t .  basis  The  a n a l y s i s i s an  f o r the p r o v i s i o n of g u i d e l i n e s  uses o f p u b l i c r e s o u r c e s and  s e l e c t i o n of  concerning  design  inadequate alternate  f o r the d e t e r m i n a t i o n o f  social  costs. Ross's various  (67)  attempt t o compare f e d e r a l payments to  housing programmes a p p l i e d  cost-benefit  to determine income r e d i s t r i b u t i o n and  a d e s c r i p t i o n of  environmental impact of a l t e r n a t e programmes p r o v i s i o n of p h y s i c a l f a c i l i t i e s and hood s u b c u l t u r e s ) . s e t s o f data.  Ross makes no  methodologies  the  the  ( i n terms o f  impact on  neighbour-  attempt to equate the  His e x p e c t a t i o n s were t h a t the  the  two  decision-maker  would a t l e a s t have a v a i l a b l e a d e s c r i p t i v e i n d i c a t i o n o f environmental impact. This the  review o f housing e v a l u a t i o n  changing p e r c e p t i o n  programme adequacy.  of relevant  efforts indicates  i n d i c a t o r s o f housing  T r a d i t i o n a l l y , housing demand  derived  q u a n t i t a t i v e programme t a r g e t s  through "housing  c a l c u l a t e d on the b a s i s o f new  household formations or through  p u b l i c housing w a i t i n g  The  lists.  deficit";  number o f s t a r t s o r p e r -  sons housed became i n d i c a t o r s o f programme performance. The to p o s t u l a t e  l i m i t a t i o n s of such i n d i c e s r e s u l t e d i n attempts an e f f e c t i v e housing demand.  further disaggregation  required  a  of the market i n t o submarkets o r g a n i z e d  a c c o r d i n g to income groups and census d a t a .  This  l o c a l i t y on  Each submarket was  the b a s i s  of  seen to p r e s e n t unique  52  c h a r a c t e r i s t i c s w i t h r e s p e c t to user need, type of housing, c r e d i t a v a i l a b l e and disaggregation  amenity.  i s that  The  assumption u n d e r l y i n g  this  the  i n h a b i t a n t s of a census t r a c t , a c i t y or even a n a t i o n share so much common c u l t u r e and p h y s i c a l s t r u c t u r e t h a t t h e i r p e r c e p t i o n s and p r e f e r e n c e s w i l l n o r m a l l y be d i s t r i b u t e d i n t h i s context. (Peterson 67 p. 19). Changes i n v i t a l s t a t i s t i c s ,  crime, and  service rates  served  as i n d i c a t o r s o f programme performance. The  more r e c e n t a p p r e c i a t i o n o f l i f e  style dif-  ferences w i t h i n p r e v i o u s l y aggregated income groups i s l e a d i n g toward a f u r t h e r d i s a g g r e g a t i o n c a t o r s are r e q u i r e d .  of housing demand.  However, housing e v a l u a t o r s  New are  indifaced  w i t h the d i f f i c u l t r e a l i z a t i o n t h a t many v i t a l a s p e c t s o f user s a t i s f a c t i o n are c l e a r l y d i f f i c u l t to cope w i t h i n the context  o f e v o l v i n g s o c i a l a c t i o n programmes.  evaluative  By-and-large  s t u d i e s undertaken by s o c i a l s c i e n t i s t s have  to combine p o l i c y r e l e v a n t v a r i a b l e s w i t h i n d i c i e s of concern.  Three e x c e p t i o n s ,  to be c o n s i d e r e d  failed  social  i n more d e t a i l  l a t e r i n t h i s c h a p t e r as p o t e n t i a l models o f user o r i e n t e d evaluations, proposals  are Gruen's p r i o r p o l i c y e v a l u a t i o n o f r e l o c a t i o n  i n Dayton, Ohio; Feagin's a n a l y s i s of an experimen-  t a l r e n t subsidy  programme i n Boston, and  Onibokun's  g a t i o n of the r e l a t i v e h a b i t a b i l i t y of p u b l i c housing p r o j e c t s i n South West  Ontario.  investi-  53 The  E f f o r t s o f F e d e r a l Housing  Agencies:  A second source of housing found  evaluative studies i s  i n the e f f o r t s of f e d e r a l housing  agencies  agencies.  r e s p o n s i b l e f o r a d m i n i s t e r i n g housing  Government  programmes  have been slow t o respond t o • t h e need f o r e v a l u a t i o n .  In the  U n i t e d S t a t e s , the Department of Housing and Urban Development i s s t r u c t u r a l l y w e l l adapted f o r the development of e v a l u a t i v e a c t i v i t i e s and techniques ....In p r a c t i c e however, the p l a n n i n g and e v a l u a t i v e u n i t s f u n c t i o n as s m a l l r e s e a r c h s t a f f s m i n i s t e r i n g to the immediate needs.... Formal program e v a l u a t i o n has been a low p r i o r i t y item and has r a r e l y been done on a s y s t e m a t i c b a s i s b e f o r e the r e c e n t a v a i l a b i l i t y o f s u b s t a n t i a l funds f o r e v a l u a t i o n of the model c i t i e s program. (Wholey 70, pp. 64 and 77). There i s , however, evidence  o f a growing concern  f o r evalua-  t i o n s which seek to i d e n t i f y and d e v i s e means f o r u t i l i z i n g 30 the b e s t and most u s e f u l p o l i c y r e l a t e d r e s e a r c h s t u d i e s . In Canada, p r i o r to the H e l l y e r Task Force  (69),  f e d e r a l o f f i c i a l s showed l i t t l e concern w i t h the adequacy o f u n i t s produced or w i t h the q u a l i t y o f l i f e i n p r o j e c t s . No reviews o f tenant s a t i s f a c t i o n w i t h the program were c a r r i e d out nor were t h e r e any i n t e r n a l e v a l u a t i o n s o f the p l a n n i n g and design o f projects. (Dennis 72, p. 179). In g e n e r a l , CMHC compared a l t e r n a t e programmes on the b a s i s of  initial  c o s t , r a t e , and the q u a n t i t y of p r o d u c t i o n .  Dennis and F i s h  (72)  i d e n t i f y cases where f e d e r a l a u t h o r i t i e s  were w i l l i n g to pay h i g h c o s t s i n hope of q u i c k ,  visible  s t a t i s t i c a l success measured by the number o f u n i t s produced.  Yet a r a t i o n a l e v a l u a t i o n of housing a l t e r n a t i v e s  over time must r e c o g n i z e the ongoing maintenance and  broader  54  s o c i a l c o n s i d e r a t i o n s f a r outweigh c o n s i d e r a t i o n s o f i n i t i a l c o s t o r o f p r o d u c t i o n measured s o l e l y i n terms o f numbers o f units  (Goetze  72, p.  70).  Most e v a l u a t i o n sponsored taken through  by CMHC has been under-  e x t e r n a l l y commissioned s t u d i e s .  Where pro-  gramme r e c i p i e n t s d i d p r o v i d e an i n p u t , i t was o n l y t o d e s c r i b e 31 the c h a r a c t e r o f t h e c l i e n t e l e .  Two o f t h e more c o n t r o -  v e r s i a l o f these s t u d i e s were e x c e e d i n g l y c r i t i c a l policy analysis.  Dennis  of existing  (72) c h a r a c t e r i z e d p o l i c y a n a l y s i s  as haphazard, l a c k i n g any s y s t e m a t i c attempt t o review  social  i s s u e s u n d e r l y i n g housing problems o r t o determine t h e e f f e c t i v e n e s s o f programmes.  The Charney Report  (71) on t h e  "Production and Adequacy o f Low Income Housing i n Canada" found t h a t : the government a l l o c a t e s approximately $1.1 b i l l i o n a year t o housing on t h e b a s i s o f very l i m i t e d i n f o r m a t i o n on t h e o v e r a l l s t a t e o f the housing stock, on the u s e f u l n e s s o f t h e housing produced o r on the o v e r a l l s t r u c t u r e of the h o u s e b u i l d i n g i n d u s t r y . (Housing and People, A p r i l 1972, p. 3 . ) . In response,  CMHC e s t a b l i s h e d an e v a l u a t i o n d i v i s i o n  designed t o make the C o r p o r a t i o n more r e s p o n s i v e t o t h e l i v i n g needs o f i t s c l i e n t s by e v a l u a t i n g Canadian Housing P o l i c i e s and programs i n terms of changing consumer demand. (CMHC In-house Paper 70, p. 7 ) . Be t h a t as i t may, the C o r p o r a t i o n has y e t t o undertake an i n t e g r a t e d e v a l u a t i o n o f e x i s t i n g programmes.  Evolving  d i r e c t i o n s i n CMHC do, however, i n d i c a t e an i n t e r e s t i n  55  cross-Canada comparative a n a l y s i s o f s p e c i f i c The  programmes.  f i r s t example i s a j o i n t CMHC-Ministry o f S t a t e f o r Urban  Affairs  proposal to evaluate  the Neighbourhood Improvement  32 Programme.  I n i t i a l p l a n s a r e f o r the two agencies  out separate, b u t complementary, e v a l u a t i o n s . s i d e r programme e f f i c i e n c y programme g o a l s . relation  CMHC w i l l  con-  and e f f e c t i v e n e s s i n l i g h t o f  MSUA's task i s t o examine the programme i n  t o a broader s e t o f urban impacts,  specifically,  p u b l i c s e r v i c e s and f i n a n c e , l o c a l p o l i t i c a l c i t i z e n p a r t i c i p a t i o n , inner c i t y , s o c i a l change, housing  to carry  institutions,  and p s y c h o l o g i c a l  and l a n d markets, i n t r a c i t y m i g r a t i o n  t e r n s , intergovernment and interdepartment e v a l u a t i v e process  relations.  patThe  being u t i l i z e d i s (1) t o determine p r o -  gramme g o a l s through an a n a l y s i s o f e n a b l i n g  legislation;  (2) t o s p e c u l a t e on p o s s i b l e impacts through a s e r i e s o f "think-tanks";  and (3) t o commission e v a l u a t i o n s o f t h e  v a r i o u s programme dimensions.  While, a t the time o f w r i t i n g ,  e v a l u a t i o n g u i d e l i n e s had y e t t o be o p e r a t i o n a l i z e d , s e v e r a l proposals 33 faction.  indicate  a  mounting concern f o r user  satis-  56 A p p l i c a t i o n s o f E x i s t i n g E v a l u a t i o n Methodologies Three g e n e r a l e v a l u a t i v e models, d e v i s e d by Wholey, C a r t e r and Wharf, and  Caputo, and  four s p e c i f i c  examples o f  housing programme e v a l u a t i o n , are worth c o n s i d e r i n g i n more d e t a i l as examples of t r a d i t i o n a l  evaluative  processes.  T h e o r e t i c a l Models Wholey's E x p e r i m e n t a l Model: Joseph Wholey's the t r a d i t i o n a l  evaluation proposal  r e s e a r c h d e s i g n o f Campbell and  i n v o l v i n g experimental provide  (70)  and  c o n t r o l groups.  programmes.  defined centrally,  (6 3)  i n the con-  Rather, Wholey's recom-  mendations focus on the a d m i n i s t r a t i v e task of e v a l u a t i o n i n a government agency.  Stanley  Wholey does not  o p e r a t i o n a l examples o f these techniques  t e x t of m u l t i - d i m e n s i o n a l  expresses  implementing  He proposes t h a t p r i o r i t i e s  t h a t e n a b l i n g l e g i s l a t i o n i n c l u d e accom-  panying e v a l u a t i o n p r o p o s a l s ,  and  the i n s t i t u t i o n o f  full  time e v a l u a t i o n departments. C a r t e r and Wharf*s Typology: A r e c e n t Canadian e v a l u a t i o n model, d e v i s e d  by  34 C a r t e r and Wharf  (73)  presents  a framework designed f o r  use  e i t h e r by a small agency o r a l a r g e government department, and  adaptable to e v a l u a t i o n of ongoing or new The  Carter-Wharf procedure i s based on the  f i c a t i o n of a typology programmes.  projects.  From the  for classifying "correct" i n i t i a l  social  identi-  development  c l a s s i f i c a t i o n of  be  57 programmes i n t o " i n q u i r y " , " e x p l o r a t o r y " or  "demonstration"  35 projects  they see the remaining e v a l u a t i v e procedure  flowing  i n a l o g i c a l sequence from: Type o f p r o j e c t — P type of r e s e a r c h d e s i g n P i d e a l evaluation ---^feasible evaluation —•» a c t u a l e v a l u a t i o n -> use of e v a l u a t i v e findings ,process —•> ongoing adjustments* outcome —> implementation > d e c i s i o n making. Included  i n the process  monitoring  and  citizen  i s the recommendation f o r ongoing involvement.  although c l i e n t and/or community p a r t i c i p a t i o n i n e v a l u a t i v e r e s e a r c h w i l l undoubtedly r e s u l t i n a d d i t i o n a l c o s t burdens, t h e i r i n p u t should be c o n s i d e r e d s e r i o u s l y . O b s e r v a t i o n s o f c l i e n t s and r e s i d e n t s about program consequences are o f t e n sound. They can h e l p programs g a i n acceptance i n communities, w h i l e p r o v i d i n g u s e f u l i n f o r m a t i o n about the program i t s e l f . (Carter and Wharf 73, p. 122). In l i m i t e d t e s t i n g C a r t e r and Wharf found t h e i r c l a s s i f i c a t i o n scheme a s s i s t e d i n c l a r i f y i n g p r o j e c t o b j e c t i v e s and  determining  the a p p r o p r i a t e  design  for evaluation.  Their  model does not, however, r e f l e c t the complex concerns of o p e r a t i o n a l i z i n g the t o t a l e v a l u a t i v e p r o c e s s .  The  authors  36 have y e t t o come to g r i p s  with  the p r a c t i c a l problems o f  d e f i n i n g programme o b j e c t i v e s , h a n d l i n g  evolving multi-  37 dimensional  concerns,  d i f f e r e n t i a t i n g between programme  system o r i g i n a t i n g problems or i n v o l v i n g users In s h o r t , they s t r u c t u r e d the  i n the  process.  obvious.  Caputo's I n t e g r a t i o n Model: Caputo's  (73) model  s p e c i f y i n g the need f o r , and  (Diagram 3) comes c l o s e s t to r o l e o f , c i t i z e n s i n the  and  58  DIAGRAM C A P U T O ' S MODEL FOR PUBLIC POLICY  (4)  EVALUATION  Program  Group  at  Evaluation  T ,T5.,T2''''T,  •n  C  (J) (1)  (4)  = T,  n  (Ul)  T (III) (3)  (3) —M  (Ul)  (.IV)  Academic & P r o f e s s ional Evaluation at X T - -jjn o-»2  _J2_  my  (2)  Public policy being evaluated  conditions existing  I  OH  prior  to  the  programme  = e v a l u a t i o n o f c o n d i t i o n s and programme e f f e c t i v e n e s s d u r i n g a c t u a l programme i m p l e m e n t a t i o n . process i e s may  T  1  -s  Interaction  <3)  Citizen  (IV)  Evaluator B  (3) (4)  URBAN  e v a l u a t i o n ; e v a l u a t i o n d a t e s and f r e q u e n c v a r y f r o m one programme t o t h e n e x t .  terminal evaluation.. ramme r e a c h e s e n d o f terminated. Solid line involvement  T h i s w i l l be d o n e when p r o g experimental phase or i s  arrows i n d i c a t e d i r e c t among p a r t i c i p a n t s .  interaction  - Broken l i n e arrows i n d i c a t e i n d i r e c t a n d i n v o l v e m e n t among p a r t i c i p a n t s . From  Caputo  and  interaction 73  p.  115.  59  evaluative process.  H i s model evolved  Gary, Indiana model c i t i e s programme. presented  - the i n c l u s i o n o f v a r i o u s  i n the e v a l u a t i o n p r o c e s s ; and  from e v a l u a t i o n s o f the Three b a s i c concepts a r e  s e c t o r s o f the community  i n t e r a c t i o n between these s e c t o r s ;  the p r o v i s i o n f o r dynamic  monitoring.  Caputo sees t h r e e groups i n v o l v e d i n g e n e r a t i n g programme e v a l u a t i o n . c a t a l y s t and c r i t i c , and  The programme e v a l u a t o r i s responsible  (A), as a  f o r the o v e r a l l  organization  implementation o f the e v a l u a t i o n and f o r conveying f i n d i n g s  to programme a d m i n i s t r a t o r s . (D) and c i t i z e n  He c o - o r d i n a t e s  the p r o f e s s i o n a l  (C) e v a l u a t i o n components.  Academic and p r o f e s s i o n a l e v a l u a t o r s ,  u s i n g the  combined e x p e r t i s e o f m u l t i - d i s c i p l i n a r y teams, p r o v i d e one i d e n t i f i c a t i o n o f programme o b j e c t i v e s and recommendations. A second e v a l u a t i o n i s conducted by c i t i z e n s who, group meetings, p e r s o n a l  contacts,  surveys,  through  and o u t s i d e  t e c h n i c a l e x p e r t i s e , are asked t o i d e n t i f y t h e i r o b j e c t i v e s , develop e v a l u a t i o n c r i t e r i a and make recommendations. A unique f e a t u r e o f the model i s the important r o l e p l a y e d by c i t i z e n e v a l u a t i o n ; i t i s a coequal r o l e t o t h a t p l a y e d by academic and p r o fessional evaluators. (Caputo 73, p. 116). 38 Caputo's " i n t e r a c t i o n p r o c e s s "  i n t e g r a t e s the  recommendations o f c i t i z e n and p r o f e s s i o n a l e v a l u a t i o n teams. Group i n t e r a c t i o n a c t s as a f i l t e r i n g d e v i c e , i t r e f l e c t s the need f o r a l t e r n a t e e v a l u a t i o n s t o be thoroughly d i s c u s s e d and analyzed b e f o r e b e i n g c o n s i d e r e d by program e v a l u a t o r s . (Caputo 73, p. 116). Where p o s s i b l e , a concensus on recommendations f o r submission  60 t o the programme e v a l u a t o r forms the output o f the group interaction. However, c i t i z e n e v a l u a t o r s , l i k e e v a l u a t o r s , can choose t o emphasize  professional  any o f s e v e r a l  measures,  r e s u l t i n g i n the s u b j e c t i v e use o f o b j e c t i v e d a t a .  So l o n g  as c r i t e r i a chosen r e s u l t i n s i m i l a r recommendations, sus i s p o s s i b l e .  When c r i t e r i a y i e l d c o n f l i c t i n g  concen-  recommenda-  t i o n s Caputo sees the programme e v a l u a t o r a c t i n g as s e l e c t o r . In essence, the programme e v a l u a t o r makes a s u b j e c t i v e d e c i s i o n on recommendations ministrators.  t o be advanced t o programme ad-  E x p e r i e n c e elsewhere suggests antagonism  and c r i e s o f "tokenism" a r i s e from c i t i z e n s and e x p e r t s a l i k e who h o l d r e j e c t e d , but, i n t h e i r minds, e q u a l l y i  proposals.  valid  39 The model's dynamic q u a l i t y i s a c h i e v e d through  c o n t i n u a l m o n i t o r i n g and feedback t o p e r m i t mid-course  cor-  r e c t i o n s and adjustments. The time component i s u s u a l l y n e g l e c t e d i n most attempts a t program e v a l u a t i o n ; the r e s u l t i s a s t a t i c a n a l y s i s which l a c k s a l o n g i t u d i n a l dimension and h i n d e r s comparab i l i t y w i t h o t h e r programs. (Caputo 73, p. 116) . From the p e r s p e c t i v e o f a user o r i e n t e d  evaluation  of a m u l t i - d i m e n s i o n a l programme, Caputo's p r o p o s a l appears to have both l i m i t a t i o n s and advantages.  H i s model f a i l s t o  p r o v i d e evidence o f an a b i l i t y t o cope w i t h e v a l u a t i n g m u l t i dimensional programmes.  The h y p o t h e t i c a l urban problems  d i s c u s s e d i n t h e paper, t r a s h removal and drug treatment, p r e s e n t n e i t h e r the complexity nor w e i g h t i n g problems  evident  61 i n an e v a l u a t i o n of the r e s i d e n t i a l environment. h i s model l a c k s the c a p a c i t y to handle d i v e r g e n t t i o n s or f o r determining  Similarily, recommenda-  whether s o l u t i o n s should be  through programme or broader system  attempted  reformulation.  Even though the i n c l u s i o n of c i t i z e n e v a l u a t i o n i s 40 designed to p r o v i d e Caputo's c r i t e r i a  the advantage o f a l t e r n a t e p e r s p e c t i v e s ,  f o r enhancing programme a p p l i c a b i l i t y  e f f e c t i v e n e s s remain economic c o s t s and b e n e f i t s . presses  He  and  ex-  concern t h a t o b j e c t i v e i n f o r m a t i o n o b t a i n e d from the model may be used s u b j e c t i v e l y . . . . c o m p l i c a t e d e v a l u a t i o n procedures may not i n s u r e o b j e c t i v e decisions. The q u e s t i o n then becomes whether the c o s t i n v o l v e d i n e s t a b l i s h i n g and monitori n g such a program i s worth the b e n e f i t s . (Caputo 73, p. 119) . Caputo's model p r o v i d e s  s e v e r a l f e a t u r e s worth  i n c l u d i n g i n a methodology f o r user o r i e n t e d His model grapples  with  evaluations.  the n e c e s s i t y to handle e v o l v i n g  needs through feedback and mid-course c o r r e c t i o n s . essence he p r o v i d e s  f o r the f o r m a t i v e  In  e v a l u a t i o n recommended  41 by S c r i v e n .  The  r e l a t i o n s h i p between v a r i o u s p a r t i c i p a n t s  i n the e v a l u a t i v e process  is illustrated.  The  i n c l u s i o n of  c i t i z e n i n p u t i n the d e f i n i t i o n o f programme o b j e c t i v e s , e v a l u a t i o n and  recommendations agrees w i t h  the broader  defi-  n i t i o n s o f c i t i z e n p a r t i c i p a t i o n i n which p a r t i c i p a t i o n i s seen as a p r o c e s s . duct and  ( S v i r i d o f f 70).  I t cannot be The  allowed  to become the  pro-  i n t e r a c t i o n between p r o f e s s i o n a l  c i t i z e n r e f l e c t s an attempt t o o p e r a t i o n a l i z e the  p o s a l s made by Friedmann's theory  of T r a n s a c t i v e  pro-  Planning.  62 I f the communication gap between planner and c l i e n t i s closed, a continuing s e r i e s o f pers o n a l and p r i m a r i l y v e r b a l t r a n s a c t i o n s between them i s needed, through which processed knowledge i s fused with p e r s o n a l knowledge and both a r e fused with a c t i o n . (Friedmann 73, p. 177). Thus the three models d i s c u s s e d are, i n e f f e c t , complementary.  C a r t e r and Wharf i d e n t i f y i n i t i a l  d i r e c t i o n s as a base f o r determining Caputo p r o v i d e s  a research  evaluative  design.  a model o f the i n t e r a c t i v e e v a l u a t i o n  process.  Wholey i n d i c a t e s the i n g r e d i e n t s o f e v a l u a t i o n i n an i n s t i t u tional setting.  What the t h r e e models do n o t attempt i s a  d e t a i l e d o p e r a t i o n a l c o n s i d e r a t i o n o f t h e i n d i c a t o r s and a n a l y s i s necessary as a base f o r r e a c h i n g mendations.  Four examples o f attempts t o o p e r a t i o n a l i z e  programme e v a l u a t i o n , i n t h e context sented  programme recom-  o f housing, are  pre-  i n the f o l l o w i n g s e c t i o n .  O p e r a t i o n a l Models o f Programme E v a l u a t i o n :  Four Examples  Attempts t o o p e r a t i o n a l i z e e v a l u a t i o n s o f housing programmes a r e seen i n the work o f CMHC E v a l u a t i o n D i v i s i o n (71), Onibokun  (71), Gruen and Gruen  (72)  and F e a g i n  (73).  CMHC's $200 M i l l i o n Programme: A 1971 a n a l y s i s undertaken by CMHC o f t h e i r 200 M i l l i o n D o l l a r Innovative  low-cost housing programme p r o -  v i d e s an i n d i c a t i o n o f government housing e v a l u a t i o n p r a c t i c e . The  e v a l u a t i o n o b j e c t i v e s were t o assess  s a t i s f i e d the o r i g i n a l i n t e n t o f being  how n e a r l y p r o j e c t s  i n n o v a t i v e and  63  low c o s t and, secondly, t o determine get low-income groups a c t u a l l y  the e x t e n t t o which  responded  tar-  t o t h i s form o f  intervention. The e v a l u a t i o n was d e v i s e d by CMHC Head O f f i c e , Ottawa.  I t involved the d i s t r i b u t i o n o f questionnaires t o  each CMHC Branch O f f i c e  r e q u e s t i n g w r i t t e n comments on:  (1) d i f f i c u l t i e s encountered  i ninitiating  low c o s t housing;  (2) a t a b u l a t i o n o f i n n o v a t i v e approaches; public reaction;  (4) q u a l i t y  v i a b i l i t y of units; of management.  (3) comments on  o f c o n s t r u c t i o n ; (5) market  and (6) t h e r o l e and r e s p o n s i b i l i t i e s  Q u e s t i o n n a i r e s were completed  sonnel from e x i s t i n g d i r e c t l y determine  agency r e c o r d s .  No attempt  was made t o  user s a t i s f a c t i o n , p r o v i d e f o r on-going  m o n i t o r i n g , o r t o separate programme impacts urban p r o c e s s e s .  by agency p e r -  from  impinging  In l i e u o f d i r e c t e v a l u a t i o n o f user  r e a c t i o n s CMHC o f f i c e s t a f f s a s s e s s e d user s a t i s f a c t i o n .  On  t h i s b a s i s the r e p o r t concluded: Those who have been accepted as purchasers o r tenants appeared t o t h e f i e l d s t a f f as b e i n g happy w i t h t h e Program. (CMHC 71, p. 46). The a n a l y s i s appears tions.  Agency s t a f f determine  t o be t y p i c a l o f agency e v a l u a goals, design the e v a l u a t i v e  instrument, and answer t h e q u e s t i o n s . stances those who a r e asked  Under such  circum-  t o do the e v a l u a t i o n are, i n  e f f e c t , e v a l u a t i n g t h e i r own e f f o r t s . that t h e i r evaluations are u s u a l l y  I t i s not s u r p r i s i n g  positive.  64  The u l t i m a t e u t i l i t y o f "in-house" e v a l u a t i o n s i s l i t t l e more than a "pat on the back" t o the s p o n s o r i n g agency. P a t t e r n s a r e a l r e a d y known.  Few a d d i t i o n a l i n s i g h t s a r e pro-  v i d e d t o a s s i s t programme r e f o r m u l a t i o n .  At b e s t ,  "in-house"  e v a l u a t i o n s o f f e r agency p e r s o n n e l a s s i g n e d t o r e g i o n a l o f f i c e s an o p p o r t u n i t y t o have an i n p u t i n t o head o f f i c e p o l i c y making. Onibokun's R e l a t i v e Onibokun  Habitability: (71) p r e s e n t s an example o f a t y p i c a l  housing e v a l u a t i o n as undertaken  by a n o n - i n s t i t u t i o n a l p l a n 42  ner.  Hxs study o f e i g h t e e n O n t a r i o housing  attempted  t o determine  projects  the r e l a t i v e h a b i t a b i l i t y o f p u b l i c  housing and the f a c t o r s i n f l u e n c i n g t h a t  habitability.  The study was based on data c o l l e c t e d by d i r e c t interview techniques.  A t h i r t y - p a g e q u e s t i o n n a i r e was  a d m i n i s t e r e d t o 199 female heads o f households housing p r o j e c t s i n t h e t h r e e Canadian Guelph, minutes.  and G a i t .  Interviews  The 190 respondents  f i e d random sampling, The respondents holds.  i n public  c i t i e s of Kitchener,  took an average  of ninety  were s e l e c t e d , through  from f i f t e e n p u b l i c housing  strati-  projects.  i n t h i s study were a l l female heads o f house-  The r a t i o n a l e was t h a t women a r e the homemakers; they  s t a y a t home and i n t e r a c t w i t h the components o f the h a b i t a b i lity  subsystems more than t h e i r male c o u n t e r p a r t s and they a r e  more r e a d i l y a v a i l a b l e f o r i n t e r v i e w s ; moreover, women a r e g e n e r a l l y r e s p o n s i b l e f o r housekeeping  and c h i l d - r e a r i n g ;  they  are l i k e l y t o have more experience w i t h the p h y s i c a l  and  s o c i a l f a c t o r s o f housing. The q u e s t i o n n a i r e was  designed to a s s i s t  the  r e s e a r c h e r t o c o l l e c t data on v a r i o u s components o f the r e s i d e n t i a l environment.  Relevant v a r i a b l e s emerged  from  l i t e r a t u r e survey, and d i s c u s s i o n s w i t h c a r e t a k e r s , housing a u t h o r i t y p e r s o n n e l , and members o f tenant Each o f the 199  respondents  point L i k e r t Scale  was  associations.  asked t o i d e n t i f y , on a  ( L i k e r t 32), her degree of  w i t h each o f the s e v e n t y - f o u r s e l e c t e d  satisfaction  attributes.  The r e l a t i v e h a b i t a b i l i t y o f each p r o j e c t c a l c u l a t e d by combining  five-  was  s a t i s f a c t i o n l e v e l s experienced f o r  each o f the s e v e n t y - f o u r s e l e c t e d a t t r i b u t e s .  Analysis  i n v o l v e d comparisons between housing p r o j e c t s and w i t h i n f o u r r e s i d e n t i a l sub-systems:  (1) the d w e l l i n g ; (2) the  environment o f which the d w e l l i n g i s a component;  (3) the  management or i n s t i t u t i o n a l arrangements under which the housing u n i t and environment are a d m i n i s t e r e d ;  (4) the  tenant, Onibokun's work i s t y p i c a l o f many h a b i t a b i l i t y studies.  He i n c l u d e s post-move i n t e r v i e w s , comparison of p a s t  and p r e s e n t environments, groups,  and an emphasis on l o c a t i o n and p h y s i c a l  of the user's environment. adequately account concept.  comparison o f treatment  and  control  composition  He a p p r e c i a t e s , but does not  f o r , change.  Habitability is a relative  I t i s q u e s t i o n a b l e whether an a n a l y s i s a t one p o i n t  i n time copes w i t h the c o m p l e x i t i e s of e v o l v i n g needs.  More  66 realistically, monitoring  evaluative studies require a continual  s t r u c t u r e to accompany them. Only vague g e n e r a l i t i e s are d i r e c t e d toward  question  of reformulating  findings.  p o l i c y based upon  His s t r u c t u r e d i n t e r v i e w q u e s t i o n s  d e t e c t i n g problems.  They do not p r o v i d e  the  research assist in  evidence o f  alter-  nate d i r e c t i o n s . Onibokun's r e s e a r c h  appears to r e s t on the  assumption  t h a t the p h y s i c a l a t t r i b u t e s of an environment are c e n t r a l t o user s a t i s f a c t i o n .  T h i s assumption i s q u e s t i o n a b l e .  p a r a t i v e a n a l y s i s o f c u r r e n t l i t e r a t u r e suggests, r e s i d e n t ' s p e r s p e c t i v e , q u a l i t y and  from the  s o c i a l relevance  r e s i d e n t i a l s e r v i c e s are more fundamental t o housing f a c t i o n than q u a n t i t y and 72 and  Turner 72).  s p a c i n g o f these s e r v i c e s  I f planners  continue  A com-  to r e l y on  of satis(Buttimer the  p h y s i c a l a t t r i b u t e s o f the environment as an i n d i c a t o r o f r e s i d e n t i a l environment s u i t a b i l i t y they w i l l be s t r e s s i n g features less Gruen and  than c r i t i c a l to consumer s a t i s f a c t i o n .  Gruen s P r i o r P o l i c y A n a l y s i s : 1  P r i o r p o l i c y a n a l y s i s u t i l i z i n g survey methods forms the base of Gruen and of a p r o p o s a l  to g e o g r a p h i c a l l y  Gruen's 19 70  research evaluation  expand the range of housing  o p p o r t u n i t i e s f o r the l e s s a f f l u e n t i n Dayton, Ohio. study was  commissioned by the Miami V a l l e y Regional  A u t h o r i t y who reluctance  The Planning  f e a r e d r e s i s t a n c e from suburban f a m i l i e s and  from lower-income households t o p a r t i c i p a t e i n  67  integrated  housing programmes.  The  (1) survey community a t t i t u d e s ; impacts t h a t low  Gruens were h i r e d  to:  (2) p r e d i c t the k i n d s o f  and moderate income placement c o u l d have  neighbourhood and  community f e a t u r e s  and  (3) recommend c r i t e r i a t o be  was  to b e n e f i t both the people who  communities i n which i t was  v a l u e d by  followed  suburbanites;  i f the new  housing  would l i v e i n i t and  located.  on  Subsequently,  the  the  Gruens used these c r i t e r i a to e v a l u a t e e x i s t i n g f e d e r a l housing mix  programmes.  The  study i n c l u d e d  l o c a l planning process.  a survey o f p a r t i c i p a n t s i n  A sample o f low  and  moderate income  f a m i l i e s , s u b u r b a n i t e s , l o c a l p u b l i c o f f i c i a l s , and men  (from v a r i o u s  f i n a n c e , b u i l d and interviews  branches o f the  industry  "inhibitors"  elsewhere i n the U n i t e d S t a t e s  business-  locate,  s e l l housing) were i n t e r v i e w e d .  i n the p l a c i n g of  income housing i n suburban neighbourhoods.  These as  lower-  Housing e x p e r t s  were asked to comment on  l o c a l examples o f mixed economic, s o c i a l , ral  that  were used to a s c e r t a i n what f a c t o r s a c t e d  " f a c i l i t a t o r s " and  the  r a c i a l or  structu-  communities. P a r t i c i p a n t interviews  composition data and  the  use  obtained basic  of e x i s t i n g services.  s t u d i e s were attempted through: scale;  (1) r a t i n g , on  (2) p i c t u r e s o f p o t e n t i a l housing u n i t s ;  c h o i c e between v a r i o u s  household  l o c a t i o n p r e f e r e n c e s ; and  of degree of acceptance of v a r i o u s augment neighbourhood s e r v i c e s .  Preference  a five (3)  forced  (4) r a t i n g s  policy alternatives  Analysis  point  of c h o i c e s  to  yielded  68 p o l i c y a l t e r n a t i v e s designed t o a c h i e v e maximum acceptance  by  both e x i s t i n g and expected r e s i d e n t s . The Gruens  1  a p p l i c a t i o n of s o c i a l science research  methods to the s o l u t i o n o f p o l i c y i s s u e s p r o v i d e s an example of an attempt  t o i n p u t user v a l u e s i n t o p o l i c y f o r m u l a t i o n .  They do not, however, attempt consultative process.  t o i n v o l v e the user i n the e n t i r e  T h e i r a n a l y s i s o f one  aspect of a m u l t i -  d i m e n s i o n a l i s s u e i n d i c a t e s the complexity o f d i s c u s s i n g v a l u e laden concepts.  As a "one-shot"  than e x i s t i n g , p o l i c y no attempt  study o f proposed, was  rather  made to determine  programme p r o p o s a l s would a c t u a l l y a l l e v i a t e  whether  identified  problems. The Gruens' study p r o v i d e s an e x c e l l e n t example o f p r i o r p o l i c y research.  T h e i r a p p l i c a t i o n of a combination  of  r e s e a r c h methods, the use of f o r c e d c h o i c e among v a r i o u s v i a b l e p o l i c y a l t e r n a t i v e s , and the c o n s u l t a t i o n w i t h  potentially  opposed s e c t o r s o f the community, o f f e r concepts a p p l i c a b l e to a u s e r - o r i e n t e d e v a l u a t i o n o f s o c i a l a c t i o n programmes. Feagin's Programme R e f o r m u l a t i o n : A f i n a l example o f the use o f survey r e s e a r c h as a b a s i s f o r programme a n a l y s i s e x i s t s i n Feagin's study o f an experimental r e n t subsidy programme i n the Boston  area.  His  e v a l u a t i o n concluded t h a t , from a v a r i e t y of p e r s p e c t i v e s r e s i d e n t s , a d m i n i s t r a t o r s and e v a l u a t o r s - the programme d i d i n f a c t p r o v i d e more a c c e p t a b l e housing a t reasonable c o s t s . Feagin's  i n t e r v i e w schedule covered a broad  range  69 of housing household  concerns activity  including  the housing  search  p a t t e r n s , neighbourhood  process,  satisfaction,  43 household  value  systems,  and  o p i n i o n s on  programme  and  o p i n i o n d a t a were  objectives. Value, through  attitude,  four to six-point  obtained  rating  statements. A time-activity activity patterns.  s c a l e s on e v a l u a t o r d e s i g n e d 44 budget summarized h o u s e h o l d  Interviews of a s s i s t e d  household,  conducted  before  45 and  a f t e r moving  similar  were m a t c h e d w i t h  circumstances;  but  not  families  receiving  in  otherwise  government  assist-  ance . In p r i n c i p l e t h e p l a n f o r m u l t i p l e i n t e r v i e w s w i t h matched samples i s v e r y a t t r a c t i v e . The e a s i e r and more common h o u s i n g r e s e a r c h p r o cedure o f i n t e r v i e w i n g r e p r e s e n t a t i v e samples o f r e s i d e n t s i n each of d i f f e r e n t k i n d s o f h o u s i n g a f t e r t h e y have moved i n l e a v e s g r e a t d o u b t as t o how much o f t h e d i f f e r e n c e b e t w e e n one h o u s i n g g r o u p and a n o t h e r i s due t o t h e s e l e c t i o n o f t y p e s o f p e o p l e who were a l r e a d y d i s t i n c t b e f o r e t h e y moved i n , and how much o f what p e o p l e s a y i n r e t r o s p e c t a b o u t t h e o l d neighbourhood r e a l l y r e f l e c t s t h e i r experience i n t h e new one. The m u l t i p l e i n t e r v i e w - m a t c h e d sample d e s i g n r e d u c e s t h e s e d o u b t s . ( F e a g i n 73, p . 73) . This  research design rests  on  s e v e r a l assumptions.  It is  assumed t h a t somewhere i n e a c h p o p u l a t i o n t h e r e w i l l least  one  family similar  matched.  The  describe  these  at  f a m i l y i n the group t o  be  i n v e s t i g a t o r must a l s o be  able to i d e n t i f y  f a m i l i e s w e l l i n advance o f the  Despite potential  to each  be  the d i v e r s i t y  f o r comparisons,  of data  analysis  move.  collected  appears  and  the  t o have b e e n  and  70  attempted on only s e l e c t e d items. was  One  the wide scope of the e v a l u a t i o n .  provide  source of problems Feagin  data on both the s o c i a l w e l f a r e  attempts  of the household  i n f o r m a t i o n on c o s t e f f e c t i v e n e s s f o r programme  does not attempt to d e f i n e e v a l u a t i o n goals  user's  perspective.  the e v a l u a t i v e  and  administrators.  Feagin  Nor  to  from  the  does he a c t u a l l y i n v o l v e the user i n  process.  S e v e r a l concepts emerge from Feagin's work which warrant c o n s i d e r a t i o n i n a u s e r - o r i e n t e d e v a l u a t i o n . questionnaire  considers  a wide range of household  a t t i t u d e and o p i n i o n data.  His  value,  Matching programme r e c i p i e n t s  w i t h a d j a c e n t market neighbours emerges as a v i a b l e b a s i s f o r e s t a b l i s h i n g a c o n t r o l group t o determine programme impact.  A n a l y s i s o f user needs and considerably  i n the f o u r e v a l u a t i o n s .  expectations The  differs  CMHC a n a l y s i s , as  an example o f i n s t i t u t i o n a l e v a l u a t i o n s , r e l i e s e n t i r e l y upon p r o f e s s i o n a l judgment o f user s a t i s f a c t i o n .  Programme  r e c i p i e n t s were, i n f a c t , unaware t h a t an e v a l u a t i o n underway.  Onibokun's study i s reasonably  was  t y p i c a l of a v a s t  number o f e v a l u a t i o n s undertaken by planners  who,  when they  c o n s u l t the user, do so on a s p e c i f i c number of p h y s i c a l environment i n d i c a t o r s .  Both Gruen and F e a g i n  extend  user  71 o p i n i o n t o v a r i o u s s o c i a l aspects o f the environment. Feagin's method, i n a d d i t i o n , attempts  t o assess  household  v a l u e s and to d i f f e r e n t i a t e between programme and system impacts.  None o f the e v a l u a t i v e designs  attempted  to a c t i v e l y  i n v o l v e the user i n a l l stages o f the p l a n n i n g p r o c e s s . The T r a d i t i o n a l E v a l u a t i o n Model The  Process: Based upon t h e p r e c e e d i n g  l i t e r a t u r e survey, a f i v e  stage model of the e x i s t i n g s o c i a l a c t i o n programme e v a l u a t i o n process emerges  (Diagram 4 ) :  Stage One: t i o n r e q u i r e s the i n i t i a l  D e f i n i t i o n o f Programme G o a l s :  Evalua-  i d e n t i f i c a t i o n o f programme g o a l s  and the t r a n s l a t i o n o f these g o a l s i n t o measurable i n d i c a t o r s of g o a l achievement a g a i n s t which changes r e s u l t i n g from gramme exposure can be measured.  pro-  Goals u t i l i z e d a r e those  s p e c i f i e d by programme a d m i n i s t r a t o r s and r e f l e c t g o a l s  identif-  i e d a t the time the programme was designed. Stage Two:  The E v a l u a t i o n Design:  e v a l u a t i o n designs appear i n the l i t e r a t u r e —  Two types o f one, a model  e v o l v e d from academic and t h e o r e t i c a l e v a l u a t i o n and,  a second,  proposals  r e f l e c t i n g e v a l u a t i o n g u i d e l i n e s u t i l i z e d by  o p e r a t i o n a l government departments. The academic model r e l i e s upon e x p e r i m e n t a l i n v o l v i n g c o n t r o l and treatment  groups t o d i s t i n g u i s h  prog-  ramme e f f e c t s from those of o t h e r f o r c e s working i n the environment.  Such r e s e a r c h designs can be o f v a r i o u s  designs  72 DIAGRAM  4  A MODEL OF THE E X I S T I N G SOCIAL ACTION EVALUATION  PROGRAM  PROCESS  DETERMINE PROGRAM GOALS  DESIGN EVALUATION .METHOD  Comparison treatment & control groups  Comparison program findings with success c r i t e r a  COLLECT DATA  DATA ANALYSIS  Comparison . findings with program goals  RECOMMENDATIONS to a d m i n i s t r a t o r s f o r reformulation  strengths.  46  The s t r o n g e s t r e q u i r e s random a l l o c a t i o n o f par-  t i c i p a n t s t o c o n t r o l and e x p e r i m e n t a l groups b e f o r e and a f t e r analysis.  In combination  these f e a t u r e s r u l e o u t "extraneous  e x p l a n a t i o n s " o f the e f f e c t s o f exposure t o the programme. Examples o f t y p i c a l  " s t r o n g " designs a r e the C l a s s i c  p r e - t e s t , p o s t - t e s t , c o n t r o l group s i t u a t i o n ?1 X ~2 and t h e 3 4 0 0 Solomon Four-Group Design l X -2 n  3  x  o  k 4  6. These designs c o n t r o l and measure t h e e x p e r i m e n t a l and,in t h e Solomon Four-Group d e s i g n , t e s t f o r  effect  p o s s i b l e i n t e r a c t i o n e f f e c t s o f the measuring process  itself.  Such designs i n c l u d e a d e f i n i t i o n o f the t a r g e t p o p u l a t i o n , drawing o f a r e p r e s e n t a t i v e sample f o r study, a l l o c a t i o n o f respondents  i n t o e x p e r i m e n t a l and c o n t r o l groups,  t i o n o f the programme t o a treatment after effects.  group and comparison o f  Programme e f f e c t i s determined  p a r i s o n o f the two groups.  administra-  through com-  I f t h e d i f f e r e n c e between ^1 and ^2  i s g r e a t e r than the d i f f e r e n c e between ^3 and ^4 t h e programme i s deemed "a s u c c e s s " . Weaker, b u t s t i l l and S t a n l e y  a c c e p t a b l e a c c o r d i n g t o Campbell  (6 3), are q u a s i - e x p e r i m e n t a l d e s i g n s .  These a r e  s o c i a l s e t t i n g s i n which t h e r e s e a r c h e r can i n t r o d u c e somet h i n g l i k e e x p e r i m e n t a l d e s i g n i n t o h i s data c o l l e c t i o n cedures  pro-  (e.g., the when and t o whom o f measurement) even though  he l a c k s the f u l l c o n t r o l over the s c h e d u l i n g o f e x p e r i m e n t a l stimuli  (over the when and t o whom o f exposure).  of a q u a s i - e x p e r i m e n t a l d e s i g n i s the time s e r i e s  One example experiment.  74 The essence o f t h i s d e s i g n i s the presence o f a p e r i o d i c measurement  process on some group or i n d i v i d u a l and the i n t r o -  d u c t i o n o f e x p e r i m e n t a l change i n t o the measurement  series.  The r e s u l t s are i n d i c a t e d by a d i s c o n t i n u i t y i n the measurement: °1 °2 °3 °4 * 0 While  °6 °7  5  V  such designs do not s a t i s f y the s t r i c t requirements o f  the e x p e r i m e n t a l d e s i g n , they do, i n a v a r i e t y o f ways, to p r o t e c t a g a i n s t the e f f e c t s o f extraneous  attempt  v a r i a b l e s on out-  come measures. Realistically,  i t must  b  e  p o i n t e d out t h a t a l l  e x p e r i m e n t a l designs are s u b j e c t , i n v a r y i n g degrees, t o sources o f e r r o r . include:  Factors jeopardizing internal  h i s t o r y , maturation, t e s t i n g , instrumentation,  s t a t i s t i c a l r e g r e s s i o n , respondent mortality.  validity  b i a s , and e x p e r i m e n t a l  Factors jeopardizing external v a l i d i t y or repre-  s e n t a t i v e n e s s are the i n t e r a c t i v e e f f e c t o f t e s t i n g and 47 m u l t i p l e treatment  interferences.  In c o n t r a s t , the I n s t i t u t i o n a l Model tends t o r e l y on the e x p e r t judgment of those r e s p o n s i b l e f o r a d m i n i s t e r i n g the programme to determine of c o n t r o l groups,  the impact on r e c i p i e n t s .  g o a l s are t r a n s l a t e d i n t o  In l i e u  predetermined  measures o f success, f o r example, the number o f housing  starts.  I f the success measure i s a t t a i n e d i n the judgment o f programme a d m i n i s t r a t o r s the programme i s r a t e d as " s u c c e s s f u l " , whether or not the programme was a c t u a l l y i n s t r u m e n t a l i n a c c o u n t i n g f o r observed p a t t e r n s .  75  Stage Three:  Data C o l l e c t i o n ;  Data t o measure p r o -  gramme a c t i o n s i s c o l l e c t e d by p r o f e s s i o n a l e v a l u a t o r s e i t h e r programme r e c i p i e n t s o r programme Stage Four:  Data A n a l y s i s :  taken by programme e v a l u a t o r s ,  administrators.  Data a n a l y s i s , under-  r e q u i r e s the comparison o f  programme f i n d i n g s a g a i n s t i n i t i a l g o a l s and o t h e r criteria  f o r programme Stage F i v e :  Programme R e f o r m u l a t i o n :  m i n i s t r a t o r s f o r programme  Recommendations  t o programme ad-  reformulation.  Assumptions: The  ral  pre-established  "success".  a r i s i n g from Stage Four are t r a n s m i t t e d  The  from  t r a d i t i o n a l e v a l u a t i o n p r o c e s s r e s t s upon seve-  assumptions: (1)  I t i s assumed t h a t programme goals  are a v a i l a b l e ,  c l e a r , simple and t h a t such g o a l s were, i n t h e f i r s t instance, (2)  valid.  In d e f i n i n g goals the  and s e t t i n g e v a l u a t i o n  priorities  f u r t h e r assumption i s made t h a t the programme  administrator  can f u n c t i o n as a s u r r o g a t e  user i n d e f i n i n g which goals  f o r the  are most r e l e v a n t t o  programme performance. (3)  The r e d u c t i o n o f g o a l s  to readily quantifiable cost  measures assumes both t h a t those c h a r a c t e r i s t i c s capable o f q u a n t i f i c a t i o n r e f l e c t s e n s i t i v e p r o gramme elements and, secondly, t h a t a v a l i d  link  e x i s t s between i n p u t , c o s t , and s e r v i c e output. (4)  The t r a d i t i o n a l e x p e r i m e n t a l design  presupposes  76 t h a t c o n t r o l groups are a v a i l a b l e and  that i t i s  f e a s i b l e to e i t h e r a l l o c a t e people randomly to treatment and  c o n t r o l groups or to match p a r t i c i -  pants to s u i t a b l e c o n t r o l s on c h a r a c t e r i s t i c s r e l e v a n t to the programme's s o c i a l (5)  The  process.  f u r t h e r assumptions t h a t the p u b l i c i s e i t h e r  a p a t h e t i c , not w e l l informed, or i s i n c a p a b l e making p o l i c y d e c i s i o n s i s i m p l i c i t i n the  of  apparent  e x c l u s i o n of programme r e c i p i e n t s from a c t i v e p a r t i c i p a t i o n i n a l l stages o f the e v a l u a t i o n  process.  L i m i t a t i o n s o f E x i s t i n g E v a l u a t i v e Research Methodologies Deficiencies implicit  i n the assumptions  underlying  the t r a d i t i o n a l e v a l u a t i o n p r o c e s s r e s t r i c t d i r e c t a p p l i c a t i o n of e x i s t i n g e v a l u a t i o n methodologies to a e v a l u a t i o n of s o c i a l a c t i o n programmes.  user-oriented Specific limitations  i n c l u d e m e t h o d o l o g i c a l weaknesses, r e s t r i c t e d s u b j e c t matter, administrative  and  bureaucratic  r e s i s t a n c e and  the currency  of  research. Methodological  Weaknesses:  E x i s t i n g e v a l u a t i o n methodologies assume a programme's o b j e c t i v e s and  i t s o p e r a t i o n a l context  administrators  can be expected to a p p r e c i a t e  and  apply  impact.  experimental research  are such t h a t programme programme  goals  designs to t e s t programme  In the r e a l world s e t t i n g o f s o c i a l a c t i o n programmes  evaluators  are  faced w i t h a complex, e v o l v i n g s o c i a l s i t u a t i o n .  The  reality  of  are  necessary  led  experiment  social  programmes, where c o n t i n u a l  to maintain  service levels,  (holding either  Treatments r e f l e c t  standard.  Obtaining  throughout  the  assistance  f r o m e q u a l l y needy segments o f  service  to the  and  user  the  valid  the  impinging  requirements  maintaining  evaluation raises  recipient  precludes  programme o r  constant.  adjustments  and  control-  variables  are  seldom  c o n t r o l groups  ethical  issue of  the  withholding  population.  Since  i s paramount:  The e v a l u a t o r , whose b a s i c f u n c t i o n i s t o p r o vide information u s e f u l to decision-makers, can h a r d l y j u s t i f y i n t e r f e r e n c e w i t h the o p t i mal o p e r a t i o n o f t h e programme and t h u s he may be c a l l e d upon t o a d a p t h i s s t r a t e g i e s t o a c c o r d w i t h t h e r e a l i t i e s o f programme l i f e . (Weiss,72,p.7). Reliance question rogate  basis  of whether the  f o r the  indicate  on  citizen  professional expertise raises expert  can  in policy  that allowing planning  indeed  f u n c t i o n as  planning.  Several  to proceed  solely  o f t h e p r o f e s s i o n a l ' s k n o w l e d g e b a s e and  the a  sur-  studies  on  the  'conventional  48 wisdom' i s h i g h l y q u e s t i o n a b l e  practice.  fundamental d i f f e r e n c e s center  on  supply  and  frequently quality, tinuing  the  demand.  Federal concerns  conflict  with  locality, to r e l y  and  user  self  upon e x p e r t  In housing,  relative for  demands f o r  control.  The  j u d g m e n t may  quantity of  and  the  recipient's  expressed  given supply  availability, i m p l i c a t i o n of be  encourage d i s c o n t i n u i t y between p r o v i s i o n of services  stress  the  to continue  conto  government  demands.  Such a  split  78 between s u p p l i e r and r e c e i v e r c o n t r i b u t e s to i n c r e a s i n g d i s s a t i s f a c t i o n as s e r v i c e s appear u n r e l a t e d to consumer demand. S e v e r a l authors  (Wholey 70) have expressed  concern  t h a t n a t i o n a l schemes f o r l a r g e s c a l e programme e v a l u a t i o n do not e x i s t . It  In the context of housing  t h i s may  prove a b l e s s i n g .  i s q u e s t i o n a b l e whether s i n g l e c r o s s - c o u n t r y  e v a l u a t i o n s based upon one  e v a l u a t i o n guide have  Housing i s a l o c a l i s s u e , responding market c o n d i t i o n s . sible.  comparative  to, at best, r e g i o n a l  A s i n g l e e v a l u a t i o n scheme may  Rather, as Schon  (71)  utility.  suggests,  not be pos-  r e s e a r c h e r s must d e v i s e  an e v a l u a t i o n framework which r e g i o n s can adapt t o t h e i r s p e c i f i c needs and generate The  t h e i r own  e v a l u a t i o n programmes.  p r o p e n s i t y to base e v a l u a t i v e r e s e a r c h on com-  p a r i s o n s of i n i t i a l programme g o a l s and formance i s o f q u e s t i o n a b l e evaluations.  subsequent g o a l  per-  a p p l i c a b i l i t y i n user o r i e n t e d  Programme o b j e c t i v e s are too f r e q u e n t l y based on  l a r g e l y u n t e s t e d assumptions whose v a l i d i t y r e s t s p r i m a r i l y on t r a d i t i o n or "common sense" and not on proven e f f e c t i v e n e s s . Cohen  (70), f o r example, p o i n t s out the r i s i n g o p p o s i t i o n ,  p a r t i c u l a r i l y i n some b l a c k communities, to t r a d i t i o n a l cesses of g o a l  formulation.  In l i g h t o f the p r e c e e d i n g  d i s c u s s i o n i t i s reason-  able to assume t h a t , l a c k i n g c i t i z e n i n p u t i n t o g o a l . . . tion, i n i t i a l  pro-  g o a l s may  be u n r e l a t e d t o user  concerns.  Furthermore, e v a l u a t i o n s based upon p a s t goals e v o l v i n g system impacts and  formula49  understate  ignore p o t e n t i a l side  effects.  5 0  79  Administrative  Resistance:  Frequently  i t i s not the p r i n c i p l e s o f  r e s e a r c h which c r e a t e d i f f i c u l t i e s , problems of adhering the  evaluative  but r a t h e r the  practical  to these p r i n c i p l e s i n the f i e l d and  face of a d m i n i s t r a t i v e c o n s i d e r a t i o n s .  r e f l e c t the s t r o n g m o t i v a t i o n  Bureaucrats  by members of an  to seek e q u i l i b r i u m and preserve  in  organization  the o r g a n i z a t i o n .  Rein  suggests t h a t the p o s s i b i l i t y of r e s e a r c h being used by an agency depends on the extent which d e a l w i t h  to which r e s e a r c h  avoids  implications  the o r g a n i z a t i o n ' s most c e n t r a l dynamics.  Thus i t turns out t h a t one o f the major o b s t a c l e s to e v a l u a t i o n r e s e a r c h i s the i n t e r e s t s i n maintenance o f a program h e l d by i t s a d m i n i s t r a t o r s . T h e i r ambivalence i s born of a two-horned dilemma: On the one hand, r e s e a r c h i s needed to demonstrate t h a t the program has an e f f e c t ; on the o t h e r hand, r e s e a r c h might f i n d t h a t e f f e c t s are n e g l i g i b l e or non-existent. (Rossi 72, p. 2 2 7 ) . 51 L e g i s l a t i o n seldom i n c l u d e s p r o v i s i o n s f o r When e v a l u a t i o n does occur  i n an i n s t i t u t i o n a l  monitoring.  setting,  inadequate funds f o r s t a f f i n g r e s u l t i n r e g u l a r employees, have n e i t h e r t r a i n i n g ,  skills  nor e x p e r t i s e , undertaking  who  evalua-  t i o n s i n a d d i t i o n t o normal d u t i e s . In such cases, minimal time resources  are a l l o c a t e d t o p l a n n i n g  c o l l e c t i n g and  analyzing  programmes R e s t r i c t e d Subject  Matter:  Evaluative research d i r e c t use  i s designed f o r immediate  i n improving the q u a l i t y of s o c i a l  and  programming.  and  80 Yet a review o f e v a l u a t i v e e x p e r i e n c e suggests  that evaluation  r e s u l t s have g e n e r a l l y not e x e r t e d a s i g n i f i c a n t i n f l u e n c e on programme d e c i s i o n s .  To a degree,  the problem l i e s i n the  g e n e r a l l a c k o f knowledge o f the decision-making i d e n t i f i e d by Dror  process  (68), Gerecke (74), Wolman (71) and Meyerson  and B a n f i e l d (55). E v a l u a t i v e r e s e a r c h has tended gramme dimensions dimensions  t o assess those p r o -  most amenable t o q u a n t i f i c a t i o n .  Such  a r e n o t n e c e s s a r i l y e i t h e r the most r e l e v a n t o r the mo  s e n s i t i v e to manipulation i n a p o l i c y context. Swain contends  F o r example,  t h a t the d e s i r e o f p o l i c y makers t o know about  r e d i s t r i b u t i v e e f f e c t s o f p o l i c y p r o p o s a l s has been i g n o r e d by many r e s e a r c h e r s : The absence o f d i s a g g r e g a t e d answers t o the fundamental q u e s t i o n s o f Who pays? and Who b e n e f i t s ? l i e s a t the core o f some d e c i s i o n makers impatience w i t h r e s e a r c h as an a i d i n p o l i c y making. (Swain 73, p. 1 4 ) . A c c e n t i n g the q u a n t i t y o f s e r v i c e s o r a c t i v i t i e s assumes a d i r e c t l i n k between c o s t and s e r v i c e and d i s r e g a r d s the q u a l i t y o f such s e r v i c e s . unique  i n t h i s emphasis.  E v a l u a t i o n research i s not  Most urban r e s e a r c h , p a r t i c u l a r i l y  i n the economic s e c t o r , attempts  t o use r e l a t i v e l y  "tough  52 minded"  c r i t e r i a which a r e simple, d i r e c t and which f r e q u e n t l y  do n o t take i n t o account scene.  the r e a l complexity o f the urban  The e v a l u a t i o n o f f e e l i n g s , o f p e r s o n a l v a l u e s , and  a s p i r a t i o n s as v a r i a b l e s determining p o l i c y c h o i c e and housing p r e f e r e n c e are l a r g e l y  ignored.  81 Currency  of The  Research: r e c e n t emergence of programme e v a l u a t i o n i s , i n  itself,  a c o n t r i b u t o r t o p r e s e n t problems.  A shortage  of  persons  t r a i n e d t o e v a l u a t e r e a l world s e t t i n g s e x i s t s i n the  r e s e a r c h community. N e i t h e r government nor the s o c i a l s c i e n c e community are o r g a n i z e d t o develop and c a r r y out l a r g e s c a l e , d i m e n s i o n a l s t u d i e s o f s o c i a l impact.  Few  a b l e i n e v a l u a t i o n and p o l i c y a n a l y s i s .  multi-  courses are  In the  avail-  university, 53  time h o r i z o n s , p h i l o s o p h i c a l d i f f e r e n c e s and reward systems are designed t o encourage r e s e a r c h based  upon t h e o r e t i c a l  problems and t o d i v e r t r e s o u r c e s from e v a l u a t i v e t o more t r a d i t i o n a l forms o f b a s i c r e s e a r c h . Rein i n d i c a t e s t h a t t e a c h e r s o f s o c i a l p o l i c y must accept the awkward c o n c l u s i o n t h a t they cannot o f f e r to t h e i r students a methodology and t e c h nology f o r doing p o l i c y a n a l y s i s . 5 4 (Rein 70, p. 214). When e v a l u a t i v e work i s undertaken  t h e r e i s no  c e n t r a l c l e a r i n g h o u s e t o b r i n g t o g e t h e r the work o f 55 b e h a v i o u r a l and p o l i c y  scientist.  Because e v a l u a t i v e r e s e a r c h r e p o r t s are o f t e n l e f t unpublished or are p u b l i s h e d i n widely scattered journals, i t i s d i f f i c u l t t o e s t i m a t e the e x t e n t of a c t u a l use...(Caro 71, p. 6 ) . T h i s i s c e r t a i n l y t r u e i n the Canadian c o n t e x t where g e n e r a l l y 56 a v a i l a b l e published evaluation l i t e r a t u r e i s sparse. r e p o r t s do e x i s t , they tend t o l i s t attempt  d i f f i c u l t i e s with  Where no  t o analyze the source o f these d i f f i c u l t i e s or t o s e t  82  f o r t h g u i d i n g procedures  to h e l p l e s s e n or overcome the  problems. Thus e v a l u a t i v e r e s e a r c h can be c h a r a c t e r i z e d as l a c k i n g i n content,  i n t e g r a t i o n , and r e l e v a n c e to p o l i c y  cerns, i g n o r i n g s o c i a l c o s t s , s t r e s s i n g e f f i c i e n c y  over  e f f e c t , seldom c o n s i d e r i n g second o r d e r impacts and such m e t h o d o l o g i c a l confused which was  containing  and assumptive weaknesses as to leave  as t o whether i t was defective.  One  the programme or the  one  research  must q u e s t i o n the v a l i d i t y  e x i s t i n g r e s e a r c h from the viewpoint gramme e v a l u a t i o n .  con-  of  o f a user o r i e n t e d p r o -  I t i s indeed q u e s t i o n a b l e whether, i n the  context of a s o c i a l a c t i o n programme, g o a l s are ever and c l e a r , t h a t c o s t s based on economic v a r i a b l e s are i n d i c a t o r s o f programme success,  simple valid  t h a t programme a d m i n i s t r a -  t o r s are a b l e to a c t as s u r r o g a t e s  for c i t i z e n input, or that  c o n t r o l s are a v a i l a b l e or programmes  static.  Emerging Trends i n E v a l u a t i o n Research - Some Optimism Remains While i t i s c l e a r t h a t the e a r l y o p t i m i s t i c  pre-  d i c t i o n s of e x t e n s i v e s c i e n t i f i c e v a l u a t i o n of s o c i a l  pro-  grammes have f a i l e d to m a t e r i a l i z e , the concern  f o r the  development o f a methodology t o e v a l u a t e the impact o f government programmes remains. An  I d e n t i f i e d Need: .  .  From among the most ardent c r i t i c s attempts are a l s o to be found  57  of e v a l u a t i o n  some of the most o p t i m i s t i c  83 statements o f the p o t e n t i a l r o l e o f programme e v a l u a t i o n . We have never seen a f i e l d e v a l u a t i o n o f a s o c i a l - a c t i o n program t h a t c o u l d n o t be f a u l t e d l e g i t i m a t e l y by good m e t h o d o l o g i s t s , and we may never see one. But, i f we are w i l l i n g t o accept r e a l - w o r l d i m p e r f e c t i o n s and t o use e v a l u a t i v e a n a l y s i s w i t h prudence, then such a n a l y s i s can p r o v i d e a f a r b e t t e r b a s i s f o r d e c i s i o n making than we have had i n the p a s t . (Williams 71, p. 121). Enough good r e s u l t s have emerged on the p l u s  side  of e v a l u a t i o n s t u d i e s t o i n d i c a t e the p o t e n t i a l o f a f u l l scale e f f o r t .  Among emerging trends which may l e a d t o more  meaningful e v a l u a t i o n s a r e : (1)  An i n c r e a s e d c o n s i d e r a t i o n o f ongoing as opposed to summative e v a l u a t i o n ;  (2)  Extension  o f e v a l u a t i o n across  agency and d i s c i p l i n e  b o u n d a r i e s ; and (3)  The r e a l i z a t i o n t h a t b e h a v i o u r a l  s c i e n t i s t s who  hope t o c o n t r i b u t e t o t h e e f f e c t i v e n e s s o f s o c i a l programmes through e v a l u a t i v e r e s e a r c h need t o concern themselves n o t o n l y w i t h methodological concerning are  the immediate  problems but a l s o with  t h e s o c i a l context  larger issues  i n which programmes  conducted.  User Involvement: The  s o c i e t a l context  includes c o n s i d e r a t i o n o f pro-  gramme impacts i n l i g h t o f user requirements.  Including  c i t i z e n s and, s p e c i f i c a l l y , programme r e c i p i e n t s i n an e v a l u a t i o n process expert.  assumes t h a t the user  i s , i n h i s own r i g h t , an  F o r example i n housing, those i n v o l v e d i n the search  84  f o r and  acceptance  o f housing accommodation develop some  s p e c i f i c b e h a v i o u r a l p a t t e r n s and a t t i t u d e s towards the housing  situation.  The e x t e n t to which one programme r e c i p i e n t  can a c t as an "expert" on what h i s " f e l l o w s " d e s i r e i s an i n t e r e s t i n g q u e s t i o n a p p a r e n t l y unanswered i n the I f c i t i z e n involvement i s t o be meaningful, and Friedmann  literature.  i n the e v a l u a t i o n process  s e v e r a l authors, among them Caputo  (73)  (73) see the n e c e s s i t y f o r e s t a b l i s h i n g n e t -  works i n v o l v i n g c i t i z e n s , r e s e a r c h e r s , p o l i c y makers and  pro-  gramme a d m i n i s t r a t o r s .  as  P l a n n i n g and e v a l u a t i o n are seen  a mutual l e a r n i n g e x p e r i e n c e - the p l a n n e r from the p e r s o n a l knowledge, the c l i e n t from the p l a n n e r ' s expertise. Caro's  The process i s by no means e a s i l y  client's  technical  accomplished.  a n a l y s i s o f the i n t e r p l a y between c l i e n t and e v a l u a t o r  i n p l a n n i n g o f the Denver Model C i t i e s programme  concluded  that c o n d u c t i n g s o c i a l r e s e a r c h i n the f i e l d o f t e n means t h a t data are slow i n coming, complex and q u a l i f i e d but, i n a gross sense o b s e r v a t i o n s o f r e s i d e n t s about the consequences o f programs may be sound and these judgments may be d e l i v e r e d w i t h a c l a r i t y and c o n v i c t i o n which appeals t o the p o l i c y maker. (Caro i n C a r t e r 73, p. 67). What Caputo and Friedmann, among o t h e r s , seem to be c a l l i n g f o r i s a methodology which c o u l d develop new  standards by f e e d i n g more  d i v e r s e i n f o r m a t i o n i n t o the p l a n n i n g p r o c e s s - namely data d e r i v e d from s o c i a l s c i e n t i f i c i n q u i r i e s about the p r e f e r e n c e of p r e s e n t and p o t e n t i a l programme u s e r s .  Explicit  criteria  to e s t a b l i s h programme g o a l s from the user's p e r s p e c t i v e , f o r  85  considering evolving value-laden multi-dimensional  programmes,  or f o r d i f f e r e n t i a t i n g programme from system impacts context o f an a c t i o n o r i e n t e d programme are enunciated  i n the  i n the  inadequately  literature.  In b u i l d i n g t h i s base of knowledge, we must not succumb to the temptation to take up the s o l u able problems and f o r s a k e the i n t r a c t a b l e dilemmas on the assumption t h a t t h i s i s the o r d e r l y route t o knowledge. S o c i a l p o l i c y i s a l l about s o c i a l o b j e c t i v e s and the v a l u e s t h a t embody the c h o i c e of s o c i a l programs. These are p r e c i s e l y the problems t h a t touch the l i m i t s o f s o c i a l s c i e n c e and r a i s e the s p e c t r e o f the a n c i e n t and s t i l l i n a d e q u a t e l y e x p l o r e d t e r r a i n where f a c t s and v a l u e s merge. (Rein 70, p. 214). The  Task Evidence  presented  i n chapters one  and two  appears  to s u b s t a n t i a t e the t h r e e assumptions upon which the s e r t a t i o n concern  i s based - t h a t c i t i z e n s are  dis-  increasingly  aware of and d e s i r o u s of p a r t i c i p a t i o n i n the s o c i a l  planning  p r o c e s s ; t h a t governments, p a r t i c u l a r i l y i n the f i e l d housing,  are responding  c o n s t i t u e n t s through  of  to p e r c e i v e d s o c i a l needs o f t h e i r  l e g i s l a t i n g s o c i a l a c t i o n programmes;  and t h a t , i n an attempt t o improve the match between  social  need and government a c t i o n s , e v a l u a t i o n methodologies have e v o l v e d to assess  policy.  However, the t r a d i t i o n a l e v a l u a t i o n p r o c e s s to be inadequate  appears  f o r the e v a l u a t i o n of programmes from both  the needs and e x p e c t a t i o n s o f the user and  from the  stand-  p o i n t of i n v o l v i n g c i t i z e n s i n t h i s aspect o f the p l a n n i n g process.  86  I f one assumes t h a t i n c l u d i n g some members o f the concerned p o p u l a t i o n  i n the e v a l u a t i o n process  will  increase  the l i k e l i h o o d o f the e v a l u a t i o n r e f l e c t i n g s e n s i t i v e p r o gramme dimensions, b e i n g proposals  r e l e v a n t t o user concerns, and t h a t  w i l l not cause f u r t h e r misunderstanding between  administrator  and user,  appears t o be needed.  then an a l t e r n a t e e v a l u a t i o n method  87  CHAPTER I I I TOWARD A USER ORIENTED METHODOLOGY FOR THE EVALUATION OF SOCIAL ACTION  PROGRAMMES  The p r e c e d i n g c h a p t e r s d i s c u s s e d the i n c r e a s e o f c i t i z e n p a r t i c i p a t i o n i n the p l a n n i n g p r o c e s s and t h e a t t e n d a n t expanding  s o c i a l consciousness o f governments.  Also  identified  were two gaps i n e x i s t i n g knowledge - how t o e v a l u a t e a government programme from t h e p e r s p e c t i v e o f the c i t i z e n user and, secondly, how the programme r e c i p i e n t might a s s i s t i n making such an e v a l u a t i o n more i n d i c a t i v e o f programme T r a d i t i o n a l programme e v a l u a t i o n seeks  impact. to provide  e v a l u a t i o n i n f o r m a t i o n through an i d e n t i f i c a t i o n o f p o l i c y o b j e c t i v e s and subsequent a n a l y s i s o f g o a l f r e q u e n t l y employing recipients.  minimal  attainment,  c o n s u l t a t i o n w i t h programme  The base upon which change i s measured i s e s t a b -  l i s h e d by programme a d m i n i s t r a t o r s r a t h e r than programme u s e r s . With t h i s concern i n mind, Chapter  I I I o u t l i n e s an  a l t e r n a t e e v a l u a t i o n process through which programme u s e r s i d e n t i f y d e s i r e d g o a l s and a s s i s t i n a s s e s s i n g the e x t e n t t o which programme output meets user e x p e c t a t i o n s .  The f i r s t  s e c t i o n s o f t h i s c h a p t e r c o n s i d e r the r e s e a r c h s e t t i n g ,  fol-  lowed by a d e s c r i p t i o n , e x p l a n a t i o n and j u s t i f i c a t i o n o f a methodology t o p r o v i d e user o r i e n t e d e v a l u a t i o n i n the c o n t e x t of  a v a l u e - l a d e n , ongoing,  programme.  multi-dimensional s o c i a l  action  The chapter concludes w i t h a p r o p o s a l t o  i d e n t i f y the p o t e n t i a l r o l e and r e s p o n s i b i l i t i e s o f programme users i n t h e e v a l u a t i o n p r o c e s s .  88  PART I THE  RESEARCH SETTING - A  To  CONSIDERATION AND  d a t e t h e r e has  been a d i s t i n c t  JUSTIFICATION  split  between  aca-  58 demic r e s e a r c h  and  u n d e r s t a n d a b l e by academic nor theory  to  account and  administrative and  world  back t o  infrequent  concern o u t l i n e d  in  context of  lowing research  THE  split  i n neither  i s made  the  encourage a t r a n s i t i o n from Differences  time h o r i z o n s  in  res-  further research  analysis.  search  and  The  i n t e r c h a n g e between b a s i c  While d i f f i c u l t i e s  the  which  theory.  k n o w l e d g e b a s e s and  f o r the  policy  reward s t r u c t u r e s  practice  ponsibility,  social policy evaluation.  will  i n e v i t a b l y a r i s e , the  i n t h i s d i s s e r t a t i o n suggests  a r e a l world planning  design attempts  to  situation.  provide both  for  d i r e c t u t i l i z a t i o n o f f i n d i n g s by: - a n a l y s i s o f an e x i s t i n g s o c i a l a c t i o n in i t s operational setting.  re-  testing  The  fol-  generality  programme  -  s e l e c t i o n of a l o c a l e f o r study i n which s e v e r a l p r o j e c t s have been i n i t i a t e d f o r c o m p a r a t i v e purposes.  -  s e l e c t i o n o f a programme w h i c h incorporates e m e r g i n g t r e n d s i n C a n a d i a n h o u s i n g p o l i c y and i s presently faced with administrative difficulties .  -  a n a l y s i s o f a programme w h i c h incorporates s e v e r a l a s s u m p t i o n s w h i c h have b e e n o f c o n c e r n to housing t h e o r i s t s .  REGIONAL SETTING On  social  action  the  assumption  that  a methodology  programmes s h o u l d be  tested  for  i n the  assessing  context  of  89  such a programme, a p o r t i o n of the N a t i o n a l Housing  Act  A s s i s t e d Home Ownership Programme, as a p p l i e d i n the G r e a t e r Vancouver R e g i o n a l D i s t r i c t , was academic r e s e a r c h . one  l o c a l e was  concern was Had  chosen as the s e t t i n g f o r  The d e c i s i o n to analyze one programme i n  based  on the r e a l i z a t i o n t h a t the  dissertation  to formulate and t e s t an e v a l u a t i o n methodology.  an a n a l y s i s o f change r e s u l t i n g from programme p a r t i c i p a -  t i o n been the d i s s e r t a t i o n f o c u s , an examination  of v a r i o u s  p o l i c i e s , over time, i n a number o f r e g i o n s would have been necessary.  However, s i n c e the r e s e a r c h s t r e s s e s p r o c e s s ,  not p r o d u c t , a n a l y s i s of a s i n g l e programme a p p l i c a t i o n i s considered  justified. S e l e c t i o n o f a s p e c i f i c r e g i o n a l example f o r  a n a l y s i s r e f l e c t s the s u g g e s t i o n of Donnison i n the Government of Ho