UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Interactive evaluation : a user-oriented process to assist housing programme reformulation McAfee, Rosemary Ann Pickard 1975

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-UBC_1975_A1 M32_6.pdf [ 18.78MB ]
Metadata
JSON: 831-1.0093605.json
JSON-LD: 831-1.0093605-ld.json
RDF/XML (Pretty): 831-1.0093605-rdf.xml
RDF/JSON: 831-1.0093605-rdf.json
Turtle: 831-1.0093605-turtle.txt
N-Triples: 831-1.0093605-rdf-ntriples.txt
Original Record: 831-1.0093605-source.json
Full Text
831-1.0093605-fulltext.txt
Citation
831-1.0093605.ris

Full Text

( INTERACTIVE EVALUATION: A USER-ORIENTED PROCESS TO ASSIST HOUSING PROGRAMME REFORMULATION by ROSEMARY ANN (PICKARD) MCAFEE B.A. University of B r i t i s h Columbia, 1962 M.A. University of B r i t i s h Columbia, 1967 A THESIS SUBMITTED IN PARTIAL FULFILLMENT • OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY i n the School of Community and Regional Planning (In t e r d i s c i p l i n a r y ) We accept t h i s thesis as conforming to the i required standard THE UNIVERSITY OF BRITISH COLUMBIA 1975 In presenting t h i s thesis i n p a r t i a l f u l f i l m e n t of the require-ments for an advanced degree at the University of B r i t i s h Col-umbia, I agree that the Library s h a l l make i t f r e e l y a v a i l a b l e for reference and study. I further agree that permission for extensive copying of t h i s thesis for scholarly purposes may be granted by the Head of my Department or by his representat-ives. I t i s understood that copying or p u b l i c a t i o n of t h i s thesis for f i n a n c i a l gain s h a l l not be allowed without my written permission. School of Community and Regional Planning. The University of B r i t i s h Columbia Vancouver, B.C., Canada. i i ABSTRACT This d i s s e r t a t i o n evolved from the i d e n t i f i c a t i o n of three trends. The f i r s t i s that governments are exhib i t i n g an increasing concern for the s o c i a l welfare of t h e i r c o n s t i t -uents. This concern i s r e f l e c t e d i n the expanded number of, and resources committed to, s o c i a l action programmes. The second i s that c i t i z e n s are becoming increasingly desirous of p a r t i c i p a t i o n i n the formulation of s o c i a l p o l i c i e s designed to provide for t h e i r welfare. And, f i n a l l y , as s o c i e t a l goals change p o l i c i e s must be altered to r e f l e c t emerging d i r e c t i o n s . Evaluation of e x i s t i n g s o c i a l action programmes p o t e n t i a l l y provides decision makers with an i n d i c a t i o n of the extent to which government actions a s s i s t i n d i v i d u a l s to share the qu a l i t y of l i f e enjoyed by society-at-large. Unfortunately, an assessment of evaluation attempts indicates findings have seldom made a s i g n i f i c a n t contribution to p o l i c y reformulation. The question arises as to whether evaluation of government programmes i s , i n i t s e l f , unattainable or whether e x i s t i n g evaluation processes merely apply inapprop-r i a t e methodologies to s o c i a l action programme s i t u a t i o n s . Con-sideration of t r a d i t i o n a l evaluation methods i n l i g h t of evolv-ing planning thought suggests the l a t t e r condition may well apply. Programme evaluation must possess the capacity to a s s i s t in the conceptualization of sensi t i v e issues to be evaluated, provide a framework for executing the study i n the context of an evolving programme, and make provision for the dissemination and use of findings i n a manner amenable to p o l i c y implement-ation . In addition, decision making through the p o l i t i c a l process involves bargaining between diverse i n t e r e s t s . To undertake t h i s task requires an appreciation of the range of alternate options for those affected whether i t i s the p u b l i c -at-large or a s p e c i f i c c l i e n t group. R e a l i s t i c tradeoffs can only be made i n l i g h t of a knowledge of personal and s o c i e t a l impacts. T r a d i t i o n a l programme evaluation models assume s t a t i c programmes for which i n i t i a l goals are a v a i l a b l e , clear and remain v a l i d over time. T r a d i t i o n a l models seldom d i r e c t l y assess user attitudes to programme dimensions and make no provision for c i t i z e n p a r t i c i p a t i o n i n the evaluation process. Such patterns appear inconsistent with emerging planning theory. Planners and p o l i t i c i a n s are c a l l i n g for new processes to pro-vide p o l i c i e s which are responsive to varied user needs and ant i c i p a t e s o c i a l costs. An analysis of e x i s t i n g evaluation procedures suggests t r a d i t i o n a l methods f a i l to f u l f i l l these c r i t e r i a . The search for a more responsive evaluation process led the researcher to speculate on the implications of i n c l u d -ing members of the concerned r e c i p i e n t population i n the eval-uation process. It was hypothesized that i f r e c i p i e n t s of gover nment programmes were d i r e c t l y involved i n the evaluation proces the i r analysis of programme guidelines would a s s i s t s e n s i t i v e r e d i r e c t i o n of government actions. The resultant user-oriented evaluation process d i f f e r s from t r a d i t i o n a l methods by involving programme re c i p i e n t s at a l l stages i n the evaluation process. Programme users i d e n t i f y desired goals and a s s i s t i n determining the extent to which programme output meets user expectations. Included i n the a l t e r -nate evaluation are procedures to locate s e n s i t i v e programme impact points, to handle the eventuality of changing user concerns, and to i d e n t i f y appropriate l e v e l s of causation. Recommendations for government agencies to incorporate user input i n future programme evaluations r e s t on a comparative test of t r a d i t i o n a l and user oriented evaluations undertaken on a s o c i a l housing programme i n the Greater Vancouver area. The test s i t u a t i o n was selected to represent a t y p i c a l planning problem requiring interagency cooperation on a multidimensional s o c i a l issue. Comparative analysis of the two evaluation-processes indicates that programme re c i p i e n t s can provide a d i s t i n c t i v e input into the evaluation stage of the planning process. Prev-iously held perceptions of programme re c i p i e n t s as re s e n t f u l toward privacy invasion and lacking the knowledge base to a s s i s t programme evaluation proved i n v a l i d . Given a sincere desire by the sponsoring agency to include user perspectives, the r e c i p i e n t i s both w i l l i n g and able to provide an evaluation of the e f f e c t s of programme p a r t i c i p a t i o n . This i s not to say that following evaluation of a programme from the r e c i p i e n t ' s perspective the programme must V. necessarily be rewritten to his s p e c i f i c a t i o n s . User specif-ic a t i o n s may involve tradeoffs society-at-large i s unwilling to accept. Ultimately decisions rest on elected representatives. The argument emerging from t h i s research i s that through i n c l u s -ion of users i n the evaluation process the decision maker w i l l at least be aware of c l i e n t reaction before reaching his decision. User p a r t i c i p a t i o n i s part of a process; i t i s not intended as the product. Findings suggest i t i s neither p r a c t i c a l nor desirable for programme rec i p i e n t s to replace agency personnel throughout the evaluation process. A more r e a l i s t i c procedure emerges through an i n t e r a c t i v e evaluation process which combines input from programme administrators, p o l i t i c a l decision-makers, and the knowledge sector with that from programme r e c i p i e n t s . An i d e n t i f i c a t i o n of the role and r e s p o n s i b i l i t i e s of each actor in the evaluation process emerges from the case study. Accept-ance of study recommendations by administrators, decision-makers and r e c i p i e n t s points toward i n t e r a c t i v e evaluation as a potent-i a l source of information for public p o l i c y generation and prog-ramme reformulation. v i TABLE OF CONTENTS Page' CHAPTER I; TOWARD CITIZEN INVOLVEMENT IN THE PLANNING PROCESS Soc i e t a l Guidance From the Top Down 2 Reaction from the Bottom Up 4 A search for meaningful involvement 5 Rejection of a u n i f i e d public i n t e r e s t 5 Concern for the pervasiveness of government actions 6 Questioning the experts bias 6 C i t i z e n P a r t i c i p a t i o n : Where we are now. Models of c i t i z e n involvement 8 Examples of c i t i z e n involvement i n the planning process 11 Goal formulation stage 12 Poli c y implementation stage 15 Evaluation stage 16 Evaluation of Social Action Programmes Limitations of e x i s t i n g evaluation methodologies . 17 The case for a user oriented evaluation methodology 19 Diss e r t a t i o n Proposal 20 Testing a user oriented evaluation methodology ... 21 The t e s t s i t u a t i o n 22 Toward an appreciation of l i m i t a t i o n s 24 CHAPTER I I : SOCIAL CONSCIOUSNESS IN A GOVERNMENT POLICY  SETTING Programme P r o l i f e r a t i o n Under an Increasing So c i a l Consciousness An example i n the Context of Canadian Housing Poli c y 28 Housing as an Economic Tool 30 The Emergence of Housing as a So c i a l Concern ... 31 Programme Evaluation as an Indicator of S o c i e t a l Directions The Nature of Evaluative Research 34 Diverse Purposes 35 Diverse D e f i n i t i o n s 38 v i i Page The Evolution of Evaluative Research As a Poli c y Tool 41 As a Concern of the Soc i a l S c i e n t i s t 43 Housing as an Evaluative Research Focus Limitations of Academic Evaluations 48 E f f o r t s of Federal Housing Agencies 53 Application of Ex i s t i n g Evaluation Methodologies Theoretical Models Wholey's Experimental Model 56 Carter & Wharf's Typology 56 Caputo's Integration Process 57 Operational Models CMHC's $200 M i l l i o n Programme 62 Onibokun's Relative H a b i t a b i l i t y 64 Gruen & Gruen's P r i o r Policy Analysis 66 Feagin's Programme Reformulation 6 8 The T r a d i t i o n a l Evaluation Model The Process Stage One: D e f i n i t i o n of Programme Goals .. 71 Stage Two: Evaluation Design 71 Stage Three: Data C o l l e c t i o n 75 Stage Four: Data Analysis 75 Stage Five: Programme Reformulation 75 The Assumptions 75 Limitations of E x i s t i n g Evaluation Methodologies Methodological Weaknesses 76 Administrative Resistance 79 Restricted Subject Matter 79 Currency of Research 81 Emerging Trends i n Evaluation Research An I d e n t i f i e d Need 82 User Involvement 83 The Task 85 CHAPTER III; TOWARD A USER ORIENTED PROCESS FOR THE  EVALUATION OF SOCIAL ACTION PROGRAMMES PART I: THE RESEARCH SETTING - A CONSIDERATION  AND JUSTIFICATION The Regional Setting 88 Sponsorship - The Greater Vancouver Regional Housing Deapartment 91 The Programme - The "300 Unit Assisted Home Ownership Programme" 9 2 Theoretical Basis of the Programme 94 The Ownership Assumption 95 The Income Mix Assumption 100 v i i i Page The L e g i s l a t i v e Basis of the Programme Pressures for Government Intervention 107 Pr o v i n c i a l Actions 10 8 Federal Actions 109 Origins: Hellyer Task Force 109 Experience i n the United States. 110 Implementation of Canadian L e g i s l a t i o n $200 M i l l i o n Innovative Pro-gramme Section 58 112 Assisted Home Ownership Pro-gramme 113 PART II; CONCEPTUALIZATION AND TESTING OF A USER  ORIENTED EVALUATION METHODOLOGY Evaluation One: The T r a d i t i o n a l Evaluation Research Design 117 Evaluation Two: A User Oriented Evaluation Process 118 Focussing on the Programme Recipient 120 Assessing Recipient Behavior 120 Assessing Recipient Attitudes 121 Assessing Recipient Opinions 123 Data C o l l e c t i o n - The Interview 124 Overcoming Limitations of T r a d i t i o n a l Evaluation Research 130 A Proposed Methodology to Identify User Concerns and S a t i s f a c t i o n Levels 132 I d e n t i f i c a t i o n of User Concerns 135 Determining S a t i s f a c t i o n Levels 138 A Proposed Methodology to Handle a Multi-Demensional Programme 141 Dimension Testing 142 Indicators of Economic and So c i a l Consequences of Ownership 143 Indicators of the So c i a l Consequences of Income Mix 144 Indicators of the Service Redistributive Consequences of Income Mix 146 Indicators of Recipient Involvement i n the Provision of Housing 149 A Proposed Methodology to Identify House-hold-Programme-System Intervention Directions 153 Conclusion 155 ix CHAPTER IV: THE USER ORIENTED EVALUATION PROCESS —  A TEST CASE STUDY A Comparative Analysis of Two Evaluation Processes I d e n t i f i c a t i o n of Programme Evaluation Goals 159 T r a d i t i o n a l Evaluation — Expert Enunciated evaluation dimensions 160 Program Demand 160 Agency Protection 162 P r i o r i t i e s for Analysis 163 Research Design 165 Success Indicators 165 User Oriented Evaluation -- User Enunciated evaluation dimensions 166 Redistributive E f f e c t s of P a r t i c i p a t i o n 166 Continuing Obligations 167 Research Design 168 A Comparison of Programme Evaluations Based upon Expert and User Enunciated Dimensions Content Comparison 169 Research Design Comparison 171 Discussion of Programme Evaluation Findings 171 The Impact of Assuming Home Ownership Economic Impact 17 2 S t a b i l i z i n g Shelter Costs 172 Economic Redistribution 178 S o c i a l Impact Redistributive E f f e c t s 180 S o c i a l Stress 181 Comparison of T r a d i t i o n a l and User Findings.... 182 The S o c i a l and Service Redistributive E f f e c t s of Income Mix Soci a l E f f e c t s Stigma Reduction 184 Neighbouring Patterns 186 Service F f f e c t s Home Space 187 Neighbourhood Space 188 Community Space 190 Comparison of T r a d i t i o n a l and User Findings.... 190 The Impact of Recipient p a r t i c i p a t i o n in the Provision of Housing Contacting Applicants 191 Continuing Agency-client Relations 192 Comparison of T r a d i t i o n a l & User Findings 194 X I d e n t i f i c a t i o n and Discussion of Program Reformulation Proposals 196 Reformulation of Ownership Guidelines 199 Continuing Agency Responsibility 201 Maximizing Redistributive E f f e c t s 203 Loan Terms 203 Reduction of Speculative Gains 206 Mobi l i t y F l e x i b i l i t y 206 Reformulation of Income Mix Guidelines 207 Quantitative Guidelines 207 Unit C r i t e r i a 208 Stigma Reduction 209 Operationalizing Integration 210 Reformulation of C i t i z e n P a r t i c i p a t i o n Guidelines213 Contacting Applicants 213 Selection of Applicants 213 Legal Aspects 214 Continuing Agency-client Relations .215 A Comparison of Program Reformulation Proposals Based Upon T r a d i t i o n a l and User Evaluations 215 Implications of Case Study Findings Implications for E x i s t i n g Housing Theory 216 Ownership i n Theory and Practice 217 Income Mix i n Theory and Practice 220 User P a r t i c i p a t i o n i n Theory and Practice 223 Implications for Canadian Housing P o l i c y 223 Uncoordinated Actions 224 Innapropriate Interventions 228 Unrealized Expectations 229 Implications to the Evaluation of Residential Environments The Use of Attitude and Behavioural Data 231 Residential Indicators 234 Performance Guidelines ...235 S a t i s f a c t i o n Ratios 236 Implications to Program Evaluation Methodologies... 237 CHAPTER V: GUIDELINES TOWARD A PROCESS TO INVOLVE PROGRAMME  RECIPIENTS IN THE EVALUATION OF SOCIAL ACTION  PROGRAMMES. Overview of the Proposed Process 240 Role and Responsibility of the User i n Programme Evaluation User Contributions to the I d e n t i f i c a t i o n of Evaluation Goals 241 Identifying Dimensions ....243 Complementary Input 244 User Contributions to Preparation of an Evaluation Design 245 x i Letters of Introduction 2 4 6 Survey Format 2 4 6 Audio-visual Aids 2 4 7 Method vs Content 2 4 7 Remuneration 2 4 8 User Contributions to Data C o l l e c t i o n 2 4 8 Invasion of Privacy 2 4 9 Cost 2 5 4 Grateful Testimonial 2 5 5 Evaluation as a Change Agent 2 5 5 User Contributions to Data Analysis 2 5 6 Analysis Rationale 2 5 7 Program Compatability 2 5 7 Quantifiable and Non-quantifiable Data 2 5 8 Aggregation 2 5 9 User Contributions to Program Reformulation Knowledge Base 2 6 0 The Role of the User i n the Evaluation Process 2 6 2 CHAPTER VI: POLICY EVALUATION AS AN INTERACTIVE PROCESS Limitations of Research Findings 2 6 7 Implications of Findings for E x i s t i n g Housing Theory and Practice 2 6 9 Implication of Findings for the Evolution of an Interactive Evaluation Process 2 7 0 Conceptualization of Issues for Programme Evaluation 2 7 1 Executing Evaluative Studies 2 7 4 Conditions necessary for Implementing Interactive Policy Evaluation 2 7 6 Role and Responsibility of C i t i z e n Recipients 2 7 7 Realizing Myths 2 7 8 Realizing R e a l i t i e s 2 7 9 Role and Responsibility of Decision-Makers 2 8 0 Role and Responsibility of Government Agencies 2 8 2 Role and Responsibility of the Knowledge S e c t o r . . . . 2 8 4 Planner as Evaluator 2 8 5 Action Orientation 2 8 7 Implementation Orientation 2 8 8 Broadening Perspectives 2 8 8 Alternate Methodologies 2 8 9 Rethinking Planning Education 2 9 2 Planning and the Evaluative Process 2 9 3 FOOTNOTES 2 9 6 BIBLIOGRAPHY 3 3 7 APPENDICES Sample questionnaires 3 6 0 x i i LIST OF ILLUSTRATIONS 1. Diagram Goal Formulation Directions . . 1 2. Diagram Models of C i t i z e n P a r t i c i p a t i o n 9 3. Diagram Caputo 1s Model for Urban Public Po l i c y Evaluation 58 4. Diagram A Model of the T r a d i t i o n a l S o c i a l Action Programme Evaluation Process 71 5. Map The Greater Vancouver Regional D i s t r i c t 90 6. Table NHA Assistance to Lower Income Households ... 108 7. Diagram Guideline for t e s t i n g an Alternate Evaluation Process 116 8. Table Duration of Programme Exposure 127 9. Table Components of a Household P r o f i l e and Dimension Indicators 133 10. Diagram Dimensions for evaluation 152 11. Table Evaluation P r i o r i t i e s 161 12. Table Incomes of Programme Families 173 13. Table Percentage of Income Expended on Housing .... 174 14. Table Resident Occupations 178 15. Table Resale A c t i v i t y 179 16. Diagram Unit, Neighbourhood and Community Concerns .. 188 17. Diagram Programme Dimensions — Evaluation and Reformulation Process 198 18. Table How I n f l a t i o n Hit Vancouver, 1972-1973 200 19. Diagram Household Value Clusters 219 20. Diagram The Canadian Housing Bureaucracy 225 21. Diagram Po l i c y , Programme and P a r t i c i p a n t Compatability 232 22. Table P r i o r i t y Concerns 235 23. Diagram The Role of the Recipient i n Programme Evaluation 263 24. Table Recommendations for a Revised Evaluation Process 272 25. Diagram Actors i n the Evaluation Process 277 26. Diagram Fusing Process and P r a c t i c a l Knowledge Bases. 286 27. Table Components of a Methodology to Reflect the Contingencies experienced i n a Real World Setting 291 28. Diagram Evaluation i n Perspective 294 x i i i ACKNOWLEDGEMENT The reader w i l l f i n d that this d i s s e r t a t i o n i s predicated on the assumption that benefits are to be derived from i n t e r r e l a t i n g experiences gained through the processed knowledge of the university with knowledge obt-ained through agency involvement and user experiences i n the housing process. Both the Greater Vancouver Regional D i s t r i c t Housing Department and Central Mortgage and Housing Corporation were supportive of this research. B i l l Casson, GVRD Housing Department Director, Jim Moodie,GVRD> and Gary Young, CMHC, were p a r t i c u l a r i t y h e l p f u l i n arrang-ing interviews and discussing program actions. They, and the recipients of the "300 Unit Housing Programme" who so w i l l i n g l y shared t h e i r housing experiences, provided a t r i a l t e st of the proposed Interactive Evaluation Process* However, without friends and colleagues writing a d i s s e r t a t i o n could have become a very lonely and f r u s t r a t -ing experience. I t was my good fortune to receive from my. committee — Dr. R.W. C o l l i e r , Dr. R.B. H o r s f a l l , and Dr. R.F. Kelly; from my friends and colleagues, most esp e c i a l l y Dr. Kent Gerecke, Rick E l l i g o t t , Emil Gutman and Roger McAfee; and from my parents, continuing encouragement and support for my endeavours. To those who shared t h i s experience, my h e a r t f e l t thanks for caring enough to see me through i t . The assistance of the Canada Council, of Miss Evelyn Popoff who undertook to spend her Christmas typing, and of a women whose theme song provided background music i f I have to I can do anything . . . . " i s acknowledged with thanks. "the adequate assessment of e x i s t i n g and innovative programs can be a v i t a l force d i r e c t i n g s o c i a l change and improving the l i v e s and environments of community mem-bers. " (Caro 71, p. 1). 1 CHAPTER I TOWARD CITIZEN INVOLVEMENT IN THE PLANNING PROCESS The "planning process" has emerged as a framework through which to guide society toward an improved q u a l i t y of l i f e . Broadly speaking the planning process includes c l a r i f y -ing one's objectives and then determining what action s h a l l be taken, by whom, when, by what methods, and at what costs, i n order to achieve desired goals.^ Just who should designate society's goals remains a question open to debate. Two sources of goal generation e x i s t . I d e n t i f i c a t i o n of s o c i e t a l goals can be undertaken by selected representatives of the s o c i e t y assisted by persons trained i n the process of s o c i e t a l guidance. A l t e r n a t i v e l y , s o c i e t a l values are i d e n t i f i e d d i r e c t l y by society at large. Recommendations are then passed to those with s c i e n t i f i c and technical knowledge for t r a n s l a t i o n into s p e c i f i c actions. Direct involvement by c i t i z e n s i n the planning process has been termed " c i t i z e n p a r t i c i p a t i o n " . The two approaches can be de-scribed as "top down" and "bottom up" goal formulation. ILLUSTRATION I GOAL FORMULATION Top Down Planner Generates Bottom Up Planners Respond Planning So c i e t a l Directions Planning 4> Society Responds Society Generates Directions 2 The s p e c i f i c process of goal generation i s a moot point when goals, generated by whatever process, are accept-able to a l l sectors of society. Planning can proceed through the subsequent stages of formulating objectives, targets, and actions. Problems a r i s e when s o c i e t a l objectives, as i d e n t i -f i e d by the two processes, d i f f e r . In the absence of con-census, s e l e c t i o n of one goal set may well lead to actions considered detrimental by opposing i n t e r e s t s . Planners have been placed i n the center of the controversy over who should determine society's d i r e c t i o n s . S o c i e t a l Guidance from the "Top Down" Placing the r e s p o n s i b i l i t y for s o c i e t a l guidance on the decisions of government bureaucrats assumes a b e l i e f i n an i d e n t i f i a b l e public i n t e r e s t , and a future capable of being designed by a ce n t r a l agency. Interpretation of the public i n t e r e s t has been assumed to be a s p e c i a l concern of the planning profession. Wheaton (70) suggests t h i s res-p o n s i b i l i t y was based upon some implied knowledge of the interdependencies of land uses. This knowledge provided plan-ners with the framework to coordinate s p e c i a l i s t plans for the benefit of society. Lacking processes to include public p a r t i c i p a t i o n , planners have frequently acted on behalf of society. Several studies have i d e n t i f i e d problems of public consultation to include the l i m i t a t i o n s laymen face i n a r t i c u l a t i n g needs i n a context amenable to po l i c y formulation, and the d i f f i c u l t i e s 3 of sensing what they may not have experienced or of imagining what does not e x i s t . Shore (67) and Loeks (70) noted the unre l i a b l e nature of c i t i z e n input. Feagin (72), i n reporting a study of persons receiving rent supplements i n the Boston area, found that most re c i p i e n t s had only a vague idea of the pro-gramme. Few recognized such bureaucratic labels as "rent subsidization". Most were unclear as to which agency or l e v e l of government was responsible for the a i d and, i n turn, what t h e i r obligations were for t h i s assistance. Feagin*s findings would not appear s u r p r i s i n g to most public o f f i c i a l s . A study conducted by Sewell (71) on the attitudes of c i v i l servants toward consulting public opinion concluded that most o f f i c i a l s believed "the public not well informed and therefore cannot make r a t i o n a l judgments". In addition, p o l i c y makers continually feared that . c i t i z e n s who appeared able to a r t i c u l a t e a desired future p a r t i c i p a t e d on the basis of an e s s e n t i a l l y personal evalua-t i o n of costs and gains. Whether such motivations r e f l e c t e d personal vested i n t e r e s t s as opposed to any broader s o c i e t a l concerns i s a judgment planners have no methodology to assess (Kafoglis 70). When p a r t i c i p a t i o n did occur, i t was often seen by planners as d i f f i c u l t , d u l l , and time consuming. Time deadlines, p r i o r i t y demands, and complex procedures were thought to preclude the p o s s i b i l i t y of meaningful c i t i z e n p a r t i c i p a t i o n . 4 Reaction from the "Bottom Up" C i t i z e n p a r t i c i p a t i o n i n the decision-making process rests on the assumption that the ordinary c i t i z e n possesses the r i g h t to p a r t i c i p a t e i n the decisions which a f f e c t his l i f e . Evidence that Canadian c i t i z e n s increasingly desire to assume a r o l e i n the planning process i s mounting. C i t i z e n protests i n eastern Canada have success-f u l l y blocked several developments not considered to be i n the best i n t e r e s t s of the community. Marathon Realty's redevelop-ment of Marlborough Street, urban renewal of Trefann Court and the Spadina expressway are cases i n point. On the West coast, s i m i l a r examples e x i s t of community groups who organized i n an attempt to d i r e c t l y influence the decision-making process. C i t i z e n actions forced Marathon Realty to"convert a proposed shopping center into r e s i d e n t i a l uses. C i t i z e n s f o r s t a l l e d a freeway disrupting the Vancouver Chinese community, and forced reassessment of land uses adjacent to Stanley Park. The Strathcona Property Owners and Tenants Association were able to pursuade governments to revise urban renewal plans for the area. Demolition and rebuilding proposals were alt e r e d i n favour of neighbourhood preservation, renovation, and r e h a b i l i -t a t i o n p o l i c i e s (Pendakur 72). The examples c i t e d are but a few selected from a broad range of c i t i z e n actions. The pressures generating c i t i z e n involvement i n diverse circumstances appear to have certai n t r a i t s i n common. 5 1. A search for meaningful involvement: Improved standards of education have been accompanied by r i s i n g expec-tations and by a more c r i t i c a l questioning of society's actions. S o c i a l unrest, (while more overt i n the United States during the l a t e 1960's), manifested i t s e l f i n Canada as a movement to improve the p o s i t i o n of the average c i t i z e n who f e l t he had v i r t u a l l y no influence on national p o l i c y and possessed no information to enable him to make judgments. Redressing i n e q u a l i t i e s became a concern of the younger seg-ments of society. P a r t i c i p a t i o n was seen not only as a r i g h t but also as a r e s p o n s i b i l i t y of the concerned c i t i z e n . The r e s p o n s i b i l i t y became increasingly evident with the "rediscovery of poverty" by the Special Senate Committee on Poverty and pub-l i c a t i o n of The Real Poverty Report (Adams 71). 2. Rejection of a U n i f i e d Public Interest: The "melting pot" theory evolved i n the United States to f a c i l i t a t e the melding of diverse ethnic and r a c i a l groups in t o a u n i f i e d American people. In Canada, presumably due to the recognition of b i - c u l t u r a l French and English o r i g i n s , the melting pot p o l i c y was not a national p r i o r i t y . Nevertheless (possibly due to the shortage of Canadian planning theorists) American design concepts prevailed. I m p l i c i t i n c i t i z e n protests i s a challenge to the u n i f i e d public i n t e r e s t assumption. Wheaton (70) suggests that there i s no general enduring public i n t e r e s t about which concensual goals can be a r t i c u l a t e d i n any meaningful way. 6 Rather, the public i n t e r e s t should be regarded as a highly dynamic complex of various i n t e r e s t s . Competition between interests requires some form of bargaining as a means of r e c o n c i l i n g c o n f l i c t i n g goals. Our society has r e l i e d on the market process to control bargaining and resource a l l o c a -t i o n . One must assume that under the market system the interests of persons who lack e f f e c t i v e demand i n the market place w i l l remain under-represented. Rea l i z a t i o n of these concerns provided currency and a c c e p t a b i l i t y to the i n c l u s i o n of non-professional c i t i z e n s i n the planning process. 3. Concern for the pervasiveness of government actions; Requirements of constituents are enunciated as s o c i a l action 3 programmes. These programmes evolve from p o l i c i e s and Lead to p r a c t i c a l actions designed to meet stated requirements. Between 1960-1968 domestic programmes i n the United States increased from 45 to 435. At the inception of the Canadian Ministry of State for Urban A f f a i r s , Secretary H. P. Oberlander noted that, i n 1970, there existed 119 federal programmes i n 27 departments dealing with some aspect of urban concern. The bureaucracy necessary to cope with the vast array of programmes has increasingly separated c l i e n t from decision-maker. Separa-t i o n has led to a questioning of the decision-maker's awareness of people's problems and a search for new avenues of communi-cation. 4. Questioning the "experts'" bias; The assumption that the expert can act as a surrogate for the user i n defining needs 4 i s questioned by Turner (72), Sewell (71) and others who 7 found divergencies between solutions recommended by pro-fessionals and those i d e n t i f i e d by the pub l i c . Among the reasons for d i f f e r i n g viewpoints between professionals and c i t i z e n s are: the i n t e l l e c t u a l e l i t i s m of professional groups; idealism of the professional; the short range per-spective of most c i t i z e n groups; d i f f i c u l t i e s of communica-ti o n across s o c i a l class and education boundaries; and a lack of understanding of goals and r e s p o n s i b i l i t i e s . Administra-t i v e behaviour studies by Wheaton (70), Dye (72) and Gans (70) suggest that planners undertake t h e i r tasks not from the per-spective of some all - p e r v a s i v e s o c i e t a l i n t e r e s t but from the viewpoint of c e r t a i n e l i t e sectors of the community. Too often planners have merely r e f l e c t e d the narrow class bias and values c h a r a c t e r i s t i c of t h e i r s o c i e t a l o r i g i n s . They have placed great emphasis on long range values...the pre-servation of nature over the needs of man.... property values rather than human values. Within property values t h e i r c h i e f concern has been with property of those who are better o f f i n society rather than the property - or rather, lack of i t - of those who were less w e l l - o f f . (Wheaton, 70, p. 154). The recognition the.1:, planning which i s responsive to established bureaucracies and to the in t e r e s t s of the developer might i n the process forsake the preferences, needs, and desires of the consuming population i s evident i n studies of housing, recreation, education, and health care f a c i l i t i e s 5 . . undertaken by Gans, Dyckman and Meyerson. Other c r i t i c i s m s of public p o l i c y , from the perspective of the low income population, emerged i n urban renewal studies.** 8 C i t i z e n P a r t i c i p a t i o n ; Where We Are now Draper (71) has defined the term " c i t i z e n " to r e f e r to the desire one has to belong to and be a part of something. In e f f e c t c i t i z e n p a r t i c i p a t i o n can extend to " c i t i z e n power" (Arnstein 69). It i s the strategy by which the "have-nots" j o i n i n determining how information i s shared, goals and p o l i c i e s are set, and programmes are operated. Various iden-t i f i a b l e c i t i z e n i n t e r e s t s e x i s t . For example, the " c i t i z e n user" (also termed ultimate c l i e n t or programme recipient) refers s p e c i f i c a l l y to persons receiving assistance under a government i n i t i a t e d s o c i a l action programme. "User-need" has come to mean the data and information that replace or supplement the t r a d i t i o n a l owner's requirements (Ferguson 72). Models of C i t i z e n Involvement: The degree of c i t i z e n p a r t i c i p a t i o n i n the decision-making process has ranged from merely receiving information to f u l l p a r t i c i p a t i o n i n decision-making. Diagram 2 on the f o l -lowing page i l l u s t r a t e s the range of c i t i z e n involvement. Several models have been created to suggest how c i t i z e n involvement i n the planning process may most success-f u l l y be accomplished. In the late 1960's a body of l i t e r a t u r e and expertise developed which sought to derive the legitimacy of the planner from the preferences of ultimate consumers. Davidoff (65) enunciated proposals for advocacy planning through which planners could a s s i s t those whose i n t e r e s t might otherwise be underrepresented i n the planning process. 9 DIAGRAM MODELS OF CITIZEN PARTICIPATION Source: Canadian C o u n c i l of Resources and M i n i s t e r s . Proceedings of the Man Conference 1973. p.15. Environment and Resources NON-PARTICIPATION INCREASING DEGREE OF CITIZEN PARTICIPATION THE AREA OF AUTHORITY TO DECIDE VESTED IN POLITICAL, BURF*""» INFORMATION MODEL PERSUATION MODEL CONSULTATION MOOEL PARTERNERSHIP MODEL CITIZEN CONTRHL MODEL AN AUTHORITY MAKES A DECISION AND INFORMS THE PUBLIC ABOUT IT. AN AUTHORITY MAKES A DECISION THEN PERSUADES THE PUBLIC TO ACCEPT IT. AN AUTHORITY DEFINES THE PROBLEM, PRESETS IT TO THE PUBLIC INVITES COMMEN-TARY & SUGGESTIONS AND AN AUTHORITY PRESCIDES THE LIMITS AND WITHIN THESE LIMIT CITIZENS SHARE & MAY EVEN ASSUME THE THEN MAKES A DECISION. DECISION-MAKING RESPONSIBILITY. CITIZENS HAVE FULL RIGHTS TO PARTICIPATE IN AND ASSUME THE RESPONSIBILITY FOR DECISIONS. A l t e r n a t e d i r e c t i o n s are suggested by Friedmann's (73) theory of t r a n s a c t i v e planning and Schon's Beyond the  Stable State (71). Both authors s t r e s s the need f o r c o n t i n u a l l e a r n i n g as the ba s i s f o r p o l i c y d i r e c t i v e s . To Friedmann the fu t u r e i s a dimension of u n r e a l i z e d p o s s i b i l i t i e s , a non-homogeneous dimension open to change and cho i c e . How we cope wit h the present w i l l shape t h i s f u t u r e . The present presents 10 a c r i s i s of values and knowledge. The c r i s i s of values i s r e f l e c t e d i n the search for p a r t i c i p a n t forms of s o c i a l organization and i n the attempts to accept a d i v e r s i t y of views as a v a l i d foundation for s o c i a l planning. To accomplish these ends Friedmann f e e l s that the scope for autonomous group action must be enlarged by reducing the influence of bureaucratic organizations through a decentra-l i z a t i o n of power. There i s also an associated c r i s i s a r i s i n g from know-ledge gaps. In most instances, neither decision makers nor c i t i z e n s are aware of c o n f l i c t i n g future aspirations and of the possible consequences of alternate actions. The b a r r i e r s to e f f e c t i v e communication between those who have access", primarily to processed knowledge and those whose knowledge rests on p r a c t i c a l experience are becoming more evident. Mess-ages may be exchanged but meanings are often not e f f e c t i v e l y communicated. Goodman i n After the Planners i d e n t i f i e s a s i m i l i a r concern for communication d i f f i c u l t i e s between planner and c l i e n t . He makes a case for demystifying the planning prof-ession through a transfer of s k i l l s to the people. Thus the task of planning i s a l l e v i a t e d as some of the r e s p o n s i b i l i t y i s shouldered by the people for whom the planning occurs. The establishment of a more s a t i s f a c t o r y form of communication i s not simply a matter of t r a n s l a t i n g the abstract and highly symbolic language of the planner into the simpler and more experience-related vocabulary of the c l i e n t . The r e a l 11 solution involves a restructuring of the basic r e l a t i o n s h i p between planner and c l i e n t (Friedmann 73 p. 172). Friedmann builds his Theory of Transactive Plan-ning on the basis of a p a r t i c i p a n t society i n which continual learning and feedback of ideas occurs. His strategy includes reducing the separation between decision makers, planning units, and people through a s o c i a l process b u i l t on dialogue, innovation, and j o i n t problem solving. The strategy i s remarkably s i m i l a r to Et z i o n i ' s (6 8) "interwoven planning" and T r i s t ' s (70) d i r e c t i v e s for t a c k l i n g the challenges of the next t h i r t y years. Examples of C i t i z e n Involvement i n the Planning Process; There i s mounting evidence from a va r i e t y of d i s c i p l i n e s concerned with s o c i a l p o l i c i e s , of a trend toward increased c i t i z e n p a r t i c i p a t i o n i n the planning process. C i t i z e n involvement i s increasingly seen as a prerequisite to relevant p o l i c y making i n Architecture: Considering the challenges of the future, design can no longer remain the exclusive mandate of a single profession or group. The design of man-made environments...will require the involvement not only of the design professions but also management, planners, s o c i a l and physical s c i e n t i s t s , economists, and, increasingly con-sumer groups....(Weiss 73 p. 19). in S o c i a l Work: Casework i s most e f f e c t i v e when the focus i s kept on the problem as the c l i e n t understands i t (Mullen 72 p. 65). 12 i n P o l i c y Sciences: One of the most important influences on ser-vices may be the nature of the c l i e n t e l e , t h e i r numbers, the severity of t h e i r needs and t h e i r motivations. Personal welfare and the i n d i v i -dual needs and resources of taxpayers are becoming an increasingly more important i n -fluence upon government decision makers than sheer aggregate of economic resources. (Sharkansky 70 p.66). i n S o c i a l Policy Planning: The reconsideration of environmental programs depends upon increasing our understanding of the needs and p r i o r i t i e s of c l i e n t groups and c l e a r l y must involve the p a r t i c i p a t i o n of these groups. (Frieden 70 p. 2 86) . and i n Residential Planning: In the f i n a l analysis i t i s only the people who experience the a c t i v i t y and i t s products who can evaluate them. (Turner 73 p. 153). Goal Formulation Stage: During recent years there have been an increasing number of operational examples of c i t i z e n involvement at various stages i n the decision making process. Possibly the most frequent stage of c i t i z e n involvement has been i n goal formulation. Early examples occurred i n the 19 30's "U.S. Action for C i t i e s " plans, and through the use of simple p o l l i n g techniques during the 19 40's and 1950's. More recent attempts to involve c i t i z e n s i n the i d e n t i f i c a t i o n of public i n t e r e s t d i r e c t i o n s include the 1964 City of Chicago report i n which a series of regional meetings were held for planners to explain proposals and to receive comments for subsequent modification of basic p o l i c i e s . Between 1966-67 the C i t y of 13 Los Angeles sought c i t i z e n assistance i n the d e f i n i t i o n of i t s planning goals. In t h i s example a series of meetings was held involving representatives of various professional s o c i e t i e s whose r e s p o n s i b i l i t i e s included the "environment". These pro-f e s s i o n a l l y oriented representatives were presumed to possess a s p e c i a l knowledge of the public i n t e r e s t . A s i m i l a r pro-cess was i n s t i g a t e d by the C i t y of Vancouver during the summer of 19 73 i n an attempt to formulate p o l i c y d i r e c t i o n s for the redevelopment of the Inner-city False Creek Basin. Various c i t i z e n planning groups, such as the Com-munity Planning Association of Canada, have performed a r o l e i n providing governments with c r i t i q u e s of proposed l e g i s -l a t i o n from a v a r i e t y of c i t i z e n perspectives. Recent examples include b r i e f s by the Canadian Council on S o c i a l Development on the 1973 Amendments to the National Housing Act and a proposal on guaranteed annual income. Non-government agencies were responsible f o r i n i t i a t i n g the 1966-67 Goals for Dallas programme. Various techniques for evoking expressions of public preferences were attempted including s o l i c i t e d essays on the problems facing Dallas, conferences of representative c i t i z e n s to formulate goals, and d i s t r i b u t i o n of printed l i t e r a t u r e to neighbour-hoods. In Canada, the "Halifax Encounter" was a s i m i l a r attempt to i d e n t i f y c i t i z e n concerns and to evoke discussion on future urban d i r e c t i o n s . In the Greater Vancouver Regional D i s t r i c t planners are attempting to formulate urban goals based upon discussions 14 with a v a r i e t y of community i n t e r e s t groups concerned and indiv i d u a l s outside the p o l i t i c a l arena. During 1972-73 meetings with some f o r t y community groups i n i t i a t e d the pro-cess of devising a "Livable Region Programme". Discussions on "What makes for l i v a b i l i t y ? " , "What works against L i v a b i -l i t y ? " , "What are the important things GVRD should be con-cerned with i f i t i s to be responsible for managing the growth of the region?" indicated c i t i z e n concerns for r e s t r i c t i n g further rapid growth i n the region, a desire to p a r t i c i p a t e i n community decisions, and a wish to see action. Proposals concerning transportation, industry, land costs, housing, community l i f e and services, p o l l u t i o n and the preservation of natural areas were further discussed during the summer of 1973 by p o l i c y committees comprised of interested c i t i z e n s . The use of consultant groups to i d e n t i f y public i n t e r e s t and preferred d i r e c t i o n s was attempted i n the Miami Valley, Ohio. In t h i s experiment the firm of Gruen and Gruen was retained to determine how best to integrate low and moderate income housing into otherwise middle-income neigh-bourhoods. In analyzing the needs and desires of both lower and middle income households the Gruens found that d i r e c t i v e s were successfully obtained from a variety of c i t i z e n groups. The analysis and discussion of tradeoffs between the two groups indicated that there were indeed common bargaining points. Their study appears to j u s t i f y the involvement of 15 c i t i z e n s i n p o l i c y formulation as a method of f a c i l i t a t i n g the long run minimization of s o c i a l costs. Policy Implementation Stage: Examples of c i t i z e n involvement i n the actual implementation of decisions are fewer. Several c i t i e s have moved to decentralized government i n an attempt to place the center of authority, decision making, and implementation as close to the ultimate user as possible. In Vancouver, Winnipeg and Toronto, for example, i n n e r - c i t y residents have been active i n upgrading t h e i r environments through c i t i z e n i n i t i a t e d programmes of renovation and i n f i l l i n g . Axworthy's (72) research on c i t i z e n groups i n Winnipeg indicates that c i t i z e n s are able to a r t i c u l a t e t h e i r concerns and to l i s t p r i o r i t i e s . C i t i z e n p a r t i c i p a t i o n trends were strengthened i n 1973 with the i n c l u s i o n of amendments to the National Housing Act which require c i t i z e n involvement i n Neighbour-hood Improvement Programmes. Among the problems emerging from c i t i z e n p a r t i c i p a -t i o n i n goal formulation and implementation i s the yet to be resolved question of who represents the people. Two claims emerge. Elected o f f i c i a l s lay claim to decision-making authority on the basis of t r a d i t i o n and law. C i t i z e n s claim authority as consumers (Arnstein 69). A prolonged debate on authority i s beyond the scope of t h i s d i s s e r t a t i o n . The author accepts the stance that programme users, c i t i z e n s affected by programme actions, and bureaucrats entrusted with 16 programme implementation, are among the interested p a r t i e s whose values and viewpoints should be considered by elected p o l i t i c i a n s . The elected o f f i c i a l s are the ultimate decision-makers. Evaluation Stage: As larger proportions of public funds are committed to s o c i a l programmes, agencies engaged i n supplying these community services are increasingly required to provide evidence of programme effectiveness as a basis for continued public support. Too frequently the s o c i a l and economic impact of programmes which cost the public m i l l i o n s of d o l l a r s were not subjected to c r i t i c a l a n a l y s i s . When evaluation of programmes dealing with the pro-v i s i o n of human resource development did occur tentative findings often suggested that programmes were having minimum impact on problems. The question of "why", the r e a l i z a t i o n of the increased complexity of problems, the expanded s o c i a l distance between p o l i c y maker and c i t i z e n , and the need for accountability encouraged government agencies to i n i t i a t e evaluative research programmes. In t h i s sense evaluation research i s considered as research i n which the s c i e n t i f i c method i s applied to analysis of public programmes to learn what happens as a r e s u l t of programme a c t i v i t i e s . Evaluation includes: the i d e n t i f i c a t i o n of programme objectives; the development of measures toward these objectives; the assessment of what 17 differences public programmes a c t u a l l y make; and the pro-j e c t i o n of what differences could be expected i f the pro-gramme was continued or expanded (Wholey 70). To date evaluation of programme impact has been undertaken by professional planners. Neither programme recipient s nor c i t i z e n s p o t e n t i a l l y affected by programme actions are r e g u l a r l y included i n the evaluation stage of the planning process. Mullen, i n discussing t h i r t e e n evaluative studies by s o c i a l workers, concluded: Although the c l i e n t ' s views were part of the evaluation, d i r e c t measurement of c l i e n t attitudes was the exception rather than the r u l e . This stance i s rather s u r p r i s i n g since p r a c t i s e has become increasingly c l i e n t oriented. (Mullen 72, p. 34). The reluctance of some projects to base evaluation on the c l i e n t ' s attitudes about changes r e s u l t i n g from the programme does not necessarily point to conservative research methods. There i s a general lack of agreement between c l i e n t s and p r a c t i t i o n e r s i n assessing treatment outcome. Thus i investigators facing various methodological problems may, Mullen suggests, have decided to resolve t h i s one by elimina-t i n g the c l i e n t as a key indicant of evaluation. Evaluation of Social Action Programmes Limitations of E x i s t i n g Evaluation Methodologies: Evaluation processes, evolved during the past ten years, have attempted to apply methodology developed i n the controlled experimental s i t u a t i o n of the laboratory to the 18 assessment of s o c i a l action programmes. Results have not 7 been impressive. The question arises as to whether evaluation of government programmes i s , i n i t s e l f , unattainable or whether exi s t i n g evaluation processes merely apply inappropriate methodologies to s o c i a l s i t u a t i o n s . Consideration of t r a d i -t i o n a l evaluation methods i n l i g h t of evolving planning thought suggests the l a t t e r condition may well apply. T r a d i t i o n a l programme evaluation models assume s t a t i c programmes i n which i n i t i a l programme goals are a v a i l -able, c l e a r and remain v a l i d over time. T r a d i t i o n a l models seldom d i r e c t l y assess user attitudes to programme dimensions and make no provision for c i t i z e n p a r t i c i p a t i o n i n the evaluative process. In most instances, programme administra-tors provide information on user need and programme impact. Unfortunately, representatives of the sponsoring agency frequen-t l y understate s o c i a l costs incurred by the programme r e c i p i e n t . E x i s t i n g evaluation models appear inconsistent with emer-ging planning theory. Planners are c a l l i n g for new processes to provide f l e x i b l e , short range planning which i s responsive to varied user needs and anticipates s o c i a l costs. However, when evaluation i s requested, the researcher presents an expert-g centered methodology requiring s t a t i c programmes and summative analysis of quantifiable programme patterns. This t r a d i t i o n a l response to a highly complex process seems inadequate par-t i c u l a r l y from the standpoint of user needs and involvement 19 of the c i t i z e n i n the evaluation stage of the planning process. The Case for a User Oriented Evaluation Methodology: This d i s s e r t a t i o n accepts the assumption that the adequate assessment of e x i s t i n g and inno-vative programmes can be a v i t a l force i n d i r e c t i n g s o c i a l change and improving the l i v e s and environments of community members. (Caro 71 p.1). While i t i s c l e a r that early o p t i m i s t i c predictions of extensive s c i e n t i f i c s o c i a l programme evaluation have f a i l e d to materialize, the concern f o r the development of a methodology to evaluate the impact of s o c i a l action programmes remains. Emerging government p o l i c i e s suggest no cut back i n s o c i a l programmes. Acceptance of the status quo r a i s e s the spectre of continuing i n e f f e c t i v e programme actions. The creation of a more e f f e c t i v e mechanism for evaluating s o c i a l action programmes i s the task posed by t h i s d i s s e r t a t i o n . The search for alternate evaluation methodologies led the author to speculate on the implications of including members of the concerned population i n the evaluation process. T h e o r e t i c a l l y , user involvement i n the evaluation process i s consistent with emerging planning theory. Inclusion of progra-mme r e c i p i e n t s should increase the l i k l i h o o d of evaluation r e f l e c t i n g dimensions s e n s i t i v e to programme operation and of future plans emerging i n response to s o c i a l needs. 20 Dissertation Proposal Programme evaluation attempts to determine the extent to which objectives are being f u l f i l l e d , the p r i n c i p l e s or theories underlying a programme which account for success or f a i l u r e , whether i d e n t i f i e d changes are due to programme measures or extraneous factors and to determine the implica-tions of changes. In e f f e c t , evaluation i s a learning pro-cess - learning why programmes succeed or f a i l and which attri b u t e s of the programme are most se n s i t i v e to change. T r a d i t i o n a l evaluation seeks to provide t h i s knowledge through an i d e n t i f i c a t i o n by programme administrators of goals and subsequent analysis of the extent to which goals are r e a l i z e d . The basis upon which change i s measured i s established by po l i c y planners not programme users. This d i s s e r t a t i o n proposes an alternate evaluative system i n which programme users i d e n t i f y desired goals and a s s i s t i n determining the extent to which programme output meets user expectations. The i d e n t i f i c a t i o n of p o s s i b i l i t i e s for user oriented evaluation i s not unique to t h i s d i s s e r t a t i o n . Several authors have noted the p o t e n t i a l advantage of evalua-t i n g programmes from the perspective of the user. Caro (71) feels f u l l appreciation of programme impact can only be accomplished through c l i e n t i n t e r p r e t a t i o n s . Caputo (73), Carter and Wharf (73) see user p a r t i c i p a t i o n as l e g i t i m i z i n g evaluative findings. Unfortunately, these authors f a i l to operationalize t h e i r concern by including user p a r t i c i p a t i o n at a l l stages i n the evaluative process. 21 Testing a User Oriented Evaluation Methodology: To a s s i s t i n es t a b l i s h i n g guidelines for a user-oriented evaluation process i t appears necessary to t e s t the proposition that the demands and expectations of need per-ceived by the ultimate c l i e n t can form an alternate and/or complementary perspective on s o c i a l p o l i c y d i r e c t i o n s . To t e s t t h i s proposition two evaluations w i l l be conducted on the same programme. One evaluation w i l l be undertaken following processes currently employed by govern-ment evaluative personnel. The second evaluation w i l l include programme rec i p i e n t s at f i v e stages i n the evaluative process. S p e c i f i c a l l y , the user w i l l be involved i n i d e n t i f i c a t i o n of evaluation goals, construction of an evaluative research design, data c o l l e c t i o n and analysis and in-programme re-formulation. The assumptions, p r i o r i t i e s and solutions of housing experts (persons responsible for s e t t i n g p o l i c y , administer-ing and evaluating programmes at federal and l o c a l levels) obtained through the t r a d i t i o n a l evaluation process w i l l be compared with those of c i t i z e n s receiving housing assistance. I f , on analysis, i t appears p o l i c y guidelines obtained from experts c l o s e l y r e f l e c t the c l i e n t ' s demand set the need for a user oriented evaluation w i l l be rejected. Rejection rests on the r e a l i z a t i o n that, while the viewpoint of the user may be of i n t e r e s t , i f the expert can adequately function as a surrogate to i d e n t i f y relevant variables the added expense in time and money of including c i t i z e n s i n the evaluation 22 process i s not warranted. I f , however, t r a d i t i o n a l programme evaluation assumptions and output do not r e f l e c t c l i e n t con-cerns the d i s s e r t a t i o n proposition w i l l appear substantiated. The recommendation w i l l be made that the user or r e c i p i e n t of a government programme should be an i n t e g r a l component i n any planning process which seeks to tackle programme evalua-t i o n . The t e s t s i t u a t i o n : Selection of a t e s t s i t u a t i o n for the proposed user oriented evaluation process was based upon several require-ments. F i r s t l y , a s i t u a t i o n r e f l e c t i n g a t y p i c a l planning problem was needed. Selection of an e x i s t i n g s o c i a l action programme provided a set t i n g upon which both planners and c i t i z e n s could focus. Secondly, the programme had to be one which involved a value laden context. In such circumstances i t would be reasonable to assume that an evaluation from the perspective of agency goals alone might y i e l d conclusions t o t a l l y inappropriate from the viewpoint of the user. Thirdly, the programme must r e f l e c t emerging p o l i c y d i r e c t i o n s . Evaluation of programme actions would then appear as a r e a l i s t i c exercise to programme administrators and users a l i k e . Fourthly, the programme should meet the p r a c t i c a l requirements of being accessible to the researcher. An Assisted Home Ownership programme being imple-mented by the Greater Vancouver Regional D i s t r i c t Housing Department appeared to f i t a l l requirements. An analysis i n 23 the context of housing leads one into an evolving, multi-dimensional issue which explores a cross section of society's s o c i a l , economic, and p o l i t i c a l a f f a i r s . I t i s a cen t r a l point i n the l i v e s of people representing shelter, a focus of s o c i a l l i f e , and the privacy of the i n d i v i d u a l household. Housing i s the largest single space user i n urban areas, thus e f f e c t i n g impacts on the o v e r a l l urban system. Housing i s consistently i d e n t i f i e d as a major urban problem linked with poverty, urban redevelopment, suburban sprawl, s o c i a l unrest, and transportation. H i s t o r i c a l l y , housing has been a focus for s o c i a l planners and reformers seeking a p o l i c y instrument open to governments wishing to promote s o c i a l and economic programmes. Contained within the Assisted Home Ownership Pro-gramme are several emerging Canadian housing p o l i c y d i r e c t i o n s - s p e c i f i c a l l y the move toward home ownership for low-income households and the attempt to disperse low income housing through-out the urban area. Data on s i m i l i a r programmes i n the United States and elsewhere i n Canada are ava i l a b l e for comparative purposes. In addition, home ownership programmes have attracted the i n t e r e s t of housing t h e o r i s t s . E x i s t i n g theory i s , however, formulated on the basis of research i n the United States and needs r e - d e f i n i t i o n i n the Canadian context. The Greater Vancouver Regional D i s t r i c t study area i s a t y p i c a l Canadian urban s e t t i n g . The region supports a population of one m i l l i o n people. Currently, Vancouver i s 24 confronted with problems of r i s i n g housing costs. The causes of problems are s i m i l a r to those experienced i n other large centers. Scarcity of serviced land, resistance to new development by l o c a l governments, and continued in-migration are among the pressures facing p o l i c y makers. Recent c i t i z e n reactions to planning proposals suggest that Vancouver residents are aware of c i t i z e n p a r t i c i p a t i o n aims and actions. Toward an Appreciation of Limitations: The proposed study has several l i m i t a t i o n s . Re-sources placed a constraint on the s e l e c t i o n of l o c a l e and number of programmes considered. However, the analysis, i n depth, of one programme i s j u s t i f i e d on the ground that the focus of the study i s to devise an evaluation framework. Evaluation of one programme provides the opportunity to compare t r a d i t i o n a l and user oriented evaluation processes. It i s r e a l i z e d that as one moves from a purely t h e o r e t i c a l study to an evaluative study the number of variables over which the researcher has control decreases. The i n a b i l i t y of the author to hold constant a considerable portion of the environment and to manipulate only a few variables may appear as a r e j e c t i o n of t r a d i t i o n a l academic research methods. However, as Schon states: In s i t u a t i o n s of public learning i t i s almost never possible to hold some variables constant while manipulating others. Any intervention a f f e c t s more than one v a r i a b l e . (Schon 71 p. 214). 25 the r e l a t i v e funding contributions necessary from various l e v e l s of government i s not considered i n d e t a i l . Since the concern i s to provide an analysis from the user's perspective i t was considered that analysis of intergovernmental appropriat-ions added another inappropriate dimension to an already complex evaluation. The absence of a c l e a r l y defined hypothesis might be considered as a l i m i t a t i o n . T r a d i t i o n a l l y , research involves the disaggregation of a problem to i d e n t i f y one dimension upon which to formulate a s p e c i f i c hypothesis. Controlled experi-ments and the a p p l i c a t i o n of s c i e n t i f i c a l l y c o l l e c t e d 'tough minded' data lead l o g i c a l l y to hypothesis t e s t i n g and theor-e t i c a l answers to p a r t i c u l a r problems. In contrast, t h i s d i s s e r t a t i o n i d e n t i f i e s a multi-dimensional issue, i n t e r a c t i n g i n a r e a l world s i t u a t i o n . The search i s for an operational framework for an analysis based upon both 'tough' and 'tender' minded data. Proofs, such as they are, evolve through immersion i n the complexity of the issue and i n the r e a l i z a t i o n of the r e s p o n s i b i l i t y implied by recommendations for s o c i a l planning and p o l i c y analysis. Conclusions at best i d e n t i f y emerging alternate d i r e c t i o n s . This chapter has outlined the author's concern for the p r o l i f e r a t i o n of s o c i a l action programmes which lack a process for including c i t i z e n p a r t i c i p a t i o n i n planning and evaluation. Realizing t h i s concern, the d i s s e r t a t i o n w i l l attempt to evolve very tentative guidelines i n d i c a t i n g the value of incorporating user input into programme evaluation. 26 Chapter II outlines i n more d e t a i l the expanding s o c i a l consciousness of governments as indicated by the growth of s o c i a l action programmes. E x i s t i n g evaluation methodologies provide the basis for the i d e n t i f i c a t i o n of a t r a d i t i o n a l pro-gramme evaluation model. Chapter III suggests an alternate user-oriented evaluation model. A case study comparison of t r a d i t i o n a l and user oriented models follows i n Chapter IV. Concluding chapters present guidelines for the i n c l u s i o n of user p a r t i c i p a t i o n i n the evaluation stage of the planning process. CHAPTER II SOCIAL CONSCIOUSNESS IN A GOVERNMENT POLICY SETTING This d i s s e r t a t i o n evolves from three assumptions. The f i r s t i s that governments are e x h i b i t i n g an increasing concern for the s o c i a l welfare of t h e i r constituents. This concern i s r e f l e c t e d i n the expanded number of, and resources committed to, s o c i a l action programmes. The second i s that c i t i z e n s are becoming increasingly desirous of p a r t i c i p a t i o n i n the formulation of s o c i a l p o l i c i e s designed to provide f o r t h e i r welfare. And, f i n a l l y , as s o c i e t a l goals change, p o l i c i e s must be altered and programmes reformulated to r e f l e c t emerging d i r e c t i o n s . Evaluation of e x i s t i n g s o c i a l action programmes may provide decision-makers with an i n d i c a t i o n of the extent to which government actions a s s i s t i n d i v i d u a l s to share the q u a l i t y of l i f e enjoyed by society at large. Chapter I documented the involvement of c i t i z e n s i n the planning process. Chapter II w i l l deal with the increase i n s o c i a l action programmes - using as a s p e c i f i c example the evolution of Canadian Federal Housing p o l i c i e s . L i t e r a t u r e reviews of these, and other programmes, indicate the pre-v a i l i n g state-of-the-art of p o l i c y analysis. This chapter concludes with the assertion that t r a d i t i o n a l processes are inadequate for the evaluation of programmes both from the perspective of the user's needs and from the standpoint of user involvement i n t h i s aspect of the planning proccess. An 28 alternate process i s proposed and tested i n subsequent chapters. Programme P r o l i f e r a t i o n Under an Increasing S o c i a l Consciousness The necessity to develop a methodology for evaluation of s o c i a l action programmes rests on the assumption that the state i s increasingly assuming the r e s p o n s i b i l i t y of providing for the s o c i a l welfare of i t ' s c i t i z e n s . This d i s s e r t a t i o n accepts the widely held assumption that one of the forces which has most changed our l i v e s since the 19 30's has been the expansion of the federal government and federal power. The extent to which the state now concerns i t s e l f with and i s an agency for change would have been inconceivable i n 1930. (Graubard 67 p. 712). Programme P r o l i f e r a t i o n i n the Context of Canadian Housing  Policy '• ; The increasing involvement of the federal government can be seen i n the f i e l d of housing. P o t e n t i a l l y , housing actions provide an a t t r a c t i v e p o l i c y instrument. The housing sector presents a va r i e t y of levers for control of the economy For example: rent control measures dampen down demands for wage increases; b u i l d i n g subsidies encourage investment and employment; rent increases r e s t r a i n consumer demand and reduc-tions or increases i n subsidies s h i f t investment between the private and public sectors. In a s o c i a l context, housing pro-vides a means of r e d i s t r i b u t i n g funds to disadvantaged sectors of society. i 29 The Canadian government's concern for the provision of housing i s r e f l e c t e d i n p o l i c y statements contained i n 9 House of Commons debates. If the Dominion, the provinces and the munici-p a l i t i e s can do anything i n these troubled times to put people i n better l i v i n g conditions, they w i l l be doing a wonderful work to make them happy and prosperous. (Carvell, Minister of Public Works, Feb. 25, 1919 p. 45). It i s the aim of t h i s government to provide good housing at reasonable cost as a s o c i a l goal and government o b l i g a t i o n . (Basford, Minister of State for Urban A f f a i r s , March 15, 1973 p. 2257). The r o l e of the Federal government i n the housing sector has expanded progressively since the introduction of the f i r s t statute i n 1919. The government o r i g i n a l l y entered the f i e l d of housing i n 1919 when shortages occasioned by World War I encouraged federal a u t h o r i t i e s to make money available to the provinces for house c o n s t r u c t i o n . ^ The f i r s t general piece of federal housing l e g i s l a t i o n was the Dominion Housing Act passed i n 1935 to a l l e v i a t e unemployment conditions. This was followed by the National Housing Acts of 1938 and 1944, culminating, i n 1954, with the National Housing Act (NHA), defined as: An act to promote the construction of new houses, the repair and modernization of e x i s t i n g houses and the improvement of housing and l i v i n g conditions. Central Mortgage and Housing Corporation (CMHC), a Crown agency incorporated by Act of Parliament i n 1945, administers the National Housing Act and coordinates the a c t i v i t i e s of the Federal Government i n Housing. The 30 Corporation has the r e s p o n s i b i l i t y for a variety of functions a f f e c t i n g housing i n i t s long-range outlook as well as i n i t s immediate requirements. CMHC i s empowered to act as an insurer of mortgage loans, as a lender or investor of public funds, as a guarantor, and as an owner of property and other assets. Programme options at present include insured mortgage loans on new and e x i s t i n g housing, d i r e c t loans to r u r a l areas, loans for home improvement, subsidized r e n t a l accommodation, subsidized home ownership, land assembly, sewerage treatment, new communities, renewal, neighbourhood improvement and assistance to sp e c i a l i n t e r e s t groups such as native peoples, students, e l d e r l y and cooperative organizations. The Corporation acts as a research agency i n f i e l d s associated with housing and assists provinces and mu n i c i p a l i t i e s i n many aspects of urban growth. Housing as an Economic Tool Despite the vari e t y of intervention options a v a i l -able, and the p o t e n t i a l attractiveness of housing as a p o l i c y t o o l , implementation of l e g i s l a t i o n p r i o r to the mid 1960's was constrained by p r e v a i l i n g a t t i t udes. Federal actions were r e s t r i c t e d to the provision of housing programmes designed to regulate the economy. I n i t i a l l y the Federal Government showed a reluctance to assume r e s p o n s i b i l i t i e s c o n s t i t u t i o n a l l y a l l o c a t e d to the 12 provinces or which, i n a market economy, could be borne by 31 13 private enterprise. These constraints meant that federal programmes placed the onus on private industry to construct units and required p r o v i n c i a l cost sharing and l o c a l i n i t i a -t i v e p r i o r to implementation. The government, through successive Housing Acts, attempted to stimulate and supplement the market for housing by increasing the flow of mortgage money and encouraging lenders to make loans on more favour-14 able terms to prospective home-owners. Provision of housing was seen to be the private r e s p o n s i b i l i t y of the i n d i v i d u a l household. Where housing assistance was made available to lower-income f a m i l i e s , i t was viewed as a temporary haven while t h e i r finances were reorgan-ized for another ass u l t on the market. This concept assumes that the poor have i t within t h e i r power to correct whatever conditions led them into poverty; a view currently shared by few (Adams 71) . The Emergence of Housing as a So c i a l Concern By 1968 a r e d i r e c t i o n of federal actions appeared necessary. It had become evident that e x i s t i n g federal p o l i c i e s had f a i l e d to a l l e v i a t e housing associated problems. During the 1960's the price of shelter rose f a s t e r than any other component of the Consumer Price Index. The National Housing Act was serving upper-middle-income people; giving them the protection of pegged i n t e r e s t rates and providing serviced land assembly projects. L i t t l e was being done for the growing numbers of people experiencing d i f f i c u l t y finding 32 accommodation. E x i s t i n g assumptions of stereotyped consumer expectations and of the a b i l i t y of the private market to f u l -f i l l a d i v e r s i t y of needs were questioned. A v a i l a b i l i t y of units and p a r t i c i p a t i o n i n the provision of housing by the 15 consumer became, at l e a s t i n r h e t o r i c , a RIGHT. Coupled with housing costs was an increasing aware-ness of the interrelatedness of urban problems. A v a r i e t y of reports published between 1967 and 1973 indicated the extent of urban housing problems. For example, the 1967 Economic Council of Canada Review i d e n t i f i e d some of the emerging issues. In 1968 a Federal-Provincial conference on Housing and the Federal Task Force on Housing and Urban Development, led by Paul Hellyer, formulated recommendations concerning (1) dual roles of private and public sectors i n housing; (2) financing changes to make mortgages avail a b l e to more consumers; (3) land cost regulations; (4) s o c i a l housing; and (5) urban management programmes. In 197o, Lithwick, i n a series of studies, i d e n t i f i e d Urban Canada: Problems and P o l i c i e s . The urban processes at work and the interrelatedness of urban issues led to recommendations concerning alternate urban futures. Probably the most c r i t i c a l report on federal actions was issued i n 19 72 by Dennis and F i s h . They analyzed and c r i t i c i z e d the Federal Housing programmes for low-income households. The Charney report followed with an analysis of the production and adequacy of housing produced for low-income f a m i l i e s . 33 Federal l e g i s l a t i o n responded with provisions to low-income groups which r e f l e c t e d a more p o s i t i v e apprecia-ti o n of demand. Public aid was transferred from the support of the private middle-income housing market to the d i r e c t assistance of low-income groups. Ottawa thereby embarked on a s o c i a l housing p o l i c y whereby the State was prepared to intervene on a s e l e c t i v e and l i m i t e d scale to a s s i s t lower-income households. For the purposes of housing assistance, lower-income families were defined as a family that receives a t o t a l family income that, i n the opinion of the Corporation, i s i n s u f f i c i e n t to permit i t to rent housing accommodation adequate for i t s need at the current r e n t a l market i n the area i n which the family l i v e s . (N.H.A. 73, p. 3). U t i l i z i n g the v a r i e t y of programmes and funds available as indicators of the i n t e n s i t y with which pro-grammes are being pursued, i t i s evident that new p o l i c y d i r e c t i o n s are emerging. In 1968, approval was given to d i r e c t loans involving 15,221 dwelling units. For the year 1970, these figures were 43,494 s e l f contained units and 20,263 hostel units. The t o t a l d i r e c t federal investments for these two years were $257 m i l l i o n and $596 m i l l i o n r espectively. Further, out of the i n i t i a l housing budget of $854 m i l l i o n , l a t e r r a i sed to a record high of $1,094 m i l l i o n , two thirds was directed towards aid to low-income groups."^ By 1973 close to h a l f of the country's housing stock of approximately 6,300,000 housing units had been constructed since the f i r s t covering l e g i s l a t i o n was enacted. 34 About one-third of these were financed i n one way or another under the National Housing Acts. Recent amendments to the NHA suggest the Federal Government i s moving from an emphasis on s t a r t s toward a mix of solutions to improve the q u a l i t y of 17 the t o t a l r e s i d e n t i a l environment. New p o l i c i e s are increasingly s e n s i t i v e to l o c a l and regional differences and to the need for choice i n tenure, l o c a t i o n and financing arrangements.^ The assumption concerning the increasing s o c i a l consciousness of governments, as indicated by funds committed and programmes enacted i n the housing sector, appears j u s t i -f i e d . As larger portions of public funds have been committed to s o c i a l programmes, governments engaged i n supplying these community services are increasingly required to provide e v i -dence of programme effectiveness as a basis for continued public support. However, when evaluation of programmes dealing with the provision of assistance for human resource develop-ment has occurred, tentative findings often suggest that pro-grammes are having minimum impact on problems. In response to questions of 'why', the r e a l i z a t i o n of the increased complexity of problems, and requirements for accountability, government agencies were encouraged to i n i t i a t e evaluative research programmes. Programme Evaluation as an Indicator of S o c i e t a l Directions The Nature of Evaluative Research The range of forces encouraging the development of evaluative research - changes i n the nature of s o c i a l problems; 35 changes i n the structure and function of public agencies; and changes i n the needs and expectations of the public - have resulted i n evaluative studies being undertaken for a v a r i e t y 19 of reasons, u t i l i z i n g various methodologies, on a wide range of programmes. The very nature of evaluative research, i t s function and d e f i n i t i o n s , r e f l e c t t h i s complexity. Diverse Purposes: E x i s t i n g evaluative research has been c l a s s i f i e d i n numerous ways. One c l a s s i f i c a t i o n i d e n t i f i e s the stage i n the planning process at which evaluation i s undertaken. P r i o r p o l i c y evaluation i s concerned with deter-mining the magnitude of need and the most suitable d i s t r i b u -t i o n of resources within a given programme, or between alternate programmes, to reach c e r t a i n stated objectives. Frequently i n p r i o r p o l i c y evaluation, experimental or demonstration projects are i n i t i a t e d and evaluated to ascertain whether proposed programmes adequately f u l f i l l stated needs. Evaluations of e x i s t i n g p o l i c y consider e f f e c t , e f f o r t and/or e f f i c i e n c y of programmes. Analysis has been undertaken during programme operation (formative evaluation) and at the conclusion or discontinuation of programme actions (summative evaluation). E f f e c t studies focus on goal f u l f i l l m e n t . Pro-grammes are evaluated with a view to determining the o v e r a l l effectiveness of a p a r t i c u l a r programme i n reaching stated objectives or of the r e l a t i v e effectiveness of alternate pro-grammes with a view to recommending future strategy d i r e c t i o n s . Riecken maintains that studies of e f f e c t s represent the maximum con-t r i b u t i o n that s o c i a l science can make to s o c i a l p ractice, since they are usually i n -tended to feedback r e s u l t s into program planning or p o l i c y making. (Riecken 72, p. 87). Programme e f f i c i e n c y studies attempt to assess the r e l a t i v e costs of. achieving the objectives set for a pro-gramme. Unfortunately, there are few successful applications to s o c i a l action programmes. Analysis of e f f o r t or programme accountability i s concerned with the kind and quantity of a c t i v i t i e s engaged i n to s a t i s f y programme objectives. T y p i c a l l y , programme input (funds) forms the basis of evaluation against output ( i . e . number of c l i e n t s served). Rossi's (69) review of evaluative studies undertaken for the U.S. O f f i c e of Economic Opportu-n i t y shows " e f f o r t studies" to be the most customary form of evaluation. In a sample of 200 out of 1,12 3 research pro-jects contracted by OEO, Rossi found that 170 (85 percent) were mainly descriptive accounts of how many people were being reached. These reports were presented p r i m a r i l y i n narrative form with " v i r t u a l l y no systematic observations on the effectiveness of programs" (Rossi 69 p. 218). This kind of assessment appears to be extremely l i m i t e d and generally lacking i n u t i l i t y unless accompanied by information on effectiveness and e f f i c i e n c y . 37 Operations analysis attempts to consider the means or operation of programme processes without s p e c i f i c attention to ends. Weiss and Rein (72), for example, suggest that studying the implementation process for large scale, m u l t i -dimensional programmes may be of more value than attempting to i s o l a t e s p e c i f i c variables. The a n a l y t i c a l techniques applied to evaluative research show no less d i v e r s i t y . A schema devised by T r i p o d i , F e l l i n and Meyer (71) groups the v a r i e t y of methodologies which have proven useful for d i f f e r e n t types of evaluation i n t o : 1. Monitoring techniques, including procedures used for the d i r e c t review of programme operations; accounta-b i l i t y audit, administrative audit, and time-motion studies. 2. S o c i a l Research Techniques, r e f e r r i n g to procedures which exclude cost considerations and which are used for developing, modifying and expanding knowledge about the programme which can be communicated and v e r i f i e d by independent in v e s t i g a t o r s : experiment, survey and case study. 3. Cost a n a l y t i c techniques, r e f e r r i n g to procedures used to appraise the r e l a t i v e value of a programme i n r e l a t i o n to programme costs: cost accounting, cost-benefit analysis, cost-outcome analysis, and operational research which binds experimental and cost-analytic methods. The d i v e r s i t y of evaluative objectives and method-ologies i s indicated by t h i s overview. Generally, the rationale behind such research has been to provide a base for improved decision-making. However, as Weiss (72) points out, i n practice evaluation i s sometimes undertaken for less noble motives: 38 Program decision-makers may turn to evalua-t i o n to delay a decision; to j u s t i f y and legitimate a decision already made; to e x t r i c a t e themselves from controversy about future directions by passing the buck; to vindicate the program i n the eyes of i t s constituents, i t s funders or the public; to s a t i s f y conditions of a government or founda-t i o n grant through the r i t u a l of evaluation. (Weiss 72, p. 14). Diverse D e f i n i t i o n s : Evaluative research covers a wide range of a c t i -v i t i e s involving the assessment of s o c i a l actions. That there exists no single accepted d e f i n i t i o n of the concept i s not s u r p r i s i n g . Rather, a v a r i e t y of meanings have been assigned to the term revealing an i n e x t r i c a b l e mixture of conceptual and operational d e f i n i t i o n s with the greater emphasis on the l a t t e r . Examination of j u s t a few of the many d e f i n i t i o n s shows that while they include some s i m i l a r i t i e s , there are divergencies both on points of scope and methodology. Wholey (70), Tripodi (71), and Freeman and Sherwood (70) view evaluation i n a somewhat narrow context. They see evaluation as focussing on s p e c i f i c programmes, the i d e n t i f i -cation of programme goals, and subsequent magnitude of change. Freeman and Sherwood, for example, see evaluation as: e s s e n t i a l l y a procedure for control i n q u a l i t y and quantity. A mechanism for determining whether the actual conduct of programs i s con-s i s t e n t with the intervention strategy out-l i n e d i n the development of the program. (Freeman and Sherwood 70, p. 70). A broader d e f i n i t i o n of evaluation i s seen i n the works of, for example, Jenkins (61), A l k i n (72) and Weiss (72). 39 To these authors, programme evaluation attempts to determine the extent to which objectives are being f u l f i l l e d , the p r i n c i p l e s or theories underlying the programme which account for success or f a i l u r e , whether i d e n t i f i e d changes are due to programme measures or extraneous factors and to determine the implications of such changes. In e f f e c t , evaluation i s a learning process - learning why programmes succeed or f a i l . Jenkins defines evaluation operationally as: the process of acquiring, analyzing and using information for decisions associated with planning, programming, implementing and re-c y c l i n g program components and a c t i v i t i e s . (Jenkins 61, p. 20 ) . As the scope of evaluative research broadens, i t merges with the generally more all-encompassing concept of p o l i c y analysis. Where p o l i c y analysis d i f f e r s from evalua-t i v e research i s i n evaluation's preoccupation with e x i s t i n g programmes. Poli c y analysis usually compares e x i s t i n g and hypothetical a l t e r n a t i v e solutions to the same problem, with the objective of specifying a l t e r n a t i v e p o l i c y and programme choices as a basis for decision-making. For the purposes of t h i s d i s s e r t a t i o n , evaluative research w i l l be interpreted i n the broader context of, for example, A l k i n (72): Evaluation i s the process of ascertaining the decision areas of concern, s e l e c t i n g appropriate information and c o l l e c t i n g and analyzing information i n order to report summary data useful to decision makers i n s e l e c t i n g among a l t e r n a t i v e s . (Alkin 72, p. 107). 40 This broader d e f i n i t i o n appears j u s t i f i e d from two perspectives. F i r s t l y , as discussed i n Chapter I, the value system of planner and c i t i z e n may indeed d i f f e r on the . . . 20 i d e n t i f i c a t i o n of programme goals and solutions. To base the proposed user-oriented evaluation process on the narrower foundation of expert-enunciated goals and to then proceed to compare the subsequent impact of these programme goals on re c i p i e n t s may well lead to an inappropriate analysis of programme impact from the viewpoint of the user. . Secondly, most conceptualizations of evaluation research are based upon an analysis of single purpose pro-grammes i n , for example, the f i e l d s of health or education. In such contexts, the t r a d i t i o n a l experimental design method of analysis appears appropriate. However, i n the context of the housing evaluation proposed as the focus of t h i s study, the more a l l encompassing multi-dimensional concerns of p o l i c y analysis appear more r e a l i s t i c . The preceeding discussion has indicated that evaluation i s i n no way a unidimensional concept. A review of evaluation research shows the o r i g i n s of these varied directions and indicates problems attendant to the t r a d i -t i o n a l evaluation process. The Evolution of Evaluative Research The growth and development of evaluative research c l o s e l y p a r a l l e l s the evolution of the public service move-ment and the changing acceptance of the r e s p o n s i b i l i t y of 41 "relevance" i n various s o c i a l science d i s c i p l i n e s . As a P o l i c y Tool: Attitudes presented i n Chapter I j u s t i f y i n g the i n i t i a l exclusion of c i t i z e n involvement i n the planning process are equally applicable i n explaining the reluctance of many agencies to undertake evaluative research. P r i o r to the mid twentieth century, the accent was on the i n i t i a t i o n of new services to meet demand rather than on the evaluation of e x i s t i n g programmes. This attitude i s exemplified by Member of Parliament Dunning, who, when questioned i n the House as to why home bui l d i n g was i n a slump, rejected the question as i r r e l e v a n t . I would not undertake to send out an i n v e s t i -gator to f i n d out why people did not b u i l d houses. The reasons why they have not done so during the l a s t number of years would range over a very wide v a r i e t y of opinions and would d i f f e r i n various parts of Canada. So I doubt any p r a c t i c a l good would come of appointing people to f i n d out why houses have not been b u i l t i n s u f f i c i e n t numbers. I would prefer to proceed on the constructive l i n e s which the b i l l contemplates by making i t easier for houses to be b u i l t i n the future. (Dunning, Hansard, June 13, 19 38, p. 3780). Such evaluation as did occur during t h i s e a r l i e r time period took the form of community appraisal forms, evaluation schedules, and grading standards for community s e l f assessment. In the United States, C. V. Chapin (21), for example, devised a series of r a t i n g sheets to i d e n t i f y problem areas i n each state. The main c h a r a c t e r i s t i c of t h i s research was i t s basis on the judgment of a few pro-42 fe s s i o n a l administrators. That these evaluative procedures were inadequate becomes increasingly evident. In 1944, Member of Parliament Stanley Knowles enunciated t h i s concern when he c r i t i c i z e d the government's slowness to i n i t i a t e research deemed necessary by the Curtis Report (44) on the s i t u a t i o n of housing i n Canada. Requests for evaluative research have been slow in reaching implementation. Only a few sectors of the govern-ment (for example, the Treasury Board) and s p e c i f i c programmes (such as the Local I n i t i a t i v e s Programme) include evaluation as part of the on-going planning process. Hartle's (72) work provides an example of the evaluative schemes being developed i n the Treasury Board. He feels the i d e a l information input into actual p o l i c y making i s a sort of i n f i n i t e matrix. The rows representing the range of.actions, programmes or interventions open to the a r b i t e r . The column headings should be thought of i n two t i e r s , the uppermost being statements of the objectives or goals that concern the a r b i t e r . Hartle, being an economist, sees the maximization of the aggregate net worth of a l l the c i t i z e n s of the nation as the source of these goals. This i s a c t u a l l y not as naive as i t sounds, as his formulation of i n d i v i d u a l net worth i s comprehensive. Hartle's process when applied to 22 the r e a l world leaves unanswered several concerns: (a) Actual behaviour of government d i f f e r s from the unitary perfection of the i d e a l model. Major Canadian c i t i e s usually have four le v e l s of government plus other autonomous 43 agencies supplying public goods. (b) Problems of uncertainty with respect to acts of other governments or agencies. (c) P o l i t i c a l and bureaucratic desires to preserve or enhance status, functions and permanence. (d) Severe r e s t r i c t i o n s on the range of actions open to each l e v e l , which i s s u f f i c i e n t to guarantee sub-optimal performance even i n the absence of i n s t i t u t i o n a l constraints. Part of the o v e r a l l reticence to a l l o c a t e funds for evaluation i s the reluctance of administrators to submit t h e i r programmes to inquiry. A second, equally r e a l i s t i c explana-t i o n , l i e s i n a perceived lack of a suitable evaluation methodology. The Evolution of Evaluative Research as a Concern of the  So~cial S c i e n t i s t : Several academic d i s c i p l i n e s have devised methodologies to evaluate s o c i a l actions. P o l i t i c a l Science: The emerging f i e l d of p o l i c y sciences attempts to account for the development of public p o l i c y by analyzing choices and assumptions underlying present and anticipated programmes. Methods employed are network analysis and case studies of p o l i c y making behaviour. Many studies which consider the behaviour of the c i t i z e n do so by using voting records as a proxy for c i t i z e n viewpoints. Economics: has u n t i l recently considered p o l i c y from the t r a d i t i o n a l market viewpoints of public finance and labour economics. Evaluative studies i n economics have t h e i r 44 antecedents i n private market research and i n project appraisals i n connection with i n t e r n a t i o n a l loans through the World Bank, whose a r t i c l e s of agreement require such evalua-t i o n . This evaluation has contributed to the development of various project appraisal techniques. Cost-benefit com-parisons, cost-effectiveness analysis, and programme budgeting are based on the assumption that the amount of money spent i n a j u r i s d i c t i o n indicates the nature and q u a l i t y 23 of services provided. Cost-benefit analysis attempts to provide a method-ology to evaluate programmes, to serve as a p a r t i a l screening device to eliminate obviously uneconomical projects, and as a base, however inadequate, for ranking and comparing projects and choosing among a l t e r n a t i v e s . Teitz (6 8) suggests cost-effectiveness analysis as an approach to quantifying urban service objectives with the costs and benefits of alternate solutions defined by government performance measures. Sewell, Davis, Scott and Ross (65) provide an example framework for cost-benefit analysis as a systematic approach to evaluation i n the s e l e c t i o n of Canadian resource development projects. The examples i n the guide r e l a t e to water resources. The p r i n c i p l e s have wider a p p l i c a t i o n . While the f i e l d of cost-benefit analysis includes a wide range of studies, techniques for assembling non-quantifiable costs and benefits for rigorous comparison are not evident. Despite the growing l i t e r a t u r e i n t h i s f i e l d there i s an attendant concern that, i n view of the d i f f i c u l t y 45 of quantifying "human benefits", the methodology may have li m i t e d a p p l i c a b i l i t y to s o c i a l planners. Faced with the need to examine programmes and objectives, economists, working for federal governments both 24 i n Canada and the United States, have moved since the mid 1960's, toward the use of Planning, Programming and Budgeting Systems (PPBS) as t h e i r main a n a l y t i c a l t o o l . To Wright (69), PPBS i s not so much a body of techniques as i t i s a viewpoint. It i s the view that the majority of government a c t i v i t y has to do with the a l l o c a t i o n of resources. As such, government a c t i v i t y should be the subject of economic analysis. The crux of PPBS i s a systematic analysis of alternate ways of meeting government objectives. PPBS i s a 25 t o t a l programming systems approach asking not A, B, or C but rather "Given our objectives and resources what i s the optimum mix of A, B, and C?", the optimum mix being the par-t i c u l a r combination of objectives, resources and scheduling which comes closest to the greatest e f f i c i e n c y and e f f e c t i v e -ness. PPBS does not make choices but i t does help to make clear the ingredients of choice and the probable consequences of a l t e r n a t i v e s . There are several c r i t i c i s m s of PPBS as a t o o l to analyze s o c i a l programmes. Herbert B a l l s (70) i n discussing PPBS i n a Canadian context suggests that more emphasis has been placed on programming and budgeting and less on planning and systems analysis than perhaps should be. The method i s 46 considered of most use on programmes with cl e a r , non-contro-v e r s i a l goals, where there i s c e n t r a l i z e d decision making and where costs and benefits can be measured - conditions seldom found i n multi-dimensional s o c i a l action programmes. Several urban economists have explored the p o t e n t i a l a p p l i c a t i on of mathematical programming models to the evalua-t i o n of alternate urban plans and to the improvement of pro-grammes. Stuart (70), using the Model C i t i e s programme as an example, develops a simple l i n e a r programming model b u i l t around a matrix of r e l a t i v e effectiveness c o e f f i c i e n t s , a set of performance standards, and appropriate programme budgets. The problem of t h i s , and s i m i l a r models, i s that the most important unresolved issues i n modelling strategy are s t i l l 2 6 the incorporation of behavioural concepts into the model. Amenity, aesthetics, s o c i a l goals and e f f e c t s are among the class of variables which have been omitted (through oversight or convenience) from most computer models. There i s then some doubt concerning the v a l i d i t y of using l i n e a r programming models as a basis for s o c i a l planning. Some evidence e x i s t s of attempts to consider the impact of the investment of human c a p i t a l i n the evaluation of poverty and education programmes. In macro economics, national economic accounting has expanded to include the invest-ment i n human resources under the rubric of s o c i a l accounts. In welfare economics, in d i f f e r e n c e curve analysis concepts, applied by L i t t l e (65) to behaviour l i n e s , t e s t the success 47 of a programme by whether the c l i e n t advanced to a "higher" behaviour l i n e . However, the o v e r a l l c r i t i c i s m of economic programme evaluation remains. Measures of success are i n v a r i a b l y only those which are e a s i l y q u a n t i f i a b l e and c h a r a c t e r i s t i c a l l y ignore non-quantifiable data. Typical of such studies i s that by Bateman (67) i n which a programme for work r e - t r a i n i n g i s evaluated s o l e l y from the c r i t e r i a of income benefits derived over programme costs. Potential benefits i n terms of user s a t i s f a c t i o n or family s t a b i l i t y are ignored. So c i o l o g i s t s , psychologists and behavioural  s c i e n t i s t s have considered the diverse needs and expectations of the c i t i z e n to a greater extent than have economists. In contrast to economists, behavioural s c i e n t i s t s have seldom approached need from a public p o l i c y , decision-making frame-work. Many s o c i a l s c i e n t i s t s have preferred to remain i n the laboratory with the "purity" of i t s c o n t r o l l e d environment. This i s p a r t l y explained by the reward structure of the s o c i a l sciences. Many academic s c i e n t i s t s value the prestige that t h e i r contributions to basic research and theory give them i n the eyes of t h e i r peers more than whatever reward might be obtained from c l i e n t s who would f i n d t h e i r work useful. (Williams 71, p. 62). In the f i e l d of housing, several examples are available of attempts to evaluate programmes u t i l i z i n g methodologies devised by s o c i o l o g i s t s , psychologists and other behavioural s c i e n t i s t s . 48 Housing as an Evaluative Research Focus Evaluative studies of programmes designed to improve the q u a l i t y of the r e s i d e n t i a l environment have been under-taken from two perspectives - that of the s o c i a l s c i e n t i s t whose concern i s to develop theories explaining man-environ-ment i n t e r a c t i o n and, secondly, of government agencies inv e s t i g a t i n g funding a l l o c a t i o n s . Limitations of Academic Research: Housing studies undertaken by academic s o c i a l s c i e n t i s t s have l i m i t e d a p p l i c a b i l i t y i n the context of user-27 oriented evaluation methodology. Behavioural s c i e n t i s t s frequently f a i l to focus on variables amenable to p o l i c y intervention. A l t e r n a t i v e l y , the researcher w i l l neglect to translate his findings into the terminology of the p o l i c y planner. Examples of such studies include the man-environment investigations of dwelling layout and l o c a t i o n undertaken by Chapin (47), Lansing (71) and Buttimer (72); Gans 1 (61) analysis of s o c i a l mix; and housing choice studies of Michelson (72). Michelson's (72) work i s a case-in-point. Policy planners at Central Mortgage and Housing expressed disappointment i n the r e s u l t s of research designed to i d e n t i f y the s o c i a l basis of family decisions on housing type and loca t i o n . During informal discussions with various CMHC employees concern was expressed that Michelson's research, while t h e o r e t i c a l l y i n t e r e s t i n g as presented, lacked 49 a p p l i c a b i l i t y i n a p o l i c y context. The p o l i c y planner was not able, unassisted, to bridge the t h e o r e t i c a l - p r a c t i c a l gap. Where attempts have been made to i n t e r p r e t s o c i a l variables i n a p o l i c y context, the r e s u l t s have been d i s -appointing. Schorr (64) tackled the task of defining and measuring the s o c i a l costs of slum l i v i n g . His indicators to compare 'slum' and 'non-slum' areas were r e a d i l y accessible and admittedly s u p e r f i c i a l , indices of disease, f i r e l o s s , crime and delinquency. A second l i m i t a t i o n of s o c i a l evaluation studies i s the focus on unique projects. Examples of such research include Chapin's (47) e f f e c t s of public housing on project residents; Festinger's (50) study of community organization and morale i n a housing project; Deutsch and C o l l i n s ' (51) work on an i n t e r -r a c i a l housing project; and Wilner's (62) implications of public housing for health and psychological adjustment. Each provides i n d i v i d u a l insights but f a i l s to assess comparative advantages of alternate environments. One of the few attempts to provide a comprehensive methodology for measuring the performance of s o c i a l programmes was devised by Alberts (70) using concepts developed i n operations research. Alberts stresses that measures of per-formance should r e f l e c t the goals of the i n d i v i d u a l and of society. He accepts, but does not cope with, the d i f f i -c u l t y of seeking quantitative measures of subjective v a r i a b l e s . Alberts determines s a t i s f a c t i o n l e v e l s based on the degree to which a person i s 'saturated' with respect to a p a r t i c u l a r i n d i v i d u a l or s o c i e t a l goal: that i s when he i s 50 i n d i f f e r e n t as to whether or not to s e l e c t a course of action. Unfortunately, Alberts neglects to tackle the question of how to determine these i n d i v i d u a l and s o c i e t a l goals from the view-point of the c i t i z e n - rather he i n f e r s them through expert opinion, his own! Housing Economists: Behavioural s c i e n t i s t s have f a i l e d to provide p o l i c y oriented housing evaluation. Housing economists, i n contrast, o f f e r several operational methodolo-gies, but understate the s o c i a l costs and benefits of programme actions. The majority of economic-based housing research has r been undertaken i n the context of U.S. Inner C i t y urban renewal programmes. Cost-benefit methodologies are used to measure 2 8 the e f f i c i e n c y of public expenditures. Rothenberg (67) i n studying a s p e c i f i c redevelopment project, maintains that the consequences of government actions should be traced back to the f u l f i l l m e n t of the wants of i n d i v i d u a l s . The l i m i t a t i o n of his work i s that his d e f i n i t i o n of "wants" uses as an indicator the increase i n land values i n the project area; a 2 9 c r i t e r i o n hardly relevant to renting residents. Rothenberg may have sought to fashion a technique to f a c i l i t a t e the empirical measurement of s o c i a l good. In r e a l i t y his research presents at best a well defined statement of the problems of determining costs and benefits, of aggregating benefits, and of defining the relevant population. Messner (67) applied Rothenberg's'framework to an Indianapolis redevelopment project. His economic analysis 51 was considered useful by planners i n the s e l e c t i o n of design and location of the project. The analysis i s an inadequate basis for the provision of guidelines concerning alternate uses of public resources and for the determination of s o c i a l costs. Ross's (67) attempt to compare federal payments to various housing programmes applied cost-benefit methodologies to determine income r e d i s t r i b u t i o n and a d e s c r i p t i o n of the environmental impact of alternate programmes (in terms of the provision of physical f a c i l i t i e s and the impact on neighbour-hood subcultures). Ross makes no attempt to equate the two sets of data. His expectations were that the decision-maker would at l e a s t have avail a b l e a d e s c r i p t i v e i n d i c a t i o n of environmental impact. This review of housing evaluation e f f o r t s indicates the changing perception of relevant indicators of housing programme adequacy. T r a d i t i o n a l l y , housing demand derived quantitative programme targets through "housing d e f i c i t " ; calculated on the basis of new household formations or through public housing waiting l i s t s . The number of s t a r t s or per-sons housed became indicators of programme performance. The l i m i t a t i o n s of such indices resulted i n attempts to postulate an e f f e c t i v e housing demand. This required a further disaggregation of the market into submarkets organized according to income groups and l o c a l i t y on the basis of census data. Each submarket was seen to present unique 52 c h a r a c t e r i s t i c s with respect to user need, type of housing, c r e d i t available and amenity. The assumption underlying t h i s disaggregation i s that the inhabitants of a census t r a c t , a c i t y or even a nation share so much common culture and physical structure that t h e i r perceptions and preferences w i l l normally be d i s t r i b u t e d i n t h i s context. (Peterson 67 p. 19). Changes i n v i t a l s t a t i s t i c s , crime, and service rates served as indicators of programme performance. The more recent appreciation of l i f e s t y l e d i f -ferences within previously aggregated income groups i s leading toward a further disaggregation of housing demand. New i n d i -cators are required. However, housing evaluators are faced with the d i f f i c u l t r e a l i z a t i o n that many v i t a l aspects of user s a t i s f a c t i o n are c l e a r l y d i f f i c u l t to cope with i n the context of evolving s o c i a l action programmes. By-and-large evaluative studies undertaken by s o c i a l s c i e n t i s t s have f a i l e d to combine p o l i c y relevant variables with i n d i c i e s of s o c i a l concern. Three exceptions, to be considered i n more d e t a i l l a t e r i n t h i s chapter as p o t e n t i a l models of user oriented evaluations, are Gruen's p r i o r p o l i c y evaluation of r e l o c a t i o n proposals i n Dayton, Ohio; Feagin's analysis of an experimen-t a l rent subsidy programme i n Boston, and Onibokun's i n v e s t i -gation of the r e l a t i v e h a b i t a b i l i t y of public housing projects i n South West Ontario. 53 The E f f o r t s of Federal Housing Agencies: A second source of housing evaluative studies i s found i n the e f f o r t s of federal housing agencies. Government agencies responsible for administering housing programmes have been slow to respond to•the need for evaluation. In the United States, the Department of Housing and Urban Development i s s t r u c t u r a l l y well adapted for the develop-ment of evaluative a c t i v i t i e s and techniques ....In practice however, the planning and evaluative units function as small research s t a f f s ministering to the immediate needs.... Formal program evaluation has been a low p r i o r i t y item and has r a r e l y been done on a systematic basis before the recent a v a i l a b i -l i t y of substantial funds for evaluation of the model c i t i e s program. (Wholey 70, pp. 64 and 77). There i s , however, evidence of a growing concern for evalua-tions which seek to i d e n t i f y and devise means for u t i l i z i n g 30 the best and most useful p o l i c y r e l a t e d research studies. In Canada, p r i o r to the Hellyer Task Force (69), federal o f f i c i a l s showed l i t t l e concern with the adequacy of units produced or with the q u a l i t y of l i f e i n projects. No reviews of tenant s a t i s f a c t i o n with the program were ca r r i e d out nor were there any i n t e r n a l evaluations of the planning and design of projects. (Dennis 72, p. 179). In general, CMHC compared alternate programmes on the basis of i n i t i a l cost, rate, and the quantity of production. Dennis and Fish (72) i d e n t i f y cases where federal a u t h o r i t i e s were w i l l i n g to pay high costs i n hope of quick, v i s i b l e s t a t i s t i c a l success measured by the number of units pro-duced. Yet a r a t i o n a l evaluation of housing a l t e r n a t i v e s over time must recognize the ongoing maintenance and broader 54 s o c i a l considerations far outweigh considerations of i n i t i a l cost or of production measured s o l e l y i n terms of numbers of units (Goetze 72, p. 70). Most evaluation sponsored by CMHC has been under-taken through externally commissioned studies. Where pro-gramme recipien t s did provide an input, i t was only to describe 31 the character of the c l i e n t e l e . Two of the more contro-v e r s i a l of these studies were exceedingly c r i t i c a l of e x i s t i n g p o l i c y analysis. Dennis (72) characterized p o l i c y analysis as haphazard, lacking any systematic attempt to review s o c i a l issues underlying housing problems or to determine the effectiveness of programmes. The Charney Report (71) on the "Production and Adequacy of Low Income Housing i n Canada" found that: the government allocates approximately $1.1 b i l l i o n a year to housing on the basis of very l i m i t e d information on the o v e r a l l state of the housing stock, on the usefulness of the housing produced or on the o v e r a l l structure of the housebuilding industry. (Housing and People, A p r i l 1972, p. 3.). In response, CMHC established an evaluation d i v i s i o n designed to make the Corporation more responsive to the l i v i n g needs of i t s c l i e n t s by evaluating Canadian Housing P o l i c i e s and programs i n terms of changing consumer demand. (CMHC In-house Paper 70, p. 7). Be that as i t may, the Corporation has yet to undertake an integrated evaluation of e x i s t i n g programmes. Evolving directions i n CMHC do, however, indicate an i n t e r e s t i n 55 cross-Canada comparative analysis of s p e c i f i c programmes. The f i r s t example i s a j o i n t CMHC-Ministry of State for Urban A f f a i r s proposal to evaluate the Neighbourhood Improvement 32 Programme. I n i t i a l plans are for the two agencies to carry out separate, but complementary, evaluations. CMHC w i l l con-sider programme e f f i c i e n c y and effectiveness i n l i g h t of programme goals. MSUA's task i s to examine the programme i n r e l a t i o n to a broader set of urban impacts, s p e c i f i c a l l y , public services and finance, l o c a l p o l i t i c a l i n s t i t u t i o n s , c i t i z e n p a r t i c i p a t i o n , inner c i t y , s o c i a l and psychological change, housing and land markets, i n t r a c i t y migration pat-terns, intergovernment and interdepartment r e l a t i o n s . The evaluative process being u t i l i z e d i s (1) to determine pro-gramme goals through an analysis of enabling l e g i s l a t i o n ; (2) to speculate on possible impacts through a serie s of "think-tanks"; and (3) to commission evaluations of the various programme dimensions. While, at the time of writing, evaluation guidelines had yet to be operationalized, several proposals indicate a mounting concern for user s a t i s -33 f a c t i o n . 56 Applications of E x i s t i n g Evaluation Methodologies Three general evaluative models, devised by Wholey, Carter and Wharf, and Caputo, and four s p e c i f i c examples of housing programme evaluation, are worth considering i n more d e t a i l as examples of t r a d i t i o n a l evaluative processes. Theoretical Models Wholey's Experimental Model: Joseph Wholey's (70) evaluation proposal expresses the t r a d i t i o n a l research design of Campbell and Stanley (6 3) involving experimental and control groups. Wholey does not provide operational examples of these techniques i n the con-text of multi-dimensional programmes. Rather, Wholey's recom-mendations focus on the administrative task of implementing evaluation i n a government agency. He proposes that p r i o r i t i e s be defined c e n t r a l l y , that enabling l e g i s l a t i o n include accom-panying evaluation proposals, and the i n s t i t u t i o n of f u l l time evaluation departments. Carter and Wharf*s Typology: A recent Canadian evaluation model, devised by 34 Carter and Wharf (73) presents a framework designed for use eit h e r by a small agency or a large government department, and adaptable to evaluation of ongoing or new projects. The Carter-Wharf procedure i s based on the i d e n t i -f i c a t i o n of a typology for c l a s s i f y i n g s o c i a l development programmes. From the "correct" i n i t i a l c l a s s i f i c a t i o n of 57 programmes into "inquiry", "exploratory" or "demonstration" 35 projects they see the remaining evaluative procedure flowing i n a l o g i c a l sequence from: Type of project — P type of research design P i d e a l evaluation ---^feasible evaluation —•» actual evaluation -> use of evaluative findings ,process —•> ongoing adjustments* outcome —> implementation > decision making. Included i n the process i s the recommendation for ongoing monitoring and c i t i z e n involvement. although c l i e n t and/or community p a r t i c i p a t i o n i n evaluative research w i l l undoubtedly r e s u l t i n a d d i t i o n a l cost burdens, t h e i r input should be considered s e r i o u s l y . Observations of c l i e n t s and residents about program conse-quences are often sound. They can help programs gain acceptance i n communities, while providing useful information about the program i t s e l f . (Carter and Wharf 73, p. 122). In l i m i t e d t e s t i n g Carter and Wharf found t h e i r c l a s s i f i c a t i o n scheme a s s i s t e d i n c l a r i f y i n g project objectives and determining the appropriate design for evaluation. Their model does not, however, r e f l e c t the complex concerns of operationalizing the t o t a l evaluative process. The authors 36 have yet to come to grips with the p r a c t i c a l problems of defining programme objectives, handling evolving m u l t i -37 dimensional concerns, d i f f e r e n t i a t i n g between programme and system o r i g i n a t i n g problems or involving users i n the process. In short, they structured the obvious. Caputo's Integration Model: Caputo's (73) model (Diagram 3) comes closest to specifying the need for, and role of, c i t i z e n s i n the 58 DIAGRAM CAPUTO'S MODEL FOR URBAN PUBLIC POLICY EVALUATION (4) (4) Program E v a l u a t o r -s (3) B (Ul) Group I n t e r a c t i o n <3) C i t i z e n E v a l u a t i o n at T C , T 5 . , T 2 ' ' ' ' T , (4) •n T (III) (3) (J) (1) (Ul) _J2_ (3) —M (IV) Academic & P r o f e s s -i o n a l E v a l u a t i o n at X o-»-P u b l i c p o l i c y being evaluated my T 2- - j j n (2) 1 ( . I V ) OH I = c o n d i t i o n s e x i s t i n g p r i o r t o t h e programme T, = T n e v a l u a t i o n o f c o n d i t i o n s a n d programme e f f e c t -i v e n e s s d u r i n g a c t u a l programme i m p l e m e n t a t i o n . p r o c e s s e v a l u a t i o n ; e v a l u a t i o n d a t e s and f r e q u e n c -i e s may v a r y f r o m one programme t o t h e n e x t . t e r m i n a l e v a l u a t i o n . . T h i s w i l l be d o n e when p r o g -ramme r e a c h e s end o f e x p e r i m e n t a l p h a s e o r i s t e r m i n a t e d . S o l i d l i n e a r r o w s i n d i c a t e d i r e c t i n t e r a c t i o n a nd i n v o l v e m e n t among p a r t i c i p a n t s . - B r o k e n l i n e a r r o w s i n d i c a t e i n d i r e c t i n t e r a c t i o n a n d i n v o l v e m e n t among p a r t i c i p a n t s . F r o m C a p u t o 73 p. 115. 59 evaluative process. His model evolved from evaluations of the Gary, Indiana model c i t i e s programme. Three basic concepts are presented - the i n c l u s i o n of various sectors of the community i n the evaluation process; i n t e r a c t i o n between these sectors; and the provision for dynamic monitoring. Caputo sees three groups involved i n generating programme evaluation. The programme evaluator (A), as a cat a l y s t and c r i t i c , i s responsible for the o v e r a l l organization and implementation of the evaluation and f o r conveying findings to programme administrators. He co-ordinates the professional (D) and c i t i z e n (C) evaluation components. Academic and professional evaluators, using the combined expertise of m u l t i - d i s c i p l i n a r y teams, provide one i d e n t i f i c a t i o n of programme objectives and recommendations. A second evaluation i s conducted by c i t i z e n s who, through group meetings, personal contacts, surveys, and outside technical expertise, are asked to i d e n t i f y t h e i r objectives, develop evaluation c r i t e r i a and make recommendations. A unique feature of the model i s the important role played by c i t i z e n evaluation; i t i s a co-equal role to that played by academic and pro-f e s s i o n a l evaluators. (Caputo 73, p. 116). 3 8 Caputo's " i n t e r a c t i o n process" integrates the recommendations of c i t i z e n and professional evaluation teams. Group i n t e r a c t i o n acts as a f i l t e r i n g device, i t r e f l e c t s the need for alternate evaluations to be thoroughly discussed and analyzed before being considered by program evaluators. (Caputo 73, p. 116). Where possible, a concensus on recommendations for submission 60 to the programme evaluator forms the output of the group i n t e r a c t i o n . However, c i t i z e n evaluators, l i k e p rofessional evaluators, can choose to emphasize any of several measures, r e s u l t i n g i n the subjective use of objective data. So long as c r i t e r i a chosen r e s u l t i n s i m i l a r recommendations, concen-sus i s possible. When c r i t e r i a y i e l d c o n f l i c t i n g recommenda-tions Caputo sees the programme evaluator acting as se l e c t o r . In essence, the programme evaluator makes a subjective decision on recommendations to be advanced to programme ad-ministrators. Experience elsewhere suggests antagonism and c r i e s of "tokenism" a r i s e from c i t i z e n s and experts a l i k e who hold rejected, but, i n t h e i r minds, equally v a l i d i 39 proposals. The model's dynamic q u a l i t y i s achieved through continual monitoring and feedback to permit mid-course cor-rections and adjustments. The time component i s usually neglected i n most attempts at program evaluation; the r e s u l t i s a s t a t i c analysis which lacks a lon g i t u d i n a l dimension and hinders compara-b i l i t y with other programs. (Caputo 73, p. 116) . From the perspective of a user oriented evaluation of a multi-dimensional programme, Caputo's proposal appears to have both l i m i t a t i o n s and advantages. His model f a i l s to provide evidence of an a b i l i t y to cope with evaluating m u l t i -dimensional programmes. The hypothetical urban problems discussed i n the paper, trash removal and drug treatment, present neither the complexity nor weighting problems evident 61 i n an evaluation of the r e s i d e n t i a l environment. S i m i l a r i l y , h i s model lacks the capacity to handle divergent recommenda-tions or for determining whether solutions should be attempted through programme or broader system reformulation. Even though the i n c l u s i o n of c i t i z e n evaluation i s 40 designed to provide the advantage of alternate perspectives, Caputo's c r i t e r i a for enhancing programme a p p l i c a b i l i t y and effectiveness remain economic costs and benefits. He ex-presses concern that objective information obtained from the model may be used sub j e c t i v e l y . . . . complicated evaluation procedures may not insure objective decisions. The question then becomes whether the cost involved i n e s t a b l i s h i n g and monitor-ing such a program i s worth the benefits. (Caputo 73, p. 119) . Caputo's model provides several features worth including i n a methodology for user oriented evaluations. His model grapples with the necessity to handle evolving needs through feedback and mid-course corrections. In essence he provides for the formative evaluation recommended 41 by Scriven. The r e l a t i o n s h i p between various p a r t i c i p a n t s i n the evaluative process i s i l l u s t r a t e d . The i n c l u s i o n of c i t i z e n input i n the d e f i n i t i o n of programme objectives, evaluation and recommendations agrees with the broader d e f i -n i t i o n s of c i t i z e n p a r t i c i p a t i o n i n which p a r t i c i p a t i o n i s seen as a process. It cannot be allowed to become the pro-duct ( S v i r i d o f f 70). The i n t e r a c t i o n between professional and c i t i z e n r e f l e c t s an attempt to operationalize the pro-posals made by Friedmann's theory of Transactive Planning. 62 If the communication gap between planner and c l i e n t i s closed, a continuing series of per-sonal and primarily verbal transactions between them i s needed, through which pro-cessed knowledge i s fused with personal know-ledge and both are fused with action. (Friedmann 73, p. 177). Thus the three models discussed are, i n e f f e c t , complementary. Carter and Wharf i d e n t i f y i n i t i a l evaluative directions as a base for determining a research design. Caputo provides a model of the i n t e r a c t i v e evaluation process. Wholey indicates the ingredients of evaluation i n an i n s t i t u -t i o n a l s e t t i n g . What the three models do not attempt i s a deta i l e d operational consideration of the indic a t o r s and analysis necessary as a base for reaching programme recom-mendations. Four examples of attempts to operationalize programme evaluation, i n the context of housing, are pre-sented i n the following section. Operational Models of Programme Evaluation: Four Examples Attempts to operationalize evaluations of housing programmes are seen i n the work of CMHC Evaluation D i v i s i o n (71), Onibokun (71), Gruen and Gruen (72) and Feagin (73). CMHC's $200 M i l l i o n Programme: A 1971 analysis undertaken by CMHC of t h e i r 200 M i l l i o n Dollar Innovative low-cost housing programme pro-vides an i n d i c a t i o n of government housing evaluation p r a c t i c e . The evaluation objectives were to assess how nearly projects s a t i s f i e d the o r i g i n a l intent of being innovative and 63 low cost and, secondly, to determine the extent to which t a r -get low-income groups ac t u a l l y responded to t h i s form of intervention. The evaluation was devised by CMHC Head O f f i c e , Ottawa. It involved the d i s t r i b u t i o n of questionnaires to each CMHC Branch O f f i c e requesting written comments on: (1) d i f f i c u l t i e s encountered i n i n i t i a t i n g low cost housing; (2) a tabulation of innovative approaches; (3) comments on public reaction; (4) qua l i t y of construction; (5) market v i a b i l i t y of units; and (6) the ro l e and r e s p o n s i b i l i t i e s of management. Questionnaires were completed by agency per-sonnel from e x i s t i n g agency records. No attempt was made to d i r e c t l y determine user s a t i s f a c t i o n , provide for on-going monitoring, or to separate programme impacts from impinging urban processes. In l i e u of d i r e c t evaluation of user reactions CMHC o f f i c e s t a f f s assessed user s a t i s f a c t i o n . On this basis the report concluded: Those who have been accepted as purchasers or tenants appeared to the f i e l d s t a f f as being happy with the Program. (CMHC 71, p. 46). The analysis appears to be t y p i c a l of agency evalua-tions. Agency s t a f f determine goals, design the evaluative instrument, and answer the questions. Under such circum-stances those who are asked to do the evaluation are, i n ef f e c t , evaluating t h e i r own e f f o r t s . I t i s not surpr i s i n g that t h e i r evaluations are usually p o s i t i v e . 64 The ultimate u t i l i t y of "in-house" evaluations i s l i t t l e more than a "pat on the back" to the sponsoring agency. Patterns are already known. Few additional insights are pro-vided to a s s i s t programme reformulation. At best, "in-house" evaluations o f f e r agency personnel assigned to regional o f f i c e s an opportunity to have an input into head o f f i c e p o l i c y making. Onibokun's Relative H a b i t a b i l i t y : Onibokun (71) presents an example of a t y p i c a l housing evaluation as undertaken by a n o n - i n s t i t u t i o n a l plan-42 ner. Hxs study of eighteen Ontario housing projects attempted to determine the r e l a t i v e h a b i t a b i l i t y of public housing and the factors influencing that h a b i t a b i l i t y . The study was based on data c o l l e c t e d by d i r e c t interview techniques. A thirty-page questionnaire was administered to 199 female heads of households i n public housing projects i n the three Canadian c i t i e s of Kitchener, Guelph, and Gait. Interviews took an average of ninety minutes. The 190 respondents were selected, through s t r a t i -f i e d random sampling, from f i f t e e n public housing projects. The respondents i n t h i s study were a l l female heads of house-holds. The rationale was that women are the homemakers; they stay at home and in t e r a c t with the components of the habit a b i -l i t y subsystems more than t h e i r male counterparts and they are more re a d i l y available for interviews; moreover, women are generally responsible for housekeeping and ch i l d - r e a r i n g ; they are l i k e l y to have more experience with the physical and s o c i a l factors of housing. The questionnaire was designed to a s s i s t the researcher to c o l l e c t data on various components of the r e s i d e n t i a l environment. Relevant variables emerged from l i t e r a t u r e survey, and discussions with caretakers, housing authority personnel, and members of tenant associations. Each of the 199 respondents was asked to i d e n t i f y , on a f i v e -point L i k e r t Scale (Likert 32), her degree of s a t i s f a c t i o n with each of the seventy-four selected a t t r i b u t e s . The r e l a t i v e h a b i t a b i l i t y of each project was calculated by combining s a t i s f a c t i o n l e v e l s experienced for each of the seventy-four selected a t t r i b u t e s . Analysis involved comparisons between housing projects and within four r e s i d e n t i a l sub-systems: (1) the dwelling; (2) the environment of which the dwelling i s a component; (3) the management or i n s t i t u t i o n a l arrangements under which the housing unit and environment are administered; (4) the tenant, Onibokun's work i s t y p i c a l of many h a b i t a b i l i t y studies. He includes post-move interviews, comparison of past and present environments, comparison of treatment and control groups, and an emphasis on location and physical composition of the user's environment. He appreciates, but does not adequately account for, change. H a b i t a b i l i t y i s a r e l a t i v e concept. It i s questionable whether an analysis at one point i n time copes with the complexities of evolving needs. More 66 r e a l i s t i c a l l y , evaluative studies require a continual monitoring structure to accompany them. Only vague g e n e r a l i t i e s are directed toward the question of reformulating p o l i c y based upon research findings. His structured interview questions a s s i s t i n detecting problems. They do not provide evidence of a l t e r -nate d i r e c t i o n s . Onibokun's research appears to rest on the assumption that the physical a t t r i b u t e s of an environment are c e n t r a l to user s a t i s f a c t i o n . This assumption i s questionable. A com-parative analysis of current l i t e r a t u r e suggests, from the resident's perspective, q u a l i t y and s o c i a l relevance of r e s i d e n t i a l services are more fundamental to housing s a t i s -f a c t i o n than quantity and spacing of these services (Buttimer 72 and Turner 72). If planners continue to r e l y on the physical a t t r i b u t e s of the environment as an i n d i c a t o r of r e s i d e n t i a l environment s u i t a b i l i t y they w i l l be stressing features less than c r i t i c a l to consumer s a t i s f a c t i o n . Gruen and Gruen 1s P r i o r P o l i c y Analysis: P r i o r p o l i c y analysis u t i l i z i n g survey research methods forms the base of Gruen and Gruen's 19 70 evaluation of a proposal to geographically expand the range of housing opportunities for the less a f f l u e n t i n Dayton, Ohio. The study was commissioned by the Miami Valley Regional Planning Authority who feared resistance from suburban families and reluctance from lower-income households to p a r t i c i p a t e i n 67 integrated housing programmes. The Gruens were hired to: (1) survey community attitudes; (2) predict the kinds of impacts that low and moderate income placement could have on neighbourhood and community features valued by suburbanites; and (3) recommend c r i t e r i a to be followed i f the new housing was to benefit both the people who would l i v e i n i t and the communities i n which i t was located. Subsequently, the Gruens used these c r i t e r i a to evaluate e x i s t i n g federal housing mix programmes. The study included a survey of p a r t i c i p a n t s i n the l o c a l planning process. A sample of low and moderate income famil i e s , suburbanites, l o c a l public o f f i c i a l s , and business-men (from various branches of the industry that locate, finance, b u i l d and s e l l housing) were interviewed. These interviews were used to ascertain what factors acted as " f a c i l i t a t o r s " and " i n h i b i t o r s " i n the placing of lower-income housing i n suburban neighbourhoods. Housing experts elsewhere i n the United States were asked to comment on l o c a l examples of mixed economic, s o c i a l , r a c i a l or structu-r a l communities. Partic i p a n t interviews obtained basic household composition data and the use of e x i s t i n g services. Preference studies were attempted through: (1) r a t i n g , on a f i v e point scale; (2) pictures of p o t e n t i a l housing units; (3) forced choice between various l o c a t i o n preferences; and (4) ratings of degree of acceptance of various p o l i c y a l t e r n a t i v e s to augment neighbourhood services. Analysis of choices yielded 68 p o l i c y alternatives designed to achieve maximum acceptance by both e x i s t i n g and expected residents. The Gruens 1 a p p l i c a t i o n of s o c i a l science research methods to the solution of p o l i c y issues provides an example of an attempt to input user values into p o l i c y formulation. They do not, however, attempt to involve the user i n the entire consultative process. Their analysis of one aspect of a multi-dimensional issue indicates the complexity of discussing value-laden concepts. As a "one-shot" study of proposed, rather than e x i s t i n g , p o l i c y no attempt was made to determine whether programme proposals would a c t u a l l y a l l e v i a t e i d e n t i f i e d problems. The Gruens' study provides an excellent example of p r i o r p o l i c y research. Their a p p l i c a t i o n of a combination of research methods, the use of forced choice among various v i a b l e p o l i c y a l t e r n a t i v e s , and the consultation with p o t e n t i a l l y opposed sectors of the community, o f f e r concepts applicable to a user-oriented evaluation of s o c i a l action programmes. Feagin's Programme Reformulation: A f i n a l example of the use of survey research as a basis for programme analysis e x i s t s i n Feagin's study of an experimental rent subsidy programme i n the Boston area. His evaluation concluded that, from a variety of perspectives -residents, administrators and evaluators - the programme did i n fact provide more acceptable housing at reasonable costs. Feagin's interview schedule covered a broad range 69 of housing concerns i n c l u d i n g the housing s e a r c h p r o c e s s , household a c t i v i t y p a t t e r n s , neighbourhood s a t i s f a c t i o n , 43 household value systems, and o p i n i o n s on programme o b j e c t i v e s . Value, a t t i t u d e , and o p i n i o n data were o b t a i n e d through f o u r to s i x - p o i n t r a t i n g s c a l e s on e v a l u a t o r designed 44 statements. A t i m e - a c t i v i t y budget summarized household a c t i v i t y p a t t e r n s . I n terviews o f a s s i s t e d household, conducted b e f o r e 45 and a f t e r moving were matched w i t h f a m i l i e s i n otherwise s i m i l a r circumstances; but not r e c e i v i n g government a s s i s t -ance . In p r i n c i p l e the p l a n f o r m u l t i p l e i n t e r v i e w s w i t h matched samples i s very a t t r a c t i v e . The e a s i e r and more common housing r e s e a r c h p r o -cedure of i n t e r v i e w i n g r e p r e s e n t a t i v e samples o f r e s i d e n t s i n each of d i f f e r e n t k i n d s o f housing a f t e r they have moved i n l e a v e s g r e a t doubt as to how much of the d i f f e r e n c e between one housing group and another i s due to the s e l e c t i o n o f types o f people who were a l r e a d y d i s t i n c t b e f o r e they moved i n , and how much o f what people say i n r e t r o s p e c t about the o l d neighbourhood r e a l l y r e f l e c t s t h e i r e x p e r i e n c e i n the new one. The m u l t i p l e interview-matched sample d e s i g n reduces these doubts. (Feagin 73, p. 73) . T h i s r e s e a r c h d e s i g n r e s t s on s e v e r a l assumptions. I t i s assumed t h a t somewhere i n each p o p u l a t i o n t h e r e w i l l be a t l e a s t one f a m i l y s i m i l a r t o each f a m i l y i n the group t o be matched. The i n v e s t i g a t o r must a l s o be a b l e to i d e n t i f y and d e s c r i b e these f a m i l i e s w e l l i n advance o f the move. De s p i t e the d i v e r s i t y of data c o l l e c t e d and the p o t e n t i a l f o r comparisons, a n a l y s i s appears to have been 70 attempted on only selected items. One source of problems was the wide scope of the evaluation. Feagin attempts to provide data on both the s o c i a l welfare of the household and information on cost effectiveness for programme administrators. Feagin does not attempt to define evaluation goals from the user's perspective. Nor does he actually involve the user i n the evaluative process. Several concepts emerge from Feagin's work which warrant consideration i n a user-oriented evaluation. His questionnaire considers a wide range of household value, attitude and opinion data. Matching programme rec i p i e n t s with adjacent market neighbours emerges as a viable basis for establishing a control group to determine programme impact. Analysis of user needs and expectations d i f f e r s considerably i n the four evaluations. The CMHC analysis, as an example of i n s t i t u t i o n a l evaluations, r e l i e s e n t i r e l y upon professional judgment of user s a t i s f a c t i o n . Programme recipients were, i n f a c t , unaware that an evaluation was underway. Onibokun's study i s reasonably t y p i c a l of a vast number of evaluations undertaken by planners who, when they consult the user, do so on a s p e c i f i c number of p h y s i c a l environment i n d i c a t o r s . Both Gruen and Feagin extend user 71 opinion to various s o c i a l aspects of the environment. Feagin's method, i n addition, attempts to assess household values and to d i f f e r e n t i a t e between programme and system impacts. None of the evaluative designs attempted to a c t i v e l y involve the user i n a l l stages of the planning process. The T r a d i t i o n a l Evaluation Model The Process: Based upon the preceeding l i t e r a t u r e survey, a f i v e stage model of the ex i s t i n g s o c i a l action programme evaluation process emerges (Diagram 4 ) : Stage One: D e f i n i t i o n of Programme Goals: Evalua-t i o n requires the i n i t i a l i d e n t i f i c a t i o n of programme goals and the t r a n s l a t i o n of these goals into measurable indicators of goal achievement against which changes r e s u l t i n g from pro-gramme exposure can be measured. Goals u t i l i z e d are those s p e c i f i e d by programme administrators and r e f l e c t goals i d e n t i f -ied at the time the programme was designed. Stage Two: The Evaluation Design: Two types of evaluation designs appear i n the l i t e r a t u r e — one, a model evolved from academic and t h e o r e t i c a l evaluation proposals and, a second, r e f l e c t i n g evaluation guidelines u t i l i z e d by operational government departments. The academic model r e l i e s upon experimental designs involving control and treatment groups to d i s t i n g u i s h prog-ramme ef f e c t s from those of other forces working i n the environment. Such research designs can be of various 72 DIAGRAM 4 A MODEL OF THE EXISTING SOCIAL ACTION PROGRAM EVALUATION PROCESS DETERMINE PROGRAM GOALS DESIGN EVALUATION .METHOD Comparison treatment & control groups Comparison program findings with success critera COLLECT DATA DATA ANALYSIS Comparison . findings with program goals RECOMMENDATIONS to administrators f o r reformulation 46 strengths. The strongest requires random a l l o c a t i o n of par-t i c i p a n t s to control and experimental groups before and a f t e r analysis. In combination these features rule out "extraneous explanations" of the ef f e c t s of exposure to the programme. Examples of t y p i c a l "strong" designs are the C l a s s i c pre-test, post-test, control group s i t u a t i o n ?1 X ~2 and the 3 4 0 0 Solomon Four-Group Design n l X -2 3 o4 x k 6 . These designs control and measure the experimental e f f e c t and,in the Solomon Four-Group design, t e s t for possible i n t e r a c t i o n e f f e c t s of the measuring process i t s e l f . Such designs include a d e f i n i t i o n of the target population, drawing of a representative sample for study, a l l o c a t i o n of respondents into experimental and control groups, administra-t i o n of the programme to a treatment group and comparison of af t e r e f f e c t s . Programme e f f e c t i s determined through com-parison of the two groups. I f the difference between ^1 and ^2 is greater than the difference between ^3 and ^ 4 the programme i s deemed "a success". Weaker, but s t i l l acceptable according to Campbell and Stanley (6 3), are quasi-experimental designs. These are s o c i a l settings i n which the researcher can introduce some-thing l i k e experimental design into his data c o l l e c t i o n pro-cedures (e.g., the when and to whom of measurement) even though he lacks the f u l l control over the scheduling of experimental st i m u l i (over the when and to whom of exposure). One example of a quasi-experimental design i s the time series experiment. 74 The essence of t h i s design i s the presence of a peri o d i c measurement process on some group or i n d i v i d u a l and the i n t r o -duction of experimental change into the measurement s e r i e s . The r e s u l t s are indicated by a d i s c o n t i n u i t y i n the measurement: °1 °2 °3 °4 * 0 5 °6 °7 V While such designs do not s a t i s f y the s t r i c t requirements of the experimental design, they do, i n a variety of ways, attempt to protect against the e f f e c t s of extraneous variables on out-come measures. R e a l i s t i c a l l y , i t must b e pointed out that a l l experimental designs are subject, i n varying degrees, to sources of error. Factors jeopardizing i n t e r n a l v a l i d i t y include: history, maturation, t e s t i n g , instrumentation, s t a t i s t i c a l regression, respondent bias, and experimental mortality. Factors jeopardizing external v a l i d i t y or repre-sentativeness are the i n t e r a c t i v e e f f e c t of t e s t i n g and 47 multiple treatment interferences. In contrast, the I n s t i t u t i o n a l Model tends to r e l y on the expert judgment of those responsible for administering the programme to determine the impact on r e c i p i e n t s . In l i e u of control groups, goals are translated into predetermined measures of success, for example, the number of housing s t a r t s . If the success measure i s attained i n the judgment of programme administrators the programme i s rated as "successful", whether or not the programme was a c t u a l l y instrumental i n accounting for observed patterns. 75 Stage Three: Data C o l l e c t i o n ; Data to measure pro-gramme actions i s c o l l e c t e d by professional evaluators from eithe r programme recipie n t s or programme administrators. Stage Four: Data Analysis: Data analysis, under-taken by programme evaluators, requires the comparison of programme findings against i n i t i a l goals and other pre-established c r i t e r i a for programme "success". Stage Five: Programme Reformulation: Recommendations a r i s i n g from Stage Four are transmitted to programme ad-ministrators for programme reformulation. The Assumptions: The t r a d i t i o n a l evaluation process rests upon seve-r a l assumptions: (1) I t i s assumed that programme goals are av a i l a b l e , cl e a r , simple and that such goals were, i n the f i r s t instance, v a l i d . (2) In defining goals and set t i n g evaluation p r i o r i t i e s the further assumption i s made that the programme administrator can function as a surrogate for the user i n defining which goals are most relevant to programme performance. (3) The reduction of goals to r e a d i l y q u a n t i f i a b l e cost measures assumes both that those c h a r a c t e r i s t i c s capable of q u a n t i f i c a t i o n r e f l e c t s e n s i t i v e pro-gramme elements and, secondly, that a v a l i d l i n k e xists between input, cost, and service output. (4) The t r a d i t i o n a l experimental design presupposes 76 that control groups are available and that i t i s f e a s i b l e to either a l l o c a t e people randomly to treatment and control groups or to match p a r t i c i -pants to s u i t a b l e controls on c h a r a c t e r i s t i c s relevant to the programme's s o c i a l process. (5) The further assumptions that the public i s e i t h e r apathetic, not well informed, or i s incapable of making p o l i c y decisions i s i m p l i c i t i n the apparent exclusion of programme rec i p i e n t s from active par-t i c i p a t i o n i n a l l stages of the evaluation process. Limitations of E x i s t i n g Evaluative Research Methodologies Deficiencies i m p l i c i t i n the assumptions underlying the t r a d i t i o n a l evaluation process r e s t r i c t d i r e c t a p p l i c a t i o n of e x i s t i n g evaluation methodologies to a user-oriented evaluation of s o c i a l action programmes. S p e c i f i c l i m i t a t i o n s include methodological weaknesses, r e s t r i c t e d subject matter, administrative and bureaucratic resistance and the currency of research. Methodological Weaknesses: Ex i s t i n g evaluation methodologies assume a programme's objectives and i t s operational context are such that programme administrators can be expected to appreciate programme goals and apply experimental research designs to t e s t programme impact. In the r e a l world s e t t i n g of s o c i a l action programmes evaluators are faced with a complex, evolving s o c i a l s i t u a t i o n . The r e a l i t y of s o c i a l programmes, where c o n t i n u a l adjustments are necessary to m a i n t a i n s e r v i c e l e v e l s , p r e c l u d e s the c o n t r o l -l e d experiment (h o l d i n g e i t h e r programme or impinging v a r i a b l e s c o n s t a n t . Treatments r e f l e c t user requirements and are seldom standard. O b t a i n i n g and m a i n t a i n i n g v a l i d c o n t r o l groups throughout the e v a l u a t i o n r a i s e s the e t h i c a l i s s u e of w i t h h o l d i n g a s s i s t a n c e from e q u a l l y needy segments of the p o p u l a t i o n . S i n c e s e r v i c e to the r e c i p i e n t i s paramount: The e v a l u a t o r , whose b a s i c f u n c t i o n i s to p r o -v i d e i n f o r m a t i o n u s e f u l to decision-makers, can h a r d l y j u s t i f y i n t e r f e r e n c e w i t h the o p t i -mal o p e r a t i o n of the programme and thus he may be c a l l e d upon to adapt h i s s t r a t e g i e s to a c c o r d w i t h the r e a l i t i e s of programme l i f e . (Weiss,72,p.7). R e l i a n c e on p r o f e s s i o n a l e x p e r t i s e r a i s e s the q u e s t i o n of whether the e x p e r t can indeed f u n c t i o n as a s u r -rogate f o r the c i t i z e n i n p o l i c y p l a n n i n g . S e v e r a l s t u d i e s i n d i c a t e t h a t a l l o w i n g p l a n n i n g to proceed s o l e l y on the b a s i s of the p r o f e s s i o n a l ' s knowledge base and ' c o n v e n t i o n a l 48 wisdom' i s h i g h l y q u e s t i o n a b l e p r a c t i c e . In housing, the fundamental d i f f e r e n c e s c e n t e r on the r e l a t i v e s t r e s s g i v e n supply and demand. F e d e r a l concerns f o r q u a n t i t y of supply f r e q u e n t l y c o n f l i c t w i t h user demands f o r a v a i l a b i l i t y , q u a l i t y , l o c a l i t y , and s e l f c o n t r o l . The i m p l i c a t i o n of con-t i n u i n g t o r e l y upon e x p e r t judgment may be to c o n t i n u e to encourage d i s c o n t i n u i t y between p r o v i s i o n of government s e r v i c e s and the r e c i p i e n t ' s expressed demands. Such a s p l i t 78 between supplier and receiver contributes to increasing d i s s a t i s -f a c t i o n as services appear unrelated to consumer demand. Several authors (Wholey 70) have expressed concern that national schemes for large scale programme evaluation do not e x i s t . In the context of housing t h i s may prove a blessing. It i s questionable whether single cross-country comparative evaluations based upon one evaluation guide have u t i l i t y . Housing i s a l o c a l issue, responding to, at best, regional market conditions. A single evaluation scheme may not be pos-s i b l e . Rather, as Schon (71) suggests, researchers must devise an evaluation framework which regions can adapt to t h e i r speci-f i c needs and generate t h e i r own evaluation programmes. The propensity to base evaluative research on com-parisons of i n i t i a l programme goals and subsequent goal per-formance i s of questionable a p p l i c a b i l i t y i n user oriented evaluations. Programme objectives are too frequently based on l a r g e l y untested assumptions whose v a l i d i t y rests primarily on t r a d i t i o n or "common sense" and not on proven effectiveness. Cohen (70), for example, points out the r i s i n g opposition, p a r t i c u l a r i l y i n some black communities, to t r a d i t i o n a l pro-cesses of goal formulation. In l i g h t of the preceeding discussion i t i s reason-able to assume that, lacking c i t i z e n input into goal formula-. . . 49 t i o n , i n i t i a l goals may be unrelated to user concerns. Furthermore, evaluations based upon past goals understate evolving system impacts and ignore p o t e n t i a l side e f f e c t s . 5 0 79 Administrative Resistance: Frequently i t i s not the p r i n c i p l e s of evaluative research which create d i f f i c u l t i e s , but rather the p r a c t i c a l problems of adhering to these p r i n c i p l e s i n the f i e l d and i n the face of administrative considerations. Bureaucrats r e f l e c t the strong motivation by members of an organization to seek equilibrium and preserve the organization. Rein suggests that the p o s s i b i l i t y of research being used by an agency depends on the extent to which research avoids implications which deal with the organization's most central dynamics. Thus i t turns out that one of the major obstacles to evaluation research i s the i n t e r e s t s i n maintenance of a program held by i t s administrators. Their ambivalence i s born of a two-horned dilemma: On the one hand, research i s needed to demonstrate that the program has an e f f e c t ; on the other hand, research might f i n d that e f f e c t s are n e g l i g i b l e or non-existent. (Rossi 72, p. 227) . 51 L e g i s l a t i o n seldom includes provisions for monitoring. When evaluation does occur i n an i n s t i t u t i o n a l s e t t i n g , inadequate funds for s t a f f i n g r e s u l t i n regular employees, who have neither t r a i n i n g , s k i l l s nor expertise, undertaking evalua-tions i n addition to normal duties. In such cases, minimal time and resources are allocated to planning c o l l e c t i n g and analyzing programmes Restricted Subject Matter: Evaluative research i s designed for immediate and d i r e c t use i n improving the q u a l i t y of s o c i a l programming. 80 Yet a review of evaluative experience suggests that evaluation res u l t s have generally not exerted a s i g n i f i c a n t influence on programme decisions. To a degree, the problem l i e s i n the general lack of knowledge of the decision-making process i d e n t i f i e d by Dror (68), Gerecke (74), Wolman (71) and Meyerson and Banfield (55). Evaluative research has tended to assess those pro-gramme dimensions most amenable to q u a n t i f i c a t i o n . Such dimensions are not necessarily either the most relevant or the mo sens i t i v e to manipulation i n a p o l i c y context. For example, Swain contends that the desire of p o l i c y makers to know about r e d i s t r i b u t i v e e f f e c t s of p o l i c y proposals has been ignored by many researchers: The absence of disaggregated answers to the fundamental questions of Who pays? and Who benefits? l i e s at the core of some decision makers impatience with research as an aid i n po l i c y making. (Swain 73, p. 14). Accenting the quantity of services or a c t i v i t i e s assumes a d i r e c t l i n k between cost and service and disregards the q u a l i t y of such services. Evaluation research i s not unique i n t h i s emphasis. Most urban research, p a r t i c u l a r i l y i n the economic sector, attempts to use r e l a t i v e l y "tough 52 minded" c r i t e r i a which are simple, d i r e c t and which frequently do not take into account the r e a l complexity of the urban scene. The evaluation of fee l i n g s , of personal values, and aspirations as variables determining p o l i c y choice and housing preference are large l y ignored. 81 Currency of Research: The recent emergence of programme evaluation i s , i n i t s e l f , a contributor to present problems. A shortage of persons trained to evaluate r e a l world settings exists i n the research community. Neither government nor the s o c i a l science community are organized to develop and carry out large scale, multi-dimensional studies of s o c i a l impact. Few courses are a v a i l -able i n evaluation and p o l i c y analysis. In the u n i v e r s i t y , 53 time horizons, philosophical differences and reward systems are designed to encourage research based upon t h e o r e t i c a l problems and to d i v e r t resources from evaluative to more t r a d i t i o n a l forms of basic research. Rein indicates that teachers of s o c i a l p o l i c y must accept the awkward conclusion that they cannot o f f e r to t h e i r students a methodology and tech-nology for doing p o l i c y analysis.54 (Rein 70, p. 214). When evaluative work i s undertaken there i s no central clearinghouse to bring together the work of 55 behavioural and p o l i c y s c i e n t i s t . Because evaluative research reports are often l e f t unpublished or are published i n widely scattered journals, i t i s d i f -f i c u l t to estimate the extent of actual use...(Caro 71, p. 6). This i s c e r t a i n l y true i n the Canadian context where generally 56 available published evaluation l i t e r a t u r e i s sparse. Where reports do e x i s t , they tend to l i s t d i f f i c u l t i e s with no attempt to analyze the source of these d i f f i c u l t i e s or to set 82 forth guiding procedures to help lessen or overcome the problems. Thus evaluative research can be characterized as lacking i n content, integration, and relevance to p o l i c y con-cerns, ignoring s o c i a l costs, stressing e f f i c i e n c y over e f f e c t , seldom considering second order impacts and containing such methodological and assumptive weaknesses as to leave one confused as to whether i t was the programme or the research which was defective. One must question the v a l i d i t y of e x i s t i n g research from the viewpoint of a user oriented pro-gramme evaluation. I t i s indeed questionable whether, i n the context of a s o c i a l action programme, goals are ever simple and clear, that costs based on economic variables are v a l i d indicators of programme success, that programme administra-tors are able to act as surrogates for c i t i z e n input, or that controls are a v a i l a b l e or programmes s t a t i c . Emerging Trends i n Evaluation Research - Some Optimism Remains While i t i s c l e a r that the early o p t i m i s t i c pre-dictions of extensive s c i e n t i f i c evaluation of s o c i a l pro-grammes have f a i l e d to materialize, the concern for the development of a methodology to evaluate the impact of govern-ment programmes remains. An I d e n t i f i e d Need: . . 57 From among the most ardent c r i t i c s of evaluation attempts are also to be found some of the most o p t i m i s t i c 83 statements of the pot e n t i a l role of programme evaluation. We have never seen a f i e l d evaluation of a so c i a l - a c t i o n program that could not be faulted legitimately by good methodologists, and we may never see one. But, i f we are w i l l i n g to accept real-world imperfections and to use evaluative analysis with prudence, then such analysis can provide a fa r better basis for decision making than we have had i n the past. (Williams 71, p. 121). Enough good re s u l t s have emerged on the plus side of evaluation studies to indicate the p o t e n t i a l of a f u l l scale e f f o r t . Among emerging trends which may lead to more meaningful evaluations are: (1) An increased consideration of ongoing as opposed to summative evaluation; (2) Extension of evaluation across agency and d i s c i p l i n e boundaries; and (3) The r e a l i z a t i o n that behavioural s c i e n t i s t s who hope to contribute to the effectiveness of s o c i a l programmes through evaluative research need to concern themselves not only with the immediate methodological problems but also with larger issues concerning the s o c i a l context i n which programmes are conducted. User Involvement: The s o c i e t a l context includes consideration of pro-gramme impacts i n l i g h t of user requirements. Including c i t i z e n s and, s p e c i f i c a l l y , programme re c i p i e n t s i n an evalua-tion process assumes that the user i s , i n his own r i g h t , an expert. For example i n housing, those involved i n the search 84 for and acceptance of housing accommodation develop some s p e c i f i c behavioural patterns and attitudes towards the housing s i t u a t i o n . The extent to which one programme r e c i p i e n t can act as an "expert" on what his "fellows" desire i s an i n t e r e s t i n g question apparently unanswered i n the l i t e r a t u r e . I f c i t i z e n involvement i n the evaluation process i s to be meaningful, several authors, among them Caputo (73) and Friedmann (73) see the necessity for e s t a b l i s h i n g net-works involving c i t i z e n s , researchers, p o l i c y makers and pro-gramme administrators. Planning and evaluation are seen as a mutual learning experience - the planner from the c l i e n t ' s personal knowledge, the c l i e n t from the planner's tec h n i c a l expertise. The process i s by no means e a s i l y accomplished. Caro's analysis of the interplay between c l i e n t and evaluator i n planning of the Denver Model C i t i e s programme concluded that conducting s o c i a l research i n the f i e l d often means that data are slow i n coming, complex and q u a l i f i e d but, i n a gross sense observations of residents about the consequences of programs may be sound and these judgments may be delivered with a c l a r i t y and conviction which appeals to the p o l i c y maker. (Caro i n Carter 73, p. 67). What Caputo and Friedmann, among others, seem to be c a l l i n g for i s a methodology which could develop new standards by feeding more diverse information into the planning process - namely data derived from s o c i a l s c i e n t i f i c i n q u i r i e s about the preference of present and p o t e n t i a l programme users. E x p l i c i t c r i t e r i a to e s t a b l i s h programme goals from the user's perspective, for 85 considering evolving value-laden multi-dimensional programmes, or for d i f f e r e n t i a t i n g programme from system impacts i n the context of an action oriented programme are inadequately enunciated i n the l i t e r a t u r e . In b u i l d i n g t h i s base of knowledge, we must not succumb to the temptation to take up the solu-able problems and forsake the i n t r a c t a b l e dilemmas on the assumption that t h i s i s the orderly route to knowledge. Social p o l i c y i s a l l about s o c i a l objectives and the values that embody the choice of s o c i a l programs. These are p r e c i s e l y the problems that touch the l i m i t s of s o c i a l science and r a i s e the spectre of the ancient and s t i l l inadequately explored t e r r a i n where facts and values merge. (Rein 70, p. 214). The Task Evidence presented i n chapters one and two appears to substantiate the three assumptions upon which the d i s -s e r t a t i o n concern i s based - that c i t i z e n s are increasingly aware of and desirous of p a r t i c i p a t i o n i n the s o c i a l planning process; that governments, p a r t i c u l a r i l y i n the f i e l d of housing, are responding to perceived s o c i a l needs of t h e i r constituents through l e g i s l a t i n g s o c i a l action programmes; and that, i n an attempt to improve the match between s o c i a l need and government actions, evaluation methodologies have evolved to assess p o l i c y . However, the t r a d i t i o n a l evaluation process appears to be inadequate for the evaluation of programmes from both the needs and expectations of the user and from the stand-point of involving c i t i z e n s i n t h i s aspect of the planning process. 86 I f one assumes that including some members of the concerned population i n the evaluation process w i l l increase the l i k e l i h o o d of the evaluation r e f l e c t i n g s e n s i t i v e pro-gramme dimensions, being relevant to user concerns, and that proposals w i l l not cause further misunderstanding between administrator and user, then an alternate evaluation method appears to be needed. 87 CHAPTER III TOWARD A USER ORIENTED METHODOLOGY FOR THE  EVALUATION OF SOCIAL ACTION PROGRAMMES The preceding chapters discussed the increase of c i t i z e n p a r t i c i p a t i o n i n the planning process and the attendant expanding s o c i a l consciousness of governments. Also i d e n t i f i e d were two gaps i n e x i s t i n g knowledge - how to evaluate a govern-ment programme from the perspective of the c i t i z e n user and, secondly, how the programme r e c i p i e n t might a s s i s t i n making such an evaluation more i n d i c a t i v e of programme impact. T r a d i t i o n a l programme evaluation seeks to provide evaluation information through an i d e n t i f i c a t i o n of p o l i c y objectives and subsequent analysis of goal attainment, frequently employing minimal consultation with programme r e c i p i e n t s . The base upon which change i s measured i s estab-l i s h e d by programme administrators rather than programme users. With t h i s concern i n mind, Chapter III outlines an alternate evaluation process through which programme users i d e n t i f y desired goals and a s s i s t i n assessing the extent to which programme output meets user expectations. The f i r s t sections of t h i s chapter consider the research s e t t i n g , f o l -lowed by a description, explanation and j u s t i f i c a t i o n of a methodology to provide user oriented evaluation i n the context of a value-laden, ongoing, multi-dimensional s o c i a l action programme. The chapter concludes with a proposal to id e n t i f y the p o t e n t i a l role and r e s p o n s i b i l i t i e s of programme users i n the evaluation process. 88 PART I THE RESEARCH SETTING - A CONSIDERATION AND JUSTIFICATION To date t h e r e has been a d i s t i n c t s p l i t between aca-5 8 demic r e s e a r c h and s o c i a l p o l i c y e v a l u a t i o n . The s p l i t i s made understandable by reward s t r u c t u r e s which i n n e i t h e r the academic nor a d m i n i s t r a t i v e world encourage a t r a n s i t i o n from th e o r y to p r a c t i c e and back to t h e o r y . D i f f e r e n c e s i n r e s -p o n s i b i l i t y , knowledge bases and time h o r i z o n s f u r t h e r account f o r the i n f r e q u e n t i n t e r c h a n g e between b a s i c r e s e a r c h and p o l i c y a n a l y s i s . While d i f f i c u l t i e s w i l l i n e v i t a b l y a r i s e , the r e -s e a rch concern o u t l i n e d i n t h i s d i s s e r t a t i o n suggests t e s t i n g i n the c o n t e x t of a r e a l w o r l d p l a n n i n g s i t u a t i o n . The f o l -lowing r e s e a r c h d e s i g n attempts to p r o v i d e both f o r g e n e r a l i t y and d i r e c t u t i l i z a t i o n o f f i n d i n g s by: - a n a l y s i s of an e x i s t i n g s o c i a l a c t i o n programme i n i t s o p e r a t i o n a l s e t t i n g . - s e l e c t i o n of a l o c a l e f o r study i n which s e v e r a l p r o j e c t s have been i n i t i a t e d f o r comparative purposes. - s e l e c t i o n o f a programme which i n c o r p o r a t e s emerging trends i n Canadian housing p o l i c y and i s p r e s e n t l y f a c e d w i t h a d m i n i s t r a t i v e d i f -f i c u l t i e s . - a n a l y s i s o f a programme which i n c o r p o r a t e s s e v e r a l assumptions which have been of concern t o housing t h e o r i s t s . THE REGIONAL SETTING On the assumption t h a t a methodology f o r a s s e s s i n g s o c i a l a c t i o n programmes sh o u l d be t e s t e d i n the c o n t e x t of 89 such a programme, a portion of the National Housing Act Assisted Home Ownership Programme, as applied i n the Greater Vancouver Regional D i s t r i c t , was chosen as the s e t t i n g for academic research. The decision to analyze one programme i n one locale was based on the r e a l i z a t i o n that the d i s s e r t a t i o n concern was to formulate and t e s t an evaluation methodology. Had an analysis of change r e s u l t i n g from programme p a r t i c i p a -t i o n been the d i s s e r t a t i o n focus, an examination of various p o l i c i e s , over time, i n a number of regions would have been necessary. However, since the research stresses process, not product, analysis of a single programme a p p l i c a t i o n i s considered j u s t i f i e d . Selection of a s p e c i f i c regional example for analysis r e f l e c t s the suggestion of Donnison i n the Govern-ment of Housing (67) that housing markets are e s s e n t i a l l y l o c a l i n nature and require analysis firmly based upon s p e c i f i c regions. When national housing programs are not securely based on a groundwork of recon-c i l i a b l e regional analysis they amount to l i t t l e more than a p o l i t i c a l game. (Donnison 67, p. 42). Selection of the Greater Vancouver Regional area as a t e s t case ( Map 5 ) presented the researcher with an environment currently experiencing some of the most d i f f i c u l t housing conditions i n Canada. Vancouver i s the t h i r d l a r g e s t metropolitan area. Immigration and physical constraints l i m i t i n g available land place pressure on the e x i s t i n g urban 90 MAP 5 GREATER VANCOUVER °. locations 91 area. The Vancouver region experienced, during the e a r l y 1970's, an escalation i n housing costs. Families on incomes under $18,000 were faced with the r e a l i z a t i o n that entering the home ownership market was increasingly improbable. At 59 the same time, changes i n Federal Income Tax Act, and the B.C. Landlord and Tenant Act, imposition of land freezes and rent controls of eight percent resulted i n a cessation of r e n t a l construction. The Vancouver area r e f l e c t s conditions as they unfortunately might develop i n other Canadian c i t i e s . However, i t might be argued that while Toronto, and to a l e s s e r extent Montreal, face s i m i l a r circumstances, the location of those c i t i e s c loser to Canadian f i n a n c i a l and p o l i t i c a l d ecision-makers might make t h e i r p l i g h t more obvious. Research i n the context of Vancouver presents a perspective less frequently brought to the attention of decision-makers. Sponsorship - The Greater Vancouver Regional D i s t r i c t Housing  Department The Greater Vancouver Regional D i s t r i c t Housing Department was formed i n December 1971 to undertake the res-p o n s i b i l i t y on behalf of the fourteen m u n i c i p a l i t i e s i n the Greater Vancouver area of i n i t i a t i n g housing projects under the National Housing Act. A c t i v i t i e s of the Department r e f l e c t the "on l i n e " requirement of d e l i v e r i n g housing i n the Vancouver area. Between 19 71 - 19 74 the Department as s i s t e d i n a range of housing a c t i v i t i e s including senior c i t i z e n projects, l o c a l input into Federal-Provincial land a c q u i s i t i o n 92 programmes, l i a i s o n and information to member mu n i c i p a l i t i e s and several "leading edge" innovative developments designed to encourage municipalites and builders to move i n more diverse directions to tackle housing needs. Two such examples included the inception of "zero l o t l i n e " developments and the integration of assisted and non-assisted housing i n pr i v a t e l y financed developments. The l a t t e r "300 Unit Special Subsidized Housing Programme" was designed to tes t procedures for integrating low and moderate income families into ownership situations i n middle-income communities. Selection of the "300 Unit Programme" as a t e s t case for t h i s d i s s e r t a t i o n evolved from several circumstances. The Programme i s an innovative attempt of a new department to implement emerging federal housing p o l i c i e s . The programme thus presented the opportunity to undertake formative evalua-t i o n research i n the context of an emerging programme. Both agencies involved, CMHC and GVRD, were eager to assess and reformulate the programme p r i o r to implementation on a broader scale. Unfortunately GVRD's lack of resources and research personnel resulted i n minimum opportunity to evaluate the 300 Unit Programme. Thus the needs of the public agency provided the researcher with an opportunity to undertake experimental research on programme evaluation methodologies i n the context of an evolving, innovative programme. In addition, housing t h e o r i s t s have speculated on the impact of two dimensions of the programme — ownership and income mix. Unfortunately, e x i s t i n g research i s la r g e l y 93 based on experience in.the United States where d i f f e r i n g r a c i a l patterns make transfer to the Canadian context of doubtful v a l i d i t y . The "300 Unit Programme" was f i r s t presented at t r i - l e v e l t a l k s i n January 19 72 as an opportunity for the GVRD Housing Department to become involved i n the provision of s o c i a l housing. A j o i n t Federal-Provincial-Municipal agreement of August 19 72 provided funds under the National Housing Act to GVRD to purchase units scattered throughout 6 0 the regional d i s t r i c t for resale to low-moderate income families on an assisted purchase plan. The f i r s t units were purchased i n September 19 72. The units acquired for resale were p r i v a t e l y constructed townhouses valued i n 1972 from $16,000 to $29,000. Purchase by low-income families was achieved through using the B. C. Home Acq u i s i t i o n Grant of $1,000 as downpayment and with monthly payments of 22 percent of adjusted household 61 income. The balance of the p r i n c i p l e , i n t e r e s t and taxes i s subsidized by the Federal-Provincial-GVRD partnership. Homes are financed over forty years at low i n t e r e s t rates. T h e o r e t i c a l l y , a t y p i c a l family with a 19 72 income of $500 per month, r i s i n g annually at 5 percent, could be expected to be i n a non-subsidy p o s i t i o n within f i v e years. The increased value of the home through normal i n f l a t i o n a r y pro-cesses was expected to r e a l i z e a p r o f i t to the assi s t e d 6 2 owner on resale* a f t e r repaying a l l subsidies. The owner could then move into the private housing market unassisted by government agencies. 94 Housing Director B i l l Casson saw the development of a subsidized Housing Scheme to a s s i s t an income group previously ignored by government housing programs to become home owners. Many low-income families have found i t d i f f i c u l t to j u s t i f y working when they might receive more benefits by being on So c i a l Assistance. Con-sequently the subsidized purchase program was developed to a s s i s t t h i s group. It was designed to function on the f a c t that a working family's income would increase due to merit raises and cost of l i v i n g b e n e f i t s . In t h i s way equity can be b u i l t up and eventually the family w i l l assume the f u l l monthly payments and no longer require government assistance. (Casson Letter of January 3, 1973). Ac q u i s i t i o n of suitable homes, advertising and sel e c t i o n of prospective home buyers was undertaken by the Greater Vancouver Regional D i s t r i c t Housing Department. Following a decision to enter the programme a prospective owner was directed to the CMHC Branch O f f i c e to discuss d e t a i l s of financing and ownership obl i g a t i o n s . Subsequent dealings concerning unit problems and mortgage payments are directed to CMHC as administrators of the mortgage. The Theoretical Basis of the GVRD Programme The t h e o r e t i c a l basis for the 300 Unit Programme i s that the s o c i a l and f i n a n c i a l implications of ownership for low-income households combined with provision of low v i s i -b i l i t y housing located throughout the urban area provide more acceptable forms of accommodation than e x i s t i n g large scale r e n t a l projects. 95 The Ownership Assumption: Assisted home ownership i s based on the assumption that ownership w i l l have p o s i t i v e s o c i a l and economic impacts on the household. Both assumptions bear examination i n the l i g h t of c o n f l i c t i n g e x i s t i n g theory. Research by Foote (60) and Rosow (4 8) indicate that ownership i s not a purely r a t i o n a l , u t i l i t a r i a n choice but i s couched i n such sentimental and symbolic values as status, commitment to community, mobility, security of tenure and control over one's own environment. P o l i c i e s encouraging ownership are based on the value-laden assumption that ownership i s the most appropriate and desirable goal for a l l Canadians. That, through owner-ship, the household's increased status and commitment to community w i l l promote such i d e o l o g i c a l values as responsi-b i l i t y and a stake i n society's future. Federal intervention in housing was j u s t i f i e d both i n Canada and the United States by such b e l i e f s on the part of p o l i t i c i a n s : The fact that a man owns his own home induces him to take a more p r a c t i c a l i n t e r e s t i n the a f f a i r s of the country and thus builds the strength and s t a b i l i t y of our national l i f e . (M. P. Rowel1, May 16, 1919, Hansard p. 2532). and o f f i c i a l s : Ownership, i n the minds of public o f f i c i a l s , i s equated with r e s p o n s i b i l i t y , while renters are considered to be less desirable c i t i z e n s . (Gruen 72, p. 81). As a r e s u l t , ownership, i n one form or another now pervades most federal housing programmes. 96 To make home ownership available to the greatest possible number of people i s a s o c i a l duty. (M.P. for Terrebonne, L Bertrand, Hansard January 28, 1954, p. 1571). c Housing th e o r i s t s have both endorsed t h i s stand The greatest benefit and incentive for up-ward mobility that can be provided by any environmental factor i s home ownership i n a compatible neighbourhood. (Gruen 72, p. 142) . and condemned i t At a time when the growing trend i s toward production of multiple r e n t a l accommodation, the federal government i s about to sponsor, a f t e r years of opposition, a program of assisted home ownership for low income households. Such sponsorship f l i e s i n the face of changing l i f e s t y l e s , increasing mobility, opposition to expressways and urban sprawl, and the e f f o r t s to lessen the gap between the status and t e n u r i a l r i g h t s of owners and tenants. (Dennis 72, p. 19). E x i s t i n g l i t e r a t u r e d i f f e r s on the importance of home ownership i n the desire to change residence. Rosow (48) i n i n v e s t i g a t i n g reasons for moving, i d e n t i f i e d possession of property and i t s attendant status, prestige and ego s a t i s -f a c t i o n as paramount. Rossi (65) concurs i n fi n d i n g that, i n addition to family structure, the desire to own was a prime motive i n moving. In contrast, Michelson finds The desire for ownership i n a l l segments of society i s high and pervasive, but while i t serves to influence some family moves, most moves require other means of explanation. (Michelson 72, p. 126). There, however, seems to be l i t t l e disagreement on the contribution of ownership towards feelings of security of 6 3 tenure and control over one's own environment. 97 There are obvious advantages i n providing modest assistance to low income families to enable them to take a r e s p o n s i b i l i t y i n t h e i r owrr-home and thus achieve a measure of i n -dependence. (Basford 73, p. 21 ). I f the s o c i a l assumptions upon which ownership i s based are open to controversy, economic assumptions are no less i n question. A s s i s t i n g low income families toward home ownership i s based on the assumption that increasing shelter costs are among the major sources of concern to lower income families and that, through s t a b i l i z i n g such expenditures, the household w i l l be better able to cope with i t s f i n a n c i a l constraints. Certainly, The increasing shelter expenditures have been among the chief sources of pressure on the budgets of the poor. (Dennis 72, p. 19). Whether home ownership provides a suitable f i n a n c i a l s o l u t i o n i s open to debate. The f i n a n c i a l consequences of ownership are expressed i n terms of s t a b i l i z i n g shelter costs and assistance i n saving such that the household w i l l , at a l a t e r date, be able to . . 64 p a r t i c i p a t e on the open housing market. GVRD Housing Director B i l l Casson operates a subsidized home ownership programme under t h i s assumption. If you can get people into t h e i r own homes we're making i n f l a t i o n work for them because i f you're renting you're just never going to get anywhere. (Casson 72, conversation) However, one must temper such expectations with the r e a l i z a -t i o n that while carrying costs are fixed, taxes, maintenance and housing's other associated costs are not. 98 In providing for ownership the assumption i s i m p l i c i t that householders both are aware of associated costs and o b l i -gations and are able to cope with unexpected demands. This assumption i s questioned by some p o l i c y researchers who see 6 5 an increasing number of defaults, owners who are unable to 6 6 maintain payments and by researchers who maintain that unanticipated costs and unrealized obligations associated with home ownership are p o t e n t i a l sources of household d i s t r e s s . Possibly the most pressing problem i s the lack of awareness by the consumer of housing associated costs. Present assistance programmes provide funds only toward the monthly mortgage payment. One questions whether t h i s i s r e a l i s t i c i n l i g h t of research into costs attendant to home ownership for persons with only marginal e f f e c t i v e demand. Families with $3,000-6,000 i n income who changed from renting to owning spent more in the f i r s t year on t o t a l consumption ( i . e . saved less and dissaved more). They spent s u b s t a n t i a l l y more on current housing expenses, home operating, and furniture and equipment but less on food, clothing, rec-reation and a l l other goods and services. (Foote 60, p. 65). People...buying a home for the f i r s t time figure out how much they can a f f o r d on a house and buy i t . Later they discover that commuting costs more than they had expected, that provision of community f a c i l i t i e s raises t h e i r taxes and generally the cost of maintaining t h e i r new home i s far higher than they expected. They have t h e i r home but unanticipated demands are placed upon them from a l l sides. (Clark 66, p. 119). There appears to be an assumption on the part of CMHC that a standard r a t i o of income to housing cost can be 99 applied both to indicate the magnitude of housing problems and the a b i l i t y of an i n d i v i d u a l household to service a mortgage. Students of housing p o l i c y have paid some attention to t h i s question and usually desig-nated a c e r t a i n proportion of income (e.g. 20% or 25%) as being the maximum desirable income-shelter r a t i o . Since these a l l o c a -tions have become part of the conventional wisdom of the housing f i e l d , i t i s par-t i c u l a r l y important to recognize that they are almost t o t a l l y a r b i t r a r y and as such have l i t t l e basis i n f a c t or theory. (Armitage and Audain 72, p. 12). Present guidelines expect a low income household to alloca t e 22 percent of t h e i r income to housing. The 22 per-cent assumption ignores the previously mentioned ownership associated expenses and assumes that a l l households place equal weighting on household expenditures. In fact, consider-able evidence i s available to suggest that d i f f e r i n g house-holds place d i f f e r i n g emphasis on the value of housing to t h e i r l i f e s t y l e . The President's Committee on Urban Housing (6 7) found that the proportion of income expended on housing varied considerably within any selected income band. Variations were 6 7 attributed to stage i n l i f e cycle and l i f e s t y l e . Michelson's (70) l i t e r a t u r e survey indicated that differences e x i s t between households with s i m i l a r gross income depending upon s o c i a l c l a s s . For example, while there i s a s i m i l a r i t y of incomes between white and blue c o l l a r workers, c l e r i c a l workers are better housed. To accomplish t h i s they spend a greater por-t i o n of t h e i r income on housing. S k i l l e d blue c o l l a r workers on the other hand spend a greater portion of t h e i r income on auto-mobiles, t e l e v i s i o n sets and the l i k e . 100 The q u a l i t y of t h e i r housing i s simply less s a l i e n t to them than i t i s to the white-c o l l a r people, for whom housing may be a greater symbol of r e s p e c t a b i l i t y . (Michel-son 70, p. 114). In l i g h t of such varied findings the question a r i s e s as to whether simple indices (22% of income) are adequate and indeed whether income i s , i n i t s e l f , a v a l i d i n d i c a t o r of housing need. To base ownership assistance programmes on the assumption of improving s o c i a l and economic conditions by pro-vidi n g standard assistance toward only one aspect of the. ownership package may be u n r e a l i s t i c . In f a c t , p u blic i n t e r -vention i n housing which favours ownership for low income families by a r t i f i c i a l l y creating e f f e c t i v e demand i n an attempt to s t a b i l i z e shelter costs may r e s u l t i n a d i s s e r v i c e to the programme r e c i p i e n t , i n the form of dissaving, and to society at large, who i n a mobile, urbanized world are decreasingly w i l l i n g or able to assume ownership (Dennis 72). Theory and practice i n providing ownership to low income households force one to question the basic assumption upon which assisted ownership programmes res t and provides a unique opportunity to te s t e x i s t i n g theory i n the Canadian context. The Housing Mix Assumption In addition to ownership, the "300 Unit Programme" incorporates a second emerging Canadian housing trend - the dispersion of low income households throughout the urban area, commonly referred to as housing mix. 101 In i t s broadest sense, housing mix includes the options of integrating units of d i f f e r i n g s i z e , tenure or 69 design and accommodating households of d i f f e r i n g age, s i z e , c u l t u r a l o r i g i n s , race and/or income i n the same neighbourhood. GVRD encourages the integration of lower income households 70 into otherwise middle income suburban neighbourhoods. This i s , i n e f f e c t , a departure from previous large project con-centrations of lower income f a m i l i e s . Zoning, federal l e g i s l a t i o n and conceptions of con-71 . . . sumer preferences have u n t i l recently e i t h e r i m p l i c i t l y or 72 e x p l i c i t l y discouraged housing mix. Segregation was in e v i t a b l e simply because programs were defined i n terms of c e r t a i n s p e c i f i c needs: we have standard programs designed for the family with c h i l d -ren, s p e c i a l programs for senior c i t i z e n s . . . . T h e o r e t i c a l l y , a developer could mix a l l these and other b u i l d i n g types i n the same development. P r a c t i c a l l y , he does not have the overhead resources for planning and negotiating with the various federal agencies that t h i s requires. (Glazer 67, p. 160). The Kel l y e r task force on housing and urban develop-ment found evidence of stigma and s o c i a l problems attendant to large housing projects. P o l i c y makers were forced to question e a r l i e r assumptions that p o s i t i v e s o c i a l consequences 73 flow from merely manipulating the physical environment. We must root from our thinking...the assump-t i o n that the physical form of our communities has s o c i a l consequences.... the truth i s that s o c i a l surroundings are more important i n determining what happens to people than are physical surroundings. (Glazer 65, p. 59). Theories were broadened to suggest that i f planners 102 could move people from poor housing conditions and place them i n better housing i n a heterogeneous s o c i a l environment the "p o s i t i v e " aspects of the more stable, higher income com-munity l i f e s t y l e would be accepted and incorporated into the lower income household's d a i l y l i f e . In addition, the socio-l o g i c a l problems of dense low income areas would be avoided and a f a i r e r basis found for sharing the f i s c a l and s o c i a l costs of dealing with urban poverty problems (Downs 73 ) Encouraged by several innovative mixed housing developments i n i t i a t e d under 19 70 and 1971 federal programmes, the NHA was amended i n 1973 to f a c i l i t a t e income mix by a s s i s t i n g low income families to obtain accommodation i n p r i v a t e l y constructed housing projects. Ontario, for example, responded p o s i t i v e l y to t h i s option by i n i t i a t i n g the "Inte-74 grated Community Housing Programme" designed to mix public housing with private i n about 4,000 units of multiple housing 75 during 1973. Reaction by Ontario builders has been favour-76 able but cautious. Emerging l e g i s l a t i o n to encourage heterogeneous, as opposed to homogeneous, neighbourhoods rests on the concern 77 for the undemocratic nature of e x i s t i n g segregation. Housing mix i s viewed as a means of reintegrating segments of society into the mainstream of urban opportunities. Integra-t i o n i s seen to enrich l i v e s , to promote tolerance through exposure to alternate l i f e s t y l e s , to decrease stigma attached to receiving housing assistance and to broaden the educational experiences of children. Such p o l i c i e s have been enacted 103 through the housing sector on the assumption that people's l i v e s are focussed on a fixed neighbourhood and, as such, the s o c i a l character of that environment w i l l a f f e c t the r e s i d i n g households. ^8 E x i s t i n g housing l i t e r a t u r e provides c o n f l i c t i n g opinions on the impact of dispersing low income households into middle income communities. Festinger (50) and Feagin (72) support emerging housing p o l i c i e s by concluding that s o c i a l i n t e r a c t i o n i s enhanced through heterogeneity. In the new neighbourhood, our data indicate that the subsidy families joined i n l o c a l a f f a i r s f a s t e r and more a c t i v e l y than poor families moving into some other types of housing, and that the net e f f e c t of the move was a heavier involvement i n friendship a c t i v i t i e s than they had c a r r i e d on before. We could detect no signs that the higher-income neighbours of the subsidy families singled them out as a s p e c i a l group, or that any sp e c i a l f r i c t i o n s developed from the neighbouring of households with d i f f e r e n t income l e v e l s . (Feagin 72, p. 145). That these studies examined heterogeneous settings i s open to question. Festinger's study l o c a l e was a student housing project. Feagin, himself, points out that Income differences were modest, and the so-c a l l e d middle income families had seen enough of the housing problems of poor blacks to be quite favourable to the general p r i n c i p l e of d i r e c t rent subsidization. (Feagin 72, p. 145). Challenges to the assumption that proximity to hetero-geneous households provides p o s i t i v e benefits are seen i n the research of Gans (61, 67), Gutman (66), Wilder (62), and Gruen (72). A l l found l i t t l e or no r e l a t i o n s h i p between l o c a t i o n 104 of d i s s i m i l a r households and s o c i a l contacts. A c l a s s i c study by Gans i n the Levittown area suggests that mixing does not always occur. Working class families l i v i n g alongside middle class ones went t h e i r own way most of the time. (Gans 67, p. 170). His conclusion to a 1961 a r t i c l e i n the Journal of the American In s t i t u t e of Planners was that heterogeneity advocated by planners was unpopular with homeowners and tenants a l i k e . Gruens' work i n Dayton, Ohio substantiates Gans' findings. They state that the majority of low and moderate income residents would expect economically d i s s i m i l a r neigh-bours to have the problems Gans discovered when he studied Levittown. It appears that friendship r e l a t i o n s h i p s con-tinue to depend on more than physical proximity ....The l o c a l answer has been that i n t e g r a t i o n cannot succeed unless the class l e v e l and customs of the two groups are approximately equal. (Gruen & Gruen 72, p.134). Their comparative studies of housing mix elsewhere i n the United States disclosed few examples of successfully integrated areas. The majority of those that do e x i s t are located i n the inner c i t y . Why s o c i a l mix has not been successful i s explained i n terms of established resident resentment and a preference for homogeneity among many people with s i m i l a r l i f e s t y l e s . Established residents are frequently suspicious of housing mix proposals. Concern focuses on the p o t e n t i a l impact to private property values and l o c a l services. This concern was not substantiated by the Gruen study. They found the placement of 105 lower income housing i n middle income neighbourhoods need not cause the price of e x i s t i n g housing to drop i f new b u i l d i n g programmes are accompanied by adequate funds for expansion and maintainance of neighbourhood amenities. The preference for homogeneity i s found i n a va r i e t y of studies. Michelson (71) suggests that to a t t a i n r e s i d e n t i a l s a t i s f a c t i o n people leave what they see as heterogeneous set-tings for s o c i a l l y more homogeneous ones. Heterogeneity and homogeneity being seen as operative factors i n the choice pro-cess. C i t i z e n s attending public meetings sponsored by GVRD to formulate goals for Vancouver (1973) indicated that while they accept the theory that housing mix i s good, they prefer to keep t h e i r own neighbourhoods homogeneous. Gruen and Gans both i d e n t i f i e d concern on the part of lower income residents for dispersed housing programmes. The Gruen study found that many low income families did not wish to migrate to suburban areas. Many recognize that the public and private services, and the friendships they need are available primarily i n the central c i t y . . . . The recognize quite r e a l i s t i c a l l y the extra costs involved i n a move to suburbia. (Gruen 67, p. 167). S i m i l a r i l y , Gans i n his Levittown study found Income s i m i l a r i t y i s valued by the less a f f l u e n t , not as an end i n i t s e l f , but because people who must watch every penny cannot long be comfortable with more aff l u e n t neighbours, p a r t i c u l a r l y when children come home demanding a toy or clothes they have seen next door. (Gans 67, p. 167). 106 Not only does the l i t e r a t u r e i d e n t i f y the preference for homogeneous neighbourhoods, also indicated are p o t e n t i a l s o c i a l problems to be faced by the dispersed lower income resident. Suggested implications of the detrimental e f f e c t s of mixing are developed by Rose (69). He notes the i s o l a t i o n of lower income families placed i n suburban areas where a v a i l a b l e r e s i d e n t i a l s i t e s are marginally located for transport, ameni-t i e s and services. The more c r i t i c a l implications of physical 79 and s o c i a l i s o l a t i o n are f e l t by the women. Gutman (63) found that working class wives, lacking necessary s o c i a l s k i l l s , had considerable d i f f i c u l t y i n adjusting to a mixed class sub-d i v i s i o n i n New Jersey. In f a c t , evidence as gathered from new towns and-housing estates throughout the world suggest that mixing groups may a c t u a l l y lead to h o s t i l i t y and con-f l i c t rather than to a more i n t e r e s t i n g and varied community l i f e . (Keller 66, p. 504). A review of e x i s t i n g housing theory suggests that the premise that dispersion of s o c i a l housing into middle income communities w i l l equalize access to urban amenities and, i n the process, provide an "improved" l i f e s t y l e for lower income Canadians i s open to question. I t may well be that undue emphasis on physical design and random placement of lower income households r e s u l t s i n increased s o c i a l and s p a t i a l i s o l a t i o n rather than any intended p o s i t i v e e f f e c t s . As for s o l i d i f y i n g a s p l i n t e r e d society, we must ask i f housing i s indeed the area i n which these problems or i g i n a t e . 107 To undertake such s o c i e t a l change o b j e c t i v e s through the housing s e c t o r g e n e r a l l y , or home ownership programmes s p e c i f i c a l l y , may w e l l be an example of a f a i l u r e to c l e a r l y i d e n t i f y the p r o b l source. The L e g i s l a t i v e B a s i s o f the GVRD Programme; GVRD's "300 U n i t Programme" i n c o r p o r a t e s w i t h i n i t , e i t h e r e x p l i c i t l y i n the l e g i s l a t i o n o r by i m p l i c a t i o n , s e v e r a l emerging Canadian housing p o l i c y d i r e c t i o n s . These i n c l u d e : 8 0 (1) The move toward ownership f o r low income groups, (2) The move away from c o n c e n t r a t i o n of lower income a s s i s t a n c e on c e r t a i n areas o f the c i t y and toward ,. . 8 1 d i s p e r s i o n , (3) The t r e n d toward a s s i s t i n g denser forms o f s i n g l e f a m i l y home ownership i n the form o f town or row 8 2 houses under condominium tenure, and, by i m p l i c a t i o n , i n c l u d e s (4) The move toward concern f o r the broader r e s i d e n t i a l 8 3 environment r a t h e r than the i n d i v i d u a l u n i t , (5) The need f o r c o n s u l t a t i o n w i t h the user as p a r t o f 84 the housing p r o c e s s , and (6) The need f o r more f l e x i b l e F e d e r a l - P r o v i n c i a l co-8 5 o p e r a t i o n i n implementing housing programmes. Canadian Governments have f l u c t u a t e d from o p p o s i -t i o n t o a s s i s t i n g home ownership f o r lower income households t o , i n 1972, advocating l e g i s l a t i o n s p e c i f i c a l l y to t h i s end. I n i t i a l o p p o s i t i o n t o p r o v i d i n g t h i s form o f tenure to lower income f a m i l i e s e v o l v e d from: 108 : Reluctance to ask some people i n the community to pay for the a c q u i s i t i o n of assets by others. : Anticipated costs to the government of such programmes. : I n a b i l i t y to f i n d any conceptual difference between rental-purchase schemes and subsidized r e n t a l schemes. : The b e l i e f that sale, construction, and lending a c t i v i t i e s associated with home ownership should remain a private concern. In addition, during the l a t e r 1950's and early 1960's, when incomes were r i s i n g more ra p i d l y than carrying costs on single family homes, an increasing number of households i n the lower t h i r d income bracket could a f f o r d standard NHA financing. While the government was w i l l i n g to lend at a t t r a c t i v e rates to p o t e n t i a l homeowners there was l i t t l e popular pressure f o r further assistance. Pressures for Government Intervention By the la t e 1960's conditions had changed markedly. As land costs, i n t e r e s t rates, and property taxes rose, i t became increasingly d i f f i c u l t for middle income households, l e t alone those i n the lower income brackets, to purchase single family homes. Table 6 i l l u s t r a t e s the changing amount of assistance to lower income households. TABLE 6 NHA ASSISTANCE TO LOWER INCOME HOUSEHOLDS 1954 6.6% NHA borrowers from lower 1/3 income bracket 1963 10.4% " 1969 15.4% " " 1971 34.6% " 1973 25.0% " " " (Canadian Housing S t a t i s t i c s 73) 109 P r o v i n c i a l Actions: P r o v i n c i a l and, l a t e r , federal governments responded to increased need. B r i t i s h Columbia has had a history of 8 6 advocating some form of assistance to home owners. In 1967 87 8 8 B.C. adopted a Ca p i t a l Grants and Second Mortgage programme to encourage home ownership. As these grants were not t i e d to income they were not s p e c i f i c a l l y directed toward a s s i s t i n g 89 lower income households. Other provinces experimented with i n t e r e s t subsidies, land cost subsidies, s h e l l housing, sweat equity, and sales tax provisions to encourage ownership. Federal Actions: The Federal government was encouraged to turn to assisted home ownership programmes by the findings of the Hellyer Task Force and comparison of experiences i n U.S. housing programmes. The Hellyer Task Force (69) i d e n t i f i e d problems created by large public housing projects where concentrations of persons on low incomes created stigma and co s t l y s o c i a l problems. Hellyer recommended that ownership options would remove uncertainty concerning duration of tenure and encourage occupier involvement i n b u i l d i n g maintenance. In addition/ ownership was seen to provide c e r t a i n administrative advan-tages : : In l i g h t of r i s i n g public housing subsidies ownership provides a way to reduce or control subsidy l e v e l s by t r a n s f e r r i n g many of the costs to the home buyer. 110 : Administrative problems are reduced as there i s no long term r e l a t i o n s h i p between buyer and developer. : The Government can continue to r e l y on the private market to construct units. Government actions remain f i n a n c i a l . : Ease Federal-Provincial r e l a t i o n s as the Provinces are seen to prefer AHOP over r e n t a l schemes. Experience i n the United States: The move toward a s s i s t i n g home ownership i n the United States was stimulated by evidence of need by both lower and middle income groups. Ghetto r e b e l l i o n s spurred Congress i n 196 8 to adopt home ownership for the poor as a national p o l i c y r e a l i z i n g that If low and moderate income households are to be able to occupy new homes...they w i l l need d i r e c t subsidies, because they have neither the income nor the c r e d i t to pay the costs of bui l d i n g new homes. (Gruen 72, p. 7).90 In the same year, Congress sensed that housing demands i n the 1970's would be strong and that without federal assistance many middle income fa m i l i e s , not otherwise considered i n need of aid, would f i n d themselves unable to secure adequate . . . 91 housing. I n i t i a l approval of assi s t e d home ownership was passed through Congress with l i t t l e opposition. The proposals 92 were backed by the r e a l estate industry and promoted through such American "ideals" as home ownership and free enterprise. 9 3 By 1970, the U.S. programme had assisted 131,000 households and seemed more l i k e l y to meet i t s 1.3 m i l l i o n s t a r t s goal by 1978 than w i l l be true for other subsidy programmes. Early evaluations of the U.S. lower-income home I l l home ownership programmes indicate several problems: (1) A 19 70 congressional report exposed corruption and . . 94 i n e f f i c i e n c y i n the programme. The report was concerned with the cost implications of a programme which l e f t the d i s c r e t i o n of house choice i n the hands of the r e c i p i e n t but gave no 95 incentive to the buyer to be concerned over p r i c e . (2) As i n a l l programmes where funds are l i m i t e d , the government must make a choice of whether to a s s i s t a few of the most needy with large i n d i v i d u a l fund a l l o c a t i o n s or to aid a larger number of less needy households. American AHOP undertook the l a t t e r course - thus r a i s i n g the question of funding some of the less needy at the expense of those most i n need of assistance. (3) D i f f e r e n t i a l assistance i s also r e l a t e d to place of residence. As housing costs fluctuate by regions, the income 96 necessary to apply for units v a r i e s . To date, the programme has worked best i n c i t i e s where an adequate number of homes are available for r e h a b i l i t a t i o n . But as land and construction costs r i s e , the programme w i l l be increasingly inapplicable i n some regions. Implementation of Canadian L e g i s l a t i o n : Based upon Canadian conditions i d e n t i f i e d by Hellyer and the U.S. experience, the Minister responsible for Housing, Robert Andras , introduced ownership options for lower income Canadians through the inauguration i n 19 70 of a $200 M i l l i o n Innovative Housing Programme. 112 The $200 M i l l i o n Programme was designed to provide housing which would reach further down the income scale ( s p e c i f i c a l l y to persons i n the $4,000-6,000 income range) through innovative construction and financing proposals. In s e l l i n g units, the government showed a major departure from pre 19 70 practice when ownership was not considered for lower 97 xncome f a m i l i e s . 9 8 Section 58: Despite some problems home ownership experiments under the Innovative Programme interested CMHC i n i n i t i a t i n g an assisted home ownership programme. In 1971 AHOP 99 was provided under Section 58 of the NHA as an a l t e r n a t i v e to public r e n t a l housing. In comparison to previous p o l i c i e s , AHOP provided ownership as opposed to r e n t a l units. Ownership options for families were to be located i n ground oriented units as opposed to high r i s e apartments and to be located throughout the urban area. The programme r e l i e d on the mechanism of lowering i n t e r e s t rates and extending amortization periods on new housing. Under these guidelines, 7,510 units involving loans to $9 8 m i l l i o n were approved during 1971. The 1971 programme was ostensibly a continuation of the 19 70 innovative programme. However, unemployment conditions in Canada i n October 19 71 encouraged the government to u t i l i z e the housing sector to s t a b i l i z e the economy through increased construction a c t i v i t y . Programme rules were changed. Maximum incomes to q u a l i f y were raised to $7,000 and assistance to the lowest income groups s a c r i f i c e d to economic s t a b i l i z a t i o n . 113 Assisted Home Ownership Programme: Obtaining neither 102 an evaluation by i n i t i a l AHOP rec i p i e n t s nor a clea r state-ment of human and s o c i a l aspects of low income housing problems, the federal government moved i n June 1972 to amend the NHA to include, i n Part IV, loans to f a c i l i t a t e home ownership for low income households. The rati o n a l e for movement i n th i s d i r e c t i o n appears to have evolved from the b e l i e f that AHOP would be less costly than public housing by removing management costs associated with r e n t a l programmes and passing maintainance costs to the purchaser. The proposal was met with mixed re-10 3 actions. The Dennis report indicated concern for the continual emphasis on quantity, the use of the market as the provider of 104 programme units, the inherently i n f l a t i o n a r y nature of the 105 programme, and the attendant problems for the low income recipient."'" 0*' Dennis recommended against AHOP i n favour of co n t r o l l i n g housing costs through extensions of non-profit r e n t a l programmes. AHOP should be li m i t e d to r u r a l areas and small towns where re n t a l i s less acceptable and pri c e i n f l a -t i o n less of a problem. 107 Following consultations with the provinces, AHOP became an amendment to the NHA i n 1973. The objectives of the new section were seen to be: 1. To extend the range of families able to enter the home ownership market, where a supply of moderately priced homes i s ava i l a b l e . 114 2. To present low income families with an alter n a t i v e to re n t a l or public housing. This does not mean that the program w i l l eliminate the need for assisted r e n t a l or for deep subsidies but i t of f e r s a choice to those families who can af f o r d home ownership with the assistance offered. 3. To encourage the bu i l d i n g industry to produce modestly-priced housing for low income families by the introduction of a continuing program. (Minister's B r i e f i n g Paper, CMHC, January 9, 1973). The Programme attempts to implement these goals by lowering i n t e r e s t rates, extending amortization periods and providing monthly subsidies to a s s i s t families with mortgage payments. The "300 Unit Programme" i s thus one example of emerging Canadian housing p o l i c y trends. On the assumption that an e x i s t i n g s o c i a l action programme i s a v a l i d research s e t t i n g to tes t the d i s s e r t a t i o n concern and that the "GVRD 300 Unit Assisted Home Ownership Programme" i s a suitable example of a multi-dimensional, ongoing programme, the remain-der of the chapter considers a methodology appropriate for a user oriented evaluation of a s o c i a l action programme. 115 PART I I CONCEPTUALIZATION AND TESTING OF A  USER ORIENTED EVALUATION METHODOLOGY A concern has been i d e n t i f i e d t h a t p r e s e n t e v a l u a t i v e procedures are inadequate f o r the e v a l u a t i o n o f s o c i a l a c t i o n programmes from the p e r s p e c t i v e both o f the user's needs and the involvement o f the user i n the e v a l u a t i v e p r o c e s s . However, recommendations f o r the replacement o f t r a d i t i o n a l programme e v a l u a t i o n models w i t h the proposed user o r i e n t e d e v a l u a t i o n model should r e s t on the demonstrated comparative advantage o f the new p r o c e s s . Thus two e v a l u a t i o n s o f one programme were proposed - one from the t r a d i t i o n a l p e r s p e c t i v e o f a d m i n i s t r a -t i o n d e f i n e d g o a l s r e l a t e d t o programme output and an a l t e r n a t e e v a l u a t i o n , o r i g i n a t i n g from user concerns and e v a l u a t i n g the exten t t o which the s o c i a l a c t i o n programme f u l f i l l s r e c i p i e n t e x p e c t a t i o n s . The two e v a l u a t i o n s have been compared (see Diagram7 ) on the b a s i s o f a v a r i e t y o f c r i t e r i a i n c l u d i n g c o n c e p t u a l i z a -t i o n o f i s s u e s and s o l u t i o n s , a m e n a b i l i t y o f p r o p o s a l s to p o l i c y i n t e r v e n t i o n , concern w i t h s o c i a l t e n s i o n s and a s p i r a -t i o n s , and the exte n t to which broader p o l i c y a l t e r n a t i v e s are p e r c e i v e d by p o l i c y makers. Recommendations f o r the replacement of t r a d i t i o n a l models w i t h a user o r i e n t e d model r e f l e c t comparative advantages emerging from the two e v a l u a t i o n com-p a r i s o n . FRCGRAM EVALUATION plica e 7 I d e n t i f i c . program dimensions of METHODOLOGY TEST Cempale, cxposjt and pKCQKcon uAQA Awareness-i d e n t i f i c a t i o n concerns phaAd 1 Prepare-pre test evaluative t o o l s Pre-test assistance pkatc, 3 Program evaluation Data C o l l e c t plicae. 4 P o l i c y -program reformulation plicae. 5 Program evaluation report to CMHC,GVRD 7 9 A b i l i t y to Evaluate program impact A b i l i t y to reformulate p o l i c y Concha ion P o t e n t i a l c o n t r i b u t i o n to evaluation A DIAGRAM 7 A PROPOSED GUIDELINE FOR EVALUATION OF THE GVRD SOCIAL ACTION PROGRAM SUCH THAT THE POTENTIAL CONTRIBUTION OF PROGRAM RECIPIENTS CAN BE ESTABLISHED AND ASSESSED AT VARIOUS STAGES IN THE EVALUATIVE PROCESS. 117 If agency personnel are found to possess the a b i l i t y to i d e n t i f y user concerns the proposal to include the programme r e c i p i e n t i n the evaluative process w i l l be rejected. I t w i l l be recommended that e x i s t i n g evaluative procedures be retained i n the i n t e r e s t of e f f i c i e n c y and economy. I f , however, the contributions of the two groups d i f f e r , the assumption that the user can provide a d i s t i n c t i v e input into the evaluation process w i l l be considered sub-stantiated. The suggestion w i l l be made that the viewpoint of the user or r e c i p i e n t of a programme should form an i n t e g r a l part of any planning process which seeks to tackle programme evaluation and p o l i c y reformulation. EVALUATION ONE: TRADITIONAL EVALUATION RESEARCH DESIGN The t r a d i t i o n a l i n s t i t u t i o n a l evaluative research design was shown i n Chapter II to involve a process of p o l i c y reformulation based upon s e l e c t i o n of goals, c o l l e c t i o n , and analysis of data by programme administrators. A research design was selected to as c l o s e l y as possible repeat t h i s expert centered evaluation process on the GVRD "300 Unit Pro-gramme" . Determination of programme goals for evaluation was obtained from l i t e r a t u r e review of documents related to the "300 Unit Programme", AHOP Guidelines, and through lengthy interviews with o f f i c i a l s at CMHC and GVRD concerned with the i n i t i a t i o n and administration of the programme. O f f i c i a l s were asked to i d e n t i f y what they saw as key variables requiring analysis. 118 Based upon programme o b j e c t i v e s and problem areas i d e n t i f i e d by o f f i c i a l s , an i n t e r v i e w schedule was c o n s t r u c t e d along the g u i d e l i n e s o f the CMHC $200 M i l l i o n D o l l a r I n n o v a t i v e low c o s t housing e v a l u a t i o n d i s c u s s e d i n Chapter I I . O f f i c i a l s were r e i n t e r v i e w e d to o b t a i n t h e i r p r e s e n t knowledge o f , and o p i n i o n s on, programme o p e r a t i o n s and impact. Interviews were conducted both s e p a r a t e l y and i n group s e s s i o n s ts s t i m u l a t e i n t e r e s t i n the e v a l u a t i o n and i n c r e a s e commit-ment to.the implementation of f i n d i n g s through the p r e s s u r e s generated by s m a l l groups. In a d d i t i o n , t o i n c r e a s e r e l i a b i l i t y , a v a r i e t y o f p o l i t i c i a n s , both l o c a l and f e d e r a l , were q u i z z e d c o n c e r n i n g t h e i r knowledge o f , and concerns r e l a t e d t o , a s s i s t e d home ownership. T h e i r c o n c l u s i o n s formed a f u r t h e r s u b s t a n t i a t i o n of i n - d e p t h i n t e r v i e w f i n d i n g s . Based upon the i n f o r m a t i o n p r e s e n t e d by o f f i c i a l s , a t r a d i t i o n a l e v a l u a t i o n o f the programme was c o n s t r u c t e d t o i n d i c a t e programme impact and s u g g e s t i o n s f o r r e f o r m u l a t i o n . T h i s r e p o r t was completed and f i l e d without p u b l i c d i s c u s s i o n pending the user o r i e n t e d e v a l u a t i o n f i n d i n g s . EVALUATION TWO; A USER ORIENTED EVALUATION PROCESS C i t i z e n s are p r e s s i n g f o r more e x t e n s i v e p a r t i c i p a -t i o n a t a l l l e v e l s o f p o l i c y f o r m u l a t i o n and implementation. However, as documented i n Chapters I and I I , involvement of users i n programme e v a l u a t i o n and p o l i c y r e f o r m u l a t i o n i s not 119 customary. The a n t i c i p a t e d c o s t s o f c i t i z e n p a r t i c i p a t i o n and the b e l i e f t h a t c i t i z e n s do not n e c e s s a r i l y possess the com-petence t o a s s i s t p l a n n e r s have been p r e s e n t e d as j u s t i f i c a t i o n f o r e x c l u d i n g programme r e c i p i e n t s from the e v a l u a t i o n p r o c e s s . The q u e s t i o n a r i s e s as to whether t h i s l a c k o f c l i e n t involvement i s j u s t i f i e d . The competence i s s u e has been c h a l -lenged by s e v e r a l s t u d i e s . Sharkansky (70) f i n d s t h a t c i t i z e n s may be e s p e c i a l l y w e l l informed -even about complex i s s u e s - when they are debated by c a n d i d a t e s f o r important o f f i c e s ; d e s p i t e the obscure nature of tax p r o p o s a l s people i n d i f f e r e n t income b r a c k e t s have i d e n t i f i e d and supported those f e a t u r e s o f the d i f f e r e n t p r o p o s a l s which o f f e r e d them the g r e a t e s t economic b e n e f i t s . (Sharkansky 70, p. 11 ). Evidence o f c i t i z e n c o n t r i b u t i o n s to o t h e r stages o f the p l a n -n i n g process i n d i c a t e t h a t u s e r s may i n f a c t add v a l u a b l e i n s i g h t s i n t o programme o p e r a t i o n based upon e x p e r i e n c e , as opposed to t h e o r e t i c a l l y d e r i v e d data. An a l t e r n a t i v e e v a l u a t i o n process p r o v i d i n g f o r d i r e c t involvement of programme r e c i p i e n t s i n the f o r m u l a t i o n of e v a l u a t i o n g o a l s , data c o l l e c t i o n , a n a l y s i s o f problems and programme r e f o r m u l a t i o n was c o n c e i v e d and t e s t e d i n an attempt to assess the h y p o t h e s i s t h a t user involvement i n p o l i c y e v a l u a -t i o n can form a v i a b l e approach to p o l i c y a n a l y s i s . The r e s e a r c h d e s i g n was s e l e c t e d such t h a t a comparison c o u l d e x i s t between user and t r a d i t i o n a l e v a l u a t i o n p r o c e s s e s and t h a t the r o l e and r e s p o n s i b i l i t y o f the user c o u l d be an a l y z e d at v a r i o u s stages i n the e v a l u a t i v e p r o c e s s . 120 FOCUSSING ON THE PROGRAMME RECIPIENT The proposed user o r i e n t e d e v a l u a t i o n r e l i e d h e a v i l y upon behaviour, a t t i t u d e s and o p i n i o n s expressed by programme r e c i p i e n t s . For the purpose of t h i s d i s s e r t a t i o n behaviour was seen to express under p a r t i c u l a r c ircumstances the use a per-son makes of an o b j e c t , w h i l e a t t i t u d e i n d i c a t e d what a person t h i n k s of the st i m u l u s o b j e c t and o p i n i o n s t a t e d whether the person would l i k e to see more or l e s s of the o b j e c t . Behaviour, a t t i t u d e and o p i n i o n data were seen to p r o v i d e complementary i n f o r m a t i o n f o r programme e v a l u a t i o n . B e h a v i o u r a l data was used to form the b a s i s f o r models of pr e s e n t programme use p a t t e r n s . An i n d i c a t i o n of s a t i s f a c t i o n e x p e r i e n c e d through such p a t t e r n s r e q u i r e d a t t i t u d i n a l comments. Suggestions f o r programme change demanded o p i n i o n s . A s s e s s i n g R e c i p i e n t Behaviour The b e h a v i o u r a l approach t o housing r e s e a r c h t a k e s , f o r example, the nature and dynamics of people's movements i n space and time as o b j e c t i v e i n d i c a t o r s of household p a t t e r n s ( B r a i l 69). In programme e v a l u a t i o n b e h a v i o u r a l data formed the b a s i s f o r de t e r m i n i n g the household's r e l e v a n t environment and f o r comparing programme e f f e c t s w i t h those generated by normal market p r o c e s s e s . There are numerous precedents f o r t a k i n g a h e h a v i o u r a l approach t o housing. M i c h e l s o n (72) , L a n s i n g (70), Gans (61) , Buttimer (72) and B r o l i n and Z e i s e l (68) are but a few of the r e s e a r c h e r s who i d e n t i f y the advantages of b e h a v i o u r a l 121 d a t a . Weiss (72, p. 40) i n d i s c u s s i n g the methodology of e v a l u a t i o n r e s e a r c h recommended b e h a v i o u r a l a n a l y s i s on the grounds t h a t changes i n a t t i t u d e s are not n e c e s s a r i l y s u f f i c i e n t to change behaviour. Deutscher (69) a l s o argues i n favour of d i r e c t b e h a v i o u r a l measures on the grounds t h a t they pose fewer v a l i d i t y problems than do procedures designed to p r o v i d e estimates o f h y p o t h e t i c a l behaviour. In a p o l i c y c o n t e x t , both L a n s i n g (70) and Freeman (70) found t h a t s t u d i e s o f user a c t i o n s encountered l e s s c r i t i c i s m from p o l i c y makers than d i d r e s e a r c h which asked people to respond to h y p o t h e t i c a l s i t u a t i o n s . CMHC p o l i c y development s t a f f member David Crenna (70) s u b s t a n t i a t e s t h i s stand by d i s m i s s i n g v a l u e s l i t e r a t u r e as b e i n g f u l l o f con-c e p t u a l b l i n d a l l e y s and responding to data based on "how people a c t u a l l y respond to the w o r l d " . A s s e s s i n g R e c i p i e n t A t t i t u d e s Having o b t a i n e d a p r o f i l e o f the household's c u r r e n t housing s i t u a t i o n through the use of b e h a v i o u r a l data, a second concern was to determine how household members e v a l u a t e t h e i r s i t u a t i o n . While a r e s e a r c h e r can a s c e r t a i n the contents o f an o b j e c t i v e environment by. i n s p e c t i o n o r a c t i v i t y r e c o r d s , e v i -dence o f use does not n e c e s s a r i l y i n d i c a t e user s a t i s f a c t i o n . The assessment of the s u b j e c t i v e environment r e q u i r e an awareness of the meanings t h a t environment h o l d s f o r the user. To c o l l e c t such i n f o r m a t i o n n e c e s s i t a t e d methods used i n a t t i t u d e r e s e a r c h . 122 In t h i s case a t t i t u d e was taken to be an enduring o r g a n i z a t i o n of m o t i v a t i o n a l , emotional, p e r c e p t u a l and c o g n i t i v e p r o c e s s e s w i t h r e s p e c t t o some asp e c t of the i n d i v i d u a l ' s world. (Scott 68 , p.204 ). A person's a t t i t u d e s are guided to a l a r g e e x t e n t by g e n e r a l , a b s t r a c t g o a l s or r u l e s which they have accumulated through pa s t experiences and a s s o c i a t i o n s . However, Kates and Wohwill (6 6) suggest t h a t i n d i v i d u a l e x p e r i e n c e s do not d i s p l a y a simple l i n e a r r e l a t i o n s h i p t o p e r c e p t i o n and a t t i t u d e s . While a t t i -tudes are enduring i n the sense t h a t r e s i d u e s are c a r r i e d over to new s i t u a t i o n s , they are s u b j e c t to change i n t h a t new a t t i t u d e s are a c q u i r e d through e x p e r i e n c e . Whether a t t i t u d e s change o r p e r s i s t i s s u b j e c t to the i n t e r a c t i o n of i n t e r n a l and e x t e r n a l i n f l u e n c e s . For t h i s reason an a n a l y s i s o f a household's a t t i t u d e s t o t h e i r housing s i t u a t i o n s h o u l d i n c l u d e an i n d i c a t i o n of the change s u s c e p t a b i l i t y o f the householder as an i n d i c a t o r o f the p o s s i b l e w i l l i n g n e s s or r e s i s t a n c e to a d j u s t e d programme dimensions. Since a t t i t u d e s are h y p o t h e t i c a l c o n s t r u c t s they can-not be measured d i r e c t l y but must be i n f e r r e d from the s u b j e c t ' s responses. The correspondence between expressed statements and a c t u a l a c t s has caused concern i n both b e h a v i o u r a l and a t t i t u -d i n a l r e s e a r c h . Dolvern (6 8) compared h y p o t h e t i c a l answers on moving w i t h a c t u a l w i l l i n g n e s s t o a c t when c o n f r o n t e d w i t h a job o f f e r . He found statements and a c t i o n c l o s e l y r e l a t e d when the concern was p r e c i s e l y d e s c r i b e d and i n v o l v e d matters which i n t e r e s t e d the respondent. 123 108 Attitude measurement attempts to provide an i n d i -cation of both the d i r e c t i o n and magnitude of a p a r t i c u l a r 109 at t i t u d e . In measuring attitudes toward housing a d i f -f i c u l t y arises i n that i n most instances one i s interested i n the attitude not to a s p e c i f i c stimulus but rather to a s i t u a -t i o n . Just as any object may be the subject of a number of attitudes so may a s i t u a t i o n . In analyzing a s i t u a t i o n , tradeoffs, p r i o r i t i e s and, i n a household group, i n t e r a c t i o n , a l l come into play i n determining which attitudes w i l l be expressed. Due to t h i s complexity, the researcher must pro-vide a r e a l i s t i c environmental display and t e s t the same attitude i n a variety of ways. I t was assumed that in-home interviews provided an environmental display relevant to the i n d i v i d u a l homeowner's concerns. Despite the fact that techniques to measure attitudes toward the environment are not well developed, there has been an increasing use of a t t i t u d i n a l surveys by planners. Examples of attitude based research include Peterson (6 7), Sonnenfeld (66), Lansing (70), Buttimer (72), Michelson (72) and Hinshaw and A l l o t t (72). Assessing Recipient Opinions Having constructed a model of the household's current housing s i t u a t i o n through the use of behavioural data, and received, through a t t i t u d i n a l studies, an i n d i c a t i o n of house-hold s a t i s f a c t i o n with past, present, and future housing s i t u a t i o n s , the f i n a l concern was to obtain, from the user's 124 p e r s p e c t i v e , o p i n i o n s c o n c e r n i n g programme dimensions r e q u i r i n g r e i n f o r c e m e n t o r r e f o r m u l a t i o n . I f an a t t i t u d e expresses what a person t h i n k s o f a sti m u l u s o b j e c t , an o p i n i o n i n d i c a t e s whether the person would l i k e to see more or l e s s o f t h a t o b j e c t . As such, o p i n i o n s are b e s t regarded as a t e n t a t i v e p e r c e p t u a l s e t toward a sti m u l u s o b j e c t . I n d i v i d u a l households were p r o v i d e d w i t h some over-view data c o n c e r n i n g programme impact and asked f o r t h e i r e v a l u a t i o n o f the programme both from the vie w p o i n t o f t h e i r own household and from t h a t o f o t h e r persons who might be needing o r d e s i r i n g s i m i l a r housing a s s i s t a n c e . DATA COLLECTION - THE INTERVIEW Sin c e u n d e r l y i n g a t t i t u d e s and v a l u e s are c o n s i d e r e d r e s p o n s i b l e f o r o p i n i o n s expressed and a c t i o n s observed, and s i n c e no s i n g l e t o o l has been developed by the s o c i a l s c i e n c e s to adequately assess b e h a v i o u r a l , a t t i t u d i n a l and o p i n i o n r e s -ponses, a v a r i e t y o f techniques were used t o c o l l e c t and analyze data. Such an approach i s not unique. Mullen's (72) i n v e s t i g a t i o n o f v a r i o u s r e c e n t s t u d i e s i n d i c a t e s t h a t By and l a r g e r e s e a r c h p r o j e c t s chose the path of d i v e r s i f i e d measurement. (Mullen 72, p. 33). I d e a l l y , i t would be advantageous to study the housing s i t u a t i o n o f a household over time t o i d e n t i f y t r e n d s . T h i s approach has been used i n E d u c a t i o n and Psychology ( l o n g i t u d i -n a l method), Anthropology and S o c i o l o g y ( h i s t o r i c a l method) and Economics (time s e r i e s method). A l l are b r o a d l y termed the "long view". In a p o l i c y c o n t e x t "long views" are u n r e a l i s t i c . 125 Requests f o r programme r e f o r m u l a t i o n seldom p r o v i d e the time necessary to undertake e x t e n s i v e m o n i t o r i n g . Surrogate i n f o r -mation on programme impact over time i s a l t e r n a t i v e l y o b t a i n e d by s u b d i v i d i n g s u b j e c t s a c c o r d i n g to the d u r a t i o n o f exposure to the programme. To an e x t e n t t h i s i s a m o d i f i c a t i o n of the "wide view", c o h o r t , or c r o s s s e c t i o n method. The p r o p o s a l to employ p e r s o n a l i n t e r v i e w s , as opposed t o m a i l e d q u e s t i o n n a i r e surveys r e f l e c t s the d e t a i l e d nature o f the data, i t s amount, complexity and the d i f f i c u l t y , as C r a i k (68) has i n d i c a t e d , o f d e v i s i n g an environmental d i s -p l a y o u t s i d e the home-neighbourhood c o n t e x t which adequately i n d i c a t e s the nature and complexity o f the r e s i d e n t i a l e n v i r o n -ment. I t i s g e n e r a l l y b e l i e v e d t h a t p e r s o n a l i n t e r v i e w pro-v i d e s the h i g h e s t r e t u r n , but a t g r e a t e s t c o s t ; telephone i n t e r v i e w s s u f f e r from sample inadequacy and m a i l q u e s t i o n -n a i r e s are l e a s t expensive but u s u a l l y s u f f e r from lower r e t u r n s . Hochstim (67) compared m a i l , telephone and i n t e r v i e w data g a t h e r i n g techniques and concluded t h a t the t h r e e s t r a t e g i e s were p r a c t i a l l y i n t e r c h a n g e a b l e when compared a c c o r d i n g t o r a t e o f r e t u r n , completeness o f r e t u r n , c o m p a r a b i l i t y of f i n d i n g s and v a l i d i t y o f responses. Only the c o s t f a c t o r v a r i e d c o n s i d e r a b l y from one s t r a t e g y to another. He con-c l u d e d t h a t the l o g i c a l procedure i s to combine the b e s t f e a t u r e s o f each s t r a t e g y s t a r t i n g w i t h m a i l , a second wave by phone, and f o l l o w e d by p e r s o n a l i n t e r v i e w s to those who s t i l l f a i l t o respond. 126 In-home interviews provided the interviewer with an opportunity to d i r e c t l y evaluate the home environment. The language of c l i e n t s - so d i f f i c u l t to i n -corporate into the formalized vocabulary of the planner - i s t i e d to s p e c i f i c operational contexts. Its meanings s h i f t with changes i n the context, and i t s manner of expression i s frequently as important as the actual words employed. (Friedmann 73, p.174 ). U t i l i z i n g both open and closed format questions provided a range of data. Open ended questions e l i c i t the main p r i o r i t i e s which respondents define for themselves, while closed questions add d e t a i l s which do not i n i t i a l l y come to mind but which are neverthe-less part of the t o t a l range of considerations which enter into the housing process for most people. Taken together these two types of questions provide a comprehensive representa-t i o n of the respondent's reasoning. (Michelson 72, p. 23). On-site evaluation by one researcher enhanced the prospect of making v a l i d comparative judgments between households. Two stages of interviews were undertaken. The objective of the f i r s t phase of programme evaluation was to gain an over-view of perceived problems and to provide an i n d i c a t i o n of dimensions to be evaluated i n more d e t a i l . Caro (71, p. 26) suggests that exploratory research u t i l i z i n g informal i n t e r -active discussions with expert and c l i e n t groups i s a v a l i d technique to obtain, i n a short period of time, a variety of opinions. The small groups representing a variety of programme reci p i e n t s act as task oriented groups to discuss issues, problems and research d i r e c t i o n s . 127 > The second phase of i n t e r v i e w i n g i n v o l v e d a random sample of programme r e c i p i e n t s , and t h e i r 'next door n e i g h -bours' who acted as a matched comparison c o n t r o l group. Randomization p e r m i t t e d the r e s e a r c h e r to extend i n f e r e n c e s beyond the data to a c o n c e p t u a l u n i v e r s e of s i m i l i a r u n i t s . In the GVRD programme a sample frame e x i s t s which c l e a r l y i n d i c a t e s the e n t i r e u n i v e r s e . The c h o i c e of a s h o r t i n t e r v i e w but l a r g e sample s i z e was r e j e c t e d i n favour of a d e t a i l e d a n a l y s i s of a s m a l l e r number of households on the grounds of the c o m p l e x i t y o f the i s s u e s being d i s c u s s e d . A l l programme r e c i p i e n t s were e l i g i b l e f o r i n c l u s i o n i n the survey w i t h the e x c e p t i o n of those e n t e r i n g the programme d u r i n g the p r e c e e d i n g t h r e e months. T h i s r e s t r i c t i o n was p l a c e d to reduce the d i s t o r t i o n which might r e s u l t s i v e n t h a t the move i n t o a new home u s u a l l y i n v o l v e s an i n c r e a s e i n n e i g h b o u r i n g and e x p e n d i t u r e s which may not c o n t i n u e as the f a m i l y s e t t l e s i n t o i t s new s i t u a t i o n . (Gans 61). When the survey was conducted f a m i l i e s had been i n t h e i r new homes f o r p e r i o d s r a n g i n g from f o u r to twenty months. TABLE 8 DURATION OF PROGRAMME EXPOSURE Less than 1/2 year 18% 1/2 to 1 year 15% 1 to 1 1/2 years 43% 1 1/2 years or more 24% A s t r a t i f i e d random sample was drawn such t h a t r e c i p i e n t s were i n c l u d e d from each of the v a r i o u s p r o j e c t s and 128 from both one and two parent households. Education, ethnic background, occupation, and income were considered sub-s t a n t i a l l y s i m i l a r . Further s t r a t i f i c a t i o n along socio-economic l i n e s was not considered necessary. In t o t a l , 42 out of 6 3 recipi e n t s were interviewed i n t h e i r homes i n two to four hour interviews. In a l l cases interviews were preceded by a personal l e t t e r from programme administrators explaining the purposes of the evaluation and requesting assistance. In most instances interviews were con-ducted with the head of the household. Where possible both adults were present i n two parent f a m i l i e s . The response rate approached 100 percent as only one family requested not to be interviewed on the grounds that t h e i r English was inadequate. Interviews were designed to provide answers to spe-c i f i c questions on family housing patterns and values and to o f f e r the opportunity for householders to f r e e l y discuss a varie t y of housing and personal concerns. Questions r e l a t i n g to s p e c i f i c programme dimensions of ownership costs, s o c i a l mix, and involvement i n the planning process offered an opportunity to compare programme r e c i p i e n t patterns with e x i s t i n g theory and to determine the extent to which r e c i p i e n t s could discuss a variety of personal issues and suggest r e f o r -mulation proposals. Available data on the family and economic s i t u a t i o n of each household at the time of ap p l i c a t i o n pro-vided a basis for comparing changed circumstances f o l -lowing programme exposure. 129 R e l i a b i l i t y o f data c o l l e c t e d , or the a b i l i t y t o y i e l d the same r e s u l t s i f the i n t e r v i e w was repeated, was heightened through the use of s p e c i f i c d i r e c t e d q u e s t i o n s and employment of the same i n t e r v i e w e r f o r a l l data c o l -l e c t i o n . V a l i d i t y o r the a b i l i t y t o measure what one t h i n k s one i s measuring was heightened through the use of s e v e r a l p i l o t t e s t s , a v a r i e t y o f approaches t o the same broad con-cer n and r e f e r r a l o f f i n d i n g s and r e f o r m u l a t i o n p r o p o s a l s back t o i n t e r v i e w e e households f o r v e r i f i c a t i o n . I n t e r v i e w s are not without t h e i r problems. In a sense housing e v a l u a t o r s are caught i n a c o n f l i c t . Many o f the t o p i c s d i s c u s s e d are of a complex and p e r s o n a l nature and n e i t h e r e a s i l y r e c o g n i z e d nor communicated through u n o b t r u s i v e measures or m a i l q u e s t i o n n a i r e s . However, the very p e r s o n a l nature o f the data makes d i s c u s s i o n d i f f i c u l t and p o t e n t i a l l y embarrassing t o the i n t e r v i e w e e . Awareness of b e i n g t e s t e d may e l i c i t c e r t a i n response s e t s which the i n t e r v i e w e e f e e l s are " d e s i r e d " . Data c o l l e c t i o n may a l s o a r t i f i c i a l l y enhance c l i e n t awareness of the programme thereby a l t e r i n g apparent or a c t u a l e f f e c t i v e n e s s . Webb (66) suggests t h i s problem may be s o l v e d by u s i n g u n o b t r u s i v e measures which d i s g u i s e the r e l a t i o n s h i p between data c o l l e c t i o n and s p e c i f i c problem. Unobtrusive measures attempt t o r e c o r d under n a t u r a l c o n d i t i o n s the r e a c t i o n o f people t o a p a r t i c u l a r s t i m u l i . They minimize the Hawthorne e f f e c t and do not c r e a t e i n the respondent r a i s e d e x p e c t a t i o n s 130 f o r improvements which the r e s e a r c h e r can, i n f a c t not p r o -duce. Two problems l i m i t the use of u n o b t r u s i v e methods i n housing e v a l u a t i o n . F i r s t we are a s s e s s i n g a s i t u a t i o n r a t h e r than a p a r t i c u l a r S — R sequence. How the s u b j e c t a c t s may r e f l e c t a m u l t i t u d e of f a c t o r s . Secondly, mere evidence of use does not n e c e s s a r i l y imply user s a t i s f a c t i o n w i t h p r e s e n t p a t t e r n s . S t u d i e s by Hyman, Wright and Hopkins (62) i n d i c a t e t h a t s e n s i t i z i n g e f f e c t s through t e s t i n g a re o f t e n n e g l i g i b l e . S i nce e v a l u a t i o n helps to focus a t t e n t i o n of people on what they are doing, e v a l u a t i o n i n i t s e l f may a c t as a change agent. Loecks (70) i n d i c a t e s as a r e s u l t of such d i s c o u r s e comes a deeper awareness of the i n t e r d e p e n d e n c i e s of competing i n t e r e s t groups. By the pro c e s s of s o c i a l i n t e r c h a n g e i n d i v -i d u a l s who p r e v i o u s l y were c o n s t r a i n e d by t h e i r own s e l f i n t e r e s t b egin to pay some a t t e n t i o n t o the i n t e r e s t s of o t h e r s . However, i n the case of problems a s s o c i a t e d w i t h income mix, extreme care was taken to i n s u r e t h a t e v a l u a t i o n d i d not c r e a t e p r e v i o u s l y n o n - e x i s t e n t or un p e r c e i v e d problems. OVERCOMING THE LIMITATIONS OF TRADITIONAL EVALUATION RESEARCH The task of f o r m u l a t i n g a user o r i e n t e d e v a l u a t i o n methodology was approached by assembling p o t e n t i a l l i m i t a t i o n s i n t r a d i t i o n a l e v a l u a t i o n methodologies. The l i t e r a t u r e survey 131 of e v a l u a t i o n r e s e a r c h methods summarized i n Chapter I I i n d i -c a t e d s e v e r a l l i m i t a t i o n s o f e x i s t i n g e v a l u a t i v e procedures when a p p l i e d t o ongoing, m u l t i - d i m e n s i o n a l , s o c i a l a c t i o n p r o -grammes . T r a d i t i o n a l e v a l u a t i o n was shown to i n v o l v e a com-p a r i s o n o f programme go a l s w i t h programme impact, e f f e c t o r e f f o r t . The e x t e n t t o which goals are met or the magnitude of change become i n d i c a t o r s o f programme success l e v e l s . As p r e v i o u s l y i n d i c a t e d , such procedures tend to be summative as opposed t o fo r m a t i v e , r e q u i r e s t a t i c programmes, and f r e q u e n t l y u n a v a i l a b l e c o n t r o l groups, and assume t h a t g o a l s were v a l i d , from the u s e r ' s p e r s p e c t i v e , i n the f i r s t i n s t a n c e . While, by t h e i r very nature, programmes d e a l w i t h v a l u e - l a d e n concepts, m i n i -mal a p p r e c i a t i o n i s o b t a i n e d o f user behaviour, a t t i t u d e o r o p i n i o n s a s s o c i a t e d w i t h the programme. E v a l u a t i o n s f a i l t o c o n s i d e r the e v o l v i n g and changing nature of user needs, f a i l to i s o l a t e user r e l e v a n t programme dimensions from among the m u l t i - d i m e n s i o n a l programme concerns, and f r e q u e n t l y n e g l e c t t o i d e n t i f y whether problems o r i g i n a t e , o r change can be accomplished, through the s o c i a l a c t i o n programme. A p p r o p r i a t e -ness o f the treatment i s seldom questioned.''"^ 0 The f o l l o w i n g s e c t i o n s d e s c r i b e , e x p l a i n and j u s t i f y a methodology to i d e n t i f y user concerns and s a t i s f a c t i o n l e v e l s , t o handle an e v o l v i n g m u l t i - d i m e n s i o n a l programme, and to i d e n t i f y household-programme-system i n t e r v e n t i o n d i r e c t i o n s . A l l appear t o be necessary i n g r e d i e n t s f o r the e v a l u a t i o n of a programme from the user's p e r s p e c t i v e . 132 A Methodology to I d e n t i f y User Concerns and S a t i s f a c t i o n L e v e l s To e v a l u a t e a programme from the user's p e r s p e c t i v e i t appears most f r u i t f u l t o i s o l a t e the f a c t o r s around which the house-h o l d s t r u c t u r e s i t s a t t i t u d e s to the r e s i d e n t i a l environment. A framework upon which to base a p r o f i l e o f each user i n c l u d e d (as i n d i c a t e d i n Diagram 9) an a n a l y s i s , employing b a h a v i o u r a l and a t t i t u d i n a l data, of the economic, s e r v i c e , s o c i a l , and p s y c h o l o g i c a l c h a r a c t e r i s t i c s o f each f a m i l y . The s e l e c t i o n of these data s e t s combines f e a t u r e s o f e x i s t i n g household c l a s s i f i c a t i o n s w i t h s e v e r a l s o c i a l dimensions p r e -v i o u s l y not u t i l i z e d i n e v a l u a t i o n r e s e a r c h . T r a d i t i o n a l c l a s s i f i c a t i o n s o f households i n t o groups assumed to e x h i b i t s i m i l a r housing demand c h a r a c t e r i s t i c s u t i l i z e v a r i o u s i n d i c a t o r s i n c l u d i n g : Stage i n l i f e c y c l e : The t h e o r e t i c a l b a s i s of stage i n l i f e c y c l e as an i n d i c a t o r i s the b e l i e f t h a t persons o f a p a r t i c u l a r age group r e f l e c t a s i m i l a r m a r i t a l s t a t u s and f a m i l y s i z e . Thus they are seen to express s i m i l a r housing demand c h a r a c t e r i s t i c s . L i t e r a t u r e u s i n g t h i s concept i n c l u d e s Whyte (57), Foote (60) and D o b riner (58). The advantage of t h i s i n d i c a t o r i s the a c c e s s i b i l i t y o f census data through which to d i v i d e the p o p u l a t i o n i n t o sub-groups. Socio-economic s t a t u s : Chapin d e f i n e s socio-economic s t a t u s as "the p o s i t i o n an i n d i v i d u a l or a f a m i l y o c c u p i e s w i t h r e f e r e n c e to the p r e v a i l i n g average standards of c u l t u r a l pos-s e s s i o n s , e f f e c t i v e income, m a t e r i a l p o s s e s s i o n s and p a r t i c i p a -t i o n i n group a c t i v i t y of the community." DIAGRAM 9 COMPONENTS OF A HOUSEHOLD PROFILE 133 PROFILE INDICATOR DATA BASE ECONOMIC PSYCHOLOGICAL SOCIAL SERVICE Spending p a t t e r n s P r o j e c t e d income r e l i a b i l i t y Role of housing i n l i f e s t y l e Past housing experience Expec ta t ions Present p a t t e r n s l o c a t i o n , t y p e , i n t e n s i t y s o c i a l i z i n g Present p a t t e r n s l o c a t i o n , type shopping, p e r s o n a l s e r v i c e s B e h a v i o u r a l & A t t i t u d i n a l B e h a v i o u r a l & A t t i t u d i n a l B e h a v i o u r a l B e h a v i o u r a l INDICATORS OF PROGRAMME DIMENSIONS STUDIED DIMENSION INDICATOR OPERATIONAL DEFINITION 1.OWNERSHIP Economic. S t a b i l i z e S h e l t e r c o s t s No d i s s a v i n g present monthly expenses not exceed income Income R e d i s t r i b u t i o n Unit Sales P4 ycholo gZcal Increased s t a t u s , s e c u r i t y Sales enable f a m i l y to p a r t i c i p a t e i n p r i v a t e market d e s i r e d s t a t u s p e r c e i v e d s t a t u s 2. INCOME MIX Social Physical Minimal stigma Minimal i s o l a t i o n Increased f i t S e r v i c e s d e s i r e d contact a c t u a l contact d e s i r e d s e r v i c e s a c t u a l s e r v i c e s 3. PARTICIPATION Increased s e l f -d e t e r m i n a t i o n d e s i r e d p a r t i c i p a t i o n a c t u a l p a r t i c i p a t i o n 134 On the assumption t h a t s i m i l a r e d u c a t i o n and occupa-t i o n a l c h a r a c t e r i s t i c s i n d i c a t e shared v a l u e s r s e v e r a l s t u d i e s have used r e a d i l y a v a i l a b l e data to s u b d i v i d e the p o p u l a t i o n i n t o housing demand by s o c i a l c l a s s . However, a l t e r n a t i v e f a c t o r s must be c o n s i d e r e d as economic c o n s i d e r a t i o n s cease to be the s o l e f a c t o r d e t e r m i n i n g access to housing. Examples of the use of s o c i a l c l a s s are found i n the work of Beshers (62), Rainwater (66), and Gans (6 7). P r e s e n t home l o c a t i o n : Based upon the argument t h a t p a s t e x p e r i e n c e leads a household to c o n t i n u a l l y a l t e r i t s housing to b e t t e r s u i t p e r c e i v e d needs, e x i s t i n g h o using l o c a -t i o n i s assumed to r e f l e c t housing v a l u e s . However, f o r those without e f f e c t i v e demand, the i n f e r e n c e o f housing v a l u e s from p r e s e n t l o c a t i o n i s q u e s t i o n a b l e . Recent r e s e a r c h c a s t s doubts on the assumptions of u n i v e r s a l behaviour p a t t e r n s based upon one v a r i a b l e such as stage i n l i f e c y c l e . M i c h e l s o n (72) found t h a t s o c i a l rank and stage i n l i f e c y c l e f a i l e d t o vary s y s t e m a t i c a l l y w i t h c h o i c e s of " i d e a l " environment. His c o n c l u s i o n s suggest t h a t c h o i c e i s not a d i r e c t r e s u l t of age or s o c i a l c l a s s but of more s u b t l e i n f l u e n c e s o f v a l u e s and l i f e style.''""'"'" S i m i l a r f i n d i n g s emerge from two o t h e r Canadian s t u d i e s : t h e r e was no marked c o r r e l a t i o n between income and o c c u p a t i o n s t a t u s and s a t i s f a c t i o n w i t h p r e s e n t housing s i t u a t i o n . (Economic Research C o u n c i l 71, p. 16). The f a c t t h a t most o f the socio-economic v a r i a b l e s t e s t e d had no s y s t e m a t i c e f f e c t upon e v a l u a t i o n s i n d i c a t e s t h a t these e v a l u a t i o n s are 135 objective i n the sense that people with d i f -f e r i n g socio-economic c h a r a c t e r i s t i c s are equally l i k e l y to make the same response.... "who you are" does not determine what you l i k e and dislike....(Condominium Research Associates 70, p. 22). If t r a d i t i o n a l indicators of housing preference groups appear inadequate, an i n v e s t i g a t i o n of alternate i n d i -cators around which households structure attitudes and expecta-tions to the r e s i d e n t i a l environment seems j u s t i f i e d . Proposed Basis for Identifying User Concerns Resource a l l o c a t i o n , i n which the d i s t r i b u t i o n of 112 time or finances i s taken to r e f l e c t household patterns, has become increasingly popular as a l i f e s t y l e i n d i c a t o r . A c t i -v i t y studies have gained t h i s popularity from s o c i a l researchers looking for techniques which e l i c i t accurate descriptions of behaviour rather than opinion. In t i m e - a c t i v i t y budgets the 113 researcher asks his respondents for an account of what they did during f i n e l y divided periods of time, where, and with whom the a c t i v i t y took place. Michelson (70) suggests that i f the researcher assumes that a c t i v i t i e s comprise a linked sequence of choices and that motivations or needs r e s u l t i n a p a r t i c u l a r cho-ice of a c t i v i t i e s , he can ascertain the quantitative and q u a l i t a t i v e aspects of s o c i a l a c t i v i t y as well as i t s s p a t i a l d i s t r i b u t i o n and gain insights to construct i n t e r a c t i o n networks with other aspects of society's s o c i a l structure. Several operational examples of time-budget a c t i v i t y analysis e x i s t . Chapin, B r a i l and Hightower see 136 behaviour patterns of i n d i v i d u a l s , f a m i l i e s , i n s t i t u t i o n s and firms as occurring i n s p a t i a l patterns that have meaning i n plan-ning for land use. (Chapin 65, p. 224). In essence they are moving toward a description of household types and of land use patterns i n terms of the kind, amount and i n t e n s i t y of a c t i v i t i e s occurring at each l o c a t i o n . Michelson has u t i l i z e d a c t i v i t y patterns as i n d i c a -114 115 tors of l i f e s t y l e , environmental compatibility, and to v e r i f y current use of f a c i l i t i e s . " ' " ' ^ Buttimer"'"^ and 118 Young and Willmott assess s o c i a l patterns through a c t i v i t y studies. The use of a c t i v i t y systems as a basis for consider-ing the p o t e n t i a l impact of alternate p o l i c i e s on people's patterns i s noted i n the Human Ecology model of the computer-119 based p o l i c y simulator - HPS. A c t i v i t y patterns have been c o l l e c t e d through 120 inferences from e x i s t i n g census data and p a r t i c i p a n t obser-121 vation. However, by-and-large, a c t i v i t y analysis which attempts to catalogue household patterns over an extended period of time has required interview or questionnaire sur-122 vey. Gathering t i m e - a c t i v i t y data i s not without i t s prob-lems : Length of programme operation as an intervening v a r i -able i n the r e l a t i o n s h i p between housing and l i f e s t y l e patterns i s uncertain. Gans (61) and Gollege (70) have considered the problem of length of time i n the programme and changing a c t i v i t y patterns. A University of Wisconsin study of shopping patterns 137 over time indicated that households frequently maintain higher a c t i v i t y patterns over the f i r s t few months a f t e r r e l o c a t i o n . With time, a greater funnelling of a c t i v i t y patterns was e v i -denced and an associated decrease i n d i f f u s e t r i p behaviour. Single respondent; The extent to which i t i s possible for one member of the household to accurately report the t o t a l a c t i v i t y patterns of the household unit i s questioned. Most survey l i t e r a t u r e has used the wife as the respondent (Gutman (66), Gans (67), Michelson (72), Wolfe (59), Young and Willmott (57)) on the assumption that i n having the most con-tact with the l o c a l area she w i l l be more l i k e l y to perceive differences. One might also mention that the wife i s i n most cases more accessible than the husband. Various studies have questioned the representativeness of t h i s approach. Morgan (61), Lipman (6 8) and Kel l y (69) suggest determining household decision-making patterns as a guide to interviewee emphasis. Kelly (69) found that differences i n perception between husbands and wives did e x i s t with respect to both decision areas and role assignments. Differences included perception of location s u i t -a b i l i t y , ease of home maintenance, s u i t a b i l i t y of home and symbolic values i n the home. Generally spouses agreed on space a l l o c a t i o n s i n the home and the aesthetics and nature of the neighbourhood. Michelson (72) notes s i m i l a r findings. Lansing (70) i n comparing a c t i v i t y reporting found that for the house-hold as a whole the number of person t r i p s reported was about one t r i p higher when the husband reports than when the wife does. 1-38 K e l l y ' s (69) r e s e a r c h seems to conclude t h a t e i t h e r spouse c o u l d adequately r e p o r t f a m i l y p a t t e r n s as A f a m i l y - b y - f a m i l y breakdown of shared and independent l e i s u r e time a c t i v i t i e s r e v e a l s t h a t , i n most f a m i l i e s , most a c t i v i t i e s t h a t the f a m i l y pursues are shared. L i k e l y , the husband and w i f e w i l l have come to share p o i n t s of view on many home f e a t u r e s over a s i g n i f i c a n t p e r i o d of l i v i n g t o g e t h e r . ( K e l l y 69, pp. 11 and 26). The r e p r e s e n t a t i v e n e s s of the p a t t e r n s ; Time-budget r e s e a r c h s e l e c t s ' y e s t e r d a y ' a n d 1 l a s t Sunday 1 as r e p r e s e n t a t i v e of d a i l y time a l l o c a t e d t o i n d i v i d u a l a c t i v -i t i e s . P o s s i b l e b i a s e s r e s u l t from the s e a s o n a l nature of a c t i v i t i e s . A c t i v i t y S a t i s f a c t i o n ; While user concerns can be i n d i c a t e d by time or funds i n v e s t e d i t i s not v a l i d t o assume t h a t a c t i v i t y p a t t e r n s r e f l e c t s a t i s f a c t i o n l e v e l s , p a r t i c u l a r i l y i n the case of households p o s s e s s i n g m a r g i n a l e f f e c t i v e demand. A c t i v i t y p a t t e r n s i n d i c a t e o n l y the r e l e v a n t environment of the household. S a t i s f a c t i o n or d i s s a t i s f a c t i o n , f o r example i n o r d i n a t e amounts of time spent on u n d e s i r a b l e a c t i v i t i e s , r e q u i r e f u r t h e r a t t i t u d e and o p i n i o n data from householders. Determining S a t i s f a c t i o n L e v e l s To i n d i c a t e those f e a t u r e s which appear c e n t r a l to r e s i d e n t i a l s a t i s f a c t i o n r e q u i r e s an a n a l y s i s of f i t between household v a l u e s and a c t i v i t y p a t t e r n s . The e x t e n t to which household v a l u e systems are congruent w i t h e x p e r i e n c e d p a t t e r n s 139 at l e a s t p a r t i a l l y determines s a t i s f a c t i o n . In e f f e c t , there i s an i m p l i e d weighting system. To date there have been few attempts to determine f a m i l y values broader than housing or to c o r r e l a t e or weigh housing needs i n r e l a t i o n t o the t o t a l f a m i l y l i f e s t y l e . Such an a n a l y s i s appears r e l e v a n t as households vary c o n s i d e r a b l y i n the p o s i t i o n housing p l a y s i n o v e r a l l p r i o r i t i e s . To some f a m i l i e s a house i s not an important component i n t h e i r mode of l i f e . S h e l t e r i s obtained by such f a m i l i e s i n a ca s u a l and o f f -hand manner by f o l l o w i n g some l i n e of l e a s t r e s i s t a n c e purchase s t r a t e g y . ( K e l l y 69, p. 16). In determining housing p o l i c y i n r e l a t i o n to o v e r a l l s o c i a l s e r v i c e s i t appears t h a t an attempt t o solve housing problems might be the i n c o r r e c t p o l i c y i n t e r v e n t i o n i f the household i n f a c t f e e l s education, employment, or other s o c i a l opportu-n i t i e s are of gr e a t e r consequence. Thus d i s c u s s i o n s of the household economic, s o c i a l and s e r v i c e p r o f i l e , obtained through a c t i v i t y a n a l y s i s , were tempered by an i n d i c a t i o n of the weighting the household places on housing i n the context of o v e r a l l needs and expec-t a t i o n s . S a t i s f a c t i o n i n d i c e s have been compiled i n a v a r i e t y 123 of ways. In t h i s study s a t i s f a c t i o n l e v e l s are expressed as a simple r a t i o of f e l t p a t t e r n s to d e s i r e d pat-124 t e r n s . The proposed s a t i s f a c t i o n index r e f l e c t s r e l a t i v e r a t h e r than absolute f i g u r e s and f u t u r e , as opposed to past, p o s i t i o n . 140 T h i s i s a departure from e x i s t i n g e v a l u a t i o n s which are i n t r i n s i c a l l y h i s t o r i c a l i n nature - comparing p a s t p a t -t e r n s w i t h the c u r r e n t housing s i t u a t i o n . I t w i l l be argued t h a t t r a d i t i o n a l a n a l y s i s s u p p l i e s summative data but c o n t r i -butes l i t t l e toward the r e f o r m u l a t i o n of programmes i n the l i g h t o f changing c i r c u m s t a n c e s . Rather, what appears neces-sary, i s an assessment o f p r e s e n t p a t t e r n s i n r e l a t i o n to p e r c e i v e d need. I t appears i m p o s s i b l e to i n d i c a t e a b s o l u t e v a l u e s as our p e r c e p t i o n of need i s c o n s t a n t l y changing. In Canada Our problem i s not p h y s i c a l s u f f i c i e n c y but r e l a t i v e s u f f i c i e n c y . . . m o s t evidence i n d i c a t e s t h a t our sense o f s a t i s f a c t i o n and d e p r i v a t i o n comes more from our r e l a t i v e p o s i t i o n i n the d i s t r i b u t i o n of l i v i n g standards than i t does from our a b s o l u t e p o s i t i o n . (Dennis 72, p. 38). In e f f e c t , one i s comparing the p r e s e n t circumstances w i t h d e s i r e d or f u t u r e e x p e c t a t i o n s . A ( p a s t r p r e s e n t ) a c t i v i t y emphasis i s r e p l a c e d by a ( p r e s e n t : f u t u r e ) o r i e n t a t i o n . Ongoing e v a l u a t i o n i s necessary s i n c e one can never e l i m i n a t e housing as a problem - housing needs are a r e l a t i v e c o n d i t i o n , the f u l f i l l m e n t o f one s e t of needs simply, as Maslow suggests, l e a d s to the p e r c e p t i o n of f u r t h e r needs demanding w i t h equal urgency, to be met. T h i s i s c o n c e p t u a l l y one o f the most d i f f i c u l t problems t o f a c e i n e v a l u a t i o n . Housing i s not a s t a t i c item but r a t h e r an e v o l v i n g or con-t i n u a l l y changing s i t u a t i o n as a f a m i l y ' s needs and e x p e c t a t i o n s a l t e r and as changing circumstances permit or p r o h i b i t a r e -assessment and r e a c t i o n to the p r e s e n t s i t u a t i o n . A broad base, 141 mixed scan of the household p r o v i d e s f o r the contingency t h a t subsequent e v a l u a t i o n s may wish t o monitor p r e s e n t l y u n i d e n t i -f i e d v a r i a b l e s . T h i s dynamic view of the r e s i d e n t i a l e n v i r o n -ment was r e c e n t l y i d e n t i f i e d by Buttimer (72) i n her attempt t o d e f i n e r e s i d e n t i a l environment success as b e i n g c o n t i n g e n t upon the e x i s t e n t i a l meaning i t a c q u i r e s f o r i t s r e s i d e n t s i n the process o f "becoming not b e i n g " . Thus the proposed methodology to i d e n t i f y user con-cerns and s a t i s f a c t i o n l e v e l s r e f l e c t s a t r e n d toward the i n c l u s i o n o f more i n t a n g i b l e v a r i a b l e s i n s o c i a l a c t i o n p r o-gramme e v a l u a t i o n by combining b e h a v i o u r a l l y - b a s e d a c t i v i t y a n a l y s i s w i t h weightings based upon user v a l u a t i o n o f p a t t e r n s . User concerns are r e f l e c t e d by time and r e s o u r c e a l l o c a t i o n s . User s a t i s f a c t i o n l e v e l s are i n d i c a t e d by a r a t i o o f p r e s e n t p a t t e r n s t o d e s i r e d p a t t e r n s . Proposed Methodology t o Handle a M u l t i - D i m e n s i o n a l Programme Programme e v a l u a t i o n t o date has tended t o r e f e r t o the t e s t o f a t o t a l product w i t h the p u r e l y p r a c t i c a l o b j e c t i v e of d e t e r m i n i n g whether exposure t o a programme was accompanied by c e r t a i n d e s i r e d e f f e c t s . R e a l i z i n g t h a t housing i n v o l v e s a complex, m u l t i - d i m e n s i o n a l system, p o s s e s s i n g economic and s o c i a l components, and t h a t any programme has an en d l e s s number of p o t e n t i a l items f o r a n a l y s i s , i t appears more r e a l i s t i c t o s e l e c t s e v e r a l components, or programme dimensions, f o r i n t e n -s i v e study. I t i s assumed t h a t the c l o s e r these v a r i a b l e s come to e x p l a i n i n g the u n d e r l y i n g p r o c e s s e s of housing dynamics, 142 as experienced by the user, the closer the evaluation w i l l come to assessing dimensions c r u c i a l to programme reformulation. Variables assessed i n d e t a i l are those i d e n t i f i e d through user behaviour patterns and attitude surveys. The formulation of a desired set of attributes by a user i s analogous to what psychologists have termed categorization. Categoriza-t i o n i s undertaken to reduce the complexity of the environment, to reduce the need for constant learning and to provide d i r e c t i o n for a c t i v i t i e s . In his book on evaluative research, Suchman sub-stantiates t h i s l i n e of reasoning: There i s greater s i g n i f i c a n c e i n evaluative research which aims at t e s t i n g of variables or p r i n c i p l e s rather than s p e c i f i c products or t o t a l programs....An evaluative study which tests the effectiveness of an approach or p r i n c i p l e of action, i n the course of evalua-t i n g the s p e c i f i c a c t i v i t y or service, may have both wider t h e o r e t i c a l and p r a c t i c a l relevance. (Suchman 67, p. 59). If programme dimensions are selected such that they r e f l e c t not only programme concerns but also broader housing p o l i c y dimensions and involve housing theory, an analysis of such dimensions w i l l contribute not only to recommendations for programme change but also to p o l i c y reformulation and to the t e s t i n g , with one c l i e n t group, of several e x i s t i n g assump-tions of housing demand within the framework provided by pres-ent theory. I n i t i a l analysis of user concerns indicates that i n the "300 Unit Programme" the s o c i a l and economic consequences of ownership for lower income f a m i l i e s , the impact of mixing lower income households i n otherwise middle income neighbourhoods, and the l e v e l of household 143 involvement i n determining housing p a t t e r n s are th r e e user o r i e n t e d programme dimensions. A l l t h r e e dimensions r e f l e c t emerging d i r e c t i o n s i n Canadian s o c i a l housing p o l i c y and can be analyzed i n the co n t e x t o f GVRD's programme. I n d i c a t o r s have been s e l e c t e d f o r each dimension t o assess household p a t t e r n s . (Refer t o Diagram 9, page 133). I n d i c a t o r s o f the Economic and S o c i a l Consequences o f Ownership Economic and s o c i a l consequences o f ownership, as d i s -cussed i n P a r t I o f Chapter I I I , were seen to i n c l u d e the s t a -b i l i z a t i o n o f s h e l t e r c o s t s , income r e d i s t r i b u t i o n , i n c r e a s e d s t a t u s , commitment t o community and c o n t r o l over the house-hold'