Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An Empirical study of judgment making in groups using qualitative controlled feedback Ali, Mirza Wazed 1978

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1978_A6_7 A45.pdf [ 7.92MB ]
Metadata
JSON: 831-1.0080252.json
JSON-LD: 831-1.0080252-ld.json
RDF/XML (Pretty): 831-1.0080252-rdf.xml
RDF/JSON: 831-1.0080252-rdf.json
Turtle: 831-1.0080252-turtle.txt
N-Triples: 831-1.0080252-rdf-ntriples.txt
Original Record: 831-1.0080252-source.json
Full Text
831-1.0080252-fulltext.txt
Citation
831-1.0080252.ris

Full Text

AN EMPIRICAL STUDY OF JUDGMENT MAKING IN GROUPS USING QUALITATIVE CONTROLLED FEEDBACK by Mirza Wazed^Ali M.A., University of Rajshahi, Bangladesh, 1968 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE i n THE FACULTY OF GRADUATE STUDIES i n the Department of Mathematics and The Ins t i t u t e of Applied Mathematics and S t a t i s t i c s We accept t h i s thesis as conforming to the required standard • THE UNIVERSITY OF BRITISH. COLUMBIA June, 1978 Mirza Wazed A l i , 1978 In presenting this thesis in partial fulfi lment of the requirements an advanced degree at the University of Br i t ish Columbia, I agree t I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the Head of my Department or by his representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. the Library shall make it freely available for reference and study. Department of The University of Br i t ish Columbia 2075 Wesbrook P l a c e V ancouver, Canada V6T 1W5 ABSTRACT This work gives an account of an empirical study on the assess-ment of judgments of individuals i n a group. The phenomenon of judg-ment or decision making i n groups appears i n various contexts. However we are interested i n situations where each member of a group i s requir ed to give independently of other members of the group, his most informed and reasoned judgment on a controversial issue. Nonetheless, i t i s of interest to gain knowledge about the importance of various judgments about the issue, and also of the arguments (or reasons) put forward by the judges to support t h e i r judgments. Such situations of judgmentmaking raise methodological problems for c o l l e c t i n g judg-mental data, and methods, such as, face-to-face discussion or the Delphi method may not be appropriate. To circumvent t h i s problem, a new method cal l e d 'Qualitative Controlled Feedback' (Q.C.F.) was developed by Press [13]. Our aim i n the present work i s to examine the workings of the method by i t s application to a r e a l world s i t u a t i o n . With this aim, judgments (and other data of interest) were collected, using a three-stage Q.C.F. survey, from a random sample group of Faculty and Staff members of the.University of B r i t i s h Columbia on a question related to the issue of whether or not the University should b u i l d an Indoor Aquatic Center on the campus. The data was analysed from an explora-tory viewpoint. i i i I t was observed that qua l i t a t i v e controlled feedback creates a good interaction ( i n the sense of exchanging arguments and'reasons) among the group members. Change i n judgment occured as subjects went from one stage to another after having qu a l i t a t i v e feedback of information. By comparing with a control group of subjects, i t was also found that q u a l i t a t i v e feedback was able to produce more r a t i o n a l judgments than without any feedback. The distributions of judgment obtained i n this empirical study bear s i g n i f i c a n t implications for decision making. The distributions were found to be bimodal and represented two opposing groups of thought. Other results involve, regression analysis, t r a n s i t i o n p r o b a b i l i t i e s of judgment change from one stage to another, analysis of judgment change behavior, importance of reasons, effect of non-response on judgment d i s t r i b u -tions and analysis of confidence i n judgment. F i n a l l y , i t was found that the method of Qualitative Controlled Feedback can be f r u i t f u l l y applied to situations of p r a c t i c a l i n t e r e s t . TABLE OF CONTENTS CHAPTER 1: INTRODUCTION 1. General Outline 2. Background 2.1 Some Basic Elements 2.2 Group Judgment 2.3 The Rational behind Controlled Feedback 3. Scope about Judgment the Method of Qualitative CHAPTER 2: THE METHOD OF QUALITATIVE CONTROLLED FEEDBACK 1. Controlled Feedback 2. Qualitative Controlled Feedback 3. Discussion 4. Objectives, of the Study 5. S t a t i s t i c s ! Background 5.1 A Regression Model for the F i r s t Stage Response 5.2 A Regression Model for the Response after Information Feedback 5.3 The Lo g i s t i c Regression 5.4 Importance of Reasons 5.5 Measure of Round-toeRound Variation i n Response V CHAPTER 3: AN EMPIRICAL APPLICATION 17 1. The Application Situation 17 2. Background of the UBC Indoor Aquatic Center 18 3. Overall Plan of the Survey 20 3.1 Population of Interest 20 3.2 Sampling Scheme 21 3.3 Number of Stages and Sample Size 22 3.4 Strategy for Questionnaire Design 23 4. F i r s t Stage . 24 4.1 Questionnaire Preparation 24 4.2 Data Collection 27 5. Second Stage 29 5.1 Questionnaire Preparation 29 5.2 Data Collection 3 5 6. Third Stage 3 5 6.1 Questionnaire Preparation 3 6 6.2 Data Collection 3 7 7. The Control Group 3 8 CHAPTER 4: DATA ANALYSIS 4 0 1. Comparison of Response Rates 4 0 2. The Empirical Judgment Distributions ^2 2.1 Graphical Representation of the Judgment Distributions 42 2.2 Means and Standard Deviations 5° 2.3 Tests of Three Hypotheses 52 Analysis of Judgment Change 3.1 A Summary 3.2 Test of a Hypothesis 3.3 Transition P r o b a b i l i t i e s of Response Regression Analysis 4.1 Regression of Z-^  on the Cue Variables 4.2 Regression of o n 4.3 Regression of on Analysis of Dropout Effect 5.1 Comparison of F i r s t Stage Distributions: Dropout vs. Nondropout 5.2 Logis t i c Regression Analysis Study of Reasons-giving Behavior 6.1 Dis t r i b u t i o n of the Number of Reasons ;6i?2nsT.r^nsifiohaProl3aBi3;it'i"es3ofoReasons 6.3 Test of a Hypothesis 6.4 Study of Reasons-giving Behavior with respect to Category of Response Testing for the Effect of Ordering of Reasons i n the Composite L i s t 7.1 Comparison between Distributions 7.2 Comparison i n terms of Reason-giving Distributions of Confidence Rating v i i CHAPTER 5: CONCLUSION 101 1. Summary of Findings 101 1.1 V e r i f i c a t i o n of Methodological Issues 101 1.2 Substantive Findings 104 2. Further Research Directions 109 3. Some Recommendations 113 4. Concluding Remarks 115 BIBLIOGRAPHY 116 APPENDICES APPENDIX A l : The F i r s t Stage Questionnaire- 118 APPENDIX A2: The Second Stage Questionnaire 125 APPENDIX A3: The Third Stage Questionnaire with Randomised Composite L i s t of Reasons 133 APPENDIX A4: The Third Stage Questionnaire with Nonrandomised Composite L i s t of Reasons 146 APPENDIX A5: The Questionnaire for the Control Group 147 APPENDIX B: Empirical Frequency Distributions of Response 148 APPENDIX CI: Frequency Table showing Change i n Response from the F i r s t Stage to the Second Stage (combined group) 149 APPENDIX C2: Frequency Table showing Change i n Response from the Second Stage to the Third Stage (combined group) 150 V l l l APPENDIX C3: APPENDIX C4: APPENDIX C5: APPENDIX C6: APPENDIX DI: APPENDIX D2: APPENDIX D3 APPENDIX E: APPENDIX F: APPENDIX G: APPENDIX H: Frequency Table showing Change i n Response from the F i r s t Stage to the Second Stage (Faculty) Frequency Table showing Change i n Response from the Second Stage to the Third Stage (Faculty) Frequency Table showing Change i n Response from the F i r s t Stage to the Second Stage (Staff) Frequency Table showing Change In Response from the Second Stage to the Third Stage Revised Expected Frequencies for the F i r s t Stage and the Control Group Distributions Revised Expected Frequencies for the Third Stage and the Control Group Distributions Revised Expected Frequencies for the F i r s t Stage and the Third Stage Distributions A Technique for Using Data Sets with. Missing Observations i n Regression S e r i a l Numbers of Reasons i n Tables- XVI, XIX, XX, XXI and Appendix G and the Corresponding S e r i a l Numbers in.Randomised and Nonrandomised L i s t s Proportion of Participants i n the Randomised and Nonrandomised L i s t Groups who gave Reason Q\ The L i s t of Reasons Arranged according to Importance Ranks 151 152 153 154 155 155 156 157 159 160 162 i x LIST OF TABLES Table I Response Rates 41 Table I I Empirical Judgment D i s t r i b u t i o n s ( i n percent) 43 Table I I I Means of Judgment Distributions 52 Table IV Standard Deviations of Judgment Distributions. 52 Table V Summary of Change i n Response with respect to Three Broad Categories 59 Table VI C l a s s i f i c a t i o n of Subjects with, respect to Judgment Change i n the Second and Third Stages 60, Table VII MLE of Transition P r o b a b i l i t i e s of Response from the F i r s t Stage to the Second Stage 62 Table VIII MLE of Transition P r o b a b i l i t i e s of Response from the Second Stage to the Third Stage 63 Table IX Results of Regression of on the Cue Variables. 66 Table X F i r s t Stage Distributions of Dropouts: and Nondropouts, -70. Table XI Results of Logistic Regression for Non-response with 10 Cue Varibles 74 Table XII Results of L o g i s t i c Regression for Non-response with 9 Cue Variables 75 Table XIII Distributions ('in % of subjects) of the Number of 1 Reasons for the F i r s t , Second and Third Stages .77 Table XIV Means and Standard Deviations of the Distributions of the Number of Reasons 77 L i s t of Tables (continued) Table XV Table XVI Table XVII Table XVIII Table XIX Table XX Table XXI Table XXII Table XXIII Table XXIV Transition P r o b a b i l i t i e s of Reasons corresponding to the F i r s t Stage and the Second Stage Transition P r o b a b i l i t i e s of Reasons corresponding to the Second Stage and the Third Stage Results of Testing H OH) Response vs. Reasons C l a s s i f i c a t i o n ( F i r s t Stage) Response vs. Reasons C l a s s i f i c a t i o n (Second Stage) Response vs. Reasons C l a s s i f i c a t i o n (Third Stage) Importance Ranks of Reasons Distributions of Subgroups with. Randomised and Nonrandomised L i s t Computed Values of u(A) Distributions of Confidence Ratings 81 82 86 89 90 91 93 •9.4. 9.7. 10.0. x i LIST OF FIGURES Fig . 1 (a) F i r s t Stage Distributionsof Judgment of the Combined Group 4-5 (b) Second Stage Dist r i b u t i o n of Judgment of the Combined Group 45 (c) Third Stage Dist r i b u t i o n of Judgment of the Combined Group 45 Fig. 2 (a) F i r s t Stage Di s t r i b u t i o n of Judgment (Faculty) 46 (b) Second Stage Dist r i b u t i o n of Judgment(Faculty) 46 (c) Third Stage Distribution of Judgment (Faculty) 46 Fig. 3 (a) F i r s t Stage Dist r i b u t i o n of Judgment (.Staff) 47 (b) Seconf Stage Dist r i b u t i o n of Judgment (Staff) 47 (c) Third Stage Dist r i b u t i o n of Judgment CStaff). 47 Fig. 4 Di s t r i b u t i o n of Judgment of the Control Group 48. Fig. 5 (a) F i r s t , Second and Third Stage Distributions: of Judgment of the Combined Group (.superimposed) 49 (b) F i r s t , Second and Third Stage Distributions, of Judgment of Faculty (superimposed) 49 (c) F i r s t , Second and Third Stage Distributions, of Judgment of Staff (superimposed) 50 Fig. 6 (a) F i r s t stage and Control Group Distributions (superimposed) 51 ."t- (b) Third Stage and Control Group Distributions (superimposed) 51 x i i L i s t of Figures (continued) Fi g . 7 (a) F i r s t Stage Dist r i b u t i o n of Response of Dropouts 71 (b) F i r s t Stage Distributions of Response of the Dropouts and the Nondropouts (superimposed) 71 Fig. 8 (a) Distr i b u t i o n of the Number of Reasons for the F i r s t Stage 78 (b) Di s t r i b u t i o n of the Number of Reasons for the Second Stage 78 (c) Dis t r i b u t i o n of the Number of Reasons. for the Third Stage 79 Fig. 9 Cumulative Frequency Diagrams for the Third Stage and Control Group Distributions of Confidence Ratings. 99 Fig . 10 Some Bimodal Judgment Distributions with. Opposing Subgroups 10-8 ACKNOWLEDGMENT I am grateful to Professor S. James Press who suggested the topic of this study and under whose direction this research was carried out. I would also l i k e to thank Professors Henry Hightower, Elizabeth Yang, Charlan Nemeth and S.W. Nash for t h e i r many helpful comments and suggestions and Professor Fred Wan for his continuing encouragement. My thanks are also due to Aftab Khan for his help i n various stages of data, c o l l e c t i o n , Abdullah-Al-Mamun' Khan, Kam-Wah Tsui and W. Samaradassa for the i r enthusiasm i n coming into discussion at various points of the study, Theresa Fong for her help i n typing-a beginning draft of the thesis. F i n a l l y , I express my thanks to a l l the participants of the. study without whose continuing active p a r t i c i p a t i o n this study could not be carried out. M. W. A. 1 CHAPTER 1 INTRODUCTION 1 General Outline This work gives an account of an empirical study on the assessment of judgments of individuals in a group. The phenomenon of judgmentmaking i n groups appears i n various contexts. However, we are interested in s i t u a -tions where each member of a group i s required to give , independently of other members, his most informed and reasoned judgment on a controversial issue. The issue i s controversial i n the sense that there i s no correct answer to the question of judgmentmaking. Nonetheless, i t i s of interest to gain knowledge about the importance of the various judgments and also of the arguments (or reasons) put forward by the judges to support t h e i r judgments. Such situations of judgmentmaking raise methodological problems for c o l l e c t i n g judgments. We w i l l discuss the problems in the following sec-t i o n . Our immediate question i s : How to overcome the problems and c o l l e c t judgmental data i n situations as described above? An answer to this quest-ion has been given by Press [13] who has developed a method for c o l l e c t i n g and analysing such data. This i s the method of 'Qualitative Controlled Feedback'. The method has been described i n Chapter 2. Our aim i n the present work i s to examine the workings of the method by i t s application to a r e a l world s i t u a t i o n . With t h i s aim, an empirical study had been undertaken whereby judgmental data was collected and analy-sed. The issue of judgmentmaking we selected for the purpose had originated in the University of B r i t i s h Columbia. The issue was whether or not the u university should b u i l d an indoor aquatic center on the campus. Judgments, on a basic question of interest related to the issue , were collected from a 2 random sample of faculty and s t a f f members of the university by using a three-stage qua l i t a t i v e controlled feedback survey. This work i s a d e t a i l -ed account of the study and i t s findings. 2 Background In t h i s section we w i l l review some material related to the psycho-logy of judgmentmaking i n groups and w i l l outline the rationale behind developing the method of quali t a t i v e controlled feedback. 2.1 Some Basic Elements about Judgment The phenomenon of judgmentmaking may be described i n terms of two sets of variables - stimulus variables and dependent variables. Stimulus variables are characteristics of stimulus objects or aspects of the stimu-lus s ituation c a l l i n g for a judgment. For example, color, loudness, d i s -tance, attractiveness, etc. characteristics of objects may serve as stimu-lus variables i n some situations of judgmentmaking. Four important depen-dent variables of judgment can be enumerated. They are::(l) the response by which the judgment i s expressed or communicated, usually simply called judgment; (2) confidence or uncertainty of judgment; (3) the time taken by the judgment process; and (4) the d i f f i c u l t y of judgment. Judgments may be c l a s s i f i e d into two broad categories - simple and complex - with respect to the nature of the stimulus variables. In case of simple judgments just one aspect or dimension of the stimulus object i s to be judged and often the stimulus variable i s c l e a r l y s p e c i f i e d , and varies along a single dimension or continuuam. For example., judging loud-ness of voices i s a simple judgment. On the other hand complex judgments (1) For details see, for instance, [7], [8]. 3 involve more than one dimension of the stimulus object and the stimulus variables may not be c l e a r l y specified. " For example, i n judging the importance of having either a hospital or a public recreation center i n a c i t y , the judge might consider such aspects as necessity, usefulness, and sim i l a r other characteristics of the two projects. These character-i s t i c s are not c l e a r l y defined and t h e i r number may vary from one judge to another. In our present study of judgment, we are concerned with situations of complex judgmentmaking. We w i l l be interested mainly i n the f i r s t kind of dependent variable, that i s , response of a subject on a question of making judgment; Wethay§halso collected data on the second variable, that i s , confidence of judgment, at a l a t e r part of the study. 2.2 Group Judgment It i s important to r e a l i z e the d i s t i n c t i o n between the terms 'judg-ment making i n groups' and 'group judgment'. We use the former term to mean the task of making a judgment by an indiv i d u a l as a member of group, the other members of which are also engaged i n sim i l a r tasks. Such groups are usually known as judgmentmaking groups or decisionmaking groups. On the other hand, by group judgment i s meant a single judgment derived from the i ndividual judgments of the group members. For example, the group may come to a consensus or the judgment of the majority may serve as the group judgment. Thus i t must be emphasised that there i s no fixed rule or c r i -Thus terio n to derive the group judgment out of the indiv i d u a l judgments; the c r i t e r i o n may d i f f e r from one context to another. In t h i s study we are b a s i c a l l y interested i n seeing how better i n f o r -med and reasoned judgments of the ind i v i d u a l members of a group can be ob -4-tained by using the method of q u a l i t a t i v e controlled feedback. Once the judgments are obtained, i t may be possible to derive group judgment from them by using some c r i t e r i o n . 2.3 The Rational behind the method of Qualitative Controlled Feedback F i r s t we w i l l review (1) the elements of the psychology of group processes, especially of face-to-face groups for judgment or decisionmak-ing, and (.2) the Delphi method for forming group judgment. Social psychologists have struggled for a long time to get an answer to the question: Which i s better, the individual or the group? However, i t was realized that the question was not quite meaningful; there are s i t u -ations where individuals do better than groups and vice versa. The next question was: Under what conditions groups perform better? I t has been found that the performance of a group depends, on the one hand, on the nature of the task, and on the other hand, on the interpersonal environ-ment of the group [3]. In p a r t i c u l a r , i t has been found that i f the task involves i n t e l l e c t u a l a c t i v i t i e s i n the sense that more ideas, views and alternatives need.- to be generated, the group performs better under cer-t a i n conditions on the interpersonal environment. Thus,'in situations of judgment or decisionmaking, where i t i s desired that a greater number of alternatives should be examined i n the l i g h t of contrasting arguments and reasons, a group of judges should perform better subject to the condition: The interpersonal environment .is such that i t fosters independent thinking, free expression of ideas and views and the sense of respect i n one member about other members' views. In other words, for a better performance of judgmentmaking groups, i t i s necessary that there should be a free i n t e r -action among the group members. However, as C o l l i n s and Guetzkow [3] have shown, various obstacles i n the interpersonal environment stand i n the way of free interaction among group members. The main sources of the obstacles are status hierarchy, personality styles , leadership styles and power of dominance. The impact of the obstacles may be that alternative views do not get representated, and also that undue weights are given, perhaps, to irrelevant factors. Effects of some other factors, such as, group cohesiveness, group norm and leadership on the performance of face-to-face decisionmaking groups has been examined by Janis [6], He has found, after analysing the d e l i -berations of actual world decisionmaking groups , that these factors may give r i s e to a phenomenon, he c a l l e d , 'groupthink'. Groupthink i s said to occur i n a group i f , i n spite of t h e i r high i n t e l l e c t u a l a b i l i t y , the group members r e f r a i n from c r i t i c a l thinking and manifest concurrence (or consensus) seeking behavior with an i l l u s o r y optimism about the success of t h e i r decision. Group pressure i s s t i l l another factor which i n h i b i t s free expression of views [1]. The objective of our above discussion was to show that, although, face-to-face discussion i s the most common method for judgmentmaking i n groups, i t has many drawbacks; situations may arise when some other method may prove useful. Delphi A different approach for s o l i c i t i n g judgment from the mem-bers of a group was i n i t i a t e d by Dalkey and Helmer [4] with a view to u t i l i z e expert judgment for forecasting future technological events. The (2) method i s named Delphi. 'The idea behind the method i s that of 1 c o n t r o l l (2) feedback 1. Each member of a panel of experts are asked to give his judg ment i n d i v i d u a l l y and independently on various aspects of future technolo (2) For a detailed d e f i n i t i o n see Chapter 2, page 8. 6' g i c a l events. The judgments are given numerically and are collected by an intermediary. The intermediary then computes some summary measures, such as, mean, median, i n t e r q u a r t i l e range etc. He, then, feeds back ( i . e . giv-es information about) one or more of the measures, or perhaps, the entire d i s t r i b u t i o n of judgments to each member individually^. 3) . Each member i s then asked again to give his judgment independently. Anonymity of the judges i s preserved and t h i s process of feedback and s o l i c i t i n g judgment i s continued u n t i l convergence or consensus i s reached. With t h i s basic p r i n c i p l e , Delphi has been used i n various ways and (4) for a large number of applications. . Although Delphi was used f r u i t f u l l y i n a number of situations for purposes of forecasting future events, i t has been c r i t i s i s e d as f a i l i n g to meet standards of a methodology for s o c i a l research.[l's] . A p r i n c i p a l objection i s r e l a t i n g the idea of quantitative feedback. When quantitative measures, such as, mean i s fed back, the pane-l i s t s are psychologically persued to move t h e i r answer towards the given mean. This i s because, i f the panelist finds that his previous round ans-wer i s far away from the mean, he often feels that his answer i s an o u t l i e r . Thus the panelists are a r t i f i c i a l l y pressurized to move towards a consessus. It has also been argued that i f , instead of forecasting future events, Del-phi i s applied to find a numerical answer to factual question, the answer produced by the consus of the judges may simply be wrong, that i s , i t may not be anywhere near the true answer. In summary ,wwhi'le by using the controlled feedback approach Delphi attempts to avoid the various obstacles and t h e i r consequences (as discuss-(3) This type of feedback i s known as quantitative feedback. (4-) A l i s t of these applications and studies i s given by Linston and Turoff 7 ed before i n t h i s section), by using a quantitative approach i t pro-duces obstacles to creating such interaction (e.g. exchange of a l t e r -native ideas) as desired by a judgmentmaking group (see f i r s t part of thi s section). The problems of face-to-face discussion for forming judgment and . , also of the Delphi procedure lead to developing a new method that could overcome them (the problems). Such a method should, at least on p r i n -c i p l e , be able to provide an opportunity to the members of a group to form t h e i r judgments independently, and s t i l l come into interaction with other members of the group (e.g. 'by review of alternative arguments and reasons put forward by other members). I t i s for t h i s objective that the method of quali t a t i v e controlled feedback has been developed. 3 Scope In Chapter 2 the method of qualitative controlled feedback has been described and some s t a t i s t i c a l background useful for data analysis i s presented. The various operational procedures, e.g. questionnaire pre-paration, data c o l l e c t i o n etc. , related to planning and conducting a three-stage qua l i t a t i v e feedback survey are described in Chapter 3. Chapter 4- i s devoted to data analysis. F i n a l l y , i n Chapter 5, we summarise the main findings of the study, make recommendations for future applications, and suggest some further research directions. 8 CHAPTER 2 THE METHOD OF QUALITATIVE CONTROLLED FEEDBACK The method can be described i n two steps - f i r s t by defining 'contro-l l e d feedback' and then by defining 'qualitative controlled feedback'. We sjtjate thesestwo steps as they appear i n Press [13]. 1 Controlled Feedback A data c o l l e c t i o n protocol using a controlled feedback involves f o r -ming a random sample group of respondents and using the following pro-cedure : (1) Asking each group member to respond to questions privately and independently of a l l other respondents ; he i s s p e c i f i c a l l y asked for an answer to each question, sometimes he i s asked to provide j u s t i f y i n g rea-sons for his answer; (2) Collecting the answers (and possibly also, the reasons for the answers) from a l l group members (pa n e l i s t s ) , recording them, and then presenting (feeding back) some summary information about the group res-ponses to each group member; each group member i s then asked to respond to the same battery of questions again , without conferring with any other group members; (3) Repeating the questioning and feedback process again, and again, u n t i l i t s t a b i l i z e s , i n the sense that a l l panelists' responses are approximately unchanging from round to round. Thus, the process might terminate at group consensus, or at a point where there are seve-r a l "judgment n u c l e i " , representing several subgroups of hardened but d i f f e r i n g positions. In paragraph (2) above, i t has been mentioned that the' group members 9 are fed some, summary information obtained from the previous round of ques-tionning (. and possibly c o l l e c t i n g reasons). Here, two kinds of feedback can usually be considered - quantitative and q u a l i t a t i v e . Quantitative feedback implies that some quantitative summary measures of information, such as, mean, median, mode, int e r q u a r t i l e range, or possibly the whole d i s t r i b u t i o n of responses are fed back. As an example, Delphi (Chapter 1) uses quantitative feedback. 2 Qualitative Controlled Feedback A qua l i t a t i v e controlled feedback data c o l l e c t i o n protocol refers to a data c o l l e c t i o n procedure i n which: (1) Panelists are asked to give a response to a basic question of inte r e s t ; i n addition, panelists are asked to record j u s t i f y i n g reasons (statements r e l a t i n g to t h e i r own values, i n addition to what they believe to be facts) for the part i c u l a r answers they are giving, to the question. The panelists are also asked for information about other explanatory (cue) variables about themselves. (2) An intermediary then merges a l l of the reasons (eliminating replicated reasons that use d i f f e r i n g wording), and forms a composite l i s t of reasons; (3) The composite of reasons i s then presented (fed back) to each panelist;}(but no quantitative data, such as the group median or mean on the previous round, i s fed back), and the group members are each asked to answer the same question again, and are asked again to provide j u s t i f y i n g reasons for t h i s round's response. They may then add to the composite, or delete some e a r l i e r given reasons, to form t h e i r i n d i v i d u a l current slate of reasons; (4-) The feedback process i s now repeated again u n t i l the process s t a b i l i z e s . (That i s , after a few rounds, panelists should no longer be ab-le to create; new reasons, and they should have hardened t h e i r positions on the basic question, since l i t t l e new information w i l l have appeared i n the fed back composite of reasons). 3 Discussion Although the basic principles have been stated above, i t i s worth-while to discuss how one goes obout obtaining judgments using the method. Given an underlying population and an issue of judgmentmaking, a basic question relevant to the issue i s prepared. An answer to t h i s basic ques-t i o n i s a judgment. To get a quantitative answer, the question may be framed i n such a way that the answer can be chosen from a given scale. An intermediary 'takes a random sample of individuals from the population to form a panel of respondents. The intermediary then s o l i c i t s from each panelist, i n d i v i d u a l l y and independently, an answer to the basic question, and also his reasons for giving the answer, and possibly information on a (6 ) set of respondent related cue variables v. When t h i s i s done for a l l the members of the sample group, the f i r s t stage of data c o l l e c t i o n i s over. The intermediary then prepares a composite l i s t of reasons out of the reasons given by the panelists at the f i r s t stage. At the second stage, the composite l i s t i s presented to each panelist. Each panelist i s then asked to answer the basic question again. However, when he answers the question, he i s required to indicate which of the reasons i n the compo-s i t e l i s t he would use to j u s t i f y his own answer, and to provide new (5) An intermediary i s an indiv i d u a l or a group of individuals who designs and conducts the data c o l l e c t i o n process. (6c) Guedvarlables are explanatory variables i n a regression sense, and re-late to idheodemogr.aphic, socioeconomic, and other sociopsychological background of the respondent. reasons not contained i n the l i s t . He may delete some given previously by the panel (including his own). After the second stage i s completed, the intermediary prepares a new composite l i s t by including new reasons (given i n the second stage)aand dropping deleted ones (deleted i n the second stage) and goes for the t h i r d stage. The process continues u n t i l the judgments s t a b i l i z e , that I s , u n t i l i t i s found that the changes i n the panelists' responses are "small". 4 Objectives of the Study Although the method of qua l i t a t i v e controlled feedback, as defined and discussed above, i s conceptually promising and t h e o r e t i c a l l y a much more sound technique for obtaining judgmental data, i t s worth can only be assessed by evaluating i t i n r e a l world situations. I t i s only when the method i s applied empirically that procedural problems can be iden-t i f i e d , and subsequently, solutions can be sought and found. To fi n d guidelines, so the method could be made operational, i t was necessary to carry out a f e a s i b i l i t y study. By studying f e a s i b i l i t y we mean examin-ing every aspect of the method i n the l i g h t of an actual application. In p a r t i c u l a r , we have i n mind the following questions: (1) Is i t possible to motivate people to take part i n a multistage qu a l i t a t i v e feedback survey? (2) How acti v e l y do participants participate throughout the survey (3) Is i t possible to create interaction among the participants through qua l i t a t i v e feedback? (4) Do people change judgments from stage to stage? We w i l l seek answers to these and other questions i n the course of our study. 5 S t a t i s t i c a l Backgroundv In t h i s section we present some s t a t i s t i c a l topics relevant to our data analysis. The f i r s t two subsections deal with regression models for response. Subsection 5.3 gives an outline of l o g i s t i c regression which w i l l be used for predicting nonresponse. Estimation of t r a n s i t i o n proba-b i l i t i e s of reasons and importance of reasons are described i n subsection 5.4-, and f i n a l l y a measure of round to round va r i a t i o n i n response i s defined i n subsec. 5.5. 5.1 TAe Regression Model for the F i r s t Stage Response We assume that the basic question of judgment can be answered numeri-c a l l y using a suitable scale. Let Z. -, denote the numerical response of respondent i to the basic question at the f i r s t stage, i = 1, 2, N, N being the t o t a l number of respondents. Let denote the value of the kth cue variable for respondent I , k = 1, 2, r , where r i s the number of cue variables. For the f i r s t stage responses, the usual multiple regression model i s adopted. Then, ( Z T J X ) = Xg + u x , ECU-L) = 0, VarCU^>•'•' = E ( U 1 ) = 0, Var ( U p = a 2 l ^ > where Z± = ( Z ^ ) denotes an Nxl vector of res ponses; X = ^i]<) denotes the Nxr matrix of cue variables; g denotes an r x l vector of unknown regression c o e f f i c i e n t s ; U-^  denotes an Nxl vector of random disturbances associated with the f i r s t stage responses; and 1^ (ienote's an i d e n t i t y matrix of order N. W.e note that the assumption of uncorrelated disturbances i s compa-t i b l e with the fact that participants respond independently of each other at the f i r s t stage. (7) Details of the s t a t i s t i c a l background are given i n Press[13] 5.2 A Regression Model f o r the Response after Information Feedback^"' Let n denote the response of the i t h subject at stage n, n = 2, 3, . . . j . We consider 1 the regression of Z^ n on n_-^ which i s given by Z i , n = a n + P n , n - l Z i , n - r + ^ n , where a Raand 3 n n_j_ are scalar constants and U£?nis a disturbance term. For the error structure i n the above model,wwe consider the fact that after information feedback, responses of the panelists are no longer un-co r r e c t e d . This i s , because, at the second and subsequent stages, each panelist views the reasons given by the other panelists, and thus, i s i n -fluenced by others i n giving response. A correlation is,'.therefore, pro-duced i n the errors Uj_ n. We also note that precisely the same informat-. ion i s fed back to each panelist on a given round. In addition, the protocol i s such that the panelists are instructed not to communicate with one another. Thus, the correlation between the responses for any pair of panelists should be the same, on a given round; the correlation might change from round to round, however, since the feedback might change. • ••' : 2 For round n, n ^2, we therefore assume E(u,- n ) = 0; Var(u- ) = o • E(u. u. ) = X , i J=~A. That i s , i n matrix form, i f u = (u. ), l ,n 3 ,n n J ' 1 ' n E(u n) = 0, 2 Var(u n) = (o n - X n) I_N + X n ee' where e denotes an Nxl vector of ones. (.8) For a more general model see Press [13:] 14 (9) 5.3 The Logis t i c Regression The l o g i s t i c regression i s useful when the dependent variable i n a regression i s dichotomous, i n d i c a t i n g , for instance, the presence or absence of some attribute i n an observation. Let Y denote the dependent variable, and l e t Y take values 1 and 0. Thus, for example, Y takes the value 1 i f the attribute i s present, and 0 i f the attribute i s absent. Let p = Prob(Y = 1); then 1 - p = Prob(Y = 0). The equation for the l o g i s t i c regression i s then given by 1 P =—5 -a -*-b£x 1 + e where x i s a * t x l vector of explanatory variables, b i s a rkxl vector of constants, a i s a scalar constant and the prime denotes transpose. 5.4 Importance of Reasons As discussed before, at each stage participants give reasons suppor-t i n g t h e i r answer to the basic question. The panel&s evaluation of the importance of reasons on each stage i s measured by P a n» "the proportion of panelists who give reason a on stage n. I t may also be of interest to estimate the importance of reasons at stage n + l ^ v ^ Estimating the importance of reasons.may be carried out according to the following scheme Let Y n(a) be a random variable taking values 1 or 0 according to whether or not a given respondent gives reason a on stage n. Then the (.9) See, for example, Nerlove and Press [111] (10) The ultimate use of estimating the importance of reasons at the next future stage i s to predict response at that stage (see Press [13]) 15 t r a n s i t i o n p r o b a b i l i t y , TT (a) = PEY^Ca) = i j ^(a) = j ] for i , j = 0,1. Here i t i s assumed that individuals tend to reassess t h e i r slate of reasons anew on each stage, merely on the basis of the composite l i s t presented to them, and thus, the dependencies of the values of Y (a) on successive stages follows a one step Markov scheme. So the probability d i s t r i b u t i o n of Y^(a) depends only on stage n-1 and not on any e a r l i e r stages. For estimating the p r o b a b i l i t i e s , l e t n^j(a) denote the number of panelists who are i n state i , at stage n, and who were i n state j , at stage n-1, for reason a, i , j = 0,1. Then the maximum l i k e l i h o o d estimators of the TT^ .J; (a) are: n 1 1 ( a ) n^Ca) + n 0 1 ( a ) 1 0 ( a ) + n Q 0 ( a ) TT (a) 01 n (a) + n (a) 00^ ' 10 v ' We next note that i 1] = E P[Y (a)) = l|Y (a) = j ] P[Y (a) = j ] i=0 n+1 n n so that an estimator of the l e f t hand side i s P a,n+1 a,n This gives: an estimate of the importance of reason a at stage n+1. 5.5 Measure of Round-to-Round Variation i n Response In order to measure the extent of var i a t i o n i n response from round to round, a quantity Qn may be defined as follows: 1 N 2 Q = - Z [z. - z. V ] n N i = i i,n i , n - l where z. , n = 2,3, ... , i s the response of the subject i at stage n. l ,n Thus Q can be calculated for n = 2, 3, ... . I t can be seen that the n • value of QC.depends on the extent of change i n response from one stage to another; i t s value being zero i f there i s no change i n response. Qn can also be used as a stopping r u l e , that i s , to decide how many times information should be fed back, and when should the process be brought to a hal t . The notion i s to stop i t e r a t i n g i f subjects are not changing t h e i r position "much" from round to round. This means that the process should be stopped when has declined to a "small" value. How "small" i s small enough must be determined subjectively. 17 CHAPTER 3 AN EMPIRICAL APPLICATION 1 The Application Situation For applying the method of qua l i t a t i v e controlled feedback, the f i r s t task for us was to fi n d an application s i t u a t i o n ; a si t u a t i o n where there was an issue of judgmentmaking with an underlying population of interest. Fortunately, at the time of i n i t i a t i n g t h i s research (June,1976), the University of B r i t i s h Columbia was starting construction of an indoor aquatic center on the campus. Construction of t h i s center had been an issue of a good deal of controversy for more than f i v e years within the university community (consisting of more than 25,000 students, faculty and s t a f f ) . In spite of the involvement of such a r e l a t i v e l y large population and a considerable amount of construction cost, no systematic e f f o r t was made (apart from two student referendums of yes-or-no type) to see what the community r e a l l y f e l t , for example, about the importance of having the center. Furthermore, the university was not commited to construct the center, beyond a small i n i t i a l preliminary construction stage, which'could have made use of the s i t e for many other purposes. Apparently the university community was divided into two major sub-groups; one i n favor* of constructing the center, and the other against construction. A t h i r d group of neutral and uninformed people also existed. Also, i t was quite natural to expect that the members of the community had something to say (reasons) as to why the center should or should not be constructed. As•has been mentioned mnffche previous chapter, qua l i t a t i v e feedback, aims at exploiting'these i n d i v i d u a l nuclei of reasons, by an i n t e r -18 acting process of feedback. Under these considerations, i t was found that the method of qualitative feedback could be applied f r u i t f u l l y to s o l i c i t judgments on a question which would be of interest to a l l members of the University community. 2 Background of the-UBC Indoor Aquatic Center It i s well known by the denizens of the Vancouver/Seatle area that ra i n occurs very frequently and that forty inches per year i s t y p i c a l . For some reason however, the existing swimming pool at the University of B r i t i s h Columbia i s outdoor, so that i t i s not used a large fraction of the time by most of the University community. In early 1970, a recreation group of the university asserted the need for a covered swimming pool on campus, and proposed that the e x i s t i n g Empire pool be covered. The Alma Mater Society (AMS) of the university took the issue into t h e i r hands. In the subsequent years, i n considera-tion of the r e l a t i v e cost and f a c i l i t i e s , the idea of covering the Empire pool was rejected i n favor of constructing a f u l l scale indoor aquatic center. The AMS also proposed that a student levy of $5.00 per student, per year, be imposed to raise the share of student contribution towards construction of the center. Controversies among the students began as to whether or not the center should be b u i l t and the levy imposed. This led to a student referendum i n October, 1972. About 4000 students p a r t i -cipated i n the referendum, and 67.3% voted i n favor of constructing the center, and of imposing the proposed student levy. The AMS could not proceed with other aspects of the center u n t i l the Board of Governors approved the student levy i n September, 1973. After t h i s , while the AMS was busy with the planning and designing phase of the center, a new move against student funding of the center started i n September, 1974. This led to another referendum as to whether or not the students were s t i l l w i l l i n g to pay the $5.00 fee. In .this November, 1974 referendum about 6000 students participated of which 71 % voted i n favor. F i n a l l y the decision-to construct the f i r s t phase of the center was taken. The t o t a l cost of construction was estimated to be $4.5 m i l l i o n i n 1974. The proposed s i t e ( i n front of the Student Union Building) was also a matter of debate i n the community. Since the construction of the center at, the s i t e would destroy- some beautiful trees and scenic beauty of the s i t e , and also that the ground could be used for needed academic buildings, objections were raised against construction of the center. Apart from these issues, i t was also a question of debate as to whether i t was r e a l l y worthwhile, with regard to the alternative needs of the university commu-n i t y , to have such alarge f a c i l i t y by spending a large amount of money that could be used for more demanding academic buildings (or other purpo-ses). However, i n view of the student referendums only, the decision to begin construction of the center was taken. It was planned that the center be constructed i n two stages. Although a ceremonial st a r t i n g was marked i n November, 1975, actual construction began i n June, 1976. F i n a l l y , i t was decided that i f the center was to be completed, i t would have to be financed by the University, Provincial and Federal Government grants, donations from Faculty and Staff of the Univer-s i t y and the general public, and from contributions from Students. The center was designed to be used for a variety of recreational and academic purposes. But should construction r e a l l y proceed beyond the i n i t i a l stage? 3 Overall Plan of the Survey-Like other sample surveys, careful planning was necessary at the beginning. However, unlike the usual surveys, a feedback survey has the d i s t i n c t i v e feature that i t i s not finished just by f i l l i n g out a ques-tionnaire only once; participants are to be followed up i n subsequent stages with feedback of information (reasons). Thus, planning a feedback survey requires much more attention than i n the usual ones. Planning i s done keeping i n mind the interrelated nature of the survey. The o v e r a l l planning may be described i n terms of four broad actions: (1) s p e c i f i c a t i o n of the underlying population, (2) selection of sampling scheme, (3) determining the number of stages and the sample size and (.4) working out strategies for questionnaire design for each of the stages. These actions are discussed i n the following subsections. 3.1 Population of Interest We are interested i n the judgment of the members of the University community on the general issue of whether or not the university should construct an indoor aquatic center on i t s campus. By i t s usual meaning, the community consists of students, faculty and s t a f f members. However, due to the experimental nature of our survey we planned the population under study to consist of only faculty and s t a f f ; students were dropped for several reasons. Students can be c l a s s i f i e d into two broad categories - "regular" and "i r r e g u l a r " . The f i r s t category consists of the f u l l - t i m e graduate stu-dents and the undergraduates who register for the eight months period from September through A p r i l . Irregular students are the part-time students and intersession and summer session students. However, when we planned the survey i n June, 1976 (Intersession), a large part of the regular students were unavailable. Most of the intersession and Summer Session students attend the university only for two months, and they are not involved i n the the usual student a f f a i r s . Also the Intersession students may not be available to participate i n a l l the stages of the survey, i f i t were to take more than two months, (which was l i k e l y to.tbe the case, since the researcher carrying out the procedural aspects of the survey (the author of t h i s thesis) was a fu l l - t i m e graduate student with many other r e s p o n s i b i l i t i e s ) . These considerations led us to decide that the population under investigation be lim i t e d to Faculty and Staff members of the University. We used the payro l l l i s t , prepared by the department of Finance of the University and published by the Data Processing Center, as our frame for drawing the sample. According to the June, 1976 s t a t i s t i c s , a t o t a l of 2,194 Faculty and 2,825 Staff members were employed i n the University, a t o t a l population size of 5,019. 3.2 Sampling Scheme In view of the nature of the population i t was decided, a p r i o r i , that a s t r a t i f i e d random sample be drawn. The population consisted of two groups of people, Faculty and Staff, who were quite d i f f e r e n t with respect to type of job, educational background, and probably, but most importantly, with respect to t h e i r attitudes towards construction of the center. A Faculty member may be expected to prefer money spending on a project which i s more of academic nature than on a project which i s more of a recreational nature. A s t a f f , on the. other hand, may not be that much., biased towards, having an academic pro j ect. In short,. on an average, a s t a f f may be expected to attach higher importance to the center than that of a Faculty member. These considerations lead us to choose a s t r a -t i f i e d random sample with two s t r a t a . 3.3 Number of Stages and Sample Size Although, i n p r i n c i p l e , a feedback survey should be continued u n t i l responses of the participants s t a b i l i z e , i n t h i s academic application, . due to l i m i t a t i o n s of time, and the fact that c l e r i c a l work had to be handled by the author only, and other factors, we fixed the number of stages beforehand, at three. S t a b i l i z a t i o n of judgment might or might not have occured within the three stages. However, i t i s expected that after a few rounds subjects are not l i k e l y to change t h e i r responses too much. The question of how many stages i s enough i s related to determining the sample s i z e . We were aware of the fact that at each stage of the survey, i t could be expected that some nonresponse would occur. Thus-; the number of participants (sample size) at the f i n a l stage was less than the same number at the f i r s t stage((about 63% response r a t e ) . A relationship between the f i n a l stage sample size and the rates of nonresponse at different stages can be established. Let r n be the non-response rate at stage n and l e t f^ denote the sample size at the begin-ning of the f i r s t stage; then the number of subjects, f , who complete K stage k i s given by the integer part of the rig h t hand side of the f o l l o -wing equation: f = f (1 - r )(1 - r ) ... (1 - r ) k • 1 1 2 k where O ^ r . ^ l f o r i = l , 2, ...,k. In p a r t i c u l a r , i f r =..r = ... = r = r (say), then In our case, with three stages, we have k = 3. We also desired to end up with a f i n a l stage sample" s i z e , f ,--of--at least 100.. With a ( l l ) ' ' rather pessimistic outlook, we took f to be 190. The next task was to allocate the sample size of 190 between the two strata of Faculty and Staff. With, a view to ending up with approxi-mately equal numbers from each group, and considering the fact that the Faculty members are more l i k e l y to move out of the university during summer ( our f i r s t data c o l l e c t i o n period), we allocated a sample size of 105 to Faculty and 85 to Staff. When the actual sample was drawn, i t was found that 9 of the Faculty members and 4- of the Staff members l e f t the university and were not available for s o l i c i t a t i o n . Thus, f i n a l l y we were able to s o l i c i t response from 81 Staff members and 96 Faculty members at the f i r s t stage, giving a revised f i r s t stage sample size of f = 177. 3.4 Strategy for Questionnaire Design To maximize consistency i n questionnaire design at various stages, we found i t convenient to set up a general strategy for questionnaire design; the thinking behind t h i s strategy was the following. Questionn-aires of a feedback survey serve two purposes: one, to get desired i n f o r -mation from the subject, and, two, to feed appropriate information back to the subject. In preparing questionnaires at each stage of the survey, one needs to be e x p l i c i t about these two types of information. Once t h i s (11) In actual practice, as w i l l be found l a t t e r , we ended up with an fg of 111 and r, = .17, r 9 = .19 and r q = .07. has been done, the next task i s to prepare questionnaire items for getting information and determine how to present the information we want to feed back. In summary, the following are the two steps we used i n preparing questionnaires: Step 1. Identify what information to 'get' and what information to feedback. Step 2. Prepare questionnaire items to get information and work out a format for feeding information back. In the rest of this chapter, we w i l l describe questionnaire preparation with reference to these two steps. 4 F i r s t Stage We have already discussed i n the preceding section the background for s t a r t i n g the survey. Questionnaire preparation for the f i r s t stage i s now described below. 4.1 Questionnaire Preparation We follow the two steps mentioned i n the l a s t section. Step 1. According to the methodology described i n Chapter 2, we need the following information from each subject, independently: (1) Numerical answer (or judgment) to a basic judgmental question, (2) Reasons supporting the answer to the basic question, and (3) Information on cue variables. Also,the panelists need to beoproVided with the background i n f o r -mation about the issue of judgmentmaking. Step.2. F i r s t we discuss the formulation of the basic question and the questionnaire items on cue variables. The Basic Question. The basic question i s intimately related to the issue of judgmentmaking. In our case, the general issue was whether or not the university should b u i l d an indoor aquatic center. However, we are not interested i n a 'yes' or 'no' type answer, as i s usually done i n voting. We are rather interested i n a value judgment which may r e l a t e to such characteristics as iimportance','necessity', ' d e s i r a b i l i t y ' , etc. Also, we need the question to be such that i t can be answered numerically using a given scale. The characteristic we chose was 'importance'; and the basic question was: " How important (necessary1)' do you f e e l i t i s for the University of B r i t i s h Columbia to complete construction of an indoor aquatic center on the campus that would be available for use by students, faculty and s t a f f and t h e i r f a m i l i e s , and the general Vancouver commu.-.' . n i t y ? " Scale. For answering the above question numerically, a scale needed to be specified. A 9-point rati n g scale was designed to represent i n -divi d u a l feelings about importance. The two extreme points of the scale were chosen to be 'Extremely Unimportant' and 'Extremely Important'; these two end points were assigned numerical values 0 and 100 respecti-vely. The whole range between 0 and 100 was then partitioned by seven other points at equal i n t e r v a l s , the i n t e r v a l being 12.5. The scale points were: 0, 12.5, 25.0, 37.5, 50.0, 62.5, 75.0, 87.5 and 100. The next task was to assign verbal equivalents to the seven remaining points. F i r s t , the point 50.0 was chosen to be the neutral point, and thus was assigned 'indifferent or neutral'. The remaining s i x points were then assigned verbal statements i n such a way that they conform to the equal numerical distance as much as possible. They are: 'Very Unimportant' (12.5), 'Moderately Unimportant' (25.0), 'Somewhat Unimportant' (37.5), . t - . ' /.<??.-. 5 ) , and 'V . - Iinrortant 'Somewhat Important' (62.5), 'Moderately Important' (75.0), and ' Very Important' (87.5). (12) Selection of Cue Variables. Considering the nature of the res-ponse variable (importance r a t i n g ) , the following cue variables were (13) selected. Selection was done on a subjective basis. Variables, such as, academic status, i . e . , whether a participant i s a faculty or s t a f f (14) member (4), 'sex'(7), 'whether or not a participant knows how to swim' (9), 'whether or not the participant l i v e s on campus'(10) are obviously relevant. In selecting some other variables, attention was given to pick up some psychological variables and variables, related to u s a b i l i t y of the center by a participant. In the f i r s t category are: 'whether or not the participant had already donated'(12), 'in case a panelist did not donate, whether or not he was w i l l i n g to donate'(13) and 'how much annual fee a participant would be w i l l i n g to pay for use of the center' '(J14). In the second category are: 'whether or not the participant's family members would use the center'(8) and 'frequency of use by the participant'(16). Two other variables are: 'whether or not the p a r t i -cipant had a swimming pool near or i n his residence'(15), and 'whether or not i t takes the participant more than 30 minutes to commute to the campus'(11). Variables 4, 7, 8, 9, 10, 11, 12, 13, and 15 are dichotomous and participants were asked to check 'yes' or 'no'; variable 14 i s quan-t i t a t i v e . For variable 16 an ordered categorical scale was used with 4 (.12) Some other cue variables were introduced i n the second and t h i r d stages; they w i l l be discussed i n S e c 5 and Sec. 6. (13) Two questions related to the students were also added i n the ques-tionnaire. However students were l a t t e r excluded from the survey. (.14) These numbers i n the parentheses are the corresponding item numb-ers i n the f i r s t stage questionnaire (see Appendix A-,). categories. Information Feeding. As has been, mentioned i n Step 1, the only-information we need to give i s the background information about the issue of judgmentmaking, which, i n our case, i s related to the construc-t i o n of the center. This was done by using a l e t t e r accompanying the questionnaire. The l e t t e r was also car e f u l l y composed to educate pane-l i s t s about the entire process of the survey and i t s objectives, and also to give necessary instructions for f i l l i n g out the questionnaire. In p a r t i c u l a r , i t was emphasized i n the l e t t e r that participants should f i l l out the questionnaire independently (without consulting with one another) and that they should give reasons for t h e i r answers. P i l o t Survey. After the preliminary questionnaire was prepared, a p i l o t survey was carried out for testing i t . The respondent group cons-isted, of 15 subjects from Faculty, Staff and Students with 5 i n each cat gory. The p i l o t survey brought out some problems with the preliminary questionnaire and demonstrated where modifications were needed; accord-ingly the questionnaire was revised. 4.2 Data Collection Data c o l l e c t i o n was done by f i r s t contacting each subject i n the sample group by telephone, and delivering the questionnaire personally. It was then l e f t with the participant to f i l l out and send back by (IS) mail. In the.early part of data c o l l e c t i o n , we attempted a face-to-face method of administering the questionnaire; i . e . , the participant, was asked to complete the questionnaire i n the presence of the i n t e r -(15) Since the respondents were a l l employees of the university they ".-. -could use the^^ viewer. However, i t was soon found that respondents were demanding more and more that the questionnaire be l e f t with them so they could f i l l i t out according to t h e i r convenience and return by mail. The reason for t h i s , according to the interviewer's experience, was that the respondents wanted more time to think about the reasons before they record them (reasons); since respondents were s o l i c i t e d during o f f i c e hours, possib-l y , they did not want to divert t h e i r attention to something which needed some careful thinking. At t h i s point, i t i s worthwhile to mention two points - quality, of response and rate of response. The issue of the relationship between quality of response and response time has been discussed by Sackman [14J. In our case, i t has been observed that the respondents wanted not only s u f f i c i e n t time to think about answer to the basic question, and t h e i r supporting reasons, but also that the time should be according to t h e i r convenience (e.g. on weekends or lunch hours). I f the respondents were pressed otherwise, the quality of response might be poor and also i t could have an adverse effect on the response rate on subsequent stages. In view of the 'follow up' nature of our survey, one of the tasks of the f i r s t stage was also to motivate participants to take part i n the l a t t e r stages of the survey. In consideration of these factors, i t was decided that the questionnaires be delivered to the respondents i n person and be l e f t with them for completing and returning by mail. A t o t a l of 147 (= 76 faculty + 71 s t a f f ) subjects participated i n the f i r s t stage out of a t o t a l of 177 (= 96 faculty + 81 s t a f f ) s o l i c i -ted. The response rate was 83.05 percent. 5 "Second Stage 5.1 Questionnaire preparation i n the second stage involves certain things which were not encountered i n the f i r s t stage; one of these i s the preparation of a composite l i s t of reasons. This l i s t has to be prepared from the reasons the participants gave at the f i r s t stage. Step 1. Each participant should again answer the basic question Calready formulated i n the f i r s t stage), and also give reasons suppor-ti n g his answer. However, now a respondent should give reasons from a composite l i s t (to be provided), and i n addition, may give new reasons not contained i n the l i s t . It may also be desirable to get information on some additional cue variables. Although, information on the cue variables should be asked at the f i r s t stage, i n actual practice, i t may happen that the inves-tig a t o r discovers some additional cue variables that were not included i n the f i r s t stage, but which are deemed to be important. We picked up three additional cue variables. In our case, they were: 'age of the (16) p a r t i c i p a n t , 'duration of membership of the participant with the university university community', and 'category of job' (e.g. administrative, c l e r -i c a l , etc.) i f the participant was a s t a f f member. Now we id e n t i f y the information f o r feeding back. F i r s t , each participant should be given a composite l i s t of reasons. Apart from (16) In f a c t , age was included i n the preliminary f i r s t stage questi-onnaire. However, i t was observed i n the pilot.survey that many female respondents declined to state t h e i r age. Anticipating that t h i s item might have adverse psychological effects on the p a r t i c i -oatiopation of females, the item was dropped from the revised f i r s t stage questionnaire. that, we need to consider the issue of whether or not a panelist should be provided with the response (answer to the basic question) he gave at the f i r s t stage as well as the reasons he gave at that stage. This issue needs some examination. F i r s t , i f a participant i s provided with his l a s t stage's answer, and reasons, two things may happen: (1) on being reminded, he may deve-lope a bias towards s t i c k i n g to his o r i g i n a l answer, and (2) he may • develope a self-consciousness and s t i l l f e e l free to revise his o r i g i n a l answer and reasons i n the l i g h t of the new information contained i n the composite l i s t of reasons. Second, i f the participants are not provided with t h e i r o r i g i n a l answers and reasons, we may run into the problem of losing uniformity i n the experiment, i . e . , some of the participants might be able to r e c a l l t h e i r answers and reasons, while others may not; t h i s would be l i k e l y to create a difference i n behavior at the second stage. Now, as has been mentioned before, the objective of qu a l i t a t i v e controlled feedback i s to provide panelists with maximum opportunity to share group information and thinking so that they can make a reasoned judgment (with minimal s o c i a l pressure). On t h i s basis, we decided that a panelist should be given the opportunity of reviewing his l a s t stage's answer and reasons. In summary, the information we wanted i s : (1) answer to the basic question,aand (2) reasons from within the composite l i s t or outside the composite l i s t , and (3) information on cue variables. On the other hand, the information we needed$:t^<prps>ide to the group includes: (1) a compo-s i t e l i s t of reasons, (2) the participant's own f i r s t stage response, and (3) the participant's own f i r s t stage reasons. Step 2. F i r s t we discuss preparation of the composite l i s t of reasons. Preparation of the Composite L i s t . Let us, f i r s t , examine the general nature of the reasons given by the participants. Each subject was asked to write his own reasons. It was found that more than 90 percent of the respondents wrote t h e i r reasons. While many of them became precise, and apparently re a l i z e d the objective of asking reasons, nonetheless, there were others whose reasons were not quite clear (at least to the investigator).. Also, reasons given by one panelist ..some-times seemed the same as those given by another; a close examination revealed differences i n meaning and emphasis. This inherent difference i n the meaning of sentences made the task of making a single reason out of two or more, d i f f i c u l t . Nonetheless, as i s done i n the case of enormous s t a t i s t i c a l data, i t i s necessary to process and summarise the verbal statements i n order to make the information contained i n them comprehensible to a respon-dent. This forces the investigator to make some kind of trade-off bet-ween dropping very minor d e t a i l s to reduce the number of conceptually d i s t i n c t reasons to a minimum, and some loss of group information. At t h i s point, we must mention the phenomenon we observed of i n t e r -preting and answering a question with respect.to different frames of reference, by different persons. This phenomenon was also observed by Campbell [2] and Speak [15]. They found from empirical research that people interpretethe same question according to. widely d i f f e r i n g frames, of reference... I t s existence, i n our case, could be i d e n t i f i e d from the reasons the panelists gave. F i r s t , l e t us look us at the basic ques-t i o n . The question asked the respondents for judgments about the impor-tance of the center for the community. It was, therefore, expected that the reasons given by a participant should have also refered to the comm-unity as a whole. However, i t was found that some of the participants ( r e l a t i v e l y few) gave reasons which may be termed as 'personal reasons'; for example, " I don't swim and w i l l never use i t " , "I enjoy swimming and want to use i t at lunch hour',' etc. This indicates that the concern-ed participants interpreted the basic question from the viewpoint of importance to himself - the frame of reference for answering the ques-t i o n were different from what was expected. Whereas questions may arise as to what to do with these reasons, the more basic question for us was: How to check against d i f f e r i n g fram-es of reference, or interpreting the question d i f f e r e n t l y , i n subsequent stages? In order to keep the respondents from repeating the same pheno-menon i n the second,sandesubsequent, stages, we dropped the personal reasons from the composite l i s t — an attempt towards a uniform frame of reference. Before going on to discuss actual preparation of the composite l i s t , we need to mention two other points. F i r s t , the number of reasons to be included i n the l i s t , and second, the number of words i n the statement of a reason. I t i s possible to prepare a very exhastive l i s t containing a l l points of views presented by the partipants; but, i n that case the l i s t may contain a very large number of reasons. A very long l i s t of reasons may have an adverse psychological effect on the participants i n that thay may f e e l discouraged to go through a l l the reasons very ca r e f u l l y . For example, they may read only the f i r s t part of the l i s t and then go to answer the basic question; t h i s i s (17) l i k e l y to have an effect on the quality of response. The second point concerns the number of words i n the statement of a reason. Salancik, et a l [1§] found, i n the context of Delphi event statements, that people tend to interprete sentences i n a variety of ways i f the number of words i n the sentence i s both too few and too great; i n the f i r s t case, because of i n s u f f i c i e n t constraints on i n t e r -pretation and i n the second case, there are too many elements to a s s i -milate into a single interpretation. Thus, a kind of balance has to be made so the number of reasons i s not too large, and also the statement of a reason i s not too big or short and i s easy to have a single interpretation. We, now, describe the par t i c u l a r technique we used i n preparing the l i s t . After studying each participant's slate of reasons very care-f u l l y , i i t was found that some sp e c i f i c 'points of view' could be iden-t i f i e d i n a participant's reasons. These ^points of views' were sorted out and noted down. I t was observed that the ^points of views' could conveniently be put into one of the two broad categories, 'pro' and 'con'; 'pro' meaning 'in favor of having an aquatic center' and 'con' meaning 'against having such a center'. The major points of view within the pro category were: 'year-round (17) In order to see whether ordering of reasons has an effect on the answer, or reason-giving, randomised and nonrandomised l i s t s were introduced i n the t h i r d stage (see Sec?e6, Appendices A3 and A4 and Sec. 7 of Chapter 4) swimming f a c i l i t y (with emphasis for winter)', 'need of the physical education department for t r a i n i n g purposes', 'the center's s o c i a l role as a mixing up media between university-people and the surrounding community', 'need for general recreation', 'the center's role for pro-moting standard of competitive swimming' , and 'shortage of swimming f a c i l i t i e s i n Vancouver'. The points of view within the con reasons could be i d e n t i f i e d as: 'university i s mainly an academic i n s t i t u t i o n ' , ' p r i o r i t y of spending money on academic rather than recreational b u i l d -ings' , 'covering the exi s t i n g pool' , 'abundance of indoor and outdoor pools i n the c i t y ' , 'center's l i m i t e d expected use to the community', 'large cost compared to li m i t e d use' , 'plenty of recreational f a c i l i t i e s on campus', 'alternative proposals for spending the money' and 'a l t e r -native for physical education department'. These points of view need not be completely mutually exclusive; but, i t i s the task of the investigator to judge which point of view f i t s best to the reasons given by a certain participant. In doing t h i s , some major details may have to be dropped. Once the points of views of each participant were i d e n t i f i e d , the next task was to sort out the state-ments from each reason's slate to different points of view groups. Care was taken to see that the ove r a l l arguments presented by a participant might not be distorted when doing t h i s . The f i n a l step was to compose a statement from the indiv i d u a l state-ments within a certain point of view group, such that i t best represented a l l of them. Thus, corresponding to each point of view, we had a reason in the composite l i s t . I t can be seen i n Appendix A2 that the reasons 1 through 8 are the pro reasons, while the reasons 9 through 17 are the con reasons. 35 Cue Var.iabl.es. We have already discussed selection of cue v a r i a -bles i n Step .1. The questionnaire item for age of the participant was prepared keeping i n mind that the participants should not be required to write t h e i r exact age. Thus, they were asked to check one of f i v e (18) age groups. A sim i l a r questionnaire item for the variable,'dura-t i o n of membership i n the UBC community' was also included i n the ques-tionnaire. 5.2 Data Collection At the second stage, response was s o l i c i t e d only from those who participated i n the f i r s t stage. The entire data c o l l e c t i o n was done by mail questionnaires using the university campus mail. Participants were informed beforehand that questionnaires were being mailed. Those who f a i l e d to return the questionnaire within a certain deadline, were reminded by telephone c a l l s , or by reminding l e t t e r s . It took about 8 weeks to complete the data c o l l e c t i o n . In the second stage, 147 persons (=76 faculty + 71 s t a f f ) were s o l i c i t e d . A t o t a l of 119 response (= 58 faculty + 61 s t a f f ) were obtained. The o v e r a l l response rate was 80.95 percent. 6 Third Stage The general procedure for conducting the survey was the same as i n the second stage. However, some new elements, such as, randomisation of the reasons i n the composite l i s t , and use of s e l f confidence ratings for assessing confidence i n judgment was. made.. We discuss, them below. (18) In contrast to our p i l o t survey i t was found that at the second stage a l l of the female participants answered t h i s item on age. 6.1 Questionnaire Preparation Step 1. Apart from the answer to the basic question, reasons from a new composite l i s t L were needed from the participants. In addition, i t • was decided that information on a new cue variable, related to the loca-t i o n of the participant's residence,• was"needed, and information about the confidence i n judgment of the participant should be asked. .Informa-t i o n on confidence was sought to see whether one's s e l f confidence i n judgment increased after feedback. As i n the second stage, each participant needed to be provided with a new composite l i s t , his second-stage response to the basic question, and the reasons he gave i n the second stage. Step 2. Preparation of the composite l i s t at the t h i r d stage was much easier, since only a few new reasons were generated by a few subje-cts. There were 9 new reasons, generated during the second stage, each, of which represented a s p e c i f i c point of view. These reasons were added to the second stage composite l i s t of 17 reasons, which gave a t o t a l of 26 reasons,,in the t h i r d stage composite l i s t . Next, i n designing the format of the composite l i s t , i t was decided that two different formats be used - one with a l l of the "pro" reasons appearing at the f i r s t part of the l i s t , followed by a l l the "con" reasons, and the other, with ran-domised ordering of reasons. The objective of doing t h i s was to see whether the two procedures would produce any differences i n response. In order that participants might recognize the new reasons, a '"' was attached to each of them. Participants were reminded of t h e i r l a s t stage's reasons by checking i n the l i s t , , (see Appendix AM) Another point related to the l i s t of reasons needs to be mentioned 37 here. Since a participant had no knowledge about the proportion of persons who gave a certain reason, i t was anticipated that he might deve-lope a tendency towards supposing that each reason was given by an equal proportion of persons. Since such a supposition might have an effect on his selecting reasons, i t was decided that participants should be e x p l i -c i t l y warned against such a supposition. In e f f e c t , an warning was i n c l -uded at the beginning of the questionnaire (see Appendix A3)., The only cue variable at t h i s stage was "location of residence". For preparing the questionnaire item the Greater Vancouver Metropolitan Area was divided into 9 regions (including an "other" category), and respondents were asked to check the l o c a l i t y i n which they l i v e d . '.) Our l a s t questionnaire item was "confidence r a t i n g " . A 9-point rating scate was used for the purpose, and participants were asked to indicate on the scale t h e i r s e l f confidence about the judgment they gave at the t h i r d stage. 6.2 Data Collection In the t h i r d stage, only those subjects who completed the second stage were s o l i c i t e d . In view of the two types of questionnaires - with randomised and nonrandomised composite l i s t of reasons - the entire group was divided into four subgroups on a'_':random selection basis. These sub-groups consisted of two subgroups of equal size from each of the groups faculty and s t a f f . Out of the two subgroups from f a c u l t y , one was ran-domly picked for the randomised l i s t , and the other for the nonrandomised l i s t . In a s i m i l a r way, the two subgroups of s t a f f were also assigned to the two types of questionnaires. Data c o l l e c t i o n was done through mail questionnaires i n a similar way as was done i n the second stage. The time taken to complete the data c o l l e c t i o n was about 8 weeks. A-total number of 119 subjects (=58 faculty + 61 s t a f f ) were s o l i c i t e d of which 111 responses (54 fac-u l t y + 57 s t a f f ) were obtained. The o v e r a l l response rate was 93.28 percent. 7 The Control Group We need to mention that a considerable amount of time (about 6 months) had passed i n going from the f i r s t stage to the t h i r d stage. The f i r s t phase of the construction of the center was well i n progress by t h i s time. In addition, some other conditions might have changed; d due to which subjects f e l t i n a different way (about the issue of judg-mentmaking) than what they f e l t at the f i r s t stage of the survey. This change in.the conditions of the experiment might have an effect on Its. r e s u l t s . As for example, i t was found i n the course of the survey, that a certain proportion of participants changed response from stage to sta-ge ; the question arises as to whether the change i n response that occured was due to feedback of information (reasons), or simply due to the pass-age of time, or, possibly, due to both feedback and time. In order to resolve questions l i k e t h i s , a new group of subjects was selected and s o l i c i t e d . This group was used as a "Control Group". The control group consisted of an independent group of people (who were not i n the experimental group at any stage), selected randomly. This group was s o l i c i t e d at the same time as the t h i r d stage. However, i n case of the control group, the f i r s t stage questionnaire (with some additional cue variables) was used (see Appendix A5). Thus:, information-given to the control group was the same as that given to the experi-mental group at the f i r s t stage; there were no reasons fed back, however, as the control group was given only the f i r s t stage questionnaire. A random sample of 110 subjects were selected for the control group and were s o l i c i t e d by using mail questionnaires; 89 responses were obtained which resulted i n a response rate of 80.91 percent. 40' CHAPTER 4 DATA ANALYSIS This chapter i s devoted to the data analysis. In Section 1, res-ponse rates at different stages have been compared. Section 2 deals with the empirical judgment d i s t r i b u t i o n s ; three hypotheses have been tested comparing the d i s t r i b u t i o n s . In Section 3, the phenomenon of judgment change from stage to stage i s considered i n d e t a i l . Section 4 i s devoted to estimation of the regression models of response which were described i n Chapter 2. In Section 5, we examine whether or not the dropouts of the survey are l i k e l y to have an effect on judgment di s t r i b u t i o n s . We studied the behavior of participants with-respect to reason-giving i n Section 6. In Section 7, we examined whether ordering of the reasons i n the composite l i s t makes any difference on the part of the participants i n giving judgmentsoor„reasons. F i n a l l y , the d i s -t r i b u t i o n s of confidence ratings are compared i n Section 8 to see whether feedback of information increases confidence i n judgment. 1 Comparison of Response Rates Table I , on the following page, gives the response rates at each stage of the survey and also for the control group. Rates for the two groups - faculty and s t a f f - are shown separately. F i r s t , we compare the response rate-' at the f i r s t stage with that of the control group, and second, between the rates of the three stages of the experimental group. I t i s seen that the rate i n the f i r s t stage i s l i t t l e higher than that of the control group. Recall that the f i r s t stage data c o l l e c t i o n was done by personally delivering the questionnaire to the Table I.. Response Rates Groups Stage Number No. of Response S o l i c i t e d response rates(%) Faculty 1 96 76 79.17 2 76 58 76.32 3 58 54 93.10 i Control 55 46 83.64 i 1 Staff 1 81 71 87.65 . 2 71 61 8 5.92 3 61 57 93.44 Control 55 43 78.18 Overall (Faculty 1 17.7 147 83.05 S Staff 2 147 119 80.95 Combined) 3 119 111 93.28 Control 110 89 80.91 participants, and then by telephone follow ups, while the control group was done completely by mail questionnaires. It i s our contention that the higher rate of response at the f i r s t stage has occured due to higher motivation to participate brought out by personal contact. Now, compar-ing the response rates of the experimental group, we find that whereas the rates at the f i r s t two stages are almost equal, the t h i r d stage rate d i f f e r s from them by at least 10 percent. An explanation for t h i s may be that participants who were highly motivated and prepared, completed the second stage, and most of them also completed the t h i r d stage; on the other hand, subjects who participated only reluctantly at the f i r s t stage, dropped out at the second stage. Thus, we may, i n general, expect a hugher response rate at the t h i r d stage compared to the e a r l i e r two stages, i n a three-stage survey. Such a conjecture seems reasonable for the time delays between stages, that we experienced i n t h i s experiment. I f interstage times were substantially reduced, the response rate pattern 42' would probably change also. F i n a l l y , 62.71 percent of a l l the subjects who were i n the sample at the beginning of the survey completed a l l the three stages of the survey. 2 The Empirical Judgment Distributions In t h i s section, we study the empirical d i s t r i b u t i o n s of judgment at the three stages of the experimental group, and also for the control group. These di s t r i b u t i o n s are of fundamental importance i n understand-ing the judgment structure of the group. They reveal various f a c t s , e.g., whether or not there exist subgroups of d i f f e r i n g judgment; i f there are, how divergent the subgroups are i n th e i r judgments; and so on. Apart from comparing the di s t r i b u t i o n s from t h e i r graphs, means and standard deviations (Subsections 1 and 2), we check three hypotheses of interest i n Subsection 3. 2.1 Graphical Representations of the Judgment Distributions Table I I gives the dis t r i b u t i o n s of judgment i n r e l a t i v e frequen-cies for the three stages and the control group. Histograms of the dist r i b u t i o n s are given i n Figures 1(a) through 3(c) and F i g . 4. In Figs. 5(a), 5(b) and 5(c) the superimposed, freehand drawn, frequency curves for the f i r s t , second the t h i r d stage are shown; they refer res-pectively to the combined group, Faculty and Staff. F i n a l l y , i n Figures 6(a) and 6(b), superimposed curves of the f i r s t stage vs. the control, and the t h i r d stage vs. the control, respectively are shown. Note that no ef f o r t has been made to s t a t i s t i c a l l y smooth'out the bumps i n the frequency curves - they were just used for approximate descriptive o 0 00 - j CD Cn CO NO H 0 -<1 cn NO 0 cn NO 0 cn O cn 0 cn O cn O CO H H H H cn CO -F H CO NO NO tn cn CO 00 H ^4 UO UO Cn 0 CD CO H H 0 CD CD CD ro NO H 00 NO uo NO NO en On CO cn • • • • •O • • • cn 00 00 NO NO NO NO cn cn H H NO 00 00 CD CD H CD H CO H H -F CO I-1 -F UO 00 cn • • • • ••J • • • • 00 -P cn cn S3 cn O H -F 0 H CO H H O H H H H NO H H H - J 00 NO H NO <] -F cn -F cn NO H uo -F 00 cn 0 H NO NO H CD H H CD NO NO H M CO NO OO O CO NO NO CO Cn 0-1 00 CO cn cn NO NO Cn NO H H NO CO H 00 00 H CD H NO M H cn NO CD NO UO UO Cn CO CO cn NO NO CD UO 0 -F 0 NO NO NO H H H NO NO H H NO O - J UO NO NO -F NO CO -F NO UO UO -F H NO H CD CD CD H NO NO H CO H 00 83 cn 00 cn 00 cn cn O O 5y NO cn <] <1 NO H cn - j CD H UO - J CD M H NO CT> H H H -F Cn "-F CD CD 4^ 0 CD • • • to • j) • • « • 00 -F NO -F CO CO 4^ 00 CO 0 H CO H HH H H H H NO H I- 1 H ao CO co CO -0 -F 00 NO 00 UO 0 0 CO CO 0 H - J 0 H -F 4T uo cn 0 <1 0 NO NO NO ~ J 00 CO -F -F CD 4r CD CO UO CD NO CD CD UO CD UO CO O CD cn Cn 00 Cn 00 NO H H 1—1 -Ul cn Cn •~J H 4^ -0 CO <1 CD 00 UO NO •F 00 CO 00 NO -F OO 00 -F ID -0 Response (Judgment) Faculty-Staff Combined Faculty Staff Combined Faculty Staff Combined jFaculty Staff Combined H-<-$ w rt C/J r t Pl oq CD oo CD O O a cx CO rt 0) e r a c o H CO-CO rt (D OP CD O O a r t 4 o 44 comparisons. Our f i r s t observation i s that the di s t r i b u t i o n s i n each of the three stages are probably bimodal. While the empirical picture was a b i t unclear at the beginning, bimodality emerged prominantly at the f i n a l stage, for each group. Thus, at the f i n a l stage, the participants became c l e a r l y divided into two d i s t i n c t groups of thought - one with favorable opinion towards construction of the center, and clustering around 75 (moderately important); and the other with unfavorable opi-nion, and clustering around 25 (moderately unimportant). I t i s interes-t i n g to note that the groups cluster around points which are at an equal distance from the neutral point. While the modal ordinate at 75 decr-eased, from e a r l i e r values, at the t h i r d stage, the ordinate at 25 i n -creased stage by stage (Figs. 1(a), 1(b) and 1(c)). The phenomenon of bimodality i n the di s t r i b u t i o n s needs some care-f u l examination. Let us suppose that the subjects were given only two options, 'yes' or 'no', to give t h e i r opinion about construction of the center. Under such circumstances, we would have expected, as i s usually the case, the group to be divided into two major subgroups - one on the 'yes' side and the other on the 'no' side - and probably a r e l a t i v e l y small group would r e f r a i n from giving any opinion, and would remain neutral. Thus, we would have observed the persons who gave ratings 0 through 37.5 to be on the 'no' side, and those with ratings 62.5 thro-ugh 100 to be on the 'yes' side. In such a case, we would have no way to ascertain the extent of the extremity i n the opinions of the two groups. However, now with a bimodal d i s t r i b u t i o n we not only know the existence of the two opposing groups, but we also know that they cluster '0 25 50 75 . J 100 - Fig.1(a) F i r s t Stage Di s t r i b u t i o n of Judgment of the Combined Group 0 25 50 75 100 Fig.1(b) Second Stage D i s t r i -bution of Judgment of the Combined Group 0 25 50 75 100 Fig.1(c) Third Stage D i s t r i -bution of Judgment of the Combined Group -F w +J o • n A cn o a o 35 r 30 25 20 1>5 10 0 25 50 75 100 0 25 50 75 100 25 50 75 100 Fig.2(a) F i r s t Stage Distribu-t i o n of Judgment of Faculty-Fig.2(b) Second Stage D i s t r i -bution of Judgment of Faculty Fig.2(c) Third Stage D i s t r i -bution of Judgment of Faculty •p cn 35 r 0 25 50 75 100 0 25 50 75 100 Fig.3(a) F i r s t Stage D i s t r i b u -t i o n of Judgment of Staff Fig.3(b) Second Stage D i s t r i -bution of Judgment of Staff 0 25 50 75 100 Fig.3(c) Third Stage D i s t r i -bution of Judgment of Staff 48 35 h 30 w tj 25 cu • n co 2 0 CM O 4-> S 15 o 0) PH 10 25 25 50 75 100 Fig. 4 Dis t r i b u t i o n of Judgment of the Control Group around the judgment neueiei 'moderately important' and 'moderately un-important'. I f these judgment neuclei were J:verymimp6rtant' and' 'very unimportant' , the difference i n opinion i n the two groups would have increased (polarised). Thus, bimodality i n the distrib u t i o n s have been useful not only i n identifying the two opposing groupssbut also to get an idea as to the extent of t h e i r difference. A comparison between the two groups - faculty and s t a f f - reveals some minor differences when compared stage by stage. The bimodality character i s prominent i n faculty d i s t r i b u t i o n s from the f i r s t stage, , whereas^it i s only i n the second stage that the s t a f f d i s t r i b u t i o n s appeared to become bimodal. The l a t t e r had undergone greater change from stage to stage, r e l a t i v e to that of faculty. Although, the.third stage distrib u t i o n s look s i m i l a r , a careful examination reveals that the two 49 % of subj ects 35r-F i r s t Stage Second Stage Third Stage Fig.5(a) F i r s t , Second and Third Stage Distributions of Judgment of the Combined Group Fig.5(b) F i r s t , Second and Third Stage Distributions of Judgment of Faculty-opponent subgroups i n s t a f f are more diverged than those of fac u l t y . ; Now we look at the Figs. 6(a) and 6(b). The shape of the control group d i s t r i b u t i o n i s veryi.much irregular and i t i s hard to find any good s i m i l a r i t y between the frequency curves i n each of the figures, except that the modes with the greatest ordinate are on the rig h t side i n l both cases; but the modal values d i f f e r (75 i n the experimental group I and 87.5 i n the control group). Thus, i t seems from inspection of the graphs that both the f i r s t stage and the t h i r d stage di s t r i b u t i o n s d i f f e r from the control group d i s t r i b u t i o n . 50 2.2 Means and Standard Deviations The means and standard deviations of the various d i s t r i b u t i o n s are shown i n Tables I I I and IV, respectively. Note that the means are not too meaningful since the dis t r i b u t i o n s are bimodal. That is,-'.alth-ough the means s t i l l denote average response, they do not r e f l e c t the point where most of the mass of t h i s d i s t r i b u t i o n l i e s . The s h i f t s of the locations and heights of the modes are more r e f l e c t i v e of d i s t r i -bution changes i n t h i s context. Nevertheless, we notice that the means decreased stage by stage for both the groups, and thus for the combined group. The mean for the s t a f f i s always larger than that f o r the f a c u l -ty. The control group means are near the f i r s t stage means (except f o r f a c u l t y ) . Table III.. Means of Judgment Distributions Groups F i r s t Stage Second Stage Third Stage Control Faculty 50.93 48.38 46.53 60.33 Staff 63.38 59.61 57.46 61.63 Combined 57.32 54.15 52.14 60.96 Table IV. Standard Deviations Distributions of Judgment Groups F i r s t Stage Second Staged Third Stage" Control Faculty 26.77 25.80 27.58 29.29 Staff 25.63 27.88 23660 27.54 Combined 26.92 27.47 29.15 28.47 In contrast to the means, the standard deviations increased stage by stage, implying a monotone increase i n pol a r i s a t i o n of the two groups due either to time effects or feedback effects. Since the t h i r d stage and control group standard deviations are comparable, i t i s not clear which effect may have generated the po l a r i s a t i o n . 2.3 Tests of Three Hypotheses In t h i s subsection, we test three hypotheses comparing the judg-ment distrib u t i o n s of the experimental group at the the f i r s t stage., ahdtthe t h i r d stage and of the control group. The hypotheses are: H : The res'p'onse'sLCo'fi the f i r s t stage experimental group and thos. of the control group came from the same population. 53 ^ A l ' ^ e ^ w o § r o u P s responses came from different populations. : The responses of the t h i r d stage experimental group and those of the control group came from the same population. : The two group of responses came from different populations. : The responses of the f i r s t , stage experimental group and those of the t h i r d stage (experimental group) came from the same population. H : The two groups of responses came from different A3 populations. Although each hypothesis has i t s in d i v i d u a l implications, i t i s more informative, i n our case, to examine the implications of the resu l t s j o i n t l y . We use the standard chi-square test of goodness of f i t for testing the hypotheses (J.9.) S t r i c t l y speaking, the chi-square test i s not applicable i n testing and H^. The basic assumptions of the test are: (1) the two samples are independent (and thus the observations between them are necessarily uncorrelated), and (2) the observations within each sa-r,plsample are independent (and thus they are necessarily uncorrelated). However, i n case of H^ ,» the second assumption i s v i o l a t e d , since the observations obtained i n the t h i r d stage are correlated. The correlation has been produced by the feedback of information i n the sense that one participant's reasons have influenced another p a r t i -cipant's response. In case of H Q 3, both the assumptions are vio-l a t e d , since, i n addition to the above type of co r r e l a t i o n , now the f i r s t stage responses are correlated with t h e i r counterparts at the t h i r d stage', (the two sets being the responses of the same individuals at two occasions). However, due to lack of more appro-priate tests at the present moment, we have used the chi-square t e s t , hoping that the correlations may not turn out to be so s e r i -ous as to upset the conclusions derived from the te s t s . For further With 9 categories i n each of the two multinomial populations, the c h i -square s t a t i s t i c , i n our case i s given by 9 2 2 i = l j = l e i j where f„ = observed frequency i n the ( i , j ) t h e c e l l , e„ = expected frequency i n the ( i , j ) t h c e l l , Under the n u l l hypothesis, the estimated value of i s given by -A f : e. . = ^ N where 2 9 9 2 fn- = -Mi i> = z f n - n and N = I 'E.-f. . 1 3=1 ^ ] i = 1 i ] i=lj=1^3 The chi-square has (9-l)(2-l)=8 degrees of freedom. While computing the chi-square, i t was found, i n each of the three cases, that some of the expected c e l l frequencies were small (less than 10). To make the test applicable, some of the c e l l s were merged^^ The resu l t s of the tests are: Computed (21) Hypothesis chi-square P-value (5 d.f.) H Q 1 10.93 0.053 H 10.87 0.054 02 H 3.17 0.674 03 discussion on t h i s point see Chapter 5, Section 2. (20) Computation of the expected c e l l frequencies are shown i n Appendi-ces DI, D2 and D3. (21) P-value i s the probability of exceeding the observed chi-square. We reject HQ^ and H a T : "the . 0 5 significance l e v e l . HQ^ cannot be rejected. Now l e t us examine the implications of the r e s u l t s . F i r s t we r e c a l l that the control group was s o l i c i t e d about 6 months l a t t e r than the f i r s t stage, and at a time when the construction of the center was well i n progress; also some other conditions might have changed (e.g., a r r i v a l of winter and reduced concern about swimming). We also note that the responses of these two independent groups ( f i r s t stage and control) were obtained using b a s i c a l l y the same questionnaire, and without any feedback of information. Thus the two groups represented the general public feelings at two different times. A difference bet-ween the response d i s t r i b u t i o n s of these two groups may, therefore, be attributed to the effect of time.((In p a r t i c u l a r , i t could be predicted that public attitude towards construction of the center would be more favorable when the construction was already well i n progress). By re-jecting HQ^, we conclude that the twoLdistributions are s i g n i f i c a n t l y d i f f e r e n t . Thus, the control group represented the time e f f e c t . On the other hand, acceptance of H implies that there i s no s i g n i f i c a n t d i f f -03 erence between the f i r s t stage d i s t r i b u t i o n and the t h i r d stage d i s t r i -bution (although the t h i r d stage was done at the same time as the cont-r o l group). This implies that.the t h i r d stage d i s t r i b u t i o n did not have a s i g n i f i c a n t time e f f e c t , (or i n p a r t i c u l a r , that the t h i r d stage d i s -t r i b u t i o n was not affected s i g n i f i c a n t l y by the fact that construction of the center was i n progress). This r e s u l t is. i n conformity with the rej e c t i o n of H Q 2 , which shows that the t h i r d stage d i s t r i b u t i o n i s different from the control group d i s t r i b u t i o n . Since the control group represented the time effect,(and that the control group and the t h i r d stage were done simultaneously), the hypothesis, H > would not have been rejected had there been a time effect i n the t h i r d stage d i s t r i -bution. The question arises: What i s that which prevented the t h i r d stage experimental group from being effected by the time effect? The answer i s 'Qualitative Feedback'. Due to qual i t a t i v e feedback, the subjects i n the t h i r d stage experimental group were exposed to a l l the 'pros' and .-'cons' about the issue of judgmentmaking. They were, thus, able to form t h e i r judg-ments on the basis of reasoned and r a t i o n a l thinking, and were not overwhelmed by the current developement of events (e.g. ongoing cons-tr u c t i o n of the center). This demonstrates the^usefulhessToffquali-t a t i v e feedback i n producing r a t i o n a l judgments. Some further comments. I t i s important to r e a l i z e that change i n individual judgments from stage to stage does not imply a change i n the judgment d i s t r i b u t i o n . While in d i v i d u a l judgments may change, the d i s t r i b u t i o n of judgments may or may not change. This can be i l l u s t r a -ted as follows: Consider any two response categories, say, 37.5 (some-what unimportant) and 62.5 (somewhat important). Suppose n^ and ^ , respectively, are the number of subjects i n these two categories at the f i r s t stage; and suppose, at the second stage, one subject changes res-ponse from 37.5 to 62.5, and another from 62.5 to 37.5, and the frequ-encies i n a l l the other categories remain fi x e d . We f i n d that, although, a change i n in d i v i d u a l judgments has occured, the distributions of judg-ments hasenot changed. However, the converse i s true, i . e . , a change in the d i s t r i b u t i o n implies a change i n the indi v i d u a l judgments. Thus, although we have found, by testing hypothesis comparing the f i r s t stage d i s t r i b u t i o n with the t h i r d stage d i s t r i b u t i o n , that the judgment d i s t r i b u t i o n s did not change s i g n i f i c a n t l y , we have no evidence about whether or not change i n judgments has occured due to feedback of information. We w i l l examine the issue of judgment change from stage to stage i n the next section. 3 Analysis of Judgment Change In t h i s section we examine the pattern of change that occured i n the subject's responses from stage to stage. F i r s t , .we present a summ-ary description of change for three broad categories i n Table V, and then, with the help of a set of t r a n s i t i o n p r o b a b i l i t i e s , we i l l u s t r a t e the pattern of change i n d e t a i l . The basic data tables that w i l l be used i n our discussion are shown i n Appendices C l and C2. They are, i n f a c t , bivariate frequency tables. For example, the entry i n the c e l l corresponding to column 5 and row 7 of Appendix C l shows that 3 subjects who gave a response 62.5 (somewhat important) at the f i r s t stage changed t h e i r response to 37.5 (somewhat unimportant) at the second stage. Thus, the diagonal entries of the table show the number of subjects i n various categories who did not change; the lower diagonal entries represent the number of subjects who changed to lower responses than t h e i r previous responses. The reve-rse i s true for the upper diagonal entries. The row and column t o t a l s are the marginal d i s t r i b u t i o n s . 3.1 A Summary Let us consider the three broad categories of response: 'less than 50', '50' and 'greater than 50'. A l l participants belonging to the f i r s t category gave responses on the'unimportant 1 side of the of the scale; those i n the second category were neutral; and those i n the t h i r d category gave responses on the 'important' side of the scale. These are the three basic groups having opposite or neutral opinions. I t i s of interest to see how they behaved i n changing opinions. Some interesting features cf response change can be observed from Table V. As can be seen from the l a s t row, nearly 30% changed response at the second stage while the same rate i s only about 21% at the t h i r d stage. From the same row we f i n d that majority of subjects who changed response, changed to give a lower r a t i n g . We w i l l see i n Section 6 that the 'con' reasons were more "appealing" than the 'pro' reasons; thus i t i s quite reasonable to believe that as an effect of feedback of the 'con' reasons, on the average, the majority of participants changed t h e i r response to a lower di r e c t i o n . Now l e t us compare the three main categories. The f i r s t thing we notice i s that the neutral respondents had undergone the greatest percentage of change i n both the second and t h i r d stages (columns (9) and (10)). This means that the nneutral sub-jects could be persuaded more to change judgment than those who had already committed. We also f i n d that most neutral subjects (75% and 66.67%) who changed response, changed i n the lower d i r e c t i o n . This i s also indic a t i v e of our e a r l i e r mentioned feedback effect of the 'con' reasons. To see the extent of var i a t i o n from round to round, and Q^ .j as defined i n Chapter 2, were calculated. For the combined group, Qn (n= 2, 3) i s given by Table V. Summary of Change i n Response with respect to Three Broad Categories Category of Response Number of Subjects No. Shanging to lower di r e c t i o n tNo. chang-ing to upper d i r -ection N%-. of f sub j ect s who changed response % of subjects (out of those who changed) changing to lower direc-t i o n % of subjects(out of those who chang ed) changing to upper dire c t i o n Stages 1 2 3 1,2 2,3 1,2 2,3 1,2 2,3 1,2 2,3 1,2 2,3 (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (13) (14) Less than 50(unimpor-ntahti).impor 30 41 42 4 9 3 3 23.33 29.27 57.14 75.00 42.86 25.00 50(indiff-erent or neutral) 13 8 7 6 2 2 1 61.54 37.50 75.00 66.67 25.00 33.33 Greater than 50 (impor-tant) 68 62 60 16 13 2 5 26.47 12.90 88.89 37.50 11.11 62.50 Overall 111 111 111 26 14 7 9 29.72 20.72 78.79 60.87 21.21 39.13 Note(l): Computations of columns (9) through (14) are as follows: (9) = E((5)+(7)) - (2)]xl00, (11) = [(5) - ((5) + (7))]xl00, (10) = C((6)+(8)) - (3)]xl00, (12) = [(6) - ((6) + (8))]xl00, (13) = 100 - (11),and (14)= 100 - 012) Note(2): Columns (6), (8), (10), (12) and (14) refer to change from the second stage to the t h i r d stage; columns (5), (7), (9), (11) and (13) refer to change from the f i r s t stage to the second stage. ID 60 1 111 E (z . - z , .) n , i n - l , i 111 i = l where z • i s the response of the i t h subject at stage n, and i = l , . . . , n, 1 111. The computed values of the Q's are: Q2 = 140.77 and Q3 = 67,57. The var i a t i o n i n response i n going from the second stage to the t h i r d stage i s 48% of that i n going from the f i r s t stage to the second stage. This shows that even i n the t h i r d stage the responses were not quite s t a b i l i z e d . 3.2 Test of Hypothesis I t i s also of interest to see i f change i n judgment by a subject i n one stage i s related to whether or not the participant changed judg-ment i n the previous stage. To be precise, l e t us consider the hypo-thesis: H: Changing judgment i n a given stage i s not related to changing judgment i n the previous stage. This hypothesis can be tested by using a 2x2 contingency table as shown below. Table VI Table VI. C l a s s i f i c a t i o n of subjects with respect to judgment change i n the Second and Third Stages w < H oo Q Pi I—I tc SECOND STAGE Changed Did not change Totals Changed 10 12 22 Did not change 24 65 89 Total' 34 77 111 The hypothesis has been tested using the usual chi-square s t a t i s -t i c for a 2x2 contingency table. The observed value of the chi-square with (2-1)(2-1) = 1 degree of freedom i s 2.03 (using the Yate's correc-t i o n for continuity) which i s exceeded with a probability of .15. We do not reject the hypothesis and conclude that change of judgment by a subject at one stage i s not related to whether or not he changed judg-ment at the previous stage. 3.3 Transition P r o b a b i l i t i e s of Response F i r s t , we w i l l define the t r a n s i t i o n p r o b a b i l i t i e s and explain t h e i r computations. Let z n be the response of a subject at stage n (n = 1, 2, 3). Then the t r a n s i t i o n p r o b a b i l i t y i s simply the condi-t i o n a l p r o b a b i l i t y , p( z n» z^ ) = P[Z n = z n | Z n _ i = zn_j_^ °^ giving res-ponse z^ (z = 0, 12.5, 100) at stage n, given that the p a r t i c i -pant gave response z n _ j _ at stage n-1. Here we assume that the subject's response at the present stage depends only on his response at the pre-(22) vious satge. The Maximum Likelihood estimate of p(zn5 z n _ i ^ i s g i v e n by f ( z n > z n - l } P ( zn' zn-1 ) = — " f (z ) n-1 where f(z^,zn_^) = number of subjects who gave response z n_i at stage n-1 and z at stage n, and f ( z .) = t o t a l number of subjects who gave n . n-1 response ^ at stage n-1. Appendix CI gives the values of f ( z 2 , z ^ ) and fCz^), and Appendix C2 gives those of f ( z 3 , z 2 ) and f ( z 2 ) for the combined group. The p r o b a b i l i t i e s p ( z 2 ,z )*and p(z^,z 2) are shown i n Tables VII and VIII respectively. Note that according to our notation zn a n d zn _ a correspond to column and row respectively. (.22) Recall that, while c o l l e c t i n g data, each subject was reminded only h i s previous stage response. 62 Table.VII. MLE of Transition P r o b a b i l i t i e s of Response from the F i r s t Stage to the Second Stage SECOND STAGE  0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.OJ .11 .38 .08 - .08 .07 .66 .06 .17 .65 .06-.06- .94 1 * Maximum Likelihood Estimates We notice i n both the tables that the largest p r o b a b i l i t i e s occur i n the diagonal c e l l s , indicating that a subject i s most l i k e l y not to change response. Starting from a diagonal c e l l as we move to the l e f t or to the r i g h t , the p r o b a b i l i t i e s decrease, the highest probability occuring i n the adjacent l e f t or adjacent r i g h t c e l l . Thus, when a subject changes response, i t i s most l i k e l y that he w i l l move to one of the next response categories. We also notice that changes took place only i n response categories 12.5 through 87.5; subjects i n the extreme two categories did not change response i n any of the stages. 4 Regression Analysis We have described the regression models of response i n Section 5 of Chapter 2. The models are estimated i n t h i s section. In -Subsec. 4.1, w CD < CO H CO OC 0 12.5 25.0 37.5 50.0 62.5 .75.0 87.5 .78 .30 .11 .60 .20 .08 .07 .03 .10 .80 .38 .20 .03 i n n n 63 Table VIII. MLE" of Transition P r o b a b i l i t i e s of Response from the Second Stage to the Third Stage THIRD STAGE 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 0 1 12.5 .100 .80 .10 25.0 .09 .82 ' .09 w ! CD < IH 00 37.5 .m .36 .43 .07 50.0 .13 .13 .62 .12 Q o 62.5 .83 .17 O w 00 75.0 87.5 100.0 r a r r m — .06 .92 .12 .08 .82 1 '•MMaximum Likelihood Estimates the regression model of the f i r s t stage responses on the cue'variables i s estimated, and regression of (second stage response) on Z ^ ( f i r s t stage response) and that of Z^(third stage response) on 7,^ are estima-ted i n Subsections 4.2 and 4.3 respectively. 4.1 Regression of Z^ on the Cue Variables In t h i s regression, the dependent variable i s the response at the f i r s t stage, which i s measured by the numerical answer the subjects gave. Thus, the observed dependent variable has values 0, 12.5,..., 100. There are, i n t o t a l , 16 cue variables of which 14 are 0-1 qu a l i -t a t i v e and the remaining two are quantitative. These variables are described below. = 1 i f the subject i s a Faculty member 0 otherwise X = 1 i f the subject i s an administrative s t a f f 0 otherwise X = 1 i f the subject i s a c l e r i c a l s t a f f 3 0 otherwise X = 1 i f the subject i s a male 4-0 i f the subject i s a female X = 1 i f the. subject's family would use the center 0 otherwise X = 1 i f the subject knows how to swim 6 0 otherwise X = 1 i f the subject already donated or was w i l l i n g to donate towards funding of the center 0 otherwise Xg = 1 i f there was a swimming pool i n or near the residence of the subj ect 0 otherwise X = 1 i f the subject would never use the center 9 0 otherwise X = 1 i f the subject would occasionally use the center 10 J 0 otherwise X = 1 i f the subject had been a member of University community for less than 5 years 0 otherwise (5 years or more) X = l t i f the subject l i v e d on the west side of the Granville street 12 0 otherwise X-^ g = 1 i f the subject l i v e d on the east side of the Granville street or i n the West End area 0 otherwise X = 1 i f the subject l i v e d i n West Vancouver, North Vancouver or i n Richmond 0 otherwise X = Age of the subject (measured as the mid-value of the age 15 i n t e r v a l of the subject) X = Amount of annual fee that the subject was w i l l i n g to pay for use of the center Data on these variables was available i n the questionnaires for 111 subjects who participated i n a l l the three stages. However, a complete set of observations on a l l the 17 variables (1 dependent + 15 explanatory) were available only f o r 96 subjects, the remaining 15 subjects had missing observations on either X or"-, X, R or both. -L O _1_0 To make use of a l l the 111 data points, we have used a certain computer program which i s designed to make good use of data sets with missing observations. The results of the regression are given i n Table IX. It gives the parameter estimates, t h e i r estimated standard errors, the F-ratios and the p r o b a b i l i t i e s of exceeding an observed F-value. In column 2, the parameter estimates are denoted by small case l e t t e r s (correspon-2 ding to the variable names). The value of R , the c o e f f i c i e n t of determination, i s .38; only 38 percent of the observed v a r i a t i o n was C23)' The part i c u l a r technique the program used i s described i n Appdx. E. Table IX. Results of Regression of Z-]_ on Cue Variables Variables Parameter Standard F-Ratio F-Proba-Estimates Errors b i l i t y Constant 60.98 18.14 • X l b x = -12.01 6.46 3.46 0.06 x 2 b 2 = -15.15 11.18 1.84 0.175 X3 b 3 = - 5 . 0 4 7.42 0.46 0.506 x 4 H = - 0.77 5.69 0.02 • 0.862 X5 1.60 5.65 0.08 0.769 X6 b = 6 - 1.78 9.18 • 0.04 0.826 X7 b 7 = 13.00 5.51 5.57 0.019 X8 b8 = 0.88 5.18 • 0.03 0.841 X9 b 9 = -25.85 8.45 <9335 • 0.003 X10 b 10 = - 7.19 6.76 1.13 • 0.290 *11 b!i= 3.42 5.56 0 .38 • 0.548 X12 b 1 2= -18.91 9.88 - 3.67 • 0.056 X13 b13 = - 5.18 11.26 0.21 0.651 X14 b = 14 -10.13 10.65 0.90 0.347 ^5 b15 = 0.58 0.25 5.43 0.021 X16 b 16 = 0.05 00.07 • 0.43 0.519 d due to the regression. However, under whatever va r i a t i o n has been ex-plained, i t i s interesting to note the significance of some of the co-e f f i c i e n t s . We may consider the c o e f f i c i e n t s b^, b^, bg, b ^ * a n d b^^ to be s i g n i f i c a n t at p-values of .06, .02, .003, .056 and .02 respec-t i v e l y . A negative value of bj_ implies that the f a c u l t y , on the averag tended to assign lower ratings to the importance of the center. A posi t i v e value of b means that those who already donated or were w i l l i n g to donate, tended to assign higher ratings; t h i s i s what could be expec-ted a p r i o r i . On the other hand, a negative value of t)^ implies that people who would not use the center at a l l gave low ratings on the average; t h i s i s also consistent with our usual expectations. However, a negative value of i - s contrary to what could be expected. The variable X i s 1 or 0 according to whether or not a subject l i v e d on 12 & the west side of the Granville street and thus close to the new aquatic center. Since respondents l i v i n g close to the center had greater oppor-tunity to use i t , they should assign higher r a t i n g , and thus, the sign of the co e f f i c i e n t should be pos i t i v e . A positive value of b , although very small, implies that r a t i n g tended to increase with age. 4.2 Regression of on Z^ The regression model described i n Chapter 2 i s given by, for n=2, Zi 2= a 2 + 3 2 1 Z i l + U.2 where, Z^2 and Z ( i = l , 2, I l l ) are the responses of the i t h sub-ject at the second and f i r s t stage r e s p e c t i v e l y , ^ and 3 2 ^ a r e constants o and U i 2 i s the errdrmtermowith E(U^2) = 0, Var(U i 2) = a and Cov(.U i 2,U j 2) = x , i ^ j i In view of the error structure, a simple least squares method of estimation w i l l not be s t r i c t l y applicable. However, for purposes of s i m p l i c i t y , and on the simplifying assumption that the correlation bet-ween the responses of two subjects might not be too large, we have used the ordinary least squares method to estimate the model. • The estimates of the coefficients and t h e i r estimated standard errors are: Estimates Standard t-values p-values Errors a 2 =0.92 2.54 0.36 0.70 B 2 1 =0.93 0.04 23.00 less than .002 Thus 3 2^ i s highly s i g n i f i c a n t and i s not s i g n i f i c a n t . The second stage response may be considered to be proportional to the f i r s t stage 2 response. The value of R i s .83, so that the correlation between Z^ and Z 2 i s .92. 4.3 Regression of on Z 2 Now we consider the regression of the t h i r d stage response on the second stage response. The regression equation i s Z i 3 = a3 + B32 Z i 2 + U i 3 where, as before, Z^2 and Z^3 are the responses of the i t h subject ( i = 1, 2, I l l ) at the second and the t h i r d stage respectively, a„ and B„„ are constants, and U.„ i s an error term with E(U._) = 0, 3 32 i3 x3 2 Var(U^3)=a^-"-and Cov(U^ 3 ,Uj 3) = X^. As i n the previous regression, we again use the ordinary least squares method (assuming similar simpl fying assumptions) to estimate the model. The results are: Estimates Standard 't-values p-values Errors a 3 = -1.76 1.72 -1.02 0.28 = 1.01 0.03 33.67 less than 0.002 Again the slope coeffi c i e n t i s highly s i g n i f i c a n t and the intercept i s 2 not s i g n i f i c a n t . The value of R i s 0.92 and thus R=0.96. 5 Analysis of Dropout Effect It has been shown i n Section 1 of t h i s chapter that there were a certain proportion of nonresponses at every stage of the survey. The group consisted of three subgroups of people - (1) people who were i n the sample but did not respond at the f i r s t stage; (2) people who par-t i c i p a t e d at the f i r s t stage but dropped out at the second stage, and (.3) people who participated at the f i r s t two stages but dropped out at the t h i r d stage. We w i l l use the term 'dropout' to mean a nonrespon-dent of subgroups two and three. In the previous sections, i n studying various d i s t r i b u t i o n s and i n other related analysis, we have used data of those 111 participants who participated i n a l l the three stages of the survey. The d i s t r i -butions at the various stages are important and t h i s i s p a r t i c u l a r l y true for the f i n a l stage d i s t r i b u t i o n , since inference about group judgment and pol i c y making would be based on i t . It i s , therefore, of interest to see whether or not the dropouts may have a serious effect on the f i n a l stage d i s t r i b u t i o n . We w i l l be interested to see whether there are good reasons, i n view of the available information, i n support of the proposition: The f i n a l stage d i s t r i b u t i o n would have been dif f e r e n t had there been no dropouts. 5.1 Comparison of F i r s t Stage Distributions: Dropout vs. • Nondropout As regards the dropouts, two kinds of information are available to us - t h e i r f i r s t stage responses and information on t h e i r background cue variables. We w i l l attempt to i n f e r , from these two sources, about the v a l i d i t y of the proposition set up i n the preceding paragraph. Suppose that the f i r s t stage d i s t r i b u t i o n of the dropouts d i f f e r s i g n i f i c a n t l y from that of the nondropouts; then we can reasonably conclude that i f the dropouts had participated throughout, they would have contributed a d i s t i n c t i v e pattern to the f i n a l stage d i s t r i b u t i o n . This argument's leads us to compare the f i r s t stage d i s t r i b u t i o n of the dropouts with that of the nondropouts from the viewpoint of goodness of f i t . Table X gives the two d i s t r i b u t i o n s i n r e l a t i v e frequencies. Table X. F i r s t Stage Distributions of Dropouts and Nondropouts Response % of Non-%dropouts . % of Drop-outs 00 5.41 2.78 12255 8.11 2.78 2550.0 9.9.101 19.44 37.5 4.50 5.56 50.0 11.71 11.11 62.5 13.51 19.44 75.0 31.53 27.78 87.5 14.41 311111 100.0 1.80 0 The freehand drawn frequency curves for the di s t r i b u t i o n s are shown in F i g. 7(b). F i g . 7(a) gives the histogram of the d i s t r i b u t i o n of -• . " •" •- ^h"' dropouts. -Q- 25 50 75 100 Fig. 7(a) F i r s t Stage D i s t r i -bution of Response of Dropouts Nondropouts Dropouts Fig. 7(b) F i r s t Stage Distributions of Response of the Dropouts and the Nondropouts response of the dropouts. As can be seen i n Fig. 7(b), both-' the d i s t r i -butions have the same modal values (with respect to both the modes) and also the troughs of the bimodal d i s t r i b u t i o n s are the same (at 37.5) The only d i s s i m i l a r i t y of the curves occures on the l e f t s i d e . ( p a r t i -c u l a r l y for the extreme three categories). But the ri g h t side has a very good s i m i l a r i t y . Considering the ov e r a l l general pattern of the dist r i b u t i o n s we may assert that the shape of the di s t r i b u t i o n s conform to each other f a i r l y w e l l . A chi-square with 3 degrees of freedom was-(24) also computed,. for testing goodness of f i t , the value of the s t a t i s -t i c being 1.18. This value i s far less than the theoretical .05 l e v e l value of 7.81. We may, therefore, conclude that so far the f i r s t stage responses are concerned, the dropouts do not come from a group very much different from the nondropouts. 5.2 Lo g i s t i c Regression Analysis It i s also of interest to see whether or not the dropouts came from some par t i c u l a r classes with respect to th e i r background cue var-iables . I f the dropouts came from a section of the population with some pa r t i c u l a r background, i t could be expected that they might have a t y p i c a l response behavior; t h i s , i n turn, due to t h e i r nonpartici-pance, would carry an effect on the f i n a l stage d i s t r i b u t i o n . For example, suppose as an extreme case, that most of the dropouts were Faculty members, then since, on the average, they gave lower ratings •('.as. was found e a r l i e r ) than the s t a f f , we could expect that the f i n a l (.24) In computing the chi-square s t a t i s t i c , some response categories were merged to makerthe expected c e l l frequencies at least 5. The following are the revised 4 categories: (0, 12.5, 25.0), (37.5, 50), (62.5) and (75.0, 87.5, 100.0). stage d i s t r i b u t i o n would have larger proportion of people on the low rating response categories and the d i s t r i b u t i o n would look different from what has been obtained. Our arguments i n the preceding paragraph lead us to enquire about the question: Is there any relationship between a person's being a dropout or a nondropout, and his background cue variables? To study t h i s kind of r e l a t i o n s h i p , we use the l o g i s t i c regression model pre-sented i n Chapter 2. Let Y be a random variable which takes the value 1, i f the sub-ject i s a nondropout, and 0 i f the subject i s a dropout. Let p be the probability of being a nondropout, i . e . , p = P(.Y = 1); then the l o g i s t i c regression model i s given by -a -b'x 1 + e where a i s a scaler constant, b i s a vector of constants, x i s a vect-or of cue variables, and the prime denotes transpose. (.25) In our case, observations on 10 cue variables were available; they are the variables X^, X 5, X y, Xg, X g, X 1 Q, and as defined i n Sec. 4.1 of t h i s chapter, and two other variables are X 1 7 = res-( 26 ) ponse at the f i r s t stage, 1 and X l g, which takes the value 1 or 0 (25) Note that we do not have information on the cue variables (for t the dropouts) which were introduced at the second and t h i r d stage. (26) X 1 7 i s the same as Z-j_ of Section 4. Table XI. Results of Log i s t i c Regression for Non-response with 10 Cue Variables Variables Coefficient Asymptotic Std. Error Asymptotic Asymptotic t-Ratio Significance Constant 1 .691 0 577 2. 930 0 .003 X l -0 .350 0 282 1. 244 0 .213 x 4 -0 .058 0 270 0. 215 • 0 .830 X5 -0 .133 0 288 0. 462 0 .644 x 7 -0 .391 0 292 1. 340 0 .180 X8 -0 .166 0 235 0. 704 0 .481 X9 -0 .493 0 437 1. 129 0 .259 X10 -0 .107 0 351 0. 304 0 .761 X16 0 .015 0 .012 1. 268 0 .205 X17 -0 .006 0 .005 .'Il 190 0 .234 X -0 .149 0 .296 0. 505 0 .614 18 according to whether or not a subject took more than half an hour to commute to the university campus. Data on a l l of these variables were available for 124 subjects. But excluding X^g, data on the remaining 9 cue variables were available for 145 subjects; there were 21 observations missing for X . Thus, i n computing the parameters 16 of the regression equation we used two approaches: (1) using 124 data points with 10 cue variables, and (2) using 145 data points with 9 cue variables. The Maximum Likelihood estimates of the parameters, asymptotic t - r a t i o s and the asymptotic significance levels for the two approaches are shown i n Tables XI and XII. 75 Table XII. Results of Logis t i c Regression for Nonresponse with 9 cue Variables Variables -Coefficient Asymptotic Std. Error Asymptotic t - r a t i o Asymptotic Significance Constant 1.244 0.485 2.567 0.010 X l -0.326 0.253 1.291 0J.97 X 4 0.074 0.247 0.299 0.765 X 5 0.111 0.247 0.448 0.654 X 7 -0.279 0.244 1.144 0253 X 8 -0.075 0.214 0.350 0.726 X 9 -0.500 0.371 1.346 0.178 X 10 . -0.274 0.331 0.829 0.407 X 117 0.000 0.005 0.023 0.982 X 18 -0.211 0.260 0.812 0.417 X It can be seen from the asymptotic significance levels i n both the tables that none of the coefficients are s i g n i f i c a n t . This implies that there i s no relationship between the probability (1-p) of being a dropout and the cue variables; that i s , knowing a subject's background variables, one cannot predict the probability of his being a dropout. This, on the other hand, implies that the dropouts were spread i n an unpredictable way over the entire sample. However, we note that the constant term i s s i g n i -f i c ant (with significance levels .003 and .01 i n Tables XI and XII respec-t i v e l y ) . This implies an o v e r a l l probability of response (and hence non-response). This pr o b a b i l i t y can be computed by setting a l l the b c o e f f i c i -ents equal to zero and using an estimated value of a i n the regression equation. Thus the probability of response i s .84 (using a=1.691) and i s .78 (using a=1.244). 76 From our above analysis we conclude that the dropouts do not form a d i s t i n c t i v e group of people with respect to response behavior at the f i r s t stage and also with respect to t h e i r background. This leads us to say that there i s no reason to believe that the response behavior of the drppouts would have been different from that of the nondropouts had the dropouts participated i n the t h i r d stage. 6 Study of Reasons-giving Behavior As has been stated before, the important aspect of qualitative feed-back i s that at each stage the subjects not only answer the basic question of judgmentmaking, but also, they give reasons supporting t h e i r answers. At the f i r s t stage, they write t h e i r own reasons i n d i v i d u a l l y and indepen-dently, and i n subsequent stages they usually check reasons from a compo-s i t e l i s t ; sometimes they also give new reasons not contained i n the l i s t . The reasons generated by the participants should prove very useful to a decision maker to get insight about opinion, and i n understanding the attitudes of the participants. I t -is worthwhile to examine and understand some basic features of the reason-giving behavior of the subjects. We have already discussed some aspects of this behavior i n Chapter 2, where we have also discussed how the composite l i s t was prepared. In this section we fo-cus attention on several other things, such as, the d i s t r i b u t i o n of the number of reasons given by the subjects, the t r a n s i t i o n p r o b a b i l i t i e s of reasons, importance of reasons, andtJthe subjetc's reason-giving behavior with respect to different response categories. In addition, we investigate whether there i s any difference i n behavior between groups with randomised and nonrandomised ordering of pro and con reasons (see Chapter 3). 6.1 Dis t r i b u t i o n of the Number of Reasons Table XIII gives the di s t r i b u t i o n s of the number of reasons, given by the subjects at the f i r s t , second and t h i r d stages. The means and standard deviations of the distrib u t i o n s are shown i n Table .'XIV. Table XIII. Distributions ( i n % of subjects) of the Number of Reasons for the F i r s t , Second and the Third Stage N 'Number of F i r s t Stage Second Stage Third Stage Reasons , 0 8.11 5.41 2.70 1 34.23 2.70 3.60 2 39.64 12.61 5.41 3 14.41 14.41 6.31 4 2.70 14.41 8.11 5 0.90 19.82 18.02 6 0 9.91 16.22 7 0 7.21 8.11 . 8 0 7.21 10.81 9 0 3.60 3.60 10 0 0.90 9.91 11 0 0.90 3.60 12 0 0.90 1.80 13 0 0 0.90 14 0 0 0 15 0 0 0 16 . 0 0 0.90 Table XIV. Means and Standard Devi-ations of the Distributions of the Number of Reasons F i r s t Sta ge Second Stage Third Stage Mean 1.72 4.58 6.15 , S.D. 0.96. 2.46. .. 3.02 '-• • 78 40 r 35 30 CO S 251 • n CH 2 ° ! O +-> g 151 CD PH 10 h 5k 0U. —I I i i i \ . I - I -2 4 6 8 10 12 Fig. 8(a) Distr i b u t i o n of the Number of Reasons f o r s t h e t F i r s t Stage 14 16-30, CO 4-> O CD • n CO 4H O • P C CD O & CD PH 25 20 15 10 I 2 4 - 6 :1 -IS.; 10 12-Fig. 8(b) Distr i b u t i o n of the Number of Reasons for the Second Stage 14 16 79 25 r 20 -15 -10 - |—I 5 - I 0 L i 1 1 1 I I I I I I I 1 I I 1 . i 1 0 2 4 6. 8 10. 12 14 16. Fig. 8(c) Dis t r i b u t i o n of the Number 0 f Reasons for the Third Stage On the average, participants gave increasing numbera of reasons, on successive stages, namely, 2, 5 and 6 reasons, at the f i r s t , second and t h i r d stages, respectively. This tendency of giving an increasing num-ber of reasons, on the average, may have some psychological implica-tions. At the f i r s t stage, each subject had to write his reasons independently and thus he had to undergo a process of thinking and deliberation, which i s not a very simple task for some people. Also, each subject had lim i t e d information. These factors might have r e s u l t -ed i n giving only 2 reasons, on the average, at the f i r s t stage. On the other hand, i n the second and t h i r d stages, subjects had only to check reasons they agreed with from a composite l i s t . In doing t h i s , we may expect, whenever a subject came accross a reason he agreed with and also which he could use to support his present judgment, he checked that reason. Since he was supplied with a broad spectrum of reasons he could pick up a larger number of reasons than he gave at the previous stage. The rate of increase i n the average number of reasons i s also noticeable. They are: 150% from the f i r s t to the second stage and only 20 % from the second to the t h i r d stage. (taking the averages to be 2, 5 and 6). This decreasing rate of increase i s also compatible with the fact that at the l a t e r stages few new reasons were generated, and also, as w i l l be seen l a t e r , that the new reasons carried less importance.' The standard deviations of the di s t r i b u t i o n s also increased from stage to stage. I t can be seen from the Figs. 8(a), 8(b) and 8(c) that the d i s t r i b u t i o n s tended to be skewed to the r i g h t . 6.2 Transition P r o b a b i l i t i e s of Reasons From the second stage onward, a subject, while checking reasons from the composite l i s t , sometimes added new reasons to his own pre-vious stage slate of reasons, or dropped reasons from the s l a t e . This phenomenon of adopting new reasons and abandoning old ones i s somewhat similar to what happens i n an ide a l face-to-face interaction where, when faced with new arguments and reasons, participants often abandon th e i r i n i t i a l arguments and accept new ones or, perhaps, while s t i l l holding the o r i g i n a l arguments, they pick some new reasons to streng-then t h e i r position. We w i l l study t h i s phenomenon within our new setting. Our analysis w i l l be based mainly on the set of estimated t r a n s i t i o n p r o b a b i l i t i e s of reasons. These p r o b a b i l i t i e s are - defined i n Chapter 2. In estimating the p r o b a b i l i t i e s , we have used data only of those 111 subjects who have completed a l l the three stages. Table XV. Transition P r o b a b i l i t i e s of Reasons, iT„(a) } corresponding to the F i r s t and Second Stages c ( i , j = 0, 1) J_ 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 .83 .20 .64 .27 .83 .30 .72 .17 .83 .13 .79 0 .90 .17 .80 .36 .73 .17 .70 .28 .83 .17 .87 .21 1 .10 0 0 .81 .19 ,33 .67 .82 .18 a 10 l i " " 11 12 13 14 15 16 17 •j' 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 .67 .14 .72 .25 .80 .08 .80 .33 .74 0 .92 .38 .76 .17 .88 0 1 .33 .86 .27 .75 .20 .92 .20 .67 .26 1 .08 .62 .24 .83 .12 1 ** a denotes the s e r i a l number of a reason i n the second stage questionnaire (Appendix A2). " P r o b a b i l i t i e s i n these c e l l s could not be calculated since both the frequencies i n these c e l l s were 0s r e s u l t i n g i n a marginal t o t a l of 0. This, rather, unfortunate situation occured because only one person who gave reason 7 at the f i r s t stage, dropped out at the second stage r e s u l t i n g i n a 0 frequency i n each of the two right c e l l s . Note again that i n estimating the p r o b a b i l i t i e s , we have used data only of those 111 subjects who have completed a l l the three stages. Table XVI. Transition P r o b a b i l i t i e s of Reasons',' T r ^ C a ) , corresponding to the Second.and Third Stages ( i , j = 0,1) a 1 2 3 4 5 6 7 J I 9 9 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 .88 • 11 .85 .03 .94 .13 .87 .08 .94 .17 .94 .12 .99 .18 .87 .09 .93 .09 1 .12 .89 .15 .97 .06 .87 .13 .92 .06 .83 .06 .88 .01 .82 .13 .91 .07 .91 a 10 11 11 13 12 ib 13 14 15 16 17 18 j 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 .92 .06 .88 .17 .93 .19 .93 .28 .94 .18 .93 .23 .86 .09 .94 .21 .89 1 1 .08 .94 .12 .83 .07 .81 .07 .72 .06 .82 .07 .77 .14 .91 .06 .79 .11 0 a 19 20 21 22 23 24 25 26 j 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 .83 0 .93 0 .91 0 .89 0 .86 1 .79 0 .92 0 .85 0 1 .17 1 .07 1 .09 1 .11 1 .14 0 .21 1 .08 1 .15 1 * The reasons i n t h i s table are numbered i n such a way that Reasons Number 1 ( i . e . ac* \ ) through 17 ( i . e . a - 1 7 ) correspond to those i n Table XV. To identify reasons i n the randomised and nonrando-mised composite l i s t s of the t h i r d stage, Appendix F should be consulted. OD ro The estimates of the p r o b a b i l i t i e s , 7 K_.(a), (i,j=0,l) are shown i n Tables XV and XVI. Each of these tables i s again a set of 2x2 tables. For example, i n Table XV, for reason number 3 (a=3), the probability i s .7 that a subject who gave t h i s reason at the f i r s t stage would again give i t at the second stage, while the probability i s only .17 that a subject who did not give the reason at the f i r s t stage would give i t at the second stage. In both the tables, the diagonal p r o b a b i l i t i e s (diagonal entries of the 2x2 tables) are large compared to those of the o f f diagonals; i t i s more l i k e l y that a subject w i l l repeat his l a s t stage's reasons at the present stage.WWe also f i n d i n the l e f t off-diagonal c e l l s of Table XV , a probability ranging from .08 to .33, indicating that there was a substantial proportion of subjects who gave a reason at the second stage while they did not give i t at the f i r s t stage. The corresponding p r o b a b i l i t i e s i n Table XVI, however, have another implication for Reas-ons 1 through 17. We note that these are the same reasons which were in the second stage composite l i s t , whileRReasons 18 through 26 were new additions i n the t h i r d stage l i s t . So each participant had the opportunity to see the Reasons 1 through 17 at the second stage, and i f desired, they could give i t then; thus, i d e a l l y the figures i n the above mentioned c e l l s should have been zero. However, the nonzero pro-b a b i l i t i e s i n these c e l l s indicate that there was a certain proportion of people who did not give these reasons at the second stage, but gave them at the t h i r d stage. I t i s hard to explain t h i s behavior, but i t might have occured due to the time e f f e c t , or simply that the said subjects did not pay much attention to those reasons at the second stage. 6.3 Test of a Hypothesis Now we focus, our attention on an issue which i s of methodologi-ca l interest. We r e c a l l from Chapter 3 that at the second stage and thirds stages of the survey we reminded each participant of his pre-vious stage response , and reasons, so he could make a conscious judg-ment. In p a r t i c u l a r , at the second stage, we gave each subject his own handwritten reasons he gave at the f i r s t stage. We asked him to check reasons from the composite l i s t . This l i s t was prepared from the reasons the participants gave at the f i r s t stage, and contained the reasons of each participant, maybe, i n a s l i g h t l y d i f f e r e n t form and composition. The point of interest here i s : whether or not a participant could recognize his reasons i n the composite l i s t . This i s important, because the efficiency of the l i s t depends partly on how e f f e c t i v e l y i t can represent the reasons of the participants. I f the l i s t contains reasons much more distorted from what a certain participant gave, then he would not agree with them, and would not check o f f those reasons i n the compo-s i t e . However, i f the l i s t were e f f e c t i v e , he would be able to recog-nize his own reasons. In that event two cases may occur - (1) the part-icipant checks the reason i f he s t i l l holds i t , or (2) he does not check i t , for he wants to abandon i t . Now, only the investigator knows which reasons i n the composite l i s t were supposed to represent a given participant's reasons. Thus, i f the p a r t i c u l a r participant checks those part i c u l a r reasons, we know that he could recognize his reasons i n the composite l i s t . However, i f he does not check them, we do not know whether or not he could recognize them. We now proceed to formalise the situ a t i o n into a hypothesis testing problem. We hypothesize: Subjects could recognize t h e i r reasons. Then, according to our discussion i n the previous paragraph, and also due to the fact that about 70% of the participants did not change t h e i r reasons (See Section 2), i n the second stage, we would expect that subjects should repeat t h e i r reasons and should be able to check reasons correct-l y . We note that t h i s expected proportion i s T r ^ ( a ) , as defined i n Chapter 2, for reason ct( ct= 1,2,..., 17). The problem of testing the above hypothesis, may now be transformed into a s t a t i s t i c a l hypothesis testing problem as follows: We test a hypothesis Hg(a): TT (a) = TT^ ( .O I) , independently for each, a, where Tr-^(.a) i s a given hypothetical value of i ^ ( c i ) , I f HQ(.a) is. rejected, we conclude that reason a could not be recognized by participants. Let n(.ct) be the number of subjects who gave reason a at the f i r s t stage. Each, of these n(.a) people were asked to check reasons indepen-dently at the second stage. We also assume that the probability of checking reason a i s the same for everyone of them; t h i s p r o b a b i l i t y , i n f a c t , i s TT_^  Cot).. The n(a) subjects then form a set of independent t r i a l s . ; a success occurs i f a subject checks. Reason a. Thus for t e s t -ing -HQCO) we can use the Binomial d i s t r i b u t i o n . We already have estimates of n^(.a) i n Table XV. Next, we need '1:0^assign's- a numerical value of ir^Ca) for each ct(a=l ,2 ,. . . ,17). Since there i s no reason to believe that TT -Q(OI) would be different for d i f f -erent a, we w i l l take the same value of Tr°^(a) for each. a. On a sub-jec t i v e basis, we take the value to be .9, i . e . , i n the population 90% or more of the subjects recognized the reason and checked i t . The Table XVII. Results of Testing H (a) a 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 31 8 7 29 7 4 * 2 4 18 6 11 4 7 5 10 1 n(a) 39 11 10 35 8 4 * 3 4 21 8 12 6 7 8 12 1 P- .037 .090 .070 .132 .5700 l * .270 1 .352 .187 .718 .114 1 .038 .341 1 value " Computations for Reasons 7 could not be carried out due to reasons explained i n page 81. Note: a stands for the s e r i a l number of reasons i n the second stage questionnaire. n^Ca) = number of subjects who gave reason a at both stages. n(a) = number of subjects who gave.the reason at the f i r s t stage. P-value = PCn-nCct) <• n°.,(a)), where n°n(.a) i s the observed value of n-^Ca). as defined above. 87 remaining 10% or l e s s , we assume, consists of subjects who recognized the reason, but abandoned i t . We have the hypothesis pairs: H Q(a): TT^Ctf) = .9 H ( a ) : TT (et) < .9 A 11 for = 1,2,...,17. For each <* , H^(») was tested separately using the Binomial d i s t r i -bution. The results of the tests are given i n Table XVII. As can be seen from the P-values, we may. reject only two hypotheses (correspond-ing to Reasons 1 and 15) out of 16 at a .05 significance l e v e l . We may conclude that the subjects could recognize t h e i r reasons i n the compo-s i t e l i s t . 6.4 Study of Reasons-giving Behavior with respect to Category of Response Tables XVIII, XIX and XX were compiled i n order to examine how participants i n different response categories d i f f e r In giving reasons.. In each table,.the reasons have been grouped Into pro and con groupings. In numbering the reasons the same number has been used f o r a reason i n a l l the tables. To i d e n t i f y the reasons i n the randomised and nonran-domised l i s t s of the t h i r d stage, Appendix F should be consulted, where the f i r s t row gives the numbers of reasons as appeared i n Tables XVIII, XIX and XX. We p a r t i t i o n each table into s i x regions: Region 1: A l l c e l l s i n categories 0 through 37.5 i n pro reasons. Region 2: A l l c e l l s i n categories 0 through. 37.5 i n con reasons. Region 3:A11 c e l l s i n categories 62.5 through 100 i n pro reasons. Region 4: A l l c e l l s i n categories 62.5 through 100 i n con reasons. Region 5: A l l c e l l s i n category 50 i n pro reasons. Region 6: A l l c e l l s i n category 50 i n con reasons. In a l l the tables, most c e l l s i n regions 2 and 3 are occupied and also the c e l l frequencies are large. This means that participants on the iimportant' side gave mainly pro reasons and those on the 'unimpor-tant' side gave mainly con reasons. This i s what could be expected. However, i t i s interesting to notice that nonzero'frequencies also occured i n Regions 1 and 4; and consistently, i n a l l the tables, f r e -quencies have a larger value i n Region 4- than those i n Region 1. Region 4 represents subjects who gave con reasons, probably along with pro reasons,, while giving an answer on the 'important' side; the reverse Is. true for subjects i n Region 1. The con reasons seem to be more appealing i n the sense that even when a suject gives a response on the important side of the scale, he cannot ignore the arguments presented i n the con reasons. Also, i f we look at Regions 5 and 6, we see that most of the neutral people gave con reasons. Another point to notice i s that as the subjects deviated more and more from n e u t r a l i t y , they also tended to give fewer and fewer con reasons. Importance of Reasons. Table XXI shows the importance of reasons as determined by the proportion of subjects (out of a t o t a l of 111) who gave a certain reason at the t h i r d stage. The reasons have been ranked i n decreasing order of importance. Thus, the highest ranking reason was given by about 49% of the subjects (Reason 4). while the same per-Table XVIII. Response vs. Reasons C l a s s i f i c a t i o n " ( F i r s t Stage) Pro Reasons Con Reasons Res-ponse poria 3 1 2 1 3 2 4 3 5 4 6 5 6 7 8 7 9 8 10 9 10 11 J.l 12 12 13 14 15 16 17 ft 0 3 1 1 1 2 1 12.5 2 1 5 3 2 1 1 1 3 5,25.0 1 3 2 22 1 ' 2 3 1 1 37.5 33 1 2 1 50.0 1 2 2 33 3 2 1 1 3 62.5 4 2 1 6 1 4 1 1 2 1 1 75.0 26 4 4 17 4 3 1 1 3 1 1 1 87.5 9 3 4 10 2 1 2 100.0 1 1 1 " No reason given ** The reason numbers refer to the s e r i a l number of reasons in the second stage questionnaire, Appendix A2, page CO CD Table IX. Response vs. Reasons C l a s s i f i c a t i o n (Second Stage) Pro Reason's ' ' Con Reasons Res=i ponse 1 2 3 4 5 6 7 8 18 19 20 21 23 24 9 10 11 12 13 14 15 16 17 22 25 26 5£ X 0 3 4 4 3 1 2 1 3 1 12.5 2 1 1 5 9 4 3 3 6 1 5 2 1 1 1 25.0 1 2 1 4 12 9 12 8 8 5 9 7 37.5 1 3 1 3 8 8 9 5 10 3 6 2 50.0 2 1 2 1 4 5 2 3 2 2 2 2 .56255 9 12 6 10 3 4 1 4 1 2 7 5 3 4 5 6 2 75.0 16 16 6 18 11 11 3 7 1 1 I 2 4 3 1 1 1 1 3 3 87.5 14 10 11 13 7 10 6 8 1 1 1 1 ' 100.0 1 2 1 1 1 1 2 * For identifying reasons i n the randomised or nonrandomised l i s t s ( i n Appendices A3 and A4) 5consult . Appendix F. ** No reason given. Table XX. Response vs. Reasons C l a s s i f i c a t i o n (Third Stage) Con Reasons Res-ponse 1 2 3 4 5 6 7 8 18 19 20 21 23 24 9 10 11 12 13 14 15 16 17 22 25 26 O 4 5 4 4 1 3 2 5 2 2 2 1 12.5 1 1 2 6 12 8 7 5 10 2 5 3 4 4 5 25.0 1 1 1 1 1 1 3 8 16 11 12 9 7 6 12 6 5 2 4 37.5 1 1 1 1 1 2 3 2 5 3 2 4 5 4 2 1 4 50.0 3 2 1 2 1 4 3 1 2 4 1 3 1 62.5 8 11 5 11 2 3 1 4 2 3 1 2 3 4 8 5 3 5 4 6 3 1 75.0 18 22 9 22 12 13 3 12 4 7 5 3 8 8 1 3 3 3 1 87.5 14 13 10 14 7 10 5 13 5 6 2 6 4 6 1 1 1 1 1 1 1 1 2 100.0 2 2 1 2 2 1 1 1 1 2 1 2 1 Total 46 53 26 54 25 28 10 31 12 19 9 11 16 24 27 51 40 31 24 33 17 42 16 13 10 1'. * For identifying reasons i n the randomised and nonrandomised l i s t s d n Appendices A3 and A4) consult Appendix F. L O H centage for the lowest ranking reason (Reason 10) i s only 9%. A l l the reasons have been arranged according to t h e i r ranks i n Appendix H. Let us consider the f i r s t f i v e high ranking reasons. It can be seen that the reasons with ranks 1, 2 and 4 are pro reasons, while those with ranks 3 and 5 are con reasons. The reasons with ranks 1 and 4 r e f l e c t on the value the community places on swimming as a source of recreation and maintaining health and f i t n e s s . Since i t rains very frequently during winter i n Vancouver, the community feels the nece-s s i t y of an indoor aquatic center. Now looking at the con reasons (ranks 3 and 5) we f i n d contrasting arguments. Since, at the time of i n i t i a t i n g construction of the center, the university was experiencing a f i n a n c i a l shortage, the community also f e l t that the money could be spent for demanding academic purposes; various alternatives for spending the money were suggested. The reasons, thus, shows the r a t i o n a l for giving opposite judgments by the two subgroups (as haswbeen revealed by the judgment d i s t r i b u t i o n s ) . 7 Testing for the Effect of Ordering of Reasons i n the Composite L i s t At the t h i r d stage of the survey, two types of composite l i s t s of reasons were fed back - one with a l l the pro reasons appearing at the f i r s t part of the l i s t followed by a l l the con reasons (this i s the nonrandomised l i s t ) , and the other randomised ordering of the reasons (this i s the randomised l i s t ) . Participants were divided into two equal subgroups by random selection. One of the subgroups were fed the non-randomised l i s t and the other the randomised one. Our objective was to 93 Table XXI. Importance Ranks of Reasons Ranks % of Ranks % of subjects subjects 1 48.64 11 23.42 2 47.74 12 22.52 3 45.95 13" 21.62 4 41.44 14 17.12 5 37.84 15 15 .32 6 36.04 16" 14.41 7 29.13 17 11.71 8" 27.93 18 10.81 9- 25.23 19 '99.91 10 24.32 20 " 9.01-21 8.11 * -Tnis rank coincides for two reasons see whether or not there was any effect of ordering of pro and con reasons on response, and giving reasons. In doing t h i s , f i r s t we w i l l compare the response d i s t r i b u t i o n s of the two subgroups by using a chi-square test to see whether the di s t r i b u t i o n s d i f f e r ; then we compare, for each reason, the proportion of people i n the two subgroups who gave the reason. 7.1 Comparision between Distributions The response distributions, of the two subgroups are shown i n Table XXII. Because of small expected c e l l frequencies, some of the 94 Table XXII. Distributions of Subgroups with Randomised and Nonrandomised L i s t ' Response 1 Randomised L i s t 1 Nonrandomised L i s t No. of-Subjs. % of Subj s. No. of Subj s. % of Subj s. 0 3 5.36 4 7.27 12.5 6 10.71 6 10.91 25.0 10 17.86 6 10.91 37.5 5 8.93 2 3.64 50.0 3 5.3.6 4 7.27 62.5 7 12.50 9 16.36 75.0 11 19.64 17 30.91 87.5 9 16.07 7 12.73 100.0 2 3.57 0 0 Total 56 55 response categories were.merged together. The 5 revised categories are (0,12.5), (25, 37.5), (50, 62.5), (75), and (87.5, 100). The computed value of the chi-square s t a t i s t i c i s 4.75 with 4 d.f.; t h i s i s far below the 5% significance value of 9.49. We may conclude that, so far as the response distributions'are concerned, we do not have evidence of d i f f e r i n g response pattern i n the two subgroups, or i n other words, randomisation of ordering of the reasons does not have an effect on the response of a subject. 7.2 Comparison i n terms of Reason-giving Next we examine whether the two groups d i f f e r i n giving reasons, at the t h i r d stage. We w i l l use data given i n Appendix G. I t gives, for 95 each reason, the number of subjects as well as proportion of subjects within a given subgroup. For example, 25% of the subjects with rando-mised l i s t gave Reason 3 while the percentage i s 21.82 i n the non-randomised l i s t group. A contingency type analysis cannot be done with t h i s table because a subject may give more than one reason and cannot be c l a s s i f i e d into one p a r t i c u l a r c e l l . The approach we w i l l follow i s to compare, for each reason, the two proportions i n the two subgroups. To formulate the hypothesis, our argument i s as follows: I f there i s no effect of randomisation then the two proportions for a given reason should be equal, i . e . , the pr o b a b i l i t y of giving a reason, 3whether the l i s t i s randomised or nonrandomised, i s constant. In other words, the ordering of reasons does not have any effect on the pr o b a b i l i t y of giving a reason. This i s the basis for our n u l l hypothesis. On the alternative side, supposing there i s an effect of ordering, our contention i s the follow-ing: Let a reason appear i n the r t h position i n the nonrandomised l i s t and i n the (r+s)th position i n the randomised l i s t (s i s nonnegative • and r l i e s between 1 and 26), then the probability of giving the reason by a subject i n the nonrandomised-list-group should be equal to .or greater than that of the randomised-list-group according to whether or not not s=0. That i s i f a reason appears f i r s t i n s e r i a l order i n one l i s t compared to the other l i s t , then the chance of giving the reason by a subject i n the f i r s t subgroup i s higher; however, i f the reason appears with the same s e r i a l number, the chance i s equal. Let us-define the following for reason a (a =1,2 ,... ,26). 96 i = s e r i a l number of the reason i n the randomised l i s t , j = s e r i a l number of the reason i n the nonrandomised l i s t , p ^ = proportion of subjects i n the randomised-list-group who gave the reason, and P2_. = proportion of subjects i n the nonrandomised-list-group who gave the reason. We set up the n u l l and alternative hypotheses as follows. H QCa): P l l = p 2 j ( i , j = 1,2,...,26) H l C a ) : P l i > P 2 j l f i < i H 1(a): P l . < p 2. i f i > j Va): Pu * p2j l f For t e s t i n g K Ca), f o r each a s e p a r a t e l y , we use, i n view of the l a r g e sample s i z e s , the normal approximation and the U s t a t i s t i c - ; d e f ined below. A A P-, • " Po-l l 2 : UCo) = p ( a ) [ l - p( M / 1 + 1 a)3 / - + -\ 56 5J where p ^ and p 2^ are the sample estimates of p-^ and p2_. respectively, and p(a) i s the o v e r a l l proportion calculated from the entire group Csee Appendix G). The numbers 56 and 55 are the sample sizes i n the randomised and nonrandomised l i s t subgroups respectively. Under the ("27) n u l l hypothesis U(a) i s distributed approximately as N(0,1) (.27). Note that we are ignoring the fact that because of feedback, from TSblSeXXIII. Computed Values of u(a) a .1 2 3 4 5 6 7 8 9 I 25 10 4 26 20 18 21 17 8 j 1 2 3 4 5 6 7 8 17 u(a) -1.62 -1.04 0.39 -1.23 0.18 -0.93 -0.03 1.42 0.17 a 10 11 12 13 14' 15 16 17 18 i 23 22 9 16 24 6 3 13 1 j 25 24 18 22 26 16 15 20 13 u(a) 0.10 -0.87 0.15 -0.05 -0.67 0.75 0.32 0.22 -0.65 a 1 19 20 21 22 23 24 25 26 i 2 5 7 11 12 14 15 19 j 10 14 9 19 12 11 21 23 u(a) 0.21 -0.38 -0.35 0.85 0.50 3.37 0.63 0.22. Note: i denotes s e r i a l number of the reason a i n the randomised l i s t , and j denotes s e r i a l number of the reason a i n the nonrandomised l i s t . 98 In view of the nature of our alternative hypothesis we reject the n u l l hypothesis according to the following r u l e : Case 1 Case 2 Case 3 i < j ; reject H^(a) i f u > u^ _g. i > j ; reject H Q(a) i f u < u, i = j ; r eject H (a) i f | u | >au^ _ 6/2 where u^ (.0 < k < 1) i s the (k x 100) percentile point of the standard normal d i s t r i b u t i o n . In p a r t i c u l a r , for 6 = .05, u = 1.64- and u ^ = --1.64 and u = 1.96. The computed values u of U are given i n Table XXIII. We f i n d that there are 16 reasons for which i< j , 9 reasons for which i > j and one reason for which i = j . I t can be seen from the values of u that none of the hypotheses can be rejected at the 5% s i g -nifecance l e v e l . We may conclude that there i s no effect of ordering of reasons i n the composite l i s t on a subject-Is giving reason. 8 Distributions of Confidence Rating To see whether information feedback increases confidence i n one's judgment, each participant i n the t h i r d stage as well as i n the control group was asked to rate himself on the basis of his confidence i n the judgment he gave by using a scale of confidence ratin g fee'e? Ap.p'dx"^  A3 ,p\ 145) The di s t r i b u t i o n s of confidence ratings with cumulative percentages are shown i n Table XXIV. For example, 95.28 percent of the subjects i n the experimental group gave ratings 5 or more; the same percentage i n the control group i s 90.28. In Fig. 9, the cumulative percentages are shown two e a r l i e r stages responses within each, group are correlated (as. well as between the two groups). We assume that this, effect i s small enough to be ignored. 99 0. L . . • • ^ ^ i 9 8. 7 6 5 H- 3 2 GFig. 9 Cumulative Frequency Diagrams for the Third Stage and Control Group Distributions of Confidence Ratings by broken l i n e diagrams. Let (x,y) (using usual notation to denote a point) be a point on one of the curves; then y percent (approximately) of the subjects gave ratings x- or more. A chi-square was also computed for testing the goodness of f i t between the dis t r i b u t i o n s (after merging the f i r s t four categories into one). The value of the chi-square with with 5 d.f. i s 3.5 and has a probability of .62 of being exceeded. We may conclude that the ratings for the two groups have come from the same population. 100 Table XXIV. Distributions of Confidence Ratings Ratings Third Stage Control No. of Subj s. % of Cumulative Subj s. % No. of Subj s. % of Subjs Cumulative % (Not at a l l 1 0 0 100 .00 1 1.15 100 .00 Confident) 2 0 0 100 .00 1 1.15 98 .86 (Not quite 3 3 2. 83 100 .00 3 3.45 97 .71 Confident) 4 2 1. 89 97 .17 3 3.45 94 .26 (Quite 5 22 20. 75 95 .28 18 20.69 90 .28 Confident) 6 12 11. 32 74 .53 4 4.60 70 .12 'nt (Very- 7 35 33. 02 63 .21 25 28.74 65 .52 Confident) 8 14 13. 21 30 .19 14 16.09 36 .78 (Absolute- 9 18 16. 98 16 .98 18 20.69 20 .69 l y Confi-dent) • Total . 106. . 87 The average ratings are also almost equal, being 6.75 for the control group and 6.77 for the t h i r d stage experimental group. (.28) 5 subjects i n the t h i r d stage and 2 subjects i n the control group did not give any ratings. 101 CHAPTER 5 CONCLUSION In t h i s f i n a l chapter we have three objectives i n mind: We w i l l review the main results and findings of the study, suggest some possible research directions and make certain recommendations so things can be done i n a better way i n future applications. 1 Summary of Findings 1.1 V e r i f i c a t i o n of Methodological Issues We raised four questions ( i n Chapter 2) r e l a t i n g to f e a s i b i l i t y and methodological aspects of the quali t a t i v e controlled feedback method for obtaining judgmental data. F i r s t , we w i l l review the findings of the study i n r e l a t i o n to these objectives and then discuss some other r e s u l t s . The f i r s t question asked was whether or not i t was possible to motivate a group of people to participate i n a multistage qua l i t a t i v e feedback survey. We give emphasis on the question of motivation because the success of a survey depends to a great extent on the moti-vation to participate on the part of the participants. Since q u a l i t a -t i v e feedback i s a new method, and also due to operational consideratr ions (e.g. writing or checking reasons) the survey demands more atten-t i o n and involvement from the participants than i n usual surveys, i t i s worthwhile to examine the issue of motivation to participate i n such a survey. 102 However, i n order to judge the extent of motivation we need to set up some c r i t e r i a . In t h i s , we note that motivation i s an important factor which i s highly related to the rate of response, although nonresponse may occure for other reasons also. S t i l l i t may be argued that unless there i s a high motivation, a participant may not be w i l l i n g to go through a l l the stages of the survey. We can, therefore, look at the response rates at different stages to get an idea about the extent of motivation. We fi n d (Table I) that 80% to more than 90% of the subjects p a r t i -cipated at the diff e r e n t stages and also that about 63% of the people who were i n the sample at the beginning of the survey, completed a l l the three stages. We also note that the survey was done mainly by mail questionnaires. Thus the rate of response may be considered rather high. This indicates that participants had a good motivation to p a r t i -cipate. We conclude that i t i s possible to motivate people to p a r t i -cipate i n a multistage feedback survey. In actual application s i t u a -t i o n s , where participants' judgments may be incorporated i n decision-making, i t can be expected that motivation would be high. Our next question was: How activ e l y do the participants p a r t i c i -pate? By t h i s we mean, whether they take part i n a l l the requirements of the survey; i n p a r t i c u l a r , 'Do they write or check reasons?','Do they answer the basic question i n the way the answer i s wanted?'. We consider these questions, because, for example, i f one participates i n a l l the stages of the survey without giving any reason to support one's judgment, we say that he does not participate a c t i v e l y ; and i f 103 most participants do t h i s , the survey i s not l i k e l y to f u l f i l l i t s purpose. (29) We found that almost a l l the subjects who participated i n at least one stage did not f i n d i t d i f f i c u l t to answer the basic question by using the rati n g scale provided. I t was also found that, except for a small f r a c t i o n (8%), a l l the participants gave at least one reason at the f i r s t stage; the average number of reasons being 2, 5 and 6 i n the. successive three stages. The cumulative number of reasons genera-ted at the second stage was 26. These facts c l e a r l y show that the res-pondents were not only active i n p a r t i c i p a t i o n , but a l s o , that they were enthusiastic i n giving reasons and supporting answers. Thus, from the viewpoint of both motivation and p a r t i c i p a t i o n , the qua l i t a t i v e feedback survey can be successfully administered. The other two ppoints are d i r e c t l y related to the successfulness of the method i n creating an environment of a "group process". We have mentioned i n Chapter 1 that the central element of group process i s the process of interaction produced by interchange of arguments and reasons; and, that as a r e s u l t of such inte r a c t i o n , a change i n judgment may occura. To see whether or not there was int e r a c t i o n , we must look into the reason-giving behavior of the participants, which we have analysed i n Section 6 of Chapter 4. It was found that participants gave more and more reasons as they went from one stage to another, r e s u l t i n g i n an C29.). There were only 2 subjects who said that the question was not quite meaningful to them. 104 increasing average number of reasons (2, 5 and 6). This tendency shows that as new reasons were introduced, a participant accepted some of them, so that a reason given by one participant convinced another p a r t i -cipant. On the other hand, we also found that a certain proportion of participants (a maximum of 33% i n the second stage, and 28% i n the t h i r d stage) dropped t h e i r o r i g i n a l reasons (we r e c a l l that each p a r t i -cipant was. reminded of his l a s t stage's reasons), probably, because they found some more convincing reasons. This phenomenon i s represen-t a t i v e of the fact that an interchange of arguments, and reasons, took. place among the participants; q u a l i t a t i v e feedback was able to create the required interaction within the group. Now that we know that there was an int e r a c t i o n , we examine, whether change i n judgment occured. It -was found i n Sec.3 of Chapter 4- that at the second and t h i r d stages, 30% and 21%, respectively, of the p a r t i -cipants changed t h e i r judgments, and that more than 41% changed either i n the second or i n the t h i r d stage. That the majority of subjects did not change reasons i s compatible with the fact that the majority of them did not change response (as was found e a r l i e r ) . 1.2 Substantive Findings Apart from the above methodological v e r i f i c a t i o n s , two other impor-tant findings have emerged from our study. One of them has•implications on the r a t i o n a l i t y of judgments ,..a nd the other i s related to the charac-ter of judgment dis t r i b u t i o n s and t h e i r implications on decisionmaking. 105 (a) Implications on the Rationality of Judgments It has been mentioned i n Chapter 1 (Subsection 2.3.3) that the ra-tionale! behind developing the method of qualitative.controlled feedback (q.c.f.) i s to give the members of a group as much opportunity as possi-ble , so that they can generate and share a common pool of information and form (and express) t h e i r most reasoned judgments, on a controversial issue, independently ( i . e . without being subject to pressures of s o c i a l conformity, authoritarianism, transference of values and similar other coercive acts). Behind q.c.f. i s /(.therefore, i m p l i c i t the hypothesis that q.c.f. would produce the most r a t i o n a l judgments on the issue. A major r e s u l t of our study l i e s with the evidence i n support of the hypothesis. This evidence wis demonstrated i n Subsection 2.3 of Chapter 4 as a re s u l t of testing three hypotheses comparing the judgment d i s -t r i b u t i o n s . We have found that the experimental group which participated at a l l the three stages, almost s t a b i l i z e d t h e i r opinions about the issue, i n a bimodal fashion, at the t h i r d stage, i n spite of the fact that the public sentiment at that time (as measured by the control group d i s t r i -bution), was r e l a t i v e l y i n favor of the construction of the center. This i s the evidence i n support of the hypothesis. Let us explain t h i s i n the following way. At the t h i r d stage of the study, the center had already been i n the process of construction for quite a while. Based on some s o c i a l psychological theories such as the cognitive dissonance theory (Festinger [5 ] ) , one can predict that people who o r i g i n a l l y were opposed to the idea of constructing the center, would have gradually changed t h e i r attitude to a favorable one since the construction was already under way,whereas people who o r i g i n a l l y were i n favor of the construction would remain so. Consequently, the o v e r a l l opinion about t h i s issue would be a favorable one. This prediction i s actu-a l l y v e r i f i e d by the r e s u l t obtained from the control group. The r e s u l t demonstrates that, at the time of the t h i r d stage, subjects i n the control group, who represented the public feelings at that time, did have a more favorable attitude toward the construction of the center than the experimental group at the time of the f i r s t stage(.Fig. 6Ca)), which represented the public feelings at that time ( f i r s t stage). The s i g n i f i c a n t difference found between the t h i r d stage experimental group and the control group, however, indicates that subjects i n the experi-mental group, at the time of the t h i r d stage, had not a l l changed t h e i r opinion toward the favorable d i r e c t i o n . AAsignifleant number of them had gone toward the opposite d i r e c t i o n instead; i . e . , they remained negative toward the issue. This finding implies that, by giving them both pro and con reasons, subjects i n the experimental group, were not overwhelmed by the current developement of the issue and therefore, were allowed to make a decision more r a t i o n a l l y than subjects i n the control group. (b) Implications for Decisionmaking Another important finding of our study i s related to the character of the judgment d i s t r i b u t i o n s . We have found that these d i s t r i b u t i o n s Csee Section 2 of Chapter 4-) are bimodal with troughs, at the neutral point of the scale. This r e s u l t suggests the hypothesis that when there 107 i s a controversial issue of 'yes' or 'no' type, the judgment d i s t r i -bution becomes bimodal, indicating the two opposing groups. One impor-tant application of these di s t r i b u t i o n s may l i e i n a decisionmaking context, where i t i s required to incorporate, i n some way or other, the judgment of the group. In that case the d i s t r i b u t i o n gives a clear idea about the extent to which the opposing groups d i f f e r i n th e i r judgments, and the strength of the individual g r o u p s T h e s e two ideas can be understood better by re f e r r i n g to the Figs.10(a) through 10 (.f). In a l l of the d i s t r i b u t i o n s , we denote the group on the l e f t as group 1 and the one on the righ t as group 2. In Figs. 10(a),(b) and; (c) each of the groups contain equal proportion of individuals. S t i l l , they d i f f e r with respect to the strength of the groups and the extent of difference between groups. In Figs. 10(a) and 10(b) the group members cluster around two judgment neuclei (the modal judgment) which are at a equal distance from the neutral position. However, i n Fig. 10(b), group 1 i s stronger i n opinion than group 2, i n the sense that members i n group 1 are more united (as can be seen from the r e l a t i v e l y sharp shape of the curve). On the other hand, i n Fig. 10(a) the two groups are of equal strength. In Fig. 10(c), both the groups are equally strong, but group 1 holds a more extreme opinion than group 2. Now i n (.30.) We note that when the group i s divided into two opposing groups, as i n our case, i t i s not quite meaningful to ta l k about a single group judgment. In such situations, i t i s perhaps, more desirable to study the entire d i s t r i b u t i o n (see discussion on group judg-ment i n Subsection 3.2, Chapter 1). Fig. 10. Some bimodal judgment distrib u t i o n s with opposing subgroups 108 109 Figs. 10(d), (e) and ( f ) , the groups are disproportionate. In Figs. 10(d) and (e) the groups cluster at an equal distance while i n Fig. 10(f) they cluster at an unequal distance from the neutral point. In Fig. 10(e), the smaller group holds a stronger opinion than the larger group. Thus, by looking at the judgment d i s t r i b u t i o n s , i t i s possible to discern the difference between the groups, and accordingly assess r i s k s before action i s taken^"^ 2 Further Research Directions Apart from the results i n the previous section, another c o n t r i -bution of t h i s empirical investigation i s that i t has also uncovered new areas of research, both i n theoretical s t a t i s t i c s and i n applied s o c i a l psychology. The research findings w i l l not only increase the ef f i c i e n c y of the method but they are also of interest- for t h e i r own sake. 2.1 Testing Goodness of F i t between two Populations with Correlated Observations Let us consider two multinomial populations R^tandcP^;-w-ith "k categories i n each. Samples of sizes n^ and respectively are ob-tained from the two populations. For testing goodness of f i t , a stan-dard test i s the chi-square test which assumes that the two samples C31) In an actual decisionmaking situation i t w i l l also be helpful to look at the reasons as w e l l as t h e i r weights (importance). 110 are independent of each other, and also, that the samples .are randomly-drawn from the respectively populations. This assumption, thus, i m p l i -es, on the one hand, that the observations within a sample must be uncorrelated, and on the other hand, that there must not be any corre-l a t i o n between the observations i n the two samples. The assumptions i n other standard tests of goodness of f i t , such as, the Kolmogorov-Smirnov t e s t , are even s t r i c t e r . This test assumes not only the same assumptions as i n the chi-square t e s t , but also , that there must not be any t i e s among two or more observations, r e s t r i c t i n g the test from being applied to grouped data. Although a modified version of the Kolmogorov-Smirnov test f o r applying to grouped and discrete data has been developed by P e t t i t t and Steph-ens [12] f o r a single sample case, the independence assumptions, are the same as those for the chi-square t e s t . The problem arises when the assumptions of zero correlation are violated by the observations, f o r instance, by observations obtained through qua l i t a t i v e feedback. Suppose data are obtained by.:applying such a protocol. Let be the sample from the f i r s t stage, and z-^ , z^,..., z^. be the sample from the mth round, before grouping, where x^ and z^ are the responses of the i t h participant. We notice f i r s t that x^ and z^ are correlated being the response of the same indiv i d u a l on two different occasions - before feedback and after feedback. Also the z^'s are correlated within themselves; t h i s i s because after feedbacks one participant's reasons, have influenced another's response. Thus a correlation among the z^'s i s produced. I l l Existence of these two types of correlations contradicts the assumption of independence between samples and within samples. Thus, i f i t i s required to test whether the f i r s t stage d i s t r i b u t i o n i s the same as that of the t h i r d stage d i s t r i b u t i o n , s t r i c t l y speaking, the usual tests described above are not applicable. A more general sit u a t i o n occurs when there i s also a correlation between the x.*s, for instance, when the x.'s are the responses at a x x c stage l a t t e r than the f i r s t stage and e a r l i e r than the mth stage. Now, when the observations are grouped, we may consider them to" have come from two multinomial populations. The problem i s then to test the goodness of f i t between the two multinomial d i s t r i b u t i o n s . I t i s in this direction that a test may be developed that takes account of the correlation structure. 2.2 Goodness of Composite L i s t The composite l i s t of reasons, used i n quali t a t i v e controlled feedback, i s the medium through which information i s fed back. The success of getting a reasoned judgment, therefore, depends greatly on how the l i s t i s prepared, or, i n other words, on the quality of the l i s t . Many questions are related to the preparation of the l i s t . Should only one person (as was done i n our case) make the l i s t ? Should a group be formed so each member of the group preparesca .'list indepen-dently, and then s i t together to form an anonymous l i s t ? or should they consult each other, face-to-face, and make the l i s t ? How should one proceed i n making the l i s t ( i . e . i n summarizing the statements given by the panelists)? Can there be any difference i n behavior due to composite l i s t s prepared i n different ways? F i n a l l y , what c r i t e r i a should be followed to judge the quality of a composite l i s t ? Much empirical research i s needed to get answers to the above questions. However, at t h i s point, we w i l l be interested only with the l a s t ques-t i o n , i . e . , 'What c r i t e r i a should be followed to judge the quality of a composite l i s t ? ' . In the following paragraphs we w i l l suggest a c r i t e r i o n by which the quality of l i s t s prepared by different approa-ches may be compared. It i s natural to assume that a good composite l i s t of reasons must be representative of the arguments given by the participants. That i s , the l i s t should be able to reproduce the potential arguments raised by the participants. This means that, i n a good composite l i s t of reasons, i f a reason i s supposed to represent a certain participant's arguments, when presented to the said p a r t i c i p a n t , he should be able to i d e n t i f y his argument i n the same reason. On the basis of t h i s p r i n c i p l e , a test can be developed to see whether a l i s t prepared by a certain procedure i s good enough to represent the participants' reasons. The test i s proposed below. Let there be N subjects each of whom i s asked to give his reasons supporting judgment on a controversial issue. A composite l i s t contain-ing reasons R-^ , . . . , R^ i s prepared by using a certain method. Let R be the set [R-j_, R 2, R^J of a l l the reasons i n the composite l i s t and Sj be the set of reasons which i s supposed to represent the j t h subject'-s reasons (j = 1,2,.. .,N); S^  contains kj elements, and S^CR. We assume that S^  i s an exhaustive set of reasons for subject j . S-'s, 113 need not necessarily be mutually exclusive. The l i s t i s presented to the j t h subject without indicating which reasons are supposed to represent his arguments. He i s then asked to check the reasons i n the l i s t he agrees with as representing his arguments. I f i t i s found that he checks a reason i n the set Sj 5 we say that a t i e has occured. Let T- (0<TJ < k •) denote the number of t i e s for the j t h subject. Define the s t a t i s t i c T = 11 + T 2 +.... + T N = Total number of t i e s for a l l subjects, where 0 < T1.J-K, and K = + + ... + k . A high observed value of T w i l l mean that a greater number of reasons given by panelists could be representated i n the composite l i s t . Thus, T can be used as a test s t a t i s t i c . The exact nature of T remains to be examined. By using the above procedure, different methods for preparing composite l i s t s can be compared. 3 Some Recommendations In t h i s section we make some recommendations designed to i m p r . r-Q'ter other applications of the q u a l i t a t i v e controlled feedback pro-cedure . ' should (1) I f possible, a larger sample/be taken at the f i r s t stage. This i s because most potential reasons are generated at the f i r s t stage; even i f some of the participants do not give reasons supporting t h e i r judgments (as was observed i n our case) many others w i l l . This w i l l ' ' help i n bringing out a variety of arguments about the issue. Also, even i f a r e l a t i v e l y high nonresponse occurs at subsequent stages, the large number of reasons generated at the f i r s t stage would serve the purpose of effective feedback of information. (2) I f possible, participants should be given the knowledge of the exact number of stages i n which they would have to participate. This would greatly i n t e n s i f y the motivation to par t i c i p a t e . Also the number of stages ;.sh<ould be kept at a lower l e v e l , say 2, 3 or 4, i n the case of a paper-and-pencil survey. (3) For wri t i n g reasons participants should be encouraged to record the reasons by attaching s e r i a l numbers (e.g. 1, 2, 3 etc.). This can be accomplished by supplying each participant with a blank sheet of paper with s e r i a l numbers printed at reasonable spaces. This i s l i k e l y to help to motivate participants to be precise in giving reasons. (4) Information on a l l cue variables should be asked at the f i r s t stage. By so doing, more background information can be obtained about the dropouts at the subsequent stages, so that I f necessary dropouts can be i d e n t i f i e d more e f f e c t i v e l y . (5) In preparing the composite l i s t , one of the following pro-cedures may be adopted, although i t i s expected that the f i r s t of the two should produce a better l i s t . Procedure 1. A small group of people should be selected for pre-paring the l i s t . Each of the group members prepares a l i s t indepen-dently, and without consulting the others. A l l the members then s i t together with t h e i r l i s t s and compare them. F i n a l l y they come up with a single composite l i s t . Procedure 2. As i n Procedure 1, a group of individuals should be selected for preparing the l i s t . - The members should s i t together and j o i n t l y review the reasons of the participants, and then prepare a l i s t . In this case, before s i t t i n g together, the ind i v i d u a l members may read the reasons independently. 4 Concluding Remarks In conclusion, i t may be said that the method of quali t a t i v e controlled feedback has been demonstrated to be a promising technique for judgmental data c o l l e c t i o n . I f conducted according to the proce-dures outlined above, the method i s l i k e l y to produce useful data for large populations under consideration. It i s reasonable to expect that, instead of a paper-and-pencil questionnaire survey, computers could be developed to c o l l e c t judg-ments and reasons, and process reasons before feedback. This would d r a s t i c a l l y reduce the time required between stages , as well as provide an opportunity for a greater number of i t e r a t i o n s . F i n a l l y , we must mention that i n our study we mainly focussed on the basic operational elements of the method, and attention was not given to derive group judgment, or to predict judgment. These are two broad areas where there i s scope for further judgment research using qua l i t a t i v e feedback. 116 BIBLIOGRAPHY [1] Asch, S.E., "Effects of Group Pressure upon the Modification and Distortion of Judgments", i n Cartwright, D. and A. Zander (Eds .) , Group Dynamics :. Research and Theory, New York: Peterson and Co., 1962 [2] Campbell, A. A., "Two Problems i n the Use of the Open Question", Journal of Abnormal and Social Psychology, Vol. 40, pp. 340-343, 1945 [3] C o l l i n s , E. B. and H. Guetzkow, A Social Psychology of Group  Processes for Decisionmaking, New York: John Wiley and Sons Inc., 1964 [4] Dalkey, Norman C. and Olaf Helmer, "The Use of Experts for the Estimation of Bombing Requirements - A Project Delphi Experiment", R-1283-PR, The Rand Corporation, 1951 [5] Festinger, Leon, A Theory of Cognitive Dissonance , E vans ton ,-I'M!.: Row, Peterson, 1957 [6] Janis , Irving L., Victims of Groupthink: A Psychological Study of;:.Foreign Policy • Decisions and Fiascoes , Boston : Houghton M i f f l i n Co., 1972 [7] Johnson, Donald M., The Psychology of Thought and Judgment, New York: Harper and Bros., 1955 [8] Johnson, Donald M., "A Systematic Treatment of Judgment", Psychological B u l l e t i n , Vol. 42, pp. 19 3-224, 1945 [9] Kish, L., Survey Sampling, New York: John Wiley and Sons Inc., 1965 [10] Linston, Harold A., and Murray Turoff, The Delphi Method: Thechniques and Applications , New York: Addison-Wesley Publishing Co. Inc., 19 75 [11] Nerlove, Marc and S. James Press, "Univariate and Multivariate Loglinear and Logistis Models", R-1306-EDA/NIH, The Rand Corporation, December, 19 73 [12] P e t t i t t , A. N. and M. A. Stephens, "The Kolmogorov-Smirnov Goodness of F i t S t a t i s t i c with Discrete and Grouped Data", Technometries, Vol. 19, No. 2, pp. 205-210, 1977 [13] Press, S. James, "Qualitative Controlled Feedback for Forming Group Judgments and Making Decisions", Paper to be published i n the Journal of the American S t a t i s t i c a l Association. 117 [14] Sackman, H., Delphi Critique - Expert Opinion, Forecasting  and Group Process, Lexington, Mass.: Lexington Books, D. C. Heathe and Co., 1975 [15] Salancik, J. R., Wegner, W. and E. Heifer, "The Construction of Delphi Event Statements", Technological Forecasting  and Social Change, Vol. 3, pp. 65-73, 1971 [16] Speak, M., "Communication Failure i n Questioning: Errors, Misinterpretations and Personal Frames of Reference", Occupational Psychology, Vol. 41,.pp. 169-179, 1967 118 APPENDIX A l . The F i r s t Stage Questionnaire THE UNIVERSITY OF BRITISH COLUMBIA 2075 WESBROOK PLACE VANCOUVER,B.C.,CANADA V6T 1W5 FACULTY OF COMMERCE AND BUSINESS ADMINISTRATION F t o i J,uly ,1976 Dear Sir/Madam: May we ask your help i n a research study we are conducting i n the Faculty of Commerce which involves group decision making. We are trying to study the general process of how groups make decisions. We have developed theoretical s t a t i s t i c a l models which w i l l predict the f i n a l decisions, when the information about the problem i s presented i n a certain way. We now need to test the theory, and that i s where you come i n . We need to coll e c t r e a l data regarding an actual deci-sion. For t h i s experiment involving group decision making, we have chosen an issue which i s of interest to a l l of us: the new indoor swimming pool (aquatic center). You w i l l be asked to provide a nume-r i c a l answer to only one judgmental question r e l a t i n g to this swimming pool (there w i l l also be some factual information requested) , and you w i l l be asked to provide reasons for your answer. The reasons are most  important. We w i l l gather a l l the reasons given by a l l respondents, and then come back to you and t e l l you.everyone else's reasons (but we won't t e l l you the numerical answers they gave, not even the average number). We w i l l then ask you the same basic question a second time, to see i f other peopleJsereasons w i l l persuade you to change your answer. We 119 expect to do this three times. During the course of the experiment you are requested not to discuss your answers or reasons with any other panelists i n our survey; i f you do, i t w i l l invalidate the experi-ment. We w i l l maintain this d i s c i p l i n e with a l l p a n e l i s t s , and w i l l keep a l l responses anonymous. Equal numbers of students, Faculty, and Staff at U.B.C. have been selected randomly, as panelists. We w i l l give you a nominal re-ward of one d o l l a r for your trouble, i f you help us complete the ex-periment ( a l l three stages of being asked the same question). We wish to complete the whole process of data c o l l e c t i o n by the end of the Summer Session 1976. To obtain f r u i t f u l results from our study we w i l l need your f u l l cooperation and hope you w i l l be w i l l i n g to give i t . Background for the Questionnaire We would l i k e to give you some background information about the aquatic center. The new aquatic center i s now under i t s f i r s t phase of construction and i s located innfnont of the Student Union Building. I f s u f f i c i e n t additional funds are coll e c t e d , construction w i l l c o n t i -nue to the f i n a l stage. A b r i e f history behind constructing the center i s as follows. In early 1970 a recreation group of the university stimulated the need for a covered swimming pool on campus, and proposed that the exis-ting Empire pool be covered. In subsequent years the idea of covering 120 the existing outdoor pool was rejected i n favor of a new f u l l scale, aquatic center. The Alma Mater Society of U.B.C. took the issue i n th e i r hands. There, was, however, a good deal of controversy among the students about whether or not such an aquatic center should be b u i l t ( i n terms of whether the money to be spent might be used for "better" purposes). F i n a l l y , the issue was settled through two student referendums and the decision to construct the center was taken. Regarding financing of the center, the students pledged to donate $925,000 (to be collected from a compulsory, annual fee of $5.00 from each student) towards a t o t a l estimated cost of $4.7 m i l l i o n . The curr-ent plan i s that the remainder of the required funds w i l l come from the University, P r o v i n c i a l and Federal Government grants, donations from Faculty, S t a f f , Alumni, and the community; the University w i l l subsidize maintenance of the center. Additional costs for lockers, towels, laundry, etc. w i l l probably be on an individual user basis. The center w i l l enable the University to provide recreational and tra i n i n g programs for both members of the academic community (students, faculty and s t a f f ) , and the general public. Such programs might i n d u -ed general swimming for a l l , in addition to Learn-to-swim, Red Cross c e r t i f i c a t i o n etc. We i n v i t e you to complete the attached questionnaire.-For example, i f you f e e l such an aquatic center i s "Extremely Unimpo tant" , put 0 i n the box. Now go on to Question B, for your reasons. Question C w i l l request factual data. B. To help persuade others in our survey, i n the next two stages, about how important the center i s , please enumerate any reasons you may have had for giving the answer to Question A. I f you do not provide reasons , your answer w i l l not persuade other respon-dents. I f you have general comments on the questionnaire (not reasons for your answer) you may give them i n Question 17, i n C. 123 C. Please provide the following background data to help us i n our evaluation. This information w i l l be kept confidential and w i l l not be used for any purpose other than t h i s survey. 1. Name: 2. Address (where i t i s easiest to reach you during the day, on campus i f Possible) : 3. Telephone Number (on campus, i f possible): 4. Status: Student Faculty Staff 5. I f you checked "Student", are you working for a degree at U.B.C? 6. Yes No 6. I f you are a student, are you: Undergraduate Graduate 7. Sex: Male Female 8. Would other members of your family, use the center? Yes No 9 Do you know how to swim? Yes No 10. Do you l i v e on campus? Yes No 11. Using your usual means of transportation to commute to the campus, do you l i v e more than t h i r t y minutes away? Yes No 12. I f you are a Staff member or a Faculty member, have you already donated $5.00 or more toward construction of the center? Yes No 13. I f you have not already donated, would you be w i l l i n g to donate $5.00 or more toward construction of the center? (This questionnaire w i l l not be usdd to ask you for a donation). 124 Yes No 14. How much annual fee would you be w i l l i n g to pay for use (lockers, towels, etc.) of the aquatic center? (The current compulsory stu-dent fee of $5.00 for construction w i l l probably not cover t h i s ) . Please write the amount i n the box. 15. Do you have a swimming pool i n your residence or near your r e s i -dence (excluding the currently existing UBC outdoor pool). Yes No 16. How frequently would you use an indoor aquatic center during the regular academic year, i f there were one available on the campus? Please put your answer i n the box: 0 - Not at a l l , 1 - Occasionally, 2 - Frequently, 3 - Very Frequently 17. Please use t h i s space to give any general comments you may have on t h i s questionnaire. I f necessary, please use the reverse side of t h i s page. 126 on the f i r s t stage and determine whether or not your o r i g i n a l judgment regarding the importance of the proposed aquatic center is s t i l l v a l i d . Perhaps i n the l i g h t of some of the reasons by other group members your judgment has now changed; perhaps not. In any case, now re-answer the o r i g i n a l importance question. (2) In studying the composite l i s t of reasons, check o f f those reasons which you f e e l you took into account i n making your importance judgment on this round. I f you f e e l there are .-...some additional reasons relevant to your current judgment that are not l i s t e d i n the composite, please add them to the l i s t . (3) Please also give some additional factual information which we we neglected to request on the f i r s t stage. We would l i k e to give you an idea about how we prepared the compo-s i t e l i s t of reasons. F i r s t we c a r e f u l l y sorted out the reasons which showed very much the same kind of argument, and then we made a single reason out of them. In order to do t h i s , sometimes we had to drop very minor d e t a i l s . We hope you w i l l find the reasons you gave on the f i r s t stage i n the composite l i s t , even though these reasons may appear in a s l i g h t l y d i f f e r e n t form from the way i n which you gave them. We would be very grateful i f you would kindly take the trouble to f i l l out the attached questionnaire, and thus help us i n carrying out our research study. Please send us back the completed questionnaire (including your f i r s t stage reasons sheet and the composite l i s t ) by campus mail, using the s e l f addressed envelope. Thank you. 127 QUESTIONNAIRE ON GROUP DECISION MAKING USING CONTROLLED FEEDBACK (Second Stage) Please car e f u l l y read the composite l i s t of reasons before answer-ing Question A; they include your own o r i g i n a l reasons. Check o f f the reasons which-are relevant i n determining your answer to the question. I f you have any new reasons , please add them at the end of the . composite l i s t . The Composite L i s t of Reasons 1. The University area has a large population with - no winter access to swimming. The Aquatic Center w i l l meet t h i s need of the univer-s i t y community; also the center, can be used by the surrounding community. 2. As a major university i n B r i t i s h Columbia UBC should possess ade-quate f a c i l i t i e s i n providing physical education and a t h l e t i c t r a i n -ing. The aquatic center can be used (as a lab) by the Physical Education Department i n giving t r a i n i n g i n swimming and other aquatic sports. 3. The Aquatic Center w i l l provide a good l i n k between the University community and the general public; also, i t w i l l improve r e l a t i o n -ships between major subgroups within the University community (students, faculty and s t a f f ) which w i l l lead to a better learning and working environment. 4. The aquatic center w i l l be a good asset to the campus since i t w i l l provide a convenient f a c i l i t y for the use of the university students-128 and employees, and the nearby community, for swimming - an exercise good for recreation, health and f i t n e s s . 5. B r i t i s h Columbia, and i n general Canada, should have the potential for f i r s t class competitive swimming, so that swimmers can show good performance i n the Olympics. The center w i l l improve quality of competitive swimming and other aquatic sports. 6. Existing f a c i l i t i e s for year-round swimming i n Vancouver are few i n Number. 7. Use o f the center for swim meets w i l l be a good source of p u b l i c i t y for UBC. 8. Development of good f a c i l i t i e s for the University i s always import-ant, whether recreational or otherwise. 9. As an academic i n s t i t u t i o n i t i s not within the r o l e of the Universi-ty to provide recreational and a t h l e t i c f a c i l i t i e s on such a grandi-ose scale. 10. The present time i s one of the severe f i n a n c i a l stringency at t h i s University. When a university i s experiencing "hard times" to keep i t s essential academic functions running and improving, any money that can be used by the university should be spent on academic, rath-er than a t h l e t i c , purposes. 11. A r e l a t i v e l y inexpensive covering device for the existing Empire pool would be much more r e a l i s t i c i n terms of expenses and need of the university (i.e.the needs of the students , faculty and s t a f f ) . 129 12. A good number of outdoor pools ( for example, pools i n community-centers) and some indoor pools (e.g. the Lord Byng High School pool, the English Bay pool) are available i n the c i t y ; so there i s l i t t l e need for the community to have an additional new pool on t h i s campus. 13. Since the proposed aquatic center would be located at a place far away from any sizeable r e s i d e n t i a l community, i t would not l i k e l y be used very e f f e c t i v e l y by the outside community. 14-. A very small fraction of students, faculty and s t a f f and (perhaps) community w i l l use the center. The large cost involved for such a lim i t e d use i s not justified.'' 15. Recreational f a c i l i t i e s on the UBC campus are already p l e n t i f u l and excellent, including one outdoor pool which meets the present needs. 16. The money could be spent on other, better, alternatives such as: (a) student housing; (b) audio-visual aids to class rooms; (c) better transportation, such as, a rapid t r a n s i t system from Blanca to campus; (d) building a covered pool outside the campus; (e) providing improved medical f a c i l i t i e s for university community members; (f) c u l t u r a l programs; (g) a variety of recreational and a t h l e t i c programs; (h) in updating women's status i n the work force; 130 ( i ) s t a f f salary; ( j ) other projects. 17. I f there i s any need for a pool by the Physical Education Depart-ment for t r a i n i n g purposes, during the 6 months period when the Empire pool cannot be used, that need could be met by the Lord Byng covered pool. New Reasons I f you have any new reasons, please put them i n the space below. Have you checked o f f your current set of reasons you w i l l take into account i n giving your "importance r a t i n g " on. t h i s stage? 131 Basic Question Question A: How important (necessary) do you f e e l i t i s for the University of B r i t i s h Columbia to complete construction of an indoor aquatic center on the campus that would be available for use by students, f a c u l t y , s t a f f , and th e i r f a m i l i e s , and the general Vancouver community.[Note: By "complete construct ion"-wwe mean, how important i s i t to have an indoor aquatic center.] Please give i n t h i s box [ ] the numerical rating which comes closest to corresponding with your own feelings, according to the following table: Extremely Unimportant 0 Very Unimportant 12 5 Moderately Unimportant 25 0 Somewhat Unimportant 37 .5 Indifferent or Neutral 50 .0 (don't care whether or not i t gets b u i l t ) ••2 5 Somewhat Important 62 .5 Moderately Important 7iS .0 Very Important 87 .5 Extremely Important 100 .0 Your answer (numerical rating) to Question A at the l a s t stage was 132 Question B. (Factual Information): Please provide the following background information to help us i n our evaluation. This information w i l l be kept confidential and w i l l not be used for any purpose other than t h i s study. 1. Name: 2. Address (on campus): 3. Telephone Number (on campus, i f possible): 4-. Age: Below 25 [ ], 25 - 34[ ] , 35 - 44 [ ], 45 - 54 [ ], 55 or more [ ]. 5. How long have you been a member of the UBC community: Less than one year [ ] , one to four years [ J , fi v e or more years [ ]? 6. I f you are a s t a f f member, are you administrative [ ] , c l e r i c a l C ] , other [ ] ? 134 formed a composite of a l l the reasons given by a l l study participants. On Round Two, we showed you the composite of reasons and then asked you to answer the same question "A" again. We also asked you to give reasons for your numerical response on the second round (your reasons on the second round might or might not have been the same as those you gave on the f i r s t round). We have now formed a composite of a l l of the Round Two reasons given by a l l participants. We want you to answer question "A" again (for the l a s t time), a f t e r you study the composite of reasons, and again we want you to give reasons for your numerical response (your new reasons might or might not overlap with your reasons given on the two e a r l i e r rounds). We are now asking you to do three things: (1) Please study the t h i r d stage composite l i s t of reasons, and then answer question "A" (your answer to question "A" on Round Two i s provided as a reminder; we are also reminding you of the reasons you gave on round two). (2) Please check' o f f which reasons i n the composite l i s t which you now f i n d as contributory towards your current response to ques-'lop t i o n "A". I f there are some new reasons you now have but which don't appear i n the composite, please give them as w e l l . (3) Please answer the two subsidiary questions following question "A". 135 We hope to receive your completed questionnaire by February 25, 1977. I f you would l i k e to receive a copy of the summary report on t h i s study, you may indicate that i n the questionnaire. Please return the entire questionnaire (you may r e t a i n t h i s l e t t e r ) by using the self-addressed envelope supplied. Upon receipt of your Round Three Questionnaire, we w i l l send you a d o l l a r , as promised e a r l i e r , as a token measure of our gratitude for your pa r t i -cipation i n the study. Sincerely, S.J. Press Faculty of Commerce M. W. A l i Institute of Applied Mathematics and S t a t i s t i c s M 136 QUESTIONNAIRE ON GROUP DECISION MAKING USING CONTROLLED FEEDBACK (Third Stage) Instruction: Please car e f u l l y read the following t h i r d stage composite l i s t of reasons, before answering question "A". Check off the reasons which you think are relevant for your question "A"-answer by putting a "/" mark i n the appropriate l e f t hand box; we have placed checks i n the ri g h t hand boxes corresponding to the reasons which you checked at the second stage. Note that the reasons with a are the new reasons added by the panelists at the second stage. I f you have any new reasons now, please add them at the end of the composite l i s t . Warning: Note ca r e f u l l y that i f a reason appears below i t means only that the reason was given by one or more respondents; i t may have been c i t e by most respondents, or i t may have been cited by only one respondent. 137 COMPOSITE OF REASONS GENERATED BY THE PANEL DURING "ROUND TWO" Check here i f Reasons checked t h i s reason i s here were given part of your by you on Round current thinking 2. [ ] "1. Since i t i s not possible to cover the Empire pool, [ ] i t i s better to have a.covered aquatic center. [ ] *2. The existing Empire pool i s w e l l used by the community [ ] - one would expect the aquatic center to be used even more . [ ] 3. The money could be spent on other, better, a l t e r - [ ] natives such as: (a) student housing; (b) audio-visual aids to classrooms; (c) better transportation, such as a rapid t r a n s i t system from Blanca to campus; (d) building a covered pool outside the campus; (e) providing improved medical f a c i l i t i e s f o r university community members; (f) c u l t u r a l programs; (g) a variety of recreational and a t h l e t i c programs; (h) i n updating women's status i n the work force; ( i ) s t a f f salary; ( j ) other projects. C ] 4. The aquatic center w i l l provide a good l i n k between [ ] 138 the university community and the general public; also [ ] i t w i l l improve relationships among major subgroups within the University community (students, f a c u l t y , and s t a f f ) which w i l l lead to a better learning and working environ-ment . [ ] *5. It i s not certain that the Lord Byng pool w i l l be C ] available for our athletes ,we should, therefore, have our own covered pool. [ ] 6. Recreational f a c i l i t i e s on the UBC campus are already [ ] p l e n t i f u l and excellent, including one outdoor pool which meets the present needs. [ ] *7. The center w i l l be used i n large part f o r academic [ ] and research purposes by a s i g n i f i c a n t cross section of the university community. [ ] 8. As an academic i n s t i t u t i o n i t i s not within the r o l e [ ] of the University to provide recreational and a t h l e t i c f a c i l i t i e s on such a grandiose scale. [ ] 9. A good number of outdoor pools (for example, pools i n [ ] community centers) and some indoor pools (e.g., the Lord Byng High School pool, The English Bay pool) are a v a i l -able i n the c i t y ; so there i s l i t t l e need for the community to have an additional new pool on t h i s campus. C J ] 10. As a major university i n B r i t i s h Columbia UBC should [ ] 139 possess adequate f a c i l i t i e s i n providing physical [ ] education and a t h l e t i c t r a i n i n g . The aquatic center can be used (as a lab) by the Physical Education Depart-ment i n giving tra i n i n g i n swimming and other aquatic sports. r. [ ] *11. Instead of being a source of p u b l i c i t y i t i s suspected [ 3 that the pool would create antipathy among the public, since what i s being constructed i s peripheral to the university's function. [ ] *12. Since the University contribution i s small enough' [ ] compared to the t o t a l cost of the center, i t i s worth-while to have an aquatic center on campus. [ ] *13. I f there i s any need for a pool by the Physical Educa'-'' [ ] t i o n Department for t r a i n i n g purposes, during the 6 months period when the Empire pool cannot be used, that need could be met by the nearby Lord Byng covered pool. [ ] "14. The center could become a source of revenue i f used [ ] for competition and swimming exhibition. [ ] "15. There are already too many " v i s i t o r s " who are allowed [ ] to take up parking space that should be reserved for students, faculty and s t a f f ; the aquatic center, i f accessible to outside v i s i t o r s , w i l l make the parking problem more acute. 140 [ 3 16. Since the proposed aquatic center would be located [ ] at a place f a r away from any sizeable r e s i d e n t i a l community i t would not l i k e l y be used very e f f e c t i v e l y by the outside community. [ ] 17. Development of good f a c i l i t i e s for the University [ 3 i s always important, whether recreational or otherwise. [ 3 18. Existing f a c i l i t i e s for year-round swimming i n [ 3 Vancouver are few i n number. [ ] "19. The University should obtain funds and complete [ ] t"/-. the Asian Center before commiting i t s e l f to another building that i t may not be able to complete due to i n s u f f i c i e n t funds. [ 3 20. B r i t i s h Columbia, and i n general Canada, should [ ] have the potential f o r f i r s t class competitive swimming so that swimmers can show good performance i n the Olympics. The center w i l l improve quality of compe-t i t i v e swimming and other aquatic sports. [ 3 21. Use of the center for swim meets w i l l be a good [ 3 source of p u b l i c i t y for UBC. [ 3 22. A r e l a t i v e l y inexpensive covering device for the [ 3 existing Empire pool would be much more r e a l i s t i c i n terms of expenses and need of the University ( i . e . , the needs of the students, faculty and s t a f f ) . 141 [ ] 23. The present time i s one of severe f i n a n c i a l s t r i n - [ ] gency at t h i s University. When a University i s experi-encing "hard times" to keep i t s essential academic functions running and improving, any money that can be spent by the University should be spent on academic, rather than a t h l e t i c , purposes. [ ] 24. A very small f r a c t i o n of students, faculty and [ ] s t a f f and.(perhaps) community w i l l use the center. The large cost involved for such a limited use i s not j u s t i f i e d . [ ] 25. The University area has a large population with no - [ ] winter access to swimming. The aquatic center w i l l meet t h i s need of the University community; also the center can be used by the surrounding community. [ ] 25. The aquatic center w i l l be a good asset to the campus [ ] since i t w i l l provide a convenient f a c i l i t y for the use of University students and employees, and the nearby community, for swimming - an exercise good for recrea-t i o n , health and f i t n e s s . New Reasons I f you have any new reasons, please put them i n the space below. Have you checked o f f your current set of reasons you w i l l take into account i n giving your "importance r a t i n g " on t h i s stag 143 BASIC QUESTION Question A. How important (necessary) do you f e e l i t i s for the University of B r i t i s h Columbia to complete construction of an indoor aquatic center on the campus that would be available for use by students, f a c u l t y , s t a f f , and t h e i r f a m i l i e s , and the general Vancouver community. (Note: By "complete construction" we mean, how important i s i t to have an indoor aquatic center). Please give i n t h i s box [ ] the numerical rati n g which comes closest to corresponding with your own feeli n g s , according to the following table: Extremely Unimportant 0 Very Unimportant 12 5 Moderately Unimportant 25 .0 Somewhat Unimportant 37 .5 Indifferent or Neutral (don't care whether or not i t gets b u i l t ) 50 .0 Somewhat Important 62 .5 Moderately Important 75 .0 Very Important 87 .5 Extremely Important 100 .0 144 Your answer (numerical rating) to Question A on the second stage was . I f you place a check i n t h i s box you w i l l receive a copy of a summary report of t h i s study. [ ]. SUBSIDIARY QUESTIONS Please provide the following subsidiary deformation which w i l l be helpful to us i n analysing the r e s u l t s . Residence 1. Please indicate with a check i n the appropriate box the location of your normal l o c a l residence i n the Vancouver metropolitan area. [ ] West side (West of Gr a n v i l l e , on UBC side of Burrard I n l e t , but i n Vancouver). [ ] East side (East of G r a n v i l l e , on UBC side of Burrard I n l e t , but i n Vancouver). [ ] West Vancouver or North Vancouver. [ ] West End (downtown area). [ ] Burnaby or Coquitlam. [ ] Richmond. B • ] Surrey, New Westminister, Delta. [ 3 Other (please indicate location on the next l i n e ) . Confidence Rating 2. Now that you have finished answering question "A", we would l i k to ask you to rate yourself, on the following scale, i n terms- o the degree of confidence you have i n that your own answer to question "A" actually represents your true feelings about the . issue. Please indicate the number below that best describes your confidence l e v e l i n your answer to Question "A". Not at a l l confident 1 2 Not quite confident 3 4 Quite confident 5 6 Very confident 7 8 Absolutely confident 9 APPENDIX AM-. The Third Stage Questionnaire with Nonrandomised Composite L i s t of Reasons. We do not reproduce the questionnaire here, since t h i s i s the same as the questionnaire i n APPENDIX A 3 except that the ordering of the reasons i n the nonrandomised composite l i s t was di f f e r e n t . The order i n which the reasons appeared can be found i n APPENDIX F. Note in that order that a l l the pro reasons appeared at the f i r s t part of the l i s t followed by a l l the con reasons. 147 APPENDIX A5. The Questionnaire for the Control Group. We do not reproduce the questionnaire here, since i t i s the same as the F i r s t Stage Questionnaire (items 5 and 6 dropped) i n APPENDIX A l with 3 additional items (item numbers 4, 5 and 6) from the Second Stage Questionnaire (APPENDIX A2) and two additional items (item numbers 1 and 2) from the Third Stage Questionnaire (APPENDIX A3) Appendix B. Empirical Frequency Distributions of Response — F i r s t Stage Second Stage Third Staj Control Response . — +-> H 3 o rd cp 4H rd p CO i rQ g V O C O -H >> p H 3 o rd t n 4H MH rd p CO i 6 a> O C O "H >> p H 3 O rd tM 4H MH rd p CO o c CJ .H >, p H 3 o rd PH tp cp rd P CO i rQ E OJ o c O -H 0 3 3 6 3 3 6 4 3 7 4 3 7 12.5 77 2 9 8 2 10 7 5 12 1 2 3 22500 7 3 10 4 7 11 7 9 16 4 3 7 37.5 2 3 5 7 7 14 5 2 7 2 2 4 50.0 6 7 13 6 2 8 4 3 7 8 2 10 62.5 8 7 15 12 6 18 11 5 16 6 10 16 75.0 18 17 35 10 15 25 12 16 28 6 8 14 87.5 3 13 16 4 13 17 4 12 16 11 12 23 100.0 0 2 2 0 2 2 0 2 2 4 1 5 Total 54 57 111 54 57 111 54 57 111 46 43 '• 89 00 Appendix CI. Frequency Table showing change i n response from fhesEirs.tgStageStootheSSegondlStagejects) SECOND STAGE 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 Total 0 6 6 12.5 7 1 1 9 25.0 3 6 1 10 37.5 1 4 5 50.0 1 5 5 1 1 133 62.5 1 3 1 10 15 75.0 1 1 2 6 23 2 35 87.5 1 15 16 100.0 2 2 Total 6 10 11 14 8 18 25 17 2 111 Note: T ie entries of the table are f ( z 2 ,z-^ ) and the row tot a l s are f (z-, ) w < H CO Q O O w CO Appendix C2. Frequency Table showing change i n response from the Second Stage to the Third Stage th -THIRD STAGE 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 Total 0 6 6 12.5 1 8 1 10 25.0 1 9 1 11 37.5 2 5 6 1 14 50.0 1 1 5 1 8 62.5 15 3 18 77,5.0 23 2 25 87.5 1 2 i a 17 100.0 r . 2 2 Total 7 12 16 7 7 16 28 16 111 .;' Not e: The entries of the table are f ( z 3 , z 2 ) and the row to t a l s are f ( z 2 ) , Appendix C3. Frequency Table showing change i n response from the F i r s t Stage to the Second Stage (Faculty) SECOND STAGE 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 Total 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 1 2 2 2 3 1 1 5 2 5 10 11 .3 3 7 7 2 6 8 18 3 0 Total 12 10 5ft cn H Appendix C4. Frequency Table showing change i n response from the Second Stage to the Third Stage (Faculty) THIRD STAGE 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 3 1 6 1 1 3 2 1 1 10 2 9 1 1 3 Total 11 12 Appendix C5. Frequency Table showing change i n response from the F i r s t Stage to the Second Stage (Staff) SECOND STAGE i — i 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 Total 0 3 3 12.5 1 1 2 25.0 1 2 3 37.5 1 2 3 50.0 1 3 2 1 7 62.5 1 1 5 7 75.0 1 1 1 13 1 17 87.5 1 12 13 100.0 2 2 Total 3 2 7 2 6 15 13 2 57 Appendix C6. Frequency Table showing change i n response from the Second Stage to the Third Stage (Staff) THIRD STAGE Q + 0 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 Total 0 •3 3 12.5 2 2 225.0 6 1 7 37.5 2 3 1 1 7 50.0 1 1 2 62.5 5 1 6 75.0 14 1 15 87.5 1 1 11 13 100.0 2 2 Total 3 5 9 2 3 5 116 12 2 57 H cn -F 155 APPENDIX DI. Revised Expected Frequencies for the F i r s t Stage and the Control Group Distributions, Response Categories F i r s t Stage Control Total ( lb served Expected Observed Expected [0,12.5] 15 13.88 10 11.13 25 [25,37.5] 15 14.43 11 11.57 26. [50] 13 12.77 10 10.24 23 [62.5] 15 17.21 16 13.80 31 [75] 35 27.20 14 21.81 49 [87.5,100] . 18 25.53 28 20.47 46 Total 111 89 200 APPENDIX D2. Revised Expected Frequencies for the Third Stage and Control Group Distributions Response Categories Third Stage Control Total Observed Expected Observed Expected [0,12.5] 19 16.10 10 12.91 29 [25] 16 12.77 7 10.24 23 [37.5,50] 14 .1155554 14 12.46 28 [62.5] 16 17.76 16 314224 32 [75] 28 23.31 14 18.69 42 [87.5,100] 18 25.53 28 20.47 46 Total 111 89 200 156 APPENDIX D3. Revised Expected Frequencies for the F i r s t Stage and the Third Stage Distributions Response F i r s t Stage Third Stage Total Categories Observed Expected Observed Expected [0,12.5] 15 17.0 19 17.0 34 [25] 10 13.0 16 13.0 26 [37.5,50] 18 16.0 14 16.0 32 [i62.5] 15 15.5 16 15.5 31 275] 35 31.5 28 31.5 63 [87.5,100] 18 18.0 18 18.0 36 Total 111 111 222 157 APPENDIX E. A Technique for Using Data Sets with Missing Observations i n Regression Let us consider a regression with 3 ri g h t hand variables X-^ , X 2, and Xg and suppose that 10 data points are available with some missing observations as shown below. I Y : X^  T 0 x7~l y l X l l X 31 Y2 X12 X32 Y3 X13 X23 X33 \ X14 X24- X34 Y5 X15 X25 X35 Y6 X16 X25 X36 Y7. X17 X27 X37 Y8 X18 X28 X38 Y9 X29 X39 Y10 X2,10 X3,10 To compute the vector of regression c o e f f i c i e n t s (b^, b 2 , b^) we need to compute -1 r i — b i S(xi) S(x 1,x 2) S(x 1 5x 3) S(x x,y) b2 = S(x 2,xp SCx^) S(x 2,x 3) S(x 2,y) A J(x 3 sx 1) S(x 3,x 2) S(X3> S(x 3 ,y) where S(.,.) stands for the sum of.squares of deviations about average Dividing each element i n the matrices by n=10 we get the parameter estimates i n terms of variances and covariances. This i s usually done when we have a f u l l set of observations. According to the technique,however, when there are missing obser-vations , as i n the above example , we compute the variances and cova-riances using only the available observations. Thus, V a r ( X ] L ) = \ .II ( X U - V 2 10 Var(X 2) = i Z ( X 2 i - X 2) X - o • Var(V =iir f . ( x v - x,)2 i = l 3 1 6 Cov(X ,X ) = I t ( X l i " X l ) ( X 2 i " V i=3 8 ( Cov(X 1,X 3) = -8 X ( X l i " hHX3i ~ V i = l 10 Cov(X 2,X 3) = i X ( X 2 i " X 2 ) ( X 3 i " V i = 3 and s i m i l a r l y for Cov(Y,X 1), Cov(Y,X 2) and Cov(Y,X3) may be computed. Fi n a l l y the parameters can be obtained by using these, variances and covariances. 159 APPENDIX F. S e r i a l Number of Reasons i n Tables XVI, XIX, XX , XXI and Appendix G and the corres-ponding S e r i a l Numbers i n Randomised and Non-J-randomised L i s t s . S e r i a l No. S e r i a l No. S e r i a l No. S e r i a l No. S e r i a l No. S e r i a l No. i n Tables i n Rando- i n Non- i n Tables i n Rando- i n Nonran-XVI,XIX, misde L i s t randomised XVI,XIX, misde l i t s domised XX,XXI, £ L i s t XX,XXI, £ L i s t Appdx. G Appdx. G 1 25 1 14 24 26 2 10 2 15 6 16 3 4 3 16 3 15 4 26 4 17 13 20 5 20 5 18 1 13 6 18 6 19 2 10 7 21 7 20 5 14 8 17 8 21 7 9 9 8 17 22 11 19 10 23 25 23 12 12 11 22 24 24 14 11 12 9 18 25 15 21 13 16 22 26 19 23 A'PPENDIX G" Proportion of Participants i n the Randomised and and Nonrandomised L i s t Groups who gave Reason a a 1 2 3 4 5 6 7 8 9 10 11 12 13 x l i 19 24 14 24 13 12 5 19 14 26 17 16 12 : .33.9 .429 .250 .429 .231 .214 .089 .339 .250 .464 .304 .286 .214 X 2j 27 29 12 . 30 12 16 5 12 13 25 21 15 12 P 2 j : . 4 9 1 . 5-27, .218 .546 .218 .291 ,091 .218 .236 .455 .382 .273 .218 1 p(a) .414 .478 .324 .487 .225 .253 .090 .279 .243 .460 .342 .279 .216 Note: i = s e r i a l number of reason i n the randomised l i s t (see Appendix F). j = s e r i a l number of reason i n the nonrandomised l i s t (see Appendix F). x-, • - number of subjects i n the randomised-list-group who gave reason i . x„^ = number of subjects i n the nonrandomised-list-group who gave reason j 2] X2j Total number of subjects who gave reason a p 2 j = — 5 p ( a ) = n r ~ l i 56 * Appendix G i s continued on the next page APPENDIX G (continued) a 14 15 16 17 18 19 20 21 22 23 24 . 25 26 X l i 15 10 22 9 5 10 4 5 8 9 18 6 9 P l i .•268 .179 .393 .161 .089 .179 .071 .089 .143 .161 .321 .107 .161 X 2 j 18 7 20 8 7 9 5 6 5 7 6 4 8 P 2 j .327 .127 .364 .146 .127 .164 .091 .109 .091 .127 .109 .073 .146 p(a) .297 .153 .378 .153 .108 .171 .081 .099 .1171 .144 .216 .090 .153 " See notes i n the previous page. 162-APPENDIX H. The L i s t of Reasons Arranged according to Importance Ranks. Importance Ranks 1. The Aquatic Center w i l l be a good asset to the campus since i t w i l l provide a convenient f a c i l i t y for the use of the university students and employees, and the nearby community, for swimming -an exercise good for recreation, health and f i t n e s s . 2. As a major University i n B r i t i s h Columbia UBC should possess adequate f a c i l i t i e s i n providing physical education and a t h l e t i c t r a i n i n g . The aquatic center can be used (as a lab) by the Physical Education Department i n giving t r a i n i n g i n swimming and other aquatic sports. 3. The present time i s one of severe f i n a n c i a l stringency at t h i s University. When a University i s experiencing "hard times" to keep :ts i t s essential functions running and improving, any money that can be spent by the University should be spent on academic, rather than a t h l e t i c purposes. 4. The University area has a large population with no winter access to swimming. The aquatic center w i l l meet t h i s need of the Univer-s i t y community; also the center can be used by the surrounding community. 5. The money could be spent on other, better, alternatives such as: (a) student housing; 163 (b) audio v i s u a l aids to class rooms; (c) better transportation; such as a rapid t r a n s i t '. :. system from Blanca to campus; (d) building a covered pool outside the campus; (e) providing improved medical f a c i l i t i e s f o r university community members; (f) c u l t u r a l programs; (g) a variety of recreational and a t h l e t i c programs; (h) i n updating women's status i n the work force; ( i ) s t a f f salary; ( j ) other projects. 6. A r e l a t i v e l y inexpensive covering device for the existing Empire pool would be much more r e a l i s t i c i n terms of expenses and need of the University ( i . e . the needs of the students, f a c u l t y , and s t a f f ) . 7. A very small f r a c t i o n of students, faculty and s t a f f and (perhaps) community w i l l use the center. The large cost involved for such a limited use i s not j u s t i f i e d . 8. A good number of outdoor pools (for example, pools i n community centers) and some indoor pools (e.g.the Lord Byng High School pool, the English Bay pool) are avaiable i n the c i t y ; so there i s l i t t l e need for the community to have an additional new pool on t h i s campus. 8. Development of good f a c i l i t i e s for the University i s always impor-164 tant, whether recreational or otherwise. Existing f a c i l i t i e s for year-round swimming i n Vancouver are few i n number. As an academic i n s t i t u t i o n i t i s not within the role of the university to provide recreational and a t h l e t i c f a c i l i t i e s i n such a grandiose scale. The aquatic center w i l l provide a good l i n k between the University community and the general public; a l s o , i t w i l l improve re l a t i o n s among major subgroups within the University community (students, U f a c u l t y , s t a f f ) which w i l l lead to better learning and working environment. B r i t i s h Columbia, and. i n general Canada, should have the potential for f i r s t class competitive swimming, so that swimmers can show good performance i n the Olympics. The center w i l l improve quality of competitive swimming and other aquatic sports. Since the proposed aquatic center would be located at a place fa r away from any sizeable r e s i d e n t i a l community i t would not like-l y be used very e f f e c t i v e l y by the outside community. The center could become a source of revenue i f used for compe-t i t i o n and swimming exhibitions. The existing Empire pool i s well used by the community - one would expect the aquatic center to be used even more. 165 The University should obtain funds and complete the Asian Center before committing i t s e l f to another building that i t may not be able to complete due to i n s u f f i c i e n t funds. Recreational f a c i l i t i e s on the UBC campus are already p l e n t i f u l and excellent, including one outdoor pool which meets the present needs. Since the University contribution i s small enough compared to the t o t a l cost of the center, i t i s worthwhile to have an aquatic center on campus. I f there i s any need for a pool by the Physical Education Depart-ment for t r a i n i n g purposes, during the 6 months period when the Empire pool cannot be used, that need could be met by the nearby Lord Byng covered pool. Instead of being a source of p u b l i c i t y i t i s suspected that the pool would create antipathy among the public, since what i s being constructed i s peripheral to the University's function. Since i t i s not possible to cover the Empire pool, i t i s better to have a covered aquatic center. The center w i l l be used i n lagge part for academic and research purposes by a s i g n i f i c a n t cross section of the University community. 166 20. There are already too many v i s i t o r s who are allowed to take up parking space that should be reserved for students, faculty and s t a f f ; the aquatic center, i f accessible to outside v i s i t o r s , w i l l make the parking problem more acute 20. Use of the center for swim meets w i l l be a good source of pub-l i c i t y for UBC. 21. It i s not certain that the Lord Byng pool w i l l be available for our a t h l e t s , we should, therefore, have our own covered pool. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0080252/manifest

Comment

Related Items