UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Multimedia as an enabling technology for enhancing organizational memory to support decision making Lim, Kai Hin 1996

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1996-091171.pdf [ 20.93MB ]
Metadata
JSON: 831-1.0087182.json
JSON-LD: 831-1.0087182-ld.json
RDF/XML (Pretty): 831-1.0087182-rdf.xml
RDF/JSON: 831-1.0087182-rdf.json
Turtle: 831-1.0087182-turtle.txt
N-Triples: 831-1.0087182-rdf-ntriples.txt
Original Record: 831-1.0087182-source.json
Full Text
831-1.0087182-fulltext.txt
Citation
831-1.0087182.ris

Full Text

M U L T I M E D I A AS A N E N A B L I N G T E C H N O L O G Y FOR E N H A N C I N G ORGANIZATIONAL M E M O R Y TO SUPPORT DECISION M A K I N G By Kai Hin Lim B . A . , The University of Minnesota, 1988 M . A . , The University of Nebraska, 1991 A THESIS SUBMITTED IN PARTIAL F U L F I L M E N T OF T H E REQUIREMENTS FOR THE D E G R E E OF DOCTOR OF PHILOSOPHY in T H E F A C U L T Y OF C O M M E R C E A N D BUSINESS ADMINISTRATION We accept this thesis as conforming to the required standard T H E UNIVERSITY OF BRITISH C O L U M B I A February 1996 © Kai Hin Lim, 1996 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholariy purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. a Department of f ^ « l i f % Cy^CSL The University of British Columbia Vancouver, Canada DE-6 (2/88) A B S T R A C T The decision making literature has portrayed human decision making as imprecise, relying too heavily on unproven heuristics, and full of shortcomings. This dissertation focuses on three of the difficulties of human decision making that can be assisted by multimedia technology. They are: poor comprehension of information, coping with ambiguous information, and first impression bias. Poor comprehension of information is the inability of a decision maker to comprehend important details when large amount of information needs to be considered (Kahneman, 1973). Coping with ambiguous information is caused by the lack of a complete understanding of ambiguous information (Weick, 1979). First impression bias is caused by people's tendency to base their decisions on their initial impression of the event (Asch, 1946). We drew upon research from the multimedia literature and identified three unique characteristics of multimedia presentations, namely complementary cues, rich language, and authentic context. The theories from multimedia and decision making literatures were applied to the domain of Organization Memory Systems (OMS) to study how OMS can be enhanced to better support decision making. OMS are systems that store information from an organization's past to support present decisions. We suggest that through the use of complementary cues, rich language, and authentic context, multimedia is capable of storing and presenting more information in OMS, as compared to text-based OMS, in a way that may alleviate the three difficulties (poor comprehension of information, coping with ambiguous information, and first impression bias). Three experiments, which contrasted a text-based OMS to a multimedia OMS, were conducted. Each focused on one of the difficulties of decision making and investigated how multimedia presentations can be used to alleviate these difficulties. Results on comprehension of information demonstrate that the use of multimedia facilitates the retention and subsequent recall of organized facts, but not isolated facts. Further, better retention and recall of organized facts lead to a higher ability to make inferences. Results on coping with ambiguous information show a task-media fit relationship. For less-equivocal tasks, text-based OMS and multimedia OMS are equally effective in reducing perceived ambiguity levels, although text-based OMS required less time than multimedia OMS to perform the task. On the other hand, for more-equivocal tasks, only multimedia OMS was able to reduce perceived ambiguity levels. Multimedia OMS was also found to be useful in reducing more than text-based OMS, but not eliminating, the influence of first impression bias when information that is inconsistent with the first impression is presented in multimedia format. Overall, this dissertation demonstrated that multimedia can be used to further enhance the capability of the current text-based OMS to better support decision making. This is achieved through the ability of multimedia OMS to alleviate three of the difficulties of decision making discussed above. T A B L E O F C O N T E N T S A B S T R A C T i i T A B L E OF TABLES v i i i T A B L E OF FIGURES tx A C K N O W L E D G E M E N T S x i C H A P T E R 1. INTRODUCTION 1 1.1 Objectives 1 1.2 The Current Status 2 1.3 Shortcomings of Text-Based OMS 5 1.4 Multimedia 6 1.5 Research Questions and Scope of this Dissertation 7 1.6 Organization of this Dissertation 8 C H A P T E R 2. THEORETICAL B A C K G R O U N D 10 2.1 Introduction 10 2.2 Organizational Memory (OM) 10 2.3 Three of the Difficulties of Decision Making Associated with Processing Text-Based Information 13 2.3.1 Poor Comprehension of Information 13 2.3.2 Coping with Ambiguous Information 18 2.3.3 First Impression Bias 20 2.4 The Contributions of Multimedia to Alleviating the Three Difficulties 24 2.4.1 Characteristics of Interactive Multimedia 24 2.5 Summary 34 C H A P T E R 3. COMPREHENSION OF INFORMATION 40 3.1 Introduction 40 3.2 Overcoming Poor Comprehension 40 3.2.1 Cognitive Perspective 41 3.2.2 The Functional Perspective 44 3.3 Hypothesis 47 3.4 Research Method 49 3.4.1 Subject Population 49 3.4.2 The Experimental System 50 3.4.3 The Experimental Tasks 53 3.4.4 The Independent Variables . 54 3.4.5 The Dependent Variables 55 3.4.6 The Pilot Study 56 iv 3.4.7 Experimental Procedures 57 3.5 Results 59 3.5.1 Subject Profile . 59 3.5.2 Evaluation of Assumptions of Statistical Tests 60 3.5.3 Overall Fact Comprehension 61 3.5.4 Comprehension of Explanative and Descriptive Knowledge 62 3.5.5 The Relationships Among System Type, Explanative Knowledge, and Inference Ability 63 3.5.6 Summary of Results 65 3.6 Discussion 65 3.6.1 The two perspectives 66 3.6.2 Low R-Squared Values in PLS Analysis 68 3.6.3 Limitations 68 3.6.4 Conclusions for Experiment 1 70 C H A P T E R 4. COPING WITH AMBIGUOUS INFORMATION 72 4.1 Introduction 72 4.2 Task vs. Information 72 4.3 Reducing Ambiguity of information 76 4.4 Hypotheses 79 4.5 The Research Method 82 4.5.1 The Experimental Materials 82 4.5.2 Experimental Task 82 4.5.3 The Independent Variables 84 4.5.4 The Dependent Variables 86 4.6 Experimental Procedure 88 4.7 Results 90 4.7.1 Evaluation of Assumptions of Statistical Tests 90 4.7.2 Efficiency of Text-Based OMS for Less-Equivocal Task 91 4.7.3 Effectiveness (Ambiguity Reduction) 92 4.7.4 Perceived Usefulness of the System 96 4.7.5 Summary of Results 97 4.8 Discussion 97 C H A P T E R 5. FIRST IMPRESSION BIAS 102 5.1 Introduction . 102 5.2 Reducing the First Impression Bias 102 5.2.1 Reinterpreting Information 102 5.2.2 Ignoring Information 103 5.3 Hypotheses 103 5.4 The Research Method 105 5.4.1 The Experimental Materials 105 5.4.2 The Experimental Tasks 106 5.4.3 Experimental Manipulation 108 .v 5.4.4 The Independent Variables 110 5.4.5 The Dependent Variables 110 5.4.6 Experimental Procedure I l l 5.5 Results 113 5.5.1 Evaluation of Assumptions of Statistical Tests 113 5.5.2 Results on First Impression Bias 114 5.5.3 Other Supporting Results for First Impression Bias 117 5.5.4 Confidence with the Decision Made 124 5.5.5 Summary of Results 125 5.6 Discussion 126 5.6.1 Unexpected Results on Confidence with Decision Made 126 5.6.2 Practical Significance 127 5.6.3 Implications for Using Qualitative Information for Personnel Decisions 128 5.6.4 Limitations . 129 C H A P T E R 6. CONCLUSIONS 131 6.1 Introduction 131 6.2 Overview of Results 131 6.2.1 Comprehension Improvement 131 6.2.2 Ambiguity Reduction 132 6.2.3 First Impression Bias Reduction 133 6.2.4 Summary 135 6.3 Limitations 136 6.3.1 Subjects' Background and the Use of Subjects in Three Experiments 137 6.3.2 Realism of the Simulated System and Tasks 138 6.3.3 Quality of Multimedia Presentation 138 6.4 Theoretical Contributions of the Research 139 6.4.1 Contributions to OMS Research 139 6.4.2 Contributions to Organizational Communication Research 140 6.4.3 Contributions to Research in Psychology 141 6.5 Practical Contributions of the Research 141 6.6 Future Research 143 REFERENCES 148 T A B L E S FOR C H A P T E R 3 162 T A B L E S FOR C H A P T E R 4 170 T A B L E S FOR C H A P T E R 5 179 FIGURES FOR C H A P T E R 3 190 v i FIGURES FOR C H A P T E R 4 196 FIGURES FOR C H A P T E R 5 201 APPENDIX A . A LIST OF SHORTCOMINGS OF H U M A N DECISION M A K I N G . . 206 APPENDIX B. INSTRUCTIONS G I V E N TO R E S E A R C H ASSISTANTS 208 A P P E N D I X C. P R E - E X P E R I M E N T A L M A T E R I A L S 216 A P P E N D I X D. E X P E R I M E N T A L M A T E R I A L S FOR EXPERIMENT 1 220 A P P E N D I X E . E X P E R I M E N T A L M A T E R I A L S FOR EXPERIMENT 2 226 A P P E N D I X F. E X P E R I M E N T A L M A T E R I A L S FOR EXPERIMENT 3 240 A P P E N D I X G. POST- E X P E R I M E N T A L M A T E R I A L S 254 A P P E N D I X H . ORDER E F F E C T OF EXPERIMENT A N D T A S K 258 A P P E N D I X I R A N D O M I Z A T I O N CHECKS 261 A P P E N D I X J. H U M A N INFORMATION PROCESSING T H E O R Y 263 v i i T A B L E OF T A B L E S Table 3.1 Summary of Dependent Variables Investigated in Experiment 1 163 Table 3.2 Descriptive Statistics for Dependent Variables In Experiment 1 164 Table 3.3 Homogeneity of Variance Tests for Dependent Variables in Experiment 1 . . 165 Table 3.4 T-test on the Overall Fact Comprehension Score 166 Table 3.5 M A N O V A Results for Experiment 1 167 Table 3.6 Means (Standard Deviations) of Explanative and Descriptive Test Scores . . . 168 Table 3.7 Summary Table for the PLS Analysis 169 Table 4.1. Summary of the Dependent Variables 171 Table 4.2 Descriptive Statistics for Dependent Variables In Experiment 2 172 Table 4.3 Homogeneity of Variance Tests for Dependent Variables in Experiment 2 . . 173 Table 4.4a T-test on the Time Taken (in seconds) to Perform Less-Equivocal Task . . 174 Table 4.4b T-test on the Time Taken (in seconds) to Perform Equivocal Task 174 Table 4.5a Repeated Measures A N O V A - Between Subjects Effects for Perceived Ambiguity Reduction 175 Table 4.5b Repeated Measures A N O V A - Within Subject Effects for Perceived Ambiguity Reduction 175 Table 4.6 The Means (Standard Deviations) for Perceived Ambiguity Reduction 176 Table 4.7a The Means (Standard Deviations) for Perceived Ambiguity Level for the Less-Equivocal Task 177 Table 4.7b The Means (Standard Deviations) for Perceived Ambiguity Level for the Equivocal Task 177 Table 4.8 T-test on the Aggregated Score on Perceived Usefulness 178 • v i i i Table 5.1 Summary of the Dependent Variables 180 Table 5.2 Descriptive Statistics for Dependent Variables In Experiment 3 181 Table 5.3 Homogeneity of Variance Tests for Dependent Variables in Experiment 3 . . 182 Table 5.4a Repeated Measures A N O V A - Between Subjects Effects for Aggregated Appraisal Score 183 Table 5.4b Repeated Measures A N O V A - Within Subject Effects for Aggregated Appraisal Score 183 Table 5.5 The Mean (Standard Deviations) for the Appraisal Score 184 Table 5.6 One-Way A N O V A for the Final Appraisal Score 185 Table 5.7 The Means (Standard Deviations) for the Final Appraisal Score by the Four Experiment Conditions 186 Table 5.8 The Percentage of Subjects Who Recommended Renewal of Contract by the Four Experiment Conditions 187 Table 5.9 T-test on the Confidence with the Decision Made 188 Table 5.10a Repeated Measures A N O V A — Between Subjects Effects for Aggregated Score on Confidence with Decision Made 189 Table 5.10b Repeated Measures A N O V A - Within Subject Effects for Aggregated Score on Confidence with Decision Made 189 ix T A B L E O F F I G U R E S Figure 3.1 The Research Model Based on the Functional Perspective 191 Figure 3.2 A n Example of the OMS User Interface 192 Figure 3.3 The User Interface for the Multimedia OMS in Experiment 1 193 Figure 3.4 The User Interface for the Text-Based OMS in Experiment 1 194 Figure 3.5 The Results of the PLS Analysis 195 Figure 4.1 A Framework for Task Equivocality and Information Ambiguity 197 Figure 4.2 The User Interface for the Multimedia OMS in Experiment 2 198 Figure 4.3 The User Interface for the Text-Based OMS in Experiment 2 199 Figure 4.4 The System Usage Order * Task Type Interaction Effect on Perceived Ambiguity Reduction 200 Figure 5.1 The User Interface for the Multimedia OMS in Experiment 3 202 Figure 5.2 The User Interface for the Text-Based OMS in Experiment 3 203 Figure 5.3 A Schematic Representation of the Experimental Design for Experiment 3 . . 204 Figure 5.4 The System Type * Time Interaction Effect on Appraisal Outcomes 205 x A C K N O W L E D G E M E N T S I would like to express my gratitude to the following people for their roles in the completion of my dissertation: Izak Benbasat, my advisor, for guiding, challenging, and inspiring me. He forced me to think carefully about every sentence I wrote and helped me to stay focused. Izak is a model researcher. Lawrence Ward, for going beyond the role of a dissertation committee member. He broadened my views of scientific research and life in general. Bob Goldstein, for challenging my assumptions and research approach and helping me to remember the bigger picture. Ricki Goldman-Segall, for forcing me to think about alternative approaches to my research and broadening my perspective. Sharon Hartung, for conducting part of the experiment, tirelessly reading my drafts, and providing much needed emotional support. Fan Chung, for implementing the experimental system, conducting part of the experiment, and checking data. Darrell Jung, for implementing the time log capturing program in the experimental system. Gray Schwartz, for shooting the videos used in the experimental system and providing audio recording support during the experiment. Drs. Paul Rannie and Ralph Durand and their staff members at the British Columbia Cancer Research Centre, for generously providing the experimental materials and allowing me to interview them and video tape their research facilities. Linda Schwartz, for making this dissertation more readable. Special thanks to Kevin Au, Mike Chiasson, Amos Shachar, and Michelle Y i k for helping to shape my ideas in the early stages of this research endeavor. Thank you. x i One of philosophy's oldest paradoxes is the apparent contradiction between the great triumphs and the dramatic failures of the human mind. The same organism that routinely solves inferential problems too subtle and complex for the mightiest computers often makes errors in the simplest of judgments about everyday events (Nisbett and Ross, 1980). C H A P T E R 1. I N T R O D U C T I O N 1.1 Objectives The objectives of this dissertation are: 1) to build a theory-based understanding of three of the difficulties associated with decision making, due either to human information processing limitations or to the lack of adequate information, that can be aided by multimedia technology, 2) to develop theories on how the unique characteristics of multimedia presentation can be used to alleviate these difficulties, and 3) to empirically test the theories developed in a laboratory experiment using materials gathered from a real organization. Drawing from the decision making literature, this dissertation identifies three of the difficulties of human decision making associated with processing text-based information that can be aided by emerging technologies. They are: poor comprehension of information, coping with ambiguous information, and first impression bias. While these three are not the only difficulties in human decision making (see Nisbett and Ross, 1980 ; Kahneman, Slovic and Tversky, 1982 for reviews of others), we chose to focus on these three because we believed that emerging technology may alleviate them. However, this is not to suggest that these are the only three difficulties that can be aided by technology. We chose to focus on these three to make the research effort manageable. Appendix A provides a listing of the other difficulties of decision making that have been identified in the literature. Poor comprehension of information is the inability of a person to comprehend important 1 details when too much information needs to be considered (Kahneman, 1973). It is a direct result of cognitive overload. Coping with ambiguous information is caused by the lack of a complete understanding of ambiguous information when some details are missing (Weick, 1979). First impression bias is caused by people's tendency to base their decisions on their initial impression of the event, and ignore or reinterpret subsequent information to fit with that first impression (Asch, 1946). Specifically, this dissertation built on literature in cognitive and social psychology and organizational communication to examine the underlying processes involved in human decision making and to develop a theory-based understanding of three of the difficulties associated with decision making. We also drew upon research from the multimedia literature and identified three unique characteristics of multimedia presentations. The theories from multimedia literature and decision making research were integrated into the domain of Organizational Memory (OM) to study how Organizational Memory Systems (OMS) can be enhanced to better support decision making. OMS are systems that "stored information from an organization's history that can be brought to bear on present decisions" (Walsh and Ungson, 1991:61). We suggest that multimedia, through its unique characteristics, is capable of storing and presenting more information in OMS, as compared to text-based OMS, in a way that may alleviate the three difficulties and better support decision making. 1.2 The Current Status Consider the following typical organizational situations: Scenario 1 Poor Comprehension of Information Sue, VP for Finance, is having one of her typically busy days. It is 8:55am, and she is preparing 2 for a meeting scheduled at 9:00am. She scans through the meeting summary of the previous meeting. Although Sue's attention is focused on reading the meeting summary, she only managed to comprehend half of what was written. Scenario 2 Coping with Ambiguous Information The special task force on new product launching is having their weekly meeting. Mike, one of the task force members, did not attend the previous meeting as he was out of town. However, Mike is fully prepared ~ this morning, he made a special effort to carefully read the transcript of the previous meeting. He knows exactly what was said and by whom. Helen, who is in charge of public relations, is reporting to the task force members on her progress on negotiation with the Product Safety Board, a government agency. Helen: "These people are really a pain in the neck. I called them several times last week trying to find out the progress on testing of our product. Well, no one was available when I called. I left several messages asking them to return my call. Three days later, someone finally returned my call, and this guy told me that it would take another two months before they start testing our product. Based on the way it is going, I think we need to push our launching date at least a month forward . . . " Mike: "Sorry for interrupting, Helen. But, didn't you say in last week's meeting that you will get the Product Safety Board to test our stuff this week? Here, it's on Page 3 of last week's meeting transcript." Helen looks at the transcript Mike handed to her. The transcript wrote that Helen said: "Right, I will get them to work on it by next week." Helen looks at Mike and laughs: "Gee Mike, that's what I said but not what I meant. I was jokingly responding to Joan's comment. I wish you could see how Joan and I laughed when I said that" Mike: "Oh, I guess I misinterpreted what you said . . . " Scenario 3 First Impression Bias John, the product manager, and Mary, the purchasing manager, are discussing from whom they should buy their next order of bolts. Mary: "Company A offers to sell us 50,000 units for 20% of what we are currently paying." John: "What kind of bolts are they? Where do they make them?" Mary: "They are of the same specifications as those we have been using, but I think they are made in Korea" John: "Oh no, I don't trust Korean products. Remember? Couple years ago we had a disaster using screws made in Korea." Mary: "John, they submitted a quality test report from an independent lab. Here's a copy of the report." Instead of reading the report as he normally would, John scans through it searching for evidence that would support his argument for not buying from this supplier. The above scenarios illustrate three common difficulties of decision making: poor comprehension of information, coping with ambiguous information, and first impression bias. The decision making literature has portrayed human decision making as imprecise, 3 relying too heavily on unproven heuristics, and full of shortcomings (e.g., Nisbett and Ross, 1980; Kahneman, Slovic and Tversky, 1982). In part, the shortcomings of human decision making are caused by a limited cognitive ability to process information. Information systems researchers have begun to recognize the importance of studying and understanding these shortcomings in human decision making to design Information Systems (IS) that are more effective in supporting decision making (Jarvenpaa, 1989; L im L . H . , 1995; and Todd and Benbasat, 1991; 1992). For the most part, IS have been very effective in supporting structured organizational decision making tasks, such as inventory control. However, higher-level organizational decision making tasks, such as strategic planning, are often unstructured, involve sifting through and interpreting ambiguous, ill-defined data. In such decision tasks, decision makers often do not have clear guidelines of what information is needed to perform the tasks. Such decision tasks are called equivocal decision tasks (Daft and Weick, 1984). Unfortunately, IS support for equivocal decision tasks has been less available and ineffective, though such decision tasks are often more vulnerable to the shortcomings of human judgment. On the other hand, as indicated in a recent Gartner Group Strategic Analysis Report, between 80% to 90% of organizational information is in documents rather than structured databases (Popkin and Cushman, 1993 as cited in Sprague, in press). These include, but are not limited to, contracts and agreements, reports, manuals and handbooks, business forms, correspondence, memos, drawings, blueprints, photographs, email and voice mail messages, video clips, script and visuals from presentations, and transcripts from meetings (Sprague, in press). Up until recently, computers could only store and process 10% to 20% of organizational 4 information (i.e., highly structured information, such as databases). It is no surprise that the remaining 80% to 90% of organizational information has been left untouched by computers. Unfortunately, such information is often the key to more equivocal, higher-level organizational decisions. This is also why IS support for higher-level decision tasks has been less available. This research suggests that multimedia technology creates a new avenue for providing IS support for such tasks. 1.3 Shortcomings of Text-Based O M S One emerging class of IS that has the potential to support equivocal decision making is Organizational Memory Systems (OMS). OMS moves beyond the limitations of IS by capturing a wide variety of organizational information, such as meeting transcripts, organizational members' know-how, and informal organizational rules and procedures. However, current OMS, which captures information using textual and numeric data (e.g., Sandoe et al's. (1991) meeting memory systems), is still inadequate for supporting highly equivocal decisions. For example, a textual summary of a meeting only provides the essential points discussed and the consensus reached. It is difficult to use a textual summary to capture seemingly subtle, yet important information, such as the dynamics of decision making, the heated arguments, and the unwilling compromises made. This contextual information is often a crucial part of equivocal decision making. At a more specific level, the limitations of text-based OMS stem from the fact that they store and present information using numbers and text, which contributes to the three difficulties of decision making mentioned above. First, comprehending textual information is a cognitively 5 intensive task. It reduces the availability of limited cognitive resources for other processes such as thinking. Second, text filters out much of the rich contextual information in social situations and tasks, thus may result in misinterpretation of the original meaning. Third, information presented using such a format is not very attention-catching and may not be noticed by a decision maker. This may contribute to first impression bias, because some information may be ignored. To effectively support decision making, an OMS must overcome or reduce the effects of these three difficulties. 1.4 Multimedia Multimedia technology presents an opportunity to build OMS that could address these three difficulties and better support decision making. Based on the findings of multimedia research, we identify several characteristics of multimedia presentations that have the potential to alleviate three of the difficulties of decision making. Broadly speaking, a multimedia computer system is one that is capable of the input or output of more than one medium, such as video, audio, text, and graphic (Dannenberg and Blattner, 1992). In recent years, multimedia technology has become a multimillion dollar industry, and it is growing at an phenomenal rate. It has made its way into the movie industry, home entertainment, and education. However, most of the research in multimedia has been conducted within the domain of education. With the exception of Rao and Goldman-Segall (1995), very little effort has been devoted to exploring its potential contributions in organizational settings. Rao and Goldman-Segall proposed using video as a means of capturing "stories" to preserve organizational memory. According to them, a story is "any free-form account of a connected 6 series of happenings (p. 333)". Stories also include reports, newsletters, memos and so on. They argued that stories provide a contextual shell for making sense of key components in many organizational interactions. They further argued that stories captured in textual format may not be adequate in many situations. A rich medium, such as video, can provide a more representative description of the situation. Building on Rao and Goldman-Segall's argument, this research empirically examines the benefits of multimedia OMS. We integrate the decision making literature and multimedia research into the domain of O M and generate several predictions about the benefits of multimedia OMS. These predictions will be tested through a laboratory experiment using a multimedia OMS implemented on a world-wide-web (web) server. 1.5 Research Questions and Scope of this Dissertation At the general level, the primary research question of this dissertation is: How can multimedia technology be used to enhance the capability of text-based OMS? At a more specific level, the research questions are, compared to text-based OMS: 1) Can multimedia presentation in OMS improve retention and recall of organizational information? If yes, under what condition (s)? 2) Can multimedia OMS reduce the perceived ambiguity level of information stored in OMS for various types of decision tasks? 3) Can multimedia presentation in OMS reduce the influence of first impression bias? At the outset, it is important to mention that this research does not address the technical 7 aspects of multimedia production and implementation. Neither does the research address the ethical issues associated with preserving and maintaining multimedia information. Further, as it will be made clear in the Theory chapter (Chapter 2), the implementation of an experimental OMS for this research was constrained by a set of real-life protocols as found in organizational situations. Also, it must be emphasized that the focus of this study is to investigate the usefulness of multimedia in enhancing text-based OMS for supporting decision making and how multimedia OMS can be used to alleviate three of the difficulties of decision making. This focus necessarily steers our discussions more toward the advantages of multimedia OMS than those of text-based OMS. However, we do not claim that multimedia OMS is useful for all organizational decision making tasks. In fact, as will be clarified in later discussion, the theoretical perspectives developed suggest a media-task fit hypothesis (i.e., under certain situations, as found in this research, multimedia OMS is better than text-based OMS, but under some other situations, text-based OMS out-performs multimedia OMS). 1.6 Organization of this Dissertation This dissertation is organized as follows. Chapter 2 reviews the literature on the following: 1) organizational memory systems, 2) three of the difficulties of decision making associated with using text-based OMS, and 3) multimedia. The review provides a theoretical foundation for developing specific theories in the subsequent three chapters, each related to one of the three difficulties discussed. Chapter 3 focuses on the first difficulty, poor comprehension of information. Two 8 theoretical perspectives, which provide different predictions regarding what situations multimedia presentation can improve comprehension, are contrasted. An experiment, which was designed to test the predictions generated based on these two perspectives, is described, and the results are reported. Chapter 4 investigates the second difficulty, coping with ambiguous information. Drawing from organizational communication research, a task-media fit hypothesis is developed and discussed. A n experiment, which tested the task-media fit framework developed, is also discussed. The results are presented and discussed within the framework developed. Chapter 5 concentrates on the third difficulty, first impression bias. A theoretical perspective, which explains how multimedia can be used to reduce the influence of first impression bias, is developed and discussed. Specific predictions are generated, and an experiment, which aimed at testing the predictions generated, is described. The results are delineated and interpreted within the theoretical perspective developed. Chapter 6 presents a summary of the three studies, points out the limitations of the dissertation, assesses the contributions of this effort, and discusses possible areas of extensions. 9 CHAPTER 2. THEORETICAL BACKGROUND 2.1 Introduction The purpose of this chapter is to provide an overview of the literature on 1) the three difficulties of human decision making associated with text-based information processing, 2) multimedia presentation, and 3) organizational memory systems (OMS). Through this review, we attempted to synthesize the theories and findings of multimedia and decision making literatures and apply them to OMS research. The review in this chapter serves as the foundation for developing the theories for each of the three experiments that will be discussed in Chapters 3, 4, and 5. The chapter begins by introducing a class of emerging technology, called Organizational Memory Systems (OMS), and explains why enhancements to text-based OMS are needed to support better decision making. Next, research related to each of the three difficulties of human decision making associated with processing text-based information is discussed. Subsequently, the three characteristics of multimedia presentation are reviewed to examine how multimedia can be used to alleviate the three difficulties associated with processing text-based information. 2.2 Organizational Memory (OM) Organizational Memory is "stored information from an organization's history that can be brought to bear on present decisions" (Walsh and Ungson, 1991:61). O M is a key element supporting organizational decision making. Organizations need past information to support day-to-day operations, to model and predict the future, to serve as a basis of change, and to avoid 10 repeating mistakes (Schwabe, 1994). The overall goal of Organizational Memory Systems (OMS) is to capture organizational past information to support future decision making (El Saway, Games, and Gonzalez, 1986; Stein and Zwass, 1995). Organizational memory is thought to be stored in several forms which include: individual members' memory (Argyris and Schon, 1978), computer-resident information, such as databases (Huber, 1990), accepted procedures and routines (Cyert and March, 1963), and even cultural artifacts, such as dress codes (Smith and Steadman, 1981). Of these, individuals are believed to be the dominant mode of storage of organizational memory. Individuals not only store organizational information and events in their memory, but also keep records and files as memory aids (Walsh and Ungson, 1991). Huber (1990) points out three difficulties associated with managing O M : 1) membership attrition; 2) information not stored, or not stored in easily retrievable ways; and 3) the existence and location of information are not always known to organization members (i.e., do we have this information and who has this information?). Traditional Information Systems (IS) are limited to capturing O M in the forms of structured information such as databases. However, 80% to 90% of organizational information is in documents that cannot be captured by structured databases (Popkin and Cushman, 1993 as cited in Sprague, in press). These include, but are not limited to, contracts and agreements, reports, manuals and handbooks, business forms, correspondence, memos, drawings, blueprints, photographs, email and voice mail messages, video clips, script and visuals from presentations, and transcripts from meetings (Sprague, in press). 11 OMS moves beyond the limitation of traditional IS by attempting to capture the rest of the 80% to 90% of organizational information. For example, Sandoe et al. (1991) describe a meeting memory system that is capable of capturing meeting discussions and generating meeting reports. Ackerman's (1994) Answering Garden records and organizes individual members' expertise to form a large pool of organizational knowledge. Morrison (1993) developed a team memory system for an on-going project or business team by enabling team members to access items, such as past meeting documentation, team membership and member information, team work-in-progress, and other data related to the project. These systems expand the boundary of traditional IS by creating central repositories of a wide variety of organizational information, including detailed meeting information, organizational members' know how, and informal rules and procedures. OMS encompasses a wide variety of information and has the potential to support a wide range of organizational decision tasks. However, current OMS are still severely limited. At a more general level, higher-level organizational decision making is often unstructured, equivocal and highly uncertain. Information relevant to such decision tasks cannot be packaged into a structured format for computer storage without simplifying, summarizing, or abstracting the information. Simplification, summary, or abstraction entails capturing those portions thought to be important and omitting others. While it might be perfectly fine to keep only the essentials under some situations, in other situations, what seems to be unimportant now may become critical for other situations in the future. Therefore, simplification may result in loss of important information. Consequently, decision makers may not have the necessary information to make 12 the decision effectively. At a more specific level; the difficulties of currently operational OMS stem from the fact that they store and present information using numbers and text. First, comprehending textual information is a relatively effort-intensive task. It reduces the amount of limited cognitive resources available for other processes such as thinking. Second, text often is a relatively "lean" medium. It filters out much of the rich contextual information in social situations and tasks, and thus may result in misinterpretation of the original meaning. Third, information presented using such a format is not very attention-catching and may not be noticed by the decision maker. This may contribute to first impression bias, because some information may be ignored. The three above mentioned difficulties of decision making associated with processing text-based information will be discussed in detail in the following sections. To better understand these difficulties, it is necessary to have a basic understanding of how humans process information and how information is stored in memory. Readers who are unfamiliar with the human information processing theory are referred to Appendix J in page 262. 2.3 Three of the Difficulties of Decision Making Associated with Processing Text-Based Information 2.3.1 Poor Comprehension of Information The most predominant information representation for organizational decision making is text and numbers. This includes formal documents, computer outputs, letters, memos, reports, and meeting summaries or transcripts. Further, as reported in Panko's (1992) survey, managers spend more than 60% of their time in meetings (including conference room, two-person, and 13 telephone meetings). Most meetings last for more than one session, with days or weeks of delay between sessions (Kinney and Panko, 1995). Based on this, to effectively perform their job (to prepare for meetings or decision making), managers may have to read great number of meeting transcripts and summaries. However, as illustrated by Scenario 2 in the Introduction chapter, decision makers do not have the capacity to fully comprehend the text-based information they read. This is because reading is a time consuming and cognitively intensive task. Further, decision makers may still forget the information after reading the materials. They would rather rely on their recollection of the information from their own memory, even though the recall process may induce errors in the information. Therefore, in approaching the problem of poor comprehension of information, this dissertation focuses on the processing effort involved in assimilating text-based information. 2.3.1.1 A Model for Text-Based Information Processing There are many models for text-based information processing (Anderson, 1976; Crothers, 1979; Norman and Rumelhart, 1975; Schank, 1975; Kintsch and Van Dijk, 1983; Van Dijk and Kintsch, 1978). One of the implicit assumptions of these models is that human's ability to process information is limited by the capacity of the small working memory, and that processing text-based information is a cognitive intensive task. Of these models, the most widely adopted one is the one proposed by Van Dijk and Kintsch (1978; 1983). In this model, text comprehension is assumed to consist of two levels of processing. At the surface level is the verbatim representation of text fragments or speech-like representation of text, called "microstructure". At the higher level is the hierarchical structure 14 that reflects the meaning of elements that are extracted from the microstructure. This hierarchical structure is called "macrostructure" and is more commonly known as the "gist" or the "theme" of a text (Van Dijk and Kintsch, 1983). As a person reads a text, he/she can work with only a small part of the text at a time. The reader carries along a small portion of the macrostructure in working memory. As new information is read, if the reader understands this information and sees a relationship between this information and the macrostructure in his/her working memory, the new text is connected to the macrostructure. If new incoming information cannot be connected to the macrostructure in the working memory, a cognitive-intensive long-term memory search is made to determine whether or not there is a macrostructure already stored, by which the new information can be related to the hierarchy. Such a search can be successful or unsuccessful. In either case, the search causes complex cognitive processing and may still result in comprehension failure (Baggett, 1989). Beside the inherent difficultly in processing text-based information, three particular characteristics of human information processing further constrain a reader's ability to comprehend text-based information. These three characteristics are: 1) cue selection (selectively attending to part of the information and filtering out the others); 2) verbal recoding (transforming printed words into speech-based code before processing); and 3) information reconstruction (filling in missing details using background knowledge). 2.3.1.2 Cue Selection In organizational decision making, often the amount of information that needs to be 15 considered or examined far exceeds a decision maker's cognitive capacity. Nevertheless, every piece of information may be potentially important for the decision. Todd and Benbasat (1991; 1992) through a series of empirical studies found that even with a decision aid, decision makers would adopt a cognitive effort minimizing strategy when the available information exceeded their cognitive capacity. Similarly, when reading a printed text, readers often selectively skim through the text to get a quick overview of the content. As a result, important information may be ignored and not considered in making the decision. Ideally, information should be presented in ways such that fewer cognitive resources are required to process it so that more information can be processed and important details are not ignored. 2.3.1.3 Verbal Recoding Empirical evidence suggests that in processing text-based information, people transform printed words into verbal code before further processing (Hardyck and Petrinovitch, 1970; Kleiman, 1975; Levy, 1977; and Makita, 1968), especially when there are large amounts of information to be processed. In most situations, this verbal recoding process occurs automatically. However, when the information is difficult to understand, the process requires the use of limited cognitive resources. As such, fewer resources are available for other processes, such as comprehension and thinking. 2.3.1.4 Information Reconstruction Information reconstruction also has important implications for textual information processing. In comprehending from spoken language, people rely heavily on what many linguistic scholars refer to as "prosodic" features of the auditory message. Prosodic features 16 include pitch, intonation, stress, and pauses that occur in spoken language. Prosodic features are crucial element of comprehension. For example, pitch tells us what the speaker wants us to do: carry out a command or answer a question. Stress tells us which words (and hence, concepts) the speaker wants us to regard as most important in focussing our attention (Pearson and Fielding, 1982). Although bold and underline can be used in written representation, they are inconvenient, distracting, and not commonly used. Further, not all prosodic cues can be translated into bold or underline. In formal written documents, where the writer has carefully constructed the sentences, prosodic features may not be important in comprehension. However, in some situations (e.g., a written transcript of a spontaneous speech such as those generated in meetings), prosodic features are important elements in helping understanding. Three possibilities exist when a reader infers prosodic features. First, the reader attempts to infer the missing prosodic cues, and succeeds. This necessarily means that extra processing effort has been used in the process. Second, the reader attempts to infer the prosodic patterns, but fails. In this case, extra processing effort has been used but does not lead to better understanding. In the first two cases, since extra processing effort is needed, this further consumes the limited capacity of working memory and may slow down the processing of other information. As a result, the reader may not have sufficient time to comprehend other information that comes later. The third possibility is that the reader ignores the missing prosodic features and relies solely on the information provided in the written text. This, of course, leads to poor comprehension of the information. In essence, all three possibilities lead to poor comprehension of information. 17 In summary, comprehending written materials is a cognitive effort-intensive task. First, readers need to recode the written representation into verbal code. Second, the prosodic patterns, which are. filtered out in written representations, have to be inferred to fully understanding the material. Both processes further reduce the limited cognitive capacity. Because of this reduced cognitive capacity, important information or details may be ignored. Finally, as described by Van Dijk and Kintsch's (1983) model, a reader has to build a macrostructure from a lower-level microstructure of the text, after which, the macrostructure developed has to be connected to the existing knowledge structure either in the working memory or long-term memory for retention. A l l of these processes are cognitively intensive. Comprehension failure can occur at any one of the above described stages. The key to improving comprehension is to present information in such a way that fewer cognitive resources are required so that more information can be processed and important details are not ignored. 2.3.2 Coping with Ambiguous Information As pointed out by Karl Weick (1979), human social systems are extremely complex with many ill-defined situations. While some issues are orderly and well-defined, many others are ambiguous and unstructured. Similarly, many organizational decision tasks are equivocal in that there are no clear guidelines on how to solve the problem, and what information is needed to perform the tasks. On the other hand, the information available may be ambiguous and subject to multiple interpretations. While in some situations, the consequence of misinterpreting information may not be serious, in others situations, misinterpretation of information can have enormous consequences. 18 For example, Trevino, Lengel and Daft (1987) suggested that the 1986 Challenger space shuttle disaster might have been caused by a misinterpretation of information. They claim that it was possible that the engineers' intuitive feelings and strength of emotion about the safety of the o-ring in the solid rocket were not properly conveyed, and hence misinterpreted by the ground commander who gave the final approval for launching. When attempting to make sense out of ambiguous information, decision makers rely heavily on subjective feelings about and interpretation of the situation (Weick, 1979). Therefore, to construct a better understanding and interpretation of the situation, decision makers require a rich and comprehensive view of the contextual information to base their interpretation1. As an example, consider the performance appraisal of a department head. Quantitative information may show that the overall productivity level of the department has been low, and that two employees have resigned during the department head's tenure. However, one can easily come up with several plausible reasons for this situation. Without further information, such as the management style of the department head, it is difficult to conclude whether this low productivity and turnover can be attributed to dissatisfaction with the department head. One possible way to understand the management style of the department head is to ask co-workers. Another possible way is to examine the past behavior of this individual. For example, information such as video clips of departmental meetings may reveal much of his/her management style. Unfortunately, textual information, such as meeting transcripts, filters out the rich The lack of contextual information can be seen as data-limited processing in Norman and Bobrow's (1975) characterization of information processing limitations. 19 contextual information and non-verbal cues required for constructing a full understanding of the situation. Further, information presented on the transcript can become highly ambiguous. Decision makers are left with their own imagination to fill in the missing contextual information, which are often subject to multiple interpretations. In short, while some types of information are concrete, concise and unambiguous, they may only provide a superficial-level understanding. Under a highly equivocal decision task, the rich contextual information behind such superficial-level information is needed to provide a complete understanding of the problem and to effectively support better decision making. This contextual information is more ambiguous in nature, because it can be subjected to multiple interpretations. To effectively support equivocal decision tasks, information has to be captured and presented in ways that the rich and comprehensive context around the information is preserved so that decision makers are less prone to misinterpret ambiguous information. 2.3.3 F irst Impression Bias Research shows that people's opinions about an event or impressions about a person are very much determined by the first piece of information about the event or person to which they are exposed2. In his classic study, Asch (1946) told his subjects that they would hear a set of qualities that belonged to a particular person and that they should try to form an impression of the target person. Next, he gave two groups of subjects two alternative descriptions of a hypothetical person: List 1: "intelligent, industrious, impulsive, critical, stubborn and envious". 2 First impression bias is a somewhat unclear term. First information bias better characterizes this limitation. However, since the literature has adopted the former, it is used in this study. 20 List 2: "envious, stubborn, critical, impulsive, industrious, intelligent". Notice that List 2 contained exactly the same adjectives as List 1, but in reverse order. List 1 begins with two positive qualities {intelligent, industrious), proceeds to three qualities that could be interpreted as either good or bad {impulsive, critical, stubborn), and ends with a negative quality (envious). List 2 begins with a negative quality and ends with two positive qualities. Subsequently, subjects were asked to rate the target person using a check list consisting of 18 trait contrasts (e.g., generous-ungenerous, wise-shrewd). Subjects who were given List 1 (positive quality first) rated the target person far more positively than those subjects who were given List 2 (negative quality first). Subjects were also asked to write a short character sketch that fitted the traits heard. Subjects who were given List 1 (positive quality first) appeared to interpret the traits "impulsive", "stubborn", and "critical" as "spontaneous", "decisive", and "discriminating". In contrast, subjects who were given List 2 (negative quality first) interpreted the same two traits as "reckless", "obstinate", and "fault-finding". Asch's study provides clear evidence that the information received first has a disproportionately large effect on judgement. However, this study was criticized on the grounds that the judgement of a few adjectives was far from making a real-world decision. Nonetheless, several subsequent studies examined this phenomenon using more realistic situations and consistently obtained the same results (e.g., Jones et al., 1968; Luchins, 1957). For example, in Jones, Rock, Shaver, Goethals, and Ward's (1968) study, subjects observed a person taking a test consisting of 30 items. In one condition, the person started off very well, correctly 21 answered almost all questions, but his performance deteriorated in the second half of the test. In the other condition, exactly the opposite occurred: the person started off very badly, but improved dramatically in the second half of the test. At the end, both persons performed equally well, each correctly answering 15 of the 30 questions. However, subjects rated the person who started off well as more intelligent, and predicted they would perform better in a subsequent test, than the person who started off slowly. These studies demonstrate that people formulate a strong opinion based on their first impression and that people are inherently biased in evaluating subsequent information. If subsequent information is consistent with their first impression, the first impression will be further enhanced. On the other hand, if subsequent information is inconsistent with the original impression, people will attempt to reinterpret the new information to fit the original impression. If this reinterpretation attempt fails, people will ignore the new information by considering it as an anomaly. Only after too many unexplainable inconsistencies will people actually start questioning the validity of the original impression. First impression bias has been examined and shown to consistently occur in various situations (Fiske and Taylor, 1991; Jones and Goethals, 1972; Lana, 1964). In fact, first impression bias reflects one of the major principles underlying perceptual organization: consistency. People often organize their perceptions of others so that all of the elements fit together into a coherent, logical pattern (Markus, 1977; Wyer, 1974). At times the need for consistency may blind the observer to various facts or events. Although first impression bias can occur in many situations, it becomes more severe when the decision maker is dealing with text4jased information. As was mentioned, the two 22 underlying processes involved in first impression bias are reinterpreting information and ignoring inconsistencies. First, text filters out much of the semantically rich information. As a result, the information may be more easily misinterpreted or reinterpreted. Second, information presented using text is not very attention-catching and may not be noticed by the decision maker. This may also contribute to first impression bias, because inconsistencies may be easily ignored. First impression bias can become especially serious in equivocal decision making situations. In dealing with any unstructured or nonroutine task (without a well-defined procedure), a decision maker usually starts by examining the information that he/she believes is most important. This first piece of information often becomes the basis of the decision, no matter what information is examined subsequently. This is because under equivocal decision making situations, much of the information is ambiguous by nature; it can be interpreted in different ways. With a strong first impression, decision makers will reinterpret other information to fit their first impressions. In summary, first impression bias occurs when the decision outcome is heavily influenced by the first piece of information that a decision maker examines. This happens because the initially formed impression guides a decision maker's subsequent information processing behavior. In a more favorable situation, in which this first piece of information provides a true portrait of the event, the decision made may not be negatively affected. However, it is likely that this first piece of information may only reflect the partial truth of the event. The resulting bias may have serious consequences for decisions. 23 2.4 The Contributions of Multimedia to Alleviating the Three Difficulties 2.4.1 Characteristics of Interactive Multimedia In general, multimedia can be characterized by three dimensions: 1) the existence of multiple, complementary cues; 2) the range of symbol systems available to convey information, which we call rich language; and 3) the potential to represent information in authentic context to facilitate easy and complete understanding of the meaning surrounding the information (Kozma, 1991). While theoretically it is possible to characterize a medium along these three dimensions, practically, it is difficult to distinguish clearly among these three in a given implementation. Multimedia systems simultaneously provide high availability of complementary cues, high language richness, and preserve the authentic context of information. 2.4.1.1 Complementary Cues Multiple representation of information in various formats has been found to lead to better retention and retrieval (e.g., Paivio, 1971). There is considerable evidence suggesting that information in long-term memory may be stored both semantically (meaning) and pictorially (images) (e.g., Kosslyn, 1980; Paivio, 1971; Shepard and Cooper, 1982). Paivio's (1971) dual-coding theory presents the central core of this argument. Briefly, the theory posits that there are at least two distinct modes of representation in memory, a verbal mode and a visual or imagery mode. The two systems are interconnected in that the information in each system can 24 be cross-referenced3. Empirical evidence suggests that imagery instructions improve memorability and aid recall (e.g., Bell, 1991; Kosslyn, 1980). Like semantic information, images stored in long-term memory can be retrieved into working memory in response to stimuli in the environment. At first glance, it appears that multiple representations, such as presenting auditory and visual information simultaneously, may cause interference, since these two sources of information may compete for our limited cognitive resources (Kahneman, 1973). Research suggests otherwise. Simultaneous presentation of congruent information in different modalities does not result in competition for cognitive resources, instead, they complement each other. For example, empirical findings in audio and visual presentations (e.g., Baggett and Ehrenfeucht, 1982; 1983; Beagles-Roos and Gat, 1983; Hayes and Kelly, 1984; Hayes, Kelly and Mandel, 1986) indicate that people can construct a mental representation of the information from either audio or visual information alone. However, when presented together, each source provides additional complementary information that retains some of the characteristics of the symbol systems of origin. The complementary effect of multiple representation on learning has also been studied extensively. For example, Levie and Lentz (1982) found that congruent pictures paired with text generally improve reading comprehension over either text alone or pictures alone. It is thought that during the retention stage, multiple representations of an object or event increase the number of possible links between information to be stored and existing knowledge in long-term memory. As a result, the information is better retained. 3 It should be added that there has been considerable debate on Paivio's notion of two memory systems. Other theorists, mainly from the Artificial Intelligence camp (e.g., Anderson and Bower, 1973; Pylyshyn, 1973), have argued for a single memory representation. 25 On the other hand, during the retrieval stage, multiple representations increase the chance of retrieval by providing more cues (stimuli). For example, one may not recognize the voice of a high school classmate whom one has not seen for 25 years. Neither can one recognize a recent photograph of this classmate. Nonetheless, when both the voice and the picture of the person are presented simultaneously, the likelihood of recall may improve substantially. In summary, there is substantial empirical evidence suggesting that complementary cues do not compete for limited cognitive resources. Instead, they increase the chances of retention and better understanding by providing more potential links with existing knowledge. They improve subsequent recall by providing more retrieval cues. 2.4.1.2 Rich Language Every symbol system has it strengths and limitations. For example, while conventional wisdom suggests that a picture is worth ten thousand words (Larkin and Simon, 1987), pictures cannot convey the meaning of conditional events or causes, such as if, nevertheless, because, or no. This is because pictures do not entail any logical connections (Salomon, 1979). On the other hand, as Hansen (1989:8) argues, "many human experiences are encoded in symbolic systems that resist logic descriptions, at least, they are not handy to communicate when translated into verbal (or mathematical) descriptions." By combining the computer with the video medium, multimedia technology brings together the symbolic and processing capabilities of various media and thus creates a new and rich symbol system of communication. It is believed that visual information is represented differently in memory as compared to linguistically presented information. Baggett (1989) suggests that visual representations contain more information and are bushier. She uses the example of "red leaf" to illustrate her 26 argument. The phrase "red leaf" when stored in memory contains only the name of an object and a modifier. On the other hand, a mental representation of a red leaf formulated from a picture would contain information about its size, color, and shape. A rich representation is particularly important in conveying information related to social situations and tasks. In studying communication patterns among organizational members, Daft, Lengel and Trevino (1987) found that "lean" media, such as electronic-mail, are very ineffective for tasks that are ambiguous and unstructured. Managers rely heavily on "rich" media, such as real-time face-to-face interactions to perform such tasks (also see McGrath and Hollingshead, 1993). This is because rich media convey nonverbal messages using a richer set of symbol systems. Research in nonverbal communication suggests that nonverbal messages are not simply alternatives to using language. One key difference between nonverbal and verbal language is that nonverbal messages tend to be much more efficient in transmitting attitudes and emotions than is language (Ekman and Friesen, 1974). Argyle and his colleagues (1970; 1971) found that when verbal messages are contradicted by nonverbal messages, the nonverbal message has more influence over people's judgements. Similarly, in capturing O M , video information would preserve the embedded social and interpersonal cues in complex, unstructured organizational activities relevant to decision making. Consider a job interview. The verbal transcript of an interview only provides a surface-level interpretation of what was said because it filters out some important information, such as facial expression, body language, tone of the voice, and the overall appearance of the candidate. Such information is critical to the decision making process, because it often adds extra meaning 27 to what was said. A video record of the interview provides a richer representation of the event than a verbal transcript of the same event. Based on the video, decision makers who did not attend the actual interview session can more easily construct their understanding and interpretation of that candidate, and hence better decisions can be made. In summary, each medium has its inherent strengths and weaknesses. For example, while pictures cannot convey the meaning of conditional events or causes (e.g., if and then), they are excellent in providing rich representations of situations. Rich representations have several advantages. First, they minimize the loss of important information. Second, they provide more potential links to existing information in long-term memory. Finally, rich representations of situations preserve the semantically rich information, such as nonverbal cues, in social interactions. 2.4.1.3 Authentic Context Research has shown that an effective way of creating understanding of events or information is to formulate mental pictures of situations (Johnson-Laird, 1983; Van Dijk and Kintsch, 1983). People construct their mental pictures by dynamically combining the current situation (the information presented) with their existing knowledge in long-term memory (Lim K . H . , Ward and Benbasat, 1995). In less complex or more structured situations people can formulate mental pictures without much difficulty. For example, most people have no difficulty constructing a mental picture of a typical classroom. On the other hand, for people who have never seen a computer-mediated group decision support room before, it is very difficult to form a mental picture of the room, unless a very detailed and lengthy verbal description is given. However, a picture of such a room, with some verbal descriptions, would easily overcome this 28 difficulty. To further illustrate the difficulty associated with constructing mental pictures in novel situations, consider an experiment conducted by the Cognitive Technology Group at Vanderbilt University (1993a). In one situation, young children were told the story of a bird who is unable to drink from a glass of water because its beak is not long enough. The children were further told that the intelligent bird put pebbles into the glass so that the water level would rise. The researchers found that the children had great difficulty imagining the situation unless they were given dynamic, visual support. It is easier for people to relate to familiar situations than unfamiliar ones. This is why people often use real-world representations to form mental pictures of situations. Therefore, when the information presented matches the real-world phenomenon, people can form mental pictures more easily. The importance of authenticity in presenting information cannot be overemphasized in learning. Within the education literature, both Situated Cognition Theory (Brown, Collins and Duguid, 1989) and Anchored Instruction Theory (Cognition and Technology Group at Vanderbilt, 1990) stress the importance of embedding the learning of facts and abstract concepts in meaningful activities related to those facts. Both theories suggest that knowledge and context are inextricably tied. Early research (e.g., Levie and Lentz, 1982; Pressley, 1977; Schallert, 1980; Stone and Glock, 1981), comparing pictures and text, concluded that pictures provide the required information to construct mental representations of the real world. This is because pictorial symbol systems share more properties with the corresponding objects and events in the real 29 world than do linguistic symbol systems4. This result should be generalizable to video, because video images share even more properties with objects and events in the real world. Researchers of the Cognitive Technology Group at Vanderbilt (1993b: 122) argued that "video allows a more veridical representation of events than text." Video is dynamic, visual, and spatial. Therefore, it allows people to form mental pictures of problem situations more easily. Park and Hannafin (1993) argued that context helps to establish connections between the information presented, the learner's existing knowledge in memory, and the environment. Therefore, "it nests meaning not in isolated or fragmented activities or descriptions, but in the events and phenomena in which they naturally occur." (p. 75) In short, when information is presented in its naturally occurring context, people need not extrapolate meaning artificially. Instead, meaning is directly perceived from the phenomenon. Therefore, information presented in authentic context reduces the requirement on cognitive effort and the chances of misinterpreting the context. Further, information presented in authentic context also implies that rich representation has been employed. Thus, it also enjoys the advantages of rich representations mentioned previously, i.e., loss of important information is minimized, more links to existing information in long-term memory are formed, and the rich semantics of social interactions are preserved. Most importantly, authentic context around the information helps to eliminate ambiguities, and thus reduces the chances of misinterpretation. With video, multimedia systems bear extraordinary capabilities to create or retain the natural contexts of events (Hannafin, 1992). 4 Perhaps with the exception of Chinese, Egyptian, and other pictorial written languages. 30 2.4.1.4 Summary on the Unique Characteristics of Multimedia Presentation Paivio's (1977) dual-coding theory posits that there are separate representations for verbal and visual information. Empirical evidence also suggest that retention and recall of information can be greatly enhanced when the information is presented in multiple formats. This is because multiple representations create more potential links to knowledge in long-term memory, and thus improve retention and recall of information. By using a set of rich symbol systems, multimedia not only conveys more information, but also combines the strengths of each medium. Specifically, rich language system is more capable of communicating the rich semantic involved in social interactions. For example, nonverbal messages are better conveyed using a richer representation, such as video or picture, and thereby the meaning can be better understood in a less ambiguous way. Finally, multimedia preserves the authentic context surrounding the information. Similar to the use of rich language, the preservation of authentic context provides additional information which helps to clarify the information being conveyed. 2.4.1.5 Interactivity While interactivity is not an unique characteristic of multimedia, it is inseparable from computer-based multimedia system implementation. Further, interactivity complements the other unique characteristics of multimedia and makes it possible to reduce the effects of the three difficulties discussed. In the three experiments that will be discussed in Chapters 3, 4, and 5, interactivity is explicitly controlled when comparing text-based OMS and multimedia OMS. Interactivity refers to the amount of control that the decision maker has in determining the range of content to be reviewed, the depth of exploration, and the time spent on each piece 31 of information. Linear video and interactive video present the two extreme examples of this dimension. In linear video, presentation sequence is predetermined, rather than being sensitive to the individual requirements of decision makers. Accordingly, the major drawback of linear presentation is that it may force decision makers through information without the necessary ongoing comprehension. In contrast, interactive video allows decision makers to access different parts of video information at wil l . For example, in using an interactive video system, a decision maker can skip through the parts that she remembers and directly reach the needed materials using a content search. The same process can only be achieved using the fast-forward and rewind features in a linear video system. This is not only time consuming, but also requires the decision maker to remember or know the approximate location of the needed information. Besides the obvious advantage that the access sequence can be controlled by the decision maker, there are several theoretically-based rationales for providing high interactivity. First, random access of information is compatible with the nonlinear processing nature of human mind (Cognitive and Technology Group at Vanderbilt, 1993b). As it was reviewed previously, Anderson's (1976) A C T views retrieval of information from long-term memory as a random process. The same stimulus given at different times may trigger different paths of retrieval. One theory in learning, Cognitive Flexibility Theory (Spiro et al., 1988; Spiro and Feltovich, 1991; Spiro and Jehng, 1990), suggests that in ill-structured, complex situations, knowledge needs to be assembled from representations (i.e., information presented), rather than retrieved directly from memory. The theory further suggests that such situations or tasks require that information be 32 represented along multiple rather than single conceptual dimensions to facilitate the "assembly" process. This is because, in ill-structured and complex situations, information related to the decision is often ambiguous (Daft and Macintosh, 1981; Weick, 1979). To gain a comprehensive understanding of the situation or task, decision makers need to examine the information from multiple perspectives. Spiro and his associates (Spiro et al., 1988; Spiro and Feltovich, 1991; Spiro and Jehng, 1990) argue that hypertext5 facilitates "cognitive flexibility" because it allows an issue to be explored in multiple ways using several different concepts or themes. As a result, a flexible and integrated knowledge structure is developed, which is more adaptive for a wide range of problems. Another advantage of the interactive system is related to the issue of individual control. A fundamental assumption of the linear model of information presentation is that decision makers process information passively based on what is given. There is an abundance of empirical evidence suggesting that people perform more effectively when they are allowed to process information or engage in learning in an active way (e.g., Carroll and Mack, 1984; 1985). As was mentioned, passive viewing of information (especially when there is a large amount of information to be examined) leads to "seeing without comprehending". Interactivity allows decision makers to engage in the viewing process in a more active way. As such, decision makers are more attentive to the information, leading to better comprehension. In summary, interactive systems appear to have several advantages over the linear mode of information presentation. First, decision makers can decide what and when they need to 5 Hypertext is a type of textual information storage and retrieval system that allows nonlinear access to stored information. 33 examine a piece of information based on their need. Second, this flexible form of information retrieval is more accommodating to the way humans process information. In an ill-structured situation, people do not simply retrieve information from memory. To interpret and make sense of the situation, decision makers assemble the given information with existing knowledge. Finally, interactivity leads decision makers to be more attentive to the information, and thus information is better comprehended. 2.5 Summary Three difficulties of current Organizational Memory Systems (OMS), which use numbers and text to store organizational memory, have been identified and explained. The first difficulty, poor comprehension of information, suggests that the stages involved in comprehending text-based information are cognitively intensive. Comprehension failure can occur at any stage. The second difficulty, coping with ambiguous information, relates to the fact that higher-level organizational decision tasks are often unstructured and highly equivocal. In such decision tasks, there are no clear guidelines on what information is needed. Decision makers need context-rich information to come up with an interpretation of the situation. Such context-rich information is often ambiguous and subject to multiple interpretations, and thus cannot be easily conveyed using text. The third difficulty, first impression bias, argues that the first piece of information that the decision maker is exposed to has a disproportionately large influence on the final decision. Text-based information is not very attention-catching and is subject to multiple interpretations. When the subsequent inconsistent information presented in text is unconsciously ignored or 34 reinterpreted by the decision maker, the bias would persist. Three characteristics of multimedia presentation have been identified. First, multimedia presentation uses complementary cues (e.g., text with picture) to present information. The use of complementary cues has been shown empirically to facilitate better retention and recall of information. Second, multimedia presentation uses a set of rich languages to convey information. These rich languages can better preserve the semantically-rich information needed for higher-level organizational decision tasks. Third, multimedia presentation preserves the authentic context surrounding the information. We suggest that through the three unique characteristics, multimedia presentation may alleviated the three difficulties of decision making associated with processing text-based information. First, research in human memory suggests that there are at least two systems of representation in memory, a verbal system and a visual system. Further, the information in each system can be cross-referenced (Paivio, 1971). Therefore, information presented in multiple formats (e.g., verbally and visually) can enhance retention and subsequent recall. Accordingly, the use of complementary cues in multimedia presentation may enhance comprehension. Also by combining several media, multimedia is capable of using rich symbol language to convey more information, at the same time, preserve important details without much distortion. With these two characteristics, multimedia may provide an avenue for improving comprehension. Second, to effectively support equivocal decision tasks, information has to be captured and presented in ways that the rich and comprehensive context around the information is preserved so that decision makers are less prone to misinterpret ambiguous information. Text-based information filters out much of the needed rich contextual information, and thus is 35 inadequate for supporting such decision tasks. Through the use of rich language, multimedia is capable of capturing the semantically-rich information in organizational interactions. Also, multimedia is capable of preserving the authentic context surrounding the information, making the original meaning of the information clearer and less ambiguous. Through these two characteristics, multimedia presentation may reduce the ambiguity level of information, and thus help decision makers to better cope with ambiguous information. Finally, to effectively overcome first impression bias, the two underlying cognitive processes in first impression bias, ignoring inconsistencies and reinterpreting information, have to be suppressed. Through the use of complementary cues, multimedia presents information in a way that information is better registered in decision maker's mind. This may make inconsistent information more vivid and more difficult to ignore. Also, using rich language, multimedia can convey more context-related information, such as nonverbal message, which may make information less ambiguous and less subject to misinterpretation. Through these two characteristic, multimedia presentation may be capable of reducing the influence of first impression bias. In the next three chapters, three experiments, which contrast a text-based OMS to a multimedia OMS, will be described. Briefly, the two OMS are equivalent in terms of their verbal information content. The multimedia OMS contained video clips of interviews. The text-based OMS replaced the video clips with text transcripts of the interviews. The transcripts only contained the verbal information of the video clips. Nonverbal information was not included in the transcripts, as would be the case for conventional meeting transcripts. Therefore, the two 36 systems are not informationally equivalent in terms of nonverbal information6; the multimedia OMS contains more information than the text-based OMS. It is necessary to point out that the two OMS contrasted only represent two specific cases of many possible implementations. We chose to focus on these two because they are more ecologically valid and yet provide clean experimental control. These two systems are more ecologically valid in that they resemble two most common and likely ways by which organizational memory is captured. The transcripts contained in the text-based OMS are similar to those of the conventional meeting transcripts used for documenting an organizational meeting, in which no nonverbal information is included. Note that it is possible to add commentary, such as "John asked sarcastically when the project would be finished". However, there are two possible problems to this approach. First, the commentaries added would only represent the interpretation of the person who wrote the transcript. As such, they could be inaccurate or even biased. Second, adding commentaries is not a common way of documenting organizational information. Also, it is possible to include charts and graphs in the text-based OMS. However, by doing so the line between multimedia and text-based OMS becomes fuzzy making it impossible to experimentally test the effects of the three unique characteristics of multimedia on the three difficulties. Similarly, for the multimedia OMS, additional features, such as animation and special video editing effects, could have been used. However, without seeing the benefits of a 6 According to Larkin and Simon, two systems or representations are informationally equivalent when "all of the information in the one is also inferable from the other, and vice versa (1987, pg 67)." 37 multimedia system, organizations are not likely to invest in such additional features. On the other hand, video recording of meetings or events are more common in organizations, especially with the increasing popularity of video conferencing. In essence, these two specific implementations of OMS were chosen for the following reasons: First, each of the two forms is the most basic, yet representative, case in its respective class (multimedia or text-based). Second, they either represent the way (text) or the most likely and convenient way (video) in the near future that real-world organizations would document information. The belief is that if the predicted relationships and benefits are realized in such basic multimedia OMS implementations, we can be confident that the relationships should be stronger if more comprehensive multimedia OMS were used in the experiments. However, it should be made clear that the multimedia OMS used in the experiment is more than a "talking head" version of the text-based OMS. While we did not add extra features to the multimedia OMS, the use of video and audio naturally included complementary cues, rich language, and authentic context in the presentation. A an example, one specific piece of information contained in the two OMS is the organizational structure of the British Columbia Cancer Research Centre (BCCRC), the background organization used in the three experiments. The multimedia OMS presented this information by showing a video clip of the Director of the Research Centre explaining the organizational structure of B C C R C . In the video clip, the Director drew an organizational chart while explaining the structure of the organization. In this case, a rich language (drawing) was employed together with the use of complementary cues (verbal descriptions with visual representation). On the other hand, the text-based OMS only contained what the Director said verbally, i.e., with a leaner language and no complementary 38 cues. The two OMS contrasted are summarized in Table 2.1 in terms of the three characteristics of multimedia presentation. Each of the following three chapters deals with one of the three difficulties discussed. The review provided in this chapter will serve as the foundation for building the theoretical perspective in each of the following three chapters. In each of these chapters, the unique characteristics of multimedia will be more closely examined to build a theory-based explanation of how multimedia presentation can be used to alleviate each of the three difficulties discussed. The ultimate goal is to investigate how multimedia presentation can be used to enhance the capability of current OMS to better support decision making. 39 CHAPTER 3. COMPREHENSION OF INFORMATION 3.1 Introduction Poor comprehension of information, as discussed in Chapter 2, is one of the difficulties of decision making. This chapter investigates how multimedia can be used to enhance the capability of text-based OMS to improve comprehension of information. Two theoretical perspectives are advanced based on the theories outlined in Chapter 2. Next, the hypotheses developed from these two perspectives are discussed, and an experiment, which was conducted to test the hypotheses, is described. Lastly, the results are presented and discussed. 3.2 Overcoming Poor Comprehension Text is the predominant form of organizational information. However, comprehending text-based information is cognitively taxing, because it involves additional cognitive processes on the part of readers. First, readers of a written text have to recode the written representation presented into verbal coding. Second, for the transcripts of speeches or conversations, to create a better understanding, readers must attempt to infer the prosodic cues, such as pitch and stress, that are absent in written representation. Comprehension failure can occur at any one of these stages. In essence, the cause of poor comprehension can be attributed to the mismatch between 1) the limited capacity of human working memory, and 2) the cognitive demand of processing text-based information. Since the limitation of working memory is beyond our control, the key to improving comprehension lies on how to present information so that fewer cognitive resources 40 are needed. The potential benefits of multimedia OMS can be viewed from two theoretical perspectives, the cognitive and functional perspectives. Although these two perspectives share the general view that multimedia information is easier to comprehend than text-based information, they are very different in terms of explaining the underlying mechanism involved in processing multimedia information. As a result, they provide very different predictions regrading under what conditions multimedia is beneficial. 3.2.1 Cognitive Perspective From the cognitive perspective, the benefits of multimedia presentation are derived directly from the reduction in cognitive load when processing multimedia information. First, multimedia systems allow the playback of audio (or with video) of the original speech. Listening to the original speech reduces the cognitive load of the decision maker. Decision makers do not undergo the verbal recoding process, as happens with reading written text. Second, the prosodic features are completely preserved in audio/video presentation, eliminating the need to infer them. The elimination of these two processes reduces the cognitive requirement in processing information, and allows cognitive effort to be devoted to thinking and relating information to knowledge in long-term memory. Consequently, overall comprehension is improved. Third, embedded complementary cues (redundancies) are presented in the video information. As discussed in Chapter 2, information presented using complementary cues creates more potential links to a decision maker's existing knowledge structure. As such the information is better retained and recalled. 41 Most important, multimedia presented information contains imagery (picture, graphic, video scene etc.), together with text or verbal description. There is considerable evidence in the field of both cognitive psychology and reading comprehension to support the argument that imagery enhances language comprehension. As was discussed in Chapter 2, Paivio's (1971) dual coding theory presents the central core of this argument. Briefly, the theory posits that there are at least two distinct modes of representation in memory, a verbal mode and a visual or imagery mode. The two systems are interconnected in that the information in each system can be cross-referenced7. Nonetheless, empirical studies comparing the use of television/video presentation and traditional textbooks consistently show that students learn better by reading text, (see Kozma, 1991 for review). While this appears to contradict the argument put forth by the cognitive perspective, a more careful look at the differences between textbook and conventional television/video presentations is necessary to understand this contradiction. The advantages of reading over watching/listening are derived from two sources. First, video presentation is transient. In conventional video presentation, information is delivered linearly at a fixed rate. Viewing video forces the viewer to absorb the information at the rate at which the speaker delivers the information, whereas the rate of reading a text is controlled by the reader (Danks, 1980; Sticht, 1972). Unlike readers, viewers (or listeners) do not have the option of reviewing the information, at least not in conventional video8 presentation. 7 It should be added that there has been considerable debate on Paivio's notion of two memory systems. Other theorists, mainly from the Artificial Intelligence camp (e.g., Anderson and Bower, 1973; Pylyshyn, 1973), have argued for a single memory representation. 8 In a conventional video presentation, viewers do not have control over the presentation. Viewing proceeds in a linear fashion from the begin to the end of the presentation as designed by the 42 Second, without giving viewers any control of the presentation, conventional video presentation treats viewers as passive information receivers. The viewer's active attention level in television viewing is low, hence results in a low level of comprehension. This "cognitive inertia" phenomenon is well-recognized in education research (e.g., Salmon, 1979). Therefore, the advantages of using audio/visual input over reading written transcripts of speech may be discounted by the transient and passive nature of video or audio presentations. Research suggests that one way to overcome these counteracting effects is to allow the viewer to have more control over the presentation (Kinzie, 1991). The interactive nature of multimedia provides such control for viewers. An interactive multimedia OMS allows its viewers to access information nonlinearly. Thus, viewers engage in more active and attentive viewing, which has been shown to improve comprehension and facilitate better construction of knowledge representations. The use of multimedia alone is a necessary but not sufficient condition. The full potential benefits of a multimedia presentation cannot be realized without an interactive feature. In summary, information presented using an interactive multimedia system should be better comprehended because such presentation requires less cognitive effort to process. Furthermore, the use of imagery and complementary cues in multimedia presentations allow information to be better retained and recalled. However, the drawback of conventional audio and video presentation, in which the decision maker has no control over the presentation speed and content, is that it may lead to a cognitive inertia problem. The provision of interactivity allows the decision maker to examine information in a more active and attentive manner, thus producer of the video. 43 largely eliminates the cognitive inertia problem. The net effect is improved comprehension of information with less cognitive effort. 3.2.2 The Functional Perspective In contrast to the cognitive perspective, the functional perspective suggests that multimedia presentation may not be beneficial in all situations. Mayer (1985) proposed a general framework for studying text-based information learning. The framework can be generalized to other contexts, including learning from information presented using multimedia. One of Mayer's key predictions of the framework is that the way information is organized and presented only affects the retention and recall of a certain kind of knowledge. Following Mayer, this study differentiates between descriptive knowledge, explanative knowledge, and inference ability. Descriptive knowledge refers to knowledge in which the key components or events are described but not explained, such as a list of facts, a series of events, or a set of procedural instructions. In contrast, explanative knowledge refers to knowledge in which the functional relationships among two or more variables are expressed, and the underlying functional rules or mechanisms are explained. For example, the ability to explain Ohm's Law in terms of electron flow, or the knowledge about the relationship between two positions or departments in an organization. Mayer used the term true understanding to refer to the ability to solve problems that are different from those the subject has previously learned. He argued that understanding is a level higher than recall. True understanding means that in addition to knowing the basic explanative information presented, a learner must be able to put this information together in a new way and apply it to a new context. We labelled this ability as inference ability. AA Based on Mayer's (1985) framework, the effect of multimedia presentation on descriptive knowledge, explanative knowledge, and inference ability can be summarized in the model depicted in Figure 3.1 (see page 189). The model predicts that the way information is being conveyed (via different types of systems, e.g., multimedia vs. text-based OMS) directly affects a decision maker's ability to encode and recall explanative knowledge (Path " A " in Figure 3.1) but not descriptive knowledge (Path "B" in Figure 3.1). In turn, a decision maker's ability to encode and recall explanative knowledge affects his/her ability to apply the knowledge to novel situations, i.e., making inferences (Path "C" in Figure 3.1). The model also suggests that system type does not directly affect the decision maker's potential ability to make inferences (Path "D" in Figure 3.1). Rather, system type indirectly affects inference ability by organizing and presenting information in such a way that it becomes easier to encode and recall explanative knowledge. Finally, the acquisition of descriptive knowledge does not affect the ability to make inferences (Path "E" in Figure 3.1). Mayer (1979) provided empirical support for the relationships described. He summarized over a dozen studies that involved learning scientific prose with the aid of an "advance organizer". A n advance organizer is a visual model that depicts or conceptualizes the main components discussed in the text. It is used to help readers organize information provided in the text. For example, a hierarchical organizational chart would be an advanced organizer for a text passage that explains the structure of an organization and the functional relationships among individual departments and positions in that organization. Notice that what Mayer called an advance organizer is in fact one form of a multimedia presentation. Through the summary, Mayer concluded that the use of an advance organizer increases recall of explanative idea units 45 (but not descriptive idea units) and increases performance on creative problem solving (i.e., making inferences). The positive relationship between the use of an advance organizer and the decision maker's ability to retain and recall explanative knowledge can be attributed to the role that the advance organizer plays. A n advance organizer focuses the decision maker's attention on the explanative information and provides a mechanism to help him/her integrate the information. With an advance organizer, rather than creating his/her own mental model, the decision maker is provided with a direct representation of the components or events. The provision of mental (or conceptual) models has been shown to improve learning (e.g., Davis and Bostrom, 1993; Kieras and Bovair, 1984; Sein and Bostrom, 1989). In studying how people learn to use a computer system, Lim K . H . , Ward, and Benbasat (1995) observed similar outcomes. They found that subjects who focused on the surface-level activities, such as detailed physical actions concerning moving icons around, did not perform well when they were asked to make inferences about novel situations. On the other hand, subjects who focused on the higher-level goals, such as relating lower-level actions to higher-level goals, performed much better in the same inference test. L im, K . H . , Ward and Benbasat's (1995) classification of surface-level vs. higher-level focus is similar to Mayer's classification of descriptive vs. explanative knowledge. They argued that a higher-level focus (explanative knowledge) causes learners to develop mental models that have higher inference potential. In summary, the functional perspective suggests multimedia may not be beneficial in all situations. While multimedia presentation facilitates the retention and recall of explanative knowledge, it does not affect the retention and recall of descriptive knowledge. Presumably, 46 this is because multimedia provides decision makers with the required mental (conceptual) models to better integrate the functional relationships among the components or events involved. Further, the acquisition of explanative knowledge directly affects the ability to apply the information learned to novel situations. 3.3 Hypothesis Two theoretical perspectives, the cognitive and functional perspectives, on how multimedia presentation could improve a decision maker's ability to better comprehend information, were delineated in Section 3.2. To recapitulate, the cognitive perspective views reading text-based information as a cognitively taxing task because decision makers have to recode written representation into verbal coding and infer the missing prosodic cues. On the other hand, Multimedia OMS is capable of capturing and playing back original speech. This eliminates the need for verbal recoding and inferring prosodic cues and reduces the cognitive requirement in processing such information. Further, multimedia OMS uses imagery and complementary cues extensively. Both of these have been shown to improve recall of information in educational research with grade-school children (see Kozma, 1991 for a review). Therefore, based on the cognitive perspective, we expect that: HI: Subjects who use multimedia OMS will score higher in a comprehension test, which requires them to recall information presented, than those who use text-based OMS. In contrast, the functional perspective proposes that the multimedia presentation may not 47 be beneficial to all situations. The functional perspective predicts that multimedia facilitates the encoding and recall of explanative knowledge but not descriptive knowledge (Mayer, 1985). In the experiment, which will be discussed in detail in Section 3.4, a reading comprehension test was used to examine the difference between descriptive and explanative knowledge acquired. The test consisted of three parts, a descriptive test, an explanative test, and an inference test. The descriptive test required subjects to recall facts that were directly presented (i.e., descriptive knowledge). In contrast, the explanative test required subjects to answer questions that involved understanding and explaining the functional relationships among/between the ideas presented (i.e., explanative knowledge). Based on the functional perspective, the following two hypotheses were formulated: H2a: Subjects who use multimedia OMS will score higher in an explanative test than those who use text-based OMS. H2b: Subjects who use multimedia OMS will score equally well in a descriptive test as compared to those who use text-based OMS. The third part of the comprehension test mentioned above was the inference test. It was used to examine the effect of multimedia presentation on higher-level understanding and the ability to draw inferences using the information presented. This test asked questions whose answers were not explicitly stated in the information presented. To correctly answer these questions, subjects had to make inferences based on the information they had examined. 48 Following Mayer, we expected that subjects who scored high in the explanative test (presumably as a result of using the multimedia OMS) would also perform well in this inference test. In other words, the type of system used does not directly affect the score in the inference test. Rather, the level of explanative knowledge acquired mediates the relationship between system type and the inference ability. Based on this, the following two hypotheses were formulated: H3a: The use of multimedia OMS will not directly contribute to a higher score in the inference test. H3b: Subjects who score high in the explanative test will also score high in the inference test. In summary, two perspectives, the cognitive and functional perspectives, were contrasted. The cognitive perspective predicted that multimedia OMS subjects would out-perform the text-based subjects in all fact-based comprehension tests. In contrast, the functional perspective predicted that multimedia would be beneficial only for retention and recall of explanative knowledge but not descriptive knowledge. The functional perspective further predicted that the score on the fact-based test would mediate the relationship between system type and outcome of the inference test. 3.4 Research Method 3.4.1 Subject Population A l l subjects were recruited from the student population of the University of British Columbia. Participation in the experiments was entirely voluntary. Subjects were recruited 49 through campus advertisements. They were provided with three incentives. First, they were paid $20 for approximately two and a half hours of participation (time required to complete Experiments 1,2, and 3). Second, an additional $30 was offered for each of the three best performers. Third, after their participation, all subjects were entitled to attend a free, one-hour world-wide-web (web) tutorial. The recruitment advertisements specified that participants must know how to operate a mouse input device. This was to avoid spending additional training time. Participants were not required to have experience on using the web. 3.4.2 The Experimental System 3.4.2.1 The Technical Implementation The OMS used in the experiments was designed explicitly for this dissertation research. It was implemented on an IBM compatible PC running a 90 M H z Pentium processor with a N E C 15" Multisync 4FGe monitor. The system was implemented as a simulated web server. Subjects used Netscape (version 1.1) to browse the experimental materials stored in the OMS. A l l experimental materials were stored on a 850 MByte hard disk. With this configuration, retrieval of information from the system, including video clips, was almost instantaneous. The video clips were digitized using Intel's Smart Video Recorder Pro (1995) at a frame output rate of 20 frames per second with screen resolution set at 340 X 240 pixels. To maintain a high resolution, the size of the video frames were made a little smaller than a quarter of the screen. Since most of the clips were interviews, this frame rate provided a fairly smooth replay quality. Although we could have chosen to digitize the video clips at a higher resolution, 340 X 240 pixels was chosen because this setting gives a reasonable image quality at a significantly 50 lower storage requirement. In the experiments, subjects sat approximately two feet away from the monitor. With the 15" N E C high-resolution monitor employed, subjects could clearly see the facial expressions of the people in the video clips. A l l video clips were digitized and stored in A V I format, a video standard supported by Microsoft. No further software or hardware compression was used, because the play-back speed of uncompressed videos is faster than that of compressed ones. Storage space was traded-off for speed to simulate a real-time OMS that could retrieve all kinds of information instantaneously, including video clips. Figure 3.2 (see page 190) shows an example of the user interface. Subjects could examine any piece of information at any time by clicking on the underlined and highlighted text (as shown in the figure), which simulates the hypertext link concept found on most web browsers. Although the system could actually be made accessible externally through the internet, the access of the system was limited to the experimental room because the current data transfer rate on the internet prohibits mnning real-time videos. The system was set-up to automatically record a time log for each subject. The log included the time spent on examining each piece of information (each page on the web). The time logs were generated by a program called Winbatch (version 5.0, 1995). A log file was automatically created each time the research assistant initialized an experimental session. Subsequently, during the course of the experiments, every time a selection was made by the subject (via clicking on the hypertext link), Winbatch would record the current time and the title of the selected information into a log file. The log file closed after the entire experimental session had ended. Therefore, each log file only contained recorded data pertaining to an 51 individual subject. 3.4.2.2 The Experimental Materials To ensure mundane realism and the generalizability of the findings, information from a real organization9 was selected. The British Columbia Cancer Research Centre (BCCRC) was used as the background case in this study. The materials used in Experiment 1 included: 1) Interview with the Director of BCCRC on various aspects of the organization, such as its organizational structure and decision making processes; 2) General introduction to the six departments of the Research Centre; and 3) Interviews with several researchers and department heads of the Research Centre. These interviews were divided into smaller segments based on the specific topic discussed. Transition text was developed to link these segments together to create "pages" on the web. Depending on the experimental condition assigned, subjects used one of the two versions of the OMS (text or multimedia). Subjects in the multimedia OMS group viewed information in real-time, full-motion video. For example, to learn about the organizational structure of British Columbia Research Centre, subjects could simply click on a hypertext link that said "organizational structure". A video clip that showed the Director of the Research Centre explaining the organizational structure would be presented. Subjects using the text-based OMS viewed the same information in text form using the same hypertext link access method. The information was the same in terms of verbal content, the only difference was the medium, namely a written transcript. The user interface used was also identical for the two systems. Figure 3.3 (see page 191) 9 Some information was altered to safeguard confidentiality. 52 is an example of the user interface in the multimedia OMS, showing a video clip playing after a hypertext link was selected in a previous screen. Figure 3.4 (see page 192) shows the corresponding user interface in the text-based OMS when the same selection was made. The written transcripts were prepared by a professional transcriber. The transcripts only included the spoken words, as would be the case for ordinary meeting transcripts. A n independent research assistant proofread the transcripts with the aid of the actual video clips. 3.4.3 The Experimental Tasks At the beginning of the experimental session, subjects were instructed to perform three tasks in the role of a newly appointed board member of the British Columbia Cancer Research Centre (BCCRC). Since they were new to the organization, their first task was to become familiar with the British Columbia Cancer Research Centre. The focus of Experiment 1 was on investigating the potential effect of multimedia on improving 1) fact comprehension and 2) the ability to draw inferences based on the facts learned. However, ambiguity of information, which is the focus of Experiment 2, can also result in poor comprehension of information. To isolate this possible confounding effect, no ambiguous information was used in Task 1. That is, all information was stated as fact for Task 1. Before starting, subjects were informed that they would be tested on their level of understanding of the B C C R C after they had examined the information. The instructions for Task 1 are included in Appendix D l . Subjects either used the text-based OMS or the multimedia OMS, depending on random 53 experimental assignment10, to examine the information stored in the OMS which provided an overview of the B C C R C . This included clarifying the relationship between B C C R C and its two sister organizations (British Columbia Cancer Agency and British Columbia Cancer Foundation), illustrating the organizational structure, describing the mission of B C C R C , and explaining the functions of each of the six departments within the B C C R C . In both conditions, subjects were given 20 minutes to browse through the information. Through pilot testing, it was concluded that 20 minutes was sufficient for this purpose. Using Netscape, subjects had the option to probe various aspects of the organization. For example, subjects could find out more about the mission of the Research Centre by "asking" the Director of B C C R C (via clicking the hypertext link which said "Mission of BCCRC"). Subjects could also "visit" a particular department of interest (by clicking on a hypertext link that has the name of that department). At the end of the 20 minutes, subjects were instructed to stop. They were then given a comprehension test, which consisted of 15 questions, shown in Appendix D2. Subjects were not allowed to refer back to information in the OMS when answering the questions. 3.4.4 The Independent Variables System type was the only independent variable for Experiment 1. Two systems, text-based OMS and multimedia OMS, were contrasted in the experiment. The two systems were equivalent in terms of verbal information content and user interface. The only difference was To ensure that the random assignment was sucessful, two t-tests and one one-way ANOVA were used to compare the experiment conditions on five variables: age, gender, year of studies, major field of studies, and language background. No statistical significant differences were found across the experimental conditions on the five variables. The results of the tests are reported in Appendix I. 54 that the multimedia OMS presented some information in the form of video clips; text-based OMS presented the same information using text transcripts. 3.4.5 The Dependent Variables There were four dependent variables in Experiment 1, comprehension of explanative knowledge, comprehension of descriptive knowledge, Overall fact comprehension, and inference ability. 1) Comprehension of explanative knowledge: Four questions which directly asked about the functional relationships of ideas presented were used to measure explanative knowledge. They are Questions 1, 2, 5, and 6 in the Comprehension Test in Appendix D2. 2) Comprehension of descriptive knowledge: Four fact-based questions were included as the descriptive test. These questions required subjects to recall facts that were explicitly mentioned in the information presented. These are Questions 3, 4, 7, and 8 in the Comprehension Test (Appendix D2). 3) Overall fact comprehension: This is the sum of score on explanative test and score on descriptive test. 4) Inference ability: Seven inference questions were included in the Comprehension Test (Questions 9-15 in Appendix D2). These questions required subjects to infer information that was not previously learned. A pretest was conducted to ensure that subjects were able to answer all the questions in the comprehension test with the information provided in the OMS. Two subjects, different from those who participated in the actual experiment, were given the test and asked to search for the answers to the questions. One of them used the text-based OMS, and the other used the 55 multimedia OMS. No time limit was placed. Both of them correctly answered all 15 questions in the test. This indicated that both systems contained the necessary information to answer the questions in the comprehension test, and that the two systems were truly equivalent in terms of information content. 3.4.6 The Pilot Study Sixteen subjects went through the pilot phase of the experiments. The allotted time for Experiment 1 was determined through the pilot study. Several modifications were made after the pilot phase. It was noticed that the entire study (including Experiments 2 and 3) took approximately three hours to complete. This was excessively long. It resulted in subjects becoming very tired toward the end of the session. Since this might affect the validity of the experimental results, it was decided to shorten the experimental tasks. First, the original open-ended questions in the comprehension test were transformed into multiple choice-type questions. This substantially reduced the time required to complete the test. Secondly, we deleted some of the information that was originally stored in the OMS to cut down the time needed to examine all the information. The change from the open-ended test to a multiple choice test was the only one that may have some consequences and thus warrant some discussion. The major concern for using a multiple choice test was that the questions might be too easy to answer. Subjects might have answered the questions correctly by guessing. In transforming the open-ended test to a multiple choice test, steps were taken to ensure that this possible problem was minimized. To discourage random guessing, for the descriptive and explanative parts of the comprehension test, subjects were told before hand that a quarter of a point would be deducted from their score for each 56 wrong answer. They were explicitly asked to leave the question blank if they could not remember the correct answer. Also, for each question, the answer choices were made similar enough that it was difficult to guess without knowing the answer. 3.4.7 Experimental Procedures The experiments were conducted one subject at a time in an isolated room. A l l experimental sessions, with the exception of the pilot study, were conducted by two research assistants (one research assistant at a time), trained together by the researcher. The training materials for the research assistants are attached in Appendix B. They were asked to follow the experimental protocols given without deviation. Also, they were required to take note of any irregularities during the experimental session, including any odd behaviors on the part of subjects that might have invalidated the data collected (e.g., not paying attention). The research assistants were not told the objectives and the hypotheses of the study until the data collection phase was completed. 3.4.7.1 The Pre-Experimental Session Upon arrival, subjects were first greeted and thanked for their participation and interest. They were then asked to sit in front of the computer. Next, they were given the Consent Form (Appendix CI) and asked to carefully read through it before signing it. After collecting the signed Consent Form, the research assistant handed out an instruction sheet, labelled General Description (see Appendix C2). This instruction sheet informed the subjects that they would be required to perform three tasks, and that in one of the three tasks11, they were required to think-aloud, i.e., verbalize their thought processes. A n exercise was also included in this instruction. 1 1 In Experiment 2. 57 Subjects were required to "think aloud" in this exercise. Next, a 10-minute tutorial on web and Netscape was introduced. This tutorial was self-paced, but the research assistant stood next to subjects to answer any questions. The tutorial is attached in Appendix C3. The entire pre-experimental session took about 15 minutes. 3.4.7.2 The Experimental Session Subjects were given the first instruction sheet for Task 1 (labelled as Task Description la in Appendix D l ) . While subjects were reading the instruction, the research assistant keyed in the pre-designated subject identification number in the computer and selected the appropriate experimental condition and task as assigned previously. Subjects were told to click an "OK" button whenever they were ready to start. The research assistant was instructed to move away from the subject working area and stay at their own desk (at another corner of the room) while subjects were performing the tasks. After subjects had examined the information in the OMS related to Task 1, the comprehension test (Appendix D2) was administered. Most subjects finished the test within five to eight minutes. The test was then collected by the research assistant. Next, subjects were given the second instruction sheet (labelled as Task Description lb in Appendix D3). In this instruction, subjects were told that the information they had just reviewed was very important and would affect their performances in the subsequent tasks. Therefore, they needed to review the information once more. They were also told that to make the review more interesting, they would examine the information using another medium. In other words, the text-based OMS subjects were subsequently switched to the multimedia OMS for the review, and vice versa for the multimedia OMS subjects. 58 Subjects were given 20 minutes for this review. No test was given at the end of the review. However, subjects were not told before-hand that there would be no test. The purpose of this review was to ensure that subjects from the two experimental conditions, text-based OMS and multimedia OMS, had the same exposure to all information before continuing on to the other two experiments. 3.5 Results Three analyses, an independent t-test, a multivariate analysis of variance ( M A N O V A ) , and a partial least squares (PLS) analysis, were performed on the data collected from Experiment 1. A l l statistical analyses were performed using SPSSX (version 6.10 for Windows) unless otherwise stated. 3.5.1 Subject Profile The two trained research assistants were told to note down any irregularities during the experiment. Of the 80 subjects, eleven were described as the follows: "not cooperative, didn't concentrate during the experiment", "just came in for cash award", "appeared very tired", and/or "very impatient, did not listen to instructions". These 11 subjects were removed from the sample, and 11 replacement subjects were run through the experiment. A l l analyses reported here pertain to the original 69 subjects and the 11 replacements. Of the 80 subjects, 54% were males and 46% were females. Their average age was 25. Half of the subjects were business students, the others majored in Sciences (27%) and Arts (23%). Undergraduate students made up 47% of the subjects and the other 53% were graduate students. A l l of the subjects used computers (either PC or Mac) regularly, with more than half 59 using computers more than once per day. However, virtually all of the subjects had little or no exposure to Netscape, the Web browser used for the study. 3.5.2 Evaluation of Assumptions of Statistical Tests There were no missing data for Experiment 1. Therefore, the sample size for all analyses in Experiment 1 was 80. The data were checked for univariate outliers. The criterion used was p < 0.001, as suggested by Tabachnick and Fidell (1989). No univariate outliers were found at this level. Secondly, for variables that were planned to be used in multivariate analysis, their Mahalanobis Distances were calculated using SPSSX's regression routine to check for multivariate outliers. For Chi-Square, with two degree of freedom and p < 0.001 (Tabachnick and Fidell, 1989), the critical value was 13.82. No multivariate outliers were found at this level. Third, normal plots and skewness and kurtosis scores were examined to assess the univariate normality assumption. The descriptive statistics are reported in Table 3.2 (see page 162). As shown in the table, all variables, with the exception of the variable DesTest (comprehension of descriptive knowledge, marked with ** in Table 3.2), had acceptable skewness and kurtosis scores. DesTest has moderate negative skewness (skewness = -0.721, z-score = -2.68). A reflection with square root transformation was applied to DesTest (Tabachnick and Fidell, 1989). Two complete sets of analyses (t-test, M A N O V A , and PLS) were performed, one on the original data, one on the transformed data. The two sets of results showed no significant departures from each other. Since the skewness was only moderate and that since results from analyses of the transformed data were similar to those from the original 60 data, the results of all analyses using the original data are presented. Fourth, for those variables to be used in multivariate analyses, linearity and homogeneity of variance assumptions were also tested. The bivariate scatterplots of the variables showed satisfactory linear relationships. The results of the univariate and multivariate homogeneity variance tests are summarized in Table 3.3 (see page 163). For ExpTest (explanative knowledge), the p-value was low at 0.03, but still greater than the acceptable limit of 0.001. Finally, it was noticed that very few subjects (only 10 out of 80) were able to correctly answer Question 9 (one of the inference test questions). This question was dropped from further analysis. As a result, only six, out of the original seven, inference questions were included in the final analyses. In summary, there were no univariate or multivariate within-cell outliers at an alpha level of 0.001. One variable was moderately negative-skewed. Two sets of analyses were performed, one for the transformed data and one for original data. These two sets of results were similar. Results of the evaluation of assumptions of linearity and homogeneity of variance were satisfactory. 3.5.3 Overall Fact Comprehension A n independent sample t:test was used to examine the difference between the two experimental groups (text-based vs. multimedia OMS) in terms of their overall fact comprehension (i.e., sum of the scores on the explanative and descriptive tests). The results are summarized in Table 3.4 (see page 164). As shown in Table 3.4, there was no statistically significant difference between the two experimental groups in their overall scores on fact-based questions (t(78)=0.84, p=0.405 ). The mean score was 5.13 (SD= 1.36) 61 for the text-based OMS group and 5.40 (SD = 1.57) for the multimedia group. Therefore, H I , which states that subjects who use multimedia OMS will better comprehend the information presented than those who use text-based OMS, was not supported. 3.5.4 Comprehension of Explanative and Descriptive Knowledge The M A N O V A routine in SPSSX was used to examine H2a and H2b. The dependent variables were comprehension of explanative knowledge and descriptive knowledge. The independent variable was the system type, i.e., whether a subject was assigned to use the text-base OMS or the multimedia OMS. The multivariate tests (Wilks' Criterion, Pillai's Trace, Hotelling-Lawley Trace and Roy's Maximum Root) showed that the combined dependent variables were significantly affected by system type (F(2,77)=5.66, p=0.005). The results of the univariate analyses are summarized in Table 3.5 (see page 165). The table shows that only for the score on explanative test was system type statistically significant (F(l,78) = 8.51, p=0.005) as a main effect. To investigate the way in which system type affected the two dependent variables, the means of these variables were examined. The means and standard deviations for the two dependent variables by system type are presented in Table 3.6 (see page 166). Results from Tables 3.5 and 3.6. together indicate that subjects who used the multimedia OMS scored significantly higher in the explanative test than those who used the text-based OMS (2.7 vs. 2.1). This supports H2a, which states that subjects who use multimedia OMS will score higher in explanative test than those who use text-based OMS. Although subjects who used text-based OMS appeared to score higher than multimedia OMS in descriptive test (3.0 vs. 2.7), this difference was not statistically significant. Thus H2b, which stated that subjects who use 62 multimedia OMS will score equally high in the descriptive test as compared to those who used the text-based OMS, was also supported. 3.5.5 The Relationships Among System Type, Explanative Knowledge, and Inference Ability Based on the functional perspective, we hypothesized that comprehension of explanative knowledge was mediating the relationship between media type and the ability to make inferences. A structural equation model was used to test this expected relationship. PLS-Graph (version 2.9.1, 1995), a software program which performs Partial Least Square (PLS) analysis, was used for this analysis. Path analysis, linear structural relations (LISREL), and PLS belong to the family of "second generation multivariate analysis". The major advantage of using such techniques is that they recognize and adjust for measurement errors (Barclay, Higgins, and Thompson, in press). As well, they allow the test of multiple relationships in one model. Although both path analysis and LISREL are more well-known, PLS was selected for two reasons. First, while LISREL is primarily used for theory testing and development, PLS is used for predictive applications (Wold and Joreskog, 1982), which is the case in this analysis. Secondly, both path analysis and LISREL require a larger sample size than PLS to achieve an equally powerful test. The results of the PLS analysis are presented in Figure 3.5 (page 193) and Table 3.7 (page 167). In Figure 3.5, the number on top of each path represents the weight (similar to a standardized beta weight in a regression analysis). The R 2 value for each endogenous construct represents the proportion of variance in the endogenous construct that is accounted for by the antecedent constructs (for a comprehensive discussion of PLS modeling see Barclay et al. , in press). The significance of the path coefficients was tested using the Jackknifing routine (Tukey, 63 1958; Wildt, Lambert, and Durand, 1982). Jackknifing routine operates by calculating the probability of obtaining a particular coefficient if the true value of the coefficient is zero. Therefore, the smaller the probability the higher the chance that the obtained coefficient is different from zero. First, Figure 3.5 shows that the direct path from the system type to inference ability (Beta=0.177) was not significant using the Jackknifing Routine, i.e., the use of text-based or multimedia OMS did not directly affect the ability to make inferences. This is consistent with our expectation, which was stated in H3a (the use of multimedia OMS will not directly contribute to a higher score in the inference test). Second, Figure 3.5 also shows that the path from the system type to explanative knowledge (Beta=0.314) was significant at 0.05 level using the jackknifing routine. On the other hand, the path from system type to descriptive knowledge (Beta=-0.155) was not significant at 0.05 level. This means that system type significantly affected the acquisition of explanative knowledge but not descriptive knowledge. This is consistent with the results of the M A N O V A analysis we just presented, which indicated that system type only affected scores on the explanative test but not on the descriptive test. Finally, Figure 3.5 shows that the path between explanative knowledge and inference ability (Beta=0.324) was significant at the 0.05 level, which means that the explanative knowledge directly influenced the score in the inference test. This supports H3b, which stated that subjects who score high in the explanative test will also score high in the inference test. Also, the path from descriptive knowledge to inference ability (Beta=0.127) was not significant at 0.05 level. This suggests that the score in descriptive test did not affect the performance in 64 the inference test. Taken together, the results of the PLS analysis demonstrate that the system type did not directly affect inference ability. Instead, the system type affected the acquisition of explanative knowledge, which in turn affected the ability to make inferences. Overall, this is consistent with our prediction that comprehension of explanative knowledge mediates the relationship between system type and inference ability. 3.5.6 Summary of Results In summary, results of Experiment 1 shows that the use of multimedia facilitated the retention and subsequent recall of explanative knowledge but not descriptive knowledge. Further, better retention and recall of explanative knowledge led to a higher ability to apply the information learned to novel situations, as indicated by the higher score on the inference test. Taken together, the results of Experiment 1 suggest that multimedia OMS presented information in such a way that it facilitated the acquisition of explanative knowledge, which in turn led to a greater ability to make inferences. 3.6 Discussion This section discusses the observed findings. First we provide a brief overview of the two perspectives outlined in the chapter and offer some explanations on why the cognitive perspective did not receive support from our results. Next, we discuss the relatively low R-squared values obtained in the PLS analysis. Finally, the findings are generalized and implications for research are outlined. 65 3.6.1 The two perspectives In the beginning of the chapter, two perspectives, the cognitive and functional perspectives, were outlined. These two perspectives provide different predictions on fact-based information comprehension in using multimedia OMS. The cognitive perspective views the advantage of multimedia as being directly derived from savings in cognitive effort in processing multimedia information. It predicts that multimedia presentation will lead to better retention and recall of all information. However, results of this study do not support this prediction. We did not observe any significant difference on the overall level of fact comprehension between text-based OMS and multimedia OMS subjects. In contrast, the functional perspective categorizes knowledge into two general classes, namely: descriptive vs. explanative knowledge. The functional perspective predicts that multimedia presentation will improve the retention and recall of explanative knowledge but not descriptive knowledge. Further, this perspective predicts that through improving the retention and recall of explanative knowledge, multimedia presentation will indirectly lead to an improvement in the ability to make inferences. However, descriptive knowledge was expected to have no effect on the ability to make inferences. Overall the results were consistent with the predictions of the functional perspective. The question is: why the outcomes observed were not in line with the predictions provided by the cognitive perspective? We will attempt to provide a post-hoc explanation to this question. One of the basic assumptions of the cognitive perspective is that reading text-based information is hard because verbal recoding, which is a necessary encoding process, is cognitively intensive. This assumption may not be valid in our case. 66 Sticht and James (1984) summarized 31 studies that compared auding12 (listening to verbal passages) and reading. They found that for children before the seventh grade, auding surpasses reading performance. Beyond seventh grade, auding and reading performances converge. They argue that this is because schooling in the middle and secondary schools require students to have extensive practice in learning from written text. Extending Sticht and James's argument, it would seem reasonable to believe that since college students have more practice in learning from written text, their reading skill may even surpass auding skill. At least, with extensive practice, verbal recoding may become an automatic process. Research has shown that very few cognitive resources are required for executing automatic processes (Kahneman, 1973; Logan, 1988). Another assumption of the cognitive perspective was that since prosodic cues were missing in written text, extra cognitive effort is required to infer the missing prosodic cues, otherwise readers will risk comprehension failure. While prosodic cues are important parts of our daily conversation, they may not play an equally important role in understanding organizational information. For example, the materials that were used for this experiment included seven interview clips. These interview clips involved three different individuals introducing various aspects of the organization. The information involved was factual. Virtually no social-emotional content existed. Although the speakers used pitch and voice inflections, these were not essential to understanding the key ideas that the speakers were conveying. In 1 2 The term auding was coined by Brown (1954). Brown argued that listening to and comprehending spoken language is different from listening to nonlanguage sounds. He argued that, just as reading is not called looking, listening while processing language signals should not be called listening. He coined the term auding to refer to the process of listening to language and processing it for comprehension. 67 other words, the potential saving in cognitive effort from not having to infer prosodic cues was not realized in our experiment. Special care had to be taken in applying theories and findings from other areas of research which is evident in the fact that some of the assumptions of the theories applied to this study were invalid. Although there are extensive theories and findings in education and psychology research on multimedia presentation, most of these theories and findings were developed in the context of school children learning. As a result, researchers must exercise caution when drawing from these areas of research. 3.6.2 Low R-Squared Values in PLS Analysis Although the PLS analysis used in this study provided significant results which supported several hypotheses, the R-squared values obtained were lower than desired, ranging from 0.17 to 0.29 (see Figure 3.5). However, in evaluating these values, one has to consider the nature of the phenomenon under investigation. Learning is affected by a large number of factors other than the learning materials themselves. It is intuitive to expect that individual differences and motivation would play key roles in learning outcome. Further, a R-squared value of less than 0.1 is not uncommon in behavioral science studies, as well as in research employing structural equation modeling (Cohen, 1988). In light of this fact, the R-squared values obtained by the PLS analysis should be considered acceptable. 3.6.3 Limitations A major limitation of this study is the type of questions used for measuring comprehension and inference ability. The experimental task reported in this chapter was only 68 one of the three tasks that the subjects performed. This created a time constraint on the length of each experiment. To ensure that subjects could finish within the time allotted for Experiment 1, multiple choice questions were used for the comprehension and inference test. Although special steps had been taken to discourage random guessing, we cannot be sure that guessing had indeed not happened. In an ideal situation, open-ended questions should have been employed. Open-ended questions would also provide a richer data set on how subjects make inferences and a deeper level of understanding on why and under what situations multimedia works better than text. Another limitation of this study is related to the amount of information subjects had to examine in the experiment. The information that subjects had to examine was predetermined. Again, due to time constraints, the information stored in the OMS was reduced to a minimum. For example, in a real-life organizational context, the amount of information that a new employee has to examine during orientation training is more than what our subjects had to examine, this means that the cognitive demands in a real-life context may be higher than that required by this experiment. This should be taken into consideration in interpreting our results. In light of this fact, more empirical research is required before firm conclusions can be made about the benefits of multimedia. As with any other research that is conducted in a controlled laboratory experimental setting, there are several limitations related to the external validity of this study, such as the subjects' background and the authenticity of the simulated system. Since these limitations are shared by the three experiments reported in this dissertation, they will be discussed in the Conclusions chapter. 69 3.6.4 Conclusions for Experiment 1 The focus of this study was the investigation of how multimedia can be utilized to enhance the capability of conventional text-based OMS with the purpose of improving information comprehension. The results indicated that multimedia only enhances the retention and recall of certain types of information (explanative knowledge) but not others (descriptive knowledge). This suggests a task (or information type) to system (or media) match hypothesis. In other words, in deciding what medium is appropriate, the system designer must consider the type of information that the system is intended to convey to its users. The notion of task-media (or task-presentation style) fit is not novel. Other researchers have proposed the same notion in other domains. Examples include graphic/table research (Jarvenpaa, 1989; Tan and Benbasat, 1990; Vessey, 1991), and media richness research (Hollingshead and McGrath, 1993). In regard to studying the role of multimedia presentation in learning from organizational information, our classification of descriptive vs. explanative knowledge is a preliminary but useful starting point for future research along this line. Regardless of the type of information that the system is intended to convey to its users, multimedia presentation clearly has the advantage of creating a deeper-level understanding, as indicated by supporting and enhancing the decision maker's inference ability. This finding has also been observed in other organizational research. Premack and Wanous (1985) through a meta-analysis of 21 experiments concluded that a new employee's future job performance is affected by the method of presentation (reading from text-based documents vs. audio-visual presentation) during his/her job orientation training. They speculated that audio-visual presentation increased job performance by showing subjects a successful "behavioral model" 70 prior to cornmencing work. Compeau and Higgins (1995) observed the same "behavioral model" effect when they used video presentation to train people to use computer systems. 71 CHAPTER 4. COPING WITH AMBIGUOUS INFORMATION 4.1 Introduction This chapter addresses coping with ambiguous information, another difficulty of decision making focused on in this study. The chapter begins by differentiating between two concepts, task equivocality and information ambiguity, to facilitate subsequent theoretical discussions. Next, the theories discussed in Chapter 2 are integrated with Media Richness Theory to test the proposed benefits of multimedia OMS in reducing the level of ambiguity of information. Four specific hypotheses were developed based on the integrated theoretical perspective. A n experiment aimed at testing these hypotheses is discussed. Finally, the results are presented and deliberated. 4.2 Task vs. Information Conceptually, it is important to differentiate between a task and the information needed to perform the task. A task can be characterized by two independent dimensions: uncertainty and equivocality. On the other hand, the information needed to perform a task can be characterized by the amount and the ambiguity level of the information (Daft and Macintosh, 1981). The focus of this study is on task equivocality (as oppose to uncertainty) and information ambiguity (as oppose to information amount). The difference between information amount and information ambiguity is obvious: the former refers to the quantity of information needed to perform the task, the latter refers to the extent to which the information needed to perform the task is subject to multiple interpretations. 72 On the other hand, the difference between task uncertainty and task equivocality is less obvious and warrants further discussion. 4.2.1 Task Characteristics: Task Uncertainty Vs. Task Equivocality Traditionally, research in organizations in general, and management information systems in particular, focuses on uncertainty in information processing (Galbraith, 1973; Turban, 1988). Uncertainty is the difference between the amount of information already possessed and the amount of information required to perform the task (Galbraith, 1973). The design philosophy behind most information systems is to reduce uncertainty by providing information, such as periodic reports and predictions, to their users. More recently, the concept of equivocality has gained interest in organizational research. Equivocality refers to ambiguity, i.e., the presence of multiple interpretations about a situation or event (Weick, 1979). Daft et al. (1987) posit that the major difference between uncertainty and equivocality is in the information processing response of managers. To resolve uncertainty, decision makers attempt to acquire more data. However, to resolve equivocality, decision makers exchange subjective views to define the problem and to resolve their disagreements. Daft et al. (1987) further suggest that in situations of uncertainty, decision makers know what information is needed and what questions to ask to obtain the needed information. In contrast, in dealing with equivocality, decision makers do not know what information is needed and what questions to ask. This means that in the case of low equivocality, there is common understanding on what information is needed to perform the task, whereas in a high equivocality situation, there is a lack of knowledge of what is needed to solve the problem or to perform the task. 73 Figure 4.1 (see page 195) presents a 2 X 2 classification of task equivocality and information ambiguity. The vertical dimension is the equivocality level of tasks. The two questions represent two types of task : a less-equivocal task (How many full-time employees?) and a equivocal task (Should A be promoted?). The "employees" question is classified as less-equivocal because a decision maker can clearly indicate what information is needed to answer the question. In contrast, the "promotion" question is classified as an equivocal task because no clear guidelines exist on what information is needed to answer the question. The horizontal dimension is the ambiguity level of information. Independent of the task, the information available to perform the task (to answer the question) can have various levels of ambiguity. For simplicity, two ambiguity levels are shown in the figure. For the less-equivocal task ("employee" question), the information would be considered as not ambiguous if the information available states that "there are 20 full-time and 5 part-time employees in Account payable Department" because it directly answers the question in an unambiguous way. On the other hand, if the information available states that "25 people work in Account Payable Department. Traditionally, about 20% of the employees are part-time employees", then the information is considered to be more ambiguous. First, it is not clear whether "20% of the employees" refers to all employees in the company or the Account Payable Department alone. Second, it is also not clear whether the percentage refers to the current one or, perhaps, the average for past few years. For the equivocal task (promotion question), assume that decision maker has decided that one piece of information that he/she requires is the opinion of A ' s immediate superior. In this case, if the information is stated as: "On a scale of 1 to 5, A ' s immediate supervisor rated him 74 as 5 for hard working", than this information would be considered as less-ambiguous. However, if the information states that" A 's superior mentioned that A is a hard working individual", than the information is clearly more ambiguous. This is because the phrase "hard working" may mean different things to different people. Although task equivocality and information ambiguity are two different concepts, empirical evidence suggests that they are related. Daft and Macintosh (1981) found that decision makers relied more heavily on ambiguous information when dealing with equivocal tasks. They argued that highly structured and less-ambiguous information is often insufficient for equivocal tasks. Keegan (1974) reported that decision makers who dealt with less-equivocal tasks (such as finance managers) tended to use documentary sources of information, whereas decision makers who dealt with equivocal tasks (such as general managers) relied more heavily on undocumented sources of information and their own intuition. Mintzberg (1972) also found that higher-level managers relied more on soft, spoken information over hard, formalized information. Presumably higher-level managerial tasks involved more equivocal decisions, and soft information provided richer insight into organizational processes, thus better supports the information need of higher-level managers. In summary, task equivocality and information ambiguity are two different yet related concepts. The information available for a task can have different levels of ambiguity independent of the task equivocality level. On the other hand, empirical evidence suggests that the information needed to perform equivocal tasks is often more ambiguous in nature, because well-structures and less-ambiguous information does not provide the rich insight that is required for equivocal tasks. 75 4.3 Reducing Ambiguity of information As was articulated in the Theory chapter (Chapter 2), misinterpretation of information can be caused by using a medium that does not have the capability to fully convey contextually rich information. One widely known theory of media choice in organizational communication and information systems research is the Media Richness Theory proposed by Daft and Lengel (1984; 1986). The central tenet of Media Richness Theory is that performance is best achieved when information processing capacities (i.e., medium used) match information processing demands (i.e., task requirement) (see Markus, 1987; Rice, 1992; and Steinfield, 1992 for review). More specifically, Media Richness Theory posits that media vary in their ability to communicate information and to facilitate understanding. A "rich" medium uses a wide variety of symbolic languages such as graphics, voice inflections, and body gestures to convey information. The theory suggests that for less-equivocal13 tasks, a lean medium such as text is sufficient to convey the information. In contrast, for equivocal tasks, a rich medium is required because it can better convey the information involved and thus facilitate a better understanding. Media Richness Theory was originally developed for human-to-human communication (mediated by technologies such as telephone or computers) for organizations. In this study we extended the theory and applied it to human-computer interaction. Although Media Richness Theory, has "face validity" and intuitive appeal, research that has examined the theory has so far produced inconsistent results. However, as pointed out by Kinney and Dennis (1994), most of 1 3 We prefer the term less-equivocal over the more commonly used unequivocal because it better reflects the nature of the variable: Equivocality level is a continuum rather than two discrete points. 76 these studies tested the theory by examining users' (typically the senders') perception of the media "fit", rather than the actual performance effects of media use. Typically, in these studies, respondents (managers) are presented with a set of hypothetical messages and are asked to choose which medium they would use for each of these messages (see Rice, 1992 for a review of these studies). There is only a handful of studies that examine the actual performance effects of media use (e.g., Hollingshead, McGrath, and O'Connor, 1992; Kinney and Watson, 1992; Kinney and Dennis, 1994; Sheffield, 1989; Valacich et al., 1994). These studies used controlled laboratory experiments to test the theory. Unfortunately, they have also produced mixed results. A common feature of these studies is the experimental tasks employed. These studies involved having two or more subjects using a communication medium to achieve a task. In such cases, a large portion of the result variance might have been caused by the social dimension of communication. Specifically, factors such as group dynamics, group history, and conformance pressure might have over-shadowed the main effect (media effect) under investigation. Rather than studying the communication among group members, this study examines Media-Richness Theory in the context of interaction between a human decision maker and a computer. By eliminating the social dimension of communication, the core of the theory can be better tested. Another issue related to the ambiguity of information is the ease of understanding and formulating mental pictures of the context around the information. It is difficult for a decision maker who was not present when an event occurred to form a complete mental picture of the event. However, a rich medium such as video would literally "bring" the decision maker into 77 the event. For example, in the context of a meeting, a written transcript of the meeting shows only a surface-level abstraction of the meeting. On the other hand, a video clip of the meeting provides a richer and more accurate representation of the meeting. From the video, the decision maker can see the interactions among meeting participants: their eye contacts, physical proximity, etc. This information provides a better perspective, and hence a better understanding of what was said and why it was said in the given context. This also helps to reduce the probability of misinterpretation. Based on the above discussion and following Media Richness Theory, we propose that since information needed for equivocal decision making is often ambiguous and subject to multiple interpretations, rich media should be used to present the information needed for such decision tasks. A rich medium, such as video, preserves much of the semantically rich information in social interactions and makes the intended meaning less ambiguous. On the other hand, less-equivocal tasks require a lean medium, because a rich medium provides too much "noise" and superfluous information for less-equivocal tasks. Decision makers may have to make extra effort to filter out this noise and superfluous information. It needs to be pointed out that an alternative possible outcome exists, even though the prevailing theory suggests that a rich medium would reduce information ambiguity by reducing the number of possible interpretations. It is plausible that information conveyed using a rich medium may lead to even more interpretations. For example, Goldman-Segall (1995) pointed out that people who have viewed Rodney King receiving a beating on video may come up with different and conflicting interpretations of the same event, which ranged from concluding that the Los Angeles Police Department is guilty of using excessive force to fully sympathizing with 78 the Los Angeles Police Department14. This example appears to suggest that an individual's social and racial background affects the way he/she assigns weights to different aspects of the information. It is certainly not uncommon for organization members to bring their personal bias into organizational decision making, perhaps even intentionally to achieve "hidden agendas". However, this is outside of the scope of this study. To keep the study focused, we assume that organizational decision makers share a set of common goals (e.g., profit maximization for profit-making organizations, and increase service level for non-profit organizations). These common goals narrow the perspectives that these decision makers bring with them in interpreting organizational information. In summary, multimedia OMS has the potential to reduce the problem of ambiguity of information. It does so by using a set of rich symbolic languages to capture the semantically rich meaning of the information. Further, it preserves the authentic context surrounding the information thereby helping decision makers to better understand the context in which the information was conveyed. 4.4 Hypotheses As was discussed in the Theory chapter (Chapter 2), conventional text-based information systems have been found to be very effective in supporting the information processing needs of low-equivocality tasks. This is because the information needed to perform low-equivocality tasks 1 4 Goldman-Segall has designed a research tool, Constellation, for layering multiple points of view in order to reach "configuration validity", an approach that aims to build broader and more robust conclusions for rich information. 79 is well-known, well-structured, concise, and without much ambiguity. Such information is called structured information, and according to the Media Richness Theory (Daft and Lengel, 1986), it can be effectively conveyed using a lean medium, such as text. On the other hand, Media Richness Theory posits that if a rich medium, such as multimedia, were used to present structured information, the "noise" contained in multimedia presentation may cause distraction. This noise interferes with a user's ability to extract key information, resulting in the user having to spend extra time. Therefore, for less-equivocal tasks, the use of multimedia presentation is less efficient than text in terms of the amount of time taken to perform the same task. HI: For less-equivocal tasks, multimedia OMS subjects will take longer to perform the task than text-based OMS subjects. For less-equivocal tasks, multimedia OMS may be less efficient than text-based OMS. This, however, does not imply that multimedia OMS is less effective than text-based OMS in reducing the ambiguity level of information (assuming that there is still a certain level of ambiguity in the information associated with the less-equivocal tasks). Multimedia OMS captures and conveys all of the information that is being captured and conveyed by text-based OMS, and in most cases, it offers more information than text-based OMS. Specific information available in text-based OMS that helps to reduce ambiguity is also available in multimedia OMS. Therefore, the following hypothesis is formulated. 80 H2: For less-equivocal tasks, multimedia OMS and text-based OMS will lead to the same amount of reduction in perceived ambiguity. On the other hand, based on Media Richness Theory, equivocal decision making involves ambiguous situations that could be subject to multiple interpretations. In dealing with such cases, decision makers need to rely on their subjective judgements to come up with an interpretation of the situation (Weick, 1979). To construct a better understanding of the problem and situation, decision makers require a rich and comprehensive view of the contextual information (Daft, Lengel, and Trevino, 1987). 5 Text-based OMS filters out much of the rich contextual information. Text is severely limited in capturing contextual information. Paradoxically, the advantage, and hence the disadvantage, of conventional text-based systems is that they present complex information in a very concise, and thus simplified form using various abstraction techniques, in this case, by eliminating sound cues and pictures. Based on this we hypothesize that: H3: For equivocal tasks, multimedia OMS will reduce perceived ambiguity more than text-based OMS will. We also expect that since subjects will perceive that the multimedia OMS will help them reduce equivocality more than the text-based OMS, subjects will also perceive the multimedia OMS as being more useful than the text-based OMS. 81 H4: Multimedia OMS will be perceived as more useful than text-based OMS. 4.5 The Research Method The same subjects who participated in Experiment 1 also participated in this experiment. The experimental system used is the same one that was employed in Experiment 1. For detailed discussions on subject population and the experimental system, please refer to Sections 3.4.1 and 3.4.2 in Chapter 3. 4.5.1 The Experimental Materials Similar to those in Experiment 1, the experimental materials used in this experiment were organized using hypertext links. The materials used focused on two of the six departments in British Columbia Cancer Research Centre (BCCRC). They included general information on the two departments that was more in-depth compared to that presented in Experiment 1. Two interviews with the researchers in these two departments were included. As with Experiment 1, the multimedia OMS presented these interviews using video clips, and the text-based OMS presented them using text transcripts. Figure 4.2 (see page 196) is an example of the user interface in the multimedia OMS, after a hypertext link was selected in a previous screen. Figure 4.3 (see page 197) shows the corresponding user interface in the text-based OMS when the same selection was made. 4.5.2 Experimental Task The task employed in this experiment consisted of evaluating the information related to 14 statements (shown in Appendix E2). Subjects were asked to evaluate the degree to which they felt that the information provided by the OMS, associated with each of the statements, was 82 ambiguous. Seven of these statements were related to facts that were explicitly stated in the OMS. The other seven required subjects to make judgements on the personalities of the four researchers who were being interviewed. The seven fact-related statements formed the less-equivocal task, and the other seven which involved making personality judgements formed the equivocal task. The rationale and justification for using these tasks will be discussed in Section 4.5.3. Subjects performed the task twice, once with the multimedia OMS and once with the text-based OMS. Half of the subjects used multimedia OMS first, and the other half used text-based OMS first. There was no time limit placed on performing the task. Subjects were given the 14 statements before-hand. They were told to review the statements before examining the information provided by the OMS. For each of the 14 statements (see Appendix E2), subjects were asked to indicate, on a -3 to +3 scale, to what extent they agreed or disagreed with the statement. Next, using the following two items (adapted from Daft and Macintosh (1981) and to be discussed in Section 4.5.4), they were asked to rate, on a -3 to +3 scale, the degree to which they felt that the information in the OMS related to the statement was ambiguous: 1) the information could be interpreted in several ways, and 2) the information could mean different things to different members of the board of trustees. Subjects were permitted and encouraged to refer back to the information stored in the OMS that they had reviewed during the evaluation of the 14 statements. This was to minimize the learning effect between the first and second evaluation. We were not interested in the improvement in evaluation as a result of seeing the same information for a second time. Rather, we were interested in the change in evaluation as a result of using another OMS which conveyed 83 information via a different medium. Subjects were asked to think-aloud when they performed the evaluation. Their verbal protocols were tape-recorded. 4.5.3 The Independent Variables There were three independent variables in this experiment: system type, system usage order, and task type. 1) System type: Two systems, text-based OMS and multimedia OMS, were compared in the experiments. The two systems were equivalent in terms of verbal information content and user interface (please refer to Figures 4.2 and 4.3 in pages 196 and 197) for examples of the interfaces used for the two systems). 2) System usage order: Subjects performed the experimental task twice, once using the text-based OMS and the other using the multimedia OMS. Half of the subjects were randomly assigned to use text-based OMS first and the other half started with multimedia OMS. Those who started with text-based OMS first would use the multimedia OMS the second time (hence multimedia-second condition); those who started with multimedia OMS would switch to text-based OMS in their second time (hence test-based-second condition). Since each subject could only be assigned to one system usage order, this variable was a between-subject factor. 3) Task type: Within each system usage, subjects performed two types of tasks: equivocal and less-equivocal. Since all subjects performed both tasks, task type was a within-subject factor. Following Daft, Lengel, and Trevino (1987), a less-equivocal task was defined as a task in which the decision maker can clearly identify what information is needed to perform the task. In contrast, an equivocal task is one in which the decision maker has less knowledge of what question to ask and what information is needed to performance the task. 84 For the less-equivocal task, subjects were required to evaluate seven fact-related statements. These statements were associated with the facts that were explicitly stated in the OMS. In evaluating each statement, subjects needed to identify and recall when and where in the OMS they had seen the information associated with statements. Assuming that subjects were conscientious and paying attention while they examined the information, they should have known precisely which specific piece of information stored in the OMS was needed to perform the evaluation. For the equivocal task, a personality judgement task was selected for two reasons. First, judging an individual's personality is highly subjective. There are no clear and commonly agreed upon guidelines or criteria on what questions to ask or what information is needed to assess and evaluate an individual's personality. The task entails interpreting the information gathered about the individual to be evaluated based on the evaluator's personal background and opinions. The second reason for choosing personality judgement task was its proven validity in predicting employees' performance, and hence its relevance in organizational decision making. In recent years there has been an increased interest in using personality measures as predictors of job performance (Barrick and Mount, 1993). This is due largely to recent meta-analytic evidence that certain personality constructs are good predictors of job performance (Barrick and Mount, 1991; Hough et al., 1990). Furthermore, the Mount, Barrick, and Strauss (1994) results revealed that supervisor, coworker, and customer assessment of subordinate/coworker's personality were at least as valid as self-ratings. Seven scales were randomly selected from the "big-five "personality inventory (Goldberg, 85 1992). These seven scales were used to construct the seven statements employed for the equivocal task. 4.5.4 The Dependent Variables Three dependent variables, efficiency, effectiveness, and perceived usefulness of the system, were used independently in the three sets of analyses: . 1) Efficiency: Efficiency of an OMS in supporting the evaluation task, was measured as the time that subjects spent in evaluating the statements. This includes the time they spent in referring back to information in the OMS. The data needed to calculate time spent on the tasks was provided by the computer logs. 2) Effectiveness: Effectiveness of an OMS in supporting the evaluation task was defined as its ability to achieve reduction of perceived ambiguity. Typically, in studies which examined Media Richness Theory using controlled experiments, effectiveness was measured in terms of actual task performance (e.g., decision quality). However, by definition, equivocal tasks involved multiple solutions, and evaluating the goodness of one solution against the others is often difficult, if not impossible. Further, we believe that the key role of an OMS is to convey information in clear, easy to understand, and unambiguous ways. Therefore, this study chose to use the ability of an OMS in helping a decision maker to reduce his/her perceived ambiguity level as an indicator for the effectiveness of the OMS. Although this is also a perceptual measure, it is very different from those used in other studies that required managers to match media with messages based on their perception. The perceptual measure used in this study required subjects to focus on specific aspects (i.e., the ambiguity level rather than a general overall perception on "fit"). 86 Two Likert-type scales (see Appendix E5), adapted from Daft and Macintosh (1981), were used to measure perceived ambiguity15. Daft and Macintosh's original instrument consisted of three items. One item was not applicable to this study; hence, it was excluded from our measures16. The internal reliability of the original instrument, as reported by Daft and Macintosh, was 0.73. As will be reported in the Results section, the modified scale has a much higher reliability score (0.89). In the experiment, subjects performed the task twice. The reduction in ambiguity was measured by the difference between subjects' first and second ratings on the 2-item scale. To the extent that OMS A is more effective than OMS B, subjects' perceived ambiguity level should decrease after using OMS A for their second evaluation, but their perceived ambiguity level should not decrease after using OMS B for the second evaluation. 3) Perceived usefulness: Perceived usefulness was defined as the degree to which a person believes that using a particular OMS would enhance his/her job performance. Davis and his colleagues (Davis, 1989; Davis, Bagozzi, and Warshaw, 1989) theorized that a user's perception of the usefulness of a system, which they called perceived usefulness, is one of the key determinants of the user's acceptance of the system. Their longitudinal study demonstrated that 1 5 Putnam and Sorenson (1982) independently developed another six-item scale to measure perceived ambiguity (equivocality). Daft and Macintosh's instrument was selected mainly because it is more parsimonious. In this experiment, subjects had to rate the perceived ambiguity level of the information related to all 14 statements (seven of which formed the equivocal task, and the other seven of which formed the less-equivocal task). A six-item scale would have required our subjects to perform 84 (14 X 7) ratings, which is too time consuming. Most importantly, subjects might react negatively toward such a highly repetitive task. 1 6 The item dropped from this study involved evaluating whether the task (problem) faced has more than one satisfactory solution. Since this study focused on the ambiguity level of information rather than solution, this item was not applicable to this study. 87 perceived usefulness is correlated with both current and future usage of the system. Other studies (Adams, Nelson, and Todd, 1992; Igbaria, Guimaraes, and Davis, 1995; Subramanian, 1994) have supported their finding. Moore and Benbasat (1995) integrated Rogers's (1983) theory of diffusion of innovations into Davis et al.'s (1989) model and found further support for the impact of perceived usefulness on the extent of system use. These studies established the theoretical and practical importance of perceived usefulness. Davis's perceived usefulness instrument consists of 10 items. The instrument has undergone rigorous psychometric testing by Davis and others (Davis, 1989; Hendrickson, 1993; Moore and Benbasat, 1991; Segars and Grover, 1993; Szajna, 1994). The reliability of the instrument, as originally reported by Davis, was 0.97. The 10-item scale (shown in Appendix E5) was adapted for this present study. Subjects were asked to rate the perceived usefulness of the second OMS they used by using the first OMS they used as the basis of comparison. 4.6 Experimental Procedure Immediately after Experiment 1, subjects were handed Task Description Ha (See Appendix E l ) and the questionnaire for this experiment (Appendix E2). In the instructions, subjects were explicitly told to read through the questionnaire before examining the information in the OMS. This was to help them prepare and focus on the information needed to respond to the questionnaire. Subjects were also told to go through the information in the OMS once without interruption. At the beginning of the experiment, they were reminded to review all information stored in the OMS. Depending on their assignment, subjects either used the text-88 based OMS or the multimedia OMS to examine the stored information. Subjects were asked to think-aloud while they responded to the questionnaire (performing the evaluation task). Prior to the beginning of the experiment, during the training session, subjects were given an exercise to practice on think-aloud (see Appendix CI for the exercise and instruction). During the experiment, subjects' verbal protocols were tape-recorded. They were allowed to refer back to the information in the OMS whenever they needed. Subjects were asked to use the black pen provided to fill in the questionnaire. After they had completed the questionnaire, the subjects were given a 5-minute rest break. Cookies and juice were provided during the break. After the break, subjects proceeded to the Part II of the experiment. Part II was a repeat of Part I, except that subjects used another OMS (either text-based or multimedia OMS, i.e., the one that was not used in Part I). The instruction for this part of the task is labelled as Task Description l ib and attached in Appendix E3. This time they were asked to use the provided red pen to write on the same questionnaire. In other words, subjects made their second evaluation by using their first evaluation as a reference. The color coding scheme was used for ease of differentiating between the first and second responses when data were entered into a computer file for analyses. Similar to the first part, they were required to verbalize their thought processes, and the verbal protocols were tape-recorded. After subjects finished Part II of the experiment, they were given the perceived usefulness questionnaire (Appendix E4). The entire experiment took about an hour to complete. 89 4.7 Results Three sets of data were collected from this experiment: 1) Data on time taken to perform the task; 2) Data on perceived ambiguity reduction; and 3) Data on perceived usefulness of the system. Three separate analyses, two independent sample t-tests and one repeated-measures A N O V A , were performed for these three sets of data. A l l statistical analyses were performed using SPSSX for Windows (Version 6.10, 1994). 4.7.1 Evaluation of Assumptions of Statistical Tests No missing data were found in the data sets, with the exception of one subject who did not complete the perceived usefulness questionnaire. Therefore, the sample size was 79 for the analysis on perceived usefulness, and 80 for all other analyses in this experiment. The descriptive statistics for all variables, as summarized in Table 4.2 (see page 170), were examined. The table shows that three variables (marked with **) have moderately high scores on either kurtosis or skewness. Frequency plots showed that there were possible outliers in these three variables. The shapes of the frequency plots indicated that the high kurtosis and skewness scores might have been caused by these possible outliers (slightly outside of three standard deviations). We used the SPSSX regression to calculate the Mahalanobis Distances to check for possible multivariate outliers. For Chi-Square with four degrees of freedom and a p-value of 0.001 (Tabachnick and Fidell, 1989), the critical value was 18.45. Three multivariate outliers were detected at this level. These three outliers were temporarily dropped from the data set. 90 A second set of descriptive statistics was obtained using the new data set. The new statistics showed that after the removal of the three outliers, for all variables, the kurtosis and skewness scores were reduced to the acceptable level (p > 0.001). As in Experiment 1, both sets of data were used to perform all the analyses. The two sets of results were not significantly different. The original data seemed to produce more conservative results than those of the new data, i.e., significant effects in the original analyses were significant at stronger levels in the second set of analyses. In light of this, and the risk of falsely removing observations that were part of the true score in the population by dropping three outliers, we decided to keep the three possible outliers in our analyses. Finally, the assumptions on univariate and multivariate homogeneity of variance and linearity were evaluated. The Cochran's and Bartlett-Box scores, together with their p-values, are reported in Table 4.3 (see page 171). The p-values in the table indicate that there were no violation of the homogeneity of variance assumption. The bivariate scatterplots for all variables were also examined. No major violations of linearity assumption were observed. In summary, there were three potential outliers at a p-value of 0.001. Two sets of analyses were performed: one using the original data and one with the possible outliers removed. The two sets of results showed no significant departure from each other. Since the two sets of results were similar, the original data were used for the analyses. Results of the evaluation of assumptions of linearity and homogeneity of variance were satisfactory. 4.7.2 Efficiency of Text-Based OMS for Less-Equivocal Task Recall that efficiency was measured in terms of the time taken to perform a task. Computer log data, together with the tape-recorded verbal protocols, were used to determine the 91 amount of time each subject spent on less-equivocal and equivocal tasks. These data were analyzed using an independent sample t-test. The independent variable was the system type, i.e., multimedia OMS vs. text-based OMS. The dependent variables were the amount of time taken to perform the less-equivocal task and the amount of time taken to performed the equivocal task. The results of the analyses are reported in Tables 4.4a and 4.4b (both in page 172). The tables also list means and standard deviations. As shown in Table 4.4a, for the less-equivocal task, subjects who used text-based OMS took significantly less time to complete the task (t(78)=2.36, p=0.021). On average, multimedia OMS subjects spent 8.2 minutes on the task while text-based subjects spent 6.5 minutes, representing a 22% reduction in time. This supports H I , which states that for less-equivocal task, multimedia OMS subjects will take a longer time to perform the task than text-based OMS subjects. Table 4.4b also shows that for equivocal tasks, the time taken for the two groups of subjects were not significantly different (t(78)=0.32, p=0.794). This means that multimedia OMS subjects and text-based OMS subjects spent about the same amount of time in performing the equivocal task. 4.7.3 Effectiveness (Ambiguity Reduction) Data associated with ambiguity reduction were formulated into a 2 (system usage order: text-based-second vs. multimedia-second) X 2 (task type: equivocal vs. less-equivocal task) mixed factor design and analyzed using the repeated-measures A N O V A routine in SPSSX. Recall that all subjects performed both tasks twice, once using the text-based OMS and once using the multimedia OMS. The independent variables were system usage order (text-based-second vs. multimedia-second) and task type (equivocal vs. less-equivocal task). The 92 system usage order was a between-subject factor while task type was a within-subject factor. The dependent measure was the reduction in ambiguity rating (the difference between subjects' first and second ratings) on the information needed to perform the tasks. Also recall that on each of the 14 statements (seven for the less-equivocal task and seven for the equivocal task as shown in Appendix E2), subjects were asked to indicate to what extent they agreed or disagreed with the statement. First, it was noticed that the ratings received by two of the seven statements included in the less-equivocal task were very different from the other five. For these two statements, their variances were much larger than the others (2.31 and 2.05 as compared to the variances of the rest of the five which ranged between 0.01 and 0.81). Upon examining the two statements, it was noticed that the wordings used for these two statements might have been confusing. These two were dropped from further analyses. Next, a t-test was used to analyze the ratings for the remaining statements to ascertain that our manipulation of less-equivocal vs. equivocal tasks was successful. The result showed statistically significant difference between the two types of tasks (t(79)=39.13, p=0.000) 1 7. The average rating was 2.61 for the less-equivocal task, and -0.13 for the equivocal task. This means that subjects agreed (or disagreed) with the statements corresponding to the less-equivocal task to a higher degree than with the equivocal task. This suggests that the experimental manipulation between less-equivocal and equivocal tasks was successful. Two items (discussed and listed in Section 4.5.2), adapted from Daft and Macintosh (1981), were used to capture the subjects' perception of the ambiguity level of the information 1 7 The scores for those negatively-worded items were transformed before performing the analysis. 93 needed to perform the task. The reliability of the two scales was first tested. The Cronbach's Alpha coefficient was 0.89, indicating that the measure was highly reliable. The scores on these two items were subsequently aggregated into an average score for further analyses. The results of the repeated-measures A N O V A are reported in Tables 4.5a and 4.5b (both in page 173). Table 4.5a shows that system usage order was significant as a main effect (F( 1,78)=8.82, p=0.004). This means that whether a subject examined information using text first and multimedia second, or multimedia first and text second led to different level of ambiguity reduction. Table 4.5b, indicates that task type was not statistically significant as a main effect (F( 1,78)=0.37, p=0.329). This indicates that there was no statistically significant difference between less-equivocal and equivocal tasks in the amount of ambiguity reduced. However, Table 4.5b shows that the interaction effect between system usage order and task type was statistically significant (F(l,78)=4.14, p=0.001). This interaction effect is depicted graphically in Figure 4.4 (see page 198). Table 4.6 (see page 174) reports the corresponding means and standard deviations for Figure 4.4. As shown in Figure 4.4, the slope for the equivocal task is much steeper than that of the less-equivocal task. In other words, the effect of system type on ambiguity reduction appeared to be stronger for the equivocal task than for the less-equivocal task. Planned comparisons showed that for the less-equivocal task the difference between multimedia-second and text-second conditions was not statistically significant (t(78)=0.28, p=0.778). On the other hand, for the equivocal task, the difference between multimedia-second and text-second conditions was statistically significant (t(78)=4.21, p=0.000). 94 The result of the first planned comparison indicated that, for the less-equivocal tasks, the two system are equally effective in reducing perceived ambiguity levels. This also supports H2, which states that for less-equivocal tasks, multimedia OMS and text-based OMS will lead to the same amount of reduction in perceived ambiguity. In contrast, the result of the second planned comparison indicated that, for equivocal tasks, multimedia OMS was more effective in reducing ambiguity levels relative to text-based OMS than text-based OMS was relative to multimedia OMS. This also supports H3, which states that for equivocal tasks, multimedia OMS will reduce perceived ambiguity more than text-based OMS wil l . To gain a better understanding of the effects of system and task type on ambiguity reduction, it is necessary to take a closer look at subjects' actual perceived ambiguity levels. Tables 4.7a and 4.7b (both in page 175) report the actual perceived ambiguity levels after each system usage for less-equivocal and equivocal tasks. For the less-equivocal task, the average rating on perceived ambiguity level was -1.87 (the scale ranged between -3 to +3, where -3 indicated highly unambiguous and +3 indicated highly ambiguous). For the equivocal task, the average rating was +1.16. The net difference between the two tasks, i.e., the task effect, was 3.03 (the range between 1.16 and -1.87). On the other hand, for text-second condition, the average rating was -0.27, and for multimedia-second condition, the average rating was -1.05. This yielded a net system effect of 0.78 (the range between -1.05 and -0.27). This clearly suggests that the effect of task dominated the effect of systems. 95 4.7.4 Perceived Usefulness of the System Perceived usefulness of the system was captured using a 10-item instrument developed by Davis (1989). Subjects were asked to evaluate on a -3 to +3 scale the usefulness of the second system they used (either text-based or multimedia, depending on the experimental condition they were assigned) using the first system they used as a basis for comparison. One of the 80 subjects did not complete this questionnaire, hence the sample size for this analysis was 79. Data collected were analyzed using the independent sample t-test routine in SPSSX. Prior to performing the t-test, the reliability of the instrument was examined. The analysis showed a Cronbach's Alpha level of 0.94, which indicated that the 10 items were highly reliable. The scores on these 10 questions were averaged to formed an aggregated score for the t-test. The results of the independent sample t-test, together with the means and the standard deviations for the two conditions, are summarized in Table 4.8 (see page 176) . The table shows that subjects perceived the multimedia OMS as being more useful than the text-based OMS in helping them to perform the task (t(78)=5.42, p=0.000, mean score -0.12 vs. 1.58, for the range of -3 to +3). This result supports H4, which states that the multimedia OMS will be perceived as more useful than the text-based OMS. This is consistent with results that show that the multimedia OMS was more effective in reducing the ambiguity associated with the equivocal tasks. The correlation between perceived usefulness and ambiguity reduction was also examined. It was found that perceived usefulness was correlated significantly with ambiguity reduction for the equivocal task (r=0.41, p=0.00), but not for the less-equivocal task (r=0.12, 96 p=0.31). 4.7.5 Summary of Results The results of this experiment indicated that for less-equivocal tasks, both text-based OMS and multimedia OMS were able to reduce the ambiguity of information. However, text-based OMS was more efficient, in terms of the amount of time needed to perform the less-equivocal task. Most importantly, only the multimedia OMS was able to reduce the ambiguity levels associated with equivocal tasks. Further, subjects perceived the multimedia OMS to be more useful than the text-based OMS in performing the task. 4.8 Discussion The results of this study on the less-equivocal task are consistent with the literature that information systems that store information as text and numbers are very efficient and effective in supporting structured, less-equivocal decision making. While multimedia OMS appears to be equally effective as text-based OMS in reducing the ambiguity of information for a less-equivocal task, text-based OMS is more efficient in terms of time needed to acquire the information. However, we must temper our conclusions since the time required to view the video clips in multimedia OMS is fixed, but the time required for the text depends on individual's reading speed and motivation. In general, a person can read a printed page faster than listening to a speaker reading the same information. Nonetheless, verbal protocols and the time logs collected while subjects were performing the task provided evidence that the differences in time taken to perform the task were not caused by the potentially different speeds in processing the two sources of information. The protocols 97 revealed that subjects who used the multimedia OMS spent more time reviewing information in the OMS to confirm what they had examined than those who used the text-based OMS. Multimedia subjects appeared to be distracted by other information in the video clips. For example, they focused part of their attention on the surrounding environment in which the interviews took place. This extra information appeared to interfere with subjects' abilities to focus on the main issue. Consequently, they needed to review the information more frequently than did text-based subjects. A n example will highlight this point. One of the statements in the less-equivocal task required subjects to evaluate the following statement: "One of the main focuses of Dr. X's lab is to determine the presence of hypoxic cells so that treatment can be adjusted accordingly. "18 Typically, the text-based OMS subjects would recognize that the above statement was explicitly stated in the information stored in the OMS. Therefore, they were able to make a relatively quick evaluation. In contrast, the multimedia OMS subjects appeared to have been distracted by other information in the video clip that mentioned the above statement. One of the multimedia OMS subjects made the following comment: "/ think this was mentioned in the video clip. But, I am not sure if it was talking about Dr. X's lab or Drs. X and Y's lab. I remember the video clip showed both of them working in the lab, so it might have been Drs X and Y's lab rather than Dr. X's lab that was doing this kind of research . . . I better look at it again (moved on to reexamine the particular video clip)." The above example is not an isolated incident. Multimedia information appeared to contain too much superfluous information that distracted subject's attention from the key 1 8 Actual names were used in the experiments but were removed here to protect the identities of the individuals involved. 98 information. The results also clearly showed that text-based OMS is inadequate in supporting equivocal decision making. When subjects were given a second OMS (different from the first one that they used) to perform the task, though both groups of subjects spent an equal amount of time on the task, only those subjects who used multimedia OMS for the second time reported a reduction in ambiguity levels. Our results also provide support for Media Richness Theory, which posits that while a lean medium is better for less-equivocal tasks, a rich medium is better for equivocal tasks. By applying Media Richness Theory to the human-computer interaction domain, this study eliminated the social dimension of human-to-human communication, which has been shown to have a significant effect on communication process and outcome (Markus, 1987; Steinfield, 1992). There were several unexpected results from the experiment that warrant further discussion. It was expected that since all statements associated with the less-equivocal tasks were, in some way, explicitly stated in the system, a subject's initial rating on the ambiguity level of information associated with these statements should be low (perhaps close to -3 on the -3 to +3 scale used). However, results showed that the initial ratings were -1.77 for subjects who used multimedia OMS first, and -1.69 for subjects who used text-based OMS first. We speculate that this was because our subjects were very cautious and conservative in judging the statements. For example, one of the statements included as part of the less-equivocal task was the following: "Researchers in Toronto, Princess Margaret Hospital, and Stanford are very interested in Dr. X and Dr. Y's research." 99 In an interview that subjects were provided with, Dr. X explicitly mentioned that she was very excited about the fact that the organizations mentioned above were interested about their research. Typically, in evaluating the ambiguity level of the information associated with the above statement, text-based OMS subjects made the following comments: "Well, Dr. X said that in the interview, but how do I know whether she wasn't lying to impress people? I would say there is some ambiguity level there. . ." Multimedia subjects gave a very similar response: "She said that, and she didn't appear to be lying. Still, I would feel more comfortable with the information if someone else said it." The same conservative and cautious behavior was also apparent when subjects performed the equivocal task. Notice that in Table 4.7b (see page 175), though their reported ambiguity levels for both experimental conditions were higher than their ratings for those of the less-equivocal (see Table 4.7a in the same page) task, they were not as high as we had expected (closer to +3). This conservative and cautious behavior might be the cause of the relatively small magnitude of ambiguity reduction observed. As shown in Table 4.6 (see page 174), the largest value for ambiguity reduction was 0.56 (on a 7-point scale, -3 to +3). At first glance, this value appears to be very low. However, in interpreting this effect, one has to take into consideration the "floor" and "ceiling" effects of subjects' actual ratings. Tables 4.7a and 4.7b indicates that the minimum and maximum ratings for perceived ambiguity level were 1.50 and -2.07 respectively. When interpreting a change (i.e., ambiguity reduction), the amount of change could be normalized using these two scores, because subjects' actual ratings were bound between these two scores rather than the original -3 to +3 range. After normalization, an 100 ambiguity reduction of 0.56 would yield a net change of 1.1, which translates into a 16% reduction. Regardless of this, the results clearly support the argument that multimedia can be used to enhance conventional OMS to help decision makers better cope with ambiguous information. Finally, this study examined two aspects of media "fit": efficiency (time taken) and effectiveness (reduction in perceived ambiguity level of information). We found that, for less-equivocal tasks, a lean medium is more efficient. On the other hand, for equivocal tasks, a rich medium is more effective. This may explain why past research in Media Richness Theory has produced mixed results, and that future research in Media Richness Theory should more closely examine both aspects of media fit. 101 CHAPTER 5. FTRST IMPRESSION BIAS 5.1 Introduction This chapter addresses first impression bias, one of the difficulties of decision making that is focused on in this study. The chapter begins by investigating how multimedia OMS may be used to reduce first impression bias. Next, specific hypotheses are discussed, and an experiment designed to test these hypotheses is described. Lastly, the results are presented and discussed. 5.2 Reducing the First Impression Bias First impression bias is mediated by two specific processes: 1) reinterpreting information to fit the first impression, and 2) ignoring disconfirming information. This suggests that to reduce first impression bias and to support objective decision making, these two processes have to be suppressed. One way to achieve this is through presenting information 1) in unambiguous ways such that multiple interpretations are eliminated or reduced (Daft and Lengel, 1986), and 2) in vivid ways such that evidence disconfirming the original bias is difficult to ignore (Nisbett and Ross, 1980). Presenting information using multimedia should provide a means to achieve a reduction in first impression bias. 5.2.1 Reinterpreting Information Using a set of rich symbolic languages, multimedia presentation can convey the original meaning of information in a less distorted and less ambiguous way. This reduces the potential for misinterpretation. For example, with video images, multimedia presentation preserves the 102 facial and body language of a speaker, and thus better conveys the original meaning of the message, making reinterpretation more difficult. 5.2.2 Ignoring Information Multimedia presents information in vivid ways using audio and video with embedded complementary cues. The use of complementary cues leads to better retention and retrieval of information, because complementary cues create more potential links to knowledge in long-term memory. For example, the impression created by watching a video clip of a person talking is more salient and lasting in comparison to reading the written transcript of the speech. With video and audio capabilities, multimedia makes disconfirming information more evident. As a result, the information is better registered in the decision maker's mind. In short, since multimedia preserves much of the original meaning of information, reinterpretation of the information is eliminated or reduced. Further, using complementary cues available in the video format, multimedia presents information in such a vivid way that information is better retained. This allows disconfirming information to be better registered in a decision maker's mind and renders it more difficult to ignore. Therefore, with multimedia, decision makers are less vulnerable to first impression bias and are more objective in assessing the presented information. 5.3 Hypotheses Past research (see Fiske and Taylor, 1991 ; Jones and Goethals, 1972 for summary) has consistently shown that people's opinions about an event are very much influenced by the first impression they have formed about the event. Further, first impression bias has been shown to 103 occur not only in experimentally-contrived situations, but also in real-life phenomena (Forgas, 1985). In Chapter 2, the underlying processes involved in first impression bias were discussed. Briefly, people form strong opinions based on their first impression and are inherently biased by their first impression in assessing subsequent information (Asch, 1946). If subsequent information matches their first impression, their original impression will be further reinforced. However, if subsequent information is inconsistent with their first impression, people will either try to reinterpret the new information to fit the original impression or ignore this new information. Through using audio and video format, multimedia OMS presents information in less-ambiguous (Daft and Lengel, 1984; 1986) and more vivid (Baggett, 1998) ways making new information that is inconsistent with the first impression more difficult to reinterpret and/or ignore. Therefore, it is expected that multimedia will be more effective in reducing first impression bias, and hence the following hypothesis is formulated. HI: Subjects who use multimedia OMS will be able to reduce the influence of first impression bias more than those who use text-based OMS. Research has shown that certainty or confidence with the decision is affected by the availability of information (e.g. Daft and Macintosh, 1981). Since multimedia OMS provides more information than text-based OMS, we expect that multimedia subjects will perceive that they have indeed received more information, and thereby will become more certain about their 104 decision. In other words, multimedia OMS is expected to enhance confidence in decision more than text-based OMS. Therefore, the following hypothesis is formulated. H2a: Multimedia OMS will result in more confidence with the decision made on the part of the decision maker than text-based OMS. H2b: Multimedia OMS will change decision maker's confidence with his/her decision more from previous level than text-based OMS. 5.4 The Research Method The same subjects who participated in Experiments 1 and 2 also participated in this experiment. The experimental OMS used in this experiment is the same one that was employed in Experiments 1 and 2. For detailed discussions on subject population and the experimental system, please refer to Sections 3.4.1 and 3.4.2 of Chapter 3. 5.4.1 The Experimental Materials Similar to Experiments 1 and 2, the British Columbia Cancer Research Centre (BCCRC) was used as the background case for the experimental task. The experimental materials used in this experiment were also presented using hypertext links. The materials included performance indicators for each of the six departments over the past five years. These were: 1) Total amount of research funding received from granting agencies (and average amount per scientist); 2) Total number of scientists (with details on the number of newly hired and attrition); 3) Total number of supporting staff; and 105 4) Total number of publications (and average number of publications per scientist). Experiment 3 also included an interview with the Department Head. This individual was the target of the appraisal decision, which will be elaborated in Section 5.4.2. The interview was divided into segments based on the topic discussed and linked by hypertext links to create simulated "pages" on the web. Subjects were required to review all available information. They were instructed to visit all hypertext links. Depending on the experimental condition assigned, subjects used one of the two versions of the OMS (text or multimedia). Subjects in the multimedia OMS group viewed information in a real-time, full-motion video format. For example, to learn about the Department Head's thoughts on keeping track of his subordinate researchers' work, subjects could simply "ask" the Department Head by clicking on a hypertext link that stated "keeping track of fellow researchers". A video clip that shows the Department Head explaining his views on this aspect would be presented. Subjects using the text-based OMS viewed the same information using the same hypertext link access method. The information was the same in terms of content. The only difference was the medium, namely written transcript versus video clips. Subjects in both conditions received the same quantitative information in the form of graphs and tables. Figure 5.1 (see page 200) is an example of the user interface in the multimedia OMS, after a hypertext link was selected in a previous screen. Figure 5.2 (see page 201) shows the corresponding user interface in the text-based OMS when the same selection was made. 5.4.2 The Experimental Tasks The experimental task involved performing a five-year appraisal of a Department Head. 106 It required subjects to evaluate the Department Head's overall performance, including leadership skills, vision, and communication skills, in his position as the Department Head. Subjects were also asked to make a final recommendation on whether to renew or terminate this Department Head's contract. The key information on which subjects based their appraisal was a series of interviews with the Department Head. In these interviews, the Department Head discussed and answered questions on nine specific aspects of managing the department. These nine aspects were: 1) Changes that he would make if he became the new department head; 2) Dealing with promised reports arriving late; 3) Balancing between his research and administrative duties; 4) Level of details he wants to know about his fellow researchers' work; 5) Handling disagreements between himself and his fellow researchers; 6) Dealing with unproductive researchers; 7) What he would do if the unproductive individuals tell him that they have been working on a potentially big discovery, but since things are still uncertain they can't tell him exactly what is going on; 8) Whether he would like his fellow researchers to approach him when they run into problems; and 9) His view of the highest level goal of the department. These nine aspects ranged from a higher-level view, such as his vision of where the department was heading, to specific details, such as how he would handle unproductive subordinates. Subjects could examine the Department Head's view on any one of these nine 107 aspects by clicking the hypertext link on the specific topic of interest. Depending on their experimental assignment, subjects examined these interviews either using multimedia OMS or text-based OMS. At the beginning of the experiment, subjects were instructed to examine every piece of information stored in the OMS. A n instrument (see Appendix F2) that consisted of 12 Likert-type scales was used to assist subjects in performing the appraisal. A 3-item Likert-type scale, which measured subjects' confidence with their decisions made, was attached at the end of the appraisal instrument. To ensure that subjects were motivated to perform diligently, they were also asked to provide a written justification for their decision. 5.4.3 Experimental Manipulation Figure 5.3 (see page 202) summarizes the design of this experiment. As highlighted in the figure, there were two experimental and two control groups. A l l subjects performed the appraisal task described above. The only difference between the experimental and control groups was that the experimental groups (both text-based experimental group and multimedia experimental group) received a biased information cue prior to examining the interview information. This biased cue is described in Appendix F4. The biased cue showed performance indicators (in terms of research funding, number of research and support staff, and number of publications) for the department under appraisal, contrasted with five other departments in the Research Centre, for the past five years. The biased cue showed that the performance of the department under appraisal had been relatively low compared to other departments. Pretest showed that subjects were able to detect this trend and formed an initial impression that the Department Head had not been managing his 108 department very well. A "dummy" personality profile of the Department Head (see Appendix F5) was given to the subjects in both experimental groups after they had examined the performance indicators. This was necessary to help subjects create a sense that they had sufficient information to evaluate the Department Head. The dummy personality profile included four statements adapted from Forer (1949). Studies have shown that people view generalized descriptions in a personality profile, such as the one used by this study, as accurate summaries of personality (Forer, 1949; Snyder and Larson, 1972; Snyder, 1974; Ulrich, Stachnik, and Stainton, 1963). This impression is created because these descriptions of personality profiles are general enough to be true of most people. For example, in Forer's (1949) study, college students were given a personality test. One week later, each student was given a typed personality report and was told that the report was based on the test he/she had taken a week ago. They were then asked to rate the accuracy of the report. Unknown to the students, all the personality descriptions were identical, but each person rated the personality profile he/she received as a highly accurate description of their own personality. This phenomenon is called the "Barnum Effect". Out of the 13 statements used by Forer (1949), four were adapted for this study as the personality profiles of the Department Head. These four were chosen based on their relatively higher acceptance rates reported by Forer and because they were more relevant to personnel decision making in organizations. These four statements are listed in Appendix F5. 109 5.4.4 The Independent Variables System type is the only independent variable for this experiment. Two systems, text-based OMS and multimedia OMS, were contrasted. Similar to those in Experiments 1 and 2, the two systems were equivalent in terms of verbal information content and the way to access this information (both through hypertext links). 5.4.5 The Dependent Variables There were two dependent variables, appraisal score and confidence with the decision made. 1) Appraisal score: Appraisal score was measured by a 12-item scale adapted from Denison's (1990) Leadership Index. The instrument taps into various aspects of leadership skills, including team building, goal emphasis, work facilitation, supervisory support, communication skills, and vision. The Leadership Index is part of a larger instrument, called Survey of Organization, which was validated using 36,000 respondents in 5,994 work groups in 130 organizations. The Leadership Index consists of 6 scales with two items per scale. The reliability of the Leadership Index was not reported. However, in Denison's (1990) study, the alpha coefficients for each of the scales ranged from 0.79 to 0.92, indicating that the reliability of the instrument is fairly high. 2) Confidence in the decision made: Confidence in the decision made was measured using a 3-item scale adopted from Aldag and Power (1986). It is defined as the extent to which the decision maker feels that he/she has reached a good decision outcome or made a good judgement. Aldag and Power reported that the reliability of the instrument was 0.84. 110 5.4.6 Experimental Procedure 5.4.6.1 Experimental Session Subjects in both control groups (text-based control group and multimedia control group) began by reading the task instructions labelled as Task Description III (Appendix F l ) . Next, they proceeded to examine the interview information stored in the OMS. They either used the multimedia OMS or the text-based OMS, depending on their assigned experimental condition. The interview information took approximately 15 minutes to examine. After they had finished examining the interview information, they were given the appraisal instrument (Appendix F2) to perform the formal appraisal task. Subjects in both experimental groups (text-based experimental group and multimedia experimental group) were given a separate instruction sheet labelled as Task Description Ilia (Appendix F3). Subjects in the experimental groups were also given two additional pieces of information before examining the interview information. The first extra piece of information was the performance indicators of the six departments within the British Columbia Cancer Research Centre, in which the department under evaluation was shown to have relatively bad performance in comparison to five other departments of the Research Centre (see Appendix F4). This piece of information was intended to be a biased cue for the experimental groups. Another piece of extra information was the fake personality profile (Appendix F5) of the Department Head under evaluation. The nature of this fake personality profile was discussed in Section 5.4.3. The purpose of providing this information was to create a feeling for the subjects that they had sufficient information to appraise the target person. After viewing these two pieces of extra information, experimental subjects were given 111 the appraisal instrument (Appendix F2) and were asked to perform a formal appraisal for the target person. This appraisal served as a manipulation check to ensure that the experimental subjects were indeed affected by the biased information cue. In other words, to the extent that this manipulation was successful, subjects who received the biased cue should give the target person a poor appraisal. The 3-item scale that measured confidence with decision made was also included at the end of the appraisal instrument. After the first appraisal, the experimental groups subjects were given the second instruction sheet for Experiment 3, labelled as Task Description Illb (Appendix F6). They were then asked to examine the interview information. Similarly, depending on their previously assigned experimental condition, they either used the text-based OMS or the multimedia OMS to examine the interview information; the same interview information given to the control groups. After they had examined the interview information, they were instructed to perform a second formal appraisal. Similarly, they were also required to indicate their confidence with the new decision made using the 3-item scale included. A l l subjects, in the experimental or control groups were required to provide a written justification of their appraisal. The entire experiment took approximately 45 minutes to complete. 5.4.6.2 Post-Experimental Session In the post-experimental session, subjects were asked to complete a demographic background questionnaire (Appendix G). They were debriefed with the information that the two pieces of information in the OMS were not real: the performance indicators and the personality profile of the target person in the appraisal task. They were told that the target person in the 112 appraisal task was actually an effective Department Head who was well-respected by his fellow members of the department, and the reason for distorting the performance indicators was to see how people reacted to different kinds of information. Finally, subjects were thanked and paid for their participation. 5.5 Results Two sets of data were collected in this experiment: data on the main decision task (the appraisal task), and data related to subject's confidence with the decision made. Two sets of analyses were performed using these two sets of data. A l l statistical analyses were performed using SPSSX (version 6.10 for Windows) unless otherwise stated. 5.5.1 Evaluation of Assumptions of Statistical Tests No missing data were found. The normality assumption was tested first. The descriptive statistics for all dependent variables, including their skewness and kurtosis scores, are reported in Table 5.2 (see page 179). The scores for all variables were within the standard criterion (p>0.001) (Tabachnick and Fidell, 1989). Second, a check was make for possible outliers. No univariate outliers were found at p > 0.001 level. We also calculated the Mahalanobis Distances for the combined variables using the SPSSX regression routine and found no multivariate outliers at p> 0.001 level. Third, the assumption of homogeneity of variance was evaluated. The Cochran's and Bartlett-Box Scores for each dependent variables, together with their corresponding p-values, are summarized in Table 5.3 (see page 180). No violation of univariate or multivariate homogeneity of variance was detected. Finally, the assumption of linearity was assessed through 113 bivariate scatterplots. The plots showed satisfactory linear relationships. In summary, there were no univariate or multivariate within-cell outliers at an alpha > 0.001. The results of the evaluation of assumptions of normality, linearity and homogeneity of variance were satisfactory. 5.5.2 Results on First Impression Bias Twelve items adapted from Denison's (1990) Leadership Index were used for the appraisal task. The reliability of this instrument was tested first. The initial Cronbach's Alpha value was 0.84. One of the 12 items, Question 2 in the questionnaire (Appendix F2), had a low correlation (0.2) with the total. This item was stated in the following way: "Dr. X19 is primarily concerned with getting the job done." We speculate that the low correlation might be have been caused by the inherent ambiguity of the item. The statement can be interpreted in two possible ways. One interpretation is that the person is strongly committed to achieve a previously set goal. It could also mean that the person does not care about other organizational or human aspects, such as the welfare of employees, and that he only wants to see the job get done. In view of its low correlation and confusing nature, the item was dropped from subsequent analyses. The resulting Cronbach's Alpha was 0.87. The scores on the remaining 11 items were aggregated into one score for further analyses. Recall that in Experiment 3 there were two experimental groups and two control groups. Subjects in the experimental groups performed the appraisal task twice, once after they had been exposed to a biased information cue, and once after they had seen (or read) the interview with 1 9 Original name was used in the experiment but is altered here to protect privacy. 114 the person under appraisal (please see Figure 5.3 in page 202). Subjects either viewed the interview in text (text-based OMS condition) or video clips (multimedia OMS condition), depending on their experimental assignment. Subjects who were assigned to the two control groups did not receive the biased information cue. They were asked to perform the appraisal only once, after they had seen the interview (again, either in text transcript form or video clips, depending on their experimental assignment). Data gathered from the two experimental groups (N = 20 for each group) were first analyzed using the repeated-measures ANOVA routine in SPSSX. The dependent variable was the aggregate appraisal score. The between-subject factor was system type, (text-based OMS vs. multimedia OMS). The within-subject factor was time sequence (i.e., first appraisal vs. second appraisal). The results of the analysis are summarized in Tables 5.4a and 5.4b (both in page 181). As shown in Table 5.4a, system type was not significant as a main effect (F( 1,38) =0.27, p=0.605). However, Table 5.4b shows that time sequence was significant as a main effect (F(l,38)=38.32), p=0.000), which suggests that subjects' first and second appraisal outcomes were different. Table 5.4b also shows a significant interaction effect between system type and time sequence (F(l,38)=4.43, p=0.042). This interaction effect is depicted graphically in Figure 5.4 (see page 203), and the corresponding means and standard deviations are reported in Table 5.5 (see page 182). Post-hoc comparisons showed that the changes from the first to second appraisal were significant for both multimedia (t(19)=5.92, p=0.000) and text-based (t(19)=2.86, p=0.01) conditions. However, as represented by the slopes of the two lines in Figure 5.4, the magnitude 115 of change is much larger for multimedia OMS condition. On average, multimedia subjects increased their appraisal score by one point (from -0.33 to 0.67) on the -3 to +3 scale. In contrast, text-based subjects only increased their rating by half a point (from -0.20 to 0.30). Another post-hoc comparison was used to test whether the changes in appraisal scores were indeed different for the two experimental groups. Results of the comparison revealed that the differences between the change in appraisal score was significant (t(38)=2.11, p=0.04). This supports H I , which states that subjects who use multimedia OMS will be able to reduce the influence of first impression bias more than those who use text-based OMS. As a supplemental support for H I , the final appraisal score was examined using a one-way ANOVA. The results of the analysis are reported in Table 5.6 (see page 183). The table shows that the final appraisal scores of the four groups were significantly different (F(3,79)=3.409, p=0.022). The means and standard deviations of the final appraisal scores of the four groups are reported in Table 5.7 (page 184). Three planned comparisons were used to investigate the differences on the final appraisal scores among the two experimental groups and the two control groups. First, the scores of the two control groups were contrasted. There were no significant differences between the two groups (t(76)=0.809, p=0.421). This suggests that without the initial biased cue, the use of different medium, in itself, did not cause any difference in judgement. Second, the score of the text-based experimental group was compared with the score of the text-based control group. Results showed that the final evaluation scores of the text-based experimental group was significantly lower than that of the text-based control group (t(76)=3.07, p=0.003). Third, the multimedia experimental group was compared with its counterpart, the 116 multimedia control group! The two groups were not significantly different (t(76) = 1.54, p=0.129). In summary, the final appraisal of the multimedia OMS experimental subjects (with biased information cue) was not significantly different from the appraisal of their control group counter-parts (without biased information cue). However, the final appraisal of the text-based OMS experimental subjects (with biased information cue) was significantly lower than that of their control group counterparts. This suggests that multimedia OMS, but not the text-based OMS, was able to reduce the influence of the biased information cue. This also provides additional support for H I . 5.5.3 Other Supporting Results for First Impression Bias Besides the key evidence presented above, two additional results provided supporting evidence for the ability of the multimedia OMS to reduce the influence of the biased cue. The first one was the final recommendation that subjects made on whether to renew the contract of the person under appraisal. The second one was subjects' written justifications for their decisions. 5.5.3.1 Final Recommendation As part of the appraisal, subjects in all experimental conditions were asked to make a recommendation whether to renew the contract of the person under appraisal. The options were as follows: 1) Renew his contract for life; 2) Renew for another five years; 3) Renew for two years; 117 4) Renew his contract but place him under probation; 5) Keep him as a scientist but not as the department head; 6) Transfer him to another department; and 7) Terminate his appointments with B C C R C . Table 5.8 (see page 185) reports the percentage of subjects in each experimental condition who recommended the renewal of the target person's contract for life and for another five years (Options 1 and 2 of the above list). Since subjects in the two control groups only performed the appraisal once, there is only one entry for each of these two conditions. Table 5.8 reports that the subjects' recommendations were consistent with the results obtained from using the 12-item appraisal instrument. For the multimedia experimental group, 10% of the subjects recommended renewing the contract for life or for five year in the first appraisal. However, after viewing the interview clips using the multimedia OMS, 40% of the subjects recommended renewing the contract for life or for five years, a four-fold increase. More importantly, the group moved closer to the means of the two control groups (45% and 50%). In contrast, 20% of the subjects in text-based OMS experiment group initially recommended contract renewal for life or for five years. However, after examining the interview transcript, this dropped to 15%. As a whole, this pattern also appears to indicate that the multimedia OMS was more effective in reducing the influence of the biased information cue than the text-based OMS. 5.5.3.2 The Written Justification At the end of the appraisal subjects were asked to provide a written justification for the 118 decision they had made. The written justification provided a useful mechanism to examine the process subjects went through in debiasing or reinforcing their first impression bias. The purpose of this section is to provide some preliminary supporting evidence to the argument that multimedia OMS can be used to reduce first impression bias. Therefore, a detailed process tracing was not performed. Overall, multimedia experimental group subjects, as compared to their text-based counter-parts, tended to express a more positive attitude towards the target person in their written justification. This included good communication skills, being caring, and showing a willingness to listen to employees' problems, etc. In contrast, text-based experimental group subjects mentioned more negative attributes, such as inflexible and autocratic. Most importantly, most text-based subjects heavily emphasized the bad performance indicators. Further, the content of the justifications written by multimedia experimental group subjects were very similar to those written by subjects in the two control groups. The followings two examples demonstrate how subjects reinterpreted the information they had seen. Example 1: Hypertext Question # 3 (How would you go about balancing your research and administrative duties?) In the experiment, nine interview segments, related to the Department Head's view on nine aspects of managing the department, were presented to the subjects. In one of the nine aspects, the Department Head was asked how he would go about balancing his own research and administrative duties. The following was his response: / don't do it very well to be quite honest with you. I very much enjoy doing research. And I guess my reaction or my approach to combining administration and research is to take what I think is an objective opinion or formulate an objective opinion on what is the minimum amount 119 of time that is required to perform my administrative functions. Now, if that is say 10 or 15 hours a week, then I will dedicate that amouni of time plus a little bit more to administrative functions. But what I would try to do is lock that in. I would try to dedicate a given number of hours to administrative functions and hopefully defined hours so that my staff and in fact everybody in the department knows that I'm wearing my administrative hat, let's say the first three hours of every morning or two or three defined days during the week, whatever. Research, on the other hand, then occupies the rest of the time and, of course, research is open-ended. If one gets involved in an experiment and excited in their experiment, then 12 or 14 or 16 hours a day to finish an experiment is not uncommon and it's totally expected. The following are some responses that appeared in the multimedia experimental group subjects' written justifications, reacting to the above statements: "I feel that he makes an excellent scientist and tries his best to balance his administrative duties as well." "He is aware of his own weaknesses, i.e., balancing administrative duties with research and has a plan to combat it with setting an initial schedule but keeping it flexible allowing for the goal of important research." "He has a balanced view of his duties. Although his passion is on research, he realizes that he needs to allocate time for other administrative duties. "He is aware of his strengths and weaknesses as researcher and administrator." The text-based experimental group subjects reacted very differently towards his responses: "He just wants to do his own research. Why hire him as a Department Head to begin with?" "Too much research emphasis and inflexible" "His first love is research, and he seems only willing to allot a number of hours to administration. Research — as many hours as it takes." "A Department Head should be more willing to commit more time for administrative duties. Administrative problems arise at any time. How can a schedule be fixed?" Example 2: Hypertext Questions # 5 (How would you handle disagreements between you and 120 your fellow researchers?) In another example, responding to a question on how he would handle disagreements between his fellow researchers and himself, the Department Head provided the following answer. Well, as much as possible, compromise. It would depend a lot on the basis for the disagreement. If the basis is on what I consider to be a philosophical issue, you know, his style versus my style or her style versus my style, then I think there is a reality that for better or for worse, I am the department chairman and with the chairmanship comes the responsibility for executing the things that need to be done. I always like that quote of my father's. He used to say I'm not always right but I'm always boss. And that is a reality. The individual that does have the leadership position has an obligation to make sure that things get done. Unfortunately, or fortunately, depending on one's point of view, things get done in the style that is adapted by the particular person that fills the chairman's position or chairman's desk at that point in time. If the individual disagrees sufficiently strongly with the management style or with the way things are being done, that individual obviously has the option of finding another organization where things are done more in the style to which he or she wishes them to be done. Now within all of that I think there is an implicit need for communication. The following are sample responses made by the multimedia experimental group subjects. "He will stand up for and defend his own beliefs." "He likes to get the job done, is able to come out with decisions decisively." "He realizes that being the department head he has to make sure that things get done, and that means he has to take a stand." "He is decisive, yet open to communication." The next four are responses from the text-based experimental group subjects. "He is too bossy." "Autocratic, my way or highway." "I would hate to work under him. He doesn 't want to listen to others." "Although he is practical and rational, he has forgotten one key quality: open-minded. 121 Problems such as tension and conflicts between the staff and himself may become obvious during his tenure." The above examples highlighted the two underlying processes of first impression bias discussed, namely reinterpreting and ignoring. In the first example, the Department Head expressed that he would allocate specific hours for administrative duties. He recognized that as a research department head, his main duty is to lead a group of researchers working on a common goal of finding a cure for cancer. Overall, he expressed a balanced view of his job duties. Presumably, the multimedia experimental group subjects were able to sense his commitment and serious intentions from the video clip. These subjects saw him as a reasonable person who recognized his own strengths and weaknesses, but is willing to work hard to further the goals of the department. In contrast, the text-based experimental subjects only focused on parts of what was said and ignored other important information. For example, they focused on the fact that the Department Head was only going to allocate 2-3 hours per day for administrative duties, which they considered insufficient for managing his department. In the task instruction (Appendix F3), subjects were explicitly told that one of the criteria for being a Department Head is that the person must be an active researcher. This is because only an active researcher can understand and appreciate the problems that other researchers face and therefore, is better able to lead the fellow scientists in achieving the mandate and mission of the department. The Department Head often plays a very difficult role. He/she must be an active and productive researcher, and most importantly, he/she has to allocate time to perform 122 administrative duties and to manage and lead the department. Despite this, the text-based experimental group subjects remained convinced that the Department Head needed to allocate more time for Administrative duties. In the second example, text-based experimental subjects focused on the phrase "I'm not always right, but I'm always boss". From the video clip, the multimedia experimental group subjects were able to sense that the Department Head said the above statement in a light-hearted way (he was smiling when he made the above statement). When he went on to say "And that is a reality. The individual that does have the leadership position has an obligation to make sure that things get done", his facial expression showed that he was very serious and committed to this premise. A l l the non-verbal cues, which made his intention clearer and less subject to misinterpretation, were filtered out in the text transcript. Having misinterpreted what was said, text-based experimental group subjects strongly believed that their first impression about this Department Head was correct, and that he was unfit for the position. Also, the text-based experiment group subjects appeared to have ignored several important positive characteristics of the Department Head that were made apparent from what he said above (Example 2). For example, in their written justifications, multimedia experimental group subjects mentioned that the Department Head was open to communication and willing to compromise. (The Department Head explicitly mentioned these, as appeared in the opening and ending statements in Example 2.) In contrast, the text-based experimental group subjects appeared to have ignored these aspects. In summary, subjects' written justifications provided supporting evidence that multimedia 123 OMS is capable of reducing the effect of first impression bias. It also provided preliminary indications on the underlying processes involved in debiasing and reinforcing first impression bias. 5.5.4 Confidence with the Decision Made A 3-item instrument adapted from Aldag and Power (1986) was used to measure the subjects' confidence with their decision. On all three items, subjects were required to rate on a -3 to +3 scale. The reliability test yielded an Cronbach's alpha level of 0.84. The scores on these three items were aggregated into a single score for further analyses. For the two experimental groups, an independent sample t-test was used to compare the differences in subjects' confidence with their decision made associated with their final appraisal. The results of the t-test are reported in Table 5.9 (see page 186). As indicated in the table, there was no statistically significance (t(38) = 1.02,p=0.315) differences between the multimedia OMS and the text-based OMS experimental groups in terms of their confidence with the decision made associated with their final appraisal. This does not support H2a, which states that multimedia OMS will result in more confidence with the decision made on the part of the decision maker than text-based OMS. To examine the change (from the first appraisal to the second appraisal) in confidence with the decision made, a repeated-measures ANOVA was used. The independent variable was the system type (text-based OMS vs. multimedia OMS). The repeated factor was time sequence (i.e., confidence with the first appraisal vs. confidence with the second appraisal). The results of the analysis are summarized in Tables 5.10a and 5.10b (both in page 187). As shown in Table 5.10a, the system type was not significant as a main effect 124 (F(l,32)=0.26, p=0.612). However, Table 5.10b shows that time sequence was significant as a main effect (F(l,32) = 18.35, p=0.000). This suggests that subject's confidence level with the decision made was different between the two appraisals. The means for the first and second appraisals were -0.34 and +0.62 respectively, which indicated that subjects felt more confident with their decision after viewing the interview information in second appraisal. Table 5.10b also shows that the interaction effect between the system type and time was not significant (F(l,32) = 1.10, p=0.302). This means that both text-based and multimedia subjects felt more confident with their decisions after examining the interview information. However, post-hoc Bonferroni comparisons on the change in confidence with decision made showed that while the change in the text-based OMS condition was significant (t( 19) =5.27, p=0.000), the change in the multimedia OMS condition was not significant (t(19) = 1.88, p=0.079). The mean score on the change of confidence with the decision made for text-based OMS was 1.20 (-0.35 to 0.84) and for multimedia OMS was 0.73 (-0.33 to 0.39). In short, the text-based OMS, but not the multimedia OMS, appeared to enhance subject's confidence with the decision. This directly contradicts H2b, which states that multimedia OMS will change decision maker's confidence with his/her decision more from previous level than text-based OMS. 5.5.5 Summary of Results In summary, results of Experiment 3 show that text-based OMS subjects continue to be biased after examining the information in textual form. Only the multimedia OMS was able to reduce the first impression bias. However, contrary to our expectation, text-based OMS enhanced subject's confidence with the decision made more than multimedia OMS did. 125 5.6 Discussion This section discusses the findings of this study and their practical implications. First, the unexpected results on confidence with decision made is examined. Next, the practical significance of the results is investigated. Finally, the practical implications of this study are discussed. 5.6.1 Unexpected Results on Confidence with Decision Made Since multimedia OMS provides more information, it was expected that multimedia subjects would feel that they have received more information. Past research has demonstrated that the amount of available information affects a decision maker's certainty level (e.g. Daft and Macintosh, 1981). Therefore, we expected that multimedia subjects would be more certain about their decision. Based on this argument, it was hypothesized that the use of multimedia OMS will enhance subjects' confidence with their decisions more than text-based OMS. The results showed the opposite: text-based OMS, but not multimedia OMS, enhanced subjects' confidence with their decisions. It was speculated that this was caused by the process of debiasing or reinforcing first impression bias. In the case of multimedia subjects, through the video clips, they had vividly seen an image of the Department Head who possesses most of the good qualities of a leader. However, this image was inconsistent with their first impression, which was created by the bias information cue. This inconsistency led to uncertainty, and thus caused multimedia subjects to feel less confidence with their decisions. On the other hand, as predicted by the theory, text-based subjects were consistently biased by the first impression created. As highlighted by the examples in Section 5.5.3.2, text-126 based subjects unconsciously ignored and reinterpreted some of the information presented after their first appraisal. They did not appear to sense strongly the inconsistency between their first impression and the information presented subsequently. Consequently, after viewing the interview information, the text-based subjects felt more confident than before with their decision. The above explanation is consistent with our argument that multimedia OMS helped subjects to debias the first impression bias. In contrast, text-based OMS not only did not reduce first impression bias, but appeared to further reinforce the bias. 5.6.2 Practical Significance The key finding of the experiment is that multimedia OMS can reduce the influence of first impression bias, and thus a more objective decision can be made. This was indicated by a statistically significant change in multimedia subjects' appraisal scores. As reported in Tables 5.5 and 5.6 (see pages 182 and 183), the average change in the multimedia subject's appraisal score were +1.0 (from -0.33 to +0.67). The change from a negative score to a positive one, together with the fact that the change was statistically significant, shows that multimedia OMS was capable of reducing the influence of first impression bias. However, since the magnitude of change (in this case, +1.0 on a 7-point scale) was small, the practical significance of this result may be questionable. In interpreting this result, one has to consider the "ceiling" effect of the "true appraisal score" for the target person. The appraisal scores given by subjects from the two control groups (in absence of the influence of the biased information cue) could be assumed to represent the "true score" of the target person. The Department Head's ability to manage the department was appraised to be at 0.94 (averaging 0.84 and 1.03, the appraisal scores given by the two control 127 groups). This score represents the "true score" without the influence of a biased information cue. This score was the "ceiling" score (imposed by the information conveyed in the interview information regarding the target person as an effective Department Head) for subjects in the experimental groups. Using the same method, the "floor" was -0.27 (averaging -0.2 and -0.33, the two lowest scores from Table 5.5). Therefore, the actual scale ranges from -0.27 to 0.94 rather than -3 to +3. In light of this, the change in appraisal score of +1.0 was actually fairly substantial (approximately 84% of the range) and arguably has a practical significance. 5.6.3 Implications for Using Qualitative Information for Personnel Decisions This study was designed to investigate the capability of multimedia OMS in reducing the effect of first impression bias. A n appraisal task was chosen as the experimental task. Also, the quantitative performance measures (performance indicators) were used as a biased cue in the experiment. It is important to emphasize that this study does not suggest that qualitative data should be used in place of quantitative data, or that one is more reliable than the other, but rather that both quantitative and qualitative data provide important insights and are a good yardstick for appraisal decisions. Nonetheless, recent research in organizational behavior has shown an increased interest in using qualitative information, such as video interviews, to evaluate managers (see Motowidlo et al., 1992 for a review). This line of research has also demonstrated the validity of video interviews in employee appraisal. For example, Motowidlo and Burnett (in press) asked subjects to view video clips of real managers' simulated job interviews. For those subjects who saw and heard the video clips, ratings of the managers were significantly correlated with the 128 actual on-the-job performance ratings of these managers given by their supervisors. This line of research in organizational behavior has demonstrated that qualitative data, when used appropriately, provide high predictive validity. Further, qualitative data are especially useful for managerial level jobs in which quantitative data on job performance are sometimes difficult to obtain or measure. 5.6.4 Limitations In this experiment, a positive information cue was presented using multimedia after a negative information cue was introduced. It is not clear what the role of multimedia OMS would be if information was presented in another order, i.e., a positive (but false) information cue is presented using multimedia, followed by a negative (but true) information cue. It is reasonable to speculate that the results would be the direct opposite of those observed in this experiment, i.e., multimedia OMS would lead to a more bias negative appraisal. In other words, multimedia is a powerful information presentation method for changing decision makers' initial impressions. One limitation of this experiment that might have confounded the findings is that people tend to identify more with the person when they "see" that person. In this experiment, since multimedia OMS subjects, but not the text-based OMS subjects, saw (in videos) the Department Head under appraisal, it is possible that the multimedia OMS subjects identified more with the Department Head than the text-based OMS subjects. As a result, the multimedia OMS subjects might have been more sympathetic or more serious in performing the appraisal, and thus potentially creating a secondary bias. Recall that in the series of three experiments reported in this dissertation, half of the subjects performed Experiment 2 (reported in Chapter 4) before Experiment 3 (reported in the 129 current chapter), and half of them performed Experiment 3 before Experiment 2. For those subjects who performed Experiment 2 first, they had seen the Department Head on video in Experiment 2 prior to performing Experiment 3. If this secondary bias had indeed occurred, the appraisal outcomes for the two groups of subjects who performed the experiments in two different orders (Experiment 2 first vs. Experiment 3 first) would be different. However, an analysis on the order effect (see Appendix H) revealed no statistically significant order effect. This rules out the possibility that multimedia OMS subjects gave the Department Head a better appraisal because they identified more with the Department Head. In conclusion, the results of this study on the appraisal score, together with the results on the confidence with the decision made and the additional analysis on subjects' written justifications, indicated a consistent pattern: multimedia OMS, but not the text-based OMS, enable the debiasing process and reduce the influence of first impression bias. The theoretical implications of this finding will be discussed next in the Conclusion chapter. 130 CHAPTER 6. CONCLUSIONS 6.1 Introduction This chapter first provides a brief overview of the results and highlights the important findings. Next, the limitations and the contributions of the research are discussed. Lastly, some directions for future research are proposed. 6.2 Overview of Results The overall goal of this research was to determine how multimedia can be used to enhance the capability of text-based Organizational Memory Systems (OMS). Three experiments were conducted. Each focused on one of the difficulties of decision making and investigated how multimedia presentations can be used to alleviate these difficulties. The results of the three experiments are summarized and discussed in this section. 6.2.1 Comprehension Improvement In regard to whether or not multimedia presentations can improve the comprehension of organizational information, the use of multimedia was found to facilitate the retention and subsequent recall of explanative knowledge but not descriptive knowledge. Explanative knowledge is organized facts connected by their underlying functional relationships. Descriptive knowledge consists of isolated facts without an explanation of the relationships between these facts. Further, better retention and recall of explanative knowledge leads to a greater ability to apply it to new situations. Taken together, results indicate that multimedia OMS presents 131 information in such a way that i t facilitates the acquisition of explanative knowledge, and this in turn leads to a greater ability to make correct inferences. The advantage of multimedia OMS in facilitating the retention and recall of explanative knowledge is attributable to the inherent features of multimedia. Specifically, multimedia presentation uses a set of rich language, such as diagrams, graphs, and videos, to organize facts making the functional relationships among them more explicit and easier to understand and remember. These visual representations also create natural redundancies (complementary cues). Complementary cues improve retention and recall because they create more potential links between the information and a decision maker's existing knowledge (Baggett, 1989). 6.2.2 Ambiguity Reduction In regard to the ability of multimedia OMS to help decision makers cope with ambiguous information, we found a task-media fit relationship. First, for less-equivocal tasks, while text-based OMS and multimedia OMS are equally effective in reducing perceived ambiguity levels, text-based OMS is more efficient than multimedia OMS in terms of time required to perform the task. Multimedia OMS provides superfluous information which may distract the decision maker's attention from main issues when dealing with less-equivocal tasks. Second, for equivocal tasks, only multimedia OMS was able to reduce perceived ambiguity levels. When subjects were given a second OMS (different from the first one they used) to perform the task, although both groups of subjects spent an equal amount of time on the task, only those subjects who used multimedia OMS for the second time reported a reduction in ambiguity levels. This clearly indicates that conventional text-based OMS is inferior, compared to multimedia OMS, in supporting equivocal decision making. However, it should 132 be emphasized that this conclusion is based on subjects' self-reported perceived ambiguity levels rather than actual task performance, which is often difficult, if not impossible, to measure when dealing with equivocal tasks (Daft and Lengle, 1986). The ability of multimedia OMS in helping decision makers to cope with ambiguous information is attributable to two of its unique characteristics: rich language and authentic context. Multimedia presentation uses a wide range of symbolic systems (rich language) to capture and describe information. Using this set of rich symbolic systems, a multimedia OMS preserves much of the semantically rich information in organizational interactions, such as body language and facial expression, making information less ambiguous. Further, multimedia presentation also preserves the authentic context surrounding the information, thereby helping decision makers to better understand the context in which the information is conveyed. This task-media fit relationship also provides support for the media richness theory, which posits that while a lean medium is better for less-equivocal tasks, a rich medium is better for equivocal tasks. 6.2.3 First Impression Bias Reduction Multimedia OMS is useful in reducing the influence of first impression bias. In the experiment, although the first impressions of both text-based OMS and multimedia OMS subjects were affected by either one of the OMS provided (one presented information using video clips, and the other presented the same information using text), for multimedia OMS, the magnitude of change was approximately double of that of text-based OMS. The written justification provided by subjects at the end of a decision task also provided important insights into the underlying processes that had occurred. Text-based OMS subjects 133 behaved as predicted by the theory. They either reinterpreted or ignored information that was inconsistent with the first impression formed. This indicates that multimedia OMS helps decision makers reduce their first impression bias. In contrast, text-based OMS does not reduce first impression bias; it may actually even reinforce it. Overall, the results indicate that only multimedia OMS is able to reduce first impression bias. The results also demonstrate that text-based OMS subjects felt more confident with the decision they had made than the multimedia subjects. Presumably, multimedia subjects expressed lower confidence levels because they perceived contradictions between their first impression and the subsequent information they examined. In contrast, text-based subjects saw the subsequent information as supporting or consistent with their first impression. Consequently, they felt more confident about their decision. This also appears to suggest that while multimedia OMS can provide a mechanism to reduce the influence of first impression bias, text-based OMS can reinforce first impression bias. A first impression bias is mediated by two specific processes: reinterpreting and ignoring subsequent inconsistent information. Two unique characteristics of multimedia presentation, rich language and complementary cues, enable multimedia OMS to suppress these two processes, and thus, reduce the influence of first impression bias. First, using a set of rich symbolic languages, multimedia presentation conveys the original meaning of information in a less distorted and less ambiguous way. This reduces the potential for misinterpretation and makes inconsistent information less subject to reinterpretation. Second, multimedia present information in vivid ways using audio and video with embedded complementary cues. The use of complementary cues leads to better retention and 134 retrieval of information, because they create more potential links to knowledge in long-term memory. As a result, the information is better registered in the decision maker's mind, and inconsistent information becomes more difficult to ignore. 6.2.4 Summary In general, the results of this research clearly highlight the limitations of text-based OMS. These limitations are due, in part, to the deficiencies of human beings in processing information, which are manifested through the three difficulties of decision making focused on in this research. Multimedia OMS was found to be useful in alleviating these three difficulties. It achieved this through the use of the three inherently unique characteristics of multimedia presentation, namely complementary cues, rich language, and authentic context. These three characteristics of multimedia presentation allow information to be presented in such a way that three of the difficulties associated with decision making are alleviated. As a result, a decision maker's ability to process information is enhanced. Specifically, complementary cues create more links between the information presented and a decision maker's existing knowledge, and thus improves retention and subsequent recall. Further, using complementary cues, inconsistent information is also better registered in a decision maker's mind. This reduces the influence of first impression bias. The use of rich language, such as graphs and figures, enables information to be organized in such a way that the functional relationships among different pieces of information become more explicit and easier to understand and remember. Rich language also preserves much of the semantically rich information in social interactions making the information less ambiguous 135 and less subject to misinterpretation. This helps decision makers to better cope with ambiguous information. Furthermore, since rich, language reduces ambiguity, it also reduces the possibilities of reinterpreting inconsistent information to fit with the biased first impression. Hence, the use of rich language also reduces the influence of first impression bias. Finally, authentic context in multimedia presentation allows the context surrounding the information to be preserved. This helps decision makers to better understand the context in which the information is conveyed. This therefore, reduces the possibilities of misinterpreting information, making it easier for the decision maker to cope with ambiguous information. 6.3 Limitations Several limitations exist in this research and warrant some discussion. Like all social science research, this study faces three conflicting objectives (Cook and Campbell, 1979; McGrath, 1982). These objectives are: 1) to generalize the findings to the respective populations (external validity), 2) to control the variables that may influence the outcome (internal validity), and 3) to examine the phenomenon of interest in a realistic setting (realism). However, since these three objectives are mutually conflicting, it is impossible to simultaneously attain all three objectives (Benbasat, 1990; Cook and Campbell, 1979). In this research, a laboratory experimental approach was chosen to investigate the research questions. This necessarily entailed gaining internal validity at the expense of lowering external validity and realism. The limitations to be discussed, which are shared by the three experiments reported, are related to the external validity and realism of the research. They are: subjects' background, realism of the simulated experimental system and experimental tasks, and 136 quality of the multimedia presentation. 6.3.1 Subjects' Background and the Use of Subjects in Three Experiments Subjects were recruited through campus advertisements, and all were college students. Essentially, this formed a convenience sample. The behaviors exhibited by college students during the experiment may not be the same as those of real-world managers. As such, one may question whether the findings are generalizable to real organizational settings. While we acknowledge this limitation, we do not see it as a serious threat. The type of tasks involved in the experiments are usually performed by organizational members who have a university level education. This resembles our subjects' background. Further, to motivate our subjects to work diligently, we told our subjects at the beginning of the experiment that each of the three best performers would receive an additional cash award. However, it is still possible that their motivational levels were not as high as that of a real-world manager, who faces potentially serious consequences if a wrong decision is made and high rewards if a proper decision outcome is reached. Another limitation related to subject selection is the use of the same subjects in all three experiments which might have resulted in fatigue on the part of the subjects. The decision to require all subjects to perform all three experimental tasks was based on a practical consideration. First, all subjects had to be trained on how to use the system. Second, all subjects had to be familiarized with the organization, which was done in Experiment 1. Therefore, the decision was whether to have 1) two sets of subjects, one set performing only Experiments 1 and 2 and the other set performing only Experiments 1 and 3, or 2) all subjects perform all the three experiments. Since twice the number of subjects would have been required 137 if two sets of subjects were used, the latter alternative was chosen. 6.3.2 Realism of the Simulated System and Tasks Every effort was made to ensure the realism of the experiments. This includes the use of authentic experimental materials from a real organization. Nonetheless, several inherent weaknesses might have discounted the desire of realism. First, to ensure that subjects were able to complete the tasks within a predetermined length of time, the information stored in the experimental OMS was reduced to a minimal level. In a more realistic organizational situation, the amount of information stored in a real OMS is more than that of the simulated OMS in the experiment, though decision makers are unlikely to examine all of it. Second, with a real OMS that stores large amounts of information, decision makers have to rely heavily on search mechanisms, such as keyword or content search. The OMS used in this research did not have a search mechanism. Subjects had to rely on their recollection to identify where they had seen the information. They then had to "visit" the particular page and search for the required information. As a result, the cognitive demand in a using a real-world OMS might be different from that required by the experiments. This should be taken into consideration in interpreting our findings. This also suggests that more empirical research is required to establish the benefits of multimedia OMS. 6.3.3 Quality of Multimedia Presentation A l l video clips used in the Multimedia OMS were produced "in-house", using the existing equipment available. It was not a professional multimedia production with special effects, such as attention-catching animation and titles and professional editing. Further, the hardware and 138 software used for digitizing video clips were low-end products due to budget constraints. A real-world organization interested in producing a multimedia OMS, is likely to invest substantially more resources in production and equipment. Nonetheless, this limitation contributes to creating a more rigorous condition under which the expected benefits of multimedia OMS were tested. 6.3.4 Separation of Effects One limitation of this dissertation, apart from external validity and realism issues, is that this research examined multimedia OMS only at a macro level. Although three unique characteristics of multimedia were identified and used to explain the benefits of multimedia OMS in alleviating three of the difficulties of decision making, the study did not explicitly test these three characteristics. It is not clear what the relative importance of each of these characteristics is. Conceptually, it is easy to differentiate these three characteristics from each other. However, it is not as easy to isolate them from each other in a practical implementation of multimedia OMS. Thereby, testing each characteristic can be a difficult challenge. This is a potential future research area. 6.4 Theoretical Contributions of the Research In spite of the limitations discussed, this dissertation provides several contributions to OMS research. It also contributes to the organizational communication and the psychology literature. 6.4.1 Contributions to OMS Research The major contribution of this research is on describing a set of mechanisms on how 139 multimedia can be used to enhance text-based OMS. Drawing from social and cognitive psychology research, we first developed a theory-based understanding of the deficiencies of human beings in processing information and identified three of the difficulties of decision making associated with processing text-based information. Next, by reviewing the literature on multimedia, we distinguished between three unique characteristics of multimedia presentation that are believed to be capable of alleviating the three difficulties identified. By integrating the literatures in social and cognitive psychology, and research findings in multimedia, we developed three sets of hypotheses. Each of them described and predicted how the three unique characteristics of multimedia can be used to solve one of the three difficulties identified. The hypotheses were tested in the domain of organizational memory using three controlled laboratory experiments. The results are useful as a basis for future research in OMS and decision making in general. 6.4.2 Contributions to Organizational Communication Research The study provides support for the controversial Media Richness Theory (Kinney and Dennis, 1994; Rice, 1992) in that the theory was useful in describing and predicting behaviors of decision makers interacting with computers. Most studies have tested the media richness theory in the context of human-to-human communication (mediated by technology such as computers or telephone). In applying the theory to the context of human-computer interaction, the study eliminated the social dimension of human-to-human interaction that has been found to be a key factor in affecting the predicted outcome of the theory. This study examined the fit between media and task from two perspectives, namely, efficiency and effectiveness. While a medium can be very efficient in terms of time required 140 to perform a task, it may not be very effective in conveying the required information to the decision maker. The results of the study clearly point out that these two perspectives must be simultaneously considered in evaluating the validity of the Media Richness Theory. 6.4.3 Contributions to Research in Psychology The psychology literature has portrayed first impression bias as a virtually "inherent" human bias. This study found that the influence of first impression bias can be reduced. The multimedia format presents information in a vivid way so that decision makers are more likely to perceive inconsistencies and less likely to ignore or reinterpret them. It needs to be emphasized that in this study multimedia only reduced the influence of first impression bias rather than eliminating it totally. Also, if the information presented in the multimedia format is biased, then it is likely that multimedia will create bias rather than reduce bias. 6.5 Practical Contributions of the Research The practical contributions of this research provide guidelines on how to effectively store and present information using OMS. For example, our results demonstrate that while descriptive information (isolated facts) can be effectively conveyed using text, explanative information (facts with explicit functional relationships among them) is better conveyed using multimedia, which makes the functional relationship more explicit and easier to understand and remember. Research has established that while lower-level organizational decision tasks are more structured and less equivocal, higher-level organizational decisions are more unstructured and highly equivocal (Daft and Lengle, 1984; Trevino, Daft, and Lengle, 1987). Our results demonstrate that for less-equivocal tasks, text-based OMS is preferable, whereas, for equivocal 141 tasks, multimedia OMS is better. This provides another guideline for capturing and presenting information using OMS: information related to lower-level organizational decision making that is more structured and less-equivocal can be stored and presented using text; information related to higher-level organizational decision making that is more unstructured and highly equivocal should be captured and presented using multimedia. Sandoe et al. (1991) suggested that completeness of information captured is one dimension of OMS. Rao and Goldman-Segall (1995) proposed that "representativeness" of the information captured is one criterion for evaluating OMS. Completeness refers to the capturing of sufficient details to provide a useful picture of the history. "Representativeness" refers to the extent to which information captured is undistorted and truly represents the context. There are several ways to preserve rich contextual information and to make the information captured more complete and representative. The best solution is, of course, that the decision maker is present when the event occurs. Clearly, this is not always possible. Besides, research has shown that human memory is faulty and biased (e.g., Neisser, 1976). Another alternative is to delegate an organizational member to take charge of recording all organizational information and events. This person can enrich the information captured in text by adding comments on emotion, body language and facial expression. There are two drawbacks to this approach. First, it may be very costly. Second, the comments added are based on one or a few individuals' interpretations, which could be biased or inaccurate. With audio and video capabilities, multimedia presents another alternative. A video recording of an event represents the best alternative to being physically present when the event occurs. Video captures most of the information, including nonverbal cues, in organizational 142 members' interactions without too much distortion. The growing trends of audio and video conferencing demonstrate that they will likely be a common part of an organization's activity in the future. Capturing video and audio information may not be as inconvenient as was once believed. Although it is still expensive to keep audio and video information, in terms of computer storage cost, this cost may be offset by elimination of the labor cost of manually transcribing and commenting information in textual form. In addition, the cost of storage is expected to decrease rapidly. It needs to be emphasized again that for certain decision tasks (e.g., a less-equivocal decisions), multimedia may provide too much "noise" and superfluous information. Further, as Rao and Goldman-Segall (1995) point out, like any other medium, video can be taken out of context (also see Goldman-Segall, 1994). Video (film) editing can be used to reflect or distort reality (Dancyger, 1993). Rao and Goldman-Segall (1995) point out that there are several problems associated with multimedia presentation, e.g., single theme and potential bias. These are related to the technical aspect of multimedia OMS implementation and is beyond the scope of this research. Interested readers are directed to Rao and Goldman-Segall (1995) and Goldman-Segall (1994). 6.6 Future Research Using multimedia to preserve organizational memory is still a relatively new concept. Much research needs to be done before the benefits of multimedia OMS can be realized. One immediate research direction is to more closely examine the three characteristics of multimedia identified, namely complementary cues, rich language, and authentic context. Although it is 143 difficult to separate the three characteristics in a given multimedia OMS implementation, this is by no means an impossible task. As an example, consider the case of describing an organizational structure, as in one of the video clips used in our OMS. To examine the benefit of complementary cues, two versions of multimedia OMS need to be contrasted. In one version, the verbal information is explicitly linked with the visual information to create strong complementary cues. In the other version, the verbal information is not explicitly linked to the visual information, thus creating weaker complementary cues. Specifically, in the case of describing the organizational structure, the first system would show a person explaining the information with the aid of an organizational chart. This person would draw the chart as he/she was talking and would constantly refer back to the chart by pointing to the position that he/she refers to in his/her discussion. In this case, the explicit reference between verbal and visual information makes extensive use of complementary cues. In the second system, an organizational chart could be drawn before the discussion. The person who presents the information would say the exact same script from the first OMS. However, this person would not point to the chart, which is always projected on the side of the speaker. In essence, no explicitly referential link is made between the verbal and visual information, thus creating weaker complementary cues. A second interesting future research direction is to compare other means of capturing and presenting organizational memory. In our research, two very basic forms of OMS were chosen for the comparison. It would be beneficial to examine whether the findings would hold for other forms of OMS. For example, commentary can be added to text transcripts of a text-based OMS 144 to enrich the system to such a point that the text-based OMS is functionally equivalent to the multimedia OMS, which contains video clips. Comparing the two systems would provide answers as to whether the benefits of multimedia are indeed derived from the three unique characteristics, as proposed, or from the fact that multimedia simply presents information in a more "natural" form. A third possible future research direction is to build on this research and develop a framework for classifying organizational information for studying the comprehensibility of organizational information. The explanative vs. descriptive classification used in the research was a preliminary one. Although the classification is a useful one, it lacks a strong theory base. Future research is required to refine and more rigorously tie the classification scheme to a theory. The notion of declarative versus procedural knowledge used by Artificial Intelligence researchers (Anderson, 1985) might be a useful starting point. The classification between declarative and procedure knowledge is strongly grounded in cognitive psychology research, and much research has been done using this classification scheme. By adopting a more commonly used and accepted classification scheme, OMS research not only can draw from some of the research findings in Artificial Intelligence related to this classification scheme, but also gains more validity because the classification is based on strong theory. Another possible research direction is to closely examine the role of multimedia OMS in changing decision makers' opinions. Our research found that first impression bias can be reduced by presenting inconsistent information in multimedia format. However, it would seem reasonable to speculate that if multimedia were used to present biased information, it may have an overall negative effect. More empirical research is needed to test this phenomenon. 145 Along the same line, multimedia appears to be a powerful means of changing decision makers' opinions. If used properly, it can be a useful tool for decision makers. However, since organizational memory is stored digitally, there are data integrity and security concerns associated with storing this information. As a worse case scenario, the history of an entire organization can be altered by a single organizational member. Both technical and ethical standards have to be developed to safe guard the integrity of information stored in multimedia OMS. Finally, it would be fruitful to replicate the three experiments in a cross-cultural setting. Specifically, in Experiment 1, one of the two perspectives (cognitive perspective) was not supported by the results. The reason was that the underlying assumption of the cognitive perspective about verbal recoding may not hold with university students who have high reading skills. This perspective may hold better with a sample of subjects who possess varying levels of English reading comprehension based on language background differences. Also, research has shown that people from the Eastern culture is better than people from the Western culture in terms of processing visual information (e.g., Schmitt, Pan, and Tavassoli, 1994). This suggests that multimedia may be more beneficial to people from the Eastern culture then people from the Western culture. Cross-cultural research has shown that the ability to perceive and decode non-verbal cues cannot be generalized across different cultures (Forgas, 1985). Experiments 2 and 3 make extensive use of nonverbal information. It would be interesting to examine whether the findings obtained in this research hold in different cultural settings. As businesses grow internationally, they will have to cope with issues associated with organizational members coming from different 146 cultural backgrounds and OMS will have to be designed with this consideration in mind. In conclusion, this dissertation has demonstrated that multimedia can be used to further enhance the capability of the current text-based OMS to better support decision making. Although cost and technical difficulties may still prohibit the wide-spread adoption of multimedia technology for preserving organizational memory, this will change as organizations realize the advantages of using the technology. We see this research effort as an important step in this direction. 147 R E F E R E N C E S Ackerman, M . "Definitional and Contextual Issues in Organization and Group Memories." Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences, pp. 191-200. (eds.) Nunamaker, J. F . Jr. and Sprague. R . H . Jr. Vol 4. Los Alamitos, C A : IEEE Computer Society Press, 1994. Adams, D. A . , Nelson, R.R. and Todd, P .A . "Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication." MIS Quarterly 16 2 (June 1992): 227-247. Aldag, R. J. and Power, D.J . "An Empirical Assessment of Computer-Assisted Decision Analysis." Decision Sciences 17 4 (1986): 572-588. Anderson, J. R. Cognitive Psychology and Its Implications. New York: Freeman, (2nd ed) 1985. Anderson, J. R. Language, Memory, and Thought. Hillsdale, NJ . : Erlbaum, 1976. Argyle, M . Salter, V . , Nicholson, H . , Williams, M . , and Burgess, P. "The Communication of Inferior and Superior Attitudes by Verbal and Non-verbal Signals." British Journal of Social and clinical Psychology 9 (1970): 222-231. Argyle, M . Alkema, F. and Gilmour, R. "The Communication of Friendly and Hostile Attitudes by Verbal and Nonverbal Signals." European Journal of Social Psychology 1 (1971): 385-402. Argyris, C. and Schon, D . A . Organizational Learning: A Theory of Action Perspective. M A : Addison-Bass, Inc., 1978. Asch, S. E . "Forming Impressions of Personality." Journal of Abnormal and Social Psychology 41 (1946): 1230-1240. Baddeley, A . D. "Working Memory and Reading." Processing of Visible language, Kolers, P. A . , Wrolstad, M . E . , and Bouma, H . (eds.) (Vol. 1). New York: Plenum Press, 1979. Baggett, P. and Ehrenfeucht, A . "Encoding and Retaining Information in the Visuals and Verbals of an Educational Movie." Educational Communication and Technology Journal 31 1 (1983): 23-32. Baggett, P. and Ehrenfeucht, A . "Information in Content Equivalent Movie and Text Stories." Discourse Processes 5 (1982): 73-99. 148 Baggett, P. "Understanding Visual and Verbal Messages." Knowledge Acquisition from Text and Pictures, pp. 101-124. Mandl, H . and Levin, J. (eds.) Amsterdam, Netherlands: Elsevier, 1989. Barclay, D. Higgins, C A . and Thompson, R. "The Partial Least Squares (PLS) Approach to Causal Modelling: Personal Computer Adoption and use as an Illustration." Technology Studies ((in press): Barrick, M . R. and Mount, M . K . "Autonomy As A Moderator of the Relationships Between the Big Five Personality Dimensions and Job Performance." Journal of Applied Psychology 78 (1993): 111-118. Barrick, M . R. and Mount, M . K . "The Big Five Personality Dimensions and Job Performance." Personnel Psychology AA (1991): 1-26. Beagles-Roos, J. and Gat, I. "Specific Impact of Radio and Television on Children's Story Comprehension." Journal of Educational Psychology 75 1 (1983): 128-137. Bell, N . Visualizing and Verbalizing for Language Comprehension and Thinking. California: Academy of Reading Publications, 1991. Benbasat, I. "Laboratory Experiments in Information Systems with a Focus on Individual: A Critical Appraisal." pp. 33-47. Benbasat, I. (ed.), Harvard Business School, 1990. Brown, D. P. "Auding as the Primary Language Ability." Unpublished Ph.D. Dissertation, Stanford University, 1954, Brown, J. S., Collins, A . and Duguid, P. "Situated Cognition and the Culture of Learning." Educational Researcher 18 1 (1989): 32-42. Carroll, J. M . , Mack, R . L . , Lewis, C . H . , Grishckowski, N . L . , and Robertson, S.R. "Exploring Exploring a Word Processor." Human-Computer Interaction 1 3 (1985): 283-307. Carroll, J. M . and Mack, R . L . "Learning to Use a Word Processor: By Doing, By Thinking, and By Knowing." Human Factors in Computing Systems, pp. 13-51. Thomas, J. and Schneider, M . (eds.) Norwood, NJ: Ablex, 1984. Cognitive and Technology Group at Vanderbilt. "Anchored Instruction and Its Relationship to Situated Cognition." Educational Researcher 19 3 (1990): 2-10. Cognitive and Technology Group at Vanderbilt. "Examining the Cognitive Challenges and Pedagogical Opportunities of Integrated Media Systems: Toward a Research Agenda." Journal of Special Education Technology 12 2 (Fall 1993a): 118-124. 149 Cognitive and Technology Group at Vanderbilt. "Integrated Media: Toward a Theoretical Framework for Utilizing Their Potential." Journal of Special Education Technology 12 2 (Fall 1993b): 71-85. Cohen. Statistical Power Analysis for the Behavioral Sciences. 1988. Compeau, D . R. and Higgins, C A . "Application of Social Cognitive Theory to Training for Computer Skills." Information Systems Research 6 2 (June 1995): 118-143. Conrad, R. "Acoustic Confusions in Immediate Memory." British Journal of Psychology 55 (1964): 75-84. Cook, T. D. and Campbell, D.T. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston, M A : Houghton Mifflin, 1979. Crothers, E . J. Paragraph Structure Inference. Norwood, NJ: Ablex, 1979. Cyert, R. M . and March, J .G. A Behavioral Theory of the Firm. Englewood Cliffs, NJ: Prentice-Hall, 1963. Daft, R. L . and Lengel. "Information Richness: A New Approach to Manager Information Processing and Organization Design." Research in Organizational Behavior, pp. 191-233. Staw, B. and Cummings, L . L . (Eds.) Greenwich, Conn.: JAI Press, 1984. Daft, R. L . , Lengel, R . H . and Trevino, L . K . "Message Equivocality, Media Selection and Manager Performance." MIS Quarterly (September 1987): Daft, R. L . and Lengel R . H . "Organizational Information Requirements, Media Richness and Structural Design." Management Science 32 5 (May 1986): 554-571. Daft, R. L . , and Macintosh, N . B . "A Tentative Exploration into the Amount and Equivocality of Information Processing in Organizational Work Units." Administrative Science Quarterly 26 2 (June 1981): 207-224. Daft, R. L . and Weick, K . E . "Toward a Model of Organizations as Interpretation Systems." Academy of Management Review 9 (1984): 284-295. Dancyger, K . The Technique of Film and Video Editing. Boston, M A : Focal Press, 1993. Danks, J. "Comprehension in Listening and Reading: Same or Different?" Reading and Understanding, pp. 1-39. Danks, J. and Pezdek, K . (eds.) Newark, D E : International Reading Association, 1980. 150 Dannenberg, R. and Blattner, M . Multimedia Interface Design. New York: A C M Press, 1992. Davis, F . D . "Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology." MIS Quarterly 13 3 (Sept 1989): 319-340. Davis, F . D. "Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology." MIS Quarterly 13 3 (Sept 1989): 319-340. Davis, F . D . , Bagozzi, R.P. and Warshaw, R.P. "User Acceptance of Computer Technology: A Comparison of Two Theoretical Models." Management Science 35 8 (Aug 1989): 982-1003. Davis, S. A . and Bostrom, R.P. "Training End Users: A n Experimental Investigation of the Roles of the Computer Interface and Training Methods." MIS Quarterly 17 1 (1993): 61-85. Delclos, V . R. and Hartman, A . "The Impact of an Interactive Multimedia System on the Quality of Learning in Educational Psychology: A n Exploratory Study." Journal of Research on Computing in Education 26 1 (Fall 1993): 83-93. Denison, D. R. Corporate Culture and Organizational Effectiveness. New York: Wiley, 1990. Ekman, P. and Friesen, W . V . "Detecting Deception From the Body or Face." Journal of Personality and Social Psychology 29 (1974): 288-298. E l Sawy, O. A . , Games, G . M . and Gonzalez, M . V . "Preserving Institutional Memory: The Management of History as an Organizational Resource." Academy of Management Best Paper Proceedings 37 (1986): 118-122. Fiske, S. T. and Taylor, S.E. Social Cognition. New York: McGraw H i l l , 1991. Forer, B. R. "The Fallacy of Personal Validation: A Classroom Demonstration of Gullibility." Journal of Abnormal and Social Psychology 44 (1949): 118-123. Forgas, J. P. Interpersonal Behaviour: The Psychology of Social Interaction. Sydney, Australia: Pergamon Press, 1985. Galbraith, J. R. Designing Complex Organizations. Reading, M A : Addison-Wesley, 1973. Goldberg, L . R. "The Development of Markers for the Big Five Factor Structure." Psychological Assessment 4 (1992): 26-42. 151 Goldman-Segall, R. "Challenges Facing Researchers Using Multimedia Data: Tools for Layering Significance." Computer Graphics 28 1 (Feb, 1994): 48-51. Goldman-Segall, R. Personal Communication. 1995. Hannafin, M . J. "Emerging Technologies, ISD, and Learning Environments: Critical Perspectives." Educational Technology Research and Development 40 1 (1992): 49-63. Hansen, E. J. "Interactive Video for Reflection: Learning Theory and a New Use of the Medium." Educational Technology 29 7 (July 1989): 7-15. Hardyck, C D . and Petrinovtich, L .R . "Subvocal Speech and Comprehension level As a Function of the Difficulty Level of Reading Material." Journal of Verbal Learning and Verbal Behavior 9 (1970): 647-652. Hayes, D. D . , Kelly, S.B. and Mandel, M . "Media Differences in Children's Story Synopses: Radio and Television Contrasted." Journal of Educational Psychology 78 5 (1986) 341-346. Hayes, D . S. and Kelly, S.B. "Young Children's Processing of Television: Modality differences in the Retention of Temporal Relations." Journal of Experimental Child Psychology 38 (1984): 505-514. Hendrickson, A . R., Massey, P .D. and Cronan, T.P. "On the Test-retest reliability of Perceived usefulness and Perceived ease." MIS Quarterly 17 2 (June 1993): 227-230. Hollingshead, A . B . , McGrath, J.E. and O'Connor, K . M . "Group Task Performance and Communication Technology: A Longitudinal Study of Computer—Mediated Versus Face-to-Face Work Groups." Small Group Research 24 3 (August, 1993): 307-333. Hough, M . H . , Eaton, N . K . , Dunnette, M . D . and McCloy, R . A . "Criterion-related Validities of Personality Constructs and the Effects of Response Distortion on those Validities." Journal of Applied Psychology 75 (1990): 581-595. Howard, D. V . Cognitive Psychology. New York, N Y : Macmillan Publishing, 1983. Huber, G . P. "A Theory of the Effects of Advanced Information Technologies on Organization Design, Intelligence, and Decision making." Academy of Management Review 15 1 (1990): 47-71. Huber, P. G. "Organizational Learning: The Contributing Processes and the Literatures." Organizational Science 2 1 (Feb, 1991): 88-115. 152 Igbaria, M . Guimaraes, T. and Davis, G.B. "Testing the Determinants of Microcomputer Usage via A Structural Equation model." Journal of Management Information Systems 114 (Spring 1995): 87-114. Jarvenpaa, S. "The Effect of Task and Graphical Format Congruence on Information Processing Strategies and Decision making Performance." Management Science 35 3 (March 1989): 285-303. Johnson-Laird, P. N . Mental Models. Cambridge, M A : Harvard University Press, 1983. Jones, E. E . and Goethals, G. "Order Effects in Impression Formation: Attribution Context and the Nature of the Entity." Attribution: Perceiving the Causes of Behavior, Jones, E . E . et al. (Eds.)ed. Morristown, N . J . : Silver Burdett/General Learning Press, 1972. Jones, E. E . , Rock, L . , Shaver, K . G . , Goethals, G.R. and Ward, L . M . "Patterns of Performance and Ability Attribution: An Unexpected Primacy Effect." Journal of Personality and Social Psychology 10 (1968): 317-341. Kahneman, D . Attention and Effort. Englewood Cliffs, NJ: Prentice-Hall, 1973. Kahneman, D . Slovic, P. and Tversky, A . Judgement Under Uncertainty: Heuristics and Biases. Cambridge, M A : Cambridge University Press, 1982. Keegan, W. J. "Multinational Scanning: A Study of information Sources Utilized by Headquaters Executives in Multinational Companies." Administrative Science Quarterly 19 (1974): 411-421. ' Kieras, D. E . and Bovair, S. "The Role of a Mental Model in Learning to Operate a Device." Cognitive Science 8 (1984): 255-273. Kinney, S. T. and Panko, R.R. "Dimensions of Conference Room Meetings." Working Paper, College of Business Administration, University of Hawaii. March 1995. Kinney, S. T. and Watson, R.T. "The Effect of Medium and Task on Dyadic Communication." Proceedings of International Conference on Information Systems, pp. 107-117. 1992. Kinney, S. T. and Dennis, A . "Re-evaluating Media Richness Theory: A Test of the Task-Media Fit Hypothesis." Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences, pp. 21-30. Nunamaker, J. F. Jr. and Sprague, R . H . Jr. 4. Los Alamitos, California: IEEE Computer Society Press, 1994. Kintsch, W. and Van Dijk, T. A . "Toward a Model of Text Comprehension and Production." Psychological Review 85 (1978): 363-394. 153 Kinzie, M . B. "Requirements and Benefits of Effective Interactive Instruction: Learner Control, Self-Regulation, and Continuing Motivation." Educational Technology Research and Development 38 1 (1991): 5-21. Kleiman, G . M . "Speech Recoding in Reading." Journal of Verbal Learning and Verbal Behavior 14 (1975): 323-339. Kosslyn, S. M . Image and Mind. Cambridge, M A : Harvard University Press, 1980. Kozma, R. "Implications of Instructional Psychology for the Design of Educational Television." Educational Communications and Technology Journal 34 (1986): 11-19. Kozma, R. B. "Learning with Media." Review of Educational Research 61 2 (1991): 179-211. Lana, R. E . "Three Theoretical Interpretations of Order Effects in Persuasive Communications." Psychological Bulletin 61 (1964): 314-320. Larkin, J. H . and Simon, H . A . "Why a Diagram is (Sometimes) Worth Ten Thousand Words." Cognitive Science 11 1 (Jan 1987): 65-100. Levie, W. and Lentz, R. "Effects of Text Illustrations: A Review of Research." Educational Communication and Technology Journal 30 4 (1982): 195-232. Levy, B. A . "Reading: Speech and Meaning Processes." Journal of Verbal Learning and Verbal Behavior 16 (1977): 623-638. Lim, K . H . , Ward, L . M . and Benbasat, I. "An Empirical Study of Mental Models and Computer System Learning." The University of British Columbia, Faculty of Commerce and Business Administration Working Paper 94-MIS-009. 1995. Lim, L . H . "Impact of Group Support Systems on Judgement Biases: A n Experimental Investigation." Unpublished Doctoral Dissertation, The University of British Columbia, 1995, Logan, G . D . "Toward A n Instance Theory of Automatization." Psychological Review 95 (1988): 492-527. Luchins, A . S. "Experimental Attempts to Minimize the Impact of First Impressions." The Order of Presentation in Persuasion, Hovland, C. (ed.) New Haven: Yale University Press, 1957. Makita, K . "The Rarity of Reading Disability in Japanese Children." American Journal of Orthopsychiatry 38 (1968): 599-614. 154 Markus, H . "Self-Schemata and Processing Information About the Self." Journal of Personality and Social Psychology 35 (1977): 63-78. Markus, M . L . "Towards a 'Critical Mass' Theory of Interactive Media: Universal Access, Interdependence and Diffusion." Communication Research 14 5 (1987): 491-511. Mayer, R. E. "Can Advance Organizers Influence Meaningful Learning?" Review of Educational Research 49 (1979): 371-383. Mayer, R. E. "Structural Analysis of Science Prose: Can We Increase Problem-Solving Performance?" Understanding Exposition Text: A Theoretical and Practical Handbook for Analyzing Exploratory Text, pp. 65-87. Britton, B. K . and Black, J.B. (eds.) Hillsdale, NJ: Lawrence Erlbaum, 1985. Mayer, R. E. "Structure Analysis of Science Prose: Can We Increase Problem-Solving Performance?" A Theoretical and Practical Handbook for Analyzing Exploratory Text, Britton, B. K . and Black, J.B. Hillsdale, New Jesey: Lawrence Erlbaum Associate, 1985. McGrath, J. E . "Dilemmatics: The Study of Research Choices and Dilemmas." pp. 69-102. McGrath, J. E . , Martin, J, and Kulka, R . A . (eds.). Berverly Hills, C A : Sage, 1982. McGrath, J. E . and Hollingshead, A . B . "Putting the 'Group' Back in Group Support Systems: Some Theoretical Issues about Dynamic Processes in Groups with Technological Enhancements." Group Support Systems: New Perspectives, pp. 78-96. Jessup, L . M . and Valacich, J.S. (eds.) New York: Macmillan, 1993. Miller, G. A . "The Magical Number Seven Plus or Minus Two: Some Limits of Our Capacity for Processing Information." Psychological Review 63 (1956): 81-97. Mintzberg, H . The Nature of Managerial Work. N Y : Harper and Row, 1972. Moore, G. and Benbasat, I. "Development of an Instrument to Measure the Perceptions of Adopting an Information technology Innovation." Information Systems Research 2 3 (1991): Moore, G . C. and Benbasat, I. "Integrating Diffusion of Innovations and Theory of Reasoned Action Models to Predict Utilization of Information Technology by End-Users." (eds.) Kautz, K . and Pries-Heje, J. Chapman and Hall, 1995. Morrison, J. "Team Memory: Information Management for Business Teams." Proceedings of the Twenty-Sixth Hawaii International Conference on System Sciences, pp. 122-131. (eds.) Nunamaker, J. F. Jr. and Sprague, R . H . Jr. Vol 4. Los Alamitos, C A : IEEE Computer Society Press, 1993. 155 Motowidlo, S. J. and Burnett, J.R. "Aural and Visual Sources of Validity in Structured Employment Interviews." Academy of Management ((in press): Motowidlo, S. J . , Carter, G.W. , Dunnette, M . D . , Tippins, N . , Werner, S., Burnett, J.R., and Vaughan, M . J . "Studies of the Structured Behavioral Interview." Journal of Applied Psychology 77 (1992): 571-587. Mount, M . K . , Barrick, M . R . and Strauss, J.P. "Validity of Observer Ratings of the Big Five Personality Factors." Journal of Applied Psychology 79 2 (1994): 272-280. Neisser, U . Cognition and Reality. San Francisco: Freeman, 1976. Nisbett, R. and Ross, L . Human Inference: Strategies and Shortcomings of Social Judgment. Englewood Cliffs, NJ: Prentice-Hall, 1980. Norman, D. A . , Rumelhart, D .E . and the L N R Research Group. Explorations in Cognition. San Francisco: Freeman, 1975. Norman, D . A . and Bobrow, D . G . "On Data-limited and Resource-limited Processes." Cognitive Psychology 7 (1975): 44-64. Norman D . A . and Rumelhart, D. E . "A System for Perception and Memory." Models of Human Memory, Norman, D. A . (ed.) New York: Academic Press, 1970. Paivio, A . "Imagery and Deep Structure in the Recall of English Nominalizations." Journal of Verbal Learning and Verbal Behavior 10 (1971): 1-12. Panko, R. "Managerial Communication Patterns." Journal of Organizational Computing 2 1 (1992): 95-122. Panko, R. R. and Kinney S.T. "Work Relationships and Media." Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences, Nunamaker, J. F . Jr. and Sprague. R . H . Jr. (eds). Los Alamitos, C A : IEEE Computer Society Press, 1994. Park, I. and Hannafin, M . J . "Empirically-Based Guidelines for the Design of Interactive Multimedia." Educational Technology Research and Development 41 3 (1993): 63-85. Pearson, D . and Fielding, L . "Research Update: Listening Comprehension." Language Arts 59 6 (1982): 617-629. Permack L . P . , and Wanous, J.P. "A Meta-Analysis of Realistic Job Preview Experiments." Journal of Applied Psychology 70 4 (1985): 706-719. 156 PLS-Graph. Version 2.9.1. Calgary, Alberta: Facutly of Management, The University of Calgary, 1995. Popkin, J. and Cushman, A . Integrated Document Management - Controlling a Rising Tide. Stamford, Conn.: Gartner Group, 1993. Pressley, M . "Imagery and Children's Learning: Putting the Pictures in Developmental Perspective." Review of Educational Research 47 (1977): 585-622. Putnam, L . L . and Sorenson, R . L . "Equivocal Messages in Organizations." Human Communication Research 8 2 (Winter 1982): 114-132. Rao, V . S. and Goldman-Segall, R. "Capturing Stories in Organizational Memory Systems: The Role of Multimedia." Twenty-Eight Hawaii International Conference on System Sciences, pp. 333-341. (eds.) Nunamaker, J. F. Jr. and Sprague, R . H . Jr. Vo l 4. Los Alamitos, C A : IEEE Computer Society Press, 1995. Rice, R. E . "Task Analyzability, Use of New Media, and Effectiveness: A Multi-site Exploration of Media Richness." Organization Science 3 4 (Nov 1992): 457-500. Rogers, E. M . Diffusion of Innovation. N Y : Free Press, 1983. Salomon, G. Interaction of Media, Cognition and Learning. San Francisco: Jossey-Bass, 1979. Sandoe, K . Olfman, L . , and Mandviwalla, M . "Meeting in Time: Recording the Workgroup Conversation." Proceedings of the Twelfth International Conference on Information Systems, pp. 261-272. 1991. Schallert, D. L . "The Role of Illustrations in Reading." Theoretical Issues in Reading Comprehension, pp. 503-524. Spiro, R. Bruce, B . , and Brewer, W. (eds.) Hillsdale, NJ: Lawrence Erlbaum Associates, 1980. Schank, R. C. "The Structure of Episodes in Memory." Respresentation and Understanding: Studies in Cognitive Science., Bobrow D . G . and Collins A . H . (eds.). New York: Academic Press, 1975. Schmit, B . H . , Pan, Y , and Tavassoli, N .T . " Language and Consumer Memory: The Impact of Linguistic Differencs between Chinese and English." Journal of Consumer Research 21 4 (Dec 1994): 419-431. 157 Schwabe, G. "Providing for Organizational Memory in Computer-Supported Meetings." Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences, pp. 171-180. (eds.) Nunamaker, J. F. Jr. and Sprague, R . H . Jr. Vo l 4. Los Alamitos, C A : IEEE Computer Society Press, 1994. Segars, A . H . and Grover, V . "Re-examining Perceived ease of use and usefulness: A Confirmatory Factor Analysis." MIS Quarterly 17 4 (Dec 1993): 517-525. Sein, M . K . and Bostrom, R.P. "Individual Differences and Conceptual Models in Training Novice Users." Human-computer Interaction 4 (1989): 197-229. Sheffield, J. "The Effects of Bargaining Orientation and Communication Medium on Negotiations in the Bilateral Monopoly Task." Unpublished Ph.D. Dissertation, University of Arizona, 1989, Shepard, R. N . and Copper, L . A . Mental Images and Their Transformations. Cambridge, M A : MIT Press, 1982. Simon, H . A . "How Big is a Chunk?" Science 183 (1974): 482-488. Smart Video Recorder Pro. Intel, 1995. Smith, G . D. and Steadman, L . E . "Present Value of Corporate History." Harvard Business Review 59 (1981): 164-173. Snyder, C. R. "Acceptance of Personality Interpretations as a Function of Assessment Procedures." Journal of Consulting and Clinical Psychology 42 1 (1974): 150. Snyder, C. R. and Larson, G.R. "A Further Look at Student Acceptance of General Personality Interpretations." Journal of Consulting and Clinical Psychology 38 (1972): 384-388. Sperling, G. "Short-term Memory, Long-term Memory and Scanning in the Processing of Visual Information." Early Experience and Visual Information Processing in Perceptual and Reading Disorders, pp. 198-215. Young, A . F. and Lindsley, D .B . (eds.) Washington, D . C : National Academy of Science, 1970. Spiro, R. J . , Coulson, R . L . , Feltovich, P.J. and Anderson, D . K . "Cognitive Flexibility Theory: Advanced Knowledge Acquisition in Ill-Structured Domains." The Tenth Annual Conference of the Cognitive Science Society, Hillsdale, NJ: Lawrence Erlbaum Associates, 1988. 158 Spiro, R. J. , and Jehng, J .C. "Cognitive Flexibility and Hypertext: Theory and Technology for the Nonlinear and Multidimensional Traversal of Complex Subject Matter." Cognition, Education, and Multimedia: Exploring Ideas in High Technology, pp. 163-205. Nix, D. and Spiro, R.J . (eds.) Hillsdale, NJ: Lawrence Erlbaum Associates, 1990. Spiro, R. J. , Feltovich, P.J . , Jacobson, M . J . , and Coulson, R . L . "Cognitive Flexibility, Constructivism, and Hypertext: Random Access Instruction for Advanced Knowledge Acquisition in Ill-Structured Domains." Educational Technology 315 (May 1991): 24-33. Sprague, R. H . Jr. "Electronic Document Management for Information Systems Managers." Management Information Systems Quarterly ((in press): SPSSXfor Windows. Version 6.1. Chicago, IL: SPSS Inc., 1994. Stein, E. W. and Zwass, V . "Actualizing Organizational Memory with Information Systems." Information Systems Research 6 2 (June 1995): 85-117. Steinfield, C. "Computer Mediated Communications in Organizational Settings: Emerging Conceptual Frameworks and Directions for Research." Management Communication Research 5 3 (Feb 1992): 348-365. Sticht, T. G. "Learning by Listening." Language Comprehension and the Acquisition of Knowledge, pp. 285-314. Freedle, R. O. and Carroll, J.B. (eds.) Washington, D . C : Winston & Sons, 1972. Sticht, T. G . and James, J .H. "Listening and Reading." Handbook of Reading Research, pp. 293-317. Pearson, P. D . (ed). N Y : Longman, 1984. Stone, D . and Glock, M . "How Do Young Adults Read Directions With and Without Pictures?" Journal of Educational Psychology 73 3 (1981): 419-426. Subramanian, G. H . "A Replication of Perceived Usefulness and Perceived Ease of Use Measurement." Decision Sciences 25 5,6 (Sept-Dec 1994): 863-874. Szajna, B. "Software Evaluations and Choice: Predictive Validation of the Technology Acceptance Instrument." MIS Quarterly 18 3 (Sept 1994): 319-324. Tabachnick, B. G. and Fidell, L .S . Using Multivariate Statistics. New York: Happer & Row Publishers, (2nd ed) 1989. Tan, J. K . H . and Benbasat, I. "Processing of Graphical Information: A Decomposition Taxonomy to Match Data Extraction Tasks and Graphical Representations." Information Systems Research 1 4 (December 1990): 416-439. 159 Todd, P. A . and Benbasat, I. "An Experimental Investigation of the Impact of Computer-Based Decision Aids on Decision Making Processes." Information Systems Research 2 2 (June 1991): 87-115. Todd, P. A . and Benbasat, I. "The Use of information in Decision Making: A n Experimental Investigation of the Impact of Computer-based Decision Aids." Management Information Systems Quarterly 16 3 (September 1992): 373-393. Trevino, L . K . , Lengel, R . H . , and Daft, R . L . "Media Symbolism, Media Richness, and Media Choice in Organizations." Communication Research 15 5 (1987): 553-574. Tukey, J. W. "Bias and Confidence in Not-Quite Large Samples." Annals of Mathematical Statistics 29 (June) (1958): 614. Turban, E. Decision Support and Expert Systems. New York: Macmillan, 1988. Ulrich, R. E . , Stachnik, T.J. and Stainton, N.R. "Student Acceptance of Generalized Personality Interpretations." Psychological Report 13 (1963): 831-834. Valacich, J. S., Mennecke, B . E . , Wachter, R . M . , and Wheeler, B . C . "Extensions to Media Richness Theory: A Test of the Task-Media Fit Hypothesis." Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences, pp. 11-20. (eds.) Nunamaker, J. F. Jr. and Sprague, R . H . Vol 4. Alamitos, C A : IEEE Computer Society Press, 1994. Van Dijk, T. A . and Kintsch, W. Strategies of Discourse Comprehension. New York: Academic Press, 1983. Vessey, I. "Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables Literature." Decision Sciences 22 (1991): 219-240. Walsh, J. P. and Ungson, G.R. "Organizational Memory." Academy of Management Review 16 1 (1991): 57-91. Weick, K . E . The Social Psychology of Organizing. Reading, M A : Addison-Wesley, 1979. Wildt, A . R., Lambert, Z . V . and Durand, R . M . "Applying the Jackknife Statistic in Testing and Interpreting Canonical Weights, Loadings, and Cross-loadings." Journal of Marketing Research 19 (February) (1982): 99-107. Winbatch. Version 5. Seattle, W A : Wilson WinDowAre, Inc., 1995. 160 Wold, H . and Jorekog, K . Systems Under Indirect Observation: Causality, Structure, and Prediction. Amsterdam: North-Holland, 1982. V o l . 2. Wyer, R. S. "Changes in Meaning and Halo Effects in Personality Impression Formation." Journal of Personality and Social Psychology 29 (1974): 829-835. 161 T A B L E S F O R C H A P T E R 3 162 Table 3.1 Summary of Dependent Variables Investigated in Experiment 1 Dependent Variable Measure 1) Explanative Knowledge Score on the four explanative questions in the comprehension test. 2) Descriptive Knowledge Score on the four descriptive questions in the comprehension test. 3) Overall Fact Comprehension Sum of the scores on the four explanative and four descriptive questions. 4) Inference Ability Score on the eight inference questions in the comprehension test. 163 Table 3.2 Descriptive Statistics for Dependent Variables In Experiment 1 Var i a b l e : FACT COMPREHENSION Mean 5 . 262 S t d Dev 1 465 K u r t o s i s . 003 S . E . K u r t 532 Skewness - . 421 S . E . Skew 269 Minimum 1. 0 Maximum 8 . 0 Var i a b l e : EXPLANATIVE KNOWLEDGE Mean 2 .400 S t d Dev 963 K u r t o s i s - .241 S . E . K u r t 532 Skewness - . 101 S . E . Skew 269 Minimum . 0 Maximum 4 . 0 V a r i a b l e : DESCRIPTIVE KNOWLEDGE * * Mean 2 . 863 S t d Dev 1 052 K u r t o s i s - . 094 S . E . Kur t 532 Skewness - .721 S . E . Skew 269 Minimum . 0 Maximum 4 . 0 Var i a b l e : INFERENCE A b i l i t y Mean 2 . 887 S t d Dev 1 312 K u r t o s i s - . 786 S . E . K u r t 532 Skewness - . 063 S . E . Skew 269 Minimum . 0 Maximum 5 . 0 * indicates a moderately high skewness score. 164 Table 3.3 Homogeneity of Variance Tests for Dependent Variables in Experiment 1 U n i v a r i a t e Homogeneity o f V a r i a n c e T e s t s V a r i a b l e . . EXPLANATIVE KNOWLEDGE C o c h r a n ' s C ( 3 9 , 2 ) = B a r t l e t t - B o x F ( l , 1 8 2 5 2 ) = .67273, 4 . 89453, p = p = . 027 . 027 ( a p p r o x . ) V a r i a b l e . . DESCRIPTIVE KNOWLEDGE C o c h r a n ' s C ( 3 9 , 2 ) = B a r t l e t t - B o x F ( l , 1 8 2 5 2 ) = . 52006 , .06202, p = p = .803 . 803 ( a p p r o x . ) V a r i a b l e . . FACT COMPREHENSION C o c h r a n ' s C ( 3 9 , 2 ) = B a r t l e t t - B o x F ( l , 1 8 2 5 2 ) = . 56913 , .74325, p = p = .389 . 389 ( a p p r o x . ) V a r i a b l e . . INFERENCE A b i l i t y C o c h r a n ' s C ( 3 9 , 2 ) = B a r t l e t t - B o x F ( l , 1 8 2 5 2 ) = .53517, .19101, p = p = . 662 . 662 ( a p p r o x . ) M u l t i v a r i a t e T e s t f o r Homogeneity of D i s p e r s i o n (EXPLANATIVE AND DESCRIPTIVE KNOWLEDGE) Matr i c e s Boxs M = 5.36151 F WITH (3,1095119) DF = 1.73754, C h i - S q u a r e w i t h 3 DF = 5 .21258, P P . 157 . 157 ( a p p r o x . ) ( a p p r o x . ) 165 Table 3.4 T-test on the Overall Fact Comprehension Score Condition N Mean S.D. t-value p-value Text-Based OMS 40 5.125 1.362 0.84 0.405 Multimedia OMS 40 5.400 1.566 166 Table 3.5 M A N O V A 2 0 Results for Experiment 1 (Independent Variable: System Type, all DF=(1,78) Variable SS M S _ ' F-value p-value Explanative Knowledge 7.20 0.85 8.51 0.005 Descriptive Knowledge 2.11 1.09 1.93 0.169 2 0 For all MANOVA tests, the F-value and p-value were the same and significant at 0.05 level for all the multivariate test statistics, namely Wilks' Criterion, Pillai's Trace, Hotelling-Lawley Trace and Roy's Maximum Root. 167 Table 3.6 Means (Standard Deviations) of Explanative and Descriptive Test Scores Variable Text-based O M S Multimedia O M S Explanative Test Score 2.10 (0.74) 2.70 (1.07) Descriptive Test Score 3.03 (1.03) 2.70 (1.07) 168 Table 3.7 Summary Table for the PLS Analysis Path Path Coefficient (Direct Effect) t-value Indirect Effect Total Effect* System Type — > Explanative Knowledge 0.314** 3.37 — 0.314 System Type ~ > Descriptive Knowledge -0.155 0.92 — -0.155 System Type — > Inference 0.177 1.80 0.121 0.298 Explanative Knowledge — > Inference 0.324** 2.63 — 0.324 Descriptive Knowledge — > Inference 0.127 1.45 — 0.127 ** p < 0.05 * Total Effect = Direct Effect + Indirect Effect R 2 for Explanative Knowledge = 0.25 R 2 for Descriptive Knowledge = 0.17 R 2 for Inference = 0.29 169 T A B L E S F O R C H A P T E R 4 170 Table 4.1. Summary of the Dependent Variables Dependent Variable Measure 1) Efficiency 2) Effectiveness 3) Perceived Usefulness Time taken to complete the task. Reduction in perceived ambiguity rating on a 2-item scale (Daft and Macintosh, 1981). Aggregate score on a 10-item scale (Davis, 1989). 171 Table 4.2 Descriptive Statistics for Dependent Variables In Experiment 2 Var i a b l e : Time t a k e n to c o m p l e t e l e s s - e q u i v o c a l t a s k Mean 442 . 475 S t d Dev 212 066 K u r t o s i s . 850 S . E . K u r t 532 Skewness . 327 S . E . Skew 269 Minimum 129 . 5 Maximum 1233 . 0 Var i a b l e : Time t a k e n to c o m p l e t e e q u i v o c a l t a s k Mean 745 . 375 S t d Dev 408 545 K u r t o s i s . 321 S . E . K u r t 532 Skewness . 3 24 S . E . Skew 269 Minimum 116 . 0 Maximum 181 38 . 0 Var i a b l e : F i r s t r a t i n g f o r l e s s - e q u i v o c a l t a s k Mean -1 .688 S t d Dev 957 K u r t o s i s - . 323 S . E . K u r t 532 Skewness .404 S . E . Skew 269 Minimum -3 .0 Maximum . 9 Var i a b l e : Second r a t i n g f o r l e s s - e q u i v o c a l t a s k * * Mean -2 .044 S t d Dev 920 K u r t o s i s 1. 270 S . E . K u r t . 532 Skewness 1. 150 S . E . Skew 269 Minimum - 3 . 0 Maximum 1 . 1 Var i a b l e : F i r s t r a t i n g f o r e q u i v o c a l t a s k * * Mean 1.404 S t d Dev 985 K u r t o s i s . 873 S . E . K u r t 532 Skewness - .689 S . E . Skew 269 Minimum -2 .0 Maximum 3 . 0 Var i a b l e : Second r a t i n g f o r e q u i v o c a l t a s k * * Mean 1.186 S t d Dev 1 118 K u r t o s i s .655 S . E . K u r t 532 Skewness - . 622 S . E . Skew 269 Minimum -2 .7 Maximum 3 . 0 V a r i a b l e .- A g g r e g a t e d s c o r e f o r p e r c e i v e d u s e f u l n e s s Mean .743 S t d Dev 1 619 K u r t o s i s - .615 S . E . K u r t 535 Skewness - .652 S . E . Skew 271 Minimum - 3 . 0 Maximum 3 . 0 indicates a variable which has moderately high kurtosis and/or skewness score. 172 Table 4.3 Homogeneity of Variance Tests for Dependent Variables in Experiment 2 U n i v a r i a t e Homogenei ty of V a r i a n c e T e s t s V a r i a b l e . . F i r s t r a t i n g f o r l e s s - e q u i v o c a l t a s k C o c h r a n ' S C ( 3 9 , 2 ) = 51370, P = • 865 ( a p p r o x . ) B a r t l e t t -Box F(1 ,18252) 02890, P = 865 V a r i a b l e . . Second r a t i n g f o r l e s s - e q u i v o c a l t a s k C o c h r a n ' s C ( 3 9 , 2 ) = 50041, P = 996 ( a p p r o x . ) B a r t l e t t -Box F(1 ,18252) 00003 , P = 996 V a r i a b l e . . F i r s t r a t i n g f o r e q u i v o c a l t a s k C o c h r a n 1 s C ( 3 9 , 2 ) = 61812, P = • 137 ( a p p r o x . ) B a r t l e t t -Box F(1 ,18252) 2 . 21150, P = 137 V a r i a b l e . . Second r a t i n g f o r e q u i v o c a l t a s k C o c h r a n 1 s C ( 3 9 , 2 ) = 63788, P = 081 ( a p p r o x . ) .Bar t l e t t -Box F(1 ,18252) 3 . 04588, P = 081 M u l t i v a r i a t e T e s t f o r Homogenei ty o f D i s p e r s i o n Matr i c e s Boxs M = 9.55669 F WITH (10, 29086) DF = .90266, P = . 530 (approx . ) C h i - S q u a r e w i t h 10 DF = 9 .02984, P . 529 (approx . ) 173 Table 4.4a T-test on the Time Taken (in seconds) to Perform Less-Equivocal Task Condition N Mean S.D. t-value p-value Text-Based OMS 40 388 235 2.36 0.021 Multimedia OMS 40 497 173 Table 4.4b T-test on the Time Taken (in seconds) to Perform Equivocal Task Condition N Mean S.D. t-value p-value Text-Based OMS 40 731 444 0.32 0.749 Multimedia OMS 40 760 375 174 Table 4.5a Repeated Measures A N O V A — Between Subjects Effects for Perceived Ambiguity Reduction Source D F SS M S F p-value System Usage Order 1 5.30 5.30 8.82 0.004 Error 78 46.88 0.60 Table 4.5b Repeated Measures A N O V A - Within Subject Effects for Perceived Ambiguity Reduction Source D F SS M S F p-value Task Type 1 0.37 0.37 0.96 0.329 System Usage Order*Task Type 1 4.14 4.14 10.84 0.001 Error 78 29.78 0.38 175 Table 4.6 The Means (Standard Deviations) for Perceived Ambiguity Reduction Experimental Condition Equivocal Task Less-Equivocal Task Text-Second -0.13 (0.62) 0.29 (0.68) Multimedia-Second 0.56 (0.82) 0.34 (0.67) 176 Table 4.7a The Means (Standard Deviations) for Perceived Ambiguity Level for the Less-Equivocal Task Experimental Condition 1st Rating 2nd Rating Text-Second -1.78 (0.94) -2.07 (0.97) Multimedia-Second -1.69 (0.99) -2.02 (0.93) Table 4.7b The Means (Standard Deviations) for Perceived Ambiguity Level for the Equivocal Task Experimental Condition 1st Rating 2nd Rating Text-Second 1.31 (0.86) 1.43 (0.93) Multimedia-Second 1.50 (1.10) 0.94 (1.24) 177 Table 4.8 T-test on the Aggregated Score on Perceived Usefulness Condition N Mean S.D. t-value p-value Text-Based OMS 39 -0.12 1.53 -5.42 0.000 Multimedia OMS 40 1.58 1.23 178 T A B L E S F O R C H A P T E R 5 179 Table 5.1 Summary of the Dependent Variables Dependent Variable Measure 1) Appraisal Outcome 2) Confidence with the Decision Made Aggregate score on a 12-item scale (Denison, 1991). Aggregate score on a 3-item scale (Aldag and Power, 1987) 180 Table 5.2 Descriptive Statistics for Dependent Variables In Experiment 3 Var i a b l e : F i r s t A p p r a i s a l S c o r e ( E x p e r i m e n t a l Groups) Mean - . 262 S t d Dev .794 K u r t o s i s 1.730 S . E . K u r t .733 Skewness - . 392 S . E . Skew . 374 Minimum -2 .4000 Maximum 1. 7000 Var i a b l e : Second A p p r a i s a l S c o r e ( E x p e r i m e n t a l Groups) Mean . 480 S t d Dev . 825 Kur t o s i s . 474 S . E . K u r t .733 Skewness - . 493 S . E . Skew . 374 Minimum -2 .0000 Maximum 1. 9000 Var i a b l e : 1 s t R a t i n g on C o n f i d e n c e w i t h D e c i s i o n Made ( E x p e r i m e n t a l Groups) Mean - . 343 S t d Dev 1 . 464 K u r t o s i s - . 882 S . E . K u r t .788 Skewness . 133 S . E . Skew .403 Minimum - 3 . 0 Maximum 2 . 3 Var i a b l e : 2nd R a t i n g on C o n f i d e n c e w i t h D e c i s i o n Made ( E x p e r i m e n t a l Groups) Mean .618 S t d Dev 1 . 290 Kur t o s i s - . 717 S . E . K u r t .788 Skewness . 012 S . E . Skew .403 Minimum - 2 . 0 Maximum 3 . 0 Var i a b l e : A p p r a i s a l S c o r e ( C o n t r o l Groups) Mean . 890 S t d Dev . 87 1 K u r t o s i s - . 695 S . E . K u r t .733 Skewness - . 368 S . E . Skew . 374 Minimum - 1.1000 Maximum 2 . 4000 Var i a b l e : C o n f i d e n c e w i t h D e c i s i o n Made ( C o n t r o l Groups) Mean 1. 314 S t d Dev 1 . 250 K u r t o s i s . 681 S . E . K u r t .788 Skewness - .794 S . E . Skew .403 Minimum -2 .3 Maximum 3 . 0 181 Table 5.3 Homogeneity of Variance Tests for Dependent Variables in Experiment 3 U n i v a r i a t e Homogenei ty of V a r i a n c e T e s t s V a r i a b l e . . F i r s t A p p r a i s a l S c o r e C o c h r a n ' s C ( 1 9 , 2 ) = B a r t l e t t - B o x F ( l , 4 3 3 2 ) = .59061, .61814, P = P = • 432 432 (approx ) V a r i a b l e . . Second A p p r a i s a l S c o r e C o c h r a n ' s C ( 1 9 , 2 ) = B a r t l e t t - B o x F ( l , 4 3 3 2 ) = .52433, . 04387 , P = • P = 834 834 (approx ) M u l t i v a r i a t e T e s t f o r Homogenei ty o f D i s p e r s i o n (For F i r s t and Second A p p r a i s a l S c o r e s ) Matr i c e s Boxs M = .64372 F WITH (3,259920) DF = .20234, p C h i - S q u a r e w i t h 3 DF = .60702, p . 895 . 895 ( a p p r o x . ) ( a p p r o x . ) U n i v a r i a t e Homogenei ty of V a r i a n c e T e s t s V a r i a b l e . . F i r s t R a t i n g on C o n f i d e n c e w i t h D e c i s i o n Made C o c h r a n ' s C ( 1 6 , 2 ) = B a r t l e t t - B o x F ( l , 3 0 7 2 ) = .54470, .12446, P = P = 724 724 ( a p p r o x . ) V a r i a b l e . . Second R a t i n g on C o n f i d e n c e w i t h D e c i s i o n Made C o c h r a n ' s C ( 1 6 , 2 ) = B a r t l e t t - B o x F ( l , 3 0 7 2 ) = . 53576, . 07954, P = • P = 778 778 ( a p p r o x . ) M u l t i v a r i a t e T e s t f o r Homogeneity of D i s p e r s i o n (For F i r s t and Second R a t i n g on C o n f i d e n c e w i t h Matr i c e s D e c i s i o n Made) Boxs M = 5.98556 F WITH (3,184320) DF = 1 .86017, p C h i - S q u a r e w i t h 3 DF = 5 .58029 , p . 134 . 134 ( a p p r o x . ) ( a p p r o x . ) 182 Table 5.4a Repeated Measures A N O V A — Between Subjects Effects for Aggregated Appraisal Score Source D F SS M S F p-value System Type 1 0.28 0.28 0.27 0.605 Error 38 38.65 1.02 Table 5.4b Repeated Measures A N O V A - Within Subject Effects for Aggregated Appraisal Score Source D F SS M S F p-value Time Sequence 1 11.03 11.03 38.32 0.000 System Type*Time Sequence 1 1.28 1.28 4.43 0.042 Error 38 10.93 0.29 183 Table 5.5 The Mean (Standard Deviations) for the Appraisal Score 1st Appraisal Score 2nd Appraisal Score Text-Based OMS -0.20 (0.87) 0.30 (0.83) Multimedia-Based OMS -0.33 (0.73) 0.67 (0.79) 184 Table 5.6 One-Way A N O V A for the Final Appraisal Score Source D F SS M S F p-value Between Groups 3 5.94 1.99 3.41 0.022 Within Groups 76 44.11 0.58 Total 79 50.04 185 Table 5.7 The Means (Standard Deviations) for the Final Appraisal Score by the Four Experiment Conditions Experimental Condition Final Appraisal Score Text-Based OMS Experimental Group (With Bias Cue) 0.30 (0.83) Multimedia OMS Experimental Group (with Bias Cue) 0.67 (0.79) Text-based OMS Control Group (Without Bias Cue) 0.84 (0.59) Multimedia OMS Control Group (Without Bias Cue) 1.03 (0.81) 186 Table 5.8 The Percentage of Subjects Who Recommended Renewal of Contract by the Four Experiment Conditions Experimental Condition Percentage of Subjects Who Recommended Renewing the Contract for Life or for Another Five Years 1st Appraisal 2nd Appraisal Text-Based OMS Experimental Group (With Bias Cue) 20% 15% Multimedia OMS Experimental Group (with Bias Cue) 10% 40% Text-based OMS Control Group (Without Bias Cue) N.A. 45% Multimedia OMS Control Group (Without Bias Cue) N.A. 50% 187 Table 5.9 T-test on the Confidence with the Decision Made Condition N Mean S.D. t-value p-value Text-Based OMS 20 0.84 1.24 > 1.02 0.315 Multimedia OMS 20 0.39 1.34 188 Table 5.10a Repeated Measures A N O V A - Between Subjects Effects for Aggregated Score on Confidence with Decision Made Source D F SS M S F j)-value System Type 1 0.79 0.79 0.26 0.612 Error 38 96.59 3.02 Table 5.10b Repeated Measures A N O V A -- Within Subject Effects for Aggregated Score on Confidence with Decision Made Source DF SS M S F p-value Time Sequence 1 15.69 15.69 18.35 0.000 System Type*Time Sequence 1 0.94 0.94 1.10 0.302 Error 38 27.37 0.86 189 FIGURES FOR CHAPTER 3 190 Figure 3.1 The Research Model Based on the Functional Perspective 191 Figure 3.2 An Example of the OMS User Interface Netscape - |BC Cancer Research Centre] File Edit View Go Bookmarks Options Directory HE Help British Columbia Cancer Research Centre Organizational Memory System TRANSLATING DISCOVERIES INTO CURES... The British Columbia Cancer Research Ceatre (BCCRC) is a non-profit organization. It is located at 601 West 10th Avenue, Vancouver, British Columbia At the first glance, the BCCRC is a collection of scientists, cancer specialists and support staff working under one roof Yet, there is so M U C H more! At its heart, the BCCRC is an alliance of some of the WORLD LEADERS in cancer research - each internationally recognized as a SPECIALIST in his or her Bald - working together as a team with clinicians, post-graduate students and dedicated support staff to solve the mystery of what causes cancer, to improve existing treatment methods and to develop new and more effective means of controlling and curing this devastating illness. WHO ARE WE... The BCCB.C, owned by the B.C. Cancer Foundation, is the LABORATORY RESEARCH A R M of the B.C_Cmier Ajsency The Agency administers all the cancer treatment clinics and satellite programs in the province. With the Centre's direct links to the cancer climes, the people of British Columbia are assured that they will have immediate access to the most 192 Figure 3.3 The User Interface for the Multimedia OMS in Experiment 1 Figure 3.4 The User Interface for the Text-Based OMS in Experiment 1 Netscape - [The Mission of the Research Centre] File Edit View Go Bookmarks Options Directory Help The Mission of the Research Centre *** Interview Transcript (with Dr. Paul Rannie, Director of B C C R C ) *** The mission of the Research Center is not too different from the mission for the Cancer Agency and that is mainly to End better and effective ways for cancer control. And cancer control means prevention, treatment, and just information dissemination about the disease. I suppose the thing that sets the Research Center apart from say a university laboratory operation is that we do have this direction towards cancer control In other words, we're not just working on curiosity-oriented research, although the curiosity component does encourage us to look in different directions. But our targets are, are more closely defined. We're looking for things that will have a direct translational outcome in the Clmic. I mean I guess for the individual researcher, the immediate objective is to get enough data to write a paper and then good enough papers to get a grant and then get a grant to approach the major problems. So the only difference in flavour between, I guess the major difference between ourselves and say a university research operation is that we are directing our research towards improving either treatment or prevention of cancer. So it's more mission-oriented in that sense. Back to the previous page. 194 Figure 3.5 The Results of the PLS Analysis F I G U R E S F O R C H A P T E R 4 196 Figure 4.1 A Framework for Task Equivocality and Information Ambiguity Q: Should A be promoted? Q: Should A be promoted? Equivocality of Task High I: On a scale of 1 to 5, A ' s immediate supervisor rated him as 5 for hard working. I: A ' s immediate supervisor mentioned that he is a very hard working individual. Q: How many full-time employees work in Account Payable Department? Q: How many full-time employees work in Account Payable Department? Low I: Twenty full-time employees work in Account Payable Department. 11: Twenty-five people work in Account Payable Department. 12: Twenty percent of the employees work part time. Low High Ambiguity of Information 197 Figure 4.2 The User Interface for the Multimedia OMS in Experiment 2 198 Figure 4.3 The User Interface for the Text-Based OMS in Experiment 2 File jEdit View Go Bookmarks Options Directory Interview with Drs. A and C Help * *** Interview Transcript *** The following interview was conducted on March 10, 1994 by K with Drs. A and C, the Head an Fox Lab, one of the six departments within the B.C. Cancer Research Center. d Deputy Head of the Terry IntCrVlGWCr; The Es have formed a long standing partnership with extent from the raising of their four children at home to the search for better treatments and eventually a cure for the form of cancer we called Leukerm a. A; I met C when we were at an institution working, not knowing each other but I had two lab co ats that I really cherished. They were 42-tall, and I had bribed the cleaning staff to make sure that I had one being cleaned ai d the other one I was wearing. And I came out in the hall one day to find this woman on the floor, eh, having wrapped d there she was pounding it with a hammer to make it into small pieces so she could use it for an exp Interviewer". So, what was the first word you said to your wife to be? I don't remember, but I don't think they were nice Interviewer: C. do you remember? C I No, but I remember they were not nice. But, they were also unacceptable, because I always lab coat and personal belongings. And, I was going to stand up for what I believe in, no matter wl J-I-Si\ ry ice in my lab coat, and eriment. thought science came before at • 199 Figure 4.4 The System Usage Order * Task Type Interaction Effect on Perceived Ambiguity Reduction Perceived Ambiguity 0.7 n 0 Q, 0.56 \ o- _ Less-Equivocal 0.34 - o 0.29 -\ \ Equivocal o -0 .13 Multimedia-Second Text-Second 200 F I G U R E S F O R C H A P T E R 5 201 Figure 5.1 The User Interface for the Multimedia OMS in Experiment 3 202 Figure 5.2 The User Interface for the Text-Based OMS in Experiment 3 Netscape -File Edit View Go Bookmarks Options Directory Help Interview Transcript *** — It would depend a lot on the basis for the disagreement If the basis is on what I consider to be a philosophical issue, you know, his style versus my style or her style versus my style, then I think there is a reality that for better or for worse, I am the lent chairman and with the chairmanship comes the responsibility for executing, eh, the things that need to be done. Eh, fs like that quote of my father's. He used to say I'm not always right but I'm always boss. And that is a reality Eh, the individual that does have the leadership position has an obligation to make sure that things get done. Unfortunately, or fortunately, depending on one's point of view, things get done in the style that is adapted by the particular person that fills the chairman's position or chairman's desk at that point in time. If the individual disagrees sufficiently strongly with the management style or with the way things are being done, that individual obviously has the option of finding another organization where things are done more in the style to which he or she wishes them to be done Now within all of mat I think there is an implicit need for communication Click here to return to the previous page. skid 203 Figure 5.3 A Schematic Representation of the Experimental Design for Experiment 3 Text-Experimental Condition Bias Cue + Dummy Personality Profile (Treatment) First Appraisal (Observation) Text Transcripts or Video Clips of interview (Treatment) X Second Appraisal (Observation) O O _ i < z UJ rr UJ x UJ Video-Experimental Condition Text-Control Condition Video -Control Condition X X T I M E m O: Observation X: Treatment 204 Figure 5.4 The System Type * Time Interaction Effect on Appraisal Outcomes Appraisal Score 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0--0.1 -0.2 H -0.3 -0.4 -0.5 -0 .20O" -0.33 o 1 st Appraisal o 0.67 •'' M u l t i m e d i a O M S 0.30 T e x t - B a s e d C M S 2nd Appraisal 205 APPENDIX A. A LIST OF SHORTCOMINGS OF HUMAN DECISION MAKING CONTENTS: A List of Shortcomings of Human Decision Making 206 Appendix A . A List of Shortcomings of Human Decision Making (Source: Kahneman and Tversky, 1984) 1) Insensitivity to prior probability of outcomes 2) Insensitivity to sample size 3) Misconceptions of chance 4) Insensitivity to predictability 5) The illusion of validity 6) Misconceptions of regression 7) Biases due to the retrievability of instances 8) Biases of imaginability 9) Illusory correlation 10) Insufficient adjustment 11) Biases in the evaluation of conjunctive and disjunctive events 12) Anchoring in the assessment of subjective probability distributions 207 APPENDIX B. INSTRUCTIONS GIVEN TO RESEARCH ASSISTANTS CONTENTS: B l — Experimental Condition Assignment B2 -- Telephone Contact B3 — Experimental Session B4 — Post-Experimental Session 208 APPENDIX B. INSTRUCTIONS TO RESEARCH ASSISTANTS This instruction is divided into four parts, as depicted in Figure 1. Experimental Condition : .Assignment 1 Phone Contact Actual Experimental Session ;RostE:xpferimental Session Figure 1. The Four Parts of this Instruction Guide 209 APPENDIX B l Experimental Condition Assignment As shown in Figure 2, there are two experimental conditions in Task 1, two in Task 2, and four in Task 3. As a result, there are 16 experimental conditions all together (2x2x4). Subjects will be randomly assigned to one of the 16 conditions. A stack of 16 cards will be used for this assignment. Each of the 16 cards has one of the 16 experiment conditions written on it. Follow the steps below: 1) Shuffle the card thoroughly; 2) Copy the resulting order of the shuffle cards onto the Subject Assignment Sheet (this will result in 16 assignments); and 3) Repeat steps 1) and 2) until you have 80 assignments. 210 A P P E N D I X B2 Telephone Contact The key during the first contact is to establish a good impression — be polite and friendly. Follow the steps below during telephone contact. 1) Thank the subject for signing up for the study and apologize for the delay in getting back to him/her; 2) Tell the subject that instead of an hour, as was originally estimated, the study will take up two and a half hours; 3) Also, we can only pay $20 for his/her participation because of our budget constraint; 4) However, other benefits for participating in the study include learning about the Web and having exposure to leading edge technology; 5) Continue to step 6) if they are still interested in participating in the study. If not, thank him/her again and hang up; 6) Write the subject name and phone number on the next available slot in the Subject Assignment Sheet (this will immediately assign a subject to one of the 16 experimental conditions); 7) Locate the next available time slot and ask if he/she can come at that time; 8) If the subject can't come in, try another time slot until a time is scheduled. Remember, his/her experimental condition is fixed, even if he/she has to reschedule later; 9) Tell the subject to come to Room 17 of Henry Angus Building couple minutes before the scheduled time; and 10) Thank him/her again before hanging up. *** If a subject fails to show up for his/her scheduled experiment, call him/her to find out why. Reschedule the subject if he/she can provide an acceptable reason. If the subject fails to show up for the second time, look for a replacement from the Subject Sign Up Sheet. Note, the replacement subject will have the experimental condition of the subjects that he/she replaced. 211 A P P E N D I X B3 Experimental Session Before Subject's Arrival 1) Start up the computer;. 2) Start up Windows; 3) Go to "Options" menu and "Expire" all hypertext links in Netscape (this will ensure that all links are marked as unvisited before the subject begins the experimental tasks); 4) Make sure that the experimental instructions are arranged in the correct order; and 5) Arrange the questionnaires in the correct order. After Subject's Arrival (Approximately 10 Minutes) 1) Greet the subject; 2) Make sure that subject is sitting comfortably in fort of the computer terminer; 3) Hand the subject an empty envelope and ask him/her to put all his/her filled questionnaires into the envelope. Ensure him/her that all data collected will be confidential. The ID number written on the questionnaires are for matching purposes only. If the subject leaves any information blank, we will have to go through the trouble of identifying him/her and call him/her later for more information; 4) Ask the subject to read and sign the Consent and Information Form; 5) Collect and file the form into the Consent From Folder; 6) Ask the subject whether he/she is familiar with using a mouse device. If yes, skip the Mouse Tutorial. If not, give him/her the MOUSE Tutorial sheet; 7) Tell the subject that all actions in Netscap involve single click. Double click will invoke two copies of the same program, and the system will crash if too many copies are invoked; 8) Start up the Web Tutorial (you can do this while explaining Step (7)); 9) Let the subject work through the tutorial by himself/herself; 10) Most the subjects will try to click on the "play" button on the sample video player control panel. You may have to explain to them that this is only a sample (template), and that he/she can't play video on this template, but he/she will be able to try one out in a moment; 11) After the subject had played the sample clip, tell him/her how to use the "pause" button; 12) Explain and demonstrate to the subject on using the sliding bar on the control panel. Let him/her know that this is very useful in the actual tasks for two reasons: a) if he/she misses a particular part of an interview, he/she can drag the bar backward a little bit to replay the part and b) he/she can forward the clip to any location, if he/she remembers the approximate location of the particular information he/she wishes to review; 13) Remind the subject that it is very important to exit out of the video clip by choosing "EXIT" from the "FILE menu ~ else, the system will clash; 14) Hand the subject the General Instruction, together with the Instruction for Think-212 Aloud; and 15) Make sure that the subject "think-aloud" on the exercise. T A S K 1 (Orientation Task - 40 minutes) 1) While the subject is performing Step 14 above, start up "OMS"; 2) Type in the appropriate subject ID and click on "ENTER"; 3) Select TIT (Task 1 Text) if the subject was assigned to T E X T condition. Select T1V (Task 1 Video) if the subject was assigned to VIDEO condition; 4) Hand the subject Task Description la and ask him/her to click on "OK" when he/she is ready; and 5) Move back to your working area and note down the time. *** Through out the experiment, let subject perform the task without interrupting him/her, unless computer error occurs. Stay at your working area. 6) When the 20-minute time limit is up, ask the subject to stop and hand him/her Questionnaire 1. Remind him/her to take about 5 minutes to finish it (most people take about 8 minutes to complete the questionnaire). If he/she finishes examining the information before 20 minutes is up, and he/she wishes to move on, you can give him/her the questionnaire. Remind him/her to put the questionnaire into the envelope provided when he/she is finished; 7) When the subject has finished Questionnaire 1, note down the time and hand him/her Task Description lb; 8) Exit the subject from the page that he/she is in. Use the mouse to select the appropriate condition (TIV or TIT, the one that was not selected previously) on the main menu. Ask him/her to click on "OK" when he/she is ready. Note down the time; and 9) Ask the subject to stop when the 15-minute limit is up. T A S K 2 (Evaluation Task - 65 minutes) 1) Hand the subject both Task Description Ha and Questionnaire 2. 2) Exit the subject from the page that he/she is in. Select the appropriate condition (T2T, if he/she was assigned to Text-Video condition. T2V, if he/she was assigned to the Video-Text condition). Ask the subject to click on "OK" when he/she is ready; 3) Turn on the microphone and check the tape recorder. Make sure that it is working. You may have to switch tape in the middle of the recording; 4) Note down the time; 5) Ask the subject to let you know when they are ready to answer the questions. You need to tape-record him/her think-aloud process; 6) Start the recorder when the subject signals to you that he/she is ready. Use the headphone to monitor the volume level. Adjust the recording level accordingly; *** Through the recording, if the subject is silent for more than 10 seconds, remind him/her to speak out aloud. 213 7) After the subject has finished Questionnaire 2. Stop the recorder, collect Questionnaire 2 from him/her and explain to him/her that he/she will revisit this questionnaire again later; 8) Hand him/her Questionnaire 2a; 9) Exit the subject from the page where he/she is in; 10) Select the appropriate next condition (T2T or T2V, whichever that was not selected previously); 11) After the subject has finished Questionnaire 2a, note down the time; 12) Allow the subject to take a 5-minute break. Encourage the subject to go out and stretch or walk around; 13) When the subject returns from the break, note down the time again; 14) Hand the subject Task Description lib and his/her original copy of Questionnaire 2; 15) Tell the subject to click on "OK" when he/she is ready. Remind him/her to let you know when he/she is ready to answer the questions again; 16) Start the recorder when he/she signals to you that he/she is ready. Use the headphone to monitor the volume level. Adjust the recording level accordingly.; *** Through the recording, if the subject is silent for more than 10 seconds, remind him/her to speak out aloud. 17) After the subject has finished Questionnaire 2. Stop the recorder; 18) Hand the subject Questionnaire 2b; 19) Exit the subject from the page where he/she is in; 20) Select T3I for Task 3; and 21) When the subject finishes Questionnaire 2a, note down the time. T A S K 3 (Appraisal Task -35 minutes) 1) Hand the subject Task Description Ilia and ask him/her to click on "OK" when he/she is ready; 2) Hand the subject Questionnaire Ilia after he/she has finished examining the performance indicators and personality test profile pages; 3) Exit the subject from the page where he/she is in. On the main Menu, select the next appropriate condition (T3T, if he/she was assigned to the text condition for Task 3 and T3V, if he/she was assigned to the video condition for Task 3); 4) After the subject has finished with Questionnaire Ilia, give him/her Task Description Illb. Ask him/her to click on "OK" when he/she is ready; 5) Note down the time; 6) Give the subject Questionnaire Illb after he/she has finished examining the relevant information; 7) Note down the time when he/she finishes Questionnaire Illb, and 8) Exit the subject from the page. 214 A P P E N D I X B4 Post-Experimental Session 1) Pay the subject and ask him/her to sign the Cash Receipt Form; 2) Let the subject know that the names of the top three performers will be posted outside Room 17 by mid-October. Winners will also be notified by phone; 3) Tell the subject that two pieces of information in the OMS are not real a) Ralph Durand Personality Profile, b) Performance Indicators. Let him/her know that Ralph Durand is a very famous researcher and good administrator. We distorted these two pieces of information to see how people change their decisions when they are given different kind of information; 4) Remind the subject not to discuss the study with him/her friends who will be participating in the study; and 5) Thank the subject. 215 APPENDIX C. PRE-EXPERIMENTAL MATERIALS CONTENTS: CI — Consent and Information Form C2 — General Instruction C3 - Instructions for Think-Aloud 216 THE UNIVERSITY OF BRITISH COLUMBIA Faculty of Commerce Vancouver, B.C. Canada V6T 1Z2 Tel: (604) 822-8396 Fax: (604) 822-8489 Dr. Izak Benbasat, & Kai H. Lim WORLD-WIDE-WEB STUDY CONSENT AND INFORMATION FORM Objective of the study: To understand how people use different types of information to make decisions. What I have to do? You wil l be asked to play the role of a newly appointed board member of an organization, and you wi l l examine this organization via information stored in a computer system. At the end, you wil l be asked to make some decisions based on the information you have examined. For how long? The whole session wi l l last approximately two and a half hours. Confidentiality and anonymity: Your participation in this study is strictly voluntary. Y o u may withdraw from the study at any time at your own discretion, without penalty. The data resulting from your participation wi l l be kept confidential. Any results reported from this study wi l l be anonymous. What do I receive for my participation? You wi l l be paid $20 upon completing the three tasks assigned to you. There wi l l be additional cash prizes of $30 awarded to each of the top three performers. Performance is measured by your total score on the three tasks that you are going to perform. Please complete the part below after you have received answers to all the questions and concerns that you have regarding this study. Please note that, should you decide to participate, you wi l l be offered a signed copy of this form for your records. Agreement to Participate: I, the undersigned, hereby agree to participate in the above mentioned study. I understand that some of my classmates and friends may also be participating in this study. I realize that my discussion of the details of this study with them may distort the results. Therefore, I agree not to discuss with any other participant any aspect of the study prior to their participation. This study is conducted by Dr. Izak Benbasat, Professor of MIS, and Kai H . Lim, Ph.D. Candidate, Faculty of Commerce and Business Administration. Name: Signature of Participant Date: Phone No. : 217 APPENDIX C2 GENERAL INSTRUCTION WORLD-WIDE-WEB STUDY H i there, and welcome. Thank you for participating in this study. The British Columbia Cancer Research Centre (BCCRC) has approached us, the researchers of this study, to design and implement a prototype computer system that captures past information of B C C R C . Your task in this experiment is to help us evaluate this system and see how this system can be used to help make better decisions. To help us achieve this objective, in some tasks, we require you to "think aloud" while you are working. In other words, you need to express verbally whatever thoughts go through your mind. We are not testing your intelligence. The purpose is simply to help us better understand your thought processes and the difficulties you may have with the system. You will be asked to perform three tasks. In all three tasks you will be playing the role of a newly appointed board member of B . C . Cancer Agency (the parent organization of BCCRC) . You will be using a computer system in performing these tasks. Every piece of information stored in the system is potentially important in helping you to perform the tasks. Therefore, we would like you to examine all the information carefully. Please do not skip any part of the information (visit all hypertext links)! 218 APPENDIX C3 INSTRUCTIONS FOR THINK-ALOUD In this study, we are interested in your running commentary on what you are attempting to do, what is going through your mind while you interact with the system and perform the evaluation task. Therefore, we will ask you to THINK ALOUD CONSTANTLY. What we mean by think-aloud is that we want you to reason in a loud voice, SAY OUT LOUD everything that passes through your mind for each step as you interact with the system and perform the task. When you read some thing alone, do you realize sometimes you read it out aloud? Sometimes, when you think about something, you actually mumble to yourself. We would like you to do that now while performing the task. The only difference is to keep your voice loud enough for the tape recorder. It does not matter if your sentences are not complete, since you are not explaining to anyone else. Just act as if you are alone in the room speaking to yourself loudly. It is most important that you keep talking. If you are silent for more than 10 seconds I will remind you to keep talking aloud. EXERCISE I Before turning to the real task, we will start with a couple of practice problems. Please talk aloud while you work on these problems. First, please add two numbers in your head, and say out aloud each step. Now T A L K ALOUD while you calculate 476 + 688 = ? 219 APPENDIX D. EXPERIMENTAL MATERIALS FOR EXPERIMENT 1 CONTENTS: D l — Task Description la D2 — Comprehension Test D3 -- Task Description lb 220 A P P E N D I X D l Task Description la Being a new board member of B C C R C , you are naturally very keen in learning more about B C C R C . Your first task is to familiarize yourself with B C C R C . The BCCRC's Organizational Memory System (OMS) will help you to achieve this. The system contains R E A L information about B C C R C . It is constructed based on the principle of Hypertext; in other words, you can get more detailed information about a particular topic/theme by clicking on a highlighted text corresponding to that topic/theme. After you have finished examining the information stored in the OMS, we will ask you to answer a set of questions to evaluate your level of understanding of B C C R C . Some questions ask you about facts that you will examined; others will require you to make inferences based on the facts. Your score on this questionnaire will be part of your overall performance score, which will be used to determine the three best performers for additional cash awards. Again, each of the three best performers will be awarded an additional $30. In this task, you do not have to think aloud. You will have 20 minutes to review information related to Task I. The lab assistant will inform you when the time is up. After that you will have 5 minutes to answer a set of questions. If you have any questions, please ask the lab assistant now. When you are ready to begin, inform the lab assistant. 221 A P P E N D I X D2 ID#: Comprehension Test Questions 1-8 are based on the facts in the OMS related to B C C R C that you have just examined. Please try your best to answer these questions. However, if you are uncertain about the answer to a particular question, leave it blank — no penalty for leaving it blank, but a quarter of a point will be deducted from each of the incorrect answers. 1) Who is the immediate superior of the Chief Operating Officer? a) Director of Operations b) Chief Executive Officer (CEO) c) Chairperson of the Board of Trustees d) Head of the Executive Committee 2) Who is the immediate superior of the Head of Medical Biophysics Department? a) Director of Radiation Oncology b) Director of Medical Oncology c) Director of Research d) Chief Operating Officer 3) Which of the following is the closest to the number of people working in B . C . Cancer Research Centre (BCCRC)? a) 230 b) 250 c) 2,500 d) 2,300 4) Which of the following is true about BCCRC? a) Its mission is to find better and effective ways for cancer control. b) It is a nonprofit organization which raises money by itself for cancer research. c) It is part of the B .C . Cancer Agency. d) More than one of the above (please indicate which ones ). 222 5) The Head of Cancer Imaging Department would like to appeal for a higher salary increase. To whom should he approach? a) The Director of BCCRC b) The Director of Human Resources c) The Chief Executive Officer (CEO) d) The Executive Committee e) The Board of Trustees 6) Using LIFE device, a) the red laser light makes cancerous cells appear blue and healthy cells appear green. b) the green laser light makes cancerous cells appear red and healthy cells appear blue. c) the orange laser light makes cancerous cells appear blue and healthy cells appear red. d) the blue laser light makes cancerous cells appear red and healthy cells appear green. 7 ) Which department created the world's first solid-state microscope for screening cancerous cells? a) Cancer Endocrinology b) Cancer Imaging c) Epidemiology, Biometry and Occupation Oncology d) Medical Biophysics e) Medical Oncology-Laboratory Operations (MOLO) f) Terry Fox Laboratory 8) Which department is involved in chemotherapy research? a) Cancer Endocrinology b) Cancer Imaging c) Epidemiology, Biometry and Occupation Oncology d) Medical Biophysics e) Medical Oncology-Laboratory Operations (MOLO) f) Terry Fox Laboratory 223 The following questions require you to make inferences based on what you have just learned about B C C R C . In other words, you may not be able to answer these questions based on the information you have just seen. You need to think and deduce you answers from your understanding of the organization. Try your best to answer these questions. However, if you are uncertain about the answer to a particular question, leave it blank — no penalty for leaving it blank, but a quarter of a point will be deducted from each of the incorrect answers 9) To which one of the following research publications do you think the Medical Oncology Department of BC Cancer Research Centre is more likely to report their research findings? a) Journal of Biochemistry Research b) Journal of Applied Biochemistry c) Journal of Radiation Science d) Journal of Radiation Therapy For Questions 10-15, please select one item from the following list that best matches the answer. Each item can be used more than once. a) B . C . Cancer Agency (BCCA) b) B . C . Cancer Research Centre (BCCRC) c) B . C . Cancer Foundation d) Cancer Endocrinology e) Cancer Imaging f) Epidemiology and Occupation Oncology g) Medical Biophysics h) Medical Oncology-Laboratory Operations i) Terry Fox Laboratory j) Radiation Oncology 10) One of the six departments within the Research Centre recently opened the first proton treatment research centre in Canada. Which department is most likely to be the one? 11) Which department is most likely to be the one pioneering the research in Autologous Bone Marrow Transplant, an important tool in treating leukemia? 12) Dr. John Dole has a Ph.D. in Electronics and Electrical Engineering. Which department is he most likely to be working for? 13) If a person would like to make a donation towards cancer research, which organization should she contact? 14) A pharmaceutical company would like to send their new drug that treats leukemia to an independent institution for further testing. Who should they contact? 15) A patient who has lung cancer would like to volunteer himself as a subject for new treatment. Who should he contact? 224 A P P E N D I X D3 Task Description lb You have just familiarized yourself with BCCRC using the OMS. Your level of understanding of B C C R C will greatly affect your performances in subsequent tasks. Therefore, we would like you to revisit this important information before moving on. To make this review more interesting, the same information will be presented to you in a new format using another medium. You will have 15 minutes for this review. Again, you do not have to think aloud. If you have any questions, please ask the lab assistant now. When you are ready to begin, inform the lab assistant. 225 APPENDIX E . EXPERIMENTAL MATERIALS FOR EXPERIMENT 2 CONTENTS: E l — Task Description Ha E2 — Task II Evaluation Questionnaire E3 -- Task Description l ib E4 — Perceived Usefulness Questionnaire 226 APPENDIX E l Task Description Ha In this task, we would like you to further explore the B C C R C . Specifically, we would like you to develop a better understanding of two departments within the Research Centre: the Terry Fox Lab and the Medical Biophysics Department. As before, the BCCRC's Organizational Memory System (OMS) will help you in accomplishing this task. When you reexamine these two departments, the system will provide you with more information related to these departments. As before, you will be asked to answer a set of questions. However, you will be given the questionnaire before you begin the task. Follow the steps below: 1) Skim through the questions first. This will help you to focus on the specific pieces of information that are required to answer the questions when you are examining the information. 2) Examine the information once without interruption. 3) Answer the questions. You can reexamine/review the information as many times as you want while you are answering the questions. We would like you to think aloud while you answer these questions. Please be as specific as you can in verbalizing your thoughts. Please use the BLACK pen provided to complete the questionnaire. In answering each question, you need to formulate your own opinion about a given statement based on the information you are examining. Again, your performance in this task will be part of your overall performance score, which will be used to determine the additional cash awards. You will be given 35 minutes to perform this part of the task. If you have any questions, please ask the lab assistant now. When you are ready to begin, inform the lab assistant. 227 WORLD-WIDE-WEB STUDY Instructions for Answering "Scaling-Type" Questions. Please circle only one number on the scale that best represents your opinion. If you think that the best answer is between two numbers, e.g., between 1 and 2, please decide which one better represents your opinion and circle only that number. Example -3 -2 -1 0 1 2 3 + + + + + + + Fully Don't Fully Disagree Know Agree 228 APPENDIX E2 ID#: Task II Evaluation Questionnaire For each of the following statements you need to respond to three separate questions. First, based on the information you have examined, please tell us the extent to which you agree or disagree with the statement. Next, you need to evaluate whether the information in the OMS that you have examined, relating to the statement in question, is clear or ambiguous: a) whether the information can be interpreted in several ways and b) whether it may mean different things to different members of the Board of Trustees (remember, you are one of the board members). Example: Statement: Dr. A has bad relationships with coworkers. First, tell us the extent to which you agree with the statement. This will require you to think about specific information in the OMS that you have examined that either support of refute this statement. Second, you need to evaluate the ambiguity level of the information in the OMS related to the statement. If the information you examine explicitly states that Dr. A has bad, or good, relationships with coworkers, then you should choose -3 (fully disagree that the information is ambiguous), or close to -3 on the two ambiguity level ratings. However, if the information in the OMS related to this statement does not explicitly state that Dr. A has bad relationships with coworkers but appear to imply that this is the case (for example, the information appears to indicates that Dr. A got upset with coworkers easily in several occasions), then you need to formulate your opinion on how strongly the information implies what was stated in the statement. In other words, with this information, how comfortable you feel in agreeing or disagreeing that Dr. A has bad relationship with coworkers. Remember, You are supposed to evaluate the ambiguity of the information in the OMS related to the statement, not the statement itself. If you have any doubts about how to answer these questions, please ask the assistant now. 229 One of the main focuses of Dr. X ' s lab is to determine the presence of hypoxic cells so that treatment can be adjusted accordingly. - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 + + + + + + + Fully Neutral Fully Disagree Agree Dr. Y is said to be very impatient. - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 + + + + + + + Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 -1 0 1 2 3 Fully Neutral Fully Disagree Agree 230 Dr. X said that Dr. Y is a "perfectionist". - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree Researchers in Toronto, Princess Margaret Hospital, and Stanford are very interested in Dr. X and Dr. Y ' s research. - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 231 DRs. A and B have five children. - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 + + + + + + + Fully Neutral Fully Disagree Agree Dr. B used his wife's lab coat for an experiment. - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 232 Dr. B would like to see Cancer Control Agency disappear, through finding a cure the disease, much like what happened to TB hospitals. 2 3 + + Fully Agree In evaluating the above statement: -3 - 2 - 1 0 1 + + + + + . Fully . Don't Disagree Know a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. -3 -2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree Dr. B is more assertive than Dr. Y . - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 233 Dr. X and Dr. Y are more enthusiastic about their research than Dr. A and Dr. B. - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree Dr. X is sloppier and less conscientious about work than Dr. Y . - 3 - 2 - 1 0 1 2 3 + + + + + + + Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 234 Dr. A is more reflective and introspective than Dr. X . - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree Dr. X is rude and opinionated. At times, she tends to dominate her husband. - 3 - 2 - 1 0 1 2 3 Fully Don't Fully Disagree Know Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 235 Dr. B gets very excited whenever he talks, and Dr. Y is more relaxed. -3 + Fully Disagree •2 • + -•1 0 - - + - -Don't Know 1 • + 2 - + -3 + Fully Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. -3 -2 -1 0 Fully Neutral Disagree 1 - + • 2 • + -3 - - + Fully Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. -3 + Fully Disagree -2 • - + • •1 0 • - + Neutral 1 • + -2 • + 3 - - + Fully Agree Dr. B is warmer than Dr. X . -3 + Fully Disagree •2 • + --1 + -0 — i — Don't Know 1 • + 2 • + • Fully Agree In evaluating the above statement: a) I found that, the information I used to evaluate this statement can be interpreted in several ways. -3 + Fully Disagree -2 • + -•1 • + -0 • - + Neutral 1 + -2 - + -3 - - + Fully Agree b) I felt that, the information used in evaluating this statement may mean different things to different members of the Board of Trustees. -3 + Fully Disagree •2 •1 0 - - + Neutral 1 • + 2 - + -3 + Fully Agree 236 APPENDIX E3 Task Description lib In this part of the task, we would like you to repeat the same analysis that you have just performed with the Terry Fox Lab and Medical Biophysics Department, but this time with the help of another medium. We will return to you the questionnaire that you have just completed. We would like you to answer the questions again using the RED pen provided. Please do not try to second guess our objective. We want you to answer the questions based on your true beliefs. Your reevaluation should be based on your current opinions, which should be formulated based on the cumulative information you have seen. In answering the two ambiguity level questions (whether the information can be interpreted in several ways and whether it may mean different things to different members of the Board of Trustees), you should ask yourself whether the new medium has made the information related to the statement under evaluation less ambiguous. If it does not, then your ratings on the two questions should remain unchange from your previous ratings. However, if you feel that the new medium made the statement less ambiguous, then your new ratings should be moved toward the left of your previous rating, i.e., disagreeing that the information is ambiguous. Please do not change your original responses! Remember to think aloud while you are answering the questions! You will have 25 minutes for this part of the task. If you have any questions, please ask the lab assistant now. When you are ready to begin, inform the lab assistant. 237 APPENDIX E4 ID#: Perceived Usefulness Questionnaire You have just used two different OMS, which contain the same factual information in two different forms, to accomplish the evaluation task. For each the following Question, circle the appropriate number on the scale based on your subjective feeling related to using the OMS for the evaluation task that you have just performed using the second OMS. In answering these questions, please use the first OMS as your basis of comparison, i.e., how good is the second OMS, as compared to the first one, in the following aspects. 1) Using the second OMS enabled me to accomplish my evaluation task more quickly. -3 -2 -1. 0 1 2 3 Fully Neutral Fully Disagree Agree 2) Using the second OMS improved the quality of the evaluation I performed. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 3) Using the second OMS made the evaluation task easier to do. - 3 - 2 - 1 0 1 2 - 3 Fully Neutral Fully Disagree Agree 4) Using the second OMS enhanced my effectiveness in completing the evaluation task. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 5) Using the second OMS gave me more control over the evaluation task. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 238 6) Using the second OMS increased my productivity in performing the evaluation task. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 7) Using the second OMS allowed me to accomplish or view more information related to understanding the two departments than would otherwise have been possible. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 8) The second OMS enhanced the quality of my evaluation judgement. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 9) The second OMS supported all the various types of activities required to complete the evaluation task. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 10) Overall, I found the second OMS useful in performing the evaluation task. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 239 APPENDIX F. EXPERIMENTAL MATERIALS FOR EXPERIMENT 3 CONTENTS: F l -- Task Instruction III F2 — Appraisal Questionnaire F3 -- Task Instruction Ilia F4 ~ Biased Information Cues (Performance Indicators) F5 - "Dummy" Personality Profile F6 -- Task Instruction Illb 240 A P P E N D I X F l Task Instruction III In this part of the study, your task is to perform a five-year appraisal of Dr. D as Head of the Medical Biophysics Department (MBD). When you examine the information, please keep in mind that your objective is to evaluate Dr. D as a department head — you are trying to understand whether he is a good department head (manager). Therefore, you should keep in mind a set of characteristics of a good manager, such as leadership skills, communications skills, vision, etc. One of the criteria for being a department head is that the person must be an active researcher in the field of Medical Biophysics. This is because only an active research can better understand and appreciate the problems that other researchers face and, consequently, better able to lead the fellow scientists in achieving the mandate and mission of the department. Although the department head must be very knowledgeable about Medical Biophysics research in general, it is unrealistic to expect he/she to be a total expert in each sub-area of research within the field. The department head often play a very difficult role. He/she must be an active and productive researcher, and on top of that, he/she has to allocate time to perform administrative duties and to manage and lead the department. 241 You will be examining Dr. D's interview while he was interviewed for the department head position about five years ago. While this interview was conducted few years ago, it is still very relevant, because Dr. D's view on managing the department and staff has not changed. The interview consists of nine topics, related to nine different aspects of managing the department. These nine topics are: 1) Changes that he would make if he became the new department head; 2) Dealing with promised reports not arriving late; 3) Balancing between his research and administrative duties; 4) Level of details he wants to know about his fellow researchers' works; 5) Handling disagreements between him and his fellow researchers; 6) Dealing with unproductive researchers; 7) What he would do if the unproductive individuals tell him that they have been working on a potentially big discovery, but since things are still uncertain they can't tell him exactly what is going on; 8) Whether he would like his fellow researchers to approach him when they run into problems; and 9) His view of the highest level goal of the department. Please carefully examine the information and evaluate Dr. D. At the end of the task, we would like you to provide a written justification of your decision. You will have 20 minutes to perform this part of the task, including writing a written justification of your decision. Again, your performance in this task will be used to determine the additional cash awards. If you have any questions, please ask the lab assistant now. When you are ready to begin, inform the lab assistant. 242 APPENDIX F2 ID#: Appraisal Questionnaire Please evaluate Dr. D considering the information you have just examined. 1) What was your decision concerning Dr.D's contract? a) Renew his contract for life b) Renew for another five years c) Renew for two years d) Renew his contract but place him under probation e) Keep him as a scientist but not as the department head f) Transfer him to another department g) Terminate all his appointment with B C C R C For each of the following statements (2 - 14), please indicate to what extent you agree with it, on the scale of -3 (fully disagree) to +3 (fully agree). 2) Dr. D is primarily concerned with getting the job done. -3 - 2 . - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 3) Dr. D has a real interest in the welfare and overall satisfaction of those who work under him. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 4) Dr. D encourages people to work as a team. - 3 - 2 -1 0 1 2 3 Fully Neutral Fully Disagree Agree 5) Dr. D is willing to listen to work-related problems. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 243 6) Dr. D is capable of balancing between administrative duties and research. -3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 7) Dr. D has low tolerance of conflict. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 8) Dr. D encourages a cooperative working environment. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 9) Dr. D is friendly and easy to approach. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 10) Dr. D has no understanding of why people working under him do what they do. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 11) Dr. D asks people affected by his decisions for their ideas and opinions. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 12) Dr. D is aware of others' feelings. - 3 - 2 - 1 0 1 2 3 Fully Neutral Fully Disagree Agree 13) Do you feel that you have sufficient information in making this decision? - 3 - 2 - 1 0 1 2 3 Definitely Neutral Definitely Insufficient Sufficient 244 Appendix F3 Task Instruction I l i a In this part of the study, your task is to perform a five-year appraisal of Dr. D as Head of the Medical Biophysics Department (MBD). First, please carefully examine a set of performance indicators of M B D for the past five years during which Dr. D was the department head. You will also be given the performance indicators for the other five departments within B . C . Cancer Research Centre to serve as a basis of comparison. To help you better understand Dr. D, you will be given a personality test profile of Dr. D . This test was part of the standard test administered to employees of B C C R C when they are hired, and this personality test has proven to be very accurate. When you examine the information, please keep in mind that your objective is to evaluate Dr. D as a department head — you are trying to understand whether he is a good department head (manager). Therefore, you should keep in mind a set of characteristics of a good manager, such as leadership skills, communications skills, vision, etc. One of the criteria for being a department head is that the person must be an active researcher in the field of Medical Biophysics. This is because only an active research can better understand and appreciate the problems that other researchers face and, consequently, better able to lead the fellow scientists in achieving the mandate and mission of the department. Although the department head must be very knowledgeable about Medical Biophysics research in general, it is unrealistic to expect he/she to be a total expert in each sub-area of research within the field. The department head often play a very difficult role. He/she must be an active and productive researcher, and on top of that, he/she has to allocate time to perform administrative duties and to manage and lead the department. You will have 15 minutes to perform this part of the task. After you have examined this information, we would like you to perform a formal appraisal of Dr. D. The end result of this appraisal is to recommend whether to renew Dr. D's contract or terminate his employment. If you have any questions, please ask the lab assistant now. When you are ready to begin, inform the lab assistant. 245 Appendix F4-1 Biased Information Cue 1 (Total Funding) Total External Funding (in Thousand) Received from Grant Awarding Agencies 1990 1991 1992 1993 1994 Cancer Endocrinology- 906 987 1099 1546 1722 Cancer Imaging 32B 403 469 684 782 Epidemiology 2055 2248 2581 2588 2752 Medical Biophysics 763 842 4 50 780 920 Medical Oncology 649 793 864 981 1033 Terry Fox Lab 1564 1686 2678 2998 3042 Total Funding Received from Grant Awarding Agencies — * — Cancer Endocrinology ffl Cancer Imaging Epidemiology — « — Medical Biophysics —~*— Medical Oncology — T e r r y FOK Lab o -I 1 1 — — i 1 1990 1991 1992 1993 1994 246 Appendix F4-2 Biased Information Cue 2 (Average Funding) Average Funding (in Thousand) Per Scientist 1990 1991 1992 1993 1994 Cancer Endocrinology- 151 165 183 258 287 Cancer Imaging 108 134 156 171 196 Epidemiology 158 173 199 199 212 Medical Biophysics 127 140 114 121 160 Medical Oncology 93 113 123 140 148 Terry Fox Lab 196 211 268 273 277 Average Funding Per Scientist — • — Cancer Endocrinology II Cancer Imaging •A Epidemiology — H — Medical Biophysics Medical Oncology — • — Terry FOK Lab 100 f 50 -• 0 -I 1 1 1 1 1990 1991 1392 1993 1994 2 4 7 Appendix F4-3 Biased Information Cue 3 (Number of Scientists) Total Number of Full-Time Scientists 1 9 9 0 1 9 9 1 1 9 9 2 1 9 9 3 1 9 9 4 Cancer Endocrinology 6 6 6 6 6 Cancer Imaging 3 3 3 4 1 4 Epidemiology 13 13 1 3 2 13 13 Medical Biophysics 6 6 S 3 ,4 6 Medical Oncology 7 7 7 7 7 Terry Fox Lab 8 8 1 0 5 l l 6 11 Total Number of Full-Time Scientists 12 -• 10 -• 8 4 6 4 -• 2 -• - • — Cancer Endocrinology -m— Cancer Imaging & Epidemiology — * — Medical Biophysics * Medical Oncology - • — Terry Fox Lab 1990 1991 1992 1993 1994 Note: 1. One new Scientist joined the team. 2. Three scientists left, and three replacements were hired. 3. Two scientists left, and only one was hked subsequently. 4 . One more scientist was hired. 5. Two scientists joined the Lab. 6. One more scientist was hired. 248 Appendix F4-4 Biased Information Cue 4 (Number of Supporting Staff) Number of Supporting Staff Members 1 1 9 9 0 1 9 9 1 1 9 9 2 1 9 9 3 1 9 9 4 Cancer Endocrinology 24 26 26 32 32 Cancer Imaging 14 14 14 15 15 Epidemiology 45 50 5 5 42 55 Medical Biophysics 25 30 20 31 31 Medical Oncology 35 35 35 35 35 Terry Fox Lab 36 36 4 3 48 48 1. Including post-docs, lab technicians, and secretaries. 40 30 20 10 Total Number of Supporting Staff Members 60 T — 50 -- — 1990 1991 1992 1993 Cancer Endocrinology * Cancer Imaging tk Epidemiology Medical Biophysics * Medical Oncology —•— Terry FOK Lab 1994 249 Appendix F4-5 Biased Information Cue 5 (Total Number of Publications) Total Number of Publications 1990 1991 1992 1993 1994 Cancer E n d o c r i n o l o g y 17 14 19 20 19 Cancer Imaging 6 7 6 9 10 E p i d e m i o l o g y 30 27 22 28 25 M e d i c a l B i o p h y s i c s 20 2 3 13 7 10 M e d i c a l Oncology 18 15 20 13 16 T e r r y Fox l a b 30 33 38 45 46 Total Number of Publications 50 45 -40 35 30 25 20 -4:-15 -• 10 5 5 0 - — 1990 - • — Cancer Endocrinology Cancer Imaging k. Epidemiology ~ k ~ T Medical Biophysics - * — Medical Oncology - • — Terry Fox Lab 1991 1992 1993 1994 250 Appendix F4-6 Biased Information Cue 6 (Average Number of Publications) Average Number o f Publications 1990 1991 1992 1993 1994 Cancer Endocrinology 2 . 8 2.3 3.2 3.3 3 . 2 Cancer Imaging 2.0 2.3 2.0 2 . 3 2.5 Epidemiology 2 . 3 2 .1 1. 7 2.2 1 . 3 Medical Biophysics 3 . 3 3.0 2 . 6 1.6 2 . 0 Medical Oncology 2 . 6 2.1 2 . 3 2 . 4 2.5 Terry Fox Lab 3 8 4.1 3 . 8 4.1 4 . 2 1990 Average Number of Publications - • — Cancer Endocrinology •if— Cancer Imaging -4t— Epidemiology x Medical Biophysics Medical Oncology - • — Terry FOH Lab 1991 1992 1993 1994 251 Appendix F5 "Dummy" Personality Profile He has the tendency to be critical of himself. At times he is extroverted, affable, sociable, while at other times he is introverted, wary, reserved. He prefers a certain amount of change and variety and becomes dissatisfied when hemmed by restrictions and limitations. He prides himself as being an independent thinker and does not accept others' opinions without satisfactory proof. 252 A P P E N D I X F6 Task Instruction I l lb In this task, you will be given another piece of information about Dr. D — his answers to the interviewer while he was interviewed for the department head position about five years ago. While this interview was conducted few years ago, it is still very relevant, because Dr. Dr. D's view on managing the department and staff has not changed. The interview consists of nine topics, related to nine different aspects of managing the department. These nine topics are: 1) Changes that he would make if he became the new department head; 2) Dealing with promised reports not arriving late; 3) Balancing between his research and administrative duties; 4) Level of details he wants to know about his fellow researchers' works; 5) Handling disagreements between him and his fellow researchers; 6) Dealing with unproductive researchers; 7) What he would do if the unproductive individuals tell him that they have been working on a potentially big discovery, but since things are still uncertain they can't tell him exactly what is going on; 8) Whether he would like his fellow researchers to approach him when they run into problems; and 9) His view of the highest level goal of the department. Please carefully examine this new information and reevaluate Dr. D. At the end of the task, we would like you to provide a written justification of your decision. You will have 20 minutes to perform this part of the task, including writing a written justification of your decision. Again, your performance in this task will be used to determine the additional cash awards. 253 A P P E N D I X G . POST- E X P E R I M E N T A L M A T E R I A L S CONTENTS: Demographic Background Questionnaire 254 A P P E N D I X G D E M O G R A P H I C D A T A ID#: The following information is strictly confidential. We need to collect this information in order to report the overall profile of our participants (e.g., how many percent are female). Your identity will not be revealed. A G E : SEX: M F STATUS: (please check only one) Undergraduate Circle your year I II III IV M B A I M B A II M.Sc. Ph.D. Other MAJOR FIELD OF STUDY: ETHIC B A C K G R O U N D : What language do you speak at home mostly? What language does your mother speak at home mostly? What language does your father speak at home mostly? How long have you been living in Northern America? 255 1. If new information relevant to a decision is made available to you, how likely would you overturn your original decision (assuming that the new information is equally important as the original information that you based your decision on? 1 2 3 4 5 6 7 + + + + + + + Very Neutral Very Unlikely Likely 2. In your daily life, do you stick to your decision after you have decided upon it, even after a piece of somewhat contradictory new evidence is presented to you? 1 2 3 4 5 6 7 + + + + + + + Very Neutral Very Unlikely Likely Within the last 12 M O N T H S , please indicate how often you have use the items listed below. (Circle one appropriate number in each row according to the following scale). 1. A Macintosh or its compatibles 2. Windows on I B M P C or its compatibles 3. A Mouse 4. Mosaic 5. Netscape 6. Other Web Browser (Please Specify: ) 7. Gopher Not Less About About About About More at than 1-3 once 2-4 once than all once times per times per once per per week per day per month month week day 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 ! 2 3 4 5 6 7 256 A P P E N D I X H . CONTENT: Order Effect of Experiment and Task 257 APPENDIX H. ORDER EFFECT OF EXPERIMENT AND TASK A l l subjects who participated in the experiments were required to go through all the three experiments. A true factorial design would have required conducting the three experiments in six different orders (e.g. 1-2-3, 2-3-1, 3-1-2, etc.). In our case, Experiment 1 had to be conducted before Experiments 2 and 3 because of the nature of the experimental tasks employed. The materials used in Task 1 provided an introduction to the organization. Subjects needed to have a basic understanding of the organization before they could perform the decision tasks in Experiments 2 and 3. Therefore, there were only two sequences in which the experiments could be conducted (1-2-3 and 1-3-2). At an individual experimental level, there were two experimental conditions in Experiment 1 (text-based OMS vs. multimedia OMS), two conditions in Experiment 2 (text-based-second vs. multimedia-second), and four conditions in Experiment 3 (text only control group, video only control group, text with biased cue experimental group, and video with biased cue experimental group). As a result, there were 16 (2 x 2 x 4) possible task orders. Taken together with the two possible sequences (1-2-3 and 1-3-2) discussed above, there were 32 (16 x 2) Experiment-task orders. Using statistical methods, we explicitly tested that the order in which experiments and tasks within experiments were conducted did not affect the results. Four one-way A N O V A tests were used to check whether the counter-balancing of the order effect was successful. The first test examined the effect of task order within Experiment 1 (text-first vs. video-first) on the performances of Experiments 2 and 3. The 258 second.test was for cases in which Experiment 2 was conducted before Experiment 3. It checked the effect of the order in which tasks within Experiments 1 and 2 were conducted on the performance of Experiment 3. The third test was similar to the second test, but it was conducted for cases in which Experiment 3 was conducted before Experiment 2. It examined the order effect of tasks within Experiments 1 and 3 on Experiment 2. Finally, the fourth test checked whether the order in which Experiment 2 or 3 was conducted first had any effect on the performances of subsequent tasks. The results of the four tests are summarized in Table H - l . As shown in the table, none of the variables show significant order effect. This indicates that the results reported in Chapters 3 ,4 , and 5 were not caused by any particular order in which tasks or experiments were performed. 259 Table H - l . Summary of the Analyses on Order Effect Description DF F-value p-value I) Order Effect of Tasks within Experiment 1 (Text-First vs. Video-First) on Experiments 2 and 3 1) Perceived usefulness (Exp 2) 1,78 0.310 0.580 2) Ambiguity reduction for less-equivocal task (Exp 2) 1,79 0.016 0.899 3) Ambiguity reduction for equivocal task (Exp 2) 1,79 0.732 0.395 4) Time taken for less-equivocal task (Exp 2) 1,79 0.034 0.854 5) Time taken for equivocal task (Exp 2) 1,79 1.066 0.305 6) Final Evaluation (Exp 3) 1,79 1.970 0.165 7) Change in Evaluation (Exp 3) 1,39 0.515 0.477 II) Order Effect of Tasks within Experiments 1 and 2 (If Experiment 2 was Conducted Before Experiment 3) on Experiment 3 1) Final Evaluation (Exp 3) 3,39 0.291 0.831 2) Change in Evaluation (Exp 3) 3,39 0.701 0.558 III) Order of Tasks within Experiments 1 and 3 (If Experiment 3 was Conducted Before Experiment 2) on Experiment 2 1) Perceived usefulness (Exp 2) 7,39 0.657 0.706 2) Ambiguity reduction for less-equivocal task (Exp 2) 7,39 1.346 0.262 3) Ambiguity reduction for equivocal task (Exp 2) 7,39 0.118 0.997 4) Time taken for less-equivocal task (Exp 2) 7,39 0.325 0.937 5) Time taken for equivocal task (Exp 2) 7,39 0.613 0.741 IV) Order Effect of Experiment 2-First vs. Experiment 3-First 1) Perceived usefulness (Exp 2) 1,78 0.392 0.533 2) Ambiguity reduction for less-equivocal task (Exp 2) 1,78 0.373 0.543 3) Ambiguity reduction for equivocal task (Exp 2) 1,79 1.438 0.234 4) Time taken for less-equivocal task (Exp 2) 1,79 1.931 0.169 5) Time taken for equivocal task (Exp 2) 1,79 1.606 0.209 6) Final Evaluation (Exp 3) 1,79 1.823 0.181 7) Change in Evaluation (Exp 3) 1,39 1.156 0.289 260 A P P E N D I X I R A N D O M I Z A T I O N C H E C K S CONTENT: Results of Randomization Checks 261 APPENDIX I RESULTS OF RANDOMIZATION CHECKS Experiment 1 Experiment 2 Experiment 3 2 1 Age t(79) = 1.13, p=0.263 t(79)=0.41, p=0.685 F(79) = 1.009, p=0.394 Year of Studies t(79) = 1.80, p=0.076 t(79)=0.56, p=0.579 F(79)=0.306, p=0.821 Gender X 2(l)=0.453, p=0.501 X 2(l)=0.503, p=0.823 ^(3) =0.955, p=0.812 Major Field of Study ^(2) =0.3964, p=0.529 ^(2) =0.135, p=0.935 ^(6) = 1.714, p=0.994 Language Background X 2(l)=0.238 p=0.626 J^(l)=0.238, p=0.626 ^(3) = 1.428, p=0.699 Age and Year of Studies are continuous variables. Gender was coded as a binary variable (l=male and 2=female). Major Field of study has three possible values (l=Business, 2=Science, and 3= Arts). Language Background is also a binary variable (l=English native speaker and 2=Non-English native speaker). For continuous variables, t-tests or one-way A N O V A s were used for the analyses. For categorical variables, Chi-Square tests were used. Unlike Experiments 1 and 2, which only have two experimental conditions, Experiment 3 consists of four experimental conditions. Since t-tests can only be used to compare two conditions, one-way ANOVA was used instead. 262 APPENDIX J . HUMAN INFORMATION PROCESSING THEORY CONTENTS: Human Information Processing 263 A P P E N D I X J . H U M A N I N F O R M A T I O N P R O C E S S I N G J . l Human Information Processing Theory Functionally, there are three subsystems within human memory: the sensory register, working memory, and long-term memory. Information from the environment entering the sensory register is stored very briefly. This enables decision makers to process large amounts of information from the external environment at very high speeds. Information that enters the sensory register is then passed to working memory. The capacity of working memory is limited, with most people retaining only seven "chunks" of information at one time (Miller, 1956; Simon, 1974). Information is processed in working memory. Information from the environment, which passes through the sensory register, is combined with information previously stored (and subsequently retrieved from long-term memory) to form new knowledge. Meanwhile, new information continues to enter working memory through the sensory register. To prevent information in working memory from being replaced by incoming information, it must be continually refreshed or rehearsed. Consequently, a large amount of information that passes through the sensory register never actually enters working memory. However, if attention is attracted by external stimuli, existing information in working memory is either replaced or distorted by incoming information. As a result, large amounts of information are lost without entering long-term memory. The limited capacity of working memory leads people to adopt various information processing strategies or heuristics to save cognitive effort. For example, when faced with a large amount of information, decision makers selectively focus on a few manageable pieces 264 of information at one time. Although these strategies reduce the decision maker's cognitive load, they often lead to suboptimal solutions or failure to consider important information (Todd and Benbasat, 1991; 1992). In fact, one fundamental principle of human information processing models (e.g., Norman and Rumelhart, 1970; Sperling, 1970) is that people do not receive and process all the sensory information that they encounter. This principle is called cue selection. Another principle of human information processing models relevant to our discussion is that recalling information involves reconstruction of the whole event from the partial information received and processed (Neisser, 1976). For example, people often use their background knowledge of events to fill in missing details. This principle is called information reconstruction. When in working memory, information can be represented in several forms, such as a visual-spatial (e.g., images) or semantic code (e.g., propositions). The prevalent representation of linguistic information is an articulatory-acoustic code (Conrad, 1964). That is, in processing text-based information, people transform the printed words into a speech-based code before further processing (Baddeley, 1979, Kleiman, 1975). Recoding is especially necessary when the materials are difficult to understand or when there are large amounts of information to be processed. This is because verbal coding facilitates working memory storage and/or processing, thereby enabling readers to integrate the information more rapidly (Howard, 1983). Although verbal recoding is thought to be an automatic process, i.e., one that occurs without consuming cognitive resources, under certain situations, verbal recoding can become a cognitive intensive task. This occurs when the information is difficult to understand 265 (Howard, 1983). After information is successfully processed in working memory, it is then transferred to long-term memory for storage. Once it enters long-term memory, it is thought to be permanently stored. Beyond this point forgetting is a problem of retrieval, i.e., the information is not lost, but rather, the retrieval mechanism was not properly invoked. Therefore, the key to effective retrieval of information from memory is to provide necessary cues to invoke the retrieval process. J.2 How is Information Structured in Memory? One line of research in human memory is based on some variation of structural models. For example, Anderson's (1976) Adaptive Control of Thought (ACT) theory represents knowledge as interconnected structures. Pieces of information are stored in nodes, which are connected by linkages. These linkages act like a cross-referencing system in a file system. Linkages not only relate information and ideas in a web-like fashion, but also connect them functionally by defining the nature of their relationship. Each node can serve as a retrieval point for a piece of information. Therefore, the more interconnected a piece of information/idea (more linkages to other nodes), the more likely and more quickly it will be retrieved (Kozma, 1986). A C T views information not as merely a collection of facts or data. Rather, to use information, decision makers need to understand the information by breaking it down, digesting it, relating it to existing highly complex web of interconnected knowledge, and building new links to the existing knowledge or restructuring old knowledge. J.3 What is the Content of Long-Term Memory? There is considerable evidence suggesting that information in long-term memory may be 266 stored both semantically (meaning) and pictorially (images) (e.g., Kosslyn, 1980; Paivio, 1971; Shepard and Cooper, 1982). Like semantic information, images stored in long-term memory can be retrieved into working memory in response to stimuli in the environment. The images retrieved into working memory can be manipulated dynamically. For example, they can be enlarged or rotated. They can be skeletal or be made more detailed. This allows decision makers to cognitively process different parts of the image at different levels of detail. Most importantly, if the same information is stored both semantically and pictorially, it is easier and more likely to be retrieved. This is because both representations can be cross-referenced (in ACT ' s term, more linkages between them and other stored information). For example, Levie and Lentz (1982) showed that congruent pictures paired with text improve reading comprehension over either text or pictures alone. 267 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0087182/manifest

Comment

Related Items