Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An instructional model for educational technology within applied health sciences Rodenburg, Dirk 1996

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-ubc_1997-0077.pdf [ 15.85MB ]
JSON: 831-1.0064570.json
JSON-LD: 831-1.0064570-ld.json
RDF/XML (Pretty): 831-1.0064570-rdf.xml
RDF/JSON: 831-1.0064570-rdf.json
Turtle: 831-1.0064570-turtle.txt
N-Triples: 831-1.0064570-rdf-ntriples.txt
Original Record: 831-1.0064570-source.json
Full Text

Full Text

AN INSTRUCTIONAL MODEL FOR EDUCATIONAL TECHNOLOGY WITHIN APPLIED HEALTH SCIENCES By Dirk Rodenburg A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF EDUCATION DEPARTMENT OF EDUCATIONAL STUDIES ADULT EDUCATION PROGRAM We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA September 1996 ^ ) D i c k R o d e n b u r g In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia. 1 agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of The Unfversity of British Columbia Vancouver, Canada Date n^^S" /j f DE-6 (2/88) Abstract This thesis will outline a model for instruction for educational technology within applied health sciences that explicitly targets the conceptual development of the student. The model is based on a first principle integration of instructional theory, instructional design, clinical/medical reasoning, and user modeling. Utilizing a patient management problem format, the model relies on three fundamental tasks carried out by the end-user: key feature selection, analogous model selection, and the choice of a clinical management strategy. It is suggested that these tasks foster deep and integrated thinking on the part of the end-user as a consequence of the necessity to articulate and compare conceptual frameworks. The model outlines a process in which content and clinical context can be "coded" and weighted to allow for some limited inferencing by the presenting system against several predefined conceptual models. The inferencing is based on a simple Baysean formula in which a look-up table of probabilities assigned to each element of each node of the clinical management problem is used to calculate the probability of a conceptual model being carried by the end-user. The inferencing and content structure is specifically designed to provide strong instructional and metacognitive support to the end-user within the targeted domain. The model proposes an extensive feedback cycle in which the end-user is given the option of reviewing each stage in the decision making process, and comparing it to the "ideal" representations of an expert. No formal evaluation of the efficacy of the proposed model is offered. Table of Contents Abstract ii Table of Contents iii List of Tables v List of Figures vi Acknowledgments vii Introduction 1 Summary 6 Chapter 1: Information Technology and Teaching 8 The adoption of a distributed learning model 8 Teaching in the Health Sciences 14 Chapter 2: The Contribution of Cognitive Science 16 Types of Knowledge 16 Declarative 16 Procedural 20 The Expert/Novice Distinction 22 Summary 24 Chapter 3: The Instructional Implications of Cognitive Science 26 Summary 32 Cognitive Science and Assessment 32 Moving Forward 35 Chapter 4: Constructivism 39 Chapter 5: Situated Cognition 47 Implications for Assessment 55 Summary 58 Chapter 6: Constructivism/Situated Cognition as a Normative Framework 61 Beliefs and Knowledge 63 Conceptual Understanding as Agency 65 Critical Theory 66 Role of the Educator 69 Summary 70 Chapter 7: Implications of Constructivism/Situated Cognition for Instructional Design 72 Challenges to Traditional ISD 73 Examples of Systems based on Constructivism/Situated Cognition 85 Simulations 89 •Summary 92 iii Chapter 8: Concept Mapping 94 Implications of Instructional Design 96 Chapter 9: User Modeling 99 Moving Forward... 102 Chapter 10: Clinical Reasoning 103 Summary 109 Chapter 11: Facilitating Medical Reasoning 112 Summary 122 Evaluating Clinical Reasoning 123 Simulations and Clinical Assessment 128 Patient. Management Problems 129 Summary 131 Chapter 12: A Mod|el for Instruction 132 The Design Process 134 Clinical Context 137 Management Strategies 138 Feedback 139 Conceptual Diagnosis 142 Summary 152 Interaction with the System: Students'Perspective 156 Chapter 13: Conclusion 158 Appendix A 161 References 171 iv List of Tables Table 1 Cognitive Dimension and Assessment Tools 37 Table 2: A Comparative Representation of Perspectives and Construct 54 v List of Figures Figure 1: Map of Intent Figure 2: Proposition "the man fixed the line" Figure 3: The Expert/Novice Distinction Figure 4: The Design Flow-chart Acknowledgments There are many silent partners who have contributed to the completion of this work, and I would like to take the time to gratefully, and publicly, thank them. Marc Broudo, without whom none of this would have been possible, and who, as a friend and college, never once acted in anything but a completely supportive and encouraging way. Gordon Page who was burdened almost daily with having to listen to me babble at the office door, but whose patience, feedback, and mentorship always helped to set me on some line of coherent reasoning. The rest of the office staff at the Division of Educational Support and Development at UBC including Margaret, Shirmilla, Maureen and Maureen, Kiko, Patrick and Charlene for creating the best work environment I've ever experienced. Daniel Pratt for his insight, mentorship and his ability to create a profoundly reflective experience. Kjell Rubenson for his support and guidance. My parents and family for their unfailing confidence in me. Dr. Rick Arseneau for his gifted reasoning and for helping to get me the job in the first place... And finally, my partner Margaret without whom I'd never have had the courage to start, and who bore an endless variety of burdens to help get me to the end. Introduction This thesis will outline a model for instruction within educational technology that explicitly targets the conceptual development of the student within specific domains. The model is a response to a number of issues that I have been actively thinking about as a consequence of my work as a multimedia developer for the Division of Educational Support, Office of the Coordinator, Health Sciences, University of British Columbia (UBC). Much of my work within Health Sciences has involved applications within medical education, and it is out of my experience within this area, and from my associations with Gordon Page and Marc Broudo, that I became interested in how educational technology might help to support a better integration between clinical practice and theory. This complements a concerted effort by the Faculty of Medicine at U B C to move from a didactic, lecture based model of delivery to a problem-based learning protocol. In essence, this paper is an attempt to meld the theoretical and practical domains of educational technology, specifically multimedia, with those of medical education. As such, it must cover a lot of ground, ground that in a number of cases overlaps. (For example, much of the current understanding within medical reasoning has been informed by cognitive science. The basic elements of those principles are now being actively incorporated into educational technology as the limitations of the earlier behaviourist model become better known.) In my exploration of issues within both medical reasoning and educational technology, some common themes seemed to emerge that I have attempted to meld into a coherent and theoretically supported model. Figure 1 outlines the approach that this thesis will take in developing this framework. The overall goal is the first principle derivation of an model, within health sciences education, that provides one possible framework for developing instructional materials delivered within an interactive multimedia environment. In deriving these principles, I must appeal to literature in three areas: learning theory, instructional design theory, and medical pedagogy and reasoning. Each of these areas contributes to issues in cognition, pedagogy, and assessment. Since the literatures covered are diverse I treat each literature separately within specific chapters, identifying those elements from each literature that will later contribute to the proposed model. A summary of the literature is provided at the end of most chapters. The final chapter is a detailed description of the proposed model and the integration of those themes and elements identified within the body of the thesis. This thesis will explore and account for the following areas of research: • the nature of medical/clinical reasoning and knowledge and the mechanisms for evaluating clinical reasoning and knowledge. Since much of this area is based on cognitive science, an examination of that literature in some limited form is essential; • instructional strategies fostering the development of "good" clinical reasoning and knowledge; • theoretical positions which are postulated to engender meaningful learning, including constructivism, cognitive feedback, conceptual mapping and situated cognition; • an analysis of the theoretical principles of instruction. • these principles as applied to instructional technology. These are not, of course, entirely discrete areas of literature and as stated, overlap does occur; for example, the general education community inform, and are informed by, medical educators and instructional technologists. But as each literature tradition has essentially a distinct focus, the task will be the creation of a coherent and meaningful framework for instructional delivery which is theoretically grounded and achievable. The essential basis for most of the discussion within these literatures is cognitive psychology. It provides a language that the other literatures, to some extent, share. This thesis, therefore, begins in Chapters 1, 2 and 3 with an exploration of the terminology and concepts within cognitive psychology, since they will be referenced frequently in subsequent chapters. It should also be recognized that the language of cognitive psychology gains ambiguity as it moves across domains, and it is important to provide the reader with a reference chapter in which this language is clearly defined. Cognitive psychology is the fundamental framework characterizing most of the current understanding about the nature of expertise, and the process of applying knowledge within a particular domain. It also forms the basis for much research within education in general, but serves as a critical platform within medical education. Constructivism is a second major theoretical framework that has, to some extent, emerged from and been characterized by those researchers working within educational technology. Although it shares some theoretical ground with cognitive psychology in terms of the nature of knowledge, the primary focus in this literature is on the meaningfulness of knowledge, the social construction of that knowledge and the critical determinants of context and the authenticity of experience. In its emphasis on the importance of "deep" understanding, constructivism is a departure from a rational/behaviourist approach to instruction (which usually targets knowledge gains as measured against criterion referenced benchmarks). These benchmarks usually employ assessment strategies that, if used alone, are argued by constructivists to be inadequate. In a constructivist framework, instruction is seen as a fluid process in which the learner plays an active role in the collection and integration of knowledge. Knowledge is also seen by constructivists to be indexed by the environment in which it is learned. Consequently, constructivism forms the theoretical umbrella for an those challenging a systems approach to instructional design and it is thus an important literature to examine. Within this theoretical umbrella, two perspectives, situated cognition and cognitive apprenticeship, are rapidly becoming key theoretical "lenses" for viewing the process of instructional design. Constructivism and situated cognition are treated Chapters 4 and 5 . Obviously constructivism and situated cognition have profound implications for instruction, assessment and consequently, instructional design. These implications are examined in detail. As previously stated, constructivism is a theoretical stance which embodies a strong emphasis on generating "deep" representations of knowledge: knowledge that is conceptual, as opposed to declarative, in nature. How to define that distinction, however, is a difficult challenge, one that is essentially epistemological in nature. In Chapter 6,1 make the argument (not new) that good instructional processes promote learner agency, the ability to make an act on personal choices. The suggestion in this chapter is that an instructional process must specifically identify and target those concepts that increase the learner's agency within the instructional domain. This has implications for instructional design and the notion of professional competence, and adds a small but critical piece to the model proposed at the end of this thesis. In Chapter 7, the implications of a construedvist/situated cognition approach for instructional design is explored. Chapter 8 deals with a related but distinct approach to the characterization of knowledge: the notion of concept mapping. Concept mapping is important to the final model, because it specifically targets conceptual development and provides the metaphor of a "map". Thus the conceptual areas targeted by an instructional process can, to some extent, be spatially represented. It begins to offer the very attractive possibility of "navigating" through a conceptual domain. The constructivist focus on the role of the relationship between knowledge, and the constructs within which that knowledge is applied, leads to the obvious conclusion that the closer an instructional process models an authentic environment the better that instruction will be (depending, of course, on how educational outcomes are evaluated). Chapter 8, therefore, also provides some review of the literature relating simulation and pedagogical efficacy. In Chapter 9, the notion of user modelling is introduced. This forms an essential part of the proposed instructional model, and is the process by which data concerning the end-user is accrued by the instructional system and applied a series of existing user models. If one of these models does, infact, accurately portray some part of the end-user's conceptual framework, the system can then dynamically move to help the end-user in a specific instructional way. This chapter represents the end of the first major area covered by the thesis: those areas that apply generally to knowledge, pedagogy and instructional design. It also represents the start of the second major area in this thesis which examines medical education in Chapters 10 and 11. It is in this area that I explore how medical knowledge, teaching and evaluation are viewed in the literature. From each of these areas I identify some elements that are used by the instructional model. These elements are the notion of prototypicality, authentic context, the expert/novice distinction in diagnostic accuracy, and the use of key features in evaluation. Finally, Chapter 12 outlines a model and process for instruction which is essentially an integration and distillation of the elements identified throughout the body of the thesis. The model is a simple one, a "close-ended" patient management problem (PMP) in which the end-user is presented with non-trivial clinical decision environment. What is different perhaps from traditional PMP, is that the structure of the environment is specifically conceptual. The model, by providing a process for coding and structuring content, attempts to allow a stand-alone system to provide instructional support, an authentic environment, and comprehensive evaluation (essentially the elements of cognitive apprenticeship) to the end-user. The system responds to user-behaviour through the utilization of a series of weighted relationships between choice elements and conceptual models. These models have been defined (pre-hoc) as likely (mis)conceptions to be carried by students. The probability that one of this defined models is being carried is calculated through simple Bayesian inferencing which in turn is dependent on a look-up table of comparative values. To allow this inferencing to take place, the end-user is asked to identify key features at each stage of the clinical environment (if considered appropriate), select one of a series of analogous models, and to select a clinical course of action. For each of these actions, the system responds by updating the probabilities of each conceptual model being carried by the end-user. A l l user behaviour is tracked and that data defines part of the feedback process. Summary This model is an attempt to actualize my belief that good supporting materials in any domain is a consequence of well planned and theoretically well grounded approaches. Although multimedia has moved past the esoteric into the mainstream of educational practice, but there is still, in my experience, a pervasive artifact of the novelty of the technology. That artifact is the belief that the technology, not the content, is the critical factor in the efficacy of an instructional package. Although the "hype" surrounding multimedia has lessened, this artifact is still a significant part of the rhetoric for many organizations dealing with teaching or training. There is, I think, an alluring and seductive side to current multimedia technology, a seduction level that currently outstrips some of the actual benefits realized from development efforts. This paper is an attempt to bring some theoretical focus to that arena. How well I've succeeded remains to be seen. / FIGURE 1: Map of Intention 7 Chapter 1: Information technology and Teaching The adoption of a "distributed learning" model There has, within the past few years, been a surge in interest in the utilization of multimedia and information technology (IT) providing for "distributed learning" among those involved in higher education. The term "distributed learning", as defined below, has been adopted by many people who seek to develop the infrastructure necessary to deliver digitally-based resource materials. Instructional technology is seen as something of a panacea for the resolution of the crisis resulting from increasing pressure to provide equal, or better, instructional services at a reduced cost. At the same time, instructional technologies are justified as contributing to egalitarian principles of access and student empowerment. These technologies can also provide the potential to increase the market base for content expertise. •"Distributed Learning: v A distributed learning environment is a learner-centred approach to education, which integrates a number of technologies to enable opportunities for activities and interaction in both asynchronous and real-time modes. The model is based on blending a choice of appropriate technologies with aspects of campus-based delivery, open learning systems and distance education. The approach gives instructors the flexibility to customize learning environments to meet the needs of diverse student populations, while providing both high quality and cost-effective learning.' Institute for Academic Technology, University of North Carolina, March, 1995" * As quoted in a paper from UBC's Center for Educational Technology ("") At the University of British Columbia, the committee primarily concerned with a move towards digitally based resources is the Advisory Committee on Information Technology (ACIT), a committee made up of a diverse group of faculty and staff. A sub-committee examining digital scholarly resources (DSRs) state: "We do suggest that, in the age of networked DSR, the roles of the scholarly researcher, the teacher, the learner, and the information provider in the university are moving closer together. Indeed, in the academic community we all play each of these roles at different times. Similarly, the format and contents of DSR and of teaching/learning materials are also converging, presenting analogous challenges and opportunities." (Working Paper, 1996) They further state: "Although, for present purposes, we do not explicitly deal with teaching materials, it is clear that the ability of teachers and students to communicate interactively through sophisticated, networked technology has major implications for the way in which teaching and learning will occur in the future. These altered ways of doing research and teaching will fundamentally change our universities." (ibid) Based on the above, it seems a "fait acompli" that instructional technology will become a ubiquitous part of student life at UBC. The reasons for the movement toward an adoption of this technology can be postulated to be primarily fueled by: • fears of reduced funding; • the perceived consequent need to increase teaching "efficiency"; • a belief in the value of access to an integrated environment providing multiple forms of media • the increase in faculty/student access to the hardware and tools required by multimedia; • at best, a desire to increase student autonomy and to focus teaching on "higher-order" processes rather than the repetitive delivery of "base-level" conceptual/factual information. At worst, an opportunity to shift responsibility for student learning from the institution to the student; • a significant reduction in the dedicated expertise required to develop computer-based training materials. In particular, in spite of current band-width restrictions, the World Wide Web (WWW) is seen as a natural repository for much of the course content available to students. Much of the rhetoric by proponents of this technology, however, seem to equate student access to information with instruction or teaching. Although one can argue that increasing access to content that may include a variety of media is important, it does not, to my mind at least, represent anything else than another channel of distribution for traditional forms of media. This is certainly true of most W W W sites developed locally at U B C ; in general, these sites are hierarchically ordered indices of content. This kind of content presentation can be important, in the same way that a brochure or textbook can be important, but it does not reflect the conditions necessary for the "information revolution" or "paradigmatic shift" in teaching that proponents suggest is taking place (this kind of suggestion places electronic brochures at a "revolutionary" level, hardly a tenable conclusion). Providing access to content, even content that is well indexed and annotated by the instructor, is only one of the roles that traditional teaching and tutorial methods fulfill. A much more important aspect of instruction is the "act" of teaching itself. Concepts of what comprises teaching or instruction are diverse, of course, and grounded in deeply held epistemological, philosophical and pragmatic considerations (Pratt, in press) In most cases, however, those considerations generally move far beyond the mere provision of content. Educators (and most lay people), I would suggest, see teaching as a far more active process, one in which the teacher or instructor acts as an "agent of change" helping (in the best cases) the learner to develop some form of meaningful understanding. This role demands an active and deliberately applied "pedagogical intelligence" which dynamically responds to changing student cognition/behaviour. It is an essential part of this paper to argue that the equating of content presentation and teaching misses the boat. To argue the contrary would be to suggest that any course at any institution can be replaced by a well written textbook. Although one can substantially support the idea that some text books are better than others (in any domain), I do not think that it can legitimately be argued that a text book can completely supplant the role of a competent instructor or teacher. One can argue that increasing the distribution of course materials is a good thing. One can better argue that if that content includes media specifically chosen to facilitate conceptual development consistent with the goals of that course then that is a good thing. Finally, if that media content is developed by, and matched to a well thought out pedagogical strategy, then that is the best thing of all. (My presumption (obviously) is that the goal of any course is the student's development of meaningful representations of the knowledge targeted by that course.) As Pratt (in press) indicates, the process of teaching demands answers to a number of questions that are deceptively easy: what am I teaching, how do I assess the learner's progress, when have I taught enough, and what is good practice? Answering these within a coherent framework is enormously difficult. Rather than leading to resolution, these questions usually lead to many others. These questions are the surface reflection of many deeper questions of theory that force the educator, in answering them, to examine her epistemological and philosophical basis for practice. When faced with these questions, not many of us are able to provide a cogent framework in which to theoretically situate ourselves. The absence of this framework, however, does not necessarily mean bad practice. For many of us, practice is as practice does, and a search for underlying assumptions may or may not help us navigate the murky waters of practice with any greater efficacy. In my experience, we make decisions "on the fly" as we teach based on our understanding of what we teach and whom we teach. These decisions, therefore, are highly contextually situated. We trust our "practical knowledge", based on past experience and our own understanding of teaching, to use the specific signposts in a situation to make "good" decisions. Thus, although we have a "pre-hoc" series of social, philosophical and epistemological assumptions that drive our practice, we do not necessarily need to articulate these in order to teach. This articulation does become important, though, when we are explicitly asked the question, "How do we know when we do well?". The concept of "good" practice can be nested in multiple levels of the educational arena, including instructional design, instruction, curriculum design and curriculum planning. Each of these stages encompasses a particular realm of theory and action, and therefore carries political and social implications as well as those of epistemology. For the educational software designer these issues remain the same, but the problems in answering them are to some extent exacerbated by the fact that the designer is not free, as is the teacher, to respond "on-the-fly" to highly contextual situations. There is, of course, a wide range of educational software types, each with its own role. Some systems are meant to act in conjunction with classroom activities as just another tool in the teachers' pool of resources. For many educational software designers and theorists, however, the promise of the emerging technologies has, for some time, been the creation of a complete, self-contained instructional system. These designers, are, in essence, seeking to do what might be considered the ultimate technical challenge in teaching; the creation of an instructional machine. This begs the question, "What is teaching?". I think it is clear that teaching must be considered as more than simply knowledge dispension. It is active intervention, to some end, in the lives of the learners. Without this active intervention, teaching would be the equivalent of reading or listening. Thus, although these are certainly part of the educational process and can be structured to promote self-directed learning, teaching is the act of helping the learner reach understanding. This implies that a third party to the learner acts on or with the learner. For the educational software designer, that third party is, of course, the computer system used by the learner; the computer is not a passive participant, it is an active player in the process of learner understanding. The rise of more affordable and more sophisticated computer technology, coupled with the advances in communications technology and data transferability, have meant a dramatic change in the way that computer assisted instructional design (CBID) is conceived. No longer confined to linear and hierarchical program structures, the recent focus of many educational software developers has been to begin looking beyond the "traditional" forms of instructional design to ones which include a "softer" and more contextually adaptable platform. In essence, the shift in focus has really been to examine pedagogy, not as a technical process of information delivery, but as a social process in which each learner and teacher bring unique and powerful perceptual and cognitive processes with them. This change has been primarily driven by the recognition that to simply provide information, as many CBID systems were designed to do, did not result in an adequate level of understanding for those using the system. Unfortunately, the level of available technology to a large extent dictated a rigid and well defined instructional protocol. There simply was little room within a technological platform to experiment with alternative forms of instructional designs. The technology has changed, of course, and with it less reliance on the sequential and tightly controlled environment provided by traditional CBID. The emergence of "smarter" machines, a more robust architecture, and new ways of looking at information storage and processing, have meant the emergence of a critique of "objectivist/behaviourist" methodologies and epistemologies. In their place, those which stress process, context and a philosophy of "psychological ecology" (Rieber, 1992) have emerged. Thus those designing instructional systems have begun seriously examining research in teaching, and providing some insights of their own. As an instructional third party, however, the computer cannot be considered autonomous in the same way that a teacher might be. What the educational system contains, and how it behaves, is a direct reflection of the software designer's own beliefs about knowledge and understanding. Thus, the reflective software designer is, like the teacher, faced with a series of epistemological and philosophical questions. (I would also argue that, to be ethically situated, she is also faced with serious social/political questions.) To be effective, the educational software designer must at some point make an effort to recognize her theoretical conceptions of education, and give "voice" to these through her instructional design. Some of the epistemological questions she needs to face are as those faced by any teacher: what does it mean to know something; what does it mean to learn something; when does the learner know he/she has learned something; when has a learner demonstrated having learned something; is there a "good" way to know something; finally, based on the previous questions, is there a "good" way to teach? Teaching in the Health Sciences Within the applied health sciences, particularly medicine, there has traditionally been a separation between the teaching that is perceived as critical "content" knowledge (eg.basic sciences) and clinical decision making. This is increasingly seen as problematic. For example, the Faculty of Medicine at U B C (among many) has decided to adopt a problem based learning curriculum as a pedagogical strategy to allow for a better integration between content knowledge and good clinical decision making. As such, the curriculum is a reflection of a move to an "anchored", contextualised and arguably more constructivist instruction/learning strategy. This kind of curriculum is hypothesized (and to some extent proven) to generate more meaningful knowledge representations than the abstracted and non-integrated approaches of traditional didactic teaching, as well as generating the kinds of "life-long learning" skills advocated by Candy (1991) among others. It seems clear that the new curriculum will demand substantial changes in the way in which a student participates in medical school; the student will be required to play a much more active and self-reliant role. Thus the student becomes responsible for: generating and following up on identified learning issues; tracking and evaluating domain-specific declarative and procedural knowledge; "charting" a course through the curriculum. My point is merely that in the face of a move to a "distributed learning" model, one in which there is a greater emphasis on student responsibility for learning and the expectation that students gain access to electronically delivered content, there needs to be an awareness of pedagogical strategies or models that are applicable to that medium. There must be assurances that these are theoretically well-grounded and feasible. As the reader will note, much of the literature covered within this paper, is based on cogntive science definitions of knowledge, and the consequent distinction between expert and novice. Before moving further into other literature, it is important to clarify, then, how researchers within cognitive science view the basis for knowledge, understanding and the instructional process. Capter 2: The Contribution of Cognitive Science Types of Knowledge: Declarative and Procedural Declarative According to cognitive psychology, knowledge can be differentiated by type: declarative ("factual") knowledge, and procedural ("how to") knowledge. The basic element of declarative knowledge is the proposition, the most primitive unit of meaning. These encapsulate - or "propose" -relationships between concepts. For example, "the man fixed the line" contains one proposition linking two concepts, the man and the line. Propositions always contain one relation affecting one or more arguments; in a proposition, relations correspond roughly to the verb, arguments to the nouns. Thus the previous proposition contains one relation - "fixed" - and two arguments - "man and line". Research indicates that what is preserved in the process of remembering the propositional elements making up declarative knowledge is not the precise semantic representation or ordering of propositions, but rather the meaning of the assertion. Propositions can be combined into propositional networks which in turn can be represented through linear ordering or as a graphical construction. In a graphical representation, each proposition can be represented by an ellipse, and each element of that proposition (argument(s) and relation) as labeled arrows. An example of a graphical representation of the proposition "the man fixed the line" is given in Figure 2. FIGURE 2: Proposition "the man fixed the line" 17 Propositions, and propositional networks, can subsumed by other propositional networks and thus form a hierarchical ordering. Thus the proposition "the man fixed the line" is made up of other, nested propositional networks which include "line" and "man", which in turn are likely (but not necessarily) made up of other nested and connected propositional networks and so on. This hierarchical ordering of propositions accounts very well for research data in which subjects were asked to judge the validity of associative statements. Hierarchical ordering predicts a positive relationship between latency time to judge the truthfulness of a proposition, and the degree of separation between the propositions in hierarchy. Thus, the proposition "birds have skin" takes longer to make a judgment about than the proposition that "birds have feathers", since "have skin" is a proposition associated with "animals" and not with birds specifically. The association hierarchy would predict the following path for "birds have skin": animals have skin - birds are animals - therefore birds have skin. This is an inferred relationship rather than the direct nature of the proposition that "birds have feathers". This kind of hierarchical propositional network, in spite of its predictive validity, seems to miss much of the (intuitively obvious) highly integrated and abstracted way that we use information. We don't seem to often use propositions at such a detailed level, and therefore the notion of "schema" - "a way of encoding what is generally true rather than what is true about a specific instance" (Anderson, pg. 134) - has arisen. Schemas allow us to make inferences about a situation very quickly, and in fact evidence suggests that what we perceive, as well as how we perceive, is a function of our currently invoked schema(s) (Anderson, pg. 135-136). Schemas, in their preservation of what is generally true, do have embedded variables: for example houses are generally made of brick, wood or stone. What falls outside of our general inference patterns (the associations that have shaped our schemas and as a consequence generate our expectations) may either generate some kind of cognitive dissonance (surprise, humour, shock - for example imagine you are confronted with a house made of stacked dimes and a roof of 100 dollar denomination bills), misinterpretation, or be missed entirely. Schemas are generally ordered hierarchically and, although they allow for variance, are structured into incidents of typicality. Certain members of a schema are likely to be considered more typical than others and occur most frequently, and this has given rise to the construct of "natural categories" (Anderson, pg. 137; Gagne et al., pg 83). Research has shown that we tend to organize our schemas along these naturally occurring instances (real world entities as well as those that are culturally generated). For example, if asked to rate the typicality of certain members of the schema "birds", chickens are less likely to be considered typical than robins; carrots are considered a more typical example of a vegetable than parsley; murder more typical of types of crimes than vagrancy (Anderson, pg. 137; Gagne et. al., pg. 84). What is extremely important for the evidence of schema formation is that we tend to abstract the most salient features of a natural category (weighted by typicality) into a "prototypicar example of that category. Thus a stimulus outline of a car that is the weighted average of all kinds of cars will tend to be more readily recognized as a car than the outline of a particular instance of a car (Gagne et. al., pg. 84-85, 155-156). This abstraction of salience into prototypical representations has been indicated in medical reasoning (Papa et. al., 1995). Another features of schemas includes the concept that how we classify an instance of an object (what schemas is invoked in response to a stimulus) is affected by the context in which that object is presented (Anderson, pg. 140), and the goals that we have set for ourselves (Gagne et. al., pg. 158). Schemas cannot only code for objects, but also for stereotypical event sequences. An example of this might be the act of entering and ordering food at a restaurant. There is generally a typical sequence to this event: enter the restaurant, wait to be seated, sit down at the table indicated to you, order drinks, examine the menu, order food, eat, pay the bill, etc.. Of course these are heavily culturally mitigated and equally obviously can lead to embarrassment or anxiety when one is placed in an atypical environment (humour often exploits this kind of scenario). To summarize, declarative knowledge is composed of (in ascending order of complexity) propositional elements, propositions, propositional networks, schemas and abstracted prototypical salience. Procedural Knowledge The following is an example from my own experience which may serve to help visualize procedural knowledge. My cousin, a craftsman in Holland, is engaging to watch regardless of the task that he's involved in. He is clearly in control of both the material and the tools he works with, and he works with a wide range of both. He is fluid and economical in movement, precise and exact in measurement and cutting, and seems to find just the right balance between functionality and artistry. Without this ability of course, it would be hard for his business to survive. I remember watching him forge new wrought iron doors for an old bake oven, something that you don't see everyday. He never seemed to perform a great deal of alteration once the basic form was crafted. Whatever the piece of hardware, it always fit after a few minor adjustments. Had I attempted the same thing I would have been endlessly fiddling during the measurement process (even if I could do the forge work!) and in spite of my efforts the results would have been indifferent at best, disastrous at worst, (and I like to think that I'm reasonably good with my hands). What then, separates his ability from mine? Gagne et. al., pg. 178, states that "When we say someone is an expert at performing a skill, we mean two things. First, the expert always seems to know how to get the job done. That is, the expert has a method or a sequence of steps that enables her to reach the goal. In addition, the expert always seems to know when to apply a particular method to achieve the maximum effect." The question remains, how did the expert become an expert? The next section will deal with procedural knowledge generally and then move to a consideration of expert/novice differences. Procedural knowledge is "how to" knowledge, or that knowledge which strategically enables one to perform a task, or series of tasks, in an efficient and (with practice) an increasing degree of expertise. Procedural knowledge is generally very domain-specific, although there is weak evidence to suggest that domain general knowledge can also be acquired (Gagne et. al., 1993). There are three stages to the acquisition of procedural knowledge, leading from novice to expert: these are the cognitive, the associative and the autonomous stages. In the same way that declarative knowledge is made up of hierarchically arranged systems of propositions and schemas, procedural knowledge is considered by cognitive psychologists to be made up of hierarchically arranged systems of "production sets", a series of purposeful IF-THEN structures. These structures or sets include not only the psychomotor elements of skillful behaviour, but also the strategic adoption of one problem solving process as opposed to several others. As one becomes increasingly "expert", these production sets become less the conscious application of remembered, or inductively discovered, rules and increasingly automated. This allows for competence in increasingly complex environments since the automaticity frees (and overcomes the limitations of) working memory. During the cognitive stage, the novice "the learner uses existing general purpose productions or weak methods to interpret declarative knowledge and to encode an initial declarative representation of conditions and actions" (Gagne et. al., pg. 179). This means the novice will invoke domain general strategies and what ever declarative knowledge he/she considers to be relevant to solve the problem and this places a great deal of load on working memory. Usually a great deal of concentration is required to complete each step of the problem solving solution and the adoption of a "think aloud" strategy to keep some kind of strategic process in place. During the associative stage, the novice begins to develop "bug-free" domain specific skills and strategies "that lose their conscious declarative character" (ibid, pg. 181). Several process are occurring to allow this proceduralization to take place. Compilation is the process of translating conscious algorithmic strategies into automatic sequences of behaviour; composition is the process of attaching several compiled productions sets together; proceduralization is the removal of the need to consciously attend to cues and thus the strategy invoked is an unconscious process of matching. The autonomous stage is a further refinement of the associative stage, in which automatic skills sets and strategic processes become faster, more tacit and increasingly responsive to subtle cues. It is the stage when schemas of the conditional IF-THEN production sets become more closely matched with specific response sets. The rate of improvement during this stage characteristically begins to level out and each improvement takes an increasingly longer exposure and execution cycle. But it is during this stage that someone is generally labeled as an expert (or at least competent practitioner) in a particular field. The Expert/Novice Distinction Research on the expert/novice distinction has suggested some characteristics that distinguish the problem solving style of the expert. Novices generally use a "means-end" or working backwards (from solution to process) strategy while experts work directly from problem to solution (Anderson, pg. 274). Experts also characterize problems by deep abstracted principles, not surface features. For example, this has been well shown in the ways in which novices and experts characterize physics problems. Given a series of physics problems, experts characterized these by their relationship to particular first principles laws, whereas novices characterized them very differently and, in general, by superficial descriptive features (ibid, pg. 277). Experts also consistently show significantly greater recall for particular characteristics of a problem. When confronted with a chess board under time limited conditions (5 seconds), and over 30 seconds later after an interfering task, experts could recall the position of a larger number of pieces and showed no loss in recall over the longer time frame. Novices usually exhibited a significant memory loss over the 30 second period. This difference is attributed to the experts' increased "library" of stored positional arrangements, and the consequent fact that memory and recall could be based on matching rather than raw encoding. Thus experts were able to "chunk" information into whole reproduction sets and the resulting encoding process is therefore the process of setting a pointer to that "chunked information". Short term memory was freed to concentrate on novel features of the particular problem. Novices had to utilize all of their short term memory to remember the location of each individual piece. Since short term memory is typically limited to 6 or 7 items, this represented the extent of their ability. Experts (paradoxically) seem to take longer to create a representation of the problem than do novices, are able to ignore perceptually salient but irrelevant features of a problem, and are better at self-monitoring the efficacy of solution choices (Gagne, pg. 216). It is also speculated that experts have a well developed "retrieval structure" in which domain-specific content can be organized and retrieved in a manner well beyond the capacity of non-experts (Anderson, pg. 284). This domain specificity is clearly a feature on expertise and little evidence exists to support the notion of transferability across domains (ibid, pg. 286; Gagne et. al., pg. 219). Both inductive and deductive reasoning depend on domain specific knowledge above all other factors (Gagne et. a l , pg. 232). Gagne et. al. argue that there are factors which can help knowledge transfer across domains. The first of these is actively targeting the development of conceptual understanding and representation before beginning to learn domain specific skills. This allows for a different (and better) knowledge organizational structure in which to embed the specific skills, and Experts also consistently show significantly greater recall for particular characteristics of a problem. When confronted with a chess board under time limited conditions (5 seconds), and over 30 seconds later after an interfering task, experts could recall the position of a larger number of pieces and showed no loss in recall over the longer time frame. Novices usually exhibited a significant memory loss over the 30 second period. This difference is attributed to the experts' increased "library" of stored positional arrangements, and the consequent fact that memory and recall could be based on matching rather than raw encoding. Thus experts were able to "chunk" information into whole reproduction sets and the resulting encoding process is therefore the process of setting a pointer to that "chunked information". Short term memory was freed to concentrate on novel features of the particular problem. Novices had to utilize all of their short term memory to remember the location of each individual piece. Since short term memory is typically limited to 6 or 7 items, this represented the extent of their ability. Experts (paradoxically) seem to take longer to create a representation of the problem than do novices, are able to ignore perceptually salient but irrelevant features of a problem, and are better at self-monitoring the efficacy of solution choices (Gagne, pg. 216). It is also speculated that experts have a well developed "retrieval structure" in which domain-specific content can be organized and retrieved in a manner well beyond the capacity of non-experts (Anderson, pg. 284). This domain specificity is clearly a feature on expertise and little evidence exists to support the notion of transferability across domains (ibid, pg. 286; Gagne et. a l , pg. 219). Both inductive and deductive reasoning depend on domain specific knowledge above all other factors (Gagne et. al., pg. 232). Gagne et. al. argue that there are factors which can help knowledge transfer across domains. The first of these is actively targeting the development of conceptual understanding and representation before beginning to learn domain specific skills. This allows for a different (and better) knowledge organizational structure in which to embed the specific skills, and can presumably be utilized across a wider range of domains. This strategy seems consistent with Pratt's (in press) developmental perspective as described by Arseneau and Rodenburg. Although the general rule of transfer - the closer the production sets for each domain are the more transfer is likely to occur - targeting the creation of mental/conceptual models can aid in lateral transfer (ibid, pg. 247). In examining vertical transfer (from lower level to higher level skills) Gagne et. al. suggest that traditional skill hierarchies identified by task analysis still serve a useful, if reductionist, approach to skill acquisition. A third factor in transfer is active and conscious evaluation of strategy effectiveness. This requires good metacogntion skills which can, to some degree, be successfully taught (Gagne et. al., pg. 255), and seems to (at least partially) correspond to Schon's (1987) conception of the "reflective practitioner". Factors impinging on transfer include "set effects" which is an inability to utilize novel strategies in problem solving. Much of this is perceptual fixedness in which objects or strategies have strong associations with a single context and thus cannot be perceived to have other functional uses (Anderson, pg. 248-249). Many "brain-teaser" puzzles rely on these associations to discourage the puzzler's ability to solve the problem. Summary Procedural knowledge, embodying both psychomotor and strategic cognitive skills, is made up of production sets and which in turn can be organized into schemas. Both the production sets and schemas can be hierarchically organized. Procedural knowledge acquisition is marked by three stages of increasing expertise: cognitive, associative, and proceduralization. Although highly domain specific, strategies such as the development of conceptual models and active reflection may help in the transfer of procedural knowledge to other domains. Royer et. al. (1993) provide a good overview of cognitive skill development in their summary of the dimensions for skill assessment proposed by Glasser et. al. (1985). The first dimension is knowledge organization and structure, the change from disconnected and isolated "pieces" of declarative knowledge into integrated and structured wholes. Depth of problem representation reflects the ability to describe a problem based on abstracted principles, not superficial/descriptive characteristics. Quality of mental models describes the sophistication and truthfulness of the conceptual models carried by the student. Efficiency of procedures is the ability to strategically implement a solution that eliminates unproductive pathways. Automaticity of performance reflects the skilled performers ability to unconsciously carry out the processes needed for solution, thus freeing working memory to concentrate on information crucial to strategic planning. Metacognition is, of course, the ability to actively monitor progress, implement changes in strategy and reflect on deeper principles. The development of each of these dimensions can serve as a benchmark for a measure of knowledge development. Having examined some of the basic constructs within cogntitive science, let us now examine what impact it has as a theoretical framework for both instruction and assessment. Chapter 3: The Instructional Implications of Cognitive Science Gagne et. al. (1993) offer some specific instructional strategies to support the development of good cognitive skills both within and across specific domains. Beginning with schema formation, the simultaneous or nearly simultaneous presentation of at least two examples will help students develop schemas that are formed on the basis of true variables. This is an important principle since a balance must be struck between the need to develop prototypical salience, and the need to keep that salience focused on the relevant points of variance and invariance. If examples are chosen which do not vary across irrelevant features then the prototypical salience will include those features as definitional. Thus that schema will include falsely representational data which, once established, can be very difficult to revoke (examples of this might be that all homosexuals are men or that all schizophrenics are paranoid). Refining schemas (which may begin by being overly general) can be furthered by the presentation of an example and non-example of schema data. If the positive exemplars are varied across relevant data, and the negative exemplars are varied across irrelevant data, a schema with the correct prototypical salience should be more likely. Matched non-examplars in which the non-exemplar differs from the positive exemplar in only one feature can help this process. (This kind of strategy might be very effective when learning visual pattern matching tasks in medicine such as radiology, dermatology and ophthalmology). Gagne et. al. and Ohlsson (1993) also suggest that a "discrepant event" or the unexpected result of the application of a schema can motivate the student to find out why the schema failed. For example, students could be asked to predict the results of an experiment or test, run the test and observe the result (White and Gunstone's (1992) POE paradigm - prediction, observation, explanation). If the result is not in accordance with what their schemas had predicted (counter-intuitive), this results in a discrepant event which usually motivates the student to find out why their schemas failed. Deliberately creating these discrepant events may foster the student's active participation in learning, providing a balance is struck between the likelihood of frustration and the student's intrinsic need to search for understanding. Mayes (1993) and Glasserfield and Steffe (1990) label this intrinsic need as the learners' search for principles underpinning understanding as opposed to mastery over a particular problem type. To foster the development of procedural knowledge, Gagne et. al. suggest that pre-requisite skill mastery is an important factor. The difficulty is creating an environment in which skill mastery can be practiced without boredom and rapid cognitive fatigue. Many researchers (for example the EGEMS group at UBC) have suggested that games in which the skills become increasingly important for better performance may offer more motivating environments. The ideal would be the creation of an environment which supports Lebow et. al.'s (1995) conception of "stolen knowledge" in which the skill development is in a sense "hidden" from the user. This is an enormously difficult design challenge. Skill acquisition is fostered by practice and feedback in which two factors seem to play an important role, spacing and the immediacy of feedback. Spaced practice seems to show greater learning outcomes than massed practice. The immediacy of feedback is important to avoid the introduction of "bugs" into the skills subset being learned. Gagne et. al. (1993) further suggest that practice example should, just as in procedural knowledge, be varied across irrelevant attributes and should be appropriately contextually situated. Students must learn when to make use of a skill set, as well as how to make use of that skill set, and thus the relations among the skills must be made explicit. Not all conscious declarative cues triggering the invocation of a skill set should be eliminated or proceduralized, since the consequence may be the development of detrimental set effects. Once complete proceduralized, skills become difficult, if not impervious, to change. Thus instruction should foster proceduralization only in cases in which circumstances are unlikely to change and/or in which speed is the critical factor. Students should also be encouraged to actively reflect on their learning, the strategies adopted, the efficacy of those strategies and the contextual cues that prompt the invocation of those strategies. These latter skills are essentially those of metacogntion. The ability to self-reflect during the learning process is a "metacognitive" skill (Novak, 1990), one which can be both discomfiting and difficult for students to grasp. We usually assume that the structure of the curriculum will serve to both teach and assess our progress, so we are generally passive players in evaluating learning outcomes. Since the active monitoring of one's own performance is not a standard part of institutional learning, students may well be wary of a process in which they are expected to observe their own learning. As well, self-reflection of the learning process involves factors that are less concrete than more standardized assessment practices, and students may respond with anxiety when asked to judge their progress by criteria that are unfamiliar and perhaps obscure. Metacognition, like any skill, takes both practice and time to assimilate. Well designed instructional strategies should encourage initial self-reflection in ways that challenge the student to think about contextually relevant responses, i.e. those responses that are meaningful within the current learning environment. If they are presented in a manner that is overly abstracted or littered with jargon then the student will either panic or cease to pay attention to them. The consequence is that the student will actually begin to be conditioned to avoid metacognitive practices in the future. My point is that metacognition, as a skill, needs the same support and instructional strategies that target specific content. Gagne et. al. cite evidence that indicates that in the absence of pre-requisite skill master, higher order skills (or vertical transfer) is inhibited. Ohlsson (1993) points out that goal-hierarchies are a natural consequence of cognitive theory. Although Gagne et. al. also suggest that skills mastery should be accompanied by a focus on developing good global conceptual understanding - so that the reasons for the skill development are clear and meaningful to the student - this echoes the reductionist approach that seems antithetical to more contsructivist epistemological positions. If instruction is simply a matter of the composition of sub-component skills sets into larger and larger procedural schemas, then as both Winn (1990) and Mayes (1993) point out, instructional design theory becomes the application of an instructional process to content. The emergence of the constructivist and situated learning positions, however, in part based on cognitive theory, has largely arisen out of a profound dissatisfaction with reductionist approaches to instructional design. The resolution to this apparent paradox lies in the recognition that a balance must be struck between the need for well conceived and contextualized practice sessions (analogous to the scaffolding and fading that occur during apprenticeships), and the need to place skills acquisition into a larger conceptual framework in which reflective, metacognitive and constructive processes are encouraged. Winn (1990) refers to this kind of framework as a "theory driven" instructional infrastructure, one that is not driven only by the mastery of certain design skills. He defines the impact of cognitive theory on instructional design over several categories: in terms of task analysis, the focus should be on characterizing the schemas and procedures that enable correct performance to occur, not on decontextualized individual component skills. The distinction between these is hard to grasp, but the essence lies in ensuring that the learner begins with, or develops over time, a well developed conceptual framework into which individual skills can be placed. Echoing this first distinction, Winn suggests that a cognitive analysis of learning objectives would tend to focus on the development of accurate schematic representations of what is to be learned. Again, the distinction is difficult to see, but lies in the understanding that from a cognitive standpoint, the stipulation for exact stimulus-response sequences and criterion referenced performance measures - although appropriate in some lower level skills acquisition stages - is less important than a focus on generating good schematic representation. Therefore, for many learners, skill acquisition may, will in fact, not be the linear sequence that the instructional designer has determined from a task analysis focused on component skills. From a cognitive position, these differences in learner characteristics is the critical distinction between cognitive and behavioural approaches to instructional design. Cognitive approaches recognize that learners bring prior knowledge, or schemas, with them, and these schemas will determine what is attended to, what is actively engaged in, and of course ultimately what is learned (prior knowledge is also a central component of the Developmental Perspective - see Arseneau and Rodenburg in Pratt, in press). Thus learning is a necessarily "messy" process, in part due to idiosyncratic nature of prior knowledge, but also due to the unpredictability of the nature of the process of meaning making. Winn also acknowledges the importance of a situated context and the recognition of the value of "plausible reasoning", the reasoning already described as that of "just plain folks". Echoing this, von Glasserfeld and Steffe (1991) refer to the distinction between a behavioural and cognitive approach to teaching as the difference between training and understanding. The former stresses production, the latter process. Good instructional methodology begins by assuming the existence of student conceptual models, and actively soliciting both the articulation and the use of these as accounts of real world phenomena. Learning takes place when a difference between what is predicted by a model, and what actually happens, occurs. This failure for the model to account for real world events leads to an internal perturbation (cognitive dissonance) and the motivation to change the conceptual model to accommodate the new experience, von Glasserfeld and Steffe go on to suggest that this process is both ad hoc and unpredictable and thus needs experienced intuition to make the most of each teaching moment. A more reductionist or systems approach is offered by Taylor (1994) who suggests a novice/expert or "Novex" analysis of the instructional domain as a means of incorporating cognitive science into instructional design. The analysis is based on the idea that instructional design should seek to understand the cognitive structures that characterize expert functioning in a specific domain and help move novices to develop similar structures. His suggestion, which is excellent, is that instructional design must work from an explicit model of memory, and work to provide instruction within the limits that the model asserts. This is the recognition that novices must devote more working memory resources to task analysis and response generation and are therefore more susceptible to "cognitive load" than the expert. The consequences are that experts have integrated heuristic strategies which allow both rapid encoding and retrieval of new information which the novice lacks; experts are able to attend to a wide range of novel conditions, and yet are still able to plan strategically while novices must adopt a "trial and error" solution paradigm. Thus the stratagem of "less is more" to the novice in a particular domain seems appropriate (Arseneau and Rodenburg, in press). The instructional strategies that Taylor offers are not overly different in substance from those offered above, but two are worth repeating. These are that an analysis should be made of the "extant" knowledge base that students typically carry with them into the instructional setting (typical conceptual models) and that this extant knowledge base must be actively considered during instructional planning; secondly, that instructional strategies must utilize a variety of means to achieve the kind of integrated knowledge base that the analysis of the experts' functioning shows. Although this section has offered some concrete examples of strategies that provide for instructional strategies that are consistent with the theoretical constructs of cognitive science, there is still a great deal in the section that is representative of the fact that instruction cannot, and should not, be prescriptive. Instructional strategies are paradigms, and should be seen as heuristic strategies, not as formulas applicable across all instructional domains (Collins et. a l , 1991). The empirical support for the constructs of cognitive theory seems persuasive, but there is clearly a great deal of dissatisfaction with the underlying assumption that learning is process driven. It can be argued that there is too much in the practice of instruction that is driven by contextual factors and the rise of a constructivist/situated learning perspective is a reflection of that dissatisfaction. A series of a priori assumptions and models do not necessarily make for good instruction, and the instructional designer must be careful before adopting a single protocol as an instructional panacea. I also argue that the instructional designer must, outside of considerations of efficacy, reflect on the normative value of each instructional strategy based on local contextual factors and on larger philosophical and epistemic principles. Summary There are some clear implications of the above discussion of cognitive science for instructional design. For declarative knowledge these can be stated as supporting the effective development of declarative propositional networks, schemas and refinement, ensuring that the hierarchical ordering of these networks and schemas are elaborated, and finally assisting in the formation of good prototypical salience without invoking rigidity. In terms of procedural knowledge, instruction should focus on developing broad and well conceived conceptual models, a contextualized representation of problems, support for the automaticity of basic skills, the promotion of composition (small sub-skills into larger skills), and the promotion of proceduralization (practice, practice, practice). Other implications for instructional design, include a focus on the adoption of high-level integrative strategies during component practice, a constant monitoring of student conceptual models, a recognition that unpredictable consequences may result from student/instruction interaction, the support for metacogntive processes, and the possibility of student's changing instructional strategies based on those metacogntive assessments. Cognitive Science and Assessment A l l assessment aimed at evaluating cognitive processes, must by necessity, be a process of inference; those inferences can be better or worse but all are subject to the accusation that they do not, in fact, evaluate what they set out to evaluate (Ramsden, 1988). Hence the debate over the value of multiple choice questions (MCQ's) (Elstien, 1993): they are usually used as a vehicle for evaluating the degree to which declarative knowledge content has been assimilated and are necessarily decontextualized frameworks. Unless they are well constructed, they allow for some recognition recall and generally do not target the elucidation of conceptual models or deeper meaning constructs (ibid). One could argue that this is in fact an artifact of the design process and not dictated by the format itself. But even if one grants that that is in fact true, it seems self-evident that procedural knowledge (performance measures) would be impossible to assess solely within a M C Q format. If cognitive science has specific implications for instructional design, then it must also have some implications for the assessment of knowledge. As stated earlier, Royer et. al. (1993) cite six dimensions (developed by Glasser et. al., 1985) in which performance at each of the stages of knowledge acquisition are targeted in a superb review article of the relationship between cognitive science and assessment techniques. Royer et. al.'s analysis is important because is a systematic effort to associate specific constructs within cognitive science with assessment techniques appropriate for that construct. The ultimate utility of such assessment measures can, in the absence of convincing supportive data, be criticized as lacking validity and reliability (do they measure what they are supposed to and do they measure that construct the same each time). But the recognition must be made that most educational processes have humble beginnings and the techniques cited in Royer et. al.'s analysis at least broadens the discourse around assessment and may plant the seeds of constructive change. Royer et. al. classify attempts to assess cognitive constructs across each of these dimensions: knowledge organization and structure, depth of problem representation, quality of mental models, efficiency of procedures, automaticity of performance, and metacogniton; these assessments they have been further subcategorized within each construct. For example, they distinguish between direct and indirect assessments of knowledge organization. One example of a direct procedure in which knowledge organization and structure - outside of criterion referenced testing - could be measured, is by constructing a situated environment (in the case of firefighters) and asking students to evaluate several given possible responses. Students repeatedly chose responses until the best response (in the eyes of the test maker) was uncovered. Another example had students list the characteristics of various electronic items under physical, functional, operational, and applicability headings. A second assessment with this group asked students to arrange components of a radar system into a "reflection" of a functioning machine. The latter strategy was also adopted by Stevens (1989) using computer-based problem-solving examinations in immunology. Students' search paths for information relevant to the problem were tracked and graphically represented; these visual representations were then analyzed to determine if they generated patterns that would be reliable indicators of conceptual models, knowledge integration and critical concepts. Although the possibility of graphically representing problem solving behaviour is a tantalizing one, since it provides a highly efficient representation of data that would otherwise be difficult to conceptualize, the assessment value of his approach is difficult to ascertain. No real attempt seems to have been made to systematically link student cognition (as measured across a variety of scales) and search behaviour. There is some suggestion, however, that unsuccessful problem solvers in two separate groups demonstrated similar search patterns. Indirect measures of knowledge organization are based on performance measures which are postulated to vary as function of knowledge integration, thus the scope of the inference is assumed to be broader than those techniques targeting knowledge directly. Examples include document/information search efficiency when confronted with a problem dependent on access to constantly updated information, SVT (sentence verification tests) which assess a student's ability to extract essential meaning from text passage; a related test, the inferential identification test (IVT) which assesses the student's ability to correctly infer implicit content within a text passage. Both SVT and IVT have some empirical support, both as predictive measures on achievement and as indices of domain knowledge (Royer et. al., 1993). Royer et. al. make the essential point that a critical determinant of the value of assessment measures, especially those that are based on cognitive theory, is based on their utilization as both relative and absolute benchmarks. Although one could develop tests that have psychometric validity, an equally important function is assessment across a time period against a student's own initial benchmarks. Changes in cognition (integration, model constructs, procedural skills) can then be clearly followed and serve not only as a criterion or mastery referenced tool, but as a diagnostic window into the student's cognitive development. Secondly, this window can not only serve the instructor, but also as a metacognitive tool for the student, promoting self-awareness of his/her learning process. Royer et. al. make a valuable distinction between assessments that have task authenticity and those that have process authenticity. The latter are those assessments that target cognitive skills that are critical components of the authentic task skill. These assessments, if well targeted, can play a vital role in the diagnosis of conceptual difficulties facing the student in a domain. Table 2 provides a summary of the assessment techniques used for each cognitive dimension based on Royer et. al.'s review. Moving Forward As will be evidenced later in the thesis, cognitive science is clearly the dominant theoretical framework within much the literature within both instructional design and medical reasoning. But there is a second series theoretical perspectives also informing both that has emerged which, while clearly adopting of some the fundamental tenants of cognitive science (the importance of context, the idiosyncratic nature of understanding), differs in its focus on reasoning as primarily mitigated by social processes. The first of these, and perhaps the theory that subsumes the others within this series, is constructivism. This framework has become the dominant framework within science education and, to some extent, educational technology. It has also had a strong influence on instruction and to some limited degree, medical education. From this staring point, the paper will then examine situated cognition and the impact these frameworks have had on instructional design, assessment and educational technology. TABLE 1: Cognitive Dimension and Assessment Tools Cognitive Dimension Assessment Tools Knowledge Organization Direct Multidimensional Scaling Concept Recall and Expert Comparison Concept Classification Response Evaluation Indirect Search Path Analysis Sentence Verification Test (SVT) Inference Identification Test (HT) Depth of Problem Representation Problem Recall after brief exposure Sorting Problems by Solution Type Judging Problem Examples Against A Criterion Problem Across Deep, Superficial Characteristics Judging Physical Action Across Standard Domain Characteristics Problem Framing Evaluation Mental Models Relational Models (Device Feature Relationships) Process Outcome Prediction under Varying Conditions Qualitative Process Models (Mental device simulation) Outcome Predictions Trouble Shooting Procedures Real-time Production Set Evaluation Assessing Appearance Models (Device Appearance Under Varying Conditions) Medical Diagnosis Based on Symptomology Metacognition Reading Comprehension Text Faulting (Deliberate Insertion of Non-congruent Passage) Planning Skills Problem Presentation, Latency of Action, Efficiency of Solution Execution Plan/Solution Plan Delimiting Effective Solution from Range of Possible Solutions Solution Strategy Means-end versus Forward Reasoning Automaticity Cognitive Load Assessment Simultaneous Presentation of Two Tasks Encapsulated Processes Stimulus-response Latency Second Task Initiation Latency 37 Procedure Procedure Sequence Sorting Efficiency Trouble Shooting Harkback Frequency 38 Chapter 4: Constructivism The rise of more affordable and more sophisticated computer technology, coupled with the advances in communications technology and data transferability, have meant a dramatic change in the way that computer assisted instructional design (CBID) is conceived. No longer confined to linear and hierarchical program structures, the recent focus of many educational software developers has been to begin looking beyond the "traditional" forms of instructional design to ones which include a "softer" and more contextually adaptable platform. In essence, the shift in focus has really been to examine pedagogy, not as a technical process of information delivery, but as a social process in which each learner and teacher bring unique and powerful perceptual and cognitive processes with them. This change has been primarily driven by the recognition that to simply provide information, as many CBID systems were designed to do, did not result in an adequate level of understanding for those using the system. Unfortunately, the level of available technology to a large extent dictated a rigid and well defined instructional protocol. There simply was little room within a technological platform to experiment with alternative forms of instructional designs. Just getting a system to consistently deliver its curriculum in one way was a technological accomplishment. The technology has changed, of course, and with it less reliance on the sequential and tightly controlled environment provided by traditional CBID. The emergence of "smarter" machines, a more robust architecture, and new ways of looking at information storage and processing, have meant the emergence of a critique of "objectivist/behaviourist" methodologies and epistomologies. In their place, those which stress process, context and a philosophy of "psychological ecology" (Rieber, 1992) have emerged. Thus those designing instructional systems have begun seriously examining research in teaching, and providing some insights of their own. Underlying the challenge to "traditional" instructional design is the central idea that instructional design must explicitly address the student's personal and idiosyncratic process of making meaningful representations of what is being taught. Constructivism, the formal theoretical stance that all knowledge is result of active processes of integration and elaboration on the part of the learner, forms the fundamental "core" of this perspective. Although is has now been widely embraced by the educational community, its development has to some extent been driven by those within CBID, as well as those within science education. Constructivism is not a monolithic theory and its proponents have many different visions of what constitutes constructivist practice. Some common principles can be identified however, and the following section describes some of these. Fundamental to a constructivist's view of learning is the premise that knowledge is not passively assimilated through observation or instruction. Rather, learning is an active process "in which meaning is developed on the basis of experience" (Merrill, 1991, pg 46). Meaning is "imposed by on the world" by the learner (Duffy and Johanssen, 1991, pg 8). In other words, the learner is an active agent in the "representational design" of knowledge, constantly caught in the dynamic flux of "equilibration": a "need for an organised and ordered world while constantly confronted by the need to adapt to an ever-changing environment" (Reiber, 1991, pg 94). The learner has constructed or created internal representations of the world based on his/her past experiences and interactions (Cunningham, 1991, pg 13; Merril, 1991, pg 47). Knowledge, outside of this process of creating constructs, makes no sense to the learner, and may be viewed as unlearnable (ibid, pg 13). Constructivists do acknowledge the existence of an external reality. Contrary to objectivists, however, constructivists do not believe that knowledge can be separated from the context in which it is embedded, which is essentially the social/cultural milieu of the community of the learner (Cognition and Technology Group at Vanderbilt (CTGV), 1991). The learner, rather, forms a series of working models of the world in which existing knowledge is reflected, and into which new knowledge is incorporated (Duffy and Johanssen, 1991, pg 9). The process of knowledge construction can be viewed as the formation of learner metaphors or "schemas" (Cunningham, 1991, pg 13). These metaphors/schemas are unique to each learner, "indexed" by the learner's experiences, and thus are never fully "shared" with another learner; knowledge is actively constructed in an individual manner and there is never an objective reality that is imparted to the learner (Duffy and Johanssen, 1991, pg 9; Merrill, 1991, pg 46). Constructivists recognise that their are "sufficient degrees of freedom in the structure of physical and epistemological worlds to allow people to construct their own personal theories of their environments, of what is 'known' or believed by others about those environments, and of themselves" (CTGV, 1991). To provide an example, a paper that exemplifies the "messiness" of conceptual understanding is Hiebert, Wearne and Taber's (1991) analysis of fourth graders' experience with decimal fractions. The instructional methodology described in the paper is arguably constructivist in its orientation, since it explicitly acknowledges the instructional value of multiple representations of knowledge and student dialogue and interaction with the materials provided, as well as the idiosyncratic nature of understanding. Their conclusion is that, for many students, understanding is marked by gradual and unpredictable changes that "make the process appear halting and erratic". Constructivists stress the reciprocal (recursive) nature of the meaning of knowledge and the context in which that meaning resides (ibid, pg 8). Therefore, internal representations shape perception of content which is derived from context which in turn shapes context to further shape content and so on. The environment in which an idea is placed is part of the meaning of that idea (ibid, pg 8). This is also true of tasks (Coles, 1991, pg 32). Although given in the context of sculpting, the quote Coles uses could be applied to virtually all processes: a plan is refined and altered by the process of carrying out the plan. How do constructivists avoid the paradox that although meaning is primarily culturally/socially mitigated, members of that culture can never share a complete construct of meaning? Constructivists speak of knowledge as a "dialectic" process of "negotiated meaning" (CTGV, 1991, pg 16). "By continually negotiating the meaning of observations, data, hypotheses, and so forth, groups of individuals construct systems that are largely consistent with one another" (ibid). This process permits individuals to come to an "agreement" about meaning without ever sharing identical constructs (Duffy and Johanssen, 1991, pg 9). Coles (1992, pg 29) offers a "conceptual continuum" in which some knowledge has clearly undergone extensive social negotiation of meaning, while other knowledge is "still very much being negotiated. Thus individual constructs of knowledge, by virtue of that knowledge's position on a negotiated continuum, would reflect a greater or lesser degree of common understanding in some absolute sense. To summarize the constructivist view of epistemology: constructivists view the world as "real" in a non-trivial way. The process of attaching meaning to information is an active one in which the participant constructs an internal representation of that information based on its social/cultural context, and his/her existing internal meaning constructs. This process, learning, can be conceived of as the learner's imposition of order on this new information. The learner's existing internal constructs are engaged in a reciprocal relationship with new information: internal constructs affect what information is perceived and how it is perceived, which changes the constructs, which changes the perception and so on. No two individuals ever generate the "same" internal representation because of differences in past experience and therefore differences in the development of previous constructs. Ultimately, meaning must be socially negotiated until some common understanding is developed which is useful and agreed to. From a constructivist platform, "learning is not solely the acquisition of new information or ideas from specific 'provinces of meaning'; for learning to occur, some qualitative change must take place in the learner's understanding... Changing conceptions demands that the learner makes connections between his\her present way of thinking and the newly presented knowledge, confronts any discrepancies, and comes to realise the personal value of the new way of thinking" (Weich, Waldie, Wong and Arseneau, 1993). Constructivism is also concerned with "negotiated" meanings of reality, and that participants within this negotiation construct reality many different ways. (This description could be likened perhaps to a weak version of the transformational perspective.) According to the constructivist perspective, meaning is constructed in relationship to two factors: previous experience, and the context in which the information is embedded (Cunningham, 1991, pg 14). This implies that what information - and how that information - is perceived, is affected and constrained by previous constructs. What seems implied by her discussion is that a constructivist approach to developing good practice within a discipline is one of context as well as content. Decision making within the "grey" areas of practice demands an "artistry" that content instruction alone cannot develop. It needs a process of "cognitive apprenticeship" which provides the learner with the mentoring needed for good skill development, and the heuristics needed to guide the practitioner well in unclear situations. Thus it follows that those engaged in research develop good research skills by doing research. The value of the kind of research protocol used, however, is not addressed. As a consequence of the constructivist view that meaning is individually constructed, yet socially negotiated to some unclear point of commonality, two methodologies have arisen as the backbone of constructivist pedagogy. The first is the recognition of the value of collaborative work to provide multiple perspectives from which meaning is then socially negotiated (Cunningham, 1991, pg 14). "Conceptual growth comes from the sharing of multiple perspectives and the simultaneous changing of our internal representations in response to those perspectives...". The end point is "self chosen positions to which they [the learners] can commit themselves" (Merrill, 1991, pg 46). The second method is the provision of an "anchored" or "situated" instructional tasks in realistic settings which closely approximate the setting in which the learning will be used (Merrill, 1991; Cunningham, 1991; Duffy and Johanssen, 1991; Weich et al, 1993). Meaning constructs, by implicit definition, make no sense out of the context in which they are formed, and are therefore of not much value. The constructivist would insist that in a realistic, collaborative setting, a negotiated "best" individual construct would be the result. Also, if the learner is perceived as active in the learning process, the more activity is demanded of the learner, the more complete are the constructs he/she forms. If these tasks are of sufficient complexity, all members of a group can contribute in a meaningful way (CTGV, 1991, pg 17). There is some disagreement within constructivism concerning the degree to which a task should be situated (Merrill, 1991, pg 47). The "extreme" constructivist view is that the "authenticity" of any task is the degree to which it reflects individual construction without prescribed knowledge (ibid; Coles, 1992, pg 29). Proffering any form of "pre-fabricated" information robs the learner of developing "good" constructs. Less extreme versions of constructivism allow for some knowledge presentation as legitimate, with the emphasis remaining on showing the learner how to "best" fit this information into a personal construct (Cunningham, 1991, pg 14). In extreme and less extreme positions, constructivists still advocate a process of inductive learning based on discovery. The degree to which discovery is left in the hands of the learner is the subject of some debate. Most would endorse a compromise position of part instruction coupled with more or less constrained choice (Coles, 1991, pg 29). Some have labelled this process "guided discovery" (Rieber, 1991, pg 96). Constructivists view the objectives of practice as helping learners "1) in discovering their conceptions and 2) to help learners in modifying those conceptions to more closely fit the socially negotiated and accepted conception" (Weich et a l , 1993). This is supported by the C T G V (1991) who state: "one of our major goals in instruction is to encourage students to develop socially acceptable systems for exploring their ideas and the differences in opinion". Constructivists clearly wish to instruct students in constructing "good" thinking skills, which are critical and derived from multiple perspectives. Thus debate and critical reflection is (ideally) a part of constructivist pedagogy (Coles, 1992). Anchored tasks and cognitive apprenticeship are ways of modelling "good" thinking skills within a particular content domain. Constructivism is student-centred in the sense that the recognized outcome of instruction is the construct of meaning which is unique to the student. The emphasis is on providing an environment in which the student makes the choices to pursue concepts when it makes sense to do so. Thus, extreme versions of constructivism would advocate a "goal-free" learning environment which is inductive and exploratory (although this is currently under debate (Coles, 1992)). The goals of constructivism seem to be the provision of an authentic learning environment in which "good" thinking skills for a specific content domain are modelled through cognitive apprenticeship and multiple perspectives. The student, in negotiating a unique meaning for him/herself within the constructivist paradigm, develops the "artistry" necessary to conduct him/herself in a manner consistent with the accepted "ideal" practitioner in that area. The theoretical (and still to be empirically established) point of greatest efficacy between student and goal-directed learning is still a matter of some debate. The formal theoretical stance that learning is primarily contextually bound is known as "situated cogntion" and although it shares, or is subsumed by, constructivist principles, it offers some specific insights which should be explored in detail. This is discussed in the next section. 46 Chapter 5 : Situated Cognition There is evidence to suggest that what is learned in an institutional context cannot readily be transferred to the "everyday" world of practice, and that what is considered knowledge by an educational institution does not reflect more than a rote or "compartmentalized" understanding of that knowledge (Biggs, 1989; Ramsdan, 1988; Masters, 1989). This is the problem of "inert" knowledge (Lebow, 1994). In the literature focused on medical education, a number of recurring themes are easily identifiable including "surface verses deep" understanding, and the inability of students to integrate disparate areas of understanding into a cohesive and elaborated whole. Thus their experiences within a clinical session are commonly felt to bear little relationship to the basic science courses they were mandated to take during the first two years of the curriculum. Balla (1989) refers to this lack of integration as the "split between the clinical and pre-clinical" in which the student has an "atomistic" understanding of the processes underlying clinical symptomology. The clinical world is necessarily uncertain and ambiguous and requires the ability to sift and weigh not only clinical cues, but conflicting and competing demands demanded by the social realities of the practice context. Balla suggests that this lack of integration is only exacerbated by clinical teachers who are often unable themselves to articulate the underlying processes. Biggs (1989) echoes this surface/deep distinction and describes a knowledge taxonomy which moves from the tacit to the theoretical. He, and others (Ramsden, 1988; Arseneau and Rodenburg, in press), argue that surface learning is generated by assessment strategies, such as performance rewards, which foster extrinsic motivation. Describing a contextual model for clinical teaching, Coles (1989) describes the experience of Southampton medical students who (spontaneously) adopted an "elaborated" learning strategy as a consequence of the ordering of the curriculum: the first set of major exams comes after the first clinical experience. Elaborated learning is, according to Coles, the process of "relating together" information which was previously disparate. Coles cites evidence indicating that those adopting the elaborated strategy did better on the exams than those who did not, with little correlation to previous performance or previous learning "style". This suggests that in the absence of a concrete context, even those students who may have "deeply" processed curriculum content will learn that information in a way which does not support actual practice. Thus Coles maintains that elaborated knowledge demands a contextual model, one in which the context is established prior to the "to-be-learnt" information. Other knowledge can be incorporated if it is explicitly related to the context already established. In this way, students perceive there task as one of relating information to the concrete example being given. In this way students are driven intrinsically to elaborating their knowledge by the rewards of unexpected insights. Although he does not explicitly state this, Coless model shares much with a theoretical stance on learning gaining a great deal of interest within both the educational mainstream and within educational technology: situated cognition. Situated cognition or learning is the theoretical stance that recognizes the "inextricability of thinking and the context in which it occurs" (Choi and Hannifin, 1995). It shares with (or is perhaps subsumed by) the constructivist's central stance that learning is meaning-making. The process of meaning making - the legitimate representation of knowledge - is the result of a unique combination of an interaction between the learner and the environment (Young, 1995; Johnson and Pratt, in press). It is the result of social processes of negotiated meaning and collaborative efforts to solve problems, and must, therefore, be a reflection of the political, economic and historical realities of the learner's experience (McLellan, 1994; Damarin, 1994; Streibel, 1993). Jonassen et al. (1995) describes the situated learning as "meaning constructed by the learner... that is indexed by the experience surrounding the learning", which assigns meaning to what is learned". The process of solving problems in a real world context leads to richer understanding. Real world contexts often offer extremely powerful cues for the acquisition of skills. Brown and Duduid (1993) provide the example of driving a car as a highly complex behavior which is a primarily social practice. There are strong implicit cues that frame the skill as an attainable outcome for all members of society and for which society provides a great deal of support. This context can be contrasted with V C R recording operation as a highly individual skill, and therefore with less implicit cues concerning successful outcomes. Consequently, the acquisition of the skills needed to program and record successfully are less likely to be attained by the general population. The achievement of competence or understanding is accomplished through a process including modeling, scaffolding, fading, articulating, reflection and exploration (Collins, et. al., 1991; McLellan, 1993; Johnson and Pratt, in press). The stages in this process are essentially those that are followed during apprenticeship teaching. Johnson and Pratt (in press) provide a description of these stages and these are summarized below. Observation allows the learners to watch practitioners with varying degrees of proficiency demonstrate the essential tasks of a domain. Observation allows learners to "develop an overall conceptual model of the various skills", understand that there are varying ways in which to approach tasks, and that practice resides in the community, not in any one individual. Modeling "occurs when the learners observe the master demonstrating how to perform different tasks, often explicitly showing learners what to do. The learners then model their efforts on those of the master." Scaffolding is the support offered by the master, and the appropriate withdrawal of that support as the learner gains skills and confidence. This process of withdrawal is known as fading. Coaching "occurs throughout the apprenticeship, and consists of overseeing the student's learning. It may include choosing tasks, scaffolding and fading, evaluating work and diagnosing problems, challenging and encouraging, working on particular weaknesses, and providing feedback." Johnson and Pratt further describe the distinction between apprenticeship and "cognitive apprenticeship" in which the learner's focus is on intellectual tasks. These are "less easily observable" and differ in three ways from traditional apprenticeships. Firstly, "teachers must expressly articulate their thinking to make it visible to the learners, and learners' thinking must also be made visible to teachers". Secondly, the context for these tasks "is often not as immediately apparent as it is in traditional apprenticeships. Because of this, teachers must explicitly place the tasks in contexts that make sense to the learners, and for which they can seem some value". Finally, teachers must help learners to understand when, and when not, a knowledge skill is transferable across domains. In describing cognitive apprenticeship, Collins et. al. (1991) bridge the research of cognitive science and the theory of situated learning by proposing a framework for learning environments, and in doing so, provide two broad categories into which the contributions of each can be integrated. These are the broad categories of "content' and "method". In describing the first category, they summarize the cognitive science distinction between expert and novice into four categories: domain knowledge, heuristic or specific "how to" knowledge, control or metacognitive knowledge (which allows for the strategic selection of heuristic processes), and learning strategies (an understanding of the exploitation of general/personal learning attributes). The second broad category, method, examines the apprenticeship teaching model as applied to intellectual skills and sequencing: the order in which new information should be presented. Collins et. al.'s cognitive apprenticeship model calls for the stages described above, but also for three stages not yet mentioned: articulation, in which the student actively and consciously describe their "knowledge, reasoning, or problem-solving processes; reflection, in "students compare their own problem-solving processes with those of an expert, another student, and ultimately, an internal cognitive model of expertise; exploration in which students are pushed into independent problem solving. Collins et. al. suggest that the latter stage, exploration, is a natural consequence of appropriate fading. The final methodological consideration described by Collins et. al. is that of sequencing, in which the "meaningfulness" of what is being learned can be maintained. Three considerations for sequencing are described: develop global before local skills, which allows students to develop a "conceptual map" in which to embed the new content; sequence according to tasks of increasing complexity; and sequence according to increasing diversity, promoting the applicability of new skills across a broader domain and contextual environment. Collins et. al. argue that explicit consideration must be given to the social character in which these skills are learned and ultimately practiced by developing situated learning tasks and fostering "communities of practice". It is in the authentic, collaborative nature of this latter consideration that a shift from extrinsic to intrinsic motivation will likely take place, and in which students learn the reality of multiple and competing perceptions. An (obviously) important distinction within the construct of situated learning is that between institutional and "everyday" cognition, or what might be characterized by "real life" versus school learning. Lebow (!994) and Griffen (1995) suggest that real life problems tend to have the following characteristics not shared by those in the classroom: they are ill-formulated and ill-structured; they are embedded in specific and meaningful contexts; they have depth, complexity, and duration; they involve cooperative relations and shared consequences; they are generally worth solving. A situated cognition perspective would also suggest that because of the contextual differences between school and real-life, even knowledge that might be applied or transferred cannot be since the contextual cues for its retrieval do not exist (Griffen, 1995). Thus the decontextualized and abstracted knowledge gained in an institutional setting remains inaccessible and of marginal functionality until integrated into the arena of practice. As a theoretical stance, situated cognition is a rejection of objectivist/transmission perspective of learning and teaching (Pratt, in press), and embraces the subjectivist assertion that meaning making is both a personal and ecological process (Choi and Hannafin, 1995). Situated cognition brings together two central components: firstly the constructivist notions of the learner as idiosyncratic agent and the recognition of learning as primarily a social negotiation of meaning ( Jonassen et al., 1995); secondly, the cognitive science recognition of context as a specific domain demanding specific expertise or problem solving heuristics (Gagne et al., 1993; Elstien, 1989). It does not share the cognitive science notion of learning/teaching as a process of the development of a production-set; it is not the imposition of an ordered series of instructional events that are deliberately applied to achieve that end (Gagne, 1978). Situated cognition is a view of learning that explicitly assumes that learning is "complex, chaotic process" (Young, 1995). In this view, a community of practitioners shares a distributed intelligence in which no single individual is the repository for all that is needed to practice successfully in that community; learning, therefore, is conversation (Jonassen et a l , 1995; Johnson and Pratt, in press). Three major principles emerge from a situated learning perspective: enculturation, knowledge as context, and differences in problem solving between expert, novice and "just plain folks" (Griffen, 1995; Brown et al. 1989; McLellan, 1994). Novices usually memorize algorithms, or "laws (McLellan, 1994), since that strategy provides them with a short-term workable mechanism for generating a solution. In an institutional setting, of course, that satisfies the primary goal of most institutional assessment practices, that of passing exams often based on "superficial knowledge of formulae and formulae manipulation techniques" (Masters, 1989). In a real world context, however, novices are unable to effectively utilize these algorithms. They cannot adapt these to be utilized in novel circumstances and are unable to detect the subtle changes in circumstance that indicate when one strategy should be adopted over another. Expert practitioners have a rich pool of experience from which to make these kinds of assessments (although the irony is that the knowledge can be tacit and automatized to the point where no reflective cognition is taking place; thus they are unable to articulate the reasons behind a particular course of action). Experts have abstracted causal models to account for the workings of the authentic context in which they operate (McLellan, 1994). "Just plain folks" also solve problems through situational experience as do the experts, but they may have a less abstracted knowledge base and utilize causal stories in place of abstracted causal models. These causal stories are closer to the experts' causal models than the students' or novices' laws (ibid). Griffen (1995) describes the distinction between just plain folks and experts: "'just plain folks' also solve problems within the context in which they are working but the knowledge base at hand is filled more with situational experiences and causal stories, not formalized, conceptual rules and causal models... Thus, both experts and just plain folks engage in the authentic activity of their culture to engage in purposeful problem-solving." The concept of "just plain folks" is important to situated learning proponents because it demonstrably indicates that meaningful learning, and as a consequence functional knowledge, is generated by everyday narratives or stories that result from experience and shared social action (McLellan, 1994). Narratives serve to index, store, explain and judge causal relationships that are important in a specific context. As Brown and Duguid (1994) state: "to understand learning, it is important to see it as a continuous social practice, rather than a process of individuals conforming to disembodied traditions." Streibel (1993) states: "... I am going to start with the assumption that learning is a social act. As soon as you make this leap in assumptions, you are led to the conclusion that knowledge itself is the result of a social act, that knowledge is social and interpersonally constructed." This vision of learning as a personal/ecological process is essentially constructivist in nature, and in stark contrast to views of learning that objectify knowledge. The latter, termed "the transmission perspective" (Pratt, in press) and as "symbolic reasoning" (Jonassen et al., 1995), seeks to efficiently codify and transmit quantifiable "chunks" of knowledge. The differences in the assumptions ascribed to both the transmission and situated learning paradigms are given in Table 1 (adapted from Jonassen et al., 1995; Choi and Hannafin, 1995; Pratt, in press). These differences have profound consequences for how assessment in conceived. TABLE 2 : A Comparative Representation of Perspectives and Construct Assumption or Transmision Situated Framework Perspective Learn ing Knowledge Objective, Subjective, independent, contextualized, stable, applied, relative, fixed situated in action, fluid Learning Objectivist, Constructivist, product process-oriented, oriented, abstract, authentic, symbolic experiential Memory Stored Connections, representation potentials Knowledge Functionally Embedded in representation equivelent to experience, the real world, personally replecation of constructed, the expert, personalized application of symbols Instruction Top-down, bottom-up, deductive, inductive, application of apprenticeship symbols Computational Symbolic Connectionist, model reasoning, probabalistic, produc embedded tion rule, symbol manipulation Context Institutional, Everyday tasks, 54 abstracted, fluid and little ambiguity competing demands, authentic environment Role of Conduit for Modelling, teacher information, scaffolding, knowledge collaborating, gatekeeper, fading, co-event control explorer, gatekeeper Role of Quantify "Problem assessment knowledge, solving is concrete assessment", outcomes for focus on tasks, choices, performance as collaborative "right or strategies, wrong" portfollios, appropriateness of solution, recognition of competing ambiguities Situated Cognition - Implications for Assessment There is a general consensus that most traditional or institutional assessment protocols "are designed to establish whether or not students can successfully reproduce facts and use procedures presented during instruction" and that "the most competent students are those who can reproduce the greatest numbers of these facts, skills and techniques" (Masters, 1989; Young, 1993; Choi and Hannifin, 1995). As stated above, reproducing this kind of knowledge is not necessarily indicative of good conceptual understanding. If situated cognition seeks to engender "deep" understanding, then clearly the methods of assessment will either foster or hinder that kind of learning outcome. Ramsden (1988) argues that students' perception of assessment has a profound impact on student learning. If the perception persists that those who develop a large store of superficially understood material will be rewarded, then (not surprisingly) appropriate strategies for 5 5 that style of learning will be adopted. This is exacerbated in areas study in which there is a high degree of curriculum overload (Arseneau and Rodenburg, in press). Situated cognition, however, shifts the onus from the mutually exclusive "right/wrong" of traditional analysis to the recognition that "even beginning learners are considered to be engaged in an active search for meaning, constructing and using naive representations or models of subject matter. Rather than being 'wrong', these representations frequently display partial understanding and are applied rationally and consistently by the individuals who use them" (Masters, 1989). These partial understandings or models are in fact a window into the conceptual framework of the student, and should be utilized as such. Instructors can then use this insight to determine a process in which these (mis)conceptions can be brought closer to the teachers own understanding of the target domain (Arseneau and Rodenburg, in press). More often than not, however, the conceptual models constructed by the student remain invisible since the evaluation process does little to probe the student's understanding (White and Gunstone, 1992). Situated cognition views the context and knowledge as, if not one and the same, then as inextricably linked. The logical consequence of this conception is that an analysis of understanding must occur "in situ", not as an abstracted "ad hoc" performance. As Young (1995) states: "... we must also acknowledge that problem solving is individual in an essential way... Some means must be adopted to capture the richness of the perception-action interaction of each student with the problem solving environment." Thus assessment must be an analysis of how the student engages with the problem, what heuristics he/she uses to solve the problem and why the student used those heuristics. Accomplishments must be assessed in the context of the authentic environment, so that if, for example, a real need in that environment is appropriate negotiation and delegation of tasks, the achievement of that end is recognized as important. In a real sense, assessment is the problem solving activity and is "an on-going embedded process... through which students diagnose their needs and seek support to bridge the gaps between apprentice and master performance" (Choi and Hannifin, 1995). As stated, situated cognition would demand that that diagnostic process take place in situ. But as Young (1995) notes, that process may have to be implicitly inferred through other routes than through a verbal protocol: "We described the paradox of verbal protocols: just when the most critical cognitive activities are occurring, the least amount of verbal recording is done." Thus, Young suggests a series of highly complex target processes for assessment, including the collaborative interactions with peers and experts, and analysis of the student's goals and intentions. Some suggestions that have been forwarded include portfolios, performance assessments, summary statistics of processes, self-assessment and reciprocal teaching (McLellan, 1993; Young, 1995; Choi and Hannifin, 1995). The problem remains, however, of how to actualize these recommendations. It is one thing to state a theoretical (or epistemic) position that context is learning, and quite another to make that position tenable for everyday use. It presupposes an extensive dialogue (of some kind) between teacher and student, one in which there is a sufficient exchange of information to allow for the process of apprenticeship or cognitive apprenticeship to take place. From the above summary of Johnson and Pratt's descriptions of the stages of apprenticeship, it would seem that coaching clearly has the strongest link to what is normally conceived of as teaching. And it would seem that it is in the quality of the performance of coaching that one apprenticeship would be better than another. Coaching, from the above description, implies the application of a very sophisticated skill-set (if that is the appropriate term, for it can be plausibly argued that effective teaching is not so much a technique but the result of a deeply held set of philosophical/epistemic principles (Arseneau and Rodenburg, in press)). The successfully application of processes such as scaffolding and fading (especially within cognitive apprenticeship) would seem to hinge on the teacher or master's ability to reach an accurate understanding of - in other words to diagnose - the student's conceptual models. This implies that the master not only have content knowledge but knowledge of effective strategies for helping the student move from some conceptual starting point to some preferred end point. This type of knowledge has been referred to as "bridging knowledge" (Arseneau and Rodenburg, in press). Let me suggest then, that the process of coaching as it has been described, is in fact the process of assessment in a situated learning context. The coach assumes the roles of diagnostician, evaluator, knowledge "gatekeeper" (Pratt, in press) and feedback provider. In an effective apprenticeship model, it can be inferred that the student is operating mostly within, or just above, his/her competency level; new areas of potential competency are not moved into until the coach is reasonably sure that the underlying competencies have been sufficiently mastered. Hence assessment has, by virtue of necessity, already taken place. A very illustrative example of this kind of assessment can be seen in Dr. Pinney's personal communication to Pratt (Johnson and Pratt, in press). That still leaves the question of how good coaches diagnose and assess conceptual models and competencies during cognitive apprenticeships, how they help extent competencies, and how they help students move their (mis)conceptions to those that are more appropriate or preferred. It is that pedagogical process that is the "meat" of the coaches' role. Summary The implications of situated cognition for assessment are: firstly, assessment should not be an ad hoc, decontextualized process, but rather one that is pursued both by inference and explicit analysis as the tasks are carried out; secondly, assessment and coaching are one and the same dynamic process; thirdly, assessment should ideally be a collaborative process in which the focus on helping student's reflect on their own, and others, problem solving strategies; fourthly, assessment should recognize that problem solving strategies should not be seen as absolutely "right or wrong", but rather as representations of a conceptual or cognitive model the student carries with him/herself. These models are crucial diagnostic tools in understanding student conceptions, and can therefore be utilized by the student/coach to help the student construct more appropriate models. Finally, I am going to suggest that both teaching and assessment within situated learning should serve to enhance the student's general efficacy in practice, but it should also serve to enhance students' autonomy or agency, and their capacity to critically reflect on what is being taught; in short it should have some emancipatory aim. In spite of Tripp's (1993) objection that situated learning is in "fundamental contradiction" with emancipatory teaching, since situated learning seeks to embed the student in a community of practice, I agree with some theorists who have suggested that situated learning - when combined with critical reflection - offers a unique opportunity/methodology for achieving these ends (Damarin, 1993; Streibel, 1993; Jamison, 1994). In its recognition of the both the subjective and the ecological, situated cognition can share much of the same ontological space that constructivism, post-modernism, cognitive science and critical pedagogy occupy (Strieble, 1993; Damarin, 1994). Although it has been postulated to provide an ideological/prescriptive basis for emancipatory learning (Damarin, 1994), it does not, however, necessarily embody an emancipatory vision. As discussed above, situated learning process does in fact, through enculturation, bring the novice into the accepted realm of expert practice (for example in cognitive apprenticeship (Pratt, in press). This necessarily means that it serves to help shape novice cognition into ways that are a reflection of the experts' understanding of their domain. Part of this domain must include accepted norms of practice in political and social terms and thus may be postulated to maintain the status quo. Often this status quo can be described as both oppressive and exclusionary. In short, situated cognition seems to have same ideological normative tension that is a reflection of all education theory. As stated, situated cognition is extremely concerned with generating "deep" and meaningful understanding. This issue of deep understanding is nested in both a pragmatic (or ends-based) realism (Arseneau and Rodenburg, in press), and, arguably, in a normative ideological position. Its pragmatic since it seeks to promote or enhance real world performance; it is ideological in that in doing so it promotes autonomous functioning within the targeted domain. If the promotion of autonomous functioning can be brought out from the arena of the domain-specific, then it is possible to suggest that a constructivism/situated learning perspective is, in fact, primarily concerned with the promotion of agency - the ability make and act on personal choices. Chapter 6: Constructivism/Situated Learning as a Normative Framework The following section will attempt to provide and argument supporting the normative value of a constructivist/situated learning perspective in instructional design, outside of the empirical assessment of its instructional efficacy. The argument will rest on the notion that adult education should, because of a just commitment to egalitarian practice, foster personal agency: the ability to make and act on personal choices. There has been, within the theory of adult education, a large body of literature within the "humanist" tradition that addresses the concept of agency through an orientation towards the individual. The language of this tradition is the language of the promotion of individual agency and includes the notions of self-actualization, self-efficacy, and personal growth and development (Elias and Merriam, 1980). This highly individuated conception of adult education has been critiqued as ideologically false in the following ways: it serves the interests of only one class (the middle class) and oppresses the lower classes through a perpetuation of political and economic hegemony; it co-opts and defuses institutions created by and/or for the underprivileged; it serves itself by creating a cultural "need" for its services; it devalues and ignores the accomplishments of those it does not legitimate (Entwistle, 1989). Sork (1994) cites several challenges that have recently faced adult educational theorists. These critiques have arisen with the emergence of Marxism, feminism, critical theory and post-modernism, and are primarily critiques of the lack of egalitarian principles. The definition of what constitutes full community participation has thus come to include the notions and effects of power, culture, class, economics and systemic oppression. These theories have helped to expand conceptions of agency past the individual to include the social constructs within which the individual lives. Thus agency is no longer a construct restricted to "boot-strapping" (Briskin, 1989), but a recognition that each individual can only act within the constraints imposed by social ordering. Thus an examination of the role of gender issues, for example, must be welcomed and actively engaged in since to ignore it is to systematically disenfranchise a segment of the population. Educational practice cannot ignore the importance of the latter issue: it simply cannot be coherently argued that an educational practice, even one concerned with highly targeted areas of expertise, which is unjustly exclusionary or discriminatory is "good" practice. In Callan's terms, this means fostering a cognitivist autonomy, one in which individuals must strive to generate personally relevant constructs of "good" living, without either blind reliance on ethical or moral authority (the extreme cognitivist view) or in the complete absence of any form of moral discourse or judgement (the extreme non-cognitivist view). Non-cogntivism seeks to further agency through social/educational practices that nurture a personally relevant capacity to act well and freely, cognitivism by providing expert analysis, support and guidance. But as Callan makes clear, the latter need not obviate the former, and it is in this acknowledgement of a role for fostering reflective personal meaning making that I think the educator's role falls. Thus good educational practice must commit to fostering agency in two ways: the first is ensuring that learners create meaningful and integrated representations of the domain of practice; secondly, actively engaging the learner in critically interrogating the assumptions they hold about a domain in a way that is both contextually and generally relevant. The latter may seem like a "soft" stance in the sense that it is not a clear unequivocal endorsement of radical emancipatory teaching, and consequently an abdication of the goal of real agency. But it is, I think, a reflection of a belief that within professional education, it is good to foster agency within that domain, and the recognition that domain competence or expertise requires "deep" representations of meaning. I would also argue that in promoting the critical interrogation of assumptions in contextually relevant ways is a necessary element of that competence or expertise. A blind adherence to the traditions of practice does not further the development of that practice, or that practice's ability to fulfil a just role in society (ethical decision making is one example of a required competence within medicine (Kopelman; 1995)). This has profound implications for ethical educational practice. If agency is the ethical outcome of a commitment to principles of a just social order, and agency is restricted by social and conceptual constructs, then educational practice must seek to help individuals shape those constructs in ways that promote agency (Callan, 1993). The question begged of course is how educators accomplish this move to greater agency, whether the chosen ground is individual or social change. In the following section I present the argument that, within professional educational practice, agency is promoted to the extent that the agent's conceptual understanding is addressed. Professional education in the absence of an explicit effort to promote conceptual understanding, for example an exclusive focus on the acquisition of disjointed declarative knowledge, is just not "good" practice (Masters, 1989). My conception of this is as follows: understanding is predicated on, or perhaps equivalent to, conceptual development; conceptual development leads to a change in beliefs, and it is beliefs, in turn, which drive action and influence understanding (Pajares, 1992). Thus the ability to act - the definition of agency - is predicated on the beliefs that the agent holds prior to and during action. In sum, and in concert with what has been postulated in the rest of this paper, "good" education is not simply a matter of adding to knowledge, but of creating the conditions which result in increased agency. Beliefs and Knowledge The relationship between belief, knowledge and action noted above -conceptual development leads to understanding leads to belief leads to action and understanding (what Friere (Giroux, 1985) and many feminists term "praxis") - is examined by a small body of literature on the relationship between knowledge and beliefs (Pajares, 1992; Ernest, 1989; Nespor, 1987). The ultimate nature of this relationship is not agreed upon, however, and some theorists contend that there is a fundamental difference between beliefs and knowledge. Beliefs are seen as powerful antecedents of action, have a high affective component, can be difficult to challenge or change, may act as the ultimate evaluative mechanism, and play an import role in perception (knowledge acquisition). Knowledge, whether procedural or declarative (Anderson, 1983), is seen as more fluid and labile, and much less capable of driving behaviour. Pajares (1992) has summarized the research concerning beliefs. He developed the following points (not all of his original points are included): 1) beliefs are formed early and tend to self-perpetuate, persevering even against contradictions 2) beliefs provide an interpretative filter 3) beliefs are prioritized according to their connection to other beliefs 4) some beliefs are more incontrovertible than others 5) change in beliefs during adulthood is the result of a "gestalt shift" I contend that in education, shifts in some beliefs (necessarily gestalt shifts - what Mezirow (1978) might call "perspective transformations") are predicated on, or equivalent to, shifts in conceptual understanding. In other words knowledge must be more than declarative to change behaviour, it must be conceptual and therefore included in the beliefs of the participant. The more central the belief, the greater the degree to which those beliefs affect perception to filter information, and the greater the educational effort necessary to create the conceptual understanding necessary to shift those beliefs. The consequence of understanding, and therefore a shift in beliefs, is increased agency. Dretske (1988) suggests much the same thing; "beliefs... are maps by means of which we steer". Models of beliefs should, in fact, "reveal the way in which what we believe helps to determine what we do". According to Dretske, beliefs are "representational structures that acquire their meaning, their maplike quality, by actually using the information it is their function to carry in steering the system of which they are a part". In short, Dretske argues that in order for information to become belief, it must be included in the highly complex web of representational knowledge already present. Thus it forms a seamless continuum with existing knowledge to create a coherent "morphology" of knowledge. This description is completely consistent with our previous discussion of constructivism and situated learning. In summary, I have argued that agency is the consequence of changes in beliefs which are in turn the consequence of conceptual understanding. Thus increasing conceptual understanding leads to increased agency. Some literature seems to confirm this view. Conceptual Understanding as Agency The idea that understanding is linked to agency is addressed by Okshevsky (1992), who suggests that "'understanding' originally constitutes a capacity that is paradigmatically expressed not within our epistemic capabilities for truth, objectivity, and knowledge, but rather through our practical competence as agents at purposive action - a competence upon which the former capabilities can be shown to be ultimately derivative in origin and structure." Thus understanding is the "originally praxeological dimension" of epistemic capabilities. Okshevsky also suggests that the capacity for understanding as "most fundamentally a capacity for understanding our own species-specific mode of 'being-in-the-world'". Understanding is the competence we express over our own modality of agency. Okshevsky (1992) claims that the human understanding of agency involves a "specific interpretation of what it is to be able to act for the sake of a goal...". This self-understanding is in the form of a "holistically-ordered" framework of references, or "concepts". These concepts are holistic structural elements pre-supposed and necessarily projected by an agent before any knowledge claim can be made. Okshevsky's conclusions are the following: our original understanding of the world and ourselves as agents is to "recognize that understanding as originally constituting a universal and generic capacity"; epistemic understanding is a derivative "of our prior generic praxeological competence"; epistemic competence presupposes praxeological competence. What the above discussion suggests to me is that knowledge is not merely a matter for epistemic consideration. Knowledge is intimately linked to agency, and knowledge which is holistically and conceptually grounded provides a "truly comprehensive" account of understanding (ibid). In other words, during the execution of an educational process, it is essential that the educator includes an explicit methodological consideration which targets conceptual understanding as an endpoint. If the previous account is correct, then the educator must address the issue of the promotion of conceptual understanding. Constructivism and situated learning offer two approaches which seem to me to be particularly important in addressing the ways in which conceptual understanding is achieved. A third theory, sharing some of the same ontological and epistemic "space" will now be considered: critical theory. Critical Theory Critical theory is a response to the absence, within educational theory and practice, of a discourse that foregrounds the profound impact of social context and processes. Consequently, therefore, critical theory moves to examine the "contradictions and asymmetries of power and privilege (McLaren, 1988). Briskin (1989), for example, provides six contradictions that women carry with them into an educational environment, including: the devaluation of mothering and the simultaneous presentation of mothering as women's work; widespread violence against women by men within in ideology of "protection" by men; discrimination against women within an ideology of "equal treatment" within traditional institutions; and the bootstrap message that those that are disadvantaged must be to blame for their disadvantage. Most critical theorists view social processes as dialectical. Thus "a social actor, both creates and is created by the social universe of which he/she is a part" (McLaren, 1988). Although still falling within the realm of critical theory, some authors seem to advocate a more deterministic ("overdeterministic" according to McLaren, 1988) framework for the "new sociology of education", in which a Marxist analysis of social processes provides a basis for an "emancipatory vision" (Collard and Law; Law and Rubenson, 1988). However, all critical theory acknowledges that many existing cultural institutions are mechanisms for the reproduction of oppression and hegemony. Many of these mechanisms and processes are invisible to even those most adversely affected by them. Critical theory, whether derived from a Marxist analysis (Law and Rubenson, 1988), Habermas's Utopian and rational "ideal speech" scenario (Guess, 1981), or Friere's "language of critique with the language of possibility" (Giroux, 1985), has as its focus this emancipatory vision. Critical theory "has as its inherent aim to the self-conscious process of enlightenment and emancipation" (Guess, 1981). This process of enlightenment and emancipation is reflexive (since those targeted are within the social processes that repress them) and leads to a "final state" in which the initial state is agreed to be false, self-delusional, illegitimate, and therefore repressive (ibid). In effect, critical theory seeks to help a group of people identify their world-picture as "ideologically false" and provide a new world-view that offers comparatively less bondage and self-delusion (Guess, 1981). In this sense, it must effect something like Mezirow's "perspective transformation" in which an agent's "knowledge of self-reflection is synonymous with becoming critically aware of the cultural and psychological assumptions that have influenced one's life" (Collard and Law). Self-reflection brings to consciousness "unconscious determinants of action" (Guess, 1981). This seems problematic in the sense that the final state reached through self-reflection must be normatively "better" than the initial state of self-delusion. How do critical theorists determine this "betterness" ordering of social structures? Habermas, as indicated above, has provided a theoretical "ideal speech" methodology for determining this normative ordering. It is predicated on the premise that agents are rational; if they could engage in completely unfettered and "free" speech" they would "rationally choose" the final state over the initial state. The initial state would be seen for what it was, a false representation of reality based on beliefs that "would be given up were they [the agents] to reflect on [them] in the light of information about the conditions under which they could have acquired [them]" (Guess, 1981). Thus the agents "epistemic principles" will change as a consequence of critical reflexive processes (ibid). This change can be associated with the process of "consciousness raising" that many social movements and educators within those movements seek to provide to their intended target group. For example, this is certainly the case within the "liberation" framework of feminist pedagogy (see Briskin, 1989; Maher, 1987 and Maher, 1987). According to McLaren (1988), critical education theorists view knowledge "as a social construction deeply rooted in a nexus of power relations". This construction is consensual, and is "heavily dependent on culture, context, custom, and historical specificity". The social functions of knowledge are rooted in the fact that "some forms of knowledge have more power and legitimacy than others". These forms include the technical (the domain of most mainstream educators), the practical (analytical descriptions of social events), and Habermas's emancipatory knowledge which "helps us understand how social relationships are distorted an manipulated by relations of power and privilege". McLaren also includes class, culture (dominant, subordinate and subcultures), hegemony and ideology as areas of concern for critical theorists. The co-option of oppositional ideologies by the dominant ideology for example, can help explain why existing hegemony is reinforced. The emphasis within critical theory on the social analysis of power and its relationship to the legitimacy of differing kinds of knowledge has enormous implications for education, and in particular adult education. It moves the discourse from a "discredited" individualism to a "rehumanizing" one of social process in which both the limiting determinants of social class and power, and the ability for each individual to be brought to, and to effect, change are reconciled (Law and Rubenson, 1988). Many educators within the critical paradigm see critical theory as providing the basis for the kind of radical change adult education has traditionally been such a prominent part of (ibid). Critical theory forces educators to confront issues around the "real" social outcomes of their practice. Do they simply reinforce existing inequities, stereotypes, and exploitive ideologies? Or do they do what educators the world over purport to do as an ideal: help each individual recognize both their own potential, and the conditions needed to help all others to reach theirs. The Role of the Educator Constructivism/situated learning and critical theory offer epistemologies that are very pertinent to the educational educator committed to good practice. I have argued that they are epistemologically committed to generated "deep" understanding and should therefore be critical components in an educative strategy. But they are also demanding in terms of implementation. The process for the educator is one that demands a great deal of consideration in two arenas. The first is the social constructs that constitute the operating environment. Particular attention must be paid by the educator not only to how those constructs appear to him/her after a systematic analysis, but more importantly how they appear to the participants participating in the process. The second arena is an analysis of two conceptual "territories": the concepts that a participant brings with him/her, and the conceptual endpoint that constitutes the aim of the educational process (Masters, 1989). This asks that the educator pay specific attention to the participant's previous knowledge, the activities and instructional systems that would promote the integration of new concepts into existing representations, and ways of evaluating the outcome that elucidates the success of these. It is incumbent on the educator to ensure a democratic process which maximizes participation, consideration for the social system already in place and how to encourage participant awareness of it, mapping the conceptual territory in terms of outcomes - which means targeting "higher-order" thinking skills - and identifying evaluative techniques that ask for an analysis of deeper understanding. This kind of teaching also asks for the recognition that the process will often be "messy", and to the onlooker (often the funder) unclear and disjointed until completion. Doing this within the constraints often imposed by a funder - in terms of time and resources - is challenging. The educator must also pay attention to, and respect, the social, cultural and personal histories of the participants and recognize and support critical reflection on the extent to which societal processes have contributed to their current position. Thus there should be explicit provision for a focus on the rights of the participants within bureaucratic structures, appeal processes, techniques for establishing more productive relationships with those administering resources, and the proper channels to challenge decisions. To accomplish his/her goals, the educator must be prepared to defend his/her position to a management that seeks a more ordered, but less useful, series of outcomes. Barndt's (1992) work with a mixed native and non-native group exemplifies the kind of instructional planning that includes the elements of critical conceptual change. Critical theory has also been implicated in good computer-based instructional design (Streibel, 1993; Jamison, 1994); this will be explored more fully in a later section. Summary Educators should: recognize ethical educational processes to be committed to agency, and thus some form of empancipatory critical reflection; recognize that agency is based on changes in conceptual understanding; seek a process rooted in an awareness of conceptual territory and outcomes; orient their needs assessments, instructional design, and evaluations on this basis. Finally, educators should recognize that agency demands a self-determination of goals based on a better awareness of the societal constructs in which the learners operate. Chapter 7: Implications of Constructivism/Situated Cognition for Instructional Technology The work "Principals of Instructional Design" (Gagne, Briggs, and Wagner, 1992) might be considered the defining prescription for translating instructional system theory into practice. It follows and builds on much pioneering work done in this area from the 1970's onward. Although not directed exclusively at CBI designers, it provides clear, consistent, and applicable methodologies for designers of computer instructional systems. These methodologies fall well within the parameters of what was and is technologically achievable, and was based on both behavioural and cognitive theory. It can be considered to have been something of a normative framework for instructional design and was used as a model by many computer instructional designers. Gagne et al. conceive of instruction as "a deliberately arranged set of external events designed to support internal learning processes." These instructional events, adopted by many computer instruction systems designers, are used to achieve the following goals in roughly the order in which they are they are presented: 1. gain attention of the learner 2. inform learners of learning objectives 3. remind learners of pervious learned content 4. clear presentation of material to be learned 5. guidance of learning by suitable semantic encoding 6. elicit performance 7. provide feedback on performance 8. assess the performance 9. provide a variety of ways to practice Gagne et al.'s instructional events, design principles, quantifiable learning outcomes, and use of learning hierarchies provide a complete methodology for CBI implementation. This makes it very attractive to CBI designers since they do not have to do their own compilation of the outcomes of a wide range of research. The process of breaking down a task into constituent components, each of which has a separate performance goal and level of mastery, is intuitively comforting. It makes linking instruction to specific learning outcomes easy to justify and demonstrate. It makes a large educational task manageable, and seems to provide a natural vehicle for the diagnosis of a student's level of cognition and progress. It also means that instructional events are much more clearly defined and therefore relatively easy to design and program. This kind of instructional design also makes the development of an authoring system possible in which even novice CBI system programmers can "code" instructional material into pre-defined components that the computer's management system can deliver in a manner faithful to the to the principles of good instructional practice. This further contributes to such a model's intuitive attractiveness to technologists. Systems using this model as a platform have been cited as successful. Wells et al. (1993) document a highly specific application in which students learned psychophysiological detection of deception question formulation. Students receiving the CBI package performed better in half the time as students receiving classroom instruction. However, this result would have to be repeated with a larger sample size to be truly convincing. Constructivist and Situated Cognitive Challenges to "Traditional" ISD Some authors have criticised this approach as falling short of producing qualitatively good design (Leshin, Pollock, Reigeluth, 1992). Guidelines for media selection are often omitted, recent advances in cognitive science are not included, and a coherent overall strategy is lost in the pursuit of a more reductionist, linearly connected and performance driven delivery system. What has emerged is the distinction between those conceptions of instructional design that remain rooted in this latter view as "traditional" and those that embrace the vision of learning generated by cognitive science and situated cognition. The distinction is symbolized by researchers moving to a change in name from instructional system design (ISD) theory to instructional design (ID) theory. ISD developers place their emphasis on a rigorous, highly structured series of ordered processes which affect all stages of the instructional cycle. It is the invocation of a "top-down", rational and systematic infrastructure which has as its core assumption the notion of knowledge transmission. In contrast, ID theory places a great more emphasis on learning as a personal, active and constructive process, the consequence of an extensive dialogue between the learners themselves, the material and the learners, and between learners and facilitators. Thus instructional design is in part a reflection of broad theoretical constructs, and in part a reflection of the context in which the learning material is typically used, and the context in which the learners will participate in the learning process. The consequence of this view, an obvious extension of the contextual important of constructivism and situated learning, is that ID theory must see the design process as an iterative, reflective and "creative process, based on intuition as well as rationality, involving divergent as well as convergent processes" (Rowland, 1993). An effective design process, according to Rowland, includes recognising that a design problem is open-ended and i l l -defined; thus problem understanding and solving may be sequential or simultaneous. Designing is also a learning process, is carried out as a reflective conversation with the materials of the situation. As a further reflection of the distinction between ISD and ID theorists, Hannafin (1992) suggests that ISD theory remains "insulated from developments of considerable consequence for improving learning, and isolated collectively from intellectual communities where significant work in next-generation learning systems has occurred". What is called for by the promise of emerging technologies is the design, not of instructional systems, but of learning environments. By fostering an active, reflective and collaborative engagement with the instructional material, learning environments support the transfer of advanced knowledge which can not be "algorithmically taught" but must be personally constructed. They help good teachers accomplish what good education should accomplish, the invocation of greater introspection and critical reflection by the student during learning. For Hannafin (1992), newer technologies make possible a fundamental shift in paradigm, from the externally controlled, convergent, reductionist nature of ISD to an understanding of the evolution of understanding, the importance of insight, and mechanisms that induce student engagement. Learning environments are internal, learner directed processes that can be supported, but not explicitly regulated, externally. This reflects a view of cognition which is not a complex rule-based process that can be algorithmically mapped, but rather is mediated by a "connectionist" process. Knowledge resides in the richness, strength and complexity of connections between data elements, not in the data elements themselves. To be effective, some education must invoke evolving learner strategies which reflect a dynamic adaptation to new information. In these situations, learning should be contextually situated. These contexts could embed a variety of complex problems. In support of this, Schott (1992) suggests that three trends in cognitive psychology call "into question the possibility of being able to unambiguously assign specific types of teaching methods to particular types of learning, as the result of an analysis of the subject matter to be taught" ( according to Schott, Gagne's conception of ISD). These trends include: the recognition that research in learning is no longer restricted to the isolation of laboratory, but examined in more naturalistic settings; the redefinition of learners as active information processors who act to construct knowledge; the learner's prior knowledge and motivation have a decisive effect on their learning and thinking abilities. The consequence for instructional technology is that a learning objective must include context in its analysis. Schott views the current challenges to the field of ID as: the teaching of more extensive abilities than before; the integration of tasks into a "natural context"; the consideration of learning processes; a cost-benefit analysis against "the background of previous knowledge". There is other support for the view that "traditional" ISD is incomplete in its analysis of learning objectives. Tennyson, Elmore, and Snyder (1992) propose a "contextual mode analysis" of a learning task for better higher order cognition development. This analysis focuses on defining the meaningfulness, complex problems, and higher order concepts and principles involved in a learning task. To form an instructional component one "clusters" problems according to shared concepts. This differs dramatically from Gagne, Briggs and Wagner's ISD in that it is not based on a strictly hierarchical format. The student is not asked to "progress" in a rigidly defined format from simple to complex skills. Instead, the student may be placed in a wide range of learning modules, all of which share some appropriate higher order concepts. The potential for self-directed exploration is much higher in this kind of a system, but the trade-off is the need for a technology that can supply the kind of intelligent curriculum management that was beyond the scope of older technologies. This kind of ID demands a system for practice sessions that "requires constant interaction between student learning (e.g. problem solving) and instructional system monitoring" (ibid). The suggestion is to use simulations of real events as domain specific learning vehicles. The technology this requires is only now becoming available. Moving even farther into the view of context-driven learning is Young (1993) who argues that ID must move to the "ecological psychology" of situated cognition. Young states that ID theorists wishing to incorporate situated learning strategies must engage in four critical tasks: select a proper generator set of situations that will afford the acquisition of knowledge; provide the necessary "scaffolding" for novices to work within a complex and realistic context; provide mechanisms to track progress, assess products and develop their own skills in utilising this concept; finally, they must define the nature and role of assessment within this framework. Good ID in this framework allows for "guided access" to successively less constrained environments as quickly as possible. Those using a situated learning perspective must constantly be able to assess the student's perceptions to detect commonly held errors of perception. Young suggests looking for three abilities within an ID for evidence of situated learning: the ability to afford transfer of knowledge from known to novel situations; the ability to provide meaning for knowledge through meaningful contexts; the ability to provide anchored situations in which multiple perspectives come to be learned. A review of the JASPER instructional series (a computer-controlled random access video disk system) is given as an example of situated learning ID. As yet the relationship between JASPER and situated learning is unclear in terms of transfer, although Young argues that it does provide anchored instruction and meaningful learning. Rieber (1992) proposes that computer-based microworlds bridge the gap between direct instruction and constructivism. A blend of behaviourial and cognitive strategies, ISD strategies are limited in scope and effectiveness. More fruitful forms of ISD will include constructivist methodologies and theory which stress experiential and discovery learning. Contrasted with the "instructivism" of traditional ISD methods, "constructivism" within new forms of ID will provide a guided-discovery process which can still satisfy the demands of a goal-oriented environment. Micro-worlds are the immediate application of constructivist theory in ID. According to Rieber, a micro-world is a "small but complete subset of reality in which one can go to learn about a specific domain through personal discovery and exploration". They "offer a compromise between the strict deductive approach suggested by ISD models and pure inductive learning advocated by experiential learning theorists. Computer microworlds are analogous to simulations except in two important characteristics: "a microworld embodies the simplest model of a domain that is deemed accurate and appropriate by an expert. Second, it offers an initial point of entry which matches the user's cognitive state so as to allow fruitful interactions to take place". Rieber also suggests that micro-worlds offer the potential to include models of cognitive apprenticeship and Vygotsky's "zone of proximal development". What Rieber really seems to be driving at is a simulation which does more than simply provide an experience that is to a greater or lesser degree similar to a "real" situation; it instructs as well. The computer controller for the simulation must diagnose a user's ability in the domains considered by the developers to be important, and adjust the simulation to "match" the diagnosis with an experience appropriate to it. It other words it must attenuate the full experience to highlight specific learning components, and dynamically building on developing skills as the learner integrates these into his/her cognitive and behaviourial repertoire. Microworlds provide a situated context for a learning experience which is "dynamically allocated" based on a diagnosis of the learner's integration of the information. This makes a lot of sense, especially within high-risk judgement arenas that involve observing, reacting, and responding to many competing, simultaneously presented variables. Poor judgement in these situations is often the consequence of attending to the wrong things at the wrong time, not observing a crucial element, and not being able to distinguish vital from inconsequential information. The need to address these issues as part of a training program has been recognized (Williams et al., 1992) Microworld instructional simulators (or to coin a word, "InSims") as tools for education are intuitively appealing if one thinks of tasks that involve multiple components. An educational software package could include a variety of InSims, each providing instruction in one area of an integrated whole. The design of such a system should, according to Rieber, balance deductive and inductive learning. It should also include contexts that provide an intrinsic motivating environment that the user can self-regulate, establish a pattern of movement from the known to the unknown, emphasise the usefulness of errors, and anticipate and nurture incidental learning. A "self-oriented feedback loop" should also be included which provides a "rich and continued stream of feedback" to students. These are general guidelines, and the road to a specific implementation of these needs many questions answered. As Rieber notes, "Mircoworlds offer a way to structure a learning experience so that a finite set of variables can be introduced at any one time". This is analogous to the situated cognition view of scaffolding. Suggesting that simulations in-and-of-themselves do not guarantee a positive orientation to learning or the benefits of contextualized learning, Lebow (1994) states that "much additional support is required to strengthen the learner's tendency to engage in intentional learning processes and to help them progressively assume responsibility for learning". Lebow identifies the components he feels to be the most salient and important characteristics of instructional simulations. These include: the ability to increase one's ability to respond to a real world simulation, and practice real decision making, problem solving, and role" playing within an appropriate context. Insims support predetermined learning outcomes by providing users with opportunities to deal with the consequences of their actions and to respond to feedback. Echoing Reiber, Lebow argues that a major potential strength of Insims is ability to allow for exploration in simplified environments, environments in which features essential to a particular conceptual construct can be isolated. A second potential strength of Insims is the ability to alter sequencing in simulations to match different, and appropriate, levels of fidelity and complexity with different level of learners. To be effective for ill-structured domains, Insims should offer multiple perspectives from within the same context and be able to provide appropriate feedback. Interestingly, Lebow suggests that fidelity as a means to increasing learning outcomes is not supported. The essential feature of a successful Insim is the processing demands placed on the learner, not the form in which the content is presented. Finally, Lebow examines the characteristics of authentic activity that are most important for InSims. These include support for higher order reasoning processes and the recognition that sometimes simplifying real world contexts to extract salience is good. Lebow also states that "instruction should influence the thinking processes students use to learn and to sustain motivation, rather than by controlling external conditions to achieve pre-set ends" and thus instructional goals should move from the transmission of information to building representations of meaning. These goals should not be replicability, communication, reliability, and control, but rather mutual inquiry, collaboration, multiple perspectives, pluralism,, autonomy, activity, reflectivity, generativity, authenticity, complexity, relevance, ownership and transformation. Brown and Duduid (1993) also examine situated cognition, with an emphasis on recognizing that learning in situated cognition requires reconceptualizing what it means to teach and instruct, the definitions of the learner and subject matter, and the role of the technology and systems used to deliver the instructional material. Learning needs to be reconstrued as "legitimate theft" - knowledge that is implicitly and non-discretely embedded in practice. Generally instructional technology is based on the notion of transmission of information. Situated cognition views the learning cycle as additive and non-discrete (such as adding color to a painting, a process in which both color and painting are transformed). The aim of any instructional system should be to provide legitimate peripheral participation (Johnson and Pratt, in press) for the newcomer in rich and productive ways. A second reconceptualizing involves the notions of implicit versus explicit knowledge. Implicit knowledge is inherent in practice and changes and evolves with it. Attempts on the part of the instructional designer to render the implicit explicit is problematic. There should also be the recognition that representation is always necessarily partial and that that partialness can be misleading. Thus any abstractions used within the system need to be a legitimate function of practice, not an intervention imposed from the outside. Instructional designers need to recognize that communities are fractious, divided, diffuse and contentious and that this needs to be reflected in the design of the system and make as much of the implicit web of practice available as possible. Instructional systems need to build connections between the learner and full practice; thus the more educational technology is constrained to essentials and individuals, the more it denies the relationship between the material to be learned and the real implementation of that knowledge. To Brown and Duduid (1993), "technology should provide an unconstrained window into practice to allow learners to see as much as it can reveal, to see in increasingly greater depths and to collaborate in exploration". It must be stated that this seems like an almost unattainable goal. Within the context of social science, Chiodo (1993) cautions that even complexly structured simulations do not necessarily lead to learning. Simulations need combined with an active and collaborative forum for discussion and reflection in which a process of "debriefing" the experience is engaged in. According to Chiodo's discussion, learning takes place in a the context of debriefing; a process which first seeks to understand an event and then help the student draw a general principle from the event which can be applied to other situations and/or contexts. To Chiodo, these accommodations of meaning form the essence of learning, for in the absence of debriefing, information may get lost or distorted. Her observations seem to me to be extremely important since (as I have stated) I maintain that education must involve consideration for agency and critical analysis. This can only be accomplished in some kind of dialogue involving multiple viewpoints and competing interests. Although generally supportive of a shift to the incorporation of constructivism/situated cognition into distance education materials, Hummel (1993) is not convinced that an approach based on mostly inductive strategies is necessarily optimal,. For example, in science education, Hummel (1993) distinguishes the student learning science from the scientist and the doing of science: "The pedagogic content of the learning experience is not identical with the syntactical structure of the discipline being studied. We should draw a clear line between the teaching/learning science and doing science." He goes on to state that: "Students do not yet possess the theoretical sophistication nor the wealth of experience of the researcher, and shouldn't be treated as junior scientists. However, the should be taught to learn problem solving in situations the expert-researcher encounters... Motives currently held for implementing practicals in a Natural Science Curriculum also seem based upon the idea or misconception that the process of learning science is or should be equivalent to the process of doing science" (emphasis mine). This point is both provocative and (I think) truthful. The role of critical pedagogy forms an essential part of my earlier contention that constructivism/situated learning perspectives have normative value in the consideration of "good" educational practice and Streiber (1993) addresses this issue directly as it applies to CBI. Structured as a series of self-reflective "queries", he offers a well written account of how an instructional designer might be informed by a critical pedagogical perspective along four dimensions: praxis, situated critical pedagogy, interpretative processes, and emancipatory evaluation. His article is an extension to CBI of the above postulate: instructional design must explicitly take both the relationship of the learner to the material to be learned and his/her relationship to the social and political milieu in which that material will be used into account. Three points must be reiterated; firstly, although it is often expressed this way, I argue that competency in a domain is not simply the "blind" application of knowledge (this will be explored more fully in a later section), but knowledge that is developed through reflective and critical inquiry (Schon, 1987); secondly, there must be the recognition that practice often operates in a complex and competing demand structure and that the "best" resolution to those demands cannot be given an absolute or unequivocal value (moreover the "best" resolution is a dynamic entity and shifts with changes in the demand structure); finally, practitioners must be concerned with playing a "just" role within a larger social order. Developing the facility to make these kinds of judgements demands more than knowledge; it demands knowledge situated in an authentic construct which has some explicit representation of agency. Steibel's queries are a reflection of these kinds of postulates. They center on learning as an active, collaborative, and socially constructed process, but it is under the headings of "interpretative" and "emancipatory evaluation" that Streibel's queries center on my understanding of the essence of critical pedagogy. These queries range from the degree to which the instructor's use of CBI opens up " the epistemological and social criteria of knowledge construction itself for critical scrutiny" to the degree that it represents coercive practices to questions concerning the degree to which critical reflection (metacognition) is supported. These queries represent another crucial challenge to a "systems" approach to instructional technology: the specific endpoints decreed by these queries are necessarily unpredictable and to a large degree based on the idiosyncratic nature of the learner and the instructional/practice environments. This thesis is echoed by Jamison (1994) who argues that to "responsibly assist societies, educational technologists would be wise to (1) examine the socio-political beliefs and practices embodied in the philosophies of their instructional plans, systems, and products and (2) develop a responsible stance towards technology (both discourse and practice)." Critical reflection is the only way in which accepted dogma can be evaluated, challenged and ultimately adequately justified or removed. He states further that the discourse in educational technology must begin to include "different ways of knowing" and the embrace the understanding that education is a human process, not the simple application of impoverished technical/rationalist algorithms. What is clear is that constructivism/situated learning are radically different perspectives from which to approach CBI. It is also clear that the demands such a perspective makes in terms of hardware and software and curriculum design means that viable, validity-proven systems will be some time in coming. As Young notes, "evaluating situations for learning will not be a simple or quick task". Situated perspectives on learning are becoming a much more important part of at least the theory of effective CBI delivery. These perspectives demand methodologies that are domain specific, and require, therefore, a context in which the information is presented. As has been noted, the changes in computer technology may help to provide a multitude of such contexts (admittedly abridged) to users in one location. Virtual reality, real-time interaction and collaborative computer networks contribute to the possibilities in this area. The following are several principles that encapsulate the incorporation of constructivism and situated learning into instructional systems: • pass control to the learner whenever and wherever possible, without compromising understanding. This might be accomplished by using a dynamic interface which changes (offers more tools?) as the learner's understanding of the system increases; • use Lloyd Rieber's notion of a microworld - "a small but complete subset of reality that in which one can learn about a specific domain" - these are syntonic which means they stress learning as a process of connection. Thus a real effort is made to make reference to a learner's previous experience. One can then nest microworlds (and interfaces) to reflect increasing conceptual understanding to some "good" point; • use InSims or instructional simulations - these are structured specifically to teach concepts as well as provide a simulation or "authentic learning environment". Thus conceptual information is made available as a consequence of a user's request during a simulation or game (i.e. why does the ball bounce in that weird way on that part of the wall? Because the texture of that part of the wall surface causes the spin of the ball to have a greater effect...) • pay attention to nature of task in terms of "authenticity" or "situatedness" - how well does it speak to a learner's past, current and future experience and culture? Learning outcomes are considered more from the question of why is that good to know rather than what is good to know. • place an emphasis on collaborative work whenever possible, and stress tasks in which multiple perspectives play a role (i.e. there is not just one way to win a game). If possible, make the task complex enough that all levels and types of thinking might contribute. Examples of Instructional Systems Theoretically Based Constructivist/Situated Learning Instructional Strategies The previous section reviewed the theoretical implications of adopting a constructivist/situated learning perspective in the design and use of educational technology. But the translation of the theoretical into the actual is not an easy task. As Tripp (1993) states: "Situated pedagogy must actually be better than traditional instruction, and not just under ideal conditions. It must survive the wear and tear of everyday use and be better under those conditions especially." The section below will review a number of instructional systems and strategies that have attempted to realize the theoretical promise of a constructivist/situated learning perspective. It must be stressed that these are presented as examples of attempts to actively integrate a constructivist/situated learning theoretical orientation into instructional practice, and provide little insight into instructional effectiveness. Although not an example of an instructional technological implementation, Griffen (1995) provides an controlled empirical assessment of a situated learning strategy in learning map skills in fourth graders. Her study, based on a pretest - differential treatment - posttest design indicated the following conclusions: traditional assessment measures demonstrated no significant outcome differences between the situated cognition group and the traditional instruction group. This rules out a treatment-assessment format interaction. Secondly, those participating in a situated cognition treatment performed significantly better on a post-treatment performance measure of map reading skills. Thirdly, no significant differences were seen in results during a novel map reading exercise, even though that exercise was based on the same performance measures that favoured the situated learning group. Her conclusions, which are reasonable given the evidence, are that situated learning did provide contextually "more robust and useful" knowledge, but that the transfer of that knowledge was mitigated by contextual factors. This lack of transfer is consistent with both cognitive learning theory (Gagne et al, 1993; Anderson, 1990) and with situated learning theory. Although not surprising, this finding raises some serious issues regarding the utility of overly contextualized information, and the need to develop some theoretical framework for developing instructional protocols allowing for greater transfer. Hummel (1993) describes a distance education strategy that mixes standard didactic teaching with contextual problem solving in a course aimed at teaching decision making around soil use in the Netherlands, an area of intense competing demands. Environmental policy dictates that soil use be multifunctional and all soil areas are utilized for one purpose at the cost of others. The course was deliberately structured to allow the student to proceed from a conceptual/didactic knowledge base to the implementation of that knowledge base under an authentic environment. The course begins with a series of structured didactic modules (first block) and then moves into a multimodal presentation of material on soil management (second block). The focus during the second block shifts to an emphasis on the application of knowledge and decisions regarding critical information, a real-life examples within an interactive video format. Finally, three interactive video cases are presented in which the student participates in an authentic decision environment. The first two of these serve to acquaint the student with the system and assessment methodology, the third as an actual assessment tool in situ. Hummel claims that "without a clear understanding of basic concepts of soil science (block 1) the student will not be capable of fulfilling the grading assessment. The cases increase in complexity. The students are given needed support initially (scaffolding) but that support is reduced over each case (fading). In a second example, students can self-select the level of support they receive during the didactic part of the content. Hummel reaches five well-conceived conclusions based on his experience with the interactive video material and the deliberate adoption of a situated learning perspective. The first is that (quoting Winn (1993) "most of the learning takes place as a result of the students' interaction with the program, not with the hard and fast instructional activities prescribed by the designer." Secondly, the higher-order decision making skills gained through the interactive video program remain "technical, not practical" since there was no dialogue between teacher and learner. Thirdly, "with respect to legitimate peripheral participation ... students first have to reach a station with in relation to the center of practice to be able to create knowledge... the newcomer who does not yet have a (soil) scientific 'gaze' can't, in any real way, create legitimate knowledge in the field of environmental soil sciences." Fouthly, situated learning is not a panacea. Initial interaction with an area of study is best done under controlled circumstances. Finally, the extent to which transfer to partially analogous situations is unclear, and may in fact (given the evidence cited above) not occur. In an effort to promote dialectical thinking, Pugh (1993) describes the use of two electronic conferencing packages, Electronic Classroom and Round Table, during student participation in a graduate course. The Electronic Classroom represented the least structured of the two packages, in which students could simply enter and share comments. Round Table offered "a discussion environment structured by the concepts of dialectical reasoning and argument analysis", in which students could selectively utilize facilities for instructional, analysis and communicative work. The course material focused on difficult and controversial cases involving multiple and competing stakeholders and students were asked to develop accurate representations of viewpoints for each of the stakeholders. Differences were noted in the content between the two packages, with the unstructured environment providing a vehicle for differing opinions and the Round Table (not surprisingly) focusing more on argument analysis. Computer aided communication was found by many students and instructors to equalize relationships. Computer-based materials have been utilized as part of a problem-based learning (PBL) curriculum in which each PBL group is given "gated" access to thirteen relevant cases (Schor et. al., 1995). The groups, which included a P B L trained facilitator, convened for each PBL session and were asked review the images and text for each case. The degree to which the content was gated was left to each author, but all cases involved the necessity for specifically entering group derived hypotheses at variable stages. Thus in one case three pieces of relevant information might be sufficient to proceed from patient history to physical exam, in others twelve pieces of information would be needed. Typed hypotheses were retained and could be reviewed at any time by both groups and individual group members. This kind of application, notwithstanding the instructional issues around the need for gating content, represents an extremely well conceived use of information technology. It provides for both collaborative and individual learning, metacognition (by reviewing and reflecting on generated hypotheses), and an accessibility level and structure that seems to empower, not disenfranchise, users. The most interesting consequence of the adoption of the system was the observation that each PBL group developed and retained an individual methodology for interaction with the material, and this underscores the need to recognize that learning is an actively social process. Finally, McMahon et al. (1992) offer a constructivist vision of the inclusion of an instructional system in the classroom. Instead of focusing on an educational system that is itself a teacher, they suggest that such an approach is simply unrealisable and therefore wrong. They offer an alternative approach in which the software system is a vehicle for the promotion of a collaborative and shared understanding about complex issues. Their system, an "empty" open-ended bubble dialogue which can be developed to fit a variety of contexts, is designed to encourage the creation and then "reflexive analysis" of a cartoon dialogue. Participants are encouraged to engage in role play and to analyse private and public dialogue in which their conception is placed against the multiple perspectives of other class members. What emerges is a rich process of interaction. Simulations and Instruction If Rieber's (1992) analysis is correct, then instructional simulations should be a part of effective instructional design but his example suffers from an educational goal that is arguably not representative of higher order cognition. In fact, the software package he describes seems a very weak example to model his microworld platform. In an earlier study, Rieber et al. (1990) also found little effect on the adult learning and retrieval tasks in a computer-based instructional system that explained and described Newtonian Laws of motion. Some effect was found in terms of latency of visual elaboration and practice, supporting the role of animated visuals in information processing and encoding, but again the evidence was weak and therefore difficult to accept unconditionally. The question remains, then, does his analysis hold for "higher" order skills which reflect constructivist/situated cognitive theory? Some research has been done on the effectiveness of simulations as instructional tools, both within and outside of computer-based instructional design. The value of simulation has been studied within police education (Williams et al., 1992). The training program was introduced in "recognition of the crucial nature of the experiential learning process and its potential role in police education". The authors stress that this training program's effectiveness relies on three factors. It involves "real" work, fosters personal reflection by discussion with peers to provide alternative points of view, and ensures that some aspects of a task a carried through to completion so that the consequences of a decision are experienced. A formal review of the program "gives particular praise to the Simulated Patrol phase of training". Though not by any means a satisfactory account of the efficacy of the program, it suggests that a simulation has some value. Where and how this value is achieved is completely unknown, however, to the reader. Shlecter and Bessemer (1992) examined the performance of students enrolled in the Armour Officer Basic course before and after SIMNET, a computer-based training simulation system, was implemented. Those trained with SIMNET had mean scores significantly higher than those without the training when the roles SIMNET provided were those enacted during the testing. Schlecter and Bessemer conclude: "interactive computer-based simulation systems that provide students with appropriate role-playing activities can train them for successful performance in dynamic vocational environments". They add that this claim must be verified. The effect that was observed in this case, although significant statistically, is too weak to be really convincing and variance for both groups was very high. Carlson and Andre (1992) examined the effect of simulation combined with conceptual change text (CCT) in overcoming student preconceptions about electric circuits. The results indicated that while CCT was far more effective in student achievement on a post-test than was traditional text (TT), the simulation provided little benefit. While this lack of effect was perhaps an artefact of hardware and software limitations, the study clearly does not indicate a strong role for simulations in the instructional package used. Jones et al (1992) provide a description of a health service management simulator. Their claim, although completely unsubstantiated, is that it can provide "an insight which may be difficult to obtain in real life". An examination of their system, even as described by the authors, makes their claim less than fully convincing. Providing more substantial support for the role of simulations, Kinzie () examined the impact of an interactive video disk (IVD) dissection simulation on learning outcomes for high school biology students in a controlled study. Their results found that the simulation was "at least as effective as real dissection in promoting student learning of frog anatomy and dissection procedures". Those exposed to the IVD simulation also performed a subsequent dissection more effectively than those viewing a video tape and those with no access to preparatory materials. Those students exposed to the IVD simulation also learned more about frog anatomy than did those who dissected without preparation. The data in this paper seems less vulnerable to criticism than in the other papers cited. Mulligan et al (1993) conducted a controlled evaluation of case-based simulations in geriatric dentistry. Their findings showed non-significant differences in outcome achieved between paper-based and the computer simulations across a wide range of measures. The description of the study did not allow a definitive evaluation of the degree of fidelity of the computer-based simulations, although one can infer from the description that the scenario descriptions were mostly text based. If this is the case, then a finding of no significance would be the expected outcome. It would mean that translating text from paper to screen has no impact on learning, a conclusion which would seem reasonable. Mulligan et al reasoned that given the finding of no difference, coupled with the record keeping potential of the computer, that utilizing computer-based simulations is a useful long term strategy. Grum (1992) studied the influence of patient vignettes on E K G interpretation by third year medical students. It was hypothesized that the presentation of a clinical vignette would facilitate E K G interpretation, especially when the student was unclear of the interpretation. The data failed to support the hypothesized relationship. In examining case based learning for orofacial pain, Clark et al (1993) used a low fidelity simulation which included some audio and graphics to gauge student response to a computer based simulation methodology. The simulations was primarily text based and used a set data gathering presentation (a heading for each e.g. medical history, physical exam etc.). They noted that the simulation was generally well received, and hypothesized that path selection analysis could play a major role in teaching by allowing the student and expert to compare differences in their case management. No substantive data was presented to support this claim. In a study of the differential effects of instructional support on learning in simulations, Veenman (1992) showed that high intelligence subjects not affected by structure level, but that a differential effect was observed for low intelligence subjects. Simulations with a high structure level supported low intelligence subjects with low metacogntive skills but interfered with those with high metacognitive skills Thus they concluded that metacognitive ability is a strong determinant of learning outcomes in instructional simulations. This has critical implications for the design of computer-based instructional materials. Summary Where does that leave us? It seems intuitively obvious that a strongly contextual educational experience would be a "good" way to learn. In support of this contention, Zelmer et al () reviewed the role of simulations specifically in nursing education and cited the following advantages of computer simulations as an instructional strategy: 1) they can provide safety in experimentation with management protocols; 2) they are easily repeatable and do not demand dedicated human resources; 3) they can provide individual pacing; 4) they support independent learning. Combine these advantages with a "broadened" view of instructional design and all should be well. Computer-based simulations should be able to partially meet the need for context-driven teaching and provide a way in which some of the more difficult features of clinical practice may be captured and utilised for pedagogical purposes. They can provide at least a modest forum for "anchored" or "situated" cognition and that, coupled with their ability to record events, should make them a valuable resource. The lack of substantial supportive data with respect to the educational efficacy of a computer-based simulated environment, however, is worrisome. There simply seems to be no conclusive evidence as yet. Chapter 8: Concept Mapping The contribution of cognitive science to educational research was the understanding that human knowledge is a highly complex phenomenon. Rather than knowledge being viewed as a series of branching and linear sequences of discrete "bits" of information, knowledge became to be defined as sets of relationships between concepts. These highly interrelated relationships in turn formed the structural basis for other larger sets and so on. Thus the metaphor for knowledge changed from "tree" to "web" to accommodate this view. The concept map is the imposition of an incomplete representation of order onto this web of understanding. The concept map was originally developed to provide both metacognitive skills and assessment appropriate to a constructivist orientation. Originally "based on the Ausubel-Novak-Gowin theory of meaningful learning" (Wandersee, 1990), concept mapping has evolved into a device used successfully for curriculum development (Edmondson, 1994; Starr and Krajcik, 1990), instruction (Okebukola, 1992; Novack, 1990; Wallace and Mintzes, 1990; Novak, 1990; Malone and Dekkers, 1984) and evaluation (Tomar and Tamir, 1990). Novak (1990) describes a concept map as "a representation of meaning or ideational frameworks specific to a domain of knowledge, for a given context of meaning". A concept is defined "as a perceived regularity in events or objects, or records of events or objects, designated by a label". Novak describes a proposition as two or more concepts linked together by words; a proposition is deemed to be a unit of psychological meaning. For any given individual, the meaning of any given concept can be defined as the representation of "all the propositional linkages the person could construct that include that concept" (ibid). It is necessarily idiosyncratic since each individual has a "unique sequence of experiences leading to unique total sets of propositions". Some of these can be shared within a culture since enough common meaning exists to allow for common understanding. For Novak, concept maps are a heuristic device for delivering and charting "meaningful" learning. Meaningful learning is the "foundation of human constructivism which is both a psychological and an epistemological phenomenon". It is a metacognitive activity in the sense that it teaches "learning how to learn". Wandersee (1990) suggests that in embedding concept meanings within a framework of propositions, "concept maps are designed to parallel human cognitive structure" or the "psychological structure of knowledge". A concept map is hierarchical rather than linear or branching, in which salient concepts are related to more general superordinate concepts. The result is a visually efficient representation of a particular domain. This representation is the gauge of the understanding of the content domain. Concept maps are generally evaluated according to two criteria, the richness of the interrelationships between concepts, and the accuracy of the word links forming the propositions between two concepts. These are not usually judged by a standard quantifiable process, but rather according to how well the evaluator feels the map demonstrates good understanding. Wallace and Mintzes (1990) do offer a methodology, however, which converts this process into a quantifiable form. Within both concept map and standard measures of understanding, there is evidence to suggest that the use of concept mapping as a pedagogical strategy is related to both academic achievement (Novak, 1990; Wallace and Mintzes, 1990; Okebukola, 1990) and a reduction in anxiety with respect to a science knowledge domain (Jegede, Alaiyemola, and Okebukola, 1990). Novak (1990) insists, however, that as a pedagogical tool, the value of the concept map lies in the act of construction, not in the simple dissemination of a master map. Wandersee (1990) suggests that cartography can inform the understanding of attempts to graphically represent scientific knowledge in the following manner: mapping and knowing are closely intertwined; maps are excellent heuristic devices; every map reflects both the data and the designer; changes in maps reflect changes in understanding; prior knowledge of the map maker greatly influences the maps created; all maps distort reality which can be exploited to achieve a communicative goal; maps have great cognitive, integrative, summative, and generative power. Implications of Concept Mapping for Instructional Systems Wandersee (1990) suggests that cartography can inform the understanding of attempts to graphically represent scientific knowledge in the following manner: mapping and knowing are closely intertwined; maps are excellent heuristic devices; every map reflects both the data and the designer; changes in maps reflect changes in understanding; prior knowledge of the map maker greatly influences the maps created; all maps distort reality which can be exploited to achieve a communicative goal; maps have great cognitive, integrative, summative, and generative power. In my opinion, Wandersee's analysis provides a powerful series of guiding principles for the design and presentation of educational or instructional software; it could provide a platform consistent with, and which answers, many of the epistemological and philosophical principles and concerns of constructivism. It also provides a pragmatic and cost effective way of unifying and integrating several areas of concern to both software designers in general, and those of instructional designers. Let me deal firstly with instructional issues. If concept maps are a legitimate way of structuring knowledge, then the act of constructing that map provides a good vehicle for structuring pedagogical content. It provides a heuristic for the hierarchical and subordinate conceptual targets that the instructional system must address, and in doing so, also provides a framework around which a contextual environment can be structured. Thus competing interpretations of a presenting context can be explicitly charted and presented based on the conceptual issues that have been identified as critical for the successful resolution of that context. Furthermore, this allows for an intriguing possibility: the first is that the choice of one of the competing interpretations provides an implicit indication that the chooser carries the conceptual (mis)conception targeted by that interpretation. Thus the possibility exists that a context arranged specifically to reflect the issues within the concept structure serves as a diagnostic tool and as a way to provide the basis for the development of a user model based on that tool. If this is true (a huge if), then a basis exists for some form of simple tutorial intelligence. Let me deal secondly with software design. Software designers have long been concerned with issues of human/computer interaction, primarily in terms of ergonomics and motivation. Out of ergonomic concerns has arisen the discipline of interface design addressed by a substantial body of literature, researching questions dealing with ease and intuitiveness of use, iconic representation, accuracy of communication and so on. Within a large information space, keeping the user informed of where he/she is with respect to the total space is of major concern (this is frequently cited as one of the major problems of a hypertext environment). The use of "maps" is a particularly attractive way of orienting the user, since it is a common skill. I am suggesting that concept maps, because of their potential link to cartographic representation, offer the inherent property of informing the user of where he/she is and where he/she has been. This property can, if properly utilized, be exploited as a navigational tool. The area of motivation is also of particular concern to software designers (as well as instructional designers). Research questions in this field are concerned with the degree to which an interface is seen as compelling and "draws" the user into an interaction with the machine. There is evidence to suggest (evidenced by game design) that a powerful method of involving the user in interaction with a software system is the notion of exploring new territory. Because of their ability to lend themselves to cartographic representation, concept maps could provide this heuristic. In other words, the user would be given the chance (metaphorically as part of the interface) to explore "uncharted" conceptual territory. Each "foray" into the conceptual unknown could lead to points of instruction based on an appropriate format. Chapter 9: User Modelling Computer-based evaluation and/or instructional tools represent a significant use of educational technology, but these systems may also provide more than a means by which content is presented. These systems can also be structured to respond to user behaviour and exploit this information as a basis from which conceptual diagnostic tools and "active" tutoring can be dynamically generated. Thus, although this represents a highly sophisticated application, educational technology may contain algorithms which create models of student cognition and to which the system responds by tailoring the presentation to match the perceived demands of the student, including what gets presented and how it gets presented (Csinger, 1996). The degree to which such "intelligent" applications have been successful as pedagogical tools is unclear. It is also unclear what algorithmic/heuristic/intelligent processes would be required in order for such a simulation to demonstrate the ability to accurately diagnose the conceptual models carried by students. Is "real" artificial intelligence required, or can "dumber", less sophisticated and more stochastically based process work as well? Fontaine et. al. (1995) describe a highly sophisticated environment which utilizes a rule-based expert system to generate, present and evaluate clinical simulations. Within clinical medicine, several of these systems have been developed and have been proven to be somewhat successful in their representation of medical knowledge (Berner et. a l , 1994). In Fontaine et. al.'s environment, clinical simulations are developed by having the "author" simply set the parameters around a particular topic. The system itself generates cases, evaluating and responding to student input and representing the results of valid requests. Student tracking and evaluation are also performed. Once developed, this kind of system allows for the rapid development and deployment of simulated cases which all make reference to the same rule-based propositional network. If that network does indeed represent medical knowledge accurately, then the system represents the highest order of developmental sophistication. It is still arguable, however, that the pedagogic utility of this kind of system is marginal, since effective teaching is clearly and substantially different than from the presentation of content (a major contention this paper). Effective teaching, as both theoretical and experimental literature indicates, should include the incorporation of relevant and meaningful feedback, guidance and support during engagement with the simulation, the ability to offer multiple and competing perspectives and to enable the student to critically reflect on the instructional process. Simulations, in and of themselves as the evidence indicates, do not necessarily make for good instructional strategies. Fontaine et. al.'s expert system offers no way of marrying the knowledge base to proven heuristics for the diagnosis and remediation of conceptual problems, and for the provision of feedback appropriate for the development of meaningful knowledge representations. Autonomous tutoring systems, in which content, pedagogical strategy and evaluation mechanisms are dynamically linked, have been developed (Fernandez-Castro and Diaz-Ilarraza; 1993), but the efficacy of these in terms of fostering "deep" understanding is unclear. What this kind of system does allow for is the drill/practice schedule that has been implicated in the development of expertise by instructional practice and cognitive theory; but the evidence seems to strongly suggest that practice only has pedagogic value when combined with feedback mechanisms that promote conceptual development. No obvious appeal to this kind of instructional strategy was described by Fontaine et. al.. Feedback to student input seemed to be limited to the indication that a differential diagnosis was correct, incorrect or premature. In the absence of any clear proof of instructional utility, the degree of dedicated expertise required to develop an expert system for each domain (demanded by cognitive theory as an integral part of expertise), makes this kind of system an expensive way to provide case simulations. Boohan (1992) describes an example of how a less sophisticated process might offer conceptual diagnostic ability. Using a questionnaire 1 0 0 format, the system, "DIAG", links Baysean probability calculations and a simple inferencing engine to assign probability values to a series of mutually exclusive and competing conceptual models. Each of the models is initially assumed to be equally likely, but probability weights are adjusted as the student answers each of the questions. Tolerance limits are set to allow for the selection of the model that is assigned the highest probability, or if that probability is too low, the selection of "NO M O D E L " . The authors claim that "the values of probability assigned with these codes have bee found to give satisfactory diagnoses of a set of a responses when compared to human judgment. There is, however, no data to support this contention. It would be very interesting to formally and empirically verify the ability of this kind of system to diagnose conceptual models. This kind of "stochastic intelligence" is completely dependent on the pedagogical intelligence of those who designed the system, but it offers a glimpse of how a simple statistical reasoning process might provide some conceptual diagnostic ability. The assumptions of such a system are that most students will carry, in a reasonably close way, one of the given competing conceptual models (this kind of "pre-hoc" strategy has also been adopted by computer scientists interested in user-modeling (Csinger, 1996)). Although this seems an enormous assumption to make, it may not, in fact, be that dramatic an assertion. If an instructional system was required to diagnose and respond appropriately to an infinite range of cognitive models then it would have to embody the kind of sophisticated reasoning engine that Fontaine et. al. (1995) describe. That end goal, a knowledge base that is a necessarily partial but accurate reflection of real-world relationships, represents an ideal state for computer-based knowledge representation. If, however, in a given instance in a given domain, an instructor has a reasonable basis from which to believe that most students carry one of a limited number of conceptual models, then the instructional system would only have to account for those to be of some utility. This strategy may offer only a partial solution, but a solution that can meet the demand for a cost-effective development cycle. The second important consideration in the value of a stochastically driven system is that, similar to an expert system, the data from which the stochastic system infers a conceptual model can be dynamically updated to increasingly better reflect the "real-world" distribution of those models. This data can be utilized by both the system and the instructional designer to optimize the inference model. Although it can be argued that this kind of simple inferencing system can only achieve an accuracy level well below that of an expert teacher, it may still provide enough utility to make its development worthwhile. This is an appealing prospect. Later in the paper, I will propose an instructional model in which a real world context can be married to a good pedagogical strategy, and in which some form of stochastic process can provide a simple method of conceptual analysis. Moving Forward... If this paper is to fulfill its mandate, it is now time to turn our attention from a focus on the literature in pedagogy and instructional design to that within medical reasoning and education. The previous review has, however, been necessary since so many of the conceptual pieces that form the basis for these areas has already been addressed in previous sections. Medical reasoning is, howver, not the simple application of these principles to a new domain. It is a specialized field of research and as such needs to be examined carefully. Chapter 10: Clinical Reasoning "Medical Problem Solving" by Elstien, Shulman and Sprafka (1978) was a landmark in the application of cognitive science research methodology to clinical reasoning, and provides the foundation on which most subsequent research has been based. The following is a short review of the current state of the understanding of clinical decision making from a cognitive psychological perspective, based on this earlier work and a later review article by Elstien (1992) entitled "Clinical Reasoning - a Ten Year Retrospective". Some of Elstien's major findings suggested that all physicians, irrespective of expertise, generate hypotheses about the probable diagnosis almost immediately after beginning a particular case. Elstein found that during initial problem representation, both successful and unsuccessful diagnosticians employed a process of generating a limited number of testing diagnostic hypotheses. These hypotheses were generated very early, occurred in the absence of explicit instructions to do so and to some extent guided subsequent data collection. Successful diagnosticians, according to Elstien's work did not generate more hypotheses nor hold more in working memory although they more accurately interpret data to test their hypothesis, suggesting more domain specific knowledge. Even experts who do not depend on the generation and testing of hypotheses in a specific case, but retrieve a solution directly from structured knowledge, do consider and evaluate alternatives in the early stages of problem formulation. This is supported by later work (Whelan, 1988). Elstein argues that such a process is a psychological necessity given the limits of working memory and the complexity of the task. Hypothesis generation is an efficient way of clustering or chunking cues into a format that works within the limits of short term memory (which can typically only hold five to seven items simultaneously). Thus hypotheses allow the clinician to cluster or constellate symptoms in an organizational framework. It is a heuristic aimed at limiting the search space and fixing and ordering the 103 content. The heuristic does not serve to generate many competing hypotheses, but rather to generate a few - the upper limit of which was seen to be five to seven in accordance with short term or working memory limitations - hypotheses deemed most likely due to experience or reasoning. In support of this, Elstein found that inconsistent cues lead to less hypothesis generation than consistent cues, suggesting that clustering can only occur in the presence of known or reasoned frameworks. Importantly, much research also suggests that domain specific knowledge is critical to problem solving; there seems to be little empirical support for the transferability of domain-general strategies. In general, according to Elstein, what separates experienced or expert clinicians' from novice clinicians' diagnostic ability seems to reside in domain specific expertise, not in strategy or process. For example, differences in chess expertise have been postulated to be more a reflection of differences in the structure of knowledge than in strategy or the types of cognitive processes; the ability of experienced physicians to organize information into bigger chunks and generally recall more information about a specific case than novices; the ability of experienced clinicians to have less difficulty with atypical cases; the tendency for expert clinicians, as high knowledge persons, to make more inferences from prior knowledge. Elstein acknowledges that experts will draw on highly structured and encoded knowledge to solve diagnostic problems, a process resembling pattern recognition. This agrees with Patel et al.'s (1990) position that expertise is distinguished by the knowledge base and the ability to bring that knowledge base to bear on a particular problem. Elstein suggests, however, that in the early stages of problem formulation that all clinicians, irrespective of expertise engage in some form of hypothetico-deductive reasoning. Also, pattern recognition cannot be brought to bear when problem is atypical or diagnostic issues are complex - in these cases a hypothetico-deductive method is used. For novice clinicians, most problems and situations are atypical and consequently not routinely solvable. Thus the principle method used for problem solving is hypothetico-deductive reasoning. This fits Whelan's (1987) description of other categories of student (mis)interpretation of clinical symptomology, based on his own clinical teaching experience. These included interpreting clinical information out of context, ignoring symptomology not fitting the preliminary diagnosis, failing to consider the importance of absent symptomology, and not eliciting key information. Elstien's findings also suggest that not only were experienced physicians more selective in data selection, but that there was considerable overlap in the data chosen by experienced physicians (note that no relationship exists between the thoroughness of data collection and accuracy of interpretation of that data - therefore evaluation of expertise should emphasize data interpretation not data gathering). Additionally, evidence indicated that experts develop an abstracted and ideal prototype of a category (a finding corroborated by Papa et. al, 1995, 1996). Personality differences were not found to play a major role in decision making. Finally, problem solving varied greatly across cases and was primarily a function of domain expertise; the existence of domain general skills across disciplines (even within broad medical categories) was not supported. In support of these contentions, and echoing expert/novice distinctions in cognitive science, Hassebrock and Prietula (1992) analyzed the verbal protocols of physicians solving case problems in pediatric congenital heart disease. These protocols - organized along three headings, knowledge states, conceptual operations, and lines of reasoning - revealed similar differences between students and physicians. Physicians used "richer, more substantive lines of reasoning", operated from abstracted principles, were able to resolve differences, and could encode and recall greater amounts of clinical information. In Elstein's work, the formation of hypotheses is a ubiquitous part of physicians' clinical reasoning. For both the novice and experienced clinician the process serves as an organizational heuristic, allowing the clustering of data into larger chunks and thereby increasing the efficiency of working or short term memory. However, the experienced clinician has access to a well structured knowledge base from which problem solutions may be drawn directly. (For the novice the hypotheses generation/testing method is the only option. This raises the issue of how that kind of hypothesis testing process might be incorporated into an instructional strategy.) Given the above, Elstien concludes that diagnostic expertise in a given domain is: • the accrual of a useful knowledge base; • rules for accessing and applying the knowledge base; • prototypes for classifying instances in a domain. In partial support of the latter point, Papa et. al. (1990) found that pattern discrimination, and not pattern matching, was the primary predictor of student diagnostic accuracy in myocardial infarct cases. Pattern discrimination, is defined as the ability of an individual to correctly recognize a case example as belonging to a particular class of diagnoses as opposed to another. Pattern matching is the ability to match a specific case to a specific diagnosis. The degree to which pattern discrimination is successful is dependent on the relative distinctiveness of other competing classes. Therefore, the better the student can represent and separate prototypical classes based on defining criteria, the better he/she will be at correctly placing a presenting case within a class of diagnoses. This process of abstracting prototypical salience is fully congruent with the cognitive psychological assertion that increasing expertise involves an increasing abstraction of conceptual models (a move to principled understanding). Papa et. al. suggest that physicians/students with more experience have better discriminated prototypes, and consequently are better able to invoke pattern discrimination as a method of reaching a correct diagnosis. It follows that if this model is correct, processes that aid in the development of better prototypical representations will foster better diagnostic skills. Further work (Papa et. al., 1996) shows a strong correlation between case "typicality" and differential diagnostic (DDx) performance by first year cardiology residents . The process for generating typicality estimates for each of the sixty-four cases finally selected (eight cases within each of eight disease classes) involved generating over thirty thousand cases based on the conditional probability estimates, by thirty-four expert physicians, for the presence of symptoms in a particular disease class. A computer-based process used these conditional estimates to simulate each physician's DDx for each of the cases. Those cases for which less than forty percent of the simulated physicians identified correctly were dropped, and typicality measures were assigned by weighting within-class and between-class variance. A typicality-gradient could then be generated by plotting typicality against the number of physicians correctly diagnosing the disease. Eight cases within each disease class, equidistant on the typicality gradient, were then selected. As typicality increased, so did the ability of the students' to achieve the correct DDx. Papa et. al. argue that typicality gradients offer a new method of generating assessment criteria; one's level of expertise would be reflected in the ability to correctly diagnose cases with increasing atypicality. There is evidence that DDx performance in one disease class is not predictive of DDx performance in another disease class. This means that instructional efforts should focus on ensuring strong matches between student ability and case typicality; a student not correctly diagnosing a case with high typicality can be assumed to have a poor disease-specific DDx conceptual framework. Other work (Murphy and Freidman; 1996) also provides some support for expert/novice distinctions in the cognitive representations of disease. Expert physicians were more likely to have well-differentiated clusters of symptomology correlated with ischemic disease than novices as measured by repertory grids, although no differences were found in the case of pneumonia. There was some indication that the clusters seen during the analysis had a correlation to diagnostic prototypes. In contrast, some weak evidence to suggest the idiosyncratically constructed nature of medical knowledge is presented by McGaghie et. al. (1996) based on the results of a Pathfinder™ analysis of pulmonary concepts. This algorithm provides a graphically represented conceptual "map" of a domain based on similarity judgments between pairwise presentation of concepts. Only limited coherence was found between expert faculty, and another study (McGaghie et. al.; 1994) showed almost no coherence. This raises some interesting questions about the value of concept maps as reliable quantitative assessment tools. McGaghie et. al. (1996) suggest that this reinforces the assumption of constructivist theorists that individual experts code knowledge in unique but efficient ways. Page (1995) provides a summary of a model of clinical decision making - proposed by Bordage (1994) - in which four type of reasoning are differentiated; these are listed in order of ascending expertise: reduced knowledge in which the physician has no cognitive basis from which to understand and solve the problem, dispersed knowledge in which a physician's declarative knowledge structure is unintegrated and discrete, elaborated knowledge in which a clinical problem can be solved by a process of the logical application of relevant knowledge, and finally compiled knowledge in which a tacit and efficient process is applied to the clinical problem in a manner similar or equivalent to pattern matching. Good clinicians are marked by the existence of an elaborated knowledge structure, which in high degrees of expertise can be considered to be "compiled". (A compiled elaborated knowledge base allow for the automatic execution of lines of clinical diagnostic reasoning, a process similar to the automaticity of procedural knowledge described by Gagne et. al. (1993)). Elaborated knowledge is characterized by deep understanding: understanding that goes beyond the recall of factual information to include meaningful and predictive understanding between presenting symptoms and their relationship to underlying causal processes. Thus nothing is seen in isolation but as part of a larger inter-related network of symptoms, disease and contributing factors; elaborated knowledge is knowledge that is encoded in networks which offer numerous points of access and retrieval. Previous research has shown that better diagnosticians were those who had organized relevant material into coherent systems of abstract properties (Bordage et. al., 1994). In agreement with Papa's work on prototypical representation, Bordage states that the definition of a clinical problem "consists of not only naming the syntax of its symptoms and signs but also of creating a pertinent network of abstract semantic relationships whose function is to delimit the appropriate diagnoses from the inappropriate ones". Therefore, instructional materials should include such a semantic network (and not be limited to "strict syntactic nomenclature or classification hierarchies"). Summary Although there is some tension between Elstein's work indicating that medical reasoning is based on a hypothetico deductive strategy, Bordage's differentiated stages of clinical reasoning, and Papa et. al.'s clear demonstrations of the impact of prototypical salience, and the cognitive science assertion that expertise is a function of highly automatic processes, there is much common ground. Diagram 2, based on the literature cited above, is a representation of diagnostic reasoning along several dimensions on which the distinctions between the novice and the expert can be plotted. These dimensions are: use of hypothetico deductive strategies, cognitive representations of discriminated prototypes, degree of integrated or elaborated knowledge, and automaticity of process. The novice is characterized by a high degree of hypothetico deductive problem solving, a poorly integrated knowledge base, poorly discriminated prototype representation, and little procedural automaticity, and a poor level of conceptual understanding and abstracted principles. The expert is characterized by the opposite ends of the scale. Elstein did acknowledge that the expert uses highly efficient, abstracted and automated representations in which a hypothetico deductive model would be an essentially tacit process. It is the novice who must resort to a "means-end", step-by-step diagnostic process. Thus the various mechanisms of diagnostic reasoning will always have the same expression in the expert: a fast, highly automated process in which the degree of 109 hypothetico deductive strategy invoked is a function of the atypicality and ambiguity of expression of symptomology. Clinical instruction can, in the recognition of the novice/expert distinction, deliberately tailor its strategies to promote those factors that bring expert reasoning closer to the novice. FIGURE 3 : The Expert/Novice Distinction Chapter 11: Facilitating Medical Reasoning - Instructional Implications The general principles that can be distilled from the work in clinical reasoning suggest the following: both novice and expert physicians actively engage with the clinical problem as the evidence unfolds but the accuracy and therefore the utility of that process depends on domain expertise; domain expertise in turn depends on both the amount of domain knowledge and the degree to which that knowledge is integrated and contextually accessible; a well integrated knowledge base allows declarative and procedural information to be "chunked", facilitating recall and decreasing cognitive load; a decreased cognitive load frees the physician to concentrate on any atypical features and strategic planning; although novices rely exclusively on some kind of hypothetico-deductive strategy, experts will usually utilize some form of tacit knowledge that relies on prototypical salience, and invoke a hypothetico-deductive strategy only when the case lacks sufficient similarity to the prototypical models. These principles are highly congruent with those that have been described in the previous section on cognitive science. Based on those principles, it should be possible to deduce, "a priori", instructional strategies that facilitate medical reasoning. Firstly, instruction should target the development of an integrated, elaborated (in Bordage's terms) and contextually relevant domain-specific knowledge base; secondly, instruction should foster prototypical salience for cases within that domain; thirdly, instruction should provide authentic contextual "anchors" (CTGV Group, 1993) that enable knowledge retrieval under actual practice conditions; finally, instruction should promote the appropriate automaticity of procedural knowledge. (This list is not to imply it in any way implies that a "proper" sequential order must be followed; these are simply the instructional "ends" that follow from the previous sections. The general guidelines as to the "means" to achieving these ends have to some extent been covered in previous sections, but each discipline has its own literature and educational researches and should, therefore, be examined separately.) Although the implementation of each of these "shoulds" is an enormous challenge, the fourth point raises some very interesting problems. As has been stated before, there has to be the recognition that for the non-expert, a balance must be struck between automaticity and conscious reflection (metacognition). Without this reflection, automaticity becomes overly contextually bound and of restricted use under even slightly varying conditions. This is the problem of transfer. As well, for the non-expert, there is the risk that incorrect procedural process will become embedded in the framework of a highly automated sequence. Although this automaticity serves the expert well (for example as a sequence of proper processes during an operation), these sequences are very resistant to change and can have serious consequences for the non-expert (musicians, for example, if they are self-taught, have an enormous difficulty in shifting to learning "proper" classical technique and often state that they must "unlearn" bad habits before learning classical technique). It can also be argued that the resistance of a discipline to change (low adaptability) is a consequence of a lack of reflective metacognition on the part of its practitioners, and that the skill of reflection is one that should be built into the cognition of every practitioner as an automatic procedural process. This point is essentially that of Schonn (1987), who described the ideal practitioner as one who constantly engages in a deliberate and conscious process of "reflection-in-action", judges the efficacy of the process, and adjusts the process towards optimization. Finally, the point should be reiterated that if educational practice is concerned with the development of an ability to judge the "just" role of a practice in larger society, then a critically reflective process is essential. The importance of critical inquiry for medical education was proposed by Kopelman (1995) in four general goals in which the practice of medicine is interrogated within the political and social constructs in which it operates: identifying and examining assumptions, broaden perspectives and self-knowledge, develop critical-thinking skills, foster tolerance and a skepticism about dogma, and cultivate empathy. Without the kind of discourse generated by these issues, Kopelman argues that the ability to competently, and therefore ethically, choose between conflicting alternatives is compromised. Since ethical competence is arguably a core component of competent medical practice, critical inquiry must play a role in the medical curriculum. (There is, among those that I have talked to in the general medical community, a recognition that ethical reasoning is not adequately addressed by the standard medical curriculum.) This kind of critical reflection is generally dependent on both an understanding of ethical constructs, and an actual context (i.e. is generally case-based) and depends on the explicit articulation and evaluation of conceptual frameworks which are then collaboratively assessed and critiqued (the case-based format used by the Division of Biomedical Ethics at UBC). For example, ethical judgment in any discipline will involve the ability to perceive situations from another party's standpoint (empathic reasoning). Without a process of critically examining the assumptions we hold about that person, and the social and cultural factors that may impact on those assumptions, we will not be able to competently judge the other person's viewpoint and consequently we will not be able to chart an ethical course in decision making (ibid). Thus our competence is affected by an inability to adequately represent competing positions. Given what seem to be the differences between novices and experts, the route to expertise must provide for extensive focused practice - and feedback - with a variety of problems. To this end, Elstein (1978) suggests that methods using hypothetico-deductive models can be useful as instructional strategies, since these could allow a student's reasoning process to be made explicit, and provide contextual opportunities in which the correctness of that reasoning can be empirically tested. Supporting contextual teaching, Bordage et. al. (1994) state that the development of elaborated knowledge in students can be encouraged by "having the students make patients and the resolution of their problems the impetus for reading and learning" . This supports storing knowledge in the "context in which it will be retrieved" - in other words, the notion of situated cognition. In agreement with Papa (1996) Bordage supports the idea that from an instructional standpoint, "less is better" for beginning to read or explore a new topic (i.e. two or three prototypical cases should formulate the nucleus of understanding the disease, not the 40 - to 50 atypical cases which would rarely occur during practice). Bordage believes that the value of on-site clinical teaching is watching a clinician deal with real world problems; he states that "its one thing to list diagnoses: its another thing to present a true differential diagnosis where justifications are offered and commitment made to a working diagnosis" (emphasis mine). Papa et. al.'s (1996) work is intriguing since it so clearly demonstrates the process of abstracting prototypical salience within specific classes. The value of well-delimited prototypical representations as instructional tools was suggested in earlier work (Eleison and Papa, 1995) in which students received five different diagnostic aids - in the form of probability estimates for the presence of each, of eighteen symptoms - to help in the diagnosis of neurological disorders. The results suggest that information that better enabled discrimination by clarifying differences between disorders on the basis of the likely presence of each symptom also facilitated diagnostic accuracy. The forms in which this information was presented, in the order in which they supported diagnostic accuracy, were: verbal coaching plus a matrix of differential diagnoses and symptom conditional probabilities, a prototypical statement of the differential diagnosis including symptoms likely to be present in seventy percent of the cases, the matrix alone, the matrix information embedded in a series of paragraphs ("hard text"), paragraphs using "soft" descriptors of probability such as "usual" or "often" ("soft text"), a control group receiving no information. One of the most interesting results of this study was the power of the prototypical description as a conveyance of essential information, since it may suggest the development of a good conceptual model as a diagnostic tool; Eleison and Papa, however, tentatively suggest that its power lay in its ability to help rule out competing diagnoses. What is clear is that without an actively supportive conceptual framework (semantic information in Eleison and Papa's words) in which to embed contextual information, no learning is achieved (as evidenced by the control group's lack of performance even after feedback on the accuracy of their diagnoses). Both these papers have instructional consequences that would seem to be clear, and which make common sense; begin with a case of high typicality, ensure the development of the good conceptual framework for that class of diseases by utilizing strategies that actively foster the development of conceptual models based on the abstracted principles (examples of these strategies are given later in this section). Combining this process with Gagne et. al.'s (1993) suggestions for matched and non-matched exemplars would help to generate a strong abstracted and principled representation of that disease process. Only then, should instruction move to the introduction of successively more atypical presentations of a particular disease class. Whelan (1987) describes a series of phenomonographic studies of beginning clinical students, which showed that there were two "qualitatively different" approaches to solving clinical problems by students: ordering, in which the students' discussion of symptoms showed little integration or underlying structural coherence (described as atomistic), and structuring in which the sympotomology and possible causal pathologies were discussed in terms of supportive evidence, relationships to basic science, and in which an underlying integrative structure was clearly discernible (described as holistic). The clinical problem solving strategies were also evaluated for evidence of overall understanding and for underlying problem solving strategies. These were classified as exclusion in which diagnostic options were ruled out on the basis of consistency with the presenting symptomology, pattern matching which associated a diagnosis solely on the basis of association with one or more symptoms, and diagnostic integration. The latter was seen as indicative of sophisticated medical reasoning since it required the use of logical, inductive, and deductive reasoning based on the relationship between pathophysiological processes and symptomology. As might be expected, good correlative relationships were seen between the use of structuring approaches, diagnostic integration and understanding; exclusion and pattern matching processes were correlated with poor overall understanding. Both ordering and structuring approaches were seen in a minority of cases, and in several, ordering approaches were associated with overall understanding. The obvious conclusion, that these students had a strong familiarity with the relationship between the sympotomology and the underlying clinical pathology, could not be determined. Whelan argues that knowledge of the "characteristically different ways in which students may approach the task of clinical problem solving offers a valuable viewpoint from which to help students who are having difficulty with the diagnostic process". Instructional strategies should aim at helping students to focus on the integrative and chronological significance of the presenting symptomology, and the recognition that the exclusion methodology (defined in the previous section) can only be successful if the complement of domain knowledge is within the expert range. Consistent with previous discussions, the focus should be on the clinical reasoning process, within the context of a specific case, rather than on the product (the correctness of the diagnosis). Recent work by Norman et. al. (1996) supports the need for contextual information. Upper year students' diagnostic accuracy was a direct function of the degree of supporting contextual information, which is "likely to be underestimated" by the instructors of clinical skills. Case-based clinical teaching is not without its problems; redundancy, lack of engagement and little value in promoting learning have been cited (Irby, 1996). In his presentation of three exemplary case-based teaching models, Irby provides a phenomonographic insight into the experience of contextualized teaching. What emerged were five common principles: anchor instruction in clinical contexts arid help foster the connection of that case from prior knowledge and to general principles of medicine; ensure that all students actively articulate coherent accounts of their models and thinking; model professional "thinking-in-action" (acknowledging and resolving competing demands, compassion, clinical skills); provide immediate feedback and help the student construct better conceptual representations; create a collaborative learning environment and stress mutual cooperation. Distinguishing between surface and deep approaches to learning, Eizenberg (1987) provides a good framework for curriculum analysis that identifies six issues that are important when developing a curriculum that, within anatomy, fosters the development of an integrated and clinically relevant knowledge base, one in which the underlying "referential" or "structural" nature of the content is addressed. These issues are the content, process, stages, context, purpose and learner characteristics. These are not treated in isolation from each other, and the development of each of these components is informed by the desire to help students create "deep" and principled representations of the content. Content organization, for example, was organized to avoid "atomistic" knowledge or the memorization of disassociated information and students are encouraged to seek meaningful representations of the essential principles of the content, and how these principles will impact on their ability to perform clinically. Instead of focusing on the examination of several body regions, in which little integration of component function with other regions was achieved, content was organized by system type. Two important scheduling changes were made, the first to correlate dissections to specifically support the region under study (as opposed to whole body study, whole body dissection), and to integrate the region under study with other subjects (study across disciplines by organ system blocks - this approach has been adopted by the Faculty of Medicine at U B C for the new problem-based learning curriculum). Eizenberg recognized the enormous impact of other contextual factors in the ability to successfully implement a more holistic teaching strategy (workload, assessment expectations from both student and institution), but makes some useful and immediately applicable suggestions for instructional intervention. One of these is encouraging metacogntion on the part of the student by helping him/her revisit their own process of learning and correlating these to stages of understanding, actively soliciting information about student intentions and relating these to the content, and providing a clinical context in which a meaningful understanding of the content under study is seen to have a relationship to clinical efficacy. An example of the impact of a simple metacognitive strategy within a situated context was the use of cognitive feedback to enhance the diagnostic accuracy of physicians, and medical students, predicting streptococcal infections in patients with sore throats (Wigton et. al., 1990). The educational intervention included a one-hour lecture, three sessions of computer-based simulated cases (24 per session), and monthly reports of the actual incidence of streptococcal infection within the intervention period. Physician estimates of the chance of infection for each of the simulated cases were based on four predictive and 3 non-predictive symptoms. Actual infection incidents were based on a pool of 286 patients and the rate was found to be accurate within several geographical areas. After each set of twelve cases were presented, the computer displayed the user's predictive function as compared to the optimal weighting of the model based on real data. The results show improving predictive diagnostic accuracy for both groups over the intervention, although physicians were less likely to maintain optimal predictive weightings than students over the six-month period. I have previously argued on normative and epistemic grounds that the focus of teaching should be on fostering conceptual change, and that in the absence of such change, no meaningful knowledge representation has taken place. The implicit assumption is that a process of conceptual diagnosis is at least partially possible. This will be explored more fully later in the paper since it plays a role in the instructional model that will be proposed. The value of a teaching strategy based on conceptual analysis, however, even in the absence of completely accurate diagnostic process, is that it provides a forum in which the accuracy of competing student conceptual representations can be discussed and evaluated (Svensson and Hogfors; 1987). This discussion can serve as entrance point in which to introduce the content that will shape "better" representations, from which students can evaluate their own progress, and provide the reassurance that alternative representations are not only possible, but likely. It also allows conceptions carried by students to be tabled and helps foster the recognition that conceptual models are not right or wrong per se, but constructed to help make sense of the world. The value of competing conceptual models is based, not on some single assigned value, but rather on their ability to predict and explain real-world phenomena, which in turn generates motivation to resolve conflicts between prediction and real world outcomes. These insights that these resolutions provide have been suggested to underlie clinical learning (Slotnick, 1996). Svensson and Hogfors used this model to promote conceptual change in mechanics students by having them work in pairs to "confront" each other's conceptual understanding of a variety of natural phenomena, for example a hockey stick hitting a standing puck (the word confront has unfortunate pejorative connotations since the exercise was designed to limit value judgments; one student described, the other actively tried to understand). One important feature of the exercise was that it involved a series of stages in the process of illustrating each conceptual model. The first was an accurate description of what would happen without any attempt to explain; the second stage involved the formulation of a model that adequately and convincingly accounts for the described phenomena; the third stage clarified the conditions under which the model would be considered to be a valid representation; the fourth stage asked the student to pick a domain-specific problem (in this case mechanics) from a list of given problems for which the student model would provide a way in which to answer questions about that problem. The value of this exercise was centered on the fact that it forced a conscious formulation, articulation and evaluation of a conceptual framework in the absence of a formulaic approach to marking the result. The stages served as a way of gradually deepening the student's commitment to the process, since in the increasingly accurate description of the outcome of the phenomena (a deceptively easy task) the student was also increasing the phenomena that the conceptual model would have to account for. In evaluating the conditions under which the generated model would be valid, the student was consciously forced to confront the specific limitations of that model, and set the stage for an appreciation of models which elegantly explained the same phenomena under a range of circumstances. The last stage served to provide a real-world context in which both the limitations of the student's conception, and the need for a robust conception, would again be consciously evaluated. Svensson and Hogfors state that it is the process of "concientization" in which the educational value of this kind of exercise resides. What is learned is essentially metacognitive; an insight that meaningful understanding differs from dissociated declarative knowledge, and that the value of a conceptual model lies in its predictive value and the degree of its reliance on underlying first principles. The latter determines the range of circumstances over which a model's predictive and explanatory power holds. I would also suggest that this exercise exemplifies the second important feature of conceptual teaching, social discourse. In describing both the natural phenomena and the conceptual model, students were engaged in a critical and complex process of negotiating a common meaning. As was indicated in the chapter on constructivism, this discourse is an essential element of knowledge construction. Interestingly, Goss (1996) describes a clinical teaching model that emulates that proposed by Svensson and Hogfors. The model has a short initial clinical encounter, followed by a "briefing" session in which students collaboratively articulate their prior knowledge and conceptual models (Goss calls this the development of an organizational framework). The second stage is the bedside encounter in which students perform complete history taking and physical examination. This is followed by a thirty minute preparatory period, a debriefing session in which students receive feedback concerning the quality of interaction with the patient after which students return to the bedside and clinical skills are reviewed. The final stages are a write-up in which the reasoning process - initial history of presenting illness, positive and negative clinical features, exam and assessment - are explicitly addressed, and a follow-up meeting with the student in which the write-up is discussed. By invoking a system of active articulation of concepts, a solid clinical context, immediate feedback, and two forms of metacogntive analysis (the debriefing and write-up processes), Goss is utilizing a model that satisfies most of the principles already discussed. His observations provide very weak support for the value of this approach by suggesting that student clinical reasoning is richer and more sophisticated as a result. A less contextual version of a similar model has been proposed by Kinderman and Humphries (1995). Koritnik et. al. (1996) provides some stronger empirical support for many of the principles discussed above and in previous sections. Instructional interventions were designed within the pharmacology section of the curriculum with the specific intention of presenting content that would "nurture cognitive skills, foster long-term retention of information, and orient students to clinical considerations simultaneously" . The theoretical underpinnings of the intervention were identified as the activation of prior knowledge, elaborating knowledge, and the provision of context. The intervention, though not described in detail, combined small group tutorials and discussion groups, student case presentations, clinical visits, and the active searching for and integration of domain specific information from a variety of sources. The students' performance in the pharmacology section of Step 1 of the U.S. Medical Liscensure Exam showed significantly stronger performances than other faculty locations, and performances above those predicted by standard regression analysis. Weak anecdotal evidence also indicated that students were more motivated, better engaged and developed better communication skills during the course. Summary Instructional strategies arrived at deductively from first principles (theory-driven), and those indicated to be effective as a consequence of instructional practice, show some congruency. The general principle on which this congruency rests is essentially a conscious, active, and critical process of applying a clearly articulated conceptual model to an authentic context, and the evaluation of that model's ability to predict and account for current and future social/clinical outcomes. Some of the more specific components of this principle might be: choice of domain, the activation of prior knowledge within a domain, the active elicitation and articulation of conceptual models, a critical evaluation of the social and ethical assumptions and consequences of those models, the choice of an authentic context with a degree of protoypticality that is a reflection of the students level of expertise, a collaborative environment, the application of the conceptual model to the authentic context, critical reflection on the model's ability to account for observed outcomes, and changes to the conceptual model based on the active resolution of incongruencies between actual and predicted outcomes. Although a sequential order is implied, many of the components overlap during the instructional process. Evaluating Clinical Reasoning Elstien (1994) argues that "cognitive research has made the assessment of clinical competence more difficult in two ways". First, it has shown that low correlations exist across cases "on various measures of problem-solving ability" (i.e. a clinical skill is domain/knowledge dependent). Thus demonstrating general clinical competence would demand a greater number of clinical cases than have been commonly used by licensure bodies. Secondly, considerable variation exists among decisions reached even when the clinical database is held constant. This strongly suggests that there are complex social and organizational factors at work in any clinical decision making process. Elstien suggests that along with the social context of clinical practice, HOW the student represents the case is an important component of the decision process, and a much greater emphasis needs to be placed on assessing the clinicians problem representation and management plans as part of any assessment process. Within current medical education, multiple choice questions (MCQ's) have dominated as an assessment technique. According to Elstein and Page (1995), this dominance is reflection of their efficiency, cost-effectiveness, scrutability and their reflection of a perspective of medicine as a discipline based on scientific principles. MCQ's do, however, suffer from a number of limitations which a number of authors conclude make them less suitable for assessing areas involving less scientific processes, especially those involving the ability to weight and judge competing and difficult to characterize demands and in which little consensus exists about the "right" course of action (Elstein, 1994; Ramsden, 1989; Masters, 1989; Page, 1995). MCQ's do not distinguish between rote learning and purposeful rational problem solving. They "tap mainly recognition memory but the structure of knowledge is better revealed by free-recall tasks than by recognition tasks" (Elstein, 1994). Elstein further suggests that "medical students and house officers could be called upon to display some competence as systematic analysis of and reflection upon indeterminate problems - problems where no single "correct" answer exists - because that is part of clinical practice in a variety of specialties" (emphasis mine). This is perhaps closer to the skills that Schon (1987) has labeled "reflection in action". To overcome the limitations of MCQ's , Elstein (1994) suggests that the one best answer problem might be expanded to include weights assigned to two or more alternatives based on a panel of experts. Thus partial grading becomes possible and the differences between alternatives can be more subtly constructed. Also, context specific local assessment on issues involving the judgment of relevant competing demands should be included. The best that can be hoped for in areas of low or no consensus is that the "physician will approach the problem, the patient, and the family equipped with tools and 1 2 4 concepts that foster mindful, attentive and effective deliberation and reflection". Fostering and evaluating this "mindfulness" is a difficult but important issue. Driven by the perceived limits of MCQ's as an evaluative mechanism for clinical skill, a range of performance-based assessment techniques have been adopted by both teaching and liscensure bodies. Swanson () offers the following summary of the impact of these techniques and "the lessons learned" within health sciences. The four performance-based methods of evaluating clinical reasoning are: • patient management problems or PMP's - this usually involves the presentations of an opening scene (generally textually represented) from which a series of management options is offered. Based on the choices made by the testee, successive scenes are generated, some of which involve further data gathering, some which initiate patient management protocols; • computer based simulations - these are generally driven by expert rule based systems that dynamically model disease states and can operate in uncued fashion; • oral examinations • standardized patients. Swanson cites the following lessons to be learned from the application of performance-based evaluation techniques: • the added "realism" of these techniques does not make test design easy or straight forward. Results indicate that there is a complex interconnection between context and knowledge dimensions which must be factored into the design. It is also clear that skills are domain specific and no generalizations can be reliably made between domains and that the differences between experts and novices lie in the depth and representation of knowledge not in problem solving strategies; • there is a discrepancy between behaviour in a simulation and in real life. Some important contextual determinants of behaviour may be lost because of partialness of a simulation and differences in performance may be based on differences in perception of the intent of the simulation; • deriving a scoring method is a hugely problematic issue; • correlations with other forms of assessment are variable and uninterpretable; • there is a general lack of cost effectiveness (small sample, general unreliability); • neither traditional or performance based assessment is a panacea. Which evaluation format is used should be determined by the skills to be assessed and often a blend is desirable. Despite the cited shortcomings of these performance-based measures, there has been a significant effort to develop clinical simulations as a means of evaluating and teaching clinical reasoning. Increasingly, the focus has been on the development of computer based simulations, in large part due to the multi-functionality of the computer. It can simultaneously serve as content provider, record-keeper, and (depending on the sophistication of the system) respond dynamically to user behaviour. Out of concerns about both PMP's and MCQ's , Page and Bordage (1995) have proposed an evaluation system in which the critical key features of a clinical problem are targeted for the assessment of clinical decision making skills. The rationale for this approach stems from the assertion that competent clinical practitioners are able to rapidly assess a clinical problem by "the effective manipulation of those few elements of the problem that are critical to its successful resolution." Thus, traditional methods of assessing competence may actually penalize the "compiled" or "elaborated" knowledge of an expert physician since he/she, as the novice/expert distinction evidence would predict, is able to make a determination of the clinical problem with greater accuracy and far fewer cues than the non-expert. Specifically targeting these "few elements" is the essential component of a key feature approach to evaluation. The stages in developing problems utilizing a key feature approach are defining a domain for which the assessee is accountable, developing an assessment "blueprint" by which problems within the domain are selected, determining clinical contexts for each question, and developing a scoring key for each question. If well designed, these questions can have high face and content validity, and a relatively stable scoring profile even without expert analysis. The formats that were considered to provide better results were write-in answer and short menu presentation. The benefit of this approach were that, in contrast to PMP's, a large enough number of questions (forty is a suggested minimum) could be given to provide a general picture of clinical decision making within the targeted domains. Since the nature of expert knowledge can be reliably shown to be domain and case specific, PMP's cannot, without a huge investment in assessment time, provide that kind of generalizable result. A key feature, defined as a "critical step in the resolution of a problem", should also focus on steps "in which examinees are most likely to make errors in the resolution of the problem and (2) it is a difficult aspect of the identification and management of the problem in practice" (Page et. al., 1995). Since key features can be embedded in a variety of presentation formats, cueing artifacts (which may compromise an accurate portrait of the clinical reasoning process or which may favor weaker examinees) can be lessened. As well, a varied number and type of clinical complexities can be accommodated. Choosing an appropriate clinical context and delimiting its key features poses a significant challenge to the author, since the presentation of relevant data must be balanced against the cueing process that may be engendered. Scoring a key feature question can be accomplished in a variety of ways including a differential weighting of each feature, a dichotomous score in which both failing to list all key features and listing too many may serve as a "wrong" answer, and equal weighting. Page et. al. recommend that each problem be given a weighting of "1" in the overall evaluation score so that a total examination score is the sum of the scores for each problem. The research conducted into key feature questions have supported the contention that they are better indicators of clinical decision making skill than MCQ's , and provide a forum in which general domain competencies can be established, a result which could not be realized with PMP's (Dauphinee, 1995; Page and Bordage, 1995). Although the promise of simulations is to provide an authentic environment which is an accurate reflection of real practice, the evidence (as suggested below) only provides weak support for that contention. In the absence of readily administered simulation assessment procedures that do embody the essential elements of real practice, the key feature approach seems to offer a rich environment that better targets the critical delimiting factors distinguishing knowledgeable from unknowledgable students and practitioners. Simulations and Clinical Assessment Supporting Elstien (1994), Melnick (1994) suggests that traditional methods of testing do not portray psychiatric, behavioural, social problems effectively since the it is difficulty to represent the role of interpersonal communications in solving these problems. Simulations offer a way to include high fidelity representations of authentic environments in evaluation processes, although evidence supporting the validity of simulations as predictors of clinical performance is not clear (Fitzgerald et. al, 1994). Gonzales-Willis' () use of standardized patients in simulated situations to assess physicians' behaviour (in this case sexual risk) is one such example. The authors of the study suggested that this kind of simulated encounter provoked a good response from physicians and was judged a good way to elicit actual practice and provide feedback on the efficacy with which physicians dealt with such situations. No data were presented to corroborate this assertion. In support for the reliability of the diagnostic capacity of simulations, Clauser et. al. (1992), using clinical ratings to model score weights for case-based clinical simulation examination, found that simple regression-based efforts to model the policies used by clinical judges can provide a scoring system that closely predicts both the rating and pass/fail decisions associated with the scale based on those ratings. Thus reliable modeling of the judgment of clinical competence would seem to be possible. Roberts et. al. (1995) also demonstrated a correlation between simulation scores and observed ward performance within a population of nursing students, therefore demonstrating some limited validity for the simulated performance as a measure of clinical performance. Reliability within items measured by the simulation was low, further supporting the contention that clinical problem is a contextually bound skill. Although not clinically based, a result recommending some caution in the adoption of simulation evaluation approaches was found by Baxter () in comparing evaluation scores derived from both computer-based simulations and hands on assessment of science learning. The results indicated that mean scores were the same for each method, but student performance varied widely on both tasks and that little correlation could be found between the two. Thus there were clearly some differences between the two methods of evaluation. This seems to potentially conflict with Melnick's (1994) assertion that "neither prior computer experience nor anxiety about computers substantially alters performance with a simulated patient". Evidence supporting clinical simulations as a reliable evaluative strategy is limited, but seems to be enough to warrant supporting their continued development. The capacity for a simulation to capture some of the contextual factors influencing "real" decision making beyond that offered by MCQ's seems an important element worth the time and resources involved in their deployment. To the best of my knowledge, however, no normative prescriptive framework has yet been established to clearly define those factors that are essential in linking simulations to the "real world" counterparts from with they are derived. As has already been noted from the work of Elstein and others, some of the critical factors influencing "real world" clinical decision making need to be adequately modeled to make simulations more relevant to the context and demands of real practice. Patient Management Problems One form of an instructional strategy utilizing hypotheses generation is the use of patient management problems. These are problems in which the student is asked to make a series of clinical management decisions based on presenting symptomology. According to Elstein (1978), the use of patient management problems (PMPs) can be valuable if based on the following principles: feedback should be based on data from a sample of experienced physicians; process feedback in which a detailed description of what the expert physician's thought processes were is not helpful - outcome feedback alone is just as effective; an essential feature of problems is the selection of clinical procedures - providing feedback as to which factors should be considered key determinants at important decision points would be helpful. Elstein (1978) suggests that PMP's can be applicable as instructional processes in self instruction, group instruction in which students compare and critique their management, and evaluation of student's general theoretical and applied knowledge base, but that, as evaluation strategies, PMPs suffer from lack of reliability verification; for example some studies have indicated that how a user performs on a PMP is different from how they perform with a real patient or on higher fidelity clinical simulations (Roberts, 1995). As previously stated, however, the extent to which this is an artifact of the real social/contextual constraints of practice, and the lack of high fidelity media included in the PMP's used by these studies, is unclear. There is also general agreement that assessing general clinical competence demands testing over a range of PMP's, since performance on PMP's has been seen to be highly problem specific and poorly correlated across domains (Swanson, 19**). Framing bias - the tendency to base clinical decisions on the way that data is presented - exists among experts and may play a role in their differential responses to P M P ' s . . Thus a clinician may differentially decide to proceed with a surgical procedure depending on whether the outcome is framed as a loss (5 % morbidity) or gain (95 % chance of survival). In general it seems that psychologically, "losses loom larger than gains" (Elstein, 1994). Important to note from an instructional standpoint, however, is that Christensen et. al. (***), in examining framing bias among novices and experts, found that both novices and experts do not show framing bias. Only physicians with modest experience were susceptible to framing bias. They reasoned that novices did not possess the knowledge needed to weigh outcomes and thus frame the problem, and recommend that students be encouraged to look at clinical problems from multiple frames. Experts operated from a more stable set of habits that are less susceptible to alterations based on presentation manipulation. Summary The adoption of performance-based evaluative measures is a response to the perceived limits of MCQ's which have dominated the traditional assessment of medical knowledge. Although the argumentation for the adoption of performance-based measures is at least theoretically supported by the literature of cognitive science and situated cognition, implementing these measures should be tempered by understanding that clinical behaviour and behaviour in a simulation may not be well correlated. The evidence is still too limited to categorically deny or accept a relationship. But a distinction must be made between performance-based measures (specifically patient management problems) as purely evaluative procedures, and as instructional strategies. A patient management problem may be very useful when structured to promote reflective reasoning; in an instructional arena, a goal is set specifically relating a conscious evaluation of factors impinging on management decisions to patient outcome. I would like to postulate a model of patient management problem in which a cycle of active cognitive process can be established, and which uses both the key feature approach to evaluation (Page and Bordage, 1994) and attempts to make explicit the conceptual model carried by the student. This approach forms the core of the instructional model which will be described in detail in the section below. Chapter 12: A Model for Instruction Based on the literature reviewed above, it would seem appropriate to ask if a process or model, aimed at increasing the pedagogical efficacy of remotely delivered and accessed health-care materials, could be developed. I have tried, during my analysis of the cited literature, to develop common threads linking research into medical reasoning, cognitive science, situated cognition and assessment. These threads I have identified as the following: expertise is domain and context specific and depends on well-integrated (elaborated) knowledge bases and good prototypical representation; learning is idiosyncratically mediated by the learners' prior knowledge and cultural/contextual circumstances; teaching is the process of fostering "deep" and accurate conceptual representations of what must be learned, as well as critically reflective metacognitive strategies; therefore, teaching must be authentically situated and specifically target conceptual understanding; finally, assessment must reflect this mission and utilize contextually and conceptually relevant methodologies. If one accepts the above as a distillation of commonality between different research interests, then the criteria that an instructional system should embody can be stated as follows: • present content in a manner consistent with a content expert's representation and understanding of that content; • provide a forum in which theoretical and/or "required" content is presented or anchored within the context of a clinical environment. That context can be enhanced, though not necessarily, by support for graphical, video or sound media; • provide the end-user with "mentoring" or "query" support in which conceptual issues are both explicitly and implicitly addressed; • explicitly promote reflection during the decision making process. This reflection is encouraged through a structured series of exercises including key feature identification and analogous concept models; • promote cognitive feedback concerning the efficacy of the end-users decision making process; 132 • provide some form of user-modeling such that deficiencies in the users conceptual understanding of the specific domain are potentially diagnosed (and potentially remedied); • offer the possibility for collaborative work, either syncronysly (i.e. "at the moment" real-time communication) or asyncronysly in the form logged user comments and reactions; • support critical reflection on social/political/ethical issues; • provide an evaluative framework; • fall within the ability of most health sciences instructors to create and utilize. Although based on the literature, the above criteria should not be viewed as the sole prescriptive methodology for all health sciences content, since there are clearly pedagogical goals for which this type of approach is simply inappropriate. But it seems equally clear that strategies must be utilized, at least occasionally, which foster the kind of metacognitive and integrative reasoning that characterizes clinical practice. I will further suggest that the process, of structuring of these kinds of integrative instructional environments, since it demands an explicit identification and ordering of the conceptual issues to be addressed within the domain, fosters better instruction both within and without instructional technology. This begs the question of how a single design could incorporate the features listed above. One process (not new by any means) that seems to meet that goal is that of a domain-specific clinical decision tree. A "close-ended" patient management problem (PMP), the process presents an initial clinical context from which the user is asked to select a course of action from a given number of choices. Contingent on this choice a further series of management choices is given, and so on. With only a limited number of levels of depth, and three to four choices at each level, it offers the potential of creating what might be described as a non-trivial environment; an environment in which it would be difficult for the end-user to easily exhaust all possible outcomes. The fact that the literature does not strongly support a relationship between behaviour as expressed in a simulation or PMP and that occurring in 133 practice is not necessarily problematic. The model proposed in this section makes use of the PMP's utility in terms of instructional goals, not as an evaluative mechanism, although this will form a significant part of the model. The instructional model, however, does not exclusively rely on the PMP's implicit evaluative function but supports it with two other processes that will measure the congruency between the inferred user's reasoning and clinical management. If no congruency exists between these, the evaluative mechanism will reflect this in both its scoring of student performance and in its cognitive feedback to the student. Thus the PMP's strengths, the provision of an "authentic" context, and the "narrative" function of an unfolding clinical story, are retained. Other mechanisms will have to provide support to the evaluative function of the PMP, and may, at the same time, allow for metacognitive development, some limited user modeling, and conceptual diagnosis. I am proposing that three mechanisms may serve that goal: the utilization of global conceptual orientation statements, a modified key feature approach developed by Page and Bordage (1995), and an analogous model strategy. These will be described more fully later in this section. The Design Process Domain and Conceptual Issues To design the kind of environment described above is not a trivial task, but it can be approached systematically and parallels the "blue-print" process described by Page et. al. (1995). It begins with the choice of a domain for which the provision of a clinical context would serve an appropriate and integrative function. The domain should be reasonably well bounded but can involve psychosocial as well as more technical/causal processes. Within the example given in Appendix 1, the domain has been determined to be the management of a non-compliant asthma patient and the goal was identified as "increase patient compliance by changing her knowledge, behaviour, and attitudes about asthma". The choice of this domain was primarily to address general instances of non-compliance (i.e. the need to integrate psychosocial factors with therapeutic options to reach a successful outcome), but some of the contextual factors are clearly domain specific (i.e., therapeutic issues specific to asthma including the nature of inhaler use, impact of smoking). Both the general and contextually specific components of the domain are a realistic and frequent issue in the lives of general practitioners. Once a domain has been determined, an explicit analysis of the conceptual territory to be covered by the system is undertaken. This is a critical step in the design process and asks the developer(s) to identify the conceptual models and declarative knowledge needed to successfully resolve issues within the chosen domain. This assumes that the developer understands that what is sought is not information as discrete units, but models that provide causal relationships and, therefore, have predictive value. Several examples of these models can be found within Appendix 1: in dealing properly with asthma, the physician needs a detailed history before making any further treatment decisions; the frequent use of an inhaler, and the consequent demand for repeat prescriptions, may be an artifact of improper inhaler technique; getting the patient to demonstrate her technique will help to assess inhaler technique; confrontation with a hostile or angry patient may exacerbate non-compliance or drive the patient away completely; under conditions of time restraint, it may be necessary to rebook the patient for an appointment in which the physician can adequately explore the problem; this may mean actively taking a role in setting that appointment, especially in a patient with a history of non-compliance; smoking and allergies have a dramatic impact on the level of expression of asthma. Since the issues targeted within the domain in Appendix 1 are psychosocial, they are essentially "softer" than those that might be raised within a domain targeting the control of a highly specific disease. This means that the models given above are not "hard and fast" but depend completely on the context in which they occur. They are also open to critique which should be actively encouraged, since active critical reflection serves to integrate and make meaningful the knowledge targeted by the developer. The conceptual models identified can then be ranked from fundamental to precise, and listed or "mapped" hierarchically with the appropriate subsumation of concepts. This ranking process will help to map out the decision environment during the clinical context stage. A second critical feature of the identification of conceptual models within the chosen domain is the identification and explicit representations of typical or probable misconceptions carried by the target audience. These will serve to shape the clinical context into management strategies that form the basis of a conceptual diagnostic process. The next step is the identification of a few key features related to each of the accurate conceptual models, and features that are the expression of the inaccurate models. These will play an important role both in the provision of feedback as well as serve to corroborate or dispute the assignment of a conceptual model to the student. Before moving to the clinical expression of domain, the final decision that must be made is the level of prototypical representation that is appropriate for the audience and for the conceptual issues targeted by that domain. As the cognitive science literature suggests, clinical experience should begin with good prototypical representations of a particular process, in which the relevant features are well demonstrated as relevant, and the irrelevant features are shown to be irrelevant. This provides the student with a chance to develop a strong conceptual model of the process rather than having to cope with, and differentiate between, a large number of potentially confounding issues. Thus an atypical representation of a disease, for example, may be desirable given the conceptual issues identified for a sophisticated audience but hinder the development of good clinical models in the naive student. The prototypicality of the case could be represented by a simple index number between 1 and 3, and the student could be given the opportunity to self-select the level of representation. 136 Clinical Context The choice of clinical context is the visible expression of the domain and the conceptual issues identified as critical within that domain. As such, it should be authentic and yet still target those issues reasonably precisely. If such a tree is well constructed ( a very difficult design challenge), it can house some of the subtleties of practice and yet help the user develop a better understanding of the factors that must be considered to successfully manage this clinical situation. The goal is to foster an integration of new (and existing) declarative knowledge into the students conceptual framework, not simply add to whatever level of fractionation in domain knowledge is already in place. The initial clinical presentation is extremely important since it serves to - simultaneously - authentically situate, set-up the bounds of the domain and orient the student. The example in Appendix 1 uses "Michelle", a 22 year old woman with a history of smoking and a "problematic" relationship with her physician. She has arrived to receive her (too often) regular refill of her inhaler and is clearly agitated and irritable. The context in this case is (hopefully) the authentic expression of the domain and related conceptual issues. The way in which the context is presented forms the narrative; the story in which the student actively participates to determine outcomes. As such it demands careful attention to legitimacy, constancy, and validity of human representation within the multiple "plot" lines that form the story. The richness of demands by these considerations can, of course, likely be met by a solely textual implementation, but other forms of media may need to be included if the conceptual issues targeted involve specific visual signs, behaviours, and interactions that cannot adequately be represented by a purely textual description. The "value-added" nature of multimedia would have to be evaluated against the increase in both cost and development time. But there is a great deal of attractiveness of the utilisation of an unfolding clinical story, one in which the student's "actions" play an integral role in eventual outcome. Although there is some danger of any systematic approach to the development and use of computer-based instructional systems resulting in a formulaic response pattern on the part of the student, I think the argument of those within situated cognition, constructivism and feminism-that people are essentially social and understand their reality through socially mediated constructs - is quite compelling. If a story about a clinical problem speaks with some authenticity then students should care about the outcome, not as an intellectual challenge or abstraction, but as a representation of the impact of clinical decision making on those affected by it. If that is true, then committing to a particular course of action should have some meaning, meaning which should provide the motivation to "do one's best" within the problem. I realise that this is something of an idealist's stance, but I have no reason as yet to reject it. Management Strategies The context is constructed to "set-up" management decisions that (again) reflect the conceptual issues identified during earlier stages of the design process. One of these paths through the problem reflects the "correct" management strategy; the others represent the presence or absence of those known or postulated misconceptions that have been identified. Since the management decisions are bounded, this kind of approach represents a trade-off between the ideal state in which the student is free to choose from an infinite management strategies, and the utility of a system not dependent on an expert system knowledge base or artificial intelligence. It is also my contention that from an instructional standpoint, the fact that the choices are bounded can help to foster a critically reflective process. In the articulation of why one, or none, of the management options is appropriate, the learner must enter an integrative and metacognitive process. This can be facilitated by 138 providing a forum in which the student can enter comments during the decision making process; these can then be appended to those that have already been entered. During the feedback process these comments can form part of the metacognitive/reflective process, and may serve to provide at least some of the benefits of a collaborative environment. The final stage in the development of each management strategy is the assignment of a weight reflecting the strategies overall utility. Feedback As stated earlier, I am proposing that three mechanisms may serve to overcome the identified limitations of PMP's as evaluation mechanisms: the utilization of global conceptual orientation statements, a modified key feature approach, and an analogous model strategy. An example of this approach is given in Figure 3. The first of these, the global conceptual organizers, serve as metacogntive tools for the activation of prior knowledge (Arseneau and Rodenburg, in press). It is the presentation of a series of statements that help orient the user to relevant issues without explicitly identifying these. Within the example in Figure 3, these are simply enabling the user to reflect on discrete factors that may or may not have importance for the solution of the problem. These statements are part of what might be called a "virtual mentor", a non-judgmental help function that the user knows can be appealed to. These statements form the first metacognitive reflective process for the user. The second mechanism asks the end user to specifically identify what he/she considers to be the most salient features of the clinical presentation at that moment in decision making. The choices are then evaluated against a series of conceptual models that have been developed pre-hoc by the developer that are known, or postulated, to be plausible but misinformed constructs. If consistency between key feature choice and a conceptual model is apparent, then the system has some justification for weighting the 139 probability of the user carrying that model slightly higher. In Appendix 1, the relationship between each key feature and the conceptual model for which it has relevance is identified. The third mechanism asks the user to select, from a series of conceptual models, the one that "best" fits their own conceptual understanding of the causal factors operating in the clinical context. These models are analogous representations of the conceptual models used with the key feature approach. They would be couched in terms that are a general representation of the causal process, without an explicit identification of the critical factors. Again, based on the level of congruency between the key features identified and the model chosen, the system could develop a user model with a higher degree of probability based on Bayesian inferencing of the kind described by Boohan (1992). Finally the user is asked to select a clinical management decision. If the first two processes have been successful in promoting an active reflective process, then the choice of a clinical management strategy is the stage at which the user must integrate their conceptual model with a clinical expression of that model. This should promote the kind of self-reflective reasoning that helps the user to query the value of a conceptual model explicitly identified against the utility of a clinical decision. This seems to fit with the instructional strategies identified in both the cognitive science and medical instructional literature. The missing element of the model proposed so far is that of active collaboration. This could be encouraged by having students actively log questions, comments or notes in a file attached to each decision point. This file could be reviewed, and added to, by the student before committing to a course of action. (Other more esoteric strategies could include the development of a "real-time" communication infrastructure, such as a "chat" facility, or video conferencing.) A number of response "behaviours" of the system are possible during the user's interaction with the system. As clinical management options are picked, the evaluation of the associated weights could continue until some threshold level for the presence of a conceptual model is reached, in which case the system might be given the leeway to intervene and suggest to the user that addressing that conceptual issue is likely to be important. The user would be given the option at that point to exit the clinical context and enter a module directly targeting the remediation of that conceptual problem. The data on which that kind of decision would be made could include weightings that are adjusted to reflect all users to that point. This kind of immediate intervention is perhaps a little pre-emptive and, since it is dependent on the accuracy of the user model, may not be that helpful or desirable. The second, more preferable, response option would be simply to wait until the user had reached the end of the clinical management problem and reconstruct their decision making process, providing a number of levels of feedback to the user. The first feedback level would be an evaluative rating, immediately on the completion of the problem, of their efficacy in terms of clinical outcome, either as a statement, or as a score, or both. This could be based on two mechanisms depending on the PMP process utilized: for a branching process, the system could use an overall ranking of that solution compared to other possible outcomes (an absolute score) as well as a "within" choice point analysis based on the ranked efficacy of each management option across all choice points. Thus a user could conceivably end up with a ranked score of (arbitrarily) "poor" in terms of clinical outcome, but still show a "moderate" level of decision making efficacy based on within-choice-point efficacy rankings. This kind of result does not necessarily conflict since the clinical outcome score would be weighted more heavily than the within point decision efficacy score. The second level of feedback would entail a reconstruction of the user's decision making path, providing the following: clinical context, key features chosen, conceptual model chosen, congruency level between model and key features, the probability weighting that that model is actually being carried by the user, the "best" conceptual model, actual choice of management strategy, the optimal management strategy, and finally a rationale for why the "best" strategy is the "best" strategy. Although this seems like an overwhelmingly large amount of information to generate and assimilate, it can, I think, be presented in a manner that would be usefully presented and it would not tax an instructional system to track these kind of data. I suggest that this level of feedback provides a better approximation of what an actual teacher might provide than the other forms of evaluative feedback usually given in CBID, and meets many of the criteria suggested by the reviewed literature. It fosters active metacognitive reflection, provides for evaluation within an authentic context, targets conceptual development, and may form the basis from which a critical theoretical perspective may be brought to bear. A third level of feedback might be the inclusion of alternative perspectives for some of the choice points. Thus a student, having played the role of therapist during the PMP, could opt to see a representation of the same situation from the perspective of the patient, her family and other concerned stakeholders. This might help to foster a critical perspective in which the student realizes that the best course of management is not necessarily that which addresses the clinical problem alone, but one which attempts to reconcile the competing psychosocial, ethical, and resource allocation demands of the various players in the problem. The Conceptual Diagnostic Process The diagnostic process could utilize (and provide the user with) an explicit representation of the (mis)conceptions identified by the user's pathway through the PMP, and some probabilistic value that he/she was carrying that conception. These probabilistic values would be a reflection of the system's (beginning with the developer's) weighting of the relationship between a particular management strategy chosen and a particular misconception. (By asking students to estimate the validity of that value, or through a testing process, a sophisticated version of the system could then adjust these weights as users participated in the PMP. Eventually, it is conceivable that - providing some convergence is noted between conception and management strategy - more optimal weightings for the general target population could be achieved.) The questions, based on probabilistic determinants, about the relationship between the presence of key features, analogous models, management strategies and the likelihood of a conceptual model being carried are: "For every subset of n key features that are present that relate to a particular conceptual model, what is the probability that the related model is being carried?" "For every subset of n key features that do not relate to a particular model, what is the probability that the related model is being carried?" "If a related analogous conceptual model is picked, what is the probability that the related conceptual model is being carried?" "If a related analogous conceptual model is not picked, what is the probability that the related conceptual model is being carried?" "If a related management strategy is picked, what is the probability that the related conceptual model is picked?" "If a related management strategy is not picked, what is the probability that the related conceptual model is picked?" "If n key features are picked, and one of either or both of the related conceptual models are picked, what is the probability that the related conceptual model is being carried?" "If n key features are not picked, and if neither of the related conceptual models are not picked, what is the probability that the related conceptual model is being carried?" As stated earlier, for any given clinical context, the key features are assigned values that link them to the clinical context, the analogous models, and each management strategy; they are also ranked, by the developer, in order of importance for the appropriate management of the problem. To demonstrate how this structure might be utilized within an instructional model, let us assume that five conceptual models are chosen to be representative of a given domain. In the absence of any garnered student data, each of the models is given an initial probability weight of 0 .20, i.e., each are equally likely. One of the models is, of course, the "right" one as determined by the developer, the other four are specifically picked as those that are known or postulated to be frequent among students. The student, after reading over the clinical context and the conceptual organizers provided by the "mentor" is then asked to pick five key features from a list often generated by the system. The student must then pick one of the five representative analogous models, or, if rejecting all five, type their own model into the field provided. The student is then free to pick a management strategy. For the key feature selection specifically, the probability that all n features related to a particular conceptual model will be picked from a list of N presented features can be calculated to be: the number of possible permutations of n OR N! (Glass and Hopkins, 1984) the number of possible permutations of N (n!)(N - n)! In our example: 10! / (5!)(1 - 5)! = 0.004 (approximately 5 per 1000). We can, therefore, state (with some degree of assurance) that if all key features linked to a particular conceptual model have been selected, it is unlikely to be the result of an entirely chance process. But the probability of a chance selection of less than n key features increases providing the selection pool remains the same size. It is likely that a set of key features will be selected by the student that is composed of pointers to competing conceptual models, and will , therefore, have some degree of ambiguity in interpretive value. The problem is in allocating appropriate weights to the presence or absence of a key feature linked to a conceptual model and how to resolve pointers to competing models in some meaningful way (what, for example, is the presence of 3 features for conceptual model " A " worth relative to 2 features present for conceptual model "B"). An additional assignment must be weighted to account for both the choice of analogous model and management strategy. Again the problem will be in determining the total weighting given to a model in the presence of competing choices. For evaluative purposes, Page et. al. (1995) have suggested that within key feature selection, one method would be to give each feature equal weight (i.e. 1 / 5). From the standpoint of the proposed instructional model, the goal is somewhat similar (the assessment of knowledge), but it is complicated by the fact that the presence or absence of key features is also used as an indication for the presence of one of a number of competing conceptual models. From the data presented by Page et. al. (1995), it was apparent that a correlation coefficient of 0.80 in experts' agreement across key features could be generated for a given clinical problem. Let us assume that this, admittedly an ideal result, is the maximal probability level that can be achieved between the selection of all key features relevant to a particular conceptual model, the choice of the analogous model, the choice of the correlated management strategy, and the fact that that model is being carried by the student. Based on this value, a Beset derivation can be determined for probability of that model against the presence of 0 through n of the key features, and the presence of either or both the related analogues model and the related management strategy. This Baysean formulation is necessary since the calculations must be based on the arbitrary assignment (in the absence of real world data) of conditional probabilities to key features, analogous models and management strategies. (These conditional probabilities can be changed with the accrual of real-world data.) Following Boohan's (1992) work, each key feature can be assigned a "code" value which acts as an index function to a "look-up" matrix of probability weightings (assigned arbitrarily). The developer assigns a strength value of 1 to 3 to each key feature and links it specifically to one or more conceptual models. Each strength value has a probability weighting assigned to it which is an indication of the probability of the model being carried given that the feature is selected. In keeping with Boohan's (1992) work, this assignment can be made as follows: Strength Value: 3 2 1 Probability that Conceptual Model is Carried Given Key Feature is Selected: 4 3 3 0 5 2 Probability that Key Feature is Selected Given that Conceptual Model is Not Carried: 2 2 3 0 8 2 Probability that Conceptual Model is Carried Given Selection of Competing Key Feature: 2 0 2 8 3 2 Probability that Conceptual Model is Carried Given Selection of Analogous Model 4 5 4 2 3 6 Probability that Analogous Model is Selected Given that Conceptual Model is Not Carried: 2 0 2 5 2 8 Probability that Conceptual Model is Carried Given Selection of Competing Analogues Model 2 8 3 2 3 5 Probability that Conceptual Model is Carried Given Selection of Related Management Strategy 4 0 3 5 3 2 Probability that Related Management Strategy is Selected Given that Conceptual Model is Not Carried: 2 5 2 8 3 2 Probability that Conceptual Model is Carried Given Selection of Competing Related Management Strategy 3 2 3 5 3 8 NOTE: A l l values are arbitrary assignments. 147 To illustrate how Baysean inferencing might work, let us calculate the probability of conceptual model " A " (CMA) being carried given that only one key feature, key feature "1" is selected. Assume that key feature "1" (KF1) has been given a strength weighting of 3 (.40) for support for C M A . Following Bayes theorem: * P(A| KF1) = P(A| KF1) P(A) P(A| KF1) P(A) + P(KF1| not A) P(not A) where: KF1) = the probability of CMA being carried given that KF1 is selected P(KF11 A) = the probability of KF1 being selected given that CMA is being carried P(KF11 not A) = the probability of KF1 being selected given CMA is not being carried P(not A) = the probability of CMA not being carried (derived from 1 -current probability of CMA) P(A) = the current probability of CMA being carried *Glass and Hopkins (1984) & Boohan (1992) Remember that all conceptual models were initially set to be equally probable at 0.20. Let us also assume that the probability of KF1 being selected (again arbitrarily) given that C M A is not being carried is .20. The calculation for the new probability that C M A is being carried given that KF1 is selected can then be calculated to be: P(A|KF1) = (.40) (.20)/(.40) (.20)+ (.20) (.80) =0.33 This new probability can be carried forward if a second key feature (strength weight 3) is selected in the same manner: P(A| KF2) = (.40) (.33) / (.40) (.33) + (.20) (.67) = 0.49 On the selection of a third key feature (strength weigh 3) P(A|KF3) = (.40) (.49)/ (.40) (.49) + (.20) (.51) = 0..65 Therefore in the presence of 3 highly weighted key features, the probability of conceptual model A being carried is .65. (If each offered key feature is also given a "critical ranking" that is a reflection of its value within that specific context (KF1 has a rank of 7, KF3 a rank of 6 and so on...), this information can be utilized to provide summative evaluative information and/or feedback to the student.) If both the analogous model (AM) and related management strategy (MS) are selected (both at strength weight 3), the inferencing can be carried forward using the values within the look-up table above. P(A| A M ) (.45) (.65) / (.45) (.65) + (.20) (.35) 0.81 P(A| MS) (.40) (.81) / (.40) (.81) + (.25) (.19) 0.87 150 Therefore given the selection of all 3 key features, the related analogous model and related management strategy, the probability of conceptual model A being carried can be calculated to be .87, an attribution reasonably consistent with Page et. al.'s (1995) finding of a 0.80 correlation level for expert modeling based on key feature identification alone. The likelihood of such a strong attribution occurring is small since conflicts between key features, analogous models and management strategies will undoubtedly result. If the number of key features asked for is greater than 3, the impact of the presence of each key feature could be lessened (for example to .35 at strength level 3), so that the inferencing is independent of the number of key features. It is also arguable, however, that the greater the number of key features selected consistent with a conceptual model, the greater the likelihood of that conceptual model being carried. Using a similar process and the appropriate weights in the look-up table, the probability of each conceptual model being carried can be calculated during all stages of the decision making process. Although these calculations are completely based on arbitrarily assigned values, an instructional system could , by tracking student data (and assuming convergence of the data) over time, adjust these weightings to more accurately reflect real-world process. This could be done, for example, by correlating the outcomes with a pre-instructional M C Q test that is known, or postulated, to have some validity as a measure of domain knowledge and conceptual constructs. The value of this kind of inferencing process may be potentially realized at two levels: if it is shown to be somewhat accurate in terms of its diagnostic power (admittedly a huge if) then remediation decisions can be made based on the probabilities assigned to the conceptual models. The second level is the explicit representation of these probabilities to the student supporting the notion of "scrutability" (Csinger, 1996) which is, in essence, a form of metacognitive feedback. The student can be given the opportunity to assess the accuracy of the system's analysis and in doing so, is performing an integrative process in which he/she must actively represent the domain from a variety of perspectives. This "multiple" viewpoint is one which the literature cited in previous chapters has indicated is extremely important in developing good and meaningful knowledge constructs. It is also a way in which critical analysis can be embedded since each conceptual model can be evaluated for assumptions, biases, and limitations. Even in the absence of any valid relationship between conceptual models, key features, management strategies, and paths chosen during participation in the PMP (an outcome which is perfectly possible), the model outlined above still fulfills an important and theoretically grounded instructional mandate. That mandate is to provide an authentic contextual environment in which reflective, metacognitive processes are invoked to help bridge the gap between the student's understanding and the expert's. What can be achieved is the integration of new and existing declarative knowledge within a specific domain into a better conceptual framework: a framework which will enable students to approach clinical problems within that domain with more accuracy, confidence and sensitivity. Summary From the developer's standpoint, the stages of structuring the content within the framework of the proposed instructional framework are given in Figure 4, and can be listed as follows: • choose a bounded domain • define the conceptual issues to be covered within that domain • ma and hierarchically rank the conceptual issues • identify "best" conceptual model to account for domain • identify common misconceptions (known or postulated) • define the level of prototypicality for contextual representation • choose clinical context within which the domain is represented • define initial clinical presentation • identify key feature list in concert with best and alternate conceptual models rank and weight key feature list develop contextually appropriate analogous models in concert with best and alternate conceptual models weigh analogous model list develop management options consistent with best and alternate conceptual models rank and weight management options develop outcomes for each management option develop rationale for best management option consistent with conceptual model FIGURE 4: The Design Process as Flow chart Choice of Domain J itllhla DdraalnUenafled Refr-eJtralar.lMj D M M M Identlflctk'JJiui of Critical, IfriiturM t Presentation of C l i n i c a l Context lllliBll 1 ::: Accurate •tfoticfcpl'itfi!! M o d t l Lde&iltEleil J Development of Clinical Management <>ptit»TLS DeirelopiuHiI of Rational* lot Optimal Siru(egi) Linking :M»rt»j>?fttttll OjMliWU la Cvncwpliul Mode &l T 1 PreMntadatt, nf Prtstaiiiiltta (rf PraKntaltfctk of I M,ansg,fmem seratfRj J I C r e a t i o n o f Outcomes for E a c h Strategy Flow Line Informs 155 Interaction with the System: The Student's Perspective Used Individually The student's interaction with the system begins with the case presentation, which is usually a textual description and some form of media representation, perhaps a picture or short video clip. Having read and examined the media, the student may wish to consult the "mentor" through a click on some appropriately designated button. The mentor provides the student with the conceptual organizers specific to that point in the case, which again may entail text and a range of media (perhaps the professor or developer talking to the student directly in a video clip). The appeal to the mentor is optional but encouraged, since it serves to activate prior knowledge and orient the student to the problem presented in the case. The student must then select n from N key features presented having first clicked the appropriate button. The student may select less than n, but not more, thus forcing the student to prioritize and evaluate each key feature against the others presented. Having selected the key features, the student must then pick an appropriate analogous conceptual model from a list generated specifically for that point in the case after clicking the appropriate button. Having done this, the student is free to pick one of the given management strategies, but may not do so until the key feature and analogous model selection has been undertaken. Depending on the implementation within the system, the student could ask to see the probabilistic weights attached to each concept model whenever an action is undertaken. In this case, the models are identified only by number, but the student can see the impact that each step in the process has on the associated weights. In this way, if convergence clearly is taking place (one model is clearly gaining probabilistic strength) the student can reflect on what common themes or threads there might be within his/her decision making. 156 The student continues through the case until some terminal point in the decision process is reached. The student is given warning that a feedback cycle is beginning and, if not opting to quit at that point, will be brought to the feedback section of the system. Within this section, the student is given numerical representation of the following: outcome ranking, clinical management score, key feature selection score, analogous model score, and finally the complete representation of each conceptual model and the probabilistic value for the likelihood of each model being carried by the student. The student can then retrace their path through the decision process, in which each decision that they had made, each key feature selection list (and those selected), each analogous model list (and that selected), and each management strategy selected can be compared to those deemed optimal by the developer. The student may opt at any time to review the "correct" conceptual model and a rationale for why it is correct. Used Collaboratively Within the context of a class, students might be asked to split into groups and to undertake the management of a clinical case. At each stage of the decision making process, student groups would be required to reach consensus about the best course of action, key features and the best representative analogous models. Used this way the system could promote collaborative team work within a slightly competitive arena. Group outcomes would never be used for evaluative purposes (except for feedback to and from the rest of the class), but as a vehicle for the process of collaborative negotiation and some form of benchmarking for both the groups and class as a whole. Chapter 13: Conclusion As suggested at the beginning of this paper, it is a foregone conclusion that educational technology will become a ubiquitous part of students' lives at U B C and most educational institutions. The following was taken from the same paper, produced by UBC's Center for Educational Technology, as that quoted in the opening chapter (""). "Rationale for using educational technology In 2001, technology-based distributed learning is ubiquitous throughout the University. As a result, there is improved access for those who cannot visit the campus regularly. This has allowed more students to be served by the University without major expansion of its physical resources. There is also an improved environment for learning: Student access is more convenient and flexible Educational information is presented in enhanced and varied formats More instructional methods are available to help develop students' responsibility for their own learning. There are increased opportunities for discussion, debate, collaboration and critical thinking, and for the development of communication skills. Instruction is offered using similar technologies to those used by students for leisure and work purposes. As before, the University aims to teach as cost-effectively as possible. The fundamental justification for embracing electronic technology however is not that it provides the same quality of education at less cost; rather, technology is used to enable the university to educate more students to even higherstandards in an increasingly complex, technological society. " The document continues: "Objectives 3. Place a greater emphasis on a learner-centred approach to teaching a) help students develop more responsibility for their own learning but within a teaching environment that provides the necessary support. b) give students the opportunity to access a wide range of learning materials in a flexible manner c) promote the learning outcomes of team building, communication, problem solving, discussion of ethics, interdisciplinary approaches d) expand the experience of students by relating the theoretical to the practical through exposure to actual, mediated or simulated experiences using educational technology e) allow students the opportunity to discuss and challenge what they are learning and to develop the social aspects of education available through the use of technology (e-mail, discussion groups)" Just what constitutes a "learner-centered" approach to learning remains unclear, unless it means (as I surmise from the above) that the responsibility for teaching lies with the student instead of the institution. This rationale makes sense if U B C is striving to increase its share of a potentially world-wide educational market. Although I endorse the thrust of much of what falls under the last heading, I find it somewhat less than credible to believe that educational technology will accomplish what can only be accomplished with great difficulty even in small group meetings and tutorial sessions: session in which discussion is often fast, furious and impassioned. By comparison, email and discussion groups are, for truly collaborative learning, a tremendously poor subsitute . I think that much of what falls under the rubrick of student empowerment is actually a guise for the institution divesting itself of the responsibility to teach, as opposed to facilitating learning. There is great danger in an uncritical and uninformed adoption of methodologies that are not well tested or proven. Lest I sound too much of a naysayer, the nature of access can, to a great extent, be met by a distributed learning model, but this model can only function well if it is combined with a pedagogical vision that is theoretically well-grounded and well-supported by a full range of human-centered strategies. I hope that this paper serves to add a tiny piece to that enormous puzzle. Appendix A 161 162 Decision Tree for Asthma Module Designed by: Churlene Fell & Dirk Rodenburg Case Presentation Temp refill & newappimt I Dr. & receptionist Smoking Refill Sal Make apptmt Make apt mi on own together Take History Demo Inhaler Take History I Demo inhaler Refill Ventolin L Make apptmt on own Dr. <fe receptionist I Ventolin Followup 1 Smoking confrontation Beclameth & follow-up Make apttnt together Beclameth followup Epilogue Epilogue 163 164 You see Michelle, who is clearly irritated and says she will be late for work, Tour ours* has mformed you mat her last ref ill was 10 days ago, and you note from her chart that the tim« frame is not unusual. You remember that she is a "problem" patient, and usually resists the request to be seen in follow-up. She has been smoking for the past five years- What strategy below would likely have the most beneficial outcome? M i l Give her a temporary refill and reschedule for 1 week later. Obtain a more detailed history jit tJi^  risk of con.fronfcufcioxi > Ask Michelle to cat! back when you have time deal with her. 165 Select Use 3 most important features at this point In the case... Time contraints (physicians) Time constraints (Michelle s) Level of medication use History of smoking Michelle's general attitude Current state of asthma Current asthma control m You see Michelle, who is I clearly irritated and says sue j will be late Cor work. Tour informed you that •fill was 10days |ou note from her t&e'time:frame is hi. You remember Id usually resists t to be seen ta She has been ?t tfte past five tat strategy below ly have the most [outcome? Cancel ) | Done { t o t a l l e d h i s t o r y c o n f r o n t a t i o n . G i v e h e r a t e m p o r a r y r e f 111 a a d r e s c h e d u l e f o r 1 w e e k l a t e r A s k M i c h e l l e t o c a l l b a c k w h e n y o u h a v e t i m e d e a l w i t h h e r . 166 www » ww twBB« • man** Pkk *ne of the rfiu».t)«a To r4rk Jew » a BEST 1 tftt current Ths p*isrt: is inftated It viwki be better and more jirodudwe to ewe her« refil and wind her cr* her *<y than to « * a eertrwtfelori III! The revet** Safculamol use ntfeales inadequate li mwagemer* of the asthma The ptystaan must lake a roote detaled history before mMatt^ any further test men!. Grvr^ j her a temporary refW and rescheduling the j^iexsirtmert afcw for more lime to lake a Metaled Mjt«y. The hisfory is necessary lo ensure proper management of Michelea' ______________ , _____ ___... UhderihesecondHions, it's belterto tet Michelle decide +twn her asfhraa is too prowemaiic to deal effec&xty. She. call when iris's ready. 167 H I N T Bo to I Review| In this case, Michelle's level of medication use, and her control of are clearly problematic. By simply refilling her prescription, the physician is not fulfilling her responsibility. Consequently, this is not an Click the "Go to Review" button on the left to get some analysis of your decision making, or the -Quit- button to leave the program WIT 168 DECISION HISTORY Overall Clinical Management Score (Lewis belter) Overall Analogous Model Score f "A" ^ (Lower to better) Overall Key Feature Score (Lower ts better) Clinical Scenario L i s t (Select One) Serin} Michelle MwJt cation Refill Con Model Most L led MwkaJproblems can only bt raAavwj by U » pfoc-*t irtAdtMtbris_s. r«lw« d ttdnuristrttJHjn protocol. TI-K- paJienl rust '.rrisrsl«r»J v?isa a retire - and b* mad* (« comply W*fx'.< (his b « i r , r>o Kntt-*r«fl«-:ffi« IntrtmenS a Pl robahility of Model Abov« Being Carri 'Best1' Conceptual Model ; ; ,___™____™_-^ ^ »_ Meh«U»'s w t at SaJtMurctttaltfyshovs * M f*f *ah«rinn»ri»fl»rj»« ti out of OOfttrd. SUOOMIIUI - M J m t n t , h e w e w , « * » _ undtratBrKfcg the ami must h*ipr**««.»wvet^wit*H*. WWh** p*)*n0 *et*t*»n w t h « * j C.mi#-lo») factors |Wf JI'yl*t hiSU>.y, Wtti*3«S), s t_Hyt« r/*«l If s t i f le i_*yf,wrap*rfcd See List of Key Features AM j See LLst of Analogous Models M S I See List of Management Strategies 169 K*y Features Key Feature UM 170 References Adams, D. (1988) Extending the Educational Planning Discourse: Conceptual and Paradigmatic Explorations in Comparative Education Review. November Adams, D. (1991) Planning Models and Paradigms in Educational Planning: Concepts. Strategies and Practices. Robert V. Carlson and Gary Awkerman (Eds.), Longman, N Y Anderson, John R. (1985) Cognitive psychology and its implications. 2nd ed. W.H. Freeman, New York Arseneau, Richard; Rodenburg, Dirk (In Press) The Developmental Perspective. Five Perspectives in Adult Education by Daniel Pratt Baker, Eva L. (1995) Computer Technology Futures for the Improvement of Assessment. Journal of Science Education and Technology v4 (1), p37-45, March Balla, J (1989) Changing concepts in clinical education: the case for a theory. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Balla, J; Elstein, A; Christensen, C (1989) Learning clinical decision analysis. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Balla, J (1989) A model for learning in the clinical professions. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Barndt, D. (1992) Revisiting the boats and canoes. Popular education around the 500 years in Convergence. Vol. X X V , Number 1, Pg. 51 Baxter, G (1995) Using computer simulations to assess hands-on science learning. Journal of Science Education and Technology. Vol. 4, No. 1, p 21 Benyon, D; Murray, D (1993) Applying user modeling to human-computer interaction design. Artificial Intelligence Review, Vol. 7, p 199 Bersky, A; Yocom, C (1994) Computerized Clinical Simulation Testing: Its use for competence assessment in nursing. Nursing and Health Care, Vol. 15, No. 3, p 120 Biggs, J (1989) Institutional learning and the integration of knowledge. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Boohan, R. (1992) DIAG: A program to diagnose students' conceptual models in science. Journal of Computer Assisted Learning, Vol. 8, p 206 Bordage, Georges (1990) Which Medical Textbook to Read? Emphasizing Semantic Structures. Academic Medicine v65 (9) suppl pS23-24 Sep Bordage, Georges (1982) The cognitive structure of medical knowledge: Categories and prototypes. Dissertation Abstracts International Nov, Vol 43(5-A), pl472-1473 Bordage, Georges (1987) The curriculum: Overloaded and too general? Medical Education. May, Vol 21(3), 183-188 Bordage, Georges (1991) Semantic Structures and Diagnostic Thinking of Experts and Novices. Academic Medicine, v66 (9) suppl pS70-72 Sep Bordage, Georges (1994) Elaborated knowledge: A key to successful diagnostic thinking. Academic Medicine, Nov Vol 69(11) 883-885 Bordage, Georges (1995) Content Validation of Key Features on a National Examination of Clinical Decision-Making Skills. Academic Medicine. v70 (4), p276-81 Bordage, Georges; Lemieux, Madeleine (1991) Semantic structures and diagnostic thinking of experts and novices. Thirtieth Annual Conference: Research in medical education 1991, Washington, DC. Academic Medicine Sep, Vol 66 (9, Suppl), p70-72 Bowles, S.; Gintis, H . (1988) Schooling in Capitalist America: Educational Reform and the Contradictions of Economic Life. Basic Books Inc. N Y Brehmer, B; Dorner, D (1994) Experiments with comptuer-simulated microworlds: excaping both the narrow straits of thje laboratory and the deep blue sea of the field study. Briskin, L . (1989) Feminist pedagogy: teaching and learning Liberation Brooks, J. G.; Brooks, M . G. (1993) The case for constructivist classrooms. Alexandria, Virginia: Association for Supervision and curriculum development Brown, John Seely(1993). Practice at the Periphery: A Reply to Steven Tripp. Educational Technology. Oct, v34 (8), p9-l 1 Brown, John Seely; Duguid, Paul (1993). Stolen Knowledge. Educational Technology. Mar, v33 (3), plO-15 Burrell, G; Morgan, G (1979) Sociological Paradigms and Organizational Analysis. Heineman, Portsmouth Callan, E (1993) Noncognitivism and Autonomy. Paideusis, 7 (1) Cervero, R. M . ; Wilson, A. L . (undated) Integrating Agency and Stucture in Program Planning Theory Chinn, Clark A. (1993) Situated Actions during Reading Lessons: A Microanalysis of Oral Reading Error Episodes. American Educational Research Journal. Summer, v30 (2), p361-92 Chiodo, J; Flaim, M (1993) The link between computer simulations and social studies learning: debriefing. The Social Studies. May/June, p 119 Choi, J; Hannifin, M (1995) Situated Cognition and Learning Environments: Roles, Structures, and Implications for Design. Educational Technology Research and Development. Vol. 43, No. 2 Christensen, C; Elstein, A; Bernstein, L , Balla, J (1991) Formal Decision Supports in Medical Practice and Education. Teaching and Learning in Medicine. Vol 3, No. 2, p 62 Christensen, C; Heckerling, P; Macesy, M ; Bernstein, L; Elstein, A (1994) Framing bias among expert and novice physicians. Academic Medicine. August Clark, G; Koyano, K; Nivichanov, B. (1993) Case-based Learing for Orofacial Pain and Temporomandibular Disorders. Journal of Dental Education. Vol. 57, No. 11, p 814 Cognition and Technology Group at Vanderbilt (1991) Some thoughts about constructivism and instructional design. Educational Technology, Sept., pg 16-17 Cole, P. (1992) Constructivism Revisited: A search for common ground. Edcuational Technology, Feb., pg 27-33 Coles, C (1989) The role of context in elaborated learning. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Collard, S; Law, M ; The impact of critical social theory on adult education: a preliminary evaluation Collins, Allan (1994). Goal-Based Scenarios and the Problem of Situated Learning: A Commentary on Andersen Consulting's Design of Goal-Based Scenarios. Nov-Dec, Educational Technology, v34 (9), p30-32 Collins, Allan ; Cognitive Apprenticeship: Making Things Visible. American Educator: The Professional Journal of the American Federation of Teachers. v l5 (3), p 6 - l l , 38-46 Conzalez-Willis, A; Rafi, I; Boekeloo, B; Bowman, M ; Sardeson, K; Taggart, V. Burnett, R; Rabin, D (1990) Using Simulated Patients to Train Physicians in Sexual Risk Asessment and Risk Reduction. Academic Medicine: September Supplement, Vol. 65, No. 9, P S7 Csinger, Andrew ; Intent Based Authoring. Unpublished Doctoral Thesis, Department of Computing Science, University of British Columbia Cunningham, D. J. (1991) Assessing Constructions and constructing assessments: a dialogue. Educational Technology. May, pg 13 Dahlgren, L . (1984) Outcomes of Learning. In F. Marton, D. Hounsell, N . Entwistle (Eds.) The Challenge of Problem Based Learning (pp. 295-307) Damarin, Suzanne K. (1993) School and Situated Knowledge: Travel or Tourism? Educational Technology, Mar, v33 (3) p27-32 Damarin, Suzanne K. (1994) The Emacipatory Potential of Situated Learning. Educational Technology, Oct, v34 (8) pi6-22 Dawson, N ; Cebul, R (1990) Advances in quantitative techniques for making medical decisions. Evaluation and the Health Professions, Vol. 13, No. 1, p 37 De Bruijn, H. (1995) Cognitive Apprenticeship in a CAL-environment for Functionally Illiterate Adults. Instructional Science. Vol 23, P 221-241 174 Dear, B. (1991) Artificial intelligence techniques: applications for courseware development in Expert Systems and Intelligent Computer-Aided Instruction Dretske, Fred (1988) Explaining behavior : reasons in a world of causes. Cambridge, Mass. : MIT Press Duffy, T. M . ; Johanssen, D. H . (1991) Constructivism: New for instructional technology? Educational Technology. May, pg 7-9 Duffy, T. M . ; Johanssen, D. H . (1991) Constructivism: New Implications for instructional technology? Educational Technology. May, pg 7-9 Edmondson, K. M . (1994) Concept maps and the development of cases for problem-based learning. Academic Medicine. Vol 69, No. 2, Feb. Eizenberg, N (1988) Approaches to learning anatomy: developing a programme for preclinical medical students. In Improving learning: new perspectives . Paul Ramsden, Nichols Publishing Company, New York Elieson, W; Papa, F (1995) The Effects of Various Knowledge Formats on Diagnostic Peformance. Academic Medicine. Yearbook of Medical Informatics, p 595 Elstein, A; Holmes, M ; Ravitch, M ; Rovner, D; Holzman, G; Rothert, M (1983) Medical Decisions in perspective; applied research in cognitive psychology. Perspectives in Biology and Medicine, Vol 26, No. 3, p 486 Elstein, A; Shulman, L ; Sprafka, S (1990) Medical problem solving: a ten-year retrospective. Evaluation and the Health Professions. Vol. 13, No. 1, p5 Elstein, Arthur S. (1993) Beyond multiple-choice questions and essays: The need for a new way to assess clinical competence. Academic Medicine. Apr, Vol 68(4), p244-249 Elstein, Arthur S. (1994) What goes around comes around: Return of the hypothetico-deductive strategy. Teaching & Learning in Medicine. Spr, Vol 6(2), pl21-123 Elstein, A (1989) On the psychology of clinical decision making: what is rational expectation. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Elstein, Arthur S. et al (1982) Psychological approaches to medical decision making. American Behavioral Scientist. May-Jun, Vol 25(5), p557-584 Elstein, Arthur S. et al (1983) Medical decisions in perspective: Applied research in cognitive psychology. ; Perspectives in Biology & Medicine. Spr, Vol 26(3), p486-501 Elstein, Arthur S.; Shulman , Lee S.; Sprafka , Sarah A. (1987) Medical problem solving : an analysis of clinical reasoning. Engel, J; Wigton, R; LaDuca, A; Blacklow, R (1990) A social judgment theory perspective on clinical problem solving. Evaluation and the Health Professions. Vol. 13, No. 1, p 63 Entwistle, H . (1989) Ideologies in Adult Education in Lifelong Education for Adults: An International Handbook. C. Titmus (Ed.), Pergamon Press Inc., Elmsford, N Y Ernest, P. (1989) The Knowledge, Beliefs and Attitudes of the Mathematics Teacher: A Model. The Journal of Edcuation for Teaching. Vol. 15, No. l ,Pg . 13 Farmer, James A., Jr. (1992) Cognitive Apprenticeship: Implications for Continuing Professional Education. New Directions for Adult and Continuing Education. Fall, 55, p41-49 Farnham-Diggory, S (1995) Paradigms of Knowledge and Instruction. Educational Review. Vol 27 (3) Fitzgerald, J; Wolf, F; Wayne, D; Barclay (1994) A perliminary study of the impact of case specificity on computer-based assessment of medical student performance. Evaluation and the Health Professions. Vol. 17, No. 3, p307 Fontaine, D.; Le Beux, P; Riou, C; Jacquelinet, C (1995) An Intelligent Computer-Assisted Instruction System for Clinical Case Teaching. Academic Medicine. Yearbook of Medical Informatics Frye, A; Hollingsworth, M ; Wymer, A, Hinds, M (1996) Dimensions of feedback in clinical teaching: a descriptiive study. Academic Medicine. Vol. 71. No. 1, January Supplement, p S79 Gagne, Ellen D (1985) The cognitive psychology of school learning 176 Garrud, P; Chapman, R; Gordon, S; Herber, M (1993) Non-verbal Communication: Evaluation of a Computer-assisted learning Package. Medical Education. Vol. 27, P 474 Gathany, N ; Stehr-Green, J (1994) Putting Life into Computer-based Training: The Creation of an Epidemiologic Case Study. Educational Technology. August, P 44 Giroux, H (1985) Introduction to The Politics of Education by Pualo Friere, Bergin & Garvey Glasser, Michael; Patients' Views of the Medical Education Setting. Journal of Medical Education. Oct, v60 (10), p745-56 Goss, J. (1996) Teaching clinical reasoning to second-year medical students. Academic Medicine. Vol. 71, No. 4, April, p 349 Greenblatt, R; Kleinmuntz, B (1984) Statistical modeling of clinical intake decisions. Journal of Clinical Psychology. Vol. 40, No. 6, P 1528 Griffin, Marlynn M . (1995) You Can't Get There from Here: Situated Learning, Transfer, and Map Skills. Contemporary Educational Psychology. Jan, v20 (1), p65-87 Guess, R (1981) The idea of a critical theory: Habermas and the Frankfurt School. Cambridge University Press. Cambridge, London Hannafin, Michael J. (1994) Learning in Open-Ended Environments: Assumptions, Methods, and Implications. Educational Technology. Oct, v34 (8), p48-55 Hannafin, R; Freeman, D (1995) An Exploratory Study of Teacher's Views of Knowledge Acquisition. Educational Technology. January/Febuary, P 48 Hannafin, Robert D. (1995) An Exploratory Study of Teachers' Views of Knowledge Acquisition. Educational Technology. Jan-Feb, v35 (1), p49-56 Hannifin, M . J. (1992) Emerging technologies, ISD, and learning environments: critical perspectives. Educational Technology. Research and Development. Vol 40, No. 1, 49-63 Hassebrock, F; Prietula, M (1992) A protocol-based coding scheme for the analysis of medical reasoning. International Journal of Man-machine Studies. Vol 37, p 613 Hay, K (1993) Legitimate Peripheral Participation, Instructionism, and Constructivism: Whose Situation is it Anyway? Educational Technology. March, P 32 Hay, Kenneth E. (1994) The Three Activities of a Student: A Reply to Tripp. Educational Technology. Oct, v34 (8), p22-27 Hiebert, J.; Wearne, D.; Taber, S. (1991) Fourth graders' gradual construction of decimal fractions during instruction using different physical representations. The Elementary School Journal. Vol 91, n 4, pg 320 - 341 Hummel, Hans G. K. (1993) Distance Education and Situated Learning: Paradox or Partnership? Educational Technology, Dec, v33 (12), p i 1-22 Irby, D (1994) Three exemplary models of case-based teaching. Academic Medicine. 1994, December, p 947 Irby, David M . (1986) Clinical Teaching and the Clinical Teacher. Journal of Medical Education, Sep, v61 (9), part 2, p35-45 Irby, David M . (1992) How Attending Physicians Make Instructional Decisions When Conducting Teaching Rounds. Academic Medicine. Oct, v67 (10), p630-38 Johnson, J; Pratt, D (In Press) The Apprenticeship Perspective. Five Perspectives in Adult Education, edited by Daniel Pratt Jonassen, D. H . (1991) Evaluating Constructivist Learning. Educational Technology. September, pg 28 Jonassen, David (1995) Constructivism arid Computer-Mediated Communication in Distance Education. American Journal of Distance Education. v9 (2), p7-26 Jonassen, David H . (1993) Constructivist Uses of Expert Systems to Support Learning. Journal of Computer-Based Instruction, v20 (3), p86-94 Jonassen, David H . (1994) Learning with Media; Restructuring the Debate. Educational Technology, Research and Development, v42 (2), p31-39 Jonassen, David H. (1995) Supporting Communities of Learners with Technology: A Vision for Integrating Technology with Learning in Schools. Educational Technology. Jul-Aug, v35 (4) p60-63 Jones, Marshall G. (1995) Using Metacognitive Theories to Design User Interfaces for Computer-Based Learning. Educational Technology. v35 (4), pi2-22, Jul-Aug Jones, R; Latif, S; Murray, S; Hedly, A (1992) Simulation Models of Health Service Management: an example of continuing medical education using a package for daycase surgery. Simulation/Games for Learning. Vol. 22, No. 1, p 57 Kant, E (1983) Ethical Philosopy. Hackett Publishing Company Indianapolis, Indiana Kessels, J; Korthagenh, F (1996) The relationship between theory and practice: back to the classics. Educational Researcher. April Kinderman, P; Humphries, G (1995) Clinical communication skills teaching: the role of cogntive schemata. Medical Education, Vol. 29, p 436 Kinzie, M ; Strauss, R; Foss, J (1993) The effects of an interactive dissection simulation on the performance and achievement of high school biology students. Journal of Research in Science Teaching, Vol. 30, No. 8, p 989 Kopelman, L (1995) Philosophy and medical education. Academic Medicine. Vol. 70, No. 9, September, p 795 Koritnick, D, Ragatz, B; Ricklin, F; Deal, D (1996) Nurturing critical thinking skills with clinical applications in a medical pharmacology course. Teaching and Learning in Medicine. Vol. 8, No. 2, p 69 Kumar, D. (1995) Intelligent Educational Systems for Anchored Instruction. Techtrends. January/Febuary, P 33 Lackey, L. M . (1993) Gender Issues in Program Planning (unpublished paper for UBC) Laurillard, D (1989) Understanding Medical Students' Problem Solving. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Law, M ; Rubenson, K (1989) Andragogy: the return of the jedi. Paper to be presented to the combined SCUTREA/AERC7CASAE conference. Trans-Atlantic Dialogue, University of Leeds Lebow, D; Wager, W (1994) Authentic Activity as a Model for Appropriate Learning Activity: Implications for Emerging Instructional Technologies. Canadian Journal of Educational Communication. Vol 23, No. 3,Ps 231-244 Lebow, David (1993) Constructivist Values for Instructional Systems Design: Five Principles toward a New Mindset. Educational Technology. Research and Development. v41 (3), p4-16 Lemieux, Madeleine ; Bordage, Georges (1992) Propositional versus structural semantic analyses of medical diagnostic thinking. Cognitive Science. Apr - Jun, Vol 16(2), 185-204 Leshin, C ; Pollock, J.; Reigeluth, C. (1992) Instructional design strategies and tactics. Educational Technology Publications. Englewood Cliffs, New Jersey Lindstrom, B; Marton, F; Ottosson, T (1995) Computer Simulation as a tool for developing intuitive and conceptual understanding in mechanics. Maher, F (1987) Inquiry teaching and feminist pedagogy. Social Education, March 1987, 186- 191 Maher, F (1987) Toward a richer theory of feminist pedagogy: a comparison of "liberation" and "gender" models for teaching and learning. Journal of Education. Vol. 169, Number 3, 91 - 100 Malone, J.; Dekkers, J. (1984) The concept map as an aid to instruction in science and mathematics. School Science and Mathematics, Vol 84 (3), March Masters, G (1989) Imrpoving the assessment of clincal reasoning. Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Mattoon, J; Klein, J (1993) Controlling challenge in instructional simulation. Journal of Educational Research, Vol. 9, No. 2, p 219 Mayes, T. (1993) Impact of cognitive theory on the practice of courseware authoring. Journal of Computer Assisted Learning, Vol. 9, p 222 McGaghie, W, Boerger, R; McCrimmon, D, Ravitch, M (1995) Agreement among medical experts about the structure of concepts in pulmonary physiology. Academic Medicine, Yearbook in Medical Informatics, p 614 McLaren, P (1988) Critical Pedagogy: A look at the major concepts in Life in Schools. Longman, New York McLellan, H (1993) Evaluation in a Situated Learning Environment. Educational Technology, March, P 38 McLellan, Hilary (1994) Situated Learning: Continuing the Conversation. Educational Technology. v34 (8), p7-8, Oct McMahon, H. ; CNiel , B. ; Quinn, A ; Loughrey, D. (1992) "Open" Software Design: A case study. Educational Technology, February, pg 43 -55 McMahon, Harry (1992); "Open" Software Design: A Case Study. (In Educational Technology. v32 (2), p43-55, Feb Melnick, D (1990) Computer-based clinical simulation: state of the art. Evaluation and the Health Professions, Vol. 13, No. 1, p 104 Merrill, M . David (1991) Constructivism and Instructional Design. (In Educational Technology. v31 (5), p45-53, May Mezirow, J (1978) Perspective Transformation. Adult Education, Vol 27, No. 2, p 100 Moore, Joyce L. (1993) The Relationship between Situated Cognition and Anchored Instruction: A Response to Tripp. Educational Technology, v34 (8), p28-32, Oct Morrison, J; Murray, T. (1994) An Experiment in Problem-based Learning. Medical Education, Vol. 28, P 139 Mulligan, R; Wood, G (1993) A controlled evaluation of comptuer assisted training simulations in geriatric dentistry. Journal of Dental Education, Vol. 57, No. 1, p 16 Murphy, G; Friedmand, C (1996) Differences in knowledge representations of experienced and inexperienced clinicians a s captured by repertory grids. Academic Medicine. Vol. 71, No. 1, January Supplement, p sl6 Nespor, J. (1987) The Role of Beliefs in the Practice of Teaching in The Journal of Curriculum Studies. Vol. 19, No. 4, Pg 317-328 Njoo, M ; de Jong, T (1993) Exploratory learning with a computer simulation for control theory: leaning processes and instructional support. Journal of Research in Science Teaching. Vol. 30, No.8, p 821 Norman, G; Brooks, L ; Regehr, G; Marriott, M ; Shali, V (1996) Impact of feature interpretation on medical school diagnostic performance. Academic Medicine. Vol. 71, No. 1, January Supplement, p S108 Norman, Geoffrey R. (1990) Sources of Observer Variation in Dermatologic Diagnosis. Academic Medicine. v65 (9), suppl pS 19-20, Sep Novak, J. D. (1990) Concept Mapping: A useful tool for science education. Journal of Research in Science Teaching. Vol 9 Novak, Joseph D. (1990) Concept Maps and Vee Diagrams: Two Metacognitive Tools to Facilitate Meaningful Learning. Instructional Science. v l9 (1), p29-52 Ohlsson, S. (1993) Impact of Cognitive Theory on the Practice of Courseware Authoring. Journal of Computer Assisted Learning. v9 (4) p 194-221, Dec Okebukola, P. A. (1990) Can good concept mappers be good problem solvers in science? Journal of Research in Science Teaching. Vol 9 Okshevsky, W. C. (1992) Episemological and Hermenueutic Conceptions of the Nature of Understanding: The Cases of Paul H. Hirst and Martin Heidegger in Educational Theory. Vol. 23, N 1, Pg 169 Page, Gordon (1995) Developing Key-Feature Problems and Examinations to Assess Clinical Decision-Making Skills. Academic Medicine. v70 (3) p 194-201, Mar Pajares, M . F. (1992) Teacher's Beliefs and Educational Research: Cleaning up a Messy Construct in Review of Educational Research. Vol. 62, No. 3, Pg. 307-332 Pajares, M . F. (1992) Teacher's Beliefs and Educational Research: Cleaning up a Messy Construct in Review of Educational Research, Vol. 62, No. 3, Pg. 307-332 Papa, F; Shores, J; Meyer, S (1990) Effects of pattern matching, pattern discrimination, and experience in the development of diagnostic performance. Academic Medicine. Vol. 65, No. 9, September Supplement, p S21 Papa, F; Stone, R; Aldrich, D (1996) Further evidence of the relationship between case typicality and diagnostic performance: implications for medical education. Academic Medicine. Vol. 71, No. 1, January Supplement, p S10 Philip C. Candy ; Self-direction for lifelong learning : a comprehensive guide to theory and practice Pratt, Daniel (In Press) Five Perspectives in Adult Education Pugh, S (1993) Using Case Studies and Collaborative Computer-Assistend Communication to Support Conceptual Learning in a Teacher-education Course on Critical Reading. Educational Technology. November, 1993, P 30 Ramsden, P (1988) Improving Learning: new perspectives. Nichols Publishing Company, New York Ramsden, P (1989) What does it take to improve medical students' learning? Learning in Medical School. Edited by John Balla, Margaret Gilson, Anne Chang, Hong Kong University Press, Hong Kong Ramsden, P. (1992) Learning To Teach in Higher Education. New York: Routiledge Ramsden, Paul (1987) Improving Teaching and Learning in Higher Education: The Case for a Relational Perspective. Studies in Higher Education. v l2 (3), p275-86 Ramsden, Paul (1993) Theories of Learning and Teaching and the Practice of Excellence in Higher Education. Higher Education Research and Development. v l2 (1), p87-97 Rando, W; Menges, R (1991) How practice is shaped by personal theories. College Teaching: From Theory to Practice. Edited by Robert Menges, Marilla Sviniki. Jossey-Bass Inc. Records, Nancy L. ; Weiss, Amy L. (1990) Clinical Judgment: An Overview. Journal of Childhood Communication Disorders. v l3 (2), pl53-65 Rieber, L . P. (1991) Computer-based microworlds: a bridge between constructivism and direct instruction. Edcuational Technology, Research and Development. Vol. 40, No. 1, pg 93-106 Roberts, J; While, A; Fitzpatrich, J (1995) Clinical problem solving using video simulation: an investigation. Medical Education, Vol 29, p 347 Rowland, G (1993) Designing and instructional design. Educational Technology. Research and Development. Vol 41, No. 1, 79-91 Royer, J; Cisero, C; Carlo, M (1993) Techniques and Procedures for Assessing Cognitive Skills. Review of Educational Research, Vol. 63, No. 2, p 201 Schon, Donald A. (1987) Educating the reflective practitioner : toward a new design for teaching and learning in the professions. Schor, N ; Troen, P; Adler, S; Williams, J; Kanter, S; Mahling, D; Sorrows, B; Skogseid, I; Berneir, G (1995) Integrated case studies and medical decision making: a novel, computer-assisted bridge from the basic sciences to the clinics. Academic Medicine. Vol 70, No. 9, p 814 Schott, F. (1992) The contributions of cognitive science and edcuational technology to the advancement of instructional design theory. Educational Technology, Research and Development. Vol 40. No. 2. 55-57 Shedletsky, L (1993) Minding Computer-mediated Communication: C M C as Experiential Learning. Educational Technology. December, P 5 Shlechter, T; Bessemer, D; Kolosh, K (1992) Computer-based simulationsystems and role-playing: an effective combination for fostering conditional knowledge. Journal of Computer-Based Instruction. Vol. 19, No. 4, p 110 Slotnick, H (1996) How doctors learn: the role of clinical problems across the medical school-to-practice continuum. Academic Medicine, Vol. 71, No. 1, p 28 Sork, T. J.; Caffarella, R. S. (1989) Planning Programs for Adults in Handbook of Adult and Continuing Education. Sharan Merriam and Phyllis Cunningham (Eds.) Jossey-Bass, San Francisco 1 8 5 Spiro, R. J.; Fetlovich, P. I ; Jacobson, M . J.; Coulson, R. L . (1991) Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Educaitonal Technology. May, pg 25 -33 Starr, M . L . ; Krajcik, J. S. (1990) Concept maps as a heuristic for science curriculum development: Toward improvement in process and productivity. Journal of Research in Science Teaching, Stevens, Ronald H. (1989) Evaluating Preclinical Medical Students by Using Computer-Based Problem-Solving Examinations. Academic Medicine. v64 (11), p685-87, Nov Streibel, Michael J. (1993) Queries about Computer Education and Situated Critical Pedagogy. Educational Technology. v33 (3), p22-26, Mar Streibel, Michael J. (1994) Misattributions about Situated Learning. Educational Technology. v34 (8), pi4-16, Oct Svinicki, M (1991) Practical implications of cognitive theories in college teaching. College Teaching: From Theory to Practice. Edited by Robert Menges, Marilla Sviniki. Jossey-Bass Inc. Swanson, D.; Norman, G; Linn (1995) Performance-Based Assessment: Lessons from the Health Professions. Educational Researcher, June, p 5 Taylor, James C. (1994) Novex Analysis: A Cognitive Science Approach to Instructional Design. Educational Technology, v34 (5), p5-13, May-Jun Tennyson, R. D.; Elmore, R. L. ; Snyder, L . (1992) Advancements in instructional design theory: contextual module analysis and integrated instructional strategies. Educational Technology, and Development. Vol 40, No. 1, 9-22 The Cognition and Technology Group and Vanderbilt (1993) Anchored Instruction and Situated Cogntion Revisited. Educational Technology, March, P 53 Thurnam, R; Mattoon, J (1994) Virtual Reality: Toward fundamental improvements in simulation-based training. Educational Technology, October, P 57 Tomer, Y . ; Tamir, P. (1990) The relations between disciplinary and pedagogical knowledge and the length of teaching experience of biology and geography teachers. Journal of Research in Science Teaching. Vol 9 186 Tripp, S. (1993) Thories, Traditions and Situated Learning. Educational Technology. March, P 70 VeenMan, M ; Elshout, J (1994) Differential effects of instructional support on learning in simulation environments. Von Glaserfeld, Ernst (1991) Conceptual Models in Educational Research and Practice. Journal of Educational Thought; Revue de la Pensee. v25 (2), p91-103, Aug Von Glasserfeld, E; Steffe, L (1991) Conceptual Models in Educational Research and Practice. The Journal of Educational Thought. Vol , 25, No. 2,p91 Wallace, J. D.; Mintzes, J. J. (1990) The concpet map as a research tool: Exploring conceptual change in biology. Journal of Research in Science Teaching. Vol 9 Wandersee, J. H . (1990) Concept mapping and the cartography of cogntion. Journal of Research in Science Teaching. Vol 9 Weinstein, C; Meyer, D (1991) Cognitive learning strategies and college teaching. College Teaching: From Theory to Practice. Edited by Robert Menges, Marilla Sviniki. Jossey-Bass Inc. Whelan, G (1988) Improving Medical Students' Clinical Problem Solving. In Improving learning: new perspectives. Paul Ramsden, Nichols Publishing Company, New York White, James A. (1991) Effects of Three Levels of Corrective Feedback and Two Cognitive Levels of Tasks on Performance in Computer-Directed Mathematics Instruction. Journal of Computer-Based Instruction. v l8 (4), p 130-34, Fall White, R.; Gunstone, R. (1992) Probing Understanding. The Falmer Press, London, New York, Philadelphia Wiech, M . ; Waldie, K. ; Wong, M . ; Arseneau, R. (1993) A constructivist perspective of adult learning. Paper submitted for A D E D 518, University of British Columbia Wigton, Robert S. (1986) The Effect of Feedback in Learning Clinical Diagnosis. Journal of Medical Education. v61 n(10), p816-22, Oct 187 Wigton, Robert S. (1990) Teaching Old Dogs New Tricks: Using Cognitive Feedback to Improve Physicians' Diagnostic Judgments on Simulated Cases. Academic Medicine. v65 (9), suppl pS5-6, Sep Williams, R; Wortley, R; Cody, J (1992) Simulations as'real'work: the use of simulated patrol in police education. Simulation/Games for Learning. Vol. 22, No. 3, pl83 Wilson, Arthur L. (1993) The Promise of Situated Cognition. New Directions for Adult and Continuing Education. n57, p71-79, Spr Winn, W. (1990) Some implications of cogntive theory for instructional design. Instructional Science, Vol. 19, p 53 Winn, W. D. (1991) The assumptions of constructivism and instructional desgin. Educational Technology, September 1991, Pg 39 - 40 Winn, William (1993) Instructional Design and Situated Learning: Paradox or Partnership? Educational Technology.v33 (3), pi6-21, Mar Winn, William (1994) Why I Don't Want to Be an Expert Sitar Player. Educational Technology. v34 (8) p i 1-14, Oct Young, M (1993) Instructional Design for Situated Learning. Educational Technology Research and Development. Vol. 41, No. 1, P 43 Young, M . (1993) Assessment of Situated Learning Using Computer Environments. Journal of Science Education and Technology. Vol 4, No. 1,P89 Zelmer, A; Zelmer, A; Shannon, B (1992) Simulations in nursing education: a review and a forecast. Simulation/Games for Learning, Vol. 22, No. 1, p 49 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items