UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Toward an evidence-informed, theory-driven model for continuing medical education White, Marc I. 2003

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2003-792684.pdf [ 16.73MB ]
Metadata
JSON: 831-1.0055048.json
JSON-LD: 831-1.0055048-ld.json
RDF/XML (Pretty): 831-1.0055048-rdf.xml
RDF/JSON: 831-1.0055048-rdf.json
Turtle: 831-1.0055048-turtle.txt
N-Triples: 831-1.0055048-rdf-ntriples.txt
Original Record: 831-1.0055048-source.json
Full Text
831-1.0055048-fulltext.txt
Citation
831-1.0055048.ris

Full Text

TOWARD A N EVIDENCE-INFORMED, THEORY-DRIVEN M O D E L FOR CONTINUING M E D I C A L E D U C A T I O N  by M A R C I. W H I T E  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF T H E REQUIREMENTS FOR T H E D E G R E E OF DOCTOR OF PHILOSOPHY  in T H E F A C U L T Y O F G R A D U A T E STUDIES (Centre for the Study of Curriculum and Instruction) We accept this thesis as conforming to the required standard  T H E U N I V E R S I T Y O F BRITISH C O L U M B I A ;  January 2003  ;  © Marc Ira White 2003  In  presenting  degree freely  this  thesis  i n partial  fulfilment  at t h e U n i v e r s i t y  o f British  Columbia,  available  copying  o f this  department publication  or  f o r reference thesis  f o r scholarly  b y his o r  o f this  a n d study.  thesis  of t h e requirements I agree  I further  purposes  gain  that  t h e Library permission  m a y b e granted  h e r representatives.  f o r financial  agree  that  shall  It  is  for an  m a k e it  f o r extensive  b y the head  understood  n o t b e allowed  shall  advanced  that without  of my  copying  or  m y written  permission.  Depart meat—of  T h e U n i v e r s i t y o f British C o l u m b i a Vancouver, Canada  Date  DE-6  (2/88)  \J ^ ^ J O ^ j  'X  j&2r  /IA£-JW c/l©vA  Abstract TOWARD A N EVIDENCE-INFORMED, THEORY-DRIVEN MODEL FOR CONTINUING M E D I C A L E D U C A T I O N  This thesis develops the basis for an evidence-informed, theory-driven educational model for planning, implementing and evaluating continuing medical education (CME). Using an historical and conceptual analysis the author argued current C M E educational planning models, based on Tyler's Curriculum Model, failed to build a systematic body of knowledge to improve learning and teaching and are founded on historical, structural, organizational and pedagogical factors that arose from research and beliefs about learning prevalent at the turn of the twentieth century. Using a case study of a three-year province-wide, evidence-informed, multi-agency, comprehensive education program to enhance family and emergency physicians' knowledge and skills regarding the diagnosis and management of whiplash-associated disorders, the thesis demonstrates the feasibility and adaptability of using the PRECEDE-PROCEED Model for C M E . The PRECEDEPROCEED Model is a community-oriented and epidemiological-based educational planning model that provides a systematic approach to identifying and organizing contextual factors influencing knowledge uptake and knowledge utilization. The case study provides a basis for modifying the PRECEDE-PROCEED Model as a tool for planning, implementing and evaluating C M E programs. The changes are intended to assist C M E planners in integrating behavioural and non-behaviour factors with theory and best practices in the actual "curriculum" or intervention-program design. In addition, the proposed modification to the PRECEDE-PROCEED Model adheres to the standards established by the Joint Committee on Standards for Educational Evaluation as to what a  comprehensive evaluation should address. Based on the case study, the thesis recommends a modification to the current PRECEDE-PROCEED model of health promotion that provides a clearer conceptual understanding of the structure, components and theoretical underpinnings of an emerging evidence-informed, theory-based curriculum model.  Table of Contents Abstract Table of Contents List of Tables List of Figures Acknowledgments Permissions Chapter One Introduction Statement of the Problem System-based Failure in Planning, Implementation and Evaluation Need for a Theory-Informed Conceptual Framework to Improve C M E Research and Practice Introduction of the PRECEDE Model to CME Central Purpose Purpose of a Conceptual Analysis Significance of this work Limitations Author's Interest in CME Introduction of Case Study Summary Structure of Literature Review Chapter Two Historical, Organizational and Pedagogical Factors Influencing C M E Practice Overview of Internal and External Forces Shaping CME Historical Perspective on the Continuum of Medical Education Early Organizational, Administrative, Regulatory, Policy Factors Shaping the Continuum of Medical Education Government Research Funding Policies: A Primary Force Changing Medical Education Organizational, Regulatory, Economic, and Environmental and Regulatory Factors Professional Development of C M E Research Practices Begins in the 1980s Pedagogical Factors Shaping Undergraduate Medical Education and C M E Practices Learning Theories Underpinning the Continuum of Medical Education More Recent Attempts at Innovation in Medical Education Curriculum Renewal at Case Western Reserve University School of Medicine (CRWU) McMaster University School of Medicine Problem-Based Learning: Pedagogical Challenges and Controversies Differences in Novice and Expert Problem-Solving Methods PBL and Implications for C M E New Developments in the Cognitive Sciences and Learning Theory Neurocognitive Psychology  ii iv viii x xi xii 1 1 1 3 5 8 9 9 10 12 17 20 21 23 23 24 25 33 40 41 44 47 57 57 62 63 66 67 68 68  Cognitive Production Theory and Related Learning Theories A Model of Social Cognition Merging of Learning Theories, Diffusion Theory and Models of Knowledge Transfer Use of Diffusion Theory in the Medical Field Importance of Informal Communication with Peer Respected Colleagues Summary Chapter Three Current C M E Planning Practices in North America Protypical CME Planning Practices Phase One Planning C M E : The Needs Assessment Challenges to Current Planning Strategies: The Needs Assessment Process Phase Two Planning C M E : Stating Educational Objectives Challenges to Current Planning Strategies: Stating Educational Objectives Phase Three: Designing Educational Activities Challenges to Current Planning Strategies - Designing Educational Activities Attempts to Incorporate Adult Learning Principles in C M E to Inform Educational Design Phase Four: Evaluation Challenges to Current Planning Strategies - Evaluation Standards in Program Evaluation General System Failure in the Planning Literature: Linking Theory to Practice Planning as a Guiding Tool or a Point of Reflection Closed Planning Models versus Open System Models C M E Accreditation Bodies : Calls for a Change to Accreditation Practices Funding Support for C M E Confusion in Purpose and Role Among Different Types of Accredited C M E Providers Summary Chapter Four Utilizing the PRECEDE-PROCEED Model as an Analytical Tool for BCWI Supporting Theories and Models Health Belief Model Theory of Reasoned Action Theory of Planned Behaviour Principles Underpinning the PRECEDE-PROCEED Framework Phases of the PRECEDE-PROCEED Model PRECEDE-PROCEED Model, Phases 1 and 2: Social and Epidemiological Diagnosis in the Context of Health Promotion Education BCWI Phases 1 and 2: Incorporating Social and Epidemiological Diagnosis in C M E Planning BCWI Phase 3: Incorporating Behavioural Diagnosis in C M E Planning Non-Clincial Factors Discussed PRECEDE-PROCEDE Phase 4: Educational and Organizational Diagnosis in the Context of Health Promotion Education BCWI Phase 4: Incorporating Educational and Organizational Diagnosis in C M E Planning  70 77 79 82 86 88 90 90 91 94 96 97 98 99 108 110 Ill 112 115 117 118 119 120 122 123 125 127 128 129 131 132 132 136 136 138 142 147 148 154  Background to the Curriculum Development Process 154 Curriculum Development Process - Translating Knowledge into Curriculum 155 Informational Obj ectives 167 Competency Objectives 168 Attitudinal Objectives <. 169 Community Organization, Environmental and Training Objectives 173 PRECEDE-PROCEED Model, Phase 5: 178 Administrative and Policy Diagnosis in the Context of Health Promotion Education 178 BCWI Phase 5: Incorporating Administrative and Policy Diagnosis in C M E Planning 179 PRECEDE-PROCEED, Phase 6: Implementation Planning in the Context of Health Promotion Education 181 BCWI Phase 6: Utilizing Implementation Planning as a Tool for C M E 182 Phase 7 PRECEDE-PROCEED: Process Evaluation Planning in the Context of Health Promotion Education 183 Phase 7 BCWI: Reconceptualizing Process Evaluation in Planning C M E 186 Rationale and Instrument Development to Assess Program Delivered Versus Program Planned 196 Phase 8 PRECEDE-PROCEED: Impact Evaluation in the Context of Health Promotion Education 202 Phase 8 BCWI: Reconceptualizing Impact Evaluation for C M E Purposes 208 Phase 9 PRECEDE-PROCEED: Outcome Evaluation in the Context of Health Promotion Education 216 Phase 9 BCWI: Reconceptualizing Outcome Evaluation for C M E Purposes 217 Chapter Five The Development of a C M E Version of The PRECEDE-PROCEED Model... 220 Phase 5 The Program Logic of the Curriculum Development Phase 223 Relationship Between the Modified PRECEDE-PROCEED Model to Evaluation Standards 226 Utility Standards 227 Feasibility Standards 232 Propriety Standards 234 Accuracy Standards 235 Other Program Evaluation Standards to Consider 237 Structural Challenges to Implementing the Modified PRECEDE-PROCEED Model 240 Specific Barriers to Implementing PRECEDE-PROCEED Model for CME Purposes 244 Encouraging Government and Foundation Investment 250 Chapter Six Challenges and Recommendations for Reconceputalizing C M E 254 Postscript 263 Bibliography 265 Appendices 280 Appendix I: Administrative and Pre-implementation Documents 281 BCWI Committee Members 281 Research Committee Terms of Reference 282 Consent Letter 283 Background to Research Activities (Speaker Handout) 285 Educational Influential Survey 288  Appendix II: Curriculum Construction Instruments Program Logic Model BCWI Content and Procedural Objectives W A D Module I and II Editorial Review Guidelines to Improve Test Construction Objective Test Item Review Data Linkage Framework Appendix III: Program Evaluation Instruments Implementation Checklist Demographic Form Memos To Myself. Interview Guide: Following Initial Programs Knowledge Test Speaker Evaluation Form Speaker Feedback Form Program Evaluation Form Post-CME Exposure Survey Diffusion Survey Program Evaluation Standards Checklist Appendix IV: Sample Materials Sample Prototypical Program Evaluation Form for Delegates Sample Speaker Feedback Form BCWI Features Instructional Design Neck Talk Brochure: Your Guide to Whiplash Recovery Neck Specific Exercises  290 290 291 291 292 294 296 297 297 298 299 300 301 302 303 304 306 308 309 310 310 311 312 313 317  vii  List of Tables Number.  Page  Table 1. Historical Factors Contributing to the Development of the Modern Medical Curriculum... 40 Table 2. Broad Education Objectives for the Medical Education at (Case) Western University 59 Table 3. Founding Educational Objectives of McMaster University Medical School (54) ' 63 Table 4. Anderson's Production Theory for Knowledge Representation (42) " 72 Table 5. Theoretical Assumptions: Development of Procedural Knowledge 73 Table 6. Structure of Medical Knowledge (103) .73 Table 7. Conceptual Links Proposed by Baldwin and Ford on Knowledge Transfer and Utilization (115) 81 Table 8. Factors Affecting Adoption of Innovation (117) 84 Table 9. Factors Influencing Successful Dissemination 86 Table 10. Summary of Factors Impacting Behaviour Change in the Medical Community (6) " . 86 Table 11. Four Principles of Good C M E and Adult Education (136) 91 Table 12. Accreditation Principles: Needs Assessment Process 92 Table 13. Methods of Data Collection Techniques for Needs Assessment (138) ~ 93 Table 14. Principles of C A C M E and A A C M E : Educational Objectives 96 Table 15. Principles of C A C M E and A A C M E : Learning Objectives 98 Table 16. Gagne's Biological-Cognitive Model of Instructional Design 100 Table 17 Comparison of Typologies of Teacher Knowledge Domains 103 Table 18. Designing Instructional Text: Evidence-Informed Style Guides (155) 105 Table 19. Aspects of Everyday Cognition 117 Table 20. Principles Identified as Indicators of Successful A H E C Programs (174) 121 Table 21. Breakdown of Accredited C M E Providers by Type and Total Income (175) 123 Table 22. Summary of Selected Principles of Health Education Applicable for C M E (179) 133 Table 23. Commonly Espoused Assumptions of Adult and Physician Learners 135 Table 24. Phases 1 and 2 Social and Epidemiological Diagnosis: BCWI Putting Principles into Practice 138 Table 25. Phase 3 Behavioural Diagnosis: BCWI Putting Principles into Practice 143 Table 26. Subclassification of Educational Objectives for Health Promotion Modified for C M E Purposes (179) " " 149 Table 27. Phase 4 Educational and Organizational Diagnosis: BCWI Putting Principles into Practice 156 Table 28. BCWI Initial Statement of Overall Program Objectives 157 Table 29. Primary and Secondary Interventions in Continuing Medical Education ( 6 ) ' 160 Table 30. Educational Vehicles Original and Modified Roles and Responsibilities (202) 162 Table 31. Summary of Predisposing, Enabling and Reinforcing Factors Believed to Facilitate or Inhibit Knowledge Transfer and Utilization 164 Table 32. Community Organization, Environmental and Community Training Objectives 173 Table 33. New BCWI Community-based Objectives 175 Table 34. Important Assumptions Underlying Common Statistical Tests (192) 206 pp  p  23  p p 35  p 4  pp46  p p  8  9 5 8  pp  106  62-3  7  p  248  6  Table 35. Relationship of the Modified PRECEDE-PROCEED Model Phases to Utility Standards (166) 228 Table 36. Relationship of the Modified PRECEDE-PROCEED Model Phases to Feasibility Standards (166) 233 Table 37. Relationship of the Modified PRECEDE-PROCEED Model to Propriety Standards (166) 234 Table 38. Relationship of the Modified PRECEDE-PROCEED Model to Accuracy Standards (166) 236 Table 39. Proposed New Planning and Evaluation Standards Arising from the PRECEDEPROCEED Model 238  ix  List of Figures Number.  Page  Figure 1. Prototypical Black Box Model (24)  5  p 7  Figure 2. Internal and External Forces Influencing C M E (6) ' 23 Figure 3. Economic Burden of Disease in Canada: Distribution of Direct and Indirect Costs by Diagnostic Category (68) ' 37 Figure 4. Research Share of Total Cost by Diagnostic Category (68) ' 37 Figure 5. Estimates of Illness in the Community and Types of Service Providers Sought (72) P38 p  p  4  1 2  p  13  6 5 9  Figure 6. A Model of the Transfer Process (115) 80 Figure 7. A Model of Stages in the Innovation-Decision Process (117) 83 Figure 8. The Health Belief Model (178) ' 130 Figure 9. Phases of the PRECEDE-PROCEED Model (30) 136 Figure 10. Relationship of Educational, Behavioural and Health Status Objectives in Planning Community Health Program (179) 150 p 6 5  p  10  p 3 5  p  1 0 7  Figure 11. Three Component Program (192) ' p  Figure Figure Figure Figure Figure  151  4 5 9  12. Intervention Mapping Process Proposed for Health Promotion Planning (193) 13. Actions Taken to Mitigate Threats to Internal Validity 14. Action Theory and Conceptual Theory: Intervention Evaluation Model (22) ' 15. Hypothetical Program to Reduce the Risk of Cardiovascular Disease 16. Mediators and Moderators (192) ' p  p  471  152 197 .. 203 204 205  pl0  200  Figure 17. Program Theory Development (192) Figure 18. Repeating Pretest-Posttest Design Figure 19. Program Component: Use of Pretest as an Advance Organizer Figure 20. Use of Group Discussion to Increase Motivation and Attentiveness Figure 21. BCWI Program Components Linked to Mediators Figure 22. Action Theory and Conceptual Theory: Linking a Program Component to a More Distal Outcome Figure 23. Moderators Considered for the Purpose of Knowledge Uptake Figure 24. Proposed Modification of the PRECEDE-PROCEED Model Figure 25. Program Logic Underlying Modification of the PRECEDE-PROCEED Model Figure 26. Constructs of Knowledge Uptake and Knowledge Utilization Figure 27. Linking Physician Behaviour Change to Patient Outcomes p-472  207 209 212 213 214 215 215 221 222 225 246  x  Acknowledgments The author wishes to express sincere appreciation to the following: Professors John Willinsky, Sam Sheps and David Lirenman for their guidance and constructive criticism as members of my thesis committee; Dr. Judith Ottoson for introducing the subject of theory-driven evaluations and providing practical guidance which informed the development of the proposed modification of the PRECEDE-PROCEED model for planning and evaluating continuing education programs in the health professions; Professor Tom Sork for introducing me to the strengths and weaknesses of rational program planning. In addition, I would like to thank members of the British Columbia Whiplash Initiative (BCWI) Steering and Research Committees for supporting me as Chair of the BCWI Research Committee. Research Committee member mentors included Dr. Stefan Grzybowski, Director of Research, Department of Family Practice, University of British Columbia (UBC) and Marc Broudo, Associate Director, Division of Educational Support and Development, Faculty of Medicine, U B C . Other notes of thanks to Daniela Pacheva and her husband George Pachev for statistical assistance and guidance, Dr. Stephen Bath for his review of evaluation instruments, Mel Tobias for logistical assistance in document retrieval and cataloguing, Natalie Jacques and the late Lloyd Margetish for their attention to detail in data entry and their sense of humour, Sheilina Dhanani as a research assistant, Joey Schibild, my wife, for her review of the manuscript and her emotional support throughout my graduate studies, Norman and Barbara White for their assistance in document retrieval and parental support, Santiago Toro Posado and Michael Marrapese for graphic design and assistance. Partial funding for BCWI research was provided by the Insurance Corporation of British Columbia (ICBC), the British Columbia Medical Services Research Foundation and The  (ICBC), the British Columbia Medical Services Research Foundation and The Woodbridge Group. M y employer, the Physical Medicine Research Foundation, provided me with a five month paid sabbatical and additional leave of absence to support my graduate studies and preparation of this manuscript. All funding was provided under the strict sponsorship guidelines of the Canadian Medical Association and the sponsorship policy of the Accreditation Council for Continuing Medical Education (ACCME).  Permissions Figure 2. Reprinted from Davis DA, Fox RD, editors. The Physician as Learner: Linking Research to Practice. Chicago, IL: American Medical Association, 1994. © American Medical Association Figure 3. and Figure 4. Moore R, Mao Y, Zhang J, Clarke K. Economic Burden of Illness in Canada. 1993. Ottawa, Environmental Risk Assessment and Case Surveillance Division, Cancer Bureau, Laboratory Centre for Disease Control, Health Protection Branch, Health Canada. © Minister of Public Works and Government Services Canada 2002 Figure 5. Reprinted from White KL, Williams TF, Greenberg BG. The Ecology of Medical Care. New Eng J Med 1961; 265:885-892. © Massachussets Medical Society 2003 Figure 6. ReprintedfromBaldwin TT, Ford JK. Transfer of Training: A Review and Directions for Future Research. Personnel Psychology, Inc 1988; 41:63-105.© Personnel Psychology Figure 7. Reprinted from Rogers EM. DIFFUSION OF INNOVATIONS, Fourth Edition (Figure 5-2, p 163). Copyright (c) 1995 by Everett M. Rogers. Copyright (c) 1962, 1971, 1983 by The Free Press. Used with the permission of The Free Press, a Division of Simon & Schuster Trade Publishing Group. Allrightsreserved. (This permission does not extend to posting the Simon & Schuster content online on the UBC ETD, part of the Networked Digital Library of Theses and Dissertations) Figure 8. Reprinted from Clark NM, Becker MH. Theoretical Models and Strategies for Improving Adherence and Disease Management. In: Shumaker SA, Schron EB, Ockene JK, McBee WL, editors. The Handbook of Health Behaviour Change. New York, NY: Springer Publishing Company, 1998: 5-32. © Springer Pubishing Company (Health Belief Model - Public Domain) Figure 9. Reprinted from Green LW, Kreuter MW. Health promotion planning: an educational and environmental approach. Mountain View, CA: Mayfield, 1991.© McGraw-Hill Companies Figure 10. Reprinted from Green LW, Ottoson JM. Community Health. 7th ed. Mosby-Year Book, Inc., 1994. © Mosby Figure 11. Reprinted from Donaldson SI. Mediator and Moderator Analysis in Program Development. In: Sussman S, editor. Handbook of Program Development for Health behaviour Research and Practice. Thousand Oaks, CA: Sage Publications, Inc., 2001: 470-496 © Sage Publications, Inc. Figure 12. Reprinted from Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Intervention Mapping: Designing theory-driven and evidence-based health promotion programs. Mountain View,CA: Mayfield Publishing Co., 2001.© McGraw-Hill Companies Figure 14. Reprinted from Chen H. Theory-Driven Evaluations. Newbury, CA: Sage Publications, 1990 © Sage Publications, Inc Figure 17. Reprinted from Donaldson SI. Mediator and Moderator Analysis in Program Development. In: Sussman S, editor. Handbook of Program Development for Health behaviour Research and Practice. Thousand Oaks, CA: Sage Publications, Inc., 2001: 470-496. © Sage Publications, Inc. Table 13. AdaptedfromMoore Jr. DE, Cordes DL. Needs Assessment. In: Rosof AB, Felch WC, editors. Continuing Medical Education: A Primer. New York, NY: Praeger Publishers, 1992: 42-51. © Greenwood Publishing Group, Inc. Table 18. AdaptedfromHartley J. Designing Instructional Text. Sterling, U.K.: Stylus-Publishing, 1994. © Kogan Page Publishing Table 21. Reprinted from ACCME. ACCME Annual Report Data 2001. 1-10. 2002. © Accreditation Council for Continuing Medical Education (public domain). Table 29. Reprinted from Davis DA, Fox RD, editors. The Physician as Learner: Linking Research to Practice. Chicago, IL: American Medical Association, 1994. © American Medical Association Appendix IV. (Thesis pp. 313-317) Reprinted from BC Whiplash Initiative 1996 © Physical Medicine Research Foundation Appendix II. (Thesis p. 292) Reprinted from Haladyna TM. Developing and Validating Multiple-Choice Test Items. Hillsdale, NJ: Lawrence Erlbaum Associates, 1994. © Lawrence Erlbaum Associates Appendix IV. (Thesis pp. 310-11) Reprinted from Green JS. Evaluation. In: Rosof AB, Felch WC, editors. Continuing Medical Education: A Primer. New York, NY: Praeger Publishers, 1992: 70-88. © Greenwood Publishing Group, Inc  xii  Chapter One  Introduction Statement of the Problem During the past forty years the continuing medical education (CME) community has been continually challenged to be more effective, accountable and responsive to an expanding group of stakeholders concerned with health care issues. (1-5) These new stakeholders include all levels of government, allied health professionals, health policy decision-makers, business, organized labour, consumers, lawyers, international C M E , undergraduate and graduate medical educators, C M E researchers, and professional education specialists. (6;7) The C M E leadership is increasingly under pressure from within and outside the profession for C M E to: (a) improve the quality and cost-effectiveness of health care (8), (b) address priority health issues informed by current population health data rather than just those needs identified by physicians (3-5;8;9), (c) reduce medical error and physician incompetence (6;7), (d) encourage physician, allied health professional and other stakeholder participation in C M E planning (4;6), (e) improve the relevance of programs offered (3), (f) improve pedagogical practices and provide a greater variety of proven efficacious education interventions (4) ' " pp  10  11  (6) ( l ) ' p  212  , (g) create a more  comprehensive 'continuing' education program, instead of the current practice of 'episodic instruction' (9) and, (h) conduct more intervention and qualitative research to identify factors that enhance or hinder behaviour change. (10; 11)  System-based Failure in Planning, Implementation and Evaluation Critics of current educational practices in medicine, including leaders in the C M E field, have argued that traditional ways of planning, delivering and evaluating C M E  programs or activities have failed to provide sufficient information to build a coherent body of knowledge to improve the efficacy of C M E . Most C M E programs in the field are not systematically evaluated, and until recently, most C M E providers primarily used selfreport satisfaction evaluations, variations on the 'happiness index', providing feedback on the participants' level of satisfaction of the facilities, audio-visual, speaker presentations and program management. (4;5;12;13) Changes to accreditation policies in the late 1980s and early 1990s have resulted in C M E providers collecting additional evaluation information, such as asking attendees to evaluate whether the C M E session or activity fulfilled explicit learning objectives. However, there is little research demonstrating that these program evaluation changes have enhanced learning, led to program improvement, or informed C M E providers how to plan, implement and evaluate more effective C M E resulting in changes in clinical practice behaviour and improvements in patient health outcomes. In the 1970s the C M E debate focused on whether C M E programs or activities were effective in improving the competence and performance of physicians and whether they resulted in improvements of health outcomes for society. (5;8;14) Unfortunately the development and implementation of systematic, well-designed research is relatively new in the field of C M E . In a comprehensive review of 238 evaluation studies between 1935 and 1982, Davis, Haynes et al. (15) found 71% of the studies were inadequate descriptive or before-after designs, most studies suffered from methodological problems with little attention or reporting on evaluation instrument development and testing, and underreporting on the populations studied and had deficiencies in statistical analysis. (16) The call to demonstrate the efficacy of C M E gave rise to hundreds of randomized controlled trials investigating the overall value of C M E programs through summative  measures (achievement scores, change in clinical behaviour). However, little attention was paid to program development or theory related to identifying determinants of behaviour change (17). Literature reviews in the 1980s provided the C M E community with strong evidence that some C M E interventions effect physicians' knowledge in the test situation, less robust evidence that some C M E interventions facilitated changes in clinical practice, and only weak evidence that C M E altered health outcomes. (18) The increased use of Randomized Controlled Trials (RCT) designs in examining the efficacy 1  of C M E in the 1980s, coupled with the growing interest in evidence-based medicine in the 1990s and the application of meta-analytic and evidence-based synthesis procedures to the C M E literature, resulted in a number of systematic reviews investigating the impact of C M E interventions on physicians' knowledge, physicians' clinical routines and patient health outcomes. These systematic evidence-based reviews (18;19) found there is no one magic bullet that leads to knowledge transfer and changes in physician behaviour, and that a combination of best-evidence educational strategies and those considered to be most promising would be more effective. It was also apparent that the commonsense belief that learning was a rational process of experts teaching practitioners, primarily by lecture, did not usually lead to the acquisition of new or updated knowledge resulting in practice change. (20;21)  Need for a Theory-Informed Conceptual Framework to Improve CME Research and Practice Although the systematic reviews based on RCT designs have provided the C M E community with valuable information as to whether some C M E interventions or combination of interventions result in changes in clinical practice or impact health  Davis et al. (15;18) and Oxman et al. (19) noted in 1984 seven articles met their inclusion criteria, in 1992 a further 43 RCTs met their criteria, by 1995 an additional 49 articles were included. 2 Clinical routines are defined as those practices that have become routinized, habitual and automatic and are believed to be a factor in why informational interventions on their own are not successful. (280) 1  outcomes, they failed to illuminate what factors lead to a program's success or failure in enhancing knowledge transfer and its utilization. (18) However, RCTs typically lack sufficient qualitative details to inform theory development or improve the conceptual understanding about phenomena or the intervening critical factors. (18) Chen (22), presenting a conceptual model for integrating qualitative and quantitative procedures to enhance evaluation methods in the social sciences, describes the strengths and weaknesses of methods-oriented evaluations and theory-driven evaluations and suggests that the "excessive advocacy of any one method might result in the exaggeration of that method's strengths and blindness to its weaknesses." (22) In discussing weaknesses with typical methods-oriented research, Chen claims the evaluation methods used simply fail to provide insight into the underlying causal mechanisms that facilitate or hinder treatment effects. Without a strong conceptual framework, both methods-oriented research and theory-driven research may be susceptible to Type II errors of inadequate measurement, for instance, not sensitive to changes in attitudes, beliefs and behaviour. (22) Green (23) posits that other factors, such as, if researchers do not explicitly investigate and report whether the planned treatment differed from the delivered treatment, or the official goals differ from the operative goals, could lead to reporting inaccurate findings and conclusions, which he refers to as Type III errors. Lipsey (24) ' p  6  refers to these types of methods-oriented evaluations as "Black Box" evaluations (see Figure 1). "Black boxes," as Ashby (1956) defined them, "are organisms, devices or situations for which inputs and outputs can be observed, but the connecting processes are not readily visible."  3  Ashby WR. (1956) Introduction to Cybernetics. London: Chapman & Hall.  Figure 1. Prototypical Black Box Model (24)  Input •  p  7  Output  With the recognition that without a strong theoretical framework most C M E evaluations will continue to use simple input-output evaluations, Robert D. Fox, the editor for The Journal of Continuing Education in the Health Professions in 1995, established a new mission for the journal. In his editorial entitled, Narrowing the Gap between Research and Practice, Dr. Fox announced the creation of new sections to challenge the continuing education leadership and research community to build a stronger theoretical framework and to improve the efficacy of continuing education in improving physician knowledge, changing physicians' routine behaviour and improving health outcomes. A primary goal for this reconceptualization of C M E was to link educational theory to practice. The reconceptualization of continuing education in the health professions is still in its early stages. The leadership in C M E research has recognized that part of the solution required to address challenges facing the field of C M E is the need to develop a strong theoretical framework to better inform research and praxis in the C M E field. (18;21;2528)  Introduction of the PRECEDE Model to CME Davis, Thomas et al. (18), in a review of 50 randomized controlled trials investigating the impact of C M E interventions, modified components of Green's (29) P R E C E D E model to categorize the type of educational interventions in the C M E literature for 4  Dave Davis (2002) attributes the idea to use this taxonomy arose from discussions with Andrew Oxman, one of the authors of the systematic review.  4  analytic purposes. PRECEDE is an acronym for determinants of behaviour change in the health promotion field. It stands for predisposing, reinforcing and enabling constructs in educational diagnosis and evaluation. The original purpose of the model was "to organize existing theories and constructs (variables) into a cohesive, comprehensive and systematic view of relations among those variables important to planning and evaluation of health promotion." (30) ' p  4 0  Predisposing factors include a person's or population's knowledge, attitudes, beliefs, values, and perceptions that facilitate or hinder motivation for change. Enabling factors are those skills, resources, or barriers that can help or hinder the desired behavioural changes as well as environmental changes...Reinforcing factors, the awards received, and the feedback the learner receives from others following adoption of the behaviour, may encourage or discourage continuation of the behaviour. ( 3 0 ) ' pp  28-29  Davis et al. (18) modified these constructs to group intervention strategies as follows: Predisposing interventions primarily involve communicating or disseminating information, enabling interventions facilitate the desired change at the practice site, and reinforcing interventions involve reminders or feedback. For analytic purposes Davis et al. grouped interventions into four types: (i) those using predisposing factors only; (ii) those using predisposing and enabling factors; (iii) those using predisposing plus reinforcing factors; and, (iv) interventions as a single multi-faceted maneuver such as chart review with a peer physician or a combination of all three types of interventions. Davis et al. found those interventions using predisposing methods produced mostly negative or inconclusive results, 9 out of 10 studies using predisposing and enabling methods attempting to change physician performance were positive, and 2 of 6 studies attempting to change health outcomes were successful. Interventions using primarily feedback and reminder systems in conjunction with predisposing factors were effective in changing physician performance in 12 out of 31 interventions instances when used alone,  or in conjunction with didactic sessions, workshops, academic detail visits or printed material. Interventions using a combination of all three strategies were effective in 8 out of 8 interventions studied in changing physician performance and 7 out of 8 studies investigating health outcome. Although it was not clear what variables were contributing to a program's success or failure, it was clear that there was no one 'magic bullet' leading to more efficacious interventions and that a combination of interventions using predisposing, enabling and reinforcing factors have a greater potential to changing physician behaviour and patient health outcomes. (18) The utilization of the construct of predisposing, enabling and reinforcing factors appears to be a fruitful topology for categorizing educational interventions and has also been discussed in the C M E literature as components to various conceptual schemes. Adelson et al. (31), for instance, defines and incorporates predisposing, enabling and reinforcing factors, into a conceptual framework for facilitating performance in an organizational setting to address potential institutional barriers to innovation. In recent years there have been a number of studies focusing on specific predisposing factors attempting to identify potential barriers to change including the identification of physicians' motivation for attendance at C M E programs (32), current opinions, attitudes and behaviours of participants, and the assessment of physicians' interest or commitment to change. (33-37) In 1991, Green and Kreuter expanded the model to include a broader range of factors believed to influence behaviour change. PROCEED is an acronym for additional elements in planning, implementation and evaluation recognizing that other institutional factors such as policy, regulatory, and organizational constructs in educational and environmental development influence behaviour change.  The P R E C E D E and the PRECEDE-PROCEED models are extensively used in the health promotion community and have been shown to be robust conceptual educational models for the planning, implementation and evaluation of health promotion programs targeting primarily lay populations. With over 1,000 studies using the PRECEDE or PRECEDE-PROCEED models, there are a growing number of rigorously evaluated, randomized clinical and field trials reported in the health promotion literature. (38;39) Based on the success of the model in the health promotion field, the NHS Centre for Reviews and Dissemination in their Effectiveness Health Care Bulletin in 1999 recommended the PRECEDE-PROCEED model for facilitating the uptake of evidence into practice. (40)  Central Purpose In this dissertation, I will argue that the most commonly used approaches to C M E planning, implementation and evaluation fail to take advantage of current research and thinking in the cognitive sciences and learning theory, program planning, and program evaluation. This system-based failure speaks to the need in C M E to experiment with other educational models that are utilized in other fields, such as the health promotion field. I will argue that the most promising conceptual model for future research and practice in the C M E field, given the current gaps and challenges facing this field, is the PRECEDE-PROCEED framework especially if it is modified for C M E purposes. This thesis uses a case study to illustrate, reflect and test the adaptability and feasibility of adopting a modified version of the PRECEDE-PROCEDE model for C M E purposes. The case study is based on a province-wide C M E program in Canada with the primary educational objective of enhancing family and emergency physicians' knowledge and management of whiplash-associated disorders.  Purpose of a Conceptual Analysis Although there is not a specific methodology for conducting a conceptual analysis, there are some conventions concerning its purposes and means of validation. Coombs and Daniels (41) posit that the purpose of a conceptual analysis is to enhance our understanding by improving "the sets of concepts or conceptual structures in terms of which we interpret experience, express purposes, frame problems and conduct inquiries." If our conceptual structures lack logical coherence, blur important distinctions, or create useless dichotomies, or we understand them so poorly that we are unable to translate them adequately into research instruments and policy prescriptions, curricular policies and research studies will fail to be fruitful. (41) ' p  1  Fruitfulness is typically associated with pragmatic ideas of usefulness such that, if the conceptual framework assists in the generation of theories and if the theories lead to a more accurate accounting of the phenomena, it is considered to be fruitful. (42) ' p  12  It is  my hope that this body of work will foster reflection on C M E praxis as well as promote and advance further experimentation with alternate conceptual models of planning, implementation and evaluation in the C M E field.  Significance of this work The importance of continuing professional development was well articulated at the turn of the 20th century by Sir William Osier. Dr. Osier, a Canadian, and possibly the founder of modern C M E , during his keynote address in London, England, on July 4, 1900, entitled, The Importance of Post Graduate Study stated, "More clearly than any other, the physician should illustrate the truth in Plato's saying, that education is a D  lifelong process."(43)  9  Osier was also a renowned scientist and recognized that with  the tremendous scientific and medical advances in the late 19th century, the gap between good medical care and the routine practices of those whose training ended when  attending medical school was widening. In 1965 there were 100 randomized controlled trials on medical innovation in the United States, by 1999 over 10,000 RCTs were published annually. (44) Today, with over 20,000 medical journals and two million articles published annually it is estimated that physicians would need to read 19 original articles daily in order to keep current. (45) Even if physicians were able to keep abreast of the research findings, there are growing concerns about the great variance in the quality of research published and evidence that most physicians are not competent in critically appraising research or translating evidence-based findings or guidelines into practice. (46;47) The expanding gap between what is known and what is practiced in today's world necessitates new ways of thinking about the purpose and role of continuing professional education. In this thesis I argue for a need to reconceptualize C M E to better meet the many challenges facing C M E and most importantly the growing need to more effectively and efficiently reduce the gap between current knowledge and clinical practice. This thesis provides an historical analysis to better understand some of the forces shaping current C M E practices in order to identify fundamental structural, pedagogical and related contextual problems requiring attention. Having identified problems with current practices, this thesis presents an alternate conceptual model originally developed in the health promotion field, and uses a case study to illustrate the adaptability and feasibility of this model to the C M E environment.  Limitations This thesis is investigating the feasibility and adoptability of a systems-based conceptual framework integrating theoretical work across disciplines to inform planning, implementation and evaluation processes in C M E for the purpose of enhancing  knowledge transfer and its utilization. There is, therefore, a need to introduce and summarize a very broad base of literature to provide both an historical understanding as to how we have ended up with our 'normative' model of operation, and how current research and other conceptual models could enrich C M E practices. The benefits of drawing on a large body of literature is the potential to integrate and synthesize methods and ideas arising from different literatures. However, a limitation in attempting to address a system-based problem informed by a broad range of literature is the need to be very selective in presenting pertinent areas of a given literature body and being unable to present an in-depth analysis of any one body of knowledge or explicate and analyze a broad range of controversies within a given field or discipline. This type of transdisciplinary research is fraught with challenges. Lattucca (48) defines disciplines as complex phenomena, " . . .as sets of problems, methods and research practices or as bodies of knowledge that are unified by any of these. They can also be defined as social networks of individuals interested in related problems or ideas." Disciplines not only pertain to certain fields of study and inquiry methods, they are also made up of various communities and subgroups that share a range of assumptions, behaviours, and beliefs about scholarship. Value judgments are made by individuals within a discipline regarding issues of research validity and legitimacy, the appropriateness of topics for investigation, the kind of questions that are valid to ask, the methods one uses, and what constitutes a valid answer. As this thesis crosses many disciplines and fields of study, I have tried to take a very pragmatic perspective on selecting literature to inform the conceptual framework rather than working from a more ideological perspective.  A major area of contention within the education community has been concerned with questions related to ideology, epistemology, and ontology, especially within the education research community. (49;50) Firestone (49), discussing the rhetoric of quantitative and qualitative research paradigms, cities Rossman and Wilsons' idea that there are two camps at the extremes of the quantitative and qualitative debate: the purists and the pragmatists. The purists assert that qualitative and quantitative methods are based on incommensurate paradigms "that make different assumptions about the social world, about how science should be conducted and what constitutes legitimate problems, solutions, and criteria of 'proof." Purists believe that method represents a logic of justification that begins with first principles about truth, reality, and the relationship between the principles inherent in the paradigm. Pragmatists view a more instrumental relationship between paradigms and methods. To pragmatists, methods are seen more as a collection of techniques and are not necessarily inherently linked to quantitative and qualitative paradigms. This thesis is more clearly aligned with a pragmatist perspective and strongly recommends that both qualitative and quantitative research and evaluation methods are needed to better inform our practices. I also suggest that greater collaboration between practitioners engaged in qualitative and quantitative research could potentially lead to improvements in research methods used by both research communities.  Author's Interest in CME But who shall parcel out His intellect by geometric rules, Split like a province into round and square? Who knows the individual hour in which His habits were first sown, even as a seed? Who shall point as with a wand and say This portion of the river of my mind Came from yon fountain? (51) " lines 2 0 8  1 5  My original training and occupation was in the field of early childhood education, a subject distant, at least in the span of ones educational development, from my current vocation and interests in the field of C M E . There are many factors influencing the twists and turns, challenges and opportunities that contribute to ones life experiences that cause me to resonate with Woodsworth's Prelude. However some selective background of myself, my work experience and interests in the general field of education and current involvement in the planning, implementation and evaluation of C M E will provide the reader with an insight into some of the pragmatic challenges facing C M E , and some of the author's past experiences contributing to the development of the proposed model. In 1974 I graduated in Early Childhood Education (ECE) from Seneca College, a technical college in Toronto, Ontario, and was hired as a co-coordinator for a cooperative daycare situated at the University of British Columbia. The early childhood program was focused on child development, theories about learning, with special attention for students in the program to become more reflective of ones own life experiences and how they might influence ones behaviour with children. After working in the field for a number of years, my interests moved from younger children to working with 'teenagers at risk' and I worked as a child care worker responsible for developing 'meaningful' after school programs for teens living in a low income housing project. My interest in cooperative endeavors, coupled with the recognition of the pitfalls of ghettoizing low income families, led to volunteer and leadership involvement in the cooperative movement, in the housing sector and the food cooperative sector. In the early 1980s I was the co-founder of a community-based not-for-profit cooperative restaurant with the goal to generate funds and resources to support international and community-oriented social action projects.  My personal interest in health education, the human potential movement and complementary health care, eventually led to my return to school to complete an undergraduate baccalaureate degree specializing in health education through Antioch University's independent study program administered through Cold Mountain Institute and Antioch West. My Faculty Advisory Committee consisted of two medical physicians, Drs. Wendy Palmer and Ron Puhky, both practicing preventative medicine, Professor Gaalen Erickson, a U B C Faculty of Education member as well as an advisor associated with Cold Mountain Institute. After graduating from Antioch West, I trained in Toronto to become a massage therapist and in 1981 became the President of Part III, Massage Section for the Association of Physiotherapists and Massage Practitioners in British Columbia, the licensing body for physiotherapists and massage practitioners. During my term as President, it was apparent that the field of massage therapy needed to improve its curriculum and examination process to better reflect the growing scientific body of literature regarding massage therapy and to create higher standards to prepare students for practicing massage therapy working under the provincial medical plan environment in British Columbia. It was also clear that massage therapists needed to establish a professional body to address more adequately some challenges, especially its legitimacy as a health profession, as well as to enhance the continuing education needs of the profession. Action plans were developed leading to a new curriculum and examination process and the birth of a professional body, The Massage Therapists Association of British Columbia. Following my initial training in massage therapy, I was fortunate to have postgraduate training with Dr. John M c M . Mennell, a noted doyen in the field of physical  medicine and rehabilitation. Dr. Mennell developed a systematic approach to the 6  assessment of common musculoskeletal disorders using manipulative methods for differential assessment, and when applicable, for treatment purposes. Within my practice as a massage therapist, it soon became apparent that from a patient's perspective, the type of diagnosis and treatment one "had" was based more on "who" the patient saw, rather than through a comprehensive systematic standardized assessment procedure. It was also very clear that the lack of communication and collaboration among health professionals, be it between physicians and therapists, physicians and specialists, or therapists and therapists, was a tremendous barrier to improving our collective knowledge-base and the development of best practices in the rehabilitation field. It was through this recognition and discussions with Dr. Mennell that led to the establishment of the Physical Medicine Research Foundation (PMRF). PMRF was established as a consumer-based registered charitable organization, the majority of its Board Members being non-health professionals, with a mandate to reduce disability and impairment from musculoskeletal conditions and improve the quality of care. To accomplish this, PMRF uses a grassroots community-based operating model, seeking the interest and involvement of a broad range of stakeholders to improve the quality and cost-effectiveness of care for people with musculoskeletal complaints with the understanding that all stakeholders must work together, as equal partners, to find better solutions for people presenting common non-malignant musculoskeletal complaints, such as back pain, whiplash, cumulative strain disorders etc. Disciplines involved with PMRF include epidemiologists, family physicians, emergency physicians,  Dr. Mennell was a consultant in physical medicine to the three armed forces in the United States and former member of the Expert Review Committee on Independent Providers of Medicare for the US Congress. He worked extensively with a number of Veteran Administration Hospitals. He was Founding Member, Past President of the North American Academy of Manipulative Medicine. Dr. Mennell published over 50 articles, five textbooks as well as producing numerous films and videos on physical medicine.  6  15  orthopaedic surgeons, physical medicine and rehabilitation specialists, occupational physicians, occupational nurses, acupuncturists, chiropractors, massage therapists, medical and allied health care educators, health services researchers, and on the whiplash prevention side, auto-engineers. PMRF facilitates and funds multidisciplinary research projects, facilitates and plans strategic planning meetings, conducts needs assessment activities, as well as plans, implements and evaluates health professional and consumer education programs. PMRF incorporated two guiding principles arising from my experience in community-based development, and training in the field of education: (i) the principle of non-hierarchal participation of health professionals, researchers and educators working together with other stakeholders for the betterment of humankind, and (ii) the commitment to cultivate and foster best practices in the creation of lay and health professional educational interventions informed by high quality research, and where possible, evidence-based research in medicine as well as in education. PMRF has held ten international conferences, over forty continuing education workshops (typically one to four day courses), four strategic planning meetings, two public forums and hosts a monthly chronic pain self-help support group. The majority of PMRF's health professional education programs are accredited under the auspices of the host university's Faculty of Medicine or Division of Continuing Medical Education. PMRF also seeks and receives accreditation from other professional associations, such as The College of Family Physicians of Canada, MAINPRO Category I program and the Royal College of Physicians and Surgeons of Canada M O C O M P program. In the mid 1990s, PMRF like most other educational planners seeking C M E accreditation, incorporated a few principles of 'adult learning theory' and the utilization  of 'behavioural learning objectives' not only in the planning process but also in the evaluation process as part of the requirement for accreditation. However, these instituted changes in planning and evaluation have, for the most part, failed to generate sufficient data to inform improvements in C M E programming. The questions rumbling in my mind were: How can C M E planners better determine what programs should be created? How can we improve the planning process and who should be involved? More specifically, what factors, aside from content, should be considered in planning, implementing and evaluating C M E to enhance knowledge uptake and utilization? And what instruments could be developed to collect better qualitative and quantitative data to inform C M E research and praxis?  Introduction of Case Study The idea for the B C Whiplash Initiative (BCWI) project, the case illustrating the application of the proposed curriculum model, arose from PMRF's 8th international symposium, Musculoskeletal Pain Emanating from the Head and Neck, held in Banff, Alberta, in October 1995. The conference was still in its planning phase just as the Quebec Task Force Report on Whiplash-Associated Disorders (QTF) released the first systematic evidence-based and consensus-based report on whiplash-associated disorders (WAD). A number of the authors of the report were invited to speak at the conference. During the conference, PMRF also held a parallel focus group consisting of invited speakers to recommend actions arising from the conference discussions and proceedings. All but one member of the focus group concluded that the QTF Report (52) represented the first evidence-based synthesis and consensus to date on the state of our knowledgebase about W A D and recommended that PMRF commit itself to disseminate the findings  of the report for the benefit of patient care through the development of a comprehensive C M E program. There were a number of unusual elements in the planning, implementation and evaluation of this C M E program that contributed to the program's uniqueness and facilitated the development of the curriculum model underpinning the program. Unlike most C M E programs organized by accredited C M E providers, the founding organizing body for this project was PMRF. After acceptance of this recommendation by PMRF's board of directors, local stakeholders were brought together to discuss whether there was a perceived need for a comprehensive community-based education project. Initial stakeholders included the B C College of Family Physicians of Canada, University of British Columbia (UBC) Division of Continuing Medical Education, U B C Department of Family Practice, Rural Education Training, U B C Family Practice Residency Program, and U B C Undergraduate Medical Education. This ad-hoc committee concluded that there was a strong need to develop educational initiatives to update physicians in British Columbia (BC) to improve the diagnosis and management of patients presenting W A D at several levels, including undergraduates, graduates, and practicing clinicians. A representative from the Insurance Corporation of British Columbia (ICBC), the public auto insurer for B C , was also invited to this initial meeting as a guest and indicated an interest in funding a provincial educational initiative at arms' length in the way of an unrestricted educational grant. In my capacity as PMRF's Executive Director and as a graduate student with the Centre for the Study of Curriculum and Instruction, I was interested in applying a more systematic planning framework to this project. I was responsible for writing the grant proposal including the development of initial educational objectives, research objectives,  research budget, project timelines, PMRF's administrative budget, and collating stakeholder proposals related to program delivery. During the proposal building phase, Marc Broudo, Assistant Director, Division of Educational Development and Support, U B C , and I conducted a review of recent systematic reviews of the C M E literature and reviewed the seminal text, The Physician as Learner: Linking Research to Practice, edited by David Davis and Robert Fox published in 1994. The book inspired an adventure in curriculum development and program evaluation through articulating a common goal to bridge the gap between health professional practices and current research using the best educational practices available. If health care practices based on the best available information are an ideal of clinicians, then, equally, delivery of C M E , based on the best evidence about its efficacy, should also become an integral ingredient in C M E provision.(6) ' p 247  Once the proposal was drafted, it was reviewed by participating bodies and, upon their agreement, funding was sought for this initiative. To our surprise and delight, funding was forthcoming from ICBC in the way of an unrestricted grant and in accordance with Canadian Medical Association policy regarding industry sponsorship of C M E programs. The BCWI, through an extensive 12 month curriculum development process and the collaboration of all stakeholders, created the largest C M E educational initiative directed to general and emergency practitioners in B.C. and the first initiative addressing the continuum of learning from undergraduate training, residency training and physicians in practice. The BCWI used many different types of educational vehicles to deliver the curriculum and to engage the interests of physicians. These included one hour grand round sessions primarily at hospitals, with some one hour sessions at group practice sessions, telemedicine sessions reaching out to rural physicians, half day and full day  sessions that included educational material on medico-legal reporting and the management of chronic pain conditions. The project also culminated in hosting a World Congress on Whiplash-Associated Disorders in February 1999, where preliminary findings on the BCWI were presented. The World Congress and the BCWI planning process led to initiating a $318,000 whiplash research and awards competition funding 9 research projects and 5 best poster awards.  Summary This chapter provided an overview of the many challenges facing the C M E field and introduced the argument that many of these challenges are a result of a system-based failure in current planning, implementation and evaluation procedures used. From both within and outside the profession there is a growing recognition that C M E needs to develop a more robust conceptual framework to better inform C M E research and practices. Since the 1980s, the C M E research community have begun to address some of the challenges facing the field. Systematic reviews of RCTs have provided a body of evidence that demonstrate certain types of C M E interventions are more effective than others in knowledge uptake and facilitating physician behaviour change, however, the lack of conceptual framework is a barrier to understanding this phenomenon or creating a more systematic approach to uncovering factors contributing to knowledge transfer. This chapter also introduced the PRECEDE-PROCEED framework and presents the central thesis that this framework, especially if it is modified to address many of the stakeholders' challenges facing C M E , is a better conceptual framework for C M E research and practice. The final part of this chapter introduced some background on myself as well as background on the case study that was used to illustrate the adoptability and feasibility of the PRECEDE-PROCEED framework for the C M E field.  Structure of Literature Review The review is informed by the literatures of medical history, medical education, medical teaching, medical innovation, principles of evidence-based medicine, curriculum development, program planning, program implementation, program evaluation, adult education, cognitive sciences, health promotion, social psychology, and a growing body of literature on knowledge transfer and diffusion. The organization of the literature review is primarily through an historical chronological lens, providing an analysis of predisposing and enabling factors that contribute to the failure of current planning, implementation and evaluation of C M E as well providing an alternative set of predisposing and enabling factors that could contribute to the creation of a better model for C M E programming. Green's original typology defines predisposing factors as those variables that facilitate or hinder motivation for change which include a person's or population's knowledge, attitudes, beliefs, values, and perceptions, whereas enabling factors are variables that can help or hinder the desired behavioural changes and/or environmental changes such as the development of skills, resources, or barriers that hinder or enable change. The argument being presented is that C M E planning as we know it is based on old models of learning theory and program planning that have become institutionalized to such a degree that reconceptualizing C M E will need to not only incorporate current knowledge about learning, planning and evaluation, but likely require changes across the continuum of medical education including institutional structures, regulatory policies, funding and governance. The identification of these factors contribute to my argument that a modified PRECEDE-PROCEED model lends itself as a fruitful framework for informing a reconceptualization of the C M E field.  In order to help the reader appreciate how we have arrived at this state of affairs, this chapter first examines and analyzes an historical perspective on the development of medical education and C M E in Canada and the United States and presents the argument that current challenges to C M E practices arise from many of the same challenges faced in undergraduate medical education and that this continuum of medical education, historically and currently, is strongly influenced by the presence, absence and effectiveness of internal and external administrative, organizational, policy and regulatory forces. The second part of the chapter presents the argument that the continuum of medical education, historically and currently, with a few exceptions, has not kept abreast of developments in our understanding about the nature of learning and that this failure in understanding, regarding how people learn, and what factors enhance or hinder knowledge uptake and its utilization contribute to the continued use of less effective models of program planning, teaching and evaluation. In the third chapter, I argue that this reliance on current models of program planning and evaluation have failed to provide sufficient information to better inform the development of a more robust conceptual framework to enhance knowledge transfer and its utilization to improve patient health outcomes. Lastly, I argue that the solution to challenges facing C M E are multi-faceted and require a radical reconceptualization of the C M E field that also has implications for undergraduate medical education.  Chapter Two  Historical, Organizational and Pedagogical Factors Influencing CME Practice Overview of Internal and External Forces Shaping CME As discussed in the introduction, C M E is in a state of crisis. It is not only under increasing pressure from many stakeholders to improve current practices, it also suffers from a lack of a robust conceptual framework to underpin its research and education practices. Michael Gammon of the A M A proposed the diagram below (see Figure 2) to represent multiple and complex forces influencing C M E during the past twenty years. (6)  Figure 2. Internal and External Forces Influencing CME (6)  p  4  Forces outside C M E (External Environment)  Government Impact: -Federal -State -Reimbursement (Regulation, Funding, Licensure)  PUBLIC of a n d Role Unions Business  ( M  (Patients) o r e Elderly)  «.  .  Technology ^  A d v a n c e d Information and Communication Technology (Telemedicine) Increased Scientific Knowledge, skills  Insurance S y s t e m s - N e w Government-run Health-care Outcomes Assessment  Greater Attention t o Preventive M e d i c i n e and Health Lifestyle  International C M E — * -  Litigation Threat —<•» N u m b e r and Distribution of Physicians (Competition)  Undergraduate and Graduate Medical Education  Certifying Boards  Patient Care (Managed care, other systems)  Recertification  M o r e Physician Employees, Less Solo Practice  Industry  Clinical Privileging and More Women Physicians  Practice Parameters N  e  w  Resources  Increased Specialization  Credentialllng Decisions  Davis and Fox (6), describing Gannon's conception of these forces, state the innermost circle represents the current players directly involved in C M E planning, research, faculty members, leaders and resources. The middle layer includes the C M E providers, C M E accreditation bodies, physician participants, and the C M E system. The outer layer are those factors that Gannon has characterized as being external factors including government interests, changes in patient demographics and expectations, advances in technology, rapid increase in scientific knowledge, undergraduate medical education, international C M E etc. The representation of the range of forces influencing C M E practice is successful from a graphical perspective in framing current conceptions of internal, organizational and external forces. However this representation does not provide sufficient conceptual linkages to better understand how these forces have shaped and continue to influence C M E practice or challenge the status quo regarding who should be actively involved in the C M E planning, implementation or evaluation process and what resources are needed to ensure C M E programming meets society's need to improve the quality of preventive and curative care.  Historical Perspective on the Continuum of Medical Education The development of C M E is intimately linked to the evolution of medical education, a progression from an apprenticeship learning model to a university-based curriculum model, the birth of post-graduate extension lectures, later to be defined as C M E . C M E followed the traditional model of undergraduate medical education which was intellectually driven by theories and principles arising from experimental science attached to a teaching style which was primarily passive in nature. (53) ' (12) The structure of all p  2  medical education was based on the curriculum design of the Hopkins Medical School  that has remained the dominant structure for medical education. (54) ' p  3  The basic  objective was to teach academic content. The assumption behind medical school curricula generally, and C M E programs in particular, was that learning was a rational process of experts teaching practitioners primarily by lecture with the occasional workshop or casebased study. It was believed that, with the acquisition of new or updated knowledge, clinical practice would change. (20)  Early Organizational, Administrative, Regulatory, Policy Factors Shaping the Continuum of Medical Education The North American medical school system began as a supplement to the apprenticeship system common in the 17th and 18th centuries. (55) Virtually all physicians in the Atlantic seaboard colonies had learned their craft by apprenticeship but from early in the 18 century many of them crossed the ocean to visit the hospitals and attend lecturers of the professors of Leydon, Paris, London and Edinburgh. A desire to teach was a natural sequel to this experience; consequently as early as 1756 classes were held in Philadelphia. (56) ' th  p  14  The movement towards a university-based medical training program began taking shape in the first half of the 19 century in France and in the latter half of the 19th th  century spread to Germany, the United Kingdom, Canada and the United States. University-based training held the promise for "high matriculation standards, an organized curriculum, regularly appointed permanent faculty, and laboratories linked with a hospital service to facilitate research and teaching." (56) ' p  Medical schools in the United States began in 1765 with the medical department of the University of Pennsylvania, Columbia College, formerly King's College. Harvard University followed in 1767, Dartmouth College in 1798 and Transylvania College in 1799. Delegates from these medical schools and societies held formal discussions about establishing standards of medical education in Northampton, Massachusetts, as early as 1827. In Canada, formal medical teaching began at the Montreal Medical Institution in  1823 with twelve students, but closed after two years. (56;57) Six years later it re7  opened and became the Faculty of Medicine at McGill University. The University of Toronto and University of Montreal's Faculties of Medicine commenced in 1843, University of Laval in 1852, Queen's University in 1854, and Dalhousie in 1867. In 1881, the University of Western Ontario's Faculty of Medicine opened its doors, soon to be followed by the University of Manitoba in 1883. By the turn of the twentieth century, 8  over 457 medical schools had started in the United States and Canada, many shortlived, most of them considered of very poor quality with a mixture of private and public 9  institutions. (55;56;58;59) This period of rapid uncontrolled explosive growth of medical schools in North America led to a mounting need both within the profession and outside of the profession to establish standards for medical practice. In 1846, the first national medical conference was held in the United States to discuss the establishment of standards of medical education. The following year, at the second national meeting, the American Medical Association (AMA) was formed to attempt to standardize instruction in medical schools, but met with little succes (54). Twenty years later, in 1867, the Canadian Medical Association (CMA) was formed to establish a standard for medical education and a uniform system of licensure at the start of the newly constituted Dominion of Canada. (60) In 1876, the American Medical College Association was formed, only to I0  Craig (1976) claims The Montreal Medical Institute began in 1823 noting that the four physicians who established the school were graduates of the University of Edinburgh. MacFarlane et al. (1964) stated the Institute began a year later, in 1824. In the more western provinces, medical schools began in 1913 at the University of Alberta, 1926 at the University of Saskatchewan, and at the University of British Columbia in 1949. MacFarlane (1964, p. 19) claims "Whether measured by standards of admission, length and character of curriculum or qualification of their teachers many were little more than a "diploma mill." The formation of national organizations and decisions about membership requirements and licensure also led to battles within and outside of the profession questioning whom and what practices were deemed to be legitimate and whether physicians should associate with non-physicians. MacDermont (1938) reports on prominent physicians in 1869 being refused admission to the C M A executive on grounds that they did not violently oppose the Parker Act or were accused of consorting with unlicensed practitioners including homeopaths. 7  8  9  1  26  be disbanded six years later as its membership representing both commercial and public institutions became untenable and failed to advance medical education standards. In 1891 the Association of American Medical Colleges (AAMC) was organized. State licensing authorities during this period were also being created to establish legal requirements for medical practice. Although there had been many singular efforts made to improve medical standards, no one organization on its own was successful in improving medical education standards across North America. Only when there were sufficient organizational and legal structures in place and an overall consensus among these stakeholders could improvements in standards take place. (54) ' p  1 5  In 1905, one year after the A M A formed the Council on Medical Education, the Council initiated an independent review of schools in the United States and Canada through the auspices of the Carnegie Foundation for the Advancement of Teaching, to report on the state of medical education to the A M A and the House of Delegates. The survey was conducted by Abraham Flexner, a noted educator , and Dr. Nathan Colwell, 11  Secretary of the A M A ' s Council on Medical Education. (56) ' p  1 9  Mr. Flexner visited 155  medical schools seeking information about: (i) the entrance requirements and their enforcement, (ii) the size and training of faculty including the pre-clinical sciences, (iii) the sum of funds available from endowment and from fees and its use, (iv) the quality and adequacy of the laboratories provided for instruction and, lastly, (v) the relationship between the medical school and hospitals including "freedom of access to beds and freedom in the appointment by the school of the hospital physicians and surgeons." (55)  Flexner a graduate of John Hopkins University founded a school that reportedly had very high applicant success rates by its students to top university programs in the U.S.A.. " (55) 11  27  The release of the Flexner Report in 1910 is a landmark year  in medical education  12  in North America. The report was a scathing assessment of the state of medical education in the United States and Canada, as medical schools in Canada suffered from similar deficiencies as those in the United States. In the matter of medical schools, Canada reproduces the United States on a greatly reduced scale. Western University (London) is as bad as anything to be found on this side of the line; Laval and Halifax Medical College are feeble; Winnipeg and Kingston represent a distinct effort toward higher ideals; McGill and Toronto are excellent. (56) ' p  2 1  Most medical historians and scholars claim the Flexner Report resulted in many changes including school closures, the end of private medical schools, changes in criteria for admission, the establishment of basic sciences curriculum taught by full-time scientists striving to advance knowledge in their field, clinical subjects taught by scholarly physicians who "devote themselves to their hospital wards", and the alignment with university-based education. (54) In Canada, MacFarlane (56)  p p  '  1 0 7 - 8  in his report,  Royal Commission on Health Services: Medical Education states that three main points arising from the Flexner Report to inform the development of medical education in Canada are: •  most proprietary schools of medicine should cease to exist,  •  only a few schools [none in Canada] sic his brackets were providing a satisfactory academic atmosphere for the teaching of physicians. 13  •  there must be at least a nucleus of full-time teaching staff in each department of a medical school.  B. Barzansky & N. Gevitz, Beyond Flexner: Medical Education in the Twentieth Century, 1992, (p. 3-8, 13) posit the reform of medical education was well underway prior to the Flexner Report, and suggests that the report was not so much revolutionary as catalytic to an already evolving process. There is little debate by medical historians that the Flexner Report was instrumental in galvanizing media interest as well as social and economic resources to support medical education. MacFarlane's claim that none of the schools in Canada provided a satisfactory academic atmosphere for teaching physicians contradicts Flexner's statement that McGill and University of Toronto Faculty of Medicine were excellent. 12  13  28  The Flexner Report also resulted in a number of unintentional changes that have played a significant role in the shaping of the modern medical school. Hudson (58)  ppl6  "  7  posits the Flexner Report was 'misconstrued', leading to two major common misconceptions attributed to the Flexner Report, although they were contrary to Flexner's own pedagogical beliefs. Flexner's critics believed that he was a strong proponent of The John Hopkins Medical School, endorsed its lockstep curriculum and pedagogical practices, later to be known as the 'Hopkins Model'. Hudson (58) ' p  17  however suggests  some of Flexner's critics had likely not read Flexner's Report and were responding to other people's interpretation of Flexner's assertion that the John Hopkins Medical School was the best medical school in North America. ' ' Flexner did state that John Hopkins 14  15  16  Medical School was the best school operating in the United States based on his review criteria, however he did not believe John Hopkins Medical School represented the gold standard for medical training. This misconception, led to the diffusion of the "Hopkins 17  Model" as being the gold standard model "that other schools, either willingly or not, would eventually follow." (54) ' p  16  C M E , historically, included any systematic training after graduation including voluntary internship, specialty training, post-graduate extension lectures, or courses. With the lack of North American standards of medical education, Flexner saw one of the Ffrangcon Roberts, 1948 criticizes the Flexner Report forjudging medical schools by their original contributions to knowledge rather than the standards of care provided by their graduates [he did recognize such an assessment would be beyond the ability of any one person]. He was also critical that Flexner "had no intimate knowledge of the intellectual difficulties which students experience (p. 3) and was concerned about the assertion that basic sciences should be taught by scientists far removed from the practitioner's need for elementary information within a given field, rather than the more advanced areas of interest of a scientist wishing to advance the field. MacFarlane (1965) notes in 1912 Flexner also completed a survey of medical education in Britain, France, Germany and Austria which encouraged the movement towards professorships associated with various and departments, rather than the use of clinicians in practice Flexner et al. (1960, p. 112) in his biography refutes the authorship of the 'full-time scheme' and states it did not originate with him and was not mentioned in 'Bulletin Number Four' or 'Bulletin Number Six'. Flexner, however, did subscribe to the belief that the full-time positions were a necessary condition to further 'teaching and cultivation' and assisted John Hopkins Medical School in receiving a 1.5 million dollar endowment for this purpose. Flexner's pedagogical perspective was similar to Dewey's perspective of the need for a flexible curriculum that would foster an inquiring scientific mind. 15  16  17  29  primary goals of post-graduate continuing education as a means of improving the competence of medical practitioners 'to repair' the damage of poor medical training. (61) The Flexner Report recommended a period of supervised clinical practice as a prerequisite for graduation. Until 1913 no medical school required an internship period, although 70% of medical graduates voluntarily took internships. (61) In 1919, the A M A Council on Medical Education established the first set of standards for internship programs, The Essentials of an Approved Internship. Similarly, the training of specialists, for the most part, was not formalized prior to 1928 when the A M A House of Delegates adopted the Essentials ofApproved Residencies and Fellowships. (61) Prior to the 1900s, practitioners would take some courses, usually in Europe, and self-proclaim their specialty. Later some specialty societies were established, however there were no licensure requirements. Flexner voiced concern about the rise of proprietary graduate and post-graduate training and advocated graduate training roles be fulfilled by university-based medical schools associated with training hospitals. This recommendation was not taken up by medical schools until the 1970s. (61) In 1920, the A M A established fifteen committees to define specialties, their training programs and their qualifying examination and certification process. During the 1930s, the American Hospital Association required specialty certification in awarding hospital privileges. The increase in specialists arose from a number of factors mentioned earlier as well as the perception during World War I that specialist surgeons in the army were more efficacious. In addition, improvements in transportation systems and the general movement towards large urban centres led to markets large enough to support full-time specialists.  From the formation of the Canadian Medical Association in 1867 until 1923, the primary focus of the C M A , and medical schools in Canada was on issues related to licensure, undergraduate medical education, the development of professional bodies, organizational financial concerns, and addressing specific legislative issues. (62) The first mention of the need for post-graduate teaching in Canada by the C M A was in a report from its Committee of Education in 1923. To this end at the beginning of 1925, two physicians, Drs. F.J. Tees of Montreal and F.W. Marlow of Toronto, and the General Secretary of the C M A delivered a series of lectures in a limited tour of eight cities in the western provinces. The first ambitious effort to deliver C M E in Canada occurred in the latter half of 1925, after the receipt of an unrestricted grant from the Sun Life Assurance Company which provided up to $30,000 to support post-graduate extension lectures. Within the first year of the program, 169 teachers delivered 513 lectures to an aggregate attendance of 17,264. (62) The program was particularly well received by practitioners living in rural areas. This funding was renewed each year until 1932, during which time 401 lecturers were provided support to deliver 832 lectures with an average attendance of 38. (62) In Canada, prior to 1950, post-graduate lectures were offered through voluntary efforts by professional associations and some universities. After World War II, the demand for short courses and refresher courses increased and universities became more active in planning and implementing post-graduate courses through the establishment of volunteer-based post-graduate committees. By 1960, in the US there were only 18 medical schools with an identifiable C M E program, however this number grew significantly as state legislation required C M E for re-registration of medical licenses, as well as by generalists and specialty professional societies. (63) In 1969, the A A M C ' s committee on C M E recommended that member schools recognize C M E among their  major responsibilities. This recommendation was not supported and the committee disbanded in 1971. In 1974, 64 years after Flexner's initial recommendation that C M E be an integrated component of university-based schools of medicine, the A A M C ' s Group on Medical Education acknowledged the increasing demand by activists and stakeholders to expand the mandate of schools of medicine to include the continuum of medical education. It was not until the late 1970s that the chair of the post-graduate committee at the University of Toronto, Faculty of Medicine became a part-time university appointee. Medical schools, already overwhelmed with managing undergraduate medical education, research centres, and teaching hospitals, many already providing some continuing medical extension programs in their teaching hospitals, were reluctant to be responsible for C M E as part of their core business. C M E activities were typically initiated by volunteer committees and later through the appointment of a part-time faculty member. As interest in C M E increased, C M E programs became seen as revenue generators for the Faculty of Medicine and these division/departments were provided with some limited additional resources to create the infrastructure to support program delivery. From a funding perspective, C M E has remained a poor cousin of Faculty of Medicines' budgets with little to no investment to support C M E research, program innovation, and program evaluation. (64) The adoption of the Hopkin's model, particularly the premise of full-time faculty members/scientists involved in their research interests, blossomed after World War II, spurred on by the US government, industry and foundation investment in biomedical research. On the positive side, this investment led to unprecedented advances in biomedical research, however this investment changed the focus of medical schools.  Government Research Funding Policies: A Primary Force Changing Medical Education Lewis and Sheps (65) ' p  4 8  posit this economic investment in biomedical research and  the decision to target these funds to departments in medical faculties rather than support the creation of external research centres, as done in some other countries, led to a significant shift in the goals and priorities in medical schools. Medical schools were no longer primarily concerned with undergraduate education, "Less emphasis was given to teaching the undergraduate medical student, and more to the research process and recruiting and developing faculty whose major interests and skills were in biomedical research." (66) ' p  4 9  This investment in biomedical research also contributed to a lack of  interest of academic medical centres to foster and support physicians oriented to community health care. (67) ' p  4 4 4  This investment in research funding had, and still has, a  profound impact on the organizational culture of departments within schools of medicine as well as their position and status within the university culture and its governance. Schools of medicine have become economic engines for universities and those department heads overseeing biomedical research continue to have the most powerful voices shaping the medical curriculum, faculty recruitment, facility expansion and research activities. Having faculty members actively participate and contribute to new knowledge in their field kept faculty members abreast of new research in the field and provided an opportunity for disseminating new knowledge for the benefit of their students. However, the usefulness of such exposure was questionable depending on the research interests of a faculty member and the learning needs of a physician in training. The socio-economic and political forces influencing the prestige of certain departments over others, and the strength and forcefulness of individual faculty members representing their own  specialized research interests, has led to the creation of a curriculum that is "warped and uneven, with important elements entirely missing or referred to hastily and inadequately."  In 1945, Joseph Wearn, the Dean of Case Western Reserve University School of Medicine (CWRU), describes the state of turmoil at his institution as follows: .. .departments had become 'feudal duchies' that were completely independent of each other. Heads of the preclinical departments were 'in active competition with each other for teaching time and exam time'. Student-faculty relationships were very poor; 'students were terrified' of the faculty in general, and of the preclinical department heads in particular. 'Any department head could dismiss a student without consultation and without an appeal process.. .the education climate was dismal, with students caught in the middle between competing departments'. (54) pp. 47-8  This turmoil was not isolated to CWRU, but was, and in many cases still is, problematic at most medical schools. With each department competing for time and resources and faculty development, and a system of promotion based on research productivity, ones prestige and position were, and continue to be, subservient to ones ability to teach. Lewis and Sheps (65) ' ' pp  n  1 3 7  posit that by mid-twentieth century the  traditional core function of medical schools, that is teaching undergraduate medical education, became a by-product of ever expanding academic medical centres responsible for managing networks of schools (in some cases, including allied health professionals, and related diagnostic and technical personnel), teaching hospitals, clinics, laboratories, training facilities, and research centres. The medical faculty is not a coherent body. It is made up of groups and individuals with distinctive interests and commitments, whose strength is a function of their relative prestige and amount of money available to them. The money comes from many different sources, much of it directly from outside agencies with special interests, rather than an institutional budget allocation determined by the university or the medical school....In most academic medical centers, the diverse research and clinical interests of the faculty provide an  intellectual smorgasbord that does not reflect many health problems that play a significant role in the lives of the public. ( 6 6 ) ' p  137-8  Lewis and Sheps describe internal and external forces transforming medical schools into multi-million dollar academic medical centres and argue they are institutions in a state of chaos and crisis, essentially devoid of effective governance and public accountability, run by "a loose federation of duchies and principalities", focused too much on resource development rather than planned development in the interest of the public good. To correct this lack of public accountability and ineffective governance  18  they argue governments need to develop a comprehensive coherent health care delivery strategy that recognizes the role academic medical centres play in training healthcare professionals, cultivating research and delivering patient care and to build a more coherent and cogent structure to enhance policy development, operational support, and fiscal control to better serve the public's interest in physicians' education, research and ..  .  r^\ PP. 217-249  t  patient-care. (66) Lewis and Sheps and others (65) ' p  1 4 1  (3;4;9) assert that a fundamental problem in  undergraduate medical education, which is also shared by C M E , is a mismatch between the educational programs delivered and the needs of society. In undergraduate medical education, the focus on biomedical research has led to an overemphasis of subjects and topics tending to concentrate on the unusual diseases, rather than reflect the knowledge and skills necessary to address the more common diseases and conditions as well as reflect changes in demographics. In C M E program current needs assessment approaches, for the most part, are based on the self-interest of practitioners rather than a systematic  Williams (1980, ) commenting on the position and role of deans states, ".. .the faculty shapes the dean—in i often picks the dean from "one of us"... [he continues, quoting a Harvard Professor] "The dean is just like a janitor, different. He sees to it that the leaks in the roof are repaired and the burned-out light bulbs changed."  analysis of data collected on medical errors, physician incompetence, and population health needs (see Chapter 3). Part of the problem stems from poor health services planning and the lack of standardized mechanisms for capturing data across the provinces. However, deeper structural, regulatory and policy problems contribute to situation. These problems become more evident when one compares the economic burden of illness in Canada in relationship to the level of funding provided for research purposes. Figure 3 below presents 1993 data on the distribution of direct and indirect costs associated with diagnostic categories. Figure 4 presents the level of funding that is provided for research by diagnostic category. There is a relationship between funding and the economic impact a disease or condition presents. Unfortunately it is mostly an inverse relationship, the greater the economic burden a disease has, the less funding it receives. To improve our practices we must not only develop better means of collecting data, we must develop mechanisms in the planning cycle that actively uses data available to better meet society's need for efficacious, cost-effective healthcare provided by well-trained health care professionals.  Figure 3. Economic Burden of Disease in Canada: Distribution of Direct and Indirect Costs by Diagnostic Category (68)  p  1 2  Cardiovascular Musculoskeletal Injuries Cancer Respiratory Nervous System/Sense Mental Disorders Other k \ \ \ \ \ \ \ \ \ N Digestive Ill-defined Endocrine and Related Genitourinary Well-patient Care Pregnancy Infectious/Parasitic Skin and Related Perinatal Conditions Birth Defects Blood Diseases 10  15  20  25  Cost ($ Billion) • Based on ICD-9  Direct ($44.1 B) S Indirect ($85.1 B)  categories  Figure 4. Research Share of Total Cost by Diagnostic Category (68)  p. 13  Blood Diseases Infectious/Parasitic Endocrine and Related Perinatal Conditions -$gEEgg&$8§§§& Nervous System/Sense Well-patient Care-J Cancer Genitourinary - j | g g | | | Cardiovascular Birth Defects - j j g & g  Digestive-g§282 Mental Disorders Skin and RelatedRespiratory-^ Pregnancy-§§ Musculoskeletal Injuries0.5  1  1.5  % of Total Cost for Diagnostic Category  Lewis and Sheps posit that population health needs of today are very different than health concerns at the turn of the century when biomedical research was much more relevant to addressing life threatening diseases responsive to antibiotic or other pharmaceutical interventions. Current population health needs require the creation of a  37  different type of physician, one who is also trained in bio-psychosocial medicine and health promotion in order to become a more effective active change agent promoting lifestyle change. (69;70) To actualize some of these changes will necessitate a realignment of research investment to further our knowledge base in the social sciences and humanities area. The current emphasis of clinical training in teaching hospitals, with its focus on tertiary care, has led to an under-emphasis of ambulatory care, yet as illustrated in Figure 5. below, less than 1% of the total of number patients seeking medical care at any point of time are found in tertiary environments. (65;71) Figure 5. Estimates of Illness in the Community and Types of Service Providers Sought (72) ' p  6 5 9  • Adult population at risk  1,000 •  750-  Adults reporting one or more illnesses or injuries per month  250'  Adults consulting a physician one or more times per month Adult patients admitted to a hospital per month Adult patients referred to another physician per month 1  Adult patients referred to a university medical center per month  The mismatch among population health needs, research investment and curriculum of undergraduate medical students therefore is not solely a medical education problem but reflects deficiencies in government leadership, health care planning, coordination and collaboration among all stakeholders involved in research funding, health professional 38  education, medical funding, and health services planning. In Canada, for instance, there is no central standardized data collection process for clearly identifying national and regional population health priorities, or tracking health professional competency. Most of the health care data available is from tertiary environments rather than ambulatory care. With little standardization among provincial data collection systems it is difficult to assess performance across provinces. The lack of investment in research in the behavioural and social sciences has left physicians in training and in practice without a sufficient evidence-informed knowledge-base to become more effective change agents to better address current population health needs. At a time when there is a growing concern and desire for greater accountability for research support, Canada lacks a coherent infrastructure to assess population health needs and facilitate a more systematic way to meet these needs. Since 1999 some action has been taken to standardize data collection.(73) These more recent developments will be discussed further in Chapter Six. Since the 1970s the European Commission office of the WHO, in particular, has voiced concerns about deficiencies in the training of health professionals and the need to reconceptualize the continuum of health professional training linking such training to population health needs, and to utilize pedagogical methods that foster lifelong learning, to remain competent, rather than the episodic C M E programming currently employed. (9;74) Present efforts in this field are often unsystematic, poorly supported, little influenced by contemporary educational science, episodic, focused more on transmitting new information than on improving competence and only incidentally related to the health and national health priorities. (9) ' p  Before these deficiencies of undergraduate and continuing medical education can be remedied, there needs to be a systematic framework in place to guide a comprehensive diagnosis of the many challenges facing current conceptions of health professional  training as well as a better understanding of the knowledge and skills available to create new models of education to support the delivery of efficacious and cost-effective preventive and curative care.  Organizational, Regulatory, Economic, and Environmental and Regulatory Factors From an historical perspective there are organizational, regulatory, economic, and environmental factors that have contributed to shaping the continuum of medical education. In Table 1 below I summarize and highlight some of these major forces shaping our current system. Table 1. Historical Factors Contributing to the Development of the Modern Medical Curriculum •  Lack of legal, organizational development and regulatory infrastructure resulting in unfettered growth in number of medical schools and little means to develop and enforce medical training standards.  •  Lack of government planning, leadership and funding to support and refine the multiple roles academic medical centres play in training medical personnel, providing medical services and supporting research activities.  •  Lack of representation of other stakeholder involvement including consumers in identifying societal health care and research needs to better link current activities to population health and information needs.  •  Mistaken adoption of the Hopkins Model as the gold standard for medical training leading to a rigid 2 plus 2 curriculum structure rather than Flexner's vision of working with educators to create more productive pedagogical models fostering the development of a scientific inquiring mind.  •  Need for better ways to identify areas of physician incompetence and linking curriculum development to address competence issues and population health needs.  The melding of all these influences created a system which has led to great advances in biomedical research, however it has generally failed to be responsive to the public's interest in meeting the population health needs in urban and rural communities with a sufficient workforce of well-trained primary care practitioners and specialists.  40  Professional Development of CME Research Practices Begins in the 1980s The lack of resources available for C M E development, planning and research coupled with the lack of a regional, national or international consensus on the purpose of C M E has resulted in each C M E provider creating their own agenda and has limited the professional development of the C M E field. However a number of developments in the 1980s contributed to improving research practices in the C M E field. In 1981, a growing professional interest in C M E research led to the birth of the first C M E journal in North America, Mobius, "dedicated to lifelong continuous learning in medical education" published by the University of California. The year 1986 represented another landmark year in C M E research with the release of the seminal monograph: Changing and Learning in the Lives of Physicians edited by Drs. Robert Fox, Paul Mazmanian, and Wayne Putman, as well as the initiation of a series of conferences, entitled, Research in C M E (RICME) organized by acknowledged leaders in the C M E research field, Drs. Robert Fox, Paul Mazmanian, David Guillion, Nancy Bennett, Wayne Putman, Jocelyn Lockyer, and David Davis, with the purpose of cultivating and promoting process and outcome-based research in the field. (63) These conferences brought together researchers in the field to share their research and discuss common issues and concerns. In 1988, Mobius, changed its name to The Journal of Continuing Education in the Health Professions (JCEHP) and became the official journal of the Society of MedicalCollege Directors of Continuing Medical Education, (later to be renamed, Society for Academic Continuing Medical Education), the Alliance for Continuing Medical Education (ACME), the Council on C M E and the Association for Hospital Medical Education. JCEHP not only acted as a communication vehicle publishing research in the  field, its editors began to shape and influence research practices and foster a recognizable community of C M E researchers. Two milestone events arose from formal and informal discussions at RICME. In 1989 an intensive one week exploratory planning conference was held in Banff, Alberta, with 20 participants exploring questions and issues in C M E resulting in a summary statement recognizing the need for C M E to be more responsive to the needs of the practitioner/learner informed by principles of adult learning, cognitive and educational psychology, to become more cognizant of external forces having an impact on C M E , to be an active agent influencing the practice of C M E and health care delivery, and to further develop practice-based delivery of C M E . (6) This 1989 conference laid the groundwork for the planning and implementation of a three day invitational consensus workshop held in 1991 at Beaver Creek, Colorado. The Beaver Creek conference, hosted by the American and the Canadian Medical Associations, was focused on a set of questions about physician learning, the assessment of physicians and the provision of C M E . The process was literature-driven with working groups initiating literature reviews in each of these areas. (6) The literature reviews and discussions for the topic area of physician learning focused on physician motivation, selfdirected learning, adoption of innovation, effects of age or stages of development, and how physicians learn from clinical encounters. The section on assessment of physicians focused on questions related to the effectiveness of chart audits in performance assessment, effectiveness of standardized patients and structured clinical examinations in competency assessment, as well as the question of how to identify incompetence. The section on the provision of C M E focused on the differences among C M E providers, the effectiveness of C M E activities, factors influencing physician participation in C M E , the  nature of individualized C M E , and the impact of practice environments on learning. The results of this consensus workshop, informed by later reviews, led to the publication of the seminal text, The Physician as Learner: Linking Research to Practice, edited by David Davis and Robert Fox. Aside from producing a state of the art review of the literature, the main thesis of the text was the recognition that in order to have an impact on the delivery, outcome, and evaluation of C M E interventions, major efforts were needed in intersectoral collaboration and research involving many stakeholders including (i) medical and other health care professions and their licensing, credentialing agencies and specialty societies, (ii) partners involved with quality assurance, utilization review, competency assessment, health services research and governments, (iii) educators and students of adult education including educational psychologists, and others interested in continuing professional education or involved with the continuum of medical education, and (iv) C M E providers and C M E researchers. (6) This seminal book, along with more recent evidence-based systematic reviews, provided the groundwork for planning, implementing and evaluating the case study described later in this thesis. The findings from these systematic reviews challenged beliefs, attitudes and values about current pedagogical practices in the field and further challenged the profession to utilize more evidence-informed educational interventions to enhance knowledge transfer. The call for pedagogical changes in the continuum of education has repeatedly been made throughout the past century. (3;53;75;76;77) Angela Towle (1), in a recent article entitled, Shifting the Culture of Continuing Medical Education: What Needs to Happen and Why Is It So Difficult? raises concerns that not much has changed in continuum of medical education since a talk she delivered 6 years ago on rethinking C M E for the 21  st  Century. Her vision in 1994 and reiterated 6 years later stated C M E must be: educationally effective in relation to health outcomes, planned systematically on the basis of needs assessment and prioritization, responsive to rapid changes in the world, inclusive of services providers and users, designed to promote self-directed learning and problemsolving based on a proven effective educational process and informed by the experience of others. Towle suggests that part of the answer is the need to bring consumer stakeholders into the educational process and proposes a clinical approach that fosters a shared decision making process. She also posits that the current stakeholders involved in the continuum of education need to reach a consensus on whether there is a need to shift the culture of C M E , what it might look like and asks the C M E community if there is sufficient interest in changing it. I suggest part of the challenge to changing the culture of the continuum of medical education, and in particular, C M E is a faulty understanding about knowledge acquisition and knowledge utilization. To better understand current practices, an historical perspective on medical pedagogical practices may help the reader understand the roots of current pedagogical practices and further recognize the need to change current practices based on newer models of learning and planning.  Pedagogical Factors Shaping Undergraduate Medical Education and CME Practices George Miller, a noted doyen in the field of medical teaching, claims that one of the most serious problems facing medical education is a deficiency in medical teaching which he posits is historically related to the lack of cross-collaboration between faculties of medicine and faculties of education. Miller ( 7 8 )  pp  '  5-20  provides an excellent  summary of individual academic and clinical leaders and single institutional attempts to improve the teaching of physicians. As early as 1811 the Austrian Imperial Commission  on Schools had issued " A n Order for the Establishment of Training Schools for Future Professors of Medicine and Related Sciences." Although there was recognition of the importance of pedagogical training to improve medical training there was little knowledge to determine what sorts of changes were required to enhance learning. In training, and especially in teaching large groups, the importance of form and method must not be underestimated. On the contrary, we must acknowledge their great pedagogical significance, and therefore spare no effort to perfect them. (78) p. 6  In 1901, the president of the Association of American Medical Colleges (AAMC), at the annual general meeting, acknowledged the need to introduce better methods of teaching and the need to study medical pedagogy. In 1908, during the period of Flexner's investigation in medical education, there was sufficient interest in improving medical pedagogy that A A M C also established a standing Committee on Medical Teaching to consider "pedagogic elements in medical education, with special reference to the training of medical teachers, methods of instruction employed and allied topics." Miller (78) ' p  9  reports there was little in the A A M C archives that suggested this committee engaged in any significant action. The development of medical pedagogy was in its infancy. Flexner, and a number of his critics, had grave concerns about the rigid lockstep curriculum and pedagogical 20  practices that became characteristics of schools as they reformed according to the Miller (1980) reporting on Theodore Billroth's 1876 book on medical education entitled, Lehren und Lernen der Medicinishen Wissenschaften (Teaching and Learning in Medical Education). Prior to 1876 most schools in the United States offered all subjects simultaneously and students repeated the same subjects and lectures during the second year (281). Harvard Medical School in 1871- 1872 introduced a three year graded curriculum with instruction in the basic subjects in the first year followed by applied and clinical courses. By 1890 approximately 30% of the schools switched to the graded program. By 1900 nearly all medical schools had a four year curriculum with time divided up equally between basic sciences and clinical disciplines, known as the 2 plus 2 lockstep curriculum which is still the dominant curriculum structure today. In 1909, The Committee of One Hundred under the auspices of the A M A ' s Council on Medical Education released a model medical curriculum consisting of 4100 hours of instruction, almost half, 1970 hours (48%), related to the basic sciences. Barzansky (1992, p. 22) states, "In actuality, about one fifth of the hours in the four year curriculum were devoted to anatomy (an average of 600-800 hours), 450 hours to physiology and biochemistry (in the best schools), 150 hours to pharmacology, and 500 hours to pathology and bacteriology. A year later, the A A M C Committee on Curriculum recommended 2,010 hours to the basic sciences and allotted 61% of these hours to laboratory work. 20  45  Hopkins Model. (58;59). Although profound changes did take place following the Flexner Report in "institutional settings, curriculum organization, subject matter content, and faculty credentials" Flexner's pedagogical advice embedded in his report was not incorporated. (78) Flexner, in his writings before and after releasing his report, asserted the lockstep curriculum was contrary to his belief that medical instruction should facilitate individual creativity, active learning and maintain 'academic flexibility'. Flexner was a proponent of active student learning noting that "purely didactic presentations were hopelessly antiquated." In Flexner'si925 book, Medical Education: A Comparative Study, speaking about the rigid lockstep curriculum, he states, "Anything more alien to the spirit of scientific or modern medicine or to university life could hardly be contrived." Flexner's own pedagogical concerns were echoed in the 1925 US Commission on Medical Education Report stating: Chief criticisms of the training in the medical sciences are directed against the presentation too early of too many details, often of temporary, miscellaneous, and inconsequential value, the overemphasis on the technical procedures of laboratory work, and the artificial separation of the subjects... The present system of detailed subject examination, which rely so largely upon memory and which are still popular in secondary schools and some colleges, tends to defeat the major purposes of training, which are not the collection of facts but the intelligent and discriminating use of knowledge which is applicable to a given problem. (56) ' p  37  Between 1921 and 1941 there were repeated surveys on medical education with many complaints centred on teaching and the lack of skilled teachers trained in pedagogy. (67) Historically, medical education arose from pedagogical beliefs prevalent at the turn of the 19 century and, except for a few instances in the mid-1950s and more recently in th  the last twenty years, have remained isolated from pedagogical developments occurring within other areas of education, or more recent research in medical learning and cognitive  sciences. When one considers the term, 'doctor' in latin means 'teacher', the predominate attitude was that a 'good' doctor or researcher would make a 'good' teacher.  Learning Theories Underpinning the Continuum of Medical Education In the late 1800s medical education teaching was heavily influenced by several assumptions on knowledge transfer arising from a curriculum rationale based on "faculty psychology ." Faculty psychology was based on two primary assumptions. Firstly the 21  belief that the mind was considered an organ like a muscle and learning was a product of mental exercise consisting of rote memorization and recitation, and secondly, the belief that certain subjects involve different exercise regimes and that students could apply this knowledge across all relevant areas. (79) In 1892, E.L. Holmes addressing the annual banquet of the Practitioners Club in Chicago, stated: 23  Despite the great advances in American medical education during the last few years, it still remains a painful fact that the methods of teaching in our colleges are unphilosophical and the means of instruction lamentably inadequate.. .What is needed to improve the instruction in our medical schools is .. .the addition to the usual corps of professors [of] a large number instructors similar in function to drill masters in the army. (78) The advocates of faculty psychology have dominated the curriculum field throughout the school system since the late nineteenth century and "loud echoes remain today." (79) p  '  7 6  In the 1890s the National Education Association in the US appointed three  committees to create curriculum policy: The Committee of Fifteen on Elementary Education, The Committee of Ten on Secondary School Studies, and The Committee on Pinar et al. (1995, p.73) stated the golden age of faculty psychology as being from 1860s - 1890 and is still present in the 1980s. Pinar, reporting on The Yale Report (quoting Sloan 1971, p.244) defines 'faculty psychology' "as a systematic and detailed psychological theory developed by some of Europe's outstanding Enlightment thinkers' who postulated human beings having three constituent faculties or powers (i) presence of will, (ii) emotions, and (ii) intellect. Key concepts underpinning 'faculty psychology' were discipline and furniture (referring to bodies of knowledge). "The aim of education is to expand the powers of the mind and store it with knowledge.. .The primary aim in a curriculum, then, should be to call into daily and vigorous exercise the faculties of the students" (Pinar et al. paraphrasing Cohen, 1974) B. Barzansky & N. Gevitz, Beyond Flexner: Medical Education in the Twentieth Century, 1992, p. 21 states, "Flexner felt that teaching the basic sciences in the context of their eventual clinical application had become [wrongly] discredited in the United States because unqualified teachers had turned such application into a mechanical drill. Miller (78) quoting E.L. Holmes' address at the annual banquet of the Practitioners Club in Chicago J.A.M.A. 18:114115, 1892 21  22  2 3  47  College Entrance Requirements. This period of curriculum development became known as the 'The Curriculum Committees'. Dr. Charles Eliot (1834-1926), the President of Harvard University, chaired the Committee of Ten and was considered a visible scholar associated with faculty psychology. (79) Dr. Eliot was also instrumental in the reorganization of medical teaching at Harvard. (56) William Harris (1835-1909), the United States Commissioner of Education between 1889-1906 and a prominent member of the Committee of Fifteen, was a leading proponent of the classical curriculum model "with a view to afford the best exercise of the faculties in their natural order, so that no one faculty is ...overcultivated or...neglected." (79) The earliest experimental work on learning was carried out by William James and his lead proponent, Edward Thorndike. Thorndike believed learning to be a physiological process a "situation (S) stimulates the nervous system, which in turn triggers a particular muscle or gland response (R)...." (66) ' p  2 2  Thorndike's physiological law of use and  disuse, the notion that connections between neurons are enhanced through exercise and diminished when not in use, supported beliefs among faculty psychologists of the importance of exercise and drill. However Thorndike , unlike faculty psychologists, 24  believed that learning transfer was possible from one situation to another only if the situation contained identical elements of content and the same patterns of procedures. (80) Thorndike's law of effect was the precursor of behaviourism depicting human behaviour in terms of stimulus and response, popularized by the early work of J. B. Watson (1913) and B. F. Skinner (1938). Behaviourists viewed all organisms, human and sub-human, as biological entities immersed in their environments and looked for linear, measurable cause and effect learning mechanisms. Tolman remarks [cited by Amsel  Thorndike's work contributed to a new conception of knowledge transfer and he was a recognized pioneer in the field of educational measurement and test development.  2 4  48  (81)], "Mental processes are, for the behaviourist, naught but inferred determinants of behaviour, which ultimately are deducible from behaviour." Watson (82) ' p  1 0 4  illustrated this belief when he stated:  Give me a dozen healthy infants, well formed, and my own special world to bring them up in and I will guarantee to take any of them at random and train him [/her] to become any type of specialist I might select - doctor, lawyer... Miller (80) describes behaviourists as follows: [Behaviourists] concluded that learning is basically a process of establishing relationships between a given stimulus and an appropriate response. The stimulus may be visual, auditory, or tactile, and the response motor or intellectual, but to each stimulus a fixed response is developed. As learning continues the response patterns accumulate and become more complex but they are always precipitated by stimuli of one sort or another [...] new learning is achieved through trial and error, which leads to the acquisition of new responses to further stimuli, and the synthesis of past and present experience into new and more complex forms of behaviour. p  4 7  Taba (71) in her seminal text, Curriculum Development and Practice, describes the behaviourist conception of learning as follows: In these behavioristic theories the higher mental functions have a very small place. Learning takes place largely by trial and error and conditioning. Thought, and individual differences in it, is secondary to the system of establishing responses. Motives can be controlled from without by conditioning, punishments, and rewards. Practice (or drill) is essential, especially when combined with applying the law of effect—that is, with rewards and punishments. Transfer is limited. A n individual transfers what he has learned in one situation to another one only if the two are similar in content or procedure. Since the behaviorists, such as Skinner and Mowrer, believe that a science of behavior must be built only on what is observable, this school will not consider such unobservable behaviors as purpose, thought, and insight. (71) ' p  From a behaviourist perspective the learner is seen to be a passive receptor with the teacher being the provider of the active stimuli shaping the learning environment and facilitating the firing of the appropriate synapses. The learning theory of the behaviourists was based on experimental scientific work with rats utilizing reward and punishment systems. With the increased interest in scientific investigation in other areas of science,  the scientific study of learning behaviour as described by Skinner and Watson was very influential between the late 1930s to the 1950s. The influence of behaviourism on medical teaching and training is, ironically, discussed by G. P. Meredith (83) in his humorous paper entitled, The Student Rat and the University Maze presented at the second annual conference of the Association for the Study of Medical Education where he portrays the inadequacies of behavourist learning theory by his clever juxtaposition reflecting on "the path of the student rat [within the University maze].. .from the D  19  experimental psychologist studying the learning of rattus rattus in the laboratory." ' v  Since it is hard to be scientific with children and students but relatively easy to be scientific with rats (who cannot answer back) the argument is that we should start with the rat, establish the basic equations of learning at that level and then gradually work up the mammalian scale adding whatever new variables are necessary at each evolutionary level until, by about the year 3,000 A . D . perhaps we reach the medical student (regarded by some as the lowest level of human life). (83) p 2 0  The behaviourists' understanding of the world was atomistic, mechanistic with a strong belief that all phenomenon belonged to a linear, rational universe that obeyed the law of cause and effect, a perspective popular with proponents of the philosophy of logical positivism. This was the milieu that gave birth to what Miller and Seller (66) coined the 'transmission meta-orientation' to curriculum development, "the function of education is to transmit facts, skills and values." ' p  5  There were alternate perspectives on learning during this period. John Dewey's seminal text, The School and Society (84), originally published in 1899, and later text, Democracy and Education published in 1916, countered the classical curriculum of routinization, memorization and recitation of mental discipline and promoted a curriculum based on the philosophy of inquiry and reflection with the purpose of enhancing active democratic practice for the betterment of humankind. (79) The premise  that one should focus on memorization of facts was antithetical to Dewey's beliefs about education as a means to cultivate social awareness and democratic vision to foster meaningful social change. The mere absorption of facts and truths is so exclusively individual an affair that it tends very naturally into selfishness. There is no obvious social motive for the acquirement of mere learning, there is no clear social gain in success.. ..Indeed, almost the only measure for success is a competitive one, in the bad sense of the that term - a comparison of results in the recitation or in examination to see which child has succeeded in getting ahead of others in storing up, in accumulating the maximum of information. (79) To Dewey, the focus of educational activity was more context-based than contentbased. Intelligence was a product of the individual's interaction with the social environment, particularly through problem-solving. Dewey's concept of problem-solving is rooted in the scientific method. In the first step of the problem-solving process the individual confronts a problematic situation that causes confusion or puzzlement he or she must resolve. In the second step the person must define exactly what the problem is. The third step, clarification of the problem consists of a careful examination or analysis of factors contributing to the problem. In the fourth step the person develops hypotheses or "if-the" statements that offer possible solutions and consider the possible consequences of each alternative. In the fifth and final step the person selects one hypothesis or alternative and implements it. If the chosen alternative is successful the person continues his of her activity; if the hypothesis does not work the individual selects another. (66) ' p  These new cognitive models of learning, rather than focusing on biological driven notions of cognition, were more concerned with 'mental events'. Learning was seen as a dynamic process concerned with the organization, processing, and storage of information. Taba (71) describes these more cognitive-based field and gestalt theories as follows: Another set of theories of behavior are referred to variously as the organismic, Gestalt, and field theories. The common feature of these theories is that they assume that cognitive processes — insight, intelligence, and organization — are the fundamental characteristics of human response, present even in the simplest perception of the environment. Human actions are marked by quality of intelligence and the capacity to perceive and to create relationships. This understanding of relationships steers man's actions. His responses are shaped by  his purposes, cognition, and anticipation. Man is also an adaptive creature who organizes each subsequent response in the light of his prior experience. ' p p  8 0 - 8 1  Miller and Seller (66) describe the cognitive family of learning theories as a blending of biological and cognitive perspectives, rather than viewing the physiological process as both the means and the ends of learning. ' p  4 7  These new cognitive perspectives saw learning as a process of discovery. [Cognitive] theorists have concluded that learning is largely a process of establishing the significance of sensory experiences, of finding and gaining understanding of relationships through the development of insight, which may be sudden and dramatic.... Learning according to these theorists is more rapid and more efficient in the presence of clear perceptual organization of sensory experiences. This means that the whole is not only greater than the sum of the parts but also precedes the parts in the act of learning, since the parts are derived by degrees from perception of the whole. The learner is seen as an active participant in the process of acquiring and using knowledge. The learning process therefore is an act of construction or more appropriately called 'reconstruction of knowledge'. Central to the vision of constructivism is the belief that we actively reconstruct our mental models of the world via schemas, otherwise known as 'cognitive maps'. From a constructivist cognitive perspective, learning is a dynamic process in which we project our "existing schemas or forms of knowing on a new situation, framing it and interpreting it according to what we already know." Learning is seen as a process beginning with simple schemas, gradually expanding, revising and linking them with other schemas. With each use or referral we build more complex representations of links and associations and deepen our knowledge and expertise base. (85) ' p  6 7  Learning is not simply a process of responding to stimuli, but  involves an active process attempting to make sense of things. As Soloway et al. (86) remarked, teachers' words are not simply engraved in the student's mind, after passing through the ear, they are acted upon and interpreted." ' p  1 9 0  Key concepts espoused by  constructivists are that understanding and learning are an active process that involve constructive or reconstructive processes which is generative in nature, involving assimilation, augmentation and self-reorganisation. This was the milieu that gave birth to what Miller and Seller (66) coined the 'transactional position' to curriculum development, "represented by the cognitive-process orientation, certain elements of the discipline orientation, and democratic-citizenship orientation." ' p  9 1  Flexner and Dewey shared similar pedagogical perspectives. Both believed knowledge is related to experience and requires active engagement and reflection rather than something that is passively received by students. Flexner believed medical schools should promote an atmosphere of scientific inquiry and provide greater flexibility concerning curriculum programming to better meet different learning needs of students. Kember et al. (87) discussing inquiry-based learning and the need for reflective thinking, quoting Dewey states: Reflective thinking, in distinction from other operations to which we apply the name of thought, involves (1) a state of doubt, hesitation, perplexity, mental difficulty, in which thinking originates, and (2) an act of searching, hunting, inquiring, to find material that will resolve doubt, settle and dispose of the perplexity. (87) ' p  1 0  Both Flexner and Dewey's pedagogical ideas regarding the need for an inquiry-based curriculum were also congruent with McMaster University School of Medicine's problem-based learning (PBL) curriculum, however current research suggests an incongruence between constructivist theory and how it is conceptualized and utilized in undergraduate medical education environments (discussed later in this Chapter). This more transactional approach to learning was supported by other noteworthy developments in curriculum development. The seminal work of Tyler (1949) Basic  Principles of Curriculum and Instruction, Bloom's (1956), Taxonomy of educational objectives: Cognitive domain. Burner's 1960s text, The Process of Education, Mager's (1962) popular book, Preparing Instructional Objectives, originally published as Preparing Objectives for Programmed Instruction, and Taba's (71) text, Curriculum Development and Practice all contributed to a more rational systematic approach to curriculum development. Tyler proposed a curriculum model to address four fundamental questions: What educational purposes should the school seek to attain? What educational experiences can be provided that are likely to attain these purposes? How can these educational experiences be effectively organized? How can we determine whether these purposes are being attained? To address these questions Tyler's curriculum model consisted sequentially of four steps: (i) specify objectives, (ii) select learning activities, (iii) organize learning activities, and (iv) specify evaluation procedures. Tyler's 1949 book became the backbone of teaching practices around the world. The book was translated into seven foreign languages and sold over 85,000 copies during 36 printings . (79) Tyler's framework will 25  be discussed in further detail in Chapter Three as current conceptions of C M E planning and accreditation are based on this framework. Bloom's seminal work, Taxonomy of educational objectives: Cognitive domain, more commonly called, "Bloom's Taxonomy" introduced a more cognitive-model of knowledge representation. Bloom's taxonomy provided a structuralist perspective on knowledge representation.  Tyler is also regarded as a pioneer of evaluation in education, the principles outlined in his 1934 text, Constructing Achievement Tests provided the basis for most evaluation efforts in common use today. (79)  The levels of knowledge representation identified by Bloom are as follows: •  Knowledge: recall of basic information  •  Comprehension: understanding the knowledge  •  Application: the ability to use generalizations and to apply principles to specific situations  •  Analysis: the ability to break down a concept or idea into its components  •  Synthesis: the ability to combine a number of unorganized elements into a unified whole  •  Evaluation: the ability to assess concepts, theories, and materials according to selected criteria.  This structuralist approach to cognition blends both transmission and transactional notions of learning. Bloom's taxonomy is a linear and sequential model with the idea that each lower step needs to be obtained in order to move to the next step. The first three steps, knowledge, comprehension and application, are the most developed components of Bloom's model. The first step is commonly interpreted as "knowing the facts", whereas the second and third steps are the process of contextualization, with comprehension representing a step where the learner understands certain relationships among sets of facts, and application representing the stage when a learner can apply this knowledge to a given situation. Under a more recent cognitive theory framework, Royer et al. (88) ' p  201  posits the  following types of measures of cognitive skills are considered: •  Measures of knowledge acquisition , organization and structure  •  Measures of depth of problem representation  •  Measures of mental models  •  Measures of meta-cognitive skills  •  Measures of automaticity of performance  •  Measures of efficiency of the procedures  26  The construct of knowledge comes from the root, to know, which gave rise to numerous other words from Latin and Greek such as cognition, note, and notion (Barnhart, 1986, p. 569).  26  55  These structuralist approaches to knowledge representation were also congruent with an alignment with a disciplines approach to learning. When the Russians launched Sputnik in 1957, the education field was under intense pressure to improve its practice especially in the sciences. In 1958, the US National Defense Education Act provided an enormous boost to a number of disciplines within and outside the education field by providing one billion dollars in federal aid "to upgrade the teaching of science, 35  technology, and foreign languages." (41) ' p  The National Academy of the Sciences, Air  Force, Rand Corporation, US Office of Education, American Association for the Advancement of Science, and Carnegie Corporation hosted a national conference, The Woods Hole Conference, inviting psychologists, scientists and mathematicians to develop a new curriculum program to enhance knowledge transfer, particularly in science and technology. The findings of the conference resulted in a new curriculum manifesto reported in Jerome Bruner's 1960s text, The Process of Education, outlining a curriculum theory based on the concept of disciplinary structure. He posited that each discipline has a particular structure and understanding each structure "enabled the student to understand how a discipline worked: how it understood its problems, what conceptual and methodological tools it employed to solve those problems...." (79) Although the 1950s and 1960s were a fertile period in the education field with over 290 books published in the curriculum field, unfortunately these rapid developments, for the most part, were not transferred to the business of educating physicians. (79) Only a few medical schools broke the discipline barrier between the faculties of medicine and faculties of education.  More Recent Attempts at Innovation in Medical Education Although there were many calls for curriculum renewal after the Flexner Report, it was not until the early 1950s that a few medical schools engaged in a substantial process of curriculum renewal to address a growing concern about the inadequacies of medical teaching and the disparity between medical school training and the practice of medicine. For the purpose of this thesis I will present a very short summary of two major attempts of innovation in medical curriculum, Case Western Reserve University School of Medicine (CWRU) and McMaster School of Medicine. Both initiatives were more cognizant of the learner being an active participant in the learning process, they led to significant pedagogical experimentation within some faculties of medicine and have a potential pedagogical contribution to make to the practice of C M E .  Curriculum Renewal at Case Western Reserve University School of Medicine (CRWU) C W R U was recognized by the Council on Medical Education, and later in the Flexner Report as a top ranked medical school in the North America. It was an early adopter of the two year basic sciences plus two year clinical sciences, known as the 2 plus 2 curriculum. It had extensive laboratories for teaching and research as well as full-time professors in the basic sciences. By 1945, when Joseph Wearn became Dean, it was clear that there were significant problems with departments (earlier I quoted Dean Wearn claiming that his departments had become feudal duchies actively competing for greater time and resources). To Wearn, the solution required a radical change to the curriculum and departmental structure.  57  A thorough re-evaluation of the curriculum is essential, with a view to improve the quality, efficiency, and coordination of the teaching. This will necessitate eradication of the sharp line now existing between clinical and preclinical years and a close integration of all departments in teaching — in short, a complete revision of the present methods of teaching . (67) ' 27  p  27  Dean Wearn struck a committee entitled, Committee on the Correlation of Instruction to report their findings to the newly formed General Faculty. The report highlighted the following deficiencies: a lack of correlation of content taught by the various departments, lack of understanding of the amount of teaching time devoted to certain subjects leading to some subjects being overemphasized while others neglected, a lack of effort to evaluate or improve teaching on a departmental level, the absence of consensus on what a graduate of the school should have been trained to do, and, a lack of information regarding other medical school education programs. (54) To address these concerns Dean Wearn recognized that the current organization structure and departmental leadership were part of the problem and his change strategy was to recruit five new department heads, which he accomplished over six years, with an explicit goal of faculty and curriculum renewal . 28  To assist him in this process of renewal, expertise was sought from noted educators in the 1940s and 1950s including "Ralph Tyler, Benjamin Bloom, Jerome Bruner, B.F. Skinner, Donald McKinnon and T.R. McConnell, among others." (78) In 1949, Wearn hired Hale Ham, a noted scientist and very popular teacher at Harvard, to coordinate the planning of the new curriculum. The integration of faculty of medicine members with pioneers in the education field led to the unique development and implementation of an organ-system based method of teaching the basic sciences linked to and relevant to Williams citing Wearn's Memorandum dated February 1945 to the hiring committee. Williams (1980, p. 461) quoting John Ellis, states, " . . .it is of interest that no other dean emulated the Wearn maneuver of directly displacing curriculum control from department chairmen to a general faculty and subject committee." 27  28  58  clinical practice. The creation of new broad education objectives (see Table 2) initiated a more interactive and flexible curricular program. Table 2. Broad Education Objectives for the Medical Education at (Case) Western University •  Curricular content should be selected for its importance and organized in a sequential and coordinated manner. Throughout the curriculum there should be correlated teaching concerning biology, the principles of medicine, and the care of the patient.  •  Content selection should be a continuing process, emphasizing basic principles, methods, and scientific evaluation of data. The teaching methods should be selected to fit the content.  •  The process of teaching should be carried out, "as a cooperative venture of faculty persons, who represent a variety of interests and departments and are organized as teaching groups to carry out a particular portion of the program."  •  Adequate numbers of faculty should be available, to allow "growth in research, teaching and education." Effective teaching should be "recognized and rewarded, be considered in selection and promotion, and that guidance be given to inexperienced or ineffective teachers."  •  The medical student should be treated as a "colleague and as a student in a graduate professional school who is given increasing responsibility for his own education..."  •  There should be longer contact between students and instructors.  Similar core educational objectives were recommended three decades later at the World Federation for Medical Education as part of the Edinburgh Declaration in 1988, and again at the World Summit on Medical and Continuing Medical Education in 1993 (26) which recommended twelve remedial principles to improve medical education including: the integration of sciences, teachers being trained as educators, the need for relevant educational settings, curriculum-based on national health needs, life-long active learning, competency-based education, etc. A similar call for curriculum renewal and the integration of sciences was echoed as well in Martson and Jones (53) report entitled, Medical Education in Transition Commission on Medical Education: The Sciences of Medical Practice, commissioned by The Robert Wood Johnson Foundation. Aside from creating an organ-based curriculum, C W R U created two other significant curricular changes, the creation of a community-based family medicine/obstetrics  59  experience program for first year students, and a student-directed scheduled free study time. The community-based medicine program provided first year students with exposure to 'live' ambulatory patients from day one. First year students were assigned to a pregnant mother and followed the mother's development and the baby to term, observed the delivery and participated in post-delivery consultations. Having clinical teaching in community-based medicine from day one was a significant departure from the 2 plus 2 curriculum program and the traditional reliance of clinical training taking place in tertiary environments. The later creation of "The Continuity Clinic" clerkship program emphasized the importance of ambulatory care and provided students with the opportunity to follow patients after hospital discharge, providing them with opportunity to observe the natural history of acute and chronic diseases, permitting the student to continue to follow the development of the mother and child they observed during their first year. The C W R U curriculum reduced the amount of laboratory time required, with core lectures occurring in the morning and three afternoons per week devoted to studentdirected learning with occasional special lectures on ethics, death and dying, home visits etc. Although C W R U introduced an emphasis on ambulatory care ten years prior to the notion of primary care specialists, it was one of the last medical schools to formally establish a Department of Family Practice. C W R U leadership, similar to other elite medical schools, believed that "good family physicians came of a student with the intellectual capacity and drive to qualify as a board-certified general internist or general pediatrician." (67) ' p  4 3 4  Since the creation of the Department of Family Practice, C W R U  has become a leader in producing graduates exiting into family and community-based practice.  CWRU's experiment in curriculum change had a major impact on medical schools in North America. By 1974, twenty-eight other medical schools, twenty-four in the US and four in Canada, had introduced interdisciplinary teaching, particularly in the biomedical sciences, as well as providing some clinical experience during the first two years. With the success of community-based training programs fostering an interest in primary care, McMaster Medical School is planning to adopt a similar community-based program in 2002 as there is growing evidence that community-based medicine programs result in more physicians choosing to practice as primary care physicians as opposed to those graduating from traditional programs. However, C W R U today suffers from many of the challenges facing medical education. Faculty, although perceived to be more humanistic, were not necessarily better teachers. Williams (67) ' p  4 7 4  states, "Clinical teachers often found it difficult to answer  the first question: What is it that you would like your students to learn in your course?" Promotion and tenure are still primarily based on research and publications rather than high performance of teaching. Although Wearn, as early as 1951, was interested in evaluation of medical teaching and created a subcommittee on evaluation, there was an ongoing struggle between evaluators and researchers. Reserve medical educators were interested in qualitative research in medical education whereas demands by the Commonwealth Fund in the early 1950s and members on the subcommittee were not interested in "nature" studies which resulted in little evaluation research being carried out. One of the main barriers to conducting research at C W R U was that the program was undergoing constant change making the evaluation of the continuing 'experiment' unmanageable. Other issues, common with educational research, include the very general nature of the stated objectives. In CWRU's case, objectives for each phase of curriculum  renewal were essentially "not too precise" nor clearly linked to a pedagogical conceptual framework which made any systematic measurement problematic. In 1956, Wearn revamped the subcommittee and applied and received support for the establishment of the Division of Research in Medical Education (DORIME). Although DORIME was prolific, DD  publishing many articles, Willaims (67)  343 4  " claims most of these articles were of  minor significance and did not adequately address major educational research questions or methodological and conceptual problems associated with education research.  McMaster University School of Medicine The next major medical curriculum innovation occurred in 1966 as McMaster University implemented a new medical school. Dr. Evans, the founding Dean of Medicine at McMaster, recruited Drs. William Walsh, William Spaulding and Fraser Mustard, who were colleagues of Dr. Evans, as well as Dr. John Anderson. These medical founders shared similar concerns regarding the highly fragmented selection and organization of knowledge and the belief that a new system was needed to address their perception concerning the lack of relevancy of traditional medical education for the practicing physician. The McMaster curriculum was patterned after CWRU's systembased curriculum with the addition of problem-based learning and tutorials. The founders of McMaster medical school initiated a survey directed to specialists and sought information about what basic science content the specialist felt was essential to practice. They ended up with "an elaborate highly detailed compendium of basic science knowledge with a marked lack of overlap from one specialty to another... [and realized] medical practice was so diffuse .. .it is impossible to identify any set of prerequisite facts that is applicable across the range of the profession." (54) ' p  From  this belief they proceeded to identify two major objectives for the McMaster undergraduate medical program (see Table 3).  Table 3. Founding Educational Objectives of McMaster University Medical School (54) pp. 62-63  •  To help students become effective solvers of problems, by enabling them to understand the principle essential to the solution of such problems, and by teaching them how to seek out and use the information acquired for their solution.  •  To foster attitudes leading to behaviour as responsible physicians and scientists in relation to patients, colleagues, and society. Such behaviour is marked by compassionate concern for patients, coupled with action to promote the public good when the physician is faced with an ethical decision.  The premise that good physicians are people who are good problem solvers became a driving force for two new movements in medicine. The McMaster model of 'problembased learning' (PBL) not only stimulated a number of medical schools to develop and adopt PBL as a curricular method of integrating sciences and clinical practice, but students also began actively searching and questioning the source and validity of material being taught, which ultimately provided the right environment for the more recent development of the evidence-based medicine movement in North America. (89)  Problem-Based Learning: Pedagogical Challenges and Controversies PBL had its roots in undergraduate medical education at McMaster University (and almost concurrently at Michigan State University's College of Human Medicine) in the late 1960s and was seen as an alternative to traditional instruction in the basic sciences that was increasingly perceived to be dehumanizing, demotivating, inefficient and even ineffective. In PBL, problems are organized in thematic blocks e.g. ischemia, inflammation etc. and are discussed in small groups. Specifically, students are presented with a problem consisting of a set of phenomena in need of an explanation. The problem has been specifically designed for instructional purposes. Eight to ten students, in a group guided 63  by a tutor, discuss the problem and generate learning issues that they believe are prerequisites for a better understanding of the phenomena in terms of its underlying processes, principles, or mechanisms. The learning issues are starting points for students. Students are provided time to engage in self-directed learning activities to gather information about the identified learning issues. In a second tutorial students report back information collected and an attempt is made to use this collected information to draw conclusions. The analysis of health problems as a means to learn basic sciences is believed to provide an integrated contextual understanding between basic sciences and clinical practice. Founders and proponents of PBL claim that their curriculum would (i) stimulate students' interest in the basic sciences more powerfully, (ii) improve medical problemsolving skills, (iii) increase understanding of basic science concepts and their relationship to clinical practice, (iv) be more satisfying for students and faculty, (vi) lead to better self-directed study and (vii) be more fun. (90) ' p  S79  To date, using a range of reliable and  valid outcome measures, there is no consistent evidence that the current utilization of PBL in an undergraduate program results in improved learning. There are a multitude of problems associated with evaluating PBL versus traditional instruction via individual studies in current literature synthesis and meta-analysis type reviews. Ross (91) ' ~ , in pp 34  5  an attempt to clarify problem-oriented learning activities, suggested a taxonomy differentiating among problem-oriented, problem-based and problem-solving curricula. Ross suggested problem-oriented curricula is one where problems are used as selection criteria for content (and method), problem-based curricula is where students work on problems as part of the course, and problem-solving curricula is where students are given specific training for solving problems. This taxonomy is not utilized in the field nor in  research. The lack of categorization of the variety of PBL instructional methods reduces the accuracy and generalizability of individual and systematic review results. (92) ' p  472  Differences in instructional design, lack of or inadequate reporting on instructional design variables, different methodology, inconsistent reporting of data collection, lack of reporting on test reliability, dissonance between findings and conclusions, make it difficult to interpret and generalize the findings of such research. The lack of robust results has resulted in some medical schools who adopted PBL curricula to revert to traditional curricula programs. Slavin (93) ' p  states that part of the problem with meta-  analysis type reviews is that they focus the reader on a few unqualified effects, for example, often the effects cannot really be understood without considering interactions and confounded variables. The specific application of PBL is poorly defined in the literature. On the macro level each school and/or course interprets and implements PBL in different ways in their curriculum, and on the micro level each instructor and student implements and responds to PBL in unique ways. The lack of a clear classification system identifying the key instructional methods presents a much higher possibility of confounding and makes researchers' discussion of small positive results, though not statistically significant, more suspect. Berkson (90) ' ° in reviewing whether PBL teaches problem-solving better than p  S8  traditional curricula, reported that experiments attempting to measure hypotheticodeductive or forward thinking processes failed to provide evidence that PBL students were better able to employ either type of reasoning more effectively (resulting in the correct diagnosis) than students from traditional curriculum. For instance, Patel et al. (94), studying the effects of conventional and PBL curricula on problem-solving reported  that McMaster medical students discussing a written case describing a patient with acute bacterial endocarditis hypothesized more but came to the correct diagnosis less often that did a group of (not clearly comparable) McGill medical students. Berkson (90) ' ° reported on studies which demonstrated that attempts to measure p  S 8  the performance of individual components of the hypothetico-deductive model, sometimes measured by patient management problem (PMP), proved highly case-specific and not generalizable, and tutoring in the individual components did not effect proficiency. Berkson also reported on a study evaluating a curriculum designed to facilitate hypothetico-deductive reasoning employed by fourth year medical students from a traditional curriculum at the University of Michigan and the problem-based curriculum at Michigan State University that showed no difference between the treatment group and controls in their use of the different components of hypothetico-deductive reasoning or in their ability to benefit from instruction in the relevant heuristics. Berkson concluded that there is a lack of definition regarding problem-solving powerful enough to guide the development of measurement tools, let alone instructional programs. There is little evidence to support the supposition that problem-based learning enhances effective decision-making. Further research is needed to clarify and measure problem-solving capacities. Differences in Novice and Expert Problem-Solving Methods Possibly relevant for use of PBL in C M E were experiments in non-medical fields. For example, Berkson (90) ' ° reporting on a study by Larkin et al. (1980) investigating p  S8  expert versus novice performance in solving physics problems, demonstrated that "experts employed sophisticated abstractions that subsumed variable amounts of data and from which they extrapolated conclusions", this being an example of forward reasoning.  Novices, those lacking knowledge and/or experience within a content-specific domain, were unable to efficiently and single-mindedly organize the data and employed hypothesis-driven reasoning. There was some evidence that novices using hypothesisdriven reasoning for certain problems may have greater difficulty with utilizing forward reasoning later. Our lack of understanding of problem-effectiveness and overall problemsolving competency, as well as the lack of test reliability reported in measures used, make it difficult to generalize these results. In spite of the rapid growth in the implementation of new medical curricula, there is confusion about what PBL is and whether there is significant value in using PBL instead of traditional instruction in undergraduate education. However, there are significant differences between undergraduate medical students using PBL, and practicing medical physicians using PBL with other experienced and knowledgeable colleagues. There is some evidence to support the hypothesis that PBL may be better placed in a C M E environment. Berksen's review suggests that 'expert's', practicing physicians, may be more effective utilizing a problem-solving format than novices. P B L and Implications for C M E The Kaufman and Mann's (95) study comparing student perceptions regarding their courses in PBL or conventional curricula demonstrated that PBL students rated their courses significantly higher than (p. 001) students in conventional curriculum on 11 of 12 items. This higher level of satisfaction within undergraduate medical students may also be translated to higher satisfaction within a C M E environment. Research on the information seeking behaviour of physicians suggests that they more likely seek advice from colleagues than any other source (96;96) and a PBL environment might provide another opportunity to fulfill this need.  Berkson (90) ' p  suggests that there is a lack of coherence between the current  application of PBL in undergraduate studies and the theoretical framework based on 'constructivism', and suggests that the unqualified extrapolation from the cited theory to current practice may not be sound. There are, however, new developments in the cognitive sciences and learning theory that may help us better link adult learning theory to a conceptual framework concerning learning processes and lead to a more informative framework to support further research in our understanding about medical learning.  New Developments in the Cognitive Sciences and Learning Theory I will be discussing three recent conceptions regarding learning (i) neurocognitive, (ii) cognitive production theory, and (iii) social cognition. The following summary also illustrates some confusion among theorists and researchers across disciplines which I believe is a result of an underdeveloped taxonomy about what is meant by 'learning' 'knowledge' and the processes involved in its acquisition and utilization. The problems of an unclear taxonomy, regarding different types of 'knowledges' and the different 'dimensions' involved in the acquisition processes, leads to similar confusion as illustrated in the following taxonomy of the animal kingdom attributed to an ancient Chinese encyclopedia entitled, The Celestial Emporium of Benevolent Knowledge: On those remote pages it is written that animals are divided into (a) those that belong to the Emperor, (b) embalmed ones, (c) those that are trained, (d) suckling pigs, (e) mermaids, (f) fabulous ones, (g) stray dogs, (h) those that are included in this classification, (i) those that tremble as if they were mad, (j) innumerable ones, (k) those drawn with a very fine camel's hair brush, (m) those that have just broken a flower vase, (n) those that resemble flies from a distance.[Borges, 1966, cited byRosch (97) ' p  1 0 8  p  27  Neurocognitive Psychology The neurocognitive development model arose from Hebb's connectionist theory and views the learning of a skill as an interlocking set of standard operating procedures that  have been inscribed on the nervous system and 'expertise', an extension of a skill, as a refinement of years of experience. Hebb, as described by Waldrop (98), describes learning and memory as the process of the brain taking in random data and immediately commencing to organise it. He states: A sensory impulse coming in from the eyes, for example would leave its trace on the neural network by strengthening all the synapses that lay along its path.... Experience would accumulate through a kind of positive feedback; the strong, frequently used synapses would grow stronger. He then postulated that the stronger pathways would then cause the brain to form cell assemblies - subsets of several thousand neurons in which the circulating nerve impulses would reinforce themselves. (98) ' p  158  Some neuroscientists believe that all aspects of the mind, including consciousness, are likely to be explained in a more materialistic way as the behaviour of large sets of interacting neurons. (99) ' p  153  Kandel and Hawkins (100) ' p  79  claim that "learning  engages a simple set of rules that modify the strength of connections between neurons in the brain." To the neuroscientist consciousness and meta-cognition is principally a neural phenomenon. Former more mentalistic psychological models, have been reclassified as 'folk psychology' whereas the real 'psychology' is based on biological modeling and biochemical investigations. Neuroscientific perspectives are primarily concerned with reductionist phenomena such as brain processing and the demonstration of information processing through biological modeling. The description of biological processes and theoretical constructs which attempt to 'make sense' of the phenomenon sometimes, unfortunately, gets interpreted as an explanation of the phenomenon itself. Part of the problem with this perspective is that the construct of 'evidence of learning' is poorly defined and generalised so that all chemical changes are equated to demonstrate learning. Neurocognitive investigations into learning concentrate on the neurophysiological level of information processing including biological aspects related to motor/perceptual  control and abnormal brain function. This microscopic perspective has provided us with some greater insight regarding certain types of learning and dysfunction. In addition there has been some limited cross-discipline research within the human-computer interface community integrating aspects of neuropsychology with human behaviour, although the scope of these investigations have been quite limited (positioning, display qualities, attention factors etc.).  Cognitive Production Theory and Related Learning Theories Anderson (42) ' p  1 9  in his influential text, Architecture of Cognition (cited >1900  times between 1983-2002 Social Sciences Citation Index) implicitly defines knowledge as that which can be remembered or acted upon. He proposes a cognitive production theoretical model for knowledge representation. I will outline some aspects of his model which will be pertinent to later discussions concerning content knowledge and learning contexts and the need to create a more grounded taxonomy for knowledge domains which may be more 'fruitful' than current conceptions for the purpose of curriculum planning in C M E . Of particular importance is the differentiation between the novice learner and the experienced learner and the need to move beyond the notion that the application of generic adult learning principles to the context of the C M E environment will improve learning. In 1983, Anderson (42) ' p  1X  proposed a theoretical model entitled, Adaptive Control  of Thought (ACT) which is a theory of basic principles of operations built into the cognitive system. His original framework has been modified and discussed in his later book, Rules of the Mind (101), published in 1993. He has coined his modified framework A C T - R ("R" for rational). His 1993 work incorporates necessary changes to be consistent with new research findings in cognitive psychology.  Unique to Anderson's A C T and A C T - R theories is his differentiation between the acquisition of declarative knowledge and procedural knowledge. Whereas a 'fact' may be committed to memory after a few seconds of study, in contrast it appears that new procedures can be created only after much practice. It is difficult to explain this huge discrepancy if both types of knowledge are encoded in the same way. (42) ' p  2 3  Declarative knowledge includes propositional knowledge i.e. the knowledge of association (that Joe hates Bill), spatial images (triangles), or strings of information (numerical order). (42) ' p  2 3  Declarative knowledge includes knowledge that people can  describe or report, whereas procedural knowledge is manifest in performance. Anderson (42) ' p  2 1 5  describes procedural knowledge as developing primarily through the act of  executing the skill; one learns by doing. When we think of skills we tend to think of motor skills like bicycling, however cognitive skills include activities such as decisionmaking, mathematical problem-solving, computer programming, clinical skills and language generation. Knowledge can involve an integration of both declarative and procedural knowledge. An activity such as typing initially involves memorizing the keyboard as a form of declarative knowledge, however, its automaticity, learned through the act of doing, may lead, though not necessarily, to a reduction of declarative knowledge. This can manifest itself in a typist needing to think of her/his finger placement to recall a letter position, rather than just mentally recalling its position on the keyboard. Most procedural knowledge involves moving from an initial transfer of declarative knowledge through a process of "doing" being transformed to procedural knowledge. The repetitive act of doing leads to a decrease of cognitive processing in a similar way that driving a car, after practice, requires less cognitive processing related to the act of doing.  There is significant research supporting this claim of disassociation between declarative and procedural knowledge especially on patients who have suffered from brain trauma or disease. The most notable case being patient H M who lost the ability to acquire new declarative knowledge, but was able to acquire new skills, or other patients who acquired the procedural skill of reading words but had substantial impairment in remembering words they had read. (100; 102) Anderson's framework for knowledge representation rests on three theoretical assumptions (see Table 4). 23  Table 4. Anderson's Production Theory for Knowledge Representation (42)  p  •  there are two long-term repositories of knowledge: a declarative memory and a procedural memory  •  the 'chunk' is the basic unit of knowledge in declarative knowledge  •  the 'production' is the basic unit of knowledge in procedural knowledge  Chunks represent an encoded set of elements in a particular relationship (propositional, temporal or spatial). In complex knowledge structures one cognitive unit (also called chunks) appears as an element of another. The interconnections of these structures and elements define a network or schema-like structure. Anderson (42)  p  '  2 3  suggests that each of these units could be called 'nodes' and that the connections between them could be called 'links'. Declarative knowledge building is seen as a process by which "a chunk is to be represented as a pattern of activation that associates the elements of the chunk and the context information." Anderson's model of cognitive skill development is based on the theoretical assumptions described below (see Table 5).  72  Table 5. Theoretical Assumptions: Development of Procedural Knowledge •  the knowledge underlying a cognitive skill begins in an initial declarative form (an elaborated example) which must be interpreted (problem-solving by analogy) to produce performance  •  as a function of its interpretive execution, this skill becomes compiled into a productionrule form  •  with practice individual production rules acquire strength and become more attuned to circumstances in which they apply  •  learning complex skills can be decomposed into learning functions associated with individual production rules (For a further elaboration of his work, see Anderson 1983, 1993)  Anderson's theory of cognition is also congruent with recent research investigating physician thinking. Irby's (103) model of clinical skills learning is based on an assortment of cognitive models and learning theories discussed in the medical literature, most notably Patel and Groen's text, Toward a General Theory of Expertise (94) and Lemieux and Bordage (104) describing semantic structures and models for representing semantic structures of medical knowledge and thinking of experts and novices. Bordage and Lemieux, as described by Irby, posit that the structure of medical knowledge can be represented as follows (see Table 6).  Table 6. Structure of Medical Knowledge (103) Compiled High Knowledge High knowledge, clinical prototypes, pattern recognition Elaborated Knowledge High knowledge, clinical prototypes, clinical reasoning in context Dispersed Knowledge Encyclopaedic knowledge, little conceptual integration, out of context Reduced Knowledge Little, disorganized knowledge, inaccessible and out of context Groen and Patel (105) ' pp  293  "  294  i n  discussing the relationship between comprehension  and reasoning in medical expertise, cited a series of studies (Johnson, 1970, Patel and Frederiksen, 1984 and Frederiksen 1979, 1981) which all noted that in routine cases, 73  similar to those that a physician frequently encounters, experts make more inferences from highly relevant information than do novices. On the other hand, novices both recall and infer more information of low relevance than do experts. Alternatively, when the case-based problem is scrambled, the differences between experts and novices disappear. Medical experts presented with routine problems are quickly able to differentiate between relevant and irrelevant information as it pertains to the task of making a diagnosis. Therefore testing simple recall or number of inferences made may bear little relevance to the ability to solve the problem nor accurately assess knowledge acquisition. This type of expert knowledge however, is different from other studies exploring differences between novices and experts in different areas of study. For instance, de Groot (1946) cited by Berkson (90), investigating recall among advanced chess players, masters and grand masters with mid-game positions, required them to recall the positions of the pieces on the board, and showed a strong correlation between level of expertise and increased ability to recall. However, with physicians it has been repeatedly shown that they have less recall than students. It is suggested that physicians, in fact, create different representations of knowledge due to a process coined as 'knowledge encapsulation' which is the packaging of lower level detailed propositions, concepts and their interrelations in an associative net under a smaller number of higher level propositions with the same explanatory power. (106). Therefore physicians, analyzing a problem, translate elaborate information into 'chunked' highly inclusive concepts. In medicine the encapsulating concepts tend to be of direct clinical relevance. For instance, an internist would take a list of symptoms and chunk them to 'patient has a septic condition'. A recent reformulation of Van Dijk and Kintsch's theory (1983) of text comprehension elucidates why current theories of text processing have difficulty explaining this anomaly. Kintsch's construction-integration model deviates from existing schema-based  conceptions of text comprehension in that it conceptualizes the processing of text as largely bottom up. It assumes that knowledge is not prestored in fixed structures, but generated in the context of the task for which it is needed. A reader attempting to understand text, words or phrases will activate their corresponding nodes and this activation will spread to other related nodes. In this way a pool of knowledge is activated that may or may not be relevant to the task of understanding the text at hand. Kintsch describes this as the construction phase in text comprehension. Further reading and hence activation of concepts and their interrelations will, however, constrain the meanings of what was previously read by deactivating irrelevant knowledge. Thus a coherent representation of the text formed occurs, integrating knowledge from earlier cycles of activation with those from later cycles of activation. In the course of this integration process the reader may have to produce bridging inferences and form macro propositions to create or maintain coherence. Prior knowledge and information from the text itself thus becomes integrated into a text base that represents meaning. Boshuizen and Schmidt (106) posit it seems reasonable that the more prior knowledge a person has the less time is needed to construct a coherent text base. They suggest that experts can be expected to produce less bridging inferences, simply because these are already part of their knowledge base and will be generated automatically. This in itself does not explain the lower recall. Groen and Patel (105) suggest that subjects reading a text about a patient transforms the discourse into different kinds of representations, such as, the text base, which is semantic representation, and a situation model which is a cognitive representation of the events, actions, persons or situation. Boshuizen and Schmidt suggest that novices and intermediates process a clinical case in a way described by Kintsch. They also report that, whereas novices and intermediates  refer to biomedical concepts while reasoning aloud, physicians rarely do and suggest that these concepts have become encapsulated into higher level concepts. Berkson (90) ' ° , reporting on a study by Larkin et al. (1980) investigating expert p  S 8  versus novice performance in solving physics problems, demonstrated that "experts employed sophisticated abstractions that subsumed variable amounts of data and from which they extrapolated conclusions." This is an example of forward reasoning. Novices, those lacking knowledge and/or experience within a content specific domain, however, were unable to efficiently and single-mindedly organize the data and employed hypothesis-driven reasoning. There was some evidence that novices, using hypothesisdriven reasoning for certain problems, may have greater difficulty with utilizing forward reasoning later. These notions of encapsulated knowledge, or Irby's idea of'compiled higher knowledge' are similar to Anderson's idea of procedural knowledge and his claim that the transformation of declarative knowledge to procedural knowledge may lead, though not necessarily, to a reduction of declarative knowledge. It appears that experts are better able to filter out irrelevant information from a case. Anderson's theory is also similar to a constructivist cognitive perspective, although constructivist proponents have not grounded their cognitive modeling of learning adequately differentiating from an operational perspective the acquisition of declarative and procedural knowledge. The benefits of Anderson's cognitive production theoretical model of knowledge representation are threefold, (i) unlike some biological models such as Hebb above, Anderson does not confuse the theoretical model of knowledge representation with the end product of learning, (ii) it is congruent with current research exploring differences between novice learners and expert learners, (iii) it provides a more promising framework to explore the development of physician tacit-knowledge and  pattern recognition to enhance our understanding regarding differences between novice learning and expert learning, which might help resolve some of the practical challenges of using PBL in undergraduate environments.  (107)  A Model of Social Cognition Social scientists, as represented by Butterworth (108), believe that knowledge acquisition is principally a social phenomenon and that the transmission of knowledge is embedded in complex value systems where information gained within one context is not necessarily easily transferred to another, from even within the same subject domain. For instance, Butterworth (108)  states that the classical Piaget perspective focused 29  30  on logico-deductive processes within the individual child and Vygotsky's perspective on problem-solving viewed thought processes and cognitive growth as being just socially-situated. Recent work on sociocognitive development has moved away from viewing contextual factors as being moderators to the learning process, to the position that contextual constraints are "inextricably linked to the acquisition of knowledge." In socioculturalism, learning is seen as the process of enculturation. Learners become collaborative meaning-makers among a group defined by common practices through Piaget's development theory drew heavily on his training in biology. Eklind (1989), reporting on Piaget's development theory stated, "Mental growth is inseparable from physical growth: the maturation of the nervous and endocrine systems, in particular, continues until the age of sixteen." Piaget coined the term "genetic" epistemology to describe his philosophy on how we know the world. Piaget (1969) believed human intelligence grew in a series of stages related to age and could not be hurried. His stages of development sensorimotor, pre-operational symbolic, concrete operational and formal operation were based on reflecting how the child deals with concepts of space, time, causality and logico-mathematical constructs. Piaget's description of how children learn utilized biological terms of accommodation and assimilation. Accommodation is the process which causes behavioural changes caused by the child adapting to external forces. Assimilation is the process of internalization which is most easily seen in watching children play. These concepts of assimilation and accommodation provided a new construct of learning theory. Learning was seen as an interactive, rational process that provided a basis for a new transactional model for understanding learning and development. 29  Vygotsky (1978), expanded Piaget's concept of intelligence by looking not only at the child's degree of organic development but also at the unity of perception, speech and action. He viewed the learning process as the mastery of mediating tools leading to high mental flexibility and problem solving. He expanded the biological model to include both a biological and a cultural perspective. He believed that instruction was critical and that timing and content were very important. Vygotsky devised the concept of "zone of proximal development", the distance between what the child could do by itself versus what a child could do with assistance. His primary concept of learning was to challenge or stretch the child's current skills through introducing gradually more difficult levels of problem solving. 30  77  language, use of tools, values and beliefs. (86) ' p  Bandura (109) in his seminal text,  Social Foundations of Thought and Action: A Social Cognitive Theory, describes social cognitive theory as follows: Social cognitive theory embraces an interactional model of causation in which environmental events, personal factors, and behaviour all operate as interacting determinants of each other. (109) ' p X1  Whereas most psychological approaches to learning look within the individual to understand how learning occurs, the central idea of social learning theory is to look outside the individual at specific types of information exchanges (verbal and non-verbal communication) with others to explain how behaviour changes. (110) The differences among sociocognitive, neurocognitive and cognitive models of learning are important, however, they are not necessarily, from a theoretical perspective, in opposition with each other. ( I l l ) Anderson's theoretical modeling of cognition does not exclude sociocognitive factors in knowledge acquisition as schemata are considered contextually rich chunks. The construct of'sociocultural cognition' places the emphasis on the sociocultural contextual ways knowledge is constructed whereas a neurocognitive framework attempts to model the information processing aspect of cognition. The notion of highly developed automated processing of high level pattern recognition, a form of procedural knowledge, is also reflected in research and discussion concerning the concept of embodied knowledge and skills, and constructs of tacit-knowledge, particularly in understanding everyday decision making and planning. (112-114) If one accepts the idea that knowledge is not simply a transmission of information but rather a complex cognitive and enculturation process, involving different cognitive processes dependent on the experience level of the learner and the familiarity of the context, then planning an educational intervention should involve a comprehensive  diagnosis of the experience level of the targeted audience as well as a contextual diagnosis identifying administrative, organization, regulatory, policy, peer and professional factors as well as individual attitudes, beliefs and practice routines that could hinder or facilitate learning. In practice, however, there is little difference between teaching practices in undergraduate medical education (novices) and C M E (experienced practitioners), which is not surprising when one recognizes that formal C M E practices arose from the same pedagogical beliefs underpinning undergraduate medical education.  Merging of Learning Theories, Diffusion Theory and Models of Knowledge Transfer Theories and knowledge about diffusion have benefited from a recent convergence of diffusion research in the fields of anthropology, sociology, rural sociology, education, public health and medical sociology, communication, marketing and management and other disciplines. (110) Knowledge transfer research and theory, although related to diffusion research, arises from other disciplines, particularly industrial psychology, and educational psychology. Whereas diffusion research is primarily addressing what factors influence knowledge uptake and its utilization, knowledge transfer theory and research tends to be is more focused on the conditions of transfer, integrating cognitive learning theories regarding knowledge acquisition and how trainees apply the knowledge, skills and attitudes gained in a training context to the job as well as how these attributes are maintained over time. (115) The traditions behind these different disciplines arise from different cognitive models. Roger's(l 10) claims that diffusion theory has many similarities to social learning theory, and that both share a central belief that "stress information exchange as essential to behaviour change, and view social factors as the main explanation of how individuals  alter their behaviour." Knowledge transfer research arising from industrial psychology tends to focus on inputs, such as instructional methods, and sequencing of training materials using principles of learning based on more neurocognitive learning models. Knowledge transfer research recognizes the importance of social systems, however views the social systems as moderators influencing knowledge uptake especially regarding reinforcement and maintenance of knowledge learned, rather than viewing knowledge acquisition as principally a social phenomenon (see Figure 6 below).  Figure 6. A Model of the Transfer Process (115) Training Inputs  Training Outputs  p  65  Conditions of Transfer  Trainee Characteristics • Ability • Personality • Motivation 2  Training Design • Principles of Learning • Sequencing • Training Content  I  Learning & Retention  Generalization & Maintenance  31 Work Environment • Support • Opportunity to use  Baldwin and Ford (115) describe the transfer process as input and output factors using behavioral terms common to the field of industrial psychology. Baldwin and Ford categorize input factors as those related to trainee characteristics, training design and work environment variables. Training outputs and training input factors, they posit, have  direct and indirect effects on conditions of transfer. Their model describes six conceptual links among the different components. Table 7. Conceptual Links Proposed by Baldwin and Ford on Knowledge Transfer and Utilization (115) •  In order for trained skills to be transferred training material must be learned and retained (linkage 6).  •  Training characteristics and work environment factors are hypothesized to have direct effects on transfer regardless of initial learning and retention. For instance, lack of motivation or lack of supervisory support may hinder transfer, (linkage 4 and 5).  •  All training inputs (trainee characteristics, training design and work environment) impact learning and retention (linkage 1, 2, and 3).  Baldwin and Ford (115) in their critical review of knowledge transfer research, used the above framework to classify the different types of research activities represented in the field in reference to learning principles commonly held within the field. In this review they present information about the research studies cited in support of a given learning principle (or lack of research). They also discuss limitations of current research with the intent to determine gaps in the knowledge-base of the field and specify the types of research needed to further understand the transfer process. While it is relatively straightforward to operationalize principles such as overlearning in controlled experimental settings with motor or memory tasks, the appropriate operationalization of learning principles in more complex organizational-training programs is problematic. For example, there is no empirical data regarding how much and in what ways a trainer should incorporate learning principles such as stimulus variability into a behaviour-modeling program to enhance the transfer of managerial skills. In addition, W. Schneider (1985) suggests that several training-design maxims (e.g. practice makes perfect) are fallacious when training for "high performance" skills. (115) Baldwin and Ford note a number of critical limitations of past research efforts which support my argument that a better conceptual framework and taxonomy of 'knowledges' is needed to better understand the linkages among cognitive models and their relationship to knowledge transfer and diffusion. A major part of the problem with knowledge transfer  research, Baldwin and Ford (115) posit, is the tendency for knowledge transfer research to focus on training input factors rather than to develop more appropriate measures of the condition of transfer. To develop appropriate measures of generalization requires a linkage of needsassessment information, the specification of training objectives, and the determination of criteria to determine how much of the knowledge, skills and behaviours learned in training are transferred to the actual job. In addition, the relevance of the skills learned for effective job performance must be determined. (115) The concern regarding the deficiencies with current measures for assessing knowledge retention and its application is not unique to researchers engaged in knowledge transfer research. A significant problem with all current cognitive models is that no one model provides a reliable and valid taxonomy of knowledge representation with reliable criteria for testing different levels of knowledge representation. (88;90;116)  Use of Diffusion Theory in the Medical Field Diffusion in the medical field has been extensively investigated addressing uptake of new pharmacological products, new surgical procedures, computer information systems, and, more recently, the uptake of clinical guidelines. Davis et al. (6) posit that most medical studies investigating knowledge and behaviour transfer have based their conceptual model on Rogers (117) seminal text, Diffusion of Innovation (see Figure 7 below).  Figure 7. A Model of Stages in the Innovation-Decision Process (117) Prior Conditions  1. 2. 3. 4.  Previous practice Felt needs/problems Innovativeness Norms of the social systems  I. KNOWLEDGE.  Characteristics of the Decision-Making Unit  1. Socio-economic characteristics 2. Personality variables 3. Communication behavior  COMMUNICATION CHANNELS  7'  'r'  II. PERSUASION  ra. DECISION  IV. IMPLEMENTATION  I I  I  Preceived Characteristics of the Innovation  1. Relative Advantage 2. Compatibility 3. Complexity 4. Trialability 5. Observability  |  i i  V. CONFIRMATION  *• 1. Adoption  - Continued Adoption • Later Adoption  • 2. Rejection -  • Discontinuance Continued Rejection  I  Rogers' diffusion model has been used in multiple fields and contexts investigating individual and/or organizational change and the adoption of new ideas and technologies. Roger (110; 117) proposed a series of stages in the innovation process recognizing that diffusion occurs over time rather than as an instantaneous act. He conceptualized the decision-making process through which an individual or group (i) has first knowledge of an innovation, (ii) forms an attitude toward the innovation, (iii) makes a decision to adopt or reject the innovation, (iv) implements the innovation, followed by, (v) confirmation, or rejection of this decision. His model also presented a number of predisposing factors influencing adoption, which he called prior conditions. These included (i) previous practice, (ii) felt needs/problems, (iii) innovativeness, and (iv) norms of social systems. Rogers (110) also produced a parsimonious set of characteristics, linking characteristics of both the innovation and of the adopter (see Table 8). These characteristics help explain  83  some of the variability why some innovations spread rapidly and others more slowly. . (118)  p 6 9  Table 8. Factors Affecting Adoption of Innovation (117) •  Relative Advantage: The degree to which an innovation is perceived as better than the idea it supersedes.  •  Compatibility: The degree to which an innovation is perceived as being consistent with existing values, past experiences and needs.  •  Complexity: The degree to which an innovation is perceived to be difficult to understand and use.  •  Trialability: The degree to which an innovation can be experimented with on a limited basis.  •  Observability: The degree to which results of an innovation are visible to themselves and others.  Rogers (110) conceptualized the diffusion of innovation as a communication and decision-making process that takes place through channels occurring over time and targets specific members of the target community. (118)  p 6 4  E h t p a c  s  e  m  m  e  model  contributes to the decision of an individual or organization whether to adopt a given innovation and whether such adoption will be maintained. The first step in the diffusion process is awareness of the innovation and its compatibility and relevance to the individual. The complexity of the innovation, the clarity of information informing the desired change, and perceived trialability may influence knowledge uptake. The second step in the process is persuasion. This can involve informal and formal clinical evidence, public and professional media, examples of successful implementation, adoption by opinion leaders in the local community. Some theorists such as Weiss (119) and Lomas (120), have paid particular attention to identifying or theorizing factors influencing the knowledge and persuasion phase of Rogers' diffusion model. Weiss, investigating the effectiveness of research utilization, looked at three different presentations of research on decision makers, research presented as data, research presented as ideas, and research  84  presented as argument. She found research presented as argument as the most compelling way knowledge can be presented to policy makers, although she warns that stripping away 'details' may omit vital elements and therefore distort research findings. Lomas and Lave building on Weiss's work, posit, that in order to advocate for change, research findings need to be condensed and translated into key messages and that these key messages or thematic messages need to be presented persuasively comparing current practice to the desired change(s). Kramer (121) " p  1 4  quoting Lave (1999) work states,  "Key messages should draw on all available research (and only profile and place in context the results of a specific research project when relevant), be compelling, and relate to a decision or set of decisions." For knowledge transfer purposes the unit of transfer are ideas rather than data. The idea of using key messages as a means to enhance dissemination has now been incorporated in the British Medical Journal and other journals including the Journal of Continuing Education in the Health Professions. (122) The third stage of Rogers' diffusion model is the decision to attempt to adopt the innovation. Trialability is influenced by a number of factors including whether the individual willing to attempt the change is able to make the change within the context of their practice or organizational structure and has the financial and human resources needed to make such a change. Huberman, cited by Kramer (121), building on Rogers' work and his own work investigating the causes and determinants by which innovations are generated, adopted, implemented, and finally institutionalized, created a Social Interaction Model which posited a number of factors that influence successful dissemination programs (see Table 9).  Table 9. Factors Influencing Successful Dissemination •  accessibility, availability and adaptability of the program  •  relevance and compatibility of the program  •  quality of the presentation  •  number of vehicles and methods used to convey the innovation  •  linkage and engagement of users  •  sustained interactivity between knowledge brokers (researchers, innovators, program leaders) and users  Davis et al. (6) provides a summary of factors believed to contribute to the adoption of innovation in the medical literature. These factors meld individual, social factors and principles that influence the uptake of medical innovation (see Table 10 below).  Table 10. Summary of Factors Impacting Behaviour Change in the Medical Community (adapted from Davis et al.) (6) •  multiple pieces of information from a variety of credible sources are required before physicians will make a change to their clinical practice  •  simpler changes require fewer sources of information and usually result in faster adoption  •  major changes requiring more complex procedures require greater number of sources and longer period needed for adoption  •  strength of the evidence supporting the innovation.  •  dependent upon collegial communication, particularly among peer-respected specialists.  •  physicians who are in association with a group practice, serve on major committees, published in medical literature, board certified, or hold an academic appointment tend to be early adopters of innovation  •  physicians must make a personal commitment to change  •  a conceptual base is necessary, as well as time to reflect whether the innovation is valid and relevant to their practice and attainable within their community, organizational context needs to support individual commitment to change  •  evidence of peers making changes  •  success of initial experience with attempting the innovation  Importance of Informal Communication with Peer Respected Colleagues Recent medical research has found that, while clinicians utilize a wide range of resources which contribute to their knowledge base and contribute to changes in clinical behavior, informal communication is consistently rated as a major source of information-  86  seeking behavior by physicians. (123) Gruppen et al. (124) found unique patterns of information-seeking behavior of family physicians with their first choices of information resources being informal consultations with colleagues (33%), consultations with community specialists (32%) and textbooks (27%). Use of journals (4%) and consultation with outside specialists (2%) were minimally important. Kaufman et al. (125) confirmed these findings in their recent survey of Nova Scotian Family Physicians who reported that formal and informal communication with colleagues and consultants were the most often used methods when seeking medical information and advice. In the medical literature different labels have been attributed to opinion leaders including gatekeepers, informal leaders, informal educators, credible messengers and educational influentials. (123; 126) The use of educational influentials (EIs)/opinion leaders in disseminating information has been shown to be an effective method of changing clinical practice. For instance, Stross et al. (127), in a randomized controlled trial, showed a significant difference in the inpatient management of chronic obstructive pulmonary disease in intervention hospitals. The Els who participated in the comprehensive educational program were involved in only 5% of cases but logged formal and informal consultations affecting another 25% of cases. Lomas et al. (128) found that clinical opinion leaders increased the proportion of vaginal births after previous cesarean sections. In Davis et al.'s (18) extensive review entitled, Evidence for the Effectiveness of C M E it was recommended that promising interventions such as the use of educational leaders deserved further testing and use. Although the credibility of the messenger may be an important factor in uptake, there are other factors that can influence judgments regarding the credibility of the content. These factors include the need for transparency, endorsement for the desired change from  credible organizations, referencing authoritative and unbiased sources of information, and adequately addressing controversial issues. (129) ' p  549  There are also a number of structural, organizational, demographic and motivational factors that may hinder or facilitate information-seeking behaviour of physicians. (130) These factors include their age and point in their career, the type of medical school they attended, their type of practice, the distance between learning centres and their practice, their access, comfort and training in new technology, whether they have a university connection or research interest, whether they are in a solo or group practice, whether they are remunerated for education, and whether they are attending C M E for accreditation purposes only. Typically these contextual factors are not considered within typical C M E programming.  Summary The first part of this chapter presented an alternate representation of internal and external forces shaping C M E practices to the model proposed by Gammon, providing the reader with an historical lens to better understand organizational, political, socioeconomic, administrative and legal factors contributing to the development of our current C M E practices. The second part of the chapter focused on the evolution of pedagogical beliefs, theories and research underpinning the continuum of medical education, outlining two attempts at medical curriculum innovation and the incongruence between the espoused theory of proponents of adult education and problem-based learning, and current research in the cognitive sciences, as well as research in medical teaching and learning. The last part of the chapter introduced research in the area of knowledge transfer, and medical diffusion. The lack of cross-discipline communication and collaboration among researchers involved in the cognitive sciences, knowledge transfer,  and diffusion has resulted in a disjointed artificial separation among different conceptions regarding what knowledge is, how it is transferred, what factors influence its utilization, and effective application resulting in improvements in this case, physician competence, and improved patient outcomes. When looking at the range of principles and factors discussed that may influence knowledge uptake, its utilization and effective application, there is a bewildering level of chaos mixing different types of knowledges with an assortment of principles and factors informing different aspects and dimensions of learning from cognitive processing, instructional design, socio-demographic and other contextual factors. The challenge for the planners of the case study was how to create greater order and coherence from this muddled array of information. The next chapter looks at a prototypical model of C M E planning recommended by the A C C M E , current accreditation policies and principles, and some planning and conceptual models arising from adult education and research into medical teaching and learning. The chapter provides a counterpoint to the planning methods used in the case study and provides an opportunity to reflect and discuss problematic issues with current planning in light of more recent ideas regarding learning theory and cognitive science research.  Chapter Three  Current CME Planning Practices in North America In Chapter Two, I argued that the continuum of medical education, historically and currently, is based on a faulty conception regarding how people learn, and has failed to create a curriculum development process that is linked to physician competence and population health needs. This chapter describes a prototypical planning process that is promoted to C M E planners interested in fulfilling accreditation policies with the intention of enhancing physician learning opportunities.  Protypical CME Planning Practices I could find very little qualitative or descriptive research investigating how C M E planners and faculty committees actually plan C M E programs. Cervero and Wilson (131) present a short case study looking at planning practices of an annual seminar for practicing pharmacists, and more recently, Golden, Parochka et al. (132) and Katz, Goldfinger et al. (133) explore the relationship of academia and industry on the planning process. However, there are a number of protypical planning frameworks for C M E and adult education planners. (134-5) For the purpose of this chapter I am using Rosof and Felch (134), second edition of Continuing Medical Education: A Primer, to describe the C M E planning process, as this text is used as a training guide and recommended reference book by A C C M E in their training programs for C M E providers and is linked to a set of principles and requirements of C M E accrediting bodies. Adelson et al.'s (135) text, Continuing Education for Health Profesionals: Educational and Administrative Methods, is similar in content to Rosof and Felch's text using the Tyler curriculum model as the C M E field's planning template.  Most planning texts in C M E and other education programs have adopted the four basic principles informed by Tyler's 1949 curriculum model. The planning principles (see Table 11) proposed in Rosof and Felch's text are more descriptive than prescriptive, however certain elements must be addressed by the C M E provider who is seeking program accreditation. Table 11. Four Principles of Good C M E and Adult Education (136)  p  4  •  Conducting a needs assessment; having some kind of mechanism for determining just what it is that physicians need to (or want to) learn.  •  Stating of educational objectives: establishing, in advance, how you plan to satisfy those needs, or put another way, what it is you expect participants to achieve by attending the CME activity.  •  Designing educational activities: deciding when and where and how you will arrange to put on the CME activity.  •  Evaluation: having a more or less formal way after the event of assessing the C M E activity and how well it achieved the objectives previously set.  I will briefly summarize each of the recommended steps discussed in Rosof and Felch's text discuss challenges to current practices.  Phase One Planning CME: The Needs Assessment In the Manual of Procedures (137) produced by the Committee on Accreditation for Continuing Medical Education for the Canadian Association of Continuing Medical Education there are several principles related to the needs assessment requirement (see Table 12).  91  Table 12. Accreditation Principles: Needs Assessment Process •  Principle 1. The provider should have a written statement of its aims, goals, major functions, and target population(s) within C M E approved by the Faculty of Medicine.  •  Principle 5. The provider should have established procedures for identifying and analyzing C M E needs, documented or perceived, of individuals or groups of intended participants.  •  Principle 6. The procedures for needs identification and allocating resources should include a coordinating mechanism for establishing priorities among needs identified. The mechanism for priority setting should reflect the provider's statement of aims, goals and major functions.  Moore and Cordes (138) p  4 2  describe the needs assessment process as follows:  (i) identifying a problem, (ii) deciding to respond to the problem, (iii) involving others, (iv) determining data collection strategy, (v) collecting the data, (vi) analyzing the data, and (vii) implementing the findings. The first step of the needs assessment process, "identifying a problem" is described as employing formal, informal and intuitive methods to identify developmental issues emerging when an individual or organization decides some changes are required from current practices to either an existing accepted standard of performance or a new standard of performance. Once an area of interest is identified, Moore and Cordes (138)  p  -  4 5  suggest, the second step is to create a planning committee, including members of the target audience, faculty, educational planners, and administrators. The role of the planning committee is to "collect, compile and analyze needs assessment data about current circumstances as well as standards." The third step is collecting data related to the identified problem. This data could include general socioeconomic, health systems, epidemiological, work setting, individual performance, and individual characteristics. The fourth step is for the planning committee to decide what sources of data are available and to seek input from the targeted audience. Moore and Cordes (138) summarize the types of data collection methods discussed in the literature (see Table 13). 92  Table 13. Methods of Data Collection Techniques for Needs Assessment (138) Unsystematic Techniques  More Formalized Expert-Centred  Hunches - professional intuition of C M E planners obtained from conversation, mass media and general observation  External consultants - process or content experts participating in a needs assessment process or one time consultation  Requests - usually from potential course directors often reflecting the perspective of on or more vocal individuals  Educational Influentials - key individuals in departments Document Analysis - studying of documents such as committee minutes, the medical literature, and other published literature.  More Formalized Learner-Centred Surveys and Interviews Questionnaires - a low cost means of reaching a large number of people, data can be easily summarized and analyzed though less informative if highly structured with little opportunity for comments. Interviews Formal and highly structured interviews with prepared questions Flexible and learner-directed interviews allowing for spontaneous topics and exploration of complex issues including uncovering feelings  Most Systematic Methods Tests and examinations - both formative and summative tests to identify learner deficiencies and progress. Observation - using actual or simulated patients. Self-Assessment - individual or in collaboration with peers various selfassessment strategies are employed to help physicians assess their needs. Group meetings - formally or informally using brainstorming or nominal group methods to help identify education needs. Patient Care Evaluations - from simple chart reviews to comprehensive research studies to identify specific deficiencies to address in the design of the CME.  C M E departments typically conduct a formal needs assessment by surveying the target audience regarding their perceived educational needs. Moore and Cordes (138)  p  report that the most common sources for data collection are course directors and potential learners and suggest other sources could include "experts in the field, potential faculty, representatives of professional groups, hospital administrators, detail persons, representatives of government agencies, researchers and patients." They also suggest a "largely untapped" variety of document sources could be used including "hospital patient 93  records, despite some limitations [...] minutes of regularly convened committees, incident reports and patient complaints, morning reports, and site visit reports from regulatory groups [...] professional literature [...] federal government [...] health statistics and technological developments." ' p  Once the data is collected, the fifth step  in this planning process is analyzing the data to ascertain (i) whether any needs exist, (ii) if they do exist to decide whether the needs are real "educational needs", that is those addressing deficits in knowledge, skills and attitudes or whether they require different action, such as "changes in staffing patterns, purchase of new equipment, and policy changes," ' p  4 8  and (iii) to prioritize the identified needs based on the "severity of the  problem, the number of patients affected, the number of staff involved, available resources, time investment required, capability and willingness of learners, and availability of previously developed programming protocols." ' p  4 9  The sixth and final part of the needs assessment process is implementing the findings. This phase involves documenting the purpose, people and methods used for conducting the needs assessment process, reporting on the findings, and translating needs into education objectives. It is suggested that a list of educational needs for the targeted community be prioritized through three different lenses, or perspectives - strategic, programmatic and individual activity. It is recommended that these needs be restated with the assistance of members of the planning committee and members of the targeted audience into behavioural descriptions of what the learner is expected to change as a result of participating in the educational activity.  Challenges to Current Planning Strategies: The Needs Assessment Process There are a number of barriers to conducting needs assessments. To conduct a needs assessment a number of human and information resources are required "including staff  time and salaries, office supplies, telephone, postage, travel, consultants, equipment, printing, and computer time." (138)  p  ' Divisions of C M E are generally poorly resourced 4  which hinders effective planning processes, survey/interview development and associated costs for conducting needs assessments. Having limited resources, comprehensive needs assessment processes are typically conducted once every three to five years depending on resources available as well as perceived need by the C M E leadership and their ability to raise sufficient funds. Unfortunately both comprehensive needs assessment processes and less formal needs assessment processes most commonly used primarily focus on physician identified educational needs rather than using data concerning physician incompetence and population health needs as primary indicators for selecting C M E topics. With limited funding and no central body systematically collecting and disseminating national, regional and local health statistics, C M E providers would have a difficult time bridging the gap between real world educational needs and what C M E programming currently offers. The WHO Report (9) ' p  19  entitled, Continuing Education of Health Personnel and its  Evaluation, recommends that a radical change is needed in how needs assessments are typically conducted. They suggest educational planning should emerge from a systematic study of existing health care patterns and the needs of health services and communities and suggest such information should be derived from national public health statistics or by regional, local or individual identification of health problems most frequently encountered. The authors of the WHO report state, "Unfortunately program objectives too often do not take into account the real needs of health services but rather are a  manifestation of the views heldby health workers, university staff or administrators concerning those needs."  Phase Two Planning CME: Stating Educational Objectives The second part of the curriculum model is formalizing learning objectives. Stating learning objectives is also a requirement for C A C M E , A A C M E and most other accreditation bodies (see Table 14). Table 14. Principles of C A C M E and A A C M E : Educational Objectives Principle 8 C A C M E •  The provider should have stated objectives for all learning activities. These objectives should be based on identified CME needs. (CACME, 2002)  Essential 3 A C C M E •  The Sponsor shall have explicit objectives for each C M E activity. The sponsor shall:  •  State the educational need(s) which the individual activity addresses.  •  Indicate the physicians for whom the activity is designed (target audience).  •  List any special background requirements of the prospective participants.  •  Highlight the instructional content and/or expected learning outcomes in terms of knowledge, skill, and/or attitudes.  •  Make these objectives known to prospective participants.  Mager's seminal work, Preparing Instructional Objectives, provides a conceptual rationale on why objectives should be identified. Rosof (139) ' p  5 4  , citing Mager states, "If  we do not know where we are going, it is difficult to select a suitable means of getting there, or, for that matter, even to know if or when we arrive." Explicating learning objectives helps the C M E planner, instructor and learner reflect and clarify what the intended outcome is for the learning activity. Learning objectives are usually constructed using action verbs to describe what the learner should know or be able to do at the end of a learning activity. Having clear objectives may also assist planners and faculty members in selecting appropriate teaching methods and materials to facilitate the learning as well as in planning evaluation methods. The rationale behind 96  constructing learning objectives is that C M E planners or faculty are able to effectively use these to design successful educational activities and evaluation instruments.  Challenges to Current Planning Strategies: Stating Educational Objectives The stating of educational objectives is more common in C M E programming due to mandatory accreditation requirements. The process of defining learning objectives is primarily left to the instructor, possibly in consultation with the C M E planner, rather than members of the target audience. There are a number of challenges to creating meaningful educational objectives. Without a strong conceptual framework linking the objectives to an evidence-informed educational design and a theory-driven evaluation, the process of defining learning objectives becomes a procedural step rather than a reflective process. Typically educational objectives are content-based rather than context-based, with many objectives either focusing on recall of information (i.e. identify three factors...) or have little relationship in applying this information to their practice environments. To change routine clinical behaviour requires a much deeper understanding regarding general and individual barriers to knowledge uptake and its utilization. Whereas some of these barriers might be identified via focus groups with the targeted audience, other barriers may be specific to an individual's work place. If one was to incorporate a more social cognitive conception of knowledge uptake, learning objectives could include having the participants identify institutional, environmental and regulatory barriers to knowledge utilization and the creation of action plans to address these barriers.  Phase Three: Designing Educational Activities The next phase of the curriculum model, designing the educational activities, is also referred to as the instructional design process. The C A C M E principle speaks to this issue (see Table 15). Table 15. Principles of C A C M E and A A C M E : Learning Objectives Principle 9 C A C M E •  The provider should develop and implement learning activities consistent in content and method with the objectives and with the intended participants.  Essential 4 A C C M E •  The provider must develop and implement learning activities consistent in content method with the objectives and with the intended participants.  This phase of the curriculum model "often gets less-than-adequate attention." Program planners tend to focus on the mechanics of the educational exercises — appropriate time scheduling, room arrangements, audiovisual resources, and so on—but usually, when questions about format comes up, they settle for the conventional assumption that continuing medical education (CME) means a visiting or local speaker delivering a lecture of common length and style. They are content if the lecture is well presented and the slides are relevant. (140) ' p  65 67  Fink Jr. and Osborn (140;141) ' pp  " present a broad range of possible educational  activities C M E planners could utilize including case review, skills sessions, simulations, self-assessment inventories, teleconferencing, computer simulations and interactive videodisc, mini-residencies, individualized learning plans, curriculum-based C M E . In addition, Fink, Jr. & Osborn (140) ' p  describe six learning principles they claim could  inform planning educational activities: (i) the learner's degree of motivation correlates directly with how that learner can incorporate new facts, concepts, skills, values, and so on, (ii) active participation on the part of the learner tends to result in higher-quality 98  learning than does passive, uninvolved experience, (iii) A problem-solving approach tends to foster both motivation and active involvement, and thereby facilitates learning, (iv) repetition and reinforcement helps the learner remember information, especially if the reinforcement comes as a result of using the information, (v) reward and positive feedback are related principles and help promote useful learning, especially if this the reward is internalized, (vi) multisensory signals (hearing and seeing) are helpful to most learners. Fink Jr. and Osborn (140) also mention three challenges facing C M E planners, especially in regard to creating innovative programming: (i) obtaining physician support/participation, (ii) obtaining good teachers, and, (iii) obtaining good facilities.  Challenges to Current Planning Strategies - Designing Educational Activities Part of the problem with Phase Three is that this is the least developed or understood part of Tyler's curriculum model and is rarely linked to current research in teaching and learning. There are major challenges to designing efficacious educational strategies without a clear understanding of what types and level of knowledge is needed, what contextual factors hinder or facilitate the process of knowledge acquisition, and lead to appropriate utilization. As this section begins to integrate some of the earlier discussion about theories of learning and problems of ineffective teaching, I will present a more comprehensive analysis about challenges to designing efficacious educational activities and identifying some bodies of research that may better inform planning practices. Under a more behavioural model of instructional design, learning technologists reduce phenomenon to the language of task analysis, breaking down human activities into inputs and outputs, entry behaviour, stimulus and reinforcement. Learners are seen as  99  human information processors. It assumes that learning occurs in predictable, systematic and controllable ways and that learners are not problematic nor particularly dynamic. Under a more biological-driven cognitive model as promoted by Gagne (142)  p  '  1 8 8  "  1 SQ  , the learning process is operationalized in a linear manner (see Table 16). Table 16. Gagne's Biological-Cognitive Model of Instructional Design •  Attention: determines the extent and nature of reception of incoming stimulation  •  Selective perception: Transforms this stimulation into the forms of object features, for storage in short-term memory.  •  Rehearsal: Maintains and renews the items stored in short-term memory.  •  Semantic Encoding: Prepares information for long-term storage.  •  Retrieval, including search: Returns stored information iri the working memory or response generator.  •  Response organization: Selects and organizes performance.  •  Feedback: Provides the learner with information about performances and sets in motion the process of reinforcement  •  Executive control processes: selects and activates cognitive strategies; these modify any or all of the previously listed internal processes.  Gagne's model of instructional design, he claims, is based on over 50 years of behavioural and educational research and is used and promoted by adult educators typically interested in instructional design.(143) However, most of Gagne's own research, and the research he cites, occurs in very specific learning environments such as schools or the military and focuses on relatively uncomplicated tasks involving immediate recall and task replication. The business of translating learning objectives into educational activities has been conceptualized in the educational literature as being the domain of teachers' knowledge. Research into teachers' knowledge has been prolific from 1983 through to 1992. Investigations since the early 1970s have moved from examining the relationship between teacher behaviour and student outcomes to considerations of teacher thinking,  100  especially in regard to the decision-making process leading to instructional actions (144) p  '  2 5 5  . Part of this change was due to the influential work of Philip Jackson, Life in the  Classroom (1968) that gave a richer description of the full complexity of the teacher task. (144) Shavelson and Stern (145) ' pp  456-7  suggest that this change in research focus was  based on two fundamental assumptions: (i) a teacher's behaviour is guided by his/her thoughts, judgments and decisions, and (ii) teachers are reasonable professionals and, similar to physicians, make judgments and carry out decisions in uncertain complex environments. This change in assumptions was based on changes in conceptions regarding research on teaching moving from a behavioural model which just looked at teacher behaviour, to one that examines the link between intentions and behaviour with the premise that this research may lead to a better understanding regarding the teaching process and inform teacher education. (145) ' pp  455-6  It was believed that to better understand the decision-  making process we needed a better conceptual framework to examine the knowledge and beliefs that inform decision-making. (146) ' p  2 0  (147) ' p  307  To meet this need, several educators/researchers have conceptualized a range of different types of knowledges which are interwoven in practice. (148-152) Each of the knowledge domains suggested by Shulman and others is not considered as being distinct and separate from each other. It is believed that in real life there are not clear boundaries between these knowledges and that there is a dynamic interaction amongst these knowledges that inform action. With this in mind, I will briefly summarize some of the distinctive characteristics among each of the knowledge domains described by a range of  educators and researchers to provide some greater context about the classification used to define the domains as well as to briefly explore some differences in typologies suggested. Shulman (149) conceptualizes a range of knowledges which underlie the teacher understanding needed to promote comprehension. These knowledges include content knowledge, general pedagogical knowledge, curriculum knowledge, pedagogical content knowledge, knowledge of learners, knowledge of educational contexts, and knowledge of educational ends. In Table 17 below you will note Shulman's conceptual framework on the left. Grossman's (146)  p  '  2 0  summary of domains of teacher knowledge merges content  knowledge with pedagogical content knowledge and curriculum knowledge with knowledge of others. Grossman also includes an additional knowledge domain, the knowledge of self. Irby (153)  p p  '  3 3 3 - 4  in the third column, exploring knowledge domains  related to clinical teaching in medicine, linked a constructivism framework with knowledge domains. Irby's qualitative study reported on knowledge domains described by six distinguished pedagogues using data from interviews, a structured task and observations.  102  Table 17 Comparison of Typologies of Teacher Knowledge Domains S h u l m a n ' s T y p o l o g y (149)  G r o s s m a n ' s T y p o l o g y (146)  Irby's T y p o l o g y (103)  Content  knowledge  - understanding of the facts, concepts and knowledge structure (organization - understanding of fundamental principles of a discipline) within their subject area  - combines subject matter knowledge with pedagogical content knowledge  - clinical knowledge is more closely related to experience with clinical cases - differences in knowledge representation between novices and experts  General  pedagogical  - knowledge of pedagogical principles and techniques - broad principles and strategies related to classroom management and organization -not bound by topic or subject matter  - knowledge of classroom organization and management and general methods of teaching  - general principles of teaching and learning conceptions - experienced teachers have large repertories of teaching strategies - knowledge of instructional resources  - understanding of the programs and materials designed for teaching particular topics and subjects  - includes knowledge of both the process curriculum development and of school curriculum across the grades  - general medical knowledge incorporating both basic sciences and clinical experience - case-based teaching scripts  Knowledge of Other  - understanding of knowledge of other subject areas that can assist in developing links with other subject matter and increase relevancy.  - combines curriculum knowledge with knowledge of other  - the need for assisting learners in building links across subject matter  Pedagogical content  - knowledge of what it means to teach a particular subject including the principles and techniques of teaching and learning.  - combined with content knowledge  - develops though the repetitive experience of teaching resulting in teaching scripts - scripts include general goals of instruction - specific representations of content (explanations, analogies, examples learning tasks)  Knowledge of learners  - including knowledge of student characteristics  - knowledge of learners and learning theory, physical, social, psychological, cognitive development of students, motivational theories and practice, ethnic, socioeconomic and diversity among students  - understanding learners prior knowledge - conceptions and misconceptions of subject matter - learner needs, motivations and abilities  Knowledge of  - ranging from the workings of the group, classroom - the governance of the school, character of the learners community and cultures  - teacher's knowledge of students, families, local community, - historical, philosophical and cultural foundations of education within a country  - includes patient population served by hospital, - historical context of therapeutic management - teaching context patient interaction or case presentation  - understanding of educational aims, goals and purposes informing pedagogical action.  - combined with knowledge of self  knowledge  Curriculum knowledge  knowledge  educational contexts  Knowledge of educational ends Knowledge of Self  - includes personal values, dispositions, strengths and weaknesses, educational philosophy, goals for students and purposes for teaching  Carter (151) in her review on research into teachers' knowledge, investigating primarily qualitative studies, reports on other emerging conceptions on teachers' knowledge and presents an alternate conceptual framework. Aside from formal subject  103  matter knowledge, teachers' institutional and occupational perspectives, and professional knowledge base (which were mentioned though not defined nor covered in her review), she proposes that research into teachers' knowledge in the classroom could be divided into two broad categories: practical knowledge and pedagogical content knowledge. Practical knowledge, she states, is "referring to the knowledge teachers have of classroom situations and the practical dilemmas they face in carrying out purposeful action in the setting." Carter describes research into pedagogical content knowledge as "representing an attempt to determine what teachers know about their subject matter and how they translate that knowledge into classroom curricular events." ' p  2 9 9  The relevance of conceptualizing the specialized knowledges of teachers is the need to rethink the development of effective educational strategies that take advantage of pedagogical practices as the need to provide sufficient training to doctors and researchers interested in medical teaching. There are also other bodies of literature that could systematically inform improvement in teaching, especially a growing body of research investigating instructional design practices and the field of cognitive ergonomics. The literature concerned with human factors and cognitive ergonomics is better known is England and Europe. Wright (154) states, "whereas conventional ergonomics is concerned with how the physical work environment matches human physical/ physiological characteristics, 'cognitive ergonomics' is concerned with how the presentation of information influences intellectual performance (understanding, memory, reasoning, etc)." The most significant work in the area is related to a range of print media, the use of different types of prose written instructions, instructional texts, as well as the use of tables and charts, with more limited research in videotape and computer assisted learning.  104  James Hartely (155) in his seminal text, Designing Instructional Text, presents some guidelines on a range of variables involved with the presentation of written information. For instance, Hartley (155) notes that the principle weakness of much instructional material is the lack of consistency in the positioning and level of relevance of its components such as listed information, numbered items, headings and subheadings, diagrammatic presentations, tables, explanatory notes, and pictorial features. The lack of consistency leaves the reader to continually ask themselves, "where am I to go from here?" ' p  1 7  For this reason he suggests the use of a master reference grid and the  development of style guides (see Table 18): Table 18. Designing Instructional Text: Evidence-Informed Style Guides (155) •  printed pages need to provide a reliable frame of reference within which the reader can move about, leave and return without confusion  •  the user must make sense of why different typefaces or conventions are used, style sheets and specification charts need to be made in advance not on a page by page basis  •  words printed in capital letters contain less distinctive information per unit of space, so this makes groups of them more difficult to read ' p 29  •  when listing information - if there is an actual sequence order use Arabic numbers or alpha order, and if sequence is not important then use bullets ' p  47  •  Misanchuk (1992) suggests that devices such as capital letters, underlining, italic and bold need to be used sparingly as they can lose their significance  •  Ellington and Race (1993) provide examples of effects of typefaces when used in slides or projection particularly the reduction of legibility  There has also been considerable research looking at prose and graphic features. This includes the use of pretests, titles, pre and post summaries, section headings, questions in text, sentence length, typographical cues, use of positive terms (more than, heavier than, thicker than) rather than negative terms (less than, lighter than, thinner than). As well there has been research into alternatives to prose — flow charts, tables, data presentation, line graphs, bar graphs, pictorial charts, use of illustrations or pictorial instructions. Wright (156), for instance, exploring the impact of presentation of information on  105  retention and understanding, showed differences among the use of a decision tree (algorithm), prose, table and short sentences with structural spacing on different outcome measures. Her results indicated that well designed tables were effective for content retention for simple problems although short sentences and algorithms were comparable when used for problem-solving. However, when working from memory, performance improved with the use of prose and lists of short sentences. Simple changes in presentation such as placing figures in columns rather than rows can in some contexts greatly influence comprehension of data information. Hartley (157) provides educators and researchers an excellent evidence-informed review of instructional text research suggesting eighty ways of improving instructional text. Wright has explored a wide range of communication/medium variables i.e. tables, algorithms, reading comprehension, print versus hypertext, print versus videotape, within a range of subject domains and experimental contexts. Wright's research attends to sociocontextual factors, including user's relevance, user's preference, and access issues. For instance, Wright (158) reporting on research investigating "reading to do" versus "reading to learn" scenarios, suggests that when interpreting experimental studies on reading it is important to specify the reading of what, by whom and for what purpose. p  '  5 1 8  Reading goals are usually up to the experimenter. Sometimes subjects will know  what and how they will be tested so they will read in such a way to maximise their score. Research cited by Wright (158), (Olshavsky 1976-77, Sticht 1978 and Samuels and Dahl, 1975) suggests that readers change reading strategies based on intrinsic or extrinsic goals e.g. reading for recall of general information, or specific details, reading for verbatim retention or a paraphrase quiz. Olshavsky (1976-77) reported 10 different reading strategies used by 24 people who were reading short stories that varied in their degree of  106  abstractness. It therefore becomes important to clearly inform the reader what you want them to be able to demonstrate and to ensure that there is congruency between your explicit outcome measure and the measure you have chosen to demonstrate this type of learning. Without paying closer attention to instructional design elements and the contextual factors (stated purpose, users relevance, awareness of test measures) among comparable treatments, research efforts are likely to be powerfully confounded by these uncontrolled variables. By looking at the many difference ways of framing teachers' knowledge as represented by Shulman, Grossman, Irby and Carter it is apparent that there is not a clear taxonomy for conceptualizing teachers' knowledge domains. Part of this problem is that there are different purposes for the creation of constructs of teachers' knowledge. Different purposes might require different conceptions of teachers' knowledge. For instance, research investigating the praxis of teaching as it is practiced is very different from research exploring teacher effectiveness or differences between novices and experts. There are many research interests involved in exploring teachers' knowledge. Some research agendas include: (i) identifying factors which influence how and why a teacher teaches in particular ways, (ii) gaining a deeper understanding about each of the factors identified, (iii) gaining insights into differences between novices and experienced teachers, (iv) understanding what qualities differentiate noted excellent pedagogues from 'less effective' pedagogues, (v) exploring teachers' perspectives on their reported perceptions, beliefs and knowledge about teaching and learning. The various conceptions of knowledge domains serve different descriptive purposes however, I suggest, that the lack of explicitness of purpose for a given conceptualization (or categorization) decreases  its utility. Research in teachers' knowledge dispels the myth that being a physician, scientist or researcher with 'content expertise' somehow qualifies them to be appropriate teachers. The old axiom "Learn One, Do One and Teach One" just does not cut the mustard even if the Latin translation of 'doctor' is 'teacher'.  Attempts to Incorporate Adult Learning Principles in CME to Inform Educational Design Since the mid 1980s there has been a growing movement in C M E to adopt adult learning theory as the philosophical underpinning for the field to inform educational planning. (159-163) For instance, in 1995, the College of Family Physicians of Canada established new C M E guidelines that explicitly state that all programs must be based on adult learning theory. The hope is that application of generic adult learning principles will enhance C M E programming and lead to better outcomes. Unfortunately adult learning theory as espoused in the literature is ill-defined and not linked to an efficacious cognitive model, or at least one that has a developing body of evidence linking theory to practice. Slotnick (160) ' p  describes adult learning characteristics as embodying the following  principles for physicians: (i) they are practical learners (ie they seek to solve problems), (ii) they wish to participate actively in their own learning; and (iii) they have multiple demands on their life and wish to meet their psychological needs of security, affiliation and self-esteem. Bennett (161) lists ten characteristics of adult learners: (i) adults of all ages have the ability to learn, (ii) adults are self-directed in their learning, (iii) experience is a resource for learning, (iv) participants look for practical learning, (v) adults learn by choice: learning is voluntary, (vi) learning is more effective when adults are actively involved, (vii) feedback is a critical part of learning, (viii) uses for learning change with different stages in a career, (ix) people learn differently — differently from one another 108  and on different occasions, (x) learners are more apt to make changes as a result of learning if they are have a clear image of what will be achieved. Adelson (162) reporting on characteristics derived from the work of Knowles describes similar characteristics associated with adult learning including adults being self-directed, being more problemcentred than subject centred, and more concerned with the immediate application and professional relevance of information. The problem with the blanket adoption of adult learning theory in C M E is that there is not a strong conceptual framework underpinning adult learning theory. The adult learning principles discussed in the literature, for instance, do not adequately assist C M E planners in making informed linkages between the desired learning objective and the types of educational activities and learning materials that could best be employed to facilitate learning. Adult learning theory does not differentiate between novices and experienced practitioners, nor is it informed by current research in the field of cognitive ergonomics, other research in the cognitive sciences, or specific research in medical teaching and learning. Despite the interest of sociocultural or socio-cognitive factors in adult education there is little research or and scholarly work directly linking sociocultural theory to adult learning. (164) Without a clear conceptual framework and taxonomy linking theory to practice, C M E planners have limited information to enhance knowledge transfer. This failure is not limited to the field of C M E , Pamela Wiggins, Vice-President, Knowledge Products and Mobilisation for the Social Sciences and Humanities Research Council of Canada (SSHRC) remarked at a knowledge transfer workshop how public 31  education systems "deliver" education to students, with the change process often  Knowledge transfer workshop hosted by the Canadian Institutes of Health Research on June 19, 2002.  appearing to be ideologically driven rather than on the basis of systematic research evidence. Without a clear conceptual framework for selecting educational activities informed by research in teaching and learning linked to meaningful educational objectives with adequate instruments for evaluation, we are left with a haphazard approach to program planning. However even if we have a more integrated model for planning C M E , action would need to be taken to address the other environmental factors that are barriers to develop and maintain innovative programming, especially if certain types of learning require more small group interactive programming rather than traditional conference programming, which will result in lower financial returns.  Phase Four: Evaluation The final phase of the curriculum model is evaluation. Principles of C A C M E and A A C M E : Evaluation Principle 9 C A C M E •  The provider should evaluate the effectiveness of the overall CME program and each learning activity.  Essential 5 A C C M E •  Periodically review the extent to which the sponsor's mission is being achieved by its educational activities.  Show that these evaluations assess: •  the extent to which educational objectives are being met  •  the quality of the instructional process  •  participants' perception of enhanced professional effectiveness  •  use evaluation methods that are appropriate and consistent in scope with the educational activity.  •  demonstrate that evaluation data are used in planning future C M E activities  From a C M E planners' perspective evaluation encompasses a broad range of activities including the effectiveness of planning procedures, budget versus actual expenditures, future programming, adequacy of facilities and effectiveness of faculty and 110  impact of C M E on the target community. Green (165) ' p  7 3  does suggest alternate types of  data collection techniques planners could use including interviews, other written questionnaires, written tests, performance tests, observations, and existing data sources. However these suggestions are more the exception than the rule.  Challenges to Current Planning Strategies - Evaluation There are many challenges to current evaluation strategies. From a planners' perspective one of the traditional reasons for conducting an evaluation of an educational program is to determine information about the perceived quality of the program and gain direction for improving future programming. (166) Most evaluation methods and instruments used in C M E have tended to focus on satisfaction measures rather than assessing the impact of C M E on physician learning and behaviour and its impact on patient health outcomes (see Appendix IV: Sample Prototypical Program Evaluation). More recent attempts by accreditation bodies to improve evaluation have led to planners restating educational objectives and having participants score how well the session has met those objectives without demonstrating whether this form of evaluation is a useful exercise for the participant or necessarily informs future planning. As mentioned in the introduction, C M E evaluations have historically relied on 'black box', 'input-output', or outcome-focused evaluation. Without the development of a theory-driven conceptual framework identifying what factors contribute to programs' successes and failures in planning, implementation, evaluation and patient outcomes, we will continue to stranded in the sea of variables unable to effectively determine what factors have contributed to program successes and failures. There have been some concerted efforts to improve the quality of evaluations  111  across educational program domains. One major attempt has been the development of standards for the profession.  Standards in Program Evaluation In 1981, the Joint Committee for Standards in Educational Evaluation (JCSEE) published standards for program evaluation and established a comprehensive set of guidelines to reflect the best practice of educational program evaluation at the time the standards were written. (166) In 1994, the second edition was published. The Standards, as they referred to in the text, are organized around four primary attributes of an evaluation: utility, feasibility, propriety and accuracy. All four primary attributes are pragmatically-based. The Standards speak to the need to create evaluations that address the need for the measurement of intended outcomes but also meet the timely information needs of various stakeholders. One of the failures of past evaluations is that for the most part, evaluations are rarely used by stakeholders to make informed decisions about program continuance, program changes, or impact policy or regulatory activities. (119;167)(168) Utility Standards: Although all attributes are important, the first attribute discussed in The Standards is "utility" which addresses the questions, why is the evaluation needed, what is the purpose of the evaluation, what are the information needs of the clients and stakeholders and what is the timeline for specific information needs. The JCSEE committee's purpose of highlighting "utility" as the first attribute to be addressed is in recognition that program evaluations must be conducted in such a manner that they address the information needs required by all stakeholders and decision-makers in order to enhance the potential utilization of the program evaluation. Far too often, professional evaluators were disappointed that their evaluations were being shelved or underutilized. The standards grouped under this category are: (i) stakeholder identification, (ii)  evaluators' credibility, (iii) information scope and selection, (iv) values identification, (v) report clarity, (vi) report timeliness and dissemination, and (vi) evaluation impact. It is believed that clearly addressing each of these components will increase the likelihood that the evaluation will be utilized and have some practical value and application. Feasibility Standards: All program evaluations require important decisions around choices made concerning research design(s) used, instruments employed, data collected procedures and analysis used. Evaluation considerations must take into account that the evaluations are typically conducted in the field, and that evaluations must address resources available, materials needed, personnel and time allocated. The best research design constructed for a given research question may produce greater evidence to make stronger claims concerning the program's merit or worth, however if the design does not take into consideration resources available, the stakeholders information needs and timelines, the evaluation will lose utility and its value may be much more limited. For instance, the project funders or stakeholders might have specified windows of opportunities where an evaluation can assist in determining program continuance, program improvement or have the greatest impact among stakeholders and decisionmakers. Propriety Standards: As the programs being evaluated can affect many people in a variety of ways, these standards "facilitate protection of the rights of people affected by an evaluation." These standards require evaluators to learn about the laws concerning privacy, freedom of information, and the protection of human subjects. The standards are: service orientation, formal agreements, rights of human subjects, human interactions, complete and fair assessment, disclosure of findings, conflict of interest, and fiscal responsibility.  113  Accuracy Standards: In order for program evaluations to be informative, there must be sufficient evidence that the evaluation is producing sound information and is comprehensive enough "to address as many of the program's identifiable features as practical and should have gathered data on those particular features judged important for assessing the program's worth or merit. The standards are: program documentation, context analysis, described purposes and procedures, defensible information sources, valid information, reliable information, systematic information, analysis of quantitative information, analysis of qualitative information, justified conclusions, impartial reporting and meta-evaluation. The Standards attempt to establish 'best practices' concerning ethical, legal and types of information and practices deemed important to conduct reasonable program evaluations, as well as to establish professional standards and codes of behaviour. Although these standards were established in 1981 and recommended in 1983 for adoption in continuing education in the health profession (17), in reviewing all research articles published in the Journal of Continuing Education in the Health Professions since 1984 I was unable to find one research article that reported they had systematically addressed each standard. To facilitate its use the JCSEE published a checklist to help planners assess and self-report on the level of compliance (see Appendix III: Program Evaluation Standards Checklist). The need to improve the quality evaluation papers on educational interventions is also evidenced by the recent establishment of the British Medical Journal's committee, Education Group for Guidelines on Evaluation and the publication of new guidelines for authors, editors, reviewers, and readers on educational interventions. (169) The B M J created this committee recognizing the rising importance of education intervention  114  research as well as the problem with current research in the field. They state, "Unfortunately many of the accounts we receive of educational interventions comprise a thin description of the innovation and an evaluation that says little more than the students liked the intervention." Although the utilization of The Standards will address BMJ's call for more in-depth description of interventions and evaluation methods, The Standards fail to provide insight on how to improve evaluation designs to specifically enhance the efficacy of educational interventions and improve the conceptual understanding of what variables should be attended to in the planning, implementation and evaluation process to link theory to practice.  General System Failure in the Planning Literature: Linking Theory to Practice The gap between theory and practice in educational planning is not solely related to C M E practice. Sork and Buskey (170), analyzing adult program planning literature between 1950 - 1983, described a series of steps common to many planning models. These steps include: (i) analysis of the planning context and client system(s) served, (ii) assessment of the client system needs (iii) development of objectives, (iv) selection and ordering of content, (v) selection, design and ordering of instructional processes, (vi) selection of instructional resources, (vii) formulation of budget and administrative plan, (viii) design of a plan for assuring participation, (ix) design of a plan for evaluating the program. Cervero and Wilson (131;171), however, claim program planning in practice is a value-laden activity, where the central activity planners engage in is negotiation. Specifically, they assert the practice of program planning in its social context is  inextricably linked to the complex world of personal and organizational power relationships and interests, where planners bring their own interests to the planning process as well as construct the program with others. Cervero and Wilson's conception of program planning is very different from the typical model described in the linear step-bystep model described above. From a C M E perspective the importance of having explicit standards and a clear universal understanding about the purpose and role of C M E is critical in today's world of competing demands and values (171), especially with the lack of financial resources for effective C M E planning and the increasing reliance of industry support. (172) Sork and Buskey (170), in their evaluation of different planning models looked at two factors, the sophistication necessary to benefit or use the planning model effectively and, the degree to which the model has an explicitly stated theoretical framework. Specifically they looked at the level of comprehensiveness of each step (steps i - ix above) described in each of the planning models reviewed to determine whether a planner could implement every step of the model effectively from the material presented by the author. In their review they outlined a number of problems with the literature they reviewed. These problems included: lack of cross referencing and absence of cumulative development in the literature, literature has a low degree of theoretical explanation, few books treat planning as a comprehensive process, lack of integration/recognition that some planning models developed for a specific context could have applicability for other contexts, heavy bias for group instruction, less literature for individual instruction, literature does not clearly look at the variety of roles played and what levels of proficiencies (skills) are necessary for planners and do not discuss the degree of group interaction in the planning process.  Planning as a Guiding Tool or a Point of Reflection Extending Cervero and Wilson's conception of planning as a process of negotiation, some social theorists suggest that, typically, people do not use a logical-deductive approach to planning. Research in the late 1980s suggests that for the most part human beings do not generally follow external plans or directions in everyday decision-making. Suchman (173), a researcher at Xerox PARC, was studying how ordinary people use Xerox machines' built-in help and diagnosis programs. In looking at how ordinary people responded to Xerox machine problems she proposed two theoretical frameworks. The first being a cognitive science-based instructional framework constructed using a hierarchical system of subprocedures for how Xerox machines 'should' be used or diagnosed. The alternative process she labeled 'situated actions', that is the 'lived' experience of Xerox users. Suchman defines situated actions, as "simply actions taken in the context of particular, concrete circumstances." Suchman (173) introduces John Seely Brown's ideas about everyday cognition (see Table 19). Table 19. Aspects of Everyday Cognition •  act on situations  •  make sense out of concrete situations  •  resolve emergent dilemmas  •  negotiate the meaning of terms  •  use plans as resources  •  socially-construct physical and social reality  Streibel (1991, ' p  117  ) in reviewing Suchman's work on the notion of situated cognition  and poses a fundamental question: "Do human beings, such as teachers or learners, follow plans no matter how tentative or incomplete those plans might be, when they solve real-world problems or do human beings develop embodied skills that are only 117  prospectively or retrospectively represented by plans?" Suchman (173) ' p  50  argues that  plans are representations of situated actions and posits that "these representations always come before or after the fact, in the form of imagined projections or recollected reconstructions." This is quite different than viewing plans as controlling procedures. Suchman (173) recommends that in the case of human beings, "plans should not be treated as 'psychological mechanisms' that control and give meaning to subsequent behaviour. Rather, plans should be treated as artifact[s] of our reasoning about action." She continues, instructional plans should not be used to control instructional interactions, rather, plans should be used for communicating about situated actions with other humans beings and reflecting on and reconceptualizing situational actions." This notion about situated learning may have an important contribution to make in how we conceptualize decision-making behaviour, however it too suffers from a lack of conceptual or research base and fails to address differences apparent in novice or expert decision making behaviour. Qualitative and quantitative research is needed to better understand how planners actually plan and what factors they perceive are influencing their decision-making and based on current conceptions about learning researchers should clearly differentiate among novices and more experienced planners.  Closed Planning Models versus Open System Models Tyler's original pedagogical teaching model was primarily addressing teaching children in the school system and arose from learning theories based on behavioural and transactional beliefs about learning. Learning was perceived to be an unproblematic learning process that involved a more linear and technical process, rather than a more complex process intrinsically linked to a process of enculturation. The Tyler model was flexible enough to accommodate learning theories arising from biological, behavioural  and behavioural-cognitive notions about learning, however I suggest the model is too limited to adequately address sociocognitive ideas. Part of the problem with the Tyler model is that it is constructed and utilized as a closed system model. It is a model where inputs and outputs are constructed in a mostly linear fashion without sufficient conceptual linkages to better identify other factors contributing to our understanding about how learners learn, informing how teachers should teach, and how planners should and actually do plan. Program planners working in the field of continuing education, as against funded public schools must also address other challenges, programs must attract participants, maintain their interest, and provide an enjoyable educational experience. The Tyler model tends to focus attention on only looking at what the program planners intended learning outcomes were, rather than identifying what learning occurred or reflecting on how it occurred without consideration as to what factors should inform the planning process. If the Tyler model does not really represent what planners actually do, as Cervero and Wilson believe, and has failed to be a fruitful model for systematically building our knowledge base on how to improve our practices, I believe it is time to look at other planning models.  CME Accreditation Bodies There are approximately 2600 accredited C M E providers in the US and Canada. The Committee on Accreditation of Canadian Medical Schools accredits undergraduate and C M E programs. Specialty residency and specialist C M E programs are accredited by the Royal College of Physicians and Surgeons of Canada, and family residency and family physician C M E programs are accredited by the College of Family Physicians of Canada. In the US, C M E accreditation is managed by the American College of Continuing Medical Education (ACCME). The Committee for Review of Recognition of the  119  A C C M E provide accrediting authority for state medical societies, which in turn accredit 1912 local C M E providers. Other accrediting bodies in the U.S. include American Osteopathic Physicians and the American Academy of Family Physicians. In Canada, medical schools and medical related societies are the only accredited bodies to provide CME.  Calls for a Change to Accreditation Practices In North America the accreditation bodies are rooted in Tyler's curriculum development model, with some minor additions, such as the requirement to maintain an organizational mission that includes C M E , and assuring there is adequate management and other resources to effectively fulfill the C M E mission. (174) Since 1985 leaders in C M E and adult educators have been questioning the current accreditation system and the adequacy of Tyler's model to inform the accreditation process. However there are many barriers to changing the current system. Mazmanian and Duff describe these barriers as follows: Many ACCME-accredited sponsors seem unprepared for major changes. They are not fully subsidized, and to offset potential fiscal deficits, they become deeply involved in the provision of conferences, with the promise of spending little money to attract a lot more. Trusting the logic of the Essentials for accountability and accreditation review, C M E planners carefully document educational needs assessment, objective setting, educational design, and budgetary decisions, and demonstrate the use of data from previously administered survey instruments and financial summaries... Some of the heavy dependence on the Essentials may be due, in part, to the limited training of those responsible for providing leadership in C M E programs. It has been estimated that over 95 percent of those responsible for carrying out continuing medical education in the United States have been trained as members of the professional groups with which they work, rather than in the field of adult education. (174) ' p  292  Mazmanian and Duff (174) suggest an alternative vision of C M E , one which is less preoccupied with the codification of rote-like accreditation, with a 'gentler' and broader  120  definition of C M E borrowing one from adult education recognizing the need to nurture the human potential of physicians. ... a process whereby (physicians) who no longer attend school on a regular fulltime basis.. .undertake sequential and organized activities with he conscious intention of bringing changes in information, knowledge, or skill, appreciation and attitudes; or for the purpose of identifying or solving personal, professional or community problems. ' p 294  Mazmanian and Duff propose the creation of "locally situated independent learning centers" bringing together medical educators, health services researchers and medical practitioners to develop "functional linkages among medical schools, hospitals, specialty societies and federally funded peer-review organizations." ' p  2 9 6  They suggest these  centres adopt the following three principles identified as indicators of successful Area Health Education Centers (AHEC) (see Table 20).  Table 20. Principles Identified as Indicators of Successful AHEC Programs (174) •  The formation of a new and permanent organization with participating schools and target communities that serve to decentralize and regionalize health professions education;  •  Strong linkage of the organization to health services delivery in the target area  •  The inclusion of key decision-makers from educational, service, and other organizations in the community who maintain authority for hiring staff and making program and budget decisions.  Part of this new vision is the creation of a new role for medical schools, providing peer mentors who will facilitate the development of formal and informal individualized curriculum informed by health services research including epidemiological reviews of community-based health care problems, and other patient care data, set of performance objectives developed by specialty societies, and standards of care or clinical guidelines. In essence, these independent learning centers define adequate health care performance, and permits self-assessment, self-monitoring, and self-directed planning of C M E activities to achieve specific objectives integral to the overall educative and health care performance of the individual practitioner. ' p  3 0 0  121  Mazmanian and Duffs vision of a different structure of C M E practice incorporates a number of ideas proposed by a series of Technical Reports on continuing education of health professionals commissioned by the European World Health Organization. However the vision is grounded more in the ideology of self-directed learning rather than addressing the pragmatic issues of systematically gathering a body of knowledge to inform learning and teaching, conducting and attending to a comprehensive diagnosis of the continuum of education, and addressing the many structural barriers to creating an alternative system of education.  Funding Support for CME Most academic or medical societies seek financial support to subsidize educational programming. Revenue sources include registration fees, exhibitor fees, and industry support. There is very little government support for continuing professional education. In Canada, the Canadian Institutes of Health Research may provide up to a maximum of $5,000 to support conferences and workshops. Industry sponsorship is a major source of revenue for C M E providers, however there is very little descriptive literature about how C M E is actually planned or what influence industry support has on educational planning. The table below (see Table 21) provides a breakdown of all C M E providers directly accredited by A C C M E and the level of industry support provided in relationship to total revenue reported, however there is little consistency in reporting financial data. (174)  Table 21. Breakdown of Accredited C M E Providers by Type and Total Income (175)  Type of CME Provider  # of CME Providers  Total Income  Communication Companies  22  86,169,205  Education Co. Other Health Care Delivery System  54 21  134,193,787 14,997,147  50 5 59  23,606,155 13,125,420 108,919,247  Hospital Consortium/Alliance Not for Profit Fdn  Income from Commercial Support 80,135,373 80,483,954 7,536,483 11,144,762  % of Commercial Support to Total Revenue 93% 60% 50%  5,865,103 47,786,242  47% 45% 44%  Physician Member Org Non Speciality  12  10,910,924  4,756,914  44%  Schools of Medicine  118  219,228,082  95,108,860  43%  196 61 10  10,696,164  4,002,796  37%  59,871,148 9,161,689  20 14  5,730,490 35,251,792  22,298,330 3,099,860 1,831,104  37% 34% 32%  9,345,539  27%  Physician Member Org Specialty Other Voluntary Health Association State Medical Society Publishing Co Education Co. Physician Owned  12  469,445,825 91,366,541 19% Government or Military 15 68,810,219 2,185,794 3% *Insurance/ Managed Care 11 1,072,287 24,336 2% Total 680 1,271,189,581 466,971,991 37% * HMO and insurance-based CME providers subsidize their own programming, not included in commerc support income.  Confusion in Purpose and Role Among Different Types of Accredited CME Providers The provision of C M E is no longer the purview of a profession or academic-based voluntary commitment to enhancing professional practice. In the US and to a lesser extent in Canada, C M E is big business, supporting a number of primary and secondary industries. Whereas there are a number of articles discussing the purpose and role of academic-based or professional society accredited C M E providers, there are very few articles in the medical literature describing the purpose and role of 'for-profit based ' or industry affiliated 'not-for-profit' accredited C M E providers. Obtaining information about more commercial-based or industry affiliated C M E providers appears to be difficult. A recent survey seeking profile information about communication companies, both non-profit and for profit, had a return rate of 25.2%. Of 182 companies identified,  p  only forty-six companies responded to a 21 item questionnaire with no information reported on the non responders. (132) Accrediting organizations and professional bodies have strict policies concerning industry-sponsored C M E . Common components being (i) the funder should provide the support in the form of an unrestricted educational grant, (ii) the C M E provider should be 100% responsible for the quality, content, and choice of speakers, (iii) speakers must disclose any relevant financial connections, (iv) the educational program should be the motivator rather than exhibits and entertainment. Although flinders should not be directly involved with the content, such conditions, however, do not prevent influence on the C M E provider choosing topics that are of interest to potential industry sponsors. (99) Katz et al. (99), investigating an academiaindustry collaboration in C M E between Harvard Medical School and a communication company, looked at 103 symposia offered at 4 Pri-Med conferences during the academic year 2000-2001. Pre-Med is a collaboration between M/C Communications, a marketing company, and Harvard Medical School "to create a wide reaching and low-cost continuing education program." The assistance of the marketing firm working with the faculty-based planning team led to a rapid growth in industry-sponsored breakfast, lunch and dinner sessions at Pri-Med conferences, with 40% of the delegates attending these symposia. O f 103 symposia offered at these four conferences, 94 were funded by a pharmaceutical company that had received US Food and Drug Administration approval within three years of the symposia for a drug related to the topic. Thirty-one percent of the C M E sponsors for these funded symposia were medical schools, 35% percent were non-profit organizations and 34% were for-profit medical education and communication companies. Katz et al. (99) ' p  4 8  warns that a reliance on this type of sponsorship would  likely lead to imbalances in C M E programming and recommends that when C M E is  funded by industry, multiple rather than single Hinders should be used, and C M E providers should guard against topic selection being influenced by funding sources. The threat of further commercialization of C M E programming is very real if alternative funds and resources are not found to support C M E practice. To address this threat I believe a clear consensus is needed on the purpose and role of C M E by public and professional stakeholders (governments, health professionals, medical schools, and consumers) concerned with the public good. This is an important question especially when considering the level of industry-sponsorship provided to the five major groups of C M E providers. For example, Pri-Med's three day courses providing 16 hours accredited study credits are offered for registration fees as low as $40 USD and are well received by physicians attending these courses. C M E planners must secure sufficient industry sponsorship in order to deliver low cost programming. With little government or alternative financial support for C M E programming, market pressures may play a more significant role in determining the types of educational programs available to physicians. The lack of alternative funding resources in the field is also a barrier to innovative programming as well as to the development and utilization of better evaluation tools. In addition, a systematic framework using international, national data, regional health data and professional data is needed to determine what educational programming is needed to enhance physician competence, adequately address population health issues and improve patient outcomes.  Summary As mentioned in Chapter One, Chen (22) posited that reliance on solely methodsoriented evaluation as currently constructed has not provided sufficient information to help us understand what is occurring in the "black box" of learning. I also suggest that  C M E planners and associated accreditation bodies have been "boxed in", relying too much on methods-oriented planning procedures associated with the Tyler Curriculum Model, and that a more open system of curriculum development is needed for the field of C M E . New concepts about learning, program planning practices and evaluation require a different conception about how to plan more efficacious educational activities. Chapter Four more formally describes the PRECEDE-PROCEED model, how it is conceptualized and used in the health promotion field, and describes how the model was interpreted for the purpose of planning, implementing and evaluating the case study.  126  Chapter Four  Utilizing the PRECEDE-PROCEED Model as an Analytical Tool for BCWI  In the introduction, I presented the recent utilization and modification of Green's (29) PRECEDE model in C M E as a framework for categorizing the type and potential efficacy of educational interventions in the C M E literature. In 1991, Green and Kreuter published a new text, Health Promotion Planning: A n Educational and Environmental Approach (30), and introduced the PROCEED part of the model, recognizing that other institutional and environmental factors influence knowledge uptake and behaviour change. Taken together, the PRECEDE-PROCEED Model, I argue, provides a more promising framework for guiding and analyzing C M E interventions such as the B C Whiplash Initiative (BCWI). This chapter will demonstrate just how the model was applied to the BCWI. The primary conceptual framework utilized for the planning, implementation and evaluation of the BCWI was based on Green and Kreuter's (30), PRECEDE-PROCEED health promotion model. The framework was not, however, used as a step-by-step guide. Rather, the model represented activities considered during the program development, implementation and evaluation, in a manner similar to Suchman (173) and Varela's (114) notion of "situated cognition", the utilization of embodied procedural knowledge, developed and learned through years of involvement with community-based organizations. The PRECEDE-PROCEED Model was familiar to some members of the steering and research committee active in the health promotion field and provided a common language  127  in identifying predisposing, enabling and reinforcing factors that could facilitate or impede knowledge uptake and behaviour change. Although the PRECEDE-PROCEED Model does not arise out of the adult education literature, I will argue that the model has a rich conceptual base that can have wider application in the field of adult education. It is also built on a set of principles that address a number of the challenges including ethics, power and interests discussed previously regarding deficiencies in current adult education models. (131) Whereas one of the main challenges discussed in the introduction of this thesis was the need to develop a stronger theoretical framework for C M E practices, the PRECEDEPROCEED Model provides an underlying planning process to identify systematically key contextual factors that can facilitate the creation and evaluation of theory-driven, enduring C M E programs.  Supporting Theories and Models Green and Kreuter's PRECEDE-PROCEED Model is founded on the notion that social context greatly influences knowledge transfer and uptake. Green and Kreuter do not explicitly define the socio-cognitive framework underpinning their model. They do refer to a number of specific models and theories in their text such as, Health Belief Model, Social Learning Theory, and Theories of Reasoned Action, that support the principles underpinning the PRECEDE-PROCEED framework. (30; 176) These theories and models, although shown to be fruitful for the purpose of facilitating individual health-related behaviour change, may also have value in the development of theoretical models to improve our understanding regarding what factors and conditions are necessary to facilitate behaviour change or other desired changes in the context of C M E . In this  128  chapter I will describe the above model and associated theories as they represent fundamental constructs underpinning the PRECEDE-PROCEED Model. As there is always some confusion over how best to define such terms as theories, concepts, constructs, models and variables to simplify and clarify, I rely on the definitions described by McKenzie and Smeltzer. (177) A theory is a set of interrelated concepts, definitions, and propositions that presents a systematic view of events or situations by specifying relations among variables in order to explain and predict the events of the situations [...] The primary elements of theories are known as concepts [...] When a concept has been developed, created or adopted for use with a specific theory, it is referred to as a construct [...] The operational (practical use) form of a construct is known as a variable. ' p  1 3 8  Models attempt to represent linkages among constructs believed to be relevant and bring together a number of theories to better understand a specific problem in a particular setting or context. The importance of these distinctions will become clear as I describe a number of theories and associated models underpinning the PRECEDE-PROCEED Model, beginning with the Health Belief Model  Health Belief Model The Health Belief Model (HBM) (see Figure 8) was developed by a group of social psychologists working at the US Public Health Service in the 1950s in an attempt to understand factors that could contribute to people adopting preventative health measures or screening tests for early prevention of asymptomatic disease. The essential premise of H B M is that behaviour change depends on two variables: the value placed by an individual on a particular goal, and the individual's estimate whether a given action or actions are likely to achieve that goal. (178) The H B M posit that demographic and other p8  psychosocial variables are influenced by various "cues to action." These cues to action are educational messages or experiences, such as mass media, advice from others,  129  reminder postcards from health care providers, and newspaper or magazine articles, that are more or less persuasive in conveying a perceived threat, and influence individual perceptions concerning the perceived risk of contracting the condition and the potential impact such a condition would have if contracted. The likelihood of changing an individual's behaviour is further influenced by an individual's beliefs as to whether the actions recommended are effective in reducing the risk, and what the perceived negative consequences are of taking the recommended action. Figure 8. The Health Belief Model (178) INDIVIDUAL PERCEPTIONS  p  1 0  MODIFYING F A C T S  L I K E L I H O O D OF ACTION  Demographic variables  Perceived benefits of  (age, sex, race, ethnicity, etc.)  preventive action  Sociopsychological variables  minus Perceived barriers to preventive action  Perceived susceptibility to Disease "X" Perceived seriousness (severity) of  Perceived threat of  Likelihood of taking recommended  Disease "X"  preventive health action  Disease " X "  A Cues to action Mass media campaigns Advice from others Reminder postcard from physician or dentist Illness of family member or friend Newspaper or magazine article  The H B M , as a first generational model, has been extensively researched with a large body of cumulative evidence in support of each construct (perceived benefit, perceived barrier, perceived susceptibility, perceived level of severity) as being significantly associated with health-related behaviour. (178)  p  '  1 0  Although the model has proven to be  fairly robust, theorists and researchers have been exploring other constructs to better  address some other intervening variables. For instance, some investigators have added a new construct to the H B M , that of "self-efficacy", recognizing the importance of the level of confidence a person has in being able to undertake a given behaviour change. From a social cognitive perspective the model's primary focus on individual attitudes and beliefs as being the primary determinant of whether a person makes a decision to change, may be too narrow. For instance, there may be economic or environmental factors preventing an individual from adopting a given behaviour change. But clearly, attitude is a critical factor in behavior change, no less so in a person acting on health information than in a physician implementing knowledge and skills learned through continuing medical education.  Theory of Reasoned Action The Theory of Reasoned Action (TRA) is a second model that the PRECEDEPROCEED Model draws on. (30; 176) It is similar to H B M , posits that a decision or intention to make a behaviour change is based on a combination of attitudes about an action, and considers an individual's perception of the impact of following a course of action within their social milieu. Rather than just seeing socio-demographic variables act as intervening factors on determinants of behaviour, T R A is based on the premise that ones perception of "normative influences" as a basis for reasoning within a given social milieu plays an important role in the prediction of behaviour. Concepts and constructs in this model might be useful to further explore the impact of undergraduate medical training on the creation of beliefs and attitudes concerning normative physician behaviour. Changes in this enculturation process could lead to new norms of behaviour more supportive of lifelong learning activities. The TRA, takes as axiomatic that individuals are rational or at least reasonable in their decision-making process.  131  Theory of Planned Behaviour The Theory of Planned Behaviour (TPB), although not specifically mentioned by Green and Kreuter, is an extension of the TRA. (178) TPB considers two primary variables - the strength of an individual's attempt to perform the behaviour and the degree of control the person has over the behaviour including knowledge, skills, time, money, willpower and opportunity. Consistent with Theory of Reasoned Action, the relative influence of attitude and subjective norm on intention is dependent on the behavioral goal. For some behaviors, the attitudinal component will be the major determinant on intention, while others, the importance of social pressure will be the primary influence. In general, the more favourable an individual's attitude toward attempting a behavior, and the more the individual believes that significant others are in favour of his or her trying, the stronger will be his or her intention to try. (178) ' p  1 1  The concepts and constructs of TPB are also relevant in the C M E environment. Theories such as TPB can be used to conceptualize and test educational strategies that use forms of peer pressure, such as the use of educational influentials, or utilize informal peer to peer communication networks. Building on existing theoretical models may lead to the development of more successful strategies for motivating physicians who are not successful in their first attempt of knowledge utilization, which has been found to be a barrier to continued knowledge uptake.  Principles Underpinning the PRECEDE-PROCEED Framework In addition to the above models and theories, the PRECEDE-PROCEED Model is based on a series of explicit principles that Green and Ottoson (179) state are derived from theories and practices discussed in adult education, social and behavioural sciences, program evaluation, epidemiology, administrative sciences, and communications, and are congruent with beliefs about conditions necessary to facilitate behaviour change for the purpose of health education and health promotion.  132  In Table 22 below I have selected principles of health promotion particularly relevant to C M E planning. I have renamed Green and Ottoson's "Principle of Health Promotion, to "Principle of Environmental Factors", as the principle speaks to the need to address organizational, legal and economic factors which are applicable to a broader range of professional education programming.  Table 22. Summary of Selected Principles of Health Education Applicable for C M E (179) pp.  95-8  Principles of: •  Cumulative Learning - Behaviour is the sum of a lifetime of personal and cultural experiences. Behaviour change requires a planned sequence of experiences and activities over time tailored to the circumstances and prior experiences of the target group. No single educational input should be expected to have significant or lasting impact on behaviour unless it is supported by other inputs.  •  Multiple Targets - Individuals' knowledge, attitudes and behaviour are intermediate to the final goals of a program. Targets must include social systems that enable its behaviour and structures/ relationships that reinforce behaviour.  •  Stakeholder Participation - Involvement of a broad base of stakeholders to identify the problems, assess their causes, and anticipate barriers to change. Early involvement influences curriculum design, implementation strategies and facilitates commitment of all those involved.  •  Situational Specificity - Utilize educational methods that are appropriate for the situation and informed by theories and principles arising from research.  •  Multiple Vehicles - Consider all vehicles for education including instructional methods, staff development, community methods (e.g. mass media), computer-based instruction. The best combination of educational methods, media and messages for some people is not necessarily the best combination for others. A variety of learning opportunities or experiences must be provided.  •  Intermediate Targets - Changes in resources, skill development and referrals enables or facilitates behaviour change. Significant others reinforce behaviour change. Behaviour change is the result of predisposing factors including changes in knowledge, attitudes, beliefs, values and perceptions. Focus on intermediate targets anticedent to the more distal outcomes desired.  •  Environmental Factors - Accomplishing voluntary behaviour change may be impaired by organizational, legal or economic factors that must be addressed with health promotion interventions.  •  Effective Administration - Adequate financial and human resources and a clear program plan with roles and responsibilities defined are required to successfully implement program planning.  133  These principles of health education and health promotion present a far broader conceptual understanding of factors to consider in program planning for the purpose of knowledge uptake and utilization than usually considered by planners using Tyler's Curriculum Model, educators adopting adult learning theory principles or the principles underpinning Gagne's model of cognitive dimensions of learning. Most of the above principles are congruent with the belief that for knowledge transfer to be effective, interventions must address environmental and social factors that hinder or facilitate adoption. For comparison purposes (see Table 23 below) I have grouped principles and assumptions discussed in the C M E and adult learning literature when describing "adult learning principles." Usually these principles and characteristics are presented in the literature as a list with little or no attempt to organize them into a useful typology. I have grouped these factors to help build a more informative nomenclature.  Table 23. Commonly Espoused Assumptions of Adult and Physician Learners Ability •  adults of all ages have the ability to learn (161)  •  people learn differently — differently from one another and on different occasion (161;180)  •  experience is a resource for learning (161;181)  Motivational Factors •  physicians are practical learners (i.e. they seek to solve problems, content relevant to practice, with immediate utility)(160;162;181;182)  •  adults are self-directed in their learning and wish to participate actively in their own learning (160-162;181-183)  •  adults learn by choice: learning is voluntary (161)  •  uses for learning change with different stages in a career (161)  •  physicians have multiple demands on their life and wish to meet their psychological needs of security, affiliation and self-esteem (160)  •  the learner's degree of motivation correlates directly with how that learner can incorporate new facts, concepts, skills, values, and so on (140)  Instructional Design Factors and Instructional Activities •  learners are more apt to make changes as a result of learning if they are have a clear image of what will be achieved (161)  •  active participation on the part of the learner tends to result in higher-quality learning than does passive, uninvolved experiences (161)  •  a problem-solving approach tends to foster both motivation and active involvement, and thereby facilitates learning (140; 183)  Cognitive Processing and Reinforcement •  repetition and reinforcement helps the learner remember information, especially if the reinforcement comes as a result of using the information (140)  •  reward and positive feedback are related principles and help promote useful learning, especially if this the reward is internalized, feedback is a critical part of learning (161)  •  multi-sensory signals (hearing and seeing) are helpful to most learners (140)  The PRECEDE-PROCEED Model, on the other hand, heavily emphasizes a comprehensive diagnosis of the nature of the educational, behavioural and attitudinal problems, the desired outcomes, the context of the clients, and the identification of potential barriers that could interfere with producing the desired outcome.  135  In order to make the case for greater utilization of the adaptability and feasibility of this model for C M E purposes, I will introduce each of the distinct phases of the PRECEDE-PROCEED Model, as it is discussed in the health promotion literature and describe how these principles were incorporated in the planning, implementation and evaluation of the BCWI. Figure 9. Phases of the P R E C E D E - P R O C E E D Model (30)  p  3 5  PRECEDE Phase 5 Administrative and policy diagnosis  HEALTH PROMOTION Education program  Phase 4 Educational and organizational diagnosis  Phase 3 Phase 2 Phase 1 Behavioral Epidemiological Social and diagnosis diagnosis environmental diagnosis  Predisposing factor*  Reinforcing factors  Behavior and lifestyle Quality of life  >  Policy regulation organization  Phase 6 Implementation  llnahling factors  Environment  Phase 7 Process evaluation  Phase 8 Impact evaluation  Phase 9 Outcome evaluation  PROCEED  Phases of the PRECEDE-PROCEED Model PRECEDE-PROCEED Model, Phases 1 and 2: Social and Epidemiological Diagnosis in the Context of Health Promotion Education In working with community health programs, the first phase of this model calls for an analysis of social problems or quality-of-life concerns. This phase identifies subjective  concerns and values of the community as well as objective data on social indicators, such as unemployment, violence and poverty. The second phase, closely linked to the first phase, involves assessing the incidence, prevalence, and cause of the health problems associated with the social problems in a given population: The sponsoring agency should use the most recent available demographic, vital, and sociocultural statistics of the subpopulation experiencing the health problem. The problem should be further analyzed on the basis of the experience of related agencies and a review of previously published reports. To gain perspective on the experience of the community with the health problem, similar data from other cities, states, or regions should be compared. (179) ' p  1 0 0  In some renditions of the PRECEDE-PROCEED Model, Phases 1 and 2 are represented graphically as one phase. (38) From an ethical perspective, the decision to move forward with a health program, after working through these first two phases, needs to be based on evidence that the issue warrants action and is remediable or can be effectively addressed, as opposed to other possible competing interests. Integral to the model is the involvement of a broad range of credible stakeholders who are in agreement that the problem is important and requires action. In the community health model, stakeholders include patients, consumers, parents and various health service providers. And while the phases below are described in a linear fashion, data and information collection for all phases begins upon the project's conception and the involvement with other stakeholders. Having set out the terms of the initial phases of the PRECEDE-PROCEED Model as establishing an ethical and practical basis for taking action, I would now turn to the BCWI to demonstrate how this model serves the particular needs of C M E .  137  BCWI Phases 1 and 2: Incorporating Social and Epidemiological Diagnosis in CME Planning For the purpose of C M E planning, the BCWI utilized Phases 1 and 2, and integrated a number of principles underpinning community health planning (see Table 22, page 133), participatory action research, and factors identified in the knowledge translation literature (see Table 24 below).  Table 24. Phases 1 and 2 Social and Epidemiological Diagnosis: BCWI Putting Principles into Practice (1) Identify and involve stakeholders from the beginning, including credible organizations and peer-respected individuals. Engage all stakeholders in the social and epidemiological diagnosis process. Identify subjective concerns of the community. (2) Gather objective data to assess the incidence, prevalence, and cause of the health problems associated with the patients presenting with W A D problems, in B C in relationship to other jurisdictions. (3) Base the decision to proceed on population health needs, rather than solely professional interests.  (1) Identify and involve stakeholders from the beginning, including credible organizations and peer-  respected individuals and identify subjective concerns of the community. Engage all stakeholders i the social and epidemiological diagnosis process To discern whether physicians and their organizations felt there was a need for C M E programming on whiplash-associated disorders, PMRF held a focus group with academic and professional organization leaders concerned with undergraduate medical education, family practice residency, family practice and C M E , to discuss whether the leadership believed there was a need for such an intervention. The meeting was called together under the leadership of Dr. Andrew Chalmers, former Chair of PMRF's Western Canada Multidisciplinary Committee and the Associate Dean of Undergraduate Medical Education. Dr. Chalmers was also former Head of the Department of Rheumatology at U B C . Don Gilbert from ICBC was invited as an observer.  138  From PMRF's perspective, it seemed reasonable that we should bring together all professional and related academic organizations to better assess and address potential learning needs through the continuum of medical education. Surprising to us this had not been done before. However, the idea of collaborating on a province-wide project was enthusiastically supported by all parties. Dr. Stephen Barron, liaison with the B C College of Family Physicians (later to become the co-chair of the BCWI steering committee) stated, "the lack of evidence-based education about whiplash-associated disorders is a pain in the physician's neck." This statement was later transformed into some of the marketing material - "Is dealing with whiplash patients a pain in your neck?" (2) Gather objective data to assess the incidence, prevalence, and cause of the health problems associated with the patients presenting with WAD problems in BC in relationship to other jurisdictions. BCWI used a number of different methods to assess the viability and relevance of the program including meetings with stakeholders, focus groups, review of needs assessments, and findings of strategic planning activities. The use of multiple methods of data collection provided subjective and objective information to help determine the need and potential relevancy of an intervention. PMRF held a series of multidisciplinary strategic planning sessions in 1986, 1988 and 1990, to identify major gaps and challenges in the musculoskeletal field and developed a strategic plan to address these gaps and challenges. One of the primary findings of these strategic sessions was the need to enhance family physicians' knowledge and skills in 32  the diagnosis and management of musculoskeletal conditions. PMRF had also reviewed two needs assessment surveys conducted by the Division of C M E , at the U B C in 1992 The undergraduate medical curriculum at most medical schools provides little clinical training in musculoskeletal medicine other than basic neurological tests and some limited history and diagnostic red flags for differential diagnosis purposes. Until the mid-1990s, and the use of evidence-based synthesis procedures, it was more difficult to glean value from non-systematic literature reviews due to problems related to selection bias and methodological problems of small sample sizes and conflicting results. 32  and 1995, looking at the perceived educational needs of family physicians and found there was a significant interest in further education on back pain, (back and neck pain were not differentiated on the surveys). In addition, PMRF runs a self-help support group for people with chronic pain conditions. From the consumer perspective anything we could do to enhance the quality of care in this area would be appreciated. Having personal experience with two rear-end motor vehicle accidents in 1988 and 1990 ,1 also 33  recognized the human face of dealing with neck and back pain in a not so perfect world of medicine, allied health care and insurance plans. During the proposal-building phase, an adhoc steering committee was created under the auspices of PMRF in partnership with the B C College of Family Physicians of Canada, U B C Division of Continuing Medical Education, U B C Department of Family Practice, Rural Education Training, U B C Family Practice Residency Program, and U B C Undergraduate Medical Education (see Appendix I: BCWI Committee Members). At the first meeting of the Adhoc Steering Committee there was frank discussion about the range of problems associated with the whiplash phenomenon. One of the first issues raised was the recognition that problems associated with W A D were not solely a medical problem. Other factors played an important role in its prevention, management and prognosis, such as the presence or absence of auto-engineering occupant protection systems , identification and effective management of psycho-social factors, and other 34  environmental factors including insurance and compensation dynamics, and level of satisfaction in ones work environment.  My second rear-end M V A took place at an intersection while I was stopped at a red light. The vehicle behind me was a public transit HandyDart bus for disabled people. The driver thought the light had changed to green! Between 1966 and 1979, the introduction of a variety of safety features in automobile design (laminated windshields.. .interior padding, lap and shoulder belts, increase side door strength...) helped reduce the vehicle accident fatality rate per mile traveled by 40%. Only three of the innovations added more than $10 to the price of the car, and in total they accounted for only 2% of the average price increase during 1975 - 1979. (World Health Report 2000, p. 6) 33  34  140  (3) Base the decision to proceed on population health needs, rather than solely professio interests. PMRF sought epidemiological and other sources of data to better understand the scope of the problem including its prevalence, the economic burden of illness on patients and society, comparative data across provinces in Canada, and provincial claims data from ICBC. Based on this data it was clear that musculoskeletal problems in general, and neck pain in particular, placed an enormous financial burden on society in terms of health care costs, compensation, and time away from work. In 1993, Health Canada released a report on the economic burden of illness. Musculoskeletal disorders and injuries were ranked second and third by diagnostic category with direct and indirect costs reaching $17.8 billion and $14.3 billion respectively. (68) Up to 85% of the population will suffer from musculoskeletal pain and of these, less than 10% of these sufferers consume 75% of the available resources. ICBC estimated W A D costs the motorists of B C about a half billion dollars annually, including wage loss, medical expense, pain and suffering awards. (184) In 1996, ICBC processed 45,500 whiplash-associated soft tissue injuries. Although most soft tissue injury claimants return to work within 30 days, 25% do not. In 1996, permanently disabled claimants made up 9% of the total number of claims, but accounted for 49%) of the total payments made. The whiplash claim rates in B C in 1995 were approximately 850 per 100,000 compared to 70 per 100,000 in Quebec and 90 per 100,000 in Saskatchewan. Claim rates in the US and around the world are very hard to interpret or compare due to differences in insurance policies, carriers, and various claims management strategies. (185) Easier to compare are crash rates. The crash rate in B C is 25% higher than the national average and 65% higher than in Alberta. This may be due to a number of factors including climate, topography, and lifestyle. (184)  141  The collection of these data was important for a number of reasons. It provided useful information to convey to all stakeholders concerning the justification of the program and the program funding needed. Population health data related to the impact of whiplash on patients and society reaffirmed the relevance of the program to the planning team target audience and was used in the curriculum to enhance participant interest and motivation. Both formal and informal data collection helped assess physicians' perceived educational needs. For instance, surveys reported that at least 42% of family physicians felt that they have been poorly trained to manage back and neck pain. The needs assessment surveys found that 71% of family physicians in B C deem further education in back pain a high priority (neck pain was not differentiated on survey) in 1988, and was still regarded a high priority in 1995. What was less clear were objective data related to the range of clinical decisions, preventive interventions, and the current medical and allied health care treatment of people presenting whiplash. (52) ' p  1 1 5  With input and interest expressed from a broad range of stakeholders supported by preliminary epidemiological data collected including its incidence, prevalence, and socioeconomic impact on society, the decision to move ahead was made.  BCWI Phase 3: Incorporating Behavioural Diagnosis in CME Planning The BCWI utilized Phase 3 of the PRECEDE-PROCEED Model with a focus on the practices of physicians in relation to their work and education regarding whiplash (see Table 25).  142  Table 25. Phase 3 Behavioural Diagnosis: BCWI Putting Principles into Practice (1) Assess undergraduate and residency training and current clinical practice (2) Evaluate the current literature that could inform improvement in practice (3) Identify and prioritize specific behaviour or other desired changes that could improve current practice based on appraisal of literature available (4) Reflect on the context of current practice, clearly identify what changes to routine clinical practice are desired (5) Identify beliefs, attitudes and values of the target audience and their milieu that could influence the adoption of innovation  (1) Assess undergraduate and residency training and current clinical practice From discussions with steering committee members involved with medical education, it was clear that physicians had no training in undergraduate or residency programs to address whiplash disorders unless it was through their own initiative. This was of little surprise as the QTF represented the first systematic review of the whiplash literature. In addition, other than the insurer's report form designed for insurance purposes, physicians had no systematic framework or decision-making aid to capture pertinent data for case history-taking or to guide and document a comprehensive physical examination. The lack of training and tools available was believed to be a major factor in physicians feeling uncomfortable with assessing, diagnosing, and managing patients presenting W A D . Members of the Steering Committee also believed that physicians would benefit from a refresher workshop in conducting a comprehensive physical examination.  (2) Evaluate the current literature that could inform improvement in practice To inform the behavioural diagnosis process all members of the Steering Committee received and reviewed copies of the scientific monograph of the QTF published in the journal SPINE. (52) This provided Steering Committee members an opportunity to reflect on the current state of knowledge and consider what behavioural changes were recommended in the literature.  143  To establish a 'gold standard ' of knowledge about W A D , the QTF collected bibliographic information on 10,382 articles published between 1980 and July 1994 on W A D . Based on a predetermined screening process using inclusion and exclusion criteria, 1204 studies were selected to be reviewed. Of these 1204 articles, once reviewed, only 294 were deemed to be scientifically admissible for a more in-depth review. Using a structured critical appraisal process, 62 studies were deemed to provide valid and useful data. The systematic process employed by the QTF was believed to be credible by members of the Steering Committee and produced a gold standard of knowledge about what was known and not known about W A D . The QTF, through a consensus process, also recommended importantly, a classification system to provide physicians with a consistent way of grouping patients' symptomatology and case history, in the belief that the use of this typology will lead to improvements in patient management. Moreover its clear it had utility for future prognostic and outcomes research.  (3) Identify and prioritize specific behaviour or other desired changes that could improve practice based on appraisal of literature available The QTF presented a number of recommendations related to clinical assessment, diagnosis and management. The core outcome objectives arising from the QTF report, as interpreted by the BCWI, were: •  Enhance physicians' knowledge, skills and confidence in the diagnosis and management of WAD  •  Improve documentation and procedures for case history taking, physical examination and utilization of the QTF proposed WAD Grade system.  •  Improve physician-patient communication especially the need to provide realistic patient reassurance  •  Reduce unnecessary diagnostic interventions (x-rays, MRI scans)  •  Reduce the use of narcotic-based pharmaceutical agents as well as muscle relaxants and long term use of NSAIDS  •  Promote active rather than passive rehabilitation.  144  (4) Reflect on the context of current practice, clearly identify what changes to routine clinical practice are desired Family physicians on the steering committee and curriculum development committee reinforced the need for the curriculum to address the context of a family physician's busy practice. Any recommended changes to clinical routines would need to fit into this milieu. In addition to the uptake of new knowledge and commitment to change clinical routines, tools would need to be developed to effectively and efficiently enable and remind a physician about the new system of diagnosing and managing patients. The steering committee agreed with the QTF assertion that part of the solution to improve physician's confidence in the diagnosis and management of W A D was the need to provide physicians with a standardized framework for managing patients, valid and reliable instruments for recording patient history and progress, and a knowledge-base informed by a systematic appraisal of the literature on natural history, diagnosis, management and prognosis. (5) Identify beliefs, attitudes and values of the target audience and their milieu that could influence the adoption of innovation The following concerns were identified: (a) Concerns about Industry Sponsorship An early concern of the steering committee was the need to have a clear separation between the content development of the project and the funder, ICBC. Some members of the Steering Committee were concerned that the funder might try to influence the content of the program. In addition, there was a concern that members of the target audience and other stakeholders might perceive the eventual education program to be tainted by possible vested interests of an insurance company.  145  (b) Negative or Ambivalent Beliefs/Attitudes about Evidence-based Medicine The adoption of evidence-based guidelines has met with resistance by some physicians. (186-191) Some contrary views include the notion that evidence-based medicine (EBM) is a form a cookbook medicine, that E B M does not sufficiently account or address individual variations, and the fear that recommendations arising from evidence-based guidelines will be inappropriately interpreted by stakeholders, particularly payers of health care services and thereby lead to a deterioration of patient care. (186; 187) Sackett et al. (187) ' argues that these concerns are antithetical to the p  1  purpose of E B M which he defines as "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients." Based on discussions with steering committee members, and reviewing internet discussions on the E B M Discussion List and opinion pieces in the medical literature, there were concerns that some physicians might have strong negative or ambivalent beliefs or attitudes about evidence-based medicine. From an educator's perspective, an important issue about this debate is that the promotion of evidence-based recommendations or guidelines per se, given such beliefs, may be a potential barrier to the adoption of better practice behaviour. (c) Concerns that Physician-Patient Communication Contributes to Iatrogenic Disability Committee members believed some physicians were providing patients with unrealistic prognostic expectations around symptom cessation, thus when symptoms did not cease undue anxiety might arise and potentially lead patients to believe they were more seriously hurt. Conversely, committee members believed some physicians did not acknowledge self-reported pain symptoms and that insufficient empathy could also lead  146  to future symptom magnification problems resulting in fostering pain behaviour and lead to iatrogenic disability. (d) Other physician-oriented concerns included •  The limited amount of time available for patient consultation.  •  Lack of patient education materials  •  Physicians' lack of confidence with clinical examination skills  •  The need to improve medical notes  •  Lack of training and financial support (fee schedules) in disability prevention or facilitating return to work  •  Marketing concerns fears that a CME program solely focusing on whiplash might not draw a large enough interest.  •  Need for further education about medico-legal reporting  At the first ad-hoc meeting of stakeholders it was recognized that whiplash was not just a medical concern, and concluded that improvements in physician diagnosis and management would not resolve the many complex social issues bearing on the social and medical consequences of people involved in a motor vehicle accident.  Non-Clincial Factors Discussed •  Need for improvement and use of auto-engineering protection features including head restraint and seat-back designs  •  Failure of legislative measures or interest in manufacturers to improve vehicle safety  •  Lack of public and stakeholder education about whiplash prevention  •  Vested interests of other stakeholders including insurers, lawyers, payers of health care services.  •  Lack of quality research investigating implications of current system-based problems including social policy issues on disability determination and compensation, impact of different legal and compensation models on WAD prognosis.  •  Need for better research in the prevention, diagnosis and management of whiplash patients  The systematic identification of a broad range of contextual factors played an instrumental role in the curriculum planning process and informed the creation of the program content, selection of instructional design features, and the development of resource materials which is further discussed in Phase 4. 147  PRECEDE-PROCEDE Phase 4: Educational and Organizational Diagnosis in the Context of Health Promotion Education Phase 4 involves the categorization of identified behavioural and environmental determinants into three groups - predisposing, enabling, and reinforcing factors that form the basis of an educational and organizational diagnosis. Once identified and integrated with data collected during the prior phases, educational programming can be developed. Predisposing factors have been characterized in different ways in the health promotion literature. Green and Kreuter's (30) definitions are as follows: •  Predisposing factors are those antecedents to behaviour that provide the rationale or motivation for the behaviour.  •  Enabling factors are the antecedents to behaviour that facilitate a motivation to be realized.  •  Reinforcing factors are factors subsequent to a behaviour that provide the continued reward or incentive for the behaviour and contribute to its persistence or repetition.  The typical predisposing factors discussed are knowledge, beliefs, values, attitudes and confidence. Enabling factors include skills and the accessibility of resources that make it possible for a motivated person to take action. Reinforcing factors are the attitudes and the climate of support facilitating behaviour change including family, peers, teachers, employers, health providers, community leaders and decision-makers. Unlike traditional C M E programming, educational objectives are not just related to content-oriented objectives but become the intermediate or sub-objectives to the behavioural or non-behavioural factors identified. Green and Ottoson (179) classify these objectives as informational objectives, attitudinal objectives, competency objectives, community organization objectives and training objectives. In the classification system below I have added environmental objectives to specifically target environmental factors, such as internal administrative policies or other environmental factors that a physician may be able to change, as against community organization objectives which relate to 148  environmental factors outside of the work environment that can influence behaviour. I have also called training objectives, community-based training objectives to clarify that these training objectives are directed at other stakeholders (see Table 26). Table 26. Subclassification of Educational Objectives for Health Promotion Modified for C M E Purposes (179) p p  1 0 6  7  •  Informational objectives relate to identifying what content, beliefs and level of understanding are necessary for a member of the target audience to address a desired change.  •  Attitudinal objectives relate to the predispositions or feeling people have towards certain health problems and specifically looks at motivational issues - what will sufficiently motivate someone to take an appropriate action  •  Competency Objectives relate to specific procedural or competency skills necessary to fulfill a specific behavioural objective.  •  Environmental Objectives relate to identifying what administrative, policy or other environmental factors need to be addressed to facilitate knowledge utilization.  •  Community Organization Objectives relate to issues related to access of services, possible changes needed in community priorities and resources.  •  Community-based Training Objectives focus on specified changes in knowledge, behaviour, and attitudes of colleagues, parents, employers, peers and others who have direct influence over the people whose behaviour needs to change and can encourage or reinforce desired changes (both from a physician and patient perspective).  The overall thrust of the educational and organizational diagnosis is the need to identify potential barriers and factors that can facilitate the desired change and then specifically develop an action plan that mitigates barriers and effectively utilizes those factors that can facilitate the desired change. In the health promotion community examples of barriers could include: ...social, psychological, and cultural barriers include citizen and staff bias, prejudice, misunderstanding, taboos, unfavorable past experiences, values, norms, social relationships, official disapproval, rumours. Communication obstacles include illiteracy and local vernacular. Economic and physical barriers include low income and inability to pay for prescribed drugs.. .Legal and administrative barriers include residence requirements to be eligible for services, legal requirements... policy or regulations that restrict program implementation. (179) p. 107  149  Facilitators of behaviour change could include past positive experience with other community-based projects, the credibility of the organizations involved, involvement and support of local leaders, the use of existing delivery systems, support of various communication channels including schools, local media, clubs, churches, neighbourhoods and ethnic organizations. Green and Ottoso (179) provide a planning overview (see Figure 10) that shows the relationship between tnhe types of training objectives and their relationship to behavioural objectives and the health status objectives of community health programs.  Figure 10. Relationship of Educational, Behavioural and Health Status Objectives in Planning Community Health Program (179) p  Objectives  Attitudinal  107  Competency  Community Organization  Informational Directed primarily at  Predisposing Factors  Training  Enabling Factors  Reinforcing Factors  Health Behaviour  To achieve With ultimate outcome  Donaldson et al. (192) P P -  Consultative  Health  458  "  460  m 0  r e recently describes this phase of planning to  consist of a number of program components "often intended to lead to immediate or proximal changes (mediators) in participants that are presumed to later cause more distal outcomes." The program activity therefore is targeting factors believed to be antecedent to the desired outcome. Donaldson et al. represents this schematically (see Figure 11).  150  Figure 11. Three Component Program (192)  p  Program Component 1 Component 2 Component 3  Theory-driven educational planning in health promotion is conceived as a process in which planners construct programming components that are thought, or for which there is evidence, to influence proximal intervening or causal mediators that in turn are believed to influence patient health outcomes. Bartholomew, Parcel et al. (193) have provided an overview schematic of ah intervention mapping process being proposed for planning health promotion programs (see Figure 12).  Figure 12. Intervention Mapping Process Proposed for Health Promotion Planning (193) p. 10 Needs assessment Review key ^ determinants  1 ~  Distinguish environmental and behavioral causes  a n d h e a l t h  p r o b  e  S  Intervention map Products  Tasks  Proximal program objective matrices  • State expected changes in behavior and environment • Specify performance objectives • Specify determinants • Create matrices of proximal program objectives, and write learning and change objectives  Theory-based methods and practical strategies  • Brainstorm methods • Translate methods into practical strategies • Organize methods and strategies at each ecological level  Program plan  • Operationalize the strategies into plans, considering implementers and sites • Develop design documents • Produce and pretest program materials with target groups and implementers  \ ^  population, quality ot life, | m  Evaluation  Adoption and implementation plan  • Develop a linkage system • Specify adoption and implementation performance objectives • Specify determinants • Create a matrix or planning table • Write an implementation plan  Evaluation plan  • • • • •  ?  Develop an evaluation model Develop effect and process evaluation questions Develop indicators and measures Specify evaluation designs 1 Write an evaluation plan X  Implementation - * Bartholomew, Parcel et al. (193) state: Intervention Mapping is a systematic process that explicates a series of steps and procedures for the development of health education programs based on theory, empirical findings from the literature, and data collected from the population. ' p  1 3  Bartholomew, Parcel et al. provide an excellent integration of the PRECEDEPROCEED Model as the foundation for intervention mapping with pragmatic suggestions  152  about various methods and strategies for planning, implementing and evaluating theorydriven programs. However missing from this recommended process is the integration of evidence-informed pedagogical research to inform the curriculum development process. The creation of theory-driven intervention research in health promotion and other fields is relatively young. Meta-analysis was first introduced in 1976 in the social sciences by Gene Glass who "galvanized the research community" with his article entitled Primary. Secondary and Meta-Analysis of Research. (194) ' p  235  Research  especially in the applied behavioural sciences provides a growing body of evidence that theory-driven programming and testing of theoretical constructs has potential for improving practice in many fields. Meta-analytic approaches are used in the field of education, for instance, to assess the impact of class size, computer-based instruction, feedback and test performance, grade retention and achievement and teacher expectancy effects. (195)  p  '  2 4 1  The utilization of theory, such as the Health Belief Model, provides  promising evidence that theory-driven programming may lead to improvements in educational planning. However there are many methodological challenges that need to be addressed to improve meta-analytic procedures and evaluation reporting practices in order to reduce errors in the use and findings of meta-analytic reports across various fields and disciplines. (196-198) Given the promising use of program theory in other areas of research concerning human behaviour, the creation of program theory for C M E purposes has greater potential in better understanding factors contributing to knowledge acquisition and knowledge utilization than current C M E planning models. To provide a better understanding about translating Phase 4 for C M E purposes I will describe how it was applied in the BCWI and my role in this process.  153  BCWI Phase 4: Incorporating Educational and Organizational Diagnosis in CME Planning Background to the Curriculum Development Process For the purpose of accurately describing this phase and my role in this activity I need to provide additional background. I was the lead project planner and primary author of the proposal. The project planning activity was a collaborative process melding the experience and knowledge of members of the adhoc steering committee. As the lead project planner, I brought my knowledge and experience in community planning including facilitating stakeholder involvement and negotiation, and pedagogical knowledge, to prepare the funding proposal that was used as the blueprint for the overall BCWI project. As Executive Director of PMRF, I was ultimately responsible for overseeing the project. The contract for the project was between PMRF and ICBC with a series of subcontracts between PMRF and the other stakeholders involved in C M E program implementation and delivery. Once the project was approved, a project coordinator and additional staff were hired to assist with program development and logistics. For the first three months of the project I was on sabbatical, fulfilling a three-month residency requirement for doctoral studies and received permission to study with Professor Patricia Wright, Senior Scientist at M R C Psychology Unit, University of Cambridge, England. Professor Wright is a wellrecognized leading researcher and scholar in the field of cognitive ergonomics. During my absence, the project began making some headway in the curriculum development process. However there were some conflicts in pedagogical styles between members of the steering committee and the hired project coordinator, which I will address later in this section, and there was a need for leadership in the research area. Upon my return, I became active in overseeing the curriculum development process, in a  similar manner as Cervero and Wilson (131) conceptualize the work of a program planner as being primarily concerned with the role of a negotiator among various interests. Marc Broudo and I were responsible for planning and applying current research in cognitive ergonomics to enhance the instructional design aspects of the final curriculum products (presentation, typographical elements, referencing and indexing systems, tabbed indexes, use of graphics, algorithms and clinical presentations). In addition to this role the Research Committee appointed and supported me in the role as Chair of the Research Committee. Phase 4 of the educational and organization diagnosis, described by Green and Kreuter (30; 176), begins the formal process of program planning, integrating program objectives with concrete actions to address factors believed to facilitate or hinder the outcomes desired. This activity was an iterative and synthesizing process bringing together data collected from initial and continuing discussions with stakeholders, reviewing the literature on whiplash-associated disorders, reviewing recent systematic reviews on C M E and related educational interventions, and the application of pedagogical related research. Once the program was implemented, feedback was provided through process evaluation activities and programming components were modified or refined. Based on our experience with the application of Phase 4,1 will propose modifications to the PRECEDE-PROCEED Model specifically to address C M E planning and programming needs, however some of these modification might also be relevant to health promotion planning. These modifications will be discussed in more detail in Chapter Five.  Curriculum Development Process - Translating Knowledge into Curriculum I have synthesized principles and various classification systems discussed in the health intervention literature concerning community health interventions including  PRECEDE-PROCEED, and pedagogical literature to organize and report on the curriculum development activities undertaken by the BCWI (see Table 27).  Table 27. Phase 4 Educational and Organizational Diagnosis: BCWI Putting Principles into Practice (1) Develop overall program planning goals. (2) Utilize recent systematic reviews in CME and educational research to inform decisions about the types of interventions and educational methods used that have shown to be effective or promising in other programs targeting a similar audience. (3) Use combination of predisposing, enabling and reinforcing intervention strategies to facilitate behaviour change. Utilize multiple formats and multiple strategies to encourage participation in one or more activity. (4) Categorize key factors identified in Phases 1 - 3 that are believed to impact knowledge transfer and utilization into predisposing, enabling and reinforcing factors. Use credible messengers and familiar dissemination channels endorsed by credible organizations. Plan for both formal and informal education inventions. (5) Identify a range of objectives including content (declarative knowledge), competency (procedural knowledge), attitudinal, environmental and community-based objectives. For each set of sub-objectives plan specific actions to mitigate factors believed to hinder uptake and take full advantage of those factors that can influence knowledge uptake. Address key non-behavioural, environmental and community-based factors which are achievable. (6) Apply pedagogical principles and standards in translating knowledge to curriculum ensuring the translation process addresses issues of content validity, clinical relevance, sequencing, readability, and the presentation is succinct and engaging. Integrate sociocontextual factors and content-related issues in the planning process. (7) Provide pedagogical training to teachers to enhance knowledge and skill development. Identify and discuss problematic content areas, and enhance and refresh interactive learning strategies. Although the principles above are not necessarily sequential steps, an early step in the process should be a careful review of systematic reviews regarding C M E and pedagogiocal-related literature to build on existing knowledge. 1. Develop overall program planning goals. The overall guiding principle for planning the BCWI intervention was that C M E planners should utilize best evidence regarding pedagogical practices and C M E research in developing C M E programming. In addition to Green and Ottoson's categories of types of subordinate objectives, the BCWI created overall program planning objectives (see Table 28). 156  Table 28. B C W I Initial Statement of Overall Program Objectives •  To develop cost-effective assessment, diagnosis and management protocols for patients presenting with WAD.  •  To produce a C M E module that would be of long term service in the training and ongoing education of family physicians in Canada and elsewhere.  •  To be an innovative teaching model informed by the CME literature, built on multidisciplinary expertise and sophisticated principles of instructional design.  •  To enable family physicians to become better equipped to improve the early intervention and management of patients.  •  To develop and test a comprehensive and rigorous evaluation instrument for determining the effectiveness of the proposed CME module which will be designed to enhance the ability of family physicians to utilize cost effective assessment, diagnosis and management protocols for patients presenting WAD.  •  To reduce the pain and suffering of the patients with WAD.  Creating overall program planning objectives provided an opportunity for stakeholders to reflect on the bigger picture. For instance, how could this project contribute to the common good? This may be an important consideration as the creation of enduring programs requires long-term commitment of stakeholders and having a greater sense of purpose may be a motivating factor. From a C M E perspective, the project held both an academic and pragmatic interest to academic stakeholders and members of the target audience. The opportunity to create an evidence-informed educational intervention and to conduct a more sophisticated evaluation of the program was also seen as an area of great importance. In the early negotiations with ICBC, PMRF made it clear that the project would not proceed unless adequate resources were available to support a comprehensive evaluation of the project. Having overall program planning objectives also provided a context for planning or considering more immediate goals and objectives. Before discussing more immediate program objectives, the next step in the planning process was to review the C M E literature and most recent systematic reviews. By looking at the literature planners can look at what factors other C M E planners and researchers have considered in categorizing predisposing, enabling and reinforcing 157  factors both in terms of determinants of individual change, but also as a way to conceptualize various educational strategies used in other studies.  (2) Utilize recent systematic reviews in CME and educational research to inform decision  types of interventions and educational methods used that have been shown to be effectiv promising in other programs targeting a similar audience. We began planning the types of educational interventions at the proposal building phase of the project, utilized Davis et al.'s (199) summary of primary and secondary interventions (see Table 29) and reviewed other systematic reviews and summaries. (6;15;18;19;199;200) Based on this review it was clear that aside from a few very specifically targeted interventions, those where economic incentives could be successfully employed, there was no magic bullet available that was proven consistently to be an effective single strategy to change physician practice, rather multiple strategies appeared to be more successful. (19;201) Davis et al. (199) reported on a systematic review of 50 Randomized Control Trials (RCTs)(18) of 74 discrete interventions that had, as a primary focus, the uptake of information. Using a modified taxonomy developed by Davis et al. (18), interventions were grouped for analysis by type of educational intervention planned. Type 1 were interventions using predisposing methods only. Predisposing methods included conveying information using didactic sessions - lectures, courses, conferences and grand rounds. Type 1 interventions generally demonstrated negative or inconclusive results. Although 7 of 11 studies displayed positive performance changes, none of the six attempts to change patient health status did so. (18) Factors contributing to changes in practice behaviour included practice strategies, case discussion and the opportunity to rehearse or consider practice behaviour. Type 2 interventions used 'enabling' strategies. Nine of the ten studies using enabling strategies included various aids that may be built 158  into a physician's routine behaviour such as flow charts, algorithms, patient education materials, consultation, and reference materials. These "enabling" strategies produced more successful outcomes on physician behaviour. Studies where knowledge testing and practice needs assessment strategies were employed also improved some aspects of physician performance. Type 3 methods were mainly feedback and/or reminders in conjunction with predisposing methods. In Davis et al.'s (18) study, 18 of 26 studies employing this type of intervention strategy to change physician behaviour had positive outcomes, and 6 of 9 studies seeking changes in patient outcomes using Type 3 methods were successful. Type 4 interventions used a combination of all three methods and produced positive results in 14 of 14 studies intending to change physician performance and 5 of 9 studies seeking patient health outcomes. Davis et al. (199) also provided a summary of primary and secondary interventions as a guide to consider programming choices (see Table 29).  159  Table 29. Primary and Secondary Interventions in Continuing Medical Education (6)  Interventions Primary  p  Synonyms, Examples  Interventions  Formal C M E  Didactic s e s s i o n s Courses Conferences Rounds  Printed Materials  Newsletters Bulletins Texts, journals Practice guidelines, clinical policies  Audiovisual M e t h o d s  Audiotapes Videotapes Videodiscs  A c a d e m i c Detailing  Physician educator visits  C o m p u t e r A i d e d Instruction Secondary  Interventions  Enabling  Flow charts, algorithms Patient education materials Patient specific information Question in practice programs Consultation  Reinforcing  Reminders Feedback  M i x e d or multi-potential  Opinion leaders/educational i n f l u e n t i a l Chart review  (3) Use combination ofpredisposing, enabling and reinforcing intervention strategies to facili  behaviour change. Utilize multiple formats and multiple strategies to encourage participat or more activity. Based on these results and considering a broad spectrum of educational research, a comprehensive educational strategy evolved using a combination of predisposing, enabling and reinforcing educational strategies. The predisposing strategies utilized in the BCWI included: (a) conducting a needs assessment (review of recent needs assessments 160  and focus group with professional leadership); (b) the involvement of credible organizations and recognized leaders within the profession; (c) use of peer-selected educational influentials (Els) as trainers; and (d) trainers' workshop regarding planning successful programs and content review. Enabling strategies included: (a) cueing of prior knowledge through the use of pretest; (b) adult-oriented educational program including lecture, case-based group discussions and hands-on clinical examination workshop; and (c) use of Els for informal diffusion. Mixed strategies (enabling and reinforcing) included: (a) instructionally designed, tab indexed comprehensive syllabus as both an instructional tool and reference manual, (b) use of 'Memos to Myself participants immediately after exposure to the core material were asked to translate their learning into specific behaviour changes (a reminder was sent to participants three weeks after the program). Reinforcing strategies included: (a) immediate feedback on knowledge test results prior to and immediately after the course, (b) distribution of patient and physician educational material post course, (c) distribution of delayed posttest results indicating correct and incorrect answers with referenced answer key, (d) distribution of summary of peers intended behaviour changes collated from Memos to Myself, and, (e) access to content information through website. Other challenges discussed in the literature for C M E were the need to provide greater variety in programming as different types of educational activities would attract different audiences and/or encoura