UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Assessment and evaluation practices in a pediatric dentistry clinical education setting Wong, Tracy Jean 2012

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2012_fall_wong_tracy.pdf [ 1.3MB ]
Metadata
JSON: 24-1.0055373.json
JSON-LD: 24-1.0055373-ld.json
RDF/XML (Pretty): 24-1.0055373-rdf.xml
RDF/JSON: 24-1.0055373-rdf.json
Turtle: 24-1.0055373-turtle.txt
N-Triples: 24-1.0055373-rdf-ntriples.txt
Original Record: 24-1.0055373-source.json
Full Text
24-1.0055373-fulltext.txt
Citation
24-1.0055373.ris

Full Text

ASSESSMENT AND EVALUATION PRACTICES IN A PEDIATRIC DENTISTRY CLINICAL EDUCATION SETTING by Tracy Jean Wong D.M.D., The University of British Columbia, 1983  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF ARTS in The Faculty of Graduate Studies  (Curriculum Studies)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  July 2012  © Tracy Jean Wong, 2012  Abstract In clinical teaching and learning settings, there is a need for assessment and evaluation practices to be focused on students' overall performance during patient care, not just technical skills in Dentistry. Competency-based education is intended to provide the framework for dental education at The University of British Columbia (UBC) in terms of curriculum content and assessment of student learning outcomes. Clinical instruction in disciplines such as Pediatric Dentistry depends on clinical practitioner-instructors who have potential to make important contributions to student development. Although they bring strengths as disciplinary experts immersed in the realities of dental practice, most are not well versed in research-based instructional strategies to engage students in critical thinking and self-directed learning for the rigours of independent practice. In a qualitative study, data were collected by the author (resident Program Coordinator of the UBC Children's Dental Program) through interviews, observation in teaching clinics, and review of documentation to inform the scope and nature of assessment and evaluation practices in the clinical educational settings of Pediatric Dentistry at UBC. Interview data also provided reflections about how clinical practitioner-instructors understand their practice. Data collected were analyzed using principles of grounded theory and merged into themes drawn from the conceptual framework of Hubball and Burt (2004) as well as the use of the UBC Faculty of Dentistry patient performance care criteria and standards for student learning. Assessment and evaluation practices in clinical settings typically ranged from predominantly directive methods (e.g. traditional teacher-driven and skills-based) that clinical practitioner-instructors experienced themselves as students, to occasional learning-centred ii  methods (e.g. instructor questioning, self-analysis, and reflection) supported by current literature. While clinical practitioner-instructors recognized the importance of student confidence and safety of patient care, most were unfamiliar with authentic methods of assessment and evaluation for competency-based dental education. Further, there was little reflection or collaboration within the community of practitioners about the effectiveness of assessment methods. These results and a research-informed approach will guide planning of faculty development initiatives (e.g., learning communities focused on learning-centred assessment and evaluation strategies) for clinical practitioner-instructors.  iii  Preface A reflective paper examining the key barriers and emergent strategies for curriculum renewal in the UBC Dentistry program context was previously published. Wong, T. J., & Hubball, H. (2011). Examination of Curriculum Reform in a Four-year Program of Pediatric Dentistry. Transformative Dialogues: Teaching & Learning Journal, 5(1), 1-12. As Clinical Assistant Professor in the Faculty of Dentistry at the University of British Columbia (UBC) and the Program Coordinator of the UBC Children’s Program, I wrote most of the manuscript. Harry Hubball provided expertise in the area of the scholarship of curriculum practice. Ethics approval was required for this research and obtained through the UBC Behavioural Research Ethics Board, Certificate Number H11-02779.  iv  Table of Contents Abstract ........................................................................................................................................... ii Preface............................................................................................................................................ iv Table of Contents ............................................................................................................................ v List of Tables ................................................................................................................................. ix List of Figures ................................................................................................................................. x Acknowledgements ........................................................................................................................ xi Dedication ..................................................................................................................................... xii 1.  Introduction ............................................................................................................................. 1  2.  Context of Pediatric Dentistry at UBC Faculty of Dentistry ................................................... 3  3.  2.1  UBC predoctoral dental program ..................................................................................... 3  2.2  Clinical training in Pediatric Dentistry............................................................................. 4  2.3  Assessment and evaluation by clinical practitioner-instructors ....................................... 5  2.3.1  Verbal one-to-one feedback ...................................................................................... 5  2.3.2  Group huddles ........................................................................................................... 6  2.3.3  Instructor rating sheet and patient care performance rubric ..................................... 7  2.4  Strengths of clinical practitioner-instructors ..................................................................... 8  2.5  Limitations of clinical practitioner-instructors ................................................................. 9  Literature Review .................................................................................................................. 11 3.1  Educational needs of the modern-day dental graduate................................................... 11  3.1.1 3.2  Competency-based dental education....................................................................... 12  Role of the clinical practitioner-instructor ..................................................................... 14  3.2.1  Disciplinary and pedagogical expertise .................................................................. 15  3.2.2  Educational reform and the clinical practitioner-instructor .................................... 15  3.3  Learning-centred pedagogy ............................................................................................ 17  3.3.1  Pedagogical content knowledge .............................................................................. 17  3.3.2  Principles based on research for good teaching and learning ................................. 18  3.3.3  A learning-centred approach ................................................................................... 19  3.3.3.1  Active learning ................................................................................................ 20  3.3.3.2  Learning as a social process ............................................................................ 21  3.3.3.3  Diversity in learners and teachers .................................................................... 21 v  3.3.3.4 3.4  A link to clinical competencies ............................................................................... 24  3.4.2  Helping students to be self-directed learners .......................................................... 25  3.4.3  Challenges to authentic methods of assessment and evaluation ............................. 26  3.4.4  Reflection on practice ............................................................................................. 27  Faculty development ...................................................................................................... 28  3.5.1  Faculty development in health professions education ............................................ 28  3.5.2  A multi-faceted approach ........................................................................................ 30  3.5.3  Challenges in planning faculty development .......................................................... 32  Methodology.......................................................................................................................... 34 4.1  Data collection................................................................................................................ 35  4.1.1  Interviews ................................................................................................................ 36  4.1.2  Observation during patient care and in group huddles ........................................... 36  4.1.3  Review of formal documented assessments and evaluations ................................. 37  4.2 5.  Assessment and evaluation in clinical practice settings ................................................. 23  3.4.1  3.5  4.  Authentic assessment and evaluation .............................................................. 22  Data analysis .................................................................................................................. 39  Results ................................................................................................................................... 41 5.1  Study participants ........................................................................................................... 41  5.2  Data collected from interviews ...................................................................................... 43  5.2.1  The context of clinical dentistry for assessment/evaluation ................................... 45  5.2.2  Planning for assessment/evaluation ........................................................................ 47  5.2.3  Programming strategies for assessment/evaluation ................................................ 50  5.2.4  Criteria and standards for assessment/evaluation ................................................... 53  5.3  Data collected through observation ................................................................................ 56  5.3.1  The dental clinic setting .......................................................................................... 57  5.3.2  Ongoing feedback and assessment for the individual student ................................ 57  5.3.3  Group huddles ......................................................................................................... 60  5.4  Data collected from documented assessments and evaluations ..................................... 61  5.4.1  Daily and global periodic assessments ................................................................... 62  5.4.2  Quantitative grades and qualitative comments ....................................................... 63  vi  6.  Discussion .............................................................................................................................. 65 6.1  6.1.2  Health promotion and community service .............................................................. 67  6.1.3  Evidence-based versus “in my experience” practice .............................................. 67  6.1.4  Real-world challenges of dental practice ................................................................ 69  6.2  Planning for assessment and evaluation ......................................................................... 69  6.2.1  Gaps in a learning-centred approach....................................................................... 70  6.2.2  Prior learning .......................................................................................................... 71  6.2.3  Student feedback and critical reflection .................................................................. 72  6.2.4  Needs assessment .................................................................................................... 74  6.3  Programming and learning strategies for assessment and evaluation ............................ 75  6.3.1  Goal-directed practice ............................................................................................. 75  6.3.2  Setting expectations around student independence................................................. 77  6.3.3  Targeted feedback ................................................................................................... 78  6.3.4  Demonstration and modeling .................................................................................. 79  6.3.5  Communities of practice ......................................................................................... 80  6.3.6  Opportunities for students to self-assess ................................................................. 81  6.3.7  Management of challenging students...................................................................... 82  6.4  7.  Assessment and evaluation within the context of clinical dental education .................. 65  Application of criteria and standards for assessment and evaluation............................. 83  6.4.1  Consistency of application of standards to clinical performance criteria ............... 83  6.4.2  Grading student independence and patient safety................................................... 84  6.4.3  Grade inflation ........................................................................................................ 85  6.4.4  Grading student performance that does not meet expectations .............................. 85  6.4.5  Toward self-directed learning and self-assessment ................................................ 87  Conclusions/Reflections ........................................................................................................ 88 7.1  Implications of inquiry-based practice for dental education .......................................... 90  7.2  Recommendations .......................................................................................................... 92  7.3  Limitations of the study.................................................................................................. 93  7.4  Further study .................................................................................................................. 94  7.5  Dissemination opportunities ........................................................................................... 95  7.6  Final reflections .............................................................................................................. 96 vii  References ..................................................................................................................................... 98 Appendices .................................................................................................................................. 110 Appendix A. UBC Dentistry Competencies for the New Practitioner .................................... 110 Appendix B. UBC Children’s Dental Program Syllabus ........................................................ 116 Appendix C. Patient Care Performance Rubric ...................................................................... 120 Appendix D. Interview Consent Form .................................................................................... 121 Appendix E. Interview Questions ........................................................................................... 122  viii  List of Tables Table 1. Research Methodology ................................................................................................... 34 Table 2. Participants...................................................................................................................... 43 Table 3. The Context of Clinical Dentistry for Assessment/Evaluation ....................................... 46 Table 4. Planning for Assessment/Evaluation .............................................................................. 49 Table 5. Programming Strategies for Assessment/Evaluation ...................................................... 52 Table 6. Criteria and Standards for Assessment/Evaluation ......................................................... 55  ix  List of Figures Figure 1. Conception of learning: Images as metaphors for interview discussions...................... 38 Figure 2. Developing, implementing, and evaluating learning-centred curricula ........................ 39 Figure 3. Global and enabling learning objectives ....................................................................... 48  x  Acknowledgements I would like to express my gratitude to my colleagues and the students in the Faculty of Dentistry who have inspired me to pursue my work in this field. Not only have my colleagues provided encouragement and support, but also many have willingly shared their perspectives as study participants. I thank Harry Hubball for his mentorship and for broadening my vision of the scholarship of teaching and learning on so many different levels. Many thanks to Tony Clarke for his feedback sprinkled with pragmatism and metaphor. Finally, my thesis would not have been completed without the polishing touches of Teresa Farn and Fang Ling (Finnie) Xu.  xi  Dedication  To my family  xii  1.  Introduction There is a call in the field of dental education to examine the needs of the twenty-first  century dental practitioner. The dental profession has recognized that “new ways of doing things” need to be implemented in clinical teaching and learning and that authentic assessment and evaluation methods must be focused on students’ overall performance during patient care, not just technical skills (Albino et al., 2008; Hendricson et al., 2007). Competency-based education provides the framework for dental education at The University of British Columbia (UBC) to define core content of curriculum and assess outcomes. A document, “UBC Faculty of Dentistry Competencies for the New Practitioner” (2006), outlines six domains of professional activity and responsibilities related to the general practice of dentistry—professionalism; practice organization; assessment of the patient and the oral environment; health promotion; establishment and maintenance of a healthy oral environment; and rehabilitation of form, function, and esthetics (Appendix A).  At UBC Dentistry, many dentists from the practice community provide clinical instruction. Because of their experience practicing in the field, these clinical practitionerinstructors have the potential to make important contributions to student learning and development. However, while they may fully recognize the importance of authentic assessment and evaluation in the realities of practice, it cannot be assumed that disciplinaryspecific expertise transfers to pedagogical expertise in a research-intensive university setting (Hendricson et al., 2007; Scarbecz, Russell, Shreve, Robinson, & Scheid, 2011). Those who frame assessment and evaluation in ways that they were taught and by the quality of their technical work and productivity may resist and fall back on traditional methods (Hendricson  1  et al., 2007: Licari & Chambers, 2007; Wong & Hubball, 2011). There appears to be a lack of awareness, capacity, and/or time for clinical practitioner-instructors to learn and adopt new evidence-based strategies for assessment and evaluation in the context of patient care.  Another barrier is the lack of availability of professional development opportunities for the clinical practitioner-instructor, without which faculty are not well prepared to help students develop as self-directed, critical-thinking, and life-long learners (Licari, 2008). Planning and implementation of an effective faculty development program that is tailored to the unique needs and circumstances of the clinical practitioner-instructor is key to enhanced pedagogical expertise, including authentic assessment and evaluation (Steinert et al, 2006; Hubball & Pearson, 2009).  The purpose of this study is to investigate: 1. What is the nature and scope of the assessment and evaluation practices conducted by clinical practitioner-instructors in Pediatric Dentistry clinical education settings at UBC? 2. What sense do the clinical practitioner-instructors make of the practice that they engage in?  2  2.  Context of Pediatric Dentistry at UBC Faculty of Dentistry Clinical instruction in dental disciplines, such as Pediatric Dentistry, is dependent on  clinical practitioner-instructors who are expert practitioners but are not well versed in research-based instructional strategies. Thus, in the context of assessment and evaluation practices, clinical practitioner-instructors bring strengths from being immersed in the realities of dental practice, but also have limitations in developing dentists ready to enter independent practice.  2.1  UBC predoctoral dental program The Faculty of Dentistry at UBC offers a predoctoral dental program that spans four  years of professional study and leads to the degree of Doctor of Dental Medicine (D.M.D.). The curriculum is built around a framework of dental competencies (UBC Faculty of Dentistry Competencies for the New Practitioner, 2006) and is a hybrid of problem-based learning and conventional lecture/clinic style. The learning community includes approximately 190 students over four years of the program, forty tenured or tenure-track and thirty professional dental educator faculty members, hundreds of sessional instructors, and seventy-five administrative and clinical support staff. The graduating class is made up of up to fifty-five students, a composite of Canadian citizens or permanent residents and graduates from international dental programs who have been admitted to the two-year International Dental Degree Completion Program.  3  2.2  Clinical training in Pediatric Dentistry As part of their final year of study, students in the predoctoral dental program  participate in a community-based learning experience in Pediatric Dentistry. As outlined in the syllabus of the UBC Children’s Dental Program (Appendix B), they learn to apply knowledge and build a repertoire of clinical skills, behaviour management strategies for children, professionalism, and practice organization. Clinical experiences for students take place in the UBC Children’s Dental Program at two sites, the Nobel Biocare Oral Health Centre at UBC in Vancouver and a community dental clinic at Douglas College in New Westminster. The UBC Children’s Dental Program is a collaborative partnership between the university (dental students, faculty, and staff) and the community (Fraser Health and Vancouver Coastal Health Authorities). Community dental staff work with families across Metro Vancouver and refer children to UBC for free dental treatment. Most of the children are from lower socio-economic, immigrant families with English as a second language and high dental needs.  All patient care in the UBC Children’s Dental Program is supervised by a group of twenty-five clinical practitioner-instructors consisting of full-time faculty, sessional instructors, and Pediatric Dentistry graduate students. Most of the clinical practitionerinstructors actively practice dentistry as pediatric specialists or general practitioners in the community and teach at UBC one half day per week. The Pediatric Dentistry graduate students are licensed dentists training to be specialists.  4  2.3  Assessment and evaluation by clinical practitioner-instructors At UBC Pediatric Dentistry the clinical practitioner-instructors bring experience and  expertise as practicing dentists from the community. Provision of patient care allows the student clinician to actively engage in integrating knowledge, skills, and attitudes of professional practice. Clinical practitioner-instructors are encouraged to continually assess students’ progress by providing feedback based on evidence through observation of students’ clinical work and performance. They are also asked to evaluate or judge students’ clinical competency using a holistic approach that consider situational factors. Opportunities to assess and evaluate students include verbal one-to-one feedback, group huddles, and formal written feedback using standards and criteria for patient care.  2.3.1 Verbal one-to-one feedback  There are multiple opportunities during each clinical session for clinical practitionerinstructors to assess and evaluate students. Verbal one-to-one feedback is continuous and ongoing as students care for patients. There are specific checkpoints for each dental procedure that students must seek instructor approval before progressing with next steps of patient treatment. For example, dental treatment cannot proceed until the medical history and treatment plan have been reviewed by the clinical practitioner-instructor. Likewise, a patient cannot be discharged until the final restoration has been checked and confirmed as satisfactory by the clinical practitioner-instructor.  5  As students practice, the experience and expertise of the clinical practitionerinstructor can allow him or her to use cues in complex, multi-faceted patient cases to provide feedback to students in training. For example, a clinical practitioner-instructor might observe that a novice student clinician is having difficulty completing a dental procedure. Verbal feedback to the student might be accompanied by a drawing of the desired cavity preparation to help the student visualize the desired outcome. In another case, the same clinical practitioner-instructor might assess another student who appears confident and is demonstrating fluidity of practice. The pediatric patient appears comfortable, the work station is organized, and the procedure is progressing in a timely fashion. These observations by the clinical practitioner-instructor provide evidence to judge that the student can proceed with independent practice and intervention is not necessary at this time.  2.3.2  Group huddles  End-of-the-day “huddles” are group discussions in which students may reflect on and discuss their cases as well as receive input from their peers and clinical practitionerinstructor. These are intended to be low stakes assessments and evaluations, targeting development of the student practitioner. Discussion topics and feedback methods are dependent upon the clinical practitioner-instructor’s preferences. Generally, the groups meet for fifteen to thirty minutes and discussions can range from technical difficulties of a procedure to a challenging pediatric patient. In the past, there has been resistance on the part of some clinical practitioner-instructors to stay and lead the group in discussion. Over a span  6  of several years and with encouragement from the program coordinator, now all clinical practitioner-instructors appear to be comfortable facilitating the huddles.  2.3.3  Instructor rating sheet and patient care performance rubric  The formal assessment and evaluation in the Pediatric Dentistry clinic sessions revolve around use of an online instructor rating sheet and patient care performance rubric (See Appendix C). The rubric and criteria have been used in the UBC general practice dentistry clinics or “Integrated Clinical Care” for some time. In its pilot year of implementation in Pediatric Dentistry, the 2010-11 academic year, the expectation was for clinical practitioner-instructors to provide both quantitative and qualitative feedback for each student after each clinical session using the online assessment instrument. The instrument is based on four broad performance criteria–professionalism (P), application of knowledge (K), clinical skills (S), and organization (O). Consideration is also given to two other variables– difficulties of the procedure (D) and degree of student independence (I). Students have immediate and full access to their assessments and evaluations.  Overall, clinical practitioner-instructors are encouraged to adopt a collegial and collaborative approach, be consistent in both qualitative feedback and quantitative grading, and follow the framework of Chickering and Gamson’s (1987) seven principles for good practice in undergraduate education. In the competency-based curriculum, the goal is for students to meet expectation levels in a broad range of competencies.  7  Regarding summative evaluation of the Pediatric Dentistry module, cumulative clinical grades make up sixty percent of the final mark while a case-based written examination accounts for the remaining forty percent. To successfully complete the Pediatric Dentistry module, a minimum grade of sixty percent in both daily clinical performance and the final examination is required. Students must also have demonstrated professional behaviour and received a passing grade for professionalism.  2.4  Strengths of clinical practitioner-instructors Clinical practitioner-instructors demonstrate commitment to teaching and learning  with students. The level of knowledge amongst clinical practitioner-instructors is progressing with regard to assessment and evaluation practices. Clinical practitionerinstructors have attempted to provide constructive feedback to students during clinical sessions and facilitate meaningful group discussions at the end of each session. Students have reported that they appreciate examples and tips that clinical practitioner-instructors share from their own practice experience.  In providing formal feedback using the online instrument, the use of clinical performance criteria to assess students has improved with encouragement and reminders from the program coordinator. There were some instances when clinical practitionerinstructors supported quantitative grades with a comprehensive descriptive evaluation of a student’s performance. For example, one wrote:  8  This is a challenging case that presents a number of treatment planning issues–to decide what to do in the best interest of the patient yet also sensitive to the family situation. You demonstrated good communication skills with both child and parent and used your time efficiently to complete an examination and develop the treatment plan. Your patient has confidence in you. Good independent work today.  2.5  Limitations of clinical practitioner-instructors Yet, while a holistic approach to assessment and evaluation might be recognized as an  important component of student learning, authentic assessment and authentic evaluation are not always realized in the Pediatric Dentistry clinical education settings for a variety of reasons. While they are expert practitioners, few clinical practitioner-instructors have a background or formal training as educators and may be unaware of the concepts of authentic assessment and evaluation. There were times when clinical practitioner-instructors defaulted to previously experienced ways of knowing. They assessed and evaluated in ways that they were taught, focusing on technical skills and procedural requirements, and downplayed the significance of other professional attributes. For example, late attendance by clinical practitioner-instructors to clinic sessions was tolerated by students, but for students tardiness would be considered unprofessional. Roth (2007) called attention to the critical need to change the culture and environment of dental education, “Students need to be treated with the respect of professional colleagues from the time they enter dental school and understand the associated obligations of ethical conduct and professional responsibility” (p. 984).  9  As well, clinical practitioner-instructors are continually balancing patient care and student learning. When patient care is prioritized, assessment and evaluation are not necessarily sustained to a level where student learning might be optimized. There are varying degrees of participation by instructors in the assessment and evaluation process; often it is compromised because of a lack of time when patient care is prioritized. There are also a myriad of clinical protocols and technologies to negotiate in the institutional setting. Accessing the assessment and evaluation instrument requires learning.  Unanticipated and common was inflation of grades by clinical practitioner-instructors in the first year of implementation of the assessment instrument. It was common for a clinical practitioner-instructor to grade most of his or her students with “exceeds expectations” across performance criteria for every clinical session. Clinical practitionerinstructors were not supporting exceptional grades with comments. There were other instances when a clinical practitioner-instructor was hesitant to interpret and judge student performance and did not comment or grade appropriately when a student did not meet expectations. The overall result was wide inter-instructor variability in quantitative feedback and inconsistency in quality of formative feedback.  10  3.  Literature Review Review of literature relevant to the research study included examination of the  educational needs of the 21st century dental practitioner and the evolving roles of clinical practitioner-instructors. Research was also focused on learning-centred pedagogy and its application to clinical educational settings. Finally, faculty development programs and strategies for “training the trainer” across multiple disciplines were explored.  3.1  Educational needs of the modern-day dental graduate In response to a fast-changing environment with new developments and technology  and increasing societal demands and expectations, there has been a shift to a competencybased education (CBE) in dentistry that emphasizes higher order learning and integration of disciplines (Chambers, 1999; Licari & Chambers, 2007). The new dental graduate is expected to demonstrate competency in a set of skills, knowledge, and values to begin the practice of dentistry. Not only is the new graduate required to be competent in procedural and technical skills, but also it is important for him or her to master skills such as patientcentred and evidence-based care, oral health prevention and promotion, and management of a general practice.  11  3.1.1 Competency-based dental education  Despite persistent advocacy for educational reform to meet the evolving oral health needs of the public, movement has been slow in this area in the dental field. In a 2009 study of dental school curricula across North America (Haden et al., 2010), participants reported that most dental school faculty members were aware of CBE as the educational model for the Commission on Dental Accreditation and the American Dental Education Association. Paradoxically, in the same study, less than one half of academic and clinic deans and department chairs in American and Canadian dental schools surveyed were able to identify the definition of competency. There was a perception that understanding and valuing of competency-based education was even lower amongst most faculty and students. Research related to effective practices in assessment related to competencies in dental education is emergent, in contrast to greater progress in other health professions (Haden et al., 2010; Hubball & Burt, 2007; Katajavuori, et al., 2009; Licari & Chambers, 2007; Marambe, Athuraliya, Vermunt, & Boshuizen, 2007).  Attempts at educational reform are evident at the Faculty of Dentistry at UBC. “UBC Faculty of Dentistry Competencies for the New Practitioner” (2006) is a document (Appendix A) that outlines six domains of professional activity and responsibilities relevant to the general practice of dentistry—professionalism; practice organization; assessment of the patient and the oral environment; health promotion; establishment and maintenance of a healthy oral environment; and rehabilitation of form, function, and esthetics. Forty-four competency statements outline what the graduating student must know and be able to do and  12  also serve as education standards for the undergraduate curriculum. Competency-based education provides a framework to define core content of curriculum and assesses outcomes.  In the domain of professionalism, the competent new practitioner should prioritize patients’ needs and interests, applying the highest standards to evidence-based practice and communicating effectively. Accountability is to individual patients, society, and the profession within provincial and national legal requirements. The practitioner understands and responds to a dynamic social environment, from which arise ethical issues and problems (related to regulatory actions, economics, social policy, cultural diversity and gender, and health care reform).  In the domain of health promotion, the practitioner should be competent in recognizing determinants of oral health, promoting health, and preventing disease with individuals, families, and community groups, including advocacy for the disadvantaged.  The dental graduate is expected to be able to responsibly organize and manage a general practice with sound skills in administration, business aspects, and management of personnel and patient care. Competencies related to assessment of the patient and the oral environment cover primary oral health care that is comfortable, functional, and esthetically acceptable, and that treats disease. The practitioner must first be able to assess and evaluate a patient, then diagnose existing conditions, and develop a treatment plan. Establishment and maintenance of a healthy oral environment is based on patient assessment. Providing preventive, therapeutic, and continued oral health care involves disease prevention and health maintenance in patients who have good oral health and management of risk factors and control of disease in patients with active oral disease. Where dental disease, congenital  13  deformity, pathosis, or traumatic incidents have compromised the dentition, then the competent dental practitioner will be able to provide treatment for the rehabilitation of form, function, and esthetics.  As the student needs to consistently demonstrate competency across many broad domains relevant to dental practice, it is logical that the desired learning outcomes for competency are assessed and evaluated. This should be one of the most important responsibilities of clinical practitioner-instructors in clinical teaching and learning.  3.2  Role of the clinical practitioner-instructor Because of their experience practicing in the field, clinical practitioner-instructors  have the potential to make important contributions to student learning and development. Bringing together dental practitioners and dental education can result in clinical training that is realistic and relevant (Roth, 2007). However, while clinical practitioner-instructors may understand the importance of instruction in the realities of practice, disciplinary-specific expertise does not necessarily transfer to pedagogical expertise in a research-intensive university setting (Hendricson et al., 2007, Scarbecz, Russell, Shreve, Robinson, & Scheid, 2011). Obstacles persist even as educational reform is being advocated in dental education.  14  3.2.1 Disciplinary and pedagogical expertise  The clinical practitioner-instructor negotiates a major shift from a well-known community of practice as the leader of a team in a dental office to an unfamiliar setting of teaching and learning in higher education. The dentist’s professional knowledge, experience, and wisdom are valued, but the application of that knowledge diverges from the routine. He or she is involved in new roles, performing new tasks, and mastering new understandings and learning in a different sociocultural context or community of practice (Lave & Wenger, 1991; Wenger, 1998). Instead of providing direct patient care, the clinical practitioner-instructor is responsible for supervising, instructing, providing feedback, assessing, and evaluating dental students as they treat patients. They are responsible for protecting the patient from harm. They are expected to facilitate group process, manage conflict, and use information technology, a set of skills which go beyond the base level of discipline-specific expertise (Scarbecz et al., 2011). In addition, there are varying degrees of knowledge, experience, and teaching perspectives of instructors in the process (Hendricson et al., 2007; Hubball, Collins & Pratt, 2005; Licari, 2008; Werb & Matear, 2004).  3.2.2 Educational reform and the clinical practitioner-instructor  While clinical practitioner-instructors grapple with their individual roles in clinical instruction, at the organizational level, a growing need for reform in clinical dental education has been identified (Hendricson et al., 2010; Roth, 2007; Spallek, O’Donnell, & Young, 2010). Roth (2007) put forward a leadership challenge to change the culture of dental  15  education, to prepare students for their professional obligations, and to maintain universitybased research to support dental practice.  Roth advocated for academic and clinical faculty to foster collegial relationships with students in supportive and positive learning environments. This is in contrast to perceptions of clinical practitioner-instructors who likely remember, as students, being taught “some techniques and procedures [that] were outdated or not relevant to contemporary dental practice”, “a mismatch between what was published as curricular requirements and what was really required to survive and succeed”, and “intimidating methods of clinical and preclinical instruction” (p. 984). Incorporating the concept of being a responsible professional throughout the dental curriculum was also emphasized. Teaching and assessment of professionalism were deemed important, given differing lifestyle expectations of the current generation of students, practice demands, changing demographics, and the economic environment.  Roth discussed the role of the clinical practitioner-instructor in developing the ability of students to think critically and problem solve. It has been argued that research must be conducted in university settings and students must be taught to critically appraise research and evaluate new knowledge, developments, and technology to discern future practice that is committed to excellence and safe for patients (Hendricson et al., 2007; Roth, 2007; Werb & Matear, 2004). After a comprehensive literature review, Coormarasamy and Khan reported that, “It is important that teachers of critical literature appraisal and evidence-based medical practice consciously find ways of integrating and incorporating the teaching of critical appraisal into routine clinical practice” (as cited in Hendricson et al., 2007, p. 1522). In spite  16  of this, evidence-based practice is restricted to the classroom more so than in patient care contexts; it is not being translated into the realm of clinical practice of educators in the health professions (Hendricson et al., 2007).  3.3  Learning-centred pedagogy While some teachers may have an intuitive sense for excellence in aspects of  teaching, learning-centred education is supported by educational research and literature (Kreber, 2002). There is potential for improved student learning with understanding of and appreciating the concepts of active learning, learning as an individual, social, and contextual process, differences among students, and provision of critical feedback (Hubball & Poole, 2003). Student learning will be further enhanced by opportunities to apply new pedagogical knowledge with the support of leaders in the field (Caffarella, 2002).  3.3.1 Pedagogical content knowledge  “When expertise in the discipline is effectively combined with knowledge of how to teach, the latter being derived from both educational theory as well as experience, we witness the construction of pedagogical content knowledge (Paulsen, 2001a, b; Shulman, 1987). It is then the construction of pedagogical content knowledge that is characteristic of expert teachers” (Kreber, 2002, p. 15). Kreber distinguished between teaching excellence and teaching expertise based on sources of teaching knowledge and self-regulation of the teacher.  17  The excellent teacher was found to focus on his or her own personal teaching experiences with hard work and reflection guiding future actions. While an excellent teacher repeatedly uses the same repertoire of skills and knowledge, the expert teacher continuously seeks out new strategies or solutions to problem in his or her teaching. He or she has intrinsic interest to develop more sophisticated levels of knowledge and skills to become an even more effective teacher. He or she would refer to pedagogical journals both in his or her own field as well as in the educational field.  Huber (2006) suggested that the first step to inquiry-based learning as a teacher is recognizing an invitation to inquiry when encountering a classroom problem, rather than being ashamed of the problem. Some may cross disciplines in their search for solutions to classroom problems and are often rewarded by discovery of different methods from outside their own discipline. Others may take a step further and become scholars in teaching and learning by sharing and making public knowledge of teaching and learning in their discipline through peer-reviewed publications and presentations (Huber, 2006; Kreber, 2002). These actions can enrich the implicit norms of scholarship within the discipline and alter the normal expectations of disciplinary teaching while articulating new practices.  3.3.2 Principles based on research for good teaching and learning  Chickering and Gamson (1987) offered seven principles for improving teaching and learning in colleges and universities based on research. Their principles are intended as a guide for faculty members, students, and administrators to apply with consideration for the  18  context of learning settings and needs of learning communities. They are broad enough for practice in undergraduate education with diverse groups of learners or to use as criteria for the evaluation of teaching effectiveness. According to Chickering and Gamson, good practice in undergraduate education:  1. Encourages contacts between students and faculty. 2. Develops reciprocity and cooperation among students. 3. Uses active learning techniques. 4. Gives prompt feedback. 5. Emphasizes time on task. 6. Communicates high expectations. 7. Respects diverse talents and ways of learning.  While Chickering and Gamson acknowledge the importance of the content, “what” is being taught, their principles emphasize pedagogy, “how” it is being taught. Regardless of the discipline, a scholarly approach is one that is learning-centred.  3.3.3 A learning-centred approach  A learning-centred approach takes into consideration the learning context and addresses the needs and circumstances of students, respecting that they may be diverse in any number of ways such as prior knowledge, abilities, and learning styles (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010; Barr & Tagg, 1995; Hubball & Poole, 2003). It connects to institutional structures and professional requirements, teaching, and disciplinary content.  19  In other words, learning-centred education is not only centred on the learner, but also on the content/discipline/institution. Moreover, there is a shift from knowledge transmission to teaching students how to learn the subject and development of higher order thinking skills (Hubball & Levy, 2004).  The intentions for learning in a course of study are planned and presented in a learning-centred syllabus (Beaudry & Schaub, 1998) which articulates the instructional goals and learning or student performance objectives, that is, what the learner will be able to know and do. Content is selected and organized around three or four major concepts, clear connections between concepts, and main topics relating to the concepts. Instructional activities, assessment, and evaluation of student performance are intimately linked to learning objectives and content.  3.3.3.1 Active learning  Planning instructional activities to engage students in active learning has been shown to help students “learn deeply” and “make meaning” from their learning (Stefani, 2008). Active learning engages students in being involved in analytical reasoning, critical thinking, decision making, and problem solving (Beaudry & Schaub, 1998; Gavalcová, 2008). It involves students in “doing things” or manipulating ideas and reflecting on how they are applying knowledge. Active teaching for active learning makes use of techniques such as effective questioning.  20  3.3.3.2 Learning as a social process  While learning is an individual process, it is also a social process so another strategy of active learning sets up interactions between learners and learning communities (Gavalcová, 2008; Lave & Wenger, 1991; Wenger, 1998). There is recognition of learners as individuals connected to a diverse and complex community with gender, cultural, ethnic, socio-economic, religious, and political differences.  3.3.3.3 Diversity in learners and teachers  The literature has recognized various approaches to ways of learning. One often-cited example comes from the work of Kolb and Kolb (2005) who described four specific ways of knowing–concrete experience, abstract conceptualization, active experimentation, and reflective observation. Using a tool, the learning styles inventory, one can determine one’s preferred learning style based on experiential learning. Another influential researcher, Gardner (1983) proposed a theory of multiple intelligences and profiled seven distinct learning styles. These were visual-spatial, bodily-kinesthetic, musical, interpersonal, intrapersonal, linguistic, and logical-mathematical. If it is believed that students have different ways of learning, then an educator ought to offer instruction and assess in multiple ways to appeal to different learning styles of students. Similarly, Pratt (2002) argued against a “one size fits all” approach to teaching and evaluation. Five teaching perspectives–transmission, developmental, apprenticeship, nurturing, and social reform–were named. Most teachers hold one or two dominant teaching  21  perspectives. Each teaching perspective embodies the set of beliefs and intentions that guide the actions of teachers holding that particular perspective. For each of the teaching perspectives, Pratt discussed the characteristics of a prototype teacher, teaching and learning strategies, assessment practices, and difficulties. The teaching perspectives inventory has been taken by 100,000 respondents of different disciplines and in more than one hundred countries over ten years (Pratt & Collins, 2000). One of the purposes of the teaching perspectives inventory is to offer a lens through which one can view and reflect upon his or her values and assumptions regarding teaching and learning.  3.3.3.4 Authentic assessment and evaluation  Authentic assessment and evaluation are contextual, interpretative, and performance based (Brualdi, 1998; Gulikers, Bastiaens, & Kirschner, 2004; Svinicki, 2005). The process requires provision of ongoing and continuous critical feedback as the learner applies disciplinary knowledge to negotiate new real-world problems and constructs his or her individual understanding in the context of the situation (Fenwick & Parsons, 2000). Components of an authentic assessment are knowledge construction through integrating a wide range of higher order thinking skills, disciplinary inquiry, and value beyond the classroom (Svinicki, 2005). Multiple opportunities for practice allow the learner to encounter challenges and make adjustments in his or her quest to put all the learning together; reflection and self-assessment are encouraged (Fenwick & Parsons, 2000; Svinicki, 2005). Brualdi (1998) distinguished the roles of informal and formal performance  22  assessments in that students are always aware of formal assessments and these are evaluative in nature.  There are advantages and disadvantages to reality-based assessments (Svinicki, 2005). Transfer of learning is more likely to be an outcome where application of knowledge and skills in context is possible. However, more time and effort by both students and instructor are required. In the complexities of real life learning settings, no two assignments will be the same, so difficulties ensue with reliable and consistent evaluations. Safety and ethical issues must be managed.  3.4  Assessment and evaluation in clinical practice settings Authentic assessment and authentic evaluation involves the provision of meaningful  feedback by expert practitioners in learning environments that replicate real-world challenges, such as clinical practice settings where students are responsible for direct patient care. These two distinct processes are interwoven into one dynamic process. Assessment is the process in which data is collected on student performance. Evaluation is a process that places judgment or value on a given situation (Brualdi, 1998; Huitt, Hummel, & Kaeck, 2001). The evaluative process will yield information regarding the worth or quality of something for which assessment has been made.  23  3.4.1 A link to clinical competencies  In dental practice, learning and development in cognitive, psycho-motor, and affective domains are equally important. How competently was a procedure performed? Was there application of knowledge learned from lectures or problem-based learning cases? How did the student manage the patient, for example, in terms of communication and pain management? Was professionalism maintained in time management, communication, and practice behaviour?  To predict future clinical competence, Val Wass, Van der Vleuten, Shatzer, and Jones (2001) emphasized the linking of assessment in medical education with clinical competencies and assessment standards. Similarly, in dental education, Chambers (1999) studied an evaluative method for competency-based education that required inferences to be made about students’ competency in diagnosis, judgment, patient management, and technical skill. This created a shift in thinking from traditional objective grading on dental procedures and productivity to interpretive assessment using a competency rating system and professional judgment or evaluation of holistic patient management. Awareness of this shift to competency-based education and more authentic assessment and evaluation practices has been inconsistent and change has been met with some resistance. Since competency-based dental education was introduced in 1993, Licari and Chambers (2007) reported that only onethird of the related literature discussed the processes of assessment and evaluation.  24  3.4.2 Helping students to be self-directed learners  Fenwick and Parsons (2000) described a method of authentic assessment and evaluation in which the instructor coaches or mentors the learner in stages through techniques such as demonstration of technical skills and “thinking out loud”. This can be applied to education in dental settings (Gerzina, McLean, and Fairley, 2005) where technical skills and managing patient behaviour are basic components of clinical training. As competence and confidence build, the learner progresses by scaffolding on to what he or she knows or is already able to do. While the ability to self-assess grows, the learner may have questions for the instructor until being able to work competently and independently.  When effectively integrated, formative assessment and evaluation can be used to help students learn to reflect on their experiences and develop the ability to self-critique. A research study by Redwood, Winning, Lekkas, & Townsend (2010) underlined the importance for students to develop critical self-assessment skills. The construct of selfassessment in dental education was defined by this group as the ability to evaluate one’s own abilities, attitudes, and performance against professional criteria and standards of competence. Along the same line, Mattheos, Nattestad, Falk-Nilsson, and Attstrom (2004) stressed the importance of ongoing and constructive self-assessment of one’s professional actions to inform future learning needs.  Evidence-based practice (EBP) integrated into clinical teaching and learning has been shown to stimulate intellectual curiosity, self-directed learning, critical appraisal, and critical thinking skills (Werb & Matear, 2004). EBP counters a historical approach of “what works in my experience”. Findings from a systematic review on evidence-based practice in medical  25  education showed that EBP integrated into clinical teaching (such as bedside teaching, literature-searching assignments based on patient symptoms, and case presentations using appraisal of evidence based on the patient’s presentation) resulted in positive outcomes in practitioner skills, attitudes, behaviour, and ultimately patient care (Coormarasamy & Khan, 2004). The authors concluded that, “It is therefore important that teachers of critical literature appraisal and EBM [evidence-based medicine] consciously find ways of integrating and incorporating teaching of critical appraisal into routine clinical practice” (p.4).  3.4.3 Challenges to authentic methods of assessment and evaluation  “Buy-in” and implementation of competencies in assessment and evaluation has been varied. Licari and Chambers (2007) suggested that faculty members were uncomfortable making professional judgments, preferring traditional methods of objective observation and procedure counts. When required to evaluate and assign grades, grade inflation in clinical performance evaluation has been problematic (Ogden, Edwards, Howell, Via, & Song, 2008). Similar to other professional schools, blurring of boundaries of instructors as mentors and evaluators in dental education is considered a significant factor. McMillan (2011) argued against grading and required quotas of procedures in favour of formative assessment and evaluation. McMillan used a constructivist’s view of learning to propose that good chairside teaching should include active learning and reflection as strategies to help students construct knowledge from their clinical encounters and apply learning to new situations.  26  In a Clinical Education Instructional Quality Questionnaire, 655 dental students were surveyed from twenty-one North American dental schools (Henzi, Davis, Jasinevicius, and Hendricson, 2006). Interaction with clinical instructors was examined as one component of students' clinical experience. While knowledgeable faculty who were eager to help was viewed positively, inconsistent and condescending feedback was problematic. The “data indicate that clinical instructors do not rely heavily on questioning strategies to guide or stimulate student thinking, rarely ask students to reflect on performance or self-assess, and often employ less than ideal strategies for providing feedback.” (p. 374).  3.4.4 Reflection on practice  For the most part, clinical instruction, assessment, and evaluation are practiced in relative isolation with time largely committed to management of students and patients. There is little opportunity for sharing of teaching perspectives and experiences amongst clinical faculty. Feedback on teaching performance and professional development is infrequent (Woolley, Emanuel, & Koshy, 2009). Just as students are encouraged to be develop their professional skills through inquiry, critical appraisal, and reflection, it seems appropriate for clinical practitioner-instructors to adopt the same attitudes and values.  27  3.5  Faculty development Scarbecz et al. (2011) identified the need for faculty development because of the  mismatch between base-level skills, which is discipline-specific expertise, and expectations of teaching faculty. Licari (2007) reasoned that the requisite implementation of faculty development in dental schools: (1) prepare faculty for understanding curricular change; (2) focus on a shift in culture to developing self-directed, critical thinkers; and (3) prepare faculty on how to assess learning. Requiring teaching faculty to participate in faculty development is one way of demonstrating that an institution places value on teaching and learning and is a means of changing institutional culture (Licari, 2008). Steinart et al (2006) argued for increased commitment to educational scholarship and for the design of faculty development to be grounded in educational theory and practice and mindful of the context of the educational environment. A multi-faceted approach, inclusive of experiential learning, formative feedback, peer support, and community building, is recommended to enhance instructional skills of clinical faculty.  3.5.1 Faculty development in health professions education  In an article commissioned by the American Dental Education Association (ADEA) Commission on Change and Innovation in Dental Education, faculty development was named as the key ingredient for curriculum reform in dental education and to change attitudes and behaviours of faculty in teaching and assessing (Hendricson et al., 2007). Most  28  studies relevant to dental faculty development strategies and teaching effectiveness have emerged from medical education.  In 2001, the Best Evidence Medical Education (BEME) Collaboration established a group to review the ‘best evidence’ in faculty development. In their systematic review of faculty development literature pertaining to Medicine, Steinert et al. (2006) concluded that, although it was not shown that student learning had improved, students noticed a change in teaching behaviours. Faculty who participated in programs expressed high satisfaction, positive change in attitudes towards teaching, and increased knowledge and teaching skills. These teaching skills were in areas such as assessing learners’ needs, promoting reflection, and providing feedback.  Davis and Haynes reported on the effectiveness of continuing medical education interventions on professional practice and health care outcomes (as cited in Hendricson et al., 2007). They found that interactive programs with learner-centred, hands-on, and active learning strategies were likely to enhance professional practice in contrast to lecture-based programs where skills learned were rarely found to be applied to patient care. To develop health professionals with critical appraisal and critical thinking skills, Coormarasamy & Khan (2004) emphasized the importance of incorporating research evidence into clinical teaching.  Hendricson et al. (2007) assembled a list of characteristics frequently associated with effective faculty development programs. Considerations in the design of faculty development programs include “use of experiential learning”, “provision of feedback to participants about their performance”, “opportunity to apply skills within the program or  29  soon after”, “use of peers to model exemplary teaching behaviours and share perspectives on teaching”, “programs designed to facilitate peer interaction and the building of collegial relationships”, and “use of a diversity of learning experiences, and opportunities for postprogram assessment of skills” (p. 1529-1530).  3.5.2 A multi-faceted approach  Scarbecz, Shreve, Robinson, and Scheid (2011) recommended a multi-faceted approach to faculty development. Following is an examination of some faculty development programs that have been implemented at UBC for the practitioner-instructor across faculties. An innovative faculty development initiative at UBC is the development of leadership portfolios to enhance scholarly approaches to undergraduate degree program reform (Hubball & Pearson, 2009). Since 1998, in an eight-month Faculty Certificate Program, a cohort of faculty from many different disciplines and academic ranks becomes a learning community focused on the Scholarship of Teaching and Learning (SoTL) and Scholarship of Curriculum Practice (SoCP). The faculty members review relevant literature, participate in action research and self-reflection in their own curricular contexts, develop strategies to enhance teaching and learning and curriculum practices, and disseminate evidence-based findings. Learning activities include the design of a learning-centred syllabus, peer review of teaching, a conference presentation of research, a self-directed learning project, and reflections on teaching and learning and curriculum practices.  30  A similar mixed-mode Faculty Certificate Program on Teaching and Learning in Higher Education was established for teachers seconded from the community to serve as practitioner-instructors in the Faculty of Education at UBC (H. Hubball, personal communication, September 8, 2011). This program was also based on a flexible and iterative framework which integrates learning context, planning, programming, and assessment and involves individual and collaborative learning experiences. It strives to meet the needs and circumstances of practicing school teachers in instructional roles in a teacher education program of a research-intensive university. Like the eight-month Faculty Certificate Program, this route specifically for seconded teachers is a blended faculty development program and includes strategic meetings over a one or two-year period, mentoring, elearning, and inquiry-based pedagogies.  According to the Office for Faculty Development and Educational Support (2011), the Faculty of Medicine at UBC provides support for the clinical teacher at different teaching sites and career stages. There are workshops and educational materials to help clinicians become more effective teachers in clinical settings. Educational primer workshop topics include “Time Saving Tips for Ambulatory Teachers”, “Teaching with Patients-The Art of Questioning”, and “Feedback and Assessment”. Educational resources include a booklet on “Teaching Skills for Community Based Preceptors”.  31  3.5.3 Challenges in planning faculty development  In planning faculty development for clinical practitioner-instructors where participation is voluntary, challenges include engaging clinical practitioner-instructors, justifying and focusing planning to meet desired capabilities in the clinical context, and setting up programs for effective transfer of learning (Caffarella, 2002; Sork, 2000; Subedi, 2004). Planning strategies include use of established communities of practice (Wenger, 1998) and implementation of active learning that is respectful and tailored to the local context.  Caffarella & Zinn (1999) described impeding and enabling factors that influence decisions for participation in professional development. Broadly, factors lie in one of four domains: (1) people and interpersonal relationships, (2) institutional structures, (3) personal considerations and commitments, and (4) intellectual and psychosocial characteristics. An unwillingness to commit time away from their busy practices and family lives would be an example of a barrier. Another obstacle is that a program focused on pedagogy would not be recognized as formal professional development and therefore ineligible for continuing education credit, a requirement for licensure.  Attendance and completion of a faculty development program does not necessarily signify that learning has been effectively applied by program participants. Caffarella (2002) argued that this “transfer of learning” must be planned with timely strategies embedded within the program that involve the program planner, the facilitators or instructors, and the learners. Barriers to transfer of learning include lack of participant “buy-in”, a perception that content is not usable, or a training environment that does not translate to the real world. A focus on relevant and practical content might enhance transfer of learning. Participants  32  might engage in active learning using cases or scenarios matched to real-world and local contexts (Caffarella, 2002; Graham et al., 2006). Prior knowledge and experience would be recognized and respected in a collaborative learning environment.  When the program planner or facilitator has close connections with participants in the workplace setting, there may be occasions to follow up with program participants. This would allow participants to apply new knowledge and skills and integrate their learning over a period of time with assistance. Opportunities might be created to allow sharing and feedback within the community of practice (Wenger, 1998). According to Wenger’s social theory of learning, learning will occur in a community of practice as members mutually engage in pursuit of and production of meaning and develop a shared repertoire of resources for learning. This potential was corroborated by the Best Evidence Medical Education Collaboration; one of their conclusions was the importance of peer and collegial relationships in effective faculty development (Steinart et al, 2006).  33  4.  Methodology A qualitative case study was conducted to address the research questions. Following  methods of Kennedy, Regehr, Baker, & Lingard, (2009), data was collected from interviews, observation, and review of documentation to inform the scope and nature of assessment and evaluation practices in the clinical settings of Pediatric Dentistry at UBC. Interview data also provided a sense of how clinical practitioner-instructors understand their practice (Table 1). Approval was obtained for the research study from UBC Behavioural Research Ethics Board.  Table 1. Research Methodology Research Questions  Data Collection  1. What is the nature and scope of the  Interview clinical practitioner-instructors  assessment and evaluation practices Observe assessment/evaluation practices by conducted by clinical practitionerinstructors in Pediatric Dentistry clinical education settings at UBC?  clinical practitioner-instructors o  With individual students  o  In group huddles  Review documentation of formal online assessment/evaluation  2. What sense do the clinical practitioner-  Interview clinical practitioner-instructors  instructors make of the practice that they engage in?  34  4.1  Data collection A qualitative study was conducted in UBC Pediatric Dentistry clinics at the Nobel  Biocare Oral Health Centre. The qualitative design using case studies was chosen because qualitative methods focus on understanding complexity and participants’ perceptions, experiences, and actions (Corbin & Strauss, 2008; Gall, Gall, & Borg, 2010).  The researcher, as part of her responsibilities as Clinical Assistant Professor at the Faculty of Dentistry, is the Program Coordinator of the UBC Children’s Dental Program at the Nobel Biocare Oral Health Centre. She conducted the interviews, observed clinical practitioner-instructors at work, and examined the formal assessment and evaluation documents. The researcher has experience as a member of the portfolio assessment team for the SoTL Leadership Program and serves on the UBC Faculty Instructional Skills Workshop Facilitator Team.  Clinical practitioner-instructors were invited by letter to participate in the study. Voluntary participants provided primary sources of data; no compensation was provided. There was purposeful sampling and representation from full-time faculty, practicing pediatric dentists, general dentists, and graduate pediatric dental students. Data were collected over a two-month period in the first term of the final year of the predoctoral dental curriculum.  35  4.1.1 Interviews  First, interview data were collected in private, individual semi-structured interviews after written consent was obtained (Appendix D). Due to scheduling conflicts and with their agreement, the two graduate students were interviewed together. Each interview began with the researcher asking the participant to view a set of six images (Figure 1) and select one or two which resonated with the participant’s perception of clinical teaching and learning. The animated images with captions were used as metaphors for conceptions of learning and intended to provoke reflection.  Clinical practitioner-instructors were then prompted to discuss the nature and scope of their assessment and evaluation practices and reflect on the meaning of their practice. A set of guiding questions (Appendix E) was used in the interviews related to verbal one-to-one feedback during clinic sessions, group wrap-up meetings or “huddles”, and written assessments and evaluations documented using the online instrument and patient care performance rubric. Thirty minutes was allotted for each interview. All interviews were audio-recorded with the consent of participants and later transcribed. Demographic data were obtained from participants regarding academic rank, number of years in clinical teaching, previous training in teaching, age, and gender.  4.1.2 Observation during patient care and in group huddles  Second, data were collected as the researcher observed interactions between participating clinical practitioner-instructors and students during clinical sessions.  36  Observation was intended to supplement and corroborate data collected from the interview process, but in reality, observation was on a regular and continuous basis, as this was also part of the role and responsibility of the program coordinator. Further data were collected at the end of the clinic session when the clinical practitioner-instructor met with his or her group of students in group huddles. The researcher participated in the discussion, mainly but not exclusively, as an observer and note-taker.  4.1.3 Review of formal documented assessments and evaluations  Third, the researcher reviewed the formal online documentation of assessment and evaluation by clinical practitioner-instructors. Consistent, norm-referenced, and, to the best extent possible, accurate ratings were checked by comparing quantitative grades and qualitative feedback. Accuracy was difficult to confirm because the nature of evaluation required interpretive judgment to be exercised by individual clinical practitioner-instructors. Since the researcher was also involved with program coordination, she had to intervene on a number of occasions during the period of data collection and provide individual feedback to clinical practitioner-instructors to elicit more thorough and consistent assessment/evaluation of student performance.  37  Figure 1. Conception of learning: Images as metaphors for interview discussions Centre for Studies in Science and Mathematics Education, School of Education. University of Leeds, by permission of Professor Jim Ryder, Leeds University  38  4.2  Data analysis Hubball & Burt’s (2004) flexible conceptual framework (Figure 2) for developing,  implementing, and evaluating learning-centred curricula in higher education contexts was used to organize and analyze data. The framework takes into account complex curriculum contexts and the iterative processes of planning, assessment, programming and evaluation in order to examine the resultant practices of curriculum reform.  Figure 2. Developing, implementing, and evaluating learning-centred curricula Hubball, H. & Burt, H. © International Journal for Academic Development, 2004, by permission  Using the framework, data were clustered into four themes and interpreted, based on grounded theory principles (Corbin & Strauss, 2008: Gall, Gall, & Borg, 2010). The themes were: 1.  Attention to the pediatric dental clinical context for assessment/evaluation  2.  Planning strategies, related to the desired learning outcomes and dental competencies, for assessment/evaluation  3.  Programming and learning strategies, related to student learning experiences in patient care, for assessment/evaluation  39  4.  Application of criteria and standards for assessment/evaluation  Analysis proceeded as follows. After the interviews were transcribed, the transcriptions were read multiple times to look for patterns and compared and contrasted with each other. They were critically examined to check the fit of the data to the themes. The researcher then inserted each meaningful piece of data (phrase, sentence, or passage) into four tables categorized by the themes, then streamlined and merged the data from which to conduct a group-level analysis. Data then were interpreted using the literature to determine how well or not they represented authentic forms of assessment and evaluation. Findings from interview data were triangulated using observational data of instructors in practice with students in the clinical setting and documentation from written assessments and evaluations of students’ performance.  Through the themes and the patterns that emerged as a result of the data analyses, the researcher drew conclusions on the nature and scope of assessment and evaluation practices by clinical practitioner-instructors in Pediatric Dentistry at UBC and the sense that clinical practitioner-instructors make of the practice in which they engage.  40  5.  Results The data collected from interviews, observation, and review of documentation  informed the scope and nature of assessment and evaluation practices in the clinical settings of Pediatric Dentistry at UBC. Interview data provided reflections of how clinical practitioner-instructors understand their practice and react to interaction with their students as individuals and in groups. The data represented multiple and sometimes opposing perspectives; some can be viewed on a continuum of beliefs, intentions, and actions while others are complex and intertwined.  5.1  Study participants Thirteen clinical practitioner-instructors were interviewed by the researcher over a  two-month period. The same thirteen were also observed interacting with dental students in clinical educational settings at the Nobel Biocare Health Centre, UBC. They represent half of the clinical practitioner-instructors involved in the 2011-12 UBC Children’s Dentistry Program. Eight clinical practitioner-instructors were responsible for supervising and instructing their own group of five to eight students in weekly clinic sessions. Two others shared responsibility for one group of students with another two clinical practitionerinstructors (not participants of the study). So, on average each of these two clinical practitioner-instructors attended UBC only once a month. Two graduate students in the combined MSc and Diploma Program in Pediatric Dentistry were not assigned a group but served as “floating” instructors and helped out where needed; one graduate student was  41  present every clinic session. One clinical practitioner-instructor, a certified dental assistant, was also interviewed to provide a different perspective. The role of this clinical practitionerinstructor was supervision and instruction of students in dental oral radiography.  The majority of study participants were pediatric dentists. Eight out of thirteen are active primarily in private dental practice in the community and instruct at UBC on a parttime or sessional basis (Table 2). Three are full-time faculty members. Two are graduate students, qualified dentists in training to be pediatric dental specialists. Most participants were between thirty-five and forty-nine years old with three participants younger than thirtyfive and two older than forty-nine. There was a balance of male (six) and female (seven) representation. The number of years of experience in clinical teaching for all clinical practitioner-instructors ranged from two to fifteen years, with an average of eight years. Only one clinical practitioner-instructor had any formal training in education.  Although thirty minutes was the anticipated duration of each interview, interview participants seemed to enjoy discussing their practice and shared their perspectives for up to forty-five minutes with little intervention by the interviewer.  42  Table 2. Participants  Private practitioners  Full-time faculty  Graduate students  5.2  Pediatric dentist  7  General dentist  1  Pediatric dentist  2  Certified dental assistant  1  In-training pediatric dentists  2  Data collected from interviews At the beginning of each interview, study participants were shown the animated  images with captions (Figure 1) and asked which one or two metaphors matched their conception of teaching and learning in clinical settings. There were three main metaphors chosen, with approximately one-third favouring either the clay being molded by the potter, the traveler led by the guide across unknown terrain, or the child throwing a stone into a pond and watching the ripples spread. In the first, clinical practitioner-instructors compared the student to an amorphous lump of clay that could be molded into the end result of a dentist ready to practice good dentistry by graduation. One explained her responsibility to “give a certain recommended shape to this pot because you have to know that when you’re shaping pottery, things can happen and you can get it spoiled, so you have to give care to put the right  43  amount of pressure in guiding the shape…direction and prompting, where to focus the knowledge and where to draw it in.”  Those study participants who identified with the guide and the traveler metaphor described a facilitation role with “the guide helping the traveler, telling how to go, which way is the safest way to go”, “without being too intrusive, without over directing”, and “as the student becomes more independent and less reliant on the instructor, the view for the student changes…they don’t need the guide anymore”. One of these study participants vehemently disagreed with the notion of teacher as molder, stating that this analogy hearkened back to past history of master-apprentice dental education and the field of dentistry now is too broad for anyone to consider oneself a master.  Participants who selected the image of a child casting a stone in a pond and the resultant outward rippling made the comparison of “an action followed by reaction and seeing the results of those actions”. One illustrated with an example, “If there’s a challenging child, [I] give them heads-up on techniques for management and let them try it out and try out different methods and see how it goes. I used to be a little more handson…now I feel that the more they experiment on their own before I intervene, the more they get out of the experience…[I] let them know that I’m there if they need my help.”  Synopses of all interview data are outlined in Tables 3 through 6 and categorized by the four themes drawn from the conceptual framework of Hubball and Burt (2004):  44  1. Attention to the dental clinical context for assessment/evaluation 2. Planning strategies, related to the desired learning outcomes, for assessment/evaluation 3. Programming strategies, related to student learning experiences in patient care, for assessment/evaluation 4. Application of criteria and standards for assessment/evaluation  5.2.1 The context of clinical dentistry for assessment/evaluation With respect to context (Table 3), interview questions were posed to explore the extent to which authentic assessment and evaluation practices are conducted. Responses span a wide spectrum with beliefs for authentic practice often not enacted in the instructional settings. Most clinical practitioner-instructors were unaware of the competency-based dental curriculum, but agreed that it is desirable for dental graduates to possess professional attributes. A limitation of clinic time was perceived as an obstacle to instruct beyond clinical and technical skills, as was a difficulty to accurately measure values and attitudes.  45  Table 3. The Context of Clinical Dentistry for Assessment/Evaluation What is your understanding of competencies for the new practitioner? (From UBC Faculty of Dentistry Competencies for the New Practitioner, 2006: Professionalism; practice management; assessment of patient and oral environment; health promotion; establishment and maintenance of a healthy oral environment; rehabilitation of form, function, and esthetics) • Unaware of competency-based dental education at UBC • Focus in school is more on technical and clinical competence; other competencies are overlooked • Limited clinic time is a big hurdle to accomplish this shift to competency-based education • Desirable to have clinicians who are respectful and caring, culturally competent, and effective communicators and who know when to refer • Important for the dental school to instill a sense of professionalism in their students. Dentistry needs to work to regain public trust because of compromised professionalism in some practices • Sometimes it is even hard to hire people in your office based on attitude; values are hard to measure • The reality of practicing in the real world is not taught well • Practice management takes time and work experience to develop • I think my role involves getting students prepared to face practicing in the real world • It is by luck that the student graduates with the necessary skills What do you bring from your experience as a dental practitioner? • Examples from my practice, such as patient behaviour management strategies, the importance of profound local anesthesia, parent management, gaining the trust and rapport of a child patient • Practical and financial things, like dental insurance, limiting excessive use of sundries, finding a practice to associate in, dealing with patients who want a discount • How to handle mistakes, communication skills to prevent a malpractice suit • How working with an experienced assistant impacts delivery of care and speed • Ergonomics • Tips that I use in my practice that are not described in the textbook  46  5.2.2 Planning for assessment/evaluation  Planning strategies (Table 4) refer to how clinical practitioner-instructors prepare for assessment and evaluation of student performance in clinical settings. Global and specific learning objectives were outlined in the module syllabus (Figure 3). In an orientation session for clinical practitioner-instructors in Pediatric Dentistry clinics, the program coordinator delineated expectations for assessment and evaluation. Additionally, two lunch hour educational enrichment seminars on assessment and a focus on feedback were recommended. When clinical practitioner-instructors did not attend these seminars, the program coordinator facilitated viewing of a podcast of one of these two seminars during clinical sessions.  Past experience as instructors and learners, as well as feedback from students, influenced the ways in which some clinical practitioner-instructors planned for assessment and evaluation. Clinical practitioner-instructors were largely unaware of students’ perceptions of their instructional effectiveness. For example, they provided feedback on students’ progress but did not follow up with students to see if feedback had been read or if it was valued by students for learning. Only one clinical practitioner-instructor spoke of providing the appropriate level of challenge to students, accessing the zone of proximal development.  While the podcast seminar on assessment in UBC Dentistry clinics was identified as helpful, it was difficult for clinical practitioner-instructors to pinpoint areas where they felt a need to develop their instructional skills. Many stated that busyness of their private practices made it difficult to conceive of making a time commitment for faculty development.  47  Figure 3. Global and enabling learning objectives UBC Children’s Dental Program Syllabus, 2011, by permission By the end of DENT 440 Pediatric Dentistry, the student will be able to: 1. Complete a comprehensive oral assessment of a child patient a. Formulate specific questions and address issues regarding the chief concern, history of present illness, past medical and dental history, family and social history b. Assess a child’s extra-oral and intra-oral structures and differentiate between normal and abnormal tissues c. Perform a radiographic examination and interpret the findings d. Recognize signs of abuse and/or neglect and make appropriate reports 2. Organize a treatment plan that will fulfill a child’s behavioural, preventive, restorative, and interceptive orthodontic needs a. Analyze and interpret data and findings b. Develop a problem list (diagnoses), a list of treatment options, and a comprehensive, prioritized, and sequenced treatment plan c. Present the case and treatment outline to parents or caregiver and child, if appropriate 3. Develop evidence-based approaches to the management of caries in the pediatric patient a. Assess caries risk b. Recommend appropriate non-surgical and surgical management of caries c. Counsel patients and parents about controlling tooth decay 4. Assess the pediatric patient and use appropriate behaviour management and effective communication strategies to make the dental experience positive for children 5. Manage the dental treatment needs and complete comprehensive dental treatment for child patients a. Administer profound local anesthesia safely for dental procedures and manage related complications b. Perform pediatric dental procedures competently, including at least one each of the following: multi-surface amalgam restoration, formocresol pulpotomy, stainless steel crown on a primary molar, and extraction c. Explain how to assess and manage dental trauma d. Self-assess accurately e. Recognize limitations and make an appropriate referral for specialist care, e.g., pediatric dentist, orthodontist, oral surgeon 6. Demonstrate professionalism and ethical practice in patient care clinics and small group participation a. Communicate effectively with patients, parents or guardians, clinical instructors, staff, other health professionals, and peers b. Demonstrate preparation for and apply knowledge to clinic practice c. Independently access, retrieve, and critically evaluate relevant information d. Organize clinical work and work area e. Practice standard infection control precautions f. Keep complete and accurate records g. Work effectively and independently in a timely fashion  48  Table 4. Planning for Assessment/Evaluation What informs the way you assess and evaluate students in the clinical setting? • • • • • • • • • • • • • •  Don’t know The articles you [program coordinator] have given us and things you have told us Through what I think is appropriate, what I have liked and not liked when I have been taught I would lay out my expectations in the first session I prepare students one week ahead if we know what they will be doing next appointment After watching the podcast on your laptop, I gained insight into how this evaluation system works I haven’t looked at the second podcast about giving descriptive feedback, but it is important. Descriptive feedback is important; you want students to be able to develop their own independent evaluative ability See over time if there is progress with the group and individually Feedback from students I try to be very fair I don’t get feedback from students, but I know they have access to my written feedback and I hope that students are learning from it I do not ask them if they read their comments Reflection  What faculty development do you need to support you as clinical practitioner-instructor? • Part of the problem is the time commitment • Focused pointers or discussion of case scenarios in 15-30 minutes before clinical sessions • How to transfer knowledge to students in a way they can understand • Have someone monitor how I teach in real time and give feedback • Learn how to give effective, consistent feedback and to critically assess • I don’t know what tools would be useful • Bring instructors together to talk about teaching • Electronic portfolios – showcase student’s work, faculty can enter formative feedback in a descriptive, not evaluative way  49  5.2.3 Programming strategies for assessment/evaluation  Clinical practitioner-instructors spent most of the interview time talking about programming strategies to improve student performance in clinical care of pediatric patients (Table 5). Cues were discerned to assess students by observing students at work, listening in on student-patient-parent interactions, and noting patient responses, organization, and time management. Clinical practitioner-instructors unanimously agreed that intervention was necessary if there was risk of serious harm to the patient. However, there appeared to be variable tolerance levels for non-intervention in everyday practice, from some taking a watch-and-monitor approach to others feeling more comfortable with a hands-on approach.  Regular and ongoing individual feedback is the norm in clinical care settings in Pediatric Dentistry. There were many modes of feedback provision described–telling, directing, prompting, questioning, coaching, and drawing diagrams to illustrate a concept or procedure. Demonstrations were universally described as an effective method of instruction, both for performing dental procedures and modeling behaviour management techniques with child patients.  When asked about students having difficulty, discussion revolved around student confidence issues and difficulty with understanding concepts or performing psychomotor skills. Supporting the unconfident student with encouragement and more opportunities for practice were strategies used. Gains in confidence appeared to be linked to gains in competence. Some clinical practitioner-instructors expressed frustration at students’ lack of understanding of concepts or skills despite numerous attempts to help, intervene, and  50  demonstrate. Others found problematic the inability of some students to link theory to practice and would guide them with questions or advise them to go home and review.  The clinical practitioner-instructor routinely met with his or her group of students at the end of the clinic session in a “wrap-up huddle”. Their patterns of meeting varied from being instructor-led with pre-determined topics to student-driven with sharing of the clinical experiences of the day. It was common for the discussion to focus on diagnosis and treatment planning of unusual or challenging cases. A few clinical practitioner-instructors expressed the value of peer teaching and learning.  51  Table 5. Programming Strategies for Assessment/Evaluation What strategies have you used in assessing students’ clinical performance? • Teach them in the way I learn best or the way I think is the successful way to learn • Check on each student constantly • Make myself available to get instruments and supplies for students • Look at the patient to see how the patient is handling the procedure • Watch the student’s progress, time-wise and efficiency-wise • See if the student is organized and things are going smoothly or if the student is jumping here and there • More handholding and guidance at the beginning of the term, later less hovering • Allow students to think their way through; if they run into trouble, they’ll ask for help • I use the acronym “KAPIT” to see if they know what they’re doing, anticipate where they’re going, prepare for the next step, initiate action, terminate, that is, know when to start and stop • Push them to achieve goals and get procedures done, my expectations are high • Wait for positive change to happen • Listening to students talk to patients and parents • Follow up on suggestions I made in the past to see if there has been improvement • Check the results • Probe with questions • Ask questions for reflection • I don’t often ask for students to self-assess their work How do you provide feedback to students engaged in patient care? • Tell, direct, and prompt to focus knowledge • Direct them how to correct something • Tell them to read up on the subject • Use the sandwich technique or balance feedback • Give feedback right at the time things are happening • “Zone of proximal development” • Ongoing encouragement on progress • Coach, for example, “talk to the child, tell him what he might be feeling” • Break a procedure down into small parts • Demonstrate and model • Draw • Recommend videotapes that relate to procedures • Intervene if I see threat of major harm; I want to protect the patient from the student • Intervene if I see the patient is out of control, not cooperating or listening to the student • Intervene if time is running out • Sometimes I want to go in and help, but now I tend to take a step back and watch them • I tend not to intervene but I will be asking questions about what the student is doing • Model professional behaviour, for example, follow infection control protocol  52  Table 5. Programming Strategies for Assessment/Evaluation What difficulties have you encountered? What strategies have you used to manage challenges? • Students lacking confidence o Encourage and be supportive o Prompt them with questions o Build more opportunities for practice o Be more hands-on to guide students o Observe and sit with student as he/she goes through a procedure o Demonstrate o Have “floater” instructor spend 1:1 time with student • Overconfident students o Have a conversation with the student and express my concern…it seemed very hard for the student to understand my point of view • Students having difficulty understanding concepts/performing a clinical procedure o Work with student to find out where the block is o Have students describe in words, then demonstrate skill o Bring student in for a private discussion o Direct student to read more o Probe understanding with questions o Draw pictures o It is frustrating…I have to find another way to do things What takes place in the group meetings “huddles” at the end of the clinic session? • Students describe to each other what they did, students teaching other students, I come in with a few words about what to consider and why something is important • Discuss unusual, challenging, or interesting cases, treatment planning, diagnoses, and treatments • I let the students manage it; they talk about their experiences or bring questions • I pick a topic for each student based on their case of the day • I make a list of things I want to talk about  5.2.4 Criteria and standards for assessment/evaluation  Interpretation of UBC Dentistry’s patient care performance rubric was varied and irregular with some clinical practitioner-instructors deriving their own meanings and others diligent about applying standards as defined in the rubric. Even those who were faithful to  53  the criteria as described had difficulty deciding the appropriate level of performance at times, for example, differentiating between when performance met or exceeded expectations. Some observed that the criteria and standards provided a launching point from which students might begin to self-assess and self-evaluate. One clinical practitioner-instructor talked of the importance of providing descriptive, not evaluative, feedback which would allow students to develop capacity for independent evaluation. Generally, comments were deemed important for student development as clinicians. There was some reluctance to give a grade of 1 or “does not meet expectations” for fear of jeopardizing the instructor-student relationship.  54  Table 6. Criteria and Standards for Assessment/Evaluation What kind of assessment/evaluation do you think is important for student learning? • Comments are more beneficial than grades/numbers • I make specific comments. Hopefully hearing it in the clinic and then reading it is useful. • In terms of developing clinicians, descriptive feedback is important. In practice, who is going to grade the clinician? It is important to know about self-directed learning, constant selfevaluation of your own work, and reflection. • It is difficult for me to be evaluative. I know it is part of my role to point out if someone is not meeting the standard. • Students need to be able to self-evaluate. Instead some seem to want me to say what I think. What do you think about the criteria outlined in the patient care performance rubric? Professionalism • Interpretation is very broad, everyone has a different interpretation • How can it be actually assessed if someone has developed professionalism? • Most students are very professional, so do I give them a grade of 4 (exceeds expectations) or should I expect that? • It is hard to have someone understand why they did not meet expectations in professionalism Application of Knowledge • I think that Clinical Skills is the application of knowledge • I ask students to tell me what they plan to do before they start, so I know that they are on the right track Clinical Skills • Straightforward Organization, Time Management, and Infection Control • No comments Degree of Difficulty • Difficult to determine, for example, is an anxious child patient considered difficult? • Some change needed because the student’s grade goes down if they are assigned a patient and/or procedure that is routine Degree of Student Independence • Difficult to determine because the students are required to show us their work at certain checkpoints • If I provide suggestions, is that minimal intervention? No intervention?  55  Table 6. Criteria and Standards for Assessment/Evaluation What feedback do you have regarding use of the patient care performance criteria? • This rubric is great. It could be applied in my office. • This is my first time seeing this rubric of performance criteria…it’s nice to have this structure • Naturally every instructor would assess students according to how he or she would do it and to the level that he or she would expect. The end result (the restoration or the child’s experience) is the most important. • I hardly ever do clinical assessments but I know I should • Still learning how to evaluate, especially on the computer • Descriptions of the criteria do not show up when doing the assessments/evaluations on the computer • We need to become accustomed to what each criteria really defines • If students do not know the criteria, then it is difficult for them to self-assess their competency • The way the grades are calculated, it’s easy to fail a student or give them a very high mark (multipliers exaggerate the bell curve) • It would have to be really bad in order for me to give a grade of 1 (did not meet expectations) • Once when I assigned a grade of 1 with specific descriptive comments, the student disagreed. That makes me really hesitant give those kind of grades…I want to avoid confrontation • Not sure if we should grade at the student level, the GP level, or the specialist level • Sometimes I need to pay more attention to ensure my grades and comments are consistent • Although I always talk to students, I’m not comfortable writing comments • A subjective assessment out of 10 would be neat to do at the end of the term/year  5.3  Data collected through observation Clinical practitioner-instructors are “on the go” constantly from the start to the end of  every clinical session in Pediatric Dentistry. Many challenging situations arise, from managing behaviour of children, communicating and gaining trust of parents, and supervising dental treatment in the hands of novices. At the same time, there is a responsibility to instruct, assess, and evaluate students in this clinical realm.  56  5.3.1 The dental clinic setting  The dental clinic is a large open space with partial partitions between dental operatories or cubicles. With thirty-two and twenty-two student-patient pairs on Tuesday and Wednesday afternoons, respectively, the energy and noise level is high. Frequently, parents accompany the child into the clinic, adding an additional dimension. Crying children are not uncommon in the general clinic area. There are enclosed operatories or ‘quiet rooms’ for cases that are very difficult and some of these patients are treated with the adjunct administration of nitrous oxide sedation. If a student had booked a quiet room, the clinical practitioner-instructor would have to physically go back and forth from their main group of students to the enclosed operatory located on the far side of a big clinic floor space. To alleviate this problem, the floater instructor was assigned the responsibility of supervising students in the quiet room or rooms.  5.3.2 Ongoing feedback and assessment for the individual student  In the main clinic area, it was common to see the clinical practitioner-instructor sit down in the operator’s chair and examine a child patient. He or she would model using child-friendly language, a soft voice, encouraging and praising the cooperative child. In one instance of treatment planning, the student looked over the clinical practitioner-instructor’s shoulder, leaned in and adjusted the mirror handle held by the clinical practitioner-instructor to better visualize the tooth. The student was then asked, “What would you do?” After the  57  student responded, the clinical practitioner-instructor said, “If this were my child, I would probably plan ‘x’. This is my rationale…”  Feedback and assessment in some situations took place with the student as operator and the clinical practitioner-instructor as assistant. Sitting next to the patient, a clinical practitioner-instructor was able to observe and coach the student. While the clinical practitioner-instructor held the child’s hand (both for reassurance and as a safety-restraint mechanism) and the parent looking on, the clinical practitioner-instructor watched, guided, and assessed the student as the student extracted a tooth.  There was frequently a waiting list for checks by the clinical practitioner-instructor generated by students on a paper towel. Paper toweling has been referred to as “UBC Dentistry stationary”, for it is also used routinely to illustrate how something should look. When a student would ask the clinical practitioner-instructor to “have a look” at a required checkpoint, feedback typically would be given to the student immediately in the form of one or two recommendations for improvement, a drawing, or approval to proceed to the next step. After a couple of attempts, if the student had been unable to achieve what was necessary, the clinical practitioner-instructor then might demonstrate it. Although demonstration was an instructional strategy, crisis intervention by a clinical practitionerinstructor was unplanned and dictated by the moment, for reasons of safety, time, or patient non-compliance, with often less-than-optimal resultant learning.  For those clinical practitioner-instructors who took a less hands-on approach, there were times when they would not be taking an active role in student supervision. It should be noted that, as students gained experience, several clinical practitioner-instructors gravitated  58  toward this strategy. They would stand back, watching and listening. Sometimes they would intervene, for example, to make a suggestion. More often, they would wait for the student to approach them for assistance or approval to proceed to the next step.  At least two clinical practitioner-instructors provided auxiliary support in the form of getting instruments and supplies from the central dispensary for students and clean-up. This enabled students to spend more time on direct patient care. One clinical practitionerinstructor brought supplemental instruments and supplies from his own office for his students’ use and a camera to take photographs of their work. At one point, the researcher witnessed a clinical practitioner-instructor helping a student by cleaning up a child’s vomit from the floor.  Observational data confirmed interview data that much of the assessment and evaluation practices were targeted to build strategies of communication with pediatric patients in order to gain compliance. Child-friendly language and metaphors (“sugar bugs”, “it’s like trying on shoes”, “the mouth prop is an elevator”), distraction, positive reinforcement, voice control, and “tell-show-do” were used every day in teaching and learning in clinic sessions by the clinical practitioner-instructors. How to manage a child’s behaviour in the dental setting was also one of the main topics for group discussions and formal assessment and evaluation at the end of the day.  59  5.3.3 Group huddles  The group discussions or “huddles” were routinely conducted after patients and families had been dismissed from the clinic; the average huddles lasted fifteen to thirty minutes. Round table discussion formats were observed in the majority of groups with short case presentations by students followed by comments, suggestions or questions from other students. The clinical-instructor would weigh in with an opinion or insight at the end. Some huddles seemed to engage students whereas, in others, students seemed tired and reluctant to participate. One huddle was particularly lively with a sharing of cake gifted by a patient’s parent to a student. Most of this group’s discussion was devoted to problem-solving one student’s difficult case. Other students were later asked to provide a quick summary of their cases.  At least two clinical practitioner-instructors used questioning to probe for understanding and to encourage participation. Appropriate and timely questioning can allow critical examination of a specific problem from multiple perspectives and exploration of solutions (Fenwick & Parsons, 2000). In a group huddle, one clinical practitioner-instructor built students’ skills in critical thinking and problem solving through series of reflective questions. He probed until students had exhausted answers, then provided his input. One such line of inquiry went like this: Why? Why does this happen? Meaning…? Why did you have difficulty? What were your choices?  60  What can we do next time?  In a different approach, other group huddles were directed by the clinical practitionerinstructor who used questions about specific cases to prompt students. It was the researcher’s observation that, in general, these groups of students appear more restrained and some students did not participate unless called upon specifically. With this style of meeting, there was more dissemination of expert information and discussion of unusual cases.  In all huddles, practical tips were offered in the context of individual patient care needs, such as how to give a nerve block (dental anesthesia), use non-verbal cures for adequate anesthesia, and punch a “mickey-mouse” rubber dam. One clinical practitionerinstructor used a ‘decision-tree’ method of analyzing and problem-solving for her students. The approach was cascade-like, “If this happens, then do this; if that happens, do that.”  Of the huddles that the researcher participated in, a small minority of clinical practitioner-instructors addressed broader issues of the practice of dentistry. Topics related to students finding a practice in which to associate after graduation, similarities and differences of dental practice in international students’ countries of origin, and how to manage clinical mistakes.  5.4  Data collected from documented assessments and evaluations The online instrument for daily assessment and evaluation of student performance  made use of the patient care performance rubric based on four broad performance criteria–  61  professionalism (P), application of knowledge (K), clinical skills (S), and organization (O). Grades of 1 to 4 for each criterion symbolized standards, respectively, of ‘does not meet expectations’, ‘borderline’, ‘meets expectations’, and ‘exceeds expectations’. Consideration was also given to two other variables: difficulty of the procedure (D) and degree of student independence (I). There was a free text box in which comments could be entered; no limit of the number of characters or words was imposed.  5.4.1 Daily and global periodic assessments  At the end of Term 1, an interim grade was calculated for students based on cumulative grades plus a “global periodic assessment”. The global periodic assessment had the same grade weighting as a single session so it was not a high stakes assessment. Guidance was given to clinical practitioner-instructors that a typical comment might indicate “progressing as expected’. It was emphasized that this was also an opportunity to provide a personal perspective on how a student’s clinical ability was progressing, highlighting strengths as well as suggestions for improvement.  With reminders and support from the program coordinator, most clinical practitionerinstructors entered some sort of assessment and evaluation daily. However, despite explicit written instructions, many clinical practitioner-instructors could not negotiate the technology to complete the global periodic assessment. Even after an easier alternative was offered for submission, less than half of the students ultimately were provided a global periodic assessment at the end of the term. Using whatever grades that were submitted for the term,  62  the mean grade for each group of students were calculated and ranged from 66% to 87%. Conversion factors were applied to align each instructor’s group with the overall class average of 80%.  5.4.2 Quantitative grades and qualitative comments  In reviewing the daily assessment and evaluation data, the numerical grades assigned were mostly 3s, indicative that expectations were met. Many 4s (exceeded expectations) were assigned, often by the same clinical practitioner-instructors, a few 2s (borderline) and very few 1s (did not meet expectations).  The type of comments and the extent of their elaboration varied widely among clinical practitioner-instructors. Some clinical practitioner-instructors specified the procedures completed by the student. One or two provided scant or no qualitative feedback to students, while two or three regularly wrote descriptive paragraphs focusing on specific aspects of patient management or clinical practice. For example, one wrote,  The ‘4’ given for professionalism reflects your attitude in treatment planning this challenging case. You have recognized that it took more than one clinical session to complete diagnosis, treatment planning and review of the case with the parent. You have also followed through in obtaining multiple consults for a comprehensive treatment plan for this patient. These skills will serve you well in practice when you encounter similarly challenging cases.  63  Most clinical practitioner-instructors, however, provided short and fairly general comments, for example, “the preparations were done well”, “worked independently”, and “good patient management”. There were several instances when 4s and 1s were not accompanied with specific comments as required for student learning. There was even confusion about the definitions of grades of 1 and 4; the student and coordinator had to draw attention to the clinical practitioner-instructor that comments were praising exemplary performance, but a grade of 1 (not met expectations) was assigned. The researcher recognized that clinical practitioner-instructors usually had finished a busy and sometimes stressful clinic session as they were fulfilling their obligation to complete assessment and evaluation duties.  64  6.  Discussion Data collected were merged into themes drawn from the conceptual framework of  Hubball and Burt (2004). Practices ranged from directive methods steeped in historical and socio-cultural tradition to authentic methods advocated by and supported by current literature. In future, the data and themes of the framework will guide and direct progress in curriculum reform in the areas of assessment and evaluation practices and implementation of faculty development strategies.  6.1  Assessment and evaluation within the context of clinical dental education Situated within the context of Pediatric Dentistry clinical education at UBC are the  competency-based curriculum and the players within learning communities–clinical practitioner-instructors, dental students, patients and families, and the program coordinator. During interviews, it was discovered that most clinical practitioner-instructors were unaware of competency-based education (CBE) in dentistry. When provided an explanation about the dental competencies that underpin the curriculum at the Faculty of Dentistry at UBC, the clinical practitioner-instructors thought it desirable for graduate dental practitioners to be competent in the broad domains of the competencies, particularly professionalism. They commented on the value of clinicians who are respectful, caring, culturally competent, and effective communicators. Knowing one’s own limitations and when to refer a patient to a specialist was considered essential.  65  6.1.1  Professionalism  Philosophically, clinical practitioner-instructors believed that it is important for the dental school to instill a sense of professionalism in its students. However, most do not see it as their responsibility to impart values or transform attitudes in dental students. They perceive their primary role as helping students learn how to gain a child’s cooperation in the dental chair in order to be successful in restoring teeth. There is a focus on diagnosis, treatment planning, behaviour management, and clinical skills acquisition. Demonstration of professional and ethical behaviour is emphasized and expected of every dental student-practitioner at UBC. This is assessed continually throughout the dental school experience. Many clinical practitioner-instructors tacitly instruct students through modeling of their own professional attitudes and behaviours. For instance, there are some who arrive fifteen minutes before the start of every patient care clinic to meet with their group of students and prepare each student for the clinic session. They follow clinic guidelines and policies despite differences from their own private dental practices and communicate respectfully with students, patients, and parents. Having role models and being treated respectfully as colleagues help students understand their own responsibilities for professional and ethical practice (Roth, 2007; Taleghani, 2004). There were some clinical practitioner-instructors, however, who exhibited less than ideal behaviours, such as routinely arriving late and questioning or openly defying infection control protocols.  66  6.1.2 Health promotion and community service  Pediatric patients are referred from the community to the UBC Children’s Dental Program because they do not otherwise have access to dental care. Financial issues often pose a significant barrier. Largely, patients come from immigrant families of low socioeconomic status. One or both parents might be employed, but families do not have private dental insurance or qualify for government assistance. The context of families’ lives and circumstances sometimes gets forgotten by students and clinical practitioner-instructors. Complex treatment options are recommended which are beyond the scope of the free pediatric dental program or require follow-up at a private practice that cannot be sustained by families. The technical rational approach to teach or learn how to do a procedure can supersede the socio-political aspect of community service (Wilson & Cervero, 1997). Health promotion is a dental competency which addresses the recognition of the social determinants of health and a thoughtful and appropriate response to individuals, groups, or communities who are disadvantaged (UBC Faculty of Dentistry Competencies for the New Practitioner, 2006).  6.1.3 Evidence-based versus “in my experience” practice  Clinical practitioner-instructors who primarily practice in the community frequently provide instruction of “things that I use in my practice that are not described in the textbook”. Chambers (2009) suggested that multiple expert models and less than ideal contexts can present opportunities for students to develop critical thinking skills. At first, students might  67  be confused by differing expert opinions, but then they may be able to solve clinical problems through reflection and by refining their own individual practices.  On the other hand, Chambers also problematized this when dentists, who are accustomed to working independently and without scrutiny, are not held accountable to provide reasons for their clinical judgment. Historically, individual opinion and “in my experience” clinical dentistry have frequently substituted for best evidence practice, resulting in inconsistencies in clinical education (Werb & Matear, 2004). Teaching evidence-based care is the standard of practice expected today, but it is not always the case. Moreover, students may not feel empowered to question the clinical practitioner-instructor.  Clinical practitioner-instructors held opposing views whether practical tips or shortcuts to improve efficiency should be offered to students. There is often more than one method of technically achieving the same outcome. One way of thinking was that senior students have enough grounding and experience in basic foundations of clinical care that there is a benefit to see how things can be done differently. On the other hand, some clinical practitioner-instructors thought that students need more practice at doing things in one way before experimenting with another. One clinical practitioner-instructor felt that some shortcuts can be successful only when four-handed dentistry is practiced. Four-handed dentistry involves a dental practitioner working with a dental assistant. Although this is authentic in real-world practice, this is not embodied in the dental school environment.  68  6.1.4 Real-world challenges of dental practice  There were clinical practitioner-instructors who welcomed students’ questions in the end-of-the-day group huddles on the realities and challenges of real-world practice. They talked about clinical ergonomics, four-handed dentistry, and how to work as a team with dental staff (dental assistants, receptionists, and hygienists) to improve delivery of care and efficiency. Difficult issues, like how to handle mistakes and communication skills to prevent a malpractice suit, were occasionally broached. One clinical practitioner-instructor gave advice about plans for practice after graduation and the advantages and disadvantages of working in a rural community. Another clinical practitioner-instructor discussed practical and financial issues, such as limiting excessive use of sundries, managing different dental insurance plans, and dealing with patients who want a discount.  One group of students had been invited to visit the dental office of their clinical practitioner-instructor. The same clinical practitioner-instructor will be organizing an overseas volunteer experience for his students to take place shortly after graduation. These extra efforts seem to be highly appreciated by students and some regard their clinical practitioner-instructors as mentors.  6.2  Planning for assessment and evaluation Planning strategies refer to the systematic and scholarly preparations of clinical  faculty as they assess and evaluate to what extent students have achieved global and specific learning objectives (Albino et al., 2008; Hubball & Burt, 2004). It includes an iterative 69  process of deliberation, reflection, and change on the part of the clinical practitionerinstructor. Planning is intended to assess and evaluate achievement of, not only curriculum and program-specific learning outcomes, but also transfer of learning of higher order thinking skills (Subedi, 2004). Critical thinking, logical reasoning, problem solving, communication skills, and an ethical attitude will benefit the dental graduate ready to enter independent practice.  6.2.1 Gaps in a learning-centred approach  Little active planning for a learning-centred approach to assessment and evaluation was reported by clinical practitioner-instructors. Disadvantageous to students and clinical practitioner-instructors alike were the lack of familiarity of dental competencies and the spectrum of knowledge, skills, and attitudes desired for transfer. Those clinical practitionerinstructors with a moderate amount of experience in clinical instruction in the pediatric program had a sense of the varying skill levels of students and oral health needs of the patient community. Since session-by-session planning tended to be patient-centred and based on treatment outcomes, individual learning plans were not explicitly made. It was not often observed that there is a link between assessment and evaluation to specific learning objectives related to the process of student learning. Post-session group meetings or huddles were generally planned ad hoc based on patient cases of the day or with students highlighting significant learning experiences and outcomes.  70  Only one clinical practitioner-instructor defined a plan for clinical instruction that he called “KAPIT”, an acronym for “knowledge”, “anticipation”, “preparation”, “initiation”, and “termination”. His methodology is intended to train students to think ahead, be organized, time-efficient, and provide a safe environment for patients. He instituted this methodology in his private practice with his dental assistants as well as to plan for the smooth flow of care of patients.  6.2.2 Prior learning  In terms of preparation for the academic term, the program coordinator reviewed the process of assessment and evaluation at an orientation session. Guidance was given throughout the term as needed or as questions arose. In addition, an in-house educational enrichment seminar on meaningful formative assessment in patient care dental clinics (Richardson, 2011) was strongly recommended to all clinical practitioner-instructors. Although readily available on the UBC Faculty of Dentistry intranet, only when the program coordinator purposefully set aside time within clinic sessions and organized private viewings of the podcast seminar did most clinical practitioner-instructors actually watch it. Themes included definitions of criteria and standards, the importance of providing meaningful comments to help students learn, consistency between quantitative grades and qualitative comments, and norm-referenced evaluations.  Those who viewed the podcast said that they gained insight on the process. They appreciated learning about formative feedback in the context of clinical teaching and learning  71  which can help students learn, as opposed to rating and ranking (Bains, 2004). The rationale behind supporting grades of “exceeds expectations” and “did not meet expectations” with specific explanatory comments was reinforced. Those who did not view the podcast reported that they assessed and evaluated students based on their own perceptions, experiences, and preferences as learners and instructors. A second educational enrichment seminar/podcast on meaningful formative feedback was offered, but most clinical practitioner-instructors did not participate or view it.  Some of the clinical practitioner-instructors referred to their participation in a short series of seminars offered by the program coordinator in 2008, “Reflection and Response: Thinking about your teaching”. This one-time voluntary faculty development program was tied in with clinical instruction sessions and engaged interested clinical practitionerinstructors in the metacognitive task of self-reflection. It provided a facilitated and shared space and time, albeit brief, with peers for critical thinking about practice in clinical instruction, feedback, and assessment practices.  6.2.3 Student feedback and critical reflection  Clinical practitioner-instructors took cues from students’ progress and attitudes to plan or change their approach to assessment and evaluation practices. In interactions with disengaged or difficult students, clinical practitioner-instructors responded intentionally but in different ways, from trying to remain objective and suppressing emotional reactions to offering extra instruction and encouragement. While clinical practitioner-instructors wanted  72  to be seen as fair and seemed to value end-of-term student evaluations, the majority did not ask students for feedback on their instructional effectiveness during the term. By the same token, although they were aware that students had access to their daily assessments and evaluations, several did not know if students reviewed them and seemed surprised at a suggestion by the interviewer to ask students.  The researcher had a unique opportunity to observe moments of critical reflection and responses by two clinical practitioner-instructors during and after interviews. In the first case, use of classroom assessment techniques was discussed (Angelo & Cross, 1993). After inquiring and learning about the One-minute Paper, the clinical practitioner-instructor used it as a method to collect, analyze, and reflect on mid-course feedback from her students. One week later, she reported back to her group how she would use their feedback to make changes to help them in their learning. If critical self-evaluative reflection on student feedback can be cultivated, then it can inform practice and serve as the catalyst for thoughtful planning and change in practice (Bamber, 2011; Saunders, 2011).  The interview process in itself allowed a second clinical practitioner-instructor to reflect on her preferred use of descriptive versus evaluative feedback. Having had experience in research and implementation of student portfolios at another institution, her perspective was that “the answer is not evaluative feedback…it’s descriptive and you want students to be able to develop their own independent evaluative ability”. Over the course of the interview, she was able to reconcile that, although it can be difficult to be evaluative, there are times it is necessary to “evaluate” or make a judgment that a student is not meeting  73  the expected standards of patient care. At the same time, it is critical to “assess” and provide the data supporting this judgment.  A new faculty member of UBC, the clinical practitioner-instructor compared her experiences as a dental educator at different institutions and expressed how she “likes the UBC huddles because the students talk about their experiences and that’s when they reflect on what they’ve done.” This clinical practitioner-instructor planned a discussion with her group of students related to perceptions of their clinical experience with children. With prompting, two students in the International Dental Degree Completion Program openly shared insights as dentists who had practiced in different countries where they encountered challenges and complexities of social, political, and cultural factors that influenced their ability to work. Here was an example of students being supported by faculty in the process of meta-cognitive task of self-reflection and self-evaluation.  6.2.4 Needs assessment  In an attempt to draw out what clinical practitioner-instructors need to plan for improved assessment and evaluation practices, they were asked to identify faculty development needs to support them. This was difficult for many to articulate, but responses included, “how to transfer knowledge to students in a way they can understand”, “how to give effective, consistent feedback and to critically assess”, and “have someone monitor how I teach in real time and give me feedback”.  74  6.3  Programming and learning strategies for assessment and evaluation Desirable programming strategies refer to the diverse methods of instruction that  help integrate learning (Hubball & Burt, 2004) through authentic assessment and evaluation. Assessment and evaluation require a scholarly approach that is informed by the higher education literature (Bamber, 2011; Hubball & Pearson, 2009). This is analogous to good clinical practice that is evidence-based and informed by scientific and dental research. What follows is an analysis of programming and learning strategies in assessment and evaluation, many of which are not evidence-based but embedded in the culture of dental education and based on “the ways in which we do things around here”. While clinical practitionerinstructors use a variety of instructional, assessment, and evaluative strategies, they expressed that they do not know how effective they are in helping students develop as practitioners.  6.3.1 Goal-directed practice  A relevant programming strategy in clinical dental education is provision of multiple opportunities for goal-directed practice at the appropriate level of challenge (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010). Ways to address goal-directed practice in clinical teaching and learning include use of a rubric that outlines performance criteria, setting expectations about practice, scaffolding learning, and modeling target performance.  Although there is intent for alignment of learning objectives, instructional strategies, and assessment in the UBC Children’s Dental Program, complexities in pediatric patient care  75  often make it difficult for learning experiences to be sequenced so that they are at the appropriate level of challenge for the student. For example, a novice student-dentist might be paired with a challenging child patient with a lot of dental treatment needs who refuses to comply for dental treatment. Another student-dentist might be responsible for the care of a cooperative child. The experiences for the students will differ dramatically but, over the course of the year, the student will encounter multiple opportunities for clinical practice with different patients, circumstances, and problems. It was variable how each clinical practitioner-instructor selected instructional strategies, for example, demonstration versus dismissal of the patient from the program and referral to a specialist. With respect to assessment and evaluation, the Patient Clinical Care Performance standards and criteria and norm-referencing were not always appropriately used.  Clinical practitioner-instructors were busy throughout clinic sessions using many different strategies to assess students’ clinical performance. They were constantly watchful of students’ procedural progress, organization, time management, and efficiency. They monitored patients to see how each was handling and responding to the procedure. However, as mentioned before, there were only a few clinical practitioner-instructors who arrived early to set specific goals and expectations for individual students with a focus to practice. It was uncommon for a clinical practitioner-instructor to discuss what specific learning outcomes or types of feedback the student needed.  76  6.3.2 Setting expectations around student independence  Clinical practitioner-instructors varied in their expectations of their group of students. One had higher expectations than the others and urged all of his students to achieve daily goals and get more procedures done in a session. This was coupled with the clinical practitioner-instructor being more involved with assisting students at the chairside, lending them instruments and supplies that he had brought in from his dental office, and cleaning up. At the opposite end of the spectrum, another clinical practitioner-instructor let students set the pace and complete a procedure as independently as possible, prioritizing independence and quality to efficiency and quantity.  Both strategies have benefits and downsides. It is advantageous to complete “quadrant dentistry” (restore all teeth in an area of the mouth at one time) for a child so that the total number of appointments to complete all treatment is minimized. On the other hand, allowing the student to negotiate the full range of clinical care on his or her own over more time can be valuable in recognizing one’s own strengths and limitations.  Findings in a study on a critical thinking course (Chambers, 2009) showed that clinical knowledge was not uniformly distributed among students and that faculty members and students significantly underestimated gaps in students’ knowledge. This might lead us to alter approaches to clinical teaching to uncover the gaps where practical and safe and allow students to practice with less instructor intervention. It can be difficult to balance student learning with service to patients.  77  For many clinical practitioner-instructors, the interview was the first time that they had an opportunity to actively reflect on what they do. They responded that they most often teach in the way that they learn best or the way they think is the successful way to learn. Experiential learning by clinical practitioner-instructors guided the ways they assessed and evaluated students’ performance. For instance, as was mentioned above, some took a more hands-on approach; others, often with more experience, tended to step back, wait and monitor. The few with some educational background tended to be more reflective on their practice and responsive to student’s needs. This was illustrated by one who discussed the importance of Vygotsky’s zone of proximal development and instructional scaffolding in student learning.  6.3.3 Targeted feedback  Research specifies targeted feedback that will be most useful in clinical assessment and evaluation: balanced, prioritized, frequent (but not overwhelming), individual, group, and peer feedback. Also significant are opportunities for students to explain how they used feedback in subsequent work. (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010). Regular and ongoing feedback is the norm in clinical dental training as students engage in patient care. Although work was continuously monitored at required checkpoints, less monitoring and hovering was reported as students gained experience, speed, confidence, and competence. At the required checkpoint of a procedure, it was observed that a clinical practitioner-instructor often looked at a student’s work, made a snap judgment of it, and then gave directions for correction if warranted. On occasion, a clinical practitioner-instructor  78  was observed to follow up at subsequent sessions on suggestions that he or she had made to see whether students had incorporated feedback. There were some spontaneous teachable moments captured that resulted in the recognition of exemplary work or correction of a deficiency (Ramani, 2003). Re-positioning of a patient by the clinical practitioner-instructor was an example where posture and ergonomics of the student practitioner improved substantially and immediately.  Teacher-centred practices were the norm traditionally in dental school assessment (Taleghani, 2004). Some clinical practitioner-instructors disclosed that they provided feedback to students by “directing them how to correct something” or “telling them to read up on the subject”. Others used questions prompting students to make associations with prior learning or experiences and construct their own knowledge (McMillan, 2011). Other ways described were to “give feedback right at the time things are happening”, “use the sandwich technique or balanced feedback”, “break a procedure down into small parts”, and “to give ongoing encouragement on progress”.  6.3.4 Demonstration and modeling  While all agreed that intervention was critical if there was any serious threat of harm to the patient or when running out of time was an issue, intervention through demonstration and modeling was also perceived as one of the most important strategies for teaching and learning. Fugill (2005) stated that demonstration needs to be managed carefully for two reasons: (1) without corresponding articulation of what he or she is doing, the knowledge and  79  reasoning underlying the demonstration might not be obvious to the student and (2) the patient or the parent might question the ability of the dental student. Thinking out loud to describe the reasoning process by an expert while modeling or demonstrating a procedure can serve three purposes. The learner gets to hear the expert as he or she fits the clinical case or problem into a general framework, translates evidence-based research of a population to a specific patient, or navigates the journey of problem solving (Reilly, 2007). This integrative instructional tactic was not utilized much in the UBC clinical settings. It was more or less assumed that students were learning through visualizing the demonstration of a procedure.  6.3.5 Communities of practice  UBC Children’s Dental Program is structured so that there is one patient assigned to each student for the clinic session. A small group of students is supervised and instructed by one clinical practitioner-instructor. As much as possible, the student-instructor groups remain constant throughout the first term of fifteen sessions, with the exception of a couple of cases where a pair or a group of clinical practitioner-instructors share responsibility for a group and alternate attending sessions. The groups become communities of practice (Wenger, 1998).  The group meetings or “huddles” at the end of the day have now become a regular occurrence in clinical teaching and learning in Pediatric Dentistry at UBC. Over the past few years, the Program Coordinator has provided rationale for the group discussions and persuaded clinical practitioner-instructors to engage students in dialogue for learning.  80  Justification and change have been uneven and slow to be accepted as critical reflection for transformative learning has not been established as mainstream in many aspects of dental curricula.  According to Wenger’s social theory of learning, learning will occur in a community of practice as members mutually engage in pursuit of and production of meaning and develop a shared repertoire of resources for learning. Members of the clinical communities of practice share experiences in the daily huddles, assist each other when possible, and have consistent assessments and evaluations by the same clinical practitioner-instructor for a period of time.  In the group with multiple, rotating clinical practitioner-instructors, consistency can be a problem. Through the interviews, it became apparent that, although they had a shared philosophy of practice and practiced in the same office, they did not communicate with each other about the progress of their group of students. Likewise, students became confused and frustrated at times if their clinical practitioner-instructor was absent and the replacement instructor had a differing opinion about treatment planning or approaches to clinical instruction.  6.3.6 Opportunities for students to self-assess  With time a limiting factor, asking students to assess their own work was not a common practice during the clinical sessions. Students are not required to explicitly monitor their own progress and shortcomings using performance criteria and standards. Although  81  self-reflection is encouraged in the huddles, it would be better to provide opportunities for students to also assess themselves in real time (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010). Clinical practitioner-instructors might ask or guide self-assessment of work in progress. The construct of self-assessment in dental education was defined by Redwood, Winning, Lekkas, & Townsend (2010) as the ability to evaluate one’s own abilities, attitudes, and performance against professional criteria and standards of competence. Effective selfassessment by individual practitioners is essential for the ongoing and constructive assessment of one’s own professional actions to inform future learning needs (Mattheos, Nattestad, Falk-Nilsson, & Attstrom, 2004).  6.3.7 Management of challenging students  Clinical practitioner-instructors mostly expressed satisfaction working with dental students. A few expressed frustration when dealing with a student who has repeated challenges in understanding a concept or performing a clinical procedure. Sometimes the transfer of learning would be unsuccessful, despite all attempts to have the student describe his or her plan, demonstration, probing questions, discussion of concerns, and poor evaluations. Having an understanding of barriers to transfer of learning, such as training design and delivery-related factors (instructional techniques as an example) and learnerrelated factors (self-efficacy, abilities and skills, motivation as examples) might be useful for clinical practitioner-instructors (Caffarella, 2002; Subedi, 2004).  82  6.4  Application of criteria and standards for assessment and evaluation Assessment strategies in the conceptual framework of Hubball & Burt (2004) refer to  the range of methods and procedures that assess and evaluate learning. Patient care performance criteria and standards are used to assess and evaluate the dental student’s clinical performance at UBC Faculty of Dentistry. Clearly defined criteria for learning outcomes and assessment and evaluation are available in writing and on the UBC Dentistry intranet. The grading scheme is communicated to clinical practitioner-instructors and students. Standards and criteria of professionalism; application of knowledge; clinical skills; organization, time management, and infection control; difficulty of procedure; and degree of student independence are outlined in the patient care performance rubric (Appendix C).  6.4.1 Consistency of application of standards to clinical performance criteria  Assessment and evaluation procedures and clinical performance criteria comply with requirements outlined by Plasschaert, Manogue, Lindh, McLoughlin, Murtomaa, Nattestad, & Sanz (2007). These include the employment of both formative and summative feedback on multiple dimensions of competence (knowledge, skills, observed behaviours, and safety), and assessment of quantity and quality of clinical performance. However, work is constantly in progress to assist clinical practitioner-instructors at UBC to understand specific criteria, to realize that evaluation standards are norm-referenced, and to be more consistent in application of criteria to standards.  83  Specifically, assessing the difficulty of a procedure posed a problem for several clinical practitioner-instructors. For some, it was unclear to what standards difficulty was measured against−the novice, the general practitioner, or the specialist. Many being proficient in pediatric dental practice would deem a routine procedure low on the difficulty scale, whereas a student performing the same procedure for the first time would find it to be quite difficult. Similarly, some clinical practitioner-instructors would consider a noncompliant child with behaviour management issues to be routine, whereas this might be a very challenging situation for the novice practitioner.  6.4.2 Grading student independence and patient safety  Grading student independence was not always simple for clinical practitionerinstructors to negotiate. It is recognized that pressure for students to act independently and to be productive is a part of the culture of dental education. This can result in a reluctance of students to ask for help when needed and have implications for patient safety.  Kennedy, Regehr, Baker, and Lingard (2009) argue that there is a boundary of safe practice that is defined by the competence level of the individual student and this boundary changes with training and skill acquisition. Thus, it is incumbent on the clinical practitionerinstructor to be learner-centred. It is a fine balance to allow a student to be in the zone of proximal development and simultaneously minimize risks to patient safety. It might be useful for clinical practitioner-instructors to explicitly outline and monitor safety boundaries to students at different levels of training. Explaining how to ask for help in a way that will  84  not negatively affect evaluation will relieve pressure from students programmed for high achievement. Finally, good practice will include mechanisms for reflection and debriefing when violations occur.  6.4.3 Grade inflation  In the previous year in which the patient care performance rubric was piloted in the UBC Children’s Dental Program, there was a predilection towards grade inflation. In the first term of the 2011-12 academic year, the Program Coordinator regularly reviewed the sessional assessments and evaluations by clinical practitioner-instructors. When qualitative comments were not included to support the quantitative grades or when students in one group were consistently “exceeding expectations”, she would follow up with the clinical practitioner-instructor to ensure that any inaccuracies or inconsistencies were amended.  6.4.4 Grading student performance that does not meet expectations  In addition to grade inflation, a reluctance to provide honest feedback when a student does not meet expectations was problematic. After assigning such a grade to a student, one clinical practitioner-instructor received an e-mail message from the student who was upset and disagreed with the grade. Although the clinical practitioner-instructor again provided his rationale for the evaluation and maintained his stance on the grade and comments, he was negatively impacted by the confrontational situation and expressed that he would hesitate to  85  give that grade again. This has been described in the dental educational literature as discomfort by faculty members having to make and defend professional judgments and the feeling by many that “we evaluate fine; the problem is willingness to make the hard decisions about incompetent students” (Licari & Chambers, 2008).  By the same token, another clinical practitioner-instructor evaluated a student with a failing grade in professionalism on two different occasions. In both circumstances, his assessment was composed of a detailed narrative of the sequence of events that led to the grade of “not meeting expectations” in professional behaviours and attitudes. His evaluation was aligned to the criteria defined for learning outcomes in the 2011-12 module syllabus, “The learner will demonstrate professionalism and ethical practice in patient care clinics and small group participation”. It was linked as well as to the UBC dental competency statements (2006) that the new practitioner must “apply accepted principles of ethics and jurisprudence to maintain standards and advance knowledge and skills and demonstrate professional behaviour that is ethical, supersedes self-interest, strives for excellence, is committed to continued professional development and is accountable to individual patients, society and the profession”.  With help from the Program Coordinator, the clinical practitioner-instructor strove to describe what happened objectively, determined what specific things were below expectations, and provided evidence to support the evaluation. He was very concerned about the events and tried hard to have the student understand the reasons for his concern. Later, a plan for remediation was set in place.  86  6.4.5 Toward self-directed learning and self-assessment  Lifelong learning requires that a professional practitioner be able to continually assess the outcome of his or her actions in practice and accordingly define a plan for his or her own learning needs (Caffarella, 2002; Mattheos, Nattestad, Falk-Nilsson, & Attstrom, 2004). It was recognized by one interviewee that the criteria and standards of the patient care performance rubric might provide the starting point for students to self-assess and evaluate professional actions, behaviours, and attitudes. However, use of the criteria and standards are restricted to the more formal written feedback at the conclusion of the session. It would be beneficial to use them as an instructional tool in real time. Another alternative would be to have students conduct self-assessments and evaluations, triangulate with instructors’ measures, and receive feedback to help students develop more insight into their personal and professional behaviours (Rees & Shepherd, 2005). Portfolios for self-reflection on clinical work are used for teaching and learning in other areas of operative dentistry at UBC.  Use of technology to do the assessments/evaluations was a challenge for some, particularly so for the global periodic assessments at the end of the term. It would have been helpful for students to be apprised of their overall progress to date by their clinical practitioner-instructor. Being advised of strengths and given areas to work on for improvement would aid self-directed learning in assessing future tasks, planning, applying strategies, monitoring performance, and reflection (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010). In the second term, students have pediatric dental rotations to the dental clinic at Douglas College, where they are supervised and instructed by any of a number of clinical practitioner-instructors.  87  7.  Conclusions/Reflections Through investigation of the current assessment and evaluation practices of clinical  practitioner-instructors in the predoctoral pediatric dental program at UBC, the following research questions were addressed:  1. What is the nature and scope of the assessment and evaluation practices conducted by clinical practitioner-instructors in Pediatric Dentistry clinical education settings at UBC? 2. What sense do the clinical practitioner-instructors make of the practice that they engage in?  With respect to Research Question 1, the main findings of this study reflected the traditional teacher-centred and skills-based practices embedded within the culture of dental education. Assessment and evaluation practices by Pediatric Dentistry clinical practitionerinstructors focused on technical competence, with an additional emphasis on behaviour management of the child patient. The latter itself is important, but cooperation of a patient must first be gained in order to achieve the former. During patient care, it was typical for clinical practitioner-instructors to be directive, telling students what to do, instead of requiring students to analyze their own practice and solve the problem at hand. Development of practice fluency was hindered due to a lack of instructor appreciation that a novice needs to learn to integrate component skills, practice, and different patient contexts (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010). Rather, goals tended to be task-based. In written assessments and evaluations, grading was completed without reference to the UBC  88  Dentistry patient care performance criteria and standards. Not infrequently, comments were sparse, non-specific, and inconsistent with quantitative grades.  Yet, there were occasional instances where learning-centred strategies were observed, such as the use of questioning to probe students’ understanding. There were clinical demonstrations when a clinical practitioner-instructor coupled visual learning with “thinking out loud” or verbal reasoning, then later reinforced the learning with discussion and opportunities for reflection in a group huddle. Overall, it was noted that assessment and evaluation practices were individual practices, silo versus social practices, and task-oriented with little reflective quality, much like a traditional dental practice.  Regarding Research Question 2, a significant finding was that most clinical practitioner-instructors were unaware of UBC Dentistry competencies for the new practitioner. Clinical practitioner-instructor’s understood their primary role as developing foundational skills of patient care (patient assessment, behaviour management, disease prevention and management, and rehabilitation). There was a general perception that the responsibility lay with others to educate students in professional and ethical behaviours, practice management, and health promotion for the community at large. Few recognized teaching as a separate skill set from practicing as a dentist. In the interviews, it surprised some to hear that considerations of how adults learn and strategies to support student learning might be compared to, respectively, evidence in evidence-based dental practice and instruments that improve effectiveness. Many based their assessment, and evaluation practices on their own experiences as students or their own preferences. Most clinical practitioner-instructors appeared to be satisfied but static in their development as instructors.  89  They had no time for relevant seminars, self-reflection, and formative student feedback regarding their effectiveness as instructors.  On the other hand, there seemed to be an instinctual understanding of knowing when something was working very well. Some clinical practitioner-instructors understood how to challenge students at the appropriate level, urging them to accomplish more or accept a patient with complex needs. Some recognized the value of peer teaching and learning and reflection that resulted from a well-managed group huddle. The researcher was heartened by the impact of the interviews on those clinical practitioner-instructors who began to dialogue and reflect amongst themselves and with the researcher.  7.1  Implications of inquiry-based practice for dental education The phenomenon under investigation included representations of both the  participants’ and the researcher’s perspectives. In this study, the researcher is the Program Coordinator of the UBC Children’s Dental Program when clinic sessions are held at the Nobel Biocare Oral Health Centre at UBC. The analysis was reliant on the coordinatorresearcher who has knowledge of the cultural and social structures of practice. She is familiar with research literature on assessment and evaluation practices in higher education and dental education contexts. Also significant in this study were relational issues of communication, trust, and transparency of the research goal to investigate the nature, scope, and meanings of assessment and evaluation practices. Through the semi-structured interviews and observation of practice, the internal researcher was able to capture the lived  90  experiences of the participants (Shreeve & Blythman, 2011) and respond to identified needs as they arose in practice. Potentially, these relational issues may help in the future acceptance of faculty development by clinical practitioner-instructors.  Study participants were very much engaged in the interview process. This was in stark contrast to an absence of time available for peer-sharing. An unexpected by-product of the study was that it opened up conversations about practice amongst clinical practitionerinstructors and between individuals and the researcher/program coordinator. If an open forum for dialogue persists, then there is more potential for change to practice through demand for knowledge and transfer of collaborative learning. One of the missing links for clinical practitioner-instructors is that clinical instruction is not a social practice outside of the individual groupings of instructor and his or her students. It would be beneficial for clinical practitioner-instructors to have time to share their experiences in the teaching and learning context of clinical dentistry. Not only can educational expertise of others be drawn upon, but also decision-making and problem-solving can be supported when critical incidents occur.  A research-informed approach has potential to stimulate awareness of and plan towards learning-centred pedagogy and authentic assessment and evaluation practices in clinical dental educational settings. Areas for faculty development might begin with strategies for goal-directed practice and targeted feedback, empowering learners to become self-directed, and interpretive judgment to judge competence across multiple domains (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010; Svinicki, 2004; van der Vleuten, 2000).  91  The researcher recognizes that she can be an agent of change and promote the value of research in authentic assessment and evaluation practices, but that the “bottom-up” approach might present challenges for clinical faculty to adopt a more evidence-based approach (Shreeve & Blythman, 2011). Dissemination of the study’s outcomes and proposals for change must be in a way that is sensitive to clinical practitioner-instructors and in language that is acceptable and accessible to faculty leaders and key stakeholders. Utilizing existing networks and relationships may enhance the potential for change and innovation.  7.2  Recommendations Recommendations to enhance assessment and evaluation practices by clinical  practitioner-instructors in the UBC Children’s Dental Program:  1. Acknowledge current contributions of clinical practitioner-instructors and encourage development of practices that strive toward authentic assessment and evaluation of student performance in clinical learning situations 2. Build in individual learning opportunities for clinical practitioner-instructors within clinical sessions. The clinical practitioner-instructor will have opportunities to access: a. Scheduled instructional support sessions with the program coordinator– formative assessment based on needs as jointly identified by clinical  92  practitioner-instructor and program coordinator, use of and interpretation of student feedback b. Peer review–as requested by a clinical practitioner-instructor to provide additional exploration or discussion of an area of concern in clinical teaching c. On-line instructional resources 3. Plan and implement four to five short faculty development seminars linked in time to clinical instruction sessions on topics such as practice and feedback, developing selfdirected learners, and diversity in teaching and learning styles. Facilitators of faculty development programs should role model good teaching practice and learning-centred strategies of: a. Active learning and reflection–using case-based scenarios b. Social practice and collegial interaction with peer teaching and learning c. Respect for diversity amongst participants, for example, differences in culture, background, and experience 4. Disseminate study results and look for opportunities to collaborate with other SoTL leaders on initiatives related to educational research and clinical teaching and learning  7.3  Limitations of the study In-depth interviews and observation of fifty percent of clinical practitioner-instructors  in the predoctoral Pediatric Dentistry program at the UBC site, may limit reliability. Although purposeful sampling was used, some perspectives might not have been represented. No participants represented the group of clinical practitioner-instructors at the Douglas 93  College dental clinic where the UBC Children’s Program is held in the second term of the academic year.  Contextual factors may limit the generalizability of finding to other settings, including other dental disciplines or community teaching clinics. For example, in dental clinics for children, communication and rapport must be established with the patient in order to be able to gain cooperation to complete a dental procedure. This is not necessarily so in the case of dentistry for adults who generally have more control of their emotions and behaviour.  Although all of the participants were involved in clinical instruction of the same group of final-year dental students, no comparison was made to distinguish between assessment and evaluation practices of primarily private practitioners in specialty or general practice, academic faculty, and graduate students. Nor was years of experience as a clinical practitioner-instructor or previous training as an educator considered to distinguish interview responses, actions during practice, or formal assessments and evaluations. Instead, the study presented the range of practices of the group of clinical practitioner-instructors.  7.4  Further study Further study might use a different sample of clinical practitioner-instructors. It  might be of interest to investigate if there is a greater emphasis on the assessment and evaluation of technical skills and procedure counts where behaviour management of children and communication strategies are not as essential. Analyzing consistency of grading and 94  comments provided by the clinical practitioner-instructors is another avenue for exploration. One might compare the value of descriptive and evaluative feedback to long-term student learning and development.  Provision of faculty development has been recommended as the next step to support curriculum reform initiatives (such as competency-based curriculum) and enhance teaching effectiveness (Hendricson et al., 2007). Further exploration is required to assess needs of clinical faculty and to design faculty development programs that will likely be attended, perceived as pertinent, and facilitate transfer of learning of concepts and skills applicable to everyday practice (Caffarella, 2002; Hendricson et al, 2007; Sork, 2001). Once implemented, research is required to evaluate faculty development programs and explore programs or program characteristics that are associated with effectiveness.  7.5  Dissemination opportunities The results of this study might be disseminated in future Educational Enrichment  seminars to clinical practitioner-instructors who serve in other dental disciplines at the Faculty of Dentistry at UBC. “Lunch and learn” seminars in the dental school are a common venue for dissemination of research by graduate students. More broadly, the research could be submitted for publication in higher education or dental education journals or for presentation at conferences, such as the American Dental Education Association annual session and exhibition. The American Dental Education Association publishes the  95  Journal of Dental Education, a monthly peer-reviewed journal. The study would add to the body of literature which calls for change and innovation in dental education.  The researcher (and Program Coordinator of the UBC Children’s Dental Program) intends to share the study findings with the clinical practitioner-instructors of the program, many who volunteered as study participants. The findings, along with learning/participation/presentations at the Educational Developers Caucus Conference (Halifax, 2012) and Investigating our Practices Conference (UBC, Vancouver, 2012) provide the starting point for planning with the community of clinical practitionerinstructors about how faculty development might be mutually envisioned and implemented. From the interviews, the researcher has already begun to gather data about needs, where supports lie, and what challenges exist. It is clear that faculty development must be planned so as to be relevant, accessible, and practical for the end users and with the end goal of enhanced student learning, development, and performance.  7.6  Final reflections Expertise as a practicing dental clinician is an entry level requirement to serve as a  clinical practitioner-instructor, but pedagogical expertise cannot be assumed and is not yet requisite in the research-intensive university. The shift from a traditional teaching-centred approach to a transformative learning-centred one in clinical dental education has been slow to emerge. This is in spite of competency-based dental education and motivation to graduate dental professionals ready to begin practice competent in many broad domains. Future  96  directions include planning a framework of faculty development strategies that are relevant to the context of clinical teaching and learning, informed by educational research, and sanctioned by faculty leaders and key stakeholders.  Although time is often identified as the barrier which perpetuates a focus on clinical or technical competence, clinical practitioner-instructors are largely unaware of the concept of competency-based education. They appear not to appreciate, or have resisted, curriculum changes and innovations to meet the needs of the modern-day dental practitioner. There is a need for faculty development initiatives to prepare clinical practitioner-instructors to improve their clinical teaching effectiveness for enhanced student learning and development.  Clinical practitioner-instructors bring a commitment to their dental students and are respected as expert clinicians. All are dedicated to the delivery of high-level care to address the dental needs of children. Within the context of clinical teaching and learning environments, most clinical practitioner-instructors recognize the rigour and expectations of the research-intensive university setting and have a willingness to learn. Developing a strong community of practice, mutual respect, active engagement, and collaboration with leaders who have scholarly expertise has the potential to empower clinical practitioner-instructors to develop their capacity. Implementation of strategies for authentic assessment and evaluation then will enhance student learning and performance. With the understanding that change can be slow and uneven to happen, clinical practitioner-instructors can be proficient in disciplinary content and practice as well as competent in pedagogical knowledge and application.  97  References Albino, J. E. N., Young, S. K., Neumann, L. M., Kramer, G. A., Andrieu, S. C., Henson, L., Horn, B., & Hendricson, W. D. (2008). Assessing dental students’ competence: Best practice recommendations in the performance assessment literature and investigation of current practices in predoctoral dental education. Journal of Dental Education, 72, 1405-1435. Retrieved May 8, 2012, from http://www.jdentaled.org.ezproxy.library.ubc.ca/content/72/12/1405.full Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2000) How learning works: 7 research-based principles for smart teaching. San Francisco, CA: Jossey-Bass. Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco, CA: Jossey-Bass. Bains, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press. Bamber, V. (2011). Evaluative practice and outcomes: issues at the self-evaluative level. In M. Saunders, P. Trowler, & V. Bamber (Eds.), Reconceptualising evaluation in higher education: The practice turn (193-199). Berkshire, England: McGraw-Hill. Barr, R. B., & Tagg, J. (1995). From teaching to learning–a new paradigm for undergraduate education. Change, 27(6), 12-25. Retrieved May 8, 2012, from http://search.ebscohost.com.ezproxy.library.ubc.ca/login.aspx?direct=true&db=aph& AN=9512100779&site=ehost-live Beaudry, M. L. and Schaub, T. (1998). The learning-centred syllabus. The Teaching Professor, 12(2), 5.  98  Brualdi, A. (1998). Implementing performance assessment in the classroom. Practical Assessment, Research & Evaluation, 6(2). Retrieved May 8, 2012, from http://pareonline.net/getvn.asp?v=6&n=2 Caffarella, R. S. (2002). Planning programs for adult learners: A practical guide for educators, trainers, and staff developers (2nd ed.). San Francisco, CA: Jossey-Bass. Caffarella, R. S., & Zinn, L. F. (1999). Professional development for faculty: A conceptual framework of barriers and supports. Innovative Higher Education, 23, 241-254. doi: 10.1023/A:1022978806131 Chambers, D. W. (1999). Faculty ratings as part of a competency-based evaluation clinic grading system. Evaluation & the Health Professions, 22, 86-106. doi: 10.1177/01632789922034185 Chambers, D. W. (2009). Lessons from students in a critical thinking course: A case for the third pedagogy. Journal of Dental Education, 73, 65-82. Retrieved May 8, 2012, from http://www.jdentaled.org/content/73/1/65.full?sid=9c08cfad-a4b3-4c95-af904a89050b5667 Chickering, A. W., & Gamson, Z. (1987). Seven principles for good practice n. AAHE Bulletin, 39(7), 3-7. Retrieved May 8, 2012, from https://engineeringdev.colorado.edu/sites/default/files/seven_principles_article.pdf Coormarasamy, A., & Khan, K. S. (2004). What is the evidence that postgraduate teaching in evidence-based medicine changes anything? A systematic review. British Medical Journal, 329. doi: 10.1136/bmj.329.7473.1017 Corbin, J., & Strauss, A. (2008). Basics of qualitative research (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc.  99  Curriculum and Teaching Effectiveness Committee Approval. (2006). UBC Faculty of Dentistry competencies for the new practitioner. Retrieved May 8, 2012, from https://secure.dentistry.ubc.ca/intranet/academicprograms/dmd/UBCCompetencyDoc ument.pdf Fenwick, T. J., & Parsons, J. (2000). The Art of evaluation: A handbook for educators and trainers (2nd ed.). Toronto, ON: Thompson Educational Press. Fugill, M. (2005). Teaching and learning in dental student clinical practice. European Journal of Dental Education, 9, 131-136. doi: 10.1111/j.1600-0579.2005.00378.x Galacová, T. 2008. On strategies contributing to active learning. Teaching Mathematics and its applications, 27, 116-122. doi:10.1093/teamat/hrn013 Gall, M. D., Gall, J. P., & Borg, W. R. (2010). Applying educational research (6th ed.). Boston, MA: Pearson Education, Inc. Gardner, H. (1993). Multiple intelligences: The theory in practice. New York, NY: Basic Books. Gerzina, T. M., McLean, T., & Fairley, J. (2005). Dental clinical teaching: Perceptions of students and teachers. Journal of Dental Education, 69, 1377-1384. Retrieved May 8, 2012, from http://www.jdentaled.org/content/69/12/1377.full?sid=d38055f1-5f044ba7-8143-e958d86333d2 Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetro, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: Time for a map? The Journal of Continuing Education in the Health Professions, 26, 13-24. doi:10.1002/chp.47  100  Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research & Development, 52(3), 67-86. doi: 10.1007/BF02504676 Haden, N. K., Hendricson, W. D., Kassebaum, D. K., Ranney, R. R., Weinstein, G., Anderson, E. L., & Valachovic, R. W. (2010). Curriculum change in dental education 2003-09. Journal of Dental Education, 74, 539-557. Retrieved May 8, 2012, from http://www.academicleaders.org/files/curriculumChange.pdf Hendricson, W. D., Anderson, E., Andrieu, S. C., Chadwick, D. G., Cole, J. R., George, M. C., Glickman, G. N., Glover, J. F., Goldberg, J. S., Haden, N. K., Kalkwarf, K. L., Meyerowitz, C., Neumann, L. M., Pyle, M., Tedesco, L. A., Valachovic, R. W., Weaver, R. G., Winder, R. L., & Young, S. K. (2007). Does faculty development enhance teaching effectiveness? Journal of Dental Education, 71. 1513-1533. Retrieved May 8, 2012, from http://www.jdentaled.org/content/71/12/1513.full Henzi, D., Davis, E., Jasinevicius, R., & Hendricson, W. (2006). North American dental students’ perspectives about their clinical education. Journal of Dental Education, 70, 361-77. Retrieved May 8, 2012, from http://www.jdentaled.org/content/70/4/361.full Hubball, H., & Burt, H. (2004). An integrated approach to developing and implementing learning-centred curricula. International Journal for Academic Development, 9, 5165. doi:10.1080/1360144042000296053 Hubball, H., & Burt, H. (2007). Learning outcomes and program-level evaluation in a fouryear undergraduate pharmacy curriculum. American Journal of Pharmaceutical Education, 71(5), Article 90. doi: 10.5688/aj710590  101  Hubball, H., Collins, J., & Pratt, D. (2005). Enhancing reflective teaching practices: Implications for faculty development programs. Canadian Journal for Higher Education, 35(3), 57-81. Retrieved May 8, 2012, from http://search.ebscohost.com.ezproxy.library.ubc.ca/login.aspx?direct=true&db=ehh& AN=20815234&site=ehost-live Hubball, H., & Levy, A. R. (2004). Graduate-level course design and assessment in epidemiology: A learning-centred approach. Journal of Faculty Development, 20(1), 10-20. Retrieved May 8, 2012, from https://newforums.metapress.com/content/j3750j6037873x65/resourcesecured/?target=fulltext.pdf Hubball, H., & Pearson, M. L. (2009). Curriculum leadership portfolios: Enhancing scholarly approaches to undergraduate program reform. Transformative Dialogues, 3(2), 1-16. Retrieved May 8, 2012, from http://kwantlen.ca/TD/TD.3.2/TD.3.2_Hubball&Pearson_Curriculum_Leadership_Po rtfolios.pdf Hubball, H., & Poole, G. (2003). A learning-centred Faculty Certificate Programme on university teaching. International Journal for Academic Development 8(1/2), 11–24. doi:10.1080/1360144042000277900 Huber, M. T. (2006). Disciplines, pedagogy, and inquiry-based learning about teaching. New Directions for Teaching and Learning, 107, 69-77. doi: 10.1002/tl.246 Huitt, W., Hummel, J., & Kaeck, D. (2001). Assessment, measurement, evaluation, and research. Educational Psychology Interactive. Retrieved May 8, 2012, from http://www.edpsycinteractive.org/topics/intro/sciknow.html  102  Katajavuori, N., Hakkarainen, K., Kuosa, T., Airaksinen, M., Hirvonen, J., & Holm, Y. (2009). Curriculum reform in Finnish pharmacy education. American Journal of Pharmaceutical Education, 73(8), Article 151. doi: 10.5688/aj7308151 Kennedy, T. J. T., Regehr, G., Baker, G. R., & Lingard, L. A. (2009). ‘It’s a cultural expectation…’ The pressure on medical trainees to work independently in clinical practice. Medical Education, 43, 645-653. doi:10.1111/j.1365-2923.2009.03382.x Kolb, A. Y., & Kolb, D. A. (2005). The Kolb Learning Style Inventory—Version 3.1 2005 technical specifications. Retrieved May 8, 2012, from http://www.whitewaterrescue.com/support/pagepics/lsitechmanual.pdf Kreber, C. (2002). Teaching excellence, teaching expertise, and the scholarship of teaching. Innovative Higher Education, 27(1), 5-23. doi:10.1023/A:1020464222360 Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York, NY: Cambridge University Press. Licari, F. (2007). Faculty development to support curriculum change and ensure the future vitality of dental education. Journal of Dental Education, 71, 1509-1512. Retrieved May 8, 2012, from http://www.jdentaled.org/content/71/12/1509.full Licari, F. W., & Chambers, D. W. (2007). Some paradoxes in competency-based dental education. Journal of Dental Education, 72, 8-18. Retrieved May 8, 2012, from http://dent.osu.edu/OAA/faculty_development/Curriculum/CurriculumDesign/Parodx es1_08.pdf Marambe, K. N., Athuraliya, T. N., Vermunt, J. D., & Boshuizen, H. P. (2007). A comparison of learning strategies, orientations and conceptions of learning of firstyear medical students in a traditional and an innovative curriculum. Annals of the  103  Academy of Medicine, Singapore, 36, 751-755. Retrieved May 8, 2012, from http://annals.edu.sg/PDF/36VolNo9Sep2007/V36N9p751.pdf Mattheos, N., Nattestad, A., Falk-Nilsson, E., & Attstrom, R. (2004). The interactive examination: Assessing students' self-assessment ability. Medical Education, 38, 378389. doi: 10.1046/j.1365-2923.2004.01788.x McMillan, W. (2011). Making the most of teaching at the chairside. European Journal of Dental Education, 15, 63-68. doi:10.1111/j.1600-0579.2010.00638.x Office for Faculty Development, Faculty of Medicine, University of B.C. Retrieved May 8, 2012, from http://www.med.ubc.ca/faculty_staff/faculty_development.htm Ogden, P. E., Edwards, J., Howell, M., Via, R. M., & Song, J. (2008). The effect of two different faculty development interventions on third-year clerkship performance evaluations. Family Medicine, 40, 333-338.Retrieved May 8, 2012, from http://www.stfm.org/fmhub/fm2008/May/Paul333.pdf Paulsen, M.B. (2001a). After twelve years of teaching the college teaching course. In D. Lieberman & C. Wehlburg (Eds.), To improve the academy, (169–192). Bolton, MA: Anker. Paulsen, M.B. (2001b). The relation between research and the scholarship of teaching. In C. Kreber (Ed.), Revisiting scholarship: Perspectives on the scholarship of teaching: New directions for teaching and learning, No.86, (19–31). San Francisco: Jossey-Bass. Pratt, D. D. (2002). Good teaching: One size fits all? New Directions for Adult and Continuing Education, 93, 5-16. doi:10.1002/ace.45  104  Pratt, D.D., & Collins, J.B. (2011). The teaching perspectives inventory at 10 years and 100,000 respondents: Reliability and validity of a teacher self-report inventory. Adult Education Quarterly, 61, 358 – 375. doi:10.1177/0741713610392763 Ramani, S. (2003). Twelve tips to improve bedside teaching. Medical Teacher, 25, 112-115. doi:10.1080/0142159031000092463 Redwood, C., Winning, T., Lekkas, D., & Townsend, G. (2010). Improving clinical assessment: Evaluating students' ability to identify and apply clinical criteria. European Journal of Dental Education, 14, 136-144. doi:10.1111/j.16000579.2009.00606.x Rees, C., & Shepherd, M. (2005). Students’ and assessors’ attitudes towards students’ selfassessment of their personal and professional behaviours. Medical Education, 39, 3039. doi:10.1111/j.1365-2929.2004.02030.x Reilly, B. M. (2007). Inconvenient truths about effective clinical teaching. The Lancet (British Edition), 370, 705-711. doi: 10.1016/S0140-6736(07)61347-6 Richardson, J. (Producer). (2011). ICC assessment: Meaningful formative assessment. Podcast retrieved May 8, 2012, from: https://secure.dentistry.ubc.ca/intranet/cir/e2/e2-ICCAssessment2010/ICC%20Assessment%202010.html Roth, K. (2007). Dental education: A leadership challenge for dental educators and practitioners. Journal of Dental Education, 71, 983-987. Retrieved May 8, 2012, from http://www.jdentaled.org/content/71/8/983.full.pdf Saunders, M. (2011). Setting the scene: the four domains of evaluative practice in higher education. In M. Saunders, P. Trowler, & V. Bamber (Eds.), Reconceptualising  105  evaluation in higher education: The practice turn (193-199). Berkshire, England: McGraw-Hill. Scarbecz, M., Russell, C. K., Shreve, R. G., Robinson, M. M., & Scheid, C. R. (2011). Faculty development to improve teaching at a health sciences center: A needs assessment. Journal of Dental Education, 75, 145-159. Retrieved May 8, 2012, from http://www.jdentaled.org/content/75/2/145.full Shreeve, A., & Blythman, M. (2011). In M. Saunders, P. Trowler, & V. Bamber (Eds.), Reconceptualising evaluation in higher education: The practice turn (177-180). Berkshire, England: McGraw-Hill. Shulman, L.S. (1987). Knowledge and teaching. Harvard Educational Review, 57, 1–22. Retrieved May 8, 2012, from http://her.hepg.org.ezproxy.library.ubc.ca/content/j463w79r56455411/?p=2528ade84 3f6412a92488eed84f4b2e8&pi=0 Sork, T. J. (2000). Planning educational programs. In A. L. Wilson & E. R. Hayes (Eds.), Handbook of adult and continuing education (171-189). San Francisco, CA: JosseyBass. Sork, T. J. (2001). Needs assessment. In D. H. Poonwassie & A. Poonwassie (Eds.). Fundamentals of adult education: Issues and practices for lifelong learning (100115). Toronto, ON: Thompson Educational Publishing, Inc. Spallek, H., O’Donnell, J. A., & Young I. J. Y. (2010). Preparing faculty members for significant curricular revisions in a school of Dental Medicine. Journal of Dental Education, 74, 275-288. Retrieved May 8, 2012, from http://www.jdentaled.org/content/74/3/275.full  106  Stefani, L. (2008). Engaging our students in the learning process: Points for consideration. International Journal for the Scholarship of Teaching and Learning, 2(1), 1-6. Retrieved May 8, 2012, from http://www.georgiasouthern.edu/ijsotl Steinart, Y., Mann K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., & Prideaux, D. (2006). A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Medical Teacher, 28, 497-526. Straight, H. S. (2002). The difference between assessment and evaluation [PowerPoint slides]. Retrieved May 8, 2012, from http://www2.binghamton.edu/academics/provost/documents/assessment-evaluationstraight.ppt Subedi, B. S. (2004). Emerging trends of research on transfer of learning. International Education Journal, 5, 591-599. Retrieved May 8, 2012, from http://ehlt.flinders.edu.au/education/iej/articles/v5n4/subedi/paper.pdf Svinicki, M. D. (2004). Authentic assessment: Testing in reality. New Directions for Teaching and Learning, 100, 23-29. doi:10.1002/tl.167 Taleghani, M. (2004). Non-graded clinical evaluation of dental students in a competencybased education program. Journal of Dental Education, 68, 644-655. Retrieved May 8, 2012, from http://www.jdentaled.org/content/68/6/644.full University of British Columbia Faculty of Dentistry. (2011). Integrated Clinical Care (ICC) Syllabus. [Patient Care Performance Care Rubric]. Retrieved May 8, 2012, from https://secure.dentistry.ubc.ca/intranet/icc_manual/documents/2011/AppendixIIPatientCarePerformanceRubric.pdf  107  University of British Columbia Faculty of Dentistry. (2011). Pediatric Dentistry, DENT 440 2011 – 2012, UBC Children’s Dental Program Syllabus. [Global and enabling objectives] van der Vleuten, C. (2000). Validity of final examinations in undergraduate medical training. British Medical Journal, 11, 1217-1219. doi: 10.1136/bmj.321.7270.1217 Wass,V., van der Vleuten, C., Shatzer, J., & Jones, R. (2001). Assessment of clinical competence. Lancet, 357, 945-949. doi:10.1016/S0140-6736(00)04221-5 Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York, NY: Cambridge University Press. Werb, S. B., & Matear, D. W. (2004). Implementing evidence-based practice in undergraduate teaching clinics: A systematic review and recommendations. Journal of Dental Education, 68, 995-1003. Retrieved May 8, 2012, from http://www.jdentaled.org/content/68/9/995.full Wilson, A. L., & Cervero, R. M. (2006). The song remains the same: the selective tradition of technical rationality in adult education program planning theory. International Journal of Lifelong Education, 16(2), 84-108. doi:10.1080/0260137970160202 Woolley, S., Emanuel, C, & Koshy, B. (2009). A pilot study of the use and perceived utility of a scale to assess clinical dental teaching within a UK dental school restorative department. European Journal of Dental Education, 13, 73-79. doi: 10.1111/j.16000579.2008.00538.x Wong, T. J., & Hubball, H. (2011). Examination of curriculum reform in a four-year program of Pediatric Dentistry. Transformative Dialogues: Teaching & Learning Journal, 5(1), 1-12. Retrieved from http://web.ebscohost.com/ehost/detail?sid=9277929e-3e5f-  108  4929-8018b8d1515eb2b8%40sessionmgr14&vid=1&hid=15&bdata=JnNpdGU9ZWhvc3QtbGl 2ZQ%3d%3d#db=ehh&AN=70268605  109  Appendices Appendix A. UBC Dentistry Competencies for the New Practitioner  INTRODUCTION This document establishes the standards for new graduates of the UBC DMD program as they enter the dental profession. These competencies are intended to provide a framework for us to educate and train a biologically oriented, technically competent, socially sensitive practitioner of dental medicine who adheres to the highest standards of professional conduct and ethics, and who can function effectively as a member of the nation's health care delivery system. In order to meet that purpose, it is necessary to identify the knowledge and skills our graduates must possess to be able to promote the oral health of patients. These competencies identify and organize the knowledge and skills UBC graduates must acquire to become competent, curious, and caring dental practitioners who treat the whole patient. The value and usefulness of these competencies are directly related to two applications. The first application is defining the core content of our pre-doctoral curriculum. By stating publicly what graduates must know and be able to do after completing our program, we establish a basis for the content of all courses. This definition sets standards for identifying relevant content and provides guidance in making decisions related to our pedagogy and course sequencing. The degree to which our pre-doctoral curriculum is relevant, complete, educationally sound, and well organized will be a direct reflection of this document. A second application is related to the issue of outcomes assessment. The quality of any curriculum must be judged by its results. UBC has established the individual competencies which a student must demonstrate to qualify for graduation and entry into the profession. For these educational standards to be of real value, the Faculty must have methods in place to measure the degree to which a student has acquired and can demonstrate the competencies needed to care for patients. Competencies for the New Practitioner should be viewed as dynamic standards which must be responsive to any clear need for change. The competencies are intended to serve as the "blueprint" for our pre-doctoral curriculum. It is recognized and understood that this education plan will require regular review in the interest of continual improvement. Competency as an Educational Concept The term “competent” is defined by Chambers as "the behavior expected of the beginning practitioner. This behavior incorporates understanding, skill, and values in an integrated response to the full range of requirements presented in practice". There is a level of skill beyond competency known as proficiency which is acquired through advanced training, leading to specialization. Competencies are: 1. a typical part of the general practice of dentistry. 2. a combination of knowledge, attitude, and skills. 3. performed in a clinical context. 4. the continued performance at or above the defined standard of care. Development of the Competencies Competencies, as defined above, are acquired in a clinical setting or in the context of patient care. A competency is the demonstrated ability to perform a clinical task or to explain and discuss a clinical concept. Because competencies are outcomes of clinical training and experience clinicians, therefore, had the greatest responsibility in identifying the initial list of essential competencies. Approved by CTEC: 18 May 06  Page 1 of 6  110  Preamble to the ACFD “COMPETENCIES FOR A BEGINNING DENTAL PRACTITIONER IN CANADA” document (2005): A competent beginning dental practitioner in Canada must be able to provide oral health care for the benefit of individual patients and communities in a culturally sensitive manner. Competency assumes that all behaviours are supported by foundation knowledge and skills in biomedical, behavioural and clinical dental science and by professional behaviour. Beginning dental practitioners in Canada must be able to apply foundation knowledge and skills to justify their decisions and actions and to evaluate outcomes. Therefore, foundation knowledge, skills and professional behaviour are understood to be a part of every competency. Competency also assumes that all behaviours are performed to an acceptable level and that the practitioner can evaluate their quality and effectiveness. Competency cannot be achieved without the ability to self-evaluate. Moreover, there are no degrees of competence: a dentist is either competent or not competent. The competencies below refer to general dental practice and include the management of patients of all ages including those with special needs. It is assumed that all oral health care is provided in an ethical manner, in accordance with legal requirements at the national and provincial level. ORGANIZATION Domains The general organization of this document (and ultimately our curriculum) is structured from the general to the more specific. Six "Domains" have been identified. These represent broad categories of professional activity and responsibilities which occur in the general practice of dentistry. The concept of Domains is intended to encourage an eventual structure and process in the pre-doctoral curriculum that is more interdisciplinary and not departmental. In this document, the Domains are indicated I-VI (see detail following Organization section). Competency Statements Within each Domain, each Competency Statement is identified as relating to that Domain's activity or concern. Competence is the ability to perform or provide a particular, but complex, service or task. For example, "The new dentist must be able to perform an examination that collects biological, psychological, and social information needed to evaluate the medical and oral condition, for patients of all ages." The complexity of this service suggests that multiple and more specific abilities are required to support the performance of any Competency. The competency statements in this document come directly from the document entitled “ACFD Competencies For A Beginning Dental Practitioner In Canada”, approved in 2005. These statements of competency are also used by the Commission on Dental Accreditation of Canada and the National Dental Examining Board of Canada. A note about the word “manage” in Competency Statements: Managing the oral health care needs of a patient may include providing education, advice, treatment by the dentist, treatment by the dentist after consultation with another health care professional, or referral of a patient to another health care professional, monitoring treatment provided, and also may include providing no treatment or observation. “Manage” assumes the use of the least invasive therapy necessary to gain a successful outcome in accordance with patient wishes. Foundational Ability Foundational ability consists of knowledge, skills, and attitudes that are prerequisite for satisfactory attainment of Competencies. Foundational knowledge is the ability to acquire and use information and correctly answer specific questions when asked, for example, in a tutorial or in an examination. Foundational skill is the ability to produce acceptable results in standardized situations, for example, creating a satisfactory full crown preparation on an artificial tooth. Foundational attitudes are positive Approved by CTEC: 18 May 06  Page 2 of 6  111  intellectual and behavioral actions, such as scheduling appointments in the patient's best interest and not at the student's convenience. The basic medical and dental sciences, behavioral sciences, and clinical sciences all provide instruction at the foundational level. Lecture, small group, seminar, and laboratory instruction provide information and psychomotor experiences that enable students to acquire and demonstrate competence in the clinical setting or context. The inclusion of any specific foundational ability in the curriculum should be based on the direct support of one or more of the Competencies. These "Foundational Abilities" are defined by Objectives. THE DOMAINS AND RELATED COMPETENCIES I. PROFESSIONALISM The competent new practitioner provides skilled care based on contemporary knowledge and therapeutics and is capable of discerning and managing ethical issues and problems in dental practice. The dental profession holds the benefit of the patient as its primary goal (CDA Code of Ethics). The practice of dentistry occurs in a rapidly changing environment where benefits to the patient are influenced by ethical issues and problems created by regulatory actions, economics, social policy, cultural diversity and gender, and health care reform. New practitioners should participate in professional and personal development activities that enhance their contribution to their communities and equip them with the knowledge and skills to provide the highest standards of dental practice. These activities should provide a thorough knowledge of community resources and expectations that will bear upon their practice of dentistry, either in private practice or in a public dental program. The general practice of dentistry includes regular involvement with large and diverse amounts of information. Patient care, office management, and professional renewal are all highly dependent upon the capacity to obtain and process information, and the ability to make decisions or take action. The competent practitioner must be prepared to practice in this dynamic environment. Related ACFD Competency Statements 3. evaluate the scientific literature and justify management recommendations based on the level of evidence available. 4. communicate effectively with patients, parents or guardians, staff, peers, other health professionals and the public. 45. apply accepted principles of ethics and jurisprudence to maintain standards and advance knowledge and skills. 47. demonstrate professional behaviour that is ethical, supersedes self-interest, strives for excellence, is committed to continued professional development and is accountable to individual patients, society and the profession. II. PRACTICE ORGANIZATION The principal goal of the pre-doctoral program is to produce graduates who will function as general practitioners in the general practice of dentistry. In addition to clinical knowledge and skills, the general dental practitioner is also required to manage a sound business operation which facilitates the delivery of quality oral health care to patients. In order to manage a general practice, the dental graduate must be able to establish a professional practice by developing practice goals and plans; implement effective office systems; make sound business decisions; manage the business aspects of practice; evaluate outcomes; manage personnel; manage patient care; and understand the legal ramifications of patient care. Approved by CTEC: 18 May 06  Page 3 of 6  112  Related ACFD Competency Statements 27. recognize and institute procedures to minimize occupational hazards related to the practice of dentistry. 46. apply basic principles of practice administration, financial and personnel management to a dental practice. III. ASSESSMENT OF THE PATIENT AND THE ORAL ENVIRONMENT Patients seek the care of a dentist to maintain a level of oral health which is comfortable, functional and esthetically acceptable to the patient, as well as for treatment of oral disease. In order to confirm or establish, and then maintain, the oral health of their patients, the general dentist must first be competent to evaluate the patient, diagnose existing conditions, and develop a treatment plan. Assessment must precede any treatment and enables the general dentist to provide appropriate primary oral health care. Related ACFD Competency Statements 5. identify the patient’s chief complaint/concern and obtain the associated history. 6. obtain and interpret a medical, dental and psychosocial history, including a review of systems as necessary, and evaluate physical or psychosocial conditions that may affect dental management. 7. maintain accurate and complete patient records in a confidential manner. 8. prevent the transmission of infectious diseases by following current infection control guidelines. 9. perform a clinical examination. 10. differentiate between normal and abnormal hard and soft tissues of the maxillofacial complex. 11. prescribe and obtain the required diagnostic tests, considering their risks and benefits. 12. perform a radiographic examination. 13. interpret the findings from a patient's history, clinical examination, radiographic examination and from other diagnostic tests and procedures. 14. recognize and manage the anxious or fearful dental patient. 15. recognize signs of abuse and/or neglect and make appropriate reports. 16. assess patient risk (including, but not limited to, diet and tobacco use) for oral disease or injuries. 17. develop a problem list and establish diagnoses. 18. determine the level of expertise required for treatment and formulate a written request for consultation and/or referral when appropriate. 19. develop treatment options based on the evaluation of all relevant data. 20. discuss the findings, diagnoses, etiology, risks, benefits and prognoses of the treatment options, with a view to patient participation in oral health management. 21. develop an appropriate comprehensive, prioritized and sequenced treatment plan. Approved by CTEC: 18 May 06  Page 4 of 6  113  22. present and discuss the sequence of treatment, estimated fees, payment arrangements, time requirements and the patient’s responsibilities for treatment. 23. obtain informed consent including the patient’s written acceptance of the treatment plan and any modifications. 24. modify the treatment plan as required during the course of treatment. IV. HEALTH PROMOTION The dental profession serves the community in both private and public practice settings. Public health is concerned with promoting health and preventing disease through organized community efforts, as well as education of individuals and family groups. These are important components of any interdisciplinary approach. Whether acting as the community advocate or serving as a resource or change agent, the dental professional should be competent to interact with others to promote activities that protect, restore and improve oral health and the quality of life. Related ACFD Competency Statements 1. recognize the determinants of oral health in individuals and populations and the role of dentists in health promotion, including the disadvantaged. 25. provide education regarding the risks and prevention of oral disease and injury to encourage the adoption of healthy behaviors. V. ESTABLISHMENT AND MAINTENANCE OF A HEALTHY ORAL ENVIRONMENT Treatment is based on patient assessment. Thus, where oral conditions are healthy and stable, the goals are disease prevention and health maintenance. Active oral disease requires management of risk factors and control of the disease processes. In order to maintain or establish a healthy oral environment, the general dentist must be competent in the provision of preventive, therapeutic and continued oral health care. Related ACFD Competency Statements 2. recognize the relationship between general health and oral health. 26. provide therapies for the prevention of oral disease and injury. 28. achieve local anesthesia for dental procedures and manage related complications. 29. determine the indications and contraindications for the use of drugs used in dental practice, their dosages and routes of administration and write prescriptions for drugs used in dentistry. 30. manage dental emergencies. 31. recognize and manage systemic emergencies which may occur in dental practice. 32. manage conditions and diseases of the periodontium, provide periodontal treatment when indicated and monitor treatment outcomes. 33. assess the risk, extent and activity of caries and recommend appropriate non-surgical and surgical therapy. 35. manage patients with orofacial pain and/or dysfunction.  Approved by CTEC: 18 May 06  Page 5 of 6  114  36. manage surgical procedures related to oral soft and hard tissues and their complications 37. manage trauma to the orofacial complex. 38. manage conditions and pathology of the pulp and provide endodontic treatment when indicated. 39. manage abnormalities of orofacial growth and development and treat minor orthodontic problems. VI. REHABILITATION OF FORM, FUNCTION AND ESTHETICS A desirable dentition is comfortable and effective in function, and socially pleasing in appearance. Dental disease, congenital deformity, pathosis or traumatic incidents may compromise any or all of these qualities to varying degrees. In order to rehabilitate a compromised dentition, the new dentist must be competent to provide treatment which restores form, function, and esthetics of defective and/or missing teeth for patients of all ages. Related ACFD Competency Statements 34. manage dental caries, tooth defects and esthetic problems and, when restoration is warranted, use techniques that conserve tooth structure and preserve pulp vitality to restore form and function. 40. recognize and manage functional and non-functional occlusion. 41. select and, where indicated, prescribe appropriate biomaterials for patient treatment. 42. manage partially and completely edentulous patients with prosthodontic needs including the provision of fixed, removable and implant prostheses. 43. make records required for use in the laboratory fabrication of dental prostheses and appliances.  44. design a dental prosthesis or appliance, write a laboratory prescription and evaluate laboratory products.  Approved by CTEC: 18 May 06  Page 6 of 6  115  Appendix B. UBC Children’s Dental Program Syllabus  Pediatric Dentistry, DENT 440 2011 – 2012 UBC Children’s Dental Program  Syllabus Module:  DENT 440/Pediatric Dentistry  Description:  This module provides senior students with experience providing clinical dental care to children  Faculty:  Dr. Rosamund Harrison, Module coordinator (Chair, Division of Pediatric Dentistry); Contact information: email rosha@dentistry.ubc.ca; phone 604-822-2094. Office hours by appointment. On sabbatical: January –December 2011 Dr. Karen Campbell, Acting module coordinator; email campbkar@dentistry.ubc.ca; phone 604-822-3001. Office hours by appointment. Dr. Tracy Wong (UBC Children’s Dental Program Patient Coordinator, Term 1 only): email: tjwong@dentistry.ubc.ca; pager 604-293-0615 Tuesday and Wednesday afternoons Term 1. Office hours by appointment. UBC, Term 1; T, W afternoons: Drs. Karen Campbell, Kavita Mathu-Muju, Reza Nouri, Anita Gartner, Azar Grakoui, Hasnain Dewji, Jong-Hyun Ban, John Hung, Peter Chan, Mark Casafrancisco, Elsa Hui-Derksen, Jennifer Yee, Louisa Leung, Christine Kim, Joy Richman, Randy Shew, Nancy Vertel*, Winnie Zhao*, Carter Ng* *Yr 2 Pediatric Dentistry Graduate students  Douglas College, Term 2; M, T, W, Th afternoons: Drs. Sam Cheung, Azar Grakoui, Lori Santos, Lisa Coveney, Geoff Grant, John Hung, Peter Chan, Mark Casafrancisco, Wendy Tang, Phoebe Tsang, Elsa Hui-Derksen, Jennifer Yee, Ella Choi*, Amin Salmasi*, Jonathan Woo*, Nancy Vertel**, Winnie Zhao**, Carter Ng**, *Yr 1 Pediatric Dentistry Graduate students (Mon) **Yr 2 Pediatric Dentistry Graduate students (Wed/Thurs)  Schedule:  UBC, Term 1: Tuesday or Wednesday afternoon (check your personal schedule), including treatment plan session on August 30 or 31 (no patients) Douglas College, Term 2: check your schedule.  Faculty competencies addressed:  See curriculum database  116  Learning Objectives: By the end of DENT 440 Pediatric Dentistry, the student will be able to: 1. Complete a comprehensive oral assessment of a child patient • Formulate specific questions and address issues regarding the chief concern, history of present illness, past medical and dental history, family and social history • Assess a child’s extra-oral and intra-oral structures and differentiate between normal and abnormal tissues • Perform a radiographic examination and interpret the findings • Recognize sins of abuse and/or neglect and make appropriate reports 2. Organize a treatment plan that will fulfill a child’s behavioural, preventive, restorative, and interceptive orthodontic needs • Analyze and interpret data and findings • Develop a problem list (diagnoses), a list of treatment options, and a comprehensive, prioritized, and sequenced treatment plan • Present the case and treatment outline to parents or caregiver and child, if appropriate 3. Develop evidence-based approaches to the management of caries in the pediatric patient • Assess caries risk • Recommend appropriate non-surgical and surgical management of caries • Counsel patients and parents about controlling tooth decay 4. Assess the pediatric patient and use appropriate behaviour management and effective communication strategies to make the dental experience positive for children 5. Manage the dental treatment needs and complete comprehensive dental treatment for child patients • Administer profound local anesthesia safely for dental procedures and manage related complications • Perform pediatric dental procedures competently, including at least one each of the following: multi-surface amalgam restoration, formocresol pulpotomy, stainless steel crown on a primary molar, and extraction • Explain how to assess and manage dental trauma • Self-assess accurately • Recognize limitations and make an appropriate referral for specialist care, e.g., pediatric dentist, orthodontist, oral surgeon 6. Demonstrate professionalism and ethical practice in patient care clinics and small group participation • Communicate effectively with patients, parents or guardians, clinical instructors, staff, other health professionals, and peers • Demonstrate preparation for and apply knowledge to clinic practice • Independently access, retrieve, and critically evaluate relevant information • Organize clinical work and work area • Practice standard infection control precautions • Keep complete and accurate records • Work effectively and independently in a timely fashion  117  Requirements for clinical activity: 1. Pre-requisites for clinical pediatric patient care: All students will have provided restorative care to an adult patient(s) before performing restorative procedures on a pediatric patient. 2. Attendance: Attendance at all scheduled UBC and Douglas College sessions is required to pass the course. Dr. Wong (Term 1 only), Dr. Harrison and the Office of the Associate Dean, Academic through the Student Services Office must approve all excused absences. For an excused absence at Douglas College, arrange for another classmate to cover your session. In the event of illness, immediately contact the Student Services Office. However, in addition you must contact o Term 1/UBC: email Dr. Wong tjwong@dentistry.ubc.ca regarding your absence from clinic o Term 2/Douglas College: leave a message at Douglas College (604-521-1391) and email rosha@dentistry.ubc.ca or call Dr. Harrison (604-822-2094) regarding your absence. A patient has been booked for you and alternative arrangements must be made. *NOTE: Departures from the above policy will result in failure of the course ‘Professionalism’ grade, and ultimately the course grade. 3. Productivity: the quality of the work that you perform is of greater importance than the quantity. However, a student who completes significantly fewer procedures than the class average will be considered to be performing below his or her peers and may not successfully complete the course. Talk to Drs. Wong, Campbell or Harrison if you have concerns about your productivity at UBC or Douglas College. If you “dismiss” a patient to private practice or have a patient “no show,” enter the appropriate procedure code. If you assist your classmates, which means staying for the entire session, make sure the code for assisting is entered. You will receive CPVs towards your clinical grade in Pediatric Dentistry for all procedures that you do at UBC and at Douglas College. Drs. Wong, Campbell and Harrison monitor the variety of procedures that you do throughout the academic year. After the mid-point of Term 2, if it appears that you have not had the opportunity to complete certain pediatric dentistry procedures e.g. one SSC, one pulpotomy, and one multi-surface restoration on a primary tooth, you will be asked and will be expected to do some extra sessions at Douglas College before the end of the academic year. 4. Clinical Skills grade: A number of components contribute to your clinical skills grade in Pediatric Dentistry. clinical grades: standardization amongst instructors is difficult to achieve in daily grading of clinical procedures. However, a student receiving comments from instructor(s) that suggest unsatisfactory clinical skills may not pass the course productivity (as above) documentation from instructors that collectively are evidence of unprofessional behaviour by a student may lead to failure in the course. attendance at Pediatric Dentistry clinics is required to pass the course. Rubric for student assessment At the end of the academic year, you will receive a final clinical grade. During the March examination period, you will write a final examination. A passing grade (≥ ≥ 60%) is required in daily clinical performance and in the final examination to successfully complete the pediatric dentistry module. Students must also get a P for Professionalism. Clinic grade 60% Final written examination score 40% *Professionalism P 100% 118  *To rectify a failing grade in Professionalism, the student will be required to submit a 1000 word essay on a designated topic in Pediatric Dentistry. The format and due date will be at the discretion of the course module co-ordinator. The written examination is based on clinical care of children. Students will answer specific questions about clinical management and may be asked to present a justified treatment plan for these cases. Your clinical grade is based on the same rubric as used for ICC clinical assessment: Sessional Clinical Assessment Clinical assessment in Pediatric Dentistry is performed sessionally using the Instructor Rating Sheet. See assessment rubric, Appendix II, of ICC manual. The axiUm swipe activates the sessional assessment process. Assessment is made in 4 broad Performance Criteria (professionalism, application of knowledge, clinical skills and organization) and consideration is also made for the difficulty of the procedure and degree of student independence. For professionalism information see Appendix I (a) and I (b) of ICC manual. Self Assessment When the axiUm swipe for a session is received, a student self assessment grade card will pop up. No self assessment is required for missed/cancelled appointments, assisting or attending clinic session swipes. No grade is applied to the student assessment. Instructor Assessment Once the Self Assessment has been swiped, an instructor grade card pops up. Typically the instructor will opt to complete it at a later time. The instructor evaluates the self assessment as part of their overall assessment. The 1, 2, 3, 4 instructor grades are converted to percentage grades as follows 1. Does not meet expectations 10% 2. Borderline 50% 3. Meets Expectations 70% 4. Exceeds Expectations 120% The percentage rating for the session will be averaged for the 4 broad performance criteria. To this rating a multiplier will be applied based on the difficulty and degree of student independence. Although individual sessions may end up with a grade higher than 100%, the maximum allowable summative grade for a term or the academic year is 100%. Extremely difficult 1.2 Independent Care 1.2 Some difficulty 1.1 Minimal Intervention 1.0 Routine 1.0 Moderate Intervention 0.8 Simple 0.9 Significant Intervention 0.6 *NOTE re: Douglas College: It is the student’s responsibility to confirm that all clinical grades are documented through axiUm by the attending instructor at the end of each session. Should axiUm access be unavailable, it remains the student responsibility to complete a hard copy version of the evaluation rubric and have it signed off by the attending instructor. Failure to do so will have a negative impact on the clinical mark, due to lack of documented grades. Student feedback on module and instructors 1) Throughout the module, students are encouraged to make suggestions about the module to Dr. Harrison or other module faculty. 2) Students will evaluate the course through web-based course assessments managed by Dentistry’s Curriculum and Teaching Effectiveness Committee (CTEC). 3) Instructors will be evaluated each year as per CTEC guidelines.  119  Appendix C. Patient Care Performance Rubric Patient Care Performance Criteria: For a student at this stage of development, please consider the following criteria in assessing this student’s performance for this session. No assessment is needed for assisting. For other instances when student contact was minimal you can omit fields or provide no assessment. NOTE: A digital signature (swipe) is still required regardless.  Professionalism - sensitivity to varying cultural, social and economic situations - ethical behaviour, proper clinic attire, dress code and grooming - teamwork, accepts feedback constructively - effective communication, verbally and written, with patients, peers and faculty - independent access, retrieval and evaluation of relevant information Application of Knowledge - answers questions relevant to planned and ongoing clinical activities - evidence of preparation prior to clinic session(s) - asks thoughtful and relevant questions - demonstrates accurate self-assessment - follows established clinical QA protocol Clinical Skills - refer to discipline-specific criteria where appropriate - quality of treatment process - quality of treatment results - quality of patient management - maintains balanced posture Organization, Time Management and Infection Control - starts on time - practices standard infection control precautions - work area clean, neat and well organized - achieves goals set at start of session - complete and accurate record keeping - finishes on time Degree of Difficulty Includes anatomic factors, extent of disease, and patient management factors Degree of Student Independence Includes verbal and/or “hands on” assistance. Instructors should demonstrate and assist as needed but intervention should decrease as the students gain experience. Students should feel safe to ask for advice and assistance at any time and instructors should offer frequent demonstration and formative feedback.  *Does Not Meet Expectations: Shows frequent lapses in these behaviours and/ or makes critical errors  Borderline: Shows occasional lapses in these behaviours, with no critical errors  Meets Expectations:  Simple  Routine  Some Difficulty  Extremely Difficult  Significant Intervention Instructor completed the treatment or provided significant hands on  Moderate Intervention More frequent demonstration or verbal direction needed  Minimal Intervention minimal hands on assistance needed, or verbal only  Independent Care No assistance required or indicated  Demonstrates most of these behaviours most of the time, with no critical errors  *Exceeds Expectations: Consistently demonstrates all of these behaviours  Adapted from Integrated Clinical Care Syllabus, Faculty of Dentistry, UBC (2011)  120  Appendix D. Interview Consent Form THE UNIVERSITY OF BRITISH COLUMBIA Department of Curriculum and Pedagogy Mailing address: Faculty of Education Scarfe Building 2125 Main Mall Vancouver, B.C. Canada V6T 1Z4 Tel: 604-822-9218 Fax: 604-822-4714 http://www.edcp.educ.ubc.ca  Interview Consent Form for Clinical Instructors Thank you for your interest in my study. Please read the following consent form before making a final decision as to whether to participate. Why you are eligible: You are eligible to participate in an interview as a Clinical Instructor at the Faculty of Dentistry, University of British Columbia (UBC). This interview will help to determine the nature and scope of assessment and evaluation practices conducted by clinical practitioner-instructors in the UBC Pediatric Dentistry. Researchers: My name is Tracy Wong. I am a Clinical Assistant Professor and Coordinator of the UBC Children’s Dental Program at the Faculty of Dentistry. Conducting this interview is part of my M.A. research thesis in Education. As a UBC graduate student, I am required to adhere to UBC ethical approval procedures and this letter serves to inform you of the data collection goals and procedures, and your rights as a research participant. Dr. Harry Hubball, Department of Curriculum and Pedagogy, Faculty of Education, UBC, is supervising this process. As the faculty member overseeing my thesis project, he is identified by UBC as the Principal Investigator of this research. He can be reached at tel: 604-822-9218, email: harry.hubball@ubc.ca. My contact information is email: tjwong@dentistry.ubc.ca. What you are expected to do/time required: An interview with Tracy Wong will take place at Nobel Biocare Oral Health Centre or Douglas College Dental Clinic at a time convenient to you and will take about thirty minutes. I will tape record the interview with your permission. Confidentiality issues: Please note your name and identity will not be revealed to the organization. There are limitations to confidentiality but the researchers ask that you not discuss the interview afterward. Risks/Benefits: Your participation in this study is entirely voluntary and you can refuse to participate or withdraw from the study at any time without jeopardy. There are no anticipated risks for participating in this interview. It is possible that talking about the topic could be an enjoyable experience for you. If you have any questions or desire further information with respect to this study, you may contact me, Tracy Wong, email tjwong@dentistry.ubc.ca or my supervising instructor, Dr. Harry Hubball. If you have any concerns about your treatment or rights as a research subject, you may contact the Research Subject Information Line in the UBC Office of Research Services at 604822-8598 or e-mail ORSIL@ors.ubc.ca. If you are willing to participate in the focus group please sign below. Your signature also indicates that you have received a copy of this consent form for your own records. _______________________________________________________________________ Subject Signature Date _______________________________________________________________________ Printed name of subject  121  Appendix E. Interview Questions Questionnaire—Assessment/Evaluation in UBC Pediatric Dentistry Clinics Clinical Practitioner-Instructor Date______________________ Demographic Information Academic Rank Tenured or Tenure Track____ Clinical Assistant Professor _____ Sessional Instructor _____ Graduate Student _____ Number of years of clinical teaching experience ____ Number of years of training in teaching_____ Program __________________________ Age (years) Gender  Under 35____ 35-49____ Over 50____ M____ F____  Assessment Practices 1. What are your expectations of a dental graduate ready to begin independent practice? 2. What informs the way you provide feedback to students? 3. What strategies have you used in assessing/evaluating students’ clinical performance in Pediatric Dentistry? 4. What do you find most useful/not useful in providing feedback to and assessing and evaluating students regarding their clinical performance in Pediatric Dentistry? Please explain. 5. What difficulties have you encountered in assessing/evaluating students’ performance? 6. What patient care performance criteria do you think are important for student learning? Please provide your rationale/comments. 7. How relevant to real-world practice are the patient care performance criteria/rubric? Faculty Development 8. What topics/formats would help you assess/evaluate students’ clinical performance? 9. What would support your development as a clinical instructor at UBC? Other 10. What other comments do you have?  122  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0055373/manifest

Comment

Related Items