UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Development of a model for assessing the quality of an oral health program in long-term care facilities Pruksapong, Matana 2008

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


24-ubc_2008_fall_pruksapong_matana.pdf [ 3.69MB ]
JSON: 24-1.0066557.json
JSON-LD: 24-1.0066557-ld.json
RDF/XML (Pretty): 24-1.0066557-rdf.xml
RDF/JSON: 24-1.0066557-rdf.json
Turtle: 24-1.0066557-turtle.txt
N-Triples: 24-1.0066557-rdf-ntriples.txt
Original Record: 24-1.0066557-source.json
Full Text

Full Text

 i      DEVELOPMENT OF A MODEL FOR ASSESSING THE QUALITY OF AN ORAL HEALTH PROGRAM IN LONG-TERM CARE FACILITIES   by  Matana Pruksapong  DDS, Chulalongkorn University, 2000      A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  DOCTOR OF PHILOSOPHY  in   THE FACULTY OF GRADUATE STUDIES  (Dental Science)     THE UNIVERSITY OF BRITISH COLUMBIA  (Vancouver)     August 2008   © Matana Pruksapong, 2008  ii ABTRACT   Background: There is little information on how the quality of oral health services in long-term care (LTC) facilities is conceptualized or assessed.  Objectives: This study aims to develop a model for assessing the quality of oral healthcare services in LTC facilities.  Methods: This study is divided into four main steps. Firstly, I examined literature for existing concepts relating to program evaluation and quality assessment in healthcare to build a theoretical framework appropriate to dental geriatrics. Secondly, I explored as an ethnographic case study a comprehensive oral healthcare program within a single administrative group of 5 LTC facilities in a large metropolis by interviewing 33 participants, including residents and their families, nursing staff, administrators and dental personnel. I also examined policy documents and made site visits to identify other attributes influencing the quality of the program. Thirdly, I drafted the assessment model combining a theoretical framework with empirical information from the case study. And lastly, I tested the feasibility and usability of the model in another dental geriatric program in northern British Columbia. I applied the assessment model by conducting 15 interviews with participants in the program, made site-visits to the 5 facilities, and reviewed documents on the development and operation of the program.   iii Results: A combination of theory-based evaluation and quality assurance provided six sequential and iterative steps for quality assessment of oral health services in LTC.  The empirical information supported the theoretical framework that a program of oral healthcare in a LTC context should be assessed for quality from multiple perspectives; it should be comprehensive; and it should include the three main attributes of quality - capacity, performance, and outcomes. Participants revealed 20 quality indicators along with suggested program objectives which encompass eight quality dimensions such as effectiveness, efficiency, and patient-centered.  Conclusion: The model provides a unique system for assessing the quality of dental services in LTC facilities that seems to meet the needs of dental and non-dental personnel in LTC.      iv TABLE OF CONTENTS  ABTRACT _________________________________________________________ii TABLE OF CONTENTS ______________________________________________iv LIST OF TABLES _________________________________________________ viii LIST OF FIGURES _________________________________________________ ix LIST OF ABBREVIATIONS____________________________________________x ACKNOWLEDGEMENTS ____________________________________________ xi DEDICATION_____________________________________________________ xiii  CHAPTER 1 INTRODUCTION _________________________________________1 1.0 OVERVIEW OF THE DOCTORAL THESIS ____________________________1 1.1 BACKGROUND__________________________________________________2 1.2 REVIEW OF THE LITERATURE ON ORAL HEALTHCARE IN LTC_________6 1.3 GAP OF KNOWLEDGE AND STATEMENT OF PROBLEM ______________10 1.4 GOALS AND OBJECTIVES _______________________________________11 1.5 RESEARCH QUESTION __________________________________________12 CHAPTER 2 METHODOLOGICAL CONSIDERATIONS AND RESEARCH DESIGN _________________________________________________________________13 2.0 INTRODUCTION ________________________________________________13 2.1 METHODOLOGICAL CONSIDERATIONS____________________________13 2.1.1 Philosophical Assumptions and Worldviews __________________________ 14 1) Critical Realism Perspective_________________________________________________ 14 2) Systems and Ecological Perspectives _________________________________________ 16 3) Population Health Perspective _______________________________________________ 17 4) Evidences for Best-practice Perspective _______________________________________ 17 2.1.2 Approaches to Inquiry _____________________________________________ 18 1) Theory-based Evaluation ___________________________________________________ 19 2) Ethnographic Case Study___________________________________________________ 19 2.2 RESEARCH DESIGN (DETAILED METHODS) ________________________21 2.2.1 Building a Theoretical Framework ___________________________________ 21 2.2.2 Describing the Program (Empirical Evidence)__________________________ 22 2.2.3 Building the Assessment Model _____________________________________ 30 2.2.4 Testing the Assessment Model ______________________________________ 30 2.3 CREDIBILITY AND RIGOR________________________________________31  v CHAPTER 3 THEORETICAL FRAMEWORK _____________________________34 3.0 INTRODUCTION ________________________________________________34 3.1 CONCEPTS ON QUALITY-OF-CARE AND HEALTH-PROGRAM EVALUATION _____________________________________________________34 3.2 BUILDING A THEORETICAL FRAMEWORK FROM A COMBINATION ____37 OF THE TWO CONCEPTS ___________________________________________37 3.2.1  Engagement _____________________________________________________ 38 3.2.2. Quality Beyond Effectiveness ______________________________________ 38 3.2.3. Evaluating Structure-Process-Outcome ______________________________ 38 3.2.4. Formative and Summative Evaluations from Multiple Sources ___________ 39 3.2.5. Knowledge Transfer with Good Program Description ___________________ 40 3.3 CONCLUSIONS_________________________________________________41 CHAPTER 4 PORTRAIT OF A COMPREHENSIVE ORAL HEALTH PROGRAM IN                       LTC FACILITIES ________________________________________43 4.0 INTRODUCTION ________________________________________________43 4.1 HISTORY, CONTEXT, AND GENERAL DESCRIPTION OF THE PROGRAM CASE STUDY _____________________________________________________43 4.2 KEY COMPONENTS OF THE PROGRAM____________________________47 4.3 THEMATIC ANALYSIS OF THE COMPLEXITY OF THE PROGRAM_______59 4.3.1 Things are Ever-changing:__________________________________________ 60 4.3.2 Conflicting Views _________________________________________________ 63 4.3.3 All Variables are Massively Entangled ________________________________ 68 4.4 LESSONS LEARNED AND POSSIBLE SOLUTIONS ___________________72 CHAPTER 5 DRAFTING A MODEL FOR ASSESSING QUALITY OF A                      COMPREHENSIVE ORAL HEALTH PROGRAM IN LONG-TERM                      CARE FACILITIES _______________________________________76 5.0 INTRODUCTION ________________________________________________76 5.1 WHAT INDICATES QUALITY OF THE PROGRAM?____________________77 5.2 WHAT IS THE THRESHOLD FOR ACCEPTABLE QUALITY? ____________81 5.3 WHAT ARE THE METHODS FOR ASSESSING QUALITY OF THE PROGRAM? _________________________________________________________93    vi CHAPTER 6 TESTING A MODEL FOR ASSESSING QUALITY OF DENTAL                      GERIATRIC PROGRAMS ________________________________102 6.0 INTRODUCTION _______________________________________________102 6.1 THE APPLICATION OF THE MODEL: ______________________________103 6.1.1 State General Purpose of Assessment and other Preparations __________ 103 6.1.2 Describe the Program_____________________________________________ 104 6.1.3 Form Specific Assessment Questions _______________________________ 104 6.1.4 Collect Information Guided by the Quality Indicators___________________ 105 6.1.5 Interpret and Report Results _______________________________________ 106 6.2 TESTING THE MODEL FOR ITS FEASIBILITY AND USABILITY ________108 6.2.1 Feasibility of the Model in a “Northern” Context_______________________ 109 6.2.2 Usability of the Model in a “Northern” Context ________________________ 112 6.3 MODIFICATION OF THE MODEL__________________________________114 CHAPTER 7 DISCUSSION AND CONCLUSION _________________________119 7.0 INTRODUCTION _______________________________________________119 7.1 SUMMARY____________________________________________________119 7.1.1 Statement of Problem_____________________________________________ 119 7.1.2 Methods ________________________________________________________ 120 7.1.3 Findings________________________________________________________ 121 7.2 FREQUENTLY ASKED QUESTIONS_______________________________123 7.2.1 What Do You Mean by Oral Health Care?_____________________________ 123 7.2.2 What Do You Mean by Quality of an Oral Healthcare? __________________ 123 7.2.3 Is this Model Only Good for a Qualitative Evaluation?   _________________ 124 7.2.4 Do I Have to Always Conduct a Comprehensive Assessment?___________ 124 7.2.5 What is the Evaluation Standard? What is Appropriate Care? ___________ 125 7.2.6 Is Anybody Else Using a Similar Assessment Approach? _______________ 126 7.3 STRENGTHS OF THE STUDY ____________________________________128 7.4 LIMITATIONS OF THE STUDY____________________________________131 7.5 IMPLICATIONS ________________________________________________132 7.6 DIRECTIONS FOR FURTHER RESEARCH__________________________133 7.7 CONCLUDING REMARKS _______________________________________134  REFERENCES____________________________________________________135  vii APPENDICES Appendix A: Certificate of Research Ethics Board Approval _____________142 Appendix B: Letter of Initial Contact _________________________________143 Appendix C: Interview and Observation Guides________________________144 C1. Interview Guide (Care recipients) ____________________________________ 144 C2. Interview Guide (Care Providers)_____________________________________ 145 C3. Interview Guide (Administrators)_____________________________________ 146 Appendix D: Observation Field Note and Protocol______________________147 D1. Observation/Field-note Protocol _____________________________________ 147 D2. Example of a Field-note:____________________________________________ 148 Appendix E: Consent Forms________________________________________150 Appendix F: Adult Care Regulations _________________________________152 Appendix G: Caries Management Program____________________________153 Appendix H: UBC/PHC Daily Mouth Care Protocol. _____________________154 Appendix I: Questionnaires for Satisfaction Surveys ___________________157 I1. Satisfaction Survey with Families of the Residents. ______________________ 157 I2. Satisfaction Survey with Nursing Staff_________________________________ 159 Appendix J: Mouth-care Protocol Audits _____________________________160 Appendix K Letter of Introduction to the Northern Program______________161 K1. Email of Initial Contact to the Northern Program________________________ 161 K2. List of Potential Interview Questions and a Summary Table Displaying the Program Components of Interest Given to the Northern Program _____________ 162 Appendix L: Evaluation Report of the Northern Program ________________164 Appendix M: Compared Data Between UBC and the Northern Programs ___194 Appendix M: Compared Data Between UBC and the Northern Programs ___195 Appendix N: The Model Worksheet __________________________________196 Appendix O: Notes on Publication and Co-authorship __________________203  viii LIST OF TABLES  Table 2.1 Matrix of predetermined topics for framework analysis________________27 Table 4.1 Characteristics of the five long-term care facilities included in my case                 study _________________________________________________________46 Table 4.2 Structural features of the GDP ____________________________________52 Table 4.3 Procedural features of the GDP ___________________________________56 Table 4.4 Outcome features of the GDP_____________________________________58 Table 4.5 History of change in provincial healthcare system during 2002-2006 ____60 Table 5.1 Capacity indicators and examples of standard of acceptability according to                 the case study _________________________________________________85 Table 5.2 Performance indicators and examples of standard of acceptability                 according  to the case study______________________________________87 Table 5.3 Outcome indicators and examples of standard of acceptability according to                 the case study _________________________________________________91 Table 6.1 Definition of the quality dimensions advocated by the Canadian Council on                 Health Services Accreditation (CCHSA)___________________________ 115    ix LIST OF FIGURES  Figure 2. 1 A Venn diagram illustrating the methodological considerations that                    influence the research design ____________________________________13 Figure 3.1 Theoretical frameworks for guiding the design of assessment model____42 Figure 4.1 A simplified program model of the GDP based on structure-process-                   outcome framework _____________________________________________47 Figure 4.2 An inclusive organizational chart of the GDP ________________________49 Figure 4.3 Three emerging themes describing complexity of the program _________59 Figure 4.4 Web of variables showing what make mouth-care work or does not work                   ______________________________________________________________69 Figure 5.1 Categories and labels of the new classification of quality indicators ____79 Figure 5.2 Categories and subcategories of quality indictors ___________________81 Figure 5.3 Incrementally increase the level of standard for each assessment cycle                  ______________________________________________________________84 Figure 5.4 Theoretical and modified versions of steps of quality assessment _____98 Figure 5.5 A PDQI (Program Description and Quality Indicators) model for assessing quality of a comprehensive oral health program in LTC facility _____________________________________________________________101 Figure 6.1 A full cycle of quality assessment of an oral health program in LTC facility                  _____________________________________________________________118   x LIST OF ABBREVIATIONS   BC   British Columbia LTC   Long-term care MeSH   Medical Subject Heading UBC   University of British Columbia GDP   Geriatric Dental Program QA   Quality Assurance IOM The (American) Institute of Medicine CQI Continuous Quality Improvement TQM Total Quality Management PROCEDE Predisposing, Reinforcing, and Enabling Factors in Educational Diagnosis and Evaluation PROCEED Policy, Regulatory, and Organizational Constructs in Education and Environmental Development P/P   PRECEDE/PROCEED (Model) SPO   Structure-Process-Outcome CODE   Clinical Oral Disorders in Elders PHC   Providence Health Care ESC    Executive Steering Committee CDA    Certified Dental Assistant RCA   Registered Care Aide RN   Registered Nurse LPN   Licensed Practice Nurse RDH   Registered Dental Hygienist GPR   General Practice Resident (Dentist) MD   Medical Doctor HEU   Hospital Employees’ Union ELDERS  Elders’ Links to Dental Education, Research, and Services PD    Program Description QI   Quality Indicator NQF   The National Quality Forum AHRQ   The Agency for Healthcare Research and Quality IHI   Institute of Health SWOT            Strength, Weakness, Opportunity, Threat CCHSA   The Canadian Council on Health Services Accreditation GODP  Geriatric Outreach Dental Program  xi ACKNOWLEDGEMENTS  I would like to express my deep gratitude to my supervisor, Dr. Michael MacEntee for his wit and wisdom. Thank you for: teaching me how to critically look into the fundamentals; providing me intellectual and practical advices; and for your patience teaching me how to write succinctly. Michael is my role model not just academic-wise but also on how to live one's life in general. Simply put, I could not ask for a better mentor. Also, my sincere gratitude goes to all the rest of my advisory committee members for all the great discussion; for your constructive comments; and for teaching and leading by example. Thank-you to Dr. Lynn Beattie for teaching me how to make abstract ideas concrete and practical. Thank-you to Dr. Arminée Kazanjian for teaching me how to think systematically, to get things organized, and most of all to know the difference between "doing the right things" and "doing things right". Thank-you to Dr. JoAnn Perry for teaching me to be kind, gentle, and see things from the perspectives of the patients and their families. Thank-you to Dr. Chris Wyatt for teaching me nuts and bolts of dental geriatric programs and for giving me opportunities to work on several sideline projects relating to the thesis. And particular thanks go to my external examiner, Dr. Paul Allison, for your time reviewing the thesis. I would also like to thank Judy, Kelly, and Phil of the ELDERS research group for keeping our group run smoothly. Thanks to my friends, Mario, Shiva, and Eduardo for all the serious discussions about life and the silly fun time we had together. Thanks to all the rest of graduate students and academic staff of the community-based research cluster for sharing ideas and experiences about something larger than working in our typical cubicle.  xii I would like to say a big thank-you and express much respect to all my research participants: all the people who care for frail elders, and the elders themselves. In addition, particular thanks go to UBC library and the IT support for all the good books available; interlibrary loan; and most of all, the online access to all the references anyone could ask for. Thank-you to UBC: for the many meaningful public lectures and activities; for being so open to let me take courses across campus; and giving me opportunity to experience my first interdisciplinary course (IHHS). Also, I would like to thank all my coursework instructors and classmates. I wouldn't have learned this much without the great educational infrastructure. In addition, much appreciation goes to the Dr. Wah Leung Endowment Fund for supporting my trip to the north to test the applicability of the model developed. Finally, I can't finish this acknowledgement without saying thank-you to my family and extended family for their love and support. Especially, I would like to thank my parents for everything and I mean "everything". And you too, Jes - Thank you!      xiii DEDICATION           To our elders Chapter 1  1 CHAPTER 1 INTRODUCTION  1.0 OVERVIEW OF THE DOCTORAL THESIS My doctoral thesis aims to develop a model for assessing the quality of a comprehensive oral health program in long-term care (LTC) facilities. This model of assessment corresponds to a full cycle of program development and therefore, can be used to guide the planning and implementation of the program, as well as to evaluate the quality of the services established. Accordingly, the model provides ways of accounting and prompting the improvement for the quality of oral health care in LTC. The intended audiences of this thesis are health professionals and dental personnel working in LTC settings, researchers, and evaluation consultants who would like to plan and evaluate oral health services in LTC facilities. Also, this thesis is useful for decision-makers who would like to develop policies to improve oral healthcare for the elderly residents of LTC facilities.  The thesis is structured according to the traditional format approved by the Faculty of Graduate Studies at the University of British Columbia (UBC). This introductory chapter begins with the background information and literature review that point out to the need for investigation. This chapter ends with research questions and objectives. Chapter 2 includes a description of detailed research design and methodological considerations that guide this thesis. Chapters 3 to 6 present results produced from the four steps of model development, as following: Chapter 1  2 First: Chapter 3 presents a theoretical framework for assessing the program developed from a combination of literatures on general health program evaluation and quality assessment of healthcare. Second: Chapter 4, based on an ethnographic case study of UBC Geriatric Dentistry Program, provides specific and empirical information about the components of a comprehensive oral health program in LTC facilities and its complexity that affect the design of assessment process. Third: Chapter 5 then combines information from theories of assessment (chapter 3) with empirical information specific to dentistry (chapter 4): resulting in a draft of the model for assessing the quality of the program. Forth: Chapter 6 presents the feasibility and usability testing of the model developed after applying the process to another oral health program in Northern BC. Subsequently, chapter 7 summarizes the thesis, discusses frequently asked questions, the implications and limitations of the thesis, as well as directions for future research.  1.1 BACKGROUND Oral health is an integral part of general health and quality of life (1). With rapid growth of the aging population around the world (2) and reports on high prevalence of poor oral health among these elders (3-7), effective oral health interventions are needed now more than ever (8). Still we are not at all sure about how to provide oral healthcare for this elderly population who, compared with other Chapter 1  3 age groups, have different needs, demands, and contextual factors that influence the type and quality of oral healthcare (9). As a dentist I asked, “What are the processes of developing an effective oral health program for the elderly population?” and “How does such program work?” These questions are very broad, but it is why I came to UBC from Thailand. Initially, I focused my research on the planning and the operation of the program, to address, for example, the physical resources needed to get the program going, the financial arrangement, and the appropriate methods of service delivery. Knowing that providing oral health care for this population faces a lot of barriers, I was looking for “solutions” in the forms of a “model of care delivery” or “evidence-based guidelines for practice” until I realised that there might not be a definite answer. I first intended to limit the extent of my dissertation to building a model of “oral health care delivery” in LTC1 along with lessons learned and recommendations gathered from a critical review of the literature and a case study of UBC Geriatric Dentistry Program. Fortunately, experience observing and learning about the UBC dental geriatric program during the first year at UBC reminded me that a full cycle of the development of any health program includes evaluation. In fact, a crucial part of the program development is evaluation (10). I, however, prefer the term “assessment” to “evaluation” because “evaluation” tends to give its contracted meaning of judging if the program should be continued or terminated when there is much broader definition to such term (11). Program evaluation, as defined by Patton (11), is the systematic collection of information about the activities, characteristics, and/or  1 I am interested in oral healthcare for all subgroups of older adults but the population of interest for my thesis is the elderly residents of LTC facilities. Chapter 1  4 outcomes of programs to make judgment or make contribution to the improvement of the program. Therefore, asking just how an oral health care program should be established and operated is not enough. My fundamental research question became: “How should an oral health care program be established, operated AND assessed?” Moreover, I noticed that at least in Canada, the public and health authorities asked for accountability, especially with this vulnerable population where special medical and ethical considerations complicate the care (9,12). The question of how to assess the quality of care inevitably surfaced. This point was confirmed at a meeting of the UBC ELDERS in 20032 where facility administrators and nursing staff requested performance indicators for daily mouth-care. In addition, the characteristics of the elderly population and organizational structure and culture of healthcare system in Thailand are different than in Canada. Instead of seeking for a definitive model of care delivery, it is more practical to aim for developing a tool that would help practitioners and policy makers diagnose the local circumstances to create interventions; as well as to help assess the impact of care and account for their programs. This approach increases the likelihood of transferability of knowledge to the next generation of program developers including myself when I go back to Thailand. The following review of the dental and nursing literature shows how knowledge has evolved on the topic of oral healthcare for the elderly population in  2 The purpose of the meeting was for the ELDERS research group at the UBC, Faculty of Dentistry to disseminate their research findings conducted in over 20 LTC facilities in Vancouver. At the same time, the research group turned to the administrators and staff of the facilities to provide information on oral healthcare issues that they feel needs attention. Chapter 1  5 general and for the residents of LTC facilities in particular. The review examined the gaps in knowledge, and consequently directed the studies in this thesis. Chapter 1  6 1.2 REVIEW OF THE LITERATURE ON ORAL HEALTHCARE IN LTC  Knowledge about oral healthcare in LTC facilities has evolved in three phases. Phase I: Prevalence Reports of Poor Oral Health and the Need for Care. In response to the demographic trends, there have been multiple epidemiological studies in the late 1980’s and early 1990’s demonstrating a high prevalence of oral disease among elderly residents of LTC facilities (e.g. 13,14). The increasing evidence of connections between oral health, general health and quality of life (1,15-17) indicate the need for improved quality of oral health care in old age and the need to improve access to oral health services for dependent elders (14). Essentially, oral health care in LTC facilities is limited largely to emergency dental treatment (18). Phase II: Confirmatory Reports on Clinical Needs, Inequitable Access to Care, and Program Initiatives. Despite the desires expressed by the residents, facility administrators, and a few government agencies, there has been little improvement in the oral care of frail elders over the last few decades. A MEDLINE search - based on the subject headings (MeSH): “(program development, or health planning, or delivery of health care, or planning techniques, or program evaluation, or health service) and (dental care for aged or geriatric dentistry)” - displayed 104 articles with the first publication in 1991. I categorized the publications as follows: Chapter 1  7 • Surveys of oral health status indicating clinical needs, inequities in access to care, and documentation of the social impact of the inequity (7,19,20) ; • Investigations on the value of oral health and its influence on general health and quality of life  (1,15-17); • Descriptions of dental practices in the context of LTC outlining the activities of dental and nursing personnel along with details of equipment needed and methods used to deliver oral healthcare (21- 25). Phase III: Outcome Research, Evaluation, and Challenges Providing Care Since the mid 1990’s, dental researchers have been developing many physical and psychosocial measures of oral health status and oral health related quality of life specific to the elderly population (26-28). However, the desired outcomes of oral health care for frail elders are unclear and data are difficult to collect (29). Most of the evaluation of oral healthcare in LTC facilities used clinical trials or tests of association to assess and document the performance of specific programs or treatments. Clinical trials have been designed to test the efficacy of mouth-care products and procedures, or educational programs for nurses and care- aides about oral healthcare (30,31). Typically, tests of efficacy operate under optimal conditions rather than under the more challenging and unstable environments of residential care. Consequently it is difficult to transfer the test results directly to the needs of frail elders. Indeed, the effectiveness of an educational program about oral healthcare was disappointingly poor when the program was offered intentionally with Chapter 1  8 minimal interference to the usual routines and political realities of the facilities (18). Moreover, most of the educational interventions failed to render a sustained benefit beyond a year to the knowledge and performance of the nursing staff, or to the health status and quality of life of the residents (32-35). In addition, studies evaluating combined strategies of care are scarce. Program evaluations of oral healthcare usually assess the impact on dental diseases, cost of care, and utilization/participation rates among the residents (36- 41). Similar to studies evaluating a single clinical or educational intervention, outcomes such as satisfaction and quality of life have been used rarely to indicate success of a multi-intervention program. Overall, it seems that measures of oral health-related quality of life have been used primarily to highlight a need for care rather than monitor the impact of care (26,42).  Also, there was little or no effort to follow the impact of the oral health program in LTC over a prolonged period (36-41). Obviously, there are limitations to the duration of longitudinal studies involving frail elders, and practical difficulties in clinical or educational trials because of the complicated environment of residential care. It has been a major challenge, for example, to find a control group or to sustain adequate sample sizes when there is a high turnover of residents and caregivers (43). It is not surprising, therefore, that most studies were limited to a “one-group-before-and-after” design, and were unable to report defensible conclusions (44,45). Research on oral health services has moved recently from focused measurements of clinical variables to broader explorations encompassing the whole processes of care with an appreciation for institutional structure and culture that Chapter 1  9 includes excessive workloads, lines of authority, and expectations of caregivers (18,46). Apparently, the success of oral health services in the facilities depends at least as much on the organisational culture, philosophical values and communication patterns as on the provision of “state-of-the-art” treatments and resources (18). Chapter 1  10 1.3 GAP OF KNOWLEDGE AND STATEMENT OF PROBLEM  In all, considerable progress has been made in identifying the variables that influence oral health outcomes and barriers to care. There is reasonable evidence that mouth-care products and techniques are efficacious (47), but there is much less information on the effectiveness of care strategies within the daily operations of residential care, and even less information on the quality of the programs as a whole. Since oral health is part of general health, there is increasing demand for quality assurance within dentistry as there is in medicine, and the task of quality assessment becomes an important concern and need, especially for services offered in LTC facilities (48-51). The assessment based on concepts of quality of care within dentistry as a whole has occurred mostly in educational institutions, hospitals, and private practices (48,52). To date, there have been only limited applications of quality assessment to oral health as part of residential care, and mostly they addressed the quality assessment of oral hygiene care with no follow-up plan of continual improvement (53).  Essentially, we are not at all sure about how to investigate the impact of oral healthcare in LTC facilities and, as a consequence, we are unsure about how to provide this service. What we need now is an assessment that not only judges if the intervention succeeds or fails, but an assessment that also helps produce the knowledge about how the program works, diagnoses, and monitors various aspects of quality of care from the people who are intimately and regularly involved with it. Consequently, as part of this doctoral thesis, I have developed an assessment model for creating, monitoring, and evaluating the quality of an oral health care Chapter 1  11 model for frail elders in LTC facilities so that program developers including myself can apply it to their own local conditions and development needs.  1.4 GOALS AND OBJECTIVES The overall goal of my doctoral thesis is to develop a generic model for assessing the development, the ongoing progress, and the impact of oral health services in LTC based on the combined concepts of quality of care and health program evaluation. The key feature of this study is to offer different groups of people involved in the program the opportunity to reflect on and give voice to their experiences, expectations, and concerns with oral health care in LTC facilities; to identify the essential components and functions of the program; and to formulate recommendations for the development of the model for quality assessment. Therefore, the two specific objectives of my dissertation are: 1) to portray how the program works (i.e. program’s parts, relationships, mechanism for change, and desired outcomes) and; 2)  to form a generic model for assessing quality guided by the information gathered as part of the preceding objective  Chapter 1  12 1.5 RESEARCH QUESTION My two major research questions corresponding with the research objectives are: 1) what is the program?: • what are the program components? • how does the multitude of factors within the program relate and interact, and in which context do they operate? 2) how can the program be assessed most effectively? • what indicates the quality of the program? • are there thresholds of acceptable quality? If yes, what are they? • how can the quality of the program be assessed? Chapter 2  13 CHAPTER 2 METHODOLOGICAL CONSIDERATIONS AND RESEARCH DESIGN  2.0 INTRODUCTION This chapter is divided into three parts. The first section presents methodological considerations relating to the design of this doctoral thesis. The second part elaborates on the detailed methods used and the third section includes discussion about credibility and rigor of the study. 2.1 METHODOLOGICAL CONSIDERATIONS As shown in Figure 2.1, research design is influenced by the interplay of at least three main topics of consideration: 1) problems and research goal; 2) philosophical assumptions and worldviews; and 3) availability of time and resources. Figure 2.1 A Venn diagram illustrating the methodological considerations that                    influence the research design.         Problem(s)  & Research goal Philosophical Assumptions & Worldviews Time & Resources Research Questions Research Design Approaches  to Inquiry Scope Chapter 2  14 My doctoral thesis addresses the problematic situation where there is no body of literature of reasonable size and quality about how a comprehensive oral health program operates in long-term care (LTC) facilities. As discussed in the introductory chapter, a major part of the problem stems from the lack of appropriate assessment. Therefore, research questions are exploratory; attempting to understand what a comprehensive oral health program in LTC is and how to track its progress and impact. This exploratory nature of the research goal and questions, in turn, suggests qualitative research inquiry. However, diverse qualitative approaches are available for selection. For example, I could have employed multiple rounds of Delphi group discussion to reach consensus among academic experts. I, however, decided to use different methods, which were shaped greatly by my personal philosophical assumptions and worldviews. The purpose of this section is to make explicit my preconceptions and interpretive lens underpinning this research project. 2.1.1 Philosophical Assumptions and Worldviews 1) Critical Realism Perspective I situate my philosophical position somewhere between a post-positivist and an interpretivist. I reject both the positivist’s oversimplified distinction between objectivity-subjectivity and the extreme interpretivist’s concept of subject-object dissolution3.  According to Searle (54), critical realism helps clear away the confusion about existence of reality and truth of knowledge by distinguishing between:  3 Suggesting there is no such thing as an objective knowledge; no epistemological privilege; no point in trying to establish "validity" of knowledge in any objective sense; and that an “objective reality” is nothing but the object of the intentional acts of human consciousness(54). Chapter 2  15  a) ontological objectivity: observer-independent matter or phenomenon such as tectonic plates, blood pressure, and death; b) ontological subjectivity: phenomena, such as pain, which is “real” but “Nobody else can feel the same pain you feel”. These include sensations, beliefs, feelings, emotions, and opinions. c) epistemic objectivity: the claim about phenomena that became fact because they can be tested whether they are true or false. Therefore, both ontological objectivity and ontological subjectivity can become epistemic objectivity. d) epistemic subjectivity: merely a matter of opinion that cannot become fact (e.g. Vanilla ice cream tastes better than Chocolate ice cream). With this view, I believe that it is still essential to strive for epistemic objectivity. However, I believe that objectivity cannot be achieved in any absolute sense, but can be achieved reasonably closely by conducting research in rigorous manner and always declaring observer/researcher’s own predisposition (i.e. what I am doing now). This understanding of epistemic objectivity is advocated by the critical realists as “modified objectivity” (55). I also adopt the view that knowledge is time and context dependent (56). In addition, I firmly believe in interdependence among factors influencing any phenomenon and that causes and effects are not always one directional or linear (57). The following sub-headings elaborate on such worldviews. Chapter 2  16 2) Systems and Ecological Perspectives  The ontological position of systems theorists and ecologists can be characterized as closer to the post-positivist than either logical-positivist or the post- modernist; while the epistemological assumption that knowledge is contextual brings it closer to post-positivism in general (58). The underlying philosophy of systems perspectives is to try to understand the complexities of the real world by viewing things as whole entities embedded in context of a larger whole (59). Ecological perspective focuses on the relationship between human behaviour and the environment (10). For example, any effort to improve health status and quality of life of a population must take into account the influence of physical and cultural environments of the ecosystem and its subsystems such as families, organizations, and communities (10). As a consequence, I agree with Green & Kreuter that to address those ecological systems in any program planning, implementation, and evaluation, “we must first be able to see them” (60); hence the first research question (see chapter 1 page 10). Note that I am a supporter of these two closely related perspectives (system and ecological perspectives) whether they are driven by either quantitative- computational simulation or qualitative research approaches. Both perspectives emphasize holism and dispute reductionism that fail to neither appreciate the multiple-directional interactions nor recognize that properties of emergent holistic systems are not reducible to those of their parts (10,11). These systems are embeded in and co-evolve with their environments (59,61). Nevertheless, the attempt to understand the whole (such as a health program as a whole) does not Chapter 2  17 mean that the investigators never become involved in component analysis or in looking at particular variables and parts of the phenomenon of interest. Rather it means that the researchers “consciously work back and forth between parts and whole, separate variables and complex interwoven constellations of variables in a sorting out then putting back together process while staying true to a strategy that emphasizes the importance of a holistic picture of the program” (11). Discovery and verification mean moving back and forth between induction and deduction. In essence, both techniques are complementary (11). 3) Population Health Perspective Population health programs aims to reduce health inequities and improve the health and quality of life of a population identified by age, gender, socio-economic status, places they live and work, or seek health services (11). It is closely related to systems and ecological perspectives in that it broadens perspective beyond one-on- one biophysical aspect of medical care to include the connections between people and their social determinants of health (10). From this perspective, health has been defined not simply as a state of disease-free but as "the capacity of people to adapt to, respond to, or control life's challenges and changes" (10). Health is not an ultimate value in itself but instrumental to it (10). 4) Evidences for Best-practice Perspective  Because knowledge is contextual, my approach to study and develop health programs is influenced greatly by the concept of “best process as best practice” (10,62). According to Green (10,62), research can promise a generalizable process for planning as part of a “best process of planning”, but research cannot necessarily Chapter 2  18 produce the best generalizable intervention. This promise provides more meaningful interventions in response to local needs; and match better with community capacity (10,62). It is not surprising why people are naturally finding clinical practice guidelines or other forms of best practices frustrating (63). Patton (11) and Green (10,62) advocate that the best practice is to emphasize evaluation; and the best evaluation is self-monitoring by those closest to the practice; those who can adjust the practice according to the results.  Moreover, rather than viewing the distant evaluation of another program as a definitive role model, local workers should view such evaluation results as suggestions that need appropriate adaptation to the local situation (10,62).  With this perspective, I decided to create an assessment model that would require the people who are involved in the program to engage throughout the process of assessment. Also, my assessment model acts as a map guiding through quality-monitoring process: thus it is not a definite recipe. 2.1.2 Approaches to Inquiry The essence of my doctoral thesis is designing an evaluation approach appropriate to the complexity of oral health programs in LTC facilities. To finish this task, I selected theory-based evaluation approach rather than method-based evaluation approach. Researchers who use method-based approach are usually attached to methodological orthodoxy: choosing among available evaluation designs and instruments, such as randomised-controlled trial, before-after experiments, without attempting to first understand the characteristics of the programs and how Chapter 2  19 they work. In relation to this view, I also used ethnographic case study to help identify such program theories in a real-world setting. 1) Theory-based Evaluation Theory in this context is not a grand theory as understood in social science but it is a theory that “represents a plausible and sensible model of how the program [or an intervention] is supposed to work” (60). Designing evaluation using theory- based approach involves two steps. Firstly, a program theory is illustrated by identifying the key program components, desired program outcomes, and how the program components would lead to the desired outcomes. Secondly, the descriptive program theory becomes the framework to guide the development, implementation, and interpretation of the evaluation (10,11,60). These two steps fits perfectly with my research questions. 2) Ethnographic Case Study Ethnography literally means a portrait (graphein) of a group of people (ethnos) (64). It is both a process and a product of the investigation. Ethnography as a process focuses on exploring the culture of the group that is “a collection of behaviour patterns and beliefs that constitute standards for deciding what is, what can be, what to do about it, and how to go about doing things” (65). Subsequently, ethnography as a product is the description of the group of people under study. Ethnographic as a product comes in many forms such as narratives, documentary, and poems (65). For this thesis, I present a case study by text and diagrammatic visualisation. Chapter 2  20 I chose ethnographic case study methods because it is responsive for exploring systematically the operation of a complicated program, such as a comprehensive oral healthcare program. It emphasises the importance of context in understanding how the program works and takes into account that the program is influenced by the beliefs, behaviours and culture of the participants (11). A close contact with persons intimately involved with its initiation and implementation, along with access to the administrative documents, offers the possibility of an extensive case study. In Ethnography, observations take place in real-world settings and people are conversationally interviewed with open-ended questions in places and under conditions that are comfortable for and familiar to them (65). Through the range of strategies it adopts, ethnography allows comparison between what people say they do or what is written in official documents (espoused theories) and what they do (theories-in-use) (60). In addition, I believe that, because in practice programs are often changing, ethnographic case study can document actual operation and impact of the program over a long period of time (prolonged engagement). Lastly, this approach permits the informants to describe what is meaningful and important to evaluate.  Chapter 2  21 2.2 RESEARCH DESIGN (DETAILED METHODS) This study is divided into four main steps. Firstly, I examined the literature for existing concepts and theories relating to program evaluation and quality assessment in health care to build a theoretical framework most closely related to oral health services in LTC. Secondly, I studied one comprehensive oral health care program in-depth by means of an ethnographic case study. This step provided intimate insights into operational details, by describing a case study of a comprehensive oral health program in LTC facilities in the form of narrative ethnography and a program logic model4.  Thirdly, I drafted the assessment model, from the synthesis of both the theoretical framework and empirical evidence obtained from an in-depth case study. The assessment model consists of 1) steps of quality monitoring; 2) a visual guide for program description and quality indication; and 3) potential standards of care for assessing the quality of the program. Lastly, I conducted a feasibility and usability of the drafted model by applying it to another oral health program outside the context of where the model was originally developed. 2.2.1 Building a Theoretical Framework I selected PROCEDE/PROCEED (P/P) model (10) and Structure-Process- Outcome (SPO) model (11) as the theoretical framework for drafting my assessment model because their underlying principles are in line with my conceptual framework or worldviews. I drew on both models because they complement one another. P/P model provides a comprehensive evaluation framework with emphasis on situating  4 This model serves as a theory in form of a diagram of the assumed causal links among program context, inputs, activities, and outcomes. Chapter 2  22 evaluation in the planning process. Its classification of predisposing, reinforcing, and enabling factors gives me a language and theory of how to code the many factors involved. However, the model does not provide a framework on the broader dimension of quality of care. SPO model and its derivatives discuss in length dimensions such as equity and acceptability of care. Similarly, P/P model speaks about the components of health programs in general terms.  These components consist of resources, organization, and policy.  The SPO model, however, specifically identifies the attributes of the program that can be placed into structure, process, and outcome. In chapter 3, I present an evolution of theories of general health program evaluation and quality assessment of healthcare and also those specific to health programs in LTC. 2.2.2 Describing the Program (Empirical Evidence) 1) Selection of Case Study UBC, Providence Health Care, and the regional health authority created together in 2002, the UBC/PHC Geriatric Dentistry Program (GDP) to attend to the dental needs of frail elders in all 6 of the facilities5 within the Providence Group. Recently, the service has been extended to other facilities outside the Group, so that today it attends to over 1,500 LTC recipients.  The GDP is selected as a case study not only because it is readily accessable but also it appears to be the most comprehensive programs available anywhere. Every resident on admission to one of the facilities receives a structured oral screening examination followed by a comprehensive diagnostic and treatment service. They also receive daily oral  5 One facility was later closed down in 2003. Chapter 2  23 hygiene assistance as required, and each facility has an oral health-related educational program for the residents and the staff. Consequently, there is a unique opportunity to identify systematically the components of the assessment model including the beliefs, behaviours and cultural attributes of the many participants. Please note that the focus of this research is not to evaluate the effectiveness of this GDP per se, but rather to identify all of its key components and relationships to build a program theory of how this type of program is supposed to work.  This, in turn, will help build a model for assessment of oral healthcare in LTC facilities for a much wider application. 2) Sampling, Recruitment and Data Collection The close contact with the initiators, administrators, clinicians and recipients of care offered by the GDP provided a rich source of information for a single case study. I selected a purposeful sample (11) of participants who have a broad base of knowledge from among those who are willing to participate. More specifically, I employed three sampling strategies to purposefully select information-rich cases for this study. Firstly, because people have conflicting values on oral healthcare; I employed maximum variation sampling criteria to select a wide range group of individuals involved in the GDP.  Within each group, information- rich participants were identified using a snowball sampling technique. Consequently, my analysis should identify important shared dimensions and common patterns among these variations. Later, I used opportunistic or emergent sampling techniques to select more participants with unexpectedly important characteristics as the fieldwork Chapter 2  24 progresses. More specifically, one recipient of care and one care-aide were recruited as a result of site visits to a LTC facility.  The number of participants is not fixed but I ended the sampling when the analysis indicated that the information collected was saturated and no new information appeared (11,55,65,66). I sought advice on potential participants from the Director of the GDP and from facility administrators. I then contacted each individual by letter or email explaining the objective and procedure of the study6 (Appendix B). Most of the participants replied shortly after receiving the letter/email. For those who did not reply, I waited for one week and contacted them by phone. In total, 33 out of 39 individuals I contacted participated in this study7. 32 participants were interviewed face to face while another person who is an administrator corresponded by phone and email. Open-ended interviews supported by interview guides8 (Appendix C1-C3) explored the opinions and beliefs of administrators, clinicians, residents and family members. Participants comprising dental personnel; 1 site operator (director of care); 4 clinical nurse leaders; 7 care-aides; 3 residents in care and 5 family members; along with 5 administrators from UBC, PHC, BC Dental Association, College of Dental Surgeons of BC, and Ministry of Health Service. Responses to requests for interviews with residents are influenced by cognitive impairment. In 3 cases, I interviewed family members at the same time as interviewing the residents,  6  This study was approved by the Ethical Review Boards at UBC and at the Providence Health Care Group (Appendix A). 7 Potential participants who refused to participate include 1 nurse, 1 family member of a resident, and 4 high-level administrators. 8 The interview guide (Appendix C) helps me to identify topics, without actually structuring questions, thereby offering flexibility to focus discussions if necessary. Chapter 2  25 which I found very helpful. The initial interview with each participant lasted from half an hour to almost 2 hours and all interviews were audio-taped and transcribed verbatim with non-verbal reactions recorded in fieldnotes.  All participants also received a copy of the transcript for verification. The interview location was convenient for the participants, and most of the time I could observe their daily activities and interactions. In addition to the audio-taped interviews, I also met with the director and manager of the program for 30-60 minutes at least biweekly during March 2005 to November 2006 to discuss the history and updates of the program day-to-day operations.  Site observations were conducted as part-participant/part-observer at 5 facilities (11). The number and charateristic of the residents in each facility will be presented in chapter 4 (see table 4.1). As a volunteer and during each visit of the interview apppointment, I observed the daily basis operation of the facilities and the GDP program and took detailed field-notes. An observation guide helped frame the observation and field note entries (Appendix D1-2). During March 2005 to November 2006 (20 months), I compiled field-notes from 20 formal observations at 5 different LTC facilities, which I used in the analysis. In addition, I also visited the UBC hub at the specialty clinic regularly and kept the memos about interesting stories and events. Document analysis on the program’s history, funding, goals, objectives, policies, activities, accomplishments, along with reports on surveys of the staff and residents were analysed for content and impact. Besides official administrative documents, I also collected other print archives such as flyers and newsletters. Chapter 2  26 3) Data Management After each fieldwork activity was completed, audiotapes were transcribed verbatim; all data were organized into 4 electronic folders and 6 paper based folders. Electronic folders include 1) transcripts of the first interviews, 2) transcripts of the follow-up interviews, 3) field-notes, and 4) memos. As for paper-based folders, two folders contain interview transcripts and signed consent forms (Appendix E); two folders are for observation field-notes; and there is another big folder for administrative documents and archives obtained from the field. The main folder contains contact information, original copy of the consent form, interview guides, ethics approval sheet, and log for research progress. 4) Analysis The interview transcripts were analysed using computer software called QSR Nvivo.9 The unit of analysis of this study is the comprehensive oral healthcare program. Therefore, the analysis consists of two layers; individual participant’s case studies combined to make up a program case study. Detailed analysis can be divided into three steps of coding10. a) Descriptive Coding: I started coding the interview transcripts by looking for attributes of participants’ background such as role, gender, career history, age group; and also the attributes of each LTC facility such as number of beds and characteristics of residents.  9 NVivo qualitative data analysis software; QSR International Pty Ltd. Version 2, 2002 10 Coding (verb) is defined as marking the segments of data with descriptive words or category names.  Chapter 2  27 b) Topic Coding (top-down): Richards (67) defines topic coding as a coding step where passages are merely allocated to pre-determined topics. These topics were created from the theoretical framework built in the previous phase. Topic coding involves minimal interpretation and aims to identify: firstly, what are key components of the program? (i.e. what are structures, processes, and outcomes?); and secondly, how do participants define quality or success of the program? Table 2.1 Matrix of predetermined topics for framework analysis (S = structure, P = process, O = outcome, Q = quality defined).  Time Key Components Planning S P O Q Initial Implementation S P O Q Adaptive Implementation S P O Q Maturity and expansion S P O Q Regardless of time Free node  When I read through the transcripts, if the passages fit the pre-determined topics, I selected the text as in-vivo code11 and placed it in the corresponding topic located in a tree of categories. This tree is a hierarchical catalogue of topics (categories and sub-categories). The place, serving as a container, where I label each topic is called a node. As new topics, other than those pre-determined, appeared; I considered whether they deserve a place in this study or not. If they were interesting, I placed them, as the in-vivo passages, to the “free node”. Once I completed the topic coding of the first transcript, I browsed the content of each topic and began the process of sorting and labelling these passages into categories and sub-categories. For example, in the “process” topic, the information suggested that  11 In-vivo code means to assign the text that is to be coded to a node, whose label is the text itself. Chapter 2  28 two main categories could be formed: the administrative processes and the clinical interventions. This categorization evolved as more transcripts were analysed. Some sub-categories were combined and others were split and re-labelled. The name of each category and sub-category came from either the term that the participants used (folk-term12) or from theories about program evaluation and quality assessment. This step of analysis ended when no new category emerged and the negative cases were explored (11,66,68). A negative case is perspective deviant from the cross-interview pattern (11,66,68). For example, one participant stated that oral hygiene care was not a key component of the program because hygiene intervention was neither efficient nor effective. I then needed to explore the issue during the follow-up interview and observations. In essence, the analysis began with individual interview cases; all interviews were compared to look for similarities and differences (11,66,68,69). The common characteristics overriding the differences confirmed the core value of what was important for this group of people. However, the different perspectives on central issues were not disregarded but further analysed as part of the next step. c) Analytical Coding (bottom-up): This layer involves further interpretive work. As these selected texts in the free node built up, I started to think about what they were about and what name should I give them. I asked, “Why is this interesting?” The answer became a new category. I developed the labels of the catergories inductively as I coded the data rather than having them a priori as the previous step. Also, at this stage I found myself working more freely using Microsoft  12 Indigenous typologies are categories that come directly out of the jargon or everyday popular talk of the field (14). Chapter 2  29 Office Word and many times physical papers. When I found similar messages I copied the text, found the category I created, and pasted it there.   I also kept a memo to record my hunches and questions. Field-notes, and administrative and archived documents also provided me with either supportive or contrasting evidence of each issue. I focused on a phrase or passages that strike me or keep recurring and asked myself,  “What is going on?”; “Why is this interesting?”; “Why do some people view the same topic differently?”; “How will this study benefit from that passage?”; What are the patterns here? Also, I frequently revisited the transcripts and rechecked the context of the selected text. I reconsidered and revised both the label of and the classification of the categories as the material built up. The outcome of this bottom-up analysis is a thematic description of what’s going on in the program - how culture of LTC influences the quality assessment of a dental program. This information helped draft the model of program evaluation that was specific to dentistry and was meaningful because it could accommodate complexity of the LTC culture. This step of analysis stopped when the thematic categorization was focused, the narrative flowed logically, the quotations supported were sufficient, and no new information emerged (66-68,70). 5) Member-check A short (~30min) follow-up interview offered each participant the opportunity to confirm the accuracy of the interpretation of the earlier interview, and to discuss program theories explaining how the program works, factors influencing evaluations, Chapter 2  30 and if there were any standards of care. I was able to contact 19 out of 34 participants13 who helped to assure that I adequately captured the program portrait. 6) Writing Ethnography of the Program I present the findings in two main sections: 1) description of the program case study and its context; and 2) thematic narrative of what is going on and how this understanding may influence the quality assessment of the program. Each theme contains categories and subcategories where I also present participants’ quotes and excerpts from other document archives. I juxtaposed the evidence with descriptions and explanations (11,66,68,70). 2.2.3 Building the Assessment Model I used inductive techniques to refine the model by seeking exceptions to the original theoretical framework and modifying the framework to accommodate all of the program elements and lessons learned that emerged from the case study. Also, I created alternative categorization of program elements indicating quality which I described in detail in chapter 5. 2.2.4 Testing the Assessment Model The primary objective of this task is to evaluate the feasibility and usability of the model of quality assessment developed. The secondary objective is to explain how the model was revised following the application. I have tested the model by applying it to review an outreach dental geriatric program in Northern BC and then  13 I conducted a face-to-face follow-up interview with all of the dental personnel (8), Site operator (1), clinical nurse leaders (4), 1 resident, 3 family members, and 2 administrators. During this study, 2 elderly residents passed away. Chapter 2  31 analysed its feasibility and usability according to criteria proposed by leading quality agencies in North America. Because this step is another study in itself, detailed methods are described in chapter 6. The rigor, credibility and ethics of this small study were maintained in the same manner as the larger ethnographic case study described in the previous step. For example, the research participants signed the same Consent Form prior to an interview (Appendix E).  2.3 CREDIBILITY AND RIGOR  The rigor of this study runs throughout the four steps of the development of the assessment model. Also, clear description of research methods is the first step to establish credibility of the study.  2.3.1 the theoretical framework was developed from well-established theories relevant to the issue being addressed. Both P/P and SPO models have been successfully applied to many healthcare settings and types of care. The concept of SPO models has evolved to include quality management initiatives in LTC. The P/P model is based on the concept of health ecology and system thinking which the American Institute of Medicine (IoM) recently endorsed as a promising way to cross the chasm of quality of care (71). In addition, I have clarified my predispositions relevant to the research studies from the outset.  2.3.2 the ethnographic case study was done in rigorous manner to ensure the credibility of the study. After the analysis, I gave the transcript and interpretation back to the participant. Member checking method gave them a chance to give comments and elaborate on their own original statement (11,66,69). Triangulation method helped to cross check the consistency of the data from different times and Chapter 2  32 different sources. Prolonged engagement (20 months) in the field helped to build trust with participants. It also helped me learning their “lingo”, the way they do things, and checking for misinformation that the informants might introduce (11). Attention to negative or deviant cases (11,66) was pursued by further exploration of alternative explanations for the information to help refine the analysis until it could explain the cases under scrutiny. I also related multiple types of data to support or contradict the interpretation. A “confluence” of evidences raised confidence of credibility of the observation, interpretation and conclusion (11,66,69). Moreover, debriefing the findings to my supervisor and presenting the preliminary findings at meetings and conferences provided external checks (11). The views of the audiences on the preliminary analysis helped to confirm or contradict the interpretation as well as to identify what was missing. In addition, being immersed in the data and writing the findings with thick description aimed to establish more credibility and transferability (11,66,69). With detailed description researchers or practitioners elsewhere can apply the information to other settings and to determine whether the findings can be transferred because of shared characteristics. The most important requirement for ethnography is to explain how things are from the native’s point of view and to be systematic in recording this information (11,64,65).  I strived for other criteria for good ethnographic case study to make explicit what is implicit and tacit to informants.  In addition, information is substantial but easy to read; quotations are used effectively; and empathy is shown for all sides (11,66,69).; .  Chapter 2  33 2.3.3 the drafted model was constructed from credible sources of theories and empirical information thus holding parallel credibility. I strive to show interplay between the empirical and theoretical” (72). Empirical evidence helps make the theories specific to oral healthcare in LTC while theories helps to frame the analysis of the data obtained from fieldwork. Also, the suggested steps of assessment and quality indicators included in the model emerged from the real experience of people who were closely involved with the program (11).   2.3.4 the testing of the model was based on criteria for quality indicator evaluation recommended by leading quality agencies(73). Moreover, since this step was a qualitative research project within itself, I applied the assessment model with in a rigorous manner and formed the usual audit-trail required by qualitative evaluation methods.  Chapter 3  34         CHAPTER 3 THEORETICAL FRAMEWORK  3.0 INTRODUCTION  A review of dental literature in chapter 1 revealed that there remains the methodological difficulty associated with evaluations to monitor quality, indicate effectiveness, and create accountability of oral health programs in LTC. It is helpful to know also how to monitor various aspects of quality of care other than biological changes in clinical outcomes. Certainly, these tasks are difficult when the focus is narrow and methods restrictive. At present, there is no widely accepted framework for assessing and improving the quality of oral health services in residential care. The objective of this chapter is 1) to review basic concepts of quality of care and health programs in general, and 2) to build a theoretical framework based on a combination of both literatures. This framework aims to guide the design of an assessment model specific to oral health programs in LTC.  3.1 CONCEPTS ON QUALITY-OF-CARE AND HEALTH-PROGRAM EVALUATION Concepts about the assessment of quality of care and the evaluation of health programs began as separate disciplines, each with a narrow scope but with many similarities. Both concepts emerged from demands for improved quality and accountability of services and programs. Initially, quality of care focused solely on the performance of healthcare personnel, whereas evaluation of a health-program addressed overall merit of performance following an intervention. Recently, both Chapter 3  35 concepts were expanded to emphasise performance and outcome as integral parts of ongoing efforts to improve the quality of care (9;10). The most cited definition of “quality of care” in recent years identifies “the degree to which health services increase the likelihood of outcomes that are consistent with current professional knowledge” (71).  Measuring quality today assumes that we know what should and can be measured. Donabedian (74,75), in a seminal effort to address this issue, created a framework with three levels of evaluation. The first level evaluates the structure of the human, financial and other material resources within which care is delivered. The second level evaluates the process or activities of care, including the technical and interpersonal activities that are addressed, whereas the third level evaluates the results or outcomes that show change in health status and quality of life from the perspective of both the recipients and of the providers of care. Subsequently, Donabedian recommended that the results of a quality assessment should lead to corrective actions at any of the preceding levels where quality is inadequate. Traditional quality assurance (QA) has expanded from the original assessment of a physician’s performance to include evaluation of multidisciplinary services, a range of organizational variables, and a variety of outcomes. Also, it has expanded to emphasize an ongoing assessment of quality as part of the service rather than merely establishing criteria for quality, evaluating them, and improving whatever needs to be improved. It is from this expansive context of healthcare that terms such as “Continuous Quality Improvement” (CQI) (76) and “Total Quality Management” (TQM) (76) were borrowed from attempts to evaluate the quality of Chapter 3  36 industrial services and products, such as automobiles. In the context of this paper, we use the term QA to include the idea of an ongoing procedural assessment, and we attribute equal importance to a program’s performance and to “patient-outcomes” as integral to an effective assurance of quality. In summary, although quality of care can never be fully assured, it is reasonable to strive towards continually improving performance (74). Along with the conceptual changes relating to quality of care, we find that health-program evaluation also has expanded to encompass systematic and empirical information about the characteristics of a program’s inputs (structure), operational activities (process), and outcomes. The aim of evaluation, as with QA, has been expanded to assess assets and capacity, improve effectiveness, judge merit, and provide information for further developments (74). Despite their similar functions, program evaluation and QA continue to differ in their jargon, which in many occasions poses confusion. For example, there are four evaluation terms that are useful but potentially confusing. “Formative” evaluation has been linked to “process”, whereas “summative” evaluation has been associated closely with “outcome”. However, the terms “formative” and “summative” relate to the intention or purpose of an evaluation, i.e. whether to develop the program or to judge it. “Process” and “outcome” evaluations, in contrast, are similar to the quality assessment framework of Donabedian, and relate to different phases of a program (60). For instance, information on the process and on the outcome of a program both contribute usefully to the formative evaluation. Similarly, many researchers find that it might be difficult to actually define the boundary between categories (structure, Chapter 3  37 process, and outcome) developed by Donabedian. For example, Closs and Tierney (77) found that it was impossible to decide whether a system of communication in the hospital might be described as structure or process. One aspect of the system includes medical notes and other documentation, constituting structure; while the other aspect involves how the document is used constituting process.  3.2 BUILDING A THEORETICAL FRAMEWORK FROM A COMBINATION       OF THE TWO CONCEPTS  Essentially, quality assessment leading to assurance is a systematic evaluation of the structure, activity, and outcome of a service or organization linked in a chain of events (Figure 3.1). Good structure increases the probability of a good process; good process increases the probability of a good outcome; and outcomes indicate quality of care (78).  When information about the program comes from a comprehensive and detailed investigation based on a combination of the concepts of QA and program evaluation, both accountability and quality improvement are addressed. There are several program evaluations or frameworks for quality assessment that are appropriate to the complexity of an oral health program for residential care (10,51,79-81). They all offer a process that: 1) engages everyone; 2) seeks quality beyond the limits of effectiveness; 3) evaluates the system as a whole; 4) considers both formative and summative factors from multiple sources and methods; and 5) transfers new knowledge for appropriate action. Chapter 3  38 3.2.1Engagement LTC involves a mixture of health and social services beyond curative treatment and disease prevention (51), and typically it engages recipients of care, caregivers, and administrators. Moreover, it offers several levels of intervention involving both healthcare providers and educators (81,82). Therefore, the success of an evaluation process depends on appropriate contacts with a range of different individuals who have different perspectives, experiences and expectations relating to health and healthcare (10,80-82). 3.2.2. Quality Beyond Effectiveness Quality of care goes beyond clinical effectiveness and program efficiency. It includes outcomes, such as QoL, personal autonomy, care-values, preferences, culture, tradition, and experience (43,51). Program success requires diverse aspects of quality, such as efficacy, efficiency, optimality, acceptability, legitimacy, and equity, to complement the general objectives of improved oral health status and function, which traditionally dominate assessments of effectiveness (83).  Equitable access to affordable services is a particular aspect of care for frail elders that needs attention in the evaluation of programs in LTC facilities (8). Recently, there is interest in oral health systems that permit prevention-oriented third-party payment. This also warrants evaluation because of its influence on use of dental services (8). 3.2.3. Evaluating Structure-Process-Outcome The desired or expected outcome of LTC is often different from acute care by attending primarily to the prevention or slowing of a decline in functional status and capacity, supplemented by help with daily activities, comfort, safety, and personal Chapter 3  39 choices (43,51). Some of the care objectives are compatible and complementary, while others incite conflict. Each recipient of care values and balances quality of care and QoL differently. Furthermore, the inevitable process of frailty can mask the visible benefit of a clinical service (43,51). Therefore, a comprehensive evaluation of the structure and process of a service should account for how and why specific outcomes have emerged (51,74). Logic models, in the form of diagrams or tables illustrating the structure, process, and outcome of a program, have been used to good effect by demonstrating links between the inputs, activities, and achievements of a program (60,82). Consequently, this process of conceptualising a program help to compose relevant and incisive questions to evaluate and understand the order of effects and the meaning of the results (60,74,82). 3.2.4. Formative and Summative Evaluations from Multiple Sources Most of the summative evaluation of oral healthcare for frail elders has judged the effectiveness of various healthcare programs, but with little attention to the suitability of the care strategies or the appropriateness of the evaluation methods. Formative evaluation provides feedback on the progress of a program and also promotes desirable mid-course changes so that the interventions can adapt to changing circumstances. Accordingly, findings from formative evaluations help clarify the theoretical base for program development (60,82,84), while summative evaluations, especially those focused solely on outcome, offer very limited explanation of why an intervention is or is not successful (60,82,84). Multiple sources of information also provide credible evidence. Qualitative research provides insights to the underlying conceptual process and operational Chapter 3  40 dynamics that are frequently overlooked or ignored by more quantitative methods. For instance, variables selected for a scientific investigation are usually pre- determined by researchers without direct or empirical insights to the complicated range of behaviours, attitudes, or perspectives of the participants (11). Typically, studies involving frail, apprehensive or cognitively impaired elders will include information whenever possible from at least three sources: 1) the residents; 2) the caregivers; and 3) families (51). Therefore, the impact of services on elderly residents is determined most effectively by multiple assessment tools and techniques, such as focus-group discussions, systematic observations of the resident’s behaviours and facial expressions, and ethnographic studies of the organizations (51). Integration of qualitative evaluation with quantitative methods can help interpret statistical results by clarifying context and process as well as confirming or questioning evidence (60,82). 3.2.5. Knowledge Transfer with Good Program Description Relaying the results of an investigation, when a program either succeeds or fails is useful (60). When people seek to develop or adopt a health program, they need information on potential components and activities suitable for their local conditions. Therefore, high-quality evidence is not merely the result of an experimentally tested intervention, but it is also a trustworthy description of the procedures, theories and psychosocial circumstances that have influenced the intervention (62).  Chapter 3  41 3.3 CONCLUSIONS The relatively slow pace of change and lack of demonstrable benefits associated with oral healthcare programs for frail residents of LTC facilities is an ongoing concern, particularly when coupled with the growing demand for accountability. Implementing a “structure-process-outcome” framework with a focus on formative and summative methods of evaluation should engage all of the participants, and involve both quantitative and qualitative evidence beyond the clinical trial and towards a more comprehensive process of evaluation. This broad approach to an evaluation framework should help assure quality and accountability of the oral health program, and produce knowledge about the programs that are more in accordance with the complexity and instability of LTC (18,62,82).  Chapter 3  42 Figure 3.1 Theoretical frameworks for guiding the design of assessment                   model.    Chapter 4  43 CHAPTER 4 PORTRAIT OF A COMPREHENSIVE ORAL HEALTH PROGRAM IN LTC FACILITIES  4.0 INTRODUCTION In this chapter I aim to answer my first research question: what is a comprehensive oral health program in LTC facility? In addition, there are three sub- questions: 1) In which context does the program operate; 2) What are the key components of the program in terms of structure, process, and outcome; and 3) How do the multitude of factors within the program relate and interact? Consequently, I divided the findings from an ethnographic case study of a comprehensive oral health program  into three sections: 1. History, context, and general description of the program case study 2. Key components of the program 3. Thematic analysis of the complexity of the program  4.1 HISTORY, CONTEXT, AND GENERAL DESCRIPTION OF       THE PROGRAM CASE STUDY In 1999, Providence Healthcare (PHC), a group of Catholic hospitals, approached UBC, Faculty of Dentistry to consider establishing a partnership program to deliver comprehensive oral health services to approximately 1,000 elders who reside in their seven long-term care facilities. The UBC/PHC Geriatric Dentistry Program (GDP) was officially initiated in 2000 with a comprehensive needs assessment using CODE (an index of Clinical Oral Disorders in Elders) (28) to Chapter 4  44 document the prevalence of dental diseases as well as demographic information and medical history of the PHC residents. The results from the needs assessment was then used to develop a business plan for the program, which was presented to the Vancouver Coastal Health Authority for funding in 2001. Capital funding from UBC Faculty of Dentistry and St. Vincent’s Foundation was used to renovate part of the UBC specialty clinic to be used for the GDP clinic and office for the program manager. Two fully equipped dental clinics were set up within two of the larger PHC facilities. Also procedures under general anaesthesia were performed in an operating room at St. Vincent Hospital Heather. Essentially, the founders adopted a “hub and satellite” model of service delivery using UBC as a central hub where a program director and a program manager oversee outreach services offered at seven PHC facilities around Vancouver.  In addition to dental services, the program is also committed to education and research. During the first year of service (starting in January 2002), the GDP provided a free clinical oral examination to 894 elderly residents of 7 PHC facilities on-site. A dental hygienist went into each facility and assessed the daily-mouth care situation; set up a mouth care protocol with the nurse leaders; provided educational in- services for nursing staff; and delivered professional cleaning to the residents. Residents who needed further dental treatment14 were cared for either at the LTC facilities or transferred to the UBC specialty clinic. For more complicated cases that needed general anaesthesia, care was delivered in an operating room at St.  14 A total of 515 (58%) residents were recommended for treatment, 265 (30%) accepted the recommendation, and 234 (26%) received treatment (40). Chapter 4  45 Vincent’s Hospital Heather. Later in 2003, PHC was restructured. St. Vincent Hospital Heather and another 97 bed LTC facility was closed down. However, the operational grant of $185,000 was secured from the health authority under the condition that the GDP extends the service to other LTC facilities in Vancouver. Three non-PHC facilities were then added to the GDP. Therefore, the number of patients participating in the program remained the same in 2004 even though two PHC facilities were closed down. In addition, the mobile dental unit at St. Vincent’s Hospital Heather was moved to an operating room at Mt. St. Joseph Hospital.  In the subsequent years, revenue from the service fees, continuing support from the health authority and other funders15, request from the recipients of care, and stipulation from the Adult Care Regulations (85) (Appendix F) have made possible the expansion of the GDP. To date (March 2008), it offers a wide range of services based on the preference of each LTC facility to over 2,500 residents in 17 LTC facilities in Vancouver. My case study, due to resource constraint, covers the GDP program within the PHC sites only.  15 e.g. PHC, UBC, Tzu Chi Foundation, St. Vincent’s and Holy Family Foundation, and some other dental corporations. Chapter 4  46 Since the restructuring in 2004, the GDP has provided comprehensive oral healthcare to five LTC facilities, which differ mainly in size and background characteristics of the elderly residents (Table 4.1). This report reflects the situation at the time of program initiation until the end of the data collection period in November 2006. Table 4.1 Characteristics of the five long-term care facilities included in my                  case study  Facility No. of Bed Level of Care Note Facility A 150 Extended Care The only facility where male > female residents. Primarily for veterans. Facility B 221 Extended Care Residents are from a variety of cultural backgrounds Facility C 84 Multi-level Care Residents are mostly Catholic Caucasians. Mostly single rooms Facility D 142 Extended Care Residential care unit adjacent to a unit specializing in rehabilitation. Residents are from a variety of cultural backgrounds. Facility E 100 Extended Care One residential care floor in a fairly large hospital. Residents are primarily Chinese speaking.  Chapter 4  47 4.2 KEY COMPONENTS OF THE PROGRAM  This part of the findings stems from a topic analysis (top-down/framework analysis) of empirical information. The main objective of this step is to identify key components and pertinent mechanisms underlying the program (Figure 4.1). Figure 4.1 A simplified program model of the GDP Based on Structure-                   Process-Outcome Framework (74,75).       Program Outcomes  • System  • Patient PROGRAM DESCRIPTION ORGANIZATIONAL STRUCTURE & CULTURE OF THE PARTNER ORGANIZATION (S) COMMUNITY CONTEXT [social, economic, political, regulatory] Program Processes Screening & Diagnosis Education & Hygiene Dental Treatment Additional Prevention & Promotion Practice Management Program Structure Plan & Policy Money Material & Technology People Space & setting PATIENT CHARACTERISTICS & FAMILY  INVOLVEMENT Chapter 4  48 4.2.1 Structure Donabedian (75) defined structure as characteristics of the people involved, of the tools and resources, and of the physical and organizational settings. There are a large number of people and organizations involved in providing care, assisting in the operations, and supporting the program financially.  Essentially, the key people of the GDP were distributed to one of the following six groups (Figure 4.2): 1) policy makers; 2) funders; 3) providers of care (including dental professionals; facility staff); 4) recipients of care (including guardians); 5) insurance companies; and 6) the public (including the Media).                         Chapter 4  49     Figure 4.2 An inclusive organizational chart of the GDP  Chapter 4  50 Subsequently, they either produced or used other structural features such as plan and policy, money, material and technology, as well as space and setting to operate the program (Table 4.2). The GDP operates under the guideline outlined in an Affiliation Agreement between UBC and PHC. Dean of UBC, Dentistry and Vice President Medicine of PHC are the signatories of the agreement (GDP Annual Report, 2002). The GDP received an annual operating grant from the health authority of about $185,000. The budget covered salaries for a UBC faculty member to direct the program, for a general practice resident dentist, for a dental hygienist educator (one day per week), and for a clinical manager (formerly called a clinic receptionist and then a clinical coordinator). The remaining amount was added to revenue from provision of dental services to cover supplies, dental assistant salaries, transportation costs, etc. (GDP Annual Report, 2002). Moreover, provincial Medical Service Plan (MSP) coverage for annual and new residents’ oral examinations could have been another source of income but the claim was rejected by MSP due to lack of appropriate procedural codes and definition of a hospital patient versus a LTC resident. The money is held by PHC and UBC invoices the hospital on a quarterly basis. The program is administered financially through the Dean’s office of UBC Dentistry and the Centre for Aging and Health of PHC. The program director is responsible for the implementation of the program and reports directly to the Executive Steering Committee (ESC) that comprises representatives from PHC, UBC, and the health authority. The ESC used to meet quarterly to review the Chapter 4  51 program and approve the budget for the following quarter. Since 2005, the ESC has met biannually. The GDP had six dentists when the program was first established. After one year, three dentists remain within the program. The other two dentists left because one had different philosophy of care and another was not comfortable working in this environment. Another dentist retired and still worked with the GDP but PHC policies do not allow this dentist to work at PHC sites. All dentists were provided with an introduction to the program (including a copy of the operations manual entitled “Oral Health Care for Persons in Residential Care” and trained to perform standardized oral assessment by the program director (GDP Annual Report, 2002). The dentists are hired as associates, and on a monthly basis paid 50% of their net deposits minus the laboratory fee. A dental hygienist was also paid 50% of the total money deposited in that month in addition to the educational in-service fee. However, in 2006, the dental hygienist who established the oral hygiene part of the program was replaced by two dental hygienists: one as an educator and another as a clinical practitioner. In addition, the GDP originally had three certified dental assistants (CDAs) and since 2004 there have been two CDAs working with the GDP for PHC sites.  The GDP also hired two administrative assistants for the program manager, who were based at the UBC central hub. Chapter 4  52 Table 4.2 Structural features of the GDP  Structural Features Characteristics of Dental Team Characteristics of LTC Facility Characteristics of Patients and Families People  - Qualification - Philosophy of Care - Personality  - Support from facility admin - Support from staff - Turnover rate of both administrator and staff - Availability of volunteer - Residents’ health - Involvement of family/friends - Paid companion? - Involvement of family council Plan & Policy - Standardized/Formalized - Document everything - Eden Alternatives - Continuity of care policy - Residents’ rights - Autonomy Money - Secured grant - Service fees - Additional fundraising - Support from UBC for   utilities and supplies - Secured grant - Additional funds from Hospital Foundations  - Donors to the foundations - Clients paying for the service fee - Tax payers Material and Technology - Dental equipments and products - Dental Units - Laminated mouth-care card - Availability of Overhead Lift - Availability of Autoclave  - Bringing mouth- care products from outside Space & Setting - UBC hub, offices of program manager and assistants - Two rooms in UBC Specialty Clinic with shared waiting and sterilization area - Number of bed - Number and layout of wings/floors - Neighbourhood system - Private room/four-bed room - Onsite dental clinic at 3 facilities - Operating room at the hospital during the first 2 years        Chapter 4  53 4.2.2 Processes  Process refers to what is done in giving and receiving care. It includes the activities of recipients in seeking care and carrying it out as well as the providers’ technical and interpersonal activities (75). The data suggested two types of activities: first, interventions and second, practice management (Table 4.3). Interventions involved screening and diagnosis; education and hygiene; dental treatment; and additional prevention and promotion such as a caries management program (Appendix G) for high-risk patients. Practice management involved administrative works and supporting activities that made the care possible. A typical day at a dental clinic   A CDA arrives at the LTC facility early with equipment from the UBC hub to set-up. The dentist usually arrives around 9AM to see about 10 patients scheduled each morning. The number of patients increases if there are more oral examinations or recalls and decreases if there is more dental treatment. The dentist usually leaves shortly after the last patient is seen while an assistant is always left to clean up, finish documentation, and return medical charts to the nursing station.  One of the most daunting tasks happening during the whole session is to locate the residents and bring them to the dentist. Not uncommonly, the residents are still in bed; having breakfast; participating in recreational activities; in a hairdressing room; having a dialysis; or having moved to an acute care hospital or to another LTC facility. If there is no volunteer assisting, the dentist and the CDA spend time finding the residents. Chapter 4  54 The GDP manager has always faxed the appointment list to the facility’s unit clerk one day in advance so that the care staff can help get the residents ready; however that did not seem to always happen. Getting residents ready for oral health appointments seems to get less attention than other appointments such as dialysis and hairdressing. A typical day at UBC hub After the dentist recommends treatment, the GDP manager and her office assistant send out consent for treatment (including professional hygiene) and consent for financial responsibility to either the resident or the next-of-kin. Obtaining the record of next of kin and scheduling care for the elderly resident brought up another challenge. LTC facilities do not have accurate records and the program manager had to make several phone calls to get up-to-date information. On one occasion, for example, the manager made a follow-up phone call to a son of a resident who needed consent for regular professional cleaning; the son said, “well, that would be very difficult as she’s now six feet under!”.  But generally this process goes well and about half of the consents were sent back accepting the recommendation. The residents who did not need dental treatment are scheduled to see a dentist again in the following year. In addition to scheduling, billing, purchasing, liaising with the facility staff and residents’ families, it is common to see the manager helping with dental appointments in the clinic including arranging transportation for the patients and their companions.  Chapter 4  55 A typical day for education and mouth-care Educational in-service and daily mouth-care faced greater challenges and participants all agree that the goal has not yet been achieved. Since the program was first initiated, five nurse leaders from PHC formed a task group concerning mouth-care for the residents and continually collaborated with the GDP dental hygienist. In 2003, they produced a daily mouth-care protocol for PHC facilities (Appendix H). The structure of each facility and culture of each floor/wing16 played a major role in the level of how much of the mouth-care protocol was accepted and sustained. The dental hygienist found that staff on the floors where the clinical nurse leaders offered to provide mouth-care, leading the care-aides by example, were easier to work with. However, it seemed that Residential Care Aides (RCAs) operated independently from the nurses. From what I observed on many occasions, mouth-care was not a priority. Nonetheless, in 2006 one of the facilities restructured the staffing model and adopted a neighbourhood system to promote continuity of care. The RCAs now cared for the same 6 residents so they got to know them better. Also, this facility aimed for incremental changes and promoted one new implementation per week.  16 The nursing care at the LTC facilities is administered separately on each wing or floor of a multi- storied building. Chapter 4  56 Table 4.3 Procedural features of the GDP  Procedural features Dental Team  LTC facility  Patients and Families Interventions Screening & Diagnosis - Dentists or GPR17 & CDA perform oral assessment for new residents and annually thereafter - Volunteers find the patients, take them to see the dentist, bring them back to their room - Accompany the patient if need be Education & Hygiene - Registered Dental Hygienist (RDH) produced and implemented care protocol with the nurse task group. - RDH developed educational media with ELDERS research at UBC - RDH provides in-services - RDH audit daily mouth-care - RDH provides professional cleaning - Nurse task group produced and implemented care protocol - Nurse initially assess oral health (MDS) - Nursing staff attend   educational in-services - Ordering, stocking and   dispensing mouth-care   products - RCAs perform daily mouth-care - Provide mouth-care   when possible Dental Treatment - Dentists or GPR & CDA deliver treatment required - Volunteers find the residents, take them to see the dentist, bring them back to their room - Consent for treatment & financial responsibility - Accompany the patient - Pay service fee Prevention & Promotion - Caries management program by dentists - RNs & RCAs dispense medicine  - Family pay for the   service Practice Management Program Manager and Assistants - Constantly updating patient records - Prepare and manage consent forms - Scheduling, faxing the appointment list to unit clerks - Billing & requisition - Ordering & purchasing supplies - Liaison with LTC facility - Preparing reports - Unit clerks fax updated list of new admission, moved, and deceased - Volunteer co- coordinator organized for 2 volunteers each dental visit - Social workers provides info re: financial arrangement and counsel the families  - Family council raise the issue of quality of care if  need be  17 GPR is a dental resident of the UBC General Practice Residency Training Program. Each one of them rotate to work with the GDP for 8 weeks. Chapter 4  57 4.2.3 Outcomes  All sources of empirical data indicated two types of outcomes which the GDP rendered.  “First, the actual provision of dental services…and then, a good enough oral health which affects their quality of life” – PHC administrator, MD  I label the first type of outcome as system outcome and the second type as patient outcome. System outcomes involved changes in predisposing factors such as knowledge, attitude, and skill of nursing staff as well as changes in enabling factors such as access to care, availability of mouth-care products, policy and individualized oral care for each resident. One of the strengths of the GDP was that they documented every outcome possible. The number of residents who received oral assessment, number of cases with treatment recommended, number of consents sent, number of consents received, number of dental visits, number of educational hours, as well as CODE records (28) monitoring for dental conditions of each resident. They also conducted a satisfaction survey among nursing staff in 2004 and among recipients of care in 2005. Both groups were generally satisfied with the program. Concerns and suggestions were considered to improve the quality of the GDP in the following year. Table 4.4 presents a list of outcomes which emerged from the case study. Participants explained that the preferred outcomes were: 1) that residents had access to care; and 2) that the program maintained or improved their oral health. Investigation of specific definitions and standards of each outcome will be presented and discussed further in chapter five.  Chapter 4  58 Table 4.4 Outcome features of the GDP  Outcome features Dental Team (Internal) LTC facility (External) Patients and Families  System Outcomes Patient Outcomes   - Productivity - Regular and timely services - Care with respect - Care responsive to needs and preferences - Evidence-based dentistry - Agile and adaptive management - Equitable access - On-site delivery - Systemized record keeping - Stable financial structure - Accountable financial records - Customized individual oral care plan - Availability of standardized mount-care products - Improved knowledge, attitude, and skill re: mouth-care - Daily mouth care performed regularly - Satisfied work life  From providers’ perspective: - Stable oral health status - No pain - No acute infection - Comfort - Function  From Recipients’ perspective: - Oral Health Related   Quality of Life - Overall Satisfaction with care  Chapter 4  59 4.3 THEMATIC ANALYSIS OF THE COMPLEXITY OF THE PROGRAM  Besides the topical coding, there were many interesting statements that did not fit the predetermined analytical framework. These statements were initially coded in the free node and later analyzed inductively for recurring themes. In all, three themes emerged to further describe the complexity of the GDP.  I found that the program is complex because things within and around it are 1) ever-changing, 2) conflicting, and 3) massively entangled. Figure 4.3 Three emerging themes describing complexity of the program.      Chapter 4  60 4.3.1 Things Are Ever-changing: a) Constant changes  During the years 2003-2006, there had been much turmoil in the BC healthcare system (Table 4.5). Restructuring of the healthcare system both regionally and within the PHC influenced the implementation of the PHC/UBC GDP.  “We just sort of start getting moving in a direction that I’d like to see and there’s a drop. Like they cut wages or somebody’s gone to a strike...where our HEU workers were pretty much fired…it’s been very difficult. Every time you think that you [are} sort of on a roll and maybe it’s time to start [mouth-care] education or time to start changing and something else happened… and you have to recover from that…so that you can move on again.”  – A clinical nurse leader   Table 4.5 History of change in provincial healthcare system during 2002-2006.  Year  Changes 2002/2003 • Hospital closure • Staffing changes (fewer RNs, more RCAs, discontinued LPNs)18 • Government imposed contract, staff losing wages (HEU strike)19 • Change admission criteria for new residents 2004 • Privatization of housekeeping and food service • Regionalization of supplies purchase and ordering 2005 • Adopting Eden Alternatives • Adopting Neighbourhood Program • PHC merged with Vancouver Coastal Health • RCAs not allowed accompanying the resident to dental visits outside the facility. Residents must hire a companion if family is not available. 2006 • New Staffing Model (RCA dispenses medication) • High turnover from relocating between facilities within the PHC • High sick time, a lot of casual staff (substitutes) • HandyDART20 restructured their service   18 RN = Registered Nurse, RCA = Resident Care Aide/Attendant, LPN = Licensed Practical Nurse 19 HEU = Hospital Employees’ Union 20 HandyDART is a public transit service with specially equipped vehicles designed to carry     passengers with physical or cognitive disabilities. Chapter 4  61 These structural changes were out of the GDP’s ability to control or manipulate. However, it is useful to understand how these changes might have impact on the implementation of the GDP.  “We moved from children and women [hospital] doing all of our products to MSJ.  [A dental hygienist] wrote up a nice product list that we needed and it was all sent to us so we had toothbrushes and denture cups and all of the toothpastes and all of that recommended stuff and then what happened was that Providence Health Care took us over and for some reasons something falls through the cracks which I am trying to pick up on right now. So we weren’t always getting the right product at the right time.” – A clinical nurse leader   “Share-buying is the way to go [after the regionalization]. So the whole region decides on products [which are often different from a recommendation by the GDP], and then they just do mass buying and send it all to one warehouse rather than it sitting in each one of the hospitals and for us it would be a top-up system so they will say to us you only need 6 bottles of mouth wash once a week. And somebody would come in and if I have 4 bottles, they’re bringing 2 bottles for that week.”  – A clinical nurse leader  Dental personnel found that initial implementation was difficult despite the external changes. All participants who are dental care providers suggested that the first few years of program development were trial and error because there was no program prototype to follow. The program is the first of its kind in Canada. “It’s like a wild wild west…you just had to try and document everything…and learn as you go” – A program director Chapter 4  62 b) Adapt as we grow  The approach that the GDP took was to “adapt as we grow”, said a dental assistant. One nurse leader also asserted, it was “learning by experience throughout by fire.” The GDP manager stated, “The thing is we didn’t really jump into it. We gradually grew into it and we’re still growing with anything new. You learn with your experience and you get better at it and you get more efficient at it and you find out what works what doesn’t work.”   I found that the official plan of procedure and written care objectives played a minor role in the initiation of the program. What was needed was a business plan for the number of the patients and their estimated cost of care based on a model of typical dental practice. Therefore, the care objectives and planned procedures that were written in the official documents did not necessarily reflect the reality of the program implementation. They were later reassessed and rewritten as people involved in the program learned more of what is appropriate, effective, and efficient. For example: “The model [of care delivery] has changed through out the years because we found out the need for [an] operating room was low.”    Chapter 4  63 In summary, I classify the stages of development of the program into five steps: 1) Need assessment (year 2000) 2) Planning year (year 2000/2001) 3) Initial Implementation (year 2002) 4) Adaptive Implementation (year 2003/2004) 5) Matured Implementation (year 2004/2005) 5) Expanding and sustaining the program (year 2005 to present)  4.3.2 Conflicting Views a) Us and them  Regarding the constant changes discussed in the first theme, I found that the two sides of the partnership did not seem to directly communicate much explanation or empathy on the situation.  “I found that out during my last audit that there were a lot of over- the-counter and outside products being used. So I developed a letter that would go out to all the families. Most sites refused to send it out. I don’t know why they don’t do those things. So they posted it whether or not family members see it” – A dental hygienist  “They [dental personnel] don’t know those factors [structural changes that hindered the ordering and dispensing recommended mouth-care products] and probably in that I am empathetic too because it is sort of a different sphere…we’ve gone through major upheavals.” – a clinical nurse leader    Chapter 4  64 Moreover, the facility staff felt that “unlike any of the other disciplines that work in residential care, the dentistry program is really not felt to be part of the interdisciplinary team”  “They don’t attend care conferences. They don’t, [try] to be part of the team…you usually…have visibility or have some presence as part of the team…but that didn’t happen.” – A clinical nurse leader   “In the care conference if it [oral health] is mentioned at all, it’s from the dietician. We don’t feel ownership [of oral healthcare program]. It’s like this independent group that comes in and does their thing and leaves.” - A clinical nurse leader –  “It [dentistry] still acts pretty much as a separate entity. We have some co-ordination that goes between unit-clerk and [the program manager] who does all of the appointments. Other than that we see [the dental hygienist who] comes up to do some care and she co- ordinates with the RNs. We rarely see the dentist upstairs and there’s a team of volunteers that comes to take charts and the residents downstairs.”  – A clinical nurse leader   “These folks come and provide independent service. I am not sure if that’s their intention or not but that’s certainly what happened.” – A clinical nurse leader    As for the intention of dental personnel, it seemed that they too would like the GDP to be more integrated into the residential care team. However, they found that the environment did not enable them to develop relationships with the facility staff.   Chapter 4  65 “[the program director] and [the program manager] were the front people…[they] got to talk with [the facility administrators and nurse leaders] but I am the one who’s there every week. I want the relationship. I was at [the facility] on a regular basis. And during that time I think there were two different administrators. I couldn’t have told you their names because I didn’t have a relationship with them. Not that I wouldn’t have wanted that, it just isn’t how the organization works.” - A dentist #1   “I didn’t have any formal introduction when I first started with any of the staff there not even a volunteer coordinator or anyone like that so I think it would have been nice [if I was introduced]. I think they just put a notice that there’s a new dentist replacing Dr. [name] but I never really got to meet the head nurses or staff nurses.” – A dentist #2    On the other hand, the nursing staff reflected that it was the nature of dentistry that had “historically been private sector using an office model” that cause them not being used to actively integrate with the interdisciplinary team.  “They haven’t been part of this BIG…machine called HEALTHCARE. So they come with a completely different mindset”   Also, one nurse asserted the disintegration was rooted in the fact that dentistry had not been part of medical care. She further reflected that “You can’t teach the nurse how to be a dental assistant or a dental hygienist when they’ve been taught nursing. Mouth-care was never high on the priority course as for education went through nursing school. It wasn’t that it was overlooked but it wasn’t the focus because there’s a dental team [laugh] and so the whole idea of educating each other and actually working as a team would be, to me, a better outcome.”   b) Different perspectives on roles and responsibilities Similar to the last quotation, expectation on roles and responsibilities played an important part in how well the GDP was integrated into residential care. The Chapter 4  66 following two side of the same story illustrate how different expectations of roles and responsibilities causes conflicts among people involved in the program.   “You need support at all levels. And we just weren’t getting it. From things like I needed someone to help me with transfer. They didn’t have a lift. And people would be sent down with no means of transferring from a wheel chair. Well, you know that’s just not acceptable. And they KNOW that this person needs to be transferred yet nobody takes responsibility for it. They’re in a hospital, there’s a head administrator. In terms of on the floor operations, the nurses…can’t come down and help transfer. They can’t designate a care-aide to work with me either. They can’t do some of the things that a chief administrator can do”  – A dentist  “…there were a lot of concerns about…demanding time of the care- aides and not communicating well…nurses were getting upset because care-aides were being pulled out from the floors and transport people down to the clinic and we didn’t build that in our work schedule right? So it’s…you implement something here, it’s going to have a ripple effect to probably a number of people…maybe the right people weren’t involved at the outset…we need to have this person at the table [saying] this [dental program] isn’t just coming in and doing an oral exam. This is what it’s going to take to make that happen. WE…let’s make it happen.” – A facility administrator  Especially with daily mouth-care, many care-aides view it as an elective part of their job description. “Mouth-care…it’s almost like a choice. If you don’t have time, you don’t do it; if you have time, you MAYBE do it.” – A clinical nurse leader – “We get new care aides who go through schooling and in the schooling they’re taught that mouth care is part of their job and they come and they think they’re going to do mouth care but six months or a year later, they’re not doing mouth care because it’s not part of the overall, among the care aides themselves expectations.” – A clinical nurse leader  Chapter 4  67 “There already is a strong part of what the care aides feel like is their job and what they feel like is not their job and some of them don’t think mouth care is their job.” – A clinical nurse leader     Moreover, another conflicting issue was around the different perspectives on “need” for oral health care. Dental personnel aim to maintain or improve clinical status of residents’ oral health, while deinstitutionalized policies such as Eden Alternatives promote independence and choice for the residents. The care-aides often said during the interviews “we are not supposed to force the residents”, hence no mouth-care if the residents are not cooperative. In addition, it is not uncommon to hear conflicting opinions whether dental care was necessary when the recipients must pay for care. “There was a feeling that the clinical issues with these elders weren’t necessarily been considered in recommendation of the treatment part. Families called social worker all the time when they got the bill and went, “what the hell is this? $3,000?” And of cause the social work had to explain, no we’re not telling you to pay that…the social worker had to do problem solving around that” – A facility administrator Chapter 4  68 4.3.3 All Variables are Massively Entangled a) Multiple causation  When I asked what makes the program challenging, first reaction from most participants were:  “It’s hard to say, there are sort of a number of things that happen all at the same time.”   “It’s difficult because there’s a language barrier; the distance; socio-economically, families are not able to pay; families are not able to take time off; families are not able to drive their family member out to appointments; families are not able to appreciate the value and importance...so there are mediating circumstances that make what kind of program at a facility.”   “I’m well aware of those things [significance of oral health and mouth-care], but those are being, if you’ll pardon the pun, BRUSHED aside by housekeeping…this last year and a half, because the nurses or care-aides are doing chores that are absolutely necessary but outside of their job description…I saw one of the nurses running down to get a bunch of spoons, they [food service] didn’t send spoons up…its ridiculous they should be doing something else.” – Resident’s family  For example, the GDP naively “started off with the educational in-service with the best intention” to increase technical knowledge and skill so that the care-aides have the ability to perform mouth-care without a careful planning to target other multitudes of variables especially those influencing “the will to perform” mouth-care (Figure 4.4).  “We thought we gave everybody information but maybe we gave too much information. We probably should have started with asking the care-aides what they thought they needed. And probably should have asked them what would make them or help them give better mouth care…maybe they don’t need to have education on how to brush their teeth.” – A clinical nurse leader  69 C hapter 4 Figure 4.4 The web of variables influencing the ability and the willingness of care-aides to                   perform daily oral hygiene for residents of long-term care facilities.  Chapter 4    70  Figure 4.4 illustrates the web of many variables that make daily mouth-care work or not work. Some of the factors such as residents’ cognitive level and cultural background of the RCAs (bottom right corner) are difficult to change and many variables regarding the working environment are out of the GDP’s control. This figure demonstrates the level of complexity that a program planner needs to consider when planning and evaluating an oral healthcare program.  b) Many variables are difficult to change or will never change  One of the most difficult variables to change was the attitude and belief of care-aides toward oral health and mouth-care. They offered four good reasons to explain why they could not or would not do mouth-care: 1) uncooperative residents; 2) high workload; 3) unclear consequence; and 4) recommended mouth-care products are not their preference and not always available.  “I don’t know why they discontinued it [the toothette]. I like it… I think maybe dentists should do more experiment on it.” – A care-aide  “It’s difficult to say. Some of them [residents] never had mouth- care and they seem to be fine…before this dental thing happened, we did still give the mouth-care but not like you have to do it everyday… and there were no major problems… OK with mouth- care it makes them [the residents] smell better…but it’s not like if you don’t brush your teeth it will be…I don’t know…it’s not the same thing [as other type of care]. I don’t know why I’ve worked for so many years and I found it’s the same [whether we do mouth- care or not]” – A care-aide  “They [residents] would do it if they were doing it before and some of them was so difficult…like [a male resident] You can’t go close to him…we tried to do it but we are not suppose to force” – A care- aide  Chapter 4    71   Of course not all care-aides were opposed to performing mouth-care but those who were had made the educational in-service very difficult, and they influenced the attitude of other care-aides overall.  “She [a dental hygienist] did it [hands-on] one time and we took the resident like [name of the resident] who’s very tough… and said now you show us how to do it…she didn’t get success. So she said, “oh, just keep doing it.”…“you go in the back and hold his arms” but he was still…Oh! he can fight that man. [laugh]…No matter how you approach him.” – A care-aide    “I hope they [dental people] try on the difficult ones themselves. You can say you should try this method and that method. They [dental people] should come experience themselves, and see if that method works for them before they tell us, “oh, this is easy you go to do this and that you will manage”…but it works only in some of them [residents]. Remember that we’re the ones doing it. It’s very hard. Another thing is when [a dental hygienist] comes; she only has 3-4 residents to do right? Just to do the teeth and with us, we do EVERYTHING…plus mouth-care.” – A care-aide   Despite the truth in the care-aides’ account, I found that they tried to generalize the difficult cases and convince themselves that they had good enough excuses to skip mouth-care for other residents. Both the dental hygienist and I found from several observations that the care-aides did have free time in the early afternoon. Also evening schedules were not as tight as in the morning. The dental hygienist and many nurse leaders had an idea of promoting mouth-care as once-a- day task: anytime before the resident was put to bed at night. However, the idea had not yet been realized because the dental hygienist discontinued her role with the GDP in 2006. The new dental hygiene educator took a new approach trying to understand the culture of each floor before planning any formal educational Chapter 4    72 intervention. He started by talking about how care-aides value their own oral health and how they perform mouth-care for themselves. However, I have not yet heard about his latest developments. One thing certain was the dental personnel could no longer recommend the importance of mouth-care through the same educational strategy because it did not work. In all, besides the multi-causational effect of the multitude of variables involved and the difficulty controlling or changing some of these variables; participants also suggested many promising solutions to make the GDP work.  4.4 LESSONS LEARNED AND POSSIBLE SOLUTIONS  Fortunately, none of the participants thought the GDP had to “reinvent the wheel”. They found that “there is lots of good information already” to help improve the program. In essence, people who are involved in the program implementation need to recognize that:  “We’re dealing with totally different groups of people depending on which facility you are in, so the need and the acceptance of the patient and family AND the staff is a different one in each facility.” - A program director   I summarized other suggestions from the participants as follow:  • “That [changes in the system] are all part of this…you have to be always diligent of what’s there and why it’s there…and who you talk to about getting things back again…it would be a waste of time if we just let it fall through.” - A clinical nurse leader  • “The program must be flexible and agile.” – UBC administrator  Chapter 4    73 • “I’d just restart a lot slower. I would definitely start with asking people if they think it’s important…don’t start with education and start from preparing people for it rather than saying we think it’s important.” - A clinical nurse leader  • “The person who starts the mouth-care program should get to know the culture of the place. And really spend time figuring out how things work there and define how to work with that culture…work with that organization within the system that they already have…rather than coming in with assumption of how things might work.” - A clinical nurse leader  • “Not trying to promise too much too quickly given that we had a limited amount of resources” – UBC administrator  • “Work with…be partner with nursing…HUGE…HUGE. If you don’t partner with nursing, it’s like you shoot your left foot. Because they’re the ones who ultimately have to work through”  - A clinical nurse leader   • “Value the input of nursing. That’s the only reason the dental hygienist made any headway with developing the protocols that were developed here.” - A clinical nurse leader  • “Organize program around nursing and ensure that it’s not just an added on.” - A clinical nurse leader  •  “A lot of things happen at the [care conference]. If you’re there, you’re part of that team. - A clinical nurse leader  • “Dental team should be included within our care conference if they’ve done some major work on the person.” – A clinical nurse leader  • “[Dental personnel] don’t have to be part of the annual review for everybody but there may be for some residents with certain flag”  - A PHC administrator, MD – • “I see that the education is not addressing the issue. Maybe bring oral care to fore front in the care conference…meet other disciplines… developing a form that the dental hygienist write a report and it [report] would go out to care conferences.” – A dental hygienist   • “Our next step was for me to go to family council when there’s a council meeting again.” – A new dental hygienist Chapter 4    74  • “There’s a model set up at [facility Z]…. just an example of another way to have the marriage of dentistry and hygiene to a facility. And they actually have an onsite staff…a dental hygienist and a dentist who work just at that site. They do attend care conference so that’s quite different.” - A clinical nurse leader     • “They [facility staff] needed someone to bring the whole thing under… who do you go to…you have to have a chain of command, a communication…I call her [the central person] the umbrella.” – A resident’s family member   Lastly, as one participant suggested,   “You have to define success maybe differently than you normally define success of other things you do in your life; because in most other aspects of your life you probably have control over. And when you’re dealing with a program with this magnitude and you’re dealing with people and you’re sensitive to the situation and it’s frustrating.”   Chapter 4    75 In conclusion, the implications of this chapter for the design of my assessment model are two fold. Firstly, this chapter identifies key components of the program that evaluators must take into account. These components provide a framework for describing the program and developing a plan for creating relevant quality indicators. I include six core components: 1) community context, 2) structure and culture of the LTC facility, 3) characteristics of the recipients of care, 4) program structure, 5) program processes, and 6) desired outcomes of the program. Secondly, thematic analysis of the complexity of the program provides guiding principles for the assessment of quality of the program. For example, when the program needs to adapt to ever-changing circumstance and when the multitude of factors are massively entangled; quality assessment of such a program must take into account the relationships among pieces within the program as well as the influences of the contextual factors and functioning of the LTC facility. In addition, when the program operates among conflicting views, the assessment should be participatory and engage as many key people involved as possible to assure that multiple perspectives are heard. Subsequently, the next chapter investigates in particular the program elements that most indicate quality and success, the assessment criteria, and methods of assessment.  Chapter 5    76 CHAPTER 5  DRAFTING A MODEL FOR ASSESSING QUALITY OF A COMPREHENSIVE ORAL HEALTH PROGRAM IN LONG-TERM CARE FACILITIES  5.0 INTRODUCTION  In the last chapter, an ethnographic analysis of a program case study identified 6 core components of the program that the evaluator has to take into account: 1) community context, 2) structure and culture of LTC facility, 3) characteristic of the recipients of care, 4) program structure, 5) program processes, and 6) expected outcomes of the program. It also yielded insight about the complexity of the program and how this complexity might affect the design of the assessment. In particular it explains that a multitude of stakeholders should be involved to assure that conflicting perspectives are heard.  And at the same time the assessment should be comprehensive enough to be able to explain procedural and outcome variations caused by massively entangled factors and ever-changing circumstances. This chapter then exploits such information to answer the second research question of this thesis: how best to assess quality of a comprehensive oral health program? The design of the assessment model evolved around three research sub-questions: 1. What indicates the quality of the program? 2. What are the thresholds of acceptable quality? 3. What are the methods for assessing quality of the program? Participants’ accounts, events observed, and published texts from an ethnographic case study provided information focusing on the opinions concerning Chapter 5    77 the description of success of the program and the experiences of how the program was or could be assessed for quality. The theoretical framework (chapter 3) also came into play by providing choices of language used to classify and label information gathered. According to Donabedian (86), two main parts must be defined when assessing quality of any health program: first, the points of information collection that indicate quality of the program hence, the quality indicators; and second, the points of reference for comparing and contrasting the information collected. The later is often referred to as standards or criteria to judge the program’s success or shortfall. Generally, when asked about what indicated quality of the program, interview participants responded quite easily, while discussion about program standards required a lot more effort for the interviewer to probe and for the interviewee to contemplate. 5.1 WHAT INDICATES QUALITY OF THE PROGRAM? Chapter 4 described the many elements of the structure, processes, and outcomes of the program and Donabedian (75) suggested that each category is a potential area for quality assessment. However, I found that, with the information collected from a case study, the distinction among the three categories is quite challenging to analyze. As described in chapter 4, participants related to the term “desired outcome” of the program in two aspects: the system outcomes and the patient outcomes. Similarly, when discussing what indicates quality of the program, they expressed that quality is: firstly, a degree of excellence of the service itself; and Chapter 5    78 secondly, in the accomplishment or outcomes reflected by the patients. As one of the administrators explained, “Outcomes are separated in my mind in terms of really needed clinical dentistry…to attend to an urgent problem or something like that, and there’s that other side…what is good oral health”  Following Donabedian’s classification (75,78), the term “the service itself” that the same administrator suggested may point to either structural or procedural elements. For example, on-site dental clinic indicates quality of amenities and responsive emergency consultation indicates quality of care process. Moreover, some desired outcomes such as integration between dentistry and LTC couldn’t be easily distinguished whether it falls into the structural or procedural category. In response to this unclear categorization, I analyzed three of the most informative interviews∞ inductively for alternative classification. The bottom-up coding revealed three main categories: 1) ultimate outcomes observed in or by the primary targets (elderly residents); 2) proximal outcomes seen in or by the care process and secondary targets (e.g. nursing staff); and 3) the interplay between the structure and process of the program that enables the program success. With the new classification (Figure 5.1), I decided to strictly label the first category as “outcome indicators”. They represent the result of the program demonstrated by the patient’s oral health, overall satisfaction with care result, and their quality of life. And, as for more proximal outcomes, such as the knowledge and performance of nursing staff on daily mouth-care and residents’ satisfaction with  ∞ an administrator, a clinical nurse leader, and a dentist Chapter 5    79 care process, I classified them as “performance indicators”. I selected the term mainly because it was brought up by the facility administrators and the nurse leaders attending the ELDERS LTC meeting in 2003.21 They suggested that “performance indicators” might help to improve mouth-care situation in the facilities because they can help holding the care-aides accountable; hence the origin of this study. Lastly, the term “capacity indicators” was chosen to represent the enabling physical and human resources as well as interactions among themselves. Elements of the program that fall under this category often depend on the policies or on the higher administrative level of responsibility. They are often out of the front line staff’s control on a day-to-day basis. Another way of viewing this classification is to use the lens of P/P framework (10). Capacity indicators correspond to the enabling Policies, Resources, and Organization (PRO). Performance indicators represent the interventions and the outputs of the processes targeting at Predisposing factors and Reinforcing factors (e.g. knowledge, attitude, skill, and satisfaction with care).  Ultimately, outcome indicators denote maintained or improved oral health, overall satisfaction and quality of life. Figure 5.1 Categories and labels of the new classification of quality                    indicators.    21 Please see Chapter 2 page 4.     Categories                                    Labels Enabling Structure + Process           Æ  Capacity indicators        Process + Proximal outcomes          Æ  Performance indicators     Ultimate outcomes or patient outcomes  Æ  Outcome indicators Chapter 5    80 After forming the categorization and definition of the quality indicators, I coded the rest of the transcripts and field-notes deductively using framework analytical method (67). Excerpts from the interviews and text from field-notes were first placed into each general level of the three indictors and later sorted to identify subcategories (Figure 5.2). As a result, capacity indicators comprise of subcategories such as 1) accessibility, 2) goals and philosophy of care, 3) record system, 4) personnel and consultants, 5) level of support from LTC facility, 6) management styles and relationships among team members, 7) work incentives, and 8) self-assessment plan. Performance indicators include 1) satisfaction with care process as perceived by staff and recipients of care, 2) productivity of dental service and educational in-service, 3) daily oral care/mouth-care assessment and planning, 4) availability of mouth-care products and reminders, 5) changes in knowledge and attitude regarding oral health among nursing staff and recipients of care, 6) the actual performance of nursing staff regarding daily mouth-care, 7) integration between dentistry and LTC and 8) financial accountability of the program. Ultimately, outcome indicators take into account both providers’ and recipients’ perspectives include oral health, oral health related quality of life, and satisfaction with treatment outcomes.  Chapter 5    81 Figure 5.2 Categories and subcategories of quality indictors.     5.2 WHAT IS THE THRESHOLD FOR ACCEPTABLE QUALITY?  As stated earlier, interview participants found it difficult to speak about what is acceptable and what is good enough considering the massively entangled factors. On the superficial level of conversation, all participants stated that quality of life should be the ultimate outcome that determines the program’s quality. As one of the nurses explained, “The biggest is the quality of life because…the world has narrowed on these people so they tend to focus on things that bring them comfort…food… drinking…it’s a very basic need for an individual and if they’re unable [to eat or drink], because of pain or whatever the conditions are in the mouth, they’re miserable.”  However, questions probing more deeply about “what is good enough for frail elders?” and “how to logistically obtain the information?” are very difficult to answer. As one of the administrators expressed, Chapter 5    82   “The problem is we don’t know what is good oral health…I don’t know how to answer that, I know oral health does have a big impact on their quality of life, to be free of pain, to be able to chew their food and those sort of things but what should be the threshold for saying that this is poor and that’s bad …teeth have not been cleaned? There’s a bad odor? I don’t know.”  During the course of the first and follow-up interviews, participants were prompted to contemplate the questions, and the answers reflected that different group of participants might have different goals (Table 5.1-5.3). For example, dental personnel were concerned more about productivity and satisfaction with dental procedures while nursing staff were concerned more about daily mouth-care. Administrators aimed at compliance with regulations, efficiency, and having no complaint from either their staff or recipient of care. Interestingly, despite some variations in term of program goals and initial struggle to find answers about quality thresholds, minimal standards regarding patient outcomes could be established. On the contrary, when discussing the level of acceptance regarding capacity and performance of the program, participants struggled more to set minimal standards of acceptability. For example, one of the clinical nurse leaders said, “I would be very happy if the mouth-care is done once a day...anytime of day. At the moment it’s not always happening.”  Similarly, a dental hygienist said,  “No resident should be going to sleep without some cleaning of the mouth, whether that’s even at the most, at the minimum level a rinsing of the mouth, flushing out the mouth debris.” Chapter 5    83  But then the same nurse and dental hygienist went on to suggest that too realistic a standard can be problematic because “it sets into place a pattern of practice of what you’re willing to accept and what you’re not willing to accept.” The dental hygienist further explained, “A threshold is how much you are willing to risk, are you willing to say you understand what oral health means for this vulnerable population…what is that risk…how serious the consequences are.”  Given the evidence of inadequate mouth-care, nurse leaders and dental professionals suggested that there must be minimal standards to hold the care- aides accountable. Nonetheless, due to a multitude of challenging factors in LTC and the different context of each facility, a nurse leader suggested that initial standards should be based on very realistic goal; and over a period of improvement initiative, expectation could be “increase[d] incrementally” (Figure 5.3). For instance, instead of setting standards right at the onset that quality daily mouth-care is brushing teeth or dentures two times a day; the program may start with minimal standard of just having the evidence-based mouth-care products available for each residents. Then the requirement may move up to having the care-aides identify individual mouth-care plan for each of their assigned residents; to being able to brush teeth or dentures once a day - anytime of day; to finally brushing teeth or dentures twice a day. One of the nurse leaders compared mouth-care with “pushing something up hill…it keeps rolling back”. She believed that “…until finally you push it right over the top, then it become the normal practice”. Therefore, if the hill is too high with no rest Chapter 5    84 stop, “in my experience people get it half way up the hill and then they go on to something else or they start thinking about something else and then it just goes back to where it started.”  Figure 5.3 Incrementally increase the level of standard for each assessment                   cycle.    Consequently, I have summarized the list of not only the minimal standards of practice but also what is considered as improved practices (Table 5.1-5.3). Then, when programs are assessed, they can be compared against the standards that fit their stage of development. These non-static goals also address the ever-changing nature of the program. (t = timeframe) Chapter 5    85 Table 5.1 Capacity indicators and examples of standard of acceptability                  according to the case study  Indicators Minimal Standard Improved Practice 1. Accessibility - Physical accessibility - Formal contact with any dental professionals (dentist/RDH/denturist): “A facility has access to somebody” - admin - Transfer to designated dental professionals - On-site delivery/Portable equipments with designated space i.e. a meeting room, a hair dressing room, a quiet room, etc. - A formal structured oral health program - Fully equipped on-site dental - “Have a lift, that’s key, to get somebody out of their wheelchair into [a dental] chair, I mean that’s number one, you have to have that” - dentist - Financial accessibility - Standardized fee guide - Cost of care is covered for the residents 2. Goals and philosophy of care - Documentation of program goals - Documentation of practice protocol/guidelines - Clear and specific action - Widely shared among team members 3. Record system - “At least have the evidence…how many patients they [dental professionals] see…what procedures they do and…what the outcomes are” - admin - “All that data computerized during the exams and during the recall exams and you can sort of follow up patients on a the long- term scale” - dentist - “[Facility] keep[ing] a list or a record of problems that they were refer to dental. How many issues were resolved? How quickly, how thoroughly.” - admin 4. Dental personnel &     Consultants - Quality of dental personnel: “Personality is big time…and the ability to adapt to any situation…I think it is a lot more important than the actual dental skill. Skill you can learn.” – CDA - “Strength of the program is we have support from specialists.” - dentist - “Because it has been a consistent team, it’s been good because the residents know who they’re going to see.”  - Clinical manager  5. Support from the LTC facility “To see the nurses be able to attend to nursing duties not kitchen duties or whatever, and we could then meet as a family council and discuss quality care, how it can be improved, how it can be maybe cut back” – resident’s family - “You need support from all level.” - dentist - “Have a volunteer who porters all the residents back and forth.” - CDA              Chapter 5    86 Indicators Minimal Standard Improved Practice 6. Management style - Communication platform available such as a study club - Central person/liaison: “You have to have a central command, not something that’s locked up at five o’clock…its twenty-four hour operation and somebody is there, and everybody is informed whether they want it or not [laughter].” –resident’s family - Flexible and agile - Open communication - Role clarity “Make sure that the staff have it written into their job description.” - RDH  7. Work incentives - “Rewarding feelings” - dentist - Reinforcement from peer and the positive care results. - “Professional growth, have recognition, opportunity for continuing education.” - RDH “Throwing a small party for care- aides every now and then because they have achieved target.” - admin - “A sense of camaraderie, a sense of belonging, a sense of pride in coming to work.” - RDH  8. Self-assessment plan - Explicit mechanism to monitor progress/impact - Ongoing monitoring and adjustment - “Sharing evaluation findings across sites” - nurse - “The accountability has to be built in…and there has to be a reporting and where, down to the care aide level they can see, ‘my goodness, this is what I’ve done and this is the impacts and these are the impacts that we’re seeing from facility to facility to facility’; and so that everyone has a sense of, across PHC the behaviour of the staff has resulted in these outcomes” Then you share [the practice and outcome] with the directors of care for each facility and then you share those results with the nursing leaders, and then you share them with the floor nurses and then the care aides and it has to be at that level of intensity.” - RDH  Chapter 5    87 Table 5.2 Performance indicators and examples of standard of acceptability                  according to the case study.     Indicators Minimal Standard Improved Practice 1. Satisfaction with care process - Patient-centred care  - Care with patience and flexibility: -“When the patients fight back, Dr. [name] just goes with the flow. You have to be very patient and flexible.” – CDA - Care with respect: “A lot have to do with touch, smile, and voice.” - CDA - “Dr. [name] has a very good hand and with making contact and even if [the resident] are not there.” - CDA - “I have watched some of the doctor from the standing up talking to this poor little heap of person down there not coming down to their level. Not touching them or sort of well, ‘open up your mouth so I can look in there’. Well that’s not going to work with dementia and Alzheimer’s.” - CDA - “If I went up to the patient and from the top and say well, ‘I’m taking you now, we’ll look at your mount and we’ll check how many teeth you have’; that’s not the way, you have to gently touch them. Half of them are in sleep anyway and you say ‘is it okay?’ and try to…BE KIND…basically.” - CDA - Personalized care involving family participation - Complaints from families, if any, are being addressed  - Timely care - On call system in place: “which puts my mind at rest” – resident’s family - “Attend to an urgent problem” and  “Timely appointments for people in pain” - nurse - “No complaint from the residents or families.” -nurse - “To be accessible and attentive to issues the nurses identify” - dentist - “…how many of the patients’ request were dealt with appropriately, quickly.” - admin - “No complaint from staff” – admin - Service “runs smoothly” – nurse  - Regular visits by dental personnel: “When things happen like dentures break or somebody else has a sore tooth it’s nice to know that the dentist will be here the next week.” - nurse - “Now somebody says oh, I’m having pain with my denture, oh no problem, we’ll have the dentist look at it next week.”  - nurse - off hour care: “Dr. [name] came and they took care of this patient on the weekend, during lunch break, or in the evening” - nurse Chapter 5    88  Indicators Minimal Standard Improved Practice 2. Productivity - Frequency - Seen by any dental professionals at least once a year -  “Repetition…always follow-up” to reinforce - nurse  - “The number of times the service is accessed and number of consult requests in balancing act with cost” - Every three month education/motivation - Reach - All new residents who give consent - All residents who have dental complaint - All consented residents - All nursing staff including casual staff/paid companion - Families who visit regularly - Scope - Clinical oral examination - Emergency care  - Comprehensive services including education and motivation - “That’s a very good service because it’s a full dentistry you can do everything there.” - nurse 3. Daily oral care/mouth- care assessment & plan  - Initial assessment by RN - Documentation of assessment  - Using standardized Minimum Data Set - “Documentation of assessment on admission…within 3 months” - nurse - Oral care plan - Written in “Hygiene” or “special need” section in the residents’ individual care plan - mouth-care explicitly indicated in the residents’ care plan  - Daily screening  - Based on a chief complaint only - Everyday by RN or RCA - “a periodic standardized assessment build it into their [nursing staff] assessments”             Chapter 5    89   Indicators Minimal Standard Improved Practice 4. Mouth-care products and reminders Mouth-care products: - Ordering and storage - Fit the unique characteristic of LTC - “Do you have what’s needed on the floor?” - According to individual needs - Inclusive (tooth brush, tooth paste, denture brush, denture cup, mouth rinse, etc.) - Standardized - Evidence-based - No outside products brought in (over the counter products)  - Distribution and placement - Hygienic placement - Product on sink/in wash room - “The right products with the corresponding card”  - Use - Evidence of usage on an audit day (e.g. the toothbrush was wet) - Denture labelled - Products are used everyday - Change products as required by the guideline - Denture labelled and stored dry Mouth-care reminders: - Wall chart - Individualized mouth care card  Incorporated into ADL wall chart - Flow sheet - Mouth-care included in the flow sheet of residents who need special attention (under hygiene or special needs).  - Incorporated mouth-care section into RCAs daily flow sheet - “Whether a flow sheet actually capturing what you want in a timely manner and is it inclusive enough to have everything that you want or it could be a piece of paper that the RNs start and the dental hygienist finishes.” - nurse - “Checklist being used to start a dialogue for accountability.” - RDH 5. Knowledge and attitude regarding oral health and mouth-care - How to do mouth-care - “Pathways of diseases” - nurse - “Know as a norm that it [oral health] is important” - RDH “Nursing Staff have trained eyes and image of disease and consequences in mind.” – RDH “Change people’s perspectives about how important it [oral health] is” – RDH  Chapter 5    90    Indicators Minimal Standard Improved Practice 6. Performance of mouth- care - “Getting mouth care on a regular basis a regular pattern [anytime of the day]” - Nurse - “No resident should be going to sleep without some cleaning of the mouth, whether that’s even at the most, at the minimum level a rinsing of the mouth, flushing out the mouth debris.” - RDH - “There ought to be no resident who after their last meal of the day is not given the opportunity or there isn’t a process to facilitate mouth care, to clean that mouth.” - RDH - “Every RCA does [or assists] mouth care on every resident”  - nurse - “Twice a day morning/before going to bed” -nurse  7. Integration between     dentistry and LTC - “…not to be someone who parachutes in and then parachutes out” - nurse - Engage LTC staff when planning – RDH - Formal communication channels between dentistry and nursing staff - Integrate with both staff and residents’ families - “The dental team should be included within our care conference if they’ve done some major work on the person.” - nurse - At least to submit a report to the annual reviews – nurse  8. Financial accountability - Formal report per request - Formal report annually Chapter 5    91 Table 5.3 Outcome indicators and examples of standard of acceptability                  according to the case study.  Indicators Minimal Standard 1. Clinical oral health status (providers’ perspectives)    - “Just to keep things stable and preventing sort of major problems like pain and infection.”  - A dentist  - “They [the residents] are happy with the outcome…beyond their own expectations too because not everybody is aware of the dental diseases that they’re affected with.” – A Program Director  - “If you’re free of infection and free of pain, you don’t have anything that’s hurting you from chewing and swallowing, that’s probably acceptable…if you don’t have loose teeth that are going to be falling out, hurting you, I don’t think that we could say you’re going to be free from plaque, that’s not reasonable, we have plaque.” – A clinical nurse leader  2. Perceived oral health (recipients’ perspectives) - “To be free of pain” – A family member  - “No complaint” – A resident  - “Before they started this program, auntie was on heavy drugs and it turned her teeth brown. And they have got her teeth looking quite good now.” - A family member  - “Now it doesn’t hurt at all” – A resident  Chapter 5    92  Indicators Minimal Standard 3. Oral health related quality of life (from multiple perspectives) - “She’s well fed, the body is assimilating the food and the medicine and instead of maybe have toast and tea or they wouldn’t eat at all” – A family member  - “to be able to chew food” – A family member  - “If people aren’t eating and smell is the big one to be honest.” – A clinical nurse leader  - “Not to make them an extreme make over or anything like that…aesthetic is sort of not a huge priority but then definitely if that’ something they want, it’s something they should have too.” – A dentist  4. Overall satisfaction with the result of care (recipients’ perspectives) - “Is their mouth feeling as the residents think it should and if the de as [the residents think] it should?” – A clinical nurse leader  - “No complaint” – A family member  - “if families call the chief administrator that tadadadadada Dr. [name] did this or that, the administrators don’t like that…and if you got the resident complaining too.” – A dentist   Chapter 5    93  5.3 WHAT ARE THE METHODS FOR ASSESSING QUALITY OF THE        PROGRAM?  According to the theoretical framework, complex programs should be assessed comprehensively by using both quantitative and qualitative methods. Suggestions from research participants reflect also the possibility of using both methods. Nurse leaders preferred a walk-through audit or a floor audit to assess daily mouth-care activities as one of them explained:  “One thing I do whenever I can is I just ask the RCA, can you take me around your 6 residents, tell me what kind of mouth-care each residents got. Just have general discussions…if [it was] pretty late in the day and I get [response like]… ‘Oh! I haven’t done it yet. I’ll do it now’ [laugh] so I’d know. Some of the care-aides are proud of taking you around tell you what they did for each resident. It’s a bit scary and it’s another quality indicator if people say, ‘oh I don’t know if she’s got her own teeth or false teeth’ when you know it’s near the end of her shift. That’s a bit worrisome too.” The same nurse leader also shared that,  “There’s one facility I worked in that did have a peer audit around basic things…‘did the person [resident] look clean and attractive’ and ‘did the person have their nails trimmed’ and care-aides would audit each other and it’s possible…there could be some [audit] system like mouth-care but we have to be very careful because you have to be in a very mature work environment to do that.”  The nurse explained that peer audit would not work in facilities where there were high tensions among the care-aides or between care-aides and management. Chapter 5    94 Consequently, the program should also be assessed by obtaining information from other sources such as the recipients of care. The nurse explained how, “We could also identify a group of residents who were able to talk [about] whether they’ve got their mouth care on a regular basis”  However, the same nurse elaborated:   “Unfortunately in this population, those people [who are able to talk] are often the people who clean their own teeth or look after their own needs, so you wouldn’t capture the right group…families would be a group to survey too [for this information]”   In fact, the program case study (GDP) has once conducted a satisfaction survey with families of the residents in 2006 (Appendix I). However, it concerned all components of the program except daily mouth-care. Moreover, the GDP also conducted a staff satisfaction survey in 2005 (Appendix I) as requested by the hospital organization and regional health authority. Both surveys used a questionnaire with 5-scale Likert-type response and a few open-ended questions. The questions reflected mostly the performance and outcome categories. Other assessments regarding educational in-services and daily mouth-care conducted by the GDP include mouth-care protocol audits by a dental hygiene educator and a master’s student who had produced an improved audit sheet (Appendix J). In addition, as part of the ongoing documentation, the GDP also keep information on 1) oral health status using CODE software22; 2) treatment recommended; 3) consent sent and received; 4) treatment provided (based on procedure instead of visit) and 5) financial records using Axium clinical management software23  22 CODE by UBC ELDERS group 23 Axium by Exan Academic, Inc. Chapter 5    95 and MS Excel spreadsheet24. These databases provide systematic information on the performance and clinical outcomes of the program. One of the dentists stated that this systemic and computerized data collection was one of the strengths of the program. He explained: “All the data collected during the exams and during the recall exams…you can follow up patients…of how many more filling have they needed after that initial treatment ...have they lose more teeth at that time? The problem with that would be…there’s a high patient turn over rate…so you can’t do a long-term follow-up [with the same patients] but if you were to take a sample of patients who were there for specific period of time and say 3-5 years…I think there is probably the possibility of doing it.”  Also, there is a possibility of observing the residents’ conditions to assess the final outcomes regarding oral health and oral health-related- quality of life. Another clinical nurse leader asserted, “Certainly sometimes you see the residents wince and grimace when they’re eating and that would indicate that they might have some pain.”   In summary, information about quality of the program comes from three main sources: 1) clinical records and administrative records; 2) interview and/or questionnaires with providers, elderly residents, and their family members; 3) observation of both providers and elderly residents. Lastly, how evaluators interpret the findings of the assessment is another important step. Participants agreed that quality of the program is reflected in a “balancing act” of multiple indicators and multiple methods of data collection. As one of the dentists pointed out:  24 Excel spreadsheet by Microsoft Office Chapter 5    96 “I saw a frail lady who needs a clearance. Clearance is a big deal. And she has no pain and no acute infection. If you only go survey that institution, that will skew everything. It’s so bad that you have got 30 decayed surfaces in that mouth and 8 teeth indicated for extraction...Skew the whole picture…what that would do to the results…You’re dead.”  Similarly, one of the dental hygienist asserted, “You have to weight a number of things out…What I do is I correlate my findings …I might be doing a clinical assessment then speaking to the nurses and the care aides and also doing some audits.”  He further explained,  “For instance, even if they said they do it [mouth-care], the toothbrushes are twiddled down or they’re dry or the products are not even in the room and so, you know, it leads one to suspect.”  Indeed, a model for assessing quality of a program this complex needs multiple steps (Figure 5.4). The empirical evidence confirmed evaluation theory that, to meaningfully assess the quality of the program, key people should be engaged to comprehensively describe the program. Moreover, participants need to, at the very beginning, state the goals of both the services and the assessment itself. The evaluator then can use the information to set appropriate assessment questions specifically to the program’s circumstance and stage of development. Later, the assessment findings consisting of a comprehensive program description and reports of met and unmet quality indicators, can then be used to adjust the program design, educate and motivate participants. In addition, experiences from the case study showed that not only the performance of the program needs Chapter 5    97 improvement; but its capacity might also be enhanced. Then, the iterative practice of assessment and adjustment of the program should increase the likelihood of improved outcomes. Chapter 5    98 Figure 5.4 Theoretical and modified versions of steps of quality assessment.                          Theoretical framework of steps of quality assurance          Empirical evidence from a program case study    Chapter 5    99 In conjunction with the previous flow chart displaying steps of quality assessment and improvement (Figure 5.4), I have also drawn a PDQI diagram guiding how to comprehensively describe a program and where to look for elements indicating quality of the program (Figure 5.5). The combination of these two parts (Figure 5.4 and 5.5) then represents “a model for assessing quality of a comprehensive oral health program in LTC facilities”. In conclusion, the model consists of 6 steps, 6 components of program description, and 20 quality indicators. The 20 quality indicators came from 8 capacity and 8 performance categories, and the last 4 quality indicators came from the outcome category. Chapter 5    100 In summary the model for assessing quality of a comprehensive oral health program in LTC facilities include: 1) Discussing with the key participants regarding general purpose of program assessment; 2) Describing the program by following the Program Description (PD) part of the PDQI diagram (Figure 5.5); 3) Forming specific assessment questions according to information gathered from step 1) and 2); 4) Collecting information guided by the Quality Indicators (QI) part of the PDQI diagram (Figure 5.5); 5) Interpreting and reporting findings in terms of met and unmet indicators according to the minimal standards selected. 6) Making recommendations to strengthen the quality elements of the program and improve unmet elements by adjusting program design as well as motivate and educate participants.  Next, application of this model to an oral health program Northern BC will be described and discussed in Chapter 6.  101 C hapter 5  Figure 5.5 A PDQI (Program description and Quality Indicators) model for assessing quality of a                   comprehensive oral health program in LTC facility.   Chapter 6    102 CHAPTER 6  TESTING A MODEL FOR ASSESSING QUALITY OF DENTAL GERIATRIC PROGRAMS  6.0 INTRODUCTION This chapter describes an application of the model of quality assessment developed in chapter 5 to an oral health program for older adults living in six long- term care (LTC) facilities in Northern British Columbia (BC), Canada. The primary objective was to evaluate the feasibility and usability of the model drafted. The secondary objective was to explain how the model was revised following application in another “real-world” setting. I selected this particular setting because the program provides care to the elders in a different context and circumstances and operates quite differently from the original program used to develop the model. I organized this chapter into three main sections: 1) the experience of model application, which are laid out according to the five steps of quality assessment; 2) an analysis of feasibility and usability; and 3) modification of the model following the assessment. Chapter 6    103 6.1 THE APPLICATION OF THE MODEL:  6.1.1 State General Purpose of Assessment and other Preparations After being asked by one of the founders of a geriatric dentistry program in Northern BC to review their program, I met with him in person in January 2007 in Vancouver. He briefly explained the program had been in operation for nearly ten years but had never been evaluated by external people before. Therefore, the general purposes of this review were: 1) to document as evidence the program delivery model which they believed was worth sharing; 2) to “ try to figure out where we [the program] are?”  and 3) “how should we [the program] proceed?” said the co- founder. In turn, I described the assessment methods, selection criteria of interviewees, and a list of administrative documents required. The founder of the program then informed the current Program Manager of our assessment plan. We used snowball sampling starting from the program manager to identify and recruit other interview participants. Shortly after the meeting, I received an email from the Program Manager who compiled contact details of all potential participants. I then e- mailed out a letter of introduction informing them of the objectives of this review, methods of information collection, and the ethical consideration employed. I was immediately able to schedule the interviews with most of the potential participants except for the facility administrators and staff who requested an additional list of interview questions. Consequently, I emailed every potential participant a list of potential interview questions and a summary table displaying the program components of my interest (Appendix K) created by adapting information from the PDQI tabular matrices. The information seemed to reduce potential participants’ Chapter 6    104 concerns and promote participation. Out of 17 identified people, only one administrator at the regional health authority refused to participate in the study. This selection of administrators and service-providers offered a wide range of experiences and perspectives on how the oral health program operates and on how the recipients of care respond to it. 6.1.2 Describe the Program Prior the field study, I compiled contextual information about the city, the local healthcare system, demographics of aging population in the city, existing LTC facilities and future assisted living projects, and news articles found on the internet about the program. In Northern BC, I began the fieldwork by conducting a two-hour interview with two co-founders of the program. I asked them about how the program was initiated, the context of the program, the details structures and operations of the program. And most importantly I asked them to articulate the goals they had set for the program and for the assessment itself. The information gathered were categorized and written according to the predetermined categories of the program’s context, structure, processes, and desired outcomes. One of the co-founders said, “This is like walking down the memory lane”. I found that they constantly triggered each other’s memory and validated each other’s account. The “pair interview” seemed to enhance the amount and also accuracy of the information provided. 6.1.3 Form Specific Assessment Questions I then used the information gathered from the first two steps to determine the stage of program development and the readiness of the program to be evaluated. Chapter 6    105 Subsequently, I reassessed the suitability of the evaluation questions posed by the participants by comparing them with the current circumstance of the program. The program at the time of study was in the mature stage of implementation (see Chapter 4 page 60), where the operation had been stable for many years. In other words, without external forces such as restructuring of LTC facilities, the structures and processes of the oral health program would remain unchanged. Therefore, the main objective of the assessment was to identify how the program can ensure its quality. Specifically, the objectives of the assessment were to 1) establish evidence of accomplishment; 2) identify how the program can be further improved; 3) share lessons learned; and 4) initiate the cycle of quality monitoring and establish a baseline for future evaluations. Consequently, specific assessment questions I formed were as follows: - 1) To what extent has the program achieved the desired outcomes? - 2) What are the areas which need improvement? - 3) What are challenges encountered and possible solutions? - 4) What are success factors and what are pitfalls? - 5) What are the next-action plans? 6.1.4 Collect Information Guided by the Quality Indicators  Throughout February and March 2007 I applied the model to review a program case study by conducting a 4-day site visit to 5 LTC facilities, interviewing 15 key people25 involved in the development and current operation of the program.  25 The participants include 2 founders of the program; a program manager; 3 dentists working in the program; a dental hygienist; a former certified dental assistant and program Chapter 6    106 After obtaining the informed consent, the author interviewed 14 participants either individually or in pairs for about one hour each; and one participant was interviewed by phone for 20 minutes. I also compiled observational field-notes and kept an activity log during the fieldwork. Due to the time constraints, this study did not involve recipients of care. In all, the three categories of quality indicators provided me with a clear and inclusive guide for points of data collection. I was able to identify where the case study fell short and comprehensively document what they have and successfully accomplished. 6.1.5 Interpret and Report Results  The 24-page report (Appendix L) is divided into five sections: I. Introduction II. Program description a) Community context and program history b) Partner organization and targeted population c) Program Structure d) Program Process: how it works e) Program desired outcomes III. Forming the evaluation questions  IV. Findings and interpretations  a) Progress and accomplishment to date  b) Lessons learned: success factors and pitfalls   coordinator; a current certified dental assistant and program coordinator; 2 facility administrators; 2 registered nurses; and 2 licensed practical nurses. Chapter 6    107 VI. Conclusion and recommendations for the next action plan   The full report as well as a PowerPoint presentation summarizing the report was sent back to the Program Manager and founders. The Program Manager also sent additional statistics regarding the financial aspects of the program operation as well as helped revise some of the wording in the report and slide presentation. The revision is mostly semantic. For example, the participants had asked me to find alternative wording for reporting that purchasing the panoramic x-ray “was a mistake”26. They gave the reason that, although the machine “was not working” for geriatric population, it was still utilized at the regional hospital for other population group. Moreover, in the slide presentation there was also information comparing the data regarding productivity and the model of delivery between the Northern BC program and the UBC program (Appendix M). It was presented at a Geriatric Dentistry Study club in Vancouver and Northern BC.  26 The term was extracted from an interview transcript. Chapter 6    108 6.2 TESTING THE MODEL FOR ITS FEASIBILITY AND USABILITY  The model for assessing quality of the program should adhere to fundamental characteristics such as feasibility and usability in their application. Therefore, my model was tested against a set of feasibility and usability criteria endorsed by several leading quality management agencies such as the National Quality Forum (NQF), the Agency for Healthcare Research and Quality (AHRQ), the Joint Commission on Accreditation of Healthcare Organizations, and the National Committee for Quality Assurance in the U.S. and also the Canadian Institute for Health Information (IHI) (73). Feasibility  The NQF defines feasibility as “the extent to which a measure can be implemented without undue burden (financial and human) and in a manner that allows for auditing or verification of results”. Practically, this means that the information used to assess a program should be readily and affordably available under typical operating conditions, and it should be considered with due regard for the confidentiality of everyone involved. Usability   The NQF defines usability as, “the extent to which intended audiences can understand the results of the measure and are likely to find the results useful for decision-making.” The criteria endorsed include: 1) Understandable and clear presentation with a dissemination strategy Chapter 6    109 2) Compelling content and useful information for decision-making (e.g. performance differences are practically and clinically meaningful; and recommendations are actionable). 3) Having high potential for working well with other indicators currently in use [in LTC] and can be interpreted and useful in the accreditation process 4) If the assessment is conducted using quantitative measures, the differences in performance levels must be statistically meaningful (statistical tests can be applied to communicate differences in performance levels greater than chance) 6.2.1 Feasibility of the Model in a “Northern” Context   Please note that the application of my model under the context of doctoral research project was not entirely realistic in terms of the cost of the assessment. In the usual circumstance, the implementation of the model would perhaps involve also a consultant fee and associated travel costs. I was granted a small amount of financial support from the UBC, Faculty of Dentistry’s Wah Leung Endowment fund to cover the flights and working hours which was approximately C$4,000. During my visit, I stayed at the program founder’s house. Therefore, if the assessment is to be conducted by an evaluation consultant, it would cost more. Consequently, I am not able to claim whether the model is financially feasible to implement. However, I believe that 4 day-site-visits and one month of interviews and document analysis has yielded sufficient information to assess the quality of a program of this size. In addition, I was able to keep the identities of all research participants confidential. The names of the long-term care facilities were replaced by a pseudonym. Chapter 6    110 Moreover, I keep all the interview tapes, transcripts, field-notes, and related documents in a case folder for the usual audit-trail required by my research method. I had expected to find a careful record of clinical records associated with the program; however, these were not accessible in a format (e.g. electronic) that could be reviewed easily. Instead, I was able to determine that the process used by clinicians to record clinical information was unstructured and difficult to analyze. The program founder told me that there was a plan to adopt an electronic record but, according to a founder of the program: “We haven’t got quite the manpower… we want to, we’ve tried, we do have a laptop here… the problem is time.”  In addition, the Program Manager told me that:  “We would like to take on a[n electronic] charting system… but we’ve struggled with it… we were wondering how it would actually benefit us, and how we would actually use it because it is quite time-consuming.”   I can conclude from these conversations with the founder and the Program Manager that the information missing from my assessment was either impractical to retrieve, or that the model is impractical in requesting this information. Nonetheless, I believe that it is reasonable to expect some indication of clinical outcomes. Therefore, I remain disposed to the view that the model should continue to seek this information. Moreover, an omission of clinical assessment from a program’s infrastructure does not necessarily indicate that the program is inadequate, but, in this case, it identifies the limitations of the program’s ability to review its achievements and improve its clinical outcomes as well as its performance. For example, the program cannot recall every resident for an annual check-up because Chapter 6    111 there is no electronic record, and it relies on the dental hygienist to review the chart manually and then to remind the dentist. Consequently, the residents do not get examined by a dentist regularly unless they have occasion to see the dental hygienist, or they have made a specific complaint to the nursing staff. On the other hand, missing information for the quality indicators of daily mouth-care and oral health education stemmed from a lack of action of the dental personnel rather than from a poor record system.  Therefore, the assessment model was able to identify a practical and appropriate need for improvements in prevention of disease and in oral health-related education.  In general, the field observations and interviews went well and the implementation of the assessment was well received by the research participants. Note that due to time constraints, I did not interview the recipients of care. Experience from interviewing and conducting a satisfaction survey with recipients of care in Vancouver, and literature review (87) reveals that acquiring such information is indeed feasible. Nonetheless, I found that the standard of some of the quality indicators (or sub-categories) were too specific and potentially irrelevant to a program in a different setting. For instance, “recipients of care must be well-informed [about the procedures]” is one of the standards for “patient-centred care” (subset of “satisfaction with care process”). In Vancouver, no treatment can be delivered unless signed consent for treatment and financial responsibility are mailed back to the office. In Northern BC, formal consent is not an issue as the cost of all basic dental treatments is covered. And even for treatment not covered by the plan, the dentists usually discussed the plan over the phone with the next of kin. Dental professionals Chapter 6    112 in Northern BC are not at all concerned so signed consent is not practiced. Also, the program does not usually contact the patient’s next of kin for basic treatment either. Research participants all felt, as a dentist asserted, that: “[Consent] is not an issue here, and there has not been any problem”.  The differences in practices also raised the issue of unclear benchmarks in geriatric dentistry. There is scientific evidence available to suggest standards of mouth-care products, dental materials, and clinical procedures; but for other components of the program, we simply do not know where the bar should be.  For instance, there is no evidence of effectiveness regarding the wall chart or flow sheets to remind the care-aides to perform mouth-care. Similarly, there is no evidence indicating how often the dental professionals should visit a long-term care facility in order to contribute directly to improve oral health of the residents, or indirectly by being present frequently enough to raise awareness of oral health among nursing staff. Therefore, the issues of “too specific standards” and “unclear benchmarks” are dealt with and later presented in the modification of the model. 6.2.2 Usability of the Model in a “Northern” Context There are two main targeted audiences who could benefit from the implementation of the assessment model: 1) the program assessed; and 2) healthcare professionals and academics who work in similar fields and would benefit from a published article about lessons learned (report of success factors and pitfalls) and a comprehensive description of the program.  I found that the assessment model provides useful information for both target groups. However, despite compelling results (report of met and unmet indicators, Chapter 6    113 and a summary table of SWOT analysis27), and clear statements of recommendations for actionable plan, I found that without any urgency or pressures to change, it is unlikely that the program would adopt the recommendation to enhance its capacity and performance: because much of what needed improvement usually involved extra hours of brainstorming and careful planning of program re- design, perhaps additional manpower, and indeed a new budget proposal. My model, like many other quality management models, has not yet managed to go from the assessment to the improvement part of the cycle. However, I believe that by having an iterative loop of assessment and waiting for the right timing, improvement will eventually occur. At the very least, what the model has contributed now is to provide a structured and systematic way to accumulate the evidence of the organization of this type of service and its impact.  As for the usability criteria of having potential in working well with other LTC indicators, it is my hope that the model being developed based on the literature of LTC and empirical insight from LTC would in the future be translated into use by the nursing team. The only set of indicators regarding oral health that is currently in use in long-term care is the Minimal Data Set (MDS), which focuses on flagging clinical problems rather than indicating the capacity and performance of the program. In addition, I have decided to increase usability of my quality indicators for the accreditation process of the facilities by incorporating various quality dimensions advocated by the Canadian Council on Health Services Accreditation (CCHSA) into the step of forming assessment questions. I will discuss this point later in the section  27 SWOT = Strength, Weakness, Opportunity, and Threat. Chapter 6    114 describing a modification of the model.  Lastly, the model has not yet been tested using a quantitative method of data collection and analysis against the criteria that “the differences in performance levels must be statistically meaningful”. The literature review suggested that risk adjustment should be applied when comparing facilities in which the residents from each facility have different levels of frailty or severity of illness. Future research can use the categories and their subsets to develop a well thought-out satisfaction survey, which is based on both theoretical and empirical foundations. 6.3 MODIFICATION OF THE MODEL   Based on my experience with the model in the Northern program, I offer the following modifications: 1. Focus on “function rather than form”: Some of the specific sub-categories of capacity and performance indicators and their standards have been removed or the definition expanded. For instance, onsite delivery is desired but it does not matter whether the program has a fully equipped onsite clinic or a mobile clinical unit, or even a simple toolbox, if it can deliver the care needed by the residents. Similarly, the formal consent form might not be required if the recipients of care are satisfied with the care process and outcomes. 2. Change the term “minimal standards” to “suggestions for care objectives”. When empirical evidence is insufficient to determine the objectives it seems appropriate for a program to offer what the recipients of care demand and what the clinicians deem to be appropriate care, despite the controversy that this will generate (12,88). Consequently, there are not yet a widely agreed upon set of Chapter 6    115 quality standards until we conduct more studies to systematically compile the standardized information using this model (or to conduct a consensus study). 3. Incorporate the quality dimensions endorsed by CCHSA (89) to help guide the step of “forming specific assessment questions”. Therefore, the program can be assessed comprehensively or in parts according to the following dimensions: population-focused, accessibility, continuity of service, safety, worklife, client- centred, efficiency, and effectiveness (Table 6.1)  Table 6.1 Definition of the quality dimensions advocated by the Canadian                  Council on Health Services Accreditation (CCHSA) (89).  Dimensions Definition Population-focused Working with community populations and recipients of care to respond to their current and future needs and expectations, and to changes in the environment. Accessibility Providing equitable services that are easily obtained by recipients in the most appropriate setting. Safety Prevention and mitigation of unsafe acts in the care system’s delivery of services and in its organizational structures. Worklife A work environment that enables optimal individual and organization wellness and outcomes. Client-centred Care and services are respectful and responsive to needs, preferences, culture, and values of individuals, families, and communities. Continuity Coordinated and consistent care/intervention/action across the continuum of services overtime. Efficiency  Resources are allocated to achieve optimal outcomes with minimal waste, re-work, and effort. Effectiveness Services/intervention/actions are based on evidence from the current and evolving state of knowledge, and achieve the desired outcomes.   Summary of the revised model for assessing quality of an oral health program in LTC facilities is as follow: Chapter 6    116 1. Discuss the purpose of the assessment with the key people who are heavily involved in the initiation and current operation of the program. Engage them in a face-to-face interview or focus group discussion. 2. Interviews, observations, and document analysis to describe the program according to the predetermined categories proposed. This is merely to write up a program logic model and determine the evaluability of the program. To complete a comprehensive program description in a final report, information from step 4 will also be incorporated. 3. Reassess the assessment questions from information previously gathered. 4. Interviews, focus groups, questionnaire surveys, site visits, document analysis, chart review to collect information as required by the quality indicators. If the assessment would be done by the LTC facilities, which focuses only on the daily mouth-care, only relevant indicators will be used. Otherwise, a full set of capacity, performance, and outcome indicators should be used because they represent a logic model of how a program should work in order to increase the likelihood of desired outcomes. 5. Interpret the indicators met and unmet from suggestions of care standards according to the stage of the program development. The report would then be written under the headings of: overall description of the program; situational analysis; progress, accomplishment, and deficiencies (indicators met and unmet); lessons learned (success stories and pitfalls), and conclusion and recommendations.  Chapter 6    117 6. To increase the likelihood of the recommendations being adopted, I decided to highlight the dissemination of the assessment findings by adding it as another step. This step should go beyond sending the report back to the program administration but perhaps to have other face-to-face meetings or set up a community of practice online so that it would gradually increase the capacity of the program assessed to move forward. Also the standardized strategies of program assessment proposed here provide a comprehensive structure for reporting and publishing the procedures and results of the assessments which I hope will contribute to the evidence of best practice to the dental and LTC communities.  Chapter 6    118 Figure 6.1 A full cycle of quality assessment of an oral health program in a                   LTC Facility.    In conclusion, testing of the model was essential because it provided information for making a best determination as to whether the assessment steps and quality indicators would perform well when implemented in other contexts. The experience of implementing the model has shown that, with modification, it has high potential for application to similar programs elsewhere.  Chapter 7    119 CHAPTER 7 DISCUSSION AND CONCLUSION  7.0 INTRODUCTION  This chapter summarizes my research findings by linking previous chapters together and relating the model I developed to existing literature. I address the many frequently asked questions I received when I presented my findings and developments at a number of research meetings. Later, I discuss the strength and limitations of my study. Also I will explain the implications of my study; and lastly, I offer and discuss directions for future developments of this work. 7.1 SUMMARY  7.1.1 Statement of Problem My doctoral thesis started with the intention to find out how best to provide oral health services to the elderly population. A literature review revealed that we know the efficacy of many products and clinical techniques; however we do not know, under “real world” circumstances, how to organize and deliver healthcare effectively or sustainably to this expanding population. Numerous challenges within the LTC environment have been reported, and dental professionals are not at all clear about appropriate oral healthcare for this population. To respond to this particular gap in knowledge, I have argued that what we need is a tool for assessing appropriately the oral health services in LTC facilities.  This tool, if it is effective, should be able to determine whether the effectiveness or ineffectiveness of a particular program is due to the planning or the implementation process, or even a Chapter 7    120 combination of both processes. It should also help to diagnose problems associated with the program and offer reasonable recommendations for improvements. Therefore, the main goal of my thesis has been to develop an assessment tool appropriate to the complicated planning, implementation, and evaluation of an oral health program for elders in residential care. 7.1.2 Methods  I employed a systematic approach for constructing an assessment model by: 1) building a theoretical framework from the literature about concepts of health program evaluation and quality assessment; 2) collecting empirical evidence through an ethnographic case study of a recently developed dental geriatric program; 3) drafting the model by making the theoretical framework specific to dental geriatrics using the information about quality indicators and suggested program objectives gathered during the case study; and 4) testing the application of the model to another dental geriatric program operating in a geographical location remote from where I developed the model.  In essence, the theoretical framework offered a structural form for the model while the ethnographic case study provided the content. The inductive qualitative methods I used identified three major themes operating within this complex dental program from which emerged the design of the model. My method also allowed me to tweak the form, and to categorize and prioritize the content, of the model so that it more accurately represented or reflected the complexity of the program. Chapter 7    121 7.1.3 Findings  • Historical review of literature reveals that concepts of quality assessment and program evaluation have different origins and nomenclature but their core concepts overlap and are complimentary. Both concepts have their own strengths and weakness. A combination of their strengths produces a theoretical framework in the form of an iterative cycle of six sequential steps for assessing a complicated healthcare program (see Figure 3.1 Page 39). • Emerging from an ethnographic case study of the Vancouver program, there are three themes with 6 sub-themes identifying the root of the complexity: 1) programs change constantly; 2) they adapt to changing circumstances as they grow, so they are never complete; 3) dental and non-dental personnel working within the programs collaborate only superficially and require constant surveillance to achieve productive integrations; 4) the personnel have unclear roles and responsibilities in oral healthcare; 5) the factors constituting productive interaction and quality of care are massively entangled for multiple reasons; and 6) many of the factors or variables are difficult if not impossible to change because they are not under the direct control of program operations. Such is the real world in which healthcare is provided in most LTC facilities. • These complex characteristics confirm the needs for assessment that is participatory, descriptive, and comprehensive, as proposed by the theoretical framework. • Ethnography is a useful tool for collecting information about a program’s components and their interactions; and the theoretical framework Chapter 7    122 provides a lens for observing and words for describing the components and their interactions. • The term “outcome” poses different meaning to different people, consequently, I eased the semantic confusion associated with program assessment by creating three new categories of a program: capacity; performance; and outcome. • There are 20 quality indicators (see Figure 5.2, page 78) encompassing 8 dimensions of quality (see Table 6.1, Page 111) all of which enter into the assessment when analyzed against a set of references. • The most difficult part of designing the assessment was formulating the references, and I found that the best way to ensure the translation of the model with empirical guidance was to have a flexible set of standards or “suggested program objectives” that can be revised and incrementally increased. • Testing the effectiveness of the model demonstrated that it is feasible and usable for targeting and assessing appropriate outcomes. However, future investigation is required to ensure that the assessment results are translated into practice. Chapter 7    123 7.2 FREQUENTLY ASKED QUESTIONS  7.2.1 What Do You Mean by Oral Health Care?  As demonstrated by my two case studies, the term oral healthcare encompasses a comprehensive practice including daily mouth-care rendered by nurses or other caregivers, educational in-service and specific dental care provided by dental personnel. Also the term includes forming relevant policy and other population health approaches to improve the oral health related outcomes for elderly residents of LTC facilities such as the Adult Care Regulations in BC (85). 7.2.2 What Do You Mean by Quality of an Oral Healthcare?  Many researchers define the term “quality of care” exclusively from “quality of outcome” (43). However, participants from this study stated that the term quality of care in their minds includes the quality of service itself and also the quality of results of such service. Therefore, empirical information from this study confirms for me that “quality of care” is involved beyond the service organization and provision. In the literature, current criticisms of quality assurance initiatives in LTC focus also on the divide between quality of care and quality of life, which typically results in particular attention to the medically oriented components of care, and much less attention to quality of life (51). Therefore, in this assessment model, quality of oral healthcare involves all aspects of care, including the care (structure and process) and the outcomes. Moreover, the dimensions of quality of care involve also effectiveness as well as efficiency, safety, continuity, client-centeredness, community-involvement, and accessibility. Chapter 7    124 7.2.3 Is this Model Only Good for a Qualitative Evaluation?   Even though both literature review and observation from this study are in agreement that qualitative methods of data collection are particularly well suited for evaluating complex phenomena, such as health programs that include also prevention and promotion components (8,11,57); my model can be used also for quantitative evaluations. Basically, I support theory-oriented assessments, which employ any methods suitable for the research questions and circumstances. My proposed categorical items for describing program components and quality indicators should serve as building blocks for various quantitative measures, such as satisfaction surveys; oral health-related quality of life measures; or systematic frameworks for reviewing medical records. For example, program objectives under the category of satisfaction with care process suggested that the instrument should measure whether or not the care is: 1) well-informed; 2) personalized; 3) encouraging family participation; 4) offered with respect; 5) responsive; and 6) timely. 7.2.4 Do I Have to Always Conduct a Comprehensive Assessment?  Yes – to capture the synergetic effects of different elements of a complicated program. However, the model can also be applied selectively for each intervention. For example, the facility administrator and nurse leader may choose to conduct only an assessment of the daily-mouth care components, especially where there is no formal dental geriatric program available in the area. Or when the program is at the early stage of development, assessment can be done predominately on the program Chapter 7    125 description (a logic model) and identification of the capacity of the program to further plan the development. For example, the purpose of assessment when I tested the model in the northern community was basically for the participants in the program “to figure out where we are”, and to find the way to establish evidence of its operation and accomplishments. 7.2.5 What is the Evaluation Standard? What is Appropriate Care?   Despite my attempt to explain the approach I developed, when I presented at a conference, some conference participants still asked, “should we have some sort of minimal standard for care?”. I argued that, yes, there should be some community standards of care, but they are distinct from minimal standards, which were formed originally in LTC to quickly identify a need of urgent attention to a more comprehensive investigation and to counter the adverse effects of substandard care (43,51). Application of minimal standards, for example, can reduce the use of physical restraint and abuse in LTC facilities. However, successful application of the standards depends upon public attention and reaction. Oral health of elders, on the other hand, appears by most accounts to attract little public attention. Therefore establishing minimal standards of oral care alone is not likely to gain much success or change of current practice. Also, research participants from this study were not able to and did not agree to formulate any fixed set of minimal standards. Certainly, it is possible to articulate a minimal standard focused on examples of unmet need that have been reported by the dental professionals. However, the problem of inadequate oral health care has been reported for three decades already with little evidence of improvement (90). My model, in contrast, is based on the principle that Chapter 7    126 the assessment should help “improve” the quality of care rather than merely identify substandard problems. Practicing at the level of a minimal standard can be seen as a sanctioning of inadequate care. The American National Commission for Quality Long-term Care (51) explained that this approach fails to address the full range of unmet needs and poor care in the community. Consequently, there are at present no widely agreed-upon standards for oral healthcare in LTC facilities, and they are unlikely to emerge until we conduct more studies systematically to compile more evidence. My model should help to guide this systematic compilation. 7.2.6 Is Anybody Else Using a Similar Assessment Approach?  The steps and approach guided by my model are in accordance with the literature on assessment of other health programs such as the Australian partnership programs (91) and the American integrated service initiatives (92) They both require a non-traditional evaluation to suit the collaborative effort and convergence of multiple healthcare disciplines. Similar approaches, such as theory-based evaluation (60,82) have been proposed to engage an array of perspectives and methods for collecting quantitative data and qualitative information. Indeed, current trends for assessing complex programs28 have forms similar to my model; (81,82) but they have yet to appear in dentistry or most other fields within the mainstream of healthcare practice. Similarly, only a few authors (88,93,94) have touched upon the concept of realistic treatment needs, treatment intentions, and the coping and  28 Atame (82) define a complex program as programs that have many implementers, different activities, different beneficiaries aiming at creating synergies that will produce results that any single initiative would not have brought about in isolation. Chapter 7    127 adaptation ability to impairment (95). As explained by the IoM in the USA, much of the health service in LTC cannot be considered in the same context as heroic rescue-type clinical outcomes. The dilemma of outcome research in LTC facilities, especially at the level of extended care, stems from the fact that the health of the elderly residents is deteriorating at different rates and will continue to deteriorate despite best practice care. This does not mean that practitioners should ignore the need to assess outcomes of care. But, as my research has shown, the assessment demanded is different than the assessment applied to acute care interventions and remedies. My model is an assessment tool applicable to the various aspects that constitute the uncertainties of chronic care along with the different expectations of everyone involved. I agree with the IoM that, by using the appropriate assessment tool for the right outcomes, research can help motivate and raise expectation by showing what can be achieved for and by frail elders. 7.2.7 Why a Strong Conceptual/Theoretical Foundation is Important for this Problem?  This question stems from a common misunderstanding that theory is ideal but somehow irrelevant to practice. Theory, according to McKenna (96), is an articulated system of concepts and proposition, and building blocks of both knowledge and practice, that begins in a practice-based problem and returns again to inform the practice. The importance of conceptual/theoretical work is to present a view of the world that helps to describe, explain, or predict events. It can also prescribe actions that will help events to occur (96). Theory offers a lens or a map for problem-solvers, and the lack of theory underlying most oral health programs in LTC facilities probably Chapter 7    128 explains why they do not operate very effectively (18). During the past few decades, researchers seem to solve problems on a functional level by achieving a desired outcome, “as a matter of doing it”, rather than “as a matter of knowing” (96). Rarely have they questioned the theoretical fundamentals of the problem to describe and explain phenomena to provide a clear lens or map to guide solutions (97). As demonstrated in the literature review (Chapter 1 and 3), the challenge remains of establishing a comprehensive and testable theory of how an oral health program works. The intent of my research has been to address this lack of theory.  7.3 STRENGTHS OF THE STUDY  I began this study with a thorough examination of the root of the problem rather than selecting a particular evaluation method (e.g. longitudinal, before and after experiment) to produce knowledge with limited applications. I challenged the traditional practice of pursuing model of care delivery through experimental evaluation, and I shifted the emphasis to go beyond “proving” inadequate oral healthcare and identifying obstacles to “improving” care through an effective assessment tool. This study is systematic in that the model was drafted with both theoretical and empirical guidance. As far as I know, it is the first time that the concepts of theory-driven program evaluation and quality assurance have been integrated into a single model for assessing a health program. The model brings the strength of both concepts to address the various issues of a complicated program, such as I found in the two dental programs I analyzed. Chapter 7    129 Moreover, the indicators and care objectives I propose emerged from the experiences and values of the people I interviewed who were closely involved in the program operations, either as providers or recipients of care. The study includes also the experiences and opinions of dental assistants and care-aides. This approach contrasts with many other approaches such as Delphi consensus that rely substantially if not wholly on the experiences and perspectives of clinicians and academic experts. I tested the transferability of the model by applying it to an outreach dental geriatric program in a relatively small metropolitan community in Northern BC. Moreover, the philosophy behind the model and the qualitative inquiry allowed me to design a flexible model for assessing an oral health program appropriate to various LTC circumstances; and, at the same time, structured to direct the evaluator efficiently to relevant information. The fact that this model can accommodate change in most of its components is particularly helpful in a demographic climate where so much change is occurring. The need and demand of future cohorts will almost certainly be different than those that I encountered. In addition, the incremental increase in the program’s objectives should ensure quality monitoring (if not improvement), along with an adaptability to address changes at different times, in different contexts, and at different stages of development. This approach also helps the care providers who are the subject of the assessment to proactively and dynamically deal with uncertainty without being concerned that their unremitting struggles will go unappreciated. Also, this model employs a new classification and definition of the capacity, performance, and outcome of a program, which should clarify the confusion of terms used traditionally Chapter 7    130 in both program evaluation (i.e. the various definition of the term impact) (60,98) and in quality assessment (i.e. unclear distinction between structure and process) (77). Application of the model encourages the assessors to silently force the program evaluation and planning to go hand in hand. Use of the term “assessment” rather than “evaluation” should help to dilute the current fixation on summative evaluations. Consequently, an assessment can be included in the planning and implementation process without undue threat or concern to the planners and implementers. As for the assessment of accomplishment, quality measurement in LTC is often criticized for having standards that do not recognize excellence in care (e.g. maximizing physical functioning or well-being). Instead, most quality indicators capture problems and adverse outcomes (e.g. pressure sores, pain, malnutrition). Similar concern has been raised about the current MDS (99).29 There is one small section relating to oral health and all 7 indicators of “oral/dental status” seek information about problems such as “some/all natural teeth lost”, “broken tooth”, “debris”,  “inflamed gums”, and “ulcers”. My model is instead based on the principle of a positive reinforcement rather than a sanctioning of bad practices. It offers to go beyond the neglected area of practice to capture positive performance and outcomes, and to indicate future possibilities. And lastly, the rigor of this study runs through out the three steps of the inquiry (see Chapter 2 page 28-30)  29 The Minimum Data Set (MDS) is a standardized process for clinical assessment of all residents in LTC facilities. This process provides a comprehensive assessment of each resident's functional capabilities and helps nursing staff identify health problems. Chapter 7    131 7.4 LIMITATIONS OF THE STUDY   This study aimed to produce a model for assessing any oral health program in a LTC facility rather than assessing either of the two programs that served as the source of my enquiry. Consequently, I did not report the tips and tricks of how to run the programs that emerged during my enquiry. Nor can I offer information on what it might cost to pay for such an assessment, because I did not experience or observe the financial costs expected by a consultant assessor.  Although possible, there is cost also to retrieving clinical records especially when they are not electronic-based. In addition, the model was tested with a small program in an intimate community of dental and medical professionals where the assessment was received with enthusiasm and support. In other types of settings or organizational arrangement, the application of the model might be more cumbersome to administer, and fraught with controversy and even resistance.  The model has emerged from and is tested with oral health programs in BC and needs to be refined in other jurisdictional contexts and social settings. For example, more quality indicators and certain program objectives might be included or adjusted to accommodate other public or governmental values. The model at present, for example, does not emphasize the perspectives of the taxpayers, because dentistry in Canada is not part of the federal healthcare expenditure. Similarly, the model is limited to dental geriatrics but with some wording adjustments the model does have a broader application in similar types of partnership programs especially in the situation where the program in the healthcare institution is contracted out. Further application and modification of the model will make it more Chapter 7    132 useful in much the same way as the P/P model (84) were developed beyond the initial framework. 7.5 IMPLICATIONS   This study indicates quality of the program structure, activities, and results, when considering all of the natural complexities of a LTC environment. It offers a comprehensive tool for assessing an oral health program in LTC where multiple perspectives are engaged. Future application of the model in other jurisdictions can provide evidence to build a body of knowledge on how these types of programs work and in what circumstance. The information can be used to inform decisions about managerial and clinical practices. It can also help to guide resource allocation. Full application of the model implies quality monitoring and improvement of a comprehensive oral health program. However, parts of the model, such as quality indicators and specific treatment objectives, can be adapted to create survey instruments examining satisfaction of care recipients and audit tools of daily mouth- care. Another example includes using outcome indicators to guide the revision of future MDS (99). Also, parts of the model can help formulate observation and interview tool to investigate oral health related quality of life of cognitively impaired elders. In conclusion, this study contributes to LTC dentistry by giving a tool for: 1) documenting and testing existing programs for improving oral healthcare; 2) developing and evaluating new programs (organizational innovations); and 3) providing evidence that help redefine fundamental goals of the services (shaping the definition of appropriate oral health care for frail elders). Chapter 7    133 7.6 DIRECTIONS FOR FURTHER RESEARCH  As stated earlier, the model requires further testing and modification. I will certainly apply the model in Thailand by first conducting a situational analysis of the oral health and LTC system there. I will use the framework for program description to document and use quality indicators to assess existing oral health promotion and disease prevention activities as well as therapeutic programs nation-wide.  At present, all programs are organized at community/senior centers and community hospitals. Therefore, I would like to extend the application of the model to frail elders who are homebound and do not access existing programs. I will document baseline information about community and healthcare system capacities using approaches and point of data collection guided by the model. The information gathered should help to make the current version of the assessment model more complete and to make the quality indicators and the suggested program objectives applicable to different cultures. Modifications or additions to the model are needed to translate assessment findings into action. As demonstrated by the program in Northern BC, the inertia and lack of structural capacity seems to inhibit further use of the assessment findings. Certainly, appropriate assessment and targeting alone do not result in quality care unless they are linked through the post-assessment planning process to the implementation of adjusted program design, education and motivation (78). For the most part, formal research on how to motivate, sustain or diffuse quality improvement in the LTC setting is lacking (43,51). A scoping review, including “grey” literature and interviews with experts on the innovations involving program design, Chapter 7    134 motivational and educational strategies, as well as relevant regulations and healthcare markets will certainly be useful (51). And once more and more studies can demonstrate the effectiveness of existing programs (that have never been assessed appropriately), their accomplishment will build up stronger evidence of how programs work, which in turn should help reinforce changes elsewhere.  7.7 CONCLUDING REMARKS    As a profession, dental practitioners and researchers around the world often advocate good oral health in terms of good biological functions of the mouth. This stance becomes more complicated when applied to frail elders in LTC and consequently poses challenges to outcome assessment and quality assurance of the services. Having more teeth is positive but they pose certain risks as old age almost certainly leads to impairment in self-care. Oral comfort and social function maybe more important to the person than mechanical function (94). Therefore, it is quality of life that drives most people of all ages to seek dental treatment and healthcare, and it is quality of life that needs to be a major focus of oral healthcare programs. This project has helped me to understand the concept of (oral) health as an instrument or resource for living (10). Reconsiderations of what actually constitute quality of an oral health program provide a more suitable assessment framework especially when the values of multiple groups of participants are considered. This thesis is another step in an effort to shift the focus from identifying problems and obstacles providing oral health care to systematically developing effective and sustainable programs for maintaining a dignified and satisfying quality of life until death. References    135 REFERENCES (1) MacEntee MI. Significance of the Mouth in Old Age. Soc Sci Med, v.45 n.9 (1997): pp.1449-1458. (2) National Institute on Aging. Why Population Aging Matters: A Global Perspective. Available at: http://www.nia.nih.gov/ResearchInformation/ExtramuralPrograms/BehavioralAndSocialRese arch/GlobalAging.htm. Accessed 6 May, 2008. (3) Kiyak HA, Grayston MN, Crinean CL. Oral health problems and needs of nursing home residents. Community Dent.Oral Epidemiol. 1993 Feb;21(1):49-52. (4) Chalmers JM. Geriatric oral health issues in Australia. Int.Dent.J. 2001 Jun;51(3 Suppl):188-199. (5) Berkey D, Berg R. Geriatric oral health issues in the United States. Int.Dent.J. 2001 Jun;51(3 Suppl):254-264. (6) Wyatt CC. Elderly Canadians residing in long-term care hospitals: Part I. Medical and dental status. J.Can.Dent.Assoc. 2002 Jun;68(6):353-358. (7) Wyatt CC. Elderly Canadians residing in long-term care hospitals: Part II. Dental caries status. J.Can.Dent.Assoc. 2002 Jun;68(6):359-363. (8) Petersen PE, Yamamoto T. Improving the oral health of older people: the approach of the WHO Global Oral Health Programme. Community Dent.Oral Epidemiol. 2005 Apr;33(2):81-92. (9) Ettinger RL. The unique oral health needs of an aging population. Dent.Clin.North Am. 1997 Oct;41(4):633-649. (10) Green LW, Kreuter MW, Green LW. Health program planning : an educational and ecological approach. 4th ed. New York: McGraw-Hill; 2005. (11) Patton MQ, Patton MQ. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks, Calif.: Sage Publications; 2002. (12) Bryant SR. Ethical issues encountered by dentists in the care of institutionalized elders. 1994 1994. (13) Rise J, Heloe LA. Oral conditions and need for dental treatment in an elderly population in northern Norway. Community Dent.Oral Epidemiol. 1978 Jan;6(1):611. (14) Berkey DB, Berg RG, Ettinger RL, Meskin LH. Research review of oral health status and service use among institutionalized older adults in the United States and Canada. Spec.Care Dentist. 1991 Jul-Aug;11(4):131-136. (15) Kiyak HA. Psychological and social factors in the dental care of the elderly. Int.Dent.J. 1983 Sep;33(3):281-291. References    136 (16) Limeback H. The relationship between oral health and systemic infections among elderly residents of chronic care facilities: a review. Gerodontology 1988 Winter;7(4):131- 137. (17) Reisine ST, Fertig J, Weber J, Leder S. Impact of dental conditions on patients' quality of life. Community Dent.Oral Epidemiol. 1989 Feb;17(1):7-10. (18) MacEntee MI, Thorne S, Kazanjian A. Conflicting priorities: oral health in long-term care. Spec.Care Dentist. 1999 Jul-Aug;19(4):164-172. (19) Kronstrom M, Palmqvist S, Soderfeldt B, Vigild M. Utilization of dental health services among middle-aged people in Sweden and Denmark. Acta Odontol.Scand. 2002 Oct;60(5):276-280. (20) Dolan TA, Atchison KA. Implications of access, utilization and need for oral health care by the non-institutionalized and institutionalized elderly on the dental delivery system. J.Dent.Educ. 1993 Dec;57(12):876-887. (21) Vigild M. A model for oral health care for elderly persons in nursing homes with an estimate of the resources needed. Acta Odontol.Scand. 1989 Aug;47(4):199-204. (22) Vigild M. Oral health care programmes for elderly in Scandinavia. Int.Dent.J. 1992 Oct;42(5):323-9; discussion 330-3. (23) Berkey DB, Ela KM, Berg RG. Advances in portable and mobile equipment systems. Int.Dent.J. 1993 Oct;43(5):455-465. (24) Koerber LG, Kelly SA. A community-based program to facilitate dental care for senior citizens. J.Indiana Dent.Assoc. 1993 Jul-Aug;72(4):18-23. (25) Henry RG, Ceridan B. Delivering dental care to nursing home and homebound patients. Dent.Clin.North Am. 1994 Jul;38(3):537-551. (26) Slade GD, Strauss RP, Atchison KA, Kressin NR, Locker D, Reisine ST. Conference summary: assessing oral health outcomes--measuring health status and quality of life. Community Dent.Health 1998 Mar;15(1):3-7. (27) Atchison KA, Dolan TA. Development of the Geriatric Oral Health Assessment Index. J.Dent.Educ. 1990 Nov;54(11):680-687. (28) MacEntee MI, Wyatt CC. An index of clinical oral disorder in elders (CODE). Gerodontology 1999 Dec;16(2):85-96. (29) Dolan TA. Identification of appropriate outcomes for an aging population. Spec.Care Dentist. 1993 Jan-Feb;13(1):35-39. (30) Frenkel H, Harvey I, Needs K. Oral health care education and its effect on caregivers' knowledge and attitudes: a randomised controlled trial. Community Dent.Oral Epidemiol. 2002 Apr;30(2):91-100. References    137 (31) Chalmers J, Pearson A. Oral hygiene care for residents with dementia: a literature review. J.Adv.Nurs. 2005 Nov;52(4):410-419. (32) MacEntee MI, Wyatt CC, Beattie BL, Paterson B, Levy-Milne R, McCandless L, et al. Provision of mouth-care in long-term care facilities: an educational trial. Community Dent.Oral Epidemiol. 2007 Feb;35(1):25-34. (33) Petersen PE. Effectiveness of oral health care--some Danish experiences. Proc.Finn.Dent.Soc. 1992;88(1-2):13-23. (34) Paulsson G, Soderfeldt B, Nederfors T, Fridlund B. The effect of an oral health education program after three years. Spec.Care Dentist. 2003;23(2):63-69. (35) Watt RG, Harnett R, Daly B, Fuller SS, Kay E, Morgan A, et al. Evaluating oral health promotion: need for quality outcome measures. Community Dent.Oral Epidemiol. 2006 Feb;34(1):11-17. (36) Vigild M. Evaluation of an oral health service for nursing home residents. Acta Odontol.Scand. 1990 Apr;48(2):99-105. (37) Petersen PE, Nortov B. Evaluation of a dental public health program for old-age pensioners in Denmark. J.Public Health Dent. 1994 Spring;54(2):73-79. (38) Mojon P, Rentsch A, Budtz-Jorgensen E, Baehni PC. Effects of an oral health program on selected clinical parameters and salivary bacteria in a long-term care facility. Eur.J.Oral Sci. 1998 Aug;106(4):827-834. (39) Vigild M, Brinck JJ, Hede B. A one-year follow-up of an oral health care programme for residents with severe behavioural disorders at special nursing homes in Denmark. Community Dent.Health 1998 Jun;15(2):88-92. (40) Wyatt CC, So FH, Williams PM, Mithani A, Zed CM, Yen EH. The development, implementation, utilization and outcomes of a comprehensive dental program for older adults residing in long-term care facilities. J.Can.Dent.Assoc. 2006 Jun;72(5):419. (41) Yoneyama T, Yoshida M, Ohrui T, Mukaiyama H, Okamoto H, Hoshiba K, et al. Oral care reduces pneumonia in older patients in nursing homes. J.Am.Geriatr.Soc. 2002 Mar;50(3):430-433. (42) Brondani MA, MacEntee MI. The concept of validity in sociodental indicators and oral health-related quality-of-life measures. Community Dent.Oral Epidemiol. 2007 Dec;35(6):472-478. (43) 20 Institute of Medicine . Committee on Nursing Homes Regulation. Improving the quality of care in nursing homes. Washington, D.C.: National Academy Press; 1986. (44) Petersen PE. The burden of oral disease: challenges to improving oral health in the 21st century. Bull.World Health Organ. 2005 Jan;83(1):3. References    138 (45) Petersen PE. Continuous improvement of oral health in the 21st century: the approach of the WHO Global Oral Health Programme. Zhonghua Kou Qiang Yi Xue Za Zhi 2004 Nov;39(6):441-444. (46) Chalmers JM, Levy SM, Buckwalter KC, Ettinger RL, Kambhu PP. Factors influencing nurses' aides' provision of oral care for nursing facility residents. Spec.Care Dentist. 1996 Mar-Apr;16(2):71-79. (47) Chalmers J, Johnson V, Tang JH, Titler MG. Evidence-based protocol: oral hygiene care for functionally dependent and cognitively impaired older adults. J.Gerontol.Nurs. 2004 Nov;30(11):5-12. (48) Schoen MH. Quality of dental care. Curr.Opin.Dent. 1991 Jun;1(3):322-328. (49) Jones JA, Kazis LE, Maas WR. Oral health for aging veterans. Making a difference: priorities for quality care. Introduction. Med.Care 1995 Nov;33(11 Suppl):NS3-5. (50) Guay AH. The oral health status of nursing home residents: what do we need to know? J.Dent.Educ. 2005 Sep;69(9):1015-1017. (51) Kohler PO, Wunderlich GS, Institute of Medicine . Committee on Improving Quality in Long-Term Care, Institute of Medicine . Division of Health Care Services. Improving the quality of long-term care. Washington, D.C.: National Academy Press; 2001. (52) Yamalik N. Quality systems in dentistry part 2. Quality assurance and improvement (QA/I) tools that have implications for dentistry. Int.Dent.J. 2007 Dec;57(6):459-467. (53) Landesman, Anne Murphy, Mary Richards, Joy Smyth, Jean Osakue,Ben. Oral hygiene in the elderly: a quality improvement initiative. Canadian journal of nursing leadership 2003;16(3):79. (54) Searle JR. The construction of social reality. New York: Free Press; 1995. (55) Kvale S. Interviews : an introduction to qualitative research interviewing. Thousand Oaks, Calif.: Sage Publications; 1996. (56) Green LA, Seifert CM. Translation of research into practice: why we can't "just do it". J.Am.Board Fam.Pract. 2005 Nov-Dec;18(6):541-545. (57) Plsek PE, Wilson T. Complexity, leadership, and management in healthcare organisations. BMJ 2001 Sep 29;323(7315):746-749. (58) Fischer F. Beyond Empirism: Policy Inquiry in Postpositivist Perspective. Policy Studies Journal 1998;26(1):129-146. (59) Checkland P. Systems thinking, systems practice. Chichester Sussex ; New York: J. Wiley; 1981. (60) Weiss CH, Weiss CH. Evaluation : methods for studying programs and policies. 2nd ed. Upper Saddle River, N.J.: Prentice Hall; 1998. References    139 (61) Glouberman S, Zimmerman B, Commission on the Future of Health Care in Canada. Complicated and complex systems : what would successful reform of Medicare look like? 2003 2002:30. (62) Green LW. From research to "best practices" in other settings and populations. Am.J.Health Behav. 2001 May-Jun;25(3):165-178. (63) Kao RT. The challenges of transferring evidence-based dentistry into practice. J.Evid Based.Dent.Pract. 2006 Mar;6(1):125-128. (64) Savage J. Ethnography and health care. BMJ 2000 Dec 2;321(7273):1400-1402. (65) Creswell JW. Qualitative inquiry and research design : choosing among five traditions. Thousand Oaks, Calif.: Sage Publications; 1998. (66) Denzin NK, Lincoln YS. The handbook of qualitative research. 2nd ed. Thousand Oaks, Calif.: Sage Publications; 2000. (67) Richards L. Handling qualitative data : a practical guide. London: SAGE Publications; 2005. (68) Coffey A, Atkinson P. Making sense of qualitative data : complementary research strategies. Thousand Oaks: Sage Publications; 1996. (69) Yin RK. Case study research : design and methods. 3rd ed. Thousand Oaks, Calif.: Sage Publications; 2003. (70) Miles MB, Huberman AM. Qualitative data analysis : an expanded sourcebook. 2nd ed. Thousand Oaks: Sage Publications; 1994. (71) Institute of Medicine . Committee on Quality of Health Care in America. Crossing the quality chasm : a new health system for the 21st century. Washington, D.C.: National Academy Press; 2001. (72) Atkinson P. Handbook of ethnography. London ; Thousand Oaks, Calif.: Sage; 2001. (73) Agency for Healthcare Research and Quality. Evaluation of the Use of AHRQ and Other Quality Indicators. 2007; Available at: http://www.ahrq.gov/about/evaluations/qualityindicators/qualindsum.htm. Accessed 7 May, 2008. (74) Palmer RH, Donabedian A, Povar GJ. Striving for quality in health care : an inquiry into policy and practice. Ann Arbor, Mich.: Health Administration Press; 1991. (75) Donabedian A. The quality of care. How can it be assessed? 1988. Arch.Pathol.Lab.Med. 1997 Nov;121(11):1145-1150. (76) McLaughlin CP, Kaluzny AD. Continuous quality improvement in health care: theory, implementations, and applications. 3rd ed. Sudbury, Mass.: Jones and Bartlett; 2006. References    140 (77) Closs SJ, Tierney AJ. The complexities of using a structure, process and outcome framework: the case of an evaluation of discharge planning for elderly patients. J.Adv.Nurs. 1993 Aug;18(8):1279-1287. (78) Donabedian A. An introduction to quality assurance in health care. Oxford ; New York: Oxford University Press; 2003. (79) The W.K. Kellogg Foundation. W.K. Kellogg Foundation: Overview - Evaluation. 1998; Available at: http://www.wkkf.org/default.aspx?tabid=75&CID=281&NID=61&LanguageID=0. Accessed 7 May, 2008. (80) Bobby Milstein, Scott Wetterhall, CDC Evaluation Working Group. A Framework Featuring Steps and Standards for Program Evaluation. Health Promotion Practice 2000 July 2000;1(3):221. (81) Pirkis J, Herrman H, Schweitzer I, Yung A, Grigg M, Burgess P. Evaluating complex, collaborative programmes: the Partnership Project as a case study. Aust.N.Z.J.Psychiatry 2001 Oct;35(5):639-646. (82) Stame N. Theory-Based Evaluation and Types of Complexity. Evaluation 2004 January 1;10(1):58-76. (83) Donabedian A. The seven pillars of quality. Arch.Pathol.Lab.Med. 1990 Nov;114(11):1115-1118. (84) Graham NO. Quality in health care : theory, application, and evolution. Gaithersburg, Md.: Aspen Publishers; 1995. (85) Ministry of Health Services, Government of British Columbia. Community Care and Assisted Living Act -- Adult Care Regulations. 2005; Available at: http://www.qp.gov.bc.ca/statreg/reg/c/commucareassisted/536_80.htm#section9.2. Accessed 6 May, 2008. (86) Donabedian A. Criteria and standards for quality assessment and monitoring. QRB Qual.Rev.Bull. 1986 Mar;12(3):99-108. (87) Pruksapong M, MacEntee MI. Quality of oral health services in residential care: towards an evaluation framework. Gerodontology 2007 Dec;24(4):224-230. (88) Mojon P, MacEntee MI. Estimates of time and propensity for dental treatment among institutionalised elders. Gerodontology 1994 Dec;11(2):99-107. (89) The Canadian Council on Health Services Accreditation. CCHSA’s revised dimensions of quality. 2007; Available at: http://www.cchsa.ca/default.aspx?page=51&cat=34. Accessed 7 May, 2008. (90) MacEntee MI. Missing links in oral health care for frail elderly people. J.Can.Dent.Assoc. 2006 Jun;72(5):421-425. (91) Pirkis J, Livingston J, Herrman H, Schweitzer I, Gill L, Morley B, et al. Improving collaboration between private psychiatrists, the public mental health sector and general References    141 practitioners: evaluation of the Partnership Project. Aust.N.Z.J.Psychiatry 2004 Mar;38(3):125-134. (92) Marquart, Jules M., Ed., Konrad, Ellen L., Ed. Evaluating Initiatives to Integrate Human Services. New Directions for Evaluation 1996 Spr 1996;69:1-103. (93) Ettinger RL. Rational dental care: part 1. Has the concept changed in 20 years? J.Can.Dent.Assoc. 2006 Jun;72(5):441-445. (94) Isaksson R, Soderfeldt B, Nederfors T. Oral treatment need and oral treatment intention in a population enrolled in long-term care in nursing homes and home care. Acta Odontol.Scand. 2003 Feb;61(1):11-18. (95) Brondani MA, Bryant SR, MacEntee MI. Elders assessment of an evolving model of oral health. Gerodontology 2007 Dec;24(4):189-195. (96) McKenna H. Nursing Theories and Models. New York: Routledge; 1997. (97) MacEntee MI. An existential model of oral health from evolving views on health, function and disability. Community Dental Health 2006 Mar;23(1):5-14. (98) Hopson RK, American Evaluation Association. How and why language matters in evaluation. San Francisco: Jossey-Bass; 2000. (99) The Centers for Medicare & Medicaid Services. Minimum Data Set: Overview. 2008; Available at: http://www.cms.hhs.gov/NursingHomeQualityInits/01_Overview.asp#TopOfPage. Accessed 6 May, 2008.   Appendices  142 APPENDICES  Appendix A: Certificate of Research Ethics Board Approval     Appendices  143 Appendix B: Letter of Initial Contact     Appendices  144 Appendix C: Interview and Observation Guides  C1. Interview Guide (Care recipients)  Research Title: A Model for Assessing the Quality of   an Oral Health Program in Long-Term Care Facilities Research questions: 1. What are the components and their relationships within a comprehensive oral healthcare program for LTC facilities? 2. What factors influence the outcome of the program? 3. What outcomes must be considered and in what way when assessing the quality of the program?  General Information: 1. F Resident       F Next of kin (Relationship to the resident __________________) 2. Name of participant (in code):  _________________________________________ 3. Date of interview:  _____________________________ 4. Interview start time:  ________________ Interview end time:  ________________  Total time for interview:  _____________ minutes ………………………………………………………………………………………………Outline the purpose of the study and the parameters of the discussion (length, audio taping, transcribing), and assure confidentiality.  Questions: Participant’s background: • Introduction • Service received. 1. Opinions on the program and experiences implementing it: • Initial involvement in the program • Opinions about the program and expected outcomes Follow-up: • Strengths and weaknesses of the program • Barriers to receiving services • Emerging issues – resolved and unresolved • Future needs 3. Opinions on assessing the quality of the program • Desired outcomes of the program   • Assessing the program for an acceptable quality of service   • Experience in assessing the quality of the program.   [Summarize the interview, and provide opportunity to elaborate on any issue raised, and to explore other items or concerns not raised. Offer possibility of further discussions and a follow-up interview to validate the interpretation and to elaborate.]   Appendices   145  C2. Interview Guide (Care Providers)  Research Title: A Model for Assessing the Quality of    an Oral Health Program in Long-Term Care Facilities Research questions: 1. What are the components and their relationships within a comprehensive oral healthcare program for LTC facilities? 2. What factors influence the outcome of the program? 3. What outcomes must be considered and in what way when assessing the quality of the program?  General Information: 1.   F Nurse  F Care-aide  F Dental personnel  F others       (specify:_____________________) 2.   Name of participant (in code):  _________________________________________ 3.   Date of interview:  _____________________________ 4. Interview start time:  ________________ Interview end time:  ________________  Total time for interview:  _____________ minutes ………………………………………………………………………………………………Outline the purpose of the study and the parameters of the discussion (length, audio taping, transcribing), and assure confidentiality.  Questions: Participant’s background • Position or title • Job duties and responsibilities • Work experiences 1. Opinions on the program and experiences implementing it: • Initial involvement in the program • Opinions about the program and expected outcomes Follow-up: • Strengths and weaknesses of the program • Barriers to organizing services • Emerging issues – resolved and unresolved   • Significant changes (i.e. funding, specific services, staffing, etc.)     If yes, describe what happened, and what prompted the change.   • Adaptability of the program to changes and the participant’s involvement. • Future needs 3. Opinions on assessing the quality of the program • Desired outcomes of the program   • Assessing the program for an acceptable quality of service    • Experience in assessing the quality of the program. 4. Other topics not already raised. • Attitudes toward work • Work plan, income and life-style expectations.  [Summarize the interview, and provide opportunity to elaborate on any issue raised, and to explore other items or concerns not raised. Offer possibility of further discussions and a follow-up interview to validate the interpretation and to elaborate.]   Appendices   146 C3. Interview Guide (Administrators)  Research Title: A Model for Assessing the Quality of      an Oral Health Program in Long-Term Care Facilities Research questions: 1. What are the components and their relationships within a comprehensive oral healthcare program for LTC facilities? 2. What factors influence the outcome of the program? 3. What outcomes must be considered and in what way when assessing the quality of the program?  General Information: 1. F Government    F UBC   F LTC facility   F Others (specify:_________________) 2. Name of participant (in code):  ________________________________________ 3. Date of interview:  _____________________________ 4. Interview start time:  ________________ Interview end time:  ________________     Total time for interview:  _____________ minutes ………………………………………………………………………………………………Outline the purpose of the study and the parameters of the discussion (length, audio taping, transcribing), and assure confidentiality.  Questions: 1.   Participant’s background: • Position or title • General duties and responsibilities • Role in the oral health care program 2.  Opinions on the program and experiences implementing it: • Initial involvement in the program • Opinions about the program and expected outcomes Follow-up: • Strengths and weaknesses of the program • Barriers to organizing services • Emerging issues – resolved and unresolved   • Significant changes (i.e. funding, specific services, staffing, etc.)     If yes, describe what happened, and what prompted the change.   • Adaptability of the program to changes and the participant’s involvement. • Future needs 3. Opinions on assessing the quality of the program • Desired outcomes of the program   • Assessing the program for an acceptable quality of service    • Experience in assessing the quality of the program.   [Summarize the interview, and provide opportunity to elaborate on any issue raised, and to explore other items or concerns not raised. Offer possibility of further discussions and a follow-up interview to validate the interpretation and to elaborate.   Appendices   147 Appendix D: Observation Field-note and Protocol  D1. Observation/Field-note Protocol Research Title: A Model for Assessing the Quality of   an Oral Health Program in Long-Term Care Facilities Research questions: 1. What are the components and their relationships within a comprehensive oral healthcare program for LTC facilities? 2. What factors influence the outcome of the program? 3. What outcomes must be considered and in what way when assessing the quality of the program?   General Information:  1. Name of observer (in code):  _____________________________________ 2. Name of observation site: _______________________________________ 3. Date of observation:  __________________  4.   Observation start time:  ________________ end time:  ________________   Total time for observation:  _____________ minutes    Description Interpretation/Reflection (Including what I don’t see, What I don’t understand) Setting Including diagrams of places   Actors-Activities/  Interesting Conversation  Participants will be identified in code /  Both verbal and non verbal, insider’s language  Others   Bulletin boards, calendars, memos, forms, newsletters    148 Appendices D2. Example of a Field-note:  General Information: 4. Name of observation site: ___YOUVILLE ___ 5. Date of observation:  Friday Oct 27th, 2005  3.   Observation start time:  ___9:00AM__ end time:  _____12:30PM____   Total time for observation:  __3.5 Hours__  Description and Interpretation/Reflection Setting Same room but the lift was installed, autoclave was brought in..  Dental clinic is between a hair salon and a meeting room. The meeting room is opposite a rehab room.  On the door, there’s a poster from CDS BC promoting oral health in old age…it was a photo of an old gentleman in with swimming cap and glasses and towel and the msg was to keep going to see the dentist twice yearly. “LOOK MA, Still no Cavities” – “After all, you still want to make mom proud”. Actors- Activities [See details from a day sheet re: # of patients, procedures] I ran up and down 2,3,4,5 FL and the basement to escort residents back and forth between their rooms or dining hall to dental clinic. Only 2FL that the residents are in special care and we need a key from a staff to call the elevater. For stairs, there’s a code (0649*)) to get out and to get in every floor through stairs, we need to push a red button by the door.  The schedule was flexible because the residents might not be ready. They may have breakfast, some may still in bed, some is sick. I try to have continual flow of bringing the patients down and also not to have them wait for more than 5 minutes in front of the dental clinic.  Today we had to rebook 3 cases: 1. in a hospital, 2. has Urinary Tract Infection and was sleeping on dining chair after having breakfast. When I tried to wake her up she said, “can’t I just sleep longer” so I told the care-aide that it’s ok we’ll reschedule the appointment. Let her rest. 3. in bed, uncooperative. He said to me, “No, I’ve seen a dentist twice already”.  The residents normally have lunch at 12:30 but today when we walked up to see the patients who could’t come down, they started serving lunch since 12:15.  Today I’ve met one of the family members, Mrs. G., that I will interview her. She’s also in a family council and she comes to see her younger sister (dementia) on the weekend and on some special occasions such as today that she would like to attended the dental appt with her sister. I got to talk to her because she told me the doctor gave Mrs. B Ativan at 9:15, so when I went back to their room again at 10:10, I told her that we still have 5 minutes before we go down to the clinic because the dentist was still seeing another patient so she asked me to sit down and we started talking. I told her who I was and finally asked her if I could interview her. She said yes and told me about the workshop re: dementia at Van Dusen garden that she learned a lot and she even made a photo copy of the document and give to nurse CS. I learned from the CDA later that she remembered this patient because she was very agitated but once her sister shown up she was all smiling and cooperate. Mrs B often say, “I wanna go home”    149 Appendices Example of a Field-note (Continue):  Description and Interpretation/Reflection   “Go home” which reminds me of the first patient who had Alzheimer’s I saw in Thailand and also my grandma when she’s not quite with it…she often said, “take me home”. Mrs. B was schedule to received impression for new CUD and CPLD but after Dr. #1 examine her oral condition, we found out she has had puree food and it is questionable whether or not she could adapt to the new denture so Dr. #1 told them that it might not be a good idea to have a new denture. Mrs. G was the one who requested the denture but after talking to Dr. #1 she understands too that this is the most appropriate way to go. Before she left, she thanked us and said,” You’re all very nice, God bless you all.“ I can sense that The dentist and the CDA was happy with the interaction they had…The CDA became all mellow and very gentle. In the elevator, Mrs. B was smiling. Mrs. G told her, I wish you would give Dr. #1 this smile when we were downstairs. When we’re back in her room, the care-aide came and ask if Mrs. B had a bowel movement...she directly asked twice, loud but very nicely and gently, “Do you have a bowel movement?” and to my surprise Mrs. B replied, “I’ll try”. Like in that moment she’s completely cognitive. As Mrs. G said, the more we know about dementia, the more we see that it’s very complicated.  There’s another family member today accompany the patient down. But this daughter of a patient didn’t know there was a dental appointment though. I was waiting for the elevator with her dad and she came in at the right time so we went down to the basement together. She said she used to be a volunteer here 20 years ago. She remembered the gift shop used to be in the basement. This gentleman was completely cognitive impaired. He can’t communicate at all but he likes owls and the daughter brought him an owl doll today…he holds it in his hands all the time. Others   In the elevator, there’s a notice [catering and retailing service] to family members that even though there’s a job action of food service (Sodexho labour action), there will be no effect to the residents. However, there’ll be no food for visitors and volunteers. The dentist was asking for a foam mouth prop, which I think it’s very handy but there wasn’t any today.   Appendices   150 Appendix E: Consent Forms (for both studies in Vancouver and in Northern BC)    Appendices   151    Appendices   152  Appendix F: Adult Care Regulations   B.C. Reg. 536/80 O.C. 2539/80 Filed November 27, 1980 Community Care and Assisted Living Act ADULT CARE REGULATIONS [includes amendments up to B.C. Reg. 278/2005, September 8, 2005] Oral health 9.2 (1) For the purposes of this section, dental health care professional means a person who is a member of (a) the College of Dental Surgeons of British Columbia, (b) the College of Dental Hygienists of British Columbia, or (c) the College of Denturists of British Columbia. (2) A licensee must encourage a person in care to obtain an examination by a dental health care professional at least once every year. (3) A licensee must ensure that a person in care is assisted in (a) maintaining daily oral health, (b) obtaining professional dental services as required, and (c) following a recommendation or order for dental treatment made by a dental health care professional providing care to the person in care. [en. B.C. Reg. 329/97; am. B.C. Reg. 217/2004, App. 1, s. 12 (d).]   Appendices   153 Appendix G: Caries Management Program     Appendices   154 Appendix H: UBC/PHC Daily Mouth Care Protocol.   H1. Routine Mouth Care Protocol for Conscious Resident      Appendices   155 H2. Routine Mouth Care Protocol for Unconscious Resident or Resident Unable to Swallow.      Appendices   156 H3. Specialized Mouth Care Protocol to Identify Oral Disorder by the Nursing Staff.        Appendices   157 Appendix I: Questionnaires for Satisfaction Surveys I1. Satisfaction Survey with Families of the Residents.  The purpose of this survey is to assess your satisfaction with the Geriatric Dentistry Program and to identify the components the program that need improving. Your responses will be treated as confidential and returning the form implies consent for the data to be used in analysis and publication.  Instructions:  Please select the most appropriate response by marking the box with an X.  If the question is not applicable to you, mark the box “N/A”. Return the completed questionnaire in the envelope provided.  Please indicate whether you agree or disagree with the following statements Strongly Agree Agree Neutral Disagree Strongly Disagree N/A 1. I found the written information provided (consent letter) to be adequate to make a decision concerning consent for treatment  2. I was able to adequately have any questions answered regarding the planned treatment, costs, and provided treatment  3. I believe that the dental care provided was appropriate  4. I believe the care was provided in a timely manner  5. I believe my relative was treated with respect  6. I believe the cost of the care was appropriate  7. I am satisfied with the clinic setting where the care was provided  8. I am satisfied with the dentist or dental hygienist who provided the care  9. I believe that my relative’s oral health has improved as a result of the care provided  10. I believe that my relative’s quality of life has improved as a result of the care provided  11. I am satisfied overall with the Dentistry Program     Appendices   158  Please select the response that comes closet to your own feelings Yearly exam by dentist Access to emergency treatment to fix dental problems Treatment to fix teeth Professional teeth cleaning Other N/A 12. What dental service do you value most?  13. What dental service do you feel needs improvements?    Additional written feedback concerning the Geriatric Dentistry Program:     Appendices   159 I2. Satisfaction Survey with Nursing Staff      Appendices   160 Appendix J: Mouth-care Protocol Audits by a Dental Hygiene Educator and a Master’s Student      Appendices   161 Appendix K: Letter of Introduction to the Northern Program  K1. Email of Initial Contact to the Northern Program  Dear [Program Manager],  I am writing this email to let you and the potential interview participants know a little more about this study. So please feel free to forward this email to all the potential participants.  Each participant will be asked different set of questions according to their roles relating to the program. But the overall goal of this study is to systematically compile information from interviews and direct observations - in order to formally document as a case study the evolution of the program, the organization, governance, and delivery of oral healthcare practices connected with the PGGODP.  As part of my doctoral thesis, I am also interested in exploring this very difficult question of how do we define success for a program such as this? Specifically, how best to evaluate this type of program when there are so many complex factors involved?  People who work within the program and in the long-term care facilities are experts. Close observations in the real settings would help us better understand how best to address this issue.  Please ensure all potential participants that they can also contact me directly should they require more information. In addition, the interview will be tape-recorded. The interview is voluntary and participants' identities will be kept confidential. We will use code names in any report we will publish.  Also, in the next couple of months, participants will be contacted again to comment on the interpretation and seek missing information.  Thank you again for your help.  Regards,  Matana       Appendices   162 K2. List of Potential Interview Questions and a Summary Table Displaying the Program Components of Interest Given to the Northern Program   OBJECTIVE AND RESEARCH QUESTIONS  Objective:  To describe as a case study the organization, governance, and delivery of oral healthcare practices connected with the PGGODP.   Key Interview Questions: Participant’s background  • Initial involvement in the program • Participant’s role in this program  Opinions and experiences with the program implementation  ƒ What are the structures and functions of the program? (See table 1) ƒ Strengths and weaknesses of the program ƒ Facilitator/barrier to providing services ƒ What are the factors influencing the outcomes and perceptions of providers and recipients of care? ƒ Significant change in the program (i.e. funding, staffing model, etc.).     If yes, describe what happened, and what prompted     the change. ƒ How do providers and recipients of care define success? ƒ key lessons learned? ƒ What participant like to see happen    Appendices   163  Categories of the program components Category of study factor Sub-category/specific case study questions Program goal/objectives Explicit/implied? # of staff, roles and responsibilities Role of public health personnel vs. private personnel Organizational structure and culture of LTC facilities Organization and practice management Level of integration between LTC facility and dentistry  Administrative processes (e.g. consent, scheduling, referral,  billing, record keeping)  System of payment and Reimbursement schemes (Capitation, salary, fee-for-service, mixed?)  How the different components of care are paid for?  How utilization and cost are monitored and managed? Types of services provided Mode of delivery Care strategies/Technology and equipments used Physical access and transportation Level of participatory Reach and exposure of each service Care delivery Perceptions of care providers How does an organization define and measure program effectiveness? Report? Patient outcomes? Program effects Perceptions of care recipients re: care experience Geography Socio-economics Culture Politics Dynamics of the context  Legislation/Ethics       Appendices   164 Appendix L: Evaluation Report of the Northern Program          The Prince George Geriatric Outreach Dental Program (PG GODP)       Program Description, Evaluation, and Recommendations                Matana Pruksapong  ELDERS Research Group UBC, Faculty of Dentistry 2199 Wesbrook Mall Vancouver, British Columbia Canada Tel   (604) 822-8879 Fax  (604) 822-3562 Appendices   165 Abbreviations   PG GODP    The Prince George Geriatric Outreach Dental Program  LTC      Long-term Care  UBC, ELDERS  University of British Columbia, Elders’ Link to Dental Education, Research, and Services  PGRH Prince George Regional Hospital  RN Registered Nurse  RCA  Residential Care Attendant  CDA Certified Dental Assistant  RDH Registered Dental Hygienist  Appendices   166 I. Introduction  The Prince George Geriatric Outreach Dental Program (PG GODP) provides basic dental care free of charge to over 310 elderly residents of five long-term care (LTC) facilities in Prince George, British Columbia. This report presents findings from a comprehensive analysis of site visits and interview data gathered from various key players of the program.  Purposes The purpose of this report are to: 1) describe the program and its development; 2) identify specific evaluation questions suitable for the stage of the program     development; 3) describe and explain evaluation findings; and 4) make recommendations for ongoing quality improvement.  Phase of evaluation This program evaluation is the PG GODP’s first external review. Essentially, I began the evaluation by exploring with the participants the goals that they had set for the program and its evaluation. This preliminary discussion offered a basis for developing suitable evaluation questions, and initiated the cycle of quality monitoring which is a baseline for future evaluations.  Methods Program evaluation in Geriatric Dentistry is difficult because there has been either standard on appropriateness of care or indicators for quality of care. UBC, ELDERS research group has proposed a framework for program evaluation specifically for oral health services in LTC facilities, which emphasizes on comprehensiveness of program description, the use of multiple quality indicators, as well as engaging various people involved in the program to participate in the Appendices   167 evaluation processes (Figure 1). The author of this report, who is a dentist and trained in qualitative research methods, conducted a 4-day site visit to 5 LTC facilities and interviewed 15 participants closely involved with the program. Participants were interviewed either individually or in pairs for about one hour each. We began the evaluation by exploring the involvement of each participant, their opinions about how it operates, and their expectations for the program. Also, we asked participants to articulate the goals they had set for the program and for its evaluation. This preliminary discussion offered a basis for developing suitable evaluation questions, and a deeper understanding of the complexity of the factors involved. The staff at four of the facility gave the interviewer a guided tour to explain the structure and operation of the facilities, and the daily activities of the nursing staff and residents. At the other facility, the dentist and dental assistant/coordinator provided dental treatment for the residents in the presence of the interviewer. The interviewer also attended a breakfast meeting of community leaders at which the program was discussed. In addition, she compiled field-notes and documents such as productivity reports, advertising brochures, memos, and websites relevant to the operations to serve as background information on the aspirations, activities and impact of the program as guided by our evaluation framework.  Appendices   168 Site visits: During February 20th to 23rd of 2007; the author conducted non- participant observations at the following sites: - Rainbow Care Home - Simon Fraser Lodge - Jubilee Lodge at Prince George Regional Hospital (PGRH) - Parkside Care Home - Laurier Manor - Centre for Health Living, Northern Health Authority - Rotary Club Breakfast Meeting, Ramada Inn - Private Practices of two of the dentists involved in the program  Interview participants: Dentistry: 2 co-founders of the program (a dental hygienist & a dentist); 3 coordinators (a public health dental hygienist & 2 certified dental assistants); 3 dentists; 1 dental hygienist  Five LTC facilities: Facility A - a Licensed Practical Nurse (LPN) Facility B - an administrator and a LPN Facility C - an administrator Facility D - a Registered Nurse (RN) Facility E - a Registered Nurse (RN)  The evaluation framework helped us to identify categories and subcategories of the program that we could subject to a content analysis that encompassed a description of the program and of quality indicators pertaining to the services offered to the facilities. The description included the context in which the program operated, such as the involvement of the community and the target population. Content, on the Appendices   169 other hand, addressed the structure, processes and desired outcomes of the program. Structure relates the staffing, financing, equipment, space and location that influence the delivery of services. Processes include descriptions of management practices and clinical services including how people involved in the program interact, whereas desired outcomes pertain to not only the official written goal but also how participants articulate what they think are the goals of the program. The accomplishments of the program relate to three dimensions: capacity, performance, and actual treatment/patient outcomes. Capacity encompasses resources, knowledge, skills, administrative structures, and level of integration required by dental personnel to address oral health related concerns. Performance indicates the quality of immediate outcomes, such as availability of mouth-care products, changes in behaviour of nursing staff, and service productivity; and lastly, the patient oral health, health, and quality of life represent the final outcomes of the program.    170        Figure 1. The evaluation framework proposed by UBC ELDERS Research Group Appendices   171 II. Program Description According to the evaluation framework proposed by UBC-ELDERS Research group, a comprehensive program description comprises of 1) community context and program history, 2) partner organizations and targeted populations, 3) program structure, 4) program processed, and 5) desired outcomes of the program. 2.1 Community Context and Program History 2.1.1 Program Initiation The PG GODP was originated in 1996 by Ms. Sharon Davalovsky, a public health dental hygienist who worked at the Northern Interior Health Unit (NIHU) at the time.  She approached the LTC facilities by offering to label dentures free of charge as a way to get in the facility and indirectly communicate about oral health care with staff and residents. During that time she witnessed the evident needs for oral care and dental treatments so she contacted Dr. Richard Wilczek and Mr. Hank Beckering, the facility administrators to set up a more comprehensive oral health service, which was realized as “a Senior Outreach Dentistry Program” in 1997.  “With those three people you’ve got public health, you’ve got the community and you’ve got a facility person and they were able to meet and talk about it and make a plan” – Program Manager  Moreover, participants felt that the plan was realized because of “the good timing”. Changes in the healthcare system in Prince George were more of opportunity than threat to the program. Also, one of the residential institutions in Vancouver Island was closing down so dental chairs and equipments were made available for bargain. Besides some funds from the NIHU, start-up funding for equipments and supplies were donations from a local Rotary Club; UBC Seniors’ Lottery Fund, and PG District Dental Society. As for the coordinator’s salary, the Northern Health Authority (NHA) was the main source of fund. Equally important, there were four volunteer dentists providing service free of charge to the residents. Appendices   172 However, after one year of implementation, they realized that the program was not sustainable and the model of care delivery did not address the need and demand. 2.1.2 Issues emerged during the first year of program implementation: The PG GODP members have identified three key challenges they have encountered during the initial period of program implementation. These issues include: z The model of service delivery: It was felt that there were too much of the oral assessment and not enough actual dental treatment done during the first year. z Logistics: Due to no fully equipped dental clinic onsite, the CDA/coordinator is responsible for transporting the portable dental unit (with air compressor) and dental supplies to the LTC facilities (Figure 2).  “[t]his is a huge challenge because we’ve had taxis deliver them and one of the taxi guys wouldn’t tie the stuff down, so stuff would be spilt and it has been a bit of a headache and the CDA’s husband at this point is kind of our support guy but really we should have somebody in place [to take care of logistics] but it’s really challenging”  z  Support from LTC facilities: It was not uncommon that dental personnel were felt as outsiders who intrude into the territory of LTC staff. However, constant presence of dental team and benefits perceived by the recipients of care has helped overcome the issue. “Over time there’s a growing acceptance of the facilities of having us there, of course. The relationship wasn’t quite as cozy nor was it as friendly and as collegial as now that it’s a very good atmosphere.” – a dentist  Appendices   173 [Removed by the author for confidentiality] Figure 2.  Transportation of equipments and supplies  2.1.3 The new PG GODP design: Year 2 In 1998, the PG GODP members brainstormed and re-designed the program based on two main guiding principles. First, “make it sustainable”; and second, “it has to be simple…not cumbersome in either paperwork or some of the other challenges” “…I said, ‘you know, I teach at the College [of New Caledonia], I get paid a set fee based on an hourly compensation’...the college doesn’t have a problem finding dentists to do that…this could work the same way’…and it has been 10 years now” – a dentist The adjusted design has been implemented and in operation for 9 years and is described in details in the following section of the report. Appendices   174 2.2 Partner Organizations and Targeted Population: Residential care is a subdivision of Home and Community Care of NHA. Furthermore, NHA is divided into three health service delivery areas where Prince George is part of the Northern Interior Health (NIH) service delivery area. In Prince George, there are six LTC facilities that participate in the PG GODP (Table 1).  Table 1. Characteristics of six LTC facilities participating in the PG GODP  Facilities Number of Bed Level of Care Notes Jubilee Lodge 72 Extended care Part of PGRH Parkside 60 Extended care Simon Fraser Lodge  116 Multi-level Privately owned, contracted RCA Rainbow 15 Alternative/transition PGRH GEM Unit 16 Alternative/transition Recently added to the program in 2007 Laurier Manor  32 Assisted Living  Data suggested important heterogeneity among the target population. Some residents are frail and cognitively impaired. Others are mobile and independent. Consequently, there are also differences in the characteristics of staff working in different facilities.  “Each facility is different…I found that at some of the facilities say [name of a facility], for example, they seemed to be doing mouth-care, they’re on top of things…not that they weren’t busy but they just didn’t feel as stressed when you went in there…and at [name of another facility], mouth-care wasn’t necessarily being done but it wasn’t because they didn’t value it, it was because they had so many other things to do and so they hoped the person on the next shift would be able to do that.” – a RDH  Moreover, due to the restructuring of LTC system and changes in admission criteria, there are also changes of resident profiles within the facility as well.  Appendices   175 “I remember when I first worked at [name of the facility], they were all mobile and go on holidays and they’re more like a hotel than a long-term care. Now it’s different.”  2.3 Program Structure:  The PG GODP is a partnership program among three main groups of people: the regional health authority responsible for general public health, the dental public health unit, and the dental professionals practicing in the community (Figure 3). [Removed by the author for confidentiality] Figure 3. Key people involved in the PG GODP as of March 2007.  Similar to healthcare organizations elsewhere in the province, restructuring, renaming, and relocation of the divisions and their offices can easily cause some confusion. At the time of this study, dental public health program was restructured and relocated to be part of the NIH, Centre for Healthy Living7 instead of the NIH Unit. The program manager of the PG GODP also works full-time as a manager for the dental public health program at the Centre for Healthy Living where she received her salary. The dental program comprises of many projects e.g. young children and pain control clinic, which were funded through the NIH. Although the PG GODP is also part of the dental public health program, it operates through a different funding and administrative structures. NIH provides ongoing dental supplies∝ and supporting staff but the Home and Community care of NHA is the one providing annual funds of C$16,500 - C$17,500/year to the PG GODP. This amount includes salary for dentists and RDH who provide care services 3 hours/visit while CDA/coordinator works 5 hours/visit. The dental care providers also get $50/visit for emergency drop- in visit such as specific examination requested or consultation with physicians or family members. It also supports the PG GODP study clubs, and two Continuing Education (CE) courses provided in 2000 and 2003 to home health and residential care aides and LPN. Therefore, officially the PG GODP is embedded in the Home  7 Together with tobacco cessation and eating well programs ∝ It is however difficult to pinpoint the real cost of supplies used as the dentists often bring supplies from their own private practice as well. Appendices   176 and Community, NHA as opposed to the NIH, dental public health program (Figure 4).  “It’s their [home and community care’s] program but the dental [public health] program [as part of Centre for Healthy Living] supports them by me helping them set up the contracts…making sure that people are taken care of…I’m sort of the person if there’s anything from the geriatric outreach program that needed to connect with somebody in Northern Health [Authority]. The coordinator [/CDA] would connect with me. For instant, so and so hasn’t been paid yet because one of the dentists has the same name as a physician in town and so his check might go to the wrong place.” – Program Manager. [Removed by the author for confidentiality] Figure 4 Organizational structure of the PG GODP  2.4 Program Process: how it works  The PG GODP provides services to residents of LTC facilities on-site including clinical oral assessment and basic dental treatment. Most facilities are equipped with a portable dental chair (Figure 5). However, participants revealed that the dental chairs are no longer used because “it’s often easier to go room to room”, said a dentist. During the site visits, I found that the dental chairs were not being used and were “scooted off to the side” in storage rooms. There were only one facility where dental chair was kept in a designated room for dentistry and hairdressing; however, a dentist who goes the particular facility revealed that,  “There is a dental chair in the basement which I haven’t used for at least in a year. The patients tend to sit with their chin on their chest so however you try to lift them up they don’t seem to want to do that and if they’re in the wheelchair you’re not going to try to wheel them to the basement and try to move them to the dental chair to see them…you look if you need to do some work you say you’re coming back next month. Then I tell the nurse to leave the patient in bed because I trained in operating theatres I can work over the head of the bed, we move the bed form the wall we put pillow in the right directions and make it into our bedside surgery, use a bedside cabinet. We use to have a lamb stand and now we use DENTLITE...great thing…[the CDA] holding the DENTLITE just to retract and I don’t use suction I use lots of gauze.” – a dentist Appendices   177    Figure 5. Portable dental chair.  The four dentists go to six LTC facilities of different size. Some dentists go to two facilities while the CDA/Coordinator and the RDH go to every facility (Figure 3). Subsequently, the frequency of dentists’ visits also depends on the size of the facilities. On average, the dentists see 8-10 residents per visit. In the big facilities, the dentists visit once a month; and in smaller facilities, they visit four times a year. The RDH has 35 hours per facility per year. “we initially started out with more days there [assisted living residence] and then we asked if we could shuffle it to other places because there didn’t seem to be the same needs and fortunately these facilities are linked under the same budget so we’re able to shuffle some things…here’s a pocket of money to provide a service and then we can adapt it to where there seems to be more need.” – Program Manager   The nursing staffs now know the routine and when they see something urgent “they will get a hold of the CDA/Coordinator or sometimes they will even phone the dental office directly because they know the dentist and there is a direct communication between them.” There is no additional paperwork regarding consent. The patients give verbal consent for treatment and there is no need for consent for financial responsibility, as the residents do not have to pay. This budget provides free basic dental care offered to every resident of LTC facilities, which includes yearly oral assessment, fillings, extractions, and hygiene. Appendices   178 “Above basic dental work, the family is notified...‘this is what we want to do, this is why, this is the cost of some kinds of things they have to pay for.’”, reported the coordinator. The contract had been signed among the partners on a yearly basis until 2006, the program has set up the contract to go for three years based on a yearly amount. “The dentists are contracted for a three-hour visit and they show up, they do their thing, they fill out the charts and off they go; where the coordinator/CDA gets there early…her contract is five hours a visit so she’ll get there and set up and then clean up, then paperwork and following up with nursing staff.” – Program Manager   The CDA/Coordinator also compiles data on service productivity, schedule the appointments with nursing staff, and fill the check requisition forms, which then sent to the Program Manager. The Program Manager oversees the operation of the program and communicates and reports productivity statistics to NIH, Preventive Public Health and NHA, Residential Care (Home and Community Care).  In addition to the dental services, the PG GODP also runs a study club four times a year. The Program Manager helps set the agenda, organizes, chair the meetings and also keep the meeting minutes.  2.5 Program Desired Outcomes  The section is still part of the program description presenting how participants articulate what they think the goals of the program are, not just what was written in the official program documents. Subsequently, the desired outcomes should be distinguished from the outputs and outcomes as part of the “findings” section. As described by one of the dentists, the overall goal of the program is: “[b]asically just to provide basic care…appropriate to the patients’ concerns, the families’ concerns…and to support the [LTC] facilities” Another dentist asserted that, “we are not looking to have high-end dentistry in the facility, the goal of this type of a program is that we want them to feel comfortable so that their quality of life is better and, of course, the health [is better].”  Appendices   179 The program manager added that another main goal is that “the nurses know what to do and who to contact in a timely manner.” III. Reassess the evaluation questions:  After describing the structure, processes, and desired outcomes of the program; more specific evaluation questions can be reassessed. Although the program seems to run smoothly, lack of clear community standards for quality of the program make it difficult to assess the success of the program. The participants agreed that the goal of the evaluation was from the external person point of view “to try and figure out where we are”. After analyzing the program structure, process, and desired outcome for its evaluability; I further analyze the progress and impact of program according to the following evaluation questions:  • To what extent has the program achieved the desired outcomes? (Quality indicators met)  • What needs improvement? (Quality indicators unmet)  • What are the success factors and the pitfalls?  • What is recommended for the next action plan? Appendices   180 IV. Findings and Interpretations 4.1 Progress and Accomplishment to date  The progress and accomplishments of program are categorized into three groups of quality indicators: 1) capacity, 2) performance, and 3) patient outcome. Capacity is the actual resources, knowledge, skills, administrative structures, and level of integration required by the program to effectively address oral health related concerns in LTC facilities; performance indicates the quality of processes and outputs (immediate outcomes) of the program; and lastly, the patient oral health, health, and quality of life represent the final outcomes of the program. Quality Indicators: a) Capacity:  The program was assessed for its capacity according to the following indicators: physical accessibility, financial accessibility, goals and philosophy of care, record, self-assessment plan, consultants, relationships and support, management styles, work incentives, and financial stability. Except the residents at Laurier Manor, most of the elderly residents have limited mobility. The PG GODP has been capable of providing care onsite. However, they did not seem to use the dental chairs installed at the facilities. The dentists revealed, “[d]ental chair doesn’t work if there’s no mechanical lift.”  One of the biggest strengths of the program is the financial stability and accessibility. Residents received basic dental care free of charge while dental personnel received reasonable compensation on a monthly basis. With this annual budget of less than C$20,000 the program is capable of convincing the health authority to continually fund the project. “For the amount of money that they [the health authority] are spending [for the PG GODP] relative to their total budget, this is money well spent and we’re really fortunate that we have some very good administrative people like Brenda and Sharon, they’re awesome at writing reports [to the health authority].”   Regarding the sources of funding for equipment and supplies, interview Appendices   181 participants did not seem to be concerned as they revealed that Prince George is a small and tight community where local fund raising for specific needs as such is not a problem.  And because of the strong sense of community, participants expressed that if the program is to expand, it is not too difficult to recruit more dentist and dental hygienist. One of the dentists joking stated that, “our group could probably coercive another one or two dentists to provide this standard of care.”; and as the program manager emphasized, “in Prince George we’re very fortunate to have a fairly strong community based dentist.” The management style also reflected this tight community atmosphere. They meet at four times a year at a study club where they also discussion operation issues and constantly synchronize program goals and philosophy of care. In addition, they also socialized informally.  As for other administrative structure, the paper work is simple and adequate to schedule responsive dental appointment as requested by the facility staff; to communicate with the facility staff about treatment recommended or performed; and to keep track of program productivity. However, all records are paper-based which make it difficult to analyze for clinical statistics and to set up a self-notification of patients due for a yearly recall. Currently, the dental hygienist identifies recall patients when she sees that a dentist has not examined the patient she saw for hygiene treatment for more than one year. Therefore, patients with no hygiene appointment cab easily fall out of the recall system. In addition, some of the dentists expressed that they would like to have more support from dental specialists, especially in oral pathology and oral medicine. Currently, the consultation is communicated via phone only. The support from facility staff and residents through out the year is the big part of encouragement for the providers. Rewarding feelings from giving back to the community is what keep the dental personnel going and overcoming all the challenges in providing care in this unique environment. Overall, the PG GODP has adequate capacity namely physical, financial, and human resources as well as administrative structures and level of integration to effectively address oral health related issues identified by the facility staff and the residents. However, if the program is to expand to comply with the ideal goal of seeing every resident at least once a year to screen and prevent serious oral health Appendices   182 related problems, the program certainly needs a more structured recall system perhaps using simple computer based or online software that offers notification function. Moreover, the number of visits of dentist and dental hygienist in larger facilities may need to be increased, which means more hours, required for the CDA/co-coordinator who has already carried the heaviest workload in the program as well.    Appendices   183 b) Performance: Performance measures of the program include information on its productivity, the quality and accountability of the processes of care, and the quality of immediate product or output of the services. It illustrated the efforts that have been made to offer oral health services to their target populations. In particular, the program was assessed for its performance according to the following indicators: interpersonal care; timeliness, scope of care; level of integration with LTC facility; technical quality of care; frequency and reach of services; changes in knowledge, attitude, and performance of nursing staff regarding daily mouth-care; planning and implementation of mouth-care protocol; and activities promoting accountability of the program. Overall, there were no complaint from the nursing staff and recipients of care regarding neither technical nor interpersonal care. They do appreciate the serviced provided on-site. The services were provided in a timely manner and responsive to urgent request. However, the recipient of care would be even more satisfied if the dentist could visit more often. “Definitely emergency type of needs are being addressed if they’re identified but like in [one of the larger facilities], there’s no way all of the residents are going to be seen [by a dentist for screening]” – a program manager  In addition, one of the dentists revealed that if the program is to increase frequency of visit, two mornings a month for residential care dentistry would still be practical.  “I could see myself doing it two mornings a month. I could work that in because I would just take a day out of the office…because I would be compensated, it’s alright…I’d work one day less a month.” – a dentist  Nevertheless, the request for more working hours was considered “a nice to have” rather than “a must”. The dental personnel also agreed with the benefit of increasing the number of visit per year but with the manpower that they have currently, the reach and frequency of the program seem to be at its full capacity. Appendices   184 “Once a month visit is not enough...if you wan to make an ideal patient care, make sure everybody was alright…now we do it on a need basis…we need to be there more often…but at the moment it’s alright.” – a dentist With the current practice, the PG GODP has been successful providing a wide scope of care from clinical examination, fillings, hygiene, extractions, denture repairs, and new dentures. There was about one case a year for root canal treatment. Therefore, the scope of care did not seem to be an issue but again the reach and frequency of care were. The example of the PG GODP productivity is presented as follows:  Table 2. Example of productivity of the PG GODP at one of the larger facility extracted from the Observation Day Report (Barager & Seemann, 2005) and The Budget Sheet 2006-2007 based on 2005-2006 statistics (Matsen, 2006).  Measures Productivity  (%) Number of resident 116 (100%) residents Residents assessed 80 (68.9%) residents Residents treated 34 (29.31%) residents Restorations 5 (4%) residents Extraction 9 (8%) residents Denture related treatment 12 (10%) residents Specific oral care plan by RDH 13 (11%) residents Denture labeling 2 (0.1%) residents Debridement 15 (13%) residents Individual consultation (one-on-one with care staff/admin) 4 visits Educational in-service 2 visits (Jan. & Mar. 2006) Number of staff at in-services 20 staff per session    Appendices   185 There are two target groups that the program aims to influence. Measuring impact on the secondary target that is the nursing staff is as important as measuring the final outcomes as they can simultaneously impact the patients’ oral health. As part of the Adult Care Regulation, each LTC facility must have an oral care plan for individual resident. Every facility has no trouble establishing an oral care plan. However, when it comes to daily performance, oral care was not obviously identified in the flow sheet where RCAs need to sign and check off that they have completed the task. On average, the mouth-care situation could be improved. The nursing staff “do like take care of the dentures, they take the dentures out and clean them”. However, they have more difficulties with brushing teeth for the residents.  “By having us in there, it makes them [nursing staff] aware that this is important…because it’s like we’re checking to see how they’re doing. I’ve seen clinicians that give these lectures and they say it’s a real horror story [about mouth-care situation in LTC facilities]…we’ve found that there are a few there like that but most of the people, the level of care is reasonable, it could be better, yes.” – a dentist  Also, there was no other form of constant mouth-care reminder such as a wall chart of individual oral care plan. However, the dental hygienist does send a report to the care conference of the residents, which would help to remind, promote, and most importantly to integrate oral health to other LTC disciplines. Another disadvantage of the PG GODP was that there was no attempt to standardize the mouth-care product used within the facilities.  “Nowhere in dentistry teaching to use the sponge but every care homes you go in you find this sponges.”  - a RDH  Despite the lack of routine mouth-care, the nursing staff seemed to be receptive with the program including the two visits of educational in-services. However, there was no evident of the effectiveness of the in-services. The PG GODP may have to not just increase the education and motivation frequency but also the program members should brainstorm for a more creative strategies fitted for each facility structure and culture. Fortunately, the program has established good Appendices   186 rapport with the LTC facilities. Initially there were feelings that the facility staff were hesitant but after several years of implementation. “I’ve been going to [name of the facility] for a long time now so they know me…and we don’t upset their routine. We don’t ask them to do too much…and sometimes they offered…so that’s quite nice.”   .   Appendices   187 c) Patient Outcomes  Patient outcomes are the final piece of jigsaw that would determine if the program is successful. However, information on clinical outcomes of the patients has not yet been collected systemically due to the structure of the paper works administered. “we would like to take on a UBC charting system…but we’ve struggled with it, [we were wondering] how it would actually benefit us and how we would actually use it because it is quite time consuming.” - Program manager  “we haven’t got quite the manpower, its one of those things that gets on the agenda and then it kind of falls off the agenda…we want to, we’ve tried, we do have our laptops here…the problem is time” – a dentist  Nevertheless, participants were able to discuss the impact of the program by assessing oral health from provider’s perspectives and recipient’s satisfaction. “You can see from her [the patient’s] face and she was grabbing your hands. She was very happy that something was done…so there are some that I actually know I’ve made an impact but there are some that I don’t know, you really don’t know because they can’t give you, they can’t tell you, it’s really hard to know.” – a dentist  Another dentist added that,  “I’m sure they [facility staff and residents] see the benefit where before they had frustrations because if they saw something they didn’t know who to contact l and now they at least have a process, they have a name to call, they can consult, it’s not uncommon for us to get a phone call.” – a dentist  Appendices   188 4.2 Lessons learned: success factors and pitfalls The participants have identified a number of challenges developing and implementing their care models within the context of LTC facilities in Prince George. Other programs attempting to initiate oral health services in LTC settings may face many similar issues. This section provide information regarding the success factors that other program might want to establish and also pitfalls that can easily be avoided.  The first lesson learned that one of the dentists brought up was about panoramic X-ray machine. The PG GODP received financial support to purchase a radiographic machine that can be adjusted to fit the height of the patient when sitting in a wheel chair. However, with most of the geriatric patients, they tend to unable to hold their faces still to finish the radiograph. It is fortunate that the machine was installed at the hospital where it can be utilized for other cases of hospital dentistry besides Geriatrics. Consequently, there has been some discussion about purchasing a portable handheld radiographic device, which is now in the researching process. Also, the program had used a portable standing lamp for several years until recently that they have switched to the illuminated mouth mirror (DentLite©). Another lesson learned regarding equipment is the on-site operating space and a dental chair. Participants suggested that if there was no designated space for the dental chair with mechanical lift, even though it could be wheeled around to available space such as a quiet room or a hair dressing room, it was not usually used and easily abandoned.  In addition to the equipment issue, another key lesson learned was regard to communication between dentistry and facility staff. Through out the year, the PG GODP has adapted and added communication documents to inform the nursing staff and recipients of care of the oral health assessment findings and treatment performed. It seems that the three sheets that they administer are simple and efficient. The screening schedule is co-entered by both the nursing staff and the program co-coordinator; the post-treatment information sheet was given to the nurse Appendices   189 in charge every visit; and the dental hygienist also provides individual report for the patient care conference.  Another key add-on of the program was the compensation for emergency or drop-in visit for the dentists. “a lot of times the dentist would just pop in if there was something and so they need to be paid for this…it’s great that they have been willing to do this because at some point you have to make it a sustainable type of program where if someone else came into it … it would be attractive to them too because you’re not necessarily always going to have somebody that’s going to donate their time… so we asked for that and that money was set aside” – Program Manager    Another factor that sustains the program is the “linkages and connections within each level of authorities”. Prince George is a small community where PG GODP personnel have informal connections with decision-makers.  When asked for other success factors, participants revealed several elements of the program as follows: first, the rights people in the program especially the centered person who links everybody; second, the right expectation which comes with experience and constant sharing of those experiences among the team members through study club; and third support from the health authorities for salary-based model of service.    Appendices   190 V. Conclusion and Recommendations 5.1 Conclusion In summary, the PG GODP has responsively addressed the oral health concern raised by the elderly residents and the nursing staff. In its first year, there was uncertainty regarding program sustainability and complaint about limited scope of care. Later in the second year the program shifted its model of delivery from volunteer dentistry to ongoing funding from NHA; since then the program has been able to provide more dental treatment as opposed to conducting largely the need assessment. Since 1998, the patients do not have to pay out of pocket for basic dental care while dental personnel receive reasonable compensation, which was seen to be the strongest feature of the PG GODP. Another uniqueness of PG GODP was regarding the formal consent issue.  The program operates under agreed upon verbal consent and throughout the ten years of implementation, it does not seem to be an issue. In all, the program works efficiently with positive feedback from the recipients of care. However, this initial review does not provide firm evidence in support of the effectiveness of the PG GODP on the residents’ clinical oral health, health, and quality of life. Further in-depth evaluation from patients’ perspectives incorporated with clinical data will shed more light on the final outcomes of care.  In addition, the PG GODP needs to be very cautious of the future of the program because, as one dentist stated, “the change is coming near. Baby boomer may have higher expectation and needs as their dental care is more complex.” The current ways of doing things are likely to be challenged by the next cohort of the population. In fact, changes in population profile is already evident as one dentist described,  “Things are going to changes. Ten years ago you would see a lot of seniors with dentures. Now we see a lot of patient coming into facility with crown and bridge work, good oral hygiene but then all of a sudden their health deteriorated, nobody is looking after their oral health and things start to break, implant is sitting around there so that’s going to be a problem in ten years time. How are we going to clean under these bridges and situation like this?” Appendices   191 In order to plan for the future, I conclude the initial assessment of the PG GODP with a summary of strength, weakness, opportunities, and threat of the program as follows:   Table 3. SWOT summary  Strength z Financial coverage for the patients z Salary-based model z Efficient administration z Services on-site z On-call service z Two-way communication between dentistry and facility staff z Connections with NHA and local communities Weakness z Heavy workload of a coordinator/CDA if the program is to expand z No recall system z No tracking records of clinical outcomes z No proactive prevention & health promotion strategies  Opportunity z Adult Care Regulations z Sense of community (social capital) z Facility accreditation requirement z Changing expectation of the recipients Threat z Changing expectation of the recipients z Changing oral health profile of the recipients z Political cycles may affect funding  5.2 Recommendations for the next action plan  Although the PG GODP has improved greatly over the years, there are still a few aspects it can develop further to prepare for the uncertain future. Note that, some recommendations are common across all sites while some problems are specific to certain facilities. In general, the next action plan for any health program include two main tasks: first, adjusting the program design to increase its capacity; and second educating and motivating both providers and recipients of care to improve performance. With enhanced capacity and performance, it is likely that the final desired outcomes could be achieved. However, the need for improvement can gradually be addressed. It is recommended to incrementally increase the program goals and continually re-evaluate the impact of any modifications. Appendices   192  Specifically, the recommendations for PG GODP include: • Gradually raised desired outcome of the program from being responsive to concerns raised by the recipients of care to proactively involve in routine screening, prevention including daily mouth-care, and health promotion. • Propose to NHA to increase frequency of clinical examination and dental treatment to twice a month in larger facilities. • Create a simple computerized database and notification system to keep track of recall schedule. • Collect and analyze the clinical data using simple and crude indicators comparing oral health status of the residents on a yearly interval. • In the facilities with ongoing changes in its administrative and physical structure, propose and lobby to have designated space for dental chair and organized a fundraising for mechanical lifts. • Continue to keep all the strong features of the program intact especially the communication between dentistry and LTC facilities. • Initiate a discussion among team members about home care service as the needs for oral care is likely to increase among elderly population residing in the community as well.  Although great step has been made in providing financial coverage to the elderly residents of LTC facilities, the program must remain vigilant in its efforts to continually self-assess for its quality; so that there is evidence to support in case the funding is cut short or if the program would like to expand. The analytical framework proposed by UBC-ELDERS Research group has shown to be applicable to the PG GODP. The context of the PG GODP confirmed the utility of structure, process and desired outcome as essential to the description, whereas consideration of capacity, performance and actual outcomes was essential to quality assessment.  The Appendices   193 findings have provided a broad basis for offering advice and practical guidance for developing the program further. It has also initiated the iterative cycle of quality monitoring which I hope will be continued at least every three years. Appendices  194 Appendix M: Compared Data Between UBC and the Northern Programs       Appendices  195  Appendix M: Compared Data Between UBC and the Northern Programs                         (Continue)       Appendices  196 Appendix N: The Model Worksheet  Introduction Purpose: The intent of this document is to provide guidance through the process of assessing oral health services in for residential care.  The guide enlists the assessment process down to 7 main steps (see Figure 6.1 page 113). Step 1  Discuss purpose of the assessment with key participants Step 2 Describe the program Step 3 Construct assessment questions Step 4  Assess indicators against objectives Step 5 Interpret and report results Step 6  Disseminate Adjust program design, motivate, educate based on information learned; Continue iterative loop (back to step 1) Intended Use and Users: This assessment model was originally developed for managers and personnel of a comprehensive oral health program for residential care. However, the evaluation process and quality indicators can be modified and applied to any area of the program. For example, facility administrator and nurse leaders can use this model to audit daily mouth-care provided by nursing staff. The model is most useful when applied for internal quality improvement initiatives. However, it can be used to conduct external review as well. Focus: This guide offers a comprehensive overview of program evaluation based on concepts of quality assurance, which is a dominant concept in residential care practices. Appendices  197 Quality Assessment Worksheet  Program name: ________________________________________________________ Assessment period             Beginning: ______________ Ending: _______________ Assessment Cycle # _____  Step 1: Discuss purpose of the assessment with key participants Stage of development  † Planning † Initial implementation † Adaptive implementation † Matured implementation † Expansion  Identify key players [Those involved and/or affected by our program]  Partnership program: † YES    † NO - list of partner organizations [please make a seperate list of contacts for potential particpants who will help with the quality assessment activities]  † administrators: government, organizations, educational institutions, facilities, hospitals † sponsors: funding agencies, donors † program manager, co-ordinator, other administrative staff † providers: nursing personnel, dental personnel † recipients and families: friends, power of atterney, public trustee, insurance company † allied health professionals: PT, OT, dietitian, social worker, etc † others: unit clerk, pastoral care, recreational staff, etc. † advocacy groups: † others (please specify):   Evaluation purposes † What’s happening here? [Gain insight for planning] (Check all that apply)  † What’s our capacity? [Gap/Asset analysis or SWOT]* † Is it working? What works? What doesn’t?     [Assess effect, summative evaluation, accountability] † How can we make it better? [Quality improvement] † To raise awareness among staff     [quality improvement through accountability] † Others [please specify]        * SWOT = Strength, Weakness, Opportunity, Threat Appendices  198 Step 2: Describe the program  Please use Figure 4.1 (see page 47) and tables below as a framework for collecting information.  Table 1. Framework for describing an oral health program in LTC facility  Program Component Categories Socio-economics, Culture Politics, Legislation/Regulatory Local Health Care System Community Context Population Demographic Frailty, Cognitive level Family involvement Family/resident council established? Characteristics of recipients of care Demographic e.g. age group, career history (veteran?), ethnicity, gender Geography (location, building types, and conditions) # of beds, # of and type of wings/neighbourhoods Government funded or private? Record system (MDS? Electronic?) Staffing model/union or contract Resident-staff ratio and is it an assignment model? Sense of community/neighbourhood/ownership? Management-staff relationship Lines of communication Central person in place? People (roles and responsibilities, staff characteristics) Characteristics of Partners - Structure and culture   of LTC facility  (other partner includes funding agencies, advocacy groups, healthy authority, Others e.g. Eden Alternative? Formal plan/policy/protocol? Oral health goals (implied or explicit/ambiguous or clear) People (roles and responsibilities, personality and characteristic e.g. training background, life history) Designated person/Oral health champion on-site? Sources of funding System of payment and Reimbursement schemes (Capitation, salary, fee-for-service, mixed?) How the different components of care are paid for? Material resources (equipments and technologies) Space and Setting (on-site/off-site?) Monitoring system in place? Program Structure Others e.g. model of care deliver – hub and satellite  Appendices  199  Program Component Categories  History of program Administrative processes (e.g. consent, scheduling, referral, billing, records keeping) How utilization and cost are monitored and managed? Types of services/modes of delivery - Screening & diagnosis (all consented residents/complaint based?) - Educational in-service (specific strategies?) - Daily oral hygiene care (by who, when, and what products?)  Other prevention & promotion activities? (e.g. caries management program for high-risk group?) - Treatment (scope and mode of delivery) Customized individual oral care plan? Recall system in place? Program Process - Practice management - Care delivery/interventions  Referral system? Consultation?  Outcome Indicators - Provider perspectives         -     Recipients’              perspectives When the objective of the program is unclear or not explicit, conduct a focus group or individual interview with key players; then report the “desired outcomes” or “expected effects” of what key informants think the program should accomplish. The findings should be categorized into three groups 1) Capacity, 2) Performance, and 3) Patient outcomes.    Methods/sources of data collection: † brainstroming meetings among key players † focus groups among key players † individual interviews among key players † questionaire surveys of key players using 5 Likert scale and open-ended reflective     questions † Observation/ Site visits † Analysis of administrative documents: proposal, annual report, newspapers, memos,     MDS, Resident Assessment Instrument.  You can present findings in the forms of tables/diagrams (logic models) and text.  Program description will help to select suitable objectives for the stage of program development; and to help intepret the findings from the assessment of quality indicators against the objectives.  Step 3: Construct assessment questions  Assossor please determine evaluability of the program and construct a set of specific assessment questions (see page 100-101).  Appendices  200 Step 4: Assess indicators against program objectives  Please use the following tables as a framework for collecting information.  Please note that indicators, when used alone, are descriptive and neutral. Standards, on the other hand, contain judgement. You may start the first period of assessment with a description of indicators comparing with minimul standard. However, you should incrementally set standards according to your incremental goals. In which case, you need to state the time frame of quality assessment AND quality improvement.  Quality Indicators Suggested Objectives Capacity 1. Accessibility - Every resident has physical access to dental professionals - The facility has a formal contact with dental professionals - A standard fee guide available Æ cost covered - Documentation of program goals - Documentation of protocol/guideline 2. Policy and philosophy of care - Team members know where the documents are kept. - Paper-based documentation of treatment records 3. Records - Electronic database available - Consistent team - Support from specialists 4. Personnel and consultants - Enough people required for the tasks 5. Management Style    - Communication platform available e.g. a study club - Central person/liaison available - Roles and responsibilities are clear - Adaptive and agile 6. Support from LTC - Have a volunteer to assist on-site - Nursing staff/other discipline as part of planning for oral health care 7. Work incentives - Reinforcement from peer and positive results - A sense of belonging/ a sense of team - Opportunity for professional growth 8. Self-assessment plan - Explicit mechanism to monitor progress and impact Performance 9. Satisfaction with care process - Patient-centred care (with flexibility, respect, patience) - Timely care (timely appointment, on-call system in place, attend to urgent problem in a timely manner) 10. Productivity of services - Frequency (at least once a year) - Reach (Residents who have concern Æ all residents) - Scope (e.g. at least with routine oral disorder screening) 11. Daily mouth-care assessment and planning - Documentation by nursing staff (e.g. MDS, oral care plan) - Daily screening by RN 12. Mouth-care product and reminders - Standardized ordering and storage - Distributed and placed in a hygienic locations - Some type of reminder present (e.g. wall chart, flow sheet) 13. Knowledge & attitude of care staff - Improved and sustained re: care techniques and pathway of oral diseases. 14. Performance of mouth-care by care staff - Evidence of mouth-care product being used - At least once a day 15. Integration between dentistry and LTC - Some form of participation at resident’s annual review - Care plan originated from a multidisciplinary team 16. Financial accountability - Documentation available annually Appendices  201  Quality Indicators Suggested Objectives Patient outcomes 17. Overall satisfaction with the result of care (from recipient’s perspective) - No complaint - Continue participation with the program 18. Perceived oral health (from recipient’s perspective) - No self-identified concern 19. Oral health status (from provider’s perspective)  - Normal chewing & swallowing - Improved, delayed, or maintained oral health status, - No pain, no acute infection, no food accumulation (stagnation) in mouth No pain 20. Oral health related quality of life (from both providers’ & recipient’s perspectives) - Looking quite good, feel comfortable - Be able to eat and engage in social activities  Sources/Methods of data collection: † focus groups among key players † individual interviews among key players † questionaire surveys of key players using 5 Likert scale and open-ended reflective questions (see Table 3) † clinical examination † observation/ site visits: formal audits, informal walk-around audit, etc. † analysis of administrative documents: annual report, existing surveys, patient records,     billing/payment records, insurance claim records, complaint records  Table 3. Sample questionnaire Indicators Definition  Likert scale Description of evidence Clarification or Comment Legislatives/ Regulations There is a regulation in place in my jurisdiction (1) Strongly disagree (2) Mostly disagree (3) No comment (4) Mostly agree (5) Strongly agree N/A Adult Care Regulations  (Since 1997) Resident- nursing staff ratio We have adequate nursing staff to perform daily oral health care (1) Strongly disagree (2) Mostly disagree (3) No comment (4) Mostly agree (5) Strongly agree N/A 1staff: 5-6 residents (Morning shift)  1staff: 10 residents (Evening shift) “But it’s not about the number” Support from administrators We have supportive administrators (1) Strongly disagree (2) Mostly disagree (3) No comment (4) Mostly agree (5) Strongly agree N/A Budget approval “But no involvement in day-to-day operations” On-site delivery/ less transfer We have on- site clinic or deliver care bed-side when needed (1) Strongly disagree (2) Mostly disagree (3) No comment (4) Mostly agree (5) Strongly agree N/A 3 on-site clinics; Documented in patient ‘s dental chart whenever care is delivered bedside. The facility provides mechanical ceiling lift Appendices  202 Sample reflective open-ended questions: ƒ What would you say are the strengths of the program? ƒ What do you wish you should have had when you first started the program? ƒ What would you like to improve? ƒ What specific strategies could you plan to improve the program? ƒ What’s your words of wisdom to for others who would like to do similar jobs?  Step 5: Interpret and report results  ƒ Incorporate all sources of findings ƒ Compare program description (logic model) with indicator assessment ƒ Does the results address all dimensions of quality? - We adopt CCHSA’s dimensions of quality but w encourage you to refer to your National/local quality framework so that your program would be better integrate with already existing LTC framework. ƒ The interpretation should allow you to report on: - Program’s capacities and SWOT (strength, opportunities, weakness, threat) - Indicators/standard met and success stories - Indicators/standard unmet - Identification of success factors (success stories) - Identification of problematic issues/barriers and other lessons learned - Identification of future intervention targets - Next action-plans  Step 6: Disseminate  ƒ We encourage you to go beyond sending the report back to the program. Please consider face-to-face meeting, follow-up focus group discussion, presentation at local conference, publications, setting up a community of practice online among practitioners in the country etc.  ƒ We highly recommend conducting a follow-up focus group discussion to discuss specific strategies for: - Infra-structural changes - Service protocol changes - Motivational activities - Educational activities  ƒ There actions will increase the likelihood of capacity an performance improvement.   Continue the loop for constant reassessment (see Figure 6.1 page 113)   We encourage you to set higher standards for the following cycles of quality assessment. Appendices  203 Appendix O: Notes on Publication and Co-authorship   Matana Pruksapong developed the research design and materials such as interview/observation guides; submitted the research proposal for ethical approval; reviewed the literature; conducted field-studies, collected information; analyzed and interpreted the findings; wrote all of the manuscripts and chapters associated with the thesis.  Michael MacEntee, as a faculty advisor and chair of supervisory committee, contributed throughout the whole process of developing, implementing, analysing, and writing the thesis. He reviewed and edited all of the texts. And he is a co- author of all publications from the thesis.  Lynn Beattie, as a member of the thesis supervisory committee from Faculty of Medicine, contributed to the development of research proposal; and reviewed each chapter. She is co-authoring a book chapter that will be published as part of “Oral Healthcare for Frail Elders”: a project accepted by Blackwell Munksgaard for publication in 2009. She will also help write a manuscript for journal article, originated mainly from chapter 4 about the complexity of an oral healthcare program in long-term care facilities.  Arminée Kazanjian, as a member of the thesis supervisory committee from Department of Health Care and Epidemiology, contributed to the development of Appendices  204 research proposal; and reviewed each chapter. She also is co-authoring the book chapter “Oral Healthcare for Frail Elders”.  JoAnn Perry, as a member of the thesis supervisory committee from School of Nursing, contributed to the development of research proposal; and reviewed each chapter. She will help write a manuscript for journal article, originated mainly from chapter 4 about the complexity of an oral healthcare program in long-term care facilities.  Chris Wyatt, as a member of the thesis supervisory committee from Faculty of Dentistry, contributed to the development of research proposal; provided insights on the initiation and operations of the case study. He reviewed each chapter and will help to prepare a journal article from chapter 4 as well as comparative case studies of the three existing models of oral healthcare delivery in BC.  


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items