Open Collections

Performance AnxietyCan performance measurement and reporting help to improve Canadian healthcare?Conference SummaryJuly 2014Eleni Wener MARachael McKendry MASteven Lewis MALeena Wu BASabrina Wong RN PhDHEALTH POLICYCONFERENCE2014CHSPRPerformance Anxiety: Can performance measurement and reporting help to improve Canadian healthcare? Conference Summary was produced by:Centre for Health Services and Policy ResearchUniversity of British Columbia201–2206 East MallVancouver, BC V6T 1Z3Phone: 604-822-4969Email: enquire@chspr.ubc.caYou can download this publication from www.chspr.ubc.ca1Contents2	 About	CHSPR2	 CHSPR’s	Health	Policy	Conferences3	 About	the	Conference3	 How	can	we	advance	performance	measurement	in	Canada?	3	 Key	messages5	 Introduction6	 Knowing	What	to	Measure7	 The	Link	Between	Measuring	and	Reporting8	 Best	Practices	for	Reporting	on	Performance10	 Using	Patient	Experience	to	Transform	Healthcare11	 SummaryUBC  C E N T R E  F O R  H E A LT H  S E R V I C E S  A ND  P O L I C Y  R E S E A R CHC O N F E R E N C E  S U MM A RY :  P E R F O R M A N C E  M E A S U R E M E N T2About CHSPRCHSPR’s Health Policy ConferencesThe Centre for Health Services and Policy Research (CHSPR) is an independent research centre based in the School of Population and Public Health of the University of British Columbia. Our mission is to stimulate scientific enquiry into health system performance, equity and sustainability. Our faculty are among Canada’s leading experts in primary health care, health care funding, variations in health services utilization, health human resources, and pharmaceutical policy. We promote interdisciplinarity in our research, training, and knowledge translation activities because contemporary problems in health care systems transcend traditional academic boundaries. We are active participants in various policy-making forums and are regularly called upon to provide policy advice in British Columbia, Canada, and abroad. We receive core funding from University of British Columbia. Our research is primarily funded through competitive, peer-reviewed grants obtained from Canadian and international funding agencies.For more information about CHSPR, please visit www.chspr.ubc.ca.CHSPR’s annual policy conference is an opportunity to present lessons learned and emerging research on a relevant issue in health services and health policy. The conference draws leaders and researchers from universities, governments, health authorities, and health and patient organizations from British Columbia (BC) and the rest of Canada. In February 2014, the conference focused on performance measurement and reporting in health care. This document presents highlights and lessons learned from the 2014 conference. To view the conference program, speaker biographies, and selected slide presentations, please visit  http://www.chspr.ubc.ca/hpc/overview.About the ConferenceHow can we advance performance measurement in Canada? Performance measurement in healthcare is aimed at enabling actors throughout the system to make better decisions. Across the globe, many jurisdictions are expanding their use of performance measurement and reporting mechanisms—with varying degrees of success. In Canada, multiple provinces, health regions, and organizations have implemented some form of performance measurement and reporting activities. The investments in these activities, to date, have been modest, but are likely to require substantial additional resources in the future. What are the most promising approaches to using these tools to improve healthcare within a Canadian context? How much progress has been made to date? How should we approach these activities in the future to ensure that we are getting the most value from our efforts? On February 25, 2014 health care decision-makers and researchers convened in Vancouver to engage in a dialogue about:1. The science of performance measurement and;2. Best practices for reporting on performance. Key messagesIncrease awareness of data usability. Many organizations have ostensibly excellent data but do not act on it. Organizations and jurisdictions that have clear concepts of how data can and should influence policy and practice are examples to learn from.Report at the level of actionability. Data on performance in healthcare are collected and used for a variety of purposes, ranging from “real time” analytics, or quality improvement, to academic research. Data on performance ought to be actionable. Thus, it is important to report performance data at the level of actionability and causation as data need to be granular enough observe meaningful variation.Report basic measures. Avoid high levels of aggregation, which tend to obscure large differences among local programs and communities; instead, disaggregate reporting to a level that will be meaningful to those who should respond to the data, e.g., a hospital unit. Examples of measures to report: How many died? What were individuals’ experiences? UBC  C E N T R E  F O R  H E A LT H  S E R V I C E S  A ND  P O L I C Y  R E S E A R CH3C O N F E R E N C E  S U MM A RY :  P E R F O R M A N C E  M E A S U R E M E N T4Merge performance measurement and quality improvement. By doing so, organizations increase the likelihood of producing relevant and actionable data that informs both suppliers and users  of care. Attract the right audience. Most members of the Canadian public will not need (or use) performance measurement reports to inform their choices about whether or where to seek care. But performance measurement does get the attention of care providers and creates a mechanism for, and the commitment to, translate it into concrete improvements. Decouple performance measurement from funding. There is a need to invest in performance-enhancing infrastructure (e.g., information systems that support coordinated care, team-based care for those with complex needs). If performance data is tied to funding, there is an incentive either to game the measures or suppress information that could affect the bottom line. Rewarding organizations for good performance while penalizing those for performing poorly helps good organizations get better while depriving organizations needing improvement the resources to improve. Rewarding organizations (teams and units) makes more sense than rewarding individuals as health care performance is most often a function of team and environmental factors. Engage with patients and foster trust. A lack of transparency has caused the Canadian public to lose trust in the health care system. Organizations and providers need to learn and understand how the public absorbs and responds to information and to continuously test different formats for making information available to the public, identifying those that resonate best. More is needed to engage patients. We have allowed patients in the door and listened to their voices but we need to do more—we need to let patients be in charge of their healthcare. The health care system belongs to the patients; thus, they should play prominent roles in its governance and administration. This thinking has been instrumental in the success of high-performing organizations such as Group Health in Puget Sound and South Central in Alaska.Rewarding organizations (teams and units) makes more sense than rewarding individuals as health care performance is most often a function of team and environmental factors. UBC  C E N T R E  F O R  H E A LT H  S E R V I C E S  A ND  P O L I C Y  R E S E A R CH5In BC, and Canada at large, we are in need of a robust performance measurement system to identify how (and whether) our healthcare system is improving and how we are benchmarking relative to other systems. Speaking at the conference, Assistant Deputy Minister Heather Davidson of the BC Ministry of Health indicated that the province is working to create such a monitoring system. The release of the Ministry’s 2014/15 Service Plan in early February 2014 brings an updated strategic plan for the Ministry, one which builds on the 2010 Innovation and Change Agenda. Lena Cuthbertson, also of the BC Ministry of Health, explained that from a patient engagement perspective, the vision is to enhance the availability of information about the quality of care that follows the patient’s journey across the care continuum. The new Service Plan builds on the four pillars that have “bent the cost curve”: 1. Health promotion, prevention and self-management; 2. Integrated primary and community care; 3. Hospital services: protocols for clinical care management; and 4. Driving productivity, efficiency and innovation across the system. Annual health care spending increases have fallen from 7% in 2010 to 2.6% in 2014. Davidson indicated she is hopeful that with commitment to a shared plan of action that identifies strategic priorities and values, with clear roles and responsibilities for all players, performance measurement and reporting strategies will improve. IntroductionHow do we ensure we measure what is important instead of simply making what we (already) measure important? C O N F E R E N C E  S U MM A RY :  P E R F O R M A N C E  M E A S U R E M E N T6What are we trying to measure the performance of? We do not always measure what is important. Often the information collected for administrative and other purposes drives and constrains our performance measurement activity. Dr. Victoria Lee, Medical Health Officer at Fraser Health, one of BC’s five regional health authorities, noted that what gets reported is what gets measured.Stephen Peckham, Director of the Centre for Health Services Studies at the University of Kent in the United Kingdom, explained that there are three questions in performance measurement: 1. Who or what is being measured? (The organization? The individual?) 2. How will these be measured? (Performance against metrics? Thresholds?) 3. Who is it being measured for? (Government? Patients? Professional? Caregiver?) Peckham highlighted that it is important to measure not only outcomes, but also the material, psychological, social and cultural influences that have impact on outcomes. Dr. Jean-Frédéric Lévesque, Chief Executive of the Bureau of Health Information in New South Wales (NSW), Australia, an organization that provides independent reports about the performance of the public healthcare system in NSW, described a dynamic model of performance measurement where performance is a shared process among multiple levels and multiple actors within those levels. Measures of accessibility, appropriateness, effectiveness, efficiency, equity and sustainability are interconnected and interact within this model; according to Lévesque, “developing measures that are attributable in time and space is one of the most challenging tasks for performance reporting organizations.”One burning question raised during the course of the conference was: How do we ensure we measure what is important instead of simply making what we (already) measure important? Peckham emphasized context and the importance of developing measures about practices aimed at meeting the needs of people with multiple health and social care needs. Generally, the current evidence suggests that we focus more on process measurements for single diseases. This is not necessarily optimal if challenges are around comorbidity, chronic disease, and end of life care. Lévesque added that we need to improve our capacity  to define what is needed to measure performance and to develop measures that move beyond single disease indicators. There was widespread acknowledgement of the complexity of measuring competence and quality of care without undermining trust relationships between providers and patients/service users. Lee argued that in order to have a successful performance measurement system, we need to facilitate a cultural (and corporate) shift that recognizes the importance of performance measurement, supported by dedicated data collection and reporting resources, flexibility in its implementation, and engagement at all levels. Knowing What to MeasureUBC  C E N T R E  F O R  H E A LT H  S E R V I C E S  A ND  P O L I C Y  R E S E A R CH7There is a clear relationship between accountability and performance measurement. Accountability can involve performance measurement. With the renewed focus after the release of the 2014/15 Service Plan, the BC Ministry of Health restructured to focus on accountabilities. In the UK and Australia there are publicly available physician report cards and reporting. The commitment of governments to be accountable to the service users by making report cards publicly available is important, becoming increasingly widespread in other jurisdictions. Dr. Jack Tu of the Institute for Clinical Evaluative Sciences in Ontario (ICES), presented valuable lessons learned in regards to publicly available healthcare report cards in Ontario, using coronary artery bypass graft (CABG) surgery as an example. Tu described how cardiac report cards in Ontario have been an effective instrument for stimulating a variety of quality improvement initiatives aimed at cardiac care. Implementing the report cards led to the identification of approximately one third of deaths as potentially ‘preventable.’ Reporting has since expanded to include other types of cardiac procedures. Tu noted that the report cards include clinical data and process of care indicators considered to be an important complement to those derived solely from outcome indicators and administrative data. Enhanced communication structures and processes resulting from collaborations between clinical researchers and surgeons have helped in creating effective report cards. Tu suggested however, we need to continue developing mechanisms for efficient and timely clinical data collection and better capacity (e.g., quality improvement teams) in the healthcare system to act on report card information. Peckham explained that the role of science in performance measurement is to explicate what ought to be measured and to improve how this gets measured in order to improve health service delivery. A desirable outcome of performance measurement is to provide information on effective interventions that will improve the desired outcomes of the health sector and overall health system. As noted by health policy analyst and conference rapporteur Steven Lewis, the morning’s presentations demonstrated inherent tensions around performance measurement, including the pressing question of: To whom do these data belong? In order to report on the performance of the system, the data must be liberated. Lewis explained that any health information enterprise needs to be linked to a theory of change. We need to understand how people respond cognitively to performance data in order to produce information that will ultimately influence their behaviour. The Link Between Measuring and ReportingWe need to understand how people respond... to performance data in order to produce information that will ultimately influence their behaviour. C O N F E R E N C E  S U MM A RY :  P E R F O R M A N C E  M E A S U R E M E N T8Canada has been slow to introduce performance measurement and reporting as a standard business practice. Yet, in order to achieve a high performing healthcare system, measurement and reporting are critical. John Abbott, formerly with the Health Council of Canada, indicated that reporting acts as an effective accountability measure when the indicators are relevant and the reporting is transparent, reflecting ideas presented in the morning panel.  Abbott highlighted that while we have done a lot of reporting in Canada, its relevance is in doubt. What a patient wants and needs to know, and what administrators, policy-makers and clinicians want and need to know often differs (and ought to differ). However, many of our reporting systems speak to the same audience, which often means that, as Abbott said, “we may be hearing, but are we in fact listening?” If we are not listening, it may be because the information presented is not directly relevant to us. Five key enablers that support improved health system performance according to the Health Council of Canada are: leadership, policies and legislation, capacity building, innovation and spread, and measurement and reporting. Going forward, we need to streamline the number of indicators in use and align these within and across the systems. We need to invest in information systems, monitor performance and progress, avoid duplicating data by using existing data and ensure analysis is shared with all parties. Ultimately, we need to demonstrate effective leadership by increasing transparency and include the perspective of the patient.Abbott indicated there is a culture of status quo when there is a lack of measurement and highlighted the importance of strong leadership in enacting change.Diane Watson of the National Health Performance Authority, an independent agency in Australia that monitors and reports on the performance of health care organizations in that country, shared some of the best practices for public reporting in the Australian context. There is extensive transparent public reporting in Australia that is easily accessible to the public on a variety of websites. The interactive webpages allow the public to compare various items, including hospitals, regional areas, physician profiles and outcomes. In addition, as part of Australia’s strategy to use public reporting as a catalyst for change they have engaged with the media. They have participated in over 700 media events (radio, television, newspaper) in order to reach the public. Watson highlighted how combining rigor in creating performance reports (a process she brought with her from Canada) with a culture in journalism leads to actively engaging in reporting. Public reporting on performance keeps organizations and clinicians honest, reinforcing the point that performance reporting attracts the attention of healthcare providers. Best Practices for Reporting on PerformanceUBC  C E N T R E  F O R  H E A LT H  S E R V I C E S  A ND  P O L I C Y  R E S E A R CH9Kira Leeb, Director of Health System Performance at the Canadian Institute for Health Information (CIHI), brought forward the idea of public consultation. Health system performance reporting is not a new idea for CIHI; however, they have refined their approach to hospital-level reporting. In 2010, CIHI released the public version of the Canadian Hospital Reporting Project’s (CHRP) interactive Public Web Tool “Health System Performance Infographic Maker” that allows users to examine how well the health system is working. CIHI worked closely with a variety of stakeholders, including a public consultation group, in developing the tool. Once the web-based tool was released, the public consultation group was approached to provide feedback on the tool (met their needs, easy to use, etc.). The feedback from the public has been excellent and a range of individuals, from students to board members, have indicated they have used it. CIHI is now developing a regional and facility level website with similar characteristics to the public website. This website will include additional information targeting decision-makers and other system managers. CIHI is committed to asking the public what they want to see in terms of public reporting on healthcare performance and believes the web-based tools will create transparency, trust and change. When involving the public and service users, we must address how we are reporting to them, what experiences we are reporting on and how this information is being disseminated. The health care system belongs to the patients; thus, they should play prominent roles in its governance and administration.C O N F E R E N C E  S U MM A RY :  P E R F O R M A N C E  M E A S U R E M E N T1 0In BC we have adopted the definition of patient experience from The Beryl Institute, “The sum of all interactions, shaped by an organization’s culture that influence patient perceptions across the continuum of care.” Lena Cuthbertson, Provincial Director of Patient-Centred Performance Measurement and Improvement at the BC Ministry of Health and Providence Health Care, explained that in BC we have the Patient Reported Experience Measures Steering Committee (PREMS) that allows the province to obtain feedback from the patients reporting on their perspectives. The PREMS uses sector specific (e.g., cancer, emergency department) surveys to collect the data. Every survey is developed with patients and providers in expert Consultation Groups. The results from patient surveys are valued. Providers value the information and the fact that it is delivered to them without them needing to personally collect the data. Surveys need to be timely and provide clear and concise information. The focus of these surveys is to provide a voice of the patients; however, this perspective is intended to complement other sources of information. Sholom Glouberman, President of Patients Canada, a patient-led and -governed organization that encourages patient and caregiver engagement in all aspects of healthcare, brought the patient’s experience to the forefront. He noted that it has been a slow process to build patient voices in a healthcare system that historically has its origins in hospital-based acute care. Now, in a system where a large proportion of patients have to manage multiple chronic conditions outside of hospital settings, he argued that clinicians, healthcare organizations, and researchers need to take a step back and listen to the experiences of patients in their navigation through our complex healthcare system. Glouberman passionately spoke about the Patients Canada procedure for indicator development and indicators for improved healthcare experience, all of which is based on patient narratives.Cuthbertson explained that measuring patient experience is not an amateur sport; it is a science and an art. Patient-centred measures need to be collected in a uniform manner, inform us on what our patients and families really think, and focus on what is important to patients and their families. Finally, patient experience reports should be available as frequently and as readily as any other management report. Using Patient Experience to Transform HealthcareUBC  C E N T R E  F O R  H E A LT H  S E R V I C E S  A ND  P O L I C Y  R E S E A R CH1 1SummaryPerformance measurement and reporting is nascent in Canada’s healthcare system. Much work is needed to develop the science and reporting of performance measurement in the healthcare system. Beyond assessing the variation within and across jurisdictions in performance, implementing a measurement system has the additional dividend of making it possible to identify innovations and combinations of innovations that are associated with a better performing healthcare system....measuring patient experience is not an amateur sport; it is a science and an art. UBC Centre for Health Services and Policy ResearchThe University of British Columbia201-2206 East MallVancouver, B.C. Canada V6T 1Z3Tel:  604.822.4969Fax:  604.822.5690Email: enquire@chspr.ubc.cawww.chspr.ubc.caAdvancing world-class health services and policy research and training on issues that matter to Canadians