UBC Faculty Research and Publications

Developing a Performance Measurement Framework and Indicators for Community Health Service Facilities… Wong, Sabrina T; Yin, Delu; Bhattacharyya, Onil; Wang, Bin; Liu, Liqun; Chen, Bowen Nov 18, 2010

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12875_2010_Article_513.pdf [ 546.2kB ]
JSON: 52383-1.0220567.json
JSON-LD: 52383-1.0220567-ld.json
RDF/XML (Pretty): 52383-1.0220567-rdf.xml
RDF/JSON: 52383-1.0220567-rdf.json
Turtle: 52383-1.0220567-turtle.txt
N-Triples: 52383-1.0220567-rdf-ntriples.txt
Original Record: 52383-1.0220567-source.json
Full Text

Full Text

RESEARCH ARTICLE Open AccessDeveloping a Performance MeasurementFramework and Indicators for Community HealthService Facilities in Urban ChinaSabrina T Wong1†, Delu Yin2†, Onil Bhattacharyya3, Bin Wang4, Liqun Liu, Bowen Chen2*AbstractBackground: China has had no effective and systematic information system to provide guidance for strengtheningPHC (Primary Health Care) or account to citizens on progress. We report on the development of the China results-based Logic Model for Community Health Facilities and Stations (CHS) and a set of relevant PHC indicatorsintended to measure CHS priorities.Methods: We adapted the PHC Results Based Logic Model developed in Canada and current work conducted inthe community health system in China to create the China CHS Logic Model framework. We used a stagedapproach by first constructing the framework and indicators and then validating their content through aninteractive process involving policy analysis, critical review of relevant literature and multiple stakeholderconsultation.Results: The China CHS Logic Model includes inputs, activities, outputs and outcomes with a total of 287 detailedperformance indicators. In these indicators, 31 indicators measure inputs, 64 measure activities, 105 measureoutputs, and 87 measure immediate (n = 65), intermediate (n = 15), or final (n = 7) outcomes.Conclusion: A Logic Model framework can be useful in planning, implementation, analysis and evaluation of PHCat a system and service level. The development and content validation of the China CHS Logic Model andsubsequent indicators provides a means for stronger accountability and a clearer sense of overall direction andpurpose needed to renew and strengthen the PHC system in China. Moreover, this work will be useful in movingtowards developing a PHC information system and performance measurement across districts in urban China, andguiding the pursuit of quality in PHC.BackgroundChina, now home to more than 1.3 billion people [1],once had an enviable primary health care (PHC) systemwhich was inexpensive and had a significant impact onpopulation health [2,3]. From 1952 to 1982, China sawrapid improvement in health; life expectancy rose from35 to 68 years and infant mortality fell from 200 to 34per 1000 live births [3]. Their approach to health careprovided nearly universal health insurance and highaccessibility through barefoot doctors to more than 90%of the population [4].Shortly after 1978, China’s universal health insurancecollapsed and there was a shift in funding from rural tourban facilities and from PHC to specialized and hospi-tal-based care. This resulted in a proliferation of specia-lists, a rapid rise in out-of-pocket expenses, excessiveuse of drugs and high-technology diagnostic tests,decreased access to care and utilization, and a growinghealth disparity gap between rural peasants and urbancity dwellers [5]. However, since 1997 the governmenthas increased health spending, emphasizing PHCrenewal through community health facilities in urbanChina [6], based on evidence that a strong PHC systemreduces health inequities across populations [7-10], and* Correspondence: deluyin@yahoo.com.cn† Contributed equally2Capital Institute of Pediatrics and Community Health Association of China.2 YaBao Road, 328#, ChaoYang District, Beijing, 100020, ChinaFull list of author information is available at the end of the articleWong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91© 2010 Wong et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction inany medium, provided the original work is properly cited.may also contribute more to improving populationhealth than specialized health services [10,11].Although people who need PHC can obtain these ser-vices from any health facility (e.g., hospital, clinic, com-munity health facility), the community health sites areespecially designed to deliver PHC, providing a basket ofcomprehensive services designed to address acute andepisodic health conditions in order to improve accessand continuity of care and increase the overall effective-ness of the health care system [12]. Indeed, the mainplace of PHC delivery is through the publicly fundedcommunity health facilities and smaller, affiliated, com-munity health stations (CHS; we herein refer to commu-nity health facilities and stations interchangeably). CHSfacilities provide residents health education, family plan-ning and rehabilitation and is a key component of com-munity development [13,14].The number of CHS facilities almost tripled by 2008,with the proportion of cities offering PHC servicesreaching 91%, through an estimated 29,127 facilities and185,050 professionals [15]. The federal government hasplanned a total of US $124 billion dollars to support thereform of the entire health system [6]. Some of theseinvestments include: US $310 million to renovate 2,400CHS facilities, US $12.4 million for CHS workforcetraining, US $34.1 million to build more CHS facilities,and investments which will increase annually with thebase of US $2.33 per person in 2009 to pay for basicpublic health services [16].Population-based information and reporting systemsare needed as policy-makers and managers seek tomonitor the performance of PHC, identify areas requir-ing structural or process modifications, assess the rela-tive impact of different strategies to catalyze renewaland account to citizens on progress [14,17,18]. Nonational information or reporting system for the com-munity based PHC system currently exists. Althoughthe CHS facilities are required to participate in monitor-ing and measurement of performance, most of this workis completed at the district and county level and hasmainly focused on examining the structure of servicedelivery (e.g. financing, organization, health humanresources, and volume of visits) [19]. This paper reportswork conducted in China since 2007 to: 1) develop andmodify a Results-Based Logic Model, a performancemanagement and accountability framework for the CHSsystem and 2) identify CHS priorities in order todevelop a useful set of relevant PHC indicators.Why use a Logic Model?A Logic Model depicts the flow of resources and pro-cesses required to produce the results desired by theorganization or program. Simply put, a Logic Modelattempts to visually convey the connection betweeninputs, activities, outputs, and outcomes [20]. It canoffer guidance in the development of an informationsystem [21] by supporting the: a) identification of rele-vant performance indicators relevant to policy makersand providers and b) development of evaluation orresearch questions aimed at examining whether perfor-mance or health outcomes improves.A common framework for delivery of PHC affords sta-keholders the opportunity to more clearly consider andcommunicate expected associations and links betweengoals and objectives, alternative courses of action, andthe attainment of results. It defines the areas in whichinformation, evaluation and evidence are needed for pol-icy, administrative and practice communities to plan,monitor, guide and report on PHC renewal [22,23].Moreover, a PHC Logic Model framework can be usefulat the organizational level for planning delivery of ser-vices and designing outcomes-based evaluations of pro-grams. Logic Model frameworks can assist policymakers, managers and providers implement targetedquality improvement efforts [22,23].MethodsWe adapted the Canadian PHC Results Based LogicModel which was developed in response to the lack of acommon performance measurement and evaluation fra-mework for understanding the PHC system. More detailabout the development and validation of the CanadianPHC Logic Model, using the Treasury Board of Canadaresults-based management accountability framework,policy analysis, research evidence, and broad consulta-tion with multiple stakeholder groups, can be foundelsewhere [22,23]. The PHC Logic Model was chosen toguide this work since it has been used to examine PHCrenewal, summarize expected outcomes of PHC, andguide analysis for the simultaneous impact of PHCactivities on outputs (e.g., type of care such as healthpromotion and qualities of care such as accessibility andcomprehensiveness of services) and outcomes through-out Canada and internationally.China CHS Logic ModelIn order to create the China CHS Logic Model, we useda staged approach. First we constructed the frameworkand indicators. Then we sought content validationthrough an intensive interactive process involving policyanalysis, critical review of relevant literature and multi-ple stakeholder consultations. Our definition of PHC inChina is based on work by the State Council of China[13], the World Health Organization [24], and interna-tionally recognized academics whose expertise is in thearea of PHC. Primary health care is defined as the firstcontact of care and the delivery of comprehensive, con-tinuous, and convenient episodic and preventive healthWong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 2 of 9care services to families. Services are provided by gen-eral practice physicians, nurses, public health workers,and other allied health professionals (e.g., pharmacists).Policy analysis and literature reviewWe collected CHS policies related to investment, facil-ities management, capacity building, register with socialhealth insurance, and so on from the published and greyliterature to identify the goals and objectives relevant toCHS service delivery and the role of CHS in China. Wethen conducted a content analysis of the China nationaland provincial policy documents, spanning 1997-2008[25-27]. This analysis consisted of a summary of the keypoints and recurring themes for each topic (e.g. facilitiesmanagement, capacity building). A review of the litera-ture identified CHS performance frameworks [22,23,28]and relevant indicators used in other countries andinternational organizations. Major databases such asMedline (PubMed interface) and CINAHL weresearched using key words (e.g., performance measure-ment frameworks and community health, qualityimprovement frameworks) and MeSH headings.Together, the analysis of policy and literature reviewprovided the foundation from which we developed theinitial China CHS Logic Model.Stakeholder consultationsA multi-stage iterative feedback and revision process wasused for stakeholder consultations. These consultationswere undertaken for a period of three months, and themodel was continually revised in response to multiplestakeholder consultations. We used our partnership withtwo health districts, BaoAn and WuHou, to recommendstakeholders who could help refine the logic model. Ourpartners suggested we conduct focus groups with a rangeof clinicians, researchers, and managers in each district.Our partners approved a list of potential participants andwe then sent letters of invitation to these individuals. Par-ticipants were chosen based on their knowledge andexpertise about CHS and their region of work in China(east, middle, and west of China).We conducted a series of four focus groups (n = 24)with CHS providers (n = 6), academics (n = 6) and eva-luation specialists and policy makers (n = 12). Duringthe course of the focus groups, our health district part-ners also suggested that we interview key decision-makers (n = 4). These in-depth interviews included lea-ders from the Ministry of Health and leaders from twopilot units: BaoAn District Health Bureau and WuHouDistrict Health Bureau. Focus groups were run sepa-rately for providers, researchers and evaluation specia-lists, and policy-makers. Focus groups were conductedby one of the authors (DY). A series of open-endedquestions asked about where the logic model categoriesand whether the connections between the different cate-gories (e.g. immediate, intermediate, and final outcomes)made sense. Examples of questions include: “What doesthe stabilization of chronic conditions mean to you?”and “Should this be an immediate outcome of servicesprovided by CHS facilities? If so, can you tell me moreabout your thinking on this?” All data were audio-recorded and summarized. The logic model was refinedbased on the focus group and interview data.Performance indicatorsBased on the Canadian Institute for Health Information(CIHI) Pan-Canadian PHC Indicators [29] and our reviewof existing performance indicators [29-34], we iterativelydeveloped indicators to measure different inputs, activities,outputs, and outcomes in the Logic Model. Developmentof the indicators was based on input from front-line provi-ders, CHS managers, policy makers, and researchers. Simi-lar to the process followed in refining the Logic Model,another series of four focus groups (n = 34) with providersand CHS managers, academic and evaluation specialists,and policy-makers was conducted. Participants were askedabout their roles and responsibilities and what theythought was important in terms of measuring their perfor-mance. We used the following criteria for choosing the setof indicators: a) importance and relevance to delivery ofPHC in China, b) potential feasibility of obtaining datausing a PHC information system, and c) evidence suggest-ing the delivery of PHC activities and services is linked tooutputs or immediate, intermediate, or final outcomes.Similar to the CIHI format [29], we developed detailedspecifications for each indicator including: (1) a clearoperational definition of the indicator; (2) explicit defini-tions of the key terms included in the definition; (3) anyinclusion or exclusion criteria; and (4) the underlyingrationale for each indicator.ResultsThe China results-based CHS Logic Model is a heuristicframework describing relationships between inputs,activities, outputs, and outcomes relevant to CHS (figure1). While the overarching structure of the framework issimilar to that of Canada’s PHC Logic Model [34], theinputs, activities, outputs and outcomes of China CHSmodel are somewhat different. These differences reflectthe contextual differences between China and Canada insocial, economic, cultural and political arenas and takeinto consideration China’s unique problems confrontedby its health system. We describe the China CHS LogicModel here [34].InputsThe foundation of China’s PHC system has two parts,resources (or inputs) and activities. Within the social,Wong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 3 of 9cultural, political, legislative, and economic and physicalcontext, decision- and policy-makers’ attention to com-munity health services are focused on inputs related tofiscal, material, and health human resources. Prioritiesfor improving the renewing of CHS are related toincreased financial investment, material resources, andhealth human resource capacity. Examples of perfor-mance measurement input indicators include:1. Amount of financial investment by the nationalgovernment for subsidization of services delivered atCHS facilities and capital infrastructure constructionin community health; Government leadership thatpublicly supports the goals of the CHS system and agradual health system shift to PHC (e.g., percent ofsub-districts who have at least one communityhealth centre)2. Material resources including physical facilities,information technology (e.g., personal healthrecords) and equipment used to support and delivercare. Moreover, the development and widespreadadoption of clinical practice guidelines in CHSsettings.3. Number and types of health human resources andtheir qualifications (e.g., knowledge and discipline-specific competencies, use of interprofessionalteams). Examples of types of health human resourcesneeded in delivery of PHC through CHS facilitiesinclude general practice physicians, nurses, andnurse practitioners.ActivitiesPHC activities, the second part of the PHC foundation,are processes intended to produce specific outputs andare prepare the PHC system to deliver services. Activitiesthat enable PHC include: policy and governance, healthcare and clinical management, individual decisions, andcommunity decisions. Priorities for measuring activitiesrelated to PHC activities include the need for CHS facil-ities to accept reimbursement from publicly fundedhealth insurance, increase coordination and collaborationwith other facilities to deliver PHC activities, improve thetraining and continuing education of health humanresources working in CHS facilities, and strengthen thecapacity of CHS facilities to delivery PHC.Examples of activity indicators include:1. Policy and governance: Expansion of the level andcoverage of health insurance, enhancing the CHS facil-ities’ ability to accept reimbursement from publicly-funded insurance carriers; Governance structure thatstrengthens coordination and formal collaborationbetween CHS facilities and other places (e.g., numberFigure 1 China Community Health Services Logic Model for Performance Measurement of Primary Health Care.Wong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 4 of 9of “two-way” referrals by facility where a patient isreferred for specialized services and that specializedservices (e.g., internal medicine) refer patients to CHSfacilities as their place of first contact with the healthsystem;2. Health care management: Increased use of inter-professional teams, Increasing accessibility of care (e.g., hours and days of operation);3. Clinical management: Increased training opportu-nities and continuing education to CHS providersthat includes the use of clinical guidelines; Integra-tion between “Western medicine” and Chinese “tra-ditional” medicine4. Community decisions: Group health promotionactivities organized by local community organiza-tions; Support and encouragement by neighborhoodcommittees for communities to use CHS facilitiesfor PHC services.OutputsServices (outputs) are divided into basic public health andmedical services with the former more important than thelatter. Services can be described in terms of type, volume,distribution, and characteristics. Examples of types of PHCservices include: primary, secondary, and tertiary preven-tion and curative, rehabilitative, palliative, and supportiveservices. Volume/utilization refers to the amount of differ-ent types of services being delivered whereas distributionrefers to how the services are allocated to individuals andcommunities (e.g., who gets how much of what service).The quality of PHC outputs, refers to the degree to whichhealth services for individuals and communities increasethe likelihood of desired health outcomes and are consis-tent with current professional knowledge. Examples ofindicators in this area include:1. Types of services and volume: Percent of CHSfacilities that provide public health services such ashealth education and immunizations; Percent ofCHS facilities incorporating rehabilitation, case man-agement (e.g., percent of patient with hypertensionwho have their care coordinated by a case manager),and use of Chinese traditional medicine;2. Utilization of CHS for public health and primarymedical care (e.g., percent of patients who have aregular doctor located at a CHS facility);3. Characteristics or qualities of CHS facilities, basedon national policy priorities [13], include: Safety,effectiveness, comprehensiveness, continuity (e.g.,percent of patients who saw a specialist and haveinformation back to their regular physician withinthree months), coordination and patient focus.OutcomesPHC outcomes are the results that should occur fromthe delivery of PHC services. Outcomes can be consid-ered immediate, intermediate, and final. While immedi-ate outcomes should be directly attributable to PHCoutputs, intermediate outcomes are those areas in whichPHC providers and stakeholders have a lesser degree ofcontrol, but for which delivery of quality PHC servicesare still expected to have some impact. Final outcomesare those areas over which PHC providers and stake-holders have the least amount of control, recognizingthat the provision and delivery of health services is onlyone of the social determinants of health. Examples ofpriority outcomes include:1. Immediate: Increased individual capacity, knowl-edge, and confidence in managing his/her health,reduced duration and effects of acute conditions, sta-bilized chronic health conditions; Satisfaction ofCHS workforce2. Intermediate: Health related outcomes (e.g.,healthy choices and behaviors), patient satisfactionand confidence in CHS facilities, and appropriate-ness of place and provider (e.g., minimize the use ofspecialists without a referral from a CHS provider).3. Final: PHC delivered through China’s CHS facil-ities was designed to attain overall improved popula-tion health (e.g., lower premature mortality), equity,and lower overall costs to the health system.In addition to developing the China CHS Logic Model,we developed a set of performance indicators (n = 287) forChina’s CHS system which were meant to measure mostcomponents of the Logic Model. Each indicator includedspecifications in order to provide sufficient detail and sothat they would be applied consistently across settings.The input indicators (n = 31) were designed to measureChina’s increasing investment in CHS facilities, healthhuman resources, and funding for PHC. Indicators in theactivities component (n = 64) of the model measure fac-tors such as health insurance coverage, education andtraining, and accessibility. Outputs indicators (n = 105)were designed to measure types and volume of PHC ser-vices and the quality of care provided. Given the particu-larly low utilization of the current CHS in China [35,36],improvements and indicators to measure these improve-ments are urgently needed. There are a total of 87 indica-tors that measure immediate (n = 65), intermediate (n =15), and final (n = 7) outcomes. Although many inputindicators are already in use, more regularized use of activ-ity, output, and outcome indicators is needed. While theset of performance indicators may appear daunting, theseindicators can be used as a whole or to form subsets ofWong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 5 of 9indicators to address different investments or priorities.Depending on the priorities for service delivery via CHSfacilities in any given year, a subset of indicators will beselected. Importantly, 24 indicators makeup the core setand used for all routine monitoring of primary care deliv-ered through the CHS. These list indicators can be alsoused to inform and prioritize the enhancement of the datacollection infrastructure over time. Table 1 shows the listof core indicators designed to measure PHC performanceand monitor investments in the CHS system.The China CHS Logic Model depicts the relationshipsbetween inputs, activities, outputs, and outcomes. It alsoprovides a heuristic for PHC decision- and policy-makers to consider the sometimes competing goals ofefficiency and effectiveness. Efficiency in the CHS sys-tem is a function of the inputs, activities, and outputs[34]. Efficiency is the extent to which an organization,policy, program or initiative is producing its plannedoutputs in relation to expenditure on resources. Effec-tiveness is the extent to which the CHS sector deliversits intended outcome or results in a desired process, inresponse to need [14].DiscussionGiven that a strong PHC system has a positive impacton population health and reduces the social-economicgradient in health [10,11,37-39] provincial and nationalgovernments in China are determined to rebuild a moreequitable PHC system to address widespread dissatisfac-tion and inability to access care due to the shift to amarket-oriented health system. Moving PHC deliveryout of hospitals into the CHS system and the develop-ment of the Logic Model and indicators provides ameans for stronger accountability and a clearer sense ofoverall direction and purpose needed to renew andstrengthen the PHC system in China. Although a net-work of facilities and a general policy framework forCHS has been constructed in the last 10 years, Chinahas lacked a systematic, standardized performance eva-luation and management information system. Moreover,no guidance of a consensus-based accountability frame-work for the CHS system has been used nor did theperformance measurement indicators comprehensivelycover the quality of care (activities) or immediate andintermediate outcomes. Therefore, these earlier type ofevaluations are not as useful in guiding future invest-ments in PHC renewal or in development of China’sPHC information system [19].A Logic Model can be useful in planning, implementa-tion, analysis and evaluation of PHC at a system andservice level [23]. This framework was used in two dis-tricts in China to generate useful practice informationabout the relationships between inputs, activities, out-puts, and outcomes [35,36]. For example, in one district,we found the incidence of measles was higher than inthe past 2 years because immunization to immigrantchildren was lagging. Using the China CHS Logic Modelas a heuristic framework, we found there was a lack ofhealth human resources to administer the immuniza-tions and that there was inadequate structural resources(e.g. a large enough space) to serve the target popula-tion. Based on our discussions with the CHS managers,we recommended increasing qualified staff during peaktimes (e.g., flu season, administration of childhood vac-cines) and increasing facility space for future CHS build-ings and renovations. Another example is that overall,the coordination between CHS facilities and other ser-vices (e.g., specialists, acute care) remains poor. Wefound there is virtually no communication between CHSfacilities and other places where patients go for theircare. We are working with CHS managers and specia-lists to find solutions regarding communication betweendifferent sites of care and working with the Chinese gov-ernment to strengthen the formal structure of care coor-dination so that the CHS becomes the primary place ofcare. Importantly, the local Health Bureau of these twodistricts commended the use of the China CHS LogicModel and its indicators.While CHS managers generally found the implemen-tation of this heuristic framework and indicators usefulin beginning to document their work, more research isneeded. We have conducted content validation of theLogic Model and PHC indicator content but will needto conduct more research with policy- and decision-makers, researchers, and evaluation specialists to providefurther refinements to the performance measurementframework. Further work is currently underway that willserve to further validate this set of indicators.The China CHS Logic Model and its indicators willalso be used, in part, to guide the development of aninformation system on measuring the quality and per-formance of the PHC sector. We will examine howexisting population-based data sources can be used tomonitor the CHS system, identify gaps in the currentdata landscape that hinder CHS performance measure-ment and recommend how these gaps might be filled.Currently, the Ministry of Health is designing the stan-dard National Resident Health Archive [40]. Based onthis work, the Community Health Association of China(CHAC) proposed the adoption and use of the ChinaCHS Logic Model and its indicators to the Ministry ofHealth. It is also expected that CHAC will use the LogicModel to guide complex analysis in order to inform thequality and performance of the CHS system.ConclusionsIn summary, A Logic Model framework can be useful inplanning, implementation, analysis and evaluation ofWong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 6 of 9Table 1 Examples of Core CHS Performance indicatorsCategory (n) Examples of Core Indicators Source ofdataInputs (31) Health Human Resources • % of qualified health care providers (physicians, nurses, nurse practitioners)in CHSHealthauthorityrecordsMaterial Resources • % of sub-districts who have at least one community health center HealthauthorityrecordsFiscal Resources • Amount of financial investment for capital infrastructure HealthauthorityrecordsActivities (64) Policy and governance level • The percentage of CHS facilities that can be reimbursed through publiclyfunded health insuranceHealthauthorityrecordsHealth care management level • % of PHC providers who completed a two-way referral of patients-a patientis referred for more specialized services or services unavailable through theCHS and that more specialized services (e.g., internal medicine) refer patientsto CHS facilities as their place of first contact with the health systemHealthauthorityrecordsClinical level • % of CHS facilities who can offer Chinese traditional medicine HealthauthorityrecordsOutputs (105) Type • % of PHC organizations who currently provide the following public healthservices (health education, illness prevention, etcCHS facilityVolume • % of patients with hypertension who have health care coordinated by acase managerCHS facilityQuality • % of patients who have a regular doctor• % of patients who were referred to other doctors and have informationback.• % of patients who report that they were given enough time to discusstheir feeling, fears and concerns• % of patients who rated the quality of CHS good or excellentPatient surveyImmediateoutcomes (65)Increased individual capacity • % of residents who have increased knowledge, skills, and confidence tomanager their healthPatient surveyReduced risk of ill-health andduration and effects of acuteconditions• Incidence rate of 0-3 year old children with low weight CHS facilityStabilization of Chronic Conditions • Control rate of patients with chronic diseases (such as hypertension) CHS facilityMaintain or improve satisfaction ofhealth care workforce• CHS provider satisfaction with CHS sector Provider surveyIntermediateoutcomes (15)Healthy Choices and Behaviors • % of population who currently engage in regular physical activity Patient surveyImprove prevention ofcomplications and acuteexacerbations• Hospitalization rate of patients with chronic diseases Patient surveyPublic acceptability of CHS • Patients’ satisfaction with CHS Patient surveyAppropriateness of place andprovider• % of patients who first see a CHS physician Patient surveyFinaloutcomes (7)Better health outcome • Decreased premature mortality Nationalreports(government)Health care system equity • Distribution of health outcome among different populations Nationalreports(government)Lower costs of health system • Health expenditure per capita in international dollars Nationalreports(government)Public satisfaction with healthsystem• Residents’ satisfaction with health system Patient surveyWong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 7 of 9PHC at a system and service level. The development andcontent validation of the China CHS Logic Model andsubsequent indicators provides a means for strongeraccountability and a clearer sense of overall directionand purpose needed to renew and strengthen the PHCsystem in China. Developing the logic model frameworkand relevant performance measurement indicators hasrequired the articulation of inputs, activities, outputs,and outcomes and extant indicators relevant to PHCrenewal in China. Although more work is needed infurther refinement of the framework, it will be useful inmoving towards developing a PHC information system,comparing common indicators across districts in China,and guiding the pursuit of quality in PHC.AcknowledgementsThis study was supported by Dr. Wong’s Michael Smith Scholar Award (CI-SCH-051) and the Canadian Institute for Health Research New Investigatoraward. We would also like to thank the generous support of BaoAn DistrictHealth Bureau and WuHou District Health Bureau.Author details1University of British Columbia, School of Nursing and Centre for HealthServices Policy Research, 6190 Agronomy Road, #302, Vancouver, BritishColumbia, V6T-1Z3, Canada. 2Capital Institute of Pediatrics and CommunityHealth Association of China. 2 YaBao Road, 328#, ChaoYang District, Beijing,100020, China. 3University of Toronto, Department of Family and CommunityMedicine and Li Ka Shing Knowledge Institute of St. Michael’s Hospital, 80Bond Street, First Floor, Toronto, Ontario, M5B 1X2, Canada. 4CommunityHealth Division, the Basic Health and Maternal and Child Care of HealthDepartment, Ministry of Health of The People’s Republic of China,XiZhimenWai Road 1, XiCheng District, Beijing, 100044, China.Authors’ contributionsSW, OB, and DY conceived of the study, participated in its design,developed the initial draft of the Logic Model and performance indicatorsand drafted the manuscript. DY also conducted the data collection forpolicy analysis, literature review, and stakeholder consultations. WB and LLparticipated in the policy analysis and interpretation of the stakeholderconsultations. BC is the principal investigator for this project. He obtainedthe funding, participated in its design and coordination of the datacollection. All authors were involved in analysis of data. All authors read andapproved the final manuscript.Competing interestsThe authors declare that they have no competing interests.Received: 3 June 2010 Accepted: 18 November 2010Published: 18 November 2010References1. National Statistics Bureau: China Statistical Yearbook 2008. Beijing, CN:National Bureau of Statistics of China; 2008.2. Hesketh TWX: Health in China: from Mao to market reform. BMJ 1997,314:1543-1545.3. Yardley J: Xinmin village journal: a deadly fever, once defeated, lurks in aChinese lake. NYT 2005.4. Blumenthal D, Hsiao W: Privatization and its discontents: the evolvingChinese health care system. N Engl J Med 2005, 353:1165-1170.5. Liu Y, Hsiao W, Eggleston K: Equity in health and health care: the Chineseexperience. Soc Sci and Medicine 1999, 49:1349-1356.6. Chen Z: Launch of the health-care reform plan in China. Lancet 2009,373:1322-1323.7. Macinko J, Starfield B, Shi L: The contribution of primary care systems tohealth outcomes within Organization for Economic Cooperation andDevelopment (OECD) countries, 1970-1998. Health Serv Res 2003,38(3):831-865.8. Shi L: Health care spending, delivery, and outcome in developedcountries: a cross-national comparison. American Journal of MedicalQuality 1997, 12, 83-84-93.9. Shi L, Starfield B, Politzer R, Regan J: Primary care, self-rated health, andreductions in social disparities in health. Health Serv Res 2002, 37(3), 529-530-550.10. Baiker K, Chandra A: Medicare spending, the physician workforce, andbeneficiaries’ quality of care. Health Aff 2004, web4 (April 7).11. Starfield B, Shi L: Policy relevant determinants of health: an internationalperspective. Health Policy 2002, 60(3):201-218.12. Pan X, Dib H, Wang X: Service utilization in community health centers inChina: a comparison analysis with local hospitals. BMC Health ServicesResearch 2006, 6:93.13. Beijing, CN: Ministry of Health: The management measures of CHSfacilities. Ministry of Health of the People’s Republic of China; 2006.14. Bodenheimer T: The American health care system: the movement forimproved quality in health care. N Engl J Med 1999, 340:488-492.15. Beijing, CN: Ministry of Health: Chinese Health Statistics summary. Ministryof Health of the People’s Republic of China; 2008.16. Beijing, CN: Ministry of Finance: Report on the 2008 central and localbudget performance and the 2009 central and local draft budget.Ministry of finance of the People’s Republic of China; 2009.17. Watson D: The development of a primary healthcare information systemto support performance measurement and research in British Columbia.Healthcare policy 2009, 5(Sp), 16-17-30.18. World Health Organization: Health systems: improving performance.Geneva: World Health Organization; 2000 [http://www.who.int/whr/2000/en/].19. Wang Y, Chen X, Cheng Y, Xiong J: Thinking of performance evaluationon regional community health services. Health Economics Research 2008,2:22-23.20. Treasury Board of Canada: Guide to the development of the results-basedmanagement and accountability frameworks. Ottawa: Treasury Board ofCanada Secretariat; 2006 [http://www.tbs-sct.gc.ca/eval/pubs/RMAF-CGRR/RMAF_Guide_e.pdf].21. Watson D: For discussion: a roadmap for population-based informationsystems to enhance primary healthcare in Canada. Healthcare Policy 2009,5(Sp), 105-106-120.22. Watson D, Broemeling A-M, Wong ST: A results-based logic model forprimary healthcare: a conceptual foundation for population-basedinformation systems. Healthcare Policy 2009, 5(Sp):35-46.23. Watson D, Broemeling A-M, Reid RJ, Black C: Results-based Logic Model toSupport Performance Measurement, Accountability, and Evaluation.Vancouver: Centre for Health Services and Policy Research; 2007.24. World Health Organization: The world health report 2008: primary healthcare now more than ever. Geneva: World Health Organization; 2008[http://www.who.int/whr/2008/en/].25. The State Council: Instruction and opinion of the development ofcommunity health services. Beijing, CN 2006.26. Beijing, CN: Ministry of Health: Facilitation and the Development ofcommunity health service. Ministry of Health of the People’s Republic ofChina; 2002.27. The Central Committee of the communist Party and State Council: PublicHealth Reform and Development. Beijing, CN 1997.28. Wong S, Watson D: Options for Consensus and Evidenced-BasedIndicators Relevant to Ontario Community Health Centres. Toronto:Association of Ontario Health Centres; 2006.29. Canadian Institute for Health Information: Pan-Canadian Primary HealthCare Indicators-Volume 1. Toronto: Canadian Institute for HealthInformation; 2006.30. Hawe P, King L, Noort M, Jordens C, Lloyd B: Indicators to help withcapacity building in health promotion. North Sydney: New South WalesHealth Department; 2000.31. Upshaw V: The national public health performance standards program:will it strengthen governance of local public health? Journal of PublicHealth management Practice 2000, 6(5), 1-2-18.32. Community Health Association of China: Assessment of demonstrationdistricts of community health services. Beijing, CN: Community HealthAssociation of China; 2005.Wong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 8 of 933. Community Health Association of China: National Census report oncommunity health service facilities in China. Beijing, CN: CommunityHealth Association of China; 2007.34. Watson D, Broemeling A, Wong ST: Results-Based Logic Model for PrimaryHealth Care: A Conceptual Foundation for Population-Based InformationSystems. Healthcare Policy 2009, 5:33-46.35. Community Health Association of China: Evaluation report onperformance of community health services in Wuhou district, ChengduCity. Beijing, CN: Community Health Association of China; 2008.36. Community Health Association of China: Evaluation report onperformance of community health services in Baoan district, ShenzenCity. Beijing, China: Community Health Association of China; 2008.37. Starfield B: Primary care reform in contemporary health care systems. InImplementing Primary Care Reform: Barriers and Facilitators. Edited by: WilsonR, Shortt S, Dorland J. Kingston, ON: McGill Queen’s University Press;2004:151-164.38. Shi L, Macinko J, Starfield B, Xu J, Regan J, Politzer R, Wulu J: Primary care,infant mortality, and low birth weight in the states of the USA. Journal ofEpidemiology & Community Health 2004, 58(5):374-380.39. Shi L, Macinko J, Starfield B, Wulu J, Regan J, Politzer R: The relationshipbetween primary care, income inequality, and mortality in US States,1980-1995. Journal of the American Board of Family Practice 2003,16(5):412-422.40. Ministry of health of the People’s Republic of China: The instructions ofthe specification of the National Residents’ Health Archive. Beijing:Ministry of Health; 2009.Pre-publication historyThe pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2296/11/91/prepubdoi:10.1186/1471-2296-11-91Cite this article as: Wong et al.: Developing a PerformanceMeasurement Framework and Indicators for Community Health ServiceFacilities in Urban China. BMC Family Practice 2010 11:91.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color figure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionSubmit your manuscript at www.biomedcentral.com/submitWong et al. BMC Family Practice 2010, 11:91http://www.biomedcentral.com/1471-2296/11/91Page 9 of 9


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items