UBC Faculty Research and Publications

An international cross-sectional survey on the Quality and Costs of Primary Care (QUALICO-PC): recruitment… Wong, Sabrina T; Chau, Leena W; Hogg, William; Teare, Gary F; Miedema, Baukje; Breton, Mylaine; Aubrey-Bassler, Kris; Katz, Alan; Burge, Fred; Boivin, Antoine; Cooke, Tim; Francoeur, Danièle; Wodchis, Walter P Feb 18, 2015

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12875_2015_Article_236.pdf [ 455.01kB ]
JSON: 52383-1.0223557.json
JSON-LD: 52383-1.0223557-ld.json
RDF/XML (Pretty): 52383-1.0223557-rdf.xml
RDF/JSON: 52383-1.0223557-rdf.json
Turtle: 52383-1.0223557-turtle.txt
N-Triples: 52383-1.0223557-rdf-ntriples.txt
Original Record: 52383-1.0223557-source.json
Full Text

Full Text

RESEARCH ARTICLE Open AccessAn international cross-sectional survey on thephysicians and 8,332 patients from 772 primary care practices completed the surveys, including 1,160 participantscompleting a Patient Values survey and 7,172 participants completing a Patient Experience survey. Overall, the responseWong et al. BMC Family Practice  (2015) 16:20 DOI 10.1186/s12875-015-0236-7Columbia (UBC), Vancouver, CanadaFull list of author information is available at the end of the articlerate was very low ranging from 2% (British Columbia) to 21% (Nova Scotia). However, the participation rate was high,ranging from 72% (Ontario) to 100% (New Brunswick/Prince Edward Island and Newfoundland & Labrador).Conclusions: The difficulties obtaining acceptable response rates by family physicians for survey participation is auniversal challenge. This response rate for the QUALICO-PC arm in Canada was similar to rates found in other countriessuch as Australia and New Zealand. Even though most family physicians operate as self-employed small businesses,they could be supported to routinely submit data through a collective effort and provincial mandate. The groundworkin setting up pan-Canadian collaboration in primary care has been established through this study.Keywords: Primary health care, Physician, Response rate, French, English, Survey, Practice* Correspondence: Sabrina.wong@nursing.ubc.ca1School of Nursing, University of British Columbia (UBC), Vancouver, Canada2Centre for Health Services and Policy Research, University of BritishQuality and Costs of Primary Care (QUALICO-PC):recruitment and data collection of placesdelivering primary care across CanadaSabrina T Wong1,2,3,4*, Leena W Chau1,2, William Hogg4,5,6, Gary F Teare7,9,10, Baukje Miedema11,12, Mylaine Breton13,Kris Aubrey-Bassler14,15, Alan Katz16,17, Fred Burge11, Antoine Boivin18, Tim Cooke19, Danièle Francoeur20 andWalter P Wodchis7,8,9AbstractBackground: Performance reporting in primary health care in Canada is challenging because of the dearth ofconcise and synthesized information. The paucity of information occurs, in part, because the majority of primaryhealth care in Canada is delivered through a multitude of privately owned small businesses with no mechanism orincentives to provide information about their performance. The purpose of this paper is to report the methodsused to recruit family physicians and their patients across 10 provinces to provide self-reported information aboutprimary care and how this information could be used in recruitment and data collection for future large scalepan-Canadian and other cross-country studies.Methods: Canada participated in an international large scale study-the QUALICO-PC (Quality and Costs of PrimaryCare) study. A set of four surveys, designed to collect in-depth information regarding primary care activities wascollected from: practices, providers, and patients (experiences and values). Invitations (telephone, electronic ormailed) were sent to family physicians. Eligible participants were sent a package of surveys. Provincial teams keptrecords on the number of: invitation emails/letters sent, physicians who registered, practices that were sent surveys,and practices returning completed surveys. Response and cooperation rates were calculated.Results: Invitations to participate were sent to approximately 23,000 family physicians across Canada. A total of 792© 2015 Wong et al.; licensee BioMed Central.Commons Attribution License (http://creativecreproduction in any medium, provided the orDedication waiver (http://creativecommons.orunless otherwise stated.This is an Open Access article distributed under the terms of the Creativeommons.org/licenses/by/4.0), which permits unrestricted use, distribution, andiginal work is properly credited. The Creative Commons Public Domaing/publicdomain/zero/1.0/) applies to the data made available in this article,Wong et al. BMC Family Practice  (2015) 16:20 Page 2 of 10BackgroundIn Canada, performance reporting in primary health care(PHC) is challenging because of the dearth of conciseand synthesized information [1]. The paucity of informa-tion occurs, in part, because the majority of PHC is pub-licly funded through a single payer (i.e. provincial andfederal governments) but delivered through a multitudeof privately owned small businesses [2], also known asPHC practices. The majority of PHC practices are familyphysician owned and operated; they employ staff, payoverhead, and provide health care services to patients.The majority of family physicians’ income is derivedfrom billing the government fees for services, which arenegotiated between each provincial government andorganized medicine in their respective jurisdictions, withsome being paid through a blend of payments [3]. Manyfamily physicians are part of a group or interprofessionalpractice, such as group practices or community healthcentres, while others provide care in a solo practice [3].Outside Canada, reporting on PHC performance alsoremains challenging [4-7]. Indeed, the activity of report-ing in PHC is not a traditional role providers are accus-tomed to. Despite PHC being publicly funded indeveloped countries, providers have not considered ittheir role to report on their performance or the perform-ance of their practice with the goal of improving thePHC sector or larger health care system.Yet, there are growing demands for performancereporting from many stakeholders including patients[8,9]. Regional case studies of performance reporting[10,11] and evidence from the hospital sector [12] indi-cate it can influence quality improvement agendas andimprove performance. Past work shows that publicreporting may improve performance [6,10,13-15], as ithas the potential to “improve the quality of care, in-crease accountability, facilitate public participation inhealth care,”([14], p.62, 15) impact societal and profes-sional values and direct attention to issues not currentlyon the policy agenda [16-18]. It may also facilitatecollaboration among stakeholders as they set a commonagenda [19]. While performance reporting in the hos-pital sector grows, performance reporting in PHC lagsbehind.The most commonly referenced performance informa-tion about PHC internationally is from the Common-wealth Fund patient and clinician surveys in industrializednations [20-26]. The surveys are based on samples of 1000patients or clinicians per country and show (for thedimensions addressed by the surveys) that PHC perform-ance in Canada is poor compared to other Organizationfor Economic Co-operation and Development (OECD)countries. In an attempt to improve the quality of infor-mation used to report on PHC performance, the inter-national QUALICO-PC (Quality and costs of primarycare) study was developed with the overarching goals of:1) examining the relationship between the strength of theprimary care system and the performance of the overallhealthcare system ([27] and 2), satisfy the demand forbenchmarking performance information and to informprimary care reform through cross national comparisons[27,28]. The purpose of this paper is twofold: 1) to reportthe methods used to recruit family physicians and theirpatients across the 10 provinces of Canada to participatein the QUALICO-PC study; and 2) to interpret patterns ofrecruitment to participate in this study. This work is im-portant in reflecting on what could be done in recruitmentand data collection for future large scale pan-Canadianand other cross-country studies.MethodsDesignThe QUALICO-PC study started as a research programfunded by the European Union (EU) including 26 memberstates and five non-EU European countries, Iceland,Macedonia, Norway, Switzerland, and Turkey [27,28]. Out-side Europe, Canada, Israel, Australia and New Zealandalso participated, funding their own participation. A totalof 34 countries participated. QUALICO-PC used a cross-sectional study design to collect self-reported data fromfamily physicians, their practices and 10 patients who wereseen by them.Survey contentA set of four surveys, designed to collect in-depth infor-mation regarding primary care activities was collected[27]. The surveys, which are described in detail else-where [27]. included concepts important to the deliveryand organization of primary care through individualpatient, physician, and practice surveys. The practicesurvey (PRA) collected information on organizationalfeatures such as design and delivery of primary care (e.g.,financing, regulation, resources), whereas the family phys-ician survey (FPS) collected information on the type oftasks and services (e.g., first contact care, prevention, con-tinuity of care, and integrated service provision) delivered.The patient experiences survey (PES) contained questionsaimed at collecting information on their experiencesincluding: coordination, continuity, quality of care, andequity in treatment of primary care. The patient valuessurvey (PVS) asked questions on the importance of access,quality of care (e.g. interpersonal communication) andservices delivered in primary care. The surveys and proce-dures for collecting the data were originally developed andvalidated by the European team [29]. Minor adjustmentswere made by the Canadian research team to align withthe different health care systems of the provinces, yetremained as close as possible to the validated Europeansurveys.Wong et al. BMC Family Practice  (2015) 16:20 Page 3 of 10Eligibility of participantsAll 10 provinces in Canada participated in QUALICO-PC;Canada’s provinces vary in geographic size and population,with Québec, Ontario, and British Columbia being thelargest provinces in terms of land mass and population.Prince Edward Island is Canada’s smallest province interms of land mass and population. Two of Canada’ssmaller provinces, New Brunswick and Prince EdwardIsland, combined their recruitment and data collectionefforts. Physicians who were working with a family/generalpractice (e.g., not specializing in a narrow set of conditionsor treatments) were eligible to participate. We maximizedrecruitment of the variety of practices where family physi-cians work by having only one physician per practiceeligible to participate. A "practice" was defined as one ormore physicians that share one of revenue, staff or pa-tients. Only patients of participating family physicianswere eligible to take part in the study. Patients had to be18 years of age or older, speak and read English or French,and not have cognitive impairment to participate.Sources and methods of participant selectionEach provincial team followed the same data collectionmethod but recruitment methods varied slightly fromprovince to province. For example, in Alberta, a noticeof the study was posted on both the Alberta MedicalAssociation and Alberta College of Family Physicianswebsites, and all related survey material was posted onthe Health Quality Council of Alberta site for physicianreference.All family physicians were recruited in collaborationwith organizations that had lists of practicing physicianssuch as the provincial chapters of the Canadian Collegeof Family Physicians. All provinces, except Québec andManitoba, used a census approach where all family phy-sicians on these organizations’ lists were recruited to bestudy participants. Some provinces (Manitoba, NewBrunswick and Prince Edward Island) worked with orga-nizations who could recruit from their membership,while other provinces (Québec) received permission touse a list of primary care physicians registered to theDepartment of Health or the Fédération des médecinsomnipraticiens du Québec. The research lead and provin-cial chapters of the Canadian College of Family Physiciansor other authorities (e.g., Departments of Family Medi-cine) gave their support in jointly inviting family physi-cians to participate in the Canadian QUALICO-PC studythrough mailed or emailed letters. Interested physiciansregistered either online or by fax. The Québec recruitmentdiffered in that physicians were called and could registerover the telephone if they did not respond by mail oremail. Additionally, Québec specifically tried to recruit astratified random sample of physicians by geographic areaand, in some cases, allowed participation by more thanone family physician from each practice to accommodategeographic areas with a lower number of practices. InManitoba recruitment was weighted so that 67% of thefamily physicians were in Winnipeg; the rest were ran-domized from non-Winnipeg regional health authoritiesto try and ensure equitable representation.ProceduresThe Canadian Primary Health Care Research andInnovation Network http://www.cphcrin-rcrissp.ca/ coor-dinated efforts across the 10 provinces [30]. They facili-tated the provincial research teams in meeting regularlythroughout the recruitment phase. This pan-Canadianworking group was responsible for planning and coord-inating data collection, data sharing, and analyses ofthe Canadian data [31]. An underlying principle duringthe recruitment phase of this working group was tominimize potential bias in recruitment by creating asupportive learning environment through sharing ethicsapplications and recruitment materials and mutualproblem solving. As much as possible data collectionfollowed a standard protocol and was led by one of theresearch team members from that province.Family physicians and patients were required to pro-vide informed consent. In some provinces, personalhealth numbers (PHNs) and full date of birth were col-lected from consenting patients for linkage to adminis-trative databases in order to determine relationshipsbetween physician, practice, patient experiences/values,utilization of medical services and health outcomes.Family physicians were compensated $200 CDN as atoken of recognition for the disruption to their work.Data were collected in 2013 and early 2014; Once datacollection started in each province, it lasted up to4 months except in Québec where data collection lastedfor 9 months. Physicians who registered were contactedby a research assistant by telephone or email to confirmeligibility and interest in participation. Each participatingphysician was couriered a package containing Scantron[32] format surveys, instructions, pens, and return courierpackages. In addition to filling out the provider (n = 1) andpractice (n = 1) surveys, physicians were asked to choose aday during which the patients to be seen represented theirregular patient panel. Patient surveys (n = 9 patient experi-ence; n = 1 patient values) were administered to consecu-tive consenting patients on this day. If the surveys werenot completed in a single day, recruitment continued forup to 3 days. Practices couriered back their completedsurveys. Practices that registered (online, telephone, fax,or email) were contacted up to a maximum of 21 times toencourage completion and return of their surveys. It is im-portant to note in cases where there were a higher numberof contacts, this was sometimes due to the research teamand family physician office leaving messages for eachother, the office asking the research team to call back, orthat the family physician was on holidays.De-identified survey data were scanned into a datafile by the provincial research team or couriered toCanmark Technologies Ltd. [33], a third party purveyorlocated in Toronto, Ontario. The provincial data fileswere subsequently merged into a national dataset forcross-jurisdictional comparisons. Any hard copies ofthe surveys originally couriered to Canmark were cour-iered back to the provincial sites to be stored in a securelocation. Separately, and only within each provincialresearch team, the personal health numbers were enterednotes were recorded by each provincial team to gain anunderstanding of recruitment challenges and reasonswhy practices were unable to participate. All recruitmentdata were managed by one research assistant in BritishColumbia.Response rates were calculated as the number of physi-cians who signed up to participate in the study dividedby the total number of invitations sent out. Participationrates were calculated as the number of physicians whowere sent a package of surveys divided by the number ofphysicians who signed up to participate in the study.Cooperation rates were calculated as the number ofWong et al. BMC Family Practice  (2015) 16:20 Page 4 of 10along with the study ID number into a secure encryptedfile. All procedures were approved by behavioural researchethics boards (BREBs) located at the institution whereeach provincial lead investigator was affiliated (BREBnumber for lead author’s institution: H12-03300). SeeTable 1 for the names of the research ethics boards.In Canada, we followed the protocol devised by ourEuropean counterparts. The target sample for Europeancountries was generally 220 family physicians from 220different practices per country with a few smaller coun-tries aiming for samples of 75 family physicians from 75different practices. In order to compare across provinces,our target sample was 220 family physicians from differentpractices in three of the most populous provinces (Alberta,Ontario and Québec) and 75 family physicians fromdifferent practices in all other provinces (with NewBrunswick and Prince Edward Island combined as asingle sampling unit).Data management and analysisUsing an excel spreadsheet, each provincial team wasasked to keep records on the number of: physician re-cruitment emails/letters sent, physicians who registered,practices who were sent surveys, and practices whoreturned the completed set of surveys. Data were alsorecorded on the number of contact attempts made topractices to encourage their participation. Finally, fieldTable 1 Research Ethics BoardsProvince InstitutionBritish Columbia University of British ColumbiaAlberta Health Quality Council of AlbertaSaskatchewan Health Quality Council of SaskatchewanManitoba University of ManitobaOntario University of TorontoUniversity of OttawaQuébec University of SherbrookeNew Brunswick/PEI Horizon Health NetworkNova Scotia Dalhousie UniversityNew Foundland & Labrador Memorial Universityphysicians who returned completed surveys divided bythe total number of physicians who received surveys.Field note data, including reasons why practices wereunable to participate were aggregated into themes.Credibility and trustworthiness [34] of the data wereundertaken by having the provincial teams discuss theresulting themes and also through international [35]and national [36] dialogue.ResultsInvitations to participate were sent to approximately23,000 family physicians across Canada. The majority ofinvitations were sent to family physicians in Ontario,followed by British Columbia, and Alberta. The fre-quency of invitations (including initial) sent ranged from2–4 across the provinces. The frequency of follow-upphone calls to encourage registered physicians and theirpractices to complete data collection and return theirpackages ranged from 1–21. The mean number rangedfrom 3.3 in Newfoundland and Labrador to 6.53 inQuébec.Across Canada a total of 792 physicians and 8,332patients from 772 primary care practices in Canadacompleted the surveys, including 1,160 participants whocompleted a Patient Values survey and 7,172 participantswho completed a Patient Experience survey (Table 2).Patient participants who also consented to link theirEthics boardBehavioural Research Ethics BoardCommunity Research Ethics Board of AlbertaUniversity of Sakatchewan Behavioural Research Ethics BoardHealth Research Ethics BoardHealth Sciences Research Ethics BoardHealth Sciences and Science Research Ethics BoardComité institutionnel d'éthique de la recherche avec les êtres humainsResearch Ethics BoardHealth Sciences Research Ethics BoardInterdisciplinary Committee on Ethics in Human Researchsurvey data to administrative data ranged from 57% inNew Brunswick/Prince Edward Island to 86% in BritishColumbia.Four sites across five provinces, Ontario, Québec, NewBrunswick/Prince Edward Island, and Nova Scotia (seeFigure 1, response rate) met their targeted sample size ofpractices through the recruitment procedures. Overall,the response rate was very low ranging from 2% (BritishColumbia) to 21% (Nova Scotia). Even with a longer datacollection period, Québec achieved only an 18% responseTable 2 QUALICO-PC completed surveys by provinceCompleted surveys BritishColumbiaAlberta SaskatchewanManitoba Ontario Québec NewBrunswick/PEINovaScotiaNewFoundland& LabradorTotalcompletedsurveysPhysician surveys, n 59 116 20 41 184 218 54 59 41 792Practice surveys, n 58 117 20 24 183 218 53 58 41 772# Patient experiencesurveys, n537 1240 185 353 1698 1798 497 544 320 7,172# Patient valuesSurveys, n90 207 33 48 282 289 69 92 50 1,160Total surveys byprovince744 1680 258 466 2347 2523 673 753 452Invitations Sentto physicians, nNB/PEIBC NLNSQCONSK MBAB4200 11,000317(3%)3033 900 481 1585 501 53865(14%)204(7%)33(4%)88(2%)104(21%)262(17%)72(13%)63379(12%)esponse Rateof practices,n(%)Wong et al. BMC Family Practice  (2015) 16:20 Page 5 of 10rticipationRate ofctices, n(%)70(80%)186(91%)26(79%)61(94%)RPapraCooperation Rate ofpractices, n(%)59(84%)131(70%)20(77%)41(67%)Figure 1 Flow diagram: Pan-Canadian QUALICO-PC Recruitment. Respup to be part of this study divided by the total number of invitations sentphysicians who were sent a package of surveys divided by the number ofrates were calculated as the number of physicians who returned completesurveys. Abbreviations: British Columbia (BC), Alberta (AB), Saskatchewan (SEdward Island (NB/PEI), Nova Scotia (NS), Newfoundland and Labrador (NL)229(72%)259(99%)79(100%)80(77%)72(100%)185(81%)41(57%)60(75%)218(84%)55(70%)onse rates were calculated as the number of physicians who signedout. Participation rates were calculated as the final number of eligiblephysicians who signed up to participate in the study. Cooperationd surveys divided by the total number of physicians who receivedK), Manitoba (MB), Ontario (ON), Québec (QC), New Brunswick/Prince.Wong et al. BMC Family Practice  (2015) 16:20 Page 6 of 10rate. However, the participation rate was high, rangingfrom 72% (Ontario) to 100% (New Brunswick/PrinceEdward Island and Newfoundland & Labrador). Thecooperation rate (see Figure 1) shows that although noprovince successfully collected data from the targetednumber of practices, once practices agreed to partici-pate and received the data collection package, the ma-jority of them were able to return completed surveys.The cooperation rate ranged from 57% (Newfoundlandand Labrador) to 84% (British Columbia and Québec).Across all provinces except New Brunswick/PrinceEdward Island and Newfoundland, there was some at-trition that affected the participation rate (n = 162). Inconfirming eligibility, 65 (40%) of the family physicianswere found to be located in the same practice as one oftheir colleagues who had already been admitted to thestudy. In some cases, one of the team members wasunable to reach the physician who had signed up online(n = 32). In other cases, once data collection procedureswere explained by our team, physicians declined to par-ticipate because many of their clients did not speak orread English or French well (n = 26), they were too busy(n = 12), or had other reasons (n = 26), such as beingaway or on maternity leave during the time data wereto be collected.There was also some attrition that affected the cooper-ation rate (n = 254) for similar reasons. Nineteen (7%)physicians did not meet our eligibility requirements orour team was unable to contact the physician (46%).Other common reasons for attrition in the cooperationrate was that some physicians (21%) reported thatadministering the survey was not feasible because appro-priate approval in working with First Nation patientshad not been sought by the practice, or stated that thesurveys would take too long to complete.Half the 792 family physicians were female and justunder half (45.3%) were 65 years or older (Table 3). Themajority (71.6%) were born in Canada with an over-whelming percentage (90.3%) indicating they were self-employed and working in group practices. Providersworked on average 41 hours per week, though there waswide variability in their reported work hours. Participat-ing practices were from a variety of geographic locations,with just under one-third reporting they delivered ser-vices in an inner city. Just under 40% of physiciansperceived that the number of elderly patients fromtheir panel was above national average. Over half re-ported being open after 6 pm with fewer practices hav-ing connections to either lab (29.8%) or x-ray (19.3%)facilities.The majority of patient respondents were female(66.4%). Many patients were between the ages of 45–64years (41.9%). Four out of every 10 patient respondentsreported not having a post-secondary education. Themajority of patients reported their health as very goodor good (73.7%) and had a chronic condition (55.6%).DiscussionInitial response rates in our study varied from provinceto province but were generally low across Canada. Thisresponse rate for the QUALICO-PC arm in Canadawas similar to rates found in other countries such asAustralia [37] (ranged from below 1% to 14.5%) andNew Zealand (12.2%). Indeed, the difficulties obtainingacceptable response rates from family physicians forsurvey participation is a universal challenge. However,our results suggest that once family physicians agree toparticipate in a survey, their participation and cooper-ation rates are high.There are some lessons learned from in the recruit-ment of family physicians for multijurisdictional stud-ies. In Canada, where funding was obtained frommultiple sources and varied by province, the low pay-ment to participating physicians made it possible for all10 provinces to participate. Yet, the low response rateis reflective of the minimal amount of resources avail-able for recruitment in this pan-Canadian study. Thisamount may seem like a high amount for participationin a study; ethics boards would suggest even higheramounts may unnecessarily influence potential partici-pants to ‘voluntarily enter the study’ [38]. From a smallbusiness viewpoint, this amount of money could beseen as less than a token of appreciation since it wouldnot cover even half the costs of disruption to theirbusinesses. Offering a larger payment in appreciation ofthe disruption to their practices might have improvedthe initial response rate but likely not enough to meetthe targeted sample size at the provincial level. Pastwork has shown that a monetary incentive significantlyimproves physicians’ response rates [39] yet, little re-mains known about how much monetary incentive isneeded to achieve adequate response rates (e.g., ≥50%)[40] amidst the known trend of decreasing responserates in this population [41,42]. There are likely otherfactors that also influence family physician participationin studies such as the lack of time in building relation-ships with the front office staff. A fine balance betweenresources available and payment for participation needsto be carefully assessed.A low response rate from family physicians is, inpart, due to structural challenges. Without a centralsource for such data, researchers and others (includingprovinces) must individually seek out family physiciansand their patients. Other sectors of the health care systemare required to regularly report on specific information toa provincial and often a pan-Canadian repository that ismade available to them for the purpose of quality im-provement and others for the purpose of planningTable 3 Respondent characteristicsProvider’s characteristics n = 792Sex, Female: n (%) 393 (49.6)Age: n (%)Under 44 19 (2.4)45-64 408 (52.2)65+ 354 (45.3)Born in Canada: n (%)Yes 563 (71.6)Self-employed or salaried: n (%)Self-employed 708* (90.3)Average hours worked per week: Mean (SD) 40.7 (12.7)Practice characteristicsSolo or group practice: n (%)Group practice 708 (90.3)Geographic profile: n (%)Inner city 236 (30.1)Suburbs 136 (17.3)Small town 153 (19.5)Mixed urban–rural 107 (13.6)Rural 152 (19.4)Patients above national average: n (%)Elderly 305 (38.6)Disadvantaged 226 (28.6)Ethnic minority 157 (19.9)Extended hours: n (%)Open after 6 pm (at least once a week) 432 (54.9)Open weekends (at least once a month) 303 (38.9)Access within practice/centre to: n (%)Lab facilities 236 (29.8)X-ray facilities 152 (19.3)Patient characteristics 8,332 (7,172 patient experiences + 1,160 patient values)Sex, Female: n (%) N = 5,447 (66.7)Age: n (%)18-30 957 (12.1)31-44 1,678 (21.2)45-64 3,320 (41.9)65+ 1,971 (24.9)Education: n (%)No qualifications, pre-primary, primary, or lower secondary 761 (9.4)Upper secondary education (grades 10–12) 2,587 (31.9)Post-secondary education (includes college, undergraduate) 4,765 (58.7)Presence of chronic condition(s): n (%)Yes 4,547 (55.5)General health status: n (%)Very good 1,859 (22.6)Wong et al. BMC Family Practice  (2015) 16:20 Page 7 of 10be needed.00Wong et al. BMC Family Practice  (2015) 16:20 Page 8 of 10health services and research. Examples of these datainclude the Discharge Abstract Database from acutecare and the Resident Assessment Instrument fromlong term and home care. Not only is does eachorganization required to report data have specific re-sources to obtain the necessary data but at the pan-Canadian level, the Canadian Institute for Health Infor-mation [43] manages the quality of the data. Currentlyprimary care has no such required reporting require-ments or the infrastructure to produce any informationusing such data.Another structural challenge in collecting data fromprimary care is that no province keeps an up-to-date listof practicing family physicians. This is somewhat sur-prising given the amount of public money spent onphysician services by government. What we did find isthat the degree to which provinces have accurate familyphysician lists varies. Some of the smaller provinces up-date their lists every few years whereas in the largerprovinces, accuracy of the lists exists at a healthauthority or other organizational (e.g. division of familypractice in British Columbia) level.As with any study, this work has limitations. Ourrecruitment strategy was to contact individual physiciansto participate in our study because no list exists to re-cruit at a practice level. Using the lists of family physi-cians we could obtain, it is likely that our denominatorused for calculating the response rate is larger than whowas actually eligible to participate (only one family phys-ician per practice who did not have a specialized prac-tice). Although we had limited resources to carry outrecruitment, more highly resourced and similar surveys(e.g. Commonwealth Fund International Health PolicySurvey [13], Commonwealth Fund International HealthPolicy Survey of Sicker Adults [44]) also have poorresponse rates (albeit higher than ours). Cross countryTable 3 Respondent characteristics (Continued)GoodFairPoorNote. All characteristics are self-reported. All categorical data do not add up to 1salaried and self-employed.surveys aimed at family physicians may be able to increaseits response rates by putting more effort (and resources)into creating situations which could increase the facevalidity of the study [45]. Finally, given the low responserate within provinces, it is not possible to compare pri-mary care practices at a provincial level. However, usefulanalyses looking for associations between the quality ofprimary care and practice and physician characteristicscan still be undertaken, similar to what has been done bythe Commonwealth fund [13,44,46].Despite these limitations and overall low response rate,the data collected through the Canadian arm ofQUALICO-PC represents the largest dataset on thequality and organization of primary care data in Canada.Through our collective recruitment efforts we have thelargest number of cooperating practices, physicians, andpatients compared to any other country participating inQUALICO-PC. These data can tell us about importantpatient experiences and values among those who haveaccess to primary care and how practice and providercharacteristics might impact patient outcomes such asactivation, the degree to which patients become engagedto manage their own care [47].ConclusionGenerally there is a need to have better information inthe area of primary care. Challenges that need to beaddressed in how to obtain better information includeincreasing response rates from primary care providersand using an appropriate sampling strategy that ac-knowledges the shift to team-based care and increasinglydifferent models of care (e.g. nurse practitioner prac-tices). Moreover, a more coordinated effort to gather thisinformation using short, valid surveys is needed inWe also note that the recruitment methods variedacross provinces. Regardless of trying to recruit usinga randomized sample or a census approach, the re-sponse rate remains low across all provinces. It is pos-sible that a more involved and longer recruitmentapproach such as what was used in Québec couldincrease participation in future practice-based primarycare studies. In order to implement this acrossCanada, more resources such as an up-to-date phys-ician list and full time staff to conduct the study would4,249 (51.6)1,797 (21.8)332 (4.0)% due to missing data. *14 of these 708 respondents indicated they are bothCanada but also internationally. Even though most fam-ily physicians operate as small businesses, they could besupported to routinely submit data through a collectiveeffort and provincial mandate. The groundwork in set-ting up pan-Canadian collaboration in primary care hasbeen established through this study. Future work toattain external validity is now needed. Establishing aninformation structure that routinely collects primarycare data will be key to developing evidence-informedpolicy and delivery of health services.Wong et al. BMC Family Practice  (2015) 16:20 Page 9 of 10AbbreviationsQUALICO-PC: Quality and Costs of Primary Care; PHC: Primary Health Care;OECD: Organization for Economic Co-operation and Development;EU: European Union; PRA: Practice Survey; FPS: Family Physician Survey;PES: Patient Experiences Survey; PVS: Patient Values Survey; PHN: PersonalHealth Number.Competing interestsThe authors declare that they have no competing interests.Authors’ contributionsSW, LC, WH, GT, and WW conceptualized the manuscript and were involvedin writing and critically revising. SW, LC completed the analysis. BM, MB,KA-B, AK, DF, FB, AB, TC were involved in the interpretation of the resultsand critically revising the manuscript. All authors collected the data and readand approved the final manuscript.AcknowledgementsWe would like to thank the physicians, office staff, and patients whoparticipated in this study. We would also like to acknowledge the support ofthe research assistants and data analysts Kevin Walker and Sandra Peterson.Financial support was provided by the Canadian Institutes of HealthInformation and the Canadian Foundation for Healthcare Improvement(CFHI). Provincial research grants were received from the Nova Scotia HealthServices Research Foundation, la Commissaire à la santé et au bien-être et laFédération des médecins omnipraticiens du Québec, and the Health SystemPerformance Research Network funded by the Ontario Ministry of Health andLong Term Care. The views expressed in this manuscript are those of the au-thors and do not necessarily represent the views of the funders.Author details1School of Nursing, University of British Columbia (UBC), Vancouver, Canada.2Centre for Health Services and Policy Research, University of BritishColumbia (UBC), Vancouver, Canada. 3Department of Family Medicine,University of British Columbia (UBC), Vancouver, Canada. 4The CanadianPrimary Health Care Research and Innovation Network (CPHCRIN), Ottawa,Canada. 5Department of Family Medicine, University of Ottawa, Ottawa,Canada. 6C.T. Lamont Primary Health Care Research Centre, Élisabeth BruyèreResearch Institute, Ottawa, Canada. 7Institute of Health Policy Managementand Evaluation, University of Toronto, Toronto, Canada. 8TorontoRehabilitation Institute, Toronto, Canada. 9Institute for Clinical EvaluativeSciences, Toronto, Canada. 10Saskatchewan Health Quality Council,Saskatoon, Canada. 11Faculty of Medicine, Dalhousie University, Halifax,Canada. 12Family Medicine Teaching Unit, University of New Brunswick,Fredericton, Canada. 13Université de Sherbrooke, Charles LeMoyne ResearchCenter Hospital, Sherbrooke, Canada. 14Faculty of Medicine, MemorialUniversity of Newfoundland, St. John’s, Canada. 15Health Sciences Centre, St.John’s, Canada. 16Departments of Community Health Sciences and FamilyMedicine, University of Manitoba, Winnipeg, Canada. 17Manitoba Centre forHealth Policy, Winnipeg, Canada. 18Department of Family Medicine,University of Montreal, Montreal, Canada. 19Health Quality Council of Alberta,Calgary, Canada. 20Institut national de santé publique du Québec (INSPQ),Québec, Canada.Received: 27 October 2014 Accepted: 29 January 2015References1. Veillard J, Huynh T, Ardal S, Kadandale S, Klazinga NS, Brown AD. Makinghealth system performance measurement useful to policy makers: Aligningstrategies, measurement and local health system accountability in ontario.Healthc Policy. Canada; 2010;5(3):49.2. Hutchison B, Levesque J-F, Strumpf E, Coyle N. Primary health care inCanada: systems in motion. Milbank Q. 2011;89(2):32.3. College of Family Physicians of Canada, Canadian Medical Association,Royal College of Physicians and Surgeons of Canada. 2007 NationalPhysician Survey [Internet]. 2007 [cited 2014 Aug 19]. Available from:http://nationalphysiciansurvey.ca/surveys/2007-survey/4. Centers for Disease Control and Prevention. Ambulatory health caredata. Survey instruments: NAMCS survey instruments [Internet]. Atlanta,GA: Centers for Disease Control and Prevention; 2011. Available from:http://www.cdc.gov/nchs/ahcd/ahcd_survey_instruments.htm.5. Fung CH, Lim Y-W, Mattke S, Damberg C, Shekelle PG. Systematic review:the evidence that publishing patient care performance data improvesquality of care. Ann Intern Med. 2008;148(2):111–23. United States: AmericanCollege of Physicians.6. Watson DE. For discussion: a roadmap for population-based informationsystems to enhance primary healthcare in Canada. Healthc Policy. 2009;5(Spec no):105. Canada.7. Canadian Institute for Health Information. Experiences With Primary HealthCare in Canada [Internet]. Ottawa, Ont: Canadian Institute for HealthInformation; 2009. p. 1–23. Available from: https://secure.cihi.ca/free_products/cse_phc_aib_en.pdf.8. Berta W, Barnsley J, Brown A, Murray M. In the eyes of the beholder:population perspectives on performance priorities for primary care inCanada. Healthc Policy. 2008;4(2):86–100.9. Shortell SM, Casalino LP. Health care reform requires accountable caresystems. J Am Med Assoc. 2008;300(1):95–7. United States: AMER MEDICALASSOC.10. Smith MA, Wright A, Queram C, Lamb GC. Public reporting helped drivequality improvement in outpatient diabetes care among Wisconsin physiciangroups. Health Aff. United States: PROJECT HOPE; 2012;31(3):570–7.11. Young GJ. Multistakeholder regional collaboratives have been key drivers ofpublic reporting, but now face challenges. Health Aff. United States:PROJECT HOPE; 2012;31(3):578–84.12. Tu J V, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, et al.Effectiveness of Public Report Cards for Improving the Quality of CardiacCare: The EFFECT Study: A Randomized Trial. J Am Med Assoc. UnitedStates: AMER MEDICAL ASSOC; 2009;302(21):2330–7.13. The Commonwealth Fund. 2011 Commonwealth Fund International HealthPolicy Survey [Internet]. New York, NY: The Commonwealth Fund; 2011.Available from: http://www.commonwealthfund.org/Surveys/2011/Nov/2011-International-Survey.aspx.14. Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reportingin health care: how do consumers use quality-of-care information? Asystematic review. Med Care. United States: Lippincott Williams & Wilkins;2009;47(1):1–8.15. Hibbard JH, Greene J, Sofaer S, Firminger K, Hirsh J. An experiment showsthat a well-designed report on costs and quality can help consumerschoose high-value health care. Health Aff. United States: PROJECT HOPE;2012;31(3):560–8.16. Powell AE, Davies HT, Thomson RG. Using routine comparative data toassess the quality of health care: understanding and avoiding commonpitfalls. Qual Saf Health Care. 2003;12(2):122–8.17. Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT tools forevidence-informed health policymaking (STP) 1: what is evidence-informedpolicymaking? Heal Res Policy Syst. 2009;7 Suppl 1:S1.18. Oxman AD, Lewin S, Lavis JN, Fretheim A. SUPPORT tools forevidence-informed health policymaking (STP) 15: engaging the public inevidence-informed policymaking. Heal Res Policy Syst. 2009;7 Suppl 1:S15.19. Van Walraven C, Dhalla IA, Bell C, Etchells E, Stiell IG, Zarnke K, et al.Derivation and validation of an index to predict early death or unplannedreadmission after discharge from hospital to the community. Can MedAssoc J. 2010;182(6):551–7.20. Blendon RJ, Schoen C, DesRoches C, Osborn R, Zapert K. Common concernsamid diverse systems: health care experiences in five countries. Health Aff.2003;22(3):106–21.21. Blendon RJ, Schoen C, Donelan K, Osborn R, DesRoches CM, Scoles K, et al.Physicians’ views on quality of care: a five-country comparison. Health Aff.2001;20(3):233–43.22. Schoen C, Davis K, DesRoches C, Donelan K, Blendon R, Strumpf E. Equity inhealth care across five nations: summary findings from an internationalhealth policy survey. Washington, DC: Commonwealth Fund; 2000. p. 1–7.23. Schoen C, Osborn R, Huynh PT, Doty M, Zapert K, Peugh J, et al. Taking thepulse of health care systems: Experiences of patients with health problemsin six countries. Health Aff. 2005;Suppl Web(W5):509–25.24. Schoen C, Osborn R, Trang Huynh P, Doty M, Davis K, Zapert K, et al.Primary care and health system performance: Adults’ experiences in fivecountries. Health Aff. 2004;Suppl Web(W4):487–50325. Schoen C, Osborn R, Squires D, Doty M, Rasmussen P, Pierson R, et al. Asurvey of primary care doctors in ten countries shows progress in use ofSubmit your next manuscript to BioMed CentralWong et al. BMC Family Practice  (2015) 16:20 Page 10 of 10health information technology, less in other areas. Health Aff.2012;31(12):2805–16.26. Davis K, Stremikis K, Schoen C, Squires D. Mirror, Mirror on the Wall, 2014Update: How the U.S. Health Care System Compares Internationally[Internet]. Available from: http://www.commonwealthfund.org/publications/fund-reports/2014/jun/mirror-mirror27. Schäfer WLA, Boerma WGW, Kringos DS, De Maeseneer J, Gress S,Heinemann S, et al. QUALICOPC, a multi-country study evaluating quality,costs and equity in primary care. BMC Fam Pract. 2011;12(1):115.28. Laberge M, Pang J, Walker K, Wong S, Hogg W, Wodchis P. QUALICOPC(Quality and Costs of Primary Care) Canada – A focus on the aspects ofprimary care most highly rated by current patients of primary carepractices. 2014.29. Schäfer WLA, Boerma WGW, Kringos DS, De Ryck E, Greß S, Heinemann S,et al. Measures of quality, costs and equity in primary health careinstruments developed to analyse and compare primary care in 35countries. Qual Prim Care. 2013;21(2):67–79.30. Hogg W, Dyke E. Improving measurement of primary care systemperformance. Can Fam Physician. Mississauga: Coll Family PhysiciansCanada; 2011;57(7):758–60.31. Hogg W, Wodchis W, Katz A, Wong ST, Cullen R, Yelland G. Bringing Canadatogether: An effective organizational structure for multi-jurisdictional healthservices research projects. Can Family Phys. In press.32. Scantron. Scantron [Internet]. 2012 [cited 2014 Sep 23]. Available from:http://www.scantron.com/.33. Canmark Technologies Ltd. Canmark Technologies Ltd. [Internet]. 2014[cited 2014 Aug 19]. Available from: http://www.canmarktech.com/.34. Thorne S. Interpretive Description. Walnut Creek, CA: Left Coast Press; 2008.35. Wong ST, Wodchis W, Hogg W, Katz A, Cooke T, Lahtinen M, et al.QUALICO-PC in Canada-Recruitment, Responses, and Initial Cross CountryComparison With Australia and New Zealand. New Zealand: Health Servicesand Policy Research Conference; 2013. Available from: http://www.hsraanz.org/Home.aspx.36. Wong ST, Katz A, Wodchis W, Teare G, Burge F, Ashton T, et al. QUALICO-PCin Canada, Australia, and New Zealand: Recruitment, Responses, and InitialCross Country Comparison With Australia and New Zealand. Toronto, ON:Canadian Association for Health Services and Policy Research; 2014.37. Parkinson A, Jorm L, Douglas KA, Gee A, Sargent GM, Lujic S, et al.Recruiting general practitioners for surveys: reflections on the difficultiesand some lessons learned. Aust J Prim Health. http://dx.doi.org/10.1071/PY13129.38. UBC Behavioural Research Ethics Office. Behavioural Guidance Notes -Reimbursements & Payments [Internet]. 2014 [cited 2014 Sep 23].Available from: http://research.ubc.ca/ore/behavioural-guidance-notes-reimbursements-payments.39. VanGeest JB, Johnson TP, Welch VL. Methodologies for improving responserates in surveys of physicians: a systematic review. Eval Health Prof[Internet]. 2007 Dec [cited 2014 Dec 24];30(4):303–21. Available from:http://www.ncbi.nlm.nih.gov/pubmed/17986667.40. James KM, Ziegenfuss JY, Tilburt JC, Harris AM, Beebe TJ. Getting physiciansto respond: the impact of incentive type and timing on physician surveyresponse rates. Health Serv Res [Internet]. 2011 Feb [cited 2014 Dec 24];46(1 Pt 1):232–42. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3034272&tool=pmcentrez&rendertype=abstract.41. Changes in telephone survey nonresponse over the past quarter century -WebSM [Internet]. [cited 2014 Dec 24]. Available from: http://www.websm.org/db/12/15984/Web Survey Bibliography/Changes_in_telephone_survey_-nonresponse_over_the_past_quarter_century/.42. Berk ML, Schur CL, Feldman J. Twenty-five years of health surveys: doesmore data mean better data? Health Aff (Millwood) [Internet]. [cited 2014Dec 24];26(6):1599–611. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17978380.43. Canadian Institute for Health Information. Canadian Institute for HealthInformation [Internet]. 2014 [cited 2014 Aug 19]. Available from:http://www.cihi.ca/CIHI-ext-portal/internet/EN/Home/home/cihi000001.44. The Commonwealth Fund. 2008 Commonwealth Fund International HealthPolicy Survey of Sicker Adults [Internet]. New York, NY: The CommonwealthFund; 2009. Available from: http://www.commonwealthfund.org/Surveys/2008/2008-Commonwealth-Fund-International-Health-Policy-Survey-of-Sicker-Adults.aspx.and take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color figure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistribution45. Van der Heide A, Deliens L, Faisst K, Nilstun T, Norup M, Paci E, et al. End-of-life decision-making in six european countries: descriptive study. Lancet.2003;362(9381):345–50.46. The Commonwealth Fund. Surveys [Internet]. New York, NY: TheCommonwealth Fund; Available from: http://www.commonwealthfund.org/Surveys.aspx.47. Hibbard JH, Greene J. What the evidence shows about patient activation:better health outcomes and care experiences; fewer data on costs. HealthAff. 2013;32(2):207–14.Submit your manuscript at www.biomedcentral.com/submit


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items