UBC Faculty Research and Publications

The challenge of obtaining information necessary for multi-criteria decision analysis implementation:… Dionne, Francois; Mitton, Craig; MacDonald, Tanya; Miller, Carol; Brennan, Michael May 20, 2013

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12962_2012_Article_167.pdf [ 237.39kB ]
JSON: 52383-1.0221250.json
JSON-LD: 52383-1.0221250-ld.json
RDF/XML (Pretty): 52383-1.0221250-rdf.xml
RDF/JSON: 52383-1.0221250-rdf.json
Turtle: 52383-1.0221250-turtle.txt
N-Triples: 52383-1.0221250-rdf-ntriples.txt
Original Record: 52383-1.0221250-source.json
Full Text

Full Text

RESEARCH Open AccessThe challenge of obtaining information necessaryfor multi-criteria decision analysisimplementation: the case of physiotherapyservices in CanadaFrancois Dionne1*, Craig Mitton1, Tanya MacDonald2, Carol Miller2 and Michael Brennan2AbstractBackground: As fiscal constraints dominate health policy discussions across Canada and globally, priority-settingexercises are becoming more common to guide the difficult choices that must be made. In this context, itbecomes highly desirable to have accurate estimates of the value of specific health care interventions.Economic evaluation is a well-accepted method to estimate the value of health care interventions. However,economic evaluation has significant limitations, which have lead to an increase in the use of Multi-Criteria DecisionAnalysis (MCDA). One key concern with MCDA is the availability of the information necessary for implementation. Inthe Fall 2011, the Canadian Physiotherapy Association embarked on a project aimed at providing a valuation ofphysiotherapy services that is both evidence-based and relevant to resource allocation decisions. The frameworkselected for this project was MCDA. We report on how we addressed the challenge of obtaining some of theinformation necessary for MCDA implementation.Methods: MCDA criteria were selected and areas of physiotherapy practices were identified. The building up of thenecessary information base was a three step process. First, there was a literature review for each practice area, oneach criterion. The next step was to conduct interviews with experts in each of the practice areas to critique theresults of the literature review and to fill in gaps where there was no or insufficient literature. Finally, the results ofthe individual interviews were validated by a national committee to ensure consistency across all practice areas andthat a national level perspective is applied.Results: Despite a lack of research evidence on many of the considerations relevant to the estimation of the valueof physiotherapy services (the criteria), sufficient information was obtained to facilitate MCDA implementation atthe local level.Conclusions: The results of this research project serve two purposes: 1) a method to obtain information necessaryto implement MCDA is described, and 2) the results in terms of information on the benefits provided by each ofthe twelve areas of physiotherapy practice can be used by decision-makers as a starting point in theimplementation of MCDA at the local level.Keywords: Physiotherapy, MCDA, Resource Allocation, Priority Setting* Correspondence: fdionne@telus.net1Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal HealthResearch Institute, School of Population and Public Health, University ofBritish Columbia, 7th Floor, 828 West 10th Avenue, Research Pavilion,Vancouver, BC V5Z 1M9, CanadaFull list of author information is available at the end of the article© 2013 Dionne et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the CreativeCommons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, andreproduction in any medium, provided the original work is properly cited.Dionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11http://www.resource-allocation.com/content/11/1/11BackgroundAs fiscal constraints dominate health policy and planningdiscussions both across Canada and globally, priority-setting exercises are becoming more common to guide thedifficult choices that must be made [1]. In this context, itis not only appropriate but also highly desirable to assessthe value of specific health care services, as an assessmentof value is necessary for priority setting on resource alloca-tion either through the use of a threshold (minimum valueper dollar spent) or through a formalized priority-settingprocess such as Program Budgeting and Marginal Analysis(PBMA).A common approach to the assessment of the value ofhealth care services is economic evaluation [2]. Economicevaluation is typically used in a threshold approach to re-source allocation, meaning that interventions costing lessthan a threshold cost per unit of benefit are deemedworthy of funding. However, there are well-known chal-lenges to the acceptability of economic evaluation as a toolto guide resource allocation decisions. A key such chal-lenge is to “ensure alignment between the objectives as-sumed in economic analyses and the objectives facingdecision-makers in reality” [3]. Specifically, economicevaluation as a priority setting tool assumes that thedecision-maker’s objective is to maximize health gain [4]but we know that other objectives are also typically pur-sued [4,5]. One solution offered is that “the simple C/E ra-tio could be supplemented by information on other healtheffects for the patient, for example a descriptive accountof expected improvements in quality of life; wider societaleffects of the intervention, for example on the number ofjobs created; and nonmonetary costs for the patientreported in natural units such as waiting time in days” [4].Such a solution can in fact be formalized through the useof Multi-Criteria Decision Analysis (MCDA): “MCDA isaimed at supporting decision makers faced with evaluatingalternatives, taking into account multiple, and often con-flictive (sic), criteria” [6]. The criteria in MCDA are the‘other health effects’, the ‘wider societal effects’ and the‘nonmonetary costs’ referred to above, or put simply, theconsiderations that a decision-maker will typically takeinto account in making a decision on resource allocation.MCDA is typically used in formal priority setting pro-cesses such as Program Budgeting and Marginal Analysis(PBMA). Like economic evaluation, MCDA has methodo-logical challenges, but in many contexts, because it for-mally includes most or all considerations relevant todecision-making, this approach, and the associated prior-ity setting frameworks, fit the decision-maker’s perspectivebetter [7,8]. One key methodological challenge of MCDAis the search for the necessary information. The necessaryinformation is often not readily available for two main rea-sons. First, some of the criteria, while relevant to decision-makers, are typically not common research subjects. Thiswould include criteria such as integration or access. Sec-ond, even when literature is available, the informationmust be contextualized before it can be used. This paperreports on an example of how this key challenge can beaddressed.In the Fall 2011, the Canadian Physiotherapy Associ-ation (CPA) embarked on a project aimed at providing avaluation of physiotherapy services that is both evidence-based and relevant to resource allocation decisions inhealth care organizations. This project originated morethan one year earlier, in 2010, when the CPA Branch Presi-dents concluded that there was a need for more informa-tion on the value of physiotherapy services and asked theCPA national staff to consider developing a document thatwould address this need. After investigating the methodo-logical alternatives, the CPA national staff decided toproceed with the MCDA framework.In this paper we report on the methods used to obtaininformation necessary for MCDA implementation andprovide a brief summary of the information that wasproduced. The objective is to show how a key challengeto the implementation of MCDA can be addressed andgive a sample of the results. The full results which are inthe final report are the starting point for an MCDA im-plementation. Actual examples of full implementation ofMCDA within a priority setting process at the local level,building on the information produced in this project, arenot included.MethodsMCDA involves the assessment of alternative actions onthe basis of a common set of criteria [9,10]. The two keyelements of the MCDA process are the alternatives to beconsidered and the criteria to be used. Possible alterna-tives are those options available to the decision-maker,for example changing the level of funding for a givenphysiotherapy service or program. The criteria representthe relevant considerations in assessing the impact ofimplementing any of the different alternatives. Criteriatherefore depend on the decision-making context. Oncepossible alternatives have been evaluated on the basis ofthe selected criteria, they can be compared and recom-mendations can be formulated. The evaluation of eachalternative provides an assessment of what would belost, in cases of a reduction in funding, and what wouldbe gained, in case on increases in funding. When con-textualized, this valuation represents the marginal valueof a service at the local level (as opposed to the total oraverage value) as the question that was posed with re-spect to each criterion was: what would be the impacton this criterion of an increase or a decrease to thecurrent volume of service. The basic steps in MCDA areoutlined in Table 1.Dionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 2 of 16http://www.resource-allocation.com/content/11/1/11The first step in the application of MCDA to the valu-ation of physiotherapy services was to determine a set ofcriteria relevant to decision making on health care re-source allocation involving such services. The perspec-tive adopted for this project was that of a decisionmaker within a health region or health service deliveryorganization, as this was the primary target audience forthis work. Based on previous priority setting work withCanadian health authorities and on the literature on pri-ority setting [11-14] an initial set of criteria was pro-posed to the CPA and, through discussion, a final list ofeleven criteria was developed (see Table 2). The criteriawere defined in such a way as to ensure that overlap wasminimized [i.e., they are meant to be mutually exclusive,as much as possible].Moving to step two, the CPA identified a set of serviceareas for assessment, based on relevant literature andsimilar briefings on value for money developed in theUnited Kingdom by the Chartered Society of Physiother-apy. The final list contained twelve service areas (seeTable 3).This research project was about the second part ofstep two which is to obtain information necessary to as-sess each alternative on the basis of each criterion. ThisTable 1 MCDA steps1 The first step is the development of relevant criteria. The criteria should be clearly defined and must relate to the overall purpose of the decisionprocess. The objective in the development of criteria is to include all considerations relevant to the decision that has to be made and to providesufficient clarity to ensure consistency in the translation of information about the alternatives into ratings.2 The second step is the identification of the possible alternatives. In this case, the alternatives are the most common physiotherapy services. Eachalternative [or in this case set of services] must be accompanied by the information required to assess it on the basis of the established criteria.3 The third step is the formal evaluation of each possible alternative. This is done by rating each alternative on each criterion and calculating acomposite score. Because the same criteria are used with all alternatives, the scores are comparable across all alternatives.4 The final step is the formulation of recommendations. First, each composite score is validated to ensure that no process errors took place. Oncethat is done, each alternative can be ranked in relation to all others. Funding recommendations are then based on this ranking.Table 2 Criteria and definitionsResource impact Impact on system-wide resource useQuality of Life This criterion deals with the absolute change in quality of life, i.e. a service that has a limited impact onquality of life could not rate to the top of the scale on this criterion.Patient/ provider satisfaction Deals with benefits of the service other than the direct impact on the underlying condition, for example, aservice that is very personalized will rate higher here because, presumably, the provider would be able toallow to a greater extent for the client’s preferences [for example, regarding the nature of the activities, thelocation, the timing, the setting- group or alone].Integration This criterion is about the continuum of care [and goes beyond the health care system]. Does the serviceaddress a gap in the continuum of care that facilitates the clients’ transition from one program or service toanother?Access This criterion measures the impact of the provision of a given service on the current utilization of otherservices, thereby possibly making these other services more accessible. For example, if a given service resultsin fewer hours per week of home care being required, then this service has freed up those hours forsomeone else to use. Some services will free up resources that way and some won’t.Equity Impact of the service on the health status of groups where there is an avoidable, unfair, and remediablehealth status gap.Effectiveness This is about the absolute effectiveness of the service. Just because a service is the best that can be donefor an underlying condition does not mean that it is highly effective. Also effectiveness is measured withrespect to the impact on the underlying condition itself or the impact on the consequences of theunderlying condition.Appropriateness This criterion deals with the high level degree of match between a given service and the overall needs ofthe population, defined as the combination of the number of persons with the underlying condition andthe impact of the underlying condition on quality of life. We should also consider here the availability ofpossible alternatives. Alternatives to be considered here can be privately provided services but also differentservices that are publicly funded. We are getting at the idea of the possibility of substitution with thiscriterion.Acceptability This deals with the relative ‘displeasure’ associated with the service delivery- amount of pain, discomfortImplementation challenges Risks associated with the implementation of given service change [for example, increased volume] but alsodegree of support- this would be measured, amongst other considerations, by the extent of public pressurein favour of a service.Impact on future use of health careservices [3+years]This criterion is about the extent to which the provision of a physiotherapy service now is likely to affect theoverall use of health care services down the road [at least three years from now].Dionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 3 of 16http://www.resource-allocation.com/content/11/1/11was done in a three part process. First, a literature re-view was undertaken to identify peer reviewed papersthat address the notion of value (as defined by the cri-teria). Search terms included the given service area alongwith ‘effectiveness’, ‘cost-effectiveness’, ‘value’ and a myr-iad of other terms relating to the identified criteria. Dueto the breadth of the search, a systematic review was notattempted; rather key papers were identified andreviewed with the intent of providing insight into a givenservice area, as opposed to a comprehensive take oneach area. Not surprisingly, for many of the criteria therewas no, or very limited, research evidence. The secondpart of the search for information or evidence was aseries of interviews with content experts for each of thetwelve service areas. These content experts (n=1 to 3 de-pending on the service area) were identified by the CPA.Through one or more phone consultations, the literaturereview for each service area was critiqued and new infor-mation was generated where no, or insufficient, litera-ture existed. This was an important part of the processas the literature only provided information on some ofthe criteria. The missing, but required, pieces of infor-mation thus came from expert opinion. It is in this com-bining of expert opinion with research finding thatMCDA provides a pragmatic approach to valuation.After drafting of an initial synthesis document by servicearea which combined the results of the literature reviewwith expert opinion, there remained a need to ensure that1) the information presented would be applicable at a na-tional level (as opposed to the provincial or regional level)and 2) the assessments would be consistent across the ser-vice areas (noting that the content experts were only fo-cusing on a single service area). For this purpose, in thethird part of the process to acquire the required informa-tion, the CPA struck a validation committee comprised ofeleven individuals from across Canada with a broad rangeof experience in physiotherapy. Over the course of 2 two-hour meetings, the synthesis document was reviewed indetail. In some cases the validation committee requestedadditional information from the literature and clarificationof points made by the content experts. The synthesisdocument was then adjusted to reflect the comments fromthe validation committee, including additional research in-formation and clarification of expert opinions, resulting inthe final synthesis by service area. Steps three and four ofthe MCDA process were not included in this project asthose steps are context-dependant by nature.Results: Key findings by service areasIn this section, we present the some of the key findingsby service area. This section is limited to select key find-ings because full presentation is beyond the scope of thispaper: we have 11 criteria and 12 service areas whichmeans 132 cells of information which in the final reportrepresented 55 pages of content. Where findings arebased on published evidence, references are provided.When there is no reference, the findings are expert opin-ion, obtained as described above. Further details, includingthe key findings for each criteria, for each of the twelveservice areas are presented in Table 4 (there again, wherefindings are from the literature, references are cited).Complete results can be found in the CPA report ‘ValuingPhysiotherapy Services’ [15]. The findings presented hererepresent the minimum starting point required to imple-ment MCDA at a local level. In the Discussion, we de-scribe how these results can be used in an MCDA exercisein a health service organization.PediatricsThe cost of providing pediatric physiotherapy servicestends to be higher than treatment for adults, however thelong-term benefits and decreased burden on future use ofcare services can be significant. Besides the expected dir-ect impact, for example, the direct impact on childrenwith juvenile idiopathic arthritis [30], with cerebral palsy[31], or with cystic fibrosis [32], there are two importantbenefits of pediatric physiotherapy that emerged: 1) thephysiotherapist typically develops a supportive relationshipwith both the child and his or her family. In this role, thetherapist is an essential source of information and educa-tion making the physiotherapist a valued link to, and guidethrough, an often-times overwhelming care process forchildren and their parents; and 2) pediatric physiotherapyservices play an important role in the transition to adult-hood. Therapists can act as a bridge between programs toensure the continuation of treatment while transitioningfrom child to adult care.Home-based servicesHome-based physiotherapy services are highly effectivefor many health conditions, including frailty in elderlyTable 3 Selected service areas for review1. Physiotherapy interventions for musculoskeletal conditions2. Physiotherapy interventions for low back pain3. Rehabilitation services in the intensive care unit4. Physiotherapy interventions for chronic disease management5. Rehabilitation services for chronic lung disease6. Rehabilitation services for cardiovascular disease7. Rehabilitation services following joint arthroplasty8. Rehabilitation services following stroke9. Physiotherapy services in the emergency department10. Home based rehabilitation services11. Rehabilitation services for falls12. Rehabilitation services for pediatricsDionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 4 of 16http://www.resource-allocation.com/content/11/1/11Table 4 Key findings by criteria service areasResourceimpactQuality of life Patient/providersatisfactionIntegration Access Equity Effectiveness Appropriateness Acceptability ImplementationchallengesImpact onfuture use ofhealth careservices1. Physiotherapyinterventions formusculoskeletalconditionsFor non-urgentMSK patients,physiotherapistsfound to behighly effectivegatekeepers tosurgical care,providingappropriateassessment andmanagement ofpatient needs;reduces costs ofoutpatient care[16]Clearrelationshipbetweenimprovedfunctioning andimpact onquality of lifePatientsatisfaction withphysiotherapytreatmentcorrelated topersonalresponsibility formanagingdisorder;recommendadjustingtreatment tomatch attitude orattempt tochange attitude[17]. As aprovider, veryrewarding areato work; client-centredapproach;increasestherapist’s driveto improve theirskillsPhysiotherapycan fill gaps forsomeone who isbelow thresholdof MSK health;helps to raiseclient tominimumthreshold so theycan then moveinto thecommunity andaccess personaltrainersLimited impacton concurrentuse of otherservices:possibly betteruse ofsurgeons’ timeDisparitybetween patientsnot privatelyinsured andthose insured;similarly with on-site access versusoff-site. Accesstied to SES; fewresources forthose with lowincomeOutpatientmultidisciplinarytreatmentprogram forsick-listedworkers highlyeffective inimprovingphysicalfunctioning,physicaldisabilities, andkinesiophobiacompared tousual care; nosignificantdifference incost-effectiveness onthe societal levelas compared tousual careOrthopedicsurgeons morelikely to referpatients to PTthan primaryphysicians; self-referral patientshad lower PTvisits thanphysician referred[17]. Need toincrease therapyresources toaddress barriersto access [18]Some servicesare quiteuncomfortable(e.g.,shoulders); butgenerally,clients do notstop due todiscomfort;have to puttreatment intobroader pictureof helping theclient whichmay, at times,be painfulPublic does notnecessarily knowwhatphysiotherapy is;people whomight benefitmay not knowhow to accessservices or areunaware of howit would bebeneficial. Needpublic and otherprofessionals tobe more aware ofskills and impactof PTCreatingindividualizedprograms andallowing forindependentcare outside ofphysiotherapycan result inlifelong changes:8 weeks post-physiotherapymay not result insignificantchanges;however, largechanges at 12-month; inaddition, if re-injury occurs,costs are muchlower2. Physiotherapyinterventions forlow back painPhysiotherapist-led painmanagementclasses offer acost-effectivealternative tousual outpatientphysiotherapyand areassociated withless healthcareuse [19]Reduces painand improvesfunctioning,especially forchroniccondition(confirmedthrough theadministrationof pre and postsurveys)Hands onindividual carethat results inpatientsatisfaction;individualizedcare witheducation is keyelement onsatisfactionEarlier positionin thecontinuum ofcare wouldproduce greaterbenefits;ironically, in ruralareas, cantypically get anMRI quicker thenPT servicesMain impact ison freeing upsurgeon’s timeby moving thetriaging activityto thephysiotherapistNo identifiablesub-populationdisproportionallyaffected by LBPalthough morewomen gettreatment thenmenSignificantimpact on risk ofworseningdisability andtime off-work[20]. About 80 to90% of all casesare resolved, i.e.patientsexperience anormal lifestyleexcept for theodd episodicrecurrenceIncidence of LBPis steady butproportion ofcases that evolveto chroniccondition isincreasing; thisprocessaccelerates accessto treatmentthereby reducingthe risk of theconditionbecomingchronic.Patients aremore likely toparticipate inexerciseprograms thatreflect theirpreferences,circumstancesand abilities;recommendcollectingpatientpreferencesbefore startingtreatment [21]Requirements fortriaging program:Cooperation fromsurgeons;Specializedtraining for thephysiotherapistLong-termimpact will beon theproportion ofcases thatbecome chronic(chronic LBPaffects mobilitywhich haspsychologicalimpacts as wellas physicalimpacts throughthe limitation onthe ability toexercise)Dionneetal.CostEffectivenessandResourceAllocation2013,11:11Page5of16http://www.resource-allocation.com/content/11/1/11Table 4 Key findings by criteria service areas (Continued)3. Rehabilitationservices in theintensive careunitWithphysiotherapy,functional abilityat time ofdischarge fromICU is higher,leading toreduced costssuch as multi-system de-conditioningwith long- termbed restImpact of ICUphysiotherapyon QoL ismainly throughprevention ofproblemsresulting froman ICU stay.These problemsare a directdeterminant ofwhere patientsgoes next, e.g.,nursing homeor own homeSignificantprovidersatisfaction inthis field inassisting peopleto move earlieralong withgreater patientconnection;physiotherapy isa constant;promotesrelationshipbuildingICU is extremelymulti-disciplinary;no practitionercan act inisolation andthereforecoordinationoccurs acrossdisciplines, in thiscontext,physiotherapistchart notes havea direct impacton how thepatient is treatedon the wardPT can affectLOS in ICUICU population isheterogeneous;equity not anissueTwo key areas ofimpact: EarlymobilityVentilatorweaningPatients arebecoming farmore complexwith co-morbidities –physiotherapistslook at patientsholistically versuspossiblefragmentation ofspecializedservicesInvolves hardwork but nodifferent thanother PTservicesSpecializedequipmentrequiredAbility to gohome earlierwithphysiotherapyservice ;however, longerterm utilization isless likely to beimpacted4. Physiotherapyinterventions forchronic diseasemanagementService is foundto be sufficientlycost-effective tobe included inthe coverageprovided bysome privately-fundedextended healthcare plansBecause of themobilityconcern, theimpact ofphysiotherapyon QoL isconnectedprimarily toincreased levelof activity andfunctioning.Many diseasespecificresearchfindingsRanges ofimprovementbut chronicdisease bydefinition willnot be ‘curative’;PT best viewedas an integralpart of multi-modal team ofcareWithoutphysiotherapy,patients wouldbe on waitlistsfor physicianservices orsurgery; assistswith filling gapsWhenphysiotherapyconductedalongsidephysicians,physicians’capacityincreasesNo impact Because patients’problems aremulti-faceted andrequire multipleinterventions (e.g.medication,surgery), PT rolein designingexerciseprograms thattake all of thesefactors intoconsideration iscentral to overalleffectivenessGrowing problem,especially with anaging populationImportant tomeasure andtrack progressas an incentiveExpertise isavailable,especially ifphysiotherapistsare used to planand superviseactivities, whileassistants provideinstruction andoverseeindividualexerciseprograms (see:CLCS model incommunitycentres inQuebec)Significantpreventionpotential thatcan have a largeimpact on futureuse of resources5. Rehabilitationservices forchronic lungdiseaseMultidisciplinary,outpatientpulmonary rehab(PR) programsubstantiallyreduced healthresources use inpatients withmoderate, severeand very severeCOPD. The meanincremental costof addingrehabilitation tostandard carewas a savings of$152 per patient[22]PR shown toimprove qualityof life (Rubi;McCarroll); PRdeals withphysicalfunction, butalso with thepsychologicalaspectsthrougheducationPatients whohave receivedPR often want tobe re-admittedafter their nextexacerbationThere is poorcontinuum ofcare for COPDpatients. Currentcare is focusedon respondingto exacerbationsUse of PRresults in lessexacerbations,fewer ER visitsand reducednumber ofunscheduledGP visitsCOPD does notdisproportionallyaffect anyspecific‘disadvantaged’groupThere is strongevidencedemonstrating areduction indyspnea,increasedexercisetolerance,improved healthrelated quality oflife and cost-effectiveness [23]COPD is asignificant chronicdisease in termsof incidence andprevalence: fourthor fifth leadingcause of deathPatientstypically wantto return totreatmentNo specializedresources needed;physiotherapistscan be trainedquickly in thespecifics of thisservice; exerciseequipment used isstandardPatients receivingPR are, in thelong run, morelikely to stay athome longer,thereforepostponinginstitutionalizationDionneetal.CostEffectivenessandResourceAllocation2013,11:11Page6of16http://www.resource-allocation.com/content/11/1/11Table 4 Key findings by criteria service areas (Continued)6. Rehabilitationservices forcardiovasculardiseaseOutpatient CRless expensivethan inpatientyet similareffectivenessCR significantlyimproved QoLscores, reduceddepression andhad positiveeffect onpsychosocialmeasuresService can befully tailored tothe client’ssituationClients comefrom diagnosisof cardiovascularcondition thentransition tolocal, ongoing,communityservices; CRplays anessential role infacilitating thistransitionImpact on theuse of otherhealth servicesis notimmediate(except forlength ofhospital stay)Women, theelderly, ethnicminority groupsaccess CR less.Very littleinformation onwhy subgroupshave lower ratesof accessCR reduces therisk of cardiacand generalmortality ratesby 25-30%There is agrowing referralrate AND agrowing uptakerate because ofincreasedawareness(referral rate) andimprovements inservices (uptakerate)The servicesare mostlyabout teachingso there is nophysical pain.Changes inlifestyle beingpromoted canbe difficult toadoptNone noted Services reducethe likelihood ofrecurrence of theproblems andreduces theseriousness offuture problems7. Rehabilitationservicesfollowing jointarthroplastyWhen comparingthe cost-effectiveness ofan acceleratedperioperative careand rehabilitationprotocol withthat of a morestandard protocolfor patientstreated with totalhip arthroplasty,beginning fromthe first visitbefore theoperation to oneyearpostoperatively, astudy found theacceleratedintervention tobe more effectivewith an averageof $4000reduction intreatment costswith a 0.08 QALYgain; also morecost-effective fortotal kneearthroplasty withno difference inQALYs [24]PT providesboth earlierfunctionality anda better endpointPostoperative,active physicaltherapy increasessatisfaction andhelps to meetpatientexpectations [25]Impact oncontinuum ofcare comes fromacceleratingpatient’sprogressionthrough the careprocessWill reducedoctor visitsMore difficult toaccess PTservices in ruralsettingsUsing teamapproach,patients hadlargeimprovements inoutcomemeasures duringtherehabilitationstay and 6-month follow-up[26]Joint arthroplastyvolume is drivenby demographicsHighacceptabilityNo significant HRor equipmentchallengesNo evidence ofimpact on futureuse of healthcare services (3+years)Dionneetal.CostEffectivenessandResourceAllocation2013,11:11Page7of16http://www.resource-allocation.com/content/11/1/11Table 4 Key findings by criteria service areas (Continued)8. Rehabilitationservicesfollowing strokeVery earlymobilisation(VEM) more cost-effective thanstandard careand improvedoutcomesResearchfindings stilllacking; recentinnovations indiagnosis,management,andrehabilitationhave resulted inmeasurableimprovementsin clinical andfunctionaloutcomes afteracute stroke;however,despiteimprovementsin medicalmanagement,quality of life isnot necessarilyimproving poststroke [27]Programs aremeant to bepatient-centered:this is the goal;limitation is inresourceconstraintswhich reducesability tocustomizetreatment plansKey componentof the continuumof care; If there isnot sufficientphysiotherapyservices LOS islonger and/or thepatient does notdo as wellVery limitedimpact on theconcurrentutilization ofother servicesIncreased odds ofproblems from apast strokeassociated withfailure to accessOT/PT services,lower monthlyincome, and ageComparingspecializedoutpatienttherapy to notreatment, 14RCTs found thattherapy-basedoutpatient rehabwas associatedwith a reductionin the odds ofpoor outcomeand increaseddaily living andpersonal activityscoresStroke is asignificantcondition interms ofincidence;physiotherapy isan integral partof its treatmentStroke causesfear in patients,which increasestreatmentacceptance rate;physiotherapyfocuses onrestoringphysical functionand in so doing,provides positivefeedbackRequires morerehab beds and/orspecialized unitsImprovedphysical functionand has directimpact on socialfunction;minimizes thefuture use ofhealth careservices9. Physiotherapyservices in theemergencydepartmentCan reduce LOSfor somepatients;facilitates flow inthe ERServicesaddress fearand uncertaintyaround riskswhendischargedPotentially betterclient satisfaction:less pain, reducesshort-termdisability,improvesfunction andsafetyImportant‘triaging’ role inthe continuumSizeableimpact on rateof return visitsto emergencyRate ofemergency visitsnot clearlyrelated to beingpart of anydisadvantagedpopulationsAt system andprovider levels,there is limitedresearchevidence on thevalue of anemergencydepartmentphysiotherapyservice; at patientlevel, there ishigh-levelevidence ofbenefits in termsof improved paincontrol andreduced disabilityin the short termThere is anincrease in EDattendances,therefore anincreased needfor emergency PTservicesSometimes‘forces’ therealization thatthe patient is ata time of lifewhere there is aloss ofindependenceand a need formobility aids orassistanceIncreased volumecomes with aneed forobservation bedsand sub-acutebedsPatients areflagged earlier forpresent andpotentialproblems andcan be followed/assisted in thecommunityDionneetal.CostEffectivenessandResourceAllocation2013,11:11Page8of16http://www.resource-allocation.com/content/11/1/11Table 4 Key findings by criteria service areas (Continued)10. Home basedrehabilitationservicesSignificant costaversion;mobilityassessment,keeping peopleindependent intheir homes;prevention offalls andproviding a safeenvironmentwithin the homecontextImpact of PTcan includeincreased socialinteraction;improvedpersonal anddomesticactivities;improvedhealth status;improvedsubjectivequality of life;reducedcaregiverburdenPatientsatisfaction isclear (buttypically is nottracked by formalinstruments); onemeasure ofsatisfaction is thatthe clients payfor subsequentvisits; verbalfeedback fromclients is verypositive; whileanecdotal, thehigh level ofsatisfaction isclearService isextremelyrelevant toserviceintegration; biggap in thecontinuum ofcare fromhospital tohome; a lot ofpeopledischarged fromthe hospital andin need ofhome-basedservice but arenot receiving itor receive it in avery limitedmanner, i.e., noactive rehabpost discharge,rather patientsare given awalker or basiclevel ofinformationReduces LOSandhospitalisationsInequities existbetweenProvinces: thosewithout financialmeans do nothave access tohome-basedrehab services insome Provinces;those withchronicconditions aremore vulnerableand need morefollow-up;currently, there isno support fromthe public systemto help theseindividualsWhen comparingadults 70 years orolder with one ormore functionalproblems whoreceived a home-basedprogramme ofoccupationaltherapy andphysiotherapy toa control group,a significantreduction inmortality ratewas found (5.6%vs 13.2%);individuals with amoderate risk ofmortality in theinterventiongroups alsoshowed asignificantreduction at16.7% vs. 28.3%[28]Home-basedtherapy increasesaccess, inparticular forpatients withgreater medicalcomplexitiesMain issue isthe paymentrequired forservicesHave to have theright provider: noteveryphysiotherapistcan provide thisservice; broadexperience base isrequired to beeffective andproficient;therapist works ontheir own whichmeans there areno secondopinions; someanxiety inproviding in-home servicesand worker safetycan be a concernHome-basedservices areexpensive withrespect to time totravel and lowvolume howeverthis needs to beconsidered inlight of potentialdecrease inutilization offuture serviceneeds; in thelong term, this isa very efficientuse of societalresources11.Rehabilitationservices for fallsTreatment forfalls was 1.8times morecostly thanimplementing afall preventionprogramSpecializedbalanceprogram forwomen withosteoporosissignificantlyimprovedquality of life,physicalfunction,symptoms,socialinteraction andoverallwellbeing [29]Falls preventionprogramming isa new field, todate has notdrawn adequateattentionNot really part ofa continuum ofcare in mostcasesFall preventionservice doesnot reduceclient’s use ofother services;greatestimpact onfuture serviceusePrograms tendto target seniorsand diabeticsExerciseprogramsignificantlyreduces the riskof death, offalling andhospitalisationor transfer to anursing homeNeed to get outin front toprovideprospectiveservices insteadof providingserviceretrospectivelyNo physicalrisks ordiscomfort butpsychological‘discomfort’ asfall preventionassociated witha loss ofindependenceMore awarenesswith health careprofessionalsgenerallySubstantialimpact especiallyin the subset ofcases where fallscan be avoidedDionneetal.CostEffectivenessandResourceAllocation2013,11:11Page9of16http://www.resource-allocation.com/content/11/1/11Table 4 Key findings by criteria service areas (Continued)12.Rehabilitationservices forpediatricsGetting rightprograms inplace early canmake a lifelongdifference inhealth outcomesand lead to verysignificantsavingsMovement isfreedom; forchildren whohave difficultygettinginvolved inactivities, theseservices openopportunity forparticipationPhysiotherapist isthe health careprofessional inclosest contactwith the patientand his/herfamily;relationship thatdevelops ispotentially unlikeany other healthcare profession;very personal innature;physiotherapistsbest understandthe child’sdisability and socan relate verywell; becomesvery strongadvocates for thepatient andfamilyThe servicedefinitelyaddresses a gap;if this service wasnot in place, bythe time thechild reachedadulthood theywould be so farbehind in theirdevelopmentthey could nevercatch upSome surgeryavoidance;somereduction inGP visitsManydisadvantagedgroups do nottypically go to thehospital forservices; if rehabservices are in thecommunity and/or school orcommunitycentre, access tohealth care ismore likely:practitioners willoften seeindividuals whohave notaccessed anyother service inthe systemMany studieshave showneffectiveness;studies aretypically small,but results areconsistent acrossconditionsChildren do notrespond as well inadult facilitiesTypically, verywell receivedBaseline servicesare not achallenge - newgrads can do thisEarly interventionhas significantimpact onreducing futureutilization ofservices, includingprevention ofsecondarysurgeriesDionneetal.CostEffectivenessandResourceAllocation2013,11:11Page10of16http://www.resource-allocation.com/content/11/1/11adults [33], ankle fractures [34], stroke [35], heart failure[36], breast cancer [37], and recovery from hip replace-ment surgery [38]. For such conditions, home based in-terventions have been shown to lower mortality ratesrelated to falls [28,39] and the risk and rate of falls inolder adults [40], reduce the number of nursing homeadmissions and hospitalisations, and decrease hospitallength of stay.Home-based physiotherapy programs are critical toservice integration, providing a much-needed link be-tween hospital and home. Home-based physiotherapyservices can also help with a social issue: social isolationis often an issue for older clients and clients with morecomplex conditions; with physiotherapists providing in-home care, patients receive regular visits and consistentmonitoring and follow-up.Intensive Care Units (ICUs)The most common use of physiotherapy in ICU is toimprove function for patients on mechanical ventilation[41]. Improving function has been shown to reduce de-pendency and promote earlier weaning, which in turndecreases hospital length of stay and increases quality oflife [42-44]. With a reduction in hospital length of stay,along with increased function and fewer patient compli-cations, physiotherapy treatment is highly cost-effective,reducing both the burden on acute care services and fu-ture health care service use [45,46]. Further, becausetreatment prevents critical weakness and increases func-tional ability [45,47,48] patients are less likely to bedischarged to a care facility and are more likely to returnto their home.Cardiovascular rehabilitationCardiac rehabilitation services support patients whentransitioning from hospital to the community by helpingwith linkages to services within the community. Thishelps to ensure that client care continues after discharge.Such linkages also help to promote social engagement,adoption of healthy behaviors and provide support forself-managed care. Along with a resulting reduction inhospitalisation rates [49] and improvements to physicalactivity, smoking cessation rates, systolic blood pressure,weight loss and total cholesterol [50,51], cardiovascularphysiotherapy services also provide a means of enhan-cing the surveillance of higher risk patients while provid-ing personalized, tailored care that leads to improvedpsychosocial function.EmergencyPhysiotherapists in emergency departments can improvepain control [52] and reduce short-term disability [53].Early access to physiotherapy for this purpose can im-pact current and future use of health care services.Physiotherapists also aid in discharge planning by pro-viding community program information and recommen-dations for mobility aids. Such assistance facilitates thecontinuation of care which in turn can alleviate patients’fear of the acute event reoccurring while supporting asafe return to the home and community.Emergency department physiotherapy programs canalso decrease hospital length of stay and wait-times, inparticular for minor musculoskeletal injuries [54]. Fur-ther since emergency departments are often a patient’sfirst point of care, clients who would benefit fromphysiotherapy interventions can be flagged early on inthe care process directly impacting current and futureuse of health care services.StrokeResearch shows that physiotherapy services for strokepatients aid in the prevention of subsequent acute eventswhile supporting a patient’s ability to live independently[55,56]. Physiotherapy services were also found to be akey component in the continuum of care, supporting pa-tients in their transition from hospital to home [35].This is particularly true when treatment is provided earlyand through a specialized stroke unit [56-58], with adose-dependent effect being present [51]. High intensityphysiotherapy programs, task-specific therapies and indi-vidual discharge planning all contribute to improvedoutcomes.Outpatient physiotherapy programs for stroke patientsare also effective. It was found that when outpatient re-habilitation programs were reduced, the length of stay inhospital increased along with rehospitalisation rates andoverall costs [59,60].Musculoskeletal conditions (MSK)With programs focusing on client self-management andindependence, physiotherapy services are highly valuedas an effective tool in the promotion of injury recoveryand prevention of acute events [61]. Furthermore, thereis a clear, positive relationship between increased phys-ical functioning and improved quality of life.While the initial costs of physiotherapists treatingMSK patients are higher because of the requirement forexperienced therapists, patients tend to require fewervisits over time. Care costs can be further reduced byusing physiotherapists in triaging of patients: experi-enced physiotherapists can act as gatekeepers to surgicalcare, providing appropriate assessment and managementof the patient’s condition [62-64].Low back painPhysiotherapy for patients with low back pain is highlyeffective in reducing both acute and chronic pain whilesignificantly limiting the risk of increased disability andDionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 11 of 16http://www.resource-allocation.com/content/11/1/11chronic conditions [65-67]. Research suggests that be-tween 80 to 90 percent of all lower back cases can be re-solved through participation in rehabilitation programs.Rehabilitation programs are also cost-effective [68-71].Prompt access to a dedicated physiotherapist for newcases of low back pain, in particular for high-risk pa-tients, often pays for itself by reducing the burden onother health care services and promoting self-managedcare. Brief, simple and early interventions that includeproviding information, reassurance and encouragementto engage in regular physical activity have resulted ineconomic gains measured one year after patients re-ceived the intervention, with no long-term negative ef-fects [71].Physiotherapists can also assist in the triaging of pa-tients to ensure that only those requiring an MRI and asurgeon consult receive a referral for such. Acting as agatekeeper to surgical care, physiotherapists are able toreduce patient treatment costs and significantly impactsurgical wait-times.Joint arthroplastyOverall, effectiveness studies indicate that patients whounderwent joint arthroplasty and participated in physio-therapy programs experienced improved outcomes [72]with the greatest health gains achieved from early inter-vention such as starting rehabilitation 24-hours post-surgery [73]. Benefits included a reduction in pain andan increase in joint motion range, strength and balance[73,74]; short-term functional milestones were alsoattained within a shorter timeframe [24,75]. Early inter-vention had a positive impact on the length of hospitalstays resulting in programs that are highly cost-effective[24,76,77]. Overall, inclusion of physiotherapy services inthe care continuum had a significant impact on treat-ment costs [78]. Discharging patients direct to homewith supportive therapy was also found to be more cost-effective than remaining in hospital with no differencefound in health outcomes.Chronic diseasesThere is strong support for the use of physiotherapy inthe prevention and treatment of chronic diseases, in-cluding hypertension, emphysema, type II diabetes andobesity [79-84]. Studies have shown that patients whoparticipated in individualized exercise programs hadfewer emergency readmissions and physician visits andgreater quality of life than patients in usual care. Physio-therapy programs also facilitate participation in commu-nity programs that enhance and maintain physicalwellbeing, and this in turn can significantly impactfuture use of health care services. Physiotherapy is an in-tegral part of the inter-professional team in the manage-ment of patients with chronic diseases.FallsPhysiotherapy is a highly effective tool in the preventionof falls and fall-related injuries both in hospital [85], aswell as in the community [86-88]. In the community,the effectiveness of physiotherapy programs is significantwith services improving the strength, motor functionand balance in older adults who had previously experi-enced a fall event [89]. These effects contribute to re-duced mortality rates, rates of hospitalisation andtransfers to a nursing home allowing individuals to liveindependently in their homes. Similarly, the implementa-tion of a falls-prevention program in an orthopaedichospital can result in a significant decrease in fall inci-dence [85], fall-related morbidity and service costs.Quality of life measures indicate that participation in afalls-prevention program improves a patient’s confidenceand reduces the fear of falling that often restricts overallphysical activity [90].Chronic lung diseaseThere is strong evidence to support the effectiveness ofpulmonary rehabilitation services for patients withchronic lung disease, with program participation corre-lated with decreased rates of dyspnea, exacerbations,and emergency room and physician visits [23,91,92].Physiotherapy services were found to be cost-effective[93-98] and in some cases a program’s net cost wasnegative (i.e. the program produced net savings): for pa-tients participating in outpatient pulmonary rehab pro-grams, evidence suggests that patient total healthresource use is lower compared to usual care. Rehabilita-tion programs also decreased medication use, the num-ber of ICU admissions over time, and assisted patientsin managing their condition, enabling them to remain intheir homes longer [99].DiscussionIn the context of choices that must be made because notall activities can be carried on as they were due to finan-cial restrictions, information about the value of anygiven intervention is very useful [100]. A commonframework for generating such information is economicevaluation where the cost per Quality-Adjusted-Life-Year (QALY) gained through a given intervention is esti-mated. The estimated cost per QALY gained howeveronly addresses the impact on the life expectancy and onthe quality of life of the clients or patients. In makingdecisions about allocating limited funding, decision-makers typically consider other objectives in addition tothe direct health impact, with equity and access, for ex-ample, being often cited [101]. Moreover, economicevaluation is focused on specific end-points which aretypically directly related to the condition, or potentialcondition, being addressed, for example, the extent toDionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 12 of 16http://www.resource-allocation.com/content/11/1/11which physiotherapy services would impact a specificmeasurement of the progress of juvenile idiopathic arth-ritis. Because of these limitations, when the CPA decidedto address what they felt was a gap in the available infor-mation on the value of physiotherapy services, it madethe decision to address this gap through the applicationof MCDA. The overarching thinking behind this deci-sion was that, as healthcare organizations face increas-ingly tougher choices, the limitations of QALYs as aresource allocation tool will push organizations towardmore formal resource allocation frameworks that useMCDA in their evaluation of alternatives and physio-therapy services will be more likely to receive fair con-sideration if the health care organizations have access toaccurate information. PBMA would be one of theseframeworks. The choice of MCDA was not primarilyguided by the relative level of difficulty in implementinga QALY approach versus MCDA. There was what wasperceived as a shortcoming in information on the valueof physiotherapy services and it was decided to put effortin the MCDA approach provided a greater potential forimpact.The result was a comprehensive report summarizingthe value of each of twelve areas of physiotherapy ser-vices with respect to each of eleven criteria that werethought to represent all relevant considerations in mak-ing decisions about funding involving those services.Some key findings in terms of benefits of physiotherapyservices are presented in this paper. It must be recog-nized that this paper is not reporting on an implementa-tion of MCDA, or of a prioritization exercise. In fact,what the CPA has done is supply health care organiza-tions in Canada and elsewhere with a base of researchwork necessary for the implementation of MCDA, aspart of a resource allocation framework such as PBMA.The findings can be used as a starting point within anylocal MCDA implementation. It is not the role of theCPA to contextualize the information, assign weights tothe criteria, or even suggest that only the criteria listedhere should be used, or to actually rate the impact ofservice volume changes. These steps are the responsibil-ity of local health care organizations. An organizationthat decides to implement the MCDA framework toguide resource allocation would have to: 1) determinelocally relevant criteria and weight them (these could bedifferent than the criteria used in this study but it is notexpected that there would be significant differences); 2)identify possible service volume change options thatmake sense in their context (which depends on theexisting mix and volume of services provided); 3) assessthe impact of each option on the basis of the selected cri-teria (this is where the information contained in the CPAreport comes into play and provides a necessary startingpoint, i.e. necessary but not sufficient information). Notethat the breakdown of areas of practice may not perfectlyfit a given local context, in which case, the relevant areasfrom the twelve used here can be combined; and 4) rankthe options and make decisions. All these steps are stand-ard practice in most prioritization framework, and are partof the PBMA, for example.Our objective here was not to provide one more caseof PBMA implementation but to address a commoncriticism of PBMA or any other process that includesMCDA: that the required information is either not avail-able or too difficult to obtain making such processesunimplementable and therefore only theoretical con-structs. . In terms of information generation, the lite-rature review posed no unusual challenge. As wasexpected, in the grid of criteria by service area, many ofthe cells were left blank after the review. The recruit-ment of experts was done by the CPA and didn’t seemvery difficult for two reasons: this was a project of theCPA and many members are very supportive of theirorganization and it is a project that many members canrelate to and specifically support. Furthermore, the de-mands are not overly burdensome as each expert wasasked to participate in one or two calls of one to twohour each. What is more challenging is explaining to theexperts what is needed from them which is to provide aresponse to the best of their knowledge and not limittheir answers to what they know is research evidence-we really wanted their expert opinion. While this did notcome naturally to some of the experts involved, allended up contributing as was needed. Putting together avalidation committee was no more challenging thanrecruiting experts for the same reasons. And just as wasthe case with the experts, it is necessary to have a fullexplanation of the process and some basic training inMCDA before the committee can start to work. Themain challenge with validating the local data to the na-tional level was understanding how much of the expertopinions were shaped by unique local circumstances.This was addressed by first identifying where this mightbe the case, going back to the local expert for further in-formation, and then reconvening the validation commit-tee. The key lessons from this experience were: 1) therehas to be experts that buy into what is being done, reluc-tant participation would defeat the process; 2) explan-ation of the process and its goals and basic training isnecessary before the experts can be asked questions.Finally, it must always be remembered that the ultimategoal is to obtain the best existing information, some-times experts feel uncomfortable with expressing theiropinion in response to a question but if it is the onlyavailable information then it becomes the best existinginformation. In our project, we found some initial hesi-tation in some cases but all experts were able to over-come it. The main limitation of this paper and theDionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 13 of 16http://www.resource-allocation.com/content/11/1/11supporting report is the extent of the resources availablefor this project. For many criteria, the principal sourceof evidence was expert opinion and this was provided ona strictly voluntary basis. There was sufficient input intothe process to produce validated results but, without adoubt, more resources would have produced a more re-fined report. However, a benefit of the MCDA approachis the transparent nature of the process which allows on-going updating of the results. As new studies are pub-lished or as more experts can devote time to thisanalysis, findings can be continually updated, by area ofservice or by criterion. And further areas of service canbe added.ConclusionAs the growth in public health care funding slows, moredifficult choices about what to fund and what not to fundmust be made. In this context, relevant and accurate infor-mation about the marginal value of any health care inter-ventions is essential for proper resource management.MCDA can be a very effective means of producing suchvaluations which can then be used in whatever prioritysetting process is implemented. However MCDA requiresevidence on aspects of value where there is typically verylittle research evidence available. In this paper we have de-scribed an approach to addressing this challenge. Theresults presented are valuable for two reasons. First, apragmatic approach to the generation of necessary evi-dence is presented. While this approach may seem ratherobvious, the fact is MCDA and priority-setting processesthat employ MCDA are often denigrated on the basis ofthe implied demands for information and the challengesthat this poses. Second, this paper also provides a glimpseof the findings that were generated which may lead somereaders to refer to the final report as a solid starting pointfor an application of MCDA involving any of the twelveareas of physiotherapy services studied.Competing interestsThis project was funded by the Canadian Physiotherapy Association. Theviews expressed in this paper are those of the authors and not necessarilythose of the Canadian Physiotherapy Association.Authors’ contributionsFD and CM have made substantial contributions to the conception anddesign of the study, to the acquisition of data, and the analysis andinterpretation of data. They have drafted the manuscript. TMD and CM havemade substantial contributions to the conception and design of the study,to the acquisition of data, and to revising the manuscript critically forimportant intellectual content. MB has been involved in revising themanuscript critically for important intellectual content.AcknowledgementsThank you to Vicki Wong and Maggie Green, CPA, for their assistance insearching the literature and identifying the 12 key practice areas.Author details1Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal HealthResearch Institute, School of Population and Public Health, University ofBritish Columbia, 7th Floor, 828 West 10th Avenue, Research Pavilion,Vancouver, BC V5Z 1M9, Canada. 2Canadian Physiotherapy Association,Ottawa, ON, Canada.Received: 28 November 2012 Accepted: 2 May 2013Published: 20 May 2013References1. Tsouraps A, Frew E: Evaluating ‘success’ in programme budgeting andmarginal analysis: a literature review. J Health Serv Res Policy 2011,16(3):177–183.2. Langer A: A framework for assessing Health Economic Evaluation (HEE)quality appraisal instruments. BMC Health Serv Res 2012, 12(1):253.3. Williams I, McIver S, Moore D, Bryan S: The use of economic evaluations inNHS decision-making: a review and empirical investigation. HealthTechnol Assess 2008, 12:7.4. Hauck K, Smith PC, et al: The Economics of Priority Setting for Health Care.A Literature Review. Washington D.C: The World Bank; 2003.5. Youngkong S, Teerawattananon Y, Tantivess S, Baltussen R: Multi-criteriadecision analysis for setting priorities on HIV/AIDS interventions inThailand. Health Res Pol Syst 2012, 10:6.http://www.health-policy-systems.com/content/10/1/6.6. Thokala P: Mutiple criteria decision analysis for health technology assessmentreport by the decision support unit. Regent Court, 30 Regent Street, Sheffield,S1 4DA: School of Health and Related Research, University of Sheffield, UKDecision Support Unit, ScHARR, University of Sheffield; 2011.7. Baltussen R, Niessen L: Priority setting of health interventions: the needfor multi-criteria decision analysis. Cost Effectiveness Resource Allocation2006, 4:14. http://www.resource-allocation.com/content/4/1/14.8. Peacock S, Mitton C, Bate A, McCoy B, Donaldson C: Overcoming barriersto priority setting using interdisciplinary methods. Health Policy 2009,92:124–32.9. Baltusen R, Niessen L: Priority setting of health interventions: the need formulti-criteria decision analysis. Cost Effectiveness Resource Allocation 2006, 4:14.10. Devin N, Sussex J: Incorporating multiple criteria in HTA: methods andprocesses. London: Office of Health Economics; 2011.11. Dionne F, Mitton C, Smith N, Donaldson C: Evaluation of the impact ofprogram budgeting and marginal analysis in Vancouver Island HealthAuthority. J Health Serv Res Policy 2009, 14:234–242.12. Mitton C, Dionne F, Damji R, Campbell D, Bryan S: Difficult decisions intimes of constraint: Criteria based Resource Allocation in the VancouverCoastal Health Authority. BMC Health Serv Res 2011, 11:169. http://www.biomedcentral.com/1472-6963/11/169.13. Wilson E, Rees J, Fordham R: Developing a prioritisation framework in anEnglish Primary Care trust. Cost effectiveness Resource Allocation 2006, 4:3.14. Astley J, Wake-Dyster W: Evidence-based priority setting. Aust Health Rev2001, 24(2):32–39.15. Mitton C, Dionne F: Valuation of Physiotherapy Services in Canada. FinalReport. Canadian Physiotherapy Association; 2012. in press.16. Larsson L, Karlqvist L, Gard G: Effects of work ability and health promotinginterventions for women with musculoskeletal symptoms: A 9-monthprospective study. BMC Musculoskelet Disord 2008, 9:105. http://www.biomedcentral.com/1471-2474/9/105.17. Freburger JK, Holmes GM, Carey TS: Physician referrals to physical therapyfor the treatment of musculoskeletal conditions. Arch Phys Med Rehabil2003, 84(12):1839–49.18. Feldman DE, Bernatsky S, Lévesque JF, Van MT, Houde M, April KT: Accessand perceived need for physical and occupational therapy in chronicarthritis. Disabil Rehabil 2010, 32(22):1827–32.19. Critchley DJ, Ratcliffe J, Noonan S, Jones R, Hurley M: Effectiveness andcost-effectiveness of three types of physiotherapy used to reducechronic low back pain disability: a pragmatic randomized trial witheconomic evaluation. Spine (Phila Pa 1976) 2007, 32(14):1474–81.20. Torstensenm TA, Ljunggren AE, Meen HD, Odland E, Mowinckel P, afGeijerstam S: Efficiency and costs of medical exercise therapy,conventional physiotherapy, and self-exercise in patients with chroniclow back pain: a pragmatic, randomized, single-blinded, controlled trialwith 1-year follow-up. Spine (Phila Pa 1976) 1998, 23(23):2616–24.21. Slade SC, Molloy E, Keating JL: People with non-specific chronic low backpain who have participated in exercise programs have preferencesabout exercise: a qualitative study. Aust J Physiother 2009, 55(2):115–121.Dionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 14 of 16http://www.resource-allocation.com/content/11/1/1122. Griffiths TL, Phillips CJ, Davies S, Burr ML, Campbell IA: Cost effectiveness ofan outpatient multidisciplinary pulmonary rehabilitation programme.Thorax 2001, 56(10):779–84.23. Evans RA, Goldstein RS: Pulmonary rehabilitation: An overview includingnew and innovative strategies. Pneumologica 2011, 50(1):47–61.24. Larsen K, Hvass KE, Hansen TB, Thomsen PB, Soballe K: Effectiveness ofaccerlated perioperative care and rehabilitation intervention comparedto current intervention after hip and knee arthroplasty. A before-aftertrial of 247 patients with a 3-month follow-up. BMC Musculoskelet Disord2008, 9:59.25. Dorr LD, Chao L: The emotional state of the patient after total hip andknee arthroplasty. Clin Orthop Relat Res 2007, 463:7–12.26. Grotle M, Garratt AM, Klokkerud M, Løchting I, Uhlig T, Hagen KB: What's inteam rehabilitation care after arthroplasty for osteoarthritis? Results froma multicenter, longitudinal study assessing structure, process, andoutcome. Phys Ther 2010, 90(1):121–31.27. Edwards JD, Koehoorn M, Boyd LA, Levy AR: Is health-related quality of lifeimproving after stroke? A comparison of health utilities indices amongCanadians with stroke between 1996 and 2005. Stroke 2010,41(5):996–1000. Epub 2010 Apr 1.28. Fontan E: Long-term effectiveness of a home intervention to reducefunctional problems: Results of a randomised, controlled trial. Cahiers del'Année Gérontologique 2010, 2(3):188–190.29. Madureira MM, Bonfá E, Takayama L, Pereira RM: A 12-month randomizedcontrolled trial of balance training in elderly women with osteoporosis:improvement of quality of life. Maturitas 2010, 66(2):206–11. Epub 2010 Apr 14.30. Long AR, Rouster-Stevens KA: The role of exercise therapy in themanagement of juvenile idiopathic arthritis. Curr Opin Rheumatol 2010,22(2):213–7.31. Martin L, Baker R, Harvey A: A systematic review of commonphysiotherapy interventions in school-aged children with cerebral palsy.Phys Occup Ther Pediatr 2010, 30(4):294–312.32. van Doorn N: Exercise programs for children with cystic fibrosis:a systematic review of randomized controlled trials. Disabil Rehabil 2010,32(1):41–9.33. Ryburn B, Foreman P: Enabling independence: Restorative approaches tohome care provision for frail older adults. Health Soc Care Community2009, 17(3):225–234.34. Lloyd JM, Martin R, Rajagopolan S, Zieneh N, Hartley R: An innovative andcost-effective way of managing ankle fractures prior to surgery–hometherapy. Ann R Coll Surg Engl 2010, 92(7):615–618.35. Langhorne P, Widen-Holmqvist L, Taylor G, Murray G, Askim T, Dennis M,Anderson C, Bautz-Holter E, Dey P, Indredavik B, Mayo N, Power M, RodgersH, Ronning OM, Rudd A, Suwanwela N, Wolfe C: Early supported dischargeafter stroke. J Rehabil Med 2007, 39(2):103–108.36. Clark AM, Haykowsky M, Kryworuchko J, MacClure T, Scott J, DesMeules M,Luo W, Liang Y, McAlister FA: A meta-analysis of randomized control trialsof home-based secondary prevention programs for coronary arterydisease. Int J Cardiol 2006, 111(3):343–51. Epub 2005 Nov 28.37. Gordon LG, Scuffham P, Battistutta D, Graves N, Tweeddale M, Newman B:A cost-effectiveness analysis of two rehabilitation support services forwomen with breast cancer. Breast Cancer Res Treat 2005, 94(2):123–33.38. Sigurdsson E, Siggeirsdottir K, Jonsson H Jr, Gudnason V, Matthiasson T,Jonsson BY: Early discharge and home intervention reduces unit costsafter total hip replacement: Results of a cost analysis in a randomizedstudy. Int J Health Care Finance Econ 2008, 8(3):181–192.39. Gitlin LN, Hauck WW, Winter L, Dennis MP, Schulz R: Effect of an In-HomeOccupational and Physical Therapy Intervention on Reducing Mortalityin Functionally Vulnerable Older People: Preliminary Findings. J AmGeriatr Soc 2006, 54:950–955.40. Gillespie LD, Robertson MC, Gillespie WJ, Lamb SE, Gates S, Cumming RG,Rowe BH: Interventions for preventing falls in older people living in thecommunity. Cochrane Database Syst Rev 2009, 2:Art.No.: CD007146. http://www.ncbi.nlm.nih.gov/pubmed/19370674.41. Malkoc M, Karadibak D, Yildirim Y: The effect of physiotherapy onventilatory dependency and the length of stay in an intensive care unit.AMED 2009, 32(1):85–8.42. Needham DM, Korupolu R, Zanni JM, Pradhan P, Colantuoni E, Palmer JB,Brower RG, Fan E: Early physical medicine and rehabilitation for patientswith acute respiratory failure: a quality improvement project. Arch PhysMed Rehabil 2010, 91(4):536–42.43. Morris PE, Goad A, Thompson C, Taylor K, Harry B, Passmore L, Ross A,Anderson L, Baker S, Sanchez M, Penley L, Howard A, Dixon L, Leach S,Small R, Hite RD, Haponik E: Early intensive care unit mobility therapy inthe treatment of acute respiratory failure. Crit Care Med 2008,36(8):2238–43.44. O'Connor ED, Walsham J: Should we mobilise critically ill patients?A review. Crit Care Resusc 2009, 11(4):290–300.45. Wild D: Pushing mobility can reduce costs, deaths in icu patientsmortality falls by 25% after protocol introduced, $3 million in annualsavings projected. Clin Anesthesiol Issue 2011, 37:8.46. Thomas AJ: Physiotherapy led early rehabilitation of the patient withcritical illness. AMED 2011, 16(1):46–57.47. Burtin C, Clerckx B, Robbeets C, Ferdinande P, Langer D, Troosters T,Hermans G, Decramer M, Gosselink R: Early exercise in critically ill patientsenhances short-term functional recovery. Crit Care Med 2009,37(9):2499–505.48. Hanekom S, Louw Q, Coetzee A: Physiotherapy management of criticallyill patients guided by an evidence based protocol is safe and effective:A preliminary study. In Conference: 23rd Annual Congress of the EuropeanSociety of Intensive Care Medicine, Volume 36. ESICM Barcelona Spain:Conference Publication; 2010:S324.49. Heran BS, Chen JM, Ebrahim S, Moxham T, Oldridge N, Rees K, ThompsonDR, Taylor RS: Exercise-based cardiac rehabilitation for coronary heartdisease. Ann Phys Rehabil Med 2011, 54(3):132–43. Epub 2011 Mar 1.50. Duarte FP, Haida A, Bousquet M, Richard L, Mauriège P, Guiraud T: Short-term impact of a 4-week intensive cardiac rehabilitation program onquality of life and anxiety-depression. Eur J Cardiovasc Prev Rehabil 2010,17(3):261–70.51. Jolly K, Taylor RS, Lip GY, Stevens A: Home-based cardiac rehabilitationcompared with centre-based rehabilitation and usual care: a systematicreview and meta-analysis. Int J Cardiol 2006, 111(3):343–51. Epub 2005 Nov 28.52. Lau PM, Chow DH, Pope MH: Early physiotherapy intervention in anAccident and Emergency Department reduces pain and improvessatisfaction for patients with acute low back pain: a randomised trial.Aust J Physiother 2008, 54(4):243–9.53. Bleakley CM, O'Connor SR, Tully MA, Rocke LG, Macauley DC, Bradbury I,Keegan S, McDonough SM: Effect of accelerated rehabilitation onfunction after ankle sprain: randomised controlled trial. BMJ 2010, 340:c1964. http://www.bmj.com/content/340/bmj.c1964.54. Taylor NF, Norman E, Roddy L, Tang C, Pagram A, Hearn K: Primary contactphysiotherapy in emergency departments can reduce length of stay forpatients with peripheral musculoskeletal injuries compared withsecondary contact physiotherapy: a prospective non-randomisedcontrolled trial. Physiotherapy 2011, 97(2):107–14. Epub 2010 Dec 16.55. Mahler MP, Zuger K, Kaspar K, Haefeli A, Jenni W, Leniger T, Beer JH: A costanalysis of the first year after stroke – early triage and inpatientrehabilitation may reduce long term costs. Swiss Med Wkly 2008,138(31–32):459–465.56. Saka O, Serra V, Samyshkin Y, Mcguire A, Wolfe C: Cost-effectiveness of strokeunit care followed by early supported discharge. Stroke 2009, 40:24–29.57. Teasell RW, Foley NC, Salter KL, Jutai JW: A blueprint for transformingstroke rehabilitation care in Canada: the case for change. Arch Phys MedRehabil 2008, 89(3):575–578.58. Lindsay P, Bayley M, McDonald A, Graham ID, Warner G, Phillips S: Toward amore effective approach to stroke: canadian best practicerecommendations for stroke care. CMAJ 2008, 178(11):1418–25.59. Salter K, Jutai J, Hartley M, Foley N, Bhogal S, Bayona N, Teasell R: Impact ofearly vs delayed admission to rehabilitation on functional outcomes inpersons with stroke. J Rehabil Med 2006, 38(2):113–7.60. Schouten LMT, Hulscher MEJL, Akkermans R, van Everdingen JJE, Grol RPTM,Huijsman R: Factors that influence the stroke care team’s effectiveness inreducing the length of hospital stay. Stroke 2008, 39:2515–2521.61. Håland Haldorsen EM, Grasdal AL, Sture Skouen J, Erling Risa A, Kronholm K,Ursin H: Is there a right treatment for a particular patient group?Comparison of ordinary treatment, light multidisciplinary treatment, andextensive multidisciplinary treatment for long-term sick-listed employeeswith musculoskeletal pain. Pain January 2002, 95(1–2):49–63.62. Belthur MV, Clegg J, Strange A: A physiotherapy specialist clinic inpaediatric orthopaedics: is it effective? Postgrad Med J 2003, 79:699–702.63. Taylor NF, Norman E, Roddy L, Tang C, Pagram A, Hearn K: Primary contactphysiotherapy in emergency departments can reduce length of stay forDionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 15 of 16http://www.resource-allocation.com/content/11/1/11patients with peripheral musculoskeletal injuries compared withsecondary contact physiotherapy: a prospective non-randomisedcontrolled trial. Physiotherapy 2011, 97(2):107–114.64. Oldmeadow LB, Bedi HS, Burch HT, Smith JS, Leahy ES, Goldwasser M:Experienced physiotherapists as gatekeepers to hospital orthopaedicoutpatient care. Med J Aust 2007, 186(12):625–8.65. Skouen JS, Grasdal AL, Haldorsen EMH, Holger U: Relative cost-effectiveness of extensive and light multidisciplinary treatmentprograms versus treatment as usual for patients with chronic low backpain on long-term sick leave: randomized controlled study.Spine (Phila Pa 1976) 2002, 27(9):901–909. discussion 909–10.66. Patrick LE, Altmaier EM, Found EM: Long-term outcomes inmultidisciplinary treatment of chronic low back pain: results of a 13-yearfollow-up. Spine (Phila Pa 1976) 2004, 29(8):850–855.67. Storrø S, Moen J, Svebak S: Effects on sick-leave of a multidisciplinaryrehabilitation programme for chronic low back, neck and shoulder pain:comparison with usual treatment. Trondheim, Norway: Clinic of PhysicalMedicine , The Norwegian University of Science and Technology.68. Pinnington MA, Miller J, Stanley I: An evaluation of prompt access tophysiotherapy in the management of low back pain in primary care.Fam Pract 2004, 21(4):372–380.69. Gatchel RJ, Polatin PB, Noe C, Gardea M, Pulliam C, Thompson J:Treatment- and cost-effectiveness of early intervention for acute low-back pain patients: a one-year prospective study. J Occup Rehabil 2003,13(1):1–9.70. Jensen IB, Bergstrom G, Ljungquist T, Bodin L: A 3-year follow-up of amultidisciplinary rehabilitation programme for back and neck pain. Pain2005, 115(3):273–283.71. Hagen M, Grasdal A, Eriksen HR: Does early intervention with a lightmobilization program reduce long-term sick leave for low back pain:a 3-year follow-up study. Spine (Phila Pa 1976) 2003, 28(20):2309–15.discussion 2316.72. Petterson SC, Mizner RL, Stevens JE, Raisis L, Bodenstab A, Newcomb W,Snyder-Mackler L: Improved function from progressive strengtheninginterventions after total knee arthroplasty: a randomized clinical trialwith an imbedded prospective cohort. Arthritis Care Res 2009,61(2):174–183.73. Sanchez LN, Castro-Sanchez AM, Mataran-Penarrocha GA, Arroyo-Morales M,del Mar S-JM, Moreno-Lorenzo C: Benefits of starting rehabilitation within24 hours of primary total knee arthroplasty: randomized clinical trial.Clin Rehabil 2011, 25(6):557–566.74. Munin MC, Rudy TE, Glynn NW, Crossett LS, Rubash HE: Early inpatientrehabilitation after elective hip and knee arthroplasty. JAMA 1998,279(11):84757.75. Lilikakis AK, Gillespie B, Villar RN: The benefit of modified rehabilitationand minimally invasive techniques in total hip replacement. Ann R CollSurg Engl 2008, 90(5):406–411.76. Raphael M, Jaeger M, van Vlymen J: Easily adoptable total jointarthroplasty program allows discharge home in two days. Can JAnesthesia 2011, 58(10):902–910.77. Husted H, Hansen HC, Holm G, Bach-Dal C, Rud K, Lande Andersen K, KehletH: What determines length of stay after total hip and knee arthroplasty?A nationwide study in Denmark. Arch Orthop Trauma Surg 2010,130(2):263–268.78. Iyengar KP, Nadkarni JB, Ivanovic N, Mahale A: Targeted early rehabilitationat home after total hip and knee joint replacement: Does it work?Disabil Rehabil 2007, 29(6):495–502.79. Adami PE, Negro A, Lala N, Martelletti P: The role of physical activity in theprevention and treatment of chronic diseases. Clin Ter 2010,161(6):537–41.80. Park YH, Song M, Cho BL, Lim JY, Song W, Kim SH: The effects of anintegrated health education and exercise program in community-dwelling older adults with hypertension: a randomized controlled trial.Patient Educ Couns 2011, 82(1):133–7.81. Tumiati R, Mazzoni G, Crisafulli E, Serri B, Beneventi C, Lorenzi CM, Grazzi G,Prato F, Conconi F, Fabbri LM, Clini EM: Home-centred physical fitnessprogramme in morbidly obese individuals: a randomized controlled trial.Clin Rehabil 2008, 22(10–11):940–50.82. Ries AL, Make BJ, Reilly JJ: Pulmonary rehabilitation in emphysema.Proc Am Thorac Soc 2008, 5(4):524–9.83. Ozdirenc M, Kocak G, Guntekin R: The acute effects of in-patientphysiotherapy program on functional capacity in type II diabetesmellitus. Diabetes Res Clin Pract 2004, 64(3):167–72.84. Bersoux S, Asbury KL, Cook CB, Verheijde JL, Larson MH, Aponte-Furlow RT,Flatten SS, Hooley SA, LaRosa CS, Seifert KM, Verona PM, Castro JC, JamesonKA: An outpatient-based clinical program for type 2 diabetes prevention.Endocr Pract 2010, 16(1):21–29.85. Cameron ID, Murray GR, Gillespie LD, Robertson MC, Hill KD, Cumming RG,Kerse N: Interventions for preventing falls in older people in nursing carefacilities and hospitals. Cochrane Database Syst Rev 2010(Issue 1):Art.No.:CD005465. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD005465.pub3/full.86. Hektoen LF, Aas E, Luras H: Cost-effectiveness in fall prevention for olderwomen. Scand J Public Health 2009, 37(6):584–589.87. Thomas S, MacIntosh S, Halbert J: Does the ‘Otago exercise programme’reduce mortality and falls in older adults? a systematic review and meta-analysis. Age Ageing 2010, 39(6):687.88. Sherrington C, Whitney JC, Lord SR, Herbert RD, Cumming RG, Close JCT:Effective exercise for the prevention of falls: a systematic review andmeta-analysis. J Am Geriatr Soc 2008, 56(12):2234–2243.89. Hauer K, Rost B, Rutschle K, Opitz H, Specht N, Bartsch P, Oster P, Schlierf G:Exercise training for rehabilitation and secondary prevention of falls ingeriatric patients with a history if injurious falls. J Am Geriatr Soc 2001,49(1):10–20.90. Bialoszewski D, Sllupik A, Lewcsuk E, Gotlib J, Mosiolek A, Mierzwinska A:Incidence of falls and their effort on mobility of individuals over 65years of age relative to their place of residence. Ortop Traumatol Rehabil2008, 10(5):441–8.91. Garvey C, Fromer L, Saver DF, Yawn BP: Pulmonary rehabilitation: anunderutilized resource in primary COPD care. Phys Sportsmed 2010,38(4):54–60.92. Waterhouse JC, Walters SJ, Oluboyede Y, Lawson RA: A randomised 2 × 2trial of community versus hospital pulmonary rehabilitation for chronicobstructive pulmonary disease followed by telephone or conventionalfollow-up. Health Technol Assess 2010, 14(6):1–164.93. Rubí M, Renom F, Ramis F, Medinas M, Centeno MJ, Górriz M, Crespí E,Martín B, Soriano JB: Effectiveness of pulmonary rehabilitation inreducing health resources use in chronic obstructive pulmonary disease.Arch Phys Med Rehabil 2010, 91(3):364–8.94. Rasekaba TM, Williams E, Hsu-Hage B: Can a chronic disease managementpulmonary rehabilitation program for COPD reduce acute rural hospitalutilization? Chron Respir Dis 2009, 6(3):157–63.95. Johnston K, Grimmer-Somers K: Pulmonary rehabilitation: overwhelmingevidence but lost in translation? Physiother Can 2010, 62(4):368–73. Epub2010 Oct 18.96. Griffiths TL, Phillips CJ, Davies S, Burr ML, Campbell IA: Cost effectiveness ofan outpatient multidisciplinary pulmonary rehabilitation programme.AMED 2001, 56(10):779–84.97. Hoogendoorn M, Van Wetering CR, Schols AM, Rutten-van Molken MPMH:Is INTERdisciplinary COMmunity-based COPD management (INTERCOM)cost-effective? Eur Respir J 2010, 35(1):79–87.98. Golmohammadi K, Jacobs P, Sin DD: Economic evaluation of acommunity-based pulmonary rehabilitation program for chronicobstructive pulmonary disease. Lung 2004, 182(3):187–196.99. Ninot G, Moullec G, Picot MC, Jaussent A, Hayot M, Desplan M, Brun JF,Mercier J, Prefaut C: Cost-saving effect of supervised exercise associatedto COPD self-management education program. Respir Med 2011,105(3):377–85. Epub 2010 Oct 29.100. Brouselle A, Lessard C: Economic evaluation to inform health caredecision-making: Promise, pitfalls and a proposal for an alternative path.Soc Sci Med 2011, 72:832–839.101. Maiwenn JA, Feenstra T, Brouwer WBF: Decisionmakers’ views onhealthcare objectives and budget constraints: results from a pilot study.Health Policy 2004, 70(1):33–34.doi:10.1186/1478-7547-11-11Cite this article as: Dionne et al.: The challenge of obtaining informationnecessary for multi-criteria decision analysis implementation: the caseof physiotherapy services in Canada. Cost Effectiveness and ResourceAllocation 2013 11:11.Dionne et al. Cost Effectiveness and Resource Allocation 2013, 11:11 Page 16 of 16http://www.resource-allocation.com/content/11/1/11


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items