UBC Faculty Research and Publications

Trade-offs, fairness, and funding for cancer drugs: key findings from a deliberative public engagement… Bentley, Colene; Costa, Sarah; Burgess, Michael M; Regier, Dean; McTaggart-Cowan, Helen; Peacock, Stuart J May 8, 2018

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12913_2018_Article_3117.pdf [ 1.23MB ]
JSON: 52383-1.0366224.json
JSON-LD: 52383-1.0366224-ld.json
RDF/XML (Pretty): 52383-1.0366224-rdf.xml
RDF/JSON: 52383-1.0366224-rdf.json
Turtle: 52383-1.0366224-turtle.txt
N-Triples: 52383-1.0366224-rdf-ntriples.txt
Original Record: 52383-1.0366224-source.json
Full Text

Full Text

RESEARCH ARTICLE Open AccessTrade-offs, fairness, and funding for cancerdrugs: key findings from a deliberativepublic engagement event in BritishColumbia, CanadaColene Bentley1* , Sarah Costa1, Michael M. Burgess2, Dean Regier1,3, Helen McTaggart-Cowan1,4and Stuart J. Peacock1,4AbstractBackground: Spending on cancer drugs has risen dramatically in recent years compared to other areas of healthcare, due in part to higher prices associated with newly approved drugs and increased demand for these drugs.Addressing this situation requires making difficult trade-offs between cost, harms, and ability to benefit when usingpublic resources, making it important for policy makers to have input from many people affected by the issue,including citizens.Methods: In September 2014, a deliberative public engagement event was conducted in Vancouver, BritishColumbia (BC), on the topic of priority setting and costly cancer drugs. The aim of the study was to gain citizens’input on the topic and have them generate recommendations that could inform cancer drug funding decisions inBC. A market research company was engaged to recruit members of the BC general public to deliberate over twoweekends (four days) on how best to allocate resources for expensive cancer treatments. Participants were stratifiedbased on the 2006 census data for BC. Participants were asked to discuss disinvestment, intravenous versus oralchemotherapy delivery, and decision governance. All sessions were audio recorded and transcribed. Transcriptswere analyzed using NVivo 11 software.Results: Twenty-four individuals participated in the event and generated 30 recommendations. Participantsaccepted the principle of resource scarcity and the need of governments to make difficult trade-offs whenallocating health-care resources. They supported the view that cost-benefit thresholds must be set for high-costdrugs. They also expected reasonable health benefits in return for large expenditures, and supported the view thatsome drugs do not merit funding. Participants also wanted drug funding decisions to be made in a non-partisanand transparent way.Conclusion: The recommendations from the Vancouver deliberation can provide guidance to policy makers in BCand may be useful in challenging pricing by pharmaceutical companies.Keywords: Public engagement, Cancer drugs, Priority setting, Canada* Correspondence: cbentley@bccrc.ca1Canadian Centre for Applied Research in Cancer Control, BC Cancer, 675West 10th Avenue, Vancouver, BC V5Z 1L3, CanadaFull list of author information is available at the end of the article© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Bentley et al. BMC Health Services Research  (2018) 18:339 https://doi.org/10.1186/s12913-018-3117-7BackgroundExpenditure on oncology drugs has risen faster in recentyears compared to other areas of health care [1–3]. Thisis due, in part, to the higher price tags associated withthe discovery of new therapies and increased rates of use[1, 4]. While health gains have been made in the treat-ment of cancer with new therapies [5–7], there is con-cern among policy makers and health-care providersthat the high costs for new cancer treatments might notbe justified by the often small increase in health benefitthey provide over less expensive drugs [7–10]. In Canadaand elsewhere, policy makers are looking for effectiveways to allocate drug budgets in the face of climbingprices and increased utilization for oncology drugs, whilebalancing the need to provide timely and equitable ac-cess to innovative treatments [1, 2]. This is focusingattention on the economic and ethical challenges ofmaking coverage decisions that are sustainable andregarded as fair, [11, 12] especially within publiclyfunded health-care systems like Canada’s [13]. Policymakers must weigh various opportunity costs and deter-mine how best to address dilemmas over sustainability,access, and fairness raised by scarcity and social need.Increasingly, health policy makers are turning to thepublic for guidance in developing solutions to policydilemmas [14, 15]. Incorporating public preferences intohealth-care decision-making processes can be conductedusing priority-setting frameworks such as multi-criteriadecision analysis and program budgeting and marginalanalysis, which use public opinion in the form of stake-holder input and locally relevant decision-making cri-teria to support the decision-making process [16, 17].One-way consultative methods such as focus groups orsurveys have traditionally been used to elicit public pref-erences in these frameworks [18]. However, deliberativeforms of public engagement are emerging as a viablepolicy-informing approach to civic participation in health-care decision making. Specifically, deliberative public en-gagement involves members of the public in a process oflearning and exchanging views explicitly directed towardscollective problem-solving, thus making it distinct fromother discussion-based consultation forums, like focusgroups [14, 19, 20]. As a method, it is highly suited to ad-dress topics in health care where some of society’s mostethically, fiscally, and politically challenging decisions aremade [14]. In Canada, deliberative engagements have beenset up periodically to advise provincial Ministries of Healthon decisions related to technology assessment [21–24],health services [25], and policies for biobanks [26].In cancer control, relatively little is known about whatCanadians value—or what values they prioritize—con-cerning different cancer interventions and their out-comes (Gibson et al, 2013: unpublished manuscript).Canadian studies focused on priority setting in cancercontrol have typically done so from a clinical or policymaker perspective [27–30]. To address this gap and tosupport decision makers’ desire for public guidance ononcology treatments [31], we conducted a public deliber-ation event in September 2014 in Vancouver, BritishColumbia (BC), on the topic of priority setting and thehigh cost of cancer drugs. The goal of the Vancouver de-liberation was to provide recommendations related tofunding decisions for cancer drugs based on the informedand civic-minded deliberations of a socially diverse groupof British Columbians. In BC, most prescription drugslisted on the province’s public drug formulary are reim-bursed through its PharmaCare program and BC Cancerspecifically covers the approved cancer drugs available inBC. To understand participants’ preferences related todrug funding coverage in BC, a well-established approachto deliberative public engagement [26] was adapted andimplemented for the project. Titled “Making Decisionsabout Funding for Cancer Drugs: a Deliberative PublicEngagement,” the deliberation brought together 24 mem-bers of the general public over two weekends (four days)to discuss how best to allocate resources for costly cancertreatments. Participants discussed disinvestment, intra-venous versus oral chemotherapy delivery, and decisiongovernance for funding cancer drugs. Thirty recommen-dations on disinvestment, cost-benefit trade-offs, andtrustworthy governance were generated from the event. Asummary of the event and the 30 recommendations isavailable elsewhere [32]. This paper details the methods ofrecruitment and deliberation used at the event, and pro-vides in-depth analysis of key recommendations made byparticipants by contextualizing their recommendationswithin the dynamics of dialogic exchange amongst allparticipants during the event. Results are presented fromour analysis of the event transcripts, which containparticipants’ inter-subjective exchanges over four days.This analysis provides a rigourous understanding of thereasons why participants may have supported—orrejected—a particular recommendation, based on our as-sessment of their articulations about what was important tothem when it comes to funding expensive cancer therapies.The Vancouver study was approved by the University ofBritish Columbia - British Columbia Cancer AgencyResearch Ethics Board. In 2016, the research team receivedadditional funding from the Canadian Partnership AgainstCancer (CPAC) to conduct an additional six public deliber-ation events, based on the BC model, in communitiesacross Canada to engage Canadians about their prioritiesfor making cancer drug funding fair and sustainable.MethodsRecruitmentThe goal of recruitment was to create a “mini public” bybringing together a diversity of perspectives within theBentley et al. BMC Health Services Research  (2018) 18:339 Page 2 of 13BC general public. A mini public is a representation ofthe larger public from which it is drawn, and thus repre-sents “the diversity of social characteristics and pluralityof initial points of view in the larger society” [33]. As aforum for deliberation, a mini public should be free fromvested or sectarian interests that can steer discussion ina particular (political) direction and thus underminecivic trust in deliberative processes [34].A market research company was engaged to overseerecruitment efforts. An online letter of invitation to par-ticipate in the deliberation was sent to the company’spanel members. The letter specified the dates and thehonorarium for the deliberation ($125/day), invited re-spondents to complete a brief demographic question-naire, and informed them they may be selected tocomplete a preference-based survey (see below). All re-spondents were stratified by age, sex, geography, ethni-city, income and education, based on the 2006 censusdata for BC, and screened for experience with chronicdisease (y/n), have children (y/n) and residence by BChealth authority (there are five BC health authorities).Here, demographic diversity was used as a proxy forexperiential diversity. The rationale for screening bychronic disease and parenthood was to include approxi-mately 50% of participants with patient and/or care-giver experience and approximately 50% who havechildren, since health status and intergenerational rela-tionships provide important perspectives on matters oflegacy, inheritance, and sustainability. We also includedparticipants from each of the five BC health authorities,because geography may affect people’s experiences withhow health-care is delivered. We oversampled for youngmales and Aboriginal individuals, since these groups aretypically underrepresented in public discourse but con-tribute to the social pluralism of BC [35]. Individualswere not eligible to participate if they were: health policymakers; employees or those with a direct financial rela-tionship with a tobacco company; individuals who hadlobbied for health advocacy groups; unavailable for bothevent weekends; or had participated in a market researchstudy in the previous six months. A pool of 80 respon-dents was created at this stage.The 80 respondents were invited to complete a 16-question preference-based survey that asked them to im-agine they had been diagnosed with a serious diseaseand to select a treatment option based on characteristicssuch as their health state before and after treatment,pain level, duration of life after treatment, and cost.Respondents were grouped into one of three categoriesbased on their preferences. From this, the research teamidentified potential participants based on their various lifeexperiences (expressed demographically) and preferences(expressed categorically). Respondents received a $25 hon-orarium for completing the survey. The preference-basedsurvey was developed specifically for this study and is inAdditional file 1.Using an algorithm developed by the research team,30 individuals were selected from the 80 respondentsand invited to participate in the event. All participantssigned a written informed consent form prior to theevent. The target sample size of 30 individuals was suffi-cient for us to stratify individuals for diversity as speci-fied in the recruitment criteria outlined above [35]. Thealgorithm was also used to find replacements if someonedropped out prior to the event.Deliberative public engagementDeliberative public engagement methods developed byBurgess and O’Doherty [26] were adapted and imple-mented for this project. The event took place over twoweekends and involved small and large group sessions ledby trained facilitators. The first weekend was designed toset the foundations for meaningful deliberation by i) pro-viding participants with sufficient and quality informationon a range of perspectives related to cancer drug funding(via expert speakers, an information booklet, and a web-site) and ii) helping people develop the skills required ofparticipants in reasoned, dialogic exchange focused oncollective decision making. The purpose of the secondweekend was to enable participants to draw on their newskills and knowledge to make recommendations on prior-ity setting in funding for cancer drugs.Small and large group sessions were also designed toserve different deliberative purposes. The small groupsessions were intended to: i) provide a smaller forum toencourage less vocal speakers to participate; ii) modeland establish deliberative norms, such as listening withrespect, being open to others’ viewpoints, and seekingand providing clarity on positions; and iii) generate abroad range of perspectives on the topic of discussion.Facilitators were to avoid drawing conclusions in thesmall groups in order to avoid identity formation andthe frustration of drawing premature conclusions thatmay need to be re-justified in the whole group setting.The purpose of the large group deliberation was to i) en-sure that participants receive the same information andinstructions; ii) introduce various viewpoints aired in thesmall group sessions to the whole group; and iii) workcollectively to formulate recommendations for policy.The deliberative questions were developed prior to theevent and in direct consultation with key decisionmakers at CPAC, the BC Ministry of Health, BC Cancer,and the pan-Canadian Oncology Drug Review (pCODR).With the exception of the governance question, alldeliberative questions were posed first in small groupsand then discussed collectively in the large group.Recommendations were made collectively in the largegroup only.Bentley et al. BMC Health Services Research  (2018) 18:339 Page 3 of 13The recommendations were composed by the partici-pants, who voted on them using electronic clickers.Consensus was not the goal of the deliberations or ofthe voting exercise. The activity of voting was used bythe research team as a tool to determine where partici-pants’ views converged and diverged. Supporting andopposing viewpoints were captured explicitly during theevent in order to help decision makers and citizens gaina better understanding of the values they share and don’tshare, so they can be incorporated into public policy in ameaningful way. Participants who voted “no” on a rec-ommendation were asked their reasons for disagreeingwith it. A participant might disagree because he/sherejected the recommendation, disagreed with its word-ing, or determined the recommendation was redundantor not necessary. Transparent accountability for partici-pants’ diverse views also supports the trustworthinessand legitimacy of recommendations for decision makersand the wider public.On the final day of the event (Day 4), participantsreviewed and re-voted as a group on recommendationsthey made the previous weekend. Re-voting gave partici-pants the opportunity to reassess earlier statements andallowed the research team to capture the reasons behindany changed perspectives.All sessions were audio recorded and transcribed.Transcript analysisThe event format affects the methods used to analyze tran-scripts. In essence, the data from transcribed proceedingsof deliberative public engagements are fundamentally dif-ferent from the qualitative data generated from interviewsor focus groups [36]. Because deliberative events facilitatecollectively reached recommendations after a process oflearning and engagement with others, deliberants’ know-ledge and sense of collectivity accrue over time, as theylearn from one another, reflect on what they’ve learned,and then direct their knowledge to collective ends. Inter-views and focus groups, on the other hand, elicit differenttypes of knowledge over much shorter time spans, with in-terviews being repeatable instances of the same phenom-ena (i.e., different subjects answering a set roster ofquestions). Deliberative proceedings thus are not amenableto straightforward thematic or content analysis, since par-ticipants’ statements on Day 1 and Day 4 cannot beweighted equally, and more significance must be accordedto statements that are more informed and morecollectively-oriented than other statements [36]. Moreover,because of the dialogic and reason-giving nature of deliber-ation, participants may change their minds over time inresponse to increased knowledge or more persuasive argu-ments. Changeability under these conditions is positivelyvalued; it is not regarded as contradictory or incoherent, asit may be if all statements were weighted equally.The transcripts were entered into NVivo qualitativedata analysis software (QSR International Pty Ltd.,Version 10, 2012) for coding. Both deductive and induct-ive coding were used. Deductive coding begins withestablished categories and codes, which are developedfrom the scholarly literature and research experience.Here, the deliberative questions and the layout of theevent (e.g., small and large group settings, Day 1, Day 2,etc.) helped set the categories in advance of the deliber-ation itself. Inevitably, coding deductively imposes con-straints on what discursive content is captured and thusavailable for analysis. To combat these constraints,inductive (or “open”) coding methods were used tocapture emergent ideas. Participants were de-identifiedand given a number (e.g., Participant 1, Participant 2, etc.)during transcript analysis.All sessions were coded and analyzed by one reviewerand a second reviewer independently coded 37% of thetranscripts. An average Kappa score of 0.67 was achievedbetween the reviewers for all nodes and sources(unweighted). Weekly norming sessions were also heldto review, track, and resolve issues as they arose.ResultsA total of 24 individuals participated in the deliberation.Participants’ characteristics are in Table 1. Chi-squaretests showed that the final 24 participants met our re-cruitment criteria. Participants made a total of 30 rec-ommendations over the four days. The full list ofrecommendations is reported elsewhere [32]. Below isan analysis of key recommendations from the Vancouverevent. The terms “Most” and “All” beside each recom-mendation indicates whether or not the recommenda-tion had consensus, even though consensus was not thegoal of the event or of making recommendations.Deliberative question on disinvestment: Under whatcircumstances is there an obligation to continue to fund acancer drug when new information suggests the drug isnot as desirable as previously determined?Recommendations: There is an obligation to continue to fund a cancerdrug if discontinued funding would have a negativeimpact on populations in rural communities andothers with limited access. (All) There is an obligation to continue to fund a cancerdrug if it is significantly easier to use compared toother drugs or treatments (for example, oral vs.intravenous drugs). (Most)In their deliberations about disinvestment, participantsvoiced concerns over equity of access and patients’ability to tolerate an alternative treatment in the eventBentley et al. BMC Health Services Research  (2018) 18:339 Page 4 of 13that a drug is removed from the public drug formulary (apublic drug formulary is a list of drugs and drug productscovered by a province’s publicly-funded health-care plan).Their concerns over equity of access were expressed interms of how delisting a drug would affect those in “ruralcommunities and others with limited access”. Participantsfelt it was unacceptable to delist a drug if doing so createdbarriers to health-care utilization for patients living inrural and remote areas, particularly if the drug beingdelisted was an oral medication that could be taken athome instead of an intravenous (IV) drug that may re-quire patients to travel long distances to receive. They alsoworried that disinvestment decisions might create haveand have-not patients:Participant 4: You have to keep the funding going forthese people [already on the drug] because you can’tjust [say]…there’s a new one for us city dwellers orwhatever. So I think …if you’re going to stop [fundingit, then] stop it…The new drug is available to allpeople. You can’t split it up.(Small group G, Day 2)“Split[ting]” the drug between urban and rural patientswould result in discriminatory practices that were un-acceptable to participants. They suggested instead thatthe obligation to fund a drug extended beyond ruralityto include patients “with limited access,” such as patientswith mental illness, and those who were homeless,housebound, First Nations, or had mobility constraints.One participant summarized the discussion from hersmall group, saying, “We were really concerned aboutfairness around the availability of drugs” (Participant 1,Large group, Day 2).Concerns over access and fairness when delisting adrug were also evident in participants’ lengthy discussionof what they meant by “significantly easier to use” intheir second recommendation. For several deliberants,“easier to use” referred to a preference for oral over IVdrugs. One participant described IV drugs as “very hardon your veins…and corrosive to your body” and wouldtake a pill over IV chemotherapy “any day” (Participant18, Large group, Day 3). Some participants wanted amore expansive definition of “easier to use” to includepatients’ ability to tolerate the replacement drug, inde-pendent of whether the replacement drug is more con-venient to use:Participant 1: We are also saying that, in addition topeople who might live in rural areas, maybesomebody is unable to take a particular pill that needsto be taken with a glass of milk because they can'tdrink milk, and this new drug can only be taken withmilk. So now all of a sudden the people who areallergic to milk cannot take this new drug, because itis not as easy to use as the old one.(Large group, Day 2)Table 1 Participants’ characteristicsCharacteristics Count N (%)SexFemale 13 (54.2%)Male 11 (45.8%)Age (years)18–24 1 (4.2%)25–34 5 (20.8%)35–49 4 (16.7%)50–64 9 (37.5%)65+ 5 (20.8%)Residence by Provincial Health AuthorityaFraser 7 (29.2%)Vancouver 8 (33.3%)Interior 4 (16.7%)Island 4 (16.7%)Northern 1 (4.2%)Experience with chronic illness (personal or as caregiver)No 16 (66.7%)Yes 8 (33.3%)EthnicityCaucasian 16 (66.7%)Chinese 3 (12.5%)South Asian 1 (4.2%)Aboriginal 1 (4.2%)Other 3 (12.5%)Highest education level attainedHigh school 6 (25.0%)Some university 2 (8.3%)University or College 16 (66.7%)Annual household incomeLess than $20,000 3 (12.5%)$20,000 - $34,999 3 (12.5%)$35,000 - $49,999 3 (12.5%)$50,000 - $79,999 7 (29.2%)$80,000 or above 8 (33.3%)Have children?Yes 13 (54.2%)No 11 (45.8%)aA map of the Health Authorities in BC can be found here: https://www2.gov.bc.ca/gov/content/health/about-bc-s-health-care-system/partners/health-authorities/regional-health-authoritiesBentley et al. BMC Health Services Research  (2018) 18:339 Page 5 of 13Once again participants worried that disinvestment deci-sions would create opportunities to discriminate be-tween categories of patients. Although taking a pill forchemotherapy may be preferable to IV treatment formany patients, it may not be the case for patients whocannot tolerate the oral treatment.Disinvestment and fairness were also linked to pa-tients’ choice. Participants argued that patients shouldbe able to remain on a drug that works for them, even ifa policy decision is made to delist the drug:Participant 4: So I think the patient should have somesay in….may decide, for whatever reason, she wouldprefer the older [drug].Participant 18: Yeah, I had to take Tamoxifen for fiveyears….I had to stop taking it because it was justmaking my life hell. So they gave me Zoladex which…didn’t give me the same side effects. So I think, yeah,like I agree with that.(Small group G, Day 2)Participant 6: I think it’s just, like, when you’re sickand stuff like that, and you’re on a certain drug, thatbeing switched from one drug to another drug [—] isjust disturbing.(Small group Y, Day 3)As these discussions show, patient choice was less aboutautonomy and more about having the chance for a betterquality of life. This point was expressed emphatically inWeekend 2’s recommendation on disinvestment: “Patientswho are taking an existing drug should have the option tostay on the existing drug even if it is more expensive than asimilar new drug.” This recommendation was nicknamed“the grandfather clause.” It showed that opportunities forfairness and a better quality of life for patients who are sta-bilized on a current treatment were important enough toparticipants that they were unwilling to trade these oppor-tunities to realize cost savings when budgets are limited.Decision scenarios 1 and 2: Trade-offs between cost andquality of life or length of lifeFour decision scenarios were presented to participantsand administered on the second weekend of the event(Day 3). The two decision scenarios are shown in Fig. 1.They were introduced because the research team feltparticipants were ready to make explicit trade-offs. TheQuality of Life (QoL) scale (Fig. 2) was also explained,including the scale’s use in health research and its sub-jective nature. They were told that most people on aver-age place themselves at 80–90 on the scale. Participantswere also given quality of life scores for Canadians(Table 2). Participants then broke into small groups todiscuss each scenario.The scenarios are constructed situations: participantswere asked to play the role of decision makers with fixedbudgets who must make a decision between two treat-ment options. The variables within the scenarios wereheld constant to force the trade-off and the specifictrade-off options—in the form of additional months oflife or extra quality of life—were also constrained. Partic-ipants’ reasons for selecting a particular trade-off aremore important than the numeric trade-off selected.Two scenarios are discussed, below.Decision scenario 1: In your opinion, how much extralength of life is needed to fund the new drug that coststwice as much as the current drug?Recommendation: There needs to be a minimum of 12 months ofadditional duration of life. (Most)Participants’ approach to this scenario varied. Somemade mathematical calculations on how much extra lifeis needed to justify twice the cost. Others felt that be-cause 60 on the QoL scale was not a good health state—and because patients’ health state in the scenario did notimprove with the new drug—they wanted greater dur-ation of life for the money. Several participants wanted a“significant improvement” in the amount of additionallife to justify twice the cost:Participant 11: I will say the one thing I have noticed asa group, none of us ha[s] picked the minimum option.Participant 10: Yeah.Participant 11: We’ve all expected a little bit more.Participant 17: Yeah, significant, yes.Participant 11: – significant improvement if we’regoing to spend twice as much(Small group B, Day 3)Participant 1: I picked 12 [months].Facilitator: 12 [months]. Is there a reason for that?Participant 1: Because three months and six months isnot a very long time, and especially if you’re lookingat a 60 percent quality of life….[P]art of that’s going tobe taken up with, like, being sick, or having to haveBentley et al. BMC Health Services Research  (2018) 18:339 Page 6 of 13treatment. And really, like, three to six months…hardly seems worth it for double the cost.(Small group Y, Day 3)Participants considered “double the cost” to be a largeexpenditure and thus wanted more than the minimumamount of additional months of life from the new drug.Concern for a better return on investment pushed par-ticipants to question the worthwhileness of investing“significant” amounts of money in a new drug thatshowed only a small increment in health benefit. Gettinggood value for money was also important in the largegroup discussion of this scenario:Participant 13: [S]ix to twelve [months means] yougot time to get your stuff together, and approach [theend of life]….[F]or some people six months is plenty,for other people a year.Participant 22: I think the 12 months and higher was ifyou looked at only money....I sat there trying to decidebetween 12 and 6 [months] because I’m like, well, ifwe’re talking dollars, 12. If we’re talking emotions, 6.(Large group, Day 3)This quotation shows participants’ desire to be goodstewards of public finances (“I think the 12 months andhigher was if you looked at only money”). It also demon-strates their effort to strike a balance between civic re-sponsibility and compassion for others (“[I]f we’retalking dollars, 12. If we’re talking emotions, 6”), andbalance collective with personal viewpoints. Decisionmakers endeavour to strike this difficult balance all thetime. Having sufficient time to “get your stuff together” orcomplete a ‘bucket list’ were additional reasons forsupporting more than minimum amounts of extendedlife for double the budget. Several felt that it was un-realistic to expect 18–24 months of extra life fromthe new drug.Not all participants supported the recommendation.Some were willing to pay twice the price for a new drugthat extended patients’ lives by fewer than 12 months:Decision scenario 1Decision scenario 2Fig. 1 Decision scenarios. Participants were given the following instructions and decision scenarios on Day 3 of the deliberation: Imagine you aredecision-makers, responsible for the cancer budget. The decision you make will affect the people in your province only. A fixed budget has beenset aside to fund one treatment. The budget is not large enough to fund both of the treatments. The budget will only fund healthcare andcannot be used for research that may improve a patient’s condition in the future. The total cost of the treatment includes all costs that arerelevant to the decision. (Decision scenario 1): Trade-off between cost and additional length of life. (Decision scenario 2): Trade-off between costand increased quality of lifeBentley et al. BMC Health Services Research  (2018) 18:339 Page 7 of 13Participant 3: I [was] saying to my group, I was like,the last week of mom's life, for five more minutes I’dhave given like $10,000. So, for me it's really hard tojust put that dollar amount on that amount of time.So I'm like six months, I would take it.Participant 24: Six for me as well.Participant 7: I would say three to six....Every momentis precious and time matters.(Large group, Day 3)Here, participants resisted trading value for moneywhen the benchmark for value was set at 12 months bythe group. Unsatisfied with the 12-month threshold, theyFig. 2 Quality of Life Scale. Note: Usual activities means things like work, study, family, and leisure activitiesTable 2 Quality of Life Scores for Canadians. Participants weregiven the following information about the age groups andaverage quality of life scores for Canadians (men and women)Age group Average score18–29 8330–39 8340–49 8150–59 8160–69 7870–79 7380+ 65Bentley et al. BMC Health Services Research  (2018) 18:339 Page 8 of 13considered fewer months of extended life to be accept-able because “every moment is precious” even whendrug costs are high and budgets are limited.Participants’ quantitative trade-offs—i.e., whether theywere willing to trade 3, 6, or 12 months of additional lifeat a “significant” cost—are less important than their rea-sons for selecting them. For this scenario, participantsaccepted that trade-offs are necessary and most wantedto see more than a minimum increase in health benefitfrom expensive drugs. Participant 7 was the onlyparticipant to consider the minimum health benefit (i.e.,3 months) as a suitable trade-off for a drug that coststwice as much.Decision scenario 2: In your opinion, how much extraquality of life is needed to fund the new drug that coststwice as much as the current drug?Recommendation: There needs to be a minimum of 20 points ofimprovement in quality of life. (Most)Overall, participants described a meaningful quality oflife as being able to think and care for oneself, spendingtime with loved ones, and living without pain or nausea;for some it meant being able to work. They related theirdescriptions to the QoL scale and the decision scenario.As expected, the points on the scale meant different thingsto different people. For instance, one person interpreted60 on the scale as being bedridden and therefore undesir-able. Some interpreted 50 on the scale as a manageablehealth state, whereas others considered 50 to be a poorhealth state. Most felt that 70 on the scale meant you werestill a productive member of society:Participant 3: I was just going to say, like, for aspecific percentage… our entire group also said 70.Because I feel like, 80 to 90 is really good. Likegenerally what we all feel now. And so I feel like youcould...justify doubling the cost feeling a little bitcrappier than you do now, but you don't really wantto go down too far. So we all ended up picking 70.Facilitator: 70?Voices: Yeah. Yes.(Large group, Day 3)Participants wanted to see a clear improvement in qualityof life from the new drug if it were to cost twice as muchas the current treatment. They quantified the improve-ment as an increase of 20 points on the QoL scale—thatis, a climb from 50 to 70 points—yet determined that 80or 90 on the scale was, as one participant put it, “optimumhealth, and maybe not that achievable” for a cancer patient(Participant 23, Large group, Day 3). One small group alsowanted to see “marked improvements” in health benefitfrom the new drug in order to send a message to pharma-ceutical companies: “By having [quality of life] at a mini-mum of 70, what it does is it also helps to promote theidea that if you want us to fund your drugs at double thecost, we want to see marked improvements” (Participant14, Small group R, Day 3).Participants who disagreed with the recommendationwere largely unpersuaded by value for money assess-ments when it came to patients’ quality of life. Theyconsidered 10 points’ increase on the scale to be an ac-ceptable magnitude of benefit despite the expense:Participant 4: I have cancer...and I'm not expected toget 70 or 80 [on the QoL scale]…. So if I could get 10percent, that is a lot in a cancer patient….Ten percentquality improvement is – I could live with that.Participant 7: To me 10 percent is good. It makes abig difference, and I totally agree with what you'resaying. I worry that we're – when we say things like20 [points on the QoL scale] and again, with the 12months [additional length of life] too, we're settingthe bar too high.(Large group, Day 3)Those who disagreed with the recommendation felt theycould justify—i.e., “could live with”—a decision to spendtwice as much on a new drug in return for small gainsin patients’ quality of life. Quality of life was clearly im-portant to all participants, with some willing to lowerthe threshold of benefit from expensive drugs in orderto get it.Deliberative question on governance: What would makedrug funding decisions trustworthy?Recommendations: There is a need for transparency around how drugfunding decisions are made, what stakeholders areinvolved, and possible conflicts of interest. (All) There is a need for an independent body that wouldoversee and review drug funding decisions andinvolve a variety of people without politicalmotivations (participants were concerned aboutpatronage). (Most)The deliberative question on trustworthy governancewas discussed on the final day of the event and in the largegroup only. Above all, participants wanted reassurancesBentley et al. BMC Health Services Research  (2018) 18:339 Page 9 of 13that drug funding decisions were made in a non-partisanand transparent way. They wanted members of drug fund-ing committees to be health-care professionals like oncol-ogists and pharmacists, and to include patients andmembers of the public. Committee members would be“hired, not appointed” (Participant 2, Large group, Day 4)to prevent pharmaceutical companies and politicalappointees from influencing decisions and to keep theprocess free of nepotism. In addition, the public wouldhave confidence in decisions reached by the committee ifits decision-making process and outcomes were placed inthe public domain:Participant 18: I'm going to, you know, trust how theymake decisions on drug funding [if I know how] theymake their decision….[I]t would be nice to know howthey make their decision in a broad way…and publishthat to the public as well.(Large group, Day 4)Participants agreed that posting this information on awebsite, along with clinical trials data and input frompatients and the public, would enhance people’s trust indecisions about funding for cancer drugs. The BCgovernment and the Canadian Agency for Drugs andTechnologies in Health (CADTH) publish their drugreviews and health technology assessments (HTAs) ontheir websites. Even after learning this was the case, par-ticipants wanted to express their strong preference fortransparency and trustworthiness in HTA-type decisionprocesses by making a recommendation to this effect.Participants also agreed on the need for increasedoversight of drug funding decisions. They wanted a com-mittee or “independent body” to have the authority toreview drug funding decisions on a regular basis. Itsmandate would be to:Participant 13: Oversee and review.Participant 11: [O]versee kind of means they have theright to kind of step in and change things….[I]f theyare just reviewing it and looking for conflicts thenthey can point those out.(Large group, Day 4)Participants believed it was important to have in place amechanism or opportunity to revisit and revise current de-cisions (i.e., to “step in and change things”) when necessary.DiscussionDeliberative engagement events are opportunities to gar-ner informed public input on pressing policy questions.The Vancouver deliberation brought together 24 BritishColumbians with various demographic and preferencecharacteristics so that a range of experiences and per-spectives could inform the consultation on setting prior-ities for cancer drug funding in BC. In addition, thestructure of the deliberation—for instance, small andlarge group discussions over two weekends, supportingdeliberation with information from expert speakers anda booklet, and using decision tools like the scenarios anda voting system—involved participants in a process ofdiscussion and reflection directed at finding collectivesolutions to funding high-cost cancer drugs when re-sources are limited.Significantly, participants accepted the premise thatbudgets are limited: no one said “fund everything.” Theyunderstood the necessity of making cost-benefit assess-ments as a way of addressing complex social problems,like health care. This was evident in their discussion ofthe decision scenarios, when they were asked to assumethe role of decision makers. Faced with a fixed budgetand a new treatment that cost twice as much as thecurrent treatment, participants-as-decision-makers byand large wanted a “marked improvement” in healthbenefit given the budget impact. While not all partici-pants agreed on 6 or 12 months of additional length oflife or 20 points improvement on the QoL scale fordouble the cost, they nonetheless supported theprinciple of getting good value for money—or a good re-turn on investment—when setting health policy. Theyaccepted cost as a unit of evaluation across contexts andtreatments.Participants also accepted using cost to set thresholds.It is worth recalling that the thresholds participants setwere hypothetical, like the scenarios themselves. For in-stance, it may be unrealistic to expect a magnitude of12 months of additional life from today’s oncology drugs,as many participants did, given the small gains in effect-iveness seen from some new cancer drugs [7–10]. It isalso unrealistic to expect quantitative or generalizablethresholds from deliberative events that can be appliedto other decisions. Numeric standards for trade-offs areonly possible using quantitative methods either toaggregate across disagreements or derive the majorityperspective. However, both of these approaches are in-consistent with qualitative research generally and, in thisinstance, with the goals of deliberative public engage-ment, which is to understand and respect the ways inwhich an informed public comes to agree or disagreeover what the most important consequences are of agiven funding decision.In setting thresholds, participants told us what was im-portant to them independent of the numeric value of thetrade-off. For example, quality of life was more import-ant to them than quantity of life, since they were willingBentley et al. BMC Health Services Research  (2018) 18:339 Page 10 of 13to accept smaller increases in quality of life from a high-cost cancer drug but demanded greater gains in lengthof life to justify the same expenditure. In other words,they wanted to see greater than minimum increments ineffectiveness from some high-cost drugs—like those thatextend life—when they rejected small health gains as in-sufficient for the price. Understanding the public’s appe-tite for types of trade-offs provides valuable guidance todecision makers when weighing the opportunity costsrelated to fair and sustainable funding decisions.Cost was not the sole factor in setting thresholds,however. It was cost and a capacity to consider others’perspectives when forming group recommendations thatpushed participants to set thresholds at greater thanminimum values. Participants demonstrated this cap-acity for civic-mindedness when they strove to balancepersonal and social perspectives when setting thresholds.For instance, some were concerned about compassionfor others’ pain (“10 percent improvement [on the QoLscale] is a lot in a cancer patient”) if the cost-benefitthreshold is set too high. Others reflected on the chal-lenge of balancing responsibility for public resourceswith the impact of policy decisions on individual lives(“If we’re talking dollars, 12 [months]. If we’re talkingemotions, 6”). Significantly, participants who self-identified as patients also took up the challenge the sce-narios presented, which was to adopt the (relatively dis-interested) perspective of a decision maker in resolvingpolicy dilemmas, as this quotation from a participant-patient shows:Participant 4: I think you're right. For myselfpersonally, 60 [on the QoL Scale] I can live with. Butif I'm looking at the cost, which this whole thing isabout, I think you're right: 70 cost-wise makes moresense. You know, if I…took the cost out, I would livewith 60. I would take anything. But cost-wise, to make[the decision] relevant and beneficial for more people,you know, you have to [look at it] that way.(Small group G, Day 3)Arguably, disinterestedness is simply an abstraction, andthus an unattainable ideal for decision makers and citi-zens alike. Moreover, we cannot know for certain whatall participants—including participant-patients—had inmind when they voted in favour of a particular recom-mendation. Yet we do know that participants entertainedalternative perspectives when they spoke of trying to jus-tify cost-benefit trade-offs that are “relevant and beneficialfor more people” alongside personal wants (“I would takeanything”) or, again, when they tried to balance costs withcompassion. These instances support previous researchthat shows the public and patients are not too self-interested to participate meaningfully in policy-type dis-cussions [37] or that patients and the public do not neces-sarily have irreconcilable perspectives on health care.They also offer decision makers additional confidence thatthe recommendations were collectively reached.Participants’ more concrete recommendations relatedto trustworthy governance. Decision makers can deriveuseful guidance from the participants’ call for drug fund-ing decisions to be made in a transparent and non-partisan way, and the need for independent governanceand review of oncology drug funding decisions in BC.Members of the public, patients, and other health-careprofessionals would compose the governing body. Underthese conditions, participants are likely to trust drugfunding decision processes and results. Although the BCgovernment and CADTH publish drug reviews on theirwebsites, neither BC nor Canada (at the national level)currently has a mechanism that provides this kind of over-sight on an ongoing basis. The practicalities of implement-ing such a body may not be concrete—participants werenot tasked with implementation considerations becausethey do not have the relevant expertise—yet the recom-mendations can nonetheless guide practical choices aboutbuilding infrastructure to identify and support publicallyacceptable trade-offs for costly cancer drugs.A limitation of this study is that the results cannot beextrapolated to other contexts. The event and its partici-pants were socially situated, meaning the event consti-tuted a specific collection of people deliberating underspecific conditions at a specific time. For this reason, therecommendations cannot be generalized to other publicsand locations. Of course, the goal was not to produce anaccount of how the population of BC would draw ontheir specific experience to express their individualpreferences. However, precisely because the recommen-dations are the result of informed and reflective deliber-ation—that is to say, they were made in the context ofconsidering alternative perspectives and reasons forthem—the principles underpinning the recommenda-tions are likely to be more durable because they werearrived at from multiple perspectives and account formultiple forms of reasoning.A second limitation of the study also relates togeneralizability in the use of decision scenarios.Constraints inherent in decision scenarios means anyrecommendations produced from them will be similarlyconstrained. In this study, the recommendationsproduced from the scenarios were constrained because:i) the scenarios inevitably focused on some decision as-sumptions and excluded others (e.g., stage of disease,treatment type, overall budget), which limited the trade-offs considered; and ii) the numeric values of certainrecommendations (e.g., “There needs to be a minimumof 12 months of additional duration of life”) are artificialBentley et al. BMC Health Services Research  (2018) 18:339 Page 11 of 13and must not be interpreted at face value. In addition,the use of tools like decision scenarios means that otherimplications of funding for cancer drugs—for instance,funding for rare cancers—were not explored in relationto issues of fairness and sustainability. Throughout theevent, the researchers sought a balance between provid-ing enough relevant information to support quality de-liberation and avoiding the provision of too much orcertain types of information that would shift the focusaway from the public policies the deliberations weremeant to inform, as per the pre-event consultations withdecision makers.ConclusionThe Vancouver deliberative public engagement eventand participants’ recommendations provide a set ofbaseline perspectives on what participants collectivelythought made for good, trustworthy decisions aboutfunding for cancer drugs in BC. Analysis of the eventtranscripts showed that participants grasped the core is-sues under consideration, and identified cost-benefit andequity trade-offs through respectful and civic-mindeddeliberation, thereby demonstrating the public can beobjective and participate meaningfully in policy-type dis-cussions. Participants accepted the principle of resourcescarcity and the consequent need of governments tomake difficult trade-offs when allocating health-care re-sources across the population. When costs are high, par-ticularly in the area of oncology drugs, they supportedthe view that cost-benefit thresholds must be set. Assuch, they expected reasonable health benefits in returnfor large expenditures, and supported the view thatsome drugs do not merit funding. Participants alsocalled for drug funding decisions to be made in a non-partisan and transparent way. Together, the recommen-dations arising from the Vancouver deliberation canprovide guidance to policy makers in BC and may be use-ful to challenge pricing by pharmaceutical companies.Additional fileAdditional file 1: Preference-based Survey. Potential participantscompleted this 16-question preference-based survey of their preferencesfor different types of treatments as part of the recruitment strategy forthe deliberation. (DOC 181 kb)AbbreviationsBC: British Columbia; CADTH: The Canadian Agency for Drugs andTechnology in Health; CPAC: The Canadian Partnership Against Cancer;HTA: Health technology assessment; IV: Intravenous; pCODR: Thepan-Canadian Oncology Drug Review; QoL: Quality of lifeAcknowledgementsWe would like to acknowledge the CanEngage team for their efforts inconducting this deliberative event.FundingThe deliberation was sponsored through research grants from the CanadianInstitutes of Health Research Partnership in Health Systems Integration (CIHR-PHSIGrant #114107), the Michael Smith Foundation for Health Research, and theCanadian Centre for Applied Research in Cancer Control (ARCC). ARCC is fundedby the Canadian Cancer Society Research Institute (Grant #019789).Availability of data and materialsThe data generated during the current study are not publicly available tosafeguard participants’ confidentiality but are available from thecorresponding author on reasonable request.Authors’ contributionsCB, SC, DR, HMC, MMB, and SP contributed to the theoretical and practicalaspects of the deliberative public engagement event, its implementation,and attended the event. CB, SC, DR and HMC developed the recruitmentmethods and criteria. DR designed the recruitment DCE and decisionscenarios. Data analysis and interpretation were performed by CB and SC. CBled on drafting the manuscript. CB, SC, MMB, and SP contributed in draftingthe manuscript. All authors contributed to revising the manuscript. Allauthors reviewed and approved the final manuscript.Ethics approval and consent to participateThis study was approved by the University of British Columbia - BritishColumbia Cancer Agency Research Ethics Board (study #H11–02226). Allparticipants signed a written informed consent form prior to the event.Competing interestsThe authors declare they have no competing interests.Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.Author details1Canadian Centre for Applied Research in Cancer Control, BC Cancer, 675West 10th Avenue, Vancouver, BC V5Z 1L3, Canada. 2W. Maurice YoungCentre for Applied Ethics, School of Population and Public Health, MedicalGenetics, Southern Medical Program, University of British Columbia, 1088Discovery Avenue, Kelowna, BC V1V 1V7, Canada. 3School of Population andPublic Health, University of British Columbia, 2206 East Mall, Vancouver, BCV6T 1Z3, Canada. 4Faculty of Health Science, Simon Fraser University, BlussonHall, Room 11300, 8888 University Drive, Burnaby, BC V5A 1S6, Canada.Received: 21 June 2017 Accepted: 12 April 2018References1. Bach PB. Limits on Medicare's ability to control rising spending on Cancerdrugs. N Engl J Med. 2009;360(6):626–33.2. Cressman S, et al. A time-trend economic analysis of Cancer drug trials.Oncologist. 2015;20(7):729–36.3. Mailankody S, Prasad V. Five years of Cancer drug approvals: innovation,efficacy, and costs. JAMA Oncol. 2015;1(4):539–40.4. Experts in Chronic Myeloid Leukemia. The price of drugs for chronicmyeloid leukemia (CML) is a reflection of the unsustainable prices of cancerdrugs: from the perspective of a large group of CML experts. Blood.2013;121(22):4439–42.5. Motzer RJ, et al. Sunitinib versus interferon alfa in metastatic renal-cellcarcinoma. N Engl J Med. 2007;356(2):115–24.6. O'Brien SG, et al. Imatinib compared with interferon and low-doseCytarabine for newly diagnosed chronic-phase chronic myeloid leukemia.N Engl J Med. 2003;348(11):994–1004.7. Schrag D. The price tag on progress — chemotherapy for colorectal Cancer.N Engl J Med. 2004;351(4):317–9.8. Davis C, et al. Availability of evidence of benefits on overall survival andquality of life of cancer drugs approved by European medicines agency:retrospective cohort study of drug approvals 2009-13. BMJ. 2017;359:j4530.9. Kantarjian HM, et al. Cancer drugs in the United States: Justum Pretium–thejust price. J Clin Oncol. 2013;31(28):3600–4.Bentley et al. BMC Health Services Research  (2018) 18:339 Page 12 of 1310. Vivot A, et al. Clinical benefit, price and approval characteristics ofFDA-approved new drugs for treating advanced solid cancer, 2000-2015.Ann Oncol. 2017;28(5):1111–6.11. de Groot F, et al. Ethical hurdles in the prioritization of oncology care.Appl Health Econ Health Policy. 2017;15(2):119–26.12. Marckmann G, In der Schmitten J. Financial toxicity of Cancer drugs:possible remedies from an ethical perspective. Breast Care (Basel).2017;12(2):81–5.13. McDonald H, et al. Challenges in striving to simultaneously achieve multipleresource allocation goals: the pan-Canadian oncology drug review (pCODR)example. J Mark Access Health Policy. 2016. 4.1. https://doi.org/10.3402/jmahp.v4.31463. Accessed 27 Apr 2018.14. Abelson J, Blacksher EA, Li Kathy K, Boesveld SE, Goold SD. Publicdeliberation in health policy and bioethics: mapping an emerging,interdisciplinary field. J Public Deliber. 2013. Vol. 9:Iss 1, Article 5. fromhttp://www.publicdeliberation.net/jpd/vol9/iss1/art5. Accessed 27 Apr 2018.15. Blacksher E, et al. What is public deliberation? Hast Cent Rep. 2012;42(2):14–6.16. Mitton C, Donaldson C. Health care priority setting: principles, practice andchallenges. Cost Eff Resour Alloc. 2004;2(1):3.17. Thokala P, et al. Multiple criteria decision analysis for health care decisionmaking–an introduction: report 1 of the ISPOR MCDA emerging goodpractices task force. Value Health. 2016;19(1):1–13.18. Peacock SJ. Public attitudes and values in priority setting. Isr J Health PolicyRes. 2015;4:29.19. O’Doherty KC. Synthesizing the outputs of deliberation: extractingmeaningful results from a public forum. J Public Deliber. 2013;9(1):203–15.20. Solomon S, Abelson J. Why and when should we use public deliberation?Hast Cent Rep. 2012;42(2):17–20.21. Stafinski T, Menon D, Yasui Y. Assessing the impact of deliberative processeson the views of participants: is it ‘in one ear and out the other’? HealthExpect. 2014;17(2):278–90.22. Menon D, Stafinski T. Engaging the public in priority-setting for healthtechnology assessment: findings from a citizens’ jury. Health Expect.2008;11(3):282–93.23. Einsiedel EF, Ross H. Animal spare parts? A Canadian public consultation onxenotransplantation. Sci Eng Ethics. 2002;8(4):579–91.24. Bombard Y, et al. Eliciting ethical and social values in health technologyassessment: a participatory approach. Soc Sci Med. 2011;73(1):135–44.25. Abelson J, et al. Does deliberation make a difference? Results from a citizenspanel study of health goals priority setting. Health Policy. 2003;66(1):95–106.26. O’Doherty KC, Burgess MM. Engaging the public on biobanks: outcomes ofthe BC biobank deliberation. Public Health Genom. 2009;12(4):203–15.27. Nadler E, Eckert B, Neumann PJ. Do oncologists believe new Cancer drugsoffer good value? Oncologist. 2006;11(2):90–5.28. Martin DK, Pater JL, Singer PA. Priority-setting decisions for new cancerdrugs: a qualitative case study. Lancet. 2001;358(9294):1676–81.29. Gibson JL, Martin DK, Singer PA. Priority setting for new technologies inmedicine: a transdisciplinary study. BMC Health Serv Res. 2002;2:14.30. Berry SR, et al. The effect of priority setting decisions for new cancer drugson medical oncologists' practice in Ontario: a qualitative study. BMC HealthServ Res. 2007;7(1):193.31. Regier DA, et al. Public engagement in priority-setting: results from apan-Canadian survey of decision-makers in cancer control. Soc Sci Med.2014;122:130–9.32. Peacock SJ, Bentley, C, Regier DA, Burgess MM. Making decisions aboutfunding for cancer drugs: a deliberative public engagement - summaryreport; 2015. from www.cc-arcc.ca/societal-values-and-public-engagement-2/.Accessed 27 Apr 2018.33. Goodin RE, Dryzek JS. Deliberative impacts: the macro-political uptake ofmini-publics. Polit Soc. 2006;34(2):219–44.34. MacKenzie M, Warren ME. In: Parkinson MJ, editor. Two trust-based uses ofminipublics in democratic systems, in deliberative systems: deliberativedemocracy at the large scale. Cambridge: Cambridge UP; 2012. p. 95–124.35. Longstaff H, Burgess MM. Recruiting for representation in public deliberationon the ethics of biobanks. Public Underst Sci. 2010;19(2):212–24.36. O'Doherty K. Synthesizing the outputs of deliberation: extractingmeaningful results from a public forum. J Public Deliber. 2013: Vol. 9: Iss 1,Article 8. Accessed 28 Apr 2018. from http://www.publicdeliberation.net/jpd/vol9/iss1/art8.37. Bruni RA. Public engagement in setting priorities in health care. CMAJ. 2008;179(1):15–18.Bentley et al. BMC Health Services Research  (2018) 18:339 Page 13 of 13


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items