Open Collections

UBC Faculty Research and Publications

Stem cells and beta cell replacement therapy: a prospective health technology assessment study Wallner, Klemens; Pedroza, Rene G; Awotwe, Isaac; Piret, James M; Senior, Peter A; Shapiro, A. M J; McCabe, Christopher Jan 30, 2018

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
52383-12902_2018_Article_233.pdf [ 1.43MB ]
Metadata
JSON: 52383-1.0363335.json
JSON-LD: 52383-1.0363335-ld.json
RDF/XML (Pretty): 52383-1.0363335-rdf.xml
RDF/JSON: 52383-1.0363335-rdf.json
Turtle: 52383-1.0363335-turtle.txt
N-Triples: 52383-1.0363335-rdf-ntriples.txt
Original Record: 52383-1.0363335-source.json
Full Text
52383-1.0363335-fulltext.txt
Citation
52383-1.0363335.ris

Full Text

RESEARCH ARTICLE Open AccessStem cells and beta cell replacementtherapy: a prospective health technologyassessment studyKlemens Wallner1* , Rene G. Pedroza2, Isaac Awotwe1, James M. Piret2, Peter A. Senior3,4,A. M. James Shapiro3,4,5 and Christopher McCabe1AbstractBackground: Although current beta cell replacement therapy is effective in stabilizing glycemic control in highly selectedpatients with refractory type 1 diabetes, many hurdles are inherent to this and other donor-based transplantation methods.One solution could be moving to stem cell-derived transplant tissue. This study investigates a novel stem cell-derived graftand implant technology and explores the circumstances of its cost-effectiveness compared to intensive insulin therapy.Methods: We used a manufacturing optimization model based on work by Simaria et al. to model cost of the stem cell-based transplant doses and integrated its results into a cost-effectiveness model of diabetes treatments. The disease modelsimulated marginal differences in clinical effects and costs between the new technology and our comparator intensiveinsulin therapy. The form of beta cell replacement therapy was as a series of retrievable subcutaneous implant deviceswhich protect the enclosed pancreatic progenitors cells from the immune system. This approach was presumed to be aseffective as state of the art islet transplantation, aside from immunosuppression drawbacks. We investigated two differentcell culture methods and several production and delivery scenarios.Results: We found the likely range of treatment costs for this form of graft tissue for beta cell replacement therapy.Additionally our results show this technology could be cost-effective compared to intensive insulin therapy, at awillingness-to-pay threshold of $100,000 per quality-adjusted life year. However, results also indicate that mass productionhas by far the best chance of providing affordable graft tissue, while overall there seems to be considerable room for costreductions.Conclusions: Such a technology can improve treatment access and quality of life for patients throughincreased graft supply and protection. Stem cell-based implants can be a feasible way of treating a widerange of patients with type 1 diabetes.Keywords: Type 1 diabetes, Stem cells, Medical device, Transplantation, Disease simulation, Cost optimization,Cost modeling, Health technology assessment, Early technology assessment, Health economicsBackgroundAlthough islet cell transplantation is effective for treatingcertain type 1 diabetes patients, some hurdles are inherentto this and other donor-based transplantation methods[1–4]. Two hurdles are the limited graft supply and graftrejection. One solution for islet cell transplantation couldbe to move from donor-harvested to stem cell-derivedtransplant tissue. That could involve production ofpancreatic progenitor cells from human embryonic stem(hES) cells. Using stem cells in general may have some ad-vantages compared to current islet cell transplantation.These advantages include the potential of producing stemcells in large quantities thereby eliminating the cell supplyproblem and possibly reducing the treatment cost perpatient.Research in that area of treatment has advanced fromproof-of-principle studies in animals, to establishing* Correspondence: wallner@ualberta.ca1Department of Emergency Medicine Research Group, Department ofEmergency Medicine, University of Alberta, 8303 - 112 Street, Edmonton, ABT6G 2T4, CanadaFull list of author information is available at the end of the article© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Wallner et al. BMC Endocrine Disorders  (2018) 18:6 DOI 10.1186/s12902-018-0233-7controllable cell manufacturing processes, and the firstclinical trials in humans [5–15]. As of 2017 clinical trialsare ongoing in Canada and the United States that use athin removable device which is implanted under the skin[7, 16]. This device has hES cell-derived pancreaticprogenitor cells within a casing to shield the tissue fromthe immune system [15]. Those cells are expected tomature to functional endocrine cells which secrete insu-lin in a glucose-dependent manner [9, 14–16]. Furtherimprovement in protection of transplant tissue could in-crease its viability and reduce graft rejection. The longterm goal of research into beta cell replacement therapyis to reverse diabetes and completely avoid the need forimmunosuppressive medication.In 2011 Weir et al. mention, “due to the need for betacell replacement therapy, much work has been done inthe past decade to generate beta cells from a variety ofcell sources” [13]. However, these efforts have had mixedsuccess. A major barrier has been in the ability to directcell lines to differentiate towards an endocrine lineage.That process was very inefficient and most cell linescould not be used. Further, technologies used in thepreservation of graftable cells, for example through cool-ing to very low temperatures, have advanced consider-ably but are still difficult and costly [17, 18]. Use ofsimpler preservation technologies makes cell tissue moreperishable but experiences in standard donor-derivedtransplantation may point towards greater affordabilityof such techniques. Still, those barriers add to existingcomplexities associated with supply logistics, regulatoryframeworks and scaling out production to multiple cellmanufacturing sites [19].Given those developments and findings, stem cell-based beta cell replacement therapy is a case study forthe necessity of prioritizing research resources whenresearching new healthcare technologies. In our studywe aimed to explore the circumstances under which astem cell-based graft tissue would be cost-effective, givenits effectiveness is comparable to state of the art islettransplantation aside from immunosuppressiondrawbacks. Our core question is if and how such a newtransplant option for beta cell replacement has a chanceof being cost-effective.MethodsTo model the cost of hES cell-derived transplant doseswe used a two part cost-effectiveness and manufacturingmodel (Fig. 1). This stochastic model is based on aprevious treatment model of type 1 diabetes [20] and thework by Simaria and colleagues [21]. Presuming equaleffectiveness with the current technology islettransplantation, aside from the immunosuppressiondrawbacks, we then ran the model to simulate marginaldifferences in clinical effects and costs, between the newstem cell-based technology and our comparator inten-sive insulin therapy. We used the models outputs to esti-mate the cost-effectiveness of this trial-stage therapy.Compared treatmentsThe form of beta cell replacement therapy was modeledas a series of identical re-extractable subcutaneous im-plant devices (‘sheets’). Each of these devices contain hES-derived cells, specifically pancreatic progenitors whichwere modified to attain the function of beta cells. Thosecells are enclosed within a casing which shields them fromthe immune system but allows the transport of nutrientsto and hormones from the encapsulated cells [14, 15]. Inour study we make the important assumption that thisshielding effect completely removes the need for immuno-suppressive medication. Patients in our study can get upto four transplantations [20]. The average proportion ofpatients with full graft function after each transplantationwas assumed to be increasing from first to third andfourth transplantation, i.e. from 15% to 70% and then 85%for the third and fourth ones [20].The comparator treatment is intensive insulin therapyinvolves frequent self-monitoring of blood glucose andmultiple daily insulin injections, further details are de-scribed elsewhere [20].Cost-effectiveness analysisWe conducted a probabilistic and structural sensitivityanalysis to investigate the cost-effectiveness of stemcell-based beta cell replacement therapy and to evalu-ate uncertainty around our results. Our simulationmodel was a discrete state-transition Markov model,which had a lifetime horizon. Its hypothetical cohortwas composed of type 1 diabetes patients withhypoglycemia unawareness in the province of Albertawho fulfill the inclusion criteria to get an islettransplant. The model took the perspective of theprovincial healthcare provider and its inputs were thesame as in the pre-existing cost-effectiveness modelFig. 1 The role of our cost estimation model within the cost-effectiveness analysis. Note that we used the explicit costoptimization model for our stem cell-based treatment onlyWallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 2 of 12[20], except for the described variations. Effectivenesswas expressed in quality-adjusted life-years (QALYs)to measure the impact of therapy on both quality oflife and life expectancy. All monetary estimates areexpressed in 2016 Canadian dollars, with necessaryadjustments made using the Canadian consumer priceindex for health and personal care [22, 23].We updated model parameters from our pre-existingmodel of unstable type 1 diabetes [20] as describedbelow. We model a future technology functioning with-out the need of immunosuppression. Therefore we hadto change the parameters that – even partially - had todo with this medication. For that we removed all disutil-ities, costs and probabilities that had only to do with im-munosuppression. The parameters for rate, costs anddisutility of initial complications were adjusted by lower-ing each by 40% because this portion was attributedsolely to immunosuppression. Specifically the rate chan-ged from 0.65 to 0.39, the cost from $600 to $360, andthe disutility from 0.05 to 0.03. Further, a study of theimpact of type 1 diabetes complications (N = 2341)served us to update our utility i.e. quality of life esti-mates [24]. Patients in that study were more compar-able to our hypothetical cohort than the ones in ouroriginal data sources. Yet they were still younger(39.3 vs. 47.0 years old) and with shorter diabetesduration (16.3 vs. 29.4 years) [20, 24]. Given that newevidence, we included neuropathy in the diabetes-related complications and adjusted our overallestimate for the complications state not only for mul-tiple complications, but also to fit the actual age andduration of diabetes in our cohort [2, 24–29]. Theutility parameter in the complications state wastherefore adjusted from 0.57 to 0.47.The cost-effectiveness model was constructed and runwith the software TreeAge Pro 2016 (Williamstown,MA, USA). The cost of goods modeling was constructedand run using Microsoft Excel. Costs and benefits werediscounted at 3%, and sensitivity analysis wereperformed at 0% and 5%. These were the rates that hadbeen recommended by the Canadian Agency for Drugsand Technologies in Health [30]. Half-cycle correctionwas applied. Our probabilistic analyses used 64,000iterations for each scenario. We estimated the value offurther research reducing the decision uncertainty byway of value-of-information analysis [31, 32]. We calcu-lated the expected value of perfect information (EVPI)and the expected value of partial perfect information(EVPPI) for the cost of goods group of parameters. Forthat EVPPI calculation we used nested Monte Carlosimulations with 600 ‘outer’ and 600 ‘inner’ loops.Additional information on our value of informationapproach, including choice of WTP thresholds, can befound elsewhere [20].Integration of cost of goods resultsTo integrate the results of the cost of goods modelinginto our cost-effectiveness model, we took the parameterrepresenting the cost per transplantation within thetransplant state in the original model, and split it up intonon-dose costs and dose costs. Based on literature weassumed the non-dose costs, including the transplantprocedure, to be about 38% of the costs per transplant-ation [33]. For that parameter of our model we used aGamma distribution with a relative standard deviation of10% (i.e. standard deviation as percentage of the mean).For the dose costs, based on our cost of goods (COG)model we used the following equation:Dose costs ¼ COGupstream  factorCOGdownstream factoradditional regulation ð1ÞHere dose costs are calculated multiplying the cost ofgoods upstream by a cost of goods downstream factorand a factor that we called “regulatory burden factor”.The cost of goods upstream came from the abovedescribed fitted distributions estimating the “pure” pro-duction cost of the cells. The downstream factoraccounted for the so-called downstream processing,which is necessary after the cells are produced, e.g. cellharvesting, volume reduction, washing, formulation forstorage or delivery (see Fig. 2). We used the regulatoryburden factor to account for the possibility of additionalregulatory burden due to stricter regulatory require-ments for a new cell production process whose productis designed to enter regular healthcare practice.The mean of the cost of goods downstream factor(multiplier) was assumed to be three, four and eight, de-pending on scenario, based on expert opinion and pub-lished literature [34]. The mean of the regulatory burdenfactor was assumed to be 1.2 and 1.8 depending on sce-nario (i.e. 20% and 80% additional costs respectively dueto regulation). Both multipliers were made probabilisticusing a Log-normal distribution and expert opinion onvariation estimates.Cost of goods modelingWe modeled the cost of goods in Microsoft Excel basedon a report describing manufacture of pancreatic pro-genitors from single cell cultures of hES cells [6]. Briefly,the modeled process consisted of thawing one or morefrozen hES cells vials and expanding them in adherentculture for about 14 days, passaging four times. Then,hES cells were cultured in suspension forming cellclusters, while a cocktail of molecular signals was addedto the media to promote stepwise differentiation of hEScells into pancreatic progenitors.The cost of goods was estimated by adapting a costminimization decisional tool for this manufacturingWallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 3 of 12process [21]. The tool selected the optimal set of dispos-able culture vessels for a user-specified annual demand,lot size, cell dose and user-specified manufacturing con-straints, i.e. maximum allowed number of culture vesselsper lot, which was set to 100. In its estimation process,the tool calculated the material (i.e. media, disposableculture vessels), labor, quality control and equipmentcosts involved in the expansion and differentiation stagesof the process for a battery of sequential culture vesselcombinations (see Fig. 2). Additional parameters utilizedduring the cost calculations were the overall yield of themanufacturing process and the expansion fold of thehES cells. In that way the upstream cost of goods wereestimated.The cost minimization decisional tool did not includethe downstream component of the manufacturingprocess (e.g., finishing, packaging, shipping), therefore95% credible ranges were derived for cost estimates infour different settings: two cell culture methods (adher-ent and suspension), and each with two supply levels (50and 500 doses per year) (also see Table 1). The credibleranges were used to fit Gamma distributions, i.e. thelower and upper bounds of the credible ranges for everycost estimate were equated to the values of cumulativedensity functions (CDF) at values of CDF = 0.025 andCDF = 0.975. The distributions were then used directlyin the health economic modeling software.Scale of manufacturingWe simulated four manufacturing modes: local produc-tion (e.g. at one University only), large scale production(one central lab produces all the doses and then shipsthem to the hospitals), and two scale-out productionmodes (local and large scale). The scale-out scenariosinvolved a network of several labs producing their owndoses at their respective location but collaborating witheach other through sharing expertise and research re-sources. We simulated one scale-out scenario for localproductions and one for large scale productions. Thelocal and large scale production scenarios assume ademand of 50 and 500 doses per year respectively. Ingeneral, the scale out approach may engage the capabi-lities of multiple local institutions and companies. Itcould, however, also contribute to unequal productquality and an increased overhead costs.We estimated the long-term capacity to performdevice implants in Canada to be 10 clinical centers. Thatestimate was derived by counting the hospitals on thelist of transplant centers by the Canadian Organ Re-placement Register in which clinicians performed isletcell transplants or other transplants of at least three dif-ferent kinds of organs [35, 36]. We took this as clinicalcapacity to carry out transplantations of beta cell re-placement devices that do not require immunosuppres-sion. In the short term there could be two centers, onefor Western Canada and one Eastern Canada.We describe the demand for and composition of thedoses of beta cell replacement tissue as follows. The an-nual demand of beta cell replacement doses was based onthe current number of islet cell transplants in Canada andassumed to be 50 per transplant center, which was derivedas linear extrapolation of transplant numbers in at theFig. 2 Illustration of cost of goods modeling in a biotechnology application. On the top one can see the different parts that compose the cost ofgoods for manufactured cell products. The bottom part portraits the upstream cost of goods, highlighted in green, as proportional to the overallcost of the treatment. That is a simplification compared to our analysis, which treats costs after cell product arrival at the hospital as independentfrom the cost of upstream cell processingWallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 4 of 12University of Alberta Hospital. Further we presumed thenumber of lots produced per year is 10, i.e. about one permonth, and a minimum of 500 million cells are requiredper dose. Those numbers were derived from consider-ations of cell quality loss over time and the productionfigures above. Based on experience in the biotechnologysector the production assumed one of two productiontechnologies, adherent or suspension cell culture ap-proach, each with optimized production set ups for thetwo demand options (50 or 500 doses per year).As a substantial simplification due to the novelty ofthe membrane technology, we presumed the cost of thedevice casing without the cells is off-set by reductions incosts through increased ability to plan transplantationtimes and processes.ResultsOur analysis shows that the use of stem cells for betacell replacement therapy can be an effective use ofhealth budget funds. However, there is substantial uncer-tainty around the costs of this technology. We calculatedthe expected range of treatment costs for hES cell-basedbeta cell tissue. Our probabilistic results indicate thatcurrently this technology could be cost-effective at aWTP threshold of $100,000 per QALY because threescenarios have ICERs substantially below that threshold(Tables 2 and 3). Specifically the ICERs of scenariosAdh20, Sus19 and Sus20 are $79,230, $89,173 and$60,111 per QALY respectivly. For the 95% Confidenceinterval values around our results please see inAdditional file 1.However, the results also indicate that large-scale pro-duction has the best chance of providing affordable grafttissue, as can be seen in scenarios ADh15, Adh16 andAdh20 in Table 2. These scenarios have the highest valuefor money for this method of cell culture. That meansthat for a given patient benefit the costs are minimized.For the suspension cell cultures the same scenarios alsohad the lowest ICERs (see scenarios Sus15, Sus16 andSus20 in Table 3).With adherent cell culture all scenarios have ICERshigher than $100,000 except scenario ‘Adh20’, which hasa 0% discount rate and a supply of 500 doses per year.On the other side all suspension cell culture scenariosalso have ICERs higher than $100,000 except for thescenarios ‘Sus19’ and ‘Sus20’, both use a 0% discountrate. Such a low discount rate does value small benefitswith a long duration more favorable than a higherdiscount rate would.Our finding that use of stem cells for beta cell replace-ment therapy can be an effective use of health budgetfunds can be confirmed by the value of information re-sults. The value of information can be seen as both ameasure of decision uncertainty as well as an indicatorof research investment value [31, 32]. In Fig. 3 we showthe expected value of research into the cost-effectivenessof the technologies under consideration. One can see allper-patient EVPI values do peak at high cost-effectiveness thresholds but there also is considerablevalue when using for instance a $50,000 threshold. Thatmeans that further research into the cost-effectiveness ofthis treatment can be worthwhile for Alberta up to theseupper limits per patient, even if one uses a strict cost-effectiveness threshold of $50,000.We found uncertainty around the mean outcomes andtherefore the need to conduct further research in thiskind of disease treatment. This becomes clear when weconsider the results in Fig. 3 and the number of patientsthat could benefit. In Alberta alone there are more than4000 patients with unstable type 1 diabetes [37–42].Extrapolating this estimate, one can expect to haveabout 500,000 patients in North America [37–43]. Whencomparing those figures with the per patient values inFig. 3, one can argue that further research in this area oftechnology can be a sound investment of health budgetfunds.We report the treatment dose costs with the produc-tion settings we used for a set of example regulatory andcost of goods downstream factors (Table 4). In this com-parison one can see the adherent cell culture with 50dose per year setting has on average no chance of beingcost effective because its mean is much higher than anyof the maximum costs. The ‘adherent 500’ setting canonly be cost effective with a 1% (or lower) discount rate,without immunosuppression and only at a less strictthreshold of $100,000. At that threshold and discountrate both the suspension cell culture settings can be costeffective without immunosuppression.Table 1 Credible ranges and fitted distributions for cost of goods upstreamProduction setting Range Gamma distributionCell culture Supply Lower bound Upper bound Mean RSD Shape RateAdherent 50 $21,300 $83,900 $47,443 34.00% 8.6505 0.0001823Adherent 500 $14,700 $73,800 $38,585 39.60% 6.3769 0.0001653Suspension 50 $16,900 $54,900 $33,193 29.42% 11.5535 0.0003481Suspension 500 $10,300 $53,100 $27,535 40.20% 6.1880 0.0002247Wallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 5 of 12Table2Resultsfordifferentscenariosusingadherentcellculture(meansperpatient)ScenarioCostBenefitICEREVPIMaximumPartialEVPIDoseCostsIndexProductionmodeSupplyperfacilityCOGdfactorRegulatoryfactorVariation(RSDa )StrategyDifferenceStrategyDifferenceWTPperQALY$50,000$100,000Scenarioswith3%discountrateComp1(Comparator3%)74,23011.12Adh1Local5041.222.5%629,181554,95113.852.73203,20318422090,957Adh2Local5041.250.0%628,936554,70713.852.73203,11467719,749135,128Adh3Local5041.822.5%876,810802,58013.852.73293,8772721143,704Adh4Local5041.850.0%873,510799,28113.852.73292,6691698061214,930Adh5Scaleoutlocal5031.222.5%504,903430,67313.852.73157,6978711,72569,691Adh6Scaleoutlocal5031.250.0%504,835430,60613.852.73157,673149332,911106,144Adh7Scaleoutlocal5031.822.5%690,050615,81913.852.73225,492112623102,737Adh8Scaleoutlocal5031.850.0%688,524614,29413.852.73224,93343215,297167,801Adh9Scaleoutlocal5081.822.5%1,616,3861,542,15613.852.73564,685019273,576Adh10Scaleoutlocal5081.850.0%1,606,9531,532,72213.852.73561,23191052443,892Adh11Largescale50041.222.5%536,915462,68513.852.73169,42012711,62178,153Adh12Largescale50041.250.0%536,730462,50113.852.73169,351150131,043124,247Adh13Largescale50041.822.5%738,478664,24813.852.73243,225243085117,352Adh14Largescale50041.850.0%736,541662,31113.852.73242,51649914,700192,416Adh15Scaleoutlarge50031.222.5%435,777361,54813.852.73132,38645324,79263,732Adh16Scaleoutlarge50031.250.0%435,661361,43213.852.73132,344300547,59196,481Adh17Scaleoutlarge50031.822.5%586,704512,47413.852.73187,65082814393,084Adh18Scaleoutlarge50031.850.0%585,166510,93613.852.73187,088111825,291148,572Scenarioswith0%discountrateComp2(Comparator0%)113,17516.09Adh19Local5041.222.5%663,514550,33920.604.51122,159139552,62090,906Adh20Scaleoutlarge50031.222.5%470,111356,93620.604.5179,23011,31530,54063,752Scenarioswith5%discountrateComp3(Comparator5%)58,5599.09Adh21Local5041.222.5%616,693558,13411.182.09267,339061490,973Adh22Scaleoutlarge50031.222.5%423,290364,73111.182.09174,70132639663,730Allscenariosusedthebasecaseassumptionswiththedescribedstructuraldeviations.CostmeasureisCanadiandollar(2016).BenefitmeasureisQALY.Allresultnumbersareroundedandincludingsamplingvariationa Relativestandarddeviation(RSD;i.e.SDaspercentageofthemean)thatwasassumedforthetwofactorsWallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 6 of 12Table3Resultsfordifferentscenariosusingsuspensioncellculture(meansperpatient)ScenarioCostBenefitICEREVPIMaximumPartialEVPIDoseCostsIndexProductionmodeSupplyperfacilityCOGdfactorRegulatoryfactorVariation(RSDa )StrategyDifferenceStrategyDifferenceWTPperQALY$50,000$100,000Scenarioswith3%discountrateComp1(Comparator3%)74,23011.12Sus1Local5041.222.5%480,575406,34613.852.73148,7905612,12659,158Sus2Local5041.250.0%479,911405,68013.852.73148,546154135,23292,836Sus3Local5041.822.5%654,137579,90613.852.73212,3424246488,524Sus4Local5041.850.0%651,401577,17113.852.73211,34045016,335141,768Sus5Scaleoutlocal5031.222.5%393,796319,56613.852.73117,01430528,62747,474Sus6Scaleoutlocal5031.250.0%393,094318,86413.852.73116,757321553,93776,931Sus7Scaleoutlocal5031.822.5%523,705449,47513.852.73164,58233808466,874Sus8Scaleoutlocal5031.850.0%521,437447,20713.852.73163,752108428,588113,389Sus9Scaleoutlocal5081.822.5%1,172,8781,098,64813.852.73402,287050169,848Sus10Scaleoutlocal5081.850.0%1,163,9741,089,74413.852.73399,026352719295,702Sus11Largescale50041.222.5%421,724347,49413.852.73127,24059028,37059,316Sus12Largescale50041.250.0%420,338346,10813.852.73126,733339951,26083,398Sus13Largescale50041.822.5%565,342491,11213.852.73179,828116978586,421Sus14Largescale50041.850.0%562,360488,13013.852.73178,736129427,942124,381Sus15Scaleoutlarge50031.222.5%349,649275,41913.852.73100,848166643,13647,205Sus16Scaleoutlarge50031.250.0%349,048274,81913.852.73100,629619264,31264,826Sus17Scaleoutlarge50031.822.5%457,207382,97713.852.73140,23238421,50567,653Sus18Scaleoutlarge50031.850.0%455,948381,71813.852.73139,772266943,714101,004Scenarioswith0%discountrateComp2(Comparator0%)113,17516.09Sus19Local5041.222.5%514,909401,73420.604.5189,173438940,83059,131Sus20Scaleoutlarge50031.222.5%383,981270,80820.604.5160,11126,451768447,205Scenarioswith5%discountrateComp3(Comparator5%)56,5589.09Sus21Local5041.222.5%468,087409,52911.182.09196,1601204259,162Sus22Scaleoutlarge50031.222.5%337,161278,60211.182.09133,44717216,41047,207Allscenariosusedthebasecaseassumptionswiththedescribedstructuraldeviations.CostmeasureisCanadiandollar(2016).BenefitmeasureisQALY.Allresultnumbersareroundedandincludingsamplingvariationa Relativestandarddeviation(RSD;i.e.SDaspercentageofthemean)thatwasassumedforthetwofactorsWallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 7 of 12Fig. 3 Expected value of perfect information results. Displayed are results of the value of further research for the scenarios using adherent (top)and suspension (bottom) cell culture techniques. All values are per patient calculations for willingness-to-pay thresholds of up to $200,000 peradditional QALY. One can see the different values between cell culture techniques and between production scenarios within each technique. Thedotted lines represent scenarios presuming an 80% increase of costs due to additional regulatory requirements compared to regulations currentlyin place. Scenarios Adh 20 and Sus 20 use a 0% discount rateTable 4 Full dose costs using example cost of goods downstream and regulator factorsProduction setting Factors Full dose costsaCell culture Supply Cost of goods downstream Regulatory Mean Lower range Upper rangeAdherent 50 3 1.2 $170,795 $76,680 $302,040Adherent 500 3 1.2 $138,906 $52,920 $265,680Suspension 50 3 1.2 $119,495 $60,840 $197,640Suspension 500 3 1.2 $99,126 $37,080 $191,160aMeans and values at the lower and upper 95% credible rangeWallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 8 of 12We report the full dose costs with the production set-tings we used for a set of example regulatory and cost ofgoods downstream factors (Table 4). In this comparisonone can see that our results point towards an increasedefficiency through a) high volume production, and b)use of adherent cell culture technique.DiscussionOur results show that the use of stem cells for beta cell re-placement therapy can be an effective use of health budgetfunds. Still, there is substantial uncertainty around thecosts of this technology. Both of those findings confirmthat methods of cost modeling combined with value-of-information analysis can be useful tools for aiding theprioritization. This especially applies to new healthcaretechnologies. Estimating the cost of transplant tissue wefound it possible for the treatment to be cost-effective atcommonly used cost-effectiveness thresholds if it greatlyreduces the need for immunosuppression. The value ofinformation as well as other results depend very much onthe assumptions in the respective scenarios. Those as-sumptions include transplantations costs, especially trans-plant tissue cost of goods and immunosuppression, aswell as discount rates.In near future stem cell therapy could be expanded toa much broader population of type 1 diabetes patients.Currently the expansion of beta cell replacement therapyin general is limited by organ supply and risks ofimmunosuppression. If outcomes were better than forislet transplantation, i.e. long term euglycemia and insu-lin independence, the lifetime costs of conventional ther-apy due to management of diabetes complications wouldbe avoided. That would include costs not considered inthis analysis which would fall outside of the budget ofthe provincial health care service, e.g. costs covered bythe federal health budget or costs for the patient’s familyor private insurance.Challenges of donor-harvested transplants in CanadaAdditional challenges of the donor-harvested approachin Canada could lead there to more readiness to adopt astem cell based therapy approach even with initiallyhigher costs. Among those challenges one needs to con-sider the relative shortage of organ donations, combinedwith great geographic distances between donors and theislet processing and transplantation site [44]. Thesefactors can lead to two kinds of costs of timely organtransport. The monetary costs are sometimes covered bydifferent regional health care services or air lines. Airtransport companies are known to occasionally shipdonor organs free of charge. Nevertheless non-monetarycosts are unavoidable. An example is the cost of organdeterioration from with progressive cold ischemia canmar graft yield later on.All those costs tend to be less for more densely popu-lated countries, or even regions with different organ dona-tion legislation, which can make a considerable differencein donor availability [45]. International coordination ofdonor organ availability could also further increase theefficient use available clinical resources. However, suchcoordination tends to come with substantial political andpractical complexities, which require further research butare beyond the topic of this study.On efficient treatment deliveryWhile high volume implant production can theoreticallybe cheaper one needs to weigh that with several consider-ations regarding demand, clinical capacity and other prac-tical limitations. One of those considerations is that stemcell-derived doses are currently as perishable as the donorderived cells. This means they have to be used withinabout 12–36 h of completion of the production process.The difference between stem cell derived tissue andharvested cells is here that one can determine the timewhen the tissue is ready. Instead of the cell dose cominginto the hospital more or less randomly at any time ofthe day or night, one can time the production process sothat the dose or doses arrive at the hospital at a prede-fined day and time of the day.In that way one can avoid the additional costs involvedin nightly or short notice transplantations. But graftdoses still have to be transplanted as quickly as possible.If several doses arrive at the same time it is also the casethat all need to be transplanted within a short period oftime. That could be accomplished if for example everyweek or every month 10 doses arrive at a hospital andare then all transplanted into 10 patients within thesame day.For that reason the number of lots produced per yearis important. Every time a lot is produced all the dosesof the lot have to be used within about one day or elsego waste. That is because currently it is not possible topreserve beta cell progenitors over long periods, e.g. viacryo-preservation. In this context, it is advantageousfrom an economic perspective to produce several lotsper year with smaller lot sizes, since it is impossible totransplant for example 500 doses in one day – even ifspread over 10 transplantation centers.Given the nature of the cells, transporting the patientsto a central location – as is done currently - might be abetter idea than transporting the cells to multiple patienthospitals across Canada. That is because transport of pa-tients may actually be more affordable than the sum of:a) the health lost through the certain quality loss in thehighly perishable cells through transport duration, b) themonetary costs from transporting the cells on a punctualjust-in-time basis, c) the costs of duplication of in-hospital infrastructure and staff training.Wallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 9 of 12LimitationsOngoing breakthroughs for example in current goodmanufacturing practice (cGMP) and mass cell expansionand limit the longevity of our estimates and modeling ef-forts. Breakthroughs include the genetic engineeringtechnique CRISPR (Clustered Regularly InterspacedShort Palindromic Repeat) and the use of modern biore-actors which aid various kinds of bioprocessing [46, 47].All those technologies have great influence on the cap-acities of researchers to generate new or more affordableways of producing transplantable tissue.We acknowledge that intensive insulin therapy as onlycomparator strategy to stem cell-based beta cell replace-ment therapy did limit the scope of our results. However,we consider the comparator and hypothetical patient co-hort in our model to be appropriate because of the patientpopulation under consideration. We explicitly limit ourstudy population to patients who: 1) do not have the de-gree of major comorbidities which would justify risks ofwhole organ transplantation, 2) are on intensive insulintreatment and 3) are candidates for islet transplantation.Still, we think further studies in a North American contextdo need to include donor-base islet transplantation as oneof the comparators. Other donor-based approaches to ad-dress type 1 diabetes could also be integrated.The cost of the semi-permeable membrane in whichthe cells are enclosed had to be estimated doe to lack ofdata. This and the fact that we did not change the followup costs and frequency compared to the study on islettransplantation are clear limitations of this study. How-ever, in light of the technology under consideration be-ing new and containing living cell tissue, the differencesbetween the actual and our estimated follow-up costsare likely smaller than for a less complex implant device.We expect that implantation would likely be anoutpatient procedure with much more limited riskscompared to islet transplantation, or even whole organtransplantation. In this study we presumed the newtreatment technology would still be require an in-patientprocedure including four days of hospital stay. Com-pared to that estimate, an out-patient procedurewould further reduce the costs of stem cell-basedbeta cell replacement therapy while increasing patientquality of life.One of the main goals of using re-extractable sheetsfor transplantation, instead of the standard cell injectioninto the liver, is to shield the cells from being attackedby the immune system. Since this is an early health tech-nology assessment of a very new technology we madethe assumption that this goal can be achieved withoutthe use of imunosuppressive drugs. If future develop-ments show that this is not the case then immunosup-pression would be necessary and with it would come theusual costs and side effects as mentioned elsewhere [20].ConclusionsUsing new grafts substantially increased the value of re-search into beta cell replacement therapy, especiallywhen also addressing the need for immunosuppression.Such a technology can improve treatment access andquality of life for patients through increased graft supplyand protection. Stem cell-based implants can be afeasible way of treating a wide range of patients withtype 1 diabetes.Additional fileAdditional file 1: 95% Confidence Interval of Results. We report the95% confidence interval for the costs and benefits of all our scenariosand the ICERs that were calculated from those values. (PDF 60 kb)AbbreviationscGMP: Current good manufacturing practice; COG: Cost of goods;CRISPR: Clustered Regularly Interspaced Short Palindromic Repeattechnology, a genome editing tool; EVPI: Expected value of perfectinformation; EVPPI: Expected value of partial perfect information; hEScells: Human embryonic stem cells; ICER: Incremental cost-effectiveness ratio;QALY: Quality-adjusted life-year; RSD: Relative standard deviation;SD: Standard deviation; Wtp: Willingness to pay per additional QALYAcknowledgementsNot applicable.FundingThis research was supported by grants from the Stem Cell Network, AlbertaInnovates Health Solutions (Collaborative Research and InnovationOpportunities) and salary support: Endowed Chair in Emergency MedicineResearch (CM), Faculty of Medicine & Dentistry at the University of Alberta(AMJS, CM, PAS), as well as the Canada Research Chairs Program (AMJS). Thefunders had no role in study design, data collection and analysis, decision topublish, or preparation of the manuscript.Availability of data and materialsThe datasets generated and/or analysed during the current study areavailable in the Open Science Framework repository (doi:10.17605/OSF.IO/JQJJZ and https://osf.io/jqjjz/). These datasets include the cost andeffectiveness results for each of the 64,000 model iterations, the mean resultsand the 95% confidence interval results. The latter are also availiable asadditional file.Authors’ contributionsKW and CM carried out data and cost-effectiveness and value-of-informationanalyses. RGP and JP performed the cost of goods analysis. KW, RGP and IAdrafted the manuscript. CM, KW and RGP participated in research design.CM, KW, RGP, IA, PAS, AMJS and JP guided the framing of the research ques-tion, provided methodological and clinical insight, and supported the datacollection. All authors had approval over the submitted manuscript and con-tributed substantially to its preparation.Ethics approval and consent to participateOur study meets the ethical standards implemented by the Research EthicsOffice of the University of Alberta, who approved the research proposalsconnected with this minimum-risk non-clinical study.Consent for publicationNot applicable.Competing interestsThe authors declare that they have no competing interests.Wallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 10 of 12Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.Author details1Department of Emergency Medicine Research Group, Department ofEmergency Medicine, University of Alberta, 8303 - 112 Street, Edmonton, ABT6G 2T4, Canada. 2Michael Smith Laboratories and Department of Chemical& Biological Engineering, University of British Columbia, 2185 East Mall,Vancouver, BC V6T 1Z4, Canada. 3Clinical Islet Transplant Program, AlbertaDiabetes Institute, University of Alberta, 2000 College Plaza, 8215 - 112 Street,Edmonton, AB T6G 2C8, Canada. 4Department of Medicine, University ofAlberta, Edmonton, Canada. 5Department of Surgery, University of Alberta,Edmonton, AB, Canada.Received: 24 April 2017 Accepted: 23 January 2018References1. Barton FB, Rickels MR, Alejandro R, Hering BJ, Wease S, Naziruddin B, et al.Improvement in outcomes of clinical islet transplantation: 1999-2010.Diabetes Care. 2012; https://doi.org/10.2337/dc12-0063.2. Senior PA, Kin T, Shapiro J, Koh A. Islet transplantation at the University ofAlberta: status update and review of progress over the last decade. Can JDiabetes Elsevier Ltd. 2012; https://doi.org/10.1016/j.jcjd.2012.01.002.3. Vantyghem M-C, Defrance F, Quintin D, Leroy C, Raverdi V, Prévost G, et al.Treating diabetes with islet transplantation: lessons from the past decade inLille. Diabetes Metab. 2014; https://doi.org/10.1016/j.diabet.2013.10.003.4. Chhabra P, Brayman KL. Overcoming barriers in clinical islet transplantation:current limitations and future prospects. Curr Probl Surg. 2014; https://doi.org/10.1067/j.cpsurg.2013.10.002.5. Kroon E, Martinson L, Kadoya K. Pancreatic endoderm derived from humanembryonic stem cells generates glucose-responsive insulin-secreting cells invivo. Nat Biotechnol. 2008; https://doi.org/10.1038/nbt1393.6. Schulz TC, Young HY, Agulnick AD, Babin MJ, Baetge EE, Bang AG, et al.A scalable system for production of functional pancreatic progenitors fromhuman embryonic stem cells. PLoS One. 2012; https://doi.org/10.1371/journal.pone.0037004.7. ClinicalTrials.gov. Identifier NCT02239354, A Safety, Tolerability, andEfficacy Study of VC-01TM Combination Product in Subjects With Type IDiabetes Mellitus. ClinicalTrials.gov. Bethesda (MD): National Library ofMedicine (US), 2016.8. Godfrey KJ, Mathew B, Bulman JC, Shah O, Clement S, Gallicano GI. Stemcell-based treatments for type 1 diabetes mellitus: bone marrow,embryonic, hepatic, pancreatic and induced pluripotent stem cells. DiabetMed. 2012; https://doi.org/10.1111/j.1464-5491.2011.03433.x.9. Schulz TC. Enabling Technologies for Cell-Based Clinical Translation ConciseReview: manufacturing of pancreatic endoderm cells for clinical trials intype 1 diabetes. Stem Cells Transl Med. 2015; https://doi.org/10.5966/sctm.2015-0058.10. Soria B, Gauthier BR, Martín F, Tejedo JR, Bedoya FJ, Rojas A, et al. Usingstem cells to produce insulin. Expert Opin Biol Ther. 2015; https://doi.org/10.1517/14712598.2015.1066330.11. Rezania A, Bruin JE, Arora P, Rubin A, Batushansky I, Asadi A, et al. Reversal ofdiabetes with insulin-producing cells derived in vitro from human pluripotentstem cells. Nat Biotechnol. 2014; https://doi.org/10.1038/nbt.3033.12. Pagliuca FW, Millman JR, Gürtler M, Segel M, Van Dervort A, Ryu JH, et al.Generation of functional human pancreatic β cells in vitro. Cell. 2014;https://doi.org/10.1016/j.cell.2014.09.040.13. Weir GC, Cavelti-Weder C, Bonner-Weir S. Stem cell approaches for diabetes:towards beta cell replacement. Genome Med. 2011; https://doi.org/10.1186/gm277.14. Desai T, Shea LD. Advances in islet encapsulation technologies. Nat RevDrug Discov. 2016; https://doi.org/10.1038/nrd.2016.232.15. Agulnick AD, Ambruzs DM, Moorman MA, Bhoumik A, Cesario RM, Payne JK,et al. Insulin-producing endocrine cells differentiated in vitro from humanembryonic stem cells function in macroencapsulation devices in vivo. StemCells Transl Med. 2015; https://doi.org/10.5966/sctm.2015-0079.16. Viacyte Inc. PEC-EncapTM (VC-01TM) – improving diabetes treatment.Viacyte Inc Available from: http://viacyte.com/products/pec%e2%80%90encap-vc-01. (Accessed 6 Jan 2017).17. Lee JE, Lee DR. Human embryonic stem cells: derivation, maintenance andcryopreservation. Int J Stem Cells. 2011;4:9–17.18. Karimi-Busheri F, Rasouli-Nia A, Weinfeld M. Key issues related tocryopreservation and storage of stem cells and cancer stem cells:protecting biological integrity. In: Karimi-Busheri F, Weinfeld M, editors.Biobanking and cryopreservation of stem cells. Cham: SpringerInternational Publishing; 2016.19. Hourd P, Chandra A, Medcalf N, Williams DJ. Regulatory challenges for themanufacture and scale-out of autologous cell therapies (June 30, 2014).In: The Stem Cell Research Community, editor. StemBook. StemBook, 2014.20. Wallner K, Shapiro AMJ, Senior PA, Mccabe C. Cost effectiveness and valueof information analyses of islet cell transplantation in the management of“ unstable ” type 1 diabetes mellitus. BMC Endocr Disord. 2016; https://doi.org/10.1186/s12902-016-0097-7.21. Simaria AS, Hassan S, Varadaraju H, Rowley J, Warren K, Vanek P, et al.Allogeneic cell therapy bioprocess economics and optimization: single-usecell expansion technologies. Biotechnol Bioeng. 2014; https://doi.org/10.1002/bit.25008.22. Statistics Canada. Table 326-0021 - consumer price index. Statistics Canada.Available from: http://www5.statcan.gc.ca/cansim/a26?lang=eng&retrLang=eng&id=3260021&pattern=&csid=. (Accessed 10 Jan 2017).23. Statistics Canada. Consumer price index, by province (monthly). StatisticsCanada Available from: http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/cpis01a-eng.htm. (Accessed 10 Jan 2017).24. Peasgood T, Brennan A, Mansell P, Elliott J, Basarir H, Kruger J. The impact ofdiabetes-related complications on preference-based measures of health-related quality of life in adults with type I diabetes. Med Decis Mak. 2016;https://doi.org/10.1177/0272989X16658660.25. O’Reilly D, Hopkins R, Blackhouse G, Clarke P, Hux J, Guan J, et al.Development of an Ontario diabetes economic model (ODEM) andapplication to a multidisciplinary primary care diabetes managementprogram. (Report prepared for the Ontario Ministry of Health and Long-term Care). Hamilton, Ontario: Program for Assessment of Technology inHealth (PATH); 2006.26. Clarke P, Gray A, Holman R. Estimating utility values for health states of type2 diabetic patients using the EQ-5D (UKPDS 62). Med Decis Mak. 2002;https://doi.org/10.1177/0272989X0202200412.27. Kirsch J, McGuire A. Establishing health state valuations for disease specificstates: an example from heart disease. Health Econ. 2000; https://doi.org/10.1002/(SICI)1099-1050(200003)9:2<149::AID-HEC501>3.0.CO;2-N.28. Tengs TO, Wallace A. One thousand health-related quality-of-life estimates.Med Care. 2000;38:583–637.29. Palmer AJ, Roze S, Valentine WJ, Minshall ME, Foos V, Lurati FM, et al. TheCORE diabetes model: projecting long-term clinical outcomes, costs andcost-effectiveness of interventions in diabetes mellitus (types 1 and 2) tosupport clinical and reimbursement decision-making. Curr Med Res Opin.2004; https://doi.org/10.1185/030079904X1980.30. Canadian Agency for Drugs and Technologies in Health (CADTH).Guidelines for the economic evaluation of health technologies. 3rd ed.Ottawa: Canadian Agency for Drugs and Technologies in Health; 2006.31. Briggs A, Sculpher M, Claxton K. Decision modelling for health economicevaluation. Handbooks in health economic evaluation. 1st ed. New Yorkand Oxford: Oxford University Press; 2006.32. Edlin R, McCabe C, Hulme C, Hall P, Wright J. Cost effectiveness modellingfor health technology assessment: a practical course. 1st ed. Heidelberg,New York, Dordrecht, London, Adis; 2015. http://www.springer.com/gp/book/9783319157436.33. Institute of Health Economics. Islet transplantation for the treatment of type1 diabetes. Edmonton, AB: Institute of Health Economics; 2013.34. Hassan S, Simaria AS, Varadaraju H, Siddharth G, Warren K, Farid SS, et al.Allogeneic cell therapy bioprocess economics and optimization:downstream processing decisions. Regen Med. 2015; https://doi.org/10.2217/rme.15.75.35. Canadian Institute for Health Information. Canadian Organ ReplacementRegister Annual Report: Treatment of End-Stage Organ Failure inCanada, 2003 to 2012. Canadian Organ Replacement Register AnnualReport. 2014.36. Canadian Institute for Health Information. Appendix B — CanadianTransplant Hospitals, Renal Programs and Independent Health FacilitiesProviding Dialysis to Chronic Renal Failure Patients as Reported to CORR.Canadian Organ Replacement Register Annual Report. 2015.Wallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 11 of 1237. Daneman D. Type 1 diabetes. Lancet. 2006. https://doi.org/10.1016/S0140-6736(06)68341-4.38. Canadian Diabetes Association. The cost of diabetes in Alberta. CanadianDiabetes Association. Available from: http://www.diabetes.ca/CDA/media/documents/publications-and-newsletters/advocacy-reports/cost-of-diabetes-in-alberta.pdf.39. Merani S, Shapiro JAM. Current status of pancreatic islet transplantation.Clin Sci (Lond). 2006; https://doi.org/10.1042/CS20050342.40. Cryer PE. The barrier of hypoglycemia in diabetes. Diabetes. 2008; https://doi.org/10.2337/db08-1084.41. Skrivarhaug T, Bangstad H-J, Stene L. Long-term mortality in a nationwidecohort of childhood-onset type 1 diabetic patients in Norway. Diabetologia.2006; https://doi.org/10.1007/s00125-005-0082-6.42. Statistics Canada. Population by year, by province and territory (Number).Statistics Canada, CANSIM, table 051–0001. Available from: http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/demo02a-eng.htm.(Accessed 10 Jan 2017).43. United Nations, Department of Economic and Social Affairs, PopulationDivision. World Population Prospects: The 2015 Revision, Key Findings andAdvance Tables. 2015. Working Paper No. ESA/P/WP.241. Available from:https://esa.un.org/unpd/wpp/Publications/Files/Key_Findings_WPP_2015.pdf. (Accessed 10 Jan 2017).44. Canadian Institute for Health Information (CIHI). Organ donations continueto fall short of meeting demand. CIHI. Available from: https://www.cihi.ca/en/types-of-care/specialized-services/organ-replacements/organ-donations-continue-to-fall-short-of. (Accessed 6 Jan 2017).45. Abadie A, Gay S. The impact of presumed consent legislation on cadavericorgan donation: a cross-country study. J Health Econ. 2006. https://doi.org/10.1016/j.jhealeco.2006.01.003.46. Pak E. CRISPR: a game-changing genetic engineering technique. Science inthe news. Harvard University Available from: http://sitn.hms.harvard.edu/flash/2014/crispr-a-game-changing-genetic-engineering-technique/.Accessed 16 Mar 201747. International Union of Pure and Applied Chemistry (IUPAC). In: AD MN,Wilkinson A, editors. Compendium of Chemical Terminology, (the “GoldBook”). 2nd ed. Oxford: Blackwell Scientific Publications; 1997. XML on-linecorrected version: http://goldbook.iupac.org (2006-) created by M. Nic, J.Jirat, B. Kosata; updates compiled by A. Jenkins. https://doi.org/10.1351/goldbook.•  We accept pre-submission inquiries •  Our selector tool helps you to find the most relevant journal•  We provide round the clock customer support •  Convenient online submission•  Thorough peer review•  Inclusion in PubMed and all major indexing services •  Maximum visibility for your researchSubmit your manuscript atwww.biomedcentral.com/submitSubmit your next manuscript to BioMed Central and we will help you at every step:Wallner et al. BMC Endocrine Disorders  (2018) 18:6 Page 12 of 12

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.52383.1-0363335/manifest

Comment

Related Items