UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Development and application of a whole disease model of oral cancer to inform health technology management Cromwell, Ian 2019

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2019_september_cromwell_ian.pdf [ 4.77MB ]
Metadata
JSON: 24-1.0379922.json
JSON-LD: 24-1.0379922-ld.json
RDF/XML (Pretty): 24-1.0379922-rdf.xml
RDF/JSON: 24-1.0379922-rdf.json
Turtle: 24-1.0379922-turtle.txt
N-Triples: 24-1.0379922-rdf-ntriples.txt
Original Record: 24-1.0379922-source.json
Full Text
24-1.0379922-fulltext.txt
Citation
24-1.0379922.ris

Full Text

DEVELOPMENT AND APPLICATION OF A WHOLE DISEASE MODEL OF ORAL CANCER TO INFORM HEALTH TECHNOLOGY MANAGEMENT  by Ian Cromwell A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in The Faculty of Graduate and Postdoctoral Studies (Population and Public Health) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) July 2019 © Ian Cromwell, 2019  ii  The following individuals certify that they have read, and recommended to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled:  Development and application of a whole disease model of oral cancer to inform health technology management Submitted by Ian Cromwell in partial fulfillment of the requirements for  The degree of  Doctor of Philosophy in   Population and Public Health  Examining committee: Nicholas J Bansback, Population and Public Health Supervisor  Stuart J Peacock, Faculty of Health Sciences, Simon Fraser University Supervisory Committee Member Gregory Werker, UBC Sauder School of Business Supervisory Committee Member Fawziah Lalji, Faculty of Pharmaceutical Sciences University Examiner Michael Law, School of Population and Public Health University Examiner Glenn Salkeld, Faculty of Social Sciences, University of Wollongong, Australia External Examiner Additional Supervisory Committee Members: Charlyn Black, Population and Public Health Supervisory Committee Member  Catherine F Poh, Faculty of Dentistry Supervisory Committee Member      iii  Abstract Whole Disease Models (WDMs) are decision analytic models characterized by their ability to reflect the policy changes that occur at multiple points within the entire clinical trajectory of a given disease. They differ from conventional ‘piecewise’ modeling approaches in their ability to reflect processes that occur ‘upstream’ and ‘downstream’ from a technology decision of interest. This dissertation describes the development of a WDM of oral cancer, and its application in generating evidence to inform Health Technology Management (HTM). The dissertation reviews the available scientific literature concerning health economic decision analytic modeling in oral cancer, and argues that a Whole Disease Model approach is appropriate for economic evaluation in this disease. A conventional piecewise Markov model is used to evaluate the cost-effectiveness of risk-guided management of oral premalignancy, and the limitations of that approach are discussed. The dissertation then describes the development and validation of the Whole Disease Model of Oral Cancer (WDMOC). The WDMOC is used to re-evaluate the risk-guided management policy, and how the cost-effectiveness of such a policy is influenced by upstream (tobacco/alcohol cessation, improved screening) and downstream (improved surgical treatment for early-stage disease, improved systemic therapy for late-stage disease) policy changes, including the hypothetical effect of a population HPV vaccination program. The WDMOC found that risk-guided management was cost saving compared to current standard practice, but was not expected to produce gains in quality-adjusted life years (QALYs). The cost-effectiveness of a risk-guided management approach was affected by upstream factors that influence malignant progression and downstream factors that prolonged survival among advanced cancers. Scenario analysis was used to estimate the impact of multiple simultaneous policy changes on the cost-effectiveness of a risk-guided approach. The WDMOC contributes a useful platform for economic evaluation that can inform HTM. Results of the analysis suggest that a risk-guided approach is cost-effective, particularly among patients with regular access to a dentist that regularly performs oral cancer screenings and in the presence of improved options for managing late-stage disease. The WDMOC was developed using an open source approach so that it can readily incorporate new information and have users in multiple policy jurisdictions.   iv  Lay Summary Health economists use decision analytic models to estimate the cost-effectiveness of new technologies in health. There are many such emerging technologies in oral cancer, but the models and evidence that exist are not sufficient to examine the impact of introducing them all. This dissertation describes the creation of a Whole Disease Model of Oral Cancer (WDMOC), which considers all parts of the disease trajectory from preclinical disease to terminal illness. The WDMOC is used to examine a new tool that helps predict a person’s risk of developing cancer, and the results are compared to a standard modeling approach. The WDMOC is then used to evaluate the impact that other policy changes across multiple parts of the health care system have on the assay’s cost-effectiveness. The WDMOC is designed to help inform evidence-based policy decisions, and can be easily updated and shared.   v  Preface Chapter 2 represents original work from the candidate. I was responsible for the design, data acquisition and analysis of the literature review. Dr. Bansback worked in a direct supervisory capacity throughout the writing of this chapter. Chapter 3 is derived from work conducted during the initial phases of my pursuit of this degree. This work has been published: Cromwell, I., Regier, D., Peacock, S.J., Poh, C.F. (2016) Cost-Effectiveness Analysis of Using Loss of Heterozygosity to Manage Premalignant Oral Dysplasia in British Columbia, Canada. Oral Oncology. 21(9): 1099-1106. I designed the Markov model and wrote most of the manuscript. Drs. Peacock and Poh were responsible for methodological, clinical, and other scientific oversight. Dr. Regier worked in a direct supervisory capacity with the candidate during the design, analysis, and interpretation of the Markov model. Chapter 4 is comprised of work that was conducted by the candidate, and of secondary analysis of data collected by others. I designed the model, conducted the expert interviews, wrote and tested the code, performed the statistical analyses, and conducted the calibration. Dr. Bansback worked in a direct supervisory capacity throughout the writing of this chapter. Dr. Poh helped the candidate assemble the expert stakeholder group, provided ongoing clinical feedback during the model design and parameterization, and assisted in the acquisition of de-identified versions of the administrative and trial datasets analyzed for parameter estimation. Dr. Werker provided input on the model design, coding, and validation/calibration, as well as input on the chapter’s structure. Chapter 5 is comprised of work that was conducted by the candidate. I conducted the analysis and comparison between models, as well as the structurally adaptation and re-analysis of primary and secondary outcome data. Dr. Bansback worked in a direct supervisory capacity throughout the writing of this chapter. Chapter 6 is comprised of work that was conducted by the candidate. I designed the scenarios with direct input from the supervisory committee. I performed the literature searches to identify parameters and conducted the cost-effectiveness analysis. Dr. Bansback worked in a direct supervisory capacity throughout the writing of this chapter. The content of all chapters represents the work of the candidate with the oversight, input, and feedback of Dr. Bansback and all members of the supervisory committee. This dissertation received institutional ethics approval from UBC (Certificate # H15-03123).   vi  Table of Contents Abstract ............................................................................................................................................ iii Lay Summary .................................................................................................................................... iv Preface .............................................................................................................................................. v Table of Contents .............................................................................................................................. vi List of Tables ................................................................................................................................... xiii List of Figures ................................................................................................................................... xv List of Abbreviations ....................................................................................................................... xvi Acknowledgments ........................................................................................................................ xviii 1. Chapter 1 - Introduction ..............................................................................................................1 1.1 Health Economics .......................................................................................................................... 1 1.2 Health Technology Assessment and Management ....................................................................... 3 1.3 Economic Evaluation ..................................................................................................................... 5 1.4 Decision Modeling in Economic Evaluation .................................................................................. 8 1.5 Types of Decision Analytic Models ............................................................................................. 10 1.5.1 Decision Trees ..................................................................................................................... 10 1.5.2 Cohort-Level State Transition Models ................................................................................ 11 1.5.3 Discrete Event Models ........................................................................................................ 12 1.6 The ‘Whole Disease Model’ Theoretical Framework .................................................................. 15 1.7 Case Study: Health Technology Management for Oral Cancer................................................... 17 1.7.1 Oral Cancer .......................................................................................................................... 17 1.7.2 Medical Management of Oral Cancers ............................................................................... 18 1.7.3 Potential Health Technologies in Oral Cancer Management .............................................. 20 1.8 A Whole Disease Model of Oral Cancer (WDMOC) .................................................................... 21 1.9 Objectives of This Dissertation ................................................................................................... 22 vii  2 Chapter 2 – Literature Review ................................................................................................... 25 2.1 Introduction ................................................................................................................................ 25 2.2 Oral Cancer Development, Progression, and Management ....................................................... 25 2.2.1 Premalignant Disease and Progression to Cancer .............................................................. 25 2.2.2 Detection of Premalignant and Malignant Disease ............................................................ 27 2.2.3 Treatment of Malignant Oral Cancer .................................................................................. 28 2.2.4 Disease Recurrence and Management ............................................................................... 29 2.2.5 Quality of Life and Health State Utility ............................................................................... 29 2.3 Review of Health Economics Literature in Oral Cancer .............................................................. 32 2.3.1 Search Strategy ................................................................................................................... 32 2.3.2 Results from Literature Review ........................................................................................... 33 2.4 Structure of Models of Oral Cancer ............................................................................................ 39 2.5 Potential Emerging Health Technologies in Oral Cancer ............................................................ 41 2.6 Discussion .................................................................................................................................... 43 2.7 Conclusion ................................................................................................................................... 44 3 Chapter 3 – Cost-Effectiveness Analysis of Genome-Guided Management of OPLs using Markov Decision Modeling .................................................................................................................... 45 3.1 Introduction ................................................................................................................................ 45 3.2 Methods ...................................................................................................................................... 46 3.2.1 Markov Model ..................................................................................................................... 47 3.2.2 Transition probabilities ....................................................................................................... 49 3.2.3 Costs .................................................................................................................................... 50 3.2.4 Health State Utilities ........................................................................................................... 51 3.2.5 Cost-Effectiveness Analysis ................................................................................................. 53 3.2.6 Probabilistic Analysis ........................................................................................................... 54 viii  3.3 Results ......................................................................................................................................... 55 3.3.1 Cost ..................................................................................................................................... 55 3.3.2 Effectiveness ....................................................................................................................... 56 3.3.3 Cost-Effectiveness ............................................................................................................... 56 3.4 Discussion .................................................................................................................................... 60 3.4.1 Limitations ........................................................................................................................... 61 3.5 Conclusion ................................................................................................................................... 62 4 Chapter 4 – Design and Implementation of a Whole Disease Model of Oral Cancer ..................... 64 4.1 Introduction ................................................................................................................................ 64 4.2 Model Design and Theoretical Framework ................................................................................. 65 4.3 Model Design process ................................................................................................................. 66 4.3.1 Stage 1: Understanding the Decision Problem ................................................................... 67 4.3.2 Stage 2: Conceptualization and Design ............................................................................... 69 4.4 Conceptual Model of the WDMOC ............................................................................................. 73 4.4.1 Natural History Model......................................................................................................... 74 4.4.2 Clinical Trajectory Model .................................................................................................... 77 4.4.3 Screening/Asymptomatic Component ................................................................................ 77 4.4.4 Oral Premalignant Lesion Component ................................................................................ 79 4.4.5 Incident Cancer Component ............................................................................................... 81 4.4.6 Follow-up Component ........................................................................................................ 83 4.4.7 Terminal Disease Component ............................................................................................. 85 4.5 Stage 3: Implementation Modeling ............................................................................................ 86 4.5.1 Approach to Simulation Modeling ...................................................................................... 87 4.5.2 Time-To-Event Processing in the WDMOC ......................................................................... 89 4.5.3 A Description of Model Functions ....................................................................................... 92 ix  4.6 Model Parameterization ............................................................................................................. 93 4.6.1 Entity Creation .................................................................................................................... 95 4.6.2 Natural History .................................................................................................................... 97 4.6.3 Screening/Asymptomatic Component .............................................................................. 100 4.6.4 Oral Premalignancy Component ....................................................................................... 102 4.6.5 Incident Cancer Component ............................................................................................. 103 4.6.6 Follow-up Component ...................................................................................................... 105 4.6.7 Terminal Disease Component ........................................................................................... 106 4.6.8 Unit Costs .......................................................................................................................... 108 4.6.9 Health State Utilities ......................................................................................................... 110 4.7 Derivation of Parameter Estimates from Data Sources ............................................................ 113 4.8 Verification, Validation, and Calibration of the WDMOC ......................................................... 118 4.9 Univariate Sensitivity Analysis .................................................................................................. 123 4.10 Discussion .................................................................................................................................. 127 4.10.1 Strengths of the approach ................................................................................................ 127 4.10.2 Limitations of the approach .............................................................................................. 128 4.11 Conclusion ................................................................................................................................. 131 5 Chapter 5 – Case Study Comparison of WDMOC to Conventional (Markov) Modeling ............... 132 5.1 Introduction .............................................................................................................................. 132 5.2 Methods .................................................................................................................................... 133 5.2.1 Model Logic ....................................................................................................................... 133 5.2.2 Parameter Estimate Data Sources .................................................................................... 135 5.2.3 Cost-Effectiveness Analysis ............................................................................................... 136 5.2.4 Secondary Model Outputs ................................................................................................ 137 5.2.5 Scenario Analysis ............................................................................................................... 138 x  5.3 Results ....................................................................................................................................... 138 5.3.1 Cost-Effectiveness Results ................................................................................................ 138 5.3.2 Secondary Outcomes ........................................................................................................ 141 5.3.3 Scenario Analysis ............................................................................................................... 142 5.4 Discussion .................................................................................................................................. 143 5.4.1 Comparison to Chapter 3 .................................................................................................. 143 5.4.2 Limitations ......................................................................................................................... 145 5.5 Conclusion ................................................................................................................................. 146 6 Chapter 6 – Impact of Upstream/Downstream Changes to Decision Environment on Cost-Effectiveness of Risk-Guided OPL Management ....................................................................... 148 6.1 Introduction .............................................................................................................................. 148 6.2 Methods .................................................................................................................................... 149 6.2.1 Description of decision problem ....................................................................................... 149 6.2.2 Scenario Analysis ............................................................................................................... 150 6.2.3 Whole Disease Model of Oral Cancer (WDMOC) .............................................................. 152 6.2.4 Scenario Creation in the WDMOC ..................................................................................... 153 6.2.5 Cost-Effectiveness Analysis ............................................................................................... 162 6.3 Results ....................................................................................................................................... 164 6.4 Discussion .................................................................................................................................. 168 6.4.1 Limitations ......................................................................................................................... 170 6.4.2 Implications of This Chapter ............................................................................................. 172 6.5 Conclusion ................................................................................................................................. 173 7 Chapter 7 – Discussion ............................................................................................................ 175 7.1 Research Objectives and Key Findings of This Dissertation ...................................................... 175 xi  7.1.1 Determine The Current State of the Literature Concerning the Health Economics of Oral Cancer and to Identify Specific Knowledge Gaps in Technologies That Are Suitable for Economic Evaluation. ........................................................................................................................................ 175 7.1.2 Illustrate The Limitations of a ‘Piecewise’ Modeling Approach When Evaluating Health Technologies. .................................................................................................................................... 176 7.1.3 Design, Implement, And Validate a Model That Can Address the Shortcomings of the Conventional Approach. ................................................................................................................... 177 7.1.4 Compare the Conventional Markov Modeling Approach with the WDMOC Approach. .. 178 7.1.5 Use the WDMOC to Evaluate the Impact That ‘Upstream’ Parameter Changes Have on the Cost-Effectiveness and Budgetary Impact of ‘Downstream’ Technology Adoption. ........................ 178 7.2 Contributions to the Literature ................................................................................................. 179 7.2.1 Economic Evaluation Methods ......................................................................................... 180 7.2.2 Knowledge in Economic Evaluations of Oral Cancer ........................................................ 181 7.3 Strengths of This Dissertation ................................................................................................... 182 7.3.1 Whole Disease Models Versus Conventional Piecewise Models ...................................... 182 7.3.2 Individual Sampling Versus Markov Models ..................................................................... 183 7.3.3 Design and Structure of the WDMOC ............................................................................... 185 7.4 Limitations................................................................................................................................. 186 7.4.1 Use of OPL Prevalence ...................................................................................................... 186 7.4.2 Second-Order Uncertainty Among Regression-Based Parameters .................................. 188 7.4.3 Data Availability ................................................................................................................ 189 7.5 Implications for Health Policy and Economic Evaluation Research .......................................... 190 7.6 Reflections on the WDM Framework ....................................................................................... 191 7.7 Areas for Future Research ........................................................................................................ 194 7.7.1 Improvement to the Analysis ............................................................................................ 194 7.7.2 Further Applications of This Work .................................................................................... 195 7.7.3 Dissemination and Knowledge Translation ....................................................................... 196 xii  7.8 Conclusion ................................................................................................................................. 197 Works Cited ................................................................................................................................... 199 Appendices .................................................................................................................................... 216 Appendix A – Literature Review Search Strategy ................................................................................. 216 Appendix B – Names and specializations of expert stakeholder group................................................ 217 Appendix C – Initial Model Structure .................................................................................................... 218 Appendix D – Strucutral elements of a discrete event simulation (DES) .............................................. 223 Appendix E – Estimating time to progression from OPL ....................................................................... 225 Appendix F – Parameter estimation from the Retrospective Oral Cancer Cohort (ROCC) ................... 228 Appendix G – Time-to-event processing in the WDMOC ..................................................................... 244 Appendix H – Cost derivation from retrospective oral cancer cohort (ROCC) ..................................... 248 Appendix I – Comparison of Cycle-based vs. Discrete Time in decision analysis models ..................... 260 Appendix J – Structural comparison of WDMOC to conventional (Markov) model ............................. 262    xiii  List of Tables  Table 2.1 – Summary of Head & Neck Cancer studies reporting EQ-5D values from Meregaglia and Cairns (2017) review .............................................................................................................................................. 31 Table 2.2 – Health Economic exercises in oral cancer from systematic literature review ......................... 35 Table 2.3 – Characteristics of decision models used in economic evaluations of oral cancer ................... 36 Table 3.1 – Parameter inputs used in the Markov model .......................................................................... 52 Table 3.2 – Summary of Cost-Effectiveness results .................................................................................... 55 Table 4.1 – Parameter estimates: entity creation ...................................................................................... 96 Table 4.2 – Parameter estimates: natural history ...................................................................................... 98 Table 4.3 – Parameter estimates: asymptomatic/screening component ................................................ 100 Table 4.4 – Parameter estimates: oral premalignancy component ......................................................... 102 Table 4.5 – Parameter estimates: incident cancer component ................................................................ 104 Table 4.6 – Parameter estimates: follow-up component ......................................................................... 106 Table 4.7 – Parameter estimates: terminal disease component .............................................................. 107 Table 4.8 – Parameter estimates: unit costs ............................................................................................ 109 Table 4.9 – Parameter estimates: health state utilities ............................................................................ 111 Table 4.10 – Calibration output target values .......................................................................................... 121 Table 4.11 – WDMOC baseline calibration outputs .................................................................................. 122 Table 4.12 – Baseline outputs for sensitivity analysis............................................................................... 124 Table 4.13 – Univariate sensitivity results by model component ............................................................. 125 Table 5.1 – Cost-effectiveness analysis results ......................................................................................... 139 xiv  Table 5.2 – Comparison of Secondary Model Outputs from a simulated cohort of 100,000 OPL patients .................................................................................................................................................................. 141 Table 6.1 – Parameter estimates used in scenario analysis ..................................................................... 161 Table 6.2 – Baseline cost-effectiveness of using the LOH assay, population and OPL+ perspectives ...... 164 Table 6.3 – Cost-effectiveness of using the LOH assay in conjunction with other policy changes........... 165 Table 6.4 – Policy changes that produce the ‘best’ and ‘worst’ cost-effectiveness values for using the LOH assay .................................................................................................................................................. 166   xv  List of Figures  Figure 1.1 - Conceptual Diagram of Questions Addressed Within Health Economics ................................. 2 Figure 2.1 - Systematic Literature Review PRISMA Diagram ...................................................................... 34 Figure 2.2 - Conceptual model of disease and decision nodes ................................................................... 40 Figure 3.1  – Structure of the Markov Model ............................................................................................. 48 Figure 3.2 – Proportion of costs attributable to each model state ............................................................ 56 Figure 3.3 – Probabilistic Analysis: incremental results from 10,000 model runs ...................................... 58 Figure 3.4 – Cost-Effectiveness Plane for A) cost per LYG and B) cost per QA ........................................... 59 Figure 4.1 – The Natural History model component .................................................................................. 76 Figure 4.2 – Screening/Asymptomatic model component ......................................................................... 78 Figure 4.3 – Oral Premalignant Lesion management model component ................................................... 80 Figure 4.4 – Incident cancer model component ......................................................................................... 82 Figure 4.5 – Cancer follow-up model component ...................................................................................... 84 Figure 4.6 – Terminal disease management component ........................................................................... 85 Figure 4.7 – Summary of simulation model implementation ..................................................................... 90 Figure 5.1 – Cost-effectiveness plane and cost-effectiveness acceptability curve of 10,000 bootstrapped values of cost and quality-adjusted survival ............................................................................................. 140 Figure 6.1 – Net Monetary Benefit of using LOH assay in OPL+ patients among different scenario combinations ............................................................................................................................................ 167   xvi  List of Abbreviations  BCCA BC Cancer BIA Budgetary Impact Analysis CADTH Canadian Association for Drugs and Technologies in Health CAIS BC Cancer Agency Information System CBA Cost-Benefit Analysis CEA Cost-Effectiveness Analysis CEAC Cost-Effectiveness Acceptability Curve COE Conventional Oral Examination COOLS Trial pan-Canadian Optically-guided Oral Lesions Surgical Trial COPD Chronic Obstructive Pulmonary Disease CT Computed Tomography CUA Cost-Utility Analysis EOL End Of Life GLM Generalized Linear Model HGL High-Grade Lesion HPV Human Papillomavirus HRQoL; QoL Health-Related Quality of Life; Quality of Life HST Health State Transition HTA Health Technology Assessment HTM Health Technology Management ICER Incremental Cost-Effectiveness Ratio IHEA International Health Economics Association ISPOR International Society for Pharmacoeconomics and Outcomes Research LOH Loss of Heterozygosticity LYG Life Years Gained M-H Metropolis-Hastings MRI Magnetic Resonance Imaging MSP Medical Services Plan NCCN National Comprehensive Cancer Network NHS National Health Service NMB Net Monetary Benefit NTD Non-Time-Dependent OCPP Oral Cancer Prevention Program OOC Oral Oncology Clinic OPL Oral Premalignant Lesion PEC Priorities and Evaluation Committee PHSA Provincial Health Services Authority QALY Quality-Adjusted Life Years ROCC Retrospective Oral Cancer Cohort RT Radiotherapy SCC; OSCC Squamous Cell Carcinoma; Oral Squamous Cell Carcinoma TAC Tobacco and Alcohol Cessation xvii  TTE Time To Event WDM Whole Disease Model WDMOC Whole Disease Model of Oral Cancer WTP; λ Willingness to Pay XRT External-beam Radiotherapy    xviii  Acknowledgments To my parents Andrew and Nina, for the years of steadfast support and sacrifice that made any of this possible, and for being a constant source of encouragement. As much as I put myself into this work, the person I was before I started was moulded, shaped, and strengthened by your love and guidance. To Bethany McElvaine, Aedan Davis, and Maria Lorenzi. Aside from me, nobody else lived and breathed this thesis, and directly felt the consequences of that, more than you did. I don’t know if I could have done this without you, but I’m eternally grateful that I didn’t have to. To my committee, Dr. Nick Bansback, Dr. Stuart Peacock, Dr. Catherine Poh, Dr. Gregory Werker, and Dr. Charlyn Black. Your guidance, expertise, and insight was crucial in helping me understand not only what I needed to do to achieve this milestone, but also what I needed to avoid doing. To my colleagues at the Canadian Centre for Applied Research in Cancer Control and at BC Cancer. For giving me advice, for helping me troubleshoot, for being a receptive ear to my various thesis-related rants, and for tolerating my various thesis-related moods. I couldn’t have asked for a more brilliant or more supportive group of people to work with. To Stavros Korokithakis, who held my hand through the entire coding process. Anything in my code that is particularly clever or well-thought-through almost certainly has Stavros to blame. To Kelly Liu, who did the painstaking work of conducting the chart review that informs huge parts of this thesis and is single-handedly responsible for shortening the timeline of this work by at least 6 months. It is also important to acknowledge that this work is made possible by the use of medical records from real people whose lives, pain, joys, and deaths are reflected in the model parameters. I hope that  To Huiying Sun, whose statistical expertise is built into the basic functioning of this model, thank you for walking me through the mechanics of turning statistical theory into actual code. To my friends, who have stayed by my side and supported me even when doing this work turned me into a really crappy friend who couldn’t be there for you as often as I needed to be. To the Vancouver music scene – the musicians, the friends, the colleagues, the fans – who have made this city feel like home. This would have been a much more miserable journey without you. To all these people, and the countless others who have offered words of support, advice, commiseration, and sympathy. I share this accomplishment with you. But I’m the only one who gets to put the letters after his name, so…  1  1. Chapter 1 - Introduction This dissertation seeks to develop a Whole Disease Model for oral cancer that will inform policy makers how to best manage health technologies ranging from screening strategies to novel therapeutic approaches that improve patient’s health. This first chapter seeks to introduce the health economics and health technology assessment disciplines that this dissertation is embedded upon. It then introduces oral cancer and the technologies that are being developed for its detection and treatment. And finally explains the relevance for a whole disease modelling approach, which sets up the objectives of the dissertation. 1.1 Health Economics Health economics is a branch of scientific inquiry concerned with the quantification and analysis of decisions relating to scarce resources as they relate to health and health care. It is concerned with understanding and predicting how decisions – policy decisions, funding decisions, behavioural decisions – affect human health. Williams defines eight characteristic questions within the field of health economics[1]: 2  Figure 1.1 - Conceptual Diagram of Questions Addressed Within Health Economics  Public domain image, adapted from[1] A seminal essay by economist Kenneth J Arrow is commonly identified as the foundation for the field of health economics[2]. In this essay, Arrow explored the limitations of classical economic theory as they relate to health and health care as ‘goods’ – something that satisfies some need or desire, and that can be exchanged for other goods in a rational and fair way. Arrow contended that factors like information asymmetry between ‘consumers’ and ‘providers’ of health care, and unresolvable uncertainty violate fundamental assumptions of economic theory, making health and health care importantly distinct from other types of goods. Arrow argued that these differences require that health care be understood and studied as a distinct set of concepts and goods, using a distinct set of methods. Health economics is grounded within economic theories that are, in turn, based in the philosophical theories of utilitarianism[3]. Health economists seek to maximize the wellbeing derived from those goods, in terms of both human health and resource investment. Utility theorists axiomatically assume that individuals are the best judge of their own well-being, and that they are capable of making comparisons and trade-offs between competing goods according to their preferences. In essence, health 3  economists investigate the ways in which the greatest health ‘bang’ can be achieved for a given ‘buck’ of investment.  In order to accomplish this, health economists may adopt the framework of welfare economics, which measures aggregate (population-level) utility as the sum of the utility experienced by all individual members of the population[3, 4]. In so doing, welfare economics recognizes that all choices will affect different people in different ways – some positive, some negative – and seeks to understand and quantify all possible effects in order to choose the option that provides the highest level of aggregate utility. Health economists may also take an “Extra-welfarist” approach, which considers the value that societies place on health outcomes, and seeks to maximize health within the resource constraints of a given decision environment[3]. The distinction between welfarism and extra-welfarism allows health economists to consider the desirability of outcomes such as equity and health-related quality of life in addition to individual preference-based estimates of utility. 1.2 Health Technology Assessment and Management Health Technology Assessment (HTA) is a multidisciplinary field of inquiry and evaluation which uses health economics to consider how to best allocate resources for health technologies (including drugs, therapies, policy changes, and other mechanisms for improving health) under scarce budgetary conditions[5-8]. HTA seeks to understand and measure the effect that changes in health care services, devices, programmes, products, and other technologies may have on society using rigorous scientific methods[6]. These effects may include economic, social, political, and other changes that affect, and are affected by, human health.  The purpose of HTA is to provide information suitable to inform policy- and policy making in health care planning[6].  4  HTA incorporates health economic theory and methods along with other disciplines such as systematic review and meta-analysis, as it seeks to achieve the balance between quality of care and resource responsibility, under the economic assumption that a dollar spent in one place is a dollar that cannot be spent in another[9]. HTA explicitly seeks to guide policy making toward comparing technologies competing for scarce resources and finding the pattern of implementation and delivery that produces the greatest possible health outcomes for the population of interest, within political, ethical, legal, and budgetary constraints[10]. HTA exercises are commonly performed on novel technologies as they are brought to the market, providing important evidence to allow policy makers (health regions, hospitals, governmental and non-governmental agencies, etc.) to make sustainable and responsible funding decisions. As the pace of new technologies increases[11], the need for timely and useful economic evidence becomes more apparent.  HTA using health economics has grown into a discipline used by policy making organizations worldwide[9]. Organizations like the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) and the International Health Economics Association (IHEA) provide standards and guidelines for health economics research in order to help policy makers in address the challenges of appropriate health care resource allocation. HTA and health economics methods are used widely by policy making authorities within the United Kingdom, where the National Institute for Health and Care Excellence (NICE) uses it to guide health care policy making[12]. The Canadian Association for Drugs and Technologies in Health (CADTH) is an independent and multi-faceted agency that uses health economics evidence to provide similar guidance to jurisdictions such as provinces and territories in Canada[5]. Within the British Columbia Cancer Agency (BCCA), the Priorities and Evaluation Committee (PEC) uses HTA and health economic evidence to guide funding of cancer therapies and other novel technologies[13, 14]. 5  More recently, experts have recognized that typical HTA practice placed an undue emphasis on the adoption of new technologies, at the expense of ensuring cost-effective use of existing technologies[15-18]. Health Technology Management (HTM) is an extension of the HTA approach that focuses the techniques and values of HTA toward technologies that are currently being used within the health care system and seeks to identify technologies that are being used sub-optimally, outside their appropriate indications, or that are failing to deliver adequate benefit to patients[15]. HTM therefore considers both the investment and disinvestment of technologies that do and do not provide good value for money, so that health care funding may be reinvested in more appropriate ways[15, 17, 18].  Economic evaluation is a key health economic method used within HTA. 1.3 Economic Evaluation Economic evaluations simultaneously consider both costs (in terms of time, money, resources) and outcomes (in terms of human health) of health care technologies as a form of evidence to support policy making [19]. These evaluations are principally related to question #5 within Williams’ framework[1], but are influenced by various aspects of health economic theory. They typically employ some combination of three different analytical frameworks to address issues of appropriate allocation between competing alternatives (A vs. B)[19]: Cost-effectiveness analysis (CEA): how does the difference in cost between A and B compare to the difference in outcomes between A and B, where the outcome of interest is a state of health or other health-related outcome?  Cost-utility analysis (CUA): how does the difference in cost between A and B compare to the difference in outcomes between A and B, where the outcome of interest is a measure of well-being (health state utility)?  6  Cost-benefit analysis (CBA): how does the difference in cost between A and B compare to the difference in outcomes between A and B, where those outcomes are expressed in terms of their monetary value?  CEA routinely compares incremental costs to incremental survival (often expressed as Life Years Gained - LYG), but it is common to see other outcomes used (e.g., cost per diagnosis, cost per case avoided, cost per cancer recurrence avoided, etc.). CEA is favoured for technologies that prolong life, either by increasing survival directly (e.g., life-saving surgeries, tumour-shrinking drugs) or by changing the rate and/or probability of life-threatening events (e.g., reducing the burden of disease through screening, avoiding relapse or progression of disease, preventative measures to reduce disease incidence). CEA permits “apples to apples” comparisons between the likely consequences of different technologies, under the assumption that a year of life is the same for all individuals, and does not depend on the population or disease under consideration. CUA can be thought of as a subcategory of CEA that uses a narrower definition of health outcome. CUA is favoured for technologies that may or may not have an impact on the length of life, but also the condition and nature of that change in survival. Health economists consider the health-related quality of life (HRQoL, sometimes abbreviated to QoL) that people experience within a given state of health. QoL can encompass a variety of domains, from functional mobility and pain to social and psychological well-being. CUA considers the possibility that people may make trade-offs between their length of survival and the associated level of QoL. Some people may, for example, prefer to spend a short amount of time in a state of health that is free from pain and disability rather than living for a long time but in constant pain and unable to complete activities of daily living. In order to reflect this potential preference, health economists consider the level of utility associated with being in a given state of health (health state utility) alongside the length of life. Health state utility is commonly estimated through the use of methods like Standard Gamble or Time-Trade-Off exercises[19, 20], where people choose between 7  different hypothetical health scenarios in order to elicit their preferences for different states of health and their components (pain, mobility, anxiety, etc.), although other methods of preference elicitation are also used[21]. The responses to these exercises can be expressed mathematically, giving a value anchored between 1.0 (a state of health equally preferable to full health) and 0.0 (a state of health equally preferable to death). CUAs most often consider the quality-adjusted life year (QALY), a measure that incorporates health utility into measures of survival. For example, a person who lives for five years in full health (i.e., utility of 1.0) is considered to have the same number of QALYs as a person who lives for ten years in a state of health with a utility of 0.5 (5 QALYs each). CBA differs from CEA and CUA in that both the cost of a health technology and the health outcomes resulting from its use can be valued monetarily[4]. The ‘benefit’ of a health technology is, therefore, expressed as the dollar value of the resulting change in health. CBA is less-commonly used than CEA or CUA, owing in part to the difficulty inherent in placing a monetary value on human health and life. It is often appropriate when the outcome of interest is expected to reduce the burden of disease in the population, and thereby reduce health care expenditure (e.g., a vaccination program to prevent acute disease). CEA and CUA commonly evaluate the relationship between the change in costs and the changes in outcomes using the Incremental Cost-Effectiveness Ratio (ICER; Δ$/ΔLYG and Δ$/ΔQALY respectively). ICERs are considered in the context of an estimate of society’s willingness to pay for an outcome (a LYG or a QALY or some other effectiveness measure), sometimes denoted as lambda (λ). If the ICER associated with adopting a new technology is smaller than λ, it can be considered cost-effective. CBA differs from CEA and CUA because both costs and outcomes are expressed in monetary terms – CBAs accordingly consider the net monetary benefit (incremental benefit minus incremental cost) or the 8  benefit-cost ratio (incremental benefit divided by incremental cost) to determine whether a technology should be adopted. The ability to compare costs and outcomes is particularly useful for guiding policy making for health care system funding. These funding decisions are always made within an environment of scarcity, as a health care system’s budget is never infinite and many costly alternatives must be weighed against one another. New technology options are constantly being proposed to improve the health of patients and the general population, and HTA provides evidence to help identify which of the options will provide acceptable value for money.  The usefulness of economic evaluation seems clear, though it is most appropriately used in conjunction with other forms of evidence. Issues of justice, equity, and political tolerability are also part of policy making frameworks[10]. Regardless, economic evaluation has an important role to play in policy making, providing policy makers with evidence-based estimates of the impact that a funding decision may have on the health care system and the population it seeks to serve. 1.4 Decision Modeling in Economic Evaluation Like other forms of evidence generation in health, economic evaluation is often performed alongside clinical trials[22, 23]. Costs and outcomes are measured prospectively through the trial, providing the data necessary to calculate cost-effectiveness (or cost-utility, or cost-benefit)[22]. However, it is also often the case that health care decisions must be made for decisions where it would be unfeasible, unethical, or impractical to conduct a clinical trial - data collection may require years of follow-up, the population may be prohibitively expensive to follow, the decision may require a policy change that cannot be delivered randomly[19]. In cases where trials are conducted, the length of follow-up may be insufficient to capture all outcomes relevant to the decision – for example, a trial’s endpoint may be the 9  number of disease events (like diagnosis of premalignancy or progression while on treatment) detected in a population, but may not be long enough to detect survival differences. In cases like this, models can be used to estimate the difference in overall survival that is expected to result from a change in the rate of these events.  In cases where a single trial is insufficient to estimate cost-effectiveness, HTA makes use of decision modeling techniques. Decision models are simplified statistical recreations of the impacts of a decision on outputs of interest, including costs and health outcomes[24]. By using known information about the decision environment - the various relevant inputs and outputs that will influence/be influenced by the decision - decision models can provide an evidence-informed approximation of what will transpire if the technology is adopted. In order to analyze the impact of a decision, a model must take the following steps[25]: 1. Define the target population, the decision environment, the important characteristics of the disease of interest, and the technology undergoing evaluation. 2. Describe the trajectory of possible events in a logical and realistic way. The relationships between different states of health and outcomes must also be considered. 3. Consider the events that may occur within the trajectory, and their associated health and cost implications, including the way those events may change as the result of the technology being assessed; and 4. Provide a computational method to estimate the measures of value (i.e., measures of cost, effectiveness). Models rely on the synthesis of the best available data to inform the value of the various parameters (variables that comprise the model). Data required for health economic decision modeling falls into four broad categories: 10  1. Non-time-dependent (NTD) probabilities: proportions that are not dependent on time – e.g., percentage of people who smoke, proportion of surgeries that result in disease-negative surgical margins, proportion of people who receive chemotherapy rather than RT; 2. Time-to-event (TTE) data: the amount of time elapsing between each event – e.g., time to develop disease, progression-free survival after treatment, death; 3. Costs: monetary value of resources used during the simulated events within the model; and 4. Utilities: a value representing a person’s preference for being in a given state of health, expressed as a value between 1.0 (perfect health) and 0.0 (a state equally preferable to death). It is common for models to derive data for their parameters from published clinical trials and meta-analyses. More sophisticated models may use stochastic (individual-level) data when it is available. Observational data and estimates derived from expert opinion are also used when appropriate. The choice of data source depends on the availability of data and the decision environment of the model[24]. As with other forms of research design, there are multiple types of modeling approaches, each suited for different types of economic evaluation questions. 1.5 Types of Decision Analytic Models Modeling approaches in HTA may include decision trees, cohort-level state transition models, and discrete event models, among other types[26, 27].  1.5.1 Decision Trees Decision trees represent, through branching probability chains, the sequence of events that may result from a choice and their likely outcomes[24, 26]. Decision trees have the advantage of being both simple and intuitive, and are ideal for evaluating the likely impact of decisions with short-term and/or well-characterized outcomes. Their use is limited, however, when dealing with decisions that happen over 11  long periods of time, particularly where events can reoccur multiple times. Additionally, complex decisions may be difficult and unwieldy to model in this way, as the number of branches in the tree becomes progressively more cumbersome with the addition of potential health outcomes, events, and decision paths.  1.5.2 Cohort-Level State Transition Models Cohort-level state transition models (often called ‘Markov’ models) simulate the progress of a hypothetical group of people through states of health over fixed intervals of time (called the model’s ‘cycle length’), according to the probability/rate at which state transitions occur[26, 28]. Markov models are often visually represented with “bubble and arrow” diagrams (see Figure 1.1), showing how members of the simulated cohort move between health states. This type of model can be understood as an expansion of the decision tree approach that incorporates the passage of time, and that allows events to repeat an indefinite number of times. Cohort transition models are among the most commonly-used models in the health economics literature[28].  Despite their popularity however, these models are limited in important ways: they typically assume that transition probabilities do not change over time, nor do they account for individual patient history (referred to as the ‘memoryless’ assumption), and they do not easily account for competing event risks within a given time frame[25, 29]. Additionally, members of a Markov cohort can only exist in one state at a time, which can make it difficult for the model to handle the effects of comorbid conditions or risk factors. Markov models also do not easily accommodate the presence of risk factors that may change over time, particularly if those factors are continuous rather than discrete in nature (e.g., body weight, intermittent risk exposure, dosage of medication received). Through design, it is often possible to 12  account for some of the limitations of this modeling approach, but like decision trees this can often lead to unworkable levels of complexity.   Figure 1.2 – Conceptual diagram of a Markov model  Finally, it is difficult to reflect the risk of competing events within a Markov model. Competing risks occur when a simulated cohort member is at risk of multiple events at a given point in time. For example, a member of a Markov simulated cohort in the ‘Healthy’ state may transition to either ‘Sick’ or ‘Death’ within a given model cycle, or may remain in the ‘Healthy’ state. Individuals who die during the cycle will not become sick (or may become sick and then die within the same cycle). If the estimates for the probability of sickness and the probability of death are estimated independently (i.e., without heed to competing risks), the transition probabilities will not reflect this possibility. This undercuts the accuracy of the inputs, and resulting outputs from the model. 1.5.3 Discrete Event Models 13  Discrete event models describe the sequence of occurring events in simulated real-time (rather than fixed cycles), and are commonly found within operations research[25-27, 29, 30]. These models are comprised of five fundamental components[29]: 1. Entities: the items (in this case, people) that flow through the simulation. Entities contain attributes (e.g., gender, age, disease history, quality of life) that accompany them through the simulation, may change during the simulation, and affect their route; 2. Events: anything that can happen to an entity within the model. Typically in disease survival models, events include treatment processes (e.g., a person has a doctor’s appointment, undergoes surgery, etc.) and disease processes (e.g., a person develops clinical disease, goes into remission, experiences recurrence, etc.); 3. Time: the speed at which entities experience events. Time can be marked by whatever is the most useful increment for the process being modeled (days, months, years, etc.); 4. Resources: the number of units of a given resource utilized by an entity during a given event; and 5. Means of execution: the logic and computational method that underlies the passage of entities through the events and the consumption of resources over time. Discrete event models have important advantages over Markov models. Primarily, events can occur at any time and do not need to be fit to a particular cycle length. This is especially advantageous when events can occur within the model over very different time scales (e.g., events a few days in length compared to ramifications spanning several years).  Second, because attributes are carried by each entity, simulated individuals may exist in several possible states of health simultaneously (e.g., it is trivially easy to distinguish between a 35 year-old woman with comorbid diabetes who is six months out of her first hospitalization and a 60 year-old man with no comorbidity who is freshly discharged from his fifth hospital visit; it is much more challenging to represent differences like this in a cohort model). 14  Third, because events do not occur at fixed intervals, discrete event models handle the risk of competing events much more easily than cohort models. Discrete event models handle competing events by using time-to-event (TTE) data to inform the sequence in which events occur. Time to death and time to sickness are both determined stochastically for each entity, with the lowest value occurring first. To use the same example from Figure 1.1, it is possible to estimate the time in which a simulated entity may develop illness (i.e., become “Sick”). Time to death (i.e., the entity experiences “Death”) can be estimated as well. These events can be considered for the entity in sequence, and the entity will experience illness and/or death at the corresponding estimated times, in the order in which they are estimated to occur. Individual sampling models, commonly used in HTA, are an example of a discrete event model where simulated people do not interact with each other[26]. Entities are simulated independently from each other, and experience events irrespective of the number of other entities being simulated. This type of model does not consider queueing for resources, disease transmission between entities, or other interactions that may theoretically occur between people in the real world. Discrete event models are typically more complicated to design and build than Markov models, and often require a breadth and depth of data that is not easily gleaned from the scientific literature[26].  The choice of modeling approach is typically dictated by the research question that a health economist seeks to address[24, 26]. Data availability, decision complexity, and the skills of the modeler also play a role. Economic evaluation using models must carefully consider different forms of uncertainty when interpreting the results.   15  1.6 The ‘Whole Disease Model’ Theoretical Framework  Decision models of the kind described above are most typically built to represent a single decision, within a single decision environment (i.e., the impact of a single health technology change). It would be inefficient to include model inputs that are not relevant to the outcome of interest, since inclusion of those inputs would not impact the output of the model. Unfortunately, this limits single-decision models (also termed ‘piecewise’) in an important way. Health care decisions are seldom made in a vacuum, and changes in the policies that govern the management of a particular disease may have a meaningful impact on the performance of a newly-adopted technology, ‘Upstream’ and ‘downstream’ are terms that describe the chronological relationship between a hypothetical policy change and a particular technology adoption of interest. If the policy affects events that occur earlier in the disease trajectory than the technology is able to influence, it is termed ‘upstream’. Conversely, policies that affect events occurring after the technology exerts its influence are termed ‘downstream’1. A good model anticipates downstream outcomes, but cannot predict the effect of ‘upstream’ events – i.e., events that occur before the decision of interest – that lie outside the model’s scope. Those effects, such as a change in the population characteristics (e.g., age, disease performance status, impact of previous treatments, synergistic effects of other technology decisions) may change a model’s inputs in a way that the model cannot account for. In order to address these limitations, a model is needed that can incorporate upstream policy changes and evaluate the simultaneous implementation of technology change at multiple points along the disease pathway[31]. Such a model, called a Whole Disease Model (WDM) would represent the decision                                                            1 It is important to note that ‘upstream’ and ‘downstream’ can only be defined in relation to another technology or policy change. They are not intrinsic properties of the technology being adopted. Accordingly, a given technology adoption decision might be ‘upstream’ from one policy, but ‘downstream’ from another. 16  environment for an entire disease, from preclinical status through all possible biological and treatment-related outcomes including death. Because the model contains many levels of policy making – screening, preclinical management, treatment, recovery, relapse, etc. – the impact of changes in any one will be reflected in the others. WDMs thereby allow researchers to simulate the impact of implementing multiple policies at the same time.  It would be theoretically possible to simply address the limitations described above by using multiple ‘piecewise’ models, such as those found in the literature, in succession. However, integrating models built in different program languages and platforms can be exceptionally challenging. A modeler wishing to reproduce and adapt a given model from the literature may not have access to the software the model was originally programmed in. While it may be theoretically possible to reflect ‘upstream’ changes by combining models that represent different parts of the disease trajectory, it is not straightforward to do so if those models use different approaches and software. There is evident value in creating a model that is easily adaptable to multiple policy questions, and that can be directly used/edited by a policy maker to increase transparency and credibility. Given that WDMs are more complex and therefore require much more data than typical ‘piecewise’ models, it is particularly valuable to build a WDM that can be updated, adapted, and re-used as needed.  The first implementation of the Whole Disease Modeling framework, conceived by Tappenden and colleagues, considered policy making in colorectal cancer[32]. Through consultation with experts, Tappenden constructed a WDM that covered the breadth of colorectal cancer management from screening to terminal disease, including natural history and treatment. This model was then used to perform cost-effectiveness analyses on eleven separate health care decisions across decision nodes spanning the colorectal disease control pathway. 17  Importantly, Tappenden was able to use the WDM model to make recommendations about disinvestment and budgetary impact. In his discussion of the exercise, Tappenden explicitly argued that the time and resource costs associated with building the WDM, while considerable, were less than what would be required to build eleven piecewise models to evaluate each facet of colorectal cancer management. He further argued that the appropriate place for the use of a WDM is during the development of clinical guidelines, which the model was used for in a subsequent study[33]. Tappenden’s colorectal cancer model and the WDM framework can be thought of, in this context, as an extension and application of existing HTA and modeling techniques to address a specific weakness of piecewise modeling. The model was able to produce the same conventional outputs (ICERs, uncertainty analysis) that piecewise HTA models would, while providing additional information that is useful for policy making. While the original implementation of the WDM framework was in cancer, the first replication exercise was in COPD, demonstrating the value of the framework in the context of other diseases[34]. 1.7  Case Study: Health Technology Management for Oral Cancer 1.7.1 Oral Cancer Oral cancer is a rare cancer, particularly when compared to cancers of the lung, breast, prostate, and colon/rectum[35]. In British Columbia, an estimated 600 people (420 men, 180 women) are diagnosed with an oral cancer annually. This low incidence and gender ratio is echoed among the national statistics, with 4650 estimated incident cases annually. Oral cancer is the ninth-most common cancer among men (representing just 3.1% of incident cancers) and fourteenth-most common among women (1.4%). 18  Despite its low incidence, oral cancer has a high case fatality rate (60% five-year survival) placing it among the deadliest cancers[36]. This fatality rate is strongly influenced by the stage at which a malignancy is detected[36]. Oral cancers are often heralded by the development of premalignant lesions that can be resected before an invasive carcinoma can develop[37]. Early-stage cancers are treatable with surgery and highly survivable, while late-stage cancers are associated with high morbidity and most often fatal[36]. Recurrences are common in late-stage cancers, and have a low cure rate[36]. Technologies that lead to earlier detection and lower rates of post-treatment recurrence have the potential to have a sizeable impact on mortality and morbidity rates in this disease. Alcohol and tobacco use sit atop the list of behavioural risk factors for developing oral cancer, but the disease is also associated with the use of betel quid and areca nut[38], factors more commonly seen in South and East Asia where incidence rates are much higher[39]. Variations in incidence are seen between different ethnic groups and different regions of the world[39, 40], and there is emerging evidence of the etiological role played by biological factors like age[41] and genetics[42]. Recent investigations have developed a body of evidence concerning the role that human papillomavirus (HPV) plays in both the development and treatment of oral cancers[43, 44]. 1.7.2 Medical Management of Oral Cancers Oral cancers are often detectable as premalignant lesions within the oral cavity, clinically called leukoplakias or Oral Premalignant Lesions (OPLs)[38]. Leukoplakias are often difficult to distinguish from reactive lesions with minimal malignant potential, making screening a challenge. If a dentist or oral hygienist detects a leukoplakia in a patient, they may be referred to an allied oral health specialist (e.g., a periodontist, an oral surgeon, oral medicine pathologist) for further investigation; the dentist may also perform this investigation themselves. If the leukoplakia shows indication of being a high-grade 19  precancerous lesion, it may be resected or referred for surveillance[37]. Leukoplakias under surveillance that show evidence of progression will be resected. Oral cancers are managed by the British Columbia Cancer Agency’s Head & Neck Tumour group[45], in conjunction with the BC Cancer Agency’s Oral Cancer Prevention Program[46]. Early-stage cancers are treated primarily through surgery, with adjuvant external beam radiotherapy being common[47]. Cancers that are not amenable to surgery due to size/location are managed with radiotherapy and chemotherapy. Late-stage cancers are managed through a combination of surgery, radiation, and chemotherapy, highly dependent on the nature of the disease presentation and the patient.  Because of their location, oral cancers can be highly morbid and treatments can cause severe disfigurement and quality of life impairment[48-53]. Even people with successfully-treated disease may experience meaningful impairments in speech, swallowing, tasting, and may experience pain and dryness. Changes to dentition including loss of teeth are common, particularly for cancers detected at a late stage. Cancers detected at an early stage, by contrast, often have fewer and milder symptom profiles and treatment-related morbidities than more invasive disease[48, 54]. Local and regional recurrences – the re-growth of cancerous tissue after an apparently-successful treatment – are common in oral cancers, especially for those diagnosed at a late stage[48]. Recurrence management is far more complex than management of a primary cancer, owing in part to the aggressive nature of the disease and the reduced effectiveness and availability of treatment options[55]. Local recurrences (tumour re-growth at the site of the original cancer) have better prognosis than regional recurrences (tumour re-growth elsewhere in the oral cavity)[56]. Reducing the rates of recurrence, either through more effective primary treatment or screening methods to diagnose disease earlier could potentially reduce overall treatment costs and increase disease survivability. 20  1.7.3 Potential Health Technologies in Oral Cancer Management The evidence on survival and quality of life suggest very strongly that prevention and earlier detection of oral cancers could yield large and meaningful reductions in mortality and morbidity. At the same time, population-level screening programs and/or improvements in primary treatment can be expensive and inefficient.  Access to dental professionals, often the first step in the detection pathway, is far from universal[57]. Developments in screening technology, for those patients with access to a dentist, are undergoing evaluation[58, 59]. Even for patients with access to a dentist with advanced screening tools, rates of compliance to screening guidelines, which vary among practitioners for reasons that are not well-characterized in the literature, are an impediment to diagnosis[57, 60]. Improvements in any of these components of screening could potentially yield a higher rate of detection of early-stage cancers and pre-malignancies. Recent discoveries in the biology of leukoplakia progression suggest the possibility of risk prediction as a component of disease management[61]. Surgical management is effective, but improvements are currently undergoing clinical testing[62].  Surgical interventions such as the dissection of the lymph nodes of the neck can reduce the risk of the disease returning after treatment, but given the high rate of morbidity and potential for life-changing disfigurement, the risks of over-treatment may outweigh the benefits of reduced recurrence rates[48, 63, 64]. Methods of delivering more appropriate treatment to patients with greater risk of recurrence after primary surgery may also yield improvements in survival and quality of life among patients who develop disease.  Medical interventions for late-stage disease may prolong life, but at a cost that most health care policy making entities may not consider good value for money. Recent developments in systemic therapies 21  have cleared regulatory hurdles for safety, but no published health economic data is available to guide policy making[65-67]. There are important economic evaluation questions to be asked at various points along the oral cancer disease pathway. Piecewise methods can be used to model each of these questions in isolation. However, as it seems likely that many of these new technologies will be implemented in relatively quick succession, it is worth considering their collective impact as well as the ‘upstream’ and ‘downstream’ impact of one on another. For example, an improvement in screening could lead to fewer late-stage diagnoses, potentially making improvements in surgical follow-up more (or less) effective than a piecewise analysis would suggest. In addition to cost-effectiveness, policy makers may be interested in budgetary impact. A cost-effective policy may be less (or more) appealing in the presence of other technologies. For example, a highly-effective curative treatment would make population screening less urgent, and less attractive financially. Conversely, a drug that is very costly may seem more reasonable to fund if upstream improvements in early detection mean that a smaller number of people will need it. 1.8  A Whole Disease Model of Oral Cancer (WDMOC) Oral cancer is also a low-incidence disease, making large-scale clinical trials difficult and expensive to conduct. Accordingly, health technology assessments in oral cancer will rely heavily on modeling techniques.  Tappenden argues that there are five principal criteria that justify the use of a WDM[31]:  1. Presence of multiple potential services not subject to formal analysis – when several components of the decision environment have not been modeled; 22  2. New technologies/approaches under evaluation - when multiple technologies may be introduced in the relatively near future; 3. Little published health economic evidence - when costs and outcomes for the disease are not well characterized in the literature; 4. Upstream changes will impact downstream cost-effectiveness - when there may be a synergistic effect between potential novel technologies; and 5. Policy-makers are likely to want more information that just cost/QALY - when budgetary impact and other types of economic evidence are likely to play a role in policy making. Oral cancer is an ideal candidate for a WDM approach, as it meets all five of Tappenden’s criteria[31]: 1. Very few formal analyses of the costs of various parts of the oral cancer pathway have been conducted - this dissertation contains the first such exercise in Canada; 2. A number of potential policy changes, treatment options, and other health technologies are currently undergoing evaluation within the BC Cancer Agency’s Oral Cancer Prevention Program . These technologies, ranging from improvements to screening and early-stage disease management to interventions for terminal disease, may be introduced into practice. HTA evidence will be needed to guide funding decisions; 3. Cost-effectiveness evidence within oral cancer is very sparse. The few oral cancer modeling exercises within the literature are far too simple to reflect the complexity of changes in treatment, or how the impact of changes in treatment might be altered by upstream policy shifts; 4. For reasons described above, the evidence suggests that upstream changes in individual characteristics and health system policies will affect budgetary impact and cost-effectiveness. Any policy that can successfully shift the trend toward earlier detection and/or reduced rates of disease recurrence may  have broad implications for the cost-effectiveness and budgetary impact of approaches for treating late-stage and recurrent disease; and 5. This information, and the model that produces it, will be useful to policy-makers. It can potentially be used not only to estimate the impacts of policies currently undergoing evaluation, but to identify fruitful avenues for future research where a change in policy could have a meaningful impact on the health care system. 1.9 Objectives of This Dissertation 23  The overarching aim of this dissertation is to describe the development of the Whole Disease Model of Oral Cancer (WDMOC), and its applicability in health economic analysis. It will do this by accomplishing the following specific research objectives:  The first objective is to determine the current state of the literature concerning the health economics of oral cancer and to identify specific knowledge gaps in technologies that are suitable for economic evaluation. This objective will be met in Chapter 2, which will summarize the current literature around the management of oral cancer including screening, surveillance, diagnosis, and treatment. Emphasis will be placed on health technology assessments and other forms of economic analysis including quality of life and health state utility. This chapter will also detail potential developments in oral cancer that will likely require health economic analysis in the near term.  The second objective of this dissertation is to illustrate the limitations of a ‘piecewise’ modeling approach when evaluating health technologies. This objective will be met in Chapter 3, which will describe a cost-effectiveness analysis of managing oral premalignancies using a genome-guided risk assay. Cost-effectiveness will be estimated using a conventional Markov decision model, and the limitations of the approach will be explored and discussed.  The third objective of this dissertation is to design and implement a Whole Disease Model of Oral Cancer (WDMOC) . This design and implementation will be described in Chapter 4, following principles and an approach laid out by Tappenden and incorporating best practice guidelines for decision modeling. This chapter will describe the structure of the model, as well as the model’s parameters and data sources from which the parameter estimates were derived. The model’s outputs will be compared to values observed in the population of British Columbia.  The fourth objective of this dissertation is to compare the Whole Disease Modeling approach with the ‘piecewise’ approach, to evaluate how the novel approach compares to a traditional one. Chapter 5 will re-examine the research question from Chapter 3 using the WDMOC. The chapter will describe the similarities and differences between the two methods in order to demonstrate that the WDMOC can provide rigorous and evidence-based outputs that are 24  comparable to standard practice, while also addressing important limitations within those standard models.  The fifth objective of the dissertation will be to estimate the impact that ‘upstream’ parameter changes have on the cost-effectiveness and budgetary impact of ‘downstream’ technology adoption. This objective will be met in Chapter 6, where the WDMOC will be used to estimate how multiple hypothetical policy changes impact the cost-effectiveness of the risk assay, both individually and in combination.   25  2 Chapter 2 – Literature Review This chapter summarizes the relevant scientific literature that underpins this dissertation topic and the methodological approach chosen.  2.1 Introduction Policy making authorities – hospitals, health regions, provinces, etc. – may wish to understand the impact that changes to the health care system might have on patients, and the extent to which those changes are proportional to the investment necessary to achieve them. To answer these questions, they may turn to the clinical and health economic literature. This chapter will briefly review and summarize this literature with respect to oral cancer from its development into premalignant disease, its detection and treatment, and the course it may take after treatment. It will then describe a structured approach to reviewing the health economics literature around technologies for oral cancers, with a particular focus on published exercises that used a decision modeling approach. A discussion of some potential changes to current standard of practice in oral cancer management will follow, including the potential cost-effectiveness questions that may arise from their uptake. Gaps in the literature as they relate to the evaluation of these new technologies will be summarized, to provide justification for the WDM approach used within this dissertation. 2.2 Oral Cancer Development, Progression, and Management 2.2.1  Premalignant Disease and Progression to Cancer As briefly described in Chapter 1, oral cancers are typically heralded by the development of premalignant growths in the oral cavity (i.e., tongue, floor of mouth, cheek, gingiva, hard or soft palate). While there are many types of abnormal growth possible within the mouth, lesions that are considered 26  precursors to cancer (i.e., oral premalignant lesions – OPLs) are typically leukoplakias (white lesions that are not attributable to another disease condition) and erythroplakias (red patches not attributable to another disease condition)[68-70].  Estimates of the incidence and prevalence of OPLs are inconsistent in the literature, owing both to the relationship between OPL development and behavioural factors like alcohol and tobacco use, and the variety of research methods used to identify cases[38, 69, 71]. An exercise conducted in a sample of community clinics in BC found potentially precancerous lesions in 0.12% of patients[72]. A similar exercise conducted in Boston, USA found an incidence of 0.09%[73]. Both studies found a statistical association between lesion development and tobacco use. The majority of OPLs will not develop into malignant disease[38, 68, 71, 74, 75]. A review of the literature found that female sex, age, and the location and size of the lesion were all risk factors for progression to cancer[69]; however, a subsequent review found it difficult to draw clear conclusions about progression risk, due in part to the heterogeneity of both disease classification and the populations in which disease was observed[71]. While anecdotal evidence exists that other factors such as socioeconomic status (SES) may independently correlate with the risk of developing oral cancer, the available clinical evidence does not support this finding[76].  A clinical trial conducted within BC identified a chromosomal characteristic known as “Loss of Heterozygosticity” (LOH) as a potential biological marker for disease development[61]. The Zhang et al. study also reported behavioural risk factors such as alcohol/tobacco use, age, and gender from their study cohort. Members of this cohort had an estimated five-year malignant transformation rate of 3.1%, 16.3%, and 63.1% for people with low-, medium-, and high-risk genomic markers respectively. The 27  previously-mentioned systematic literature review found an average annual malignant transformation rate of 3.8%, estimated from among the three studies that reported such a rate[71]. 2.2.2 Detection of Premalignant and Malignant Disease A person with an OPL but no serious symptoms (i.e., pain or discomfort) may not notice the lesion at all before it becomes cancerous. These undetected OPLs may be detected during the course of routine oral care by an oral health professional (e.g., dentist, dental hygienist, oral medicine specialist). Conventional oral examination (COE) has some evidence to support its efficacy and cost-effectiveness[77, 78]; however, the lack of evidence of survival benefit from randomized controlled trials (RCTs) places a serious limitation on the ability of organizations to issue concrete screening guidelines and recommendations[77, 79, 80].  Current evidence suggests that, due to issues of access, public knowledge, and the lack of concrete clinical evidence, population-level screening programs are challenging to properly implement[81, 82]. The effectiveness of such programs is also subject to their acceptance and uptake from oral health professionals, and the evidence about such uptake is mixed[83-85]. A national survey of dental hygienists in Canada found that hygienists believed that oral screening was conducted primarily by them (with overlapping care provided by a dentist), and that they felt comfortable with and capable of detecting oral cancers during routine appointments[86]. Despite their perceived level of knowledge about screening, previous studies have suggested that barriers exist to implementation of regular screening[72, 87]. The BC College of Dental Surgeons formally launched its guidelines for oral cancer screening in 2008[37], recommending annual visual screening to all individuals age 40 and older. The guidelines also set out recommendations for referral and management of potentially precancerous lesions and oral cancers 28  among oral health professionals and the BCCA. The Oral Cancer Prevention Program (OCPP) is a collaborative effort between oral health professionals aimed at providing coordinated care to British Columbians, including outreach to specific communities who are more likely to develop cancer[46, 88]. Research suggests that these guidelines, when followed, are effective in detecting premalignancy[57]. However, given the lack of a central registry of screening activities, it is difficult to estimate how many oral health professionals adhere to the recommendations. 2.2.3 Treatment of Malignant Oral Cancer Management of oral cancers is dependent on the characteristics and extent of the disease[45, 47, 56, 89]. Oral cancers can be classified into four approximate stages[89]: Stage I (T1N0M02): Tumour is ≤ 2 cm in its greatest dimension, no nodal involvement, no metastasis Stage II (T2N0M0): Tumour is larger than 2 cm but ≤ 4 cm in its greatest dimension, no nodal involvement, no metastasis Stage III (T3NXMX or T1/2N1M0): Tumour is greater than 4 cm in greatest dimension and/or disease is present in a single regional lymph node Stage IV (T4aNXM0 or T1/2/3N2M0 or TXN3M0 or T4bNXM0 or TXNXM1): Tumour is invading surrounding tissues, disease in multiple regional lymph nodes, and/or distant metastasis. Tumours are most commonly found on the tongue, floor of mouth, and the lower lip[89]. Diagnosis typically involves computed tomography (CT) scan and may also include Magnetic Resonance Imaging (MRI) in addition to physical exam[56]. Early-stage (stage I/II, as well as high-grade dysplasias and in situ disease) cancer is typically treated with surgery[45, 47, 56, 89]. Depending on disease and patient characteristics, treatment may also include external beam radiation therapy[56, 89]. Diseases with                                                            2 TNM (primary Tumour, regional lymph Nodes, Distant Metastasis) is a tumour staging categorization system commonly used for most cancers.  29  regional and/or metastatic involvement may be managed with systemic therapy and radiation, and may or may not include surgery. Estimated survival is highly associated with stage at detection[36, 56, 89]. Following treatment, patients are managed by their oncological team – their otolaryngologist head & neck surgeon, radiation oncologist, and medical oncologist – to monitor the tumour site for evidence of disease recurrence and/or progression (in the case of incurable disease)[45, 90]. Time to recurrence/progression is also strongly influenced by stage at diagnosis as well as other clinical features[48, 56, 89, 91, 92]. 2.2.4 Disease Recurrence and Management Management of recurrent oral cancer is highly individualized, based on numerous clinical and patient-specific factors[47, 55]. Patients may receive surgery if their recurrent tumour is resectable, and will likely receive radiation if they have not previously done so (although reirradiation is possible for some patients, radiation toxicity is a serious concern). Systemic therapy is often prescribed to patients with recurrent disease, but there is mixed evidence supporting its efficacy[55].  Prognosis following recurrence depends on a variety of factors including eligibility for salvage surgery[55], previous treatment[93], recurrence interval[94], stage at presentation[94], genetic factors[95], and site of recurrence[96], among others. Follow-up and surveillance for recurrent disease is once again managed by the clinical team. Multiple recurrences are possible, but rare. Recurrent disease has very poor prognosis[55, 93, 94, 96]. 2.2.5 Quality of Life and Health State Utility Oral cancer, even when treated, is associated with a variety of adverse health-related quality of life (QoL) outcomes[97, 98]. Survivors of oral cancers report issues with dentition, swallowing, speaking, and 30  salivation that can persist for several years after treatment[99, 100]. The type and degree of these adverse outcomes is affected by stage at diagnosis, treatment type, and tumour site[97, 100, 101], among other factors. An emerging body of evidence suggests that patient-reported QoL can be prognostic, with poor QoL at time of treatment being associated with poor survival outcomes[97, 98, 100, 102].  A recent review of the literature summarized exercises that estimated health state utility in head and neck cancers[103]. This review found that utility values for recurrent and metastatic disease were not well characterized, and found a great deal of uncertainty among evaluations of identical states of health, especially between different utility-elicitation instruments. The authors note that, for the purposes of health economic evaluation and model-building, the three-dimension EuroQoL tool (EQ-5D-3L) may be preferred as these tools are recommended by agencies like NICE. Accordingly, the results of studies reporting values from the EQ-5D-3L were abstracted from their respective manuscripts. These values are presented in Table 2.1.   31  Table 2.1 – Summary of Head & Neck Cancer studies reporting EQ-5D values from Meregaglia and Cairns (2017) review Study Year Country Population N Health state EQ-5D Mean EQ-5D SD Truong[104] 2017 USA Advanced stage (III/IV) head/neck cancer patients, treated with chemotherapy + radiationa 199 Pre-treatment 0.82 0.14     178 2 weeks from end of tx 0.65 0.19     167 3 months after tx 0.77 0.15     147 1 year after tx 0.87 0.13     136 2 years after tx 0.91 0.12     117 3 years after tx 0.89 0.14     101 4 years after tx 0.87 0.16     64 5 years after tx 0.88 0.16 Pottel[105] 2015 Belgium Older HNC (65+) treated with RTb 81 Baseline 0.61 0.23        4th week of tx 0.47 0.24        2 months after tx 0.55 0.29        5 months after tx 0.58 0.31        1 year after tx 0.52 0.37        2 years after tx 0.4 0.4        3 years after tx 0.34 0.41 Govers[106] 2016 Netherlands Early-stage (T1-T2) oral cancer Cross-sectional evaluation of 4 different patient groups (by type of treatment)c 104 Full population 0.83 0.02       86 No previous recurrence or second primary 0.84 0.02       53 No adjuvant RT/chemo 0.85 0.03 Pickard[107] 2016 USA Stage III/IV head/neck cancers, retrospectively collected from clinical trials. All pts had received some chemotherapy 50 Full population 0.76 0.15 Rogers[108]  2006 UK Oral cancer patients previously treated with surgery 224 Full population 0.75 0.02 Noel[109] 2015 Canada Head/Neck cancer patients treated with surgery or RT within past 3 months-3 years Recurrence, metastatic disease removed 54 Primary surgery 0.83 0.19       13 Received chemo 0.76 0.17       87 No chemo 0.83 0.18       47 T1-T2 cancer 0.83 0.18       20 T3-T4 cancer 0.83 0.09       100 Full population 0.82 0.18 Marcellusi[110] 2015 Italy Head/Neck patients treated within past 20 months 79 Full population 0.69 0.3 Del Barco Morillo[111] 2016 Spain Metastatic head/neck cancer patients in clinical triald  Baseline 0.7 N/A       Follow-up 0.6-0.8 N/A SD – standard deviation; tx – treatment; RT – radiation therapy a – Numbers presented reflect people with p16-positive disease; b – mean values provided through personal communication with author, Sept 28th 2017; c – values presented represent people who had neck dissection as largest subgroup presented (differences between subgroups were not significant); d – median values presented in paper, no means or SDs OPLs are associated with less QoL impairment than invasive cancers, which in turn are less debilitating than recurrences, which suggests that intervention early in the development of the disease may yield substantial benefits in terms of avoided morbidity and mortality[112].  32  2.3 Review of Health Economics Literature in Oral Cancer One of the key criteria for the development of a WDM is that the disease of interest is not adequately explored within the existing literature[31]. Indeed, if models exist that can adequately span the breadth of all decisions within the disease, it is difficult to justify the time and effort required to construct a WDM. In order to demonstrate that this dissertation project fills a necessary gap in the health economics modeling literature, a review of the existing literature was undertaken. 2.3.1 Search Strategy The objective of the search was to identify all published examples of health economic exercises in oral cancer. In order to do this, the literature was searched using a search strategy designed to capture all relevant studies. Eligible studies were those that reported any economic outcome (i.e., cost, cost-effectiveness, cost-utility) from an original investigation of a population of oral cancer patients – this would exclude, for example, reviews of economic exercises. Exercises where oral cancer patients were part of a larger study cohort (e.g., head & neck cancer patients generally) were also excluded, unless the economic indices for those patients was reported separately. Because oral cancers are often classified in the literature within the larger umbrella of “head and neck cancers”, the search strategy involved finding all exercises with a “head and neck” focus that may include oral cancers, as well as oral cancers specifically. The search strategy used in this chapter was based on three recently-published health economic literature reviews[113-115], using similar inclusion and exclusion criteria, search terms, and scope. 33  The EMBASE, and PubMed databases3 were searched to identify eligible studies. Results from searches were included based on manuscript titles. The search terms for the review are detailed in Appendix A  Abstracts of included papers were reviewed by the candidate to exclude any paper that did not include any economic outcomes. The full text of all remaining papers was reviewed to determine whether they included cancers of the oral cavity (cancers of the oral cavity and oropharynx were considered ‘oral cancers’ for the purposes of the review). 2.3.2 Results from Literature Review The titles from a total of 1531 potentially-eligible papers were reviewed, yielding 185 potentially-relevant papers. Based on abstract review, 91 health economic exercises in head & neck (including oral) cancer were included in the review. Of those, 50 did not specify results for oral cancer separate from the general population of head and neck cancers, yielding a final list of 40 economic evaluations of oral cancer. These findings are described in Figure 2.1.                                                            3 The Cochrane Library database was reviewed as well. Seventeen potential results were found from the identified search terms, and none met the eligibility criteria. 34  Figure 2.1 – Literature Review PRISMA Diagram  Of these exercises, a sizeable majority (76%) used methods other than modeling to produce cost or cost-effectiveness estimates. The nature of these studies is summarized in Table 2.2. Table 2.3 summarizes the nine remaining studies that used some form of decision modeling. They will be described in greater detail below:   35  Table 2.2 – Health Economic exercises in oral cancer from literature review Study Year Country Study Type Study Population Modeling? Wesley[116] 1992 India CEA Prospective No Funk[117] 1998 USA Costing Prospective No Smeele[118] 1999 Canada Costing Retrospective No van Agthoven[119] 2001 Netherlands Costing Retrospective† No van der Meij[120] 2002 Netherlands CUA Modeled Decision Tree Zavras[121] 2002 Greece Costing Retrospective No Nijdam[122] 2004 Netherlands Costing Prospective No Smeele[123] 2006 Canada Costing Prospective No Speight[124] 2006 UK CUA Modeled Markov Hollenbeak[125] 2007 USA Costing Administrativeǂ No Nijdam[126] 2007 Netherlands Costing Prospective No Epstein[127] 2008 USA Costing Administrative No Lin[128] 2008 Taiwan Costing Administrative No Subramanian[129] 2009 India CEA RCT No Han[130] 2010 China Costing Prospective No Lee[131] 2010 Taiwan Costing Administrative No Dedhia[132] 2011 USA CEA Modeled Markov Kim[133] 2011 UK Costing Prospective† No Jacobson[134] 2012 USA Costing Administrative No Park[135] 2012 South Korea Costing Prospective† No Govers[136] 2013 Netherlands CUA Modeled Markov Li[137] 2013 USA Costing Administrative No O’Connor[138] 2013 EU Costing RCT No Goyal[139] 2014 India Costing Prospective No Lee[140] 2014 USA Costing Administrative No Sheets[141] 2014 USA Costing Retrospectiveǂ No Enomoto[142] 2015 USA Costing Administrative No Govers[143] 2015 Netherlands CUA Modeled Markov Hollenbeak[144] 2015 USA Costing Administrative† No Lee[145] 2015 Taiwan Costing Administrative No Simons[146] 2015 Belgium CUA Prospective† Markov Acevedo[147] 2016 USA CEA Modeled Markov Cromwell[148] 2016 Canada CEA Modeled Markov Forner[149] 2016 Canada Costing Retrospective No Perrier[150] 2016 France Costing Prospectiveǂ No van der Linden[151] 2016 Netherlands CUA RCT Markov Jayakar[64] 2017 New Zealand Costing Prospective† No Keeping[152] 2017 UK Costing Retrospective† No Khoudigian-Sinani[153] 2017 Canada CEA Modeled Decision Tree Kumdee[154] 2018 Thailand CEA Modeled Markov CEA – Cost-effectiveness analysis; CUA – Cost-utility analysis; RCT – Randomized control trial † – cost values in this study were based on a subgroup of head and neck cancer patients ǂ – cost values in this study were derived from a regression analysis of a head and neck cancer study sample    36  Table 2.3 – Characteristics of decision models used in economic evaluations of oral cancer Study Model Type A B C D E F G H I Acevedo[147] Markov          Cromwell [148] Markov          Dedhia[132] Markov          Govers (2013)[136] Markov          Govers (2015)[143] Markov          Khoudigian-Sinani[153] Decision Tree          Kumdee[154] Markov          Simons[146] Markov          Speight[124] Markov          van der Linden[151] Markov          van der Meij[120] Decision Tree          The model could be used to evaluate changes in management of: A – Healthy Individuals; B – Oral Premalignancy; C -Primary Treatment; D – Post-Treatment Remission; E – Recurrence Treatment; F – Post-Recurrence Remission; G – Second Recurrence; H – Persistent/Metastatic Disease; I – End of Life van der Meij et al. (2002) The model in this exercise is designed to evaluate screening among people with oral lichen planus – a condition of the oral epithelium. The authors believed that people with oral lichen planus may be predisposed toward oral cancer, and that screening within this population would be more cost-effective than population-level screening of healthy individuals. The decision tree considered the costs and outcomes associated with screening, oral cancer treatment (grouped into ‘stage I’ and ‘stage II+’), but could not incorporate any other policy changes. The model also did not consider premalignant lesions aside from oral lichen planus.  Speight et al. (2006)[124] The Markov model developed by Speight and colleagues is among the most comprehensive in the available literature. The model considers the trajectory of a population of cancer-free people, some of whom have a premalignant lesion. These people may develop cancer, which may progress through four stages and be detected either through regular screening or symptomatic presentation. The model does not, however, consider changes that may occur after the initial detection and treatment of cancer, including recurrence. Despite its limitations, the model is well-supported by evidence and considers a great deal of the full breadth of the disease.  Dedhia et al. (2011)[132] 37  The Markov model developed by Dedhia and colleagues is similarly comprehensive, albeit less intricate than the Speight model. This model considers the trajectory of a population of high-risk men (i.e., men who regularly use tobacco and/or alcohol) being screened for oral cancer. The model considers movement from full health to premalignant disease, to early-stage (stage I/II) and late-stage (stage III/IV) cancer. Like the Speight model[124], however, outcomes and decisions affecting recurrence (or after) are not modeled. Govers et al. (2013)[63] This cost-effectiveness analysis of neck dissection in early-stage cancers incorporates a treatment decision tree into a three-state Markov model (regional failure, no regional failure, death). No interventions before or after treatment can be incorporated into the model. Govers et al. (2015)[143] The scope of this model was similar to the previous exercise, albeit in a different patient population (metastatic disease rather than early-stage disease). The Markov component of the model had a death state and four living health states: failure/no failure, shoulder morbidity/no shoulder morbidity. As in the previous example, the model did not consider the impact of decisions before or after the decision scenario of interest. Simons et al. (2015)[146] This model technically did not concern oral cancer specifically, but rather included oral cancer patients as a subset of a hypothetical cohort of head and neck cancer patients. The model is comprised of four health states, all occurring after the diagnosis and treatment of non-metastatic disease – progression-free survival, recurrence, metastatic disease, and death. The model does not include the development of de novo disease, either from full health or from premalignant disease. Importantly, it also did not include a treatment decision node (treatment efficacy was assumed and adapted to disease site by relative risk), but could theoretically be adapted to include one. Cromwell et al. (2016)[148] Further information about this model will be presented in Chapter 3 of this dissertation. Briefly, this model included states for premalignant disease management through to 38  invasive/recurrent/persistent disease. End-of-life disease management was not considered within the model, nor were people with no (or undiagnosed) oral disease. Acevedo et al. (2016)[147] This Markov model concerned oral cancer patients with node-negative disease, evaluating the use of elective neck dissection to prevent recurrence. This model had four disease states – no evidence of disease (following initial treatment), salvaged after recurrence, multiple recurrence or metastatic disease, and death. The model did not consider disease states before or including initial treatment, and did not explicitly consider end-of-life treatments. van der Linden et al. (2016)[151] This model of sentinel node biopsy in early stage node-negative cancers is structurally identical to the model published in Govers[63], and does not consider decisions outside of incident treatment. Khoudigian-Sinani et al. (2017)[153] This decision tree concerns the diagnosis of oral cancer, using a tool that is potentially less subjective than current standard of practice (biopsy and histopathology). The only health state considered in this analysis was post-treatment survival, and the model did not consider decision nodes outside diagnosis and treatment. Kumdee et al. (2018)[154] This Markov model, examining oral screening in Thailand, was based on the structure of the Speight model[124], with epidemiological and treatment information adapted to fit a Thai population. From the literature described above, only a small number of oral cancer models have been published, most of which are limited to treatment-related decisions. The model by Speight is the most comprehensive and well-evidenced in the existing literature, despite being one of the oldest. Only one study found in the review evaluated any decisions concerning recurrent disease, and explicitly considered second recurrence as an outcome. One model in the review considered the impact of changes in end-of-life care, although this period of life is notoriously difficult to characterize[155]. 39  These models were created in a variety of different modeling software packages – Excel, TreeAge, R –  meaning that combining or otherwise adapting the existing models is challenging. It is also worth noting that all of the currently-existing models are either decision trees or Markov models, with the limitations that accompany those approaches (especially the lack of memory of previous treatments). It seems reasonable to conclude that a whole disease approach would require the creation of an entirely new model, rather than a simple adaptation/combination of ones that currently exist. 2.4 Structure of Models of Oral Cancer The goal of a whole disease model is to represent the full breadth of the disease, from full health through to death. A whole disease model is thereby capable of evaluating potential policy changes at multiple points during disease management. The structure of the models found in the literature search was analyzed to investigate their breadth. In this analysis, the breadth of oral cancer was conceptualized in terms of potential decision nodes: A. Healthy Individuals: Does the model consider the impact of changes to screening of healthy individuals for oral disease? B. Oral Premalignancy: Does the model allow for changes in the management of people with oral disease that may develop into cancer at a later date? C. Primary Treatment: Does the model evaluate the impact of different ways of managing incident cancers (at any stage)? D. Post-Treatment Remission: Does the model allow for changes in follow-up and post-treatment management after initial treatment? E. Recurrence Treatment: Can the model evaluate changes in the treatment of local or regional recurring disease? F. Post-Recurrence Remission: Does the model allow for changes in follow-up and post-treatment management after recurrence? G. Second Recurrence: Can the model evaluate changes in the treatment of second recurrences? H. Persistent/Metastatic Disease: Does the model allow for changes in the way that incurable disease is managed? I. End of Life: Can the model be used to evaluate changes occurring in treatments occurring at the end of a patient’s life? 40  These decision nodes, and their relationship to disease trajectory, are described in Figure 2.2.  N.B. – “Dead from other cause” can occur from within any health state. Based on this investigation of the literature, no model currently exists that can address all nine of these decision nodes, though a number of them could theoretically be adapted to address as many as six. This suggests that the currently-available modelling literature is unable to reflect the full impact of policy changes that occur upstream or downstream from the decision nodes in each (i.e., the policy question being evaluated). Three models consider newly-developed but undetected oral premalignancy and Figure 2.2 - Conceptual model of disease and decision nodes 41  simulate the trajectory to primary treatment, but cannot reflect the impact that changes in technologies may affect rates of primary and secondary disease recurrence, or interventions at the end of life. With two exceptions, no available model considers these downstream events in a way that would allow economic evaluation of interventions therein. The available literature is therefore poorly equipped to estimate the economic impact of technologies whose influence is experienced late in the disease trajectory, nor is it able to adequately reflect cumulative changes that may occur as a result of the simultaneous adoption of multiple technologies. Examples of such technologies are summarized in the following section. 2.5 Potential Emerging Health Technologies in Oral Cancer In addition to a paucity of available decision models, another important criterion for the creation of a WDM is that there should be a number of new technologies in need of health economic evaluation[31]. A chief strength of the WDM approach is that it can evaluate the impact of the introduction of multiple decisions simultaneously. Accordingly, it is worthwhile to highlight some emerging technologies in oral cancer management. As discussed in section 2.1.1, the development of oral premalignancy and cancer is associated with the use of alcohol and tobacco. Health Canada[156] and the World Health Organization[157] have identified tobacco use reduction as public health priorities, while the Canadian Cancer Society advises Canadians to limit their alcohol intake as a means of reducing cancer risk[158]. In addition to reductions in lung cancers and heart disease, programs aimed at reducing population levels of alcohol and tobacco intake would likely affect oral premalignancy prevalence, and the development of invasive cancers.  Improvements in oral cancer screening could potentially shift the distribution of detected cancers toward earlier disease, resulting in improvements in survival[77]. Despite its promise, population-based 42  oral cancer screening faces several challenges, not the least of which is the cost of implementing screening for a disease that is relatively rare among the general population[124]. The identification of subgroups that are particularly susceptible to disease may result in a more favourable cost-effectiveness profile and easier implementation of organized screening. Additionally, new approaches to screening are available to oral health professionals, including toludine blue staining, brush cytology, and the use of chemiluminescent and autofluorescent tools[58]. There is a great deal of uncertainty around the progression of OPL to invasive disease. Current standards of practice may over-treat some diseases that will never develop into life-threatening disease, while at the same time under-treating some premalignancies that would be life-extending if treated immediately. Many potential treatments for OPL have been proposed, but the evidence supporting them is weak and mixed[159]. The emerging genetic evidence concerning the risk associated with LOH suggests a future path for new risk-stratified methods of managing OPL[61]. The original impetus of this dissertation began with the inception of the pan-Canadian Optically-guided Oral Lesions Surgical (COOLS) Trial, investigating the use of autofluorescent technologies in the surgical theatre[62]. Further developments in risk-prediction and surgical management are currently undergoing evaluation[160]. Genomic advances may help predict patient response to different types of treatment, allowing oncologists to take increasingly personalized approaches to oral cancer management[56, 161]. Novel chemotherapeutic agents may dramatically improve survival in advanced and recurrent cancers[55]. Any of these technological changes could have large-scale impacts on patient health and resource utilization/allocation within the health care system. None of them have undergone formal economic evaluation. Evaluations of any one of these technologies may be impacted by upstream and/or 43  downstream changes, particularly if multiple technologies are adopted simultaneously. The currently available modeling literature does not have the capacity to reflect these types of changes, suggesting the need for a more robust and adaptable model. The WDM framework is the appropriate method to address this challenge.  2.6 Discussion The literature review conducted in this chapter highlights many of the challenges inherent in conducting economic evaluations to inform oral cancer policy-making. There are a wide variety of potential places where clinical and public health policy changes could impact the incidence and survivability of oral cancer, from health behavior changes and early detection through to interventions at invasive and advanced disease. Oral cancer’s relatively low incidence also means that economic evaluation alongside clinical trials is a prohibitively resource- and labour-intensive approach to answering these health economic questions, especially given the pace with which the policy environment seems poised to change. As these technologies enter current practice, the models that do exist will become increasingly out-of-date, and their recommendations will increasingly cease to accurately reflect standard treatment approaches. The literature search method used in this chapter had important limitations to consider. While the search strategy was based on previously-conducted literature reviews, this review did not consider all possible repositories for health economic literature (i.e., indexed databases other than Medline and EMBASE, manual review of the sources cited in the included papers, grey literature). The decision models discussed in this review did not undergo a quality appraisal process, and the candidate was the sole reviewer. Consequently, it is possible that this review does not include an exhaustive list of all economic evaluations performed in an oral cancer context. Despite these limitations, the review 44  suggests that the existing health economics modeling literature does not seem well-equipped to evaluate the number of rapidly-emerging technologies and innovative approaches to oral cancer control. 2.7 Conclusion This chapter describes the development, management, treatment, survival, and morbidity of oral cancer from premalignancy through to advanced disease. A review of the available literature suggests that while decision models in oral cancer exist, there are notable limitations to the types of health economic questions they are equipped to address. These limitations suggest that policy makers will not have sufficient evidence to evaluate the cost-effectiveness impact of the adoption of novel health technologies, especially if those technologies are adopted concurrently. A modeling approach is needed that reflects the full breadth of disease and the impact that different policy decisions may have, both individually and in tandem. Such a model will need to be both powerful and able to adapt alongside the evolution of practice. It will also need to be able to reflect the diversity of demographic and health system factors that influence disease incidence and practice, as they change between different health jurisdictions. The following chapter describes, in detail, the challenges and limitations inherent in piecewise decision modeling.45     3 Chapter 3 – Cost-Effectiveness Analysis of Genome-Guided Management of OPLs using Markov Decision Modeling This chapter will describe a cost-effectiveness analysis exercise in oral cancer using a conventional Markov modeling approach. This exercise estimates the incremental costs and outcomes of using a genomic assay to guide management of oral premalignant lesions (OPLs), compared to a conventional management approach within the context of BC Cancer (BCCA). The limitations of this modeling approach will be discussed at the end of this chapter, for the purpose of explaining why a more comprehensive modeling approach (i.e., a Whole Disease Model approach; WDM) is useful and necessary in the context of this disease. 3.1 Introduction Cancers of the oral cavity have an age-standardized incidence rate of 9% in Canada[162], with similar rates experienced in other countries with industrialized economies like the United States[163] and United Kingdom (UK)[164]. Despite its relatively low incidence (compared to malignancies of the colon or breast), oral cancers have high case mortality with an overall five-year survival rate of 60-63%[165]. Early detection has a meaningful impact on survivability – locally controlled oral cancers have five-year survival rates of 75-93%; cancers that have spread to other tissue sites have 20-52% five-year survival rates[166]. Given that more than 40% of oral cancers are diagnosed at late stages with either regional or distant diseases, the argument for early detection is a strong one – early detection increases the proportion of early-stage, curable cancers[167]. 46     Early detection of lesions in the oral cavity is usually performed by a community dentist[168, 169]. Suspicious lesions are referred for diagnostic biopsy, where they may be identified as a low-grade dysplasia (LGD; mild or moderate dysplasia) and monitored on an ongoing basis, or as a high-grade dysplasia which is referred for treatment (usually surgery)[169]. The majority of the LGDs will not develop into cancer and the incorporation of better risk identification techniques into routine oral health management is recommended[168]. A recent prospective study showed that a specific molecular panel of biomarkers, using loss of heterozygosity (LOH) is the most significant predictor of progression of an OPL to an invasive cancer, superseding clinical and pathological features[61]. Using this biomarker test, patients presenting with an OPL can be stratified into high-, intermediate-, or low-risk group that corresponds to the likelihood of developing cancer. Theoretically, patients in the “low” or “intermediate” risk category may receive less frequent follow-up monitoring without appreciably increasing their risk of developing cancer, which would change their pattern of health care system resource use from current standard of care. The economic evaluation of risk-guided OPL management can be addressed through conventional Markov modeling methods. This modeling approach was chosen because it is typical for health care decision analysis of this type and in this disease area, as described in Chapter 2. The purpose of this chapter is to evaluate risk-guided management using this conventional method, and to explore the ways in which it may be insufficient to address the policy question. 3.2 Methods 47     Cost-effectiveness analysis was performed using a conventional Markov modeling approach. Incremental cost-effectiveness ratios (ICERs) were calculated based on the costs and outcomes of the model. Probabilistic sensitivity analysis (PSA) was performed to investigate the impact that uncertainty in model parameters had on the cost-effectiveness of the change in management. 3.2.1 Markov Model A cohort-based Markov model was constructed in the R language (R Foundation for Statistical Computing, Austria). The model simulated a hypothetical cohort of people diagnosed with an OPL in the province of British Columbia (BC), Canada between 20 and 80 years of age. Nearly all cancers in BC are managed within the auspices of BC Cancer (BCCA), a provincial entity responsible for cancer care and research. Cancer treatment is provided by oncologists, pathologists, and other cancer care professionals within the BCCA. Oral precancerous lesion care across BC is primarily conducted in Vancouver at Vancouver General Hospital (VGH) or at the BCCA Oral Oncology Clinic (OOC) [170]. The model designed for this exercise had two arms (Figure 1): an “Assay Informed” arm in which the schedule of follow-up and management for a person with an OPL was informed by the results of the molecular test, and an “Assay Naïve” arm in which people with OPLs received care according to current standard of practice. The model structure is described below and illustrated in Figure 3.1.  48     Follow-up in the Assay Informed arm was scheduled according to a person’s risk group – either “low”, “intermediate”, or “high”. People in the “low-risk” group returned for a re-appraisal of their lesion (including the assay and biopsy) every five years. People in the intermediate-risk group were assessed (with assay and biopsy) every two years. People in the “high-risk” group were treated with surgery immediately, as though they had a high-grade precancerous lesion (HGL; severe dysplasia or carcinoma in situ). It was possible in the model to be diagnosed with cancer during any follow-up appointment – this cancer may be an HGL or a squamous cell carcinoma (SCC). People in the Assay Naïve arm returned for a follow-up appraisal of their lesion every six months, regardless of risk group. Cancer (either HGL or SCC) could be diagnosed at any follow-up appointment. Figure 3.1  – Structure of the Markov Model 49     People with a detected HGL were managed surgically. HGLs removed with positive surgical margins (i.e., evidence that cancerous cells exist within a margin of apparently-healthy tissue drawn around the lesion) required a second surgery. Patients with successfully treated HGL were followed every six months for five years, after which point they were considered to be in remission, and were discharged from the health care system (i.e., no more follow-up visits). Patients whose HGL was not controlled by treatment were considered to have persistent/metastatic disease. People with a detected SCC were managed either through surgery or with external-beam radiation therapy (XRT). Following treatment, patients were followed up to detect recurrence of their disease. Patients experiencing a recurrence were considered to have persistent/metastatic disease, and were treated with chemotherapy and palliative care. Patients who lived five years beyond their initial diagnosis with no recurrence were considered to be in remission. People with persistent/metastatic disease state were managed with palliative care until they die of oral cancer. People in all states (pre-cancer, locally controlled disease, remission, persistent/metastatic disease) could die from causes unrelated to cancer. The Markov model had a time horizon of ten years, meaning that all simulated patients either had a resolved OPL, were in remission (cancer-free for 5 years after diagnosis) or had died of cancer or another cause. The cycle length of the model was six months. The cycle tree method was used for half-cycle correction[171]. A health care system perspective was adopted for costs in this study. 3.2.2 Transition probabilities Risk stratification into “high”, “intermediate” and “low” groups was estimated based on results from the Oral Cancer Prediction Longitudinal Study[61]. The associated risk of developing cancer for these risk 50     groups was taken from the same study[61]. The probability of moving into another risk category (i.e., from low to intermediate, from intermediate to high) was assumed to be zero (0%) for this exercise – this assumption was examined in sensitivity analysis. This study used BC Cancer Registry data to identify a retrospective cohort of 148 patients who had developed an oral cancer from a monitored OPL in British Columbia between February, 2004 and November, 2011. Data from this cohort was used to estimate the probability of developing an HGL or an SCC. The probability of requiring a second surgery for HGL and of local control following treatment for HGL was also estimated from this dataset. This study also used data from a second retrospective cohort of 864 patients diagnosed and treated for SCC in BC between January, 2000 and September, 2009. These data were used to estimate the proportion of SCCs treated primarily with surgery vs. with XRT. The proportion of SCC surgeries requiring neck dissection was estimated from preliminary (blinded) data from the pan-Canadian Optically-guided Oral Lesions Surgical (COOLS) Trial[62]. Risk of SCC recurrence[172], of death from persistent/metastatic oral cancer[173], and relative risk of death according to age[174] were estimated from published studies. Age-specific death rates published by Statistics Canada were used to estimate the probability of dying from causes other than cancer from all health states[175]. 3.2.3 Costs The cost of the assay was assumed to be $500 (all costs expressed in 2013 Canadian dollars – CAD). The cost of medical appointments, biopsy, surgical resection, and neck dissection were taken to be the medically insured cost of a doctor’s visit as established in the provincial Medical Services Plan (MSP) fee 51     schedule[176]. Additionally, the cost associated with a patient’s out-of-pocket expenses such as travel and accommodations were included, estimated based on preliminary (blinded) results of 383 patients from the COOLS Trial[62]. Participants in the study were issued a questionnaire that asked them to respond to questions about distance travelled, method of travel, and any other expenses incurred as a result of their visit to the OCC. A fixed cost of $0.50 per kilometer was applied to trips taken by car based on reimbursement values used by the BC Provincial Health Services Authority (PHSA). The cost of XRT was estimated by applying a fixed per-fraction cost of $325.50 to a schedule of 25-30 fractions per person. The per-fraction cost is based on budgetary numbers from the Vancouver Cancer Centre. The cost of chemotherapy was based on a health economic study conducted by Hannouf and colleagues[177], which used a costing model that synthesized data from hospital drug formularies and the Ontario Case Costing Initiative. The cost of resources used in follow-up surveillance for locally controlled cancers were based on the MSP fee schedule. The cost of the first twelve months and all subsequent months of persistent/metastatic disease was taken from a hospital-based cohort study conducted by Speight and colleagues in the United Kingdom (Canadian estimates were not available in the literature)[124]. Costs, originally published in 2006 UK Pounds (£), were first converted to Canadian dollars based on the mid-year currency exchange rate, then inflated according to the Consumer Price Index to 2017 Canadian dollars. Costs following cancer remission were assumed to be zero. 3.2.4 Health State Utilities 52     Health state utilities were applied to each state in the model. Utilities reflect a person’s preference for a health state, anchored between 1.0 (perfect health) and 0.0 (equivalent to death). Estimates for each health state were taken from a study conducted by Downer and colleagues, based on a standard gamble exercise conducted in a convenience sample of 100 members of the general public in the UK[178]. Health utility experienced by people in remission was assumed to be 1.0. A summary of all values used in the model is provided in Table 3.1. Table 3.1 – Parameter inputs used in the Markov model Description Value (SE) Distribution Source Probabilities Proportion of people who are "low risk" 0.47 0.03 Dirichlet Zhang (2012) Proportion of people who are "intermediate risk" 0.43 0.03 Dirichlet Zhang (2012) Proportion of people who are "high risk" 0.10 0.02 Dirichlet Zhang (2012) Probability of moving up to a new risk category 0   Assumption Rate of developing cancer in "low risk" group 0.031 over  5 years 0.023 Beta Zhang (2012) Rate of developing cancer in "intermediate risk" group 0.163 over  5 years 0.036 Beta Zhang (2012) Rate of developing cancer in "high risk" group 0.631 over  5 years 0.090 Beta Zhang (2012) Probability of cancer being SCC 0.68 0.01 Beta SCC cohorta Probability of HGL treated with surgery 1.0 -  Precancer cohortb Probability of HGL surgery requiring neck dissection 0 -  Assumption Probability of second surgery for HGL 0.02 0.01 Beta Precancer cohortb Probability of locally controlled HGL after treatment 1 - Beta Precancer cohortb Probability of SCC treated with only surgery 0.71 0.02 Beta Precancer cohortb Probability of SCC surgery requiring neck dissection 0.371 0.0317   Rate of SCC recurrence in first year 0.2 0.03 Beta Ganly (2013) Rate of SCC recurrence after first year 0.1 over 4 years 0.03 Beta Ganly (2013) Oral cancer mortality rate 0.681 over 5 years 0.04 Beta Mucke (2009) Relative rate of cancer death, age <55 1.0 - Ref.  Relative rate of cancer death, age 55 – 64 1.5 0.43 Normal Rogers (2009) 53     Description Value (SE) Distribution Source Relative rate of cancer death, age 65 – 74 1.6 0.46 Normal Rogers (2009) Relative rate of cancer death, age 75+ 3.4 0.97 Normal Rogers (2009) Costs Cost of genetic assay 500 -  Assumption Direct cost of medical appointments 250 -  MSP code 03770 Indirect cost of medical appointments 67.15 306.22 Gamma COOLS Trialc Cost of biopsy 150 -   Cost of resection 1889.27 -  MSP – fee code 02279 Cost of dissection 1231.05 -  MSP – fee code 02470 Cost of course of RT 325.5   BCCA Costing exercise Number of courses of RT 27.5 0.7 Normal BCCA Clinical guidelines Cost of course of chemotherapy 4478 750 Gamma Hannouf (2012) Cost of asymptomatic follow-up 75 -  MSP, BCCA Consultancy fee Cost of 1st 12 months with metastatic disease 11,639 16,719 Gamma Speight (2006) Cost of subsequent 12 months with metastatic disease 2150 8940 Gamma Speight (2006) Costs after 5 years of cancer-free survival 0 -  Assumption Utilities Utility for pre-cancerous lesion 0.92 0.18 Beta Downer (1997) Utility for locally controlled cancer 0.88 0.20 Beta Downer (1997) Utility for persistent/metastatic disease 0.68 0.33 Beta Downer (1997) Utility for disease in remission 1.0 -  Assumption a – a retrospective cohort of 864 people diagnosed with squamous cell oral carcinoma in British Columbia between January, 2000 and September, 2009. b – a cohort of 148 patients who had developed oral cancer from monitored precancerous lesions in British Columbia between February, 2004 and November, 2011. c – intervention-blinded survey data from an ongoing clinical observation of 400 people newly-diagnosed HGL or SCC MSP – British Columbia Medical Services Plan; COOLS – Canadian Optically-guided Oral Lesions Surgical Trial; BCCA – British Columbia Cancer Agency  3.2.5 Cost-Effectiveness Analysis The difference in total years of life between the two arms was defined as the incremental effectiveness (ΔE) as life years gained (LYG). Incremental effectiveness was also expressed in quality-adjusted life years (QALYs) – i.e., number of years spent in each health state, multiplied by the utility associated with that health state. 54     The sum of all costs experienced by people in the model was calculated for both arms in a similar way. Incremental cost (ΔC) was defined as the difference between the sum of costs between the two arms. Costs and outcomes (LYG, QALYs) were discounted annually at a rate of 1.5% to reflect time preference[5]. ICERs were calculated as the ratio of incremental costs to incremental effectiveness (ΔC/ ΔE), expressed as cost (in dollars) per LYG and per QALY. ICERs are typically compared to a ‘threshold’ value, denoted as (λ) that represents policy makers willingness to pay for an additional LYG or QALY. If the ICER is below λ, the associated intervention or program is considered to be cost-effective. 3.2.6 Probabilistic Analysis Probabilistic analysis was performed using Monte Carlo Simulation. A total of 10,000 iterations were drawn from the input distributions (see Table 3.1) to generate a range of ICERs. The ICERs were plotted on the cost-effectiveness plane[179]. The cost-effectiveness plane is divided into four quadrants, representing positive/negative incremental cost (on the Y axis) and positive/negative effectiveness (on the X axis). ICERs associated with new technologies are commonly found in the north-east quadrant (i.e., costs more and is more effective, compared to current practice). Since the value of λ varies across policy making contexts, it is often useful to consider the proportion of PSA-sampled ICERs that lie below the threshold (i.e., percentage of ICERs that are cost-effective) at various levels of willingness to pay. This is done through the use of Cost-Effectiveness Acceptability Curves (CEACs)[19]. These curves illustrate the value of λ at which a certain percentage (such as 50% or 55     95%) of ICERs lie – suggesting the level of willingness to pay that an intervention is, for example, 95% likely to be cost-effective. 3.3 Results Costs, survival, and incremental cost-effectiveness are described in Table 3.2. Table 3.2 – Summary of Cost-Effectiveness results Arm Estimate Mean 95% CI Assay Informed       Cost $3,198 2,648 – 4,579  LYG 8.98 7.89 – 9.30  QALY 8.33 3.67 – 9.22 Assay Naive       Cost $7,239 5,676 – 12,210  LYG 8.94 7.84 – 9.28  QALY 7.85 3.13 – 8.79 Incremental       Cost -$4,041 -7,972 – -2,785  LYG 0.043 0.014 – 0.099  QALY 0.480 0.337 – 0.655  Cost/LYG Assay Informed Dominates  Cost/QALY Assay Informed Dominates   3.3.1 Cost The mean per-person cost of oral pre-cancer and cancer management was $3,198 per patient (95% CI: 2,648 – $4,579) in the Assay Informed arm, compared to $7,239 per patient in the Assay Naïve arm (95% CI: 5,676 – 12,210). Costs for both cohorts were primarily generated in the asymptomatic phase, by people who did not develop cancer – 68% (SD: 12.1%) in the Assay Informed arm, and 67% (SD: 18.6%) 56     in the Assay Naïve arm. A summary of the proportion of total cost represented by each state can be seen in Figure 3.2.  Figure 3.2 – Proportion of costs attributable to each model state  3.3.2 Effectiveness People managed according to the Assay Informed protocol experienced an average of 8.98 LYG (95% CI: 7.89 – 9.30), compared to 8.94 LYG (95% CI: 7.84 – 9.28) in the Assay Naïve arm. The Assay Informed arm had an average of 8.33 QALY per person (95% CI: 3.67 – 9.22), compared to 7.85 QALY (95% CI: 3.13 – 8.79) in the Assay Naïve arm. 3.3.3 Cost-Effectiveness Incremental cost between the two model arms was -$4,041 (95% CI: -7,972 – -2,785). Incremental effectiveness was 0.043 LYG (95% CI: 0.014 – 0.010), or 0.480 QALYs (95% CI: 0.337 – 0.655). Use of the 57     assay dominated – i.e., cost less and was more effective than – standard care in this model. A summary of these results can be seen in Figure 3.3. 58       Figure 3.3 – Probabilistic Analysis: incremental results from 10,000 model runs 59      Incremental costs and effectiveness were plotted on the cost-effectiveness plane (Figure 3.4). ICERs tended to fall in the southeast quadrant (less costly, more effective), with a few in the northeast quadrant (more costly, more effective).  Figure 3.4 – Cost-Effectiveness Plane for A) cost per LYG and B) cost per QA   60     3.4 Discussion This chapter’s findings suggest that a genetic assay with the capability of determining the risk that people with precancerous oral lesions will progress to develop oral cancer is cost-effective if it allows for different schedules of patient follow-up. A Markov model was used to estimate the costs and effectiveness of such an assay, in which ‘low-risk’ and ‘intermediate-risk’ patients were seen on a schedule elongated from standard care. Patients identified as ‘high-risk’ had their lesions resected immediately to reduce the incidence of oral cancer. Under this scenario, overall costs to the health care system were lower and average patient survival was higher (i.e., use of the assay dominated standard care). The cost-effectiveness findings were primarily driven by two factors. First, the reduction in cost was largely due to the reduced number of clinic visits among people who did not develop cancer. Second, people who were at high risk were treated immediately with a very high predicted cure rate. As a result, the rate of cancer morbidity and mortality (with associated costs) was lower in the assay-informed arm. Our model showed that, by using the assay and the adjusted schedule, the rate of oral cancer decreased by an average of 51.1%, and people who were “high risk” had a decreased mortality rate of 12.7%. People who were ‘high-risk’, and most likely to go on to develop cancer, represented only 2% of the total cost in the ‘standard care’ arm, while those who were ‘low-risk’ were responsible for 38%. Under the “assay informed” scenario, that proportion of total cost dropped to 25%. This suggests that a population who would not be expected to have appreciably different survival could be expected to have dramatically lower health care costs. 61     3.4.1 Limitations Any model is a simplification of reality, and this model is no exception. The clinical management of oral cancer is more complex than could be feasibly represented here. While the model structure allowed for the possibility of two surgeries for HGL, it does not treat local recurrence differently from regional or distant recurrence. Locally-recurring oral cancers may be successfully treated, allowing the possibility of long-term remission. All recurrences are assumed to be regional and terminal, which likely overestimates the risk and costs of treating cancer.  The model also does not account for stage progression of undetected (or indeed, detected) cancers. For the sake of simplicity, the model assumes that cancers detected during screening are early-stage (i.e., stage I) and that they do not progress to a later stage during the screening intervals. This assumption was made in order to simplify the complex natural history underlying oral cancer management, and was reflective of the fact that most patients undergoing regular observation will have their cancers detected at an early stage – it is unlikely that a person’s disease would progress to a late stage while they are under regular surveillance. This assumption means, however, that progression of interval cancers will not be reflected in estimates of either costs or effectiveness. While it is theoretically possible to incorporate the growth of cancers during the screening interval, doing so would dramatically increase the complexity of the model’s structure and data needs (a phenomenon sometimes termed ‘state explosion’).  Finally, for similar simplifying reasons, the model does not reflect the way that individual demographic characteristics of the simulated cohort affect their clinical trajectory. While cancer and non-cancer 62     mortality were adjusted to reflect population aging, it was not possible to adjust other important factors like OPL progression, treatment efficacy, or recurrence rates, in the same way. Furthermore, other factors known to be related to oral cancer disease progression and severity – smoking, alcohol use, and sex – were not reflected in this analysis. The model necessarily assumes that the study populations from which parameter estimates were drawn are reflective of the general population, and it is not possible to investigate the effect that ‘upstream’ changes in those demographic factors might have on the cost-effectiveness of the genomic assay.  A Markov approach was chosen for this chapter as it most closely resembles the current state of the modeling literature with respect to oral cancer (see Chapter 2). A Discrete Event Simulation (DES) approach would be able to incorporate the effect of the demographic and clinical factors described above, but estimates of this type are not available in the available clinical literature. Estimating necessary parameter values to populate a DES model that includes these factors would require the collection and analysis of new data, or secondary analysis of existing data.  3.5 Conclusion The previously-described model suggests that using a genomic assay to risk-stratify the management of OPLs is less costly and more effective than the current standard of practice. The reduction in clinic visits for the majority of precancerous patients who will never go on to develop cancer will reduce health care expenditure, and the early identification and treatment of high-risk lesions will result in improved patient outcomes. This approach had several shortcomings, including the inability to reflect the impact 63     of decisions made upstream and downstream from the modeled pathways, and to reflect individual-level heterogeneity.  It is theoretically possible to address the shortcomings identified in the traditional Markov approach by using a WDM that incorporates DES functionality. By modeling the full disease trajectory, from undetected premalignancy through to terminal disease, the model can reflect structural factors that might influence detection and management of OPLs. Using the entity-based time-to-event modeling approach of a DES will allow the model to reflect how individual characteristics might affect the cost-effectiveness of the LOH assay. A WDM model that is more comprehensively parameterized will also be better equipped to incorporate more elements of clinical care than the model described in this chapter. The next chapter of this dissertation will describe the design and construction of a Whole Disease Model of Oral Cancer, which improves upon the model described in this chapter and better reflects the complexity of premalignant and invasive cancer care.  64     4 Chapter 4 – Design and Implementation of a Whole Disease Model of Oral Cancer This chapter will describe the design, implementation, and calibration of a WDM of oral cancer (WDMOC) that addresses the limitations of the existing modeling literature with respect to their ability to generate evidence to guide policy-making. 4.1 Introduction In Chapter 2, no models were identified that reflect the full complexity  of the oral cancer disease trajectory. The implication of Chapter 3 is that it is not possible to analyze the combined effect of multiple simultaneous policy changes without the ability to reflect how upstream policy changes affect, and are affected by, downstream ones. Existing models also lack the necessary level of parameter complexity to adequately reflect management of premalignancies and invasive cancers. Given the broad range of interventions possible within the oral cancer management pathway, and given the difficulty of using conventional modeling techniques to represent changes to that pathway, a novel approach to economic evaluations of oral cancer is warranted. The Whole Disease Modeling framework provides such a novel approach. The central purpose of the WDM framework is to model the breadth of the entire disease, including preclinical management and detection through treatment to the end of life. Whole disease models should be able to reflect the impact of multiple simultaneous policy and/or technology changes, while adhering to recommended guidelines for model design and implementation.  65     This chapter will describe the process through which the WDMOC was created, following methods described by Tappenden[31]. First, the process through which the model was conceived and designed will be described. Next, the model’s characteristics and basic architecture will be summarized. The implementation of the model – i.e., the specific way in which the model was programmed – will be described briefly as well. A description of the statistical techniques employed by the model, and how they were applied to the values and sources of the model’s parameters, will follow. Finally, the model’s baseline output will be described and compared to values observed in the real world to evaluate its validity. 4.2  Model Design and Theoretical Framework The conceptual framework for the WDMOC was designed drawing on the work by Tappenden[31] which defines a whole disease model as one that: 1. Includes preclinical and post-diagnostic pathways for individuals who may or may not develop a given disease in their lives; 2. Captures different service pathways from system entry to discharge or death; 3. Represents events, costs and outcomes, and structural relationships between these to a level of detail that allows the point at which technologies may change (decision node) to be transferred across the modeled pathway; 4. Allows for the economic evaluation of individual or multiple service changes WDMs are conceptually defined by three principal attributes:  Boundary: the populations represented within the model – the people who interact with and are affected by changes made within the system the model seeks to represent  Breadth: the phenomena, costs, and consequences included within the model – the types of processes, services, resources, and outcomes that the model will reflect 66      Depth: the level of detail used to describe and valuate each phenomenon, cost, and consequence included in the model. Three principles follow from these attributes: 1. The model boundary and breadth should capture all relevant aspects of the disease and its treatment – from preclinical disease through to death 2. The model should be developed such that the decision node is conceptually transferable across the model 3. The costs and consequences of service elements should be structurally related A whole disease model of oral cancer must represent the clinical experience of both people with oral cancer (and pre-cancer) and those who do not experience the disease but would be affected by changes in the system (i.e., the general population), in such a way that several potential changes can be evaluated simultaneously. The model must be granular enough to accurately reflect all relevant costs and outcomes, while being broad enough to estimate the impact that upstream changes will have on all downstream events. 4.3 Model Design process Designing a whole disease model is a five-stage process: 1. Understanding the decision problem: defining who will use the model and what types of economic questions it will be used to answer; 2. Conceptualization and design: building a conceptual representation of the processes that the model will simulate; 3. Implementation modeling: the creation of the model itself, using computer software. Implementation typically requires a time-to-event approach. This stage includes model calibration and uncertainty analysis; 4. Model checking: ensuring that the model entities and processes are behaving as expected. This is an ongoing process during model development; 67     5. Engaging with the decision: incorporating the results of the model into a policy making process and/or framework. The development of the WDMOC, through these stages, is described below. 4.3.1 Stage 1: Understanding the Decision Problem There are three key elements of this stage: establishment of a stakeholder group, immersion in relevant evidence, and agreeing what is to be evaluated and why. Establishment of a Stakeholder Group A multidisciplinary stakeholder group was established to guide the development of the WDMOC. This group included health care professionals whose areas of expertise are represented within the full scope of the decision problem being modeled. Ten individuals were included based on both their familiarity with oral cancer management and care, and their previous relationship with researchers in the Oral Cancer Prevention Program at the BCCA. The scope of practice within the stakeholder group is described below, and includes five surgeons, three oncologists (surgical, radiation, medical), and seven frontline community practitioners. A description of the stakeholder group can be found in Appendix B.  Each member of the stakeholder team was approached for a one-on-one interview to provide input on the structure of the model. Most interviews were conducted in person, with some conducted via telephone. Members were provided with a draft version of the conceptual model and a document explaining the model’s purpose and summarizing its design. During the course of the interview, members were asked “What structural elements in the model do not match current practice?”; “What could/should be changed about those elements to more closely match current practice?”; and “What 68     are important research questions within your scope of practice that the model could/should be used to address in the future? How might the model have to change to reflect those?” The model structure and inputs were updated iteratively over the course of these interviews and re-presented to members of the committee until broad agreement was reached. Immersion in Relevant Evidence Chapter 2 describes the relevant evidence consulted in the construction of the WDMOC. The model was also designed according to principles and guidelines set out by International Society for Pharmacoeconomics and Outcomes Research (ISPOR)[30]. The Canadian Association for Drugs and Technologies in Health (CADTH) issue similar guidelines for model-based economic evaluations in Canada[5]. The use of these guidelines ensures consistent quality between modeling exercises, and provides modelers with a set of analytical principles and tools to ensure that the model output is relevant and useful to guide policy making. They also provide recommendations for data sources, statistical analysis, and the form that model inputs should take. The guidelines are written flexibly, allowing modelers some leeway to customize their guidance to fit the particular decision being modeled.  Generally, the CADTH guidelines were consulted to ensure that the model was appropriate for a Canadian policy making context. The characterization of the decision problem, the comparator group, and methodological issues such as discounting and probabilistic analysis were conducted according to the recommendations published by CADTH. The ISPOR guidelines were used to inform technical issues, such as calculating competing risks, parameter estimation, and incorporating parameter uncertainty. Agreeing What is to be Modeled and Why 69     Chapter 3 can be thought of as an initial attempt to address the third element of determining what is to be modeled and why, for a single decision context. Through the model design process, and through comparison to existing models, the gaps in the literature were discovered. The committee and stakeholder group provided ongoing input regarding the balance between model complexity and the practicality of deriving model parameter estimates, given these gaps. The stakeholder group provided additional insight into the number of novel technologies that could be evaluated from a more comprehensively designed model. It was apparent from the nature of the gaps in knowledge, and from the complexity and variety of emerging technologies, that several components of the oral cancer management/treatment pathway required modeling in a more granular way than was possible with the approach from Chapter 3. This novel granular approach should be able to credibly evaluate single decisions, but also evaluate multiple decisions in concert given the number of technologies that are likely to require evaluation in the near future. Chapters 5 and 6 describe an approach to these single- and multiple-decision evaluations, respectively. The question of ‘why’, as described in Chapter 1, relates to the overall goals of HTA: to guide policy making in such a way that produces the greatest possible health outcomes for the population of interest (in this case, British Columbians) from a given level of budgetary constraint. 4.3.2 Stage 2: Conceptualization and Design  The structural arrangement of the conceptual model (i.e., how the various elements were organized, and how the relationships between them were described) was informed by a text written by Jaime Caro and colleagues[180]. This text was used primarily as the theoretical background for the discrete event 70     simulation methodology, and formed the basis for much of the programming and the way the model is presented visually. Elements from previously-published models were also consulted in the design of the WDMOC. The oral cancer screening model published by Speight and colleagues[124] was consulted to establish elements of the model’s breadth, particularly with regard to the development, detection, and management of preclinical disease. The model developed in Chapter 3 was developed with the Speight model in mind, while adding some necessary depth to the management of both detected OPLs and detected invasive cancers. Much of this depth, particularly with respect to the management of invasive disease and recurrence, was also taken from clinical guidelines published by the National Comprehensive Cancer Network (NCCN)[47], and by the British Columbia Cancer Agency (BCCA)[37, 46]. In order to translate a conceptual model into an empirical one, data about each step and event within the process is required. However, this kind of data is not always available. Accordingly, model design requires balancing the complexity of the real world and the pragmatic limitations of data availability. Many components of the conceptual model were informed from retrospective cohorts of people who had been treated for oral cancer and pre-cancer within the BCCA. These cohorts are discussed in greater detail in section 4.5. An initial draft of the model was presented to members of the stakeholder group during the interviews, for their expert feedback (see Appendix C – Initial Model Structure) Some key findings from these interviews were incorporated into the model’s final structure, including: 71      Treatment options for invasive cancers – what patient and disease factors influence the type of treatment prescribed. Estimates of treatment duration.  Relationship between community dentists and oral health specialists with respect to detection, referral, surveillance, and treatment of premalignant lesions (OPLs).  Role of HPV with respect to cancer incidence and implications for treatment The stakeholder group also identified a number of ways in which the model’s structural assumptions simplify detection, management, and treatment of OPLs. These limitations are discussed in detail in Chapter 7. Through this process, the boundary, breadth, and depth of the WDMOC were determined for the conceptual model:  Model boundary Based on feedback from the committee and the stakeholder group, The WDMOC was designed to simulate a population of adult British Columbians who would be at risk of developing oral cancer within their lifetimes. Because anyone could potentially develop oral disease, this includes all adult members of the population, excluding those who currently have oral cancer. Because changes to the availability of dental care would likely impact the rate at which preclinical oral disease can be detected, people who do not have access to a dentist were also included. The model’s structure is summarized in Section 4.3, and described in Figures 4.1 through 4.6. It should be noted that the model does not adequately reflect the extent to which regional factors influence availability and type of treatment, especially in the context of premalignancy. While the scope 72     of the model is provincial, it is worth noting that many aspects of the model structure are reflective of the Lower Mainland of British Columbia (i.e., Vancouver and the surrounding area) rather than being truly representative of the whole province. This regional bias and its implications will be discussed in greater depth in Chapter 7, but briefly it was agreed that this represented a reasonable ‘starting point’ upon which a more comprehensive model that reflects the complexity of practice outside the Lower Mainland could be built in the future. Model breadth The WDMOC is designed to reflect the full treatment/management pathway of oral cancer from premalignant disease to death. The model is divided into five principal ‘components’, each reflecting a related group of health care services used to address clinically meaningful stages of disease progression. The structure of each component will be described in greater detail in section 4.5. Model depth The level of detail used within each model component required a balance between the complexity necessary to adequately address the technologies undergoing assessment and the availability of data to inform parameters. Many cancer screening and treatment processes are highly individualized, and it is impractical (if not impossible) to build a model that is capable of reflecting all possible options for all possible people. Several simplifying assumptions were made, each of which will be discussed in Chapter 7. Because the ultimate purpose of the WDMOC is to evaluate the impact of changes in health policy and technologies, members of the stakeholder group contributed suggestions of potentially impactful 73     and/or emerging technologies (including policies, programs, and services) for future evaluation beyond the initial implementation of the model as described in this dissertation. These suggestions, and the steps necessary to implement them within the WDMOC, will be described in Chapter 7. 4.4 Conceptual Model of the WDMOC The WDMOC simulates the trajectory of hypothetical people ( ‘entities’) through the oral cancer pathway from preclinical disease through the development of invasive cancer to death from terminal illness. The following section will describe the structure of the WDMOC, and the path that entities can follow from creation to termination. The WDMOC involves the creation of entities (simulated people), whose disease status is informed by a “Natural History” model, and whose health care interactions are informed by a “Clinical Trajectory” model. The Natural History model describes the development of de novo OPL and its progression to invasive squamous cell carcinoma (SCC) of increasing severity. The Clinical Trajectory model describes the health care system processes through which premalignancies and invasive SCC are detected and managed, and is divided into five interacting ‘components’. The structure of the model, by each component, is presented in the following section. The structure and entity path assumptions were determined iteratively through review of the literature, input from the stakeholder team, and the available data to inform parameter estimates (which will be described in greater detail in section 4.5). Entity Creation 74     The process by which an entity is created precedes both the Natural History and Clinical Trajectory models. Entities are assigned a set of personal characteristics that informs the way they will move through the model:  Age at start of model  Sex (binary M/F)  Smoking status (ever/never, by sex)  Alcohol use (heavy/non-heavy, by sex)  Access to a dentist  Date of death from causes other than oral cancer This list of characteristics were drawn from the literature reviewed in Chapter 2, with input from members of the stakeholder group who agreed that they were reasonably comprehensive for this initial model. The entity may also start the model with an undetected OPL, based on their age and sex. This prevalence-based approach was used in a previously published decision model[124], and its limitations will be discussed in Chapter 7. The OPL is assigned a risk profile of ’high‘, ’medium‘ or ’low‘, based on its level of loss of heterozygosticity (LOH), informed by data published in the literature (see section 4.5). 4.4.1 Natural History Model Based on demographic characteristics (age, sex), newly-created entities may have an OPL that can be detected through screening. An OPL may progress to SCC, or it may spontaneously resolve (i.e., return to normal, non-diseased epithelial tissue), based on the entity’s age, sex, smoking status, and LOH risk profile.  75     SCCs start at stage I and may progress to a higher stage (i.e., I  II  III  IV), or may present symptoms that drive a person to seek medical care outside routine care, at which point the disease is detected and diagnosed – this part of the process occurs in the ‘Incident Cancer’ component of the Clinical Trajectory model. Based on expert input, the WDMOC assumes that terminal undetected stage IV cancers (i.e., an entity can die from an undetected stage IV cancer) are detected symptomatically three months before death or less – this part of the process occurs in the ‘Terminal Disease’ component of the Clinical Trajectory model. The process is illustrated in Figure 4.1.   76     Figure 4.1 – The Natural History model component    77     It is important to note that this modeling approach uses prevalent OPL cases rather than incident ones. This approach was adapted from a previously-published oral cancer screening model[124], but places meaningful limits on the WDMOC’s function. The reasons for this choice and the implications of the resulting limitations will be discussed in Chapter 7. 4.4.2 Clinical Trajectory Model The Clinical Trajectory model is divided into five components, each representing a set of health care system processes for management of oral cancer at various stages. Entities move through the components according to their disease status (i.e., their progression within the Natural History model) and their clinical history (i.e., the events that have happened previously in the Clinical Trajectory model). The five components are organized as follows: 1. Screening/Asymptomatic 2. Oral Premalignant Lesion (OPL) 3. Incident Cancer 4. Follow-up 5. Terminal Disease Entities pass through these components from the start of the model run (i.e., the creation of an entity) until they reach a terminal state, which simulates either death from oral cancer or from another cause. The paths that entities can take through each component are described in detail below. 4.4.3 Screening/Asymptomatic Component Entities with access to a dentist will be seen at regular intervals for a dental checkup. If the entity has developed a premalignant lesion, it may be detected in a routine exam. If no lesion or other abnormality 78     is detected, the entity will return for their next appointment after a period of time. The component is described graphically in Figure 4.2. Figure 4.2 – Screening/Asymptomatic model component  79      Based on input from the community practitioners within the stakeholder group, entities with a detected lesion are asked to return in three weeks. If the lesion persists beyond at three-week follow-up, the entity is referred to an oral health specialist (periodontist, oral medicine specialist, oral surgeon) for additional scrutiny. The specialist will perform a biopsy of any lesion that is deemed suspicious for premalignancy.  Premalignant lesions are detected in this way, and referred for pre-malignant management (OPL component). Invasive cancers may also be detected in the course of routine dental care and are referred for curative treatment (Incident Cancer component). Entities with non-malignant lesions and/or lesions that resolve within the three-week period return for routine dental checkups after a period of time. It is possible for the screening procedure to return a false negative (i.e., the entity has premalignant or malignant disease, but a negative test), in which case they will not be re-screened until the next screening appointment. Their disease may progress during this time. Entities with no dental access will not have premalignant lesions detected, and any invasive cancer that may result can only be detected symptomatically. If they do not develop oral cancer, they will eventually die of another cause. The rate of non-oral cancer related death is described in section 4.5. 4.4.4  Oral Premalignant Lesion Component 80     Entities with a detected premalignant lesion will undergo regular evaluations by a specialist for evidence of progression to malignant disease. If progression is suspected, the entity will undergo a diagnostic biopsy. The entity’s OPL may be biopsied at regular intervals as well, after a period of time (based on stakeholder group input). If no progression is suspected or detected, the entity will return for another evaluation after a period of time. Detected invasive cancers are referred for treatment (Incident Cancer component). The component is described graphically in Figure 4.3. Figure 4.3 – Oral Premalignant Lesion management model component    81     4.4.5  Incident Cancer Component Entities with a detected invasive cancer will undergo a diagnostic workup to determine the disease stage. If a stage I disease was detected through screening (either in the Screening/Asymptomatic component or the OPL component), it may be classified as either a high-grade lesion (HGL) or a squamous cell carcinoma (SCC), reflecting the possibility that some lesions may be referred for treatment before they develop invasive malignant characteristics. Stage I cancers detected symptomatically are assumed to be SCC. The model treats all cancers of stage II or higher as SCC – a simplifying assumption made based on feedback from the stakeholder committee. The component is described graphically in Figure 4.4. 82     Figure 4.4 – Incident cancer model component  Cancers at all stages undergo a Treatment Process, based on stage at diagnosis. Treatment falls into three categories: surgery alone, surgery with adjuvant external beam radiotherapy (RT), or another treatment that may include any combination of surgery, chemotherapy, and RT. All HGL are managed with surgery alone. These assumptions were informed by feedback from the stakeholder group and the availability of data as described in section 4.5. 83     Based on demographic and clinical characteristics (stage, age at detection, sex, and treatment type), an entity is assigned a time at which they either experience disease recurrence or death from disease. After a period of time, entities begin attending post-treatment follow-up (Follow-up component). 4.4.6 Follow-up Component After their treatment is finished, entities return at regular intervals to see their oncologist(s) to evaluate their tumour site for evidence of disease progression. The frequency of evaluations is determined by the length of time since the entity was treated, becoming less frequent as the entity approaches ten years of follow-up care, reflecting clinical practice guidelines and input from the stakeholder group. If the entity is alive and disease-free after ten years, it is assumed to be in full remission and will die of a cause other than oral cancer at the time determined in the Natural History model. The component is described graphically in Figure 4.5.   84     Figure 4.5 – Cancer follow-up model component   Death from disease or recurrence occurring during the follow-up period is managed within the Terminal Disease component.   85     4.4.7  Terminal Disease Component Entities may enter this component either as a result of a detected recurrence, an undetected Stage IV cancer that is within three months of death, or a cancer undergoing follow-up that is within three months of death. The component is described graphically in Figure 4.6.  Figure 4.6 – Terminal disease management component  86     Recurring cancers are diagnosed and then undergo a Recurrence Treatment Process. Treatment falls into four categories: curative including surgery, curative not including surgery, palliative, or none. These categories are based on the parameter estimation process described in section 4.5. A time to death from disease or second recurrence is calculated based on the entity’s demographic and clinical characteristics (stage at first diagnosis, age at recurrence, sex, recurrence treatment type). If the entity has received curative treatment (surgery, non-surgery) they return to the Follow-up component. Entities with a second recurrence undergo a similar Recurrence Treatment Process, but treatment is assumed to be identical for all patients, and time to additional recurrence or death is based only on age and sex, based on the parameter estimation process described in section 4.5. Entities receiving palliative or no treatment are managed on a monthly basis until they are within three months of death, at which point they are receive End-of-Life (EOL) care and die from disease. It is possible for someone to undergo full remission with best supportive or palliative care within the model. Entities entering the Terminal Disease component as a result of terminal undetected stage IV cancer, or whose cancer is within three months of death from disease, receive EOL care and will die from disease. The model assumes that it is not possible for these cancers to go into full remission. 4.5  Stage 3: Implementation Modeling An individual sampling model using a time-to-event approach was chosen[26]. Individual sampling models are a type of discrete event simulation model in which entities do not interact. Interaction between entities is of particular importance when a model needs to account for queuing for scarce 87     resources, or in the case of infectious disease modeling where entities can influence each other’s disease status. The WDMOC does not consider queuing, and assumes that all resources (including specialist care) are available instantaneously when the entity needs them. This is a common assumption in health economic decision modeling. The WDMOC was programmed in the Python language (Python Software Foundation, Delaware, USA). Python was chosen in favour of the commercial software used to create Tappenden’s original Whole Disease Model. Python is an open-source and web-ready language that is free to license. While further reasoning behind this choice is detailed in Chapter 7, The primary rationale for this choice was to make a model that could easily be adapted, updated, and re-used by different researchers in different contexts. No model, no matter how well-designed, can adequately reflect the full breadth and depth of any decision environment. Whole disease models are still designed to reflect a single jurisdiction (i.e., a province, a health authority, a country), and simplifying assumptions that may be valid within one jurisdiction may not apply in another. Accordingly, in order to be useful beyond its original context, these models should be easy to edit and share across health care policy making jurisdictions, which implies the need for open-source and free software. Designing models in this way allow them to be adapted to quickly and seamlessly reflect not only differences between jurisdictions, but also technological and policy innovations that may be developed in the future. 4.5.1  Approach to Simulation Modeling Individual sampling models simulate the movement of ‘entities’ (simulated people, in this case) through an environment (the parts of the health care system relevant to oral cancer, in this case). Entities 88     experience changes to their characteristics and use resources over the course of their simulated time; these changes and resources occur during particular ‘events’ that occur at each step of the process. A population can be simulated by generating several entities and running them through the same environment.  A crucial step in any economic evaluation is assessing the impact that uncertainty (i.e., unknown information that is relevant to the decision being evaluated) has on its outputs.  Uncertainty in economic evaluation is described within four categories[181]: 1. Stochastic uncertainty (or ‘first-order’ uncertainty) concerns the the random variability in outcomes that occurs between people with identical characteristics (e.g., people of the same age, sex, disease type, etc.). It is analogous to random error in a regression analysis. 2. Parameter uncertainty (or ‘second-order’ uncertainty) concerns the variability that surrounds each value the model uses to estimate its outputs. the level of uncertainty or imprecision in the estimation of a particular model parameter (e.g., cost of a resource, time to developing symptoms, probability of a false positive, etc.). It is analogous to the standard error of a coefficient estimate in a regression analysis. 3. Heterogeneity concerns characteristics of the population being modeled that may impact the magnitude of costs and outcomes. Rates of recurrence may be influenced by age and sex, and will differ between subgroups within each population. 4. Structural uncertainty concerns the assumptions that are inherent to the design of the model. For example, even a good model may necessarily exclude potential outcomes, simplify the relationships between events, and use data estimated independently from different populations. These structural decisions may affect the output of the model in unknown ways. 89     By allowing model input values (i.e., the variables that inform the model) to vary between each entity, individual sampling models can reflect first- and second-order uncertainty. First-order uncertainty was reflected in the random draws from the uniform distribution used to evaluate the assignment of characteristics and the sequencing of events – these random draws mean that two entities with identical characteristics will not necessarily follow identical paths through the model.  Second-order uncertainty was reflected in the random draws for the values of each model parameter, from their underlying distribution – this process means that the value of a given parameter (e.g., probability of receiving a certain type of treatment) is different for each entity, based on the uncertainty around that parameter.  This process of probabilistic random sampling is known as Monte Carlo simulation, and is a commonly-used technique in decision modeling[182]. Heterogeneity and structural uncertainty were evaluated through the use of sensitivity analysis, which will be described in section 4.8.  4.5.2  Time-To-Event Processing in the WDMOC The general structure of the time-to-event approach used in the WDMOC relies on five principal types of programs. These are: 1. Sequencer: directs progress of entities from creation to a terminal condition (death) 2. Clock (‘CheckTime.py’): identifies current simulated time and schedules next occurring event 3. Natural History processes: describe an entity’s trajectory through the natural history of oral precancer and undetected invasive cancer 4. System Processes: describe an entity’s trajectory through the health care system (screening, cancer treatment, follow-up) 90     5. Global processes: contain functions that are used by other programs The relationship between these functions is described in Figure 4.7. The Sequencer creates a new entity, and that entity is assigned characteristics that will be used to determine its natural history and whether or not it will receive screening. Entities are then assigned a natural history – times at which different disease events will occur (development of precancer, progression to invasive disease, symptomatic detection, etc.). The next event to occur (either a natural history event or a system process event) is read by the CheckTime program and the system clock is advanced to that time. The Sequencer then runs the appropriate programs (system and/or global Figure 4.7 – Summary of simulation model implementation 91     processes) that update the entity’s characteristics and determine the next event to occur. Resource units and utilities are also appended to each entity as events occur and health status changes.  This process continues on a loop until the entity reaches a terminal state, at which point a new entity is created and the process restarts. The model runs until a user-defined number of entities has been simulated. The cohort of simulated entities can then be analyzed to determine survival, quality-adjusted survival, and cost (through resource unit costing) for each entity. All model parameters are read in from a master spreadsheet containing estimates of the mean and standard deviation of each variable. By adjusting values in the spreadsheet, the model can simulate multiple cohorts moving through the same (or similar) policy environments. Incremental cost-effectiveness analysis can be conducted by comparing mean costs and outcomes in these cohorts. Because the master spreadsheet can be adjusted in ways that affect multiple decisions within the model’s breadth, it is possible to analyze the cost-effectiveness of several policy decisions simultaneously. Additionally, the approach used in the implementation of the WDMOC allows for entire subsections of the treatment pathway (e.g., a chance in surgical management of early-stage oral cancers) to be programmed de novo and inserted into the appropriate place within the overall model. By making a small adjustment to the Sequencer, entities can be routed through the newly-programmed subsection, allowing for the model to be updated and/or customized to a variety of policy-making settings. 92     An example of such a de novo substitution is presented in Chapter 5. An example of multiple simultaneous evaluations is presented in Chapter 6. Chapter 7 will discuss how these two methods can be easily incorporated. 4.5.3  A Description of Model Functions There are five basic types of functions used in this type of simulation modeling. A discussion of these nodes follows, and examples of each type are included in Appendix D. 1. Entity – an entity (a simulated person) takes the form of a Python library that can be expanded to add any useful information such as age, treatment flags, current time and time to next event, among others. 2. Parametric sampling – all values in the model are generated probabilistically, based on a mean and standard deviation, as well as a value denoting the assumed parametric distribution of the parameter uncertainty (e.g., Beta distributed, Gamma distributed, etc.). These parametric assumptions follow ISPOR guidelines[30, 182]. Some model parameters are input as coefficients from generalized linear equations. 3. Probability nodes – values sampled through the above process are compared to a randomly generated number from a uniform distribution. If the randomly-generated number meets a given condition (i.e., is greater or less than the sampled value) then a model-defined outcome will arise (e.g., an entity’s characteristics will change, an event will be scheduled, a natural history event will be encoded, etc.) 4. Time – time to next event is handled through the ‘CheckTime.py’ program. Briefly, the next event to occur (Natural History, System Process, Clinical History) is sampled from a parametric distribution based on the entity’s characteristics at various points throughout the entity’s simulated life. The various values of time are compared during each loop of the Sequencer, and the next to occur is scheduled. When that time is reached, the entity’s characteristics are updated to reflect the occurrence of the event. 5. Resources, utilities, and events – lists that record the resource used or events occurring, and the system time at which they occurred. These can be compared at the end of the model run. 93     Once the specified number of entities has been simulated, the lists of resources, utilities, and events can be compared in incremental analysis. Resources are converted to costs through a unit costing approach, in which a monetary value is estimated for each resource unit.  Costs are discounted to account for society’s preference for goods now rather than in the future – a concept known as future time preference[5, 19]. Discounting is a distinct concept from currency inflation. Discounting was applied according to the following formula:                                      Where   is an annual discount rate, and   is the amount of time in the future that the cost occurs, expressed in years. An annual discounting rate of 1.5% was used, based on guidelines established by CADTH[5]. Unit costs are applied to resources utilized by each entity, and are then discounted and summed to estimate the total costs experienced by that entity over the course of their trajectory through the model. Simulating multiple entities and estimating costs for each produces an estimate of mean costs for the simulated population. Mean costs from different simulated populations can be compared in incremental analysis to produce the incremental cost term (∆C) of an Incremental Cost Effectiveness Ratio (ICER – see Chapter 1).  4.6 Model Parameterization 94     The probabilities, times to event, resource unit costs, and utilities that govern the model’s behavior are represented as model parameters – numerical estimates of variables used within the model. Parameterization of the model was undertaken using both primary data and secondary data. Primary data is data that was collected directly from participants in research studies and retrospectively-collected datasets. This stochastic data allowed for several model parameters to be estimated in ways that make the operation of the WDMOC reflect multiple aspects of clinical realities at a depth that is not typically possible with published (secondary) data. Secondary data is collected from sources in the scientific literature. It is common to need to convert the best available data into a form that is useable by the model through statistical methods. The ways in which both these types of data were converted into useable model data are described below. Primary Data Primary data for this project was derived from two sources: 1. A Retrospective Oral Cancer Cohort (ROCC), comprised of the electronic medical records of 864 patients previously treated for oral cancer in the province of British Columbia between January 1, 2001 and December 31, 2015 (diagnosed between January 1, 2001 and December 31, 2009). These records were identified by a member of the Oral Cancer Control Program’s research team through the Provincial Cancer Registry. A chart review was conducted by this same researcher to identify relevant clinical dates (e.g., diagnosis, treatment, recurrences, death) for each person within the cohort. This dataset was linked through Popdata BC and the BC Cancer Agency’s Information System (CAIS) to identify resources used from diagnosis to death, censoring, or loss to follow-up. 95     2. Anonymized data from the clinical trial conducted by Zhang and colleagues (2012) exploring the role that LOH plays in oral cancer development. Times between lesion detection and progression to cancer were observed within this cohort, as well as some basic demographic information (age, sex, tobacco use, alcohol use). These sources were analyzed to determine statistical associations between individual entity characteristics and times to events of interest. Parameter inputs were derived from these data sources using linear regression methods, described in section 4.6.  Secondary Data A variety of secondary data sources were used to build the WDMOC, each with its own appropriate method of being incorporated into the simulation process. These parameterization methods are described in section 4.6. The parameters used in each component of the WDMOC are described in the following sections. The methods used to valuate each parameter are based on assumptions about the statistical distribution that each parameter takes. The process by which estimates of each parameter were derived for each entity are described in section 4.6.  4.6.1 Entity Creation Newly-created entities are assigned demographic and disease characteristics, derived from four sources: population statistics published by Statistics Canada, figures published by the Canadian Dental Association, values published in the Speight et al oral screening model, and values published in the Zhang et al trial. Parameter inputs are summarized in Table 4.1. 96     Table 4.1 – Parameter estimates: entity creation Parameter Mean SD Distribution Source Smoking prevalence (British Columbia)   Beta [183] Men 0.174 0.0003   Women 0.113 0.0002   Has access to a dentist (British Columbia) 0.688 0.0002 Beta [184] Prevalence of oral premalignancy   Lognormal [124] Men     <50 -3.132 0.258   50-59 -2.817 0.234   60-69 -2.788 0.261   70-79 -3.040 0.370   80+ -2.670 0.520   Women     <50 -4.132 0.311   50-59 -3.817 0.283   60-69 -3.788 0.310   70-79 -4.040 0.403   80+ -3.671 0.545   LOH Risk Score Count†  Dirichlet [61] Low 130    Medium 120    High 28    † – counts, rather than means, were used to calculate the probabilities and uncertainty using the Dirichlet distribution. Smoking prevalence Statistics Canada publishes smoking rates for each province by sex. These values were expressed as probabilities assuming a Beta distribution. Access to a Dentist The percentage of British Columbians with regular access to a dentist was estimated from rates published by the Canadian Dental Association. This value was expressed as probabilities assuming a Beta distribution. 97     Prevalence of Oral Premalignancy Newly-created entities are assigned a probability of starting the model run with an undetected premalignancy, based on the underlying prevalence of OPLs in the population. This prevalence, by age and gender, was estimated based on values published in the Speight et al model. These values were originally estimated from an opportunistic screening study conducted in the general population of the UK. Individual-level data was analyzed using logistic regression. The published coefficients of the log odds from the regression were used to calculate probabilities in the WDMOC assuming a lognormal distribution. Progression risk score Entities with an OPL were assigned an LOH risk score based on the prevalence of high, medium, and low-risk LOH profiles published in the Zhang et al trial. Counts of each risk group were converted to probabilities assuming a Dirichlet distribution. 4.6.2  Natural History Parameters for the WDMOC’s natural history processes were derived from two sources: secondary analysis of the Zhang et al trial cohort, and the values published in the Speight et al oral screening model. Parameter inputs are summarized in Table 4.2.   98     Table 4.2 – Parameter estimates: natural history Parameter Mean SD Distribution Source Time to OPL progression to Stage I cancer   Weibull GLM Zhang et. al cohort Intercept (β0) 9.296 0.744   Sigma (σ) 0.801 0.106   Age 0.0013 0.010   Sex     Men Ref.    Women -0.0804 0.257   LOH Risk Score     Low Ref.    Medium -0.868 0.449   High -1.249 0.391   Time to symptomatic detection of cancer   Weibull [124] Stage I 0.27 0.19   Stage II 0.56 0.27   Stage III 0.68 0.28   Stage IV 0.71 0.3   Time to undetected cancer progression   Weibull [124] Stage I to Stage II 0.53 0.27   Stage II to Stage III 0.59 0.25   Stage III to Stage IV 0.67 0.25   Time to death from Stage IV   Weibull [124] Men     <50 0.378 0.251   50-59 0.439 0.251   60-69 0.487 0.251   70-79 0.670 0.251   80+ 1.00 0.01   Women     <50 0.320 0.224   50-59 0.372 0.224   60-69 0.412 0.224   70-79 0.566 0.224   80+ 0.873 0.224    Time to OPL progression to Stage I cancer 99     These values were estimated from a secondary analysis of data from the Zhang et al trial. Time to progression was fit to a parametric Weibull survival curve. Details on this process are available in Appendix E. Coefficients from a GLM regression with a Weibull link function of observed survival with multiple covariates (age, sex, LOH risk score) were used to estimate survival times and probabilities for each entity. Time to Symptomatic Detection of Cancer The time between developing an invasive cancer and that cancer being detected through symptomatic presentation was estimated using transition probabilities published in the Speight et al model. The authors calculated these values by eliciting expert opinion from health care practitioners concerning the proportion of people whose cancer would be detected in the absence of a routine screening program. These assumptions were reviewed by the committee and were determined to be appropriate within a BC context. These probabilities were converted to time-to-event values using the Weibull method of moments. Time to Undetected Cancer Progression The time that it takes for an undetected cancer to advance in stage was estimated using transition probabilities published in the Speight et al model. These values were originally estimated by eliciting expert opinion from health care practitioners concerning what proportion of undiagnosed patients, at each stage of disease, would progress to the next stage within a year. These probabilities were converted to time-to-event values using the Weibull method of moments. 100     Time to Death from Undetected Stage IV Cancer The time between developing an undetected Stage IV cancer and death from disease was estimated using transition probabilities published in the Speight et al model. These values were originally estimated from a retrospective analysis of survival in oral cancer patients undergoing treatment, and making an assumption of a more dire prognosis for undetected cancers, using an exponential regression process. These values were reported by age and sex. The probabilities were converted to time-to-event values using the Weibull method of moments. 4.6.3 Screening/Asymptomatic Component Values concerning the passage of asymptomatic people through regular dental appointments were derived from three principal sources: the Speight at al oral cancer screening model, an opportunistic screening study from the US, and a similar study conducted in British Columbia. Parameter inputs are summarized in Table 4.3. Table 4.3 – Parameter estimates: asymptomatic/screening component Parameter Mean SD Distribution Source Regular appointment interval (days) 180 30 Normal Assumed Return appointment interval (days) 21 3 Normal Assumed Sensitivity of visual screening 0.965 0.012 Beta [124] Specificity of visual screening 0.848 0.043 Beta [124] Probability of non-OPL lesion 0.0452 0.0071 Beta [73] Probability that non-OPL lesion resolves within interval 0.290 0.0408 Beta [72] Probability of non-OPL biopsy 0.111 0.0524 Beta [72] Probability of attending return appointment 0.350 0.0242 Beta [72]    101     Appointment intervals The WDMOC assumes that all entities are managed according to guidelines, returning for regular dental visits every six months (180 days). If a suspicious lesion (either OPL or another type) is found during the course of an appointment, an entity returns after 21 days for re-evaluation. Data on the number of days between appointments was not available, so parameter uncertainty was assumed and these assumptions were verified by the stakeholder committee. These values were assumed to be normally distributed. Sensitivity and Specificity of Visual Screening Values describing the ability of routine dental screening to accurately detect lesions in the mouth were derived from corresponding values published in the Speight et al oral screening model. These values were originally synthesized through a meta-analysis of multiple studies evaluating the effectiveness of routine screening. These values were expressed as probabilities assuming a Beta distribution. Non-premalignant lesions The WDMOC accounts for the presence of oral lesions that are not premalignant but will nevertheless trigger a response from a dentist if they are detected (i.e., they will be asked to return for re-evaluation). The rate at which these lesions occur was estimated based on values published in a population screening study in Boston.  Non-premalignant lesions may resolve spontaneously upon return screening. Non-resolving lesions may be biopsied if they are suspected to be premalignant. Some entities may choose not to return for re-102     evaluation. Values for each of these outcomes were estimated based on results published in an evaluation of an oral cancer screening education program in British Columbia. All parameters were expressed as probabilities, assuming a Beta distribution. 4.6.4 Oral Premalignancy Component Parameter values concerning management of people with detected OPLs were derived from the Speight oral cancer screening model and values published in the Cromwell et al OPL management model (i.e., the inputs from Chapter 3). Parameter inputs are summarized in Table 4.4. Table 4.4 – Parameter estimates: oral premalignancy component Parameter Mean SD Distribution Source Appointment interval (days) 180 30 Normal Assumed Sensitivity of visual screening 0.965 0.012 Beta [124] Specificity of visual screening 0.848 0.043 Beta [124] Probability of SCC at detection 0.678 0.038 Beta [148]  Appointment interval The WDMOC assumes that all entities return for regular follow-up visits every six months (180 days). Values were assumed to be normally distributed. Sensitivity and Specificity of Visual Screening The accuracy of visual screening in OPL was derived from the same source as described in the Asymptomatic/Screening component. Probability of SCC at detection 103     Premalignant lesions undergoing observation may be surgically resected before progression to invasive cancer (SCC). These high-grade lesions (HGL) are managed surgically but have a different prognosis to SCC. In order to reflect this, the WDMOC assumes that a proportion of OPLs under surveillance are detected as HGLs. This proportion was calculated in the Cromwell et al cost-effectiveness model, and was derived from a retrospective cohort of OPL patients undergoing follow-up at the Vancouver Cancer Centre. This value was expressed as a probability, assuming a Beta distribution. 4.6.5 Incident Cancer Component Parameters concerning treatment type and outcomes were derived from the ROCC. Parameter inputs are summarized in Table 4.5.   104     Table 4.5 – Parameter estimates: incident cancer component Parameter Mean SD Distribution Source Treatment type Count†  Dirichlet ROCC Stage I     Surgery 235    Surgery + RT 26    Other 58    Stage II     Surgery 124    Surgery + RT 56    Other 120    Advanced stage     Surgery 46    Surgery + RT 43    Other 145    Treatment time (days) 90 0 Normal Assumed Time to First Event   Weibull GLM ROCC Intercept (β0) 11.034 0.334   Sigma (σ) 1.320 0.045   Age -0.0424 0.005   Sex     Men Ref.     Women 0.364 0.117   Cancer Stage     I Ref.     II -0.376 0.145   Advanced -0.874 0.154   Treatment type     Surgery alone Ref.     Surgery + RT -0.796 0.191   Other treatment -0.869 0.130   Time to First Event – Death   Weibull GLM ROCC Intercept (β0) 11.216 0.328   Sigma (σ) 1.272 0.044   Age -0.043 0.005   Sex     Men Ref.     Women 0.307 0.114   Cancer Stage     I Ref.     II -0.376 0.142   Advanced -0.925 0.150   Treatment type     Surgery alone Ref.     105     Parameter Mean SD Distribution Source Time to First Event – Death     Treatment type cont.     Surgery + RT -0.739 0.184   Other treatment -0.817 0.126    Treatment Type Treatment type was simplified to encompass three basic approaches – surgery alone, surgery + RT, and ‘other’. The ‘other’ category is primarily comprised of people receiving chemotherapy and radiotherapy, but other combinations were seen in the data as well. Treatment type was counted by stage at presentation, and expressed as probabilities assuming a Dirichlet distribution. Time to First Disease Event Time to first recurrence or death was estimated for each entity using the competing events approach. Briefly, time to a first event (either recurrence or death) was sampled from a parametric Weibull curve fit to the observed survival data, and the probability of the event occurring at that time was calculated using the hazard function. The corresponding probability was calculated from a second curve of time to death. These probabilities were compared to a random draw, and the nature of the event (recurrence or death) was determined. Details on this process are available in Appendix F. Coefficients from a linear Weibull GLM regression of observed survival with multiple covariates (age, sex, stage at detection, treatment type) were used to estimate survival times and probabilities for each entity. 4.6.6 Follow-up Component 106     The time between each follow-up appointment was derived from NCCN guidelines. The WDMOC assumes some variability around guideline adherence using a Normal distribution. Parameter inputs are summarized in Table 4.6. Table 4.6 – Parameter estimates: follow-up component Parameter Mean SD Distribution Source Interval between follow-up appointments (days)    [47] 0 to 3 years post-treatment 90 10 Normal  3 to 5 years post-treatment 180 20 Normal  5 to 10 years post-treatment 365 50 Normal   4.6.7 Terminal Disease Component Parameter inputs are summarized in Table 4.7.   107     Table 4.7 – Parameter estimates: terminal disease component Parameter Mean SD Distribution Source Treatment type Count†  Dirichlet ROCC Recurrence     Surgery 62    No Surgery 48    Palliative 20    No Treatment 17    Time to Second Event   Weibull GLM ROCC Intercept (β0) 7.687 0.632   Sigma (σ) 1.185 0.083   Age -0.008 0.009   Sex     Men Ref.    Women 0.028 0.229   Recurrence treatment type     Treatment includes surgery Ref.    Treatment does not include surgery -0.691 0.261   Palliative Care -1.656 0.325   No Treatment -1.414 0.378   Time to Second Event – Death   Weibull GLM ROCC Intercept (β0) 8.080 0.647   Sigma (σ) 1.182 0.086   Age -0.008 0.009   Sex     Men Ref.     Women -0.244 0.233   Recurrence treatment type     Treatment includes surgery Ref.     Treatment does not include surgery -0.814 0.271   Palliative Care -1.957 0.332   No Treatment -1.165 0.389   Time from Second Recurrence to Death   Weibull GLM ROCC Intercept (β0) 5.946 2.434   Sigma (σ) 1.249 0.304   Age -0.036 0.041   Sex     Men Ref.     Women -0.244 0.233    † – counts, rather than means, were used to calculate the probabilities and uncertainty using the Dirichlet distribution.  108     Recurrence Treatment Type Treatment type was simplified into three categories – treatment including surgery, treatment not including surgery, and treatment flagged as ‘palliative’. A fourth category was included for people whose charts indicated that they were not prescribed any treatment, either curative or palliative. In the retrospective cohort analysis described in Section 4.5, this fourth category had a statistically significantly different survival curve (see Appendix F), and so was treated as a distinct population within the model. Treatment type was expressed as probabilities using a Dirichlet distribution. Time to Second Disease Event Time to second event (second recurrence or death following recurrence) was calculated through the same process as time to first event (see above). Coefficients from a linear Weibull GLM regression of observed survival with multiple covariates (age, sex, recurrence treatment type) were used to estimate survival times and probabilities for each entity. 4.6.8 Unit Costs The costs of treatment were estimated via a linkage exercise between the retrospective cohort (ROCC) and data held by the BCCA and the Ministry of Health. The linkage exercise is described in Appendix H. Briefly, unit costs were applied to retrospective records of tests, appointments, hospitalizations, chemotherapy drugs, radiotherapy, and provincially-insured drug prescriptions for each member of the cohort over a three-month period from each clinical event (initial treatment, recurrence, second recurrence) and preceding death. Estimates for unit costs were derived from Medical Services Plan 109     (MSP) reimbursement rates[185], sources in the published literature, and expert opinion where necessary (see Appendix H for full description). All costs were expressed in 2017 Canadian dollars, adjusted for inflation using the Consumer Price Index for Health Care[186].  Unit costs for resources aside from cancer treatments were estimated primarily from MSP fee-for-service (FFS) reimbursement rates[185]. It is important to note here that dental appointments are not covered under provincial insurance, and are typically paid either out-of-pocket or through private insurance. Accordingly, these costs are not borne by the health care system as conventionally understood, and any differences in cost due to a change in dental appointment rates would not necessarily affect health care system budgets. Unit costs are presented in Table 4.8. Parameter values were estimated for each entity assuming a Gamma distribution, except for FFS values which were assumed to be equal for all entities. Table 4.8 – Parameter estimates: unit costs Parameter Mean SD Distribution Source Dental Appointment $43.10 N/A FFS [187] Specialist Appointment $254.91 N/A FFS MSP – 03770 Dental Screening $0.00 N/A FFS Assumption Biopsy $250.40 N/A FFS MSP – 03773 OPL surveillance appointment $59.51 N/A FFS MSP – 03785 Diagnostic Workup $591.29 N/A FFS MSP† Treatment   Gamma ROCC Stage I     Surgery $9,268.55 $10,758.64   Surgery + RT $21,219.00 $17,525.96   Other $7,630.33 $9,051.55   Stage II     110     Surgery $19,299.16 $25,503.44   Surgery + RT $26,059.51 $18,567.71   Other $10,279.56 $8,660.23   Advanced Stage     Surgery $38,185.45 $40,438.87   Surgery + RT $34,925.65 $22,256.47   Other $14,848.61 $16,238.45   Parameter Mean SD Distribution Source Treatment     Recurrence     Includes Surgery $29,262.14 $42,730.52   Does Not Include Surgery $17,066.59 $20,314.99   Palliative $20,778.66 $31,106.02   No Treatment $11,119.52 $8,518.00   Second Recurrence $16,616.98 $27,427.74   End of Life $17,930.25 $21,977.09   Follow-up – 1 to 3 $154.00 $75.00 Gamma MSPǂ Follow-up – 3 to 5 $80.67 N/A FFS MSP – 33512 Follow-up – 5 to 10 $80.67 N/A FFS MSP – 33512 Follow-up appointment – final $154.00 $75.00 Gamma MSPǂ Death from Natural Causes $0 N/A  Assumption FFS – fee for service; MSP – medical service plan; ROCC – retrospective cancer cohort; N/A – these are fixed costs that do not have any parameter uncertainty †- Diagnostic workup includes ‘Diagnostic Examination and Consultation’ (MSP 03770), CT Scan (MSP 08693) and a PET Scan[188] ǂ- This figure is a frequency-weighted estimate of appointments based on treatment type, reflecting the fact that patients may see one or multiple members of their medical team (surgeon, radiation oncologist, medical oncologist) at a given follow-up appointment. 4.6.9 Health State Utilities Health statue utilities were retrieved from sources cited in a recent systematic literature review[103]. Exercises performed in oral cancer or general head & neck cancer patient cohorts were considered eligible. The EQ-5D-5L was the most commonly-used utility measure in the literature review, and was therefore used in this exercise. Three published studies were included from this review, based on 111     similarity of the health states in those exercises to those found in the WDMOC. Parameter inputs are summarized in Table 4.9. Table 4.9 – Parameter estimates: health state utilities Parameter Mean SD Distribution Source Well (no disease) 1.0 N/A  Assumption Undetected OPL 0.92 0.18 Beta [124] Detected OPL 0.92 0.18 Beta [124] Undetected cancer     Stage I 0.84 0.02 Beta [106] Stage II 0.84 0.02 Beta [106] Stage III 0.82 0.14 Beta [104] Stage IV 0.82 0.14 Beta [104] Detected Cancer     Undergoing Treatment 0.65 0.19 Beta [104] During Follow-up 0.82 0.18 Beta [109] Recurring cancer     Undergoing treatment 0.65 0.19 Beta [104] During Follow-up 0.82 0.18 Beta [109] Incurable/Terminal disease 0.68 0.33 Beta [124] End of Life 0.68 0.33 Beta [124] Cancer in Full Remission 1 N/A  Assumption  Utility values for detected and undetected OPLs were derived from the same source as the Speight model[124, 178]. These values were taken from general population evaluations of health states related to oral conditions including cancer. The WDMOC assumes no difference between health state utility for detected and undetected OPLs. Utility values for undetected early-stage cancer (stage I, II) were derived from an exercise by Govers et al [106], from a population of patients with early-stage cancer whose disease was managed with watchful 112     waiting. No data was available in the literature regarding health state utilities for early-stage cancers prior to diagnosis, but this estimate is similar to baseline values estimated in the COOLS Trial for early-stage SCC and HGLs.  Utility values for stage III and IV cancers were derived from the baseline (pre-treatment) values taken from a cohort of clinical trial participants with stage III/IV head and neck cancers[104]. Utility values for cancers undergoing treatment were derived from the same Truong et al. clinical trial, using EQ-5D-3L values reported by patients at the end of their treatment[104]. Use of this estimate assumes that treatment-related utility does not differ across disease stage. Utility values for cancers in remission less than ten years after treatment were derived from a study of a consecutively-recruited cohort of previously-treated head and neck cancer patients between 3 months and 3 years following treatment[109]. Utility values for terminal and end-of-life stages of disease were derived from the same source as the Speight model[124, 178]. People with no disease or with disease in full remission (after ten years follow-up) were assumed to have a utility of 1.0. Utility values were applied to each entity assuming a Beta distribution.   113     4.7 Derivation of Parameter Estimates from Data Sources The following section describes the statistical approach used to produce individual estimates of each model parameter, based on the type of data used for the parameter estimates. Generalized Linear Modeling (GLM) techniques were used to derive time-to-event values from primary data sources. GLM is a mathematical expression of the relationship between two or more variables through the statistical fitting of a linear equation. The predicted value of some dependent variable (Y) can be estimated through a linear predictor of a number of independent variables (  ) and a link function that estimates the mean that is derived from assuming a given statistical distribution:               GLM regression functions can be fit assuming a variety of statistical distributions, including the Weibull distribution, when using the corresponding link function. The Weibull function is highly flexible, and can be used to approximate a number of time-to-event functions. The Weibull probability density function can be expressed in terms of two parameters,   and   (referred to as the scale and shape parameters respectively):                              where t is the time whose probability is being described.  114     The scale parameter  can be expressed as    , allowing for mathematical predictions of time t elapsing between two events to be calculated as a function of other independent variables (    and their coefficients of association ( 0,  ). GLM regressions were performed on time-to-event data from the Zhang trial data and the ROCC to determine the associated coefficients and Weibull parameters (  and  ). This approach was used to take advantage of the stochastic nature of these datasets, and to reflect multiple entity characteristics simultaneously. A Python function was written to generate random draws from a Weibull distribution based on an entity’s characteristics, the regression coefficients associated with those characteristics, and Weibull parameters from a best-fitting curve. Regression analysis was conducted using the LIFEREG function in SAS 9.4 (SAS Institute, USA), which returns estimates of  0,  n, and . Given these parameters, it was possible to use linear regression methods to generate predicted time-to-event values for an entity with a given set of characteristics using the function numpy.random.weibull() in Python. Beta Distributed Values Time-independent probabilities are estimated using the Beta distribution. This distribution is bound between zero and one and its probability density function can be expressed as a function of two parameters   and  : 115                                   where  describes the bounds of the distribution between zero and one. Probabilities can be sampled from the Beta distribution given the value of a mean probability (p) and standard deviation ( ) using the following equation, derived from the method of moments for the Beta distribution:                              This pair of equations can be applied to a mean and standard deviation to sample a random probability, which can be called using the function numpy.random.beta(α,β) in Python. Normally (Gaussian) Distributed Values Normally distributed variables are sampled from the Gaussian distribution. This distribution describes continuous values bound between negative and positive infinity and its probability density function can be expressed as a function of its mean (μ) and standard deviation ( ):                               116     This equation can be used to sample a random value based on the mean and standard deviation, which can be called using the function numpy.random.normal(μ, ) in Python. Gamma Distributed Values Estimates of unit cost are samples from the Gamma distribution. This distribution is bound by 0 and infinity and its probability density function can be expressed as a function of a shape parameter   and a scale parameter  :                           where  is a complete gamma function. Values can be sampled from the Gamma distribution given the value of a mean (μ) and standard deviation ( ) using the following equations, derived from the method of moments for the Gamma distribution:               This pair of equations can be applied to a mean and standard deviation to sample a random value, which can be called using the function numpy.random.gamma( , ) in Python. Dirichlet Distributed Values 117     The Dirichlet distribution is related to the Beta distribution, and can be used to randomly sample probabilities (values between 0 and 1). The Dirichlet distribution is multinomial, meaning it can be used to sample multiple random probability values that sum to 1 (i.e., mutually exclusive probabilities that cumulatively represent all possible outcomes) The distribution is bound by 0 and 1 and its probability density function can be expressed as a function of a multivariate Beta function  and a vector of k integers (                               Random probabilities can be sampled from a Dirichlet distribution using the function numpy.random.dirichlet(  in Python. Log-normal Distributed Values The log-normal distribution is related to the Normal distribution, and can be used to randomly sample continuous values bound between negative and positive infinity. Its probability density function can be expressed as a function of a logarithmic mean (μ) and associated standard deviation ( ):                                    Random values can be sampled from a log-normal distribution using the function numpy.random.normal( ,  ) in Python. 118     Transition Probabilities Transition probabilities that are published as a mean and standard deviation can be used to generate time-to-event values randomly sampled from a one-parameter Weibull distribution using a two-step process. In the first step, a transition probability value tp is sampled from a Beta distribution using a given mean probability (p) and standard deviation ( ). In the second step, the Weibull function is assumed to have a shape parameter  equal to 1[180]. In the presence of this assumption, the Weibull distribution is equal to the exponential distribution:                      where              . Random values can be sampled from an exponential distribution using the function numpy.random.exponential( ) in Python. The above methods were used to derive parameter estimates for each entity as it moved through each component of the model. 4.8 Verification, Validation, and Calibration of the WDMOC In order to draw useful conclusions from a model, it is necessary to ensure that the model produces estimates that match its structural assumptions while matching values seen in the population it is 119     intended to represent – in this case, the population of British Columbia. This is accomplished through a three-part process of model verification, validation, and calibration. Model verification is the process of ensuring that the model is doing what it is intended to do. The WDMOC is made up of multiple processes, functions, and processes that govern the behaviour of entities as they are created and move through the simulated system. Verification was conducted on each process as it was constructed to ensure that it performed its intended function (i.e., that the values produced by each function matched the expected parameter input values and their associated distributions). Model validation refers to the process of ensuring that the behaviour of the entities within the model follows their expected trajectory – that movement through the model corresponds to the theoretical path that real people would take across their natural history of disease and through the health care system. Validation exercises were conducted within each model component (e.g., Asymptomatic/Screening, OPL Management, Invasive Disease, etc.) on an entity-by-entity basis to ensure that events were occurring in the expected chronological order and with plausible values. This process was repeated to look at entities as they moved through the entire model, from entity creation to death. The final stage of the process is model calibration. Even in cases where the model’s inputs may be drawn from representative sources, it is reasonable to expect that parameter interactions within the structure of the model may result in output values that differ from those observed in the real world. 120     Accordingly, it is typically necessary to ‘calibrate’ a model by making reasonable adjustments to its parameter values and/or structural assumptions[24]. The first step in model calibration involves choosing a set of ‘target’ values that are important to the model’s predictive validity. It is most appropriate to choose a set of targets that reflect multiple points across the disease process being modeled, as using a single ‘target’ may obscure intermediate values that are not accurate, yet nonetheless produce reasonable values of the single target. The following calibration targets were chosen: 1. Prevalence of oral premalignancy at time of diagnosis by age and sex. 2. Prevalence of cancer at time of diagnosis by age, sex, and stage 3. Age at death among entities with oral cancer These targets represent initial, intermediate, and final values that are related to an entity’s clinical trajectory within the model (i.e., age and sex are statistically related to premalignant progression; age, sex, and stage are related to treatment response and survival). The second step in model calibration involves assessing how well the model’s output fits reasonable values for the targets. While there are several methods that can be used to assess goodness of fit, the underlying complexity of the WDMOC, the heterogeneity in parameter data sources, and the relative lack of data for many parameter values suggested that a simpler approach was the most practical. Accordingly, an acceptable window approach was chosen. The acceptable window method compares the mean and variance of the identified target outputs to the mean of real-world values for those same 121     targets. Adjusted parameter values were accepted when model output for the mean and standard deviation of the calibration targets were similar. The values and data sources used in the acceptable window calibration process are described in Table 4.10. The model was run for one hundred iterations of 25,000 entities per iteration (for a total of 2,500,000 entities), to ensure at least 10 cancers diagnosed at each stage for each iteration. Values of the target outputs were calculated for each iteration, and the mean and standard deviation were calculated across all iterations (i.e., for the full run). The results of this process are provided in Table 4.11. Table 4.10 – Calibration output target values Target  Mean Source Prevalence of OPL 0.9% [72, 73]  Men 55% [61]  Women 45% [61]  Age 59 [61] At cancer diagnosis    Men 58.5% ROCC  Women 41.5% ROCC  Age 65.6 ROCC  Stage I 37.3% ROCC  Stage II 35.0% ROCC  Advanced Stage 27.4% ROCC Age at death from cancer 70.6 ROCC    122     Table 4.11 – WDMOC baseline calibration outputs Target Reference Model Output Distance (z score)   Mean SD  Prevalence of OPL 0.9% 0.908% 0.027% 0.22 Men 55% 63% 4.8% 1.67 Women 45% 37% 4.8% -1.67 Age 59 59.0 2.31 -0.18 At cancer diagnosis     Men 58.5% 58.6% 12.9% 0.008 Women 41.5% 41.4% 12.9% -0.008 Age 65.6 62.2 1.9 -1.79 Stage I 37.3% 36.4% 8.0% -0.11 Stage II 35.0% 31.1% 8.1% -0.48 Advanced Stage 27.4% 26.7% 7.4% -0.09 Age at death from cancer 70.6 64.96 9.71 -0.41  After each run, select parameters were manually adjusted in order to produce outputs that more closely matched the target values. This process was repeated until the model’s outputs match the target values. The following calibrating adjustments were made to the WDMOC:  The mean and standard deviation for the ‘starting age’ parameter was adjusted to reflect the age distribution of detected premalignant lesions within the Zhang et al study of malignant transformation[61];  The age and sex parameters for premalignancy prevalence were adjusted (from their baseline values within the Speight model) using the published Relative Rate values from a population prevalence study within British Columbia[72];  A constant relative prevalence value was introduced to adjust the overall prevalence of premalignant lesions to values within the same study;  A parameter representing the likelihood that an entity will receive regular screening from their dentist was created, in order to adjust the percentage of premalignancies and cancers that are detected symptomatically vs. through screening; The output suggests that the calibrated WDMOC produces estimates of prevalence and age/stage distribution that approximate the target values. There is a noticeable gender difference at the time 123     premalignancies are detected, and the age at which cancers are detected is slightly lower in the model output than in the target data. This early age at cancer detection carries forward to produce an earlier death from cancer in the model output than in the target values. These discrepancies will be discussed later in this chapter. 4.9 Univariate Sensitivity Analysis  Following verification, validation, and calibration, the final step of quality assurance in model design is conducting univariate sensitivity analysis, in order to investigate the impact that changes in single parameter values have on the model’s outputs. In this exercise, univariate sensitivity was conducted in the following way:  A group of model outputs was selected, chosen to represent key informative outputs of the model;  A parameter input was adjusted to 10% of its baseline value. This value was chosen arbitrarily, but allows for a constant and comparable level of change across all model parameters;  An analysis set of 100 iterations of 25,000 entities (2,500,000 entities in total) was run with the adjusted value. The mean value of each model output was calculated for the analysis set.  The resulting means were compared to mean output from a baseline set with all parameters at baseline values (100 iterations, 25,000 entities, resulting in a total of 2.5m entities)  Parameter sensitivity was calculated as the sum of z scores between the means of the high (i.e., +10%) and low (i.e., -10%) analysis sets:                                                                      124     The following output results were selected: OPL prevalence, cancer prevalence, cancer stage at detection, age at death from cancer, and mean cost and QALY per entity that began the model with an OPL. These values are summarized in Table 4.12. Table 4.12 – Baseline outputs for sensitivity analysis Model Output Baseline value  Mean SD Prevalence of OPL 0.908% 0.027% Prevalence of cancer 0.150% 0.02% Stage at cancer detection   HGL 5.75% 3.3% Stage I 36.4% 8.0% Stage II 31.1% 8.1% Stage III/IV 26.7% 7.4% Age at cancer death 64.96 9.71 Mean cost† $5,294.38 $172.61 Mean QALY† 15.35 0.71 † – these values were calculated for entities who begin the model with an OPL Because the majority of entities in the model do not experience disease, the model was most sensitive to changes in the natural history parameters, particularly starting age. Table 4.13 lists the five parameters that exerted the largest influence on each output variable.   125     Table 4.13 – Univariate sensitivity results by model component Model Output Parameter Model Component Change (ΔZ score) Prevalence of OPL    OPL prevalence conversion factor Entity Creation 6.98  Cohort starting age Entity Creation 3.46  Prevalence of oral premalignancy Entity Creation   Men, 50-59  3.47  Men, 60-69  3.33  Women, 50-59  2.89 Prevalence of cancer    OPL prevalence conversion factor Entity Creation 1.91  Prevalence of oral premalignancy Entity Creation 0.88  Men, 60-69 Entity Creation 0.85  Men, under 50 Entity Creation 0.85  Women, 50-59 Entity Creation 0.81  Time to OPL progression to Stage I cancer Natural History   Smoking Status – Ever  0.82 Stage at cancer detection   HGL     Time to undetected cancer progression  Natural History   Stage I to Stage II  0.72  Prevalence of oral premalignancy Entity Creation   Women, 70-79  0.59  Return appointment interval Screening 0.56  Has access to dentist Entity Creation 0.52  Time to symptomatic detection of cancer  Natural History   Stage I  0.50 Stage I     Time to undetected cancer progression Natural History   Stage I to Stage II  0.69  Time to symptomatic detection of cancer  Natural History   Stage I  0.51  Smoking prevalence – Women Entity Creation 0.47  Prevalence of oral premalignancy Entity Creation   Women, 50-59  0.42  Screening adherence Screening 0.36 Stage II     Time to undetected cancer progression Natural History   Stage I to Stage II  1.11  Time to symptomatic detection of cancer  Natural History   Stage II  0.78  Prevalence of oral premalignancy Entity Creation   Women, 60-69  0.54 126     Model Output Parameter Model Component Change (ΔZ score)  Women, 50-59  0.39  Time to undetected cancer progression  Natural History   Stage II to Stage III  0.48 Stage III/IV     Time to symptomatic detection of cancer  Natural History 0.69  Stage II    Prevalence of oral premalignancy  Women, 60-69 Entity Creation 0.66  Screening adherence Screening 0.66  Time to undetected cancer progression  Natural History   Stage II to Stage III  0.50  Return appointment interval Screening 0.41 Age at cancer death    Time to Second Event – Death Terminal Disease   Sigma (σ)  1.20  Age  0.99  Cohort starting age Entity Creation 0.99  Time to First Event – Death Invasive Cancer   Treatment type – Surgery + RT  0.98  Cancer stage – Advanced  0.96 Mean cost† Time to OPL progression to Stage I cancer Natural History   Sigma (σ)  9.52  Intercept  (β0)  6.17  Time to First Event Invasive Cancer   Age  7.49  Intercept  (β0)  3.85  Time to First Event – Death Invasive Cancer   Age  6.02 Mean QALY†     Cohort starting age Entity Creation 6.52  Utility – Well (no disease) Multiple 1.62  Utility – Undetected OPL Screening 0.88  Prevalence of oral premalignancy Entity Creation   Men, <50  0.76  Men, 60-69  0.65  The model is not sensitive to changes in most individual model parameters, but can experience relatively large changes in important output values from even small changes in key parameters, particularly those concerning the age of the cohort and the prevalence and natural history of oral disease. Costs are highly 127     sensitive to changes in the risk of premalignant transformation and cancer survival. The full implications of this parameter sensitivity will be discussed in greater detail in Chapter 7, but a brief discussion follows. Pre-malignant disease is not as well-characterized in oral cancer as it is among other diseases where screening is widely practiced – cancers of the cervix, breast, and prostate. The sensitivity of the model to changes in natural history parameters seems to make a strong case for the value of future research in the population prevalence, incidence, and development of novel premalignancies. It also suggests that the requirements of the Whole Disease Framework can best be met in disease areas where pre-symptomatic disease trajectory is well understood. The cost-effectiveness analyses that follow this chapter are focused on novel technology adoption that occur after premalignancies have been diagnosed, which limits the impact that natural history parameter sensitivity has on cost-effectiveness estimates. Nevertheless, it remains imperative to recognize this impact within the context of this dissertation as a whole. 4.10 Discussion 4.10.1 Strengths of the approach The WDMOC was designed and implemented in accordance with guidelines set out for the WDM framework. It considered the full breadth of the disease pathway, from preclinical disease to death. The depth of the model was sufficient to reflect potential policy changes at multiple levels, guided by factors that were associated with preclinical disease progression and survival after diagnosis of invasive disease. 128     The model’s boundary was adults in British Columbia, calibrated to resemble a population with newly-diagnosed OPLs. These model characteristics make the WDMOC suitable for evaluating the impact of policy changes on a population of adults with newly-diagnosed OPL, but the decision node of the model can be moved to consider populations with invasive cancer or other disease sequelae. 4.10.2 Limitations of the approach The nature of modeling requires simplifying assumptions to be made, and the WDMOC as described above is no exception. Parameters and pathways governing screening and treatment were limited by data availability. The individual sampling approach that was chosen for the model’s design does not reflect health care system wait lists. Assumptions were made in the model’s calibration as well. The implication of these assumptions will be described in the following section.  4.10.2.1 Clinical Processes Chapter 2 highlighted the relative lack of high-quality data on screening frequency and practice, particularly as it relates to how demographic factors like age and smoking status affect screening frequency. The screening process described in the WDMOC assumes that all entities with access to a screening dentist are treated identically at regular intervals, based on available guidelines and the opinion of the expert stakeholder group. Cancer treatments were similarly simplified, although better data were available to guide these assumptions. Treatment options were collapsed into categories (surgery, surgery + RT, other) based on a regression analysis exercise (see Appendix F), which consequently limits the ability of the model to 129     reflect the full complexity of possible treatment approaches. The 90-day window for treatment costs was also chosen based on the available data and expert opinion, and does not reflect every potential treatment regimen that a given patient could experience. Local, locoregional, and regional recurrences were treated as synonymous events due to the limited number of events in the data. These simplifications limit the ability of the model to reflect the full complexity of screening practice and invasive disease management, and the results produced by the WDMOC should be interpreted with these limitations in mind. It is worth noting, however, that the level of evidence used within the WDMOC is either at par with (i.e., directly drawn from) or a step above what is available in the extant literature. 4.10.2.2 Individual Sampling and Queuing The model assumes that referrals are instantaneous and resources are always available. This is a simplifying assumption that is common in health economic decision models, especially since data on queueing is not typically available. In future cases where there is a compelling reason to believe that waiting is meaningfully associated with the cost and/or effectiveness of a new technology, the effect of queuing could be reflected in the model by adding a ‘wait time’ parameter. The consequence of not including queueing means that referrals between specialists (e.g., referral to OPL management, referral to an oncologist, time between referral and treatment, etc.) happens on a time scale that may be days/weeks earlier than what would be observed in real-world practice. 130     4.10.2.3 Model Calibration The calibration method chosen for the WDMOC produced a population that developed cancer slightly earlier than the calibration targets. This discrepancy was due to the data sets used to derive parameter estimates. Values related to OPL progression were derived from literature-published estimates and a clinical trial, while values related to oral cancer survival were derived from a population-based cohort. Because these values were drawn from separate populations, it is reasonable to expect that the values they produce would not necessarily match up exactly. The clinical trial population underwent more rigorous regular observation than is typical for people in the general population, and was drawn from a population that is likely subject to selection biases that are typical for clinical trials (e.g., better educated, wealthier, fewer comorbidities, etc. than the general population). The committee agreed that it was important for prevalent cases of oral cancer to resemble the age distribution of the population from which parameter estimates were derived, given that age is significantly associated with time to progression to cancer (see Table 4.2). Because the statistical association between sex and cancer survival was stronger than the corresponding association to progression to invasive cancer, the model was calibrated to ensure that the sex distribution matched the cancer survival cohort. When interpreting the TTE estimates for progression from OPL to invasive cancer, it is also important to note that the underlying study from which the estimates are drawn considered time from OPL detection to progression, rather than the time from development. The earlier age of cancer detection will also be a product of this discrepancy, and the lack of available data about OPL incidence. 131     Despite the limitations of the data sources, the model’s outputs were broadly similar to values observed in the calibration targets. 4.11 Conclusion This chapter described the process through which a Whole Disease Model of Oral Cancer was conceived and designed, using the guidelines set out by Tappenden. This description included a summary of the model’s structure and implementation, as well as data sources and statistical procedures. The model’s outputs were compared to expected values, and then the impact of changes in each parameter on key outputs was explored through sensitivity analysis.  The next chapter considers the same cost-effectiveness question as explored in Chapter 3 – cost-effectiveness of a risk-guided approach to premalignant management – but this time uses the WDMOC. Chapter 6 describes an exercise exploiting the ability of a Whole Disease Model to evaluate the ‘upstream’ impact of policy changes on the ‘downstream’ cost-effectiveness of new technologies.  132     5 Chapter 5 – Case Study Comparison of WDMOC to Conventional (Markov) Modeling This chapter will describe the use of the WDMOC to evaluate a single ‘piecewise’ decision, comparing the WDMOC approach to the conventional Markov approach described in Chapter 3. 5.1 Introduction In Chapter 3, an economic evaluation was conducted to estimate the cost-effectiveness of a molecular assay for risk-stratified management of OPLs using a conventional Markov decision analytic model. This model found that using a genomic assay to classify OPLs by their LOH-related risk was a cost-effective method of managing OPLs, compared to standard practice[148]. This model was limited by the fact that it was a ‘piecewise’ model, considering a limited scope of the disease that was most relevant to this single decision, but not other aspects of the oral cancer disease pathway. This limitation, common to piecewise models, invites the question of whether a more comprehensive model would provide additional relevant information to the question of cost-effectiveness. This chapter describes an exercise in which the WDM described in the previous chapter (the WDMOC) was used to re-evaluate the cost-effectiveness of the assay in a population of patients with a detected OPL. The WDMOC considers screening by community dentists and can reflect a more nuanced picture of disease progression and treatment than was practicable with the Markov model in Chapter 3. Further, the WDMOC is based on a more comprehensive set of parameter estimates, with many of these estimates informed by better sources of data. Finally, the DES nature of the model will introduce the 133     effect of demographic factors (age, sex, smoking status, etc.) and clinical factors (treatment history) that were also not practicable within a Markov framework. Each of these factors is expected to influence the cost-effectiveness of the assay. 5.2 Methods A whole disease model of oral cancer (WDMOC) was constructed in the Python programming language (Python Software Foundation, Delaware, USA) (see Chapter 4).  5.2.1 Model Logic The model logic of the WDMOC is described fully in §4.3, and is summarized below.  The WDMOC is an individual sampling time-to-event simulation model, in which entities (simulated patients) move through the model according to its underlying structure. The WDMOC’s structure is divided into 5 separate components: 1. Screening/Asymptomatic Disease: entities undergoing regular dental appointments. 2. OPL Management: entities undergoing regular follow-up for a detected premalignant lesion 3. Incident Cancer Treatment: entities with detected cancers undergo treatment depending on their disease characteristics 4. Post-treatment Followup: entities undergoing regular post-treatment follow-up appointments with their oncologist 5. Terminal disease: entities with incurable disease undergo best supportive care until dying of their disease. The WDMOC was used to simulate two separate health policy decisions (model arms). In the ’Assay Informed‘ arm, an assay was used to guide OPL surveillance appointments based on risk category. In the ’Assay Naïve‘ arm no assay is used and OPL surveillance was conducted according to standard practice 134     (once every 6 months). After 5 years, the OPL is presumed to be noncancerous and the patient is discharged from regular follow-up. Entities with access to a community dentist will continue to see that dentist on a regular basis. Simulated patients started in the ’OPL Management‘ component with a diagnosed OPL that may progress to invasive cancer according to its LOH risk score, estimated for each entity based on data from a previously conducted clinical trial. In the Assay Informed arm, patients were managed according to their risk score, people with ’high risk‘ lesions receiving immediate surgical treatment with an attenuated surveillance schedule for those with ’medium‘ and ’low‘ risk lesions (3 years and 5 years, respectively).  OPLs could progress to invasive cancer over time based on LOH risk score, age, sex, and smoking status based on results from a previously-published clinical trial. Cancers could progress from Stage I  Stage II  Advanced Stage (III and IV). Stage I cancer could be detected either as a high-grade lesion (HGL) or an invasive squamous cell carcinoma (SCC).  Cancers at all stages was treated with surgery alone, surgery and radiation, or some other combination of therapies based on stage at diagnosis. Following treatment, patients could experience disease recurrence, based on demographic and clinical characteristics (stage, age at detection, sex, and treatment type). Recurrence treatment and the possibility of death from disease were estimated in a similar way. After treatment for incident cancer or recurrence, patients were followed regularly by their oncology team until three months before their death, at which point they received end-of-life care. 135     Patients surviving at least five years after their initial diagnosis without disease progression were considered to be in full remission and were discharged from regular care. 5.2.2 Parameter Estimate Data Sources Parameter estimate calculation and data sources are described fully in section 4.5, and is summarized below. Simulated entities in the WDMOC move between model components and experience events at discrete time intervals (days, or fractions thereof). The value of each time interval is drawn from an underlying statistical distribution based on the entity’s characteristics and model parameter values. Values for other model parameters (i.e., non-time-dependent probabilities, unit cost estimates, health utility values) were estimated in a similar way. The sources of data used to estimate each parameter are described in detail in Chapter 4. A brief summary follows. Newly-created entities were assigned demographic and disease characteristics, derived from four sources: population statistics published by Statistics Canada[189], figures published by the Canadian Dental Association[184], values used in a previously published oral screening model[124], and values used in a previously  published clinical trial of patients with OPLs[148]. Parameter estimates for the natural history processes (OPL progression to invasive cancer, cancer stage progression) were derived from two sources: secondary analysis of the Zhang et al trial cohort[61], and the values published in the Speight et al oral screening model[124]. Parameter values concerning management of people with detected OPLs were derived from a values published in a Markov model of oral cancer screening[124], 136     and values published in a previously published Markov model of risk-guided OPL management[148]. Values concerning the passage of asymptomatic people through regular dental appointments were derived from three principal sources: the oral cancer screening model[124], an opportunistic screening study from the US[73], and a similar study conducted in British Columbia[72]. Parameters concerning treatment type and outcomes for incident and recurring cancers were derived from the manual chart review of a retrospective cohort of 864 patients previously treated for oral cancer in the province of British Columbia between January 1, 2001 and December 31, 2015 (diagnosed between January 1, 2001 and December 31, 2009). Resources used at each event were recorded for each entity. A unit costing approach was applied to these resources to estimate total cost for each entity. The costs of treatment were estimated via a linkage exercise between the same retrospective cohort and data held by BC Cancer and the British Columbia Ministry of Health (including chemotherapy drugs, radiotherapy fractions, resources utilized in a hospital setting, provincially insured fee-for-service billings, Medical Services Plan billings, and prescriptions covered under PharmaCare). The cost of the assay was assumed to be $500. Health statue utilities were retrieved from sources cited in a recent systematic literature review[113]. Health utility experienced by people in remission was assumed to be 1.0.  5.2.3 Cost-Effectiveness Analysis By simulating a large population of entities, the model was able to simulate a reasonable range of costs and quality-adjusted survival values for that population. Incremental cost-effectiveness analysis was 137     conducted by comparing costs and outcomes for populations simulated within the model’s two arms. Mean estimates for costs and QALYs, with 95% Confidence Intervals, were estimated through a bootstrapping process, where cost and survival values for a cohort of 10,000 entities were sampled (with replacement) 1,000 times from an underlying simulated population of 10,000.  Costs and outcomes in both the conventional Markov model and the adapted WDMOC were discounted for future time preference at an annual rate of 1.5%, expressed as a continuous daily discount rate[5]. Costs for both were expressed in 2017 Canadian Dollars, adjusted for inflation using the Consumer Price Index for health care. A ten year time horizon (3,650 days) was chosen for the analysis, representing a point at which all entities are either dead (from cancer or another cause), in 5-year remission from a detected cancer, or unlikely for their OPL to progress to invasive cancer. Incremental cost-effectiveness ratios (ICERs) were calculated, in which the incremental mean cost experienced by patients in each arm was compared to incremental quality-adjusted life years (QALYs). The impact of decision uncertainty (probabilistic analysis) was expressed on the cost-effectiveness plane and by generating cost-effectiveness acceptability curves (CEACs), which consider the proportion of bootstrapped ICERs that lie below a threshold of society’s willingness to pay for an additional QALY (a value denoted as  ). A CEAC illustrates the probability that an intervention is cost-effective for a varying value of  . 5.2.4 Secondary Model Outputs The following secondary model outputs were compared in addition to total cost and QALY: 138      Number of entities that developed invasive cancer  Stage at which cancer was initially detected  Number of entities that died, either of cancer or of another cause. 5.2.5 Scenario Analysis Cancers could be detected either during specialist follow-up appointments or at regular dental appointments. No data was available on the proportion of OPL patients that had regular access to a dentist. Consequently, three scenarios were evaluated wherein this proportion was adjusted. In the baseline scenario, the proportion of people with access to regular dental appointments was assumed to be equal to the default WDMOC value. A second scenario assumed that everyone has regular access to a dentist, while a third considered a population with no access to a dentist. An additional scenario analysis was performed where low-risk OPLs were followed up every three years, while intermediate-risk OPLs were followed up every year. 5.3 Results 5.3.1 Cost-Effectiveness Results The WDMOC-generated estimates of cost, survival, and quality-adjusted survival are presented in Table 1. Assay-informed care resulted in a mean cost of $7,680 per patient (95% CI: 6,710 – $8,650), compared to $11,451 per patient under standard practice (95% CI: 10,473 – 12,429). Survival was similar between both model arms. Quality-adjusted survival was slightly higher in the Assay Naïve arm than the Assay 139     Informed arm, but mean values were within the confidence bands between arms suggesting that this difference was not statistically significant. Results are summarized in Table 5.1. Table 5.1 – Cost-effectiveness analysis results Arm Estimate Mean 95% CI Assay Informed       Cost $7,680 6,710 – 8,650  LYG 7.99 7.84 – 8.14  QALY 7.44 7.28 – 7.60 Assay Naive       Cost $11,451 10,473 – 12,429  LYG 7.99 7.84 – 8.14  QALY 7.49 7.33 – 7.65 Incremental       Cost -$3771 -5,151 – -2,391  LYG -0.004 -0.22 – 0.21  QALY -0.051 -0.28 – 0.18  Cost/LYG N/A     Cost/QALY N/A    CI – confidence interval; LYG – life years gained; QALY – quality-adjusted life years N/A – ICERs with ∆E < 0 are not straightforward to interpret and were not calculated  Risk-guided management of OPLs resulted in a reduction in cost to the health care system (∆C = -$3,771; 95% CI -$-5,151 – -$2,391) with non-significantly lower quality-adjusted survival (∆E = -0.051 QALY; 95% CI -0.28 – 0.18). Probabilistic analysis results are shown in Figure 5.1. 67% of sampled QALY values were below zero (i.e., assay informed care produced a reduction in quality-adjusted survival an estimated 67% of the time) – values that fall within this range are not typically considered cost-effective at any value of  .   140     Figure 5.1 – Cost-effectiveness plane and cost-effectiveness acceptability curve of 10,000 bootstrapped values of cost and quality-adjusted survival    141     5.3.2 Secondary Outcomes Secondary model outcomes are presented in Table 5.2, based on a simulated cohort of 100,000 entities. The overall number of detected cancers was lower in the Assay Informed arm (relative rate = 0.81). Despite this, there was a higher rate of cancers in the Assay Naïve arm that were detected at Stage II or higher. Overall cancer mortality was similar between the two arms. Table 5.2 – Comparison of Secondary Model Outputs from a simulated cohort of 100,000 OPL patients Scenario Output Assay Informed† % Assay Naïve % Relative Rate Baseline       Cancers 14,145   17,389  0.81  HGL 2,049 0.14 4,267 0.25 0.59  Stage I 6,617 0.47 10,075 0.58 0.81  Stage II 3,498 0.25 2,585 0.15 1.66  Advanced Stage 1,981 0.14 462 0.03 5.27  Deaths 28,869  28,546  1.01  From cancer 5,940 0.21 5,971 0.21 0.98  From other cause 22,929 0.79 22,575 0.79 1.00 All see a dentist           Cancers 13,977   17,375   0.80  HGL 3,387 0.24 4,448 0.26 0.95  Stage I 8,057 0.58 10,324 0.59 0.97  Stage II 2,153 0.15 2,260 0.13 1.18  Advanced Stage 390 0.03 343 0.02 1.41  Deaths 27,678  28,633  0.97  From cancer 4,839 0.17 5,654 0.20 0.89  From other cause 22,839 0.83 22,979 0.80 1.03 None see a dentist           Cancers 13,823   17,322   0.80  HGL 1,611 0.12 4,111 0.24 0.49  Stage I 5,896 0.43 10,003 0.58 0.74  Stage II 3,952 0.29 2,755 0.16 1.80  Advanced Stage 2,364 0.17 453 0.03 6.54  Deaths 28,938  28,815  1.00  From cancer 6,103 0.21 5,966 0.21 1.02  From other cause 22,835 0.79 22,849 0.79 1.00 142     Scenario Output Assay Informed† % Assay Naïve % Relative Rate 3yr and 1yr follow-up           Cancers 14,064   17,642  0.79  HGL 2,609 0.19 4,251 0.24 0.77  Stage I 7,362 0.52 10,195 0.58 0.91  Stage II 2,933 0.21 2,788 0.16 1.32  Advanced Stage 1,160 0.08 408 0.02 3.57  Deaths 28,171  28,968  0.97  From cancer 5,418 0.19 6,059 0.21 0.92  From other cause 22,753 0.81 22,909 0.79 1.02 HGL – high-grade lesion † – The number of detected cancers in the Assay Informed arm does not include premalignant lesions that are identified as “high risk” at baseline testing with the assay. Note: A relative rate less than 1.0 favours the use of the assay, while a relative rate larger than 1.0 favours current standard of practice.  5.3.3 Scenario Analysis In a scenario where all entities regularly saw a dentist, incremental cost was -$11,004 (95% CI: -12,278 – -9,729) and incremental QALY was 0.008 (95% CI: -0.22 – 0.24). In a scenario where no entities regularly saw a dentist, incremental cost was -$1,052 (95% CI: -2,400 – 295) and incremental QALY was -0.042 (95% CI: -0.27 – 0.19). In a scenario where low-risk lesions were evaluated every three years and intermediate-risk lesions were followed up annually, incremental cost was $-1,437 (95% CI: -2,776 – -97) and incremental QALY was -0.024 (95% CI: -0.25 – 0.20). The stage distribution of incident cancers was influenced by access to regular screening (see Table 2). While the overall distribution followed a similar pattern in both alternative scenarios, the relative rates were biased toward 1.0 (no difference in stage distribution) in the scenario where all entities regularly visit a dentist, and away from 1.0 in the scenario where entities do not regularly see a dentist. The oral 143     cancer mortality rate is also affected by dental screening, with a relative morality rate of 0.89 in the scenario where all entities see a dentist. 5.4 Discussion Cost-effectiveness analysis using the WDMOC found that use of a genomic assay to provide risk-guided management to patients with identified oral premalignant lesions reduced overall health care costs, but did not improve survival or quality-weighted survival. The results suggest that assay-informed care results in lower overall cancer incidence, but the longer followup interval between screening appointments results in cancers progressing to a later stage before being detected. Use of the assay is more cost-effective in a population with regular access to a dentist, as there is greater probability of cancers being detected in the interval between screening appointments. Caution is warranted when interpreting the cost-effectiveness findings. Raw and quality-adjusted survival estimates were both close to zero, but lay on opposite sides of the X axis in the cost-effectiveness plane. This suggests that, if the mean estimates are accurate reflections of the true effectiveness, cost-effectiveness is driven primarily by lower health state utility values for patients who progressed to advanced cancer. The uncertainty around incremental effectiveness suggests that improved estimates of health state utility and cancer stage progression rates could have a meaningful impact on the cost-effectiveness of this technology. 5.4.1 Comparison to Chapter 3 144     Analysis of this decision problem using the WDMOC produced a different result from the Markov model described in Chapter 3, in which incremental survival results favoured the use of the assay. The secondary and scenario analyses, taken together, suggest that this difference in findings is driven by factors that were not reflected in the Markov model. These factors include biological complexity – cancer stage progression, the possibility of symptomatic detection; and structural relationships between health care components – the effect that regular dental screenings have on stage distribution and mortality.  In order to determine the factors that explain this difference more conclusively, an exercise was conducted in which the structure of the WDMOC was adapted to more closely resemble the Markov model. This exercise is described in Appendix I, but will be summarized briefly here. The WDMOC was structurally adjusted to reflect assumptions made in the Markov model as follows:   Community dentists were not reflected in the care pathway – entities were managed entirely at an OPL clinic;  Cancers were assumed to be the same stage (i.e., no interval progression to later stage cancers) and were described by a single survival function (i.e., survival for all entities was drawn from the same transition probability parameter, without considering stochastic characteristics like sex or treatment type);  Cancers could not be detected symptomatically between OPL follow-up appointments;  Remission was not possible following recurrence (i.e., all recurrences were eventually terminal); The resulting model produced similar outcomes and decision recommendations (i.e., use of the LOH assay yielded higher QALYs at lower cost compared to standard practice), with important differences that were the product of discrete event vs. cycle-based time processing in the WDMOC vs. the Markov model respectively. These remaining differences are also described in Appendix I.  145     This change illustrates the value of a Whole Disease Model. It is reasonable to expect that a more complex model will produce different outcomes from a simpler one. It is uncommon, however, to see models that can reflect all relevant processes for a decision problem, which is precisely the issue that the WDM framework seeks to address. In the above example, modeling the interplay between community dentist and specialist care had a noticeable impact on the cost-effectiveness findings, as did a more nuanced approach to disease natural history – both factors that were not included in the model presented in Chapter 3 or any other model currently published in the literature. Differences of this type are seen in other studies that have compared DES and Markov modeling approaches. Karnon found that DES and Markov models produce similar outcomes from identical parameter sets, but that the differences are expected to be more pronounced in the presence of survival data derived from secondary clinical data (as was the case in this dissertation)[190]. Subsequent comparisons of Markov and DES models[191-195], including those in a cancer context[196, 197], suggest that DES models produce more accurate cost-effectiveness estimates, and are differentially affected by time horizon and overall model complexity when compared to Markov approaches. 5.4.2 Limitations Despite its increased complexity compared to a conventional model, the WDMOC still faces a number of structural limitations. The screening behaviour and accuracy of community dentists is far more complex than the WDMOC is able to simulate. Dentists have a far wider and more varied scope of practice than the WDMOC was able to reflect. Smoking history, age, and previous evidence of disease are likely to change the way a dentist treats their patients; the WDMOC did not have sufficient evidence to reflect 146     such a change. The estimates of stage progression are drawn from expert opinion rather than observational data, and do not reflect factors like LOH risk score or other patient-level factors. Given how close incremental effectiveness is to zero, it is reasonable to suspect that addressing these limitations through better data may exert an important influence on cost-effectiveness. Despite these limitations, the result produced by this model is based on a much more comprehensive structure and set of data than the original Markov model. Importantly, the model’s secondary results highlight areas where further research and a more comprehensive modeling approach could produce still-better estimates of cost-effectiveness. 5.5 Conclusion The WDMOC-based analysis suggests that use of a genomic assay to manage OPLs results in cost savings to the health care system, but does not improve quality-adjusted survival. Use of the assay reduces the overall rate of cancer, but the longer observation period between screening appointments produces a higher number of late-stage cancers. The cost-effectiveness of the assay was influenced by the proportion of patients with access to regular dentist appointments and the interval between screening appointments. Use of the WDMOC highlighted areas where improvements in the model’s structure and inputs are very likely to produce better estimates of cost-effectiveness. The model sensitivity analysis conducted in Chapter 4 found that small changes in a number of parameters, both “upstream” and “downstream” from the policy change described in this exercise (i.e., the use of the molecular assay), influenced key model outputs including cost and quality-adjusted 147     survival. Chapter 6 will describe an exercise that utilizes the full potential of the WDM framework, examining the impact that changes in those parameters may (or may not) have on the cost-effectiveness of using the genomic assay.   148     6 Chapter 6 – Impact of Upstream/Downstream Changes to Decision Environment on Cost-Effectiveness of Risk-Guided OPL Management This chapter will describe the application of the WDM framework through use of the WDMOC to evaluate how ‘upstream’ and ‘downstream’ decisions can affect the cost-effectiveness of a given technology. The chapter will also explore the impact that multiple simultaneous technology adoption affects cost-effectiveness, providing evidence that can be used for Health Technology Management (HTM). 6.1 Introduction In Chapter 3, a conventional Markov decision analytic modeling approach was used to evaluate the cost-effectiveness of a molecular assay that estimates the risk that a patient with an OPL will progress to invasive cancer. The model found that by reducing the frequency of follow-up visits for low- and intermediate-risk patients and immediately treating high-risk patients, use of the assay was associated with reductions in health care costs and an improvement in quality-adjusted survival. The Markov approach was limited in scope and considered a restricted number of health care system, demographic, and clinical factors. These limitations occasioned the creation of a more comprehensive model that considers a wider breadth of events, including ones that may not appear directly relevant to the question of OPL follow-up. The construction of such a model, the WDMOC, was described in Chapter 4. The WDMOC was then used to re-evaluate the assay’s cost-effectiveness, an exercise described in Chapter 5. The WDMOC-based analysis found that while the assay was cost-saving and reduced the overall incidence of cancer, it 149     did not produce a meaningful difference in terms of quality-adjusted survival as a consequence of a shift to late-stage cancers for patients with a long interval between screening appointments. This effect was moderated by the availability of community dental screening and the length of the interval between OPL follow-up screening appointments. The cost-effectiveness of any health technology adoption may be affected by changes in other health care policies. For the purpose of this chapter, ‘upstream’ and ‘downstream’ are defined relative to the adoption of the assay –events occurring before diagnosis of OPL are considered ‘upstream’, while ‘downstream’ events are defined as those occurring after the assay is used to prescribe OPL management.  Screening availability is an example of an effect that is ‘upstream’ from the assay that may affect its cost effectiveness – if more dentists were available and screened more regularly, the assay would be more cost-effective. There may be other policy changes that similarly affect cost-effectiveness, both upstream and downstream of the assay and OPL followup. Policy-makers deciding whether or not to adopt the assay can be guided by economic evaluation evidence that includes the effect of other policy options they may be considering now, or may consider in the future. The WDMOC is suited to provide exactly this type of evidence. This chapter will describe the use of the WDMOC to consider the effect that a slate of potential policy changes may have on the cost-effectiveness of the molecular assay, both individually and in concert. 6.2 Methods 6.2.1 Description of decision problem 150     This exercise concerned the use of a genomic assay to guide management of oral premalignancy. The decision problem has been described in Chapter 3 and again in Chapter 5. Briefly, oral premalignant lesions (OPLs) display a genetic property called Loss of Heterozygocity (LOH) that is associated with the risk of progression to invasive oral squamous cell carcinoma (OSCC). It is currently standard practice for patients with a detected OPL to be evaluated on a regular basis to investigate for evidence of progression to OSCC. The use of an assay that measures LOH would potentially allow patients with high-risk OPL to seek immediate treatment before progression occurs, while simultaneously allowing longer screening intervals for intermediate- and low-risk patients. As described in chapter 5, such a change would result in differences in both costs and outcomes within this patient population. 6.2.2 Scenario Analysis Univariate and probabilistic analyses are useful for understanding the impact that uncertainty around model parameters may have on a model’s outputs. Scenario analysis is a form of model analysis in which parameters and pathways within the model are adjusted to explore the impact of structural changes to the model from its baseline[5]. Scenario analysis can be used to simulate the effect that a hypothetical change (a scenario) may have on the outcome of interest. Scenarios can be created within the model by adjusting parameter values (one at a time or in a multivariate way), and/or by modifying structural relationships between model elements (e.g., by introducing a new treatment or other health system process that does not exist under baseline conditions).  In so doing, it can generate evidence to provide policy makers with useful estimates of what would happen to existing policies in the face of additional changes that are possible/likely to occur in the future.  151     Scenario analysis was performed to estimate the impact that five potential policy changes may have on the cost-effectiveness of using the LOH assay: A Tobacco and Alcohol Cessation Program: tobacco and alcohol use are causally linked to the development of oral premalignancy. A program to help smokers quit, coupled with interventions to reduce heavy drinking, would likely exert an influence on the population prevalence of OPLs. Improved Screening in Community Dental Offices: visual exams, typically performed by dentists and dental hygienists in the community, are useful but not perfect methods of detecting premalignancies and cancer. The routine use of a tool that capitalizes on the autofluorescent properties of OPLs may help dentists detect disease earlier, when it is more easily curable. Conversely, it may also result in a higher number of false positive referrals, which would increase costs without any benefit to patients. Improved Surgical Management of Early-Stage Cancers: early-stage oral cancers are typically managed through surgery. The same autofluorescent properties of oral malignancy and premalignancy can theoretically be used in the surgical theatre to increase the precision of these surgical interventions, lowering the rate of local recurrence and thereby reducing cancer mortality. A New Drug For Advanced-Stage Cancers: advanced cancers may be managed through systemic therapy (chemotherapy). While the cure rate for these cancers is low, scientific breakthroughs in cancer biology periodically produce new drugs that are able to dramatically increase survival in incurable illness. These new drugs are typically very expensive, but provide meaningful improvements in length and quality of life. Vaccination for Human Papillomavirus (HPV): high-income countries around the world, including Canada, have begun vaccinating young people against cancer-causing strains of HPV to prevent cervical cancer. HPV is also recognized as a causal factor in the development of oral cancer, but the evidence on the specific mechanisms behind a causal relationship are not well known. HPV may affect the rate of OPL development, progression to OSCC, and survival rates after OSCC treatment. The potential impacts, across multiple components of the disease trajectory, make WDM an ideal tool to evaluate the impact of HPV vaccination. Two of the proposed scenarios (Tobacco/Alcohol Cessation; Improved Screening) were ‘upstream’ from the LOH assay, as they affect people who have not yet reached the point of OPL management. Two of 152     the scenarios (Improved Surgery; New Drug) were ‘downstream’. The HPV Vaccination scenario was related to disease outcomes both ‘up-‘ and ‘downstream’ from the assay, given that it is hypothesized to affect both disease incidence and post-detection survival. 6.2.3 Whole Disease Model of Oral Cancer (WDMOC) Policy analyses were performed using a Whole Disease Model of Oral Cancer (WDMOC). The model has been previously described in Chapter 4 and Chapter 5. Briefly, the WDMOC is an individual sampling time-to-event simulation model, in which entities (simulated patients) move through the model according to its underlying structure, divided into 5 separate components: 1.  Screening/Asymptomatic Disease: entities undergoing regular dental appointments. 2. OPL Management: entities undergoing regular follow-up for a detected premalignant lesion 3. Incident Cancer Treatment: entities with detected cancers undergo treatment depending on their disease characteristics 4. Post-treatment Followup: entities undergoing regular post-treatment follow-up appointments with their oncologist 5. Terminal disease: entities with incurable disease undergo best supportive care until dying of their disease. The WDMOC was used to simulate two separate health policy decisions (model arms). In the “Assay Naïve” arm no assay is used and OPL surveillance was conducted according to standard practice (once every 6 months). In the “Assay Informed” arm, patients are managed according to their risk score, people with “high risk” lesions receiving immediate surgical treatment with an attenuated surveillance 153     schedule for those with “medium” and “low” risk lesions (2 years and 5 years, respectively). Entities with access to a community dentist continue to see that dentist on a regular basis.  After 5 years, the OPL is presumed to be noncancerous and the patient is discharged from regular follow-up. If an entity progresses to cancer, that cancer is treated and the entity may experience recurrence, remission, and/or death from disease. Entities may also die of causes unrelated to oral cancer. Simulated patients in the scenario analysis began in the “Screening/Asymptomatic Disease” component. Some entities begin the simulation with an OPL, the probability of which is determined from associated demographic factors (age, sex, smoking, alcohol use). They moved through the model according to the parameters that govern disease and health care system events. Resources used at each event were recorded for each entity. A unit costing approach was applied to these resources to estimate total cost for each entity. Health state utility values were applied at each event as well, and were used to calculate quality-adjusted life years gained (QALYs). In Chapter 5, the WDMOC found that using the assay reduced the rate at which cancers developed in a population with detected OPLs (14% vs. 17%). Using the assay did not improve the overall mortality rate, due to a higher rate of late-stage cancers developing during the longer screening interval (14% vs. 3%). 6.2.4 Scenario Creation in the WDMOC Each scenario was created by modifying parameter values and/or introducing new parameters and event relationships into the WDMOC. 154     6.2.4.1 Tobacco/Alcohol Cessation The Tobacco/Alcohol Cessation (TAC) scenario considers the case of a cohort in whom programs had been instituted to reduce rates of tobacco and alcohol use. It was created by adjusting the frequency of smoking and alcohol use among modeled entities. A ‘smoking cessation’ and/or ‘tobacco cessation’ resource was applied to entities who were ever smokers and/or heavy alcohol users under baseline conditions. Each entity’s risk of tobacco and/or alcohol use was re-revaluated against the incremental efficacy of the cessation program, resulting in the possibility of a change in smoking and/or alcohol status from ‘ever smoker’ to ‘never smoker’ or from ‘heavy alcohol use’ to ‘non-heavy alcohol use’, respectively.  Parameter values for cost and the incremental program efficacy for tobacco cessation were derived from a cost-effectiveness evaluation of a population-based smoking cessation program conducted in Boston, USA[198]. Values for alcohol reduction were derived from a cost-effectiveness evaluation of a brief intervention program to reduce alcohol overuse in the UK[199]. The effects of tobacco and alcohol reduction on costs and survival outcomes outside the context of oral cancer were not included in the scenario analysis – this limiting assumption will be discussed in section 6.4. This process is hypothesized to produce a cohort with lower-than-baseline alcohol and tobacco use. This will result in a lower prevalence of OPL in the population, with lesions that are less likely to progress to cancer. Among OPL+ patients, lesions should progress to cancer more slowly, increasing the likelihood that they will be detected at an early stage and cured. Since early-stage cancers are more likely to be cured than late-stage ones, fewer fatal late-stage cancers are expected to develop during the longer 155     screening interval in the ‘Assay Informed’ arm than under baseline conditions. This is expected to increase the overall survival rate within that arm, meaning that the TAC scenario is hypothesized to increase the NMB associated with the LOH assay. 6.2.4.2 Improved Screening The Improved Screening scenario considers the case of a cohort in which community dentists have access to, and use, a device that has a higher sensitivity for detecting oral cancer and precancer than conventional oral examination (COE). Under this scenario, an entity with a lesion whose pathogenicity has not been determined is examined using COE. If that lesion is detected using COE, they are asked to return in three weeks. If they return, and the lesion has not resolved, the lesion is re-evaluated using the high-sensitivity device. If the re-evaluation yields a positive result, the lesion is biopsied. If the lesion is an OPL or a cancer, the entity moves to the OPL Management or Incident Cancer components (respectively) for further medical management.  Parameter values for the additional screening using the device were estimated from cost and efficacy values associated with the VELscope device (LED Medical Diagnostics, Inc.; Vancouver, BC). The VELscope is a device that uses fluorescent visualization (FV) to examine lesions in the oral epithelium. Growths in the mouth have different autofluorescent properties to healthy epithelial cells when they are exposed to certain wavelengths of light. The VELscope is designed to emit wavelengths of light that can be used to identify potentially premalignant lesions at a high rate of sensitivity, with a lower rate of specificity (i.e., a higher rate of false positive referrals than using COE alone). Estimates for the sensitivity and specificity of the VELscope were estimated from values published in a cross-sectional 156     evaluation of 260 patients with suspicious oral lesions who were randomly assigned to receive either COE or examination using the VELscope[200]. There is no billing code for a cancer screening procedure, so the additional cost of using the device was assumed to be equivalent to an additional dental appointment[187]. This process is hypothesized to increase the probability that OPLs will be referred for management earlier in their development. This is expected to produce an increase in the population rate at which OPLs are detected, meaning more people will be managed using the assay, with fewer false negatives. This is expected to produce a lower overall cancer rate in the “Assay Informed” arm, with more curable cancers (HGLs and Stage I) prevented by early intervention. This is expected to be accompanied by a higher rate of false positive results, which will increase costs in the overall population, biasing incremental costs toward zero. This scenario may lead to an improvement in NMB from the reduced cancer rate, but given the higher false positive rate this potential improvement is expected to be small. 6.2.4.3 Improved Surgery The Improved Surgery scenario considers the case of a cohort in which otolargynologist head and neck surgeons use a device that uses the same FV technology as the VELscope to draw ‘margins’ of healthy tissue around oral lesions during surgery. Use of such a device has the potential to reduce the rate of local recurrence (LR – the return of cancer after a surgery that was thought to be curative). Such a difference would theoretically result in improved survival, but would likely come with some procedure cost that is greater than that of standard surgery. 157     Parameter values for the reduced rate of LR was derived from a previously-published evaluation of FV-guided surgery conducted in British Columbia[201]. In this trial, 246 patients with either early-stage OSCC or high-grade premalignant lesions (HGL) were randomized to receive either FV-guided surgery or conventional surgery under white light (WL) conditions. Multivariable Cox regression analysis was used to estimate the hazard ratio (HR) of LR between trial arms, along with its degree of statistical uncertainty (standard error – SE). The mean and SE were used to sample normally-distributed values of the HR, which was applied to values of estimated survival sampled from the WDMOC’s default parameters. Because the FV-guided surgery has not yet undergone a formal cost-effectiveness evaluation, no estimate of its additional cost was available in the literature. Preliminary unpublished data from a trial[62] that evaluated the length, clinical complexity (in terms of labour), and disposables cost from 400 patients treated with FV- vs. WL-guided surgery suggests that the two procedures are equal in resource utilization and cost. Additionally, as the FV device is a camera that does not have additional consumables and can be used for multiple patients with negligible wear and tear, the per-patient cost of the device is unlikely to generate a great deal of additional cost. A conservative approach (i.e., biased away from favourable cost-effectiveness of the FV camera) was taken and the cost of FV-guided surgery was assumed to represent a 25% increase above the cost of a surgical resection[185]. This scenario is hypothesized to reduce rates of recurrence and mortality among early-stage cancers, which are more commonly found in patients whose OPL management is “Assay Naïve”. Improving survival in both “Assay Informed” and “Assay Naïve” arms, especially among early-stage cancers, is expected to bias incremental survival toward zero, reducing NMB.  158     6.2.4.4 New Drug The New Drug scenario considers the case of a cohort for which a highly effective new drug treatment is available for patients with advanced stage (stage III/IV) oral cancer. Patients who would normally be treated with standard chemotherapy will instead be treated with this new drug. The new drug carries a high level of cost, but provides a high level of clinical benefit above standard systemic therapies. Because relatively few people will develop late stage cancers and benefit from the new drug, this scenario is hypothesized to have only a marginal effect on the net benefit of the assay. After review of the literature, the authors know of no recent trials of drugs in oral cancer that are expected to have dramatic survival benefit for patients. This scenario rather represents a hypothetical “what if” analysis of a drug that may one day become available. Parameter values for this scenario were derived from the recently-published KEYNOTE-407 Clinical Trial[202]. This trial considered the incremental effectiveness of pembrolizumab (Merck, trade name Keytruda) against placebo in patients also treated with standard systemic therapies for metastatic non-squamous cell lung cancer. The mean and SE values of the HR were randomly sampled for each entity and applied to values of estimated survival sampled from the WDMOC’s default parameters. The additional cost of the hypothetical ‘blockbuster’ drug was estimated from a cost-effectiveness analysis of pembrolizumab, using the published estimates of incremental cost[203], converted to 2017 CDN from 2017 USD. Given that late-stage cancers are more likely to occur in the “Assay Informed” arm, a reduction in the mortality associated with these cancers is hypothesized to result in an overall reduction in cancer mortality. This will increase the incremental survival between the two arms. However, since these 159     cancers represent a small percentage of overall cancers, and the cost of the drug is high, the potential increase in NMB resulting from the additional survival may be outweighed by the additional cost. 6.2.4.5 HPV Vaccination The HPV Vaccination scenario considers the case of a cohort that has previously been vaccinated with a quadrivalent vaccine. This vaccine, in current use in British Columbia[204], protects against oncogenic strains of HPV, including strains 16 and 18 that are associated with oral cancer[205]. A review of the literature suggests that the level of evidence supporting the link between OSCC and HPV is unclear[205-207]. The literature suggests three potential relationships between HPV and oral cancer: 1. HPV may be associated with the development of OPLs. HPV has been found in greater prevalence among OPLs than non-premalignant controls in clinical studies. A causal mechanism for this relationship has not yet been determined. 2. HPV may be associated with the progression from oral premalignancy to cancer. Some clinical evidence suggests an association between HPV and p16 protein development – p16 is also associated with oral cancer[206]. HPV prevalence is also higher among newly diagnosed oral cancer patients than among disease-free controls[43, 205, 208]. There is no established causal mechanism for this association either. 3. Cancers associated with HPV may have different post-treatment survival characteristics than those arising from other causal factors (e.g., smoking, alcohol). There is no consensus among recent studies on the relationship between HPV infection and patient survival, with some studies finding higher rates of recurrence and mortality among HPV-associated cancers[209-211] and others finding no relationship[212, 213]. The HPV Vaccination scenario incorporated all three of these potential relationships, to elicit the strongest potential effect on the cost-effectiveness of the LOH assay. The WDMOC was adjusted in the following ways to simulate the effect of an HPV vaccination program: 160     Step 1 – OPL Development: overall OPL prevalence data was derived from studies that did not measure HPV status, meaning that it is not possible to know which OPLs are potentially caused by HPV. To account for the potential causal relationship, the probability that an entity’s OPL was caused by HPV was weighted by an estimate of the population prevalence of HPV, and the odds that a newly-detected OPL is HPV 16/18 positive (HPV+). The scenario assumes that HPV+ OPLs would not have formed in people who have been vaccinated against HPV – an entity’s probability of being vaccinated was estimated from the HPV vaccine’s expected uptake rate. Step 2 – OPL Progression: an entity with an OPL that was deemed to be unrelated to HPV (HPV-) through the process in Step 1 was assumed to have a different HR for progression to invasive cancer from HPV-associated OPLs. Mean and SE values of this HR were randomly sampled for each entity and applied to values of estimated progression time from the WDMOC’s default parameters. Step 3 – Post-treatment survival: an entity whose HPV- OPL has progressed to invasive cancer is assumed to have a different HR for local recurrence from HPV+ OPLs. Mean and SE values of this HR were randomly sampled for each entity and applied to values of estimated local recurrence rates (‘First Event’ and ‘First Event – Death’) from the WDMOC’s default parameters. Resource utilization for the vaccination program was based on an assumption that male and female members of the cohort received two doses of the vaccine at age 14[204, 214]. This scenario is hypothesized to produce a population with a smaller number of OPLs that are less likely to progress to cancer, and for whom treatment is more effective. Given that late-stage cancers are more common in the Assay Informed arm, improving treatment for these cancers will produce higher incremental survival between the arms, increasing NMB. Parameter estimates for the five scenarios described above are presented in Table 6.1.   161     Table 6.1 – Parameter estimates used in scenario analysis Scenario Parameter Mean SE Distribution Source Tobacco/Alcohol Cessation      Tobacco program efficacy rate 0.097 0.026 Beta [198]  Tobacco program cost $35 $5 Gamma [198]  Alcohol program efficacy rate 0.123 0.031 Beta [199]  Alcohol program cost $35a –  [185, 199] Improved Screening      Screening test sensitivity 100% –  [200]  Screening test specificity 74% –  [200]  Screening cost $17.40b –  [187] Improved Surgery      HR: local recurrence 0.16 0.2 Lognormal [201]  Additional cost of surgery $100c –  [185] Blockbuster Drug      HR: local recurrence 0.56 0.11 Lognormal [202]  Additional drug cost $78,609 $8,157 Gamma [203] HPV Vaccination      HPV population prevalence 10%  Beta [215-217]  HPV vaccine coverage rate 0.67 0.06 Beta [218]  OR: OPL is HPV+ 4.0 1.2 Lognormal [208]  HR: progression to invasive cancer, HPV- 1.0 – Lognormal d  HR: local recurrence, HPV- 0.5 0.2 Lognormal [209, 210]  Cost to be vaccinatede $360 –  [214] SE – Standard Error; HR – hazard ratio; HPV – human papillomavirus; OR – odds ratio; OPL – oral premalignant lesion a – the Purshouse paper used a per-minute estimate for the cost of a brief intervention. For the purpose of this analysis, a fee-for-service value (MSP 16100 – General Practice Visit) was used instead. b – in the absence of an estimate for the cost of a VELscope examination, this value is derived from the cost of a recall oral examination (MSP 01202 – Recall Oral Examination). c – in the absence of an estimate for the cost of FV-guided surgery, this value represents an assumed 25% increase above the cost of a partial glossectomy (MSP 02478 – Glossectomy, partial for carcinoma), on the assumption that FV-guided surgery will be somewhat more complicated than standard surgery d – No studies in the literature estimated the incremental or relative effect of HPV on OPL progression. An RR of 1.0 was assumed for the baseline, and adjusted in sensitivity analysis. e – vaccination costs were discounted to reflect the present value at the beginning of the model, under the assumption that each vaccinated entity received two doses at age 15. For example, if the entity began the model at age 45, the cost of the vaccination program was discounted over 30 years.   162     6.2.5 Cost-Effectiveness Analysis Each scenario was run on a simulated cohort of 1.2 million entities, representing a population that, under baseline assumptions, is expected to have at least 10,000 OPLs. By simulating a large population of entities, the model was able to simulate a reasonable range of costs and quality-adjusted survival values for that population. Incremental cost-effectiveness analysis was conducted by comparing costs and outcomes for populations simulated within the model’s two arms. Mean estimates for costs and QALYs, reflecting parameter uncertainty, were estimated through a bootstrapping process, where cost and survival values for OPL+ entities were sampled (with replacement) 1,000 times. Costs and outcomes in both the conventional Markov model and the adapted WDMOC were discounted for future time preference at an annual rate of 1.5%, expressed as a continuous daily discount rate[5]. Costs for both were expressed in 2017 Canadian Dollars, adjusted for inflation using the Consumer Price Index for health care. A ten year time horizon (3650 days) was chosen for the analysis, representing a point at which all entities are either dead (from cancer or another cause), in 5-year remission from a detected cancer, or unlikely for their OPL to progress to invasive cancer. Mean values for incremental cost and QALYs (‘Assay Informed’ minus ‘Assay Naive’) were calculated for each bootstrapped sample. These values were used to calculate the Net Monetary Benefit (NMB) of using the assay to guide OPL management. NMB is a summary statistic expressing the expected value of an intervention in monetary terms. It incorporates society’s willingness to pay for a QALY (commonly expressed as λ), allowing for the conversion of QALYs to dollars – this converted amount represents the 163     value (‘benefit’) that society gains in the form of improved survival. By subtracting the cost required to produce that benefit, NMB expresses the expected return on investment for a new health technology.                  NMB values larger than zero reflect policy changes where the benefits outweigh the costs (i.e., the policy change is cost-effective). Percent cost-effectiveness (%CE) is a summary measure that incorporates decision uncertainty – the extent to which statistical variation in parameter values create variation in the likely cost-effectiveness of policy change. %CE is calculated by calculating the proportion of bootstrap-sampled results where both NMB and ΔE are greater than zero. This metric represents the probability that a policy change will result in improved health at a cost that is within society’s willingness to pay for it. The further %CE values move away from 50%, the more information a policy maker has about whether or not a policy change will be cost-effective. Estimated NMB and %CE of using the LOH assay to guide OPL management were calculated at baseline (i.e., under policy status quo), for each scenario individually, and in combination. A policy change that increases NMB and %CE above baseline levels means that use of the LOH assay becomes more cost-effective than it would be within the current policy environment. Larger NMB values represent greater value for money. A λ value of $100,000 per QALY was chosen for this exercise. The scenarios considered in this exercise will impact people outside the specific context of managing OPLs, which is where the LOH assay takes effect. Both “upstream” and “downstream” policies will affect 164     the broader population, most of whom will not have an OPL that is detected and managed or any subsequent disease experience. For example, the screening scenario will produce a larger number of false positive referrals, resulting in higher costs with no change in health for people without OPLs. Two perspectives are therefore considered for this analysis: the full population and the subset of the population with a detected OPL. 6.3 Results Results (Incremental cost, QALYs, NMB and %CE) for the cost-effectiveness of the LOH assay, and each scenario, are presented in Table 6.2. Under baseline conditions (i.e., no alternative scenarios), use of the LOH assay had a population NMB (NMBpop) of $17.00 and a population %CE (%CEpop) of 50.32. Among people with an OPL, the NMB (NMBOPL) was $1340.18, and the %CE (%CEOPL) was 51.78. These results taken together suggest that, compared to the status quo, introduction of the LOH assay is expected to produce cost-effective outcomes both at the population level, and among patients with a detected OPL, with a great deal of uncertainty about whether or not the assay is truly cost-effective.  Table 6.2 – Baseline cost-effectiveness of using the LOH assay, population and OPL+ perspectives Perspective Incremental Cost Incremental QALY Net Monetary Benefit Percent Cost-Effective Population  -$3.27 0.0001 $17.00 50.3 OPL+ patients only -$876.47 0.005 $1,340.18 51.8 QALY – quality-adjusted life years. Societal Willingness to Pay for a QALY ( ) = $100,000  165     NMB and %CE results for each scenario are presented in Table 6.3. These values reflect how cost-effective use of the LOH assay would be in combination with the alternative policies represented within each scenario. With the introduction of new policy changes, the use of the assay became more cost-effective, with the greatest change in NMBOPL associated with the Tobacco/Alcohol Cessation scenario (NMBOPL = $9,543; %CEOPL = 75.6%). The smallest change from baseline was associated with improved surgery (NMBOPL = $4,202; %CEOPL = 61.5%). In a scenario where all policy alternatives were implemented simultaneously, the NMBOPL of the LOH assay was $7,736 (%CEOPL = 70.9%), suggesting that use of the assay would still produce good value for money in the face of a changed policy environment – better value for money, in fact, than under the status quo. Table 6.3 – Cost-effectiveness of using the LOH assay in conjunction with other policy changes Scenario NMBpop %CEpop  NMBOPL %CEOPL Baseline $17 50.3  $1,340 51.8 Tobacco/Alcohol Cessation -$51 50.0  $9,543 75.6 Improved Screening -$36 50.1  $4,390 62.1 Improved Surgery $13 50.2  $4,202 61.5 Blockbuster Drug -$15 49.5  $7,914 71.2 HPV Vaccination -$13 49.3  $9,302 76.4 Implement All Policies  -$53 49.4  $8,778 70.9 NMB – Net Monetary Benefit; %CE – percent of ICERs that are cost-effective; pop – values for full simulated population; OPL – values for patients with a detected OPL. Societal Willingness to Pay for a QALY ( ) = $100,000  Population cost-effectiveness (NMBpop and %CEpop) were largely unchanged with the introduction of the new policies, compared to baseline. This is likely a reflection of the fact that only a very small minority of the simulated population developed an OPL, and thus most entities did not receive the LOH assay at all. 166     Use of the LOH assay produced the most additional value for money (NMBOPL = $14,208; %CEOPL = 85.6%) where the Screening, Blockbuster Drug, and HPV Vaccination policies were adopted. The assay produced the least additional value for money (NMBOPL = $236; %CEOPL = 48.6%) in a scenario where the Screening, Surgery, and HPV Vaccination policies were adopted. In fact, under this combination of programs the LOH assay became less cost-effective than it was predicted to be under baseline conditions. Table 6.4 describes the five “best” and “worst” scenario combinations (i.e., those producing the most- and least-favourable NMBOPL values). The range of NMBOPL values found within the various scenarios (and scenario combinations) is presented in Figure 6.1. Table 6.4 – Policy changes that produce the ‘best’ and ‘worst’ cost-effectiveness values for using the LOH assay Scenario NMBOPL %CEOPL ‘Best’ 5 scenarios   Screening + Drug + HPV $14,208 85.6 TAC + Screening + HPV $11,738 81.8 TAC + Drug + HPV $11,675 79.6 Drug + HPV $11,493 81.0 TAC $9,543 75.6    ‘Worst’ 5 scenarios   Surgery $4,202 61.5 TAC + Screening + Surgery $3,792 59.3 Screening + Drug $2,261 52.6 Surgery + Drug + HPV $800 49.2 Screening + Surgery + HPV $236 48.6 TAC – Tobacco/Alcohol Cessation; HPV – HPV vaccination; Drug – Blockbuster Drug; NMBOPL – net monetary benefit for OPL+ patients; %CEOPL – percent of ICERs that are cost-effective for OPL patients. Societal Willingness to Pay for a QALY ( ) = $100,000    167     Figure 6.1 – Net Monetary Benefit of using LOH assay in OPL+ patients among different scenario combinations  TAC – Tobacco/Alcohol Cessation; HPV – HPV vaccination; Drug – New Drug; the red bar indicates the baseline NMB value. Societal Willingness to Pay for a QALY ( ) = $100,000 Due to the lack of conclusive data in the available literature, HPV was not associated with a change in the rate of OPL progression to invasive cancer under baseline conditions (i.e., HR = 1.0). The HR for OPL progression was arbitrarily adjusted to 1.25 and 0.80 (±25%) to investigate the impact it had on NMB and %CE. NMBOPL and %CEOPL were directly correlated with this change in survival, with and upper value of $11,869 (79.7% cost-effective) and a lower value of $4,258 (60.0% cost-effective). Use of the LOH assay was cost-saving at both a population level and among OPL+ patients in nearly all scenario combinations. At the population level, a combination of the Screening and Blockbuster Drug 168     policies produced the greatest cost savings (∆Cpop = -$4.58; NMBpop = -$77.49; %CEpop = 49.7), and most cost-saving among OPL+ patients (∆COPL = -$1,829; NMBOPL = $8,051; %CEOPL = 69.50) when the Tobacco/Alcohol Cessation and HPV Vaccination policies were adopted simultaneously. A table containing the full analysis results is available in Appendix 6.1. 6.4 Discussion The WDMOC was able to evaluate the impact that multiple potential changes in the policy environment may have on the cost-effectiveness of a program of interest, both alone and in combination. These policies can occur ‘upstream’ or ‘downstream’ from the program of interest, or have effects that are felt at multiple points along the disease pathway. This exercise used the example of an LOH assay to guide the management of OPLs. The model was also able to determine the combination of policy changes that would make the LOH most cost-effective, and most cost-saving. The pattern of findings suggests that the cost-effectiveness of the assay is most sensitive to policies that affect the rate of downstream disease events (progression to cancer under the Tobacco/Alcohol Cessation policy, disease recurrence under the HPV Vaccination policy), and much less sensitive to policies whose effects are immediately proximate to the assay (screening, early-stage surgery). The findings also suggest that reducing late-stage mortality disproportionately favours using the assay, resulting in a greater overall survival benefit for assay-informed care. The analysis also suggests that changes in NMB were more strongly influenced by changes in quality-adjusted survival than they were by changes in cost. This is partially a result of the fact that, under baseline conditions, using the assay produced a near-zero survival benefit. Any technology that produced a change in incremental survival 169     between model arms, even a small one, would yield a noticeable change in NMB as a result. This change in NMB would be much more subtle if a smaller value of λ had been chosen. The value of using a Whole Disease Model illustrated that the change in cost-effectiveness depends on the policy making perspective being taken. While ‘piecewise’ models tend to consider the context of whether or not a technology change is cost-effective for a single patient population (the OPL+ perspective), this exercise also considered the change in cost-effectiveness when considering the full population, including people who may not develop the disease. Such a perspective may bear a closer resemblance to how policy makers actually make resource allocation decisions within fixed budgets (i.e., resource allocation decisions may be made by a health minister or a regional health authority without heed to trade-offs in each specific disease area affected by the decision). Use of the model in this way allows for the comparison of several potential policies of interest, which could provide important priority-setting information to these policy makers about how their most important decisions might work in tandem. The ability of the WDMOC to evaluate multiple outcomes of interest allows policy makers to consider whatever priorities they think are most valuable (e.g., most cost-effective vs. least costly vs. most effective). The analysis also found that the model can also be used to re-assess existing resource allocation decisions when new decisions are being made, in order to account for their impact on existing services.  This finding underlines the value of being able to evaluate the way that multiple policies interact across the disease decision environment. A policy maker could consider the impact not only of foreseeable policy changes (i.e., by subjecting newly proposed technologies to economic evaluation) but of 170     hypothetical changes that may occur in the future (e.g., drugs undergoing clinical trials, secular demographic changes), which could allow them to proactively plan the delivery and management of a variety of health services – timing which programs may need to be phased out or given additional resources. This consideration of resource allocation could also be used to guide research, giving academics and granting agencies a sense of what kinds of changes would produce meaningful shifts in costs and outcomes among existing programs. It is important to recognize that, at a population level, the vast majority (>99%) of simulated people did not develop any disease at all. It is also the case that some people may develop cancer without going through OPL management at all (i.e., they present symptomatically or their cancer develops between regular dental screenings). Since the costs and outcomes of population-wide programs like HPV vaccination and screening are experienced by these people, it is relevant to include them in the analysis. From Table 1 one can see that population-level changes in cost-effectiveness were much smaller than within the OPL+ population. 6.4.1 Limitations Models are limited by a number of assumptions, and many of the WDMOC’s assumptions are described in Chapters 4 and 5. Beyond those, this exercise made additional assumptions in both how the model simulated each policy scenario and the information used to power those simulations. Each scenario was a simplified version of an example of policies that could be enacted, and the values were drawn from recent available evidence from decisions with varying levels of similarity to what actual policy questions might look like. The New Drug scenario, for example, was drawn from an example in an entirely 171     different disease area, and the authors know of no drug that is currently undergoing phase III investigation for use in oral cavity cancers.The effect of this limitation is especially true of the HPV Vaccination scenario, where the paucity of data in the literature required a number of assumptions to be made about both the nature and magnitude of the policy change and its likely impact. The recent literature provides no consensus on the role that HPV plays in oral cavity cancer, only that there is strong association at multiple points along the disease pathway. Each of the potential pathways was populated with results from recent studies, but these studies were conducted in different populations using different techniques and finding different levels of association between exposure and outcome. The Tobacco/Alcohol Cessation scenario did not consider the impact of lower smoking rates on outcomes outside the context of oral cancer. Reducing these rates would likely reduce the overall rate of all-cause mortality in the population, and would result in fewer deaths from non-cancer causes in the simulated population. The combined effect of this model assumption introduces a slight bias away from the null hypothesis (i.e., use of the LOH assay is not cost-effective), as cancer-specific mortality becomes more influential by comparison. Given the short time horizon of the model (ten years), the size of this bias is likely very small. It is worth noting, however, that beyond the effects such programs would have on cancer-specific mortality, population programs to modify cancer risk factors are likely to reduce the risk of other illnesses simultaneously, providing a larger return on the initial investment to run them. This effect is also true of the HPV Vaccination scenario, which would likely produce reduced rates of all HPV-related cancers, thus reducing all-cause mortality. 172     These scenarios should be thought of as “what if?” examples rather than being an evaluation of specific policy options. As researchers develop new information about policies, and as policy makers are identifying policy priorities of interest, the exercise described above shows how that new information can be incorporated into the WDMOC as it becomes available. 6.4.2 Implications of This Chapter This exercise showed the value of a Whole Disease Modeling approach. The WDMOC was used to evaluate ‘upstream’ and ‘downstream’ policy changes, and found that the policy impact of those changes was highly variable both individually and in conjunction. This flexibility and capacity for multiple simultaneous evaluations are the main arguments in favour of WDM over a conventional modeling approach. As can be seen from the above results, these attributes of the WDM approach provide additional useful information above what is typical for decision modeling exercises. It is worth noting that while this chapter sought to address the question “what is the potential effect of new policies on the cost-effectiveness of an existing policy”, the analysis conducted in this chapter still produced values that could be used to answer the more conventional question of “what is the potential cost-effectiveness of introducing a new policy”. It is possible to estimate the cost-effectiveness of each of the policy scenarios by comparing outputs from the “Assay Naïve” arm of the scenario of interest to the “Assay Naïve” arm of the baseline case. These comparisons are comparing the effects of the policy scenarios, with a conventional approach to OPL management. This approach to answering the second question was eschewed in this chapter in favour of exploring the first one, but the WDMOC is equally equipped to answer both. 173     Budgetary impact assessment is another strength of the WDM approach, as it allows policy-makers to consider not only whether an intervention provides good value for money, but how much money must be spent to achieve that value. A population-wide screening program might appear cost-effective, but the additional cost to institute such a policy may be prohibitive without additional funding or disinvestment from currently-running programs.  While a budgetary impact assessment was not performed in this chapter, the WDMOC could easily be used for this purpose as well, by considering and comparing the sum of health care costs for two or more equivalent simulated populations. The Canadian Agency for Drugs and Technologies in Health (CADTH) suggest a shift in policy making toward Health Technology Management (HTM) over the conventional Health Technology Assessment (HTA) framework[5, 15]. The key difference between HTM and HTA is that HTM explicitly considers the evaluation and re-evaluation of existing/approved technologies, whereas HTA considers the marginal impact of adopting a novel technology[15]. The exercise described above is aligned with the HTM approach, providing information about the impact that subsequent policy changes may have on the decision of interest. This is an application of the Whole Disease Modeling approach, and is of particular value in disease areas where multiple policy changes are being considered in the coming years. This type of analysis would not be possible with a typical ‘piecewise’ model. 6.5 Conclusion This chapter described the use of the WDMOC to evaluate ‘upstream’ and ‘downstream’ policy changes for potential future technologies, and found that policy combinations and decision context were highly influential on the cost-effectiveness of use of a LOH assay to guide OPL management. The exercise 174     described above illustrates the power and value of a Whole Disease Modeling approach, which can address the impact of potential changes in policy not only for newly adopted technologies, but on the cost-effectiveness of existing technologies (or technologies under consideration for adoption). Now that the WDMOC has been designed and implemented, the next and final chapter will discuss the dissertation as a whole, including future applications of the WDMOC and the implications of the model approach chosen.     175     7 Chapter 7 – Discussion The preceding chapters have described the design, construction, and application of a Whole Disease Model of Oral Cancer (WDMOC) that is suitable for use by researchers and policy-makers to generate cost-effectiveness evidence on potential changes in the use of new and existing health technologies. This chapter will discuss the key findings from the key research objectives of this dissertation in the context of the wider literature. It will also identify and summarize some important strengths and limitations of the methods chosen in this dissertation. Finally, this chapter will present the implications for practice, and highlight options for future research that were beyond the scope of the research objectives that informed the previous chapters. 7.1 Research Objectives and Key Findings of This Dissertation This dissertation has addressed five principal research objectives. 7.1.1 Determine The Current State of the Literature Concerning the Health Economics of Oral Cancer and to Identify Specific Knowledge Gaps in Technologies That Are Suitable for Economic Evaluation.  This objective was accomplished in Chapter 2, in which a structured literature review was conducted to characterize the capacity of the literature to evaluate health technologies in oral cancer and to identify some potential technologies that may require evaluation in the near future. The review found that there is insufficient capacity in the existing literature to evaluate the variety of technologies that are expected to emerge in the coming years at various points along the disease trajectory (improvements in 176     screening, OPL management, changes in treatment). There is also limited capacity for existing models to reflect the effect that individual risk factors have on disease development and survival.  If health policy makers wish to make evidence-based resource allocation decisions regarding oral cancer, their ability to do so is limited by the breadth and depth of currently-existing models, which are few in number and cannot reflect the impact of ‘upstream’ and ‘downstream’ policy changes on cost-effectiveness. The WDM framework is appropriate to address exactly these circumstances. 7.1.2 Illustrate The Limitations of a ‘Piecewise’ Modeling Approach When Evaluating Health Technologies.  Chapter 3 met this objective by using a conventional Markov model-based approach to evaluate the cost-effectiveness of using a molecular assay to stratify the management of oral premalignant lesions (OPLs) by risk category. By reducing the frequency of screening among patients with low- and intermediate-risk OPLs, use of the assay was less costly and more effective than (i.e., it dominated) conventional surveillance. However, the model made simplifying assumptions made about stage progression and interval screening, and did not reflect the impact that community dentists would have on the likelihood that interval cancers would be detected.  Given that factors like access to community dentists vary within the population, it is important to understand how this variation affects the cost-effectiveness of proposed interventions. This is especially true when those factors can be influenced by changes in policy that may be ‘upstream’ or ‘downstream’ from the intervention of interest. There are several potential technologies in oral cancer prevention, 177     control, and treatment that will require economic evaluation, and the Markov model described in Chapter 3 underwent peer review and publication, suggesting that it is broadly representative of the current literature. The limitations of existing ‘piecewise’ modeling approaches revealed in this chapter suggest that there is value in a model that address these factors across the entire disease trajectory. 7.1.3 Design, Implement, And Validate a Model That Can Address the Shortcomings of the Conventional Approach.  This took the form of a Whole Disease Model of Oral Cancer (WDMOC), as described in Chapter 4. The structure of the model was designed with input from a team of clinical experts and the available literature, and considered disease-related events from the development of an OPL through community screening and management to invasive disease treatment and terminal illness. To estimate model parameters, two retrospective cohorts were analyzed to estimate time-to-event (TTE) parameters (progression to invasive cancer, survival after detection and treatment) in a way that reflects individual factors (genomic risk score, age, sex, treatment received). Retrospective analysis based on a data linkage to provincially-insured services was used to estimate treatment costs. Other model parameters were derived from the literature.  The WDMOC was designed to be a platform to generate evidence on how policy changes at multiple points along the clinical trajectory of oral cancer, either individually or in combination. The discrete event simulation (DES) approach used in the WDMOC allows it to reflect the influence that the distribution of demographic and clinical characteristics in a population may have on costs and quality-adjusted survival. This represents an improvement in the capacity of the existing modeling literature for 178     oral cancer, designed to address the specific shortcomings of that literature and based in part on parameter values that were developed from primary data sources within this dissertation. 7.1.4 Compare the Conventional Markov Modeling Approach with the WDMOC Approach.  Use of the more comprehensive WDMOC found that while the assay was still cost-saving, there was no improvement in survival. While using the assay produced a reduction in the overall cancer rate, the longer screening interval for low-risk OPLs meant that the few cancers that did develop in that group were disproportionately late stage and deadly. Consequently, overall cancer-related mortality was equivalent to the arm representing standard practice. The cost-effectiveness of the assay was moderated by seeing a community dentist, and having a shorter screening interval. Re-addressing of the previous policy scenario showed the specific impact that the WDM approach can have beyond conventional modeling techniques. The model produced a different result from the conventional Markov, and made it possible to understand the specific reasons for that difference. These reasons can potentially be addressed through subsequent policy changes, increasing the likelihood that a technology will be cost-effective. The WDMOC is a useful health economic tool that can provide a wide variety of nuanced outputs at a stochastic level, making it possible to understand cost-effectiveness impacts of policy changes with a high degree of detail. 7.1.5 Use the WDMOC to Evaluate the Impact That ‘Upstream’ Parameter Changes Have on the Cost-Effectiveness and Budgetary Impact of ‘Downstream’ Technology Adoption.  179     The cost-effectiveness of the molecular assay was analyzed under a variety of different policy scenarios, each representing a likely technology that may be adopted in the future. These technologies would be implemented both ‘upstream’ and ‘downstream’ from the assay, spanning multiple components of the disease pathway. The analysis showed that the cost-effectiveness of the assay was most sensitive to technologies that affected the progression rate of OPLs. The analysis also showed the WDMOC’s capacity to evaluate multiple technologies either simultaneously or in conjunction, which is a useful feature for Health Technology Management (HTM) approaches in which existing technologies may be re-evaluated in the light of changes to the policy environment. 7.2 Contributions to the Literature  Tappenden and colleagues developed the first WDM for colorectal cancers,[32] and was used to estimate the cost-effectiveness of multiple potential technologies in that disease. This model was constructed to serve a specific function, which was the development of a set of National Health Service (NHS) clinical guidelines that would be informed by cost-effectiveness evidence. Tappenden identified the lack of evidence concerning treatment flows and resource utilization for patients with diagnosed cancer as a limitation of this initial exercise, and noted that such estimates could come from a retrospective data analysis. A WDM for chronic obstructive pulmonary disease (COPD) was created by Sadatsafavi and colleagues[34]. At the time of writing, this model has not yet been used to conduct economic evaluation in the published literature. The authors also note that their model does not reflect diagnostic or treatment processes, and that it does not reflect parameter uncertainty in its current form. 180     The WDMOC is the third application of the WDM framework. It is based on the kind of retrospective data analysis suggested by Tappenden, and includes the diagnostic and treatment processes that are suggested by Sadatsafavi (while also incorporating parameter uncertainty). In addition to these developments within the context of the WDM framework, this dissertation  makes a number of important contributions to the field of health economic evaluation in the context of Health Technology Assessment and Management (HTA and HTM, respectively). This dissertation also contributes new knowledge to economic evaluations of the management of oral premalignant lesions and oral cancer. 7.2.1 Economic Evaluation Methods  The principal objective of this dissertation has been to create a comprehensive model that can be used to conduct decision analysis at multiple points throughout the oral cancer disease trajectory. This capability was explored to its full extent in Chapter 6, where the alternative policies being analyzed were interventions taking place before the onset of disease, at the detection of premalignancy, during primary treatment, and for late stage cancers. The model could be used to evaluate policy interventions post-recurrence, during follow-up, and at the end of life as well. The signature feature of a WDM is the capacity to evaluate ‘upstream’ and ‘downstream’ changes from a policy of interest. This feature is what separates WDMs from conventional ‘piecewise’ models, which only reflect processes that are directly relevant to the policy question of interest. The original implementation of the WDM framework showcased the ability of a WDM to evaluate the cost-effectiveness of various combinations of potential policy changes individually and in concert[32]. In Chapter 6 of this dissertation, this WDM feature was extended to investigate a related but novel 181     question of HTM – how do subsequent policy changes affect the projected cost-effectiveness of a technology of interest? This dissertation demonstrates that the WDM is perfectly suited to address these kinds of policy questions as well. 7.2.2 Knowledge in Economic Evaluations of Oral Cancer This dissertation contributes important policy-relevant knowledge to the scientific literature about the health economic impact of premalignant management of OPLs. Namely, that managing premalignancy via a genomic risk score is expected to be cost-saving, but not life-prolonging compared to standard practice. This relationship is moderated by access to regular screening, the length of OPL follow-up, and the rate of stage progression of undetected disease. This dissertation also generated new information about oral cancer survival, as well as the costs of treatment. The retrospective cohort analysis conducted for the model parameterization in Chapter 4 was more comprehensive (in terms of the variety of resource inputs) and larger (in terms of the sample size) than other exercises available in the health economic literature, as described in Chapter 2. Being able to draw costs and survival from the same population was a particular strength of the approach. The exercise conducted in Chapter 6 also suggests that the cost-effectiveness of OPL risk prediction is sensitive to the introduction of new programs that reduce the risk of progression (i.e., tobacco and alcohol cessation, HPV vaccination) and the availability of effective downstream treatment for late-stage disease. Early-stage interventions such as screening and surgery did not have the same impact on cost-182     effectiveness. Previously-published decision models, including the example provided in Chapter 3, would not have been able to estimate these impacts either alone or in combination. Finally, this dissertation produced primary research into the natural history of oral cancer from the time of OPL detection. A secondary analysis conducted on data from the Zhang trial cohort[61] used multivariate regression techniques to estimate the impact that risk factors (age, alcohol/tobacco use, sex) had on time to progression. This regression analysis controlled for underlying LOH, which is an extension beyond the originally-published trial results[61]. Furthermore, this dissertation was able to estimate the impact that sex, age, stage, and treatment type had on not only overall survival from oral cancer, but also time to first and second recurrence and survival beyond those points. These estimates, which represent an improvement over what is available in the clinical literature, can be used to inform future economic evaluations. 7.3 Strengths of This Dissertation Principally, the WDMOC’s design and implementation represent meaningful improvements above conventional modeling approaches like those in the currently-available literature on oral cancer. The chief strengths fall into three general categories: strengths of a WDM over conventional modeling, strengths of an individual sampling DES over Markov modeling, and strengths due to the model’s design. 7.3.1 Whole Disease Models Versus Conventional Piecewise Models The WDM framework requires the inclusion of processes and events that are relevant to the disease of interest, even those occurring before disease is detected. As a result, the WDMOC was able to describe 183     the relationship between different elements of the health care system, e.g., the relationship between OPL surveillance and community dentists when it comes to detecting progression to cancer. Chapter 6 also found that ‘downstream’ elements of the health care system (particularly treatments for late-stage disease) play an important role in the cost-effectiveness of early intervention. In so doing, economic evaluations using the WDMOC go beyond typical piecewise modeling approaches in that they are not bound to a specific decision context.  The impact of these improvements, taken together, was evident in the differences between the cost-effectiveness estimates produced by the Markov model described in Chapter 3, and by the WDMOC in Chapter 5. The cost-effectiveness of risk-guided OPL management was moderated by the availability of screening dentists, symptomatic detection of disease, cancer stage progression, and late-stage cancer survival. These factors were not reflected in the original ‘piecewise’ Markov model presented in Chapter 3, and many of the parameters needed to inform them were derived from the primary data analysis described in Chapter 4. The supplementary analysis conducted in Appendix I describes which aspects of this difference are due to the WDMOC’s increased parametric complexity, and which are due to implementation factors like TTE processing instead of cycle-based processing. 7.3.2  Individual Sampling Versus Markov Models The individual sampling approach chosen for the WDMOC meant it was also able to reflect how individual factors (age, sex, smoking, treatment type, etc.) influenced important TTE values like cancer stage progression, and the relationship between stage at detection and survival. The TTE processing 184     used in the WDMOC also exerted an influence – events could transpire in a shorter simulated time than a fixed cycle length approach allows.  This dissertation also included a conventional model (Chapter 3), which allowed for a side-by-side comparison of the new and old approaches. This comparison was able to distinguish between differences that are due to the model’s parameter complexity (i.e., the inclusion of more natural history and treatment variables), the structural features of a WDM itself (i.e., including other parts of the disease pathway like community dental screening), and the differences between TTE and cycle-based time processing.  As discussed in Chapter 5, many of these features can be understood as improvements above a Markov approach rather than a facet of the Whole Disease nature of the WDMOC[190-197]. Nevertheless, the WDMOC is a much more complex model than the Markov model described in Chapter 3. This is due in part to the fact that it simulates more components of the health care system (e.g., community screening, end-of-life care, second recurrence), at a greater depth (e.g., multiple stage-specific treatment regimens, incorporation of entity characteristics into TTE processing) than is typical for any of the Markov models described in Chapter 2. It is also due to improved parameter estimates from richer data sources where such were available. Many of these sources took the form of secondary clinical data for TTE estimates including survival. Given the conclusions from the Markov and DES comparisons described in Chapter 5 (i.e., DES is better for complex models and those that use secondary data analysis for TTE), the DES method in the WDMOC was chosen as most appropriate. The DES method is also recommended by Tappenden[31]. 185     7.3.3 Design and Structure of the WDMOC The WDMOC was designed as a number of interacting components and processes, in order to reflect the main factors that influence oral cancer development, progression, detection, treatment, and management. Accordingly, the model’s structure was designed to be flexible and easy to modify, with each component and process programmed independently. The natural history processes were programmed independently of health care system processes, in a way that allows the seamless integration of new system processes into the pre-existing natural history. The WDMOC was also designed with a comprehensive array of parameters – over 170 in the base case alone. Data for parameter estimates was read into the model from an easy-to-edit format (Microsoft Excel table) with no need to make any corresponding changes in the code itself. These parameters were drawn from a variety of data types – probabilities, costs, counts, transition probabilities, regression coefficients – derived from both primary and secondary sources. The WDMOC can be adapted to reflect additional data types as the need may arise.  These design features also allowed the WDMOC to incorporate new and emerging data, including data that is not necessarily well-developed and is highly speculative. This capacity for data incorporation drove the scenario analysis conducted in Chapter 6, where data on the impact of the proposed policies (particularly HPV vaccination) was not well characterized and had potential impacts on multiple points within the disease pathway, both in terms of natural history and health care management. As new research emerges on the costs and effects of novel approaches to oral cancer management, the 186     WDMOC is uniquely poised to incorporate those changes without requiring a major re-design of the model’s structure or code. The modeling approach, in addition to producing costs and QALYs, can also output a potentially infinite variety of fields, allowing for stochastic examination of simulated individuals for demographic, clinical, and other useful values for analysis. This latter feature, which allowed the kind of in-depth analysis and interpretation of outputs conducted in Chapter 5, would be very difficult to replicate in a Markov model. 7.4 Limitations The approach adopted in this dissertation has some important limitations as well. These limitations must be considered both when interpreting the results of the analysis chapters, and when considering future directions for research. 7.4.1 Use of OPL Prevalence The principal limitation of the WDMOC is that it relies on OPL prevalence data. There is no data in the available literature, or from primary sources, to describe incidence of new OPL in the population, or the factors that influence its development among asymptomatic people. The WDMOC was therefore designed to simulate a population wherein some entities start with OPL, while everyone else will never develop oral disease (though they may still have non-precancerous lesions). Any WDM will face the problem of estimating incidence of disease before it is detectable. In the original WDM for colorectal cancer, Tappenden used an optimization algorithm (Metropolis-Hastings; M-H) to 187     estimate incidence – the development from preclinical disease to invasive cancer. This process relied on a variety of prevalence estimates including the underlying population prevalence of premalignant colorectal adenomas and the stage distribution of detected colorectal cancers. The M-H approach was not used in this dissertation. This choice was made in light of the fact that while OPL incidence was not known, reasonable values were available for premalignancy progression and the demographic characteristics of incident cancers (i.e., age, sex, smoking status, alcohol use). This data was stochastic and allowed for the model to reflect the association between these demographic factors and OPL progression, relative to LOH risk score. The M-H algorithm would not have been able to reflect the influence of these individual demographic factors on incidence, as the posterior distribution used to estimate the prior distribution would need to rely on two unknown quantities – the distribution of LOH in the asymptomatic population (i.e., prior to developing disease) and the distribution of LOH among incident cancers by stage. Given that the focus of the analytic chapters (5 and 6) relied on these demographic factors, the choice was made to design the WDMOC in such a way that used existing values from previously-published models and allowed for the substitution of the stochastic OPL progression data. A consequence of this choice is that the model cannot be used to estimate year-over-year population cancer incidence. Instead, cancer rates can only be estimated among those who begin the model with an undetected OPL. Estimating the impact of policy change in a defined cohort rather than the broader population, as is the case for the WDMOC, is common practice in decision modeling. Nevertheless, the 188     use of prevalence rather than incidence is an important limitation to the uses of the WDMOC, and findings should be interpreted with this in mind. 7.4.2 Second-Order Uncertainty Among Regression-Based Parameters Second-order uncertainty, also called parameter uncertainty, allows models to reflect the impact that statistical error around each model parameter has on model outputs. This dissertation reflected second-order uncertainty in all parameters, save those derived from regression analysis. One of the chief strengths of the implementation approach taken in this dissertation is the ability to quickly and easily adjust the WDMOC’s parameters within a common spreadsheet. The WDMOC randomly samples a value for each parameter, based on its mean and standard error at the time the corresponding value is called from the spreadsheet. While the standard error around each coefficient from a regression analysis could be calculated, they cannot be randomly sampled using the same technique. Because each dependent variable in the regression analysis is correlated to the others, incorporating them into the full analysis would require the use of the covariance matrix. Using the covariance matrix in this way would have severely limited the ability to adjust regression-based parameters. It was necessary to make these kinds of adjustments in order to incorporate new TTE data (e.g., the effect of HPV vaccination on cancer survival in Chapter 6) into the WDMOC. A choice was therefore made to program the WDMOC in such a way that allowed for these adjustments but did not reflect parameter uncertainty. As a consequence of this limitation, the WDMOC results likely underestimate the overall amount of uncertainty around cost and QALY results, as well the percentage likelihood that a policy change is cost-effective (%CE). This introduces a bias away from the null case for probabilistic analysis, but not for 189     mean ICERs. It would also not be appropriate to conduct VOI analysis for regression-based parameters. This limitation notwithstanding, the univariate sensitivity analysis conducted in Chapter 4 provides insight into the extent to which model results are robust to changes in regression-based parameters. 7.4.3 Data Availability Simplifying assumptions had to be made about several crucial model parameters across all disease components. Many of these simplifications (e.g., screening adherence, primary disease treatment) were driven by the relative paucity of studies about oral cancer in the literature. The limitations created by the absence of this data are common in health care modeling. They nevertheless raise concerns about the external validity of the model’s results, especially if those results are meant to apply outside the Lower Mainland of British Columbia. Users of the WDMOC in other policy-making jurisdictions (i.e., other provinces, countries, etc.) may wish to substitute data that is more reflective of their local clinical and demographic reality. The input structure of the WDMOC is designed to make this easy.  The validation, calibration, and sensitivity analysis steps taken in Chapter 4 are the appropriate tools for addressing the impact of these limitations on the model’s overall performance, but in the absence of stronger data it was not possible to know how well the model predicted certain outcomes like screening guideline adherence rates.  This limitation highlights an important challenge of modelling oral cancer: while the WDM framework is highly appropriate for this disease, the lack of clinical and epidemiological research presents a very real challenge for health economists. It is likely the case that many other disease areas that are theoretically 190     amenable to a WDM approach lack even the level of data used in this dissertation. One advantage of the approach to WDM implementation used in this dissertation is that it can rapidly and easily incorporate new information as future research is conducted. 7.5 Implications for Health Policy and Economic Evaluation Research This dissertation suggests that the LOH assay is likely to be cost saving and reduced cancer incidence, but may not produce an improvement in overall survival. The cost-effectiveness of using the LOH assay was also strongly influenced by the policy environment. In the presence of other policy changes – particularly tobacco/alcohol cessation programs and potentially HPV vaccination – the LOH assay becomes more favourable than under baseline conditions. This finding suggests that the assay’s cost-effectiveness is strongly influenced by factors that affect the rate of OPL progression. Beyond the particular policy implications for this particular technology, the WDMOC may be used to perform additional health economic evaluations in oral cancer anywhere in the disease trajectory. The work in this dissertation should facilitate simple and rapid evaluations of other health technologies. The methods that were developed to program the WDMOC may be used for WDMs in other disease areas as well. Oral cancer is not the only disease area where ‘upstream’ and ‘downstream’ policy changes can affect cost-effectiveness, and the work conducted in this dissertation will provide methods, tools, and important topics to consider for the next team of researchers who approach the WDM framework.  Tappenden did not explicitly consider HTM as an application for the WDM framework, but this dissertation suggests that WDMs are ideally suited for conducting HTM, as they can generate evidence 191     about how the cost-effectiveness of extant health technologies can be affected by policy changes elsewhere in the decision environment. Health economic researchers and policy-making organizations that rely on cost-effectiveness evidence should consider the potential benefit of WDMs specifically for HTM. The WDMOC was also used to evaluate prospective policy scenarios for which the evidence was unclear and had a high degree of uncertainty, suggesting that health economists should consider their use in early evaluation of technologies that have yet to undergo clinical trial testing. This type of evaluation can help guide research priorities and identify places where new discovery can yield meaningful results.  7.6 Reflections on the WDM Framework The timeline for the model’s development (three years) was much longer than what is typical for decision models, the simplest of which can often be completed in a matter of days. This long timeline raises the important pragmatic question of whether creating WDMs is “worth it”, in terms of the amount of labour required to produce one.  The entire coding structure of the WDMOC, including methods for entity creation, parametric sampling, TTE processing, was a novel development for this dissertation. Now that these methods have been worked out, the timeline for development of an additional WDM should be considerably shorter.  The term ‘open source’ describes a general category of software design where the code is made available, at no cost, to any person or group who wishes to examine it. Models developed using open source approaches are easier to share, edit, replicate, and modify than those developed in proprietary 192     software. Python is free to acquire and operate, while proprietary software packages can cost $2000 or more[219]. The whole of the WDMOC, including parameter inputs, can be attached to an e-mail (total file size = 1.1 MB), meaning it can be shared between collaborators at negligible additional cost, lowering barriers to access.  Tappenden suggests five specific criteria to justify use of a WDM[31]: 1. Presence of multiple potential components of the decision environment that have not undergone formal analysis; 2. Multiple new technologies needing to be evaluated; 3. A dearth of available health economic evidence; 4. Upstream changes that are expected to affect downstream cost-effectiveness; 5. A need for more information than just cost/QALY Work on this dissertation suggests that these criteria are accompanied by important guidelines that should be considered when undertaking the work necessary to create a WDM:  A reasonable expectation that the decision environment will change – a chief strength of the WDM is its ability to reflect changes including and in addition to the adoption of novel technologies. A WDM that can be updated and re-used as these changes occur is a valuable tool that endures beyond the single set of technologies it was programmed to evaluate;  Designed with the ability to easy incorporate new information – similarly, a WDM should be designed with the expectation that its parameters and its structure will be updated to reflect emerging epidemiological and clinical research. A rigidly-designed WDM will quickly become obsolete, making the time required to create one hard to justify;  Flexible enough to be adaptable to other health care jurisdictions – it should be possible for the parameters and structure of a WDM to be changed to reflect the population characteristics, standard of clinical practice, and health care resources of regions other than the one in which it 193     was originally constructed. Ideally, only one WDM would ever need to be constructed ‘from scratch’ for a given disease, and then adapted for use by other researchers in other contexts;  Shareable and/or free to access – the previous three points rest on the idea that a WDM is accessible to other researchers for them to make adaptations/modifications as new information becomes available and/or the decision context changes. The less accessible a WDM is, the less useful it becomes and the more anchored it becomes to decisions being made in a particular place at a particular time; An open source approach fits the original spirit of the WDM framework, allowing for multiple and ongoing evaluation of multiple policy alternatives. Using these can help to justify the labour required to create one in the first place. The methods developed within this dissertation are capable of incorporating multiple types of input data , read from a single easy-to-use interface (an Excel document).  It is important to note that a WDM is not appropriate for all circumstances. Policy questions require responsive and timely answers, and policy and clinical pathways can be accurately modeled by the techniques in the current literature. As argued in Chapter 1 and Chapter 2, oral cancer is particularly well-suited to the creation of a WDM. Researchers and policy makers may evaluate the relative merits of a WDM versus a piecewise approach and make their own decisions based on the clinical and policy circumstances within their specific context. In places where they feel the need to reflect upstream and downstream decisions simultaneously, they may choose a WDM. HTA exercises conducted within Canada are governed by guidelines set by CADTH[5]. These guidelines do not explicitly consider modeling a whole disease, but they do speak directly to many of the same 194     design and quality assurance principles that are required within the WDM framework. This dissertation demonstrates that a WDM can be constructed within these guidelines, using an identical reference case, modeling methods, and outputs of interest. As more WDMs are designed and published, guidelines like CADTHs may need to be adapted, but there is no apparent conflict between the state of the art as defined by CADTH and the use of the WDM framework. 7.7 Areas for Future Research There are important next steps for the development of the WDMOC and the methods identified within this dissertation. 7.7.1 Improvement to the Analysis It will be valuable and important to address two of the chief limitations of the WDMOC in its current form. First, it will be important to develop methods to model incidence of OPL and improve estimates of asymptomatic disease progression. This development will allow not only for more reasonable estimates of the impact of screening programs and other decisions that are upstream from invasive cancer, but will also permit year-over-year population-level forecasting instead of simple cohort simulation. This may require an adaptation and extension of the M-H algorithmic approach used by Tappenden and the collection of new empirical evidence on OPL prevalence and the distribution of LOH. Sadatsafavi used a regression-based approach to estimate COPD incidence from prevalence data – this approach could also potentially be adapted and used within the WDMOC[34]. 195     It will also be important to develop methods to incorporate parameter uncertainty into regression-based parameters. Doing so will allow VOI analysis, which can be used to identify future research priorities (i.e., variables for which the value of additional information is highest). Doing so will require would require the incorporation of the correlation matrix from the TTE regression analysis in order to achieve covariate balance, which would in turn require an extension of the existing implementation method in Python with the input of biostatistical expertise.   7.7.2 Further Applications of This Work At time of writing, the candidate has three imminent opportunities to apply the methods developed in this dissertation to additional economic evaluations. The pan-Canadian Optically-guided Oral Surgical Trial (COOLS)[62] evaluated the use of a FV camera in the surgical theatre – the ‘Improved Surgery’ scenario in Chapter 6. The WDMOC will be used to conduct the economic evaluation for this trial, as the candidate and members of the dissertation committee are among the trial’s investigators (C Poh is Principal Investigator, S Peacock is a co-Investigator, the candidate has been involved with this trial since its inception). There is another opportunity to apply the DES modelling techniques from this dissertation to cervical cancer. A team led by Dr. Gina Ogilvie will be examining the cost-effectiveness of using HPV testing to screen for cervical cancers, based on the Human Papillomavirus For Cervical Cancer screening trial (HPV FOCAL)[220]. The modeling methods developed in this dissertation are highly appropriate for this clinical population, where individual factors like age and screening adherence are expected to exert an influence on cost-effectiveness. The candidate and members of this dissertation committee are among 196     the investigator team for this project as well (S Peacock is a co-Investigator, the candidate has worked with this team regularly over the past 7 years). 7.7.3 Dissemination and Knowledge Translation In addition to these opportunities for further use, there are important and valuable opportunities for the work developed in this dissertation to be shared within and outside the academic community. This dissertation has value for health economists and the broader scientific community. The work described in chapters 4, 5, and 6 are all suitable for publication in the scientific literature. Chapter 3 is an adaptation of previously-published work, and the activities in this dissertation provide additional context to that work. As described in previous sections, the WDMOC was designed to facilitate the generation of rich cost-effectiveness evidence for economic evaluations. The primary data analysis used for parameterization is also a novel contribution to an under-studied field. Health policy researchers may also find value in the results of this dissertation, particularly the use of WDMs for HTM. The candidate is a member of the Canadian Centre for Applied Research in Cancer Control (ARCC). Membership in this network provides the candidate with access to researchers, policy-makers, patient groups, and other relevant health policy stakeholders. A member of the committee (S Peacock) is co-director of ARCC and can support the candidate in engaging the network. The findings within this dissertation, and the potential to use the WDMOC to conduct rapid economic evaluations of new oral cancer technologies may also interest a clinical audience. The candidate has previously worked with the PanCanNOC, a pan-Canadian network of oral cancer specialists. This group is 197     comprised of practitioners and researchers from provinces across Canada, many of whom are acknowledged experts in their fields and whose recommendations help guide policy making at local and provincial levels. A member of the committee (C Poh) is among the executive within the network, and can support the candidate in engaging its members. Finally, the open source programming methods within this dissertation may appeal to researchers who are particularly interested in building decision models. The candidate has also been in ongoing contact with members of Decision Analysis in R for Technologies in Health (DARTH), a team of researchers and model developers based out of the University of Toronto[221]. This team has developed methods and issued guidelines for the development of decision models in the R language[222]. The group has expressed interest in the candidate’s work and in developing similar methods and guidelines for DES modeling in R and in Python. 7.8  Conclusion The Whole Disease Modeling (WDM) framework is most appropriate in circumstances where policy decisions need to be made about multiple potential health technologies, there is little available health economic evidence to guide such decisions, and those technologies are expected to have impacts in places across the disease trajectory both individually and in concert. This dissertation identified oral cancer as such a circumstance, and described the design and implementation of a WDM of oral cancer (WDMOC) using innovative methods and based on evidence derived from the literature and from primary sources. Results from the WDMOC were compared to those derived from conventional ‘piecewise’ methods using a Markov model to demonstrate the impact that the more comprehensive 198     approach had on the cost-effectiveness of risk-guided management for oral premalignancies. The WDMOC was also used to demonstrate how changes in the policy environment might impact this estimate of cost-effectiveness, extending the use of the framework to conduct health technology management. Future uses of the WDM framework should consider using open source methods as a means of ensuring that these difficult-to-implement models can be updated to incorporate new information as it emerges, and can be made available to a variety of knowledge users.  199     Works Cited 1. Williams A: Health Economics: The Cheerful Face of the Dismal Science? In: Health and Economics. edn. Edited by Williams A. London: Palgrave Macmillan; 1987. 2. Arrow KJ: Uncertainty and the Welfare Economics of Medical Care. The American Economic Review 1963, 53(5):32. 3. Brouwer WB, Culyer AJ, van Exel NJ, Rutten FF: Welfarism vs. extra-welfarism. Journal of health economics 2008, 27(2):325-338. 4. Buchanan J, Wordsworth S: Welfarism versus extra-welfarism: can the choice of economic evaluation approach impact on the adoption decisions recommended by economic evaluation studies? Pharmacoeconomics 2015, 33(6):571-579. 5. CADTH: Guidelines for the Economic Evaluation of Health Technologies: Canada. In. Edited by CADTH. Ottawa; 2017. 6. Health technology assessment [http://www.who.int/health-technology-assessment/about/Defining/en/] 7. Banta D: The development of health technology assessment. Health Policy 2003, 63(2):121-132. 8. Banta D, Jonsson E, Childs P: History of the international societies in health technology assessment: International Society for Technology Assessment in Health Care and Health Technology Assessment International. Int J Technol Assess Health Care 2009, 25 Suppl 1:19-23. 9. Gold MR, Siegel JR, Russell LB, Weinstein MC: Cost-Effectiveness in Health and Medicine. Oxford, UK: Oxford University Press; 1996. 10. Cromwell I, Peacock SJ, Mitton C: 'Real-world' health care priority setting using explicit decision criteria: a systematic review of the literature. BMC health services research 2015, 15:164. 11. Cressman S, Browman GP, Hoch JS, Kovacic L, Peacock SJ: A Time-Trend Economic Analysis of Cancer Drug Trials. Oncologist 2015, 20(7):729-736. 12. Massetti M, Aballea S, Videau Y, Remuzat C, Roiz J, Toumi M: A comparison of HAS & NICE guidelines for the economic evaluation of health technologies in the context of their respective national health care systems and cultural environments. Journal of market access & health policy 2015, 3. 13. Wilson KS, Barnett JB, Shah A, Khoo KE: The BC Cancer Agency Compassionate Access Program: outcome analysis of patients with esophagogastric cancer. Curr Oncol 2009, 16(5):9-14. 14. Ismail Z, Peacock SJ, Kovacic L, Hoch JS: Cost-Effectiveness Impacts Cancer Care Funding Decisions in British Columbia, Canada, Evidence from 1998 to 2008. Int J Technol Assess Health Care 2017:1-6. 15. Bryan S, Mitton C, Donaldson C: Breaking the addiction to technology adoption. Health economics 2014, 23(4):379-383. 200     16. Soril LJ, MacKean G, Noseworthy TW, Leggett LE, Clement FM: Achieving optimal technology use: A proposed model for health technology reassessment. SAGE open medicine 2017, 5:2050312117704861. 17. Harris C, Green S, Elshaug AG: Sustainability in Health care by Allocating Resources Effectively (SHARE) 10: operationalising disinvestment in a conceptual framework for resource allocation. BMC health services research 2017, 17(1):632. 18. Scotland G, Bryan S: Why Do Health Economists Promote Technology Adoption Rather Than the Search for Efficiency? A Proposal for a Change in Our Approach to Economic Evaluation in Health Care. Medical decision making : an international journal of the Society for Medical Decision Making 2017, 37(2):139-147. 19. Drummond MF, Sculpher, M.J., Torrance, G.W., O'Brien, B.J., Stoddart, G.L.: Methods for the Economic Evaluation of Health Care Programmes (3rd ed.). Oxford, U.K.: Oxford University Press; 2005. 20. Bansback N, Tsuchiya A, Brazier J, Anis A: Canadian valuation of EQ-5D health states: preliminary value set and considerations for future valuation studies. PloS one 2012, 7(2):e31115. 21. Regier DA, Peacock SJ, Pataky R, van der Hoek K, Jarvik GP, Hoch J, Veenstra D: Societal preferences for the return of incidental findings from clinical genomic sequencing: a discrete-choice experiment. CMAJ 2015, 187(6):E190-197. 22. Briggs A, Claxton K, Sculpher M: Decision Modelling for Health Economic Evaluation. Oxford, UK: Oxford University Press; 2006. 23. Ramsey SD, Willke RJ, Glick H, Reed SD, Augustovski F, Jonsson B, Briggs A, Sullivan SD: Cost-effectiveness analysis alongside clinical trials II-An ISPOR Good Research Practices Task Force report. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2015, 18(2):161-172. 24. Roberts M, Russell LB, Paltiel AD, Chambers M, McEwan P, Krahn M: Conceptualizing a model: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--2. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2012, 15(6):804-811. 25. Caro JJ: Pharmacoeconomic analyses using discrete event simulation. Pharmacoeconomics 2005, 23(4):323-332. 26. Barton P, Bryan S, Robinson S: Modelling in the economic evaluation of health care: selecting the appropriate approach. Journal of health services research & policy 2004, 9(2):110-118. 27. Brennan A, Chick SE, Davies R: A taxonomy of model structures for economic evaluation of health technologies. Health economics 2006, 15(12):1295-1310. 28. Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM: State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--3. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2012, 15(6):812-820. 201     29. Caro JJ, Moller J, Getsios D: Discrete event simulation: the preferred technique for health economic evaluations? Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2010, 13(8):1056-1060. 30. Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Moller J: Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--4. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2012, 15(6):821-827. 31. Tappenden P, Chilcott J, Brennan A, Squires H, Stevenson M: Whole disease modeling to inform resource allocation decisions in cancer: a methodological framework. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2012, 15(8):1127-1136. 32. Tappenden P, Chilcott J, Brennan A, Squires H, Glynne-Jones R, Tappenden J: Using whole disease modeling to inform resource allocation decisions: economic evaluation of a clinical guideline for colorectal cancer using a single model. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2013, 16(4):542-553. 33. Atkin W, Brenner A, Martin J, Wooldrage K, Shah U, Lucas F, Greliak P, Pack K, Kralj-Hans I, Thomson A et al: The clinical effectiveness of different surveillance strategies to prevent colorectal cancer in people with intermediate-grade colorectal adenomas: a retrospective cohort analysis, and psychological and economic evaluations. Health Technol Assess 2017, 21(25):1-536. 34. Sadatsafavi M, Ghanbarian S, Adibi A, Johnson K, FitzGerald JM, Flanagan W, Bryan S, Sin D, Canadian Respiratory Research N: Development and Validation of the Evaluation Platform in COPD (EPIC): A Population-Based Outcomes Model of COPD for Canada. Medical decision making : an international journal of the Society for Medical Decision Making 2019, 39(2):152-167. 35. Canadian Cancer Statistics 2017 [ cancer.ca/Canadian-CancerStatistics-2017-EN.pdf] 36. Survival Statistics for Oral Cancer [http://www.cancer.ca/en/cancer-information/cancer-type/oral/prognosis-and-survival/survival-statistics/?region=on] 37. College of Dental Surgeons of British Columbia: Guideline for the Early Detection of Oral Cancer in British Columbia 2008. In.; 2008. 38. Lambert R, Sauvaget C, de Camargo Cancela M, Sankaranarayanan R: Epidemiology of cancer from the oral cavity and oropharynx. European journal of gastroenterology & hepatology 2011, 23(8):633-641. 39. Gupta N, Gupta R, Acharya AK, Patthi B, Goud V, Reddy S, Garg A, Singla A: Changing Trends in oral cancer - a global scenario. Nepal journal of epidemiology 2016, 6(4):613-619. 40. Jemal A, Bray F, Center M, Ferlay J, Ward E, Forman D: Global cancer statistics. CA: A Cancer Journal for Clinicians 2011, 61:69-90. 41. Hussein AA, Helder MN, de Visscher JG, Leemans CR, Braakhuis BJ, de Vet HCW, Forouzanfar T: Global incidence of oral and oropharynx cancer in patients younger than 45 years versus older patients: A systematic review. Eur J Cancer 2017, 82:115-127. 202     42. Ali J, Sabiha B, Jan HU, Haider SA, Khan AA, Ali SS: Genetic etiology of oral cancer. Oral Oncol 2017, 70:23-28. 43. Chen X, Zhao Y: Human papillomavirus infection in oral potentially malignant disorders and cancer. Archives of oral biology 2017, 83:334-339. 44. Taberna M, Mena M, Pavon MA, Alemany L, Gillison ML, Mesia R: Human papillomavirus related oropharyngeal cancer. Annals of oncology : official journal of the European Society for Medical Oncology 2017. 45. Head & Neck Cancer [http://www.bccancer.bc.ca/health-info/types-of-cancer/head-neck] 46. Oral Cancer Prevention Program [http://www.bccancer.bc.ca/health-professionals/clinical-resources/oral] 47. National Comprehensive Cancer Network: NCCN Clinical Practice Guidelines in Oncology: Head and Neck Cancers. In. Edited by Network NCC; 2013. 48. Brown JS, Rogers SN, Lowe D: A comparison of tongue and soft palate squamous cell carcinoma treated by primary surgery in terms of survival and quality of life outcomes. International journal of oral and maxillofacial surgery 2006, 35(3):208-214. 49. Hammerlid E, Bjordal K, Ahlner-Elmqvist M, Boysen M, Evensen JF, Biorklund A, Jannert M, Kaasa S, Sullivan M, Westin T: A prospective study of quality of life in head and neck cancer patients. Part I: at diagnosis. The Laryngoscope 2001, 111(4 Pt 1):669-680. 50. Korfage A, Schoen PJ, Raghoebar GM, Bouma J, Burlage FR, Roodenburg JL, Vissink A, Reintsema H: Five-year follow-up of oral functioning and quality of life in patients with oral cancer with implant-retained mandibular overdentures. Head Neck 2011, 33(6):831-839. 51. Nagy J, Braunitzer G, Antal M, Berkovits C, Novak P, Nagy K: Quality of life in head and neck cancer patients after tumor therapy and subsequent rehabilitation: an exploratory study. Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation 2014, 23(1):135-143. 52. Pandey M, Devi N, Ramdas K, Krishnan R, Kumar V: Higher distress relates to poor quality of life in patients with head and neck cancer. International journal of oral and maxillofacial surgery 2009, 38(9):955-959. 53. Rogers SN, Harvey-Woodworth CN, Hare J, Leong P, Lowe D: Patients' perception of the financial impact of head and neck cancer and the relationship to health related quality of life. The British journal of oral & maxillofacial surgery 2012, 50(5):410-416. 54. Bjordal K, Ahlner-Elmqvist M, Hammerlid E, Boysen M, Evensen JF, Biorklund A, Jannert M, Westin T, Kaasa S: A prospective study of quality of life in head and neck cancer patients. Part II: Longitudinal data. The Laryngoscope 2001, 111(8):1440-1452. 55. da Silva SD, Hier M, Mlynarek A, Kowalski LP, Alaoui-Jamali MA: Recurrent oral cancer: current and emerging therapeutic approaches. Frontiers in pharmacology 2012, 3:149. 56. Chinn SB, Myers JN: Oral Cavity Carcinoma: Current Management, Controversies, and Future Directions. Journal of clinical oncology : official journal of the American Society of Clinical Oncology 2015, 33(29):3269-3276. 203     57. Laronde DM, Bottorff JL, Hislop TG, Poh CY, Currie B, Williams PM, Rosin MP: Voices from the community--experiences from the dental office: initiating oral cancer screening. J Can Dent Assoc 2008, 74(3):239-241. 58. Jitender S, Sarika G, Varada HR, Omprakash Y, Mohsin K: Screening for oral cancer. Journal of experimental therapeutics & oncology 2016, 11(4):303-307. 59. Giovannacci I, Vescovi P, Manfredi M, Meleti M: Non-invasive visual tools for diagnosis of oral cancer and dysplasia: A systematic review. Medicina oral, patologia oral y cirugia bucal 2016, 21(3):e305-315. 60. Laronde DM, Corbett KK: Adjunctive screening devices for oral lesions: their use by Canadian Dental Hygienists and the need for knowledge translation. International journal of dental hygiene 2015. 61. Zhang L, Poh CF, Williams M, Laronde DM, Berean K, Gardner PJ, Jiang H, Wu L, Lee JJ, Rosin MP: Loss of heterozygosity (LOH) profiles--validated risk predictors for progression to oral cancer. Cancer Prev Res (Phila) 2012, 5(9):1081-1089. 62. Poh CF, Durham JS, Brasher PM, Anderson DW, Berean KW, MacAulay CE, Lee JJ, Rosin MP: Canadian Optically-guided approach for Oral Lesions Surgical (COOLS) trial: study protocol for a randomized controlled trial. BMC Cancer 2011, 11:462. 63. Govers TM, Hannink G, Merkx MA, Takes RP, Rovers MM: Sentinel node biopsy for squamous cell carcinoma of the oral cavity and oropharynx: a diagnostic meta-analysis. Oral Oncol 2013, 49(8):726-732. 64. Jayakar R, Choi J, MacKinnon C, Tan S: The cost of major head and neck cancer surgery. The New Zealand medical journal 2017, 130(1455):111-119. 65. Blasco MA, Svider PF, Raza SN, Jacobs JR, Folbe AJ, Saraf P, Eloy JA, Baredes S, Fribley AM: Systemic therapy for head and neck squamous cell carcinoma: Historical perspectives and recent breakthroughs. The Laryngoscope 2017. 66. Harrington KJ, Ferris RL, Blumenschein G, Jr., Colevas AD, Fayette J, Licitra L, Kasper S, Even C, Vokes EE, Worden F et al: Nivolumab versus standard, single-agent therapy of investigator's choice in recurrent or metastatic squamous cell carcinoma of the head and neck (CheckMate 141): health-related quality-of-life results from a randomised, phase 3 trial. The lancet oncology 2017, 18(8):1104-1115. 67. Singer S: Quality of life after nivolumab treatment for head and neck cancer. The lancet oncology 2017, 18(8):993-994. 68. van der Waal I: Potentially malignant disorders of the oral and oropharyngeal mucosa; terminology, classification and present concepts of management. Oral Oncol 2009, 45(4-5):317-323. 69. Napier SS, Speight PM: Natural history of potentially malignant oral lesions and conditions: an overview of the literature. Journal of oral pathology & medicine : official publication of the International Association of Oral Pathologists and the American Academy of Oral Pathology 2008, 37(1):1-10. 70. Yardimci G, Kutlubay Z, Engin B, Tuzun Y: Precancerous lesions of oral mucosa. World journal of clinical cases 2014, 2(12):866-872. 204     71. Warnakulasuriya S, Ariyawardana A: Malignant transformation of oral leukoplakia: a systematic review of observational studies. Journal of oral pathology & medicine : official publication of the International Association of Oral Pathologists and the American Academy of Oral Pathology 2016, 45(3):155-166. 72. Laronde DM, Williams PM, Hislop TG, Poh C, Ng S, Zhang L, Rosin MP: Decision making on detection and triage of oral mucosa lesions in community dental practices: screening decisions and referral. Community dentistry and oral epidemiology 2014, 42(4):375-384. 73. Villa A, Gohel A: Oral potentially malignant disorders in a large dental population. Journal of applied oral science : revista FOB 2014, 22(6):473-476. 74. Amagasa T, Yamashiro M, Uzawa N: Oral premalignant lesions: from a clinical perspective. International journal of clinical oncology 2011, 16(1):5-14. 75. Yanik EL, Katki HA, Silverberg MJ, Manos MM, Engels EA, Chaturvedi AK: Leukoplakia, Oral Cavity Cancer Risk, and Cancer Survival in the U.S. Elderly. Cancer Prev Res (Phila) 2015, 8(9):857-863. 76. Johnson S, Corsten MJ, McDonald JT, Chun J: Socio-economic factors and stage at presentation of head and neck cancer patients in Ottawa, Canada: a logistic regression analysis. Oral Oncol 2010, 46(5):366-368. 77. Speight PM, Epstein J, Kujan O, Lingen MW, Nagao T, Ranganathan K, Vargas P: Screening for oral cancer-a perspective from the Global Oral Cancer Forum. Oral surgery, oral medicine, oral pathology and oral radiology 2017, 123(6):680-687. 78. Downer MC, Moles DR, Palmer S, Speight PM: A systematic review of measures of effectiveness in screening for oral cancer and precancer. Oral Oncol 2006, 42(6):551-560. 79. Warnakulasuriya S, Fennell N, Diz P, Seoane J, Rapidis A: An appraisal of oral cancer and pre-cancer screening programmes in Europe: a systematic review. Journal of oral pathology & medicine : official publication of the International Association of Oral Pathologists and the American Academy of Oral Pathology 2015, 44(8):559-570. 80. Brocklehurst P, Kujan O, O'Malley LA, Ogden G, Shepherd S, Glenny AM: Screening programmes for the early detection and prevention of oral cancer. The Cochrane database of systematic reviews 2013(11):CD004150. 81. Petti S, Scully C: How many individuals must be screened to reduce oral cancer mortality rate in the Western context? A challenge. Oral diseases 2015, 21(8):949-954. 82. Gogarty DS, Shuman A, O'Sullivan EM, Sheahan P, Kinsella J, Timon C, O'Neill JP: Conceiving a national head and neck cancer screening programme. The Journal of laryngology and otology 2016, 130(1):8-14. 83. Lim K, Moles DR, Downer MC, Speight PM: Opportunistic screening for oral cancer and precancer in general dental practice: results of a demonstration study. British dental journal 2003, 194(9):497-502; discussion 493. 84. LeHew CW, Epstein JB, Kaste LM, Choi YK: Assessing oral cancer early detection: clarifying dentists' practices. Journal of public health dentistry 2010, 70(2):93-100. 205     85. Hassona Y, Scully C, Shahin A, Maayta W, Sawair F: Factors Influencing Early Detection of Oral Cancer by Primary Health-Care Professionals. Journal of cancer education : the official journal of the American Association for Cancer Education 2016, 31(2):285-291. 86. Clarke AK, Kobagi N, Yoon MN: Oral cancer screening practices of Canadian dental hygienists. International journal of dental hygiene 2017. 87. Tax CL, Haslam SK, Brillant M, Doucette HJ, Cameron JE, Wade SE: Oral cancer screening: knowledge is not enough. International journal of dental hygiene 2017, 15(3):179-186. 88. Walker BB, Schuurman N, Auluck A, Lear SA, Rosin M: Suburbanisation of oral cavity cancers: evidence from a geographically-explicit observational study of incidence trends in British Columbia, Canada, 1981-2010. BMC public health 2015, 15:758. 89. Rivera C: Essentials of oral cancer. International journal of clinical and experimental pathology 2015, 8(9):11884-11894. 90. Tiwana MS, Wu J, Hay J, Wong F, Cheung W, Olson RA: 25 year survival outcomes for squamous cell carcinomas of the head and neck: population-based outcomes from a Canadian province. Oral Oncol 2014, 50(7):651-656. 91. Rogers SN, Brown, J.S., Woolgar, J.A., Lowe, D., Magennis, P., Shaw, R.J., Sutton, D., Errington, D., Vaughan, D.: Survival Following Primary Surgery for Oral Cancer. Oral Oncology 2009, 45:201-211. 92. Huang TY, Hsu LP, Wen YH, Huang TT, Chou YF, Lee CF, Yang MC, Chang YK, Chen PR: Predictors of locoregional recurrence in early stage oral cavity cancer with free surgical margins. Oral Oncol 2010, 46(1):49-55. 93. Tam S, Araslanova R, Low TH, Warner A, Yoo J, Fung K, MacNeil SD, Palma DA, Nichols AC: Estimating Survival After Salvage Surgery for Recurrent Oral Cavity Cancer. JAMA otolaryngology-- head & neck surgery 2017, 143(7):685-690. 94. Mucke T, Wagenpfeil, S., Kesting, M.R., Holzle, F., Wolff, K.D.: Recurrence Interval Affects Survival After Local Relapse of Oral Cancer. Oral Oncology 2009, 45:687-691. 95. Agra IM, Carvalho AL, Pinto CA, Martins EP, Filho JG, Soares FA, Kowalski LP: Biological markers and prognosis in recurrent oral cancer after salvage surgery. Archives of otolaryngology--head & neck surgery 2008, 134(7):743-749. 96. Kernohan MD, Clark JR, Gao K, Ebrahimi A, Milross CG: Predicting the prognosis of oral squamous cell carcinoma after first recurrence. Archives of otolaryngology--head & neck surgery 2010, 136(12):1235-1239. 97. Moore KA, Ford PJ, Farah CS: Support needs and quality of life in oral cancer: a systematic review. International journal of dental hygiene 2014, 12(1):36-47. 98. Barrios R, Montero J, Gonzalez-Moles MA, Baca P, Bravo M: Levels of scientific evidence of the quality of life in patients treated for oral cancer. Medicina oral, patologia oral y cirugia bucal 2013, 18(4):e578-584. 99. Abendstein H, Nordgren M, Boysen M, Jannert M, Silander E, Ahlner-Elmqvist M, Hammerlid E, Bjordal K: Quality of life and head and neck cancer: a 5 year prospective study. The Laryngoscope 2005, 115(12):2183-2192. 206     100. Nordgren M, Jannert M, Boysen M, Ahlner-Elmqvist M, Silander E, Bjordal K, Hammerlid E: Health-related quality of life in patients with pharyngeal carcinoma: a five-year follow-up. Head Neck 2006, 28(4):339-349. 101. Crombie AK, Farah CS, Batstone MD: Health-related quality of life of patients treated with primary chemoradiotherapy for oral cavity squamous cell carcinoma: a comparison with surgery. The British journal of oral & maxillofacial surgery 2014, 52(2):111-117. 102. Tarsitano A, Pizzigallo A, Ballone E, Marchetti C: Health-related quality of life as a survival predictor for patients with oral cancer: is quality of life associated with long-term overall survival? Oral surgery, oral medicine, oral pathology and oral radiology 2012, 114(6):756-763. 103. Meregaglia M, Cairns J: A systematic literature review of health state utility values in head and neck cancer. Health and quality of life outcomes 2017, 15(1):174. 104. Truong MT, Zhang Q, Rosenthal DI, List M, Axelrod R, Sherman E, Weber R, Nguyen-Tan PF, El-Naggar A, Konski A et al: Quality of Life and Performance Status From a Substudy Conducted Within a Prospective Phase 3 Randomized Trial of Concurrent Accelerated Radiation Plus Cisplatin With or Without Cetuximab for Locally Advanced Head and Neck Carcinoma: NRG Oncology Radiation Therapy Oncology Group 0522. Int J Radiat Oncol Biol Phys 2017, 97(4):687-699. 105. Pottel L, Lycke M, Boterberg T, Pottel H, Goethals L, Duprez F, Rottey S, Lievens Y, Van Den Noortgate N, Geldhof K et al: G-8 indicates overall and quality-adjusted survival in older head and neck cancer patients treated with curative radiochemotherapy. BMC Cancer 2015, 15:875. 106. Govers TM, Schreuder WH, Klop WM, Grutters JP, Rovers MM, Merkx MA, Takes RP: Quality of life after different procedures for regional control in oral cancer patients: cross-sectional survey. Clinical otolaryngology : official journal of ENT-UK ; official journal of Netherlands Society for Oto-Rhino-Laryngology & Cervico-Facial Surgery 2016, 41(3):228-233. 107. Pickard AS, Jiang R, Lin HW, Rosenbloom S, Cella D: Using Patient-reported Outcomes to Compare Relative Burden of Cancer: EQ-5D and Functional Assessment of Cancer Therapy-General in Eleven Types of Cancer. Clinical therapeutics 2016, 38(4):769-777. 108. Rogers SN, Miller RD, Ali K, Minhas AB, Williams HF, Lowe D: Patients' perceived health status following primary surgery for oral and oropharyngeal cancer. International journal of oral and maxillofacial surgery 2006, 35(10):913-919. 109. Noel CW, Lee DJ, Kong Q, Xu W, Simpson C, Brown D, Gilbert RW, Gullane PJ, Irish JC, Huang SH et al: Comparison of Health State Utility Measures in Patients With Head and Neck Cancer. JAMA otolaryngology-- head & neck surgery 2015, 141(8):696-703. 110. Marcellusi A, Capone A, Favato G, Mennini FS, Baio G, Haeussler K, Bononi M: Health utilities lost and risk factors associated with HPV-induced diseases in men and women: the HPV Italian collaborative study group. Clinical therapeutics 2015, 37(1):156-167 e154. 111. Del Barco Morillo E, Mesia R, Adansa Klain JC, Vazquez Fernandez S, Martinez-Galan J, Pastor Borgonon M, Gonzalez-Rivas C, Caballero Daroqui J, Berrocal A, Martinez-Trufero J et al: Phase II study of panitumumab and paclitaxel as first-line treatment in recurrent or metastatic head and neck cancer. TTCC-2009-03/VECTITAX study. Oral Oncol 2016, 62:54-59. 207     112. Rana M, Gellrich NC: Comparison of health-related quality of life of patients with different precancer and oral cancer stages. Clinical oral investigations 2015, 19(2):481-488. 113. Hettiarachchi RM, Kularatna S, Downes MJ, Byrnes J, Kroon J, Lalloo R, Johnson NW, Scuffham PA: The cost-effectiveness of oral health interventions: A systematic review of cost-utility analyses. Community dentistry and oral epidemiology 2017. 114. McManus E, Sach T, Levell N: The Use of Decision-Analytic Models in Atopic Eczema: A Systematic Review and Critical Appraisal. Pharmacoeconomics 2017. 115. Monten C, Veldeman L, Verhaeghe N, Lievens Y: A systematic review of health economic evaluation in adjuvant breast radiotherapy: Quality counted by numbers. Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology 2017. 116. Wesley RS, Kutty VR, Matthew B, Sankaranarayanan R, Nair MK: Economic comparison of two strategies of oral cancer screening. Health policy and planning 1992, 7(3):6. 117. Funk GF, Hoffman HT, Karnell LH, Ricks JM, Zimmerman MB, Corbae DP, Hussey DH, McCulloch TM, Graham SM, Dawson CJ et al: Cost-identification analysis in oral cavity cancer management. Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery 1998, 118(2):211-220. 118. Smeele LE, Irish JC, Gullane PJ, Neligan P, Brown DH, Rotstein LE: A retrospective comparison of the morbidity and cost of different reconstructive strategies in oral and oropharyngeal carcinoma. The Laryngoscope 1999, 109(5):800-804. 119. van Agthoven M, van Ineveld BM, de Boer MF, Leemans CR, Knegt PP, Snow GB, Uyl-de Groot CA: The costs of head and neck oncology: primary tumours, recurrent tumours and long-term follow-up. Eur J Cancer 2001, 37(17):2204-2211. 120. van der Meij EH, Bezemer PD, van der Waal I: Cost-effectiveness of screening for the possible development of cancer in patients with oral lichen planus. Community dentistry and oral epidemiology 2002, 30(5):342-351. 121. Zavras A, Andreopoulos N, Katsikeris N, Zavras D, Cartsos V, Vamvakidis A: Oral cancer treatment costs in Greece and the effect of advanced disease. BMC public health 2002, 2:12. 122. Nijdam W, Levendag P, Noever I, Uyl-de Groot C, van Agthoven M: Cost analysis comparing brachytherapy versus surgery for primary carcinoma of the tonsillar fossa and/or soft palate. Int J Radiat Oncol Biol Phys 2004, 59(2):488-494. 123. Smeele LE, Goldstein D, Tsai V, Gullane PJ, Neligan P, Brown DH, Irish JC: Morbidity and cost differences between free flap reconstruction and pedicled flap reconstruction in oral and oropharyngeal cancer: Matched control study. The Journal of otolaryngology 2006, 35(2):102-107. 124. Speight PM, Palmer S, Moles DR, Downer MC, Smith DH, Henriksson M, Augustovski F: The cost-effectiveness of screening for oral cancer in primary care. Health Technol Assess 2006, 10(14):1-144, iii-iv. 125. Hollenbeak CS, Stack BC, Jr., Daley SM, Piccirillo JF: Using comorbidity indexes to predict costs for head and neck cancer. Archives of otolaryngology--head & neck surgery 2007, 133(1):24-27. 208     126. Nijdam W, Levendag P, Fuller D, Schulz R, Prevost JB, Noever I, Uyl-de Groot C: Robotic radiosurgery vs. brachytherapy as a boost to intensity modulated radiotherapy for tonsillar fossa and soft palate tumors: the clinical and economic impact of an emerging technology. Technology in cancer research & treatment 2007, 6(6):611-620. 127. Epstein JD, Knight TK, Epstein JB, Bride MA, Nichol MB: Cost of care for early- and late-stage oral and pharyngeal cancer in the California Medicaid population. Head Neck 2008, 30(2):178-186. 128. Lin CS, Lee HC, Lin CT, Lin HC: The association between surgeon case volume and hospitalization costs in free flap oral cancer reconstruction operations. Plastic and reconstructive surgery 2008, 122(1):133-139. 129. Subramanian S, Sankaranarayanan R, Bapat B, Somanathan T, Thomas G, Mathew B, Vinoda J, Ramadas K: Cost-effectiveness of oral cancer screening: results from a cluster randomized controlled trial in India. Bulletin of the World Health Organization 2009, 87(3):200-206. 130. Han S, Chen Y, Ge X, Zhang M, Wang J, Zhao Q, He J, Wang Z: Epidemiology and cost analysis for patients with oral cancer in a university hospital in China. BMC public health 2010, 10:196. 131. Lee CC, Ho HC, Jack LC, Su YC, Lee MS, Hung SK, Chou P: Association between surgeon volume and hospitalisation costs for patients with oral cancer: a nationwide population base study in Taiwan. Clinical otolaryngology : official journal of ENT-UK ; official journal of Netherlands Society for Oto-Rhino-Laryngology & Cervico-Facial Surgery 2010, 35(1):46-52. 132. Dedhia RC, Smith KJ, Johnson JT, Roberts M: The cost-effectiveness of community-based screening for oral cancer in high-risk males in the United States: a Markov decision analysis approach. The Laryngoscope 2011, 121(5):952-960. 133. Kim K, Amonkar MM, Hogberg D, Kasteng F: Economic burden of resected squamous cell carcinoma of the head and neck in an incident cohort of patients in the UK. Head & neck oncology 2011, 3:47. 134. Jacobson JJ, Epstein JB, Eichmiller FC, Gibson TB, Carls GS, Vogtmann E, Wang S, Murphy B: The cost burden of oral, oral pharyngeal, and salivary gland cancers in three groups: commercial insurance, Medicare, and Medicaid. Head & neck oncology 2012, 4:15. 135. Park YD, Kang JO, Kim SJ, Kwon HJ, Hwang JH, Hwang KS: Estimation of the costs of smoking-related oral disease: a representative South Korean study. International dental journal 2012, 62(5):256-261. 136. Govers TM, Takes RP, Baris Karakullukcu M, Hannink G, Merkx MA, Grutters JP, Rovers MM: Management of the N0 neck in early stage oral squamous cell cancer: a modeling study of the cost-effectiveness. Oral Oncol 2013, 49(8):771-777. 137. Li R, Fakhry C, Koch WM, Gourin CG: The Effect of tumor subsite on short-term outcomes and costs of care after oral cancer surgery. The Laryngoscope 2013, 123(7):1652-1659. 138. O'Connor R, Pezier T, Schilling C, McGurk M: The relative cost of sentinel lymph node biopsy in early oral cancer. Journal of cranio-maxillo-facial surgery : official publication of the European Association for Cranio-Maxillo-Facial Surgery 2013, 41(8):721-727. 139. Goyal S, Tiwari VK, Nair KS, Raj S: Risk factors and costs of oral cancer in a tertiary care hospital in Delhi. Asian Pacific journal of cancer prevention : APJCP 2014, 15(4):1659-1665. 209     140. Lee MK, Dodson TB, Nalliah RP, Karimbux NY, Allareddy V: Nine-year trend analysis of hospitalizations attributed to oral and oropharyngeal cancers in the United States. Oral surgery, oral medicine, oral pathology and oral radiology 2014, 118(1):47-67. 141. Sheets NC, Wheeler SB, Kohler RE, Fried DV, Brown PM, Chera BS: Costs of care in a matched pair comparison of intensity-modulated radiation therapy (IMRT) versus conventional radiation therapy (CRT) for the treatment of head and neck cancer. American journal of clinical oncology 2014, 37(6):539-544. 142. Enomoto LM, Schaefer EW, Goldenberg D, Mackley H, Koch WM, Hollenbeak CS: The Cost of Hospice Services in Terminally Ill Patients With Head and Neck Cancer. JAMA otolaryngology-- head & neck surgery 2015, 141(12):1066-1074. 143. Govers TM, Patel S, Takes RP, Merkx T, Rovers M, Grutters J: Cost-effectiveness of selective neck dissection versus modified radical neck dissection for treating metastases in patients with oral cavity cancer: A modelling study. Head Neck 2015, 37(12):1762-1768. 144. Hollenbeak CS, Kulaylat AN, Mackley H, Koch W, Schaefer EW, Goldenberg D: Determinants of Medicare Costs for Elderly Patients With Oral Cavity and Pharyngeal Cancers. JAMA otolaryngology-- head & neck surgery 2015, 141(7):628-635. 145. Lee CC, Chang TS, Wu CJ, Yang CC, Chen PC: Determinants of End-of-Life Expenditures in Patients with Oral Cancer in Taiwan: A Population-Based Study. PloS one 2015, 10(5):e0126482. 146. Simons PA, Ramaekers B, Hoebers F, Kross KW, Marneffe W, Pijls-Johannesma M, Vandijck D: Cost-Effectiveness of Reduced Waiting Time for Head and Neck Cancer Patients due to a Lean Process Redesign. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2015, 18(5):587-596. 147. Acevedo JR, Fero KE, Wilson B, Sacco AG, Mell LK, Coffey CS, Murphy JD: Cost-Effectiveness Analysis of Elective Neck Dissection in Patients With Clinically Node-Negative Oral Cavity Cancer. Journal of clinical oncology : official journal of the American Society of Clinical Oncology 2016, 34(32):3886-3891. 148. Cromwell I, Regier DA, Peacock SJ, Poh CF: Cost-Effectiveness Analysis of Using Loss of Heterozygosity to Manage Premalignant Oral Dysplasia in British Columbia, Canada. Oncologist 2016, 21(9):1099-1106. 149. Forner D, Phillips T, Rigby M, Hart R, Taylor M, Trites J: Submental island flap reconstruction reduces cost in oral cancer reconstruction compared to radial forearm free flap reconstruction: a case series and cost analysis. Journal of otolaryngology - head & neck surgery = Le Journal d'oto-rhino-laryngologie et de chirurgie cervico-faciale 2016, 45:11. 150. Perrier L, Morelle M, Pommier P, Boisselier P, Coche-Dequeant B, Gallocher O, Alfonsi M, Bardet E, Rives M, Calugaru V et al: Cost Analysis of Complex Radiation Therapy for Patients With Head and Neck Cancer. Int J Radiat Oncol Biol Phys 2016, 95(2):654-662. 151. van der Linden N, Flach GB, de Bree R, Uyl-de Groot CA: Cost-utility of sentinel lymph node biopsy in cT1-T2N0 oral cancer. Oral Oncol 2016, 53:20-26. 152. Keeping ST, Tempest MJ, Stephens SJ, Carroll SM, Simcock R, Jones TM, Shaw R: The cost of oropharyngeal cancer in England: A retrospective hospital data analysis. Clinical 210     otolaryngology : official journal of ENT-UK ; official journal of Netherlands Society for Oto-Rhino-Laryngology & Cervico-Facial Surgery 2017. 153. Khoudigian-Sinani S, Blackhouse G, Levine M, Thabane L, O'Reilly D: The premarket assessment of the cost-effectiveness of a predictive technology "Straticyte" for the early detection of oral cancer: a decision analytic model. Health economics review 2017, 7(1):35. 154. Kumdee C, Kulpeng W, Teerawattananon Y: Cost-utility analysis of the screening program for early oral cancer detection in Thailand. PloS one 2018, 13(11):e0207442. 155. De Oliveira C, Pataky R, Bremner KE, Rangrej J, Chan KK, Cheung WY, Hoch JS, Peacock S, Krahn MD: Estimating the Cost of Cancer Care in British Columbia and Ontario: A Canadian Inter-Provincial Comparison. Healthcare policy = Politiques de sante 2017, 12(3):95-108. 156. Canada's Tobacco Strategy [https://www.canada.ca/content/dam/hc-sc/documents/services/publications/healthy-living/canada-tobacco-strategy/overview-canada-tobacco-strategy-eng.pdf] 157. Implementing Tobacco Control [https://www.who.int/tobacco/control/en/] 158. Limit Alcohol [http://www.cancer.ca/en/prevention-and-screening/reduce-cancer-risk/make-healthy-choices/limit-alcohol/?region=on] 159. Lodi G, Franchini R, Warnakulasuriya S, Varoni EM, Sardella A, Kerr AR, Carrassi A, MacDonald LC, Worthington HV: Interventions for treating oral leukoplakia to prevent oral cancer. The Cochrane database of systematic reviews 2016, 7:CD001829. 160. Elective vs Therapeutic Neck Dissection in Treatment of Early Node Negative Squamous Carcinoma of Oral Cavity (NCT00193765) [https://clinicaltrials.gov/ct2/show/NCT00193765?term=%22Oral+cancer%22&rank=10] 161. Ribeiro IP, Barroso L, Marques F, Melo JB, Carreira IM: Early detection and personalized treatment in oral cancer: the impact of omics approaches. Molecular cytogenetics 2016, 9:85. 162. Statistics CCSsACoC: Canadian Cancer Statistics 2014. In: Canadian Cancer Statistics. Edited by Society CC. Toronto, ON; 2014. 163. Oral Cancer Incidence (New Cases) by Age, Race, and Gender [http://www.nidcr.nih.gov/DataStatistics/FindDataByTopic/OralCancer/OralCancerIncidence.htm] 164. Oral cancer incidence by sex and UK region [http://www.cancerresearchuk.org/health-professional/cancer-statistics/statistics-by-cancer-type/oral-cancer/incidence] 165. Statistics and Outlook for Mouth Cancers [http://www.cancerresearchuk.org/about-cancer/type/mouth-cancer/treatment/statistics-and-outlook-for-mouth-cancers] 166. Survival rates for oral cavity and oropharyngeal cancer by stage [http://www.cancer.org/cancer/oralcavityandoropharyngealcancer/detailedguide/oral-cavity-and-oropharyngeal-cancer-survival-rates] 167. Early Detection, Diagnosis and Staging [http://www.oralcancerfoundation.org/cdc/cdc_chapter5.php] 168. Rosin MP, Poh CF, Elwood JM, Williams PM, Gallagher R, MacAulay C, Lam WW, Auluck A, Zhang L, Hislop TG: New hope for an oral cancer solution: together we can make a difference. J Can Dent Assoc 2008, 74(3):261-266. 211     169. Program BOCP: Guideline for the Early Detection of Oral Cancer in British Columbia, 2008. In. Edited by Columbia CoDSoB. Vancouver, BC; 2008. 170. Oral Cancer Prevention Program - Clinical Services [http://www.bccancer.bc.ca/health-professionals/professional-resources/oral/clinical-services ] 171. Naimark DM, Kabboul NN, Krahn MD: The half-cycle correction revisited: redemption of a kludge. Medical decision making : an international journal of the Society for Medical Decision Making 2013, 33(7):961-970. 172. Ganly I, Goldstein D, Carlson DL, Patel SG, O'Sullivan B, Lee N, Gullane P, Shah JP: Long-term regional control and survival in patients with "low-risk," early stage oral tongue cancer managed by partial glossectomy and neck dissection without postoperative radiation: the importance of tumor thickness. Cancer 2013, 119(6):1168-1176. 173. Mucke T, Wagenpfeil S, Kesting MR, Holzle F, Wolff KD: Recurrence interval affects survival after local relapse of oral cancer. Oral Oncol 2009, 45(8):687-691. 174. Rogers SN, Brown JS, Woolgar JA, Lowe D, Magennis P, Shaw RJ, Sutton D, Errington D, Vaughan D: Survival following primary surgery for oral cancer. Oral oncology 2009, 45(3):201-211. 175. Statistics Canada: Table  102-0504 -  Deaths and mortality rates, by age group and sex, Canada, provinces and territories, annual,  CANSIM In.: Statistics Canada; 2015. 176. MSP Payment Schedule [http://www2.gov.bc.ca/gov/content/health/practitioner-professional-resources/msp/physicians/payment-schedules/msc-payment-schedule] 177. Hannouf MB, Sehgal C, Cao JQ, Mocanu JD, Winquist E, Zaric GS: Cost-effectiveness of adding cetuximab to platinum-based chemotherapy for first-line treatment of recurrent or metastatic head and neck cancer. PloS one 2012, 7(6):e38557. 178. Downer MC, Jullien JA, Speight PM: An interim determination of health gain from oral cancer and precancer screening: 1. Obtaining health state utilities. Community Dent Health 1997, 14(3):139-142. 179. Drummond MF, Barbieri, M., Cook, J., Glick, H., Lis, J., Malik, F., Reed, S., Rutten, F., Sculpher, M., Severens, J.: Transferability of Economic Evaluations Across Jurisdictions: Report of an ISPOR Task Force. In. Edited by ISPOR; 2008. 180. Caro JJ, Moller J, Karnon J, Stahl J, Ishak J: Discrete Event Simulation for Health Technology Assessment. Boca Raton, Florida, USA: CRC Press; 2016. 181. Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD: Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical decision making : an international journal of the Society for Medical Decision Making 2012, 32(5):722-732. 182. Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD: Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--6. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2012, 15(6):835-842. 183. Table 105-0501 - Smokers, by sex, provinces and territories  (Percent)  212     184. Contacted a Dentist, Dental Hygienist or Orthodontist in the Past 12 Months, Ages 12 and Over, Canada and Provinces, 2007 and 2009 [https://www.cda-adc.ca/en/services/fact_sheets/contactwithdentist20072009.asp] 185. MSP Payment Schedule [http://www2.gov.bc.ca/gov/content/health/practitioner-professional-resources/msp/physicians/payment-schedules/msc-payment-schedule] 186. Statistics Canada: Consumer Price Index, Health and Personal Care, by Province (Catalogue no. 62-603-X). In. Edited by Canada S; 2018. 187. Suggested Fee Guide [https://www.bcdental.org/YourDentalHealth/YourDentalHealth.aspx?id=10240] 188. Ontario Health Insurance Program: Schedule of Benefits. In: Schedule of Benefits - Physician Services Under the Health Insurance Act. Edited by Ontario Ministry of Health and Long-Term Care. Toronto: Ontario Ministry of Health and Long-Term Care; 2017. 189. Canada S: Mortality, Summary List of Causes. In. Ottawa: Statistics Canada; 2008. 190. Karnon J: Alternative decision modelling techniques for the evaluation of health care technologies: Markov processes versus discrete event simulation. Health economics 2003, 12(10):837-848. 191. Simpson KN, Strassburger A, Jones WJ, Dietz B, Rajagopalan R: Comparison of Markov model and discrete-event simulation techniques for HIV. Pharmacoeconomics 2009, 27(2):159-165. 192. Noon KM, Montgomery SM, Adlard NE, Kroes MA: When does economic model type become a decisive factor in health technology appraisals? Learning from the expanding treatment options for relapsing-remitting multiple sclerosis. Journal of medical economics 2018, 21(10):983-992. 193. Standfield L, Comans T, Scuffham P: Markov modeling and discrete event simulation in health care: a systematic comparison. Int J Technol Assess Health Care 2014, 30(2):165-172. 194. Standfield LB, Comans TA, Scuffham PA: An empirical comparison of Markov cohort modeling and discrete event simulation in a capacity-constrained health care setting. The European journal of health economics : HEPAC : health economics in prevention and care 2017, 18(1):33-47. 195. Kansal A, Tafazzoli A, Leipold R, Sarda S: Comparison of a Markov Cohort Model and a Discrete-Event Simulation for Economic Analyses of Treatments for Multiple Sclerosis. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research 2014, 17(7):A398. 196. Jahn B, Rochau U, Kurzthaler C, Paulden M, Kluibenschadl M, Arvandi M, Kuhne F, Goehler A, Krahn MD, Siebert U: Lessons Learned from a Cross-Model Validation between a Discrete Event Simulation Model and a Cohort State-Transition Model for Personalized Breast Cancer Treatment. Medical decision making : an international journal of the Society for Medical Decision Making 2016, 36(3):375-390. 197. Degeling K, Franken MD, May AM, van Oijen MGH, Koopman M, Punt CJA, MJ IJ, Koffijberg H: Matching the model with the evidence: comparing discrete event simulation and state-transition modeling for time-to-event predictions in a cost-effectiveness analysis of treatment in metastatic colorectal cancer patients. Cancer epidemiology 2018, 57:60-67. 213     198. Levy DE, Klinger EV, Linder JA, Fleegler EW, Rigotti NA, Park ER, Haas JS: Cost-Effectiveness of a Health System-Based Smoking Cessation Program. Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco 2017, 19(12):1508-1515. 199. Purshouse RC, Brennan A, Rafia R, Latimer NR, Archer RJ, Angus CR, Preston LR, Meier PS: Modelling the cost-effectiveness of alcohol screening and brief interventions in primary care in England. Alcohol and alcoholism 2013, 48(2):180-188. 200. Rana M, Zapf A, Kuehle M, Gellrich NC, Eckardt AM: Clinical evaluation of an autofluorescence diagnostic device for oral cancer detection: a prospective randomized diagnostic study. European journal of cancer prevention : the official journal of the European Cancer Prevention Organisation 2012, 21(5):460-466. 201. Poh CF, Anderson DW, Durham JS, Chen J, Berean KW, MacAulay CE, Rosin MP: Fluorescence Visualization-Guided Surgery for Early-Stage Oral Cancer. JAMA otolaryngology-- head & neck surgery 2016, 142(3):209-216. 202. Paz-Ares L, Luft A, Vicente D, Tafreshi A, Gumus M, Mazieres J, Hermes B, Cay Senler F, Csoszi T, Fulop A et al: Pembrolizumab plus Chemotherapy for Squamous Non-Small-Cell Lung Cancer. N Engl J Med 2018, 379(21):2040-2051. 203. Georgieva M, da Silveira Nogueira Lima JP, Aguiar P, Jr., de Lima Lopes G, Jr., Haaland B: Cost-effectiveness of pembrolizumab as first-line therapy for advanced non-small cell lung cancer. Lung Cancer 2018, 124:248-254. 204. Immunize BC - HPV Vaccination [http://www.immunizebc.ca/diseases-vaccinations/hpv] 205. Hubbers CU, Akgul B: HPV and cancer of the oral cavity. Virulence 2015, 6(3):244-248. 206. Speight PM, Khurram SA, Kujan O: Oral potentially malignant disorders: risk of progression to malignancy. Oral surgery, oral medicine, oral pathology and oral radiology 2018, 125(6):612-627. 207. Raj AT, Patil S, Gupta AA, Rajkumar C, Awan KH: Reviewing the role of human papillomavirus in oral cancer using the Bradford Hill criteria of causation. Disease-a-month : DM 2018. 208. Dalla Torre D, Burtscher D, Edlinger M, Solder E, Widschwendter A, Rasse M, Puelacher W: Comparison of the prevalence of human papilloma virus infection in histopathologically confirmed premalignant oral lesions and healthy oral mucosa by brush smear detection. Oral surgery, oral medicine, oral pathology and oral radiology 2015, 119(3):333-339. 209. Lee LA, Huang CG, Tsao KC, Liao CT, Kang CJ, Chang KP, Huang SF, Chen IH, Fang TJ, Li HY et al: Human Papillomavirus Infections are Common and Predict Mortality in a Retrospective Cohort Study of Taiwanese Patients With Oral Cavity Cancer. Medicine 2015, 94(47):e2069. 210. Nulton TJ, Kim NK, DiNardo LJ, Morgan IM, Windle B: Patients with integrated HPV16 in head and neck cancer show poor survival. Oral Oncol 2018, 80:52-55. 211. Jansen L, Buttmann-Schweiger N, Listl S, Ressing M, Holleczek B, Katalinic A, Luttmann S, Kraywinkel K, Brenner H, Group GCSW: Differences in incidence and survival of oral cavity and pharyngeal cancers between Germany and the United States depend on the HPV-association of the cancer site. Oral Oncol 2018, 76:8-15. 212. Palve V, Bagwan J, Krishnan NM, Pareek M, Chandola U, Suresh A, Siddappa G, James BL, Kekatpure V, Kuriakose MA et al: Detection of High-Risk Human Papillomavirus in Oral Cavity 214     Squamous Cell Carcinoma Using Multiple Analytes and Their Role in Patient Survival. Journal of global oncology 2018(4):1-33. 213. Adnan Ali SM, Awan MS, Atif S, Ali N, Mirza Y: Correlation of human papillomavirus infection and clinical parameters with five-year survival in oral squamous cell carcinoma. The Journal of laryngology and otology 2018, 132(7):628-635. 214. HPV Vaccines [http://convio.cancer.ca/site/PageServer?pagename=SSL_ON_TM_CervicalCancer_Vaccine#.XA7lkjhKhpg] 215. Ogilvie GS, Cook DA, Taylor DL, Rank C, Kan L, Yu A, Mei W, van Niekerk DJ, Coldman AJ, Krajden M: Population-based evaluation of type-specific HPV prevalence among women in British Columbia, Canada. Vaccine 2013, 31(7):1129-1133. 216. Chaturvedi AK, Graubard BI, Broutian T, Pickard RK, Tong ZY, Xiao W, Kahle L, Gillison ML: NHANES 2009-2012 Findings: Association of Sexual Behaviors with Higher Prevalence of Oral Oncogenic Human Papillomavirus Infections in U.S. Men. Cancer Res 2015, 75(12):2468-2477. 217. Gillison ML, Broutian T, Pickard RK, Tong ZY, Xiao W, Kahle L, Graubard BI, Chaturvedi AK: Prevalence of oral HPV infection in the United States, 2009-2010. JAMA : the journal of the American Medical Association 2012, 307(7):693-703. 218. Bird Y, Obidiya O, Mahmood R, Nwankwo C, Moraros J: Human Papillomavirus Vaccination Uptake in Canada: A Systematic Review and Meta-analysis. International journal of preventive medicine 2017, 8:71. 219. Simul8 Online Store [https://www.simul8.com/store/] 220. Ogilvie GS, van Niekerk D, Krajden M, Smith LW, Cook D, Gondara L, Ceballos K, Quinlan D, Lee M, Martin RE et al: Effect of Screening With Primary Cervical HPV Testing vs Cytology Testing on High-grade Cervical Intraepithelial Neoplasia at 48 Months: The HPV FOCAL Randomized Clinical Trial. JAMA : the journal of the American Medical Association 2018, 320(1):43-52. 221. Decision Analysis in R for Technologies in Health [http://darthworkgroup.com/] 222. Krijkamp EM, Alarid-Escudero F, Enns EA, Jalal HJ, Hunink MGM, Pechlivanoglou P: Microsimulation Modeling for Health Decision Sciences Using R: A Tutorial. Medical decision making : an international journal of the Society for Medical Decision Making 2018, 38(3):400-422. 223. Cromwell I, van der Hoek K, Melosky B, Peacock S: Erlotinib or docetaxel for second-line treatment of non-small cell lung cancer: a real-world cost-effectiveness analysis. Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer 2011, 6(12):2097-2103. 224. Cromwell I, van der Hoek K, Malfair Taylor SC, Melosky B, Peacock S: Erlotinib or best supportive care for third-line treatment of advanced non-small-cell lung cancer: a real-world cost-effectiveness analysis. Lung Cancer 2012, 76(3):472-477. 225. Cromwell I, Ferreira Z, Smith L, van der Hoek K, Ogilvie G, Coldman A, Peacock SJ: Cost and resource utilization in cervical cancer management: a real-world retrospective cost analysis. Curr Oncol 2016, 23(Suppl 1):S14-22. 215     226. Canadian Institute for Health Information, - Your Health System [https://yourhealthsystem.cihi.ca/hsp/indepth?lang=en#/theme/C9001/2/]     216     Appendices Appendix A – Literature Review Search Strategy EMBASE 1. Oral Cancer 2. Head and Neck 3. Cancer 4. Cost effectiveness 5. Cost utility 6. Cost 7. Health Economic 8. Economic 9. 1 AND 4; 1 AND 5, 1 AND 6, 1 AND 7, 1 AND 8 10. 2 AND 3 AND 4; 2 AND 3 AND 5; 2 AND 3 AND 6; 2 AND 3 AND 7; 2 AND 3 AND 8 PubMed 1. (“Head and Neck Cancer”) AND (Costs and Cost Analysis[MeSH] OR Cost Effectiveness OR Cost Utility OR Economic) 2. (Oral Cancer[MeSH]) AND (Costs and Cost Analysis[MeSH] OR Cost Effectiveness OR Cost Utility OR Economic)   217     Appendix B – Names and specializations of expert stakeholder group 1.       Scott Durham – Otolaryngologist H&N Surgeon, VGH 2.       Priscilla Walsh – Periodontist 3.       Cheryl Ho – Medical Oncologist 4.       Brad Forster – Oral surgeon 5.       Denis Nagy – Dental surgeon 6.       Jonn Wu – Radiation Oncologist 7.       Samson Ng – Oral medicine specialist 8.       Denise Laronde – Dental Hygienist and oral medicine researcher 218     Appendix C – Initial Model Structure    219        220        221        222        223      Appendix D – Strucutral elements of a discrete event simulation (DES) The conceptual model for the WDMOC is comprised of 9 different structural elements, each of which describes a different function within the model’s code:  Origin Node: This is where all entities within the model begin  Entity Paths: These describe how entities may move between different elements. Dotted-line paths describe movement that happens across model components.  Decision Nodes: A process where one of multiple potential paths are possible for a given entity. The code samples an underlying probability of each path occurring, and compares that sampled value to a randomly-generated value from a uniform distribution to determine which one occurs.  Characteristic Nodes: A point at which an entity is assigned a new characteristic – a treatment flag (e.g., surgery only/surgery + RT/other/etc.), a demographic value (e.g., Male/Female, Ever/Never Smoker, etc.), or other information that governs its movement through subsequent parts of the model.  Resource Nodes: Similar to characteristic nodes, these are points at which entity resource utilization is applied (e.g., a medical appointment, a treatment, a health service, etc.).  Temporal Nodes: Describes the passage of time between model events. An entity ‘waits’ for a number of days before moving to the next node.  Destination Nodes: These describe an entity moving across different model components. They correspond to the dotted-line entity paths.  Terminal Nodes: A point at which an entity’s route through the model ends. Within the WDMOC, the terminal nodes signify death either from oral cancer or from another cause.  Subroutines: For simplicity’s sake, some complex groups of processes that lie along multiple potential paths are summarized as a subroutine.   224     Figure A4.3.1 – Examples of Structural Elements    225     Appendix E – Estimating time to progression from OPL Objective: To estimate the “OPL_prog” parameter for the model (time for OPL to progress to stage I cancer). These estimates must reflect patient age, sex, smoking status, and LOH risk score. Data from 289 participants in the Zhang et al study[61] underwent secondary analysis. Demographic (age, sex, smoking status) and LOH risk score was identified for each participant, as well as time to confirmed progression from premalignant to invasive disease. LOH risk score was categorized, as in the Zhang et al study, as “high”, “medium”, or “low” risk based on trial protocol definitions. Twelve (12) participants with an unclassified risk score were excluded from the analysis, resulting in a final cohort size of 277. Table A4.4.1 – Patient demographics Variable (N = 277)  Number SD/Percent Age at recruitment  58.7 12.0 Sex Male 152 58.5%  Female 354 41.5% Smoking History Never smoker 75 27.1%  Ever smoker 202 72.9% LOH risk score Low 100 36.1%  Medium 54 19.5%  High 123 44.4%  Survival Analysis  Median survival in the participant cohort was 4103 days, and mean survival was 3423.4 (SE: 98.59). The clear majority (84.5%) did not progress to cancer within the observation period of 226     the trial (i.e., were censored). Smoking status was significantly associated with time to progression, as was LOH risk score.     Multivariate linear regression was performed on time-to-progression values, assuming a Weibull distribution. Age, sex, and LOH Risk Score were included as covariates. Outputs from the regression analysis are presented in the table.   227     Table A4.4.2 – Coefficients from regression analysis Parameter   Estimate SE 95% Confidence Limits Pr > ChiSq Intercept (β0)   9.296 0.744 7.838 – 11.6878 <.0001 Sigma (σ)   0.801 0.106 0.618 – 1.4115   Age   0.0013 0.010 -0.018 – 0.0208 0.8966 Sex Male Ref.       F -0.080 0.257 -0.585 – 0.424 0.7547 Smoking Never Ref.       Ever 0.593 0.267 0.0695 – 1.116 0.0264 LOH Risk Score Low Ref.       Medium -0.868 0.449 -2.015 – -0.483 0.0533  High -1.249 0.391 -1.749 – 0.0123 0.0014    228     Appendix F – Parameter estimation from the Retrospective Oral Cancer Cohort (ROCC) Objective: To estimate the “ClinHist” parameters for the model (generally, time to recurrence and death). These estimates must reflect patient treatment type, stage, and other individual characteristics a model entity may potentially have.  ClinHist_timeRecurrence  ClinHist_timeDeadOfDisease Description of patient population A retrospective cohort of 886 patients diagnosed and treated with oral cancer was identified from a linkage between the BC Cancer Agency’s Oral Biopsy service and the BC Cancer Registry. Members of the cohort, once identified, had their medical charts extracted to determine dates of treatment, recurrence, regional failure, and death (from disease or from another cause). Demographic (age, sex, smoking history, etc.) and clinical characteristics (tumour site, stage, location, etc.) and types of treatment received by each cohort member were also identified and recorded from this chart review – a full list of variables is included in Appendix A. Each member of the cohort has an identified “date_init” and “date_last” which were considered equivalent to date of diagnosis and death (or censoring). Cohort members who do not have a “date_init” (n = 4) or whose “date_init” and “date_last” were identical (i.e., they contribute zero days to the analysis; n = 26) were removed from the analysis, leaving a final cohort of 856 patients. Patient demographics 229     Demographic characteristics of the sample are presented in Table A4.5.1. Former and current smokers were considered “ever smokers” and were combined into the same category.  Table A4.5.1 – Demographic characteristics of ROCC Variable (N = 856)  Mean SD/Percent Age at first treatment  65.6 14.0 Sex Male 501 58.5%  Female 355 41.5% Smoking History Never smoker 203 23.7%  Ever smoker 587 68.6%  Unknown 66 7.7% Stage at diagnosis I 319 37.3%  II 300 35.0%  III 106 12.4%  IV 128 15.0%  Unknown 3 0.4%  Clinical characteristics Four clinical milestones were identified for each patient, where applicable: 1) date of initial treatment, 2) date of first recurrence, 3) date of second recurrence, and 4) date of death or censoring. Date of initial treatment was defined as the ‘date_init’ variable in the cohort. Date of recurrence was defined as either ‘LR_Date’ or ‘RR_Date’ (locoregional or regional recurrence, respectively), whichever date was the earlier of the two. Date of second recurrence was defined as either ‘LR2_Date’ or ‘RR2_Date’, whichever date was the earlier of the two. In cases where a cohort member had a local recurrence and a regional recurrence and a second recurrence (either local or regional; n = 7), the first two of those dates were taken to be the date of second recurrence. Date of death or censoring was defined as ‘date_last’ in the cohort datafile. 230     Overall survival (OS) was measured from date of initial treatment to date of death or censoring. Cohort members who died of oral cancer (N = 282) were considered to have experienced the ‘dead of disease’ event. Cohort members who did not die (N = 283), or who died of a cause unrelated to cancer (N = 291) were censored at ‘date_last’. Time to First Recurrence (TFR) was measured from the date of initial treatment to date of recurrence. Cohort members who did not experience a recurrence (N = 709) were censored at ‘date_last’. Time from First Recurrence to Death (TFRD) was measured from date of recurrence to ‘date_last’. Time to Second Recurrence (TSR) was measured from the date of first recurrence to the date of second recurrence. Only cohort members who had experienced a first recurrence (N = 147) were included in this analysis. Cohort members who did not experience a second recurrence (N = 124) were censored at ‘date_last’. Time from Second Recurrence to Death (TSRD) was measured from date of second recurrence to ‘date_last’. Median survival times are presented in Table A4.5.2. 231     Table A4.5.2 – Median Time-To-Event Values Parameter  Value Overall Survival (N = 856)   Median (days) 1052  Censored observations (%) 574 (67%) Time to First Recurrence (N = 856)   Median (days) 836.0  Censored observations (%) 709 (83%) Time from First Recurrence to Death (N = 147)   Median (days) 377  Censored observations (%) 67 (46%) Time to Second Recurrence (N = 147)   Median (days) 762  Censored observations (%) 124 (84%) Time from Second Recurrence to Death (N = 23)   Median (days) 228  Censored observations (%) 9 (39%)  Treatment characteristics Treatment type was identified at the time of initial treatment (‘prim_tx_type’) and at recurrence (‘LR_tx_type’ and ‘RR_tx_type’) and grouped into categories. For concurrent therapies (i.e., surgery with adjuvant RT), therapies that occurred within three months of the primary treatment in the absence of recurrence were considered to be given simultaneously. This information is presented in Table 3. Primary treatment type (i.e., the treatment approached used for the first presentation of the oral cancer) was collapsed into four categories: surgery alone, surgery with radiotherapy, other adjuvant treatment (including brachytherapy, chemotherapy, and chemoradiotherapy), and no treatment. Treatment at time of recurrence was collapsed into four categories: surgical management, non-232     surgical management (i.e., any curative treatment that does not include surgery), palliative care, and no treatment. Table A4.5.3 – Treatment Received by Cohort Parameter N  % Primary Treatment Type (N = 856)    Surgery Alone 406 47.43  Surgery + RT 125 14.60  Other Adjuvant Treatment 310 36.21  No Treatment 15 1.75 Recurrence Treatment Type (N = 147)    Managed Surgically 62 42.18  Managed Non-Surgically 48 32.65  Managed Palliatively 20 13.61  No Treatment 17 11.56 Had Second Recurrence 23 2.3  Statistical Regression Methods Survival times (in days) were fit to a Weibull regression model using the LIFEREG Procedure in SAS 9.4. A Weibull distribution was chosen as it allows for estimation of baseline hazard as well as proportional hazard, making it ideal for use in this type of analysis. Akaike Information Criterion (AIC) and Bayes’ Information Crierion (BIC) statistics suggested that a Weibull approximation fit the cohort data similarly to a Gamma or Lognormal approximation.  Age, sex, and smoking status were chosen as demographic covariates in the regression analysis. Stage at diagnosis was collapsed into three categories: stage I, stage II, and advanced stage (III and IV) – cancers 233     of unknown stage were excluded from the analysis. Treatment type, both primary and recurrence, were also included in the model. It is worthwhile to note that the results of this regression should not be taken to suggest that a given type of treatment is more effective than another. This analysis assumes that each patient was recommended the most appropriate type of treatment for their individual clinical presentation, and that they enjoyed the maximum survival benefit from whatever treatment they received. Survival Analysis Overall Survival Age, stage at diagnosis, and treatment type were all significantly associated with overall survival.   234     Figure A4.5.1 – Overall Survival in Days  Total Failed Censored Percent Censored 856 282 574 67.06 235     Table A4.5.4: Overall Survival Regression Coefficients Parameter   Estimate SE 95% Confidence Limits Pr > ChiSq Intercept   12.1050 0.5397 11.0473 – 13.1628 <.0001 Sigma   1.4102 0.0720 1.2760 – 1.5585   Age   -0.0364 0.0074 -0.0509 – -0.0219 <.0001 Sex Male Ref.       F 0.2028 0.1874 -0.1645 – 0.5701 0.2792 Smoking Never Ref.       Ever -0.2358 0.2160 -0.6591 – 0.1875 0.2750 Stage I Ref.       II -0.5226 0.2471 -1.0070 – -0.0382 0.0345  Adv -1.3667 0.2584 -1.8733 – -0.8602 <.0001 Treatment Type Surgery Alone Ref.       Surgery + RT -0.9073 0.2673 -1.4312 – -0.3834 0.0007  Other -0.9437 0.2268 -1.3882 – -0.4992 <.0001  No Treatment -3.0385 0.5296 -4.0765 – -2.0005 <.0001  236     Time to First Recurrence (TFR) Treatment type was related to recurrence time, with patients requiring RT experiencing recurrence at an earlier average date than other patients. Figure A4.5.2 – Time to First Recurrence in Days  Total Failed Censored Percent Censored 856 147 709 82.83 237     Table A4.5.5: Time to First Recurrence Regression Coefficients Parameter   Estimate SE 95% Confidence Limits Pr > ChiSq Intercept   10.3039 0.6232 9.0824 – 11.5254 <.0001 Scale   1.3754 0.0961 1.1994 – 1.5773   Age   -0.0087 0.0092 -0.0267 – 0.0093 0.3425 Sex M Ref.       F 0.2774 0.2476 -0.2079 – 0.7626 0.2626 Smoking Never Ref.      Smoking Ever - 0.1331 0.2843 -0.6902 – 0.4241 0.6397 Stage T1 Ref.       T2 -0.3536 0.2810 -0.9043 – 0.1971 0.2082  adv -0.3102 0.3396 -0.9759 – 0.3555 0.3611 Treatment Type Surgery Alone 0.0000       Surgery + RT -0.9004 0.3145 -1.5167 – -0.2840 0.0042  Other -0.4612 0.2938 -1.0371 – 0.1147 0.1165  No Treatment 28.3841 63509.82 -124449 – 124505 0.9996    238     Time from Recurrence to Death Figure A4.5.3 – Time From Recurrence to Death in Days  Total Failed Censored Percent Censored 147 67 80 54.42  239     Table A4.5.6 – Time From Recurrence to Death Regression Coefficients Parameter   Estimate SE 95% Confidence Limits Pr > ChiSq Intercept   9.2803 0.9231 7.4712 – 11.0895 <.0001 Scale   1.2543 0.1201 1.0398 – 1.5132   Age   -0.0123 0.0134 -0.0386 – 0.0140 0.3588 Sex M 0.0000       F -0.5217 0.3190 -1.1468 – 0.1035 0.1019 Smoking Never Ref       Ever -0.5840 0.3900 1.3485  -0.1804 0.1343 Recurrence Treatment Surgical 0.0000       Non-Surgical -1.0934 0.4074 -1.8918 – -0.2949 0.0073  Palliative -2.7061 0.4510 -3.5901 – -1.8222 <.0001  No Treatment -1.7166 0.5271 -2.7496 – -0.6836 0.0011    240     Time to Second Recurrence Time to second recurrence was not statistically associated with any of the parameters in the model. Figure A4.5.4 – Time to Second Recurrence in Days  Total Failed Censored Percent Censored 147 23 124 84.35  241     Table A4.5.7: Time To Second Recurrence Regression Coefficients Parameter   Estimate SE 95% Confidence Limits Pr > ChiSq Intercept   10.2014 2.3781 5.5405 – 14.8623 <.0001 Scale   1.9706 0.3723 1.3608 – 2.8537   Age   -0.0058 0.0356 -0.0756 – 0.0640 0.8697 Sex M Ref.       F 1.1526 0.9755 -0.7595 – 3.0646 0.2374 Smoking Never Ref.       Ever 0.1752 0.9612 -1.7088 – -2.0591 0.8554 Recurrence Treatment Surgical 0.0000       Non-Surgical 1.1131 1.0785 -1.0007 – 3.2269 0.3020  Palliative 2.4629 2.1279 -1.7077 – 6.6336 0.2471  No Treatment -0.3380 1.2823 -2.8514 – 2.1753 0.7921   242     Time from Second Recurrence to Death Women in the sample had shorter survival time than men, but this may be due to the small number of observations rather than being a true effect. Figure A4.5.4 – Time From Second Recurrence to Death in Days  Total Failed Censored Percent Censored 23 9 14 60.87    243     Table A4.5.8: Time From Second Recurrence to Death Regression Coefficients Parameter   Estimate SE 95% Confidence Limits Pr > ChiSq Intercept   5.9464 2.4339 1.1761 – 10.7167 0.0146 Scale   1.2485 0.3036 0.7751 – 2.0110   Age   0.0355 0.0409 -0.0447 – 0.1156 0.3855 Sex M Ref.       F -2.4923 0.9150 -4.2857 – -0.6989 0.0065   244     Appendix G – Time-to-event processing in the WDMOC Time-to-event (TTE) sampling is accomplished through one of two processes. 1. Sampled from a transition probability Some TTE parameters are encoded as a mean (x) and a standard deviation (y) of a transition probability. In those cases, a random sample is drawn from an exponential distribution based on those values using the method of moments. The Python code for this process is as follows:  # Step 1: generate random estimate of the transition probability x = self.mean y = self.se bdist_alpha = x*((x*(1-x)/y**2) - 1) bdist_beta = (1-x)*(x/y**2*(1-x) - 1) est_tp = numpy.random.beta(bdist_alpha, bdist_beta)  # Step 2: generate random draw from exponential distribution lmbd = -(math.log(1.0 - est_tp)/365.0) beta = 1/lmbd samp_value = numpy.random.exponential(beta) return samp_value  2. Sampled from a parametric (Weibull) distribution Other TTE parameters are derived from a generalized linear model (GLM) regression with a Weibull link function. The output of the GLM is an intercept  0 with coefficients ( 1   n). Each coefficient corresponds with values (X1 … Xn) stored on the entity (the entity’s age, sex, smoking status, treatment type, etc.). The GLM also outputs a Sigma value ( ) that corresponds to the shape of the distribution.  Sampling values from the empirical distribution described by the GLM function (based on an entity’s individual characteristics) can be accomplished through the use of the function numpy.random.weibull(self.shape)*self.scale where: Shape = 1/  Scale = exp( 0 + X1 1    Xn n) Both of these methods produce a TTE estimate t, expressed in days. 245     Competing Risks The GLM-based analysis allowed the WDMOC to account for competing risks – the risk of death vs. recurrence following primary treatment for cancer, and the similar risk of death vs. recurrence following recurrence treatment. This was accomplished in three steps: 1. Estimate the time to the next event through parametric sampling  The time to the next event (t) is estimated using the parametric sampling process described above. The distribution coefficients describing intermediate event (in this case, recurrence) are used.  2. Estimate the probability of each risk occurring at that time  The probability of an event occurring at a given time is described by the cumulative distribution function (CDF) of the parametric distribution for that event, based on the shape and scale of the parametric function and the value of t from Step 1: probevent = 1 - numpy.math.exp(-(t/self.scale)**self.shape) Probabilities for each event (i.e., recurrence or death following treatment) are generated in this way. 3. Determine which event occurs, based on relative probability Once probabilities for competing events have been estimated, the relative probability is compared to a randomly-sampled value from a uniform distribution: event_prob = prob2/prob1 self.probEst = random.random() if probEst < event_prob: event_type = 2 elif probEst >= event_prob: event_type = 1 If the relative probability is less than or equal to the randomly-sampled probability (i.e., event type = 1), then the intermediate event (recurrence) occurs at time t. Otherwise, the competing event (death) occurs at time t. The code governing the parametric sampling and competing risk functions follows:  246     Glb_GenTime – A Function to Generate Parametric Sampling of TTE Values class GenTime:     def __init__(self, estimates, regcoeffs):         self._estimates = estimates         self._regcoeffs = regcoeffs      def readVal(self, entity, param):                # Is the parameter being estimated contained within the Excel sheet?         if param in self._regcoeffs:                 # The sum of the coefficients starts at zero             coeff = 0                   # For a given factor of a parameter within the Excel sheet             for factor in self._regcoeffs[param].keys():                                        # Identify the intercept                 if factor == 'Intercept':                                 Intercept = self._regcoeffs[param]['Intercept']['mean']                                      # Identify the shape parameter from the output                                  elif factor == 'Sigma':                                     Sigma = self._regcoeffs[param]['Sigma']['mean']                                  # Identify values for all other coefficients                 elif factor in entity.__dict__.keys():                        value = getattr(entity, factor)                                          if self._regcoeffs[param][factor]['vartype'] == 2:                         coeff += self._regcoeffs[param][factor]['mean'] * value                     else:                         coeff += self._regcoeffs[param][factor][value]['mean']                                          # Produce an estimate of time from the regression             mu = Intercept + coeff                      shape = 1/Sigma             scale = math.exp(mu)                          self.mu = mu             self.shape = shape             self.scale = scale                  # Randomly sample an event time for the entity from a Weibull distribution                 def estTime(self):                          estimate_time = numpy.random.weibull(self.shape)*self.scale         return estimate_time 247          # Estimate the probability (CDF) of being alive at a given time     def estProb(self, time):         estimate_probability = numpy.math.exp(-(time/self.scale)**self.shape)         return estimate_probability  Glb_CompTime – A Function to Evaluate Competing Risks class CompTime:     def __init__(self, estimates, regcoeffs):         self._estimates = estimates         self._regcoeffs = regcoeffs         self.probEst = random.random()              def Process(self, entity, tte1, tte2):         # Draw two survival functions for the entity         event1 = GenTime(self._estimates, self._regcoeffs)         event2 = GenTime(self._estimates, self._regcoeffs)         # Any event         event1.readVal(entity, str(tte1))         # Competing event         event2.readVal(entity, str(tte2))                  # 1 - Draw random value for time to next event                 event_time = event1.estTime()         # 2 - Estimate probability of that value occurring within first event         prob1 = 1 - event1.estProb(event_time)         # 3 - Estimate probability of that value occurring within second event         prob2 = 1 - event2.estProb(event_time)         # 4 - Calculate relative probability that event is the competing event         event_prob = prob2/prob1         # 5 - Evaluate relative probability against random probability         if self.probEst < event_prob:             event_type = 2         elif self.probEst >= event_prob:             event_type = 1                  return (event_time, event_type)   248     Appendix H – Cost derivation from retrospective oral cancer cohort (ROCC) Objective: to estimate the treatment costs of people with oral cancer, based on a linked analysis of medical records.  These estimates will reflect stage and treatment type, as used by the model.  Treatment – Stage I (Surgery, Surgery + RT, Other)  Treatment – Stage II (Surgery, Surgery + RT, Other)  Treatment – Advanced (Surgery, Surgery + RT, Other)  Treatment – Recurrence (Surgery, Nonsurgery, Palliative, No Treatment)  Treatment – Second Recurrence  Treatment – End of Life Description of patient population This analysis used the same retrospective cohort described in the Clinical History analysis (Retrospective Oral Cancer Cohort; ROCC). Briefly, 856 previously-diagnosed oral cancer patients in British Columbia were identified through the BC Cancer Registry, and their medical charts were reviewed by a researcher in order to identify critical clinical events (dates of diagnosis, recurrence, death). Table A4.7.1 – Demographic characteristics of cohort Variable (N = 856)  Mean SD/Percent Age at first treatment  65.6 14.0 Sex Male 501 58.5%  Female 355 41.5% Smoking History Never smoker 203 23.7%  Ever smoker 587 68.6%  Unknown 66 7.7% Stage at diagnosis I 319 37.3%  II 300 35.0%  III 106 12.4%  IV 128 15.0%  Unknown 3 0.4% 249      Description of linkage strategy Anonymized records from the ROCC were sent by the principal investigator to the British Columbia Cancer Agency’s (BCCA) Cancer Agency Information System (CAIS). The CAIS data stewards added data fields describing chemotherapy and radiotherapy resource utilization, as well as records of appointments and tests within the BCCA.  The linked dataset was then sent to PopData BC, where it was linked to resource utilization data from the following sources: 1. Canadian Institute of Health Information (CIHI) Discharge Abstract Database (DAD). This dataset contains information about resource utilization at hospitals, through the Resource Intensity Weight variables (inpatient, ambulatory RIWs). 2. BC’s Medical Services Plan (MSP) billings. This dataset contains information about all provincially-insured fee-for-service (FFS) services received by members of the cohort. Importantly this dataset does not include services provided within the BCCA. 3. BC’s PharmaNet billings. This dataset contains information about all provincially-insured medication prescriptions and their unit costs. This does not include chemotherapy or other supportive drugs (anti-emetics, analgesics, etc.) received within the BCCA. The resulting linked dataset contained dates and resource records for the full cohort, from diagnosis to death or censoring. Unit Costing – BCCA CAIS Care at the BCCA is not delivered on a fee-for-service basis. As a result, it is not straightforward to derive unit costs for these services. An approach was taken that mirrors and improves upon efforts from 250     previously-published exercises using CAIS data sources[223-225]. BCCA CAIS data contains three basic types of records: radiotherapy delivery, chemotherapy delivery, and appointments and tests. Radiotherapy delivery The radiotherapy records from CAIS were released summarized by course. Each course record included start and end dates, treatment intention, and the total number of fractions given. Double counting can easily occur from these records, as boost fractions (i.e., fractions delivered in addition to the guideline-prescribed number) may be coded separately from the guideline-prescribed fractions. To avoid this double counting, records were sorted using the following sequential steps: 1. Duplicate records with the same start/end date and number of fractions were removed 2. Duplicate records with the same start/end date and a lower number of fractions were removed 3. Duplicate records with the same end date were removed Start date, rather than end date, was chosen as the resource utilization date. This choice was made to ensure that the maximum number of resources were included, rather than excluding those that sat on the ‘border’ of eligibility (i.e., those courses of RT starting before three months but ending after three months). A unit cost of $325 was applied to each fraction, based on an internal BCCA costing exercise wherein the annual operating costs of the Vancouver Cancer Centre (VCC) were divided by the number of fractions of 251     RT delivered in that year. This is an admittedly crude measure, but is more up-to-date than amounts available in the published literature. One of the biggest weaknesses of using this unit costing approach is that Head and Neck cancers tend to be more complex to treat than solid tumours in tissues like the breast. Moulds are needed to steady the head, and more complex pre-treatment planning is needed to ensure that sensitive tissues in the head and neck are not irradiated by mistake. This requires more resources and staff hours than average. As a result, it is likely that the unit costing approach used in this exercise underestimates the true cost of treatment. Chemotherapy delivery The data sharing agreement between the Ministry of Health and the BC Cancer Agency meant that we could not link direct drug costs from CAIS to the PopData file. Dispensed units are not reported consistently between hospitals, so it was not possible to use list prices as unit costs. The drug costs are provincially-negotiated, and as such are accessible to BCCA staff but cannot leave the Agency. In order to address this issue, the following steps were taken: 1. A file was obtained with agency ID, drug name, amount dispensed, protocol code, and billed amount for each systemic therapy record from each member of the ROCC cohort.  2. Prescriptions with protocol codes that were for other tumour types (i.e., non head-neck) were removed. Drugs with no protocol code were left in. 3. The total cost for each drug, for each cohort member, was calculated. 4. The 7 most frequently-prescribed drugs (representing 60% of the total number of prescriptions) were reviewed. An average per-prescription cost was calculated for each, in order to find a per-252     prescription cost that would most closely match the actual amount paid for each cohort member:  Cisplatin: averages were calculated by time period (before 2006, 2006-2010, after 2010)  Fluouracil: the mean of all prescriptions was used  Etoposide: means were calculated for records with a Head & Neck protocol code, and one for those without a protocol code  Methotrexate: the mean of all prescriptions was used  Carboplatin: means were calculated for records with a Head & Neck protocol code, and one for those without a protocol code  Leucovorin: the mean of all prescriptions was used  Docetaxel: before 2005, the mean was used, after 2005 the average of head-neck codes was used. The mean of all prescriptions was used for all remaining drugs. 5. The calculated per-prescription amounts were applied to the records in the linked PopData dataset, producing a unit cost for each record. This process produced unit costs that are less accurate than what would be achievable if the billed amounts were available, but more accurate than what would have been produced if we had used list prices. It should also be noted that the costs for chemotherapy do not include associated staff costs (nursing, administration, etc.). Appointments and tests Because appointments and tests at the BCCA are not delivered on a fee-for-service basis, there are no billing codes associated with the records retrieved from CAIS. These records contain the date of the 253     service provided and codes generally describing the type, nature, and location of the service. The following steps were taken to apply unit costs to these records: 1. Records describing the following sources were excluded from the analysis, being deemed either redundant with other cost estimate sources (RT unit costs, CIHI DAD), or outside the scope of the analysis:  Hospitalizations  Memos, notes, referrals  Radiotherapy records (including mould room)  Chemotherapy dressings/equipment  Resources related to second primary cancers 2. Remaining records were classified according to type 3. Unit costs were estimated for each record type using MSP reimbursement rates for similar services. If MSP reimbursement was unavailable, literature sources and/or expert opinion were consulted. A full list of unit types, counts, and costs is presented in a Table at the end of this Appendix. Unit Costing – CIHI DAD All hospitalization records were eligible to be included in the analysis. Total RIW for each hospitalization record was multiplied by the cost per weighted case (CWC) for the hospital associated with that record, by hospital number. Hospital numbers were determined through a list published by CIHI[226]. Any record corresponding to a hospital number that was not included in the list was assigned the provincial average CWC. The product of the CWC and the RIW was taken to be the unit cost of the record. 254     Unit Costing – MSP Billings All billings were eligible to be included in the analysis. Billed amounts (‘paidamt’) for each service are included within the datafile. The amount paid for each record was taken to be the unit cost of the record. Unit Costing – PharmaNet Billings All billings were eligible to be included in the analysis. Billed amounts (‘paidamt’) for each prescription are included within the datafile. The amount paid for each record was taken to be the unit cost of the record.   255     Results Costs within three months from the event of interest are presented. Cohort members with fewer than 3 months of costs were excluded from the analysis, meaning that this figures under-represent those who are diagnosed with disease so severe that they experience another event (recurrence, death) within 3 months. Table A4.7.2 – 90-Day Costs of primary treatment Value  Mean SD 95% CI Stage I       Surgery $9,225 $10,676 $7,838 – $10,612  Surgery + RT $27,895 $17,805 $18,407 – $37,383  Other $7,889 $9,132 $5,237 – $10,540 Stage II       Surgery $19,454 $25,046 $15,056 – $23,853  Surgery + RT $26,687 $19,000 $20,911 – $32,464  Other $10,280 $8,660 $8,595 – $11,964 Advanced Stage       Surgery $37,800 $37,958 $26,528 – $49,072  Surgery + RT $38,083 $23,453 $29,326 – $46,841  Other $15,000 $16,088 $12,015 – $17,986    256     Table A4.7.3 – Cost breakdown of primary treatment Value Mean SD 95% CI Total cost $16,142 $20,210 $14,702 – $17,582 BCCA Appointments $385 $653 $341 – $429  6.53% 15.90% 5.40% – 7.66% BCCA Tests $14 $59 $10 – $19  0.18% 1.01% 0.10% – 0.25% Chemotherapy $6 $41 $4 – $9  0.16% 2.62% -0.02% – 0.35% Hospitalizations $9,659 $17,472 $8,472 – $10,846  47.81% 36.99% 45.18% – 50.45% MSP $2,112 $2,837 $1,920 – $2,305  24.10% 26.85% 22.18% – 26.01% Pharmanet $208 $411 $180 – $236  4.10% 10.84% 3.32% – 4.87% Radiotherapy $2,288 $4,098 $2,009 – $2,566  17.13% 31.14% 14.91% – 19.34%  Table A4.7.4 – 90-day Costs of Recurrence Treatment Value Mean SD 95% CI Management Including Surgery $29,262 $42,731 $13,306 – $45,218 Management not Including Surgery $17,067 $20,315 $10,293 – $23,840 Palliative $20,779 $31,106 $4,203 – $37,354 No Treatment $11,120 $8,518 $543 – $21,696    257     Table A4.7.5 – Cost breakdown of recurrence treatment Value Mean SD 95% CI Total cost $21,561 $31,384 $14,911 – $28,211 BCCA Appointments $56 $308 $35 – $77  8.48% 15.17% 5.26% – 11.69% BCCA Tests $6 $39 $3 – $8  0.63% 2.07% 0.19% – 1.07% Chemotherapy $9 $205 -$5 – $23  1.50% 8.02% -0.20% – 3.20% Hospitalizations $1,786 $10,904 $1,046 – $2,527  51.79% 37.56% 43.83% – 59.75% MSP $264 $1,261 $179 – $350  26.19% 26.62% 20.55% – 31.83% Pharmanet $42 $218 $27 – $57  5.04% 9.54% 3.02% – 7.06% Radiotherapy $109 $900 $48 – $170  6.37% 18.46% 2.46% – 10.28%  Second Recurrence Because of small sample size, proportional costs could not be reported. Mean treatment cost was $16,617 (SD: $27,428; 95% CI: $1428 – $31,806).   258     A4.7.6 – 90-Day costs at End of Life Value Mean SD 95% CI Total cost $17,930 $21,977 $16,104 – $19,756 BCCA Appointments $211 $525 $176 – $247  3.85% 8.40% 3.16% – 4.55% BCCA Tests $18 $67 $14 – $23  0.28% 1.05% 0.19% – 0.37% Chemotherapy $22 $264 $4 – $40  0.34% 2.91% 0.10% – 0.58% Hospitalizations $9,890 $18,478 $8,635 – $11,145  58.41% 38.97% 55.17% – 61.65% MSP $802 $1,308 $713 – $891  19.48% 27.16% 17.22% – 21.74% Pharmanet $556 $1,301 $468 – $645  13.82% 22.35% 11.96% – 15.68% Radiotherapy $504 $2,023 $366 – $641  3.82% 13.14% 2.72% – 4.91%  259     Table A4.7.7 – Unit costs of BCCA Appointments Unit Cost Count Source Complex patient visit 89.71 1 MSP P33527 CT Scan 98.99 73 MSP 08693 Dental new patient 256.18 133 MSP 03770 Dental visit 59.81 5686 MSP 03785 Dermatology visit 52.69 2 MSP 00210 Doppler ultrasound 59.5 6 MSP 08664 ECG 100.7 7 MSP 08638 Fine needle biopsy 53.41 8 MSP S00844 Gastroscopy 50.75 2 MSP 10742 Genetic counselling 506 5 Personal communication – Gillian Mitchell Hospital visit (orthopaedic) 30.35 199 MSP 51008 Hygiene 150 207 Personal communication – Denise Laronde Mammogram 101 1 MSP 08610 Nutrition 100 3437 Personal communication – Angie Bowman Occupational medicine visit 50.81 6 MSP 33907 Oncologist visit 80.67 5657 MSP 33512 Oncology consult 169.06 864 MSP 33510 Oncology follow-up 39.14 199 MSP 33508 Orthopedist new patient 104.17 31 MSP 51010 Patient + Family Counselling 50 470 Personal communication – Gina Mackenzie Psychiatric consultation 126.17 53 MSP 00625 Psychiatrist (new pt) 237.95 16 MSP 00610 Psychology 50 412 Personal communication – Gina Mackenzie Pulmonary function test 81.41 11 MSP S00945 Radiography 49.2 160 MSP 08602 Social Work 50 34 Personal communication – Gina Mackenzie Speech Path 41 27 Personal communication – Cindy Reynolds Symptom management 169.05 94 MSP 33510 Telephone call 24.05 1462 MSP G10003 Thoracentesis 99.83 1 MSP S00749 Tube Nutrition 337.5 62 Personal communication – Angie Bowman Ultrasound (abdomen) 107.55 17 MSP 08648 Ultrasound (pelvic) 107.53 2 MSP 08653 Voiding study 19.27 1 MSP S00732 260     Appendix I – Comparison of Cycle-based vs. Discrete Time in decision analysis models Table A5.1 – Hypothetical Markov model Cycle State A State B State C State D 1 1000 0 0 0 2 500 500 0 0 3 250 650 100 0 4 125 710 75 90 … … … … … N 0 0 0 1000  The above table illustrates a four-state Markov model with initial cohort size N = 1000, and health states A, B, C, and D where D is an absorbing (death) state. Transition probabilities are as follows:  State A to State B = 50%  State B to State C = 20%  State C to State D = 90% No other health state transitions are possible in this simplified example. If each cycle represents six months of time, this means that it is structurally impossible for anyone to experience State C before two cycles (one year) have elapsed, and impossible for anyone to experience mortality before 3 cycles have.   261     Figure A5.2 – Hypothetical Individual Sampling model   The above figure illustrates an individual sampling model that replicates the model logic of the hypothetical Markov model above. The dashed lines represent time-to-event values that are sampled from the same values as the transition probability values.  Because transition probabilities are sampled in probabilistic analysis from mean values with statistical uncertainty around them, it is possible that the sampled value for time from A to B may be shorter than six months in some cases. It is, however, not possible for events to occur in less than six months in the Markov model. This effect compounds through the model’s process. This effect compounds In a pathway where sampled values of times to event B, C, and D are all shorter than the mean value for a given model run. Over multiple iterations of this process, time-to-event processing will result in systematically shorter overall times to event than cycle-based processing will. As a result, more events are likely to occur within the same time period. 262     Appendix J – Structural comparison of WDMOC to conventional (Markov) model This appendix will describe an exercise in which the WDMOC was made to more closely resemble the structure of the Markov model described in Chapters 3 and 5.  Methods Two models, a conventional Markov model and a Whole Disease Model of Oral Cancer (WDMOC) were used to estimate the cost-effectiveness of managing OPL according to an LOH ‘risk score’ derived from a hypothetical genomic assay. Each model had two arms: an “Assay Informed” arm in which the assay was used to guide OPL surveillance appointments and an “Assay Naïve” arm in which no assay was used and OPL surveillance was conducted according to standard practice (once every 6 months). Patients in both models start with a diagnosed OPL that may progress to invasive cancer according to its LOH risk score. In the “assay informed” arm of each model, patients are managed according to their risk score, people with “high risk” lesions receiving immediate surgical treatment with an attenuated surveillance schedule for those with “medium” and “low” risk lesions (3 years and 5 years, respectively). Progression to invasive cancer, which may be either a high-grade lesion (HGL) or an invasive squamous cell carcinoma (SCC), may be detected at a follow-up appointment. Once detected, HGL and SCC are treated surgically, with the possibility of adjuvant external-beam radiotherapy (XRT) and/or neck dissection for SCC. Following treatment, patients are followed up regularly by an oncologist for 5 years or until they experience a recurrence. Recurrence, if it occurs, is managed until the patient dies of disease. All patients may die of causes other than oral cancer at any point during the simulation. 263     Markov Model A conventional ‘piecewise’ Markov model was constructed using the R programming language. The model simulated cohort of patients moving through different simulated health states. The model was composed of six health states: low-, intermediate-, and high-risk LGDs; locally controlled oral cancer, representing an invasive oral cancer that has undergone successful treatment and does not show signs of progression; persistent/metastatic disease, representing cancers that are refractory to curative treatment; and remission, representing a locally controlled cancer that has shown no signs of disease return for at least 5 years.  Simulated patients in a Markov model move between health states at fixed time intervals known as ‘cycles’, each cycle representing six months of time. Resource utilization was applied at each cycle, for each health state. Health state utility values, representing a patient’s well-being and anchored between 1 (full health) and 0 (a state equivalent to death), was applied in the same way. The model was evaluated for a hypothetical cohort of 1,000 patients, and run multiple (10,000) times. Values for each run were estimated from a set of model parameters, with each run’s parameters drawn from an underlying statistical distribution. This probabilistic Monte Carlo simulation allows the estimation of the impact that parameter uncertainty has on cost-effectiveness. Incremental cost-effectiveness analysis was conducted by comparing costs and outcomes for populations simulated within the model’s two arms.   264     Whole Disease Model The WDMOC was adapted to match the structure of the Markov model in the ways described in Table A5.1. Table A5.1 – Structural Adaptations made to the WDMOC Model Component Element Original Update Natural History Symptomatic detection of cancer Cancers may be detected symptomatically, which is influenced by stage All cancers are considered Stage I and are detected at follow-up appointments Screening/Asymptomatic No modifications were made to this component N/A N/A OPL Management Frequency of surveillance appointments Once every 6 months for all entities Assay Informed: Low-risk OPL – 5 years Medium-risk OPL – 3 years High-risk – immediate resection Assay Naïve: Once every 6 months for all entities OPL Management OPL prevalence Prevalence according to age, sex distribution All entities start with OPL 265     Incident Cancer Treatment Treatment modality Entities treated according to demographics and stage, using surgery, surgery + RT, or ‘other’ All entities treated surgically, some receiving RT, some receiving neck dissection Incident Cancer Treatment Post-treatment survival Entity survival is based on demographic characteristics and treatment received Survival is drawn from the same parametric function for all entities Incident Cancer Treatment Cancer stage Stage at diagnosis divided into HGL, I, II, advanced Stage at diagnosis is HGL or Invasive (assumed to be Stage I) Incident Cancer Treatment Post-treatment events First and second recurrences are possible, recurrence treatment can be curative Recurrence treatment is not curative, second recurrences not possible Post-treatment Followup Followup appointment interval Frequency declines over time until full remission @ 5y Every 6 months until full remission @ 5y  Costs and outcomes in both the conventional Markov model and the adapted WDMOC were discounted for future time preference at an annual rate of 1.5%. Costs for both were expressed in 2017 Canadian Dollars, adjusted for inflation using the Consumer Price Index for health care. A ten year time horizon – 20 cycles in the Markov model, 3650 days in the WDMOC) was chosen for the analysis, representing a point at which all entities are either dead (from cancer or another cause), in 5-year remission from a detected cancer, or unlikely for their OPL to progress to invasive cancer. 266     Data Sources and Parameter Estimates All data sources and parameter estimates for both models are as they are presented in Chapter 3 and Table A5.2: Table A5.2 – Parameter Values in both models Description Value (SE) Distribution Source Probabilities Proportion of people who are "low risk" 0.47 0.03 Dirichlet Zhang (2012) Proportion of people who are "intermediate risk" 0.43 0.03 Dirichlet Zhang (2012) Proportion of people who are "high risk" 0.10 0.02 Dirichlet Zhang (2012) Probability of moving up to a new risk category 0   Assumption Rate of developing cancer in "low risk" group 0.031 over  5 years 0.023 Beta Zhang (2012) Rate of developing cancer in "intermediate risk" group 0.163 over  5 years 0.036 Beta Zhang (2012) Rate of developing cancer in "high risk" group 0.631 over  5 years 0.090 Beta Zhang (2012) Probability of cancer being SCC 0.68 0.01 Beta SCC cohorta Probability of HGL treated with surgery 1.0 -  Precancer cohortb Probability of HGL surgery requiring neck dissection 0 -  Assumption Probability of second surgery for HGL 0.02 0.01 Beta Precancer cohortb Probability of locally controlled HGL after treatment 1 - Beta Precancer cohortb Probability of SCC treated with only surgery 0.71 0.02 Beta Precancer cohortb Probability of SCC surgery requiring neck dissection 0.371 0.0317   Rate of SCC recurrence in first year 0.2 0.03 Beta Ganly (2013) Rate of SCC recurrence after first year 0.1 over 4 years 0.03 Beta Ganly (2013) Oral cancer mortality rate 0.681 over 5 years 0.04 Beta Mucke (2009) Relative rate of cancer death, age <55 1.0 - Ref.  Relative rate of cancer death, age 55 – 64 1.5 0.43 Normal Rogers (2009) Relative rate of cancer death, age 65 – 74 1.6 0.46 Normal Rogers (2009) Relative rate of cancer death, age 75+ 3.4 0.97 Normal Rogers (2009)   267     Description Value (SE) Distribution Source Costs Cost of genetic assay 500 -  Assumption Direct cost of medical appointments 250 -  MSP code 03770 Indirect cost of medical appointments 67.15 306.22 Gamma COOLS Trialc Cost of biopsy 150 -   Cost of resection 1889.27 -  MSP – fee code 02279 Cost of dissection 1231.05 -  MSP – fee code 02470 Cost of course of RT 325.5   BCCA Costing exercise Number of courses of RT 27.5 0.7 Normal BCCA Clinical guidelines Cost of course of chemotherapy 4478 750 Gamma Hannouf (2012) Cost of asymptomatic follow-up 75 -  MSP, BCCA Consultancy fee Cost of 1st 12 months with metastatic disease 11,639 16,719 Gamma Speight (2006) Cost of subsequent 12 months with metastatic disease 2150 8940 Gamma Speight (2006) Costs after 5 years of cancer-free survival 0 -  Assumption Utilities Utility for pre-cancerous lesion 0.92 0.18 Beta Downer (1997) Utility for locally controlled cancer 0.88 0.20 Beta Downer (1997) Utility for persistent/metastatic disease 0.68 0.33 Beta Downer (1997) Utility for disease in remission 1.0 -  Assumption a – a retrospective cohort of 864 people diagnosed with squamous cell oral carcinoma in British Columbia between January, 2000 and September, 2009. b – a cohort of 148 patients who had developed oral cancer from monitored precancerous lesions in British Columbia between February, 2004 and November, 2011. c – intervention-blinded survey data from an ongoing clinical observation of 400 people newly-diagnosed HGL or SCC MSP – British Columbia Medical Services Plan; COOLS – Canadian Optically-guided Oral Lesions Surgical Trial; BCCA – British Columbia Cancer Agency  Incremental cost-effectiveness ratios (ICERs) were calculated for both models, in which the incremental mean cost experienced by patients in each arm was compared to incremental quality-adjusted life years (QALYs). Probabilistic analysis was expressed on the cost-effectiveness plane and by generating cost-effectiveness acceptability curves (CEACs), which consider the proportion of sampled ICERs that lie below a threshold of society’s willingness to pay for an additional QALY (a value denoted as  ). A CEAC illustrates the probability that an intervention is cost-effective for a varying value of  .   268     Results Cost-Effectiveness Summary – Markov Model The Markov model’s estimates of cost, survival, and quality-adjusted survival are presented in Table A5.3. The experimental protocol (Use of the assay and early treatment of high-grade lesions) resulted in cost savings (∆C = -$3701; 95% CR -$7,714 – -$2271) with increased quality-adjusted survival (∆E = 0.470 QALY; 95% CR 0.324 – 0.657). Use of the assay dominated (i.e., cost less, and was more effective than) standard practice. Table A5.3 – Cost-effectiveness results from Markov Model Arm Estimate Mean 95% CR Assay Informed       Cost $3,241 2,549  4,623  LYG 9.23 7.33  9.71  QALY 8.51 3.70  9.60 Assay Naive       Cost $6,897 5,049  11,802  LYG 9.18 7.28  9.68  QALY 8.04 3.17  9.17 Incremental       Cost -$3701 -7,360  -2,304  LYG 0.048 0.015  0.112  QALY 0.469 0.323  0.656  Cost/LYG -$76,942     Cost/QALY -$7,800       Cost-Effectiveness Summary – Whole Disease Model The Whole Disease Model’s estimates of cost, survival, and quality-adjusted survival are presented in Table A5.4. The experimental protocol also resulted in cost savings (∆C = -$2318; 95% CR -$2423 – -269     $1925) with increased quality-adjusted survival (∆E = 0.172 QALY; 95% CI 0.103 – 0.244). Use of the assay dominated standard practice.  Table A5.4 – Cost-effectiveness results from Whole Disease Model Arm Estimate Mean 95% CR Assay Informed       Cost $2,580 461  4,721  LYG 8.14 3.54  9.28  QALY 7.52 2.37  9.28 Assay Naive       Cost $4,916 2,299  7,532  LYG 8.01 3.20  9.28  QALY 7.34 1.98  9.28 Incremental       Cost -$2358 -2,423  -2,211  LYG 0.120 0.057  0.181  QALY 0.171 0.102  0.244  Cost/LYG -$19,580     Cost/QALY -$13,735     Secondary Outcomes – Comparison Between Models A number of secondary measures were calculated for each model, and are presented in Table A5.5. Outputs were similar between the two different models in terms of cancer incidence and type, but were noticeably different in the rate of recurrences and deaths from cancer. This difference occurs despite the two models having identical parameter inputs.    270     Table A5.5 – Comparison of Secondary Model Outputs Output Markov Model Whole Disease Model Rate (per 1,000) Assay Informed Assay Naïve Assay Informed Assay Naïve Cancers 149.0 229.5 138.8 218.7 SCC 102.8  158.3 91.7 146.9 Recurrences 19.6 36.7 28.7 53.0 Deaths from cancer 12.6 24.3 26.3 49.4 Deaths from other cause 121.9 121.1 230.1 229.2  Based on these results, the use of a genomic assay to provide risk-guided management to a patient with an OPL is expected to reduce health care system costs and improve quality-adjusted survival. This finding holds irrespective of which modeling technique is used. The model results are not identical – notable differences exist between some outputs. Potential explanations for these differences will be discussed later in this appendix. Discussion Differences in secondary outcomes  The difference in recurrence and disease mortality rates owes in part to the computational structure of a Markov model, in which transitions are evaluated at regular intervals rather than continuously. This results in a higher probability that cancer recurrences will happen earlier – it is possible for an entity to develop recurrence and die within six months in a discrete model, whereas that would require at least 2 cycles in a Markov model with a cycle length of >6 months. As a result these kinds of downstream events will systematically occur later in cycle-based time than in discrete time.  271     The model also shows a noticeable difference in the number of people dying from causes other than cancer. While this is likely also influenced by the issue of discrete versus cycle-based time, there is an additional explanation to consider. The parameter estimates for “time to death from natural causes” were necessarily drawn from different statistical measures. Markov models require event risk (in this case, death from causes other than cancer) to be expressed as a per-cycle transition probability, which were computed from age-specific mortality rates published by Statistics Canada. Discrete event models require event risk to be expressed as a time to an event, which was computed based on a measure of age at death published by Statistics Canada. Although these numbers are drawn from the same underlying population, they do result in different distributions as they are measures of separate but related concepts (risk of death at a given age vs. age at which a person is expected to die).  Cancer incidence is similar between the two models, as are the relative values of the secondary measures (see Table A5.5). Recurrence and mortality rates are directly related to the absolute value of both costs and outcome measures (people who die earlier do not generate additional costs beyond their deaths), and will exert influence on incremental cost and survival values as well. It is reasonable, therefore, to conclude that the observed differences in incremental costs, outcomes, cost-effectiveness, and secondary outcome measures are likely due to the aforementioned effects of how the two models handle time, and how the two models estimate risk of death from a non-cancer cause. Structural Limitations In order to compare the two distinct approaches, it was necessary to make a number of meaningful adaptations to the structure of the original WDMOC. There were several simplifying assumptions made 272     in the Conventional model that the WDMOC was designed to improve upon. (see Table A5.1). These adaptations were based on simplifying assumptions that were necessary for the Markov modeling approach to function. The Whole Disease Model, as discussed in Chapter 4, considers many more details that are likely to have meaningful impact on the model’s results when considering this research question. It is important to consider the effect these structural limitations may have on cost-effectiveness, particularly when considering the effect of upstream factors like sex and smoking rates, and downstream factors like the relationship between stage at detection and post-recurrence survival. Despite the issues described above, both the Markov model and the WDMOC found that risk-guided OPL management dominated usual care. Structural adaptations were made for the two models to be reasonably comparable, and the WDMOC’s estimates are more likely to be accurate by dint of the way discrete event models handle time. Further work is needed to understand the relationship between these adaptations and the model’s estimates of cost-effectiveness. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0379922/manifest

Comment

Related Items