Science, Faculty of
Statistics, Department of
DSpace
UBCV
Lobay, Rachel
Master of Science - MSc
University of British Columbia
In multiple sclerosis (MS) clinical trials, any elevation of a patientâs baseline Expanded Disability Status Scale (EDSS) score at a follow-up visit that is sustained or increased at the next visit is known as confirmed disability worsening (CDW). It is not clear how to handle patients who have an initial EDSS elevation (known as initial disability worsening (IDW)) immediately preceding dropout or at the last visit of the trial as they lack a confirmatory visit. Our objective was to assess the impact of different approaches to impute those missing CDW statuses on the estimated treatment effect in a time to first CDW analysis. Guided by a MS clinical trial dataset of 718 patients, we constructed a larger simulated dataset where all patients had EDSS scores from baseline to the last study visit. From this complete dataset, we obtained several samples of 300 patients and imposed dropouts on them by using probabilities that depended on whether IDW occurred at the immediately preceding visit. Then, 3 investigations were conducted to ascertain in what circumstances and to what extent different strategies for the imputation of the CDW statuses led to clearly different treatment effect results. First, we investigated the impact of magnifying the probabilities of dropout alone. In our next 2 investigations, we magnified the dropouts in conjunction with magnifying the treatment effect by modifying the EDSS data simulation procedure. In the first, we used Markov chain models to simulate the EDSS data, while in the second we incorporated more patient-specific information in the simulation through use of mixed-effects proportional odds models. We found that magnifying the dropouts alone had little impact on the estimated treatment effects. In contrast, under extreme treatment and dropout factors, the imputation strategies that accounted for treatment group led to estimated treatment effects that were clearly stronger than those from the other strategies in both the Markov chain and proportional odds model-based investigations. Since this was only observed under extreme circumstances, the imputation strategies are not expected to lead to clearly different estimated treatment effects in a time to first CDW analysis of a similar MS clinical trial.
Impact of Informative Censoring on Estimated Treatment Effects for Confirmed Disability Worsening in Multiple Sclerosis Clinical Trials by Rachel Lobay B.Sc., Thompson Rivers University, 2018 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Statistics) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) August 2020 © Rachel Lobay, 2020 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled: Impact of Informative Censoring on Estimated Treatment Effects for Confirmed Disability Worsening in Multiple Sclerosis Clinical Trials submitted by Rachel Lobay in partial fulfillment of the requirements for the degree of Master of Science in Statistics. Examining Committee: John Petkau, Statistics Supervisor Lang Wu, Statistics Supervisory Committee Member iii Abstract In multiple sclerosis (MS) clinical trials, any elevation of a patientâs baseline Expanded Disability Status Scale (EDSS) score at a follow-up visit that is sustained or increased at the next visit is known as confirmed disability worsening (CDW). It is not clear how to handle patients who have an initial EDSS elevation (known as initial disability worsening (IDW)) immediately preceding dropout or at the last visit of the trial as they lack a confirmatory visit. Our objective was to assess the impact of different approaches to impute those missing CDW statuses on the estimated treatment effect in a time to first CDW analysis. Guided by a MS clinical trial dataset of 718 patients, we constructed a larger simulated dataset where all patients had EDSS scores from baseline to the last study visit. From this complete dataset, we obtained several samples of 300 patients and imposed dropouts on them by using probabilities that depended on whether IDW occurred at the immediately preceding visit. Then, 3 investigations were conducted to ascertain in what circumstances and to what extent different strategies for the imputation of the CDW statuses led to clearly different treatment effect results. First, we investigated the impact of magnifying the probabilities of dropout alone. In our next 2 investigations, we magnified the dropouts in conjunction with magnifying the treatment effect by modifying the EDSS data simulation procedure. In the first, we used Markov chain models to simulate the EDSS data, while in the second we incorporated more patient-specific information in the simulation through use of mixed-effects proportional odds models. We found that magnifying the dropouts alone had little impact on the estimated treatment effects. In contrast, under extreme treatment and dropout factors, the imputation strategies that iv accounted for treatment group led to estimated treatment effects that were clearly stronger than those from the other strategies in both the Markov chain and proportional odds model-based investigations. Since this was only observed under extreme circumstances, the imputation strategies are not expected to lead to clearly different estimated treatment effects in a time to first CDW analysis of a similar MS clinical trial. v Lay Summary In multiple sclerosis clinical trials, confirmed disability worsening (CDW) occurs when there is an elevation in the Expanded Disability Status Scale (EDSS) score from the patientâs baseline score and that elevation is sustained or increased at the next visit. It is unclear what statistical approach is best to handle patients who have an elevated EDSS score before dropout or at the last visit of the trial as they lack a confirmatory visit. Our objective was to fill-in those missing CDW statuses using different imputation strategies and to assess their impact on the estimated treatment effect in a time to first CDW analysis. We examined whether those strategies led to clearly different estimated treatment effects and whether such differences emerged under realistic magnifications of the treatment effect and the dropout rates. Our findings have direct implications for how time to first CDW analyses should be conducted in multiple sclerosis clinical trials. vi Preface This thesis is original and unpublished work by the author, Rachel Lobay, under the supervision of Professor John Petkau. The author conducted all simulations and statistical analysis in this thesis, using a dataset that was provided by the supervisor. vii Table of Contents Abstract ......................................................................................................................................... iii Lay Summary .................................................................................................................................v Preface ........................................................................................................................................... vi Table of Contents ........................................................................................................................ vii List of Tables ................................................................................................................................ xi List of Figures ............................................................................................................................. xiii Acknowledgements .................................................................................................................. xviii Dedication ................................................................................................................................... xix Chapter 1: Introduction ................................................................................................................1 1.1 Types of MS .................................................................................................................... 2 1.2 The Expanded Disability Status Scale (EDSS) ............................................................... 4 1.3 The phases of clinical trials ............................................................................................. 4 1.4 Confirmed disability worsening ...................................................................................... 6 1.5 Overview of this thesis .................................................................................................... 8 1.6 Main steps of this project ................................................................................................ 8 Chapter 2: Preliminary Analysis of the Schering Dataset .......................................................10 2.1 Introduction to the Schering dataset ............................................................................. 10 2.2 Collapsing the EDSS scores into states ........................................................................ 11 2.3 Nearly monotone missingness in the analysis dataset and dropouts ............................. 12 2.4 Initial disability worsening ........................................................................................... 13 2.5 Confirmed disability worsening .................................................................................... 15 viii 2.6 Logistic regression model for the presence of CDW .................................................... 17 2.7 Cox survival model for time to first CDW ................................................................... 19 Chapter 3: Generation of the Complete Simulated Dataset ....................................................21 3.1 Treatment group assignment ......................................................................................... 21 3.2 Baseline EDSS state generation .................................................................................... 21 3.3 Generation of follow-up EDSS states ........................................................................... 22 3.4 Generation of the 39 month CDW statuses .................................................................. 25 3.5 IDW and CDW instances in the complete simulated dataset ....................................... 25 3.6 The treatment effect hazard ratio and CDW rates in the complete simulated dataset .. 27 Chapter 4: Differing Dropouts Based on Presence of IDW .....................................................29 4.1 A dropout procedure that depends on presence of IDW ............................................... 29 4.2 Distributions of the count of dropouts by visit ............................................................. 30 4.3 Numbers of dropouts across the dropout factors by group ........................................... 36 4.4 The counts of patients requiring CDW status imputation ............................................. 38 4.5 The counts of patients requiring CDW status imputation by group ............................. 42 4.6 Imputation strategies used in the simulation study ....................................................... 48 4.7 Obtaining the treatment effect hazard ratios ................................................................. 51 4.8 Overall results for the average treatment effect hazard ratios ...................................... 52 4.9 Key results for the base set dropout probabilities and the dropout factor of 100 ........ 56 4.10 Key results from the average CDW rates ..................................................................... 58 4.11 Similar performances across the imputation strategies ................................................. 59 4.12 Confusion matrices to assess correct classification ...................................................... 60 4.13 Summary ....................................................................................................................... 62 ix Chapter 5: Differing Treatment Effects Based on Transition Matrices .................................64 5.1 Generation of follow-up EDSS states ........................................................................... 64 5.2 Treatment effects for the complete simulated datasets ................................................. 65 5.3 The counts of patients requiring CDW status imputation ............................................. 67 5.4 The counts of patients requiring CDW status imputation by group ............................. 69 5.5 Overall results for the average treatment effect hazard ratios ...................................... 71 5.6 Key results based on the sample hazard ratios .............................................................. 76 5.7 Key results from the average CDW rates ..................................................................... 81 5.8 Similar performances across the imputation strategies ................................................. 82 5.9 Confusion matrices to assess correct classification ...................................................... 83 5.10 Summary ....................................................................................................................... 86 Chapter 6: Differing Treatment Effects Based on a Proportional Odds Model ....................89 6.1 Introducing the proportional odds model ...................................................................... 89 6.2 The mixed-effects proportional odds model ................................................................. 90 6.3 The response variable and the potential covariates ....................................................... 91 6.4 Implementation of the proportional odds model in R ................................................... 92 6.5 Determining the predictors in the proportional odds model ......................................... 93 6.6 Assessing the proportional odds assumption ................................................................ 95 6.7 Fitting proportional odds models with the treatment by time interaction term ............ 99 6.8 Generation of the complete simulated datasets ........................................................... 101 6.9 The counts of patients requiring CDW status imputation ........................................... 103 6.10 Overall results for the average treatment effect hazard ratios .................................... 109 6.11 Key results for the most extreme model-based factors ............................................... 112 x 6.12 Key results from the average CDW rates ................................................................... 118 6.13 Similar performances across the imputation strategies ............................................... 119 6.14 Confusion matrices to assess correct classification .................................................... 120 6.15 Summary ..................................................................................................................... 123 Chapter 7: Conclusion ...............................................................................................................125 Bibliography ...............................................................................................................................127 xi List of Tables Table 2.1 Patient EDSS scores by visit from the analysis dataset. ............................................... 11 Table 2.2 Patient EDSS states by visit from the analysis dataset. ................................................ 12 Table 2.3 Dropouts by group and by visit in the analysis dataset. ................................................ 13 Table 2.4 All IDW instances in the analysis dataset by visit. ....................................................... 14 Table 2.5 Dropouts immediately following IDW in the analysis dataset. .................................... 15 Table 2.6 IDW instances with unavailable confirmatory visits and no prior CDW instances. .... 15 Table 2.7 First CDW instances in the analysis dataset by visit. ................................................... 16 Table 2.8 All CDW instances in the analysis dataset by visit. ..................................................... 16 Table 2.9 Potential covariates to include in logistic regression for the presence of CDW. .......... 17 Table 3.1 Distribution of the baseline EDSS states by treatment group from the analysis dataset........................................................................................................................................................ 21 Table 3.2 Placebo group transition matrix of counts from baseline to 3 months. ........................ 22 Table 3.3 Treatment group transition matrix of counts from baseline to 3 months. .................... 22 Table 3.4 Placebo group transition matrix of counts from 3 to 6 months. ................................... 22 Table 3.5 Treatment group transition matrix of counts from 3 to 6 months. ............................... 23 Table 3.6 Placebo group transition matrix of counts from 33 to 36 months. ............................... 23 Table 3.7 Treatment group transition matrix of counts from 33 to 36 months. ........................... 23 Table 3.8 Aggregated transition matrix for the placebo patients from the analysis dataset. ........ 24 Table 3.9 Aggregated transition matrix for the treatment patients from the analysis dataset. ..... 24 Table 3.10 All IDW instances in the simulated dataset by visit. .................................................. 25 Table 3.11 First CDW instances in the simulated dataset by visit. ............................................... 26 xii Table 3.12 All CDW instances in the simulated dataset by visit. ................................................. 27 Table 4.1 Empirical probabilities of dropout given IDW at the preceding visit. .......................... 29 Table 4.2 Empirical probabilities of dropout given no IDW at the preceding visit. ..................... 30 Table 4.3 Treatment effect HRs and CDW rates averaged over the 100 samples under all dropout factors. ........................................................................................................................................... 55 Table 5.1 Treatment effect HR [95% CI] from the complete datasets for the 3 matrix-based scenarios. ....................................................................................................................................... 66 Table 5.2 Treatment effect HRs [MC SE] and CDW rates from the base and the 3 treatment scenarios with , = 3 in conjunction with the base set of dropout probabilities and the dropout factor of . = 100. ....................................................................................................................... 75 Table 6.1 Empirical cumulative logits at 3 months for the groups from the analysis dataset. ..... 95 Table 6.2 Treatment effect HRs and 95% confidence intervals for the complete simulated datasets. ....................................................................................................................................... 102 Table 6.3 Treatment effect HRs [95% CI] from the complete dataset and the averaged HRs [MC SE] from the 100 samples after the imposition of the most extreme dropout factor of . = 100...................................................................................................................................................... 112 Table 6.4 Treatment effect HRs and CDW rates under the base and the treatment factor of 30 in conjunction with the base set of dropouts and the dropout factor of . = 100. ......................... 113 xiii List of Figures Figure 1.1 RRMS where pre-relapse functionality is completely recovered (left) or partially recovered (right). ............................................................................................................................. 2 Figure 1.2 SMPS with a relatively steady progression over time (left) or with a few relapses and remissions in the relatively steady progression over time (right). .................................................. 3 Figure 1.3 Primary-progressive multiple sclerosis where there are periods of stabilization and relapse. ............................................................................................................................................ 3 Figure 1.4 The EDSS scores of a patient with 2 instances of CDW. ............................................. 7 Figure 2.1 Final Cox survival model that resulted from stepwise variable selection. .................. 20 Figure 3.1 Kaplan-Meier curves for first CDW instance by treatment group with the log-rank p-value. ............................................................................................................................................. 28 Figure 4.1 Dropouts in the placebo group by visit under the base set of dropout probabilities. .. 31 Figure 4.2 Dropouts in the placebo group by visit under the dropout factors of . = 1.5, 3.9, 7.4, and 10. .......................................................................................................................................... 32 Figure 4.3 Dropouts in the placebo group by visit under the dropout factors of . = 15, 25, 50, and 100. ........................................................................................................................................ 33 Figure 4.4 Dropouts in the treatment group by visit under the base set of dropout probabilities. 34 Figure 4.5 Dropouts in the treatment group by visit under the dropout factors of . = 1.5, 3.9, 7.4, and 10. .................................................................................................................... 35 Figure 4.6 Dropouts in the treatment group by visit under the dropout factors of . = 15, 25, 50, and 100. .................................................................................................................... 36 xiv Figure 4.7 Placebo group dropouts across all visits in the 100 samples under each dropout factor........................................................................................................................................................ 37 Figure 4.8 Treatment group dropouts across all visits in the 100 samples under each dropout factor. ............................................................................................................................................ 38 Figure 4.9 Patients requiring CDW status imputation in the 100 samples under the base set of dropout probabilities. .................................................................................................................... 40 Figure 4.10 Patients requiring CDW status imputation in the 100 samples under the dropout factors of . = 1.5, 3.9, 7.4, and 10. ............................................................................................ 41 Figure 4.11 Patients requiring CDW status imputation in the 100 samples under the dropout factors of . = 15, 25, 50, and 100. ............................................................................................ 42 Figure 4.12 Placebo patients requiring CDW status imputation in the 100 samples under the base set of dropout probabilities. .......................................................................................................... 43 Figure 4.13 Placebo patients requiring CDW status imputation in the 100 samples under the dropout factors of . = 1.5, 3.9, 7.4, and 10. .............................................................................. 44 Figure 4.14 Placebo patients requiring CDW status imputation in the 100 samples under the dropout factors of . = 15, 25, 50, and 100. .............................................................................. 45 Figure 4.15 Treatment patients requiring CDW status imputation in the 100 samples under the base set of dropout probabilities. .................................................................................................. 46 Figure 4.16 Treatment patients requiring CDW status imputation in the 100 samples under the dropout factors of . = 1.5, 3.9, 7.4, and 10. .............................................................................. 47 Figure 4.17 Treatment patients requiring CDW status imputation in the 100 samples under the dropout factors of . = 15, 25, 50, and 100. .............................................................................. 48 Figure 4.18 Treat. eff. HRs across the 100 samples under the base set of drop. prob. ................ 57 xv Figure 4.19 Treat. eff. HRs across the 100 samples under . = 100. ......................................... 57 Figure 4.20 Proportions of correctly classified CDW and no CDW patients across the 100 samples under the base set of dropout probabilities (left) and under the dropout factor of . = 100 (right). .......................................................................................................................... 62 Figure 5.1 Patients requiring CDW status imputation in the 100 samples under the treatment scenarios 1, 2, and 3 with , = 3 and the dropout factor of . = 100. ...................................... 68 Figure 5.2 Placebo patients requiring CDW status imputation in the 100 samples under the treatment scenarios 1, 2, and 3 with , = 3 and the dropout factor of . = 100. ...................... 70 Figure 5.3 Treatment patients requiring CDW status imputation in the 100 samples under the treatment scenarios 1, 2, and 3 with , = 3 and the dropout factor of . = 100. ...................... 71 Figure 5.4 First treat. scen. treat. eff. HRs across the samples under , = 3 and the base set of drop. prob. ..................................................................................................................................... 78 Figure 5.5 First treat. scen. treat. eff. HRs across the samples under , = 3 and . = 100. ...... 78 Figure 5.6 Second treat. scen. treat. eff. HRs across the samples under , = 3 and the base set of drop. prob. ..................................................................................................................................... 79 Figure 5.7 Second treat. scen. treat. eff. HRs across the samples under , = 3 and . = 100. ... 79 Figure 5.8 Third treat. scen. treat. eff. HRs across the samples under , = 3 and the base set of drop. prob. ..................................................................................................................................... 80 Figure 5.9 Third treat. scen. treat. eff. HRs across the samples under , = 3 and . = 100. ..... 80 Figure 5.10 Proportions of correctly classified CDW and no CDW patients across the 100 samples from the first treatment scenario with , = 3 and the base set of dropout probabilities (left) and the dropout factor of . = 100 (right). ......................................................................... 84 xvi Figure 5.11 Proportions of correctly classified CDW and no CDW patients across the 100 samples from the second treatment scenario with , = 3 and the base set of dropout probabilities (left) and the dropout factor of . = 100 (right). ......................................................................... 85 Figure 5.12 Proportions of correctly classified CDW and no CDW patients across the 100 samples from the third treatment scenario with , = 3 and the base set of dropout probabilities (left) and the dropout factor of . = 100 (right). ......................................................................... 86 Figure 6.1 The proportionality assumption for the treatment predictor at 3, 6, 9, 12, 15, and 18 months. .......................................................................................................................................... 96 Figure 6.2 The proportionality assumption for the treatment predictor at 21, 24, 27, 30, 33, 36 months. .......................................................................................................................................... 97 Figure 6.3 The proportionality assumption for the treatment, baseline EDSS state, sex, and time predictors. ...................................................................................................................................... 98 Figure 6.4 Proportional odds model including both the treatment and treatment by time predictors. .................................................................................................................................... 100 Figure 6.5 Proportional odds model including only the treatment by time predictor. ................ 101 Figure 6.6 Patients requiring CDW status imputation in the 100 samples under the treatment factor of 1 and the dropout factor of . = 100. ......................................................................... 104 Figure 6.7 Patients requiring CDW status imputation in the 100 samples under the treatment factors of 5, 10, 20, and 30 (as indicated by 1, 2, 3, and 4) and the dropout factor of . = 100...................................................................................................................................................... 105 Figure 6.8 Placebo patients requiring CDW status imputation in the 100 samples under the treatment factor of 1 and the dropout factor of . = 100. ......................................................... 106 xvii Figure 6.9 Placebo patients requiring CDW status imputation in the 100 samples under the treatment factors of 5, 10, 20, and 30 (as indicated by 1, 2, 3, and 4) and the dropout factor of . = 100. .................................................................................................................................... 107 Figure 6.10 Treatment patients requiring CDW status imputation in the 100 samples under the treatment factor of 1 and the dropout factor of . = 100. ......................................................... 108 Figure 6.11 Treatment patients requiring CDW status imputation in the 100 samples under the treatment factors of 5, 10, 20, and 30 (as indicated by 1, 2, 3, and 4) and the dropout factor of . = 100. .................................................................................................................................... 109 Figure 6.12 Treat. eff. HRs across the samples under the base treat. factor and the base set of drop. prob. ................................................................................................................................... 116 Figure 6.13 Treat. eff. HRs across the samples under the base treat. factor and . = 100. ...... 116 Figure 6.14 Treat. eff. HRs across the samples under the treat. factor of 30 and the base set of drop. prob. ................................................................................................................................... 117 Figure 6.15 Treat. eff. HRs across the samples under the treat. factor of 30 and . = 100. .... 117 Figure 6.16 Proportions of correctly classified CDW and no CDW patients under the base treatment factor and the base set of dropout probabilities (left) and the dropout factor of . = 100 (right). ................................................................................................................................. 122 Figure 6.17 Proportions of correctly classified CDW and no CDW patients under the treatment factor of 30 and the base set of dropout probabilities (left) and the dropout factor of . = 100 (right). ......................................................................................................................................... 123 xviii Acknowledgements I am deeply grateful to my supervisor, Dr. John Petkau, whose passion and dedication to his work is inspiring. I have learned a lot about statistics and conducting statistical research from our discussions, where he posed insightful questions and gave thoughtful responses to my endless questions. His guidance, patience, and sense of humour have made the process of writing this thesis an excellent experience. It has been a privilege to work with him. xix Dedication To my parents, Natalie and Douglas Lobay, and my sister, Jessica Lobay, for their continued support and encouragement for me to keep chipping away at this thesis. 1 Chapter 1: Introduction Multiple sclerosis (MS) is a neurological disease in which the protective covering (myelin sheath) of nerve cells in the brain and spinal cord are damaged. The protective covering is essential for the transmission of nerve impulses, which are used to communicate between the nerve cells in the brain and spinal cord. Common symptoms of MS are balance or dizziness problems, tremors, tingling and numbness, vision problems (such as double vision), memory problems, extreme fatigue, and paralysis. Yet, the symptoms are unpredictable in that they are known to differ from person to person and to change within the same person. For example, some symptoms of MS can alleviate when a person is in remission. There is no cure for MS, only ways to manage the symptoms. A common way to medically manage symptoms is though pharmacological treatments. Non-medical ways are often mobility-oriented such as physiotherapy or exercise programs. The main goal of MS research is to cure the disease. There are some demographic patterns as to who typically develops the disease. According to the MS Society of Canada, many of the people who are diagnosed with MS are between 20 to 49 years old. Additionally, women are 3 times more likely to get the disease than men. This information and much of what follows in this introductory chapter can be found at the MS Society of Canada website: https://mssociety.ca, the MS International Federation website: https://www.msif.org, and the National MS Society website: http://www.nationalmssociety.org. 2 1.1 Types of MS There are 4 types of MS, as designated by the MS Society of Canada: 1. Clinically isolated syndrome (CIS) 2. Relapsing-remitting multiple sclerosis (RRMS) 3. Secondary-progressive multiple sclerosis (SPMS) 4. Primary-progressive multiple sclerosis (PPMS) CIS is the earliest form of the disease, in which there is a single incident where neurological symptoms manifest that are consistent with MS. When a person with such symptoms is examined by a medical professional, another anomaly in the brain or spinal cord is often uncovered. RRMS is characterized by unpredictable onsets or attacks of symptoms, which are known as relapses, and remissions between the onsets. During a relapse, new symptoms may emerge or existing symptoms may worsen. During a remission, the patient either completely recovers their pre-relapse functionality or partially recovers it as shown in Figure 1.1. Figure 1.1 RRMS where pre-relapse functionality is completely recovered (left) or partially recovered (right). In time, people who initially have RRMS usually progress to the secondary progressive form of the disease. SPMS is often regarded as a more advanced stage of the disease because there are 3 less relapses and remissions as the patientâs condition begins to steadily worsen through accruing disability as shown in Figure 1.2. Figure 1.2 SMPS with a relatively steady progression over time (left) or with a few relapses and remissions in the relatively steady progression over time (right). The fourth type of multiple sclerosis, PPMS, is defined by a relatively steady progression of the disease and disability accumulation from onset as is depicted in Figure 1.3. Whereas about 85% of patients are diagnosed with RRMS, about 15% are diagnosed with PPMS. People with PPMS have no definitive remissions although they may have brief periods of stabilization. As well, approximately 5% exhibit signs of occasional relapses from the onset of the disease. Figure 1.3 Primary-progressive multiple sclerosis where there are periods of stabilization and relapse. 4 1.2 The Expanded Disability Status Scale (EDSS) MS-related disability stems from the lack of relapse recovery and disease progression. For patients with RRMS, lack of relapse recovery is the primary cause of MS-related disability. On the other hand, for patients with SPMS, disability is caused by both incomplete relapse recovery and disease progression (Kappos, Polman, Pozzilli, Thompson, and Dahlke 1998). The Expanded Disability Status Scale (EDSS) was developed to assess of the degree of disability in MS patients by measuring neurological impairment (Kurtzke 1983). Neurological impairment is assessed via a neurological examination where different functional systems of the central nervous system are evaluated by a trained medical professional (typically a neurologist). The EDSS is used as part of the in-clinic assessments in both clinical trials and clinical practice. EDSS scores are ordinal and start from 0, which indicates that the neurology of the patient is normal, and increase by half-points up to 10, which indicates death caused by MS. 1.3 The phases of clinical trials Clinical trials are used by MS researchers to find improved and novel treatments. In these trials, clinical study measures such as EDSS are assessed for each patient at every visit from baseline until the end of the study. More generally, clinical trials investigate the impact of health interventions such as new treatments, technologies, surgeries or diagnostic tools on people. The primary purpose of clinical trials is to assess the efficacy and safety of a new health intervention. Provided that the pre-clinical laboratory results of a health intervention are promising, clinical trials may commence, in which the health intervention will be assessed in 4 phases. Each phase must be passed before moving on to the remaining phases. 5 The 4 distinct clinical trial phases are described below as defined by the MS International Federation. It is important to note that, regardless of the phase, if the safety and health of the participants is at risk, then testing of the health intervention will be stopped. Phase I A Phase I trial is a trial of 1 or more years that normally includes between 20 to 80 volunteers. These volunteers are usually comprised of some people who have MS and some who do not. The aim of a Phase I trial is to assess the safety of the health intervention in question. For example, when the health intervention is a drug, a range of different doses of the drug are tested on groups of patients. Animal studies usually help to determine the range of doses used in a Phase I trial. Phase II Once the health intervention is deemed to be safe in a Phase I trial, it can be tested in a Phase II trial. A Phase II trial is a 1 to 3 year trial that normally includes between 100 to 300 volunteers who have MS. The aims of this trial are to assess both safety and efficacy of the health intervention. If the health intervention is a new drug, then in a Phase II trial it will be compared to a placebo or a drug that is presently used for the same purpose. In most Phase II studies, the patients are not informed of whether they received the new drug or the comparator. If different doses of the new drug are used, the goal is to find the dosage that has the most positive effect on the patients. Phase III Once the health intervention is deemed both to be safe and to have a positive impact on MS patients by passing a Phase II trial, it can be put to the test in a trial that is conducted for 2 to 3 years on a larger group of about 1000 to 3000 MS patients. In a Phase III trial, when the health intervention is a drug, it is compared to a placebo or another drug that is used for the same 6 purpose. Patients are randomly assigned to the intervention and comparator groups. The aims of this trial are to assess efficacy and side-effects relative to the comparator, and to gather other information that will facilitate the safe use of the intervention. Such a large-scale trial may be conducted in multiple centres or sites in different countries. If a Phase III trial has demonstrated that the health intervention has a clear positive impact on the trial patients, then the researchers may endeavour to secure a marketing license for the drug from the responsible department of the government such as the US FDA. The marketing license allows the drug to be prescribed by medical professionals. Phase IV Once the health intervention has passed the first 3 phases and has a marketing license, a Phase IV trial may be conducted. The main aims of this trial are to monitor the safety and effectiveness of the health intervention while it is on the market and available to patients. 1.4 Confirmed disability worsening In MS clinical trials, patientsâ EDSS scores are often used to generate a new response variable called confirmed disability worsening (CDW) or confirmed progression. Typically, CDW is defined as a 1 or more unit increase from the baseline EDSS score that is sustained over 2 consecutive visits. The elevated EDSS score at the first visit indicates initial disability worsening (IDW), while the elevated EDSS score at the second visit indicates CDW. The second visit serves as a confirmatory visit so that it is more likely the initial increase in the EDSS score is not a mere fluctuation in the patientâs scores. We define time to (first) CDW as the time from the baseline visit until the visit of IDW that immediately precedes the patientâs first CDW. 7 To illustrate these concepts, Figure 1.4 depicts the EDSS scores of a patient at 3-monthly visits from baseline to 36 months. The baseline EDSS score of the patient is 3. IDW that does not lead to CDW is shown at the 6 month visit. Since the patientâs EDSS score is not at least 1 unit higher than the baseline score at the 9 month visit, the patient does not have CDW in that case. In contrast, at the 12 and 30 month visits, the patientâs EDSS scores are 1 score higher than the baseline EDSS score and both of these IDW instances have confirmatory visits where the patient also had EDSS scores that are at least 1 unit higher than baseline. Hence, both the 12 and 30 month cases represent instances of CDW. For this patient, the time to first CDW is 12 months. Figure 1.4 The EDSS scores of a patient with 1 instances of CDW. 8 1.5 Overview of this thesis This thesis is comprised of a project using a dataset from a Phase III SPMS clinical trial, which will be referred to as the Schering dataset. Based on the Schering dataset, we constructed a simulated dataset of 2000 patients in which patients had the potential to drop out. Some simulated patients have an IDW at the visit immediately preceding dropout or at the last study visit. For each such IDW instance, the corresponding CDW status is missing as no confirmation visit is available. This project focuses on the impact of such missing CDW statuses in the simulated dataset with dropouts imposed. In particular, we fill-in the missing CDW statuses of patients with unavailable confirmation visits (and no prior CDW instances) by using different imputation strategies. Our objective is to ascertain in what circumstances and to what extent the different imputation methods lead to clearly different estimated treatment effect hazard ratios in an analysis of time to first CDW. This project is inspired by the work of Riester, Xu, Elkins, and Lipkovich (2015). 1.6 Main steps of this project The project will be executed in 4 main steps, which will be described in more detail in the upcoming chapters: 1. A complete simulated EDSS dataset will be generated reflecting the stochastic structure of the Schering dataset. 2. Dropouts of increasing magnitude will be imposed on samples from the simulated dataset. The impact of several different imputation strategies on the estimated treatment effect will be assessed in those samples. 9 3. Three Markov chain models will be used to generate complete simulated EDSS datasets that reflect differing stochastic structures in the treatment group. Then, we will assess whether the estimated treatment effects are clearly different across the imputation strategies in samples of these simulated datasets for each set of dropouts. 4. A mixed-effects proportional odds model with different scaling factors to magnify the treatment effect will be used to generate complete simulated EDSS datasets. Then, we will assess whether the estimated treatment effects are clearly different across the imputation strategies as in the previous investigations. In addition to more detailed descriptions of the individual investigations, an overall summary and key conclusions will be provided for the project. 10 Chapter 2: Preliminary Analysis of the Schering Dataset 2.1 Introduction to the Schering dataset In the Schering dataset that was generated by the MS clinical trial, there were 718 patients: 358 patients in the placebo group and 360 patients in the treatment group. The division of the patients by sex was that 439 were female, while only 279 were male. Patient EDSS scores were recorded at scheduled visits of Screen, Day 1, and every 3 months thereafter until the last visit at 36 months. The first treatment administration was on Day 1. Since the patients were only required to have either a Screen or a Day 1 EDSS score, a clear definition was necessary for the baseline EDSS score. If a patient had an EDSS score at Day 1, that score was used as the baseline EDSS score; otherwise, the Screen EDSS score was used. We omitted 4 patients who only had baseline EDSS scores because of their lack of contribution to the treatment effect. We shall refer to this dataset of 714 patients as the analysis dataset to distinguish it from the Schering dataset. Table 2.1 shows the distribution of EDSS scores for each visit. For the Day 1 visit, the number of missing EDSS scores was 84 among the analysis dataset patients. For the follow-up visits from 3 up to 36 months, there were 668 missing EDSS scores. The count of missing EDSS scores generally increased with visit, although there was a particularly large number of missing scores at the 33 month visit. 11 Visit EDSS score 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0 NA Baseline 1 1 25 85 80 55 70 76 214 105 2 0 3M 4 10 27 70 82 53 75 68 197 113 11 3 1 6M 7 7 38 62 76 58 62 66 190 114 13 5 3 1 12 9M 2 2 15 31 54 80 55 63 51 206 115 17 4 4 2 13 12M 4 6 12 27 50 67 50 56 59 202 121 20 7 10 23 15M 1 2 5 14 29 44 68 44 61 50 187 127 37 6 5 2 32 18M 1 4 7 27 49 59 51 58 51 181 127 37 14 6 1 41 21M 5 3 9 22 41 58 52 56 46 184 123 36 17 6 4 52 24M 5 3 13 20 41 61 37 57 52 165 135 38 18 12 3 1 53 27M 2 5 14 22 39 44 47 51 47 158 136 38 23 18 3 1 66 30M 2 6 13 17 44 43 36 52 46 155 127 34 26 18 8 1 86 33M 1 5 8 4 12 34 41 28 35 43 128 107 43 27 18 5 175 36M 2 6 8 18 35 37 43 36 35 142 125 47 28 32 6 114 Table 2.1 Patient EDSS scores by visit from the analysis dataset. 2.2 Collapsing the EDSS scores into states Table 2.1 shows that patients from the analysis dataset had EDSS scores in 17 of the score categories. Too few counts in an EDSS score category may make it difficult to estimate an effect for that category. Therefore, the EDSS scores were collapsed into 6 ordinal EDSS states to facilitate analysis. Letters A to F will be used to represent the 6 states: A: EDSS â¤ 3.5 B: EDSS = 4.0, 4.5 C: EDSS = 5.0, 5.5 D: EDSS = 6.0 E: EDSS = 6.5, 7.0 12 F: EDSS â¥ 7.5 Table 2.2 shows that the distribution of the EDSS states by visit has no 0 counts after baseline (in contrast to the EDSS scores). With the exception of state F at the earlier visits, all states had reasonable counts at all visits. Visit EDSS state A B C D E F NA Baseline 112 135 146 214 107 0 0 3M 111 135 143 197 124 3 1 6M 114 134 128 190 127 9 12 9M 104 135 114 206 132 10 13 12M 99 117 115 202 141 17 23 15M 95 112 111 187 164 13 32 18M 88 110 109 181 164 21 41 21M 80 110 102 184 159 27 52 24M 82 98 109 165 173 34 53 27M 82 91 98 158 174 45 66 30M 82 79 98 155 161 53 86 33M 64 69 78 128 150 50 175 36M 69 80 71 142 172 66 114 Table 2.2 Patient EDSS states by visit from the analysis dataset. 2.3 Nearly monotone missingness in the analysis dataset and dropouts We say a patient dropped out at a particular visit if they had no EDSS information for that visit and beyond. The missingness in the analysis dataset was not exactly monotone, where each patient is completely observed until he or she drops out. There were 173 patients who had at least 1 visit that was intermittently missing (in which the visits before and after are observed). All other instances of missingness in the analysis dataset were monotone. As we wish to focus on 13 the dropout pattern, the intermittently missing visits were filled-in with the lowest adjacent EDSS state of the patient, making the missingness in the analysis dataset monotone. The count of dropouts by visit and by group is presented in Table 2.3. The total numbers of patients who dropped out in the placebo and treatment groups were comparable at 54 and 60 patients, respectively. The main trend was that the number of patients who dropped out generally increased over time, with the greatest number of dropouts at 36 months for both groups. Interestingly, at 36 months, nearly double the patients dropped out in the treatment group as in the placebo group. Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 0 3 2 1 1 6 5 2 8 4 9 13 Treatment 0 4 1 3 1 3 2 1 8 6 7 24 Table 2.3 Dropouts by group and by visit in the analysis dataset. 2.4 Initial disability worsening For present purposes, initial disability worsening (IDW) is defined as an increase in EDSS state of least 1 from the baseline EDSS state. The number of patients that had at least 1 instance of IDW was 454, where 239 were from the placebo group, while 215 were from the treatment group. The number of patients who had more than 1 IDW instance was 402. Hence, most patients who had IDW had more than 1 instance. Table 2.4 presents the frequency distribution of all IDW instances in the analysis dataset by visit. 14 Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 53 77 101 126 136 147 155 154 163 165 165 170 Treatment 41 60 88 107 112 116 127 141 144 150 149 141 Total 94 137 189 233 248 263 282 295 307 315 314 311 Table 2.4 All IDW instances in the analysis dataset by visit. A key observation was that the count of IDW generally increased with time for both groups. The 30 month visit had the highest total number of IDWs of 315 (165 in the placebo and 150 in the treatment). There was a total of 2988 IDW instances in the analysis dataset (the placebo patients had 1612 instances, while the treatment patients had 1376 instances). Another important observation was that the placebo group had more IDW instances than the treatment group at every visit, which suggested that there would tend to be more CDW instances in the placebo group at every visit. Table 2.5 shows the distribution of the number of dropouts immediately following an IDW instance by visit. Since IDW could first occur at 3 months, we started with the 6 month count. The number of dropouts given IDW at the immediately preceding visit increased over time. While the numbers of such dropouts were very small (0 or 1) from 6 to 15 months, the numbers of such dropouts were larger (8 or more) from 27 to 36 months. The largest number of such dropouts was 19 at 36 months. This aligned with our expectation that patients who accrued disability worsening at later visits would be more likely to drop out because the patients who were getting worse later in the trial were less likely to have perceived a beneficial effect from the trial. 15 Visit 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M # of drop. given IDW 1 0 1 1 6 5 1 10 8 9 19 Table 2.5 Dropouts immediately following IDW in the analysis dataset. In this project, the treatment effect was to be estimated after the missing CDW status was imputed for every patient who had an IDW instance with an unavailable confirmatory visit (and no prior CDW instances). Table 2.6 shows that the number of patients with such a data pattern was only 26. The largest total number of such IDW instances was 17 at the end of study visit. Interestingly, the majority of these IDW instances were in the placebo group. In particular, the placebo patients had 11 such instances at 36 months, while the treatment patients had only 6. In total, the placebo group had 16 such IDW instances, while the treatment group had 10. Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 0 0 0 0 2 0 0 0 2 1 0 11 Treatment 1 0 0 0 0 0 0 1 0 0 2 6 Total 1 0 0 0 2 0 0 1 2 1 2 17 Table 2.6 IDW instances with unavailable confirmatory visits and no prior CDW instances. 2.5 Confirmed disability worsening Recall that confirmed disability worsening (CDW) occurs when an instance of IDW is followed by another IDW instance at the subsequent visit. The number of patients in the analysis dataset with at least 1 CDW instance was 391, where 212 were from the placebo group, while 179 were from the treatment group. The frequency distribution of first CDW instances by visit is displayed in Table 2.7. There were no first CDW instances at 36 months because that was the last study visit so no confirmatory visit was available. 16 Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 44 25 27 28 14 21 16 5 14 11 7 0 Treatment 24 33 27 21 12 10 11 14 8 12 7 0 Total 68 58 54 49 26 31 27 19 22 23 14 0 Table 2.7 First CDW instances in the analysis dataset by visit. The larger counts of first CDW instances were concentrated in the first year of follow-up. Interestingly, the placebo group generally had more first CDW instances by visit than the treatment group, which was indicative of a positive treatment effect; most notably, there were 20 more CDW instances at 3 months in the placebo group than in the treatment group. The number of patients with more than 1 CDW instance was 344, so the vast majority of patients with at least 1 CDW instance had more than 1. There were 38 patients who had the largest number of CDW instances, which was 11. The frequency distribution of all CDW instances by visit is shown in Table 2.8. The larger counts of CDW instances were concentrated at the later visits. The placebo group had a higher number of CDW instances at all visits, which indicated a strong positive treatment effect. The greatest number of CDW instances was 287 (154 in the placebo and 133 in the treatment group) at 30 months. This is not surprising as SPMS is characterized by a relatively steady worsening of MS over time. Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 44 66 89 113 121 131 141 137 149 154 153 0 Treatment 24 55 76 92 101 111 118 128 128 133 130 0 Total 68 121 165 205 222 242 259 265 277 287 283 0 Table 2.8 All CDW instances in the analysis dataset by visit. 17 The total number of CDW instances was 2394. The proportion of IDW instances that led to CDW was quite high at 0.801. The total number of CDW instances in the placebo group was 1298, while the total number in the treatment group was 1096. The proportions of IDW instances that led to CDW were similar in the placebo (0.805) and treatment groups (0.797). 2.6 Logistic regression model for the presence of CDW We plan to fill-in the missing CDW statuses due to unavailable confirmatory visits using logistic regression imputation. The potential covariates to include in the logistic regression model for the presence of CDW in the analysis dataset are shown in Table 2.9. Note that the time covariate was discrete because it indicated patient visits with medical professionals, which were scheduled every 3 months from baseline to 36 months, but we treated it as a continuous covariate in the analysis. Covariate Description Type Treatment Treatment indicates whether the patient belonged to the placebo or the treatment group. Binary Baseline EDSS state The patientâs EDSS state at the baseline visit. Ordinal Time The time in months at which EDSS values were recorded for each patient by a medical professional. Continuous Age The age in years of each patient at their entry into the study. Continuous Sex The sex of the patient. Binary Centre The location of the centre at which each patient was treated in the study. Nominal Relapse Relapse indicates if the patient had a relapse in the 2 years before the study (yes or no). Binary Table 2.9 Potential covariates to include in logistic regression for the presence of CDW. Since centre has many categories (32 locations in total), it was excluded from the logistic regression analysis. The remaining covariates were considered as predictors in the logistic 18 regression models for the presence of CDW. The effects of those predictors were assessed using likelihood ratio tests for nested logistic regression models. Only patients with IDW followed by a confirmatory visit entered the logistic regression analysis. Additionally, patients who had an IDW instance immediately before dropout or the end of study and no prior CDW instances were excluded because, for those patients, any previous IDW instances that were followed by a confirmatory visit necessarily led to no CDW; including those patients would bias the logistic regression fit toward no CDW. For every patient who entered into the logistic regression analysis (of which there were 223 placebo and 205 treatment patients), only 1 of the eligible IDW instances was randomly drawn and the corresponding patient information was incorporated in the model. We included treatment and time in the initial logistic regression model because they were integral to the structure of the dataset. We also included baseline EDSS state because it was expected to be an important indicator of CDW status: patients with higher baseline EDSS states were farther along with the disease and expected to be worsening to a greater extent than patients with lower baseline EDSS states. The order in which we added covariates to the model could have impacted which covariates were significant in the likelihood ratio tests because different combinations of covariates were likely to be more strongly related than others. Since there were only 3 remaining covariates to be tested and all could have been reasonably strong indicators of CDW, we randomly picked the addition order of sex, age, and then relapse. According to the likelihood ratio tests, none of sex, age or relapse significantly improved the model fit. Hence, the predictors of treatment, time, and baseline EDSS state will be used for the imputation of the missing CDW statuses in future chapters. 19 2.7 Cox survival model for time to first CDW In future chapters, we will fit Cox survival models to simulated datasets based on the analysis dataset and express the treatment effect on time to first CDW by the hazard ratio. Therefore, stepwise variable selection by minimizing the Akaike information criterion (AIC) was performed on a semi-parametric Cox survival model for time to first CDW using all analysis dataset patients to identify the covariates that should be included in these model fits. The stepwise variable selection alternated between forward and backward selection steps. The AIC (for which the degrees of freedom multiple used for the penalty is E = 2) corresponds to a p-value threshold of around 0.157 for covariate entry and removal when only 1 variable is considered. We reduced the threshold to the more stringent 0.05 by setting E = 3.841. The covariates considered in the stepwise variable selection procedure were the same as in the logistic regression analysis of the preceding section save for time. The resulting final model included the covariates of treatment and baseline EDSS state (see Figure 2.1). The hazard ratio for treatment was 0.739 (95% CI [0.605, 0.902]), which indicated that being in the treatment group reduced the hazard of first CDW by about 26.1%, which supported that there was a positive treatment effect. 20 Figure 2.1 Final Cox survival model that resulted from stepwise variable selection. 21 Chapter 3: Generation of the Complete Simulated Dataset We generated a dataset of 2000 patients so that we could conduct a larger-scale simulation study on a dataset that was representative of the analysis dataset. In this chapter, we describe how the simulated dataset was constructed and some basic summary information. 3.1 Treatment group assignment To generate the full dataset of 2000 patients, we randomly assigned half of the patients to each of the placebo and treatment groups, reflecting the roughly equal size of the placebo and treatment groups from the analysis dataset. 3.2 Baseline EDSS state generation The frequency distribution of the baseline EDSS states by treatment group from the analysis dataset is shown in Table 3.1. A chi-square test of homogeneity (p-value 0.259) indicated that the distributions of the baseline EDSS states were reasonably similar for the placebo and treatment groups, as expected due to randomization. Treatment Group Baseline EDSS State A B C D E F Placebo 46 72 69 112 56 0 Treatment 66 63 77 102 51 0 Table 3.1 Distribution of the baseline EDSS states by treatment group from the analysis dataset. In the simulated dataset of 2000 patients, we randomly assigned a baseline EDSS state to each patient. The proportion of patients in each state was based on the baseline distribution that was collapsed across the treatment groups from the analysis dataset. 22 3.3 Generation of follow-up EDSS states To generate the follow-up EDSS states for the patients in the simulated dataset, we incorporated the transition matrices of the analysis dataset groups for each pair of adjacent visits. For example, the transition matrices of the EDSS states for the placebo and treatment groups from baseline to 3 months, from 3 to 6 months, and from 33 to 36 months are provided in Tables 3.2 to 3.7. Baseline EDSS State 3 Month EDSS State A B C D E F A 36 8 2 0 0 0 B 9 51 9 2 0 1 C 0 6 52 10 0 1 D 0 5 7 80 20 0 E 0 0 0 8 48 0 F 0 0 0 0 0 0 Table 3.2 Placebo group transition matrix of counts from baseline to J months. Baseline EDSS State 3 Month EDSS State A B C D E F A 58 7 1 0 0 0 B 8 48 6 1 0 0 C 0 10 55 11 1 0 D 0 1 8 79 13 1 E 0 0 3 6 42 0 F 0 0 0 0 0 0 Table 3.3 Treatment group transition matrix of counts from baseline to J months. 3 Month EDSS State 6 Month EDSS State A B C D E F A 39 5 0 0 0 0 B 6 53 7 3 0 0 C 0 10 47 13 0 0 D 0 0 6 79 14 1 E 0 0 0 4 59 4 F 0 0 0 0 0 2 Table 3.4 Placebo group transition matrix of counts from J to K months. 23 3 Month EDSS State 6 Month EDSS State A B C D E F A 55 8 2 0 1 0 B 14 42 8 2 0 0 C 0 12 49 11 1 0 D 1 4 9 69 11 0 E 0 0 0 10 44 1 F 0 0 0 0 0 1 Table 3.5 Treatment group transition matrix of counts from J to K months. 33 Month EDSS State 36 Month EDSS State A B C D E F A 23 4 1 0 0 0 B 3 29 1 1 1 0 C 0 4 23 12 0 0 D 0 0 1 62 10 1 E 0 0 0 2 71 11 F 0 0 0 1 4 36 Table 3.6 Placebo group transition matrix of counts from JJ to JK months. 33 Month EDSS State 36 Month EDSS State A B C D E F A 39 3 0 0 0 0 B 4 34 4 0 0 0 C 0 5 38 3 1 0 D 0 1 2 58 8 0 E 0 0 1 3 75 3 F 0 0 0 0 2 15 Table 3.7 Treatment group transition matrix of counts from JJ to JK months. If we assumed that the transition probabilities of the placebo and treatment groups were stationarity, we could aggregate these transition matrices to obtain a single transition matrix for each group. Scaling these aggregated matrices of counts by their row sums led to the 2 transition matrices of empirical probabilities shown in Tables 3.8 and 3.9. 24 Current EDSS State Next EDSS State A B C D E F A 0.825 0.149 0.020 0.007 0.000 0.000 B 0.093 0.724 0.147 0.025 0.009 0.002 C 0.006 0.110 0.696 0.170 0.015 0.002 D 0.000 0.010 0.048 0.793 0.141 0.007 E 0.000 0.000 0.004 0.067 0.862 0.067 F 0.000 0.000 0.000 0.015 0.100 0.886 Table 3.8 Aggregated transition matrix for the placebo patients from the analysis dataset. Current EDSS State Next EDSS State A B C D E F A 0.869 0.112 0.013 0.003 0.003 0.000 B 0.095 0.757 0.127 0.022 0.000 0.000 C 0.004 0.099 0.717 0.162 0.018 0.000 D 0.001 0.009 0.060 0.791 0.137 0.002 E 0.001 0.000 0.010 0.098 0.849 0.042 F 0.000 0.000 0.000 0.000 0.121 0.879 Table 3.9 Aggregated transition matrix for the treatment patients from the analysis dataset. The aggregated transition matrices for the placebo and treatment patients had comparable probabilities in each row. We performed a chi-square test of homogeneity with a first-order Rao-Scott corrections for each current EDSS state (as in Decady and Thomas 2000). The tests for the current EDSS states of A, C, and E indicated that the distributions of the next EDSS state were similar across the groups (p-values of 1, 0.281, and 1, respectively). In contrast, the tests for the current EDSS states of B, D, and F suggested there were differences in the distributions of the next EDSS state (p-values < 0.001). Hence, use of these transition matrices to generate the follow-up EDSS states has the potential to produce a simulated dataset with a clear treatment effect. 25 3.4 Generation of the JM month CDW statuses In addition to generating the follow-up EDSS states, we also used the stationary transition matrices to generate 39 month EDSS states, which will only be used to check the proportions of correct CDW and no CDW classifications for the 36 month CDW statuses (as will be explained in the next chapter). Otherwise, those 39 month EDSS states will not be utilized so that the structure of the simulated dataset is as similar to the analysis dataset as possible. 3.5 IDW and CDW instances in the complete simulated dataset As we used stationary transition matrices to generate the follow-up EDSS states, the structure of the complete simulated dataset may be somewhat different from the analysis dataset. In this section, we highlight some of the key IDW and CDW information of the simulated dataset. The number of patients that had at least 1 instance of IDW was 1383, where 733 were from the placebo group, while 650 were from the treatment group. The number of patients who had more than 1 IDW instance was 1289. Hence, most patients who had IDW had more than 1 instance. Table 3.10 depicts the frequency distribution of all IDW instances in this dataset. Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 133 253 331 397 437 474 493 512 523 545 566 572 Treatment 117 207 274 336 379 391 401 431 445 443 452 468 Total 250 460 605 733 816 865 894 943 968 988 1018 1040 Table 3.10 All IDW instances in the simulated dataset by visit. A key observation was that the count of IDW increased as the visit time increased (as in the analysis dataset). The 36 month visit had the highest number of IDW at 1040 (572 in the placebo and 468 in the treatment group). Overall, there was a total of 9670 IDW instances (the 26 placebo patients had 5236 instances, while the treatment patients had 4344 instances). Another important observation was that the placebo group had more IDW instances than the treatment group at every visit (as in the analysis dataset), which suggested that the placebo group would tend to have more CDW instances at every visit. The total number of patients in the simulated dataset with at least 1 CDW instance was 1284, where 680 were from the placebo group, while 604 were from the treatment group. The frequency distribution of first CDW by visit is displayed in Table 3.11. As in the analysis dataset, there were no CDW instances at the 36 month visit because that was the last visit included in the dataset (there was no confirmatory visit). Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 127 115 92 80 49 51 38 40 28 27 33 0 Treatment 107 91 84 72 59 39 32 44 26 23 27 0 Total 234 206 176 152 108 90 70 84 54 50 60 0 Table 3.11 First CDW instances in the simulated dataset by visit. The larger counts of first CDW instances were concentrated in the first year of follow-up. As in the analysis dataset, the placebo group generally had more instances than the treatment group at each visit and this trend was most prominent at earlier visits. For example, there were 20 more instances of first CDW in the placebo group at 3 months, whereas there were only 6 more at 33 months. The vast majority of patients with CDW had more than 1 instance of it as there were 1162 patients with more than 1 CDW instance in the simulated dataset. 140 patients had the largest number of CDW instances in the dataset, which was 11. 27 The frequency distribution of all CDW instances in Table 3.12 shows that the larger counts were concentrated at the later visits as in the analysis dataset. For instance, the highest CDW counts for the placebo and treatment groups were 526 and 429, respectively, at 33 months. As in the analysis dataset, the placebo group had higher CDW counts than the treatment group at every visit, which indicated a positive treatment effect. Across the treatment groups, the greatest number of CDW instances was 965 at 33 months, while the total number of CDW instances in the complete simulated dataset was 8019 (where 4421 were observed in the placebo group and 3598 were observed in the treatment group). The proportion of IDW instances that led to CDW was 0.837, which was higher than the analysis dataset proportion of 0.801. The proportion was higher in the placebo (0.844) than in the treatment group (0.828), which suggested a slight positive treatment effect. Yet, both were higher than the analysis dataset proportions (which were around 0.800). Group Visit 3M 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Placebo 127 235 313 381 409 447 464 483 507 519 536 0 Treatment 107 183 258 312 344 362 375 404 411 413 429 0 Total 234 418 571 693 753 809 839 887 918 932 965 0 Table 3.12 All CDW instances in the simulated dataset by visit. 3.6 The treatment effect hazard ratio and CDW rates in the complete simulated dataset We conducted a time to first CDW analysis using all patients in the complete simulated dataset. As an initial description of this, we obtained the Kaplan-Meier curves by treatment group. Figure 3.1 shows that the treatment curves diverged and, over time, it became more evident that the treatment patients were doing better than the placebo patients. The log-rank p-value of < 0.001 strongly indicated that the survival curves of the treatment group populations differed. 28 A Cox model, adjusted for baseline EDSS state, fitted to the aforementioned subset of the complete simulated dataset led to a treatment effect hazard ratio of 0.811 with a corresponding 95% confidence interval of [0.727, 0.905]. The hazard ratio indicated that there was a reduction in the risk of first CDW in the treatment group compared to the placebo group of 18.9% at any time point in the study. The raw CDW rate, which was the proportion of patients who had at least 1 CDW instance among the patients who had at least 1 IDW instance, was very high at 0.928 in the placebo group and 0.929 in the treatment group. These near identical rates may have suggested that CDW occurred later on average in the treatment group, which can be observed in the Kaplan-Meier curves from Figure 3.1. Figure 3.1 Kaplan-Meier curves for first CDW instance by treatment group with the log-rank p-value. 29 Chapter 4: Differing Dropouts Based on Presence of IDW We obtained multiple samples of the simulated dataset and imposed dropouts on those samples to investigate the impact of imputing the missing CDW status for every patient who had a missing confirmation visit (and no prior CDW) in a time to first CDW analysis. In this chapter, our objective was to observe whether any considered imputation method led to clearly different treatment effect hazard ratios when dropout situations of increasing extremity were imposed. 4.1 A dropout procedure that depends on presence of IDW From the complete simulated dataset of size 2000, we obtained 100 samples that each had 300 patients. Next, a dropout process that was plausibly missing at random (MAR) was applied on each sample. The probability of dropout depended on whether the patient had IDW at the preceding visit. For each visit from 3 up to 36 months, the empirical probabilities of dropout given IDW or no IDW at the preceding visit from the analysis dataset provided the base set of dropout probabilities that were applied to each simulated patient. Tables 2.4 to 2.6 contain information leading to these base dropout probabilities. More explicitly, Tables 4.1 and 4.2 show the empirical probabilities of dropout given IDW or no IDW at the preceding visit from the analysis dataset. The key observation is that all probabilities are very small (below 0.05) save for those at 36 months (which are slightly above 0.05). Visit 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Num. 1 0 1 1 6 5 1 10 8 9 19 Denom. 94 137 189 233 248 263 282 295 307 315 314 Prop. 0.011 0.000 0.005 0.004 0.024 0.019 0.004 0.034 0.026 0.028 0.061 Table 4.1 Empirical probabilities of dropout given IDW at the preceding visit. 30 Visit 6M 9M 12M 15M 18M 21M 24M 27M 30M 33M 36M Num. 6 3 3 1 3 2 2 6 2 7 18 Denom. 620 570 515 467 450 426 399 383 356 338 323 Prop. 0.010 0.005 0.006 0.002 0.007 0.005 0.005 0.016 0.006 0.021 0.056 Table 4.2 Empirical probabilities of dropout given no IDW at the preceding visit. In our construction of more extreme dropout situations, we did not alter the base set of dropout probabilities given no IDW. To scale up the dropouts following instances of IDW, modified probabilities of dropout given IDW at the preceding visit were obtained by expressing the corresponding odds as a multiple of the base set of the odds of dropout given no IDW. There were 8 such dropout factors that we investigated: . = 1.5, 3.9, 7.4, 10, 15, 25, 50, and 100. For the most extreme dropout factor of . = 100, the average number of patients who dropped out in the 100 samples of size 300 was approximately 202. Such a large number of dropouts is not very likely to occur in a real-world MS clinical trial. Hence, no factors larger than the 100 were considered. 4.2 Distributions of the count of dropouts by visit Figures 4.1 to 4.6 depict the distributions of the dropouts in the placebo and treatment groups from the 100 samples of the simulated dataset for each of the dropout factors. Assuming that the dropouts imposed on the simulated dataset samples were at least somewhat representative of the dropouts from the analysis dataset, we expected that the number of patients who dropped out would follow similar patterns to those shown Table 2.3 (at least under lower dropout factors). In particular, we expected the number of dropouts to be comparable in the groups under lower dropout factors. This was observed as the dropout counts for the placebo and treatment groups 31 were quite similar under all factors. The highest number of dropouts was consistently near the end of the study for both groups under lower dropout factors. The overall pattern of an increasing number of dropouts with visit became less pronounced under the more extreme dropout factors. This was because the modified probabilities of dropout given IDW were increased to such an extent that the patients had good chances of dropping out at any of the visits. For example, under the dropout factor of . = 100, the probability of dropout at 6 months given IDW at 3 months was 0.494, at 9 months given IDW at 6 months was 0.346, and at 12 months given IDW at 9 months was 0.369. The probability of dropout given IDW was high at all visits under the extreme dropout factor of . = 100. Figure 4.1 Dropouts in the placebo group by visit under the base set of dropout probabilities. 32 Figure 4.2 Dropouts in the placebo group by visit under the dropout factors of N = O. P, J. M, Q. R, and OS. 33 Figure 4.3 Dropouts in the placebo group by visit under the dropout factors of N = OP, 1P, PS, and OSS. 34 Figure 4.4 Dropouts in the treatment group by visit under the base set of dropout probabilities. 35 Figure 4.5 Dropouts in the treatment group by visit under the dropout factors of N = O. P, J. M, Q. R, and OS. 36 Figure 4.6 Dropouts in the treatment group by visit under the dropout factors of N = OP, 1P, PS, and OSS. 4.3 Numbers of dropouts across the dropout factors by group Figures 4.7 and 4.8 depict the total counts of placebo and treatment group dropouts for the 100 samples across all visits and under each of the dropout factors. For both groups, the medians increased as the dropout factor increased, but the treatment group had slightly smaller medians, which may have been because that group had fewer IDW instances (see Table 3.10). Additionally, both groups had similar spreads, which increased as the dropout factor became 37 more extreme. This was likely due to the magnification of the probabilities of dropout given IDW and since the same conditional probabilities of dropout were applied to both groups. Figure 4.7 Placebo group dropouts across all visits in the OSS samples under each dropout factor. 38 Figure 4.8 Treatment group dropouts across all visits in the OSS samples under each dropout factor. 4.4 The counts of patients requiring CDW status imputation Since the total number of missing confirmatory visits was very small in the analysis dataset (Table 2.6 reports only 26 instances in the analysis dataset of 714 patients), we did not expect imputation to have much impact on the estimated treatment effect under the less extreme dropout factors. Under the base set of dropout probabilities, the number of simulated patients requiring CDW status imputation was typically quite small as can be seen in Figure 4.9. The median count was 0 at all visits except at 36 months where it was around 7. 39 Figure 4.10 shows that the counts of patients requiring CDW status imputation were quite low at all visits save for 36 months under . = 1.5, 3.9, 7.4, 10, while Figure 4.11 shows that the counts of such patients at 3, 6, and 9 months were noticeably larger under . = 50 and 100. Under the base set of dropout probabilities and low dropout factors, the number of patients requiring CDW status imputation generally increased with visit (although the increase was minor until 36 months). However, as the dropout factor increased, that trend reversed. Under the extreme dropout factor of . = 100, the number of patients requiring imputation generally decreased with visit (the highest median number of such patients was at 3 months). This was because as we magnified the conditional dropout probabilities, the patients who had at least 1 IDW instance became more likely to drop out after their first IDW instance. Hence, fewer patients remained as candidates to drop out after their later IDW instances. This was supported by the probabilities of dropout for patients with IDW. For example, under the base set of dropout probabilities, the probability of dropout at 6 months given an IDW instance at 3 months was 0.011 whereas under the dropout factor of . = 50 it was 0.328 and under . = 100 it was 0.494; the corresponding probabilities of dropout at 9 months given an IDW instance at 6 months were 0, 0.209 and 0.346. Overall, the trend reversal was the major visible difference in the boxplots as the dropout factor became more extreme. 40 Figure 4.9 Patients requiring CDW status imputation in the OSS samples under the base set of dropout probabilities. 41 Figure 4.10 Patients requiring CDW status imputation in the OSS samples under the dropout factors of N = O. P, J. M, Q. R, and OS. 42 Figure 4.11 Patients requiring CDW status imputation in the OSS samples under the dropout factors of N = OP, 1P, PS, and OSS. 4.5 The counts of patients requiring CDW status imputation by group We separated the above counts by treatment group to examine if there were differences in the counts for the groups that might be expected to impact the treatment effect; see Figures 4.12 to 4.17. For lower dropout factors, the number of patients requiring imputation was roughly the same in both groups, which was not surprising because treatment did not play a role in the generation of dropouts and we expected the distributions of IDW with unavailable confirmatory 43 visits to show similar patterns as in Table 2.6. However, for the most extreme dropout factor of . = 100, the median number of patients to impute was slightly higher in the placebo group than in the treatment group (particularly at the earlier visits of 6 to 9 months). Figure 4.12 Placebo patients requiring CDW status imputation in the OSS samples under the base set of dropout probabilities. 44 Figure 4.13 Placebo patients requiring CDW status imputation in the OSS samples under the dropout factors of N = O. P, J. M, Q. R, and OS. 45 Figure 4.14 Placebo patients requiring CDW status imputation in the OSS samples under the dropout factors of N = OP, 1P, PS, and OSS. 46 Figure 4.15 Treatment patients requiring CDW status imputation in the OSS samples under the base set of dropout probabilities. 47 Figure 4.16 Treatment patients requiring CDW status imputation in the OSS samples under the dropout factors of N = O. P, J. M, Q. R, and OS. 48 Figure 4.17 Treatment patients requiring CDW status imputation in the OSS samples under the dropout factors of N = OP, 1P, PS, and OSS. 4.6 Imputation strategies used in the simulation study We investigated 8 strategies to impute CDW status for patients who had missing confirmation visits. The purpose was to assess whether 1 or more of these strategies gave an estimated treatment effect that was clearly different from the others in a time to first CDW analysis. The first 2 imputation strategies are standard for analyzing clinical trial data. In the none confirmed strategy, none of the patients with missing confirmatory visits were imputed to have 49 CDW. In the all confirmed strategy, all of the patients with missing confirmatory visits were imputed to have CDW. The none confirmed and the all confirmed imputation strategies are extreme strategies because they provide the minimum and maximum possible number of CDW instances in a sample. Yet, this does not imply that they are extreme strategies in terms of their resulting treatment effect hazard ratios. The third, fourth, and fifth strategies were k-nearest neighbour (kNN) single imputation with 3, 5, and 7 nearest neighbours considered, respectively. The neighbours considered were every IDW instance with a confirmatory visit, save for those from the patients requiring imputation because their IDW were always followed by no CDW, which would bias the statuses of the neighbours toward no CDW. The variables used in the distance calculations were visit time, baseline EDSS state, and treatment group. Since the counts of CDW varied across the visits, we wanted to consider visit time as a factor in the kNN imputation strategy. The k-nearest neighbour CDW statuses yield each of the imputed CDW statuses by majority vote. In the kNN function from the VIM package, variations of the Gower distance metric are used. The distance between observations T and U is VW,X = YZ[W,X,ZZ]^ YZZ]^ where E is a label for the variables used in the imputation, _ is the total number of variables used in the imputation, YZ is the weight, and [W,X,Z is the measure of the kth variableâs contribution (Kowarik and Templ 2016). The formula for [W,X,Z depends on the variable type. For binary and nominal variables, a binary distance is used: [W,X,Z = 0 Yâab cW,Z = cX,Z1 Yâab cW,Z â cX,Z For continuous variables, if eZ is the range of variable E, then 50 [W,X,Z = cW,Z â cX,ZeZ . Ordinal variables are first converted to integer values for successive categories. This conversion assumes equal distances between the categories. Then, [W,X,Z is determined in the same way as in the continuous variable case. The final 3 imputation strategies were multiple imputation strategies. Rubin (1987) recommended that between 2 to 10 imputations were sufficient in multiple imputation. Hence, g = 10 imputations were performed for every sample. The sixth imputation strategy assumed that once an IDW instance occurred, the probability of CDW was the same for all patients at all visits. The IDW instance and accompanying CDW status used to calculate the probability of CDW given IDW were obtained by randomly drawing an IDW instance with an available confirmatory visit from each sample patient with at least 1 such instance. Only patients who did not require imputation were included because of the issue of bias toward no CDW as was explained for the kNN strategies. We shall refer to this strategy, which used random sampling from a Bernoulli distribution, as the Bernoulli strategy. The seventh imputation strategy used logistic regression with presence of CDW as the response. We included the covariates of treatment, baseline EDSS state and visit time because they were significant in the logistic regression investigation of the analysis dataset. The patient information used in the logistic regression strategy was structured the same as in the analysis dataset investigation. Also, note that the patient information was exactly the same as in the Bernoulli strategy. To impute the missing CDW statuses, Bernoulli sampling was applied, which used the predicted probabilities from the fitted logistic regression. The eighth imputation strategy was a Bayesian logistic regression method, which considered the same predictors and the same patient information as in the logistic regression imputation 51 strategy. The MICE package in R was used to implement this strategy. MICE uses chained equations (ie. a fully conditional specification) to yield multivariate imputations, where imputation is done on a variable-by-variable basis using conditional densities. Since MICE focuses on 1 variable at a time, a clear advantage is that it can impute multiple variables of different types. As we are only imputing 1 variable, CDW status, we will just provide the steps of the MICE algorithm for Bayesian logistic regression (as in Van Buuren 2019): 1. A fit of the logistic regression model hijTk l(n = 1 o)) = opq yields q, the estimated regression coefficients, and r q , the unscaled estimated covariance matrix of q. 2. qâ is drawn from t q, r q . To accomplish that, first the Cholesky decomposition, ruv, of r q is calculated. Next, a t 0, 1 value is drawn for each coefficient and stored in vector w. Then, qâ = q + rO1 â w is calculated, where rO1 â w may be thought of as a perturbation for the q values. 3. For each of the missing binary statuses, a predicted probability is calculated by using hijTky^ opqâ . 4. Bernoulli sampling is used to impute the missing status for each predicted probability. MICE generates g = 10 imputed datasets via implementing the above algorithm in-parallel g times (Van Buuren and Groothuis-Oudshoorn 2011). 4.7 Obtaining the treatment effect hazard ratios After imputation using 1 of the above strategies, a semi-parametric Cox survival model for time to first CDW was fit to all of the sample patients in each imputed dataset. Adjustment for baseline EDSS state was incorporated as suggested by the results for the analysis dataset. 52 For the single imputation strategies, an estimated treatment effect log hazard ratio was obtained for each of the 100 samples and the corresponding estimated hazard ratio was obtained via a back-transformation. Then, the Monte Carlo standard error, which is the standard deviation of the estimates divided by the square root of the number of samples, was calculated. In contrast, the multiple imputation strategies produced a log hazard ratio for each of the g = 10 imputed datasets for each sample. These were averaged as recommended in Rubinâs rules for pooling multiple imputation estimates (see Rubin 1987; Dong and Peng 2013). For Rubinâs rules to apply, the sampling distribution of the estimated parameter should approximately follow a normal distribution (Rubin 1987; Heymans and Eekhout 2019). Since hazard ratios cannot be negative, the log hazard ratios were instead averaged across the g = 10 imputed datasets (as recommended in Marshall, Altman, Holder, and Royston 2009). The average log hazard ratio was then back-transformed to obtain the estimated hazard ratio for that sample. Then, the Monte Carlo standard error was obtained as for the single imputation strategies. 4.8 Overall results for the average treatment effect hazard ratios Table 4.3 shows the average treatment effect hazard ratio over the 100 samples and the corresponding Monte Carlo standard error (MC SE) under the base set of dropout probabilities and under all of the dropout factors. The results from the analysis of the complete dataset are also included to allow for direct comparison. Since kNN (E = 3, 5, 7) led to essentially identical results for each dropout factor, Table 4.3 presents only 1 set of results for all 3 E values. The main result from the table was that the estimated hazard ratios for the 8 imputation strategies were effectively indistinguishable under every dropout factor when their MC SEs were 53 taken into account. That is, there was no clear differentiation in the estimated hazard ratios across the 8 imputation strategies. To provide some perspective on the impact of the most extreme dropout factor, the number of dropouts under the base set of dropout probabilities and under the most extreme dropout factor were obtained. The number of patients who dropped out increased from an average of approximately 48 out of the 300 sampled patients under the base set of dropout probabilities to 202 out of the 300 sampled patients under the dropout factor of . = 100. So, even though the number of dropouts increased to about two-thirds of the sampled patients on average, the treatment effect hazard ratios remained indistinguishable across the imputation strategies. Although there were slightly more IDW instances in the placebo than in the treatment group from the complete simulated dataset (see Table 3.10), the numbers of IDW instances were still fairly comparable between the groups. Hence, the number of dropouts was likely to increase to nearly the same extent in both groups as the dropout factor increased. Additionally, since the proportion of IDW instances that led to CDW was quite similar in the groups from the complete simulated dataset (0.844 for the placebo vs 0.828 for the treatment group), the number of imputed CDW instances was likely to be roughly the same in both groups (even when the imputation strategies incorporated treatment group). Therefore, it was unlikely that any dropout factor would have a great impact on the estimated treatment effect. Table 4.3 shows that the average treatment effect hazard ratios (HRs) were fairly similar to the complete dataset hazard ratio when the MC SEs were taken into account, which indicated good recovery of the âtrueâ treatment effect under all considered dropout factors. Yet, the average hazard ratios were consistently slightly larger than the complete dataset hazard ratio, which indicated that the treatment effect was slightly weaker under the dropout factors. This 54 overestimation was likely due to the lack of incorporation of the treatment group in the generation of the dropouts. Due to this, the number of dropouts and the number of imputations was very similar in both groups. The similar numbers of imputations coupled with the similar proportions of CDW given IDW in the groups led to a slight dilution of the treatment effect in the imputed samples, resulting in weaker treatment effects in those samples. 55 Imputation strategy Single imputation Multiple imputation All confirmed None confirmed kNN (: = <, >, ?) Bernoulli Log. Reg. Baye. Log. Reg. Complete data. P T P T P T P T P T P T P T CDW rate 0.928 0.929 HR [P>% CI] 0.811 [0.727, 0.905] Base set dr. Avg. CDW rate 0.959 0.960 0.917 0.915 0.959 0.959 0.954 0.955 0.956 0.956 0.955 0.954 Avg. HR [MC SE]1 0.836 [0.012] 0.838 [0.013] 0.836 [0.012] 0.837 [0.012] 0.836 [0.012] 0.835 [0.012] Dr. [ = . > Avg. CDW rate 0.960 0.960 0.917 0.918 0.959 0.959 0.954 0.955 0.956 0.955 0.955 0.954 Avg. HR [MC SE] 0.833 [0.013] 0.838 [0.013] 0.833 [0.013] 0.834 [0.013] 0.832 [0.013] 0.833 [0.013] Dr. [ = <. P Avg. CDW rate 0.960 0.961 0.893 0.895 0.959 0.959 0.952 0.952 0.954 0.953 0.951 0.951 Avg. HR [MC SE] 0.835 [0.013] 0.842 [0.013] 0.835 [0.013] 0.836 [0.013] 0.835 [0.013] 0.835 [0.013] Dr. [ = ?. ] Avg. CDW rate 0.962 0.962 0.868 0.874 0.962 0.961 0.951 0.952 0.953 0.952 0.951 0.949 Avg. HR [MC SE] 0.831 [0.012] 0.842 [0.013] 0.831 [0.012] 0.832 [0.012] 0.830 [0.013] 0.830 [0.013] Dr. [ = ^ Avg. CDW rate 0.963 0.963 0.853 0.855 0.963 0.962 0.951 0.951 0.952 0.950 0.949 0.947 Avg. HR [MC SE] 0.831 [0.013] 0.839 [0.013] 0.831 [0.013] 0.832 [0.013] 0.828 [0.013] 0.830 [0.013] Dr. [ = > Avg. CDW rate 0.964 0.964 0.823 0.819 0.963 0.963 0.947 0.948 0.950 0.947 0.946 0.941 Avg. HR [MC SE] 0.834 [0.013] 0.835 [0.013] 0.833 [0.013] 0.835 [0.013] 0.830 [0.013] 0.830 [0.013] Dr. [ = _> Avg. CDW rate 0.967 0.967 0.769 0.757 0.965 0.965 0.946 0.946 0.947 0.942 0.941 0.935 Avg. HR [MC SE] 0.835 [0.012] 0.830 [0.013] 0.834 [0.012] 0.835 [0.012] 0.831 [0.012] 0.830 [0.012] Dr. [ = >^ Avg. CDW rate 0.972 0.971 0.670 0.668 0.970 0.967 0.943 0.942 0.946 0.937 0.934 0.923 Avg. HR [MC SE] 0.832 [0.012] 0.842 [0.014] 0.830 [0.012] 0.833 [0.012] 0.824 [0.012] 0.822 [0.012] Dr. [ = ^^ Avg. CDW rate 0.978 0.978 0.542 0.539 0.976 0.976 0.945 0.945 0.946 0.938 0.922 0.914 Avg. HR [MC SE] 0.831 [0.013] 0.843 [0.016] 0.832 [0.013] 0.831 [0.012] 0.825 [0.013] 0.826 [0.013] Table 4.3 Treatment effect HRs and CDW rates averaged over the ^^ samples under all dropout factors. 1 Average hazard ratio for the imputation strategy of interest with the Monte Carlo standard error across the 100 samples. 56 4.9 Key results for the base set dropout probabilities and the dropout factor of !"" Only the results for the base set of dropout probabilities and the most extreme dropout factor are discussed in more detail in this section because the results from the intermediate dropout factors were very similar. Figure 4.18 presents the estimated treatment effect hazard ratios from the 100 samples under the base set of dropout probabilities. The average treatment effect hazard ratios were almost identical for the 8 imputation strategies, varying only between 0.835 to 0.838. No imputation strategy gave an estimated treatment effect that was clearly different than those from the other imputation strategies. Next, we shift our focus to the most extreme dropout factor of ) = 100. Figure 4.19 shows that the medians and spreads of the hazard ratios were very similar across the imputation strategies. Table 4.3 indicates that the average estimated treatment effect hazard ratios were comparable to those from the base set of dropout probabilities as they ranged from 0.826 to 0.843. There was more differentiation across the strategies under the more extreme dropout factor as the none confirmed strategy led to a slightly, but perhaps negligibly, higher average treatment effect hazard ratio and a slightly larger spread than the rest. Considered together, our observations indicate that the method we used to magnify the dropouts did not appear to affect the estimated treatment effect much regardless of the imputation strategies considered. 57 Figure 4.18 Treat. eff. HRs across the !"" samples under the base set of drop. prob. Figure 4.19 Treat. eff. HRs across the !"" samples under / = !"". 58 4.10 Key results from the average CDW rates In Table 4.3, we also included the average CDW rate for the groups. The purpose was to give some indication as to why the hazard ratios differed across the imputation strategies. The all and none confirmed strategies always had the highest and lowest CDW rates, respectively, as expected. Interestingly, the kNN strategies had CDW rates that were almost the same as the all confirmed rates. The kNN strategies were likely to have high CDW rates because the nearest neighbours could have been any of the IDW visits with available confirmatory visits (from the patients not requiring imputation) and most IDW instances led to CDW in the complete simulated dataset. As was observed in the complete simulated dataset, the groups had very similar rates of CDW across all of the imputation strategies for each dropout factor. Under every dropout factor, the average CDW rates for both groups were quite large (above 0.910), save for the none confirmed strategy. The lowest average CDW rate was 0.539 for the treatment and 0.542 for the placebo patients from the none confirmed strategy under the dropout factor of ) = 100. The highest average CDW rate was 0.978 for both groups from the all confirmed strategy under the dropout factor of ) = 100. Although the average CDW rates differed greatly for those imputation strategies, their average hazard ratios were similar. This may have been because the average CDW rate was based on only the patients who had at least 1 IDW and not all of the patients in the sample, which the hazard ratios were based on. Instead of focusing on the raw average CDW rates for the groups, the differences in those rates for each imputation strategy may be more relevant to the treatment effect. Under the base set of dropout probabilities and the dropout factor of ) = 100, the differences in the average 59 CDW rates were close to 0 for each of the considered imputation strategies, which pointed to weaker treatment effects as were observed in Table 4.3. Additionally, those differences were comparable, suggesting that that the treatment effects were likely to be similar, which was also observed in that table. 4.11 Similar performances across the imputation strategies The kNN imputation strategies gave nearly the same average treatment effect hazard ratios and those results were nearly the same as the all confirmed results, suggesting that CDW was almost always imputed. We attributed that to the fact that the kNN strategies considered every IDW with a confirmation visit (from the patients who did not require imputation) as the potential neighbours and most of those instances of IDW led to CDW in the simulated dataset. For that reason, the kNN imputation strategies performed similarly to the all confirmed imputation strategy under all dropout factors. The Bernoulli strategy produced very similar average treatment effect hazard ratios to the all confirmed strategy likely for that same reason. The logistic regression and Bayesian logistic regression strategies gave very similar average treatment effect hazard ratios regardless of the dropout factor. In general, those strategies produced the lowest average treatment effect hazard ratios, which may have been because they incorporated more patient information than many of the other strategies (specifically, treatment group, time, and baseline EDSS state). So, the treatment effect was likely to be more pronounced under those strategies. 60 4.12 Confusion matrices to assess correct classification Under the different single imputation strategies, we constructed 2 Ã 2 confusion matrices to examine the proportion of correctly classified imputed CDW statuses for each of the 100 samples. Each imputed CDW status was compared to the corresponding value from the complete dataset. Recall that we generated month 39 visits in the complete dataset to establish the true values for the month 36 CDW statuses that required imputation. For the multiple imputation strategies, we created a confusion matrix for each of the 10 imputations from each sample. To obtain 1 matrix per sample, we obtained the element-wise averages of the confusion matrices that pertained to each sample. Then, we calculated the proportions of correct classifications from each of the resulting matrices. For the all confirmed case, the proportion of correct CDW classifications was exactly 1 because all of the imputed patients were imputed to have CDW. Similarly, the proportion of correct no CDW classifications was exactly 0 in that case. Likewise, in the none confirmed case, the proportion of correct no CDW classifications was exactly 1 and the proportion of correct CDW classifications was exactly 0. For the other imputation strategies, Figure 4.20 shows the proportions of correct CDW and no CDW classifications across all 100 samples under the base set of dropout probabilities and under the dropout factor of ) = 100. The results for only the 2 extreme situations were shown because the plots were not very different across the intermediate dropout factors. Except for the none confirmed strategy, there was a general trend across all dropout factors that the proportions of correct CDW classifications were tending toward or exactly equal to 1, while the proportions of correct no CDW classifications were tending toward or exactly 0. This may reflect the fact that the proportion of patients with at least 1 CDW instance out of those who 61 had at least 1 IDW instance was very high under all imputation strategies (save for the none confirmed strategy) as in the complete simulated dataset. Correspondingly, since there was a very small proportion of patients who did not have at least 1 CDW instance out of those who had at least 1 IDW instance, the proportion of correct no CDW classifications was expected to tend toward 0 under these strategies. It was interesting that there was very little variability in the proportions of correct classifications for the kNN strategies under both dropout situations. This may have been because all instances of IDW with available confirmation visits from the patients not requiring imputation were considered as potential neighbours in the kNN strategies. Since most instances of IDW led to CDW, it was plausible that nearly all of the patients who required imputation would be set to have CDW. The selection pool formulation also explained the similarities in the proportions of correct CDW classifications for the kNN and the all confirmed strategies under both dropout situations. In contrast, the 3 multiple imputation strategies had more variability in their boxplots, which was likely due to the randomness introduced by the multiple imputations. Additionally, the variability was generally lower under the more extreme dropout factor of ) = 100, which may have been because there were more imputations for patients who had IDW as the dropout factor increased (that were likely to have been imputed as CDW by the above reasoning). 62 Figure 4.20 Proportions of correctly classified CDW and no CDW patients across the !"" samples under the base set of dropout probabilities (left) and under the dropout factor of / = !"" (right). 4.13 Summary Nine dropout situations of increasing extremity were implemented on 100 samples of 300 patients from the complete simulated dataset of 2000 patients. The base set of dropout probabilities were simply the analysis dataset probabilities of dropout at each visit given IDW or no IDW at the preceding visit. For every non-base set of dropout probabilities, the no IDW 63 dropout probabilities were unchanged, while the IDW probabilities of dropout were modified by scaling-up the base set of no IDW dropout probabilities on the odds scale. After the dropouts were imposed on a sample of the simulated dataset, 5 single and 3 multiple imputation strategies were used to fill in any missing CDW statuses. Then, the estimated treatment effect hazard ratio was ascertained from a time to first CDW model for each of the imputation strategies. The major result was that none of the imputation strategies led to clearly different estimated treatment effect hazard ratios under any of the dropout factors. This result was not entirely surprising because the distributions of IDW were fairly comparable in both groups from the complete simulated dataset. Hence, when the dropouts were magnified, the number of patients who had IDW and then dropped out and, thus, the number of CDW statuses requiring imputation would likely increase to nearly the same extent in both groups. Coupled with the fact that the proportion of the IDW instances that led to CDW was very similar in both groups, it was unlikely that any dropout factor would have a great impact on the estimated treatment effect. Since the different imputation methods did not lead to clearly different treatment effects under any of the considered dropout factors, we decided to investigate whether this occurred in the presence of stronger treatment effects. In the next chapter, we explore 3 scenarios that are intended to magnify the treatment effect, where we modify of the probabilities from the treatment groupâs stationary transition matrix. 64 Chapter 5: Differing Treatment Effects Based on Transition Matrices In this chapter, our main objective was to investigate whether any of the previously introduced imputation methods led to clearly different treatment effect hazard ratios in 3 transition matrix-based treatment scenarios. In this investigation, the follow-up EDSS states of patients in the treatment group were simulated using a modified stationary transition matrix, while the EDSS states of the placebo patients were generated from the groupâs original stationary transition matrix. Otherwise, the investigation proceeded as in the previous chapter. 5.1 Generation of follow-up EDSS states In the investigation from the previous chapter, henceforth referred to as the base treatment scenario, the follow-up EDSS states of the patients were generated using the placebo and treatment stationary transition matrices given in Tables 3.8 and 3.9. The same placebo transition matrix was used here, but the treatment transition matrix was modified according to 3 treatment scenarios from Healy, Chitnis, and Engler (2011). The 3 Healy transition matrix-based treatment scenarios are as follows: 1. The probabilities of moving to higher EDSS states (probabilities of progression) are decreased by a constant factor, while the probabilities of moving to lower EDSS states are not changed. In matrix terms, the elements above the main diagonal are all divided by a factor, 3, that is greater than 1. Then, the elements on the diagonal, which correspond to the probability of remaining in the same EDSS state, are modified accordingly to ensure that each row still has a sum of 1. 65 2. The probabilities of moving to higher EDSS states are not changed, while the probabilities of moving to lower EDSS states (probabilities of improvement) are increased by a constant factor. In matrix terms, the elements below the main diagonal are all multiplied by a factor, 3, that is greater than 1. Then, the elements on the diagonal are modified accordingly to ensure that each row still has a sum of 1. 3. The probabilities of moving to higher EDSS states are decreased by a constant factor, while the probabilities of moving to lower EDSS states are increased by a constant factor. In matrix terms, the elements above the main diagonal are all divided by a factor, 3, that is greater than 1, while the elements below the main diagonal are all multiplied by the same factor. Finally, the elements on the diagonal are modified accordingly to ensure that each row still has a sum of 1. The factors considered were 3 = 1.5 to 3 in steps of 0.5 for a total of 4 factors (not including the base scenario). We did not consider factors beyond 3 = 3 because larger factors led to treatment effect hazard ratios less than 0.1 in at least 1 of the scenarios, which are likely too extreme to be realistic representations of MS clinical trials. After the complete datasets of 2000 patients were generated using the stationary transition matrices described above, the investigation proceeded exactly as in the previous chapter. Overall, the investigation was a 3 Ã 4 Ã 9 factorial design because there were 3 treatment scenarios, 4 factors used to create the treatment group transition matrix, and 9 dropout situations in total. 5.2 Treatment effects for the complete simulated datasets For each complete simulated dataset, Table 5.1 shows the treatment effect hazard ratio and the corresponding 95% confidence interval from the Cox model adjusted for baseline EDSS state. 66 For reference, the treatment effect hazard ratio from the base scenario was 0.811 (95% CI [0.727, 0.905]). Scenario Factor 8 1.5 2.0 2.5 3.0 1 0.519 [0.463, 0.582] 0.386 [0.341, 0.436] 0.319 [0.280, 0.363] 0.263 [0.229, 0.301] 2 0.649 [0.582, 0.725] 0.588 [0.525, 0.658] 0.515 [0.459, 0.579] 0.443 [0.392, 0.499] 3 0.469 [0.417, 0.528] 0.293 [0.256, 0.335] 0.205 [0.177, 0.238] 0.139 [0.118, 0.165] Table 5.1 Treatment effect HR [9:% CI] from the complete datasets for the ; matrix-based scenarios. All factors from each scenario led to greatly decreased hazard ratios in comparison to the base scenario (indicating stronger treatment effects) and their 95% confidence intervals had no overlap with the confidence interval for the base scenario. For each scenario, the hazard ratio decreased substantially as the treatment factor increased. Since the placebo group transition matrix was not modified, the distribution of CDW instances among the placebo patients was the same in all treatment scenarios and for all factors. Hence, save for random variation, these differences in the hazard ratios were being driven entirely by the distributions of CDW instances among the treatment patients. In the first and second treatment scenarios, differences in the numbers of CDW instances in the treatment group (and the resulting hazard ratios) were likely due to the different impacts of the treatment factors on the diagonal probabilities of the treatment group transition matrix (Table 3.9). While increasing the probabilities of improvement led to the probabilities along the diagonal becoming lower, decreasing the probabilities of progression led to those probabilities becoming higher. So, as the treatment factor became more extreme in the first scenario, the diagonal probabilities (which were already quite close to 1 in the original treatment group 67 transition matrix) became closer and closer to 1 and all states became more and more like absorbing states. The higher probabilities of treatment patients remaining in the same states under more extreme treatment factors suggested that CDW was less likely to occur in that group, leading to stronger treatment effects. In contrast, as the treatment factor became more extreme in the second scenario, the probabilities of treatment patients moving to lower EDSS states became higher, also resulting in fewer CDW instances in that group. As expected, the treatment effect hazard ratios were smallest in the third scenario for each treatment factor, 3. Additionally, the second scenario consistently led to higher hazard ratios than the first scenario for a fixed 3, suggesting that more treatment patients had CDW in the second than in the first scenario. Since the treatment patients had higher probabilities of moving to higher EDSS states in the second scenario (where the probabilities of progression were unchanged), there were likely greater numbers of IDW and CDW instances in the second scenario treatment group than in the first. This suggested that, for a fixed 3, the treatment effect would be less pronounced in the second scenario than in the first. 5.3 The counts of patients requiring CDW status imputation Figure 5.1 shows the number of patients from each sample who required imputation by visit for the most extreme treatment effect factor 3 = 3 in conjunction with the most extreme dropout factor ) = 100. In all 3 scenarios, the number of patients to be imputed was highest at the 3 month visit and generally decreased thereafter. Our explanation is as in Section 4.4: under ) = 100, the dropout probabilities were magnified to such an extent that patients were very likely to dropout after their first IDW. Hence, there were simply less patients remaining that could drop out at the later visits. 68 The boxplots show relatively large numbers of patients who required their CDW statuses to be imputed in all 3 scenarios. The medians from these boxplots indicated that the number of patients requiring imputation in each sample was around 76 in the first treatment scenario, 95 in the second treatment scenario, and 70 in the third treatment scenario. Even so, it was unclear if these counts were large enough to induce differences in the treatment effect hazard ratios across the imputation strategies. Figure 5.1 Patients requiring CDW status imputation in the !"" samples under the treatment scenarios !, <, and ; with 8 = ; and the dropout factor of / = !"". 69 5.4 The counts of patients requiring CDW status imputation by group To check whether there were differences in the number of cases to impute for the groups, we separated the boxplots by treatment group in Figures 5.2 and 5.3. Since the placebo group was unaffected by the treatment scenarios, we expected that the 3 plots would be identical to the dropout ) = 100 plot from Figure 4.14 (save for randomness), which was observed. The numbers of CDW statuses to be imputed were fairly comparable in the groups from the second scenario (only slightly larger in the placebo group, based on the medians). In the treatment group, we anticipated that increasing the probabilities of improvement would have less impact on decreasing the number of CDW statuses to be imputed than decreasing the probabilities of progression by the same factor since IDW and CDW are markers of disease progression and not improvement. This was supported as the number of CDW status imputations was much lower in the treatment group relative to the placebo group in the first and third scenarios than in the second. Yet, the median number was below 4 patients at all visits in those 2 scenarios, suggesting that the number of treatment group CDW statuses to be imputed may have not been large enough to have a noticeable impact. That is, there were very few opportunities for the imputation strategies to yield clearly different treatment effects at most visits. Perhaps this occurred because visit (and other important factors) were not incorporated into the generation of the post-baseline EDSS states in the 3 matrix-based approaches. The trend that the number of patients requiring CDW status imputation typically decreased with visit was clear in both groups from the second treatment scenario. This trend was also evident in the placebo groups from the first and third treatment scenarios, but not as clear in the treatment groups likely because of the smaller counts for those groups. 70 Figure 5.2 Placebo patients requiring CDW status imputation in the !"" samples under the treatment scenarios !, <, and ; with 8 = ; and the dropout factor of / = !"". 71 Figure 5.3 Treatment patients requiring CDW status imputation in the !"" samples under the treatment scenarios !, <, and ; with 8 = ; and the dropout factor of / = !"". 5.5 Overall results for the average treatment effect hazard ratios The same Cox model as in previous chapters was fitted to each imputed sample of size 300 and each sampleâs estimated treatment effect hazard ratio was obtained. Table 5.2 shows the estimated treatment effect hazard ratios averaged over the samples under the most extreme treatment effect factor (3 = 3) for each scenario in conjunction with the base set of dropout probabilities and the dropout factor of ) = 100. The corresponding results for the base 72 treatment scenario, which were already discussed in Section 4.8, are included for reference. We chose to present the results for the most extreme treatment and dropout factors because the intermediate factors led to similar patterns. For the most extreme dropout factor of ) = 100, there was greater discernment across the imputation strategies in the second treatment scenario than in the other scenarios likely due to the larger number of CDW statuses that were required to be imputed in the treatment group (see Figure 5.3). In particular, the logistic regression strategies led to the lowest average hazard ratios, while the none confirmed and kNN (= = 3) strategies led to marginally higher hazard ratios. Although these groupings of the hazard ratios were not definitive, it was evident that these 4 average hazard ratios were clearly lower than those from the remaining strategies. A possible reason for this is the treatment group patients had higher probabilities of attaining lower EDSS states in this scenario and, hence, were less likely to have CDW. Due to the higher probabilities of attaining lower states and the relatively lower probabilities of remaining in the same or moving to higher states, a smaller fraction of the missing CDW statuses from the treatment group were imputed as CDW in the strategies where treatment group was incorporated, leading to stronger treatment effects for those strategies. For the kNN strategies, = = 3 led to lower (stronger) hazard ratios because there were likely fewer no CDW than CDW instances overall. So, the no CDW instances held more weight as neighbours when the = value was smaller, but were outnumbered when = was larger. The second treatment scenario led to the greatest median number of CDW statuses to impute in the treatment group. Under the none confirmed strategy, this may have led to a substantial number of no CDW in that group and a corresponding strengthened positive treatment effect. Despite these observed differences, the average hazard 73 ratios from the considered imputation strategies were not substantially different as the largest difference was only about 0.072 units on the hazard ratio scale. In the third scenario with the most extreme dropout factor, the kNN (= = 3) and the logistic regression average hazard ratios were marginally smaller than the rest, likely for the same reasons given above. We believe these hazard ratios were lower on average than the corresponding none confirmed hazard ratio because these strategies accounted for the smaller fraction of CDW in the treatment group, which bolstered the positive treatment effect. However, the differences across the imputation strategies were even smaller in this scenario than in the second; hence, they were regarded as insubstantial. In the first scenario, the average treatment effect hazard ratios had a great degree of similarity across the imputation strategies regardless of the dropout factor, which may have been due to the very high probabilities of remaining in the same EDSS state in the modified treatment group transition matrix for this scenario (as explained in Section 5.2). As a result, treatment patients that had an IDW instance were highly likely to have an accompanying CDW instance. Since the probabilities of IDW leading to CDW were very high in both groups and the number of imputations in the treatment group was lower, it was unlikely that any imputation strategy would result in clear differences in the average hazard ratios across the strategies. The first scenario led to average hazard ratios that were all similar to the corresponding complete dataset hazard ratio, likely for the reasons given above. On the other hand, in the second and third scenarios, the imputation strategies that accounted for treatment group led to average hazard ratios that were generally closer in absolute value to the complete dataset hazard ratio, indicating better recovery of the âtrueâ treatment effect for those strategies, likely for the reasons discussed above. 74 The above discussion was based solely on the average hazard ratios. So, we next look at the collections of the treatment effect hazard ratios from the samples for each of the 3 matrix-based treatment scenarios. 75 Situation Summary Imputation strategy Single imputation Multiple imputation All confirmed None confirmed kNN (: = <) kNN (: = >) kNN (: = ?) Bernoulli Log. Reg. Baye. Log. Reg. P T P T P T P T P T P T P T P T Base trt. & base set dr. Avg. CDW rate 0.959 0.960 0.917 0.915 0.958 0.957 0.959 0.959 0.959 0.960 0.954 0.955 0.956 0.956 0.955 0.954 Avg. HR [MC SE]2 0.836 [0.012] 0.838 [0.013] 0.835 [0.012] 0.836 [0.012] 0.837 [0.012] 0.837 [0.012] 0.836 [0.012] 0.835 [0.012] Base trt. & dr. Z = [\ Avg. CDW rate 0.978 0.978 0.542 0.539 0.973 0.971 0.977 0.978 0.978 0.978 0.945 0.945 0.946 0.938 0.922 0.914 Avg. HR [MC SE] 0.831 [0.013] 0.843 [0.016] 0.830 [0.013] 0.832 [0.013] 0.832 [0.013] 0.831 [0.012] 0.825 [0.013] 0.826 [0.013] 1 trt. sc. & base set. dr.3 Avg. CDW rate 0.958 0.952 0.922 0.901 0.957 0.952 0.957 0.952 0.958 0.952 0.953 0.946 0.954 0.945 0.953 0.944 Avg. HR [MC SE] 0.264 [0.006] 0.266 [0.005] 0.264 [0.005] 0.264 [0.005] 0.264 [0.005] 0.264 [0.005] 0.264 [0.005] 0.264 [0.005] 1 trt. sc. & dr. Z = [\ Avg. CDW rate 0.978 0.973 0.554 0.532 0.974 0.965 0.977 0.972 0.978 0.972 0.945 0.937 0.945 0.927 0.919 0.897 Avg. HR [MC SE] 0.260 [0.005] 0.269 [0.007] 0.259 [0.005] 0.260 [0.005] 0.260 [0.005] 0.260 [0.005] 0.258 [0.005] 0.258 [0.005] 2 trt. sc. & base set. dr Avg. CDW rate 0.958 0.798 0.923 0.753 0.958 0.795 0.958 0.795 0.958 0.797 0.951 0.788 0.955 0.785 0.954 0.784 Avg. HR [MC SE] 0.467 [0.007] 0.466 [0.007] 0.466 [0.007] 0.466 [0.007] 0.467 [0.007] 0.467 [0.007] 0.463 [0.007] 0.462 [0.007] 2 trt. sc. & dr. Z = [\ Avg. CDW rate 0.978 0.890 0.541 0.428 0.972 0.801 0.977 0.829 0.977 0.849 0.905 0.811 0.943 0.748 0.925 0.729 Avg. HR [MC SE] 0.530 [0.008] 0.483 [0.010] 0.478 [0.008] 0.493 [0.008] 0.505 [0.008] 0.523 [0.007] 0.461 [0.008] 0.458 [0.007] 3 trt. sc. & base set. dr Avg. CDW rate 0.962 0.786 0.923 0.762 0.962 0.783 0.962 0.783 0.962 0.783 0.955 0.782 0.959 0.778 0.958 0.778 Avg. HR [MC SE] 0.136 [0.003] 0.141 [0.003] 0.136 [0.003] 0.136 [0.003] 0.136 [0.003] 0.137 [0.003] 0.136 [0.003] 0.136 [0.003] 3 trt. sc. & dr. Z = [\ Avg. CDW rate 0.980 0.876 0.546 0.453 0.973 0.786 0.978 0.817 0.980 0.851 0.924 0.822 0.947 0.738 0.918 0.709 Avg. HR [MC SE] 0.154 [0.003] 0.155 [0.004] 0.140 [0.003] 0.145 [0.003] 0.150 [0.003] 0.154 [0.003] 0.135 [0.003] 0.134 [0.003] Table 5.2 Treatment effect HRs [MC SE] and CDW rates from the base and the < treatment scenarios with ] = < in conjunction with the base set of dropout probabilities and the dropout factor of Z = [\. 2 Average hazard ratio for the imputation strategy of interest with the Monte Carlo standard error across the 100 samples. 3 Shorthand for first treatment scenario with the most extreme treatment factor of b = 3 and the base set of dropout probabilities. 76 5.6 Key results based on the sample hazard ratios We discuss each of the 3 treatment scenarios in turn. Since all of the factors in each of the treatment scenarios presented similar results, the discussion focuses on the results of the most extreme treatment factor, " = 3, as presented in Table 5.2. Figures 5.4 and 5.5 depict the boxplots of the treatment effect hazard ratios from the first treatment scenario in conjunction with the base set of dropout probabilities and with the dropout factor of % = 100. The median hazard ratios and spreads across the samples were nearly identical for almost all of the imputation strategies. The only apparent difference was that the spread of the hazard ratios for the none confirmed strategy was slightly larger than the others under the dropout factor of % = 100. For the second treatment scenario with the base set of dropout probabilities, Figure 5.6 shows that the median hazard ratios and spreads of the treatment effect hazard ratios did not noticeably vary. Under the most extreme dropout factor of % = 100, Figure 5.7 shows that the none confirmed, kNN (( = 3) and the logistic regression imputation strategies generally led to slightly lower hazard ratios than the other strategies (likely for the reasons given in the previous section). Also, the none confirmed strategy had a larger spread of hazard ratios than the other strategies. Yet, the middle 50% of the none confirmed hazard ratios were quite similar to those from the other strategies. Overall, the differences between the imputation strategies were clearly insubstantial on the hazard ratio scale. For the third treatment scenario with the base set of dropout probabilities, Figure 5.8 shows very similar treatment effect hazard ratios across the imputation strategies. On the other hand, for the dropout factor of % = 100, Figure 5.9 shows that the logistic regression strategies gave slightly lower medians and boxes than the other strategies. Yet, those boxes still almost entirely 77 overlapped with those from the other imputation strategies. Additionally, the hazard ratios from the none confirmed strategy were slightly more variable than the others. Although Figure 5.9 shows slight differences in the hazard ratios across the imputation strategies, these differences were likely insubstantial. Therefore, we could not conclude that the treatment effect hazard ratios were clearly different across the imputation strategies. 78 Figure 5.4 First treat. scen. treat. eff. HRs across the samples under + = , and the base set of drop. prob. Figure 5.5 First treat. scen. treat. eff. HRs across the samples under + = , and - = .//. 79 Figure 5.6 Second treat. scen. treat. eff. HRs across the samples under + = , and the base set of drop. prob. Figure 5.7 Second treat. scen. treat. eff. HRs across the samples under + = , and - = .//. 80 Figure 5.8 Third treat. scen. treat. eff. HRs across the samples under + = , and the base set of drop. prob. Figure 5.9 Third treat. scen. treat. eff. HRs across the samples under + = , and - = .//. 81 5.7 Key results from the average CDW rates For the first treatment scenario with " = 3, the average CDW rates were comparable across the imputation strategies, except for the none confirmed strategy which had clearly lower rates; this discrepancy increased substantially as the dropout factor increased. Under both dropout situations displayed, the rates were higher for the placebo group for each imputation strategy, but the differences between the group rates were fairly small across the strategies. These indicated comparable treatment effect hazard ratios across the strategies, which can be seen in that table. In the second treatment scenario with " = 3, the average CDW rates were comparable across the imputation strategies, save for the none confirmed strategy which had slightly lower rates under the base set of dropouts and markedly lower rates under the most extreme dropout factor. The differences were fairly similar for all strategies under the base set of dropout probabilities, suggesting similar hazard ratios, which were observed in Table 5.2. Under the dropout factor of % = 100, the differences for the logistic regression strategies were the most pronounced followed closely by the none confirmed strategy, suggesting stronger treatment effects for those strategies (as can be seen in the table). Additionally, the differences for the kNN strategies became smaller as ( increased, which indicated that the treatment effect hazard ratios would likely become weaker as ( increased (which were observed). In the third treatment scenario with " = 3, the average CDW rates were fairly similar across the imputation strategies, except for the none confirmed strategy, which led to slightly lower rates under the base set of dropouts and markedly lower rates under the most extreme dropouts. The differences in the average CDW rates were very similar under the base set of dropout probabilities, which suggested similar treatment effect hazard ratios across the imputation 82 strategies. Under the dropout factor of % = 100, the noticeably larger differences for the logistic regression strategies indicated slightly stronger treatment effects, as can be seen in the table. Interestingly, the none confirmed difference was more comparable to the all confirmed difference than to the differences for the logistic regression strategies, suggesting that the none confirmed and the all confirmed treatment effect hazard ratios would be similar (as can be seen in Table 5.2). Finally, the pattern in the kNN differences was as in the second scenario. 5.8 Similar performances across the imputation strategies For all considered treatment scenarios, the kNN (( = 3, 5, 7) imputation strategies gave very similar average treatment effect hazard ratios under the base set of dropouts, which indicated that their nearest neighbours consistently led to the same CDW status imputations. However, as the dropout factor became more extreme, the hazard ratios for ( = 3 more closely resembled those from the logistic regression strategies, likely due to reason given in Section 5.5. For larger (, the kNN and the all confirmed strategies gave quite similar treatment effect hazard ratios regardless of the treatment and dropout factors considered, which indicated that many of the IDW instances were followed by CDW in the simulated datasets. Additionally, the all confirmed and Bernoulli strategies had very similar treatment effect hazard ratios regardless of the treatment and dropout factors considered, likely for the reason given in Section 4.11. The logistic and Bayesian logistic regression strategies led to similar average treatment effect hazard ratios under every treatment scenario and dropout factor combination. Under the extreme treatment and dropout factors (save for the first treatment scenario), the logistic regression strategies led to noticeably lower average hazard ratios, likely because smaller 83 fractions of the missing statuses were imputed as CDW in the treatment group, as was explained previously. 5.9 Confusion matrices to assess correct classification The proportions of correct CDW and no CDW classifications in all 100 samples for the base treatment scenario in conjunction with the base set of dropout probabilities and with the dropout factor of % = 100 are presented in Figure 4.20. Figures 5.10 to 5.12 present very similar results for all 3 treatment scenarios (with the most extreme treatment factor of " = 3): all of the depicted imputation strategies led to nearly or exactly 1 for the proportion of correct CDW classifications and nearly or exactly 0 for the proportion of correct no CDW classifications. We only displayed 1 plot for the kNN strategies as all 3 ( values gave very similar results. The only notable difference from Figure 4.20 was the slightly increased variability in the proportions of correct no CDW classifications from the second and third treatment scenarios. This was likely due to the lower proportions of treatment patients that were imputed to have CDW in these scenarios, as explained in Section 5.5. Otherwise, the same lines of reasoning used to explain Figure 4.20 may be used to explain Figures 5.10 to 5.12. 84 Figure 5.10 Proportions of correctly classified CDW and no CDW patients across the .// samples from the first treatment scenario with + = , and the base set of dropout probabilities (left) and the dropout factor of - = .// (right). 85 Figure 5.11 Proportions of correctly classified CDW and no CDW patients across the .// samples from the second treatment scenario with + = , and the base set of dropout probabilities (left) and the dropout factor of - = .// (right). 86 Figure 5.12 Proportions of correctly classified CDW and no CDW patients across the .// samples from the third treatment scenario with + = , and the base set of dropout probabilities (left) and the dropout factor of - = .// (right). 5.10 Summary Three transition matrix-based treatment scenarios that were adapted from Healy et al. (2011) were used to construct complete simulated datasets of 2000 patients. In the scenarios, we modified portions of the treatment group stationary transition matrix (Table 3.9) while leaving the placebo group stationary transition matrix (Table 3.8) unchanged to induce stronger treatment effects. The 3 matrix-based treatment scenarios are: 87 1. The probabilities of progression are decreased by a constant scaling factor, while the probabilities of improvement are unchanged. 2. The probabilities of progression are unchanged, while the probabilities of improvement are increased by a constant scaling factor. 3. The probabilities of progression are decreased by a constant scaling factor, while the probabilities of improvement are increased by the same scaling factor. Once a complete simulated dataset was generated, the dropout factors considered earlier were implemented on 100 samples of 300 patients. Then, the imputation strategies considered earlier were used to fill in the CDW statuses of patients with missing confirmatory visits (and no prior CDW). The key result was that the second and third treatment scenarios with the most extreme treatment and dropout factors led the kNN (( = 3) and the logistic regression imputation strategies (as well as the none confirmed strategy in the second scenario) to have slightly stronger treatment effect hazard ratios than the other strategies. This may have been because the treatment group patients had higher probabilities of attaining lower EDSS states in those scenarios. Therefore, a smaller fraction of such patients were imputed to have CDW when treatment group was taken into account. In the second scenario, it was plausible that the none confirmed strategy resulted in stronger treatment effect hazard ratios because that scenario led to the greatest median number of CDW statuses to impute in the treatment group. Imputing all of those statuses as no CDW likely strengthened the positive treatment effect. Overall, since the largest difference in the average hazard ratios across the imputation strategies was only about 0.072, these differences were unlikely to be of practical importance. 88 On the other hand, in the first scenario, no combination of treatment and dropout factors led to clearly different treatment effect hazard ratios across the imputation strategies. We suspected this was because it was highly likely that treatment group patientsâ IDW instances would be confirmed as CDW in this scenario. If a treatment patient attained a higher state than at baseline, the high probabilities of remaining in the same state in the modified treatment group transition matrix would almost guarantee confirmation. The very high probabilities of CDW given IDW in both groups coupled with the low number of imputations in the treatment group made it highly unlikely that the imputation strategies would lead to clearly different treatment effect hazard ratios in this scenario. In the next chapter, we investigate a model-based approach to generate the follow-up EDSS states that was more tailored to the analysis dataset. As in the transition matrix-based investigation, our aim was to observe whether any of the model-based treatment factors in conjunction with any of the dropout factors led to clearly different treatment effect hazard ratios across the imputation strategies. 89 Chapter 6: Differing Treatment Effects Based on a Proportional Odds Model In this chapter, our primary objective was to observe whether any of the different imputation strategies led to clearly different treatment effect hazard ratios when we magnified the treatment effect by using a model-based approach. We first fit a series of proportional odds models to the analysis dataset to find a suitable model structure. Then, in the simulation of the follow-up EDSS states, model-based treatment factors were used to magnify the treatment effect through scaling a treatment predictor in the resulting proportional odds model. 6.1 Introducing the proportional odds model A proportional odds model is the most common type of ordinal logistic regression model. The model is based on the log odds of being in or below an ordered response category versus being in any of the higher categories. Let 3 indicate the ordinal outcome variable, 4 be a 5 Ã 1 vector of covariates, and 7 be a 5 Ã 1 vector of coefficients. Suppose that 58 = 9 3 = : 4) so that the cumulative probabilities for the ordinal response categories are 9< = 9(3 â¤ ( | 4) = 58 <8@A for ( = 1, 2, â¦, C. Then, the general form of this model is logit 9< = log IJAKIJ = L< â 4N7 (( = 1, 2, â¦, C â 1) in which L< are the C â 1 strictly increasing intercept parameters that indicate the cutoffs (boundaries) between the categories of the ordinal response (McCullagh 1980; Christensen 2018). These intercept parameters vary depending on each cumulative logit, while the regression coefficients are the same regardless of the cumulative logit considered. McCullagh (1980) referred to this as the proportional odds model because of the property log I OP< 4Q)/I(OS<|4Q)I OP< 4T)/I(OS<|4T) = logit[9 3 â¤ ( 4Q)] â logit[9 3 â¤ ( 4T)] = (4T â 4Q)â²7, 90 which means that the log cumulative odds ratio for any 4Q and 4T is proportional to the 4Q and 4T distance (McCullagh 1980). 6.2 The mixed-effects proportional odds model When ordinal response data is clustered with multiple covariates, then the mixed-effects proportional odds model may be used to find relationships between those variables. Such models include both fixed and random effects, where the fixed effects aim to capture the relationship between a predictor and the response variable, while the random effects aim to capture specific aspects of the random variability in the response variable. In clinical trials with repeated measurements over time for each patient, the repeated measurements within each patient form natural clusters and are expected to be more similar than measurements between patients. This within-patient correlation can be incorporated by including a random intercept for each of the patients. For our purposes, let X = 1, â¦, Y indicate the patient and Z = 1, â¦, [ indicate the time point. Suppose that 5]< = 9(3] = () so that the cumulative probabilities for the ordinal response category ( are 9]< = 9(3] â¤ () = 5]8<8@A (where the dependence on covariates is suppressed in the notation). Let 4^_ be a 5 Ã 1 vector of covariates, 7 be a 5 Ã 1 vector of coefficients, `^_ be the random effects design vector, and a^ be the (patient-specific) vector of random effects from the density function b a^ . Then, the general form of this model is logit 9]< = log 9]<1 â 9]< = L< â (4^_N 7 + `^_N a^) for strictly increasing intercept parameters L< where ( = 1, 2, â¦, C â 1 (Hedeker and Gibbons, 2006). 91 For maximum likelihood estimation, let d^ be the ordinal response vector for patient X where there are [\trepeated observations for that patient. If we denote e]< = 1 3] = ( 0 fgâijkXli, then the contribution of patient X to the conditional likelihood is 9 d^ a^ = m<@Ano]@A 9 3] = ( a^)poqJ where 9 3] = ( a^) = 9]< â 9],<KA with 9]s = 0. The marginal density of d^ is the integral of this conditional likelihood contribution â d^ = 9 d^ a^ b a^ ta^. It follows that the marginal log-likelihood for all Y patients is simply log u = log â d^v@A . The most common choice of wxy(/, za) for the random effect density function, b â , will be utilized throughout what follows. 6.3 The response variable and the potential covariates Our ordinal response variable was EDSS state, where the states range from A to F as specified previously. Our possible covariates included - gji}g was set to 1 if patient X belonged to the treatment group and 0 if that patient was in the placebo group - gX:i] indicated the visit month Z of patient X and was implemented as a continuous covariate with values 3, 6, 9, â¦, 36 - Ã}li indicated the initial EDSS state (A, B, C, D, or E) of patient X - lià was set to 0 if patient X was female and to 1 if the patient was male. - }bi denoted patient Xâs age in years at entry into the study 92 - jiÃ}5li was set to 1 if patient X had a relapse in the 2 years before the study and 0 otherwise The models to be considered also included a patient-specific random effect, 9ÃÃ. We included a random patient effect for each patient because the repeated EDSS states were expected to be closely related (and correlated) within each patient. The cumulative probabilities for a mixed-effects proportional odds model with main effects for all of the above covariates can be expressed as: logit 9]< = L< â ÃAgji}g â ÃÃgX:i] â ÃÃ¡Ã}li â ÃÃ lià â ÃÃ¢}bi â ÃäjiÃ}5li â 9ÃÃ where X = 1, â¦, 714 for the patients, Z = 1, 2, 3, â¦, 12 for the post-baseline visits, and ( = 1, 2, â¦, 5 for the EDSS state categories A, B, C, D, and E. Although baseline EDSS state had more than 2 categories, we used a single variable as shorthand for the predictor variables that represented its categories for simplicity. In proportional odds models, the usual convention is to include negative signs for each of the regression coefficients to facilitate a more natural interpretation: a larger predictor value is indicative of a response that is in a higher category (in our case, a higher EDSS category). Hence, a negative treatment regression coefficient, ÃA, (where the placebo group is the reference group) indicates that there is a smaller log odds of attaining higher EDSS states for patients in the treatment group, which would be expected for a beneficial treatment effect. 6.4 Implementation of the proportional odds model in R We implemented our proportional odds models by using the clmm2 function from the ordinal package in R. Since maximum likelihood calculations can be difficult for proportional odds 93 models with both fixed and random effects (particularly for those with more than 1 random effect), different estimation methods are available. The clmm2 function can use the Gauss-Hermite quadrature method to approximate the marginal likelihood function for parameter estimation. The Gauss-Hermite quadrature method uses the approximation % Ã iKåçtà â % Ã k8@AèêKÃª where % Ã is an arbitrary function, : is the number of nodes (ie. points that are sampled), Ã are the Hermite polynomial roots, and k are the associated weights. For more details on Hermite polynomials, refer to Abramowitz and Stegun (1972). For more details on the Gauss-Hermite quadrature method, refer to Haneuse (2019). In the clmm2 function, the default number of quadrature nodes is 1, which corresponds to a Laplace approximation (see Liu and Pierce 1994). The Gauss-Hermite quadrature method has a trade-off between approximation accuracy and computational time. Typically, the larger the number of nodes used, the greater the accuracy and computational time (Haneuse 2019). Since the purpose of fitting the proportional odds model to the analysis dataset was to construct a complete simulated dataset that reflected the structure in the analysis dataset, it was not critical to focus on accuracy. Hence, we opted for the default of 1 node. 6.5 Determining the predictors in the proportional odds model We performed manual forward selection by using a series of likelihood ratio tests to find which predictors were important in a proportional odds model fitted to the analysis dataset. Note that our manual forward selection was conducted using only the proposed fixed effects for the model. The initial model had only treatment group as a predictor. A likelihood ratio test was performed to assess whether each additional predictor significantly improved the model fit. 94 Hence, the order in which the predictors were tested was important. The order used was based on our judgement and prior knowledge of the most important predictors. First, we chose to inspect the addition of the time predictor because patients were likely to move to higher EDSS states as time progressed as SPMS is characterized by relatively steady disease progression. Then, baseline EDSS state was considered for the same reason as in the logistic regression model for the presence of CDW. Thirdly, prior relapse was considered because a relapse was likely indicative of further disease progression and, hence, a propensity to move to higher EDSS states. Next, sex was considered because there was likely an association with disease progression as there were more women in the Schering placebo group than in the treatment group and the baseline EDSS states were generally higher in that group (see Table 2 in Kappos et al. 1998 for the baseline characteristics of the Schering dataset). Finally, age was considered because older patients likely had more time for the disease to progress and were more susceptible to such progression. The likelihood ratio test for time provided evidence that time improved the model in addition to treatment (p-value < 0.001). Baseline EDSS state further improved the model (p-value < 0.001), but prior relapse did not (p-value 0.109). Sex provided further improvement (p-value 0.007), but age did not (p-value 0.750). Since there were many orders to test the variables that we did not use in our manual stepwise selection, we also used automatic stepwise selection, combining forward and backward selection to choose a model. For the same reason as in Section 2.7, we set ( = 3.841 to make the threshold more stringent at 0.05. The final model included the same predictors as identified by the likelihood ratio tests. 95 Since we anticipated a positive treatment effect that may increase over time, we next conducted a likelihood ratio test for the treatment by time interaction. Although the p-value of 0.055 was not quite below ì = 0.05, we opted to include this interaction term for consideration in subsequent work. 6.6 Assessing the proportional odds assumption The key assumption of the proportional odds model is the proportionality assumption. Graphically, the lines of the cumulative logits should be parallel across the different levels of each covariate. We chose to visualize the proportional odds assumption for each covariate that was to be included in the model by plotting the empirical cumulative logit values for each of the ordinal categories versus the discrete covariate levels. Table 6.1 shows an example of how the empirical cumulative logit values (for the response of EDSS state at 3 months) were obtained for the placebo and the treatment groups. The same approach was used for each of the discrete predictors in the model. Response â EDSS state at 3 months Placebo Group A B C D E F Count 45 70 70 100 68 2 Cumulative Odds 45/310 115/240 185/170 285/70 353/2 Logit â1.930 â0.736 0.085 1.404 5.173 Treatment Group Count 66 66 73 97 56 1 Cumulative Odds 66/293 132/227 205/154 302/57 358/1 Logit â1.491 â0.542 0.286 1.667 5.881 Table 6.1 Empirical cumulative logits at , months for the groups from the analysis dataset. 96 We begin with the treatment by time interaction predictor, for which we obtained separate plots for the treatment predictor at every visit from 3 to 36 months as shown by Figures 6.1 and 6.2. The key observation was that the lines on the plots of logit values versus treatment group were approximately parallel for every visit. Hence, it seemed reasonable to proceed with the proportionality assumption for the treatment by time predictor. Figure 6.1 The proportionality assumption for the treatment predictor at ,, î, ï, .Ã±, .Ã³, and .Ã² months. 97 Figure 6.2 The proportionality assumption for the treatment predictor at Ã±., Ã±ô, Ã±ö, ,/, ,,, ,î months. Figure 6.3 depicts the plots that we used to assess the proportionality assumption for the remaining 4 predictors of treatment, baseline EDSS state, sex and time. The proportional odds assumption seemed reasonable for treatment since the 5 lines on the plot for that predictor were approximately parallel. Also, the treatment group had higher cumulative logit values than the placebo group for each category. Since larger log odds indicate higher cumulative probabilities and greater chances of being in lower EDSS states, this pattern was consistent with a beneficial treatment effect. The 5 lines on the plot for the sex predictor were also approximately parallel. In 98 contrast, there were departures from parallelism in the baseline EDSS state plot. However, the departures appeared modest, so it seemed reasonable to proceed with the proportional odds assumption for baseline EDSS state. Similarly, there was a lack of parallelism for the time predictor. In particular, the line corresponding to cutoff 5 had some spikes at earlier visits (before 18 months) after which it became more steadily decreasing. Presumably, the spikes were due to the odds not being very well-determined because the denominator of that odds was small as there were very small counts in the last category (as can be seen in Table 6.1). That aside, any other departures were minor so it seemed reasonable to proceed with the proportional odds assumption for time. Figure 6.3 The proportionality assumption for the treatment, baseline EDSS state, sex, and time predictors. 99 6.7 Fitting proportional odds models with the treatment by time interaction term Based on the preceding sections, we fit the following proportional odds model to the analysis dataset: logit 9]< = L< â ÃAgji}g â ÃÃgX:i] â ÃÃ¡Ã}li â ÃÃ lià â ÃâgX:i] â gji}g â 9ÃÃ. Figure 6.4 shows the summary of this model from R. There are 2 treatment-based predictors in this model: the treatment and the treatment by time interaction terms. Since it would be difficult to develop a systematic approach to magnify the treatment effect using both coefficients, we wanted to simplify the model. The model fit suggests that the treatment by time predictor is significant (p-value 0.007) whereas the treatment predictor only approaches significance (p-value 0.078). Hence, it seemed reasonable to drop the treatment predictor and to fit the reduced model: logit 9]< = L< â ÃAgX:i] â ÃÃÃ}li â Ãálià â ÃÃ gX:i] â gji}g â 9ÃÃ. Figure 6.5 shows the model summary from R. In this model, it is easy to modify the treatment by time interaction term to magnify the treatment effect. The model can be re-expressed as an equation for the placebo group logit 9]< = L< â ÃAgX:i] â ÃÃÃ}li â Ãálià â 9ÃÃ (6.1) and an equation for the treatment group logit 9]< = L< â (ÃA + ÃÃ )gX:i] â ÃÃÃ}li â Ãálià â 9ÃÃ, (6.2) where the only difference is the âÃÃ gX:i] term in the treatment group model, which arose from the interaction term. 100 Figure 6.4 Proportional odds model including both the treatment and treatment by time predictors. 101 Figure 6.5 Proportional odds model including only the treatment by time predictor. 6.8 Generation of the complete simulated datasets Using the models represented by Equations (6.1) and (6.2) with the parameter estimates as in Figure 6.5, a complete simulated dataset of 2000 patients was generated. Simulated patients were assigned baseline EDSS states and to the treatment groups as in previous chapters. Patients were randomly assigned a sex based on the proportions of male and female patients in the analysis dataset. Then, a random effect was generated for each patient based on a normal distribution with a mean of 0 and the standard deviation as specified in Figure 6.5. To generate the follow-up EDSS states for each patient in the simulated dataset, the following 2-step procedure was used: 102 1. Each patientâs covariate values and random effect were used to obtain their cumulative probabilities at a particular visit. 2. A uniform number was randomly generated. The cumulative probability range in which this uniform random number fell determined the patientâs EDSS state at this visit. To magnify the positive treatment effect, we multiplied the treatment by time coefficient (ÃÃ = â0.0167) by positive factors greater than 1. In addition to the base case, where all coefficients were as in Figure 6.5, there were 4 additional cases, where we multiplied the interaction term coefficient (ÃÃ ) by the factors 5, 10, 20, and 30. We shall refer to the base case treatment factor as the (base) treatment factor of 1 and the other 4 factors as the model-based treatment factors. We stopped at the factor of 30 because the datasets generated from larger factors had very low treatment effect hazard ratios (around or below 0.1) in the Cox model for time to first CDW, which were not likely realistic representations of MS clinical trials. Using the above approach, we generated complete simulated datasets of 2000 patients for each of the treatment factors. Table 6.2 shows the estimated treatment effect hazard ratios for those datasets. In comparison to the base treatment factor, all of the other factors led to substantially lower hazard ratios and their 95% confidence intervals had no overlap with the base case. These lower hazard ratios indicated greater treatment effects than in the base case; for instance, the hazard ratio was below 0.16 for the most extreme factor of 30, which indicated a very exaggerated treatment effect in comparison to the base case hazard ratio of about 0.85. Treat. factor 1 5 10 20 30 HR [95% CI] 0.850 [0.759, 0.951] 0.526 [0.463, 0.597] 0.333 [0.288, 0.385] 0.203 [0.171, 0.241] 0.155 [0.128, 0.186] Table 6.2 Treatment effect HRs and ïó% confidence intervals for the complete simulated datasets. 103 Following the generation of the complete simulated datasets, 100 samples were obtained from each and dropouts were imposed as before. Then, the same imputation strategies as in previous chapters were used to fill in the CDW statuses that required imputation. Under each imputation strategy, time to first CDW was analyzed using the same Cox model from the previous chapters. 6.9 The counts of patients requiring CDW status imputation The counts of patients requiring CDW status imputation in the 100 samples under each of the treatment factors in conjunction with the most extreme dropout factor of % = 100 are shown in Figures 6.6 and 6.7. For all treatment factors, the boxplot counts appeared large enough to exert an influence on the treatment effect as the median number of CDW status imputations in a sample was about 145, 120, 114, 102, and 97 for the treatment factors of 1, 5, 10, 20, and 30, respectively. For every treatment factor, there was a clear trend that the 3 month visit had the largest number of patients requiring CDW status imputation (as its median was about 39 patients), which agreed with our expectation that the largest number of imputations would be at the earliest visit under the most extreme dropout factor (based on Section 4.4). The remaining visits had very low numbers of patients (as their median numbers were all less than 15). The pattern in the number of CDW statuses requiring imputation was very similar in all 5 plots, although the number of imputations decreased slightly for months 6 to 36 as the treatment factor increased. This likely occurred because there were fewer treatment patients who had IDW instances when the positive treatment effect was magnified. 104 Figure 6.6 Patients requiring CDW status imputation in the .// samples under the treatment factor of . and the dropout factor of - = .//. 105 Figure 6.7 Patients requiring CDW status imputation in the .// samples under the treatment factors of Ã³, ./, Ã±/, and ,/ (as indicated by ., Ã±, ,, and Ã´) and the dropout factor of - = .//. Since the overall counts seemed large enough so that the hazard ratios may be differentially impacted by the imputation strategies, we separated these boxplots by treatment group in Figures 6.8 to 6.11 to see whether the number of imputations differed by group. Note that increasing the treatment factor had no effect on the placebo probabilities. Therefore, all 5 plots for the placebo group (Figures 6.8 and 6.9) were identical save for random variation. 106 Although the same pattern from Figures 6.6 to 6.7 was somewhat evident in both groups, the numbers of imputations by visit were clearly lower in the treatment than in the placebo group for the non-base treatment factors. The differences between the groups were most apparent under the most extreme treatment factor of 30 as the medians and spreads for the 12 to 36 month imputations were all 0 in the treatment group. We anticipated that there would be a lower number of CDW statuses to impute in the treatment group because we expected a substantial positive treatment effect under the most extreme treatment factor. Figure 6.8 Placebo patients requiring CDW status imputation in the .// samples under the treatment factor of . and the dropout factor of - = .//. 107 Figure 6.9 Placebo patients requiring CDW status imputation in the .// samples under the treatment factors of Ã³, ./, Ã±/, and ,/ (as indicated by ., Ã±, ,, and Ã´) and the dropout factor of - = .//. 108 Figure 6.10 Treatment patients requiring CDW status imputation in the .// samples under the treatment factor of . and the dropout factor of - = .//. 109 Figure 6.11 Treatment patients requiring CDW status imputation in the .// samples under the treatment factors of Ã³, ./, Ã±/, and ,/ (as indicated by ., Ã±, ,, and Ã´) and the dropout factor of - = .//. 6.10 Overall results for the average treatment effect hazard ratios For each of the treatment factors, Table 6.3 shows the treatment effect hazard ratios from the complete dataset and the average hazard ratios for the imputation strategies obtained under the dropout factor of % = 100. Overall, there was some separation in the average hazard ratios for all model-based treatment factors. In particular, the kNN (( = 3, 5, 7) and the logistic regression strategies typically led to lower average hazard ratios than the all confirmed and the 110 Bernoulli strategies. When we accounted for the MC SEs, it became evident that the kNN (( = 3, 5, 7) and the logistic regression average hazard ratios were clearly stronger than the rest for the treatment factor of 5 and beyond. This finding was unsurprising because we expected that CDW would occur less often in the treatment group due to the strong propensity for treatment patients to remain in lower EDSS states under the more extreme treatment factors. Thus, a smaller fraction of the missing statuses in the treatment group was likely imputed as CDW than in the placebo group under the strategies that incorporated treatment group, which led to the stronger treatment effects. Yet, even under the most extreme treatment factor of 30, the average hazard ratios differed by 0.085 units at most, indicating that the differences across the imputation strategies were not of practical importance. Interestingly, the none confirmed strategy was comparable to the kNN (( = 3, 5, 7) and the logistic regression strategies for the base treatment factor, while for the treatment factor of 10 and beyond it behaved more like the all confirmed and Bernoulli strategies. The none confirmed strategy was likely to lead to a stronger treatment effect under lower treatment factors due to the larger numbers of statuses to impute in the treatment group in such cases, whereas there were fewer statuses to impute under the larger treatment factors. Another interesting observation was that the kNN strategies with larger ( led to lower average hazard ratios than the logistic regression strategies for the treatment factors of 10 and beyond. This may have been because there was a trend that, as the treatment factor increased, the number of IDW instances (and, hence, CDW instances) in the treatment group decreased. So, when the treatment factor was greatly magnified, the differences in the number of CDW instances in the groups became more pronounced. This was supported as the average hazard ratios increased across the increasing ( values for the base treatment factor (and for the treatment 111 factor of 5 to a lesser extent), suggesting that the neighbours were largely comprised of CDW instances regardless of the treatment group. As the treatment factor increased from 10 to 30, the hazard ratios decreased across the increasing ( values, suggesting that the neighbours from the treatment group had fewer CDW instances, which led to the more pronounced treatment effects. In comparison to the complete dataset hazard ratio, the strategies that did not incorporate treatment group (the all confirmed, none confirmed, and Bernoulli strategies) led to systematically higher average treatment effect hazard ratios under every treatment factor. Those strategies led to rather poor recovery of the âtrueâ treatment effect hazard ratio in each case. Yet, the all confirmed and Bernoulli average hazard ratios generally became closer in the absolute sense to the âtrueâ treatment effect hazard ratio as the treatment factor became more extreme. This may have been because there was a comparable number of CDW statuses to impute in the groups under the lower treatment factors, while there was a greater number of statuses to impute in the placebo group under the more extreme treatment factors. The more statuses that were imputed as CDW in the placebo group, the stronger the treatment effect. For that same reason, the none confirmed average hazard ratios were generally closer in absolute value to the âtruthâ under the low treatment factors (the base factor and the treatment factor of 5) than they were under the higher treatment factors. Although the strategies that incorporated treatment group (the kNN and the logistic regression strategies) led to slightly higher average hazard ratios than the complete dataset hazard ratio, they were almost always closer to the âtruthâ than the strategies that did not utilize treatment group. Therefore, the kNN and the logistic regression strategies led to better recovery of the âtrueâ treatment effect overall. 112 Treatment factor Imputation strategy4 Single imputation Multiple imputation Complete Dataset All confirmed None confirmed kNN (ú = ,) kNN (ú = Ã³) kNN (ú = ö) Bernoulli Log. Reg. Baye. Log. Reg. Base trt. f. 0.850 [0.759, 0.951] 0.941 [0.015] 0.865 [0.022] 0.865 [0.017] 0.863 [0.018] 0.879 [0.017] 0.929 [0.015] 0.869 [0.018] 0.867 [0.018] Ã³ trt. f. 0.526 [0.463, 0.597] 0.636 [0.011] 0.602 [0.017] 0.548 [0.013] 0.554 [0.015] 0.555 [0.015] 0.631 [0.010] 0.577 [0.013] 0.581 [0.012] ./ trt. f. 0.333 [0.288, 0.385] 0.430 [0.007] 0.424 [0.013] 0.364 [0.010] 0.352 [0.010] 0.349 [0.010] 0.430 [0.007] 0.374 [0.010] 0.391 [0.009] Ã±/ trt. f. 0.203 [0.171, 0.241] 0.283 [0.006] 0.284 [0.010] 0.224 [0.008] 0.212 [0.008] 0.208 [0.008] 0.282 [0.006] 0.218 [0.007] 0.233 [0.007] ,/ trt. f. 0.155 [0.128, 0.186] 0.235 [0.005] 0.218 [0.008] 0.173 [0.007] 0.158 [0.006] 0.150 [0.006] 0.230 [0.005] 0.170 [0.005] 0.178 [0.005] Table 6.3 Treatment effect HRs [ïó% CI] from the complete dataset and the averaged HRs [MC SE] from the .// samples after the imposition of the most extreme dropout factor of - = .//. 6.11 Key results for the most extreme model-based factors Table 6.4 presents the results for the base treatment factor in conjunction with the base set of dropout probabilities and the dropout factor of % = 100 as well as for the treatment factor of 30 in conjunction with the dropout factor of % = 25 and beyond (to show some of the dropout factors leading up to the clearest separation of the treatment hazard ratios). Our focus is on the treatment factor of 30 to examine in more detail if the differences across the imputation strategies were insubstantial on the hazard ratio scale in the most extreme case considered. 4 Average hazard ratio for the imputation strategy of interest with the Monte Carlo standard error across the 100 samples. 113 Imputation strategy Single imputation Multiple imputation All confirmed None confirmed kNN (" = %) kNN (" = &) kNN (" = ') Bernoulli Log. Reg. Baye. Log. Reg. P T P T P T P T P T P T P T P T Base trt. f. & base set dr. Avg. CDW rate 0.861 0.804 0.758 0.696 0.850 0.797 0.855 0.798 0.857 0.800 0.821 0.763 0.825 0.757 0.800 0.742 Avg. HR [MC SE]5 0.881 [0.014] 0.877 [0.016] 0.886 [0.015] 0.882 [0.014] 0.880 [0.015] 0.880 [0.015] 0.870 [0.015] 0.882 [0.015] Base trt. f. & dr. 5 = 677 Avg. CDW rate 0.953 0.926 0.368 0.322 0.851 0.758 0.885 0.786 0.903 0.816 0.772 0.740 0.763 0.679 0.732 0.650 Avg. HR [MC SE] 0.941 [0.015] 0.865 [0.022] 0.865 [0.017] 0.863 [0.018] 0.879 [0.017] 0.929 [0.015] 0.869 [0.018] 0.867 [0.018] %7 trt. f. & base set of dr.6 Avg. CDW rate 0.859 0.433 0.760 0.424 0.846 0.430 0.852 0.428 0.854 0.427 0.811 0.429 0.821 0.426 0.799 0.428 Avg. HR [MC SE] 0.155 [0.004] 0.183 [0.005] 0.157 [0.004] 0.155 [0.004] 0.155 [0.004] 0.168 [0.004] 0.164 [0.004] 0.172 [0.004] %7 trt. f. & dr. 5 = 8& Avg. CDW rate 0.911 0.536 0.592 0.344 0.863 0.432 0.882 0.437 0.892 0.430 0.764 0.447 0.796 0.393 0.763 0.397 Avg. HR [MC SE] 0.180 [0.005] 0.197 [0.005] 0.155 [0.005] 0.151 [0.005] 0.147 [0.005] 0.186 [0.005] 0.156 [0.005] 0.168 [0.005] %7 trt. f. & dr. 5 = &7 Avg. CDW rate 0.932 0.614 0.490 0.289 0.862 0.444 0.882 0.444 0.897 0.435 0.741 0.473 0.777 0.388 0.744 0.386 Avg. HR [MC SE] 0.206 [0.005] 0.207 [0.006] 0.163 [0.006] 0.157 [0.006] 0.151 [0.006] 0.206 [0.005] 0.161 [0.005] 0.169 [0.005] %7 trt. f. & dr. 5 = 677 Avg. CDW rate 0.951 0.706 0.357 0.217 0.838 0.452 0.869 0.439 0.903 0.438 0.705 0.503 0.750 0.388 0.717 0.389 Avg. HR [MC SE] 0.235 [0.005] 0.218 [0.008] 0.173 [0.007] 0.158 [0.006] 0.150 [0.006] 0.230 [0.005] 0.170 [0.005] 0.178 [0.005] Table 6.4 Treatment effect HRs and CDW rates under the base and the treatment factor of %7 in conjunction with the base set of dropouts and the dropout factor of 5 = 677. 5 Average hazard ratio for the imputation strategy of interest with the Monte Carlo standard error across the 100 samples. 6 Treatment effect factor of 30 in conjunction with the base set of dropouts. 114 The treatment effect hazard ratios for the base treatment factor in conjunction with the base set of dropout probabilities are depicted in Figure 6.12. These average hazard ratios were all nearly the same as the lowest was 0.870 under the logistic regression imputation strategy, while the highest was 0.886 under the kNN (& = 3) imputation strategy. When the MC SEs were taken into account, the average hazard ratios were not clearly differentiable across the strategies. Figure 6.13 shows the treatment effect hazard ratios for the base treatment factor in combination with the dropout factor of * = 100. That figure and Table 6.4 both support that the hazard ratios were quite similar, especially when the MC SEs were taken into account. Even though the hazard ratios for the all confirmed and Bernoulli strategies were slightly larger than the others, the MC SEs from all of the imputation strategies indicated that their hazard ratios were not clearly different. Hence, we concluded that there was no clear separation across the imputation strategies. Next, we discuss the results for the treatment factor of 30. Figure 6.14 depicts the treatment effect hazard ratios for the base set of dropout probabilities. Although most of the imputation strategies had very similar hazard ratios, the none confirmed strategy led to slightly higher hazard ratios. We believe this was due to there being more CDW statuses to impute in the placebo group than in the treatment group in this case. Hence, when all of the missing CDW statuses were set to no CDW, the treatment effect became weaker. Yet, since the largest difference in the average hazard ratios was 0.028 (refer to Table 6.4), they were not regarded as noticeably different across the imputation strategies. Figure 6.15 depicts the treatment effect hazard ratios for the treatment factor of 30 in combination with the dropout factor of * = 100. For the most extreme treatment factor, this was the clearest case where the hazard ratios from the kNN (& = 3, 5, 7) and the logistic 115 regression imputation strategies were clearly lower than those from the other imputation strategies (save for perhaps the none confirmed strategy). While the spread for the none confirmed strategy extended down to the stronger hazard ratios from the kNN (& = 3, 5, 7) and the logistic regression strategies, it also extended up to the weaker hazard ratios from the remaining strategies. Therefore, we could not say that the none confirmed strategy led to clearly distinct hazard ratios from the other strategies based on its spread of hazard ratios. Under the treatment factor of 30 in combination with the dropout factors of * = 25, 50, similar patterns in the average hazard ratios emerged as were observed under the most extreme dropout factor of * = 100. Most notably, the kNN (& = 3, 5, 7), logistic regression, and Bayesian logistic regression strategies led to slightly lower average hazard ratios than the all confirmed and Bernoulli strategies. Yet, as was previously noted for the dropout factor of * = 100, these differences were unlikely to be of practical importance. 116 Figure 6.12 Treat. eff. HRs across the samples under the base treat. factor and the base set of drop. prob. Figure 6.13 Treat. eff. HRs across the samples under the base treat. factor and . = /00. 117 Figure 6.14 Treat. eff. HRs across the samples under the treat. factor of 10 and the base set of drop. prob. Figure 6.15 Treat. eff. HRs across the samples under the treat. factor of 10 and . = /00. 118 6.12 Key results from the average CDW rates For the base treatment factor, the average CDW rates were fairly comparable across the imputation strategies except for the clearly lower rates from the none confirmed strategy, which decreased considerably as the dropout factor increased. For the base set of dropout probabilities and the most extreme dropout factor, the rates were slightly larger in the placebo group than in the treatment group for every imputation strategy, which indicated a slightly positive treatment effect for every strategy. Under the base set of dropout probabilities, all of the differences in the average CDW rates were quite similar, which suggested that their treatment effect hazard ratios would be comparable, as was observed in Table 6.4. In contrast, the average CDW rates under the dropout factor of * = 100 were more variable likely due to the greater numbers of CDW statuses to impute with more dropouts. The largest differences in the CDW rates were observed for the kNN and the logistic regression strategies, which supported their more pronounced treatment effect hazard ratios. In contrast, the all confirmed, none confirmed, and Bernoulli strategies had noticeably smaller differences, which supported their less pronounced treatment effect hazard ratios. For the most extreme treatment factor of 30, as the dropout factor increased, the none confirmed strategy rates decreased, while the all confirmed strategy rates increased (from Table 6.4). The average CDW rates were fairly similar across the other imputation strategies. For all imputation strategies, the average CDW rates were clearly higher in the placebo group than in the treatment group. Under the base set of dropout probabilities, the differences were substantially larger for this extreme treatment factor than for the base treatment factor, which suggested stronger positive treatment effects in this case, as were observed. The none confirmed strategy led to the smallest difference between the group rates (mainly due to a much lower 119 average CDW rate for the placebo group than in the other strategies), which indicated a weaker treatment effect hazard ratio, as was observed in Table 6.4. Under the dropout factor of * = 100, the largest differences in the rates between the groups were again observed for the kNN and the logistic regression strategies, indicating stronger treatment effect hazard ratios for those strategies. In contrast, the differences for the other strategies were smaller, suggesting less pronounced treatment effects, as were observed. 6.13 Similar performances across the imputation strategies Under the more extreme treatment and dropout situations, the average treatment effect hazard ratios for the kNN (& = 3, 5, 7) strategies generally increased in strength as & increased (from Tables 6.3 and 6.4). The more pronounced treatment effects for larger & suggested that many of the IDW instances did not lead to CDW in the treatment group, as was noted in Section 6.10. Additionally, as discussed in that section, the kNN results were somewhat similar to the results for the logistic regression strategies. The all confirmed and Bernoulli strategies gave quite similar hazard ratios because the majority of IDW were likely to lead to CDW when the imputation strategy did not incorporate treatment group. As was observed in the previous chapters, the logistic regression approaches had quite similar average treatment effect hazard ratios. The minor variability in their results was likely due to the randomness introduced by multiple imputation. Also, for the non-base treatment factors and particularly for the larger dropout factors, the logistic regression strategies gave average hazard ratios that were more similar to the kNN (& = 3, 5, 7) strategies than to the 120 strategies that did not incorporate treatment group (see Table 6.3), likely for the reason given in Section 6.10. 6.14 Confusion matrices to assess correct classification Figure 6.16 depicts the proportions of correct CDW and no CDW classifications for the base treatment factor in conjunction with the base set of dropout probabilities and with the dropout factor of * = 100. Figure 6.17 depicts the same information for the treatment factor of 30. Only the results for the base set of dropout probabilities and the most extreme dropout factor are shown because the trends were comparable for the intermediate dropout factors. The key difference between the matrix-based and the model-based investigations was that the post-baseline EDSS states were generated differently, which seemed unlikely to greatly impact the proportions of correct classifications. Yet, there were some noticeable differences in the proportions for the multiple imputation strategies. The medians for all of the multiple imputation strategies in the model-based investigation were lower for CDW (between 0.5 to 0.6) and higher for no CDW (between 0.3 to 0.5) regardless of the dropouts (save for the Bayesian logistic regression strategy under the base set of dropout probabilities). Interestingly, the Bayesian logistic regression strategy medians were around 0.5 and 0.7 (for CDW and no CDW, respectively) under the base set of dropout probabilities. Both the lower medians for CDW classifications and the higher medians for no CDW classifications suggested that there were usually higher proportions of patients who did not have CDW in the model-based than in the matrix-based simulated datasets, which was supported by the average CDW rates from Tables 5.2 and 6.4. Specifically, the model-based average CDW rates indicated that there were lower proportions of treatment patients who had at least 1 CDW instance out of those who had at least 121 1 IDW instance than in the matrix-based investigation. These lower proportions were likely due to incorporating the treatment effect over time (and other important factors for CDW such as baseline EDSS state) into the generation of the post-baseline EDSS states from the model-based simulated dataset. Additionally, we expected that the random patient effects helped to capture the trends of patients who were worsening faster (that is, the patients who had many CDW instances) and those patients who were more slowly or not worsening (the patients who had very few or no CDW instances), the latter of whom were more likely to be treatment patients when there was a positive treatment effect. Since the remaining differences between the plots from the 2 investigations were relatively minor, refer to the discussion of Figures 4.20 and Figures 5.10 to 5.12 for our key observations about these plots. 122 Figure 6.16 Proportions of correctly classified CDW and no CDW patients under the base treatment factor and the base set of dropout probabilities (left) and the dropout factor of . = /00 (right). 123 Figure 6.17 Proportions of correctly classified CDW and no CDW patients under the treatment factor of 10 and the base set of dropout probabilities (left) and the dropout factor of . = /00 (right). 6.15 Summary In this chapter, we found that a proportional odds model with the predictors of sex, baseline EDSS state, and the treatment by time interaction captured the main structure of the analysis dataset together with the patient-specific random effects. We used that mixed-effects proportional odds model with the base and magnified model-based treatment factors to generate the follow-up EDSS states of the simulated datasets. The investigation of the different imputation strategies then proceeded as in the matrix-based investigation. 124 The key result was that for the most extreme treatment factor of 30 with the extreme dropout factor of * = 100, the hazard ratios from the kNN (& = 3, 5, 7) and the logistic regression imputation strategies were noticeably lower than those for the other imputation approaches. This may have been because these strategies took treatment into account. Since treatment patients were more likely to remain in lower EDSS states under the more extreme treatment factors, it was likely that greater fractions of the statuses were imputed as no CDW for the treatment patients, resulting in more pronounced treatment effects. Yet, the differences across the imputation strategies were unlikely to be important in the context of MS clinical trials because the average hazard ratios differed by 0.085 units at most. The different imputation strategies led to treatment effect hazard ratios that were clearly distinct only under fairly extreme model-based treatment factors. Since these indicated greatly exaggerated treatment effects in comparison to that from the Schering trial, we do not expect that the different imputation strategies would lead to clearly different treatment effect results in a SPMS trial that is similar to the Schering SMPS trial. 125 Chapter 7: Conclusion In phase III SPMS clinical trials, a common outcome is time to first confirmed disability worsening (CDW). CDW occurs when there is an initial elevation in the EDSS score at a visit and that elevation in EDSS is confirmed (sustained or increased at the next visit). It is not evident what statistical approach should be used to handle the right-censored patients who are missing confirmation visits. Those patients have the potential for CDW, but they are typically treated as not confirmed in the analysis. Our objective was to examine how the estimated treatment effect hazard ratio from a time to first CDW analysis was affected by different imputation approaches that were used to assign CDW statuses to those right-censored patients. Guided by a phase III SPMS clinical trial dataset of 718 patients (that was referred to as the Schering dataset), a complete simulated dataset of 2000 patients was constructed by generating follow-up EDSS states using separate stationary transition matrices for the placebo and treatment patients. Samples of 300 patients were obtained from the complete simulated dataset and dropouts were imposed on each of these based on the conditional probabilities of dropout given that IDW occurred at the preceding visit. After the required CDW statuses were imputed in a sample, a time to first CDW analysis gave the estimated treatment effect hazard ratio. We conducted 3 separate, but related, investigations to understand in what circumstances and to what extent different imputation methods led to clearly different estimated treatment effects. First, we investigated the impact of magnifying the conditional probabilities of dropout by using factors of increasing magnitude. Our next 2 investigations focused on magnified treatment effects in conjunction with the earlier dropout factors. More precisely, our second investigation focused on Markov chain models where the probabilities of disease progression and 126 improvement from the earlier treatment group stationary transition matrix were scaled by positive factors. Our third investigation focused on including more information from the Schering dataset to generate the follow-up EDSS states by using a mixed-effects proportional odds model. As a result, the proportional odds model allowed us to simulate follow-up EDSS states that were likely more representative of the clinical trial data. Our first simulation study indicated that magnifying the conditional probabilities of dropout had little impact on differentiating the hazard ratios across the imputation approaches. In contrast, in both the Markov chain and model-based treatment effect investigations, the imputation strategies that accounted for treatment group (that is, the kNN and the logistic regression strategies) gave estimated treatment effect hazard ratios that were clearly stronger than those from the other strategies under extreme treatment and dropout situations. Yet, as the largest differences in the average hazard ratios were about 0.072 and 0.085 in the Markov chain and model-based investigations, respectively, they were not large enough to be of practical importance. We emphasize that distinct treatment effect hazard ratios across the imputation strategies emerged only under fairly extreme treatment and dropout circumstances. Therefore, the right-censored patients with missing confirmatory visits only had a strong enough influence on the hazard ratios to lead to clear differences under the most exaggerated treatment and dropout factors explored. Since such extreme scenarios are unlikely to be realized in practice, we do not expect that the different imputation methods under investigation would lead to clearly different treatment effect results in phase III SPMS trials that are similar in structure to the Schering SMPS trial. 127 Bibliography Abramowitz, M., and Stegun, I. A. (1972), Handbook of Mathematical Functions, Dover Publications Incorporated. Christensen, R. H. B. (2018), âCumulative Link Models for Ordinal Regression with the R Package Ordinalâ, Submitted in Journal of Statistical Software. Decady, Y. J., and Thomas, D. R. (2000), âA Simple Test of Association for Contingency Tables with Multiple Column Responsesâ, Biometrics, 56(3), 893â896. Dong, Y., and Peng, C. Y. (2013), âPrincipled Missing Data Methods for Researchersâ, SpringerPlus, 2(1), 222, DOI: 10.1186/2193-1801-2-222. Haneuse, S. (2019), âGeneralized Linear Mixed Modelsâ, Harvard School of Public Health, Available at https://cdn1.sph.harvard.edu/wp-content/uploads/sites/565/2019/09/245Spr19_Part-5_GLMMs.pdf. Healy, B., Chitnis, T., and Engler, D. (2011), âImproving Power to Detect Disease Progression in Multiple Sclerosis through Alternative Analysis Strategiesâ, Journal of Neurology, 258(10), 1812â1819. Heymans, M., and Eekhout, I. (2019), Applied Missing Data Analysis with SPSS and (R) Studio. Available at https://bookdown.org/mwheymans/Book_MI. Kappos, L., Polman, C., Pozzilli, C., Thompson, A., and Dahlke, F. (1998), âPlacebo-Controlled Multicentre Randomised Trial of Interferon β-1b in Treatment of Secondary Progressive Multiple Sclerosisâ, The Lancet, 352(9139), 1491â1497. Kowarik, A., and Templ, M. (2016), âImputation with the R Package VIMâ, Journal of Statistical Software, 74(7), 1â16. 128 Kurtzke, J. F. (1983), "Rating Neurologic Impairment in Multiple Sclerosis: An Expanded Disability Status Scale (EDSS)â, Neurology, 33(11), 1444â1452. Liu, Q., and Pierce, D. A. (1994), âA Note on Gauss-Hermite Quadratureâ, Biometrika, 81(3), 624â629. Marshall, A., Altman, D. G., Holder, R. L., and Royston, P. (2009), âCombining Estimates of Interest in Prognostic Modelling Studies after Multiple Imputation: Current Practice and Guidelinesâ, BMC Medical Research Methodology, 9, 57, DOI: 10.1186/1471-2288-9-57. McCullagh, P. (1980), âRegression Models for Ordinal Dataâ, Journal of the Royal Statistical Society: Series B (Methodological), 42(2), 109â127. MS Society of Canada, https://mssociety.ca. MS International Federation, https://www.msif.org. National MS Society, http://www.nationalmssociety.org. Riester, K., Xu, L., Elkins, J., and Lipkovich, I. (2015), âMS Disease Progression: A Multiple Imputation Approach for Handling Missing Dataâ, Joint Statistical Meetings. Available at https://ww2.amstat.org/meetings/jsm/2015/onlineprogram/AbstractDetails.cfm?abstractid=314579. Rubin, D. B. (1987), Multiple Imputation for Nonresponse in Surveys, New York: John Wiley & Sons. Van Buuren, S. (2019), Flexible Imputation of Missing Data, Chapman and Hall/CRC. Available at https://stefvanbuuren.name/fimd. Van Buuren, S. V., and Groothuis-Oudshoorn, K. (2011), âMICE: Multivariate Imputation by Chained Equations in Râ, Journal of Statistical Software, 45(3), 1â67. 129 Zhu, F. (2011), âEvaluation of Analysis Approaches to Longitudinal Ordinal Responses in Multiple Sclerosis Clinical Trialsâ, M.Sc. thesis, The University of British Columbia.
Thesis/Dissertation
eng
Statistics
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Impact of informative censoring on estimated treatment effects for confirmed disability worsening in multiple sclerosis clinical trials
Text