UTILITY OF THE ACS NSQIP SURGICAL RISK CALCULATOR TO ACCURATELY PREDICT POSTOPERATIVE OUTCOMES AFTER COLON RESECTION by Rihab Al Lawati M.D., Sultan Qaboos University, 2012 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in The Faculty of Graduate and Postdoctoral Studies (Surgery) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) July 2016 ©Rihab Al Lawati, 2016 ii Abstract BACKGROUND: Predicting potential complications from surgery is a crucial step to aid decision to operate. The American College of Surgeons (ACS) initiated the National Surgical Quality Improvement Program (NSQIP) which collects and analyses patients’ outcomes from surgery. ACS NSQIP developed a Surgical Risk Calculator (RC) to predict risks of postoperative complications. Aim of this study was to assess RC accuracy for predicting complications in patients undergoing colon resection. METHODS: Validation study with secondary use of administrative data conducted in a tertiary care center. Patients who received colorectal procedures in our Enhanced Recovery After Surgery (ERAS) program from November 2013 to December 2015 were enrolled. RC predictions were calculated and compared with observed NSQIP outcomes within 30 days follow-up. Observed versus predicted outcomes were compared. RC accuracy was assessed by graphical examination of the model calibration for outcomes that exceeded 50 events. Predicted versus observed length of stay (days mean±SD) was compared. RESULTS: A total of 368 patients were enrolled. RC predicted versus observed outcomes (n) were: serious complication 40.3 vs. 51; any complication 60.5 vs. 70; surgical site infection (SSI) 31.8 vs. 51; pneumonia 5.8 vs. 15; cardiac complication 16.5 vs. 9; urinary tract infection 9.8 vs. 11; venous thromboembolism 4.8 vs. 4; acute renal failure 16.6 vs. 5; return to operating room 14.6 vs. 6; death 4.2 vs. 2; Discharge to iii facility 20.2 vs. 12. Good calibration was observed for any complication and serious complications. SSI was underestimated but RC adjustment by surgeon improved SSI prediction. Length of stay was inaccurately predicted: 4.4±1.3 predicted versus 8.6±12.1 days observed (p <0.01, Wilcoxon Rank Sum Test). CONCLUSION: Application of RC in our population closely predicts serious and any complication but less accurately predicts SSI unless adjusted by surgeon and inaccurately predicts length of hospital stay. All outcomes including the above require analysis of greater number of events to permit final conclusions on RC use. iv Preface • Chapter two – Validation study with secondary use of administrative data was conducted as collaboration between Vancouver General Hospital Department of Surgery the Vancouver Coastal Health Department of Clinical Quality and Safety for NSQIP. The supervising committee consisted of Dr. Garth Warnock (as the Principal Investigator) and Drs. Kelly Mayson and Penny Brasher as Co-investigators. - Under the supervision of the research committee, I was responsible of formulating the research objectives, study design, obtaining the ethical approval, data collection, data analysis and writing the manuscript. - Ms. Tracey Hong (Department of Clinical Quality and Safety) provided the study participants under ERAS protocol who underwent colorectal procedures. - Dr. Penny Brasher (Center for Clinical Epidemiology and Evaluation) provided assistance in the project design and data analysis. - Mr. Markus Zurberg (Department of Clinical Quality and Safety) provided an insight about the current situation and how NSQIP assists in quality assurance. - Ethical approval was obtained from the UBC Clinical Ethics Board (UBC CREB No. H16-00821). v 8 Table of Contents Abstract …………………………………………………………………………………………………………… ii Preface ……………………………………………………………………………………………………………. iv Table Of Contents …………………………………………………………………………………………… v List Of Tables …………………………………………………………………………………………………… vii List Of Figures …………………………………………………………………………………………………. viii List Of Abbreviations ………………………………………………………………………………………. ix Acknowledgments ……………………………………………….…………………………………………. x Dedication ………………………………………………………………………………………………………. xi Chapter 1 Introduction And Literature Survey ……………………………….. 1 1.1 Background ……………………………………………………………………...…………………….. 1 1.1.1 Mortality and morbidity conference ………………….……………………………… 1 1.1.2 UBC Mortality and Morbidity Tool ………………………………………………..…... 2 1.1.3 RIisk assessment tools/models ………………………………………………………….. 3 1.1.4 National Surgical Quality Improvement Program (NSQIP) …………..…..…. 3 1.1.5 ACS NSQIP Surgical Risk Calculator ……………………………………………………. 4 1.1.6 Enhanced Recovery After Surgery (ERAS) …………....…………………………… 6 1.2 Overall project hypothesis ……………………………………………………………………… 7 1.3 Overall project objective ………………………………………………………………………… 7 Chapter 2 Utility Of The ACS NSQIP Surgical Risk Calculator To Accurately Predict Postoperative Outcomes After Colon Resection: A Validation Study ………………………………………………………………… 2.1 Methods …………………………………………………………………………………………………. 8 2.1.1 Study design ……………………………………………………………..……………………….. 8 2.1.2 Patient selection (inclusion and exclusion criteria) ……………………………… 8 2.1.3 Data collection and definitions ……………………………………………………..……. 8 2.1.4 Data analysis …………………………………………………………………..…………………. 11 vi 2.2 Results ……………….………………………………………………………………………..………….. 13 2.2.1 Baseline characteristics of the study population ……………………………….... 13 2.2.2 Frequency of complications …………………………………………………………….….. 16 2.2.3 Length of postoperative hospital stay ……………………………………………….... 17 2.2.4 Evaluation of the accuracy of ACS NSQIP Surgical Risk Calculator ………… 18 2.2.5 Less frequent outcomes ………………………………………………………………………. 25 2.3 Discussion ………………………………………………….……………………………………………. 25 2.3.1 Main findings ………………………………………………………………………………….….. 26 2.3.2 Limitations …………………………………………………………………………………….….… 28 2.4 Conclusion …………………………………………………………………………………………….… 29 2.5 Future plan …………………….……………………………………………………………………..... 30 Bibliography ……………………………………………………………………………………………………. 31 Appendices ……………………………………………………………………………………………………… 35 Appendix A: UBC Mortality & Morbidity Tool webpage …………………………...…… 35 Appendix B: ACS NSQIP Surgical Risk Calculator software example …………….…. 37 vii List of Tables Table 2-1: Data collection form variables ……………………………………………………….... 10 Table 2-2: Characteristics of the study population and procedures …………………… 15 Table 2-3: Frequency of complications recorded in NSQIP database ………………… 16 Table 2-4: Length of postoperative hospital stay comparison ……………………………. 17 Table 2-5: Total predicted vs. total observed outcomes …………………………………….. 25 viii List of Figures Figure 2-1: Calibration graph example …………………………………………………………… 12 Figure 2-2: Flow diagram of patients through the study ……………………………..… 13 Figure 2-3: Distribution of predicted vs. observed length of hospital stay …….. 18 Figure 2-4: Distribution of the predicted SSI …………………………………………………. 19 Figure 2-5: Comparison of predicted vs. observed SSI without risk adjustment 20 Figure 2-6: Comparison of predicted vs. observed SSI with risk adjustment level 1 ……………………………………………………………………………………………………………. Figure 2-7: Comparison of predicted vs. observed SSI with risk adjustment level 2 ……………………………………………………………………………………………………………. Figure 2-8: Distribution of predicted any complication ………………………..……….. 21 Figure 2-9: Comparison of predicted vs. observed any complication without risk adjustment ……………………………………………………………………………………………… Figure 2-10: Comparison of predicted vs. observed any complication with risk adjustment level 1 …………………………………………………………………………………..……. Figure 2-11: Distribution of predicted serious complication ……………………….... 23 Figure 2-12: Comparison of predicted vs. observed serious complication without risk adjustment ………………………………………………………………………….….... Figure 2-13: Comparison of predicted vs. observed serious complication with risk adjustment level 1 …………………………………………………………………………….……. 20 21 22 22 24 24 ix List of Abbreviations VGH Vancouver General Hospital UBC University of British Columbia ACS NSQIP American College of Surgeons - National Surgical Quality Improvement Program. NSQIP National Surgical Quality Improvement Program. ERAS Enhanced Recovery After Surgery. M&M Mortality and Morbidity MRN Medical Record Number RC Risk Calculator UTI Urinary Tract Infection SSI Surgical Site Infection PE Pulmonary Embolism DVT Deep Vein Thrombosis VTE Venous Thromboembolism COPD Chronic Obstructive Pulmonary Disease DM Diabetes Mellitus HTN Hypertension CHF Congestive Heart Failure OR Operation Room CPT Current Procedural Terminology ASA American Society of Anesthesiologists Cm Centimeters Kg Kilograms x Acknowledgments “We always aim for the best science” The inspiring sentence I frequently heard from Dr. Warnock and which immensely encouraged me to always aim for the best quality outcomes throughout my project period. No words can express my deep and genuine gratitude to my supervisor, Dr. Warnock, for his sincere dedication, keen interest and overwhelming support throughout my master’s journey. His constructive advice, guidance and kind co-operation enabled me to complete and accomplish this task. I am also thankful to my committee members Dr. Kelly Mayson for her input and guidance, Dr. Penny Brasher for her opinions in the statistical analysis plan and content revision, Ms. Tracey Hong for her constant cooperation during the data collection phase, Mr. Markus Zurberg for his valuable opinions during the period of finalizing the study design. My special Thanks go to Dr. Alice Mui (MSc program coordinator) who always was there to facilitate any obstacles appeared through the way and Dr. Morad Hameed for accepting to be part of the defense committee. Finally, I would be honored to express my sincere appreciation and gratitude to Dr. Hani Al Qadhi for his ongoing support, keen encouragement and for making this opportunity of pursuing my ambitions possible. xi Dedication I dedicate this thesis to: My perfect match, Sadiq for his continuous support and encouragement, without him this would not have been possible. My loving parents, who raised me to the person I am today and who taught me that no matter how hard the journey might be, ambitions must be chased and goals must be reached. My wonderful three sisters, Noor, Bedoor and Lubna, with whom I shared my day-to-day details and difficulties until this work was finally formulated. Despite the distance their presence will always be of a great value to me. My only best friend, Safiya, without hearing her advises and weekly complaints my journey would not have been as joyful as it was. My son, Ali, Who I cannot wait to see him grow up into a successful gentleman with enough confidence, knowledge and wisdom to choose the right path in life. 1 Chapter 1 Introduction And Literature Survey 1.1 Background Complications after surgery increase mortality substantially (1). Colorectal surgery itself carries a significant mortality risk, with reported rates of 1–6% for elective surgery and up to 22% in the emergency setting (2-5).Understanding the risks and possible complications of any surgical intervention before proceeding with it is an important issue. The process of this decision is shared between physicians and patients. In order to reach a final decision and provide the informed consent that is required before any intervention, the patient has to have a thorough understanding of the potential risks of surgery (6,7). Providing accurate information to patients about potential complications is essential. Historically, health institutions adopted what is called a minimum standard program in which every staff is required to review and analyze at regular intervals their clinical experience and discuss it in regular meetings (8,9). With some modifications over the years, this experience nowadays is called morbidity and mortality meeting or morbidity and mortality conference. 1.1.1 Mortality and morbidity conference The morbidity and mortality conference has a long history in the academic pathway as many medical centers use it as a teaching tool to provide trainees real examples of medical errors or problematic decision-making situations. Although this type of meeting has not always resulted in lessons to prevent future error, it has been increasingly used as a part of the measures that are taken to enhance patient safety and quality of care (10). 2 Currently, this type of meeting is considered a key educational tool which is essential for training programs accreditation and a critical contribution to quality assurance (11). From these meetings, critical information can be gathered to inform knowledge about certain cases and complications. As part of the educational process, trainees must learn to report errors and quality issues, to participate in the wider picture of providing a better environment for patient safety. Trainees should also be engaged in projects to improve systems of care, decrease health care disparities and improve patient outcomes (12). If this process is conducted with attention to best practices such as non-punitive review, debriefing, and follow up on systems improvements it can support building strong safety cultures in medicine (13). 1.1.2 UBC Mortality and Morbidity Tool To be contemporary with the recommendations and apply the best quality measures, UBC has established Mortality and Morbidity (M&M) Tool in 2013. Many modifications were introduced to the tool over the previous years to improve its quality. Appendix A demonstrates the website page of the tool and the categories that must be completed in order to report a case of mortality or morbidity, for example, patients’ last name, MRN, type of complication and Clavien Dindo grade. Recently the tool was approved by British Columbia Freedom of Information and Protection of Privacy Act and The Vancouver Costal Health Authority. Since August 2015, the tool has been used for residents in the UBC General Surgery Program to report complications for presentations and discussions at a weekly M&M conference. Furthermore, a database was established and a summary provided for feedback to surgeon attending faculty on the frequency of complications on their teams. Surgical team members are requested to provide feedback to front line care providers on their team-encountered patients’ mortality and morbidity. This experience was prepared as an abstract submitted for presentation in one of the 3 future NSQIP meetings (personal communication; Mr. Markus Zurberg, Dec 2015). Despite potential advantages of M&M reporting tool such as the one we have developed, literature is showing a very low reporting rate for custom reporting tools in other centers (9) which made it necessary to find other ways to improve patient safety by creating a variety of pre-procedural risk assessment tools. 1.1.3 Risk assessment tools/models Many institutions created tools for risk assessment and risk prediction to help in the process of decision making. Some of the currently available models in the literature include Cleveland Clinic Foundation colorectal cancer model and the Physiological and Operative Severity Score for enUmeration of Mortality and morbidity (POSSUM). These models either solely assess risk of mortality or are difficult and complex to assess at patient’s bedside (14). Other models which tried to simplify the previously mentioned ones include the Colorectal preOperative Surgical Score (CrOSS), which may be easier to apply but still encounters the same problem of assessing the risk of mortality only without addressing the other important aspects of postoperative morbidity (15). In order to solve this issue, ACS NSQIP developed a model called the Surgical Risk Calculator (RC) which addresses postoperative morbidity and mortality. 1.1.4 National Surgical Quality Improvement Program (NSQIP) The American College of Surgeons (ACS) initiated a program called National Surgical Quality Improvement Program (NSQIP) that collects high-quality, standardized clinical data on preoperative risk factors and postoperative complications from patients who have surgery in more than 500 hospitals in North America and selected international sites (16,17). Clinical reviewers in these centers 4 are extensively trained to collect data in different methods like chart review, surgeon and patient interview to ensure high quality of the collected data (18). This program applied high-quality data that they collected to develop a tool to predict surgical risk in the form of software that predicts postoperative mortality and morbidity and from here it gains its importance, as it is not addressing mortality only like the previous models but it also includes morbidity assessment (17). 1.1.5 ACS NSQIP Surgical Risk Calculator The ACS NSQIP Surgical Risk Calculator is generated from 1.4 million patients’ information gathered between 2009 till 2012 from all the NSQIP-participating institutions (19,29). Those institutions ranged from rural community hospitals to large academic and university-affiliated centers representing a wide variety of surgeries in variable clinical settings (29). The initial RC first released in 2013 (29) was an online software requesting information input of demographics, functional status, comorbidities and American Society of Anesthesiologists (ASA) class. Figure 1, Appendix B shows the risk calculator software interface and a screen shot of the risk factor entry screen. This data is entered by the surgeon or anesthesiologist who is assessing the patient in the preoperative period. All the risk factors can be entered using a drop down menu of each category which makes the process of entering the data easier and quicker. The information required to generate the risk estimates includes procedure name, patient age, sex, comorbidities (DM, HTN, cardiac events, etc.), ASA class, wound class and others. Figure 2 (appendix B) is the risk generated report screen which demonstrates the way that risk estimates are presented in each category starting from serious complication and ending with discharge to nursing or rehabilitation facility. Clear definitions are provided by NSQIP explaining what each category of outcomes indicates and the outcome definition will appear in a pop up dialogue if clicked on the question mark sign beside each outcome. For example, “any 5 complication” defined as all the NSQIP recorded morbidity and they are: Superficial incisional SSI, deep incisional SSI, organ space SSI, pneumonia, unplanned intubation, PE, DVT, ventilator > 48 hours, acute renal failure, UTI, cardiac arrest, myocardial infarction, return to the operating room, systemic sepsis. While Serious complication included all the outcomes mentioned in any complication except superficial incisional SSI and ventilator > 48 hours. The report also provides an explanation on how to interpret the results by showing a sample at the bottom of the reporting page. The sample shows that the bolded black line represents average patient risk and the concerned patient risk is demonstrated in three methods: graphically, percentage of estimated risk and chance of outcome (below average, average, above average). The report also provides an estimation of the length of hospital stay postoperatively. Finally, the calculator gives the surgeon or physician assessing the patient the option of adjusting the risks because the calculator doesn’t capture all risk factors. The risks adjustment has three levels, level 0 with no adjustment, level one is when the risk is somewhat higher than estimated and level two is when the risk is significantly higher than estimated. So far, the literature presents scattered papers studying the accuracy of ACS NSQIP Surgical Risk Calculator in estimating postoperative complications in certain types of surgeries. Bilimoria et al concluded in their study that ACS NSQIP Surgical Risk Calculator level of prediction was reasonable and this was demonstrated by c-statistics results which ranged between 0.806 to 0.944 for most of the outcomes (20). Some studies showed that complications might not be accurately estimated (21). Other studies showed that complications were effectively estimated in patients with average risk factors but less so in predicting complications in patients with lots of risk factors (22). These studies were performed in surgical specialties of gynecology, orthopedics and surgical oncology. One study compared the data collected using a traditional M&M tool with data collected using the ACS NSQIP techniques concluding that the M&M tool considerably underreported for both in-hospital and post-discharge complications and deaths compared with ACS NSQIP techniques (9). No study in the literature was identified that addresses utility of the 6 RC in the context of general surgery cases rather than subspecialized fields like surgical oncology, gynecology and orthopedics surgery. Recently, ACS NSQIP Surgical Risk Calculator was updated. New prediction equations were based on larger and more recent samples of surgical patients. The updated calculator tends to assign, for highest risk patients, higher predicted risk of mortality than the old calculator. Otherwise, the prediction for other outcomes is almost similar to the version used in this study (personal communication with Tracey Hong, May 2016). Hence, the idea of this project came to study the accuracy of ACS NSQIP Surgical Risk Calculator in the setting of elective colorectal surgery procedures which are enrolled in ERAS program. ERAS database combined with standardized NSQIP reporting allows for reliable definitions to test the RC tool. 1.1.6 Enhanced Recovery After Surgery (ERAS) The project aims to use the Enhanced Recovery After Surgery (ERAS) database as a source for the sample population that will be studied. ERAS program (also called fast track perioperative care) is an evidence-based collection of protocols that patients undergoing elective surgeries are recommended to follow (23,24). Literature shows that undergoing colorectal surgery involving bowel resection carries s 15% to 20% rate of complications (25,26). In an effort to reduce post colorectal surgery complications and decrease the length of hospital stay, Kehlet et al. (27) was the first to describe in detail the fast track or the enhanced recovery after surgery protocols. This was achieved mainly by harnessing the physiological principles to improve patient outcomes by reducing the profound stress response induced by surgery, there by reducing postoperative complications, minimizing hospital stay, and ultimately reducing health costs without compromising patients' safety (28,30). Moreover, the aim is to provide pain and stress-free pathway to full recovery. 7 Combining all the above factors, the main objective of this study is to examine the evidence that supports and validates the accuracy of the level of risk prediction generated by the RC in our patient population and how accurately it may help in predicting postoperative complications. The ultimate aim is to observe if pre-surgical application of the RC will highlight ways to predict and reduce postoperative complications further in patients entering ERAS programs (30). 1.2 Overall project hypothesis: We hypothesize that the RC predicts postoperative complications that can be detected through routine NSQIP screening. 1.3 Overall project objective: The overall objective of this thesis is to provide a more robust ability to predict and prevent postoperative complications in a population of patients who are undergoing scheduled elective colon resections using a standardized procedure care protocol. 8 Chapter 2 Utility Of The ACS NSQIP Surgical Risk Calculator To Accurately Predict Postoperative Outcomes After Colon Resection: A Validation Study. 2.1 Methods 2.1.1 Study design This project is a validation study with secondary use of administrative data. This study compared observed postoperative outcomes for patients undergoing elective colorectal procedures in VGH to the predicted outcomes generated by the RC. This comparison was conducted to assess validity of the surgical risk prediction model provided by NSQIP in our patient population. Data for a cohort of patients who underwent elective colorectal surgery under ERAS protocol at VGH during the period from November 2013 to December 2015, and who were selected for the VGH NSQIP sample was extracted from the ERAS database. 2.1.2 Patient selection (inclusion, exclusion criteria) All adult (≥18 years old) patients of all ages who underwent colorectal procedure under Enhanced Recovery After Surgery program in Vancouver General Hospital were included. Any patients presented and operated in an emergency setting were excluded from the study. 2.1.3 Data collection and definitions Consecutively treated patients enrolled in ERAS who underwent elective colorectal procedures between the periods from November 2013 to December 2015 were identified through reviewing the lists of OR slates for patients who underwent colorectal procedure under ERAS during the above mentioned period. Information 9 on patients’ demographics, functional status, smoking, medical background (HTN, DM, COPD, previous cardiac history, ventilation dependence, cancer, acute renal failure, dialysis, ascites and sepsis) and procedural details (procedure name, date, CPT code and description) was abstracted from the ERAS database and entered into Excel spreadsheets as summarized in table 2-1. Admission, discharge dates and discharge destination were also obtained to calculate the length of hospital stay. Duration of hospital length of stay was defined as the total days postoperatively in hospital from the surgery date until discharge. NSQIP outcomes detected through the 30 days follow up included any complication, serious complication, pneumonia, cardiac complications, surgical site infection, urinary tract infection, acute renal failure, ileus, deep vein thrombosis, pulmonary embolism, unplanned intubation, ventilation more than 48 hours, death and discharge destination as shown in table 2-1. All categories were defined by NSQIP standard definitions. The category “any complication” included an aggregate of all the NSQIP-recorded morbidities including: Superficial incisional SSI, deep incisional SSI, organ space SSI, pneumonia, unplanned intubation, PE, DVT, ventilator > 48 hours, acute renal failure, UTI, cardiac arrest, myocardial infarction, return to the operating room, systemic sepsis. “Serious complication” was defined as all the outcomes mentioned in any complication except superficial incisional SSI and ventilator > 48 hours. 10 Table 2-1 Data collection form variables Preoperative variables (ERAS database) 30 days follow up variables (NSQIP database) Age Sex Date of admission Date of discharge Date of surgery Procedure name CPT code and description Height in (cm) & weight (kg) DM HTN Smoking Dyspnea Ventilation dependence Disseminated cancer Functional status COPD Previous cardiac history/ CHF Acute renal failure/ dialysis Steroid use Ascites Systematic sepsis ASA class Wound class Pneumonia Cardiac complications SSI UTI Ileus DVT PE Acute renal failure Transfusion Sepsis Return to OR Death Unplanned intubation Ventilation >48 hours Discharge destination If study data was missing from the ERAS or NSQIP database the information was obtained by chart reviews of preoperative assessment reports in the Vancouver General Hospital Patient Care Information System (PCIS) files. Since all of the patients were managed according to the standard of care at VGH, the project was approved to be of minimal risk to patient confidentiality by the University of British Columbia Clinical Research Ethics Board which approved the study protocol to be conducted with a waived consent (UBC CREB No. H16-00821). 2.1.4 Data analysis Descriptive statistics were used to describe the demographic, preoperative and operative characteristics of the study population. Discrete variables were 11 summarized by frequencies and percentages. Continuous variables were summarized by mean (standard deviation). A table of the number (percent) of patients with a NSQIP-recorded complication was constructed. Postoperative outcomes were broken down by NSQIP category, i.e., serious complication, any complication, pneumonia, cardiac complication, SSI, UTI, VTE, acute renal failure, return to OR, death and discharge to nursing or rehab facility. The NSQIP predicted and the observed lengths of hospital stay postoperatively were compared visually using histograms. A table showing the mean, standard deviation, range of hospital stay was constructed and the Wilcoxon rank sum test was used to compare the two groups. Evaluation of the accuracy of the ACS NSQIP Surgical Risk Calculator: To assess the model performance, calibration measures were assessed for the outcome categories which had at least 50 events. Calibration of a prediction model generally studies the agreement between observed outcome frequencies and predicted probabilities (31). In our study the calibration of the model was assessed graphically by comparing predicted and observed risks for the categories serious complication, any complication and SSI. The predicted risk for each outcome was cut into approximately equally-sized "bins". A bin, is away of sorting data by evenly distributing the data set in carefully chosen categories, as demonstrated in figure 2-1 (x-axis). 12 Figure 2-1 Calibration graph example: The thick line indicates perfect calibration; the thin line shows the relationship between the predicted and observed probabilities. Circles indicate observed events per quintile of predicted probabilities After that, the mean predicted risk for each bin was calculated (x-axis) and plotted versus the observed risk (y-axis) (Figure 2-1) and a linear model line was fit to the data (thin line). Finally the line of equality was superimposed on the graph (Thick line). The line of equality is used as a reference in comparing two sets of data expected to be identical. If the prediction model is perfectly accurate, the fitted linear model line should follow the line of equality. In the example provided in figure 2-1 the linear model line is shifted to the left of the line of equality, which indicates that the observed events are higher than predicted. For outcomes that did not have a sufficient number of events to assess calibration, we provided a table of observed versus predicted events. For length of postoperative hospital stay, a p value <0.05 was considered statistically significant. All data was collected using Microsoft Excel (2011) and data analysis was carried out using R program (version 3.2.3). 13 2.2 Results 2.2.1 Baseline characteristics of the study population Figure 2-2 shows the selection of patients to be included. A total of 416 patients were enrolled in ERAS program to undergo a colorectal procedure during the period from Nov 1st, 2013 to December 31st, 2015. Study population colorectal procedures included left and right hemicolectomy, sigmoid and segmental colon resection, rectosigmoid and rectal excision. Diagnoses of colon resection were malignancy, benign polyp, diverticular disease and inflammatory bowel disease. Out of these, thirty-seven were excluded because they were not sampled by NSQIP and another eleven patients were excluded because of missing data. Figure 2-2 Flow diagram of patients through the study 14 Finally, a total of 368 patients were included in the study. Table 2-2 shows the demographics, procedure measures and operative characteristics of the population. The age of the study population ranged between 24 and 100 years and the mean was 69 years with a standard deviation of 13.4 and 54.1 % of them were males. The mean population BMI was 26.7 (SD= 5.3) and the preoperative measures showed that 45.4% of the sample were hypertensive and 15.8 % were with a previous cardiac history.57.3 % were classified as ASA class 2 and 71.2 % of the total procedures performed were minimally invasive surgery. 15 Table 2-2 Characteristics of the study population and procedures (n=368) Demographics Mean±SD or Number % Age (years) 69±13.4 ⎯ Males – number (%) 199 54.1 Body Mass Index (kg/m2) 26.7±5.3 ⎯ - Height (cm) 163.5±18.8 ⎯ - Weight (kg) 74.1±18.5 ⎯ Preoperative measures Number % Functional status - Independent 365 99.2 - Partially Dependent 3 0.8 Diabetes* 37 10.1 Hypertension 167 45.4 Smoker** 29 7.9 Dyspnea on moderate exertion 18 4.9 Ventilator dependent 5 1.6 Disseminated Cancer 17 4.6 COPD† 14 3.8 Previous cardiac history 58 15.8 Dialysis 3 0.8 Recent steroids use 15 4.1 Sepsis†† 5 1.4 Operative characteristics Number % ASA Class ∞ - Class 1 15 4.1 - Class 2 211 57.3 - Class 3 128 34.8 - Class 4 14 3.8 Procedure - Laparoscopic technique 262 71.2 - Open technique 105 28.5 Wound Class - Clean-Contaminated 350 95.1 - Contaminated 10 2.7 - Dirty-Infected 8 2.2 Data are shown as number (%) or mean (standard deviation). •Diabetes category including Insulin and non-insulin dependent; ••smoking recorded within a year from the surgery; †COPD, Chronic Obstructive Pulmonary Disease; ††Sepsis includes its different stages; ∞ ASA, American Society of Anesthesiologists. 16 2.2.2 Frequency of complications The occurrence of a complication was recorded by NSQIP is shown in table 2-3. There were a total of 70 patients with recorded NSQIP morbidity, representing 19 % of the total sample population. Fifty-one of the total complications met the criteria of serious complication representing 13.9% of the total population of the study. Surgical site infection was the highest represented complication against all other recorded outcomes reaching up to 13.8 % of the total complications. Postoperative blood transfusion and ileus scored 10.3 % and 8.7 % respectively but these two complications are not predicted by the tool that we are evaluating in this study so they were not further included in the analysis. Table 2-3 Frequency of complications recorded in NSQIP database NSQIP-recorded complication Number Percent % Serious complication 51 13.9 Any complication 70 19 Pneumonia 15 4.1 Cardiac complication 9 2.4 SSI 51 13.9 UTI 11 3 DVT 3 0.8 Ileus 32 8.7 Acute renal failure 5 1.4 PE 1 0.3 Transfusion 38 10.3 Sepsis 20 5.4 Return to OR 6 1.6 Death 2 0.5 Intubation 10 2.7 Ventilation >48 hours 11 3 17 2.2.3 Length of postoperative hospital stay Length of postoperative hospital stay is shown in table 2-4. The mean of the predicted length of stay was 4.4±1.3 days (range 2.5-10.5, median 4). Corresponding stay was 8.6±12.1 days (range 1-171, median 6). The difference between predicted and observed lengths was significant (p < 0.01, Wilcoxon rank sum test). Table 2-4 Length of postoperative hospital stay comparison Length of postoperative hospital stay (days) Predicted Observed P value * Mean (days) 4.4 8.6 < 0.01 SD 1.3 12.1 Range of hospital stay (days) 2.5 - 10.5 1 - 171 * Wilcoxon rank sum test Figure 2-3 is shows the distribution of the predicted length of postoperative hospital stay versus the observed length of postoperative hospital stay. It clearly illustrates a wider pattern of distribution. Multiple outliers are shown in the graph of observed length of stay. 18 Figure 2-3 Distribution of predicted vs. observed length of hospital stay (The numerical values in the observed graph are the outliers detected in that group) 2.2.4 Evaluation of the accuracy of ACS NSQIP Surgical Risk Calculator In this part of the analysis the calibration of the risk prediction model was assessed for the following outcomes: 1. Surgical site infection: Surgical site infection was the highest detected complication as shown previously in table 2-3 and the pattern of its predicted distribution is shown in figure 2-4. The calibration of the model for this outcome was assessed without increasing the level of surgeon adjusted risk in the RC in figure 2-5 and showed that the observed outcomes are higher than the predicted outcomes, which means that this model is underestimating the risk of surgical site infection. 19 Figure 2-4 Distribution of the predicted SSI To further assess the model, the same process was repeated for but with introduction of the RC surgeon adjusted risk according to the RC report screen. As shown in figure 2-6 and figure 2-7 respectively, predicted outcomes for SSI were much more accurate and closer to the observed outcomes in level 1 adjustment but in level 2 the predicted outcomes exceeded the observed outcomes. From the assessment of the model calibration taking in consideration its three levels (without adjustment, adjustment level 1, adjustment level 2), prediction with risk adjustment level one provided the closest prediction of postoperative surgical site infection in our patient population. 20 Figure 2-5 Comparison of predicted vs. observed SSI without risk adjustment Figure 2-6 Comparison of predicted vs. observed SSI with risk adjustment level 1 21 Figure 2-7 Comparison of predicted vs. observed SSI with risk adjustment level 2 2. Any complication: This category of outcomes included the sum of all the predicted morbidity outcomes by NSQIP and figure 2-8 is showing the pattern of its distribution. Figure 2-8 Distribution of predicted any complication 22 Figure 2-9 and 2-10 are the graphs of the model calibration for outcome category “any complication” without risk adjustment and with risk adjustment level 1 respectively. For this outcome, the model showed a similar level of prediction without risk adjustment, but a lower agreement of prediction at higher adjustment. Figure 2-9 Comparison of predicted vs. observed any complication without risk adjustment Figure 2-10 Comparison of predicted vs. observed any complication with risk adjustment level 1 23 3. Serious complication: This category of outcomes shows the distribution in figure 2-11. The calibration graph showed that observed risks were slightly higher than predicted (figure 2-12) but using the level 1 adjustment produced larger discrepancies (figure 2-13). Figure 2-11 Distribution of predicted serious complication 24 Figure 2-12 Comparison of predicted vs. observed serious complication without risk adjustment Figure 2-13 Comparison of predicted vs. observed serious complication with risk adjustment level 1 25 2.2.5 Less frequent outcomes The rest of the outcome categories were not assessed graphically because the number of the detected events per category was very small. Pneumonia and UTI were the only two categories that showed a larger number of observed complications than the predicted ones. The outcome category discharge to nursing or rehab facility showed less number of observed events than predicted and this was the same for all other complications. Table 2-5 Total predicted vs. total observed outcomes Outcome category Total predicted Total observed Pneumonia 5.8 15 Cardiac complications 16.5 9 UTI 9.8 11 VTE 4.8 4 Acute renal failure 16.6 5 OR return 14.6 6 Death 4.2 2 Discharge to facility 20.2 12 2.3 Discussion Validation studies are crucial to evaluate any prediction model performance. Two qualities are assessed, including calibration (compare observed and predicted event rates for a group of patients) and, discrimination (quantify the model ability to distinguish between patients who do or do not experience the event of interest) or both (36). Testing the generalizability of these models before recommending them for clinical use is essential (32). The more diverse the setting the model is tested and found accurate, the more it will generalize and become widely applicable (33, 34). In any surgical field, surgeons are frequently asked to provide prognostic assessment 26 for operative and postoperative risks (39). To do so, surgeons usually depend on the literature that is based on results of aggregate data or they depend on their personal experience and they often worry that their assessment will prove incorrect (33,35). The American College of Surgeons provided a tool that may guide surgeons to accurately predict postoperative risk and make the process of assessment consistent in different surgical environments worldwide. The ACS NSQIP Surgical Risk Calculator is a model that was built on a multi-institutional, well-collected data of a large sample size. However, this tool is still underutilized in the clinical field. One of the possible reasons for this is that this tool remains incompletely tested to prove its validity and gain trust of surgeons and stakeholders to incorporate it in the preoperative assessment protocols. 2.3.1 Main findings Our study applied this tool on a sample of patients from a single tertiary center and compared the predictions to our actual experience. Starting with the length of postoperative hospital stay, the observed stay showed multiple outliers. Even after we tried to exclude the outliers, the observed distribution was still different than the predicted one. The RC did not show an accurate prediction, as the range of the prediction was very small despite having multiple patients with many risk factors. This may indicate that the tool incompletely adjusts the risk on the individual level. This observation agrees with other studies in the literature which had similar findings (22). Our study population was ERAS which has designed to reduce postoperative length of hospital stay, yet we still did not see accuracy. To test the model performance our study assessed the calibration graphically. Three outcome categories of any complication, serious complication and SSI were chosen to be assessed by this method and the reason for choosing them was that they contained a reasonable number of events. In order to rigorously assess calibration at least 100 events are necessary (31). 27 Despite the fact that our sample population is on standardized procedure care protocols within ERAS under high compliance with perioperative protocols to reduce SSI, we still found that SSI scored the highest. The model prediction for this outcome showed an overall underestimation of the postoperative SSI. In order to improve the model prediction, we generated the calibration graphs for the predicted outcomes with risk adjustment in its two levels. This step was done to examine if simple risk adjustment will modulate the model performance. This particular step adds to the concepts in previous literature. Our analysis for this outcome showed by adjusting the risk to level one, the calibration graph demonstrated a better agreement between the predicted and the observed outcomes. Hence, we can suggest that to improve the tool performance for this outcome in our patient population, the first level risk adjustment can be employed. The same steps of testing the agreement between the predicted and observed outcomes were performed for the outcome categories any complication and serious complication. For these two variables, the model showed fairly good calibration. Our study did not perform the same steps of assessment and analysis for the rest of the outcome categories (Pneumonia, Cardiac complications, UTI, VTE, acute renal failure, OR return, death) because the number of events detected was too small to draw conclusions from it. A comparison between the sums of observed outcomes and the sums of the detected outcomes were performed and it showed that VTE predicted and observed events were similar, while Pneumonia and UTI showed higher observed events and all other outcome categories showed a lower number of observed events than predicted by the model. However, these findings must be interpreted cautiously given the small number of events. We observed a total of 19% of overall complications in our sample population. This percent is still considered high and greater efforts are required to address this problem in order to find new methods for improvement. It is possible the prospective application of the RC in advance of surgery for this population could 28 allow better preoperative preparation and enhance partnerships between patients, surgeons, anesthesiologists and nurses, but this possibility remains unproven. As ERAS standardized protocols improve in this population, the overall complication rate is trending down in slow steps and this was noticed in comparison to a recent study done in the same center with similar characteristics of the sample population which showed an overall complication rate of 22%. SSI remained the outcome of concern with a rate of 14% in both studies (30, personal communication with Dr. Garth Warnock, March 2016). Accurate assessment of treatment risks is an important aid for good decision-making and a recent study published in the literature found that providing surgeons with objective data from a well-validated risk calculator resulted in improved and less varied judgments of operative risks that more closely approximate the risk calculator values (38). Due to this fact, our study tried to focus on one of the tools that are currently available and proof its validation or suggest a simple "fix" to improve the model and make the tool better utilized clinically. The current study aims to determine if the RC is accurate in a colon resection population. This is unique in the literature and essential step to incorporating this tool in preoperative clinics. The calculator can be completed by surgeon or anesthesiologists in the preoperative assessment period where they can have the chance to discuss the results with patients to make a shared decision about the best quality of care provided focusing on individual patient risk factors and needs. 2.3.2 Limitations There were several limitations in this study that were recognized. First, the reliance on historical data which exposed the study to the issue of missing data and followed by patients exclusion due to this fact. Secondly, the sample size and we were unable to rigorously assess calibration, thus our findings should be interpreted cautiously. Third, our sample population was based on patients treated in a single tertiary referral center. Therefore, the results are unlikely to be generalizable to 29 other centers. Despite the limitations, our study has made unique contribution to the literature by assessing the calibration of the RC prediction model, including prediction among the three levels of risk adjustment. This allowed us to test if the model can be improved by simply raising the level of risk adjustment instead of just stating that this model is not appropriately predicting postoperative outcomes. 2.4 Conclusion Prediction models for treatment risks are of crucial benefits for surgeons to predict their decision to operate. They can also allow patients to understand and comprehend the measures of the possible risks associated with their treatment in order to reach a final decision and sign the informed consent. For a risk prediction model implementation in a clinical system, it must have a rigorous proof of its validity and generalizability. Our study focused on the ACS NSQIP Surgical Risk Calculator, tool generated from a very well collected data and vast sample size. Calibration of the model was mainly examined in three categories of NSQIP outcomes for any complication, serious complication and SSI categories which showed: any complication and serious complication predictions were fairly accurate but SSI prediction was lower than the actual. By adjusting the level of risk, model predictions improved notably. Length of postoperative hospital stay was examined too but the model showed inaccurate predictions for this variable. Operationally, results from the study to date would mean using the values predicted by the RC for all outcomes including those less frequent, because we have no evidence to support not using them. We conclude that this tool is a potentially useful tool in this population of colon resection patients but we recommend further analysis to be conducted to test all the ACS NSQIP Surgical Risk Calculator predicted outcomes after obtaining larger 30 number of events to properly validate the model and reach a solid conclusion about its level of accuracy as planned for the ongoing part of this study. 2.5 Future plan All that have been done so far is considered as phase I of the project that we proposed and it is the initial step to move forward towards the ultimate goal that we have set and trying to achieve. Further data will be collected for validating the tool in the study population giving our center continues to collect NSQIP data on all colon resection patients who enter ERAS program. Also continuing the project will give us the privilege of increasing the sample size, which will increase the overall number of events and the number of events per category of complication. The next step will be moving to phase II where we will be collecting the data prospectively to avoid the problem of missing data when depending on hospital records alone. This will allow us to assess the model calibration for more outcome categories and perhaps recommend it as a useful preoperative tool placed on the chart for patients, surgeons, nurses and anesthesiologist to incorporate into the preadmission clinic protocols or define a certain recommendation to prepare patients for a safe surgical experience. 31 Bibliography 1. Sandblom G, Videhult P, Crona Guterstam Y, Svenner A, Sadr Azodi O. Mortality after a cholecystectomy: a population-based study. HPB. 2015 Mar;17(3):239–43. 2. Population-based audit of colorectal cancer management in two. 2005 Nov 10;:1–6. 3. Operative mortality in colorectal cancer: prospective national study. 2003 Nov 18;:1–6. 4. Slim K, Panis Y, Alves A, Kwiatkowski F, Mathieu P, Mantion G. Predicting Postoperative Mortality in Patients Undergoing Colorectal Surgery. World J Surg. 2005 Dec 13;30(1):100–6. 5. Ragg JL, Watters DA, Guest GD. Preoperative Risk Stratification for Mortality and Major Morbidity in Major Colorectal Surgery. Diseases of the Colon & Rectum. 2009 Jul;52(7):1296–303. 6. Knops AM, Legemate DA, Goossens A, Bossuyt PMM, Ubbink DT. Decision Aids for Patients Facing a Surgical Treatment Decision. Annals of Surgery. 2013 May;257(5):860–6. 7. Schenker Y, Fernandez A, Sudore R, Schillinger D. Interventions to Improve Patient Comprehension in Informed Consent for Medical and Surgical Procedures: A Systematic Review. Medical Decision Making. 2011 Feb 4;31(1):151–73. 8. Brand RA. Biographical Sketch: Ernest Amory Codman, MD (1869–1940). Clinical Orthopaedics and Related Research®. 2012 Dec 18;471(6):1775–7. 9. Hutter MM, Rowell KS, Devaney LA, Sokal SM, Warshaw AL, Abbott WM, et al. Identification of Surgical Complications and Deaths: An Assessment of the Traditional Surgical Morbidity and Mortality Conference Compared with the American College of Surgeons-National Surgical Quality Improvement Program. Journal of the American College of Surgeons. Elsevier; 2006 Nov;203(5):618–24. 10. Ackerman AD. Morbidity and Mortality Conference. Pediatric Critical Care Medicine. 2016 Jan;17(1):94–5. 32 11. Aaronson E, Wittels K, Nadel E, Schuur J. Morbidity and Mortality Conference in Emergency Medicine Residencies and the Culture of Safety. WestJEM. 2015 Nov 12;16(6):810–7. 12. Weiss KB, Bagian JP, Wagner R. CLER Pathways to Excellence: Expectations for an Optimal Clinical Learning Environment (Executive Summary). Journal of Graduate Medical Education. 2014 Sep;6(3):610–1. 13. McVeigh TP, Waters PS, Murphy R, O'Donoghue GT, McLaughlin R, Kerin MJ. Increasing Reporting of Adverse Events to Improve the Educational Value of the Morbidity and Mortality Conference. Journal of the American College of Surgeons. American College of Surgeons; 2013 Jan 1;216(1):50–6. 14. Kong CH, Guest GD, Stupart DA, Faragher IG, Chan STF, Watters DA. Recalibration and Validation of a Preoperative Risk Prediction Model for Mortality in Major Colorectal Surgery. Diseases of the Colon & Rectum. 2013 Jul;56(7):844–9. 15. Kong CH, Guest GD, Stupart DA, Faragher IG, Chan STF, Watters DA. Colorectal preOperative Surgical Score (CrOSS) for mortality in major colorectal surgery. ANZ J Surg. 2015 Mar 30;85(6):403–7. 16. Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does Surgical Quality Improve in the American College of Surgeons National Surgical Quality Improvement Program. Transactions of the Meeting of the American Surgical Association. 2009;127:7–20. 17. Ingraham AM, Richards KE, Hall BL, Ko CY. Quality Improvement in Surgery: the American College of Surgeons National Surgical Quality Improvement Program Approach. Advances in Surgery. Elsevier Inc; 2010 Sep 1;44(1):251–67. 18. Henderson WG, Daley J. Design and statistical methodology of the National Surgical Quality Improvement Program: why is it what it is? AJS. Elsevier Inc; 2009 Nov 1;198(S):S19–S27. 19. Bilimora KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY, et al. Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: A Decision Aid and Informed Consent Tool for Patients and Surgeons. Journal of the American College of Surgeons. American College of Surgeons; 2013 Nov 1;217(5):833–3. 33 20. Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY, et al. Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: A Decision Aid and Informed Consent Tool for Patients and Surgeons. Journal of the American College of Surgeons. 2013 Nov;217(5):833–3. 21. Szender JB, Frederick PJ, Eng KH, Akers SN, Lele SB, Odunsi K. Evaluation of the National Surgical Quality Improvement Program Universal Surgical Risk Calculator for a Gynecologic Oncology Service. International Journal of Gynecological Cancer. 2015 Mar;25(3):512–20. 22. Cologne KG, Keller DS, Liwanag L, Devraj B, Senagore AJ. Use of the American College of Surgeons NSQIP Surgical Risk Calculator for Laparoscopic Colectomy: How Good Is It and How Can We Improve It? Journal of the American College of Surgeons. American College of Surgeons; 2015 Mar 1;220(3):281–6. 23. Walter CJ, Collin J, Dumville JC, Drew PJ, Monson JR. Enhanced recovery in colorectal resections: a systematic review and meta-analysis. Colorectal Disease. 2009 May;11(4):344–53. 24. Wind J, Polle SW, Fung Kon Jin PHP, Dejong CHC, Meyenfeldt von MF, Ubbink DT, et al. Systematic review of enhanced recovery programmes in colonic surgery. Br J Surg. 2006;93(7):800–9. 25. Colorectal Resection and Primary Anastomosis in Patients Aged 70 and Older: Prospective Study. 2003 Nov 18;:1–5. 26. Retchin SM, Penberthy L, Desch C, Brown R, Clement , et all. Perioperative management of colon cancer under medicare risk programs. Arch Intern Med. 1997;157:1878-1884. 27. Kehlet H, Mogensen T. Hospital stay of 2 days after open sigmoidectomy with a multimodal rehabilitation programme. British Journal of Surgery. 1999, 86:227-230. 28. Gouvas N, Tan E, Windsor A, Xynos E, Tekkis PP. Fast-track vs standard care in colorectal surgery: a meta-analysis update. Int J Colorectal Dis. 2009 May 5;24(10):1119–31. 29. CS data collection, analysis and reporting. http://site.acsnsqip.org/program- specifics/data-collection-analysis-and-reporting. 34 30. Bodie G, Mayson K, Hong T, Panton N, Meneghetti A, Warnock GL, et al. Impact of An Enhanced Recovery After Surgery (ERAS) Program on Outcomes of Patients Undergoing Elective Colon Resection. Manuscript in preparation. 31. Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. Journal of Clinical Epidemiology. 2005 May;58(5):475–83. 32. Frank E, Harrell JR, Kerry I, Lee and Daniel B. Mark.Tutorial in biostatistics multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy and measuring and reducing errors. Statistics in Medicine. 1996; 15:361–387. 33. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Annals of Medicine. 1999 March 130; 6. 34. Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009 May 28;338 (may28 1):b605–5. 35. Neuman MD, Charles LB. What we talk about when we talk about risk: refining surgery's hazards in medical thought. The Milbank Quarterly. 2012; 90(1) :135–159. 36. Janssen KJM, Moons KGM, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. Journal of Clinical Epidemiology. 2008 Jan;61(1):76–86. 37. Steyerberg EW, Borsboom GJ, Van Houwelingen HC, Eijkemans MJ, Habbema JD. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Statistics in Medicine. 2004; 23:2567–2586. 38. 39. Sacks GD, Dawes AJ, Ettner SL, Brook RH, Fox CR, Russell MM, et al. Impact of a Risk Calculator on Risk Perception and Surgical Decision Making. Annals of Surgery. 2016 May;:1–7. Sacks GD, Dawes AJ, Ettner SL, Brook RH, Fox CR, Maggard-Gibbons M, et al. Surgeon perception of risk and benefit in the decision to operate. Annals of Surgery. 2016 May. 35 Appendices Appendix A: UBC M&M Tool webpage Figure (1) UBC M&M Tool website page 36 Figure (2) UBC M&M Tool website page 37 Appendix B: ACS NSQIP Surgical Risk Calculator software example Figure (1) Data required in the RC software Figures (1) & (2) Screenshots of the ACS NSQIP Surgical Risk Calculator (http://riskcalculator.facs.org). (A) Risk factor entry screen. (B) Example of report screen. 38 Figure (2) Example of report screen Figures (1) & (2) Screenshot of the ACS NSQIP Surgical Risk Calculator (http:// riskcalculator.facs.org)(A) Risk factor entry screen. (B) Example of report screen.
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Utility of the American College of Surgeons National...
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Utility of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk… Al Lawati, Rihab 2016
pdf
Page Metadata
Item Metadata
Title | Utility of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator to accurately predict postoperative outcomes after colon resection |
Creator |
Al Lawati, Rihab |
Publisher | University of British Columbia |
Date Issued | 2016 |
Description | BACKGROUND: Predicting potential complications from surgery is a crucial step to aid decision to operate. The American College of Surgeons (ACS) initiated the National Surgical Quality Improvement Program (NSQIP) which collects and analyses patients’ outcomes from surgery. ACS NSQIP developed a Surgical Risk Calculator (RC) to predict risks of postoperative complications. Aim of this study was to assess RC accuracy for predicting complications in patients undergoing colon resection. METHODS: Validation study with secondary use of administrative data conducted in a tertiary care center. Patients who received colorectal procedures in our Enhanced Recovery After Surgery (ERAS) program from November 2013 to December 2015 were enrolled. RC predictions were calculated and compared with observed NSQIP outcomes within 30 days follow-up. Observed versus predicted outcomes were compared. RC accuracy was assessed by graphical examination of the model calibration for outcomes that exceeded 50 events. Predicted versus observed length of stay (days mean±SD) was compared. RESULTS: A total of 368 patients were enrolled. RC predicted versus observed outcomes (n) were: serious complication 40.3 vs. 51; any complication 60.5 vs. 70; surgical site infection (SSI) 31.8 vs. 51; pneumonia 5.8 vs. 15; cardiac complication 16.5 vs. 9; urinary tract infection 9.8 vs. 11; venous thromboembolism 4.8 vs. 4; acute renal failure 16.6 vs. 5; return to operating room 14.6 vs. 6; death 4.2 vs. 2; Discharge to facility 20.2 vs. 12. Good calibration was observed for any complication and serious complications. SSI was underestimated but RC adjustment by surgeon improved SSI prediction. Length of stay was inaccurately predicted: 4.4±1.3 predicted versus 8.6±12.1 days observed (p <0.01, Wilcoxon Rank Sum Test). CONCLUSION: Application of RC in our population closely predicts serious and any complication but less accurately predicts SSI unless adjusted by surgeon and inaccurately predicts length of hospital stay. All outcomes including the above require analysis of greater number of events to permit final conclusions on RC use. |
Genre |
Thesis/Dissertation |
Type |
Text |
Language | eng |
Date Available | 2016-07-23 |
Provider | Vancouver : University of British Columbia Library |
Rights | Attribution-NonCommercial-NoDerivatives 4.0 International |
IsShownAt | 10.14288/1.0306923 |
URI | http://hdl.handle.net/2429/58526 |
Degree |
Master of Science - MSc |
Program |
Surgery |
Affiliation |
Medicine, Faculty of Surgery, Department of |
Degree Grantor | University of British Columbia |
GraduationDate | 2016-09 |
Campus |
UBCV |
Scholarly Level | Graduate |
Rights URI | http://creativecommons.org/licenses/by-nc-nd/4.0/ |
AggregatedSourceRepository | DSpace |
Download
- Media
- 24-ubc_2016_september_allawati_rihab.pdf [ 1.04MB ]
- Metadata
- JSON: 24-1.0306923.json
- JSON-LD: 24-1.0306923-ld.json
- RDF/XML (Pretty): 24-1.0306923-rdf.xml
- RDF/JSON: 24-1.0306923-rdf.json
- Turtle: 24-1.0306923-turtle.txt
- N-Triples: 24-1.0306923-rdf-ntriples.txt
- Original Record: 24-1.0306923-source.json
- Full Text
- 24-1.0306923-fulltext.txt
- Citation
- 24-1.0306923.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>

http://iiif.library.ubc.ca/presentation/dsp.24.1-0306923/manifest