Algorithms & Software for Intelligent Patient Monitoring by Christopher J. Brouse B.A.Sc, The University of Ottawa, 2004 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Science m The Faculty of Graduate Studies (Electrical and Computer Engineering) The University Of British Columbia June, 2007 © Christopher J. Brouse 2007 Abst rac t In Canada, more deaths occur after a patient experiences an adverse event in a hospital than from breast cancer, motor vehicle accidents and AIDS combined. Current technology contributes to the problem, by adding.greater complexity to the clinician's workload. There is still significant promise that computerized assistance can im-prove patient safety. A clinical monitoring expert system, incorporating algorithms and expert knowledge, could automatically diagnose problems and provide advice on how best to avoid hazard. This thesis investigates the design and performance of algorithms and software for an intelligent patient monitor, which forms the foundation of a clinical monitoring expert system. An algorithm has been developed for detecting electrocautery noise in the electrocardiogram (ECG) using wavelet analysis. Electrocautery noise can lead an expert system to make incorrect diagnoses. In 15 surgical cases spanning 38.5 hours of ECG data, we achieved a false positive rate of 0.71% and a false negative rate of 0.33%. While existing hardware approaches detect activation of the noise source without any ability to assess its impact on the measured ECG, our software approach detects the presence of noise in the signal itself. Furthermore, the software approach is cheaper and easier to implement in a clinical environment than existing hardware approaches. A software framework, called iAssist, has been developed for intelligent patient monitoring. The framework is extensible, flexible, scalable, and in-teroperable. It supports plugins to perform data acquisition, signal process-ing, graphical display, data storage, and output to external devices. iAssist currently incorporates the electrocautery noise detection algorithm as a plu-gin for artifact rejection, as well as two plugins to detect change point events in physiological trends. In 38 surgical cases, iAssist detected 868 events, of which clinicians rated more than 50% as clinically significant and less than 7% as artifacts. Clinicians found iAssist intuitive and easy to use. ii Table of Contents Abstract ii Table of Contents iii List of Tables vi List of Figures vii Acknowledgements x 1 Introduction 1 1.1 Medical Errors Harm Patients 1 1.2 Current Technology May Not Help 2 1.3 Clinical Monitoring Expert Systems Can Improve Patient Safety 4 1.4 An Intelligent Patient Monitor as Expert System Foundation 6 2 A Wavelet Approach to Detecting Electrocautery Noise in the ECG 8 2.1 Background 8 2.2 Method 13 2.2.1 Wavelet Analysis 13 2.2.2 Algorithm 15 2.2.3 Testing 16 2.3 Results 17 2.4 Discussion 18 3 iAssist: An Intelligent Patient Monitor 21 3.1 Background 21 3.2 Requirements 21 3.2.1 Data Acquisition . 22 3.2.2 Physiological Signal Processing . 22 Table of Contents 3.2.3 Data Storage 23 3.2.4 Graphical User Interface 24 3.2.5 Output to External Devices 24 3.2.6 Nonfunctional Requirements 25 3.3 Platform 26 3.3.1 Advantages of Java . 26 3.3.2 Java Myths & Controversy 27 3.4 Framework Architecture 28 3.4.1 Extensibil i ty 31 3.4.2 Flexibi l i ty 36 3.4.3 Scalability 46 3.4.4 Interoperability 51 3.5 Functional Area Design 55 3.5.1 Da ta Acquisit ion 57 3.5.2 Physiological Signal Processing 59 3.5.3 Da ta Storage 62 3.5.4 Graphical User Interface 63 3.5.5 Output 64 3.6 Event Feedback 65 3.7 F ina l Configuration & Testing 65 3.8 Cl in ica l Study 70 3.8.1 Algorithms 70 3.8.2 Usabil i ty 70 3.9 Discussion 72 4 Conclusion &: Future Work 74 4.1 Significance & Contributions 74 4.2 Future Work 75 Bibliography 78 Appendices A Clinical Data Collection Ethics Certificate 84 B iAssist Clinical Study Ethics Certificate 86 iv Table of Contents C iAssist Clinical Study Subject Forms 88 C l Subject Information h Consent 89 C.2 Subject Demographics 93 C.3 Post-Study System Usability Questionnaire 94 v List of Tables 2.1 Algorithm results for MaxWidth = 24 and MinDist = 4. Units in number of five second ECG blocks. * indicates FPs, and t indicates FNs 3.1 iAssist algorithms study results. A — artifact, CI — clinically insignificant, CS = clinically significant, CSAT = clinically significant with action taken, CSI = clinically significant due to intervention, NR = not rated 3.2 iAssist usability study results, with range from 1 (good us-ability) to 7 (poor usability). * SYSOBT is a custom element, and not a validated factor List of Figures 1.1 Physiological monitor with "Silence Alarms" button 3 1.2 Clinical monitoring expert system concept. Areas highlighted in grey are addressed in this thesis 7 2.1 Scene from a typical operating room: (A) The ESU (B) Inci-sion produced by the ESU (C) Resulting electrocautery noise (mistaken by the physiological monitoring equipment as ven-tricular fibrillation) 9 2.2 Typical ECG with P wave, QRS complex, and T wave 11 2.3 Electrocautery noise of weak (a), medium (b), and high (c) power 12 2.4 Standard ECG features: wandering baseline (a), motion ar-tifact (b), poor lead contact (c). . : 12 2.5 ECG partially corrupted by electrocautery noise (a), and its thresholded wavelet coefficients (b) 15 2.6 Error rates for tuning of MaxWidth, with MinDist = 4. . . 18 2.7 Error rates for tuning of MinDist, with MaxWidth — 24. . . 19 3.1 Example of the reductionist approach to engineering, as ap-plied to automobile design. The automobile can be decom-posed into smaller modules, forming a design hierarchy. . . . 29 3.2 The iAssist workflow consists of the six steps shown here. Each block represents a step, and provides both the name of the action and the functional area that performs it 31 3.3 The iAssist extensible framework is analogous to a puzzle. Different pieces can be connected, provided they have the correct shape (interface). iAssist is more general than a puz-zle, however, in that multiple pieces can be simultaneously connected to the same slot 32 List of Figures 3.4 UML diagram showing a portion of the List interface, and implementing classes ArrayList and LinkedList. The inter-face is highlighted in grey, to distinguish it from the concrete implementing classes 34 3.5 UML diagram showing the iAssist plugin interface hierarchy. Intermediate interfaces are highlighted in grey, to distinguish them from the terminal interfaces 35 3.6 Visual representation of a simple high level plugin intercon-nection relationship 38 3.7 An example of iAssist plugin instances. Two signal process-ing plugins are available to iAssist: Wavelet Transform and Change Point Detection. Plugins are highlighted in grey to distinguish them from instances. 39 3.8 An example two-level signal processing analysis hierarchy. The Wavelet Transform is first applied to the ECG and plethys-mogram. Scale Analysis is then applied to the resulting wavelet decompositions 40 3.9 An example of a MIMO, multi-level DAG. Data flow from the source vertices (A, B, C) through to the sink vertices (H, I). . 41 3.10 A simple example of a DAHG. Vertex A is the source, and vertices B, C, D, and F are the sinks. A single edge connects vertices A, B, and C. Another edge connects vertices A, D, and E. A final edge connects vertices E and F. Note that edge directionality has been omitted for visual simplicity 42 3.11 Example of an iAssist signal processing ("algorithm") plugin XML deployment descriptor. This descriptor will configure iAssist with two instances of a single plugin. Note that many descriptor details have been omitted for simplicity. 44 3.12 The parallelized iAssist workflow consists of the three steps shown here. Each vertical grouping represents a step, and all blocks contained within can execute in parallel. Each block provides both the name of the action and the functional area that performs it 48 List of Figures 3.13 A simple MIMO, multi-level DAG, with highlighted depen-dence relationships. The highlighted regions represent depen-dence among vertices, and should not be confused with the highlighted regions in Figure 3.10. Here, vertex F is de-pendent on D, which is itself dependent on A. As a result, vertices F, D, and A are all interdependent. Vertices I, G, D, and A are likewise interdependent, as are vertices H, E, C, and B. In highlighting these dependencies, independence be-comes apparent. Any two vertices are independent, provided they are not members of the same highlighted grouping. For example, vertex F is only dependent on vertices D and A, and is thus independent of all others. 49 3.14 The ISO/IEEE 11073 medical package model [57]. This UML diagram illustrates the relationships among the model's ele-ments. Those elements used in iAssist are highlighted in grey. 54 3.15 iAssist configured to serve as a device adapter. Custom-written input plugins translate from vendor-specific formats and protocols into ISO/IEEE 11073 for manipulation inside iAssist. The standardized data are sent in various combina-tions to custom-written output plugins, which translate back to vendor-specific formats and protocols 56 3.16 The electrocautery noise detector acts as a switch. If the ECG is clean, the ECG-derived heart rate is passed through to the output. Otherwise, the Sp02-derived heart rate is passed. . . 61 3.17 iAssist's main window, showing graphs and change point re-sults for six trends 66 3.18 iAssist's event feedback system 67 3.19 Plugin interconnections in the final iAssist configuration that our research group presently employs. iAssist processes and plots six physiological trends. Note that the ASCII File data-store plugin is omitted, for visual simplicity. . 69 ix Acknowledgements I would like to express my deepest gratitude to my supervisors, Dr. Guy Dumont and Dr. Mark Ansermino. Their guidance, support, and encour-agement throughout my research has been invaluable, and without them this thesis would not have been possible. x Chapter 1 Introduct ion 1.1 Medical Errors Harm Patients In Canada, more deaths occur after a patient experiences an adverse event in a hospital than from breast cancer, motor vehicle accidents and AIDS combined [1]. The Canadian Adverse Events study indicates that adverse events occur in 7.5% of hospital admissions [2]; this rate is similar in other developed countries [3]. Of these adverse events, 37-51% have been judged preventable [4]. Research beginning in the 1960s confronted anesthesiologists with the reality that patients were frequently injured by the same medical care that was intended to help them [5]. These findings created incentive to better address patient safety. In the 1970s and 1980s, rising malpractice insurance costs for anesthesiologists made this incentive all the more compelling [6]. In response to these issues, Harvard Medical School developed the first practice parameters for diagnosis, management, and treatment of specific clinical problems [7]. Guidelines from the American Society of Anesthesiologists (ASA) soon followed, outlining basic monitoring standards, objectives, and methods [8]. The guidelines have been amended since their inception, most recently in 2005. The current ASA standards are: • Standard I: Qualified anesthesia personnel shall be present in the room throughout the conduct of all general anesthetics, regional anes-thetics and monitored anesthesia care. • Standard II: During all anesthetics, the patient's oxygenation, ven-tilation, circulation and temperature shall be continually evaluated. The practice of surgical anesthesia has reduced its error rate nearly seven-fold since these parameters and guidelines were developed [9]. Anal-yses of the ASA Closed Claims Study reveal that respiratory events consti-tuted the single largest class of injury (34%) prior to establishment of the ASA monitoring guidelines in 1986 [10]. Death or brain damage occurred in 85% of cases. From 1986 to 2000, however, respiratory events decreased 1 Chapter 1. Introduction while cardiovascular events increased, occurring in approximately the same proportion (28%) [11]. Many adverse outcomes resulting in closed claims are still considered preventable with better monitoring [10], [12]. Patients pay the heaviest price for adverse outcomes. In addition to pain and suffering, consequences include loss of income, loss of function, loss of limb, and in the worse case, loss of life [13]. Families and loved ones suffer by association. Accordingly, there is an urgent need to provide insight into the nature of adverse events and to develop mechanisms by which they can be systematically reduced on a large scale. The consequences of adverse events have spurred a renewed national and international effort to improve patient safety. 1.2 Current Technology May Not Help Technological advances in patient monitoring have played a significant role in the effort to improve patient safety. Many new and improved sensors have been developed over the last 15 years, including pulse oximetry, automated oscillotonometric blood pressure measurement and continuous exhaled gas analysis. Routine monitoring of increasing numbers of physiological param-eters, including oxygenation, ventilation, circulation, and temperature, has also been implemented as a minimum standard of care. However, there is still much debate as to whether monitoring devices alone have been able to reduce adverse patient outcome [4], [11]. There is a compelling argument that advances in patient monitoring have actually increased the chances of human error, by adding greater complexity to the clinician's workload. In addition, the monitor may become a distrac-tion when things go wrong. One of the main causes of this paradox is that, while a monitoring system is capable of detecting abnormalities, it may not ensure that this information is effectively transferred to the clinician. Infor-mation transfer must be improved if advances in monitoring technology are to translate into better patient outcomes. The unintelligent alarm systems currently employed are a significant hin-drance to information transfer. Abnormal clinical conditions are signaled by a primitive audible or visual alarm, automatically triggered when a single parameter fluctuates beyond preset upper or lower thresholds. The thresh-olds assume that each parameter (e.g. heart rate, blood pressure, etc.) will remain within the same range for each patient over time. Natural phys-iological rhythms cause these parameters to fluctuate, however, and the acceptable range varies for each individual. False alarms are therefore in-2 Chapter 1. Introduction Figure 1.1: Physiological monitor with "Silence Alarms" button. evitable, especially since the alarm is based on a single parameter. Known clinical interventions and artifacts from electrocautery noise or movement exacerbate the situation, to the point that more than 90% of alarms can be dismissed as insignificant, 1/3 of which are triggered by artifacts [14]. This high false positive rate renders alarms ineffective because they are fre-quently ignored or disabled entirely [15], [16]. Is it any wonder that the "Silence Alarms" button is one of the most clearly marked and prevalent on physiological monitors? Advances in sensor technology within the operating room have resulted in an exponential growth in the amount of physiological data collected during a typical surgical procedure (>5MB/hr). However, only a small fraction of this information is relayed to the clinician at the bedside using a visual display or auditory signals; the remainder is unused and discarded. At present, the clinician must identify episodes of change through continued observation of the constant stream of largely uninterpreted data produced by current monitoring systems. Previous research has shown that automated post hoc analysis of this vast collection of perioperative physiological data has a high sensitivity and specificity in identifying incidents associated with increased hospital mortality that the clinician may have missed [17]. 3 Chapter 1. Introduction The clinician is unassisted in continuously observing the parameters on the monitors, and determining when a clinically significant event has oc-curred. Unlike the alarm systems, the clinician is able to understand and integrate the underlying human physiology and its rhythms. Yet main-taining attention to detect subtle changes in parameters, even occasionally, across multiple monitors is very difficult—especially during very long sur-gical procedures. To further complicate matters, the clinician is required to manage many considerations simultaneously, including the patient, mon-itors, fluid and drug delivery, and teaching students. A balance must be reached between each responsibility, and these demands can exceed the in-formation processing capacity of even highly trained, focused clinicians [18]. Indeed, attentional.overload has been identified as a fundamental cause of human error in clinical monitoring [16], often leading to poor clinical out-comes [19], [20]. 1.3 Clinical Monitoring Expert Systems Can Improve Patient Safety Despite the problem with present-day monitor alarms, there is significant promise that assistance from computerized information can improve vigi-lance, standardize clinical protocols, enhance situational awareness and re-duce errors in anesthetic practice [21]. Computers are able to monitor a large volume of diverse data rapidly while humans are limited to monitoring a maximum of seven parameters at any given time [22]. Computers could therefore improve clinician performance by executing repetitive tasks accu-rately, particularly those tasks for which humans are ill-suited, such as trend analysis and parameter surveillance. Computerized systems are also able to provide memory aids and enforce standardization of clinical guidelines. As a result, computers may reduce human errors and improve clinical outcomes. A clinical monitoring expert system could aid clinicians with patient monitoring, by diagnosing problems and providing advice on how best to avoid hazard. A number of clinical monitoring expert systems have been described [23], [24], [25], [26], [27]. These systems typically employ rule-based reasoning on patient data, using medical domain ontologies and pre-set domain knowledge. They produce a patient diagnosis, presented via a single unified alarm. While these systems show much promise, they have not been adopted into standard clinical practice. The reasons are manifold [28], including the following: 4 Chapter 1. Introduction • Many systems place insufficient emphasis on the need to capture high quality data for the decision support engine. • Decision support systems are often not based on the best available clinical knowledge. • Technology-led solutions often ignore the manner in which users will employ them. • Decision support systems are usually black boxes, and tend to treat users as "learned intermediaries." Such systems fail to address the broader legal and ethical issues that arise when healthcare profession-als blindly follow computerized advice, rather than using their own judgement. We aim to address these shortcomings of previous work, with the long-term goal of developing a clinical monitoring expert system that can be adopted into standard clinical practice. We believe we can address the problems listed above, by employing a multi-stage approach to diagnosing patient conditions: 1. Noise and artifact detection and rejection. Ensure that high quality data are used for processing, and will advise subsequent stages when only poor quality data are available. 2. Change point detection in physiological trends. Detect clini-cally significant changes in patient state, which are often indicative of adverse conditions. 3. Decision support system. Perform rule-based reasoning on patient data and change point detection results. Will incorporate an extensive clinical knowledge base, acquired from anesthesiology domain experts. The system will receive raw waveforms and trends from standard physi-ological monitors for processing. Each stage of the system will extract key information from the data for use in the next stage, until the system reaches a diagnosis. Diagnoses will be presented to the user as unified alarms, as-piring to both high sensitivity and specificity. Furthermore, alarms will be accompanied by an explanation feature, to ensure the system is not a black box. 5 Chapter 1. Introduction 1.4 An Intelligent Patient Monitor as Expert System Foundation While development of a full clinical monitoring expert system is a long-term project, there are clear incremental-components that we can build gradually over time. Thus far, we have created algorithms to serve as part of the first and second stages of the system. Figure 1.2 illustrates our clinical monitoring expert system concept. Individual algorithms in the first and second stages are collected to form an intelligent patient monitor. When enhanced with a decision support engine, it forms a complete clinical monitoring expert system. Chapter 2 of this thesis describes an electrocautery noise detection al-gorithm [29], to be used as part of the first processing stage. The algorithm will help ensure that our expert system receives high quality heart rate trend data for processing. Furthermore, we have already developed two change point detection al-gorithms [30], [31], to form part of the second processing stage. These algorithms are intended to detect clinically significant changes in six dif-ferent physiological trends: heart rate, respiratory rate, oxygen saturation, end-tidal carbon dioxide, minute volume, and non-invasive blood pressure. Our algorithms require a signal processing framework to interconnect them, provide data, and validate them through clinical study. We have therefore created a software framework to incorporate our current and future algorithms, and to serve as a user' interface for clinicians. This framework, called iAssist, is based on the Java 5.0 platform, and is described in Chapter 3 of this thesis. As iAssist does not yet employ rule-based reasoning, it cannot be called an expert system. Instead, iAssist will serve as an intelligent patient monitor in the near term. We aim to construct a solid foundation of event detection algorithms, before incorporating high-level decision support. Adoption by clinicians is essential to the success of any practical expert system, and clinical studies will help expedite our system's adoption by ensuring that it meets clinicians' needs. Chapter 3 will show through clinical study that iAssist can detect signif-icant clinical events with far greater accuracy than unintelligent threshold-based alarms, producing fewer false and insignificant alarms. It will also show that iAssist is intuitive and easy to use, helping to facilitate its adop-tion into clinical practice. 6 Figure 1.2: Clinical monitoring expert system concept. Areas highlighted in grey are addressed in this thesis. Chapter 2 A Wavelet Approach to Detecting Electrocautery Noise in the E C G 2.1 Background Electrocautery noise is a significant obstacle to be overcome in the real time implementation of clinical monitoring expert systems. The electrosurgical unit (ESU) that produces the noise is used extensively in surgery. Its ability to quickly cut tissue and coagulate blood vessels has made the ESU an indispensable tool for surgeons. To cut and cauterize, the ESU passes a high frequency current from an active electrode to a grounding plate via the patient's body. The active electrode contacts only a small amount of tissue, resulting in a high current density that causes heating which destroys tissue. As a side effect, the current passing through the patient's body corrupts measurements of small potentials at the skin surface, manifesting itself as high frequency additive noise. This noise is often so powerful that it reduces the measured potential's signal to noise ratio (SNR) to near zero. Figure .'2.1 shows the source and result of electrocautery noise in a typical operating room. The clinical monitoring expert system that we are developing will make extensive use of physiological trends in determining the patient's state. The system will receive these trend data from the traditional physiological mon-itoring equipment that is widely used today. Such monitoring equipment derives the trends from the measured physiological signals, and the signals corrupted by electrocautery noise may lead to erroneous trend data which can cause the recipient expert system to make incorrect decisions. Expert systems are in need of a noise detection component that can flag trend data as potentially corrupted, thereby identifying artifacts before these can lead to incorrect decisions. As mentioned in Chapter 1, many expert systems place insufficient emphasis on the need to capture high quality data for the 8 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG Figure 2.1: Scene from a typical operating room: (A) The E S U (B) Incision produced by the E S U (C) Resulting electrocautery noise (mistaken by the physiological monitoring equipment as ventricular fibrillation). 9 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG decision support engine [28], and we aim to address this shortcoming. A hardware approach to detecting electrocautery noise has been used [32]. The ESU's activation switch is affixed with a hardware device that notifies signal acquisition systems that the ESU is in use, so that acquisition can be halted. The device was found to halt acquisition each time the ESU was used. The developers of the NIM-2 facial nerve integrity monitor at the Michigan Ear Institute have taken a similar approach [33]. The NIM-2 incorporates a muting probe that attaches to the ESU's cabling to detect when it is activated, so that monitoring can be halted. The drawback of these hardware device designs, however, is that they detect only the source of the noise without any ability to assess its impact on the acquired signal. In many cases, such as when there is a great distance between the electro-cautery current path and the recording electrodes, the ESU will have little to no impact on measurements. These designs will nevertheless halt the ac-quisition of otherwise clean and useful signals, resulting in undesirable gaps in the acquired signals. A software approach to noise detection could eliminate the hardware designs' inherent drawback by analyzing the ESU's effect on the received physiological signal. The received signal is a superposition of the true phys-iological signal and the electrocautery noise: R(t) = S{t) + N(t) (2.1) In cases where the ESU is in use but the noise N(t) is too weak to corrupt the physiological signal S(t), the software approach will recognize that the received signal is clean and usable, whereas the hardware approach will incorrectly believe it to be corrupted. A software approach can therefore allow more, of the recorded signal to be used, aiding the expert system in making informed decisions. Of particular importance to expert systems is the heart rate (HR) trend, which is most commonly derived from the electrocardiogram (ECG) signal. The ECG is a measure of the time-varying electrical potential in the heart, and is recorded using electrodes attached to the patient's chest or back. Since the recorded ECG potentials are measured at the skin surface and are small in amplitude, this physiological signal is extremely susceptible to electrocautery noise. Because of its importance and widespread use, the ECG signal is the focus of this algorithm. A typical ECG is shown in Figure 2.2. Examples of ECG signals corrupted to various degrees by electrocautery noise are shown in Figure 2.3.. A solution to the problem must detect only 10 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG Figure 2.2: Typical ECG with P wave, QRS complex, and T wave. the presence of electrocautery noise, without being misled by other ECG features. In particular, the solution must be blind to such standard ECG features as wandering baseline, motion artifacts, and noise resulting from poor lead contact. Examples of ECG signals exhibiting these features are shown in Figure 2.4. We therefore present a software solution to detect electrocautery noise in the ECG. In recent years, much of the research into blind signal separation has focused on finding a basis in which one constituent signal is sparse and the other is not [34]. Our method is in keeping with this Zeitgeist, as it has proven very effective in practice. Though our aim is not to separate the received signal into its constituents, the same approach can be applied to the simpler goal of detecting the presence of a given constituent. If there exists a discriminating basis in which S(t) is sparse and N(t) is not, then a projection of R(t) onto ^ will reveal the presence of N(t). The choice of discriminating basis is highly dependent upon the differ-ence in structure between S(t) and N(t). The ECG has a mostly smooth, continuous structure; conversely, electrocautery noise is typically highly dis-continuous. Considering the difference in structure of our two signals, a wavelet basis seems an appropriate choice for signal discrimination. The strength of wavelet analysis is in detecting signal discontinuities [35]; there-11 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG ! i (a) 500 1000 1500 f. e 0 otherwise Such thresholding achieves sparseness for the clean ECG in the wavelet basis. Conversely, electrocautery noise is neither sparse nor compressible in this basis, so the thresholding operation has minimal impact on its coeffi-cients. Figure 2.5 illustrates the wavelet decomposition of an ECG partially corrupted by electrocautery noise. Once the thresholded decomposition is obtained, a number of isolated HWR(u,s) = 14 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG 500 1000 Sample/Coeficient Figure 2.5: ECG partially corrupted by electrocautery noise (a), and its thresholded wavelet coefficients (b). groups of closely-spaced nonzero coefficients result from the QRS complexes. By performing a localized Z°-norm operation on each coefficient group, we can effectively measure the duration of its discontinuities. The Z°-norm of a vector is defined as: N l o = £ > i | ° (2.5) i Since the ECG's QRS complexes are always brief in time, they result in a short discontinuity duration. In contrast, the ESU is typically used over a much longer time period, so it produces electrocautery noise discontinuities over a correspondingly long duration. The ?°-norm of each coefficient group is therefore used to discriminate standard ECG features from electrocautery noise. 2.2.2 Algorithm The algorithm has been developed using the principles explained in the previous section. It receives an ECG signal block, and returns the duration of electrocautery noise that corrupts the block. It executes in four steps: 15 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG The ECG block is decomposed to a single level using a Daubechies-10 analyzing wavelet. A hard thresholding operation is applied to the resulting wavelet coef-ficients. All coefficients with amplitude smaller than the threshold are nullified, leaving those greater than the threshold unaffected. We are now left with a number of isolated groups of closely-spaced nonzero coefficients. A localized Z°-norm operation is performed on each coefficient group to determine its width. Non-zero coefficients can be separated by up to a user-specified number of zeroed coefficients and still be treated as belonging to the same group. The group widths are inspected to determine whether they are the result of standard ECG features or electrocautery noise. Any group wider than a user-specified number of coefficients is treated as noise. The widths of the groups attributed to noise are summed and returned as the total duration of electrocautery noise in the signal block. Using the algorithm's returned duration, an expert system can estimate the reliability of the HR trend derived from the ECG signal block. 2.2.3 Testing Following Ethics approval (Appendix A), ECG data were collected from 15 cases, amounting to 38.5 hours of data. Data were collected from the anes-thesia monitors using on-line clinical data collection software (S/5 Collect® - GE Medical) at a sampling rate of 300 Hz. The cases span operations from start to finish, including the periods of ECG lead attachment, induction of anesthesia, surgery, patient awakening, and ECG lead detachment. The S/5 Collect software can process and output ECG data in block sizes of one second to 30 seconds. In a typical expert system, the highest resolution required for HR would not be less than five seconds. This was therefore chosen as the ECG block size in S/5 Collect. Electrocautery noise detection was tested on ECG blocks of the same size, as the algorithm's intent is to flag potential ESU-related artifacts in the HR trend. Prior to testing, each five second ECG block was inspected and flagged as either clean or corrupted in order to establish a solid baseline against which the algorithm's output could be compared. Inspection was performed in three steps: 16 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG 1. The E C G block was inspected visually for signs of electrocautery noise, and an ini t ia l assessment was made. High frequency corruption such as that shown in Figure 2.3 was assessed as electrocautery noise. 2. The result from a prototype hardware E S U detector was consulted when available (the hardware was present for 13 of the 15 cases). The prototype proved to be somewhat unreliable, often missing obvious periods of E S U use. Nevertheless, it did help to characterize the noise and to distinguish it from other sources of E C G corruption. The proto-type's result occasionally imposed a revision of the ini t ia l assessment. 3. The result from the software noise detection algorithm was consulted. We stress that the algorithm's result was only observed after a noise assessment had been made visually and verified in hardware, and that it rarely resulted in a change in this assessment. Interestingly, the al-gorithm's result did occasionally detect very weak electrocautery noise that had been overlooked in prior steps. The algorithm was then executed on each E C G block, and the output compared to the baseline results. Blocks that were clean in the baseline but flagged as noisy by the algorithm were denoted as false positives (FPs) . Conversely, blocks that were noisy in the baseline but flagged as clean by the algorithm were denoted as false negatives (FNs). Note that although the algorithm is able to judge the duration of electrocautery noise in the E C G blocks, only the presence or absence of noise was considered for testing, as it is difficult to achieve a precise measure of noise duration in the baseline. 2.3 Results Depending on the application, and the expert system receiving the data, more importance may be assigned to either F P s or F N s . The algorithm can be tuned to be more sensitive (more F P s but fewer FNs) or more spe-cific (fewer F P s but more FNs) , using two parameters. One parameter, MaxWidth, defines the maximum allowable width of a clean coefficient group. A n y group wider than MaxWidth is treated as electrocautery noise. The other parameter, MinDist, defines the minimum separation distance between distinct coefficient groups. A n y two nonzero coefficients that are separated by less than MinDist are treated as part of the same group. Testing results indicate that setting MaxWidth = 24 and MinDist = 4 provides a good tradeoff between F P s and FNs . Figure 2.6 plots the error 17 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG Table. 2.1: Algorithm results for MaxWidth = 24 and MinDist = 4. Units in number of five second ECG blocks. * indicates FPs, and f indicates FNs. Baseline Assessment Clean Noisy Algorithm Result Clean 26330 4t Noisy 187* 1226 rates resulting from different values of MaxWidth, with MinDist fixed at a value of 4. Figure 2.7 plots the error rates resulting from different values of MinDist, with MaxWidth fixed at a value of 24. 1.1 0,3' 1 > 1 1 1 1 1 1 1 23 24 25 26 27 28 29 30 31 32 Maximum Clean Width (MaxWidth) Figure 2.6: Error rates for tuning of MaxWidth, with MinDist = 4. Table 2.1 provides a detailed breakdown of the results for the chosen parameter values. 2.4 Discussion With proper tuning, the wavelet-based software algorithm can detect elec-trocautery noise in the ECG with high accuracy. In fact, the algorithm's 18 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG Minimum Separation Distance (MinDist) Figure 2.7: Error rates for tuning of MinDist, with MaxWidth — 24. effective error rate may be lower than it appears. It is worth noting that 63 of the 187 FPs result from the following signal abnormalities: • High frequency noise caused by poor lead contact at the beginning and end of surgery. • High frequency artifacts, possibly caused by an external noise source or disruption of the lead contact. • Closely-spaced motion artifacts. Though not electrocautery noise, these abnormalities have an equally disruptive effect on the physiological monitoring equipment's ability to de-rive the HR trend data. Since our method has been developed to detect electrocautery noise only, the abnormalities were treated as FPs. Because of their disruptive effect, however, detection of the abnormalities is not nec-essarily detrimental. The algorithm does have limitations. We observed cases in which elec-trocautery noise manifests itself as a smooth and continuous low frequency sinusoid, in contrast to the discontinuous high frequency noise addressed by this algorithm. Wavelet analysis is not adept at detecting such noise, as this basis cannot discriminate it from the ECG. Indeed, all FNs presented in the 19 Chapter 2. A Wavelet Approach to Detecting Electrocautery Noise in the ECG previous section are the result of this manifestation of electrocautery noise. It is worth noting, however, that the low frequency electrocautery noise does not have a significant impact on the monitoring equipment's ability to derive the HR trend data. Nonetheless, this limitation could be addressed through the addition of Gabor atoms (short-time Fourier transform) to detect this manifestation of electrocautery noise. Another concern is that heart rhythm abnormalities such as ventricular fibrillation, which often result in a highly discontinuous ECG signal, could be mistaken for electrocautery noise. This problem may be addressed through parallel analysis of the plethysmogram, which is generally immune to elec-trocautery noise. A plethysmogram arrhythmia detector could override the ECG electrocautery detector. 20 Chapter 3 iAssist: A n Intelligent Patient Moni tor 3.1 Background Chapter 1 explained the need for a clinical monitoring expert system for patient monitoring, and described our staged approach to developing such a system. Initially, individual building blocks can be created and tested in the lab-oratory against pre-recorded patient data. Eventually, however, we must perform validation in broader clinical studies on live patients. Such studies will provide us with results from a large number of surgical cases and pa-tients. The result we collect will guide us in improving our existing building blocks, and in creating new ones. To perform such clinical studies, we must combine the individual build-ing blocks together into a working patient monitor that clinicians can use in a peri-operative environment. We have created an intelligent patient monitor to address this need. The monitor's objective is to assist clinicians in providing quality care to patients, and to assist our research group in conducting clinical trials on our algorithms. In keeping with the monitor's purpose, we called it iAssist. This chapter describes iAssist in depth. It begins with an examination of high-level requirements, and then proceeds to discuss the manner in which we address these. It describes iAssist's platform, architecture, design, de-velopment, and testing. The chapter concludes with the results of our first clinical study of iAssist. 3.2 Requirements We must consider the requirements that iAssist is to satisfy. Many deliv-ered software systems do not meet users' needs, due in part to ineffective requirements engineering [37]. Indeed, failure to properly gather require-21 Chapter 3. iAssist: An Intelligent Patient Monitor merits is the single biggest cause of software project failure [38]. Good up-front understanding of requirements (both in breadth and depth) leads to smoother development execution and a more successful end product. It would be shortsighted to design iAssist based solely on present-day require-ments. By considering possible future requirements and designing to accom-modate them, we will mitigate future redesign and software maintenance efforts [39]. The following subsections detail iAssist's present and future require-ments, divided into functional areas. The last subsection is devoted to general nonfunctional requirements. 3.2.1 Data Acquisition In order for iAssist to process physiological signals, it must have the abil-ity to acquire data from physiological monitoring sources both offline (pre-recorded) and online (in real time). At British Columbia's Children's Hospital, where iAssist is intended for testing and distribution in the near term, most operating rooms are equipped with Datex/Ohmeda AS/3® physiological monitors. Datex/Ohmeda has provided our research group with a LabVIEW®-based data acquisition sys-tem called S/5 Collect®, which records physiological data from the AS/3® monitors. S/5 Collect® can save recorded data in text files for subsequent offline viewing and analysis, and can transmit the data, to generic recipient systems over TCP/IP using custom data frames. In the near term, iAssist must acquire data sent by S/5 Collect® over TCP/IP in real time, and load pre-recorded S/5 Collect® data stored in text files. In the medium term, iAssist must have the ability to acquire data directly from the AS/3® monitors in real time. In the long term, we must plan for a future large scale iAssist distribution, in which users may employ hardware other than the Datex/Ohmeda AS/3® monitors. Ideally, iAssist should be compatible with any physiological monitor. 3.2.2 Physiological Signal Processing Our research group has already developed two change point detection al-gorithms [30], [31], and these must be incorporated in iAssist. These algo-rithms have significant differences in their approach, but share a common input/output interface: both receive a single time varying trend value as input, and both output a numerical change point result. 22 Chapter 3. iAssist: An Intelligent Patient Monitor iAssist must also incorporate the electrocautery noise detection algo-rithm detailed in the Chapter 2, to reject heart rate trend artifacts. Based on the algorithm's ECG quality estimate, iAssist can decide whether change point detection should be performed on the ECG-based heart rate, or on an alternate heart rate measure such as that from the arterial blood pressure or the pulse oximeter. This algorithm must therefore receive several inputs of different types: the ECG waveform, the ECG-based heart rate trend, and an alternate heart rate trend. Furthermore, our research group plans to develop algorithms for cal-culating time-varying statistical properties in physiological trends. These algorithms will receive a single time-varying trend value as input, and pro-duce multiple statistical values as outputs. Eventually, our research group aims to produce multivariate change point detection algorithms that process multiple interrelated trends. These algo-rithms could produce more accurate results by using a priori knowledge of trend relationships. This class of algorithm will receive multiple trends as input, and produce a single change point value as output. Finally, iAssist will be used in the near- and medium-term as a test bed for the algorithms described above. A user may wish to adjust an algorithm's tuning parameters, for example to increase or decrease its sensitivity. iAssist should provide the means to run the same algorithm against the same data with multiple different tuning parameters, allowing users to "batch tune" an algorithm. In considering all requirements described above, it is clear that iAssist must support multiple classes of algorithm running simultaneously on vari-ous trends and waveforms. These algorithms may receive and produce any number of inputs and outputs, respectively. iAssist must support an algo-rithm hierarchy, as the outputs of some algorithms will be required as inputs to other algorithms. Finally, iAssist must support the simultaneous execu-tion of multiple copies of the same algorithm against the same inputs, with varying tuning parameters. 3.2.3 Data Storage When a user is batch tuning an algorithm (as described in the previous section), he may wish to compare results in an external data analysis appli-cation such as Matlab®. iAssist must therefore store all physiological trend and waveform data, algorithm results, and associated tuning parameters. Storage must be performed in such a way as to allow extraction of the data into a format that external applications can read and interpret. 23 Chapter 3. iAssist: An Intelligent Patient Monitor In the long term, iAssist may be widely distributed across multiple op-erating rooms, and store its data in a shared SQL database. iAssist must therefore support data storage in a variety of formats and repositories. 3.2.4 Graphical User Interface iAssist must have a graphical user interface (GUI) in order for the user to interact with the system. The user must also have the means to view time-varying historical trend and waveform data for each surgical case. His-torical data contain contextual meaning that helps anesthesiologists with monitoring. In addition, iAssist must provide the means for users to visually observe algorithm results. For example, if an algorithm detects an increase in the heart rate trend, the user should be alerted to this significant event via an "increase" indicator overlaid on the heart rate trend graph. In the future, iAssist may require more complicated overlays on the graphs, including the results of the trend statistics algorithms described earlier. Finally, iAssist's GUI must be designed in consideration of its intended environment and users. iAssist will be used primarily in the operating room in real time, by anesthesiologists who are often overloaded and unable to dedicate much of their attention to the system. The operating room envi-ronment is already overly cluttered with numerous tools, monitors, electric cables, and gas/fluid tubes. Our aim should be to minimize additional envi-ronment clutter. Delivering iAssist on a standard computer, with user inter-action via traditional peripherals including mouse and keyboard, is there-fore impractical. iAssist should feature a touch screen interface, minimize required user interactions, and present information in such a manner as to expedite its transfer to the user. The iAssist GUI should also be designed to approximate the appearance of traditional physiological monitors already in widespread use, to help users feel comfortable using the system. 3.2.5 Output to External Devices iAssist must have the ability to output data and algorithm results to any number of external devices, which may consist of hardware or software. Two members of our research group, postdoctoral fellow Pierre Barralon and M.A.Sc student Ginna Ng, are developing a tactile display device for use in the operating room. This wearable hardware device is intended to alert anesthesiologists of significant adverse patient events, such as a dramatic increase or decrease in heart rate, respiratory rate, and more. Since iAssist's 24 Chapter 3. iAssist: An Intelligent Patient Monitor underlying algorithms are capable of detecting such events, iAssist is the ideal driver for the tactile display. In the long term, iAssist may be used to control automated drug infusion pumps, using algorithms such as the wavelet-based anesthetic value (WAV) for depth of anesthesia estimation [40]. 3.2.6 Nonfunctional Requirements In addition to the functional requirements described in previous sections, there are a number of nonfunctional requirements that iAssist must meet. Nonfunctional requirements pertain to the software's quality; in particular, its reliability, accuracy, performance, scalability, security, and liveness [41]. While nonfunctional requirements are typically less directly visible to the end user than are their functional counterparts, they are just as impor-tant to a software system's ultimate success. Nonfunctional requirements are particularly important in real-time- point-of-care medical systems like iAssist. iAssist must be reliable, to ensure it is always available to assist the anesthesiologist in the operating room whenever needed. In particular, sys-tem crashes are unacceptable. It must also be accurate, to ensure that "true" physiological data and algorithm results are presented to the user. Presentation of inaccurate information may lead an anesthesiologist to take inappropriate action, possibly resulting in patient harm. The performance of iAssist must also be sufficient to meet the real-time data processing requirements of its intended point-of-care environment, in order for it to assist the anesthesiologist in a timely fashion. Were iAssist unable to keep pace with the patient's physiological data, the anesthesiol-ogist could be alerted to adverse events well after they occurred, possibly when corrective action is no longer feasible. Liveness is also an important nonfunctional consideration in iAssist. Liveness pertains to a system's availability and responsiveness; for exam-ple, a system with poor liveness may "hang" periodically, forcing the user to wait for control to return before he may interact with it. In iAssist's in-tended medical point-of-care environment, poor liveness can result in undue distractions and delays for the anesthesiologist. Finally, iAssist must be scalable to support an arbitrary number of run-ning algorithms, trend/waveform graphs, and output devices. Failure to meet scalability requirements can result in a system of poor performance or liveness under heavy load. 25 Chapter 3. iAssist: An Intelligent Patient Monitor 3.3 Platform We must choose a platform on which to architect the system. In the case of iAssist, this choice is largely driven by the nonfunctional requirements; in particular, those of reliability, performance, and scalability. We chose the Java Standard Edition (JSE) 5.0 platform for iAssist. 3.3.1 Advantages of Java Java offers a number of compelling advantages over other programming lan-guages, particularly for a system with the complex requirements of iAssist. This section details these advantages, and explains their importance to iAs-sist. Comparisons will be made primarily with C++, the most commonly used programming language for desktop software systems. The most widely recognized advantage of Java—and indeed its original promise—is seamless cross-platform compatibility [42]. While other lan-guages require that software must be custom designed and compiled for each intended platform, Java software can be compiled once and run on any hardware or operating system. This "write once, deploy anywhere" promise is not always easy to achieve in practice, even with Java. In particular, differences in operating system GUI capabilities can result in application limitations or inconsistencies. Nevertheless, Java brings the developer much closer to the cross-platform compatibility goal than can any other language. This is advantageous for iAssist, as end users may not wish to entrust their patients' safety to the notoriously unreliable Windows platform. Though written on a Windows PC, iAssist can be run on Mac OS, Unix, Linux, and more. Furthermore, Java provides an excellent balance between functionality and ease of development unparalleled in any other programming language of its class. Software developers are often more productive when develop-ing in Java compared to C++ [43]. Java applications are easier to design, implement, and debug, resulting in faster development cycles and an over-all shorter time from concept to deployment. This productivity gain can be attributed in part to the Java software development kit (SDK), which includes a rich set of libraries to facilitate development. The SDK pro-vides important application building blocks, including data structures, file I/O, and GUI frameworks. Many of these building blocks are unavailable in other programming languages, requiring that the developer custom-write them. iAssist makes extensive use of the Java SDK. Most importantly for iAssist, software developed in Java is recognized 26 Chapter 3. iAssist: An Intelligent Patient Monitor to be more robust and of higher quality overall than that developed in other programming languages, such as C++ [43]. One important technology Java provides is the "garbage collector," a background thread that automati-cally frees memory when no longer needed. The garbage collector abstracts away the details of memory reclamation from the user. In other languages, such as C++, the developer must manually write code to reclaim memory. This renders software development more difficult and error prone. Software developed in C++ is often plagued by "memory leaks," as developers occa-sionally forget to perform the manual memory reclamation., Memory leaks result in a gradual increase in the software's memory footprint and ulti-mately cause system crashes. For various reasons Java's garbage collector cannot completely eliminate memory leaks; however, it dramatically reduces their prevalence and severity. Finally, Java provides an extensive and powerful library of concurrency tools. Concurrency is requisite for scalability, and developing reliable con-current systems is among the most difficult challenges in software engineer-ing. Java's concurrency library greatly facilitates concurrent development, and helps ensure it is done properly and efficiently. 3.3.2 Java Myths & Controversy Java is perhaps the most controversial programming language in use today; indeed, no discussion of Java is complete without mention of its shortcomings and the myths surrounding them. This section presents a number of common statements proclaimed by Java dissidents, and disputes or clarifies them as appropriate. • Java is slower than C. While this statement is true, it is largely exaggerated. In order to achieve cross-platform compatibility, Java was designed as an interpreted language. Java code is compiled into "bytecode" for the Java Virtual Machine (JVM) and not the desti-nation hardware (Intel x86, IBM Power, etc.). At runtime, the JVM translates Java bytecode into platform-specific machine code, slowing execution in the process. In the early days of Java, this translation resulted in a significant performance penalty. Sun has since improved its JVM, and with the advent of the just-in-time (JIT) compiler Java's performance now approaches that of C/C++ code. • Java cannot be used for real-time systems. This statement is largely false, though there are indeed certain types of real-time sys-tem for which Java is inappropriate. By definition, real-time systems 27 Chapter 3. iAssist: An Intelligent Patient Monitor receive data as they are generated, and must process them under time constraints [44]. Time-constraints define the operational deadlines from real-world event (data received) to system response (data pro-cessed) . A system with "hard" constraints must provide a response at a precise and unvarying moment in time; in contrast, a system with "soft" constraints can tolerate some delay. Java is inappropriate for hard real-time systems-, as its garbage collector adds some inherent un-certainty in the response time. Java is perfectly acceptable for many soft real-time systems, as the garbage collector can only delay the re-sponse time by a few hundred milliseconds. iAssist is a soft real-time system, and Java is an acceptable platform for development. • Java's abilities are extremely limited. This statement is largely a holdover from the early days of Java, when it was used primarily for creating Web applets (small applications that run in a Web browser). Java has since gained acceptance for a broader range of applications, and can be used to develop everything from personal desktop applica-tions to distributed enterprise server systems. 3.4 Framework Architecture We designed iAssist using the top-down reductionist approach to engineer-ing, whereby the problem is incrementally decomposed into successively smaller modules [45]. The result is a hierarchical problem whose individ-ual modules are smaller in scope and more manageable than they would be otherwise. This approach allows for module reusability, easier design and development, and often leads to a better-engineered solution than alterna-tives. Figure 3.1 illustrates the reductionist approach to engineering, as applied to automobile design. At any given level of the hierarchy, there are a number of modules that must be designed. Design of a module requires an understanding of four aspects: 1. The module's purpose and internal workings. 2. The sub-modules that comprise the module. 3. The manner in which sub-modules interact to achieve their collective purpose. 4. The manner in which the module interacts with its super-module in the hierarchy. 28 Chapter 3. iAssist: An Intelligent Patient Monitor Engine Electrical System Body Brake System Figure 3.1: Example of the reductionist approach to engineering, as applied to automobile design. The automobile can be decomposed into smaller mod-ules, forming a design hierarchy. 2!) Chapter 3. iAssist: An Intelligent Patient Monitor In the automobile example provided in Figure 3.1, the top-level auto-mobile is decomposed into its comprising modules. The automobile can be designed to interconnect these modules, before the modules themselves are designed. Of particular note is the fact that the higher level hierarchy modules impose their own design requirements on their comprising sub-modules. Clearly, we must design the higher level modules before proceeding to the lower levels. The top-down design approach is therefore well suited for this task. For the purposes of this thesis, only the top two levels of the iAssist design are relevant; details of lower-level design are too numerous and too small in scope to warrant treatment here. This chapter describes the top-level architecture and the functional areas that comprise it. This section describes the top-level architecture, and the next will describe the lower-level functional areas. Among the two levels of iAssist design discussed herein, the most diffi-cult task is in architecting the top-level framework, which must interconnect all comprising functional areas such that they operate together to meet the overall system requirements. For example, Section 3.2.2 details the hierar-chical nature of individual physiological signal processing algorithms; this hierarchy imposes design constraints on how the algorithms must commu-nicate with each other. Furthermore, the requirements impose constraints on how different functional areas must communicate. For example, the al-gorithms must receive physiological trends and waveforms from the data acquisition component. At the highest level, iAssist's functional areas must interact to accom-modate the desired workflow, depicted in Figure 3.2. The workflow steps can be enumerated as follows: 1. Acquire physiological data from an input source. 2. Display the data on-screen. 3. Process the data using the signal processing algorithms. 4. Display the signal processing results on-screen. 5. Store the data and signal processing results. 6. Output the data and signal processing results to external devices or applications. 30 Chapter 3. iAssist: An Intelligent Patient Monitor 4. Display Results flf-*^ (GUI) JF""^ Figure 3.2: The iAssist workflow consists of the six steps shown here. Each block represents a step, and provides both the name of the action and the functional area that performs it. Each step must be performed in sequence, and the workflow must repeat until the user instructs iAssist to stop or until all input data are consumed. The following subsections describe the iAssist framework architecture, which includes four facets: extensibility, flexibility, scalability, and interop-erability. We devote one subsection below to each facet, and discuss how each contributes to iAssist's requirements and workflow. 3.4.1 Extensibility Extensibility allows iAssist to meet future requirements, by acquiring data from new sources, incorporating new signal processing algorithms, storing data in new formats, outputting data to new devices, and displaying data in new ways. Extensibility supports the iAssist workflow depicted in Figure 3.2, by allowing users to seamlessly integrate various functional components into the flow. Users can integrate any number of input sources, signal pro-cessing algorithms, datastores, GUI components, and output devices, all while maintaining the core iAssist workflow. 31 Chapter 3. iAssist: An Intelligent Patient Monitor Figure 3.3: The iAssist extensible framework is analogous to a puzzle. Dif-ferent pieces can be connected, provided they have the correct shape (inter-face). iAssist is more general than a puzzle, however, in that multiple pieces can be simultaneously connected to the same slot. Extensibility must be custom designed into each functional each has its own unique requirements. For example, Data Acquisition can be modeled as an output-only functional area, as it provides data to iAssist, but does not receive anything in return. In contrast, others such as Data Storage can be modeled as an input-only functional areas, as they receive data from iAssist but do not produce anything in return. Signal Processing can be modeled as a combination of these two, as it is an input-output functional area. Every major functional area must exist outside the framework, yet in-clude the ability to connect and extend the framework where appropriate. The iAssist extensible framework is analogous to a puzzle, in which unique pieces can connect to different areas, and each piece's placement is specified by the shape of its connector. Five types of piece can connect to the iAssist puzzle, each in its own place as per its functional area: data acquisition, signal processing, data storage, GUI, and output. The framework puzzle is illustrated in Figure 3.3. This framework's extensibility allows the user to add and remove com-32 Chapter 3. iAssist: An Intelligent Patient Monitor ponents (called "plugins" in iAssist) at will, without making any changes to the core framework. The same iAssist framework can be used different tasks, provided the user connects the appropriate plugins. Extensibility im-plies interchangeability, so that the user can apply different plugins for the same purpose as needed. Automobiles provide a simple real-world example of interchangeability. All vehicles of a certain class (e.g. automatic compact cars) present a com-mon interface to the driver: all have a steering wheel and gas and brake pedals. Provided a vehicle adheres to this interface, anyone with a driver's license can operate it. For our purposes, iAssist is analogous to the driver, and the plugins to the vehicles. iAssist must specify the common interfaces for each type of plugin it is able to operate, and any plugin available to iAssist must adhere to these. The List is a typical programmatic example of interchangeability through common interfaces. Provided by Java, it defines methods for adding/ remov-ing entries to/from a list, iterating over the list, obtaining the size of the list, obtaining subsets of the list, and more. Any number of implementa-tions of the List interface may exist, including ArrayList and LinkedList, shown in Figure 3.4. If an object (ObjectA) needs to obtain a list of data from another (ObjectB), it should do so via the List interface, rather than through the direct implementing class. The specific implementation is thus abstracted away from ObjectA, allowing ObjectB to use any implementation appropriate for its purposes. Furthermore, this abstraction allows ObjectB to change its List implementation in the future without breaking its connec-tion to ObjectA. The use of interfaces for abstraction is among the major principles of object oriented programming. In iAssist, plugins are likewise made interchangeable through common interfaces. The inner workings of the individual plugins should be abstracted away from the iAssist extensible framework, and vice-versa. Relating to the automobile example above, the driver should not be concerned with how the vehicle executes his commands (e.g. brake pedal is connected to the brake pads, and friction between the pads and the wheels causes the car to slow) nor should the vehicle be concerned with how the driver will use it (e.g. brake, then gas, then turn, etc.). Such abstraction keeps the framework general and versatile, which facilitates plugin development by allowing for future plugins of unanticipated purposes. Furthermore, abstraction often simplifies plugin development, and good design methodology dictates that abstraction should be maximized. iAssist's plugin framework employs a hierarchy of interfaces to define each functional area. The hierarchy is shown in Figure 3.5. Some interfaces 33 Chapter 3. iAssist: An Intelligent Patient Monitor List :+size():int-h«::^ vadd(value Object) boolean +remove(va!ue Object) boolean +iterator() Iterator +subList(from Index: int,:tolndex: frit): List • • • -data -data LinkedList Figure 3.4: UML diagram showing a portion of the List interface, and im-plementing classes ArrayList and LinkedList. The interface is highlighted in grey, to distinguish it from the concrete implementing classes. in the hierarchy are terminal; that is, they must be directly implemented by any plugin intended for use within iAssist. Other interfaces are intermediate, providing a logical collection of methods that can be reused by others, but cannot be implemented individually by iAssist plugins. Together, these interfaces form an inheritance structure adherent to the principles of object oriented, programming. Each interface presents a logical collection of functionality. When ac-cumulated through the inheritance hierarchy into the terminal interfaces, this functionality provides iAssist with the means to interact with plugins as required. The following is a brief overview of each interface's purpose; each item describes only the functionality provided directly by the interface, and not by its superinterfaces: • Plugin: This intermediate interface is the root of all plugins. It pro-vides iAssist with the means to access the plugin's name, and to ac-cess/modify its internal configuration. • DataOutputPlugin: This intermediate interface defines a plugin that sends data to iAssist. It provides descriptions of the data that the plugin can send, as well as the means for iAssist to receive these data. 34 Plugin +getName(): String +getDefaultConfig(): Config +getConfig():Config ' j :+setConfig(value:Config):void;; +reset():void DataOutputPlugin +getOutputNames():List , +setOutputQueues(value:List):voidr sDatalnputPlugin +getlnputNames():List - . , +setlnputQueues(value:List):void:> InputRlugir. AigorithmPlugin +getGraphlcon():Graphlcon GraphPlugin +newDisplayableCmpnt():DisplayableComponent +setlconQueues(valueiist):v CO Cn Figure 3.5: UML diagram showing the iAssist plugin interface hierarchy. Intermediate interfaces are highlighted in grey, to distinguish them from the terminal interfaces. Chapter 3. iAssist: An Intelligent Patient Monitor • DatalnputPlugin: This intermediate interface defines a plugin that receives data from iAssist. It provides descriptions of the data that the plugin can receive, as well as the means for iAssist to send these data. • AlgorithmPlugin: This terminal interface defines a plugin that can perform signal processing tasks. It provides the means for iAssist to receive graphical representations of algorithm results, so that these can be presented to the user in the GUI. . • GraphPlugin: This terminal interface defines a plugin that can graph data in the iAssist GUI. It provides the means for iAssist to create and customize GUI components. In particular, it can receive graphical results from the AlgorithmPlugins, to be presented to the user. • InputPlugin, OutputPlugin, &; DatastorePlugin: These termi-nal interfaces define plugins that can perform data acquisition, data output, and data storage, respectively. They do not provide any ad-ditional functionality to iAssist, and serve only as logical placeholders in the interface hierarchy. For the sake of brevity, we will not discuss the data structures passed to and returned from the plugin interface methods. While there are many interesting design details behind these structures, their discussion is beyond the scope of this thesis. 3.4.2 Flexibility Flexibility builds upon extensibility, and it is required in all functional areas. Once plugins are connected to the iAssist extensible framework, flexibility serves as a mechanism to model their relationships. At the highest level, the plugin relationships are depicted in Figure 3.2. iAssist must connect the input plugin to the datastore, signal processing, GUI, and output plugins. It must also connect the signal processing plugins to the datastore, GUI, and output plugins. These interconnections, which allow the iAssist workflow to proceed as required, must be flexible enough to allow for the extensibility described in the previous section. When the user changes input plugins, for example, iAssist must have the ability to incorporate the new plugin into the workflow. More generally, the user may add, remove, or exchange any number of plugins across various functional areas, and iAssist must connect them for execution. 36 Chapter 3. iAssist: An Intelligent Patient Monitor Though these high level relationships are relatively simple to model, flex-ibility becomes more complicated at lower levels. One level of the complexity stems from the multiple input, multiple output (MIMO) nature of iAssist plugins. The input plugins alone can produce a vast array of data to iAssist, including over 100 different physiological trends and waveforms. The signal processing plugins may require data from any number of these trends and waveforms, and iAssist must connect the plugins together via their individ-ual inputs and outputs. Plugin inputs and outputs must also be connected in the proper order. For example, a signal processing plugin with two inputs may expect the heart rate trend as its first input and the electrocardiogram as its second. If iAssist were to reverse the input order, the plugin could not function. The same logic applies to plugin outputs. This level of plu-gin interconnection is shown in Figure 3.6. As mentioned earlier, iAssist must have the ability to dynamically rearrange the interconnections to meet changing user requirements. A deeper level of complexity emerges from the configuration of plugins, in order to reuse them for different purposes. Consider, for example, a signal processing plugin that performs the wavelet transform. It receives a time domain signal as input, and outputs the signal's scale domain decomposi-tion. An iAssist user may wish to apply the wavelet transform to multiple signals (e.g. the electrocardiogram and the plethysmogram). It would be impractical to install two identical wavelet transform plugins to meet this requirement, assigning one plugin to each signal. More generally, it would be impractical to continually add or remove copies of the same plugin as requirements change (e.g. processing a third signal, the arterial blood pres-sure). Ideally, a single wavelet transform plugin should be applicable to any number of different signals. iAssist addresses this scenario by supporting multiple distinct instances of a given plugin. An instance is an in-memory copy of a template. Just as Java allows a programmer to create multiple copies (instances) of a class (by invoking the "new" operator), iAssist must allow users to create multiple instances of a given plugin. In the wavelet transform example above, the user must only install one wavelet transform plugin, and configure iAssist to create a number of instances from it. Each instance would be assigned to a single signal. Furthermore, some plugins require custom tuning for each signal. For example, different physiological waveforms may require decomposition via different mother wavelets or to different levels of decomposition, as required by the waveform's custom characteristics. Our change point detection al-gorithms [30], [31] are another example. These can operate on a number 37 Chapter 3. iAssist: An Intelligent Patient Monitor Chapter 3. iAssist: An Intelligent Patient Monitor Change Point Detection Instance 1 Configuration 1 Figure 3.7: An example of iAssist plugin instances. Two signal processing plugins are available to iAssist: Wavelet Transform and Change Point De-tection. Plugins are highlighted in grey to distinguish them from instances. of different physiological trends, but require custom sensitivity, safeguards, and other tuning parameters to produce correct results. Each instance can therefore have its own configuration, specifying which signals it is receiving and which tuning parameters to apply. There is a final layer of complexity in the interconnections between dif-ferent signal processing plugins. These interconnections apply only to signal processing plugins, as these are the only plugins that may communicate with others in the same functional area. Signal processing plugins may require a hierarchical configuration to perform compound actions, in which one ac-tion depends on the results of another. We must therefore elaborate on the 39 Chapter 3. iAssist: An Intelligent Patient Monitor Wavelet Transform Instance 1 Configuration 1 b Waveform Result Wavelet Transform Instance 2 Configuration 2 j> Waveform Result Scale Analysis Instance 1 Configuration 1 Scale Analysis Instance 2 Configuration 2 b Scale Decomp. Result i j> Scale Decomp. Result Figure 3.8: An example two-level signal processing analysis hierarchy. The Wavelet Transform is first applied to the ECG and plethysmogram. Scale Analysis is then applied to the resulting wavelet decompositions. iAssist workflow shown in Figure 3.2, adding more depth to Step 3. Returning to our wavelet transform example, a user may wish to use iAssist for time-scale analysis. He would configure iAssist with two signal processing plugins: one performs the wavelet transform, and the other an-alyzes the resulting scale domain representation for components at various scales. The output of the wavelet transform plugin must be connected to the input of the analysis plugin, and the plugins must be executed in the correct order. The user would employ two plugins for the sake of reusability; in this configuration, the standard wavelet transform plugin could be used to process other signals for unrelated purposes. Figure 3.8 illustrates this plugin hierarchy. This hierarchical nature is necessary for our research group's present-day purposes. The electrocautery noise detection algorithm, described in a previous chapter, is integrated into iAssist as a signal processing plugin, as are our change point detection algorithms [30], [31]. The electrocautery noise detector rejects heart rate artifacts, providing a clean heart rate to the change point detector. We therefore require two layers of signal processing plugins in iAssist at present, and will likely require further layering in the 40 Chapter 3. iAssist: An Intelligent Patient Monitor Figure 3.9: An example of a MIMO, multi-level DAG. Data flow from the source vertices (A, B, C) through to the sink vertices (H, I). future as we develop more advanced and interrelated algorithms. We must design the signal processing plugin interconnections to be as general as possible, allowing for any number of related plugin instances that pass results through the hierarchy. Furthermore, iAssist must allow for any number of independent hierarchies. The structure that best meets these requirements is the directed acyclic hypergraph (DAHG). A graph is a set of vertices connected by edges, and is a generalization of a tree structure, in which subtrees can be shared by different parts of the tree. A graph can have any number of root and leaf vertices (called sources and sinks, respectively). A directed graph has directed edges, so that the edge between two vertices can only be traversed in one direction. An acyclic graph is constrained such that it does not have feedback loops. Finally, a hypergraph is a generalization of a graph, in which a single edge can connect any number of vertices. A DAHG is a combination of all the properties described above [46]. DAHGs are difficult to illustrate in two dimensions, so we provide a detailed image of a directed acyclic graph (DAG) in Figure 3.9, and a very basic DAHG in Figure 3.10. iAssist models the relationships between signal processing plugin in-stances as a DAHG. Plugin instances are the vertices, and the data de-41 Chapter 3. iAssist: An Intelligent Patient Monitor Figure 3.10: A simple example of a D A H G . Vertex A is the source, and vertices B , C , D , and F are the sinks. A single edge connects vertices A , B , and C . Another edge connects vertices A , D , and E . A final edge connects vertices E and F . Note that edge directionality has been omitted for visual simplicity. 42 Chapter 3. iAssist: An Intelligent Patient Monitor pendencies are the edges. Source vertices are those instances that depend only on data produced by the input plugin, and not on the results of any other algorithms. Sink vertices are those instances on which there are no dependencies; that is, no other algorithm depends on their results. Execu-tion flows from the source vertices to the sink vertices, and execution of the entire signal processing DAHG constitutes Step 3 in the iAssist workflow, depicted in Figure 3.2. All intermediate and terminal results are sent to the datastore for storage, and any of these can be sent to the GUI plugins for display or the Output plugins for external use. It is thus clear that iAssist signal processing is a great deal more complex than is apparent in the top level workflow. The iAssist signal processing DAHG must be dynamic; that is, the user must be able to add, remove, or rearrange plugin instances to meet his re-quirements. iAssist must construct and modify the signal processing DAHG in response to configuration changes. Considering all aspects of flexibility described above, iAssist must have the means to store user configurations, and to load/apply them at runtime. iAssist addresses this using extensible markup language (XML) deployment descriptors. One deployment descriptor is created for each functional area: input, signal processing, datastore, GUI, and output. When the user con-figures plugins, iAssist stores the configurations in the descriptors. The descriptors specify the following: • Which plugins are being used. The user may have more plugins installed than he wishes to use on a given surgical case. The descriptors allow him to load and execute only those that are required. • Which plugin instances to create. As plugins may have multiple instances, the descriptors specify these. • The tuning parameters to apply to each instance. Each plugin instance may require custom tuning, and the descriptors specify the tuning parameters and their values. • The data input/output mappings. Data must flow from between plugins for processing, and the mappings specify interconnections to control the data. flow. iAssist employs the XML data format both because it is an industry standard and because it supports hierarchical data representations. An ex-ample of iAssist's signal processing XML deployment descriptor is provided in Figure 3.11. 43 | | 8 en-CD »-< C o CD £3 13 Figure 3.11: Example of an iAssist signal processing ("algorithm") plugin XML deployment descriptor. This descriptor will configure iAssist with two instances of a single plugin. Note that many descriptor details have been omitted for simplicity. Chapter 3. iAssist: An Intelligent Patient Monitor All elements in the deployment descriptors are keyed by name; that is, each element must have its own unique name among elements of the same type. iAssist uses the element names to distinguish them. Keyed elements are listed below, and examples can be found in Figure 3.11: • Plugins. Each plugin must have a name that is unique among all plugins of its type (input, signal processing, etc.). • Instances. Each instance must have a name that is unique among all instances of the same plugin. • Inputs. Each input must have a name that is unique among all inputs of the same plugin. • Outputs. Each output must have a name that is unique among all outputs of the same plugin. • Configuration Groups. Each group must have a name that is unique among all groups of the same plugin. • Configuration Properties. Each property must have a name that is unique among all properties of the same group. Note that the plugin instances and data inputs and outputs shown in Figure 3.11 contain attributes called codes and reference identifiers (e.g. . channeLcode, metric_code, vmd_ref_id). These attributes are used for inter-operability, and will be explained in Section 3.4.4. When iAssist is launched, it loads all deployment descriptors and con-figures itself accordingly. This operation involves the following steps: 1. Create the necessary plugin instances. 2. Apply custom tuning parameters to each instance. 3. Analyze the specified data inputs and outputs for each instance, ensur-ing they constitute a coherent and properly organized signal processing DAHG and top-level workflow. 4. Connect the instances together to form the DAHG and top-level work-flow. 45 Chapter 3. iAssist: An Intelligent Patient Monitor 3.4.3 Scalability Recall that iAssist may be configured to have any number of plugins and instances thereof. In addition, individual plugins themselves may have ar-bitrary complexity. A signal processing plugin that calculates statistical properties of a physiological trend (e.g. mean, variance, etc.) would have very low computational complexity. In contrast, our statistical change point detection algorithm [30] has relatively high computational complexity. Fu-ture multivariate analysis algorithms may be even more complex. iAssist's extensibility and flexibility allow it to have configurations with very high overall computational complexity. When iAssist is running in real time, it must process incoming data sets quickly. It is therefore important for iAssist's architecture to be scalable, so that it can handle high computational complexity, even when faced with a rapid rate of real-time data acquisition. While it is impossible to guarantee that iAssist can handle any processing requirement under any real-time data load, the software can be architected to help it scale well and to take advantage of new computing hardware in the future. For decades, software has been written sequentially for computer pro-cessors, and has scaled well over time as new processor technologies are released. That is', the software takes immediate and significant advantage of new processors, without requiring any system redesign. Software that was written in the late 1990s for Intel's Pentium ® processor, for exam-ple, benefited from an immediate and striking performance increase'when the Pentium @ II was released. New processors consistently increased their clock speeds to process instructions at a faster rate, thus allowing existing programs to execute faster. Recently, however, chipmakers began to encounter limits on processor clock speed that they could not surpass. Driving processors' transistors at faster rates required ever more electrical power, and caused the processors to emit ever more heat. A barrier was reached at approximately 4 GHz, at which point it became impractical to increase the clock speed further. In order to continue increasing processor performance, chipmakers were forced to embrace a paradigm shift in processor architecture [47], [48]. Recent processor architectures address the heat and power issues by adding more parallel processing cores, while reducing the clock speed. At present, processors containing two to four cores are available in standard PCs and notebooks, and the number of cores per processor is expected to increase dramatically with time. Indeed, Intel recently demonstrated a prototype of an 80-core processor, capable of executing instructions remarkably faster 46 Chapter 3. iAssist: An Intelligent Patient Monitor than any other processor of its size [49]. While still only a research prototype, Intel plans to market a similar processor within five years. Though these new processor architectures promise to deliver unprece-dented performance, there is a drawback. Each software thread can only run on a single processing core; software must therefore be multithreaded in order to utilize multiple cores simultaneously. Single-threaded software programs, including most desktop applications available today, cannot take advantage of multicore processors. Though these applications typically do include multiple threads, they perform most of their tasks in a single thread. When they are executed on a multicore processor, only one core actively performs computations while the others remain idle. Thus, unlike histor-ical advances in processor technology, multicore processors do not provide significant performance increases to existing software applications. Software applications must undergo a paradigm shift towards highly con-current, multithreaded architectures in order to fully benefit from current and future processor technologies [50]. Threads running on two or more processing cores must be independent of each other in order to execute in parallel. An application's logic must be divided into independent tasks for parallel processing, and such independence is not always obvious or even possible. Architecting highly concurrent software is therefore very challeng-ing. Some software programs are more parallelizable than others, by the nature of tasks they perform. User interfaces are particularly difficult to parallelize, and this is still an open area of research in computer science [51]. Fortunately, iAssist's extensible plugin architecture is well suited to highly concurrent execution. There is a great deal of independence be-tween its five functional areas that can be exploited for parallel processing. Consider the iAssist workflow illustrated earlier in Figure 3.2. Though this workflow was presented sequentially, it can be reorganized in a parallel struc-ture, exploiting functional area independence. For example, incoming data can be displayed, stored, outputted, and processed simultaneously, as these actions are independent. Figure 3.12 illustrates this parallelized workflow. There can be further exploitable independence within the signal pro-cessing DAHG, depending on iAssist's configuration. Figure 3.13 presents a simple signal processing DAG, highlighting dependence and independence. Note that we present the signal processing relationships as a DAG (instead of a full DAHG), for visual simplicity. Any two signal processing plugin instances can execute in parallel, provided they are independent. When independence in the signal processing DAHG is exploited for par-allelism, the iAssist workflow in Figure 3.12 is parallelized even further. 47 Chapter 3. iAssist: An Intelligent Patient Monitor Figure 3.12: The parallelized iAssist workflow consists of the three steps shown here. Each vertical grouping represents a step, and all blocks con-tained within can execute in parallel. Each block provides both the name of the action and the functional area that performs it. 48 Chapter 3. iAssist: An Intelligent Patient Monitor Figure 3.13: A simple MIMO, multi-level DAG, with highlighted depen-dence relationships. The highlighted regions represent dependence among vertices, and should not be confused with the highlighted regions in Figure 3.10. Here, vertex F is dependent on D, which is itself dependent on A. As a result, vertices F, D, and A are all interdependent. Vertices I, G, D, and A are likewise interdependent, as are vertices H, E, C, and B. In highlighting these dependencies, independence becomes apparent. Any two vertices are independent, provided they are not members of the same highlighted group-ing. For example, vertex F is only dependent on vertices D and A, and is thus independent of all others. ^ Chapter 3. iAssist: An Intelligent Patient Monitor Results are made available incrementally, as vertices of the DAHG are pro-cessed. Some actions of Step 3 can thus be performed before Step 2 is com-pletely finished, essentially blurring the distinction between the two steps. For example, the results of vertex A in Figure 3.13 can be displayed, out-putted, and stored as soon as they are available, and before the entire DAHG is processed. Java provides an excellent parallelism mechanism in its Concurrency package, called BlockingQueue. Designed primarily for concurrent producer-consumer problems, BlockingQueue is threadsafe. BlockingQueue is a first-in, first-out (FIFO) queue that forces consumer threads to wait for the queue to become non-empty when retrieving elements. Waiting threads are suspended (blocked) until data are available for retrieval, allowing other threads to execute in the interim [51]. iAssist achieves most of its parallelism by employing BlockingQueue for communication between plugin instances. Each plugin instance executes as a dedicated thread, and can be considered a producer, consumer, or both, depending on its functional area. A BlockingQueue connects plugin instance pairs. The producer instance puts data onto the queue; the consumer in-stance blocks until data are available (producer finishes), and then retrieves them. In the signal processing DAHG, BlockingQueues serve as the edges. By running each plugin instance in a dedicated thread, and connecting instances with BlockingQueues, iAssist exploits independence automatically. iAssist need not determine which plugin instances and functional areas are independent (and thus parallelizable) at runtime, nor must it arbitrate or-dered execution. iAssist must simply start all plugin instance threads, and they will automatically execute when possible. As a result, the data them-selves guide and arbitrate execution among dependent and independent plu-gin instances. Considering that iAssist's flexibility permits an arbitrary number of plu-gins and plugin instances, there is also the potential for arbitrary indepen-dence. Taken together, independence among the various functional areas and signal processing tasks will allow iAssist to scale very well from today's single or dual core processors to massively multicore processors in the fu-ture. iAssist will scale automatically, as independence and parallelism are arbitrated solely by the flow of data. Adding concurrent tasks to iAssist will not magically increase perfor-mance on all systems, however. A processing core can only execute a single task at any given time. When the number of concurrent tasks exceeds the number of available processing cores, tasks must wait until a core is available to execute them. On a dual-core system, only two tasks can be executed con-50 Chapter 3. iAssist: An Intelligent Patient Monitor currently, and all others must wait their turn—even if they are independent. As a result, although iAssist has been architected for high parallelism, the vast majority of its capabilities will be unrealized on present-day processors. Just as older sequential software used to gain immediate advantage from new processors with higher clock speeds, iAssist's highly concurrent archi-tecture will allow it to gain immediate advantage from new processors with more processing cores. 3.4.4 Interoperability Modern hospitals employ technologies for many purposes, and from many different vendors. A hospital's operating rooms may have physiological mon-itors from Datex/Ohmeda, while its intensive care unit may have monitors from Draeger. At the same time, the hospital may keep its records in an anesthesia information management system (AIMS) from Siemens. These devices store their data in custom, vendor-specific formats that are incom-patible with those of competing or dissimilar devices.-Traditionally, these incompatibilities have been acceptable, as clinicians have not often needed to connect devices together. Data recorded in the operating rooms or intensive care unit were not used outside their original environment. AIMS data were not used outside the AIMS system. Recently, a number of academics in the domain of health informatics have called for greater interoperability between medical devices [52], [54], [53]. Interoperability will allow for better record keeping and smarter pa-tient monitoring, with the goal of improving patient safety and reducing operational costs. In the future, data from various devices will be combined together and made accessible over internal networks, allowing for ubiquitous data .access. Medical device interoperability will therefore be essential in the future. Recently, efforts have been made to standardize medical data represen-tation, storage, and sharing, resulting in several standard specifications: • Health Level 7 (HL7). An ANSI standard for clinical and admin-istrative data messaging [55]. • I S O / I E E E 11073. A joint ISO and IEEE standard for modeling point-of-care device data [56], [57], [58]. • Systemized Nomenclature of Medicine Clinical Terms (SNO-M E D C T ) . An international standard nomenclature for medical data representation. 51 Chapter 3. iAssist: An Intelligent Patient Monitor While there is some overlap amongst these three standards, they are largely complementary. SNOMED is designed as a very high-level ontology for collecting and relating medical terms, particularly for handling variations in expression. For example, the terms "myocardial infarction" and "heart attack" are both used to describe the same event, and the SNOMED ontol-ogy collects them together as such. In contrast, ISO/IEEE 11073 models the representation of different physiological signals, trends, and alerts; in particular, the manner in which the raw data are represented and related. For example, the ECG is assigned a unique identifier, and is stored as an array of numerical data. Finally, HL7 is designed as a messaging standard for transmission of medical data. For example, it specifies the sequence in which terms must be transmitted. The three standards can be used together in a complete medical system. SNOMED is intended to be the highest level, with ISO/IEEE 11073 beneath, and finally HL7 at the lowest level. Because iAssist is intended for use as a point-of-care medical device, it employs the ISO/IEEE 11073 standard for representation of clinical data, and "speaks" this format natively. It may be upgraded to employ SNOMED and HL7 in the future, but these are unnecessary for our purposes at present. The ISO/IEEE 11073 medical package model is illustrated in Figure 3.14. iAssist utilizes a hierarchical subset of the model: • V M O . This is a Virtual Medical Object. It is the abstract base class for all medical-related objects in the model. It provides consistent naming and identification across the model. • V M D . This object is a Virtual Medical Device. It is an abstrac-tion for a medical device, which can be hardware or software. iAssist treats input plugin instances as VMDs, as they acquire data from de-vices such as physiological monitors. For example, a Datex/Ohmedia AS/3® monitor is a VMD. iAssist itself is also modeled as a VMD. Each VMD contains a collection of Channels. • Channel. This object is a logical grouping of related physiological measurements. Examples of Channels in a physiological monitor in-clude the ECG Analyzer, Blood Pressure Analyzer, Gas Analyzer, and more. Each Channel contains a collection of Metrics. • Metric. This object represents a physiological measurement. For ex-ample, the ECG analyzer module of a physiological monitor (a Chan-nel) records the patient's ECG, and produces both the raw ECG wave-form and a derived heart rate value. The ECG and heart rate are 52 Chapter 3. iAssist: An Intelligent Patient Monitor Metrics inside the ECG Analyzer channel. A Metric object can con-tain a single Numeric value (in the case of a trend like heart rate), or a Sample Array (in the case of a waveform like the ECG). iAssist does not use the Metric object directly; rather, it uses the Complex Metric object. • Complex Metric. This object is closely related to the Metric ob-ject. Indeed, Complex Metric is a Metric that can contain a grouping of strongly related Metrics. iAssist uses Complex Metric (rather than Metric) to store multiple Numeric or Sample Array objects. Specifi-cally, Complex Metric allows iAssist to store time-varying trend and waveform values together. Hereafter, Complex Metrics will be.referred to simply as Metrics, for simplicity. • Numeric. This object stores a single numeric value and associated information. For example, it may hold a heart rate value and units, such as 75 beats per minute. iAssist uses a collection of Numerics, in-side a Complex Metric, to track a patient's time-varying physiological trends. • Sample Array. This object stores a collection of numeric values and associated information. In particular, it is meant to store waveform samples recorded over a given period of time. For example, it may hold ECG samples recorded over a five-second period. iAssist uses a collection of Sample Arrays, inside a Complex Metric, to track a patient's time-varying physiological waveforms. The ISO/IEEE 11073 standard also includes a mechanism for uniquely identifying VMDs, Channels, and Metrics. Each VMD must be assigned a string "reference identifier," or reference ID, unique among all VMDs. Each Channel must be assigned an integer "code" and string reference ID, unique within its parent VMD. Likewise, each Metric must be assigned a code and reference ID, unique within its parent Channel. Objects are identified by the codes and reference IDs of their hierarchy. For example, the standard allows us to identify a Metric having code 1000 and reference ID "Some_Metric," within a Channel having code 1500 and reference ID "Some-Channel,", within a VMD having reference ID "Some_VMD." iAssist models itself as a VMD, with a unique reference ID to distinguish it from any VMD used as an input data source. It also models every sig-nal processing plugin instance as a Channel under the iAssist VMD, with 53 Chapter 3. iAssist: An Intelligent Patient Monitor Figure 3.14: The ISO/IEEE 11073 medical package model [57]. This UML diagram illustrates the relationships among the model's elements. Those elements used in iAssist are highlighted in grey. 54 Chapter 3. iAssist: An Intelligent Patient Monitor user-assigned code and reference ID auto-generated from the plugin and in-stance names. Since iAssist enforces unique plugin and instance names, the Channel reference IDs are guaranteed unique. Finally, iAssist models every signal processing output as a Metric under the plugin instance Channel, with user-assigned code and reference ID auto-generated from the plugin, instance, and output names. As before, these names are all unique, so the reference ID is guaranteed to be unique. iAssist's Channel and Metric codes are stored in the deployment descrip-tors, which were discussed in Section 3.4.2. Each input of a plugin instance is identified in the deployment descriptors by VMD reference ID, Channel code, and Metric code. These unique identifiers allow iAssist to properly connect the various plugin instances together. Figure 3.11 provides exam-ples of this identification. iAssist requires that data acquisition plugins input data in the ISO/IEEE 11073 format, translating from the source format when necessary. Like-wise, it is the output plugins' responsibility to translate from iAssist's native ISO/IEEE 11073 format into the format of the recipient system. The ad-vantage of this translation scheme is abstraction; the core iAssist framework must only be concerned with the ISO/IEEE 11073 format, and not with the vendor-specific formats of any other connected device. Since the data acquisition and output plugins perform the translation, iAssist is particularly well-suited to serve as an adapter to connect incompat-ible devices. Custom plugins can be written to communicate with present-day medical devices, translating between vendor-specific and standardized formats. Figure 3.15 provides a visual example of how iAssist may be configured as an adapter. Because iAssist is designed around an international standard data model, it can help achieve the goal of medical device interoperability. In the near term, it can serve as an adapter to bridge communications between dissimilar devices. In the long term, it will be able to communicate natively with future standardized medical devices. 3.5 Functional Area Design The iAssist framework is incapable of performing the intended workflow on its own. Plugins must be connected to perform the individual workflow steps of each functional area—data acquisition, signal processing, GUI, data storage, and output. This section is devoted to the design of iAssist's functional area plugins. 55 Figure 3.15: iAssist configured to serve as a device adapter. Custom-written input plugins translate from vendor-specific formats and protocols into ISO/IEEE 11073 for manipulation inside iAssist. The standardized data are sent in various combinations to custom-written output plugins, which translate back to vendor-specific formats and protocols. Chapter 3. iAssist: An Intelligent Patient Monitor The discussion focuses on the plugins that have been developed, as well as those that are currently in development or planned for the near future. This section also discusses testing methodologies used in each functional area for plugin validation. In reading this section, note the simplicity of the plugin designs. This simplicity is testimony to the level of abstraction achieved in architecting the high-level iAssist framework. Plugins need not be concerned with many details of iAssist itself, and can focus almost entirely on their own specific purposes. 3.5.1 Data Acquisition Our research group requires connectivity to Datex/Ohmeda AS/3® physi-ological monitors, for data acquisition both online (in real time) and offline (pre-recorded data). This section describes three iAssist input plugins; two have already been developed and are fully operational, and the third is not yet complete. File Monitor We have created an input plugin for acquiring pre-recorded data for offline processing, for tuning and validating our algorithms in the laboratory. These pre-recorded data originate from the AS/3® monitors used at the British Columbia's Children's Hospital. Datex/Ohmeda has provided us with a tool, called S/5 Collect®, that acquires data from AS/3® monitors. This LabVIEW®-based research tool connects to the AS/3® monitors via a standard RS-232 serial connector, and decodes the AS/3® data frames to extract waveforms, trends, and alerts. S/5 Collect® can then save these data to disk in S/5-formatted ASCII text files. File Monitor employs a standard Java library to facilitate the reading of data from text files. Once data are read, File Monitor parses the S/5 formatted data and translates it into the standard ISO/IEEE 11073 format. S/5 uses string-based identifiers for each trend and waveform. For example, heart rate from the ECG is identified as "HR(ECG)," while the correspond-ing ISO/IEEE 11073 Complex Metric is identified in the domain model spec-ification by code "16770" and reference ID "MDC_ECGJ3EART_RATE." File Monitor performs the translation using a lookup table that maps the S/5 identifiers into the ISO/IEEE 11073 identifiers. Once translated, the data are put onto output queues for the iAssist 57 Chapter 3. iAssist: An Intelligent Patient Monitor framework to receive, and File Monitor repeats the process, reading the next set of data from the text files. S/5 Monitor We have created an input plugin for acquiring online data for real-time processing, for validating our algorithms live in an intra-operative setting. These data originate from AS/3® monitors. The S/5 Collect® tool, described earlier, also includes the ability to transmit AS/3® data over the standard TCP/IP protocol, using S/5 data frames. In addition, S/5 Collect® can replay a recorded case, simulating a real-time environment. S/5 Monitor uses S/5 Collect® as a broker between iAssist and the AS/3® monitoring hardware. S/5 Monitor employs a standard Java library to facilitate TCP/IP com-munications. Once data are received, S/5 Monitor parses each S/5 data frame and translates it into the standard ISO/IEEE 11073 format. As with File Monitor, S/5 Monitor performs the translation using a lookup table. Once translated, the data are put onto output queues for the iAssist framework to receive, and S/5 Monitor waits until new data are available from S/5 Collect®. AS/3 Monitor File Monitor's and S/5 Monitor's reliance on S/5 Collect® as a broker between the AS/3® monitors and iAssist is undesirable. Our research group is using S/5 Collect® under a research license from Datex/Ohmeda, which is restrictive and would prohibit a more widespread deployment of iAssist in the future. Furthermore, using S/5 Collect® and iAssist in tandem is cumbersome. Finally, communication between S/5 Collect® and iAssist requires the overhead of a TCP/IP link. It would therefore be beneficial for us to develop an input plugin that eliminates the S/5 Collect® broker, instead communicating with the AS/3® monitoring hardware directly. We have developed a Java-based application—called DatexRelay—that reproduces some of S/5 Collect's® features. It communicates directly with AS/3® monitors over an RS-232 connection and translates the AS/3® data frames into S/5 Collect® frames. It can then save the data to disk in S/5-formatted ASCII text files, and can transmit the S/5-formatted data over TCP/IP. At present, iAssist can use DatexRelay in lieu of S/5 Collect® for both online and offline processing. Since DatexRelay employs the S/5 Collect® 58 Chapter 3. iAssist: An Intelligent Patient Monitor data format, iAssist can communicate with it using the pre-existing File Monitor and S/5 Monitor input plugins. DatexRelay is still not ideal, how-ever, as it doesn't relieve the cumbersome process of configuring two separate applications to work in tandem over a TCP/IP link. In the near future, we will transform DatexRelay into an iAssist input plugin, and rechristen it AS/3 Monitor. Testing File Monitor and S/5 Monitor, both described above, require testing to ensure they function properly as iAssist input plugins. Testing ensures that the plugins can acquire data, translate them to the ISO/IEEE 11073 format, and pass them to iAssist. We performed testing as follows: 1. Begin with a known data set from a surgical case, recorded using S/5 Collect®. 2. Configure iAssist to use the desired input plugin, as well as the ASCII File data storage plugin. 3. Supply the data set to the input plugin. File Monitor loads the data from ASCII files directly. S/5 Monitor requires that S/5 Collect® transmit the data over TCP/IP in pseudo real-time. 4. Run iAssist against the data set, from beginning to end. iAssist will generate an ASCII file containing the data it received. 5. Compare the iAssist-generated ASCII file with the original input data set, ensuring each data point is the same in both data sets. Both plugins passed testing against multiple data sets. In testing the plugins, we also implicitly tested iAssist's Data Acquisition and Data Stor-age functional areas, as well as the ASCII File datastore plugin. 3.5.2 Physiological Signal Processing Our research group requires three signal processing algorithms. One al-gorithm is the electrocautery noise detector described in Chapter 2, and two are change point detectors for physiological trends [30], [31]. All three plugins have been developed and are fully operational. 59 Chapter 3. iAssist: An Intelligent Patient Monitor Electrocautery Noise Detector This plugin is designed to detect the presence of electrocautery noise in the ECG, and provide an appropriate heart rate trend value to the heart rate change point detector. The electrocautery noise detector helps ensure that the change point detector processes the patient's true heart rate value, and not one artificially inflated by the presence of electrocautery noise in the ECG. The electrocautery noise detector receives three values as inputs. The inputs must be provided in the following order: 1. An ECG waveform. 2. A heart rate trend value, derived from the ECG. 3. A heart rate trend value, derived from an alternate source (e.g. arterial blood pressure or pulse oximeter). It produces a single value as output: a heart rate trend value. The plugin processes the received ECG waveform, using the wavelet-based electrocautery noise detection algorithm described in depth in Chapter 2. If the ECG is deemed clean, the plugin outputs the ECG-derived heart rate value. Otherwise, the ECG is deemed noisy, and the plugin outputs the alternate heart rate value. In essence, the electrocautery noise detector acts as a switch, allowing one of the two received heart rate values to pass through from input to output, depending on the cleanliness of the received ECG. Figure 3.16 illustrates the switch. Statistical Change Point Detector This plugin is designed to detect significant changes in four physiological trends: heart rate (HR), end tidal carbon dioxide (EtC02), exhaled minute ventilation (MVexp), and respiratory rate (RR). The change point detector receives a trend value as input, and produces a change point indicator as output. The plugin models the trends using a dynamic linear growth model, whose noise covariances are estimated by an adaptive Kalman Filter based on a recursive expectation-maximization method. It detects change points by adaptive local cumulative sum (CUSUM) testing. Our research group developed the underlying algorithm [30]. 60 Chapter 3. iAssist: An Intelligent Patient Monitor HR(EGG) ECG HR(Sp02) HR Figure 3.16: The electrocautery noise detector acts as a switch. If the ECG is clean, the ECG-derived heart rate is passed through to the output. Otherwise, the Sp02-derived heart rate is passed. When processing has finished, the plugin produces a change point value as output. A value of +1 indicates an increase, -1 indicates a decrease, and 0 indicates no change. Cusum Masking Change Point Detector This plugin is designed to detect significant changes in two physiological trends: mean noninvasive blood pressure (NIBPmean) and oxygen satura-tion (Sp02). The change point detector receives a trend value as input, and produces a change point indicator as output. The plugin detects change points using CUSUM testing on a filtered residual from an exponential moving average filter. It applies a "mask" (custom-created based on clinician knowledge) to the time-varying trend. Our research group developed the underlying algorithm [31]. When processing has finished, the plugin produces a change point value as output. A value of +1 indicates an increase, -1 indicates a decrease, and 0 indicates no change. 61 Chapter 3. iAssist: An Intelligent Patient Monitor Testing The electrocautery noise detector, statistical change point detector, and cusum masking change point detector, all described above, require testing to ensure they function properly as iAssist signal processing plugins. Testing ensures that the plugins can receive data from iAssist, process them properly, and pass the results to iAssist. All plugins' underlying algorithms were originally developed in Matlab®, and this Matlab® code serves as an excellent "known good" baseline for testing. We performed testing as follows: 1. Begin with a known data set from a surgical case, recorded using S/5 Collect®. 2. Run the desired algorithm's Matlab® code against the data set, to obtain a baseline result set. 3. Configure iAssist to use the desired input plugin, the desired signal processing plugin, as well as the ASCII File data storage plugin. 4. Supply the data set to the input plugin. 5. Run iAssist against the data set, from beginning to end. iAssist will acquire the data, process it using the signal processing plugin, and generate an ASCII file containing the plugin result set. 6. Compare the iAssist-generated ASCII file result set with the baseline result set, ensuring each data point is the same in both. All three plugins passed testing against multiple data sets. In testing the plugins, we also implicitly tested iAssist's Data Acquisition, Physiological Signal Processing, and Data Storage functional areas, as well as the ASCII File datastore plugin. 3.5.3 Data Storage Our research group requires one data storage plugin, to store data and results to disk. ASCII File This plugin is designed to write all iAssist data to an ASCII file on the local computer's file system. It stores all data acquired by the input plugins, as 62 Chapter 3. iAssist: An Intelligent Patient Monitor well as all results of the signal processing plugins that process these data internally. Data are stored in ISO/IEEE 11073 format. ASCII File receives an arbitrary number of values as inputs. These values are formatted as ISO/IEEE 11073 Complex Metrics, and they can be received at any time and in any order. The plugin does not produce any outputs to iAssist. 'The plugin creates one file for each VMD. Recall that each input plugin instance is represented as a VMD, as is iAssist itself. In a basic configuration with a single input plugin, ASCII File will produce two files: one containing data acquired from the input source, and one containing data generated by iAssist's signal processing plugins. ASCII File models the data as a table within each file. Table columns are keyed to Channel and Complex Metric identifiers, by their integer code and string reference ID. Each table cell contains a data point. When the plugin receives new data, it looks up the appropriate ASCII file based on the VMD reference ID. It then looks up the appropriate table column within the file, based on the Channel and Complex Metric codes and reference IDs. Finally, it appends the new data point to the column. Testing The ASCII File plugin does not require custom testing. The plugin and the Data Storage functional area were validated implicitly when the Data Acquisition and Physiological Signal Processing plugins were tested. The plugin passed this prior testing against multiple data sets.-3.5.4 Graphical User Interface Our research group requires one GUI plugin, to display time-varying physi-ological trend data. Trend Graph This plugin is designed to receive time-varying physiological trend data, and plot them together as a graph on-screen. It receives a trend value as input, as well as any number of displayable icons. It does not produce any outputs. The Trend Graph plugin creates a polygon to representing the data, based on its display size on-screen as well as its time window and scale. It then paints this polygon on-screen, using a standard Java GUI library. Once the trend data are plotted, Trend Graph overlays any icons pro-vided as inputs. The icons are graphical representations of signal processing 63 Chapter 3. iAssist: An Intelligent Patient Monitor plugin results. The change point detection plugins, for example, will pro-vide arrows pointing upwards (increasing trend) or downwards (decreasing trend). When overlaid on the graph, they provide visual notification to the user that a significant change is occurring in the patient. Testing Trend Graph requires testing to ensure it functions properly as an iAssist GUI plugin. Testing ensures that the plugin can properly plot trend data, and overlay the required information. We performed testing as follows: 1. Begin with a known data set from a surgical case, recorded using S/5 Collect®. 2. Configure iAssist to use the desired input plugin, a change point de-tector plugin, and the Trend Graph plugin. 3. Supply the data set to the input plugin. 4. Run iAssist against the data set, from beginning to end. iAssist will process the data it receives, and plot both the data and change point detector results on-screen. 5. Compare the plotted trend data with the original input data set, en-suring the graphical representation matches the given data. 6. Compare the plotted icon overlays with the known change point de-tector results, ensuring the proper icons are displayed at the correct points in time. The Trend Graph plugin passed testing against multiple data sets. 3.5.5 Output We have not yet developed any output plugins, however one will be needed soon. Tactile Display 1 We are currently researching and developing a tactile display, which can convey information to clinicians in the operating room using the medium of touch rather than the traditional media of sound and vision. The tactile 64 Chapter 3. iAssist: An Intelligent Patient Monitor display will convey information in vibration patterns applied to the user's body. The aim of the tactile display is to alert the clinician to significant patient events detected in iAssist. To support the tactile display, a plugin will be designed to receive the results of the change point detector plugins, translate them into vibration patterns and sequences, and send these vibrations to the tactile display. 3.6 Event Feedback Since iAssist will be used to conduct clinical studies for our algorithms, we incorporated an event feedback system into the user interface. When a signal processing algorithm detects an event (e.g. a change point), iAssist generates an event record. Users can open records and provide feedback about the event: • Usefulness: Users select a point on a 10cm visual analog scale (VAS) to rate the event's usefulness. Usefulness ranges from "frustrating" (0) to "very useful" (10). • Classification: Users classify the event using a dropdown list of op-tions: "artifact" (A), "clinically insignificant" (CI), "clinically signifi-cant" (CS), "clinically significant with action taken" (CSA), and "clin-ically significant due to intervention" (CSI). • Notes: Users can enter freeform notes to provide more detailed event feedback. iAssist stores all event feedback in text files for subsequent analysis. Screenshots of iAssist's main window and event feedback system are provided in Figures 3.17 and 3.18, respectively. 3.7 Final Configuration & Testing This section describes the final configuration of iAssist that our research group has used for real-world testing. Our high-level configuration require-ments are listed here: • iAssist must receive data using either File Monitor or S/5 Monitor. • iAssist must perform pre-processing of the ECG using the Electro-cautery Noise Detector. 65 Chapter 3. iAssist: An Intelligent Patient Monitor Afl Trends Individual Trends Event Log Figure 3.17: iAssist's main window, showing graphs and change point results for six trends. 66 Chapter 3. iAssist: An Intelligent Patient Monitor Chapter 3. iAssist: An Intelligent Patient Monitor • iAssist must detect significant change points in the HR, EtCC-2, RR, and MVexp trends, using the Statistical Change Point Detector signal processing plugin. • iAssist must detect significant change points in the SpC-2 and NIBP-mean trends, using the Masking Change Point Detector signal pro-cessing plugin. • iAssist must store all input data and signal processing results using the ASCII File datastore plugin. • iAssist must display the received data for all six trends being processed, using the Trend Graph GUI plugin. We created the configuration listed above using iAssist's XML deploy-ment descriptors. In the final configuration, the plugins are interconnected as shown in Figure 3.19. Unfortunately the Electrocautery Noise Detector was not ready in time for our initial testing, and was omitted from the configuration. The true final configuration will be ready for future testing, however. We tested this modified iAssist configuration, to ensure that all func-tional areas of iAssist operate properly together as a whole. Testing involved four stages: 1. Offline testing in the lab. We tested iAssist offline, on pre-recorded surgical cases. iAssist processed numerous surgical cases, and its func-tionality and results were validated visually. 2. Pseudo-online testing in the lab. We tested iAssist in a pseudo-online setup, on pre-recorded surgical cases played back in real-time simulation. iAssist processed the same surgical cases tested previously, and its functionality and results were validated visually. 3. Online testing in the operating room. We tested iAssist in a true online setup, on live surgical cases in the operating room. Dr. Mark Ansermino and his anesthesia residents used iAssist in their day-to-day activities, and validated its functionality and results visually. 4. Clinical study in the operating room. We conducted a clinical study of iAssist, on live surgical cases in the operating room. Thirteen anesthesiologists and anesthesia residents used iAssist in their day-to-day activities, and provided feedback on its functionality and results 68 Chapter 3. iAssist: An Intelligent Patient Monitor Electrocautery Noise Detector > ECG > HR(ECG) > HR(Sp02) Figure 3.19: Plugin interconnections in the final iAssist configuration that our research group presently employs. iAssist processes and plots six phys-iological trends. Note that the ASCII File datastore plugin is omitted, for visual simplicity. 69 Chapter 3. iAssist: An Intelligent Patient Monitor using iAssist's Event Feedback system. This study will be discussed in greater detail in Section 3.8. Bugs were discovered in all but the last stage of testing. Bugs ranged from minor to major in severity, but we successfully resolved all that were discovered. Overall, iAssist performed very well in all stages of testing. 3.8 Clinical Study Following Ethics approval (Appendix B), we conducted an iAssist clini-cal study in real time alongside current physiological monitors in operating rooms at British Columbia's Children's Hospital. Thirteen anesthesiologists evaluated iAssist during a variety of pediatric surgical total of 38 cases of at least one hour duration were evaluated. The clinical studies in-volved two aspects of iAssist; study design and results are presented for each aspect. 3.8.1 Algori thms This study was designed to evaluate two change point detection algorithms [30], [31]. Subjects rated the events produced by the algorithms in real time, using iAssist's event feedback system. Subjects also noted missed events. Table 3.1 presents the type of events and their classifications. The mean number of events per case was 22.8 (a 23.4). The usefulness ratings ranged from 0 to 9.3 (for an EtC02 CSAT event). Subjects reported only six significant events as being missed by the algorithms. 3.8.2 Usabil i ty Subjects completed a demographics data form. The form provided us with subject information, including sex, age, years of anesthesia experience, and computer proficiency. Subjects completed a usability questionnaire after using iAssist for the first time. The questionnaire is a modified version of the standard post-study system usability questionnaire (PSSUQ) [59], com-prising 18 questions. Subjects responded to each question on a VAS, with results ranging from 1 ("strongly agree," good usability) to 7 ("strongly dis-agree," poor usability). Each question also provided a N / A (not applicable) option, and a freeform notes section for detailed feedback. All subject forms are provided in Appendix C. _ 70 Table 3.1: iAssist algorithms study results. A = artifact, CI = clinically insignificant, CS = clinically significant, CSAT = clinically significant with action taken, CSI = clinically significant due to intervention, NR = not rated. Total Events A ( % ) CI (%) CS (%) CSAT (%) CSI (%) N R (%) EtCC-2 212 5.2 24.1 19.8 5.2 27.4 18.4 H R 253 6.3 36.4 24.5 9.5 9.9 13.4 MVexp 145 5.5 20.7 17.9 2.1 34.5 19.3 NIBPmean 124 3.2 28.2 36.3 16.1 9.7 6.5 R R 86 2.3 9.3 9.3 3.5 47.7 27.9 SpC-2 48 37.5 20.8 6.3 2.1 10.4 22.9 Total 868 6.8 26.0 21.4 7.1 22.1 16.6 Chapter 3. iAssist: An Intelligent Patient Monitor Table 3.2: iAssist usability study results, with range from 1 (good usability) to 7 (poor usability). * SYSOBT is a custom element, and not a validated factor. PSSUQ Factor Complete Excl . Outliers O V E R A L L 3.1 2.8 S Y S U S E 3.0 2.7 I N F O Q U A L 2.9 2.6 I N T E R Q U A L 3.4 3.3 SYSOBT* 3.7 3.5 The PSSUQ rates iAssist's usability across four factors: overall (OVER-ALL), system usefulness (SYSUSE), information quality (INFOQUAL), and interface quality (INTERQUAL). Our questionnaire also included a question supplemental to the standard PSSUQ, to assess obtrusiveness in the oper-ating room. We treated the results of this question as a distinct "element" (SYSOBT), as it has not been validated with factor analysis. As per the standard PSSUQ scoring [59], questionnaire results are aver-aged across all subjects and all questions comprising each factor. One question was removed from further analysis as it was not applicable to the use cases in this study. Two subjects were treated as outliers based, on their demogrpahic data. These subjects rated themselves as somewhat "uncomfortable" with computers, and understandably had difficulty using iAssist. Table 3.2 presents the usability results for each PSSUQ factor, plus the custom SYSOBT element. The table includes results both overall and excluding outliers. 3.9 Discussion In successfully incorporating our data acquisition, signal processing, data storage, GUI, and output plugins into the iAssist framework, we have im-plicitly validated the system's architecture and design. Testing in offline and online environments validated the system's functionality and performance. The single-processor computing hardware employed for testing was unable to fully validate iAssist's scalability; however, we will perform further vali-dation in the future using multi-processor hardware. 72 Chapter 3. iAssist: An Intelligent Patient Monitor In real-time clinical study, over 50% of events iAssist detected were rated as clinically significant (CS, CSAT, CSI), and less than 7% were rated as artifacts. In comparison, more than 90% of traditional threshold-baesd alarms are insignificant, 1/3 of which are triggered by artifacts. These re-sults demonstrate that our algorithms can detect changes in patient state with accuracy far superior to that of traditional alarms. There is still much room for improvement, however. The SpC>2 trend, for example, was the source of many artifacts. This is related to the fact that the algorithm failed to detect small changes over a long period of time, and these may be clinically significant. We hope to improve the tuning of this algorithm. Furthermore, subjects described some artifacts in the HR trend as being caused by electrocautery noise. Future inclusion of the electrocautery noise detector plugin, using the wavelet analysis algorithm described in Chapter 2, should eliminate these. Finally, while the CSI events are significant, their detection is irrelevant to clinicians. We may suppress these in the future by capturing information about clinical interventions a priori. The usability study results are also encouraging. Most subjects enjoyed using iAssist, and felt the software was intuitive and easy to use. Freeform subject feedback on the PSSUQ indicates that the touch screen interface was the most significant hindrance to usability. Improper touch screen calibra-tion led to difficulty interacting with iAssist. We may address this problem in the future through improved calibration and larger user interface features. 73 Chapter 4 Conclusion & Future Work This concluding chapter reviews the work done under this M.A.Sc program, emphasizing the significance and contributions of this thesis. It also proposes new directions to further this work. 4.1 Significance &; Contributions Medical errors in anesthesia can result in adverse patient outcomes. Since the 1960s, engineers have been developing new physiological sensors and monitors to help reduce errors and improve patient safety. Unfortunately, even state-of-the-art monitors employ unintelligent, single-parameter, threshold-based alarms, which produce far too many false alarms to be useful. Fur-thermore, the increasing number of physiological sensors produce far too much data for clinicians to monitor themselves. False alarms and a deluge of data often cause clinicians to miss episodes of significant change in patient state, and thus medical errors persist. Technology still has the potential to improve patient safety, via clinical monitoring expert systems that integrate multiple parameters, presenting patient diagnoses as unified intelligent alarms. This thesis describes two contributions to our development of a clinical monitoring expert system, summarized below. Electrocautery Noise Detection The development of the wavelet-based electrocautery noise detector was spurred by the need for high quality heart rate data. Heart rate is used for change point detection and eventually for rule-based reasoning. A noisy ECG can result in artificially inflated heart rate values, leading our expert system to make incorrect diagnoses. It was important to develop a purely algorithmic solution. While existing hardware solutions are available, their design makes them overly sensitive and cumbersome to use. Our novel wavelet-based algorithm does not have such drawbacks. Using the time-frequency localization property of wavelets, 74 Chapter 4. Conclusion