Applied Science, Faculty of
Electrical and Computer Engineering, Department of
DSpace
UBCV
Yu, Zhou
Master of Applied Science - MASc
University of British Columbia
The brain-computer interface (BCI) has emerged as a potential and radically new mode of communication for users with neuromuscular impairments since it provides a communication channel based on human brain activity as opposed to peripheral nerves and muscles. It is a practical problem to detect user commands among spontaneous Electroencephalograph signals. Low-Frequency Asynchronous Switch Design (LF-ASD) is one of the leading means of addressing this problem. Although the performance of the LF-ASD is encouraging, it is not yet sufficient for real world application. The main goal of this research study is to improve the design of the LF-ASD BCI technique, and then evaluate the performance of this modified design. In this work, the Energy Feature related to Voluntary Movement Related Potential (VMRP) was determined. An energy normalization transform was proposed corresponding to this Energy Feature. A simulation model was set up and EEG data from five able-bodied subjects was applied for offline evaluation. The impact of this normalization transform was evaluated in two studies: the impact of this transform on the low frequency EEG and the impact on the performances of the LF-ASD. By analyzing the experimental results, the characteristics of this normalization filter were determined. In Study 1, it was determined that the proposed normalization transform has two major benefits to the low frequency EEG components (0-4 Hz), which is used by the LF-ASD. First, it can decrease the input scale variance. Consequently, it resulted in more stable feature sets and then a higher successful classification rate. Second, it can increase the separation between the VMRP and idle data. Another side benefit is that the proposed normalization transform can also adjust the input scale automatically. In Study 2, the performances of the LF-ASD with and without this normalization transform were compared. For four out of the five subjects, this transform increased the successful classification rate (True Positive rate with the corresponding False Positive rate at 1%) by 7.7%, 8.3%, 8.5% and 18.9% respectively. By applying an alternative energy normalization transform, the performance increased by 0.4% for the fifth subject. In the future with the parameters of the LF-ASD, especially the codebook in the Feature Classifier, derived from the normalized data, the performance could be further improved. The two studies also showed that, although this transform is non-linear in the broadband (0-64 Hz), it does not distort the features used by the LF-ASD. Therefore, it would not hamper the performance of the LF-ASD. The work concluded with the introduction of potential features related to VMRP in the phase spectrum.
4589336 bytes
application/pdf
The Performance Enhancement of the LF-ASD Brain-Computer Interface: An Energy Normalization Transform by Zhou Yu B.A.Sc , Harbin Engineering University, 1995 A THESIS SUBMITTED IN P A R T I A L F U L F I L L M E N T OF THE REQUIREMENTS FOR THE D E G R E E OF Master of Applied Science in THE F A C U L T Y OF G R A D U A T E STUDIES (Department of Electrical and Computer Engineering) We accept this thesis as conforming to the required standard The University of British Columbia Vancouver, B.C., V6T 1Z4 September 2002 © Zhou Yu, 2002 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of El^-fr,'/J H U^^^ ^ ^ ^ \ The University of British Columbia Vancouver, Canada Date DE-6 (2/88) A B S T R A C T The brain-computer interface (BCI) has emerged as a potential and radically new mode of communication for users with neuromuscular impairments since it provides a communication channel based on human brain activity as opposed to peripheral nerves and muscles. It is a practical problem to detect user commands among spontaneous Electroencephalograph signals. Low-Frequency Asynchronous Switch Design (LF-ASD) is one of the leading means of addressing this problem. Although the performance of the LF-ASD is encouraging, it is not yet sufficient for real world application. The main goal of this research study is to improve the design of the LF-ASD BCI technique, and then evaluate the performance of this modified design. In this work, the Energy Feature related to Voluntary Movement Related Potential (VMRP) was determined. An energy normalization transform was proposed corresponding to this Energy Feature. A simulation model was set up and E E G data from five able-bodied subjects was applied for offline evaluation. The impact of this normalization transform was evaluated in two studies: the impact of this transform on the low frequency E E G and the impact on the performances of the LF-ASD. By analyzing the experimental results, the characteristics of this normalization filter were determined. In Study 1, it was determined that the proposed normalization transform has two major benefits to the low frequency E E G components (0-4 Hz), which is used by the LF-ASD. First, it can decrease the input scale variance. Consequently, it resulted in more stable feature sets and then a higher successful classification rate. Second, it can increase the separation between the V M R P and idle data. Another side benefit is that the proposed normalization transform can also adjust the input scale automatically. In Study 2, the performances of the LF-ASD with and without this normalization transform were compared. For four out of the five subjects, this transform increased the successful classification rate (True Positive rate with the corresponding False Positive rate at 1%) by 7.7%, 8.3%, 8.5% and 18.9% respectively. By applying an alternative energy normalization transform, the performance increased by 0.4% for the fifth subject. In the future with the parameters of the LF-ASD, especially the codebook in the Feature Classifier, derived from the normalized data, the performance could be further improved. The two studies also showed that, although this transform is non-linear in the broadband (0-64 Hz), it does not distort the features used by the L F - A S D . Therefore, it would not hamper the performance of the LF-ASD. The work concluded with the introduction of potential features related to V M R P in the phase spectrum. ii C O N T E N T S Abstract ii Contents iii List of Tables v List of Figures vi Acknowledgements viii Chapter 1. Introduction 1 1.1 R E S E A R C H G O A L S 2 1.2 O V E R V I E W OF T H E THESIS 2 Chapter 2. Background 4 2.1 B R A I N - C O M P U T E R INTERFACE (BCI) R E S E A R C H 4 2.1.1 BCI system design 5 2.1.2 Existing system designs 6 2.2 CHARACTERISTICS OF M O V E M E N T - R E L A T E D E E G 11 2.3 N O R M A L I Z A T I O N THEORY 12 2.3.1 Normalization based on radar Maximum Likelihood Estimation 13 2.3.2 Normalization of Electrocardiogram signals 14 Chapter 3. Design of the modified LF-ASD 16 3.1 T H E PROPOSED MODIFIED L F - A S D 16 3.1.1 Design 16 3.1.2 Optimal parameter determination 16 3.1.3 Design rationale 17 3.1.4 Concerns related to the modified LF-ASD design 18 3.2 STUDY TO D E T E R M I N E E E G POWER CHARACTERISTICS R E L A T E D TO V M R P DETECTION 18 Chapter 4. Evaluation of the modified LF-ASD 24 4.1 OBJECTIVES OF T H E STUDIES 24 4.2 TEST D A T A CHARACTERISTICS 24 4.2.1 Power characteristics of the bipolar EEG 25 4.2.2 Time amplitude characteristic of the low frequency EEG (0-4Hz) 26 4.3 E V A L U A T I O N M E T H O D O L O G Y 29 4.3.1 Methods to evaluate the impact of the Energy Normalization Transform 29 4.3.2 Effect of Energy Normalization Transform on the features related to VMRP 35 Chapter 5. Results and Discussion 37 i i i 5.1 O V E R V I E W 37 5.2 RESULTS 37 5.2.1 The optimal parameter values 37 5.2.2 Effect of the ENT on low frequency EEG 40 5.2.3 The Effect of the ENT on the LF-ASD performance 42 5.2.4 Demonstration that the ENT does not distort the features used in the LF-ASD.... 45 5.2 DISCUSSION 46 5.3.1 Results discussion 46 5.3.2 Potential features related to VMRP in the magnitude and phase spectrum 49 Chapter 6. Conclusions 52 6.1 S U M M A R Y OF CONTRIBUTIONS 53 6.2 SUGGESTED F U T U R E W O R K 53 References 55 Appendix A. Tables 59 A . 1 G L O S S A R Y OF ABBREVIATIONS 59 Appendix B. Procedures of data acquisition 61 B . l H A R D W A R E DESCRIPTION 62 B .2 D A T A ACQUISITION S T R A T E G Y 63 Appendix C. Additional study results 64 iv L I S T O F T A B L E S Table 3.1 Average Power of E E G recorded from FCi-Cj across different runs and subjects 18 Table 4.1 The subject information 25 Table 5.1 The optimal value of Wo over five subjects in Channels 1-6 (in sample) 38 Table 5.2 The optimal W N for the five subjects over channels (in sample) 40 Table 5.3 The separation between the active and idle data with and without the ENT 41 Table 5.4 Standard deviations of the low frequency E E G with and without normalization over subjects 42 Table 5.5 The overall performance improvement for the five subjects in terms of Area-under-ROC-Curve (PFp<l%) 44 Table 5.6 Performance improvement with normalization in terms of PTP corresponding with a specific PFP 44 Table 5.7 Averaged separation increase over channels with the ENT across subjects 46 Table 5.8 Performance of LF-ASD with different normalization schemes for Subject CB ...48 v L I S T O F F I G U R E S Figure 1.1a) The original LF-ASD without the Energy Normalization Transform; b) The modified L F - A S D with the Energy Normalization Transform 2 Figure 2.1 Functional Model of a BCI System 5 Figure 2.2 The simple model of the LF-ASD 9 Figure 2.3 Electrode distribution of the LF-ASD 10 Figure 2.4 An example of a codebook in the Feature Classifier in Channel 1 vs. Channel 4.11 Figure 2.5 The typical ensemble average of the IVMRP recoded from C 3 + 2 - C 3 12 Figure 2.6 a) Chinese character without normalization, b) The Chinese character with normalization 13 Figure 2.7 Normalization based on radar maximum likelihood estimation 14 Figure 2.8 Block diagram of the algorithm for suppressing impulse noise and normalizing baseline background drift 15 Figure 3.1 Procedure to Analyze the Energy Feature related to V M R P 19 Figure 3.2 For Subject PP, average power of the data falling into the Observation Windows vs. the window length (W 0 ) a) in the broadband; b) in the high-frequency band; c) in the low-frequency band 21 Figure 3.3 Analysis of the idle and active data in the frequency domain for Subject PP's E E G in Channel 1 ( F r F C i ) over 80 trails 22 Figure 4.1 The definition of M l , M2 and Idle2 26 Figure 4.2 Ensemble average of the low frequency EEG (Subject PP) centred at a finger switch with one second before and after the centred in Channels 1-6 27 Figure 4.3 The density distributions of M l vs. M2 for Subject PP in Channel 1 28 Figure 4.4 (For Subject PP in Channel 1) a)The density distribution of M2 vs. Idle2 (Idlel); b)The density distribution of M l vs. Idle2 (Idlel) 28 Figure 4.5 Evaluation of the impact of the Energy Normalization Transform 29 Figure 4.6 Procedure of the optimal Observation Window Size determination 31 Figure 4.7 Procedure of the optimal Normalization Window Size determination 31 Figure 4.8 Evaluation of the ENT impact on the LF-ASD performance 32 Figure 4.9 Definition of True Positive, True Negative, False Negative and False Positive... 33 Figure 4.10 An example of ROC Curves 34 Figure 5.1 Determination of the Optimal Wo for Channel 1 of Subject PP 38 Figure 5.2 Optimal W N determination for Channel 1 of Subject PP 39 Figure 5.3 Density distribution of signal amplitude for active and idle E E G data in Channel 1 for Subject PP a) without normalization, and b) with normalization 4C vi Figure 5.4 The ROC Curves (PFP<=1%) of Subject PP with different W N values and the corresponding ROC Curve without normalization 43 Figure 5.5 Comparison of spectral properties between E E G with and without ENT (WN=51) for Subject PP, a) Magnitude spectra of M l and M2 activity relative to Idle2, and b) Phase spectra of M l , M2 and Idle2 45 Figure 5.6 (For Subject CB in the broad band,) Average power of data falling in the Observation Window centered at finger switch activations vs. observation window size (Wo) 47 Figure 5.7 The ROC Curves of Subject CB with different normalization schemes and the corresponding ROC Curve without normalization 48 Figure 5.8 Magnitude and phase spectrums of Idlel, Idle2, M l and M 2 for Subject K T in Channel 1 50 Figure B . l Hardware description of the LF-ASD 62 Figure C l (For Subject CB's E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band 66 Figure C.2 (For Subject MP's E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band 67 Figure C.3 (For Subject MB's E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band 69 Figure C.4 (For Subject KT's E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band 70 Figure C.5 Determination of the optimal Normalization Window Size (DOM vs. WN) in Channels 1-6 a) for Subject CB; b) for Subject M P ; c) for Subject PP; d) for Subject KT; e) for Subject M B 73 Figure C.6 ROC Curves of the L F - A S D with and without the E N T a) for Subject M P ; b) for Subject M B ; c) for Subject PP; d) for Subject CB; e) for Subject K T 76 vii A C K N O W L E D G E M E N T S The work presented here would not have been possible without the support and contribution of many people. I would like to thank the participants in the pilot and evaluation studies, thank Dr. Peter Lawrence as my co-supervisor. I would like to thank my colleague, Dr. Steve Mason and Research Assistant Regi Bohringer, who recruited the subjects as well as record E E G signal at the Neil Squire BCI lab. I am very grateful to Dr. Steve Mason who gave me tremendous support for understanding the underlying issues and guiding me throughout the experimental tasks as well as defining and formalizing my thesis report. My deepest thank goes to Dr. Gary Birch for his enthusiastic guidance, constant support and valuable feedback. I would not have been able to complete this work without his patience and understanding. I dedicate this thesis to my family and friends for their unrelenting support and encouragement so that I would complete my studies. Also, I dedicate this work to my five years old niece suffering a brain injury. Z H O U Y U The University of British Columbia August 2002 viii Chapter 1 I N T R O D U C T I O N Over the last three decades, research in Human Computer Interaction (HCI) has been focused on obtaining effective and efficient communication between computers and humans. As a result of recent advances in signal processing technologies and increased computing power, novel sensing modalities, such as speech, vision-based gesture recognition, facial expressions, eye tracking, force-sensing and the electroencephalograph (EEG), have been introduced as potential input signals that could be embodied in a HCI system. Brain Computer Interface (BCI) research is a branch of HCI research that interprets a user's intent from electrical activity of the brain. The term BCI has been formally defined in [1] as a "communication system that does not depend on the brain's normal output pathways of peripheral nerves and muscles". In a BCI system, electrical brain activity is recorded from electrodes implanted under the skull or attached on the surface of the scalp. The recorded signal is then analyzed by a signal-processing unit, which produces control signals that relate to the intent of the human being. These control signals are then used as input to intelligent devices. One of the ultimate goals of the BCI research is to develop an improved interface for individuals with a high-level of impairment, such as those with severe stages of Amyotropic Lateral Sclerosis (ALS), Multiple Sclerosis (MS), or high-level spinal-cord injuries (SCI). The realization of a BCI would allow these people to effectively control devices such as wheelchairs, robotic assist appliances, computers and neural prostheses. Measurement of the brain electrical activity by E E G using electrodes placed on the surface of the scalp is the most common basis for BCI research. Implanted electrodes are generally considered too invasive. Other methods such as magnetoencephalography are too expensive and cumbersome to be practical. For intermittent control applications, the leading non-invasive (EEG-based) technique is the LF-ASD [2]. (Refer to Chapter 2 for a review of BCI technologies) Although the performance results achieved to date are encouraging, the L F - A S D error rates are not good enough for real world applications. For example, with False Positive rates around 1%, the successful classification rates are in the range from 44% to 81% with the L F _ A S D [3] [2;4;5]. It has been postulated that the performance of the LF-ASD could be improved by normalizing the input signal energy in a manner shown in Figure 1.1 b [6]. This idea is based on observations [7-9] that E E G high-frequency signal energy decreased significantly during 1 periods of movement. It is hypothesized that normalization would relatively increase the strength of the low-frequency features used by the LF-ASD during movement, and thus improve the separation between idle and active class EEG. Consequently, this is expected to lead to improve the performance of the LF-ASD . SIN T Low-Pass Filter SLPF _ Feature Extractor SFE Feature Classifier SFC* — — • SIN Energy Normalization Transform SN Low-Pass Filter SNLPF Feature Extractor SNFE Feature Classifier SNFC • • b) Figure 1.1 a) The original LF-ASD without the Energy Normalization Transform; b) The modified LF-ASD with the Energy Normalization Transform Energy normalization is expected to have a secondary benefit that can make the system relatively independent of the input scale. As seen in [3], the Feature Classifier of the LF-ASD is amplitude dependent. The codebook in the LF-ASD is static and supposed to be matched to the input signal with a specific scale. By utilizing the Energy Normalization Transform, the modified LF-ASD will be less sensitive to variations in input amplitude, and the input scale could be automatically adjusted. Thus, the error rate could be improved. In this work, a new transform, the Energy Normalization Transform, is proposed to improve the design of the LF-ASD. 1.1 Research Goals The goal of this work was to test the hypothesis that the Energy Normalization Transform would decrease the error rates of the LF-ASD and make the LF-ASD design less sensitive to fluctuations in EEG amplitude. In order to evaluate the modified design over the original LF-ASD design (shown in Figure L i b and a), two studies were conducted. The goals of the first study were 1) to determine the energy characteristics of bipolar EEG related to movement potential, and 2) determine the impact of the Energy Normalization Transform on low frequency EEG by comparing the characteristics of signals SLFP and SNLFP (shown in Figure 1.1). The goal of the second study was to determine the impact of the Energy Normalization Transform on the performance of the LF-ASD by comparing error rates in the system output SFC and SNFC. 1 . 2 Overview of the Thesis Chapter 2 presents a summary of relevant research that provided the groundwork for the development of the modified LF-ASD with the energy normalization transform. Chapter 3 provides details of the algorithm of the improved system design and also its rationale. In Chapter 4, the methodology to study the impact of the Energy Normalization Transform on both low frequency data and the performance of the LF-ASD is specified. The signal 2 characteristics related to both voluntary movement-related potential ( V M R P ) and idle E E G are also introduced in Chapter 4. The results from the two studies are presented and discussed in Chapter 5. Chapter 6 contains conclusions and suggestions for future work. A list of abbreviations related to this work is provided in Appendix A . Appendix B describes how the E E G data was recorded. Appendix C contains plots that offer additional comprehensive results to show the impact of energy normalization on low frequency E E G data and the performance of the L F - A S D over subjects in various bipolar channels. 3 Chapter 2 B A C K G R O U N D Over the last three decades, research on BCI has been focused on finding an alternative communication channel to the existing interface techniques. To date there are more than 20 research groups working on BCI worldwide. Two categories of switches, continuous and intermittent, are under development by various BCI labs. In this chapter, three topics are presented. First, the major BCI techniques are reviewed and evaluated. Second, the characteristics of the EEG signals related to voluntary movement related potential (VMRP) are reviewed. Third, a few applications of normalization transform used in fields such as radar, EKG, and Chinese handwriting recognition are discussed. 2 . 1 Brain-Computer Interface (BCI) Research BCI researchers study the electrical activity detected from the brain. Brain cells communicate by producing tiny electrical impulses and the cumulative effect of those signals can be detected by electrodes over various areas of the brain. These signals have multiple applications, such as bioengineering application, human subject monitoring, and neuroscience research. Since EEG has good time resolution and can be obtained by inexpensive equipment through noninvasive acquisition, it is regarded as a good signal source for BCI research. The methodology to acquire the electrical brain signal falls into two categories: invasive acquisition and non-invasive acquisition. In invasive acquisition, the electrodes are implanted under the skull; while in non-invasive acquisition, surgery is not needed. For example, electroencephalography (EEG) and magnetoencephalography (EMG), positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) are signal sources based on non-invasive acquisition, while EcoG is a technology based on invasive acquisition. They are all potential technologies, which can be used in a BCI. Since EEG is obtained using non-invasive acquisition with inexpensive equipment and research results have shown that EEG is capable of revealing information related to human intentions [1], it is regarded as a good signal source to be used in a practical BCI for people with disabilities. 4 2.1.1 BCI System Design To date, researchers have designed a few BCI applications for different applications. To provide a context of those designs, a functional model of a BCI system is discussed in this section. Mason and Birch [10] proposed the functional model, which provided a protocol to describe a BCI design. The simplified functional model of a BCI system is provided in Figure 2.1 Device user-reported error Figure 2.1 Functional Model of a BCI System The BCI control component is device-independent. Its function includes signal recording, signal amplifying, feature extraction and feature classification. It performs like an interpreter between the human brain and the output of logical level signals. There are two types of BCI control: intermittent and continuous. Intermittent BCI Control permits users to take control of the system intermittently when they desire. Continuous BCI control expects users continuously producing control signals. Control Interface translates logic signals from the BCI Control into signals with semantic meanings; it shows the user the interpretation results of the BCI Control, and then allows the user to make dynamic adjustment. For example, when a subject was using a virtual keyboard BCI system, the user's intent was interpreted and fed back to him through a monitor, such that the user could adjust his brainwave dynamically during the operation. In addition, the On/Off Mechanism is in the control interface and it defines how the host system is turned on and off. Device controller can translate the semantic signal into a physical control signal, which is used to drive the device. "It is an important problem for practical applications: detection of the user commands without the timing cues provided by structured trails." [11] Based on this thought, the applications of BCI design fall into two categories: continues application and intermittent application. Continuous applications need users to produce control signal all the time, while intermittent applications allow users to give control intent signal intermittently at their own pace. 5 2.1.2 Existing System Designs 2.1.2.1 BCI Research at the Wadsworth Centre The Wadsworth Centre in New York [12] [13] [12] is one of the leading BCI research centres in the world. One of their well-known projects is a BCI system, which could move a cursor towards a given target on the screen [12]. It is a continuous application, and it has a continuous BCI Control. No report exists about the successful classification rate when the subject is in the idle state. The features, which the designers use for control of the cursor, are the amplitudes of the signal at the mu (8-12 Hz) and beta (18-26 Hz) band over the sensory-motor cortex. It has been proved that movement or preparation for movement is typically accompanied by a decrease in the mu or beta band rhythms, and this decrease in the mu and beta band has been labelled as "event related desynchronization (ERD)" [13]. In one study, subjects were required to drive a one-dimensional cursor to a target. The chance performance is 50%. 16 subjects including three spinal cord injured and two A L S subjects were reported to have achieved the accuracy rates ranging from 51% to 94% after training [12]. For six of the sixteen subjects who had completed at least 20 training sessions, the last four to five sessions from each subject were used to report the results. The other ten subjects completed ten sessions, and only sessions nine and ten were reported. Subjects varied greatly in their learning rates, and accuracies higher than 90% were expected to take several months to develop. No information about the performance difference between the subjects with disabilities and the able-bodied subjects was reported. When they applied this technology to develop a precise one-dimensional control and choose among up to 8 different targets, they achieved information transfer rates up to 20-25 bits/min [14]. In another study [15], a cursor was designed to move in two dimensions, where the chance performance was 25%. The three able-bodied subjects, who had been trained and achieved good result in one dimension cursor moving sessions, were introduced to the experiment. After five to six training sessions of moving the cursor in two dimensions, accuracies of 60% to 65% were reported over the three subjects. 2.1.2.2 The Graz BCI System The Graz BCI system [16-18] is a famous BCI Design to detect a subject's finger flexion. This application is a continuous application and has a continuous BCI Control. No report of the Graz BCI system exists about its successful classification rate when the subject is in the idle state. The feature that Pfurtscheller and his colleagues chose to study was the changes in amplitude in the mu, beta rhythm and other frequencies bands from 5 to 30 Hz associated with preparation of specific movements [16-18]. In the prototypes of this system[16], the subject was asked to press a micro-switch with either his/her left index finger or right index finger upon presentation of a stimulus cue on a computer monitor. The E E G was recorded from electrodes overlying the sensory-motor areas, and the signals were analyzed in subject-specific frequency bands and classified on-line using a neural network. The result of the detection was then fed back to the subject via a monitor. The first 3-4 training sessions were for setting up the neural net by non-supervised learning. Every session lasted around 1 hour. After that, in 6-7 subsequent sessions, user feedback, which defined how well the classifier could recognize the user's intention, was 6 offered to the users for supervised learning, and the neural net parameters were adjusted between daily sessions. Trials that did not match both left and right hand finger movement were rejected. In the last test session with two choice trials (i.e. left hand vs. right hand imaginary), three users can reach accuracies of 91.3%, 68.2%, and 87.3%. This group's recent research was focused on autoregressive frequency analysis and use of alternative spatial filters to improve classification. It was reported that about 90% of people were capable of using this system with accuracies similar as those above [11]. 2.1.2.3 VEP-Based BCI The VEP-Based BCI [19] can figure out which letter a subject is looking at in an 8 by 8 letter array shown on the screen. This application is a continuous application and has a continuous BCI Control. No report of the VEP-Based BCI system exists about its successful classification rate when the subject is in the idle state. Since the feature of V E P depends on the gaze direction (therefore, on extra-ocular muscles and the cranial nerves that activate them [11]), VEP-based BCIs are dependent BCIs. Sutter [19] studied steady-state visual evoked potentials (VEPs) recorded from the visual cortex of the brain. He presented a 64-symbol array (an 8 X 8 grid on a CRT screen) to subjects who were asked to concentrate visually on a letter of their choice. The colour of each letter could be changed rapidly at a unique frequency between red/green to evoke detectable signals using scalp electrodes over the visual cortex. The response to the visual pattern on the screen depends on the letter (direction) that the subject was looking at. This response was then compared with templates attained during a training session. From these comparisons, the system determined which symbol the user was looking at. The prototype system was tested with over 70 able-bodied and approximately 20 severely disabled subjects. After one hour of training, one subject with A L S was able to reach communication rates of 10 to 12 words/minute using implanted electrodes, while able-bodied subjects achieved communication rates of 10 to 12 words/minute [11]. 2.1.2.4 P300-Based BCI P300 is an event-related brain potential (ERP) elicited by rare, task-relevant events and has a latency of approximately 300 milliseconds. The amplitude of the P300 varies directly with the relevance of the eliciting events and inversely with the probability of the stimuli. The P300-Based BCI [20] was designed to tell which character the subject wants to pick out of a 6 by 6 matrix. This application is a continuous application and has a continuous BCI Control. No report of the P300-Based BCI system exists about its successful classification rate when the subject is in the idle state. The P300-Based BCI is different from the VEP-based BCI. The P300-Base BCI system records E E G from the parietal site, while the VEP-Based BCI acquires E E G from the scalp over the visual cortex. The potential generation of P300 depends mainly on the user's intent, and not on the precise orientation of the eyes[20]. In people with visual impairments, auditory or tactile stimuli might be used. A P-300-based BCI has an apparent advantage in that it requires no initial user training: P300 is a typical or naive response to a desired choice. However, P300 and related potentials are likely to change over time. In the long term P300 might habituate so that BCI performance deteriorates (or 7 improves). Therefore, appropriate adaptation by the translation algorithm is likely to be important [11]. Using the features of the P300, Farwell and Donchin [21] developed the P300-Based BCI system that can tell which character the subject wants to pick out of a 6 by 6 matrix on a screen displaying visual stimuli that elicits unique response related to each cell in the matrix. This P300-Based BCI could serve as a keyboard emulation system providing a mean communication rate of 2.3 characters per minute with 95% accuracy [21]. 2.1.2.5 The Thought Translation Device (TTD) The TTD [22;23] was designed to interpret the binary choice made by the brain. This application is a continuous application and has a continuous BCI Control. No report of the TTD BCI system exists about its successful classification rate when the subject is in the idle state. Slow Cortical Potentials (SCPs) of E E G measured at the vertex is the feature related to human intention, which the TTD use. Birbaumer and his colleagues built their system by studying slow cortical potentials (SCPs) in a 2-second rhythm [22;23]. They demonstrated that subjects with A L S could use this technique for conducting a binary choice through the alphabet in order to select letters or words from a language support program. After prolonged training of hundreds of sessions, three subjects with A L S were able to achieve self-control. Mean percentage accuracies for the binary synchronous selection of tasks based on the last 20 sessions for these three subjects were reported to be 86.7%, 46.2%, and 66.1% for the selection task with corresponding 51.5%, 74.0%, and 76.2% for the rejection task respectively. Two of these A L S subjects were reported to be able to write their own correspondence by using the TTD at an average speed of 2 minutes per each letter selection [22;23]. A recent report said that the improved TTD can help subjects have two choice accuracies of 65-90% at the speed of 0.15-3.0 letters per min [11]. Although these rates are low, the system is useful to and highly valued by people with high-level locked in disabilities. 2.1.2.6 Cross-Correlation Based Brain Interface (CCBBI) CCBBI [24] was designed to differentiate a movement related E E G from spontaneous E E G signal. This application is an intermittent application and has an intermittent BCI Control. The designer studied the motor related signal in EcoG, which was recorded by invasive electrodes under the skull. Using the trigger-averaged event related potential (ERP), a template of V M R P was obtained. Then by cross-correlation of the averaged template with the input of continuous EcoG, the cross-correlation coefficients were obtained. If the coefficients were greater than a threshold value, the interface would be in the activated state. With the higher signal quality of ECoG over E E G , it was reported that the True Positive rate of this system could be higher than 90%, with the corresponding False Positive rate of 1%. However, the invasive data acquisition is a major limitation. 2.1.2.7 The OPM (Outlier Processing Method) The O P M [25] was designed to differentiate a movement related E E G from spontaneous E E G signal. This application is an intermittent application and has an intermittent BCI Control. The O P M is based on the premise that E E G activity measured from the scalp can be 8 modeled as the summation of event-related potentials (ERPs) and statistically independent background E E G activity. In this approach, a generalized robust maximum likelihood estimate is utilized to provide a robust estimate of the ongoing, underlying E E G process, which in turn is subtracted from the original E E G activity to yield an estimate of the outlier potential. The time series of outliers produces waveform patterns that provide single-trial event-related information. In addition, O P M has the potential to differentiate between different types of movement, such as left hand, right hand and foot movement. It was reported that the True Positive rate of this system was around 25%, with the corresponding False Positive rate of 1%. 2.1.2.8 The LF-ASD (Low Frequency -Asynchronous Switch Device) The L F - A S D [3] is an intermittent application and it has an intermittent BCI Control that was designed to recognize potentials related to VMRPs in E E G measured over the sensori-motor and supplementary motor cortices. The translation algorithm uses features extracted from the 0-4Hz band in six bipolar E E G channels. VMRPs is recognized in ongoing E E G rather than in the E E G associated with externally paced trials. Thus, it can detect of the user commands without the timing cues provided by structured trails [11]. By analyzing E E G with a custom wavelet, the features related to voluntary movement related potential (VMRP) periods showed a definite difference from that in the idle periods. L F - A S D is a mainstream intermittent BCI application. To provide a context of its design, the signal processing components of the L F - A S D are shown in Figure 2.2. Feature Extractor SFE ^ Feature Classifier w Figure 2.2 The simple model of the L F - A S D The LF-ASD includes a Low Pass Filter (LPF) with the cut-off frequency at 4 Hz, a Feature Extractor, and a Feature Classifier. The input of the L F - A S D is a six-dimension bipolar E E G signal recorded from F,- FC, , Fz- FCz, F 2 - F C 2 , F C i - C i , FCz-Cz and F C 2 - C 2 on the scalp. Refer to Figure 2.3. Before being inputted into the L F - A S D , the record E E G was amplified and sampled at 128 Hz. 9 Figure 2.3 Electrode distribution of the L F - A S D The function of the LPF is to decrease the interference with the features in the low frequency band. In the Feature Extractor, the features were derived using Equation 2.1. _ | £,(«)• Ej(n) Ei(n)>0and £ . ( « ) > 0 ^ ^ u I 0 otherwise where the elemental features, Ej and Ej, were defined by E{(n) = eki (n + ai)-eki (n + p.), i = 1, 2,..., M (2.2) Ej(n) = ek](n + aj)-ekj(n+Bj),j = 1, 2,...,M (2.3) where ek(n) is the kth observed, bipolar E E G signal, n indicates discrete samples of time, a,-, Pi, aj, and Pj are system delay parameters, and M is the number of features evaluated. The sub-subscript / used in ek(n) associates the pair of delay parameters a,- and Pi to the bipolar signal ek(n) and similarly for subscript j. The notation does not imply that different bipolar signals were used in obtaining the pair of Ej(n) and E/n). In order to increase the robustness of the signal detection to trial-by-trial latency variation, these feature values were collapsed over l/8 t h of a second into the aggregate features defined by G^n) = max(giJ(n-S),gij(n-7),...,gij(n + l)) (2.4) where max(.) represents the maximum value in the data set. The extracted feature values are then sent to the Feature Classifier to matched a codebook representing active (containing VMRP) and the idle EEG. The Feature Classifier implements a one-nearest-neighbor (1-NN) classifier, which determines if the features belong to V M R P (active) or spontaneous (idle) EEG. The codebook is a template representing the idle and active pattern in the Feature Classifier. An example of the codebook is shown in Figure 2.4. 10 x 10 Codebook sampleof the Feature Classifier 2;5 •S-1:5h The Idle Pattern The Active Pattern '••5|S!*: 1 1.5 2 Featureamplitude in Channel 1 x.10 Figure 2.4 An example of a codebook in the Feature Classifier in Channel 1 vs. Channel 4 Note: Since it is impossible to plot the codebook in 6-dimension hyperspace, this example shows the codebook in 2 dimensions only. As indicated in Chapter 1, Birch and Mason evaluated the performance of the LF-ASD. Its True Positive rate was in the range of 44%-81% with the corresponding False Positive of 1% [3] [4;5]. Although the L F - A S D is a leading intermittent control application and its performance is encouraging, its correct classification rate is still not good enough for real world applications. Hence research directed at improving this performance is required. 2 . 2 Characteristics of Movement-Related E E G E E G is a complex, unstable signal and it is easily corrupted by noise from other sources, such as eye movement, muscle movement and other brain electrical activity unrelated to V M R P . Therefore, single trial analysis is difficult. One effective way to study E E G is to employ the ensemble average of the signal or features. The averaged signal is used as a template for characteristic analysis. L.Deecke [26] averaged the bipolar E E G signal containing movement potentials from location of C3+2-C3, which is close to location of F C i -C i . As indicated earlier, two electrodes over F C i - C i are used in the L F - A S D . Therefore, one could expect that the characteristics of the bipolar signal recorded from F C i - C i should be similar to that recorded from C3+2-C3. The averaged result centred at the onset of a finger switch was provided by L.Deecke and shown in Figure 2.5. 11 Figure 2.5 The typical ensemble average of the IVMRP recoded from C3+2 -C 3 In Figure 2.5, the averaged signal related to the movement potential can be divided into two parts. In the first part, which is called the movement preparation period, the curve bends towards positive. In the second part, which is called the movement period, the curve goes in the negative direction. In addition, signal recorded in the idle periods is approximately a zero mean random process [27]. The difference between E E G around VMRPs and in idle periods exists not only in the time domain, but also in the frequency domain. Jasper and Penfield [9] studied the mu (8-12Hz) rhythm. They reported that the signal amplitude in the mu band dropped dramatically during periods of V M R P in six out of nine subjects' EEG. They also noticed that this amplitude decrease varied over subjects. Mason furthered the observation in the broadband [7]. He found that, when the movement potential occurred, the energy of E E G in the higher frequency band decreased significantly. Therefore, the energy in the broadband at VMRPs would decrease, comparing with that in the idle periods. This observation was made on aggregate data. Therefore, it is possible that a single trial test result does not always follow this rule. However, it does suggest that some form of signal-energy normalization may improve features of active E E G in the low frequency band. 2.3 Normalization theory In real world applications, scientists have found that the performance of a design is usually undesirably influenced by some factors represented in the single trial signal. Normalization is defined as an operation to make the system performance independent of these factors [28]. For example, in the research on Chinese character recognition, the style of handwriting is usually individualized. It is hard to match the handwriting to the print-style fonts. In order to decrease this dependence of recognition accuracy on the individualized writing style, a two-dimension normalization operation could be applied. One of the well-known algorithms is to normalize the Chinese characters by relatively equalizing the density of the black-dots within the character space [29]. The comparison with and without normalization operation with this principle is shown in Figure 2.6. After the normalization operation, the position of the character was adjusted to be independent of individual preference, and then the successful classification rate increased [29]. 12 Figure 2.6 a) Chinese character without normalization, b) The Chinese character with normalization. Cochran [28] reviewed the existing normalization approaches. Assuming the input signal to a normalization transform is x(t) and the none-zero norm of the input is TJ(x), the output signal of a normalization transform is y(t) - x(t) lrj(x). T] (x) is usually defined as following: The value of 7] is the integration of x(t) in the normalization window. dt 1. T](x) = j\x(t) within normalization window 2. Also in some cases, r/(x) is defined as its maximum (or minimum) value within the normalization window. rj(x) = sup (x) within normalization window where sup(.) means to get the maximal (minimal) value in the normalization window. 3. r] (x) can be defined in terms of the measurement of "size" of the signal that is meaningful within the context. In addition, Cochran [28] pointed out that most of the normalization approaches are "highly non-linear" operations. There are inherent difficulties in approximating the normalization algorithm even locally by a linear operator. Since a Fourier transform is usually applied to the analysis of the linear and time-invariant system, "this non-linear nature of the signal normalization makes frequency response analysis of the normalization filter meaningless." This issue will be taken into account and is discussed in more detail in Chapter 5. Although to date most E E G researchers used very simple methods, such as low-pass or band-pass filtering, to decrease interference from noise, normalization operations were widely applied in radar, E C G , and other one-dimension signal processing. Those examples provide reference to E E G normalization. 2.3.1 Normalization Based on Radar Maximum Likelihood Estimation In radar or sonar signal processing, in order to decrease dependence of the correct decision rate on the noise power, researchers employ a normalization operation with the observation input divided by the noise power. The approach is shown in Figure 2.7 [30]. 13 N o r m a l i z a t i o n C 1 a s s i f i e r L (.) i— t " ' N o i s e P o w e r E s t i m a t i o n Figure 2.7 Normalization based on radar maximum likelihood estimation. Since the noise power is unknown, an estimation is made in block A . In the white Gaussian noise case, L(.) can be replaced by a Matched Filter, while A is the estimation of the noise power, which corresponds to the norm, T] (x), in Cochran's review. Based on different assumptions, there are several approaches to estimate the noise power. The performance of the estimation and corresponding classification depend on the density distribution of the noise. 2.3.2 Normalization of Electrocardiogram Signals Chu and Delp [31] proposed a morphological normalization method to eliminate both spike noise (with sharp amplitude but short duration) and slow background baseline drift in a E C G signal. They named the two basic normalization operators as Dilation and Erosion. Dilation and Erosion are basic normalization operations. The Dilation operation outputs the maximal value of the input signal in the normalization window, while the Erosion operation outputs the minimal value of the input signal in the normalization window. This is similar to sup(.) operation in Cochran's review, but it is more complex in the following procedure. Dilation and Erosion are used in tandem in the E C G normalization transform. Dilation followed by Erosion is defined as Closing, while Erosion followed by Dilation is defined as Opening. If the normalization window size is small, the Opening and Closing algorithm can be applied to eliminate (positive and negative) spikes with short duration. If normalization window size is large, Opening and Closing algorithm can be used to obtain the baseline background drift. Then by using the original signal minus it, baseline background drift can be eliminated. In a real application, in order to eliminate both the (positive and negative) spikes and slow baseline drift, opening and closing are usually combined as shown in Figure 2.8. To achieve slow baseline drift, the normalization window size applied in the second step is hundreds of times larger than those in the first step. 14 Opening Closing Opening Closing i r Input Closing Opening ^Average f* Closing Opening •(Average f* Eliminating the spike noise with small Normalization Window Deriving the baseline background noise with a large Normalization Window Output Figure 2.8 Block diagram of the algorithm for suppressing impulse noise and normalizing baseline background drift 15 Chapter 3 D E S I G N O F T H E M O D I F I E D L F - A S D As described in Chapter 2, the LF-ASD is one of the leading intermittent switches. Although its performance is encouraging, it is not good enough for real-world applications. The LF-ASD needs to be modified to improve its error rate. In this chapter, first, a signal-processing approach, Energy Normalization Transform, and the modified LF-ASD are proposed. The rationale of the new design is also briefly introduced. Second, power characteristics of the test data are studied in detail to support the design rationale of the proposed new design. 3.1 The Proposed Modified LF-ASD 3.1.1 Design The proposed design of the modified LF-ASD was shown in Figure 1.1b and the original design of the L F - A S D was shown in Figure 1.1 a. In the original design, the input to the low-pass filter (LPF), denoted as S,N is a six-dimension bipolar E E G signal recorded from F i - F C i , Fz-FCz, F 2 - F C 2 , FC1-C1, FCz-Cz and F C 2 - C 2 sampled at 128 Hz. The cut-off frequency of the LPF is 4 Hz. The Feature Extractor of the LF-ASD extracts custom features related to VMRPs. The Feature Classifier implements a one-nearest-neighbor (1-NN) classifier, which determines if the input signals are related to a user state of voluntary movement or passive (idle) observation. In the modified design, the Energy Normalization Transform is before the LPF, and all the other components are the same as defined in [3]. The thought was originated from the radar signal normalization [30]. The Energy Normalization Transform was implemented using Equation 3.1 J 2 ] 5 ^ ( n - j ) y.v=-(vt,„-D/2 where S (ri) is the input to the LF-ASD, $N(ri) is the output of the transform, and WN , which defines the length of a data window, is called Normalization Window Size. 3.1.2 Optimal Parameter Determination The optimal value of WN is the only parameter in the Energy Normalization Transform. It was determined by measuring the capability to achieve the largest separation between the 16 Voluntary Movement Related Potential (VMRP) and the idle data under a given separation rule. The method of its determination is listed in the following paragraphs: First, by putting an observation window centred at the onset of voluntary movement related potential (VMRP), an active data set containing V M R P and an idle data set containing spontaneous E E G can be determined. Data contaminated by ocular artefact are discarded. Second, the optimal WN could be obtained by exhaustive search for the maximal class separation between the active and idle data set. Since small WN values may cause distortion, small WN values are ignored in the exhaustive search. The detail description about how to separate the idle and active E E G signal and the method of the exhaustive search in this work is provided in Section 4.3.1. 3.1.3 Design Rationale The design of the modified LF-ASD is based on several BCI researchers' observations and the study results related to the E E G power variation in this work. Two factors, the instantaneous power decrease around movement potentials and the E E G power variation over time and subjects, could result in false classifications of the original L F - A S D . The Energy Normalization Transform (ENT) can combat these two negative factors. Therefore, it was postulated that ENT could improve the system performance. 3.1.3.1 Power Variation Study results in Section 3.2 shows that power of the bipolar E E G varies over time and from subject to subject. This may cause undesirable amplitude variation. Since the LF-ASD was designed to extract features from signal amplitudes, this power variation might result in undesirable signal amplitude variation, and consequently cause false classifications during operation. The proposed ENT can decrease the sensitivity of the system to power variation, and consequently decrease the error rate of the L F - A S D . 3.1.3.2 E E G Signal Power Decrease during Periods of VMRP Jasper and Penfield [9] reported the mu rhythm (8-12 Hz) decrease and Pfurtscheller et al. [8] reported the beta rhythm (18-26 Hz) decrease when people are in a state of tension or movement. Mason [7] found that the signal power in the frequency components greater than 4Hz decreased significantly during V M R P periods, while at the same time the signal power in the frequency components less than 4Hz did not decrease. This power decrease in the high frequency band may cause the overall power in the broadband (0-64Hz) to decrease during VMRPs. In this work, this phenomenon is named as the Energy Feature related to V M R P . In addition, for the L F - A S D , the codebook vectors representing the idle pattern in the Feature Classifier usually has a smaller norm value than the codebook vectors representing the active pattern. (An example of codebook vectors used by the L F - A S D is provided in Figure 2.4.) The power decrease around VMRPs reduces the distance between idle and active codebook. The proposed ENT is designed to increase signal power around VMRPs , and then it could increase the distance between the idle and active codebooks, and thereby increases the successful classification rate. 17 3.1.4 Concerns Related to the Modified LF-ASD Design Since the ENT is not a linear operation as discussed in the Chapter 2, a major concern is whether or not the ENT distorts the features related to the V M R P . If these features were distorted, it would potentially hamper the performance of the Feature Classifier of the L F -ASD. The answer to this concern is that it does not distort the features, and the demonstration will be provided in Chapter 5. It turns out that by properly choosing the filter parameters, the ENT does not distort the magnitude and phase spectrum of the V M R P and the idle E E G in 0-4 Hz band. Therefore it does not distort the features related to V M P R . 3.2 Study to Determine E E G Power Characteristics Related to VMRP Detection Although the researchers [7] [32;33] have reported the existence of the Energy Feature related to V M R P , further observation across more subjects was required. In this section, power characteristics of the E E G data in the Neil Squire BCI lab were tested over 5 subjects. 3.2.1 Power Variation over Time and Subjects: A session of E E G is defined as the data recorded from a subject on the same day. In a session, if the data are recorded over several periods into several data sets, each set is defined as a run of E E G . Each run lasts around 2 minutes in this work. The test data in this work were acquired from five subjects. Data with eye blink contamination was discarded. (For details regarding how the. E E G data was acquired, see Appendix B.) These subjects were asked to perform the same set of tasks involving voluntary right finger flexions over eight separate runs in a session. To prove E E G power varies over time, the average power of the E E G signal in each E E G run was calculated, as in Equation 3.2 below, and compared. Length of a run / ^ Sample Amplitude 2<n) / Average Power - n=0 /T , (3.2) / Length of a run Table 3.1 Average Power of E E G recorded from F C i - C i across different runs and subjects I Runl Run2 Run3 Run4 Run 5 Run 6 Run 7 Run 8 [Subjectl 6.44 7.07 6.28 7.12 6.86 6.68 5.63 6.15 Subject2 8.14 7.47 11.97 12.26 11.93 12.24 12.06 11.78 Subject3 7.07 7.04 6.31 6.06 6.42 7.07 7.07 7.98 Subject4 4.00 3.29 3.76 3.10 3.49 3.84 3.42 3.57 Subject5 2.59 2.71 2.67 2.89 2.64 2.78 2.56 2.81 Table 3.1 shows that E E G power varies over both time and subjects. Since the Feature Classifier of the LF-ASD is sensitive to the input amplitude variation [3], the E E G power 18 variation over time hampers the system performance. The impact of power variation over time on the system performance is evaluated in Chapter 4 and Chapter 5. 3.2.2 Power Decrease during the Period around V M R P (Energy Feature of VMRP) The subjects were asked to trigger a finger switch upon the presentation of a stimulus cue on a computer monitor. In between trials, subjects were in a passive observation state. Thus, the EEG signal, containing both active and idle data, and a control signal revealing the finger switch activation time were obtained. EEG contaminated by Ocular Artefact were discarded. [HPF >4Hz Onset of Finger Switch MHz Concatenating ttrials into a set I EEG input LPF (MHz Idle VMRP Idle Observation Window 04Hz -> • I Broadband signal Separating the data containing VMRP from Broadband signal the idle data. Calculating average power by Equation 3.3 Concatenating trials into a set Calculating average power by Equation 3.3 Concatenating trials into a set Calculating average power by Equation 3.3 Figure 3.1 Procedure to Analyze the Energy Feature related to VMRP The experiment, as shown in Figure 3.1, was designed to obtain the average signal power of the EEG data around finger switch activations in the high (>4Hz), low (0-4Hz) and broad (>=0Hz) frequency band respectively. As shown in Figure 3.1, first, the original EEG data was filtered into three signals: low frequency component (0-4Hz), high frequency component (>4Hz) and broadband signal (>=0FIz). Second, the active data around VMRPs were obtained by an Observation Window centred at the onset time of the finger switch. Data falling into the Observation Window belongs to the active pattern, while the data falling out of the Observation Window represents the idle pattern. As the Observation Window Size (Wo) increases, more data belonging to the idle pattern may fall into the Observation Window. (Note: Wo is different from WN- Wo defines how to separate active and idle data in the original EEG runs, while WN is the normalization window size defined in Equation 3.1.) Third, for a specific Wo and in each frequency band, the data in the Observation Windows were put together into an active data set representing the active EEG. Fourth, for the high frequency, low frequency and broadband EEG signal respectively, the average power of the active EEG data was derived by .Equation 3.3. Assuming total sample number in the active data set is N, then N / ^ Sample Amplitude 2 00 / Average Power of the active data = " = 0 A (3-3) If (due to the energy decrease in the high frequency band) the overall signal power decrease around VMRP is true, both in the broad and high frequency band, the average power of the signal in the active data set should increase consistently against the increase in the Wo (i.e. more idle data falls into the active set). The study results are provided below: 1. In the broadband, the average power of the data in the active set vs. Wo. 19 Power of VMRP in the Broadband vs. Length of the Observation Window 2 <?° j Channel 1 SO 10O' 1 4Q;. 1:60 18Q 2QO SO 1Q0 1 4Q 1 6Q 180 2O0 'Channel 3. GO OO lOO 12Q. 14Q 1GO ISO 2QO | Channel 4 [ 60 .80 1QO 120 1 4Q 1 6Q 1 8 Q ^ ^ ^ _ _ 2QQ Channel 5 | 6(3 SO lOO 12Q T40 16Q 1 SO 2QO Channel 6 | 20 40 SO SO 100 120- 140 160 180 200 Length of the Observation Window - Wo (In Sample Numbers) a) 2. In the high frequency band, the average power of the data in the active set vs. W a Average Power of VMRP in the High-frequency Band vs. Length of the Observation Window 4 i — 1 1 1 S 3 ft, f r 60 80 10O : 120 140 1 6Q 180 200 60 80 100, 120 140 160 180 200 RO SO 1QO 12Q 140 16Q 18Q 2O0 40 60 100 120 140 160 180 200 6Q OO 1Q0 120 14Q 160 180 2Q0 20 40 60 80 100 120 140 1 OO 180 200 Length of the Observation Window -Wo (In Sample Numbers) b) 3. In the low frequency band, the average power of the data in the active set vs. W 0 . 20 P o w e r o f V M R P i n L o w - F r e q u e n c y b a n d v s . L e n g t h o f t h e O b s e r v a t i o n W i n d o w 25 r . r. r 1 r n n | - I L I U t L L 1 I. . ' 20 40 60 80 100 120 146 160 180 200 L e n g t h o f t h e O b s e r v a t i o n W i n d o w - W o ( In S a m p l e N u m b e r s ) C) Figure 3.2 For Subject PP, average power of the data falling into the Observation Windows vs. the window length (W 0 ) a) in the broadband; b) in the high-frequency band; c) in the low-frequency band. The results of the other four subjects' E E G are provided in Appendix C. From the results above, the following observations were made: 1. The average power of high frequency components (>4Hz) increased consistently with increasing observation window size. In other words, the power of the high frequency E E G decreased around VMRPs. This was consistent over different subjects. 2. The power of the low frequency components (0-4Hz) did not decrease around VMRPs. This rule was consistent over different subjects. 3. The power of the broadband E E G (>=OHz) increased with increasing observation window size (Wo). This showed that the power of the E E G in the broadband dropped during periods of V M P R activity. In this work, this phenomenon is named "Energy Feature related to V M P R " . This feature was consistent over most signals of the subjects. 4. The study results across subjects and across channels (Provided in Appendix C) showed that, in the broadband, the Energy Feature related to V M P R was stronger in the 1st, 2 n d and 3 r d channel, while it was less strong in the 4 t h, 5 t h, 6 t h channels. The signal powers in the front electrodes (Channels 1-3) in broadband had a stronger decrease than that in the back electrodes (Channel 4-6). For example, for all the subjects in the study, in Channels 1-3 power of the broadband E E G decreased consistently in the range of 5.8%- 41.7% during VMRPs, while in Channels 4-6, for four out of five subjects in the study, power of the broadband E E G decreased in the range of 3.1%-29.1% around VMRPs. For the fifth subject's (Subject CB) in Channels 4-6, power of the broadband E E G did not drop around V M R P , but had a little increase. Further study on Subject C B ' E E G showed that power increase during 21 V M R P periods was caused by the power increase in the low frequency components, although the power in the high frequency band decreased. In order to compare the power distribution between the spontaneous E E G and E E G around VMRPs in the frequency domain, the magnitude spectrum of active and idle E E G were derived in the study. The derivation method is provided below. Similar to the method of separating data containing VMRPs from the idle data described in Figure 3.1, in the broadband, the E E G signal was separated into an active and an idle data set using an Observation Window centred at the onset of the finger switch. In order to decrease the spectral leakage in the frequency domain analysis, in this step a Hamming Window replaced the rectangle window as the Observation Windows to separate data around VMRPs from idle data. After that, an FFT was applied on the windowed data for the active and idle magnitude spectrums. Finally the ensemble average of the magnitude spectrum of both V M R P and idle patterns were achieved and are shown in Figure 3.3. This study was free of eye-blink contamination. mu Rhythm Observation Idle-VMRP 3500 2500 1500 -20 0: 20 Frequencyjn Hz Figure 3.3 Analysis of the idle and active data in the frequency domain for Subject PP's E E G in Channel 1 ( F i - F d ) over 80 trails Observations were made as follows: When the subject was in a state of being relaxed or idle, signal energy in the 8-12 Hz and 18-26Hz band increased, i.e. mu and beta rhythm appeared. While the subjects were in movement states, signal power in the mu and beta band decreased dramatically. The signal energy in the high frequency band decreased significantly during movement potential periods. 22 Overall, conclusions drawn from experimental results in Section 3.2 are as follows: During the V M R P periods, due to the significant power decrease in the high frequency band (>4Hz), broadband power decreases, but power in the low frequency band (0-4Hz) does not decrease around VMRPs. Since the power decrease is one of the features related to movement activities, which was named the "Energy Feature related to V M R P " in this work. Generally, this Energy Feature related to V M R P is consistent over subjects and across channels. The Energy Feature related to V M R P is stronger in the 1-3 channels than that in the 4-6 channels. For a minority of the subjects (One out of five in this work), the Energy Feature does not apply to some of the signals in the 4 t h - 6 th channels. 23 Chapter 4 E V A L U A T I O N O F T H E M O D I F I E D L F - A S D To evaluate the impact of the Energy Normalization Transform (ENT) described in Chapter 3, two studies were conducted. The first study analyzed the impact of the ENT on the low frequency components of bipolar E E G (SNLFP in Figure 1.1b). The second study analyzed the impact of the ENT on the performance of the L F - A S D (SNFcin Figure L ib ) . In Section 4.2, the characteristics of the test data are discussed, which is the foundation for further discussions. The methodologies of evaluation for the two studies are provided in Section 4.3.1, which includes a method to quantify the separation between idle and V M R P E E G data, methods to quantify the performance of the (modified) L F - A S D , and a method used in this work to determine the optimal parameter values of the ENT. In addition to the two studies, a demonstration method, which shows that the ENT, a non-linear operation in the broadband (>=0 Hz), does not distort the features related to V M R P in the low frequency band (0-4Hz), is provided in section 4.3.2. 4.1 Objectives of the Studies 1. Evaluate the error rate of the modified LF-ASD. 2. Determine the impact of the Energy Normalization Transform on the low frequency components of bipolar EEG. 3. Determine the impact of the Energy Normalization Transform on the performance of the modified LF-ASD 4. Determine the characteristics of the Energy Normalization Transform: whether or not the normalization-transform distorts the features related to V M R P detection; the influence of the different W N (Normalization Window Size) values on the performance of the modified L F - A S D ; analysis about how the ENT improves the system performance. 5. Find and introduce new potential features of the voluntary movement related potential. 4.2 Test Data Characteristics Pre-recorded data from five right-handed able-bodied subjects were chosen for the two studies in this chapter. The detailed acquisition process is provided in Appendix B. Since it is much easier to measure the movement-related potential of an able-bodied subject than that of a subject with a disability, Voluntary Movement-Related Potential (VMRP) of able-bodied subjects was studied in this work as an initial phase. According to Decety and Boisson [34], 24 the E E G related to IVMRP recorded from people with spinal cord injuries (SCI) was not different from that of able-bodies subjects, while Bozorgzadeh [2;35] mentioned there were slight differences in averaged IVMRP between able-bodied subjects and subjects with SCI. After this initial exploration, which proves the fact that the signal processing approach can effectively help able-bodied subjects improve error rates, the application can be extended to subjects with a disability. This has been left for future studies. For each subject, his or her E E G was recorded in four runs in each session. In each run, subjects were instructed to trigger a finger switch by cues shown on a screen, while passive observation periods were recorded between the finger switch activations. For each run, approximately twenty periods of V M R P free of ocular artifact were collected. In addition, for each subject, two extra runs of E E G lasting two minutes were recorded in a period of passive observation. E E G section contaminated by Ocular Artifact was discarded. The data used in this study was from the same session for each subject. The impact of the ENT over different sessions was left for future studies. A l l the subjects in this work did not customize the system before the data was acquired. In a real application, in order to obtain better performance, a BCI system may get feedback from the user, and then this information is used to adapt the parameters of the BCI system components, such as the codebook in the Feature Extractor. This procedure is named customization. Since this work was an offline study and its purpose was to improve the performance of the L F - A S D with a signal processing approach instead of seeking help from human factors, customization is beyond its scope. Consequently in this work, by using generic L F - A S D parameters, which were derived from a non-normalized E E G , across all subjects, the relative performance improvement with and without the Energy Normalization Transform was the focus. In the future, customization may be applied to further improve error rates. The subject information is shown in Table 4.1. Table 4.1 The subject information P.P. K.T M.B M.P C B . Gender Female Female Male Female Male Age 54 31 39 35 57 4.2.1 Power Characteristics of the Bipolar EEG As shown in Table 3.1, a characteristic of the bipolar E E G is its power variation over time and across different subjects. Power variation results in unstable features in the Feature Extractor, which makes the Feature Classification problem more difficult and, hence, introduces higher error rates. Its impact to the LF-ASD is shown in Chapter 5. It has been shown in Chapter 3 that, compared with that of the idle data nearby, the power in the E E G signal during V M R P periods decreased. The significant power decrease in the high-frequency band accounts for this phenomenon. For details, please refer to Figure 3.2 (a) (b) (c). 25 4.2.2 Time Amplitude Characteristic of the Low Frequency E E G (0-4Hz) The ensemble average of the low frequency E E G (SLFpin Figure 1.1a) around VMRPs was derived from pre-recorded E E G by an offline study. The pre-recorded E E G signal was obtained as described in Appendix B. Then a rectangle observation window was imposed and centred at the onset of the finger switch, which gave a measurement of the V M R P , as shown in Figure 4.1. The centre of the observation window, which was marked by the onset of the finger switch, divided the observation window into front and rear. The data falling into the front window were named M l ; the data falling into the rear window were named M2; the data falling in between the observation windows were named Idle2. For comparison with Idle2, another E E G session was recorded when the subject was in a state of passive observation and this data set was named Idlel. E E G contaminated by Ocular Artifact was discarded. <TI> t . ; : .«~ . r v . < t on Window F i n g e r — — S w i t c h A c t i v i t i o n ISO 200 250 Figure 4.1 The definition of M l , M2 and Idle2 The ensemble averages of the low frequency data representing V M R P were produced by time locking and centering the signal to the onset of a finger switch with one second pro and post to the center respectively. For subject PP, the ensemble averages of the low frequency E E G from Channels 1-6 are shown in Figure 4.2. 26 Ensemble Average of the non-normalized signal for Subject P.P - 1 M1, . M2 n i Channel 1" Idle2 ^ i-Idle? 50 100 150 200; ;250 ...... , , ' , Channel 2" 1 1 ,. too 0 -100 100: -100: 50 100 150 200; 250 •§ 100 1 1 I r 1 — r r Channel 3" ' | -100 I I I — T T l ; Channel '4-I I 100 h 0 -100 50 100 150 200 250 I ! 1 ' Channel 5-: • • ^ -•• •. > —<• \ I I 100 0 -100 50 •i'oo .150 200 !250 100 ••,! J — |. I; "Channel 6 0 100 i i 50 100 '150 " Finger Switch Onset; 200: 250 Figure 4.2 Ensemble average of the low frequency E E G (Subject PP) centred at a finger switch with one second before and after the centred in Channels 1-6 This ensemble average of the low frequency signal recorded from FC1-C1 (Channel 4) is similar to that of C3+2-C3 reported by L.Deecke [26] (See Chapter 2). It also has a positive movement preparation section and a negative movement activation section. But the signal of F C i - C i has a stronger movement activation section, while the signal of C3+2-C3 has a stronger movement preparation section. By ensemble averaging of the low frequency E E G centred at the onset of V M R P , the following observations can be made: 1. Before the onset of the finger switch, the averaged curve of M l bends above zero and the mean amplitude values of M l are slightly positive. 2. After the onset of the finger switch, the averaged curve of M2 bends quickly and strongly negative and then gradually, it bends back towards zero. 3. In the idle period, Idle2 is approximately a zero-mean random process. The density distributions of the M l and M 2 in Channel 1, which was recorded from F i -F C i , are shown in Figure 4.3. 27 Amplitude Figure 4.3 The density distributions of M l vs. M2 for Subject PP in Channel 1 0.0451 (a) (b) Figure 4.4 (For Subject PP in Channel 1) a)The density distribution of M2 vs. Idle2 (Idlel); b)The density distribution of M l vs. Idle2 (Idlel) By observing the relationships of M l , M2 and Idle2, conclusions were drawn as follows: 1. In terms of density distribution, Idle2 was similar to Idlel. Therefore, from this point on, Idle2 is used to represent the passive observation data. 2. The density distributions of active data (M1/M2) showed separation from the idle data (Idlel/Idle2). However, the difference between means of active and idle data was not very large. This indicates the difficulty of differentiating the active from the idle trials. 3. Compared to the separation between M l and Idle2, M2 and Idle2 showed much larger separation. Therefore, from this point on in this work, the data separation between VMRP and the passive observation was represented by the separation between M2 and Idle2. 4. The mean of M2 was negative; the mean of Idle2 was approximately zero; the mean of M l was positive, but its absolute value was very small. 28 4.3 Evaluation Methodology In this section, methods to evaluate the impact of the Energy Normalization Transform and determine its characteristics are introduced. First, the impact of the Energy Normalization Transform is studied in two phases: the impact on the low frequency EEG (SNLFP in Figure 1.1b) and the impact on the performance of the LF-ASD (SN Fc in Figure 1.1b). Second, distortion caused by the ENT is discussed. In addition, a theoretical formula to calculate the error rate of the LF-ASD is provided. 4.3.1 Methods to Evaluate the Impact of the Energy Normalization Transform 4.3.1.1 Determination of the ENT Impact to the Output of the LPF To study the impact of the ENT on low frequency EEG, the characteristics of low frequency signal SNLPF, SELPF and SLPF, which were defined in Figure 4.5, were compared. ENTOPT is the energy normalization transform with optimal normalization window size. While "ENT with WN = the run length" applies the total sample number of the data in an EEG run as the normalization window size of the ENT. The rationale of this evaluation method is provided in the following sections. ENT C LPF ENT with WN= the run length LPF LPF Feature Extractor SNFE Feature Classifier Feature Extractor SEFE Feature Classifier Feature Extractor SfE Feature Classifier Figure 4.5 Evaluation of the impact of the Energy Normalization Transform As discussed in Chapter 1, the Energy Normalization Transform was postulated to have two major benefits to the low frequency EEG. First, it desensitizes the system to the variance of the input signal; second, it captures the Energy Feature related to VMRP, and then increases class separation. Another side benefit of ENT is that it can automatically adjust the mean scale of the input EEG to match the static codebook in the Feature Classifier better. ENT can decrease the signal variance of the low frequency EEG. This can be evaluated by comparing the standard deviation of SNLFP and SLPF, as shown in Figure 4.5. The signal with smaller deviation should produce more stable features in the Feature Extractor. Thus, it makes the system insensitive to input signal variance and improves system error rates. To prove that the ENT can increase class separation between the active and idle data by capturing the Energy Feature of VMRP, in Figure 4.5, the filtered EEG with normalization (SNLPF) was compared to the filtered EEG without normalization (SELPF)- The "ENT with WN = the run length" is applied to adjust the amplitude of EEG in each run. This results in the data amplitude in each EEG run ranging from -1 to 1 approximately. Thus, regardless the 29 mean amplitude difference, active (idle) data in a run are similar to active (idle) data in another runs after the "ENT with W N = the run length". By this means, data across runs could be combined together to compute the class separation by a statistic approach. "ENT with W N = the run length" was only used to analyze the characteristics of the non-normalized data. Since that WN (total sample number of data in a run) is too large, the operation can only adjust the scale of data in a run but it cannot affect the power relative during movement potential periods. Thus, to evaluate the benefit of increased class separation by capturing the Energy Feature related to V M R P , the signal SELPF, in Figure 4.5, can represent the low frequency E E G components without normalization, while SNLPF, in Figure 4.5, is applied to represent the low frequency E E G components with ENTOPT- However, using SELPF to represent non-normalized low frequency E E G , the comparison results between signal with and without normalization was conservative, because, in reality, the compensated signal (SELPF) has better quality than the original signal (SLPF). In order to compare the characteristics of SNLPF and SELPF and then determine class separation increase with the ENT by capturing the Energy Feature related to V M R P , it is necessary to quantify the class separation between the active data and the idle data. In this work, the separation between the two data sets is indicated by Difference Of Means (DOM), which is defined by Equation 4.1 below: DOM = [mean(active EEG) - mean(idle EEG)]I variability of the idle EEG (4.1) The D O M between M2 and Idle2, as defined in Figure 4.1, is named DOM22, and the D O M between M l and Idle2 is named DOM12. A larger D O M means better class separation between active and idle signal. As discussed in Section 4.2.2, M2 have larger difference from Idle2 than M l , thus DOM22 is much larger than DOM12. Therefore, DOM22 is applied to represent the class separation between active and idle E E G data in this work. "Variability of the idle E E G " in Equation 4.1 is defined by the range in which 98% of Idle2 data fall. The observation window size used to obtain M2 and Idle2 are described below. As described in Section 3.1.2, normalization window size (WN) is the only parameter involved in the ENT. But in order to obtain the optimal WN value, we need to get active (M2) and idle (Idle2) E E G data. Thus, like described in Figure 4.1, we need to know the optimal value of observation window size (W 0 ) . Wo is a parameter for evaluation, which determine the best data sets representing the idle and active EEG for analysis, (i.e. by using the finger switch as the measurement of movement, the parameter Wo defines how to separate M l , M2 and Idle2 from the E E G signal,) as shown in Figure 4.1; while WN is the normalization window size in the ENT, which is defined in Equation 3.1. During evaluation, the optimal combination of Wo and WN values should correspond to the best separation (DOM) between the V M R P and the idle data. In this work, by exhaustive searching through the combinations of WN and Wo, it was found that the optimal W N and Wo could be independent of each other. (The detailed information is provided in Figure 5.1 in Section 5.2.1.1.) Therefore, the optimal parameter values of WN and Wo were determined separately as follows: a) Method to Determine the Optimal Value of W 0 Since D O M is the indicator of separation between the idle and active class and the optimal value of WN and Wo are independent of each other (See Figure 5.1), the optimal Wo 30 can be obtained by exhaustive search for the maximal DOM while fixing Normalization Window Size (WN) at a specific value. (Separating data into an Onset of the finger switch a c t i v e set and an idle set) W N is fixed. ENT(WN) LPF -Idle2 Ml M2 Observation Window •*• Wo is a variable. •* Idle2 Computing the DOM between the idle and active class Figure 4.6 Procedure of the optimal Observation Window Size determination b) Method to Determine the Optimal Value of W N After the optimal Wo has been determined Wo was fixed at its optimal value and the optimal W N was determined by exhaustive search for the maximal DOM, as shown in Figure 4.7. Since very small Normalization Window Sizes may cause undesirable distortion to the original signal, results corresponding to the very small normalization window sizes were ignored. Onset of the finger (Separating the original signal into an active dataset and an idle data set) W N is a variable. ENT(WN) LPF "*"Idle2 Ml | M2 -r Observation Window (W 0is fixed at its optimal value.) Idle2 Computing the DOM between the idle and active data Figure 4.7 Procedure of the optimal Normalization Window Size determination With the optimal combination of W N and Wo, the DOM with the ENT can be achieved. In addition, corresponding to the optimal Wo, the maximal DOM for non-normalized data (SELPF) can be obtained as well. The difference between the maximal DOM with normalization and the DOM without normalization indicates the capability of the ENT to improve the data separation. 4.3.1.2 Determination of the ENT Impact on the LF-ASD Performance As discussed in Section 4.3.1.1, the ENT has two major benefits and one side benefit to the filtered EEG signal. The two major benefits are: first, it decreases the EEG scale variance and hence makes the features generated more stable; second, it captures the Energy Feature related to VMRP, which increases the class separation between the idle and active data. These two major benefits can result in system performance improvement. 31 SLNLPF Feature Extractor SLNFE Feature Qassifier SLNFC ENT with a large WN LPF w SlPF k SFF FC LPF Feature Extractor Feature Qassifier SNLPF w SNFE SNFC ENTnpr sN LPF Feature Extractor Feature Qassifier w w w w Figure 4.8 Evaluation of the E N T impact on the L F - A S D performance The model, shown in Figure 4.8, was designed to evaluate the L F - A S D performance improvement by the two factors. To evaluate the influence of scale/power variance of the input signal to the L F - A S D system performance, a large normalization window size was applied to the normalization filter shown as "ENT with a large W N " block in the model. This large normalization window cannot capture the Energy Feature related to V M R P , but it can decrease the scale variance of the input E E G . This large normalization window size was determined by observing a plot of D O M vs. WN with optimal Wo, as shown in Figure 5.2, and details of its derivation are provided in Section 5.2.1.2. The operation of "ENTOPT" can both decrease input scale variance and capture the Energy Feature related to V M R P . sFc, sLNFC and SNFC , which are defined in Figure 4.8, are output of the L F - A S D . sFC is the original non-normalized output; sLNFC represents the output of the L F - A S D with the input scale variance decreased; sNFC represents the output of the LF-ASD with both input scale variance decreased and Energy Feature related to V M R P captured. The difference between sLNFC and sFC shows how much the scale variance influences the system performance. The difference between sLNFC and S N F C shows how much the system performance can be improved by capturing the new Energy Feature related to V M R P . By this means, the performance improvements by the two factors of the ENT can be evaluated individually. The output of the L F - A S D has 2 states only, indicating the active pattern or the idle pattern. By comparing the BCI switch decision result with the control signal system performance evaluation can be obtained in terms of True Positive, False Positive, True Negative and False Negative. These definitions are illustrated in Figure 4.9. When BCI output = 1, it indicates the movement potential is detected. When the BCI output = 0, it indicates that idle state is detected. The finger switch activation indicates the real movement potential event, which was applied as a control to compare with the BCI decision. An observation window is applied around finger switch activation. Note: the size of the observation window in Figure 4.10 is 128 samples (total duration of one second), with one quarter-second pre finger switch activation and three quarter-seconds post finger switch activation. This division is based on the fact that M2 shows larger separation from Idle2 than M l . 32 Onset of the finger switch Onset of the finger switch BCIoutput=l True Negative I True Positive t False Positive False Negative j I I. -\-—r i • * I True Negative - 1 i i Observation Window Observation Window Figure 4.9 Definition of True Positive, True Negative, False Negative and False Positive If the output of the Feature Classifier is the active pattern and appears within the observation window, the LF-ASD is assumed to have made the correct decision; this event is named True Positive (TP). If the output of the Feature Classifier is the idle pattern and no output indicating the active pattern appears in the observation window, the LF-ASD is assumed to have made a wrong decision; this event is named False Negative (FN). If the output of the Feature Classifier is the idle pattern and no output indicating the active pattern appears beyond the observation window, the LF-ASD is assumed to have made a correct decision; this event is named True Negative (TN). If the output of the Feature Classifier is the active pattern and appears beyond the observation window, the LF-ASD is assumed to have made a wrong decision; this event is named False Positive (FP). In case of the LF-ASD, once the BCI switch is on, it will hold on for a period (hold on samples). The probabilities of the four events are named PTP, PFN, PTN and PFP respectively. PTP and PFP can be obtained by Equation 4.2. The evaluation is ocular artefact free. Number of TP events .. PTP = (4-2) Number of finger switch activations n Number of FP events p • 1 FP — Number of samples falling out of observation windows - hold on samples of FP Equation 4.3 shows the relationship of PTP, PFN, PTN and Ppp. PFN =1-PTP ' ' (4.3) PTN =1- PFP Thus, a pair of PTP and PFP could represent the system performance without redundancy. While a series of PTP and PFP values can represent the system performance on different conditions. By plotting PTp against PFP and connecting the dots corresponding to different pairs of PTP and PFP, a Receiver Operating Characteristic Curve (ROC Curve) [36] can be obtained to represent the system performance on different conditions. In order to obtain different PTP and PFP pairs, a scalar was applied on the codebook in the Feature Classifier. This scalar can change the relative distance of the feature set to the codebook representing the active or idle class, resulting in different corresponding PTP/PFP pairs (i.e. different points on the ROC curve). Every ROC Curve goes from (0,0) to (1,1). It shows the performance of the system on all conditions. In this evaluation, every W N value corresponded to a ROC Curve; different WN values correspond to different ROC Curve, indicating the system performance for various values of WN. 33 An example of ROC Curve is shown in Figure 4.10. False Positive, Figure 4.10 An example of ROC Curves In this work, two metrics were provided to quantify the difference between system output with and without normalization (sLNFC, sNPC and sFC ), as shown in Figure 4.8: PTp corresponding to a specific PFp and Area-under-ROC-Curve. The first is a point on the ROC Curve; the second is the area surrounded by a ROC Curve and axes. a) . PTP corresponding to a specific PFP An evaluation study of a subject's frustration level caused by BCI system error [37] showed that a user's frustration becomes quite prominent with a False Positive rate (PFP) between 1.7% and 4%. Over 4%, the users regard the system not useable. Therefore, in this work, for the purpose of practical application, when PFP value was at 0.2%, 0.4% and 1% respectively, the corresponding PTp values with and without the normalization operation were compared. Each of them corresponds to a point on the ROC Curve. b) . Area-under-ROC-Curve (PFP<=1%) Area-under-ROC-Curve is an indicator of the overall performance of a classifier. A larger area indicates a better performance. For example, in Figure 4.10, the chance performance of the classifier is represented by the diagonal ROC Curve, where the values of PTp and PFP are equal to each other. In this case, the Area-under-ROC-Curve is 0.5. In Figure 4.10, the best performance is represented by the right-angle ROC Curve, which starts at (0,0) goes through (0,1) and then terminates at (1,1). This indicates that Pjp value can be as high as 100% with the corresponding PFP value of 0%. In this case, Area-under-ROC-Curve is 1. As a performance indicator of a real application, Area-under-ROC-Curve always varies in the range of 0.5 to 1.0. Specifically, in this work, only the ROC-Curve section with small PFp values is interested. Therefore, only when Ppp <= 1%, Area-under-ROC-Curve was 34 calculated. By comparison of the Area-under-ROC-Curve values (PFP <= 1%), overall performance (PFP <= 1%) of the two systems can be compared. The theoretical error rate of the system depends on the probability of the active event or the probability of the idle event. If these values had been known, the error rate of the system could have been calculated easily by the formula below. P(.) represents probability of an event; P< A | B > represents the conditional probability of A given B in Equation 4.4. P(error) = P(VMRP | Idle) x P(Idle) + P(ldle | VMRP) x P(VMRP) However, in the real world, the probabilities of the idle event and the active event are not predictable. Therefore, the theoretical error rate of the system is not a practical metric to evaluate the system performance. 4.3.2 Effect of Energy Normalization Transform on the Features Related to VMRP According to Cochran's work [28], which has been discussed in Chapter 2, most normalization operations are non-linear. In the broadband, the proposed energy normalization transform, given in Equation 4.5, is a non-linear operation as well. S^(n) is input and Su(n) is output of ENT. However, because the LF-ASD extracts the features only from the information located in the low frequency band (0-4Hz), it is significant whether or not this transform distorts the features in the 0-4 Hz band. The distortion to the signal depends on the value of the Normalization Window Size (WN). Some cases are discussed below: 1. The size of the normalization window is 1. In this case, the ENT formula can be simplified as SN(n) can be either 1, when SIN(n) is positive, or - 1 , when SIN(n) is negative. Thus, the output values are normalized into 1 or -1 . This operation is not a linear operation in the 0-4Hz band, and it seriously distorts features related to V M R P . Therefore, ENT with a small normalization window size may cause distortion of the features used by the L F - A S D , which may hamper the system performance. 2. The size of the normalization window is very large. In this case, the formula for the ENT will be simplified as: (4.4) = PFP x P(ldle)+ (1 - PTP) x P(VMRP) (4.5) (4.6) SN(ri) = Energy per sampleof the data in the normalizdton window (4.7) 35 If the normalization window size is very large, the energy per sample of the data in the normalization window should be the same as the energy per sample of the data in that EEG run. Consequently, the denominator of Equation 4.7 would become a constant scalar, and SN(n) would become the product of SIN(n) with a constant. In this case, the density distributions of the signal with and without normalization would be identical except the scale. Therefore, the ENT with a very large W N cannot help to differentiate the VMRP and the idle class. 3. The normalization window size is properly chosen. To determine whether or not the ENT with the optimal WN value distorts the features used by the LF-ASD in the 0-4Hz band, in the frequency domain the responses of the ENT to different input classes are considered. There are three potential input classes: Idle, M l and M2. If the ENT does not distort both the phase and the magnitude spectrums of each class in the 0-4 Hz band, then it will not distort the features related to VMRP detection in the LF-ASD. 36 Chapter 5 R E S U L T S A N D D I S C U S S I O N 5.1 Overview According to the evaluation methodologies described in Chapter 4, Simulink blocks were implemented and the experimental results were determined. In this chapter, the experimental results are reported first followed by a discussion of the results. The results are organized into four areas: the optimal parameter values of the modified L F - A S D ; the two major benefits of the ENT to E E G components in the low frequency band; the enhanced performance of the modified L F - A S D measured by Area-under-ROC-Curve (PFP<=1%) and P T p corresponding to a specific Ppp; and a demonstration that the Energy Normalization Transform (ENT) does not distort the features related to V M R P detection in the LF-ASD. The discussion section shows that the ENT could increase the separation between the idle and the active E E G class and can desensitize the system to the input scale variance. As a result of these two benefits, this transform can improve the error rate of the L F - A S D . 5.2 Results 5.2.1 The Optimal Parameter Values 5.2.1.1 Determination of the Optimal Observation Window Size (W0) According to the procedure shown in Figure 4.6, by fixing W N (Normalization Window Size) at a given value and then changing Wo, a curve showing D O M vs. Wo can be achieved. Thus, four curves are shown in Figure 5.1, corresponding to WN at 21, 23, 51 and 101 respectively. In addition, a curve derived from the low frequency E E G without the normalization operation is shown in Figure 5.1. 37 Figure 5.1 Determination of the Optimal Wo for Channel 1 of Subject PP For Subject PP in Channel 1, Figure 5.1 shows: 1) Regardless of the Normalization Window Size, DOMs, the indicators of data separation, reach the maximal value unanimously at the same Wo- This shows that Wo determination is independent of that of WN. 2) A normalized signal with a reasonable normalization window size always has better separation than the corresponding non-normalized signal. The optimal observation window size (Wo) defines the best data set representing VMRP and the idle pattern in terms of DOM. Moreover, it has been shown that the optimal Wo determination can be independent of optimal W N . Therefore, in the case of Figure 5.1, the optimal Wo is 62. Table 5.1 lists the optimal Observation Window Size over the five subjects in Channels 1-6. Table 5.1 The optimal value of W 0 over five subjects in Channels 1-6 (in sample) PP KT MP MB CB Channel 1 62 30 54 46 58 Channel 2 62 30 54 48 58 Channel 3 62 30 54 48 55 Channel 4 70 30 80 30 58 Channel 5 70 30 88 50 60 Channel 6 80 30 88 55 70 5.2.1.2 Determination of the Optimal W N Since determination of optimal W N can be independent of W 0 , as in the procedure shown in Figure 4.7, by fixing Wo at the optimal value, which was determined in Section 5.2.1.1, and then exhaustively searching through W N , the optimal W N was achieved. Figure 5.2 38 shows an example Channel 1. of the impact of different W N values on D O M for Subject PP in Figure 5.2 Optimal WN determination for Channel 1 of Subject PP In Figure 5.2, the curve of D O M shows four phases following the change in W N : 1. The Oscillation Phase (W N is between 1 to 40.): In this phase, in terms of a single sample, the ENT increases the separation between the idle and active E E G class. However, ENT with a small WN would distort the features related to V M R P and would result in many False Positives. (For details, please refer to Section 4.3.2.) Therefore, the optimal WN was not selected from this phase. 2. Optimal Value Determination Phase (W N is between 40 and 80.): The normalization operation improves the separation between the idle and active data. At the same time, it causes little feature distortion. (For details, refer to the analysis in Section 4.3.2.) 3. Steep Falling Phase (W N is between 80 and 250.): Following the increase in normalization window size, D O M decreases steeply, indicating that the ENT is losing its ability to capture the Energy Feature of V M R P . 4. Flat Phase (W N is greater than 250.): When W N is larger than 250, the ENT has lost its ability to capture the Energy Feature related to V M R P . However, the separation between the active and idle data is still greater that that of non-normalized EEG. Following a further increase in W N , the curve gradually bends towards the best performance of the non-normalized signal (For details, please refer to Section 4.3.2.). In conclusion, the optimal WN should correspond to the maximal value of the curve in the Optimal Value Determination Phase. In the case shown in Figure 5.2, the optimal separation corresponds approximately to WN at 51. Table 5.2 shows the optimal WN values over the five subjects and over the six channels of the LF-ASD. 39 Table 5.2 The optimal W N for the five subjects over channels (in sample) PP KT MP MB CB Channel 1 51 55 40 55 51 Channel 2 47 51 40 40 55 Channel 3 51 51 40 51 55 Channel 4 51 51 48 49 51 Channel 5 51 51 51 40 55 Channel 6 55 51 47 51 47 5.2.2 Effect of the ENT on Low Frequency EEG As discussed in Chapter 4, the ENT has two major benefits to the low frequency EEG components: improved separation between the movement-related and non-movement-related EEG; decreased scale variance. 5.2.2.1 Improved Separation between the Idle and Active E E G In general, for all the subjects, this Energy Normalization Transform increased the separation between movement related and non-movement-related data. For example, Figure 5.3 shows that class separation in Subject PP's Channel 1 (Fi-FCi) increases with the ENT. After the ENT, the mean of the active data moved towards -1. Consequently, the DOM between active and idle data increased by 45%, from 0.340 to 0.493. Table 5.3 lists DOMs between the active and idle EEG class with and without the ENT over channels and over subjects. It is noted that for EEG in Channels 5-6 of subject CB, the proposed ENT did not increase data separation. The reason is provided in the discussion section. Without Normalization With Normalization 0 . 0 4 5 - ' 0 . 0 3 r V _ A m p l i t u d e A m p l i t u d e -(a) (b) Figure 5.3 Density distribution of signal amplitude for active and idle EEG data in Channel 1 for Subject PP a) without normalization, and b) with normalization Since the active data is composed of M l and M2, the impact of ENT on Difference Of Means between M l and Idle2 (DOM12) was also studied. However, experimental results showed that the ENT did not have an obvious impact on DOM12. For example, for subject 40 PP 's E E G in Channel 1, the D 0 M 1 2 with and without normalization were both 0.022, which was less than 10% of the corresponding D O M 2 2 , as indicated in Table 5.3. Therefore, in terms of optimal classification, the D O M 1 2 seems much less important than D O M 2 2 , and the E N T impact on D O M 1 2 is not shown in detail in this work. A l l in al l , the E N T did not deteriorate D O M 1 2 . Table 5.3 The separation between the active and idle data with and without the E N T Subject Channel DOM without Normalization DOM with Normalization 0.493 Percentage Increase Subject PP 1 0.340 45.0% 2 0.327 0.431 31.8% 3 0.350 0.483 38.0% 4 0.190 0.258 36.3% 5 0.160 0.196 22.5% 6 0.190 0.238 25.3% Subject KT 1 0.172 0.228 32.6% 2 0.148 0.194 31.1% 3 0.175 0.268 53.1% 4 0.171 0.227 32.8% 5 0.150 0.219 46.0% 6 0.252 0.343 0.230 36.1% 15.0% Subject MP 1 0.200 2 0.232 0.265 14.2% 3 0.114 0.150 31.6% 4 0.101 0.107 6.0% 5 0.052 0.052 0.0% 6 0.07 0.093 32.9% Subject MB 1 0.332 0.414 24.7% 2 0.328 0.440 34.2% 3 0.335 0.473 41.2% 4 0.210 0.252 20.0% 5 0.152 0.187 23.0% 6 0.147 0.179 21.8% Subject CB 1 0.319 0.405 26.9% 2 0.358 0.432 20.7% 3 0.240 0.320 33.3% 4 0.421 0.453 7.6% 5 0.504 0.492 -2.3% 6 0.410 0.396 -3.4% 5.2.2.2 Decreased Scale Variation of the Low Frequency E E G For E E G in the first channel for Subject P P , the standard deviation of the low frequency E E G without normalization is 1.902, while the standard deviation of the normalized low frequency E E G is 1.30. The standard deviations of the low frequency E E G with and without normalization over the five subjects in the first channel are shown in Table 5.4. 41 Table 5.4 Standard deviations of the low frequency E E G with and without normalization over subjects Subject P P Subject K T Subject C B Subject M P Subject M B Without E N T 1.30 1.23 1.41 1.21 1.31 With E N T 1.91 1.82 2.10 1.85 1.93 In conclusion, the E N T can decrease the amplitude variance of the signal. In addition, the E N T can also automatically adjust the mean input amplitude, which, as indicated earlier, is its side benefit. 5.2.3 The Effect of the ENT on the LF-ASD Performance The effect of the E N T on the L F - A S D performance was evaluated by PTP corresponding to a specific P Fp and Area-under-ROC Curve (<=1%). (Since in the real world people are interested in system performance at small PFP only, in this work the Area-under-ROC-Curve with corresponding PFP values only no greater than 1% was calculated.) Two factors resulted in the performance improvement of the L F - A S D : captured Energy Feature related to V M R P and decreased scale variance. As discussed in Section 5.2.1.2, when the W N is larger than 250 (in the Steep Falling Phase), the E N T can no longer capture the Energy Feature related to V M R P ; however, it can help to decrease scale variance. Therefore, in the evaluation design as shown in Figure 4.8, the E N T with W N value of 251 was applied as the " E N T with a large W N " to decrease scale variance of the input signal only, while the E N T with the optimal W N value was supposed to be capable of both capturing the Energy Feature related to V M R P and decreasing the scale variance. Thus, the effect of the Energy Feature related to V M R P capture and scale variance decrease can be approximately separated. 5.2.3.1 Evaluation Based on the Area-Under-ROC-Curve (PFP<=1 %) Figure 5.4 shows an example of R O C Curves (in interested section) with and without normalization across different W N values. It shows that Area-under-ROC-Curve (Ppp<=l%) of the modified L F - A S D is larger than that of the original L F - A S D . The impact of the E N T for other subjects, in terms of Area under R O C Curve (PFp<=l%), is provided in Table 5.5. R O C Curves with and without normalization for other subjects are provided in Appendix C. It is noted that, although E N T improved the performance for the other four subjects, it did not improve performance for subject C B . The reason for it is provided in discussion part. 42 ROC Curve (Part) of the LF-ASD with and without ENT for Subject PP Without ENT 15 Figure 5.4 The ROC Curves (PFp<=l%) of Subject PP with different W N values and the corresponding ROC Curve without normalization 5.2.3.2 Evaluation Based on P T P Corresponding to a Specific P F P Table 5.6 lists the performance of the LF-ASD with and without normalization in terms of Pjp Corresponding to a Specific P F P , across five subjects. 43 CB w 1 H Z P 3 oo c ,CT ct' o | 18.7% oo bo VO VO # VO 4^ 8P z 3 § s-^ I oo IO VO O 33 z =/ II sr to U i # to # bo # to U l *d -n - D II 23.4% to 00 is) to to oo oo O ^ 4.7% VO # to to vo # 3 ° 3 2 £ S. 66.4% U i O 4*. # 4^ 4^ 4^ # 4^ O N VO # U i z 3 § 3-a ° • c r-• U i VO O N to 8P oo U l 4^ VO # U i O N li to =. U i ^ O N O U i U i 4^ 00 O N - J U l to n o 4*. # 4^ # # 4^ # <3 £• -5 2 3 3-e> -6.3% b # to o U i # O N # g ° 2 < 3 £5 g EL ft — VO o vo VO oo to O N O N Z * U l & # ° ~ 3 g 00 VO —1 VO 00 to 00 to vb # vo # II —^ to a. U l ~ II 00 oo 00 oo oo VO o 00 U l O ,^ bo b # 4* b z -1.8% I oo U l # oo # oo vo 3 2 . K E. » c r ST tn bs T l CD O 0) 3 •a 3 < CD 3 CD 3 ? 3 -3 o N &5 O 3 3 CD H •o o o cl cn O 3 a 5' era 3" P to T3 CD O Bi o ' T l 3 O N — 00 Z c P CT 3 o-o < ?o > 3 I- O 3 EL o 9 3 - 3 - 3 o ct. § s g. I- O 3 ^ 3- n ^ - ^ n O < C6 a . < Ct P p "a 3 V 5 C 5 ft O cn O S» 2 S " — C L Ct p 5 tc 3 O Ct =• O CD g O =r O » c "S. ^ n cj " 3 2 ^ 0 . p o < ct ct 7 o p S B ' S Ct 3 c » o 3 . 0 p T3 C ~" ct < § a. ft a rr 3 ^ 5^ ft 00 ^ 3 ct T 3 Ct n ct 3 00 ct ct S 3 K p T3 Ct A ft H tn In H 3" CD O < CD CD 3 O CD 3 •a i-i O < CD 3 CD 3 3" CD < CD cn 3 CD' o 3 CD > CD &3 I c 3 a CD * i 1 o o I n c < CD A 5.2.4 Demonstration that the ENT Does Not Distort the Features Used in the LF-ASD As mentioned in Chapter 4, in the 0-4 Hz band, if the ENT does not distort the magnitude and phase spectrums of M l , M2 and Idle2, it will not distort the features related to VMRP that are used in the LF-ASD. The frequency spectrums of M l , M2 and Idle2 pre and post the ENT are compared in Figure 5.5 a and b where W N is optimal. W i t h o u t N o r m a l i z a t i o n W i t h N o r m a l i z a t i o n -50 0 50 Frequency in Hz (a) Without Normalization <D 1 tn re o. 0 -1 4 2 So -4 Ml -50 50 M2 -50 50 o 1 0. 0 I Idle2 I 2 .0) 1: W re Q. 0 4 CD 0 -4 2 0) 1 Ui ro £ 0 -50 0 50 Frequency in Hz . With Normalization M1 -50 50 M2 -50 50 Idle2 -50 0 50 Frequency in Hz -50 0 50 Frequency in Hz (b) Figure 5.5 Comparison of spectral properties between EEG with and without ENT (WN=51) for Subject PP, a) Magnitude spectra of M l and M2 activity relative to Idle2, and b) Phase spectra of M l , M2 and Idle2 Figure 5.5 shows that the ENT did not distort the phase and magnitude spectrums in the 0-4Hz band. Thus, it did not distort the features used by the LF-ASD 45 5.2 Discussion 5.3.1 Results Discussion 5.3.1.lOptimal Signal Processing Parameter Determination According to Figure 5.2 and Table 5.2, the optimal observation window size is independent of the normalization window size; the optimal Normalization Window Size is approximately 51 over different channels across the five subjects studied. Further studies are required to see if this observation holds over a larger population. 5.3.1.2 Impact of ENT on the Low Frequency E E G Components The ENT had two major contributions to the low frequency EEG: first, it decreased the low frequency EEG scale variance; second, it increased the separation between active and the idle EEG class. In addition, a side benefit of the normalization transform is that it can adjust the input scale automatically. 1 Decreased scale variance: The Energy Normalization Transform helped to decrease the standard deviation of low frequency EEG amplitude. This shows that this normalization can desensitize the system to input amplitude variation. The decreased scale variance resulted in more stable features in the Feature Extractor, which could match the static codebook better in the Feature Classifier. 2 Increased separation between the active and idle EEG class: When the normalization window size was at the optimal value, the ENT can best increase class separation by capturing the Energy Feature related to VMRP. The averaged separation increase over channels with normalization across all subjects is shown in Table 5.7. Table 5.7 Averaged separation increase over channels with the ENT across subjects Channel 1 2 3 4 5 6 DOM 28.8% 26.4% 39.4% 20.5% 17.8% 22.5% Improvement According to the results in Table 5.7, with the ENT, the separation improvement in the first three channels was more pronounced than that in the last three channels. Referring to the details shown in Table 5.3, data separation in Channels 1-3 across all subjects increased consistently with the proposed ENT. The same was true for Channels 4-6, for all subjects except Subject CB. The reason why the ENT did not increase the class separation for Subject CB is discussed in detail in the next section. In conclusion, the ENT can increase the separation between the active and idle EEG class. And it is expected to improve the performance of the LF-ASD. But there is variation. 5.3.1.3 The Impact of ENT on Performance of the LF-ASD The performance improvement was caused by two factors of ENT: decreased input scale variance and the Energy Feature related to VMRP. For 4 out of 5 subjects, with the corresponding False Positive rate at 1%, the proposed normalization transform increased the True Positive rate by 7.7%, 8.3%, 8.5% and 18.9%. Thus, the overall performance of the LF-46 ASD for these subjects was improved from 66.1%-82.7% to 85.0%-90.4%. For the fifth subject (Subject CB), who had the highest non-normalized accuracy of 90.5%, the performance remained at 90% with normalization. Average Power of data in Observation Window centered at MP vs. the window size for Subject CB 8:5 r 7.5 40 80 Channel 2 ,60 180 200 Channel 3 140 180 200 : Channel 4 180 200 : Channel 5-8.5 % 8:5 r-180 200 Channel 6 "20 40 60 80 100 120 140 160 180 200 Size of Observation Window (Wo) Centred at the F|nger Switch (In Sample Numbers) Figure 5.6 (For Subject CB in the broad band,) Average power of data falling in the Observation Window centered at finger switch activations vs. observation window size (Wo) Since the ENT did not improve the error rate for Subject CB (except at PFp=0.1%), the characteristics of his EEG were studied and shown in Figure 5.6. It is noted: 1. In Channels 1-3, the Energy Feature related to VMRP existed. However, compared with other subjects (Refer to Appendix C), the Energy Feature was less strong. 2. In Channels 4-6, EEG signal power around VMRPs is even stronger than (or as strong as) that the idle periods. This is not consistent with other subjects' EEG. Further details (in Figure C l show in Appendix C) show that, for Channels 4-6 of Subject CB, although the power in the high frequency band decreased, the power in the low frequency band also increased. This resulted in the signal power in broadband (>=0Hz) increase around VMRP. As a consequence, the proposed ENT cannot help to increase the data separation. As a solution, an alternate normalization scheme was proposed and tested. The ENT with W N at 251 was applied to Channels 4-6, while the data in Channels 1-3 was normalized with W N at 51 to capture the Energy Feature. For this subject, the performances of LF-ASD with different normalization schemes are compared in Figure 5.7. For Subject CB, results in Table 5.8 compare the performance of different normalization schemes. 47 ROC Curves (Part) of the LF-ASD with and without the ENT for Subject C B Figure 5 . 7 The ROC Curves of Subject CB with different normalization schemes and the corresponding ROC Curve without normalization Table 5 .8 Performance of LF-ASD with different normalization schemes for Subject CB Area under ROCC (PFP<=1%) P T P with PFP=0.1% P T P with PFP=0.4% P T P with PFP=1% Non-Normalized 0.064 18.7% 66.4% 90.5% Original Scheme 0.061 23.4% 60.1% 88.7% Alternative Scheme 0.0625 21.7% 61.1% 90.9% Relative to other subjects, Subject CB's ROC Curves with and without normalization, as shown in Figure 5.7, are very close to each other. The weak Energy Feature in his E E G signal accounts for this. With the corresponding PFP = 1 % , for Subject C B , the modified LF-ASD with the alternative normalization scheme improved PTP by 0.4%. Thus with different normalization schemes, PTP improved from 6 6 . 1 % - 9 0 . 5 % to 85 .0%-9 0 . 9 % with corresponding PFP = 1 % over the five subjects. In terms of Area-under-ROC-Curve ( P F P = 1%), the modified L F - A S D with ENT does not seem better than the system without ENT (0 .00625 vs. 0 .00640) . This result may be due to the following reasons: 48 1. The shape of the non-normalized codebook is slightly different from that of the normalized codebook. This in fact may help the system without normalization perform better. If the Energy Feature in the signal were strong, this slight influence could be negligible; but if the Energy Feature is not strong, as in this case, the influence from this factor could be noticed. If the codebook had been changed into a normalized one, with the increased data separation in Channels 1-3, which is indicated by D O M , the new normalization scheme should have achieved better performance. 2. The parameters in the Feature Extractor are optimal to non-normalized data, and that is also in the favour of the system without normalization. 3. In this offline evaluation, it is possible that the amount of data is not enough, in which case the performance of the original system was exaggerated. It is somewhat suspect that Subject CB achieved a PTP with corresponding PFP at 1% higher than 90% without normalization. According to the previous evaluation work, the original L F - A S D has not achieved this type of performance. Typically, PTp without normalization should be less than 80% with the corresponding PFP at 1% [35]. In conclusion, if more data were collected from this subject, the set of codebook and other parameters of the system were obtained from normalized data, the improved system with the optimal normalization scheme should have better performance than the system without normalization. In the future, several approaches can be applied to further improve the system performance. The Feature Classifier in this work was not customized to the subjects data in this work. Instead, a standard (generalized) codebook and parameters were used, which were derived from a set of non-normalized data from a previous subject not used in this study. The system performance is expected further improved by customization and applying the codebook generated from the normalized EEG. In addition, the shape of the current normalization window is a rectangle window and it may not be. optimal. Other window shapes may be more suitable to capture the Energy Feature related to V M R P , and achieve better performance. 5.3.2 Potential Features Related to VMRP in the Magnitude and Phase Spectrum There were four classes of E E G data used in this study. These were Idlel, Idle2, M l and M2. The phase and magnitude spectrums of these signal classes, which were acquired by the procedure introduced in Section 4.2.2, are provided in Figure 5.8. 49 Magnitude Spectrum Phase Spectrum 0 50 Frequency in Hz Frequency in Hz Figure 5.8 Magnitude and phase spectrums of Idlel, Idle2, M l and M2 for Subject K T in Channel 1 1. Features in the magnitude spectrum: The features related to V M R P in the magnitude spectrum have been studied thoroughly by a few researchers over the last five decades. Jasper and Penfield [9] observed the mu rhythm (8-12 Hz) decrease during the movement potential periods; Pfurtscheller et al. [8] observed beta rhythm (18-26 Hz) decrease during movement potential periods. Researchers named these observations Event Related Desynchronization (ERD). However, according to Jasper and Penfield [9], some subjects may not have distinctive E R D . For example, Jasper and Penfield [9] reported 3 in 9 subjects in their study did not have mu rhythm in idle periods. In this study, magnitude spectrums, as shown in Figure 5.8, were derived from E E G in Channel 1 of subject KT . Observations from this data were: a) . Idle2 has strong mu and beta rhythm. Therefore, Idle2 can best represent the pattern of idle E E G . b) . M2 does not have mu and beta rhythm. Therefore, M2 can best represent the pattern of V M R P . c) . Although Idlel is idle E E G , it does not have mu rhythm but beta rhythm. d) . Although M l is active EEG, it has both mu and beta rhythm. In conclusion, the features related to V M R P , mu and beta rhythm in the magnitude spectrum, did not seem to be distinctive for differentiating the active or idle class. 2. Features in the phase spectrum 50 Previous BCI researchers appear to have ignored the study of the features related to V M R P in the phase spectrum. Most likely, this is due to the fact that most BCI researchers are neural scientists or psychologists. Therefore they do not have a DSP background. In this work, the phase spectrum related to V M R P was studied. The results are shown in Figure 5.8 and observations are made as follows: a) The phase spectrums of Idlel and Idle2 seemed to be very similar in the broadband. The phases of both Idlel and Idle2 have an amplitude spike value of 1.5 at 0 Hz, and oscillate slightly within the range between -0.2 and 0.2 when the frequency is larger than 3 Hz. b) The phase spectrum of M2 is distinctively different from the other classes. It has an amplitude of 3.14 at 0 Hz, and then falls to -1 at 4.5 Hz; the second peak appears with the amplitude of +1 at 7.5 Hz. Between 10 and 55 Hz, the curve oscillates within the range of -0.5 to +0.5. Between 55 Hz and 64 Hz, the curve oscillates within the range of -1 to +1. c) The phase spectrum of M l is unique among the four patterns as well. It has an amplitude of +1 at 0 Hz, which oscillates greatly between 0 and 8 Hz: Between 10 and 50 Hz, the curve oscillates slightly within the range of -0.5 to +0.5. Between 50 Hz and 64 Hz, the curve oscillates strongly within the range of -1.2 to +1.2. The phase of M l seemed to be different from that of Idlel (or Idle2) in the 0-8Hz band, while the phase of M2 is different from that of Idlel (or Idle2) in the 50-64Hz band. It is first found that information in the 50-64 Hz band is useful for V M R P detection. In conclusion, since the phases of Idlel and Idle2 are very similar and the phases of either M l or M2 have distinctive differences from that of Idlel (or Idle2), information in the phase spectrum is sufficient for the V M R P detection. To date E E G related to V M P R for subject PP and K T were analyzed in the phase spectrum, and their spectrums are very similar (Figure 5.5 and 5.8). However, it is too ambitious to draw conclusions based on only two subjects' EEG. In the near future, it would be worthwhile to analyze the phase spectrum for more subjects. 51 C H A P T E R 6 C O N C L U S I O N S The evaluation with data from five able-bodied subjects indicates that the proposed system with Energy Normalization Transform (ENT) has better performance than the original. The ENT separates signal classes better, decreases input variance to the Feature Extractor, decreases overall error rates, and makes the overall design less sensitive to variations in input scale. This study has verified the original hypotheses that energy normalization would increase separation between signal classes leading to a decrease in the error rate of the L F - A S D . Study results showed that the proposed transform had two major benefits to the system performance. First, the proposed transform increased the difference between the active and idle signal classes. It can increase the class separation in the range from 17.8% to 39.4% in channels 1-6. Second, it desensitized the system performance to the input signal amplitude variance. The filtered signal variance with ENT decreased from around 1.9 to 1.3. These two improvements (wider separation and more stable feature sets in the Feature Extractor, which could match the static codebook in the Feature Classifier better) resulted in decreased overall error rates. For 4 out of 5 subjects, with the corresponding False Positive rate (PFP) at 1%, the proposed transform increased the system performance by 7.7%, 8.3%, 8.5% and 18.9% respectively in terms of True Positive rate (PTP). Thus, the overall performance of the L F -ASD for these subjects was improved from 66.1%-82.7% to 85.0%-90.4%. For the fifth subject (Subject CB), who had the highest non-normalized accuracy of 90.5%, the performance did not change notably with normalization. In the future with the codebook derived from the normalized data, the performance could be further improved. As a side benefit, the ENT can also make design less sensitive to the input scale. In the broad band, the Energy Normalization Transform is a non-linear transform. However, for bipolar E E G signals used by the L F - A S D , although the Energy Normalization Transform distorted the phase spectrum in the high frequency band, it had no visible distortion to the features related to V M R P located in the 0-4 Hz band. In conclusion, the proposed transform does not cause distortion to signal features in a specific frequency band used by the L F - A S D . Study results of idle/active data characteristics showed that, in general, signal power in channels 1-6 decreased during periods around movement potential. This conclusion is consistent with the observations discussed in Section 3.1.3 [7-9], but some variation was noticed. The signal power in the front electrodes (Channels 1-3) had a stronger decrease than that in the back electrodes (Channels 4-6). For example, for all the subjects in the study, average power of E E G signals in Channels 1-3 decreased consistently by 5.8%- 41.7%. While in Channels 4-6, for four out of five subjects in the study, E E G average energy decreased by 3.1%-29.1%, but for the other subject (Subject CB) his E E G signal power in Channel 4-6 did not drop around V M R P . 52 Study results also show that the signal power decrease happened approximately 0.75 second before the finger switch activation until 0.75 seconds after the finger switch activation. 6.1 Summary of Contributions The main contributions of this work are: 1. A design of a new signal processing method, the Energy Normalization Transform, that improves the LF-ASD system performance and does not distort the features related to V M R P . 2. A methodology for the optimal parameters of the filters in the Energy Normalization Transform. 3. A Simulink module that implements the Energy Normalization Transform and a Simulink system that implements the modified LF-ASD. 4. Two studies that evaluated the effect of the Energy Normalization Transform on the filtered E E G data and on the performance of the L F - A S D . 5. Enhanced understanding of the energy drop in E E G during movement periods. 6.2 Suggested Future Work Suggestions for modifying the proposed system and evaluation methodologies are: 1. Evaluate the performance of the modified L F - A S D on-line. 2. Evaluate the performance of the modified L F - A S D on a larger pool of subjects, including those with spinal-cord injuries. 3. Explore additional shapes of the normalization window in the ENT, which might fit the characteristics of the E E G related to movement potential better than rectangle window. 4. Study the effect of codebook customization on the modified LF-ASD, and determine the relationship between the effect of the Energy Normalization Transform and the effect of codebook customization. It is expected that codebook customization could further improve the performance of the modified LF-ASD. 5. Customize the normalization scheme to the users. (Refer to Chapter 5 for discussion of the normalization schemes.) It has been noted that E E G characteristics of Subject CB are different from the other four subjects in this study. Consequently, the alternative scheme of Energy Normalization Transform is more suitable for Subject CB. A future user of the modified L F - A S D could choose a normalization scheme, which works best for him or her. 6. Study the potential features observed in the phase spectrum. In Chapter 5, potential features related to V M R P detection in the phase spectrum were observed. They should be tested against a large amount of data and an increased number of subjects. 7. In pursuit of a multi-function switch, combination of the O P M (See Section 2.1.2.7) and the LF-ASD may be pursued because of the potential of the O P M to 53 differentiate between different types of movement, such as left hand, right hand and foot movement. 54 R E F E R E N C E S [I] Wolpaw, J. R., Birbaumer, N . , Heetderks, W. J., McFarland, D. , Peckham, P., Schalk, G., Donchin, E., Quatrano, L . A . , Robinson, C. J., and Vaughan, T. M . , "Brain-Computer Interface Technology: A Review of the First International Meeting," IEEE Trans Reh Eng, vol. 8, no. 2, pp. 164-173, 2000. [2] Mason, S. G., Bozorgzadeh, Z., and Birch, G. E. "The L F - A S D Brain-Computer Interface: On-line identification of imagined finger flexions in subjects with spinal cord injuries." Proceedings of ASSETS 2000. 2000. A C M . [3] Mason, S. G. and Birch, G. E., " A Brian-Controlled Switch for Asynchronous Control Applications," IEEE Trans Biomed Eng, vol. 47, no. 10, pp. 1297-1307, 2000. [4] Birch, G. E. and Mason, S. G., "Current Trend in Brain-Computer Interface Research at the Neil Squire Foundation," Transaction of Neural Science and Rehabilitation Engineering, 2002. [5] Birch, G. E., Bozorgzadeh, Z., and Mason, S. G., "Initial on-Line Evaluation of the LF-ASD Brain-Computer INterface with Abie-Bodied and Spinal-Cord Subjects Using Imagined Voluntary Motoe Potentials," Transaction of Neural Science and Rehabilitation Engineering, 2002. [6] Personal Communication with G.E.Birch and S.G.Mason. 2002. [7] Mason, S. G., "Finger Movement Detection in E E G " PhD Thesis, UBC, Jan. 1997. [8] Pfurtscheller, G. , Neuper, C , and Flotzinger, D., "EEG-based discrimination between imagination of right and left hand movement," Electroencephalography and Clinical Neurophysiology, vol. 103 pp. 642-651, 1997. [9] Jasper, H . and Penfield, W., "Electrocortiograms in man: Effect of voluntary movement upon the electrical activity of the precentral gyrus," Arch.Psychiat.Nervenkr., vol. 183 pp. 163-174, 1949. [10] Mason, S. G. and Birch, G. E., " A General Framework for Describing Brain-Computer Interface Design and Evaluation," IEEE Trans Reh Eng, (accepted) [II] WolPaw, J. R., Birbaumer, N . , McFarland, D. J., Pfurtscheller, G. , and Vaughan, T. M . , "Brain Computer Interfaces for Communication and Control," Clinical Neurophysiology, vol. 113, no. 2002, pp. 767-791, Jan.2002. 55 [12] Wolpaw, J. R., McFarland, D. , and Vaughan, T. M . , "Brain-Computer Interface Research at the Wadsworth Center," IEEE Trans Reh Eng, vol. 8, no. 2, pp. 222-226, June2000. [13] Pfurtscheller, G. and Aranibar, A. , "Evaluation of the event-related desynchronization (ERD) preceding and following voluntary self-paced movement," Electroencephalography and Clinical Neurophysiology, vol. 46 pp. 138-146, 1979. [14] McFarland, D., Sarnacki, W. A. , Vaughan, T. M . , and Wolpaw, J. R., "EEG Based Brain Computer Interface Communication Effect of Target Number and Trail Length on Information Transfer Rate," Soc Neurosci Abstr 2000, vol. 26 pp. 1228, 2000. [15] Wolpaw, J. R., McFarland, D. , and Pfurtscheller, G., "EEG-based Communication: Improved Accuracy by Reponse Verification," IEEE Trans Reh Eng, vol. 6, no. 3, pp. 326-333,1998. [16] Kalcher, J., Flotzinger, D. , Pfurtscheller, G., Golly, S., and Neuper, C. Graz Brain-Computer Interface II: Studies for an EEG-Based Communication Device. 1996. Vancouver, Canada, Proceedings of the RESNA '95 Annual Conference. [17] Pfurtscheller, G. , Neuper, C , Schlogl, A . , and Lugger, K. , "Separability of E E G Signals Recorded During Right and Left Motor Imagery Using Adaptive Autoregressive Parameters," IEEE Trans Reh Eng, vol. 6, no. 3, pp. 316-325, 1998. [18] Pfurtscheller, G. , Neuper, C , Guger, C , Harkam, W., Ramoser, FL, Schlogl, A. , Obermaier, B., and Pregenzer, M . , "Current Trends in Graz Brain-Computer Interface (BCI) Research," IEEE Trans Reh Eng, vol. 8, no. 2, pp. 216-219, June2000. [19] Sutter, E. E., "The brain response interface: communication through visually-induced electrical brain responses," J.Micro comp.App., vol. 15 pp. 31-45, 1992. [20] Polich, J., "P300 in Clinical Applications," Electroencephalography: Basic Principles, Clinical Applications and Related Fields, pp. 1073-1091, 1999. [21] Farwell, L . A. and Donchin, E., "Talking off the top of your head: towards a mental prosthesis utilizing event-related brain potentials," Electroencephalography and Clinical Neurophysiology , vol. 80 pp. 510-523, 1988. [22] Kuebler, A . , Kotchoubey, B., Hinterberger, T., Ghanayim, N . , Perelmouter, J., Schauer, M . , Fritsch, C , Taub, E., and Birbaumer, N . , "The thought translation device: a neurophysiological approach to communication in total motor paralysis," Experimental Brain Research, vol. 124, no. 4, pp. 223-232, 1999. [23] Birbaumer, N . , Kuebler, A. , Ghanayim, N . , Flinterberger, T., Perelmouter, J., Kaiser, J., Iversen, I., Kotchoubey, B., Neumann, N . , and Flor, H . , "The Thought Translation Device (TTD) for Completely Paralysed Patients," IEEE Trans Reh Eng, vol. 8, no. 2, pp. 190-193, June2000. 56 [24] Levine, S. P., Huggins, J. E., Bement, S. L . , Kushwaha, R., Schuh, L . A . , Rohde, M . M . , Passaro, E. A . , Ross, D. A . , Elisevich, K. V. , and Smith, R. J., " A Direct Brain Interface Based on Event-Related Potentials," IEEE Trans Reh Eng, vol. 8, no. 2, pp. 180-185, June2000. [25] Birch, G. E., "Single Trial E E G Signal Analysis using Outlier Information." Ph.D. The University of British Columbia, 1988. [26] Deecke, L . and Kornhuber, H . , "Comparison of Bereitschaftspotential, pre-motion positivity and Motor potential preceding voluntary flexion and extension movements in man," in Kornhuber, H . H . and Deecke, L . (eds.) Progress in Brain Research Volume 54 - Motivation, Motor and sensory processes of the brain: Electrical potentials, behaviour and clinical use Amsterdam: Elsevier/North-Holland Biomedical Press, 1980, pp. 171-176. [27] McEwen, J. A . and Anderson, G. B., "Modeling the Stationarity and Gaussianity of Spontaeous Electroencephographic Activity," IEEE Trans Biomed Eng, vol. BME-22, no. 5, pp. 361-369, 1975. [28] Cochran, D. , " A Consequence of Signal Normalization in Spectrum Analysis," Acoustics, Speech, and Signal Processing, 1988.ICASSP-88, pp. 2388-2391, 1988. [29] Lee, S. W., Park, J. S., and Tang, Y. Y. , "Performance Evaluation of Nonlinear Shape Normalization Methods for the Large-Set Handwriting Chracters," IEEE Signal Processing, pp. 402-405, 1993. [30] Boubet, M . , "Performance of Normalized Match Filter," IEEE Signal Processing, pp. 2704-2707, 1989. [31] Chu, C. -H. H . and Delp, E. J., "Impulsive noise suppression and background normalization of electrocardiogram signals using morphological operators ," Biomedical Engineering, IEEE Transactions, vol. 36 pp. 262-273, 1989. [32] Chiarenza, G. A . , Hari, R., Karhu, J. J., and Tessore, S., "Brain Activity associated with skilled finger movements: Multichannel magnetic recordings," Brain Topography, vol. 3, no. 4, pp. 433-439, 1991. [33] Dumermuth, G. and Molinari, L . , "Spectral analysis of E E G background activity," in Gevins, A . S. and Remond, A . (eds.) Handbook of Electroencephalography and clinical neurophysiology: Methods of analysis of brain electrical and magnetic signals New York: Elsevier, 1987. [34] Decety, J. and Boission, D. , "Effect of brain and spinal cord injuries on motor imaginary," Eur Arch Psychiatry Clin Neurosci, vol. 240 pp. 39-43, Jan. 1990. 57 [35] Bozorgzadeh, Z., Birch, G. E., and Mason, S. G. The L F - A S D BCI: On-line Identification of Imagined Finger Movements in Spontaneous E E G with Abie-Bodied Subjects. 2000. Istanbul, Proceedings of ICASSP 2000. [36] Green, D. M . and Swets, J. A. , Signal Detection Theory and Psychophysics New York: John Wiley and Sons, Inc, 1966. [37] Forrest, G. , Zikov, T., and Yu, Z., "Evaluation of an Asynchronous Switch Application using Frustration Metrics for potential use in Biological Driven Switches (especially EEG)," Procedding of Human Computer Interface (UBC), 2001. 58 Appendix A T A B L E S A . l Glossary of Abbreviations S I N > S L P F . S F E , Spc Signals of the original LF-ASD defined in Figure 1.1a S I N > S N . S N L P F > S N F E . S N F C Signals of the modified LF-ASD with optimal window size defined in Figure 1.1b S I N > S E N > S E L P F > S E F E > S E F C Signals of the modified LF-ASD with W N = the length of the run defined in Figure 4.5 S I N > S L N . S L N L P F . S L N F E > S L N F C Signals of the modified LF-ASD with W N = 250 defined in Figure 4.8 59 Symbol Expanded Term AID Analog to Digital ALS Amyotropic Lateral Sclerosis Area under ROC Curve Area of region confined by the Receiver Operating Characteristic Curve BCI Brain-Computer Interface CRT Cathodic Ray Tube EcoG Electrocorticogram EEG Electroencephalograph EMG Electromyograph ENT Energy Normalization Transform ENTOPT Energy Normalization Transform with optimal normalization window size EOG Electro-oc ulograph ERD Event-Related Desynchronization fMRI Functional Magnetic Resonance Imaging FN False Negative FP False Positive HCI Human-Computer Interface IVMRP Imagined Voluntary Movement-Related Potential Idle 1 The EEG recorded in the passive observation period Idle 2 The EEG in the passive observation period between the periods of two finger switch activations LF-ASD Low-Frequency Asynchronous Switch Design MEG Magnetoencephalography M l Data of voluntary movement-related potential falling into the front part of the Observation Window which is centred at the onset of the finger switch activations M2 Data of voluntary movement-related potential falling into the rear part of the Observation Window which is centred at the onset of the finger switch activations MI Primary Motor Area MS Multiple Sclerosis OPM Outlier Processing Method PET Positron Emission Tomography ROC Curve Receiver Operating Characteristic Curve SCI Spinal Cord Injury SCP Slow Cortical Potential SMA Supplementary Motor Area VEP Visual Evoked Potential VMRP Voluntary Movement-Related Potential wN Size of the normalization window Wo Size of the observation window centered at finger switch activations 60 Appendix B P R O C E D U R E S O F D A T A A C Q U I S I T I O N 61 B.l Hardware Description A m p l i f i e r Headbox , — control terminal Figure B . l Hardware description of the L F - A S D As shown in Figure B . l , E-Cap with non-invasive electrodes is applied to acquire the mono-polar EEG. The location of the recording electrodes is in S M A and M I as shown in Figure 2.3. In order to keep the analysis free of the Ocular Artifact contamination, electrodes are applied to record E O G from a subject's eye-lips. For the purpose of evaluation, a finger switch is used to record the on/off information into a control signal. An amplifier is applied to adjust the mean amplitude of E E G mono-polar that makes the signal power relatively stable. In A/D interface and A/D board, the signal is sampled at 128 Hz. Then the digital signal is saved onto hard disk for offline analysis or input into the L F - A S D for on-line single trial analysis. The display monitor is used for showing instruction to the user, giving cues about when to activate the finger switch or imagine the finger switch activation. The researchers use the control terminal to direct the study. 62 B.2 Data Acquisition Strategy Five able-bodied subjects participated in the study. For each subject, eight runs of E E G were recorded. For each run, approximately 20 finger movements free of ocular artifact were recorded. In addition, for each subject, in the passive observation period, A E E G signal, which last for 4 minutes, was recorded to represent the idle state E E G . A computer program, which was designed to display a red, yellow or blue dot on the display terminal, was activated in the experiment. The dots with different colors were prompted to a subject in a sequence each per time. In the orientation phase, a subject was expected to activate the finger switch at the appearance of the yellow dot and remain in a passive observation state for the rest of the time. After orientation phase, in order to keep the V M R P free of the visual evoked potential (VEP) contamination, yellow dots were no longer displayed to the users. Since the red and blue dots were still presented to the subject at the original pace and order, with a blank screen where the yellow dot was previously, the subject should have been able to predict the transition time of the non-displayed yellow dots and then activate (or imagine to activate) the finger switch. The experiment was designed in two phases: the training phase, which including the orientation phase, and the data acquisition phase. The training phase was designed for the subjects to learn the synchronization between finger switch activations (for an able-bodied subject, or imaginary right hand finger flexion, for a subject with a disability,) and the appearance of yellow dots, both shown and unshown. At the end of this training phase, the subjects were expected to be able to activate (or imagine to activate) the finger switch at the exact time when a yellow dot should have appeared. Through a display monitor, a researcher could observe the synchronization between the appearance of a yellow dot and the finger switch activation (or, for people with a disability, a B C I switch activation) to tell whether the subject had learned to predict precisely. The data acquisition phase followed the training phase. In this phase, the able-bodied subjects, who sat on a chair, were required to follow the instructions shown on the screen to conduct a right index finger flexion, which triggered a micro switch. The programmed cues instructed the subjects when to activate the finger switch and when to be in a state of passive observation. The finger switch activations were marked into a control signal used as the measure of the switch activation time. (This method can also applied to a subject with a disability. He can conduct imaginary right hand switch activation and the transition time of non-displayed yellow dots, which was programmed and supposed to be synchronized with the imaginary finger flexion after training, can be recorded as the control signal.) A t the same time, E O G was recorded. E E G trials contaminated by ocular artefacts were discarded by referring to this E O G signal. 63 Appendix C A D D I T I O N A L S T U D Y R E S U L T S 64 A v e r a g e E n e r g y P e r S a m p l e j n ; a . w i n d o w c e n t e r e d a t M P f o r S u b j e c t - C . B 6 0 0 0 1 0 0 ' 1 2 0 1 4 0 1_6Q 1 8 0 . 2 0 0 jpj £0 4 0 6 0 8 0 1 0 0 ; 1 2 0 1 4 0 1 6 0 1 8 0 2 0 0 6 0 8 0 1 0 0 1 2 0 - 1.40 1 6 0 1 8 0 2 0 0 8 1 — 4 0 6 0 8 0 -TOO 1 2 0 1 4 0 1 6 Q 1 8 0 200 4 0 6 0 8 0 1 0 0 . 1 2 0 1 '40 1 6 Q 1 8 0 2 0 0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 ' 1 4 0 ' 1 6 0 1 8 0 S i z e : ; o f O b s e r v a t i o n W i n d o w : C e n t r e d : a t t h ' e F i n g e r S w i t c h (a) A v e r a g e L o w - F r e q u e n c y E n e r g y p e r S a m p l e i h : a w i n d o w ; c e n t r e d ' a t t h e F i n g e r S w i t c h f o r S u b j e c t C . B 3 . 4 | - r - — , -r. .... r - r . ., ' —1 I I l L i 1 1 1 , , v ' ^ - T ~ 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0 2 0 0 S i z e o f O b s e r v a t i o n W i n d o w C e n t r e d a t t h e F i n g e r S w i t c h (b) Average High- f requency Energy per Sample' in a window centrediat the Finger Switch for. Subject C B 5" A.^P-f 5^ SO 1 OO 1 2Q 14Q' 1QO; 1 2b 1 4Q- 16Q i S O 2QQ iOQ 12Q 14Q 16Q 18Q 2QQ i O O 1 2 0 1 4 0 I S O 1 S O 2 0 0 i a o 2QQ i O Q 12Q 1 4Q 1ISO 18Q 200 GO OO i O O 120 140 1GO 1 SO 200 Size-6f^Observation Window Centred at tho Finger Switch c) Figure C.1 (For Subject CB's E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band Energy Per Sample of Broad Band in a window centered at M p ;for; Subject M : P 1 OO ; 12Q 14Q 1 6 0 18Q SOP SO 1QQ. 12Q 1-4Q : 1 SO :2QO SO 1 0 0 120- 1 4Q 1 80 :2QO IOQ" 12Q 1'4Q T 6 0 16Q 2 0 0 1 2 0 1 4Q i SO '2QO SO 80; IOO; 120' 140' 160 180 Size of.Observation Window Centred .at .the Finger Switch (a) 66 Energy Per Sample of Low frequency bandi'nia wihdow:centered at M P for Subject M.P £ i.s. 100 °7. 1 1.5 20 80: " 'J26 40 60 80 100: 120 140 160 180 200 180 200 60 80 100 120 140 160 180 200 Size of Observation Window Centred at the Finger Switch b) Average Energy per Sample of High-Frequency band in a: windowcentred at the Finger Switch for'SubjectMLP 1.5 L— o> 80 100 120 140 160 180 Size of Observation Window Centred^ at the Finger Switch c) Figure C.2 (For Subject MP's E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band 67 Energy Per Sample of BroadBahd/ ih a window centered at Finger Switch for Subject M.B -S> 2.o i— E £ 1 S „ Si 4:5^ 2 3 ' . £ S .20 3 4 i— 40 60 100 120 14Q: 1 60 180 20Q 60 80 100 120 140 i 160 '180 200 60 80 100; 120 140 160 180 200 60 80 IOO 12Q 140 160 180 200 6Q 80 100' 120 140 160 180 200 40 60 80 100 120 140 1CO 100 200 Size-of^Obseryation WindoW::Centred:-at the Finger Switch (a) Average Energy per Sample of HightFrequencybandin a'window centred atthe Finger Switch for Subject M B 2. , , r r. 1 1 1 ' . . . 1.5 L C 1 i . r Q-015 20 40 40 60 80 100- 120 140 160 180 200 40 60 80 100. 120 140 160 180 200 40 60 80 100' 120. 140: 160 180 200 40 60 80 10b: 120 140 160 180 200 60 80 100 120 140 160 180 200 60 80 100 120 140 160 180 200 ISize of Observation Window Centred at :the*Finger-Switch (b) Energy Per Sample of Low frequency band in a window centered at Finger Switch for Subject M.B .•£ 1 ^ 1 . 3 i i i 0 :5 0 . 5 T 1.5'L-1 20 80: 100; 120 140 160 180 Size of Observation Window Centred at'the Finger Switch (c) Figure C.3 (For Subject M B ' s E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band Energy : Per Sample of Broad Band iri a^window centered : :at Finger Switch for Subject K.T 100: Si 10: L U 5 10 1 — 2 0 2 0 -60 80 100 120 140 160 180 .200 Size of Observation Window Centred at the Finger Switch (a) 69 Average Energy per Sample of High-Frequency '.band iri a window centred at the Finger Switch" for Subject rCT 6'h 0 '— 1!??-5 1 -2C?° 10 0 ! 10 20 40 40 40 40 60 60 100' 140, 160 120 -160 100 120 140 160 80 100 140 100 160 80 100 120 140 160 180 180 Size of Observation Window.Centred at the Finger Switch 200 200 180 200 ,200 (b) Energy Per Sample of Low frequency band in a window centeredat : Finger Switch for Subject K:T E. CO CO <u CL. 4 3.5 3.5 40 60 80; 100 120 140 160 180 Size of Observation Window Centred at the Finger Switch 200 (c) Figure C.4 (For Subject KT's E E G in Channels 1-6) a) Power around V M R P vs. length of the observation window centred at the finger switch in the broadband; b) Power around V M R P vs. length of the observation window centred at the finger switch in the low frequency band; c) Power around V M R P vs. length of the observation window centred at the finger switch in the high frequency band Note: The E E G energy variation condition near the finger switch activation for Subject PP has been provided in Figure 3.2. 70 DOM vs Normalization Window Size in Channel 1-6 for Subject CB 0.5 0.4 -0.3, O Q 0 . 5 : - • 0 . 4 -0 . 4 ° 0 . 3 -0 . 2 ^ 0: 0 . 5 -0:4 ^ 0 0:55 0.5 0.45 I 0.45 0:4 100 1 0 0 100 100 1 0 0 200 300 200 300 200. 300 300 200 300 With normalization Without normalization 400 500 600 With; Normalization Without Normalization 400: 500 600 With ::N ormalization Without Normalization. 500 600 With Normalization Without Normalization 400 500 ,600 Without Normalization With Normalization 500 600 With Normalization Without Normalization 200 300 400 Normalization Window Size 500 600 O Q 0:3 0.25 0.2 0:4! 0.3 0,2 O.i' 0.15 0.1 0.12 0.11 0.1 0.06 0.05 0.04( 0:1. 0:05 100 100 a) DOM.vs., Normalization Window Size in channels 1-6 for Subject MP 200 300 200: 300 200 200 With normalization Withoutnormalization 600 With ;:N ormalization Without Normalization 5 0 0 600 With Normalization Without Normalization 400 500 600 With Normalization Without "Normalization | — ^ - • .I — I " " I: Without Normalization VVith'"Normalization ~ -^ /"^^ : 400 500 600 With Normalization Without Normalizati on 300 400 Normalization Window Size 500 b) 71 DOM'vs. Normalization Window :;Size;,ih;Channe.r~1-6 for-Subject PP 0.5 0.4 0:3 0.5 0.4 0:3 0.15 0.2' 0.15 0.1 O.25' 0.2 0.15 100 100 100 100 200. 200' 20O: -•With normalization 'Without normalization 300 400 500 ;600 With Normalization Without Normalization: 300 300 300 400; 500 ,600 With: Normalization Without Normalization, 400 500 600 With Normalization Without Normalization; — ^ ^ :—... .| — — : : • • • : - — • • — : ~" : ' T " Wjthput: Normalization -r——• With Normalization -I i 1 A J 400 500 600 With Normalization Without N ormalizati on 200 300 400 Normalization Window Size 500 600 0.3 0.2 0.3 -0.2 -0.5 p 0.4 -0.3 -0:35 0.3 0.25 0.3 0.25 0 2 0.45' 0;4^ 0.35 100 100 100 -100 100 c ) DOM vs. Normalization Window Size in; Channel 1-6for Subject KT 200: 200 200 200 300 300 300 300 With normalization Without; normalization 400; 500 600 With Normalization Without N ormalizati on 400; 500 600 With Normalization Without Normalization 400; 500 600 With Normalization 1 , •••• 1 • I- " 'I p | • — — • :l — T T Without Normalization -• .. . With Normalization i 1 •:•••'•' | _1 J 400 500 600 With Normalization Without Normalization 200 300 400 Normalization Window Size 500: 600 d) 72 0:6 - -0.4 -0; O 0.6 0.4 0:3; 0.25 0.2 " 'i 0.2 0.18 0.16 i 0.2 0.15 100 100 100 100 •100 DOM vs. Normalization Window Size in channels 1-6 for Subject M B 200^ 300 200 300 200; 300 200 300 With normalization Without normalization 400i 500 600 With Normalization Without .'Ki ormalization i400 500' ;eoo Without Normalization With Normalization 400 500. 600 With Nb'rmalization Without Normalization / ' V ^ ^ ^ ^ ^ n n 1 Without Normalization • —r—With Normalization , ~ ~ L " ~ ~ — - ~ ~ - , r- • , - • -400 5 0 0 : 600 With Normalization Without Normalization 200 300 400 Normalization W i n d o w S i z e 500 600 e) Figure C.5 Determination of the optimal Normalization Window Size (DOM vs. W N ) in Channels 1-6 a) for Subject CB ; b) for Subject M P ; c) for Subject PP; d) for Subject KT; e) for Subject M B 73 The ROC Curve of the LF-ASD.with iNbrmalizatibn-arid without Nornalizatibnfbr .Subject MP 1I - *~ Without ENT 0.C02 0X04 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0 02 False Positive a) The ROC Curve of the LF-ASD with ENT for Subject MB Without ENT Wn = 21 -e- Wn = 51 - +•- Wn = 251 0.005 False Positive 0.015 b) d) The ROC Curve of the LF-ASD with Normalization and without Normalization for Subject KT False Positive e) Figure C .6 ROC Curves of the LF-ASD with and without the ENT a) for Subject M P ; b) for Subject M B ; c) for Subject PP; d) for Subject CB; e) for Subject K T 76
Thesis/Dissertation
eng
Electrical and Computer Engineering
Vancouver : University of British Columbia Library
For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
Graduate
The performance enhancement of the LF-ASD brain-computer interface : an Energy normalization transform
Text