Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

On Line Single Trial Detection of Index Finger Flexions from Spatiotemporal EEG Lisogurski, Dan 1998

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-ubc_1998-0267.pdf [ 2.78MB ]
JSON: 831-1.0065126.json
JSON-LD: 831-1.0065126-ld.json
RDF/XML (Pretty): 831-1.0065126-rdf.xml
RDF/JSON: 831-1.0065126-rdf.json
Turtle: 831-1.0065126-turtle.txt
N-Triples: 831-1.0065126-rdf-ntriples.txt
Original Record: 831-1.0065126-source.json
Full Text

Full Text

Line Single Trial Detection of Index Finger Flexions from Spatiotemporal E E G by Dan Lisogurski B.A.Sc, University of Waterloo, 1996 A THESIS SUBMITTED IN PARTIAL F U L F I L L M E N T O F T H E REQUIREMENTS F O R T H E D E G R E E O F Master of Applied Science in T H E F A C U L T Y O F G R A D U A T E STUDIES (Department of Electrical and Computer Engineering) We accept this thesis as conforming to the required standard The University of British Columbia April 1998 © Dan Lisogurski, 1998 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of £LCCjf~2cAL Arid CoHfo T€/L €./*6-S4)e* <r The University of British Columbia Vancouver, Canada DE-6 (2/88) Abstract A Brain Computer Interface (BCI) detects commands directly from the operator's brain and provides an output which can be easily interpreted by a computer. Much of the research to date has focused on differentiating between several possible commands rather than between control signals and the idle state. For most practical applications, it is essential that the operator is able to rest without issuing unintended commands. Mason proposes an Asynchronous Signal Detector (ASD) which allows existing BCI techniques to function outside the laboratory. The ASD continuously monitors the electroencephalograph (EEG) identifying segments which contain commands and acts as a switch which selectively relays E E G to a Control Signal Classifier (CSC). Existing BCI methods may be used for the CSC in order to determine which command the operator intended to issue. Alternatively, the ASD can function as a stand alone system capable of recognizing a single control signal. The goal of this research was to implement an ASD capable of identifying Voluntary Movement Related Potentials (VMRPs) from a continuous sampling of surface electrodes spatially distributed over the motor areas of the brain. Features were extracted from E E G components below 4 Hz as in Mason's Low Frequency ASD (LF-ASD) and classified using Learning Vector Quantization (LVQ). A revised version of Mason's ASD was implemented as an on line system. Two able bodied subjects each participated in three sessions as a preliminary evaluation of the ASD in detecting right handed index finger flexions. Subject training was also briefly investi-gated by providing the participants with feedback one second after movements were ii detected. Training appeared to take place rapidly with the percentage of detected movements for both subjects increasing from 24.0% in the first session to 45.3% and 49.3% respectively during the second attempt. During the three sessions for each subject 90.4% up to 99.2% of the idle E E G collected was correctly classified. Further research is required to evaluate the ASD as a generic V M R P detector and to test the method with participants who have little or no motor control. iii Contents Abstract ii Contents iv List of Tables vii List of Figures viii Acknowledgements ix 1 Introduction 1 1.1 Research Goals 3 1.2 Overview 4 2 Background 5 2.1 Technical Overview 6 2.2 Single Trial ERP Extraction 6 2.2.1 The Outlier Processing Method 7 2.3 E R D Based Methods 7 2.3.1 M u - E R D Interface 8 2.3.2 The Graz BCI Project 8 2.3.3 Feature Selection . 9 2.4 Asynchronous Signal Detection 9 iv 2.5 Subject Training 10 3 Signal Detector Design 11 3.1 Feature Extractor 11 3.1.1 Filtering 14 3.2 Feature Classifier 16 3.2.1 Learning Vector Quantization 16 3.2.2 Distinction Sensitive LVQ 21 3.3 Decision Module 22 3.4 Discussion 23 3.4.1 Classifier Selection 23 3.4.2 System Delay 24 4 Off Line Validation and Design 25 4.1 E E G Data 25 4.2 Analysis 26 4.3 ASD Tuning 27 4.3.1 DSLVQ as a Classifier 31 4.3.2 Number of Active Training Vectors 32 4.4 Cross Subject Training 32 4.5 Summary of ASD Modifications 33 5 O n Line Validation 36 5.1 Experiment Design 36 5.2 Results 41 5.3 Participant's Comments 42 5.4 Post Processing 43 5.5 Discussion 44 v 6 Conclusions 46 6.1 Summary of Contributions 48 6.2 Suggested Future Work 49 References 50 Appendix A List of Acronyms 53 Appendix B Subject Information and Consent Forms 54 B . l Subject Instructions 54 B.2 Consent Form 55 B.3 Subject Information Form 56 B.4 LAT-24 R Handedness Inventory [29] 57 vi List of Tables 2.1 Performance of Mason's ASD (Best Subject, tuned to reduce FP) . . 10 3.1 Mason's Feature Delays (128 Hz Samples) 14 3.2 LVQ Training Algorithms 24 3.3 System Delay 24 4.1 Comparison of ASD to Mason's Results 27 4.2 Effect of ASD Parameters 29 4.3 Revised ASD Performance for Subject 3 30 4.4 Comparison of Revised and Original ASD Performance 31 4.5 DSLVQ Comparison for Subject 3 32 4.6 Cross Subject Training 34 4.7 Summary of ASD Parameters 35 5.1 Participant Information 40 5.2 On Line ASD Performance 41 5.3 Vectors Selected for On Line LVQ Training 45 vii List of Figures 1.1 Mason's Two Stage BCI 3 3.1 Block Diagram of Mason's L F - A S D 11 3.2 E E G Electrodes (feature set electrodes shaded) 12 3.3 Feature Delays 13 3.4 Frequency Response of Mason's Filter ' . . 15 3.5 LVQ Discriminant 17 3.6 K Means Example 18 3.7 LVQ Window 20 3.8 DSLVQ Distances 22 3.9 Decision Module Window 23 4.1 LS (17) Filter Frequency Response 30 5.1 Experiment Display 37 5.2 Data Collection Hardware 38 viii Acknowledgements Many people contributed to this work in some way. I would like to thank the partici-pants in the pilot studies and experiments as well as my co-supervisor Peter Lawrence. Steve Mason's research established the basic methodology applied in this thesis and I would also like to thank him for the discussions which were useful in understanding his work. Peter Liddle from the Department of Psychology and Michael Young of the British Columbia Institute of Technology were generous in loaning the E E G ampli-fiers used in this research. I am grateful to Robert Hare for making his laboratory facilities available and in helping me understand some of the fundamentals of E E G . I would also very much like to Vicky Ressl for her encouragement and advice. Gary Birch always managed to find time to meet on short notice and provide feedback when I needed it. I would like thank him for his flexibility in allowing me to choose a direction for this project as well as all of his guidance. D A N L I S O G U R S K I The University of British Columbia April 1998 ix Chapter 1 Introduction A Brain Computer Interface (BCI) detects commands directly from the operator's brain and provides an output which can be easily interpreted by a computer. The simplest BCI detects only the presence of a control signal. Single switch interfaces have already been developed to emulate computer keyboards [1] such that one reliably detected command could provide a usable, although relatively slow, interface. A more advanced system capable of differentiating between multiple control signals may significantly improve the speed and efficiency of the BCI. The short term impacts of this research area are likely to benefit only those with severe disabilities who have few other options for communication and independence. Individuals with the acute stages of amyotropic lateral sclerosis, multiple sclerosis, or injuries to the upper spinal cord often have very limited motor control making even the design of a single switch interface challenging. End users will benefit from greater independence with the ability to control devices such as a powered wheelchair or robot in addition to an improved ability to communicate with others. The hardware and development costs may be offset over the long term by reduced needs for attendant care and an improved quality of life. Measurement of the electrical activity of the brain or electroencephalograph (EEG) using electrodes placed on the surface of the scalp is the most common basis 1 for BCI research. Implanted electrodes are generally considered too invasive although they have been used in some cases. Other methods such as magnetoencephalography are too expensive and cumbersome to be practical. The single switch BCI can make two possible errors. A False Positive (FP) is the detection of an unintended command in idle E E G . A False Negative (FN) occurs when the BCI fails to detect a control signal issued by the operator. In addition, a BCI capable of detecting several control signals can generate other types of errors by confusing one command with another. Much of the research in this area has focused on distinguishing between commands rather than between a control state and an idle or resting state. For a system which can be used without the supervision of an assistant it is essential that the BCI not only determine which control signals are present but also allow the operator to rest without producing false positives resulting in issuing unintended commands. Many researchers have evaluated the error rates of their BCIs in terms of how often short segments (1-4 seconds) of E E G are misclassified. This provides a very biased indicator of performance since control signals will be issued infrequently in many applications. Outside the laboratory the BCI must be able to process several minutes of data without false positives while the operator rests or is busy processing information [2]. Mason [3] outlines a framework for the design of a BCI with two separate mod-ules called the Asynchronous Signal Detector (ASD) and the Control Signal Classifier (CSC). The ASD is designed to continuously monitor the E E G and to detect the presence of control signals. The CSC can be optimized to classify the E E G with the knowledge that a command is present. The ASD acts as a switch which disables the CSC until a control signal is detected, as illustrated in Figure 1.1. Mason argues in favor of the two stage approach since the spatially distributed features which are gen-erally better for an ASD are not necessarily optimal for the CSC. The CSC typically requires more local information. An architecture with two separate modules allows 2 EEG ASD -oo-r > C S C BCI output Figure 1.1: Mason's Two Stage BCI each stage to be simpler as well as more specialized. Used alone the ASD can act as a BCI capable of recognizing a single command. A reliable ASD allows the BCI to be used out of the laboratory in an unsuper-vised environment. After a careful review of the literature, Mason evaluated four off line ASD designs on E E G collected from five subjects. Mason's L F - A S D performed well but several important questions remain. The effects of subject training cannot be studied without an on line implementation to provide feedback. The robustness of the ASD to involvement of the operator in various cognitive tasks is also unknown. Obtaining the time-locked trials used in training the L F - A S D is much more challeng-ing when the operator is physically unable to perform the movement. In addition, this technique makes the assumption that individuals with motor dysfunction can produce E E G patterns similar to Mason's able bodied subjects, despite the inability to actually make the voluntary movement. Although the latter issue is beyond the scope of this research, an on line implementation will provide the tools required for future investigation. 1.1 Research Goals The goal of this research is to implement a reliable software based asynchronous signal detector which is fast enough for an on line control task. The resulting system will function as a BCI with one level of control and be suitable for use in conjunction with a control signal classifier in identifying multiple commands. Once the training stage is complete the operator will not require any assistance to use the BCI. The implemen-3 tation of this interface also allows a preliminary investigation into the effectiveness of subject training and provides a tool for future research in this area. 1.2 Overview Chapter 2 presents a summary of relevant research concluding that Mason's L F - A S D provides a good starting point for the design of an on line system. Chapter 3 contains a detailed description of the L F - A S D algorithm and several ideas are proposed to improve performance or solve problems in making the transition to a on line system. In Chapter 4 Mason's results are replicated using the original data to establish a baseline for comparing proposed changes. In addition, a revised version of the ASD both improves performance and reduces the delay in processing E E G to 640.5 ms. An experiment designed to validate the on line system is explained in Chapter 5 and the results from two new subjects are presented. Acceptable F P rates were achieved while detecting up to 49.3% of the finger flexions. Chapter 6 contains conclusions and a list of acronyms is available in Appendix A. 4 Chapter 2 Background Recent reviews of BCI techniques [2] [3] group the methods into two distinct cate-gories. The first class are based on the detection of patterns which appear in the E E G due to an externally generated stimulus such as Visually Evoked Potentials (VEP). In one of the more successful methods Sutter [4] gets the operator to gaze at one of 64 symbols arranged on the monitor in an 8 x 8 grid. The symbols alternate between red and green in different sequences. The correct symbol is identified based on the correlation between E E G sampled over the visual cortex and templates produced by averaging during a training phase. This method has led to communication at up to twelve words per minute when used with implanted electrodes. While this interface may be well suited to certain applications, it is not practical for a generic system since it requires attending to the monitor rather than the task and a reasonable level of eye control is assumed. For this reason it is preferable to focus on the second class of BCIs which are based on self initiated potentials including Voluntary Movement Related Potentials (VMRPs). 5 2.1 Technical Overview Surface electrodes placed on the subject's scalp detect a weighted sum of the electrical activity of the brain. Artifact from nearby sources of noise including the electrical ac-tivity from the muscles (EMG) or due to eye movements and blinks (EOG) may often occur in much larger magnitudes than the desired signals. Event Related Potentials (ERPs) are patterns within the E E G associated with an event or stimulus. Examples include evoked potentials which result from exposing the operator to a flash of light or other stimulus. ERPs are also produced by the operator during a V M R P related to an index finger flexion. The background E E G signal is typically on the order of 100 fiV. In contrast an ERP may be 10 fj,V in magnitude making single trial extraction challenging due to the low Signal to Noise Ratio (SNR). In addition, E E G is a non stationary coloured noise process. Ensemble averaging of multiple time-locked trials is sometimes used to improve the SNR. This method may be effective in building a template of the ERP during the BCI training phase but it is not practical for the implementation since it requires the operator to repeat the command multiple times and and a trigger is needed to time-lock the trials. 2.2 Single Trial ERP Extraction Several methods attempt to extract ERPs on a single trial basis and classify the resulting waveform as one of several possible signals. Lange et al. [5] reported an improvement of 18 to 23 dB from an initial SNR of -10 to -20 dB respectively. The potentials extracted were due to a finger tapping task. The robust evoked potential estimator used in this work is based on the Box-Jenkins model. They have also evaluated a method which uses a priori templates of the ERP [6]. Hansson et al. [7] extracted ERPs using an adaptation of the Prony method for coloured noise from signals with an SNR of 0 dB. Although techniques such as these may be candidates for the control signal classifier, the remainder of this chapter will be limited to methods 6 specifically designed as a BCI which have been tested in their ability to discriminate between the active and idle states. 2.2.1 The Outlier Processing Method The Outlier Processing Method (OPM) [8] is a robust signal estimation technique which has been effective in single trial ERP extraction. The method does not require any a priori information about the signals, but the algorithm is quite sensitive to several parameters which must be tuned appropriately. The O P M assumes that the noise (EEG) can be modeled as an autoregressive process with time varying parameters. The ERPs appear as patchy additive outliers to the background E E G . The model parameters are computed with a robust autoregressive estimator and the noise is estimated using a technique which can be described as a robust modification of the Kalman filter. The ERP is extracted by subtracting the estimated noise process from the measured E E G signal. This technique has yet to be implemented in real time but should be possible using an inexpensive Digital Signal Processor (DSP) such as the Texas Instruments TMS320C3x series [9]. Birch [8] uses dynamic time warping to determine how well the ER P matches a template in order to classify the signal. Although the technique works well for relatively long movements (approximately 4 seconds) a much higher rate of false positives has been observed with shorter segments [3]. 2.3 E R D Based Methods The theory behind E R D is that certain events cause a desynchronization in neural activity or Event Related Desynchronization (ERD) measurable by a decrease in signal power for the frequency band involved. The opposite effect is called Event Related Synchronization (ERS). 7 2.3.1 Mu-ERD Interface Wolpaw and McFarland [10] [11] [12] [13] have developed an interface where the operator learns to move a cursor up or down on the monitor by consciously varying the power in the mu (8-12 Hz) frequency band. Some subjects have also been trained to achieve cursor control in two dimensions [14]. The sum of the 10 Hz E E G activity over the left (FC3-CP3 electrodes) and right (FC4-CP4 electrodes) side of the head was linearly transformed into vertical movement. Another linear equation mapped the difference of the two magnitudes to horizontal movement. Larger sums or differences resulting in moving the cursor up or right respectively. Four of five subjects who trained over a period of 6-8 weeks achieved accuracies up to 70% in selecting a target placed either at the top of bottom of the screen [14]. The other subject was unable to become proficient in this task. Some of the subjects reported initially imagining that they were floating or running to move the cursor up or down respectively. As the training progressed this was no longer necessary. It should be noted that from the imagery it appears possible that the subject may have controlled the cursor by varying their attention level, which may also affect this frequency band. During each trial the subject must select a target by moving the cursor. There is no discussion of an idle or no movement state. 2.3.2 The Graz BCI Project The Graz BCI Project [15] [16] uses E R D to accomplish one dimensional cursor control. Data was collected from surface electrodes placed over the contralateral sen-sorimotor cortex with an upper cutoff frequency of 15 Hz. Trials lasted 1.125 seconds and the the researchers experimented in dividing the trial into several windows. Fea-tures were defined as the average power over each window. Six neural networks were compared in their performance for this classification problem [17]. The features were classified using either the LVQ1 or LVQ3 algorithms which are explained in Chapter 3. Once again, Pfurtscheller's group assumed a left or right handed finger flexion dur-8 ing each trial. The idea of an idle class is briefly discussed in the context of rejecting trials which do not seem to match either movement [15]. 2.3.3 Feature Selection Estimates of the optimal electrode positions for detecting E R D have been produced by analyzing data recording from multiple sites by the Graz BCI Project. A new method called Distinction Sensitive Learning Vector Quantization (DSLVQ) [18] [19] [20] was compared with a genetic algorithm for feature selection [21] in the ability to choose relevant electrodes for E R D based features. DSLVQ is explained in Chapter 3. The performance of the two methods was similar, but DSLVQ was preferred due to the simplicity of the implementation when LVQ is already used for classification. This technique has also been applied to Wolpaw's work in collaboration with Pfurtscheller [22]. 2.4 Asynchronous Signal Detection Mason [3] has proposed a Low Frequency ASD (LF-ASD) based on features extracted from the 1-4 Hz E E G activity. This method, explained in detail in Chapter 3, at-tempts to exploit spatial and temporal redundancies to detect VMRPs. The results were compared to an ASD using only temporal information as well as two ASDs de-signed by Mason based on mu-ERD and the O P M respectively. All four methods were evaluated off line using E E G collected from five subjects. The percentages of correctly classified idle points and movements (hits) are presented in Table 2.1 for the subject with the best performance. Although Mason's ASD is designed to detect VMRPs from various movements, it has only been evaluated in recognizing a right handed index finger flexion from a right handed subject. 9 Table 2.1: Performance of Mason's ASD (Best Subject, tuned to reduce FP) ASD Design Idle Hits L F - A S D 98.4% 60.0% L F - A S D f 94.5% 55.1% M u - E R D 28.2% 71.6% O P M 71.3% 51.1% f Simplified version without spatial information 2.5 Subject Training Several researchers have shown that subjects can learn to control their E R D . Mason only provided feedback based on how well the subject's index finger flexion matched a template of the desired movement. Since the data analysis was done off line it was not possible to provide feedback suitable for training the subjects to produce more separable features. Mason simulated training effects by eliminating nine of the weakest trials from the analysis and hits increased by 10-20% [3]. In order for the system to be practical, it must be shown that subjects with very limited motor control can learn to produce these potentials. An abstract by Kuebler et al. [23] describes a successful attempt in training two subjects with severe motor paralysis to modify their slow cortical potentials. Other evidence includes the ability of amputees to trigger myoelectric patterns in stump muscles from phantom hand movements which were useful in controlling hand prostheses [24]. However, there is no conclusive evidence that Mason's ASD will work on the population for which it has been designed since it has only been evaluated on able bodied subjects. 10 Chapter 3 Signal Detector Design Mason's work was selected as the starting point for the design of an ASD since he is the only known researcher to have focused on this aspect of a BCI. In addition, the off line evaluation reported in [3] seems promising. A block diagram of Mason's LF-ASD is shown in Figure 3.1. Features extracted from the E E G are either classified as idle or active. Active E E G contains a VMRP. The output from the classifier is modified by a decision module. The following sections discuss each block in detail. Several changes are proposed to improve the design and better adapt the system to an on line implementation. The outcome of these modifications is presented in Chapter 4. 3.1 Feature Extractor Mason analyzed data from finger position, E O G , and the fourteen monopolar elec-trodes (referenced to Oz) shown in Figure 3.2. The electrode pairs over the Sup-plementary Motor Area (SMA) and primary motor area (MI) are indicated. The EEG Feature Feature • . Decision Extractor Classifier Module Output Figure 3.1: Block Diagram of Mason's L F - A S D 11 Figure 3.2: E E G Electrodes (feature set electrodes shaded) electrode positions conform to the International Ten-Twenty System [25]. Eighteen bipolar electrode pairs were simulated by subtracting combinations of adjacent elec-trodes arranged in either anterior-posterior or laterally placed pairs. Bipolar record-ings were thought to be more robust. Mason noted power differences between the active and idle states in the 1-4 Hz band. He hypothesized that patterns visible in the ensemble averages occurred on a single trial basis but were buried in noise and may be detectable if the spatial and temporal redundancies of the brain were exploited. Mason defined elemental features E\ and E2 as the difference of a filtered signal at two points in time as calculated in (3.1) and (3.2) where is the potential difference measured between the simulated bipolar electrode pair i. The delay terms 8 were selected to maximize the difference of medians between features extracted from active and idle data. Mason initially estimated these values from the ensemble averages of the 1-4 Hz components in the odd numbered active trials from one subject. The trigger point was defined as the start of the finger flexion. The delays were based on the peak near the trigger, the first local minimum after the trigger, and the local 12 M-V 3 0 4 3 1 2 3 2 0 3 2 8 3 3 6 3 4 4 3 5 2 3 6 0 3 6 8 3 7 6 3 * 4 3 9 2 4 0 0 4 0 8 f»16 4 2 4 4 3 2 4 4 0 4 4 8 Samples (128 Hz) Trigger Point ( V M R P ) Figure 3.3: Feature Delays maximum which follows as illustrated in Figure 3.3. An optimization process was used to adjust the initial estimate. Compound features were formed to emphasize samples where two large elemental features appear simultaneously. After an initial evaluation using cross correlation, Mason simplified the calculation to (3.3). The scalar weight W defined in (3.4) prevents inverse signal relationships (negative differences) from producing large feature values. For robustness the compound features are maximized over a window as shown in (3.5). Ei[n] = ei[n + Si] — ej[n + 62 (3-1) W (3.2) (3.3) (3.4) E2[n] = ei[n + S3] - e,[n + 64] gi[n] = Wx E^n] x E2[n] 1 Ei[n] > 0 andE2[n] > 0 0 else Gi[n] = max{5,[n - 8], &[n - 7], • • •, c/;[n], • • •, g^n + 7], &[n + 8]} (3.5) Mason selected symmetrical electrode pairs and constrained the delays to be equal from left to right. Although it has not been evaluated, this methodology was 13 Table 3.1: Mason's Feature Delays (128 Hz Samples) Feature Electrode Pair t Si s 2 83 5, 1 F1-FC1 -1 25 0 50 2 Fz-FCz -1 25 0 50 3 F2-FC2 -1 25 0 50 4 FC1-C1 -1 15 -12 30 5 FCz-Cz -1 15 -12 30 6 FC2-C2 -1 15 -12 30 f See Figure 3.2 for electrode locations intended to make the feature set applicable to various VMRPs since there is no emphasis on contralateral potentials. Mason selected the features in Table 3.1 from the many possible combinations. The feature and delay selections were based on data from one subject and appeared to work reasonably well for all of the subjects. Mason was able to improve the performance by customizing the delays to each individual. Note that the maximum delay is 50 samples in addition to the eight samples over which the values are maximized in (3.5). Since these samples are at 128 Hz the system must acquire 453 ms of data past the point in time where the features are to be extracted. 3.1.1 Filtering Mason implemented a 121 order bandpass (1-4 Hz) zero phase Finite Impulse Re-sponse (FIR) filter based on a hamming window. The non causal filter was applied as in (3.6). The FIR coefficients are stored in b which is a vector of length N . The order must be odd to achieve symmetry about the midpoint resulting in exactly zero phase in the passband. The frequency response of Mason's filter is shown in Figure 3.4. An on line implementation this filter would introduce a delay of 60 samples. At Mason's sampling rate of 128 Hz this adds in a delay of 469 ms before the features can be 14 o g - 2 0 o ^ - 4 0 cz O) CO s - 6 0 - 8 0 MVYYYYVYVVVVYVVVVV IAAAAAAAAA AAAAAAAAA WAAAAAAAf AAAf 1000 h a> 5 0 0 10 2 0 3 0 40 F r e q u e n c y (Hz) 5 0 3 0 4 0 F r e q u e n c y (Hz) 5 0 Figure 3.4: Frequency Response of Mason's Filter 60 60 TV extracted. y[n - (N - l)/2] = £ b[k]x[n - k] (3.6) fc=0 A small delay in operator feedback is acceptable for this application although it should be minimized. The operator is likely to perceive only the fraction of the delay which occurs after the voluntary movement has been completed. Delay can be reduced by decreasing the filter order. However, the sensitivity of the feature separability to the filter is unknown. A sharper cutoff or better stop band attenuation may not necessarily improve the performance of the system since it is possible that some of the E E G components in the transition bands contributed to Mason's success. Therefore the design procedure adopted in this work was to implement an off line system and compare the performance using Mason's filter to various other designs with lower order and a similar frequency response. Considering the passband shown in Figure 3.4 it seemed reasonable to try a low pass filter with 0-4 Hz passband. In addition, a shorter filter length has the following advantages. 1. Less data to reject after detecting artifact. 15 2. Dependence on fewer data points surrounding the finger flexion onset increases the probability that the detected patterns are due to movement planning and initiation rather then the proprioceptive feedback which occurs slightly later. This is considered advantageous since it is still unknown how well this method will work in the case of a disabled operator. 3. Savings of 6 features x 128 Hz = 768 multiplications per second for each zero which can be eliminated from the filter design. 3.2 Feature Classifier The features are classified using Learning Vector Quantization (LVQ) [26]. Mason has already shown the method to work for the L F - A S D [3] and this technique has also been applied to the classification of E R D [15] [17] [20] [22]. The following issues were considered in choosing the method. 1. Small number of active training vectors. 2. Features are not well represented by a Gaussian probability density function. 3. Real time classification required. 4. Ability to tune the classifier in favor of false positive or false negative errors. 3.2.1 Learning Vector Quantization Although Learning Vector Quantization (LVQ) is often described as a neural network, the statistical basis for this classifier is easily understood. The codebook consists of one or more vectors from each possible class. A 1-Nearest Neighbor (1-NN) classifier is applied to each feature vector resulting in the classification of input vectors with the same class as the nearest (Euclidean distance) codebook vector. The locations of the codebook vectors are determined during the training process. The result is 16 70 60 50 ,t40 £ to "-30 20 10 0 0 5 10 15 20 25 30 35 40 45 50 Feature 3 Figure 3.5: LVQ Discriminant a piecewise linear discriminant. Two of the six features for one of Mason's subjects are plotted in Figure 3.5. The plot also contains the discriminant corresponding to a 1-NN classifier using the three codebook vectors from each class which are shown. Each point is classified based on the region in which it falls. The class of the codebook vector contained within each region defines the class of points which fall inside the boundary. Kohonen [26] proposes several algorithms for training the codebook vectors so that the 1-NN classifier will result in a good decision boundary. Several of the algorithms described in the following subsections explain how LVQ can approximate the Bayes decision rule without any a priori knowledge about the probability density function for the features. The first step is the selection of the number of codebook vectors used to represent each class. These vectors are initialized using the K means algorithm described below which separates the training vectors for each class into K clusters represented by their means. 17 • Idle - + + Active + o Idle Codebook + * Active Codebook Discriminant + + * A c t i Y- e + + * Active + ~~' ~H — '• + p Idle .+.• .:-*AS»ve + —^ + I I Cluster 1 III Cluster 2 / \ Cluster Mean Feature 1 Initialization Feature 1 Iteration Process Feature 1 Convergence Figure 3.6: K Means Example The K Means Algori thm The active and idle vectors are clustered in two separate runs of the algorithm. Each training vector is randomly initialized as a member of one of K possible clusters. The means are calculated by averaging the vectors in each cluster. Vectors are reassigned to the cluster with the nearest (Euclidean distance) mean. The means are updated again and the algorithm continues to iterate until the means converge. The process is illustrated in Figure 3.6 where K=2. The resulting cluster means can be used as codebook vectors where applying the 1-NN decision rule will result in classifying each input vector with the class of the nearest cluster. However, it is preferable to fine tune the codebook further since the K means algorithm does not generally reflect the minimum probability of error. In addition, with a 1-NN classifier not all of the clusters defined by K means affect the decision boundary. Fine tuning is accomplished by LVQ training using one of the following algorithms. LVQ1 Training At each iteration x is selected randomly from the set of training vectors. The nearest codebook vector to x in terms of Euclidean distance is called mi. Only one vector from the codebook (m,) is modified during each iteration. If x and mj belong to the 18 same class then (3.7) is applied to move the codebook vector towards the training vector. If this is not the case then the codebook vector is moved away from the training vector using equation (3.8). The learning rate a[n] is usually set to decrease from an initial value (typically 0.05) down to zero over a fixed number of iterations. The codebook vectors are initialized (rrii[0]) with the results of applying K means to the training data. rriifn + 1] = m;[n] + a[n](x[n] — mi[n]) (3.7) rrii[n + 1] = nii[n] — a[n](x[n] — m;[n]) (3.8) LVQ2 Training Kohonen [26] proposes LVQ2 in order to better approximate the Bayesian decision surface. The closest (m;) and second closest (mj) codebook vectors to x are identified at each iteration. If rrii and mj belong to the incorrect and correct classes respectively then both codebook vectors are trained using (3.9) and (3.10). However, the training only takes place if the window constraint in (3.11) is satisfied. The scalar value w is determined experimentally and represents the relative width of the window. Kohonen recommends 20% (w = 0.2) as a reasonable starting point. The scalar distances dj and dj are measured between x and each of the codebook vectors. rrii[n + 1] = rrii[n] — ct[n](x[n] — mj[n]) (3.9) mj[n + 1] = mj[n] + a[n](x[n] — mj[n]) (3.10) . [di oL-i 1 — w min < - 3.11) ldj di1 1 4- w Figure 3.7 shows an example in 1 dimension of how the LVQ discriminant is updated during training to approximate the Bayes decision rule. Each time x falls in the shaded window on the wrong side of the mid plane the two codebook vectors are adjusted using (3.9) and (3.10). Since the probability of obtaining an idle vector x from the training set is higher, more corrections should result in a shift of the codebook vectors in the direction of the arrows, towards the Bayes decision surface. 19 p P(Idle) P(x I Idle) Idle Codebook Vector ] Active Codebook Vectdr P(Active) P(x I Active) .1 LVQ Discriminant (Vector Mid Plane) ! Bayes Discriminant (Minimum Probability of Error) Figure 3.7: LVQ Window Kohonen points out that the correction to the closest vector will be always be greater than the correction to the next closest vector since adjustments are only made when x is on the wrong side of the mid plane [(x — rrij) < (x — nx,)]. Too many iterations or too large a learning constant may result in overshooting the optimal placements. LVQ2.1 Training To help stabilize LVQ2 over a longer training period the two closest codebook vectors are trained at each iteration where one vector is the correct class and the other vector belongs to the incorrect class. This results in training where x falls within the window , on either side of the mid plane. Training proceeds using (3.9) and (3.10) with the exception that mi and m.j now represent the correct and incorrect codebook vectors respectively. L V Q 3 Training The process is further stabilized with the LVQ3 algorithm. LVQ2.1 training takes place when the proper conditions are met. When both of the two closest vectors belong to the same class as x and the windowing constraint in (3.11) is satisfied, 20 training takes place according to the following equations. The stabilizing factor e is typically set at 0.2 and makes the process less sensitive to the number of iterations. rrii[n + 1] m.i[n] + ea[n](x[n] — nii[n]) (3.12) m.j[n + 1] m j N + «*[ft](x[n] — nij[n]) (3.13) Mason's L V Q Implementation Mason applied LVQ3 to the classification problem but did not use a window constraint in the case where both codebook vectors were from the correct class. LVQ2 was applied instead of LVQ2.1 when the appropriate conditions were met. The learning rate a[n] decreased linearly from 0.05 to zero over 5000 training iterations and the window was defined by w = 0.2. 3.2.2 Distinction Sensitive LVQ More recently Pregenzer et al. [18] have proposed a method called Distinction Sen-sitive Learning Vector Quantization (DSLVQ) which uses a weight vector w to vary the influence of each feature. The weights are adjusted iteratively in parallel with LVQ3 training according to (3.14). The second advantage of this method is that the resulting weights provide an indication of the relative information in each feature. The threshold operator prevents any of the weights from falling below a prede-termined minimum value or achieving a value of 1.0. At each iteration the distance between the training vector x and closest codebook vector of the same class (mj) is called dj. The distance to the closest codebook vector of an incorrect class (mi) is labeled d, as shown in Figure 3.8. The case where dj < dj results in a negative value of wrij in (3.16) which leads to a reduction in the weight w, in (3.14). Since the weights are updated at each iteration in parallel with the codebook vectors the constant k < 1.0 in (3.14) scales down the learning rate. Updates only take place if the window constraint for LVQ3 learning is satisfied. 21 a 1^ m i dj ""di*" mj Figure 3.8: DSLVQ Distances w[n + 1] = norm[threshold[w[n] + ka[n] x (wn[n] - w[n]))J (3.14) norm(y) = Y (3.15) 2^i=i \yi\ wn[n] = norm(dj[n] — dj[nj) (3.16) d|[n] = |x[n] - mi[n]| (3.17) djN = | x [ n ] - m j [ n ] | (3.18) 3.3 Decision Module The decision module refines the output of the feature classifier by further exploiting temporal information. Although not every trial produced active classifications, Mason found that between two and four active points typically occurred around the finger flexion in most trials when the features were extracted every 1/16 seconds. In contrast only about 8% of the idle trials resulted in one to four active points from the feature classifier. The Control State Density (CSD) is defined as the number of times that the classification module detected a control signal in a window of length A centered around the point being evaluated [3]. The system reports a V M R P when the control state density exceeds l0. While evaluating the features every 1/16 seconds, Mason selected A = 5 (312.5 ms) and l0 = 2 or 3 depending on the desired trade off between false positives and false negatives. Mason reports no improvement in increasing A beyond five. Figure 3.9 shows how the window is applied to the classifier output c[n]. 22 c[n-2] c[n] c[n+2] • •• o o i o o o o o ^ o ^  o 'rT N v ' A=5 Figure 3.9: Decision Module Window This non causal design requires the results of (A — l)/2 future classifications resulting in an additional delay of 2 x ^ = 125 ms for the on line ASD. 3.4 Discussion Two issues are addressed in Chapter 4 before building the on line ASD in Chapter 5. These include reducing the overall system delay and the selection of best LVQ algorithm and parameters. 3.4.1 Classifier Selection Table 3.2 summarizes the LVQ algorithms described in this chapter. In theory LVQ3 or DSLVQ appears to be the best method, but various trade offs must be considered. For example, Flotzinger et al. [15] reported a smaller standard deviation (1-4 vs. 10-30) in the classifier's performance when the codebook was trained with LVQ1 and LVQ3 respectively. DSLVQ may be useful in providing a measure of the relative information in each feature by comparing the weights which result from the codebook training. Note that the weights are not affected by redundant or correlated features. However, unequal weighting of lateral features will likely result in a movement specific ASD rather than the general movement detector which is the goal of this research. It may be possible to correct for this by applying an appropriate constraint in training the weights. The results of this investigation are presented in the next chapter. 23 Table 3.2: LVQ Training Algorithms Algorithm Procedure Window LVQ1 Each iteration trains the closest codebook vec-tor. No LVQ2 The nearest and second nearest vectors to x are trained at each iteration provided they be-long to the incorrect and correct classes re-spectively. Yes LVQ2.1 The two closest codebook vectors are trained provided they belong to different classes. Yes LVQ3 In addition to the rules for LVQ2.1, the nearest two codebook vectors are trained if the both belong to the correct class. Yes Mason's LVQ LVQ3 with LVQ2 is used instead of LVQ2.1. LVQ2: Yes LVQ3: No DSLVQ LVQ3 using weighted distances to vary the in-fluence of each feature. Yes 3.4.2 System Delay Table 3.3 shows that Mason's L F - A S D requires a delay of 1.047 seconds before sam-ples can be classified. Although the delay in the feature extractor may vary slightly between subjects, a significant reduction is not possible using this method. The de-cision module delay will also be difficult to reduce since removal of this temporal information would likely increase the false positives. Changing the filter order and design will likely provide the best solution in minimizing system delay. Table 3.3: System Delay Subsystem Delay (ms) Filter 469 Feature Extractor 453 Decision Module 125 Total 1047 24 Chapter 4 Off Line Validation and Design Before attempting to process E E G on line, an off line L F - A S D was replicated to validate Mason's work and test the implementation. The error rates obtained from this system were compared to those reported by Mason and formed a baseline for testing modifications to the algorithm and parameters. 4.1 E E G Data Data from Mason's five subjects [3] was available. Each file contained the 18 bipolar electrode combinations for one trial which consisted of four seconds of data sampled at 128 Hz. A index finger flexion was situated at the three second mark (trigger) of the active trials. Training data was taken from the odd numbered trials. As in Mason's work [3], a single feature vector was extracted from each active trial at the trigger point and from the idle trials every 1/8 seconds. The classifier was evaluated on features extracted every 1/16 seconds from the even numbered trials. The two second window centered at the trigger in the active trials was considered a special case (pre and post movement data) and was not classified, with the exception of a ± 400 ms window about the trigger point where the detection of a V M R P was considered a hit. This window was selected based on Mason's work [3] since there 25 was little activity in from 300 to 600 ms both before and after the trigger. Higher rates of FPs were observed in the regions located from ± 600 to 1000 ms. Approximately 50 active and 50 idle trials were available from each subject. 4.2 Analysis Mason evaluated the ASDs in two ways. The one shot evaluation involved training the classifier using odd numbered trials and evaluating the performance with the even numbered trials. The results were improved with a nine fold cross validation procedure where the classifier was trained using random sets including 8/9ths of the data and tested on the remaining vectors. Cross validation can be useful with a small number of feature vectors since more data can be used for training. This is not possible for the on line implementation since the test data will not be available a priori. For this reason the results in the section are calculated and compared in the manner of the one shot evaluation. LVQ learning involves the selection of random feature vectors from the training set resulting in some variance to the convergence of the codebook. Due to the decaying learning constant and window constraint the training process is sensitive to both the choice of vectors as well as the order in which they are selected. Note that Mason's one shot evaluation reports only the results of one particular run for each subject. Table 4.1 shows the performance of an ASD designed to simulate the L F - A S D . The percentages of correctly classified idle points and hits are shown. Standard deviations generally ranged up to about 12% for the false negatives and were typically smaller for the false positives. This is not surprising since the number of hits is based on approximately 25 trials which makes the performance more sensitive to small changes in the codebook affecting the classification of only a few vectors. The results seem in line with Flotzinger et al. [15] where LVQ3 standard deviations ranged from 10% to 30%. 26 Table 4.1: Comparison of ASD to Mason's Results / 0 - 2 k = 3 Subject Mean 95% Confidence Mason Mean 95% Confidence Mason 1 Idle 74.0% [59.1, 89.0] 86.9% 87.3% [75.3, 99.2] 93.9% 1 Hits 37.9% [19.7, 56.1] 31.6% 21.8% [ 4.3, 39.3] 27.9% 2 Idle 88.8% [82.7, 94.8] 96.0% 94.5% [90.7, 98.7] 97.7% 2 Hits 61.5% [34.3, 88.6] 60.0% 49.6% [43.1, 56.1] 48.0% 3 Idle 91.6% [84.8, 98.5] 96.1% 96.3% [92.1, 100] 97.7% 3 Hits 61.7% [47.7, 75.7] 73.1% 43.5% [29.0. 58.0] 53.8% 4 Idle 87.5% [78.8, 96.2] 94.1% 94.4% [89.6, 99.1] 90.3% 4 Hits 46.7% [33.8, 59.6] 50.0% 37.3% [27.7, 47.0] 37.5% 5 Idle 79.8% [62.0, 97.6] 89.9% 89.7% [79.4, 100] 85.7% 5 Hits 49.0% [30.3, 67.8] 31.8% 30.7% [ 8.8, 52.5] 47.8% All of Mason's results fell within a 95% confidence interval for the mean and standard deviation of the revised ASD (calculated over 101 runs) with the exception of the FPs for Subject 2 (l0 = 2) which was 1.2% above the interval. From this result it seems reasonable to believe that the ASD evaluated here was achieving results similar to Mason's work. 4.3 ASD Tuning As outlined in Chapter 3 the delay in the system can be reducing by selecting a lower order filter for the feature extractor. In order to investigate the effects of changing the filter design on performance, features were extracted using Mason's filter, no filter (the data was already band limited from 0-64 Hz) and a variety of other filter designs. Several lengths were evaluated, including even orders (despite the delay not being exactly zero). The following filter design methods were used. 1. Specification of the filter response in the frequency domain. 2. Minimizing the squared error to the desired frequency response. 27 3. Minimizing the maximum ripple (equiripple design) using the Remez Exchange Algorithm. The features were extracted for all subjects using each of the filter designs. Codebooks were trained using the five algorithms described previously in Chapter 3 (LVQ1, LVQ2.1, LVQ3, DSLVQ, and Mason's LVQ) and the features were classified. In total this required 2525 runs (101 runs x 5 classifiers x 5 subjects) per filter. Use of a filter generally resulted in an improved accuracy and lower standard deviation. However, the relatively high order selected by Mason did not seem to be required and the low pass design (0-4 Hz passband) produced good results. Based on the preliminary analysis it was decided to proceed with four filter designs which included Mason's filter (121 order), no filtering (for comparison purposes), a Remez design (11 order), and a Least Squares (LS) design (17 order). The two new filters seemed to provide an acceptable trade off between minimum order and maximum performance. In addition to the filter and LVQ algorithm, there are several other parameters which affect performance. Table 4.2 lists parameters introduced in Chapter 3 which were tuned in this work. Although a systematic approach to this problem is challeng-ing due to the large inter-dependence in parameters, the effect of each was evaluated separately and is shown in Table 4.2. Mason selected an equal number of training vectors from each class resulting in a discriminant based on P(Idle)=P(Active). It was thought that selecting vectors from the training set at random (proportional training) would provide a better representation of the Bayesian decision surface. As expected, this generally resulted in a lower rate of false positives as well as decrease in hits. Overall, Mason's classifier had the largest variance in performance. L V Q l and LVQ3 typically performed best, with a slightly higher performance and variance from LVQ3. Many of the classifier parameters including the number of codebook vectors for each class, learning rate, number of iterations, and width of the window were unchanged from Mason's design. Experimenting with these parameters did not 28 Table 4.2: Effect of ASD Parameters Parameter Effect Feature Extractor Filter Design (order) Feature Separability Classifier LVQ Algorithm Idle Codebook Vectors Active Codebook Vectors Training Iterations Learning Constant: a[0] Window Constraint: w Training Vector Ratio by Class Number of Active Training Vectors Performance F P / F N Trade Off and Stability F P / F N Trade Off and Stability Stability and Performance Stability and Performance Stability F P / F N Trade Off Classifier Performance Decision Module Decision Module Design Mason Width of window: A Use of Temporal Information Required CSD for Active: lQ F P / F N Trade Off improve performance enough to justify the change. The decision module was also unchanged from Mason's work but /„ = 3 was selected to reduce false positives. Based on Mason's simulated subject training, mentioned briefly in Chapter 2, it was thought that the ASD performance may be improved if weak trials were discarded. Therefore, active vectors where the sum of the six features was less then 1.0 were not used in training the LVQ codebook. Table 4.3 shows the means ± one standard deviation for Subject 3 using this method and one particular set of parameters (shown as the revised ASD in Table 4.7). Similar tables were generated for the remaining subjects and all were compared to other runs where the parameters were varied. Figure 4.1 shows the frequency response of the LS (17) filter which generally performed best. Extracting the features with this filter generally resulted simultaneously in superior performance as well as a shorter system delay (62.5 ms) when compared with Mason's filter (469 ms). 29 Table 4.3: Revised ASD Performance for Subject 3 Classifier Correct None Mason (121) LS (17) Remez (11) L V Q l Idle Hits 97.3%± 0.4 48.4%± 6.2 97.5%± 0.3 44.3%± 3.3 96.5%± 0.3 64.6%± 2.8 96.9%± 0.4 60.2%± 4.1 LVQ2.1 Idle Hits 98.0%± 0.3 37.2%± 7.8 97.6%± 0.5 42.2%± 4.7 96.6%± 0.4 61.5%± 4.6 97.1%± 0.5 55.2%± 5.3 LVQ3 Idle Hits 97.4%± 0.6 45.9%±10.9 97.4%± 0.5 44.4%± 5.1 96.0%± 0.6 65.3%± 5.7 96.5%± 0.7 62.5%± 8.0 DSLVQ Idle Hits 97.3%± 0.8 47.9%±12.0 97.4%± 0.6 43.8%± 5.4 96.2%± 0.5 64.0%± 5.3 96.6%± 0.7 60.4%± 7.8 Mason Idle Hits 97.5%± 0.5 41.3%± 9.0 97.5%± 0.7 42.6%± 4.5 96.1%± 0.4 65.7%± 4.9 96.2%± 0.8 63.8%± 7.0 T 1 r F r e q u e n c y (Hz) 8oo r • i 6 0 0 k 4 0 0 h • ' ' ' i • 10 2 0 3 0 4 0 5 0 60 F r e q u e n c y (Hz) Figure 4.1: LS (17) Filter Frequency Response ° 2 0 0 L . o 30 Table 4.4: Comparison of Revised and Original ASD Performance Subject Revised Original Significance 1 Idle 1 Hits 97.1%± 1.2 27.1%± 6.9 87.3%± 6.1 21.8%± 8.9 p < 0.001 p < 0.001 2 Idle 2 Hits 94.4%± 0.6 75.0%± 5.1 94.5%± 2.0 49.6%± 3.3 p > 0.05 p < 0.001 3 Idle 3 Hits 96.1%± 0.5 64.5%± 5.2 96.3%± 2.1 43.5%± 7.4 p > 0.05 p < 0.001 4 Idle 4 Hits 95.5%± 0.7 40.5%± 4.5 94.4%± 2.4 37.3%± 4.9 p < 0.001 p < 0.001 5 Idle 5 Hits 95.0%± 1.0 38.1%±10.4 89.7%± 5.3 30.7%±11.2 p < 0.001 p < 0.001 In Table 4.4 the revised ASD was compared to the original results using Ma-son's filter and parameter selections with l0 = 3. The revised method showed a significant improvement (p < 0.001) in the mean percentage of correct classifications for all cases except Subjects 2 and 3 where a decrease in the FP rate was observed. However, these decreases were not statistically significant (p > 0.05). 4.3.1 D S L V Q as a Classifier As described in Chapter 3, DSLVQ is a modification of LVQ3 with an unequal weight-ing for each feature. In keeping with Mason's methodology of designing an ASD capable of detecting a generic VMRP, it is necessary to equalize the weights related to the electrode pairs over each of the SMA and the MI (shown in Figure 3.2). Two methods were attempted. The first approach allowed training to proceed normally and the average values were calculated over 101 runs. These means were then aver-aged by region (SMA and MI). The second method assigned wn[fc] the average value of d{[k] — dj[fc] by region in (3.16) at each iteration during training. The resulting weights were also averaged over 101 runs. Both methods produced almost identical results as shown in Table 4.5. The standard deviation for the case where the weights 31 Table 4.5: DSLVQ Comparison for Subject 3 Region: SMA MI Feature: 1 2 3 4 5 6 Normal DSLVQ Mean by Region Adjusted in Training 0.213 0.175 0.138 0.175 0.175 0.175 0.173 0.173 0.173 0.158 0.176 0.139 0.157 0.157 0.157 0.160 0.160 0.160 were adjusted during training was 0.012 for all features. In general DSLVQ produced results similar to those from LVQ3 both with the original weight vector as well as with the adjustment during training. The variance was generally larger, most likely due to the changing weights in addition to the normal variability in the algorithm. Therefore, use of DSLVQ in the ASD was rejected since the performance was not too different and did not consistantly improve on LVQ3. In addition, the algorithm is slightly more complex. 4.3.2 Number of Active Training Vectors It was necessary to estimate the number of active trials to be collected before training the on line system. The number of active vectors supplied to the training algorithm was varied. This was investigated using data from Mason's subjects as well as from pilot studies for the on line system described in the following chapter. It was estimated that collecting 25 artifact free movements from each subejct should provide reasonbale results. In addition, this number is consistant with the vectors used for training in the off line analysis. 4.4 Cross Subject Training Recording time-locked VMRPs from a subject with a motor disability will be signif-icantly more challenging since there is no actual movement to which the V M R P can 32 be synchronized. It would be advantageous if a codebook trained for an able bodied subject can be used initially, and possibly modified after further data collection. This idea was tested, although it must be noted that all of the subjects were able bodied. Codebooks were established based on the training data for each of Mason's subjects. An additional codebook was trained using the pooled data from all five subjects. These codebooks were then used to classify the test data from each subject. The revised ASD was used in this work. The mean ± one standard deviation over 101 runs for each case is shown in Table 4.6. The pooled training set in the far right col-umn generally produced better results than the worst case shown, but it seems that a better strategy may be to allow the subject to try several off the shelf codebooks and select the one which performs best. 4.5 Summary of ASD Modifications Table 4.7 contains a summary of the parameters found to work best based on the off line study using Mason's data. Along with the algorithms presented in Chapter 3 this completes the description of the ASD to be used for the on line system described in Chapter 5. 33 10 00 c o o -H -fl 06 Tt* 01 i—I co o i - H CM ' " 3 CO 0 5 LO c o i—H -H CM LO Oi Oi oo -H LO 00 CO LO 00 T-i T** -H -H T** >-! ^ cd LO 0$ 10 i—! oo -H -H 0 i - H CO CO 01 CM 00 i—l 0 co -H -H 01 CM CO CO Oi CO i—H CO -H -fl i—H oo oo oo l - H p i—H CO HH -tH CO CM l - H OI T** CO <3 10 TJH 00 O CO -H -H ^ J £ LO oo 00 o 01 i - H CO oo -H C- Oi CS LO I - H o i -H -H LO LO Tt* CD Oi LO I>- LO 0 T}* -H - H 1 LO LO LO O 01 Tt* CO CO 10 t>- i - H 0 CO -H -H r-^  c o 01 CM oq o -H i - H CO Oi oi +1 CM l - H co LO CM 0 L6 -H +1 i - H LO CO Tl* 01 CO l - H LO -H -H Oi Tl* CO o i Oi T1* CM CO 03 10 t> CO 0 Tl* -H +1 J £ as r>- oq CO i—H 01 c o CO o -H T** O T * * o -H -H oo oo O TJ* i—H CM -H -H LO LO 0 co 01 LO CO c3 10 CM Oi i—H CO -H -H i - H i - H Oi CM CO i - H -H LO Oi O Oi CM O It* Oi Oi oq i—H CO I -H -H CO T l * i—H Oi T l * o 1^ ICO -2 -2 T3 us r H i - H CO CO CU CO CM CM CO CO CD CO c o c o CO CO Tt* Tl* CO CO cu <u CU CU H EH rcu CP 34 Table 4.7: Summary of ASD Parameters Parameter Revised ASD L F - A S D General Sampling Rate 128 Hz 128 Hz Total System Delay 640.5 ms 1047 ms Feature Extractor Filter Design (order) LS (17) Mason (121) Delay 5i Table 3.1 Table 3.1 Frequency of Features Extraction 62.5 ms 62.5 ms Classifier LVQ Algorithm LVQ3 Modified LVQ3 Idle Codebook Vectors 3 3 Active Codebook Vectors 3 .3 Training Iterations 5000 5000 Learning Constant: a[0] 0.05 0.05 Learning Constant Decay Linear Linear LVQ3 Constant: e 0.2 0.2 Window Constraint: w 20% 20% Training Vector Ratio by Class Proportional Equal DSLVQ Learning Constant: k 0.1 N / A Number of Active Training Vectors 25 N / A Decision Module Decision Module Design Mason Mason Width of window: A 5 5 Required CSD for Active: l0 3 2 or 3 35 Chapter 5 On Line Validation An on line version of the ASD was developed and tested using an experiment similar to the one described by Mason [3]. During the data collection it is necessary to keep the participant involved in some task to minimize the chance that the shifts in attentiveness are detected rather than VMRPs. Mason used a simple video game for this purpose. 5.1 Experiment Design The participant was seated with eyes 100 cm from a black and white monitor with the display shown in Figure 5.1. Both balls contained within the box move at a constant and moderate speed of approximately 4 cm/s. The box occupies a visual angle of 3.5° from top to bottom in order to minimize eye movements. One ball moves freely within the rectangle bouncing off the walls and the other ball. The controlled ball is constrained to move along either a horizontal or vertical path passing through the center of the box. The subjects were asked to read the directions shown in Appendix B . l prior to the experiment. The participant wore an ElectroCap1 connected to a custom built isolated 1Electro-Cap International, Inc., Eaton, Ohio, USA. 36 Figure 5.1: Experiment Display bioelectric amplifier system [27]. The amplifiers have an input impedance greater than 1 GQ and a C M R R of approximately 112 dB at 60 Hz. The digital impedance meter built into the amplifier headbox was used to ensure that no electrode exceeded 5 Kfi and were generally all below 3 KQ. The analog Butterworth filters in the amplifier were set to a passband of 0.1-30 Hz (-12dB/octave) except for the E O G channel which was filtered between 1.0-30 Hz. A bipolar E O G channel measured the potential difference from an electrode placed below the right eye to a second electrode positioned to the right of the eye. All channels were sampled at 128 Hz by a PC equipped with a 12 bit Data Translations D T 2801-A analog to digital converter [28]. The participant also wore a glove custom built for Mason's experiment and described in [3]. The glove measured finger flexions, using piezoelectric sensors located over the knuckles. In order to deal with E O G artifact, the ASD would always result in an idle clas-sification for data acquired when the difference between the E O G electrodes exceeded ± 2 5 /J,V. Prior to each experiment the setting was verified by ensuring that blinks caused the software to switch to an artifact rejection mode. Processing resumed two 37 X-Windows Workstation 1 EEG Electrodes Operator *» EOG Electrodes Data Glove H EEG ^ Amplifiers Glove 1 L Amplifiers Rlhcmcl I ) ' l2x ( ) l - \ A / D Cmncr l iT T SOIIW.IIL' PC RS212 IVn Experimenter A Monitor l l . i i . l D^k Figure 5.2: Data Collection Hardware seconds after the voltage returned within the allowable limits. The A M D - K 6 233 MHz computer required approximately 60% utilization dur-ing the experiment, with a large percentage due to the graphics. A multitasking operating system (Linux 2.0.30) acquired and processed the data simultaneously. In addition, the software controlled a diagnostic display for the experimenter and the display for the subject which appeared on a second monitor. The data was stored on disk for optional post processing. The hardware block diagram is shown in Figure 5.2. A consistent sampling rate was achieved using the hardware clock on the A / D board which transfered samples to the PC over the ISA bus using D M A . This was verified by sampling a 16 Hz 100 aV peak to peak sine wave while the software was in full operation. The same sine wave was used to calibrate the offset for each channel and scale the signals to fiV. After calibration, the participant was was asked to perform self paced move-ments similar to those demonstrated by the experimenter. The movement was initi-ated with a flexion at the metacarpophalangeal joint followed by a trigger pull motion 38 which involved flexing the proximal and distal interphalangeal joints. A template was constructed by averaging the data glove output voltage from two finger flexions con-sidered acceptable upon visual inspection by the experimenter and approved by the subject as being reproducible. During the training phase the subject performed self paced movements and attempted to cause one of the balls to flash and change direction by reproducing a movement which correlated sufficiently with the template. This threshold was ad-justed slightly during the experiment in order to ensure that the movement was always challenging and may elicit more cognitive involvement from the subject. A random number generator initiated a break for the subject every four to seven movements. The classifier was trained after 25 movements free of ocular artifact were col-lected. The ratio of active to idle vectors aquired in the training set was thought to be representative of the testing phase of the experiment. Therefore, training vectors were selected randomly from either class since this method performed well in Chap-ter 3. From pilot studies it seemed reasonable to collect 75 movements during the testing phase in order to keep the data collection to about one hour and minimize the probability of fatigue affecting the results. The ASD continuously monitored and classified the E E G and the ball flashed and changed direction when a false positive or hit (correctly identified movement) occurred. The subject was instructed to try to reproduce movements which caused the ball to flash. During both stages of the experiment, the box border would flash to indicate to the subject which movements were ignored due to ocular artifact or because they were too soon after the last movement. This was intended to minimize confusion during training in cases where a movement which would have resulted in a hit was ignored. In Mason's experiment the feedback to the subject was always based on the data glove correlation and the E E G was post processed. To prevent contamination of the signals with visual evoked potentials, a one second delay was added between the 39 Table 5.1: Participant Information Subject Sex Age L A T 24-R | 1 Male 31 11 2 Male 23 12 f 0=Left Handed, 12=Right Handed movement and the start of the ball flashing. The delay was preserved in this work to allow for more flexibility if any post processing was required. Many of the mu-ERD BCI experiments involved 6-8 weeks of training with a small number of participants. It seems natural that a relatively long time period is required in learning to control these features since people are not generally skilled in controlling 8-12 Hz E E G power. In the case of a V M R P based system it is hy-pothesized that a much shorter training period will suffice. Mason [3] has already reported success without providing any feedback to the participant other than from the data glove. Able bodied participants are typically quite familiar with tasks such as an index finger flexion. The participant may learn to increase the separability or repeatability of the features as training progresses, resulting in lower error rates. The training period will likely be longer for participants with motor dysfunction, especially if the onset was not recent. Two subjects each participated in three sessions held during the same time slot every other day. Specific information for each participant is given in Table 5.1. Each session consisted of up to 30 minutes for setup and an explanation of the task followed by about one hour of self paced index finger flexions. The L A T 24-R [29] Handedness Inventory indicated that both subjects were strongly right handed. A sample of the forms and questionnaires completed by each subject are shown in Appendix B.2. The experiment was approved by the Department of Research Services at the University of British Columbia (B89-279) and each participant signed a copy of Appendix B.2 giving their informed consent to participate in the study. 40 Table 5.2: On Line ASD Per: Session Actual Subject 1 Subject 2 Class Idle Active Performance Idle Active Performance 1 Idle 5012 63 98.8% 2595 80 97.0% Active 57 18 24.0% 57 18 24.0% 2 Idle 8416 158 98.2% 1590 169 90.4% Active 41 34 45.3% 38 37 49.3% 3 Idle 7186 61 99.2% 1733 115 93.8% Active 61 14 18.7% 48 27 36.0% ormance 5.2 Results The performance of each subject is shown in Table 5.2. Samples contaminated by E O G artifact were not included in the results. Both scored 18 hits out of the 75 movements during their first session. The performance was often patchy where 4/5 movements may be detected followed by several misses. The movements were self paced so a different number of idle data points were collected from each subject. The false positive rates were quite low and often appeared in groups due in part to the design of the decision module window. From pilot studies it was decided to count a hit if an active classification appeared at, or up to 0.5 seconds after the trigger. Unexpectedly, Subject 2 produced several false positives approximately 1 second prior to the movement onset. In other trials the hits were at or just after the trigger. Occasionally, false positives were detected at other times during the experiment. The number of hits approximately doubled for each subject in the second session. The increase in FPs for Subject 2 was largely due to those which occurred about a second preceding the movement. Surprisingly, both subjects performed worse in the final session. It was not clear how an active output from the ASD should be interpreted near the trigger point. These points may have been false positives but were likely 41 related to movement planning. To get a different measure of the FP rate, a small investigation was added at the end of the third session. Subject 1 was instructed to watch the screen and not to make any movements. He was asked simple questions involving the addition or multiplication of single digit numbers and instructed to think of the answer but not to actually reply. Data was also collected during periods of silence. Only one cluster made up of four consecutive false positives was detected over a total collection time of 110 seconds. Data containing ocular artifact was not classified or counted towards the 110 seconds. The codebook from session 2 was used in this exercise since it was thought that session 3 was too biased in terms of rejecting false positives. The same experiment was repeated with Subject 2 using the codebook from the third session. However, idle data was only correctly classified in 88.6% of the samples, The false positives appeared in small patches and too frequently for the ASD to be practical for most tasks. Note that this data was collected at the end of the session and the subject was blinking frequently (often every 2-3 seconds) so fatigue may have been a factor. 5.3 Participant's Comments Both participants had many similar comments about the ASD. At one point both were frustrated that movements were not being detected and found that nothing seemed to work while they felt this way. Part way through the first day, both subjects reported that they had developed a reasonably good sense of which movements would be detected before receiving feedback from the ASD. Both found that the ASD worked better when they would really concentrate on making the movement. However, in the third session, Subject 1 reported that movements which he expected to work were not usually detected and the VMRPs which were recognized were those he did not expect. He found this confusing and was unable to adapt to the new codebook in the 42 75 movements collected. Both subjects reported that many of their false positives were detected just when they were thinking about making a movement. This appeared to happen far too often to be due to chance, but this phenomena was not investigated further. However, it provides some very preliminary evidence that it may be possible to activate the ASD simply by planning a movement and that proprioceptive feedback may not be required. Subject 1 did not seem particularly interested in the position of the balls moving about the box. He usually watched the ball which he could control and performed self paced movements. Alternatively, Subject 2 appeared to pay a lot of attention to the display and felt that it was much easier to produce a detected movement when the balls were close and he could cause them to collide if the direction were changed. Neither subject felt that their attention level varied greatly, but no quantitative measure was used to verify this claim. 5.4 Post Processing Post processing the third session for Subject 1 with the codebook trained during the second session resulted in a hit rate of 38% with 98.4% of the idle E E G classified correctly. This indicates that the poor performance during this session is most likely due to the codebook used on that day. They subject would probably have achieved even more hits with this codebook had the feedback not been confusing. Similarly, applying the codebook from the second session also improved the performance of Subject 2 with 45.7% hits and 92.4% correctly classified idle. The second day for Subject 2 was reprocessed with l0 = 2 and the hits increased to 67.5% but only 84.3% of the idle was correctly classified. When this was repeated for Subject 1 a hit rate of 49.2% resulted with 96.2% of the idle vectors correct. The idle data collected from Subject 2 at the end of the third session was 43 reprocessed using the codebook from session 2. There was a small decrease in false positives (1.8%) but they still occurred far too often to be considered acceptable. 5.5 Discussion One possible explanation for the higher false positive rate of Subject 2 may be that in attending to the display he would begin to plan a movement as the balls approached each other. The result could be an earlier active classification by the ASD as compared to Subject 1 who mostly ignored the second ball and moved when he felt ready. This would also account for the FPs which occurred about a second before movements made by Subject 2. The decrease in hits for both subjects during the third session may be at-tributed any one of several factors. Training a new LVQ codebook from scratch may have caused confusion once they started to develop some proficiency with the previous system. A better solution may have been to slightly fine tune the codebook from the previous session using a small learning constant or to leave it alone. During training the subjects seemed to have a different strategy. As seen in Table 5.3 the number of active training vectors for Subject 2 decreased as the sessions progressed while they increased for Subject 1. Subject 1 seemed to take more and more time in trying to produce good training movements. Subject 2 took less time probably due to a higher level of proficiency with the task as well as in estimating how much time to wait after blink or in between movements without receiving a warning for moving too soon. The small number of active training vectors in session 3 for Subject 1 is likely a part of the reason that the FPs were lowest on this day and so many of the movements were missed. To test this theory, an additional 25 movements in session 3 were collected using the codebook from session 2 immediately following the scheduled data collection. At first the subject seemed quite able to remember what had been done on that day 44 Table 5.3: Vectors Selected for On Line LVQ Training Session Subject 1 Subject 2 Idle Active Idle Active 1 2 3 4836 164 4886 114 4945 55 4713 287 4336 664 3961 1039 as he was initially successful in 7/10 movements. However, the next 15 movements only resulted in one additional hit. This brings up the possibility that the subject was simply less interested in the task and was not putting as much thought into the movements on this day. The unexplained high FP rate in the idle data for Subject 2 which was collected after the final session remains. This result was unexpected since false positives seemed to occur more frequently in this data then during the active experiment. Many factors such as fatigue, level of attention for the idle task, or a conditioned response to planning movements when the balls were in certain positions may have contributed, but further investigation will be required to explain this result. One additional possibility was considered in explaining the large FP rate for this data. Since Mason observed higher rates of FPs in the second both before and after the movement, the pre and post movement features were not used as part of the idle training set. However, it was observed that Subject 2 often had FPs approximately one second before the movement. It is likely that several of the vectors resulting in FPs were collected during training and that the idle training set was contaminated by FPs occurring slightly more than one second before the movement. As shown in Table 5.3 the number of idle training points collected for Subject 2 decreased over the three sessions. It is possible that the few pre movement FPs pulled the codebook vectors too close to the active case resulting in the higher FP rate observed. One possible solution may be to discard a longer segment of the pre movement data during training. 45 Chapter 6 Conclusions The asynchronous signal detector was successful in identifying index finger flexions both off line using Mason's data as well as on line when two new subjects were given feedback based on the ASD output. A revised version of the ASD increased the average number of detected movements (hits) from between 3.2% to 25.4% (p < 0.001) when Mason's subjects were processed off line. False positive errors decreased between 1.1% and 9.8% (p < 0.001) for three of the subjects and no significant change (p > 0.05) occurred for the other two. The revised ASD also reduced the processing delay by 39% to 640.5 ms by substituting a lower order filter design for the feature extractor. Two subjects participated in an on line evaluation of the revised ASD. The system identified 24% of the finger movements made by both participants during their first session which increased to 45.3% and 49.3% during the second session. This preliminary investigation indicated that subject training may occur quite rapidly as compared to E R D based BCIs where many of the subjects improved slowly over several weeks. The more rapid effect is likely due to the subject's familiarity in controlling finger flexions as compared with the more abstract idea of modifying the power in specific frequency bands of their E E G . However, the small sample size does not provide any conclusive evidence and further investigation is recommended. 46 Performance decreased from the second to the third session for both Subject 1 and Subject 2 with 18.4% and 36% hits respectively. The results improved when the data was classified using the LVQ codebook from session 2 which suggests that the decrease may be explained by the classifier converging differently on this day and confusing the participants already accustomed to a different movement. It was concluded that future work should proceed with small updates in the codebook once the subject demonstrates a reasonable level of proficiency in using the ASD. This work also addressed the problem of training an ASD for individuals with limited motor control who cannot provide time-locked training vectors. The off line evaluation of Mason's subjects indicated that a codebook trained for one subject may be used on others, although the performance is generally not as good. In addition, all of the results are based on delay parameters estimated from Mason's Subject 3 so customized delays do not seem essential although Mason found that customization improved performance. From these results and some preliminary evidence of rapid subject training it may be feasible to apply an off the shelf codebook and allow subject training to compensate for the sub optimal parameters. From the algorithm described in Chapter 3 it seems feasible to implement the ASD using a single Digital Signal Processor (DSP) making the system both inex-pensive and portable. Six differential (bipolar) E E G amplifiers, one bipolar E O G channel, and analog to digital converter(s) would also be required. Overall, the revised ASD performed well but an investigation with a larger subject group is required to generate statistically significant results and properly evaluate subject training effects. This preliminary investigation has demonstrated promising results which indicate that the methods explained in this report present a feasible solution for the single trial detection of index finger flexions from surface electrodes. 47 .1 Summary of Contributions 1. Verification of Mason's work using the original data set followed by further validation on E E G collected from two new subjects. 2. Modifications and tuning of the classifier resulting in improved system perfor-mance. 3. Mason's 121 order filter previously used in the feature extractor was replaced by a 17th order filter. This change reduced the overall delay required to process E E G by 39% and simultaneously resulted in superior performance when the features were classified. The total delay from movement onset is 640.5 ms. 4. Preliminary evaluation indicating that rapid subject training may occur. 5. An investigation of cross subject training which addressed the problem of train-ing a classifier for a subject who cannot provide time-locked movements due to a motor related disability. This issue may be resolved using a classifier tuned for an able bodied subject although some tuning will be required to achieve optimal performance. However, this evaluation makes the assumption that a disabled subject produces similar V M R P patterns. 6. The on line implementation provides a tool which allows for further evaluation of the ASD in order to answer several of the issues addressed in the following section on directions for future research. 48 2 Suggested Future Work 1. Identify a training algorithm which makes suitable updates to the codebook either after each session or adaptively on line. The characteristics should not change rapidly enough to confuse the BCI operator. 2. Evaluate the performance of the ASD in the detection of movements other then a right hand index finger flexion. 3. Test several subjects with various motor related disabilities to determine their ability to produce the same detectable V M R P patterns. 4. Combine the ASD with a suitable CSC for an on line evaluation of the system's ability to distinguish between multiple control signal such as a left hand or right hand index finger flexion. 5. Investigate methods for the removal of ocular artifact. The current system is too restrictive if the subject is required to make eye movements in order to receive visual feedback for a specific task. 6. Use DSLVQ to evaluate the information content of other electrode pairs, possi-bly using the same 9 monopolar signals to create a symmetric montage including electrodes placed laterally or diagonally in addition to the anterior-posterior combinations selected by Mason. 7. Investigate the ratio between the elemental features E\ and E2 (as described in Chapter 3) at the trigger point. False positives may be reduced by adding a constraint which requires a specific range for Ei/E2 in order for an active classification to result. 8. A study involving a larger number of participants may lead to a better under-standing of subject training and other issues addressed in Chapter 5. 49 References [1] F. Shein, G. Hamannand N. Brownlow, J. Treviranus, M. Milner, and P. Parnes. Wivik: A visual keyboard for windows 3.0. In Proceedings of the 14-th Annual Conference of RESNA, pages 160-162, 1991. [2] T. M. Vaughan, J. R. Wolpaw, and E. Donchin. EEG-based communication: Prospects and problems. IEEE Transactions on Rehabilitation Engineering, 4(4):425-430, December 1996. [3] S. G. Mason. Extraction of Single-Trial Index Finger Flexions From Continuous Spatiotemporal EEG. PhD thesis, University of British Columbia, Faculty of Ap-plied Science, Department of Electrical Engineering, 2356 Main Mall, Vancouver, B.C. , V6T 1Z4, C A N A D A , November 1996. [4] E . E . Sutter. The brain response interface: Communication through visually-induced electrical brain responses. Journal of Microcomputer Applications, 15:31-45, 1992. [5] D. Ff. Lange and G. F. Inbar. A robust parametric estimator for single-trial movement related brain potentials. IEEE Transactions on Biomedical Engineer-ing, 43(4):341-347, April 1996. [6] D. H. Lange, Ff. Pratt, and G. F. Inbar. Modeling and estimation of single evoked brain potential components. IEEE Transactions on Biomedical Engineering, 44(9):791-799, September 1997. [7] M. Hansson, T. Gansler, and G. Salomonsson. Estimation of single event-related potentials utilizing the prony method. IEEE Transactions on Biomedical Engi-neering, 43(10):973-981, October 1996. [8] G. E. Birch, P. D. Lawrence, and R. D. Hare. Single-trial processing of event-related potentials using outlier information. IEEE Transactions on Biomedical Engineering, 40(l):59-73, January 1993. 50 [9] D. Lisogurski. Real time hardware for single-trial processing of E E G . Technical Report (Unpublished), December 1997. [10] J. R. Wolpaw, D. J. McFarland, G. W. Neat, and C. A. Forneris. An EEG-based brain-computer interface for cursor control. Electroencephalography and clinical Neurophysiology, 78(3):252-259, March 1991. [11] D. J. McFarland, G. W. Neat, R. F. Read, and J. R. Wolpaw. An EEG-based method for graded cursor control. Psychobiology, 21(1):77-81, 1993. [12] J. R. Wolpaw and D. McFarland. Development of an EEG-based brain-computer interface. In Proceedings RESNA 1995 RECREAbility, pages 645-649, 1995. [13] G. W. Neat, D. J. McFarland, C. A. Forneris, and J. R. Wolpaw. EEG-based brain-to-computer communication: System description. Proceedings IEEE En-gineering in Medicine and Biology Society, 12(5):2298-2300, 1990. [14] J. R. Wolpaw and D. J. McFarland. Multichannel EEG-based brain-computer communication. Electroencephalography and clinical Neurophysiology, 90:444-449, 1994. [15] D. Flotzinger, J. Kalcher, and G. Pfurtscheller. E E G classification by learning vector quantization. Biomedizinische Technik, 37(12) :303-309, December 1992. [16] J. Kalcher, D. Flotzinger, C. Neuper, S. Golly, and G. Pfurtscheller. Graz brain-computer interface II: Towards communication between humans and computers based on online classification of three different E E G patterns. Medical and Bio-logical Engineering and Computing, 34(5):382-388, September 1996. [17] I. Ivanova, G. Pfurtscheller, and C. Andrew. Al-based classification of single-trial E E G data. In Engineering in Medicine and Biology Society, IEEE 17th Annual Conference, pages 703-704, 1995. [18] M . Pregenzer, D. Flotzinger, and G. Pfurtscheller. Distinction sensitive learning vector quantization — A new noise-insensitive classification method. In IEEE International Conference on Computational Intelligence, volume 5, pages 2890-2894, 1994. [19] M . Pregenzer, G. Pfurtscheller, and D. Flotzinger. Selection of electrode positions for an EEG-based brain computer interface (BCI). Biomedizinische Technik, 39(10):264-269, October 1994. 51 [20] G. Pfurtscheller, J. Kalcher, C. Neuper C, D. Flotzinger, and M . Pregenzer. O n -line E E G classification during externally-paced hand movements using a neural network-based classifier. Electroencephalography and clinical Neurophysiology, 99(5):416-425, November 1996. [21] D. Flotzinger, M. Pregenzer, and G. Pfurtscheller. Feature selection with dis-tinction sensitive learning vector quantization and genetic algorithms. In IEEE World Congress on Computational Intelligence, pages 3448-3451, 1994. [22] G. Pfurtscheller, D. Flotzinger, M . Pregenzer, J. R. Wolpaw, and D. McFar-land. EEG-based brain computer interface (BCI). Search for optimal electrode positions and frequency components. Medical Progress Through Technology, 21(3):111-121, 1995-1996. [23] A. Kubler, B. Kotchoubey, Ff. P. Salzmann, H. Schleichert, and N. Birbaumer. Operant self-regulation of the brain without motor mediation. Psychophysi-ology:Abstracts of the Thirty-Seventh Annual Meeting, 34(Supplement 1):S55, August 1997. [24] P. Herberts, C. Almstrom, R. Kadefors, and P. D. Lawrence. Hand prosthesis control via myoelectric patterns. Acta Orthopaedica Scandinavica, 44(4-5) :389-409, 1973. [25] H. H. Jasper. The ten-twenty electrode system of the International Federation. Electroencephalography and clinical Neurophysiology, 10:371-375, 1958. [26] T. Kohonen. The self-organizing map. Proceedings of the IEEE, 78(9):1464-1480, September 1990. [27] SA Instrumentation Company. A Custom-Built Research System for Univer-sity of British Columbia Department of Psychology. Unpublished, San Diego, California, USA, 1997. [28] Data Translation. DT2801 Series User Manual. Data Translations, 100 Locke Drive, Marlboro, MA, 1992. [29] S. Coren. The Left-Hander Syndrome: The Causes and Consequences of Left Handedness. Free Press/Macmillan, New York, 1992. 52 Appendix A List of Acronyms A / D Analog to Digital ASD Asynchronous Signal Detector BCI Brain Computer Interface CSC Control Signal Classifier CSD Control State Density DSLVQ Distinction Sensitive Learning Vector Quantization DSP Digital Signal Processor E E G Electroencephalograph E M G Electromyograph E O G Electrooculargraph E R D Event Related Desynchronization ERP Event Related Potential ERS Event Related Synchronization F N False Negative FP False Positive LF-ASD Low Frequency Asynchronous Signal Detector LS Least Squares LVQ Learning Vector Quantization MI Primary Motor Area NN Nearest Neighbor O P M Outlier Processing Method SMA Supplementary Motor Area V E P Visual Evoked Potential V M R P Voluntary Movement Related Potential 53 Appendix B Subject Information and Consent Forms This appendix contains copies of the instructions and forms (based on Mason's ex-periment [3]) given to each subject. B . l Subject Instructions • A right handed index finger flexion will be demonstrated and you will be asked to make the same movement. The glove will determine if your movement is a good enough match. • The box on the screen is now covered, but will contain two moving balls once the experiment begins. One ball will move either horizontally or vertically. After each finger movement there will be about a one second delay. If the movement was good then the ball will start to flash and then change direction soon after. • After several movements the box will be covered again and you can take a break. It is preferred that you try to delay talking or moving (other then your finger) until the break if possible. • If you make a movement after a blink or too soon after another movement then the border will flash. Wait at least 5 seconds after the double border returns before the next finger movement. Try not to blink around the time of a movement but don't worry if this happens. • Try to time your movements to make the two balls collide, however the main goal is to reproduce a good movement which makes the ball flash. Concentrate more on making a few good movements rather then many. 54 B.3 Subject Information Form Subject Information Form E X T R A C T I O N O F M O T O R - R E L A T E D P O T E N T I A L S F R O M S I N G L E - T R I A L E E G Name Address Phone Number ( ) " Sex M / F Age Neurological Disorders: MS, ALS, Epilepsy, ... Medications: anti-depresent, anti-anxiety, anti-histomene, ... Head Injuries: length of unconsciousness, lasting effects, age at injury A r m / H a n d / S h o u l d e r : breaks, sprains, operations, flexion impediments Pre-Experiment Consumption: smoking, coffee, meals, alcohol Other: sleep, alertness, general anxiety, stress Dexterity special skills: piano, guitar, attitude (perfectionist) Please do not write below this line. Subject # Day 1 Day 2 Day 3 Experiment Date Start time End time Notes: 56 .4 LAT-24 R Handedness Inventory [29] «1> ftt tU <u to I I I ] .»= JH J= **- £ w - £ « • — ' * E u tu n ^ ^ - «- — <9 w «<i t* n > *«* TJ — c * o #- • -P — » T> => *# *W 4U 4> O *2 "> ( A M « -CI * * A C < V U J X k C — CJ w M L . J« «  1 4 ^ . - Q * rri — >^  i. C *» b r 9 ai *a 3 C O k- W 1? JC i— •» K 5 C 3 e o o «*- 9 o p « • S a c 3* *- w . L . *_ — 5? £5- =• 3 =» 3 1 X» Q o 9 a 2 5 5 ** *> *• I 5 ^ £ i? £• ? ~ ^ E 3 if e J* * * 3 » * £ £. O O o e> S S X 31 K X jo a 00 o «> Q ft U 4* * i < : C: I ? o a >» *v o o -a 11 1 1 I I 1 1 1 1 1 1 1 1 1 I 11 1 I I 1 1 1 "l "l I I | 1 1 1 1 1 1 } 1 I 1 1 ! 1 1 l i 1 1 1 1 1 1 1 I 1 1 1 1 1 I. i 1 1 1 % 1 I I I I o u 4, . 2 ^ 2 o"B 3 3 & • * ^, *   & ZM *f , O O >t U U. ' - "O •'O r~ 1m rm i v * r => « ut 1 K rm O *-» a, -*= ^: x-" Xl Xi ^ ^ ^ ^ JZ q**j ^ O ^ C O k. n ^ 4 * -- t Q ^ rf! ^ •* * B _ — e * * Jt — ^ f O 1 — 3fe ^ «• w. U * * a B«j >> £ m 3 3 C7 3 •n O =» >1 O * D £ O ^ ^ •— O *> J= M * > L/ .C U «-• C r« f -it. ^ C u : ,7 w ' a g : *» f= o O - = * -»4 *• ? C "c ! 0> rm ' <= — LT XS r--" •* rm O ^; fc_ U*-> O CJ fa l - w o. -4 O c n r' 01 H ft U &,= g| S 3 : - g -i s =• g.: s 2 - S S : *•* -E T » — "to 6 r L O > C < U c a f e - * ) q> 5 T> 1 "QC JZ C <_ C U *^ C U * • C« <r- o 4 = • • o l «J •-• c d (S a A . -j * U •I r— •* A O D ~> TJ TJ Q •« r— l»- O — 3 * J — 0> — 3 * > — «i — o - C k - -tz C C 10 IT C JZ J? J= O C «r-C s>- a ^ - o ^ • w e ^ - O 5 > » »-< » — W »j o a? r-. «r vi 3 •> o •— TJ -a » » - >> M «J T » US • L> © J= C 3 41 a C L J< Tl ISO £ D M D U + J | -T3 ZI a> =• o C => ft ^z V O — V- o -T^ -«Z V . u 57 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items