Applied Science, Faculty of
Electrical and Computer Engineering, Department of
DSpace
UBCV
Zhang, Wenwen
Master of Applied Science - MASc
University of British Columbia
The pervasiveness of wearable sensors has contributed to plenty of daily activity data and greatly improved the ability to link data analytics with healthcare applications. Gait analysis is an important part of the healthcare field to help monitor the patient’s progress with gait disturbances such as Stroke and Parkinson’s. The traditional method to get gait parameters is the GaitRite walkway, which is cumbersome and requires professional training in setting up every time. With the assistance of wearable sensors, we are able to record the motion of the foot without those limitations. Zero Update Position and Timing (ZUPT) has been used to analysis of normal people’s walking for a long time. However, ZUPT has been validated on the walking of normal people only. In this article, we test the ZUPT on pathological patients. We compare the results of normal and abnormal walking patterns in terms of step recognition and gait parameter prediction accuracy. To help track and monitor patient disease progress with better accuracy, we propose Machine Learning (ML) algorithms to improve the accuracy of healthcare application estimation results based on the time series data. Those patients also suffer from gait impairments which may cause fatal dangers such as falls. In the meantime, gait parameters are also significant signs of the progress of the disease. To monitor the gait condition, we collect datasets with different gaits, comparing and predicting gait parameters for the patients. Compared to the ZUPT algorithm, the final prediction from ML algorithm of gait parameters achieves greater performance.
Machine Learning based Gait Parameter Analysis forNormal and Pathological PatternsbyWenwen ZhangBsc. Electric Engineering, Tianjin University, 2020A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS FOR THE DEGREE OFMaster of Applied ScienceinTHE FACULTY OF GRADUATE AND POSTDOCTORALSTUDIES(Electric and Computer Engineering)The University of British Columbia(Vancouver)April 2023© Wenwen Zhang, 2023The following individuals certify that they have read, and recommend to the Fac-ulty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled:Machine Learning based Gait Parameter Analysis for Normal and Patho-logical Patternssubmitted by Wenwen Zhang in partial fulfillment of the requirements for the de-gree of Master of Applied Science in Electric and Computer Engineering.Examining Committee:Prof. Peyman Servati, Professor, Electric and Computer Engineering, UBCSupervisorProf. Calvin Kuo, Assistant Professor, Biomedical Engineering, UBCSupervisory Committee MemberProf. Jane Z. Wang, Professor, Electric and Computer Engineering, UBCSupervisory Committee MemberiiAbstractThe pervasiveness of wearable sensors has contributed to plenty of daily activitydata and greatly improved the ability to link data analytics with healthcare applica-tions. Gait analysis is an important part of the healthcare field to help monitor thepatient’s progress with gait disturbances such as Stroke and Parkinson’s. The tradi-tional method to get gait parameters is the GaitRite walkway, which is cumbersomeand requires professional training in setting up every time. With the assistance ofwearable sensors, we are able to record the motion of the foot without those limi-tations.Zero Update Position and Timing (ZUPT) has been used to analysis of normalpeople’s walking for a long time. However, ZUPT has been validated on the walkingof normal people only. In this article, we test the ZUPT on pathological patients.We compare the results of normal and abnormal walking patterns in terms of steprecognition and gait parameter prediction accuracy.To help track and monitor patient disease progress with better accuracy, wepropose Machine Learning (ML) algorithms to improve the accuracy of healthcareapplication estimation results based on the time series data. Those patients alsosuffer from gait impairments which may cause fatal dangers such as falls. In themeantime, gait parameters are also significant signs of the progress of the disease.To monitor the gait condition, we collect datasets with different gaits, comparingand predicting gait parameters for the patients. Compared to the ZUPT algorithm,the final prediction from ML algorithm of gait parameters achieves greater perfor-mance.iiiLay SummaryThis article discusses the use of wearable sensors in healthcare for gait analysis,specifically for patients with gait disturbances such as Stroke and Parkinson’s.Wearable sensors offer advantages over traditional methods like GaitRite walk-ways, as they are less cumbersome and require no professional training. The articleexplores the use of a ZUPT algorithm for step recognition and gait parameter pre-diction accuracy in normal and pathological patients. To improve accuracy, the ar-ticle proposes machine learning algorithms based on time-series data. The datasetscollected show that the machine learning algorithm outperforms the ZUPT algo-rithm in predicting gait parameters for patients. The use of wearable sensors andmachine learning algorithms can greatly improve the monitoring and treatment ofpatients with gait impairments, which can cause fatal dangers such as falls.ivPrefaceThis dissertation is an original intellectual product of the author, Wenwen Zhang.The fieldwork reported in Chapter 3 was covered by UBC Ethics Certificate num-ber H21-02052.The research showcased in this thesis has been carried out at the Flexible Elec-tronics and Energy Lab (FEEL), situated within the Department of Electrical andComputer Engineering at the University of British Columbia. Dr. Peyman Servatihas overseen all of the research endeavors. Dr. Calvin Kuo has supervised thework on the gait analysis in Chapter 3. The data is collected with GaitRite systemsupported by Vancouver Coastal Hospital and Dr. Calvin Kuo.Chapter 3 focus on the normal and abnormal gait analysis, especially on thegait disturbance scenarios of Stroke and Parkinson’s patients. We compared theperformance of traditional ZUPT and ML performance in terms of gait parameteranalysis. We described the details of the gait analysis results for each participant.Publications and PresentationsThe work in Chapter 3 is presented at the 2022 BMES Annual Meeting Copy-right ©2023 - Biomedical Engineering Society.W. Zhang, C. Kuo, and P. Servati1 A Wearable Sensor System for MeasuringPathological Gait Parameters ..In the Chapter 3, W. Zhang contributes to ethics preparation, data collection,data pre-processing, and machine learning model training. C. Kuo helps with ethicspreparation, data collection, and data process procedures. C. Kuo and P. Servatiadvise on the project.Other contributions: A. Tashakori, W. Zhang, Z. J. Wang, P. Servati SemiPFL:vPersonalized Semi-Supervised Federated Learning Framework for Edge Intelli-gence.W. Zhang contributes to the baseline model training and data preparation. A.Tashakori proposed the hyper network and autoencoder. Z. J. Wang and P. Servatiadvise on the project.viTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiList of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 AI and Personalized Healthcare . . . . . . . . . . . . . . . . . . . 11.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.1 Wearable Electronics . . . . . . . . . . . . . . . . . . . . 21.2.2 Personalized Healthcare . . . . . . . . . . . . . . . . . . 31.3 Organization and Structure . . . . . . . . . . . . . . . . . . . . . 32 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1 Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 5vii2.1.1 Supervised Learning . . . . . . . . . . . . . . . . . . . . 62.1.2 Unsupervised Learning . . . . . . . . . . . . . . . . . . . 62.1.3 Semi-supervised Learning . . . . . . . . . . . . . . . . . 72.2 Federated learning . . . . . . . . . . . . . . . . . . . . . . . . . 82.3 Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.4 Convolutional Neural Network . . . . . . . . . . . . . . . . . . . 102.5 Recurrent Neural Networks . . . . . . . . . . . . . . . . . . . . . 152.6 Long Short Term Memory . . . . . . . . . . . . . . . . . . . . . 162.7 Unity Real-Time Development Platform . . . . . . . . . . . . . . 183 Gait Analysis in Pathological Gait . . . . . . . . . . . . . . . . . . . 203.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.2 Data Collection Process . . . . . . . . . . . . . . . . . . . . . . . 203.2.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . 203.2.2 Measurement Protocols . . . . . . . . . . . . . . . . . . . 213.3 Sensor Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.3.1 Normal Pattern . . . . . . . . . . . . . . . . . . . . . . . 233.3.2 Pathological Gait . . . . . . . . . . . . . . . . . . . . . . 243.4 Data pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . 263.4.1 Scale and drift . . . . . . . . . . . . . . . . . . . . . . . 263.5 ZUPT Performance on Pathological Gait . . . . . . . . . . . . . . 283.5.1 Step Segmentation . . . . . . . . . . . . . . . . . . . . . 283.6 Gait Parameter Prediction . . . . . . . . . . . . . . . . . . . . . . 373.6.1 Gravity Subtraction . . . . . . . . . . . . . . . . . . . . . 373.6.2 Velocity and Displacement Update . . . . . . . . . . . . . 393.6.3 Machine Learning Predicted Gait Parameters . . . . . . . 484 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . 514.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53A Gait Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61viiiA.1 MATLAB Program . . . . . . . . . . . . . . . . . . . . . . . . . 61A.2 Statistical Results for Each Participant . . . . . . . . . . . . . . . 63ixList of TablesTable 3.1 Statistical results of missing steps . . . . . . . . . . . . . . . . 32Table 3.2 Statistical results of overcounted steps . . . . . . . . . . . . . 32Table 3.3 Statistical results of Heel Strike Event (HS) difference in second 33Table 3.4 Statistical results of Toe Off Moment (TO) difference in second 33Table 3.5 Statistical results of stride length difference (absolute value) incentimeter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Table 3.6 Statistical results of stride length difference (absolute value) incentimeter ([cm]) (ML method), and improvement compared toprevious method Table 3.5. . . . . . . . . . . . . . . . . . . . 49Table A.1 Statistical results of missing steps for each participant . . . . . 64Table A.2 Statistical results of overcounted steps for each participant . . . 65xList of FiguresFigure 2.1 A simple example to demonstrate how semi-supervised learn-ing is possible [12]. Reprinted with permission © 2023 SpringerNature Switzerland AG. Part of Springer Nature. . . . . . . . 7Figure 2.2 Typical Federated Learning (FL) client-server architecture [25].Copyright ©2020 by Morgan & Claypool. . . . . . . . . . . . 8Figure 2.3 Horizontal Federated Learning (HFL) structure [25]. Copy-right ©2020 by Morgan & Claypool. . . . . . . . . . . . . . . 9Figure 2.4 The ongoing discrete Kalman filter cycle. The time updateprojects the current state estimate ahead of time. The mea-surement update adjusts the projected estimate by an actualmeasurement at that time [16]. Copyright ©NC-Chapel Hill,2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Figure 2.5 A complete picture of the operation of the Kalman filter [16].Copyright ©NC-Chapel Hill, 2006 . . . . . . . . . . . . . . . 11Figure 2.6 The structure illustration of CNN [48]. Copyright © arXiv, 2015 13Figure 2.7 The workflow of a CNN [13, 14]. Reprint with © 2023 SpringerNature Switzerland AG and Copyright © arXiv, 2016. . . . . 13Figure 2.8 Common types of activation function [13, 49]. Reprint withCopyright © 2023 Springer Nature Switzerland AG and Copy-right © arXiv 2016 . . . . . . . . . . . . . . . . . . . . . . . 15Figure 2.9 The structure of RNN [15, 54]. Reprint with Copyright © 2019Elsevier B.V. and Copyright © arXiv, 2017 W.Y . . . . . . . . 16xiFigure 2.10 (a). The structure of LSTM; (b) The internal workflow ofLSTM cell [56]. Reprint with Copyright © 2023 IEEE. . . . . 17Figure 3.1 (a). GAITRite system. (b). Inertial Measurement Unit (IMU)setup. (c). Foot-mounted IMUs. . . . . . . . . . . . . . . . . 22Figure 3.2 IMU signals in normal walking of left (a) and right foot (b). . . 23Figure 3.3 IMU raw signals in normal walking of left (a) and right foot (b). 24Figure 3.4 IMU signals for Parkinson gaits of left (a) and right foot (b)with ground truth data from GAITRite system. . . . . . . . . 25Figure 3.5 IMU signals for Stroke gaits of (a) normal foot and (b) draggingfoot with ground truth data from GAITRite system. . . . . . . 26Figure 3.6 Left and right IMU signal with z axis facing (a). downward.(b). upward . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Figure 3.7 IMU signals for normal gaits of (a) left foot and (b) right footwith ground truth data from GAITRite system and estimatedvalue from ZUPT algorithm. (c). Zoom in on details for onenormal walking step. . . . . . . . . . . . . . . . . . . . . . . 29Figure 3.8 IMU signals for Parkinson gaits of (a) left foot and (b) right footwith ground truth data from GAITRite system and estimatedvalue from ZUPT algorithm. (c). Zoom in on details on missingsteps of Parkinson’s case. . . . . . . . . . . . . . . . . . . . 29Figure 3.9 IMU signals for Stroke gaits of (a) normal foot and (b) drag-ging foot with ground truth data from GAITRite system andestimated value from ZUPT algorithm. (c). Zoom in on detailsof normal foot patterns. (d). Zoom in on details of draggingfoot pattern with late TO prediction. . . . . . . . . . . . . . . 30Figure 3.10 (a). Step count errors are normalized by correctly recognizedsteps. (b). TO and HS events time difference in second unit. . . 31Figure 3.11 Missing step counts normalized by correctly recognized stepsfor each participant. . . . . . . . . . . . . . . . . . . . . . . . 34Figure 3.12 Over counted step errors normalized by correctly recognizedsteps for each participant. . . . . . . . . . . . . . . . . . . . . 35xiiFigure 3.13 Functional divisions of the gait cycle [66]. Copyright ©2015Sto¨ckel, Jacksteit, Behrens, Skripitz, Bader and Mau-Moeller. 36Figure 3.14 (a). Normal walking pattern with low thresholds and con-nected steps. (a). Normal walking pattern with high thresholdsand split steps. . . . . . . . . . . . . . . . . . . . . . . . . . 36Figure 3.15 Quaternions of sensor data fusion for a normal walking trial. . 37Figure 3.16 Quaternions of sensor data fusion for a normal walking trialaffected by drift problem. . . . . . . . . . . . . . . . . . . . . 38Figure 3.17 Comparison of acceleration before gravity subtraction (a) andafter gravity subtraction (b). . . . . . . . . . . . . . . . . . . 40Figure 3.18 Integration results of velocity and displacement from acceler-ation. (a) Velocity (b) Displacement. . . . . . . . . . . . . . . 41Figure 3.19 Calculated gait parameters. ”pre” and ”gr” in the legend standsfor prediction values and ground truth respectively. (a) Swingand stance time. (b) Stride Length and stride time. . . . . . . 42Figure 3.20 Summaries of the stride length prediction value with unit cm.The x-axis and y-axis are the predicted and ground truth valuesof stride length in centimeter separately. . . . . . . . . . . . . 43Figure 3.21 Stride length comparison of different walking types. The blueone is predicted results and the original one is the ground truthvalue from GaitRite. . . . . . . . . . . . . . . . . . . . . . . 44Figure 3.22 Stride time comparison for predicted and the ground truth valueof different walking types in time unit s. (a). Normal walkingpattern. (b). Parkinson’s pattern. (c). Stroke pattern. (d).Summaries of three types of walking pattern in box plot. . . . 45Figure 3.23 The stance time and swing time box error of the normal andpathological walking patterns. . . . . . . . . . . . . . . . . . 46Figure 3.24 The stride length difference between the predicted values andground truth data in length unit centimeter. . . . . . . . . . . 47Figure 3.25 The MSE results for stride length prediction and ground truthvalues of normal and pathological gait. . . . . . . . . . . . . 48Figure 3.26 The input data into the network. The practical input data has alarger range than the predicted and ground truth range. . . . . 49xiiiFigure 3.27 The prediction results and ground truth data on the validationdataset in length unit meter ([m]). . . . . . . . . . . . . . . . 50xivList of AbbreviationsAI Artificial IntelligenceCNN Convolutional Neural NetworkECG ElectrocardiogramEMG ElectromyographyFL Federated LearningHFL Horizontal Federated LearningHS Heel Strike EventIMU Inertial Measurement UnitIOT Internet of ThingsLSTM Long short-term memoryML Machine LearningNLP Natural Language ProcessingRNN Recurrent Neural NetworksSTD Standard DeviationTO Toe Off MomentVFL Vertical Federated LearningZUPT Zero Update Position and TimingxvAcknowledgmentsIt has been so difficult but fun to finally end my journey as a master’s student at theUniversity of British Columbia. I could have never been so far without the help,support, and guidance of many people. Here, I would like to sincerely appreciatethe people who have given me a hand when I get into difficulty.Prof. Peyman Servati : I am immensely lucky to have worked under the super-vision of Prof. Servati. I am perpetually inspired by his unlimited creativ-ity, great tenacity, and strong leadership. Prof. Servati has taught me whatchange and progress research can make to the whole society. And he is alsoa distinguished advisor who creates and provides opportunities for studentswho need help. I sincerely thank his great support and guidance during mymaster’s years.Prof. Calvin Kuo : I am also incredibly fortunate to have worked with Prof.Calvin Kuo in the Biomedical Engineering department. He has providedme with valuable advice on the ethics of the gait parameter, the data collec-tion protocol, and the data process methods. I am frequently awestruck byProf. Kuo’s thoughtful considerations in designing the experiment. He is sorichly experienced in clinical products and patients’ conditions that he canalways think ahead about potential problems. I wouldn’t have proceeded sofast on the gait project without his help and advice.Dr. Zenan Jiang : Zenan has provided me with excellent insights and consideratefeedback always. I have learned how to work with great deadlines and dealwith anxiety when I confronted difficulties in academic life. Researching isxvibattling with unknown problems and pushing the envelope of ourselves. I’dlove to thank her for selflessly sharing those with me.Amir Servati, Fatemeh (Elham) Zabihi : Amir Servati, Fatemeh (Elham) Zabihi,and all coworkers from Feel lab and Texavie team, we have experienced agreat time together and contributed brilliant projects as an integrated team.Thank you all for the majestic teamwork.I would have never forgotten the priceless experience we explored together.I would love to thank NSERC for funding me for the master’s period. My bestwishes to the University of British Columbia. I also want to sincerely appreciatethe Friedman Scholarship to fund me as a visiting scholar at the University ofCalifornia, Berkeley. Those excellent experiences have broadened my insights intothe digital healthcare area, giving me instructive enlightenment on contributing tohuman society with the knowledge we have.Finally, I’d love to thank my family for their support and endless love. I wishto thank my boyfriend Hao Zhang, who has always encouraged me to confront thedifficulties in research and given me a lot of valuable advice on illustrations.I am immensely lucky to have met all the interesting and creative people, I amgrateful to them all.xviiDedicationTo the people who are struggling for high-quality medical resources and who aresuffering from the pain of disease.xviiiChapter 1IntroductionWith the development of Machine Learning (ML) models, Artificial Intelligence(AI) has been extensively utilized in multi-disciplinary fields to understand andmanipulate data [1], including natural language processing, biology data analysis,etc. The proliferation of wearable technologies has also witnessed the sheer vol-ume of data from edge devices providing unlimited possibilities for the promotionof ML and AI in the healthcare and medicine area. By adopting the large amountof data generated daily in the AI methods, healthcare and medicine tend to benefitincredibly from the precision assistance from ML. In this article, we demonstratedthe effectiveness of AI in wearable sensor feature extractions. The wearable sen-sor has been developed to assist the medical diagnoses and assessments. We firstdiscuss how wearable sensors and AI can contribute to personalized healthcare.And then we applied two specific examples on the wearable sensors and healthcareplatforms. We focus on tactile knee sleeves and sports injury prevention. We alsodemonstrate the foot-wear sensors’ usage in the gait parameter predictions.1.1 AI and Personalized HealthcareTime series data are arranged in chronological order, including statistical informa-tion and meaningful summaries [2–4]. It’s helpful in diagnosing past behavior,and also estimating promising future actions. Time series data has emerged asone of the most representative data due to the massive digitalization of healthcare.1Medicine and healthcare need to start thinking about the prediction of the future ina mathematical way. Persuasive time series data in the healthcare areas encompassa wide range of information such as Electrocardiogram (ECG) [5], Electromyog-raphy (EMG) [6], Inertial Measurement Unit (IMU) [7], etc. Those data can trackthe human physical index and daily activities for a long time outside the laboratorycontext. By applying ML algorithms to time series data, we are inspired to extractmore useful information and make more precise estimations on future diagnoses.ML in time series data is a relatively young area, but one that has shown greatpromise. Current methods for time series data are not originally developed for timeseries data specifically but are proven to have achieved great performance on it [8].In this thesis, we focus on classification and forecasting tasks on the time seriesdata. We discuss data privacy problems, insufficient label issues, etc. We proposehealthcare applications that benefit patients for disease recovery.1.2 Motivation1.2.1 Wearable ElectronicsThe wide spread of electronic technologies has promoted the proliferation of wear-able technology by embedding integrated circuits into accessories that can be di-rectly worn on the body [9]. Their linkage of data analytics can also advance theback-end devices such as mobile software.Wearable sensors are compact, convenient, and can be easily integrated intoportable devices. Moreover, wearable devices are a new method to track daily ac-tivities and can replace the usage of cameras in a lot of scenarios [10]. Without theparticipation of cameras, the data collection process can be extended into larger anduncontrolled environments. Data privacy issues can be mitigated without recordingsensitive information and personal recognition [10].To monitor the post-disease recovery progress, we utilize wearable electronicsto record patients’ daily activities and guide their recovery process. Our pose esti-mation project can reconstruct the motion video from the stress sensors. In futureplans, we will guide the patients to do appropriate exercises by developing gameswith virtual figures and instructions. In the gait project, we collect data from three2different gaits: normal walking, Parkinson’s, and Stroke gaits. We analyze individ-ual gaits and predict gait parameters such as stride length, stride time, etc. Thoseparameters can be the reference for clinicians and physicians to track the patientprogress. We apply ML methods to those wearable time series data to improve theaccuracy of estimation.1.2.2 Personalized HealthcareAfter working on two specific healthcare applications with wearable sensors, werealize that there exist some issues in directly applying current ML methods to thetime series data from the edge devices. First of all, the data from different partic-ipants have discrepancy data distributions due to their distinctive behavior habits.To improve higher accuracy in the time series data, we propose providing person-alized models in the federated learning stream for specific participants accordingto their own data distribution under the supervision of the cloud server. What’smore, supervised learning requires a huge amount of labels for each edge device,which is labor-extensive and not realistic. We incorporated semi-supervised learn-ing methods to reduce the energy to label collected data from edge devices. Thesemi-supervised learning algorithm we propose here is validated based on publiclyaccessible datasets online from the UCI machine learning repository [11]. Thisrepository records a large volume of time series data. What’s more, by only shar-ing the parameters and model weighs, we constrain the data leakage possibility andsecure client data privacy,In conclusion, we propose to solve three distinguishing issues in wearable sen-sor technology:• Data distribution discrepancies in the edge devices.• Label insufficiency in wearable electronics.• Data privacy in sharing.1.3 Organization and StructureThis thesis presents two healthcare applications based on wearable electronics. Wepropose corresponding ML algorithms to solve classification and forecasting is-3sues separately. Wearable electronics provide time series data from daily life andset patients free from laboratory context for data collection. The emergence ofwearable technologies can provide more data without constraining participants ina controlled environment. Those data might be representative and natural. Later,we found that the labeling process for the subjects is labor-intensive. What’s more,some data even require professional experts’ participation, which greatly limits theprogress of wearable sensor development. To save energy for labeling, we proposea semi-supervised federated learning algorithm to provide personalized codes forwearable devices.Chapter 2 covers the basic background of machine learning methods such asthe difference between supervised learning and unsupervised learning [12]. Weunderline the background of semi-supervised learning, which will be used in thesubsequent chapter. We explain the fundamental knowledge and basic cells in theML such as Convolutional Neural Network (CNN) [13, 14], Long short-term mem-ory (LSTM) [15] and Recurrent Neural Networks (RNN). Besides, we introducetraditional algorithms pervasively adopted in the time-series data inference such asthe Kalman filter [16].Chapter 3 shows the details of another of our healthcare applications: gaitparameters dataset and prediction. We collect data from different walking types:normal walking, Parkinson’s, and Stroke patterns. We analyze the distinctive walk-ing data based on the acceleration and gyroscope values. We apply the Zero UpdatePosition and Timing (ZUPT) algorithm [17–19] to different types of walking dataand compare the performance separately. We analyze the disadvantages of the tra-ditional algorithms and propose the ML improve the accuracy in gait parameterestimation.4Chapter 2Background2.1 Deep LearningWith the evolution of Internet of Things (IOT), various wearable technologies getaccess to the Internet, and we could get our hands on mass data from edge devices.Thanks to all those data, we are able to track daily human activities like activityrecognization [20–22], fall recognization [23], and sleep monitor [24]. However,most of the success of flexible technologies is dependent on fully-supervised mod-els, which require high-quality annotations for model learning under a controlledenvironment. And it’s extremely cost-prohibitive and energy-expensive to labelthe incoming data from end devices. Large volumes of data from different partic-ipants are generated every day. Moreover, even though the labels are available ateach node, nonnegligible issues like limited computational sources at distributeddevices, data privacy in communications, and domain shift will still exist when in-corporating data from those edge devices. Just like the process of human learning,computer programs do not need to know all the details but do analogical reasoningby extracting the characteristics of things, that is, connections. For example, stu-dents can learn from experience and solve similar but new problems by studyingexisting examples. The most common deep learning is classification. When train-ing data features and labels are input, the learning process is to find the relationshipbetween features and labels so that when unlabeled data is input, its label can beinferred according to its features.52.1.1 Supervised LearningThe task of supervised learning is to learn a model so that the model can make agood prediction of its corresponding output for any given input [12]. Learn a func-tion (model parameter) from the given training data set. When new data arrives,you can predict the results according to this function. The training set of supervisedlearning includes input and output, which can also be said to be characteristics andobjectives. The goals in the training focus are marked by people. Supervised learn-ing is the most common classification (attention and clustering) problem. Throughexisting training samples (known data and its corresponding output) to train, an op-timal model is obtained (this model belongs to a set of functions, and the optimalmeans that it is optimal under a certain evaluation criterion). Then, all inputs aremapped to the corresponding outputs using this model, and the output is simplyjudged to achieve the purpose of classification. It also has the ability to classifyunknown data. The goal of supervised learning is often to let computers learn theclassification system that we have created.Common supervised learning algorithms:Regression: The regression problem is to fit a curve of (x, y) to achieve theminimum L of the cost function.L( f ,(X ,Y )) = || f (X)−Y ||2 (2.1)Classification: Assuming that Y is a finite number, it can be regarded as a classlabel. The classification problem should first have a known label data training clas-sifier. In the classification process, cost function l (X, Y) is the negative logarithmof the probability that X belongs to class Y.L( f ,(X ,Y )) =−log( fY (X)) (2.2)2.1.2 Unsupervised LearningUnsupervised learning refers to mathematical modeling in which the input data isnot marked and there is no definite result. If the category of sample data is un-known, the sample set needs to be classified (clustering) according to the similarity6Figure 2.1: A simple example to demonstrate how semi-supervised learningis possible [12]. Reprinted with permission © 2023 Springer NatureSwitzerland AG. Part of Springer Nature.between samples in an attempt to minimize the intra-class gap [12]. For example,in real life, many times we cannot know the label of the sample in advance, andthere is no category corresponding to the training sample, so we can only learnclassifier design from the sample set that has no sample label.The goal of unsupervised learning is not to tell the computer how to do things,but to let the computer learn how to do things by itself. The idea of unsupervisedlearning is to specify clear classifications but to adopt some form of incentive sys-tem when successful. Its goal is not to produce a classification system, but to makethe decision of maximum return. This idea well summarizes the real world. Com-puters can stimulate correct behavior and punish wrong behavior. In this way, anexcellent machine-learning model can be trained.2.1.3 Semi-supervised LearningDeep learning requires a large amount of labeled data. Even in the age of bigdata, there are few clean and usable labeled data, which leads to the combina-tion of deep learning and semi-supervised learning. Semi-supervised learning is amachine learning algorithm used in the training data mixed with labeled data andunlabeled data (shown in 2.1) [12].7Figure 2.2: Typical Federated Learning (FL) client-server architecture [25].Copyright ©2020 by Morgan & Claypool.Generally, semi-supervised learning methods include [12]:Consistent regression: If a small real disturbance is added to the unmarkeddata (not infinitesimal, but very small), the classification of the data will not change.Agent labeling method: label unlabeled training sets based on some heuristicmethods using labeled training sets, and then generate some additional trainingcases.Generation model: learning features on one task can be transferred to othersubsequent tasks. The generation model that can generate images from the datadistribution p(x)must learn transferable features for a given task p(y|x)with thegoal of y.2.2 Federated learningFederated Learning (FL) [25, 26] has gained a lot of attention on addressing prob-lems of model training on a large volume of decentralized data [27], and user data8Figure 2.3: Horizontal Federated Learning (HFL) structure [25]. Copyright©2020 by Morgan & Claypool.privacy preserving [28–30]. FL has been ubiquitously applied to a lot of areas suchas anomalies detection [31, 32], Natural Language Processing (NLP) [33] and envi-ronment monitoring [34]. Most recently, FL has been extensively utilized in clinicalapplications and healthcare platforms [35, 36], especially when FL can tackle do-main shift and is friendly to sensitive medical data privacy issue in healthcare. Forexample, [36, 37] proposed a federated learning frame to estimate the COVID clin-ical symptoms generalize across heterogeneous. Keyboard prediction by Google[27] and keyword estimation in research engine by Firefox [38, 39] are outstandingexamples of using FL at edge devices without disturbing clients’ data privacy. Butsince there exists data transmission between servers and clients, FL brings chal-lenges to efficient communication [40–43]. [44, 45] adopted FL in breast cancerprediction and fMRI datasets.A typical structure for FL: client-server model is displayed in Figure 2.2 from[25]. Generally, FL is classified into two categories: Horizontal Federated Learn-ing (HFL) and Vertical Federated Learning (VFL). To focus on the topics of wear-able technologies in this article, we will only discuss the HFL, which is also calledthe client-server model. For more details on VFL, please refer to [25]. HFL usu-ally refers to datasets at various locations sharing some common latent featuringspaces but distinct from each other in sample spaces [46]. When applied in ourscenarios, wearable sensors for different users will collect the same type of data9such as ECG, accelerometer, gyroscope, etc., but with shifted data domains due toindividuals’ unique characteristics like different behavior and physical conditions,as is displayed in Figure 2.3.2.3 Kalman FilterKalman filtering method is an algorithm (also we can call it attitude estimation) thatuses the state equation of a linear system to optimally estimate the overall systemstate by observing and calculating the system input and output observation data (asshown in figure 2.4 and figure 2.5). This algorithm is often used in aircraft control(four-axis, six-axis, eight-axis, etc.), autopilot, and other navigation applications.Since the observation data includes the influence of noise and interference in thesystem, the optimal estimation is called the filtering process. If the measurementvariance is available, the Kalman filter can calculate the optimal estimation solutionof the state of the dynamic system from a series of noisy data. Stanley Schmidt firstdeveloped and applied the Kalman filter for the NASA Apollo program [16, 47].When considering the traditional digital filter, generally speaking, it can onlybe realized under the condition that the signal and noise/idler signal have differentfrequencies/bandwidths. The main idea of the Kalman filter is to process the noisyinput signal and observation signal in the linear state space to obtain the systemstate (in the time domain). Among them, the system state is the set of the minimumparameters that synthesize the effects of all historical inputs and disturbance signalsof the system on the system. After that, knowing the system state can determine theentire behavior of the system together with future inputs and system disturbances.For example, in digital image signal processing, the Kalman filter can restoreblurred images (remove noise). After assuming some statistical properties of noise,we can use Kalman’s algorithm to obtain the real image with the minimum meansquare error from the blurred image in a recursive way, so that the blurred imagecan be restored.2.4 Convolutional Neural NetworkWith the rapid development of machine learning and deep learning, the sizes andcategories of datasets are increasing, which makes the traditional neural network10Figure 2.4: The ongoing discrete Kalman filter cycle. The time updateprojects the current state estimate ahead of time. The measurement up-date adjusts the projected estimate by an actual measurement at thattime [16]. Copyright ©NC-Chapel Hill, 2006Figure 2.5: A complete picture of the operation of the Kalman filter [16].Copyright ©NC-Chapel Hill, 200611unable to meet the corresponding demands in computing speed, memory, and com-puting time [48, 49]. For example, in a 1000-pixel*1000-pixel image, the computerneeds to calculate 3 ∗ 106 (RGB three primary colors). If there are 1000 such im-ages in the dataset, the result of the calculation is 3∗109. This calculation is veryhuge and requires high memory and speed of the computer, but it is only a verysmall dataset. What should I do if I encounter a very large dataset (10k+ images)?In 1987, Alexandra et al. proposed the first convolution neural network, whichis called Time Delay Neural Network (TDNN) [50]. TDNN is used in the con-volution neural network for speech recognition. The algorithm uses FFT to pre-process the input signal, and the Hidden layer is composed of two one-dimensionalconvolution cores to extract translation invariant features in the frequency do-main. In 1988, Wei proposed the translation invariant artificial neural network(SIANN), which is a two-dimensional convolution neural network [51]. In 1989,Yann first proposed a convolution neural network applied to computer vision prob-lems, namely the famous LeNet [49]. In 2006, the deep learning theory pro-posed [52], and convolution neural networks received extensive attention. Withthe support of GPU cluster computing, Convolution Neural Network (CNN) hasbecome very mature in the application of multiple researchers and has developedmany variants [49].CNN is a feed-forward neural network, and the simulated neurons can coverthe responses of the surrounding cells. CNN is connected by a full connectivitylayer and the top of multiple convolution layers, and the weight layer and poolinglayer are associated [48].The structure and whole workflow of CNN are shown in figure 2.6 and 2.7.Generally speaking, the input layer of a one-dimensional convolution neural net-work receives one dimensional or two dimensional arrays, in which the one di-mensional array is usually time or spectrum sampling; A two dimensional arraymay contain multiple channels. The input layer of a two dimensional convolu-tion neural network receives two-dimensional or three-dimensional arrays. CNNis widely used in the field of computer vision, so usually, three-dimensional inputdata is assumed in advance, that is, two-dimensional pixels and RGB channels onthe plane [48].Since the gradient descent algorithm is used for learning, the input features12Figure 2.6: The structure illustration of CNN [48]. Copyright © arXiv, 2015Figure 2.7: The workflow of a CNN [13, 14]. Reprint with © 2023 SpringerNature Switzerland AG and Copyright © arXiv, 2016.of the convolution neural network need to be standardized. More specifically, itis required to normalize the input data in the channel or time/frequency dimen-sion before inputting the learning data into the convolution neural network. If theinput data is pixels, the original pixel values distributed in [0, 255] can also benormalized to [0,1] interval. The standardization of input features is conducive toimproving the learning efficiency and performance of convolution neural networks.Hidden Layer: the hidden layer of the convolution neural network includes theconvolution layer, pooling layer, and full connection layer. The convolution core inthe convolution layer contains weight coefficients, while the pooled layer does not.13Taking LeNet-5 as an example, the order of three common types of structures in thehidden layer is usually: input - convolution layer - pooled layer - fully connectedlayer – outputc˜itelecun1998gradient.Convolution layer: convolution layer is used to extract features of input data,which depends on multiple convolution cores contained in it. Each element of theconvolution kernel corresponds to a weight coefficient and a bias vector. Eachneuron in the convolution layer is connected to multiple neurons in the area closeto the previous layer, and the size of the area depends on the size of the convolutionnucleus (receptive field):zl+1(i, j) = [zl⊗ω l+1](i, j)+b = Σklk=1Σ fx=1Σ fy=1[zlk(s0i+ x,s0 j+ y)ω l+1k (x,y)]+b(2.3)(i, j) ∈ {0,1, ...Ll+1};Ll+1 = Ll +2p− fs0 +1 (2.4)Here, b is bias vector, zl and z(l+1) Represent convolution input and output oflayer l+1. Ll+1 is the size of zl+1. Z(I, j) corresponds to the pixel’s position in thefigure, K is the number of channels, f is the size of convolution kernel, s0 is theconvolution step and p is the number of padding layers.Activate function: in the neural network, there is a functional relationshipbetween the input and output of the hidden layer and the output layer nodes, whichis called the excitation function.Common activation functions are: Sigmoid, Rectified Linear Unit (ReLu),Softmax, Softplus, Hyperbolic tangent, etc [13, 49](some of them are shown infigure 2.8).Pooling Layer: after feature extraction in the convolution layer, the output fea-ture map will be transferred to the pooling layer for feature selection and informa-tion filtering. The pooling layer contains a preset pooling function, whose functionis to replace the results of a single point in the feature map with the feature mapstatistics of its adjacent areas.CNN is widely used in computer image recognition, object recognition, actionrecognition, natural language processing, and attitude estimation [49]. Among14Figure 2.8: Common types of activation function [13, 49]. Reprint withCopyright © 2023 Springer Nature Switzerland AG and Copyright ©arXiv 2016them, attitude estimation is to output the human attitude in the form of coordinatesin the image.The first convolution neural network used in attitude estimation is DeepPose,which trains and outputs pictures with coordinate points in the way of supervisedlearning [53]. In addition, the application of convolution neural networks to localattitude estimation is also studied. For video data, the convolution neural networkwith sliding window can be used to estimate the pose frame by frame [49].2.5 Recurrent Neural NetworksThe Recurrent Neural Network (RNN) is a neural network used to process serialdata (such as time series data) [15]. Compared with the general neural network,it can handle changing sequential data. For example, when we understand themeaning of a sentence, it is not enough to understand each word of the sentence15Figure 2.9: The structure of RNN [15, 54]. Reprint with Copyright © 2019Elsevier B.V. and Copyright © arXiv, 2017 W.Yin isolation. We need to deal with the whole sequence connected by these words;When we process video, we can not only analyze each frame separately but alsoanalyze the whole sequence connected by these frames. RNN can solve such prob-lems well. RNN structure is as shown in figure 2.9In figure 2.9, x is the input of data in the current state, and h is the input of thelast node received. y is the output in the current node state, while h ’is the outputpassed to the next node. In general, y uses h ’to input a linear layer (e.g., dimensionmapping), and then uses softmax [55] for classification to obtain the required data.The RNN workflow is shown at the bottom of the figure 2.9.2.6 Long Short Term MemoryLong short-term memory (LSTM) is a special RNN, which is mainly used tosolve the problem of gradient disappearance and gradient explosion during longsequence training. In short, LSTM can perform better in longer sequences thanordinary RNNs. LSTM structure and internal cellular workflow are shown in thefigure 2.10 (a) and (b).Compared with RNN, which has only one transmission status of ht , LSTM has16Figure 2.10: (a). The structure of LSTM; (b) The internal workflow of LSTMcell [56]. Reprint with Copyright © 2023 IEEE.two transmission statuses, one is a ctcell state, and one is a ht hidden state. It canbe understood here that: ht in RNN for ct in LSTM).z f ,zi,zo(forget, information,output) is a gated state that is converted to a value between 0 and 1 through asigmoid activation function after the splicing vector is multiplied by the weightmatrix. And z is the result converted into a value between - 1 and 1 through a tanhactivation function.⊙is the Hadamard Product and ⊕ is the adder of the matrices.Workflow of LSTM:1. Forget the stage. The calculated z f is used as the forgotten door to controlwhich c(t − 1)in the previous state needs to be left and which needs to beforgotten. In this stage, the input from the previous node is selectively for-gotten. The best way to explain this method is: ”Forget the unimportant,remember the important”.2. Select the memory stage. At this stage, the input xt will be selected andmemorized. Record the important information and discard the unimportantinformation. The current input content is represented by the z calculatedpreviously. The selected gating signal is controlled by zi.3. Add the results from the above two steps to get the ct transferred to the next17state, as shown in the formula in the figure.4. Output stage. This stage will determine which outputs will be considered asthe current state. It is mainly controlled by zo. In addition, the co obtained inthe previous stage is transformed through the tanh activation function.In summary, LSTM controls the transmission state through the gated state,remembers the information that needs to be memorized for a long time, and forgetsthe unimportant information, which has strong operability and high accuracy forlong-time sequence learning and prediction; Unlike ordinary RNN, there is onlyone memory superposition mode mechanically. However, due to the introductionof a lot of content, the parameters become more, which also makes the trainingmore difficult.2.7 Unity Real-Time Development PlatformUnity3D is a multi-platform comprehensive game development tool developed byUnity Technologies, which allows you to easily create interactive content such as3D video games, building visualization, real-time 3D animation, and other typesof interactive content. It is a fully integrated professional game engine [57].What are the characteristics of Unity?1. Visual programming interface completes all kinds of development work,efficient script editing, convenient for development;2. Automatic instantaneous import, Unity supports most 3D models, bones andanimation are directly imported, and mapped materials are automatically convertedto U3D format;3. Multi-platform development and deployment of works can be completedwith one click;4. The bottom layer supports OpenGL and Direct11, a simple and practicalphysics engine, a high-quality particle system, easy to use, and realistic effect;5. Support JavaScript, C #, and Boo scripting languages;6. Unity has excellent performance, outstanding development efficiency, andcost-effective advantages;187. Support from a single application to large-scale multiplayer online gamedevelopment19Chapter 3Gait Analysis in Pathological GaitIn this chapter, we will discuss the gait parameters analysis in pathological forcamera-free clinician health monitoring and personal health care. In this project,we collected and analyzed the data via algorithms. Then we compared the resultsof the traditional zero-velocity update algorithm and machine learning.3.1 IntroductionZUPT is one of the most widely-used algorithms in predicting spatial gait param-eters in normal walking patterns of healthy subjects with an accuracy of gener-ally less than 10% error [58]. In this chapter, two typical gait disorders caused byParkinson’s and Stroke will be investigated by ZUPT algorithm from IMU data [17],against ground truth data collected from the GAITRite system.3.2 Data Collection Process3.2.1 Experimental SetupThe experimental platform was composed of a GAITRite system and a set of wear-able sensors. Participants were asked to wear one IMU sensor on the top of eachfoot measuring triaxial linear accelerations and triaxial angular velocities usinghypoallergenic tape and stepped on the GAITRite walkway as in Figure 3.1. Thedetails of the hardware platform are displayed here.20GAITRite: GAITRite has been used as the golden standard to provide theground truth data related to gait analysis [59, 60]. GAITRite mat is able to detectmechanical pressure with embedded sensors while not affected by the time andmeans of force [61]. Data is analyzed by specifically designed GAITRite to providegait parameters like stride length, stride time, and cadence.MCU: Adafruit4884 from Arduino is a flexible and open-source prototype usedas the main board to process and save electric signals during walking. The mainboard communicates with an SD card and saves acceleration and gyroscope data toit at 1000Hz.IMU: ICM20649 is a 6-axis IMU, collecting triaxial linear accelerations at ±30gand triaxial angular velocities at ±4000deg/s during experiment. Data were simul-taneously sampled from both sensors at 1000Hz during walking.3.2.2 Measurement ProtocolsThe experiment is consented to by the UBC Research Ethics Protocol H21-02052.A total of 10 subjects (six females against four males) were recorded for algorithmvalidation. All participants are healthy subjects aged between 19 and 60 years oldwithout a history of any musculoskeletal injuries in the past year that would affectnormal locomotion. Each participant is required to walk in three different patterns:(1). Normal gait trial: participants walked at comfortable self-selected speeds tothe end of the GAITRite carpet. (2). Parkinson’s gait trial: participants simulatedgait impairment symptoms of Parkinson’s like shuffling steps and increased ca-dence [62] according to the research results and attached video. (3). Stroke gaittrial: participants mimicked the asymmetry in gait [63] during walking by refer-ring to the study results of actual Stroke patients. Each pattern is repeated sixtimes in a randomized order to guarantee data authenticity. Participants are askedto stay stationary for a couple of seconds before and after the trials for calibration.Acceleration and gyroscope sometimes suffer from mean value drifts, and thosestationary periods can provide a reference for calibration in later signal processing.21Figure 3.1: (a). GAITRite system. (b). IMU setup. (c). Foot-mounted IMUs.3.3 Sensor SignalsThe output signals of different walking patterns are different. There exist extrasteps at the start and end of each trial for IMU data, which come from the stepsoutside the GAITRite mat. The extra steps may be caused by the first move fromthe ground to the GAITRite mat which is unable to be detected before the first HeelStrike Event (HS) event or participants move accidentally before/after the trial. Theconcrete signals of three walking patterns will be discussed separately.22Normal-Left20100-10-20Acceleration (m/s )220100-10-20Time (s)Acc X Acc ZabNormal-RightGround truthAcc Y0 15105Figure 3.2: IMU signals in normal walking of left (a) and right foot (b).3.3.1 Normal PatternThe normal walking signal is displayed in Figure 3.2. For better visualization, thesignal has been processed with a low pass filter for extra noise and a high pass filterfor mean value drift. Generally, in a normal walking pattern, four or five steps cancover the GAITRite carpet length. The green line represents the ground truth dataexported from the GAITRite system. The green line stays high till the first footcontact moment happens and turns into low status. In later detection, when thegreen line goes low, the foot contacts the ground. For easier statements, we referhigh status of the green line as the ‘1’ status and the low status as the ‘0’ status.After the first contact, 1 stand for the status of the foot off the ground, and 0 standsfor the status of the foot on the ground.Comparing Figure 3.2 (a) and (b), we can see no asymmetry between legs isobserved and the cadence in normal walking is pretty stable. The raw data ofnormal walking is in Figure 3.3. The acceleration of x and y should be 0 when23Normal-Left20100-10-20Acceleration (m/s )220100-10-20Time (s)Acc X Acc ZabNormal-RightAcc Y0 15105Figure 3.3: IMU raw signals in normal walking of left (a) and right foot (b).stationary while the z-axis value should be close to g. However, since the left andright feet are mounted with independent IMUs, the drift and tilt angles on eachfoot are different. We will discuss in detail how to calibrate and remove the driftsin acceleration caused by the tilt of IMU in later sections. The stationary part at thebeginning and the end of the trial can be used as calibrations for such drift and latergravity subtraction.3.3.2 Pathological GaitThe characteristic of Parkinson’s gait is shuffling and smaller steps with increasedcadence. As in Figure 3.4, the Parkinson’s gait exhibit more irregular steps withintermittence shuffling steps between normal steps. Compared to normal walk inFigure 3.2, the single support time of Parkinson’s pattern is less consistent, wereshuffling steps last shorter both swing and stance time than normal steps. When240 5 10 15 20 25time (s)-10010acceleration m/sLeft-ZUPT-May212-Shuffle110 5 10 15 20 25time (s)-10010acceleration m/sRight-ZUPT-May212-Shuffle11Shuffle-RightShuffle-Left100-10Acceleration (m/s )2100-100 252015105Time (s)abAcc XAcc Y Gro nd truthAcc ZFigure 3.4: IMU signals for Parkinson gaits of left (a) and right foot (b) withground truth data from GAITRite system.walking through the GAITRite carpet with the same length, more steps (gener-ally more than 10 steps) are required for the Parkinson’s pattern, corresponding toincreased cadence.The major effect of a hemiplegic stroke is paralysis for only one side of thebody, which will render an asymmetry motion of gait [63]. During foot contactevents, HS moment of the affected side is often lost because of decreased strengthof affected side body muscles. The motion and speed of the dragging foot arerelatively slow, which reflect in IMU detection results in smaller triaxial linearaccelerations and smaller triaxial angular velocities in comparison to normal walkin Figure 3.2, as is displayed in Figure 3.5. More steps (generally more than 10steps) are also required to cover the same carpet length in contrast to a normalwalking pattern with increased cadence. It’s obvious to observe that the IMU valuesbetween the dragging foot and normal foot are different. The normal foot has250 5 10 15 20 25time (s)-10010acceleration m/sLeft-ZUPT-May212-Stroke10 5 10 15 20 25time (s)-10010acceleration m/sRight-ZUPT-May212-Stroke1Dragging footNormal footAsymmetric Asymmetric 0 25201510520100-10-20Acceleration (m/s )220100-10-20Time (s)abAcc XAcc YAcc ZGround truthFigure 3.5: IMU signals for Stroke gaits of (a) normal foot and (b) draggingfoot with ground truth data from GAITRite system.a more similar range to the normal walk (Figure 3.2), while the dragging footdisplays slower motion than the normal foot.3.4 Data pre-processingNoise [64] is an inevitable part of the signal during the data collection process dueto sensor position, soft tissue movement, and manufacturing of IMU itself. To getmore accurate results, data pre-processing is indispensable to produce before ZUPTalgorithm can be applied.3.4.1 Scale and driftThe most frequent noise in IMU is the scale and drift. When putting IMU station-arily with one axis (let’s assume it’s z-axis, it’s applicable to another axis as well)260 10 20 30 40 50 60 70 80 90-10-50left0 10 20 30 40 50 60 70 80 90-10-50right0 806040200-5-10-5-10Time (s)LeftRightAcc X Acc YAcc ZAcc X Acc YAcc Z0 10 20 30 40 50 60 70 80 900510left0 10 20 30 40 50 60 70 80 900510right0 806040201501050Time (s)LeftRightAcc X Acc YAcc ZAcc X Acc YAcc Za bFigure 3.6: Left and right IMU signal with z axis facing (a). downward. (b).upwardfacing downward, there should be a gravitational constant on the z-axis and alwayszero values for the x and y-axis. And when converting the IMU over, there will bea negative gravitational constant on the z-axis. However, most of the time, thereexist shifts that cause z-axis values to deviate from the gravitational constant. Theleft foot doesn’t show much difference when flipping the IMU up and downward asin the Figure 3.6(a), and the absolute value of both orientations seems very close(+9.8m/s2 and -9.9m/s2). Yet there is clear drift and scale in the right foot as inthe Figure 3.6(b), where the positive and negative IMU gravity is +8.3m/s2 and-11.45m/s2. Those shifts are the characteristics of the hardware and will exist nomatter in what orientation the IMU is. To fix this problem, we need to analyze thevalues when flipping each axis’s orientation. Leave IMU quietly for 90 seconds andalign it to the gravity direction as vertically as possible. Collect data with each axisseparately.Let’s use g standing for the gravitational constant. Assuming when one axis(let’s say z-axis) is facing critically downward, the value of the corresponding axis(z-axis) is gˆ, where gˆ should be close to gravity value g. When the IMU is facingupward, the value of the corresponding axis (z axis) is gˆ′, and the value of gˆ′ shouldbe close to the opposite of gravity value: −g. Assume that the drift value andscale of the z-axis are constant numbers and presented as zo f f and zscale. Then the27method to figure out zo f f and zscale is to calculate matching pair of zo f f and zscalethat make equation 3.1 minimum:fmin = [(gˆ+ zo f f )∗ zscale−g]2+[(gˆ′+ zo f f )∗ zscale+g]2 (3.1)The MATLAB procedure to calculate the requested solution of zo f f and zscaleis attached inSection A.1 of Appendix A.3.5 ZUPT Performance on Pathological GaitVarious algorithms to quantify gait parameters with inertial measurement units(IMUs) have been developed recently, among which ZUPT [18, 19] has been demon-strated as an accurate algorithm for quantifying gait parameters in normal healthywalking. However, its performance in patients with gait pathologies is still unclear.We will investigate ZUPT on the pathological gaits in this section. There are twophases in ZUPT [65] algorithm. The first part is ZUPT foot segmentation, where weneed to recognize the foot-ground contact period.3.5.1 Step SegmentationFoot segmentation is the foundation of ZUPT algorithm and has a major influencein the later process to integrate foot velocities and calculate gait parameters. Anymissed or overcount steps will cause a huge error in estimating the gait parameters.Thus, it’s equally important to evaluate how ZUPT performs in the foot segmenta-tion phase.The detector we use in this article is the accelerator magnitude detector. Theacceleration should be passed through a low pass filter and a high pass filter toremove pulse noise and mean value drift noise. The result of ZUPT algorithmin normal walking patterns is displayed in Figure 3.7, which achieved a betterperformance than pathological patterns. The ZUPT estimation is accurate in normalpatterns in terms of step segmentation. Very limited missed or overcounted stepsappear in normal walking, although normal gaits have early predictions of Toe OffMoment (TO) and late predictions of HS.The IMU signals of Parkinson’s pattern exhibit visible inconsistent cadence28ShuffleNormal100-10Acceleration (m/s )2100-100 108642Time (s)abAcc X Acc YGround truthAcc ZPredicted value126.4 6.6 6.8 7 7.2time (s)-10010acceleration m/sLeft-ZUPT-May212-Normal10Normal100-10Ti e (s)c0 1Figure 3.7: IMU signals for normal gaits of (a) left foot and (b) right footwith ground truth data from GAITRite system and estimated value fromZUPT algorithm. (c). Zoom in on details for one normal walking step.0 5 10 15 20 25time (s)-20-1001020acceleration m/sLeft-ZUPT-May18-Shuffle10 5 10 15 20 25time (s)-20-1001020acceleration m/sRight-ZUPT-May18-Shuffle1Shuffle-RightShuffle-Left20100-1-2Acceleration (m/s )220100-1-20 252015105Time (s)ab17 17.5 18 18.5time (s)-10010acceleration m/sLeft-ZUPT-May212-Shuffle11ShuffleTi e (s)c0 .0.100-10Acc X Acc YGround truthAcc ZPredic ed valueFigure 3.8: IMU signals for Parkinson gaits of (a) left foot and (b) right footwith ground truth data from GAITRite system and estimated value fromZUPT algorithm. (c). Zoom in on details on missing steps of Parkinson’scase.29Dragging footNormal footAsymmetric Asymmetric 0 25201510520100-10-20Acceleration (m/s )220100-10-20Time (s)abAcc X Acc YGround truthAcc ZPredicted value15.5 16 16.5 17time (s)-10010acceleration m/sLeft-ZUPT-May212-Stroke1022 22.5 23 23.5time (s)-10010acceleration m/sRight-ZUPT-May212-Stroke10Dragging footNormal footAsymmetric Asymmetric Time (s)d100-10100-100 1.50.5 10 1.50.5 1cFigure 3.9: IMU signals for Stroke gaits of (a) normal foot and (b) draggingfoot with ground truth data from GAITRite system and estimated valuefrom ZUPT algorithm. (c). Zoom in on details of normal foot patterns.(d). Zoom in on details of dragging foot pattern with late TO prediction.during walking. Small and shuffling steps intersect with normal steps which resultsin fluctuating thresholds in stationary period judgment and more missing steps.Shuffling steps have a shorter swing phase and stance phase than normal steps, andall steps in Parkinson’s pattern display a shorter gait cycle compared to the normalwalking gait Figure 3.7.The asymmetry of Stroke gait leads to the asymmetry of ZUPT predictions. Theestimation of the normal foot in the Stroke pattern is relatively accurate, comparedto the dragging foot, as is shown in Figure 3.9 (a) and (b). The dragging foot hasa larger number of late TO estimations, as in Figure 3.9 (d), caused by its slowmotion during walking. Additionally, because of the obscure TO and HS moment,it’s easy to miss out on steps in dragging the foot as well.In summary, we quantified two indexes in the first phase of ZUPT as a criterionto evaluate the algorithm performance on different walking data: 1). The num-ber of missed and overcounted steps normalized against the total number of stepsas identified by GAITRite and 2). The timing errors for TO and HS as is shown300.80.60.40.2MissingOver countingStep counts errors (normalized)Shuffle AsymmetryNormala0.0Walk type-1012Difference (s)Walk typeb Normal Shuffle AsymmetryToe offHeel strike3Figure 3.10: (a). Step count errors are normalized by correctly recognizedsteps. (b). TO and HS events time difference in second unit.in Figure 3.10. Since each pattern of walking has a different number of steps inone trial, constricted to the length of the GAITRite walkway, we normalized themissing steps and overcounted steps by the corrected steps for better comparison.From Figure 3.10 (a), normal walking has the best overall performance in termsof step segmentation with an average of 0.23% missing steps and 0.44% over-counted steps. The statistical data of missing step error is listed in Table 3.1 andTable 3.2. Parkinson’s patterns have more missing and overcounted steps withaverages of 10.45% and 1.71%. ZUPT performs worst on the Stroke gait withan average of 26.16% missing steps and 3.4% overcounted steps. Due to the in-consistent steps within pathological gaits, Parkinson’s (Standard Deviation (STD)0.1591 for missing steps and 0.0401 for overcounted steps) and Stroke gait (STD0.2487 for missing steps and 0.0669 for overcounted steps) results are more de-viated than the normal walking pattern (STD 0.0199 for missing steps and 0.0329for overcounted steps), which means ZUPT achieves the most stable performanceon normal walking while pathological gait situations are more varied and discretepoints may happen in error plotting (Figure 3.10). In contrast with missing stepsin Figure 3.11 where the maximum value can reach 0.94, the maximum value ofover-counted steps only goes for 0.33 after normalization. Overall, pathologicalgaits are easier to miss steps because of slower foot motion, and the Stroke patternwith asymmetric walking pattern miss most steps.31Table 3.1: Statistical results of missing stepsWalking Type Normal Parkinson StrokeAverage 0.0023 0.1045 0.26161st Quartile 0 0 0Median 0 0.0667 0.20003rd Quartile 0 0.1645 0.4444STD 0.0199 0.1591 0.2487Table 3.2: Statistical results of overcounted stepsWalking Type Normal Parkinson StrokeAverage 0.0044 0.0171 0.03401st Quartile 0 0 0Median 0 0 03rd Quartile 0 0 0.0625STD 0.0329 0.0401 0.0669The timing results in Figure 3.10 (b) show that all gaits have a similar earlyestimation of TO and late predictions of HS time difference, except for the Strokepattern, which has a number of situations with late TO prediction and early HS pre-dictions. The details of statistical results are displayed in Table 3.3 and Table 3.4,where the positive numbers stand for early TO prediction and late HS prediction.The TO and HS time difference results are calculated from only correctly recog-nized steps, where missing steps are ignored. The smaller values of the averageand quartile of Pathological gaits may be caused by more varied data in those gaitsand more negative estimations. The average Stroke timing difference (-0.0017) iseven negative, which indicates lots of steps are missing TO and HS events. Thelowest STD of normal walking (0.0756 for HS & 0.0524 for TO) suggests most con-sistent performance than Parkinson (0.2038 for HS & 0.09167 for TO) and Strokegaits (0.1851 for HS & 0.1529 for TO).If we look into the details of each specific participant’s performance, we canfind more varied pathological gait results that cause larger errors as is shown in Fig-ure 3.11 and Figure 3.12. For each participant, the ZUPT performance on Parkin-32Table 3.3: Statistical results of HS difference in secondWalking Type Normal Parkinson StrokeAverage 0.1678 0.0824 0.00311st Quartile 0.1270 0.0260 0Median 0.1740 0.0560 0.03903rd Quartile 0.2050 0.0860 0.0890STD 0.0756 0.2038 0.1851Table 3.4: Statistical results of TO difference in secondWalking Type Normal Parkinson StrokeAverage 0.0998 0.0561 -0.00171st Quartile 0.0680 0.0120 0Median 0.0860 0.0670 0.00603rd Quartile 0.1203 0.1040 0.0780STD 0.0524 0.0917 0.1529son and Stroke pattern is more fluctuating. For the missing step case, all most nomissed steps are spotted except for P1 and P5, where the max value is constrainedunder 0.2 with median & average value as 0. In Parkinson’s case, some subjectsget really good performance compared to normal walking cases such as P2, P8,and P9. But there are also situations where Parkinson’s participant data is worsethan Stroke results such as P4. The case is the same with Stroke data. Pathologicalgait data usually has a more varied data distribution, and for the different partic-ipants, commonly Parkinson’s data has better performance than Stroke data. Thedetails of the statistical results for each participant are attached in the Table A.1 ofSection A.2 in Appendix A.Comparing missing steps (Figure 3.11) with overcounted steps (Figure 3.12),we can find that normal walking is actually more likely to overcount steps whilepathological gaits are easier to miss steps. That’s because the normal walkingpattern has a very strong and clear TO and HS event, which is easily recognizedby the acceleration magnitude detector. Yet the motion when the foot is lifted tothe highest position and ready to fall on the ground is relatively slow and easier to330.250.50.751.001.25-0.5-0.250Mssing step counts (normalized)Participant numberP1 P2 P3 P4 P5 P6 P7 P8 P9 P10NormalParkinsonStrokeFigure 3.11: Missing step counts normalized by correctly recognized stepsfor each participant.be missed out. This phase in the gait cycle is also referred to as mid swing [66] inFigure 3.13. The values of overcounted steps for pathological gaits are way smallerthan the values in missing steps. Almost all the participants have a median valueof zero for overcounted steps except for the Parkinson patterns in P3 (0.0769),while the median value for missing steps of pathological gaits varies from 0 to0.3. The maximum value of pathological gaits also reduces from max 0.9 (missingsteps, P10) to 0.3 (overcounted steps, P3). The opposite is true for normal gaits.Maximum value increases from max 0.1 (missing steps, P1) to 0.3 (overcountedsteps, P1). The STD and average values of overcounted steps also are increased fornormal walking. The details of average value, mean value, maximum and STD forovercounted steps are also attached in Table A.2 of Section A.2 in Appendix A.In one word, mid-swing in the normal walking pattern is more likely to bejudged as a foot-ground contact period, which will generate over-counted steps340.10.20.30.40.5-0.10overcunted step counts (normalized)Participant numberP1 P2 P3 P4 P5 P6 P7 P8 P9 P10NormalParkinsonStrokeFigure 3.12: Over counted step errors normalized by correctly recognizedsteps for each participant.and interrupt the gait parameter calculation process as in Figure 3.14 (a). On thecontrary, pathological gaits miss steps with greater possibility instead of over-countsteps. That’s because both Parkinson’s and Stroke patients have a slower motionon either one or both of the feet, which means they have obscure TO and HS events.This can lead to totally lost steps. By adjusting the threshold to a lower place,overcounted steps caused by misjudgment of the mid-swing phase as a stationaryperiod can be partially avoided. The lower threshold can also decrease missed stepsin pathological gaits. However, the lower threshold can render in misjudgment ofthe stance phase as off ground period. The acceleration theoretically should be veryclose to zero at stationary state, but noises such as soft tissue artifacts [67], andaccident shaking during a single support phase may contribute to non-zero valuesin the foot-ground contact period. The vibrating during the single support phase in35Figure 3.13: Functional divisions of the gait cycle [66]. Copyright ©2015Sto¨ckel, Jacksteit, Behrens, Skripitz, Bader and Mau-Moeller.100-10Acceleration (m/s )100-100 108642 12Normal100-10Time (s)0 1Time (s)5.5 6 6.5 7 7.5time (s)-10010acceleration m/sLeft-ZUPT-May111-Normal3Normal100-10Tim (s)0 1 2Acc X Acc Y Ground truth Predicted valueAcc Z0 8642abFigure 3.14: (a). Normal walking pattern with low thresholds and connectedsteps. (a). Normal walking pattern with high thresholds and split steps.360 5 10 15 20time (s)-0.4-0.200.20.40.60.8quaternionswxyzFigure 3.15: Quaternions of sensor data fusion for a normal walking trial.pathological gaits will be more obvious and will affect more in the step recognitionprocess more. In those cases, ZUPT will be confused and recognize several stepsas one giant step due to a low threshold as is shown in Figure 3.14 (b). ZUPT issensitive to the choice of threshold, that’s also the reason for the bad performanceon pathological gaits. Each step may require a very different threshold for bestperformance, and this is pretty difficult for the unsupervised method to realizewithout feedback from labels and ground truth data.3.6 Gait Parameter Prediction3.6.1 Gravity SubtractionAfter we divide each step, it’s time to move on and get the ultimate gait parame-ters from the segmentation result. To find the stride length, we need to integrateacceleration and figure out the velocity. Since the errors and drifts will be accu-mulated during the integration process, we adopt the ZUPT algorithm [18] that has370 5 10 15 20time (s)-0.500.51quaternionswxyzFigure 3.16: Quaternions of sensor data fusion for a normal walking trial af-fected by drift problem.been ubiquitously utilized in gait analysis. We reset the velocity always to zerowhen detecting the foot on the ground. The displacement results of walking areintegrated from velocity. The correction for each step of velocity will greatly im-prove displacement integral results. Before we operate the integration procedure,we need to subtract the gravity from IMU acceleration [68]. However, since wedon’t have a magnetometer embedded in our experiment, the quaternion results aremore subject to the IMU drift. We leverage the powerful library of MATLAB tocalculate quaternions. The code for calculating quaternion and subtracting gravityis displayed in Listing A.2 of Section A.1 in Appendix A.The sensor data fusion results for normal walking are displayed in Figure 3.15.The orientation change of the gait cycle is obvious in this example. There are fivemajor steps in the trial, and quaternions tilt most during the swing phase. Comparedto acceleration, gyroscope value is more vulnerable to drifts, which may render38shifted quaternion results as in Figure 3.16. In the drift situation, the start pointof quaternions is quite reasonable, however, they slowly drift away with time andeventually end with evident unrealistic status. Gait cycles like the swing phase andstance phase are still very distinguishable in the drifted context and as distinctiveas in the non-drift context Figure 3.15. Drifts happen more when the data is notcalibrated with magnetometers. Qaurternions should have consistent gait cycleswith the acceleration and gyroscope value, which go back to the initial status (1,-0.1, 0.15, 0) after each step completion in Figure 3.16. However, due to the drift,the quaternion values can’t fall back to the consistent position at the stance phase.The errors accumulate with steps and finally shift away.Assuming that the accelerations of the x, y, and z-axis are represented as accx,accy, and accz. The magnitude of accelerations will not be changed after the gravitysubtraction. Image the orientation of the foot is a 3D vector composed of vectorsfrom the x, y, and z planes. Then only the projection of the vector on each plane isconverted, and the 3D orientation vector remains unchanged. ˜accx, ˜accy, and ˜acczare denoted as the vectors after the gravity subtraction. Then we have the Equa-tion 3.2 established.√|accx|2+ |accy|2+ |accy|2 =√| ˜accx|2+ | ˜accy|2+ | ˜accz|2 (3.2)Before gravity subtraction Equation 3.2 (a), the magnitude of the stationaryphase is close to the gravity value g = 9.8m/s2. After gravity subtraction Equa-tion 3.2 (b), the stationary part is around zero with a small variation.3.6.2 Velocity and Displacement UpdateAfter eliminating the effect of gravity g, we could integrate the acceleration andcalculate the velocity and displacement values. Depending on the segmentationresults of Section 3.5, we force the velocity to zero when detecting foot on theground. The integration results of velocity and displacement integration are illus-trated in Figure 3.18. When swing phase starts, foot is off ground and velocitystarts to increase. Velocity theoratically should decrease to zero when swing phasecompletes and stance phase starts. However, due to the noise and imperfect seg-mentation timing, the velocity doesn’t drop to zero when swing phase ends Fig-39Figure 3.17: Comparison of acceleration before gravity subtraction (a) andafter gravity subtraction (b).ure 3.18 (a).Similarly, displacement should grow step by step. Since we experimented onflat ground, there is no displacement in the z-direction. However, Figure 3.18 (b)shows a small displacement on the z-axis. Moreover, although the displacementin the x direction increases by step, it implies that the participant moves backwardsomehow during the swing phase. That’s also caused by the imperfect timing anddrifts in the IMUs.40Go backwardFigure 3.18: Integration results of velocity and displacement from accelera-tion. (a) Velocity (b) Displacement.The stride length, stance time, swing time, and stance time calculated by eachstep are displayed in Figure 3.19. The predictions and ground truth data for swing,stance, and stride time are close. Figure 3.19 is the chart with two different y-axes to better compare the ground truth and predicted variables of the same kind.Left and right axis have individual scale to fit the range. Time prediction valuescome from the segmentation results, however, stride length is integrated from theacceleration and velocity, which is more subject to IMU noise and shifted values.The summary of the stride length prediction value is displayed in Figure 3.20.The x-axis and y-axis are the predicted and ground truth values of stride lengthseparately. The regression (blue) line in the figure is the y = x function. The411 2 3 4Step (1 unit = 1count)0.40.50.60.7Swing time (s)0.40.50.60.70.80.9Stance time (s)swing/stance TimeSwing-time-preSwing-time-grStance-time-preStance-time-gr1 2 3 4Step (1 unit = 1count)80100120140Stride Length (cm)1.11.151.21.251.3Stride Time (s)strideLength/strideTimestrideLength-prestrideLength-grstrideTime-prestrideTime-gr(a)(b)Figure 3.19: Calculated gait parameters. ”pre” and ”gr” in the legend standsfor prediction values and ground truth respectively. (a) Swing andstance time. (b) Stride Length and stride time.closer the scattered dots are to the regression line, the more accurate the predictionvalues are. As observed, most of the data points are above the regression line,which reveals that the prediction values always underestimate the stride length.Different colors of data points denote separate types of walking patterns. From thecomparison, it’s easy to postulate that, the common normal walking stride length isthe largest among the three types both for predicted values and ground truth data.And Parkinson’s stride length is the smallest, as is shown in Figure 3.21.This is consistent with the inference in previous Section 3.5. Figure 3.21 re-42Figure 3.20: Summaries of the stride length prediction value with unit cm.The x-axis and y-axis are the predicted and ground truth values ofstride length in centimeter separately.ports overall lower predicted data than ground truth. In the ground truth data (or-ange box), there are outliers only in the Parkinson’s trials, which are caused byirregular steps during shuffling walking patterns. The fluctuation for Parkinson’swalking type is also the largest among all the categories. In the predicted values(blue box), there are outliers from all kinds of walking types, some of which areeven larger than two meters. Those outliers are abnormal outcomes from ZUPT al-gorithm, most probably explained by the connected steps. Connected steps happenmost frequently in Parkinson’s walking mainly because of the shuffling, but existin other types of walking as well.Figure 3.22 reports the discrepancy between estimated and ground truth values.The x-axis represents the estimated number while the y-axis denotes the groundtruth, similar to Figure 3.21. It’s indicated that normal walking pattern have morestable strides and the best prediction results from more tight data points in Fig-ure 3.21 (a). Parkinson’s pattern generally has a shorter stride time than the Strokeand normal pattern. Moreover, both Parkinson’s and Stroke patterns have larger43PredictionGround truthStride Length [cm]Figure 3.21: Stride length comparison of different walking types. The blueone is predicted results and the original one is the ground truth valuefrom GaitRite.variations than normal walking, which means pathological patterns are harder topredict than normal walking. Additionally, in pathological patterns, most of the es-timated stride time is smaller than the ground truth values, reported by Figure 3.21(a-c). More abnormal data points are displayed in pathological gaits as outliers.Figure 3.21 (d) also reveals that Normal walking has more consistent and regularstride time in different participants and trials compared with pathological gaits. Theexceptionally large predicted values are mainly due to unrecognized HS momentsand connected steps mentioned before Figure 3.14.The swing and stance time discrepancy are illustrated in the Figure 3.23. Forswing time, predictions are larger than ground truth, and correspondingly, the pre-dictions are smaller in stance time than ground truth. Especially, for Stroke, pre-dictions are more varied in both swing and stance time compared to other typesof walking patterns. The variation is largely the result of asymmetrical walking of44Ground truthPredictiona b cc dFigure 3.22: Stride time comparison for predicted and the ground truth valueof different walking types in time unit s. (a). Normal walking pattern.(b). Parkinson’s pattern. (c). Stroke pattern. (d). Summaries of threetypes of walking pattern in box plot.left and right legs which creates a larger range of gait cycle. This is more obviousin the swing time both for prediction and ground truth data. Normal walking typestill has the most stable predicting and ground truth value while Parkinson’s typeis easier to have abnormal data points in the outlier.The difference between ground truth and estimated stride length is displayed45PredictionGround truthPredictionGround truthFigure 3.23: The stance time and swing time box error of the normal andpathological walking patterns.Table 3.5: Statistical results of stride length difference (absolute value) incentimeterWalking Type Normal Parkinson Stroke OverallAverage 37.428870 18.352208 31.938853 24.9080341st Quartile 30.260269 4.764179 16.262603 8.012589Median 38.826751 10.845871 27.263453 19.3812443rd Quartile 44.851722 23.780752 43.386656 37.794387STD 15.068690 21.545938 20.653407 21.864072in Figure 3.24. Since most of the ground truth values are larger than the predictedvalue, we print the figure by subtracting the estimated stride length from the groundtruth data. The average error for normal walking, Parkinson’s, and Stroke patternsare described in Table 3.5. These results suggest that Normal walking patternsusually have larger stride lengths than Parkinson’s and Stroke patterns. The strokepattern has the largest range of stride length from 16 cm to 43 cm. Part of thereason is that the asymmetrical walking patterns may render stride length as longas normal walking from the functional side and shorter stride length from the mal-functions side. The overall stride length for Parkinson’s is the smallest due to theshuffling gaits.46NormalParkinson StrokeType2001000delta_stride_lengthdifferenceFigure 3.24: The stride length difference between the predicted values andground truth data in length unit centimeter.The MSE results of stride length prediction are displayed in Figure 3.25. Thereis no obvious improvement in normal walking data results than Stroke and Parkin-son’s walking type. However, this is only the result of correctly recognized steps.For missing and over-counted steps, it’s evaluated in previous Section 3.5. Themissing and over-counted steps will cause a much worse effect on the gait pa-rameter calculation. Fortunately, ZUPT secures the calculation results of correctlyrecognized steps. However, due to the absence of magnetometers and imperfectHS and TO timing, the predictions are inaccurate and it’s hard to compare the dif-ference based on the inaccurate results. Therefore, the MSE errors are high for alltypes.In conclusion, the ZUPT algorithm works well on Normal walking patternsin terms of step segmentation, while the performance on the pathological gaits(Parkinson and Stroke) is less satisfying. A key factor for ZUPT to miss or over-count steps on the pathological pattern easily is the varied cadence and velocity47Normal Parkinson StrokeType0.00.20.40.6StridelengthStridelength_MSEFigure 3.25: The MSE results for stride length prediction and ground truthvalues of normal and pathological gait.which is unable to be detected by a single threshold. In the stride length and inte-gration results, there is no evident discrepancy in normal and pathological walkingoutcomes. The major source for the errors is the inaccurate quaternions calcu-lation. The extra noise and reference frame convert difficulties are the commonissues for inaccurate collected data from IMUs. The quaternions will greatly influ-ence the subsequent integration process which leads to under-estimated final gaitresults like stride length.3.6.3 Machine Learning Predicted Gait ParametersTo overcome the quaternion calculation problems and output better performancein the stride length prediction, we utilized machine learning methods to learn thestride length representations. To let the model learn the overall feature of eachstep, we input each step segmentation data into the CNN network and output thepredicted stride length result. To avoid the influence of imperfect timing, we don’t486 6.5 7 7.5 8 8.5 9 9.5 10time (s)-10010acceleration m/sLeft-ZUPT-May212-Normal10input dataAcceleration m/s²ti e (s)Figure 3.26: The input data into the network. The practical input data has alarger range than the predicted and ground truth range.Table 3.6: Statistical results of stride length difference (absolute value) incentimeter ([cm]) (ML method), and improvement compared to previousmethod Table 3.5.Statistical Value ImproveAverage 13.9694 43.94%1st Quartile 7.3250 8.58%Median 11.6500 39.89%3rd Quartile 18.0250 52.31%STD 10.3968 52.45%throw the accurate step segmentation into the network, we include extra time beforethe TO and after HS moment as is shown in Figure 3.26.We randomly select ten trials as validation data. In the training, we mix alltypes of walking patterns together to train a general model for input data, no matterit’s normal walking, Parkinson’s, or Stroke data. Figure 3.27 illustrates the estima-tion result on the validation data, which has achieved better overall performanceon all three categories of walking patterns with stride length varied from 20 cm to120 cm. The model learns well about presentations of distinctive walking typesand achieves great performance in predicting stride length in a large range.The statistical outcome of the ML algorithm is displayed in the Table 3.6. Com-49Figure 3.27: The prediction results and ground truth data on the validationdataset in length unit meter ([m]).pared with the statistical results by traditional methods (Kalman filter and integra-tion) Table 3.5, ML reaches more accurate estimations. The average errors betweenthe prediction and ground truth values have reduced from 24.90cm to 13.96cm,(43.94% improved). Similarly, the improvement on the 1st quartile, median, 3rdquartile, and STD results have been improved 8.58%, 39.89%, 52.31%, 52.45%separately.In summary, the ZUPT algorithm performs well on normal walking patterns butlacks satisfying accuracy in pathological gaits in step segmentation. The Kalmanfilter, sensor fusion, and integration results based on the ZUPT are greatly subject tothe noise and drifts in the IMUs, with average errors of 24cm in stride length predic-tion. To improve the experiment results, we adapt CNN networks to compensatefor the errors that happened in the sensor fusion and integration algorithm, whichhas achieved an average of 43.94% progress and reduced the errors in stride lengthestimation to 13.97cm (average value). Overall, ML can greatly improve the stridelength predictions compared to traditional methods by avoiding difficult issues likeaccurate timing for HS and TO moments and sensor fusion errors.50Chapter 4Conclusions and Future WorkThis work focuses on the interdisciplinary field of wearable sensors and healthcareareas. We first proposed a gait dataset for pathological gaits including Stroke andParkinson’s patients. We collected ground truth data from the GaitRite system.We have discussed the difficulties of utilizing traditional ways to do gait parame-ter prediction by gravity subtraction and second integration from acceleration anddemonstrate the effectiveness of applying machine learning algorithms in gait pa-rameter prediction. Except for the gait parameter prediction for pathological pa-tients, we also demonstrated the accurate prediction of pose estimation for tactileknee sleeves.4.1 ConclusionsGait Analysis: The gait analysis for patients with gait disorders is playing a vitalrole in clinical diagnosis and assessments. Previous researchers demonstratethe ZUPT algorithm performance and traditional methods to calculate thegait parameters. However, without the assistance of magnetometers, the ac-celeration drifts away quickly after several steps. To solve the issues, weutilized machine learning methods to predict the stride length where gravitysubtraction isn’t indispensable anymore. We got very accurate results fromML methods compared with the traditional methods.514.2 Future WorkGait Analysis: We have healthy subjects mimicking the pathological gaits in theprevious studies. While the data are still meaningful, future data collec-tions from real patients are anticipated to further validate and compare withthe current data. However, wearable sensors will suffer from domain shiftsand data privacy issues when connecting to the Internet. Federated learningmight be a good solution to this problem. We are motivated to add in thispart to achieve better accuracy of gait analysis in the future.Pose Estimation: Pose estimation has been a popular computer vision task for along period. And the state-of-art AI methods have opened up new insightsin this area. The wearable sensors can provide more flexible and portablemethods for daily activity tracking and support guidance for sports injuryrehabilitation. We can try to restore the lower body locomotion from moresensors.As the field of gait analysis and pose estimation continues to evolve, therewill likely be many new insights and advancements in both research andtechnology. As such, it will be important to stay up-to-date with the latestdevelopments and to continue exploring innovative approaches to enhanceour understanding of human movement patterns and improve patient out-comes.52Bibliography[1] Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, VolodymyrKuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado,Sebastian Thrun, and Jeff Dean. A guide to deep learning in healthcare.Nature medicine, 25(1):24–29, 2019. → page 1[2] Aileen Nielsen. Practical time series analysis: Prediction with statistics andmachine learning. O’Reilly Media, 2019. → page 1[3] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart,Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressivebody capture: 3d hands, face, and body from a single image. In Proceedingsof the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pages 10975–10985, 2019.[4] Subramanian Sundaram, Petr Kellnhofer, Yunzhu Li, Jun-Yan Zhu, AntonioTorralba, and Wojciech Matusik. Learning the signatures of the human graspusing a scalable tactile glove. Nature, 569(7758):698–702, 2019. → page 1[5] Shweta H Jambukia, Vipul K Dabhi, and Harshadkumar B Prajapati.Classification of ecg signals using machine learning techniques: A survey.In 2015 International Conference on Advances in Computer Engineeringand Applications, pages 714–721. IEEE, 2015. → page 2[6] Jamileh Yousefi and Andrew Hamilton-Wright. Characterizing emg datausing machine-learning tools. Computers in biology and medicine, 51:1–13,2014. → page 2[7] Hyerim Lim, Bumjoon Kim, and Sukyung Park. Prediction of lower limbkinetics and kinematics during walking by a single imu on the lower backusing machine learning. Sensors, 20(1):130, 2019. → page 2[8] Romain Tavenard, Johann Faouzi, Gilles Vandewiele, Felix Divo, GuillaumeAndroz, Chester Holtz, Marie Payne, Roman Yurchak, Marc Rußwurm,53Kushal Kolar, et al. Tslearn, a machine learning toolkit for time series data.J. Mach. Learn. Res., 21(118):1–6, 2020. → page 2[9] Yiwen Gao, He Li, and Yan Luo. An empirical study of wearabletechnology acceptance in healthcare. Industrial Management & DataSystems, 2015. → page 2[10] Shyamal Patel, Hyung Park, Paolo Bonato, Leighton Chan, and MaryRodgers. A review of wearable sensors and systems with application inrehabilitation. Journal of neuroengineering and rehabilitation, 9(1):1–17,2012. → page 2[11] Arthur Asuncion and David Newman. Uci machine learning repository,2007. → page 3[12] Xiaojin Zhu and Andrew B Goldberg. Introduction to semi-supervisedlearning. Synthesis Lectures on Artificial Intelligence and MachineLearning, 3(1):1–130, 2009. → pages xi, 4, 6, 7, 8[13] Yann LeCun, Le´on Bottou, Yoshua Bengio, and Patrick Haffner.Gradient-based learning applied to document recognition. Proceedings ofthe IEEE, 86(11):2278–2324, 1998. → pages xi, 4, 13, 14, 15[14] Jayanth Koushik. Understanding convolutional neural networks. arXivpreprint arXiv:1605.09081, 2016. → pages xi, 4, 13[15] Alex Sherstinsky. Fundamentals of recurrent neural network (rnn) and longshort-term memory (lstm) network. Physica D: Nonlinear Phenomena,404:132306, 2020. → pages xi, 4, 15, 16[16] Greg Welch, Gary Bishop, et al. An introduction to the kalman filter.SIGGRAPH 2001, 1995. → pages xi, 4, 10, 11[17] Giulia Pacini Panebianco, Maria Cristina Bisi, Rita Stagni, and SilviaFantozzi. Analysis of the performance of 17 algorithms from a systematicreview: Influence of sensor position, analysed variable and computationalapproach in gait timing estimation from imu measurements. Gait & posture,66:76–82, 2018. → pages 4, 20[18] Isaac Skog, Peter Handel, John-Olof Nilsson, and Jouni Rantakokko.Zero-velocity detection—an algorithm evaluation. IEEE Transactions onBiomedical Engineering, 57(11):2657–2666, 2010. → pages 28, 3754[19] Sang Kyeong Park and Young Soo Suh. A zero velocity detection algorithmusing inertial sensors for pedestrian navigation systems. Sensors,10(10):9163–9178, 2010. → pages 4, 28[20] Siamak Yousefi, Hirokazu Narui, Sankalp Dayal, Stefano Ermon, andShahrokh Valaee. A survey on behavior recognition using wifi channel stateinformation. IEEE Communications Magazine, 55(10):98–104, 2017. →page 5[21] Liming Chen, Jesse Hoey, Chris D Nugent, Diane J Cook, and Zhiwen Yu.Sensor-based activity recognition. IEEE Transactions on Systems, Man, andCybernetics, Part C (Applications and Reviews), 42(6):790–808, 2012.[22] Nishkam Ravi, Nikhil Dandekar, Preetham Mysore, and Michael L Littman.Activity recognition from accelerometer data. In Aaai, volume 5, pages1541–1546. Pittsburgh, PA, 2005. → page 5[23] Muhammad Mubashir, Ling Shao, and Luke Seed. A survey on falldetection: Principles and approaches. Neurocomputing, 100:144–152, 2013.→ page 5[24] Akara Supratak, Hao Dong, Chao Wu, and Yike Guo. Deepsleepnet: Amodel for automatic sleep stage scoring based on raw single-channel eeg.IEEE Transactions on Neural Systems and Rehabilitation Engineering,25(11):1998–2008, 2017. → page 5[25] Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu.Federated learning. Synthesis Lectures on Artificial Intelligence andMachine Learning, 13(3):1–207, 2019. → pages xi, 8, 9[26] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federatedmachine learning: Concept and applications. ACM Transactions onIntelligent Systems and Technology (TIST), 10(2):1–19, 2019. → page 8[27] Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, AlexIngerman, Vladimir Ivanov, Chloe´ Kiddon, Jakub Konecˇny´, StefanoMazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, DanielRamage, and Jason Roselander. Towards federated learning at scale: Systemdesign. In A. Talwalkar, V. Smith, and M. Zaharia, editors, Proceedings ofMachine Learning and Systems, volume 1, pages 374–388, 2019. → pages8, 955[28] Dinh C Nguyen, Ming Ding, Pubudu N Pathirana, Aruna Seneviratne, JunLi, and H Vincent Poor. Federated learning for internet of things: Acomprehensive survey. IEEE Communications Surveys & Tutorials,23(3):1622–1658, 2021. → page 9[29] Mohamed Amine Ferrag, Othmane Friha, Leandros Maglaras, HelgeJanicke, and Lei Shu. Federated deep learning for cyber security in theinternet of things: Concepts, applications, and experimental analysis. IEEEAccess, 9:138509–138542, 2021.[30] Ahmed Imteaj, Urmish Thakker, Shiqiang Wang, Jian Li, and M HadiAmini. A survey on federated learning for resource-constrained iot devices.IEEE Internet of Things Journal, 9(1):1–24, 2021. → page 9[31] Yi Liu, Sahil Garg, Jiangtian Nie, Yang Zhang, Zehui Xiong, Jiawen Kang,and M Shamim Hossain. Deep anomaly detection for time-series data inindustrial iot: A communication-efficient on-device federated learningapproach. IEEE Internet of Things Journal, 8(8):6348–6358, 2020. → page9[32] Weishan Zhang, Qinghua Lu, Qiuyu Yu, Zhaotong Li, Yue Liu, Sin Kit Lo,Shiping Chen, Xiwei Xu, and Liming Zhu. Blockchain-based federatedlearning for device failure detection in industrial iot. IEEE Internet ofThings Journal, 8(7):5926–5937, 2020. → page 9[33] Mingqing Chen, Rajiv Mathews, Tom Ouyang, and Franc¸oise Beaufays.Federated learning of out-of-vocabulary words. arXiv preprintarXiv:1903.10635, 2019. → page 9[34] Utkarsh P Chinchole and Shital Raut. Federated learning for estimating airquality. In 2021 12th International Conference on ComputingCommunication and Networking Technologies (ICCCNT), pages 1–7. IEEE,2021. → page 9[35] Nicola Rieke, Jonny Hancox, Wenqi Li, Fausto Milletari, Holger R Roth,Shadi Albarqouni, Spyridon Bakas, Mathieu N Galtier, Bennett A Landman,Klaus Maier-Hein, et al. The future of digital health with federated learning.NPJ Digital Medicine, 3(1):1–7, 2020. → page 9[36] Micah J Sheller, Brandon Edwards, G Anthony Reina, Jason Martin, SarthakPati, Aikaterini Kotrotsou, Mikhail Milchenko, Weilin Xu, Daniel Marcus,Rivka R Colen, et al. Federated learning in medicine: facilitating56multi-institutional collaborations without sharing patient data. ScientificReports, 10(1):1–12, 2020. → page 9[37] Ittai Dayan, Holger R Roth, Aoxiao Zhong, Ahmed Harouni, AmilcareGentili, Anas Z Abidin, Andrew Liu, Anthony Beardsworth Costa,Bradford J Wood, Chien-Sung Tsai, et al. Federated learning for predictingclinical outcomes in patients with covid-19. Nature medicine,27(10):1735–1743, 2021. → page 9[38] Florian Hartmann. Federated learning. Masterarbeit am Institut furInformatik der Freien Universit at Berlin, Arbeitsgruppe TheoretischeInformatik, Master thesis, 2018. → page 9[39] Florian Hartmann, Sunah Suh, Arkadiusz Komarzewski, Tim D Smith, andIlana Segall. Federated learning for ranking browser history suggestions.arXiv preprint arXiv:1911.11807, 2019. → page 9[40] Solmaz Niknam, Harpreet S Dhillon, and Jeffrey H Reed. Federatedlearning for wireless communications: Motivation, opportunities, andchallenges. IEEE Communications Magazine, 58(6):46–51, 2020. → page 9[41] Jakub Konecˇny`, H Brendan McMahan, Felix X Yu, Peter Richta´rik,Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategiesfor improving communication efficiency. arXiv preprint arXiv:1610.05492,2016.[42] H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, andBlaise Agu¨era y Arcas. Communication-efficient learning of deep networksfrom decentralized data, 2017.[43] Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally. Deepgradient compression: Reducing the communication bandwidth fordistributed training. arXiv preprint arXiv:1712.01887, 2017. → page 9[44] Holger R Roth, Ken Chang, Praveer Singh, Nir Neumark, Wenqi Li, VikashGupta, Sharut Gupta, Liangqiong Qu, Alvin Ihsani, Bernardo C Bizzo, et al.Federated learning for breast density classification: A real-worldimplementation. In Domain adaptation and representation transfer, anddistributed and collaborative learning, pages 181–191. Springer, 2020. →page 9[45] Xiaoxiao Li, Yufeng Gu, Nicha Dvornek, Lawrence H Staib, PamelaVentola, and James S Duncan. Multi-site fmri analysis using57privacy-preserving federated learning and domain adaptation: Abide results.Medical Image Analysis, 65:101765, 2020. → page 9[46] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aure´lien Bellet,Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles,Graham Cormode, Rachel Cummings, et al. Advances and open problems infederated learning. Foundations and Trends® in Machine Learning,14(1–2):1–210, 2021. → page 9[47] Richard J Meinhold and Nozer D Singpurwalla. Understanding the kalmanfilter. The American Statistician, 37(2):123–127, 1983. → page 10[48] Keiron O’Shea and Ryan Nash. An introduction to convolutional neuralnetworks. arXiv preprint arXiv:1511.08458, 2015. → pages xi, 12, 13[49] Saad Albawi, Tareq Abed Mohammed, and Saad Al-Zawi. Understanding ofa convolutional neural network. In 2017 International Conference onEngineering and Technology (ICET), pages 1–6. Ieee, 2017. → pagesxi, 12, 14, 15[50] Alexander Waibel, Toshiyuki Hanazawa, Geoffrey Hinton, KiyohiroShikano, and Kevin J Lang. Phoneme recognition using time-delay neuralnetworks. IEEE Transactions on Acoustics, Speech, and Signal Processing,37(3):328–339, 1989. → page 12[51] Wei Zhang, Jun Tanida, Kazuyoshi Itoh, and Yoshiki Ichioka. Shift-invariantpattern recognition neural network and its optical architecture. InProceedings of Annual Conference of the Japan Society of Applied Physics,pages 2147–2151, 1988. → page 12[52] Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing thedimensionality of data with neural networks. science, 313(5786):504–507,2006. → page 12[53] Alexander Toshev and Christian Szegedy. Deeppose: Human poseestimation via deep neural networks. In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition, pages 1653–1660, 2014. →page 15[54] Wenpeng Yin, Katharina Kann, Mo Yu, and Hinrich Schu¨tze. Comparativestudy of cnn and rnn for natural language processing. arXiv preprintarXiv:1702.01923, 2017. → pages xi, 1658[55] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization withgumbel-softmax. arXiv preprint arXiv:1611.01144, 2016. → page 16[56] Sepp Hochreiter and Ju¨rgen Schmidhuber. Long short-term memory. NeuralComputation, 9(8):1735–1780, 1997. → pages xii, 17[57] Sue Blackman. Beginning 3D Game Development with Unity 4: All-in-one,multi-platform game development. Apress, 2013. → page 18[58] Ross Stirling, Jussi Collin, Ken Fyfe, and Ge´rard Lachapelle. An innovativeshoe-mounted pedestrian navigation system. In Proceedings of Europeannavigation conference GNSS, volume 110, 2003. → page 20[59] Belinda Bilney, Meg Morris, and Kate Webster. Concurrent related validityof the gaitrite® walkway system for quantification of the spatial andtemporal parameters of gait. Gait & posture, 17(1):68–74, 2003. → page 21[60] Hylton B Menz, Mark D Latt, Anne Tiedemann, Marcella Mun San Kwan,and Stephen R Lord. Reliability of the gaitrite® walkway system for thequantification of temporo-spatial parameters of gait in young and olderpeople. Gait & posture, 20(1):20–25, 2004. → page 21[61] Andrew L McDonough, Mitchell Batavia, Fang C Chen, Soonjung Kwon,and James Ziai. The validity and reliability of the gaitrite system’smeasurements: A preliminary evaluation. Archives of physical medicine andrehabilitation, 82(3):419–425, 2001. → page 21[62] Anat Mirelman, Paolo Bonato, Richard Camicioli, Terry D Ellis, Nir Giladi,Jamie L Hamilton, Chris J Hass, Jeffrey M Hausdorff, Elisa Pelosin, andQuincy J Almeida. Gait impairments in parkinson’s disease. The LancetNeurology, 18(7):697–708, 2019. → page 21[63] Sandra J Olney and Carol Richards. Hemiparetic gait following stroke. parti: Characteristics. Gait & posture, 4(2):136–148, 1996. → pages 21, 25[64] William Premerlani and Paul Bizard. Direction cosine matrix imu: Theory.Diy Drone: Usa, 1, 2009. → page 26[65] Isaac Skog, John-Olof Nilsson, and Peter Ha¨ndel. Evaluation ofzero-velocity detectors for foot-mounted inertial navigation systems. In2010 International Conference on Indoor Positioning and IndoorNavigation, pages 1–6. IEEE, 2010. → page 2859[66] Tino Sto¨ckel, Robert Jacksteit, Martin Behrens, Ralf Skripitz, Rainer Bader,and Anett Mau-Moeller. The mental representation of the human gait inyoung and older adults. Frontiers in Psychology, 6:943, 2015. → pagesxiii, 34, 36[67] Andrea Cereatti, Tecla Bonci, Massoud Akbarshahi, Kamiar Aminian,Arnaud Barre´, Mickael Begon, Daniel L Benoit, Caecilia Charbonnier,Fabien Dal Maso, Silvia Fantozzi, et al. Standardization proposal of softtissue artefact description for data sharing in human motion measurements.Journal of Biomechanics, 62:5–13, 2017. → page 35[68] Sebastian OH Madgwick, Andrew JL Harrison, and Ravi Vaidyanathan.Estimation of imu and marg orientation using a gradient descent algorithm.In 2011 IEEE International Conference on Rehabilitation Robotics, pages1–7. IEEE, 2011. → page 3860Appendix AGait AnalysisA.1 MATLAB ProgramThis section provides related MATLAB program related to Chapter 3.A.1 procedure includes the minimum search for zo f f and zscale pair in Sec-tion 3.4.1.A.2 is the code for gravity subtraction formulation without magnetometer inSection 3.4.1.Listing A.1: MATLAB procedure to remove stationary drifts from IMUs1 f u n c t i o n [ x , y , z ]= s c a l e d r i f t r e m o v e ( mean value )2 %% mean value conta ins the mean value o f s t a t i o n a r y IMU readings when ...f ac ing downward & upward separa te ly from x , y and z ax is .3 g = 9.80665; % g r a v i t a t i o n a l constant4 x0 = [ 0 , 1 ] ; % s t a r t po in t , x0 ( 0 ) stands f o r zo f f and x0 ( 1 ) stands f o r zscale5 opt ions = opt imset ( ' MaxFunEvals ' ,2000) ;67 fun = @( z ) ( ( ( mean value{1 ,1} (3)+z ( 1 ) ) * z ( 2 ) −g ) ˆ2+ ...( ( mean value{2 ,1} (3)+z ( 1 ) ) * z ( 2 ) +g ) ˆ 2 ) ;8 z = fminsearch ( fun , x0 , op t ions ) ;9 disp ( num2str ( ( mean value{1 ,1} (3)+z ( 1 ) ) * z ( 2 ) ) )10 disp ( num2str ( ( mean value{2 ,1} (3)+z ( 1 ) ) * z ( 2 ) ) )1112 fun = @( y ) ( ( ( mean value{3 ,1} (2)+y ( 1 ) ) * y ( 2 ) −g ) ˆ2+ ...( ( mean value{4 ,1} (2)+y ( 1 ) ) * y ( 2 ) +g ) ˆ 2 ) ;13 y = fminsearch ( fun , x0 , op t ions ) ;6114 disp ( num2str ( ( mean value{3 ,1} (2)+y ( 1 ) ) * y ( 2 ) ) )15 disp ( num2str ( ( mean value{4 ,1} (2)+y ( 1 ) ) * y ( 2 ) ) )1617 fun = @( x ) ( ( ( mean value{6 ,1} (1)+x ( 1 ) ) * x ( 2 ) −g ) ˆ2+ ...( ( mean value{5 ,1} (1)+x ( 1 ) ) * x ( 2 ) +g ) ˆ 2 ) ;18 x = fminsearch ( fun , x0 , op t ions ) ;19 disp ( num2str ( ( mean value{6 ,1} (1)+x ( 1 ) ) * x ( 2 ) ) )20 disp ( num2str ( ( mean value{5 ,1} (1)+x ( 1 ) ) * x ( 2 ) ) )21 endListing A.2: MATLAB procedure to remove gravity from acceleration1 f u n c t i o n [ g acc , s t a t u s d r i f t ] = mykalmen ( data , fs , f i lename )2 %This i s a kalmen f i l t e r f o r IMU data3 % data ( : , 1 : 3 ) : accelerometer readings , data ( : , 4 : 6 ) : gyroscope readings .45 decim =1;6 fuse = i m u f i l t e r ( ' SampleRate ' , fs , ' Decimat ionFactor ' , decim , ...' L inearAcce lera t ionDecayFactor ' , 0 . 8 ) ;78 q = fuse ( data ( : , 1 : 3 ) , deg2rad ( data ( : , 4 : 6 ) ) ) ;9 q = compact ( q ) ;10 S t a r t s t a t e = S t a r t S t a t e ( data ( : , 1 : 3 ) ) ;11 q s = mean( q ( 1 : S t a r t s t a t e , : ) ,1 ) ;12 i f sum( abs (mean( q ) −q s ) > [ 0 .1 ,0 .1 ,0 .1 ,0 .1 ] )13 s t a t u s d r i f t =1;14 else15 s t a t u s d r i f t =0;16 end1718 f i g u r e (name=f i lename ) ; p l o t ( q )19 legend ( 'w ' , ' x ' , ' y ' , ' z ' )20 t i t l e ( ' Ca lcu la ted q ' )2122 % f i n d out the i n i t i a l index23 a s =[0 ,0 , norm ( mean( data ( 1 : S t a r t s t a t e , 1 : 3 ) ,1 ) ) ] ;2425 % sub t rac t g r a v i t y26 g acc=GravitySub ( data ( : , 1 : 3 ) ,q , a s ) ;2728 end2930 f u n c t i o n [ g acc ]= GravitySub ( acc , q , a s )31 g acc = zeros ( s ize ( acc ) ) ;32 f o r i = 1 : leng th ( acc )6233 acc temp = [0 acc ( i , : ) ] ;34 acc temp = q u a t m u l t i p l y ( q u a t m u l t i p l y ( q ( i , : ) , acc temp ) , quatcon j ( q ( i , : ) ) ) ;35 g acc ( i , : ) = acc temp ( 2 : 4 ) − a s ;36 end37 end3839 f u n c t i o n [ Windowini t ]= S t a r t S t a t e ( acc )40 acc mag = s q r t ( acc ( : , 1 ) . * acc ( : , 1 ) + acc ( : , 2 ) . * acc ( : , 2 ) + acc ( : , 3 ) . * acc ( : , 3 ) ) ;41 Windowsize =50;42 WindowPeriod=zeros (1 , leng th ( acc mag ) ) ;43 t h resho ld =0.02; % f i n d out the s t a t i o n a r y pa r t4445 f o r i =1 :1 : leng th ( acc mag ) −Windowsize46 i f var ( acc mag ( i : i +Windowsize ) )>t h resho ld % use var iance as c r i t e r i o n47 WindowPeriod ( i : i +Windowsize ) =ones ( s ize ( WindowPeriod ( i : i +Windowsize ) ) ) ;48 break ;49 end50 end51 Windowini t= f i n d ( WindowPeriod==1 , 1) ;52 endA.2 Statistical Results for Each ParticipantThis section attaches statistical results related to each participant.Table A.1 shows the average value, median value, maximum and STD of miss-ing steps for different participants. The value is normalized by correctly recognizedsteps extracted from GAITRite systems.Table A.2 displays also the average value, median value, maximum and STD ofover counted steps for different participants. The value is normalized by correctlyrecognized steps extracted from GAITRite systems.63Table A.1: Statistical results of missing steps for each participantWalking Type(Normal/Parkinson/Stroke)Average Median Maximum STDP1(NormalParkinsonStroke)0.01190.04400.3567000.34850.14290.18180.75000.04120.06940.1909P200.01300.2032000.062500.09090.700000.03300.2631P300.06560.158800.03850.105600.23080.500000.08190.1509P400.25480.275100.03130.25000.92310.571400.38380.1768P50.01430.23600.353300.20200.24100.20000.50000.81820.05350.13690.3193P600.15980.190300.12500.118100.40000.500000.12270.2061P700.12320.219300.09100.111100.33330.777800.09870.2501P800.11010.358700.06460.316700.37500.909100.10080.2818P900.00480.1735000.166700.06670.500000.01780.1486P1000.0100.3041000.25000.07690.937500.02700.325564Table A.2: Statistical results of overcounted steps for each participantWalking Type(Normal/Parkinson/Stroke)Average Median Maximum STDP1(NormalParkinsonStroke)0.02780.02160.04420000.33330.18180.27270.09620.05510.0815P20.02080.04350.03040000.16670.18180.22220.05890.05770.0730P300.05720.066400.0769000.16670.333300.05580.1171P4000.0417000000.1250000.0616P5000.0153000000.10000.053500.0359P600.00600.032900000.07140.250000.02060.0732P700.02360.04510000.16670.15390.25000.03040.04180.0732P800.01800.025600000.16670.100000.04360.0431P9000.0271000000.2222000.0632P10000.0119000000.0833000.030365
Thesis/Dissertation
eng
Electrical and Computer Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Servati, Peyman
Machine learning based gait parameter analysis for normal and pathological patterns
Text