Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Gait sensing on smartphones to support mobile exercise applications and games 2012

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
ubc_2012_fall_schneider_oliver.pdf [ 6.67MB ]
Metadata
JSON: 1.0052125.json
JSON-LD: 1.0052125+ld.json
RDF/XML (Pretty): 1.0052125.xml
RDF/JSON: 1.0052125+rdf.json
Turtle: 1.0052125+rdf-turtle.txt
N-Triples: 1.0052125+rdf-ntriples.txt
Citation
1.0052125.ris

Full Text

Sensing Gait on Smartphones to Support Mobile Exercise Applications and Games by Oliver Stirling Schneider B.Sc. Computer Science, University of Saskatchewan, 2010 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Science in THE FACULTY OF GRADUATE STUDIES (Computer Science) The University Of British Columbia (Vancouver) August 2012 c© Oliver Stirling Schneider, 2012 Abstract To encourage and support physical activity in increasingly sedentary lifestyles, many are turning to mobile technology. Modern smartphones are equipped with a wealth of sensors, including Global Positioning Systems (GPS) and accelerom- eters, suggesting great potential to be integrated with fitness and exercise applica- tions. So far, GPS-enabled devices have been used to support running, cycling, or even exercise games that encourage people to be physically active, but GPS- enabled devices lack fine-grained information about the user’s activity. Accelerom- eters have been used to some effect to detect step count and walking cadence (step rate), and even to classify activity (distinguishing walking from cycling, for exam- ple), but require a known carrying location and orientation. In this work, we ex- amine the role of location in two application areas - real-time cadence estimation and gait classification - and develop algorithms to accommodate diverse carrying locations. In the first application area, real-time cadence estimation, our algorithm (Ro- bust Real-time Algorithm for Cadence Estimation, or RRACE) uses a frequency- domain analysis to perform well without training or tuning, and is robust to changes in carrying locations. We demonstrate RRACE’s performance and robustness to be an improvement over existing algorithms with data collected from a user study. In the second application area, gait classification, we present a novel set of 15 gaits suitable for exercise games and other fitness applications. Using a minimal amount of training for participants, we can achieve a mean of 78.1% classifica- tion for all 15 gaits and all locations, an accuracy which may be usable now in some applications and warrants further investigation of this approach. We present findings of how our classification scheme confuses these gaits, and encapsulate ii insights in guidelines for designers. We also demonstrate that our classification performance varies dramatically for each individual even when trained and tested on that individual, suggesting strong individual differences in performing gaits. Our innovative methodology for simple and quick collection of accelerometer data is also described in detail. Future work includes planned improvements to both al- gorithms, further investigation of individual differences, and extension of this work to other application areas. iii Preface There are several collaborators without whom this work could not have happened as it did. In this preface, I describe the relative contributions of all people involved in the research presented in this thesis, all resulting publications, and ethics ap- proval. At the moment of writing, there are three publications planned that are associ- ated with this thesis. The first planned publication reflects the cadence estimation work, primarily described in Chapter 3. The second planned publication describes the gait classification work, described primarily in Chapter 4, Chapter 5, and Chap- ter 6. The third planned publication describes a software package, GaitLib, devel- oped as a result of software development conducted during this research. The details of these collaborations are described below. The cadence estimation work was conducted with the following people: PhD student Idin Karuei, undergraduate research assistant Bryan Stern, undergradu- ate research assistant Michelle Chuang, and supervisor and principal investigator Karon MacLean. Idin was the primary researcher for this work. Oliver’s con- tribution to this work included planning and helping to conduct the main study; implementing, testing, and training the time-domain algorithm; reimplementing the cadence algorithm for further testing; consultation when analyzing the results; and contributing major sections to the journal manuscript, which the group has prepared as a result of this work. Chapter 3 has been written to include the entire work for framing and readability, with Oliver’s contributions elaborated upon and his non-contributions minimized. Essentially, the chapter is a “fish-eye” view of the manuscript, highlighting Oliver’s contributions. The gait classification work includes the following collaborators: Idin Karuei, iv post-doctoral researcher Kerem Altun, undergraduate research assistant Michael Ming-An Wu, and Karon MacLean. This work was primarily done by Oliver. Dr. Maclean provided a great deal of mentorship, guidance, and feedback. Idin was available for consultation, given his experience with the cadence detection. Dr. Altun provided valuable information regarding machine learning and sensor (accelerometer and gyroscope) use. Michael provided advice with insights drawn from his development of the GaitLib software system. Finally, the GaitLib software system involved the following contributors: Idin Karuei, Bryan Stern, Michelle Chuang, Michael Ming-An Wu, and Karon MacLean. Michael was the primary developer for this system and writer of the planned pub- lication. Oliver was a major contributor and principal architect for the software package, and provided mentorship and advice to Michael’s development. Idin, Bryan, and Michelle provided the original software development with the cadence detection software package. Michael took the software developed for both of these studies and, with help from Oliver, Idin, and Dr. MacLean, developed it to incor- porate gait classification. At the time of this writing, all three publications are at different stages. The ca- dence estimation manuscript is titled “RRACE: Robust Real-time Algorithm for Cadence Estimation” by Idin Karuei, Oliver Schneider, Bryan Stern, Michelle Chuang, and Karon MacLean; this paper is about to be submitted for review to a top-tier journal. The gait classification manuscript, based on Chapter 4, Chap- ter 5, and Chapter 6 of this thesis, is in preparation for submission to a journal or conference with Oliver as the primary author. The GaitLib software publication, tentatively titled “Coding Gait: A Library for Gait Recognition in Mobile Phones” by Michael Ming-An Wu, Oliver Schneider, Idin Karuei, and Karon MacLean, is in preparation for submission to a conference. Finally, this work was approved by the University of British Columbia (UBC) Behavioural Research Ethics Board (BREB), with ethics number #H01-80470. All study forms can be found in Appendix A. v Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Application Space . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1.1 Exercise Applications . . . . . . . . . . . . . . . . . . . 6 2.1.2 Exergames . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.3 Mobile Context-Aware Applications . . . . . . . . . . . . 10 2.1.4 Positioning this work in the application space . . . . . . . 11 2.2 Real-Time Mobile Gait Analysis . . . . . . . . . . . . . . . . . . 12 2.2.1 Step detection and Cadence Estimation . . . . . . . . . . 12 vi 2.2.2 Activity Detection and Gait Classification . . . . . . . . . 13 2.2.3 Gait-Based Individual Identification . . . . . . . . . . . . 18 2.2.4 The Role of Location . . . . . . . . . . . . . . . . . . . . 18 3 Cadence Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.1 The RRACE Algorithm . . . . . . . . . . . . . . . . . . . . . . . 22 3.2 Cadence Study Data Collection . . . . . . . . . . . . . . . . . . . 23 3.2.1 Experiment Design, Metrics and Subjects . . . . . . . . . 24 3.2.2 Results for Outdoor Validation of 4 Second Window RRACE 25 3.3 Comparing RRACE with a Threshold-based Time-domain Algorithm 28 3.3.1 Implementation of Time-based Algorithm for Comparison 28 3.3.2 Training of the Time-based Algorithm . . . . . . . . . . . 30 3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.4.1 The nature of RRACE’s Error . . . . . . . . . . . . . . . 34 3.4.2 RRACE Performs Best with the Arm, Bag, Belt, and Front Pocket Locations . . . . . . . . . . . . . . . . . . . . . . 34 3.4.3 RRACE is Robust to Subject Differences . . . . . . . . . 34 3.4.4 RRACE Window Length of 4 seconds is Best: . . . . . . . 35 3.4.5 RRACE is Sensitive to Very Slow Speeds . . . . . . . . . 35 3.5 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . 35 4 Gait Classification Data Collection . . . . . . . . . . . . . . . . . . 37 4.1 Gait Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.2 GaitLogger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.3 Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.3.1 Participant Recruitment . . . . . . . . . . . . . . . . . . 45 4.3.2 Setup and Participant Briefing . . . . . . . . . . . . . . . 45 4.3.3 Data Collection . . . . . . . . . . . . . . . . . . . . . . . 46 4.3.4 Study Conclusion . . . . . . . . . . . . . . . . . . . . . . 48 4.4 Data Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4.2 Data Quality . . . . . . . . . . . . . . . . . . . . . . . . 48 vii 5 Gait Classification Algorithm Development . . . . . . . . . . . . . . 51 5.1 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.1.1 Window Sizes . . . . . . . . . . . . . . . . . . . . . . . . 52 5.1.2 Axis Sets . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.1.3 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.2 Algorithm Selection . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.3 Location And Gait . . . . . . . . . . . . . . . . . . . . . . . . . 59 5.3.1 Analysis Design . . . . . . . . . . . . . . . . . . . . . . 59 5.3.2 Computational Setup and Performance . . . . . . . . . . 60 5.3.3 Results - All Participants . . . . . . . . . . . . . . . . . . 60 5.3.4 Results - By-Participant . . . . . . . . . . . . . . . . . . 68 6 Gait Classification Discussion . . . . . . . . . . . . . . . . . . . . . 81 6.1 Overall Success . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.2 Gait is a Complex Ecosystem Involving Gaitset and Location . . . 82 6.3 Individual Differences Have a Strong Influence on Gait Recognition 85 6.4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 A Supporting Materials . . . . . . . . . . . . . . . . . . . . . . . . . . 103 A.1 Cadence Estimation Study Forms . . . . . . . . . . . . . . . . . . 103 A.1.1 Consent Forms . . . . . . . . . . . . . . . . . . . . . . . 103 A.2 Gait Classification Study Forms . . . . . . . . . . . . . . . . . . 106 A.2.1 Consent Forms . . . . . . . . . . . . . . . . . . . . . . . 106 A.2.2 Recruitment Email . . . . . . . . . . . . . . . . . . . . . 109 A.2.3 Demographic Information Form . . . . . . . . . . . . . . 111 A.3 Other Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 A.3.1 Participant Reimbursement Form . . . . . . . . . . . . . 113 viii List of Tables Table 2.1 Overview of related work for real-time activity detection and gait classification with data from smartphones or loosely-carried accelerometers. Please note that this table is intended to high- light the most prominent features of these works, but there are additional differences in classification strategy, extracted fea- tures, signal processing, and methods for calculating accuracy. Please see the text for these details. . . . . . . . . . . . . . . . 17 Table 3.1 Experiment design . . . . . . . . . . . . . . . . . . . . . . . . 24 Table 3.2 Error Ratio differences by body location for four-second win- dow RRACE through an unpaired Z-test. The second column contains the mean Error Ratio of each body location. . . . . . 27 Table 3.3 Error Ratio differences by window sizes of RRACE , with walk- ing speed and body location lumped. Window sizes are ordered by increasing ER mean. . . . . . . . . . . . . . . . . . . . . . 27 Table 3.4 RRACE Error Ratio differences by body location for all win- dow sizes and walking speeds (unpaired Z-test). Locations are ordered by increasing ER mean. . . . . . . . . . . . . . . . . 27 Table 3.5 Unpaired Z-test comparison of error ratios of the best and the worst versions of the frequency-based algorithm and the best of each category of time-based algorithm. Algorithm variants are ordered by increasing ER mean. . . . . . . . . . . . . . . . . 33 Table 4.1 Gaits and descriptions used in our data collection study . . . . 39 ix Table 4.2 Self-reported statistics from 12 participants . . . . . . . . . . . 48 Table 4.3 Summary of recorded accelerometer data . . . . . . . . . . . . 49 Table 4.4 Summary of low-quality accelerometer data . . . . . . . . . . 49 Table 4.5 Summary of GaitLogger sample period over all recorded data (including between trials) . . . . . . . . . . . . . . . . . . . . 50 Table 4.6 Summary of recorded gyroscope data . . . . . . . . . . . . . . 50 Table 4.7 Summary of low-quality gyroscope data . . . . . . . . . . . . 50 Table 5.1 Violations of normality in the results. . . . . . . . . . . . . . . 57 Table 5.2 Top 20 algorithms ranked by mean percent correct. All results shown are the mean value of 10 iterations of 10-fold cross- validation. Analysis was conducted on data from all participants but limited to the Front Pocket location and the Pedestrian gaitset. 58 Table 5.3 Mean gaitset/location accuracy rate over all participants. All results shown are the mean value of 10 iterations of 10-fold cross-validation. Analysis was conducted using XYZ/2s/RF500. 63 Table 5.4 Mean gaitset/location accuracy rate, by-participant. All results shown are the mean value of 10 iterations of 10-fold cross- validation for every participant. Analysis was conducted using XYZ/2s/RF500. . . . . . . . . . . . . . . . . . . . . . . . . . 70 Table 5.5 Top 20 of 336 gaitset/location/participant data cells by mean percent correct. All results shown are the mean value of 10 iterations of 10-fold cross-validation. Analysis was conducted using XYZ/2s/RF500. Three participants are present in this group. 75 Table 5.6 Worst 20 of 336 gaitset/location/participant data cells ranked by mean percent correct. All results shown are the mean value of 10 iterations of 10-fold cross-validation. Analysis was con- ducted using XYZ/2s/RF500. Five participants are present in this group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 x List of Figures Figure 3.1 Experiment walkway, start and end points. . . . . . . . . . . . 24 Figure 3.2 Error Ratio as a function of speed condition for 4-second win- dow RRACE. Please note that dark-red boxplots have a larger range for Error Ratio and light-green boxplots have a smaller one. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Figure 3.3 Example of the MPTrain time-based step detection algorithm . 29 Figure 3.4 Error Ratio, ordered by median, of (a: green) 4 window sizes of RRACE (first four) and time-based algorithm trained on: (b: cyan) all subjects’ body locations (last one), (c: pink) all body locations of each single subject, (d: yellow) one body location of all subjects, and (e: grey) single body location of one subject. . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Figure 4.1 GaitLogger main screen . . . . . . . . . . . . . . . . . . . . 42 Figure 4.2 GaitLogger study runner mode . . . . . . . . . . . . . . . . . 43 Figure 4.3 GaitLogger log mode . . . . . . . . . . . . . . . . . . . . . . 44 Figure 4.4 Participant apparatus . . . . . . . . . . . . . . . . . . . . . . 47 xi Figure 5.1 Bar-and-whiskers plot of the algorithm selection study. In this work, bar-and-whiskers plots report on quantiles; that is, the bar in the middle represents the median, the box represents the 25th percentile to the 75th percentile, and the ends of the whiskers represent the extremes of the dataset’s range exclud- ing outliers. All bar-and-whiskers plots were produced using the R statistical programming language’s bwplot function in the lattice library. . . . . . . . . . . . . . . . . . . . . . . 56 Figure 5.2 Bar-and-whiskers plot of 10 means of 10-fold cross-validation results of the XYZ/2s/RF500 algorithm trained on all partic- ipants by carrying location and gaitset, arranged to highlight differences in gaitset. . . . . . . . . . . . . . . . . . . . . . . 64 Figure 5.3 Bar-and-whiskers plot of 10 means of 10-fold cross-validation results of the XYZ/2s/RF500 algorithm trained on all partic- ipants by carrying location and gaitset, arranged to highlight differences in carrying location. . . . . . . . . . . . . . . . . 65 Figure 5.4 Confusion matrix for a 10-fold cross validation of the cho- sen algorithm (XYZ/2s/RF500) trained on all participants, all gaits, and all carrying locations. Darker squares represent a higher classification rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Figure 5.5 Confusion matrices for a 10-fold cross validation of the cho- sen algorithm (XYZ/2s/RF500) trained on all participants and all carrying locations for each gaitset. The algorithm thus at- tempts to distinguish only between gaits in the gaitset. Darker squares represent a higher classification rate. Numbers pre- sented are percents of classification rates for each actual gait (that is, rows sum to 100%). . . . . . . . . . . . . . . . . . . 67 Figure 5.6 Bar-and-whiskers plot of the Location-Gaitset-Participant Anal- ysis, aggregated by Location and Gaitset. . . . . . . . . . . . 69 Figure 5.7 Bar-and-whiskers plot of the Location-Gaitset Analysis. By- participant analyses are presented by gaitset and location. . . 71 xii Figure 5.8 Bar-and-whiskers plot of the Location-Gaitset Analysis, to high- light gaitset performance for each of participants 1 through 6. Notice that participants tend to consistently perform well or poorly independent of gaitset/location conditions. This graph is continued in Figure 5.9. . . . . . . . . . . . . . . . . . . . 72 Figure 5.9 Bar-and-whiskers plot of the Location-Gaitset Analysis, to high- light gaitset performance for each of participants 7 through 12. This graph is a continuation from Figure 5.8. . . . . . . . . . 73 Figure 5.10 Bar-and-whiskers plot of the Location-Gaitset Analysis. By- participant analyses are presented without any other factors. . 74 Figure 5.11 Confusion matrix for the worst-performing gaitset/location/- participant combination using XYZ/2s/RF500: All Location- s/All Gaits/P10. Darker squares represent a higher classifi- cation rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). . . . . . 76 Figure 5.12 Confusion matrix for the second worst-performing gaitset/lo- cation/participant combination using XYZ/2s/RF500: All Lo- cations/All Gaits/P11. Darker squares represent a higher clas- sification rate. Numbers presented are percents of classifica- tion rates for each actual gait (that is, rows sum to 100%). . . 77 Figure 5.13 Confusion matrix for the third worst-performing gaitset/loca- tion/participant combination using XYZ/2s/RF500: Arm/All Gaits/P11. Darker squares represent a higher classification rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). . . . . . . . . . . 78 Figure 5.14 Confusion matrix for the fourth worst-performing gaitset/lo- cation/participant combination using XYZ/2s/RF500: Back- pack/All Gaits/P10. Darker squares represent a higher classi- fication rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). . . . . . 79 xiii Acknowledgments There are several people that I would like to thank. I am very grateful for the supervision of Dr. Karon MacLean, who helped me through this process every step of the way. I must thank Dr. Michiel van de Panne, my second reader, for his quick reading and thorough comments. Thank you to Idin Karuei, who managed his way through early drafts and was always ready when I had a question, and to Kerem Altun and Mike Wu for all their help. Another thank you to my fellow master’s students, Shathel Haddad, Anna Flagg, and Diane Tam - we did it! I owe a big thank you to my family. Kevin Schneider, Karen Wood, Samuel Schneider, and Anna Schneider gave me much-appreciated advice, and were al- ways around when I needed to take a break. Finally, I could not have done this without Elyse Clancy’s love, support, and constant encouragement. She kept me on track when needed, and answered my worries with her sense of humour. The steady supply of coffee on late nights did not hurt either. My research was funded by the Natural Science and Engineering Research Council of Canada. xiv Dedication Mom, Dad, Sam, Anna, Elyse - this is for you. xv Chapter 1 Introduction If I could not walk far and fast, I think I should just explode and perish. — Charles Dickens We are in a health crisis. Over 38% of Canadians, and 10% adults worldwide, are classified as overweight or obese [58, 63]. Obesity is a risk factor for many dis- eases, including heart disease and stroke, diabetes, and some cancers [63]. Though a result of both genetic and environmental factors [42], successful prevention of obesity can be achieved by reducing sedentary behaviours and increasing active behaviours. Sedentary behaviours, often including activities such as watching tele- vision or playing video games, can increase the likelihood of being overweight [62]. However, turning sedentary activities into active ones can more than double energy expenditure [37]. Exercise, more than diet, has been shown to be effective for achieving healthy weight loss [50]. Indeed, simply increasing step count can provide overall health benefits [7]. All of this points towards a need for people to to be active; unfortunately, 31% of adults worldwide aged 15 and over were insuf- ficiently active in 2008 [64]. There exists a need to better support and encourage physical activity. Recently, many technological solutions have arisen to promote or facilitate physical activity. Modern smartphones1 have a wealth of sensing techniques, such as accelerometers, gyroscopes, and Global Positioning System (GPS) units. These 1“A cell phone that includes additional software functions (as e-mail or an Internet browser) on their person.” [44] 1 sensors allow for novel mobile applications to support physical activity in a natural way, such as matching a song’s beat to a users’ running cadence (i.e., step rate) [47]. This support for physical activity can help people maintain their exercise or weight goals [9]. Knowledge of this physical activity can also be provided to context-aware applications, such as a mobile song recommendation system, that take into account both physical walking information as well as time and location [15]. Others have shown that automatically sensing higher-level activities, such as cycling or climbing, is important for supporting persuasive fitness applications [9]. In particular, automatically sensing different gaits2 as higher-level activities lends itself well to applications involving physical activity. Walking and running are common exercise activities; even everyday choices, such as taking the stairs to the office rather than the elevator, are closely coupled to physical activity. Mo- bile exergames3 can also benefit from sensing novel gaits as an input modality, as many only use location-based sensing or approximations of overall physical activity. Movement on foot is often a context in which users carry smartphones, al- lowing applications and games to naturally use smartphones’ sensing capabilities. All of this points towards the ability to sense gait on smartphones as a promising potential means of supporting physical activity. However, techniques for sensing a user’s gait are not yet robust enough to be widely adopted on commodity smartphones. Although GPS has been widely used for fitness and exercise applications, accelerometers (which sense more fine- grained information) often require major constraints, such as carrying external sensors or placing the smartphone in a known location and orientation [45, 47]. Wearing external sensors is intrusive and social inhibitive [9, 21], costs additional money, and ties the application to a hardware platform that could become obso- lete. Using the smartphone’s internal sensors solves these problems and leverages the pervasiveness of smartphones, but limiting them to a known location and ori- entation ignores the diverse ways that people carry their mobile devices [12]. To truly support physical activity and exercise in a mobile context, modern sensing 2“A manner of walking or moving on foot.” [44]. In this work, we use it to refer to diverse movements, but all are bipedal, directional movements on land, or analogous movements with land vehicles (such as cycling, skateboarding, and driving a car). 3Exercise games; see Section 2.1.2 for a definition. 2 techniques must be able to handle diverse carrying locations on the mobile devices themselves. 1.1 Contributions In this work, we strive to develop better techniques for accelerometer sensing that can handle different carrying locations robustly. We do this for two types of sensed entities: cadence estimation and the classification of the user’s gait. Both of these are suitable for mobile exercise applications, the burgeoning area of exergames, and mobile context-aware applications. We make the following contributions to modern science: • A demonstration that carrying location has a major effect on mobile device’s accelerometer signals. Carrying location must be considered in future work involving smartphones. • A robust cadence estimation algorithm, Robust Real-time Algorithm for Ca- dence Estimation (RRACE), that is robust to carrying location. We include: – A thorough validation study of the RRACE algorithm on mobile smart- phones. – A thorough comparison of RRACE with the (previously) best-performing published cadence estimation algorithm. • A gait classification algorithm developed to recognize a wide variety of gaits with a variety of carrying locations. Though not robust to individual dif- ferences, this algorithm can be rapidly trained or personalized by collecting only 30 seconds of each gait. We include: – A selection of gaits suitable for exercise applications and games, as well as a set of criteria used to determine these gaits. – A demonstration that we can use accelerometer data to infer the users’ gait in real-time (with lag time of only a 2 seconds). – Characterization of influence that carrying location, individual differ- ences, and types of gait have on classification, and evidence that gait analyses must consider individual differences. 3 • A methodology for rapid gathering and experimentation of gait analysis data and techniques. • Plans for future work: how we can use gait detection and feedback to help combat obesity and physical inactivity. 1.2 Thesis Outline We begin by reviewing the relevant literature in Chapter 2. Next, we cover our work in cadence estimation, including descriptions of the RRACE algorithm, our methodology, a validation study of RRACE, and an analysis of a status quo algo- rithm Chapter 3. The next 3 chapters focus on the problem of gait classification, starting with a description of our gait selection process and gait classification data collection methodology in full in Chapter 4. After, we detail our gait classifica- tion algorithm, including design decisions and performance results in Chapter 5. We then discuss our analysis of the gait classification algorithm in Chapter 6. We conclude with a summary of our findings and plans for future work in Chapter 7. 4 Chapter 2 Related Work Only thoughts reached by walking have value. — Friedrich Nietzsche This work is related to many different application areas and draws upon a di- verse area of previous work. In this chapter, we will review the related work. First, in Section 2.1, we discuss application areas related to robust gait analysis and how this work is positioned in consideration of these works. After, in Section 2.2, we review previous work in robust mobile gait analysis. 2.1 Application Space Gait analysis has a wide variety of applications. In this section, we review the space of major application areas for this work. Though not exhaustive, this review will give a thorough introduction to the prominent motivating application areas and the body of research contained there-within. First, in Section 2.1.1, we explore mobile exercise applications not imple- mented as games. Second, in Section 2.1.2, we present an overview of the bur- geoning area of exercise games, focusing on mobile exercise games and their input modalities. Next, in Section 2.1.3, we examine work on context-aware applications on a mobile platform. Finally, in Section 2.1.4, we explain how our methodology is designed to support these varied application areas simultaneously. 5 2.1.1 Exercise Applications Using on-person sensors to support exercise is not a novel concept. Pedometers have long been a fitness tool for dedicated walkers and runners, both for meeting fitness goals (such as a target daily step count) and for general awareness of activity. As mobile devices become more prevalent, many mobile applications have been developed to encourage or support physical fitness and exercise. Examples of these range from self-reported logging of exercise and nutrition to smartphone-based pe- dometers and GPS-based location sensing for running or cycling [19, 41, 47, 60]. More elaborate sensing solutions have also been developed, such as the Nike+ project [45]. Nike+ is a suite of solutions for exercise monitoring and sensing. It includes a mobile application that uses GPS to record runs, and specialized equip- ment to provide additional sensing if the user chooses: the Nike+ FuelBand, a wrist-based sensor and display that tracks step count and overall movement; a heart rate monitor, connected via Bluetooth; Nike shoes, orchestrated with addi- tional sensors; or even several brands of exercise machines. Previously, Nike also included the Nike+ Sensor, which was placed in the user’s shoe. Step detection techniques are often used to estimate walking cadence (i.e., step rate). This can be used as part of an exercise logging application, or to motivate users by matching music tempo to the users’ current or desired walking or run- ning cadence. Oliver’s & Flores-Mengasand’s MPTrain system uses song tempo to help the user achieve a desired heart rate using a chest-mounted accelerometer and heart-rate sensor [47]. MPTrain’s interface was later extended to include a glanceable display and competitive running support as the TripleBeat system [14]; these changes made TripleBeat more effective than MPTrain at encouraging users to maintain their training heart rate, although only half their participants enjoyed the competitive aspect. Elliott & Tomlinson provide a prototype “PersonalSound- track” that matches music to the users’ current walking pace (detected with a hip- mounted accelerometer that detects user steps) [16]. All of these works sought to encourage users to maintain or reach a certain cadence. Recent studies have shown that pedometers can influence physical activity lev- els. Lin et al. use Prochaska’s Transtheoretical Model of Behaviour Change to encourage participants to increase their daily step count [39]. Their results sug- 6 gest that tying step count to an ambient display (in this case, with fish avatars) may encourage people to walk more. Consolvo et al. show that more than simple step count is needed [9]. They developed a mobile fitness journal connected to a pedometer, and found that though pedometers were motivating, there was a need to support other activity beyond walking via cumulative step count. Because pe- dometers did not register other activities, such as cycling or climbing, participants were frustrated when the system failed to report their full set of activities (both to themselves and to groups when a social network sharing was used). As well, ex- ternal pedometers were large and unsightly - unsuitable for a pervasive application used in day-to-day life. Consolvo et al.’s follow-up work, the UbiFit system, improved matters by using a multimodal external sensor, the Mobile Sensing Platform (MSP), to detect varied activities [10]. UbiFit provides an ambient visual metaphor of a user’s exercise ac- tivity with a garden that grows with your activity. Each flower’s type is determined by the type of exercise (such as a walk, etc.), and butterflies appear as weekly goals are met. Drawing from [39], positive enforcement was used exclusively: there were no wilting flowers. The MSP inferred walking, running, cycling, using an elliptical trainer, and using a stair machine, but also allowed users to manually log activities. The MSP was relatively accurate: 77% of inferred activities were unmodified in the UbiFit log [8], but only 39% of recorded activities were inferred (the rest were manually entered into the system) [11]. A twelve week study reinforces that having ambient (in this case, glanceable) display of the garden improved the maintenance of a regular exercise schedule. Motivation to exercise is especially desirable in today’s youth. Poole et al. in- vestigated the role of ubiquitous computing in schools as an interventionist method for youth physical activity [48]. Physical activity levels encounter an “adolescent slump”, where physical activity levels drop dramatically around age 12. They insti- gated the “American Horsepower Challenge”, a pedometer based pervasive health game for middle school students aged 9-13, and found that rituals and structured time are important for such an intervention. They used a virtual race accessible on a website to encourage students to compete to collect a greater number of steps. The desire to bring exercise to youth has been one of the primary motivators for recent work into exergames. 7 2.1.2 Exergames Since the release of Dance Dance Revolution in 1988 [53], many exergames have been developed, and an increasingly large body of research was been created. Re- cent work tends to focus on the use of exercise games as persuasive applications to promote physical activity in a variety of ways, from using stationary bicycles [56, 61] to the dynamic construction of virtual worlds based on your activity level and location in the physical world [13]. There is unfortunately a great discrepancy in terminology between authors, in- cluding “active game”, “exercise game”, and “exergame.” In this work, we will use Oh and Yang’s definition of exergame, which is based upon a recent survey of exergame literature: “An exergame is a video game that promotes (either via using or requiring) players’ physical movements (exertion) that is generally more than sedentary and includes strength, balance, and flexibility activities” [69]. Please note that this definition includes location-based games that do not explicitly re- quire strength, balance, or flexibility. A subset of these games, mobile exergames, are defined in this thesis as exergames that are not tied to a traditional desktop com- puter or stationary device. In this section, we will discuss exergames, focusing on a mobile context. Sinclair, Hingston, & Masek provide a thorough motivation for exercise games [53]. There is a rising epidemic of obesity, often caused by poor eating habits and lack of exercise. Their review of the literature suggests that exercise is the “best, safest, and most effect way to prevent obesity” [53]. As well, electronic sources of entertainment, such as TV, Internet, and videogames, consume a large amount of children’s time. Encouraging children to make healthy lifestyle choices is important for combatting obesity, as it can establish early lifestyle habits. Mobile devices have many unique advantages for exercise games. By not being constrained to a single location, experiences can vary to help keep user interest. As well, the sensing options in modern smart phones are very powerful (as we will describe in Section 2.2), and lend themselves well to pervasive games played in day-to-day life. Finally, mobile exergames provides an exciting opportunity to add interactivity to outdoor activities. Location is often used in mobile exercise games, with sensing typically done by 8 GPS. Jensen et al. developed a location-based game called PH.A.N.T.O.M., which uses GPS to identify a users’ location; the user then interacts with the game to complete a military-themed mission [28]. This is similar to Geocaching, a popular activity of hiding items in locations around the globe, and having other people use a shared service to find items or hide their own [23]. Another location-based game is Stanley et al.’s PiNiZoRo, a GPS-based game designed for families that allows non-programmers to provide their own scenarios, or “quests”, to the game [57]. Sensing can also be done by wifi access points, as in Bell et al.’s “Feeding Yoshi” game [3]. By using location to provide input to games, users can be encouraged to walk (or even jog or run) more; simply increasing step count can provide overall health benefits [7]. Heart rate is another common input modality to exercise games. Wylie and Coulton implemented the classic game Space Invaders on a mobile phone, modified to include random bonus stages that required a certain heart-rate to pass [66]. These stages would provide a bonus to the player, such as a shield or bomb powerup in the game. The authors recommended a direct link between the level of exertion and the bonus received for this style of exercise game. Sinclair et al. second this notion of intensity with the concept of game flow: by balancing the intensity of the workout with the user’s level of fitness and including proper warm ups and cool downs, they suggest that games can be more engaging to the user [53]. Stach et al. also looked at using heart rate as input, but scaled it to balance games for players of different fitness levels [56]. Davis et al. combined both heart rate and location as input in ‘Ere Be Dragons [13]. ‘Ere Be Dragons constructs a digital world based on the user’s real-world location (detected by GPS). In this world, the landscape flourishes when the user’s heart rate is elevated, and is impoverished otherwise (similar to UbiFit’s garden representation [10]). Accelerometers are also a powerful method of input for exercise games via gestures, overall activity level, or (less commonly) gait analysis. Fujiki, Kaza- kos, Pavlidis, Levine, and others developed NEAT-O-Games (NEAT meaning Non- Exercise Activity Thermogenesis) [20, 21, 32]. Their goal was to increase “energy expenditure of all physical activities other than volitional sporting-like exercise,” such as moving around to do chores [20, 32]. They created simple games that used 9 hip-mounted external accelerometers to infer an overall physical activity level. They developed two main games: NEAT-o-Race and NEAT-o-Sudoku [20, 21]. Two studies suggest that people might adopt these games and exercise more with them, but further study is needed [32]. However, at the time of this writing there are no exercise games that use gait as an input modality; that is, no exercise games directly take into account how the user is moving. Previous work has focused on non-mobile (i.e., situated) contexts. Stach et al. classified existing exergames (or “active games”) into a set of abstract inputs: gesture, stance, point, power, continuous control, and tap [55]. Brehmer et al. then developed a toolkit to handle all these types of inputs [4]. Their General Active Input Model (or GAIM) toolkit allows programmers to specify interactions with these abstract inputs rather than deal with individual hardware. Although some aspects of gait can be fit into these abstract specifications, emphasis was on non-mobile input techniques. Hardware types include accelerometers (mostly for gestures), cameras, ergometers (e.g., stationary bicycles and other resistance devices), pressure sensors, and touch sensors. We believe that gait analysis tech- niques provide a unique means of input for future exercise games. 2.1.3 Mobile Context-Aware Applications Accelerometers have been used for sensing context other than gait, such as the wearer’s manner of transportation, the device’s carrying location, or remote health monitoring. Froelich et al. developed a similar application to UbiFit, designed to keep users aware of their traveling habits and aid those who want to environmentally-friendly modes of transport [18]. An ambient display showed either a tree or a family of polar bears, and as more green options were chosen the displays become more lush and environmental. Vehicle transportation was detected by location change via GSM cell towers, but the type of vehicle could not be inferred: participants self- reported their methods of transportations after transportation was completed. Au- tomatic sensing of the means of transportation could be valuable for this. They also used the MSP (mobile sensing platform) to distinguish sitting, standing, walking, running and cycling activities, but participants did not like wearing an additional 10 sensor. This suggests that using an everyday mobile device for sensing might be more accessible for end users. Biometric data, including gait information, can also be used as a security mea- sure for mobile devices. Briggs and Olivier suggest a “Biometric Daemon”, a robot pet or partner that has a one-to-one relationship to a user, which can be used for authentication or mediation in work or personal matters [6]. Tamviruzzaman et al. propose a method to use location and gait features to infer the current user of a mobile phone [59]. Remote health monitoring, sometimes subsumed under the term “eHealth”, provides a means to help risk detection or aid rehabilitation. Zheng et al. used accelerometers as part of a comprehensive remote health monitoring environment [71]. Their present prototype uses accelerometers to measure gross levels of activ- ity, such as walking around the house, standing, sitting, and lying. Worringham, Rojek, and Stewart used GPS-based speed, GPS-based location, single lead elec- trocardiograph (ECG), and heart rate monitor for recovering cardiac patients [65]. Remote monitoring is an increasingly important application area, and gait sensing can be a part of that. Komninos, Wallace, and Barrie detected step count to help infer user affect (e.g., an agitated user might walk more quickly) [35]. They endeavored to make this more robust by detecting a mobile device’s location-on-person, but were only able to distinguish between a trouser pocket and one of 4 jacket pockets. How- ever, they showed that if this knowledge was gained, step detection could be im- proved dramatically. Dornbush et al. use a number of mobile sensors, including accelerometers, to classify songs for users based on their context [15]. 2.1.4 Positioning this work in the application space Mobile sensing techniques are a strong method for providing novel interactions for exercise applications, exergames, and context-aware applications. We propose to extend this work in two fundamental ways. First, we plan to investigate a number of gaits suitable for these varied application areas. As a compelling domain, we will focus on exercise applications and games. However, by looking at a variety of gaits previously uninvestigated, we will gain valuable insight into the general 11 capabilities of mobile sensing platforms for mobile context-aware applications. Second, mobile devices are able to provide compelling pervasive applications. They can be used in the background of everyday life, making them more accessi- ble than non-mobile exercise equipment or computers. However, mobile devices can be carried in many different ways. Previous sensing approaches, discussed in Section 2.2, often require knowledge of the mobile device’s carrying location and orientation. We aim to study gait analysis in a way that makes it robust to carrying location and orientation, and thus able to operate in the background without requir- ing a particular carrying location. As well, given our primary domain of exercise applications and games, and to allow for interactive context-aware applications, we are focusing on real-time sensing1. We should note that we are focusing on cyclic motions, not ephemeral movements (such as the foot gestures in [52]). 2.2 Real-Time Mobile Gait Analysis Because of the wide variety of application areas for mobile gait analysis (described in Section 2.1), there is a large body of work conducted on gait analysis using ac- celerometers. In this section, we describe the related body of work used to analyze gait using methods that can work in real-time or robustly in the wild. In Sec- tion 2.2.1, we describe previous work related to detected footsteps and cadence estimation in real-time. In Section 2.2.2, we describe previous work related to classifying gait or physical activity in real-time. In Section 2.2.3, we describe re- lated work on other application areas, including individual identification of users through gait. Finally, in Section 2.2.4, we discuss the role that carrying location plays in these algorithms. 2.2.1 Step detection and Cadence Estimation Step detection and cadence estimation are powerful tools. However, it can be dif- ficult to develop a robust method: pedometer accuracy can vary by age, weight, BMI, and (most powerfully) walking speed [43], and by carrying location (as we 1With the problem of gait detection, real-time sensing is defined as follows: given a sensing task for a time t, we have only knowledge of the past and present (data for t ′ ≤ t), and a desire to calculate with a delay on the order of seconds. Exact acceptable delay time depends on application. 12 show in Chapter 3). Though robustness remains a challenge, Melanson et al. found that pedometers in a fixed location can be quite accurate [43]. In particular, piezo- electric pedometers perform better than spring-loaded pedometers, achieving 97% accuracy with walking speeds of 1.8-2.0 miles per hour (MPH). The most accu- rate published cadence estimation method on a mobile device uses a step-detection method with chest-mounted accelerometer in a known orientation [47]. Elliott & Tomlinson provide a prototype “PersonalSoundtrack” that matches music to the users’ current walking pace [16]. As well, Nishiguchi et al. showed that smart- phone accelerometers can produce gait parameters similar to external accelerome- ters [46]. They attached an external accelerometer to a smartphone, and compared the results of gait parameters calculated from the two signals. Gait parameters were found to be consistent between the two devices, lending credibility to smartphone accelerometers as sensors. It is worth mentioning that one of the gait characteris- tics, peak frequency, was found using a frequency-based approach. We also use a frequency-based approach in our cadence estimation algorithm, RRACE (see Sec- tion 3.1), although we use different filtering and numerical procedures to accom- plish this. Though smartphone accelerometers were found to be valid sensors, we encoun- tered major barriers in accessing them through the smartphone operating system; see Section 4.4.2 for a discussion. The main consequence of these barriers is a variable sampling rate, which we accommodate in our algorithms. To our knowl- edge, this has not been discussed in any detail in the literature (though Nishiguchi et al. briefly mention the use of interpolation to handle this in [46]). 2.2.2 Activity Detection and Gait Classification Gait analysis is a complex field, bridging kinesiology, engineering, artificial in- telligence, and human-computer interaction. Here, we limit our discussion to works relevant to robust or pervasive activity detection and gait classification via accelerometer signals, with a focus on real-time analysis for an interactive system. Statistical features are often used for activity detection. Figo et al. discuss preprocessing techniques for using mobile sensors for context-aware applications [17]. They classify preprocessing techniques into three domains for work: the time 13 domain, the frequency domain, and discrete representation domains. In both the time and frequency domains, algorithms can use basic descriptive statistics, such as mean, median, variance, standard deviation, minimum, maximum, correlation, and cross-correlation, or other mathematical functions, such as the signal-magnitude area (SMA) or the DC component of frequency. Discrete representation domain analysis involves symbolic string based algorithms such as the Levenshtein dis- tance and Dynamic Time Warping (DTW). Most analyses with a smartphone or single inertial mass-unit (IMU) have a known location or orientation. Huynh and Schiele looked at the best features and window size2 needed for activity detection of 6 activities: standing, walking, jog- ging, skipping, hopping, and riding a bus [25]. They gathered 200 minutes of data from two participants with a multisensor on a backpack strap, and determined the best feature and window size for each activity. Features were always a fre- quency spectrum band. Best window sizes by activity were: hopping 4s, skipping 2s, jogging 1s, riding bus 1s, walking 1s, standing 0.5s. Classification results var- ied depending on activity; hopping and walking performed the best (F-scores not reported, but graphically look like they resemble 90%), while the other activities performed more poorly. They suggest that unequal sample sizes for activities might influence the results. When contextual information about the phone (such as the phone’s orientation) is not known, it can sometimes be inferred. Kawahara, Kurasawa, and Morikawa (2007) used a manually created rule-based system to infer phone location and ori- entation, and then used this information to classify activity for a exercise coaching system [31]. Training on three subjects and testing on a fourth, they achieved be- tween 96.7% to 100% accuracy of four activities (sitting, standing, walking, and running), and 97.4% to 98.7% accuracy for three locations: pants, chest pocket, and bag (trained on a variety of bags brought by subjects). One notable technique in their approach is the use of events to help select locations - for example, if some- one leaned forward in a chair, it would help infer the chest pocket location. Their accelerometer sensor, attached to a mobile phone, read samples at 20 Hz. Window sizes varied depending on features. 10 minutes of data was collected from each 2A window is a subsection of a signal, which is used for calculating features. The window size is the length of such a subsection. 14 subject. A hierarchical classification considering carrying location improved classifi- cation [33]. Khan et al. found that, by dividing their classifier into recognizing resting activities at all locations, dynamic activities at upper body locations (breast pocket, inner jacket pocket), and dynamic activities at lower body locations (left front trouser pocket, right front trouser pocket, and rear trouser pocket) improved classification from 58.7% to 94.4%. The activities detected were resting (sitting/s- tanding/lying down), walking upstairs, walking downstairs, walking, running, cy- cling, and vacuuming. They used 8 subjects with a minimum of 2 minutes per location/gait/subject combination, extracting features from 1 second windows. Zhang et al. also used a hierarchical classification based on motion [70]. Their algorithm uses manually tuned thresholds of acceleration change to detect whether the subject is moving or not, then uses an support vector machine (SVM) classifier to classify the activity into one of three non-moving activities (sitting, standing, ly- ing down) or one of three moving activities (walking, posture transition, and gentle motion). This improved performance from a naı̈ve SVM classifier on all 6 activi- ties from a mean of 63.8% to a mean of 82.8%. The algorithm was trained on one subject, then tested on 10 subjects (including the training subject). Though they also assumed perfect detection of motion vs. no motion, this suggests that hierar- chical algorithms based on gait sets can improve performance. Their algorithm ran on a single mobile phone attached to the belt in a horizontal orientation. Iso & Yamazaki used the Kohonen self-organizing map (KSOM) on features extracted with wavelet packet decomposition, resulting in 80% accuracy for gait classification [27]. Robustness was achieved by generating pseudodata through rotational transformation of their original data. Evaluation was with only two sub- jects but with phones located in the breast pocket and the hip pocket. So far, the most robust system has been presented by Yang [68]. By estimat- ing the gravity vector, Yang was able to separate motion into vertical and hori- zontal components of the gravity plane. Analysis achieved 88-90% accuracy on sitting/cycling/driving/running/walking/standing with a 10s window. The system was trained on three subjects and tested on a fourth; there were approximately 10- 30 minutes per activity of data, with no knowledge about orientation or carrying location. 15 In summary, gait classification approaches on cell phones have achieved high accuracy (90%), which is suitable for our application areas. We must note that methodologies vary considerably between studies, with different data windows (often longer than ours), a large amount of training data (a larger amount than we collect, as we aim for rapid collection of data to facilitate experimentation with the analysis of novel gaits), whether or not classification is done in real-time (our work strives for real-time classification), and the number of subjects involved in a study (often very small numbers; we collect data from 12). See Chapter 4 for more information on our methodology, and Table 2.1 for a summary table of the work in this section. 16 Table 2.1: Overview of related work for real-time activity detection and gait classification with data from smartphones or loosely-carried accelerometers. Please note that this table is intended to highlight the most prominent features of these works, but there are additional differences in classification strategy, extracted features, signal processing, and methods for calculating accuracy. Please see the text for these details. Publication Collected Data Locations Considered Activities/Gaits Accuracy Khan et al. 2010 [33] 2+ minutes per data cell for each of 8 subjects 5: breast pocket, in- ner jacket pocket, and left, right, and rear trouser pockets 7: resting, walking upstairs/downstairs, walking, running, cy- cling, and vacuuming 94.4% Zhang et al. 2010 [70] 2 hours overall for 10 subjects 1: belt (horizontal ori- entation) 6: sitting, standing, lying down, walking, posture transition, and gentle motion 82.8% (trained on 1 subject, tested on all 10) Yang 2009 [68] 10-30 minutes per gait for each of 4 participants N/A: freely carried 6: sitting, cycling, driv- ing, running, walking, standing 88-90% (trained on 3, tested on 4th) Kawahara, Kura- sawa, & Morikawa 2007 [31] 10 minutes from each of 4 subjects 3: pants, chest pocket, and bag 4: sitting, standing, walking, and running 96.7% to 100% (trained on 3, tested on 4th) Iso & Yamazaki 2006 [27] 15 minutes per gait for each of 2 subjects 2: breast pocket and hip pocket 5: walking, going up- /down stairs, walking rapidly, and running 80% Huynh & Schiele 2005 [25] 200 minutes overall for 2 subjects 1: backpack strap 6: standing, walking, jogging, skipping, hop- ping, and riding a bus 90% 17 2.2.3 Gait-Based Individual Identification A highly related area to gait classification is gait-based individual identification. As mentioned in Section 2.1.3, discerning a device’s user by gait has been proposed as a means for security or personalization [6, 59]. We review robust methods for this style of sensing here. Although a different problem from cadence estimation or activity detection, we can draw from their efforts to be robust to different gaits, carrying locations, and orientations. Previous efforts have been quite successful at identifying a user based on the output of smartphone sensors. Sprager and Zazula used principal components anal- ysis (PCA) with a support vector machine (SVM) with cumulants (higher order statistics similar to moments) as features to achieve an average of 90.8% accuracy when identifying 6 individuals walking at 3 walking speeds: slow, normal, and fast [54]. However, the phone was in a known orientation and location for all partic- ipants: the cell phone was affixed at the hip in a landscape orientation. Gafurov and Bours recently examined 100 subjects using a “natural walking style” with a template matching system, and achieved an identification rate of 81.4% [22]. Again, location and orientation of the accelerometer was known. Although many locations, such as shoe, pocket, wrist, waist, and hip, have been examined by them- selves previous work, we are unaware of any who has investigated location [22]. Kobayashi, Hasidi, and Nobuyuki Otsu worked on developing a rotation-invariant feature set by using an autocorrelation matrix of the frequency spectrum [34]. This was used for individual gait identification with 58 subjects as they walked freely in daily life, and was able to work with approximately 49% mean accuracy. How- ever, the phone was held in a single location - the hand. This again suggests that robust sensing given different orientations and carrying locations is a challenging problem. 2.2.4 The Role of Location A major shortcoming of most gait analysis approaches is that they are rarely ro- bust to carrying location. In 2007, Cui, Chipchase, and Ichikawa conducted a cross-cultural study on mobile phone carrying locations and personalization [12]. Through a series of street interviews in 11 cities on 4 continents, they discovered 18 that people carried phones using a variety of methods. These were coded into the following categories: bags, trousers/skirts, belt case/clip, upper body, hands, neck, “not with me”, and for 1.97% of men and 6.11% of women, “other”. The re- searchers tabulated reasons for the various carrying locations; the majority of users (67.34%) chose their carrying location based on “instrumental” reasons, such as: ease of carrying or fetching their phone, detecting notifications, or security of the phone. Cui et al. also found that the detection rate of phone notifications varied by position, and catalogued reasons for personalization of phones, such as through custom cases, straps, or stickers. Some researchers have considered carrying location in their analysis of IMU sensors. Kunze et al. used the magnitude (Euclidean norm) of accelerometer sen- sors to detect different locations on the person [36]. Accelerometers were affixed to four locations (head, breast, trousers, and wrist) using a strap. Classification was conducted using a one second sliding window to detect whether the subject was walking, and if so, to detect location. Using non-real-time analysis, they were able to accomplish a precision of 100% for the detection of walking segments, however, no recall was reported. For real-time performance, they achieved their best performance by a C4.5 decision tree achieved accuracy of 69% - 95% (mean 82%). This was for a single gait (walking vs. not walking). Given that the user was walking, location detection was a mean of 89.81% accurate for 4 locations: wrist, head, left trouser pocket, and left breast pocket. Using non-real-time analysis and event recognition, they were able to achieve a precision (not accuracy) of 100%. Basically, they were reducing false positives as much as possible. As mentioned in Section 2.2.2, hierarchical classification considering carrying location improved classification [33]. Khan et al. found that, by dividing their clas- sifier into recognizing resting activities at all locations, dynamic acitivities at upper body locations (breast pocket, inner jacket pocket) and dynamic activities at lower body locations (left front trouser pocket, right front trouser pocket, and rear trouser pocket) improved classification from 58.7% to 94.4%. The activities detected were resting (sitting/standing/lying down), walking upstairs, walking downstairs, walk- ing, running, running, cycling, and vacuuming. They used 8 subjects with a mini- mum of 2 minutes per location/gait/subject combination. 1 second windows with no overlap were used. 19 Komninos, Wallace, and Barrie looked detecting a mobile device’s carrying location, using it to improve an algorithm for step detection [35]. Unfortunately, they were only successful in differentiating between two sets of locations: trouser pocket, and jacket pockets. They did show that, if phone location was known, then step detection could be improved dramatically. However, they only used 3 basic features: mean, standard deviation, and the ratio between the two (as “intensity”). Thus, robust treatment of carrying location remains an open problem in gait analy- sis. 20 Chapter 3 Cadence Detection My grandmother started walking five miles a day when she was sixty. She’s ninety-seven now, and we don’t know where the hell she is. — Ellen DeGeneres In this chapter we describe the development and validation of a novel real-time frequency-based cadence detection algorithm, RRACE, that is robust to carrying location and orientation. First, in Section 3.1, we describe this algorithm. Second, in Section 3.2, we describe the main data collection procedures as well as partici- pant demographics. Third, we describe the implementation and tuning of a gold- standard time-based cadence detection algorithm in Section 3.3. We demonstrate that this time-based reference algorithm is sensitive to location-on-person and in- dividual differences. We then show that the frequency-based algorithm is more accurate and robust to carrying location and individual differences by comparing it with best-case scenario tunings of the time-based cadence detection algorithm in 3.3.2. Finally, we discuss major results about RRACE’s performance in Sec- tion 3.4, and then conclude with the summary of our findings and future work in Section 3.5. As described in the Preface, this work is the result of a collaboration. As such, portions of the work are written to demonstrate the relative contributions of the author. Parts primarily done by other contributors (Section 3.1, Section 3.2.1, 3.3.2, Section 3.4) have been reduced but included for framing, while the author’s main contributions are described in full. The full manuscript is currently being prepared 21 for submission to a peer-reviewed journal [30]. 3.1 The RRACE Algorithm Our cadence-detection algorithm, Robust Real-time Algorithm for Cadence Esti- mation (RRACE), performs a spectral analysis on a four-second window of sam- pled 3-axis accelerometer data. Our approach ensures that the algorithm is robust to orientation, and accommodates non-uniform sampling rate present to smartphones. RRACE accomplishes this with the following implementation details. Supporting orientation-invariant information: To estimate overall movement from a triaxial accelerometer signal and to make it independent of orientation, we use the magnitude (Euclidean or `-2 norm) of the three accelerometer axes, x, y, and z, as our signal, as in [38]. This is a simple solution for creating robustness to orientation, which we later show to be effective. Accommodating Non-uniform Sampling (FASPER): Most smartphones supply ac- celerometer data that is not sampled at a constant rate, making the most common spectral analysis methods (such as the Fast Fourier Transform (FFT)) inappropri- ate. The Lomb-Scargle periodogram approach (also known as Least-squares spec- tral analysis) is appropriate for nonuniformly sampled data [40, 51]. In particular, the FASPER algorithm (“Fast Calculation of the Lomb Peri- odogram”) [49] employs four parameters: the vector time series along, the time coordinate of each sample, an output gain (we used 0.25), and an oversampling parameter (controls resolution of the computed spectrum; we used 4.0). It com- putes the relative power for each of a discrete set of frequencies. The maximum frequency computed is the average Nyquist frequency scaled by the supplied gain [49]. RRACE uses FASPER to find the spectrum of the overall movement of the device. We report window cadence as the most powerful frequency in the spec- trum. We define our algorithm’s latency as half the window length (e.g., a 4 second window has a latency of 2 seconds). Android-Based Validation Platform: For our validation study, we used six Google Nexus One smartphones running Android OS version 2.3.4 (Gingerbread). Our 22 main application was implemented in Java, the most commonly used programming language for Android development. Numerical programming algorithms (includ- ing FASPER) were implemented in C for speed benefits and because of readily- available C implementations [49]. We used the Java Native Interface (JNI) to con- nect the two languages. 3.2 Cadence Study Data Collection In this section, we describe the data collection for the Cadence study. Apparatus: The experimental apparatus consisted of six Google Nexus One smart- phones, an external GPS receiver connected to one of the phones via Bluetooth, our reference cadence measurement consisting of two FSR (force sensing resistor) sensors [26] connected to a Bluetooth-enabled Arduino board [2] to detect foot- falls, two laptops (one main computer (a laptop) for logging trials and a second (a small netbook) to log footfalls sent from the Arduino board via Bluetooth), a stop watch, and two flags for experimenters to send timing signals to each other. The study required three experimenters to run. Prior to the experiment, subjects were asked to wear pants with front and back pockets, but exact pocket locations were not controlled. The Arduino board and six phones were synchronized with the main computer at the start of the experiment. The GPS receiver, netbook and Arduino were put in the bag. The bag had a filled weight of approximately 2 kg. See Table 3.1 for general phone locations, which were chosen as the places people used most frequently for their mobile phones while commuting [12]. Trial Length and Speed Measurement: Our goal was to collect 20 seconds of walk- ing data for each trial and compute average step frequency. We asked subjects to walk a known distance, either 30m or 60m (marked by small flags along the walk- way), depending on whether 20 seconds had elapsed by the time the 30m point (first end time) had been reached (Figure 3.1 ). Timespan was manually recorded via stopwatch. 23 Figure 3.1: Experiment walkway, start and end points. Table 3.1: Experiment design Factor Number of Levels Factor Levels Window Size 4 1, 2, 4, or 8 seconds Body Location 6 back pocket, bag (backpack), dominant hand (held), front pocket, hip (mounted on belt), upper arm (mounted) Condition 5 typical (0), fastest (2), leisurely (-2), faster than typical (1), slower than typical (-1) Repetition 2 first time, second time 3.2.1 Experiment Design, Metrics and Subjects The design was within-subjects repeated-measures, with independent variables of window size, body location, and speed condition (Table 3.1). The five speed con- ditions and their repetitions (10 trials) were randomized. An indoor pilot study conducted on a treadmill indicated good performance for both 4- and 8-second windows; we added 1- and 2-second windows, to capture any possible advantage of lower latency and computational load. Metrics and Analysis: In this first study, we assessed RRACE’s accuracy by com- paring it to our FSR-based reference (Section 3.2). Our primary metric was the “Error Ratio” or ER, defined as the ratio of RRACE’s measurement “error” (the difference between the frequency measurement produced by RRACE, Fa, and the reference frequency, Fr) to the reference frequency Fr: ER = |Fa−Fr| Fr (3.1) 24 where Fr = 1 TFSR TFSR =The median of the last three intervals of each of the two feet to filter errors caused by false positives (extra footfall detected) or false negatives (footfall missed). We conducted analysis with Linear Mixed-Effects Models, using unpaired Z- test1 comparisons for post-hoc analysis. We used a level of significance of p = 0.05, applying a Bonferroni correction to counteract the multiple comparisons problem. Note that we report differences between effect levels as z-scores, and because z-scores are normalized by standard deviation, differences between means in our analysis are analogous to Cohen’s d statistics of effect size. Subjects: Eleven individuals (6 female and 5 male), aged 21-30 years (mean=25.2, SD=3.3), 155-179cm tall (mean=165.9, SD=7.0), and weighing 46-80kg (mean=59.1, SD=10.0) volunteered. No subjects had physical impairments. 3.2.2 Results for Outdoor Validation of 4 Second Window RRACE We chose the 4 second window RRACE as the baseline for our algorithm. Theo- retically, 4 seconds is enough to detect a wide range of walking cadences; this was verified by our pilot study. To improve clarity of the results we divide the analysis into two parts: (a) first, we analyze the four second window RRACE and (b) then, we analyze the use of alternative window sizes. Because the phones blacked out a few times during the study and we had missing data, for the analysis, we used Linear Mixed Effect Models which is robust to missing data. Main Effects: Body location has a significant effect on ER. Front pocket, belt, arm, and bag (light-green box plots of Figure 3.2) are significantly more reliable than back pocket and hand (dark-red box plots of Figure 3.2). Walking speed also has a significant effect on ER; the slowest speed performs more poorly. 1The Z-test is appropriate because of our large sample sizes; we calculate cadence every 200ms. 25 Interaction Effects: Interaction effects of both location / condition and of location / window size on ER are significant. Arm, bag, and front pocket are more consistent and remain below 5% under all speed conditions, with their minimum at the middle (typical speed). Belt produces the lowest Error Ratio at faster than typical speed, back pocket produces lower error ratios at higher speeds and hand produces lower error ratios at lower speeds (Figure 3.2). Slowest (−2) Typical (0) Fastest (2) 0. 00 0. 05 0. 10 0. 15 0. 20 Arm Speed (kph) Er ro r R at io Slowest (−2) Typical (0) Fastest (2) 0. 0 0. 1 0. 2 0. 3 0. 4 0. 5 Back Pocket Speed (kph) Er ro r R at io Slowest (−2) Typical (0) Fastest (2) 0. 00 0. 05 0. 10 0. 15 0. 20 Bag Speed (kph) Er ro r R at io Slowest (−2) Typical (0) Fastest (2) 0. 00 0. 05 0. 10 0. 15 0. 20 Belt Speed (kph) Er ro r R at io Slowest (−2) Typical (0) Fastest (2) 0. 00 0. 05 0. 10 0. 15 0. 20 Front Pocket Speed (kph) Er ro r R at io Slowest (−2) Typical (0) Fastest (2) 0. 0 0. 1 0. 2 0. 3 0. 4 0. 5 Hand Speed (kph) Er ro r R at io Figure 3.2: Error Ratio as a function of speed condition for 4-second window RRACE. Please note that dark-red boxplots have a larger range for Error Ratio and light-green boxplots have a smaller one. 26 Table 3.2: Error Ratio differences by body location for four-second window RRACE through an unpaired Z-test. The second column contains the mean Error Ratio of each body location. Difference from Body Location ER Front Pocket Bag Arm Belt Back Pocket Front Pocket 0.028 - - - - - Bag 0.031 not sig - - - - Arm 0.036 0.003 0.001 - - - Belt 0.055 0.022 0.020 0.014 - - Back Pocket 0.079 0.045 0.043 0.037 0.030 - Hand 0.114 0.078 0.076 0.071 0.052 0.026 Table 3.3: Error Ratio differences by window sizes of RRACE , with walking speed and body location lumped. Window sizes are ordered by increasing ER mean. Difference from Window Size ER 8 Seconds 4 Seconds 2 Seconds 8 Seconds 0.058 - - - 4 Seconds 0.058 not sig - - 2 Seconds 0.071 0.011 0.011 - 1 Second 0.115 0.054 0.054 0.041 Table 3.4: RRACE Error Ratio differences by body location for all window sizes and walking speeds (unpaired Z-test). Locations are ordered by increasing ER mean. Difference from Body Location ER Bag Arm Front Pocket Belt Back Pocket Bag 0.041 - - - - - Arm 0.048 0.005 - - - - Front Pocket 0.055 0.012 0.004 - - - Belt 0.071 0.028 0.020 0.013 - - Back Pocket 0.105 0.061 0.053 0.046 0.030 - Hand 0.128 0.083 0.076 0.069 0.053 0.019 27 3.3 Comparing RRACE with a Threshold-based Time-domain Algorithm As described in Section 2.2.1, there are many commercial pedometers on the mar- ket, but most use proprietary algorithms that have not been released to the public. We compared our frequency-based algorithm to MPTrain’s algorithm [47]. MPTrain uses two low-pass filters. One removes noise in the original accelerom- eter signal, producing a smoothed signal; the second has a lower cutoff frequency, and its output is used as a dynamic threshold. Footsteps are detected when the smoothed signal crosses the dynamic threshold from above to below (Figure 3.3). Because the MPTrain accelerometer is required to be situated on the user’s torso and oriented to detect accelerations in the superior-inferior axis (e.g., y-axis), it detects footsteps on both feet. Foot falls are translated to instantaneous steps-per- minute (SPM) using the following formula: SPMi = (int) 60.0∗SamplingRate #SamplesSinceLastStep (3.2) Finally, the MPTrain algorithm applies a median filter to the instantaneous SPM to calculate estimated SPM. The MPTrain study reported a constant sampling rate of 75Hz for accelerometer data, achieved by using an external chest-mounted device to capture the accelerometer signal. The authors report that the MPTrain algorithm is comparable to those found in commercial pedometers by [43], but provide no details on cadence accuracy. 3.3.1 Implementation of Time-based Algorithm for Comparison We implemented a modified MPTrain algorithm. Since no implementation details were reported for either low-pass filter, and no window was given for the median filter, these elements had to be reconstructed. As well, we made other modifica- tions to accommodate the variable sampling rate found in smartphones, and mea- sured cadence in steps-per-second instead of steps-per-minute to compare it with RRACE. Because we measured steps-per-second, and to avoid rounding error, our value is a real number, and is not converted to an integer as in MPTrain’s algorithm. 28 0 1 2 3 4 4 6 8 10 12 14 16 18 Time (seconds) A cc el er om et er  V al ue  (x , y , z , o r m ag ni tu de ) Time-Based Algorithm for Subject 10 - Arm Raw Signal Smoothed Signal Threshold Detected Steps Figure 3.3: Example of the MPTrain time-based step detection algorithm Finally, given that we do not have a sensor in a known orientation, we also consider each of four different axes in our analysis: x, y, z, and mag (the magnitude of the vector). Our implementation follows. For efficiency and simplicity, we implemented low-pass filters as an exponentially- weighted moving average (EWMA). An EWMA is defined as follows: Si = αxi +(1−α)Si−1 (3.3) where Si is the i-th smoothed (low-passed) value, xi is the i-th raw accelerometer value, and α is the smoothing parameter (with 0≤α < 1). The two low-pass filters (for accelerometer data smoothing and dynamic threshold) employ two parameters, α and β (with α < β ). These parameters were tuned on our data, as described in Section 3.3.2. As in the MPTrain algorithm, steps are detected when the smoothed signal crosses the dynamic threshold from above to below. The difference between step 29 times is used to calculate instantaneous cadence by the formula: 1/CurrentDifferenceBetweenSteps For example, if the two previous footsteps were detected at StepTimei = 100ms and StepTimei+1 = 600ms and we wanted the instantaneous cadence at any time t ≥ 600ms, the instantaneous cadence would be 1/(600−100) = 0.002 steps per millisecond, or 2.0 SPS. Final cadence estimates were calculated by averaging each instantaneous cadence estimate with one previous estimate (i.e., a 2-sample smoothing filter). 3.3.2 Training of the Time-based Algorithm To compare the MPTrain time-based algorithm as favourably as possible to RRACE, we optimized the low-pass filter smoothing parameters α and β (Section 3.3.1) for several datasets involving different combinations of subjects and locations-on- person (LOP): • All data (all subjects and LOPs): 1 set • Each subject (over all LOPs): 11 sets • Each LOP (over all subjects): 6 sets • Each subject-LOP combination (e.g. Subject 1, Arm) minus 9 (missing data): 11×6−9 = 57 sets We thus considered 75 parameterizations of the time-based algorithm in a thorough search to compare the time-based algorithm as favourably as possible to RRACE. During analysis, data was only scored on the dataset on which it was tuned. This was done to give a best-case scenario of tuning the algorithm for a certain indi- vidual and/or LOP, which could be the situation in real-world use cases with one individual using a personal device in a consistent way. Within a dataset, we used a uniform search for the best combination of smooth- ing parameters (α and β ) with a granularity of 0.05 (i.e., α ∈ {0.05,0.1, ...,1.0} and β ∈{0,0.05, ...,1.0}), one of the three axes or magnitude (γ ∈{x,y,z,m}where 30 m = √ x2 + y2 + z2), as well as four scaling factors (δx,y,z ∈ { 12 ,1,2,4} for individ- ual axes and δm ∈ { 14 , 12 ,1,2} for magnitude, which scaled the cadence estimated values to compensate for possible harmonics). The best of all these 20 αs × 20 β s × 4 axes × 4 δ s = 6400 combinations for each dataset was determined by having the lowest mean squared ER by comparing to the FSR golden standard. Analysis of Time-based Algorithm We found it was not possible to train the time-based algorithm to work on all body locations and for all subjects with an Error Ratio below 5%; the best we could reach was ER = 74%. The best time-based algorithm for all LOP on one subject achieved ER = 18%, but this was a best case scenario and may not be achievable for all users. We found that tuning the time-based algorithm for one best-case body location on all subjects was more feasible: it achieved ER = 12% for bag. If we tune the algorithm for one body location of each subject we may even get a lower error ratio; when tuned for Subject10’s arm , the algorithm reached ER = 7.8% (Figure 3.4). Comparisons with Frequency-based Algorithm Figure 3.4 shows boxplots of Error Ratio of all the RRACE and time-based algo- rithm variants ordered by the median of Error Ratio (ER). We divided them into five categories: (a: green) 4 window sizes of RRACE and time-based algorithm trained on: (b: cyan) all subjects’ body locations, (c: pink) all body locations of each single subject, (d: yellow) one body location of all subjects, and (e: grey) single body location of one subject. Because it is unproductive to compare each of these algorithms with the rest, we have chosen the best of each category in addition to the worst-case RRACE variant (one-second window) which are marked red on Figure 3.4. This is a highly conservative comparison which tends to favor the time-based algorithm. We used the same data for verification of each time-based algorithm that was used for their training and secondly, the Error Ratio of all versions of the frequency-based algo- rithm is measured across all body locations of all subjects. RRACE was not trained or tuned in this comparison. 31 * All Subjects' Body Locations Subject3 Subject5 Hand Subject4 Subject1 Subject8 Subject2 Front Pocket Subject1's Front Pocket Back Pocket Subject2's Front Pocket Subject11 Subject4's Hand Subject2's Hand Subject7 Subject8's Front Pocket Subject3's Front Pocket Subject3's Back Pocket Subject7's Hand Subject7's Arm Subject11's Belt Subject3's Hand Subject7's Belt Subject1's Belt Subject10's Belt Subject3's Belt Subject10's Front Pocket Subject9's Back Pocket Subject11's Front Pocket Subject6's Front Pocket Subject1's Hand Subject4's Front Pocket Subject6's Belt Subject4's Belt Subject11's Arm Belt Subject11's Hand Subject6 Subject9 Subject5's Belt Subject5's Back Pocket Subject9's Belt Subject2's Belt Subject6's Back Pocket Subject9's Hand Subject1's Bag Subject6's Arm Subject5's Front Pocket Subject9's Arm Subject11's Back Pocket * Subject 10 Arm Subject3's Bag Subject1's Back Pocket Subject9's Bag Subject6's Bag Subject7's Bag Subject2's Back Pocket Subject4's Arm * Bag Subject8's Belt Subject8's Hand Subject8's Back Pocket Subject5's Hand Subject10's Back Pocket Subject1's Arm Subject2's Arm Subject11's Bag Subject5's Bag Subject10's Hand Subject2's Bag Subject10's Bag Subject8's Bag * Subject10's Arm * Freq−Based 1 Freq−Based 2 Freq−Based 4 * Freq−Based 8 0. 0 0. 2 0. 4 0. 6 0. 8 All Algorithms Error Ratio Figure 3.4: Error Ratio, ordered by median, of (a: green) 4 window sizes of RRACE (first four) and time-based algorithm trained on: (b: cyan) all subjects’ body locations (last one), (c: pink) all body locations of each single subject, (d: yellow) one body location of all subjects, and (e: grey) single body location of one subject. 32 Table 3.5: Unpaired Z-test comparison of error ratios of the best and the worst versions of the frequency-based algorithm and the best of each category of time-based algorithm. Algorithm variants are ordered by increasing ER mean. Difference with Algorithm ER 8-Sec RRACE Subject10’s Arm 1-Sec RRACE Bag Subject10 8-Sec Window 0.058 - - - - - RRACE (a) Subject10’s 0.078 0.012 - - - - Arm (e) 1-Sec Window 0.115 0.054 0.029 - - - RRACE (a) Bag (d) 0.119 0.055 0.032 not sig - - Subject10 (c) 0.179 0.102 0.081 0.046 0.040 - All Subjects’ 0.735 0.675 0.650 0.618 0.609 0.537 Body Locations (b) Thus, the single “fair” comparison is between either version of RRACE (colour green in Figure 3.4), and the time-based algorithm trained on all subjects and all body locations (colour cyan in Figure 3.4). Table 3.5 summarizes these compar- isons. Unlike Figure 3.4 where entries are ordered by median, the algorithm vari- ants here are ordered based on mean, therefore subject10’s arm which came after the 1-second RRACE comes before it in this table. 3.4 Discussion The goal of this research was to develop a cadence measurement algorithm for mobile phones equipped with accelerometers that is robust and works out-of-the- box with an Error Ratio of 5% or less (comparable to Yang et al.’s waist-mounted cadence measurement device [67] and MPTrain of Oliver & Flores-Mangas [47]). First, we will review the nature of RRACE’s error, its performance on different body locations and robustness to subject differences, and compare it with the time- based algorithm. Then we will examine its main weakness, and finally we will discuss the best choice for window size. 33 3.4.1 The nature of RRACE’s Error Our data shows that very few outliers with extremely large or small values are responsible for some of the error in RRACE’s readings. We believe there are two types of outliers: (a) random readings as a result of irregularities in the signal, and (b) harmonic readings which happen when the main frequency component gets smaller than its harmonics. These outliers may be avoided by filtering the outcome of RRACE. The rest of the error is caused by hardware measurement error and delay from the 4 second window. 3.4.2 RRACE Performs Best with the Arm, Bag, Belt, and Front Pocket Locations Based on our experiment, four body locations contain enough information for RRACE to make accurate cadence estimations: arm, bag, belt, and front pocket, with Error Ratios ranging from 3%-5%. By this criteria, RRACE works (i.e., can access sufficiently consistent motion to meet this ER) at four out of six body loca- tions, with negligible difference among these. Front pocket was the body location most robust to speed change. Note that, even though RRACE performs poorly with back pocket and hand, it still performs better than the time-based algorithm did at these locations. In contrast, the time-based algorithm was highly sensitive to body location. It was almost impossible to tune the time-based algorithm for three of the body locations, front pocket among them. That is, for three body locations (front pocket, back pocket, and hand), the time-based algorithm could not be tuned to perform well; for the other three (arm, bag, and belt), the time-based algorithm can perform well (with a lowest ER of 11.5%), but requires knowledge of and tuning for the particular location. The body location with the lowest ER is bag, but its ER of 11.5% is still almost double the Error Ratio of the 8-second and 4-second window RRACE. 3.4.3 RRACE is Robust to Subject Differences As discussed in 3.3.2, the time-based algorithm was very sensitive to subject dif- ferences; it could not be trained to work on all body locations of all subjects, and 34 when trained on single body locations ER = 12% was the best result which could only be achieved for bag. RRACE worked for all of our subjects with no prior tuning to compensate for subject differences with much lower ER. 3.4.4 RRACE Window Length of 4 seconds is Best: Our results showed that highest accuracy (lower error ratio) is achieved at larger window sizes. The difference in Error Ratio is substantial for 1 vs 2-second win- dows, and for 2 vs 4-second windows, but not for 4 vs 8-second windows. A 4- second window size seems the ideal length among our candidates as a compromise between responsiveness and accuracy. 3.4.5 RRACE is Sensitive to Very Slow Speeds Our outdoor validation results showed that, like other pedometers, RRACE is sensitive to speed. The highest Error Ratio belongs to the slowest speed with ER = 6.3%. We attribute this worsened performance to two possible causes: (a) At lower speeds, walking cycles take longer and fewer cycles are captured in a fixed window size. This weakens RRACE. Mitigation requires use of a larger window size, e.g., by dynamically changing the window size to fit the speed. (b) The kind of walking performed by our subjects becomes less autonomous and more irregular when they are asked to walk at very low speeds, especially because users can easily choose to walk as slowly and irregular as they want, while at high speeds step interval is bounded by subject’s physique. The time-based algorithm is less affected by walking speed because it just de- tects single steps, no matter how irregular or distant from each other they are. Thus one practical approach might be to shift to a time-based algorithm when low speeds are detected. 3.5 Conclusion and Future Work In this paper we introduced a new algorithm for measuring cadence from ac- celerometer data from smart phones, called RRACE. We also presented an exper- iment design for verification of our algorithm. Our validation study showed that RRACE performs well under different speed conditions, providing 5% or lower 35 error for four of the six common body locations examined: front pocket, bag, arm and belt, consistent with previous work in a single location [67]. Our algorithm was also robust to orientation and subject differences, and thus can be said to work ‘out of the box’, without requiring calibration or parameter tuning. RRACE’s primary weakness is a drop in performance for slow and irregular walking. We compared RRACE with a state-of-the-art published time-based algorithm which we tuned in every way possible. Our highly conservative comparisons show that RRACE is substantially more accurate than the time-based algorithm tuned for any subset of the data. Our data suggests that RRACE is also superior to the time-based algorithm in terms of independence from body location and robustness to user differences. The exception is for very low and/or irregular speeds; these are situations which may not be considered walking in most applications of a cadence detection method. As well, our algorithm provides general guidelines for window size and ro- bust spectral analysis. This information can be used to inform solutions to more complex real-time gait analysis problems, such as activity detection for fitness or rehabilitation applications, or individual gait identification for mobile security. A discussion of its use for these purposes is available in Chapter 5. Our algorithm can be improved in several respects: (a) Reduce estimation out- liers by using smart filters which will increase the accuracy of our algorithm even more; (b) Insert an algorithmic step that adjusts window size based on current ca- dence, to optimize the accuracy/responsiveness tradeoff; (c) Employ cadence to measure other useful information about gait such as stride length and type of gait. As well, one limitation of our analysis is the comparison with a threshold-based time-domain algorithm, which does not consider autocorrelation-based algorithms; future work is planned to compare RRACE with this approach. 36 Chapter 4 Gait Classification Data Collection Well sir, I have a silly walk and I’d like to obtain a Government grant to help me develop it. — Eric Idle, Monty Python’s Flying Circus Cadence estimation provides a promising means for exercise and fitness appli- cations, but it is only one aspect of gait. Previous work with pedometers shows that a single gait parameter is not enough for applications encouraging physical activity [9]. Inferring higher-level activities and gaits lends itself to a host of new applica- tions, including fitness journals and novel interaction modalities for exergames. In this chapter, we describe the gait data collection methodology and summary results, to be used for a gait classification algorithm described in Chapter 5. First, in Section 4.1, we report our methodology for selecting a set of gaits suitable for this study and the established application space, described in Table 4.1. We then describe the Android-based data collection application, GaitLogger, in Section 4.2, and our study procedure in Section 4.3. Finally, we summarize the collected data, including demographics and data quality, in Section 4.4. 37 4.1 Gait Selection In order to inform gait as input for mobile exercise applications and exercise games, we wanted to explore a wide variety of gaits. Previous work, surveyed in Sec- tion 2.2.2, has investigated 2 to 6 distinguishable gaits, often relevant for context- aware applications. We had four primary goals: (a) to push boundaries for context- aware applications, investigating gaits that were similar (such as distinguishing jogging from running); (b) to explore novel gaits suitable for exercise applications, by involving different muscle groups; (c) to explore novel gaits that could be easily linked by metaphor to a child-friendly exercise game, such as marching like a sol- dier or walking on a tight rope; and (d) to connect modes of travel involving vary- ing levels of physical activity, such as cycling or rollerblading. These four goals informed the generation of our gaits; as such, our target gaits were divided into four gaitsets, sets of gaits grouped by their relationship to our four goals. Because of this emphasis on unorthodox gaits, we wanted data to be collectible in a quick and practical manner, both in our data collection and in target application spaces. We hope to support iterative development and experimentation in games or other interactive applications; this has the added benefit of facilitating personalization of our algorithms. With this in mind, we developed a set of gaits suitable for an exercise or context-aware game or application. After piloting, we retained 15 gaits orga- nized into three gaitsets (Pedestrian, Exercise, and Game) corresponding to our first three goals; for scope and logistical reasons, our fourth goal of travel methods (the would-be Locomotion gaitset) was considered but left to future work. Explor- ing 15 gaits allowed us to collect 30 seconds of each gait easily within 20 to 30 minutes, without exertion being a problem for participants (some gaits, such as twofoothop, can be physically demanding when maintained for 30 seconds). To limit scope, all gaits had to involve directional movement and be bipedal - that is, we did not allow in-place activities like jumping jacks or walking on hands and knees. Our gaits are described in Table 4.1. The final list of 15 gaits were refined during piloting from a longer list of brain- stormed gaits. We discovered during initial brainstorming that some gaits were consistently interpreted by participants (e.g., walking on heels, walking like a pen- 38 Category Label Verbal description Pedestrian walk Walk normally fastwalk Walk quickly jog Jog, if asked for clarification: slow run run Run, not necessarily a flat out sprint, but a run ascendstairs Walk up the stairs as you normally would until you reach the top descendstairs Walk down the stairs as you normally would until you reach the bottom Exercise toes Walk on your toes heels Walk on your heels liftknees Walk while lifting your knees high in the air kickingbum Walk while kicking your bum. You don’t actually have to hit it, but do that motion Game twofoothop Hop with two feet penguin Walk like a penguin narrow Walk as if on something narrow, like a tightrope walkbackwards Walk backwards walksideways Walk sideways Table 4.1: Gaits and descriptions used in our data collection study guin) but others were not (e.g., skipping, walking like a zombie). To facilitate our user study and allow for a robust application, we decided to only use gaits that were consistently interpreted by a simple verbal or written description. That is, a user could read an instruction from a smartphone application and understand the movement. We are thus directly measuring participants’ interpretation of several gaits, important for a game or application to be distributed to a wide audience. This would also reduce any bias from the experimenter demonstrating a gait, and avoided logistical and possibly biased approaches of giving diagrams or videos of each gait. To accomplish this, many gaits were combined, dropped, or had their description refined through iterative piloting. A longer list of explored gaits, in- cluding our gaitset descriptions used during brainstorming and piloting, follows. This list includes the would-be Locomotion gaitset that was dropped after brain- 39 storming. • “Pedestrian” - Everyday walking suitable for context-aware applications. These would be suitable for supporting interior GPS or casual exercise appli- cations. In this gait subset we stressed a gradient of similar gaits to expand upon previous work on context-aware applications. “Pedestrian” gaits in- clude: – Walking – Walking quickly – Jogging/slow run – Running – Ascending stairs – Descending stairs • “Exercise” - Gaits that exercise different leg muscle groups, inspired by ply- ometrics, warmups, and stretches. Because numerous exercise movements exist, we chose ones that would exercise different muscle groups, and heavily used our scoping criteria of bipedal, directional movement.“Exercise” gaits include: – Walking on toes – Walking on heels – Walking while lifting knees high in the air – Walking while kicking bum – Squat walk – Lunges – Walking sideways (either grapevine or shuffle) • “Game” - Gaits that are exciting for kids and suitable for exercise games. These might resemble iconic characters such as zombies, gaits already asso- ciated with children’s games such as skipping, or abstract motions for which 40 metaphors are easily developed such as hopping (can easily be tied to being an animal, such as a frog or kangaroo). “Game” gaits include: – Hopping on two feet – Walking backwards – Walking like a zombie – Walking or marching like a soldier – Skipping – Tip toeing or sneaking – Walking on a tightrope – Walking on something narrow As well, we developed one category of gaits that are not necessarily bipedal, but important to our application area: • “Locomotion” - Not necessarily bipedal, but in the same spirit, locomo- tion gaits are methods of movement that involve vehicles or other devices. These gaits are also closely related to context-aware applications and previ- ous work. “Locomotion” gaits include: – Cycling – Riding a skateboard – Rollerblading – Riding on a bus – Driving or riding in a car During the study, we were careful to stick to the developed descriptions. When the participant asked if they were doing the gait correctly, we responded with a “Yes” regardless of what they were actually doing. Inquiries about misunderstood words (such as asking whether the heel was the front or the back of the foot) were answered concisely. On one occasion, a participant understood “jog” to be the same as “walking quickly”. This was the only situation in which we asked them to repeat the gait; in this case, asking them to do a “slow run”. The same researcher ran all participants. 41 4.2 GaitLogger Figure 4.1: GaitLogger main screen To collect data, we developed an Android application, GaitLogger (Figure 4.1). GaitLogger was developed in Java 1.6 using the Software Development Kit (SDK) for Android 4.0. GaitLogger Modes GaitLogger has two modes: Study Runner mode (Figure 4.2), which is used to run the experiment, and Log mode (Figure 4.3), used to record gait. GaitLogger’s Study Runner mode, used by the experimenter, writes participant information to a Comma Separated Value (CSV) file. This includes the participant number, time stamps used in the synchronization process (described below), and time stamps for the beginning and end of each trial, where one gait is recorded. Study Runner mode also allows randomization of gait order to facilitate data log- ging. 42 Figure 4.2: GaitLogger study runner mode GaitLogger’s Log mode allows the researcher to enter phone information re- quired for the study, connects to the accelerometer or gyroscope, and logs the raw sensor data to a file. The phone information is saved in a CSV file, and includes the participant ID, the phone’s location-on-person (LOP), and the phone ID. Log mode also allows selection of which sensor to access: accelerometer, gyroscope, or both. Due to issues described in Section 4.4.2, we only sampled from the accelerometer in our study. GaitLogger Implementation Details When logging data, GaitLogger creates a Service object [1], which informs the Android operating system that it is to operate in the background. This Ser- vice, GaitLoggerService, has native code section written in the C programming language. This native code section connects to the accelerometer and/or gyroscope to sample data at the fastest possible sampling rate, which is determined by the 43 Figure 4.3: GaitLogger log mode (a) Logging (b) Not Logging operating system; our empirically measured sampling rates are reported in Sec- tion 4.4.2. Data is buffered to reduce bottlenecking from input/output (I/O) opera- tions when saving to the file. GaitLogger connects to this native code section using the Java Native Interface (JNI). Sensor data is also saved to a CSV file, which can be collected after the study. GaitLogger Use In Study Seven Samsung Galaxy Nexus smartphones were used in the experiment, each loaded with GaitLogger. Six were placed on the participant using the same loca- tions as in our cadence estimate study (see Chapter 3): front pocket, back pocket, hip, hand, arm, and backpack. These six phones were in Log mode. The final phone was held by the experimenter, and was in Study Runner mode. The experimenter entered participant ID on all seven phones. One by one, each of the six logging 44 phones was synchronized with the experimenter’s phone by pressing a button on the the logging phone and the experimenter’s phone simultaneously. This method of synchronization is only as exact as the experimenter’s motor control, but this is close enough for our purposes: each trial lasts 30 seconds, and synchronization has an error rate of a fraction of a second. During the study, the experimenter logged beginning and end times for each gait on the Study Runner phone. 4.3 Study Design Our study followed the overall structure described below: 1. Participant Recruitment, described in Section 4.3.1. 2. Setup and Participant Briefing, described in Section 4.3.2. 3. Data Collection, described in Section 4.3.3. 4. Study Conclusion, described in Section 4.3.4. In the following subsections, we elaborate on the study design. 4.3.1 Participant Recruitment Participants were recruited through email lists to University student pools, com- monly used for this purpose. Emails linked to an online Doodle (cite) poll, which offered a number of sign-up times. Please see Section A.2.2 for email and adver- tising details. 4.3.2 Setup and Participant Briefing Before participants arrived, each phone was charged and had GaitLogger installed and running (Section 4.2). One phone was put into Study Runner Mode (Fig- ure 4.2), into which the anonymous participant identification number (hereafter referred to as “participant ID”) was entered. The other six phones were then put into Log mode. The experimenter then synchronized all the phones’ clocks using GaitLogger, entered location-on-person and participant ID, and started logging the accelerometer data. Phones were placed in any necessary cases; for example, the 45 phone going on the participant’s arm was slipped into a commercially available exercise arm band, and the phone placed in the backpack location was placed loose in a backpack. Once participants arrived, they were given consent forms (Section A.2.1) to read and sign, then given a copy for their records. After this, participants filled out demographic information (Section A.2.3). Participants were then read a briefing from a script and given time to ask questions. Phones were then given to the par- ticipant to be placed in the appropriate locations. The “arm” phone was placed on the participant’s self-reported non-dominant arm. The “hand” phone was held in the dominant hand, although this was not strictly enforced during the study when it would interfere with their normal walking gait (e.g., the participant might use their dominant hand to hold a railing when walking up or down stairs). Participants were instructed to place pocket or belt phones where they would normally be placed, and to adjust the backpack so that it was comfortable. See Figure 4.4 for a picture of the participant apparatus. 4.3.3 Data Collection Once the phones were in place, we collected accelerometer data (and for four par- ticipants, gyroscope data; see Section 4.4.2) using GaitLogger to record the sensor signals. Aside from Ascend Stairs and Descend Stairs, participants performed each gait described in Table 4.1 for exactly 30 seconds each in an outside, paved, rela- tively level and straight location. Ascend Stairs and Descend Stairs were performed on a square spiraling indoor staircase, with participants instructed to either walk all three floors to the top of the stairs or all three floors to the bottom. Because As- cend Stairs and Descend Stairs were limited by the height of the staircase, these gaits were not necessarily performed for 30 seconds each. However, typical time to travel up or down the stairs exceeded 30 seconds. Outside gaits were performed in a random order. For logistical reasons, Ascend Stairs and Descend Stairs were performed either at the beginning or end of the trial; this was counterbalanced by participants, as was the order of Ascend Stairs and Descend Stairs. 46 (a) Front (b) Back (c) Arm (d) Belt Figure 4.4: Participant apparatus 47 4.3.4 Study Conclusion After the study, phones were retrieved from participants, and the purpose behind the gaits was explained. Participants received $10 in compensation for participat- ing in the study. 4.4 Data Summary In this section we summarize the collected data for our study. Specifically, we de- scribe our participants (Section 4.4.1) and the data collected (Section 4.4.2). Data logs were used in gait classification algorithm development, described in Chapter 5. 4.4.1 Participants 12 people (6 female) participated in the study. All were right-handed. Self-reported age, weight, and height were gathered during the study using a basic questionnaire (Section A.2.3). Summary statistics can be found in Table 4.2. 4.4.2 Data Quality Overall, collected accelerometer data were very consistent. Mean sample period was 16.95 ms, corresponding to 59.00 Hz, with a mean of 1901.76 data points per participant/location/gait trial. Unfortunately, due to occasional gaps in data col- lection, the result of recording data on a smartphone with the operating system in control of sampling events, 6 trials had total recorded time less than 30 sec- onds. This may have been caused by background processes interfering with the data recording; however, this was isolated to only a few trials. A summary of the overall accelerometer data can be found in Table 4.3. A summary of accelerometer Age (years) Height (cm) Weight (kg) Minimum 21 155 46 Median 26 171.5 69.5 Mean 25.3 171.5 68 Standard Deviation 2.96 9.71 14.14 Maximum 31 183 86 Table 4.2: Self-reported statistics from 12 participants 48 Observations present Recorded Time per data cell Span (seconds) Minimum 74 28.04 Median 1 662 31.26 Mean 1 901.76 32.45 Standard Deviation 731.78 3.87 Maximum 6 932 56.22 Table 4.3: Summary of recorded accelerometer data Observations Recorded Participant ID Location Gait present per Time Span data cell (seconds) Participant 8 backpocket jog 89 28.22451 Participant 8 backpocket kickingbum 74 29.26617 Participant 8 backpocket toes 104 29.04672 Participant 8 backpocket walkbackwards 111 28.95335 Participant 8 backpocket walksideways 86 28.04027 Participant 10 frontpocket descendstairs 685 29.53482 Table 4.4: Summary of low-quality accelerometer data trials with less than 30 seconds of recorded time can be found in Table 4.4. Sample periods for both accelerometer and gyroscope can be found in Table 4.5. Gyroscope data were also collected for participants 6, 9, 10, and 12 by repeat- ing the experiment after the accelerometer-recording experiment, in order to de- termine relative data quality and informativeness for our purposes. We found that the gyroscope data were not as consistent as accelerometer data. Gyroscope data had a mean sample period of 80.05 ms, corresponding to 12Hz, and had a mean of 378 data points per trial. In fact, 3 trials had absolutely no data. Because of this poor performance, possibly related to the onboard sensor chip’s caching scheme, we decided to only analyze accelerometer data in Chapter 5. A summary of the recorded gyroscope data can be found in Table 4.6. A summary of accelerometer trials with less than 30 seconds of recorded time can be found in Table 4.7. 49 Accelerometer (ms) Gyroscope (ms) n 6 756 537 405 945 Minimum 0 0 Median 8.3 35.95 Mean 16.95 80.05 Standard Deviation 28.75 101.6 Maximum 7 389.38 16 284.8 Table 4.5: Summary of GaitLogger sample period over all recorded data (in- cluding between trials) Observations Observations Recorded Recorded present per present per Time Time Span, data cell non-empty Span non-empty data cell (seconds) (seconds) Minimum 0 8 0.00 0.35 Median 378 379 31.18 31.18 Mean 399.87 403.23 31.84 32.1 Standard Deviation 133.35 128.74 5.64 4.84 Maximum 1169 1169 54.59 54.59 Table 4.6: Summary of recorded gyroscope data Observations Recorded Participant ID Location Gait present per Time Span data cell (seconds) Participant 12 backpocket walk 396 16.40 Participant 12 frontpocket walk 8 0.349761962 Participant 12 hand walk 0 0 Participant 6 hand descendstairs 0 0 Participant 6 backpack descendstairs 0 0 Table 4.7: Summary of low-quality gyroscope data 50 Chapter 5 Gait Classification Algorithm Development He followed in his father’s footsteps, but his gait was somewhat erratic. — Nicolas Bentley In Chapter 4, we described the collection of raw accelerometer logs from 12 participants performing 15 gaits in 6 locations. In this chapter, we describe our analyses for developing a gait classification algorithm using the collected ac- celerometer logs. We begin by describing our feature extraction method, including window size, handling of orientation through different vectorizations of accelerom- eter data (referred to in this chapter as “axis sets”), and statistical features in Sec- tion 5.1. We then describe a preliminary analysis to compare window size, axis, and machine learning classification techniques in Section 5.2, by examining their effect on a single location and gaitset. Finally, we present two analyses with the best performing algorithm to examine the influence of location and gaitset in Sec- tion 5.3. 5.1 Feature Extraction In this section, we describe our methods for extracting features for classification. We investigate two window sizes and three methods of generating sets of axes. 51 Statistical features are taken from an axis set on a single window, and thus each generated window corresponds to a single observation for machine learning algo- rithms. 5.1.1 Window Sizes Because we only collected 30 seconds of data and both 2 and 4 second windows performed well in our cadence experiment (see Chapter 3), we separated our data in 2 second windows with 1 second (50%) overlap, and 4 second windows with 2 second (50%) overlap. Overlapping windows allowed us to generate more ob- servations, at the expense of having some correlation between observations. We examined the effect of window size in Section 5.2. 5.1.2 Axis Sets For each set of features, we had to choose a set of axis projections, created out of our original 3 accelerometer vectors. We examined three axis sets: XYZ Using the original x, y, and z sensor data from the triaxial accelerometer. VH Calculating the gravity vector by taking the mean of the x, y, and z axes, and separating these axes into a vertical component and a horizontal component (as in [68], but with our 2 or 4 second window rather than 10 seconds). MAG Using the magnitude (Euclidean or `-2 norm) of the x, y, and z axes to have a rotationally-invariant feature set (as in our cadence detection algorithm; see Chapter 3). These axis sets explore the tradeoff between robustness to orientation and in- formation: XYZ provides all captured information, but is sensitive to differences in orientation, while VH and MAG give robustness by reducing the amount of orientation information captured. 5.1.3 Features Features are extracted using the Python programming language, drawing upon the Scipy software package [29]. Features were chosen to represent a large number of 52 basic statistical observations of both time and frequency domain data: our gaits can have subtle differences, and we wanted to capture as much information as possible. The spectral domain of each window is calculated using the Fast Lombe-Scargle Periodogram (FASPER) algorithm [49], as in our cadence estimation algorithm RRACE (described in Chapter 3). • For each axis in a specified axis set, we take the following features: – Minimum value – Maximum value – Mean value – Variance – Skewness – Kurtosis – 25th percentile – Median (50th percentile) – 75th percentile – A ten-bin histogram, normalized to have the proportion of each value in each bin – The most powerful spectral frequency (cadence as estimated by RRACE) – The weakest spectral frequency – A weighted average of spectral frequencies by spectral power – Spectral variance – Spectral entropy – A ten-bin histogram of spectral powers, normalized to have the propor- tion of each value in each bin • In addition, we look at basic correlations between the original three axes: – Pearson correlation coefficient for the x-axis and the y-axis – Pearson correlation coefficient for the x-axis and the z-axis 53 – Pearson correlation coefficient for the y-axis and the z-axis – P-value for the Pearson correlation coefficient for the x-axis and the y-axis – P-value for the Pearson correlation coefficient for the x-axis and the z-axis – P-value for the Pearson correlation coefficient for the y-axis and the z-axis • Finally, we take the signal magnitude area (sum of the Euclidean or `-2 norm of every x,y,z tuple). 5.2 Algorithm Selection With our features chosen in Section 5.1, we had to choose the right parameters for our algorithm. Specifically, we needed to decide on window size, the axis set to be used, and which classifier is the most successful at handling our data. We examined the window sizes and axis sets described in Section 5.1.3. We also examined 6 commonly used classification algorithms, all with implementa- tions in Weka [24]: ZeroR A baseline classifier. ZeroR finds the class with the greatest number of instances. This is equivalent to accuracy due to chance when instances are balanced. • Referred to as “ZeroR” in analysis. Naı̈ve Bayes A common classifier, employing Bayes’ Rule to construct an esti- mate of prior probability (probability of a class given the observed features) from a collection of posterior probabilities (probability of a set of features given a class). • Referred to as “NB” in analysis. J48 A Java implementation of the C4.5 decision tree algorithm [24]. A C4.5 deci- sion tree is constructed by iteratively selecting a feature that gives the highest information gain, and splitting on that feature. 54 • Referred to as “J48” in analysis. Multilayer Perceptron A commonly used neural network algorithm. • Referred to as “MP” in analysis. Random Forest An algorithm that uses bagging to randomly select smaller sets of a features, constructing a decision tree for each one. Decisions are made by majority response of these trees. In this implementation, we try two vari- ations by manipulating the number of trees constructed. As the number of trees approaches infinity, the accuracy of a Random Forest classifier con- verges [5]. Through piloting we found accuracy converged between 100 and 500 trees. We thus try those two parameters: • Random Forest with 100 trees (Referred to as “RF100” in in analysis) • Random Forest with 500 trees (“RF500”) Support Vector Machine (SVM) An algorithm that constructs a number of hyper- planes in a high-dimensional feature space to divide the space into regions corresponding to each class. Because SVM is a commonly used algorithm for gait recognition, we combine it with Principal Components Analysis (PCA) to reduce the feature set. We examine four variations of SVM: • SVM without PCA (Referred to as “S” in analysis) • SVM with PCA tuned to explain 95% of the variance (“S+P”) • SVM with PCA tuned to explain 95% of the variance but with a max- imum of 15 features (“S+P15”). We chose this because it is a rule-of- thumb to have 10 times the number of observations • SVM with PCA tuned to explain 95% of the variance but with a maxi- mum of 30 features (“S+P30”) 55 Figure 5.1: Bar-and-whiskers plot of the algorithm selection study. In this work, bar-and-whiskers plots report on quantiles; that is, the bar in the middle represents the median, the box represents the 25th percentile to the 75th percentile, and the ends of the whiskers represent the extremes of the dataset’s range excluding outliers. All bar-and-whiskers plots were produced using the R statistical programming language’s bwplot function in the lattice library. 56 Table 5.1: Violations of normality in the results. Axis Window Algorithm Shapiro-Wilk W Shapiro-Wilk P-Value XYZ 4s J48 0.65 0.00023 MAG 4s NB 0.81 0.012 MAG 2s ZeroR 0.63 0.00012 VH 2s ZeroR 0.63 0.00012 XYZ 2s ZeroR 0.63 0.00012 MAG 4s ZeroR 0.64 0.00017 VH 4s ZeroR 0.64 0.00017 XYZ 4s ZeroR 0.64 0.00017 Note that though these classifiers can require significant time to train, we are primarily concerned with their computational costs during evaluation (so that they can run in real-time on a smartphone). With this in mind, all the classifiers are expected to run in real-time on smartphones. ZeroR runs in constant time (as it al- ways outputs the same class). J48, Random Forests, and Multilayer Perceptron are all tree- or graph-based; classification corresponds to traversing these structures. Naı̈ve Bayes as well is relatively efficient, as we did not use a kernel estimator; for each feature/class pair there is a normal distribution describing estimated posterior probability, from which probabilities for each class can be easily calculated. SVM is expected to be efficient because it only involves the calculation of an exponential kernel function. This must be repeated for each class or pair of classes (as SVM is a binary classifier), but is still expected to be tractable on a smartphone. An analysis was carried out looking at these three factors, using the collected accelerometer data (described in Chapter 4). Datasets for each axis size and win- dow size were generated with a program written in the Python programming lan- guage, using the Scipy software package for statistical features [29]. The gen- erated datasets were stored in the ARFF file format, the standard file format for Weka datasets. For each axis set, window size, and classifier combination, we con- ducted 10 iterations of 10-fold cross validation. We aggregated results for each cross-validation, resulting in 10 estimates of mean axis set/window/classifier ac- curacy. Results follow. To make this computationally tractable and to focus on 57 Table 5.2: Top 20 algorithms ranked by mean percent correct. All results shown are the mean value of 10 iterations of 10-fold cross-validation. Analysis was conducted on data from all participants but limited to the Front Pocket location and the Pedestrian gaitset. Axis Window Algorithm Mean percent correct XYZ 2s RF500 83.70 XYZ 2s RF100 81.85 XYZ 4s RF500 80.95 XYZ 4s RF100 79.50 XYZ 2s S+P15 72.84 XYZ 2s S+P30 72.83 XYZ 4s S+P30 72.79 MAG 4s RF500 72.28 XYZ 4s S+P15 71.39 VH 4s RF500 71.21 MAG 2s RF500 71.20 XYZ 4s S+P 71.14 MAG 4s RF100 70.80 MAG 2s RF100 69.99 XYZ 2s S+P 69.93 VH 4s RF100 69.48 VH 2s RF500 68.91 XYZ 4s MP 67.76 VH 2s RF100 67.58 XYZ 2s MP 66.47 interpretable results, we used a single location (Front Pocket) and a single gaitset (Pedestrian), described in Chapter 4. The location and gaitset were chosen a pri- ori, and because they were the most similar to locations and gaits used in previous work. The Pedestrian gaitset is expected to be the most difficult to classify, as it has the most gaits (6) and was designed around subtle differences between gaits (e.g., walking and walking quickly, jogging and running). We thus had 10 observations in each of the 3×2×10 = 60 datasets, generated with Pedestrian/Front Pocket data from all 12 participants. Results are visible in Figure 5.1. 58 We planned an Analysis of Variance to compare the different factors. How- ever, the Shapiro-Wilk test of normality failed on the residuals of 8 data cells at the 5% level of significance (see Table 5.1). We thus were unable to detect statisti- cal effects for each of the three factors (axis set, window size, and classifier), and could not conduct analysis through planned contrasts. Instead, because we hoped to choose a single best algorithm for our subsequent analysis on gaitset and loca- tion, we took the highest-ranking algorithm: XYZ/2s/RF500. See Table 5.2 for the ranked accuracy of the top 20 tested algorithms. Further analysis of these differ- ent classification parameterizations is left to future work; possible approaches to achieve normality include larger sample sizes (i.e., a greater number of iterations of 10-fold cross-validation), non-parametric analysis (such as bootstrap methods), and transformations of the response variables. 5.3 Location And Gait After choosing our algorithm in Section 5.2, we conducted our main algorithm analysis with location, gaitsets, and participants as factors, in that we examine the benefit of knowledge of these different factors by training classifiers that assume specific levels (such as an algorithm that is trained only on Exercise gaits with Front Pocket as the location). We report our analysis design in Section 5.3.1, and our results in Section 5.3.3 and Section 5.3.4. Discussion of these results can be found in Chapter 6. 5.3.1 Analysis Design A second analysis was carried out, in which we compared the effects of carrying location, gaitset (one of Pedestrian, Exercise, Game, or All), and participant, using the same dataset as in Section 5.2 (fully described in Chapter 4). We used the best performing algorithm in our Algorithm Selection experiment (Section 5.2): we used a 2 second window on the raw X/Y/Z signals with 50% (or 1 second) overlap, extracting features as described in Section 5.1, using a Random Forest classifier with 500 trees implemented in Weka [24]. We conduct this analysis twice, once training and testing on all 12 participants (“All Participants”, Section 5.3.3), and once training and testing on each participant 59 (“By-Participant”, Section 5.3.4). For each location, gaitset, and (for by-participant analysis) participant, we conducted 10 iterations of a 10-fold cross-validation. Our results analyze the mean of each cross validation, giving 10 data points for each location/gaitset(/participant) combination. 5.3.2 Computational Setup and Performance These analyses were conducted on a Macbook Pro laptop with a 2.7 GHz Intel i7 processor and 8 GB of RAM. Training the All Locations/All Gaits/All Partici- pants data cell with the RF500 algorithm exceeded the 8GB memory capacity of the laptop. We should note that these computational costs are only present when training the algorithm, and that the RF500 algorithm, once trained, can perform efficiently on mobile devices. All other datasets were analyzed on this laptop, al- though conducting 10 10-fold cross-validations can take several hours depending on the dataset. 5.3.3 Results - All Participants Results of the cross-validation experiment over all participants suggests a trend in gaitset, but potential interactions between gaitset and carrying location; see Fig- ure 5.2 for results grouped by gaitset, Figure 5.3 for results grouped by carrying location, and Table 5.3 for mean classification rates. Unfortunately, due to a lack of computing power, the All Locations/All Gaits data cell could not be generated. Confusion matrices have been produced for all gaits, and gaits grouped by gaitset, over all locations and all participants. As the All Locations/All Gaits/All Participants data cell could not be computed with RF500, we used RF100 as a substitute in the confusion matrix, presented in Figure 5.4. Major patterns in the all gaits confusion matrix follow, using “→” to indicate mis-classification. Note that mis-classification is not necessarily bidirectional; when it is, we use “↔”. • many gaits→ ascendstairs and descendstairs • many gaits→ narrow and walkbackwards, although not as strongly as with ascendstairs and descendstairs. • Major confusions between two gaits are: 60 – walk→ fastwalk – walk→ ascendstairs – jog→ fastwalk – ascendstairs→ descendstairs – fastwalk→ descendstairs – toes→ narrow – walksideways→ narrow – walkbackwards→ kickingbum Major patterns in the confusion matrices by gaitset are: • Pedestrian shows a number of confusions. As in the all gaits confusion ma- trix (Figure 5.4), many gaits were mis-classified as ascendstairs, and several were mis-classified as jog. Both of the stairs were often correctly distin- guished from jog and run. Few gaits were confused with run and walk. Major individual confusions include: – run→ jog – To a lesser extent, walk↔ fastwalk – To a lesser extent, jog↔ fastwalk • Exercise demonstrates few standout confusions between its gaits. The most confused gait was heels, in that it has the darkest column excluding the di- agonal element, followed by liftknees. The strongest individual confusion is liftknees→ heels. Toes and kickingbum were confused with each other more than with the other two gaits. • Game demonstrates many confusions; several gaits were classified as nar- row. Strong individual confusions are: – walksideways→ narrow – walksideways→ walkbackwards – walkbackwards→ narrow 61 – penguin→ walkbackwards – To a lesser extent, walkbackwards→ walksideways – To a lesser extent, penguin→ twofoothop As in Section 5.2, we planned an Analysis of Variance to compare the different factors. However, like Section 5.2, the Shapiro-Wilk test of normality failed on the Exercise/Arm data cell residuals at the 5% level of significance (W=0.83, p=0.037). We thus were unable to detect statistical effects for each of the two factors (location and gaitset), and could not conduct analysis through planned contrasts. Again, further analysis of these different classification parameterizations is left to future work. 62 Table 5.3: Mean gaitset/location accuracy rate over all participants. All results shown are the mean value of 10 iterations of 10-fold cross- validation. Analysis was conducted using XYZ/2s/RF500. Gaitset Location Mean percent correct Exercise Back Pocket 87.2 Exercise Front Pocket 86.6 Exercise Arm 85.3 Game Front Pocket 85.0 Exercise Belt 84.8 Exercise Backpack 84.6 Game Back Pocket 84.5 Pedestrian Front Pocket 83.7 Exercise Hand 83.4 Pedestrian Back Pocket 82.6 Game Arm 81.7 All Gaits Front Pocket 80.1 Game Backpack 79.9 All Gaits Back Pocket 79.8 Game Belt 79.6 Pedestrian Belt 79.5 Game Hand 78.9 Pedestrian Backpack 78.9 Pedestrian Arm 78.4 Pedestrian Hand 77.2 All Gaits Arm 74.5 All Gaits Belt 74.4 All Gaits Backpack 74.3 All Gaits Hand 73.5 63 Figure 5.2: Bar-and-whiskers plot of 10 means of 10-fold cross-validation results of the XYZ/2s/RF500 algorithm trained on all participants by carrying location and gaitset, arranged to highlight differences in gaitset. 64 Figure 5.3: Bar-and-whiskers plot of 10 means of 10-fold cross-validation results of the XYZ/2s/RF500 algorithm trained on all participants by carrying location and gaitset, arranged to highlight differences in carrying location. 65 Figure 5.4: Confusion matrix for a 10-fold cross validation of the chosen al- gorithm (XYZ/2s/RF500) trained on all participants, all gaits, and all carrying locations. Darker squares represent a higher classification rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). 66 (a) Pedestrian (b) Exercise (c) Game Figure 5.5: Confusion matrices for a 10-fold cross validation of the chosen algorithm (XYZ/2s/RF500) trained on all participants and all carrying locations for each gaitset. The algorithm thus attempts to distinguish only between gaits in the gaitset. Darker squares represent a higher clas- sification rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). 67 5.3.4 Results - By-Participant Analysis of all participants gives some insight, but training the XYZ/2s/RF500 al- gorithm on each participant might improve results with the short data collection time of 30s per gait and location, and reveals variability in individual differences; see Figure 5.7, Figure 5.8, and Figure 5.9 for graphs of the effect of participants by gaitset and location, and Figure 5.10 for a graph involving only the effect of participants. Top performing mean data classification rate was 100.0%, present in three data cells (involving two participants): Game/Front Pocket/P3, Game/Back- pack/P9, and Exercise/Backpack/P3. The worst performing mean data classifica- tion rate was All Gaits/All Locations/P10 with a mean of 63.2% classification rate. See Table 5.5 and Table 5.6 for the top 20 and bottom 20 ranked datasets, respec- tively. Confusion matrices for the four worst ranked data cells are presented in Figure 5.11, Figure 5.12, Figure 5.13, and Figure 5.14. As with Section 5.2 and Section 5.3.3, we planned an Analysis of Variance to compare the different factors. However, as before, the Shapiro-Wilk test of normality failed on 55 of 336 data cell residuals at the 5% level of significance. We thus were unable to detect statistical effects for each of the three factors (location, gaitset, and participant), and could not conduct analysis through planned contrasts. Further analysis of these different classification parameterizations is again left to future work. 68 Figure 5.6: Bar-and-whiskers plot of the Location-Gaitset-Participant Analysis, aggregated by Location and Gaitset. 69 Table 5.4: Mean gaitset/location accuracy rate, by-participant. All results shown are the mean value of 10 iterations of 10-fold cross-validation for every participant. Analysis was conducted using XYZ/2s/RF500. Gaitset Location Mean percent correct Exercise Belt 92.2 Exercise Hand 91.0 Exercise Arm 90.1 Exercise Back Pocket 89.8 Exercise Front Pocket 89.7 Exercise Backpack 88.7 Game Back Pocket 88.6 Game Front Pocket 88.2 Pedestrian Hand 87.9 Game Hand 87.5 Game Arm 87.4 Exercise All Locations 87.3 Pedestrian Front Pocket 87.0 Game Belt 87.0 Pedestrian Belt 86.7 Game Backpack 86.3 Pedestrian Back Pocket 86.0 Pedestrian Arm 85.6 Pedestrian Backpack 84.7 Game All Locations 84.1 All Front Pocket 83.0 Pedestrian All Locations 82.7 All Hand 82.3 All Back Pocket 82.1 All Belt 81.6 All Arm 80.7 All Backpack 80.1 All All Locations 78.1 70 Figure 5.7: Bar-and-whiskers plot of the Location-Gaitset Analysis. By- participant analyses are presented by gaitset and location. 71 Figure 5.8: Bar-and-whiskers plot of the Location-Gaitset Analysis, to high- light gaitset performance for each of participants 1 through 6. Notice that participants tend to consistently perform well or poorly independent of gaitset/location conditions. This graph is continued in Figure 5.9. 72 Figure 5.9: Bar-and-whiskers plot of the Location-Gaitset Analysis, to high- light gaitset performance for each of participants 7 through 12. This graph is a continuation from Figure 5.8. 73 Figure 5.10: Bar-and-whiskers plot of the Location-Gaitset Analysis. By-participant analyses are presented without any other factors. 74 Table 5.5: Top 20 of 336 gaitset/location/participant data cells by mean per- cent correct. All results shown are the mean value of 10 iterations of 10-fold cross-validation. Analysis was conducted using XYZ/2s/RF500. Three participants are present in this group. Gaitset Location Participant Mean percent correct Game Front Pocket P3 100.0 Game Backpack P9 100.0 Exercise Backpack P3 100.0 Game Belt P3 99.9 Pedestrian Belt P3 99.7 Game Front Pocket P9 99.7 Pedestrian Hand P3 99.4 Game Belt P9 99.4 Pedestrian Back Pocket P3 99.4 Pedestrian Front Pocket P9 99.4 Game Backpack P3 99.3 Game Arm P3 99.3 Pedestrian Belt P9 99.2 Pedestrian Backpack P9 99.2 Exercise Back Pocket P9 99.2 Game Back Pocket P9 99.0 All Gaits Backpack P3 98.9 Exercise Back Pocket P3 98.9 Exercise Front Pocket P2 98.8 Game Arm P9 98.8 75 Figure 5.11: Confusion matrix for the worst-performing gaitset/location/par- ticipant combination using XYZ/2s/RF500: All Locations/All Gait- s/P10. Darker squares represent a higher classification rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). 76 Figure 5.12: Confusion matrix for the second worst-performing gaitset/loca- tion/participant combination using XYZ/2s/RF500: All Locations/All Gaits/P11. Darker squares represent a higher classification rate. Num- bers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). 77 Figure 5.13: Confusion matrix for the third worst-performing gaitset/loca- tion/participant combination using XYZ/2s/RF500: Arm/All Gait- s/P11. Darker squares represent a higher classification rate. Numbers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). 78 Figure 5.14: Confusion matrix for the fourth worst-performing gaitset/lo- cation/participant combination using XYZ/2s/RF500: Backpack/All Gaits/P10. Darker squares represent a higher classification rate. Num- bers presented are percents of classification rates for each actual gait (that is, rows sum to 100%). 79 Table 5.6: Worst 20 of 336 gaitset/location/participant data cells ranked by mean percent correct. All results shown are the mean value of 10 iterations of 10-fold cross-validation. Analysis was conducted using XYZ/2s/RF500. Five participants are present in this group. Gaitset Location Participant Mean percent correct All Gaits Backpack P4 71.7 All Gaits Backpack P10 71.1 All Gaits All Locations P8 70.3 All Gaits Backpack P11 70.2 All Gaits Hand P8 70.1 All Gaits Backpack P1 70.1 Pedestrian Belt P10 70.1 All Gaits Arm P8 69.7 All Gaits Arm P5 69.5 All Gaits Belt P10 68.6 Pedestrian All Locations P10 68.4 All Gaits Front Pocket P11 68.3 All Gaits Back Pocket P11 67.6 All Gaits Front Pocket P10 67.3 All Gaits Arm P10 67.1 All Gaits Backpack P5 67.0 All Gaits Back Pocket P10 66.9 All Gaits Arm P11 66.7 All Gaits All Locations P11 66.6 All Gaits All Locations P10 63.2 80 Chapter 6 Gait Classification Discussion Everywhere is walking distance if you have the time. — Steven Wright In this chapter, we discuss the findings of the Gait Classification algorithm de- veloped in Chapter 5: a Random Forest classifier with 500 trees extracting features from the original x, y, and z axes in 2 second windows. Our major findings include: overall success of classification in Section 6.1, a discussion of the complex inter- play between location and gaitset in Section 6.2, and an examination of the effects of individual differences in Section 6.3. Finally, we discuss the limitations of our study in Section 6.4. 6.1 Overall Success Overall, the classification scheme performed well, with a varying range of success. Before we begin our discussion, we should note that comparisons to previous work are difficult due to differences in application areas and methodology (such as the number of subjects, the amount of recorded data, and the types and numbers of gaits). Despite this, we still make an effort to compare our results with previous work whenever possible. When trained and tested over all participants, accuracy ranged from 73.5% (All Gaits/Hand) to 87.2% (Exercise/Back Pocket), although we unfortunately and no- tably exclude All Gaits/All Locations (due to limitations of computational power; 81 see Section 5.3.2 for more details), which is expected to be the worst performer. All are larger than chance, which ranges from 6.66̄% (All Gaits) to 25% (Exercise) depending on gaitset. See Table 5.3 for details. As we only collected 30 seconds for each gait/participant/location section, we also investigated the improvement when the classifier is trained upon participant- specific data. Overall, accuracy improved by about 5%, although due to non- normality of the data we cannot verify that this difference is statistically significant. However, it suggests that a short personalization session might improve classifica- tions rates. Even with personalization, though, there was a great deal of variation in accuracy: mean classification rates ranged from 63.2% to 100.0%. We attribute this large range of success to differences in gaitset, carrying location, and partici- pant, and discuss these factors in detail in Section 6.2 and Section 6.3. The lowest mean classification rate, 63.2%, is well above chance (6.66̄% for all 15 gaits). In- deed, for all locations and all gaits, after training on an individual for 30 seconds for each gait, the mean classification rate across all 12 participants was 78.1%. For non-real-time analysis we expect that both personalized and non-person- alized algorithms (that is, trained on the user’s data or not) could be sufficient for extremely high classification rates: Kunze et al. used a majority vote of several 1 second windows with 82% classification rate in 1 or more minute sequences to achieve 100% classification rate of walking vs not walking [36]. For real-time applications, a classification rate of 78.1% could be used with novel application design or game mechanics to improve perceived recognition rates. We also note that many of our gaits are very similar, and that careful pruning of the selection of gait can also improve performance: our Exercise gaitset had a mean classification rate of 87.3% over all body locations when personalized, and achieved a mean classification rate of 92.2% when it was known to be mounted on the user’s belt. Thus, our results are promising for robust classification of a wide variety of gaits. 6.2 Gait is a Complex Ecosystem Involving Gaitset and Location We have several main conclusions about gaitset and location. Altogether, they paint a picture of subtle interactions involving many different factors. We comment 82 upon our main findings in this section, and explore what patterns emerge while respecting the complexity of these factors. There is no clear cut ranking of gaitset performance As our results demonstrate, the success rate of gait recognition depends upon a variety of features, ranging from 73.5% (All Gaits/Hand) to 87.2% (Exercise/Back Pocket) with non-personalized algorithms, and from 63.2% (All Gaits/All Lo- cations/P10) to 100.0% (three different data cells) for personalized algorithms (trained on the individual). Drawing from Figure 5.6, we see that this wide varia- tion does not distinguish a clear ranking of locations or gaitsets in terms of accu- racy. Even if there were statistically significant differences between the presented factors, interactions between the factors are present between gaitset and location: Game has a higher median than Pedestrian in some locations (such as Back Pocket or Arm), and a lower median in others (such as Belt and Front Pocket). In aggregate, Exercise performs best and All Gaits performs worst However, we can still draw some conclusions about the relative performance of the different gaitsets. In general, the best performing gaitset tended to be Exercise, and the worst performing gaitset was the expected worst performer, All Gaits. In the all participant analysis, the top three performing data cells are Exercise data sets, and the worst 4 performing data cells are All Gaits data sets (Table 5.3). In the by-participant analysis, the top 6 performing algorithms are Exercise data sets, and the worst 6 performing algorithms are All Gaits data sets (Table 5.4). In both all participant and by-participant analyses, the best performer for every location was Exercise, and the worst performer for every location was All Gaits. Although we do not yet have statistical analysis to support this claim, the small ranges of our data sets (visible in the bar-and-whiskers plot of Figure 5.2) make a promising case. A close look at the by-participant results (Figure 5.8 and Figure 5.9) shows that these results might change dramatically depending on the user when the algorithm is personalized. We elaborate on this effect of individual differences in Section 6.3. Gait similarity in a gaitset tends to affect classification performance Though there could be several reasons for the difference in performance of dif- ferent gaitsets, such as the number of gaits within each gaitset, our findings suggest that the types of gaits chosen within each gaitset are an important factor. As we can 83 see in Figure 5.5, Exercise’s gaits have fewer standout confusions with each other than the gaits in Pedestrian and Game, which is not surprising considering that Ex- ercise is the best performing gaitset. Pedestrian has a number of confusions that suggest differences between these gaits are quite subtle, such as walk → fastwalk and jog→ fastwalk.1 This is an expected result, and is consistent with the design of the Pedestrian gaitset to include very similar gaits (see Section 4.1). For the Pedestrian gaitset, then, we note that distinguishing between different categories of interpreted speeds of walking is challenging, and it may be best to only classify walking and jogging or running, or refer to cadence directly in applications. In the Game gaitset, most gaits were mis-classified as narrow, especially walk- sideways and walkbackwards. The least confused gaits were narrow→ walkside- ways, and penguin→ walksideways. We thus suggest that narrow, walksideways, and walkbackwards are similar, but different from twofoothop and penguin (which, unexpectedly, had a strong confusion with penguin → twofoothop). Ultimately, many of these gaits are similar to normal walking, and those that differ were more easily recognizable. The effect of carrying method is subtle The smartphone’s carrying location has a subtler effect on classification per- formance. Looking at all participants analysis (Figure 5.3), we can see that Front Pocket and Back Pocket tend to perform better than other locations for the Pedes- trian, Game, and All Gaits gaitsets; this could also be the case in Exercise, but if so the effect is less clear. However, once we look at the by-participant bar- and-whiskers plots (Figure 5.8 and Figure 5.9), we find that this varies strongly with individuals. In several cases (such as P1/All Locations and P1/Arm) Exercise performs best and All Locations performs worst, as they do in aggregate, but in several cases (such as P1/Hand or P1/Front Pocket) we no can no longer draw the same conclusions. This effect of individual differences is quite prominent, and is discussed next in Section 6.3. 1Recall that “→” signifies mis-classification; thus, “walk → fastwalk” signifies that walk was frequently mis-classified as fastwalk. 84 6.3 Individual Differences Have a Strong Influence on Gait Recognition Individual differences add an important dimension to the relationship between lo- cation and gaitset. Looking at Figure 5.7, we see that that adding participant as a factor diminishes the range of each set of results. This suggests that participant, location, and gaitset all contribute to the success or failure of training an algo- rithm. This is especially remarkable given that in this analysis, each algorithm was trained upon the participant, location, and gaitset combination. Furthermore, the confusion matrices for the four worst performing participant/location/gaitset data cells (Figure 5.11, Figure 5.12, Figure 5.13, and Figure 5.14) show wide variation. This suggests that individual differences are a major factor in the recognizability of these gaits. This could be due to a number of confounded variables, from gait interpretation to the clothing worn by a participant (see Section 6.4 for a longer discussion of confounding variables). Finally, looking at participants as a single factor in Figure 5.10 suggests that individual differences do not account for all variability; ranges are quite large. However, we note that three participants seem to perform considerably better than others: P2, P3, and P9; in fact, the 18 best performing data cells were from P3 and P9. No distinguishing demographic features stand out for these three participants: both sexes are represented, heights and weights vary (155-183cm and 51-64kg respectively), and no notable behaviours were observed during the study. This suggests that these gaits may work very effectively for certain individuals with minimal personalization (30 seconds per gait and location). If this is from gait interpretation, then it is possible that, with instruction for users, a wide variety of gaits could be effectively classified. This wide variation in individual differences for activity recognition is consis- tent with literature (Zhang et al. report 69% to 95.3% with their best classifier [70], Kunze et al. report 72% to 93% with a binary classifier [36]), but we are not aware of any previous work that has commented on or explored individual differences and their impact on gait. Furthermore, previous work has used a non-personalized algorithm for all participants; we demonstrate this effect even when it is personal- ized via training on each participant, location, and gaitset, suggesting that it is not 85 merely error from the classifier, but an intrinsic component of each participant’s gaits that must be considered in future gait analysis. We thus suggest that future work attempting gait classification with gaits that might be susceptible to individual differences, such as the interpretation of unusual gaits or gaits with subtle differences must accomodate these differences. We aim to pursue this in our future work; see Section 7.2 for more details. 6.4 Limitations There are several limitations to our study that must be considered for interpretation and future work. Major limitations involve the amount of data collected, confounds in experimental factors, and the selected algorithm for analysis. Data collection When collecting data, we only collected 30 seconds per gait/participant/loca- tion data cell in order to satisfy a compromise between covering a large number of gaits and an adequate number of samples per gait. Although our results suggest that this can be very successful for some users with a limited amount of data col- lection, other participants did not perform as well. Overall, we expect that more data will lead to better collection rates. In fact, previous work has often collected more data and used larger window sizes than 2-seconds. We therefore have shown a lower-bound, but not explored the effect of increasing data size. Because of this limitation, we used 50% overlap in our windows, which may bias our results (as data points are not all independent). However, this is also representative of our target application - we plan on using an overlapping window for possible exercise applications or games using this classification algorithm. As well, our results and conclusions may not generalize to all populations. Although our 12 participants have a range of heights, weights, and ages (Table 4.2), they were all younger adults without any medical conditions that would affect their gaits. The influence of age (such as gaits performed by children or the elderly) and effects of medical conditions (such as a limp) were not considered, and must be left to future work. Additional considerations include using a single commercially-available phone, and not including handbags or purses as a carrying location in data collection. 86 Confounds There are some confounds found in our experimental factors. First, we only collected data once from each participant. Therefore, individual differences in participants are involved with many different factors, including but not limited to: interpretation of gait description, physical gait characteristics, clothing, and phone orientation in its location-on-person. Our goal was to develop an algorithm that is robust to many of these factors, but given the effect of individual differences on classification rates, we recommend that future work investigate these confounding factors. Another major confound is that the gaitsets do not have a consistent number of gaits: Pedestrian has 6 gaits, Game has 5, and Exercise has 4. We thus have not investigated the role of number of gaits compared with the types of gaits in a gaitset. Further analysis is planned but left to future work. Selected Algorithm We selected a single algorithm, which used the original x, y, and z axes, a 2 second window, and a Random Forest classifier with 500 trees (see Section 5.2). It is possible that the gait classification ecosystem is even more complicated, in that difference classification schemes could be more or less effective depending on different gaitsets, locations, and individuals. This could be contingent on axis set, window size, classification algorithm, or any combination of the three factors. 87 Chapter 7 Conclusion Meandering leads to perfection. — Lao Tzu Mobile exercise applications and games show great promise to encourage ex- ercise and fitness, increasingly important in our modern world. Modern mobile devices come with a wealth of sensors, including accelerometers, to support these applications. However, to truly manage ubiquitous gait sensing, one must take into account a variety of factors, including the carrying location of the device as well as the individual differences of the gait itself. Cadence estimation, a simple gait analysis task, is achievable through accelerometers on smartphones in a manner ro- bust to these factors. However, gait classification remains a more challenging task that, though achievable, is heavily influenced by carrying location and individual differences. In this chapter, we summarize and conclude our research. First, in Section 7.1, we review our major findings from these two types of sensing. We then layout future directions in Section 7.2. 7.1 Conclusions In this section, we review the overall results from our two main studies. Our cadence estimation study, presented in Chapter 3, demonstrated that we can robustly estimate a simple gait parameter, cadence, in real-time on mobile de- 88 vices. We present the algorithm to do this, RRACE: Robust Real-time Algorithm for Cadence Estimation, which uses a frequency-domain analysis of the accelerom- eter signal, transformed to be orientation-invariant. We demonstrate that RRACE is robust to carrying location, individual differences, and moderate to fast speeds, although it is sensitive to slow speeds (where our participants’ walking becomes erratic). Overall, it was able to estimate cadence with 5% or lower error rate. We also compared RRACE to the most accurate published time-based cadence estimation algorithm. We found that this time-based algorithm is extremely sensi- tive to carrying location and individual differences. Though some algorithms are affected by these factors, it is possible to overcome these differences using robust sensing techniques. Our gait classification study, presented in Chapter 4, Chapter 5, and Chapter 6, demonstrated similar themes in the more challenging problem of classifying user gait. First, we presented 15 different gaits that, in general, were consistently inter- preted during piloting. We also provide 3 gait sets suitable for different purposes: Pedestrian gaits, indicative of common context-aware gaits; Game gaits, with sim- ple metaphors that are suitable for an exercise game (especially for kids); and Ex- ercise gaits, gaits that involve different muscle groups in the legs. We found that it was feasible to classify gait with quick personalization (30 seconds of recording per location and gait): our worst-case scenario, classifying gait over all carrying locations and 15 different gaits, had a mean personalized performance of 78.1%. This improved when the carrying location was known and the set of gaits was re- duced, achieving a mean best score of 92.2% with the Exercise gaitset located at the belt. These results were accomplished using a Random Forest classifier with 500 trees analyzing the original x, y, and z signal axes using a 2 second window, and can be computed on a smartphone in real time. This algorithm was determined by a study of different classification algorithm parameterizations. During analysis of the classification algorithm, we found that carrying loca- tion, gaitset, and individual differences all influence gait classification success, even when considered in training. This reflects our findings with the time-based algorithm in the cadence estimation study; ultimately, gait is a complex entity, and sensing with smartphone accelerometers in a variety of locations is a difficult problem. However, we have been able to draw major conclusions. First, that the se- 89 lection of a set of gaits influences classification rates, with many similar gaits being confused by the classifier. With this in mind, the Exercise gaitset tends to perform best in general, and attempting to classify all 15 gaits is in general the most chal- lenging problem. We have presented details of which gaits were mis-classified as other gaits, to provide guidelines for future design with gait classification. Carrying location has a subtler effect than gait selection, but in general suggests front and back pockets to be the best carrying locations for gait classification; this is strongly affected by each participant’s individual differences. Finally, and on that note, we found that while classification rates per individuals were in general promising, some individuals had extremely high classification rates regardless of location or gaitset, suggesting that individual differences might be the most influential factor in gait classification. We encourage others to investigate this phenomenon with further exploration. 7.2 Future Work Given our findings, we have several possible directions for future work, includ- ing improvements for both algorithms, and an expansion into different application areas. We believe that our cadence estimation algorithm, RRACE, can be improved through harmonic analysis. At this time, RRACE only analyzes the most powerful frequency observed; by examining more than this, the algorithm might be able to become even more accurate. As well, RRACE does not perform well at lower speeds. An adaptive algorithm, such as switching to a step detection technique when speed or cadence is reduced, could remedy this flaw. Finally, smoothing filters could be applied to improve the cadence estimation. For gait classification, we find that the question of individual differences to be unanswered but important. We plan on investigating individual differences in a number of ways: investigating the role of instruction for gaits, to see whether simple verbal descriptions of gaits are sufficient; examining whether the perfor- mance of the gait itself is the problem, or the carrying locations or clothing has an impact; and investigating whether these individual differences can be character- ized and understood. Improvements to the algorithm itself might be accomplished 90 by developing a more intelligent classification model (e.g., using a Markov-model approach to consider previous classifications, or Bayes-net-like approach using dif- ferent classifiers in tandem to make a more accurate classification of gait, carrying location, and user). Finally, we hope to expand this work into other application areas beyond sup- porting fitness and exercise. Examining subtler gait characteristics, such as asym- metry or limping, could be beneficial as an early warning system for those who might suffer debilitating diseases, or for remote monitoring of rehabilitation. Us- ing individual differences to recognize the user of a mobile device could be useful for security or context-aware applications. Finally, a better understanding of the role of location could help us develop context-aware applications, such as recog- nizing when a user is carrying the phone in his or her pocket; for example, by turning the ringer volume up when the phone is in a pocket to improve detection rate. Ultimately, there is great potential for accelerometer sensing in mobile devices, with applications ranging from health and fitness to simple interactive context- aware applications. This work has provided valuable information about some of the factors that influence gait analysis in real-time on a mobile smartphone, including carrying location and individual differences. We hope to transform gait sensing to be ubiquitous and robust, to help improve fitness, health, and other applications in the world. 91 Bibliography [1] Android Open Source Project. Android Developers, 2012. URL http://developer.android.com/index.html. → pages 43 [2] Arduino. http://www.arduino.cc/, 2012. URL http://www.arduino.cc/. → pages 23 [3] M. Bell, M. Chalmers, L. Barkhuus, M. Hall, S. Sherwood, P. Tennent, B. Brown, D. Rowland, S. Benford, M. Capra, and A. Hampshire. Interweaving mobile games with everyday life. In Proceedings of the SIGCHI conference on Human Factors in computing systems, number April in CHI ’06, pages 417–426. ACM Press, 2006. URL http://doi.acm.org/10.1145/1124772.1124835. → pages 9 [4] M. Brehmer, T. C. N. Graham, and T. Stach. Activate Your GAIM : A Toolkit for Input in Active Games. In ACM FuturePlay 2010, pages 151–158, 2010. → pages 10 [5] L. Breiman. Random Forests. Machine Learning, 45:5–32, 2001. → pages 55 [6] P. Briggs and P. L. Olivier. Biometric daemons. In Proceeding of the twenty-sixth annual CHI conference extended abstracts on Human factors in computing systems - CHI ’08, pages 2423–2431, New York, New York, USA, Apr. 2008. ACM Press. ISBN 978160558012X. doi:10.1145/1358628.1358699. URL http://dl.acm.org/citation.cfm?id=1358628.1358699. → pages 11, 18 [7] C. B. Chan, E. Spangler, J. Valcour, and C. Tudor-Locke. Cross-sectional Relationship of Pedometer- Determined Ambulatory Activity to Indicators of Health. Obesity research, 11(12):1563–1570, Dec. 2003. ISSN 1071-7323. URL http://dx.doi.org/10.1038/oby.2003.208. → pages 1, 9 92 [8] T. Choudhury, S. Consolvo, B. Harrison, J. Hightower, A. LaMarca, L. LeGrand, A. Rahimi, A. Rea, G. Borriello, B. Hemingway, P. P. Klasnja, K. Koscher, J. A. Landay, J. Lester, D. Wyatt, and D. Haehnel. The Mobile Sensing Platform: An Embedded Activity Recognition System. Pervasive Computing, 7(2):32–41, 2008. → pages 7 [9] S. Consolvo, K. Everitt, I. Smith, and J. A. Landay. Design requirements for technologies that encourage physical activity. In Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI ’06, pages 457–466, New York, New York, USA, Apr. 2006. ACM Press. ISBN 1595933727. doi:10.1145/1124772.1124840. URL http://dl.acm.org/citation.cfm?id=1124772.1124840. → pages 2, 7, 37 [10] S. Consolvo, P. Klasnja, D. W. McDonald, D. Avrahami, J. Froehlich, L. LeGrand, R. Libby, K. Mosher, and J. A. Landay. Flowers or a Robot Army? Encouraging Awareness & Activity with Personal, Mobile Displays. In Proceedings of the 10th international conference on Ubiquitous computing - UbiComp ’08, pages 54–63, New York, New York, USA, Sept. 2008. ACM Press. ISBN 9781605581361. doi:10.1145/1409635.1409644. URL http://dl.acm.org/citation.cfm?id=1409635.1409644. → pages 7, 9 [11] S. Consolvo, D. W. McDonald, T. Toscos, M. Y. Chen, J. Froehlich, B. Harrison, P. Klasnja, A. LaMarca, L. LeGrand, R. Libby, I. Smith, and J. A. Landay. Activity Sensing in the Wild: A Field Trial of UbiFit Garden. In Proceeding of the twenty-sixth annual CHI conference on Human factors in computing systems - CHI ’08, pages 1797–1806, New York, New York, USA, Apr. 2008. ACM Press. URL http://dl.acm.org/citation.cfm?id=1357054.1357335. → pages 7 [12] Y. Cui, J. Chipchase, and F. Ichikawa. A cross culture study on phone carrying and physical personalization. In N. Aykin, editor, UI-HCII’07: Proceedings of the 2nd International Conference on Usability and Internationalization, pages 483–492, Berlin, Heidelberg, July 2007. Springer-Verlag. ISBN 978-3-540-73286-0. URL http://dl.acm.org/citation.cfm?id=1769821.1769881. → pages 2, 18, 23 [13] S. B. Davis, M. Moar, R. Jacobs, M. Watkins, C. Riddoch, and K. Cooke. Ere Be Dragons : heartfelt gaming. Digital Creativity, 17(3):157–162, Jan. 2006. ISSN 1462-6268. doi:10.1080/14626260600882430. URL http://www.tandfonline.com/doi/abs/10.1080/14626260600882430. → pages 8, 9 93 [14] R. de Oliveira and N. Oliver. TripleBeat. In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services - MobileHCI ’08, pages 255–264, New York, New York, USA, Sept. 2008. ACM Press. ISBN 9781595939524. doi:10.1145/1409240.1409268. URL http://dl.acm.org/citation.cfm?id=1409240.1409268. → pages 6 [15] S. Dornbush, A. Joshi, Z. Segall, and T. Oates. A Human Activity Aware Learning Mobile Music Player. In Proceedings of the 2007 conference on Advances in Ambient Intelligence, pages 107–122, June 2007. URL http://dl.acm.org/citation.cfm?id=1563773.1563780. → pages 2, 11 [16] G. T. Elliott and B. Tomlinson. PersonalSoundtrack. In CHI ’06 extended abstracts on Human factors in computing systems - CHI ’06, pages 736–741, New York, New York, USA, Apr. 2006. ACM Press. ISBN 1-59593-298-4. doi:10.1145/1125451.1125599. URL http://dl.acm.org/citation.cfm?id=1125451.1125599. → pages 6, 13 [17] D. Figo, P. C. Diniz, D. R. Ferreira, and J. a. M. P. Cardoso. Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing, 14(7):645–662, Mar. 2010. ISSN 1617-4909. doi:10.1007/s00779-010-0293-9. URL http://www.springerlink.com/content/0685p2hp8737012m/. → pages 13 [18] J. Froehlich, T. Dillahunt, P. Klasnja, J. Mankoff, S. Consolvo, B. Harrison, and J. A. Landay. UbiGreen: investigating a mobile tool for tracking and supporting green transportation habits. In D. R. Olsen, editor, Proceedings of the 27th international conference on Human factors in computing systems (CHI 2009), volume 09 of CHI ’09, pages 1043–1052. ACM SIGCHI, ACM, 2009. ISBN 9781605582467. doi:10.1145/1518701.1518861. URL http://portal.acm.org/citation.cfm?id=1518861. → pages 10 [19] Y. Fujiki. iPhone as a physical activity measurement platform. In Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems - CHI EA ’10, pages 4315–4320, New York, New York, USA, 2010. ACM Press. ISBN 9781605589305. doi:10.1145/1753846.1754146. URL http://portal.acm.org/citation.cfm?doid=1753846.1754146. → pages 6 [20] Y. Fujiki, K. Kazakos, C. Puri, I. Pavlidis, J. Starren, and J. Levine. NEAT-o-Games: Ubiquitous Activity-based Gaming. In CHI ’07 extended abstracts on Human factors in computing systems - CHI ’07, pages 94 2369–2374, New York, New York, USA, Apr. 2007. ACM Press. ISBN 9781595936424. doi:10.1145/1240866.1241009. URL http://dl.acm.org/citation.cfm?id=1240866.1241009. → pages 9, 10 [21] Y. Fujiki, K. Kazakos, C. Puri, P. Buddharaju, I. Pavlidis, and J. Levine. NEAT-o-Games: Blending Physical Activity and Fun in the Daily Routine. Computers in Entertainment, 6(2):Article 21, July 2008. ISSN 15443574. doi:10.1145/1371216.1371224. URL http://dl.acm.org/ft gateway.cfm?id=1371224\&type=html. → pages 2, 9, 10 [22] D. Gafurov and P. Bours. Improved Hip-Based Individual Recognition Using Wearable Motion Recording Sensor. In T.-h. Kim, W.-c. Fang, M. K. Khan, K. P. Arnett, H.-j. Kang, and D. Ślzak, editors, Security Technology, Disaster Recovery and Business Continuity, volume 122 of Communications in Computer and Information Science, pages 179–186. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010. ISBN 978-3-642-17609-8. doi:10.1007/978-3-642-17610-4. URL http://www.springerlink.com/content/h1458811385x5527/http: //www.springerlink.com/content/m183188075tt9871/. → pages 18 [23] Groundspeak Inc. Geocaching, 2012. URL http://www.geocaching.com/. → pages 9 [24] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The WEKA data mining software. ACM SIGKDD Explorations Newsletter, 11(1):10–18, 2009. ISSN 19310145. doi:10.1145/1656274.1656278. URL http://doi.acm.org/10.1145/1656274.1656278. → pages 54, 59 [25] T. Huynh and B. Schiele. Analyzing features for activity recognition. In Proceedings of the 2005 joint conference on Smart objects and ambient intelligence innovative context-aware services: usages and technologies - sOc-EUSAI ’05, pages 159–164, New York, New York, USA, Oct. 2005. ACM Press. ISBN 1595933042. doi:10.1145/1107548.1107591. URL http://dl.acm.org/citation.cfm?id=1107548.1107591. → pages 14, 17 [26] InterlinkElectronics. http://www.interlinkelectronics.com/, 2011. URL http://www.interlinkelectronics.com/. → pages 23 [27] T. Iso and K. Yamazaki. Gait analyzer based on a cell phone with a single three-axis accelerometer. In Proceedings of the 8th conference on Human-computer interaction with mobile devices and services - MobileHCI ’06, pages 141–144, New York, New York, USA, Sept. 2006. ACM Press. 95 ISBN 1595933905. doi:10.1145/1152215.1152244. URL http://dl.acm.org/citation.cfm?id=1152215.1152244. → pages 15, 17 [28] K. L. v. Jensen, R. Krishnasamy, and V. Selvadurai. Studying PH. A. N. T. O. M. in the wild. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction - OZCHI ’10, pages 17–20, New York, New York, USA, Nov. 2010. ACM Press. ISBN 9781450305020. doi:10.1145/1952222.1952228. URL http://dl.acm.org/citation.cfm?id=1952222.1952228. → pages 9 [29] E. Jones, T. Oliphant, P. Peterson, and Others. Scipy: Open source scientific tools for Python, 2012. URL http://www.scipy.org/. → pages 52, 57 [30] I. Karuei, O. Schneider, B. Stern, M. Chuang, and K. MacLean. RRACE: Robust Real-time Algorithm for Cadence Estimation. In preparation for a top-tier journal, 2012. → pages 22 [31] Y. Kawahara, H. Kurasawa, and H. Morikawa. Recognizing User Context Using Mobile Handsets with Acceleration Sensors. In 2007 IEEE International Conference on Portable Information Devices, pages 1–5. IEEE, May 2007. ISBN 1-4244-1039-8. doi:10.1109/PORTABLE.2007.12. URL http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4216907. → pages 14, 17 [32] K. Kazakos, Y. Fujiki, I. Pavlidis, T. Bourlai, and J. Levine. NEAT-o-Games: Novel Mobile Gaming Versus Modern Sedentary Lifestyle. In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services - MobileHCI ’08, pages 515–518, New York, New York, USA, Sept. 2008. ACM Press. ISBN 9781595939524. doi:10.1145/1409240.1409333. URL http://dl.acm.org/citation.cfm?id=1409240.1409333. → pages 9, 10 [33] A. M. Khan, Y.-K. Lee, S. Lee, and T.-S. Kim. Accelerometer’s position independent physical activity recognition system for long-term activity monitoring in the elderly. Medical & Biological Engineering & Computing, 48(12):1271–1279, Dec. 2010. ISSN 1741-0444. doi:10.1007/s11517-010-0701-3. URL http://www.ncbi.nlm.nih.gov/pubmed/21052854. → pages 15, 17, 19 [34] T. Kobayashi, K. Hasida, and N. Otsu. Rotation invariant feature extraction from 3-D acceleration signals. In 2011 IEEE International Conference on 96 Acoustics, Speech and Signal Processing (ICASSP), pages 3684–3687. IEEE, May 2011. ISBN 978-1-4577-0538-0. doi:10.1109/ICASSP.2011.5947150. URL http://ieeexplore.ieee.org/xpl/freeabs all.jsp?arnumber=5947150. → pages 18 [35] A. Komninos, R. Wallace, and P. Barrie. Mobile empathy. In Proceedings of the 5th Nordic conference on Human-computer interaction building bridges - NordiCHI ’08, pages 483–486, New York, New York, USA, Oct. 2008. ACM Press. ISBN 9781595937049. doi:10.1145/1463160.1463225. URL http://dl.acm.org/citation.cfm?id=1463160.1463225. → pages 11, 20 [36] K. Kunze, P. Lukowicz, H. Junker, and G. Tr. Where am I : Recognizing On-body Positions of Wearable Sensors. In LOCA05: International Workshop on Location and Context-Awareness, volume 3479, pages 264–275, 2005. → pages 19, 82, 85 [37] L. Lanningham-Foster, T. B. Jensen, R. C. Foster, A. B. Redmond, B. A. Walker, D. Heinz, and J. A. Levine. Energy expenditure of sedentary screen time compared with active screen time for children. Pediatrics, 118(6): 1831–1835, Dec. 2006. ISSN 1098-4275. doi:10.1542/peds.2006-1087. URL http://pediatrics.aappublications.org/cgi/content/abstract/118/6/e1831. → pages 1 [38] J. Lester, B. Hannaford, and G. Borriello. Are You With Me ? Using Accelerometers to Determine if Two Devices are Carried by the Same Person. In In Proceedings of Second International Conference on Pervasive Computing (Pervasive 2004), volume 3001/2004, pages 33–50, 2004. → pages 22 [39] J. Lin, L. Mamykina, S. Lindtner, G. Delajoux, H. Strub, P. Dourish, and A. Friday. FishnSteps: Encouraging Physical Activity with an Interactive Computer Game. In P. Dourish and A. Friday, editors, 8th International Conference on Ubiquitous Computing (Ubicomp 2006), volume 4206 of Lecture Notes in Computer Science, pages 261–278, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-39634-5. doi:10.1007/11853565. URL http://www.springerlink.com/content/vtw615807853xn0v/. → pages 6, 7 [40] N. R. Lomb. Least-squares frequency analysis of unequally spaced data. Astrophysics and Space Science, 39(2):447–462, 1976. ISSN 0004640X. 97 doi:10.1007/BF00648343. URL http://www.springerlink.com/index/GX8721K80P8V878W.pdf. → pages 22 [41] I. MapMyFitness. MapMyRun, 2012. URL http://www.mapmyrun.com/. → pages 6 [42] A. Marti, M. J. Moreno-Aliaga, J. Hebebrand, and J. A. Martı́nez. Genes, lifestyles and obesity. International Journal of Obesity, 28:29–36, Nov. 2004. ISSN 0307-0565. URL http://dx.doi.org/10.1038/sj.ijo.0802808. → pages 1 [43] E. L. Melanson, J. R. Knoll, M. L. Bell, W. T. Donahoo, J. O. Hill, L. J. Nysse, L. Lanningham-Foster, J. C. Peters, and J. A. Levine. Commercially available pedometers: considerations for accurate step counting. Preventive Medicine, 39(2):361–8, 2004. ISSN 00917435. doi:10.1016/j.ypmed.2004.01.032. URL http://www.ncbi.nlm.nih.gov/pubmed/15226047. → pages 12, 13, 28 [44] Merriam-Webster. Merriam-Webster Online Dictionary, 2012. URL http://www.merriam-webster.com/dictionary/digital. → pages 1, 2 [45] Nike Inc. Nike+ Project, 2012. URL http://nikeplus.nike.com/plus/. → pages 2, 6 [46] S. Nishiguchi, M. Yamada, K. Nagai, S. Mori, Y. Kajiwara, T. Sonoda, K. Yoshimura, H. Yoshitomi, H. Ito, K. Okamoto, T. Ito, S. Muto, T. Ishihara, and T. Aoyama. Reliability and validity of gait analysis by android-based smartphone. Telemedicine and e-Health, 18(4):292–6, May 2012. ISSN 1556-3669. doi:10.1089/tmj.2011.0132. URL http://online.liebertpub.com/doi/abs/10.1089/tmj.2011.0132. → pages 13 [47] N. Oliver and F. Flores-Mangas. MPTrain: a mobile, music and physiology-based personal trainer. In M. Nieminen and M. Röykkee, editors, 8th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI’06), pages 21–28. ACM, 2006. ISBN 1595933905. doi:10.1145/1152215.1152221. URL http://portal.acm.org/citation.cfm?id=1152221. → pages 2, 6, 13, 28, 33 [48] E. S. Poole, A. D. Miller, Y. Xu, E. Eiriksdottir, R. Catrambone, and E. D. Mynatt. The place for ubiquitous computing in schools. In Proceedings of the 13th international conference on Ubiquitous computing - UbiComp ’11, pages 395–404, New York, New York, USA, Sept. 2011. ACM Press. ISBN 98 9781450306300. doi:10.1145/2030112.2030165. URL http://dl.acm.org/citation.cfm?id=2030112.2030165. → pages 7 [49] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical recipes in C: The art of scientific computing: Second edition. Cambridge University Press, 1992. ISBN 0521431085. URL http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20\&amp; path=ASIN/0521431085. → pages 22, 23, 53 [50] R. Ross, I. Janssen, J. Dawson, A.-M. Kungl, J. L. Kuk, S. L. Wong, T.-B. Nguyen-Duy, S. Lee, K. Kilpatrick, and R. Hudson. Exercise-induced reduction in obesity and insulin resistance in women: a randomized controlled trial. Obesity Research, 12(5):789–98, May 2004. ISSN 1071-7323. doi:10.1038/oby.2004.95. URL http://dx.doi.org/10.1038/oby.2004.95. → pages 1 [51] J. D. Scargle. Studies in astronomical time series analysis. II - Statistical aspects of spectral analysis of unevenly spaced data. The Astrophysical Journal, 263(2):835–853, 1982. ISSN 0004637X. doi:10.1086/160554. URL http://articles.adsabs.harvard.edu/full/1982ApJ...263..835S. → pages 22 [52] J. Scott, D. Dearman, K. Yatani, and K. N. Truong. Sensing foot gestures from the pocket. In Proceedings of the 23nd annual ACM symposium on User interface software and technology - UIST ’10, pages 199–208, New York, New York, USA, Oct. 2010. ACM Press. ISBN 9781450302715. doi:10.1145/1866029.1866063. URL http://dl.acm.org/citation.cfm?id=1866029.1866063. → pages 12 [53] J. Sinclair, P. Hingston, and M. Masek. Considerations for the design of exergames. In Proceedings of the 5th international conference on Computer graphics and interactive techniques in Australia and Southeast Asia - GRAPHITE ’07, pages 289–296, New York, New York, USA, Dec. 2007. ACM Press. ISBN 9781595939128. doi:10.1145/1321261.1321313. URL http://dl.acm.org/citation.cfm?id=1321261.1321313. → pages 8, 9 [54] S. Sprager and D. Zazula. A cumulant-based method for gait identification using accelerometer data with principal component analysis and support vector machine. WSEAS Transactions on Signal Processing, 5(11):369–378, Nov. 2009. ISSN 1790-5022. URL http://dl.acm.org/citation.cfm?id=1853844.1853846http: //dl.acm.org/citation.cfm?id=1736242.1736261. → pages 18 99 [55] T. Stach, T. C. N. Graham, M. Brehmer, and A. Hollatz. Classifying Input for Active Games. In Proceedings of the International Conference on Advances in Computer Enterntainment Technology - ACE ’09, page 379, New York, New York, USA, Oct. 2009. ACM Press. ISBN 9781605588643. doi:10.1145/1690388.1690465. URL http://dl.acm.org/citation.cfm?id=1690388.1690465. → pages 10 [56] T. Stach, T. C. N. Graham, J. Yim, and R. E. Rhodes. Heart Rate Control of Exercise Video Games. In GI ’09 Proceedings of Graphics Interface 2009, pages 125–132. Canadian Information Processing Society, 2009. ISBN 9781568814704. URL http://portal.acm.org/citation.cfm?id=1555912\&dl= GUIDE\&coll=GUIDE\&CFID=102065872\&CFTOKEN=69748589. → pages 8, 9 [57] K. G. Stanley, I. Livingston, A. Bandurka, R. Kapiszka, and R. L. Mandryk. PiNiZoRo : A GPS-based Exercise Game for Families. In Futureplay ’10 Proceedings of the International Academic Conference on the Future of Game Design and Technology, pages 243–246, 2010. → pages 9 [58] Statistics Canada. CANSIM Table 105-0501 and Catalogue No. 82-221-X, 2011. URL http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/ health81a-eng.htm. → pages 1 [59] M. Tamviruzzaman, S. I. Ahamed, C. S. Hasan, and C. O’brien. ePet: When Cellular Phone Learns to Recognize Its Owner. In SafeConfig ’09: Proceedings of the 2nd ACM workshop on Assurable and usable security configuration, pages 13–18, New York, New York, USA, Nov. 2009. ACM Press. ISBN 9781605587783. doi:10.1145/1655062.1655066. URL http://dl.acm.org/citation.cfm?id=1655062.1655066. → pages 11, 18 [60] C. Tudor-Locke. Taking Steps toward Increased Physical Activity: Using Pedometers To Measure and Motivate. President’s Council on Physical Fitness and Sports Research Digest, 17(3):3–10, 2002. URL http://eric.ed.gov/ERICWebPortal/search/detailmini.jsp? nfpb=true\& % &ERICExtSearch SearchValue 0=ED470689\ &ERICExtSearch SearchType 0=no\&accno=ED470689. → pages 6 [61] R. Waters, R. Waters, D. Anderson, J. Barrus, D. Brogan, M. Casey, S. McKeown, T. Nitta, I. Sterns, and W. Yerazunis. Diamond Park and Spline: A Social Virtual Reality System with 3D Animation, Spoken Interaction, and Runtime Modifiability. Presence: Teleoperators and Virtual Environments, 6(4):461–480, 1997. URL 100 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.2.9500. → pages 8 [62] S. L. Wong and S. T. Leatherdale. Association between sedentary behavior, physical activity, and obesity: inactivity among active kids. Preventing Chronic Disease, 6(1):1–13, Jan. 2009. ISSN 1545-1151. URL http://ukpmc.ac.uk/articles/PMC2644601. → pages 1 [63] World Health Organization. WHO — Obesity and overweight, 2012. URL http://www.who.int/mediacentre/factsheets/fs311/en/index.html. → pages 1 [64] World Health Organization. WHO — Physical Inactivity: A Global Public Health Problem, 2012. URL http://www.who.int/dietphysicalactivity/factsheet inactivity/en/index.html. → pages 1 [65] C. Worringham, A. Rojek, and I. Stewart. Development and Feasibility of a Smartphone, ECG and GPS Based System for Remotely Monitoring Exercise in Cardiac Rehabilitation. PLoS ONE, 6(2):1–6, 2011. URL http://dx.plos.org/10.1371/journal.pone.0014669. → pages 11 [66] G. Wylie, C. Miguel, and P. Coulton. Mobile persuasive exergaming. In ACE ’08 Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, pages 338–341, 2008. URL http://dx.doi.org/10.1109/ICEGIC.2009.5293582. → pages 9 [67] C.-C. Yang, Y.-L. Hsu, K.-S. Shih, and J.-M. Lu. Real-time gait cycle parameter recognition using a wearable accelerometry system. Sensors, 11 (8):7314–7326, Jan. 2011. ISSN 1424-8220. doi:10.3390/s110807314. URL http://www.mdpi.com/1424-8220/11/8/7314. → pages 33, 36 [68] J. Yang. Toward Physical Activity Diary: Motion Recognition Using Simple Acceleration Features with Mobile Phones. In Proceedings of the 1st international workshop on Interactive multimedia for consumer electronics - IMCE ’09, pages 1–10, New York, New York, USA, Oct. 2009. ACM Press. ISBN 9781605587585. doi:10.1145/1631040.1631042. URL http://dl.acm.org/citation.cfm?id=1631040.1631042. → pages 15, 17, 52 [69] S. Yang. Defining Exergames & Exergaming. In Proceedings of Meaningful Play 2010, pages 1–17, 2010. URL http://meaningfulplay.msu.edu/proceedings2010/mp2010 paper 63.pdf. → pages 8 101 [70] S. Zhang, P. Mccullagh, C. Nugent, and H. Zheng. Activity Monitoring Using a Smart Phones Accelerometer with Hierarchical Classification. In 6th International Conference on Intelligent Environments, pages 158–163, 2010. → pages 15, 17, 85 [71] H. R. Zheng, C. Nugent, P. McCullagh, Y. Huang, S. M. Zhang, W. Burns, R. Davies, N. Black, P. Wright, S. Mawson, C. Eccleston, M. Hawley, and G. Mountain. Smart self management: assistive technology to support people with chronic disease. Journal of Telemedicine and Telecare, 16(4): 224–227, 2010. URL http://dx.doi.org/10.1258/jtt.2010.004017. → pages 11 102 Appendix A Supporting Materials A.1 Cadence Estimation Study Forms A.1.1 Consent Forms 103 Version 2.1 / August 17, 2011 / Page 1 of 2  PARTICIPANT’S COPY CONSENT FORM Department of Computer Science 2366 Main Mall Vancouver, B.C.  Canada  V6T 1Z4 tel:   (604) 822-3061 fax:  (604) 822-4231 Project Title: Gait Measurement Algorithm Verification  (UBC Ethics #H01-80470) Principal Investigator: Karon MacLean, Professor, Dept. of Computer Science, 604-822-8169 Co-Investigator: Idin Karuei, PhD Candidate, Dept. of Computer Science, 604-827-3982 Oliver Schneider, M.Sc. Student, Dept. of Computer Science Michelle Chuang, B.Sc. Student, Dept. of Computer Science   The purpose of this experiment is to test the gait measurement algorithm that we have developed on android phones. In this experiment, you will be asked to walk on a sidewalk at different speeds ranging from very slow to brisk walking.  You will be asked to carry android phones in your front and back pocket and a backpack provided by us, attached to your arm and belt, and held in one of your hands.  REIMBURSEMENT: We are very grateful for your participation. However, you will not receive compensation of any kind for participating in this project.  TIME COMMITMENT: 1 × 30 minute session CONFIDENTIALITY: You will not be identified by name in any study reports. Data gathered from this experiment will be stored in a secure Computer Science account accessible only to the experimenters.  You understand that the experimenters will ANSWER ANY QUESTIONS you have about the instructions or the procedures of this study. After participating, the experimenter will answer any other questions you have about this study. Your participation in this study is entirely voluntary and you may refuse to participate or withdraw from the study at any time without jeopardy. Your signature below indicates that you have received a copy of this consent form for your own records, and consent to participate in this study.  If you have any concerns about your treatment or rights as a research subject, you may contact the Research Subject Info Line in the UBC Office of Research Services at 604-822-8598. 104 Version 2.1 / August 17, 2011 / Page 2 of 2  RESEARCHER’S COPY CONSENT FORM Department of Computer Science 2366 Main Mall Vancouver, B.C.  Canada  V6T 1Z4 tel:   (604) 822-3061 fax:  (604) 822-4231 Project Title: Gait Measurement Algorithm Verification  (UBC Ethics #H01-80470) Principal Investigator: Karon MacLean, Professor, Dept. of Computer Science, 604-822-8169 Co-Investigator: Idin Karuei, PhD Candidate, Dept. of Computer Science, 604-827-3982 Oliver Schneider, M.Sc. Student, Dept. of Computer Science Michelle Chuang, B.Sc. Student, Dept. of Computer Science  The purpose of this experiment is to test the gait measurement algorithm that we have developed on android phones. In this experiment, you will be asked to walk on a sidewalk at different speeds ranging from very slow to brisk walking. You will be asked to carry android phones in your front and back pocket and a backpack provided by us, attached to your arm and belt, and held in one of your hands.  REIMBURSEMENT: We are very grateful for your participation. However, you will not receive compensation of any kind for participating in this project.  TIME COMMITMENT: 1 × 30 minute session CONFIDENTIALITY: You will not be identified by name in any study reports. Data gathered from this experiment will be stored in a secure Computer Science account accessible only to the experimenters.  You understand that the experimenters will ANSWER ANY QUESTIONS you have about the instructions or the procedures of this study. After participating, the experimenter will answer any other questions you have about this study. Your participation in this study is entirely voluntary and you may refuse to participate or withdraw from the study at any time without jeopardy. Your signature below indicates that you have received a copy of this consent form for your own records, and consent to participate in this study.  If you have any concerns about your treatment or rights as a research subject, you may contact the Research Subject Info Line in the UBC Office of Research Services at 604-822-8598.  You hereby CONSENT to participate and acknowledge RECEIPT of a copy of the consent form: PRINTED NAME ________________________________ DATE ____________________________ SIGNATURE ____________________________________ 105 A.2 Gait Classification Study Forms A.2.1 Consent Forms 106 Version 1.0 / December 16, 2011 / Page 1 of 2  PARTICIPANT’S COPY CONSENT FORM Department of Computer Science 2366 Main Mall Vancouver, B.C.  Canada  V6T 1Z4 tel:   (604) 822-3061 fax:  (604) 822-4231 Project Title: Gait Classification Algorithm Development  (UBC Ethics #H01-80470) Principal Investigator: Karon MacLean, Professor, Dept. of Computer Science, 604-822-8169 Co-Investigators: Oliver Schneider, M.Sc. Student, Dept. of Computer Science         Idin Karuei, PhD Candidate, Dept. of Computer Science, 604-827-3982  The purpose of this experiment is to gather data for a gait classification algorithm that we are developing on android phones. In this experiment, you will be invited to perform several gaits, such as walking, jogging, running, ascending or descending stairs, or riding a bicycle or bus. If uncomfortable with any requested gait(s) or activities, you may choose to omit it/them and it will not affect your reimbursement. You will be asked to carry several android phones in these locations, which we will be using to measure parameters of your motion: in your front and back pocket, in a backpack provided by us, attached to your arm and belt, and held in one of your hands when safe to do so.  REIMBURSEMENT: We are very grateful for your participation. You will receive monetary compensation of $10 for this session.  TIME COMMITMENT: 1 × 45 minute session CONFIDENTIALITY: You will not be identified by name in any study reports. Data gathered from this experiment will be stored in a secure Computer Science account accessible only to the experimenters.  You understand that the experimenter will ANSWER ANY QUESTIONS you have about the instructions or the procedures of this study. After participating, the experimenter will answer any other questions you have about this study. Your participation in this study is entirely voluntary and you may refuse to participate or withdraw from the study at any time without jeopardy. Your signature below indicates that you have received a copy of this consent form for your own records, and consent to participate in this study.  If you have any concerns about your treatment or rights as a research subject, you may contact the Research Subject Info Line in the UBC Office of Research Services at 604-822-8598. 107 Version 1.0 / December 16, 2011 / Page 2 of 2  RESEARCHER’S COPY CONSENT FORM Department of Computer Science 2366 Main Mall Vancouver, B.C.  Canada  V6T 1Z4 tel:   (604) 822-3061 fax:  (604) 822-4231 Project Title: Gait Classification Algorithm Development  (UBC Ethics #H01-80470) Principal Investigator: Karon MacLean, Professor, Dept. of Computer Science, 604-822-8169 Co-Investigators: Oliver Schneider, M.Sc. Student, Dept. of Computer Science         Idin Karuei, PhD Candidate, Dept. of Computer Science, 604-827-3982  The purpose of this experiment is to gather data for a gait classification algorithm that we are developing on android phones. In this experiment, you will be invited to perform several gaits, such as walking, jogging, running, ascending or descending stairs, or riding a bicycle or bus. If uncomfortable with any requested gait(s) or activities, you may choose to omit it/them and it will not affect your reimbursement. You will be asked to carry several android phones in these locations, which we will be using to measure parameters of your motion: in your front and back pocket, in a backpack provided by us, attached to your arm and belt, and held in one of your hands when safe to do so.  REIMBURSEMENT: We are very grateful for your participation. You will receive monetary compensation of $10 for this session.  TIME COMMITMENT: 1 × 45 minute session CONFIDENTIALITY: You will not be identified by name in any study reports. Data gathered from this experiment will be stored in a secure Computer Science account accessible only to the experimenters.  You understand that the experimenter will ANSWER ANY QUESTIONS you have about the instructions or the procedures of this study. After participating, the experimenter will answer any other questions you have about this study. Your participation in this study is entirely voluntary and you may refuse to participate or withdraw from the study at any time without jeopardy. Your signature below indicates that you have received a copy of this consent form for your own records, and consent to participate in this study.  If you have any concerns about your treatment or rights as a research subject, you may contact the Research Subject Info Line in the UBC Office of Research Services at 604-822-8598.  You hereby CONSENT to participate and acknowledge RECEIPT of a copy of the consent form: PRINTED NAME ________________________________ DATE ____________________________ SIGNATURE ____________________________________ 108 A.2.2 Recruitment Email 109 T H E  U N I V E R S I T Y  O F  B R I T I S H  C O L U M B I A  Department of Computer Science 201-2366 Main Mall Vancouver, B.C.  Canada  V6T 1Z4 tel:   (604) 822-3061 fax:   (604) 822-4231  Human Gait Classification with Smart Phones Principal Investigator: Karon MacLean, Professor, Dept. of Computer Science, 604-822-8169 Co-Investigator: Oliver Schneider, MSc Student, Dept. of Computer Science         Idin Karuei, PhD Candidate, Dept. of Computer Science, 604-827-3982 Version 1.0 / 22 December, 2011  The following message will be used to recruit participants for our study. We will distribute this message using either or both of the following methods: • Emailing the recruitment message to mailing lists maintained by the Computer Science department or our research group, such as a list of department graduate students (often used for this kind of purpose) and a list of persons who have expressed an interest in being study participants. • Uploading the recruitment message to an online forum posting, such as craigslist.ca.  From: Oliver Schneider and Idin Karuei Subject: Call for Study Participants - $10 for Human Gait Classification with Smart Phones  The SPIN Research Group in the UBC Dept. of Computer Science is looking for participants for a study recognizing user gait. You will be compensated $10 for your participation in a single 45-minute session.  We will place Android phones in your front and back pocket, a bag or knapsack, on your hip, on your arm, and in your hand, and measure parameters of your movement during several types of typical activities. You will be asked to move freely outdoors in a safe place, such as an otherwise unused running track, and asked to do several different gaits for a few minutes each, such as walking, jogging, running, ascending and descending stairs. You may be asked to ride a bicycle (safety equipment will be provided) or a bus.  Please visit <URL> to sign-up for the experiment. You can contact me if you have any questions.  Oliver Schneider MSc Student, UBC Computer Science oschneid@cs.ubc.ca   110 A.2.3 Demographic Information Form 111 	
   	
   	
   Date:	
   	
   _______________	
   	
   	
   	
   	
  	
  ID:	
   	
  	
   _______________	
   	
   	
   	
   Checklist:	
   Frontpocket	
  	
  Height:	
  	
   _______________	
   	
   	
   	
   	
   	
   Backpocket	
  	
  Weight:	
   _______________	
   	
   	
   	
   	
   	
   Backpack	
  	
  Age:	
   	
   _______________	
   	
   	
   	
   	
   	
   Belt	
  	
  Sex:	
   	
   _______________	
   	
   	
   	
   	
   	
   Arm	
  	
  Dominant	
  Hand:	
   _______________	
   	
   	
   	
   	
   Hand	
  	
  Order:	
  	
   A	
  (stairs	
  first,	
  up	
  then	
  down)	
   	
   	
   Water	
  	
   	
   	
  B	
  (stairs	
  first,	
  down	
  then	
  up)	
   	
   	
   Whistle/Stopwatch	
  	
   	
   	
  C	
  (stairs	
  last,	
  up	
  then	
  down)	
  	
   	
   	
  D	
  (stairs	
  last,	
  down	
  then	
  up)	
  	
  	
  	
  	
  Date:	
   	
   _______________	
   	
   	
   	
   	
  	
  ID:	
   	
  	
   _______________	
   	
   	
   	
   Checklist:	
   Frontpocket	
  	
  Height:	
  	
   _______________	
   	
   	
   	
   	
   	
   Backpocket	
  	
  Weight:	
   _______________	
   	
   	
   	
   	
   	
   Backpack	
  	
  Age:	
   	
   _______________	
   	
   	
   	
   	
   	
   Belt	
  	
  Sex:	
   	
   _______________	
   	
   	
   	
   	
   	
   Arm	
  	
  Dominant	
  Hand:	
   _______________	
   	
   	
   	
   	
   Hand	
  	
  Order:	
  	
   A	
  (stairs	
  first,	
  up	
  then	
  down)	
   	
   	
   Water	
  	
   	
   	
  B	
  (stairs	
  first,	
  down	
  then	
  up)	
   	
   	
   Whistle/Stopwatch	
  	
   	
   	
  C	
  (stairs	
  last,	
  up	
  then	
  down)	
  	
   	
   	
  D	
  (stairs	
  last,	
  down	
  then	
  up)	
   112 A.3 Other Forms A.3.1 Participant Reimbursement Form 113 T H E  U N I V E R S I T Y  O F  B R I T I S H  C O L U M B I A Page 1 of 1 revised 7/23/2012  Experiment Name: _____________________ BREB Ethics Approval # ________________    Date Name Number Paid Signature Email                   114

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
Germany 23 9
China 9 0
United States 7 6
France 5 1
Canada 5 4
Argentina 2 0
Austria 2 0
Tanzania 1 0
India 1 0
City Views Downloads
Unknown 32 18
Beijing 9 0
Ashburn 4 0
Toronto 3 3
Cupertino 2 0
Vancouver 2 1
Buenos Aires 2 0
Redmond 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}

Share

Share to:

Comment

Related Items