A MIXED-METHODS STUDY COMPARING SKILL PERFORMANCE BETWEEN INDOOR AND OUTDOOR ENVIRONMENTS AMONG EXPERIENCED SCOOTER USERS by Sharon Liana Jang B.Kin., The University of British Columbia, 2015 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Rehabilitation Sciences) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) October 2019 © Sharon Liana Jang 2019 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, a thesis entitled: A mixed-methods study comparing skill performance between indoor and outdoor environments among experienced scooter users submitted by Sharon Liana Jang in partial fulfillment of the requirements for the degree of Master of Science in Rehabilitation Sciences Examining Committee: Ben Mortenson, Occupational Sciences and Occupational Therapy Supervisor Bill Miller, Occupational Sciences and Occupational Therapy Supervisory Committee Member Laura Hurd Clarke, School of Kinesiology Supervisory Committee Member Bonita Sawatsky, Department of Orthopaedics Additional Examiner iii ABSTRACT Background: The Wheelchair Skills Test (WST) has been validated to assess scooter users’ skill performance and mobility confidence. Traditionally, the WST has been performed using a standardized indoor course; however, it has been suggested that an outdoor community-based setting may be a suitable alternative. However, no previous research has compared scooter skill performance in an indoor versus outdoor environment. Objectives: 1) To explore participants’ experiences with scooter use, 2) to determine the distribution of WST scores and how scores compare between indoor and outdoor environments, 3) to determine correlations between the indoor lab-based WST scores, the outdoor community-based WST scores, and the WST-Q scores, and explore participants’ perspectives on the representativeness of the WST, testing preferences, and suggestions for improvement, and 4) to determine the practicality of performing the WST in an outdoor community-based setting.. Methods: For this mixed-methods study, 20 scooter users who have used their devices for ≥ 3 months were recruited. Each participant was randomized to complete the WST twice – once in their community and once indoors within a two-week period. While testing in the community, detailed observations were made of the setting and the 28 representative skills on the WST (e.g., curbs, hills). Semi-structured interviews were conducted after completion of the WST in both environments. Results: Indoor and outdoor scores were not strongly correlated (r= 0.306, p=0.190) and demonstrated wide limits of agreement. The outdoor WST scores were weakly correlated with subjective capacity, confidence, performance, and mobility; however, this may have been related to a low variation in scores. When searching for WST obstacles in the community, the majority were easily found. . Whereas most participants preferred performing the WST in their community due to convenience and familiarity, they also perceived the indoor course as reflective of their community setting. iv Conclusion: These findings suggest that skills testing in the community is feasible; however, skills testing in indoor and outdoor settings are not comparable. Community-based testing may provide a better reflection of day to day performance of experienced users, but may not reflect user’s capacity in novel environments. v LAY SUMMARY Training is recommend to improve mobility scooter safety. The Wheelchair Skills Test (WST) is one of the most validated skills test and is intended for use in an indoor clinical environment, or an outdoor community-based setting. However, WST scores in both setting have not previously been compared. Therefore, this study aimed to 1) compare performance in a standardized indoor environment to a community-based outdoor environment; 2) explore participants’ perceptions of skills testing in both settings and 3) determine the practicality of community administration. It was found that the scores were not comparable between settings, but participants thought the indoor skills course reflected the outdoor environment well. The majority of WST obstacles were easily found in the community. Community-based testing may be a better indication of day-to-day skill performance, but the indoor setting may be useful to assess how people may perform in novel environments among experienced users. vi PREFACE The conception of this study was based on discussions with my supervisor Dr. Ben Mortenson on performance of the WST among scooter users, which is a relatively new topic. The study design was developed by myself, with guidance and input from Dr. Ben Mortenson, Dr. Laura Hurd Clarke, and Dr. Bill Miller. All data collection and analysis was conducted by myself, with consultation from my supervisor. This study was approved by the University of British Columbia’s Research Ethics Board [certificate #H17-01763]. At the present time, the work presented in this thesis has not been submitted for any publications. vii TABLE OF CONTENTS ABSTRACT ....................................................................................................................................... iii LAY SUMMARY ................................................................................................................................ v PREFACE .......................................................................................................................................... vi TABLE OF CONTENTS ................................................................................................................ vii LIST OF TABLES .............................................................................................................................. x LIST OF FIGURES .......................................................................................................................... xi ACKNOWLEDGEMENTS ............................................................................................................ xii CHAPTER 1: INTRODUCTION .................................................................................................... 1 1.1 Scooter use .................................................................................................................. 1 1.2 Importance of wheelchair skills ................................................................................... 3 1.3 Wheelchair Skills Test ................................................................................................. 4 1.4 Validity ....................................................................................................................... 7 1.4.1 Messick’s Sources of Validity..................................................................................... 8 1.4.2 Threats to validity ..................................................................................................... 10 1.5 Relevance of the environment in testing..................................................................... 11 1.6 Applicability of a measure ............................................................................................... 12 1.7 Theoretical Considerations .............................................................................................. 12 Objectives and Hypothesis ..............................................................................................................15 CHAPTER 2: METHODOLOGY .................................................................................................16 2.1 Design ............................................................................................................................. 16 2.2 Participants...................................................................................................................... 18 2.2.1. Inclusion and exclusion criteria ................................................................................ 18 2.2.2 Sample size .............................................................................................................. 18 2.3 Recruitment ..................................................................................................................... 19 2.4 Quantitative Methods ...................................................................................................... 19 2.4.1 Outcome measures .................................................................................................... 19 2.4.2 Procedure ................................................................................................................. 21 2.4.3 Statistical Analyses ................................................................................................... 23 2.5 Qualitative Methods ........................................................................................................ 24 2.5.1 Interviews ................................................................................................................. 24 2.5.2 Methods.................................................................................................................... 26 2.5.2 Trustworthiness Strategies ........................................................................................ 29 2.5.3 Reflexivity ................................................................................................................ 30 CHAPTER 3: RESULTS .................................................................................................................32 3.1 Demographics ................................................................................................................. 32 3.1.2 Average participant scores on outcome measures.......................................................... 34 3.2 Determining how scores on the indoor course compare to scores on the outdoor community-based setting (objective two) .............................................................................. 36 3.2.1 Comparing scores between environments (objective two) ......................................... 38 viii 3.2.2 Observing the correlation and association between indoor lab-based and outdoor community-based WST performance (objective two) ........................................................ 38 3.2.3. Observing the agreement between scores on the indoor lab-based versus outdoor community-based environment (objective two) ................................................................. 39 3.3 Participants’ experiences with scooter use (objective two) ............................................... 40 3.4 Correlations with other measures (objective three) .......................................................... 43 3.4.1. Determining the relationship of the outdoor community-based WST scores with other measures: the external aspect of validity (objective 3.1) .................................................... 43 3.4.2 Participant perspective on the representativeness of the WST items (objective 3.2) ... 46 3.4.3. Participants’ comparisons of the indoor lab-based versus outdoor community-based environment (objective 3.2) ............................................................................................... 47 3.4.4. Participants’ perspectives on confidence (objective 3.3)........................................... 49 3.5 Determining the applicability of performing the WST outdoors (objective four) .............. 51 3.5.1 Participants’ perception on whether certain skills are required on the WST (objective four) .................................................................................................................................. 53 3.5.2 Participants’ perception on safety (objective four) .................................................... 54 3.5.3 The perceived benefits of performing the WST (objective four) ................................ 55 CHAPTER 4: DISCUSSION ..........................................................................................................56 4.1 Comparing performance between the indoor lab-based and outdoor community-based environments......................................................................................................................... 57 4.2 Participants’ experiences with scooter use ....................................................................... 60 4.3 Messick’s Evidence for Validity ...................................................................................... 62 4.3.1 Analysis from the Spearman Correlation ................................................................... 62 4.3.2 Participants’ perspectives on the representativeness of the WST ............................... 64 4.3.3 Participants’ sense of confidence .............................................................................. 64 4.4 Determining the applicability of the WST in an outdoor community-based setting .......... 67 4.5 Personal Reflection ......................................................................................................... 70 CHAPTER 5. CONCLUSION ........................................................................................................71 5.1 Limitations and Future directions .................................................................................... 71 5.2 Conclusion ...................................................................................................................... 73 5.2 Relevance and Application .............................................................................................. 73 References ..........................................................................................................................................75 Appendices .........................................................................................................................................83 Appendix 1. Literature search on the Wheelchair Skills Test ................................................. 83 Appendix 2: Reporting Guidelines ........................................................................................ 91 2.1. STROBE .................................................................................................................... 91 2.2 COREQ ....................................................................................................................... 93 2.3 GRAMMS ................................................................................................................... 97 Appendix 3: Measures ........................................................................................................... 98 Demographics ................................................................................................................... 98 Wheelchair Skills Test ..................................................................................................... 101 Life Space Assessment .................................................................................................... 103 Appendix 4: Outdoor Environment Measures and Notes ..................................................... 105 Appendix 5: Interview guide ............................................................................................... 107 Appendix 6: Sample of powerpoint slides shown to participants during the interview ......... 109 ix Appendix 7: List of Codes ................................................................................................... 110 The perceived benefits of performing the WST (objective four) ...................................... 112 Appendix 8: Additional Information on Clinimetrics ........................................................... 113 x LIST OF TABLES Table 1. 1 Reliability values for the WST version 4.1 reported in the literature .............................. 6 Table 2. 1 Participant pseudonyms and device use………………………………………………20 Table 3. 1 Participant Demographics.................................................................................................32 Table 3. 2 Participants’ scooter measurements .................................................................................33 Table 3. 3 Participant mean scores on outcome measures ...............................................................34 Table 3. 4 Number of skills performed by each participant on the WST ........................................35 Table 3. 5 Tally of reported daily activities performed with a scooter ...........................................40 Table 3. 6 Correlation of WST performance and confidence with other measures........................45 Table 3. 7 Suggested WST Improvements (objective 3) ..................................................................51 Table 3. 8 Number of participants in each city in the Lower Mainland ..........................................51 Table 3. 9 Environmental measures ...................................................................................................52 xi LIST OF FIGURES Figure 3. 1 Distribution of outdoor community-based WST scores ................................................36 Figure 3. 2 Distribution of indoor lab-based WST scores ................................................................37 Figure 3. 3 Distribution graph for the total WST-Q scores ..............................................................37 Figure 3. 4 Correlation between performance scores of the WST outside and the WST inside....39 Figure 3. 5 Bland-Altman Plot ...........................................................................................................40 xii ACKNOWLEDGEMENTS First and foremost, I would like to acknowledge my thesis supervisor, Dr. Ben Mortenson, for all of his continued support and guidance throughout this project. All of your advice and help has truly helped shaped my development as a researcher and has helped me realize my passion of qualitative research. Thank you for all you have taught me! In addition, I would like to thank my committee members, Dr. Laura Hurd Clarke and Dr. Bill Miller, for all of your suggestions and insights on this project. I would also like to acknowledge all the participants who helped out with this study – this project would not be possible without you! Thank you for your time and eagerness to help me with this study. To all the past and present members of the Mortenson Lab, thank you for all of your help and emotional support! Special thanks to Jessica Chow, for accompanying me on my assessments and for your encouragement; to Rosemary Cheung, for all of your help with accessing files on the drive when I was off site and for all your emotional support; and to the research coordinators, Leena Chau and Jodine Perkins, for all your administrative assistance and help with dealing with UBC! To my family and friends, thank you for helping me stay sane through my journey through grad school. To my mom and my sister, I am grateful for all of your unwavering love and support. Special thanks to Kevin, for your support and constant reminders of “you can do this!” and “you’re almost there.” 1 CHAPTER 1: INTRODUCTION 1.1 Scooter use According to the 2012 Canadian Survey of Disability (Statistics Canada, 2012), there are 1.9 million people in Canada with some form of mobility impairment. To facilitate mobility, 80% of these individuals rely on an assistive device (e.g., manual wheelchairs, powered wheelchairs, scooters, and walkers) (Smith et al., 2016; Statistics Canada, 2012). Although there are a variety of assistive devices to choose from, individuals who are still able to walk may turn to scooters as they facilitate long distance travel and require less physical effort for those with greater mobility limitations (Edwards and McCluskey, 2010; Smith et al., 2016). In 2013, there was an estimated 108,550 scooter users in Canada (Smith et al., 2016). Given that these data were from 2013, Canada’s population is aging (Statistics Canada, 2017), and the increased acceptability of mobility aids, the number of individuals who use scooters is expected to increase. Scooters are defined in this study as three- or four-wheeled devices that are battery powered and controlled with a tiller. Scooters can vary in regard to the number of wheels, the type of drive, and the weight and size. Traditional scooters are three-wheeled, four-wheeled, or hybrid four-wheeled. Three wheeled scooters are generally rear-wheel drive and are more suited for indoor or tight spaces as they are highly maneuverable (Dutta et al., 2010); however, these scooters are subject to poor stability. Four-wheeled scooters are generally larger than three-wheeled scooters, and are available in front wheel drive, rear wheel drive, and all-wheel drive. They are better suited for outdoor use or use on uneven terrain and provide good stability. However, their larger size makes them less maneuverable in small spaces (Dutta et al., 2010). Hybrid four-wheeled scooters are also available for purchase. These scooters have two front 2 wheels that are located close together, mimicking a three-wheeled scooter. The hybrid four-wheeled scooter thus provides greater stability than a three-wheeled scooter, and more maneuverability than a four-wheeled scooter. Travel scooters are a light-weight three-wheeled scooter. They are characterized by a triangular based composed of three metal tubes, a non-adjustable seat, and a tiller. Travel scooters are lighter than traditional scooters (e.g., 35 lbs vs 78-180 lbs), and are easily folded and transported. However, in order to achieve the light weight and transportability, some features have been removed from or altered on the scooter. First, there is no internal battery charger – the battery must be removed from the scooter in order to be charged, opposed to inserting a plug into the scooter. Secondly, speed is less precise on a travel scooter, as it is controlled with a throttle (versus a lever handles found on the traditional scooters). Thirdly, the scooter is controlled by a motor located in one of the three wheels (opposed to having a rear-wheel drive, whereby both wheels are powered). Lastly, the speed at which travel scooters are able to attain are lower than the traditional scooter. The prevalence of scooters may be related to the multiple reported benefits that have been associated with their use. Four cross-sectional surveys (ACCC et al., 2012; Brandt et al., 2004; Edwards and McCluskey, 2010; Samuelsson and Wressle, 2014) were identified in a scoping review (Mortenson and Kim, 2016) which found that scooter use is associated with an increased sense of independence and freedom, mobility, and quality of life for users. These associations may be related to the scooter users’ increased ability to move around the community (e.g., increased number of trips taken outside, decreased assistance required, and reduced barriers to participation) (Edwards and McCluskey, 2010; Petterson et al., 2006; Samuelsson and Wressle, 2014). 3 1.2 Importance of wheelchair skills Although scooter users have reported many positive benefits with the use of these devices, accidents are a concern. Among scooter users, the reported average accident rates vary from 1.54 (Hoenig et al., 2007 to 15 (Australian Competition and Consumer Commission (ACCC) et al., 2012) accidents per person per year (Mortenson and Kim, 2016). These accidents include bumping into other objects and other people, or tipping over and falling. Scooter skills has been recommended for safety (Mortenson et al., 2005), as a relationship between skill and minor incidents has been identified (Formiatti et al., 2013). Currently, formal training is rarely provided to scooter user, as only a quarter of scooter users reported receiving training in two studies (ACCC et al., 2012). Given the benefits of skills training, methods to improve and promote training are required. While there is a dearth of literature on relationships among skill level and confidence among scooter users, there is some evidence among the parallel literature from powered wheelchairs. Relationships among scooter skills and confidence and participation have been identified. Among powered wheelchair users, skill level has been associated with confidence (Rushton et al., 2014), which in turn has the potential to impact community participation (Sakakibara et al., 2013; Sakakibara et al., 2014) through the possibility of improving mobility and reduced participation limitation (Barker et al, 2006). Furthermore, high levels of skills have been identified as an independent predictor of participation (Hosseini et al., 2012); skills may have both a direct and indirect impact on participation frequency (Mortenson et al., 2012). Scooter skills may have an indirect impact on participation through increasing mobility; users are given the independence to travel to places that they would not be able to normally access without their scooters (Mortenson et al., 2012; Smith et al., 2014). This is supported by the finding that 4 power mobility skills have been statistically significantly associated with leisure time physical activity (Phang et al., 2012). 1.3 Wheelchair Skills Test The WST is one of the most validated and widely used wheelchair skills tests (Kilkens et al., 2003; Mortenson et al., 2008). The WST is an outcome measure that was developed in 1996 to assess the ability of wheelchair users to safely perform skills required in their everyday lives (Kirby et al., 2015). Since its conception, the WST has gone through six versions, each addressing feedback and including changes: versions 1.0, 2.4, 3.4, 4.1, 4.2 and 4.3. In version 4.3, different forms have been developed for different mobility devices: manual wheelchair users, power wheelchair users, and scooter users. However, a new version of the WST has been released after the start of this study, which includes one form that may be applied to all types of wheeled mobility devices. The WST may be conducted objectively (i.e., individuals are evaluated by a rater) or subjectively with the WST – Questionnaire (WST-Q) (i.e., individuals rate themselves). Three studies have found that objective and subjective performance scores are highly correlated among manual wheelchair users (r = 0.91 (Mountain et al., 2004); r = 0.95 (Newton et al., 2002)) and powered wheelchair users (r = 0.89 (Rushton et al., 2012)), and that subjective scores are statistically significantly higher than objective scores among manual and powered wheelchair users (Mountain et al., 2004; Newton et al., 2002; Rushton et al., 2012). The over estimation of scores on the WST-Q was generally on the more advanced skills. The differences between the WST-Q and WST scores may reflect differences between performance anxiety of being tested in the lab versus being tested in a familiar environment (i.e., the community) (Kirby et al., 2015). 5 Due to the over-estimation of skills among scooter users, Rushton et al. (2012) suggest that WST-Q scores should be interpreted with caution. The WST 4.1 incorporates aspects of performance and safety into the score and evaluates various skills. The WST assesses 29 skills for scooter users. Skills included in the test vary from basic skills (e.g., turning on/off the power, adjusting seating position, moving forwards in a straight line), to more advanced skills (e.g., going up/down curbs, going over pot holes, etc.). The WST is usually performed using a standardized indoor obstacle course that incorporates prescribed ramps, curbs, gaps, and soft surfaces (i.e., gravel). According to the WST Manual (Kirby et al., 2015), the test setting for the WST should be “reasonably quiet, private, free from distractions, and well lit” (p. 8); however, it is also noted that comparable challenges in the existing natural or built environment may be used. My review of the extant literature indicates that no study has explicitly evaluated how the setting may influence performance of and on the WST; however, two studies have conducted the WST in the community. In a study by Best et al., (2005), anecdotal evidence supported that all WST skills in the participants’ homes and the community were “reasonably similar” to the specifications in the WST manual. In a study by Kirby et al., (2016), it was reported that the WST was conducted in the homes of veterans with spinal cord injuries; however, no description of the home environments or the utility of the WST was provided. Since its conception, the measurement properties of the WST have been evaluated in five studies. The WST has high reliability across all versions for different mobility devices, as can be seen below in Table 1. Boxes indicated with ‘n/a’ indicate reliabilities that have yet to be evaluated in the literature. The external aspect of validity of the WST has also been examined for the WST for scooter. Among scooter users, the WST has been statistically significantly 6 correlated with WST-Q confidence domain (i.e., confidence) (r = 0.466, p = 0.038) and subjective scooter skills (r = 0.547, p = 0.013). In addition, males perform better than females on the WST (Mortenson et al., 2017). Table 1. 1 Reliability values for the WST version 4.1 reported in the literature Intra-rater reliability (ICC) Inter-rater reliability (ICC) Test-retest reliability (ICC) Manual Wheelchair (Lindquist et al., 2010) 0.950 (95% CI: 0.880-0.984) 0.855 (95% CI: 0.683-0.953) 0.901 (95% CI: 0.786-0.971) Powered Wheelchair (Smith et al., 2017) Range from 0.923-0.998 0.940 (95% CI: 0.862-0.985) n/a Scooters (Mortenson et al., 2017) n/a n/a 0.889 (95% CI: 0.741-0.954) In my search of the literature, the WST has been used as an outcome measure in 42 studies as of July 2017, which can be found in Appendix 1. Of these studies, 11 used the WST as an outcome measure in observation of the efficacy of the WST-P. Other studies have used the WST to evaluate wheelchair skills capacities of specific groups (e.g., rugby players (Furmanuik et al., 2010), basketball players (Saltan et al., 2017), and caregivers (Rushton et al., 2017)), in addition to different clinical populations (e.g., people with hemiplegia (Charbonneau et al., 2013; Jung et al., 2015; Kirby et al., 2005), individuals with spinal cord injuries (Nelson et al., 2010), and stroke survivors (Mountain et al., 2010)). The WST has been used to assess new technologies (e.g., intelligent powered wheelchairs (Boucher et al., 2013), pushrim activated wheelchairs (Best et al., 2006), and one hand drivable wheelchairs (Jung et al., 2015)). In addition, the WST has been used to evaluate new wheelchair training programs, such as the program “Enhancing participation in the community by improving wheelchair skills (EPIC 7 Wheels)” (Giesbrecht et al., 2013), a program that used a motor learning approach (Morgan et al., 2017), and a boot camp approach to teaching occupational therapy students wheelchair skills (Giesbrecht et al., 2015). The WST has also been used in validation studies, which includes the relation of wheelchair skills to the constructs of quality of life and community integration (Hosseini et al., 2010), mobility (Lemay et al., 2012), leisure time physical activity (Phang et al., 2012), life space mobility (Sakakibara et al., 2014), self-efficacy (Sakakibara et al., 2014) and goal satisfaction (MacGillivary et al., 2017). 1.4 Validity Validity has been defined as the extent to which an instrument measures the construct it was developed to evaluate (Raykov and Marcoulides, 2011). Historically, according to Raykov and Marcoulides (2011), validity of performance assessment has been defined in three categories: content, criterion, and construct. Content validity specifically evaluates the degree to which a test accurately represents a construct or performance of interest (Raykov and Marcoulides, 2011). Content validity is subjectively evaluated, as no measures exist that can be used to assess it (Portney and Watkins, 2007). Criterion validity represents the extent to which one can predict scores on another variable from the scores on the given instrument (Raykov and Marcoulides, 2011). Criterion validity is further divided into two types: predictive validity (extent to which future performance can be predicted based off current performance) and concurrent validity (extent to which two test scores taken roughly at the same time predict scores on a construct) (Raykov and Marcoulides, 2011). Lastly, construct validity is the degree to which explanatory constructs effect the performance on the test (Raykov and Marcoulides, 2011). According to Messick (1995, p. 741), validity is “not a property of the test or assessment as such, but rather the meaning of the test.” Messick (1995) proposed that there should only be 8 one type of validity – construct validity – although it should take into consideration aspects of content and criterion validity by combining both evidence and the actual or potential implications of score interpretation and use. He argues that separating validity (i.e., into content, criterion and construct) creates an incomplete picture of validity; criterion or content validity alone does not accurately reflect the overall validity of a measure, but are rather complementary forms of validity (Messick, 1995). 1.4.1 Messick’s Sources of Validity According to Messick (1995), there are six aspects of validity that can be used to define validity: content, substantive, structural, generalizability, external, and consequential. These six sources of validity are interrelated, and together provides a more complete justification of score use and interpretation. These aspects are be described below. The content aspect relates to “evidence of content relevance, representativeness, and technical quality” (Messick, 1995, p. 6). This aspect refers to the process of determining the boundaries of a construct to be measured; what skills and knowledge are involved. When determining what tasks should be selected for assessment when evaluating a construct, the relevance (i.e., the task should be pertinent to the construct being measured) and representativeness (i.e., the task should represent the entire construct being measured) of the task should be considered. The relevance and representativeness are often determined through expert judgement (Messick, 1995). A measure with a good content source of validity will cover all the meaningful and important aspects of the domain, and be technically sound (Carter, 2009). The substantive aspect refers to the use of empirical evidence and appropriate theories to define the boundary of the domain to be tested (Messick, 1995). Empirical evidence and theories should be used to identify the appropriate sampling of skills of a given domain, and provide 9 support to the professional judgements made in the content aspect of validity. This aspect is important to validity as not all skills evaluated on an assessment may be related to the domain. It is important to ensure that the majority of the skills being evaluated are reflective of the construct being measured. To ensure so, the tasks or skills on an assessment should be selected based on theory, and should take place in a real-life, meaningful setting to the study participant. The structural aspect refers to the internal structure of an assessment and the rubric the assessment is scored on (Carter, 2009). The measure should consist of tasks that are related to each other, and should reflect how skills are related within the construct of interest. To ensure good structural validity, Messick (1995) suggests that each item on a scoring rubric should be related back to the underlying processes that produce the effect. Generalizability refers to the extent to which results can be applied to the population, and the ability to make inferences across varying conditions (Messick, 1995). In order to be able to generalize results, the assessment must be representative of the construct and have tasks that are correlated with other tasks representing the construct. Messick (1995) cautions that interpretations of a score may only be generalizable within the boundaries of the construct; generalizations may only be made to what the evidence is meant to demonstrate within the boundaries of a construct. Some factors that contribute to the boundaries of a construct include domain sampling and reliability. Given the limitation time in completing assessments (i.e., assessments have to be administered in a timely manner; we cannot test a participant forever), an entire domain may not be completely sampled. Generalizations may only be made to what has been measured in the domain. Secondly, generalizations are limited to the reliability of an assessment; scores are impacted by time and across multiple raters. 10 The external aspect refers to convergent and discriminant evidence, and observes the extent to which the measure correlates to other similar measures, and does not correlate to measures it should not be related to (Messick, 1995). The degree to which the given measure relates or does not relate to other measures in a domain provides support to the scores on the measure of interest. The consequential aspect observes the impact of the meaning and values of the interpretation of scores for a given group of individuals (Messick, 1995). A central theme to this aspect is the interpretations of the outcomes of a measure – do the interpretation of the data lead to positive, beneficial outcomes or negative and potentially harmful outcomes? Although assessments are often used to show benefits, it must also show that adverse consequences are mitigated (Messick, 1995). The consequential aspect of validity therefore includes evidence that evaluates the intended and unintended effects of score interpretation. If the direction of the interpretations is negative, one must ensure that the results did not arise from construct underrepresentation or construct irrelevant variance (Carter, 2009). 1.4.2 Threats to validity Messick (1995) identified two potential threats to construct validity: construct underrepresentation and construct-irrelevant variance. Construct underrepresentation occurs when important aspects of the construct are omitted (i.e., the assessment does not include all the essential dimensions). Construct-irrelevant variance occurs when the assessment is too broad and incorporates dimensions outside of the construct. More specifically, there are two types of construct-irrelevant variance: construct-irrelevant difficulty and construct-irrelevant easiness. Construct-irrelevant difficulty occurs when aspects of an assessment are especially difficult for specific groups of people, thus affecting their score. For example, adding a skill of “transferring 11 into a car” may be difficult for scooter users who are do not transfer daily into a vehicle. Construct-irrelevant easiness occurs when aspects of an assessment assist individuals to respond correctly or appropriately through the provision of clues or if the measure is familiar to the user. Therefore, when one is evaluating the construct validity of a measure, care must be taken to ensure that the measure provides adequate coverage of the construct, while ensuring that the items are not biased towards or against a specific group and does not provide any assistance. 1.5 Relevance of the environment in testing The practice of occupational therapy highlights the importance of the impact the environment can have on performance. When evaluating skills used to perform activities (e.g., going shopping, proficiently using a scooter to travel in the community, visiting family and friends), it is best understood in context (Bottari et al., 2006). For example, according to the Ecology of Human Performance model (Dunn et al., 1994) performance of skills evaluated out of context may lead to misinterpretations; lab-based testing may not accurately reflect performance in a naturalistic setting (e.g., community settings). There is limited research evaluating performance on various domains of activities of daily living (including ambulation) in a clinic-based setting versus home-based settings (Stoker et al., 2012). Given the influence of environmental context on performance, the scooter skills should be evaluated in a more natural environment (e.g., the scooter user’s community). While there has been no research specific to wheeled mobility skills, one review found that individuals with traumatic brain injury perform significantly better in a familiar home environment (Bottari et al., 2006). In addition, another study found that motor scores (which included ambulation) of individuals with dementia were higher on the structured Assessment for Independent Living Skills measure in a home-based setting (Hoppes et al., 2003). A model by Shumway-Cook and Woollacott (1985) posited that 12 movement is specific to environments, because of the complex interaction among the person, task, and environment. Performance of skills and activities are therefore environment specific; multiple factors in regard to the perception of the individual, the nature of the task, and factors in the environment (such as light, temperature, noise level) may impact performance. 1.6 Applicability of a measure There is little consensus about what makes a measure applicable or pragmatic (Auger et al., 2006); however, Auger et al. propose four main operational criteria that can be used for this purpose, namely response burden, examiner burden, format compatibility, and score distribution. Response burden includes factors such as administration time and invasiveness (i.e., physical or emotional), and acceptability, which includes factors such as number of refused items, and response rates. Examiner burden observes the hindrances experienced by the researcher or clinician, and includes factors such as administration time, scoring, training, and environmental requirements. Format compatibility relates back to Messick’s concept of consequences (1995), whereby the measure should be compatible with a specific population to avoid biases (e.g., construct underrepresentation). An applicable measure should be in a format that is compatible with the characteristics of the target population, which may include factors such as age, gender, culture, language, and ability levels (Auger et al., 2005). Overall, a measure is considered applicable if it is low in response and examiner burden, and has good format compatibility and a normal score distribution. 1.7 Theoretical Considerations This study was guided by Urie Bronfenbrenner’s Human Ecological Theory (Bronfenbrenner, 1995). According to Bubolz and Sontag (1993), a strong value that underlie the human ecology theory include survival and betterment (i.e., being for the better) of all people. A 13 premise of this theory is that there is an interdependency between a person and their environment; the quality of life of humans is interdependent with the quality of their environments (Rosa and Tudge, 2013). Bronfenbrenner proposed four main ecosystems (or environments in which a person interacts with: the microsystem (most proximal setting to the person in which the majority of their interactions occur), the mesosystem (relationships between two or more microsystems), the exosystem (an environment that an individual does not actively participate in, but is still influenced by), and the macrosystem (institutional systems, including policies and culture). Years after Bronfenbrenner developed his theory, he then introduced a model – the bioecological model of human development. This model encompasses four elements that influence developmental outcomes: process, person, context, and time. The proximal process element refers to the developmental processes that occur within an individual. Bronfenbrenner notes that development of an individual not only stems from genetics, but also from interactions with other individuals and objects in their immediate environment over an extended period of time (Brofenbrenner and Morris, 1998). The person element refers to three general categories of characteristics of an individual that will facilitate development: generative forces, resource characteristics, and demand characteristics (Brofenbrenner and Morris, 1998). Generative forces are characteristics that will initiate or maintain the momentum of the aforementioned element of proximal processes, and include motivation, tendencies to take initiative, curiosity, and prioritizing long term goals over immediate gratification (Rosa and Tudge, 2013). Resource characteristics are traits that influences an individual’s ability to effectively engage in proximal processes (e.g., skill level, knowledge, experience). Lastly, demand characteristics are qualities that may hinder development (e.g., poor temperament, passiveness). The context element refers to the four ecosystems in Brofrenbrenner’s original theory. Lastly, the time element refers to the 14 historical period through which the individual is living in, in addition to the conditions and events occurring at the time (Bronfrenbrenner, 1995). There are multiple assumptions made by the human ecology theory (Bubolz and Sontag, 1993). Firstly, all living beings have some common cognitive processes, which can be described and understood using a similar abstract concept. Secondly, the person and environment are seen as an interdependent relationship and must be analyzed as a system. The interdependency of humans and the environment allow for humans to be impacted by their environment, and for the environment to be impacted by humans. Thirdly, all parts of the environment are related and have an influence on each other. Fourth, although environments do not directly determine human behaviour, they positively create opportunities for individuals, but also impose limitations and constraints on them as well. Lastly, it is assumed that individuals have varying amounts of freedom and control with respect to the level of interaction with their environment. The present study focuses on the interactions between scooter users and their microsystem environments (i.e., the communities of the participants) in regard to users’ scooter skills. Given the bidirectional relationship between the person and the environment, data was collected on both the person (via measuring their skills and interviewing them) and on the environment (via photos and taking physical measurements of hills, curbs, and potholes found in the community). As the individual and environment need to be analyzed as a system, mixed-methods were used to develop a more holistic understanding of how the environment plays a role in the development of scooter skills. The notion of mesosystems (i.e., the influence of being directly involved with two or more environments) was observed by exploring the different types of environments scooter users travel in, and looking at the impact of the frequency of scooter use on users’ development of skills. Although not directly measured quantitatively, the influence of 15 the macrosystem (e.g., policies in regard to scooter training) was considered as part of our qualitative interviews. . Objectives and Hypothesis Given the increased popularity of scooters and the safety concerns around its use, more research is required on the use of the Wheelchair Skills Test for scooter users. While much research has been done on and with the WST, little has been conducted on the WST for scooter users, and none have directly evaluated the use of the WST in a community-based setting. Conducting tests in a meaningful, real-life setting may influence the interpretation of the scores – tests conducted out of environmental context may result in misconstruction of the data. The human ecological theory supports this notion by identifying the bi-directional relationship between the person and their performance and the environment. The purpose and primary objective of this study was to compare scooter skill performance on the WST in an indoor standardized course (indoor lab-based course) to skills in the outdoor unstandardized community setting (outdoor community course). This project was guided by the question: how well do scores obtained in the community relate to those collected in an indoor lab-based course. The objectives of this study are therefore to: 1) Quantitatively determine the distribution of scores and how scores compare between indoor lab-based and outdoor community-based administration including level of agreement. It was hypothesized that scores between the indoor and outdoor environments would not be statistically significantly correlated (p < 0.05). 2) Qualitatively explore participants’ experiences with scooter use to explore the content aspect of Messick’s definition of validity 16 3) Observe validity through 1) quantitatively observing the correlations among the indoor lab-based WST scores, the outdoor community-based WST scores, and the WST-Q scores, and by qualitatively exploring 2) participants’ perspectives on the representativeness of the WST and WST-Q in both settings, 3) testing preferences, and 4) suggestions for improvement. It is hypothesized that in both environments, skills confidence and life space travelled would be positively correlated; however, the magnitude of the correlation between skills confidence and life space travelled with the performance of the WST in the outdoor community-based setting would be higher, as the participants may be more comfortable operating their devices in familiar settings. The magnitude of the correlation between life space travelled and scores on the WST in the outdoor community-based setting is hypothesized to be higher. 4) Quantitatively and qualitatively determine the applicability of performing the WST in different settings by comparing the features in the outdoor community to the indoor lab-based course, and exploring the perceptions of participants in regard to the necessity of specific items on the WST, safety, and perceived benefits of performing the WST. Based on my previous experience of conducting the WST in an outdoor community-based environment, it is expected that ≥90% of WST skills would be found in the community, and all skills will be within ±15% of the measurements of the indoor lab-based course. CHAPTER 2: METHODOLOGY 2.1 Design This study used a mixed-methods approach, as combining quantitative and qualitative methods can help create a deeper understanding (Kroll et al., 2005). More specifically, a sequential explanatory strategy (Kroll et al., 2005) was used, whereby the quantitative portion 17 preceded the qualitative. With this strategy, the qualitative data was intended to highlight perceptions, values, and attitudes, contextualize findings and to explore causal pathways (Kroll et al., 2005). In contrast quantitative methods were used in this study to evaluate how similar the scores on the WST were between the indoor lab- and the outdoor community-based environments, and to determine the practicality of performing the WST in an outdoor community-based setting. The qualitative piece in this study was used to support the findings from the quantitative part; the participants’ perceptions and experiences were used to explain the comparison of scores between the indoor lab-based environment and the outdoor community-based environment. Across-method triangulation was used to perform triangulation between the data; the qualitative data corroborated the data from the quantitative. In addition, data-analysis triangulation was completed through combing the use of both content analysis and thematic analysis for the qualitative data. The quantitative section of this paper has been guided by the Strengthening the Reporting of OBservational studies Epidemiology (STROBE) cross-sectional guidelines, and the qualitative section of this paper has been guided by the COnsolidated criteria for REporting Qualitative research (COREQ) guidelines (Tong et al., 2007). The Good Reporting of A Mixed Methods Study (GRAMMS) guidelines were also used. These forms can be found completed in Appendix 2. This research adopts a post-positivist perspective. The positivist perspective is based on the idea that knowledge arises from observable findings from a direct source, and that truth is objective; it is independent from the researcher and arises from facts present in the tangible reality (Clark, 1998). The positivist view fails to acknowledge the researcher’s biases and involvement in the research, and does not accept data from sources such as the understandings and experiences of individual people. In contrast, the post-positivist perspective acknowledges 18 the positivist stance that research requires logical reasoning, precision, and evidence, but also places importance on unobservable data (Clark, 1998). In this sense, quantitative and qualitative research are not mutually exclusive; qualitative data may be used to support quantitative results. The post-positivist perspective also acknowledges that the researcher is a tool in the research process; each scientist brings with them their own perspectives and biases that may influence the findings of the study (Clark, 1998). 2.2 Participants 2.2.1. Inclusion and exclusion criteria To be recruited for this study, participants needed to have used a powered wheelchair or a scooter for 3 months or more, be able to communicate in English, and be 19 years of age or older. Individuals with cognitive impairments that prevented them from providing consent or reliably completing measures, or those who live in nursing homes were excluded. A minimum of 3 months of scooter use was selected, as previous studies have used this timeframe to classify users as experienced (Kirby et al., 2015; Mortenson et al., 2017). We wanted to avoid including new users because training effects are more likely in this population. Individuals who reside in long-term care facilities were excluded from the study, as they require institutional approval, and residents are not permitted to freely leave the home. 2.2.2 Sample size Using G*Power, the sample required to conduct a matched-pairs t-test with a two-tailed α = 0.05, a power of 0.8, and a large effect size of 0.7 (Cohen, 1992), 19 participants are required; however, a total of 20 participants was recruited in anticipation of dropouts. 19 2.3 Recruitment Participants were recruited in the Metro Vancouver area via posters placed around GF Strong, and ICORD. Participants who had previously participated in other studies and had indicated their interest in participating in future studies were contacted by telephone or email. 2.4 Quantitative Methods 2.4.1 Outcome measures All outcome measures can be found in Appendix 3. A demographics form was used to gather information, including each participant’s age, gender, employment status, and living situation. Participants were also asked about their diagnoses and their use of mobility aids (e.g., duration of use, which device they use, etc.). The primary outcome measure for this study is the objective WST. The WST consists of 29 skills that vary in difficulty, and is scored on scale from 0-2, where 0 = unable to perform skill/unable to safely perform skill, 1 = participant performs skill with difficulty, and 2 = skill is performed safely and confidently. Due to safety concerns, the skill “getting from the ground into the wheelchair” was omitted part way through data collection for 13 participants. The WST has been validated for scooter users and has good reliability (ICC = 0.889) and a Cronbach’s alpha of 0.74 (Mortenson et al., 2017). Subjective skills and confidence were measured using the WST-Questionnaire (WST-Q). The WST-Q is a 28-item form that asks participants whether they think they can perform a skill perfectly, with difficulty, or not at all. The WST-Q also includes a confidence aspect, whereby participants rate how confidently they can perform a skill. The options provided are fully confident, somewhat confident, or not confident. Each provided option for performance and 20 confidence is then converted into a numerical value (0, 1, or 2), which is then tallied to find a score for each domain. A weighted score was calculated using the equation provided in the WST manual (Kirby et al., 2015), which accounted for the skills that were not possible or if a testing error occurred. The total WST-Q score was calculated by summing the weighed scores of each domain. The total WST-Q score has been found to be correlated with total WST scores among scooter users (r = 0.547, p = 0.013) (Mortenson et al., 2017); however, reliability of the WST-Q has not been observed for scooter users, but has been for powered wheelchair users. The test-retest reliability of the WST-Q among powered wheelchair users is ICC = 0.78, and the limits of agreement between baseline and 1 month ranged from 72.2%-100% (Rushton et al., 2014). Mobility was measured using the Life Space Assessment (LSA). The LSA measures mobility by inquiring participants about how often they move between various five life spaces that vary from small (i.e., do you leave your bedroom) to large (i.e., do you leave your city). The LSA has good reliability and validity (Baker, Bodner, and Allman, 2003), and has previously been used among powered mobility device users (Auger et al., 2010). Scores are calculated by multiplying the 3 values: the life space (yes = 1, no = 0), the frequency of travel in the last month (< once a week = 1, 1-3x/week = 2, 4-6x/week = 3, daily = 4), and by the type of equipment used (personal assistance = 1, equipment only = 1.5, no equipment or personal assistance = 3). Each of the life spaces have an increasing value as the size of the life space increases (i.e., bedroom = 1, outside home = 2, neighbourhood = 3, city = 4, outside of your city = 5). In addition, a separate score indicating how often individuals travelled with or without their scooter (i.e., scooter use frequency) was calculated by multiplying the total score for each life space level by 1 if a scooter was used or 0 if a scooter was not used. The LSA has an ICC of 0.96 over a two-week period, and has been correlated with physical performance, activities of daily living, 21 depressive symptoms, and self-reported health to provide evidence of external validity (Baker et al., 2003). 2.4.2 Procedure 2.4.2.1 Comparison of performance between environments This study required 2 assessment sessions. Upon receiving written consent and enrolment, participants were randomly assigned to their first testing environment, either the indoor lab-based environment (i.e., GF Strong Rehabilitation Center or ICORD), or the outdoor community-based environment. The outdoor community-based environment consisted of the immediate surroundings outside of the participants’ homes within a 2-block radius (i.e., the immediate area outside of their homes; their neighbourhood). The environment order participants performed the WST in was randomized to prevent order effects (Portney and Watkins, 2000). Randomization was conducted using the randomize function in Microsoft Excel. Participant IDs were listed in three blocks of six and one block of two; even numbers were selected for block sizes should a participant drop out. In addition, the order of measures within each session was randomized. During the first assessment, all participants completed a demographics form, the WST-Q, and the LSA. The WST was performed twice, once in the indoor lab-based environment and once in the outdoor community-based environment. The WST-Q confidence domain was additionally conducted concurrently with the WST in each environment; participants were asked to rate how confident they felt performing a skill prior to performing it in each environment. All participants performed the WST and the WST-Q confidence domain in their second environment within a two-week timeframe. The WST and the WST-Q confidence domain was performed 22 twice within this time limit to reduce the impact of any changes (e.g., in health, etc.) that may occur. 2.4.2.2 Applicability of the WST in the outdoor community-based setting The applicability of the WST was measured based on Auger et al.’s (2005) four criteria of applicability: examiner burden, score distribution, response burden, and format compatibility. Examiner burden was measured in the time it takes to find all the skills within a two-block radius of the participant’s home. Timing started when the researcher arrived in front of the participant’s property, and ended when I got back from searching for skills. Score distribution was determined by creating distribution plots and visually analyzing it. Response burden was evaluated through interviews with the participants, focusing on questions around their thoughts on the time it took to complete the test and perceptions of safety. Format compatibility was evaluated through assessing whether all skills could be found in the outdoor community-based setting, and how comparable the outdoor community-based environment was to the standardized indoor lab-based course. To compare how similar the challenges found in the outdoor community-based setting were to the indoor lab-based setting, data were collected about the similar indoor lab-based skills found in the community. These data were documented using a form developed by myself for this study with the guidance of my supervisor, which can be found in Appendix 4. The lengths and angles of the ramps, the height of the curb, and the dimensions of the gap were measured and recorded. The angles of the ramps were measured at 3 locations on each ramp using a digital inclinometer (i.e., once at the bottom, once in the middle, and once at the top). The soft surface that was used (as recommended in the WST manual (Kirby et al., 2015) in the indoor lab-based course was gravel; however, in the outdoor community-based environment, grass or gravel was used, and the type was noted. 23 Furthermore, the weather, temperature, and surface conditions (e.g., wet or dry) on the day of performance were documented. Photos were taken of all the comparable WST challenges (e.g., ramps/hills, potholes, curbs) found in the community. If I was not able to find slopes with the exact same angles as the indoor lab-based environment, I found and used two slopes available within the one-block radius of the participants’ home instead and noted it down. An effort was made to find slopes of varying steepness; a steeper slope and a gradual slope were sought after. 2.4.3 Statistical Analyses 2.4.3.1 Descriptive statistics to describe the sample Descriptive statistics were used to analyze demographics and to describe the study sample. A Shapiro-Wilk test was run to determine the normality of the data. Mean scores and standard deviations were calculated for each measure. To evaluate objective two, a Bland-Altman plot was used to determine agreement in measuring wheelchair skills in an indoor lab-based versus outdoor community-based environment. When evaluating the plot, the value between the limit of agreement dictates whether two measures agree. The acceptable limit is based not on statistics, but clinical judgement (Bland and Altman, 2003). The statistical test used to determine whether scores differed between environments, as part of objective two, depended on the distribution of scores: a paired sample t–test for parametric data or Wilcoxon signed rank test for non-parametric data. In addition, a one-way ANOVA was used to determine whether scores were dependent on the type of scooter used (four-wheeled, three-wheeled, or travel). An independent t-test was also run to determine whether scores differed among participants who have previously performed the indoor lab-based WST before and those who did not. 24 Given that the skill of ‘getting from ground into scooter’ was been omitted part way through data collection, this skill was omitted from statistical analysis. Details of the incident are reported below in the results section. To evaluate objective 3.1, correlations were performed to examine the validity of indoor lab-based and outdoor community-based scores on mobility, WST-Q (subjective) scooter skills, and skills confidence. Values from 0.1-0.29 are considered small, 0.3-0.49 is considered medium, and 0.5 or greater are considered large (Cohen, 1992). If the data was normally distributed, a Pearson correlation was conducted. Should the data be skewed, a Spearman correlation was be used. To evaluate the fourth objective, the outdoor community-based measurements of each skill (e.g., curbs, slopes) were compared. A paired t-test was used to determine whether skills in the outdoor community-based environment were significantly different from those on the indoor lab-based standardized skills course. The three measurements taken of each slope found in the participants’ communities (i.e., top, middle, and bottom) were averaged, and compared with the values of the indoor lab-based course. Differences ≥ 20% are considered to be clinically, significantly different, as reported by previous studies (Kirby et al., 2015; Newton et al., 2002). 2.5 Qualitative Methods 2.5.1 Interviews Semi-structured interviews were conducted with 18 of the 20 participants upon completion of the WST in both environments; the participant who was injured during the testing was unable to complete the interview and data from another interview was not available as the recording was corrupted. A list of participant pseudonyms and descriptors can be found below in table 2.1 25 Table 2. 1 List of pseudonyms and device use Pseudonym Age Years of experience using a scooter Other mobility devices used Lisa 45 1.91 John 87 18.2 Samantha 72 7.5 Powered wheelchair Angela 79 6 Kevin 52 10.75 2-wheeled walker Kyle 89 13.75 Cane, 4-wheeled walker Sandra 64 6.3 Medically prescribed footwear, leg brace, 2-wheeled walker, 4-wheeled walker, manual wheelchair Brandon 67 5.91 Cane, 4-wheeled walker Rachel 65 2.5 Crutches, 2-wheeled walker Mandy 60 3.08 Crutches, Medically prescribed footwear Vanessa 54 1.91 Crutches, 4-wheeled walker Brad 69 4.92 Crutches, 4-wheeled walker, manual wheelchair Elise 75 10.83 Medically prescribed footwear, 2-wheeled walker, 4-wheeled walker Jill 80 17.83 Medically prescribed footwear, 4-wheeled walker Barbara 71 8.17 Cane, powered wheelchair 26 Pseudonym Age Years of experience using a scooter Other mobility devices used Paul 59 12.75 Crutches, leg brace, 4-wheeled walker Jarrod 61 4.16 Crutches Mary 74 35.17 4-wheeled walker, manual wheelchair I conducted all the interviews, and identify as a female with 2 years of experience with interviewing. Interviews lasted an average of 20 minutes, and half of interviews took place in the homes or apartment lobbies of the participants, while the other half took place in a research lab. All interviews were conducted one-on-one (i.e., only the researcher and the participant were present). A complete interview guide can be found in Appendix 5. Questions were asked around performance and confidence, opinions on the utility of the WST, and environment preference. Participants were asked how they thought they performed in each environment, shown their scores on the WST in both environments, and were asked for possible explanations for any differences in score. The utility of the test was explored through the opinion of the participants on the burden of the test, in addition to whether they believed all the skills required for community driving were included. Lastly, environment preference was explored in terms of which environment participants found to be a more accurate assessment of skills, and whether they felt more comfortable in one environment over the other. 2.5.2 Methods Upon the completion of the creation of the interview guide, the interview was piloted with another individual in the lab. After the test run, the order of the questions was changed, and additional questions were added (i.e., on confidence). Prior to each interview, I would input the 27 WST scores into an excel sheet and briefly compare the scores in each environment. When interviewing each participant, results from the WST and photos were used as prompts. An example can be found in Appendix 6. WST scores were shown to participants after they indicated in which environment they thought they performed better. Revealing WST scores from both environments was used to guide the interview, as it was hoped that doing so would help the participants think about why they performed better on certain skills in a specific environment. Furthermore, photos taken of the skills performed in the outdoor community-based environment and the indoor lab-based course on a laptop screen were shown to help participants with recall. I kept a researcher journal that included personal reflections (e.g., my perspectives on how similar/different the community was) and descriptions of the participant (e.g., reasons performance may have been affected, such as fatigue). Personal reflections of how similar each of the outdoor community settings were to the indoor lab-based setting was documented in addition to the objective measurements taken of the environment in the form in Appendix 1. All interviews were digitally recorded and transcribed verbatim. All transcriptions were password protected and anonymized – participants were given pseudonyms, which are used in the results below. As interviews were used to support the quantitative data as well as to explore participants’ perceptions of the performing the WST, thematic and content analyses were conducted; a thematic analysis was used to address the first objective of exploring the participants’ experiences with scooter use, and a content analysis was conducted when analyzing data on an individual question level. NVivo 11 was used to code the data. 2.5.2.1 Thematic Analysis The thematic analysis was conducted in accordance with the six steps indicated by Braun and Clarke (2006). As per the first step, the researcher familiarized herself with the data by 28 reading all the transcripts through once, noting initial ideas. Secondly, the researcher went back through all the transcripts and began identifying codes. The graduate students’ supervisor assisted with the coding process by independently coding the first two transcripts. They proceeded to discuss similarities and differences. Discrepancies between the coding were deliberated and agreed on. Upon completion of coding all of the transcripts, the codes were organized into potential themes, based on common or related ideas. Themes were then refined in the fourth step, which included omitting or collapsing smaller themes. Once the themes were finalized, names were provided for each theme. A complete list of codes can be found in Appendix 7. 2.5.2.2 Content Analysis To address objectives one, three, and four, a content analysis was used to inductively identify mutually exclusive response options for specific open-ended questions. Content analysis commonly consists of a word-frequency count, whereby the most frequently mentioned words are assumed to reflect importance or concerns (Stemler, 2001). Elo and Kyngas (2007) state that content analysis consists of three phases: preparation, organizing, and reporting. In the preparation phase, the unit of analysis is selected; one needs to decide what to analyze in detail. The unit of analysis may consist of a word or a theme, which may be identified as a sentence or a portion of a page (Elo and Kyngas, 2007). In this study, themes for the units of analysis include the representativeness of the WST, preferred testing environments, and confidence to address the third objective of this study, and the necessity of items on the WST, safety, and perceived benefits of the WST to address the fourth objective of this study. The preparation phase ends with the researcher immersing herself in the data by reading and re-reading the interviews. The next phase, organization, consisted of analyzing the data, beginning with open coding of the data. 29 During this process, notes were created while reading through the interviews (Elo and Kyngas, 2007). Following this, the codes were categorized and grouped, to describe phenomena. In the last phase, the data were reported, as outlined below. 2.5.2 Trustworthiness Strategies As per the post-positivist views on qualitative research, trustworthy strategies were used to promote credibility, dependability, transferability, and confirmability (Morrow, 2005). Credibility is concerned with internal consistency, or the rigor of the research (Morrow, 2005). To facilitate credibility of the data, researcher reflexivity was maintained with the use of a researcher journal, as described above. This journal encouraged me to reflect on what external factors and biases may have influenced their analysis. Additionally, a relaxed, trusting rapport was built between the participants and the researcher in attempt to make the participants feel comfortable in discussing their honest opinions in the interviews. Transferability refers to the extent to which the findings of a study may be generalized. To promote transferability, detailed descriptions were kept of the participant and the interview environment, and notes on the researcher-participant relationship were documented. There is often a tension between transferability and anonymity; details about the participant were reported while still protecting their identity. Although many details were collected about the participants, only a limited description (i.e., duration of scooter use) was provided in this document along with their pseudonym. Dependability is concerned with the consistency of the study across time and analysis techniques (Morrow, 2005). To facilitate dependability, analytic memos were kept; thoughts and ideas of emerging themes or patterns were noted during the on-going data analysis. Confirmability is the acknowledgement that research is never objective (Morrow, 2005). This is highlighted through the personal reflections section later in this document. Methodological 30 triangulation of the data was used in this study given the mixed-methods methodology employed (Thurmond, 2001). Methodological triangulation provides a more holistic perspective, as the qualitative data was used to explain findings from the quantitative data (Thurmond, 2001). Data triangulation was also completed through looking for convergence and divergence of the information collected through quantitative and qualitative methods. Additionally, upon completion of the data analysis, a lay summary was created and sent out to all participants for member checking. 2.5.3 Reflexivity 2.5.3.1 Researcher Position I acknowledge that I am a young, able-bodied, female who is educated with a university degree. I grew up living with my mother who suffered from a brain aneurism before I was born; although not first hand, I have therefore been exposed to what it is like to live with a physical disability. Like the participants in this study, my mother can ambulate, but not proficiently. In addition, I have worked with people with spinal cord injuries and diseases over the past five years, and specifically with scooter users for the past three years. The opportunity to work with these individuals on a more intimate level has provided me further insight on what it is like to live with a disability and on the rehabilitation process. This experience has helped me develop a deeper understanding on the challenges faced by individuals with a spinal cord injury, but more importantly, how individuals with physical disabilities can regain their independence in everyday life. This knowledge has allowed me to approach the current study with some knowledge about the experiences of living with a disability, and to feel more comfortable addressing disability in general. 31 2.5.3.2 Researcher bias and assumptions Working with scooter users for over three years has led me to hold a potential bias and assumptions of this population. First, the fact that I had previously established connections with many of the participants (i.e., 65%) in the study may have created some bias in regard to their performance level as I had more in depth knowledge on how they use their scooters. For example, knowing that a participant has relied full time on their scooter for more than 10 years leads me to assume that they are very proficient with their scooter skills. In addition, these pre-established relationships may have impacted the researcher-participant power dynamic when conducting interviews. The power-dynamic shifts from a researcher-participant dynamic to more of a friendship dynamic, as I have known some of these participants for years. This relationship may lead to two different potential situations in interviews: 1) the participants may feel comfortable opening up with me and will not hold back, thus providing rich, in depth detail, or 2), participants may provide truncated responses. Truncated responses may arise due to participants assuming that I already know a lot about them, and that minute details do not need to be mentioned again. Secondly, working in the field of scooter use may have led to a positive bias for scooter use. I have seen how much independence and freedom a scooter can provide for an individual who has limited mobility and have heard numerous raving stories from scooter users on how using a scooter has “changed their lives.” Despite hearing negative stories in regard to scooter use (e.g., accessibility, accidents), the positives always seem to triumph the negatives. My positive view on scooter use may have impacted my perspectives when questioning participants about their scooter use in the interview. For example, more time was spent discussing the positive aspects of scooter use, while the negative aspects did not receive as much 32 attention. In addition, when clarifying/summarizing the participant’s discussion, I would often emphasize the positives (e.g., “so overall, you would say that using a scooter has been fairly helpful?) which may lead the participants away from thinking about their negative experiences with the scooter. CHAPTER 3: RESULTS 3.1 Demographics The participants’ demographics can be found in Table 3.1. Overall, the 20 participants were recruited for the study and interviews were conducted with 19, although a total of 18 were used for analysis. The average time between testing sessions was 7.76 days. Generally, participants in this study were quite experienced scooter users who relied on multiple devices to get around, and most were retired. Since one method of recruitment was through participants from previous studies, 9 participants (45%) had previously completed the indoor WST course. Table 3. 1 Participant Demographics Variable Average (SD)/ N (%) Age (years) 67.1 (11.54) Female 12 (60) Retired 15 (75) Rely on >1 Assistive Device for mobility 17 (85) Average duration of scooter use (years) 8.98 (7.91) Average use per week 0-1 days 2-3 days 4-5 days 6-7 days 5 (25) 5 (25) 2 (10) 8 (40) Participants who have done the 9 (45) 33 Variable Average (SD)/ N (%) indoor WST prior to the study Diagnoses Multiple sclerosis Post-polio Spinal cord injury Arthritis Other 9 2 2 2 5 Other assistive devices used in a typical week Cane Crutches Medically prescribed footwear Leg brace Two-wheeled walker Four-wheeled walker Manual wheelchair Powered wheelchair 4 6 5 2 4 8 3 3 Various types of scooters were used in this study. The main scooters used in this study were three wheeled scooters, four wheeled scooters, and travel scooters. Scooter measurements for the group are listed in table 3.2. Table 3. 2 Participants’ scooter measurements Measurement n/Mean (SD) Range Number of wheels Four wheel Three wheel 7 12 34 Measurement n/Mean (SD) Range Three wheel travel scooter 3 Length (meters) 1.14 (0.19) 0.84-1.55 Width (meters) 0.63 (0.22) 0.50-1.52 Diameter of front wheel (cm) 22.6 (4.18) 18-34 Diameter of rear wheel (cm) 23.55 (4.32) 18-35 3.1.2 Average participant scores on outcome measures The mean scores for the outcome measures can be found in Table 3.2. On average, participants scored well on all measures, although they did not use their on a frequent (i.e., >5 days) basis. On the WST-Q, participants were fairly confident scooter users, and scored high on the WST in both environments. Despite high WST scores, participants did not travel outside of the city very often, and even less with their scooters. Table 3. 3 Participant mean scores on outcome measures Measure (range of scores) Score (mean (SD)) WST Outside (0-100) 92.68 (7.76) WST Outside Confidence (0-100) 91.04 (5.56) WST Inside (0-100) 88.86 (8.62) WST Inside Confidence (0-100) 85.02 (10.65) WST-Q Total (0-300) 250.80 (27.41) WST-Q Confidence (0-100) 86.92 (11.25) WST-Q Capacity (0-100) 91.61 (8.66) WST-Q Frequency (0-100) 72.27 (16.43) LSA Frequency 34.24 (7.08) LSA Scooter 23 (11.95) When conducting the WST, not all skills were tested, as some were not possible (i.e., the scooter lacked a part or the skill could not be completed because of an absent part) or testing errors occurred (i.e., obstacles were unable to be found in the community). Table 3.4 specifies the number of skills performed for each participant. 35 Table 3. 4 Number of skills performed by each participant on the WST Community Lab-based Participant # of WST skills completed (/28) # testing errors # skills not possible WST obstacles not found # of WST skills completed (/28) # testing errors # skills not possible 1 28 0 0 - 27 1 0 2 21 7 0 Steep hill, soft surface, threshold, gap, low curb 26 2 0 3 27 1 0 Gap 27 1 0 4 26 2 0 Threshold, gap 26 2 0 5 23 5 0 Slight incline, gap, low curb 27 1 0 6 24 4 0 Steep incline, threshold, gap, 27 1 0 7 25 3 0 Slight incline, gap, curb 27 1 1 8 25 3 0 Gap, low curb 27 1 0 9 27 1 0 Gap 27 1 0 10 27 1 3 Gap 18 1 9 11 25 3 0 Hinged door, threshold, gap 27 1 0 12 24 1 0 Soft surface, gap, low curb 28 0 0 13 23 5 0 Soft surface, gap, curb, battery charger 27 1 0 14 21 6 1 Hinged door, soft surface, gap, low curb 27 1 0 15 26 3 0 Hinged door, gap 26 1 1 16 26 3 0 Hinged door, gap 26 2 0 17 27 1 0 Gap 27 1 0 18 25 3 3 Hinged door, threshold, gap 0 0 19 27 1 3 Hinged door 20 1 7 20 28 0 0 - 24 1 3 36 3.2 Determining how scores on the indoor course compare to scores on the outdoor community-based setting (objective two) The outdoor community-based WST scores were not normally distributed (p = 0.010) according to the Shapiro-Wilk test, although they were not more than twice the standard error of measurement, which is also a guideline for skewness (IBM Corporation, 2012) (i.e.,the scores had a negative skew of -0.827 (SE = 0.512)). A histogram of the outdoor community-based WST scores is found below in Figure 3.1. Figure 3. 1 Distribution of outdoor community-based WST scores The scores on the inside lab-based WST were normally distributed according to the Shapiro-Wilk test (p = 0.287), although scores were slightly negatively skewed (-0.382, SE = 0.512). A distribution is seen in Figure 3.2. 37 Figure 3. 2 Distribution of indoor lab-based WST scores The WST-Q total scores were not normally distributed according to the Shapiro-Wilk test (p = 0.029), and was negatively skewed (-0.772). The distribution graph can be found in Figure 3.3. Figure 3. 3 Distribution graph for the total WST-Q scores A ceiling effect (i.e., ≥ 20% of the sample having a perfect score) was observed for the WST outdoor community-based community scores. A subgroup analysis identified that three out of 38 seven (42.8%) participants who completed the entire WST (including the getting up off the ground skill) received a maximum score, while four out of 13 (30.7%) participants who completed the WST with the ground skill omitted received the maximum score. While no other measure exhibited a ceiling effect according to this definition, 20% scored 96% on the WST inside, and 15% scored 100% on the WST-Q capacity. 3.2.1 Comparing scores between environments (objective two) Given that the indoor lab based WST scores were normally distributed, but the outdoor community-based WST scores were not, both the paired sample t-test and the Wilcoxon signed rank test were run. With the paired t-test, the scores between the indoor lab-based and outdoor community-based environments were significantly different statistically (t = 1.861, p = 0.002).. However, when the Wilcoxon signed rank test was run, no significant difference was found between the scores in each of the environments (Z = -1.771, p = 0.077), although the p-value was almost significant. 3.2.2 Observing the correlation and association between indoor lab-based and outdoor community-based WST performance (objective two) The association between WST performance in indoor lab-based and outdoor community-based environments was found to be low. A Spearman correlation (see figure 3.4) revealed low correlation (r = 0.306) of scores between the indoor lab-based and outdoor community-based environments. 39 Figure 3. 4 Correlation between performance scores of the WST outside and the WST inside The ANOVA analysis revealed that scores were not significantly different among the different types of scooters in the outdoor community-based environment (F(2,17) = 0.385, p = 0.686) and in the indoor lab-based environment (F(2,17) = 1.233, p = 0.316). In addition, those who have previously completed the indoor lab-based WST did not perform statistically significantly better than those who did not (t = 0.564, p = 0.362). 3.2.3. Observing the agreement between scores on the indoor lab-based versus outdoor community-based environment (objective two) A Bland-Atman plot of WST scores (Figure 3.5) revealed a limit of agreement ranging from -19.09 – 24.78 (43.87%). The Bland-Altman plot also suggested a slight systematic bias, as participants scored better in the outdoor community-based environment. 606570758085909510070 75 80 85 90 95 100Indoor WST ScoresOutdoor WST ScoresCorrelation of Outdoor vs Indoor Scores 40 Figure 3. 5 Bland-Altman Plot 3.3 Participants’ experiences with scooter use (objective two) Content analysis revealed that three out of twenty participants (15%) in this study relied on their scooter for full time mobility and thus their community participation, while all other participants used their scooter in combination with another assistive device (e.g., walkers, canes, walking poles. Table 3.5 represents the total frequency count of each activity reported. Table 3. 5 Tally of reported daily activities performed with a scooter Activity N (%) Going shopping 16 (80) Going to get fresh air 9 (45) Going to get exercise 7 (35) Going to get coffee 4 (20) Going out to a restaurant 4 (20) Visiting family 3 (15) Going out to read 3 (15) 41 Activity N (%) Going to medical appointments 3 (15) Socializing/going for coffee 2 (10) Going to the library 2 (10) Doing household chores (e.g., cooking, cleaning) 1 (5) Going to the community center 1 (5) Many participants relied on their scooter for community participation. Jill, a scooter user of 17 years, emphasized the importance of using her scooter:, “Oh, I couldn’t do it without it, it’s [community participation] totally dependent on the scooter.” A lot of participants regarded their scooters as a large contributing factor to their community participation. Brandon, a scooter user of 5 years, noted, “oh [having a scooter has had a] huge [impact]! Before I had it, I didn’t do much, well I was more mobile but I did a lot less, but after I got the scooter I realized that I should’ve gotten it [I: Earlier?] A year or two earlier.” A thematic analysis revealed two main themes regarding participants’ experiences with scooter use: finding the scooter beneficial, and finding the scooter a hindrance. In the first theme, participants generally found their scooters to be helpful in regard to participation. Many participants found that using a scooter allowed them to participate in a greater variety of activities. As Mandy, a scooter user of 3 years, noted, “Well, I am able to access some areas that I wouldn’t be able to access otherwise.” Some participants also expressed that using a scooter was beneficial for saving energy. Brandon explained “If I need to try to walk anywhere, I can’t go very far. But if I can take my scooter, then I have the energy to do things." Using the scooter as a vehicle to travel longer distances allowed participants to reserve their energy and apply it 42 towards their intended activity (e.g., shopping, spending time with friends). Some participants also perceived that using a scooter gave them more “freedom from their medical conditions” through alleviating the effects of their condition (e.g., short of breath, inability to ambulate long distances). For example, one women with chronic obstructive pulmonary disorder (COPD) explained "Oh god, [having a scooter is] 100 percent [helpful]. I can’t, with my breathing I wouldn’t, I’d be totally housebound" (Angela, scooter user of 6 years). In the second theme, participants noted how using a scooter occasionally acted as a hindrance to participation in regard to feeling physically discomfort, not optimally performing in inclement weather, and accessibility. Some participants discussed how a lack of tire suspension on their scooters resulted in them feeling every bump they went over. While this only resulted in discomfort for some, for others, it resulted in stiffness and pain. As an example, Mandy described: “Yeah, and then I come home and normally I am relatively tired after I use my scooter because you know, the bumping, it’s –the surfaces are not always very user friendly, so I find that I am a little bit stiff afterwards." There were differing opinions in regards to using the scooter performance in inclement weather. While some participants continued to use their scooters in the rain and snow with caution, others found themselves to be housebound in miserable weather. Lisa, a scooter user of 2 years, explained: If it [the weather] is really harsh I can’t [go out]. I’ve actually had to cancel church before. It started raining heavily and in a very short period of time I got so soaked [I] had to turn around and come back. It was too wet. And then you get really cold, and it’s really windy. It doesn’t matter how much you bundle up, once you get wet its freezing. In contrast, two participants noted that inclement weather was not an issue for them. Participants also discussed the various barriers they faced in the built environment, including public transit 43 and stores. While some participants accessed busses and trains without difficulties, others noted struggles and difficulties, resulting in them being unable to travel outside of their neighbourhood. While trains are generally more accessible for scooter users, some participants noted having troubles accessing the elevators, and that elevator maintenance seems to occur quite frequently. Brandon explained one experience: The other day I actually had to go to New West [for a medical appointment] so what I did was take Canada Line down to Waterfront, took the Expo Line back to Columbia Street, discovered the elevator didn’t work at Columbia Street. […] I had to go back a station since they don’t tell you on the train that the elevator is out of service Participants additionally noted that not all stores were accessible to scooters – quite a few stores had inaccessible aisles (either too narrow or obstructed with displays). Samantha, a scooter user of 8 years, described one experience that she had: Sometimes [when] shopping […] I find the scooter is hard to get around the clothes because the piles they have in the various areas shopping are made for people who walk in single file and the scooter is difficult to maneuver around tight corners. 3.4 Correlations with other measures (objective three) 3.4.1. Determining the relationship of the outdoor community-based WST scores with other measures: the external aspect of validity (objective 3.1) Spearman correlations between WST scores in the indoor lab-based and outdoor community environments and other measures are presented in Table 3.5. Strong, positive correlations were found between the outdoor community-based WST scores and outdoor confidence, indoor confidence, and subjective capacity. Outdoor community-based WST scores were weakly negatively correlated with the WST-Q total scores and confidence and frequency 44 domains, in addition to both LSA scores. The full correlation matrix can be found below in Table 3.6. 45 Table 3. 6 Correlation of WST performance and confidence with other measures WST In WST Out Confidence WST In Confidence WST-Q Capacity WST-Q Confidence WST-Q Frequency LSA Scooter LSA Frequency WST Out (r (ρ)) 0.306 (0.190) 0.682** (0.001) 0.664** (0.001) 0.583* (0.007) 0.040 (0.867) -0.009 (0.970) -0.129 (0.588) -0.240 (0.389) WST In (ρ (p)) x 0.443 (0.051) 0.345 (0.137) 0.271 (0.247) 0.206 (0.384) 0.445* (0.049) 0.211 (0.371) -0.073 (0.760) WST Out Confidence (ρ (p)) x x 0.760** (0.000) 0.617** (0.004) 0.258 (0.273) 0.337 (0.146) 0.388 (0.091) 0.155 (0.514) WST In Confidence (ρ (p)) x x x 0.675** (0.001) 0.227 (0.337) 0.194 (0.411) 0.144 (0.546) 0.017 (0.942) WST-Q Capacity (ρ (p)) x x x x 0.398 (0.082) 0.050 (0.835) 0.014 (0.953) -0.154 (0.516) WST-Q Confidence (ρ (p)) x x x x X 0.632* (0.003) 0.030 (0.900) -0.071 (0.768) WST-Q Frequency (ρ (p)) x x x x x x 0.423 (0.063) 0.065 (0.785) LSA Scooter (ρ (p)) x x x x x x x 0.472* (0.031) 46 3.4.2 Participant perspective on the representativeness of the WST items (objective 3.2) The perceptions of participants in the study supported the representativeness of the WST; the content analysis revealed that participants perceived both environments to be similar - 45% of participants thought they performed the same in both environments. As Samantha noted, “They feel similar and they use the same techniques in either one.” In contrast, 40% perceived themselves to perform better in the outdoor community-based environment, while 15% thought they performed better indoors. Some of these participants attributed their perceived improved performance in the community due to a sense of familiarity. Samantha explained: “Because in the community I’ve done all these things before. So I’m more familiar with them. If you take me to a different locality I didn’t know it might have been different, it’s just the familiarity of my own neighbourhood." Another participant explained how familiarity contributes to a perceived improved score: "I perceive there is more room, but you’re saying it's not, so that might be the familiarity" (Rachel, scooter user of 2.5 years). Six participants in the study noted a lack of distraction in the indoor lab-based environment. Lisa summarized, “[The] inside is very controlled. The floors are more smooth, you know, the inside I can’t- as far as [built challenges] it’s very accurate. It’s just the other factors we are missing.” A few participants discussed the impact of bright weather in the environment. Kevin, a scooter user of 11 years, described: Outdoors, I mean, sometimes the sun is blinding. One thinks one goes for instance off the sidewalk onto the street level through a ramp or whatever but there is no ramp [laugh] and that’s where one kind of gets a rude awakening. The most commonly reported outdoor distractions were moving cars and pedestrians. Jarrod noted: 47 As you noticed there were a few cars trying to get into the laneway all of a sudden, so I guess […] [a difference] between the two [environments] would be you got to pay attention more so, clearly outside […] as opposed to when we were inside. While many participants noted outdoor distractions, some concurrently reported that outdoor distractions were not an issue, while others thought it simply did not impact their performance. Elise described her experience outside in this way: “I never paid attention at all - I didn’t feel that there were any distractions at all.” 3.4.3. Participants’ comparisons of the indoor lab-based versus outdoor community-based environment (objective 3.2) In the interviews, participants discussed how reflective the indoor lab-based course was of the outdoor community-based environment, and debated which environment allowed for a more accurate performance of skills. The discussion of representativeness arose in 17 interviews. Five participants found the indoor lab-based course to be representative of the outdoor community-based environment, while 12 participants noted that the indoor environment was “similar to things that I have accommodate [in the community]” (John). Jarrod described, “Given what you’ve set up, I thought it was it was very good, you’ve done a good job of setting it up to the best that you can, obviously for an outdoor typical experience, as far as setting it up indoors.” John, a scooter user of 20 years, supported Jarrod’s notion by further explaining, “Well I thought it really does emulate what we deal with on a pretty daily basis - things that aren’t level, things are kind of... yeah, like the [slight and steep slopes and the cross slope].” Many participants in the study thought that going up the various ramps, going in and out of the bathroom, and turning in a small space were very useful skills to know, and thought it was well simulated indoors. The one skill that participants debated was going over gravel. The gravel used in the indoor lab-based course is relatively large, and is unlike most soft surfaces (i.e., grass, 48 small/packed gravel) encountered in their communities. Samantha explained the various factors that a couple of other participants also considered when driving on gravel: “I couldn’t do the ditch and I didn’t wanna do the gravel. Even if it [the gravel] were smaller it depends how thick it is and how fast you’re going at it.” This resulted in a fear of tipping in some participants. Elise, a scooter user of 11 years, who was currently borrowing a scooter while hers was getting repaired, explained: “The only reason I didn’t go on the gravel and that sort of thing is because the scooter. Because this scooter is pretty light in [comparison] to the one I have. Mine’s heavy, and I feel like I would tilt on it with this one.” There were mixed perceptions of which environment provided a more accurate assessment of skills. Seven participants thought the indoor lab-based environment provided a more accurate assessment, while six participants thought the outside was a more accurate assessment, and four participants thought both environments were accurate. One reason why participants perceived the indoor lab-based course to be more accurate was that they found the indoor lab-based course to be more encompassing of a variety of challenges. Brandon explained: [The indoor is more accurate because there are] more challenges and you can… outside you are limited by the, where you happen to be and what challenges are available. Here you could create a regular minefield of challenges because you’re controlling it. In addition, some participants viewed the indoor lab-based course to be more challenging, perhaps because it was a novel setting. Brad, a scooter user of 5 years, explained how he was more challenged in the indoor lab-based environment: “Because it [the indoor course] was more complex, more challenges in terms of making turns, going on different angles. I’m more used to being outside. And when I’m inside, I’m not used to the situation like at ICORD.” In contrast, a couple of participants thought the outdoor community-based environment was a more accurate 49 assessment of skills because they found the community setting to be more relevant. Jill explained, “Because that’s what I do all the time so, it’s more accurate, for example, watching for potholes. There’s more to watch for, but it’s more familiar.” Content analysis revealed that participants were almost evenly split in regards to which environment they preferred performing the WST in; seven participants (39%) preferred the indoor lab-based environment, six (33%) preferred the outdoor community-based environment, and five (3%) did not have a preference. Many participants preferred performing the WST indoors as it provided more variation in skills than their community setting. Kyle, a scooter user for 14 years, explained: “It was more extreme, more different. Where around the house wasn’t much.” In addition, participants preferred the indoor lab-based environment due to its novelty. These participants found the indoor lab-based WST to be more “fun” and interesting: “[I preferred the indoor environment] just because it was more interesting... the neighborhood I’ve done lots of time” (Paul, a scooter user of 2 years). Meanwhile, other participants preferred the outdoor community-based environment due a perception of it being “easier”. One participant noted that they preferred the outdoor community-based environment due to the indoor lab-based setting feeling like too much of a test setting: “Well, it’s more what I’m used to, so being tested in the environment at PARC, it kind of feels like a test” (Sandra). 3.4.4. Participants’ perspectives on confidence (objective 3.3) In addition to the confidence ratings provided in each environment and on the WST-Q, participants were asked about their confidence in the interviews. A content analysis revealed that one participant felt more confident inside (5%), eight (40%) felt more confident outdoors, and nine (45%) felt equally confident. The sole participant who felt more confident in the indoor lab- 50 based environment had the least experience with using their scooter. For him, the indoor lab-based environment was preferred given that there were fewer distractions, and found the outdoor distraction to be a hindrance to their testing. An often reported reason for why participants felt more confident in the outdoor community-based environment was familiarity. Samantha explained: “I mean my neighborhood is very familiar to me so I know what's coming up.” A common reason for feeling equally confident was related to the perception that skill testing performance came down to experience – if you are experienced enough, you can test well in any environment. When one participant was asked why they felt the same level of confidence in both environments, he explained, “I guess it’s just a confidence in using the scooter” (Jarrod, scooter user of 4 years). Another participant noted, “The more confident you are, the better [scooter user] you are going to be” (Sandra). When one participant was asked about which environment they felt more confident performing in, she thought that confidence was not comparable in the different situations. She explained: They’re very different. I don’t think they’re comparable. I think they were equal because the indoor one was novel [and] the outdoor one is very familiar, [but] the indoor had no distractions, the outdoor one has distractions. So it’s kind of equal and balanced (Jill) 3.4.5 Participants’ suggestions for the WST (objective 3.4) While the indoor lab-based testing sites were perceived to be fairly representative of their communities, participants also had many suggestions for improving the course, including the modification of the course and the addition of some skills. Many of these suggested skills were functional (i.e., used to access the community), and included tasks such as navigating busy/tight spaces and accessing various doors. See table 3.7 below for a complete list. 51 Table 3. 7 Suggested WST Improvements (objective 3) Suggestion Number of participants who mentioned the suggestion Approaching fast moving doors (elevator simulation) 2 Making the ramp longer 3 Adding surface textures (i.e., rougher, bumpier) 1 Angled curb cuts 2 Maneuvering through a crowd 3 Navigating a tight space 2 Opening a weighed door 1 Steeper hills 2 Turning out of a dead end 2 3.5 Determining the applicability of performing the WST outdoors (objective four) The participants in this study resided in various neighbourhoods around the Lower Mainland, including areas in Vancouver, Richmond, Burnaby, New Westminster, Coquitlam, Port Moody, and North Vancouver. The number of neighbourhoods sampled in each city is listed below. Table 3. 8 Number of participants in each city in the Lower Mainland City Number Vancouver 12 Burnaby 2 North Vancouver 1 New Westminster 1 Coquitlam 1 Port Moody 1 Richmond 1 The details taken of the environmental measures can be found in Table 3.9. Most of the community-based testing occurred on sunny days, and the skills in the outdoor community-based 52 environment were similar to the indoor lab-based skills. On average, it took 10 minutes to explore the neighbourhoods to identify skills within a one-block radius. While the time to complete the WST in both environments was not recorded, all participants thought that the timing of the tests were reasonable. Table 3. 9 Environmental measures Variable Mean (SD)/N [%] Actual/Recommended Value (Kirby et al., 2015) Weather during outdoor assessment Sunny Cloudy Rainy 13 [65] 6 [30] 1 [5] Surface Wet Dry 15 [75] 5 [25] Mean long distance (m) 114.47 (67.67) 100 Slight inclines found 14 [70] Mean slight incline (°) 4.7 (1.54) 5 Mean slight length (m) 8.89 (10.14) 2.5 Steep inclines found 19 [95] Mean steep incline (°) 7.48 (2.28) 10 Mean steep length (m) 7.67 (8.66) 2.5 Cross slopes found 20 [100] Mean cross slope incline (°) 5.45 (2.07) 5 Mean cross slope length (m) 5.48 (7.85) 2.5 Mean cross slope width (m) 3.07 (1.98) 1.5 Curbs found 13 [65] Mean height (cm) 4.96 (0.94) 5 Soft surfaces found 17 [85] Soft surface used: Grass 16 [59] Gravel Mean length (m) 7.18 (3.89) 2 Abbreviations: ° = degrees, m = meters, cm = centimeters 53 3.5.1 Participants’ perception on whether certain skills are required on the WST (objective four) When scoping community settings for similar WST tasks, two of the most uncommonly found skills were low curbs and potholes. This raised the question of whether these skills should still be tested for on the WST. When asked in the interview, 15 participants thought that the curb skill was still good to know and 13 participants still thought the pothole skill was important. Brad noted that sometimes you do not have a choice when you encounter these obstacles: Yeah, I mean you have to… you try and pick the most um… accessible course when you’re going from A to B but you often don’t have a choice. Sometimes you have disruptions like the construction so you have to take the alternative route, which isn’t exactly what you were looking for. So I do think you have skills and training or whatever on all different kinds of terrains Only one participant thought the pothole skill was unnecessary, but only because they viewed it as being “not applicable” to scooter users (i.e., the scooter itself is unable to go over a pothole). Two other participants shared this view of the low curb skill. Six participants also noted that when they encountered potholes or curbs, they would often avoid them or go around them if possible. One participant suggested the importance of avoidance over performance: “No, it’s more avoidance, than anything else. The real skill set to learn is how to get these things [scooters] out [of potholes when you are in the situation]” (Brandon). In addition, a couple of participants discussed how the pothole skill in the indoor lab-based environment does not fairly reflect potholes in the community, but did simulate going over train tracks – a skill which was sometime encountered. 54 3.5.2 Participants’ perception on safety (objective four) There were differing perceptions of safety among the participants when performing the WST in the two environments. Content analysis revealed that 11 participants felt equally safe, 2 felt safer outdoors, and 5 felt safer indoors. Among those who did not feel equally as safe in both environments, more participants felt safer in the indoor lab-based environment. One reason was the perception of more available help in the case of mishap or emergency, as described by Kyle, who stated: “If I fall down there is a lot of people to help, find help easier”. Another reason for feeling safer inside was related to the perception that it was a more controlled environment. Jill explained: “There were no cars, there was nothing else to keep track of.” Kyle related this sense of safety to his sense of confidence and willingness to try more skills. For example, Kyle felt safer inside, and therefore only performed the pothole skills indoors but not in the community. He emphasized, “If you see a hole [outside] you might not even think about trying it - you aren’t going to try it outside.” Interestingly, most of the participants who felt safer in the indoor lab-based environment also preferred performing the WST indoors as well. One participant experienced an accident in the study when performing the WST in the indoor lab-based course during the “gets from ground into scooter” skill (where the participant is required to lower themselves to the ground and get back into their scooter). Although it was emphasized that he did not have to perform a skill should he feel uncomfortable, he was determined to attempt the skill. When trying to lower himself down onto the ground, the strength in his arm gave out. This resulted in him falling on this arm and breaking it. When I conversed with the participant, he explained that he had felt confident, but had over-estimated his abilities. To prevent future incidents from happening, this skill was removed from further testing. While the WST manual provides instructions for safety, it does not indicate how to determine whether 55 it is safe for a particular individual of a given ability to perform this skill. In this case, the participant indicated he was confident in performing this skill but fell and sustained an arm fracture, which required medical intervention and rehabilitation. This participant, however, still completed the WST in both environments, and his scores were used in analysis. 3.5.3 The perceived benefits of performing the WST (objective four) Participants perceived the WST to be beneficial for experienced and new users. Although most participants had many years of scooter experience, the WST provided a reaffirmation of their skills; some participants were surprised by their abilities to use the scooter. This is illustrated when Angela explained that the WST “Was good, it was very, very good. [The WST] gives me a positive outlook, you know, ‘cause when you ask me to [first] do these [skills] I wasn’t sure if I knew how to do them, so it’s good, I enjoyed it.” A couple of participants also explained that they are more likely to try out a skill they are unsure about since they felt safer. Kyle noted: “Outside I probably wouldn’t have gone on those rocks because I don’t know if my wheels will go - you think twice about it outside.” Participants also highlighted how the WST would be beneficial for new uses. Mary, a scooter user of 35 years, expressed: “I think it was a very good test, I think everyone should go through it when you are starting out with a scooter.” A few participants noted that the indoor lab-based test was better for training as it encompassed a larger variation in skills that could not always be found in the immediate neighbourhood. In contrast, other participants noted that the indoor lab-based situation feels more “test-like”, and argued that training in the outdoors was more realistic due to all the common distractions: “I think it’s [the distractions] good because you learn to handle the scooter in different situations” (Jill). 56 CHAPTER 4: DISCUSSION This is one of the first studies we are aware of that compares the performance on an indoor standardized scooter skills assessment course to an outdoor community-based setting among scooter users. Given the significant difference with the parametric test and trend with the non-parametric test, it appears that Skills testing scores in the indoor lab-based and outdoor community-based environments are not comparable. However,, participants perceived that the indoor lab-based version of the wheelchair skills test accurately reflected a community-based setting. The findings from this study are strengthened by combining quantitative and qualitative data; this study not only compares performance quantitatively, but also explored the participants’ perspectives on performance in the two environments. The qualitative data in this study were used to help explain and elaborate on findings from the quantitative data. The participants appeared to be similar to those in previous studies demographically. On average, participants in the present study were older adults (i.e., average of 67 years old), and the majority were female. The demographics of the study sample is comparable to the demographics of scooter users across Canada, in which 63% of scooter users were reported to be women, and the average age of scooter users is 68 (Smith et al., 2017). A bidirectional relationship between the environment and person (i.e., their performance level) was highlighted by participants who used travel scooters. Participants who used a travel scooter encountered more challenges, both in the indoor-lab based environment and outdoor community-based environment. While participants’ may possess the resource characteristics required for proficient scooter use, the environment must also be conducive to performance. For example, in this study, travel scooters were not powerful enough to navigate through the indoor lab-based soft surface, while the traditional scooters had no issue. In order to maximize 57 performance, participants should have the proper skills, and the environment should be as accessible as possible for all devices. 4.1 Comparing performance between the indoor lab-based and outdoor community-based environments. Scores of the indoor lab-based WST and the WST-Q were normally distributed in this study, but there were questions about the skewness of the outdoor community-based WST scores. As a rule of thumb, if twice the standard error is less than the skewness value, the distribution is considered normal (IBM Corporation, 2012). Although WST scores from the outdoor community-based environment were statistically significantly skewed, the absolute skewness value was not greater than twice the value of the standard error. In addition, the power of the Shapiro-Wilk test is low for sample sizes less than 50 (Razali and Yap, 2011), as in the present study. When the distribution of the data is skewed towards the extremes, the t-test becomes less robust to type I (Sawilowsky and Blair, 1992). Given that the scores from the outdoor community-based environment were skewed, the chances of obtaining a false positive (i.e., a type I error) is increased. This may explain why scores were significantly different using the parametric test (i.e., a t-test) opposed to a non-parametric test (i.e., the Wilcoxon Signed Rank test), although a trend towards significance was noted with the non-parametric test. However, given questions about the degree of skewness this raises additional questions regarding the need for non-parametric testing. The size of the sample would also increase the change of Type I error. Ultimately, although the results are mixed, the trend in the non-parametric testing combined with the significant difference the parametric testing appear to not support the hypothesis for the first objective in that the indoor lab-based WST scores and the outdoor community-based WST scores are not statistically significantly different. 58 This trend in skewness may have been a result of the absence of the more challenging items, such as curbs and potholes. If these items were present for outdoor testing, the scores may have been lower. Higher scores in the absence of the more difficult WST items may suggest that scooter performance is based on the interdependency of an individual’s skill level and the environment. Whereas capacity can be an indicator of an individuals’ resource characteristics (i.e., their skill, knowledge, and expertise with scooter use), capacity is also influenced by the environment. For example, an environment that is too narrow for a scooter to go through cannot be accessed, regardless of the skill level of the scooter operator. The scores on the indoor lab-based WST in the present study were high, and are slightly higher than the average scores found in a previous study (mean 87.3, standard deviation, 7.5) (Mortenson et al., 2017). The indoor lab-based WST scores may have been higher in the present study as 45% of the participants had previous experience with the indoor lab-based course. This previous exposure may have led to less performance fear and anxiety, resulting in improved performance. The WST-Q scores in the present study were high; however, scores were not higher than what a previous study on experienced scooter users found (mean 91.3, standard deviation 8.33). A ceiling effect was observed for outdoor community-based WST scores, but not the indoor lab-based. Although the indoor lab-based scores did not meet the requirement of the definition of a ceiling effect (i.e., ≥ 20% of the sample attaining the maximum score), 2 participants obtained the maximum score. One reason for the presence of a ceiling effect may be due to the scoring system of the WST. In the version used for this study (4.2), participants were scored on a scale from 0-2 (fail, pass with difficulty, pass). In the newest version of the WST (5.0), individuals are scored on a scale from 0-3 (fail, partial pass, pass, advanced pass). Including a greater range in scores may help reduce the ceiling effect and may allow for a more 59 precise score. The presence of a ceiling effect in the outdoor community-based setting may imply that testing experienced scooter users in a familiar environment may not accurately discriminate between those with good scooter skills and those with poor scooter skills. In the outdoor community-based setting, a high score may reflect the scooter user’s familiarity with the environment rather than their scooter skills; a familiar setting does not seem to challenge scooter users as much in comparison to a novel setting This effect of a high average score is also seen among the confidence scores. While a ceiling effect was not found for confidence based on the definition by de Vet et al. (2011) (i.e., ≥20% of participants obtaining the maximum score), mean confidence scores were still high (i.e., 15% obtained the maximum score). This may be related to the participants’ long experience of scooter use. The absence of a ceiling effect is in line with previous findings among a sample of powered wheelchair users (Rushton et al., 2014). According to previous WST literature, a clinically meaningful difference in WST scores is 20% (Kirby et al., 2004). Given that the limits of agreement in the present study is 43.87%, it is suggested that using the WST in indoor lab-based and outdoor community-based environments may not be used interchangeably; however, each setting may be useful for different purposes. The outdoor community-based setting may be a better testing site for individuals who are looking to transition from rehabilitation back into the community, given that the community is a setting in which the scooter will be used frequently. However, testing in an unfamiliar environment may be beneficial for training new scooter users to navigate unfamiliar environments. Recently, the topic of potentially requiring a license to drive a scooter has been brought to light. Should this occur, an unfamiliar test site may also be more beneficial for skills testing. 60 The Bland-Altman analysis also revealed a slight systematic bias – on average, participants scored slightly better in the outdoor community-based environment versus the indoor lab-based environment. The improved performance in an outdoor community-based setting may be related to higher confidence levels in the outdoor community-based setting and feeling familiar with the environment. Since the outdoor WST was conducted in a familiar setting to participants (i.e., right outside of their homes), many participant felt comfortable manoeuvring through obstacles and using their scooter skills as they have could practice these challenges over time. The WST scores in the indoor lab-based and outdoor community-based environments were statistically significantly different. Familiarity may have also played a role in this finding. 4.2 Participants’ experiences with scooter use The participants in the present study used their scooters similarly to scooter users in other studies. Previous research has noted that many participants use their scooters to travel longer distances (Smith et al., 2016), and often use them to engage in community activities such as shopping, socializing with friends, and going to medical appointments (Brandt et al., 2004; May et al., 2010; Fomiatti et al., 2013). Additionally, the meanings ascribed to scooter use is similar to previous findings in that individuals often perceive that using a scooter promotes their sense of independence and ability to partake in community activities (Brandt et al., 2004; May et al., 2010; Samuelsson and Wressle, 2014). Thoreau (2015) noted that there has been scarce information on the satisfaction and wellbeing of scooters, and there has been little research since. A contribution of this study included the hindrances highlighted by the participants, which impact their overall satisfaction with their device use. This study identified a potential issue that may arise from scooter use including the lack of suspension on scooters, which results in a bumpier ride, and thus causes or worsens soreness and pain. Further research is suggested to 61 observe the hindrances experienced by scooter users, and to explore other negative aspects of using scooters. As scooter users and powered wheelchair users both rely on a battery-powered device, inclement weather has been posed as a barrier in the present study and has been supported by previous studies. Some challenges previously reported in using these powered mobility devices in inclement weather include feeling cold (especially with the hands), spinning out in the snow, and frozen batteries (Ripat et al., 2015). Snow has particularly been identified as a barrier that can result in reduced outings for these powered mobility users, or even cause them to become homebound (Fomiatti et al., 2013; Mortenson et al., 2015; Ripat et al., 2015). While it has been recommended by some scooter manufactures (Pride Mobility, 2006) to not use scooters in inclement weather, many individuals still opt to do so. Additional skills for using a scooter in the rain and/or snow should be considered in a future revision of the WST. Obstacles in the built environment were nothing new to scooter users. Previous studies have observed negative experiences held by scooter users in the built environment, specifically in regards to accessing buildings (e.g., grocery stores, bathrooms) and public transportation (Fomiatti et al., 2013; May et al., 2010). While skills training is suggested to improve performance in the community (Kirby et al., 2015), some of these issues can only be addressed by a change in societal practice. The hindrances highlighted by the participants in this study illustrate how scooter mobility is influences by a variety of factors that are not directly related to wheelchair skills. For example, the lack of accessibility in indoor built spaces may not become any more accessible if the scooter is unable to enter the space to begin with; however, improved scooter skills might facilitate access to places that are marginally more accessible. 62 4.3 Messick’s Evidence for Validity 4.3.1 Analysis from the Spearman Correlation The results from the Spearman correlation did not fully agree with the a priori hypotheses. The correlations did support the hypothesis that the outdoor community-based scores would be more strongly correlated with the outcome measures than the indoor lab-based scores. This may reflect a greater shared variance between the indoor lab-based environment and the outdoor community-based environment; that is, both environments are similar. However, the outdoor community-based correlations were not all positive as expected. While the outdoor community-based WST scores were positively and strongly correlated with WST indoor confidence and subjective capacity, they were weakly correlated with the WST-Q total scores, WST-Q confidence, and the WST-Q frequency domain, and not positively correlated with either of the LSA scores. The low correlations may have been noise in the data, as the spearman correlation values were close to zero. In line with a previous measurement properties study conducted on scooter users and the WST that found the WST was positively and statistically significantly correlated to the WST-Q (r = 0.547, p = 0.013) (Mortenson et al., 2017), the present study also found that indoor skills testing was moderately correlated to the total WST-Q score; however, it was only statistically significant on the frequency domain. This could indicate that scooter users do not use their full repertoire of skills to perform activities in the communities; however, if an individual is more skilled, they may be more likely to perform the skill more frequently. Given that the outdoor community-based setting were often missing skills (particularly the pothole and low curb), the indoor lab-based setting may be indicated of a higher level of skill should a scooter user be able to score highly on the WST. 63 The LSA had a weak correlation to scores in both environments and to confidence exhibited in both environments, while the LSA scooter score has a small correlation to the WST scores in both environments, and was not correlated to confidence. The directionality of the correlation did not agree with the a priori hypothesis; however, the outdoor community-based WST scores did have a greater correlation magnitude for the LSA frequency than the indoor lab-based score. Again, the low correlation may be related low variation in score; however, this indicates that the frequency people travelled with their scooter is not related to scooter skills or confidence. This contradicts the findings by Sakakibara et al. (2014), who found a relationship between confidence and life spaces travelled amongst manual wheelchair users. The scenario may differ among scooter users, as many scooter users rely on multiple devices given that they are often used during a period of transitioning health (Thoreau, 2015). In the present study, only 3 participants relied on their scooter full time for mobility; many participants were still able to walk short distances with the use of a mobility aid (e.g., walker, cane, walking poles). The majority of participants used their scooters in their neighbourhoods or to travel longer distances. This may reflect the high scores on the WST, but low scooter use as most of the participants did not fully rely on their scooters to get around. The low correlation may have additionally been related to ceiling effects, which has been defined as having more than 20% of the sample attaining the maximum score (de Vet et al., 2011). In addition, scores were negatively skewed which demonstrated that scores were clustered at the higher end of the range of scores. The low variations in score may have resulted in a weak correlation. 64 4.3.2 Participants’ perspectives on the representativeness of the WST As noted in the introduction, the representation aspect of validity is often judged by experts. In a sense, the participants in the present study may be regarded as “expert” scooter users given that they have been using their scooters for an average of ~7 years. Most participants believed that the indoor lab-based course was representative of the outdoor community-based course; however, some felt that the indoor lab-based course was unable to fully and accurately recreate an outdoor community-based setting. Some aspects mentioned may be addressed in future refinement of an indoor lab-based skill course, such as more textured surfaces; however, factors such as wind, rain, and fast moving objects (e.g., people, cyclists, dogs) may be more difficult to recreate. In this regard, performing the WST in the outdoor community-based setting may have better substantive evidence for ecological validity. 4.3.3 Participants’ sense of confidence The participants’ sense of confidence was further explored in the first theme of the thematic analysis, the impact of confidence. While one participant felt more comfortable in the indoor lab-based environment, the majority felt more confident in the community-based setting. According to Bandura’s self-efficacy theory (1977), one factor that plays a role in one’s level of confidence is mastery experience, or how proficient one is at a task. Given that the many of the participants frequently used their scooters in their communities, mastery experience of using a scooter outdoors may have played a role in their confidence outside. The one participant reported above who was more confident in the indoor lab-based environment was a newer user who was only beginning to use their scooter more often. Given the lack of experience, a quieter, distraction free environment may have been preferred. Bandura’s concept of mastery experience may also explain why some participants felt equally confident in both environments. While some 65 participants were equally confident due to being experienced with their scooter, Jill’s belief that confidence is environment specific may potentially be related to Bandura’s notion that self-efficacy is situation specific (1977); each environment has its own challenges that may impact confidence. Among the participants, there were mixed opinions about which environment provided a more accurate assessment of skills. The Wheelchair Skills Test was originally developed for manual wheelchair users, and has been expanded to include powered wheelchairs and scooters. With this history, construct underrepresentation and construct-irrelevant difficulty may be present in the WST for scooter users, as the skill level of powered mobility device users partially lays in the capability of the device. For example, the skill of “turns in place” (i.e., turning within a 1.5 meter x 1.5 meter box) (Kirby et al, 2015) may be difficult for some scooters given that they can be up to 1.4 meters in length. Additionally, other skills such as ‘going over a soft surface’ may be difficult for some scooter users as the motor of the scooter is not powerful enough to propel the user through coarse gravel; instead, some scooters may spin out. However, Kirby et al. (2015) found that the skills on the WST have good content evidence for construct validity. In support of this, some participants in the current study argued that the indoor lab-based course was more representative of the scooter skills required for community participation, as it was more encompassing of a variety of skills. Although the majority of skills were found in the outdoor community-based setting, most often 1-2 skills would be missing (usually the pothole or low curb skills). Some participants preferred to perform the WST in the indoor lab-based environment as it allowed them to demonstrate their full range of abilities. Although content evidence is provided with testing in the indoor lab-based environment, this may not hold true for the outdoor community-based environment. Other participants in the present study 66 argued that the outdoor community-based environment provided better representation; it was more representative of the environment in which they would normally be using their scooter. In this regard, these participants viewed the outdoor community-based environment as the more applicable skills testing environment with better ecological validity. The WST manual suggests that the test should be conducted in a setting that is “quiet, private, free from distractions and well lit” (Kirby et al., 2015, p. 24); however, some participants in the present study thought that the indoor lab-based setting was lacking natural distractions (e.g., sounds of cars/people, bright sun light, cyclists). Some participants thought that learning to perform skills in a busy environment might be more transferrable to a community setting. While standardization is important in research to compare and generalize results, unless the WST is being used to compare scores for research, a quiet, standardized environment may not be the best for testing. In general, a controlled measure may be highly structured (i.e., more controlled) or low in structure. The former is more reliable, but the latter is more representative of real life (Domholdt, 2005). This resonates with Messick’s (1995) concept of construct validity. As noted in the introduction, Messick’s (1995) notion of content and substantive as sources of evidence requires a measure: 1) to have set boundaries of the construct domains that will be assessed, and 2), to include tasks that are substantive, or in a real-life setting with meaning. The WST in the outdoor community-based setting provides a substantive setting for the skills setting; in this regard, it may be seen to have good construct validity. In addition, participants thought that the indoor lab-based WST course was representative of the outdoor community-based community, and found that the skills that were tested were all good ones to know. As described in the introduction, this is evidence of construct validity, as Messick (1995, 67 p. 742) defines construct validity as “an integration of any evidence that bears on the interpretation or meaning of test scores.” 4.4 Determining the applicability of the WST in an outdoor community-based setting A unique aspect of this study was the observation of the applicability of the WST in an outdoor, community-based setting. Based on the findings, the WST is feasible to conduct in the community given that it took on average 10 minutes to find most of the obstacles on the WST, and that ≥ 95% of items could be found in all communities. These findings supported the a priori hypotheses made in that ≥90% of WST items were found in the community and that all skills were within ±15% of the measurements of the indoor lab-based course.. The ease of finding all items in the community is in line with the findings reported by Best et al. (2005); assessors in that study found reasonably comparable skills in and around the participants’ homes. The two obstacles that were the most challenging to find in neighbourhoods were potholes and low curbs. While a lack of potholes and low curbs is a positive feature of the natural environment, participants still thought that these skills were important to know on the rare occasion that they did come across these obstacles. The ease of finding most items in the outdoor community-based environment may suggest low examiner burden. Although clinicians/examiners would have to factor in an additional 10 minutes to their assessment time should they wish to perform the WST in the community, it may reduce the burden of travelling for scooter users, who may then be more encouraged to schedule a time to perform the test. Another caveat to performing the WST in an outdoor community-based setting may be an increased cost of health care; the additional time required by clinicians to travel to the communities of their clients and to conduct the WST in the outdoor community-based setting may ultimately result in greater costs for health care services. 68 Most of the obstacles found in the outdoor community-based setting were similar to the indoor challenges. The angles of the slopes found in the outdoors were similar to the ones present in the indoor lab-based course; however, they were longer. An exception to this was the steep slopes found in the community; they were slightly less steep than the slopes presented indoors. Although the low curb was not commonly found, those in the community were very close to the low curb in the lab. Given that the indoor lab-based setting was developed to train new wheelchair and scooter users the skills to safely navigate the community (Kirby et al., 2015), it can be argued that the indoor lab-based course should be more difficult than the community-based setting. One major difference in obstacles was the type of soft surface used. In both indoor research sites, (coarse) gravel was used, while in the outdoor, community-based setting, grass was used as a soft surface for all but one participant. Many participants found that the gravel used in the indoor setting was too large and potentially even dangerous; it had been suggested to use turf instead. However, it is also important to note that the use of coarse gravel is specific to the study; the WST manual provides options for a soft surface, which includes a gym mat, sand, or carpet over open cell foam (Kirby et al., 2015). In the interviews, participants generally agreed that the indoor lab-based course reflected the outdoor community-based setting. The WST was originally developed to assess the skills required to safely use a manual wheelchair in everyday life (Kirby et al., 2015). As such, many of the obstacles in the indoor lab-based course are meant to simulate the outdoor community-based environment. Many participants thought the WST did a good job of this, as many thought the test encompassed the various skills required in their everyday lives. This finding may represent good format compatibility, per the criteria described by Auger et al. (2006), as the 69 participants perceived the skills in the WST to be applicable to their everyday use of their scooters. Performing the WST outdoors involved low response burden. All participants thought that the WST was performed in a reasonable amount of time and did not feel any physical threats when performing the WST outdoors; however, participants did report a varying sense of safety in the different environments. Although the majority of participants did not perceive any difference in feelings of safety between environments, among the rest, many preferred the indoor lab-based environment. One reason for preferring the indoor lab-based environment includes feeling more reassured as there was a perception of more help available in the indoor lab-based environment. This may be because both indoor lab-based testing sites are located within a hospital setting, with doctors and other clinicians in close proximity should an accident occur. The only accident that occurred in this study was in the indoor lab-based environment, and not in the outdoor community-based environment. Additionally, this participant expressed a sense of confidence in performing the skill they were unable to successfully perform. This sense of confidence could be related back to the notion that some participants felt more confident in the indoor lab-based environment and were willing to try new skills due to a perceived comfort in knowing there is help nearby if needed. Participants discussed in their interviews how performing the WST benefitted them despite years of experience, and their thoughts on the importance of the WST. This indicates a low respondent burden, and a good respondent acceptability given that the scores were meaningful to the participants as well. A main benefit participants reported from performing the WST was a reaffirmation of skills. Despite years of experience, some participants learned that their scooters could perform skills (e.g., go up/down curbs) that they did not think it could do. 70 This may be related to the Heisenberg principle, which when applied in a social science setting, suggests that the act of observation changes the object observed (Smith, 2005). In this sense, the participant’s skill levels may have been influenced from being observed. One reason for this change may be related to the participants’ sense of safety in a testing situation; some participants were willing to try more skills that they would not normally attempt given the presence of a researcher. In this sense, the act of observation may positively impact participants’ performance abilities; some participants were pleasantly surprised in their new-found scooter abilities. According to the definition provided by Auger et al. (2006), performing the WST in an outdoor community-based setting generally has good applicability. It exhibits low response and examiner burdens, and suggests good formal compatibility. The one applicability criteria that was not met was having a normal score distribution. In the present study, the scores were negatively skewed as a ceiling effect was observed. Although this may indicate that performing the WST in an outdoor community-based setting may not be able to capture lower scores, as reported above, the inflation of scores may have been related to the great amount of experience the participants had with their scooters, and because they were highly familiar with the outdoor environment. 4.5 Personal Reflection As previously noted, I had prior connections with multiple participants prior to their participation in the study. During the interview, the participants who I had previous connections with seemed more comfortable during the interview; however, they may have assumed that I have had pre-existing knowledge of their scooter use, and required more probing. For example, when asking about a typical day of using the scooter, some participants provided truncated responses and required additional probing (e.g., “I believe you also partake in dragon boating, 71 can you tell me how your scooter helps you with that?”). In contrast, participants whom I had no previous experience with generally discussed more openly and in-depth about their scooter use and experiences. For example, one participant seemed to view me as someone who could potentially influence change, and was therefore eager to discuss issues with the test and with their scooter use more generally rather than discussing the topic at hand (i.e., his experience with the WST). This may impact the dependability of the study, as the interviews may not have been consistent across participants given their previous history with me. Secondly, my positive bias of scooter use may have impacted my interview skills around asking participants how the use of a scooter has been beneficial them. More specifically, I failed to go more in depth in asking some participants why they thought their scooters were helpful, and accepted simple responses such as “it’s just amazing” (Lisa). This may have influenced the credibility of the study, as my bias may have influenced my interviewing and my analysis of the data. CHAPTER 5. CONCLUSION 5.1 Limitations and Future directions This study encountered several limitations. These involved a limited variation in scores, having an unstandardized outdoor testing environment, lacking specific environmental details, a confounding variable of familiarity, and including a large variation in scooter design. To begin, there was a limited variation in scores; since all of the participants in this study were experienced scooter users, a ceiling effect was present in the study. Having scores clustered at one end of the range may have affected the correlations observed. A modification that could have been made to this study to account for the factor of familiarity is to include new scooter users, although this may influence the homogeneity of the sample. 72 Second, while the indoor lab-based skills test was standardized, the outdoor community-based test environment was not. While all participants completed the same types of obstacles in the community, the test environment was unique to each participant. Some environmental factors that varied per environment include the weather, cars present, and the number of pedestrians on the street. Specific details of the weather were not documented, as it was categorized as “sunny, cloudy, or rainy” and “wet or dry” surfaces. Although this was documented, this does not consider the type of wet weather. For example, one participant performed the WST after a week of snow, and the ground was cold, frozen and wet. This may have impacted their performance on the WST. In this study, a confounding variable of familiarity had arisen in the study as the outdoor community-based testing took place in the participants’ neighbourhoods. Should this study be conducted again, it would be advisable to conduct the WST in the same outdoor community-based environment that is unfamiliar to participants (e.g., the community outside of the research lab, and have an exclusion criteria of individuals who have been to the neighbourhood before). Lastly, a large variation of scooters were used in this study, which may have impacted performance on the WST. For example, some scooters were too large to complete some tasks, such as turning around in a tight space. As a result, WST scores may not accurately reflect a participants’ true skill level as it is partially dependent on the ability of the scooter as well. The results from the present study suggest that outdoor community-based training may be feasible, given that the obstacles found in the community-based setting are very similar to those found in an indoor lab-based setting. Further research is required to determine the measurement properties of the WST in a community-based environment (e.g., reliability), and whether the 73 findings from this study are applicable for manual and powered wheelchair users as well. In addition, further work could be conducted to map out how the specific skills in the WST translate to community-based activities. 5.2 Conclusion This is one of the first studies to compare performance of the WST in an indoor lab-based versus community setting, and to compare community-based challenges to the indoor lab-based built-course. The findings from this study suggest that skills testing in indoor lab-based and outdoor community-based settings are not comparable, although these findings may have been influenced by the distribution of the outdoor WST scores. Community testing may be a better reflection of experienced users’ day-to-day skills; however, it might also be helpful to assess scooters users in unfamiliar settings if we want to encourage users to use the devices in novel settings. Participants in this study thought highly of the WST, and suggested that new scooter users should go through the test when they first get their scooters. 5.2 Relevance and Application In the literature, the WST is primarily used as a research tool; however, this study informs the clinical utility of the tool. Most of the reported WST-based studies conducted the WST in indoor standardized lab-based environments. This is an ideal set up for research as it is consistent; however, a standardized environment may not be optimal for all clinical purposes. Although clinicians would be required to take more time with their assessments, conducting the WST may be appropriate and feasible in an outdoor community-based setting. Testing in the community may provide the clinician a better sense of what skills a new scooter may need to improve on to functionally operate their devices in and around their homes. Given the increasing 74 usage of scooters, it is important that these individuals receive the training required for safe and effective use, which is guided by appropriate pre-post testing. This may ultimately facilitate their community reintegration. 75 References Auger, C., Demers, L., & Swaine, B. (2006). Making sense of pragmatic criteria for the selection of geriatric rehabilitation measurement tools. Archives of gerontology and geriatrics, 43(1), 65-83. Auger, C., Demers, L., Gélinas, I., Miller, W. C., Jutai, J. W., & Noreau, L. (2010). Life-Space Mobility of Middle-Aged and Older Adults at Various Stages of Usage of Power Mobility Devices. Archives of Physical Medicine and Rehabilitation, 91(5), 765–773. Australian Competition and Consumer Commission. (2012). Mobility scooter usage and safety survey report, (September), 1–10. Baker, P. S., Bodner, E. V., & Allman, R. M. (2003). Measuring Life-Space Mobility in Community-Dwelling Older Adults. Journal of the American Geriatrics Society, 51(11), 1610–1614. Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioural change. Psychological review, 84(2), 191-215. Barker, D. J., Reid, D., & Cott, C. (2004). Acceptance and meanings of wheelchair use in senior stroke survivors. American Journal of Occupational Therapy, 58(2), 221–230. Best, K. L., Kirby, R. L., Smith, C., & MacLeod, D. A. (2005). Wheelchair skills training for community-based manual wheelchair users: A randomized controlled trial. Archives of Physical Medicine and Rehabilitation, 86(12), 2316–2323. Best, K. L., Kirby, R. L., Smith, C., & Macleod, D. A. (2006). Comparison between performance with a pushrim-activated power-assisted wheelchair and a manual wheelchair on the Wheelchair Skills Test. Disability and Rehabilitation, 28(4), 213-220. Bland, J. M., & Altman, D. G. (2003). Applying the right statistics: analyses of measurement studies. Ultrasound in Obstetrics and Gynecology, 22(1), 85-93. Bottari, C., Dutil, É., Dassa, C., & Rainville, C. (2006). Choosing the most appropriate environment to evaluate independence in everyday activities: Home or clinic?. Australian Occupational Therapy Journal, 53(2), 98-106. Boucher, P., Atrash, A., Kelouwani, S., Honoré, W., Nguyen, H., Villemure, J., ... & Pineau, J. (2013). Design and validation of an intelligent wheelchair towards a clinically-functional outcome. Journal of Neuroengineering and Rehabilitation, 10(1), 58. Brandt, A., Iwarsson, S., Stahle, A. (2004). Older people's use of powered wheelchairs for activity and participation. Journal of rehabilitation medicine, 36(2), 70-77. 76 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. Bronfenbrenner, U. (1995). The bioecological model from a life course perspective: Reflections of a participant observer. In P. Moen, G. H. Elder, & K. L¨uscher (Eds.), Examining lives in context: Perspectives on the ecology of human development (599-618). Washington, DC: American Psychological Association. Bronfenbrenner, U., & Morris, P. A. (1998). The ecology of developmental processes. In W. Damon & R.M.Lerner (Eds.), Handbook of child psychology, Vol. 1: Theoretical models of human development (993 – 1023). New York: Wiley. Bubolz, M.M., & Sontag, M.S. (1993). Human ecology theory. In P.G. Boss, W.J. Doherty, R. LaRossa, W.E. Schumm, & S.K. Steinmetz (Eds.), Sourcebook of family theories and methods: a contextual approach (419-448). New York: Plenum Press. Carter, S.L. (2009). Organization of social validity data. In The social validity manual: a guide to subjectiev evaluation of behaviour interventions in applied behaviour analysis (211-223). London, UK: Academic Press. Charbonneau, R., Kirby, R. L., & Thompson, K. (2013). Manual wheelchair propulsion by people with hemiplegia: within-participant comparisons of forward versus backward techniques. Archives of Physical Medicine and Rehabilitation, 94(9), 1707-1713. Chen, W. Y., Jang, Y., Wang, J. Der, Huang, W. N., Chang, C. C., Mao, H. F., & Wang, Y. H. (2011). Wheelchair-related accidents: relationship with wheelchair-using behavior in active community wheelchair users. Archives of Physical Medicine and Rehabilitation, 92(6), 892–898. Clark, A. M. (1998). The qualitative-quantitative debate: moving from positivism and confrontation to post-positivism and reconciliation. Journal of advanced nursing, 27(6), 1242-1249. Cohen, J. (1992). A Power Primer. Psychological Bulletin, 112(1), 155–159. De Vet, H. C. W., Terwee, C. B., & Bouter, L. M. (2003). Current challenges in clinimetrics. Journal of Clinical Epidemiology, 56(12), 1137–1141. Dunn, W., Brown, C. & McGuigan, A. (1994). The ecology of human performance: A framework for considering the effect of context. American Journal of Occupational Therapy, 48(7), 595-607. Dutta, T., King, E.C., Holliday, P.J., Gorski, S.M., Fernie, G.R. (2010). Disability and Rehabilitation: Assistive Technology, 6(1), 67-76. Elo, S., Kyngas, H. (2007). The qualitative content analysis process. Journal of advanced nursing, 62(1), 107-115. 77 Fava, G. A., & Belaise, C. (2005). A discussion on the role of clinimetrics and the misleading effects of psychometric theory. Journal of Clinical Epidemiology, 58(8), 753–756. Fava, G. A., Tomba, E., & Sonino, N. (2012). Clinimetrics: The science of clinical measurements. International Journal of Clinical Practice, 66(1), 11–15. Fehr, L., Langbein, W. E., & Skaar, S. B. (2000). Adequacy of power wheelchair control interfaces for persons with severe disabilities: a clinical survey. Journal of Rehabilitation Research and Development, 37(3), 353–60. Feinstein, A. R. (1983). The Development of Clinimetrics. Annuals of Internal Medicine, 99:843-848. Feinstein, A. R. (1987). Clinimetric perspectives. Journal of Chronic Diseases, 40(6):635-640. Furmaniuk, L., Cywińska-Wasilewska, G., & Kaczmarek, D. (2010). Influence of long-term wheelchair Rugby training on the functional abilities of persons with tetraplegia over a two-year period post-spinal cord injury. Journal of Rehabilitation Medicine, 42(7), 688-690. Giesbrecht, E. M., Miller, W. C., Eng, J. J., Mitchell, I. M., Woodgate, R. L., & Goldsmith, C. H. (2013). Feasibility of the Enhancing Participation In the Community by improving Wheelchair Skills (EPIC Wheels) program: study protocol for a randomized controlled trial. Trials, 14(1), 350. Giesbrecht, E. M., Wilson, N., Schneider, A., Bains, D., Hall, J., & Miller, W. C. (2015). Preliminary evidence to support a “boot camp” approach to wheelchair skills training for clinicians. Archives of Physical Medicine and Rehabilitation, 96(6), 1158-1161. Hoenig, H., Pieper, C., Branch, L. G., & Cohen, H. J. (2007). Effect of Motorized Scooters on Physical Performance and Mobility: A Randomized Clinical Trial. Archives of Physical Medicine and Rehabilitation, 88(3), 279–286. Hoppes, S., Davis, L. A., & Thompson, D. (2003). Environmental effects on the assessment of people with dementia: a pilot study. The American journal of occupational therapy, 57(4), 396-402. Hosseini, S. M., Oyster, M. L., Kirby, R. L., Harrington, A. L., & Boninger, M. L. (2012). Manual wheelchair skills capacity predicts quality of life and community integration in persons with spinal cord injury. Archives of Physical Medicine and Rehabilitation, 93(12), 2237–2243. Hsieh, H.F., Shannon, S.E. (2005). Three approaches to qualitative content analusis. Qualitative health research, 15(9), 1277-1288. Hubbard, S. L., Fitzgerald, S. G., Vogel, B., & Reker, D. M. (2007). Distribution and cost of wheelchairs and scooters provided by Veterans Health Administration. Journal of Rehabilitation Research and Development, 44(4), 581. 78 IBM Corporation. (2012). Display statistics. Retrieved from: https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.modeler.help/dataaudit_displaystatistics.htm Jung, H. S., Park, G., Kim, Y. S., & Jung, H. S. (2015). Development and evaluation of one-hand drivable manual wheelchair device for hemiplegic patients. Applied Ergonomics, 48, 11-21. Karmarkar, A.M., Dicianno, B.E., Cooper, R., Collins, D.M., Matthews, J.T., Koontz, A., et al. (2011). Demographic profile of older adults using wheeled mobility devices. Journal of Aging Research, 1-11. Kilkens, O. J. E., Post, M. W. M., Dallmeijer, A. J., Seelen, H. a M., & van der Woude, L. H. V. (2003). Wheelchair skills tests: a systematic review. Clinical Rehabilitation, 17, 418–430. Kirby, R. L., Swuste, J., Dupuis, D. J., MacLeod, D. A., & Monroe, R. (2002). The Wheelchair Skills Test: A pilot study of a new outcome measure. Archives of Physical Medicine and Rehabilitation, 83(1), 10–18. Kirby, R. L., Dupuis, D. J., MacPhee, A. H., Coolen, A. L., Smith, C., Best, K. L., … Bonaparte, J. P. (2004). The wheelchair skills test (version 2.4): Measurement properties. Archives of Physical Medicine and Rehabilitation, 85(5), 794–804. Kirby, R. L., Adams, C. D., MacPhee, A. H., Coolen, A. L., Harrison, E. R., Eskes, G. A., ... & Dupuis, D. J. (2005). Wheelchair-skill performance: controlled comparison between people with hemiplegia and able-bodied people simulating hemiplegia. Archives of Physical Medicine and Rehabilitation, 86(3), 387-393. Kirby, R.L., Smith, C., Parker, K., McAllister, M., Boyce, J., Rushton, P.W.,… Brandt A. (2015). The Wheelchair Skills Program Manual. Published electronically at Dalhousie University, Halifax, Nova Scotia, Canada. www.wheelchairskillsprogram.ca/eng/manual.php Kirby, R. L., Miller, W. C., Routhier, F., Demers, L., Mihailidis, A., Polgar, J. M., … Sawatzky, B. (2015). Effectiveness of a Wheelchair Skills Training Program for Powered Wheelchair Users: A Randomized Controlled Trial. Archives of Physical Medicine and Rehabilitation, 96(11), 2017–2026. Kirby, R. L., Mitchell, D., Sabharwal, S., McCranie, M., & Nelson, A. L. (2016). Manual wheelchair skills training for community-dwelling veterans with spinal cord injury: A randomized controlled trial. PLoS ONE, 11(12), 1–20. Koontz, A.M., Brindle, E.D., Kankipati, P., Feathers, D., Cooper, R.A. (2010). Design features that affect the maneuverability of wheelchairs and scooters. Archives of Physical Medicine and Rehabilitation, 91(5), 759-64. Kroll, T., Neri, M.T., Miller, K. (2005). Using Mixed Methods in Disability an Rehabilitation Research. Rehabilitation Nursing, 30(3), 106-113. 79 Lemay, V., Routhier, F., Noreau, L., Phang, S. H., & Ginis, K. M. (2012). Relationships between wheelchair skills, wheelchair mobility and level of injury in individuals with spinal cord injury. Spinal Cord, 50(1), 37-41. Lindquist, N.J., Loudon, P.E., Magis, T.F., Rispin, J.E., Kirby, R.L., Manns, P.J. (2010) Reliability of the performance and safety scores of the wheelchair skills test version 4.1 for manual wheelchair users. Archives of Physical Medicine and Rehabilitation, 91(11), 1752-1757. MacGillivray, M. K., Sawatzky, B. J., Miller, W. C., Routhier, F., & Kirby, R. L. (2017). Goal satisfaction improves with individualized powered wheelchair skills training. Disability and Rehabilitation: Assistive Technology, 1-4. Messick, S. (1989). Meaning and values in test validation: the science and ethics of assessment. Educational Researcher, 18(2), 5-11. Messick, S. (1995). Validity of psychological assessment: validation of inferences from persons' responses and performance as scientific inquiry into score meaning. American Psychologist, 50(9), 741-749. Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52(2), 250-260. Mortenson, W. B., Miller, W. C., Boily, J., Steele, B., Odell, L., Crawford, E. M., & Desharnais, G. (2005). Perceptions of power mobility use and safety within residential facilities. Canadian Journal of Occupational Therapy, 72(3), 142–152. Mortenson, W. B., Miller, W. C., & Auger, C. (2008). Issues for the selection of wheelchair-specific activity and participation outcome measures: A review. Archives of Physical Medicine and Rehabilitation, 89(6), 1177-1186. Mortenson, W. B, Demers, L., Fuhrer, M. J., Jutai, J. W., Lenker, J., & DeRuyter, F. (2012). How Assistive Technology Use by Individuals with Disabilities Impacts Their Caregivers. American Journal of Physical Medicine and Rehabilitation, 91(11), 984–998. Mortenson, W.B., Hammell K.W., Luts, A., Soles, C., Miller, W.C. (2015). The power of power wheelchairs: mobility choices of community-dwelling older adults. Scandinavian journal of occupational therapy, 22(5), 394-401. Mortenson, W.B., & Kim, J. (2016). Scoping review of mobility scooter-related research studies. Journal of Rehabilitation Research and Development, 53(5), 531–540. Mortenson, W.B., Hurd Clarke, L., Goldsmith, C.H., Jang, S., Kirby, R.L. (2017). Measurement properties of the Wheelchair Skills Test for scooters among experienced users. Disability and Rehabilitation: Assistive Techology, 0(0), 1-6. Mountain, A. D., Kirby, R. L., & Smith, C. (2004). The wheelchair skills test, version 2.4: Validity 80 of an algorithm-based questionnaire version. Archives of Physical Medicine and Rehabilitation, 85(3), 416–423. Mountain, A. D., Kirby, R. L., Eskes, G. A., Smith, C., Duncan, H., MacLeod, D. A., & Thompson, K. (2010). Ability of people with stroke to learn powered wheelchair skills: a pilot study. Archives of Physical Medicine and Rehabilitation, 91(4), 596-601. Newton, A. M., Kirby, R. L., MacPhee, A. H., Dupuis, D. J., & MacLeod, D. A. (2002). Evaluation of manual wheelchair skills: Is objective testing necessary or would subjective estimates suffice? Archives of Physical Medicine and Rehabilitation, 83(9), 1295–1299. Pettersson, I., Törnquist, K., & Ahlström, G. (2006). The effect of an outdoor powered wheelchair on activity and participation in users with stroke. Disability and Rehabilitation: Assistive Technology, 1(4), 235-243. Portney, L.G., & Watkins, M.P. (2015). Foundations of Clinical Research. Philadelphia: F.A. Davis Company Phang, S. H., Martin Ginis, K. a., Routhier, F., & Lemay, V. (2012). The role of self-efficacy in the wheelchair skills-physical activity relationship among manual wheelchair users with spinal cord injury. Disability and Rehabilitation, 34(8), 625–632. Pride Mobility. (2006). Owner's Manual. Retrieved from: https://www.pridemobility.com/pdf/owners_manuals/intl_scooter/uk/uk_destiny_om.pdf Raykov, T., & Marcoulides, G.A. (2017). Introduction to Psychometric Theory. New York: Routledge. Razali, N.M., & Wah, Y.B. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. Journal of statistical modeling and analytics, 2(1), 21-23. Ripat, J.D., Brown, C.L., Ethans, K.D. (2015). Barriers to wheelchair use in the winter. Archines of physical medicine and rehabilitation, 96(6), 1117-1122. Rosa, E.M., & Tudge, J. (2013). Urie Bronfenbrenner's theory of human development: Its evolution from ecology to bioecology. Journal of Family Theory and Review, 5(4), 243-258. Rushton, P. W., Kirby, R. L., & Miller, W. C. (2012). Manual wheelchair skills: Objective testing versus subjective questionnaire. Archives of Physical Medicine and Rehabilitation, 93(12), 2313–2318. Rushton, P.W., Kirby, R.L., Routhier, F., Smith, C. (2014). Measurement properties of the wheelchair skills test-questionnaire for powered wheelchair users. Disability and Rehabilitation: Assistive Technology, 11(5), 400-406. 81 Samuelsson, K., & Wressle, E. (2014). Powered wheelchairs and scooters for outdoor mobility: a pilot study on costs and benefits. Disability and Rehabilitation: Assistive Technology, 9(4), 330-334. Sakakibara, B. M., Miller, W. C., Souza, M., Nikolova, V., & Best, K. L. (2013). Wheelchair skills training to improve confidence with using a manual wheelchair among older adults: A pilot study. Archives of Physical Medicine and Rehabilitation, 94(6), 1031–1037. Sakakibara, B. M., Miller, W. C., Eng, J. J., Backman, C. L., & Routhier, F. (2014). Influences of wheelchair-related efficacy on life-space mobility in adults who use a wheelchair and live in the community. Physical Therapy, 94(11), 1604-1613. Sakakibara, B. M., Miller, W. C., Routhier, F., Backman, C. L., & Eng, J. J. (2014). Association between self-efficacy and participation in community-dwelling manual wheelchair users aged 50 years or older. Physical Therapy, 94(5), 664-674. Saltan, A., Bakar, Y., & Ankarali, H. (2017). Wheeled mobility skills of wheelchair basketball players: a randomized controlled study. Disability and Rehabilitation: Assistive Technology, 12(4), 390-395. Sawilowsky, S. S., & Blair, R.C. (1992). A more realistic look at the robustnesss and type II error properties of the t Test to departures from population normality. Psychological bulletin, 111(2), 352-360. Shumway-Cook, A., & Woollacott, M. H. (1985). The growth of stability: postural control from a developmental perspective. Journal of motor behavior, 17(2), 131-147. Smith, E. M., Sakakibara, B. M., & Miller, W. C. (2014). A review of factors influencing participation in social and community activities for wheelchair users. Disability and Rehabilitation. Assistive Technology, 3107(September), 1–14. Smith, E.M., Giesbrecht, E.M., Mortenson, W.B., Miller, W.C. (2016). Prevalence of wheelchair and scooter use among community-dwelling Canadians. Physical Therapy, 96(8), 1135-1142. Smith, E.M., Low, K., Miller, W.C. (2017). Interrater and intrarater reliability of the wheelchair skills test version 4.2 for power wheelchair users. Disability and Rehabilitation, 1-6. Stemler, S. (2001). An overview of content analysis. Practical assessment, research, and evaluation, 7(17), 1-10. Streiner, D. L. (2003). Clinimetrics vs. psychometrics: An unnecessary distinction. Journal of Clinical Epidemiology, 56(12), 1142–1145. Statistics Canada. (2017). Age and sex, and type of dwelling data: Key results from the 2016 Census The Daily Statistics canada Catalogue no. 11-001-X. Statistics Canada Catalogue No. 11-001-X, Ottawa. Retrieved from http://www.statcan.gc.ca/daily- 82 quotidien/170503/dq170503a-eng.pdf Stoker, K., Heinz, A., Baran, R., Sato, N., Kohl, M. (2012). Environmental contexts: considerations for therapy. Practice theory CATS. https://commons.pacificu.edu/theory/2 Thurmond, V. A. (2001). The point of triangulation. Journsal of Nursing Scholarship, 33(3), 253–258. 83 Appendices Appendix 1. Literature search on the Wheelchair Skills Test Authors Title Year Type (P = Psychometrics, T = training, O = Outcome measure) WST Version Population Al Lawati Z, Kirby RL, Smith C, Mackenzie D, theriault C, Matheson K Getting MWC over a threshold using the momentum method: a descriptive study of common errors 2017 O WST 214 AB students put into MWC Kirby RL, Worobey LA, Cowan R, Presperin-Pedersen J, Heinemann AW, Dyson-Hudson TA, et al WC Skills Capacity and performance of MWC users with SCI 2016 O WST MWC; 117 Kirby RL, Dupuis DJ, MacPhee AH, Coolen AL, Smith C, Best KL, et al The WST (v. 2.4): Measurement Properties 2004 P WST MWC; 169; AB; 129 Lindquist NJ, Loudon PE, Magis TF, Rispin JE, Kirby RL, Manns PJ Reliability of the performance and safety scores of the WST 4.1 for MWC 2010 P WST MWC; 11; 4 raters Mortenson WB, Hurd Clarke L, Golsmith CH, Jang S, Kirby RL Measurement properties of the WT for scooter users 2017 P WST Scooter; 20 84 Kirby RL, Swuste J, Dupuis DJ, MacLeod DA, Monroe R WST: Pilot study of a new outcome measure 2002 P WST (1.0) 24 MWC Best KL, Kirby RL, MacLeod DA Comparison between performance with a pushrim-activated power-assisted wheelchair and a MWC on the WST 2006 O WST (2.4) MWC; 30 able-bodied individuals Furmanuik L, Cywinska-Wasilewska G, Kaczzmeaek D Influence of long-term wheelchair rugby training on functional abilities of persons with tetraplegia over a 2 year period post SCI 2010 O WST (2.4) MWC; 40 male tetralegia Kirby RL, Adams CD, MacPhee AH, Coolen AL, Harrison ER, Eskes GA, Smith C, et al. WC Skill performance: controlled comparison between people with hemiplegia and able-boded people simulating hemiplegia 2005 O WST (2.4) MWC; 4 (20 AB, 20 WC with hemiplegia) Nelson AL, Groer S, Palacios P, Mitchell D, Sabharwal S, Kirby RL, Gavin-Dreschnack D, et al. WC -Related falls in veterans with SCI residing in the community: a prospctive cohot study 2010 O WST (2.4) 702 SCI (MWC, PWC, and scooter) Sakakibara BM, Miller WC, Souza M, Nikova V, Best, KL Wheelchair skils training to impove confidence with using a MWC among older adults: a pilot study 2013 O WST (2.4) 20 MWC Pradon D, Pinsault N, Zory R, Routhier F Could mobility performance measures be used to evaluate WC skills? 2012 O WST (3.2) MWC; 40 with Sci 85 Kirby RL, Walker R, Smith C, Macleod DA, Thompson K MWC handling skills by caregivers using new and conventional rear-anti-tip devices: a RCT 2009 T/O WST (3.2) MWC; 16 caregiver/WC user dyads Boucher P, Atrash A, Kelouwani S, Honore W, Nguyen H, Villemure J, Routhier F, Cohen P, Demers L, Forget R, Pineau J Design and validation of a intelligent wheelchair towards a clinically-functional outcome 2013 O WST (4.1) PWC; 17 Charbonneau R, Kirby RL, Thompson K MWC Propulsion by people with hemiplegia: within-participant comparisons of forward vs backwards techniques 2013 O WST (4.1) MWC; 18 hemiplegic Giesbrecht EM, Miller WC, Eng JJ, Mitchell IM, Woodgate RL, Goldsmith Ch Feasibility of the Enhancing Participation in the Community by improving WC Skills (EPIC Wheels) program: study protocol for a RCT 2013 O WST (4.1) MWC; 40 Hosseini SM, Oyster ML, Kirby RL, Harrington AL, Boninger ML MWC Skills capacity predicts QoL and community integration in persons with SCI 2012 O WST (4.1) MWC; 214 SCI Lema V, Routhier F, Noreau L, Phang SH, Ginis KA Relationships between WC skills, WC mobility, and level of injury in individuals with SCI 2012 O WST (4.1) MWC; 54 Nagy J, Winslow A, Brown JM, Adams L, O'Brien Pushrim kinetics during advanced WC skills in MWC with SCI 2012 O WST (4.1) MWC; 23 86 K, Boninger M, Nemuanitis G Phang SH, Martin Ginis KA, Routhier F, Lemay V The role of self-efficacy in the WC skills- PA relationship among MWC with SCI 2012 O WST (4.1) MWC; 54 Sakakibara BM, Miller WC Prevalence of low mobility and self-managemetn self-efficacy in MWC users and the association with WC skills 2015 O WST (4.1) MWC; 123 Sakakibara BM, Miller WC, Eng JJ, Backman CL, Routhier F Preliminary examination of the relation between participation and confidence in older MWC users 2013 O WST (4.1) MWC; 54 Saltan A, Ankarali H The role of trunk stabilzation in functional classification levels 2016 O WST (4.1) MWC; 113 Sawatzky B, Hers N, MacGillivray MK Relationships between wheeling parameters and wheelchair skills in adults and children with Sci 2015 O WST (4.1) MWC; 16 adults, 8 children Smith C, Kirby Rl MWC Skills caacity and safety of residents of a long term care facility 2009 O WST (4.1) MWC; 13 Sorrento GU, Archambault PS, Routhier F, Dessureault D, Boissy P Assessment of joystick control during the performance of powered wheelchair driving tasks 2011 O WST (4.1) PWC; 26 (10 expert, 13 novice) Ozturk, A, Dokuztug, U. Effectiveness of a WST program for community-living users of MWC in Turkey: a RCT 2011 T WST (4.1) MWC (community dwelling); 24 87 Morgan KA, Tucker SM, Klaesner JW, Engsberg JR A motor learning approach to training wheelchair propulsion biomechanics for new MWC users: A pilot study 2017 O WST (4.2) MWC; 6 SCI Jung HS, Park G, Kim YS, Jung HS Development and evaluation of one-hand drivable MWC device for hemiplegic patients 2015 O WST (4.2) MWC; 30 Smith EM, Low K, Miller WC Interrater and intrarater reliability of the WST v. 4.2 for PWC users 2017 P WST (4.2) PWC; OTs; 10 Mortenson WB, iller WC, Polgar JM Measurement properties of the LLDI among individuals who use PWC as their primary means of mobility 2014 O WST-P (4.1) PWC; 115 Giesbrecht EM, Wilson N, Schneider A, Bains D, Hall J, Miller WC Preliminary evidence to support a boot camp approach to wheelchair skills training for clinicians 2015 O WST-Q MWC; 65 OT students Mountain D, Kirby RL, Smith C WST 2.4 - Validity of the WST-Q 2014 P WST-Q MWC; 20 Newton AM, Kirby RL, MacPhee AH, Dupuis DJ, MacLeod DA Evaluation of MWC Skills: Is objective testing necessary or would subjective estimates suffice? 2002 P WST-Q MWC; 21 and their therapist Sakakibara BM, Miller WC, Eng JJ, Backman CL, Routhier F Influences of WC-related efficacy on life space mobility in adults who use a WC and live in the community 2014 O WST-Q (4.1) MWC; 124 Sakakibara BM, Miller WC, Routhier F, Association between self-efficacy and participation in community-dwelling MWC aged 50+ 2014 O WST-Q (4.1) MWC; 124 88 Backman CL, Eng JJ Saltan A, Bakar Y, Ankarali H Wheeled mobility skills of WC basketball players: a RCT 2017 O WST-Q (4.1) 111 players, 85 non-players (MWC) Inkpen P, Parker K, Kirby Rl Manual Wheelchair Skills Capacity versus Performance 2012 O/P WST-Q (4.1) MWC; 26 Rushton PW, Kirby RL, Routhier F, Smith C Measurement properties of the WST-Q for PWC 2014 P WST-Q (4.1) PWC; 72 Rushton PW, Kirby RL, Miller WC MWC Skills: Objective Testing Versus Subjective Questionnaire 2012 O/P WST-Q (4.1), WST (4.1) MWC (community dwelling); 89 Rushton PW, Labbe D, Demers L, Miller WC, Mortenson WB, Kirby RL Understanding the burden experienced by caregivers of older adults who use a PWC: a cross sectional tudy 2017 O WST-Q (caregivers) 35 caregivers of PWC users Best KL, Kirby RL, Smith C, MacLeod DA WS Training for Community-Based MWC Users: A RCT 2005 T WSTP MWC; 15M, 5F Kirby RL, Mitchell D, Sabhawwal S, McCranie M, Nelson AL MWC Skills training for community-dwelling veterants with SCI: A RCT 2016 T WSTP MWC; 106 SCI (vets) MacPhee AH, Kirby RL, Coolen AL, Smith C, MacLeod DA, Dupuis DJ WSTP: A RCT of WC users undergoing initial rehabilitation 2004 T WSTP MWC; 35 Worobey LA, Kirby RL, Heinemann AW, Effectiveness of group WC skills training for people ith SCI: A RCT 2015 T WSTP MWC; 114 SCI 89 Krobot EA, Dyson-Hudson TA, Cowan RE, et al. Coolen AL, Kirby RL, Landry J, MacPhee AH, Dupis D, Smith C, et al. WC Skills training program for clinicans - a RCT with OT students 2004 T WSTP (2.4) MWC; 82 OT students Kirby RL, Mifflen NJ, Thibault DL, Smith C, Best KL, Thompson KJ, MacLeod DA The MWC handling skills of caregivers and the effect of training 2004 T WSTP (2.4) MWC; 24 (caregivers) Mountain AD, Kirby RL, Eskes GA, Smith C, Duncan H, MacLeod DA, Thompson K Ability of people with stroke to learn PWC skills: a pilot study 2010 O WSTP (3.2) PWC; 10 with stroke Routhier F, Kirby RL, Demers L, Depa M, Thompson K Efficacy and retention of the french-canadian version of the WSTP for MWC users: a RCT 2012 O WSTP (3.2) MWC; 39 MacGillivary MK, Sawatzky BJ, Miller WC, Routhier F, Kirby RL Goal satisfaction improves with individualized PWC skills training 2017 O WSTP (4.1) PWC; 17 Kirby RL, Miller WC, Routhier F, Demers L, Mihailidis A, Effectiveness of a Wheelchair Skills Training Program for Powered Wheelchair Users: A Randomized Controlled Trial 2015 T WSTP (4.1) PWC; 116 90 Polgar JM, Rushton PW, et al Mountain AD, Kirby RL, Smith C, Eskes G, Thompson K Powered wheelchair skills training for persons with stroke: a RCT 2004 T WSTP (4.1) PWC; 20 Filess-Douer O, Vanlandewijck YC A systematic reviw of WSTs for MWC users with SCI: towards a standardized outcome measure 2010 Abbreviations: O = observational, M = measurement properties, T = training, MWC = manual wheelchair, AB = able bodied, PWC = power wheelchair, WST = Wheelchair Skills Test, WST-Q = Wheelchair Skills Test-Questionnaire, WSTP = Wheelchair Skills Training Program 91 Appendix 2: Reporting Guidelines 2.1. STROBE STROBE Statement—checklist of items that should be included in reports of observational studies Item No Recommendation Page Title and abstract 1 (a) Indicate the study’s design with a commonly used term in the title or the abstract 1 (b) Provide in the abstract an informative and balanced summary of what was done and what was found 2 Introduction Background/rationale 2 Explain the scientific background and rationale for the investigation being reported 1-10 Objectives 3 State specific objectives, including any prespecified hypotheses 10 Methods Study design 4 Present key elements of study design early in the paper 11 Setting 5 Describe the setting, locations, and relevant dates, including periods of recruitment, exposure, follow-up, and data collection 13-14 Participants 6 (a) Cohort study—Give the eligibility criteria, and the sources and methods of selection of participants. Describe methods of follow-up Case-control study—Give the eligibility criteria, and the sources and methods of case ascertainment and control selection. Give the rationale for the choice of cases and controls Cross-sectional study—Give the eligibility criteria, and the sources and methods of selection of participants 12-13 (b) Cohort study—For matched studies, give matching criteria and number of exposed and unexposed Case-control study—For matched studies, give matching criteria and the number of controls per case n/a Variables 7 Clearly define all outcomes, exposures, predictors, potential confounders, and effect modifiers. Give diagnostic criteria, if applicable 13-14 Data sources/ measurement 8* For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group n/a Bias 9 Describe any efforts to address potential sources of bias 15 Study size 10 Explain how the study size was arrived at 13 Quantitative variables 11 Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why 17 Statistical methods 12 (a) Describe all statistical methods, including those used to control for confounding 17 (b) Describe any methods used to examine subgroups and interactions 17 (c) Explain how missing data were addressed n/a (d) Cohort study—If applicable, explain how loss to follow-up was addressed n/a 92 Case-control study—If applicable, explain how matching of cases and controls was addressed Cross-sectional study—If applicable, describe analytical methods taking account of sampling strategy (e) Describe any sensitivity analyses Results Page Participants 13* (a) Report numbers of individuals at each stage of study—eg numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up, and analysed 27 (b) Give reasons for non-participation at each stage 27 (c) Consider use of a flow diagram n/a Descriptive data 14* (a) Give characteristics of study participants (eg demographic, clinical, social) and information on exposures and potential confounders 27 (b) Indicate number of participants with missing data for each variable of interest n/a (c) Cohort study—Summarise follow-up time (eg, average and total amount) 27 Outcome data 15* Cohort study—Report numbers of outcome events or summary measures over time 28-29, Case-control study—Report numbers in each exposure category, or summary measures of exposure n/a Cross-sectional study—Report numbers of outcome events or summary measures n/a Main results 16 (a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, 95% confidence interval). Make clear which confounders were adjusted for and why they were included 33-36 (b) Report category boundaries when continuous variables were categorized n/a (c) If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period n/a Other analyses 17 Report other analyses done—eg analyses of subgroups and interactions, and sensitivity analyses 19 Discussion Key results 18 Summarise key results with reference to study objectives 48 Limitations 19 Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias 61-62 Interpretation 20 Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence 61-62 Generalisability 21 Discuss the generalisability (external validity) of the study results 62 Other information Funding 22 Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based n/a 93 2.2 COREQ Number Item Guide questions/description Page number Domain 1: Research team and reflexivity Personal Characteristics 1. Interviewer/facilitator Which author/s conducted the interview or focus group? 21 2. Credentials What were the researcher's credentials? E.g. PhD, MD 21 3. Occupation What was their occupation at the time of the study? 21 4. Gender Was the researcher male or female? 21 5. Experience and training What experience or training did the researcher have? 21 Relationship with participants 6. Relationship established Was a relationship established prior to study commencement? 25 7. Participant knowledge of the interviewer What did the participants know about the researcher? e.g. personal goals, reasons for doing the research 25 94 Number Item Guide questions/description Page number 8. Interviewer characteristics What characteristics were reported about the interviewer/facilitator? e.g. Bias, assumptions, reasons and interests in the research topic 25-27 Domain 2: study design Theoretical framework 9. Methodological orientation and Theory What methodological orientation was stated to underpin the study? e.g. grounded theory, discourse analysis, ethnography, phenomenology, content analysis 20-25 Participant selection 10. Sampling How were participants selected? e.g. purposive, convenience, consecutive, snowball 15 11. Method of approach How were participants approached? e.g. face-to-face, telephone, mail, email 15 12. Sample size How many participants were in the study? 15 95 Number Item Guide questions/description Page number 13. Non-participation How many people refused to participate or dropped out? Reasons? n/a Setting 14. Setting of data collection Where was the data collected? e.g. home, clinic, workplace 21 15. Presence of non-participants Was anyone else present besides the participants and researchers? 21 16. Description of sample What are the important characteristics of the sample? e.g. demographic data, date 20-21, 27 Data collection 17. Interview guide Were questions, prompts, guides provided by the authors? Was it pilot tested? 22 18. Repeat interviews Were repeat interviews carried out? If yes, how many? n/a 19. Audio/visual recording Did the research use audio or visual recording to collect the data? 22 20. Field notes Were field notes made during and/or after the interview or focus group? 24 96 Number Item Guide questions/description Page number 21. Duration What was the duration of the interviews or focus group? 21 22. Data saturation Was data saturation discussed? 23. Transcripts returned Were transcripts returned to participants for comment and/or correction? 25 Domain 3: analysis and findings Data analysis 24. Number of data coders How many data coders coded the data? 23 25. Description of the coding tree Did authors provide a description of the coding tree? 85-87 26. Derivation of themes Were themes identified in advance or derived from the data? 23 27. Software What software, if applicable, was used to manage the data? 23 28. Participant checking Did participants provide feedback on the findings? TBA Reporting 29. Quotations presented Were participant quotations presented to illustrate the themes / findings? Was each 30-32, 38-47 97 Number Item Guide questions/description Page number quotation identified? e.g. participant number 30. Data and findings consistent Was there consistency between the data presented and the findings? 31. Clarity of major themes Were major themes clearly presented in the findings? 30-31 32. Clarity of minor themes Is there a description of diverse cases or discussion of minor themes? 38-47 2.3 GRAMMS Guideline Section: page Describe the justification for using a mixed methods approach to the research question Methods: p. 14 Describe the design in terms of the purpose, priority and sequence of methods Methods: p. 14 Describe each method in terms of sampling, data collection and analysis Methods (quantitative data collection): p.17-19 Methods (quantitative analysis): p. 19-21 Methods (qualitative data collection): p.20-22 Methods (qualitative data analysis): p.23-24 Describe where integration has occurred, how it has occurred and who has participated in it Discussion: 47 Describe any limitation of one method associated with the present of the other method Describe any insights gained from mixing or integrating methods Discussion: p.48 98 Appendix 3: Measures Demographics Sociodemographics Section 1: Personal Information 1) Age:(years) ______________ 2) Country of Birth: ________________ 3) Gender: 1= Male 2=Female 4) Do you live? (circle all that apply) 1 = alone 2 = with family (eg. Spouse, children, siblings, parents) 3 = with friends/roommates 4 = with a paid caregiver 5 = other please specify: _______________________________ 5) Highest Education Level: 1 = no formal education 2 = primary or elementary school 3 = high school 4 = college/university/trade school 5 = post - graduate school 6 = other Please specify other: 6) Current Employment Status: (circle all that apply) 1 = employed 2 = unemployed 3 = volunteer 4 = student 5 = retired 7) Do you ever have difficulty getting around your home without the help of devices or other people? Section 2: Mobility 99 1=No 2=Yes If yes, 7a) What do you rely on to help you move around your home? 1= an assistive device 2= personal assistance 3= both 8) Do you ever have difficulty getting around your community without the help of devices or other people? 1=No 2=Yes If yes, 8a) What do you rely on to help you move around your community? 1= an assistive device 2= personal assistance 3= both 9) During a typical week do you use: No Yes No Yes 9a. cane 1 2 9f. scooter 1 2 9b. crutches 1 2 9g. walker without wheels 1 2 9c. medically prescribed footwear 1 2 9h. 2 wheeled walker 1 2 9d. orthosis (leg brace) 1 2 9i. 4 wheeled walker with seat 1 2 9e. prosthesis (artificial leg) 1 2 9j. manual wheelchair 1 2 9k. power wheelchair 1 2 9) What is your primary assistive device you use for mobility? (write the item 9a-9j) ___________ By your primary device we mean the one that you use most often, or are most dependent on, for mobility. 10) When did you begin using your primary assistive device? (YYYY/MM) _______/_______ 11) On average, how often do you use your device? 1 = 0 – 1 days per week 2 = 2 – 3 days per week 3 = 4 – 5 days per week 4 = 6 – 7 days per week 11a) What was the source of funding for your device? 100 1 = Private 2 = Insurance plan (eg. Blue Cross, Workers Compensation Benefits) 3 = Government funding 4 = Other. Please specify:____________________________________ Section 3: Diagnoses 12) Primary health problem accounting for scooter use (circle one) a. Spinal cord injury e. Multiple Sclerosis b. Stroke f. Arthritis c. Chronic Obstructive Pulmonary Disease (COPD) g. Other d. Amputation If other, please specify: _______________________________ 12a) Number of years you have had this problem: _________ 13) Please list any other health problems that affect your use of your powered mobility device _____________________________________________________________________ 14) What is your gross annual household income? 1 = <14 999 2 = 15 000 – 29 999 3 = 30 000 – 44 999 4 = 45 000 – 59 999 5 = 60 000 – 74 999 6 = 75 000 – 89 999 7 = 90 000 – 114 999 8 = 115 000 – 129 999 9 = 130 000 – 145 000 10 = > 145 000 11 = Prefer not to answer 101 Wheelchair Skills Test Response options: Scoring: 0 = Fail, 1 = Pass with difficulty, 2 = Pass 102 103 Wheelchair Skills Test – Questionnaire Response options: Capacity: • 0 = No • 1 = Yes with Difficulty • 2 = Yes Confidence: • 0 = Not at all confident • 1 = Somewhat confident • 2 = Fully confidence Frequency • 0 = Never • 1 = Sometimes • 2 = Always Life Space Assessment 104 LIFE –SPACE The next questions refer to your activities just within the past month. During the past four weeks have you… FREQUENCY In the last four weeks, how often have you been to _________________? How did you get there? DEVICES Did you use aids or special equipment to get to ________? Did you use your power mobility device? ASSISTANCE Did you need help from another person to get to________? Yes No Less than once a week 1-3 times a week 4-6 times a week Daily Yes No Yes No Yes No LIFE-SPACE 1 Been to other rooms of your home besides the room where you sleep? Ο Ο Ο 777 Ο 888 Ο 999 (LS1) Ο Ο Ο Ο Ο 777= if not applicable Ο 888=don’t know or refused Ο 999 = question not asked (LS1F) Ο Ο Ο 777 Ο 888 Ο 999 (LS1A) Ο Ο Ο 777 Ο 888 Ο 999 (PMD1) Ο Ο Ο 777 Ο 888 Ο 999 (LS1H) LIFE-SPACE 2 Been to an area outside your home such as your porch, deck, or patio, hallway (of an apartment building) or garage, in your own yard or driveway? Ο Ο Ο 777 Ο 888 Ο 999 (LS2) Ο Ο Ο Ο Ο 777= if not applicable Ο 888=don’t know or refused Ο 999 = question not asked (LS2F) Ο Ο Ο 777 Ο 888 Ο 999 (LS2A) Ο Ο Ο 777 Ο 888 Ο 999 (PMD2) Ο Ο Ο 777 Ο 888 Ο 999 (LS2H) LIFE-SPACE 3 Been to places in your neighbourhood, other than your own yard or apartment building*? Ο Ο Ο 777 Ο 888 Ο 999 (LS3) Ο Ο Ο Ο Ο 777= if not applicable Ο 888=don’t know or refused Ο 999 = question not asked (LS3F) Ο Ο Ο 777 Ο 888 Ο 999 (LS3A) Ο Ο Ο 777 Ο 888 Ο 999 (PMD3) Ο Ο Ο 777 Ο 888 Ο 999 (LS3H) LIFE-SPACE 4 Been to places outside your neighbourhood, but within your own town? Ο Ο Ο 777 Ο 888 Ο 999 (LS4) Ο Ο Ο Ο Ο 777= if not applicable Ο 888=don’t know or refused Ο 999 = question not asked (LS4F) Ο Ο Ο 777 Ο 888 Ο 999 (LS4A) Ο Ο Ο 777 Ο 888 Ο 999 (PMD4) Ο Ο Ο 777 Ο 888 Ο 999 (LS4H) LIFE-SPACE 5 Been to places outside your town? Ο Ο Ο 777 Ο 888 Ο 999 (LS5) Ο Ο Ο Ο Ο 777= if not applicable Ο 888=don’t know or refused Ο 999 = question not asked (LS5F) Ο Ο Ο 777 Ο 888 Ο 999 (LS5A) Ο Ο Ο 777 Ο 888 Ο 999 (PMD5) Ο Ο Ο 777 Ο 888 Ο 999 (LS5H) 105 Appendix 4: Outdoor Environment Measures and Notes Outdoor Environment Measures and Notes Participant ID:_________________ Date:___________________________ Neighbourhood: ______________________ Section 1: Weather What is the weather like? a. Sunny b. Cloudy/overcast c. Rainy Is the ground wet or dry? a. Wet b. Dry Section 2: WST Measures Length of short distance (m): ___________ Length of long distance (m): ___________ Slight incline: Bottom Angle: ___________ Middle Angle: ___________ Top Angle: ___________ Length (m): ___________ Steep incline: Bottom Angle: ___________ Middle Angle: ___________ Top Angle: ___________ Length (m): ___________ Cross Slope: Bottom Angle: ___________ 106 Middle Angle: ___________ Top Angle: ___________ Length (m): ___________ Height of curb (cm): ___________ Gap: Distance (cm): ___________ Depth (cm): ___________ Soft surface: a. Grass b. Gravel c. Other Length of soft surface (m): ___________ Section 3: Additional Notes 107 Appendix 5: Interview guide 1. What does a typical day of using your scooter look like? 2. Tell me about some of your experiences in your community, using your scooter a. Probes: is using your scooter helpful for community participation? Does using your scooter interfere? How so? What impact does using a scooter have on your community participation? 3. What did you think about doing the test inside? Outside? a. Which environment did you prefer? Why? b. How did you feel performing inside? Outside? i. Probe: did you feel embarrassed performing outside? Did you feel more safe inside? ii. More distractions outside? c. Which environment did you feel more confident performing in? Why? d. Which environment do you think provided a more accurate assessment of your skills? Why? i. How does in compare to out? Okay, to refresh your memory of the environments, I’m going to scroll through a couple of comparison photos Show photos of environment i) Now that you’ve seen the comparisons, do you still agree with your previous statement that ___ environment is more accurate? If not, why so? 4. How do you think your scores compared between the 2 environments? Why? 108 *Interviewer to show scores from both environments, with differences highlighted a. Why do you think there are differences between scores here? *Show differences in skill performance* 5. Why do you think you did better on certain skills inside than outside? a. How can we change the ___ environment to make it more reflective of the other environment? 6. This item (e.g., pothole) was hard to find, do you think its important to know that skill? 7. How did you feel about the time it took to complete the WST in each environment? 8. How could the WST be improved? a. Probe: Were there any skills you thought weren’t needed or that were missing? 109 Appendix 6: Sample of powerpoint slides shown to participants during the interview 110 Appendix 7: List of Codes Participants’ experiences with scooter use (objective one) Category Code How the scooter has been helpful to participants Allowing for more activities Helping participants to save their energy Freedom from health conditions Hindrances of scooter use Causing stiffness or pain Difficulties maneuvering in tight spaces Facing inclement weather Participant perspective on the representativeness of the WST items (objective three) Category Code Which environment participants thought they performed better in Thinking they performed the same in both environments Thinking they performed better in the outdoor environment Thinking they performed better in the indoor environment Feeling familiar with the outdoor environment Finding the indoor course too controlled Experiencing more distractions outdoors Participants’ perspectives on confidence (objective three) Category Code Which environment participants felt more confident in Feeling more confident outside Feeling more confident inside Feeling equally confident Reasons why participants felt more confident inside Experiencing less distractions Reasons why participants felt more confident outside Feeling more familiar with the setting Reasons why participants felt equally confident Thinking that skills testing comes down to experience Thinking that confidence isn’t comparable between environments 111 Participant comparisons of the indoor lab-based versus outdoor community-based environment (objective three) Category Code Participants’ perception on how representative the indoor course was of their neighbourhood Finding the indoor course to be representative of the outdoor environment Finding the indoor course to be similar to the outdoor environment Thoughts on the gravel skill Finding the indoor gravel difficult Not feeling confident with the indoor gravel Feeling unsafe going over gravel Which environment provided a more accurate assessment of skill Finding the indoor course to be a more accurate representation of scooter skills Finding the outdoor course to be a more accurate representation of scooter skills Finding both environments to be an accurate representation of skills Reason for why an environment was a more accurate representation of skills Finding the indoor to be more encompassing of a variety of challenges Finding the inside to be more challenging - novel setting Finding the outdoor to be a more relevant setting Which environment participants preferred performing the WST in Preferring the indoor environment Preferring the outdoor environment Not having a preference for environment Reasons for preferring the indoor environment More variation of skills inside Finding the indoor environment novel Finding the indoor environment “fun” Reasons for preferring the outdoor environment Feeling familiar with the community setting - easier Finding the indoor setting to be too “test-like” Participants’ perception on whether certain skills are required on the WST (objective four) Category Code Thoughts on curb skill Thinking curbs are a necessary skill 112 Thinking curbs may be unnecessary “I’d avoid them” Thoughts on pothole skill Thinking potholes are an important skill Unsure if the pothole is a necessary skill Thinking the pothole skill is similar to train tracks “I’d avoid them” Participants’ perception on safety (objective four) Category Code Feelings of safety Feeling safer indoors Feeling safer outdoors Feeling equally safe Reasons for feeling safer indoors Perceiving there to be more help Finding less distractions indoors The perceived benefits of performing the WST (objective four) Code Finding the WST beneficial More willing to try skills 113 Appendix 8: Additional Information on Clinimetrics Development Clinimetrics is a term that was originally coined by Alvan Feinstein in 1987. Clinimetrics has been referred to as the conversion of basic clinical phenomena (e.g., severity of a condition, symptoms, etc) that cannot be measured in numbers into basic scientific data (Feinstein, 1983, Feinstein, 1987). More specifically, it is a domain that is concerned with rating scales, indexes, instruments, or other expressions that are used to measure these basic clinical phenomena (Feinstein, 1987). Feinstein originally conceptualized Clinimetrics in response to a concern that clinical data was not being valued as it was viewed as being “too soft” and lacked the quality required of scientific evidence (Feinstein, 1983). Feinstein (1983) makes the argument for the use of soft data in clinical practice as they can be important factors to consider in diagnosis and can affect results of randomized control trials and statistics, and can impact cost/benefit or risk/benefit analyses. As a result, the field of Clinimetrics was designed for the field of medicine, and focuses on the conversion of these soft data into “hard data”. Is clinimetrics different from psychometrics? There is current debate on the existence of the field of clinimetrics. The first is around whether clinimetrics is different from psychometrics. There is much support for the fact that clinimetrics is a different field than psychometrics. De Vet et al. (2003) argues that clinimetrics is different than psychometrics, as clinimetric instruments are intended to measure multiple constructs with a single index, while psychometric instruments measure a single construct using multiple items. In addition, de Vet et al. (2003) makes the claim that clinimetric measures are developed based on what is important to patients or clinicians; clinimetrics is focused on patient reported outcomes (Fava et al., 2011). 114 On the contrary, David Streiner has been a big advocate against the field of clinimetrics, as he believes that psychometrics and clinimetrics measure the same domain, thus not making a new contribution (2003). Streiner (2003) highlights that a main supposed difference between the fields is how measurements are developed; however, Streiner argues that both are developed in a similar method, that is, by starting with a large number of items and using statistical methods. Nonetheless, Streiner (2003) accepts that they are different in the development process by how much statistics and clinical judgements weighed. Furthermore, Streiner disagrees with statement made by the popular belief that psychometric scales are supposedly homogenous while clinimetric scales are heterogeneous, as strictly classifying all measures in a field as one type overlooks the diversity of the domain (Streiner, 2003). Should clinimetrics be abolished? The second area of debate is on whether the field of clinimetrics should be abolished. Based on the previous debated on whether clinimetrics is different from psychometrics, Streiner (2003) argues that the term clinimetrics should be faded from use. Streiner (2003) explains that having the term around may lead to confusion and misunderstandings between psychometrics and clinimetrics, and limits researchers from being exposed to the advances in psychometrics. As Streiner (2003) believes that clinimetrics is simply a portion of psychometrics, he believes that the clinimetrics is problematic and should be abolished. Others have debated for the existence of clinimetrics. Clinimetrics provides a method to reliably assess soft data, which is important for clinical research and practice (e.g., patient care). (Fava et al., 2011). Furthermore, De Vet et al. (2003) argue that clinimetrics is a term that is more appealing to clinicians, and to maintain their involvement in the field. Despite debate, De Vet et al. (2003) also believe that clinimetrics is different from psychometrics as clinimetric 115 measures are multidimensional and are constructed based on what is deemed important by clinicians or patients. Although there is question on whether the field of clinimetrics should abolished, De Vet et al. (2003) alternatively suggest for better integration of the two fields.