UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Reliability and validity of the Wheelchair Outcome Measure Garden, Jennifer Anna 2009

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2009_fall_garden_jennifer.pdf [ 3.14MB ]
Metadata
JSON: 24-1.0067673.json
JSON-LD: 24-1.0067673-ld.json
RDF/XML (Pretty): 24-1.0067673-rdf.xml
RDF/JSON: 24-1.0067673-rdf.json
Turtle: 24-1.0067673-turtle.txt
N-Triples: 24-1.0067673-rdf-ntriples.txt
Original Record: 24-1.0067673-source.json
Full Text
24-1.0067673-fulltext.txt
Citation
24-1.0067673.ris

Full Text

RELIABILITY AND VALIDITY OF THE WHEELCHAIR OUTCOME MEASURE  by  JENNIFER ANNA GARDEN B.H.K., The University of British Columbia, 1999 M.Cl.Sc. (OT), The University of Western Ontario, 2001  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in The Faculty of Graduate Studies (Rehabilitation Sciences)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  August 2009  © Jennifer Anna Garden, 2009  ABSTRACT Study Design: Prospective test re-test study. Objectives: To examine the reliability and validity of the Wheelchair Outcome Measure (WhOM) and discuss its clinical implications. Background: Review of the literature indicates the need for a client-centred wheelchair outcome measurement tool that measures participation outcomes related to wheelchair use. To date there have been no studies of the reliability and/or validity of the WhOM. Methods and Measures: A total of 50 participants who had a spinal cord injury participated in the study. There were three time periods in which the participant was required to answer questions. During the first session, demographic information was collected and the WhOM was completed along with the four additional measures. During the second session, the WhOM satisfaction scores were measured again. The entire WhOM was re-administered during the third by a second rater. A relative reliability index (intra-class correlation coefficient, ICC2,1 for test re-test and ICC2.2 for inter-rater reliability) was employed. Rater agreement for identified WhOM participation outcomes was determined using Kappa coefficients. Spearman‟s correlation coefficient was used to examine construct validity. Results: Test re-test agreements were high (ICC2,1: 0.90) and inter-rater agreements were high (ICC2,2: 0.90). Substantial agreement between raters for identified participation outcomes was achieved (K> 0.71). The subscale of assistive device scale of the Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST) demonstrated a positive relationship with the WhOM (r > 0.65). Items on the Assessment of Life Habits (LIFE-H) demonstrated a positive relationship with the WhOM (r ranged from 0.51 – 0.62). Both the Psychosocial Impact of Assistive Devices Scale and the Return to Normal Living Index failed to meet the hypothesis (r > 0.50). Conclusion: The WhOM is a new and useful tool for clinicians and researchers who work in the field of wheelchair prescription and research. Test re-test and inter-rater agreements were high for individuals with a diagnosis of a spinal cord injury. Evidence of validity was demonstrated with the QUEST and items from the LIFE-H.  ii  TABLE OF CONTENTS ABSTRACT ................................................................................................................................. ii TABLE OF CONTENTS .......................................................................................................... iii LIST OF TABLES ..................................................................................................................... vi LIST OF FIGURES .................................................................................................................. vii CO-AUTHORSHIP STATEMENT ....................................................................................... viii ACKNOWLEDGEMENTS ...................................................................................................... ix 1  INTRODUCTION .............................................................................................................. 1 1.1 Background ................................................................................................................. 1 1.2 Wheelchairs and Wheelchair Seating ....................................................................... 1 1.3 Abandonment of Assistive Technology Generally ................................................... 2 1.4 Measurement of Participation Outcomes ................................................................. 3 1.5 Model and Theory Framing this Research ............................................................... 4 1.6 Development of the WhOM ....................................................................................... 5 1.7 Reliability and Validity of Wheelchair Measurement ............................................. 6 1.8 Study Purpose.............................................................................................................. 8 References ................................................................................................................................ 9  2 RELIABILITY OF THE WHEELCHAIR OUTCOME MEASURE AMONG INDIVIDUALS WITH SPINAL CORD INJURIES ............................................................ 12 2.1 Introduction: Reliability of the Wheelchair Outcome Measure among Individuals with Spinal Cord Injuries ................................................................................ 12 2.2 Reliability ................................................................................................................... 13 2.3 Purpose, Objectives and Hypotheses ...................................................................... 15 2.3.1 Purpose.................................................................................................................. 15 2.3.2 Primary Objectives................................................................................................ 15 2.3.3 Primary Hypotheses .............................................................................................. 16 2.4 Methods ...................................................................................................................... 16 2.4.1 Design and Participants......................................................................................... 16 2.4.2 Protocol ................................................................................................................. 19 2.4.2.1 2.4.2.2 2.4.2.3 2.4.2.4  Rater Training ............................................................................................................................. 19 Time 1 (T1)................................................................................................................................. 19 Time 2 (T2)................................................................................................................................. 19 Time 3 (T3)................................................................................................................................. 20  2.5 Data Collection .......................................................................................................... 20 2.5.1 Description of the WhOM .................................................................................... 20 2.5.2 Data Analysis ........................................................................................................ 21 2.6 Results ........................................................................................................................ 23 2.6.1 Test-Retest Reliability .......................................................................................... 23 2.6.2 Inter-Rater Reliability ........................................................................................... 28 2.7 Discussion................................................................................................................... 35 2.7.1 Test-Retest Reliability .......................................................................................... 36 2.7.2 Inter-Rater Reliability ........................................................................................... 37 iii  2.8 Limitations ................................................................................................................. 38 2.9 Conclusion ................................................................................................................. 39 References .............................................................................................................................. 40 3 VALIDITY OF THE WHEELCHAIR OUTCOME MEASURE AMONG INDIVIDUALS WITH SPINAL CORD INJURIES ............................................................. 42 3.1 Introduction ............................................................................................................... 42 3.2 Types of Validity ....................................................................................................... 43 3.3 Purpose, Objectives and Hypotheses ...................................................................... 44 3.3.1 Purpose.................................................................................................................. 44 3.3.2 Objectives ............................................................................................................. 44 3.3.3 Hypotheses ............................................................................................................ 45 3.4 Methods ...................................................................................................................... 45 3.4.1 Design and Participants......................................................................................... 45 3.4.2 Protocol ................................................................................................................. 47 3.4.3 Measures ............................................................................................................... 47 3.4.3.1 3.4.3.2 3.4.3.3 3.4.3.4  Life Habits (LIFE-H) .................................................................................................................. 47 Psychosocial Impact of Assistive Devices Scale (PIADS) ......................................................... 48 Quebec User Evaluation of Satisfaction with Assistive Technology 2.0 (QUEST) ................... 49 Reintegration to Normal Living Index (RNLI) ........................................................................... 49  3.4.4 Data Analysis ........................................................................................................ 50 3.5 Results ........................................................................................................................ 50 3.6 Discussion................................................................................................................... 55 3.7 Limitations ................................................................................................................. 58 3.8 Conclusions ................................................................................................................ 58 References .............................................................................................................................. 59 4  CONCLUSIONS AND IMPLICATIONS ...................................................................... 61 4.1 General Findings ....................................................................................................... 61 4.2 Measurement Standards .......................................................................................... 61 4.2.1 Reliability and Validity ......................................................................................... 61 4.3 Suggested Modifications ........................................................................................... 62 4.3.1 Participation Outcomes ......................................................................................... 63 4.3.2 Scoring .................................................................................................................. 63 4.4 Clinical Implications ................................................................................................. 64 4.4.1 Further Research Studies ...................................................................................... 64 References .............................................................................................................................. 65  APPENDIX A: WhOM............................................................................................................. 66 APPENDIX B: RECRUITMENT ADVERTISEMENT AND LETTER OF INITIAL CONTACT................................................................................................................................. 69 APPENDIX C: PRIMARY DATA COLLECTION SHEET ............................................... 70 APPENDIX D: EXAMPLES OF PARTICIPATION OUTCOMES CLASSIFIED INTO HILL’S CRITERIA .................................................................................................................. 71 APPENDIX E: DETAILED VALIDITY HYPOTHESIS FOR QUESTIONS IN LIFE-H SUBSCALES ............................................................................................................................. 72 APPENDIX F: LIFE-H SHORT FORM ................................................................................ 76 iv  APPENDIX G: PSYCHOSOCIAL IMPACT OF ASSISTIVE DEVICES SCALE (PIADS) ..................................................................................................................................................... 97 APPENDIX H : QUEBEC USER EVALUATION OF SATISFACTION WITH ASSISTIVE TECHNOLOGY (QUEST) ................................................................................ 98 APPENDIX I : REINTEGRATION TO NORMAL LIVING INDEX .............................. 103 APPENDIX J: UNIVERSITY OF BRITISH COLUMBIA BEHAVIOURAL RESEARCH ETHICS BOARD CERTIFICATE OF APPROVAL ......................................................... 104  v  LIST OF TABLES Table 2.1 Descriptive Statistics and Test Re-test Values for Satisfaction Scores ...................... 24 Table 2.2 Descriptive Statistics and Test Re-test Values for Satisfaction x Importance Scores 24 Table 2.3 Descriptive Statistics and Inter-Rater Reliability Values for Satisfaction Scores for Matched Items ................................................................................................................. 29 Table 2.4 Descriptive Statistics and Inter-Rater Reliability Values for Importance Scores for Matched Items ................................................................................................................. 30 Table 2.5 Descriptive Statistics and Inter-Rater Values for Sat x Imp Scores for Matched Items ......................................................................................................................................... 30 Table 2.6 Rater Agreement for Participation Outcomes Categorized into Hill's Criteria .......... 35 Table 3.1 Participant Characteristics Sample (n=50) ................................................................. 51 Table 3.2 Spearman's Correlation Coefficients (p) for Satisfaction Multiplied by Importance Score for the WhOM Total, Inside and Outside Scores.................................................. 51 Table 3.3 Spearman's Correlation Coefficients for Satisfaction x Importance for the WhOM Total, Inside and Outside Scores .................................................................................... 52 Table 3.4 Spearman's Correlation Coefficients for the WhOM Total, Inside and Outside Satisfaction Scores and Selected Life-H Items ............................................................... 54  vi  LIST OF FIGURES Figure 2.1 Participant Flow for Recruitment and Data Collection ............................................. 18 Figure 2.2 Bland and Altman Graph for Test-Retest Inside WhOM Satisfaction Score ............ 25 Figure 2.3 Bland and Altman Graph for Test-Retest Outside WhOM Satisfaction Score ......... 26 Figure 2.4 Bland and Altman Graph for Test-Retest Total WhOM Satisfaction Score ............. 26 Figure 2.5 Bland and Altman Graph for Test-Retest Inside WhOM Sat x Imp Score ............... 27 Figure 2.6 Bland and Altman Graph for Test-Retest Outside WhOM Sat x Imp Score............. 27 Figure 2.7 Bland and Altman Graph for Test-Retest Total WhOM Sat x Imp Score................. 28 Figure 2.8 Bland and Altman Graph for Inter-Rater Inside WhOM Sat Score Matched Items . 30 Figure 2.9 Bland and Altman Graph for Inter-Rater Outside WhOM Sat Score Matched Items 31 Figure 2.10 Bland and Altman Graph for Inter-Rater Total Whom Sat Score Matched Items .. 31 Figure 2.11 Bland and Altman Graph for Inside WhOM Importance Score .............................. 32 Figure 2.12 Bland and Altman Graph for Outside WhOM Importance Score ........................... 32 Figure 2.13 Bland and Altman Graph for Total WhOM Importance Score ............................... 33 Figure 2.14 Bland and Altman Graph for Inside WhOM Satisfaction x Importance Score for Matched Items ................................................................................................................. 33 Figure 2.15 Bland and Altman Graph for Outside WhOM Satisfaction x Importance Score for Matched Items ................................................................................................................. 34 Figure 2.16 Bland and Altman Graph for Total WhOM Satisfaction x Importance Score for Matched Items ................................................................................................................. 34 Figure 3.1 Recruitment and Protocol .......................................................................................... 46  vii  CO-AUTHORSHIP STATEMENT  Dr. William C. Miller identified the design and research program for this thesis. Jennifer Garden performed the research and data analysis with Dr. Miller‟s guidance. The manuscript was prepared by Jennifer Garden with Dr. Miller‟s guidance and revisions.  viii  ACKNOWLEDGEMENTS This thesis would not have been possible had it not been for the patience, dedication and excellent teachings of Dr. Bill Miller. Thank you for accompanying me along this journey of six years. You are an exceptional teacher and mentor and I have learned a great deal from you.  Thank you to the members of my committee, Dr. Lyn Jongbloed and Dr. Tal Jarus. I really appreciate the feedback and the effort that everyone put in to help me finish this thesis.  Words cannot describe my gratitude to Sandy, my husband, for supporting me when I needed it most. You have had a great deal of patience over the past six years and I really appreciate the faith you have in me. Even though they are only 15 months old, I want to also thank my beautiful twins Hannah and Charlie for helping me finish. I know I am a better mother to you for having finished this thesis and I owe a great deal of it to you!  To my parents, also known as my editors, I cannot thank you enough. I know I have lost count of the number of times I sent you drafts to review. I really appreciate your endless support, encouragement and gentle reminders to „get that thesis done‟!  Thank you as well to my fellow graduate students, Erica Botner-Marigold, Adrienne Hol, Paula Rushton, and Jocelyn Harris as well as friends, Lisa Bennett, Sandy Barclay, Renee Lieu, and Nicole Penner. You have inspired me, encouraged me and had faith in me to complete this journey.  Finally, thank you to the participants and to Kathy Norton. This study would not have been possible had it not been for the generous amount of time that you gave to participation in the study.  ix  1  1.1  INTRODUCTION  Background According to the Canadian Paraplegic Association (2006), there are approximately  36,000 Canadians living with a spinal cord injury (SCI) and each year there are 1,050 new injuries reported. People who experience a SCI have an enormous initial disruption to their daily lives. In particular, their ability to ambulate may be compromised. New (2005) and Barbeau Ladouceur, Norman, Pepin, and Leroux (1999) reported that over 50% of individuals with SCI use a wheelchair as their primary means for mobility. Despite liberating an individual with respect to mobility, using a wheelchair may pose additional barriers to participating in daily activities, which can negatively affect a person‟s quality of life. Wheelchairs are designed to enhance people‟s lives by overcoming barriers and increasing their participation in life activities. The World Health Organization (WHO) (2001) defines participation as the extent of a person‟s involvement in life situations in relation to impairments, activities, health condition and contextual factors. Increased participation in life activities leads to an enhanced quality of life (Scherer, 2000; Bender Pape, Kim, & Weiner, 2002). It would seem that the benefits of assistive technology (AT) are abundant, especially in light of today‟s advances in technology (wheelchairs, for example, are lighter, more compact and easier to manoeuvre). However, researchers caution that because of the wide array of products available, a person‟s abilities, needs and environment need to be considered (Scherer, 2000; Devitt, Chau, & Jutai, 2003; Scherer & Cushman, 2001). Complex interactions among the person and their abilities during the execution of daily activities, in addition to environmental factors must be accounted for when choosing and prescribing a wheelchair. These considerations are crucial to the proper fit and functional use of the wheelchair.  1.2  Wheelchairs and Wheelchair Seating Wheelchairs provide SCI survivors with mobility that may enable and enhance their  participation in daily activities. There are generally two broad classifications of wheelchairs: power wheelchairs which are battery powered and usually controlled by a joystick; and manual 1  wheelchairs which are self-propelled, using hands and/or feet. Wheelchairs have varying design features that can be customized to meet an individual‟s needs: weight, durability and cost vary accordingly (Hunt, Boninger, Cooper, Zafonte, Fitzgerald, & Schmeler, 2004). However, wheelchairs are like shoes: one size does not fit all. There is a unique fit for each individual based on their needs and physical abilities, which is precisely the reason for their adjustable nature. Wheelchair prescription is a specialized area of practice in which a seating therapist assesses and recommends proper seating components such as size, weight, specific dimensions of the wheelchair, suitable cushioning, and adjustability. A wheelchair may be abandoned if it is not properly fitted to the person or if it is not suited to the environment for which it is intended. As a result, the person may be unable to accomplish daily activities independently (Kittel, Di Marco, & Stewart, 2002; Phillips & Zhao, 1993).  1.3  Abandonment of Assistive Technology Generally Abandonment of AT is problematic and costly (Scherer & Cushman, 2001; Phillips &  Zhao, 1993; & Bender Pape, et al., 2002). Phillips and Zhao (1993) report that mobility aids are frequently abandoned. Reasons for abandonment are numerous and include lack of consideration of the user‟s opinion, loss of functional abilities of the user, poor device performance, expectations held by others, financial and environmental factors, and predisposition to technology (Scherer & Cushman, 2001; Kittel, et al., 2002; Day, Jutai, & Campbell, 2002; Bender Pape, et al., 2002; & Scherer, 2000). Chaves, Boninger, Cooper, Fitzgerald, Gray, and Cooper (2004) investigated factors related to the wheelchair, individual impairment and environment that affect the perception of participation of persons with SCI in three settings: the home, the community, and during transportation. These findings indicated limited wheelchair participation in each of the three settings (Chaves et al., 2004). For individuals who rely on wheelchairs to overcome barriers, improper fitting of a wheelchair may result in decreased functional abilities, freedom, independence, and ultimately quality of life (Phillips & Zhao, 1993; Kittel et al., 2002).  Kittel et al., (2002) conducted a qualitative study with individuals with SCI to determine reasons for wheelchair abandonment. Two areas were reported as contributing to wheelchair abandonment. The first is the impact that the wheelchair had on the person‟s function: for 2  example, sitting tolerance and ability to mobilize outside the home or within the immediate environment without assistance to transfers to a vehicle. The second area that contributed to abandonment was the person‟s psychosocial well being related to the design, size, weight, and manoeuvrability of the wheelchair. One participan; found that “…her wheelchair caused so much damage [it] prompted [her] to position herself at a distance from tables, counters, and other furniture, creating an impersonal space and subsequent social barrier between herself and others” (Kittel et al., 2002, p. 110).  Wheelchairs are expensive and often range from $1,500 to over $30,000 dollars, depending on varying wheelchair features. Given these values, abandonment of wheelchairs is a waste of a significant resource (Kittel, et al., 2002, Scherer & Cushman, 2001, & Scherer, 2000). Many funding agencies supply the monies necessary to procure a wheelchair and its devices; therefore, abandonment of such devices is costly to these agencies and to society as a result of poorly utilised resources. Studies have demonstrated that it is important to consider the user‟s goals and long term needs when completing intervention strategies and prescribing AT such as wheelchairs (Kittel, et al., 2002; Scherer & Cushman, 2001; Dickerson, 1996; Phillips & Zhao, 1993; Demers, Monette, Lapierre, Arnold, & Wolfson, 2002).  1.4  Measurement of Participation Outcomes Several measurement tools exist that reflect client priorities which can be used when  addressing wheelchair and seating needs (Tugwell, Bombardier, Buchanan, Goldsmith, Grace, & Hanna, 1987; Stolee, Rockwood, Fox & Streiner, 1992; and Law, Baptiste, Carswell, McColl, Polatajko, & Pollock, 1994). The McMaster Toronto Arthritis Patient Preference questionnaire (MACTAR) is a functional index that measures change in impaired activities selected by each patient in a baseline interview. It uses predetermined questions, however, and therefore may not fully capture client priorities (Verhoeven, Boers, & van der Liden, 2000). Goal Attainment Scaling traditionally involves the client identify the goals while the scoring is completed by the therapist; however, because the scoring involves determining the importance and difficulty with the problem area, it does not capture the client‟s scores but rather the investigator‟s scores (Rushton & Miller, 2002). Finally, the Canadian Occupational Performance Measure (COPM) requires the client to identify occupational performance issues, 3  their importance, perceived performance, and satisfaction to construct a score, but does not capture specific wheelchair user information that is essential when prescribing or modifying wheelchairs and seating components (Barlow, 1998). Clearly, there is a need for a new and client-centred measurement tool that captures participation outcomes from a client‟s point of view.  1.5  Model and Theory Framing this Research The WHO‟s International Classification of Functioning, Disability, and Health (ICF)  can be used to describe how individuals live with their health conditions (World Health Organization, 2005). The ICF classifies health and health domains according to: 1) body functions and structures (level of the organ), 2) activities (tasks and actions by an individual), 3) participation (involvement in a life situation), and 4) environmental factors (World Health Organization, 2005). The ICF effectively classifies human functioning based on a universal and interactive model, which integrates the social and environmental aspects of disability and health (National Centre for Health Statistics, 2005). In using the WHO‟s ICF Model, we can classify many of the existing wheelchair measurement tools that assess outcomes. Currently, assessment tools used for wheelchair seating look at the level of body functions and structures, specifically body positioning, swallowing, respiratory function, pressure sore management, and psychosocial impact of the assistive technology (Day & Jutai, 1996; Herzberg, 1993; Krasilovsky, 1993; Miller, Miller, Trenholm, Grant, & Goodman, 2004). Other assessments measure the level of activity through evaluation of wheelchair skills such as propulsion time, length of reach when seated in chair, and transfers from a wheelchair to another surface (Kirby, Depuis, MacPhee, Coolen, Smith, Best, Newton, Mountain, MacLeod, & Bonaparte, 2004; Kilkens, Dallmeijer, de Witte, van der Woude, & Post, 2004). Typical of many rehabilitation measures, these tools reflect the clinician‟s concerns (Mortenson, Miller, & Auger, 2008). Kittel et al. (2002) described in their study that when the user of AT is involved in decision-making regarding a particular device, they are more likely to retain the device. In addition, a client-centred approach can provide a reliable, valid, and responsive measure of outcome after intervention (Carpenter, Baker, & Tyldesley, 2001; McColl, Paterson, Davies, Doubt, & Law, 2000). Measuring an individual‟s 4  participation outcomes while taking into consideration environmental contexts provides an assessment of participation in life activities, the third category in the ICF model. Participation in this context is defined as, “the nature or extent of a person‟s involvement in life situations reflecting the positive orientation of the contemporary equalization of opportunities for persons with disabilities” (Noreau & Fougeyrollas, 2000, p. 170). The Wheelchair Outcome Measure (WhOM) was developed to address the need for an evidence-based outcome measure that takes into account the user‟s chosen participation outcomes. It is designed to evaluate the efficacy of wheelchair system prescription based in terms of the users‟ ability to achieve participationoriented outcomes while using the wheelchair.  Models are important because they guide practice. Through research, models are developed and tested to justify outcomes such as the prescription of a wheelchair. To date there are no assessment tools available that effectively measure participation related to wheelchair use. The only client-specific wheelchair outcome measure that identifies outcomes at the participation level is the WhOM (Mortenson, Miller & Miller-Polgar, 2007; Mortenson, et al., 2008). Participation in life activities is a domain considered essential for function according to the ICF (World Health Organization, 2005). In fact, when wheelchair-specific measures were evaluated using the ICF as a framework, the WhOM was noted to be the only measurement tool which had the potential to measure participation across all ICF domains (Mortenson, et al., 2008). In addition, the WhOM allows the client to identify participation outcomes which supports client centred practice, a core principal of occupational therapy (CAOT, 2002).  The WhOM is an important step towards addressing the need for an evidence-based outcome measure that takes into account the user‟s participation outcomes. It is designed to evaluate the efficacy of wheelchair system prescription. Reliability and validity testing are required to ensure it captures what it is intended to capture and that it reproduces results over time.  1.6  Development of the WhOM The WhOM was developed using a qualitative study that recruited a heterogeneous  sample of individuals for participation. Initial interviews were conducted with a total of 34 5  interviewees, including 13 wheelchair prescribers, 14 wheelchair users, six wheelchair associates and one wheelchair manufacturer. Participation from these individuals provided face validity for the WhOM, a subjective judgment about whether or not a measurement makes sense intuitively and is reasonable (Hulley, Cummings, Browner, Grady, Hearst, & Newman, 2001).  The primary objective was to determine how individuals used their wheelchairs, with specific questions addressing which activities were performed while using the chair, where people went, and what specific tasks people would not be able to do without a wheelchair. The variety of informant groups interviewed provided different perspectives about the study question and allowed for triangulation of the data.  The current version of the WhOM is separated into three areas: questions related to activities that are accomplished inside or around the home; questions related to activities that are accomplished in the community and outside of the home; and questions related to comfort, positioning, and history of pressure sores. A unique aspect of the WhOM is that both the item selection of the participation outcomes and assessment of outcome are determined by the client. A copy of the WhOM can be found in Appendix A.  1.7  Reliability and Validity of Wheelchair Measurement Measurement is an important in that it helps us understand if the tools that clinicians use  are effective and appropriate for interventions. Clinicians are trained to be evidenced-based in their practice and there is an ever-growing need to demonstrate that interventions are effective. One of the primary goals of therapeutic intervention is to effect change (Streiner & Norman, 2003) and enable a person to achieve meaningful participation in their chosen occupational performance (CAOT, 2002; Dickerson, 1996). Assessment tools should be valid, reliable, and clinically meaningful in order to accurately demonstrate evidence-based outcomes in wheelchair seating and mobility interventions. „Validity‟ describes the extent to which a tool measures that which it is intended to measure (Streiner & Norman, 2003). However, the many different types of validity make its 6  measurement complex, especially in the field of rehabilitation, where clinicians typically measure abstract concepts such as „quality of life‟, „pain‟ or „participation‟ for which few if any „gold standard‟ tools exist (Hulley, et al., 2001). When a „gold standard‟ does not exist, construct validity is measured. Construct validity, like other types of validity, evaluates an instrument‟s ability to confirm a hypothesis and make accurate inferences about the variable(s) being measured (Streiner & Norman, 2003; Desrosiers, et al., 2004). Portney and Watkins (2000) suggest that to determine construct validity of an instrument, it is important to consider how it relates to other tests of the same and different constructs, thus looking at convergent and discriminant validity. Specifically, convergent validity indicates how closely two scales reflect the same underlying phenomenon and yield similar results and a high correlation (Steiner & Norman, 2003; Portney & Watkins, 2000). Discriminant validity indicates that low correlations would be expected when looking at instruments that are believed to assess different phenomena. Correlation coefficients measures are commonly used to examine the magnitude of the relationship between two variables (Portney & Watkins, 2000).  New instruments should have correlations that are strong enough to demonstrate a relationship between the constructs that both instruments purport to measure but not so strong that they appear to be measuring exactly the same construct (Streiner & Norman, 2003). Portney and Watkins (2000) suggest a general guideline for evaluating correlations coefficients as: 0.00 to 0.25 indicate a weak relationship; values ranging from 0.25 to 0.50 indicate a moderate to good relationship; and values above 0.75 are considered good to excellent. „Reliability‟ describes the extent to which an instrument is consistent and free from error, thereby producing reproducible and dependable results (Portney & Watkins, 2000). There are two types of measurement error. The first is systematic error, which is predictable and occurs consistently in one direction, e.g., always hitting a dart board on the right side of the bull‟s eye. The second type of error is random error, which is deviation in measurements that occurs purely by chance (Portney & Watkins, 2000). Random errors are unpredictable and can be influenced by factors such as the person rating behaviours or by factors related to the subject/participant, e.g., fatigue and mistakes. Portney and Watkins (2000), explain that an  7  assumption is made that if enough measurements are conducted, random errors will cancel out and the average score is a good estimate of the true score.  There are several different types of reliability. Some common examples include: 1) testretest: the ability of an instrument to produce the same results consistently over time, 2) intrarater: the stability of the data recorded by one person over several administrations to obtain similar results, and 3) inter-rater: the stability of scores between two or more people who administer the instrument to the same group of subjects (Portney & Watkins, 2000).  In summary, there is a clear need for a client centred wheelchair outcome measurement tool that measures participation outcomes related to wheelchair use. The WhOM is a new and potentially useful tool for clinicians and researchers who work in the field of wheelchair prescription and research. To date there have been no studies of the reliability and/or validity of the WhOM.  1.8  Study Purpose The following chapters will examine the reliability and validity of the WhOM and  discuss its clinical implications. The chapters have been set up using a manuscript thesis format such that chapters two and three represent components of the study which will be published. Chapter two addresses the reliability of the WhOM, chapter three addresses its validity, and chapter four synthesizes the findings and provides a discussion for the clinical implications of this measure. Study specific objectives and hypotheses are presented in chapters two and three.  8  References Barbeau, H., Ladouceur, M., Norman, K.E., Pepin, A., & Leroux, A. (1999). Walking after spinal cord injury: evaluation, treatment, and functional recovery. Archives of Physical Medicine and Rehabilitation, 80, 225-35. Barlow, I. Reliability and clinical utility of selected outcome measures with adult clients of seating clients [master‟s thesis] London, Ontario, Canada: University of Western Ontario, 1998. Bender Pape, T. L., Kim, J, & Weiner, B. (2002). The shaping of individual meanings assigned to assistive technology: a review of personal factors. Disability and Rehabilitation, 24(13), 5-20. Canadian Association of Occupational Therapists (2002). Enabling occupation: An occupational therapy perspective. Ottawa, ON: CAOT Publications ACE. Canadian Paraplegic Association. Retrieved on June 23, 2006, from http://canparaplegic.org/national/level2.tpl?var1+story&var2+20001027122552 Carpenter, L., Baker, G. A., & Tyldesley, B. (2001). The use of the Canadian Occupational Performance Measure as an outcome of pain management programme. Canadian Journal of Occupational Therapy, 68, 16-22. Chaves, E.S., Boninger, M.L., Cooper, R, Fitzgerald, S.G., Gray, D.B., & Cooper, R. A. (2004). Assessing the influence of wheelchair technology on perception of participation in spinal cord injury, Archives of Physical Medicine in Rehabilitation, 85, 1854-57. Day, H. & Jutai, J. (1996). Measuring the psychosocial impact of assistive devices: the PIADS. Canadian Journal of Rehabilitation, 9(3), 159-168. Day, H. Jutai, J., & Campbell, K. A. (2002). Development of a scale to measure the psychosocial impact of assistive devices: Lessons learned and the road ahead. Disability and Rehabilitation, 24(1-3), 31-37. Demers, L., Monette, M., Lapierre, Y, Arnold, D.L., & Wolfson, C. (2002). Reliability, validity, and applicability of the Quebec User Evaluation of Satisfaction with assistive Technology (QUEST 2.0) for adults with multiple sclerosis. Disability and Rehabilitation, 24(1-3), 21-30. Desrosiers, J., Noreau, L., Robichaud, L., Fougeyrollas, P., Rochette, A., & Viscogliosi, C. (2004). Validity of the assessment of life habits in older adults. Journal of Rehabilitation Medicine, 36, 177-82. Devitt, R., Chau, B., & Jutai, J.W. (2003). The effect of wheelchair use on the quality of life of persons with multiple sclerosis. Occupational Therapy Practice and Research, 17(3-4), 63-79. Dickerson, A. E. (1996). Should choice be a component in occupational therapy assessments? Occupational Therapy in Health Care, 10, 23-32. Herzberg, S. R. (1993). Positioning the nursing home resident: An issue of quality of life. American Journal of Occupational Therapy, 47, 75-77. Hulley, S. B., S. R. Cummings, Browner, W. S. Grady, D., Hearst, N. & Newman, T. B. (2001). Designing clinical research (2nd ed.). Philadelphia: Lippincott Williams & Wilkins. Hunt, P.C., Boninger, M.L., Cooper, R.A., Zafonte, R.D., Fitzgerald, S.G., & Schmeler, M.R. (2004). Demographic and socioeconomic factors associated with disparity in wheelchair customizability among people with traumatic spinal cord injury, Archives of Physical Medicine and Rehabilitation, 85, 1859-64. 9  Kilkens, O. J., Dallmeijer, A. J., de Witte, L. P., van der Woude, L. H., & Post, M. W. (2004). The wheelchair circuit: construct validity and responsiveness of a test to assess manual wheelchair mobility in persons with spinal cord injury. Archives of Physical Medicine and Rehabilitation, 85(3), 424-31. Kirby, R. L., Dupuis, D. J., MacPhee, A. H., Coolen, A. L., Smith, C. Best, K. L., Newton, A. M., Mountain, A. D., MacLeod, D. A. & Bonaparte, J. P. (2004). The Wheelchair Skills Test (Version 2.4): Measurement properties. Archives of Physical Medicine and Rehabilitation, 85, 794-804. Kittel, A., Di Marco, A., & Stewart, H. (2002). Factors influencing the decision to abandon manual wheelchairs for three individuals with a spinal cord injury. Disability and Rehabilitation, 24(1-3), 106-114. Krasilovsky, G. (1993). Seating assessment and management in a nursing home population. Physical and Occupational Therapy in Geriatrics, 11, 25-38. Law, M., Baptiste, S., Carswell, A., McColl, M., Polatajko, H., & Pollock, N. (1998). Canadian Occupational Performance Measure Manual (3rd Ed.) Toronto: CAOT Publications. McColl, M.A., Paterson, M., Davies, D., Doubt, L., & Law, M. (2000). Validity and community utility of the Canadian Occupational Performance Measure, Canadian Journal of Occupational Therapy, 67(1), 22-30. Miller, W.C., Miller, F., Trenholm, K., Grant, D., & Goodman, K. (2004). Development and preliminary assessment of the measurement properties of the Seating Identification Tool (SIT). Clinical Rehabilitation, 18, 317-25. Mortenson, W.B., Miller, W.C., & Auger, C. (2008). Issues for the Selection of WheelchairSpecific Activity and Participation Outcome Measures: A Review, Archives of Physical Medicine and Rehabilitation, 89, 1177 – 86. Mortenson, W.B., Miller, W.C., & Miller-Polgar, J. (2007). Measuring wheelchair intervention outcomes: Development of the wheelchair outcome measure. Disability and Rehabilitation, 2, 275-85. National Centre for Health Statistics, Classification of Diseases, Functioning, and Disability. Retrieved on February 23, 2005, from http://www.cdc.gov/nchs/about/otheract/icd9/icfhome.htm New, P. W. (2005). Functional outcomes and disability after nontraumatic spinal cord injury rehabilitation: results from a retrospective study. Archives of Physical Medicine and Rehabilitation, 86, 250-261. Noreau, L. & Fougeyrollas, P. (2000). Long-term consequences of spinal cord injury on social participation: the occurrence of handicap situations. Disability and Rehabilitation, 22(4), 170-180. Phillips, B. & Zhao, H. (1993). Predictors of assistive technology abandonment. Assistive Technology, 5, 36 – 45. Portney, L.G. & Watkins, M.P. (2000). Foundations of clinical research applications to practice (2nd ed.). New Jersey: Prentice Hall Inc. Rushton, P.W. & Miller, W.C. (2002). Goal attainment scaling in the rehabilitation of patients with lower extremity amputations: a pilot study. Archives of Physical Medicine and Rehabilitation, 83: 771 – 775. Scherer, M. J. (2000). Living in the state of stuck: How assistive technology impacts the lives of people with disabilities (3rd ed.). Cambridge: Brookline Books.  10  Scherer, M.J. & Cushman, L.A. (2001). Measuring subjective quality of life following spinal cord injury: a validation study of the assistive technology device predisposing assessment. Disability and Rehabilitation, 23(9), 387-393. Streiner, D. L. & Norman, G. R. (2003). Health Measurement scales: a practical guide to their development and use (3rd ed.). New York: Oxford University Press. Stolee, P., Rockwood, K., Fox, R.A., & Streiner, D.L. (1992). The use of goal attainment scaling in a geriatric care setting. Journal of the American Geriatric Society, 40, 574578 Tugwell, P., Bombardier, C., Buchanan, W.W., Goldsmith, C.H., Grace, E., and Hanna, B. (1987). The MACTAR patient preference disability questionnaire: an individualized functional priority approach for assessing improvement in physical disability in clinical trials in rheumatoid arthritis. Journal of Rheumatology, 14, 446-451. Verhoeven, A.C., Boers, M., & van der Linden, S. (2000). Validity of the MACTAR Questionnaire as a functional index in a rheumatoid arthritis clinical trial. Journal of Rheumatology, 27, 2801-9. World Health Organization. (2002). ICF: international Classification of Functioning Disability and Health. Geneva: WHO.  11  2  2.1  RELIABILITY OF THE WHEELCHAIR OUTCOME MEASURE AMONG INDIVIDUALS WITH SPINAL CORD INJURIES 1  Introduction: Reliability of the Wheelchair Outcome Measure among Individuals with Spinal Cord Injuries In Canada, there are approximately 36,000 individuals living with a spinal cord injury  (SCI) and each year there are approximately 1,050 new cases (Canadian Paraplegic Association, 2006). The majority of individuals who experience a SCI rely on a wheelchair as their primary means of mobility (Barbeau, Ladouceur, Norman, Pepin, & Leroux, 1999; New, 2005). The prescription and fit of a wheelchair is unique to each individual and requires careful consideration of that individual‟s physical and cognitive abilities, the environment in which they reside and the types of activities that they undertake. A wheelchair or its seating components (cushion and back rest) that are not customized properly to the person or to his or her environment may prohibit this person from accomplishing daily activities independently, and in many cases the wheelchair is abandoned (Garber, Reynold, & Monga, 2002; Kittel, Di Marco, & Stewart, 2002). Wheelchairs are expensive and can cost anywhere from $1,500 to over $30,000 depending on the features. It is important that proper wheelchair prescription involve the individual who will be using the wheelchair: this ensures that the prescriber has a better understanding of the activities in which the wheelchair user participates. The World Health Organization‟s (WHO) International Classification of Functioning, Disability and Health (ICF) defines participation as “involvement in a life situation” (World Health Organization, 2005, p. 123). The ICF qualifies participation as the activities that a person undertakes in their current environment, including factors such as the physical, social and attitudinal environment (World Health Organization, 2002). Funding agencies require evidence of the effectiveness of the wheelchair being prescribed. Mortenson, Miller, & Auger, (2008) used the WHO‟s ICF as a framework to identify and evaluate wheelchair specific outcome instruments that are useful for measuring activity and participation. In total, they  1  A version of this chapter will be submitted for publication. Garden, J. And Miller, W.C. Reliability of the Wheelchair Outcome Measure. 12  found eleven wheelchair-specific measures and of those, reliability evidence was limited (Mortenson, et al., 2008).  The Wheelchair Outcome Measure (WhOM) is a flexible and client-centred outcome measurement tool. The wheelchair user identifies and rates satisfaction level with participation outcomes related to activities performed while using their wheelchair (Mortenson, Miller & Miller-Polgar, 2007). In order for a measurement tool such as the WhOM to be useful in clinical and research settings, it needs to provide accurate and meaningful results. Measurement properties which provide information regarding the accuracy of an instrument include: reliability, validity and the ability to detect change (Finch, Brooks, Stratford & Mayo, 2002).  2.2  Reliability Classical Test Theory states that any observation is made up of two components: a true  score and error associated with observation. In other words, reliability is the ratio of true variance to observed total variance (Finch, et al, 2002; Steiner & Norman, 2005). The error, defined as the difference between the obtained score and the true score, can be associated with three different sources: 1) the examiner, where the examiner may make an error when administering the measurement tool, 2) the examination, where there are errors in the measurement tool, protocol or environmental distractions and 3) the examinee, where there may be a lack of consistency of the person responding (e.g., where responses are affected by memory loss) (Portney & Watkins, 2000; Streiner & Norman, 2005). In order for a measure to be reliable, it must provide consistent results with only small errors of measurement (Steiner & Norman, 2005; Portney & Watkins, 2000). There are several different types of reliability that are examined in clinical tools. The following represent important forms of reliability that are necessary for interview-oriented tools such as the WhOM: 1) Test-retest reliability examines the extent to which a measure is reliable when repeated on different occasions (Finch, et al, 2002; Streiner & Norman, 2005); 2) inter-rater reliability examines the extent to which different raters provide a reliability estimate of the same attribute; and 3) intra-rater reliability examines the extent to which the same rater provides a reliability estimate of the attribute on different occasions (Finch, et al, 2002; Streiner 13  & Norman, 2005). While other forms of reliability exist, such as internal consistency and alternate form reliability, these forms are not relevant when the client generates his or her own outcomes: this is the case with the WhOM. Therefore, these forms of reliability are not addressed in this thesis.  Determination of reliability also depends upon the type of data that is collected (Portney & Watkins, 2000; Rankin & Stokes, 1998; Steiner & Norman, 2005). Data that have distinct categories with no obvious order, such as yes/no or blood types, can be tested using the Kappa statistic (Portney & Watkins, 2000; Rankin & Stokes, 1998). The Kappa statistic looks at the proportion of observed agreements as well as the proportion of agreements expected by chance (Portney & Watkins, 2000). There are, however, limitations to the Kappa statistic: it is based on proportions, which makes it necessary to have sufficient participants; otherwise the data could skew results (Portney & Watkins, 2000). Portney and Watkins (2000) describe the greatest limitation to the Kappa statistic is its specificity to nominal; each subject must be placed in an exclusive category and therefore continuous data must be treated differently. Researchers differ in opinion as to which statistics should be used to examine reliability of continuous data (interval and ratio) (Rankin & Stokes, 1998). Expressing reliability in terms of an intra-class correlation coefficient (ICC) reflects both correlation and agreement and has become the preferred index (Portney & Watkins, 2000). There are different types of ICC‟s based on models used. The type of ICC used depends on the design and potential use of the results (Rankin & Stokes, 1998). Rankin and Stokes (1998) discuss different forms of ICC‟s that can be used to interpret inter-rater reliability. The ICC is expressed with two integers. The first is used to describe the type of case (usually related to the design of the study) and the second describes how many units of analyses there are (Rankin & Stokes, 1998).  The Bland and Altman method, also known as the limits of agreement, is used to estimate absolute reliability and provides additional information because it assesses individual agreement of the various data points (Rankin & Stokes, 1998). Rankin and Stokes (1998) suggest it is helpful to use an ICC accompanied by the Bland and Altman method because using an ICC alone will not detect bias (e.g. a rater overestimating a measurement systematically). It also allows for a nice visual representation of the degree of agreement to visually inspect if data 14  points are evenly distributed above and below the mean and have an equal spread along the x axis. However, equally important if using the Bland and Altman method is to ensure that the limits of agreement are indicated to explain the range of error.  Another way to provide additional information to the reader about the magnitude of disagreement between measurements is to use the standard error of measurement (SEM), which is expressed in the same units as the original measurement. This also provides an estimate of absolute reliability, which is the extent to which a score varies on repeated measurement: the higher the value, the less reliable as there is more error. The SEM is useful to clinicians because it provides an estimate of reliability about the extent to which replicate measurements performed on the same individual yield similar values (Finch, et al, 2002). With this information, a clinician can determine if a statistically significant change has occurred from the baseline measurement to measurement post-treatment, providing an excellent method to determine if their treatment was effective for a group of individuals.  In order for therapists to use an objective measure to determine whether treatments are successful and effective, the measurement tool must first have reliability to ensure that it provides reliable and consistent results with minimal error. The present study was undertaken to examine the reliability properties of the WhOM, a tool which will be very helpful to therapists and clients who require a wheelchair.  2.3  Purpose, Objectives and Hypotheses  2.3.1 Purpose The purpose of this chapter is to report on the test-retest and inter-rater reliability of the WhOM in individuals who have sustained a spinal cord injury. A second purpose is to examine rater agreement on items listed on the WhOM.  2.3.2 Primary Objectives 1. To examine one week test-retest reliability of the WhOM. 2. To examine inter-rater reliability of the WhOM. 3. To examine inter-rater agreement for participation outcomes using the WhOM. 15  2.3.3 Primary Hypotheses 1. Test-Retest Reliability: The WhOM will demonstrate good reliability with an intraclass correlation coefficient (ICC2,1) > 0.80 or higher. 2. Inter-rater: The inter-rater reliability of the WhOM will be ICC2,2 > 0.80. 3. Inter-rater Agreement: The participation outcomes classified into Hill‟s criteria recorded by each rater will have an overall agreement of Kappa > 0.60.  A correlation coefficient of 0.80 was chosen for both hypotheses one and two because it is reported that ICC‟s ranging from 0.75 – 1.00 are considered good to excellent (Portney & Watkins, 2000). It was hypothesized that the overall Kappa would be rated as substantial according to Landis & Koch, (1977).  2.4  Methods  2.4.1 Design and Participants A prospective one week test-retest design was used with a consecutive sample of 50 individuals living in the community who have mobility impairments as a result of a spinal cord injury. Potential participants were identified and selected using a current database of individuals interested in participating in research. In addition, therapists were approached to ask clients on their current caseload to participate, and a notice was posted in a local magazine to identify possible participants. Finally, a snowballing technique was also employed whereby a past participant may approach a friend who may also be interested in the study. (Please see the four recruitment strategies as outlined in Figure 2.1.) Copies of the letter describing the study and the recruitment notice that was posted in the magazine BC Paraplegia can be viewed in Appendix B. Those individuals who were interested were instructed to contact the study coordinator who screened each person by telephone to ensure they met the inclusion criteria. Participants were included if they: 1) were aged 20 years or older; 2) used a wheelchair as their primary means for mobility for at least four hours each day; 3) had a diagnosis of complete or incomplete spinal cord injury at any level; and 4) were able to read, speak and understand English.  16  Individuals with cognitive impairments (as screened with the Cognitive Competency Screening Evaluation (CCSE) scoring below 24) were excluded from the study, as their responses may not be reproducible (Xu, Meyer, Thornby, Chowdhury, & Quach, 2002). In addition, subjects were excluded if they had received a new wheelchair in the past six months or if they were currently undergoing any modifications to their present wheelchair or seating components during the re-test interval. Wheelchair seating often requires several fittings to obtain the correct position and it was felt that any modifications made during the testing period would affect the stability of participant responses.  A sample size of 50 was determined using a sample size calculation based on a directional hypothesis where alpha is set at 0.05 and Beta is 0.20 with an expected ICC > to 0.80 (Donner & Eliasziw, 1987).  Individuals who met the study criteria were invited to their first appointment. Participants received $25.00 for each of their two visits during the data collection period. Written and informed consent was obtained from each participant and the study was approved by the University of British Columbia Behavioural Research Ethics Board.  17  Recruitment  200 letters of information sent out to individuals from database at GF Strong Research Lab  Advertisement in BC paraplegic magazine  Therapists working in the community and seating clinics provided information about the study to people in their caseloads  Snowball effect  19 individuals responded to the letter and were screened for eligibility by a research assistant over the phone and scheduled for an appointment  34 individuals were recruited by referral from a therapist or by the snowball effect. They were contacted and screened for eligibility over the telephone and scheduled for an appointment.  53 individuals recruited  Session 1 (n = 52) rater one  Informed consent  Cognitive Capacity Screen Evaluation  Demographic data  WhOM & 4 Validity scales (random order)  Scheduled appointment for second visit – one week One participant discovered to be ineligible Session 2 (n = 51) rater one  Re-administration of WhOM by rater one (only satisfaction scores re-rated)  Schedule appointment for telephone administration of the WhOM for 48 – 72 hours One participant withdrew Session 3 (n = 50) rater two  Re-administration of the WhOM  One participant withdrew  Figure 2.1 Participant Flow for Recruitment and Data Collection 18  2.4.2 Protocol 2.4.2.1 Rater Training Two occupational therapists (OT) were recruited and trained to administer the WhOM questionnaire. Both OTs had a minimum of 5 years experience with seating in either a community setting or in a rehabilitation setting. The first rater completed a pilot test on one subject, which was videotaped. She was then provided with feedback about the semi-structured interview technique employed. The second rater was trained by first using the original video. Both raters, therefore, had the necessary knowledge of wheelchair seating and interview skills required to complete the study.  Upon the start of data collection all difficulties which presented themselves during the study were discussed and resolved by both raters (e.g. capturing participation outcomes instead of activities).  Three data collection time points were used in order to capture all of the relevant forms of reliability. The participant was required to answer a set of questionnaires at each time point.  2.4.2.2 Time 1 (T1) The purpose of this session was to administer the WhOM and obtain participation outcomes which the subjects completed using their wheelchairs. At T1 the first rater screened the participant for cognitive impairments to determine eligibility. Demographic data such as age, sex, marital status, type of wheelchair, date of injury, and length of time with current wheelchair, were also collected at this time. A copy of the demographic questions is attached in the Appendix C.  2.4.2.3 Time 2 (T2) One week later a second data collection period was arranged. This timeline was chosen in order to minimise recall bias and to ensure the wheelchair and its seating components had not changed. The participant was asked to re-evaluate the satisfaction score of their original chosen participation outcomes which were identified at T1. This was completed by Rater One in order to provide test-retest reliability data. 19  2.4.2.4 Time 3 (T3) Time three (T3) occurred 48 to 72 hours after T2 in order for us to capture inter-rater reliability. It was hoped that this time period would minimize recall bias but provide a close enough retesting period to ensure the wheelchair and seating components had remained stable. The subject was contacted via telephone by a second rater and the WhOM was fully readministered (e.g. participant chose participation outcomes for a second time). The second rater was blinded to the results of first rater. We chose to collect T3 data over the telephone in order to minimize the burden of travel to the participant and improve the response rate for the study. In addition, it provides an opportunity to collect inter-rater reliability over the telephone, which would be the mostly likely method to collect follow-up data for clinical practice.  2.5  Data Collection  2.5.1 Description of the WhOM The WhOM was developed using the ICF as a theoretical framework to solicit information on participation outcomes performed while using a wheelchair. The WhOM can be administered in less than 30 minutes and is divided into two sections: 1) the therapist asks open ended questions to allow the client to identify up to five participation outcomes each for a) inside the home (e.g. using the wheelchair to cook) and b) outside the home and in the community. The therapist uses a semi-structured interview guide to collect information by asking “Some people use their wheelchair to participate in activities in or around the home, such as preparing meals, watching TV, or gardening. What activities in your home do you use your wheelchair to perform?” These participation outcomes can form the basis to guide intervention and to determine the success of intervention.  Once participation outcomes are identified, the client is asked to score the importance of each one using a scale numbered 0 (no importance) – 10 (very important). The same scoring procedure is used to rate their satisfaction with each participation outcome. To obtain initial scores during the assessment, the value of the importance score is multiplied with the satisfaction score. The same method is completed for each identified participation outcome. A total score can be obtained by summing the scores and dividing by the number of participation outcomes. Alternatively, the therapist can use just the satisfaction scores to address the 20  outcome participations identified by dividing the sum of satisfaction scores by the number of participation outcomes. This provides a score range from 0-10 whereas using the importance multiplied by the satisfaction score provides a score range from 0 – 100. These participation outcomes form the basis for intervention planning for the therapist and client. Once intervention is complete the client is asked to re-rate their satisfaction for each previously identified participation outcome.  2.5.2 Data Analysis Descriptive statistics such as age, marital status, years of wheelchair use and years of disability were collected during the initial testing session to enable the researchers to describe the population being studied. Two types of reliability were examined: test re-test and interrater. Test re-test reliability was assessed using ICC‟s as well as SEM and Bland and Altman were calculated to provide additional information about reliability. Portney and Watkins (2002) suggest the use of ICC‟s as the preferred index for reliability analysis because it reflects both correlation and agreement. Traditionally, reliability studies focussed on reporting coefficients that were not typically useful in a clinical setting because the coefficient‟s meaning was not easily translated into a measurement error that was in the same units as the original measurement. Therefore, it is now important to also report the SEM (Stratford, 2004). Rankin and Stokes (1998), suggest that Bland and Altman method be used to complement an ICC to allow for clinical interpretation.  To analyse test re-test reliability an ICC model of ICC2,1 was employed. The first integer refers to the type of study being performed. In this case, because there is a need to generalize to other raters, the first integer is two. The second integer refers to the number of raters. Inter-rater reliability was assessed using an ICC model of ICC2,2 - again, because the raters within the same population (therapists) will be using the WhOM, the first integer is two. The second integer is also two as there were two raters and individuals were measured twice. Rankin and Stokes (1998), advise that an ICC2,k model is inter-rater reliability study where the results will be used to generalize it to other therapists/raters who will use the tool in a clinical or research setting. Rankin and Stokes (1998) indicate this is an appropriate use of this model for its  21  intended purpose. Use of this model of ICC will answer the question about agreement between raters and if raters are interchangeable, as is the case in clinical settings.  We chose a slightly higher ICC criterion of (0.80) for our standard. While there is considerable debate in the literature as to what is should be (Streiner & Norman, 2003), Portney and Watkins (2000), states that a criterion of (0.75) demonstrates good reliability. We choose (0.80) because we felt that the WhOM needed to demonstrate strong reliability.  The results for test re-test of the WhOM are presented with satisfaction scores separated out from the importance score to get a better sense of the participants‟ satisfaction with the participation outcome. In addition, the overall score (satisfaction multiplied by importance) is reported. For each area inside, outside and total scores are presented as it is felt that participants completed different participation outcomes in their chairs from indoor to outdoor. It was important, therefore, to separate out these scores to get a better sense of how the wheelchair impacted their satisfaction in different environments. Inter-rater reliability scores are also reported with satisfaction only, importance only and satisfaction multiplied by importance for indoor, outdoor, and total scores. Because inter-rater reliability had each rater soliciting participation outcomes on different occasions from subjects, these items were not always the same, and therefore only identical items were reported in the inter-rater reliability scores.  In order to capture succinctly the type of participation outcomes that people identified while using their wheelchairs, the over five hundred participation outcomes were categorized using Hill‟s criteria (Juster & Stafford, 1985). Doing so enables easier assessment of agreement of the types of participations that the raters captured (see objective/hypothesis 3). Hill used ten broad categories of activities to examine patterns of time use of American adults in the 1970‟s. The ten categories of activities include: 1) market work, 2) house and yard work, 3) child care, 4) services/shopping, 5) personal care, 6) education, 7) organizations, 8) social entertainment, 9) active leisure, 10) passive leisure. Each participation outcome identified by the subject was categorized using Hill‟s Criteria for each rater. These areas were then analysed for agreement between raters using Kappa for each category of Hill‟s criteria as well as for overall agreement.  22  Agreement for items between raters was examined using Kappa. Landis and Koch (1977) provide criteria for interpreting the Kappa statistic as follows: < 0 no agreement, 0.0 – 0.20 slight agreement, 0.21 – 0.40 fair agreement, 0.41 – 0.60 moderate agreement, 0.61 – 0.80 substantial agreement and 0.81 – 1.00 almost perfect agreement.  Data analyses were performed using SPSS for Windows version 17. Statistical significance was set at p<0.05.  2.6  Results Of the 53 individuals who qualified for this study (Figure 2.1), one participant was  found to be ineligible as this person used more than one wheelchair for participation in outcomes; two other individuals could not be contacted in follow-up sessions of testing. Therefore, a total of 50 individuals participated in the study. Of the 50 participants, 84% were male and 16% were female. There was a greater number of individuals with tetraplegia (64%) than paraplegia (36%) and their mean age was 43.7 years (SD=10.7, range 20 - 66). Fourteen percent of the participants were married or living common law while 70% were either single or divorced; one person was widowed. Most participants used a manual wheelchair (66%) and the mean length of time using a wheelchair was 5.7 + 4.7 years.  Test re-test was completed one week apart with an average of 7.55 days between testing. Inter-rater reliability was used between the second administration of the instrument and 3 days after. The average for this time was 3.51 days.  2.6.1 Test-Retest Reliability The ICC‟s for test-retest for satisfaction scores are reported in Table 2.1. ICC‟s for the total WhOM, indoor participation satisfaction and outdoor participation satisfaction were calculated separately. The total WhOM test-restest reliability for satisfaction scores is ICC = 0.83 (95 CI 0.72, 0.90). The mean for time two is higher than the mean of time one and the ICC is lower for outside than it is for inside participation outcomes.  23  Test re-test ICC‟s for satisfaction multiplied by importance scores are reported in Table 2.2. The total WhOM score, satisfaction multiplied by importance score is ICC = 0.90 (95 CI 0.83 – 0.94). The mean for T2 is higher than the mean for T1. The ICC for the total score is high and the ICC for inside scores is higher than for outside scores. Overall, the CIs are tight for satisfaction scores as well as for satisfaction multiplied by importance scores for test-retest reliability.  The SEMs for satisfaction and for satisfaction multiplied by importance are all low. The Bland and Altman plots appear below in Figures 2.2 – 2.4 for each of the three scores (inside, outside and total WhOM) for satisfaction as well as for satisfaction multiplied by importance. The mean difference between each testing time was close to zero and the data points were fairly equally distributed above and below the mean difference line. There were some outliers in each graph. In almost all cases these outliers represent the same participants.  Table 2.1 Descriptive Statistics and Test Re-test Values for Satisfaction Scores Mean T1 (SD) Mean T2 (SD) Test-Retest ICC  SEM for Test Retest  MEASURE  WhOM  Satisfaction T1 – T2  INSIDE  7.90 (1.54)  7.92 (1.47)  .83 (.72 - .90)  0.63  OUTSIDE  7.83 (1.56)  7.93 (1.50)  .80 (.68 - .88)  0.70  TOTAL  7.86 (1.41)  7.93 (1.33)  .83 (.72 - .90)  0.58  Table 2.2 Descriptive Statistics and Test Re-test Values for Satisfaction x Importance Scores  Mean T1 (SD) Mean T2 (SD) Test-Retest ICC  SEM for Test Retest  MEASURE  WhOM  Sat x Imp T1 – T2  INSIDE  67.52 (21.00)  70.42 (19.18)  .87 (.78 - .92)  7.57  OUTSIDE  72.55 (16.50)  72.70 (16.93)  .83 (.72 - .90)  6.80  TOTAL  70.04 (17.23)  71.56 (16.58)  .90 (.83 - .94)  5.45  24  Figure 2.2 Bland and Altman Graph for Test-Retest Inside WhOM Satisfaction Score  25  Figure 2.3 Bland and Altman Graph for Test-Retest Outside WhOM Satisfaction Score  Figure 2.4 Bland and Altman Graph for Test-Retest Total WhOM Satisfaction Score  26  Figure 2.5 Bland and Altman Graph for Test-Retest Inside WhOM Sat x Imp Score  Figure 2.6 Bland and Altman Graph for Test-Retest Outside WhOM Sat x Imp Score  27  Figure 2.7 Bland and Altman Graph for Test-Retest Total WhOM Sat x Imp Score  2.6.2 Inter-Rater Reliability The inter-rater reliability ICC‟s were in general higher than test re-test ICC‟s for inside and outside activities; however, they were similar when considering the overall WhOM scores. Table 2.3 presents the inter-rater reliability for satisfaction scores for identical items between T2 and T3. The mean at T3 is consistently higher for all scores (inside, outside, and total). Table 2.4 presents the inter-rater reliability for importance scores for identical items between T1 and T3. Means for both measurement times were similar and ICC‟s for inside were higher than for outside scores. The mean scores were consistently higher at T3 than T2. The ICC‟s were very similar for inside and outside activities. Table 2.5 presents the inter-rater reliability for satisfaction multiplied by importance scores for identical items between T2 and T3.  The Bland and Altman plots appear below for each of the three scores (inside, outside and total WhOM) for satisfaction scores, importance scores, and satisfaction multiplied by importance scores (Figures 2.8 – 2.16). There were outliers which were the same individuals. Some of the data points were not evenly distributed and at times there are clusters of scores at  28  the top end of the range of scores, indicating a possible ceiling effect. The mean difference between testing times was generally close to zero. Table 2.3 Descriptive Statistics and Inter-Rater Reliability Values for Satisfaction Scores for Matched Items  MEASURE  WhOM  Mean T2 (SD)  Mean T3 (SD)  Inter-rater ICC  SEM for Inter-Rater  SATISFACTION T2 – T3  INSIDE  7.93 (1.64)  8.10 (1.63)  .87 (.77 – 93)  0.59  OUTSIDE  7.95 (1.56)  8.13 (1.54)  .89 (.80 - .94)  0.52  TOTAL  7.94 (1.43)  8.12 (1.44)  .91 (.85 - .95)  0.43  29  Table 2.4 Descriptive Statistics and Inter-Rater Reliability Values for Importance Scores for Matched Items Mean T1 (SD) Mean T3 (SD) Inter-rater ICC  SEM for Inter-Rater  MEASURE  WhOM  IMPORTANCE T1 – T3  INSIDE  8.67 (1.24)  8.74 (1.09)  .76 (.58 - .87)  0.61  OUTSIDE  8.93 (0.99)  8.93 (1.07)  .74 (.53 - .85)  0.50  TOTAL  8.80 (0.95)  8.84 (1.00)  .78 (.61 - .87)  0.45  Table 2.5 Descriptive Statistics and Inter-Rater Values for Sat x Imp Scores for Matched Items Mean T2 (SD) Mean T3 (SD) Inter-rater ICC  SEM for Interrater  MEASURE  WhOM  Sat x Imp T2 – T3  INSIDE  70.20 (19.03)  71.40 (19.29)  .88 (.79 - .93)  6.59  OUTSIDE  72.73 (16.94)  73.73 (18.14)  .88 (.79 - .93)  5.87  TOTAL  70.88 (17.13)  72.57 (17.81)  .90 (.83 - .94)  5.42  Figure 2.8 Bland and Altman Graph for Inter-Rater Inside WhOM Sat Score Matched Items 30  Figure 2.9 Bland and Altman Graph for Inter-Rater Outside WhOM Sat Score Matched Items  Figure 2.10 Bland and Altman Graph for Inter-Rater Total Whom Sat Score Matched Items 31  Figure 2.11 Bland and Altman Graph for Inside WhOM Importance Score  Figure 2.12 Bland and Altman Graph for Outside WhOM Importance Score  32  Figure 2.13 Bland and Altman Graph for Total WhOM Importance Score  Figure 2.14 Bland and Altman Graph for Inside WhOM Satisfaction x Importance Score for Matched Items  33  Figure 2.15 Bland and Altman Graph for Outside WhOM Satisfaction x Importance Score for Matched Items  Figure 2.16 Bland and Altman Graph for Total WhOM Satisfaction x Importance Score for Matched Items  34  The following (Table 2.6) indicates the number of agreements for each rater related to the participation outcomes which were categorized using Hill‟s criteria. The columns containing the N values represent the number of items that each rater identified as a participation outcome in that area. Kappa values are also presented for each area as well as for an overall Kappa agreement. The Kappa values vary, from a low range of 0.40 for shopping to a high of 0.88 for child care. The overall Kappa was 0.71 representing a substantial agreement between raters.  Please see Appendix D for a list of examples of participation outcome and how they were classified into Hill‟s criteria. Table 2.6 Rater Agreement for Participation Outcomes Categorized into Hill's Criteria  Hill’s Criteria  N for Rater One (R1)  N for Rater Two (R2)  Kappa Value  Interpretation (according to Landis et al., 1977)  Work House and Yard work Shopping Personal Care Entertainment Active Leisure Passive Leisure Organization Education Child Care  28 34 33 40 37 41 36 3 10 5  29 35 41 45 35 42 42 1 8 4  .72 .58 .40 .46 .41 .50 .43 .49 .73 .88  Substantial agreement Moderate agreement Moderate agreement Moderate agreement Moderate agreement Moderate agreement Moderate agreement Moderate agreement Substantial agreement Almost perfect agreement Substantial agreement  OVERALL KAPPA  2.7  .71  Discussion This is the first study to examine the reliability of the WhOM. The standards which  were chosen to examine reliability were selected based on interpretations of ICC magnitudes as outlined by Portney & Watkins, 2002).  35  2.7.1 Test-Retest Reliability Test-retest reliability refers to the stability of the instrument and that it will measure a variable consistently over time (Portney & Watkins, 2002). Test-retest for satisfaction-only scores were all above 0.80 and therefore met the stated hypothesis. Overall the CI‟s are tight which indicates that we can confidently state that the population value for the ICC lies somewhere within these values. The ICC‟s for inside participation outcomes were higher than outside participation outcomes. It is unclear why this might occur; however, it is plausible that these types of participation outcomes are familiar to individuals and there is therefore less variability in what they do. In such a case, the participants would be less likely to experience variability in this area whereas participation in the community might vary on a daily basis.  The Bland and Altman plots and SEM coefficients provide additional information about absolute reliability to determine if real statistical change has occurred in a group of individuals. The SEM calculated for test re-test scores provides an indication of the range of scores that can be expected on retesting. Also, scores which fall above or below the SEM added or subtracted with the mean score for an individual demonstrate a real change. All the SEMs calculated for test re-test were small, indicating there was minimal error with re-testing participants. Moreover, a one unit change in satisfaction scores, as an example, suggests that real change beyond statistical noise has occurred.  The Bland and Altman graphs provide a good visual representation of the agreement in scores and of any bias or outliers that exist. All the Bland and Altman graphs for test re-test reliability demonstrate some outliers (data points outside the 95% confidence interval). With a few exceptions the outliers consisted of the same individuals. Post hoc analyses of these individuals did not reveal any insight as to why they experience such a change either between raters of collection time points (retesting) related to the variables we had. Additional data exploring whether or not these cases experienced a change in their health or wheelchair setup might have provided us with better insight.  36  The mean difference for scores was close to zero, indicating strong overall agreement for repeated measurement of participants. For almost every plot the data points were evenly distributed above and below the zero line indicating minimal bias due to change related to retesting (no learning effect) or rater.  Because test re-test scores were based on the participant identifying their participation outcome and rating its importance and satisfaction score at time one then re-rating the satisfaction score at time two, test re-test reliability only applies to individuals who maintain the same participation outcomes. This is a likely scenario for clinical use of the WhOM; however, it would be beneficial to provide additional reliability data in a future study for participants who decide to choose an additional participation outcome.  2.7.2 Inter-Rater Reliability Inter-rater reliability concerns the variation between two or more raters who measure the same group of subjects (Portney & Watkins, 2002). Inter-rater reliability was assessed using time two and time three. The second rater collected the scores three days after the first rater and did so over the telephone. Some variations in styles of interviewing as well as the method of data collection (over the phone versus in person) may be reasons to attribute error. However, inter-rater reliability is very good (>0.80) for the average scores for the total WhOM, inside, and outside scores and met the stated hypothesis. The inter-rater reliability ICC scores are higher than for test-retest reliability. In addition, means for time three (rater two) were also higher, indicating a systematic bias. Scores are positively skewed: this produces a ceiling effect, defined as 20% or greater of the responses at the maximal level of the scale (Andersen, 2000), which makes detect any improvements in scores difficult. We did expect higher scores overall as we tested individuals who had stable seating and who were presumably reasonably satisfied with their current wheelchairs. Higher scores at time three might also be explained by learning effect although the Bland Altman plots suggest that this did not happen. In addition, individual differences in taking the measurement as well as the method of collection at time three (over the telephone) could account for higher means at time three. Finally, the short time period between testing may have enabled participants to remember their scores.  37  To have a better understanding of the magnitude of disagreement between measurements the SEM values and Bland and Altman graphs are presented. The SEM calculated for inter-rater scores reflects the extent of expected error in different raters: these SEMs are all close to zero.  There were more outliers than expected in the Bland and Altman graphs for inter-rater reliability; however, these outliers were the same individuals across plots. Again, post hoc analyses did not reveal any insight as to why these individuals consistently fell outside the boundaries.  There was a trend for scores to be on the upper end of the scale when only satisfaction and importance scores were analysed suggesting a ceiling effect. The scores were more evenly distributed when satisfaction was multiplied by importance scores. In particular, there was a ceiling effect for the WhOM inter-rater reliability inside, outside and total scores for importance. Up to 32% of the important scores were recorded at the maximal level of the scale (10). This is not to be unexpected, though, as individuals were asked to choose participation outcomes that were important to them and, therefore, they would likely rate them at a high importance. Inter-rater agreement for items categorized using Hill‟s criteria met the stated hypothesis. This indicates that the raters were able to elicit similar participation outcomes from participants using the semi-structured interview techniques. This suggests that if the WhOM is conducted by different therapists, similar participation outcomes will likely be obtained.  2.8  Limitations A number of limitations listed below may have affected the findings of this study. Data  collected at time two was not a complete re-administration of the WhOM. Instead, only satisfaction scores were asked to be re-rated. Had the entire tool been re-administrated, some of the results may have changed for test re-test. 38  Collection of data at time three was completed over the telephone. Although this is a perceived intended use of the WhOM in clinical practice, it may have contributed to additional error, such as the ceiling effect which was observed at T3. In addition, reliability coefficients were higher as were the means at time three: this may be due to a learned effect from the tool.  We chose an ICC criteria of (0.8) or higher to indicate good reliability. We should have chosen one which was a bit lower and not set standards so high. As Portney and Watkins (2000) criteria suggest, an ICC = 0.75 or higher indicates good reliability. If we had selected this criterion the ICC‟s for importance scores would also have consistently demonstrated good reliability.  Collecting information regarding the participants‟ health status and wheelchair stability (changes in seating) at all data collection times would have provided additional information to investigate outliers. This should be considered in future studies.  Finally, we used a small sample that was collected using convenience sampling techniques. Moreover, our sample had a large number of individuals with tetraplegia which does not reflect the general population of individuals with SCI. These limitations suggest that generalizing to the larger population of individuals with SCI should be done with caution.  2.9  Conclusion This is the first study to examine the reliability of the WhOM for a sample of  individuals who have SCI. Based on the results of this study, researchers and clinicians can feel confident using the WhOM to identify participation outcomes for individuals who use wheelchairs. This study provides strong evidence of reliability for test re-test and inter-rater agreement in a small sample of individuals with SCI who use wheelchairs as their primary source of mobility.  39  References Andresen, E.M., (2000). Criteria for assessing the tools of disability outcomes research. Archives of Physical Medicine and Rehabilitation, 81(12 Suppl 2):S15 – 20. Barbeau, H., Ladouceur, M., Norman, K.E., Pepin, A., & Leroux, A. (1999). Walking after spinal cord injury: evaluation, treatment, and functional recovery. Archives of Physical Medicine and Rehabilitation, 80, 225-35. Canadian Paraplegic Association. Retrieved on June 23, 2006, from http://canparaplegic.org/national /level2.tpl?var1+story&var2+20001027122552 Donner, A. & Eliasziw, M. (1987). Sample size requirements for reliability studies. Statistics in Medicine, 6, 441-448. Donner, A. & Eliasziw, M. (1987). Sample sizse requirements for reliability studies. Statistics in Medicine, 6:441 – 448. Finch, E., Brooks, D., Stratford, P.W., & Mayo, N.E. (2002). Physical rehabilitation outcome measures: A guide to enhanced clinical decision making (2nd ed.). Hamilton: B.C. Decker. Garber, S.L., Bunzel, R. & Monga, T.N. (2002). Wheelchair utilization and satisfaction following cerebral vascular accident. Journal of Rehabilitation Research and Development, 39(4), 1-13. Juster, F.T. & Stafford, F.P. (Eds.). (1987). Time, goods, and well-being. Michigan: Institute for Social Research. Kittel, A., Di Marco, A., & Stewart, H. (2002). Factors influencing the decision to abandon manual wheelchairs for three individuals with a spinal cord injury. Disability and Rehabilitation, 24(1-3), 106-114. Landis, J.R. & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174. Mortenson, W.B., Miller, W.C., & Auger, C. (2008). Issues for the Selection of WheelchairSpecific Activity and Participation Outcome Measures: A Review, Archives of Physical Medicine and Rehabilitation, 89, 1177 – 86. Mortenson, W.B., Miller, W.C., & Miller-Polgar, J. (2007). The development of the Wheelchair Outcome Measure (WhOM). Disability and Rehabilitation, 2, 275-85. New, P. W. (2005). Functional outcomes and disability after nontraumatic spinal cord injury rehabilitation: results from a retrospective study. Archives of Physical Medicine and Rehabilitation, 86, 250-261. Portney, L.G. & Watkins, M.P. (2000). Foundations of clinical research applications to practice (2nd ed.). New Jersey: Prentice Hall Inc. Stratford, P. (2004). Getting more from the literature: Estimating the standard error of measurement from reliability studies. Physiotherapy Canada, 55, 225-229. Streiner, D. L. & Norman, G. R. (2003). Health Measurement scales: a practical guide to their development and use (3rd ed.). New York: Oxford University Press. Rankin, G. & Stokes, M. (1998). Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clinical Rehabilitation, 12, 187-199. World Health Organization. (2002). ICF: international Classification of Functioning Disability and Health. Geneva: WHO. Xu, G., Meyer, J.S., Thornby, J., Chowdhury, M., & Quach, M. (2002). Screening for mild cognitive impairment (MCI) utilizing combined mini-mental-cognitive capacity 40  examinations for identifying dementia prodromes. International Journal of Geriatric Psychiatry, 17, 1027-1033.  41  3  3.1  VALIDITY OF THE WHEELCHAIR OUTCOME MEASURE AMONG INDIVIDUALS WITH SPINAL CORD INJURIES2  Introduction Many individuals with spinal cord injuries (SCI) use a wheelchair as their primary  means for mobility. When a wheelchair does not fit properly, it may restrict the activities in which a person can participate. There are many reasons to account for why wheelchairs may not meet the person‟s needs. Some of these reasons are: the environment may not suit the wheelchair that was originally prescribed; the user‟s skills and/or physical abilities may have changed over time; and the components on the wheelchair, such as the seating, may have changed over time and no longer fit the user properly.  Typically, during a wheelchair assessment, the therapist and wheelchair-user work towards obtaining a wheelchair that will meet the person‟s needs and physical abilities as well as suit the environment in which it will be used. Thus, there are a variety of factors to consider. Current assessment tools which address the individual at the participation level are not wheelchair specific (Barlow, 1998). Others only address aspects of wheelchair seating at the level of body functions and structures, (Day & Jutai, 1996; Herzberg, 1993; Krasilovsky, 1993; Miller, Miller, Trenholm, Grant, & Goodman, 2004) or at the level of activity through evaluation of wheelchair skills (Kirby, Depuis, MacPhee, Coolen, Smith, Best, Newton, Mountain, MacLeod, & Bonaparte, 2004; Kilkens, Dallmeijer, de Witte, van der Woude, & Post, 2004). Typical of many rehabilitation measures, these tools generally reflect the clinician‟s concerns. Kittel, Di Marco, & Stewart, (2002) determined that when a wheelchair user is involved in decision-making regarding a particular assistive device, they are more likely to retain and use the device. In addition, a client-centred approach can provide a reliable, valid, and responsive measure of outcome after intervention (Carpenter, Baker, & Tyldesley, 2001; McColl, Paterson, Davies, Doubt, & Law, 2000). Measuring an individual‟s goals while taking into consideration  2  A version of this chapter will be submitted for publication. Garden, J. And Miller, W.C. Validity of the Wheelchair Outcome Measure. 42  environmental contexts provides an assessment of participation in life activities. The Canadian Measure for Occupational Performance is one tool that does consider a person‟s chosen occupations or activities. However, because it has very broad categories and is useful in many settings, it was found to lack the specificity that clinicians require when completing wheelchair seating assessments (Barlow, 1998). The Wheelchair Outcome Measure (WhOM) was developed to address the need for an evidence-based outcome measure that takes into account the user‟s participation outcomes. It is designed to evaluate the effectiveness of the wheelchair system prescription in terms of facilitating the user‟s ability to participate in life activities while using the wheelchair.  An important aspect of a measurement instrument is that it measures what it is intended to measure (Portney & Watkins, 2000). Unlike reliability, validity looks at „what‟ is being measured (Streiner & Norman, 2003). A definition of validity provided by Portney and Watkins (2000; p.79) is, “…the extent to which an instrument measures what it is intended to measure… plac[ing] an emphasis on the objectives of a test and the ability to make inferences from test score or measurements”.  There are many different ways to measure validity; however, the type of validation can generally be related to two circumstances: there are other tools or scales that measure similar constructs and attributes; and there are no tools that exist to measure the attribute or construct (Streiner & Norman, 2003). The latter case is termed construct validity, which is the type of validity that was tested in this study.  3.2  Types of Validity There are many different types of validity. The following section provides an overview  of these. Face validity describes an instrument‟s ability to test what it is supposed to test, and for this to occur, the definition of the concept being measured needs to be clear (Portney & Watkins, 2000). In a previous study, the WhOM demonstrated face validity (Mortenson, Miller, Miller-Polgar, 2007).  43  Construct validity refers to the ability of an instrument to measure an abstract construct, for instance, participation (Portney & Watkins, 2000). Cook and Beckman, (2006), discuss validity as a property of an inference and not of the instrument. They point out that “an instrument‟s scores will reflect the underlying construct more accurately or less but never perfectly” (p. 166.e10). Therefore, construct validation is an ongoing process as there can always be more learned about the construct and testing of its predictions (Portney & Watkins, 2000). When assessing construct validity some indicate that it is important to address how an instrument relates or does not relate to other instruments of the same and different constructs, which is termed convergent and discriminant validity (Streiner & Norman, 2005, Portney & Watkins, 2000; Finch, Brooks, Stratford, & Mayo, 2002). To have a better understanding of measuring the construct of participation in life activities while using a wheelchair, construct validity was measured by comparing the WhOM to several existing instruments that measure similar traits. Mortenson, Miller, & Auger, (2008), used the World Health Organization‟s (WHO) International Classification of Functioning, Disability and Health (ICF) as a framework to identify and to evaluate outcome measures useful for measuring activity and participation specific to wheelchairs. Out of eleven instruments, they determined that only six had sufficient validity information. This is true of the WhOM as well. Therefore, the purpose of this study is to document the construct validity of the WhOM in individuals who have a SCI and use a wheelchair as their primary means of mobility by comparing scores to several other measures known to measure similar constructs.  3.3  Purpose, Objectives and Hypotheses  3.3.1 Purpose The purpose of this study is to examine the validity of using the WhOM on individuals who have a spinal cord injury.  3.3.2 Objectives To examine construct validity of the WhOM using the following four measures: the Psychosocial Impact of Assistive Devices Scale (PIADS); the Quebec User Evaluation of 44  Satisfaction with Assistive Technology (QUEST); the Assessment of Life Habits (LIFE-H); and the Return to Normal Living Index (RNLI).  3.3.3 Hypotheses a. Scores on the WhOM will demonstrate a moderate (r > .50) positive relationship with scores on the PIADS subscales (competence, adaptability, and self-esteem). b. Scores on the WhOM will demonstrate a moderate (r > .50) positive relationship with scores on the QUEST (devices subscale) c. Scores on the WhOM will demonstrate a moderate (r > .50) positive relationship with scores on the RNLI. d. Scores on the total, inside, and outside WhOM as well as satisfaction scores for the total, inside, and outside WhOM will demonstrate a moderate (r > .50) positive relationship with scores on the following items within the LIFE-H subscales: personal care, housing, mobility, community life, employment, and recreation. The items in these subcategories of the LIFE-H are hypothesised to be correlated with satisfaction of participation outcomes with a wheelchair because each question necessitates the use of an assistive device, such as a wheelchair, to accomplish participations in life activities. The inside, outside and total scores for the WhOM have been separated out because different questions in the subscales of the LIFE-H are completed in inside and outside environments, therefore it will be useful to determine correlations based on the separate environments. For a detailed list of the specific hypotheses for each subsection, please refer to Appendix E.  3.4  Methods  3.4.1 Design and Participants The cross-sectional data used in this study were taken from a prospective one week testretest design with a consecutive sample of 50 individuals from Vancouver, British Columbia who have mobility impairments as a result of a spinal cord injury. Potential participants were identified and selected using four strategies as outlined in Figure 3.1. A letter describing the study was provided to potential subjects requesting that those interested contact the study coordinator. Those who were interested were screened by telephone by the study coordinator to ensure that they met the inclusion criteria. Participants were included if they were 20 years of 45  age or older, used a wheelchair as their primary means for mobility for at least four hours each day, had a diagnosis of complete or incomplete spinal cord injury at any level and were able to read, speak and understand English.  Individuals with cognitive impairment, as screened using the Cognitive Competency Screening Evaluation (CCSE), who scored below 24 were be excluded from the study (Xu, Meyer, Thornby, Chowdhury, & Quach, 2002). The CCSE was chosen as it is a motor-free assessment tool that does not require the individual to write, as disability in this area may be the case with this population. In addition, subjects were excluded if they had received a new wheelchair in the past six months or if they received a new wheelchair or seating component during the retest interval. Because wheelchair seating often requires several fittings to obtain the correct position, it was felt that these individuals‟ seating and positioning were not stable and were likely to change. These criteria were used to create a stable sample so that we could examine reliability in a separate arm of this study.  Individuals who met the study criteria were invited to their first appointment. Participants received $25.00 for each of their two visits to GF Strong during the data collection period. See Appendix B for samples of the letter of initial contact and recruitment advertisement.  Recruitment  GFS Database  Ad in BC Paraplegia  Therapist caseload  Snowball effect  Screening  53 participants  Figure 3.1 Recruitment and Protocol 46  3.4.2 Protocol As outlined in Figure 3.1, participants were recruited through several means and asked to contact the study research assistant if they were interested in participating in the study. The study was fully explained to the individual; if they were interested they were screened by the research assistant to determine whether or not they met the inclusion criteria for the study. If participants fit the criteria they then signed the consent form. The CCSE was administered to screen for cognitive impairments, determining eligibility. Demographic data such as age, sex, marital status, type of wheelchair, date of injury, and length of time with current wheelchair were then collected. A copy of the demographic questionnaire is available in Appendix C. The questionnaires used in this study were randomly administered to minimise response bias related to order effects.  3.4.3 Measures 3.4.3.1 Life Habits (LIFE-H) The LIFE-H is a questionnaire intended to gather information on life habits (regular activities and social roles in society) and what people accomplish in their environments (Noreau, Desrosiers, Robichaud, Fougeyrollas, Rochette, & Viscogliosi, 2004). The Disability Creation Process (DCP) provided the conceptual background for the development of the LIFEH (Noreau, Fougeyrollas, & Vincent, 2002). The DCP considers one‟s social participation as a result of the interaction between an individual‟s characteristics and capabilities (intrinsic factors) and environmental attributes in the life milieu (extrinsic factors) (Noreau, et al., 2002). The short form of the LIFE-H is comprised of 69 items which are categorised into 12 areas that include intrinsic (e.g. individual characteristics) and extrinsic (e.g. environmental attributes) elements of the DCP. The 12 areas include: nutrition, fitness, personal care, communication, housing, mobility, responsibility, interpersonal relationships, community life, education, employment and recreation. Participants are asked to determine their level of accomplishment for each life habit, the type of assistance that they require, and their level of satisfaction in performing the habit. The LIFE-H gives an item score ranging from 0, total assistance [not accomplished or achieved], to 9, optimal social participation [performed without difficulty or assistance]. A score for each item, each category (mean of items), or the mean of all categories can be obtained. Noreau, et al., (2004) report reliability coefficients of the global score (mean 47  of all categories) of the LIFE-H as excellent (ICC and 95% confidence intervals (95% CI): 0.95 (0.91–0.98) for test retest, and 0.89 (0.80-0.93) for inter-rater. The LIFE-H is reportedly able to discriminate between different levels of participation in daily activities (Desrosiers, Noreau, Robichaud, Fougeyrollas, Rochette, & Viscogliosi, 2004). The LIFE-H has been used in previous studies to measure long-term consequences of SCI on social participation (Noreau & Fougeyrollas, 2000) and is well suited to use in this study given its ability to measure participation in life activities. See Appendix F for a copy of the LIFE-H.  3.4.3.2 Psychosocial Impact of Assistive Devices Scale (PIADS) The PIADS is a 26-item self-report tool that is designed to assess how assistive technology (AT) influences the competence (12 items), adaptability (6 items) and self-esteem (8 items) of individuals who use assistive devices. The response scale for the PIADS ranges from - 3 to 3. Negative 3 indicates the maximum negative impact, 0 indicates no perceived impact and +3 indicates maximum positive impact (Jutai & Day, 2002). The score of – 3 was incorporated to be sensitive to negative impacts that may exist with the AT (Day, Jutai, & Campbell, 2002). The PIADS has good internal consistency (Cronbach‟s alpha of 0.87 to 0.95) and reportedly has good test-retest stability at the item levels (Day, et al., 2002). Content validity was determined by discussions with people with disabilities (Day et al., 2002). In a study by Day et al., (2002) they report that the PIADS has positive correlations with the Pleasure and Dominance on Mehrabian and Russell‟s PIADS scales that purport to tap environmental impact on emotional responses. In addition, individual items on the PIADS showed good corrected item-total correlations, defined as above 0.20 (Day et al., 2002). The PIADS scale has been used in different populations to measure AT users‟ perception of device satisfaction affecting quality of life as well as being used with individuals who have SCI. Jutai (2001) reported that individuals who have SCI who evaluated the perceived impact of their wheelchair on their quality of life were more likely than individuals with degenerative diseases to perceive it as a positive impact on their lives, especially in the domain of self-esteem. While the PIADS may be assessing the construct of quality of life, the WhOM, addresses importance and satisfaction with participation in chosen activities while using a wheelchair which is also related to quality of life (Day et al., 2002; Demers, Monette, Lapierre, Arnold, & Wolfson, 2002). Therefore it seems plausible that there may be a relationship between these two 48  variables. In addition, the PIADS is specifically designed to be used for assistive technology purposes and has been used in a validation study with the QUEST, another validity outcome measure to be used in this study (Demers et al, 2002). See Appendix G for a copy of the PIADS.  3.4.3.3 Quebec User Evaluation of Satisfaction with Assistive Technology 2.0 (QUEST) The QUEST is a measure of satisfaction with assistive technology. It consists of 8 items designed to explore satisfaction with the device (AT), 4 items to explore satisfaction with services related to the device, and 1 item to ask overall satisfaction with the assistive device. (Demers, et al., 2002). The QUEST yields a total score and two subscale scores: device and services. Response items range from 1 (not satisfied at all) to 5 (very satisfied). There is also a space to comment next to each item for the purpose of identifying the sources of the user‟s satisfaction or dissatisfaction (Demers, et al., 2002). The QUEST has good test-retest stability (ICC= 0.82, 0.82, 0.91) in studies of individuals using mobility devices (e.g. walkers, manual and power wheelchairs, and scooters) and has moderate Pearsons correlation coefficient (r =0.45) with the competence dimension and a weaker correlation coefficient (r =0.35) with the remaining two dimensions (adaptability and self-esteem) of the PIADS (Demers, et al., 2002). The authors of the QUEST note that further assessment of the services subscale is needed; however, for the purpose of this present study, the area of services is not the focus- only the area of „device‟ was analysed. The QUEST has been used in previous studies looking at seating and positioning as well as mobility (Demers, et al., 2002). The QUEST purports to measure satisfaction with the assistive device; in the case of this study, a wheelchair. Demers et al., (2002), indicate that satisfaction is closely linked with quality of life. The present study addresses satisfaction with participation outcomes while using a wheelchair and can be seen as being related to the overall quality of life. Therefore, the QUEST examines similar components of the construct of satisfaction with participation. See Appendix H for a copy of the QUEST.  3.4.3.4 Reintegration to Normal Living Index (RNLI) The RNLI is an 11-item self-report measure of global function designed to assess how an individual has returned to life after an injury or illness, such as a spinal cord injury (WoodDauphinee, & Williams, 1987; Wood-Dauphinee, Opzoomer, Williams, Marchand, & Spitzer, 49  1988). Scores range from 0 (does not describe my situation) to 2 (fully describes my situation). The scores are summed to provide an overall score ranging from 0 to 22 with higher scores indicating better reintegration. The RNLI can be interviewer administered (face to face or over the telephone), self-completed or completed by proxy (Wood-Dauphinee, & Williams, 1987; Wood-Dauphinee, et al, 1988). For the purpose of this study, given that some individuals may not have the ability to write, the interviewer recorded the scores. The RNLI is an easilyadministered tool that is completed in under 10 minutes and is widely used in rehabilitation research (Clarke, Back, Badley, Lawrence, & Williams, 1999; Daverat, Petit, Kemoun, Dartigues, & Barat, 1995; Friedland & Dawson, 2001).  3.4.4 Data Analysis Descriptive statistics were calculated for all participants. Statistical assumptions were assessed to ensure that the assumptions were not violated beyond the tolerance of the statistic. Spearman‟s correlation coefficient was used because the data were not normally distributed when observing plots of skew and kurtosis. In addition, Spearman‟s correlation coefficient is a more conservative estimate of validity: if the magnitude of the relationships were achieved, we would feel more confident about the hypothesized associations. For the purposes of this study we used a correlation coefficient of r > .50. Portney and Watkins, (2000) suggest correlations ranging from 0.50 to 0.75 are moderate to good. Data were analysed using SPSS version 17. Statistical significance was set at a p<0.05 level.  3.5  Results A total of 50 individuals with SCI provide data for the study. The mostly male sample  was approximately middle age on average and most had tetraplegia. Details about the sample characteristics are presented in Table 3.1.  50  Table 3.1 Participant Characteristics Sample (n=50)  The only measure that demonstrated moderate to good (r>.5) correlation with the total WhOM score was the QUEST (ρ = 0.65). Interestingly while the magnitude of the indoor and total WhOM scores were correlated strongly with the QUEST the outside WhOM score was not. We rejected our hypothesis for the PIADS and RNLI as neither measure was associated with the WhOM. Some individual items of the Spearman‟s correlations coefficients for the PIADS, RNLI, and QUEST measures and total WhOM, inside and outside score are listed in Table 3.2. Bolded items met our hypothesized relationship. Table 3.2 Spearman's Correlation Coefficients (p) for Satisfaction Multiplied by Importance Score for the WhOM Total, Inside and Outside Scores Measures  Subcategories or items from the measures  WhOM Total  WhOM Inside  WhOM Outside  0.187 0.057 0.228 -0.114  0.124 0.071 0.163 -0.047  0.130 -0.079 0.154 -0.169  0.645**  0.511**  0.197  PIADS Competency score Adaptability score Self-Esteem score RNLI QUEST  Score for assistive device subscale  ** Correlation is significant at the 0.01 level (2-tailed) * Correlation is significant at the 0.05 level (2-tailed) Bold values indicate hypothesized values met  Of the 26 items of the LIFE-H used in this study only questions related to recreation and employment met the stated hypothesis when examining WhOM scores calculated to include 51  importance x satisfaction. The Spearman‟s correlations coefficients for the LIFE-H subcategory questions and total, inside and outside WhOM scores range from a high of rho=0.59 for the outside WhOM score and „entering and exiting your home‟ to a low of rho=0.10 between the inside WhOM score and “participating in spiritual or religious practices”. Specific coefficients for the relationships examined are listed in Table 3.3. Table 3.3 Spearman's Correlation Coefficients for Satisfaction x Importance for the WhOM Total, Inside and Outside Scores Measures  Subcategories or items from the measures  WhOM Total  WhOM Inside  0.242  0.207  0.168  0.142  0.219  0.163  0.347*  0.288  WhOM Outside  Life-H Personal Care Attending to your personal hygiene Dressing/undressing upper extremities Dressing/undressing lower extremities Housing Maintaining your home Maintaining the grounds of your home Entering and exiting your home Moving around within your home Moving around outside of your home  0.295 0.313* 0.123  0.315 -0.078  0.235 0.087  0.391**  0.263  0.310*  0.354*  0.375**  0.244  0.225  0.441**  0.486**  0.299*  0.370**  0.415**  0.486**  0.358*  0.453**  0.281  0.396*  Mobility Getting around on streets or sidewalks Getting around on slippery or uneven surfaces Community Life Getting to public buildings in your community Entering and getting around in public buildings in your comm. Getting to commercial establishments in your community Entering and moving around in commercial estab in community Participating in social or community groups  52  Measures  Subcategories or items from the measures Participating in spiritual or religious practices  WhOM TOTAL  WhOM Inside  0.187  WhOM Outside 0.104  Employment Taking part in unpaid activities (Volunteering) Getting to your principal place of occupation Entering and moving around in your principal place of occ.  0.522**  .443**  0.547**  0.396*  0.405*  0.552**  0.590**  Recreation Participating in sporting or recreational activities Participating in artistic, cultural or craft activities Going to sporting events Going to artistic or cultural events Participating in tourist activities Taking part in outdoor activities  0.411*  0.367*  0.431**  0.192  0.164  0.183  0.538** 0.159 0.208 0.378*  0.530** 0.247 0.261 0.507**  ** Correlation is significant at the 0.01 level (2-tailed) * Correlation is significant at the 0.05 level (2-tailed) Bold values indicate hypothesized values met  A greater number of statistically significant correlations were observed when investigating the association between the WhOM and Life-H satisfaction scores. These include items involving housing, community life, employment, and recreation. However most of these scores did not meet the hypothesized level of relationship that we listed. Details of the Spearman`s correlation coefficients for the WhOM total, inside and outside satisfaction scores and the Life-H subscales are listed in Table 3.4.  53  Table 3.4 Spearman's Correlation Coefficients for the WhOM Total, Inside and Outside Satisfaction Scores and Selected Life-H Items Measures  Subcategories or items from the measures  WhOM Satisfaction Total  WhOM WhOM Satisfaction Satisfaction Inside Outside  Personal Care Attending to your personal hygiene  0.211  0.148  Dressing/undressing upper extremities  0.167  0.137  Dressing/undressing lower extremities  0.279*  0.229  Maintaining your home Maintaining the grounds of your home Entering and exiting your home Moving around within your home Moving around outside of your home  0.221 -0.027 0.341* 0.238 0.415*  0.221  Getting around on streets or sidewalks Getting around on slippery or uneven surfaces  0.389**  0.435**  0.226  0.234  0.531**  0.507**  0.367*  0.417**  Housing  0.342* 0.240  0.254 -0.068 0.308* 0.504**  Mobility  Community Life Getting to public buildings in your community Entering and getting around in public buildings in comm. Getting to commercial establishments in your community Entering and moving around in commercial established in community Participating in social or community groups Participating in spiritual or religious practices  0.548**  0.505** 0.417**  0.485**  0.370*  0.484**  0.400  0.348  Employment Taking part in unpaid activities (Volunteering)  0.497**  0.455**  0.555**  54  Subcategories or items from the measures  Measures  Getting to your principal place of occupation Entering and moving around in your principal place of occ.  WhOM Satisfaction Total  WhOM WhOM Satisfaction Satisfaction Inside Outside  0.473**  0.520**  0.622**  0.697**  Recreation Participating in sporting or recreational activities Participating in artistic, cultural or craft activities Going to sporting events Going to artistic or cultural events Participating in tourist activities Taking part in outdoor activities  0.545**  0.478**  0.540**  0.346  0.306  0.365  0.563** 0.177 0.268 0.522**  0.497** 0.152 0.363* 0.576**  ** Correlation is significant at the 0.01 level (2-tailed) * Correlation is significant at the 0.05 level (2-tailed) Bold values indicate hypothesized values met  3.6  Discussion This is the first study to empirically examine the validity of the WhOM in a sample of  individuals who have SCI. Support for construct validity was demonstrated as some of the hypotheses in terms of the magnitude and direction of the relationships with the QUEST and a subset of items from the Life-H were confirmed. However, other postulated relationships with items from the Life-H as well as the PIADS were supported in terms of the direction of the relationship but the magnitude was lower than was proposed a priori.  Specific questions from the satisfaction scale of the LIFE-H that are related to mobility demonstrated moderate correlations with the WhOM. The LIFE-H questionnaire was hypothesized to have moderate correlations with the total WhOM, inside and outside scores. The employment category questions related to volunteerism and entering and moving around in the principal place of occupation all demonstrated moderate correlations with the total and outside scores of the WhOM. The recreation category questions, which relate to attending sporting events and participating in outdoor activities, also had moderate correlations with outside and total WhOM scores. These questions are related to wheelchair use and allow the 55  individual to participate in volunteer, outdoor activities and occupation. It is clear, therefore, why they are correlated with the WhOM as they relate to the person‟s ability to participate in life activities while using a wheelchair.  The remaining LIFE-H items in the areas of personal care, housing, mobility, and community life failed to meet the magnitude of the stated hypotheses but did demonstrate statistically significant correlations. There may be several reasons why these questions did not meet our cut score criteria. The most obvious reason is that our criterion of a correlation of 0.50 was simply too high. In retrospect we should not have used a single threshold. However because there is no existing literature using the WhOM, we had to start somewhere: it is clear that we were overly optimistic.  In some cases we did not even achieve a statistically significant association. The most obvious explanation of this is that the WhOM and the selected LIFE-H items were not assessing the same construct. For instance, under the Housing section of the LIFE-H, a very low negative correlation was observed with “maintaining the grounds of your home”. It seems plausible that few people would select this as an „important‟ WhOM outcome and, therefore, there would be no overlap with this item.  A somewhat surprising result is that despite capturing up to 45 items related to personal care (Table 2.5) using the WhOM, we observed no association with the LIFE-H personal care questions (hygiene and dressing/undressing). It may be that individuals completed these activities without the use of their wheelchair and instead completed them while sitting on their bed; however, it is more likely that few of our participants selected this as an important WhOM goal. The final result would be that there would be a low or no association.  The PIADS measures how assistive devices influence the competence, adaptability and self-esteem of the user. Again, there were no statistically significant correlations with the WhOM, perhaps because the PIADS taps into elements of psychosocial response to receiving assistive technology rather than participation within mobility activities. This may partially explain the lack of association. However, our observation was that the PIADS was difficult for 56  the participants to understand. For instance, the PIADS requests that the subject state whether their self esteem is „better‟ because of device provision. Self esteem is a complex construct that is not well understood in lay terms. Without easy-to-understand measurement items, it is difficult to speculate on the quality of data that the tool is capturing. Collecting the participants‟ level of education may have provided additional insight into their ability to comprehend aspects of the PIADS.  Interestingly, there was absolutely no association between the WhOM and the RNLI and, in fact, the direction of the relationship was in opposite from what we anticipated. It is difficult to know for sure why we obtained this result; however, it is possible that satisfaction with performance of participations using the wheelchair just wouldn‟t adequately correlate with a measure of global function of reintegration into the community. Even post hoc analyses between the first three items of the RNLI, which are all mobility-oriented, and the WhOM scores revealed a correlation coefficient <0.10. The response options on the RNLI ask the participant to indicate whether the items reflect their situation or not. The lack of shared variance may therefore simply be because the constructs measured were too different.  Another potential explanation is that the on average, the participants were 16.1 years post injury. While the maximum score a person can obtain on the RNLI is 22 our samples‟ mean (data not shown) was 18.5 (range 16-22). Therefore, our sample was relatively well integrated into the community. The lack of variation in the RNLI scores might also explain the lack of a relationship with the WhOM.  Finally, the hypothesis that the WhOM would have a moderate or better correlation with QUEST (subscale for assistive devices) was confirmed although the outside WhOM was lower than expected. Given that both measures addresses satisfaction with assistive technology (in this case the wheelchair) the strength and magnitude of the association is not a surprise. It presents good support of the construct validity of the WhOM.  57  3.7  Limitations Several limitations were evident in this study. The bipolar nature of the PIADS scoring  (-3 to +3) may have lead to the null correlation with the WhOM. One way to remedy this maybe to make the scale on continuous positive scale (e.g. 1 to 7), however the current version for scoring uses an algorhythm and we did not foresee this potential problem. In future studies, a different scoring method may demonstrate significant positive correlations.  Additional information, such as education, would have supported further discussion of use of the PIADS. In addition, had the number of participants been higher, it is more likely that a correlation would have been found. Our anticipated relationships between the WhOM and other measures were not high: they did not share a lot of variance because they measure different constructs. Therefore, in future studies, a lower criterion (e.g. ρ = 0.4) with these other measures would be more appropriate. In spite of having a large magnitude of relationships (e.g. ρ = 0.47) some of our relationships were not statistically significant. This may have occurred because fewer people responded to items on the tool (e.g. LIFE-H). When individuals are allowed to choose whether or not to respond to items, this leads to a smaller n. A larger sample size would be required to demonstrate a statistically significant relationship.  3.8  Conclusions This is the first study to examine validity of the WhOM in individuals with a diagnosis  of a spinal cord injury. We have demonstrated preliminary support for the construct validity with scales that measure mobility and satisfaction. Further studies should be undertaken which will examine further the construct of mobility and satisfaction with scales which address participation using the ICF model.  58  References Barlow, I. Reliability and clinical utility of selected outcome measures with adult clients of seating clients [master‟s thesis] London, Ontario, Canada: University of Western Ontario, 1998. Carpenter, L., Baker, G. A., & Tyldesley, B. (2001). The use of the Canadian Occupational Performance Measure as an outcome of pain management programme. Canadian Journal of Occupational Therapy, 68, 16-22. Clarke, P. J., Black, S. E., Badley, E. M., Lawrence, J. M., & Williams, J. I. (1999). Handicap in stroke survivors. Disability and Rehabilitation, 21(3), 116-123. Cook, D.A. & Beckman, T.J. (2006). Current concepts in validity and reliability for psychometric instruments: theory and application. The American Journal of Medicine, 119(2), 116.e7-116.e16. Deverat, P., Petit, H., Kemoun, G., Dartigues, J. F., & Barat, M. (1995). The long term outcome in 149 patients with spinal cord injury. Paraplegia, 33, 665-68. Day, H. & Jutai, J. (1996). Measuring the psychosocial impact of assistive devices: the PIADS. Canadian Journal of Rehabilitation, 9(3), 159-168. Day, H. Jutai, J., & Campbell, K. A. (2002). Development of a scale to measure the psychosocial impact of assistive devices: Lessons learned and the road ahead. Disability and Rehabilitation, 24(1-3), 31-37. Demers, L., Monette, M., Lapierre, Y, Arnold, D.L., & Wolfson, C. (2002). Reliability, validity, and applicability of the Quebec User Evaluation of Satisfaction with assistive Technology (QUEST 2.0) for adults with multiple sclerosis. Disability and Rehabilitation, 24(1-3), 21-30. Desrosiers, J., Noreau, L., Robichaud, L., Fougeyrollas, P., Rochette, A., & Viscogliosi, C. (2004). Validity of the assessment of life habits in older adults. Journal of Rehabilitation Medicine, 36, 177-82. Donner, A. & Eliasziw, M. (1987). Sample sizse requirements for reliability studies. Statistics in Medicine, 6:441 – 448. Friedland, J. F. & Dawson, D. R. (2001). Function after motor vehicle accidents: A prospective study of mild head injury and posttraumatic stress. The Journal of Nervous and Mental Diseases, 189(7), 426-434. Finch, E. Brooks, D., Stratford, P.W., & Mayo, N.E. (2002). Physical rehabilitation outcome measures: A guide to enhanced clinical decision making (2nd ed.). Hamilton: B.C. Decker. Herzberg, S. R. (1993). Positioning the nursing home resident: An issue of quality of life. American Journal of Occupational Therapy, 47, 75-77. Jutai, J. (2001). The spychology of assistive technology. Rehabilitation Psychology, 43(3), 334 335. Jutai, J. & Day, H. (2002). Psychosocial impact of assistive devices scale (PIADS). Technology & Disability, 14, 107 – 111. Kilkens, O. J., Dallmeijer, A. J., de Witte, L. P., van der Woude, L. H., & Post, M. W. (2004). The wheelchair circuit: construct validity and responsiveness of a test to assess manual wheelchair mobility in persons with spinal cord injury. Archives of Physical Medicine and Rehabilitation, 85(3), 424-31.  59  Kirby, R. L., Dupuis, D. J., MacPhee, A. H., Coolen, A. L., Smith, C. Best, K. L., Newton, A. M., Mountain, A. D., MacLeod, D. A. & Bonaparte, J. P. (2004). The Wheelchair Skills Test (Version 2.4): Measurement properties. Archives of Physical Medicine and Rehabilitation, 85, 794-804. Kittel, A., Di Marco, A., & Stewart, H. (2002). Factors influencing the decision to abandon manual wheelchairs for three individuals with a spinal cord injury. Disability and Rehabilitation, 24(1-3), 106-114. Krasilovsky, G. (1993). Seating assessment and management in a nursing home population. Physical and Occupational Therapy in Geriatrics, 11, 25-38. McColl, M.A., Paterson, M., Davies, D., Doubt, L., & Law, M. (2000). Validity and community utility of the Canadian Occupational Performance Measure, Canadian Journal of Occupational Therapy, 67(1), 22-30. Miller, W.C., Miller, F., Trenholm, K., Grant, D., & Goodman, K. (2004). Development and preliminary assessment of the measurement properties of the Seating Identification Tool (SIT). Clinical Rehabilitation, 18, 317-25. Mortenson, W.B., Miller, W.C., & Auger, C. (2008). Issues for the Selection of WheelchairSpecific Activity and Participation Outcome Measures: A Review, Archives of Physical Medicine and Rehabilitation, 89, 1177 – 86. Mortenson, W.B., Miller, W.C., & Miller-Polgar, J. (2007). Measuring wheelchair intervention outcomes: Development of the wheelchair outcome measure. Disability and Rehabilitation, 2, 275-85. Noreau, L., Desrosiers, J., Robichaud, L., Fougeyrollas, P., Rochette, A., & Viscogliosi, C. (2004). Measuring social participation: reliability of the LIFE-H in older adults with disabilities. Disability and rehabilitation, 26(6), 346-52. Noreau, L. & Fougeyrollas, P. (2000). Long-term consequences of spinal cord injury on social participation: the occurrence of handicap situations. Disability and Rehabilitation, 22(4), 170-180. Noreau, L., Fougeyrollas, P., & Vincent C. (2002). The LIFE-H assessment of the quality of social participation. Technology and disability, 14, 113-118. Portney, L.G. & Watkins, M.P. (2000). Foundations of clinical research applications to practice (2nd ed.). New Jersey: Prentice Hall Inc. Streiner, D. L. & Norman, G. R. (2003). Health Measurement scales: a practical guide to their development and use (3rd ed.). New York: Oxford University Press. Wood-Dauphinee, S.L. & Williams, J. I. (1987). Reintegration to normal living as a proxy to quality of life. Journal of Chronic Disability, 40, 491-99. Wood-Dauphinee, S.L., Opzoomer, M. A., Williams, J. I., Marchand, B., & Spitzer, W. O. (1988). Assessment of global function: the reintegration to normal living index. Archives of Physical Medicine and Rehabilitation, 69(8), 583-90. Xu, G., Meyer, J.S., Thornby, J., Chowdhury, M., & Quach, M. (2002). Screening for mild cognitive impairment (MCI) utilizing combined mini-mental-cognitive capacity examinations for identifying dementia prodromes. International Journal of Geriatric Psychiatry, 17, 1027-1033.  60  4  4.1  CONCLUSIONS AND IMPLICATIONS  General Findings The Wheelchair Outcome Measure (WhOM) was developed to address the need for an  evidence-based outcome measure that takes into account the user‟s participation outcomes. It is designed to evaluate the efficacy of wheelchair system prescription based in terms of the users‟ ability to participate in life activities while using the wheelchair.  The purpose of this thesis was twofold: to provide estimates of the reliability of the WhOM (Wheelchair Outcome Measure) and to provide support for the validity of the WhOM. Results from this study indicate that the WhOM has excellent reliability (test-retest and interrater) and demonstrates moderate correlations with instruments that purport to measure satisfaction and importance with assistive devices, such as wheelchairs. The WhOM does not demonstrate significant correlations with instruments that tap into certain psychosocial aspects of satisfaction with assistive devices. In this final chapter, measurement standards will be discussed with respect to the WhOM and minor modifications to improve the overall use of the tool will be suggested. Clinical implications for using the WhOM will be outlined and further studies are recommended in order to ensure the WhOM can be used with other populations, can be administered in a different language and can be used as a responsiveness measure.  4.2  Measurement Standards  4.2.1 Reliability and Validity As outlined in the first chapter of this thesis, there is a need for valid, reliable, and clinically meaningful assessment tools to demonstrate evidence based outcomes in wheelchair seating and mobility interventions. Validity is an essential component to determine the psychometrics of an instrument. It describes the extent to which a tool measures that which it is intended to measure (Streiner & Norman, 2003), and is therefore a crucial aspect of accurate measurement. Reliability ensures that an instrument can reproduce results dependably as well as being consistent and free from error (Portney & Watkins, 2000).  61  The second chapter of this thesis examined reliability of the WhOM among individuals with a diagnosis of spinal cord injury. Reliability values reported include test-retest, interrater and agreement between raters. Test-retest and interrater demonstrated very good reliability as did agreement between raters for items identified. Hill‟s criteria were used to classify items into 10 categories in order to compare agreement of selected participants between raters. It would have been more useful to classify these items using the International Classification of Functioning, Disability and Health (ICF). Use of the ICF to categorize each item would have provided additional information in the context of participation outcomes. A recent study is underway which links these data with the ICF and found that individuals living with SCI in the community identified participation outcomes related to community, social, and civil life, domestic life and mobility categories of the ICF.  Validity of the WhOM was examined in the third of this thesis. The WhOM demonstrated moderate to good correlations with the subscale assistive devices of the QUEST and subscale questions of the LIFE-H which both encompass constructs of mobility. The WhOM does not correlate significantly with instruments that measure psychosocial aspects of assistive devices and return to normal living, as is the case with the PIADS and RNLI.  It would have been useful to gather information about the education of participants as the PIADS scale has complex language which, in the opinion of the rater, affected the administration of the tool and likely led individuals to choose varying answers which skewed results. The LIFE-H items related to mobility did not meet our stated hypothesis. The likely reason for this is a small sample size and that our criteria for the correlation were too high.  Overall, the WhOM demonstrated good reliability and moderate to good validity with instruments that purport to measure participation and satisfaction with assistive devices. Some modifications to the tool itself could be considered in future studies.  4.3  Suggested Modifications This study used version two of the WhOM. In the instructions for administration  section, it instructs the rater to “ask the client to identify activities they perform”. It would be 62  helpful to change this wording to participation outcomes to maintain the language on which the WhOM is based, the ICF model. Changing these words would then enable the rater to obtain participation outcomes rather than activities, a distinction which is made in the ICF manual and literature (World Health Organization, 2002; Mortenson, Miller & Auger, 2008).  4.3.1 Participation Outcomes During administration of the WhOM, participants are asked to list five participation outcomes. Often during the testing sessions, participants would come up with more than five outcomes. It would be helpful to have additional room on the form to list these and then ask the participant to narrow down the list to the top five. After this is accomplished, the rater can ask the participant to rate importance and satisfaction with these outcomes.  Strategies for administering the WhOM are outlined in the manual; however, interview skills are essential in order to assist the client in listing participation outcomes related to wheelchair use. Therapists should be provided with background information on the ICF model and be able to differentiate between activity and participation. This would allow for gathering outcomes related to involvement in life situations (participation), as defined by the World Health Organization, (2002). A video to assist with training therapists would be helpful, specifically to address difficult scenarios: for example, when asked to identify what participation outcomes the person completes while in the wheelchair inside their home, the person‟s response is “everything”.  During the testing it became evident to this particular rater that instead of asking participants to rate each item‟s importance and then satisfaction, it would be easier for participants to list each participation outcome, then to rate the importance of all outcomes before moving to rate satisfaction. This is suggested as it appears that some participants may have had difficulty with the term „satisfaction‟ and asking them to jump from importance to satisfaction for each item may have been confusing.  4.3.2 Scoring Clinicians and researchers using the WhOM can choose to use the satisfaction scores as standalone scores and have a range of scores for the satisfaction of the participation outcome 63  which range from 0 to 10. Alternatively, they can chose to score the WhOM with satisfaction multiplied by importance and the range of the WhOM score is from 0 to 100. Reliability evidence is provided for both scoring options; however, caution should be used when interpreting the scores from satisfaction only or importance only as there may be a ceiling effect, especially for importance scores as seen in this study. An overall satisfaction multiplied by importance score is advised as scores had a more even spread and higher ICC‟s as well as tighter confidence intervals.  4.4  Clinical Implications  4.4.1 Further Research Studies Now that psychometric properties of the WhOM are known to some extent, it is important to move toward testing it to determine its responsiveness and ability to detect clinically important differences when it is used in a clinic setting. It would also be useful to expand the population so that its use can be generalized to include other populations. The WhOM has the potential to be a very valuable tool, to be used clinically with various populations, to include the individual during the wheelchair prescription or treatment phase for wheelchair seating and to capture important participation outcomes from a client-centred perspective.  It would also be helpful to continue testing the validity of the WhOM, as Portney and Watkins (2000) indicate that construct validation is an ongoing process as there can always be more learned about the construct and testing of its predictions.  64  References Mortenson, W.B., Miller, W.C., & Auger, C. (2008). Issues for the Selection of WheelchairSpecific Activity and Participation Outcome Measures: A Review, Archives of Physical Medicine and Rehabilitation, 89, 1177 – 86. World Health Organization. (2002). ICF: international Classification of Functioning Disability and Health. Geneva: WHO.  65  APPENDIX A: WhOM Part I: PARTICIPATION Instructions for Administration: Ask the client to identify activities they perform in their wheelchair that are important to them by asking the two questions outlined below. Have the client score the importance of these activities and then ask them to rate their current level of satisfaction in performing these activities. If the client has scored their satisfaction with an activity ≤ 7, determine the underlying conditions (wheelchair/seating device or environmental barriers) that impair performance of this activity to assist with intervention planning. 1) Some people use their wheelchairs because they want to participate in activities in or around their home, such as the preparation of meals, watching TV, or gardening. What activities in your home would you use your wheelchair to perform?  0  Initial assessment Participation goals: Eg. Walking the dog Visiting my sister Watching a hockey game  Use this numerical scale to help fill in the table: 1 2 3 4 5 6 7 8 9  Date:  10  Reassessment Importance How important is this activity to you? (0 - 10) 0 = Not at all important 10 = Extremely important  Satisfaction 1 How satisfied are you with your current level of performance of this activity? (0 –10) 0 = Not satisfied at all 10 = Extremely satisfied  Importance x Satisfaction 1  Date:  Satisfaction 2 How satisfied are you with your current level of performance of this activity? (0 – 10) 0 = Not satisfied at all 10 = Extremely satisfied  Importance x Satisfaction 2  i. ii. iii. iv. v.  Total of importance x satisfaction 1 scores =  Change in satisfaction = satisfaction score 2  Total of importance x satisfaction 2 scores =  – satisfaction score 1  =  66  2. Some people use their wheelchairs because they want to participate in activities outside of their home such as dog walking, going for coffee, to work or to the park. What activities outside of your home or in your community would you use your wheelchair to perform?  0  Use this numerical scale to help fill in the table: 1 2 3 4 5 6 7 8 9  10  Initial assessment Date: Participation goals: Eg. Walking the dog Visiting my sister Watching a hockey game  Reassessment Date: Importance How important is this activity to you? (0 - 10) 0 = Not at all important 10 = Extremely important  Satisfaction 1 How satisfied are you with your current level of performance of this activity? (0 –10) 0 = Not satisfied at all 10 = Extremely satisfied  Importance x Satisfaction 1  Satisfaction 2 How satisfied are you with your current level of performance of this activity? (0 – 10) 0 = Not satisfied at all 10 = Extremely satisfied  i.  ii.  iii.  iv.  v.  Total of importance x satisfaction 1 scores =  Change in satisfaction = satisfaction score 2  Total of importance x satisfaction 2 scores =  – satisfaction score 1  =  Importance x Satisfaction 2  67  Part II: BODY FUNCTION 0  Use this numerical scale to help fill in the table: 1 2 3 4 5 6 7 8 9  Initial assessment Date: Questions  10  Reassessment Date: Time 2  Time 1  How would you rate your comfort while sitting in your wheelchair? (0 – 10) 0 = Not at all comfortable  10 = Extremely comfortable  How satisfied you are with the way your body is positioned in your wheelchair? (0 – 10) 0 = Not at all satisfied  10 = Extremely satisfied  Over the past month have you had any episodes of skin breakdown on your bottom? (please circle)  Y  N  Y  N  3a. If yes, in your opinion, how severe has your skin breakdown been? (0 - 10) 0 = Extremely severe  10 = Not at all severe  Score 1 Total =  Change = Score 2  - Score 1  =  Score 2 Total =  68  APPENDIX B: RECRUITMENT ADVERTISEMENT AND LETTER OF INITIAL CONTACT  Attention all wheelchair users! Do you use your wheelchair as your main way of getting around?  Are you over 20 years old? We would like to talk to you about your wheelchair mobility and comfort.  We need your participation for a research project, entitled “The WhOM: Reliability and Validity of a Client-Specific Outcome Measure of Wheelchair Intervention”  in cooperation with The University of British Columbia.  You will be paid a total of $50 and you will be interviewed 3 times over the course of 2 weeks.  The first 2 interviews will be held at GF Strong Rehabilitation Hospital in Vancouver and the last interview will be over the phone. The first interview will take 1½ -2 hours and the subsequent interviews will take less than 30 minutes of your time.   If you are interested in participating or if you would like more information please contact principal investigator Dr. Bill Miller or research coordinator, Priscilla Hsu. Phone: 604-XXX-XXXX  69  APPENDIX C: PRIMARY DATA COLLECTION SHEET Date of Birth  Gender  _____________ (dd/mm/yyyy)  Male 0  Female 1  Marital Status  Single  0  Married  1  Common Law  2  Separated  3  Divorced  4  Widowed  5  Diagnosis  Years of disability  _____________________  ___________ (yy/mm)  Years of wheelchair use  Time with current wheelchair  ___________  ___________  (yy/mm)  (yy/mm)  Type of wheelchair  Type of seat cushion  Type of back cushion  70  APPENDIX D: EXAMPLES OF PARTICIPATION OUTCOMES CLASSIFIED INTO HILL’S CRITERIA Hill’s criteria  Participation outcomes identified using the WhOM  Market Work  Taxidermy Sharpening knives/scissors  House and Yard Work  Dusting and sweeping Sweeping the floors  Child Care  Taking daughter to daycare Playing with son  Services/Shopping  Grocery shopping Ordering food  Personal Care  Brushing teeth Shaving  Education  Attending school/classes Studying  Organizations  Going to church Attending community meetings  Social Entertainment  Going to the movies Going to concerts  Active Leisure  Camping Sailing  Passive Leisure  Reading Watching television  71  APPENDIX E: DETAILED VALIDITY HYPOTHESIS FOR QUESTIONS IN LIFE-H SUBSCALES 1. Scores on the total, inside, and outside WhOM as well as satisfaction scores for the total, inside, and outside WhOM will demonstrate a moderate (r > .50) positive relationship with scores on the following LIFE-H subscale: a) personal care: Attending to your personal hygiene, dressing and undressing the upper half of your body, dressing and undressing the lower half of your body. b) housing: maintaining your home, maintaining the grounds of your home, entering and exiting your home, moving around within your home, moving around outside your home c) mobility: getting around on streets or sidewalks, getting around on slippery or uneven surfaces d) community life: getting to public buildings in your community, entering and getting around in public buildings in your community getting to commercial establishments in your community, entering and moving around in commercial establishments in your community, participating in social or community groups, participating in spiritual or religious practices e) employment: taking part in unpaid activities, getting to your principal place or occupation, entering and moving around in your principal place of occupation f) recreation: participating in sporting or recreational activities, participating in artistic, cultural or craft activities, going to sporting events, going to artistic or cultural events, participating in tourist activities, taking part in outdoor activities.  2. Validity: Scores on the WhOM inside subscale will demonstrate a moderate (0.50) to high (0.75) correlation with scores on the following LIFE-H subscale questions: a) personal care: Attending to your personal hygiene, dressing and undressing the upper half of your body, dressing and undressing the lower half of your body. b) housing: maintaining your home, entering and exiting your home, moving around within your home c) employment: taking part in unpaid activities  72  d) recreation: participating in sporting or recreational activities, participating in aartistic, cultural or craft activities  3. Validity: Scores on the WhOM subscale outdoor will demonstrate a moderate (0.50) to high (0.75) correlation with scores on the following LIFE-H subscale questions: a) housing: maintaining your home, maintaining the grounds of your home, entering and exiting your home, moving around outside your home b) mobility: getting around on streets or sidewalks, getting around on slippery or uneven surfaces c) community life: getting to public buildings in your community, entering and getting around in public buildings in your community, getting to commercial establishments in your community, entering and moving around in commercial establishments in your community, participating in social or community groups, participating in spiritual or religious practices d) employment: taking part in unpaid activities, getting to your principal place or occupation, entering and moving around in your principal place of occupation e) recreation: participating in sporting or recreational activities, participating in artistic, cultural or craft activities, going to sporting events, going to artistic or cultural events, participating in tourist activities, taking part in outdoor activities.  4. Validity: Satisfaction scores on the total WhOM will demonstrate a moderate (0.50) to high (0.75) correlation with scores on the following LIFE-H subscale questions: a) personal care: Attending to your personal hygiene, dressing and undressing the upper half of your body, dressing and undressing the lower half of your body. b) housing: maintaining your home, maintaining the grounds of your home, entering and exiting your home, moving around within your home, moving around outside your home c) mobility: getting around on streets or sidewalks, getting around on slippery or uneven surfaces d) community life: getting to public buildings in your community, entering and getting around in public buildings in your community getting to commercial establishments in your community, entering and moving around in commercial establishments in your 73  community, participating in social or community groups, participating in spiritual or religious practices e) employment: taking part in unpaid activities, getting to your principal place or occupation, entering and moving around in your principal place of occupation f) recreation: participating in sporting or recreational activities, participating in artistic, cultural or craft activities, going to sporting events, going to artistic or cultural events, participating in tourist activities, taking part in outdoor activities.  5. Validity: Satisfaction scores on the WhOM inside subscale will demonstrate a moderate (0.50) to high (0.75) correlation with scores on the following LIFE-H subscale questions: a) personal care: Attending to your personal hygiene, dressing and undressing the upper half of your body, dressing and undressing the lower half of your body. b) housing: maintaining your home, entering and exiting your home, moving around within your home. c) employment: taking part in unpaid activities d) recreation: participating in sporting or recreational activities, participating in aartistic, cultural or craft activities  6. Validity: Satisfaction scores on the WhOM subscale outdoor will demonstrate a moderate (0.50) to high (0.75) correlation with scores on the following LIFE-H subscale questions: a) housing: maintaining your home, maintaining the grounds of your home, entering and exiting your home, moving around outside your home b) mobility: getting around on streets or sidewalks, getting around on slippery or uneven surfaces c) community life: getting to public buildings in your community, entering and getting around in public buildings in your community getting to commercial establishments in your community, entering and moving around in commercial establishments in your community, participating in social or community groups, participating in spiritual or religious practices d) employment: taking part in unpaid activities, getting to your principal place or occupation, entering and moving around in your principal place of occupation 74  e) recreation: participating in sporting or recreational activities, participating in artistic, cultural or craft activities, going to sporting events, going to artistic or cultural events, participating in tourist activities, taking part in outdoor activities.  75  APPENDIX F: LIFE-H SHORT FORM  76  77  78  79  80  81  82  83  84  85  86  87  88  89  90  91  92  93  94  95  96  APPENDIX G: PSYCHOSOCIAL IMPACT OF ASSISTIVE DEVICES SCALE (PIADS)  97  APPENDIX H : QUEBEC USER EVALUATION OF SATISFACTION WITH ASSISTIVE TECHNOLOGY (QUEST)  Quebec User Evaluation of Satisfaction with assistive Technology QUEST (Version 2.0) Technology device: _____________________________________ User name: ____________________________________ Date of assessment :____________________________ The purpose of the QUEST questionnaire is to evaluate how satisfied you are with your assistive device and the related services you experienced. The questionnaire consists of 12 satisfaction items.  For each of the 12 items, rate your satisfaction with your assistive device and the related services you experienced by using the following scale of 1 to 5. 1 not satisfied at all  2 not very satisfied  3 more or less satisfied  4 quite satisfied  5 very satisfied   Please circle or mark the one number that best describes your degree of satisfaction with each of the 12 items.  Do not leave any question unanswered.   For any item that you were not "very satisfied", please comment in the section comments.  Thank you for completing the QUEST questionnaire.  98 © L. Demers, R. Weiss-Lambrou & B. Ska, 2000  1 not satisfied at all  2 not very satisfied  3 more or less satisfied  4 quite satisfied  5 very satisfied  ASSISTIVE DEVICE How satisfied are you with, 1. the dimensions (size, height, length, width) of your assistive device? Comments:  1  2  3  4  5  2. the weight of your assistive device? Comments:  1  2  3  4  5  3. the ease in adjusting (fixing, fastening) the parts of your assistive device? Comments:  1  2  3  4  5  4. how safe and secure your assistive device is? Comments:  1  2  3  4  5  5. the durability (endurance, resistance to wear) of your assistive device? Comments:  1  2  3  4  5  6. how easy it is to use your assistive device? Comments:  1  2  3  4  5  7. how comfortable your assistive device is? Comments:  1  2  3  4  5  8. how effective your assistive device is (the degree to which your device meets your needs)? Comments:  1  2  3  4  5  99 © L. Demers, R. Weiss-Lambrou & B. Ska, 2000  1 not satisfied at all  2 not very satisfied  3 more or less satisfied  4 quite satisfied  5 very satisfied  SERVICES How satisfied are you with, 9. the service delivery program (procedures, length of time) in which you obtained your assistive device? Comments:  1  2  3  4  5  10. the repairs and servicing (maintenance) provided for your assistive device? Comments:  1  2  3  4  5  11. the quality of the professional services (information, attention) you received for using your assistive device? Comments:  1  2  3  4  5  12. the follow-up services (continuing support services) received for your assistive device? Comments:  1  2  3  4  5   Below is the list of the same 12 satisfaction items. PLEASE SELECT THE THREE ITEMS that you consider to be the most important to you. Please put an X in the 3 boxes of your choice. 1.  Dimensions  7.  Comfort  2.  Weight  8.  Effectiveness  3.  Adjustments  9.  Service delivery  4.  Safety  10. Repairs/servicing  5.  Durability  11. Professional service  6.  Easy to use  12. Follow-up services 100  © L. Demers, R. Weiss-Lambrou & B. Ska, 2000  QUEST Scoring Sheet This page is for scoring the answers to your questions. DO NOT WRITE ON THIS PAGE.  Number of non-valid responses __________________________________  Device subscale score __________________________________________ For items 1 to 8, add the ratings of the valid responses and divide this sum by the number of valid items in this scale.  Services subscale score _________________________________________ For items 9 to 12, add the ratings of the valid responses and divide this sum by the number of valid items in this scale.  Total QUEST score ____________________________________________ For items 1 to 12, add the ratings of the valid responses and divide this sum by the number of valid items.  The 3 most important satisfaction items: ____________________________________ ____________________________________ ____________________________________  101 © L. Demers, R. Weiss-Lambrou & B. Ska, 2000  QUEST (version 2.0)  1 not satisfied at all  2 not very satisfied  102  © L. Demers, R. Weiss-Lambrou & B. Ska, 2000  3 more or less satisfied  4 quite satisfied  5 very satisfied  APPENDIX I : REINTEGRATION TO NORMAL LIVING INDEX  Reintegration to Normal Living Index (RNL)  Statement  Record 1 response for each statement Does not describe my situation 0  Partially describes my situation 1  Fully describes my situation 2  1. I move around my living quarters as I feel is necessary (wheelchairs, other equipment or resources may be used) 2. I move around my community as I feel necessary (wheelchairs, other equipment or resources may be used) 3. I am able to take trips out of town as I feel are necessary (wheelchairs, other equipment or resources may be used) 4. I am comfortable with how my self-care needs (dressing, feeding, toileting, bathing) are met (adaptive equipment, supervision and/or assistance may be used) 5. I spend most of my days occupied in a work activity that is necessary or important to me (could be paid employment, housework, volunteer work, school, etc.) (adaptive equipment, supervision and/or assistance may be used) 6. I am able to participate in recreational activities (hobbies, crafts, sports, reading, television, games, computers, etc.) as I want to (adaptive equipment, supervision and/or assistance may be used) 7. I participate in social activities with family, friends and/or business acquaintances as is necessary or desirable to me (adaptive equipment, supervision and/or assistance may be used) 8. I assume a role in my family which meets my needs and those of other family members (Family means people with whom you live and/or relatives with whom you don’t live but see on a regular basis) (adaptive equipment, supervision and/or assistance may be used) 9. In general I am comfortable with my personal relationships 10. In general I am comfortable with myself when I am in the company of others 11. I feel that I can deal with life events as they happen  Total =  103  APPENDIX J: UNIVERSITY OF BRITISH COLUMBIA BEHAVIOURAL RESEARCH ETHICS BOARD CERTIFICATE OF APPROVAL  104  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0067673/manifest

Comment

Related Items