UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A comparison of participation instruments based on the International Classification of Functioning, Disability… Noonan, Vanessa Karine 2009

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2009_fall_noonan_vanessa.pdf [ 1.71MB ]
Metadata
JSON: 24-1.0067308.json
JSON-LD: 24-1.0067308-ld.json
RDF/XML (Pretty): 24-1.0067308-rdf.xml
RDF/JSON: 24-1.0067308-rdf.json
Turtle: 24-1.0067308-turtle.txt
N-Triples: 24-1.0067308-rdf-ntriples.txt
Original Record: 24-1.0067308-source.json
Full Text
24-1.0067308-fulltext.txt
Citation
24-1.0067308.ris

Full Text

A COMPARISON OF PARTICIPATION INSTRUMENTS BASED ON THE INTERNATIONAL CLASSIFICATION OF FUNCTIONING, DISABILITY AND HEALTH by VANESSA KARINE NOONAN B.Sc.(P.T.), Queen’s University, 1994 M.Sc., The University of British Columbia, 1997  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Health Care and Epidemiology)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  June 2009 © Vanessa Karine Noonan, 2009  ABSTRACT Purpose and Objectives: The purpose of this study was to compare participation instruments based on the International Classification of Functioning, Disability and Health (ICF). It was hypothesized that information from these instruments would not be equivalent due to differences in how the concept of participation was operationalized. Methods: Eleven participation instruments were identified in the literature. Content validity was assessed by mapping the content in 8 of 11 instruments to the ICF classification. Individuals treated for spinal conditions at an acute hospital were followed-up and 545 took part in the empirical study. Subjects completed five participation instruments [Impact on Participation and Autonomy (IPA), Keele Assessment of Participation (KAP), Participation Measure-Post Acute Care (PM-PAC), Participation Objective Participation Subjective (POPS), World Health Organization Disability Assessment Schedule II (WHODAS II)]. A sub-sample (n=139) was used to assess test-retest reliability. Measurement properties, including score distribution, internal consistency, test-retest reliability, dimensionality, convergent/discriminant validity and known-group validity were assessed. Results: The eight participation instruments adequately covered the concept of participation but two instruments (Participation Scale, WHODAS II) contained irrelevant content. In the empirical study all instruments demonstrated considerable ceiling effects, except for the POPS. Internal consistency of the domains was >0.70. The IPA and WHODAS II had the highest values for test-retest reliability, with intraclass correlation coefficients >0.70. The minimal detectable change, as a percentage of the scale score range was on average between 20% and 30%. A confirmatory factor analysis of the IPA, PM-PAC and WHODAS II demonstrated adequate model fit. Correlations were generally higher among similar domains  ii  of the WHODAS II, IPA, KAP and PM-PAC and as expected the lowest correlations were observed with the objective domains of the POPS. All instruments demonstrated knowngroup validity. Conclusions: More direct comparisons of these instruments are needed to advance our understanding of this concept and assist users. The IPA, PM-PAC and WHODAS II have similar measurement properties. The KAP was designed for population-based studies and the POPS includes objective and subjective information, which may explain some of the differences observed. To date, there is no gold standard and future studies should continue testing these instruments.  iii  TABLE OF CONTENTS Abstract  ........................................................................................................................... ii  Table of Contents .................................................................................................................iv List of Tables .......................................................................................................................vi List of Figures ................................................................................................................... viii List of Abbreviations............................................................................................................ix Acknowledgements ..............................................................................................................xi Co-Authorship Statement .................................................................................................. xiii 1  Introduction .........................................................................................................1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8  2  Overview of Disability.........................................................................................1 Models of Disability: An Historical Overview......................................................1 An Overview of the International Classification of Functioning, Disability and Health ..................................................................................................................4 Conceptualization and Measurement of Participation ...........................................6 Research Needs and Study Justification................................................................9 Study Objectives and Thesis Organization .........................................................11 Summary ...........................................................................................................12 References .........................................................................................................15 A Review of Participation Instruments Based on the International Classification of Functioning, Disability and Health.................................................................20  2.1 2.2 2.3 2.4 2.5 3  Introduction .......................................................................................................20 Methods.............................................................................................................22 Results ...............................................................................................................27 Discussion..........................................................................................................34 References .........................................................................................................62 Content Comparison of Instruments Assessing Participation Based on the International Classification of Functioning, Disability and Health ......................76  3.1 3.2 3.3 3.4 3.5 4  Introduction .......................................................................................................76 Methods.............................................................................................................78 Results ...............................................................................................................83 Discussion..........................................................................................................87 References .......................................................................................................106 Comparing the Reliability of Five Participation Instruments.............................111  4.1 4.2 4.3 4.4 4.5  Introduction .....................................................................................................111 Methods...........................................................................................................114 Results .............................................................................................................121 Discussion........................................................................................................124 References .......................................................................................................148  iv  5  Comparing the Validity of Five Participation Instruments ................................154 5.1 5.2 5.3 5.4 5.5  6  Introduction .....................................................................................................154 Methods...........................................................................................................156 Results .............................................................................................................166 Discussion........................................................................................................170 References .......................................................................................................195 Discussion, Conclusions and Recommendations ..............................................202  6.1 6.2 6.3 6.4 6.5  Summary of Results .........................................................................................202 Study Strengths and Limitations.......................................................................210 Recommendations............................................................................................217 Conclusions .....................................................................................................226 References .......................................................................................................229  Appendices........................................................................................................................235 Appendix A. UBC Research Ethics Board Certificate........................................................235 Appendix B. Missing Data ................................................................................................236 B.1. Overview of Terminology ..................................................................................236 B.2. Summary of Missing Data in this Study..............................................................236 B.3. Management of Missing Data.............................................................................238 B.4. Discussion..........................................................................................................239 B.5. References..........................................................................................................241 Appendix C. Additional Analyses Related to Chapter 4 (Reliability) .................................246 Appendix D. Additional Analyses Related to Chapter 5 (Validity).....................................275  v  LIST OF TABLES Table 2.1. Overview of participation instruments.................................................................42 Table 2.2. Description of questions, response options and scores.........................................45 Table 2.3. Health conditions included in development, testing and use of participation instruments ........................................................................................................49 Table 2.4. Evidence related to reliability..............................................................................52 Table 2.5. Evidence related to validity.................................................................................55 Table 2.6. Evidence related to responsiveness......................................................................59 Table 3.1. Examples of linking questions to ICF categories .................................................95 Table 3.2. Summary of the data abstracted from the participation instruments .....................96 Table 3.3. ICF categories in the component activities and participation ...............................97 Table 3.4. ICF categories in the components body functions and environmental factors.....100 Table 3.5. Number of questions with ICF categories (%, number questions with ICF categories/total questions) ................................................................................102 Table 3.6. Content validity of the participation instruments ...............................................105 Table 4.1. Characteristics of the study respondents for the entire sample ...........................131 Table 4.2. Descriptive information and floor/ceiling effects for the participation instruments based on the entire sample (n=545) ..................................................................133 Table 4.3. Internal consistency and standard error of measurement for the entire sample (n=545) ............................................................................................................135 Table 4.4. Test-retest reliability for the participation instruments (n=139) .........................138 Table 4.5. Summary* of the study results for the score distribution and reliability.............141 Table 5.1. Characteristics of the study respondents for the entire sample (n=545) ..............179 Table 5.2. Descriptive information for the participation instruments for the entire sample (n=545)* ..........................................................................................................181 Table 5.3. Results of the item-to-scale tests for the entire sample (n=545) .........................184 Table 5.4. Results of the confirmatory factor analysis for the entire sample .......................186 Table 5.5. Strength of the correlations among similar participation instrument domains using the entire sample (n=545).................................................................................188 Table 5.6. Correlations* among participation domains and disease-specific instruments for the entire sample (n=545)†...............................................................................190 Table 5.7. Correlations* among the participation instrument domains and the quality of life instrument using the entire sample (n=545) ......................................................193 Table 5.8. Summary* of the study results for validity ........................................................194  vi  Table 6.1. Summary* of study results ................................................................................228 Table B.1. Missing data for the questions within the instruments for the entire sample (n=545) ............................................................................................................242 Table B.2. Missing data for the subjects completing the instruments for the entire sample (n=545) ............................................................................................................245 Table C.1. Descriptive information and floor/ceiling effects for the participation instruments for the three spine groups .................................................................................247 Table C.2. Internal consistency for the participation instruments for the three spine groups ........................................................................................................................254 Table C.3. Test-retest reliability for the three spine groups ................................................256 Table C.4. Standard error of measurement and minimal detectable change for the three spine groups..............................................................................................................258 Table D.1. Descriptive summary for the disease-specific and Life Satisfaction-11 instruments for the three spine groups .................................................................................276 Table D.2a. CFA results for IPA baseline model for the entire sample (n=545) .................277 Table D.2b. CFA results for IPA adjusted model for the entire sample (n=545) .................279 Table D.3a. CFA results for PM-PAC baseline model for the entire sample (n=510)* .......281 Table D.3b. CFA results for PM-PAC adjusted model for the entire sample (n=510)* .......283 Table D.4a. CFA results for WHODAS II baseline model for the entire sample (n=545) ...285 Table D.4b. CFA results for WHODAS II adjusted model for the entire sample (n=545)...287 Table D.5. Correlations* among similar participation domains for the entire sample (n=545) ........................................................................................................................289 Table D.6. Correlations* among participation domains and LiSat-11 questions for the entire sample (n=545) ................................................................................................294 Table D.7. Known-group validity hypothesis testing*........................................................298  vii  LIST OF FIGURES Figure 1.1. The ICF Model. .................................................................................................13 Figure 1.2. The ICF Classification. ......................................................................................14 Figure 4.1. IPA mean scores and the minimal detectable change as a percentage of each domain score range ..........................................................................................142 Figure 4.2. KAP mean question scores and the minimal detectable change as a percentage of each question (sub domain) score range ...........................................................143 Figure 4.3. PM-PAC mean scores and the minimal detectable change as a percentage of each domain score range ..........................................................................................144 Figure 4.4. POPS Objective and Subjective mean scores and the minimal detectable change as a percentage of each domain score range......................................................145 Figure 4.5. WHODAS II mean scores and the minimal detectable change as a percentage of each domain score range ..................................................................................147 Figure C.1. IPA Bland and Altman Plots ...........................................................................261 Figure C.2. KAP Bland and Altman Plots..........................................................................263 Figure C.3. PM-PAC Bland and Altman Plots ...................................................................266 Figure C.4. POPS Bland and Altman Plots ........................................................................269 Figure C.5. WHODAS II Bland and Altman Plots .............................................................272 Figure D.1a. IPA baseline CFA model for the entire sample ..............................................309 Figure D.1b. IPA adjusted CFA model for the entire sample..............................................311 Figure D.2a. PM-PAC baseline CFA model for the entire sample......................................313 Figure D.2b. PM-PAC adjusted CFA model for the entire sample .....................................315 Figure D.3a. WHODAS II baseline CFA model for the entire sample................................317 Figure D.3b. WHODAS II adjusted CFA model for the entire sample ...............................319  viii  LIST OF ABBREVIATIONS CFA  Confirmatory factor analysis  CFI  Comparative fit index  ICC  Intraclass Correlation Coefficient  ICF  International Classification of Functioning, Disability and Health  ICIDH  International Classification of Impairments, Disabilities and Handicaps  IPA  Impact on Participation and Autonomy  IRT  Item response theory  KAP  Keele Assessment of Participation  LiSat-11  Life Satisfaction-11  MDC  Minimal detectable change  MIC  Minimal important change  NDI  Neck Disability Index  ODQ  Oswestry Disability Questionnaire  PARTS/M  Participation Survey/Mobility  PIPP  Perceived Impact of Problem Profile  PM-PAC  Participation Measure-Post Acute Care  PM-PAC-CAT  Participation Measure-Post Acute Care-Computerized Adaptive Test  POPS  Participation Objective Participation Subjective  P-Scale  Participation Scale  RMSEA  Root mean square error of approximation  ROPP  Rating of Perceived Participation  SCI  Spinal cord injury  ix  SEM  Standard error of measurement  SRFM  Self-Reported Functional Measure  SRMR  Standardized root mean residual  VGH  Vancouver General Hospital  WHODAS II  World Health Organization Disability Assessment Schedule II  x  ACKNOWLEDGEMENTS I would like to acknowledge the tremendous support and guidance I received from my supervisor Dr. Jacek Kopec. His expertise in the area of measurement and his willingness to share his time enabled me to complete my doctoral degree and dissertation and for that I am extremely grateful. In addition, I would like to sincerely thank my committee members. It is because of the mentoring I received from Dr. Marcel Dvorak that I decided to pursue my doctoral degree. He offered all the resources available within the Vancouver Spine Program and provided his clinical expertise, which were both instrumental in being able to conduct this study, and for that I cannot thank him enough. In addition, I would like to thank Dr. Luc Noreau who was so supportive of my work and was always willing to share his immense knowledge on community integration. Thank you also to Drs. Joel Singer and Louise Mâsse who were always so supportive and willing to assist me. I would also like to thank the staff at the Vancouver Spine Research Program; in particular I would like to thank Dale Dreffs who helped me execute this study, the study coordinators Keely Johnston and Anna Chan who were invaluable in conducting this study as well as Hongbin Zhang who assisted me with some of the analyses. I would also like to offer a sincere thank you to all of the study subjects who generously offered their time and enabled me to learn what participation really means. Most of all, I would like to thank my family. The extraordinary love and support I receive from both my husband Chris and my parents Maureen and Godfrey have enabled me to accomplish my goals, and for that I dedicate this work to them. As for my children Nathan, Lauren and my upcoming third child, thank you for bringing such joy to my life and  xi  constantly reminding me what is important. I would also like to recognize and thank my brother Greg, all of my other family members, friends and colleagues who provided me with such unyielding support throughout the years. In closing, I would like to thank all the faculty, staff and students at the School of Population and Public Health for creating such a rich learning environment and sharing all of their knowledge and time. Thank you also to the Canadian Institutes of Health Research, the Vancouver Foundation, the Paetzold Chair in Spinal Cord Injury Clinical Research and the University of British Columbia for their financial support.  xii  CO-AUTHORSHIP STATEMENT This study was primarily conceived, conducted and written by the doctoral candidate. The data analyses were conducted in part by the doctoral candidate. The co-authors of manuscripts from this study made contributions only as is commensurate with a supervisory committee or as experts in an area of study. All of the co-authors provided direction and support and offered critical feedback on the manuscripts submitted for publication. The doctoral candidate was responsible for the writing and final content of all the manuscripts.  xiii  1 INTRODUCTION 1.1 Overview of Disability Disability has a significant impact on both the individual and society. In 2006, 4.4 million people in Canada reported having limitations in their activities, equaling a disability rate of 14.3%.1 Data from the Canadian post-censal survey, Participation and Activity Limitation Survey (PALS), reported that the disability rates increase with age for both men and women.1 In persons aged 15 to 24 the disability rate is 4.7% and it increases to 56.3% in persons over the age of 75.1 Problems associated with mobility (difficulty walking, climbing stairs and standing for 20 minutes) are the most commonly reported disability, affecting 11.5% of Canadians.1 In addition, 11.1% of Canadians have difficulty with everyday tasks such as getting dressed or undressed, cutting up food or picking up an object from the floor.1 In Canada the disability rate increased by 1.9% between 2001 and 2006.1 It is expected that disability rates will continue to rise. Reasons cited for this increase include an aging population, advances in medical and trauma care enabling more people to survive, expansion in the definition of disability and more accurate estimates of the number of people affected.2  1.2 Models of Disability: An Historical Overview There are numerous definitions of disability and over time there have been important developments in how disability is viewed. This next section will briefly describe four models that present how disability has been conceptualized from the medical, rehabilitation and social perspectives and more recently from an integrated or biopsychosocial perspective. The intent is to highlight how the conceptualization of disability has evolved over time and to provide examples of models from each of these perspectives.  1  1.2.1 Medical Model of Disability The medical or biomedical model is based on the idea that disability follows a linear process starting with etiology of a disease that produces pathology and ultimately results in disability.3 This model was developed during the introduction of modern medicine in the 19th Century.4 Persons with disabilities were often expected to benefit from medical treatment and the disability was viewed to be inherent to the individual.4;5 In the medical model the focus was on the disease and the goal was to cure the person so that he or she could return to normal functioning within society.5 Although the medical model was useful in understanding the etiology of disease, it was limited in understanding how persons with chronic disabilities were able to function in society.5  1.2.2 Rehabilitation Model of Disability Following the Second World War the rehabilitation model of disability was introduced to describe the rehabilitation process used to re-integrate veterans with chronic conditions back into society.4 Rehabilitation models of disability evolved from the sociological paradigm called functionalism that focuses on how people function within society. Society is viewed as a system containing inter-related functioning parts (e.g. family, education system) and in order for the system to function effectively, persons with disabilities must be able to fulfill their expected roles.5 In this paradigm the role of heath care professionals is to treat persons with disabilities so they can return to the community and become contributing members of society.5 The two most common rehabilitation models are 1) Nagi’s Disablement Model originally published in 19656 and later revised in 19767 and 2) the International Classification of Impairments, Disabilities and Handicaps (ICIDH) published in 1980 by the World Health 2  Organization5. These rehabilitation models differ from the medical model by recognizing the consequences of disease or pathology at both an individual- and societal-level.5 Nagi’s model specifically acknowledged the social environment which was an important contribution.8 The medical model and rehabilitation model do have some similarities. Firstly, both models view disability as residing within the individual. Secondly, the focus still remains on the disease or pathology and the resulting functional limitations.5  1.2.3 Social Model of Disability In contrast to the medical and rehabilitation model, the social model views disability as a socially created problem.4 Disability is a normal part of life in this model and problems in the social and physical environment limit persons with disabilities in their daily life.4 The Independent Living Model is one of the most common social models and was published in 1979.5;9 The Independent Living Model criticizes the medical and rehabilitation models for making persons with disabilities dependent on medical care instead of enabling them to be consumers and activists.5  1.2.4 Biopsychosocial Model of Disability The final perspective, the biopsychosocial perspective, attempts to integrate aspects of both the medical and social models.10;11 In the biopsychosocial model, disability results from an interaction among biological, personal and social factors.10 The International Classification of Functioning, Disability and Health (ICF) is an example of a biopsychosocial model. It was published in draft form (Beta-1 and Beta-2 version of the ICIDH-2) in 1997 and 199912 and then officially in 200111. The ICF captures aspects of both the medical and social models of disability in an attempt to provide a “coherent view of different perspectives 3  of health from a biological, individual and social perspective” (page 20). Unlike the ICIDH, the ICF does not focus on disease but instead describes health and health-related states.11 Features of the later rehabilitation models, which stressed the importance of the interaction of a person and the environment, are incorporated in the ICF and as a result the distinction between rehabilitation models and biopsychosocial models is not always made.10 Today the ICF has been used in both the research13;14 and the clinical setting15;16 and adopted by 191 countries8. The ICF has been described as having “great promise to provide a synthesis of earlier models of disablement and to provide….. a universal language with which to discuss disability and related phenomena” (page 727)10. The ICF has also been criticized for reasons such as 1) it is still focused on the health condition and the resulting functional limitations5; 2) there is a lack of clarity in the terminology used in the ICF17-19; 3) it excludes the concept of quality of life19;20; and 4) the essential aspect of disability, the dynamic interaction between the person and their environment (person-environment interaction) is stated more clearly in the rehabilitation models8. A detailed description of the ICF is provided in Section 1.3.  1.3 An Overview of the International Classification of Functioning, Disability and Health In addition to the ICF model (see Figure 1.1) there is also an ICF classification which groups health and health-related domains.11 The ICF classification is described in a hierarchical structure from the perspective of the body, the individual, and society. It includes two parts: 1) functioning and disability and 2) contextual factors. Within these two parts are four components: 1) body functions and structures, 2) activities and participation, 3) environmental factors and 4) personal factors.  4  1.3.1 Functioning and Disability Functioning and disability comprises body functions, body structures, activities and participation. Body functions are the physiologic functions of the body systems such as heart function and sleep function and body structures are anatomical parts of the body such as the organs, limbs and their components. Impairments are problems in body functions or structures such as a significant deviation or loss.11 In the ICF model, activity is the execution of a task or action by an individual. Activity limitations are difficulties an individual may have in executing activities.11 Participation is defined as the involvement in a life situation and participation restrictions are problems individuals may experience in the involvement in life situations.11 Although the ICF components activity and participation are differentiated in the conceptual model, in the classification the activity and participation chapters (also referred to as domains) are combined and provided in a single list. Four options for differentiating these two concepts have been suggested and will be described in Section 1.4.1.  1.3.2 Contextual Factors Environmental factors make up the physical, social and attitudinal environment in which people live and conduct their lives. These factors are external to individuals and can have a positive or negative influence on the individual’s performance as a member of society, on the individual’s capacity to execute actions or tasks, or on the individual’s body functions or structures.11 Environmental factors interact with the components of 1) body functions and structures and 2) activities and participation. Personal factors are not described in ICF classification because of the large social and cultural variance associated with them. These 5  factors are the particular background of an individual’s life and living, and comprise features of the individual that are not part of a health condition or health states. Examples of personal factors include gender, race, age, lifestyle and coping styles.  1.3.3 ICF Units of Classification In the ICF classification, the components are labeled with letters: body structures (s), body functions (b), activities and participation (d), and environmental factors (e). As mentioned previously, personal factors are not specified. Within each component the categories are organized hierarchically and assigned a numeric code (see Figure 1.2). The categories are nested so the chapters include all the detailed subcategories. The first-level category is the chapter number (1 digit) then there is the second-level category (2 digits), third (1 digit) and sometimes fourth-level (1 digit). An example demonstrating the coding from the activities and participation component is d4 Mobility (chapter heading/first-level category), d450 walking (second-level category), and d4500 walking short distances (thirdlevel category).  1.4 Conceptualization and Measurement of Participation The ICF concept of participation has recently received considerable attention in the literature. Although the terminology in disability models may differ, the idea of measuring a person’s participation in his or her life activities has evolved and helps to understand the impact of disability. In this next section the following aspects of participation relevant to this study will be described and include 1) differentiating the concepts of activity and participation and 2) operationalizing the concept of participation.  6  1.4.1 Differentiating the Concepts of Activity and Participation Although activity and participation are differentiated in the biopsychosocial model, in the classification they are combined and there is a single list of domains (chapters) covering various actions and life areas. The user is provided with four options on how activity can be distinguished from participation: 1) divide the activity and participation domains and do not allow for any overlap; 2) allow for partial overlap between activity and participation domains; 3) define participation as broad categories within the domains and activity as the more detailed categories, with either partial or no overlap; and 4) allow for complete overlap in the domains considered to be activity and participation.11 It has been suggested that activity and participation are distinct concepts that must be differentiated conceptually and operationally.20;21 Some suggest that activity reflects basic tasks (e.g. the ability to climb stairs) and participation reflects more complex life roles (e.g. preparing meals).22 Others have stated that activity is at an individual-level and performed alone, whereas participation is at a social-level and performed with others.20 The lack of clarity in the model has affected the measurement of participation. Perenboom and Chorus23 reviewed participation instruments and evaluated them according to the ICF classification to determine how well they assess participation. The authors reported that very few instruments just assess participation and the distinction between activity and participation varied considerably.23 Nordenfelt24 suggested that rather than trying to conceptually distinguish between activity and participation it may be preferable to combine them and refer to them as ‘actions’ based on philosophical action theory. The actions then could be qualified based on their simplicity or complexity rather than using the capacity and performance qualifiers. Jette et al.25 recently published data supporting Nordenfelt’s proposal24. These authors conducted an exploratory factor analysis using questions assessing physical functioning and disability and 7  were not able to reproduce the ICF domains.25 Instead the factors consisted of multiple ICF activity and participation domains and reflected the complexity of the action rather than the content. For example, a question such as ‘managing your money’, or ‘keeping track of expenses and paying bills’ loaded on the applied cognition factor which is different than classifying them as part of the domain ‘Major life areas’ in the ICF classification. Differentiating the concepts of activity and participation is an important issue that needs to be considered when reviewing results from this study. Although this study will not focus on differentiating activity and participation in detail, it will highlight how these two concepts have been differentiated when developing participation instruments.  1.4.2 Operationalizing the Concept of Participation There have been some important advances in how the concept of participation is operationalized. In the ICIDH26 the term handicap is used instead of participation. Handicap is defined as “the disadvantage of a given individual resulting from an impairment or a disability that limits or prevents the fulfillment of a role that is normal (depending on age, sex and social and cultural factors) for that individual” (page 182)26. Handicap is measured by determining how much a person deviates from roles fulfilled by an able-bodied member of society.27 Instruments such as the Craig Handicap Assessment Reporting Technique27, which assess handicap based on the ICIDH, uses objective or quantitative information and a comparison is made with societal norms. It is possible to measure objective aspects of participation and not assess handicap. For example, the frequency a person engages in work activities could be measured quantitatively but not compared to societal norms. The term handicap was replaced with the term participation in the ICF due to its pejorative connotations.28 The definition of participation in the ICF refers more to the personal 8  fulfillment of roles rather than fulfilling roles deemed important by society, which is an important change.29 It was recognized that the individual’s experiences and preferences were not captured in some handicap instruments and this information is required to understand the individual’s needs and problems.30 Today this perspective is referred to as assessing subjective aspects of participation.28 Subjective aspects of participation rely on ratings from the person regarding aspects of participation such as performance (e.g. difficulty, limitations) and satisfaction. The change in terminology in the ICF compared to the ICIDH as well as the perspective in which participation is measured has resulted in a new era of measuring participation.  1.5 Research Needs and Study Justification If participation is going to be a meaningful outcome of rehabilitation, it is critical that instruments are available to measure participation. Although the idea of participating in life roles is not new, the term participation as defined in the ICF is relatively new and as a result it has not been clearly defined.20 It has been recommended that in order to advance our understanding of disability, new instruments operationalizing the concepts in the ICF model are developed and then tested so consensus can be obtained on how participation should be defined and quantified.20 Rather than trying to retrofit existing instruments to the ICF classification, instruments should be developed using the ICF.20 To date, there has not been a comparison of participation instruments developed using the ICF. The intent of this study was to address the recommendations stated above by first identifying and then comparing participation instruments developed using the ICF. Persons with spinal conditions are an ideal population to empirically evaluate instruments assessing participation as these conditions are very prevalent and cause tremendous 9  disability. Low back pain will affect one in five adults in Canada.31 It is reported to cost $100 billion per year in the United States, with the majority of these costs due to lost wages and productivity from an inability to work31;32. Spinal cord injuries (SCI) are another substantial health problem in Canada. Traumatic events causing SCI typically occur in males in their thirties who will live a normal lifespan with their disability and the likelihood these individuals will be able to fully participate in life activities is low.33 Only 14% of persons with traumatic SCI are employed at six months following discharge from rehabilitation and the majority are either unemployed or on disability (25% and 35%, respectively).34 Persons with SCI (traumatic and non-traumatic) also report severe limitations in self-care, recreation, fulfilling their family role and education.35 Finally, with an aging population one of the most notable trends is the increase in spinal injuries from falls in the elderly.36 These types of injuries often result in spinal column fractures without neurological involvement37 but nonetheless profoundly affect all aspects of participation including self-care, mobility and community life.38 Participation in life activities is an important outcome to both the individual and to society as a whole. It is imperative that clinicians and researchers have instruments to measure participation outcomes. Clinicians must ensure outcomes of treatment are meaningful to the individual and researchers need to assess interventions and provide evidence to guide government policy. Results from this study will not only provide information on how to measure participation in persons with spinal conditions but will also contribute to our understanding on how best to operationalize the concept of participation, which is relevant to all health conditions and to the disablement and enablement process.  10  1.6 Study Objectives and Thesis Organization The overall purpose of this study was to compare participation instruments based on the ICF. There were four study objectives: 1) to identify and compare all participation instruments based on the ICF in the existing literature; 2) to assess the content validity of participation instruments identified in the literature review; 3) to empirically assess the score distributions and reliability of participation instruments in persons with spinal conditions; and 4) to empirically assess the cross-sectional construct validity in persons with spinal conditions. It was hypothesized that there would be differences in the content of the instruments as well as the measurement properties, due to differences in how the ICF concept of participation was operationalized in each of the instruments. This study was conducted to fulfil the requirements for a doctoral dissertation. The results from this study were written in a manuscript-based format, which is a collection of manuscripts (chapters) suitable for submission to a journal, prefaced by an introductory and concluding chapter. The appendices include additional study information as well as empirical results on the three spinal diagnostic groups, which comprise the study sample, since it was not possible to include this much detail using the manuscript-based format. In total there are six chapters. The second chapter provides details on the systematic search that was conducted to identify the participation instruments as well as describes their reported measurement properties. In the third chapter, an evaluation of the content validity of participation instruments identified in the literature review is reported. The fourth chapter 11  describes the empirical results for the score distributions, internal consistency and test-retest reliability of the instruments in persons with spinal conditions. Chapter five contains the empirical findings of the cross-sectional construct validity in persons with spinal conditions. Finally, the sixth chapter summarizes the study findings, discusses the strengths and weaknesses of this study and provides recommendations for future research.  1.7 Summary There have been tremendous advances in our understanding of disability since the medical model was first introduced in the 19th Century. Today the concept of participation, as defined in the ICF, reflects the interaction between the person and his or her environment. Participation has been cited as an important rehabilitation outcome and there has been considerable progress in developing instruments assessing participation since the first draft of the ICF was published in 1997. To date, it is not known how these instruments compare. The purpose of this study was to compare participation instruments developed using the ICF. It was hypothesized that information from these instruments would not be equivalent due to differences in how the concept of participation was operationalized.  12  Figure 1.1. The ICF Model.  © World Health Organization, 2001, by permission.  13  Figure 1.2. The ICF Classification39.  © Quality of Life Research, 2007, by permission.  14  1.8 References 1. Statistics Canada. Participation and Activity Limitation Survey 2006: Analytical Report. 1-38. 2007. Ottawa. 2. Smart J. Defining Disability. Disability, Society and the Individual. Gaithersburg: Aspen Publishers Inc., 2001:3-31. 3. Minaire P. Disease, illness and health: theoretical models of the disablement process. Bull.World Health Organ 1992;70:373-9. 4. Kaplan D. The definition of disability. [The Center for an Accessibility Society]. 2006; Available at: http://www.accessiblesociety.org/topics/demographicsidentity/dkaplanpaper.htm. Accessed August 25, 2006. 5. Lutz BJ, Bowers BJ. Understanding How Disability is Defined and Conceptualized in the Literature. Rehabil Nurs 2003;28:74-8. 6. Nagi SZ. Some conceptual issues in disability and rehabilitation. In: Sussman MB, ed. Sociology and Rehabilitation. Washington, DC: American Sociological Association, 1965:100-13. 7. Nagi SZ. An epidemiology of disability among adults in the United States. Milbank Mem Fund Q Health Soc 1976;54:439-67. 8. Masala C, Petretto DR. From disablement to enablement: Conceptual models of disability in the 20th century. Disabil Rehabil 2008;30:1233-44.  15  9. Dejong G. Independent living: from social movement to analytic paradigm. Arch Phys Med Rehabil 1979;60:435-46. 10. Jette AM. Toward a common language for function, disability, and health. Phys Ther 2006;86:726-34. 11. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization, 2001. 12. World Health Organization. International Classification of Functioning and Disability Beta-2 Draft Full Version. Geneva: World Health Organization, 1999. 13. Cieza A, Stucki G, Weigl M et al. ICF Core Sets for low back pain. J Rehabil Med 2004;69-74. 14. Cieza A, Stucki G. New approaches to understanding the impact of musculoskeletal conditions. Best Pract Res Clin Rheumatol 2004;18:141-54. 15. Rosenbaum P, Stewart D. The World Health Organization International Classification of Functioning, Disability, and Health: a model to guide clinical thinking, practice and research in the field of cerebral palsy. Semin Pediatr Neurol 2004;11:5-10. 16. Stucki G, Cieza A, Ewert T et al. Application of the International Classification of Functioning, Disability and Health (ICF) in clinical practice. Disabil Rehabil 2002;24:281-2. 17. Imrie R. Demystifying disability: a review of the International Classification of Functioning, Disability and Health. Sociol Health Illn 2004;26:287-305.  16  18. Schuntermann MF. The implementation of the International Classification of Functioning, Disability and Health in Germany: experiences and problems. Int J Rehabil Res 2005;28:93-102. 19. Wade DT, Halligan P. New wine in old bottles: the WHO ICF as an explanatory model of human behaviour. Clin Rehabil 2003;17:349-54. 20. Whiteneck G. Conceptual models of disability: past, present, and future. In: Field MJ, Jette AM, Martin L, eds. Workshop on Disability in America: A New Look. Washington, D.C.: National Academies Press, 2006:50-66. 21. Institute of Medicine. Definition and monitoring of disability. In: Field MJ, Jette AM, eds. The future of disability in America. Washington, D.C.: The National Academies Press, 2007:35-64. 22. Jette AM, Haley SM, Kooyoomjian JT. Are the ICF Activity and Participation dimensions distinct? J Rehabil Med 2003;35:145-9. 23. Perenboom RJ, Chorus AM. Measuring participation according to the International Classification of Functioning, Disability and Health (ICF). Disabil Rehabil 2003;25:577-87. 24. Nordenfelt L. Action theory, disability and ICF. Disabil Rehabil 2003;25:1075-9. 25. Jette A, Tao W, Haley SM. Blending activity and participation sub-domains of the ICF. Disabil Rehabil 2007;29:1742-50.  17  26. World Health Organization. International Classification of Impairments, Disabilities and Handicaps. A manual of classification relating to consequences of diseases. Geneva: World Health Organization, 1980. 27. Whiteneck GG, Charlifue SW, Gerhart KA et al. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil 1992;73:519-26. 28. Noreau L, Fougeyrollas P, Post M et al. Participation after spinal cord injury: the evolution of conceptualization and measurement. J Neurol Phys Ther 2005;29:147-56. 29. Cardol M, de Haan RJ, de Jong BA et al. Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Arch Phys Med Rehabil 2001;82:210-6. 30. Carr AJ, Thompson PW. Towards a measure of patient-perceived handicap in rheumatoid arthritis. Br J Rheumatol 1994;33:378-82. 31. Cassidy JD, Cote P, Carroll LJ et al. Incidence and course of low back pain episodes in the general population. Spine 2005;30:2817-23. 32. Katz JN. Lumbar disc disorders and low-back pain: socioeconomic factors and consequences. J Bone Joint Surg Am 2006;88 Suppl 2:21-4. 33. Fisher CG, Noonan VK, Dvorak MF. Changing face of spine trauma care in North America. Spine 2006;31:S2-S8. 34. Canadian Institute for Health Information. Life after traumatic spinal cord injury: from inpatient back to the community. 1-8. 2006. Ottawa.  18  35. Lund M.L., Nordlund A, Nygard L et al. Perceptions of participation and predictors of perceived problems with participation in persons with spinal cord injury. J Rehabil Med 2005;37:3-8. 36. Kannus P, Niemi S, Palvanen M et al. Continuously increasing number and incidence of fall-induced, fracture-associated, spinal cord injuries in elderly persons. Arch Intern Med 2000;160:2145-9. 37. Irwin ZN, Arthur M, Mullins RJ et al. Variations in injury patterns, treatment, and outcome for spinal fracture and paralysis in adult versus geriatric patients. Spine 2004;29:796-802. 38. Tinetti ME, Williams CS. The effect of falls and fall injuries on functioning in community-dwelling older persons. J Gerontol A Biol Sci Med Sci 1998;53:M112M119. 39. Geyh S, Cieza A, Kollerits B et al. Content comparison of health-related quality of life measures used in stroke based on the International Classification of Functioning, Disability and Health (ICF): a systematic review. Qual Life Res 2007;16:833-51.  19  2 A REVIEW OF PARTICIPATION INSTRUMENTS BASED ON THE INTERNATIONAL CLASSIFICATION OF FUNCTIONING, DISABILITY AND HEALTH1 2.1 Introduction Since the introduction of the International Classification of Impairments, Disabilities and Handicaps (ICIDH)1 by the World Health Organization (WHO) in 1980 there has been tremendous interest in understanding how individuals with a health condition live their lives. Although the ICIDH was a significant step in understanding the disablement process, the model had limitations, most notably it did not include external factors such as the environment which is necessary to understand the genesis of handicap.2 In 1997 a revision of the ICIDH called Beta Version of ICIDH-2 was released and by 2001 the World Health Assembly approved the International Classification of Functioning, Disability and Health, referred to as the ICF.3 In the ICF the concept of participation replaced the ICIDH concept of handicap. Participation is defined in the ICF as involvement in a life situation and participation restriction is defined as problems an individual may experience while involved in life situations.4;5 This is an important change from assessing handicap which focused on the disadvantages for an individual in life roles considered normal (based on age, sex, social and cultural factors). In addition, the ICF model recognizes the importance of contextual factors, which include personal factors (e.g. age, coping style) and the environment (e.g. physical surroundings), that are seen to interact with the individual and influence their level of function. The evolution from viewing disability as a consequence of a disease or disorder in the ICIDH towards a biopsychosocial perspective, which incorporates aspects of social 1  A version of this chapter has been published. Noonan VK, Kopec JA, Noreau L, Singer J, Dvorak MF. (2009). A review of participation instruments based on the International Classification of Functioning, Disability and Health. Disabil Rehabil. May 19:1-19 (Epub ahead of print).  20  models of disability in the ICF has paved the way for a new generation of health instruments.3 The perspective of how to assess participation has also evolved over time. Original measures of handicap6;7 are primarily based on observable information such as the frequency an individual performs roles (e.g. hours worked) and measure objective aspects of participation8. Carr and Thompson9 were the first to comment on the limitations of measuring objective information, stating that the individual’s perspective on the impact of the disease and the problems they experience in performing their life roles is not captured. This led to the development of instruments which assess the cognitive, emotional and motivational aspects of participation as perceived by the individual, and measure subjective aspects of participation.10 Although the ICF model does not include a subjective dimension, the replacement of the term handicap with participation and the inclusion of a broad range of life roles make the model compatible with capturing subjective information. In 1999 Cardol et al.11 conducted a literature review and identified 20 instruments that assess handicap and reviewed how handicap was defined and measured (objective versus subjective). Since that time the ICIDH-2 and ICF have been published and new instruments have been developed using this conceptual model. To date, there has not been a review of instruments developed using the ICIDH-2 or ICF to assess how participation has been operationalized. Therefore, the purpose of this review was to 1) identify instruments developed to assess participation; 2) describe how participation has been operationalized; and 3) summarize the measurement properties of the instruments in various health conditions. This review may assist clinicians and researchers in selecting a participation instrument and identify areas for future research.  21  2.2 Methods 2.2.1 Identification of Studies A systematic search of the literature was conducted in September 2007 to identify all instruments that assess participation and used the ICIDH-2 or ICF model. The search terms were grouped and included terms related to 1) the conceptual model (ICF, ICIDH-2, WHO); 2) participation (participation, handicap, patient participation, consumer participation, community re-integration, community integration, social adaptation, social adjustment, independent living, daily life activity, instrumental activities of daily living, quality of life); and 3) instrument (questionnaire, instrument, instrument evaluation, health survey, health assessment questionnaire, psychometrics, disability evaluation, outcome assessment, rehabilitation). Seven databases were searched [Medline; CINAHL; EMBASE; HaPI; Psyc (Info, Articles, Books)]. Once the instruments were identified then the name of each instrument was searched as a keyword in the databases listed above. Review articles on the ICF as well as on participation and handicap instruments were included and reference lists of all articles selected were reviewed. The systematic search was updated in March 2008.  2.2.2 Selection Criteria Articles were selected if the instrument was based on the ICIDH-2 or ICF conceptual model. An instrument was considered to assess the ICF concept of participation if it included a minimum of 3 domains from 1) the ICIDH-2 participation dimension12 or 2) Chapters 3 to 9 in the activities and participation component in the ICF, which is one of the suggested options for operationalizing participation5. In addition, an instrument assessing participation had to be designed for use in the community but did not need to use the ICIDH-2 or ICF 22  terminology. Instruments based on this definition of participation were then included if they met the following inclusion criteria: either self-administered or interview-administered, generic in content, developed for adults and published in the English language. Since the ICIDH-2 was first released in 1997, the search included articles published between 1997 and March 2008.  2.2.3 Data Extraction Data was extracted based on the criteria outlined by the Medical Outcomes Trust (MOT)13 and Fitzpatrick et al.14. One person extracted all of the data (VKN) and a description of the data fields is provided below.  2.2.4 Overview of the Instruments For each instrument the following information was recorded: the number of questions (require a response from the respondent, including screening questions); subdomains (include single or multiple questions which are part of a domain); domains (assess an underlying dimension); amount of time required for the respondent to complete it; the different formats for administering it; and the original language it was developed in as well as the number of languages it has been translated into. In addition, the wording of questions, the response options for the scale(s) (including the number of points and the wording) and the scoring was recorded.  23  2.2.5 Characteristics of the subjects Data was extracted on the health conditions of the subjects used to develop and test the instrument as well as the countries where the testing was conducted to assist in interpreting the meaning of the scores.  2.2.6 Reliability Reliability is the degree to which an instrument is free from random error.13 Two types of reliability were extracted. Internal consistency uses between-item correlations to assess the homogeneity of a multi-item scale. Cronbach’s coefficient alpha is most commonly used as a measure of internal consistency and accepted minimal standards are 0.70 for group comparisons and 0.90-0.95 for individual comparisons.13 Modern measurement methods such as Rasch analysis report a person separation reliability which is similar to internal consistency and values greater than 0.70 are considered adequate.15 Reproducibility of the instrument assesses the amount of random error that occurs over time in repeated assessments between the same interviewers (intra-rater reliability), different interviewers (inter-rater reliability) or the same subjects (test-retest reliability), which are assumed to be stable.14 The intraclass correlation coefficient (ICC) is calculated using analysis of variance and describes how much of the total variability in scores is due to differences between individuals and how much is due to measurement error.16 A reliability coefficient of 0.90 is recommended if measurements are used for individual respondents and for group comparisons 0.70 is acceptable.13;14 Instruments using binary or ordered categorical scales can be assessed using kappa (κ) or weighted kappa (κw), respectively. Kappa values <0.20 are considered poor, 0.21-0.40 slight, 0.41-0.60 moderate, 0.61-0.80 good and 0.81-1.00 very high.17 Since (κw) is affected by the value of the weights the 24  guidelines above cannot be applied.17 The consistency of responses among repeated administrations of an instrument is assessed using the standard error of measurement (SEM), which is the square root of the error variance.16 Information using the SEM enables the user to determine if an instrument is suitable for monitoring changes over time. The SEM can be used to calculate the minimal detectable change (MDC) (also called the smallest real difference) using the formula MDC=1.96 X √2 X SEM.18;19 The MDC represents the smallest within-person change in score that can be detected in an individual beyond measurement error, with p<0.05.18  2.2.7 Validity Validity assesses whether the instrument measures what it intends to measure.13 It is not a property of an instrument but rather is the meaning or interpretation that can be derived from the instrument scores for a specific purpose.20;21 For this paper, face, content and construct validity were considered to be the most relevant for patient reported instruments14 and were assessed. Face validity examines whether the instrument appears to measure what it intends to measure, and content validity assesses how well the questions cover the health components being measured.14 Construct validity assesses the theoretical relationship of the questions to each other and to hypothesized scales.17 It includes evidence assessing the dimensionality of the scales using factor analysis and modern measurement methods such as confirmatory factor analysis and Rasch analysis. Item-to-scale correlations can also be used to assess homogeneity of the scales and the minimum correlation expected between an item and the scale, where the item is removed is 0.20.14;22 In addition, it consists of evidence examining the relationship between the participation subscale scores and/or total scores with other variables (also referred to as known-group validity) including sociodemographic (e.g. age, 25  marital status); socioeconomic (e.g. education, employment status); clinical (e.g. diagnostic groups, duration of symptoms) and patient-reported variables (e.g. pain ratings, scales from patient-reported instruments). Lastly, correlations with other clinical or patient-reported instruments are used to determine if scores from the participation instruments are associated with instruments measuring similar constructs (convergent validity) or different constructs (discriminant validity). All information available related to validity of the instruments was abstracted and it was noted whether a priori hypotheses were stated regarding the expected relationships being tested.13  2.2.8 Responsiveness Responsiveness is often referred to as sensitivity to change and it refers to an instrument’s ability to detect change over time.13 Various statistical measures are used and commonly reported ones include correlation with other change scores, effect size (change score for an instrument is divided by the standard deviation of the baseline measure of the instrument) and standardized response mean (change score for an instrument is divided by the standard deviation of the change score).13;14 Effect sizes of 0.2, 0.5 and 0.8 are considered small, medium and large respectively.23 Other authors state that responsiveness must include individuals’ assessment of whether or not a meaningful change has occurred using health transition questions or global assessments of change.13 When evaluating whether an instrument is responsive it is necessary to consider the type of intervention, time between assessments and the health condition being treated since the responsiveness of an instrument is influenced by the effectiveness of the intervention.13 It is preferable to assess responsiveness using longitudinal data comparing a group that is expected to change with a group that is expected to remain stable.13 Responsiveness is also affected by the instrument’s 26  scale and extreme scores (very low or high levels of participation) may make it impossible to report changes in these health states.14  2.3 Results 2.3.1 Identification of Instruments A review of the literature in September 2007 identified 3087 articles. The titles of these articles were reviewed, 78 appeared to meet the inclusion and exclusion criteria and the abstracts were then reviewed. Fifty two out of the 78 abstracts appeared to meet the inclusion and exclusion criteria and the full article was reviewed. Ten instruments were included: Impact on Participation and Autonomy (IPA)24-26, Keele Assessment of Participation (KAP)27, PAR-PRO28, Participation Measure-Post Acute Care (PM-PAC)29, Participation Objective Participation Subjective (POPS)30, Participation Scale (P-Scale)31, Participation Survey/Mobility (PARTS/M)32, Perceived Impact of Problem Profile (PIPP)15, Rating of Perceived Participation (ROPP)33, and World Health Organization Disability Assessment Schedule II (WHODAS II)34. The Participation Measure-Post Acute Care Computerized Adaptive Test version (PM-PAC-CAT)35 was added when the systematic search was updated in March 2008.  2.3.2 Description of Instruments The 11 instruments are described in Table 2.1. Seven instruments (IPA, KAP, PARTS/M, PIPP, P-Scale, ROPP, WHODAS II) include questions with content from Chapters 4 (Mobility) through 9 (Community, social and civic life) in the activities and participation list in the ICF classification. Four instruments (PAR-PRO, POPS, PM-PAC, PM-PAC-CAT) exclude Chapter 5 (Self-care) and two instruments (PAR-PRO, POPS) focus 27  only on transportation issues covered in Chapter 4. One instrument (PM-PAC-CAT) includes questions with content from only three chapters [Chapter 4, Chapter 6 (Domestic life), Chapter 9]. Four instruments include questions with content from additional ICF chapters in activities and participation, specifically, P-Scale and WHODAS II [Chapter 1 (Learning and applying knowledge)]; WHODAS II [Chapter 2 (General tasks and demands)]; and PM-PAC, P-Scale, ROPP and WHODAS II [Chapter 3 (Communication)]. The PIPP asks about mental functions which is part of the ICF component body functions. Table 2.2 provides examples of questions, the metric of the scales as well as the scores produced for each of the instruments. Only the WHODAS II has population norms available.34  2.3.3 Characteristics of the Subjects The participation instruments were developed and tested in a wide range of health conditions (Table 2.3). Seven instruments have been developed and tested in only one country (KAP, PAR-PRO, PM-PAC, PM-PAC-CAT, POPS, PARTS/M, ROPP) and four have been tested in multiple countries (IPA, PIPP, P-Scale, WHODAS II).  2.3.4 Reliability Data on the internal consistency and reproducibility of the instruments is presented in Table 2.4. Most of the instruments either met or exceeded values 0.70 for internal consistency. In some of the instruments, there were domains below 0.70, such as in selfcare36;37 and getting along with people38;39 in the WHODAS II (36-question). No information was available on the internal consistency of the POPS or KAP.  28  Evidence on the reproducibility of the participation instruments was primarily assessed using ICCs. There is evidence for the IPA, ROPP, PARTS/M, P-Scale and WHODAS II being used for group-level comparisons. The PM-PAC (with the exception of the role functioning and economic life domain) also met the criterion of having an ICC greater than 0.70. The objective and subjective summary scores for the POPS met the group-level criterion however, the intra-rater reliability for some of the domain scores were low with 9 out of the 10 domain scores being <0.70.30 The test-retest reliability for the KAP response options were also low ranging from 0.34 – 0.64 κw.27 No evidence was available on the reproducibility of the PAR-PRO, PM-PAC-CAT or the PIPP. The ROPP33 and the WHODAS II38 (36-question) were the only instruments with data on the SEM and MDC.  2.3.5 Validity The sources of input used in developing the content and testing content and face validity are listed in Table 2.5. A variety of methods were used ranging from focus groups and qualitative interviews24;27;31;32 to expert panels28;31. Dimensionality of the instruments was reported for 8 of the 11 instruments (Table 2.5). Exploratory factor analysis was used in testing the IPA24;25;40, PAR-PRO28, P-Scale31, PARTS/M32, PM-PAC29 and WHODAS II34. Dimensionality of the IPA40, PM-PAC29, PM-PAC-CAT35 and WHODAS II34;39 was further evaluated using confirmatory factor analysis. Rasch analysis was used to assess dimensionality in the IPA41-43, PAR-PRO28 and PIPP15;44 and non-parametric or parametric item response theory (IRT) was used to assess the PM-PAC29, PM-PAC-CAT35 and WHODAS II (12-question)45. There was evidence to support the item-to-scale correlations for the IPA40, P-Scale31 and PM-PAC29. All instruments met the minimum value of 0.20.14 No information on dimensionality was located for the KAP, POPS or ROPP. 29  All of the instruments have evidence supporting construct validity, except for the ROPP which has only recently been published (Table 2.5). Hypotheses were supported regarding the negative effect of coma duration46 on the IPA perceived participation score as well as sociodemographic factors such as age having no effect on the perceived participation score in persons with Parkinson’s Disease41. A priori hypotheses testing the convergent validity of the IPA with instruments measuring similar constructs in instruments such as London Handicap Scale and the Short Form-36 (SF-36) have been confirmed.25;40;41;46;47 However, there have been mixed results in terms of the discriminant ability of the IPA25;40;41 since associations between dissimilar constructs had higher correlations than expected. Studies using the KAP have demonstrated that sociodemographic variables such as age48 and gender48 as well as socioeconomic variables such as education49 impacted the scores in older adults. Convergent validity of the KAP was supported by comparing similar domains in the IPA and Reintegration to Normal Living (RNL).27 Discriminant validity for the KAP using these same instruments was not as strong as expected.27 Construct validity testing for the PAR-PRO demonstrated it can differentiate among diagnostic groups.28 Sociodemographic factors such as age negatively impacted the PARPRO score but gender had no effect in various health conditions.28 The ability of the PM-PAC to differentiate groups based on clinical variables such as diagnosis and severity has mostly been supported.29;50 Sociodemographic factors such as age, gender and race had no effect on PM-PAC summary scores as hypothesized50 The association between the PM-PAC has been compared to the Medical Outcomes Study Social Support Survey and demonstrated a weak to moderate correlation, particularly with the social and home participation summary score (r=0.408 at 1 month and r=0.344 at 6 months post  30  discharge from rehabilitation).51 The PM-PAC-CAT was able to distinguish between diagnostic groups.35 Scores generated by the CAT were compared to a fixed length version (PM-PAC-53) containing questions from the item banks and there was no difference in scores.35 In terms of the POPS, hypotheses stating that the severity of traumatic brain injury would affect the participation subjective scores have been partly supported.30 Convergent validity has been demonstrated by the participation subjective scores having a stronger correlation with instruments tapping subjective assessments such as Flanagan Quality of Life Scale compared to the participation objective scores30. However, very few of the POPS objective questions or scores correlated with the Global Fatigue Index as hypothesized52. Expected low correlations between the objective and subjective total scores of the POPS have been confirmed (0.21-0.23).30 There is some evidence to support the construct validity of the P-Scale. The P-Scale scores correlated with experts’ rating of participation restriction (Spearman rank correlation=0.44) as well as with individuals’ self-assessment of participation.31 The P-Scale was able to differentiate between individuals with and without a health condition and positive correlations with the Eyes Hands and Feet impairment instrument for leprosy subjects was supported as hypothesized.31 Construct validity of the PARTS/M has been partly supported in studies reporting no effect of age or gender.53 Education had a positive effect on the perceived choice and satisfaction of the PARTS/M scales as hypothesized, however, marital status did not have any effect which was unexpected.53 Convergent validity of the PARTS/M has been supported using the RNL and the Personal Independence Profile-2 and -3.32  31  The PIPP has undergone some construct validity testing. Questions in the PIPP did not have any substantial differential item functioning for variables such as age, gender, education level.15;44 The PIPP has demonstrated convergent validity compared to the (EuroQoL) EQ5D as hypothesized.15;44 Construct validity of the WHODAS II has been extensively tested. The WHODAS II (36-question) was able to differentiate among diagnostic groups36;37;39;54-59 and disease severity60. The effects of sociodemographic variables such as age45;54;55;58;59;61-64 and socioeconomic variables such as education45;54;62-64 have demonstrated mixed effects in various populations; however, patient reported variables such as depression59;61;64;65 consistently had a negative impact on WHODAS II scores. Finally, the WHODAS II has been compared most frequently to the SF-3636;39;60;66/SF-36 Veterans version38 and domains measuring similar constructs demonstrated strong correlations as expected; for example, the WHODAS II domain getting around was highly correlated with the SF-36 physical function domain (range: r=-0.65 to -0.79)36;38;39;60;66.  2.3.6 Responsiveness Evidence on the responsiveness of the participation instruments was available for the IPA, PM-PAC-CAT, P-Scale and WHODAS II (see Table 2.6). The IPA participation domains, family role, autonomy outdoors, and work and education, were most responsive following three months of rehabilitation.26 The family role, leisure and work problem questions were also responsive in a variety of health conditions.26 There is also some preliminary evidence supporting the responsiveness of the P-Scale after 9 to 12 months following the initial assessment in subjects with health conditions such as leprosy and spinal cord injury.31 The effect sizes for the three domains in the PM-PAC-CAT were 32  assessed after 3 months following discharge from rehabilitation and ranged from 0.50 to 0.58, which is a moderate effect size.35 In terms of the WHODAS II, effect sizes were similar to other generic measures such as the SF-3636;39;60;66 (WHODAS II, 36-question) and SF-1267 (WHODAS II, 12-question). Change scores for the WHODAS II were also highly correlated to disease-specific measures such as the Bath Ankylosing Spondylitis Functional Index66 (WHODAS II, 36-question) in subjects with ankylosing spondylitis and symptom measures such as the Social Phobia Scale67 (WHODAS II, 12-question) in subjects with anxiety disorders. Responsiveness is also affected by the score distribution. Information on the difficulty of the questions was available for most of the instruments. The IPA domains, social life and relationships40;41;43;46 and autonomy indoors40;43;46 are frequently considered easy and the domain considered most difficult varies. High levels of participation were reported for individuals with incomplete spinal cord injury43 as well as those visiting general physicians40, indicating that the IPA is most suitable for individuals with moderate disability. During the pilot testing of the KAP in individuals living in the community, 53% of the sample (n=575) reported no participation restrictions.27 However, the KAP was developed for population studies and the authors stated that it may not be detailed enough for clinical practice.27 In the PAR-PRO the easiest questions, socializing in and outside the home, produced a floor effect in individuals with moderate to severe disability.28 For the PM-PAC, the results were analysed using IRT.29 The threshold values ranged from -1.94 to 1.05 and 76% of the questions had a negative threshold, suggesting that the instrument is designed for individuals with significant participation restrictions.29 High levels of participation were also noted in the P-Scale, with 40% of the sample having no participation restrictions.31 In terms of the  33  PIPP, low scores were reported for the impact on relationships subscale, indicating very few subjects reported that their health problems impacted their relationships.15 In the ROPP, individuals with Parkinson’s disease and multiple sclerosis frequently had a score of zero, which means they had very good participation or the question was not applicable, in domains such as personal care and social relationships.33 For the WHODAS II (36-question) high levels of participation were reported in the self-care domains for a wide range of health conditions, with 33.0-70.3% of individuals reporting no problems39, as well as in the mobility domain for conditions such as depression.36 Only a few studies considered measurement error when assessing responsiveness. One study assessed the reproducibility of baseline measurements (test-retest reliability) and included a control group.68 Another study used the reliable change index to determine the ability of the instrument to measure a real change in symptoms.67  2.4 Discussion There has been considerable progress in the conceptualization and measurement of participation since Cardol et al’s.11 review of handicap instruments in 1999. The most notable development was replacing handicap with participation in the ICIDH-2. This change in how participation is conceptualized has instigated the development of these 11 instruments, of which 7 were published between 2006 and 2008.  2.4.1 Operationalizing the Concept of Participation The domains in the 11 participation instruments vary, especially regarding whether selfcare is included. In the POPS self-care is not considered to be part of participation since it is not related to fulfilling life roles.30 Self-care is also not included in the PAR-PRO, PM-PAC 34  or PM-PAC-CAT and in the WHODAS II self-care is included but is considered to be part of activity as opposed to participation34. The variability in how activity is distinguished from participation is not surprising considering that in the ICF classification activity and participation domains (chapter headings) are listed together and the user decides how to structure their relationship.5 Future work should include comparing the content of the instruments by linking them to the ICF classification using standardized rules69 to help determine how the concept participation has been operationalized. This may further refine how participation is conceptualized in future revisions of the ICF. This conceptual clarity is important because in order to understand the relationship between concepts in the ICF model, participation instruments should be pure in content and not contain questions assessing other concepts such as activity.70 There is a great deal of variation in how participation is operationalized which reflects the purpose of the instrument. After reviewing the questions included in the instruments, 9 (IPA, KAP, PARTS/M, PIPP, PM-PAC, POPS, P-Scale, ROPP, WHODAS II) assess subjective participation asking about autonomy, level of participation compared to a peer, problem/impact or distress caused by the participation restriction, satisfaction or amount of difficulty. Three instruments (IPA, KAP, ROPP) specifically assess autonomy, defined as the ability to do something the way and when one wants to. In the P-Scale a peer comparison is used to assess subjective participation since in developing countries the concept of autonomy is not part of the culture.31 Dijkers71 however, questioned whether by defining a peer using demographic, economic or socio-cultural characteristics, it may understate the impact on participation for a particular health state. In addition, many instruments assess participation at multiple levels, asking about the perceived participation restriction as well as  35  the importance, impact or satisfaction since not all restrictions are deemed equivalent and this will enable rehabilitation professionals to focus on areas relevant to the person. From a measurement perspective, clarity should be obtained regarding the relationship between the concepts of participation and quality of life since instruments such as the Life Satisfaction (9 or 11 question version)72;73 used to assess quality of life also ask about satisfaction with selfcare, family life and relationships. Objective participation is measured by frequency [PAR-PRO, PM-PAC (not included in domains scored), POPS, PARTS/M] and environmental supports used (PARTS/M). The PAR-PRO is the only instrument to just assess objective participation, comparing frequency of participation prior to the health condition, the current level and the ideal frequency for each of the participation tasks. Although objective participation has been criticized since it does not focus on the needs of the individuals11, the information obtained from objective instruments can be used to evaluate rehabilitation interventions such as the provisions of equipment (e.g. adapting a car) to determine if it increases the frequency of participation (e.g. driving to work). Both types of information may be useful to clinicians74 and some instruments recognize this, capturing both (PARTS/M, PM-PAC, POPS).  2.4.2 Reliability, Validity and Responsiveness The evidence related to reliability, validity and responsiveness for the participation instruments is impressive considering most of the instruments have only been recently published. There is sufficient evidence on the internal consistency for the domains and overall scores for most of the instruments supporting their use in group-level comparisons in the health conditions assessed. Four instruments (IPA, P-Scale, ROPP, WHODAS II) have evidence to support their summary scores being used for individual-level comparisons. This 36  is probably due to the fact that these are the instruments with the largest number of questions.14 Measuring internal consistency in the KAP and POPS may not be applicable. The KAP reports participation restrictions using individual questions, categories based on the number of restrictions or an overall score with the total number of restrictions.27;48;49;75 In the POPS different types of information are included in the domains (frequency for objective domains and combined satisfaction and importance ratings for subjective domains) and high correlations among the questions are not necessarily expected.76 In terms of the reproducibility of the participation scores, most studies used ICCs and met the requirement of 0.70 for group-level comparisons. Two instruments (IPA, ROPP) have evidence to support their use for individual-level comparisons. The test-retest reliability for the KAP and the POPS were low and may reflect true changes in participation27;30 and more testing is needed. Two instruments (ROPP, WHODAS II) have data on the SEM and MDC. Reporting SEM and MDC should be included in future studies since the MDC indicates how much change is needed to detect differences beyond measurement error for an individual, which is useful information for clinicians.19 Content validity of the instruments was assured by involving individuals with health conditions in the development process for eight of the instruments. No detailed information was available for the POPS or WHODAS II. The PAR-PRO did not specify that individuals with health conditions were involved in assessing the instrument content. As indicated by Cardol et al.11, it is imperative that developers of new instruments involve individuals with health conditions to ensure that all important aspects of participation are addressed. Dimensionality was assessed in eight of the instruments. Problems with exploratory factor analysis have been identified and include instability of factors after the first one or two  37  factors have been extracted.14;17 The use of modern measurement methods such as confirmatory factor analysis, Rasch analysis and IRT may provide additional support for the underlying dimensions measured. Some of these modern measurement methods assume domains/scales measure a single underlying dimension and this assumption will not necessarily be appropriate in all cases.77 For example, in the POPS different aspects of participation are included in the domains (frequency for objective domains, combined importance and satisfaction ratings for subjective domains) which are not necessarily related and so testing unidimensionality is likely not applicable.30 In addition, often questions need to be removed in order to fit the model, which may impact the content validity.41-43 Construct validity was also assessed by comparing the scores obtained from the participation instruments to other instruments as well as sociodemographic, socioeconomic, clinical or patient-reported variables. The WHODAS II has been the most extensively tested (Table 2.5). Some of the authors stated a priori hypotheses regarding the expected relationship with the other variables; however, very few quantified the magnitude of the expected relationship. In addition, only the KAP has been compared to the IPA and more head-to-head comparisons of participation instruments included in this review are needed. Scores from participation instruments such as the IPA, PARTS/M, PM-PAC, and WHODAS II (both the 12 and 36 question versions) have been compared to the SF-12/SF-36, which is one of the most widely used generic health status instruments. More research is needed to identify how participation instruments and generic health status instruments differ. Linking the instruments to the ICF classification using standardized linking rules was done for the WHODAS II and SF-3678 which enables the content of the instruments to be compared. Future research should do this for all the instruments.  38  There is some evidence to support the ability of participation instruments to assess change. Evidence on responsiveness is available for the IPA, PM-PAC, P-Scale and WHODAS II. The IPA was not as responsive as expected following three months of rehabilitation but the authors suggested that this may be due to a lack of true change.26 The change scores in the WHODAS II (12- and 36-question) following health interventions were similar to change scores reported by the SF-36.36;39;66;67 There was variability in how responsiveness was reported. Effect sizes and/or standardized response means were reported in all studies which enable results to be compared. In addition, the results are not influenced by sample size, which is preferable. However, these types of measures are assessing the ability to detect a treatment effect and in order for these results to be meaningful, and to understand if the instrument is fulfilling its purpose, the expected effect sizes should be stated a priori.18 This could not be found in any of the studies reviewed. Scores indicating high levels of participation were common and occurred when participation was measured in relatively healthy populations27;31;36, which may limit the usefulness of these instruments in this group if the purpose is to detect change. As noted by Hyland79 and others39, responsiveness is affected by the interaction between the treatment, the instrument’s scale and population and all need to be considered. Future studies must also consider individuals’ assessments of whether or not a meaningful change has occurred to assist in interpreting the results (anchor-based) since measures such as effect size are based on statistical methods (distribution-based).18 Based on this review it is evident that the field is rapidly progressing and none of the instruments could be considered a gold standard at this time. Clinicians and researchers should determine the type of information required about participation before selecting an  39  instrument since the instruments vary in content and how participation is measured. To date, there is more evidence to support these instruments in research studies compared to clinical practice since it is easier to detect a significant difference when assessing groups due to a smaller variance of the sample mean.80 For these instruments to be useful clinically the SEM should be reported and the MDC calculated to determine if real change can be detected in individuals. Instruments with scores indicating high levels of participation may not be able to detect change and some participation instruments may be better suited for health conditions which have moderate to severe participation restrictions. The use of ‘normative data’ from individuals with similar health conditions may provide important information clinically but it is also important to capture the desires and needs of the individual. In summary, this article reviewed 11 instruments developed to assess participation based on the ICF, with 7 of the 11 instruments being published in the past two years. Conceptually, participation needs to be distinguished from the ICF concept of activity and its relationship with quality of life should also be determined.81 The WHODAS II has the greatest body of research supporting its use; however instruments such as the IPA are increasingly being administered. Future research should empirically assess these participation instruments in various health conditions to determine if they provide similar findings. Furthermore, including multiple participation instruments in a single study will enable the instruments to be directly compared. More work is also needed to establish the MDC and the minimal important change for these instruments so they can be used to evaluate clinical interventions. The use of modern measurement methods such as IRT need to be further examined. Computer adaptive tests such as the PM-PAC-CAT, enable participation to be measured precisely with reduced respondent burden and advances in measurement methods will offer  40  new possibilities for measuring participation in the future. Participation is considered a key outcome in rehabilitation82 and future work in this area will ensure the information obtained from these instruments is meaningful and can enhance the lives of individuals living with various health conditions.  41  domestic management activities; socialization factor; physical vigour; generative activities (e.g. work, education)§  mobility; self-care; domestic life; interpersonal interactions & relationships; major life areas; community, social & civic life self-care; mobility; participation; relationships; psychological wellbeing  20  159/20  46/23  PARTS/M  PIPP  KAP  PAR-PRO  15-45 min  autonomy indoors; family role; autonomy outdoors; social life & relationships; work & education  NR  25-40 web; 60-90 paper  NR  2-4 min  Respondent burden  Domains  mobility; self-care; domestic life, interpersonal interactions & relationships; major life areas; community, social & civic life  #Questions*/ subdomains (if applicable) 39‡  15/11  IPA  Instrument  Table 2.1. Overview of participation instruments  English-UK  Dutch  Original language  self-admin; intervieweradmin  self-admin  EnglishAustralia  English-USA  interviewer- English-USA admin; proxy intervieweradmin  self-admin  self-admin  Alternate forms†  2  NR  NR  NR  3  42  Number of translations  domestic life; major life areas; transportation; interpersonal interactions & relationships; community, recreational & civic life mobility; self-care; interpersonal interactions & relationships; major life areas; community, social & civic life; communication; learning & applying knowledge personal care; mobility; communication; social relationships; domestic life & caring for others; education; work & employment; economic life; social & civic life  78/26  36/18  69/22  P-Scale  ROPP  PM-PACCAT  POPS  NR  communication; mobility; domestic life; interpersonal relationships; role functioning; work & employment; education; economic life; community, social & civic life  15-30 min  20 min  NR  1.5 - 18 min  Respondent burden  Domains  mobility; domestic life; community, social & civic life  #Questions*/ subdomains (if applicable) 51  79  PM-PAC  Instrument  English-USA  English-USA  Original language  self-admin  intervieweradmin  Swedish  English  interviewer- English-USA admin  self-admin  self-admin  Alternate forms†  1  6  NR  NR  NR  43  Number of translations  #Questions*/ subdomains (if applicable) 36; 12 understanding & communicating; getting around; self-care; getting along with people; household & work activities (life activities); participation in society  Domains 20 min 5 min  Respondent burden self-admin; intervieweradmin; proxy selfadmin; proxy intervieweradmin  Alternate forms† English  Original language  minimum of 16  Number of translations  44  Notes: * questions require a response from the person and include screening questions; sub-domains can include single or multiple questions which are part of a domain † sometimes the self-admin version was read by an interviewer but it did not require special instructions or training ‡ in the English version of the IPA 2 additional questions were added, for a total of 41 questions § these are 4 subfactors identified beyond the unidimensional construct participation and are not official domains  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NR, not reported; PARTS/M, Participation Survey/Mobility; PIPP, Perceived Impact of Problem Profile; PM-PAC, Participation Measure-Post Acute Care; PM-PAC-CAT, Participation Measure-Post Acute Care-Computerized Adaptive Test; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II  WHODAS II  Instrument  5 (none – daily)*  categorical time intervals  Rate patient’s typical degree of participation in these life situations in the year prior to this hospitalisation or episode of illness. E.g. light housework.  How much time do you require for dressing on a typical day?  PAR-PRO  PARTS/M  illness, physical impairment, pain, fatigue, not limited When dressing how much choice do you have compared 4 (a lot – no choice) to others without mobility limitations?  5 (all the time – none of the time)  During the past 4 weeks, I have moved around in my home as and when I have wanted.  KAP  Is your participation in dressing limited by…  3 (no problems – major problems)  Response options # of points (anchors) My chances of fulfilling my role at home as I would like 5 (very good – very poor) are…  Examples of questions  If your health or your disability affect your activities in and around your home, to what extent does this cause you problems?  IPA  Instrument  Table 2.2. Description of questions, response options and scores  45  4 component scores (temporal, health-related, evaluative, supportive) for each of the 6 domains; 6 domain scores; 1 overall score  overall participation rating  report number of questions with a participation restriction (response options: some/little/none) and can categorize the number of participation restrictions; can also report the value for individual questions  5 perceived participation domain scores; can also report the perceived problem questions  Score  6 (no impact – extreme impact)* 6 (no distress – extreme distress)* 5 (not at all limited – extremely limited) 5 (very satisfied – very dissatisfied) NR 4 (all – none)  How much impact has your current health problems had on your ability to relate to relatives?  How much distress has been caused by the impact of your health problem on your ability to relate to relatives?  How much are you currently limited in getting around your home?  How satisfied are you with how much you can help family and friends?  NR  In a typical week, do you do all, most, some or none of the cleaning in the house?  PM-PACCAT  POPS  PM-PAC  PIPP  5 (all of the time – never)  How often do you use accommodations, adaptations or special equipment to dress?  Response options # of points (anchors) 4 (very satisfied – dissatisfied) 4 (a great deal – none)  How satisfied are you with your participation in dressing?  Examples of questions  How much help from another person do you require for dressing?  PARTS/M cont.  Instrument  46  objective and subjective subscale scores for each of the 5 domains;  3 domain scores  7 domain scores; 2 overall scores (social and home community)  impact subscale and distress subscale scores for each of the 5 domains  Score  WHODAS II  ROPP  P-Scale  POPS cont.  Instrument  Response options # of points (anchors) 3 (more, less, same)  5 (not restricted – severely participation restriction score restricted) 2 (yes, no) 2 (yes, no)  Full participation is when uses one’s money in the way one wants. My participation is…  I am satisfied with my level of participation…  I want support to change my level of participation..  47  6 domain scores; 1 overall score  4 (no problem – large)  (if answer to above question is sometimes or no) How big a problem is it to you?  In the past 30 days, how much difficulty did you have in 5 (none – extreme/cannot making new friends? do)  4 (yes, sometimes, no, irrelevant)  participation restriction score  2 summary scores (participation objective, participation subjective)  Score  Are you as socially active as your peers are? (e.g. in religious/community affairs)  How important is cleaning the house to your satisfaction 5 (most – not at all) with life?  Would you say the amount you engage in cleaning the house is satisfactory to you?  Examples of questions  Notes: * in the analysis the scale was modified  48  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NR, not reported; PARTS/M, Participation Survey/Mobility; PIPP, Perceived Impact of Problem Profile; PM-PAC, Participation Measure-Post Acute Care; PM-PAC-CAT, Participation Measure-Post Acute Care-Computerized Adaptive Test; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II  Table 2.3. Health conditions included in development, testing and use of participation instruments Instrument Health condition (number of individuals) Country Netherlands24-26;42;46;47;83IPA chronic idiopathic axonal polyneuropathy 87 84 85 86 ; Italy41 ; Sweden43;88;89; (n=56) ; haemophilia (n=43) , (n=127) ; UK40;42;90 healthy adults (n=60)40;42; hypoxic brain injury due to cardiac arrest (n=16)46; multiple sclerosis (n=35)90, (n=377)87, (n=60)40;42; Parkinson’s disease (n=100)41; rheumatoid arthritis (n=51)40;42; spinal cord injury *(n=161)43;89, *(n=157)88, (n=42)40;42; neurological disorders, rheumatic disorders, coronary and pulmonary disorders, amputation (n=63)47; neuromuscular disease, fibromyalgia, rheumatoid arthritis, spinal cord injury, stroke (n=126)25;83; neuromuscular disorders, stroke, hand injuries, rheumatoid disorders (n=49)26;42; various neuromuscular diseases, traumatic hand injury, other disabilities24 KAP  adults over 50 years (n=1117)27, *(n=7878)48;49, adults over 50 years with knee pain *(n=2252)75  UK27;48;49;75  PAR-PRO  orthopaedic, stroke, neurologic, brain injury, cardiac and pain, arthritis (n=594)28  USA28  PARTS/M  cerebral palsy, post polio, multiple sclerosis spinal USA32;53 cord injury, stroke, (n=604)32; spinal cord injury (n=255)53  PIPP  mobility impairments due to: amputation (injury, cancer, vascular disease), central nervous system disease (multiple sclerosis), stroke, degenerative conditions (arthritis), spinal cord injury, (n=169)15; mobility impairments (n=210)44, (n=210)91  Australia15; Malaysia91; Thailand44  PM-PAC  brain injury, cardiopulmonary, debility due to illness, fractures, joint replacement, joint or muscular pain, Parkinson’s disease, post surgical, spinal cord injury, stroke, (n=395)29; chronic obstructive pulmonary disease, Guillan Barré syndrome, lower extremity fractures, multiple sclerosis, Parkinson’s disease, post myocardial infarction, post surgical, stroke, traumatic brain injury, *(n=435)50, *(n=342)51  USA29;50;51  49  Instrument Health condition (number of individuals) PM-PAC- brain injury, cardiopulmonary conditions, debility due to illness, fractures, joint replacements, CAT multiple sclerosis, neuropathy, orthopaedic surgery, Parkinson’s disease, post-surgical recovery, spinal cord injury, stroke (n=94)35  35  Country  USA  POPS  adults (n=121)30, (n=85)52; traumatic brain injury (n=454)30, (n=223)52  USA30;52  P-Scale  leprosy (n=254)92;93, (n=264)94; leprosy, poliomyelitis, spinal cord injury, other disabilities (n=724)31  Brazil31;94; India31;94; Nepal31;92;93  ROPP  multiple sclerosis (n=29), other disabilities (n= 23), Parkinson’s disease (n=27), spastic paresis (n=6)33  Sweden33  WHODAS II (36question)  adults (n=198)37, †(n=2125)45, (n=30)58, (n=4149)56; adults over 65 years (n=1204)61, (n=840)63; adult onset hearing loss (n=380)38;68; ankylosing spondylitis (n=214)66; asthma, chronic obstructive pulmonary disease, coronary heart disease, diabetes mellitus, obesity (n=308)39;97; back pain (n=76)36; back pain, osteoarthritis, rheumatoid arthritis (n=296)39;97; blindness (n=74)55; breast cancer (n=119)39;97, (n=284)57; cerebral palsy (n=89)55; deafness55; depression (n=73)36, (n=65)39;97, (n=405)100; diabetes (n=4357)65, (n=233)55; epilepsy (n=82)55; multiple sclerosis (n=136)55; psychotic disorders (n=20)96; schizophrenia (n=54)54, (n=60)58;101; spinal cord injury ‡(n=311)95; stroke (n=116)39;97, (n=32)99, (n=64)55; systemic sclerosis *(n=337)64, *(n=402)60; trauma *(n=97)98, *(n=101)62  Afganistan95; Australia96 ; Austria45; Cambodia45 ; Canada45;56;60;64; China45; Cuba45; Germany39;97; Greece45; India45; Ireland55; Italy45; Japan45; Korea61; Lebanon45; Luxenbourg45; Netherlands45;66; Nigeria45; Norway62;98; Peru45; Poland57; Puerto Rico37; Romania45; Spain45; Sweden99; Tunisia45; Turkey45;58;63;101; UK45; USA36-38;45;54;65;68;100  WHODAS II (12question)  adults (n=124)59; anxiety disorders (n=169)67  Australia67; France59  50  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PARTS/M, Participation Survey/Mobility; PIPP, Perceived Impact of Problem Profile; PMPAC, Participation Measure-Post Acute Care; PM-PAC-CAT, Participation Measure-Post Acute Care-Computerized Adaptive Test; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II Notes: * same study sample for the stated health condition † used WHODAS II (12- and 36-question) ‡ version of WHODAS II not stated (12- or 36-question)  51  TR 5 response options for 11 subdomains/domains‡ (κw)=0.34-0.6427 TR participation restriction for 11 subdomains/domains‡ (κ)=0.20-0.7127 NR TR domains at 6-8 weeks (Pearson r)=0.77-0.9132 TR components at 6-8 weeks (Pearson r)=0.75-0.9332 NR  TR domains* at 1-15 days ICC=0.61-0.8629 NR  IC=0.7728  IC domains=0.71-0.9232; IC components=0.64-0.9732  PSR impact domains=0.69-0.7915; PSR distress domains=0.73-0.8315; PSR impact domains=0.79-0.8944  IC domains*=0.72-0.8929  NR  PAR-PRO  PARTS/M  PIPP  PM-PAC  PM-PAC-CAT  TR participation domains* at 2 weeks ICC=0.83-0.9125; TR participation domains at 2 weeks ICC=0.91-0.9740; TR participation questions* at 2 weeks (κw)=0.56-9025; TR problem questions* at 2 weeks (κw)=0.59-0.8725; TR participation questions at 2 weeks (κw)=0.64-0.9240  NR  IPA  KAP  Reproducibility  Internal consistency; Person separation reliability IC participation domains=0.81-0.9125; IC participation domains=0.88-0.9385;86; IC participation overall=0.9785; IC participation domains=0.86-0.9440; IC participation overall*=0.8687; PSR participation overall=0.9443; PSR problem overall=0.8243; PSR participation overall*=0.9341; PSR problem overall*=0.7141;  Instrument  Table 2.4. Evidence related to reliability  52  InterRR overall within 4 weeks ICC=0.8031; IntraRR overall at 4-12 weeks ICC=0.8331 TR domains for perceived participation, satisfaction & selection at 2 weeks are primarily (κ/κw) > 0.7033, TR perceived participation overall ICC=0.9733, MDC=7.933 TR domains† at 10 days ICC=0.46-0.8037; TR overall† at 10 days ICC=0.75-0.8337; TR domains*at 2 weeks ICC=0.81-0.9338; TR domains* at 10 weeks ICC=0.71-0.8938 TR overall at 12 weeks ICC=0.8954; InterRR at 5 days*=(κw)= (-0.09) – 0.8596; MDC at 2 weeks*=9.2 to 25.038; MDC at 10 weeks*=8.8 to 25.038  IC overall=0.9231  IC perceived participation total score=0.9033; IC perceived participation domains=0.50-0.8933  IC domains†=0.47-0.9337; IC overall†=0.92-0.9737; IC domains*=0.68-0.9138; IC overall*=0.9438; IC domains=0.65-0.9136; IC overall=0.9536; IC domains=0.69-0.9739 IC domains=0.69-0.9739  ROPP  WHODAS II (36-question)  53  Abbreviations: IC, internal consistency; ICC, intraclass correlation coefficient; IntraRR, intra-rater reliability; InterRR, inter-rater reliability; IPA, Impact on Participation and Autonomy; (κ), kappa chance corrected agreement; KAP, Keele Assessment of Participation; (κw), weighted kappa; MDC, minimal detectable change; NR, not reported; PARTS/M, Participation Survey/Mobility; PIPP, Perceived Impact of Problem Profile; PM-PAC, Participation Measure-Post Acute Care; PM-PAC-CAT, Participation Measure-Post Acute Care-Computerized Adaptive Test; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; PSR, person separation reliability; ROPP, Rating of Perceived Participation; TR, test-retest reliability; WHODAS II, World Health Organization Disability Assessment Schedule II  IntraRR objective domains at 1-3 weeks ICC=0.28-0.6930; IntraRR objective overall at 1-3 weeks ICC=0.7530; IntraRR subjective domains at 1-3 weeks ICC=0.42-0.6830; IntraRR subjective overall at 1-3 weeks ICC=0.8030  P-Scale  POPS  Reproducibility  Internal consistency; Person separation reliability NR  Instrument  Notes: * does not include the work/education domain/questions † based on 3 sites ‡ time for test-retest not stated  54  55  RNL32; PIP232; PIP332; subscales‡ of PARTS/M53  sociodemographic‡53; socioeconomic‡53; patientreported32  PCA used to determine scoring which supports using domains & overall scores32  clinicians, researchers, family members32  PARTS/M  NR  sociodemographic28; clinical28  in PCA 4 factors explain 44% of variance; good Rasch person & item fit statistics28  covered in instrument development28  PAR-PRO  IPA27; RNL27  sociodemographic48;49; socioeconomic49; clinical49; patient-reported49  NR  patients & healthy adults over 50 years27  KAP  LHS25;40; SF-3625;40; SIP-6825; FLP40; UPDRS-ADL&ME41; HY41; SE41; PDQ-3941; CFQ46; FAI46; QOLIBRI46; LiSAT988; IPA Part & IPA Prob41; HHD†84; SMS†84; SODA†84; SWT†84; FSS†84; BBS†84; Life-H SF47  sociodemographic41;42;83-85;89; socioeconomic83; clinical41;46;84;85;87;89; patient-reported83-85;87;89  in PCA 4 factors explain (exclude work & education) 67% of variance25; Rasch analysis supports participation & problem experience scales derived from 4 or 5 domains after removal of questions41-43 &/or changes to scale41;42; CFA of domains had a CFI=0.99, RMSEA=0.10, NFI=0.98, 5 factors explain 72% of variance40; ISC participation domains=0.340.8940  clinicians, researchers, patients, persons with disabilities24; patients90  IPA  Convergent/discrimin ant validity-other instruments used*  Construct validity- variables assessed  Construct validitydimensionality  Face/Contentsource of input  Instrument  Table 2.5. Evidence related to validity  PM-PAC-53 fixed length test35 BDI30; Life 330; FQOLS30; BISQ30; PO and PS subscale and total scores30; GFI52  clinical35  sociodemographic30; clinical30 ; patient-reported30  CFA of 3 domains had a CFI=0.945, RMSEA=0.07835 NR  used previous work on PMPAC29  used an instrument from a previous study30  PM-PACCAT  POPS  56  HACE51; MOS-SSS51  sociodemographic50; clinical29;50; patient-reported29;50;51  CFA of 7 domains had a CFI=0.940 & RMSEA=0.07629; PCA of 2 higher-order factors explain 62% of variance of 7 domains29; ISC 7 domains=0.38-0.7829  patients, persons with disabilities29  PM-PAC  EQ5D15;44; distress & impact subscales of PIPP15;91  sociodemographic15;44;91; socioeconomic15  good Rasch person & item fit statistics after removal of questions and changes to scale15;44; Rasch analysis indicated problems differentiating between impact & distress scales44  persons with disabilities, impairments, researchers15  PIPP  Convergent/discrimin ant validity-other instruments used*  Construct validity- variables assessed  Construct validitydimensionality  Face/Contentsource of input  Instrument  57  SF-3636;39;60;66; SF-36 Vet38; PHQ36; RM36; WLQ36; APHAB38; HHIE38; PGWB66; ASQoL66; DFI66; Pain rating66; BASDI66; BASFI66; DFI66; HAQS66; QOLI37; PANSS54;58; HAM-D54; DRS54; UPSA54; QWB54; FIM95; WMSR (digit span & memory)58; WCST58; Physician assessments60; SHAQ60; VSDAI60; CES-D60; MPQ60; MDSS60  sociodemographic36;45;54;55;58;59;61-65; socioeconomic45;54;58;61-65;98; clinical36;37;39;45;54-66;68;96;100; patient-reported39;45;54;56;59;62-66;101  all questions load onto 1 of 6 domains & 6 domains load onto general disablement factor (loadings>0.70); (36 questions)34; factor structure replicated using PCA except for household & work domain (life activities) which identified them as 2 separate factors39; CFA & non parametric item response theory support dimensionality (12 questions)45  WHODASII assessed in sites (12 and 36 worldwide34 questions)  NR  NR  NR  clinicians, patients, administrators33  ROPP  EHF31  sociodemographic94; socioeconomic94; clinical31;92-94; patient-reported31;94  in EFA first factor explains 90% of variance31; ISC overall=0.32-0.7331  expert input and multiple sources from extensive development work31  P-Scale  Convergent/discrimin ant validity-other instruments used*  Construct validity- variables assessed  Construct validitydimensionality  Face/Contentsource of input  Instrument  Notes: * all instruments correlated with the participation instruments’ scores are included † only used IPA Autonomy Indoors and IPA Autonomy Outdoors scales ‡ only used PARTS/M bladder care, leaving the home, working inside the home and employment subscales  58  Abbreviations: APHAB, Abbreviated Profile of Hearing Aid Benefit; ASQoL, Ankylosing Spondylitis Quality of Life Questionnaire; BASDAI, Bath Ankylosing Spondylitis Disability Activity Index; BASFI, Bath Ankylosing Spondylitis Functional Index; BBS, Berg Balance Scale; BDI, Beck Depression Inventory; BISQ, Brain Injury Screening Questionnaire; CES-D, Centre for Epidemiologic Studies Depression scale; CFA, confirmatory factor analysis; CFI, Comparative Fit Index; CFQ, Cognitive Failures Questionnaire; DFI, Dougados Functional Index; DRS, Mattis Dementia Rating Scale; EFA, exploratory factor analysis; EHF, Eye Hands Feet score; FAI, Frenchay Activities Index; FSS, Fatigue Severity Scale; FIM, Functional Independence Measure; FLP, Functional Limitations Profile; FQOLS, Flanagan Quality of Life Scale; GFI, Global Fatigue Index; HACE, Home and Community Environment; HAM-D, Hamilton Depression Rating Scale; HAQ-S, Health Assessment Questionnaire for Spondyloarthropathies; HHD, hand-held dynamometry; HHIE, Hearing Aid Handicap for the Elderly; HY, Hoehn and Yahr scale; IPA, Impact on Participation and Autonomy; IPA Part, IPA Perceived Participation Score; IPA Prob, IPA Perceived Problem Score; ISC, item-toscale correlation; KAP, Keele Assessment of Participation; Life-H SF, Life-Habits Short Form; LiSAT-9, Life Satisfaction-9; LHS, London Handicap Scale; MDSS, Medsger Disease Severity Scale; MOS-SSS, Medical Outcomes Study-Social Support Survey; MPQ, McGill Pain Questionnaire Short Form; NFI, Normed Fit Index; NR, not reported; PANSS, Positive and Negative Syndrome Scale; PARTS/M, Participation Survey/Mobility; PCA, principal components analysis; PDQ-39, Parkinson’s Disease Questionnaire-39; PGWB, Patient Global Well Being; PHQ, Patient Health Questionnaire; PIP, Personal Independence Profile; PIPP, Perceived Impact of Problem Profile; PM-PAC, Participation Measure-Post Acute Care; PM-PAC-CAT, Participation Measure-Post Acute Care-Computerized Adaptive Test; POPS, Participation Objective Participation Subjective; PO, Participation Objective; PS, Participation Subjective; QOLI, Lehman’s shortened Quality of Life Interview; QOLIBRI, Quality of Life After Brain Injury; QWB, Quality of Well-Being; RM, Roland Morris; RMSEA, Root Mean Square Error of Approximation; RNL, Reintegration to Normal Living; SE, Schwab and England’s Activities of Daily Living Scale; SF-36, Short Form-36; SF-36 Vet, Short Form-36 Veterans Version; SHAQ, Scleroderma Health Assessment Questionnaire; SMS, Sensory Modality Sum score; SODA, Sequential Occupational Dexterity Assessment; SWT, modified Shuttle Walk Test; UPDRS-ADL & -ME, Unified Parkinson’s Disease -Activities of Daily Living & -Motor Examination; UPSA, UCSD Performance-Based Assessment; VSDAI, Valentini Scleroderma Disease Activity Index; WCST, Wisconsin Card Sorting Test; WHODAS II, World Health Organization Disability Assessment Schedule II; WLQ, Work Limitations Questionnaire; WMSR, Wechsler Memory Scale Revisited  Health condition various health conditions26  various health conditions35  leprosy, spinal cord injury, poliomyelitis31  Intervention and follow-up out-patient rehabilitation with follow-up at 3 months26  no intervention, change between discharge from rehabilitation with follow-up at 3 months35  compared subjects expected to change (e.g. rehab) to subjects not expected to change subjects after 9-12 months31  Instrument IPA  PM-PACCAT  P-Scale  Table 2.6. Evidence related to responsiveness  59  value not reported, states it was according to expectation in the article31  mobility domain ROC+SE=0.587+0.079 (p=0.318)35; community, social and civic life domain ROC+SE=0.684+0.074 (p=0.034)35; domestic life domain ROC+SE=0.655+0.077 (p=0.076)35  overall rating of change provided by patients (absolute values)35  statistically significant improvement (p<0.05), minimum of 10 points & in the expected direction31  mobility domain SRM=0.44, ES=0.5835 community, social and civic life domain SRM=0.44, ES=0.5035; domestic life domain SRM=0.42, ES=0.5035  participation domains AUC=50-92%26; problem domains AUC=56-74%26  9 transition indices  NA  Responsiveness statistic participation domains SRM=0.1-1.326; problem domains SRM=0.4-1.526  Criterion for change NA  NA  NA  NA NA  various health conditions39;97  depression and back pain36 ankylosing spondylitis66 adult onset hearing loss68  rehabilitation, follow-up NR39;97  usual primary care with follow-up at 3 months36  spa treatment with follow-up at 4 weeks66  hearing aid intervention with follow-up at 10 weeks68  WHODAS II (36question)  Responsiveness statistic overall score ES=0.57-0.6967  overall score †,‡ ES=0.2068; participation in society domain †ES=0.1368; understanding and communication domain †ES=0.5268  overall score ES=0.3966; overall score SRM=0.4166  60  overall score ES=0.60-0.6536; subscales scores ES=0.060.7236  overall score ES=(0.16)(-0.69)97; overall score SRM=(0.16)-(-0.68)97; subscale scores ES=(-0.01)(-0.75)39; subscale scores SRM=(-0.01)-(-0.69)39  clinical improvement based overall score *ES=0.30-0.82, on symptom measures assess ability to detect symptom changes67  Criterion for change NA  Health condition anxiety disorders67  Intervention and follow-up cognitive behavioural group treatment programme, follow-up NR67  Instrument WHODAS II (12question)  Intervention and follow-up power wheel-chair prescription with follow-up at 3-5 months99  Health condition stroke99  Criterion for change NA  Responsiveness statistic overall score §ES=0.2699; subscale score ES=0.13-0.8499  Notes: * effect sizes are the differences in change scores between those individuals who demonstrated reliable improvement on a symptom measure and those who did not, divided by pooled standard deviation of the change scores for the two groups † for denominator of the ES used a pooled standard deviation of the baseline and 10 week scores ‡ does not include the work/education questions § 31 question version of WHODAS II, excludes work/education questions and a question on sexuality  61  Abbreviations: AUC, area under the curve; ES, effect size; IPA, Impact on Participation and Autonomy; NA, not applicable; NR, not reported; PM-PAC-CAT, Participation Measure-Post Acute Care-Computerized Adaptive Test; P-Scale, Participation Scale; ROC, receiver operator characteristic curve; SE, standard error; SRM, standardized response mean; WHODAS II, World Health Organization Disability Assessment Schedule II  Instrument WHODAS II (36question) cont.  2.5 References 1. World Health Organization. International Classification of Impairments, Disabilities and Handicaps. A manual of classification relating to consequences of diseases. Geneva: World Health Organization,1980. 2. Badley EM. The genesis of handicap: definition, models of disablement, and role of external factors. Disabil Rehabil 1995;17:53-62. 3. Gray DB, Hendershot GE. The ICIDH-2: developments for a new era of outcomes research. Arch Phys Med Rehabil 2000;81:S10-S14. 4. Grimby G, Smedby B. ICF approved as the successor of ICIDH. J Rehabil Med 2001;33:193-4. 5. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization, 2001. 6. Whiteneck GG, Charlifue SW, Gerhart KA et al. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil 1992;73:51926. 7. Harwood RH, Rogers A, Dickinson E et al. Measuring handicap: the London Handicap Scale, a new outcome measure for chronic disease. Qual Health Care 1994;3:11-6. 8. Noreau L, Fougeyrollas P, Post M et al. Participation after spinal cord injury: the evolution of conceptualization and measurement. J Neurol Phys Ther 2005;29:14756. 62  9. Carr AJ, Thompson PW. Towards a measure of patient-perceived handicap in rheumatoid arthritis. Br J Rheumatol 1994;33:378-82. 10. Ueda S, Okawa Y. The subjective dimension of functioning and disability: what is it and what is it for? Disabil Rehabil 2003;25:596-601. 11. Cardol M, Brandsma JW, de Groot IJ et al. Handicap questionnaires: what do they assess? Disabil Rehabil 1999;21:97-105. 12. World Health Organization. International Classification of Functioning and Disability Beta-2 Draft Full Version. Geneva: World Health Organization, 1999. 13. Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11:193-205. 14. Fitzpatrick R, Davey C, Buxton MJ et al. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2:i-74. 15. Pallant JF, Misajon R, Bennett E et al. Measuring the impact and distress of health problems from the individual's perspective: development of the Perceived Impact of Problem Profile (PIPP). Health Qual Life Outcomes 2006;4:36. 16. Stratford P. Reliability: consistency or differentiating among subjects? Phys Ther 1989;69:299-300. 17. Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. 1 ed. Chichester: John Wiley and Sons, 2000.  63  18. Terwee CB, Dekker FW, Wiersinga WM et al. On assessing responsiveness of healthrelated quality of life instruments: guidelines for instrument evaluation. Qual Life Res 2003;12:349-62. 19. Beckerman H, Roebroeck ME, Lankhorst GJ et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 2001;10:571-8. 20. Messick S. Validty of psychological assessment: validation of inferences from persons' responses and performance as scientific inquiry into score meaning. Am Psychol 1995;50:741-9. 21. Streiner DL, Norman GR. "Precision" and "accuracy": two terms that are neither. J Clin Epidemiol 2006;59:327-30. 22. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 2 ed. Oxford: Oxford University Press, 1995. 23. Cohen J. Statistical power analysis for behavioural sciences. New York: Academic, 1977. 24. Cardol M, de Haan RJ, van den Bos GA et al. The development of a handicap assessment questionnaire: the Impact on Participation and Autonomy (IPA). Clin Rehabil 1999;13:411-9. 25. Cardol M, de Haan RJ, de Jong BA et al. Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Arch Phys Med Rehabil 2001;82:210-6.  64  26. Cardol M, Beelen A, van den Bos GA et al. Responsiveness of the Impact on Participation and Autonomy questionnaire. Arch Phys Med Rehabil 2002;83:1524-9. 27. Wilkie R, Peat G, Thomas E et al. The Keele Assessment of Participation: a new instrument to measure participation restriction in population studies. Combined qualitative and quantitative examination of its psychometric properties. Qual Life Res 2005;14:1889-99. 28. Ostir GV, Granger CV, Black T et al. Preliminary results for the PAR-PRO: a measure of home and community participation. Arch Phys Med Rehabil 2006;87:1043-51. 29. Gandek B, Sinclair SJ, Jette AM et al. Development and Initial Psychometric Evaluation of the Participation Measure for Post-Acute Care (PM-PAC). Am J Phys Med Rehabil 2006;86:57-71. 30. Brown M, Dijkers MP, Gordon WA et al. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004;19:459-81. 31. Van Brakel WH, Anderson AM, Mutatkar RK et al. The Participation Scale: Measuring a key concept in public health. Disabil Rehabil 2006;28:193-203. 32. Gray DB, Hollingsworth HH, Stark SL et al. Participation Survey/Mobility: psychometric properties of a measure of participation for people with mobility impairments and limitations. Arch Phys Med Rehabil 2006;87:189-97.  65  33. Sandstrom M, Lundin-Olsson L. Development and evaluation of a new questionnaire for rating perceived participation. Clin Rehabil 2007;21:833-45. 34. World Health Organization. WHODAS II. [http://www.who.int/icidh/whodas]. November 2001; Accessed March 10, 2006. 35. Haley SM, Gandek B, Siebens H et al. Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes. Arch Phys Med Rehabil 2008;89:275-83. 36. Chwastiak LA, Von Korff M. Disability in depression and back pain: evaluation of the World Health Organization Disability Assessment Schedule (WHO DAS II) in a primary care setting. J Clin Epidemiol 2003;56:507-14. 37. Chavez LM, Canino G, Negron G et al. Psychometric properties of the Spanish version of two mental health outcome measures: World Health Organization Disability Assessment Schedule II and Lehman's Quality Of Life Interview. Ment Health Serv Res 2005;7:145-59. 38. Chisolm TH, Abrams HB, McArdle R et al. The WHO-DAS II: psychometric properties in the measurement of functional health status in adults with acquired hearing loss. Trends Amplif 2005;9:111-26. 39. Posl M, Cieza A, Stucki G. Psychometric properties of the WHODASII in rehabilitation patients. Qual Life Res 2007;16:1521-31.  66  40. Sibley A, Kersten P, Ward CD et al. Measuring autonomy in disabled people: Validation of a new scale in a UK population. Clin Rehabil 2006;20:793-803. 41. Franchignoni F, Ferriero G, Giordano A et al. Rasch psychometric validation of the Impact on Participation and Autonomy questionnaire in people with Parkinson's disease. Eura Medicophys 2007;43:451-61. 42. Kersten P, Cardol M, George S et al. Validity of the impact on participation and autonomy questionnaire: A comparison between two countries. Disabil Rehabil 2007;29:1502-9. 43. Lund ML, Fisher AG, Lexell J et al. Impact on participation and autonomy questionnaire: internal scale validity of the Swedish version for use in people with spinal cord injury. J Rehabil Med 2007;39:156-62. 44. Misajon R, Pallant JF, Manderson L et al. Measuring the impact of health problems among adults with limited mobility in Thailand: further validation of the Perceived Impact of Problem Profile. Health Qual Life Outcomes 2008;6:6. 45. Rehm J, Ustun B, Saxena S et al. On the development and psychometric testing of the WHO screening instrument to assess disablement in the general population. Int J Methods Psychiatr Res 1990;8:110-22. 46. Middelkamp W, Moulaert VR, Verbunt JA et al. Life after survival: long-term daily life functioning and quality of life of patients with hypoxic brain injury as a result of a cardiac arrest. Clin Rehabil 2007;21:425-31.  67  47. Lemmens J, van Engelen ISM, Post MW et al. Reproducibility and validity of the Dutch Life Habits Questionnaire (LIFE-H 3.0) in older adults. Clin Rehabil 2007;21:853-62. 48. Wilkie R, Peat G, Thomas E et al. The prevalence of person-perceived participation restriction in community-dwelling older adults. Qual Life Res 2006;15:1471-9. 49. Wilkie R, Peat G, Thomas E et al. Factors associated with participation restriction in community-dwelling adults aged 50 years and over. Qual Life Res 2007;16:1147-56. 50. Jette AM, Keysor J, Coster W et al. Beyond function: predicting participation in a rehabilitation cohort. Arch Phys Med Rehabil 2005;86:2087-94. 51. Keysor JJ, Jette AM, Coster W et al. Association of environmental factors with levels of home and community participation in an adult rehabilitation cohort. Arch Phys Med Rehabil 2006;87:1566-75. 52. Cantor JB, Ashman T, Gordon W et al. Fatigue after traumatic brain injury and its impact on participation and quality of life. J Head Trauma Rehabil 2008;23:41-51. 53. Schopp L, Clark MJ, Hagglund KJ et al. Life activities among individuals with spinal cord injury living in the community: perceived choice and perceived barriers. Rehabil Psychol 2007;52:82-8. 54. McKibbin C, Patterson TL, Jeste DV. Assessing disability in older patients with schizophrenia: results from the WHODAS-II. J Nerv Ment Dis 2004;192:405-13.  68  55. Gallagher P, Mulvany F. Levels of ability and functioning: using the WHODAS II in an Irish context. Disabil Rehabil 2004;26:506-17. 56. Wang J, Adair CE, Patten SB. Mental health and related disability among workers: a population-based study. Am J Ind Med 2006;49:514-22. 57. Pyszel A, Malyszczak K, Pyszel K et al. Disability, psychological distress and quality of life in breast cancer survivors with arm lymphedema. Lymphology 2006;39:185-92. 58. Ertugrul A, Ulug B. The influence of neurocognitive deficits and symptoms on disability in schizophrenia. Acta Psychiatr Scand 2002;105:196-201. 59. Norton J, de Roquefeuil G, Benjamins A et al. Psychiatric morbidity, disability and service use amongst primary care attenders in France. Eur Psychiatry 2004;19:164-7. 60. Hudson M, Steele R, Taillefer S et al. Quality of life in systemic sclerosis: psychometric properties of the World Health Organization Disability Assessment Schedule II. Arthritis Rheum 2008;59:270-8. 61. Kim JM, Stewart R, Glozier N et al. Physical health, depression and cognitive function as correlates of disability in an older Korean population. Int J Geriatr Psychiatry 2005;20:160-7. 62. Soberg HL, Bautz-Holter E, Roise O et al. Long-term multidimensional functional consequences of severe multiple injuries two years after trauma: a prospective longitudinal cohort study. J Trauma 2007;62:461-70.  69  63. Donmez L, Gokkoca Z, Dedeoglu N. Disability and its effects on quality of life among older people living in Antalya city center, Turkey. Arch Gerontol Geriatr 2005;40:213-23. 64. Hudson M, Thombs BD, Steele R et al. Clinical correlates of quality of life in systemic sclerosis measured with the World Health Organization Disability Assessment Schedule II. Arthritis Rheum 2008;59:279-84. 65. Von Korff M, Katon W, Lin EH et al. Potentially modifiable factors associated with disability among people with diabetes. Psychosom Med 2005;67:233-40. 66. van Tubergen A, Landewe R, Heuft-Dorenbosch L et al. Assessment of disability with the World Health Organisation Disability Assessment Schedule II in patients with ankylosing spondylitis. Ann Rheum Dis 2003;62:140-5. 67. Perini SJ, Slade T, Andrews G. Generic effectiveness measures: sensitivity to symptom change in anxiety disorders. J Affect Disord 2006;90:123-30. 68. McArdle R, Chisolm TH, Abrams HB et al. The WHO-DAS II: measuring outcomes of hearing aid intervention for adults. Trends Amplif 2005;9:127-43. 69. Cieza A, Geyh S, Chatterji S et al. ICF linking rules: an update based on lessons learned. J Rehabil Med 2005;37:212-8. 70. Dijkers MP, Whiteneck G, El Jaroudi R. Measures of social outcomes in disability research. Arch Phys Med Rehabil 2000;81:S63-S80.  70  71. Dijkers M. Comments on van Brakel et al.'s Participation Scale. Disabil Rehabil 2006;28:1360-2. 72. Fugl-Meyer AR, Branholm I, Fugl-Meyer KS. Happiness and domain-specific life satisfaction in adult northern Swedes. Clin Rehabil 1991;5:25-33. 73. Fugl-Meyer AR, Melin R, Fugl-Meyer KS. Life satisfaction in 18- to 64-year-old Swedes: in relation to gender, age, partner and immigrant status. J Rehabil Med 2002;34:239-46. 74. Johnston M, Nissim EN, Wood K et al. Objective and subjective handicap following spinal cord injury: interrelationships and predictors. J Spinal Cord Med 2002;25:1122. 75. Wilkie R, Peat G, Thomas E et al. Factors associated with restricted mobility outside the home in community-dwelling adults ages fifty years and older with knee pain: an example of use of the International Classification of Functioning to investigate participation restriction. Arthritis Rheum 2007;57:1381-9. 76. Streiner DL. Being inconsistent about consistency: when coefficient alpha does and doesn't matter. J Pers Assess 2003;80:217-22. 77. Magasi SR, Heinemann AW, Whiteneck GG. Participation following traumatic spinal cord injury: an evidence-based review for research. J Spinal Cord Med 2008;31:14556.  71  78. Cieza A, Stucki G. Content comparison of health-related quality of life (HRQOL) instruments based on the international classification of functioning, disability and health (ICF). Qual Life Res 2005;14:1225-37. 79. Hyland ME. A brief guide to the selection of quality of life instrument. Health Qual Life Outcomes 2003;1:24. 80. Dekker J, Dallmeijer AJ, Lankhorst GJ. Clinimetrics in rehabilitation medicine: current issues in developing and applying measurement instruments. J Rehabil Med 2005;37:193-201. 81. Whiteneck G. Conceptual models of disability: past, present, and future. In: Field MJ, Jette AM, Martin L, eds. Workshop on Disability in America: A New Look. Washington, D.C.: National Academies Press, 2006:50-66. 82. Cicerone KD. Participation as an outcome of traumatic brain injury rehabilitation. J Head Trauma Rehabil 2004;19:494-501. 83. Cardol M, de Jong BA, van den Bos GA et al. Beyond disability: perceived participation in people with a chronic disabling condition. Clin Rehabil 2002;16:2735. 84. Erdmann PG, Teunissen LL, Van Genderen FR et al. Functioning of patients with chronic idiopathic axonal polyneuropathy (CIAP). J Neurol 2007;254:1204-11. 85. Van Genderen FR, Van Meeteren NL, Heijnen L et al. The use of a disability model in haemophilia research. Haemophilia 2005;11:472-80.  72  86. Van Genderen FR, Westers P, Heijnen L et al. Measuring patients' perceptions on their functional abilities: validation of the Haemophilia Activities List. Haemophilia 2006;12:36-46. 87. Wynia K, Middel B, De Ruiter H. et al. Stability and relative validity of the Multiple Sclerosis Impact Profile (MSIP). Disabil Rehabil 2008;30:1027-38. 88. Lund ML, Nordlund A, Bernspang B et al. Perceived participation and problems in participation are determinants of life satisfaction in people with spinal cord injury. Disabil Rehabil 2007;29:1417-22. 89. Lund M.L., Nordlund A, Nygard L et al. Perceptions of participation and predictors of perceived problems with participation in persons with spinal cord injury. J Rehabil Med 2005;37:3-8. 90. Vazirinejad R, Lilley JM, Ward CD. The 'Impact on Participation and Autonomy': acceptability of the English version in a multiple sclerosis outpatient setting. Mult Scler 2003;9:612-5. 91. Misajon R, Manderson L, Pallant JF et al. Impact, distress and HRQoL among Malaysian men and women with a mobility impairment. Health Qual Life Outcomes 2006;4:95. 92. Cross H, Choudhary R. STEP: an intervention to address the issue of stigma related to leprosy in Southern Nepal. Lepr Rev 2005;76:316-24.  73  93. Cross H, Choudhury R. Self-care: a catalyst for community development. Asia Pacific Disabil Rehabil J 2005;16:100-14. 94. Nicholls PG, Bakirtzief Z, Van Brakel WH et al. Risk factors for participation restriction in leprosy and development of a screening tool to identify individuals at risk. Lepr Rev 2005;76:305-15. 95. Deconinck H. The health condition of spinal cord injuries in two Afghan towns. Spinal Cord 2003;41:303-9. 96. Chopra PK, Couper JW, Herrman H. The assessment of patients with long-term psychotic disorders: application of the WHO Disability Assessment Schedule II. Aust NZ J Psychiatry 2004;38:753-9. 97. Posl M. Evaluation of the World Health Organization Disability Assessment Schedule II (WHO DAS II) - German Version. 1-62. 2004. Ludwig Maximilian University. 98. Soberg HL, Finset A, Bautz-Holter E et al. Return to work after severe multiple injuries: a multidimensional approach on status 1 and 2 years postinjury. J Trauma 2007;62:471-81. 99. Pettersson I, Tornquist K, Ahlstrom G. The effect of an outdoor powered wheelchair on activity and participation in users with stroke. Disabil Rehabil Assist Technol 2006;1:235-43.  74  100. Schulberg HC, Lee PW, Bruce ML et al. Suicidal ideation and risk levels among primary care patients with uncomplicated depression. Ann Fam Med 2005;3:523-8. 101. Ertugrul A, Ulug B. Perception of stigma among patients with schizophrenia. Soc Psychiatry Psychiatr Epidemiol 2004;39:73-7.  75  3 CONTENT COMPARISON OF INSTRUMENTS ASSESSING PARTICIPATION BASED ON THE INTERNATIONAL CLASSIFICATION OF FUNCTIONING, DISABILITY AND HEALTH2 3.1 Introduction Participation is cited as central to a person’s quality of life and well-being1. The reduction of disabilities and improving participation for individuals with disabilities are therefore important goals of rehabilitation.2 Working for pay, attending school and joining in community activities are all examples of life situations that comprise participation. Participation is defined in the International Classification of Functioning, Disability and Health (ICF) as the involvement in a life situation and participation restrictions are defined as problems an individual may experience in the involvement in life situations.3 Although the idea of participation is not new, participation as defined in the ICF is a relatively new concept and as a result the conceptualization and measurement of participation continues to evolve.4 Whiteneck5 in his critique of the ICF recommended that new instruments operationalizing the concepts in the ICF are developed and then tested to assess the relationship between the concepts in the ICF model. In particular, Whiteneck5 recommended that the measurement properties of instruments must first be assessed to obtain consensus on how each concept within the ICF is quantified before it will be possible to test the relationship between concepts. He stated that it is important to develop new instruments which are designed to measure the ICF concepts rather than trying to retrofit existing instruments to the ICF classification.5  2  A version of this chapter has been submitted for publication. Noonan VK, Kopec JA, Noreau L, Singer J, Chan A, Mâsse LC, Dvorak MF. Content comparison of instruments assessing participation based on the International Classification of Functioning, Disability and Health.  76  It is therefore necessary to first assess how the concepts such as participation are being measured by examining the content validity of instruments. Content validity assesses how well the questions cover the health components being measured6 and this is important since it determines if the content of the instrument is appropriate for measuring the construct or concept.7 A recently proposed method for evaluating content validity involves identifying the content within the questions of an instrument and linking or mapping this content to the ICF classification. This methodology has been recommended since the ICF classification provides a standardized framework for evaluating content8 and to date, this methodology has been used to compare the content of both generic and disease-specific instruments.8;9 In 2003 Perenboom and Chorus2 reviewed the literature and examined how existing generic instruments assess the concept of participation. These authors concluded that most of the instruments assess one or more domains related to participation described in the ICF but none of them measured all the domains.2 Although this study provides an important contribution to the literature, the results cannot be used to understand how the ICF concept of participation has been operationalized since only a few of the instruments included in the review were developed based on the ICF.2 A draft version of the ICF was published in 1997 [(International Classification of Impairments, Disabilities and Handicaps-2 (ICIDH-2)] and the first version was officially published in 2001 and so an updated review needs to be conducted.3 In addition, the methodology for linking the content in the questions to the ICF was published in 2002 and revised in 2005, and this methodology was not used in the study by Perenboom and Chorus2. Thus, to address the recommendations proposed by Whiteneck5, it is important to build on the work of Perenboom and Chorus2 and examine the content of instruments developed  77  based on the ICF using published methodology for linking the content to the ICF. The purpose of this study was to conduct a detailed content analysis of participation instruments developed using the ICF, to elucidate how participation has been operationalized. Content validity of instruments was ranked according to whether the questions adequately cover the ICF concept of participation and whether any questions were irrelevant by including content which is not part of participation as defined in this study. Findings from this study will assist in determining which instrument(s) best cover the concept of participation as defined by the ICF and will serve to identify possible revisions of these instruments needed in order to test the relationships between the concepts in the ICF model.  3.2 Methods A systematic search of the literature was conducted to identify all instruments that were developed to assess the ICF concept of participation. Each instrument was then reviewed to evaluate the content in the questions and to link the content to the ICF classification. Finally, the instruments were rated according to two criteria which were used to assess content validity (described below). An overview of the ICF model and classification is provided below to assist in explaining how the instruments were selected and how the content was linked.  3.2.1 Overview of the ICF The ICF consists of two parts: functioning and disability and contextual factors. Functioning and disability contains the components body structures, body functions, and activities and participation. Activity is defined as the execution of a task or action by an individual.3 Although activity and participation are differentiated in the model, in the 78  classification they are combined and there is a single list of domains covering various actions and life areas. The user is provided with four options on how activity and participation can be considered: 1) divide activity and participation domains and do not allow for any overlap; 2) allow for partial overlap between activity and participation domains; 3) operationalize participation as broad categories within the domains and activity as the more detailed categories, with either partial or no overlap; and 4) allow for complete overlap in the domains considered to be activity and participation.3 Contextual factors comprise the background of a person’s life and living which interact with the individual and determine their level of functioning.3 They include environmental and personal factors. Environmental factors comprise the physical, social and attitudinal environment in which people live.3 These factors are external to individuals and can have a positive or negative influence on an individual’s performance as a member of society, on an individual’s capacity to execute actions or tasks, or on an individual’s body functions or structures.3 Personal factors are the particular details of an individual’s life and living and include factors such as gender, age and coping style.3 A detailed classification of environmental factors was first introduced in the ICF and currently a classification does not exist for personal factors. In addition, the ICF model includes health conditions (disorders or diseases) that are classified using the World Health Organization’s etiological classification, the International Classification of Diseases-10 (ICD-10). In the ICF classification the components are labeled with letters: body structures (s), body functions (b), activities and participation (d), and environmental factors (e). As mentioned previously, personal factors are not specified. Within each component the categories are organized hierarchically and assigned a numeric code. The categories are nested so the  79  chapters also referred to as domains, include all the detailed subcategories. The first-level category is the chapter number (1 digit) then there is the second-level category (2 digits), third (1 digit) and sometimes fourth-level (1 digit). An example demonstrating the coding from the activities and participation component is d4 Mobility (chapter/first-level category), d450 walking (second-level category), and d4500 walking short distances (third-level category).  3.2.2 Instruments A systematic search of seven databases [Medline; CINAHL; EMBASE; HaPI; Psyc (Info, Articles, Books)] was conducted to identify all the instruments that assess participation and were based on the ICIDH-2 or ICF model. The ICIDH-2 was first released in 1997 and so the search included articles published between 1997 and March 2008. Instruments including questions covering a minimum of three chapters in the ICIDH-2 participation dimension10, or three chapters from the ICF Chapters 3 to 9 in the activities and participation component in ICF classification, were considered to assess participation3. Instruments which met this definition of participation were then included if they fulfilled the inclusion criteria: were designed to assess participation in the community, either self-administered or interviewadministered, generic in content, developed for adults and published in English.  3.2.3 Linking to the ICF The questions in the instruments were assigned ICF categories, also known as linking or cross-walking. First the content contained within each of the questions and, if applicable, response options (response scale) was identified using standardized linking rules11. This content is referred to as the meaningful concept(s) in the published methodology.11 The 80  meaningful concept(s) capture all of the ideas or information contained within a question and these concepts are used to identify the ICF categories in the classification. Examples of the meaningful concepts extracted from the questions and the assigned ICF categories are provided in Table 3.1. To determine if contextual factors and health conditions are included in the participation instruments, relevant information stated in the instructions was also used to identify meaningful concepts, which is a modification to the published linking rules. For example, if the instructions stated the respondent should consider the impact of their health condition or the use of assistive devices when thinking about participating in certain life roles, then ‘health conditions’ and ‘assistive devices’ were included as meaningful concepts. Any terms referring to a time period (e.g. in the past four weeks) and qualifiers such as ‘difficulty’, ‘satisfaction’ or ‘importance’ were not considered to be meaningful concepts. Once all the meaningful concepts were identified, then the meaningful concepts were linked to the most suitable ICF category. The meaningful concept was classified as ‘not definable’ if there was not enough information to select the most precise ICF category and if a meaningful concept was not included in the ICF (e.g. suicide attempts) it was coded as ‘not covered’. A meaningful concept was coded as a ‘personal factor’ if it was about age or other factors that relate to the background of the person. Meaningful concepts such as health, illness or physical disability were coded as ‘health condition’. One coder was primarily responsible for identifying the meaningful concepts and two coders linked all the meaningful concepts in the instruments. The two coders reviewed their results and discussed the questions where different ICF categories or codes were selected. Another coder was consulted if there were any questions regarding the meaningful concepts,  81  the ICF categories or ICF codes and made a final decision. All the coders were familiar with the ICF and the linking rules11.  3.2.4 Analysis First a descriptive analysis was conducted. The number of meaningful concepts linked to categories in the ICF components as well as the number of meaningful concepts which could not be linked was reported. The most precise categories selected from the components activities and participation (d-categories) as well as body functions (b-categories), body structures (s-categories) and environmental factors (e-categories) were recorded and reported up to the second-level ICF categories. Agreement between the two coders was calculated as the percent agreement for the ICF categories (levels 1 to 3), which also included the ICF codes which could not be linked. Only the initial assessment of agreement for the ICF categories and codes assigned by the two coders was considered for the percentage of agreement analysis; it did not include any changes resulting from a third coder’s input. Second, content validity for each instrument was examined by evaluating the coverage (do the questions adequately cover the concept) and whether the questions are all relevant (do all the questions include content related to participation). Since the instruments have a different number of questions, the percentage of questions containing ICF categories from the chapters included in the activities and participation component was first calculated. Similar estimates were made for the ICF components body functions and environmental factors as well as for ‘health conditions’ and ‘not defined/not covered’. Instruments were then evaluated based on whether they contain questions with ICF category codes in 5 out of 7 ICF chapters (first-level categories) considered aspects of participation; as mentioned previously, the chapters considered aspects of participation include ICF Chapter 3 82  Communication (d3) to Chapter 9 Community, social and civic life (d9) in the activities and participation component. Coverage in 5 out of a possible 7 chapters was selected since this covers at least two-thirds of the chapters used to operationalize participation in this study and this was felt to be acceptable coverage. Relevance of the questions was evaluated by examining if all the questions have one or more meaningful concepts linked to ICF categories in Chapters 3 Communication (d3) to Chapter 9 Community, social and civic life (d9). Since it is possible that an instrument contains meaningful concept(s) related to participation but an ICF category could not be identified, meaningful concepts linked to ‘not defined’ and ‘not covered’ were reviewed to determine if the meaningful concepts were similar to the content included in the Chapters 3 Communication (d3) through Chapter 9 Community, social and civic life (d9).  3.3 Results 3.3.1 Identification of the Participation Instruments A review of the literature in September 2007 identified 3087 articles. After reviewing the articles based on the two-stage eligibility process ten instruments12-19 met the inclusion criteria: Impact on Participation and Autonomy (IPA)12;20, Keele Assessment of Participation (KAP)18, PAR-PRO14, Participation Measure-Post Acute Care (PM-PAC)13, Participation Objective Participation Subjective (POPS)21, Participation Scale (P-Scale)17, Participation Survey/Mobility (PARTS/M)22, Perceived Impact of Problem Profile (PIPP)15, Rating of Perceived Participation (ROPP)16, and World Health Organization Disability Assessment Schedule II (WHODAS II)19. The Participation Measure-Post Acute Care-Computerized Adaptive Test version (PM-PAC-CAT)23 was added when the systematic search was updated in March 2008. For eight of the instruments (IPA, KAP, PARTS/M, PM-PAC, POPS, P83  Scale, ROPP, WHODAS II) a copy of the instrument was available and so these instruments were included in the content analysis.  3.3.2 Linking the Meaningful Concepts to the ICF A total of 1351 meaningful concepts were identified in the 8 instruments. If examples are used to describe an aspect of participation then all the examples were coded as meaningful concepts and linked to ICF categories. Two instruments include screening questions (KAP, PARTS/M) and the meaningful concepts in the screening questions were also included. In the P-Scale there are a total of 36 questions; however, only 18 questions were assessed in for the P-Scale since the meaningful concepts are not explicitly stated in 18 questions which ask “how big a problem is it to you?” as a follow-up to the first question. In addition, there was no impact on the results by only including 18 questions from the P-Scale. A summary of the meaningful concepts linked to ICF categories is included in Table 3.2. The PARTS/M has the highest number of meaningful concepts (n=545). Sixty nine percent (932/1351) of the meaningful concepts were linked to categories in the component activities and participation. Three instruments (PARTS/M, P-Scale, WHODAS II) have meaningful concepts linked to body functions and all the instruments with the exception of the ROPP, have meaningful concepts linked to the component environmental factors. Four of the eight instruments (IPA, PARTS/M, PM-PAC, WHODAS II) have meaningful concepts which could not be linked to the ICF. Out of the 199 meaningful concepts that could not be linked to the ICF, 160 were coded as health conditions and 39 were either ‘not defined’ or ‘not covered’. Examples of meaningful concepts that were coded as ‘not defined’ include ‘other activities’ (PM-PAC), ‘days away from home’ (PM-PAC) and ‘staying by yourself for a few days’ (WHODAS II). Examples of meaningful concepts coded as ‘not covered’ include 84  ‘control over your own life’ (IPA) and ‘impact on your family’ (WHODAS II). The meaningful concept ‘health condition’ was identified in 4 of the 8 instruments (Table 3.2). Instructions in the WHODAS II and IPA ask the respondent to consider all the questions in the context of difficulties due to health conditions and so all questions contain meaningful concepts linked to ‘health conditions’. No meaningful concepts were linked to personal factors. Table 3.3 lists the categories from the activities and participation component that the meaningful concepts were linked to. All of the instruments contain meaningful concepts linked to the following ICF Chapters Mobility (d4), Domestic life (d6), Interpersonal interactions and relationships (d7), Major life areas (d8), and Community, social and civic life (d9). Table 3.4 describes the categories within the ICF components body functions (bcategories) and environmental factors (e-categories) included in the instruments. In the PARTS/M, for each of the 20 aspects of participation assessed there is a question which asks if either ‘pain’ (linked to b280 pain) or ‘fatigue’ (linked to b4552 fatigability and the second-level category is b455 exercise tolerance functions), limits participation. The PScale contains a meaningful concept ‘confidence’ (linked to b126 temperament and personality functions). The WHODAS II contains three questions which ask about ‘remembering to do important things’, being ‘emotionally affected’ and ‘living with dignity’, which were linked to b144 memory, b152 emotional functions and b1 Mental functions, respectively. Seven instruments include meaningful concepts which were linked to categories in the ICF component environmental factors. Six instruments (IPA, KAP, PARTS/M, PM-PAC, POPS, P-Scale) either ask about the use of aids or assistance in either a specific question or  85  ask the respondent to consider these factors when considering aspects of participation; these meaningful concepts were linked to categories within e1 Products and technology or e3 Support and Relationships. In three instruments (IPA, P-Scale, WHODAS II,) there are questions asking about attitudes of others, where the meaningful concepts were linked to categories in e4 Attitudes. The PM-PAC has two questions that ask about ‘filing your taxes’ / ‘completing forms for insurance or disability benefits’ and the e-categories were related to the instructions which tells the respondent to consider any assistance or services available to them (meaningful concepts linked to categories in e3 Support and relationships and e5 Services, systems and policies). The percentage of observed agreement between the two coders for the ICF categories and codes ranged between 91-100% for the first-level ICF categories, 77-95% for the secondlevel ICF categories and 77-94% for the third-level ICF categories. The percentage of agreement was not assessed for the IPA since this instrument was linked to the ICF in a previous study conducted by one of the authors (VKN).  3.3.3 Assessing Content Validity In terms of content coverage, the number of questions with content from the ICF Chapters 3 to 9 ranges from 0 (0%) for d3 Communication (IPA, PARTS/M) and d5 Selfcare (POPS) to 58 (36%) for d9 Community, social and civic life in the PARTS/M (Table 3.5). All of the instruments met the criteria of having questions containing content covering 5 out of 7 ICF Chapters (d3 Communication to d9 Community, social and civic life) in the activities and participation component. In terms of relevance, there are questions in the IPA, PM-PAC and WHODAS II which do not contain d-categories from d3 Communication to d9 Community, social and civic life but the meaningful concepts were considered to be related to 86  participation (e.g. WHODAS II ‘staying by yourself’). There are two instruments (P-Scale, WHODAS II) that have questions which do not contain d-categories from d3 Communication to d9 Community, social and civic life and were considered to assess something other than participation. The P-Scale has one question which just contains meaningful concepts linked to e-categories (e4 Attitudes) and another question asking about ‘feeling confident trying new things’ which was linked to d1 Learning and applying knowledge. The WHODAS II has nine questions which do not contain meaningful concepts related to d3 Communication to d9 Community, social and civic life; these questions have meaningful concepts linked to b1 Mental functions, d1 Learning and applying knowledge, e4 Attitudes, health condition and ‘not covered’ or ‘not defined’ and were therefore not considered related to participation (e.g. ‘barriers or hindrances in the world around you’). The IPA, KAP, PARTS/M, PM-PAC, POPS and the ROPP met both criteria for content validity. A table summarizing the results on the content validity is provided in Table 3.6.  3.4 Discussion 3.4.1 Defining the Concept of Participation In this study an instrument was included if its domains cover a minimum of three chapters from the ICF Chapters 3 to 9 in the component activities and participation. This broad definition of participation was used since there is no consensus regarding how activity is differentiated from participation2;5;5;24-27 and selecting chapter headings provided objective criteria. Perenboom and Chorus2, however, considered a question to be assessing participation if it asked about actual or perceived participation (involvement, autonomy, social role) (page 578) and so different results would be obtained using this definition.  87  3.4.2 Content Validity of the Participation Instruments Although all the instruments adequately met the requirement for including content from 5 of the 7 ICF chapters, there are differences in the actual content. All of the instruments include content from ICF Chapters Domestic life (d6), Interpersonal interactions and relationships (d7), Major life areas (d8), and Community, social and civic life (d9). There are differences, however, in whether the Chapters Communication (d3), Self-care (d5) and certain aspects of Mobility (d4) are considered part of participation. Four instruments (PM-PAC, P-Scale, ROPP, WHODAS II) intend to assess d3 Communication based on the original articles and ICF categories from d3 Communication were noted for all these instruments. Meaningful concepts linked to categories in d3 Communication were also identified in the KAP and POPS, which was likely unintentional. In the KAP and POPS there are questions which contain meaningful concepts linked to multiple ICF chapters, including communication, but the latter was not the major focus. For example, in the POPS the question “How many times do you speak with your neighbour?” includes the meaningful concept ‘conversation’ which was coded as d350 conversation but it is only a minor meaningful concept and the major meaningful concept is ‘relationship with neighbour(s)’, coded as d7501 informal relationships with neighbours. In some instruments such as the PM-PAC, assessing communication is a major focus (“How much are you limited in watching or listening to the television or radio?”). Empirical findings suggest that it is difficult to demonstrate discriminant validity among participation domains12;18and this may be a result of overlapping content. In future studies it may be beneficial to identify and code the major and minor meaningful concepts, since this could assist with developing a priori hypotheses regarding expected correlations between instrument domains.  88  All of the instruments contain meaningful concepts linked to categories in d5 Self-care with the exception of the POPS. When the POPS was developed self-care was not included since participation was operationalized as “engagement in activities that are intrinsically social, that are part of household or other occupational role functioning, or that are recreational activities occurring in community settings” (page 463) and self-care did not qualify.21 The PM-PAC does not intend to assess self-care13 but there were two meaningful concepts linked to categories in d5 Self-care. One question in the PM-PAC asks about ‘exercising’ which was coded as d570 managing diet and fitness and the other question asks about ‘providing self-care to yourself’, which was coded as d5 Self-care. In terms of mobility, all of the instruments contain meaningful concepts linked to categories in d4 Mobility and all the instruments intend to include content from this chapter. Three instruments (IPA, PARTS/M, WHODAS II) operationalize moving in the home using specific phrases such as ‘getting out of bed, getting out of a chair…’ (PARTS/M) or ‘getting up and going to bed’ (IPA). In the other instruments, mobility includes broader statements such as ‘moving around the home’ (KAP, ROPP) or ‘getting around’ (PM-PAC, P-Scale, ROPP) and in the POPS mobility only includes using transportation. In terms of the relevance of the content, the P-Scale and WHODAS II were considered to have content not related to the concept of participation. When the P-Scale was developed participation was considered to include content from Chapter Learning and applying knowledge (d1)17 and the differences in how participation was operationalized explains why this question did not meet the criteria for content relevance. There was one questions in the P-Scale that did not meet the criteria for content relevance; the question “In your home, are the eating utensils you use kept with those used by the rest of the household?” includes the  89  meaningful concepts ‘family member attitudes’ and ‘eating utensils’ that were considered to be primarily assessing environmental factors (‘e410 individual attitudes of immediate family members’ and ‘e115 products and technology for personal use in daily living’, respectively). This question only asks about the observable consequences of others’ attitudes and so it was not considered to be related to the concept of participation. Similarly, the WHODAS II contains questions with meaningful concepts linked to d1 Learning and applying knowledge which is expected since this instrument was developed to assess the concepts of activity and participation.19 The WHODAS II, like the P-Scale also contains questions which do not contain any d-categories (e.g. “How much of a problem did you have because of barriers or hindrances in the world around you?”) and so it was not just due to the differences in how participation was operationalized.  3.4.3 Linking the Meaningful Concepts to the ICF The methodology published by Cieza et al.28 was used to identify and link meaningful concepts to the ICF. Our results for the d-categories selected for the WHODAS II can be compared to a study by Cieza and Stucki9, which also linked the WHODAS II to the ICF. It is difficult to compare the results from these two studies directly since Cieza and Stucki9 used an older version of the linking rules28 and we modified the linking rules by including ‘health condition’ as a meaningful concept if it was included in the instructions. Cieza and Stucki9 identified 38 meaningful concepts and in our study we had 45 not including coding ‘health condition’, however, we did not include the five questions in the WHODAS II on general health and it appears that Cieza and Stucki9 did. Both studies had the same number of meaningful concepts linked to body functions (n=3), environmental factors (n=1) and ‘not defined’ (n=2). There were some differences. We linked 38 meaningful concepts to 90  categories from activities and participation and Cieza and Stucki9 linked 30 meaningful concepts and we linked 1 meaningful concept to ‘not covered’ whereas they linked 2 meaningful concepts. Overall, it appears that the linking rules were applied similarly in the two studies. It has been recognized that there are a number of challenges with using the linking rules (e.g. establishing the meaningful concepts contained in the assessment items) and future work will further enhance the standardization of these rules.29  3.4.4 Participation and Other ICF Categories It was informative to examine the ICF categories within each question to determine which other ICF categories were used in conjunction with the categories from the ICF component activities and participation. The ICF states that disability is a dynamic process which results from the interaction of the ICF components (body structures, body functions, activities and participation) and the contextual factors (environment, personal factors).3 It is helpful, therefore, to identify what is asked in relation to participation. For example, in the PARTS/M for every participation topic area (e.g. dressing, working inside the home) there is a question asking whether participation is impacted by pain and/or fatigue. Clinically it is useful to determine the impact of factors such as pain and fatigue, because similar to environmental factors they can be potentially modified in order to enhance participation. As stated by Nordenfelt26 and others30, activity and participation must occur in an environment. In the ICF there is reference to a ‘standard environment’ versus ‘usual environment’ and this distinction is one way activity is differentiated from participation.3 It is interesting how environmental factors asking about assistance or equipment are included in some instruments (IPA, KAP, PARTS/M, PM-PAC, POPS, P-Scale) but not in other instruments (ROPP, WHODAS II). The PARTS/M specifically assesses the use of assistance 91  and the frequency in which accommodations, adaptations or special equipment is used. Asking about the use of equipment and assistance is important clinically since if a person does not have a suitable environment, this can be modified to enhance their participation. Further qualitative and quantitative studies will determine if respondents inherently consider their environment when answering the questions. Similar to environmental factors, there is variation in whether a participation restriction is attributed to a health condition. In the WHODAS II and IPA, the instructions state that the respondent should consider his or her health condition. In the PARTS/M there are specific questions which ask if the person’s participation is limited by their illness or physical impairment. Dubuc et al.31 demonstrated the importance of specifying whether the participation restriction is a result of a health condition or not, especially for areas which are highly influenced by environmental factors. By asking if the participation restriction is a result of a health condition, it underestimated the influence of the environment since subjects focussed on the implications due their health and did not often consider the restrictions in the physical and social environment.31 More research should assess the best way to assess these influencing factors. The PARTS/M offers the advantage of asking specific questions related to these areas and this may provide a means to determine their effects. None of the questions contain meaningful concepts coded as personal factors, which is not surprising since this data is collected separately (e.g. age, gender) in research studies. Further studies should compare questions that either attribute or do not attribute participation to factors such as the environment or health conditions to determine if these phrases influence a person’s response.  92  3.4.5 Study Limitations and Conclusions There are several limitations to this study which need to be considered when interpreting the results. The methodology for assessing content validity was developed for this study which limits comparisons to other studies but it may be useful in future studies. The criteria used for evaluating content validity in this study assume that it is desirable to have an instrument cover the majority of areas within a multidimensional concept such as participation and so it may not be suitable for instruments that focus on selected areas such as employment. As very few studies have linked the instruments used in this study to the ICF classification, the results from this study should be confirmed in other studies. Interpreting the questions and determining the meaningful concepts can be influenced by culture and the experience of the coders. Future enhancements to the ICF linking rules and more examples demonstrating how the meaningful concepts are identified and coded will enhance the methodology in these types of studies. In summary, this study linked eight instruments measuring participation to the ICF classification. Benefits of linking the content of instruments to the ICF has been described in various studies8;9;32 and these benefits include enabling users to review the content as part of the selection process, to provide a standardized approach to comparing the content and to inform revisions of existing instruments. An enhancement to the linking methodology enabled the role of contextual factors to be examined within each question. Including contextual factors in the ICF is an important step forward and empirical research comparing results from instruments that either include and or do not include contextual factors will further advance the measurement of participation. The instruments all contain content from the ICF Chapters Domestic life (d6) to Community, social and civic life (d9), but there is variability in whether content from ICF Chapters Communication (d3), Mobility (d4) and 93  Self-care (d5) is included. All the instruments contain content covering five chapters in the ICF component activities and participation; however, the WHODAS II and KAP have questions with concepts not considered aspects of participation. The differences observed in the eight participation instruments regarding content, inclusion of environmental factors, and attributing participation restrictions due to health should be considered when selecting an instrument.  94  d730 relating with strangers health condition  health condition  Notes: * the text in italics is the instrument’s instructions which was considered to include meaningful concepts and is a modification to the linking rules  95  e3 support & relationships e120 products & technology for personal indoor & outdoor mobility & transportation  assistance from others use of gadgets/machines  dealing with strangers  ICF Category Assigned d4600 moving around within the home  Meaningful Concept moving around in my home  Abbreviations: ICF, International Classification of Functioning, Disability and Health; KAP, Keele Assessment of Participation; WHODAS II, World Health Organization Disability Assessment Schedule II  In the last 30 days how much difficulty did you have in dealing with people you do not know. This questionnaire asks about difficulties due to health conditions.* (WHODAS II)  Question During the past 4 weeks, I have moved around in my home, as and when I have wanted to. It does not matter if you require the help of other people or from gadgets and machines. * (KAP)  Table 3.1. Examples of linking questions to ICF categories  66 40 26  82 3  2 7  9  103 14  40 379 60  85  27 22  55 66  PM-PAC 117  PARTS/M 479 135 9  POPS 144 1 42 4  P-Scale 47 153  ROPP 153  36 3  39  3 38 1  WHODAS II 42  96  Abbreviations: ICF, International Classification of Functioning, Disability and Health; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PARTS/M, Participation Survey/Mobility; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II  Number of meaningful concepts not linked to ICF categories Health conditions Not defined or not covered  Number of meaningful concepts linked to ICF categories Body function Activity/Participation Environmental factors  KAP 49  IPA 121  Table 3.2. Summary of the data abstracted from the participation instruments  d4 Mobility d410 Changing basic body position d415 Maintaining a body position d450 Walking d455 Moving around d460 Moving around in different locations d470 Using transportation d475 Driving  d3 Communication d310 Communicating with-receivingspoken messages d325 Communicating with-receivingwritten messages d330 Speaking d335 Producing non-verbal messages d345 Writing messages d350 Conversation d355 Discussion d360 Using communication devices & techniques  d2 General tasks & demands  ICF category d1 Learning & applying knowledge d155 Acquiring skills d160 Focusing attention d175 Solving problems  43  2  2  PARTS/M  10  1  KAP  1 1  IPA  Table 3.3. ICF categories in the component activities and participation  7  4  4  1  3  4 2  PM-PAC  6  POPS  12 3  10  1  P-Scale 1  6  10  12  3 12 3  1  ROPP  97  9  2  1 1 1  2  1  2 1 1  WHODAS II  d8 Major life areas d820 School education d825 Vocational training d830 Higher education d840 Apprenticeship d845 Acquiring, keeping & terminating a job  d7 Interpersonal interactions & relationships d720 Complex interpersonal interactions d730 Relating with strangers d740 Formal relationships d750 Informal social relationships d760 Family relationships d770 Intimate relationships  d6 Domestic life d620 Acquisition of goods & services d630 Preparing meals d640 Doing housework d650 Caring for household objects d660 Assisting others  d5 Self-care d510 Washing oneself d520 Caring for body parts d530 Toileting d540 Dressing d550 Eating d560 Drinking d570 Looking after one’s health  ICF category  1 1 1  1 2 1 1  1 4 1  5  7  4  4  3  1 4 2 2  1 2 1 1  1  3  1  KAP 1 1  IPA 1 2  7 12 35  21 21 2  2  18 18 5  18  PARTS/M  7 1 2 3 1 4  1 12 8  9 6 1 1 6  PM-PAC 1  3  12 6 3  3 3  3 6 4  1  POPS  4  1  1 1  4 3 3 1 3 4  P-Scale 2  3 3  3  6  3 7 6 3  6  2  6 3 3 2  ROPP 7  98  3  5  1  2  1  1  2  1 1  1  WHODAS II  1  2  6  2 2  1 1  2  KAP  IPA  62 14 28  9 9 7 28  PARTS/M  2 19 3 3  3  1  PM-PAC  6 3 24 6  3 3 3 6  POPS  2 5 4 2  1  P-Scale  6 3 4  1  3 3  1  ROPP  3 1 1  1  WHODAS II  99  Abbreviations: ICF, International Classification of Functioning, Disability and Health; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PARTS/M, Participation Survey/Mobility; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II  d9 Community, social and civic life d910 Community life d920 Recreation & leisure d930 Religion & spirituality d950 Political life & citizenship  ICF category d8 Major life areas cont. d850 Remunerative employment d855 Non-remunerative employment d860 Basic economic transactions d865 Complex economic transactions d870 Economic self-sufficiency  Environmental Factors (e) e1 Products & technology e115 Products & technology for personal use in daily living e120 Products & technology for personal indoor & outdoor mobility & transportation e125 Products & technology for communication e130 Products & technology for education e135 Products & technology for employment e140 Products & technology for culture, recreation & sport 2 1 1 1  5 1 2 5 2  1 5  2 15  2  2  3  24 8  20  PARTS/M  b4 Functions of the cardiovascular, hameatological, immunological & respiratory systems b455 Exercise tolerance functions  KAP  20  IPA  b2 Sensory functions b280 Pain  ICF category Body Functions (b) b1 Mental functions b126 Temperament & personality functions b144 Memory functions b152 Emotional functions  1  PM-PAC  9  POPS  Table 3.4. ICF categories in the components body functions and environmental factors  1  1  P-Scale  ROPP  100  1 1  1  WHODAS II  1  e4 Attitudes e410 Individual attitudes of immediate family members e425 Individual attitudes of acquaintances, peers, colleagues, neighbours & community members  20  1  PARTS/M  3 3  7  PM-PAC  POPS  1  2  P-Scale  ROPP  1  WHODAS II  101  Abbreviations: ICF, International Classification of Functioning, Disability and Health; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PARTS/M, Participation Survey/Mobility; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II  e5 Services, systems & policies e575 General social support services, systems & policies  11  32  e3 Support & relationships  1  KAP  IPA  ICF category Environmental Factors (e) e1 Products & technology cont. e145 Products & technology for the practice of religion & spirituality  0 (0)  0 (0)  4 (10)  5 (12)  9 (22)  d3 Communication # questions containing categories from d3 (%)  d4 Mobility # questions containing categories from d4 (%)  d5 Self-Care # questions containing categories from d5 (%)  d6 Domestic life # questions containing categories from d6 (%)  0 (0)  d2 General tasks & demands # questions containing categories from d2 (%)  Activities & Participation d1 Learning & applying knowledge # questions containing categories from d1 (%)  Number of questions in the instrument  IPA 41  4 (27)  1 (7)  2 (13)  1 (7)  0 (0)  0 (0)  KAP 15  17 (11)  29 (18)  19 (12)  0 (0)  0 (0)  0 (0)  PARTS/M 159  4 (8)  2 (4)  8 (16)  7 (14)  0 (0)  0 (0)  PM-PAC 51  18 (23)  0 (0)  6 (8)  6 (8)  0 (0)  0 (0)  POPS 78  2 (11)  1 (6)  3 (17)  1 (6)  0 (0)  1 (6)  P-Scale 18*  7 (10)  10 (14)  13 (19)  13 (19)  0 (0)  0 (0)  ROPP 69  Table 3.5. Number of questions with ICF categories (%, number questions with ICF categories/total questions)  102  4 (11)  3 (8)  5 (14)  2 (6)  0 (0)  3 (8)  WHODAS II 36  Health Conditions # questions containing meaningful concepts coded as health conditions (%)  Environmental Factors # questions containing categories from environmental factors (ecategories) (%)  Body Functions # questions containing categories from body functions (b-categories) (%)  d9 Community, social & civic life # questions containing categories from d9 (%)  d8 Major life areas # questions containing categories from d8 (%)  d7 Interpersonal interactions & relationships # questions containing categories from d7 (%)  41 (100)  34 (83)  0 (0)  6 (15)  9 (22)  9 (22)  IPA  0 (0)  11 (73)  0 (0)  2 (13)  5 (33)  1 (7)  KAP  20 (13)  40 (25)  20 (13)  58 (36)  25 (16)  22 (14)  PARTS/M  1 (2)  7 (14)  0 (0)  11 (22)  14 (27)  11 (22)  PM-PAC  0 (0)  9 (12)  0 (0)  30 (38)  12 (15)  24 (31)  POPS  0 (0)  3 (17)  1 (6)  4 (22)  3 (17)  3 (17)  P-Scale  0 (0)  0 (0)  0 (0)  7 (10)  15 (22)  13 (19)  ROPP  103  36 (100)  1 (3)  3 (8)  2 (6)  5 (14)  5 (14)  WHODAS II  1 (2)  0 (0)  KAP 26 (16)  PARTS/M 7 (14)  PM-PAC 0 (0)  POPS 0 (0)  P-Scale 0 (0)  ROPP  3 (8)  WHODAS II  104  Notes: * the P-Scale has 36 questions but only 18 questions are considered since the meaningful concepts are not explicitly stated in 18 questions which ask “How big a problem is it to you?” for each question  Abbreviations: ICF, International Classification of Functioning, Disability and Health; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PARTS/M, Participation Survey/Mobility; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II  Not Defined/Not Covered # questions containing meaningful concepts coded as not defined/ not covered (%)  IPA  Table 3.6. Content validity of the participation instruments  Instrument  Criteria #1: Questions cover ICF category codes in at least five ICF Chapters d3 to d9*  IPA KAP PARTS/M PM-PAC POPS P-Scale ROPP WHODAS II  yes yes yes yes yes yes yes yes  Criteria #2: All questions contain one or more meaningful concepts linked/related to categories in ICF Chapters d3 to d9* yes† yes yes yes† yes no yes no† ‡  Abbreviations: ICF, International Classification of Functioning, Disability and Health; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PARTS/M, Participation Survey/Mobility; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; P-Scale, Participation Scale; ROPP, Rating of Perceived Participation; WHODAS II, World Health Organization Disability Assessment Schedule II Notes: * d3 Communication to d9 Community, social and civic life † contains ‘not defined’ or ‘not covered’ codes that are similar in content to d3 Communication to d9 Community, social and civic life ‡ the WHODAS II also has questions which do not contain any meaningful concepts linked or related to content in d3 Communication to d9 Community, social and civic life  105  3.5 References 1. Reinhardt JD, Stucki G. Rheumatoid arthritis and participation--the social animal revisited. J Rheumatol 2007;34:1214-6. 2. Perenboom RJ, Chorus AM. Measuring participation according to the International Classification of Functioning, Disability and Health (ICF). Disabil Rehabil 2003;25:577-87. 3. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization, 2001. 4. Noreau L, Fougeyrollas P, Post M et al. Participation after spinal cord injury: the evolution of conceptualization and measurement. J Neurol Phys Ther 2005;29:147-56. 5. Whiteneck G. Conceptual models of disability: past, present, and future. In: Field MJ, Jette AM, Martin L, eds. Workshop on Disability in America: A New Look. Washington, D.C.: National Academies Press, 2006:50-66. 6. Fitzpatrick R, Davey C, Buxton MJ et al. Evaluating patient-based outcome measures for use in clinical trials. Health Technol.Assess 1998;2:i-74. 7. Sireci SG. The construct of content validity. Soc Indic Res 1998;83-117. 8. Stucki A, Stucki G, Cieza A et al. Content comparison of health-related quality of life instruments for COPD. Respir Med 2007;101:1113-22.  106  9. Cieza A, Stucki G. Content comparison of health-related quality of life (HRQOL) instruments based on the international classification of functioning, disability and health (ICF). Qual Life Res 2005;14:1225-37. 10. World Health Organization. International Classification of Functioning and Disability Beta-2 Draft Full Version. Geneva: World Health Organization, 1999. 11. Cieza A, Geyh S, Chatterji S et al. ICF linking rules: an update based on lessons learned. J Rehabil Med 2005;37:212-8. 12. Cardol M, de Haan RJ, de Jong BA et al. Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Arch Phys Med Rehabil 2001;82:210-6. 13. Gandek B, Sinclair SJ, Jette AM et al. Development and Initial Psychometric Evaluation of the Participation Measure for Post-Acute Care (PM-PAC). Am J Phys Med Rehabil 2006;86:57-71. 14. Ostir GV, Granger CV, Black T et al. Preliminary results for the PAR-PRO: a measure of home and community participation. Arch Phys Med Rehabil 2006;87:1043-51. 15. Pallant JF, Misajon R, Bennett E et al. Measuring the impact and distress of health problems from the individual's perspective: development of the Perceived Impact of Problem Profile (PIPP). Health Qual Life Outcomes 2006;4:36. 16. Sandstrom M, Lundin-Olsson L. Development and evaluation of a new questionnaire for rating perceived participation. Clin Rehabil 2007;21:833-45.  107  17. Van Brakel WH, Anderson AM, Mutatkar RK et al. The Participation Scale: Measuring a key concept in public health. Disabil Rehabil 2006;28:193-203. 18. Wilkie R, Peat G, Thomas E et al. The Keele Assessment of Participation: a new instrument to measure participation restriction in population studies. Combined qualitative and quantitative examination of its psychometric properties. Qual Life Res 2005;14:1889-99. 19. World Health Organization. WHODAS II. [http://www.who.int/icidh/whodas]. November 2001; Accessed March 10, 2006. 20. Sibley A, Kersten P, Ward CD et al. Measuring autonomy in disabled people: Validation of a new scale in a UK population. Clin Rehabil 2006;20:793-803. 21. Brown M, Dijkers MP, Gordon WA et al. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004;19:459-81. 22. Gray DB, Hollingsworth HH, Stark SL et al. Participation survey/mobility: psychometric properties of a measure of participation for people with mobility impairments and limitations. Arch Phys Med Rehabil 2006;87:189-97. 23. Haley SM, Gandek B, Siebens H et al. Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes. Arch Phys Med Rehabil 2008;89:275-83.  108  24. Badley EM. Enhancing the conceptual clarity of the activity and participation components of the International Classification of Functioning, Disability, and Health. Soc Sci Med 2008. 25. Jette AM, Haley SM, Kooyoomjian JT. Are the ICF Activity and Participation dimensions distinct? J Rehabil Med 2003;35:145-9. 26. Nordenfelt L. Action theory, disability and ICF. Disabil Rehabil 2003;25:1075-9. 27. Schuntermann MF. The implementation of the International Classification of Functioning, Disability and Health in Germany: experiences and problems. Int J Rehabil Res 2005;28:93-102. 28. Cieza A, Brockow T, Ewert T et al. Linking health-status measurements to the international classification of functioning, disability and health. J Rehabil Med 2002;34:205-10. 29. Xiong T, Hartley S. Challenges in linking health-status outcome measures and clinical assessment tools to the ICF. Adv Physiother 2008;10:152-6. 30. Fougeyrollas P, Beauregard L. An interactive person-environment social creation. In: Albretch GL, Seelman K.D., Bury M, eds. Handbook of Disability Studies. Thousand Oaks: Sage Publications, 2001:171-94. 31. Dubuc N, Haley SM, Kooyoomjian JT et al. Assessing disability in older adults: the effects of asking questions with and without health attribution. J Rehabil Med 2004;36:226-31.  109  32. Geyh S, Cieza A, Kollerits B et al. Content comparison of health-related quality of life measures used in stroke based on the International Classification of Functioning, Disability and Health (ICF): a systematic review. Qual Life Res 2007;16:833-51.  110  4 COMPARING THE RELIABILITY OF FIVE PARTICIPATION INSTRUMENTS3 4.1 Introduction As disability rates continue to rise with an aging population, advances in medicine and improved disability surveillance, there will be a greater need to understand how health conditions impact a person’s life. The concept of participation, defined as the involvement in life situations in the International Classification of Functioning Disability and Health (ICF)1, is therefore receiving considerable attention in the literature. Since the ICF was published in draft form in 1997 a recent review conducted in 2008 identified 11 new participation instruments developed using the ICF. (Chapter 2) There have been tremendous advances in how participation is operationalized. Initially participation was assessed based on objective information (e.g. the number of hours a person works).2 However, it has been recognized that it is also important to measure how a person perceives his or her participation3 and the recent review of participation instruments (Chapter 2) noted that instruments are now assessing more subjective information4;5. Since measuring participation using the ICF model is a relatively recent development, more research evaluating the measurement properties of participation instruments is needed. There has also been a recent increase in the number of participation instruments and so information is needed to help users select an instrument for a given study. Relatively few studies include multiple participation instruments, so it is not known how well the instruments compare. A direct comparison of participation instruments, including both subjective and objective participation instruments is therefore needed.  3  A version of this chapter will be submitted for publication. Noonan VK, Kopec JA, Noreau L, Singer J, Mâsse LC, Zhang H, Dvorak MF. Assessing the Reliability of Five Participation Instruments in Persons with Spinal Conditions: I. How do they Compare?  111  We selected five of the eleven instruments identified in the literature review (Chapter 2). These instruments include Impact on Participation and Autonomy (IPA)6, Keele Assessment of Participation (KAP)5, Participation Measure-Post Acute Care (PM-PAC)7, Participation Objective and Participation Subjective (POPS)8, and World Health Organization Disability Assessment Schedule II (WHODAS II)9. The IPA and KAP assess perceived autonomy in participation (doing something the way and when one wants to). The PM-PAC and WHODAS II primarily assess perceived difficulty or limitations in participation. The POPS includes quantifiable information regarding frequency of participation along with subjective information asking about the importance and satisfaction for various types of activities. All of these instruments can provide valuable information to clinicians and researchers. Six instruments were excluded because they were 1) too specific (Participation Survey/Mobility just measures lower extremity mobility10), 2) designed to assess participation in developing countries ( Participation Scale11), 3) administered by interview or computer (PAR-PRO12 and Participation Measure-Post Acute Care-Computer Adaptive Test13), 4) too similar to other instruments (Rating of Perceived Participation14 was too similar to the IPA and KAP), or 5) not available (Perceived Impact of Problem Profile15). Participation instruments can be used to assess individual as well as group differences. Clinicians need to know if an instrument is able to detect individual changes to be useful clinically while researchers often assess group differences. Assessing floor and ceiling effects as well as reliability using standardized criteria provides evidence regarding what instrument is best suited for what purpose. Floor and ceiling effects limit an instrument’s ability to detect changes or differences in individuals or between groups.16 In these five participation instruments ceiling effects have been reported. For example, 70.3% of  112  individuals with conditions such as diabetes reported no problems with self-care in the WHODAS II17 and 53% of community-dwelling individuals had no participation restrictions using the KAP5. However, since these instruments have not been compared in individuals in a single study, it is not known how the floor and ceiling effects in the instruments compare. Reliability is the degree to which an instrument is free from random error.18 Two types of reliability are frequently assessed. The first is determining if the questions within a multiitem scale are homogenous or are internally consistent. The second type of reliability assesses whether the information provided by individuals remains stable over time (test-retest reliability in self-administered instruments). Test-retest reliability can be assessed using the intraclass correlation coefficient (ICC) and expressed as a ratio between 0 and 1. Since the ICC is calculated using the ratio of variability between individuals and the total variability (variability between individuals and measurement error) it describes an instrument’s ability to differentiate among individuals in the sample studied.19-21 Test-retest reliability can also provide information on the consistency of responses using the standard error of measurement (SEM) by calculating the variability of measurements on the same individual.19-21 The SEM assesses the absolute measurement error (how close scores are on repeated measures) and it is reported in the unit of the scale.20 As more instruments are being used in clinical practice there has been a growing interest in using the SEM to calculate the minimal amount of change in a score that must be observed beyond the absolute measurement error, referred to as the minimal detectable change (MDC).22 To date, information on the internal consistency and test-retest reliability using ICCs for these five instruments has been published; however it is difficult to compare the instruments since the results are based on different health  113  conditions and the information is collected in different settings.(Chapter 2) In addition, very little has been reported on the SEM and MDC for these instruments. A direct comparison of five participation instruments would provide important information regarding the use of these instruments in individuals with spinal conditions. In addition, it would enable the instruments to be compared directly, which would help improve our understanding of how participation has been operationalized and provide recommendations for clinicians and researchers. Therefore, the purpose of this study was to compare the floor and ceiling effects, internal consistency, test-retest reliability (using ICCs and SEM) and MDC in five participation instruments (IPA, KAP, PM-PAC, POPS, WHODAS II).  4.2 Methods 4.2.1 Recruitment and Study Procedures Individuals admitted to the Vancouver General Hospital (VGH) Acute Spine Program, in Vancouver, British Columbia Canada between 2001 and March 2005, were eligible if they had a diagnosis of 1) a traumatic or non-traumatic spinal cord injury (SCI), 2) a spinal column facture without neurological involvement, or 3) a spinal degenerative disease (e.g. disc herniation, spondylosis). Individuals were excluded if they were deceased; could not be contacted; did not speak English; had a cognitive deficit; were not able to physically complete the instruments (e.g. age, a person with a SCI who is ventilator dependent); or were recently discharged from hospital within the last three months and were not able to do regular activities (e.g. because they were prescribed bed rest because of a pressure sore). In each of the three diagnostic groups a sample size of approximately 200 individuals was targeted. These diagnoses were selected since it provided a mechanism to compare different health 114  conditions and it was possible to access these former patients. Individuals eligible for the study were randomly selected from the hospital database until the target sample size was achieved or until all eligible individuals had been contacted. The study was approved by the Behavioural Research Ethics Board at the University of British Columbia (Appendix A) and all individuals provided written informed consent. Individuals were contacted by mail and asked to complete a questionnaire. A study coordinator followed up with all the potentially eligible participants to determine if the questionnaire was received approximately one week after the questionnaire was mailed. A reminder letter with another copy of the questionnaire was mailed if individuals indicated that they were interested in participating in the study but did not return the questionnaire. All returned questionnaires were checked by the study coordinator and if there were sections with missing data then the subjects were contacted and asked if they would be willing to complete them. No subsequent contact was made if subjects indicated that they did not want to be re-contacted and subjects were not asked to complete any questions related to income or intimate relationships. Test-retest reliability was assessed using a sub-sample of individuals from each diagnostic group. If individuals agreed to participate in the reliability study they were asked to complete the instruments twice within 10 days. A target sample size of 50 individuals per group was based on a sample size estimation23 of 124 subjects, which used an ICC of 0.75 obtained from the previous studies using these instruments (Chapter 2).  115  4.2.2 Data Elements Data was obtained from hospital databases and from a questionnaire completed by the respondents and included sociodemographic/socioeconomic data, clinical data, and domain/total scores from participation instruments.  4.2.2.1 Sociodemographic and Socioeconomic Data Data on age and gender were obtained from the hospital database. Respondents were asked about marital status, racial background, living situation (living alone or with someone), education, employment and compensation status related to their spine condition.  4.2.2.2 Clinical Data Diagnosis and treatment information for each respondent was obtained from hospital databases. The following diagnostic categories were included 1) traumatic and non-traumatic SCI (myelopathy); 2) spinal column fractures due to trauma; and 3) spinal degenerative conditions (stenosis, disc degeneration, spondylolisthesis, spondylosis). All surgical procedures for each individual were recorded. Neurology data for individuals with traumatic SCI consisted of the first neurological assessment during the acute admission using the International Standards for the Neurological Classification of SCI (ISNCSCI).24 Comorbidities were assessed using one section of the Self-Administered Comorbidity Questionnaire25 which measures the presence or absence of 14 comorbid conditions. One point is assigned for each comorbid condition producing a maximum score of 14 points.  4.2.2.3 Participation Instruments A description of the five participation instruments is provided below. 116  The IPA6 assesses the perceived impact of a health condition or disability on participation and autonomy in the following domains autonomy indoors (e.g. self-care); family role (e.g. housework); autonomy outdoors (e.g. visiting friends, leisure time); social life and relationships; and work and education. Based on 31 questions, the perceived participation score was calculated for each domain. A lower score indicates better perceived participation. The KAP5 contains 11 questions asking about autonomy in conducting life activities in the sub-domains mobility (2 questions); self-care; domestic life (3 questions); interpersonal interactions and relationships; major life areas (3 questions); and community, social and civic life. The mean score for each question was calculated in order to compare domains within the participation instruments, with a lower score indicating a better perceived participation. Participation Measure – Post Acute Care7 is designed to assess participation in the community. It contains a total of 51 questions and 42 questions are used to create a score for the domains communication; mobility; domestic life; interpersonal relationships; role functioning; work and employment; education; economic life; and community, social and civic life. A higher score indicates better participation. The POPS8 assesses participation in 26 life activities in both an objective (frequency) and a subjective manner (importance and level of satisfaction). A scoring algorithm provided by the developers was used to calculate the overall objective and subjective scores as well objective and subjective domain scores for domestic life; major life areas; transportation; interpersonal interactions and relationships; and community, recreational and civic life. Objective scores are based on z scores which represent the difference between the frequency information for each question compared to reference data from a sample including traumatic brain injury (TBI) and healthy controls. The domains were weighted based on the perceived  117  importance of the activity in the reference sample. Subjective scores are obtained by multiplying the individual’s importance score by the satisfaction score and range from -4 (important area that a person wants to do more or less of the activity) to +4 (important area that a person is satisfied with the amount of activity). The POPS was originally developed to be interviewer-administered and a self-administered version was developed and tested for use in this study. The scoring algorithm was modified slightly when the raw (non-imputed) data was used. It has been reported that subjects often omit questions if they do not engage in the activity26 and in the algorithm the subjective questions are not scored if subjects respond ‘don’t know’ to either the importance or satisfaction question, which could result in considerable missing data. To maximize the information provided by the subjects, domain scores were, therefore, calculated even if less than half of the subjective questions were answered within a domain. The WHODAS II9 assesses daily functioning using domains covered in the activities and participation component of the ICF. There are 36 questions and the domains include understanding and communicating, getting around, self-care, getting along with people, life activities (household/work activities), and participation in society. A scoring algorithm was provided by the World Health Organization. Separate scores were calculated for individuals who were working and not working for the life activities domain as well as for the total score since four questions related to work/school were not relevant for all individuals. A lower score indicates better participation.  118  4.2.3 Statistical Analysis For each instrument the score distribution, internal consistency and test-retest reliability (ICC and SEM) were evaluated. In this paper results are presented only for the overall sample since the primary purpose of this study was to compare the instruments. The score distributions were assessed using descriptive statistics (mean, standard deviation, range). The percentage of individuals with the lowest level of participation and the highest level of participation were recorded and values greater than 15% were considered to be substantial floor and ceiling effects, respectively.22 Internal consistency assesses the homogeneity of a multi-item scale and it is evaluated using Cronbach’s alpha coefficient.18 It has been recommended that a minimum of 0.70 is required for group comparisons and 0.90 - 0.95 is needed for individual comparisons.18 Testretest reliability was assessed using a two-way random effects model (ICC 2,1), with absolute agreement to account for any systematic variability between the two administrations.27 Recommended minimum values are 0.70 and 0.90 for individual and group-level comparisons, respectively.16;18 For instruments consisting of categorical scales, a weighted kappa coefficient was used. Although some suggest that it is difficult to apply criterion when assessing weighted kappa28, for the purpose of this study 0.70 was used as the minimal standard22. In this study SEM was calculated from the square root of the within-subject variance obtained from ANOVA (the square root of the sum of the between measures variance and the residual variance). Systematic differences between the test and retest were included when calculating the SEM, as recommended in the literature.20 The SEM can be used to calculate the MDC (MDC=1.96 X √2 X SEM) which represents the smallest within-person change in score that can be detected in an individual beyond measurement error, with p<0.05.22 The 119  MDC as a percentage of the scale score range was also calculated to compare the instruments. Bland and Altman29 recommend that plots displaying the differences between the first and second administration of the instrument against the average of the domain or total scores can provide a visual display of the agreement. Limits of agreement were calculated using the formula ‘mean change + 1.96 X standard deviation of these changes’ and are included in the plots.29 The Bland and Altman plots were reviewed to determine if there were any systematic differences over the range of the mean values for the domain scores. The limits of agreement provide similar information as the MDC but they are useful to determine if the SEM depends on the scale score when reviewing the graphs. The percentages of missing data for the first and second administration of the participation instruments was less than 10% at the question-level (except for two questions in the second administration of the POPS which asked about attending school, where the missing data was 12.8% and 14.1%). For the overall sample (n=545) missing data was imputed using the procedure for multiple imputation using SAS (PROC MI), however, only one simulated version of the data set was created. Variables potentially related to the reason for the missing data and variables known to be associated with the participation scores were included in the model. The imputation was done within each instrument (imputation did not borrow information from other instruments) and data pertaining to work and education were only imputed for individuals who indicated that they were involved in these activities. The imputed data was used to estimate the score distribution and the internal consistency. Only the raw data (non-imputed) was used to assess test-retest reliability including the ICC, SEM estimates and Bland and Altman plots. Additional information is included in Appendix B for  120  the amount of question- and person-level missing data, the type of missing data and the methods used to impute the data.  4.3 Results 4.3.1 Recruitment A total of 545 individuals participated in the study. The response rates for individuals eligible in each of the groups were 59% (145/246) SCI group, 58% (187/320) spinal column group and 62% (213/345) spinal degenerative group. The overall response rate was 60% (545/911). Average time of discharge from VGH to study follow-up was approximately 4 years and ranged from 3 months to 7.5 years. A total of 139 individuals completed the reliability study. The mean time and standard deviation (SD) between the first and second administration of the instruments was 14.70 (5.60) days and ranged between 7 and 31 days. Seventeen individuals did not complete the second administration within this time period and were excluded in the test-retest analysis.  4.3.2  Subject Characteristics  An overview of the individuals who participated in this study is described in Table 4.1. The average age of the overall group was 51.5 (16.6) years. Sixty-seven percent (n=367) of the overall group were males, with slightly fewer males in the degenerative group (56%) compared to the SCI group (79%). Sixty-two percent of individuals were married or were living with a partner but 78% lived with someone. Education level was similar among the three groups and in the overall sample 49% had either attended college or university. There were differences in employment status, with the SCI group having the highest unemployment (7%) compared to the spinal column group and spinal degenerative group (2%). A 121  comparison of individuals who participated in this study and those who were eligible, but did not participate revealed the study participants were older (46.97 versus 39.98 years) on admission to VGH and there were fewer men (67% versus 73%) compared to nonparticipants.  4.3.3 Score Distributions The scores for each of the instruments are reported in Table 4.2. Ceiling effects were present in the IPA, KAP, PM-PAC and WHODAS II (Table 4.2). The KAP had the highest percentages of ceiling effects, ranging from 56.7% to 75.8% in the eleven questions. All of the IPA domains also demonstrated ceiling effects ranging from 29.4% to 49.5%, affecting less of the sample compared to the KAP. Both the PM-PAC and the WHODAS II had a few domains which did not have ceiling effects. The PM-PAC domain community, social and civic life had a perfect score in 15% of the overall group, almost meeting the criteria. In the WHODAS II life activities for the non-working group, 13.6% of the sample had a perfect score and the total score was the best possible score in 2.5% and 12.5% for the non-working and working group respectively. The POPS was the only instrument that did not suffer from ceiling effects. In the objective domains of the POPS (except for the domestic life domain) the data regarding frequency was used and since the questions are open-ended ceiling effects were not possible. A floor effect was noted in the POPS objective major life areas domain. None of the other instruments demonstrated any floor effects. A summary of the scores and the floor and ceiling effects for the three groups (SCI, spinal column, spinal degenerative) are described in Appendix C, Table C.1.  122  4.3.4 Internal Consistency and Test-retest Reliability Internal consistency was assessed using Cronbach’s alpha and the results are reported in Table 4.3. Additional analyses for each of the three groups for Cronbach’s alpha are provided in Appendix C (Table C.2). The internal consistency was good (values greater than 0.70) in all the instruments and the IPA was the only instrument to have values for internal consistency greater than 0.90. Internal consistency was not assessed in the KAP and the POPS. In this study the KAP was reported using individual questions and even the overall score with the number of participation restrictions (each question dichotomized into yes or no) would likely not have high correlations among the questions. In the POPS different aspects of participation are included in the domains which are not necessarily related and so measuring internal consistency is also likely not applicable.30 In comparing the results from the first and second administration for all the instruments, most of the domain scores were not significantly different (see Table 4.4). The ICC values (95% confidence intervals) for the IPA ranged between 0.83 (0.77, 0.88) for social life and relationships and 0.88 (0.84, 0.92) for the family role domain. In the PM-PAC, all the ICCs were greater than 0.70 with the exception of the communication domain which had an ICC of 0.59 (0.47, 0.69). The ICC values for the POPS total objective and subjective score were 0.82 (0.75, 0.87) and 0.82 (0.76, 0.93), respectively. Overall, the POPS objective domains had higher ICC values, with the exception of the interpersonal interactions and relationships domain, where the subjective ICC domain value was higher than the objective value (0.72 versus 0.61). A comparison of the test-retest data for the POPS subjective domains using the original and the slightly modified scoring algorithm (generating a domain score even if less than half of the questions were scored) revealed that there was no impact on the ICCs and more subjects were included in the analysis for transportation and major life areas. The 123  weighted kappa values for the KAP ranged from 0.47 (0.27, 0.67) for economic life to 0.79 (0.66, 0.92) for domestic life (question #6). Additional test-retest data for the three spine groups is outlined in Appendix C, Table C.3. Estimates for the SEM and the MDC for each of the five instruments are summarized in Table 4.3 and additional data for each of the three groups is outlined in Appendix C Table C.4. The MDC as a percentage of the scale, on average ranged from 20 to 30%. The estimate of 13.5% for the education domain in the PM-PAC and 67.0% for the education question in the KAP were based on small samples. Graphs were produced to demonstrate the mean score for the first administration of the reliability data and the MDC as a percentage of each domain score range (Figures 4.1 to 4.5). Information from the graphs demonstrate that due to the high ceiling effects it would not be possible to detect improvements beyond measurement error for most of the instruments (IPA, KAP, PM-PAC, WHODAS II). It also would not be possible to detect deterioration in three POPS objective domains (major life areas; interpersonal interactions and relationships; community, recreational and civic life) due to floor effects. The Bland and Altman plots for the five instruments are included in Appendix C (Figures C.1 to C.5). Overall the differences between the two tests for each domain were not dependent on the domain scores in the five instruments.  4.4 Discussion To our knowledge this is the first direct comparison of participation instruments based on the ICF. Overall, internal consistency estimates for the instruments’ domains were acceptable; however, large ceiling effects were present in most of the instruments. The testretest reliability data suggest that the instruments are able to discriminate at a group-level. 124  Estimates of the SEM and MDC indicate it would be difficult to detect improvements at an individual-level due to the ceiling effects. A summary of the results is included in Table 4.5. The results regarding the ceiling effects are consistent with previous studies.5;17 We observed large ceiling effects in domains related to self-care, economic life, and interpersonal interactions and relationships (IPA, KAP, PM-PAC, WHODAS II). The domains related to work as well as community, social and civic life had the least problems, but the percentage of the sample with a perfect score was still greater than 15% for most instruments. The KAP suffered from the largest number of ceiling effects. Over 56% of the sample had a perfect score for each question and it was as high as 75.8% for self-care. The IPA domains autonomy indoors as well as social life and relationships are considered the least difficult, which is also consistent with other studies31-34. Ceiling effects were not an issue in the POPS because of scoring algorithm, which is an advantage. There is no maximum value for the estimates of frequency which are used to calculate the objective POPS domains (except domestic life). However, floor effects were a problem for the POPS objective major life domain. In terms of the POPS subjective domains, ceiling and floor effects were not common because it was rare for individuals to be completely unsatisfied or satisfied in all important areas for all questions within a domain. The ceiling effects observed in this study may result from individuals either recovering or adapting to their spinal condition and so consequently participation restrictions may not be a problem. The KAP was developed to assess participation restrictions at a population-level5 and so it would be expected that individuals in the general population would not have as many participation restrictions compared to individuals with health conditions. This instrument is more likely to demonstrate ceiling effects since the KAP was reported using the  125  questions as compared to domains scores in the other instruments which include multiple questions. Future studies should administer these instruments prior to and following interventions in order to determine if these instruments are responsive. Estimates of internal consistency were very good in the IPA, PM-PAC and WHODAS II. All of the IPA domains had a Cronbach’s alpha values between 0.90 and 0.96 and IPA was the only instrument which met the criteria for both individual- and group-level comparisons. The social life and relationship domain had the lowest value for internal consistency (0.90) and this is supported by other studies6;34;35. Internal consistency was also lowest in the domain getting along with people (0.81) in the WHODAS II and other studies assessing individuals with health conditions such as stroke, breast cancer, diabetes and osteoarthritis have reported similar findings17. Internal consistency was not assessed in the KAP and the POPS. In the KAP, one or two questions are included for each aspect of participation. The POPS objective domains include quantifiable information and it does not make sense to assume that a person who works will also attend school.8;36 In fact, it is more likely that a person who works will not attend school due to time constraints.8;36 It has been suggested that not all the measurement criteria are necessarily relevant when evaluating participation instruments and it is important to consider how participation is operationalized in an instrument before the measurement properties are assessed.36 Test-retest reliability estimates based on the ICC values were adequate in the five participation instruments assessed in this study and are similar to results from other studies. Domains in the IPA, WHODAS II and the PM-PAC (except the communication domain) all met the criterion of having an ICC>0.70 suggested for measuring group differences and these results are similar to those from previous studies6;7;37;38. Very few studies have demonstrated  126  that these participation instruments are able to achieve ICC values>0.90 recommended for individual comparisons. The study of IPA by Sibley et al.34 was one of the few studies which reported ICC values ranging from 0.91 to 0.97, with an interval of 2 weeks between tests. The higher values reported by Sibley et al.34 may be due to the lower scores and therefore more variability in their sample which would produce higher ICC values39. Sibley et al.34 also did not state the type of ICC used which may also explain the differences between the two studies39. It is interesting that test-retest reliability was higher for the objective participation domains compared to the subjective participation domains in the POPS, which was not the case in individuals with TBI8. These variations may be due to differences in the sample variability and type of ICC used.39 Results from this study add new information regarding the absolute measurement error and how much change is needed to detect differences beyond measurement error. Estimates of SEM and the MDC have been reported previously for the WHODAS II in adults with acquired hearing loss.37 Estimates for SEM and MDC were higher in this study for 5 of the 7 domains; for example, in the domain ‘getting along with people’ the SEM was 9.04 versus 7.23 and the MDC was 25.04 versus 15.0, in this study compared to the study by Chisolm et. al.37. There are variations in the type of data and calculations used to calculate SEM and MDC20 (e.g. SEM can be calculated using Cronbach’s alpha or within-subject variance from test-retest studies) and the methods used to obtain these estimates were not clearly described by Chisolm et. al.37 , which may explain some of the differences observed. Future studies should include detailed methodology regarding the calculations used so studies using similar instruments can be compared.  127  The values for the MDC as a percentage of the scale were on average between 20% and 30% and ranged from 13.5% and 67%. Other studies have also reported values ranging between 26% and 39% for instruments such as the low vision quality of life (LVQOL) and Vision-Related Quality of Life Core Measure (VCM1).40 For the Sickness Impact Profile measurement error accounted for 9.26% in the total score and was as high as 40.27% for questions asking about alertness.19 The authors concluded that the SIP was likely not responsive enough to detect changes in individuals who had a stroke, considering the baseline values. In this study, due to the high ceiling effects it would not be possible to detect improvements beyond measurement error for the majority of domains in the IPA, KAP, PM-PAC and WHODAS II. The MDC estimates are based on individual-level changes and group-level MDC estimates would be lower since the MDC is divided by the √n (sample size)41 and so participation instruments would be much better at detecting grouplevel changes. It is important to note that the MDCs reported in this study do not necessarily represent the differences that are expected to be clinically relevant, referred to as minimal important change (MIC). Estimates of MDC are a measurement property of a particular instrument.19 For an instrument to be useful clinically the MDC should be less than the change considered to be clinically relevant.19 Future studies must further assess changes that are meaningful to individuals receiving a particular intervention. It has been recommended that the anchorbased methodology, whereby an external criterion is used to measure important change as opposed to distribution-based approaches which rely on statistical properties of the sample be used to determine MIC.22  128  There are several limitations to this study which must be considered. Although the sample included three types of spinal conditions and vary in demographics and clinical symptoms, these results cannot be generalized to other health conditions. Future studies should continue to compare instruments and include individuals with more disabling health conditions given the problems experienced with ceiling effects in this study. Finally, as mentioned previously, this study was a cross-sectional assessment of participation following an acute care admission for three different spinal diagnoses. In order to determine if the instruments are useful in the clinical setting future studies should assess the instruments’ MIC before any conclusions can be made regarding their role in clinical assessment. In conclusion, this study compared the score distributions, internal consistency and testretest reliability for five participation instruments. Results from this study can be used by researchers and clinicians to select instruments appropriate for a given purpose. The IPA, PM-PAC and WHODAS II had similar measurement properties in individuals with spinal conditions. The KAP was developed to assess participation in population-based studies and results from this study also suggest that it may not have the measurement properties required clinically to assess various aspects of participation. Future studies should continue to revise and test the POPS since it captures both objective information (frequency) and subjective information (importance and satisfaction) in various activities which is unique. Evaluating test-retest reliability using the SEM and calculating the MDC indicated that measuring changes at an individual-level in all of the instruments may be difficult due to measurement error. Rather than asking individuals to answer all domains included in instruments measuring concepts such as participation, future consideration should be given to selecting relevant domains depending on the user’s purpose. This would provide the user with  129  flexibility in selecting domains within instruments that have the best measurement properties and studies directly comparing instruments will help provide the necessary information to accomplish this.  130  Table 4.1. Characteristics of the study respondents for the entire sample SCI (n=145)  Spinal Column (n=187)  Spinal Degenerative (n=213)  Overall (n=545)  Variable  Description  Gender  male  79%  71%  56%  67%  Marital Status  single married/partner divorced/widowed  31% 55% 14%  25% 60% 15%  8% 69% 23%  20% 62% 18%  Racial Background  Caucasian  80%  88%  87%  86%  Living Support  live with someone  75%  79%  79%  78%  Education  high school college/university graduate  43% 49% 8%  36% 54% 10%  38% 45% 16%  39% 49% 12%  Employment  employed unemployed volunteer/retired unable to work  32% 7% 32% 26%  70% 2% 19% 9%  50% 2% 32% 14%  52% 3% 28% 15%  Compensation  yes  59%  17%  19%  29%  Spinal Procedures  yes  86%  48%  98%  78%  AIS traumatic SCI only (n=123)  AIS A AIS B AIS C AIS D  42% 15% 18% 24%  131  Variable  Description  mean (SD) (range) Age  at follow-up  Comorbidity (0 to 14)  score at follow-up  Motor Score (0 to 100)  on admission  SCI (n=145)  Spinal Column (n=187)  Spinal Degenerative (n=213)  Overall (n=545)  48.7 (17.4) (21 to 86)  46.8 (16.2) (21 to 85)  57.6 (14.5) (24 to 90)  51.5 (16.6) (21 to 90)  1.0 (1.4) (0 to 8)  0.9 (1.3) (0 to 6)  1.5 (1.5) (0 to 7)  1.2 (1.4) (0 to 8)  51.9 (26.2) 0 to 96  Abbreviations: AIS, ASIA Impairment Scale; SCI, spinal cord injury; SD, standard deviation  132  Table 4.2. Descriptive information and floor/ceiling effects for the participation instruments based on the entire sample (n=545) Instruments (score range) IPA (0 to 4) Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work and Education (n=356) KAP (1 to 5) Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6 (n=286) Interpersonal Interactions & Relationships Economic Life Work (n=327) Education (n=193) Community, Social & Civic Life (n=412) PM-PAC (1 to 5) Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment (n=299) Education (n=64) Economic Life Community, Social & Civic Life  Overall mean (SD)  Overall range  % Worst Score  % Best Score  0.55 (0.77) 0.99 (0.97) 1.14 (1.14) 0.62 (0.70) 0.99 (1.12)  0 to 3.57 0 to 4.00 0 to 4.00 0 to 3.00 0 to 4.00  0.0 0.2 1.5 0.0 1.7  49.5 29.4 31.0 41.1 38.2  1.40 (0.73) 1.69 (0.97) 1.37 (0.78) 1.62 (0.95) 1.45 (0.81) 1.58 (0.87) 1.49 (0.82)  1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00  0.4 1.5 1.1 1.8 1.1 1.4 0.9  70.3 56.7 75.8 61.5 69.5 60.1 66.6  1.48 (1.00) 1.57 (1.10) 2.05 (1.48) 1.70 (1.08)  1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00  5.7 5.8 14.0 3.6  74.7 71.6 58.0 60.9  4.63 (0.66) 4.26 (0.93) 4.32 (0.87) 4.08 (0.94) 3.54 (1.19) 4.19 (0.97) 4.43 (0.78) 4.59 (0.76) 4.03 (0.90)  1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 2.00 to 5.00 1.00 to 5.00 1.17 to 5.00  0.4 0.2 0.6 0.4 4.0 1.0 0.0 0.6 0.0  58.2 43.3 44.8 30.8 16.7 39.1 43.8 66.6 15.0  133  Instruments (score range) POPS (subjective domains -4 to 4) Objective Domestic Life Objective Major Life Areas Objective Transportation Objective Interpersonal Interactions & Relationships Objective Community, Recreational & Civic Life Objective Participation Total Subjective Domestic Life Subjective Major Life Areas Subjective Transportation Subjective Interpersonal Interactions & Relationships Subjective Community, Recreational & Civic Life Subjective Participation Total WHODAS II (0 to 100) Understanding & Communicating Getting Around Self-Care Life Activities (Non-working; n=162) Life Activities (Working; n=383) Getting Along with People Participation in Society Total Score (Non-working; n=162) Total Score (Working; n=383)  Overall mean (SD)  Overall range  % Worst Score  % Best Score  -0.15 (0.91) 0.79 (1.76) -0.80 (0.56) 0.88 (2.54)  -2.22 to 2.03 -0.98 to 10.69 -1.31 to 3.17 -1.59 to 20.09  0.6 27.5 2.0 0.7  2.0 0.0 0.0 0.0  0.43 (1.37)  -1.16 to 10.06  1.1  0.0  0.24 (0.91) 1.00 (1.28) 0.28 (1.44) 0.89 (1.41) 0.99 (1.19)  -1.29 to 4.34 -3.00 to 4.00 -3.33 to 3.33 -4.00 to 4.00 -3.38 to 3.75  0.0 0.0 0.0 0.2 0.0  0.0 0.4 0.0 0.6 0.0  0.70 (0.96)  -2.80 to 3.20  0.0  0.0  0.77 (0.88)  -2.77 to 2.92  0.0  0.0  11.48 (16.69) 31.33 (27.57) 13.74 (22.20) 45.56 (30.95)  0 to 80.00 0 to 100.00 0 to 100.00 0 to 100.00  0.0 1.3 0.9 10.5  48.1 22.4 61.0 13.6  21.64 (23.93)  0 to 100.00  1.0  33.2  16.07 (19.79) 26.93 (22.43) 29.91 (17.26)  0 to 100.00 0 to 91.67 0 to 76.09  0.2 0.0 0.0  40.2 17.4 2.5  18.20 (17.58)  0 to 84.91  0.0  12.5  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PMPAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SD, standard deviation; WHODAS II, World Health Organization Disability Assessment Schedule II 134  Table 4.3. Internal consistency and standard error of measurement for the entire sample (n=545)  Instruments (score range) IPA (0 to 4) Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work and Education KAP (1 to 5) Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6 Interpersonal Interactions & Relationships Economic Life Work Education Community, Social & Civic Life PM-PAC (1 to 5) Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment Education Economic Life Community, Social & Civic Life  # Cronbach’s Questions alpha  SEM  MDC  MDC % of Scale (%)  7 7 5 6 6  0.94 0.95 0.95 0.90 0.96  0.25 0.30 0.42 0.28 0.35  0.70 0.83 1.18 0.76 0.96  17.5 20.8 29.0 19.0 24.0  1 1 1 1 1 1 1  NA NA NA NA NA NA NA  0.31 0.54 0.33 0.39 0.40 0.26 0.33  0.88 1.05 0.91 1.09 1.10 0.72 0.91  22.0 26.3 22.8 27.3 27.6 18.0 22.8  1 1 1 1  NA NA NA NA  0.62 0.48 0.97 0.40  1.73 1.34 2.68 1.10  43.3 33.5 67.0 27.5  6 5 3 3 4 5 4 3 9  0.91 0.93 0.85 0.85 0.92 0.90 0.84 0.84 0.90  0.29 0.26 0.34 0.42 0.58 0.42 0.19 0.30 0.34  0.80 0.73 0.94 1.17 1.61 1.16 0.54 0.84 0.93  20.0 18.3 23.5 29.3 40.3 29.0 13.5 21.0 23.3  135  Instruments (score range) POPS* (subjective domains -4 to 4) Objective Domestic Life Objective Major Life Areas Objective Transportation Objective Interpersonal Interactions & Relationships Objective Community, Recreational & Civic Life Objective Participation Total Subjective Domestic Life Subjective Major Life Areas Subjective Transportation Subjective Interpersonal Interactions & Relationships Subjective Community, Recreational & Civic Life Subjective Participation Total  # Cronbach’s Questions alpha  SEM  MDC  MDC % of Scale (%)  8 3 2 8  NA NA NA NA  0.28 0.56 0.23 1.20  0.79 1.54 0.64 3.33  NA NA NA NA  5  NA  0.95  2.62  NA  26 16 6 4 16  NA NA NA NA NA  0.34 0.70 1.03 0.99 0.65  0.93 1.93 2.86 2.74 1.81  NA 24.1 35.8 34.3 22.6  10  NA  0.67  1.86  23.2  52  NA  0.43  1.19  14.9  6  0.90  6.37  17.64  17.6  5 4 4 8 5 8 32 36  0.85 0.85 0.91 0.94 0.81 0.90 0.94 0.96  8.24 6.18 15.44 7.45 9.04 8.01 5.51 4.69  22.82 17.12 42.77 20.64 25.04 20.64 15.26 12.99  22.8 17.1 42.8 20.6 25.0 20.6 15.3 13.0  WHODAS II (0 to 100) Understanding & Communicating Getting Around Self-Care Life Activities (Non-working) Life Activities (Working) Getting Along with People Participation in Society Total Score (Non-working) Total Score (Working)  136  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; MDC, minimal detectable change; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SEM, standard error of measurement; WHODAS II, World Health Organization Disability Assessment Schedule II Notes: * the score range for the POPS objective domains varies for each domain  137  4.76 (0.47) 4.33 (0.88) 4.44 (0.76) 4.21 (0.90) 3.59 (1.17) 4.26 (0.93) 4.71 (0.58) 4.73 (0.59) 4.10 (0.84)  PM-PAC (1 to 5) Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment (n=69) Education (n=13) Economic Life Community, Social & Civic Life  -0.10 (0.91) 0.59 (1.63) -0.82 (0.48) 0.70 (2.27)  0.45 (0.63) 0.90 (0.88) 1.03 (1.11) 0.58 (0.69) 0.83 (1.00)  IPA (0 to 4) Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education (n=71)  POPS† (subjective domains -4 to 4) Objective Domestic Life Objective Major Life Areas Objective Transportation Objective Interpersonal Interactions & Relationships  Test (SD)  Instruments (score range)  -0.06 (0.93) 0.47 (1.37) -0.80 (0.50) 0.34 (1.46)  4.76 (0.43) 4.39 (0.86) 4.46 (0.78) 4.32 (0.81) 3.74 (1.10) 4.28 (0.83) 4.67 (0.54) 4.71 (0.65) 4.19 (0.80)  0.42 (0.63) 0.84 (0.88) 0.95 (1.07) 0.55 (0.65) 0.71 (0.87)  Retest (SD) 0.03 (-0.03 to 0.09) 0.05 (-0.02 to 0.12) 0.09 (-0.01 to 0.19) 0.03 (-0.04 to 0.09) 0.12 (0.005 to 0.23)  Mean Difference (95% CI)  -0.04 (-0.10 to 0.03) 0.12 (-0.04 to 0.25) -0.02 (-0.08 to 0.03) 0.36 (0.08 to 0.64)  0 (-0.07 to 0.07) -0.06 (-0.12 to 0.05) -0.02 (-0.10 to 0.06) -0.10 (-0.20 to -0.002) -0.15 (-0.28 to -0.008) -0.02 (-0.17 to 0.12) 0.04 (-0.13 to 0.21) 0.03 (-0.05 to 0.10) -0.09 (-0.17 to -0.01)  Table 4.4. Test-retest reliability for the participation instruments (n=139)  0.30 0.08 0.43 0.01*  0.99 0.07 0.66 0.05 0.04* 0.74 0.64 0.47 0.03*  0.34 0.16 0.09 0.26 0.04*  p-value  0.90 (0.87, 0.93) 0.86 (0.81, 0.90) 0.78 (0.71, 0.84) 0.61 (0.49, 0.70)  0.59 (0.47, 0.69) 0.91 (0.87, 0.93) 0.81 (0.74, 0.86) 0.76 (0.68, 0.82) 0.74 (0.65, 0.81) 0.78 (0.66, 0.86) 0.88 (0.65, 0.96) 0.77 (0.69, 0.83) 0.83 (0.77, 0.88)  0.84 (0.78, 0.88) 0.88 (0.84, 0.92) 0.85 (0.80, 0.89) 0.83 (0.77, 0.88) 0.86 (0.79, 0.91)  ICC (95% CI)  138  WHODAS II (0 to 100) Understanding & Communicating Getting Around Self-Care Life Activities (Non-working; n=53) Life Activities (Working; n=86) Getting Along with People Participation in Society Total Score (Non-working; n=53) Total Score (Working, n=86)  POPS cont. Objective Community Recreational & Civic Life Objective Participation Total Subjective Domestic Life Subjective Major Life Areas Subjective Transportation Subjective Interpersonal Interactions & Relationships Subjective Community, Recreational & Civic Life Subjective Participation Total  Instruments (score range)  17.37 (20.52) 13.29 (17.37) 20.11 (20.45) 22.74 (17.51) 13.79 (15.03)  1.11 (1.03)  1.01 (0.99)  17.60 (21.50) 12.58 (16.83) 23.29 (20.20) 24.37 (15.65) 14.97 (15.55)  0.99 (1.08)  0.93 (1.07)  8.20 (13.74) 26.96 (26.01) 9.19 (17.58) 32.02 (30.96)  0.06 (0.67) 1.24 (1.20) 0.74 (1.75) 1.29 (1.54) 1.22 (1.26)  0.17 (0.88) 1.12 (1.23) 0.51 (1.69) 1.29 (1.69) 1.16 (1.22)  9.14 (14.14) 28.99 (26.80) 9.28 (16.88) 32.64 (30.00)  0.35 (1.33)  Retest (SD)  0.52 (1.85)  Test (SD)  0.23 (-2.04 to 2.50) -0.71 (-2.85 to 1.44) 3.17 (1.33 to 5.01) 1.63 (-0.49 to 3.75) 1.18 (-0.23 to 2.58)  0.94 (-0.57 to 2.45) 2.02 (-0.10 to 3.96) 0.09 (-1.38 to 1.56) 0.61 (-5.46 to 6.69)  -0.10 (-0.20 to 0.003)  -0.06 (-0.22 to 0.10)  0.11 (0.03 to 0.18) -0.12 (-0.29 to 0.04) -0.23 (-0.49 to 0.03) -0.004 (-0.24 to 0.24) -0.06 (-0.22 to 0.10)  0.17 (-0.06 to 0.39)  Mean Difference (95% CI)  0.84 0.52 0.001* 0.13 0.10  0.22 0.04* 0.90 0.13  0.06  0.48  0.01* 0.14 0.08 0.98 0.45  0.14  p-value  0.87 (0.81, 0.92) 0.72 (0.63, 0.79) 0.85 (0.78, 0.89) 0.89 (0.82, 0.94) 0.91 (0.86, 0.94)  0.79 (0.72, 0.85) 0.90 (0.87, 0.93) 0.87 (0.83, 0.91) 0.74 (0.59, 0.84)  0.82 (0.76, 0.93)  0.61 (0.49, 0.70)  0.82 (0.75, 0.87) 0.67 (0.57, 0.75) 0.64 (0.53, 0.74) 0.63 (0.51, 0.72) 0.72 (0.63, 0.79)  0.66 (0.55, 0.74)  ICC (95% CI)  139  1.30 (0.55) 1.48 (0.73) 1.23 (0.46) 1.49 (0.72) 1.35 (0.64) 1.45 (0.70) 1.34 (0.63) 1.21 (0.59) 1.51 (0.92) 2.18 (1.52) 1.54 (0.86)  1.43 (0.93) 1.49 (0.89) 1.64 (1.16) 1.50 (0.90)  Retest (SD)  1.26 (0.50) 1.53 (0.77) 1.23 (0.54) 1.47 (0.75) 1.34 (0.63) 1.45 (0.68) 1.37 (0.67)  Test (SD)  0.21 (0.06 to 0.36) -0.01 (-0.17 to 0.15) -0.54 (-0.95 to -0.13) -0.04 (-0.15 to 0.07)  -0.04 (-0.12 to 0.03) 0.04 (-0.05 to 0.13) 0 (-0.08 to -0.08) -0.02 (-0.11 to 0.08) -0.01 (-0.11 to 0.08) 0 (-0.10 to 0.10) 0.03 (-0.05 to 0.11)  Mean Difference (95% CI)  0.006* 0.87 0.01* 0.48  0.26 0.34 1.00 0.76 0.76 1.00 0.45  p-value  0.47 (0.27, 0.67) 0.67 (0.49, 0.85) 0.56 (0.32, 0.80) 0.67 (0.55, 0.79)  0.60 (0.45, 0.76) 0.61 (0.49, 0.73) 0.54 (0.40, 0.68) 0.61 (0.49, 0.73) 0.49 (0.35, 0.63) 0.79 (0.66, 0.92) 0.65 (0.53, 0.77)  Weighted Kappa (95% CI)  Notes: * statistically significant (p<0.05) † the score range for the POPS objective domains varies for each domain  Abbreviations: IPA, Impact on Participation and Autonomy; ICC, intraclass correlation coefficient; KAP, Keele Assessment of Participation; PMPAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SD, standard deviation; WHODAS II, World Health Organization Disability Assessment Schedule II  KAP (1 to 5) Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6 Interpersonal Interactions & Relationships Economic Life Work (n=75) Education (n=39) Community, Social & Civic Life (n=101)  Instruments (score range)  140  Table 4.5. Summary* of the study results for the score distribution and reliability Criteria  IPA  KAP  PM-PAC  POPS  WHODASII  +  +  +  +++  ++  +++  NA  +++  NA  +++  2) Test-retest Reliability (ICC/weighted kappa)  ++  +  ++  +  ++  3) Test-retest Reliability (SEM/MDC)  ++  +  ++  ++  ++  Score Distribution (Floor/Ceiling) Reliability 1) Internal Consistency  Abbreviations: ICC, intraclass correlation coefficient; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; MDC, minimal detectable change; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SEM, standard error of measurement; WHODAS II World Health Organization Disability Assessment Schedule II Notes: * Ratings: +++ met criteria; ++ partially met criteria; + results primarily did not meet criteria  141  Figure 4.1. IPA mean scores and the minimal detectable change as a percentage of each domain score range  The bars in the figure illustrate the MDC as a percentage of each domain score range, where a lower score indicates better participation. Since the sample mean domain scores are low (high ratings of participation), the figure illustrates that it would be difficult to detect further improvements. Abbreviations: IPA, Impact on Participation and Autonomy; MDC, minimal detectable change Notes: * IPA domains: 1=Autonomy Indoors; 2=Family Role; 3=Autonomy Outdoors; 4=Social Life and Relationships; 5=Work and Education o mean score for the first administration of the IPA (n=139)  142  Figure 4.2. KAP mean question scores and the minimal detectable change as a percentage of each question (sub domain) score range  The bars in the figure illustrate the MDC as a percentage of each question (sub domain) score range, where a lower score indicates better participation. Since the sample mean question scores are low (high ratings of participation), the figure illustrates that it would be difficult to detect further improvements. Abbreviations: KAP, Keele Assessment of Participation; MDC, minimal detectable change Notes: * KAP questions (sub-domains): 1= Mobility #1; 2= Mobility #2; 3= Self-Care; 4= Domestic Life #4; 5= Domestic Life #5; 6=Domestic Life #6; 7=Interpersonal Interactions and Relationships; 8=Economic Life; 9=Work; 10=Education; 11=Community, Social and Civic Life o mean score for the first administration of the KAP (n=139)  143  Figure 4.3. PM-PAC mean scores and the minimal detectable change as a percentage of each domain score range  The bars in the figure illustrate the MDC as a percentage of each domain score range, where a higher score indicates better participation. Since the sample mean domain scores are high (high ratings of participation), the figure illustrates that it would be difficult to detect further improvements. Abbreviations: PM-PAC, Participation Measure-Post Acute Care; MDC, minimal detectable change Notes: * PM-PAC domains: 1= Communication; 2= Mobility; 3= Domestic Life; 4= Interpersonal Relationships; 5=Role Functioning; 6=Work and Education; 7=Education; 8=Economic Life; 9=Community, Social and Civic Life o mean score for the first administration of the PM-PAC (n=139)  144  Figure 4.4. POPS Objective and Subjective mean scores and the minimal detectable change as a percentage of each domain score range  The bars in the figure illustrate the MDC as a percentage of each domain score range, where a higher score indicates more frequent participation. Since the sample mean domain scores are low (low frequency), the figure illustrates that it would be difficult to detect further deteriorations in the domains Major Life Areas; Transportation; Interpersonal Interactions and Relationships; Community, Recreational and Civic Life Abbreviations: POPS, Participation Objective Participation Subjective; MDC, minimal detectable change Notes: * POPS domains: 1= Domestic Life; 2= Major Life Areas; 3= Transportation; 4= Interpersonal Interactions and Relationships; 5=Community, Recreational and Civic Life; 6=Objective Total Score o mean score for the first administration of the POPS (n=139) + the lowest scores for the domains are: Domestic Life (-2.22); Major Life Areas (-0.98); Transportation (-1.31); Interpersonal Interactions and Relationships (-1.59); Community, Recreational and Civic Life (-1.16); Objective Total Score (-1.45)  145  Figure 4.4. POPS Objective and Subjective mean scores and the minimal detectable change as a percentage of each domain score range cont.  The bars in the figure illustrate the MDC as a percentage of each domain score range, where a higher score indicates an important area that an individual is satisfied with the amount of activity. Since the sample mean domain scores are high (satisfied with the amount of activity in important areas) and the MDC is large in domains such as Transportation, the figure illustrates that it would be difficult to detect further improvements. Abbreviations: POPS, Participation Objective Participation Subjective; MDC, minimal detectable change Notes: * POPS domains: 1= Domestic Life; 2= Major Life Areas; 3= Transportation; 4=  Interpersonal Interactions and Relationships; 5=Community, Recreational and Civic Life; 6=Subjective Total Score o mean score for the first administration of the POPS (n=139)  146  Figure 4.5. WHODAS II mean scores and the minimal detectable change as a percentage of each domain score range  The bars in the figure illustrate the MDC as a percentage of each domain score range, where a lower score indicates better participation. Since the sample mean domain scores are low (high ratings of participation), the figure illustrates that it would be difficult to detect further improvements. Abbreviations: MDC, minimal detectable change; WHODAS II, World Health Organization Disability Assessment Schedule II Notes: * WHODAS II domains: 1= Understanding and Communicating; 2= Getting Around; 3= Self-Care; 4= Life Activities (Non-working); 5=Life Activities (Working); 6=Getting Along with People; 7=Participation in Society; 8=Total Score (Non-working); and 9=Total Score (Working) o mean score for the first administration of the WHODAS II (n=139)  147  4.5 References 1. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization, 2001. 2. Whiteneck GG, Charlifue SW, Gerhart KA et al. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil 1992;73:519-26. 3. Carr AJ, Thompson PW. Towards a measure of patient-perceived handicap in rheumatoid arthritis. Br J Rheumatol 1994;33:378-82. 4. Cardol M, de Haan RJ, van den Bos GA et al. The development of a handicap assessment questionnaire: the Impact on Participation and Autonomy (IPA). Clin Rehabil 1999;13:411-9. 5. Wilkie R, Peat G, Thomas E et al. The Keele Assessment of Participation: a new instrument to measure participation restriction in population studies. Combined qualitative and quantitative examination of its psychometric properties. Qual Life Res 2005;14:1889-99. 6. Cardol M, de Haan RJ, de Jong BA et al. Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Arch Phys Med Rehabil 2001;82:210-6. 7. Gandek B, Sinclair SJ, Jette AM et al. Development and Initial Psychometric Evaluation of the Participation Measure for Post-Acute Care (PM-PAC). Am J Phys Med Rehabil 2006;86:57-71.  148  8. Brown M, Dijkers MP, Gordon WA et al. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004;19:459-81. 9. World Health Organization. WHODAS II. [http://www.who.int/icidh/whodas]. November 2001; Accessed March 10, 2006. 10. Gray DB, Hollingsworth HH, Stark SL et al. Participation Survey/Mobility: psychometric properties of a measure of participation for people with mobility impairments and limitations. Arch Phys Med Rehabil 2006;87:189-97. 11. Van Brakel WH, Anderson AM, Mutatkar RK et al. The Participation Scale: Measuring a key concept in public health. Disabil Rehabil 2006;28:193-203. 12. Ostir GV, Granger CV, Black T et al. Preliminary results for the PAR-PRO: a measure of home and community participation. Arch Phys Med Rehabil 2006;87:1043-51. 13. Haley SM, Gandek B, Siebens H et al. Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes. Arch Phys Med Rehabil 2008;89:275-83. 14. Sandstrom M, Lundin-Olsson L. Development and evaluation of a new questionnaire for rating perceived participation. Clin Rehabil 2007;21:833-45. 15. Pallant JF, Misajon R, Bennett E et al. Measuring the impact and distress of health problems from the individual's perspective: development of the Perceived Impact of Problem Profile (PIPP). Health Qual Life Outcomes 2006;4:36.  149  16. Fitzpatrick R, Davey C, Buxton MJ et al. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2:i-74. 17. Posl M. Evaluation of the World Health Organization Disability Assessment Schedule II (WHO DAS II) - German Version. 1-62. 2004. Ludwig Maximilian University. 18. Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11:193-205. 19. Beckerman H, Roebroeck ME, Lankhorst GJ et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 2001;10:571-8. 20. de Vet HC, Terwee CB, Knol DL et al. When to use agreement versus reliability measures. J Clin Epidemiol 2006;59:1033-9. 21. Stratford P. Reliability: consistency or differentiating among subjects? Phys Ther 1989;69:299-300. 22. Terwee CB, Bot SD, de Boer MR et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34-42. 23. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 2 ed. Oxford: Oxford University Press, 1995. 24. Marino RJ, Barros T, Biering-Sorensen F et al. International standards for neurological classification of spinal cord injury. J Spinal Cord Med 2003;26 Suppl 1:S50-S56.  150  25. Sangha O, Stucki G, Liang MH et al. The Self-Administered Comorbidity Questionnaire: a new method to assess comorbidity for clinical and health services research. Arthritis Rheum 2003;49:156-63. 26. Dijkers M. Comments on van Brakel et al.'s Participation Scale. Disabil Rehabil 2006;28:1360-2. 27. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420-8. 28. Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. 1 ed. Chichester: John Wiley and Sons, 2000. 29. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135-60. 30. Streiner DL. Being inconsistent about consistency: when coefficient alpha does and doesn't matter. J Pers Assess 2003;80:217-22. 31. Franchignoni F, Ferriero G, Giordano A et al. Rasch psychometric validation of the Impact on Participation and Autonomy questionnaire in people with Parkinson's disease. Eura Medicophys 2007;43:451-61. 32. Lund ML, Fisher AG, Lexell J et al. Impact on participation and autonomy questionnaire: internal scale validity of the Swedish version for use in people with spinal cord injury. J Rehabil Med 2007;39:156-62.  151  33. Middelkamp W, Moulaert VR, Verbunt JA et al. Life after survival: long-term daily life functioning and quality of life of patients with hypoxic brain injury as a result of a cardiac arrest. Clin Rehabil 2007;21:425-31. 34. Sibley A, Kersten P, Ward CD et al. Measuring autonomy in disabled people: Validation of a new scale in a UK population. Clin Rehabil 2006;20:793-803. 35. Van Genderen FR, Van Meeteren NL, Heijnen L et al. The use of a disability model in haemophilia research. Haemophilia 2005;11:472-80. 36. Dijkers MP, Whiteneck G, El Jaroudi R. Measures of social outcomes in disability research. Arch Phys Med Rehabil 2000;81:S63-S80. 37. Chisolm TH, Abrams HB, McArdle R et al. The WHO-DAS II: psychometric properties in the measurement of functional health status in adults with acquired hearing loss. Trends Amplif 2005;9:111-26. 38. McKibbin C, Patterson TL, Jeste DV. Assessing disability in older patients with schizophrenia: results from the WHODAS-II. J Nerv Ment Dis 2004;192:405-13. 39. Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clin Rehabil 1998;12:187-99. 40. de Boer MR, de Vet HC, Terwee CB et al. Changes to the subscales of two visionrelated quality of life questionnaires are proposed. J Clin Epidemiol 2005;58:1260-8.  152  41. de Vet HC, Bouter LM, Bezemer PD et al. Reproducibility and responsiveness of evaluative outcome measures. Theoretical considerations illustrated by an empirical example. Int J Technol Assess Health Care 2001;17:479-87.  153  5 COMPARING THE VALIDITY OF FIVE PARTICIPATION INSTRUMENTS4 5.1 Introduction It is increasingly recognized that a person’s ability to participate in life situations is an important rehabilitation outcome that needs to be measured.1 The World Health Organization’s revised model of disability, the International Classification of Functioning, Disability and Health (ICF) includes participation as one of the three major components that comprise functioning and health.2 Participation is defined as “the involvement in life situations” and participation restrictions reflect the problems that an individual may experience in those life situations.2 The predecessor of the ICF, the International Classification of Impairments, Disabilities and Handicaps (ICIDH) included the term handicap. Handicap is defined as “the disadvantage of a given individual resulting from an impairment or a disability, that limits or prevents the fulfillment of a role that is normal (depending on age, sex, and social and cultural factors) for that individual” (page 182)3. The comparison to ‘normal’ used in the ICIDH was based on the medical model. This did not reflect the advances offered by social models of disability which view that disability as resulting from problems within society and not within the individual. Because the instruments developed to assess handicap4;5 were no longer congruent with the ICF, which combines aspects of both the medical and social models of disability2, new instruments were needed. A recent review of the literature (Chapter 2) identified eleven instruments which were developed using the ICF. Although there has been tremendous progress in developing new instruments to measure the concept of participation, it is currently not known how the 4  A version of this chapter will be submitted for publication. Noonan VK, Kopec JA, Noreau L, Singer J, Mâsse LC, Zhang H, Dvorak MF. Assessing the Validity of Five Participation Instruments in Persons with Spinal Conditions: II. How do they Compare?  154  instruments compare. One of the recommendations from the review (Chapter 2) was that more direct comparisons of existing instruments measuring participation are needed. Recently the content validity (Chapter 3) and the reliability (Chapter 4) of five participation instruments included in this review were evaluated. The five instruments include Impact on Participation and Autonomy (IPA)6, Keele Assessment of Participation (KAP)7, Participation Measure-Post Acute Care (PM-PAC)8, Participation Objective and Participation Subjective (POPS)9, and World Health Organization Disability Assessment Schedule II (WHODAS II)10. Next the construct validity of these five instruments needs to be assessed. Validity assesses whether the instrument measures what it intends to measure.11 Validity is not a property of an instrument, but rather it is the meaning or interpretation that can be derived from the instrument scores for a particular purpose.12;13 Face, content and construct validity are commonly evaluated in self-reported instruments.14 Face validity examines whether the instrument appears to measure what it intends to measure and content validity assesses how well the questions cover the health components being assessed.14 Construct validity evaluates the theoretical relationship of the questions to each other and to hypothesized scales.15 Specifically, construct validity assesses whether the questions all measure one underlying construct in a domain, which is referred to as dimensionality15 or evidence based on internal structure. Construct validity also includes examining relationships between hypothesized similar or dissimilar domains in other instruments, referred to as convergent or discriminant validity15. Relationships can also be examined between groups of individuals based on sociodemographic variables such as age or clinical variables such as diagnosis, which is referred to as known-group validity.15  155  To date, there have been reviews and assessments of the content and face validity (Chapters 2 and 3) but little is known regarding how the construct validity of participation instruments compare empirically. Comparing the construct validity of instruments, all purporting to measure participation, will help to determine if underlying constructs considered to comprise the concept of participation are unidimensional. It will also help elucidate whether differences in how this concept is operationalized (e.g. asking about difficulties, limitations or frequency) are captured in the resulting domain scores. In addition, comparisons to other instruments measuring concepts such as health status in disease-specific instruments or quality of life (also referred to as subjective well-being) in life satisfaction instruments will further enhance our understanding of how participation is conceptualized and operationalized. Since several instruments have been developed, results from this study will assist clinicians and researchers in selecting an instrument. The purpose of this paper was, therefore, to evaluate the construct validity of five participation instruments. Specifically, the unidimensionality, convergent/discriminant validity and known-group validity was assessed.  5.2 Methods 5.2.1 Recruitment and Study Procedures A retrospective review of the spine database for the Acute Spine Program at Vancouver General Hospital (VGH), was performed to identify individuals who were admitted between 2001 and March 2005 with a diagnosis of 1) traumatic or non-traumatic spinal cord injury (SCI), 2) a spinal column facture without neurological involvement, or 3) a spinal degenerative disease (stenosis, disc degeneration, spondylolisthesis, spondylosis). Individuals were excluded if they were deceased; could not be contacted; did not speak 156  English; had a cognitive deficit; were not able to physically complete the instruments (e.g. age, a person with a SCI who is ventilator dependent); or were discharged from hospital within the last three months and were not able to do regular activities (e.g. if bed rest was prescribed due to a pressure sore). A sample size of 200 individuals per diagnostic group was targeted and eligible individuals were randomly selected until the sample size was achieved. The study was approved by the Behavioural Research Ethics Board at the University of British Columbia (Appendix A) and all individuals provided written informed consent. All potentially eligible individuals were mailed a questionnaire and were contacted approximately one week later to determine if they received it. A reminder letter and another copy of the questionnaire was sent if individuals mentioned they were interested in participating in the study but had not sent in their completed questionnaire. The study coordinator reviewed the returned questionnaires to check if there was missing data. If there were sections with missing answers then the study coordinator contacted the subjects and asked if they would be willing to answer the questions. If however, subjects indicated in the questionnaire that they did not wish to be contacted then no further contact was attempted and no subjects were asked to answer questions related to income or intimate relationships.  5.2.2 Data Elements Data was obtained from hospital databases and from a questionnaire completed by the respondents and included clinical data, sociodemographic/socioeconomic data and domain/total scores from participation instruments.  157  5.2.2.1 Clinical Data Information on diagnoses was obtained from hospital databases. For individuals with SCI, the motor score from the first acute neurological assessment (International Standards for the Neurological Classification of SCI) was recorded.16 Comorbidities were assessed using one section of the Self-Administered Comorbidity Questionnaire17 which asks about the presence or absence of 14 comorbid conditions. One point is assigned for each comorbid condition producing a maximum score of 14 points.  5.2.2.2 Sociodemographic and Socioeconomic Data The sociodemographic variables collected were age, gender, marital status, racial background and living support at the time of follow-up. The city where the person lived was used to classify the location as either an urban or rural setting using the methodology from Statistics Canada18. Socioeconomic variables collected from participants at the time of follow-up included education, employment and compensation status.  5.2.2.3 Participation Instruments Five instruments designed to assess participation based on the ICF were selected as previously described in Chapter 4. The IPA6 assesses the person’s perception of how health impacts participation and autonomy in the domains autonomy indoors (e.g. self-care); family role (e.g. housework); autonomy outdoors (e.g. visiting friends, leisure time); social life and relationships; and work and education. Based on 31 questions, the perceived participation score was calculated for each domain, where a lower score indicates better perceived participation. Since this study commenced before the publication by Sibley et al.19, which  158  suggested adding an additional question on ‘helping or supporting people’, it was not possible to evaluate this slightly revised version. The KAP7 is a participation measure that contains 11 questions asking about autonomy in conducting life activities. It covers mobility (2 questions); self-care; domestic life (3 questions); interpersonal interactions and relationships; major life areas (3 questions); and community, social and civic life. The mean response for each question was calculated in order to compare the domains within the participation instruments and a lower score indicates better perceived participation. Participation Measure – Post Acute Care 8 assesses participation in the community. It contains 51 questions and 42 questions are used to create the domains communication; mobility; domestic life; interpersonal relationships; role functioning; work and employment; education; economic life; and community, social and civic life. A higher score indicates better participation. The POPS9 assesses participation in 26 life activities in both an objective (frequency) and a subjective manner (importance and level of satisfaction) . A scoring algorithm provided by the developers was used to calculate the overall subjective and objective scores as well domain scores for domestic life; major life areas; transportation; interpersonal interactions and relationships; and community, recreational and civic life. Objective scores are based on z scores which represent the difference between the frequency for each question compared to reference data from a sample including persons with traumatic brain injury and healthy controls. The scores are weighted based on the perceived importance in the reference sample. A higher score indicates a greater frequency compared to reference data, weighted based on importance. Subjective scores are obtained by multiplying the individual’s  159  importance score by the satisfaction score and range from -4 (important area that a person wants to do more or less of) to +4 (important area that a person is satisfied with the amount of activity). The POPS was originally developed to be interviewer-administered. A selfadministered version was developed in consultation with the instrument’s authors and piloted so it could be used in this study. The WHODAS II10 assesses daily functioning using domains covered in the activities and participation component of the ICF. There are 36 questions and the domains include understanding and communicating, getting around, self-care, getting along with people, life activities (household/work activities), and participation in society. The instrument was scored using a scoring algorithm provided by the World Health Organization. Separate scores were calculated for individuals who were working and not working for the life activities domain and the total score since four questions related to work/school were only applicable if a person worked or attended school. Scores were calculated for each domain and a lower score indicates better participation.  5.2.2.4 Disease-Specific Instruments The Neck Disability Index (NDI)20 was used for individuals with either a degenerative or spinal column injury of the cervical spine. It contains 10 questions and measures interference in activities due to cervical pain. The response options range from 0 to 5 and a lower score indicates less disability. The NDI has been used in individuals with a variety of cervical spine conditions.21;22 The Oswestry Disability Questionnaire (ODQ) Version 2.0 23 was used for individuals with a degenerative or spinal column injury of the thoracic or lumbar spine. The ODQ assesses interference in activities due to back pain. It contains 10 questions related to pain, 160  personal care, lifting, walking, sitting, standing, sleeping, sex life, social life and travelling. The response options range from 0 to 5 and a lower score indicates less disability. The ODQ has been used in individuals with spinal column and spinal degenerative conditions.24 The Self-Reported Functional Measure (SRFM)25;26 was used for individuals with traumatic SCI and non-traumatic SCI. This instrument was designed to assess physical functioning in individuals with SCI and is a self-report measure based on the Functional Independence Measure27. The SRFM response options range from 1 to 4 and a lower score indicates greater independence (less reliance on assistance).  5.2.2.5 Subjective Well- Being Instrument The Life Satisfaction 11 (LiSat-11)28 assesses quality of life, also referred to as subjective well-being. The questions ask about satisfaction in life domains including satisfaction with life in general, vocation, financial situation, leisure; social/friends/family, sexual life, family life, physical health and mental health. Each question has a 6 point response scale ranging from very satisfying to very dissatisfying and a lower score indicates a higher quality of life. The LiSat-11 has been used in individuals with SCI29 as well as individuals in the general population28;30.  5.2.3 Statistical Analysis The aspects of construct validity assessed included dimensionality, convergent and discriminant validity, and known-group validity. Dimensionality was assessed by evaluating the item-to-scale correlations and by conducting a confirmatory factor analysis (CFA). The item-to-scale correlations are corrected for overlap by removing the question from the scale when calculating the total score and a correlation of >0.40 is recommended15;31, which is 161  more rigorous than a minimum value of 0.20 recommended by others14;32. In addition, the question should have a higher correlation with the domain (scale) it belongs to as compared to the other domains. A summary of the number of correlations where the correlation between the questions and their hypothesized domains were higher compared to the other domains was calculated. The CFA analysis partitions out the unique variance that is not accounted for by the latent factor and produces an estimate of the proportion of the variance of the indicator (question) which is explained by the latent factor33; this is not accounted for in estimates of item-toscale correlations. Since the purpose of using CFA was to test if the proposed factor structure fit in this study sample (referred to as a strictly confirmatory approach) no modifications were made to the models except for allowing correlated errors within a factor and not across factors.34. Correlated errors can result from method effects (e.g. similarly worded questions resulting in a high correlation between responses) Since the response options in the instruments’ scales were 5 points, the data was considered to be continuous and robust maximum likelihood estimation was used to account for the non-normal data distribution.35 All analyses were conducted using Lisrel 8.08 (Scientific Software International, 1996). Model fit was evaluated using three fit indices. The root mean square error of approximation (RMSEA) is an index of population discrepancy and considers the complexity of the model.33 It has been suggested that a RMSEA value less than 0.05 is considered to be a close fit and an upper value of 0.080 is considered reasonable fit.36 The comparative fit index (CFI) is an incremental fit index and a value near 1.0 indicates a close fit of the data to the model and values close to or greater than 0.95 are recommended.37 The standardized root mean residual (SRMR) is an absolute fit  162  index and values less than 0.08 are recommended.37 To maximize the sample size for the CFA the domains associated with work and education were not included since these domains were not applicable to many subjects in the sample. In the PM-PAC four questions in the community, social and civic life domain (10e, 10g, 10h, 10i,) and two questions in the communication domain (10j, 10k) were not included due to the large percentage of missing data from having a ‘not applicable/don’t do’ response and this data was not imputed. These same questions were not included in the initial CFA published for the PM-PAC8, with the exception of question 10k. Convergent and discriminant validity were assessed by examining 1) the association between similar participation domains among the instruments, 2) the associations between participation domains and scores from disease-specific instruments and 3) associations between domains in participation instruments and questions in the LiSat-11. Correlations were assessed using Spearman rho. Values greater than or equal to +0.70 were considered strong, values between +0.50 to +0.69 were considered moderate, values between +0.31 to +0.49 were considered fair and values less than +0.30 were considered weak.38 It was hypothesized that similar domains among the participation instruments would have a strong or moderate correlation, with the exception of the POPS objective domains scores where only a fair or weak correlation was expected. For the disease-specific instruments, higher correlations were expected in the domains related to mobility, self-care, domestic life and major life areas (work and/or school) and lower correlations were expected in domains related to communication, interpersonal interactions and relationships as well as economic life. Since the POPS measures frequency (objective POPS domains) and satisfaction/importance (subjective domains), lower correlations were expected between the  163  POPS and the disease-specific instruments. In terms of the LiSat-11, higher correlations were expected between the participation domains and questions in the LiSat-11 containing similar content. Specifically, a strong to moderate correlation was expected between the participation domains related to interpersonal interactions and relationships and the LiSat-11 question asking about satisfaction with the amount of contact with friends and acquaintances. A fair to weak correlation was expected between the participation domains related to interpersonal interactions and relationships and the LiSat-11 question asking about satisfaction with the person’s financial situation. Relationships between the participation domains and other study variables were hypothesized to assess known-group validity. The clinical variables tested to determine their relationship with participation scores included motor score (SCI group), traumatic versus non-traumatic SCI, and level of injury (spinal column and spinal degenerative group). It was hypothesized that scores in participation domains would be lower in subjects with lower motor scores (greater impairment) in the domains related to mobility, self-care, domestic life, and community, social and civic life but not for communication domains. No differences in scores were expected for various levels of injury (cervical versus lumbar) or for traumatic versus non-traumatic SCI. The patient-reported variable describing back pain was assessed. It was expected that individuals reporting more severe back pain (using question one from the ODQ) would have lower participation in domains related to interpersonal interactions and relationships as well as community, social and civic life. A similar analysis was conducted assessing individuals reporting moderate pain on the ODQ using the same questions. Finally, the impact of the demographic variables age and gender were assessed. It was expected that individuals over the age of 65 would have worse participation in domains related to mobility,  164  self-care, and domestic life but not in domains related to interpersonal interactions and relationships. No difference was expected in domains related to mobility or community, social and civic life based on gender. Since there are differences in instruments not all domains were included in the hypothesis testing; for example, the POPS mobility domain was not included in the hypothesis regarding motor scores since the POPS only assesses transportation rather than physical mobility. Hypotheses were tested using either linear or ordinal regression with backward stepwise variable selection to adjust for relevant covariates. A univariate analysis was first conducted and then an adjusted effect was estimated controlling for clinical (motor score, diagnosis), sociodemographic variables (age, gender, marital status, living support) and socioeconomic factors (employment, compensation) depending on what was relevant in each analysis. A p-value <0.05 was considered statistically significant. A hypothesis was considered to be supported if the effect was statistically significant in both the unadjusted and adjusted analysis and in the correct direction (increase or decrease in score as expected). An index was created for each instrument comparing the number of hypotheses supported out of the total number assessed. It has been recommended that 75% of hypotheses should be supported.39 The percentages of missing data for the participation instruments (IPA, KAP, PM-PAC, POPS, WHODAS II) were all less than 10%. For the disease-specific instruments (NDI, ODQ, SRFM) and quality of life (LiSat-11) the amount of missing data at the question-level was also less than 10%, except for the SRFM where it was 10.3% to 11.7% since 12 subjects received the wrong version of the questionnaire (received the questionnaire containing the ODQ or NDI instead of the SRFM). For the overall sample (n=545), missing data was imputed using the multiple imputation procedure in SAS (PROC MI) except only one  165  additional data set was created. Variables potentially related to the reason why the data was missing and also variables known to be associated with the participation scores were included in the model. Data pertaining to work and education was only imputed for individuals who indicated that they were involved in these activities and data from the other instruments was not used in the imputation (did not use responses from other instruments). Additional information on the amount of missing data at the question- and subject-level as well as the type of missing data and the methods used to impute the data are described in Appendix B.  5.3 Results 5.3.1 Recruitment A total of 545 individuals participated in the study. There were 145 in the SCI group, 187 in the spinal column group and 213 in the spinal degenerative group. The response rates for all eligible individuals in each of the groups were 59% (145/246) in the SCI group, 58% (187/320) in the spinal column group and 62% (213/345) in the spinal degenerative group. The overall response rate was 60% (545/911). Individuals were contacted approximately 4 years following discharge from VGH.  5.3.2 Subject Characteristics A description of the sample is provided in Table 5.1. Sixty-seven percent of the sample was male (367/545). The mean age and standard deviation (SD) at the time of follow-up was 51.5 (16.6) years. A comparison of individuals who participated in this study and those who were eligible, but did not participate revealed that individuals in the sample were older (46.97 versus 39.98 years) on admission to VGH and there were fewer men (67% versus 73%). 166  5.3.3 Descriptive Statistics for the Instruments Results for the five participation instruments are described in Table 5.2. A comparison of the mean and median values for the participation instruments revealed that the data was not normally distributed. In terms of the disease-specific instrument, the SRFM score and SD was 1.72 (0.71) for the SCI group. Data on the ODQ were available for 272 individuals from the spinal column and spinal degenerative group and the mean and SD was 1.14 (0.89). The mean NDI score was 1.13 (0.84) (n=128). Additional data in each of the three groups for the participation instruments is outlined in Appendix C, Table C.1 and in Appendix D, Table D.1 for the disease-specific and LiSat-11 instruments.  5.3.4 Unidimensionality Dimensionality was assessed in three of the five instruments. It was not assessed in the KAP or the POPS. Results from the KAP are reported using individual questions or by determining the number of participation restrictions (with each question dichotomized into yes or no) and so it is unlikely the KAP was intended to be unidimensional .7 In the POPS scoring algorithm, the questions included in the domains are not necessarily intended to be related but instead comprise an index, often referred to as a clinimetric approach40. The item-to-scale correlations were all greater than 0.40 in the IPA, PM-PAC and WHODAS II, which suggests that the questions were strong indicators of the domains (see Table 5.3). Results also indicated that there were questions which had stronger correlations to other domains as opposed to their own domain. In both the IPA and the WHODAS II the questions asking about sexual or intimate relationships, which are part of domains assessing interpersonal relationships, correlated with domains assessing community, social and civic 167  life as well as work/education. In the PM-PAC, the education questions had strong correlations with questions from other instrument domains; for example, the question 2a ‘getting around your home’ which had an item-to-scale correlation of 0.72 in the mobility domain, also had a similar correlation with the education domain. A CFA was conducted on the IPA, PM-PAC and WHODAS II but not on the KAP and POPS for reasons described above. A first-order CFA model was assessed in the IPA and PM-PAC and a second-order CFA model was assessed in the WHODAS II. Overall the models demonstrated adequate fit. All of the models had a RMSEA less than 0.08, including the 90% CI but only the PM-PAC had a value less 0.05, including the lower 90% CI. The CFI indices ranged from 0.98 to 0.99, suggesting good fit. The SRMR values ranged from 0.060 in the IPA to 0.085 in the WHODAS II. Three correlated error terms were added within a factor for the IPA and the WHODAS II models whereas only one was added in the PM-PAC, suggesting the PM-PAC had a superior factor structure in this sample compared to the other instruments. Additional results describing the unadjusted and adjusted models are included in Appendix D for the IPA (Figures D.1a and D.1b, Tables D.2a and D.2b), PMPAC (Figures D.2a and D.2b, Tables D.3a and D.3b) and the WHODAS II (Figures D.3a and D.3b, Tables D.4a and D.4b). The standardized factor loadings were all greater than 0.40, suggesting adequate loading on the hypothesized factor (see Table 5.4).33 There were just three questions which had a standardized loading <0.60. In the PM-PAC two questions in the communication domain, including ‘reading books’ and “watching or listening to television and radio” had standardized loadings of 0.57 and 0.56 respectively. The WHODAS II had one question, asking about difficulty with ‘sexual activities’ which had a loading of 0.53.  168  5.3.5 Convergent/Discriminant Validity The correlations among similar participation domains are summarized in Table 5.5 and the actual values are provided in Appendix D (Table D.5). Overall, correlations were generally higher among the WHODAS II, IPA, KAP and PM-PAC. As expected the lowest correlations were observed between the objective domains of the POPS and the other instruments. Correlations were lower than expected between the subjective POPS domains and the other participation domains in the IPA, KAP, PM-PAC and WHODAS II. Correlations between the participation domains and the disease-specific instruments generally supported our hypotheses. Higher correlations were observed between the diseasespecific instruments and the domains related to mobility, self-care, domestic life, work or education and community, social and civic life (Table 5.6). Overall, correlations were highest among the ODQ and the participation domains. The association between the participation domains and questions in the LiSat-11 measuring similar content was as expected (Table 5.7), except a higher correlation was observed among the PM-PAC domain economic life and the LiSat-11 question asking about satisfaction with finances (rho= -0.51). The values for the correlations between the participation instruments and the LiSat-11 are provided in Appendix D (Table D.6).  5.3.6 Known-Group Validity The known-group validity indices (# of hypotheses supported/# hypotheses tested) for each the participation instruments were: IPA=95.2% (20/21); KAP=77.4% (24/31); PMPAC=94.4% (17/18); POPS=75.0% (24/32); and WHODAS II=84.0% (21/25). None of the participation instruments demonstrated any statistically significant differences related to 169  traumatic versus non-traumatic SCI or for level of injury in the spinal column and spinal degenerative groups as expected. The IPA, PM-PAC and WHODAS II were able to detect differences based on motor score for the SCI group in the domains related to mobility, selfcare, domestic life (working group only) and community, social and civic life but not in domains related to communication (PM-PAC, WHODAS II) as expected. The POPS was not able to detect as many changes in participation due to back pain in the domains interpersonal interactions and relationships or community, social and civic life as compared to the other participation instruments. Differences due to age were not detected in domains related to mobility (KAP), self-care (KAP, WHODAS II) or domestic life (KAP, PM-PAC, POPS subjective, WHODAS II) which was unexpected. The IPA was the only instrument to detect differences due to age for interpersonal interactions and relationships (domain social life and relationships), which was also unexpected. For additional details on the hypothesis testing see Appendix D (Table D.7).  5.4 Discussion The purpose of this study was to compare the cross-sectional construct validity of five participation instruments. Results from this study indicate that given the challenges in measuring a multi-dimensional concept such as participation these instruments demonstrate good construct validity in individuals with spinal conditions. The POPS instrument performed differently compared to the other four instruments. This is not surprising since it is the only instrument to include objective domains and the subjective domains combined ratings of importance and satisfaction. In reviewing these results it is important to consider that the measurement of participation is in the developmental stages and as a result there is no gold standard that can be used as a comparison. Instead these results may help explain if 170  these instruments are measuring similar or different things. A summary of the results is provided in Table 5.8. Unidimensionality was assessed by examining the item-to-scale correlations by conducting a CFA on the IPA, KAP and the WHODAS II. All the instruments demonstrated good item-to-scale correlations. For the IPA, results in this study were generally better than those reported by Sibely et al.19. The question regarding ‘spending my own money’ had an item-to-scale correlation of only 0.34 whereas in our study a value of 0.65 was obtained. The results from the two studies may be due to differences in the percentage of women in the study by Sibley et al.19 and our study (58% versus 33%). It is interesting that in two instruments (IPA, WHODAS II), the question asking about sexual/intimate relationships had cross-correlations with domains related to community, social and civic life as well as work. Based on other studies7;19 it is not surprising that areas of participation overlap. A recent study by Anderson et al.41 reported that sexual function is a priority for individuals living with SCI and has a tremendous impact on their life which further supports the need to include these types of questions. Since there is only one question included in each instrument it is not possible to develop a separate domain. The measurement properties of questions asking about sexual relationships should be assessed in individuals with different types of health conditions before suggesting any changes. Results from the CFA provided additional information pertaining to the factor structure. Confirmatory factor analysis is recommended over exploratory factor analysis when the factor structure has been established since it enables the hypothesized factor structure to be tested empirically.33 In this study the standardized factor loadings for the PM-PAC were similar to the results reported by Gandek et al.8. The lowest factor loading (0.53) was for a  171  question in the community, social and civic life domain whereas in our study it was for the question ‘watching or listening to television and radio’ (0.56). Sibley et al.19 conducted a CFA on the IPA and these authors reported seven factor loadings less then 0.60 whereas in our study all the factor loadings were greater than 0.63. In the studies by Gandek et al.8 and Sibley et al.19 the sample sizes were 291 and 213, respectively, compared with 545 in our study; therefore, our study may provide more robust estimates. Finally, a second-order factor model was tested for the WHODAS II10 and so the same analysis was done in our study. It is difficult to compare the results since very few details of the analysis are provided for the WHODAS II and the number of questions in the domains differs. The higher-order factor in the WHODAS II is a general disability factor and not a participation factor since this instrument proposes that communication, mobility and self-care are part of activity and not participation.10 In the PM-PAC, however, aspects of communication and mobility are considered to be part of participation. Currently, much debate exists around what domains are included in activity versus participation42-44 and so it is not surprising that such differences exist. The WHODAS II was the only instrument in which a second-order factor structure was used, which proposes that the general disability factor accounts for the interrelationships among the factors. In both the IPA and PM-PAC the relationships among the factors are not specified and no higher-order factor related to participation is proposed so dimensionality is assessed only within the domains. Convergent validity was assessed in this study by examining the relationship between similar domains. Since all the instruments used the ICF as a conceptual model, in this study we mapped domains within the instruments to the ICF chapter headings, also referred to as ICF domains. Overall the correlations were strong (rho > +0.70) to moderate (+0.50 to  172  +0.69) between similar domains within the IPA, KAP, PM-PAC and WHODAS II. Since the IPA and KAP are both designed to assess autonomy in participation it was interesting that the correlations between these two instruments were not higher in comparison to the others such as the WHODAS II which asks about difficulty in the various aspects of participation. For example, the IPA domain autonomy indoors, which covers questions related to self-care, had a correlation of rho=0.63 with the KAP, but a correlation of rho=0.66 was observed between the IPA and WHODAS II. The KAP only has one question on self-care compared to seven in the IPA and four in the WHODAS II, and so the use of broad or general questions may explain the lower correlation. Results from this study also highlight the importance of considering the content of the questions contained within domains. For example, the PM-PAC and the WHODAS II both have domains assessing aspects of communication and the correlation between these two domains was lower than expected, rho=(-0.46). However, on further inspecting the individual questions comprising these domains, it was apparent that the questions ask about different things. In the WHODAS II, the questions are related to comprehension and having conversations whereas in the PM-PAC it includes questions asking about keeping in touch with others as well as reading books. Given the different examples provided in these two instruments, it is not unexpected that the correlation was only moderate. Similarly, the way in which participation was operationalized greatly impacted the relationships between similar domains. In the POPS it was expected that objective assessment of participation would not correlate highly with subjective estimates based on previous studies9, however, it was surprising the correlations were not higher among the subjective domain of the POPS and the other instruments. The POPS subjective domains  173  include a rating of satisfaction that is weighed by importance which may explain the unexpected results. Overall the correlations between the subjective domains in the POPS had a fair correlation (+0.31 to +0.49) with the other instruments. There were a few associations that were above rho=0.50, for example, the domain assessing interpersonal, interactions and relationship in the POPS and PM-PAC had a correlation of rho=(-0.52). On examining the questions in these domains both ask about satisfaction with relationships (the POPS assessment of satisfaction is weighted by importance), which explains why a higher correlation was observed. These results reinforce that in evaluating the construct validity of an instrument it is important to consider both the content and how the questions are asked since these can greatly affect the observed relationship. Results from this study also suggest that it may be important to distinguish between difficulty/limitations/autonomy and satisfaction/importance when measuring participation, which is done in the Life-Habits instrument, since both provide different information45;46. The relationships between the participation domains and instruments measuring health status were also examined. As expected, higher correlations were observed between domains assessing mobility, self-care, domestic life, and major life areas (work and/or school) and lower correlations were observed with domains assessing communication, interpersonal interactions and relationships, and economic life. Correlations were lowest between the disease-specific instruments and POPS as expected. Overall, the correlations were higher for the ODQ and the participation instruments as compared to the SRFM and NDI. Questions in the SRFM primarily focus on assistance required for self-care and mobility and so as expected, higher correlations were observed with similar participation domains (convergent validity) and lower correlations were observed between dissimilar domains (discriminant  174  validity) such as interpersonal relationships. To our knowledge, the disease-specific instruments used in this study have not been previously compared to participation instruments. The ODQ and NDI measure pain and assess the effect of pain (a body function in the ICF) on aspects of participation. The SRFM assesses the need for assistance, which is considered an environmental factor in the ICF, on aspects of mobility and self-care. So it is possible that the disease-specific instruments assess more the influence of other ICF components (e.g. ICF component body functions) on participation. The participation instruments seem to be more ‘pure’ measures assessing participation directly and have a broader coverage of domains considered to be part of participation. More work is needed to further clarify the concepts of health status and participation and inform users which instrument(s) is best for what purpose. In terms of the correlations between the five instruments measuring participation and the LiSat-11 which measures quality of life, as expected higher correlations were observed between similar content areas (interpersonal interactions and relationships) and lower correlations between different content areas. It is interesting to note that none of the correlations were strong (>0.70), even with the POPS subjective domains which combines questions on the importance and satisfaction. In the POPS, since the rating of importance (range 0 to 4, with a higher number indicating an important area to a person’s satisfaction with life) is weighted by how satisfied a person is with the amount of activity (multiplied by 1 if dissatisfied and +1 if satisfied), the importance factor weights the response more than satisfaction, which may explain why higher correlations were not observed. The assessment of known-group validity was the final aspect of cross-sectional construct validity assessed. All the instruments included in this study met the criteria of having 75% of  175  the study hypotheses supported. The IPA had the greatest number of hypotheses supported 95.2% and the POPS had the lowest (75%). The PM-PAC had very similar results to the IPA, with 94.4% of all hypotheses supported. Other studies have also reported fewer hypotheses supported than expected using the POPS.9;47 As mentioned previously, the POPS operationalizes participation differently compared to the other instruments and this must be considered when interpreting these results. In measuring participation, it is important to consider not only if the person is able to do it but also his or her interests and values.48;49 As a result it has been suggested that optimal participation may vary for different individuals.49 If participating in leisure activities is not important to an individual, then he or she may or may not report having difficulty in leisure and scores on instruments such as the WHODAS II will vary. However, if the person does not enjoy leisure activities then his or her rating of the importance or satisfaction with leisure activities would be close to zero on the POPS. The low correlations observed between the subjective domains of the POPS with similar domains in the other instruments support this idea. Data obtained from the POPS is likely more subjective compared to the other instruments which may explain why the number of hypotheses supported are lower. Recently it has been reported that terminally ill patients underwent a response shift as a result of treatment and the reference used to answer the questions changed as well.50 Overall answers to the questions in the European Organization for Research and Treatment of Cancer Core Quality of Life Questionnaire (EORTC QLQ-C30) indicated that they had fewer limitations and this was because subjects interpreted the questions differently (e.g. ignoring activities that they could not perform).50 This suggests that further qualitative studies are needed to examine what factors individuals consider when responding to questions included  176  in participation instruments and these studies must consider the type of health condition. In this sample there was a mixture of spinal conditions and each one is unique. Individuals in the SCI group with neurological impairment would be expected to undergo a significant response shift as they adapt to having neurological impairment. Similar experiences would also likely occur with the spinal column and spinal degenerative groups since the experience of sustaining a trauma without any resulting neurological deficit or dealing with on going pain may also lead to similar response shifts. This will need to be considered when participation instruments are used to evaluate interventions and participation is assessed over time. When presenting these results it is also important to acknowledge the limitations of this study. Only the cross-sectional construct validity was assessed and future research should assess the ability of these instruments to assess clinically important changes following a treatment intervention. In addition, we were able to conduct most of the analyses within each of the three subgroups (see Appendix D); however it was not possible to conduct a confirmatory factor analysis within each group due to the small sample sizes and this should be done in future studies. In summary this study examined the cross-sectional construct validity of five participation instruments. Based on the criteria used to evaluate construct validity in this study the PM-PAC, IPA and WHODAS II performed differently in individuals with spinal conditions compared to the KAP and the POPS. It is important to note that the KAP was developed to assess participation at a population-level and consequently the level of detail was sacrificed for brevity. In terms of the POPS, results from this study suggest that it assesses different aspects of participation compared to the other four instruments which  177  should not make it less valid. It was the only instrument to capture objective or quantifiable information. The POPS also asks about satisfaction and importance in life areas which is unique and the PM-PAC also contains a few questions asking about satisfaction with relationships. Results from this study would suggest that information obtained from the POPS may supplement information obtained from the other participation instruments. However, since quality of life instruments also assess satisfaction (e.g. LiSat-11, Quality of Life Index51) and importance (e.g. Quality of Life Index51) in various life domains, future research should determine the relationship between participation and quality of life as well as how these concepts differ. Clinicians and researchers should consider the type of information required about the concept of participation before selecting an instrument. Overall, the five participation instruments assessed in this study appear to be promising but more evidence is required to demonstrate the construct validity in other health conditions.  178  Table 5.1. Characteristics of the study respondents for the entire sample (n=545)  Variable  SCI (n=145)  Spinal Column (n=187)  Spinal Degenerati ve (n=213)  Overall (n=545)  Gender  male  79%  71%  56%  67%  Marital Status  single married/partner divorced/widowed  31% 55% 14%  25% 60% 15%  8% 69% 23%  20% 62% 18%  Racial Background  Caucasian  80%  88%  87%  86%  Living Support live with someone  75%  79%  79%  78%  Urban or Rural urban  95%  93%  96%  94%  Education  high school college/university graduate  43% 49% 8%  36% 54% 10%  38% 45% 16%  39% 49% 12%  Employment  employed unemployed volunteer/retired unable to work  32% 7% 32% 26%  70% 2% 19% 9%  50% 2% 32% 14%  52% 3% 28% 15%  Compensation  yes  59%  17%  19%  29%  Spinal Procedures  yes  86%  48%  98%  78%  AIS traumatic SCI only (n=123)  AIS A AIS B AIS C AIS D  42% 15% 18% 24%  179  Variable mean (SD) (range) Age  at follow-up  Comorbidity (0 to 14)  score at follow-up  Motor Score (0 to 100)  on admission  SCI (n=145)  Spinal Column (n=187)  Spinal Degenerative (n=213)  48.7 (17.4) (21 to 86)  46.8 (16.2) (21 to 85)  57.6 (14.5) (24 to 90)  51.5 (16.6) (21 to 90)  1.0 (1.4) (0 to 8)  0.9 (1.3) (0 to 6)  1.5 (1.5) (0 to 7)  1.2 (1.4) (0 to 8)  Overall (n=545)  51.9 (26.2) 0 to 96  Abbreviations: AIS, ASIA Impairment Scale; SCI, spinal cord injury; SD, standard deviation  180  Table 5.2. Descriptive information for the participation instruments for the entire sample (n=545)* Instruments (score range)  Mean (SD)  IPA (0 to 4) Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education (n=356)  0.55 (0.77) 0.99 (0.97) 1.14 (1.14) 0.62 (0.70) 0.99 (1.12)  KAP (1 to 5) Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6 (n=286) Interpersonal Interactions & Relationships Economic Life Work (n=327) Education (n=193) Community, Social & Civic Life (n=412) PM-PAC (1 to 5) Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment (n=299) Education (n=63) Economic Life Community, Social & Civic Life  Range  Median  IQR  0 to 3.57 0 to 4.00 0 to 4.00 0 to 3.00 0 to 4.00  0.14 0.86 1.00 0.33 0.75  1.00 1.71 2.00 1.00 1.67  1.40 (0.73) 1.69 (0.97) 1.37 (0.78) 1.62 (0.95) 1.45 (0.81) 1.58 (0.87) 1.49 (0.82)  1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00  1.00 1.00 1.00 1.00 1.00 1.00 1.00  1.00 1.00 0.00 1.00 1.00 1.00 1.00  1.48 (1.00) 1.57 (1.10) 2.05 (1.48) 1.70 (1.08)  1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00  1.00 1.00 1.00 1.00  1.00 1.00 2.00 1.00  4.63 (0.66) 4.26 (0.93) 4.32 (0.87) 4.08 (0.94) 3.54 (1.19) 4.19 (0.97)  1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00 1.00 to 5.00  5.00 4.60 4.67 4.33 3.75 4.60  0.50 1.20 1.00 1.33 1.75 1.40  4.43 (0.78) 4.59 (0.76) 4.03 (0.90)  2.00 to 5.00 1.00 to 5.00 1.17 to 5.00  4.75 5.00 4.29  1.00 0.67 1.33  181  Instruments (score range) POPS† (subjective domains -4 to 4) Objective Domestic Life Objective Major Life Areas Objective Transportation Objective Interpersonal Interactions & Relationships Objective Community, Recreational & Civic Life Subjective Domestic Life Subjective Major Life Areas Subjective Transportation Subjective Interpersonal Interactions & Relationships Subjective Community, Recreational & Civic Life WHODAS II (0 to 100) Understanding & Communicating Getting Around Self-Care Life Activities (Non-working; n=162) Life Activities (Working; n=383) Getting Along with People Participation in Society Total Score (Non-working; n=162) Total Score (Working; n=383)  Mean (SD)  Range  Median  IQR  -0.15 (0.91) 0.79 (1.76) -0.80 (0.56) 0.88 (2.54)  -2.22 to 2.03 -0.98 to 10.69 -1.31 to 3.17 -1.59 to 20.09  0 0.65 -0.22 0.15  1.29 2.93 1.29 2.25  0.43 (1.37)  -1.16 to 10.06  0.17  1.31  1.00 (1.28) 0.28 (1.44) 0.89 (1.41) 0.99 (1.19)  -3.00 to 4.00 -3.33 to 3.33 -4.00 to 4.00 -3.38 to 3.75  1.25 0.33 1.50 1.25  1.85 2.00 2.00 1.50  0.70 (0.96)  -2.80 to 3.20  0.80  1.20  11.48 (16.69)  0 to 80.00  5.00  15.00  31.33 (27.57) 13.74 (22.20) 45.56 (30.95)  0 to 100.00 0 to 100.00 0 to 100.00  25.00 0.00 50.00  50.00 20.00 42.50  21.64 (23.93)  0 to 100.00  16.67  37.50  16.07 (19.79) 26.93 (22.43) 29.91 (17.26)  0 to 100.00 0 to 91.67 0 to 76.09  8.33 25.00 29.89  25.00 37.50 26.09  18.20 (17.58)  0 to 84.91  13.21  25.47  Abbreviations: IPA, Impact on Participation and Autonomy; IQR, inter-quartile range; KAP, Keele Assessment of Participation; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SD, standard deviation; WHODAS II, World Health Organization Disability Assessment Schedule II  182  Notes: * the sample size is noted if it is less than the total sample (n=545) † the score range for the POPS objective domains varies for each domain  183  7 7 5 6 6  6 5 3 3 4 5 4 3 9  6 5 4  IPA (0 to 4) Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education  PM-PAC (1 to 5) Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment Education Economic Life Community, Social & Civic Life  WHODAS II (0 to 100) Understanding & Communicating Getting Around Self-Care  Instruments (range)  0.74 0.73 0.77  0.76 0.80 0.72 0.74 0.83 0.77 0.70 0.75 0.69  0.82 0.85 0.88 0.79 0.87  0.69 to 0.82 0.62 to 0.81 0.60 to 0.83  0.65 to 0.85 0.72 to 0.89 0.71 to 0.74 0.64 to 0.80 0.75 to 0.88 0.67 to 0.81 0.67 to 0.78 0.59 to 0.77 0.43 to 0.80  0.73 to 0.82 0.66 to 0.87 0.84 to 0.89 0.60 to 0.83 0.81 to 0.92  Median Item Correlation # of Inter-Item Range Within the Questions Correlation Domain  Table 5.3. Results of the item-to-scale tests for the entire sample (n=545)  0.26 to 0.59 0.30 to 0.62 0.37 to 0.65  0.20 to 0.66 0.37 to 0.72 0.41 to 0.66 0.22 to 0.57 0.30 to 0.74 0.26 to 0.73 NA 0.23 to 0.58 0.16 to 0.75  0.52 to 0.71 0.55 to 0.80 0.65 to 0.80 0.45 to 0.70 0.61 to 0.80  Item Correlation Range With All Other Domains  30/30 (100) 29/30 (97) 24/24 (100)  48/48 (100) 39/40 (98) 24/24 (100) 24/24 (100) 32/32 (100) 40/40 (100) NA 32/32 (100) 69/72 (96)  28/28 (100) 27/28 (96) 20/20 (100) 22/24 (92) 24/24 (100)  # Correlations Within Domain> Other Domains (%)  184  0.85 0.80 0.70 0.71  4 8 5 8  0.78 to 0.85 0.45 to 0.76 0.64 to 0.78  0.83 to 0.90 0.45 to 0.68 0.24 to 0.57 0.38 to 0.71  0.36 to 0.57  Item Correlation Range With All Other Domains  40/40 (100) 27/30 (90) 47/48 (98)  20/20 (100  # Correlations Within Domain> Other Domains (%)  185  Abbreviations: IPA, Impact on Perceived Autonomy; KAP, Keele Assessment of Participation; NA, not applicable; PM-PAC, Participation-MeasurePost Acute Care; POPS, Participation Objective Participation Subjective; WHODAS II, World Health Organization Disability Assessment Schedule II  WHODAS II cont. Life Activities (Non-working; n=162) Life Activities (Working; n=383) Getting Along with People Participation in Society  Instruments (range)  Median Item Correlation # of Inter-Item Range Within the Questions Correlation Domain  Table 5.4. Results of the confirmatory factor analysis for the entire sample Instruments IPA (n=545) Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education PM-PAC (n=512) Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment Education Economic Life Community, Social & Civic Life WHODAS II (n=545) Understanding & Communicating Getting Around Self-Care Life Activities Getting Along with People Participation in Society  Standardized Loadings on First-Order Factor 0.73 to 0.91 0.70 to 0.91 0.88 to 0.91 0.63 to 0.89  RMSEA (90% CI)*  CFI*  SRMR*  0.071 (0.066, 0.075)  0.99  0.060  0.054 (0.049, 0.059)  0.99  0.064  0.98  0.085  NA  0.56 to 0.87 0.76 to 0.94 0.78 to 0.82 0.71 to 0.92 0.76 to 0.92 NA NA 0.67 to 0.88 0.74 to 0.80  0.73 to 0.89  0.069 (0.065, 0.073)  0.70 to 0.81 0.63 to 0.90 NA 0.53 to 0.86 0.72 to 0.80  186  Instruments WHODAS II (n=545)† Understanding & Communicating Getting Around Self-Care Life Activities Getting Along with People Participation in Society  Standardized Loadings on Second-Order Factor 0.73 0.81 0.73 NA 0.72 0.90  Abbreviations: CFI, comparative fit index; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual; WHODAS II, World Health Organization Disability Assessment Schedule II Notes: * the RMSEA, CFI and SRMR are estimates of the overall model fit † the WHODAS II has a second-order factor structure (general disability factor accounts for the correlations among the first-order factors) whereas the IPA and PM-PAC just have a firstorder factor structure .  187  NA 6/10 2/2 10/18 3/5  8/14 0/1 3/5  NA 4/6 2/2 6/8 3/5  6/8 NA 3/5  Communication (d3)  Mobility (d4)  Self-Care (d5)  Domestic Life (d6)  Interpersonal Interactions & Relationships (d7)  Major Life Areas (d8) -Work/Education  Major Life Areas (d8) -Economic Life  Community, Social & Civic Life (d9)  PM-PAC  POPS-OBJ  POPS-SUBJ  0/4†  NA  0/7†  0/4†  0/7†  NA  0/5†  NA  0/4  NA  0/7  1/4  1/7  NA  0/5†  NA  3/5  NA  5/8  3/5  9/14  2/2  4/6  0/1  WHODAS II  Abbreviations: ICF, International Classification of Functioning, Disability and Health; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; WHODAS II, World Health Organization Disability Assessment Schedule II  3/5  0/1  11/18  4/5  6/8  NA  4/6  0/1  # strong or moderate correlations / # correlations assessed*  KAP  IPA  ICF Domains  Table 5.5. Strength of the correlations among similar participation instrument domains using the entire sample (n=545)  188  189  Notes: * Strong correlation >0.70; Moderate correlation = + 0.50 to + 0.69; Fair correlation = + 0.31 to + 0.49; Weak correlation <0.30 and Spearman’s rho correlation was used. The numbers of correlations vary among the instruments depending on the domains or questions (subdomains) relevant to the ICF chapters in the activities and participation component. Not all instruments cover each content area in the ICF (e.g. self-care, economic life) and are therefore not applicable. Correlations among domains within instruments (e.g. PM-PAC’s education and work/employment domains) were not counted. † strong or moderate correlations were not expected  Table 5.6. Correlations* among participation domains and disease-specific instruments for the entire sample (n=545)†  IPA  Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education  KAP  Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6 Interpersonal Interactions & Relationships Economic Life Work Education Community, Social & Civic Life  POPS  Obj Domestic Life Obj Major Life Areas Obj Transportation Obj Interpersonal Interactions & Relationships Obj Community, Recreational & Civic Life Subj Domestic Life Subj Major Life Areas Subj Transportation  SRFM (n=145) 0.59 0.50 0.49 0.38 0.57  ODQ (n=272) 0.64 0.73 0.75 0.66 0.69  NDI (n=128) 0.52 0.71 0.67 0.41 0.66  0.47 0.37 0.47 0.29 0.31 0.23‡ (n=59) 0.28  0.60 0.63 0.50 0.60 0.57 0.62 (n=162) 0.53  0.41 0.46 0.32 0.49 0.43 0.43 (n=65) 0.50  0.28 0.45 (n=76) 0.38 (n=45) 0.33  0.46 0.51 (n=175) 0.45 (n=102) 0.54  0.31 0.46 (n=76) 0.47 (n=46) 0.51  -0.36 -0.29 -0.19 -0.16‡  -0.21 -0.33 -0.05‡ -0.27  -0.01‡ -0.33 -0.17 -0.16‡  -0.17  -0.18  -0.08‡  -0.30 -0.20 -0.14‡  -0.42 -0.26 -0.19  -0.31 -0.23 0.01‡  190  POPS cont.  PM-PAC  Subj Interpersonal Interactions & Relationships Subj Community, Recreational & Civic Life Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment Education Economic Life Community, Social & Civic Life  WHODAS II  Understanding & Communicating Getting Around Self-Care Life Activities (Non-working) Life Activities (Working) Getting Along with People Participation in Society  SRFM (n=145) -0.19  ODQ (n=272) -0.31  NDI (n=128) -0.14‡  -0.05‡  -0.27  -0.12‡  -0.22 -0.51 -0.38 -0.30  -0.49 -0.68 -0.67 -0.54  -0.55 -0.52 -0.61 -0.44  -0.21 -0.37 (n=53) -0.39 (n=24) -0.22 -0.44  -0.73 -0.60 (n=168) -0.51 (n=26) -0.48 -0.78  -0.63 -0.65 (n=78) -0.52 (n=13) -0.31 -0.68  0.10‡  0.46  0.45  0.56 0.67 0.21‡ (n=58) 0.45 (n=87) 0.22 0.47  0.79 0.51 0.62 (n=69) 0.66 (n=203) 0.54 0.73  0.57 0.40 0.57 (n=35) 0.58 (n=93) 0.43 0.58  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NDI, Neck Disability Index; ODQ, Oswestry Disability Questionnaire; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SRFM, Self-Reported Functional Measure; WHODAS II, World Health Organization Disability Assessment Schedule II Notes: * Strong correlation >0.70; Moderate correlation = + 0.50 to + 0.69; Fair correlation = + 0.31 to + 0.49; Weak correlation <0.30 and Spearman’s rho correlation was used  191  Notes cont: † Higher correlations (strong/moderate) were expected between the participation domain scores related to 1) mobility, 2)self-care, 3) domestic life, 4) major life areas (work/education) and 6) community, social and civic life and the disease-specific instrument overall scores. Lower correlations (fair/weak) were expected between the participation domain scores related to 1) communication, 2) interpersonal interactions and relationships and 3) major life areas (economic life) and the disease-specific instrument overall scores. Lower correlations were also expected between the POPS objective and subjective domain scores and the disease-specific instrument overall scores. ‡ non-significant correlation  192  fair (0.45)  fair (0.39)  moderate (0.54)  Interpersonal Interactions & Relationships  Social Life & Relationships moderate (0.60)  KAP  IPA  moderate (-0.51)  moderate (-0.68)  Interpersonal Relationships  PM-PAC  weak (-0.11)  weak (-0.28)  Interpersonal Interactions & Relationships  POPS-OBJ  fair (-0.35)  moderate (-0.54)  Interpersonal Interactions & Relationships  POPS-SUBJ  fair (0.37)  moderate (0.51)  Getting Along with People  WHODAS II  193  Notes: * Strong correlation (>0.70); Moderate correlation (+ 0.50 to + 0.69); Fair correlation = + 0.31 to + 0.49; Weak correlation <0.30 and Spearman’s rho correlation was used † A strong/moderate correlation was expected between the participation domains related to interpersonal interactions and relationships and the LiSat-11 question asking about satisfaction with the amount of contact with friends and acquaintances (except for the POPS objective Interpersonal Interactions and Relationships domain). A fair/weak correlation was expected between the participation domains related to interpersonal interactions and relationships and the LiSat-11 question asking about satisfaction with the person’s financial situation.  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; LiSat-11, Life Satisfaction-11; NDI, Neck Disability Index; ODQ, Oswestry Disability Questionnaire; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SRFM, Self-Reported Functional Measure; WHODAS II, World Health Organization Disability Assessment Schedule II  Financial Situation†  LiSat-11 Contact with Friends†  Instrument  Table 5.7. Correlations* among the participation instrument domains and the quality of life instrument using the entire sample (n=545)  Table 5.8. Summary* of the study results for validity Criteria  IPA  KAP  PM-PAC  POPS  WHODASII  Dimensionality 1) Item 2) CFA  ++ ++  NA NA  +++ +++  NA NA  ++ ++  ++  ++  ++  +  ++  Convergent/Discriminant 1) Participation Instruments 2) Disease-Specific Instruments 3) Quality of Life  +++  ++  +++  +++  +++  +++  +++  ++  +++  +++  Known-Groups  +++  ++  +++  ++  ++  Abbreviations: CFA, confirmatory factor analysis; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; WHODAS II World Health Organization Disability Assessment Schedule II Notes: * Ratings: +++ met criteria/results as expected; ++ partially met criteria/results partially as expected; + results primarily did not meet criteria/results primarily not as expected  194  5.5 References 1. Cicerone KD. Participation as an outcome of traumatic brain injury rehabilitation. J Head Trauma Rehabil 2004;19:494-501. 2. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization, 2001. 3. World Health Organization. International Classification of Impairments, Disabilities and Handicaps. A manual of classification relating to consequences of diseases. Geneva: World Health Organization, 1980. 4. Harwood RH, Rogers A, Dickinson E et al. Measuring handicap: the London Handicap Scale, a new outcome measure for chronic disease. Qual Health Care 1994;3:11-6. 5. Whiteneck GG, Charlifue SW, Gerhart KA et al. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil 1992;73:519-26. 6. Cardol M, de Haan RJ, de Jong BA et al. Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Arch Phys Med Rehabil 2001;82:210-6. 7. Wilkie R, Peat G, Thomas E et al. The Keele Assessment of Participation: a new instrument to measure participation restriction in population studies. Combined qualitative and quantitative examination of its psychometric properties. Qual Life Res 2005;14:1889-99. 8. Gandek B, Sinclair SJ, Jette AM et al. Development and Initial Psychometric Evaluation of the Participation Measure for Post-Acute Care (PM-PAC). Am J Phys Med Rehabil 2006;86:57-71. 195  9. Brown M, Dijkers MP, Gordon WA et al. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004;19:459-81. 10. World Health Organization. WHODAS II. [http://www.who.int/icidh/whodas]. November 2001; Accessed March 10, 2006. 11. Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11:193-205. 12. Messick S. Validty of psychological assessment: validation of inferences from persons' responses and performance as scientific inquiry into score meaning. Am Psychol 1995;50:741-9. 13. Streiner DL, Norman GR. "Precision" and "accuracy": two terms that are neither. J Clin Epidemiol 2006;59:327-30. 14. Fitzpatrick R, Davey C, Buxton MJ et al. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2:i-74. 15. Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. 1 ed. Chichester: John Wiley and Sons, 2000. 16. Marino RJ, Barros T, Biering-Sorensen F et al. International standards for neurological classification of spinal cord injury. J Spinal Cord Med 2003;26 Suppl 1:S50-S56.  196  17. Sangha O, Stucki G, Liang MH et al. The Self-Administered Comorbidity Questionnaire: a new method to assess comorbidity for clinical and health services research. Arthritis Rheum 2003;49:156-63. 18. Statistics Canada. 2006 Census Dictionary. [Statistics Canada Catalogue no.92-566XWE]. 2007; Available at: http://www12.statcan.ca/english/census06/reference/dictionary/index.cfm. Accessed February 16, 2008. 19. Sibley A, Kersten P, Ward CD et al. Measuring autonomy in disabled people: Validation of a new scale in a UK population. Clin Rehabil 2006;20:793-803. 20. Vernon H, Mior S. The Neck Disability Index: a study of reliability and validity. J Manipulative Physiol Ther 1991;14:409-15. 21. Hart DL, Wright BD. Development of an index of physical functional health status in rehabilitation. Arch Phys Med.Rehabil 2002;83:655-65. 22. Riddle DL, Stratford PW. Use of generic versus region-specific functional status measures on patients with cervical spine disorders. Phys Ther 1998;78:951-63. 23. Fairbank JC, Pynsent PB. The Oswestry Disability Index. Spine 2000;25:2940-52. 24. Slover J, Abdu WA, Hanscom B et al. The impact of comorbidities on the change in short-form 36 and oswestry scores following lumbar spine surgery. Spine 2006;31:1974-80.  197  25. Hoenig H, McIntyre L, Sloane R et al. The reliability of a self-reported measure of disease, impairment, and function in persons with spinal cord dysfunction. Arch Phys Med Rehabil 1998;79:378-87. 26. Hoenig H, Branch LG, McIntyre L et al. The validity in persons with spinal cord injury of a self-reported functional measure derived from the functional independence measure. Spine 1999;24:539-43. 27. Granger CV, Hamilton BB, Linacre JM et al. Performance profiles of the functional independence measure. Am J Phys Med Rehabil 1993;72:84-9. 28. Fugl-Meyer AR, Melin R, Fugl-Meyer KS. Life satisfaction in 18- to 64-year-old Swedes: in relation to gender, age, partner and immigrant status. J Rehabil Med 2002;34:239-46. 29. Lund ML, Nordlund A, Bernspang B et al. Perceived participation and problems in participation are determinants of life satisfaction in people with spinal cord injury. Disabil Rehabil 2007;29:1417-22. 30. Fugl-Meyer AR, Branholm I, Fugl-Meyer KS. Happiness and domain-specific life satisfaction in adult northern Swedes. Clin Rehabil 1991;5:25-33. 31. McHorney CA, Ware JE, Jr., Lu JF et al. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994;32:40-66.  198  32. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 2 ed. Oxford: Oxford University Press, 1995. 33. Brown TA. Confirmatory Factor Analysis for Applied Research. New York: Guildford Press, 2006. 34. Joreskog KG. Testing structural equation models. In: Bollen KA, Long JS, eds. Testing structural equation models. Newbury Park: Sage Publications Inc., 1993:294-316. 35. Muthen B, Kaplan D. A comparison of some methodologies for the factor analysis of non-normal Likert variables. Br J Math Stat Psychol 1984;38:171-89. 36. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, eds. Testing structural equation models. Newbury Park: Sage Publications Inc., 1993:136-62. 37. Hu L, Bentler PM. Cuttoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria Versus New Alternatives. Structural Equation Modeling 1999;6:1-55. 38. McHorney CA, Ware JE, Jr., Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care 1993;31:247-63. 39. Terwee CB, Bot SD, de Boer MR et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34-42. 40. Feinstein AR. Clinimetric perspectives. J Chronic Dis 1987;40:635-40.  199  41. Anderson KD, Borisoff JF, Johnson RD et al. The impact of spinal cord injury on sexual function: concerns of the general population. Spinal Cord 2006. 42. Jette A, Tao W, Haley SM. Blending activity and participation sub-domains of the ICF. Disabil Rehabil 2007;29:1742-50. 43. Reinhardt JD, Stucki G. Rheumatoid arthritis and participation--the social animal revisited. J Rheumatol 2007;34:1214-6. 44. Whiteneck G. Conceptual models of disability: past, present, and future. In: Field MJ, Jette AM, Martin L, eds. Workshop on Disability in America: A New Look. Washington, D.C.: National Academies Press, 2006:50-66. 45. Fougeyrollas P, Noreau L, Bergeron H et al. Social consequences of long term impairments and disabilities: conceptual approach and assessment of handicap. Int J Rehabil Res 1998;21:127-41. 46. Noreau L, Fougeyrollas P, Vincent C. The LIFE-H: assessment of the quality of social participation. Technol Disabil 2002;14:113-8. 47. Cantor JB, Ashman T, Gordon W et al. Fatigue after traumatic brain injury and its impact on participation and quality of life. J Head Trauma Rehabil 2008;23:41-51. 48. Dijkers MP, Whiteneck G, El Jaroudi R. Measures of social outcomes in disability research. Arch Phys Med Rehabil 2000;81:S63-S80. 49. Rochette A, Korner-Bitensky N, Levasseur M. 'Optimal' participation: A reflective look. Disabil Rehabil 2006;28:1231-5.  200  50. Westerman MJ, Hak T, Sprangers MA et al. Listen to their answers! Response behaviour in the measurement of physical and role functioning. Qual Life Res 2008;17:549-58. 51. Ferrans CE, Powers MJ. Quality of life index: development and psychometric properties. ANS Adv Nurs Sci 1985;8:15-24.  201  6 DISCUSSION, CONCLUSIONS AND RECOMMENDATIONS 6.1 Summary of Results The overall purpose of this study was to compare participation instruments developed using the International Classification of Functioning, Disability and Health (ICF). The literature review identified 11 instruments, with 7 of the 11 instruments being published in the past two years of the literature review (between 2006 and 2008). Due to the recent increase in the number of instruments, it is important that more direct comparisons of these instruments are conducted, not only to advance our understanding of this concept but also to assist clinicians and researchers in the selection of instruments. Our hypothesis was that information from these instruments would not be equivalent due to the differences in how the concept of participation was operationalized and this was supported by our results. In this study the content validity of 8 of the 11 instruments was assessed. All of the eight instruments adequately cover the concept of participation based on the criteria used in this study. Each instrument contained questions with content covering a minimum of five chapters from the ICF Chapters 3 (Communication) to 9 (Community, social and civic life). There are differences among these instruments as to which ICF chapters (domains) are included. For instance the Participation Objective Participation Subjective (POPS) and the Participation Measure-Post Acute Care (PM-PAC) do not consider self-care to be an aspect of participation and in the POPS only transportation is covered from the ICF Chapter Mobility. These differences reflect the lack of consensus in how the concepts of activity and participation should be differentiated. Two instruments [Participation Scale, World Health Organization Disability Assessment Schedule II (WHODAS II)] contain questions with content not considered to be aspects of participation and therefore received lower ratings for  202  content validity. As an example, the WHODAS II has a question which asks about problems due to barriers or hindrances in the world around you which had no meaningful concepts (content) related to the ICF Chapters 3 to 9. By linking the meaningful concepts in these instruments to the ICF classification it was possible to conduct a detailed content analysis of the instruments which was invaluable in assessing content validity and interpreting the empirical results. Five of the eight instruments [Impact on Participation and Autonomy (IPA), Keele Assessment of Participation (KAP), PM-PAC, POPS, WHODAS II] were empirically tested and compared in terms of score distribution (floor/ceiling effects), internal consistency, testretest reliability, unidimensionality, convergent and discriminant validity, and known-group validity. Three of the instruments were not included because they were developed for other cultures (Participation Scale), too similar to other instruments included (Rating of Perceived Participation), or the instrument was too specific (Participation Survey/Mobility). We evaluated each instrument’s measurement properties based on criteria published in the literature and our a priori expectations. A summary of the ratings for the measurement properties is provided in Table 6.1 for the five instruments. The presence of substantial ceiling effects (highest possible participation scores for greater than 15% of the sample) was the largest problem that all the instruments suffered (except the POPS), thereby making it difficult to detect changes over time or participation in highly functioning individuals. Ceiling effects were greatest in the KAP, ranging from 56.7% for the second mobility question to 75.8% for the self-care question. All the individuals included in this study were admitted and treated at Vancouver General Hospital (VGH) for a spinal condition. The instruments were administered following discharge and  203  reflect participation in the community (cross-sectional). In reviewing the presence of ceiling effects, it is important to consider the purpose of the assessment. In this study, we did not purposefully target participants currently experiencing symptoms but rather included all participants. It has been suggested that the goal of rehabilitation should be achieving an optimal level of participation for each particular person depending on their needs and desires.1 If this is the case then perhaps the ceiling effects in this study demonstrate that individuals who have sustained a spinal condition and received treatment are participating as they want without too many constraints. In this study 38.3% (n=104/272) of the individuals with a spinal degenerative condition or spinal trauma had moderate to extremely severe pain and the participation instruments (except in the POPS) were able to detect lower group differences. If the purpose of collecting information on participation is to assess the effectiveness of an intervention, which is expected to improve participation, then it would be important to determine if the instruments are able to detect the effect of interventions. In this study we assessed both aspects of reliability, internal consistency and test-retest reliability. The internal consistency was good in all the instruments and the values were greater than 0.70, which is recommended for group-level comparisons2. The IPA was the only instrument to have values for internal consistency greater than 0.90, which is recommended for individual-level comparisons2. Cronbach’s alpha can be higher in domains with more questions.3 The PM-PAC had four domains where the values were in the mid 0.80’s and these domains had fewer questions compared to the IPA domains (3 to 4 versus 5 to 7 questions), which may contribute to the lower values. Test-retest reliability needs to be established to ensure the assessments of participation remain stable. Measuring stability in the area of participation is challenging since unlike  204  measuring range of motion, levels of participation can change depending on the life circumstances. Since the intraclass correlation coefficient (ICC) is calculated using the ratio of variability between individuals and the total variability (variability between individuals and measurement error), it describes an instrument’s ability to differentiate among individuals.4-6 The IPA and the WHODAS II had the highest values for test-retest reliability using ICCs (2, 1), which accounted for any systematic variability differences between the two administrations.7 All the ICCs in these two instruments were greater than 0.70 which has been recommended for group-level comparisons2 and the WHODAS II mobility domain (getting around) had an ICC value 0.90 which is recommended for individual-level comparisons2. The POPS demonstrated adequate test-retest reliability in domains assessing objective participation in areas expected to remain stable, such as domestic life and work. Lower values were noted for objective participation in the domains ‘interpersonal interactions and relationships’ and ‘community, recreational and civic life’, which may reflect large within person changes thereby making it harder to discriminate among individuals. The KAP also did not perform as well as the other instruments and it may be due to the use of a broad or general question covering an area of participation such as selfcare. In addition, single questions are also known not to be as reliable as multi-item scales.8 Test-retest reliability was also assessed for these instruments using the standard error of measurement (SEM). The SEM provides information regarding the consistency of responses by calculating the variability of measurements on the same individual and there has been a growing interest in measuring the minimal detectable change (MDC), which can be calculated using the SEM. For an instrument to be useful clinically, it is important that a meaningful change [referred to as the minimal important change (MIC)] is greater than the  205  MDC, otherwise it will not be possible to detect with a certain precision the effectiveness of interventions. To our knowledge this is the first study to report MDC for the majority of these instruments. Other studies have investigated the MDC as a percentage of the scale using different instruments and results from our study are similar to those reported for the Sickness Impact Profile in individuals with chronic stroke.4 In both our study and Beckerman et al.’s4 study, the baseline values indicated individuals had high ratings (very few problems) and it would not be possible to detect any improvement above the MDC due to ceiling effects at an individual-level. Since it is easier to detect a significant difference when assessing groups compared to individuals due to a smaller variance of the sample mean, the participation instruments would be better at detecting group-level differences.9 We did not find any reports of the MIC for these five instruments and so this will need to be addressed in future research before any conclusions can be made regarding their ability to assess the effectiveness of clinical interventions. The assessment of construct validity is an on-going process that requires developing a ‘nomological network’, linking the theoretical framework with an empirical one.10 In this study we investigated many aspects of validity in an attempt to understand how the information obtained from the participation instruments compare. It is important to consider that the instruments were designed for different purposes and measure participation differently. The KAP was developed to assess participation restrictions at a populationlevel11 and so the need for brevity often requires using a single question to cover broad areas of participation (e.g. self-care). The IPA and KAP assess autonomy (doing something the way and when someone wants), the PM-PAC primarily asks about limitations in various aspects of participation and the WHODAS II measures difficulty. The POPS is the only  206  instrument to assess objective aspects of participation by asking about the frequency of doing specific activities. In the POPS subjective aspects of participation are measured by producing a combined rating of satisfaction and importance. Results from this study demonstrate how the KAP and the POPS differ from the IPA, PM-PAC and the WHODAS II and given the differences in the KAP and POPS compared with the other instruments, it is not too surprising. Assessments of unidimensionality were empirically assessed in the IPA, PM-PAC and the WHODAS II. Unidimensionality and item-to-scale correlations were not assessed in the KAP and the POPS since these two instruments were not necessarily designed and tested to measure unidimensional aspects of participation using multi-item scales. Results from this study suggest that overall the factor structure was supported when administering these instruments to individuals with spinal conditions. The concept of participation is an extremely broad concept and can range from moving around in one’s house to managing finances and even voting in civic elections, depending on how it is operationalized. Even within one of these areas it is challenging to develop domains measuring a single underlying latent factor and often various types of mobility, including moving around the house to moving around using transportation are assessed within one domain. Overall the results from the item-to-scale correlation and the confirmatory factor analysis (CFA) demonstrated that the instruments have well structured domains, especially given the type of concept. Questions asking about sexual and intimate relationships did not seem to fit as well with their domain compared to other areas such as mobility or self-care. However, it is important not to let the measurement models and data drive the construction of participation instruments since relying on factor analysis to determine the questions may result in instruments excluding  207  important areas that are relevant in the conceptualization of participation.12 This raises the issue of needing appropriate criteria and methods to evaluate the multi-dimensional concept of participation and future work in this area is greatly needed. It has been suggested that methods such as classical test theory and item response theory (including Rasch) be abandoned and participation instruments are constructed using alternative methods which focus on the face validity (e.g. clinimetrics), so the needs and desires of individuals can be captured.12 The domains within each of the participation instruments were mapped to an ICF chapter heading (also referred to as ICF domains). By doing this it enabled us to compare how the instruments covered the ICF chapters. Results from the convergent validity analysis demonstrated that the instrument domain name and ICF chapter did not necessarily result in strong correlations. Only by considering both the content and how the question was asked was it possible to interpret the correlations. The POPS and the KAP performed differently compared to the other three instruments in this area as well. Lower correlations were expected among the objective domains of the POPS compared to all the other participation domains based on the literature13. However, it was surprising that higher correlations were not observed among the subjective domains of the POPS with the other participation instruments. These results support the idea that the POPS is measuring a different aspect of participation compared to the others. Comparisons were also made between the participation domains and disease-specific instruments which assess the concept of health status. This type of comparison is important since many clinicians and researchers currently assess health status and it is important to determine how the concepts of participation and health status compare. Results from this  208  study supported the hypothesis that 1)higher correlations would be observed in domains related to mobility, self-care, domestic life, major life areas (work and/or school), and community, social and civic life and 2) lower correlations would be expected in domains related to communication, interpersonal interactions and relationships, and economic life. On further examination of the content of the Neck Disability Index (NDI), Oswestry Disability Questionnaire (ODQ) and Self-Reported Functional Measure (SRFM), it appears that the disease-specific instruments assess the impact of other ICF components (body functions or environment) on aspects of participation. The participation instruments in contrast are generally more ‘pure’ measures asking directly about mobility or work and cover more aspects of participation. A similar comparison was made with the Life Satisfaction-11 (LiSat-11) instrument since the relationship between participation and quality of life is not well understood. In this study none of the instruments had a strong association with ratings of satisfaction in various life domains. The correlations, however, were higher as expected in domains measuring similar content. It was surprising that the POPS subjective domains did not have higher correlations with satisfaction ratings since both instruments ask about satisfaction. In the POPS the satisfaction rating is combined with the importance rating and this may result in the differences observed. Brown et al.13 also did not report strong correlations between the subjective domains of the POPS and the Flanagan Quality of Life Scale which asks about importance in various life areas. These findings support the comments made by Brown et al.13 suggesting that the assessment of satisfaction and importance do not necessarily fit with the methods used in classical test theory. Brown et al.13 do not support separating the ratings of importance and satisfaction, since this assumes that importance and satisfaction each have  209  a single underlying construct and some theories suggest that individuals use various standards to determine an overall rating of satisfaction. All of the five instruments were able to differentiate among known-groups, with the number of hypotheses supported ranging from 75% in POPS to 95.2% in the IPA. In this study the known-groups were formed based on a range of clinical, patient-reported and demographic variables previously reported in the literature. Hypotheses were considered to be supported if 1) there was a statistically significant difference between the groups (e.g. < 65 years versus >65 years old) in both the unadjusted and adjusted model and 2) the estimate of the effect was in the expected direction. In summary, the IPA, PM-PAC and WHODAS II had similar measurement properties. Both the KAP and the POPS fill a unique and important gap. The KAP was designed to be used in population-based studies and the POPS provides objective and subjective information (importance/satisfaction) that is not captured in the other three instruments. It is important to consider these differences when interpreting the evaluation criteria since these criteria may not be well suited to these instruments. More work is needed in this area before one participation instrument can be considered the gold standard.  6.2 Study Strengths and Limitations 6.2.1 Study Strengths There are a number of strengths in this study which make the results an important contribution to the literature. The most important strength of this study was including five instruments which were developed using the ICF to measure the concept of participation. To date, there was only one study which included two of the instruments (IPA, KAP).11 There has been a strong interest in developing instruments which assess participation since the ICF 210  was first published in 1997 and therefore it is important that studies include multiple instruments to understand how the information obtained from these instruments compares. In addition, by selecting instruments developed using the ICF it provided a framework whereby we could make comparisons among the domains using the ICF chapter headings. For example, the POPS only includes transportation from the ICF Chapter Mobility whereas the other instruments include multiple aspects of mobility ranging from moving around the house to travelling outside a person’s town. Another strength of this study is the large sample size (n=545) with each of the spinal groups having a sample size ranging from 145 in the spinal cord injury (SCI) group to 213 in the spinal degenerative group. Studies using these instruments previously reported data on samples with 200 to 300 individuals.13 Eligible individuals were randomly selected until the target sample size was achieved or until all eligible individuals were contacted. The sample in this study was also well defined as the spinal diagnosis was made by a physician. As a result we feel confident that the individuals were assigned to the correct group. By using a hospital database which prospectively captures all acute in-patient admissions for spine patients at VGH, we were able to identify participants for this study and also obtain high quality clinical information (diagnosis, neurological assessment). All individuals who participated in this study also provided detailed information pertaining to sociodemographics (e.g. marital status, living support), socioeconomic information (e.g. level of education; employment) and health information (e.g. comorbidities). The clinical and self-reported data enabled us to comprehensively study the measurement properties of the instruments. For example, the neurology data for subjects with SCI allowed us to test hypotheses regarding the expected effect of motor impairment on participation (known-group validity). The  211  clinical and self-reported data was also used to make necessary adjustments in our statistical models for confounding effects of variables such as age and gender, which enabled us to obtain accurate estimates of the effects. The rigorous study methods and procedures used to collect the data provide credibility to the results. A pilot study was initially conducted with 67 subjects and it provided us with valuable information regarding the study procedures and format of the instruments within the questionnaire. In the actual study, numerous attempts were made to locate potentially eligible individuals in order to have a representative sample. For those individuals who were eligible to participate, the study coordinator contacted them within one week of mailing the questionnaire to ensure the questionnaire was received. In cases where the individuals indicated that they were interested in participating in the study but had not returned a questionnaire, a reminder letter and another copy of the questionnaire was sent. The response rate in this study was 60% (545/911) for the overall sample, which is good for this type of sample.14;15 In addition, all questionnaires received were checked to ensure the data was complete. If there were sections a number of missing answers then the study coordinator contacted the subjects and asked if they would be willing to answer the questions. If however, subjects indicated in the questionnaire that they did not wish to be contacted then no further contact was attempted and no subjects were asked to complete questions asking about personal information (finances or personal relationships). These procedures enabled us to have less than 10% missing data at the question-level for the participation instruments. Including instruments which assess concepts such as health status (disease-specific instruments) as well as quality of life (LiSat-11) in this study is an important attribute. Currently it is not well understood how the concept of participation differs from these other  212  concepts and so empirical evidence comparing these instruments is an important contribution. By including these instruments it was possible to empirically examine if the five participation instruments were similar to, or different from these other instruments. If instruments measuring health status and quality of life had not been included it would only have been possible to speculate as to these relationships. In addition, this is the first study to our knowledge which has compared participation instruments with disease-specific instruments since in the past most of the comparisons have been made with generic health status instruments such as the Short Form-36.16;17  6.2.2 Study Limitations It is also necessary to discuss the limitations with this study which have been highlighted in Chapters 3, 4 and 5 and are summarized below. In this study it is possible that individuals may have experienced a true change when completing the second questionnaire used to assess test-retest reliability. Overall, the responses for the second administration of the instruments were higher for some domains related to work and community life (IPA, PM-PAC, WHODAS II) suggesting that the respondents’ ratings of participation did improve or they answered the questions differently (e.g. learning effect). A test-retest interval of 10 days was targeted in order to try and obtain an accurate assessment. We excluded individuals (n=17) who did not complete the second administration of the instruments within 7 to 31 days since it is possible that true changes may have occurred. However, in this study we did not ask the respondents if there was any change in their participation which would affect their answers and this should be included in future studies.  213  As with any study involving self-reported instruments, there was missing data. Overall the amount of missing data was very low. The percentages of missing data for the first and second administration of the IPA, KAP, PM-PAC and WHODAS II were all less than 10%. The POPS has two questions asking about attending school, where the missing data was 12.8% and 14.1% on the second administration (reliability). In the SCI group, the amount of missing data for the SRFM at the question-level ranged between 10.3% and 11.7% since some participants (n=12) received the wrong version of the questionnaire (received the questionnaire containing the ODQ or NDI instead of SRFM). All individuals completed at least one of the instruments and variables potentially related to the reason why data was missing and also variables known to be associated with the participation scores were included in the model (e.g. age, spine diagnosis). In addition, the imputation did not use information from the other instruments in order to obtain an accurate comparison of the instruments. Our analysis of the missing data revealed that there were differences in the subjects with missing data compared to those with complete data. Subjects with missing data tended to be older and did not rate their health as high as compared to those with complete data and it would have biased the sample if they had been removed from the analysis (see Appendix B for more details). Since the instruments included in this study have only recently been developed some slight modifications were needed. In the case of the IPA, the data collection phase had already begun when the paper by Sibley et al.18 was published which included a question asking about ‘helping or supporting others’. There were also other minor changes to the wording of the questions; for example in our study the IPA work and education domain only includes paid work and education, based on the original studies published by Cardol et  214  al.19;20. However, in the version of the IPA published by Sibley et al.18 modifications were made to include unpaid as well as paid work. Based on a comparison of our results with these other studies it appears that these minor changes did not have a large impact on the results. In terms of the POPS, this instrument was originally developed to be an intervieweradministered instrument and the initial testing included individuals with traumatic brain injury.13 We contacted the developers and asked for their advice regarding changing the format to be self-administered.(personal communication Brown, October 3, 2006) It was felt that the POPS could easily be changed into a self-administered format and we piloted the POPS in a sample of 67 individuals and solicited feedback regarding the format. We felt comfortable using the POPS reference data to calculate the objective domain scores since the frequency and importance ratings for the objective questions were similar for the healthy individuals and persons with traumatic brain injury, suggesting the data is generalizable.13 Following this study we have spoken with the developers of the POPS and are considering comparing the reference data for the two samples. The response options and scoring algorithm for the POPS should possibly be revised if this instrument is going to be selfadministered in order to maximize the number of questions used to generate subjective domain scores. In this study it was noted that subjects tended to omit questions if they did not participate in the activity (e.g. volunteer work, use public transport) and based on the original POPS scoring algorithm scores are not generated if subjects select the response ‘don’t know’. With regard to the PM-PAC, the scores were originally developed using an item response theory (IRT) macro.21 For all of the scores with the exception of the community, social and  215  civic life domain score, the authors stated that there were no differences between the scores derived from the IRT macro and scoring the domains using simple sum scoring. We were not able to obtain a copy of the IRT macro. Based on the evaluations of internal consistency, item-to-scale, item discriminant validity and the CFA using our data, we felt comfortable with generating a score for the community, social and civic life domain. However, it is possible that results from our study may not be comparable to the results reported by Gandek et al.21 for this domain due to the different scoring procedures used. In terms of our analyses, since the data was not normally distributed we accounted for this by using Spearman rho correlation coefficients and by using robust maximum likelihood in the estimation of the CFA models, which is known to be robust for the type and amount of non-normality observed in our data. However, for the known-group validity analysis we used multiple regression under the assumption the data was normally distributed. In terms of the CFA analysis, it has been suggested that instruments containing ordinal data similar to this study should use polychoric correlations with diagonally weighted least squares estimation using the asymptotic covariance matrix.22 This approach requires a larger sample size compared to using estimation methods for continuous data.22 It was decided that given our sample size and given that robust maximum likelihood estimation can handle some nonnormality this would be the best method. Robust maximum likelihood estimation uses the asymptotic covariance matrix to compute a weight matrix, and adjusts the fit statistics and standard errors to account for the non-normal data distribution.22 Finally, in this study we did not assess the time required by the subjects to complete the instruments (respondent burden) and this should be included in future studies. The number of questions in each of the five instruments range from 15 (including screening questions) in  216  the KAP to 78 in the POPS and this information will help users select an instrument. In addition, the ability of the instruments to detect meaningful change over time was also not assessed and should be addressed in future studies. Results from this study did estimate the MDC (Chapter 4) and it would be helpful to further quantify what individuals with spinal conditions perceive to be important or meaningful change so users have information on the MIC.  6.3 Recommendations 6.3.1 Clinical and Research Implications As with any study it is important to determine how the results from this study can impact clinical practice and research. Before selecting an instrument it is important that the purpose is clearly identified and the following questions should be answered: 1) ‘what aspects of participation are relevant to meet the patient/person’s needs or for research purposes?’, 2) ‘is subjective or objective information or both types needed on the relevant aspects of participation?’, and 3) ‘does the influence of environmental factors or the health condition(s) on aspects of participation matter (should the questions tell the person to think of his or her health/environmental influences on participation when selecting a response versus not mentioning these factors)?’. Answers to these types of questions will assist the user in selecting the correct type of participation instrument and then the measurement properties of relevant instruments can be compared and an appropriate instrument can be selected. It is also important to recognize that the measurement properties will also provide users with information that is needed to answer the questions stated above. For example, if a societal or environmental intervention is being assessed then it is important to consider if instruments providing objective or subjective information are more reliable and responsive. However, we 217  would encourage users to first consider the questions stated above rather than focussing on the measurement properties since without a clear understanding of the purpose, there is a risk of not meeting the measurement objectives. Information from this study will be valuable to clinicians and researchers who are interested in measuring participation. In Chapter 2, a review of the literature is provided which includes not only the five instruments empirically assessed in this study but also six additional instruments. The content of eight participation instruments is provided in Chapter 3. In Chapters 4 and 5, the empirical results assessing aspects of reliability and validity in individuals with spinal conditions are described and the empirical results are summarized in Chapter 6. To help synthesize all of the information described in this study an overview of the findings are summarized in Table 6.1. In considering these results it is important to remember that the instruments described in this study are not exhaustive and there are many other instruments assessing participation which have been developed using other conceptual models. For example, the Assessment of Life Habits (Life-H)23;24 instrument assesses participation but used the Disability Creation Process model and so it was not included in this study. As mentioned previously there is a lot of work being done currently to develop new instruments to assess participation and during the time of this study another new instrument has been published, called the ICF Measure of Participation and Activities questionnaire (IMPACT-S).25  6.3.2 Future Research Throughout this study we have suggested numerous areas that require future research. Some of these recommendations include further subgroup analysis and interpretation of results in each of the three spinal conditions to better understand the information obtained 218  from these instruments. Other recommendations include empirically assessing these instruments in individuals with different health conditions and conducting future studies to assess measurement properties such as responsiveness and the MIC. However, results from this study have highlighted a few significant areas that must be addressed in future research to advance the conceptualization and measurement of participation. These areas include 1) determining which measurement methods are suitable for developing and assessing the measurement properties of participation instruments and 2) conceptually differentiating the concepts participation, activity, health status and quality of life. Both of these areas are discussed below.  6.3.3 Measurement of Participation In measuring such a broad and complex concept such as participation, it is important to carefully consider the type of measurement model used and specifically, determining the relationship between the questions and the underlying latent factor. Fayers et al.26 stated that indicators, such as the assessment of symptoms are causal because a change in symptoms could affect a person’s rating of his or her quality of life. However, a change in a person’s rating of their quality of life does not necessarily mean a corresponding change in their symptoms. In contrast, if a domain was developed to measure depression and the questions asked about feeling sad or levels of energy, then it is expected that a change in the person’s depression would result in a corresponding change in the answers to these questions. Instruments measuring an underlying unidimensional concept such as depression use measurement methods such as exploratory factor analysis and item-to-total correlations to demonstrate that the questions are homogeneous. The questions are outward manifestations of the underlying construct and are referred to as effect indicators.27 For instruments 219  developed using causal indicators, such as the Apgar scale, the questions are selected based on clinical judgement and the most relevant questions are included. It does not make sense to do factor analysis or assess the inter-item-correlations since the questions are heterogeneous and selected based on relevance and not statistics. Feinstein28 introduced the term ‘clinimetrics’ to describe the development of instruments such as the Apgar scale used in medicine. This term was chosen to conceptually differentiate these types of scales from the homogenous scales developed using more traditional methods in education and psychology, referred as psychometrics. However, debate continues in the literature whether clinimetrics should be referred to as a separate discipline or instead be seen as types of psychometric theory.27;29 The POPS is an example of an instrument which was not necessarily developed to be unidimensional. Brown et al.13 stated that testing the POPS using a Rasch analysis would imply that the domains or total scores are measuring single underlying dimension. Instead the authors argued that such methods are not applicable in this case13 and the questions are more likely causal indicators rather than effect indicators. It has been suggested that in the area of quality of life not much attention has been paid as to whether questions are causal or effect indicators and often instruments contain a mixture of both.26 Given the complexity of measuring a concept such as participation, careful consideration should be given as to whether the questions comprising participation domains are causal or effect indicators. Developers of instruments should describe the type of measurement model and assess the appropriate measurement properties. Results from this study suggest that for some of the participation domains such as interpersonal interactions and relationships, the poor inter-item-correlations and standardized loadings for questions asking about sexual  220  activities are possibly causal indicators rather than effect indicators. It does not necessarily make sense to assume that an improvement in a person’s interpersonal relationships will be manifested by an improvement in his or her sexual or intimate relationships. Since sexual relationships are an important aspect of participation that needs to be measured, careful consideration is needed before removing it just because it does not have a high item-to-scale correlation. These types of questions may be better assessed from the perspective of casual indicators rather than effect indicators and recommendations have made regarding how this should be done.30 Dijkers31 has suggested that it is unreasonable to expect that all questions will have positive inter-item correlations unless the sample ranges from persons with minimal disabilities to those who live in institutional settings. It is also possible to find negative correlations (e.g. people with full-time jobs will not be able to manage their household fulltime).31 Dijkers’31 point regarding the sample is important since the inter-item-correlation improves with a more heterogeneous sample. This study did include a heterogeneous sample and the inter-item correlations will likely be lower if analyzed within each subgroup. The type of measurement model also has implications regarding the suitability of some IRT models which require the measurement of a single underlying dimension. When Rasch, which is a type of IRT, was applied to instruments such as the IPA, four questions were removed since they did not fit the model.32 In one study the question asking about ‘intimate relationships’ was removed since it did not fit the model.32 However, as mentioned previously, if conceptually these types of questions are a critical aspect of participation then the model should not necessarily dictate the content and other models should be considered.12 More research is needed to determine the methods and the criteria used to  221  assess participation instruments to reflect the challenges in trying to measure ‘optimal participation’ since doing more of a given activity may not be the desired outcome and the person’s values must be acknowledged.12;31 There have been significant developments in the areas of CFA and IRT and there will be an opportunity to apply new emerging latent variable methodologies such as factor mixture models whereby latent classes can be identified within a heterogeneous sample.22 Finally, advances in other areas of measurement will be applicable to assessing participation. New approaches such as the Day Reconstruction Method, which uses a structured approach to recalling past events, have been suggested as ways to collect subjective information more reliably and should be considered in future studies.33 New techniques such as global positioning systems have been used to document how an individual interacts in his or her environment and are now starting to be applied when measuring participation and can provide objective information.34 All of these advances will assist us in measuring and interpreting information related to participation. There is a need to consider both objective and subjective aspects of participation since both types of information can be relevant depending on the user’s perspective. The POPS is the only instrument which contains both subjective and objective information. There are additional questions in the PM-PAC for some domains (e.g. interpersonal interactions and relationships) which assess objective information but were not scored and this should be tested in future studies. Future work should consider the differences in how participation can be measured (e.g. difficulty, limitations, autonomy, satisfaction, importance, frequency, amount of assistance, etc.) and how the information is interpreted (subjective ratings versus comparing to societal norms or data from other health conditions). This type of work will enable us to better measure  222  participation and facilitate users in selecting the appropriate instrument depending on their purpose.  6.3.4 Differentiating the Concepts of: Participation, Activity, Health Status and Quality of Life One of the areas for future research identified by Whiteneck35 in his review of the ICF model was the need to further distinguish the concepts of activity and participation. Currently the concepts of activity and participation are defined in the ICF model. In the ICF classification a single list of domains is provided and the user is provided with various options as to how they can be further distinguished.36 Results from this study revealed differences in how these two concepts are operationalized in the instruments. Based on the content analysis performed in Chapter 3, there are differences in whether Communication (ICF Chapter 3), aspects of Mobility (ICF Chapter 4) and Self-care (ICF Chapter 5) are included as domains within participation. In order for the field to advance there should be consensus on how to clearly differentiate activity from participation as it is difficult to compare instruments if they do not contain the same content. Currently confusion exists regarding how the concepts of participation, health status and quality of life are different. In the past it has been difficult to determine the relationship between subjective handicap and quality of life since both are often measured by assessing satisfaction in life domains.37 Furthermore, there is a misconception that quality of life instruments assess participation.38 Similarly, instruments assessing health status have been reported to assess participation since both types of instruments contain questions asking about social functioning or social roles.38-40  223  To accurately assess the relationship between the concept of participation and other concepts such as health status and quality of life, these concepts need to be well-defined and clearly operationalized. For example, if a researcher is interested in determining how improving interpersonal relationships affects quality of life and the PM-PAC was selected to measure interpersonal relationships and the LiSat-11 was used to assess quality of life, since both instruments ask the same questions (satisfaction with relationships) then the association detected between these two concepts will result from measuring the same thing and not because of any unique relationship. Therefore, it is important to consider how to operationalize the subjective dimension of participation if quality of life is assessed using life domains. Currently quality of life is not included in the ICF because the classification focuses on disease or disability constructs that are ‘objective and exteriorised signs of the individual (page 251)’36. The ICF manual states that quality of life assesses how people feel about their life and is part of the construct subjective well-being (p. 251)36 In the 2007 Institute of Medicine’s report Future of Disability in America41 it is suggested that quality of life should be added to the ICF model and the relationship between the ICF components (body structures and functions, activity and participation, contextual factors) and the concept of quality of life be established. It has been well documented that the subjective dimension of functioning is important and cannot be ignored.42;43 To date there have been several proposals made regarding how the subjective dimension43-45 and quality of life35;44;45 could be incorporated into the ICF model, but no consensus has been reached. Future research exploring the relationship between the ICF components and quality of life as well as the subjective dimension will further clarify how these concepts are operationalized.  224  In terms of the relationship between the concepts of health status and participation, there are issues with overlap in content. The term health status emerged from the World Health Organization’s definition of health as “A state of complete physical, mental, and social wellbeing and not merely the absence of diseases and infirmity”46. Instruments assessing selfreported health status commonly include physical, mental and social dimensions and it has been suggested that health status includes the ICF components 1) body functions and body structures, 2) activity and 3) participation.44 This study included five participation instruments and three disease-specific instruments. The ODQ and NDI focus on the impact of pain (body function) on aspects of participation and the SRFM assess the need for assistance (environmental factor) on aspects of mobility and self-care. The participation instruments appear to be more ‘pure’ measures not asking about the impact of other factors such as pain or fatigue on aspects of participation in the questions. However, the instructions in the WHODAS II, for example, states that this instrument asks about ‘difficulties due to health conditions’ and so perhaps the ODQ and NDI would likely overlap if the person considers his or her pain when answering the WHODAS II questions. There are likely differences in whether the person is asked about his or her participation versus the impact of health or pain on aspects of participation. Future research should determine the relationship between the concepts of health status and the ICF components. This will assist users in determining which concepts are operationalized in instruments and whether the questions are asking about participation versus the effect of other variables such as health on participation. In addition, future qualitative studies should examine how individuals answer the questions and what factors they consider when selecting their response.  225  6.4 Conclusions This study compared instruments assessing the concept of participation. There have been tremendous advances in this area since the concept of participation was first introduced in draft form in 1997. Seven of the eleven instruments identified in the literature search were published in the last two years. Due to this recent increase of participation instruments it is critical that these instruments are compared to advance our understanding of how the concept of participation is operationalized and to assist users of these instruments in selecting an appropriate instrument for a given purpose. To our knowledge this work has not been done and our study will help fill this gap. Results from this study supported our hypothesis that information from these instruments would not be equivalent due to the differences in how the concept of participation was operationalized. As expected, we found differences in how the instruments were developed based on the literature review, in the content, what was assessed (objective versus subjective information), whether the respondent should consider his or her health and environmental influences as well as the measurement properties. Understanding these differences is important for the field to move forward. By examining the similarities and differences in how the instruments were conceived, developed and the empirical evidence generated by their use, it will enable us to have meaningful dialogue regarding the required next steps. Continued research using these instruments will further advance our understanding of the concept of participation, how it differs from other concepts such as activity and quality of life and will inform us as to how best to operationalize it. Given the differences in the instruments, to date there is no gold standard for measuring participation and the selection of an instrument should be driven by the user’s purpose. The 226  ideal participation instrument would cover all relevant aspects of participation, collect objective as well as subjective information and be responsive. Due to the diverse needs of the users (administrator, clinician, researcher) and time constraints of daily life, it is also unlikely that one instrument will emerge as the gold standard. Instead it may be preferable to focus not on the instruments themselves but on the domains within the instruments. This would enable users to examine the content, measurement model and measurement properties for a given domain and then combine different domains depending on what is important in a given study or population. If this approach was taken to measuring multi-dimensional concepts such as participation, then the results from this study would assist in providing an inventory of information for the domains to assist the user. The ICF model could provide users with an overarching conceptual model and help users identify the concepts that should be measured. However, more clarity is needed regarding the relationship among concepts within the ICF model as well as with concepts such as quality of life, so it is clear what is being measured. There have been significant advances in our ability to measure participation over the last few years and continued work in this area will enable us to measure this important concept. In the future, the number of persons with disabilities will continue to increase as the population ages. There will be new medical discoveries and technologies developed as well as new governmental policies assisting persons with disabilities to be active in the workplace that will all need to be evaluated. Participation is positioned to be the most relevant outcome to understand the effect on functioning and disability and continued work in this area is essential.  227  Table 6.1. Summary* of study results Criteria Score Distribution (Floor/Ceiling) Reliability 1) Internal Consistency 2) Test-retest Reliability (ICC/weighed kappa) 3) Test-retest Reliability (SEM/MDC) Content Validity  IPA  KAP  PM-PAC  POPS  WHODASII  +  +  +  +++  ++  +++ ++  NA +  +++ ++  NA +  +++ ++  ++  +  ++  ++  ++  +++  +++  +++  +++  ++  NA NA  +++ +++  NA NA  ++ ++  ++  ++  ++  +  ++  Cross-Sectional Construct Validity Unidimensionality 1) Item ++ 2) CFA ++ Convergent/Discriminant 1) Participation Instruments 2) Disease-Specific Instruments 3) Quality of Life  +++  ++  +++  +++  +++  +++  +++  ++  +++  +++  Known-Groups  +++  ++  +++  ++  ++  Abbreviations: CFA, confirmatory factor analysis; ICC, intraclass correlation coefficient; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; MDC, minimal detectable change; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SEM, standard error of measurement; WHODAS II World Health Organization Disability Assessment Schedule II Notes: * Ratings: +++ met criteria/results as expected; ++ partially met criteria/results partially as expected; + results primarily did not meet criteria/results primarily not as expected  228  6.5 References 1. Rochette A, Korner-Bitensky N, Levasseur M. 'Optimal' participation: A reflective look. Disabil Rehabil 2006;28:1231-5. 2. Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11:193-205. 3. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 2 ed. Oxford: Oxford University Press, 1995. 4. Beckerman H, Roebroeck ME, Lankhorst GJ et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 2001;10:571-8. 5. de Vet HC, Terwee CB, Knol DL et al. When to use agreement versus reliability measures. J Clin Epidemiol 2006;59:1033-9. 6. Stratford P. Reliability: consistency or differentiating among subjects? Phys Ther 1989;69:299-300. 7. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420-8. 8. Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. 1 ed. Chichester: John Wiley and Sons, 2000.  229  9. Dekker J, Dallmeijer AJ, Lankhorst GJ. Clinimetrics in rehabilitation medicine: current issues in developing and applying measurement instruments. J Rehabil Med 2005;37:193-201. 10. Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychol Bull 1955;52:281-302. 11. Wilkie R, Peat G, Thomas E et al. The Keele Assessment of Participation: a new instrument to measure participation restriction in population studies. Combined qualitative and quantitative examination of its psychometric properties. Qual Life Res 2005;14:1889-99. 12. Dijkers MP. Issues in the conceptualization and measurement of participation: an overview. Presentation at the American Academy of Physical and Rehabilitation Medicine, October 14, 2008. Slides 1-70. 13. Brown M, Dijkers MP, Gordon WA et al. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil 2004;19:459-81. 14. Fisher CG, Noonan VK, Smith DE et al. Motor recovery, functional status, and healthrelated quality of life in patients with complete spinal cord injuries. Spine 2005;30:2200-7. 15. Noreau L, Proulx P, Gagnon L et al. Secondary impairments after spinal cord injury: a population-based study. Am J Phys Med Rehabil 2000;79:526-35.  230  16. Chwastiak LA, Von Korff M. Disability in depression and back pain: evaluation of the World Health Organization Disability Assessment Schedule (WHO DAS II) in a primary care setting. J Clin Epidemiol 2003;56:507-14. 17. Posl M, Cieza A, Stucki G. Psychometric properties of the WHODASII in rehabilitation patients. Qual Life Res 2007;16:1521-31. 18. Sibley A, Kersten P, Ward CD et al. Measuring autonomy in disabled people: Validation of a new scale in a UK population. Clin Rehabil 2006;20:793-803. 19. Cardol M, de Haan RJ, de Jong BA et al. Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Arch Phys Med Rehabil 2001;82:210-6. 20. Cardol M, Beelen A, van den Bos GA et al. Responsiveness of the Impact on Participation and Autonomy questionnaire. Arch Phys Med Rehabil 2002;83:1524-9. 21. Gandek B, Sinclair SJ, Jette AM et al. Development and Initial Psychometric Evaluation of the Participation Measure for Post-Acute Care (PM-PAC). Am J Phys Med Rehabil 2006;86:57-71. 22. Brown TA. Confirmatory Factor Analysis for Applied Research. New York: Guildford Press, 2006. 23. Fougeyrollas P, Noreau L, Bergeron H et al. Social consequences of long term impairments and disabilities: conceptual approach and assessment of handicap. Int J Rehabil Res 1998;21:127-41.  231  24. Noreau L, Fougeyrollas P, Vincent C. The LIFE-H: assessment of the quality of social participation. Technol Disabil 2002;14:113-8. 25. Post MW, de Witte LP, Reichrath E et al. Development and validation of IMPACT-S, an ICF-based questionnaire to measure activities and participation. J Rehabil Med 2008;40:620-7. 26. Fayers PM, Hand DJ, Bjordal K et al. Causal indicators in quality of life research. Qual Life Res 1997;6:393-406. 27. Streiner DL. Clinimetrics vs. psychometrics: an unnecessary distinction. J Clin Epidemiol 2003;56:1142-5. 28. Feinstein AR. Clinimetric perspectives. J Chronic Dis 1987;40:635-40. 29. de Vet HC, Terwee CB, Bouter LM. Current challenges in clinimetrics. J Clin Epidemiol 2003;56:1137-41. 30. Atkinson MJ, Lennox RD. Extending basic principles of measurement models to the design and validation of Patient Reported Outcomes. Health Qual Life Outcomes 2006;4:65. 31. Dijkers MP, Whiteneck G, El Jaroudi R. Measures of social outcomes in disability research. Arch Phys Med Rehabil 2000;81:S63-S80. 32. Lund ML, Fisher AG, Lexell J et al. Impact on participation and autonomy questionnaire: internal scale validity of the Swedish version for use in people with spinal cord injury. J Rehabil Med 2007;39:156-62.  232  33. Smith DM, Brown SL, Ubel PA. Mispredictions and misrecollections: challenges for subjective outcome measurement. Disabil Rehabil 2008;30:418-24. 34. Seekins T, Ipsen C, Arnold NL. Using ecological momentary assessment to measure participation: a preliminary study. Rehabil Psychol 2007;52:319-30. 35. Whiteneck G. Conceptual models of disability: past, present, and future. In: Field MJ, Jette AM, Martin L, eds. Workshop on Disability in America: A New Look. Washington, D.C.: National Academies Press, 2006:50-66. 36. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization, 2001. 37. Fuhrer MJ. The subjective well-being of people with spinal cord injury: relationships to impairment, disability, and handicap. Top Spinal Cord Inj Rehabil 1996;4:56-71. 38. Desrosiers J. Muriel Driver Memorial Lecture. Participation and occupation. Can J Occup Ther 2005;72:195-204. 39. Perenboom RJ, Chorus AM. Measuring participation according to the International Classification of Functioning, Disability and Health (ICF). Disabil Rehabil 2003;25:577-87. 40. Salter K, Jutai JW, Teasell R et al. Issues for selection of outcome measures in stroke rehabilitation: ICF Participation. Disabil Rehabil 2005;27:507-28.  233  41. Institute of Medicine. Definition and monitoring of disability. In: Field MJ, Jette AM, eds. The future of disability in America. Washington, D.C.: The National Academies Press, 2007:35-64. 42. Hammel J, Magasi S, Heinemann A et al. What does participation mean? An insider perspective from people with disabilities. Disabil Rehabil 2008;30:1445-60. 43. Ueda S, Okawa Y. The subjective dimension of functioning and disability: what is it and what is it for? Disabil Rehabil 2003;25:596-601. 44. Post M, Noreau L. Quality of life after spinal cord injury. J Neurol Phys Ther 2005;29:139-46. 45. Post MW, de Witte LP, Schrijvers AJ. Quality of life and the ICIDH: towards an integrated conceptual model for rehabilitation outcomes research. Clin Rehabil 1999;13:5-15. 46. World Health Organization. The first ten years of the World Health Organization. Geneva: World Health Organization, 1958.  234  APPENDICES APPENDIX A. UBC RESEARCH ETHICS BOARD CERTIFICATE The University of British Columbia Office of Research Services Behavioural Research Ethics Board Suite 102, 6190 Agronomy Road, Vancouver, B.C. V6T 1Z3  CERTIFICATE OF APPROVAL- MINIMAL RISK RENEWAL PRINCIPAL INVESTIGATOR:  DEPARTMENT: UBC/Medicine, Faculty of/Health Care & Jacek Kopec Epidemiology/Epidemiology & Biostatistics INSTITUTION(S) WHERE RESEARCH WILL BE CARRIED OUT: Institution  Vancouver Coastal Health (VCHRI/VCHA)  UBC BREB NUMBER: H06-80684  Site  Vancouver General Hospital  Other locations where the research will be conducted:  N/A  CO-INVESTIGATOR(S): Joel Singer Marcel F.S. Dvorak Vanessa Noonan  SPONSORING AGENCIES: Physiotherapy Foundation of Canada Unfunded Research - "A Comparison of Questionnaires which Assess Participation in Life Activities in Individuals with Spinal Conditions" PROJECT TITLE: A Comparison of Questionnaires which Assess Participation in Life Activities in Individuals with Spinal Conditions EXPIRY DATE OF THIS APPROVAL: August 15, 2008 APPROVAL DATE: August 15, 2007 The Annual Renewal for Study have been reviewed and the procedures were found to be acceptable on ethical grounds for research involving human subjects. Approval is issued on behalf of the Behavioural Research Ethics Board  235  APPENDIX B. MISSING DATA B.1. Overview of Terminology Missing data is a common occurrence in research studies. Rubin1 in 1976 proposed a classification for describing the mechanisms for missing data. This classification includes three types of missing data 1) missing completely at random (MCAR),2) missing at random (MAR) and 3) missing not at random (MNAR).2 In Rubin’s classification the word ‘mechanism’ describes the relationship between the condition of data that are missing and the observed or missing variables rather than the cause of the missing data.3 The first type of missing data mechanism, MCAR, occurs if for missing data on a given variable Y, the probability of missing data on Y is not related to the value of Y or to any other variables in the data set.4 If the assumption of MCAR is supported then the subset of subjects with complete data can be considered a simple random sample of the larger data set.4 The second type of missing data mechanism, MAR, occurs when missing data on a variable Y is not related to the value of Y after controlling for other variables in the analysis. It is not possible to confirm this assumption because the missing data cannot be analyzed. Since the mechanism for the missing data in MCAR and MAR is random, it is considered ignorable and does not have to be modeled.4 The final type of missing data mechanism, MNAR, occurs when the missing data mechanism is not ignorable and there is a relationship between the missing data on a variable Y and the value of Y.3  B.2. Summary of Missing Data in this Study When describing the missing data it is important to consider both the amount and the pattern. For each of the instruments, missing data patterns were generated using SAS 9.1.3 (SAS  236  Institute, 2005). Using the missing data patterns it was possible to report the missing data at the level of the questions (Table B.1) and for the subjects (Table B.2). For these analyses only the questions which were not optional were evaluated and questions which were optional in each of the instruments were excluded. The missing data for the questions was generally very low and the majority of the missing data was less than 10%. There was a greater amount of missing data for the Self-Reported Functional Measure (SRFM) since there were 12 subjects in the spinal cord injury (SCI) group who did not complete the correct version of the questionnaire [completed the Neck Disability Index (NDI) or Oswestry Disability Questionnaire (ODQ) instead of the SRFM]. The questions and domains in the instruments which ask about personal areas such as relationships and in particular sexual functioning had greater amounts of missing data. There was also a higher amount of missing data for the Participation Objective Participation Subjective (POPS) subjective domains. In the literature it has been reported that individuals often omit questions if they do not engage in the activity.5 This was also our experience and there were greater amounts of missing data for the POPS subjective questions which subjects did not participate in (e.g. education, work and using public transport). In terms of the missing data at the level of the subjects, the majority of the subjects completed all of the questions for each of the instruments. For the ODQ, 99% of the subjects (268/272) answered all 10 questions but for the POPS, the amount of complete data for the subjects was lower with only 33% answering all questions due to the reasons stated above. However, for the POPS 75% of the sample (411/545) completed 90% of the questions.  237  B.3. Management of Missing Data The missing data was imputed using the multiple Markov chain Monte Carlo (MCMC) method. Starting values were based on expectation maximization (EM) maximum likelihood estimates. Demographic and clinical data were also incorporated in the joint distribution for the imputation and included: age at injury (SCI group, spinal column group); age on admission to Vancouver General Hospital (VGH); age at follow-up; gender; live in an urban or rural setting; number of admissions to the Acute Spine Program at VGH; diagnosis (SCI group, spinal column group, spinal degenerative group); neck or back (spinal column group, spinal degenerative group); and International Standards for the Neurological Classification of SCI6 (SCI group only). A single imputation was produced and it was done within each instrument (imputation did not borrow information from other instruments). Data pertaining to work and education was only imputed for individuals who indicated that they were involved in these activities. The imputation was conducted using SAS 9.1.3 (SAS Institute, 2005). To test if the missing data was MCAR, an analysis was conducted to compare the subjects with complete data for each instrument to subjects who had data missing in one or more questions. Any questions which were optional (e.g. questions asking about work, education, sexual life) were not included in the analysis. The two groups (complete data versus any missing data) were tested to determine if there were any differences based on age, gender, diagnosis (SCI group, spinal column group, spinal degenerative group) and the subject’s rating of his or her general health (question one in the Short Form-36, Version 2.0). The Little’s chi-square test for assessing MCAR was also conducted. Results from this analysis indicated that for the participation instruments (Impact on Participation and Autonomy, Participation Measure-Post Acute Care, POPS, World Health Organization Disability Schedule II) the assumption of MCAR was not supported and the missing data was assumed to be MAR. Subjects with missing data for 238  these instruments tended to be older and report lower ratings of general health. For the Keele Assessment of Participation, the disease-specific instruments (SRFM, NDI, ODQ) and the Life Satisfaction-11, the assumption of MCAR was supported. In this study the imputed data was used for the analyses involving score distribution, Cronbach’s alpha coefficient and validity testing (correlations, known-group hypothesis testing, confirmatory factor analysis). Analyses pertaining to test-retest reliability, which included assessing intraclass correlation coefficients, standard error of measurement and minimal detectable change used the raw (non-imputed) data.  B.4. Discussion During the data collection phase, study procedures were incorporated to minimize the amount of missing data. When the study coordinator received the questionnaire from the subjects all of the instruments were checked to see whether the questions were completed. If there were sections with missing data, then the subjects were re-contacted and asked to provide answers to the missing data. However, if the subjects indicated that they did not want to answer any further questions then the subjects were not re-contacted. In addition, no subjects were asked to answer personal questions with missing data related to intimate relationships or income. For the analyses pertaining to score distribution, internal consistency and all of the validity testing a decision was made to use the imputed data set. If the raw (non-imputed) data was used then subjects with incomplete data would have been eliminated by the statistical software packages using either listwise or pairwise deletion, which removes the subject from the analysis or any domains with incomplete data, respectively.4 If data is removed using either listwise or pairwise deletion then this can result in larger standard errors and biased parameter estimates if the data is not MCAR.4 The analyses testing the assumption of MCAR revealed that the subjects 239  with missing data tended to be older and reported poorer general health and if the data was eliminated using either listwise or pairwise deletion it would have biased the sample and the data. In addition, the sample size and power would also have been reduced and it would have made the comparison among the various instruments difficult due to varying sample sizes. In this study, only one imputed data set was created rather than creating three to five data sets, which is often recommended when using multiple imputation.4 Multiple data sets were not created since it would have required additional analyses for all of the various measurement properties reported on in this study (e.g. score distribution, known-group validity, etc.). Since the overall percentage of missing data at a question- and subject-level was primarily less than 10%, this was not expected to have a significant impact on the study results and a decision was made to conduct the analyses using a single imputed data set.  240  B.5. References 1. Rubin DB. Inference and missing data. Biometrika 1976;63:581-92. 2. Rubin DB. Multiple imputation after 18+ years. J Am Stat Assoc 1996;91:473-89. 3. McKnight PE, McKnight KM, Sidani S et al. Classifying missing data. Missing data: a gentle introduction. New York: Guildford Press, 2007:40-64. 4. Allison PD. Missing data. Thousand Oaks: Sage Publications, 2001. 5. Dijkers M. Comments on van Brakel et al.'s Participation Scale. Disabil Rehabil 2006;28:1360-2. 6. Marino RJ, Barros T, Biering-Sorensen F et al. International standards for neurological classification of spinal cord injury. J Spinal Cord Med 2003;26 Suppl 1:S50-S56.  241  Table B.1. Missing data for the questions within the instruments for the entire sample (n=545) Instrument IPA Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education KAP Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6 Interpersonal Interactions & Relationships Economic Life Work Education Community, Social & Civic Life PM-PAC Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment Education Economic Life Community, Social & Civic Life  # of Questions Assessed  # of Subjects  Range of Item Missing Data  7 7 5 6 0*  545 545 545 545 NA  1.1% to 2.2% 1.3% to 2.8% 1.7% 1.3% to 3.1% NA  1 1 1 1 1 0 1  545 545 545 545 545 NA 545  1.3% 1.7% 1.5% 1.7% 1.7% NA 4.4%  1 0 0 0  545 NA NA NA  5.0% NA NA NA  6 5 3 3 4 0 0 3 9  545 545 545 545 545 545 545 545 545  1.7% to 2.2% 1.3% to 1.8% 2.0% to 2.4% 1.5% to 1.8% 2.4% to 2.6% NA NA 1.7% to 2.2% 1.7% to 2.4%  242  Instrument  POPS Objective Domestic Life Objective Major Life Areas Objective Transportation Objective Interpersonal Interactions & Relationships Objective Community, Recreational & Civic Life Subjective Domestic Life Subjective Major Life Areas Subjective Transportation Subjective Interpersonal Interactions & Relationships Subjective Community, Recreational & Civic Life  # of Questions Assessed  # of Subjects  Range of Item Missing Data  6* 3 2 8  545 545 545 545  1.3% to 2.0% 2.4% to 3.3% 2.2%to 2.3% 2.2% to 8.4%  5  545  1.8% to 2.8%  12* 6 4 16  545 545 545 545  1.5% to 2.8% 5.9% to 9.2% 2.8% to 6.6% 2.2% to 8.3%  10  545  1.7% to 5.5%  WHODAS II Understanding & Communicating Getting Around Self-Care Life Activities Getting Along with People Participation in Society  6  545  0.7% to 1.1%  5 4 4* 5 8  545 545 545 545 545  0.7% to 1.3% 0.2% to 0.4% 0.4% to 0.6% 0.2% to 6.8% 1.3% to 3.9%  NDI  9*  128  1.6% to 8.6%  ODQ  9*  272  1.1% to 1.5%  145  10.3% to 11.7%  545  1.5% to 9.7%  SRFM LiSat-11  13 9*  243  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; LiSat-11, Life Satisfaction-11; NDI, Neck Disability Index; ODQ, Oswestry Disability Questionnaire; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SRFM, Self-Reported Functional Measure; WHODAS II World Health Organization Disability Assessment Schedule II Notes: * questions that are optional (e.g. work, education) in the instruments were not included  244  13 9  SRFM  LiSat-11  489/545  126/145  268/272  117/128  463/545  178/545  494/545  514/545  505/545  0%  0/545  3/145  0/272  0/128  72/545  233/545  32/545  0/545  27/545  1 to 10%  46/545  1/145  0/272  3/128  7/545  109/545  9/545  23/545  2/545  11% to 30%  0/545  0/145  1/272  5/128  1/545  14/545  1/545  0/545  3/545  31% to 50%  0/545  0/145  0/272  0/128  1/545  5/545  1/545  0/545  1/545  51% to 70%  1/545  0/145  0/272  1/128  0/545  0/545  3/545  1/545  3/545  71% to 90%  7/545  15/145  3/272  2/128  1/545  6/545  5/545  7/545  4/545  91% to 100%  Notes: * questions that are optional (e.g. work, education) in the instruments were not included  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; LiSat-11, Life Satisfaction-11; NDI, Neck Disability Index; ODQ, Oswestry Disability Questionnaire; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SRFM, Self-Reported Functional Measure; WHODAS II World Health Organization Disability Assessment Schedule II  9  ODQ  32  WHODAS II 9  72  POPS  NDI  33  7  KAP  PM-PAC  25  Number of questions*  IPA  Instrument  Number of Subjects with % of Missing Data  Table B.2. Missing data for the subjects completing the instruments for the entire sample (n=545)  245  APPENDIX C. ADDITIONAL ANALYSES RELATED TO CHAPTER 4 (RELIABILITY)  246  1.60 (1.17) 0 to 4.00 0.86 (0.72) 0 to 2.83 1.40 (1.29) 0 to 4.00 (n=78)  Autonomy Outdoors  Social Life & Relationships  Work & Education  Mobility #2  2.03 (1.11) 1.00 to 5.00  1.61 (0.87) 1.00 to 5.00  1.31 (0.95) 0 to 3.71  Family Role  KAP (1 to 5) Mobility #1  0.88 (0.95) 0 to 3.57  mean (SD) range  4.1%; 39.3%  1.4%; 57.2%  7.7%; 21.8%  0%; 22.8%  3.4%; 11.0%  0%; 12.4%  0%; 26.2%  % worst score; % best score  SCI (n=145)  Autonomy Indoors  IPA (0 to 4)  Instrument (score range)  1.33 (0.64) 1.00 to 4.00  1.22 (0.55) 1.00 to 4.00  0.70 (0.91) 0 to 3.83 (n=144)  0.46 (0.64) 0 to 2.83  0.73 (0.93) 0 to 3.40  0.66 (0.83) 0 to 3.71  0.30 (0.54) 0 to 2.57  mean (SD) range  0%; 75.4%  0%; 82.4%  0%; 48.6%  0%; 54.0%  0%; 49.2%  0%; 47.1%  0%; 66.3%  % worst score; % best score  Spinal Column (n=187)  1.79 (0.99) 1.00 to 5.00  1.43 (0.72) 1.00 to 4.00  1.09 (1.17) 0 to 4.00 (n=132)  0.60 (0.70) 0 to 3.00  1.18 (1.15) 0 to 4.00  1.06 (1.00) 0 to 4.00  0.52 (0.72) 0 to 3.00  mean (SD) range  247  0.9%; 51.6%  0%; 68.5%  0.7%; 35.1%  0%; 40.4%  1.4%; 28.6%  0.5%; 25.4%  0%; 50.7%  % worst score; % best score  Spinal Degenerative (n=213)  Table C.1. Descriptive information and floor/ceiling effects for the participation instruments for the three spine groups  1.63 (0.98) 1.00 to 5.00 1.74 (1.0) 1.00 to 5.00 1.57 (0.86) 1.00 to 5.00 1.64 (0.92) 1.00 to 5.00 (n=59) 1.68 (0.90) 1.00 to 5.00 1.54 (1.07) 1.00 to 5.00 1.87(1.32) 1.00 to 5.00 (n=76)  Domestic Life #4  Domestic Life #5  Domestic Life #6  Interpersonal Interactions & Relationships  Economic Life  Work  mean (SD) range  10.5%; 59.2%  6.2%; 71.0%  0.7%; 54.5%  1.7%; 55.9%  1.4%; 61.4%  2.1%; 53.8%  2.1%; 62.1%  % worst score; % best score  SCI (n=145)  Self-Care  KAP cont.  Instrument (score range)  1.42 (0.92) 1.00 to 5.00 (n=130)  1.32 (0.83) 1.00 to 5.00  1.38 (0.72) 1.00 to 5.0  1.51 (0.76) 1.00 to 5.00 (n=108)  1.29 (0.64) 1.00 to 5.00  1.42 (0.76) 1.00 to 5.00  1.20 (0.56) 1.00 to 4.00  mean (SD) range  3.1%; 76.2%  3.2%; 81.8%  0.5%; 72.2%  0.9%; 61.1%  0.5%; 78.1%  1.6%; 69.5%  0%; 85.6%  % worst score; % best score  Spinal Column (n=187)  1.53 (1.10) 1.00 to 5.00 (n=121)  1.58 (1.15) 1.00 to 5.00  1.46 (0.84) 1.00 to 5.00  1.61 (0.95) 1.00 to 5.00 (n=119)  1.50 (0.88) 1.00 to 5.00  1.72 (1.04) 1.00 to 5.00  1.33 (0.73) 1.00 to 5.00  mean (SD) range  248  5.8%; 74.4%  7.5%; 71.4%  1.4%; 69.0%  1.7%; 61.3%  1.4%; 67.6%  1.9%; 59.2%  1.4%; 76.5%  % worst score; % best score  Spinal Degenerative (n=213)  3.74 (0.98) 1.40 to 5.00 4.00 (0.99) 1.00 to 5.00 3.91 (0.92) 1.00 to 5.00 3.27 (1.19) 1.00 to 5.00 3.98 (1.14) 1.00 to 5.00 (n=53)  Mobility  Domestic Life  Interpersonal Relationships  Role Functioning  Work & Employment  4.55 (0.74) 1.00 to 5.00  1.87 (1.15) 1.00 to 5.00 (n=106)  Community, Social & Civic Life  PM-PAC (1 to 5) Communication  1.96 (1.28) 1.00 to 5.00 (n=45)  mean (SD) range  1.9%; 30.2%  6.2%; 11.0%  0.7%; 22.1%  0.7%; 28.3%  0%; 17.9%  0.7%; 52.4%  4.7%; 50.9%  6.7%; 53.3%  % worst score; % best score  SCI (n=145)  Education  KAP cont.  Instrument (score range)  4.30 (0.92) 1.00 to 5.00 (n=137)  3.93 (1.08) 1.00 to 5.00  4.27 (0.83) 1.67 to 5.00  4.55 (0.71) 1.00 to 5.00  4.64 (0.63) 2.40 to 5.00  4.70 (0.58) 1.00 to 5.00  1.49 (0.90) 1.00 to 5.00 (n=151)  1.94 (1.50) 1.00 to 5.00 (n=80)  mean (SD) range  0.7%; 47.4%  2.7%; 26.7%  0%; 38.0%  0.5%; 57.2%  0%; 64.2%  0.5%; 61.5%  1.3%; 70.2%  16.2%; 63.8%  % worst score; % best score  Spinal Column (n=187)  4.16 (0.92) 1.00 to 5.00 (n=109)  3.38 (1.19) 1.00 to 5.00  4.05 (1.00) 1.00 to 5.00  4.33 (0.87) 1.00 to 5.00  4.27 (0.94) 1.00 to 5.00  4.63 (0.66) 1.50 to 5.00  1.78 (1.15) 1.00 to 5.00 (n=155)  2.24 (1.56) 1.00 to 5.00 (n=68)  mean (SD) range  249  0.9%; 33.0%  3.8%; 11.7%  0.5%; 30.5%  0.5%; 44.6%  0.5%; 42.7%  0%; 59.6%  5.2%; 58.7%  14.7%; 52.9%  % worst score; % best score  Spinal Degenerative (n=213)  3.69 (0.95) 1.17 to 5.00  Community, Social & Civic Life  0.34 (1.53) -0.98 to 5.09 -0.83 (0.53) -1.31 to 2.12 1.19 (3.27) -1.59 to 20.09  Obj Major Life Areas  Obj Transportation  Obj Interpersonal Interactions & Relationships  -0.44 (0.95) -2.22 to 1.91  4.43 (0.91) 1.00 to 5.00  Economic Life  POPS* (subjective domains -4 to 4) Obj Domestic Life  4.09 (0.87) 2.00 to 5.00 (n=25)  mean (SD) range  -0.74 (0.66) -1.31 to 3.17 1.12 (2.36) -1.59 to 15.46  0.7%; 0%  1.27 (1.76) -0.98 to 9.32  -0.06 (0.88) -1.89 to 2.03  4.33 (0.81) 1.33 to 5.00  4.69 (0.61) 2.00 to 5.00  4.66 (0.63) 2.75 to 5.00 (n=27)  mean (SD) range  1.6%; 0%  1.1%; 0%  16%; 0%  0%; 2.6%  0%; 30.5%  0%; 73.3%  0%; 59.3%  % worst score; % best score  Spinal Column (n=187)  2.1%; 0%  37.9%; 0%  1.4%; 0.7%  0%; 3.4%  1.4%; 56.6%  0%; 20.8%  % worst score; % best score  SCI (n=145)  Education  PM-PAC cont.  Instrument (score range)  0.45 (2.01) -1.55 to 14.72  -0.83 (0.49) -1.31 to 2.53  0.69 (1.83) -0.98 to 10.69  -0.02 (0.85) -2.22 to 1.93  3.99 (0.87) 1.62 to 5.00  4.62 (0.74) 1.00 to 5.00  4.54 (0.88) 2.25 to 5.00 (n=12)  mean (SD) range  0%; 0%  250  2.8%; 0%  30.5%; 0%  0.5%; 2.3%  0%; 9.4%  0.5%; 67.6%  0%; 58.3%  % worst score; % best score  Spinal Degenerative (n=213)  0.22 (1.12) -1.16 to 5.07 0.10 (0.93) -1.29 to 4.34 0.71 (1.33) -3.00 to 3.29 0.08 (1.46) -3.00 to 3.33 0.99 (1.41) -3.00 to 4.00 0.82 (1.22) -2.38 to 3.75 0.68 (1.03) -2.80 to 3.20 0.66 (0.88) -2.06 to 2.78  Obj Participation Total  Subj Domestic Life  Subj Major Life Areas  Subj Transportation  Subj Interpersonal Interactions & Relationships  Subj Community, Recreational & Civic Life  Subj Participation Total  mean (SD) range  0%; 0%  0%; 0%  0%; 0%  0%; 0.7%  0%; 0%  0%; 0%  0%; 0%  0.7%; 0%  % worst score; % best score  SCI (n=145)  Obj Community, Recreational & Civic Life  POPS cont.  Instrument (score range)  0.83 (0.74) -1.29 to 2.67  0.69 (0.81) -2.00 to 3.20  1.11 (1.02) -2.25 to 3.12  0.76 (1.25) -3.00 to 4.00  0.40 (1.36) -2.67 to 3.33  1.18 (1.09) -2.00 to 4.00  0.40 (0.80) -1.11 to 3.78  0.39 (1.28) -1.16 to 10.03  mean (SD) range  0%; 0%  0%; 0%  0%; 0%  0%; 0.5%  0%; 0%  0%; 0.5%  0%; 0%  1.6%; 0%  % worst score; % best score  Spinal Column (n=187)  0.80 (1.00) -2.77 to 2.92  0.74 (1.03) -2.60 to 3.00  1.01 (1.28) -3.38 to 3.00  0.94 (1.53) -4.00 to 4.00  0.31 (1.48) -3.33 to 3.00  1.03 (1.38) -3.00 to 4.00  0.18 (0.85) -1.25 to 4.14  0.60 (1.57) -1.16 to 10.06  mean (SD) range  0%; 0%  0%; 0%  0%; 0%  251  0.5%; 0.5%  0%; 0%  0%; 0.5%  0%; 0%  0.9%; 0%  % worst score; % best score  Spinal Degenerative (n=213)  50.73 (26.76) 0 to 100.00 27.03 (29.86) 0 to 100.00 52.07 (31.66) 0 to 100.00 (n=58) 29.84 (26.44) 0 to 100.00 (n=87) 22.70 (21.39) 0 to 100.00 36.01 (21.12) 0 to 91.67 34.76 (14.98) 1.09 to 76.09 (n=58)  Self-Care  Life Activities (Non-working)  Life Activities (Working)  Getting Along with People  Participation in Society  Total Score (Non-working)  13.03 (17.27) 0 to 80.00  mean (SD) range  0%; 0%  0%; 4.8%  0.7%; 21.4%  2.3%; 17.2%  15.5%; 6.9%  2.8%; 40.7%  3.4%; 5.5%  0%; 40.0%  % worst score; % best score  SCI (n=145)  Getting Around  WHODAS II (0 to 100) Understanding & Communicating  Instrument (score range)  23.39 (17.02) 0 to 56.52 (n=33)  20.48 (20.32) 0 to 79.17  12.25 (18.50) 0 to 66.67  16.21 (20.84) 0 to 95.83 (n=154)  34.24 (26.58) 0 to 90.00 (n=33)  6.79 (12.93) 0 to 60.00  17.05 (20.12) 0 to 75.00  9.63 (14.70) 0 to 70.00  mean (SD) range  0%; 9.1%  0%; 28.9%  0%; 54.5%  0%; 46.1%  0%; 27.3%  0%; 71.7%  0%; 40.1%  0%; 53.5%  % worst score; % best score  Spinal Column (n=187)  28.98 (18.16) 0 to 71.74 (n=71)  26.39 (23.17) 0 to 91.67  14.87 (18.72) 0 to 75.00  22.51 (24.06) 0 to 100.00 (n=142)  45.49 (31.11) 0 to 100.00 (n=71)  10.80 (18.45) 0 to 100.00  30.69 (25.76) 0 to 100.00  11.90 (17.48) 0 to 70.00  mean (SD) range  0%; 1.4%  0%; 16.0%  0%; 40.4%  252  1.4%; 28.9%  11.3%; 12.7%  0.5%; 65.3%  0.5%; 17.8%  0%; 48.8%  % worst score; % best score  Spinal Degenerative (n=213)  28.11 (19.29) 0 to 84.91 (n=87)  mean (SD) range 0%; 3.4%  % worst score; % best score  SCI (n=145)  13.25 (14.42) 0 to 66.98 (n=154)  mean (SD) range 0%; 22.1%  % worst score; % best score  Spinal Column (n=187)  17.51 (17.23) 0 to 81.13 (n=142)  mean (SD) range  0%; 7.7%  % worst score; % best score  Spinal Degenerative (n=213)  Notes: * the score range for the POPS objective domains varies for each domain  253  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SCI, spinal cord injury; SD, standard deviation; WHODAS II, World Health Organization Disability Assessment Schedule II  Total Score (Working)  WHODAS II cont.  Instrument (score range)  Table C.2. Internal consistency for the participation instruments for the three spine groups Instrument  SCI (n=145)  Spinal Column (n=187)  Spinal Degenerative (n=213)  0.93 0.92 0.94 0.86  0.94 0.95 0.95 0.92  0.95 0.95 0.96 0.92  0.96 (n=76)  0.94 (n=144)  0.96 (n=130)  KAP  NA  NA  NA  POPS  NA  NA  NA  0.90 0.91 0.83 0.79  0.91 0.89 0.83 0.84  0.92 0.93 0.92 0.88  0.91 0.93 (n=53) 0.80 (n=25) 0.86 0.89  0.93 0.90 (n=137) 0.84 (n=27) 0.80 0.92  0.92 0.87 (n=109) NA (n=12) 0.82 0.89  0.89  0.90  0.90  0.79 0.84 0.90 (n=58) 0.93 (n=87) 0.78  0.84 0.76 0.90 (n=33) 0.91 (n=154) 0.85  0.85 0.85 0.91 (n=72) 0.92 (n=141) 0.80  0.85  0.90  0.92  IPA Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education  PM-PAC Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment Education Economic Life Community, Social & Civic Life WHODAS II Understanding & Communicating Getting Around Self-Care Life Activities (Non-working) Life Activities (Working) Getting Along with People Participation in Society  254  Instrument WHODAS II cont. Total Score (Non-working) Total Score (Working)  SCI (n=145)  Spinal Column (n=187)  Spinal Degenerative (n=213)  0.90 (n=58) 0.96 (n=87)  0.95 (n=33) 0.96 (n=154)  0.95 (n=71) 0.96 (n=142)  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SCI, spinal cord injury; WHODAS II, World Health Organization Disability Assessment Schedule II  255  Table C.3. Test-retest reliability for the three spine groups Instrument  IPA Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work &Education PM-PAC Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment Education Economic Life Community, Social & Civic Life POPS Obj Domestic Life Obj Major Life Areas Obj Transportation Obj Interpersonal Interactions & Relationships Obj Community, Recreational & Civic Life Obj Participation Total Subj Domestic Life Subj Major Life Areas Subj Transportation Subj Interpersonal Interactions & Relationships Subj Community, Recreational & Civic Life Subj Participation Total  SCI (n=38)  Spinal Column (n=45)  Spinal Degenerative (n=54)  ICC (95% CI)  ICC (95% CI)  ICC (95% CI)  0.78 (0.62, 0.88) 0.84 (0.71, 0.91) 0.78 (0.62, 0.88) 0.73 (0.54, 0.85)  0.91 (0.84, 0.95) 0.89 (0.80, 0.94) 0.81 (0.67, 0.89) 0.88 (0.80, 0.93)  0.82 (0.71, 0.89) 0.90 (0.83, 0.94) 0.90 (0.83, 0.94) 0.85 (0.75, 0.91)  0.86 (0.64, 0.95)  0.80 (0.61, 0.90)  0.90 (0.80, 0.95)  0.80 (0.65, 0.89) 0.81 (0.65, 0.90) 0.68 (0.47, 0.82) 0.78 (0.62, 0.88)  0.51 (0.25, 0.70) 0.89 (0.81, 0.94) 0.77 (0.61, 0.87) 0.66 (0.45, 0.80)  0.55 (0.34, 0.71) 0.95 (0.92, 0.97) 0.90 (0.83, 0.94) 0.81 (0.70, 0.89)  0.60 (0.35, 0.77) 0.61 (0.18, 0.85) (n=15) NA (n=5) 0.71 (0.51, 0.84) 0.86 (0.74, 0.92)  0.91 (0.84, 0.95) 0.94 (0.88, 0.97) (n=29) NA (n=6) 0.69 (0.49, 0.82) 0.73 (0.55, 0.84)  0.70 (0.53, 0.81) 0.58 (0.25, 0.79) (n=25) NA (n=2) 0.82 (0.72, 0.89) 0.84 (0.74, 0.90)  0.93 (0.88, 0.97) 0.81 (0.67, 0.90) 0.87 (0.76, 0.93) 0.28 (0.0, 0.55)  0.90 (0.82, 0.94) 0.86 (0.76, 0.92) 0.85 (0.74, 0.92) 0.45 (0.19, 0.66)  0.87 (0.79, 0.92) 0.90 (0.83, 0.94) 0.60 (0.40, 0.74) 0.80 (0.68, 0.88)  0.62 (0.38, 0.78)  0.93 (0.87, 0.96)  0.50 (0.28, 0.67)  0.70 (0.50, 0.83) 0.64 (0.41, 0.80) 0.62 (0.37, 0.79) 0.39 (0.07, 0.63) 0.65 (0.42, 0.80)  0.78 (0.62, 0.87) 0.60 (0.39, 0.76) 0.79 (0.57, 0.86) 0.81 (0.67, 0.89) 0.73 (0.55, 0.84)  0.87 (0.79, 0.92) 0.72 (0.56, 0.82) 0.61 (0.38, 0.76) 0.64 (0.45, 0.77) 0.75 (0.61, 0.85)  0.55 (0.27, 0.74)  0.53 (0.28, 0.71)  0.67 (0.49, 0.79)  0.76 (0.59, 0.87)  0.82 (0.70, 0.90)  0.84 (0.74, 0.90) 256  Instrument  WHODAS II Understanding & Communicating Getting Around Self-Care Life Activities (Non-working) Life Activities (Working) Getting Along with People Participation in Society Total Score (Non-working) Total Score (Working) KAP Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6 Interpersonal Interactions & Relationships Economic Life Work Education Community, Social & Civic Life  SCI (n=38)  Spinal Column (n=45)  Spinal Degenerative (n=54)  ICC (95% CI)  ICC (95% CI)  ICC (95% CI)  0.49 (0.21, 0.69)  0.80 (0.66, 0.88)  0.90 (0.83, 0.94)  0.88 (0.77, 0.93) 0.87 (0.76, 0.93) 0.54 (0.07, 0.81) (n=17) 0.88 (0.72, 0.95) (n=21) 0.79 (0.63, 0.88)  0.71 (0.53, 0.83) 0.67 (0.47, 0.81) 0.78 (0.44, 0.93) (n=13) 0.92 (0.85, 0.96) (n=32) 0.70 (0.51, 0.82)  0.92 (0.87, 0.95) 0.85 (0.75, 0.91) 0.89 (0.77, 0.95) (n=23 ) 0.81 (0.65, 0.90) (n=33) 0.67 (0.50, 0.79)  0.83 (0.69, 0.91) 0.69 (0.34, 0.88) (n=17) 0.89 (0.76, 0.96) (n=21)  0.75 (0.52, 0.86) 0.88 (0.65, 0.96) (n=13) 0.91 (0.82, 0.96) (n=32)  0.91 (0.85, 0.95) 0.97 (0.93, 0.99) (n=23) 0.88 (0.76, 0.94) (n=33)  Weighted Kappa Weighted Kappa (95% CI) (95% CI) 0.52 (0.28, 0.76) 0.49 (0.14, 0.84) 0.49 (0.26, 0.72) 0.39 (0.12, 0.65) 0.39 (0.16, 0.63) 0.34 (0, 0.70) 0.33 (0.13, 0.52) 0.70 (0.52, 0.89) 0.43 (0.18, 0.69) 0.47 (0.21, 0.74) 0.57 (0.03, 1.00) 0.75 (0.55, 0.94) (n=13) (n=21) 0.57 (0.33, 0.81) 0.65 (0.47, 0.82)  Weighted Kappa (95% CI) 0.69 (0.45, 0.92) 0.73 (0.57, 0.88) 0.74 (0.56, 0.92) 0.73 (0.54, 0.92) 0.54 (0.33, 0.75) 0.88 (0.71, 1.00) (n=26) 0.72 (0.53, 0.92)  0.35 (0.03, 0.67) 0.64 (0.23, 1.00) (n=20) NA (n=9) 0.70 (0.50, 0.90)  0.43 (0.12, 0.73) 0.52 (0.23, 0.81) (n=27) NA (n=14) 0.69 (0.51, 0.86)  0.71 (0.46, 0.97) 0.84 (0.68, 1.00) (n=28) NA (n=16) 0.60 (0.35, 0.85  Abbreviations: CI, confidence interval; ICC, intraclass correlation coefficient; IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SCI, spinal cord injury; WHODAS II, World Health Organization Disability Assessment Schedule II  257  0.27 0.66  POPS Obj Domestic Life Obj Major Life Areas  Education  Community, Social & Civic Life  0.40 0.47 0.50 0.54 0.44 0.28 (n=13) 0.38 0.72 0.72 (n=20) NA (n=9) 0.38  Interpersonal Interactions & Relationships Economic Life Work  KAP Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6  0.75 1.82  1.07  NA  1.99 2.00  1.05  1.10 1.31 1.39 1.49 1.23 0.77  0.29 0.56  0.22 0.23 (n=28) 0.83 (n=16) 0.42  0.26 0.35 0.26 0.30 0.39 0.31 (n=21) 0.37  0.81 1.56  1.16  2.30  0.61 0.64  1.01  0.72 0.97 0.72 0.84 1.09 0.85  0.29 0.47  0.76 0.45 (n=27) 0.82 (n=14) 0.39  0.30 0.33 0.21 0.33 0.37 0.19 (n=26) 0.26  0.79 1.29  1.07  2.28  2.11 1.25  0.72  0.83 0.91 0.59 0.92 1.01 0.54  Table C.4. Standard error of measurement and minimal detectable change for the three spine groups SCI Spinal Column Spinal Degenerative Instrument (n=38) (n=45) (n=56) SEM MDC SEM MDC SEM MDC IPA Autonomy Indoors 0.33 0.93 0.13 0.35 0.26 0.72 Family Role 0.36 0.99 0.24 0.67 0.30 0.83 Autonomy Outdoors 0.53 1.46 0.40 1.12 0.36 0.99 Social Life & Relationships 0.35 0.97 0.23 0.64 0.25 0.69 Work & Education 0.41 1.14 0.36 0.99 0.28 0.79 (n=15) (n=31) (n=27)  258  Economic Life Community, Social & Civic Life  Education  PM-PAC Communication Mobility Domestic Life Interpersonal Relationships Role Functioning Work & Employment  POPS cont. Obj Transportation Obj Interpersonal Interactions & Relationships Obj Community, Recreational & Civic Life Obj Participation Total Subj Domestic Subj Major Life Areas Subj Transportation Subj Interpersonal Interactions & Relationships Subj Community, Recreational & Civic Life Subj Participation Total  Instrument  0.98 2.02 2.88 3.25 1.68 1.73 1.07  0.35 0.73 1.04 1.17 0.61 0.62 0.38  0.96 0.90  NA  0.47 1.14 1.39 1.02 2.04 1.75  1.54  0.56  0.17 0.41 0.50 0.37 0.74 0.63 (n=15) NA (n=5) 0.35 0.33  0.63 3.67  MDC  0.23 1.32  SEM  SCI (n=38)  0.28 0.16 0.25 0.50 0.32 0.23 (n=29) NA (n=6) 0.26 0.34  0.39  0.70  0.33 0.64 0.78 0.66 0.60  0.42  0.17 1.33  0.71 0.95  NA  0.77 0.45 0.70 1.39 0.89 0.65  1.07  1.94  0.90 1.76 2.17 1.82 1.67  1.16  0.48 3.68  Spinal Column (n=45) SEM MDC  0.35 0.19 0.25 0.39 0.64 0.42 (n=25) NA (n=2) 0.30 0.34  0.49  0.68  0.33 0.72 1.21 1.06 0.72  1.37  0.27 0.99  0.83 0.94  NA  0.98 0.54 0.68 1.08 1.76 1.16  1.37  1.88  0.92 1.99 3.36 2.94 1.99  3.79  0.74 2.74  Spinal Degenerative (n=56) SEM MDC  259  24.79 24.82 63.91  8.95 8.96 23.07 (n=17) 7.57 (n=21) 7.65 8.28 7.79 (n=17) 5.32 (n=21) 14.74  21.18 22.94 21.59  20.97  22.01  MDC  7.95  SEM  SCI (n=38)  9.15 3.80 9.41 (n=13) 6.04 (n=32) 9.29 9.63 5.04 (n=13) 4.05 (n=32)  6.41  11.22  25.75 26.68 13.96  16.73  25.35 10.53 26.05  17.76  Spinal Column (n=45) SEM MDC  6.85 5.35 10.27 (n=23) 8.53 (n=33) 9.67 6.14 3.27 (n=23) 4.84 (n=33)  4.99  13.40  26.79 17.01 9.05  23.62  18.97 14.82 28.46  13.81  Spinal Degenerative (n=56) SEM MDC  260  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; MDC, minimal detectable change; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SCI, spinal cord injury; SEM, standard error of measurement; WHODAS II, World Health Organization Disability Assessment Schedule II  Total Score (Working)  WHODAS II Understanding & Communicating Getting Around Self-Care Life Activities (Non-working) Life Activities (Working) Getting Along with People Participation in Society Total Score (Non-working)  Instrument  Figure C.1. IPA Bland and Altman Plots  261  262  Figure C.2. KAP Bland and Altman Plots  263  264  265  Figure C.3. PM-PAC Bland and Altman Plots  266  267  268  Figure C.4. POPS Bland and Altman Plots  269  270  271  Figure C.5. WHODAS II Bland and Altman Plots  272  273  274  APPENDIX D. ADDITIONAL ANALYSES RELATED TO CHAPTER 5 (VALIDITY)  275  Table D.1. Descriptive summary for the disease-specific and Life Satisfaction-11 instruments for the three spine groups Instrument (score range)  NDI (0 to 5)  SCI (n=145) mean (SD) range NA  Spinal Column (n=187) mean (SD) range 0.94 (0.78) 0 to 3.50  Spinal Degenerative (n=213) mean (SD) range 1.45 (0.84) 0 to 3.20  ODQ (0 to 5)  NA  0.88 (0.79) 0 to 3.00  1.31 (0.92) 0 to 3.60  SRFM (1 to 4)  1.72 (0.71) 1.00 to 4.00  NA  NA  2.68 (1.31) 1.00 to 6.00 2.88 (1.38) 1.00 to 6.00 3.06 (1.47) 1.00 to 6.00 2.89 (1.30) 1.00 to 6.00 2.45 (1.15) 1.00 to 6.00 4.06 (1.70) 1.00 to 6.00 2.40 (1.40) 1.00 to 6.00 2.08 (1.02) 1.00 to 6.00 1.90 (1.08) 1.00 to 6.00 3.11 (1.32) 1.00 to 6.00 2.51 (1.29) 1.00 to 6.00  2.05 (1.08) 1.00 to 6.00 2.35 (1.26) 1.00 to 6.00 2.66 (1.30) 1.00 to 6.00 2.43 (1.21) 1.00 to 6.00 2.18 (1.06) 1.00 to 5.00 3.05 (1.66) 1.00 to 6.00 1.66 (0.82) 1.00 to 5.00 1.83 (0.84) 1.00 to 5.00 1.63 (0.82) 1.00 to 4.00 2.60 (1.27) 1.00 to 6.00 2.30 (1.20) 1.00 to 6.00  2.46 (1.33) 1.00 to 6.00 2.54 (1.38) 1.00 to 6.00 2.74 (1.40) 1.00 to 6.00 2.80 (1.44) 1.00 to 6.00 2.28 (1.18) 1.00 to 6.00 3.31 (1.68) 1.00 to 6.00 1.74 (1.01) 1.00 to 6.00 2.03 (1.11) 1.00 to 6.00 1.90 (1.10) 1.00 to 6.00 3.24 (1.40) 1.00 to 6.00 2.54 (1.42) 1.00 to 6.00  LiSat-11 (1 to 6) Overall Vocation Financial Leisure Contact with Friends Sexual Life Self-Care Family Life Partner Relationship Physical Health Mental Health  Abbreviations: LiSat-11, Life Satisfaction-11; NA, not applicable; NDI, Neck Disability Index; ODQ; Oswestry Disability Questionnaire; SD, standard deviation; SRFM, Self-Reported Functional Measure 276  Table D.2a. CFA results for IPA baseline model for the entire sample (n=545) Model Indices Satorra-Bentler Scaled Chi-Squared χ2 (df) p-value RMSEA (90% CI for RMSEA) CFI SRMR Standardized Loadings for Items Autonomy Indoors 1a 1b 1c 1d 1e 1f 1g Family Role 2a 2b 2c 2d 2e 2f 2g Autonomy Outdoors 3a 3b 3c 3d 3e Social Life and Relationships 4a 4b 4c 4d 4e 4f Correlations Among Latent Factors AI 1.00 AI 0.81 FR 0.78 AO 0.66 SR  1112.11 (269) <0.001 0.076 (0.071; 0.081) 0.99 0.054 0.85 0.87 0.91 0.91 0.84 0.81 0.75 0.90 0.82 0.86 0.88 0.87 0.91 0.70 0.88 0.91 0.91 0.91 0.89 0.82 0.84 0.79 0.65 0.90 0.91 FR  AO  SR  1.00 0.88 0.71  1.00 0.77  1.00 277  Abbreviations: AI, Autonomy Indoors factor; AO, Autonomy Outdoors factor; CI, confidence interval; CFA, confirmatory factor analysis; CFI, comparative fit index; df, degrees of freedom; FR, Family Role factor; RMSEA, root mean square error of approximation; SR, Social Life and Relationships factor; SRMR, standardized root mean residual  278  Table D.2b. CFA results for IPA adjusted model for the entire sample (n=545) Model Indices Satorra-Bentler Scaled Chi-Squared χ2 (df) p-value RMSEA (90% CI for RMSEA) CFI SRMR Standardized Loadings for Items Autonomy Indoors 1a 1b 1c 1d 1e 1f 1g Family Role 2a 2b 2c 2d 2e 2f 2g Autonomy Outdoors 3a 3b 3c 3d 3e Social Life and Relationships 4a 4b 4c 4d 4e 4f Correlations Among Latent Factors AI FR 1.00 AI 0.80 1.00 FR 0.77 0.88 AO 0.63 0.70 SR  986.08 (269) <0.001 0.071 (0.066; 0.075) 0.99 0.060 0.85 0.88 0.93 0.91 0.84 0.79 0.73 0.90 0.82 0.86 0.88 0.87 0.91 0.70 0.88 0.91 0.91 0.91 0.89 0.86 0.89 0.84 0.63 0.81 0.82 AO  SR  1.00 0.76  1.00  279  Abbreviations: AI, Autonomy Indoors factor; AO, Autonomy Outdoors factor; CFA, confirmatory factor analysis; CI, confidence interval; CFI, comparative fit index; df, degrees of freedom; FR, Family Role factor; RMSEA, root mean square error of approximation; SR, Social Life and Relationships factor; SRMR, standardized root mean residual  280  Table D.3a. CFA results for PM-PAC baseline model for the entire sample (n=510)* Model Indices Satorra-Bentler Scaled Chi-Squared χ2 (df) p-value RMSEA (90% CI for RMSEA) CFI SRMR Standardized Loadings for Items Mobility 1 2a 2b 2c 2d Role Functioning 5a 5b 5c 5d Community, Social and Civic Life† 10a 10b 10f 10n 11 Domestic Life 16a 16b 16c Interpersonal Relationships 13a 13b 13c Economic Life 16d 16e 16f Communication‡ 10b 10c 10l 10m  864.31 (303) <0.001 0.060 (0.056; 0.065) 0.99 0.065 0.80 0.76 0.91 0.94 0.83 0.76 0.85 0.94 0.89 0.74 0.76 0.80 0.78 0.79 0.78 0.78 0.82 0.87 0.92 0.71 0.67 0.80 0.88 0.64 0.62 0.86 0.83  281  Correlations Among Latent Factors Mob RF CSC 1.00 Mob 0.60 1.00 RF 0.80 0.80 1.00 CSC 0.78 0.71 0.82 DL 0.47 0.52 0.61 IR 0.53 0.41 0.53 EL 0.47 0.47 0.69 Com  DL  IR  EL  Com  1.00 0.57 0.70 0.61  1.00 0.39 0.52  1.00 0.67  1.00  Abbreviations: CI, confidence interval; CFA, confirmatory factor analysis; CFI, comparative fit index; Com, Communication factor; CSC, Community, Social and Civic Life factor; df, degrees of freedom; DL, Domestic Life factor; EL, Economic Life factor; IR, Interpersonal Relationships factor; Mob, Mobility factor; PM-PAC, Participation Measure-Post Acute Care; RF, Role Functioning factor; RMSEA, root mean square error of approximation; SRMR, standardized root mean residual Notes: * some subjects were removed for this analysis if they answered ‘don’t do this/not applicable’ for questions in the communication and community, social and civic life domains † does not include questions 10e, 10g, 10h, 10i ‡ does not include questions 10j and 10k  282  Table D.3b. CFA results for PM-PAC adjusted model for the entire sample (n=510)* Model Indices Satorra-Bentler Scaled Chi-Squared χ2 (df) p-value RMSEA (90% CI for RMSEA) CFI SRMR Standardized Loadings for Items Mobility 1 2a 2b 2c 2d Role Functioning 5a 5b 5c 5d Community, Social and Civic Life† 10a 10b 10f 10n 11 Domestic Life 16a 16b 16c Interpersonal Relationships 13a 13b 13c Economic Life 16d 16e 16f Communication‡ 10b 10c 10l 10m  755.64 (302) <0.001 0.054 (0.049; 0.059) 0.99 0.064 0.80 0.76 0.91 0.94 0.83 0.76 0.85 0.94 0.89 0.74 0.76 0.80 0.79 0.79 0.78 0.78 0.82 0.87 0.92 0.71 0.67 0.80 0.88 0.57 0.56 0.87 0.85  283  Correlations Among Latent Factors Mob RF CSC 1.00 Mob 0.60 1.00 RF 0.80 0.80 1.00 CSC 0.78 0.71 0.82 DL 0.47 0.52 0.61 IR 0.53 0.41 0.53 EL 0.47 0.47 0.69 Com  DL  IR  EL  Com  1.00 0.57 0.70 0.60  1.00 0.39 0.53  1.00 0.66  1.00  Abbreviations: CI, confidence interval; CFA, confirmatory factor analysis; CFI, comparative fit index; Com, Communication factor; CSC, Community, Social and Civic Life factor; df, degrees of freedom; DL, Domestic Life factor; EL, Economic Life factor; IR, Interpersonal Relationships factor; Mob, Mobility factor; PM-PAC, Participation Measure-Post Acute Care; RF, Role Functioning factor; RMSEA, root mean square error of approximation; SRMR, standardized root mean residual; UC, Understanding and Communicating factor Notes: * some subjects were removed for this analysis if they answered ‘don’t do this/not applicable’ for questions in the communication and community, social and civic life domains † does not include questions 10e, 10g, 10h, 10i ‡ does not include questions 10j and 10k  284  Table D.4a. CFA results for WHODAS II baseline model for the entire sample (n=545) Model Indices Normal Theory Chi-Squared χ2 (df) p-value RMSEA (90% CI for RMSEA) CFI SRMR Standardized Loadings for Items on First Order Factor Understanding and Communicating 2a 2b 2c 2d 2e 2f Getting Around 3a 3b 3c 3d 3e Self-Care 4a 4b 4c 4d Getting Along with People 5a 5b 5c 5d 5e Participation in Society 8a 8b 8c 8d 8e 8f 8g 8h  1475.46 (345) <0.001 0.078 (0.074; 0.082) 0.97 0.090 0.78 0.79 0.88 0.79 0.73 0.75 0.86 0.78 0.67 0.68 0.87 0.90 0.90 0.63 0.87 0.80 0.86 0.69 0.84 0.53 0.78 0.73 0.69 0.75 0.80 0.73 0.72 0.75  285  Standardized Loadings for First Order Factor on Second Order Factor Understanding and Communicating 0.73 Getting Around 0.76 Self-Care 0.72 Getting Along with People 0.72 Participation in Society 0.91 Correlations Among Latent Factors UC GA 1.00 UC 0.55 1.00 GA 0.53 0.55 SC 0.52 0.54 GAP 0.67 0.69 PS 0.73 0.76 Disab  SC  GAP  PS  Disab  1.00 0.52 0.66 0.72  1.00 0.65 0.72  1.00 0.91  1.00  Abbreviations: CI, confidence interval; CFA, confirmatory factor analysis; CFI, comparative fit index; df, degrees of freedom; Disab, General Disability factor; GA, Getting Around factor; GAP, Getting Along with People factor; PS, Participation in Society factor; RMSEA, root mean square error of approximation; SC, Self-Care factor; SRMR, standardized root mean residual; UC, Understanding and Communicating factor; WHODAS II, World Health Organization Disability Assessment Schedule II  286  Table D.4b. CFA results for WHODAS II adjusted model for the entire sample (n=545) Model Indices Normal Theory Chi-Squared χ2 (df) p-value RMSEA (90% CI for RMSEA) CFI SRMR Standardized Loadings for Items on First Order Factor Understanding and Communicating 2a 2b 2c 2d 2e 2f Getting Around 3a 3b 3c 3d 3e Self-Care 4a 4b 4c 4d Getting Along with People 5a 5b 5c 5d 5e Participation in Society 8a 8b 8c 8d 8e 8f 8g 8h  1521.27 (342) <0.001 0.069 (0.065; 0.073) 0.98 0.085 0.78 0.79 0.88 0.79 0.73 0.75 0.77 0.74 0.70 0.71 0.81 0.90 0.90 0.63 0.77 0.80 0.86 0.69 0.84 0.53 0.78 0.73 0.69 0.75 0.80 0.73 0.72 0.75  287  Standardized Loadings for First Order Factor on Second Order Factor Understanding and Communicating 0.73 Getting Around 0.81 Self-Care 0.73 Getting Along with People 0.72 Participation in Society 0.90 Correlations Among Latent Factors UC GA 1.00 UC 0.60 1.00 GA 0.54 0.59 SC 0.53 0.58 GAP 0.66 0.73 PS 0.73 0.81 Disab  SC  GAP  PS  Disab  1.00 0.53 0.66 0.73  1.00 0.65 0.72  1.00 0.90  1.00  Abbreviations: CFA, confirmatory factor analysis; CFI, comparative fit index; df, degrees of freedom; Disab, General Disability factor; GA, Getting Around factor; GAP, Getting Along with People factor; PS, Participation in Society factor; RMSEA, root mean square error of approximation; SC, Self-Care factor; SRMR, standardized root mean residual; UC, Understanding and Communicating factor; WHODAS II, World Health Organization Disability Assessment Schedule II  288  WHODAS II Self-Care IPA Autonomy Indoors KAP Self-Care  Self-Care (d5)  WHODAS II Getting Around IPA Autonomy Indoors KAP Mobility #1 KAP Mobility #2 POPS Obj Transport POPS Subj Transport PM-PAC Mobility  Mobility (d4)  WHODAS II Understanding & Communicating PM-PAC Communication  Communication (d3)  -0.46  1.00  1.00  0.66 1.00  IPA Autonomy Indoors  0.57 0.63 1.00  KAP Self-Care  0.61 1.00  1.00  WHODAS II Self-Care  0.55  0.68  1.00  KAP Mobility #1  IPA Autonomy Indoors  WHODAS II Getting Around  1.00  PM-PAC Communication  WHODAS II Understanding & Communicating  0.66 0.76 1.00  0.60  KAP Mobility #2  Table D.5. Correlations* among similar participation domains for the entire sample (n=545)  -0.17 -0.16 -0.20 1.00  -0.15  POPS Obj Transport  -0.12 -0.20 -0.20 0.02† 1.00  -0.10  POPS Subj Transport  -0.71 -0.58 -0.65 0.22 0.11 1.00  -0.73  289  PM-PAC Mobility  0.64 0.66 1.00  0.78 1.00  0.63  KAP Domestic Life #4 0.45  IPA Family Role  WHODAS II Getting Along with People IPA Social Life & Relationships KAP Interpersonal Interactions & Relationships  1.00 1.00  0.69  Interpersonal Interactions and Relationships (d7) WHODAS II IPA Getting Along Social Life & with People Relationships  WHODAS II Life Activities (Non-work; n=163) WHODASII Life Activities (Working; n=383) IPA Family Role KAP Domestic Life #4 KAP Domestic Life #5 KAP Domestic Life #6 POPS Obj Domestic Life POPS Subj Domestic Life PM-PAC Domestic Life  Domestic Life (d6)  -0.26 -0.20  1.00  POPS Obj Interpersonal Interactions & Relationships -0.30  -0.33 -0.12 -0.17 -0.21 1.00  -0.18  -0.39  0.46  1.00  -0.35  -0.41  -0.52  -0.63  -0.52  290  PM-PAC Interpersonal Relationships  1.00  -0.74 -0.57 -0.57 -0.58 0.26  -0.68  -0.56  PM-PAC Domestic Life  -0.49 -0.44 -0.40 -0.47 0.34  -0.41  -0.56  POPS Subj Domestic Life  POPS Subj Interpersonal Interactions & Relationships -0.40  POPS Obj Domestic Life  0.66  0.65 0.64 0.56 1.00  0.59  KAP Domestic Life #6 0.59  KAP Interpersonal Interactions & Relationships 0.51  0.64 0.77 1.00  0.60  KAP Domestic Life #5 0.45  KAP Education  Major Life Areas (Work and Education) (d8) WHODASII IPA Life Work & Activities Education (Working) 1.00 0.74 WHODASII Life Activities (n=332) (Working; n=383) IPA Work & 1.00 Education KAP Work  POPS Obj Interpersonal Interactions & Relationships POPS Subj Interpersonal Interactions & Relationships PM-PAC Interpersonal Relationships KAP Education 0.49 (n=167) 0.55 (n=160) 0.58 (n=158) 1.00  0.52 (n=289) 0.55 (n=274) 1.00  -0.31  -0.31  -0.24 -0.37  -0.31  -0.26 (n=383)  POPS Subj Major Life Areas  POPS Obj Interpersonal Interactions & Relationships 1.00  -0.35  -0.25 (n=383)  POPS Obj Major Life Areas  KAP Interpersonal Interactions & Relationships  KAP Work  Interpersonal Interactions and Relationships (d7) cont. WHODAS II IPA Getting Along Social Life & with People Relations  -0.75 (n=296) -0.48 (n=253) -0.55 (n=139)  -0.67 (n=298)  PM-PAC Work & Employ  -0.75 (n=58) -0.55 (n=52) -0.76 (n=59)  -0.73 (n=58)  PM-PAC Education  0.52  1.00  291  -0.70 (n=356) -0.53 (n=327) -0.52 (n=193)  -0.69 (n=383)  PM-PAC Role Functioning  1.00  0.22  PM-PAC Interpersonal Relationships  POPS Subj Interpersonal Interactions & Relationships 0.32  Major Life Areas (Economic Life) (d8) KAP PM-PAC Economic Economic Life Life 1.00 -0.41 KAP Economic Life 1.00 PM-PAC Economic Life  Major Life Areas (Work and Employment) (d8) cont. WHODASII IPA KAP Life Work & Work Activities Education (Working) POPS Obj Major Life Areas POPS Subj Major Life Areas PM-PAC Work & Employ PM-PAC Education PM-PAC Role Functioning KAP Education  0.38 (n=63) 0.75 (n=43) 1.00  0.17 1.00  1.00  0.40 (n=63)  0.14  0.18  PM-PAC Education  1.00  PM-PAC Work & Employ  POPS Subj Major Life Areas  POPS Obj Major Life Areas  292  0.73 (n=299) 0.72 (n=63) 1.00  0.27  0.39  PM-PAC Role Functioning  1.00 0.72 1.00  1.00  0.77  KAP Community, Social & Civic Life 0.62  IPA Autonomy Outdoors  0.24  1.00  1.00  0.19  0.33  1.00  -0.80 -0.65  PM-PAC Community, Social & Civic Life -0.71  -0.30 -0.30  POPS Subj Community, Recreational & Civic Life -0.28  -0.25 -0.13  POPS Obj Community, Recreational & Civic Life -0.21  Notes: * Spearman’s rho correlation was used † non-significant correlation  293  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; WHODAS II, World Health Organization Disability Assessment Schedule II  WHODAS II Participation in Society IPA Autonomy Outdoors KAP Community, Social & Civic Life POPS Obj Community, Recreational & Civic Life POPS Subj Community, Recreational & Civic Life PM-PAC Community, Social & Civic Life  Community, Social and Civic Life (d9) WHODAS II Participation in Society  0.56 0.58  0.40 0.41 0.40 0.41 0.44 0.48 (n=286) 0.45 0.28 0.44 (n=327) 0.38 (n=193)  0.62 0.60  0.44 0.49 0.43 0.44 0.43 0.51 (n=286) 0.48 0.30 0.36 (n=327) 0.36 (n=193)  Mobility #1 Mobility #2 Self-Care Domestic Life #4 Domestic Life #5 Domestic Life #6  Interpersonal Interactions & Relationships Economic Life Work  Education  KAP  0.54 0.58  0.60 0.65  Autonomy Indoors Family Role Autonomy Outdoors Social Life & Relationships Work & Education  IPA  0.35 0.33 (n=327) 0.31 (n=193)  0.39  0.32 0.34 0.34 0.37 0.40 0.40 (n=286)  0.47  0.45  0.45 0.50  0.41  0.49  0.60  0.52 0.60  0.49  Contact with Friends  0.54  0.30 0.32 0.39 0.41 (n=327) (n=327) 0.39 0.44 (n=193) (n=193)  0.49  0.46 0.40 0.52 0.47 0.41 0.37 0.48 0.41 0.46 0.44 0.49 0.44 (n=286) (n=286)  0.57  0.56  0.59 0.67  0.55  Vocation Financial Leisure 0.50  Overall QoL 0.56  LiSat-11  0.32 0.34 (n=327) 0.44 (n=193)  0.43  0.41 0.46 0.38 0.35 0.33 0.44 (n=286)  0.50  0.58  0.52 0.54  0.49  Sexual Life  0.37  0.49  0.41 0.43  0.40  0.41 0.36 0.28 0.32 0.23 (n=327) (n=315) 0.42 0.32 (n=193) (n=180)  0.47  0.12† (n=327) 0.17† (n=134)  0.18  0.26  0.19 0.21 0.23 0.23 0.23 0.25 (n=235)  0.28  0.35  0.26 0.26  0.21  0.24 0.41 (n=327) 0.43 (n=193)  0.42  0.46 0.51 0.40 0.48 0.46 0.46 (n=286)  0.62  0.55  0.61 0.65  0.53  Family Partner Physical Life Relationship Health  0.52 0.32 0.50 0.35 0.54 0.33 0.40 0.34 0.44 0.37 0.49 0.45 (n=286) (n=281)  0.51  0.53  0.59 0.58  0.67  SelfCare  Table D.6. Correlations* among participation domains and LiSat-11 questions for the entire sample (n=545)  294  0.28 0.30 (n=327) 0.32 (n=193)  0.45  0.39 0.44 0.36 0.42 0.43 0.50 (n=286)  0.58  0.55  0.53 0.54  0.42  Mental Health  -0.48 -0.49 -0.54 -0.49 -0.50 (n=299) -0.57 (n=63) -0.35 -0.48  -0.50 -0.51 -0.67 -0.55 -0.50 (n=299) -0.51 (n=63) -0.32 -0.57  Mobility Domestic Life  Role Functioning  Work & Employment  Education  Economic Life Community, Social & Civic Life  Interpersonal Relationships  -0.43  -0.40  Communication  PM-PAC  -0.34 -0.39  -0.40 (n=63)  -0.39 (n=299)  -0.35  -0.39 -0.40 -0.51  -0.32  0.40 (n=412)  Contact with Friends  -0.44  -0.44 -0.48 -0.68  -0.51  -0.31 -0.60  -0.53 (n=63) -0.36 -0.52  -0.63 (n=63)  -0.53 -0.41 (n=299) (n=299)  -0.53  -0.48 -0.55 -0.64  -0.44  0.53 0.51 (n=412) (n=412)  0.46 (n=412)  0.52 (n=412)  Community, Social & Civic Life  Vocation Financial Leisure  Overall QoL  KAP cont.  LiSat-11  -0.32 -0.51  -0.44 (n=63)  -0.34 (n=299)  -0.43  -0.46 -0.46 -0.53  -0.38  0.41 (n=412)  Sexual Life  -0.34  -0.33 -0.42 -0.55  -0.34  -0.42 -0.53  -0.51 (n=63)  -0.32 -0.38  -0.29 (n=58)  -0.35 -0.25 (n=299) (n=286)  -0.42  -0.53 -0.57 -0.48  -0.40  -0.13 -0.26 (n=397)  -0.38 (n=33)  -0.17 (n=224)  -0.22  -0.21 -0.28 -0.43  -0.24 (n=397)  0.19 (n=306)  -0.33 -0.62  -0.47 (n=63)  -0.56 (n=299)  -0.59  -0.51 -0.52 -0.54  -0.41  0.50 (n=412)  Family Partner Physical Life Relationship Health  0.47 0.35 (n=412) (n=393)  SelfCare  295  -0.33 -0.48  -0.41 (n=63)  -0.50 (n=299)  -0.49  -0.38 -0.47 -0.61  -0.41  0.45 (n=412)  Mental Health  POPS  -0.16 -0.25 -0.15 -0.18  -0.16  -0.35 -0.34 -0.14 -0.39  -0.30  -0.29 -0.18 -0.24  -0.16  -0.39 -0.33 -0.10 -0.44  -0.29  Obj Major Life  Obj Transportation  Obj Interpersonal Interactions & Relationships  Obj Community, Recreational & Civic Life  Subj Domestic Life  Subj Major Life  Subj Transportation  Subj Interpersonal Interactions & Relationships  Subj Community, Recreational & Civic Life  -0.34  -0.35  -0.17  -0.33  -0.28  -0.24  -0.11  -0.10  -0.13  -0.09  -0.34  -0.45  -0.15  -0.34  -0.40  -0.18  -0.24  -0.14  -0.18  -0.14  Vocation Financial Leisure  -0.16  Overall QoL  Obj Domestic Life  LiSat-11  -0.37  -0.54  -0.17  -0.34  -0.42  -0.19  -0.28  -0.17  -0.21  -0.15  Contact with Friends  -0.24  -0.41  -0.14  -0.31  -0.35  -0.18  -0.29  -0.14  -0.31  -0.19  Sexual Life  -0.21  -0.28  -0.08†  -0.22  -0.35  -0.19  -0.12  -0.15  -0.29  -0.30  SelfCare  -0.29 (n=511)  -0.35 (n=511)  -0.09 (n=511)  -0.29 (n=511)  -0.33 (n=511)  -0.18 (n=511)  -0.23 (n=511)  -0.16 (n=511)  -0.19 (n=511)  -0.14 (n=511)  -0.15 (n=397)  -0.27 (n=397)  -0.01† (n=397)  -0.18 (n=397)  -0.22 (n=397)  -0.10† (n=397)  -0.12 (n=397)  -0.02† (n=397)  -0.11 (n=397)  -0.02† (n=397)  -0.25  -0.32†  -0.10  -0.29  296  -0.31  -0.39  -0.12  -0.31  -0.34  -0.14  -0.16  -0.32  -0.22  -0.11  -0.23  -0.10  Mental Health  -0.20  -0.10†  -0.30  -0.13  Family Partner Physical Life Relationship Health  0.45 0.41 0.35  0.51  0.51 0.55  0.49 0.43 0.38  0.55  0.50 0.61  Getting Around  Self-Care  Life Activities(Non-working; n=162)  Life Activities – (Working; n=383)  Getting Along with People  Participation in Society  0.53  0.37  0.41  0.35  0.36  0.40  0.58  0.46  0.50  0.41  0.40  0.46  0.42  0.53  0.51  0.45  0.38  0.37  0.39  0.44  Contact with Friends  0.52  0.54  0.45  0.30  0.40  0.46  0.36  Sexual Life  0.52  0.43  0.45  0.47  0.59  0.52  0.36  SelfCare  0.40  0.42  0.30  0.37  0.28  0.30  0.35 (n=511)  0.26  0.31  0.26 (n=283)  0.20 (n=114)  0.10  0.20  0.26 (n=397)  0.65  0.45  0.60  0.43  0.39  0.55  0.42  Family Partner Physical Life Relationship Health  0.57  0.47  0.48  0.34  0.30  0.38  0.47  Mental Health  Notes: * Spearman's rho correlation was used † non-significant correlation  297  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; LiSat-11, Life Satisfaction-11; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; QoL, quality of life; WHODAS II, World Health Organization Disability Assessment Schedule II  WHODAS II Understanding & Communicating  0.41  Vocation Financial Leisure 0.46  Overall QoL 0.43  LiSat-11  Instrument  Domain  Study Variable  Is the hypothesis significant after adjustments (p<0.05)? Is the estimate of the effect in the correct direction?  Overall, is the hypothesis supported?  d6 - Domestic Life & Motor Score  d5 - Self-Care & Motor Score  d4 - Mobility & Motor Score  WHODAS II  IPA KAP KAP KAP PM-PAC POPS POPS WHODAS II  IPA KAP WHODAS II  IPA KAP KAP PM-PAC WHODAS II  Family Role Domestic Life #4 Domestic Life #5 Domestic Life #6 Domestic Life Obj Domestic Life Subj Domestic Life Life Activities (Non-work; n=58) Life Activities (Work; n=87)  Autonomy Indoors Self-Care Self-Care  Autonomy Indoors Mobility #1 Mobility #2 Mobility Getting Around  motor score  motor score motor score motor score motor score motor score motor score motor score motor score  motor score motor score motor score  motor score motor score motor score motor score motor score  yes  yes yes yes no yes yes yes no  yes yes yes  yes yes yes yes yes  yes  yes yes yes NA yes yes yes NA  yes yes yes  yes yes yes yes yes  yes  yes yes yes no yes yes yes no  yes yes yes  yes yes yes yes yes  298  In individuals with SCI, expect lower participation scores in the Mobility (d4), Self-care (d5) and Domestic life (d6) domains in individuals with more neurological impairment (lower motor score). Do not expect any difference in scores for communication. [Note: did not include POPS transportation for Mobility (d4); the magnitude of the effect was based on 10 point motor score increase].  Hypothesis  Table D.7. Known-group validity hypothesis testing*  PM-PAC WHODAS II  WHODAS II  POPS  POPS  Communication Understanding & Communicating  motor score motor score  motor score  motor score  motor score  motor score  motor score  motor score  no no  yes  no  no  yes  no  yes  Is the hypothesis significant after adjustments (p<0.05)?  NA NA  yes  NA  NA  yes  NA  yes  Is the estimate of the effect in the correct direction?  yes yes  yes  no  no  yes  no  yes  Overall, is the hypothesis supported?  d6 - Domestic Life & Traumatic vs Non-traumatic Family Role Domestic Life #4 Domestic Life #5  IPA  KAP  KAP  traumatic vs non-traumatic  traumatic vs non-traumatic traumatic vs non-traumatic  no  no  no  NA  NA  NA  yes  yes  yes  299  For individuals with traumatic SCI versus non-traumatic SCI ( myelopathy), expect no difference in any of the participation domain scores. (Note: selected d6 and d9 as two different types of participation areas)  d3 Communication & Motor Score  KAP  PM-PAC  Autonomy Outdoors Community, Social & Civic Life Community, Social & Civic Life Obj Community, Recreational & Civic Life Subj Community, Recreational & Civic Life Participation in Society  IPA  d9 – Community, Social & Civic Life & Motor Score  Domain  Instrument  Hypothesis  Study Variable  Life Activities (Work)  WHODAS II  POPS  POPS  PM-PAC  Community, Social & Civic life Community, Social & Civic life Obj Community, Recreational & Civic Life Subj Community, Recreational & Civic Life  Life Activities (Non-work)  WHODAS II  KAP  Subj Domestic Life  POPS  Autonomy Outdoors  Obj Domestic Life  POPS  IPA  Domestic Life  PM-PAC  d9 - Community, Social & Civic Life & Traumatic vs Non-traumatic  Domestic Life #6  KAP  d6 - Domestic Life & Traumatic vs Non-traumatic cont.  Domain  Instrument  Hypothesis  traumatic vs non-traumatic traumatic vs non-traumatic traumatic vs non-traumatic traumatic vs non-traumatic traumatic vs non-traumatic  traumatic vs non-traumatic traumatic vs non-traumatic traumatic vs non-traumatic traumatic vs non-traumatic traumatic vs non-traumatic traumatic vs non-traumatic  Study Variable  no  no  no  no  no  no  no  no  no  no  no  Is the hypothesis significant after adjustments (p<0.05)?  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  Is the estimate of the effect in the correct direction?  yes  yes  yes  yes  yes  yes  yes  yes  yes  yes  yes  300  Overall, is the hypothesis supported?  WHODAS II  d9 - Community, Social & Civic Life & Traumatic vs Non-traumatic cont. Participation in Society  Domain  traumatic vs non-traumatic  no  Is the hypothesis significant after adjustments (p<0.05)?  NA  Is the estimate of the effect in the correct direction?  yes  Overall, is the hypothesis supported?  d6 - Domestic Life & Level of Injury Family Role Domestic Life #4 Domestic Life #5 Domestic Life #6 Domestic Life Obj Domestic Life Subj Domestic Life Life Activities (Non-work) Life Activities (Work)  IPA  KAP  KAP  KAP  PM-PAC  POPS  POPS  WHODAS II  WHODAS II  (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar 1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  NA  NA  NA  NA  NA  NA  NA  NA  NA  yes  yes  yes  yes  yes  yes  yes  yes  yes  301  In individuals with spinal column injuries and spinal degenerative conditions, expect no difference based on level of injury in any of the participation domain scores. (Note: selected d6 and d9 as two different types of participation areas)  Instrument  Hypothesis  Study Variable  Community, Social & Civic life Community, Social & Civic life Obj Community, Recreational & Civic Life Subj Community, Recreational & Civic Life Participation in Society  KAP  (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar (1) cervical (2) lumbar 1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  1 vs 2 = no  Is the hypothesis significant after adjustments (p<0.05)?  NA  NA  NA  NA  NA  NA  Is the estimate of the effect in the correct direction?  yes  yes  yes  yes  yes  yes  Overall, is the hypothesis supported?  (d7) -Interpersonal Interactions & Relationships & Back Pain Social Life & Relationships  Interpersonal Interactions & Relationships  IPA  KAP  (1) none (2) moderate (3) fairly severe (1) none (2) moderate (3) fairly severe  1 vs 2 = yes; 1 vs 3 = yes  1 vs 2 = yes; 1 vs 3 = yes  yes; yes  yes; yes  yes; yes  yes; yes  302  In individuals with spinal column injuries and spinal degenerative conditions, expect individuals who report pain (moderate or fairly severe/worst possible) will have lower participation scores in domains related to: Interpersonal interactions and relationships (d7) and Community, social and civic life (d9) compared to individuals not reporting pain.  WHODAS II  POPS  POPS  PM-PAC  Autonomy Outdoors  IPA  d9 - Community, Social & Civic Life & Level of Injury  Domain  Instrument  Hypothesis  Study Variable  Community, Social & Civic life Obj Community, Recreational & Civic Life  PM-PAC  POPS  Community, Social & Civic Life  KAP  Getting Along with People  WHODAS II  Autonomy Outdoors  Subj Interpersonal Interactions & Relationships  POPS  IPA  Obj Interpersonal Interactions & Relationships  POPS  d9 - Community, Social & Civic Life & Back Pain  Interpersonal Relationships  PM-PAC  (d7) -Interpersonal Interactions & Relationships & Back Pain cont.  Domain  Instrument  Hypothesis  (1) none (2) moderate (3) fairly severe (1) none (2) moderate (3) fairly severe (1) none (2) moderate (1) none (2) moderate (3) fairly severe  (1) none (2) moderate (3) fairly severe (1) none (2) moderate (3) fairly severe (1) none (2) moderate (3) fairly severe (1) none (2) moderate (3) fairly severe  Study Variable  1 vs 2 = yes; 1 vs 3 = yes 1 vs 2 = no; 1 vs 3 = yes  1 vs 2 = yes; 1 vs 3 = yes  1 vs 2 = yes; 1 vs 3 = yes  1 vs 2 = yes; 1 vs 3 = yes  1 vs 2 = no; 1 vs 3 = yes  1 vs 2 = no; 1 vs 3 = no  1 vs 2 = yes; 1 vs 3 = yes  Is the hypothesis significant after adjustments (p<0.05)?  NA; yes  yes; yes  yes; yes  yes; yes  yes; yes  NA; yes  NA; NA  yes; yes  Is the estimate of the effect in the correct direction?  no; yes  yes; yes  yes; yes  yes; yes  yes; yes  no; yes  no; no  yes; yes  303  Overall, is the hypothesis supported?  Participation in Society  WHODAS II  (1) none (2) moderate (3) fairly severe (3) fairly severe (1) none (2) moderate (3) fairly severe 1 vs 2 = yes; 1 vs 3 = yes  1 vs 2 = no; 1 vs 3 = yes  Is the hypothesis significant after adjustments (p<0.05)?  yes; yes  NA; yes  Is the estimate of the effect in the correct direction?  yes; yes  no; yes  Overall, is the hypothesis supported?  d9 - Community, Social & Civic Life & Gender  IPA KAP  Autonomy Outdoors Community, Social & Civic life  male vs female male vs female  no no  NA NA  yes yes  304  For the demographic variable, gender, expect no difference in any of the participation domain scores (applies to all group) and tested Mobility (d4) and Community, social and civic life (d9). Expect a difference in the domains related to Mobility (d4), Self-care (d5), Domestic life (d6) for subjects over the age of 65 years but not in domains related to Interpersonal interactions and relationships (d7). d4 - Mobility & IPA Autonomy Indoors male vs female no NA yes Gender KAP Mobility #1 male vs female no NA yes KAP Mobility #2 male vs female no NA yes PM-PAC Mobility male vs female no NA yes POPS Obj Transportation male vs female yes NA no POPS Subj Transportation male vs female no NA yes WHODAS II Getting Around male vs female no NA yes  Subj Community, Recreational & Civic Life  POPS  d9 - Community, Social & Civic Life & Back Pain cont.  Domain  Instrument  Hypothesis  Study Variable  d4 - Mobility & Age  PM-PAC  d9 - Community, Social & Civic Life & Gender cont.  Autonomy Indoors Mobility #1 Mobility #2 Mobility Obj Transportation Subj Transportation Getting Around  KAP  KAP  PM-PAC  POPS  POPS  WHODAS II  Community, Social & Civic Life Obj Community, Recreational & Civic Life Subj Community, Recreational & Civic Life Participation in Society  Domain  IPA  WHODAS II  POPS  POPS  Instrument  Hypothesis  < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years  male vs female  male vs female  male vs female  male vs female  Study Variable  yes  yes  yes  yes  yes  yes  yes  NA  no yes  NA  yes  NA  NA  NA  NA  Is the estimate of the effect in the correct direction?  no  yes  no  no  no  no  Is the hypothesis significant after adjustments (p<0.05)?  yes  yes  yes  yes  no  no  yes  yes  yes  yes  yes  305  Overall, is the hypothesis supported?  Family Role Domestic Life #4 Domestic Life #5 Domestic Life #6 Domestic Life Obj Domestic Life Subj Domestic Life Life Activities (Non-work) Life Activities (Work)  KAP  KAP  KAP  PM-PAC  POPS  POPS  WHODAS II  WHODAS II  Self-Care  WHODAS II  IPA  Self-Care  KAP  d6 – Domestic Life & Age  Autonomy Indoors  IPA  d5 - Self-Care & Age  Domain  Instrument  Hypothesis  < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs >65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years  > 65 years < 65 years vs > 65 years < 65 years vs > 65 years  < 65 years vs  Study Variable  NA NA  no no  NA  yes  yes no  NA  NA  no no  NA  NA  no  no  yes  NA  no  yes  NA  yes  Is the estimate of the effect in the correct direction?  no  yes  Is the hypothesis significant after adjustments (p<0.05)?  no  no  no  yes  no  no  no  no  yes  no  no  yes  306  Overall, is the hypothesis supported?  WHODAS II  POPS  POPS  Obj Interpersonal Interactions & Relationships Subj Interpersonal Interactions & Relationships Getting Along with People  Interpersonal Interactions & Relationships Interpersonal Relationships  KAP  PM-PAC  Social Life & Relationships  IPA  d7 - Interpersonal Interactions & Relationships & Age  Domain  Instrument  Hypothesis  < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years < 65 years vs > 65 years  Study Variable  no  no  no  NA  NA  NA  NA  NA  no no  NA  Is the estimate of the effect in the correct direction?  yes  Is the hypothesis significant after adjustments (p<0.05)?  yes  yes  yes  yes  yes  no  307  Overall, is the hypothesis supported?  308  Notes: * Depending on the hypothesis tested, adjustments were considered for the following variables: age at follow-up; time to follow-up; gender; education; employment; marital status; living alone/with someone; living in urban or rural setting; receiving compensation; diagnostic study group; comorbidity score.  Abbreviations: IPA, Impact on Participation and Autonomy; KAP, Keele Assessment of Participation; NA, not applicable; PM-PAC, Participation Measure-Post Acute Care; POPS, Participation Objective Participation Subjective; SCI, spinal cord injury; WHODAS II, World Health Organization Disability Assessment Schedule II  Figure D.1a. IPA baseline CFA model for the entire sample Excludes Work and Education domain; (n=545)  309  Abbreviations: AI, Autonomy Indoors factor; AO, Autonomy Outdoors factor; CFA, confirmatory factor analysis; FR, Family Role factor; IPA, Impact on Participation and Autonomy; SR, Social Life and Relationships factor  310  Figure D.1b. IPA adjusted CFA model for the entire sample Excludes Work and Education domain; (n=545)  311  Abbreviations: AI, Autonomy Indoors factor; AO, Autonomy Outdoors factor; CFA, confirmatory factor analysis; FR, Family Role factor; IPA, Impact on Participation and Autonomy; SR, Social Life and Relationships factor  312  Figure D.2a. PM-PAC baseline CFA model for the entire sample Excludes Work & Employment; Education; CSC10e, CSC10g, CSC10h, CSC10i, Com10j, Com10k; (n=510)*  313  Abbreviations: CFA, confirmatory factor analysis; Com, Communication factor; CSC, Community, Social and Civic Life factor; DL, Domestic Life factor; EL, Economic Life factor; IR, Interpersonal Relationships factor; Mob, Mobility factor; PM-PAC, Participation Measure-Post Acute Care; RF, Role Functioning factor Notes: * some subjects were removed for this analysis if they answered ‘don’t do this/not applicable’ for questions in the communication and community, social and civic life domains  314  Figure D.2b. PM-PAC adjusted CFA model for the entire sample Excludes Work & Employment; Education; CSC10e, CSC10g, CSC10h, CSC10i, Com10j, Com10k; (n=510)*  315  Abbreviations: CFA, confirmatory factor analysis; Com, Communication factor; CSC, Community, Social and Civic Life factor; DL, Domestic Life factor; EL, Economic Life factor; IR, Interpersonal Relationships factor; Mob, Mobility factor; PM-PAC, Participation Measure-Post Acute Care; RF, Role Functioning factor Notes: * some subjects were removed for this analysis if they answered ‘don’t do this/not applicable’ for questions in the communication and community, social and civic life domains  316  Figure D.3a. WHODAS II baseline CFA model for the entire sample Second Order Factor Analysis (n=545); excludes the domain Life Activities  317  Abbreviations: CFA, confirmatory factor analysis; Disab, General Disability factor; GA, Getting Around factor; GAP, Getting Along with People factor; PS, Participation in Society factor; SC, SelfCare factor; UC, Understanding and Communicating factor; WHODAS II, World Health Organization Disability Assessment Schedule II  318  Figure D.3b. WHODAS II adjusted CFA model for the entire sample 2nd Order Factor Analysis (n=545); excludes the domain Life Activities  319  Abbreviations: CFA, confirmatory factor analysis; Disab, General Disability factor; GA, Getting Around factor; GAP, Getting Along with People factor; PS, Participation in Society factor; SC, SelfCare factor; UC, Understanding and Communicating factor; WHODAS II, World Health Organization Disability Assessment Schedule II  320  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0067308/manifest

Comment

Related Items