The Co-Worker Acceptance of Disabled Employees (CADE) Scale: A Study to Gather Evidence of Content Validity by Jonathon Steven Breen A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Social Work) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) November, 2018 © Jonathon Steven Breen, 2018 ii Doctoral Committee The following individuals certify that they have read, and recommended to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: The Co-Worker Acceptance of Disabled Employees (CADE) Scale: A Study to Gather Content Validity Submitted by Jonathon Steven Breen in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Social Work Examining Committee: Dr. Timothy Stainton, Social Work__________________________ Supervisor Dr. Anita Hubley, Educational and Counselling Psychology_______ Supervisory Committee Member Dr. Susan Forwell, Occupational Science and Occupational Therapy Supervisory Committee Member Dr. Anne Gaderman, Population and Public Health______________ University Examiner Dr. Rachelle Hole, Social Work, UBC-O_______________________ University Examiner Dr. Susanne Bruyère, Disability Studies, Cornell University________ External Examiner iii Abstract The acceptance of disabled employees and job seekers within the workplace is a key determinant of their employment success. The principal purpose of this dissertation is the development of the Co-Worker Acceptance of Disabled Employees (CADE) Scale. It is intended as a measure of the degree to which co-workers accept these employees within the workplace. The CADE Scale is supported by both theoretical and empirical evidence. The former, theoretical validity evidence, is considered within a newly developed model of disability, the Difference Model. That evidence, which is intended to mitigate a previously undescribed form of error associated with scale development, and which I refer to as ontological error, is applied to the development of the CADE Scale. The latter, empirical validity evidence, has been gathered from data generated through a content validity study. The Standards for Educational and Psychological Testing (2014) indicates that evidence based on scale content is one of the five principal and, typically, one of the first sources of empirical validity evidence to be considered. To meet this requirement, these data were gathered within a judgmental study with input from 21 Subject Matter Experts from Canada, the United States, Great Britain and Australia. This group included individuals with and without disabilities. Each represented one or more of three skill sets – experience delivering disability-related training or workshops; experience as a person with a disability in a corporate iv environment; an understanding of workplace dynamics, either as a practitioner or an academic, as these dynamics relate to employees with disabilities. This dissertation is presented within a framework of emancipatory disability research. As such, it is purposeful; that purpose, as articulated by many within the field of disability, is an attempt to improve life opportunities for individuals living with disabilities. In support of this purpose, I have included several “text box” summaries of my personal, professional, and academic relationships to this research. My intent is to provide an opportunity for ontological critique and, ultimately, to assist in meeting my principal purpose. v Lay Summary In Canada and other developed countries, the unemployment rate for people with disabilities is double that of the general population. The attitudes of others within the workplace significantly affect the employment success of people with disabilities. However, there are few tools available to measure these attitudes. The Co-worker Acceptance of Disabled Employees (CADE) Scale has been developed to fill this gap. This scale is a questionnaire containing 20 statements that illustrate examples of workplace circumstances involving people with disabilities. Each of these is written so that respondents will choose to agree or disagree depending on whether they believe that people with disabilities are different from others in the workplace. These responses are analyzed to determine the degree of this perceived difference. If the CADE Scale is administered both before and after disability-related training, changes to these attitudes can also be measured. This provides an indicator whether the training was effective and, if measured again later, whether these changes to attitude are retained. vi Preface This dissertation is original, independent research by the author, J. Breen. The research reported herein was covered by Ethics Certificate number H15-02011 granted on April 6, 2016 by the Behavioural Research Ethics Board of the University of British Columbia in Vancouver, Canada. A summarized version of Chapter 3 has been accepted by an academic journal with a focus on disability [Breen, J. (in press). Attitudes toward employees with disabilities: A systematic review of self-report measures. Australian Journal of Rehabilitation Counselling.] A self-report scale designed by the author to measure employer attitudes toward people with disabilities in Armenia has been published [Breen, J., Havaei, N., & Pitassi, C. (2018). Employer attitudes toward hiring persons with disabilities in Armenia. Disability and Rehabilitation, https://doi.org/10.1080/09638288.2018.1459882]. I was the lead investigator and was principally responsible for concept formation, data collection and manuscript development. Havaei, N was responsible for the majority of the data analysis and participated in the editing of the manuscript. Pitassi, C participated in the concept formation and in the editing of the manuscript. None of the main findings of the research within this dissertation have been published. vii Table of Contents Abstract .................................................................................................................. iii Lay Summary ........................................................................................................... v Preface .................................................................................................................... vi Table of Contents.................................................................................................... vii List of Tables ............................................................................................................ x List of Figures .......................................................................................................... xi List of Abbreviations ................................................................................................ xii Acknowledgments .................................................................................................. xiii Chapter 1: Introduction ............................................................................................. 1 The Issue .............................................................................................................. 1 Defining Employers and Co-workers ........................................................................ 5 Goal, Purpose and Objectives ................................................................................. 6 Conceptual and Operational Definitions ................................................................. 17 Gap Analysis ........................................................................................................ 20 The Methodological Framework ............................................................................ 22 Overview ............................................................................................................. 23 Chapter 2 ........................................................................................................ 23 Chapter 3 ........................................................................................................ 23 Chapter 4 ........................................................................................................ 24 Chapter 5 ........................................................................................................ 24 Key Points ........................................................................................................... 25 viii Chapter 2: Attitudes, Acceptance and Models of Disability ......................................... 26 Defining disability. ............................................................................................ 26 The relationship between attitude and acceptance. ............................................ 36 Models and Disability ........................................................................................... 40 Models of Disability .............................................................................................. 54 The Medical Model ............................................................................................... 56 The Social Model ................................................................................................. 60 The Embodiment Model ....................................................................................... 67 The Difference Model ........................................................................................... 71 Critiques, Gaps and Othered Groups ..................................................................... 80 Key Points ........................................................................................................... 83 Chapter 3: Systematic Review .................................................................................. 84 Background ......................................................................................................... 84 Introduction ........................................................................................................ 85 Objectives ........................................................................................................... 88 Methods .............................................................................................................. 89 Results ................................................................................................................ 95 Description of measures. .................................................................................. 98 Discussion ......................................................................................................... 118 Strengths and Weaknesses ................................................................................. 126 Key Points ......................................................................................................... 128 Chapter 4: The CADE Scale – Theory, Development, and Analysis ............................ 129 ix A Brief Review of Validity Theory ........................................................................ 129 Ontological Validity ............................................................................................ 133 Content Validity ................................................................................................. 142 Judgmental studies ........................................................................................ 142 The CADE Scale ................................................................................................. 143 A Judgmental Study. ...................................................................................... 143 Methodology. ................................................................................................. 160 Data Analysis ................................................................................................. 166 Key Points ......................................................................................................... 188 Chapter 5: Conclusions, Limitations and Future Research ........................................ 189 References ........................................................................................................... 196 Appendix A: Initial 40 Item Scale ........................................................................... 228 Appendix B: Final 20 Item Scale............................................................................. 230 Appendix C: Databases and search parameters ....................................................... 231 Appendix D: SME Introductory Letter ..................................................................... 234 Appendix E: Informed Consent Form ...................................................................... 237 Appendix F: SME Self-Identification Form ............................................................... 241 Appendix G: SME Response Form .......................................................................... 243 Appendix H: Review of full texts and additional references ...................................... 258 Appendix I: SME narrative responses ..................................................................... 260 x List of Tables Table 1.1: Sources of Validity Evidence ...................................................................... 7 Table 1.2: Models of Disability ................................................................................... 9 Table 3.1: Thirteen Database Searches ................................................................... 93 Table 3.2: Summary of Database Searches ............................................................... 95 Table 3.3: Weight of Evidence Chart ...................................................................... 118 Table 3.4: Scale Weight of Evidence Scores ............................................................ 120 Table 3.5: Scale Items and Models of Disability ....................................................... 121 Table 3.6: Item Valences, Theoretical Assumptions, and Psychometric Data ............. 124 Table 4.1: Subject Matter Expert Demographic Information ..................................... 168 Table 4.2: Co-worker Response Categories ............................................................. 172 Table 4.3: Forty-item Relevance Scores (Hiring & Performance) ............................... 173 Table 4.4: Forty-item Relevance Scores (Accommodations & Supervision/Dismissal) . 174 Table 4.5: Forty-item Clarity Scores (Hiring & Performance) .................................... 175 Table 4.6: Forty-item Clarity Scores (Accommodations & Supervision/Dismissal) ....... 176 Table 4.7: SME Relevance Scores (20 item scale) .................................................... 177 Table 4.8: SME Clarity Scores (20 item scale) ......................................................... 178 Table 4.9: SME Scores and Comments ................................................................... 179 xi List of Figures Figure 1.1: Workplace Acceptance of PWDs .............................................................. 18 Figure 3.1: Records Flow Diagram ........................................................................... 97 Figure 4.1: Ontological Validity Input Matrix ........................................................... 139 xii List of Abbreviations ACP - Average Congruency Percentage APE Model – Associative and Propositional Evaluation Model BCHRT - British Columbia Human Rights Tribunal CAP – Canadian Astronaut Program CHRC - Canadian Human Rights Commission CVI - Content Validity Index EEOC - Equal Employment Opportunity Commission EFA - Exploratory Factor Analysis JAN - Job Accommodation Network Non-PWDs - People without Disabilities PWDs - People with Disabilities SME - Subject Matter Expert UBC – University of British Columbia xiii Acknowledgments I would like to express my appreciation to my supervisor, Dr. Tim Stainton, and to my other committee members, Dr. Anita Hubley and Dr. Susan Forwell. Their collective experience, wisdom, and patience have provided me with the remarkable degree of support and guidance required to complete this epic journey. 1 Chapter 1: Introduction The Issue The World Health Organization (WHO, 2001) reports that over one billion people live with disabilities (PWDs)1. The United Nations Convention on the Rights of Persons with Disabilities states that PWDs are entitled to all of the rights and freedoms afforded all people. This includes the right to employment and all of the benefits associated with this entitlement (United Nations, 2006). Employment is understood to be a fundamental and necessary component of the well-being of all people, including PWDs (Dutta et al., 2008). In addition, policy makers and disability activists consider that employment is central to the social inclusion of PWDs (Shaw & Priestley, 2011). Shaw and Priestley state that “All the other situations from which physically impaired people are excluded are linked, in the final analysis, with the basic exclusion from employment” (citing UPIAS/Disability Alliance, 1976, p. 119). The unemployment rate of PWDs continues to be significantly higher than that of the general population. According to Burke et al. (2013), these rates are continuing to increase. In Canada, overall unemployment on the part of PWDs is approximately double that of the general population. Additionally, within the broad group of PWDs, 1 The terms used to designate this group vary according to region and political perspective. In the United Kingdom and the European Union, where the strong social model of disability is predominant, the common phrase is “disabled people”. This phrase is intended to bring focus to the social barriers imposed on those living with disabilities (Oliver, 1990). However, in Canada and the United States, and elsewhere where the minority group model of disability is more prevalent, the phrase “people with disabilities” is more typical. This phrase is described as people first language; it is intended to indicate that the person should be the primary focus, with the disability understood as an attribute of that individual (Titchkosky, 2001). I have selected the semiotic of PWD to describe this group within this dissertation. As described below in more detail (see Chapter 2), this symbol can be understood as people with differences as well as people with disabilities. 2 people who identify as living with moderate and extreme disabilities2 demonstrate the highest rates of unemployment; these groups are reported to have rates of unemployment that are two to three times higher than those with mild disabilities (Turcotte, 2014). These high unemployment rates within the disability community have been most commonly linked to three social and psycho-emotional factors. These include the legislative environment, the norms and values within different organizations, and the personal characteristics of PWDs and co-workers (Stone & Colella, 1996). This latter factor includes the type of impairment, attributes of supervisors and co-workers, and previous experience with PWDs (Louvet, 2007). However, efforts to address each of these three factors, as delineated by Stone and Colella, have not led to improved rates of employment. Legislative supports have been found to be challenging to implement and, in some cases, have made matters more difficult by blunting the political demands of PWDs (Campbell, 2005). Although many countries, including Canada, the United States, and the United Kingdom, have made significant political and legislative efforts to outlaw discrimination in employment, these high rates of unemployment have continued to be the norm among PWDs (Barnes & Mercer, 2003; Clayton et al., 2011; Kidd, Sloane, & Ferko, 2000). Employers also continue to demonstrate organizational norms and values that suggest widespread concerns regarding the hiring of PWDs. In an American survey of 3,797 employers, the United States Department of Labor found that a high percentage 2 Statistics Canada reports that scores for calculating the severity of disability are based on a formula that includes the number of disability categories identified by an individual as well as the frequency and degree of difficulty associated with the limitation of activities (Statistics Canada, 2013). 3 of respondents demonstrated practices and beliefs that would, to a significant degree, preclude PWDs. For example, 72.6 percent of these respondents indicated that the nature of the work required in their organizations could not be effectively performed by PWDs (Domzal, Houtenville, & Sharma, 2008). Employers also consider PWDs to be a poor fit in jobs that require a significant degree of interpersonal contact. In two French studies, 284 management undergraduates rated PWDs as significantly less suitable for positions that required a high level of public interaction, and similarly unsuitable for positions that were deemed to be “male jobs” (Louvet, 2007, p. 300). A recent study of employer attitudes in Armenia found that employers were concerned over the possible negative perceptions of employees and customers toward PWDs in the workforce (Breen, Havaei, & Pitassi, 2018). An Australian study of disability-related human rights complaints found that employer policies and practices were significant barriers to employment for PWDs (Darcy, Taylor, & Green, 2016). In a review of almost 30,000 employee surveys from 14 American companies, Schur, Kruse, Blasi, and Blank (2008) found that these corporate cultures are frequently “based on the assumption that employees are able-bodied” (p. 384) and that the perceived fit, or lack thereof, between employees and these cultural norms are strong indicators of employee success. Neither changes to these legislative climates nor various efforts to reform organizational values has been sufficient to result in increased employment rates for PWDs (Barnes & Mercer, 2003; Campbell, 2005). Corker (1999a) argues that these attitudinal barriers will not necessarily be resolved even following the removal of structural barriers. 4 Within the personal characteristics component of the triad described by Stone & Colella (1996), current research indicates that the negative attitudes of others within the workplace are considered to be the most significant determinant of employment success of PWDs (Antonak, 1982; Antonak & Livneh, 2000; Burke et al., 2013; Copeland, Chan, Bezyak, & Fraser, 2010; Vornholt, Uitdewilligen, & Nijhuis, 2013). These negative attitudes toward PWDs are further parsed, on the one hand, as being driven principally by initial, affect-based responses (Vilchinksy & Findler, 2010) which are, in turn, thought to be pervasive and largely unconscious (Antonak, 1988; Pruett & Chan, 2006). Others, however, consider these attitudes to be cognitively-based and purposeful. In a review of 37 studies of employer attitudes toward employees with disabilities, the negative attitudes of others within the workplace were demonstrated to be the main barrier to employment opportunities for people with disabilities (Hernandez, Keys, & Balcazar, 2000). That same review also indicated that employers typically offer relatively positive responses when queried about their more global attitudes toward PWDs but more negative attitudes when asked about hiring intentions and support for accommodations. Similar discrepancies in employer attitudes are reported by Copeland et al. (2010). These findings are further supported by Hernandez et al. (2000) who state that employers recognize that it is politically correct to espouse generally positive attitudes toward PWDs but that these do not carry over into practical employment situations. Although a broad spectrum of workplace training programs have been developed and implemented to address these challenges of attitude and awareness within various 5 organizations, insufficient research has been conducted which measures the effects of these interventions (Brostrand, 2006). However, employment rates among PWDs continue to remain at less than half those of the general population (M. Russell, 2002; Turcotte, 2014). Defining Employers and Co-workers The term employer has been variously defined in workplace attitude studies related to PWDs. In a review of 37 studies of employer attitudes toward PWDs, Hernandez et al. (2000) found that, while some studies indicate that employers mean either business owners or senior management, others also include human resources personnel and other employees with the authority to hire and fire. Domzal et al. (2008) report that, in a study of employer attitudes toward PWDs involving 3,797 businesses, all interviews were conducted with senior executives with the authority to hire across all organizational strata. Copeland et al. (2003) acknowledge that this term is not well defined in this context. Many researchers refer to the attitudes of others within the workplace as including “employers and co-workers” when considering their effect on PWDs (Antonak & Livneh, 2000, p. 211; McCarthy, 1988, p. 250). However, in their influential work describing the factors which affect PWDs within the workplace, Stone and Colella (1996) state that the attitudes of all others who come in contact with PWDs are of significance. These include all those “who might interact with or make judgments about disabled individuals (p. 356). Following Stone and Colella, I have adopted the term co-worker and applied it very broadly within this dissertation to mean senior 6 management, supervisors, human resource personnel, and all other employees within the workplace who may have contact with or influence over PWDs. Goal, Purpose and Objectives The principal goal of this dissertation is to add to the knowledge available to support improved employment opportunities for PWDs, specifically as these may be improved through an enhanced understanding of workplace attitudes toward PWDs. As such, this dissertation is presented under a rubric of emancipatory disability research. Emancipatory disability research, as articulated within the field of disability, is purposeful - an attempt to improve life opportunities for individuals living with disabilities (Mercer, 2002; Oliver, 1992a). To meet this goal, my purpose is to develop a new resource, the Co-Worker Acceptance of Disabled Employees (CADE) Scale, which will measure workplace attitudes toward PWDs. For such a scale to be of value to research, the validity of the inferences generated must be supported. Validity is defined as “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. (U.S.). (2014). [AERA et al.], 2014, p. 11). Messick (1990) states that “test validation is empirical evaluation of the meaning and consequences of measurement” (p. 2) and that scores from tests are to be considered in a broad sense, including both “quantitative and qualitative summaries” (p. 1). Validity is considered to be the most fundamental consideration in scale development; “without 7 validity, a test, measure, or observations and any inferences made from it are meaningless” (Hubley & Zumbo, 1996, p. 207). Within this context, and as shown below in Table 1.1, five sources of validity evidence may be considered in support of these interpretations of test scores. These include evidence based on test content, response processes, internal structures, relations to other variables, and the consequences of testing (AERA et al., 2014). Table 1.1: Sources of Validity Evidence SOURCES OF VALIDITY EVIDENCE SOURCE DETAIL Test content Evidence based on test content includes an examination of the relationship between the content of a test and the construct that it is designed to measure. Test content may refer to the items or questions within a test, the scoring method, and the test administration process. Response processes Evidence based on response processes is an examination of the cognitive processes that are actually engaged in by test respondents. This evidence may be generated by analyses that include questioning individual test takers regarding response strategies, monitoring physiological responses or comparing response times. Internal structures Evidence based on the analysis of the internal structure of a test allows for a comparison between the relationships among test items and the construct under consideration. Relations to other variables Evidence based on the relationship of test scores to other variables includes comparisons with predicted external performance criteria, other tests measuring the same, similar or different constructs. Consequences of testing Evidence based on the consequences of testing includes a consideration of these consequences that are beyond those intended by the developer of the test; this evidence is particularly salient when these consequences have negative implications for a subset of test takers. However, the CADE Scale, at this formative stage, requires the generation of sufficient evidence of content validity to warrant proceeding with its development. Evidence of content validity, in essence, provides the means by which the structure of a scale may be evaluated to determine its “quality and appropriateness” in supporting score-based inferences (Sireci, 1998a, p. 103). 8 Within this dissertation, I provide both theoretical and empirical evidence of content validity in support of the CADE Scale. The generation of theoretical validity evidence in support of the CADE Scale includes two distinct but related efforts. The first is the development of a new model of disability, referred to as the Difference Model. This development followed my review of current models of disability; models which are intended to “direct our attention to particular elements of disability” and which are the basis for driving particular intervention strategies (Dirth & Branscombe, 2017, p. 413; also see Hahn, 1993; Smart, 2009; Zola, 1993). The Difference Model is an ontological alternative to current models of disability. Each of these latter models has been positioned at different locations along what I have identified as a dimension of responsibility. I present the Difference Model, as shown below in Table 1.1, as a new articulation of voice (Young, 1990) that is based upon a different dimension of disability, that of perception of difference. The Difference Model, with its understanding of disability as perceived difference, offers a possibility of contributing to the resolution of the employment challenges of PWDs by circumventing the ongoing, and increasingly accepted as irresolvable, debate over where disability should be positioned along the dimension of responsibility. This debate is illustrated through an examination of the debate over how to interpret the meaning of the term disability itself (Beaudry, 2016). Disability is described by Shildrick (2012) as “a shifting nexus of both physical and mental states that resists full and final definition” (p. 3). The Difference Model, as the name implies, understands PWDs as 9 being framed within a perception of difference that is sufficiently significant to be of social notice. Table 1.2: Models of Disability MODELS OF DISABILITY MODEL RESPONSIBILITY LOCATION Medical Individual Personal Social Society Political Embodiment Individual & Society Social Difference None Social Notice The Difference Model will add to the validity evidence associated with the CADE Scale by providing a specific theoretical framework against which each of the items included in this scale may be screened. As noted by Barnes (2014), “researchers adopting an emancipatory framework must make their standpoint clear at the onset, and ensure that choices of research methodology and data collection strategies are logical, rigorous and open to scrutiny” (p. 42). This description of the Difference Model is intended to provide a clearly delineated ontology against which items to be considered for inclusion within the CADE Scale may be screened, in an effort to reduce “construct underrepresentation or construct-irrelevant components” (AERA et al., 2014, p. 21), which are understood to be the principal challenges to validity within scale development (Messick, 1990). The second of these objectives which address theoretical validity evidence is the development of a conduit through which the Difference Model may be explicitly applied to the development of the CADE Scale. I have identified a previously undescribed form of ontological evidence through which the Difference Model may be explicitly applied to 10 the development of the CADE Scale. This process serves as a means to bring required theoretical evidence to bear on the process of scale development (AERA et al., 2014; Messick, 1990). In essence, this process provides a lens through which items considered for inclusion in the CADE Scale can be screened to ensure a clearly delineated relationship to a specific model of disability; in this instance - the Difference Model. Ontological evidence addresses what Young (1990), in a consideration of the politics of difference, offers as a critique of those engaged in theoretical considerations of social structures; that critique is stated as the failure to “acknowledge the particularity of the voice of their writing” (p. 14). Young also suggests that, without a consideration and clear articulation of the worldview which informs such theoretical constructions, there is no basis against which they may be compared and evaluated. Hacking (1999), in a consideration of various understandings of disability, argues that care must be taken to articulate the basis of these understandings in order to support the intended interpretations. Ontological evidence, similar to consequential validity, considers the implications of evidence that extend “beyond the interpretation or use of scores intended by the test developer” (AERA et al., 2014, p. 19). However, in the former instance, these implications support the development of the scale itself; in the latter instance, this evidence supports “proposed interpretations for their intended uses” (p. 19). In addition, ontological evidence is gathered prior to administration, while evidence based on consequences of tests is gathered following a specific test application (Hubley & Zumbo, 1996; Messick, 1990, 1998). 11 A consideration of ontological evidence adds to the validity evidence supporting the intended uses of CADE Scale test scores. Ontological evidence rests on an exploration and articulation of the antecedent perspective upon which the test is constructed. By delineating this perspective, the relevance of items within the scale itself may be more effectively evaluated as pertaining to and broadly representing the construct under consideration. A consideration of ontological evidence also supports theoretical transparency, allowing theoretical discourse to benefit from being more clearly “situated in concrete social and political practices” (Young, 1990, p. 5). Ontological evidence also adds to the tools available to define the construct under consideration. As noted by Kane (2001), “the variable of interest is not out there to be estimated; the variable of interest has to be defined or explicated” (p. 324). Ontological evidence is investigated through a consideration of information that exists antecedent to scale development, thereby providing a chronological dimension to the nomological network within which the scale exists. This chronological analysis supports the requirement, particularly in areas of the social sciences with little supporting theory, that interpretations of complex constructs require detailed, clear and explicit evidence (Kane, 2001, 2016). The recognition of disability as a complex and undertheorized construct is acknowledged by many authors (see, for example, Beaudry, 2016; Oliver, 1990; Silvers, 2003). Empirical evidence in support of the CADE Scale is presented through a review and analysis of the data generated through a content validity study of the CADE Scale. As noted above, content validity evidence is recognized as one of five principal sources 12 of validity evidence and is considered to be contained within the scale itself (AERA et al., 2014). In this study, empirical evidence in support of the CADE Scale has been generated through an analysis of input from 21 Subject Matter Experts (SMEs). Each of the SMEs provided both quantitative and narrative responses to the perceived relevance and clarity of an initial set of 40 items, (see Appendix A), developed for consideration, review, and possible inclusion in the scale. These 40 items were initially developed following my review and synthesis of four lines of research. These included a systematic review of current scales, an analysis of the several categories of workplace-related complaints filed by PWDs with Canadian and American organizations charged with oversight and resolution of such complaints3, personal discussions with non-government organizations (NGOs) involved in providing employment supports and advice to PWDs, and in prior personal discussions with three of the SMEs who subsequently volunteered to participate in this study. My analyses of the quantitative and narrative SME responses informed my selection of the 20 items included in the CADE Scale (see Appendix B). The SMEs, as a group, were treated as “an advisory board rather than a representative of a larger population” (L. Russell, 2013, p. iii). Although the SMEs were not provided with a detailed summary of either the Difference Model of disability or the concept of ontological evidence, I included information in the instruction materials indicating that the original 40 items were to be evaluated for relevance and clarity in terms of the ability of each item to raise the 3 These included the Canadian Human Rights Commission, the British Columbia Human Rights Tribunal, and the American Equal Employment Opportunity Commission. 13 possibility of a perceived difference between disabled employees and others. In addition, I communicated to each of the SMEs that this perception by co-workers of workplace-related differences between PWDs and others is hypothesized as the key determinant to acceptance. I also advised that this workplace acceptance is, in turn, perceived to be a determinant of employment success for PWDs. In my introductory letter to SMEs, I presented this relationship between acceptance and perceived difference as follows: I have located this measure of acceptance along a continuum of perceived difference. Therefore, for purposes of this study, I have described acceptance as “the degree of perceived difference attributed to disabled employees within a workplace environment”. In other words, I have predicated an inverse relationship between acceptance and perceived difference; the more perceived difference, the less acceptance. (see Appendix C for introductory letter). Response data received from the SMEs were analyzed through two different methods. First, all ratings of the original 40 items for relevance and clarity were analyzed through a Content Validity Index (Lynn, 1986), which provides recommendations for including or excluding each item based on specific levels of item-level and scale-level agreement amongst SMEs. Second, narrative comments regarding specific items, item categories, and overall perceptions of the scale effectiveness were reviewed to determine whether SMEs had additional recommendations pertaining to 14 specific items or whether the scale appeared to effectively capture the construct under consideration. Each of these objectives outlined above is delineated and supported within the text of this dissertation. Based on this process, and the information provided, I believe that I have been able to meet the goal which has motivated my research program and the development of the CADE Scale. To expand briefly upon the initial statement of my goal, I am attempting to influence the future employment success of PWDs. My current research and previous work have led me to hypothesize that the employment success of PWDs is significantly hampered by the negative attitudes of others within the workplace (Antonak, 1988; Antonak & Livneh, 2000; Domzal et al., 2008). In the future, by being able to accurately measure and compare the various and varying levels of workplace acceptance of PWDs, attitudinal changes may be tracked over both the short and long terms. In the short term, pre- and post- measurements connected to various forms of disability-related training could allow for needed evaluations of the effectiveness of these interventions (Brostrand, 2006). These measurements could reveal quantified changes in attitudes toward PWDs that would inform the structure and focus of future workplace interventions, including disability awareness training. In the longer term, monitoring of those attitudes, and any subsequent changes, could influence strategic planning within individual organizations, between different organizations, and between industry categories, thereby supporting the employment success of PWDs. 15 As a final point within this introductory section, it should be noted that I have included five italicized ‘text box commentaries’ on my personal, professional, and academic relationships to and motivations for this research. I have included these commentaries to provide additional opportunities for ontological critique and considerations of researcher bias (Creswell, 2013) as well as a broadly based triangulated description of my position as a researcher. As described by Maxwell (2012), triangulation is a method of making use of several methods and sources to gather and present information, thereby “reduc(ing) the risk that your conclusions will reflect only the systematic biases or limitations of a specific source or method” (p. 106). Within these five commentaries, I have employed the qualitative technique of narrative inquiry as a means to present specific examples as applied coherence of the more general and theoretical (Breen, 2017). Narrative inquiry is understood as a method intended to generate coherent and meaningful representations of experience (Reisman, 2008). Within this dissertation, I have included these commentaries to illustrate my relationship to disability and how that relationship has influenced my understanding of its construction and application. I have offered this description at the beginning of this dissertation for two reasons. First, it pertains to the larger goal which has informed the development of the CADE Scale and its supporting evidence, including the newly conceptualized category of ontological evidence and the Difference Model of disability. Second, it serves as an indicator of the proposed uses to which the CADE Scale may be put (AERA et al., 2014). Text Box Commentary #1 16 In 2004, I was hired by the Yukon Government to create and administer the new Workplace Diversity Employment Office, with a mandate to increase its low rate of employment of PWDs. At the time, that rate was approximately 4.4 percent of the Yukon Government (YG) workforce, which totalled over 4,500 individuals. Although this rate was not dissimilar from most other Canadian jurisdictions, including the federal government, the government had been coming under pressure from the community to address this issue. The size of the Yukon, approximately 35,000 people, made it susceptible to relatively small lobbying efforts on the part of its constituents. During the first year of my mandate, my focus was on three areas. The first was to convince the Yukon disability community that the government was interested in hiring PWDs. The second was to convince supervisors and managers within YG that this was possible, cost-effective, and beneficial. Each of these efforts was based on what I now refer to as the Difference Model of disability. I demonstrated to each of these groups that disability was just a part of a new, expanded ‘normal’. For example, all YG job advertisements were screened to remove any unintentional barriers to employment for PWDs. These included removing an anachronistic requirement for a driver’s licence for many positions that did not require driving. Interestingly, no one had considered that these could be seen as a barrier by some PWDs. My next effort followed my review of an internal equity-based census which revealed that, similar to many jurisdictions, individuals who were hard of hearing made up the largest, single category of PWDs currently employed by YG. My research indicated that many of these people had difficulty with their telephones, which, although they had volume controls, did not work well for those wearing hearing aids. I sourced a simple electronic device that could be attached to a telephone and which solved this problem. My office purchased 100 of these devices and a global email was sent to all YG employees, advising them that these were available at no charge for anyone who could benefit. Three things happened. First, we distributed all of these devices within two weeks. Second, we began to receive communications from other employees with disabilities and managers who had not until then believed that the government was willing to provide funding for accommodations. Third, every employee within YG now knew that government was taking this matter seriously. By 2007, an internal census revealed that 10.5 percent of YG employees were PWDs. By 2013, this had increased to 12 percent. (All figures confirmed through personal communications with YG official in November, 2017.) 17 Conceptual and Operational Definitions Conceptual definitions are intended to define the universe or domain of content to be measured in a manner that provides a detailed description of the target construct (Beck & Gable, 2001; Clark & Watson, 1995). These constructs have been defined as “theoretical concepts of varying degrees of abstraction and generalizability which facilitate the understanding of empirical data” (Anastasi, 1986, p. 5). These broad, theoretical definitions also require additional explanations of their various components in order to “bridge the gap between language and experience” (Waltz, Strickland, & Lenz, 2005, p. 35). The conceptual definition of the workplace relationship to disability that is the subject of this dissertation is as follows: The acceptance of PWDs is a measure of the workplace-related attitudes exhibited by co-workers toward this group. Acceptance is determined by the degree to which PWDs are considered by co-workers to be different from others within the workplace. This determination of difference is defined as a continuum ranging from a neutral position to one in which it is perceived as sufficiently different to be detrimental to either the organization or the individual. This separation between organizational and personal success is analogous to what Waltz and Bausell (1981) refer to as the separation between values and interests within affective measures. Within the context of the perception of PWDs by co-workers, organizational considerations include perceptions of the potentially negative effect of differences on corporate strategies, policies or expenditures. Personal considerations 18 include these perceptions of PWDs as they may affect individual interests such as financial reward and workload. These perceptions, as illustrated in Figure 1.2, are indicated by cognitive, affective and behavioural responses to PWDs by co-workers in each of the four principal aspects of employment. These include hiring, performance, accommodation, and supervision/dismissal. Figure 1.1: Workplace Acceptance of PWDs Operational definitions are defined as the domain or construct under consideration in terms of the observations that measure it (Beck & Gable, 2001). Operationalization of a concept involves moving from an abstract idea to a concrete delineation of how it can be measured. “It involves making a concept explicit in terms of the observable indicators associated with it and/or the operations that must be carried out in order to measure it” (Waltz et al., 2005, p. 23). This requires the identification of the various dimensions of its meaning, the observables associated with those dimensions or attributes, and the manner in which these observables will be measured. Responses Influenced by Perception Impact on Organizational & Personal Success Difference Cognitive Affective Behavioural Acceptance 19 I have operationalized the construct of the acceptance of PWDs in the workplace as a measure of the degree to which co-workers perceive PWDs as different from others within four specific aspects of the employment continuum. As stated by Waltz et al. (2005), many constructs must be inferred because they cannot be observed directly. These “require the use of special devices or instruments” (p. 26). The CADE Scale will serve as that means of operationalization and will “provide meaning by defining a concept in terms of the observations and/or activities that measure it” (p. 26). In a reference specific to disability, Altman (2001) acknowledges that, due to the complexity of disability as a social phenomenon “one needs to convert the concept through a procedure, either observational or through inquiry, which establishes the empirical existence of the concept in question” (p. 101). This concept can be best defined by reducing the focus of that definition to one that clearly and economically represents a specific aspect of this complex experience (Altman, 2001). According to Sireci (1998a), operational definitions of constructs are most often written to reflect test specifications. This understanding is supported by Antonak (1988) within the context of disability. According to that author, an operational definition is appropriately demonstrated by a given technique that accounts for theoretical assumptions. However, the term acceptance , as it applies to PWDs, has been defined in a variety of ways; I have chosen to operationalize it in a manner that will be of practical value (Link & Phelan, 2001) by providing a means to quantify co-worker attitudes toward PWDs. Therefore, the acceptance of PWDs by co-workers is operationalized as: 20 The degree of perceived difference attributed by co-workers to disabled employees within a workplace environment as measured by the CADE Scale. The categories within these perceived differences, as represented within the CADE Scale, are based on the most common workplace-related complaints filed by disabled employees. Although the categorizations of these complaints vary somewhat between jurisdictions, both between Canada and the United States, and between provincial and federal jurisdictions within Canada, these complaints are most frequently associated with dismissal, followed in order by lack of accommodations, performance/adverse treatment, and hiring procedures. As considered in more detail in Chapter 4, these four categories are supported through my analysis of the most common workplace areas of complaint filed by disabled employees through the Canadian Human Rights Commission (CHRC), the British Columbia Human Rights Tribunal (BCHRT), and the American Equal Employment Opportunity Commission (EEOC). Gap Analysis There are a number of instruments available to measure attitudes toward PWDs (see, for example, Antonak, 1982; Findler, Vichinsky, & Werner, 2007; Yuker, Block, & Younng, 1970). However, very few of these measures address attitudes specifically within the workplace (McLaughlin, Bell, & Stringer, 2004; Popovich, Scherbaum, Scherbaum, & Polinko, 2003). In addition, these few have been demonstrated to face a range of challenges; they have been most frequently developed to measure attitudes toward specific disabilities resulting in poor generalizability, they have not typically 21 engaged employees or co-workers as respondents (see, for example, Gething & Wheeler, 1992; Loo, 2004), and they have not considered the recognized differences between overall attitudes toward PWDs and those specific to employment environments (Araten-Bergman, 2016; Burke et al., 2013). Overall, there is a paucity of research directed specifically toward the analysis of workplace attitudes toward PWDs and their subsequent treatment within organizations (Popovich et al., 2003). The psychometric soundness of these few instruments has also been questioned (Antonak, 1988; Findler et al., 2007). This limited empirical evidence, specifically as it pertains to research and program development regarding the employment of PWDs, is broadly acknowledged within disability studies research (Goodley, 2011; Shakespeare, 2014). The systematic review of scales used to measure workplace attitudes toward PWDs, which I conducted in 2016, (see Chapter 3) supports these findings. As described below, that review demonstrates that current measures have typically presented limited psychometric soundness and validity evidence in support of their conclusions. In addition, none of these measures provides any direct reference to a specific framework or model of disability within which they were created and against which scale items could be evaluated. Similar to the other models of disability discussed above, the Difference Model serves to describe the relationship between PWDs and the larger community. And, also similar to the other models of disability, that description is based on premises that reflect its specific ontological underpinnings. These premises, and the resulting descriptions, frame proposed strategies to address the many agreed upon social 22 challenges, including employment, that face PWDs. However, as I present below in Chapter 3, my systematic review of current scales intended to measure attitudes toward PWDs indicates that none of these explicate a coherent theoretical relationship between PWDs and the larger community; none provide data that would demonstrate a coherent ontological position upon which they were formulated. As a final consideration within this gap analysis, I conducted a literature review of a wide array of scales designed to measure attitudes toward a range of othered groups. My intent was to investigate whether a scale similar to what I was developing was already available within a different context. I reviewed a total of 51 scales, which included summaries of 43 scales (Gamst, Liang & Der-Karabetian, 2011) as well as a review of the complete sets of items in a selection of eight scales designed to measure generic racial attitudes and prejudices (Davis & Engel, 2011). These included at least one from each of the following categories - race/ethnicity/diversity (41), women/gender (6), general prejudice (2), prostitution (1), and poverty (1). These scales demonstrated consistent differences from the CADE Scale, principally in terms of their levels of specificity and a lack of intersectionality with disability, particularly with regard to workplace accommodations. The Methodological Framework This dissertation is presented in three parts. First, is an examination of the Difference Model of disability in order to understand the framework (Damasio, 2010) within which the CADE Scale has been crafted. Next, a description of ontological evidence is offered, as the conduit through which the Difference Model may be applied 23 to the development of the CADE Scale (see Figure 1.2). Finally, I will examine the empirical process utilized to generate the CADE Scale, including the recruitment of SMEs, the procedures implemented to gather and analyze the data provided by those SMEs, and the findings generated from those analyses. Figure 1.2: Relationships between Theory and Empirical Evidence Overview Chapter 2 provides an historical and political summary of disability and reviews current models of disability, including the newly introduced Difference Model. Each of these models is considered as they may apply to and inform the CADE Scale. A preliminary review indicates that current measures of co-worker attitudes toward PWDs demonstrate limited psychometric analyses for validity and reliability. Chapter 3 is a systematic review of disability-related scales that have been CADE Scale Ontological Evidence Difference Model of Disability 24 developed to measure attitudes toward PWDs within a workplace environment or which have been subsequently applied for that purpose. These include scales that were developed between 2005 and 2015 as well as scales that had been developed earlier but which are still in active use. Each of these measures is summarized and reviewed for empirical and theoretical validity evidence. These measures demonstrate a range of empirical validity evidence; however, none provide evidence of theoretical validity. Chapter 4 begins with an overview of the ongoing development of validity theory. Second, it considers ontological evidence as a newly described form of evidence which supports the content validation process as this has been applied to the development of the CADE Scale. Within this context, ontological evidence is representative of the means and degree to which a theoretical framework has been applied to test development. Finally, this chapter describes the methodology and the resultant findings from this content validity study. Chapter 5 is a summary of my research findings, a consideration of their practical and theoretical contributions, a number of limitations, and directions for future research. The most significant of these latter directions is a continuation of accumulating validity evidence for the CADE Scale through the implementation of initial validation studies. These will allow for further statistical analyses of items, further consideration of the dimensionality of the scale, and its potential for generalizing across various populations. 25 Key Points PWDs experience very high rates of unemployment, a significant barrier to social inclusion. The negative attitudes of others within the workplace are understood as the most significant influence on the employment success of PWDs. The CADE Scale has been developed to measure co-worker attitudes, and any changes in these attitudes subsequent to various workplace interventions. The CADE Scale is supported by demonstrations of both theoretical and empirical validity evidence. 26 Chapter 2: Attitudes, Acceptance and Models of Disability Within this chapter, I consider the construct of disability and its historical and political relationships with the community within which it exists. I also describe three current models of disability, as well as the newly developed Difference Model. Defining disability. Defining disability poses a number of challenges. First, it is not necessarily clear who counts as disabled or not. Due to the inherent fluidity of the construct (Shildrick, 2012), individuals may be within the group or not, at various times and in various places (Shakespeare & Watson, 2001). Varying cultural norms and a broad range of available medical technologies are of significant influence in determining who is, and who remains, a member of this group (Ginsberg & Rapp, 2013). There are significant differences in language use both between and within the several models of disability (Beaudry, 2016; Thomas, 1999b). These various interpretations are often applied by different groups for different purposes (Altman, 2001). In addition, the premises used to support various models of disability may be drawn from a broad spectrum of either cognitive or affective mental states (Goodey, 2016; Goodley, 2001; Priestley, 1998), resulting in a complex and “essentially contested concept” that is often interpreted according to previously held political values (Silvers, 2003, p. 473). Finally, disability may be purposefully defined for either political or funding purposes to include only those with specific categories of impairments (Americans With Disabilities Act, 1990; Statistics Canada, 2006) or those who exceed a politically pre-determined threshold of impairment (Osipov, 2013). In addition, the understandings of and relationships to disability by the larger community are many and varied (Thomas, 27 2007). Although neither disability nor normality is now considered “an immutable, permanent fact of the matter” (Breen, 2015; Waldschmidt, 2005, p. 191), society has always sought some means to categorize its membership. As Davis (2000) describes in an analysis of the divide between the able-bodied and those with disabilities, “Cultures perform an act of splitting. … These violent cleavages of consciousness are as primitive as our thought processes can be” (p. 53). Shakespeare (1994) states that identity itself “is being strengthened by the isolation and rejection of anomaly” (p. 294). Stainton & Swift (1996) argue that the role of difference is essential “in the formation of both collective and individual identity” (p. 78). Similarly, Connolly (2002) states that identity itself is established relative to socially recognized differences, which are converted “into otherness, into evil, or one of its numerous surrogates” (p. 64). Campbell (2005) argues that disability is beyond “a mere fear of the unknown” (p. 109) and is viewed as an “ontological terror” (p. 110). Fujiura and Rutkowski-Kmitta (2001) state that “We are driven to classify and count the human condition…Such has been the case with disablement” (p. 69). Paul Hunt, recognized as an influential theorist who contributed significantly to the development of the social model of disability (Thomas, 1999b), provides an early reference to the perception of others as they encounter a PWD. Hunt (1996) states that, “An impaired and deformed body is a ‘difference’ that hits everyone hard at first. Inevitably it produces an instinctive revulsion, has a disturbing effect!” (p. 151). Hughes (2005) states that considerations of the term disability occur at the level of the cognitive, as illustrated by the use of terms such as preferable and evaluation, both of which fall into the cognitive, rather than the affective, realm. In addition, that 28 author notes that “disability becomes discursively constituted as a physical or mental deficit” (pp. 83-84). However, other researchers suggest that this cognitive analysis is actually a secondary step, which follows behind prior emotional responses and serves to create and maintain a coherent worldview (Damasio, 2010; Guidano, 1987, 1995). Mik-Meyer (2016), in a study of PWDs in employment situations, finds that co-workers construe these individuals as different and “spontaneously engage in the ‘othering’ of employees with impairments” (p. 1342). Bhaskar and Danermark (2006) argue that these cognitive mechanisms are subsequent to “their extra-cognitive causes and their physical grounds, and must be viewed as both transient and corrigible” (p. 296). This chronological ordering of affect and cognition has been further analyzed by Vilchinsky and Findler (2010). These authors state that, in a study of 404 students, encounters with subjects in wheelchairs revealed that affective responses preceded cognitive responses and that these “initial spontaneous negative emotional response[s]” (p. 298) were followed by “compensatory positive cognitions and behavioral [sic] tendencies” (p. 298). Wolfensberger (1972), in a consideration of how society has typically accorded disability a relationship to difference, which he describes as deviance, developed a typography that offered eight categories of that deviance within which PWDs have, at various times, been perceived. These were all variations of what was more generically described as the ‘other’ and included PWDs as Subhuman Organisms, Menaces, Unspeakable Objects of Dread, Objects of Pity, Holy Innocents, Diseased Organisms, Objects of Ridicule, and Eternal Children. This relationship between disability and difference, either compared with or used interchangeably with otherness, is anything 29 but a recent phenomenon. Although it is clear that specific categories and characterizations of disability have been fluid over time within various cultures, each has created a dichotomy between the normals (Goffman, 1963) and those with disabilities which were sufficiently different to cause social rejection. Reaching back to the classical Greek era, it is clear that societies have, at various times, established various differencing criteria. Edwards (1997) points out that, in that culture, there was no abstract word to describe the category of physical disability as a predetermined set of limitations. Instead, the concept of disability was based on the ability to engage in appropriate participation in the community. This community model of disability determined the status of individuals with physical disabilities. An acceptable level of participation led to social inclusion; an insufficient level led to rejection. Following the classical period in the Western world, according to Gregory (2007), were “centuries of indifference and hostility toward the psychiatrically and mentally impaired…. (T)hey were alternately ignored, persecuted, or tortured” (p. 53). Some researchers have argued that the rise of capitalism and modernism have not only created the generic classification of disability itself but also exacerbated the difficulties faced by PWDs (Oliver, 1990). However, the generalization of this argument is challenged by Thomas (1999b), who maintains that although capitalism has added to the challenges facing PWDs, caution should be exercised regarding the idealization of the existence of PWDs in pre-industrial society. This position is also taken by Barnes and Mercer (2003), who note that there is little evidence available to support claims that PWDs were accepted into feudal societies. In addition, these authors acknowledge that a high mortality rate 30 among PWDs was commonplace in pre-industrial societies. Charlton (2006) notes that “few people with physical disabilities survived for very long in pre-capitalist economies” (p. 219). These determinations and the resulting social implications of difference have not been limited to Western geography and thought. As examples, a tribe in New Guinea, called the Wogeo, has buried alive infants with obvious deformities but accepts as tribal members those who acquire disabilities later in life. The Punan people of Borneo consider those with severe disabilities to be non-human and do not give them names. Like the ancient Greeks, many African languages have no word for disability and only name those specific limitations that impact their lives (Ginsberg & Rapp, 2013); other categories of disability appear to be essentially ignored (Darling, 2013). Returning to European cultures, the re-emergence of empiricism during the Renaissance period of the late 15th and early 16th centuries led to a new view of society based on this scientific revolution, which quickly spread from the natural sciences to human affairs (Rogers, 1995). This newly developed application of measures intended to quantify social relations required a means to delineate and measure social boundaries. The solution was the creation of the other, a construct based on unreason and depravity, and which included those with disabilities (Stainton, 2004). Other writers have gone further and described disability as being beyond inclusion in a generic category of otherness. Siebers (2008) states that “disability is the other other that helps make otherness imaginable” (p. 48). Goodey (2016) refers to society’s perception of individuals with intellectual disabilities as members of an “extreme outgroup” (p. 54), 31 which “remains the undrained sump from which other forms of dehumanization can always draw” (p. 76). By the mid-20th century, disability had come to be viewed from a medicalized perspective (Parsons, 1952), with an increasingly recognized stigmatization of PWDs (Goffman, 1963). Prior to that period, human bodily representations were seen as necessarily flawed, failing to attain the perfection exhibited by the gods (Davis, 2006a). However, as theoretical analyses became more empirically driven, including various social and technical applications of neo-Darwinian thought, individuals who diverged from these new social norms came under increased censure for their differences, which had come to be viewed as deviations from the norm rather than as representations of magico-religious punishment. These new responses to those with differences, created within a period which saw the early development of industrialization, market economies and the emergence of empiricism, included a determination to resolve what was seen as a social problem. This determination was clearly demonstrated within the eugenics movement. First named by Francis Galton in 1883, this popular and widespread movement advocated the improvement of the human species through a variety of direct interventions. These interventions included the sterilization of those deemed unfit to reproduce and the killing of institutionalized populations (Davis, 2006a). People with disabilities often fit into both of these categories (Pernick, 1997). As the eugenics movement became accepted as ‘science’ by the greater population, its tenets became influential in supporting such diverse cultural indicators as consumer products and legislative change. 32 The former often associated with images of acceptable, ‘biologically perfect’ body images and the latter with the so-called ‘ugly laws’, prohibitions against marriage for PWDs, and involuntary institutionalization (Snyder & Mitchell, 2006). Of particular note was the belief that those considered “to be ‘mentally deficient’ were not only inferior but a clear and present danger to the community” (Stainton, 2008, p. 485). As Gregory (2007) points out, the normal distribution curve, now typically referred to as the bell curve, was developed by Gauss in the 1780s as a mathematical representation of a specific statistical distribution of observations. However, it soon became apparent that the normal curve could be applied to social and biological observations. In 1825, Adolph Quetelet, a Belgian astronomer and statistician, was one of the first mathematicians to recognize that the normal distribution curve, or bell curve, appeared to be applicable to social data (Rogers, 1995). One of its first applications to human statistical analysis was based on a measurement of the heights of French military conscripts. The resulting distribution revealed that conscript heights clustered around the mean, or average, and that gradually the sample numbers were reduced as a function of their increasing distance from the mean (Gregory, 2007). Quetelet interpreted these findings as indicating that “the average man is seen as nature’s ideal and deviations from this are nature’s mistakes” (p. 115). Additional experiments with various biological and social measures also revealed similar results. However, the next step, taken by Quetelet and others, transformed the descriptive value of the bell curve into a prescriptive social litmus test. Deviations from the average, or mean, of the bell curve were interpreted as error or mistakes. Davis 33 (2006a) notes that, when the notion of the mean, in societal terms, replaced that of degrees of imperfection, a space was created for disability. And, as theoretical analyses became more focused on social applications of neo-Darwinian thought, disability became more closely defined with deviance as a social analysis and increasingly conflated with “criminal activity, mental incompetence, sexual licence” (p. 9). However, although Quetelet’s findings may have applied to the categorization of physical representations of disability, they were insufficient in terms of determining the degree and direction of difference required to create negative perceptions of PWDs. To return to a consideration of height, the average Canadian man is five feet, nine inches (176 cm.) tall and the average Canadian woman is five feet, four inches (162 cm.) tall (Statistics Canada, 2006). The implications associated with the degree of difference from these means are significant and not necessarily linear. For example, individuals who exceed the average height may find themselves able to excel at various endeavors, such as professional sports. These endeavors may result in prestige, financial reward and social status. Although much beyond what is typically considered to be normal, the height of a star basketball player may be viewed positively. The well-known, and very successful, 1992 “Be Like Mike” advertising campaign featuring Michael Jordan (1.98 m or 6’ 5”), illustrates this point. However, as shown in Figure 2.1, beyond a certain point, human height becomes increasingly perceived as different and negative. For example, Robert Pershing Ludlow, born in Illinois in 1918, is considered by medical historians to be the tallest person who ever lived. When he died, he was 2.72 meters (8’ 11.1”) tall (Taura, 2011). Due to his 34 size, he was unable to engage in many typical life functions. However, his difference, only became a disability at some point during his growth. Perceptions of difference become negative at a socially determined point along a particular continuum. These negative differences have been variously applied to a wide range of social and activity limitations (Nagi, 1976) and continue to inform current models of disability. Figure 2.1: The Abnormal Curve of Disability A common theme emerges from these historical examples of the relationship between society and disability. That commonality is whether a particular individual or group has, within a particular social environment, risen above “the horizon of social notice (italics in original)” (Breen, 2017, p. 2723). Within the context of disability, this tipping point is determined by whether a particular impairment, within a particular Gradations of Difference Disability Options: Different x Bad = Disability Different x Good = No Disability Event Horizon BAD GOOD BAD Difference 35 culture, is perceived as being sufficiently different and negative to be of social notice (Breen, 2017). It has not been historically common (Ginsburg & Rapp, 2013) or necessary to have a particular difference named as a disability; only two factors have been required to generate this notice and incur censure – those of difference and a negative perception of this difference (Vehmas & Watson, 2014). Without that negative perception, disability does not exist. Nagi (1976) points out that specific differences may or may have not have historically fallen into this category. However, it is clear that when that point has been reached, society has imposed various sanctions. Disability, once it is sufficiently differencing to generate social notice, demands action. Priestley (1998) notes that “the degree to which they (PWDs) are integrated or excluded will vary according to predominant cultural perceptions of that difference” (p. 81). Disability, understood as a fluid social construct (Shildrick, 2012), is only perceived as such when this negative difference reaches a critical degree of difference within a particular social environment (Nagi, 1976). For example, beyond a certain point in our current academically-focused environment, challenges with written text may be perceived as sufficiently negatively different to be considered as a disability; however, in past agrarian societies this particular challenge may have been of little notice or concern (Oliver, 1990). Within the context of disability, the horizon of social notice can be defined as “the point at which the differences of others are perceived as sufficiently negative to impose a label of disability”. 36 The relationship between attitude and acceptance. The two terms of attitude and acceptance, and the relationship between them, have been described across a broad spectrum of understandings by various researchers. Definitions of attitude are most frequently described as containing a mental process that is used to evaluate a stimulus (Eagly & Chaiken, 1993; Fishbein & Ajzen, 1975; Gawronski & Baudenhausen, 2006). This mental process is commonly understood as a multidimensional construct which contains cognitive, affective and behavioural components (Eagley & Chaiken, 1993; Gawronski & Baudenhausen, 2006). The evaluation that is determined by the mental process is described as residing along a valenced continuum from negative to positive (Findler et al., 2007; Fishbein & Ajzen, 1975). Finally, the stimuli which generate these evaluations can include objects, ideas and psychological constructs (Fishbein & Ajzen, 1975). First described by Thurstone (1928) as being measurable, attitudes were stated as “the sum total of a man’s [sic] inclinations and feelings, prejudice or bias, pre-conceived notions, fears, threats, and convictions about any specified topic” (p. 529). Attitudes were described shortly thereafter as “a neural state of readiness” which was based on experience and which influenced individual responses to objects or situations (Allport, 1935, p. 810). Fishbein (1963) proposes a differentiation between attitudes and beliefs, wherein attitudes are the evaluative dimension and beliefs are the probability dimension of a concept under consideration. This process of separating attitudes into component parts supported representations of attitudes by many researchers as being most heavily influenced by cognitive processes (Eagly & Chaiken, 37 1993; Fishbein & Ajzen, 1975). Fishbein and Ajzen (1975) argue that the individual is, essentially, a rational processor of information and that evidence did not support an important role for “emotional or affective factors” (p. 215). However, other researchers suggest that attitude formation is initially based on affective processes that either precede cognitive analysis (de Jonge, 2012; Damasio, 1999, 2010; Zajonc, 1980, 2000) or operate in conjunction with or parallel to cognitions (Gawronski & Baudenhausen, 2006; Guidano, 1995). According to Zajonc (2004), this question of the primacy of affective or cognitive aspects of attitude formation will only be resolved by empirical demonstrations. As noted by Fiske (2004), following a review of academic abstracts within the database PsycINFO between the years 1954 and 2000, the cognitive and affective perspectives of attitude formation have been represented relatively equally since 1984. Two additional points are of note with regard to attitudes and their relationship to PWDs. First, although there is an ongoing debate over the roles played by cognition and affect within the formulation of attitudes, each of the three current models of disability described below demonstrates a cognitively-based perspective on the relationship between PWDs and the larger community. However, the Difference Model is differentiated from these by its affect-based perspective. Each of these is considered in more detail below. Second, a growing body of research indicates that attitudes, when applied to PWDs, demonstrate significant discrepancies within different contexts, particularly when associated with employment. In a review of studies of employer attitudes toward PWDs, 38 which had been conducted between 1987 and 2012, Burke et al. (2013) state that employers indicated generally positive attitudes toward PWDs. However, those authors note that these same employers were most frequently reluctant to hire PWDs, most commonly believing that PWDs “cannot effectively perform the nature of the work required” (p. 23). A similar response was given as the principal concern regarding the employment of PWDs by 74 percent of 3,797 employers who participated in a national study conducted by the American Department of Labor (Domzal et al., 2008). In that study, employers indicated generally positive attitudes toward PWDs but most frequently referred to work requirements as a barrier to employment. Of these 3,797 respondents, only 19 percent indicated that they had employed PWDs. Definitions of acceptance also vary considerably within the research literature. Acceptance has been considered as a means of predicting attitude change (Eagly & Telaak, 1972) and has also been understood as a means to quantify changes to specific attitudes (Thurstone, 1928). Within the context of disability, researchers have used the term acceptance as a specific, quantifiable measure of attitude, wherein increasingly positive attitudes are understood to be more accepting of disability (Gething & Wheeler, 1992). Similarly, increasingly negative attitudes are understood as being less accepting, with “nonaccepting attitudes associated with a view of these people as separate or different, the usual implication being that the difference implies deficiency or inferiority” (p. 75). Others, although agreeing that acceptance is a means of measuring attitude, argue that this acceptance of PWDs is based, not on a denial of difference, but “on the 39 absence of impugning the disabled person’s moral character because of the disability” (Taylor & Bogdan, 1989, p. 27). Additional understandings of acceptance are also current. A systematic review of disability awareness interventions for school-age children indicates that acceptance and attitude are often used somewhat interchangeably (Hunt & Hunt, 2004). McLaughlin, Bell, and Stringer (2004), in an employment-related study that considered the relationship between acceptance and disability type, define acceptance “specifically in terms of an incumbent employee’s attitude toward a coworker [sic] who has a disability … and employment judgments about that coworker [sic] (with respect to hiring, promoting, and retaining)” (p. 303). In a literature review of factors affecting the acceptance of disabled employees, Vornholt et al. (2013) acknowledge this variety of interpretations and state that “the term ‘acceptance’ has never been clearly defined when it comes to the acceptance of employees with disabilities at work” (p. 464). Throughout this dissertation, and in my introductory letter to SMEs (see Appendix C), I follow Gething and Wheeler (1992) and make use of the term acceptance as a quantified measure of attitude, with a greater measure of acceptance indicating a more positive attitude toward PWDs in the workplace. Text Box Commentary #2 In 2016, I was contracted by the United Nations Industrial Development Organization (UNIDO) to develop a national strategy for enhancing employment opportunities for PWDs in Armenia. I travelled to Armenia and worked with a diverse group of stakeholders to plan this course. This group included representatives from government, NGOs, the disability community, UNIDO, and private employers. This latter group was, however, not well represented. In an 40 attempt to gather additional information from the employer community, I was invited to meet privately with representatives from several large organizations. At one such opportunity, I attended a large manufacturing organization with two members of the Armenian branch of UNIDO. We arrived, with me using my wheelchair, at a facility with several stairs and no ramp. As we considered our options, two employees approached, picked me and my wheelchair up, and deposited me at the top of the stairs. We were ushered into the employer’s office where it became clear that this interview had been granted as a formality and that there was little interest in the topic of disability and employment. However, as our conversation progressed, it became equally clear that the employer had become interested in observing me as a wheelchair user and as someone who had been introduced as an international consultant from Canada. By the end of our meeting, this employer stated that he was in the planning process of building a new facility and that he was now going to instruct the architect to install a ramp at the front door. Models and Disability Disability studies can be characterized as an effort to examine, conceptualize, interpret and locate those individuals whose physical, cognitive or affective differences are sufficiently apparent to be of social notice (see, for example, Altman, 2001; Shakespeare, 2014; Siebers, 2008; G. Williams, 2001). If we accept that these considerations are based on interpretations of available information, then the discovery of new information allows for new theoretical development. As noted by Shakespeare (1996), recent political developments within the context of disability have opened up opportunities for new stories to be constructed. These nexus, labelled turning points by Charmaz (1991), allow for the possibility of changes in identity on the part of PWDs. This potential for change is supported by Barnes (2003), who recognizes a difference between what that author describes as the natural world and the social 41 world, in that the rules that govern the former are regarded as universal while those that govern the latter are “variable across time, place, cultures and context” (p. 11). Hacking (1999) describes how the iterative nature of the social sciences, and the awareness of the participants within this process, separate the social from the natural sciences. Described by Martinez (2007) as a “looping or feedback effect” (p. 216) this process is of no consequence within the natural sciences but, within the social sciences, it can support changes within those conceptualizations and interpretations. To extend this logic, the recognition and naming of new categorizations, models, and, consequently, new options for intervention, provide opportunities for the creation of new identities and theoretical constructs. In keeping with an emancipatory research framework intended to improve the life opportunities for PWDs (Mercer, 2002; Oliver, 1992a), various interventions will be of value proportionate to their ability to improve these life opportunities. The Difference Model is put forward within this dissertation as the most recent in a line of paradigmatic exemplars which are intended to frame social descriptions of disability and, consequently, provide guidance for improving the lives of PWDs. As described below, the Difference Model is differentiated from these other models in that it presents disability as an attribute that may be located above or below the horizon of social notice independently of modification to either the individual or to the physical environment; thereby providing a new opportunity for the acceptance of those of us who are now perceived as different. 42 Three models of disability are currently supported and critiqued by various disability theorists (see, for example, Oliver, 1990; Shah & Mountain, 2007; Shakespeare, 2006a, 2014; Siebers, 2008). However, before turning to each of these current models of disability and the newly proposed Difference Model, I will contextualize this collection from several directions. First, I will provide a description of the concept of models within the social sciences. Second, I will describe two challenges associated with model creation. The first of these challenges is applicable broadly to model development and evaluation. The second is specifically applicable to models of disability and is concerned with the effects that ongoing social relational and linguistic developments may have on disability-related models over time. Third, I will consider the relationship between disability and other minority or minoritized social groups that are typically considered as stigmatized, with a focus on the “race-class-gender triad” (Davis, 2001, p. 535). Within the social sciences, models have been described as simplified pictures of a part of the real world which contain only some characteristics of that reality (Oliver & Barnes, 2012). Models are distinguished from theories in that the latter are explanatory, while the former are more narrow and are intended only to illustrate or represent specific features, properties or characteristics of a particular construct (Beckett & Campbell, 2014). In separate analyses specific to disability, Oliver (1990) states that “models are merely ways to help us explain, rather than aid understanding” (p. 40); Oliver and Barnes (2012) describe models as “simplified representations of a complex social reality” (p. 23). However, Kools et al. (1996) caution that the selection of a 43 particular model should not be without prior consideration of possible options; the “investigator is methodologically obligated to audition many competing dimensions” (p. 324) in order to determine which provides the most useful analysis and to avoid “bias[ing] the ways in which data were being collected and analyzed” (p. 325). It is possible to create several models of the same construct, each of which has been intended to consider a different aspect of that particular construct. Each of these models may be based on a different structural or procedural dimension of a specific construct, depending upon its purpose (Lave & March, 1975). However, if the selection of that specific dimension as a focal point for a particular model is based upon prior commitments to sociological or politico-ideological positions, the model itself will be inherently biased, creating what has been termed a perspectivized model (Schatzman, 1991). Disability theorists have frequently acknowledged this biasing influence of pre-existing socio-political preconceptions on the development of models of disability. Shakespeare (1994) states that the social model of disability is based on a pre-existing, anti-capitalist ideology. This claim is supported by Corker (1999b), who notes that the development of models of disability have “often [been] designed from within particular perspectives conceived at particular times in history which do not reflect ‘disability’ and ‘society’ as we understand it today” (p. 628). Priestley (1998) creates a four-quadrant matrix of individual versus social and materialist versus idealist dualities to demonstrate the four main approaches to considerations of disability theory. That author then argues 44 that each of these is based on premises that support these models but that none meet the criterion of being testable. Other writers recognize and acknowledge their reliance on prior personal experiences to inform model development. Oliver (2009) notes that “social theories are grounded in the knowledge the theorist has gained through personal experience [and] are of course utterly persuasive to the theorist” (p. 19, citing Gouldner, 1975, p. 70). Similarly, Thomas (1999a) acknowledges that any attempts to explain disability-related questions are driven by the previously developed theoretical framework one applies to the social world. That author, in a reference to her interpretations of the disability-related experiences particular to women, suggests that “like all sociologists I inevitably construct an account…according to the contours of my own intelligence and experiential biography” (p. 48). Rioux (1997) extends this argument by stating that researchers must acknowledge their worldviews, in recognition of the fact that these will influence any subsequent interpretations of disability. Rioux also asserts that researchers have a moral imperative to consider and disclose the underlying premises of their research, particularly in the field of disability, where this research affects a particularly marginalized group of people. Support for this argument, within the context of vocational rehabilitation counselling, comes from Crisp (2011). Crisp notes that the reported success of a particular school of counselling, observed through a meta-analysis of 186 studies involving more than 14,000 clients, was ultimately attributed in large part to the allegiance of the authors of that meta-analysis. Those authors were found to be 45 supporters of the counselling technique which they determined to be the most effective within that study. A number of safeguards have been proposed to address this ontological challenge to the premises supporting various models of disability. Kools et al., (1996) argue for a comparative analysis to safeguard against the incorporation of premises that influence analyses of models toward a specific goal, whether purposeful or inadvertent. Another approach to safeguard against this challenge is to consider how best to evaluate a particular model. Altman (2001) states that the value of a model is determined by its ability to correctly predict new facts. Specifically, this determination of value requires that a model must be testable and should be evaluated based on its ability to consider empirical problems in the real world. Most importantly, this relationship of a model to the real world can only be determined by evaluating the truth of that model’s derivations or predictions, rather than engaging in debate over the truth of its assumptions (Lave & March, 1975). Another challenge associated with the interpretation of models of disability is the impact of changing social relationships and language usage. Abberley (1987) states that the premise of oppression, as contained within the social model of disability, was first used as an equivalent to exploitation (citing Oliver, 1986) but later was interpreted more structurally, as “an ‘obvious’ but difficult to substantiate characteristic of ‘social relations under capitalism’ ” (p. 6). Abberley also maintains that, to be considered oppression, activities must contain a degree of purposefulness. However, Barnes and 46 Mercer (2003) argue that various social activities may generate oppression through inadvertence. Shakespeare (2014) notes that disablism, understood as containing oppression in a manner similar to institutionalized racism, includes “processes, attitudes and behaviour which amount to discrimination through unwitting prejudice, ignorance, thoughtlessness and racial stereotyping” (p. 29). Owens (2015) summarizes this dilemma by stating that “oppression is a nebulous concept, poorly understood and undertheorized” (p. 389). Similar to these changes in definition associated with oppression, the term disability has been variously defined and characterized within the several models of disability (Haegele & Hodge, 2016). Disability has been variously considered as: a social construction within a range of cultural contexts (Priestley, 1998); as resulting from social forces unwilling or unable to support impairments (Oliver, 1990); a form of oppression based on these social restrictions (Thomas, 1996b); a result of the meanings that perceptions of disability have on social identities (Swain & French, 2000); as akin to impairments related to bodily systems (Altman, 2001; Watson, 2012); as an interrelation between the biological and the social (Anastasiou & Kauffman, 2013; Shakespeare & Watson, 2001); and as a perception by others of insufficiency (Galvin, 2003). Silvers (2003) acknowledges this broad spectrum of definitions and states that the term disability is sufficiently underspecified so as to allow for a host of interpretations based on political values. Shildrick (2012) chooses to withhold a definition of disability, considering that state to be “a shifting nexus of both physical and mental states that resists full and final definition” (p. 3). 47 Thomas (1999b) notes that, even within the same paper, writers often present different interpretations of disability, at times conflating or otherwise misunderstanding a social relational interpretation and a property interpretation of that term. That author describes the former as a disadvantage or restriction caused by social forces and the latter as a personal characteristic which, when coupled with social factors, results in restricted activity. Thomas maintains that confusion over these differing interpretations, and their appropriate applications, has led to a number of unproductive debates with each side arguing in support of its conclusions based on dissimilar premises. Concerns have also been raised about how the rapid and ongoing changes in the meaning of the term disability have caused this term to be understood differently by different groups (Silvers, 2003). These categorical interpretations are “slippery, fluid, heterogeneous, deeply intersectional, and thus resistant to definition” (Shildrick, 2012, p. 4). It is also suggested that these differences in interpretation have been purposefully developed to meet specific organizational or structural requirements. For administrative purposes, disability is often considered to be a multidimensional construct subject to change over time; from a medical perspective, disability is typically defined with a focus on impairment as it relates to particular bodily systems; within a scholarly or academic approach, the construct of disability is described through various efforts to provide a conceptual framework that will contain this phenomenon (Altman, 2001). 48 Researchers have also focused on the relationship between understandings of the term disability and the evolving cultural politics of language (Corker, 1999a; Peters, 1999). Priestley (1999) maintains that the evolving nature of the language of disability itself may be inadvertently reinforcing the stereotypes that theorists are intending to eliminate. Based on changing perceptions and prejudice, these fluctuations have influenced how PWDs are perceived by others and themselves (Haegele & Hodge, 2016). Corker (1999a) states that it is clear that finding a new word to represent a minority group or culture has been the obvious choice in other circumstances. And, when a word becomes inappropriately value-laden or pejorative, it is easier to move on. These changing applications of language are also seen by many individuals living with impairments as being detrimental to their self-images and creating unwanted identities of disability (Peters, 1999; Priestley, 1999). There is an emerging discussion among disability theorists over the effect that these several challenges exert on the understanding of disability. In a recent consideration of the several models of disability, Beaudry (2016) states that proponents of the different models may be operating at cross-purposes, being driven more by concerns over supporting pre-existing political positions than by considerations of current topics affecting PWDs. That author recommends that the concept of disability itself should either be left sufficiently broad and inclusive to allow for these considerations or, alternatively, that the concept of disability itself could be abandoned. This latter option would allow for meaningful dialogue “without being hindered by 49 debates about the nature of disability” (p. 210). Other recommendations regarding the future use of this term have also been put forward. Linton (1998) states that the term should be retained but with a new meaning. Oliver and Barnes (2012) argue that the term ‘disability’ should be maintained for its political value, while acknowledging that there is a wide range of understandings of the term across cultures and time. Galvin (2003) recommends the dissolution of the disability identity as a means to avoid confusion based on prior definitional interpretations. Although there continues to be significant debate with regard to understanding disability, there is general agreement among researchers that attitudes toward PWDs are relatively negative. Goffman’s (1963) book Stigma: Notes on the Management of Spoiled Identity has been widely cited and “inspired a profusion of research on the nature, sources and consequences of stigma” (Link & Phelan, 2001, p. 363). In that book, stigma is defined as an “attribute that is deeply discrediting” which reduces the stigmatized individual “from a whole and usual person to a tainted, discounted one” (Goffman, 1963, p. 3). This definition follows earlier social theorists who argued that the unfamiliarity of objects or people led to negative attitudes based on uncertainty (Gething & Wheeler, 1992). In a review of the sources of negative attitudes specifically toward PWDs, Livneh (1988) offers a perspective which considers six possible dimensions. These are positioned as affective/cognitive, conscious/unconscious, past experience/present situation, internal/external, and theoretical/empirical. Considerations of each of these dimensions list 63 possible antecedent motivators, which include a range of psychic 50 threats to observers, as well as several demographic variables and personality characteristics of the PWDs and/or observers. When these motivators are considered in toto, two common themes emerge. The first is that the resulting attitudes contain a significant degree of ambivalence (Katz, Hass, & Bailey, 1988); the second is that the principal psycho-social motivators held out in support of various explanations apply pre-existing theoretical understandings of social relational interactions as supporting evidence. In an overview of the literature regarding what is described by Gething and Wheeler (1991) as the prevailing negative community attitudes toward PWDs, these authors suggest that these attitudes are informed by a range of underlying emotions; these include fear of the unknown, threats to personal or cultural security, fear of vulnerability, guilt, discomfort with tragedy, and aversion to weakness. Others suggest that these same negative attitudes result from somewhat different influences. Stone and Colella (1996) state that these are influenced by the nature of the disability, the attractiveness of the individual, and expectancies of the observer. According to some researchers, these attitudes are acquired through experience but are subject to change (Antonak & Livneh, 2000; Hunt & Hunt, 2004). Goffman (1963) supports the perspective of perceived difference, as a generator of stigma, as one of relationship rather than attribute. That author notes that what was needed was “a language of relationships, not attributes” (p. 3). This relational application of stigma has been extended to include the purposeful imposition of that designation by others (Parker & Aggleton, 2003). However, most subsequent 51 interpretations have, to a large extent, migrated the understanding of stigma from one associated with relationships to one associated with personal attributes (Fine & Asch, 1988; Link & Phelan, 2001). Research has linked these negative associations of stigma and personal attributes to a wide range of ‘discrediting’ circumstances. Examples within the sphere of disability include urinary incontinence, leprosy, cancer, mental illness, wheelchair use, and unemployment (Link & Phelan, 2001). These understandings, or impression formations, of others have been described by Fiske and Neuberg (1990) as occurring along a continuum with category-based processes at one end and the particular attributes of the referent at the other, with the former having priority over the latter. These authors have attempted to reconcile these two perspectives which have been applied to support the process of impression formation. As they note, “When Asch (1946) asked whether impression formation operates by holistic, gestalt processes or by elemental, algebraic processes, there began a running debate within impression formation research” (p. 61). Fiske and Neuberg (1990) describe a five-stage process that is initially weighted toward a matching of an individual to a predetermined category but which evolves into one more focused on personal attributes if there is not a reasonable fit with the original category. This process begins as “perceivers initially categorize others immediately upon encountering information sufficient for cuing a meaningful social category” (p. 4). This notion of meaningful implies a cognitive process that “determines whether the target is minimally interesting or personally relevant enough to warrant further processing” (p. 4). 52 This construction of difference, categorized through a cultural framework, is also supported by Connolly (2002). That author, taking a purposeful, cognitive approach, states that “an identity is established in relation to a series of differences that have become socially recognized” (p. 64). Connolly argues that the individual identity, in this case a member of the normal group (Goffman, 1963), requires that perceived difference is purposefully converted into otherness in order to support the individual’s self-identity. Dhamoon (2009) suggests that the most frequently described framework used to consider the relationship between identity and difference has been constructed within a context of social power, with difference being associated with negative social connotations by the majority of normals (Goffman, 1963). That author, through what is described as a Foucauldian perspective, argues that identity is produced as an effect of making meaning of difference, whether through a coercive or productive process. Within this framework “identity is difference” (italics in original), (p. 11); identity is understood as a relationship generated through a meaning-making effort based on perceptions of difference. Dhamoon “use(s) the term “identity/difference politics” to signal a relationship between these two concepts” (p. 11). This relationship between identity and power is supported by Gamson (1995) with reference to sexual identities. The queer movement is described as presenting itself as a difference that is entitled to acceptance, while remaining as a part of the outgroup. However, this author also notes that, although the creation of a clear, collective identity has supported lesbians and gay men in making themselves an 53 effective force within the larger community, “disrupting these categories, refusing rather than embracing ethnic minority status, is the key to liberation” (p. 391). This dilemma is “shared by other identity movements (racial, ethnic, and gender movements, for example)” (p. 391). Although, according to Gamson, fixed identity categories may be the basis for political power, they are also seen as the basis for oppression. In each of these examples, regardless of whether a cultural or a social power framework is favoured, each is based on a premise of intentionality. The creation of the identity/difference relationship is purposeful, whether initiated by those on the ingroup or outgroup side of the equation and whether perceived as based on relational characteristics or on individual attributes. Each is based on a process of meaning making. However, others argue that these attitudes are automatic and innate, preceding the meaning making of cognition (Pruett & Chan, 2006; Zajonc, 1980, 2000, 2004); still others argue that attitudes are formed within two separate streams of mental activity, those based on cognitive processes, which require higher level mental activities, and those which are more immediate affect-based responses to external stimuli (Gawronski & Baudenhausen, 2006). Text Box Commentary #3 In 1982, I started a company (AmbuTech Inc.) that designed and manufactured health care products*. Principal among these products were canes – canes for support, canes used by people who were blind or who had low vision. One specialty product was designed for those who required both support and a visible indicator that the user was blind. My role was designer, manager, marketer, and, for the first few years, almost everything else. Gradually, as our 54 products gained acceptance in Canada, the United States and internationally, I began to hire employees to take on many of these responsibilities. In 1986, I made the decision to only hire PWDs and, by 1988, seven of my eight employees were PWDs. These included a lathe operator, several production and assembly workers, a salesperson, and my office administrator. There were no special jobs for PWDs, there were employees who were PWDs. Accommodating various disability-related needs was commonplace. We developed a ‘buddy’ system for one employee who had previously sustained a significant head injury. This injury had left him with virtually no short-term memory. After any break, a co-worker would demonstrate the last assembly task that he had been performing and he would be able to return to his work. Our lathe operator, who was blind, spent much of his time cutting aluminum tubing to exacting specifications. We built a special jig that would allow the work to be done by ‘feel’ instead of sight. Another employee had an uncontrollable condition that would bring on several seizures every week. All employees received training to support him during those times. By the time I sold that company in 1991, it was clear to me that what is considered ‘normal’ is more about what you believe than about the status quo. *(Chishom et al., 1988) [MacLean’s Magazine]. Models of Disability The three most prominent models of disability – the medical, social and embodiment models – provide a description of disability that is intended to address commonly agreed upon discriminatory social challenges. Each of these models contains assumptions that change is both possible and necessary. Each agrees that life could, and should, be better for those with disabilities. As described in more detail below, these models are differentiated, not by adopting distinct dimensions within the construct of disability, but by their positioning along a common dimension of political and social responsibility. These different understandings, whether focused on the individual PWD (medical model), society (social model), or a combination of the two 55 (embodiment model), inform correspondingly different methodologies for resolving these agreed upon social challenges. These three models also share one additional feature - each considers attitudes toward PWDs principally from a cognitive perspective associated with intentionality and meaning making. Finally, each maintains that, with the implementation of appropriate changes, the difference associated with disability, or at least the negative aspects of that difference, should sink back below the horizon of social notice and, essentially, disappear. However, as demonstrated by the widespread and continued low employment rates for PWDs (Burke et al., 2013; Turcotte, 2014), none of these three models has been successfully applied in reducing that particular differencing aspect of disability. In one of the very few examples of a disability theorist addressing this relationship, Oliver (2013) notes that the social model “has barely made a dent in the employment system” (p. 1025). However, new options have recently been developed that allow for a further consideration of the relationship between disability and society. As noted by Shakespeare (1996), new narratives are being created which allow for new interpretations of disability. In addition, recent scientific developments promise further considerations of this relationship. This relationship between PWDs and society can now be altered, at least in some cases, through relatively commonplace technological means (Breen, 2015; Wolbring, 2008). It is within this new context, which contains the potential for a change that goes beyond crafting a new position along the dimension of responsibility, that I will describe the Difference Model of disability. But first I will 56 provide a summary of each of the medical model, the social model, and the embodiment model. The Medical Model The medical model of disability emerged in the late 19th century. Based on a humanist interpretation of the social environment, the medical model replaced the magico-religious notion of universal imperfection with defined boundaries between the acceptable and the unacceptable (Foucault, 1988; Stainton, 2004). The developing interventionist nature of medical science (Barnes, Mercer, & Shakespeare, 1999) coupled with the ethical support provided by the application of newly developed statistical tools to the social sciences, in which the norm became associated with the good, were combined to repair or transform those considered to be deviant (Rogers, 1995). This deviance could now be overcome, and medical science became the recognized corrective instrument. Each of the several iterations of the medical model was a response to moral or religious explanations of disability. Whether premised upon modernist ideology (Barnes, Mercer, & Shakespeare, 1999), the industrial revolution (Oliver, 1990), language (Goffman, 1963), or a rift between reason and unreason (Stainton, 2004), new scientific methods, and the medicalization of disability, were understood to be a progressive way to understand and treat illness (Oliver, 1990). The emerging medical model became an application of what was then considered to be ‘cutting edge’ technology and newly discovered scientific understandings of individual impairments (Shakespeare, 2006b). 57 Regardless of the intent that motivated attempts to relieve suffering and to repair what were seen as physical, emotional and mental aberrations, the medical model was premised on incapacity, illness and an authoritarian relationship between care providers and recipients. In addition, the perception of disability as individualized personal tragedy and incapacity was well entrenched within the medical community (Barnes, Mercer, & Shakespeare, 1999) and accepted as such by various policy-making bodies (Birch & Longmore, 2009). The medical model has been criticized for its lack of consideration of the subjective experience of impairment, including the social causes of at least some illnesses (Oliver, 1990). However, the medical model cannot be so easily dismissed. Although often portrayed in archaic, 19th century imagery associated with medical misconceptions and gruesome procedures inflicted upon those with disabilities for “their own good” (Barnes, Mercer, & Shakespeare, 1999), the medical model developed within a period of ontological reconstruction. Modernism, with its emphasis on the individual and a positivist worldview, was replacing long held religious dogma that stifled scientific exploration and precluded experimentation as a basis for knowledge. This transition allowed for scientific solutions to be applied to individual impairments (Shakespeare, 2006b). The development of the medical model was the embodiment of this transition. Although proponents of the social model argue against the continued acceptance of the medical model, that model continues to be acknowledged for its transitional value (Oliver, 1990). The medical model introduced the notion that disability could be remediated. The way to that remediation included the 58 application of technological discoveries, newly developed surgical techniques, and pharmaceutical interventions (Aptowicz, 2014). These were the first step taken to resolve the perception of disability as difference. By remediating those with disabilities, they would be transformed into what Goffman (1963) would later refer to as normals. A mid-twentieth century medical model analysis of the relationship between those with disabilities and society was presented by Parsons (1951) who described illness as deviant behaviour. He saw those with disabilities as adopting a sick role, which was a failure to meet the expectations of individual participation and contribution within society. Parsons maintained that those who were ill were not at fault for their illness but did have a moral responsibility and corresponding duty to make every effort to return to a state of proper functioning. This requirement of recovery included the seeking and accepting of appropriate medical advice (Shilling, 2002). Within this description, the behavioural requirements of PWDs included their support for technological interventions. More current descriptions of the medical model of disability include those which focus on the absence of disease and the ability to carry out typical bodily activities and functions (Boorse, 1977). The concept of disability continues to be defined as a deviation from structural or functional norms, caused by trauma, disease or other health conditions (Bickenbach et al., 1999). The medical model, at its core, focuses on illness and the body in which it is seen to be situated. The intent is to intervene in a manner that will reduce functional limitations based on biological impairments. This focus is still seen as predominant, 59 although many current proponents and practitioners of the medical model acknowledge the environmental impact on both the cause and consequences of disease (G. Williams, 2001). Proponents of the medical model argue that it is being misrepresented (Shah & Mountain, 2007). Scientific rigour, characterized by “observation, description and differentiation, which moves from recognizing and treating symptoms to identifying disease aetiologies and developing specific treatments” (p. 375) is, according to these authors, inaccurately portrayed as relegating people with disabilities to the status of the problem. It is argued that this latter approach, as presented by those opposed to the medical model, paints a “caricature of a reductionist, mechanistic, disability-enhancing approach, taken by powerful doctors towards patients” (p. 375) in an effort to bolster their own opposing positions. Many supporters of the medical model maintain that it continues to be invoked as a theoretical scapegoat by a variety of writers in need of a construct to oppose (McKee & Chappel, 1992). However, although the medical model is generally accepted as a curative model that has demonstrated success when intervening in the acute stages of impairment, its most significant criticism is that it does not adequately address the realities of life of those with stable, functional limitations (Fougyrollas & Beauregard, 2001). Regardless of the limitations associated with the medical model, it has been widely acknowledged as having increased survival rates and life expectancy for many disabled people (French & Swain, 2001; S. Williams, 1999). To illustrate this point, prior 60 to the 1940s, eighty to ninety percent of those who acquired a spinal cord injury died within a few weeks. As antibiotics and other medications improved during and after World War Two, the number of survivors grew exponentially. Over the next thirty years, survivors of spinal cord injuries were often able to return to their communities where they encountered another problem, that of interacting with a population that had no experience with disabled people (Trieschmann, 1988). This experience serves to foreshadow the development of the social model described below. The Social Model The social model of disability rejects the medical model for what is claimed to be its authoritarian stance relative to those it is purported to serve, its assumptions about those with impairments as living lives of “personal tragedy”, and its inability to differentiate between chronic illness and impairment. However, the social model reserves its most pointed criticism of the medical model for failing to distinguish the socially created construct of disability from individually experienced impairments (Oliver, 1990; Oliver & Barnes, 2012; Shakespeare, 2006b). By introducing a conceptual dichotomy between impairment and disability, the social model shifts the blame and responsibility (Siminski, 2003), and what Goffman (1963) called the stigma of disability, from the individual to society. This reformulation allowed for the notion of personal tragedy, as imputed to the medical model, to be decoupled from impairment (Barnes, Mercer, & Shakespeare, 1999; Oliver, 1990). However, the separation and clarification of each of these terms did not spring fully formed into existence. 61 Early theoretical differentiations between the concepts of impairments, functional limitations, and disability were put forward by Nagi (1976) who suggested that one’s gaze must extend beyond individual limitations to understand disability, which that author considered to be a combination of individual characteristics and social requirements. In that same period, the Union of the Physically Impaired Against Segregation (UPIAS, 1976) put forward a proposal that clearly distinguished between the constructs of impairment and disability. The former was described as the functional limitation within the individual; the latter was described as the limitations imposed on those with impairments by social and physical barriers. This theoretical shift also put forward an acceptable option to the medical model that was politically accessible to PWDs. Within this new model, the personal tragedy of PWDs within the medical model was transformed into one of oppression on the part of others (Oliver, 1990). This shift to a social oppression theory of disability presented disabled people “as the collective victims of an uncaring or unknowing society rather than as individual victims for the tragedies that have befallen them.” (p. 2). This new relationship was also articulated by the World Health Organization in its International Classification of Functioning, Disability and Health. That model asserts that “disability is not an attribute of an individual, but rather a complex collection of conditions, many of which are created by the social environment” (WHO, 2001, p. 28). These concepts were also being considered by a number of researchers who argued 62 that the social environment is more of a barrier than individual impairments (Shakespeare, 2004). The strategic value that this overall shift of perspective had for many PWDs was significant. Disability, if interpreted as a socially constructed barrier, justified PWDs in actively seeking remediation from the injustices and oppression created by that barrier (Oliver, 2004). The social model shifted the onus from that of Parsons’ (1951) requirement for the repair or correction of individuals to that of social change (Shakespeare, 2004). The responsibility for remediation was repositioned from the individual to the social environment. As delineated by Shakespeare and Watson (2001), the social model of disability was further parsed into the strong and weak models. The former, also called the UK model, included three key elements. These were: disabled people are an oppressed group, impairment is distinguished from oppression, and disability is defined as social oppression. The weak social models of disability do not define disability as social oppression. These include the American minority rights model and the Nordic or relational model (Gustavsson, 2004). The American model, beginning in the late 1960s, was developed by those involved in the Independent Living Movement and by the growing number of disabled veterans of the Vietnam War (Braddock & Parish, 2001). Members of both of these groups rejected what were the current social expectations of their ongoing dependency on a range of medical and rehabilitation professionals. Both groups also 63 became involved in the burgeoning social protest movements of the time that were advocating for civil rights (Stark, 2001). This changing social climate in the period following the Vietnam War included a shift in relationship between the disability community and the majority population in North America. A new self-assertiveness was fostered, in part, by disabled war veterans who had not lived through a childhood and adolescence as people with disabilities. Consequently, they had not been indoctrinated into a medical model assertion of disability as being equated with illness (Oliver, 1990), and did not know that they were ‘exceeding their place’ when demanding increased services and independence. The American model defined disability as the relationship between impairments, functional limitations and the social environment (Chan et al., 2009). Even without assuming a necessary connection between disability and oppression, this growing understanding of the interaction between impairment and the environment as a source of disability provided the impetus for the North American disability community’s rights-based advocacy for self-determination. The American social model of disability, unlike the UK model, adopted the political position of a social minority in search of its rights of citizenship. Within a minority paradigm, disability became a suitable subject of legislative reform in a manner similar to that demanded by African-Americans and other minority groups (Safilios-Rothschild, 1976, as cited in Barnes, Mercer, & Shakespeare, 1999). An attempt to address this demand was made by the American federal government with the passage of the Americans with Disabilities Act (ADA) in 1990 (Danermark & Gellerstedt, 2004). 64 American legislative reform, culminating with the ADA, has served as the single, most comprehensive example of a legislative solution to establishing disability rights (Bickenbach, Jerome, Chatterji, Badley, & Ustun, 1999). However, ongoing concerns have been raised regarding the lack of improved employment statistics in the years following the implementation of this legislation (Donoghue, 2003; Burkhauser & Stapleton, 2004). The American civil rights model has most often been critiqued by those supporting the strong social model. Barnes (1999) castigates American and Canadian writers for using the terms “disabled people” and “people with disabilities” interchangeably. According to Barnes and many other UK writers, the ‘people first’ language of the North American model does not sufficiently acknowledge that disability has been imposed on those with impairments through social oppressions (see, for example, Barnes, 1999; Shakespeare & Watson, 2001). However, Titchkosky (2001) argues that, in keeping with its rights-based approach, the North American model approaches disability as an attribute of the person, with the ‘person first’ language signifying that relationship. The Nordic model developed from the self-advocacy movement of the 1970s within which PWDs began to demand more autonomy and control over their daily lives (Barnes, Mercer, & Shakespeare, 1999). Supported in recent works by Bhaskar and Danermark (2006) and Shakespeare (2014), this approach positions disability within a multi-layered, critical realist ontology, suggesting that “people are disabled by society and (italics in original) by their bodies and minds” (p. 5). 65 These have included arguments against the necessity of oppression as a component of disability (Shakespeare, 2006a; Shakespeare & Watson, 2001); the problem of conceding that the impaired body ‘belongs’ to medicine by abandoning impairment as a component of disability and viewing it only through a medical lens (Hughes & Paterson, 1997); and a consideration of the individuality of disability through an analysis of subjective phenomena such as pain (Siebers, 2006). Researchers have also provided analyses and interpretations of specific aspects of the social model from a variety of perspectives. Thomas (1999b, 2007), writing from a feminist, social model perspective, describes the recognition of what are termed impairment effects as a means to resolve the binary of disability and impairment. Although maintaining a position in support of the social model, that author argues that it is necessary to distinguish the results of impairments that are socially caused and, therefore, considered to be disabilities, from those that are inherent to particular limitations, considered to be impairment effects. Several other writers with a feminist perspective see value in exploring the perspective of the social model within a social constructionist context. Concerns are raised that individual factors, which can limit activity regardless of available accommodations, are not well considered within the social model (Crow, 1996). In addition, the individual experience of various persistent limitations is not explained by disability as oppression (French, 1996, as cited in Shakespeare & Watson, 2001). As Oliver (1992b) was delineating the boundaries of the social model within which social change could occur, others were offering more broadly based options. Zola 66 (1989) posited a universalism of disability. Although in support of a civil rights framework, that author argued that only through a ‘universalist’ approach can disability be demystified and the notions of different and special be eliminated. Chronic illness and disability await the entire population. Therefore, policies and processes that recognize this population risk, and which eliminate the separation of those with disabilities from the general population, are needed. Shakespeare (2004) notes that several aspects of the strong social model have been counterproductive and unnecessarily alienating of others. These include the anti-capitalist thought that has influenced the strong social model, the counterproductive rift between the strong and weak social models, and the rejection of social model variants proposed by non-disabled researchers. He also notes that the social model movement has been dominated by those with physical impairments and has been much less successful in appealing to members of the Deaf community, those with mental health issues, seniors with disabilities, and others. Oliver (1992b, 2013) responds to claims regarding the theoretical inadequacy of the social model by stating that it was never intended to fill this role. It had always been intended as a practical tool to support practice and to produce political and social change, including a recognition of the essential and causal nature of the relationship between society and disability. However, Oliver (2013) also acknowledges that implementation of the social model has not had any impact on rates of employment for PWDs. In addition, that author states that “it is time to either re-invigorate the social model or replace it” (p. 1026). Finally, Shakespeare (2004) argues that the time has 67 come to reject the strong social model. “Rather than a social theory of disability, we need a social theory of embodiment” (p. 20). The Embodiment Model The embodiment model of disability has provided a response to the social model, with a more materialist grounding to disability. Garland-Thomson (2006) points out that the social model is insufficient to address bodily issues associated with disability. Disability theorists, according to Barnes and Mercer (2010), are now engaged with new theories of the body and considering how the body has become key to understanding identity. As Hughes and Paterson (1997) argue, the exclusion by the social model of impairment as a conceptual component of disability creates an “untenable separation between body and culture” (p. 326). Furthermore, this separation limits disability by requiring it to be dependent on oppression for its existence (Shakespeare, 2006b). The embodiment model of disability attempts to consider and place the subjectivity of the lived, bodily experience within the life-world (Turner, 2001). Bringing the body, with its concomitant impairments, back into the discussion of disability allows for identity, and identity politics, without requiring the construct of disability as oppression. Identity politics offer a positive representation to its members and to others, with difference from the norm as the common theme (Darling, 2013). Although identity theory, as extracted from Symbolic Interactionism, is capable of encompassing a range of self-perceptions, disability politics represents these differences as affirmative identities rather than the tragedy model of disability (Swain & French, 2000). Paradoxically, these various identities, including disability, require the 68 maintenance of the challenged division for their own existence, even if only the attitudes directed towards those acknowledged differences (Barnes & Mercer, 2001). Particularly in the United States, disability scholars and activists have argued for the inclusion of disability into what Davis (2001) refers to as the “race-class-gender triad” (p. 535). The intent of each of these identities is to win full rights of citizenship. However, this model of disability is critiqued as being distinguished from other identities in several ways. Its membership is not clearly defined. Individuals can and do leap that divide, in either direction, for a number of reasons including age, accident, recovery or definitional change. Unlike identities such as ethnicity and gender, the identity of disability is heterogeneous and somewhat arbitrary (Davis, 2013; Ginsberg & Rapp, 2013; Shakespeare & Watson, 2001). A further distinction from other identities is that disability is seen as inherently containing some degree of impairment and can only be understood if it includes limitations to the body and activity restrictions (Shakespeare & Watson, 2001; Thomas, 2004). Although these limitations may be caused by environmental or social influences, “it is the individual body within which illness is situated” (G. Williams, 2001, p. 125). Identity politics also draws additional critiques. Sherry (2004) categorizes these principally as the debates between constructionism and essentialism. The constructionist perspective describes identity as resulting from cultural and social conditioning and, therefore, capable of adjustment. The essentialist understanding of identity is one that is inherent, permanent and natural. According to Galvin (2003), identity politics is unavoidably essentialist, regardless of whether construed as 69 irreducible due to biological or humanistic causes, leaving the construct with little value. This position is contrasted with that of Oliver (1992b), who supports an essentialist perspective, criticizing social constructionists and social idealists alike for “denying the essential nature of disability and this existence of disablism throughout society.” (p. 22). This challenge has also been addressed by Imrie (2004), who attempts to bring these opposing viewpoints together. That author argues that the impaired body can be viewed as being constructed of physiological, social and discursive components, existing at the intersection of the social, biological, and psychological. This position is closely aligned with that put forward by the International Classification of Functioning, Disability and Health (WHO, 2001), which considers disability to be a variation of functioning caused by any combination of individual impairment, activity limitations, or participation restrictions (Imrie, 2004). This debate has also been considered by Davis (2006b) within a description of the challenge facing disability identity politics if it is included within the collective of oppressed minorities. Davis anticipates a first-level critique of reductionist essentialism, countered by rejoinders demonstrating that identities are socially constructed. But this argument for a social construction of identity is actually what that author uses to raise a more basic concern. That is, whether a social construction of identity leaves space for any core identity. Davis (2006b) refers to this as merely a politically motivated position of “strategic essentialism” (p. 236).This same objection is raised by Hacking (1999) who claims that without a biological basis for disability, a social construction of that identity 70 may not be possible, leading to the retention of a construct of identity simply for strategic purposes, even without evidence of a theoretical or biological underpinning. It is also suggested that, rather than abandoning identity politics altogether, a ‘universalist’ approach to impairment is most appropriate, whether that approach is based on the temporary nature of able-bodiedness (Shakespeare & Watson, 2001; Zola, 1989), a common morality based on a universal ontological identity (Timpanaro, 1975, as cited by Hughes, 2009), or a universalism of injustice and bodily limitations (Davis, 2006b). Finally, as noted by Hughes (2009), each of these raises another concern. If everyone is impaired, a biological essentialism and reductionism akin to that critiqued in the medical model re-emerges. However, Shildrick (2012) states that any superimposed binary between PWDs and others is so unstable as to be meaningless, arguing that any perception by members of the normative majority of a loss of appropriate cognitive functions, control over one’s body, or a state of interdependency, generates “a deep-seated anxiety that devalues difference” (p. 2). Shakespeare and Watson (2001) attempt to avoid the critique of biological essentialism by distinguishing those with impairments who can claim oppression from those who cannot. The International Classification of Functioning, Disability and Health (WHO, 2001), has attempted to avoid this pitfall by presenting disability as falling somewhere between the social model and the medical model, allowing for the joint creation of disability by bodily and social means (Bickenbach et al., 1999; Imrie, 2004). However, this debate is discounted by M. Russell (2002) who notes that the low rates of employment of PWDs are rooted in 71 purposefully discriminatory attitudes of employers, which are not significantly affected by arguments related to identity politics. That author states that a “traditional civil rights model would not serve to provide equal [employment] opportunities for disabled people in the labour force” (p. 127). M. Russell argues that the premise of discriminatory employer attitudes being driven by irrational factors capable of remediation is incorrect. It is the rational, albeit incorrect, decisions made in the name of pre-existing capitalist financial goals that discriminate against PWDs. Whether disability is conceptualized within an essentialist or constructionist framework is irrelevant; it is employers’ perceptions of PWDs as both incapable and expensive, each of which raises concerns over profit, that perpetuate low employment rates. The Difference Model Before presenting a description of the Difference Model, I will offer a brief explanation of the name itself and a further rationale for its development. The name of the Difference Model has been chosen for two reasons. First, it provides a more neutral representation of those who live within this category that is not engaged with the various politically influenced phrases such as disabled people, people with disabilities, the disabled, or the impaired (see, for example, Barnes, 1999; Titchkosky, 2001). A neutral definition also avoids the implication that disability is a devaluing characteristic or that PWDs are inadequate (Silvers, 2003). Second, it provides a reference that can be understood differently by individuals with differing political values. This includes individuals who may see themselves as excluded or atypical but not necessarily living under a banner of disability. This latter group includes seniors with late-in-life 72 challenges, the Deaf community, some with learning difficulties, and youth, many of whom do not consider themselves appropriately described within the current nomenclature (Priestley, 1999; Shakespeare, 2014). The Difference Model offers a framework for understanding disability as part of an expanded normal, not as a characteristic that, with appropriate individual or social modifications, is able to fit within current parameters of normalcy. This is accomplished by adopting the perception of difference as the dominant descriptive dimension of disability. As noted by Schatzman (1991), models are typically developed with a focus on a specific, dominant dimension, which serves as the principal indicator in determining the underlying meaning associated with that model. Within the several current models of disability, the relationship of responsibility, with its implications of blameworthiness, has served as the principal dimension upon which each has been constructed. The dimensional dominance of responsibility is rejected by the Difference Model in favour of that which informs these several variations of blame – the perception of difference. This perception is understood as the underlying affective response to difference that informs the different cognitive meaning making processes described within each of the medical, social and embodiment models of disability. Within the Difference Model, disability is a personal attribute that generates valenced responses by others as informed by perceptions of social norms. To accommodate PWDs, the social ground held by the normals requires an expansion of what Goffman (1963) called the in-group. This is quite different than an 73 acknowledgement that PWDs are entitled to be treated as if they were within the in-group. The Difference Model has been designed based on this understanding of the lives of PWDs and to support specific strategies designed to improve those lives. The first of these strategies is that of the CADE Scale. It is intended as a means to quantify the degree of difference by which disabled employees are perceived by co-workers and measure the effects of interventions designed to reduce this perception. The Difference Model does not rely on a cognitively-based process, as put forward by Connolly (2002), to determine that PWDs do not fit into preconceived categories. Neither does it require a purposeful meaning making process, as suggested by Dhamoon (2009), to create an identity based on difference. The Difference Model postulates that it is the initial, affect-based response (Guidano, 1987, 1995) to difference that creates what Davis (2000) describes as “…violent cleavages of consciousness (that) are as primitive as our thought processes can be” (p. 53). The cognitive role itself is understood as a secondary step, which follows behind prior emotional responses (Damasio, 1999, 2010), and serves principally to create and maintain a coherent worldview that creates this particular relationship between identity and difference. The cognitive approaches to attitude formation put forward by Connelly (1991), Dalhoon (2009) and others each serve to provide some degree of clarity to this relationship between identity and difference. However, each of these explanations has begun partway along the creation of an identity/difference continuum, at a point 74 following the recognition of a prior affective relationship necessary for any meaning to be made. The Difference Model postulates that responses to disability are initially generated as affective reactions. . It is only when these initial reactions are reviewed by our more cognitively-based functions do we begin to follow the meaning-making pathways described above. To illustrate, Buller (2005) notes that fear responses to potentially dangerous phenomena in our environment are relatively immediate but “not very sophisticated” (p. 143). It takes a cognitive analysis to determine whether a fleeting image is one of a snake or only a coil of rope. Our first reaction to the possibility of danger is acognitive, immediate and cautious. I have hypothesized that disability, when initially perceived as sufficiently different, evokes this affective response similar to a variety of other ‘hardwired’ fears, as described by Seligman (1971), and which include coiled snakes, spiders (Mineka & Ohman, 2002) and babies’ fear of heights (Buller, 2005). Examples of these psychological reflexes have been demonstrated to exist within the first few days of life (Zajonc, 2004). This type of affect-based response, understood as requiring only the most basic mental processes, is described by Gawronski and Baudenhausen (2006) within the Associative and Propositional Evaluation (APE) model of attitude formation. Differentiating attitude formation into two distinct processes, the associative and the propositional, these authors found that the former is ”best characterized as automatic affective reactions resulting from the particular associations that are activated 75 automatically when one encounters a relevant stimulus” (p. 693). As such, these do not require “much cognitive capacity or an intention to evaluate an object” (p. 694). These responses are held to be largely automatic, spontaneous, and outside of cognitive control. The Difference Model, in a consideration of the relationship between the normals and PWDs, further characterizes the relationship between identity and difference to be one of inadvertence; that is, without intent. Difference is not sought out. It is the remainder. This relationship to difference distinguishes disability from the range of other discredited groups that include gender, race, and class. Even though these latter groups may be considered by some normals to display characteristics, or relational aspects, that display “an undesirable difference” (Goffman, 1963, p. 14) in a manner similar to disability, these other discredited groups do not evoke an initial acognitive response to this deeper perception that is specifically activated by a primal response to difference. Within this context, disability appears to stand alone. Many researchers state that the current models of disability are intended to be descriptive rather than explanatory (Oliver, 1990; Shakespeare, 2014; Thomas, 1999b, 2007). In addition, the construct of disability itself is considered to be sufficiently undertheorized for the development of a theory of disability (Beaudry, 2016; Oliver, 1992b). However, as noted by Oliver (1998), “all health care and research are influenced by theories” (p. 1446). I attempt to add some clarification to that influence by providing an explicit description of the ontological framework within which the Difference Model of disability exists. 76 I propose applying critical realism as an ontological framework within which to position disability. Critical realism attempts to address the apparent inability to agree upon either the ontology or the epistemology of disability (Bhaskar & Danermark, 2006). Critical realism also attempts to move beyond what it considers to be an overly superficial debate between positivism and social constructionism. It attempts to analyze the structures and mechanisms which underlie these phenomena and to create theories that are more than summaries of data (Alvesson & Sköldberg, 2009). A critical realism analysis of a changing social reality is intended to explain how several understandings of disability can co-exist. Within a critical realist analysis, reality is structured in three levels or domains. The empirical (or social) domain includes that which we can observe and experience, the actual domain refers to events that occur independently and without the necessity of being observed, and finally, the real domain is that which includes the deep mechanisms that bring about the particular events and activities of the empirical and the actual (Alvesson & Sköldberg, 2009; Collier, 1994). Reality is complex, containing active human participants as well as social and structural boundaries and influences (Houston, 2010). Bhaskar and Danermark (2006) speak of several levels of social interaction, moving from the intra-individual to the individual to the inter-individual. These writers place the construct of disability into what is referred to as a necessarily laminated system, meaning that it exists within a matrix of physical, psychological, medical, social and cultural influences. They argue that this necessary complexity explains disability, with a recognition that the particular 77 combination and relationship of these components will vary from one individual to the next, providing room for several theoretical perspectives and causing the essence of essentialism to disappear. Finally, critical realism recruits the reasoning tools of abduction and retroduction in support of explaining observed phenomena (Craig & Bigby, 2015). The former, abduction, is considered to be a creative reasoning process within which the connectivity between observed phenomena and possible explanations, which include environmental influences, may be considered (Danermark, Ekstrom, Jakobsen, & Karlsson, 2002). The latter, retroduction, is the analytical process applied following the question “what must be true for this to be the case?” (Craig & Bigby, 2015, p. 315, citing Bhaskar, 1989). With its focus on complexity, variations in perception, and the application of creative processes with which to determine relationships between phenomena and explanations, critical realism offers a theoretical framework that supports the Difference Model of disability. In addition, through the application of the retroductive process, critical realism supports a delineation of the premises which inform that model. The Difference Model faces one additional challenge that goes to the heart of its own legitimacy. Even if it is accepted that none of the other models has been able to generate its predicted benefits to PWDs, especially employment benefits, why would the Difference Model be able to meet this challenge? This solution is associated with the concept of essentialism. As noted above, critics of the medical model have named it as biologically essentialist (Oliver, 1990). The social model is held out by its most well-known proponent as being socially essentialist (Oliver, 2013). The embodiment model 78 faces a critique of both biological and humanistic essentialism (Hacking, 1999). Each of these models, to account for this essentialist critique, moves the fixed point of disability from outside of the space occupied by the normals to one that exists inside that space. To accomplish this, the medical model repairs disability (Parsons, 1951); the social model removes disability (Oliver, 1990); the embodiment model deconstructs disability (Davis, 2006b). Within the Difference Model, the construct of disability, whether considered within a constructionist or biological sense, is not exposed to that critique. The Difference Model construes disability as an attribute created by the normals to label those who are sufficiently different to have come to social notice. However, that attribute is a function of a relational “symbol of difference rather than a synonym for a person” (Dhamoon, 2009, p. 11). It describes the relationship, not the individual. As described by Connelly (1991), “an identity is established in relation to a series of differences that have become socially recognized” (p. 64). There is no essentialist premise attached to this identification. It is acknowledged as arbitrary, intended to reflect that which a particular society, at a particular time, deems to be beyond the pale of acceptability, rather than normalcy. In this sense, disability is a description of attributes that fall outside of what is construed as acceptable, rather than those identified within what is construed of as normal. However, acceptability and normalcy are not necessarily equivalent concepts. Within the Difference Model, to consider PWDs to be subject to a critique of essentialism would require that essentialism is comprised of a continuously changing collection of attitudes toward what is considered acceptable. 79 The importance of this consideration of essentialism and disability rests with how disability is coming to be viewed within our technologically driven society. The influence of new technologies has created an increasingly common view of disability as a mutable attribute rather than a fixed characteristic or state (Breen, 2015). As discussed below, new possibilities allow for new theories, especially when current problem-solving efforts have failed (Kuhn, 1962/2012). It is the potential for the mutability of disability-related attributes which opens the way for the Difference Model and the expansion of the social ground currently occupied by the normals by allowing for a new, less unacceptably different analysis of these characteristics. Text Box Commentary #3 (Excerpted from a paper I wrote in 2007 entitled “Self-narrative: A life in process”) An event that comes to mind is the experience that I went through when I applied to the Canadian Astronaut Program (CAP) shortly after the Challenger exploded in 1987. With a partner who was a doctor, I developed two scientific experiments that were suitable both for micro-gravity and for someone with a disability such as mine. One focused on the psychological effects on astronauts who were to be in space for extended periods and the other had to do with intra-muscular electrical stimulation. The CAP initially showed some interest but gradually cut off our correspondence. Finally, I received a confidential telephone call from one of the staff. He advised me that, although the experiments were intriguing, there was no chance whatsoever that the program would consider drafting a disabled astronaut. After all, he advised, in an emergency situation, it would not be fair to expect the others to have to provide me with support. The fact that an emergency would probably mean that we would all be scattered across the sky was not to be considered. Although I did not realize this until later on, that experience was a significant contributor to my decision to move my career from that of operating a manufacturing company to working on behalf of people with disabilities. A related incident that I recall from the late 1980s occurred during a keynote address that I was delivering to an audience in Regina. The event that I was attending was part of a 80 national celebration of what was then called National Access Awareness Week. I was speaking about how it was necessary for mainstream society to accept people with disabilities even though we were different from others rather than choosing the easier philosophical position of maintaining that we were really all just the same. This was the period when such sugary phrases as “differently-abled” and “we all have disabilities” came into vogue. The notion of being different and yet still entitled to acceptance and respect seemed to be a more difficult concept than does one that overlays a blanket of similarity as a justification for acceptance. During the course of this speech, an image went through my mind that was so powerful that I put away my notes and related it to this audience as a way to explain this conceptual difference. The image that appeared to me was one of me running down a country road. But what I saw was only an image of me from the waist up, moving along behind a row of bushes. I realized that I had never seen myself run in my dreams. Any dream situations that required movement on my part were tailored to prevent requiring an image of me running or walking. Hopping or flying seemed to be okay. I attributed the hopping to the fact that I can stand, with all of my weight only on my right foot. I assumed that the image of flying was fairly common. This all went through my mind in just a split second. I used this image to try and explain to this audience the difference between the imposition of similarity and the acceptance of difference. Critiques, Gaps and Othered Groups As noted above, in an effort to explain the process involved in developing the CADE Scale, I have presented the Difference Model as an alternative to current models of disability and identified that measures currently available to quantify the attitudes of co-workers toward PWDs are typically “without formal evaluation or detailed psychometric analyses” (Antonak, 1988, p. 110). Specifically, there is an absence of a scale, supported by theoretical and empirical validity evidence (AERA et al., 2014), which has been demonstrated to effectively measure co-workers’ acceptance of disabled employees. I have developed the CADE Scale to address this gap. 81 As a final consideration within this gap analysis, I conducted a literature review of a wide array of scales designed to measure attitudes toward a range of othered groups. My intent was to investigate whether a scale similar to what I was developing was already available within a different context. I reviewed a total of 51 scales, which included summaries of 43 scales (Gamst, Liang, & Der-Karabetian, 2011) as well as a review of the complete sets of items in a selection of eight scales designed to measure generic racial attitudes and prejudices (Davis & Engel, 2011). These included at least one from each of the following categories - race/ethnicity/diversity (41), women/gender (6), general prejudice (2), prostitution (1), and poverty (1). These scales demonstrated consistent differences from the CADE Scale, principally in terms of their levels of specificity and a lack of consideration of disability, particularly with regard to workplace accommodations. Overall, the scales that were reviewed addressed attitudes toward generalized characteristics associated with a particular referent or with specific social rights associated with the referents. For example, the Attitude Toward Poverty Scale – Short Form, included an item that stated “There is a lot of fraud among welfare recipients” (Yun & Weaver, 2010). The Attitudes Toward Feminism Scale included “Whether or not they realize it, most women have been exploited by men” (Byrne, Felker, Vacha-Haase, & Rickard, 2011). In that scale, items were principally associated with the rights of women as compared to men. Similar political or social characterizations were found within many of the scales directed toward race/ethnicity and general diversity. However, three scales, the Attitude 82 Toward Diversity Scale, (Davis & Engel, 2011), the Organizational Diversity Inventory (Hegarty & Dalton, 2010), and the Reaction to Diversity Inventory (DeMeuse & Hostager, 2001), focused specifically on employment. In each case, they were intended to measure attitudes toward diversity in the workplace. Scale items addressed topics associated with women, racial minorities and generalized prejudice. Although these three scales were at a similar level of specificity as the CADE Scale, in that they presented items associated with co-workers, supervisors, and hiring/promoting, in no case was there a mention of disability as it may have pertained to any of these othered groups. Consequently, attitudes toward PWDs within any of these groups or disability-related workplace accommodations were not measured. The Attitudes Toward Black Males Scale (ATBM) is of particular note. It was developed by substituting the word “Black” for “disability” as presented in the Attitude Toward Disabled Persons Scale (ATDP) discussed below in Chapter 3. As noted by the author of the ATBM, “Nearly all of the items of the ATDP were retained…. Additional items also were included based on a review of the professional literature” (Bryson, 1998, para. 2). A 47-item self-report was developed with eight subscales, each with a focus on attitudes toward a broad range of social relationships. One of these subscales focused on employment. All mentions of disability were removed. The review of the scales above supports the presumption that the CADE Scale, as a measure of attitudes toward disability, will add to the body of knowledge associated with diversity within the workforce. This literature review reveals no evidence to indicate a focus on attitudes that are either specifically or peripherally 83 related to disability and employment. Neither does it reveal any practical considerations associated with disability, such as workplace accommodations. Differentiating the CADE Scale from this more broadly based set of attitude-focused scales directed toward social ingroups and outgroups, provides evidence for its uniqueness within a sphere of social relationships that is larger than disability. Key Points Disability, as influenced by the social and political environment, is an unstable construct. The medical, social, and embodiment models of disability each describe different cognitively-based aspects of the relationship between PWDs and society. The difference model is differentiated from the others through its presumption of an affect-based influence on others’ attitudes toward PWDs. 84 Chapter 3: Systematic Review Background The integration of PWDs into the Canadian workforce remains of significant concern, with representation rates having remained virtually unchanged since the enactment of the Canadian Employment Equity Act in 1986 (Rioux & Patton, 2014). According to Statistics Canada, (Turcotte, 2014), the unemployment rate for PWDs in 2011 was 11 percent, compared with six percent for the general population. That same report indicates that the employment rate for PWDs, which compares the number of employed individuals to a defined total population, is only 49 percent, compared to 79 percent for the general population. Other developed countries, including the United States, report similar results (Chan et al., 2010; Copeland et al., 2010; Jasper & Waldhart, 2013; Schur et al., 2005). Stone and Colella (1996) have linked these high rates of unemployment within the disability community to the legislative environment, the norms and values within different organizations, and personal characteristics of PWDs and co-workers. Efforts to address each of these three factors have not led to improved rates of employment. Legislative supports have proven to be ineffective (Campbell, 2005) and employers continue to demonstrate organizational values that suggest widespread concerns regarding the hiring of PWDs (Domzal et al., 2008). Employer policies and practices continue to be significant barriers to employment for PWDs (Darcy, Taylor, & Green, 2016). 85 However, it is the attitudes of others within the workplace that are understood as being the most significant determinant of employment success of PWDs (Antonak, 1980; Antonak & Livneh, 2000; Burke et al., 2013; Copeland et al., 2010; Vornholt et al., 2013). Hernandez et al. (2000) state that “employer attitudes are now the main obstacle for people with disabilities in the employment arena” (p. 5). These authors also indicate that employers often respond positively when queried about their general attitudes toward PWDs; however, these same employers typically offer significantly more negative responses when asked about hiring intentions and support for workplace accommodations. Introduction This systematic review reports on self-report measures employed to gauge workplace attitudes toward disabled employees. The review includes 13 scans of five academic databases for variants of disability, attitude, procedure/design/employment, and outcome. Inclusion criteria include age (adult), specific references to disability and competitive employment, and that these surveys provide examples of quantitative analysis. These scans include measures that were either published between January, 2005 and March, 2015 or were published prior to this period but remain in active use. The database scans generated 1192 articles. All but 49 were removed following reviews of titles and abstracts. Full-text reviews eliminated another 40. The strengths and weaknesses of each measure are summarized and categorized according to the PICO model (population, intervention, comparison and outcome), as developed by Gough, Oliver, & Thomas (2012). None of these studies offers a consideration of and 86 positioning within a theoretical framework of disability, revealing a common gap in the validity evidence presented in support of these measures. This systematic review discovered four applicable attitude measures developed between 2005 and 2015 and five additional measures developed before that period but which are still in common usage. Of these nine measures, four were developed specifically to measure employer attitudes toward PWDs. The remaining five, although designed to measure attitudes toward PWDs within a variety of social settings, either indicated a possible goal of measuring employment-related attitudes toward people with disabilities or have subsequently been applied for this purpose. Prior to this review, preliminary title searches of Google Scholar, Medline(OvidSP), and the Cochrane Collaboration resulted in zero results when ‘systematic review’ was added to variants of ‘disability AND employment AND attitude’, providing a preliminary indication that a systematic review of disability related to attitude and employment has not been previously conducted. Although this initial scan was not sufficiently broad to confirm a lack of prior work, the several database searches that comprised the body of this review also failed to discover any previous systematic reviews of co-worker attitudes toward PWDs. By measuring these attitudes, researchers are able to determine the effectiveness of training and other workplace interventions (Brostrand, 2006), determine norms within and between organizations and monitor longer term changes. However, the value of a specific measure is directly related to both its associated theoretical and empirical validity evidence (Messick, 1990). Validity itself is defined as 87 “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (AERA et al., 2014, p. 11). This understanding is supported by Hubley and Zumbo (1996), who state that “without validity, a test, measure, or observations and any inferences made from it are meaningless” (p. 207). Commenting on what were perceived as the pervasively negative attitudes directed toward PWDs at the time, Stiller (1976) noted that “attitude change toward the disabled likely will be most successful when efforts are directed to the affective state of the nondisabled person (i.e., the “gut” level) in a meaningful way” (p. 75). Negative attitudes towards PWDs have more recently been described as “a pervasive, automatic, and yet unconscious development in society” (Pruett & Chan, 2006, p. 204). Based on these determinations that attitudes contribute significantly to the low levels of employment of PWDs (Antonak, 1980; Antonak & Livneh, 2000; Burke et al., 2013), a review of the measures used to evaluate this construct is both timely and necessary if this situation is to be improved. How, then, are attitudes understood? According to Antonak and Livneh (2000), attitudes are latent psychological processes that are acquired through experience and which are displayed or evoked by specific circumstances. Attitudes are also commonly understood as constructs that contain cognitive, affective and behavioural components (Findler et al., 2007). Eagley and Chaiken (1993) describe an attitude as “a psychological tendency that is expressed by evaluating a particular entity with some degree of favour or disfavour” (p. 1). 88 Objectives Systematic reviews are intended to add value to the category of knowledge under consideration. The objectives of this systematic review of co-worker attitudes toward PWDs are to discover available measures, determine the degree of validity evidence demonstrated within each of these measures, and provide an assessment of their value in determining these workplace attitudes. Based on these findings, this review will consider the next steps required to support an improved level of labour force attachment by PWDs. Typically, systematic reviews are used to compare interventions or other processes, either by degree or kind, most often following the commonly used PICO model (Gough, Oliver, & Thomas, 2012). For the purposes of this review, it is the fourth pillar of the PICO model, that of outcome, that is of primary consideration. In this context, outcome quality is measured by the degree of validity evidence demonstrated by each of the included studies. It can be stated as the following research question - “To what degree do current measures of workplace attitudes toward PWDs provide theoretical and empirical validity evidence to support their interpretations of test scores?”. To address this question, each of these measures has been evaluated following a review of the several study designs, their methodological quality, and their value in enhancing the employment opportunities of employees and job seekers with disabilities. These measures have also been reviewed to determine whether a theoretical framework of disability has been offered, or can be inferred, which would 89 add validity evidence and support the selection of items contained within each of these studies (Boland, Cherry, & Dickson, 2014). The purpose of this systematic review is to collect and evaluate the several instruments that have been developed in order to determine their value in measuring attitudes toward PWDs in the workplace. I have limited my research to quantitative analyses and measures, which are able to generate standardized data that may be compared within and between studies. This consideration of the ability of these instruments to effectively measure attitudes associated with PWDs and employment will assist in the determination of whether current measures suitably measure this construct or whether a knowledge gap exists; if the latter, this review and analysis will be of value in informing future developments. Methods To gather the measures discussed within this review, I conducted a computerized search of five psychological, medical, and sociological databases. Databases searched were Medline, PsycInfo, Google Scholar, Cinahl, and Cochrane Collaboration. The phrases work disability, Return To Work (RTW), and self-reports of disabled persons were excluded as being beyond the scope of this review. Preliminary searches of the first two phrases had resulted in material focused on disabilities acquired during the course of employment, with an emphasis on disability management procedures. In the latter case, searches had generated a range of qualitative analyses. Although qualitative research allows for in-depth analysis of meanings associated with various 90 findings (Creswell, 2013; Mason, 2002), the construct of employer/co-worker attitude as a principal component of the challenges faced by disabled employees and job seekers is already an accepted premise of this review (Stone & Colella, 1996). Therefore, these qualitative analyses were excluded. One additional decision informed the methodology of this review. A preliminary review of the sourced materials indicated a clear delineation in the literature between studies that are intended to consider and evaluate a specific event, study or group response and those that are intended to be developed and applied comparatively across a range of groups and circumstances. I refer to these two categorizations, respectively, as Single Evaluations and Transferable Measures. Several authors, although using a range of descriptors, distinguish between these two categorizations. The former, Single Evaluations, are intended to gather sufficient data from individual studies to allow for “conceptual saturation” (Gough, Oliver, & Thomas, 2012, p. 109). This process identifies a sufficient number of studies to allow for a broad and representative configuration across the range of relevant research, thereby allowing for the potential development of new or more coherent theory. However, as noted above, the construct of attitude is already commonly accepted as a significant factor in the success of PWDs in the workplace. The second study format, Transferable Measures, is an exhaustive aggregation of all relevant studies and is more appropriate for an examination of a construct intended to have applicability across different contexts, times, and groups. Aggregative studies are associated with a consideration of theories with similar conceptual 91 frameworks in an effort to gain greater confidence in their results (Boland, Cherry, & Dickson, 2014; Gough, Oliver, & Thomas, 2012). This description corresponds to the nine instruments described below as Transferable Measures and it is these studies that will be considered within this systematic review. Intended principally as a descriptive systematic review, this study will also consider the range and trends exhibited by these measures (Gough, Oliver, & Thomas, 2012). Systematic reviews within the social sciences typically make use of quality evaluation tools that provide one of the following types of results: generation of scales that provide a numerical value of quality ’for each study, an overall checklist associated with several indicators, or categorical ratings that allow for comparative results (Boland, Cherry, & Dickson, 2014; Gough, Oliver, & Thomas, 2012). I have selected the latter approach, within a three dimensional framework, to arrive at a comparative quality assessment for each of the included scales. These dimensions, proposed within a variation of the Weight of Evidence Model (Gough, 2007), include the quality of execution of the study according to generic standards, the appropriateness of the study itself for answering the research question of the systematic review, and the applicability of the study (including its population, sample size and context) to the review question (Gough, Oliver, & Thomas, 2012). Each of these dimensions, summarized as format, content, and focus, is discussed below. The several individual search parameters within each of these databases, including keyword variations, dates, and steps, resulted in 13 individual searches. Keywords were generated from within four categories following a PICO format (Boland, 92 Cherry, & Dickson, 2014) and included variants of disability, attitude, procedure/design/employment, and outcome. See Table 3.1 for a summary of these 13 searches. Procedural details for each of these searches are shown in Appendix C. 93 Table 3.1: Thirteen Database Searches # Date Database Key Words A/O Title A/O Abstract A/O Add’l Criteria Total 1 Apr. 9 Medline (OvidSP) disabled person (expanded) + employ* and integrat* or accept* or attitude* and measure* or questionnaire* or empirical or quantitative 234 2 Apr. 9 Medline (OvidSP) disab* + (employ* or work*) + (attitude* or accept* or integrat*) and 2005-2015, English 11 3 Apr. 9 Medline (OvidSP) (measure* or questionnaire* or tool* or evaluat*) + disab* + (empl* or work) + (attitude* or accept* or integrat*) or measure* or questionnaire* or tool* or evaluat*) + disab* + (empl* or work) + (attitude* or accept* or integrat*) and 2005-2015, English 39 4 Apr. 9 Medline (OvidSP) disability + attitude + employment + factors or disability + attitude + employment + factors and 2005-2015, English 105 5 Apr. 9 Medline (OvidSP) disab* + (employ* or work) + (attitude* or accept* or integrat*) or disab* + (employ* or work) + (attitude* or accept* or integrat*) and 2005-2015, English 11 6 Apr.13 Medline (OvidSP) valid* + disab* + (employ* or work or organiz*) and 1998-2015, English 78 7 Apr. 9 PsycInfo disab* + (attitude* or accept* or integrat*) + (employ* or work) and Jan. 2005-Mar.2015, “apply related words”, English, 18+ years 18 8 Apr. 9 PsycInfo disability + attitude + employment and 2005-2015, peer reviewed journals 2 9 Apr.11 PsycInfo disab* + (employ* or work) + (scale* or factor* or questionnaire* or tool* or measure*) + (accept* or integrat* or attitude*) or disab* + (employ* or work) + (scale* or factor* or questionnaire* or tool* or measure*) + (accept* or integrat* or attitude*) 537 10 Apr.23 Cochrane Collabor-ation disab* or disab* or disab* 25 94 # Date Database Key Words A/O Title A/O Abstract A/O Add’l Criteria Total 11 disability + employment + (attitude or acceptance or integration) and 2005-2015 9 12 Apr. 9 Google Scholar disability + attitude + (acceptance or integration or employment) 37 13 Apr.11 Cinahl Science disab* + (employ* or work) + (measure* or questionnaire* or factor* or scale or rat*) and 2005-2015, English, peer-reviewed, abstract available, apply related words 73 TOTAL 1179 Abbreviations: KW = key words; TI = title; AB = abstract A/O = and/or Asterisk (*) = wildcard ending 95 Titles and abstracts of peer reviewed articles were used as the first two screening tools. All remaining articles were then reviewed in their entirety. All of these screenings included a confirmatory scan for the required inclusion criteria, which included – age (adult), date of publication (2005-2015, or earlier if still in active use), specific references to disability and competitive employment, and a quantitative data analysis. Results Nine Transferable Measures were extracted through 13 computerized searches of psychological, medical and sociological databases. Databases searched were Medline, PsycInfo, Google Scholar, Cinahl, and Cochrane Collaboration. These searches resulted in a total of 1179 articles. Of these, 1078 were discarded based on titles, 12 were discarded as duplicates, and another 65 discarded following reviews of abstracts. This resulted in 24 database articles. Full-text reviews of these 24 articles resulted in the addition of another 25 full-text reviews. Therefore, 49 studies were reviewed in their entirety. Table 3.2: Summary of Database Searches SUMMARY OF DATABASE & ADDITIONAL REFERENCE SEARCHES (Title & Abstract Reviews) Search Database Total Removed Reason Bal Sub Removed Reason Bal Total 1. Medline 234 202 Title 32 48 37 Abstract 11 11 2. Medline 11 4 Title 7 Abstract 3. Medline 39 31 Title 8 Abstract 4. Medline 105 104 Title 1 Abstract 5. Medline 11 11 Title 0 Abstract 6. Medline 78 78 Title 0 Abstract 7. PsycInfo 18 10 Title 8 24 13+1*=14 Abstract 10 10 8. PsycInfo 2 1 Title 1 Abstract 9. PsycInfo 537 522 Title 15 Abstract 96 Search Database Total Removed Reason Bal Sub Removed Reason Bal Total 10. Cochrane 25 25 Title 0 0 0 Abstract 0 0 11. Google Scholar 9 7 Title 2 3 2 Abstract 1 1 12. Google Scholar 37 36 Title 1 Abstract 13. Cinahl 73 59 Title 14 14 12 Abstract 2 2 Subtotal Add’l References 1179 25 1078 0 12 (duplicates) 89 25 89 25 65 0 24 25 24 25 TOTAL 1204 1078 114 114 65 49 49 *could not acquire Full text reviews of the remaining 49 studies resulted in the removal of an additional 40 scales, resulting in nine studies that addressed attitudes toward PWDs. These 40 studies were removed for several reasons, including – studies were literature reviews (8), not related to disability (2), not quantitative studies (6), not within time period (11), single evaluations (6), not employment (4), not attitudes (3). Details of this review process are shown below in Figure 3.1. 97 Figure 3.1: Records Flow Diagram Records identified by database searches (n = 1167) (n = 1180) Full documents reviewed (n = 49) Full-text articles screened out (n = 40) Included in data extraction and review (n = 9) Additional records identified through other sources* (n = 25) Removed 1131 (title or abstract) Lit review (8) Not disability (2) Not quantitative (6) Not within dates (11) Single evaluation (6) Not employment (4) Not attitude (3) n = 1192 Removed 12 duplicates n = 49*Additional records identified through review of titles of references in articles retrieved from databases. 98 Description of measures. Descriptions of each of the nine measures are provided below. In each of the studies within which this information has been made available, these descriptions include a summary of the stated premises, purpose, methodology, and results. Additional data are compiled within the general headings of format, content, focus, and strengths and weaknesses. These latter categories have been summarized and calculated to provide relative scores within the Weight of Evidence Model (See Table 3.3 following scale summaries) based on the scoring guidelines shown below: Assessment Criteria: Format (High - 3) – Clear and detailed description of methods, results and data collection process. Interpretation of data and results based on satisfactory validity evidence. Format (Medium - 2) – Satisfactory description of methods, results and data collection process. Interpretation of data and results based on moderate validity evidence. Format (Low -1) - Unsatisfactory description of methods, results and data collection process. Interpretation of data and results based on poor validity evidence. Content (High -3) – Clear and detailed description of survey construction and methodology. Content (Medium -2) – Satisfactory description of survey construction and methodology. 99 Content (Low -1) – Unsatisfactory description of survey construction and methodology. Focus (High -3) – Diverse participant groups with good generalizability to employment. Focus (Medium -2) – Satisfactory participant groups with moderate generalizability to employment. Focus (Low -1) - Unsatisfactory participant groups with poor generalizability to employment. 1. Yuker et al. (1970). The Attitudes Toward Disabled Persons Scale (ATDP) The authors hypothesized that the attitudes of employees with disabilities toward their own disabilities would be more reliable indicators of work performance than would the type, severity, or age of onset of disability. Therefore, the ATDP was originally intended to measure the attitudes of disabled employees about themselves. However, during the course of its initial development phase, throughout the 1960s, it was determined that it could be applied with equal validity to others regarding these employees. The fundamental assumption of the measure was that disabled people, whether viewed by themselves or others, were distinguished by residing along a single continuum of perceived difference as compared with the non-disabled population. The original scale was administered to approximately 15,000 subjects by the authors and other researchers during the 1960s, prior to its publication in 1970. During the development of the ATDP, PWDs included only those with physical impairments as 100 represented by individuals employed at the workshops managed by the Viscardi Center in Albertson, New York. The ATDP is considered to be the most widely used scale in measuring attitudes toward disabled persons (Pruett & Chan, 2006; McCaughey & Strohmer, 2005) and is still currently in use (Meyers & Lester, 2016). Format – The original scale, ATDP - Form O, is a 20-item, self-administered questionnaire with a Likert-type response format (-3 to +3 with no neutral score). Two additional scales, Forms A and B, were also developed. Each of these latter scales contains 30 items and is of a similar format to the original scale. These were developed in an effort to determine whether longer scales could result in increased reliability and validity. However, these latter two scales were not found to yield sufficiently different results to warrant the additional length. Content – The questionnaire is comprised of two general categories of statements - how PWDs should be treated and about their reality as PWDs. Focus – Although no definition of disability is provided, the ATDP is stated as being developed to measure attitudes toward individuals with physical disabilities. Strengths and Weaknesses – The ATDP was reviewed for reliability, faking, acquiescence, social desirability, criteria validity, construct validity, and content validity. Factor analysis indicated a unidimensional construct associated with difference, although the authors acknowledged that a construct as complex as attitude may contain a set of underlying factors. Although apparently technically sound, the items within the ATDP do not capture the full range of disability. As such, it addresses only a subset of PWDs. In addition, a review of the ATDP items reveals an implicit pejoration 101 within the descriptors. Although it could be argued that the negative connotations associated with these descriptors have developed over time, there is a general sense of a flawed existence embedded within many of the questions. Although respondents are asked to rate their degree of agreement or disagreement with each item, almost all of these statements portray PWDs as being diminished. For example, Item #14 states “You should not expect too much from disabled people.” (Yuker et al., 1970, p. 113). These images create the possibility that respondents will be influenced by the language rather than by their own perceptions of PWDs (Schwartz, 2008). Theoretical framework – Yuker et al. (1970) state that attitudinal measures should be based on evidence, not theory. These authors explicitly reject a theoretical positioning of disability as contributing to validity evidence, and are in favour of a strictly empirical analysis. 2. Antonak (1982). Scale of Attitudes Toward Disabled Persons (SADP) The author states that the SADP has been developed to provide a contemporary, easily administered and scored, psychometrically sound instrument to measure attitudes toward PWDs. Factor analysis resulted in a three-factor solution, labelled Optimism-Human Rights, Behavioral [sic] Misconceptions, and Pessimism-Hopelessness. A criterion validity coefficient of .54 was found when tested against a reduced version of the ATDP-Form O. Several items in the scale address attitudes toward individuals with intellectual or psychiatric disabilities. This differs from the ATDP, which specifically considers physical disabilities. The SADP study explicitly mentions its potential value as 102 a tool to measure changes in attitudes following interventions. However, no evidence of this use has been found (Palad et al., 2016). Format – The SADP is a 24 item self-report with a Likert-type response format, with three subscales. (n = 225; 28 males and 197 females, of which 146 were undergraduate students, 46 were graduate students, and 33 were in continuing education programs.) Content – Similar to the ATDP, the SADP embeds a variety of negative assumptions regarding PWDs into its statements. For example, Item 12a states that “Disabled individuals are able to adjust to life outside an institutional setting.” The assumed relationship of disability to institutionalization, even if that relationship can be ‘overcome’, illustrates an unrecognized bias. Although the SADP is not described as being specific to employment, six items (#s 11, 16, 18, 20, 21 & 23) measure responses to various aspects of employment as these apply to PWDs. Focus – The SADP has been developed as a response to the earlier ATDP, which is considered by the author to be no longer a reliable or valid instrument in measuring attitudes toward PWDs. The ATDP is described as out-of-date in terms of its language (Lam et al., 2010) and too simplistic in its conceptualization of attitudes toward PWDs as being unidimensional. Strengths and Weaknesses - The SADP is explicitly stated to be a tool to measure changes in attitudes following interventions such as training programs. However, according to Palad et al. (2016), no evidence of this use has been discovered. In addition, although critical of the validity claims of the earlier ATDP, the SADP appears to 103 be measuring a different construct from that considered by the ATDP. The questions within the latter scale are much more indicative of attitudes directed toward individuals with intellectual or psychiatric disabilities than were the questions in the ATDP. Although the scale demonstrates a variety of tests for reliability and validity, it contains an underlying conceptual bias. Theoretical framework – Although there are no direct references to disability theory or models of disability, the author notes a positive correlation between attitudes toward PWDs and perceived difference. However, there appears to be an admixture of ontological positioning of PWDs. For example, the author states that items within the second subscale, that of Behavioral [sic] Misconceptions, refer to misconceptions about PWDs that include that “they enjoy repetitive, menial work (and) are often arrested as criminals, and should be incarcerated in institutions, rather than rehabilitated to live and work in general society” (p. 24). This statement appears to indicate that, although PWDs may not necessarily be dull or evil, they may require some form of support or repair to function in society. 3. Gething et al. (1992). The Interaction with Disabled Persons Scale (IDP) The IDP was developed to address perceived shortcomings within the ATDP. These shortcomings were stated as the use of outdated language and an overgeneralization of results. The IDP was intended to consider more latent aspects of attitude associated with strangeness, unfamiliarity, and what was described as perceived discomfort in social situations. Factor analysis revealed six subscales, described as Discomfort in 104 Social Situations, Coping/Succumbing Framework, Perceived Level of Information, Vulnerability, Coping, and Vulnerability. However, Factors 4 and 6, as shown in Table 6 of the study, have the same name. Each is comprised of different items. However, this anomaly is not addressed in the study. Format – The IDP is a 20-item scale with a Likert-type response format. Two groups (n=87, n=53) were administered the IDP. Participants included students, and employees from the private and public sectors. Content – Several of the items force responses based on subgroups of disability. For example, Item #11 states that “I can’t help staring at them”. This statement implies that disability is visible, thereby reinforcing the concept that disability equates to perceived physical difference. The visibility of disability appears to be an unstated assumption underlying the study. Focus – The focus of the scale is to gauge the relationships generated between attitudes and discomfort on the part of the observer. Strengths and Weaknesses – Validity evidence was generated through initial reliability evaluation, an exploratory factor analysis, and evaluation of the consistency of the scale with its nomological network. The study’s first hypothesis, that the level of prior contact with PWDs would have a significant, positive effect on IDP scores was supported. However, it was very difficult to understand the relationships between the different sample groups that were used for various analyses. Within the six factor solution of the data analysis, three items loaded twice at a cut-off score of 0.3, raising some concern over the validity of the analysis. This concern has been raised by several researchers 105 (Iacono, Tracy, Keating, & Brown, 2009; Loo, 2001a). In addition, the two participant groups (n=87, n=53) used for the factor analyses are, based on the length of the scale, considered to be too small to offer a meaningful degree of validity evidence (DeVellis, 2017). Others (Forlin & Fogarty, 1999) have supported the six factor analysis. The study also determined that more positive attitudes are held by women than men and by younger than older participants. However, Loo (2001b), in a recent Canadian validity study of the IDP, demonstrated no significant differences in scores between men and women and among age groups, questioning the reliability and validity of the study. The authors also note that their second study includes an analysis of test-retest reliability of the IDP which was administered immediately prior to and following respondents’ participation in a disability-awareness training session. These results were held to indicate that the IDP could be used successfully to measure attitude change. However, Furr and Bacharach (2014) state that the administration of test/retest procedures within periods of less than two to eight weeks may be influenced by “carry over or contamination effects” (p. 130). The authors indicate that this instrument was capable of measuring attitudes across categories of disability although no evidence was provided. In addition, other than the test/re-test analysis described above, no evidence of additional test/re-test use has been found (Palad et al., 2016). Theoretical framework – This study puts forward an operational definition of attitudes toward PWDs “in terms of level of discomfort reported by a person during interaction with people with disabilities” (p. 76). Those levels of discomfort are considered to be 106 generated by community views that PWDs are different and separate, and that these characteristics imply inferiority, stigma and deficiency. However, that explanation of difference as a principal mediator of attitude does not correspond to the multidimensional analysis of the IDP Scale. 4. Greenwald et al. (1998). Measuring Individual Differences in Implicit Cognition: The Implicit Association Test (IAT) The IAT was originally developed to test the relationship between implicit and explicit attitudes as they applied to race. However, a minimum of 18 studies have since applied the IAT to the measurement of attitudes toward PWDs, with similar results. In a review of these studies, Wilson & Scior (2014) note that two of these latter studies include measures of implicit attitudes toward PWDs in employment situations and that the respondents in one study included co-workers of PWDs. However, neither the specific study nor the number of these respondents was provided. Typical results indicate moderate to strong implicit negative attitudes toward PWDs, with these attitudes being more negative than those stated explicitly. In addition, men and older people are more implicitly negative than are women and younger people. The IAT is a computerized test that measures differences in the response speeds of participants when they are asked to pair positive and negative comparators with specific descriptors associated with, in this instance, PWDs. The authors state that the IAT can be delivered effectively with relatively small number of respondents, with the original three groups that made up the above study (n1 = 32, n2 = 32, n3 = 26) each 107 using 150 stimulus words. The IAT is intended to account for response bias from explicit responses to socially sensitive topics. Format – The process involved in determining the scores of the IAT is difficult to interpret or reproduce from the material provided. For example, in one study, the authors indicate that data from eight of the original 41 participants were removed due to higher than anticipated error rates. Although these error rates are attributed to responses that were overly rapid, no information is provided on how these determinations were made. In addition, many of the comparators used as examples of pleasant-meaning or unpleasant-meaning images that were associated with target concepts appear to be culturally and value-laden (for example, flowers versus insects), leading to the possibility of culturally skewed results. Content – The three groups that provided data for the original results were quite small and several had their responses removed from the calculations due to what were perceived as high error rates in the completion of their scales. The potential for confusion within the process appears to be high. Focus – Further studies following the original appear to have allowed for more satisfactory generalization and evidence of validity and reliability. However, little clear evidence has been provided. Strengths and Weaknesses – The principal strength of the IAT is its ability to separate the implicit from the explicit attitudinal positions of participants. There is significant convergent validity evidence for this separation from a range of studies (Wilson & Scior, 2014). 108 Theoretical framework – The several applications of the IAT intended to evaluate implicit attitudes toward PWDs have either not reported disability categories, or have selected individual or generic categories of disability without theoretical consideration. 5. Loo (2004). Attitudes Toward Employing People with Disabilities: A Test of the Sympathy-Discomfort Categories (ATEPD) The ATEPD was administered to a convenience sample of Canadian management, human resource, and organizational behaviour undergraduates. The author notes that there has been very limited testing of the attitudes of those who are or will be in a position to directly influence the employment of PWDs, especially in Canada. Overall, women were found to have only slightly more positive attitudes toward PWDs than did men. Format – The ATEPD is a 13-item questionnaire made up of three analytically derived subscales. These include organizational policies, organizational climate, and working with PWDs. A Likert-type response format is used. In addition, there are three open-ended questions at the end of the survey. Details of these latter results were not discussed within the study. Content – The convenience sample was made up of 231 participants. Of these, 129 were men and 102 were women. Age range was from 19 to 51 years (M = 22.89, SD = 4.43). Focus – The focus on employment-related attitudes is a useful addition to the range of more generalized surveys. This respondent group brings more knowledge of the reality of employment-related situations than most of the participant groups within this review. 109 Strengths and Weaknesses – Testing a Canadian group of management students provides useful comparative data when determining attitudes toward PWDs. Attitudes of this group could be compared to others, particularly managers of PWDs, to measure possible differences based on experience. However, the study itself appears to have several weaknesses. At least two items (#2, #13) contain arithmetic errors in the descriptive statistics section. Both show Total Sample Means as being outside of the range of Male and Female Means. Finally, there is no reproduction of the three open-ended questions referred to as being attached to the end of the 13-item questionnaire. Theoretical framework – The author reports that “research over the past several decades has shown that persons with disabilities are seen as different” (p. 2201). These perceptions, often based on assumptions of isolation and emotional challenges, result in the acceptance of stereotypical attributes and characteristics. “Such differences indicate that persons with disabilities are viewed as lesser persons” (p. 2201). 6. Gilbride et al. (2006). Development and Validation of the Employer Openness Survey (EOS) The EOS was developed to provide information to rehabilitation professionals about employers’ interests in hiring PWDs. The authors had previously developed a set of employer characteristics related to employer openness. These included employer culture, work design issues, and employer experience and support. This latter characteristic was considered to be determined by the relationship between the employer and the vocational rehabilitation professional. 110 Format – The EOS is an 18-item, open-ended questionnaire designed to measure the level of openness demonstrated by employers in hiring, accommodating and promoting PWDs. Scoring is accomplished by raters assigning a rating of between 1 and 5, based on how closely the respondent’s answers correspond to predetermined response descriptions. The original questions were reviewed by employers, evaluated for construct validity, field tested again with employers who had hired PWDs at the recommendation of rehabilitation professionals, and then reviewed again by rehabilitation professionals. Overall, the measure appears to be well validated within its context. Reliability tests were also clearly described. Overall, eighty-three employers were selected to participate in the development process through recommendations of various rehabilitation counsellors. Content – Although a description of a final developmental phase was provided, conducted with 67 participants (28 employers and 39 rehabilitation professionals), there was no description of the instrument being used beyond this stage. Focus – Although intended specifically for employers, the group selected for an evaluation of construct validity was chosen from rehabilitation counsellors in attendance at either of a National Rehabilitation Association conference or an Association of Vocational Rehabilitation in Alcohol and Substance Abuse conference. Strengths and Weaknesses – The specific validity and reliability measures, as described, appear to demonstrate adequate results. However, this measure demonstrates significant flaws. First, all of the employers involved in the developmental and validation processes had previously demonstrated positive responses to job seekers with 111 disabilities. Second, although no definition of disability is provided within the measure, the survey itself poses questions that imply a target population that would be more valued for dependability than ability. Theoretical framework – Although not stated explicitly, the EOS is framed within a concept of disability that rates dependability over skills and learning speed. For example, scoring for Item Eight is less positive if an employer were to mention “learning speed” as part of a response to “If you had to choose, would you select someone who you knew to be a quick learner or someone who you knew to be very dependable?” (p. 88). In addition, the survey contains an underlying premise that PWDs will be supported in job acquisition and training by a rehabilitation professional. 7. Pruett & Chan (2006). The Development and Psychometric Validation of the Disability Attitude Implicit Association Test (DA-IAT) The DA-IAT was developed to provide a measure of implicit attitudes of rehabilitation professionals toward PWDs. The authors note that there is evidence that rehabilitation professionals hold many of the same negative attitudes toward PWDs as do members of the larger community. However, no supporting evidence is provided. In addition, the authors state that there has been no research in rehabilitation medicine or psychology specifically designed to measure these attitudes within this group. Development of the DA-IAT was based on the theory and methodology described within the Implicit Association Test (See #5 above). Specifically, the intent was to eliminate the potential for social desirability bias in self-reports when addressing socially sensitive topics. Although not specifically considering attitudes toward PWDs and employment, 112 this measure of attitudes of rehabilitation professionals who will be in a position to directly influence the placement and employment of PWDs could be of significance in determining appropriate training for those specialists. Test results indicated that scores of the DA-IAT had no relationship with scores generated by the Attitudes Toward Disabled Persons scale (See # 1 above). Also, analysis of demographic data, including age and gender, did not predict DA-IAT scores. The predominant predictor was found to be prior contact with PWDs. Format – Although the DA-IAT was found to be uncorrelated with the ATDP scale, that latter scale had been modified for purposes of this application. The potential for error, based on this adjustment, was not addressed. In addition, the test itself was not described in the study, other than it had employed a series of disability-related symbols instead of disability-related words. This use of symbols was explained as a means to overcome the inherent complexity of the construct of disability. For example, images of a wheelchair, a guide dog, and a pair of crutches were used, in tandem with what were considered to be congruent or non-congruent words, to represent various aspects of disability. Content – The DA-IAT results were based on a survey of 223 rehabilitation counselling students from ten American universities. Although the authors acknowledged that the original computer-based IAT demonstrated superior accuracy to subsequent paper-based questionnaires, the DA-IAT measure used the latter format, an iteration of which was developed specifically for this test. 113 Focus – This study did not provide a definition of disability. With a cohort of rehabilitation counselling students as the subjects, an implicit bias toward an understanding of PWDs as being in need of those professional services appears possible. Strengths and Weaknesses – As noted above, this study demonstrates several significant flaws that make its results susceptible to a variety of challenges. As such, it appears to have very poor generalizability to employment. Theoretical framework – No explicit consideration of a theoretical framework is offered in this study. However, the assumption that rehabilitation professionals are required to support PWDs, whether in employment or other aspects of daily living, indicates an assumption of dependence and the need for ongoing professional assistance. 8. Ozawa & Yaeda (2007). Employer Attitudes Toward Employing Persons with Psychiatric Disabilities in Japan (ATEP) The ATEP was developed specifically to measure employer attitudes toward individuals with psychiatric disabilities. According to these authors, people with psychiatric disabilities in Japan have a much lower workforce attachment (0.5%) than do people with physical disabilities (11.4%) and those with intellectual disabilities (24.8%). Survey responses indicated that attitudes toward individuals with psychiatric disorders varied by employment sector, with the service sector being less subject to concerns over what was described as activity limitation than was the manufacturing sector. The age of employers also correlated with perceptions of activity limitation, with older employers showing more negative attitudes. Employer motivation to hire people 114 with psychiatric disabilities also correlated positively with prior experience employing PWDs. Format – The draft instrument contained 67 items, based on a comprehensive literature review. A pilot survey was then conducted and factor analyzed to produce four subscales – employer motivation, activity limitation, prejudice and fear, and attention distribution. The items were then reviewed and reduced by a panel of content experts. The final survey contained 24 items and was designed with a Likert-type response format. Note: Although this study was written in English, it did not list the 24 survey items. However, the corresponding author, Akhiko Ozawa, provided me with an additional source where 12 of these items had been reproduced in English. (Personal email correspondence, November 30, 2017). Content – This survey was completed by 358 employers, including corporate executives and human resources personnel, all of whom represented Japanese businesses with over 100 employees. A stratified sampling process provided evidence for internal validity. Focus – The authors state that the ATEP scale was developed in response to a lack of available measures demonstrating assessments of reliability and validity within this category of disability as it pertains to employment. The authors also noted that the response rate to the original 1600 mailed surveys was only 22.4 percent and that this could both limit the generalizability of the study and generate limited validity evidence. Strengths and Weaknesses – The subscales within the measure are not clearly defined and allow for various interpretations. In addition, the two factors selected as 115 explanatory variables within a multiple regression analysis of the Employer Motivation subscale explained only 16.5 percent of the variation related to that subscale. Although a lack of knowledge about and contact with individuals with psychiatric disabilities were both cited as being related to an unwillingness to hire, the conclusion that this unwillingness may be resolved by the provision of additional knowledge and experience to employers is not well supported in this study. Theoretical framework – The authors note that “(e)mployer motivation for employing people with psychiatric disability correlates with fear, prejudice and prior experiences” (p. 105). However, these authors also note that individuals with psychiatric disabilities may also be poorly represented in the workforce due to “limited competence…limited ability to form and maintain personal relations…limited ability to learn…and social status problems” (p. 105). Finally, the authors stated that the low response rate to the survey may have been, in part, influenced by “well-publicized incidents of violence by people with psychiatric disability (which) occurred during the study” (p. 112). These several comments indicate that attitudes toward PWDs, at least those with psychiatric disabilities, are influenced significantly by an assumption of difference from the norm. 9. Findler et al. (2007) The Multidimensional Attitudes Scale Toward Persons with Disabilities (MAS): Construction and Validation The MAS was developed following a review of eight scales designed to measure attitudes toward various categories of PWDs. The authors summarized these scales as falling within one of three dimensions. These included unidimensional/multidimensional, direct/indirect, and general/specific. This latter dichotomy was applied to both 116 disabilities and context, with a specific mention of employment of PWDs. Based on a critical review of each of these scales, the authors determined that the psychometric soundness and validity of each of these instruments was of concern. The MAS scale, comprised of three dimensions, including affect, cognition, and behaviour subscales, is intended to address these concerns. Following Antonak and Livneh (2000), the MAS is presented in this study as an indirect measure, beginning with written presentations of disability-related vignettes that varied across disability and gender. These presentations are followed by a 47-item questionnaire that rates the agreement of the participants with the stated feelings, thoughts or behaviours of the vignette characters. Responses are gathered through a 5-point Likert-type response format (n=132). Validity evidence was gathered through a correlation of the MAS with the Rosenberg Self-Esteem Scale and with the ATDP scale. However, no data were provided that would have allowed a consideration of whether these two scales demonstrated adequate convergent or discriminant validity evidence. Results indicated that, when scores on the three subscales were compared, cognitive and affective scores revealed more negative attitudes than did behaviours. This was interpreted as a modifying effect being applied by social norms and expectations on behaviours in spite of more negative, individually-held beliefs and attitudes. Format – The exploratory factor analysis used a relatively high cut off score of 0.4. Without any published data, it is difficult to know the number of items that may have loaded between 0.3 and 0.4 but which were excluded from the analysis. The factor 117 analysis data were not provided in the study. However, the size of the sample (n=132) is not considered sufficient to generate a meaningful factor analysis with a 47-item measure (DeVellis, 2017). In addition, what was stated to be a satisfactory Cronbach’s alpha of .89 was only achieved with the ATDP scale following the removal of three items from that 20-item scale. The claimed evidentiary value demonstrated by this alpha is therefore reduced. Content – The 47-item MAS questionnaire was completed by 132 participants, most of whom were stated to be college students. The questionnaire was divided into three subscales to reflect a three-dimensional conception of attitude. These subscales included affects (16 items), cognitions (20 items), and behaviours (11 items). Participant demographic information, including gender, age, and education levels, was gathered. Focus – This scale was developed to address concerns over the psychometric soundness and validity evidence provided by previous scales, particularly those that had been developed and administered as direct measures of attitude. Strengths and Weaknesses – A particular strength of the study was its development of a variable vignette that allowed for a focus by participants on what they determined a character in the story would think, feel or do. However, as noted above, the psychometric quality of the study is of concern, in part due to a lack of reported data. In addition, the authors acknowledged that the snowball recruiting process and the use of college students as subjects could affect the generalizability of the study. Finally, the size of the group was too small to provide a satisfactory factor analysis of a 47-item 118 scale. A ratio of between 5 to 10 subjects per item is typically recommended, although this ratio may be somewhat relaxed with larger populations of over 200 if factor analyzing fewer than 40 items (DeVellis, 2017). Neither of these guidelines was followed in this example. Theoretical framework – There was no explicit consideration of a theoretical framework within this study. However, the authors state that positive attitudes can facilitate inclusion and acceptance of PWDs by “family, friends, and potential employers” (p. 166). Conversely, negative attitudes can lead to the creation of social barriers, including vocational challenges. Discussion The modified Weight of Evidence Model, as illustrated below in Table 3.3, shows that the nine measures scored between 7 and 16 on an 18-point scale when considered for their format, content, and focus, as well as for stated considerations of reliability and validity evidence. Within this model, points were allocated for “test/retest” only if a measure stated that it could be used to gauge the effect of an intervention through its application before and after that intervention. Only two measures – Antonak (1982), and Gething and Wheeler (1992) - made such a statement. Table 3.3: Weight of Evidence Chart Weight of Evidence Chart Study Format Content Focus Psychometric Information TOTAL Validity Reliability Test/Retest 1. Yuker, Block & Younng (1970) 3 3 3 3 3 1 16 2. Antonak (1982) 3 3 2 3 3 2 16 3. Gething & Wheeler (1992) 3 2 2 2 2 2 13 4. Greenwald, McGhee & Schwartz (1998) 2 2 3 3 3 1 14 5. Loo (2004) 1 2 3 3 3 1 13 119 Study Format Content Focus Psychometric Information TOTAL Validity Reliability Test/Retest 6. Gilbride, Vandergoot, Golden & Stensrud (2006) 1 1 1 3 3 1 10 7. Pruett & Chan (2006) 2 1 2 1 1 1 7 8. Ozawa & Yaeda (2007) 2 1 2 1 2 2 11 9. Findler, Vilchinsky & Werner (2007) 2 2 1 2 2 1 10 Assessment Criteria: Format (High - 3) – Clear and detailed description of methods, results and data collection process. Interpretation of data and results based on satisfactory validity evidence. Format (Medium - 2) – Satisfactory description of methods, results and data collection process. Interpretation of data and results based on moderate validity evidence. Format (Low -1) - Unsatisfactory description of methods, results and data collection process. Interpretation of data and results based on poor validity evidence. Content (High -3) – Clear and detailed description of survey construction and methodology. Content (Medium -2) – Satisfactory description of survey construction and methodology. Content (Low -1) – Unsatisfactory description of survey construction and methodology. Focus (High -3) – Diverse participant groups with good generalizability to employment. Focus (Medium -2) – Satisfactory participant groups with moderate generalizability to employment. Focus (Low -1) - Unsatisfactory participant groups with poor generalizability to employment. Adapted from the Weight of Evidence Framework (Gough, Oliver, & Thomas, 2012, p. 162) The two scales with the highest scores, the ATDP (Yuker et al., 1970) and the SADP (Antonak, 1982), provide significant degrees of reliability and empirical validity evidence. However, each is subject to critiques based on outdated language and the potentially misleading and biasing social images within which PWDs are portrayed. One of these, the ATDP, also explicitly rejected the need to consider theoretical validity evidence for psychometric support. The SADP, similar to each of the others, failed to provide theoretical evidence of validity. In a comparison of the scores within the Weight of Evidence Chart above, each of the five scales that were created before 2005 demonstrates higher scores than any of those created after this date (see Table 3.4). 120 Table 3.4: Scale Weight of Evidence Scores This systematic review did not include theoretical positionings of disability as one of its search parameters. In addition, a subsequent review of each of these nine studies revealed that none made any references to specific models of disability. However, of the nine studies, four provided complete sets of scale items which were framed as self-reports with Likert-style response formats. These were further analyzed to determine whether each item was primarily influenced by the medical, social or embodiment models of disability. As noted above, the ATEP provided only a partial list of the items in English translation and was not included in that analysis. A second study, the EOS, was also excluded. Although the items were provided, they were focused on employer attitudes toward general inclusiveness within the workplace rather than being specific to PWDs. For purposes of this analysis, I assigned items to one of the three current models of disability based on whether each item represented responsibility for disability 121 as deriving from flaws or limitations found within the PWD (medical model), as residing outside the individual and within the social environment (social model), or as a combination of the two (embodiment model). I also assigned several items to an ‘unknown’ category if this categorization of responsibility was not apparent. A review of the four scales, as illustrated below in Table 3.5, demonstrates that each contains a majority of items that represent a medical model of understanding disability and a minority of items informed by either the social or embodiment models, or which were categorized as ‘unknown’. Each of the scales included items representing each of the three models of disability, with 48 percent of all items representing the medical model. Table 3.5: Scale Items and Models of Disability Classification of Scale Items by Model of Disability # Scale Name Items Pos Neg Medical Social Embodiment Unknown 1. Attitudes Toward Disabled Persons (ATDP-O) 20 5 15 13 2 4 1 2. Scale of Attitudes Toward Disabled Persons (SADP) 24 12 12 15 6 1 2 3. Interactions with Disabled Persons Scale (IDP) 20 3 17 3 3 2 12 4. Attitudes Toward Employing Persons with Disabilities Scale (ATEPD) 13 6 7 6 3 4 0 Item Totals 77 26 51 37 14 11 15 Percentages 100% 34% 66% 48% 19% 13% 22% Summary psychometric information and statements indicative of the authors’ understandings of PWDs are provided below in Table 3.6 for each of the nine scales. A review of these statements yields a result similar to that obtained with regard to specific items within the scales. Neither provides explicit or implicit evidence to support a single underlying model of disability. 122 A review of the included measures also reveals that a definition of disability was not typically provided to participants. In addition, the understandings, and subsets, of PWDs considered within the several studies varied significantly (see, for example, Antonak, 1982; Yuker et al., 1970). In some cases, (Yuker et al., 1970; Findler et al., 2007), PWDs were represented by a single image or a narrow subgroup, with attitudinal results being extrapolated to a more broadly based construct of disability. These nine measures provide varying degrees of reliability and validity evidence. In addition, items within these studies include the presentation of scenarios and descriptions of disability with inappropriately negative qualities and characteristics (see, for example, Yuker et al., 1970; Findler et al., 2007). Akin to racial profiling and other inherently negative descriptions of minority groups, presentations of disability within such a framework cannot help but influence the attitudes that are the subject of the measures themselves, thereby affecting overall validity claims. A negative association between traits or circumstances and a reference group may “contribute to automatic negative evaluations of minority groups, even if the stereotype is not endorsed” (Bohner & Dickel, 2011, citing Devine (1989), p. 399; Schwartz, 2008). None of these scales provides evidence that would indicate its development within a specific structure or model of disability. As shown above in Table 3.5, each of the scales for which analysis was possible indicates that items were drawn from each of the three current models of disability. Without an indication of the ontological perspective applied when scale items have been created by the researcher, it is difficult to determine a basis upon which responses to specific items should be valenced. For 123 example, Item 5 in the ATDP Scale states that “Disabled people are the same as anyone else” (Yuker et al., 1970, p. 113). A medical model of disability presents PWDs as being sick and in need of professional intervention (Parsons, 1951); within this understanding, PWDs are, by definition, different from others. Within a social model of disability, disability itself is a function of social neglect or malfeasance and not inherent in an individual (Oliver, 1990). As described by DeVellis (2017), different items within a single scale may relate positively or negatively to a single construct; as a solution to this possibility, those items “whose correlations with other items are negative” (p. 140) may be reverse scored to create a set of responses which coherently address a particular construct. However, as described above, without a means to determine the intended valence of a particular item, this advice becomes moot. These challenges may be addressed through the provision of a clear articulation of the ontological perspective of disability applied to the formation of future instruments intended to measure attitudes toward PWDs within employment environments. These instruments could then be more accurately evaluated for evidence of theoretical validity as well as that of empirical validity. For example, if a Social Model of disability was endorsed, each item of the proposed instrument could be considered relative to its relationship to that model. As demonstrated below in Chapter 4, this consideration of evidence of ontological validity is incorporated into the development of the CADE Scale. Table 3.6: Item Valences, Theoretical Assumptions, and Psychometric Data # Scale Name Item Valences Assumptions Empirical Analyses Comments 1 Attitudes Toward Disabled Persons (ATDP) (Yuker, Block & Younng, 1970) Neg =15 Pos = 5 States that the degree of perceived difference between PWDs and others determines whether attitudes are positive or negative. Greater perceived difference results in more negative attitudes. This correlation applies to both the self-reflections of PWDs and others. PWDs (referring to those with physical disabilities) are represented by those working in what would today be referred to as affirmative industries or social enterprises. (i.e. paid employment of PWDs in supported, clustered environments. Considered by authors to be otherwise unemployable.) -test/retest reliability -response bias -factor analysis The ATDP is comprised of 20 items which focus on the characteristics of PWDs or on the manner in which they should be “managed”. The authors stated that the items were developed through a literature review and consultation with psychologists working in the field; no details were provided. Disability is most frequently characterized as a negative attribute. 2 Scale of Attitudes Toward Disabled Persons (SADP) (Antonak, 1982) Neg =12 Pos = 12 No hypothesis regarding the nature of disability or any description of what may create attitudinal variance is provided. The scale item topics and assumptions present an image of disability that focuses on the potentially negative attributes of damaged people. Participants are expected to provide responses that judge this degree of damage as presented. -inter-item correlations -factor analysis -reliability This 24 item scale was developed to address concerns over the psychometric properties of the ATDP Scale and to provide more modern language, imagery and social context. 3 Interaction with Disabled People Scale (IDP) (Gething & Wheeler, 1992) Neg =17 Pos = 3 States that theorists regard attitudes toward PWDs as typically negative, with these negative attitudes based on perceptions of difference that imply inferiority and deficiency. Indicates that the ATDP and the SADP are outdated in terms of language and subject to validity challenges. Claims that previous measures of attitude toward PWDs which have been based on difference are subject to criticism and are insufficient. This scale considers discomfort with social interactions to be a more latent factor underlying negative attitudes toward PWDs. Disability is considered to be a generic state which generates various degrees of discomfort. -content validity analysis -reliability (test/retest & Cronbach’s alpha) -factor analysis Items were generated initially by analyzing open-ended descriptions by 633 individuals of how they felt when meeting a PWD. Responses were then analyzed relative to variables cited in the literature as associated with negative attitudes. These items were then reviewed by PWDs and others. The scale items imply that PWDs are in need of help, vulnerable, and expected to be patient with others. 4 The Implicit Association Test (IAT) (Greenwald et al., 1998) n/a No hypothesis regarding the nature of disability is provided. However, respondent reactivity, especially as it relates to social desirability, was of particular concern. The IAT has since been used for at least 18 studies that focused on various disabilities within the general categories of intellectual and physical disabilities. Overall, implicit attitudes were scored as more negative than explicit attitude scores generated by self-report questionnaires. -response latency analyses -convergent/ discriminant validity analyses Of the 2009 participants in 18 studies, 85 had an employment relationship with PWDs. Of the 11 studies that conducted both explicit and implicit measurements, six found small significant relationships between the two. The remaining five studies found no significant relationship. 125 # Scale Name Item Valences Assumptions Empirical Analyses Comments 5 Attitudes Towards Employing People with Disabilities Scale (ATEPD) (Loo, 2004) Neg = 7 Pos = 6 No hypothesis regarding the nature of disability is provided. The ATEPD Scale, when evaluated through a discomfort/sympathy matrix, reveals that the discomfort dimension plays a dominant role in attitudes toward PWDs. In addition, high discomfort correlated with more negative scores in operational categories such as productivity and being a burden. -response bias analyses -ANOVA analyses of response categories Items in this scale are specifically directed toward employment circumstances. Participants were business students, not individuals who had worked with PWDs. 6 Employer Openness Survey (EOS) (Gilbride et al., 2006) Neg = 4 Pos = 1 No hypothesis regarding the nature of disability or any description of what may create attitudinal variance is provided. The EOS was developed to provide rehabilitation professionals with a tool to gauge the openness of employers to hiring PWDs. Employer attitude was seen as a significant contributor to the construct of openness. The scale is administered by rehabilitation professionals to employers, with scoring based on pre-determined response categories. -content validity/pilot testing (incl. interrater correlation analysis) Items for this 18 item scale, principally open-ended, were generated by the authors and then reviewed by employers. There was no reference to a literature review as a means to generate items. No response data from the completed survey were provided. 7 Disability Attitude Implicit Association Test DA-IAT) (Pruett & Chan, 2006) n/a No hypothesis regarding the nature of disability or any description of what may create attitudinal variance is provided. Developed as a means to measure implicit attitudes toward PWDs. Scores indicated no relationship with the ATDP Scale nor with demographics (age, gender) of participants. The most significant predictor of positive attitudes toward PWDs was previous contact. -convergent/ discriminant validity analyses -independent sample t tests -multiple regressions This study made use of a combination of 132 symbols intended to represent either disability or able-bodiedness. These were coupled with positively or negatively valenced words. 8 Attitudes Toward Employment of Psychiatric Disability Scale (ATEP) (Ozawa & Yaeda, 2007) Neg = 9* Pos = 0 No hypothesis regarding the nature of disability is provided. The ATEP was developed to gauge employer attitudes toward employees with psychiatric disabilities. The scale was delivered by mail to 1598 employers with 358 responses. Frequency of exposure and the number of psychiatrically disabled employees were associated with more positive attitudes. -factor analysis -reliability analysis (Cronbach’s alpha) This study contained 27 items developed through a literature review and secondary review by 10 SMEs. Pilot study was conducted with 27 participants (not described). *Only 12 items were available in English. Nine of these items were specific to PWDs. 9 Multi-dimensional Attitude Scale Toward Persons with Disabilities (MAS) (Findler, Vichinsky & Werner, 2007) n/a No hypothesis regarding the nature of disability or any description of what may create attitudinal variance is provided. This scale was developed as an indirect measure of attitude, making use of paper-based vignettes that involved disabled/able-bodied who were either male or female. It was presented as containing cognitive, affective and behavioural subscales. -factor analysis -convergent validity analysis This scale contains 47 items. Concurrent validity was demonstrated by comparison with the ATDP Scale. However, three items were removed from the ATDP before Cronbach’s alpha (α = .89) was determined. 126 Strengths and Weaknesses This review has a number of strengths. It is the first systematic review of instruments developed to measure attitudes toward PWDs in employment. By assembling four instruments that have been created between 2005 and 2015 as well as by including five earlier measures that are still commonly in use, it is able to offer a perspective of the relationship between PWDs and employment that would otherwise be difficult to observe. In particular, three threads of development are apparent. First, the understanding of attitude as a significant contributing factor moves from a unidimensional (Yuker et al., 1970) to a multidimensional interpretation (Antonak, 1982; Stone & Colella, 1996). Second, data gathering methodologies have developed indirect measurement techniques to address considerations of bias within self-reports of such a socially sensitive topic (Antonak, 1982; Greenwald, McGhee & Schwartz, 1998). Third, there is a recognition that the cognitions, affects, and behaviours of those whose attitudes impact the employment of PWDs are not necessarily congruent within individuals (Pruett & Chan, 2006; Ozawa & Yaeda, 2007; Findler et al., 2007). Neither are these attitudes congruent within individuals within employment and other social environments (Burke et al., 2013; Domzal et al., 2008). By conducting this systematic review, it has been possible to compare the psychometric properties of these several measures. As importantly, it has been possible to observe a common gap within this same set of measures. As observed within this review, the manner in which a variety of instruments have been created to measure attitudes toward PWDs has undergone significant developments within this period as 127 various critiques have been raised and addressed. However, based on the instruments reviewed herein, these measures have not been sufficiently validated through an evidentiary perspective, such as measuring changes in attitude following training or educational interventions. This position is supported by Brostrand (2006) who noted that Tilting at Windmills, one of the most well-known and respected American disability awareness training programs, although widely endorsed, has never been formally evaluated to gauge its effect on participants. This systematic review has several limitations. These include having only a single reviewer to evaluate sourced materials and a restriction to English language reports in peer-reviewed journals (Gough, Oliver, & Thomas, 2012: Littell, Corcoran, & Pillai, 2008). In particular, a review of grey literature was excluded due to time and resource restrictions. In addition, although the studies were conducted in five different countries (six from the USA and one each from Canada, Japan, Israel, and Australia), all nine of the measures were reviewed in English, raising concerns over generalizability and a lack of cultural diversity. Finally, although additional validity evidence may have been available following subsequent applications of each of these scales, this review considered only the evidence provided within the initial papers in which each of these was introduced. A steady development in the complexity and sophistication of measures intended to examine attitudes toward PWDs has been observed within this review. However, as discussed above, the potential for ontological bias associated with these measures 128 indicates the requirement for further research, particularly through an investigation into the effect of explicit theoretical frameworks being specified within future studies. Finally, of the nine instruments identified within this review, only four gathered data directly from employers, some of whom were recruited based on predispositions toward positive responses. In addition, by not providing definitions of disability to respondents who were most typically undergraduate students with unstated degrees of exposure to PWDs in the workplace, several of the studies risked assigning degrees of validity evidence based on data that may have been unrelated to the construct under consideration. Any new instruments would benefit from a consideration of these limitations. Key Points The negative attitudes of co-workers within the workplace are the most significant barrier to employment success for PWDs. Current measures of co-worker attitudes do not provide theoretical validity evidence in support of inferences made from test scores. The limited validity evidence provided by current measures reveals a knowledge gap in the consideration of co-worker attitudes toward PWDs. 129 Chapter 4: The CADE Scale – Theory, Development, and Analysis Validity, as it is applied to measurement instruments, is defined as “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (AERA et al., 2014, p. 9). Within this context, five sources of validity evidence are considered to support these interpretations of test scores. These include evidence based on test content, response processes, internal structures, relations to other variables, and the consequences of testing. Of these, evidence based on test content is not directly related to these interpretations of test scores but provides a foundation upon which these interpretations may be framed (Messick, 1980). The CADE Scale, at this formative stage, requires the generation of sufficient evidence of content validity to warrant proceeding to the next step in its development. The next step, that of initial validation studies (DeVellis, 2017), will be the first in which validity evidence from these additional sources may be gathered. To understand the implications and purpose of content validity as it applies to the CADE Scale first requires a consideration of the more broadly based concept of validity as it applies to measurement instruments. A Brief Review of Validity Theory Validity, according to Messick (1990), is “an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of interpretations and actions based on test scores or other modes of assessment” (p. 5). Validity is now generally conceived as a unitary concept in which several lines of validity evidence are subsumed within an overarching framework of 130 construct validity (see, for example, Hubley & Zumbo, 1996; Messick, 1980, 1989, 1995; Sireci, 1998a; cf. DeVellis, 2017). The unitary validity theory asserts that the purpose of measurement is “to make inferences from observed test scores to unobservable constructs” (Sireci, 1998a, p. 84), which requires evaluating the construct validity of those inferences. However, the unitary conception of validity has not always represented the manner in which validity has been understood. The concept of construct validity was described by Cronbach and Meehl (1955) as a specific form of validity, which also included the separate aspects of content validity and criterion validity. Shortly thereafter it was argued that all validity is construct validity (Loevinger, 1957). This latter understanding of validity has also undergone significant development; from a narrow, clearly defined set of measures that exist as a function of validity within the scale itself (Loevinger, 1957) to more broadly-based conceptions of validity that rest on the interpretation of test scores for specific purposes. These interpretations are also understood to incorporate considerations that include consequential implications, including various social and axiological relationships (Hubley & Zumbo, 2011; Messick, 1998). However, this concept of validity is not universally accepted. DeVellis (2017) presents a more narrowly framed image of validity that is restricted to content, criterion, and construct validity. Messick (1995) states that all aspects of validity are contained within a more comprehensive notion of construct theory, which includes six distinguishable aspects. These include “content, substantive, structural, generalizability, external, and 131 consequential aspects of construct validity.” (p. 741). All validity claims are forms of evidence that inform inferences about the meanings of scores. In addition, the unified theory of validity supports objective elements within the scale development process itself as well as the inclusion of subjective elements peripheral to this core process. As noted by this author, these subjective elements could exist as both precursors to and as consequences of the scale development process. Although Messick (1995) acknowledged the potential for antecedent validity claims, the focus was clearly on those considered to be consequential. This is evidenced by the inclusion of the consequential as one of the six distinguishable aspects of validity mentioned above, with no mention of the antecedent. Current commentaries on validity also address what are referred to as value implications (Hubley & Zumbo, 2011; Messick, 1998). These are described as the personal or social values implied by a researcher’s interest in a particular topic, and the theoretical underpinnings reflected by these interests. Although various commentaries acknowledge the broad and pervasive effect of values throughout the testing process (Brennan, 2001), they do not provide guidance regarding how best to specifically address the effects that the input of values at the earliest stage of a chronologically-based developmental process will have on its outcome. As an example, Brennan indicates that researchers have a responsibility to disclose “personal politics” (p. 12) as these may influence value-based assumptions. However, no strategy or process is provided. 132 In the above summary of the concept and definitions of validity, it is clear that understandings of validity have undergone a continuing evolution. If we first consider Loevinger’s (1957) categorization of validity, it is described as a matrix of validity indicators that is a reconfiguration of a division wherein the “APA Committee on Psychological Tests (had) divided validity into four types, content, concurrent validity, predictive validity, and construct validity.” (p. 653). Loevinger (1957) had reconfigured this matrix to provide a more “logically coordinate” (p. 653) model which placed construct validity at the top of a pyramid-shaped structure containing three primary subdivisions of substantive, structural and external validity. These three subdivisions each contained further refinements and subcategories. For example, external validity was further subdivided into predictive and concurrent validity. However, the challenges associated with maintaining a rigid, structured definition and application of validity that is separate from its relationship to changing interpretations and meanings have become apparent over time. This point is clearly illustrated when we consider Loevinger’s (1957) statement, in a reference to construct validity, that “these items as a group are good indications of hysteria and hysteroid behavior” (p. 652). Although the group of items may be good indicators of some construct, it is not in today’s environment likely to be the one they originally addressed. Similarly, within the context of the unitary theory of validity as put forward by Messick (1980, 1989), content validity came to be considered as something distinct from other forms of validity evidence because it applied to characteristics within an instrument rather than to inferences associated with test scores. Messick (1980) 133 recommended that the term content validity be replaced by “content relevance”, “content representation” or “content coverage” depending upon the particular intention. However, the categorization of content validity, counter to arguments put forward by Messick (1980), and now more commonly referred to as evidence based on test content, is generally considered to again belong within the framework of construct validity. Listed as one of the five sources of validity evidence by AERA et al. (2014), evidence based on test content is considered to provide “important validity evidence [that] can be obtained from an analysis of the relationship between the content of a test and the construct it is intended to measure” (p. 14). This ongoing process of revision and reconceptualization of validity is recognized by Messick (1980), who states that construct validation is a “continuous, never ending process developing an ever-expanding mosaic of research evidence” (p. 1019). That author also notes that “validity is an evolving property and validation a continuing process” (Messick, 1995). Ontological Validity Evidence I argue that there is a further opportunity to continue with the development of the concept of validity beyond that described above. Specifically, a researcher’s antecedent ontological position must be specifically taken into consideration as an integral part of any assessment of validity. That ontological position creates clear propensities and limitations, conscious or otherwise, that inform scale development and implementation. Furthermore, these antecedent determinants exert influence on the 134 balance of the research process, including the selection of a research question to which a scale is to be applied and the construction of the items within the scale itself. Non-evidential aspects of validity development have been discussed by Hubley & Zumbo (2011) and others as a part of the overall consideration of validity. Messick (1989) states that “test validity and social values are intertwined and that evaluation of intended and unintended consequences of any testing is integral to the validation of test interpretation and use” (p. 13). Messick (1998) also notes that “value implications both derive from and contribute to score meaning, (therefore) different value perspectives may lead to different score implications” (p. 37). These value implications require that we are “more cognizant of and question how our values may shape our theories, the development and evaluation of our measures, and our interpretations of research findings” (Hubley & Zumbo, 2011, p. 223). Messick (1994) argues that value implications and social consequences are intrinsic to the meaning and outcomes of testing. These are not something that has recently been added in, rather they have always been intrinsic, but only recently exposed as such. The unified theory of validity “makes explicit what has been latent all along, namely, that validity judgments are (italics in original) value judgments” (Messick, 1994, p. 748). As indicated above, the ongoing consideration of values, and their iterative relationship with research findings, occupies considerable space within current considerations of validity. Although this focus on value implications has primarily been consequential to scale applications, there has been acknowledgement that antecedent 135 ontological positions are also of interest (Hubley & Zumbo, 2011; Messick, 1995). Although not using the language of antecedent validity, Messick (1995) states that score meanings are supported by an explicit evaluation of “the theoretical rationale underlying score interpretation” (p. 745). Finally, Messick (1998) indicates that validity is, to an extent, dependent on values and “also on evidence of consequences and antecedent conditions” (p. 38). Although these acknowledgements of ‘antecedent conditions’ are clear, I argue that the connection between the antecedent ontological underpinnings and the balance of the process involved in scale development and application has not yet been sufficiently considered. This gap in an explicit consideration of these antecedent influences creates a risk of undermining any resultant determination of content validity within scale development. In turn, this failure within the scale development process results in an unrecognized gap in overall validity evidence when considering “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (AERA et al., 2014, p. 9). I offer a solution to this dilemma by proposing the inclusion of an additional contribution to validity evidence, that of ontological validity evidence. As illustrated below, in Figure 4.1, the process directly associated with the mechanics of scale development is separate from the ontological position of the scale developer. Of particular note is that without an understanding of the ontological position assumed by the scale developer, although the overall structure of the scale may be unaffected, the meanings of these scores may be understood quite differently. 136 Figure 4.1 illustrates this separation. The antecedent ontological input may affect the output without disturbing the internal process associated with the actual development of the scale. The process is the same; the result is different. As noted earlier, the three most widely considered theoretical frameworks for disability are the medical model, the social model and the embodiment model (Davis, 2006b). In addition, I have introduced the Difference Model within this dissertation. As presented above, each of these first three occupies a different position along a continuum related to responsibility, with the Difference Model positioned along a dimension related to a perception of difference. However, the procedure involved in scale development, in this example that of the CADE Scale, would not necessarily vary significantly in terms of its internal structure regardless of which of these models was of most influence. Each could produce the same scale categories and items. However, the difference would be discernable at the output level, on the “Co-worker acceptance of PWDs”. Without a recognition and consideration of this antecedent aspect of the environment in which the scale is developed, a clear understanding and interpretation of the inferences generated from these scale scores is made more difficult. This point is illustrated, as noted above, with an example item from the ATDP Scale, (Yuker et al., 1970). That item states that “Disabled people are the same as anyone else” (p. 113). Without a process to determine the ontological perspective within which this item has been presented within the scale, this item may as easily be considered to support the equality of PWDs or to support their difference. 137 The recognition and acknowledgement of the ontological environment in which a scale is framed, particularly at the early stages of development, serves to begin to address the issues described above. Although not framed within the language of ontological validity evidencemany researchers, within the field of validity studies and elsewhere, acknowledge the relationship and impact of the worldview of researchers upon their interpretations of their research. Kane (2001), in a consideration of current validity theory, states that the “interpretations of observations always rely on prior assumptions” (p. 334). Messick (1995) indicates that the unified theory of validity allows for both objective and subjective elements. These latter elements, including self-knowledge, may exist as both precursors to and consequences of scale development. In that same study, value implications are considered to encompass “ideologies about the functions of science or about the nature of the human being as a learner or as an adaptive or fully functioning person” (p. 748). Schatzman (1991), in a consideration of the dimensionality of various social constructs, recommends a consideration of researchers’ “commitments to received conception, sociological or political-ideological” (p. 311) positions, as a means to aid in the interpretation of data. Kools et al. (1996) raise a similar caution, stating that research perspectives require explanation because each “provides a different configuration to the data and results in a different interpretation of meaning” (p. 319). Rioux (1997), within a consideration of the effects of discrepant worldviews as these pertain specifically to disability-related research, states that “it is a moral imperative to disclose the normative basis of the research in a 138 field like ours, where the research affects the single most identifiably marginalized class of people” (p. 102). Although there is a broad recognition of the possible effect of researchers’ various worldviews, the question now to be considered is whether there is a means to address these ontological considerations in a manner that would generate additional validity evidence within scale development; in particular, within the development process of the CADE Scale. 139 Figure 4.1: Ontological Validity Evidence Input Matrix Theory Assumptions Scale Categories Items Axiology 1 2 Ontology 3 4 Co-worker Acceptance of PWDs Epistemology Individual Responsibility Social Responsibility Combined Responsibility Perceptions of Difference Hiring Performance Accommodations Supervision/ Dismissal 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Models of Disability 140 A means to address ontological validity evidence may be found within the range of qualitative techniques utilized to acknowledge, if not measure, the potential for bias brought to bear on the interpretation of data generated from social interactions. An application of one or more of these qualitative techniques, when considering the interpretation of quantitative measures, may be justified when the quantitative analyses are, at least to some extent, dependent on the acceptance of underlying assumptions. As stated by the Oxford English Dictionary (1997), an assumption is that which is taken for granted as the basis for an argument or action. Following this definition, although quantitative processes may maintain an internal coherence, they do so only with the forbearance and to the degree allowed by the surrounding assumptions. Specifically, a researcher’s antecedent ontological position creates clear propensities, limitations, and biases (Creswell, 2013), that inform scale development and implementation. Qualitative research provides a number of examples of how best to acknowledge and consider the impact of these a priori assumptions. Maxwell (2012) argues that all knowledge is theory-laden and, as such, cannot be established as having insight into ontological reality beyond its own defined limitations. Creswell (2013) offers an overview of eight distinct methods, several combinations of which may be used to examine and ameliorate researcher bias which may be introduced into any study. Of particular note is the requirement of “clarifying researcher bias from the outset of the study … so that the reader understands the researcher’s position and any biases or assumptions that impact the inquiry (emphasis in original)” (p. 251). That author also identifies researcher reflexivity as a means to consider a set of possible biases, 141 recognizing that interpretations by readers and the consequent impact on participants may be influenced by those biases (Creswell, 2013). This value of reflexivity is supported by Maxwell (2012) who also describes a commonly employed framework of triangulation for its consideration. That author describes triangulation as a method of making use of several methods and sources to gather and present information, thereby “reduc(ing) the risk that your conclusions will reflect only the systematic biases or limitations of a specific source or method” (p. 106). This reflection and the consequent transparency generated by this process of reflexivity is intended to create a more fulsome validation process. If the above arguments are accepted, that understandings are interpretive, there is a corresponding obligation to describe and offer for inspection the framework from within which these understandings are derived. Otherwise, statements related to evidence of validity are at least partially unavailable for critique. If validity is influenced by the interplay between an internally coherent epistemology and a particular ontological framework which informs that internal coherence, there is significant value to be found in making visible these ontological assumptions. Presenting that ontological framework for inspection will provide the opportunity for a more broadly based and coherent analysis and critique of validity evidence. Therefore, ontological validity evidence is operationalized as a representation of the degree to which theoretical rationales are integrated into the development of assessment measures. 142 Content Validity Content validity is the degree to which the elements of an assessment instrument are “relevant to and representative of the target construct for a particular assessment purpose” (Haynes, Richard, & Kubany, 1995, p. 238). This evidence is based on the degree to which the items within the instrument adequately represent the construct without being irrelevant (Messick, 1989). These are the “two major threats to construct validity, namely, construct underrepresentation…and construct-irrelevant variance” (Messick, 1989, p. 14). Test content includes the wording and format of each individual test item, the administration and scoring processes, and the methods used to generate these items. According to Sireci (1998a), there is a general consensus that four elements inform the quality of evidence of content validity. These include “domain definition, domain relevance, domain representation, and appropriate test construction procedures” (p. 101). This latter effort may employ logical or empirical analyses of proposed test content or from expert judgments on the part of Subject Matter Experts. However, as noted by Beck & Gable (2001), “There are no set criteria for deciding to what degree an instrument has achieved content validity. Evidence of content validity is primarily judgmental in nature” (p. 207). Judgmental studies are commonly recommended as an effective method to gather validity evidence based on test content (AERA et al., 2014; Beck & Gable, 2001; Grant & Davis, 1997; Lynn, 1986; Rubio, Berg-Weger, Tebb, Lee, & Rauch, 2003). The test elements considered within these assessment instruments include the scale items 143 themselves, the manner in which data are gathered, and other associated components including the instructions to the SMEs. I will describe each of these components within a methodological framework that is comprised of two broad categories, the developmental stage and the judgment-quantification stage as these generally pertain to an affect-based study (Lynn, 1986) and to the CADE Scale in particular. The CADE Scale According to Polit and Beck (2006), when developing a new scale, evidence based on content is a necessary component required to draw conclusions about scale quality. These authors suggest that there is general agreement that evidence based on content “concerns the degree to which a sample of items, taken together, constitute an adequate operational definition of a construct” (p. 490). A Judgmental Study. This content validation study is intended to generate and evaluate expert input as evidence of content validity in support of this initial stage of the CADE Scale development process. The purpose of this evaluation is to determine whether this evidence is sufficient to warrant the next developmental step, that of administering the CADE Scale through initial evaluation studies (Rubio et al., 2003) within the private and public business sectors. The CADE Scale has been developed within a three-step framework that includes domain identification, item generation, and instrument formation (Lynn, 1986, citing Carmines & Zeller, 1979; Nunnally, 1978; Williamson, 1981). Each of these steps is described below. 144 Domain Identification – The process recommended for domain identification varies depending on whether the scale is a cognitive or affective measure (Grant & Davis, 1997; Lynn, 1986). In the latter circumstance, such as with the CADE Scale, Lynn recommends that the first step is a “thorough review of the literature on the topic of the measure so that all dimensions and subdimensions can be identified” (p. 383). This first step, that of domain identification, has been implemented through a literature review of disability-related research as reported on in my description of the several models of disability in Chapter 2 and through the systematic review of disability-related measures of attitudes as described in Chapter 3. Within this dissertation, the domain being investigated is the workplace acceptance of PWDs by co-workers. The conceptual definition that I have put forward is - The acceptance of PWDs is a measure of the workplace-related attitudes exhibited by co-workers toward this group. Acceptance is described as a function of the degree to which PWDs are considered by co-workers to be different from others within the workplace. Within this context, acceptance is operationalized as -The degree of perceived difference attributed by co-workers to disabled employees within a workplace environment as measured by the CADE Scale. This domain was further analyzed, and subsequently categorized into four subdimensions. These include - the hiring process; workplace performance; accommodations4; and supervision, discipline and termination. 4 In many jurisdictions, including the United Kingdom and Australia, disability-related accommodations are referred to as workplace adjustments. 145 These subdimensions were developed through my review of employment-related human rights complaints filed by job seekers and employees with disabilities with the British Columbia Human Rights Tribunal (BCHRT), the Canadian Human Rights Commission (CHRC), the American Equal Employment Opportunities Commission (EEOC), and with additional input from the Job Accommodation Network (JAN). My review of all BCHRT decisions between 2005 and 2014 indicated that 38 decisions were handed down that were related to both disability (Section 8, BC Human Rights Code) and employment (Section 13, BC Human Rights Code). These data from the BCHRT were obtained through a review of all adjudicated complaints filed with the BCHRT that included the two keywords disability and employment. Each of these cases was then reviewed for references to the specific aspects of employment that had led to the complaints. Of these 38 decisions, 23 cases included complaints regarding accommodations, two cases included adverse treatment during employment, and 23 included complaints regarding dismissal. None included complaints related to hiring practices. This list does not include cases that were settled prior to going to a tribunal for adjudication or which were abandoned after initiating a tribunal due to settlement or other causes. Data from the CHRC were obtained following an informal internal review by that organization conducted at my request. Previously, I had been advised by the CHRC that this organization does not keep statistics that allow for a categorization of workplace-related complaints by PWDs that would reveal the specific aspect of employment involved in individual complaints. However, following my request, the CHRC conducted 146 an informal review of recently adjudicated cases that were associated with employment and disability. It was reported to me that this review supported the fourfold categorization that I had described. I was also advised that, since 2011, within Canadian workplaces that fall under the jurisdiction of the CHRC, complaints that had previously been categorized under accommodations or performance /adverse treatment have been combined under a general complaint heading of adverse treatment. This category encompasses the broad range of workplace activities and relationships that do not include the hiring or termination processes. (Personal communications with Michael Vasseur, Program Officer, CHRC, between February 19 and 22, 2016). In the United States, these data were gathered through a review of charges5 that had been filed with the EEOC, as provided through the Disability Statistics website of Cornell University, which tracks all disability-related complaints filed with the American EEOC6. Within the period of 2012-2014, 59 percent of employment-related charges were related to discharge, 35 percent were related to reasonable accommodations, 28 percent to harassment/discipline, and 6 percent to hiring practices7. American data were also obtained from the Job Accommodation Network (JAN), University of West Virginia (personal communications with the Executive Director of JAN between January 27 and February 10, 2016). That organization provides employment-related advice to employees and job seekers with disabilities across the United States and in Canada8. At my request, JAN had conducted an internal review and analysis to determine the most 5 In the United States, human rights complaints are referred to as charges. 6 As retrieved from Cornell University website on Feb. 5, 2016 https://www.disabilitystatistics.org/eeoc/tableau.cfm?report=1 7 This total exceeds 100%. These and other charges often included more than one category. 8 www.askjan.org 147 common categories of concern raised by its clients. Their conclusions also supported these four categories of complaint. Item Generation - The second step in the development of a judgmental study is the drafting of an initial set of items for inclusion in the scale. Recommendations regarding the overall ratio of initial items to an anticipated final number vary considerably. DeVellis (2012) recommends a ratio of between fifty percent and three or four hundred percent more than expected in the final scale, depending on the degree of difficulty in generating items. Miller, McIntire, and Lovlar (2011) indicate that a general rule is to write twice as many items as anticipated for inclusion within the test. Lynn (1986) notes that “it is always better to generate too many items than too few” (p. 383). Each of these recommendations for generating a relatively large set of initial items has included an expectation that a significant percentage will be removed or modified during a following analytical stage, including those items that are insufficiently supported by SMEs (Lynn, 1986; Waltz et al., 2010). ). In addition, during subsequent pilot testing, items may be misinterpreted, deemed to be too easy or too difficult, or answered differently based on gender (Miller et al., 2011). A final review by the researcher, based on this input and his or her professional experience, will also help to determine the items selected for the scale (Beck & Gable, 2001; Waltz et al., 2005). I reviewed the numbers of items contained within the scales described within the systematic review as a means to compare completed scales specific to attitudes toward PWDs with these more general guidelines regarding numbers of items. The mean 148 number of items from the five scales with Likert-type response formats, and which indicated the number of items, was 20.2 items. Second, I hypothesized that an initial number of 40 items reviewed by the SMEs would be reduced by as much as half, resulting in a scale that would be similar in length to those within the systematic review. Although there are no clearly delineated rules regarding scale length (Hinkin, 1998), this issue is addressed by DeVellis (2017) who notes that, although respondents may be more willing to answer a shorter scale than a longer one, there is “a trade-off between brevity and reliability” (p. 147). In addition to determining the overall length of a scale, researchers indicate that consideration must be given to the proportional representation of items to be included within the applicable categories or dimensions (DeVellis, 2012; Haynes et al., 1995; Hubley & Zumbo, 2013). Maintaining consistency between the internal structure of the scale and that of the construct domain creates “structural fidelity” (Messick, 1995, p. 746). Therefore, the item categories included within the original CADE Scale were derived from the four aspects of employment that have been the most commonly grieved by PWDs as generating discriminatory treatment within the workplace. Although the categorizations of these complaints vary somewhat between jurisdictions, both between provincial and federal jurisdictions within Canada, and between Canada and the United States, these complaints have been most frequently associated with dismissal, followed in order by lack of accommodations, performance/adverse treatment, and hiring procedures. These four categories provided a framework for the development of items for inclusion in the CADE Scale as a means to meet the 149 requirements of fully representing the construct under consideration while excluding irrelevant aspects (Lynn, 1986; Messick, 1980; Waltz et al., 2010). These considerations resulted in my decision to generate an initial set of 40 items, with four subsets of ten items that corresponded to the four subdimensions within the scale (see Appendix A). My drafting of each of these specific 40 items followed my literature review and a consideration of data obtained through four separate sources of inquiry. As recommended by Antonak (1988), I have provided an account of each of these steps. First, I conducted a thorough review of employment-related items within the scales considered above in Chapter 3 to determine which of these may have addressed issues of perceived workplace differences between PWDs and co-workers. Second, I held preliminary discussions with three SMEs to determine their understandings of employment-related difficulties experienced by PWDs. These discussions confirmed the four-part categorization of workplace-related attitudes toward PWDs as described above. Third, I drew upon my own professional experience (Beck & Gable, 2001; DeVellis, 2017), including a consideration of the questions and comments raised within over 100 disability-awareness training sessions and workshops that I have delivered to employees of various Canadian public and private organizations over the past 30 years. The most frequent of these queries focused on developing strategies to resolve perceived differences, whether these were associated with performance, attitudes or behaviours. Finally, following an initial draft of each of these items, I reviewed each to confirm that it represented the Difference Model of disability by providing a clear comparison or distinction between PWDs and others in the workplace. This final 150 evaluation was intended to confirm the ontological coherence of the several items and, therefore, to provide evidence of ontological validity within the scale. Instrument Formation - This final step in the development of the CADE Scale, as described by Lynn (1986), centres around “the assertion by a specific number of experts that the items are content valid… and that the entire instrument is content valid” (p. 383). This panel of SMEs provides ratings, comments and recommendations on the relevance and clarity of proposed items to the content domain (Beck & Gable, 2001; DeVellis, 2017; Lynn, 1986). SMEs can include individuals with professional certification in a related field, those who have presented or published papers on the general topic, or have conducted related research. Overall, SMEs should have expertise with the theory and concepts which inform the instrument under development and possess relevant training, experience and qualifications (Grant & Davis, 1997). SMEs are also described as those who have “worked extensively with the construct in question or with related phenomena” (DeVellis, 2017) and should include individuals with personal experience of the construct under consideration (Beck & Gable, 2001; Grant & Davis, 1997). Sireci (1998b) offers three general recommendations when selecting SMEs. These include selecting individuals with familiarity with the content to be tested, variability within different aspects of the domain to be content tested, and that the selected SMEs should be representative of the larger pool from which they are selected. However, this latter recommendation, that SMEs be representative of a population of all possible SMEs, is not universally accepted. Lynn (1986) notes that groups of SMEs are more typically 151 selected from those who are willing to volunteer. Others have indicated that SMEs are often selected from professional colleagues with experience in the particular construct (DeVellis, 2017). L. Russell (2013) recommends that SMEs may be selected, not as being generally representative of possible SMEs, but as an advisory group with at least some members with contradictory viewpoints. This process of instrument formation (Lynn, 1986) is comprised of a three-part methodology – recruiting a cohort of SMEs, providing instructional and ratings materials to the SMEs, and analyzing SME responses to determine the validity evidence resulting from this judgmental study. The first step, recruiting a cohort of SMEs, requires determining a suitable number of SMEs, establishing appropriate qualifications, and then locating and approaching a sufficient number of possible participants. Recommendations for numbers of SMEs vary from a minimum of two to a maximum of 20 (Grant & Davis, 1997; Tilden, Nelson, & May, 1990), with more typical numbers being between 5 and 10 (Beck & Gable, 2001). Hubley and Palepu (2007) indicate that “typically, somewhere between three to ten experts is recommended, although a minimum of five SMEs is recommended to control for chance agreement” (p. 2). Lynn (1986) recommends that a minimum number of five experts would be sufficient to control for chance agreement, but that in circumstances where a larger number of experts was not available, a minimum of three could be adequate. That author also notes that no maximum number of these experts has been established, “although it is unlikely to exceed 10” (p. 383). 152 Lynn indicates that this maximum number of experts may be limited by how many can be identified by the developer of the study. The number of SMEs is also influenced by the specific construct that informs the study. Davis (1992) notes that, with complex constructs, experts from several disciplines may be appropriate. If not reasonable to find SMEs with expertise in all areas, those with subsets of different skills may be selected. Finally, where appropriate, a broad geographic selection of SMEs should be considered to assist with identifying colloquialisms and regionally-based technical language (Grant & Davis, 1997)9. Within my SME recruitment process, I addressed each of these three characteristics - expertise, number, and geographic distribution. First, in recognition of the complex nature of the construct under consideration, I developed a threefold categorization to represent different skill sets, expertise, and perspectives associated with PWDs in the workplace. As noted below, these three categories were also linked to the behavioral, cognitive, and affective aspects of attitude formation (Eagley & Chaiken, 1993). These categories included: Extensive interaction with employees through delivery of disability-related training (trainer [behavioural]); This first group includes those with extensive expertise in employment-related attitudes toward PWDs. These are individuals who have developed and provided organizational training programs and who are familiar with the conceptual framework 9 The value of this latter recommendation, as it applies to both geographic and political influences, is clearly illustrated by a consideration of the debate within disability studies over the use of the term people with disabilities versus disabled people. Other significant differences in language uses within the disability field are also either geographic (e.g. seizure versus fit) or political (e.g. complaint versus charge). 153 (Davis, 1992). Each of these was required to have delivered a minimum of 50 training sessions to employees of either public or private organizations. Understanding of the major structural and functional components of the workplace, including their relationship to disability, either from an academic or practice perspective (organizational experience [cognitive]); The second group includes individuals with expertise in organizational human resources. These include representatives from academic and corporate environments. Each of these has a minimum of five years of experience with either academic research associated with employment and PWDs or at a human resources director level with private or public organizations. Experience as a PWD who has participated in a variety of professional and personal relationships with co-workers and who has observed the effects of disability on those relationships (PWD participant [affective]). This final group is comprised of PWDs with personal and professional experience in the workplace as employees with disabilities. However, this group is distinguished from what are referred to as experiential experts (Grant & Davis, 1997; L. Russell, 2013; Schilling et al., 2007). Experiential experts (EEs) are typically individuals with direct experience with the construct under consideration and are considered frequently to be in need of assistance, as members of the target population, in determining “whether an instrument’s content reflects the target population’s conceptualization of the construct of interest and in identifying unclear terms and language” (L. Russell, 2013). These EEs are also often considered to be in need of support to approach the task from a more 154 broad perspective than their own experience (Stewart, Lynn, & Mishel, 2005). However, this group of PWD participants was recruited from a selection of individuals who had worked within the field of disability management for a minimum of five years within large organizations, many of which have been mandated by either federal or provincial/state legislation to comply with employment equity goals. In this way, the concerns expressed regarding the need for support, and the possible limitations associated with this group, do not apply. For this reason, although members of the PWD participant group, as PWDs, do have direct experience with the construct, they also bring their professional expertise, and have not been considered as a separate group of EEs. This categorization informed my recruitment strategy both in terms of search parameters and numbers. My search included approaching professional and academic colleagues (Lynn, 1986) as well as several other authorities, authors and practitioners within the field. I also began with an assumption that recruiting a sufficient number of SMEs would require contacting a considerably larger number of possible candidates. I had determined that a minimum of five SMEs representing each category would be required (Hubley & Palepu, 2007; Lynn, 1986). This determination resulted in a requirement to recruit a minimum of 15 SMEs. I began this recruitment process by generating a list of 33 possible SMEs, each of whom I either knew personally or knew to represent at least one of these three categories. As noted above, I had further qualified candidates as follows: trainers were to have provided a minimum of 50 training sessions; workplace experts and PWD participants were to have had a 155 minimum of five years of workplace experience. My initial list included seven trainers (Canada , USA , Australia ); sixteen with organizational experience (Canada , USA , UK ); and ten as PWD participants (Canada , USA , UK ). Of these, 24 were male and nine were female. Beginning on May 16, 2016, I contacted each SME either by telephone (12), by email (20), or through Linked In (1). Each of these contacts included a brief description of my research efforts to develop the CADE Scale and to request their participation as an SME. Twenty-six of the SMEs agreed to participate in the study. The second step, providing materials to each of the SMEs who had agreed to participate, was carried out by mailing individual information packages10 that included the following documents: 1. SME introductory letter (see Appendix D) – the introductory letter provided the SMEs with an overview of the workplace challenges facing PWDs, a description of the purpose and value of this study, and an explanation of why the individual SMEs were selected (Waltz et al., 2010). Also included were a description of the content domain and a detailed summary of the tasks required of the SMEs, which included a rating of the relevance and clarity of each item to that domain (Grant & Davis, 1997). There is some debate over whether SMEs should be made aware of the supporting theory and the intended use of the scale prior to completing their assigned tasks. Sireci (1998b) raises a concern that if SMEs are told of the theoretical underpinnings or specific objectives of a test, a variety of response biases may be 10 During this period (May 1–August 31, 2016) I was living in the UK while attending the University of Leeds. 156 introduced. For example, if SMEs are told to “rate relevance of test items to the content areas they are intended to measure” (p. 301), a pre-existing knowledge of the purpose of the scale may influence the interpretation of intentionality. However, Waltz et al. (2010) recommend that SMEs be provided with information about the purpose of the study and a list of appropriate definitions. Lynn (1986) recommends that SMEs be provided with information that describes the content domain. Grant & Davis (1997) also support this position, stating that, as a means to avoid confusion by the SMEs over the conceptual basis for the test, they should be provided with information about “the conceptual underpinnings of the instrument” (p. 270). Based on my consideration of the arguments put forward above, I provided SMEs with an operationalized description of the construct and an overview of the intended purpose of this new scale. 2. SME informed consent form (see Appendix E) – the informed consent form provided a brief description of the study purpose and procedures, the inclusion criteria (including meeting at least one of the SME criteria), a working definition of disability11 (to assist SMEs in determining their inclusion in that category), the potential emotional risks associated with participation as an SME, the potential benefits to PWDs, and our commitment to confidentiality. Each SME was also advised that they could terminate their involvement in the study at any time. Finally, I indicated that each SME would receive a $20.00 (CDN) gift certificate from the Starbucks Coffee Company as compensation for contributing their time and expertise to this study. 11 Adapted from the definition developed by the Government of Yukon in 2006 for internal census purposes. 157 3. SME self-identification form (see Appendix F) – the self-identification form confirmed whether SME participation was on an individual basis or on behalf of an organization. If the latter, SMEs were asked to indicate whether approval from that organization was required12. Finally, SMEs were asked to indicate which of the three categories of expertise (trainer, workplace expert, PWD participant) were applicable. 4. SME rating form (see Appendix G) – the rating form provided an introductory section that included a review of the four aspects of employment to be considered, the responsibility to rate each item within these categories for both relevance and clarity, and instructions regarding the inclusion of additional items regarding aspects of employment on the part of PWDs that may be underrepresented (Beck & Gable, 2001). SME were also requested to offer comments on individual items, specific sections, or on the overall employment experience of PWDs. Also included in this section was a reminder that SMEs were rating items in terms of their applicability to the workplace attitudes of others toward PWDs, not whether they, the SMEs, agreed or disagreed with each of these statements. As recommended by Grant and Davis (1997), this key element of the SME task was highlighted to reinforce its significance. This reminder was highlighted by hand in each SME rating form with yellow felt marker. The body of the rating form was divided into five sections, one for each of the four employment subdimensions and a final section within which overall comments could be provided. Each of the sections provided a reiteration of the purpose and procedure of the rating task, with space provided for comments following each item; 12 Only one SME answered ‘yes’ to this question. However, a subsequent telephone conversation determined that this SME was unsure if this was necessary and agreed to investigate further and provide an appropriate confirmation if necessary. No further response was received. 158 these could include recommendations for additional items or to make editorial changes to existing items (Beck & Gable, 2001). In addition, the ‘Relevance’ and ‘Clarity’ headings, as well as the rating categories for each of these, were reproduced at the top of each page. As recommended by Rubio et al. (2003) rating scales for both of these headings were included for each item, allowing SMEs to evaluate each item completely at the same time. 5. Materials packages – the materials packages also included stamped, addressed envelopes to be returned upon completion to me at the School of Social Work at the University of British Columbia (UBC). Although, for reasons of confidentiality, all SMEs had been requested to mail their response information, I received two requests, based on disability-related needs, to forward the materials electronically and to accept electronic responses. Each of these SMEs acknowledged and accepted responsibility for any potential lapses in confidentiality based on these requests. I agreed to this change in procedure. I had originally advised all SMEs that they would receive their information packages within two weeks of confirming their participation and that all documents, including the SME Response Form, should be mailed to me within three weeks of receipt. However, several factors intervened with this original timetable. First, the postage stamps required for the return envelopes from Canada, the USA and Australia were not available for purchase from the UK postal service. This required me to order these stamps from the postal services of each of these countries, adding approximately two weeks to the delivery time of the information packages. Second, the mail delivery 159 periods to each of these countries varied considerably, with one package arriving in Canada within one week and several others to other Canadian locations taking up to three weeks. In addition, one package was either lost by the postal service or misplaced following receipt. As of July 26th, 16 packages had been received at UBC. I then sent an email reminder to each of the remaining SMEs, requesting that they complete the documents as soon as possible. One of these SMEs indicated that they had not received their package and I forwarded a replacement. As of August 16th, three additional packages had been received at UBC. I sent a second email reminder to those still outstanding. Two more packages were received by September 20th. At that point, I determined that the period for receiving SME responses was complete and so advised those who had not yet forwarded their materials. This process resulted in my receipt of a total of 21 responses from SMEs. The third step, instrument formation, includes an analysis of quantitative and narrative SME response data and applying these data to the original 40-item CADE Scale. The quantitative response data include ratings of relevance and clarity, while the narrative data include recommendations for rewording of individual items, suggestions for additional content coverage, and for the removal of several items that are considered to be irrelevant. The result of these analyses is a scale that is based on sufficient evidence of content validity to address the twin threats of construct underrepresentation and construct irrelevance (Messick, 1994). This requires that each of the several aspects of 160 the construct under consideration is sufficiently represented and that those that are irrelevant to the construct are excluded (Lynn, 1986; Waltz et al., 2010). Methodology I employed the Content Validity Index (CVI) as put forward by Lynn (1986) as a means to evaluate the quantitative data generated by the SMEs. The CVI is a commonly used measure of inter-rater agreement which indicates the proportion of SMEs that endorse an item as content valid (Hubley & Palepu, 2007; Polit, Beck, & Owen, 2007); and, when applied at a scale level, is used to determine whether the “total instrument [is] to be assessed as content valid” (Lynn, 1986, p. 383). The first of these, the measure of item-level agreement, is referred to as the I-CVI; the latter, the measure of scale level agreement, is referred to as the S-CVI. Since its development by Lynn, the CVI has been frequently applied in a range of disciplines, including sociology, psychology, occupational therapy, social work and others (Schilling et al., 2007). The CVI is a 4-point ordinal scale (from 1=not relevant, 2=somewhat relevant, 3=quite relevant, to 4=extremely relevant) used to determine SME ratings of individual items for both relevance and clarity (Lynn, 1986). No rationale is provided for the four-point scale, either by Lynn or by Waltz and Bausell (1981), who had earlier described the CVI as a practical solution to quantifying “the judgment of content specialists by using an index of interjudge agreement as a measure of item content validity” (p. 71). However, Lynn notes that a four-point scale “should provide the instrument developer with sufficiently delineated information” (p. 384) and is to be preferred over either a 161 three or five-point scale “because it does not include the ambivalent middle rating” (p. 384). As described by Lynn (1986), “the actual CVI is the proportion of items that received a rating of 3 or 4 by the experts” (p. 384). These ratings for each item are then divided by the number of SMEs to obtain the proportion of whom are in agreement, determined by those who selected either a three or four on the rating scale, about the relevance of the item (Polit et al., 2007). Lynn also provides a guideline for establishing a minimum acceptable level of agreement by SMEs. As first suggested by Waltz and Bausell (1981), and based on the standard error of proportion, Lynn calculated this index as a means to determine the number of SMEs required to agree on item relevance. If there are five or fewer SMEs, the item-level CVI must be 1.0; that is, all must agree that the item is content valid. If the number of SMEs is greater than five, the index determines acceptable levels of content validity “beyond the .05 level of significance” (p. 384). As the number of experts increases, this level of agreement can be less than universal but still exhibit sufficient evidence of content validity. For example, with 10 experts, a minimum of eight agreeing to the relevance of each item is necessary to achieve a sufficient level of content validity. Although the CVI indicates that an agreement level of .78 would be sufficient for 10 experts, agreement by seven SMEs would yield an index of .70; therefore, eight are required. However, not all researchers agree with this minimum level. Davis (1992) recommends that inter-rater agreement should be greater than .70 and, with new instruments, this should be greater than .80. 162 Lynn (1986) also specifies that a CVI rating should be calculated for the entire instrument determined “by the proportion of total items judged content valid” (p. 384). This scale-level requirement, or S-CVI, is supported by Grant & Davis (1997), who also note that SMEs should be asked to consider each item in terms of representativeness within specific dimensions, and to comment on the comprehensiveness of the entire set of items. Polit et al. (2007) consider two possible options in calculating the content validity of S-CVI scores. While recommending a minimum acceptability rating of .80, these authors describe both of these methods, which reveal considerably different results. The first, referred to as the universal agreement method, is based upon the proportion of items which are universally agreed upon by all of the SMEs. For example, if three SMEs all agreed that seven specific items were content valid (scored either three or four) but did not agree about three remaining items, the S-CVI would be .70 (seven out of ten items were found to be universally agreed upon). The second method, referred to as the average agreement method, calculates the S-CVI by determining the level of agreement for each item, adding these scores, and dividing by the number of items. Similar to the above example, if three SMEs all agreed that seven items were content valid (scored either three or four) but in each of the remaining three items, one SME disagreed (scored either a one or a two), the first seven items would be scored as 1.0 and the last three would be scored as .67. If all scores were to be added together and divided by the number of items, the S-CVI would 163 be .90 ([7 x 1.0] + [3 x .67]=.90). Waltz et al. (2005) recommend that, for new measures, the scale-level CVI should be a minimum of 90 percent. I have used the average agreement method for calculating the S-CVI for two reasons. First, an average score more clearly represents the sentiments of the group of SMEs by taking into account the actual scores for each item, providing additional evidence for a consideration of the relevance of each item. Second, with a large group of SMEs, as is the case in this study, the possibility of at least some degree of disagreement is increased (Lynn, 1986; Rubio et al., 2003). Therefore, even very highly scored items would not be able to meet the test of universal agreement. The CVI has been criticized for its failure to account for chance agreement, resulting in “an inflated estimate of content validity” (Beckstead, 2009, p. 1274). That author recommends using an alternative index based on Cohen’s kappa statistic, and also recommends Fleiss’ kappa (Fleiss, 1971, as cited in Beckstead, 2009), both of which are intended to account for chance agreement. However, Beckstead (2009) also acknowledged that, regardless of the index employed, obtaining the opinions of SMEs is always a means to arrive only at an estimate of the relevance of any particular item or items, acknowledging that the quality of any such estimate “is a function of the number of experts sampled” (p. 1280). In addition, Polit et al. (2007) note that as the number of SMEs increases, the probability of chance agreements and disagreements correspondingly decreases. Lynn (1986) acknowledges the possibility of chance agreement within the CVI scale, but maintains that this is addressed by “the selection of the number of experts 164 and the minimum number that must agree” (p. 384). Polit et al. (2007) also state that the principal weakness of the CVI is its failure to account for chance agreement among the SMEs. These authors computed a modified kappa statistic that evaluates each item score for chance agreement based on the number of SMEs and the proportion of agreement, arguing that for 16-member groups of SMEs, item-level agreement levels of .78 coupled with scale level CVI scores of .90 will result in “excellent content validity” (p. 467). These authors also maintain that the issue of chance agreement can be managed by adhering to the levels of agreement required by various numbers of SMEs as described by Lynn (1986). In addition, in selecting between these options, Polit et al. (2007) argue that the CVI index has several advantages. These include its focus on consensus rather than consistency, ease of calculation, and particularly its “focus on agreement of relevancy, rather than agreement per se” (p. 459, italics in original). This argument is supported by L. Russell (2013), who argues in favour of the CVI because of its focus on SME endorsement of the individual items rather than agreement. Russell notes that ”many measures of overall inter-rater agreement … do not distinguish between high agreement that an item is relevant and high agreement that an item is not relevant” (p. 30). It is this endorsement, or lack thereof, “which is the information needed to make decisions about revising the instrument content” (p. 31). The CVI has also been criticized for its collapsing of the four-point scale made available to SMEs for rating purposes into a two-point scale for purposes of calculating the resulting levels of agreement. Beckstead (2009) argues that this technique causes available information to be lost and artificially inflates levels of agreement among SMEs; 165 it not only “distort(s) the proportion of judgments which agree exactly, but we change the probability of chance agreement” (p. 1280). This author also points out the high risk of sampling error, based on small sample sizes of SMEs who are typically non-randomly selected (Lynn, 1986), adds to the challenges associated with interpreting SME responses as being representative of the larger expert population. L. Russell (2013) has responded to each of these two critiques. The first, that of collapsing the four-point scale, is considered within the context of how it is used throughout the process. When presented to SMEs, the purpose of the scale, as noted by Beckstead (2009) is to provide these experts with an opportunity to transmit information to the researcher. At this point in the process, providing a dichotomous scale would risk limiting SME responses to extreme opinions, with no opportunity to express more moderate views. However, when scale data are subsequently analyzed to determine relevancy of particular items, that purpose has changed (L. Russell, 2013). That purpose becomes how to determine whether the SME ratings provide evidence for validity and whether various items should be removed or adjusted. The question is no longer ‘how relevant (clear, etc.) is this item?’, but rather ‘is this item relevant (clear, etc.) enough for the purposes of content validation?’ These are judgments that rest with the researcher(s) conducting the content validation study, rather than the experts who provided the ratings (L. Russell, 2013, p. 36, italics in original). With regard to the representativeness of the SMEs, L. Russell (2013) argues that this may not be the most useful function of this cohort. Rather, SMEs may serve as an 166 advisory group “who have expertise that is relevant to the construct of interest … [and] who are not likely to be representative of the ‘population’ of experts” (p. 40, italics in original). This non-representative group may provide a more broadly based range of insights. L. Russell also argues that this advisory group should be specifically requested to provide descriptive information regarding item ratings, recommendations for revisions, and suggestions for additional items. Although often viewed as optional (DeVellis, 2017), or with little guidance regarding the manner in which these data have been employed (Halliday, Porock, Arthur, Manderson, & Wilcock, 2012; Haynes et al., 1995; cf. Hubley & Palepu, 2007), this narrative feedback will provide explanations for “why they [items] are not endorsed, and not only which elements would benefit from revision, but also what revisions to make” (L. Russell, 2013, p. 43). Data Analysis The SME cohort. My original intent within this study was to calculate the combined response data from SMEs – trainers, workplace experts, and PWD participants – to determine I-CVI ratings and S-CVI ratings within a single cohort. Many studies that report on the recruitment of SMEs provide guidance and recommendations regarding the numbers of SMEs and general qualifications (see, for example, Hubley & Palepu, 2007; Lynn, 1986; Waltz et al., 2010); the use of SMEs who may provide different categories of professional expertise, including contact with the instrument’s target population, to considerations of complex constructs (Davis, 1992; L. Russell, 2013); and the recruitment of SMEs from different geographic locations as a means to 167 address idiomatic language that may be associated with a particular construct (Grant & Davis, 1997). As noted above, some writers assert that it is useful to involve members of the target population in the content validation process (Grant & Davis, 1997; L. Russell, 2013; Schilling et al., 2007). These experiential experts (EEs) can be of assistance in providing additional insights into the conceptualization of the target construct, clarification of terminology, and idiomatic language use (L. Russell, 2013). In addition, some researchers have categorized SMEs and EEs differently for scoring purposes within a CVI context, thereby allowing for a comparative analysis between these two groups (see, for example, L. Russell, 2013; Schilling et al., 2007). However, others have considered SME scores as a whole, particularly when large numbers of SMEs have been employed. Chien and Norman (2004), in a content validity study employing 15 SMEs and five EEs, combined all 20 scores when calculating CVI scores. Similarly, Rubio et al. (2003) recruited six SMEs and six EEs to participate in a content validity study of a caregiver well-being scale. Of the six SMEs, five were in academia and one worked with caregivers. All of the response data were analyzed together, although the authors noted that the results may have been different if the data had been “distinguish[ed] between lay and professional experts” (p. 102). In that study, no mention was made with regard to separating the professional experts into subcategories. Similarly, other researchers employing a minimum of 10 SMEs with differing areas of related expertise have calculated CVI scores collectively, without 168 separating SMEs into subcategories (see, for example, Chen, Horner, & Percy, 2003; Zamanzadeh et al., 2015). As shown below in Table 4.1, the information provided by SMEs in the Self-identification Form revealed that five of the SMEs indicated expertise in all three of the specified categories and six indicated expertise in two of these categories. In addition, eight of the 21 SMEs indicated that they were PWDs. Although not anticipated when I first recruited this cohort of SMEs, having this number of SMEs with disabilities presented an opportunity for a comparative analysis of responses provided by PWDs and those who did not identify as PWDs (non-PWDs). Therefore, all of the I-CVI and S-CVI calculations were conducted to provide separate subtotals for PWDs and non-PWDs, as well as overall total scores. Table 4.1: Subject Matter Expert Demographic Information # Country Response Received Trainer PWD Experience Workplace Expert 1. USA Aug. 18/16 X X X 2. USA July 26/16 X X X 3. Canada June 20/16 - - X 4. Canada June 20/16 - X X 5. Canada July 22/16 - - X 6. USA June 21/16 - - X 7. Canada June 23/16 X X X 8. USA June 30/16 X - - 9. USA June 28/16 X X X 10. Canada July 24/16 X - X 11. Canada July 26/16 - - X 12. Canada July 6/16 X - X 13. Canada July 17/16 X - - 14. Canada July 4/16 - X X 15. Canada July 26/16 - X X 16. Canada July 27/16 X X X 17. UK July 29/16 - - X 18. UK July 14/16 X - X 19. UK July 7/16 - - X 20. UK July 1/16 - - X 169 # Country Response Received Trainer PWD Experience Workplace Expert 21. Australia Sept. 20/16 X - X Subtotals 11 8 19 Of these 21 SMEs, four were female and 17 were male; eight SMEs had PhDs, two had LLBs, six had Master’s degrees, and three had undergraduate degrees. The education level of the remaining three SMEs is not known. Removal of outlier data. Written instructions provided to SMEs confirmed the requirement that the SME response forms were to be completed following a consideration of each of the items for their relevance to the construct. Responses were not to be based on whether SMEs agreed with the specific statements. This admonition was similar to that recommended by Schilling et al. (2007). However, three of the SMEs (#s 2, 12 & 17) provided repeated comments regarding how various items should not be necessary or were inappropriate because respondents should have a greater understanding of PWDs13. The ratings of items for relevance provided by these three SMEs (M=13.67, SD=1.53) were also much below those provided by the other 18 SMEs (M=34.8, SD=4.2). However, these differences were not apparent when rating items for clarity. In that case, the scores of these three SMEs (M=39.33, SD=1.15) were slightly higher than those of the other 18 SMEs (M=36.56, SD=3.43). As recommended by Waltz et al. (2010), data from individual SMEs should be eliminated if score differences are likely attributable to a lack of knowledge or misinterpretation on the part of the SME. Although no specific recommendations regarding the degree of discrepancy are provided, the removal of outlier data is also 13 Although SME comments are shown in Appendix I, I have not grouped these comments by SME in order to retain individual confidentiality. 170 recommended by many other researchers (see, for example, Haynes et al., 1995; Hubley & Palepu, 2007; Lynn, 1986). Based on the narrative feedback and the scores of these three SMEs, I determined that these responses should be considered as outlier data and removed from CVI calculations. As shown in the four tables below, (Table 4.3, Table 4.4, Table 4.5, Table 4.6), subsequent calculations of SME scores were conducted following the removal of these outlier scores. SME response data. Although the seven PWDs’ quantitative response data were initially analyzed separately from those provided by the non-PWDs, the I-CVI weighted averages calculated for each item were identical to those calculated for the overall averages of the total number of SMEs for relevance (see Table 4.3 and Table 4.4) and for clarity (see Table 4.5 and Table 4.6). However, the two groups of SMEs did not score each item identically. Of the 20 highest scoring items for relevance, average scores revealed an S-CVI of .98 for non-PWDs and an S-CVI score of .96 for PWDs. When the final 20 items were selected for inclusion in the scale, the S-CVI scores dropped slightly to an S-CVI score of .94 for non-PWDs and .90 for PWDs. Of the items selected for the 20-item version of the CADE Scale, the average score for each was between the acceptable lower limit of .78 (14 SMEs) and 1.0 (18 SMEs). However, as shown below in Table 4.7, this lower limit was not achieved for every item by both subgroups of SMEs. Rationale for item selection. Thirty-three of the original 40 items produced relevance scores that were equal to or above the minimum recommended CVI score of 171 .78 (Lynn, 1986) when the SMEs were considered as one cohort. When separated into two cohorts, PWDs generated relevance ratings of .78 or greater for 26 items and non-PWDs generated 34 items of .78 or greater (see Tables 4.4 and 4.5). Based on this input, the narrative feedback provided by the SMEs (McKenzie, Wood, Kotecki, Clark, & Brey, 1999; L. Russell, 2013), and my own professional experience (Beck & Gable, 2001; Waltz et al., 2005), I selected five items for inclusion within each of the four sub-categories as a means to support construct representativeness (Lynn, 1986; Polit et al., 2007). This final selection also included a review of each item to confirm that it continued to represent the Difference Model of disability. As noted above, all of the initial 40 items had been drafted to represent different aspects of the Difference Model of disability as a means to generate evidence of ontological validity. Following the redrafting or rewording of several items based on narrative input from SMEs (see Table 4.9), this final review was intended to ensure that the ontological relationship between each of the final 20 items and the Difference Model had not been lost. Specifically, I reviewed each item to ensure that it presented a clear comparison or distinction between PWDs and others in the workplace. In addition, I reviewed these items to confirm that there was a relatively equal representation of cognitive, affective or behavioural aspects of attitude within each of the four categories of the construct. Details of these response categories are shown below in Table 4.2. 172 Table 4.2: Co-worker Response Categories Category Cognitive Affective Behavioural Totals Hiring 3 2,4 1,5 5 Performance 10 6,7,9 8 5 Accommodations 14,15 12 11,13 5 Supervision/Termination 17,20 16 18,19 5 Totals 6 7 7 20 173 Table 4.3: Forty-item Relevance Scores (Part 1 - Hiring & Performance) SME Category: E – PWD participant T – Trainer W – Organizational experience Outliers Removed (#2, 12 & 17) Section Hiring (Items 1-10) Performance (Items 11-20) SME Cat. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Subtotal 1. E/T/W 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 9 2. E/T/W 3. E/T/W 0 0 1 1 0 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 16 4. E/T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 5. E/T/W 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 18 6. E/W 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 19 7. E/W 1 1 1 0 1 1 1 1 1 1 0 1 1 1 0 0 1 1 1 0 15 8. E/W 1 1 1 1 0 0 1 0 1 1 1 1 1 1 0 1 1 1 1 1 16 Subtotal 5 4 5 4 4 5 6 5 6 6 4 7 7 7 5 6 7 7 7 6 9. T/W 1 1 1 0 0 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 16 10. T/W 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 1 0 0 13 11. T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 12. T 13. T 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 14. W 1 1 0 1 0 1 1 0 1 1 0 1 1 0 0 1 1 1 1 0 13 15. W 1 1 1 0 1 1 1 0 1 1 0 1 1 1 0 1 1 1 0 1 15 16. W 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 19 17. W 18. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 19. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 19 20. W 0 0 1 1 0 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 16 21. W 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 18 Subtotal 10 10 10 9 7 11 11 8 10 11 7 11 11 8 7 9 11 10 9 9 TOTAL 15 14 15 13 11 16 17 13 16 17 11 18 18 15 12 15 18 17 16 15 Percentages % % % % % % % % % % % % % % % % % % % % PWDs 71 57 71 57 57 71 86 71 86 86 57 100 100 100 71 86 100 100 100 86 non-PWDs 91 91 91 82 64 100 100 73 91 100 64 100 100 73 64 82 100 91 82 82 Weighted Ave* 83 78 83 72 61 89 94 72 89 94 61 100 100 83 67 83 100 94 89 83 Overall Ave. 83 78 83 72 61 89 94 72 89 94 61 100 100 83 67 83 100 94 89 83 *Formula for Weighted Average = [.39 x (PWD%)] + [.61 x (non-PWD%)] 174 Table 4.4: Forty-item Relevance Scores (Part 2 - Accommodations & Supervision/Dismissal) SME Category: E – PWD participant T – Trainer W – Organizational experience Outliers Removed (#2, 12 & 17) Accommodations (Items 21-30) Supervision/Termination (Items 31-40) SME (Sub) 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Total 1. (9) 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 1 1 1 1 26 2. 3. (16) 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 0 1 33 4. (20) 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 39 5. (18) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 38 6. (19) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 38 7. (15) 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 33 8. (16) 1 0 0 1 0 1 1 1 0 1 0 1 0 1 1 1 1 1 1 0 29 Subtotal 7 5 6 7 6 7 7 7 3 7 6 7 6 5 5 7 7 7 6 5 9. (16) 1 0 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1 0 1 32 10. (13) 1 1 0 1 1 1 0 0 1 1 0 1 1 1 1 1 1 1 1 0 28 11. (20) 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 38 12. 13. (20) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 40 14. (13) 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 32 15. (15) 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 34 16. (19) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 39 17. 18. (20) 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 38 19. (19) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 39 20. (16) 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 35 21. (18) 1 0 1 1 0 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 35 Subtotal 11 5 10 11 10 11 9 10 9 11 9 11 10 10 11 11 11 11 10 11 TOTAL 18 10 16 18 16 18 16 17 12 18 15 18 16 15 16 18 18 18 16 16 Percentages % % % % % % % % % % % % % % % % % % % % PWDs 100 71 86 100 86 100 100 100 43 100 86 100 86 71 71 100 100 100 86 71 non-PWDs 100 45 91 100 91 100 82 91 82 100 82 100 91 91 100 100 100 100 91 100 Wted Ave.* 100 55 89 100 89 100 89 94 67 100 83 100 89 83 89 100 100 100 89 89 Overall Ave. 100 56 89 100 89 100 89 94 67 100 83 100 89 83 89 100 100 100 89 89 *Formula for Weighted Average = [.39 x (PWD%)] + [.61 x (non-PWD%)] 175 Table 4.5: Forty-item Clarity Scores (Part 1 – Hiring & Performance) SME Categories: E – PWD participant T – Trainer W – Organizational Experience Outliers removed (#2, 12 & 17) Section Hiring (Items 1-10) Performance (Items 11-20) SME Cat. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Subtotal 1. E/T/W 0 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 17 2. E/T/W 3. E/T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 4. E/T/W 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 18 5. E/T/W 1 0 1 1 1 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 16 6. E/W 1 1 1 1 0 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 17 7. E/W 1 0 1 0 1 0 0 1 1 1 0 1 1 1 1 0 1 1 1 0 13 8. E/W 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 18 Subtotal 6 5 7 5 6 5 6 5 4 7 6 6 7 5 7 6 6 7 7 6 9. T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 10. T/W 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 18 11. T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 12. T 13. T 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 14. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 19 15. W 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 19 16. W 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 17 17. W 18. W 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 19 19. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 19 20. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 21. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 Subtotal 10 11 10 11 11 9 11 10 10 11 11 11 11 11 10 11 10 11 11 10 TOTAL 16 16 17 16 17 14 17 15 14 18 17 17 18 16 17 17 16 18 18 16 Percentages % % % % % % % % % % % % % % % % % % % % PWDs 86 71 100 71 86 71 86 71 57 100 86 86 100 71 100 86 86 100 100 86 non-PWDs 91 100 91 100 100 82 100 91 91 100 100 100 100 100 91 100 91 100 100 91 Weighted Ave* 89 89 94 89 94 78 94 83 78 100 94 94 100 89 94 94 89 100 100 89 Overall Ave. 89 89 94 89 94 78 94 83 78 100 94 94 100 89 94 94 89 100 100 89 *Formula for Weighted Average = [.39 x (PWD%)] + [.61 x (non-PWD%)] 176 Table 4.6: Forty-item Clarity Scores (Part 2 – Accommodations & Supervision/Dismissal) SME Categories: E – PWD participant T – Trainer W – Organizational Experience Outliers removed (#2, 12 & 17) Accommodations (Items 21-30) Supervision/Dismissal (Items 31-40) SME (Sub.) 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Total 1. (17) 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 35 2. 3. (20) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 40 4. (18) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 38 5. (16) 0 1 1 1 0 0 0 1 1 1 1 1 0 1 0 1 1 1 1 1 30 6. (17) 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 36 7. (13) 1 1 1 0 0 1 1 1 0 1 0 1 1 0 0 1 1 1 1 1 27 8. (18) 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 36 Subtotal 6 7 7 6 5 6 6 7 5 7 3 6 6 6 5 7 7 7 7 7 35 9. (20) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 40 10. (18) 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 35 11. (20) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 40 12. 13. (19) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 40 14. (19) 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 37 15. (19) 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 38 16. (17) 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 36 17. 18. (19) 0 1 1 1 1 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 36 19. (20) 1 1 1 1 0 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 37 20. (20) 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 38 21. (20) 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 39 Subtotal 6 10 11 11 10 10 11 11 9 10 8 11 11 10 11 11 11 11 11 11 Total 12 17 18 17 15 16 17 18 14 17 11 17 17 16 16 18 18 18 18 18 Percentages % % % % % % % % % % % % % % % % % % % % PWDs 86 100 100 86 71 86 86 100 71 100 43 86 86 86 71 100 100 100 100 100 non-PWDs 55 91 100 100 91 91 100 100 82 91 73 100 100 91 100 100 100 100 100 100 Weighted Ave* 67 94 100 94 83 89 94 100 78 94 61 94 94 89 89 100 100 100 100 100 Overall Ave. 67 94 100 94 83 89 94 100 78 94 61 94 94 89 89 100 100 100 100 100 *Formula for Weighted Average = [.39 x (PWD%)] + [.61 x (non-PWD%)] 177 Table 4.7: SME Relevance Scores (20 item scale) SME Categories: E – PWD participant T – Trainer W – Organizational Experience Outliers removed (#2, 12 & 17) Section Hiring Performance Accommodations Supervision/Dismissal SME Cat. H1 H2 H3 H4 H5 P6 P7 P8 P9 P10 A11 A12 A13 A14 A15 S16 S17 S18 S19 S20 Total 1. E/T/W 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 14 2. E/T/W - - - - - - - - - - - - - - - - - - - - - 3. E/T/W 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 16 4. E/T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 5. E/T/W 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 19 6. E/W 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 18 7. E/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 8. E/W 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 1 1 1 1 18 Subtotal 5 4 5 6 6 7 7 7 7 7 7 6 7 7 5 5 7 7 7 6 9. T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 19 10. T/W 1 1 1 0 1 1 1 1 0 1 1 0 1 0 0 1 1 1 1 1 15 11. T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 12. T - - - - - - - - - - - - - - - - - - - - - 13. T 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 14. W 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 19 15. W 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 19 16. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 17. W - - - - - - - - - - - - - - - - - - - - - 18. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 19. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 20. W 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 18 21. W 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 19 Subtotal 10 10 10 10 11 11 11 10 9 11 11 10 11 10 10 11 11 11 11 10 TOTAL 15 14 15 16 17 18 18 17 16 18 18 16 18 17 15 16 18 18 18 16 Percentages % % % % % % % % % % % % % % % % % % % % Average PWDs 71 57 71 86 86 100 100 100 100 100 100 86 100 100 71 71 100 100 100 86 89 non-PWDs 91 91 91 91 100 100 100 91 82 100 100 91 100 91 91 100 100 100 100 91 95 Weighted Ave* 83 78 83 89 94 100 100 94 89 100 100 89 100 94 83 89 100 100 100 89 93 Overall Ave. 83 78 83 89 94 100 100 94 89 100 100 89 100 94 83 89 100 100 100 89 93 *Formula for Weighted Average = [.39 x (PWD%)] + [.61 x (non-PWD%)] 178 Table 4.8: SME Clarity Scores (20 item scale) SME Categories: E – PWD participant T – Trainer W – Organizational Experience Outliers removed (#2, 12 & 17) Section Hiring Performance Accommodations Supervision/Dismissal SME Cat. H1 H2 H3 H4 H5 P6 P7 P8 P9 P10 A11 A12 A13 A14 A15 S16 S17 S18 S19 S20 Total 1. E/T/W 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 18 2. E/T/W - - - - - - - - - - - - - - - - - - - - - 3. E/T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 4. E/T/W 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 19 5. E/T/W 1 0 1 0 1 1 1 1 0 1 0 1 0 1 1 0 1 1 1 1 14 6. E/W 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 18 7. E/W 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 17 8. E/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 Subtotal 6 5 7 4 7 6 6 7 5 7 6 7 6 7 7 5 7 7 7 7 9. T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 10. T/W 1 1 1 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 17 11. T/W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 12. T - - - - - - - - - - - - - - - - - - - - - 13. T 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 14. W 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 19 15. W 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 19 16. W 0 1 0 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 17 17. W - - - - - - - - - - - - - - - - - - - - - 18. W 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 19 19. W 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 19 20. W 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 19 21. W 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 Subtotal 10 11 10 10 11 11 11 11 11 10 6 10 10 11 11 11 11 11 11 11 TOTAL 16 16 17 14 18 17 17 18 16 17 12 17 16 18 18 16 18 18 18 18 Percentages % % % % % % % % % % % % % % % % % % % % Ave. PWDs 86 71 100 57 100 86 86 100 71 100 86 100 86 100 100 71 100 100 100 100 90 non-PWDs 91 100 91 91 100 100 100 100 100 91 55 91 91 100 100 100 100 100 100 100 95 Weighted Ave* 91 89 94 78 100 94 94 100 89 94 67 94 89 100 100 89 100 100 100 100 93 Overall Ave. 89 89 94 78 100 94 94 100 89 94 67 94 89 100 100 89 100 100 100 100 93 *Formula for Weighted Average = [.39 x (PWD%)] + [.61 x (non-PWD%)] 179 Table 4.9: SME Scores and Comments *Outliers removed (Total=18; PWDs=7; non-PWDs=11) 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 1. (H1) 83 89 It would be fair if disabled job seekers were required to disclose the likely progression of their disability as part of the hiring process. (H1) Disabled applicants should provide job-related details of their disability during the interview process. (H1) SMEs: -Replace ‘the likely progression’ with a more clear phrase. -Unclear whether question is intended to refer to the individual’s circumstance or the general condition. Breen: -Rewritten to be more clear, focused on employment-related circumstance and framed as an item that reflects a specific behaviour. -Retained. 2. (H2) 78 89 When hiring disabled employees, consideration should be given to how well they will be accepted by other members of the organization. (H2) When hiring disabled employees, consideration should be given to how well their disability will be accepted by co-workers. (H2) SMEs: -The item should focus on the degree of acceptance associated with the disability, not the other characteristics of the person. Breen: -Although this was the only retained item with a score below .80, it addresses a concern recognized by many SMEs. It also scored at the minimum recommended cut-off point. -Retained. 3. (H3) 83 94 Job candidates with disabilities are less likely than others to have all of the skills necessary for technical positions. (H3) Applicants with disabilities are as likely as other candidates to have all of the required job skills. (H3) SMEs: -Too narrowly focused on technical skills/positions. -Goes to the core of others’ beliefs about PWDs. -Too ambiguous with regard to what constitutes a technical position. Breen: -Rewritten to be more generally applicable and to be presented as a positive comparison with others. This item will be reverse scored. -Retained. 180 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 4. (H4) 72 89 A job seeker who indicates on an application form that he has a disability should include the medical diagnosis. (H4) SMEs: -Disclosure is already an option for PWDs. -Mandatory disclosure transgresses human rights legislation. -Does not address performance restrictions. Breen: -Removed. 5. (H5) 61 94 Job candidates with disabilities are more likely than others to exaggerate their skills in an interview. (H5) SMEs: -Unclear wording of the statement. -Not sure of the purpose of this item. Breen: -Removed. 6. (H6) 89 78 An applicant for a job that is similar to yours has advised that she requires a disability-related accommodation for her interview. This could also indicate that she may not be able to satisfactorily perform all aspects of this job. (H6) SMEs: -The phrase ‘disability-relate accommodation’ may be unfamiliar to some. -It is not clear how this relates to ‘acceptance’. Breen: -The conditional aspect of the item makes it potentially confusing. -Removed. 7. (H7) 94 94 Job candidates with disabilities are less likely than others to have all of the skills necessary for physically demanding positions. (H7) SMEs: -Many disabilities are unrelated to the physical demands of particular jobs. -The item is too general. Breen: -Also repetitive (see H3). -Removed. 8. (H8) 72 83 At a technology-oriented customer service desk, a “tech” with a visible disability would instill less confidence than one who was able-bodied. (H8) SMEs: -Too specific to particular forms of disability and to particular jobs. Breen: -Removed. 181 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 9. (H9) 89 78 I would be comfortable introducing a new disabled employee to my work group. (H9) I would be less comfortable welcoming a new employee into my work group if they had a disability. (H4) SMEs: -This should be restated to make the comfort level more dependent on whether the person has a disability. -The item should differentiate between a general discomfort in introducing a new employee and one in which the new employee has a disability. Breen: -Rewritten to create a more clear relationship between the comfort level of the person introducing the new employee and the fact that the new employee has a disability. -Retained. 10. (H10) 94 100 Job candidates with disabilities are less likely than others to have the emotional stability that is necessary to succeed at very stressful jobs. (H10) Job candidates with disabilities are less likely than others to have the emotional stability necessary to succeed at very stressful jobs. (H5) SMEs: -Very strong link to perceptions/attitudes/beliefs. -This assumption can affect the hiring process. Breen: -Slightly shortened for clarity. -Retained. 11. (P1) 61 94 For purposes of receiving annual cash bonuses, employees with disabilities should be rated on the effort that they put in instead of how well they perform. (P1) SMEs: - Not all organizations provide cash bonuses; this item may be inappropriate for some respondents. Breen: -Removed. 12. (P2) 100 94 I would be concerned that working with disabled colleagues as part of my team could negatively affect my own performance evaluation. (P2) I would be concerned that having a disabled person as part of my work unit could negatively affect my own performance evaluation. (P1) SMEs: -This is the core issue of discrimination. -Change ‘working’ to ‘having’ for a more broadly-based concept. Breen: - Adjusted for greater clarity of the workplace relationship between the PWD and others. -Retained. 13. (P3) 100 100 Having a person with a disability as a member of my team would add extra burdens to my job. (P3) SMEs: -Very relevant. A common misperception. -Too similar to #12 (P2). Breen: -Removed. 182 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 14. (P4) 83 89 I would be less comfortable reporting to a supervisor with a visible disability than to someone without a disability. (P4) SMEs: -Seen as relevant but confusing regarding areas of discomfort. Breen: -Removed. 15. (P5) 67 94 I would be less likely to become friends outside of our workplace with a disabled colleague than with others. (P5) SMEs: -Too unrelated to the workplace. -There may be legitimate reasons. Breen: -Removed. 16. (P6) 83 94 Working with disabled co-workers would distract me from my own responsibilities. (P6) SMEs: -Understood by SMEs as a common sentiment. Breen: -Similar to P7 and P8 below. Removed. 17. (P7) 100 89 The inclusion of disabled employees can be detrimental to a project team. (P7) The inclusion of disabled employees can be detrimental to a project team. (P2) SMEs: -Provides an opportunity for a response that is framed within a concern for other team members while still presenting an opportunity to voice any concerns. Breen: -Retained. 18. (P8) 94 100 Compared to others, disabled employees are more likely to miss work due to illness. (P8) Employees with disabilities are more likely than others to miss work. (P3) SMEs: -A widely held belief. -Would be more effective without the qualifier of ‘due to illness’. Breen: -Rewritten for clarity and to remove the limitation of absence being associated with illness. -Retained. 19. (P9) 89 100 I would be more comfortable working with co-workers who did not have disabilities. (P9) I would be more comfortable working with others who did not have disabilities. (P4) SMEs: -Relevant. Breen: -Wording change from ‘co-workers’ to ‘others’ to broaden applicability in the event that co-workers are distinguished from supervisors by some respondents -Retained. 183 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 20. (P10) 83 89 I would be concerned about a disabled co-worker’s ability to perform my job while I was on an extended vacation. (P10) SMEs: -Provide a more broadly based context, such as ‘away from work’. -Does not separate disability from specific individual skills unrelated to disability. Breen: -Too unclear. -Removed. 21. (A1) 100 67 If necessary, disabled employees should be entitled to have peripheral responsibilities within their job descriptions modified, even if this requires corresponding changes in others’ duties. (A1) If medically necessary, employees with disabilities should be entitled to have their responsibilities modified, even if this means changing co-workers’ duties. (A1) SMEs: -Remove ‘peripheral’. -This language is too complex and unclear. Breen: -Rewritten for clarity. -Additional focus on “medical necessity” to indicate that modifications to responsibilities have been evaluated and deemed necessary and appropriate by a healthcare professional. Procedural and equipment/built environment accommodations within the workplace most frequently require formal approval prior to implementation -Retained. 22. (A2) 56 94 Employees with certain disabilities should be allowed to wear headphones to listen to music at work if this helps them concentrate. (A2) SMEs: -Too specific. -May also apply to others. Breen: -Removed. 23. (A3) 89 100 I would have to be careful about my language so as not to inadvertently offend a disabled co-worker. (A3) I would need to be careful about my language so I did not accidentally offend a disabled co-worker. (A2) SMEs: -A common belief. -Have had personal experience with co-worker discomfort over language use. Breen: -Slightly simplified. -Retained. 184 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 24. (A4) 100 94 Disabled employees typically expect some degree of help in performing their day-to-day tasks. (A4) Employees with disabilities typically expect some help in performing their day-to-day tasks. (P5) SMEs: -Addresses a commonly held belief. -Item should be moved to ‘Performance’ section. Breen: -Item re-labelled as P5. -Retained. 25. (A5) 89 83 It would be fair for an employee who requires an ergonomic chair to go to the top of our workgroup waiting list, even if others have been waiting longer for new chairs. (A5) SMEs: -Too specific to a particular job and disability. -Unclear. Most office chairs now marketed as ‘ergonomic’. Breen: -Too specific. -Removed. 26. (A6) 100 89 I would be unhappy about being requested by a supervisor to assist a disabled colleague with performing a brief, daily task. (A6) I would be unhappy about being requested by a supervisor to assist a disabled co-worker with performing a brief, daily task. (A3) SMEs: -Addresses a common co-worker concern over tasks outside of their job description and comfort level. Breen: -‘Co-worker’ substituted for ‘colleague’ to remove a possible connotation of an intervening personal relationship. -Retained. 27. (A7) 89 94 My employer should make training available to managers and other employees to help us understand disability-related challenges within the workplace. (A7) SMEs: -The purpose of the item is unclear. -Is this ‘available’ or ‘mandatory’? Breen: -Would not be applicable or available to all co-workers. - Removed. 28. (A8) 94 100 Supervisors should not have to be burdened with additional responsibilities associated with disabled employees. (A8) Supervisors should not have to accept additional responsibilities to manage disabled employees. (A4) SMEs: -Raises commonly held belief that PWDs create extra work for others. Breen: -Rewritten to focus on individual responsibility and behaviour of the supervisor. -Retained. 185 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 29. (A9) 67 78 The need to include disabled employees in workplace-sponsored social events limits opportunities for others. (A9) SMEs: -Too unclear, particularly with regard to ‘opportunities’. Breen: -Removed. 30. (A10) 100 94 If people with invisible disabilities require any special treatment in the workplace, they should explain their specific disability to co-workers. (A10) SMEs: -Relevant; however, reference to ‘invisible disabilities’ may be either confusing or limiting. Breen: -Similar disclosure issue as in H1. -Removed. 31. (S/T1) 83 61 People with disabilities are entitled to have different rules about their working hours if they p medical reasons. (S/T1)* SMEs: -Raises issue of accommodations rather than supervision. Breen: *Typographical error in original document. Should read “provide”. Pointed out by several SMEs. -Removed. 32. (S/T2) 100 94 I would feel more uncomfortable dismissing a disabled employee for poor performance than I would about another employee. (S/T2) SMEs: -A common concern, resulting in ongoing poor performance by a PWD. Breen: -Similar to #37 below. -Removed. 33. (S/T3) 89 94 If criticized for poor performance, disabled employees are more likely than others to blame external circumstances. (S/T3) SMEs: -This item is making assumptions about PWDs, rather than co-worker/supervisors. Breen: -Removed. 34. (S/T4) 83 89 Disabled employees should always be given additional opportunities to correct inappropriate behaviour. (S/T4) SMEs: -This item does not address whether the behaviour is connected to a disability. -Responses may vary based on disability category. -This would be the same for everyone, unless the intent is to provide extra opportunities for PWDs. Breen: -Somewhat similar in sentiment to #39 below. -Removed. 186 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 35. (S/T5) 89 89 The supervision of disabled employees, as compared to others, can be more difficult because of the potential for human rights claims. (S/T5) The supervision of disabled employees can be more difficult because of possible claims of discrimination. (S/T1) SMEs: -Language requires clarification. -‘Claims’ should be ‘complaints’. Breen: -Reworded to make the language more applicable to different jurisdictions and less connected specifically to formal or legislated complaint processes. -Retained. 36. (S/T6) 100 100 As a supervisor, I would not know how to respond to a disabled employee who blamed a poor performance review on her disability. (S/T6) As a supervisor, I would be particularly uncomfortable following up with an employee who blamed a poor performance review on their disability. (S/T2) SMEs: -Concern over gender bias. Breen: -Reworded to now acceptable use of ‘their’ as an individual, non-gender-based possessive. -Changed ‘would not know’ to ‘particularly uncomfortable’ to indicate a discrepancy between PWDs and others and also to provide more of a balance of affect-based items within the final 20-item scale. -Retained. 37. (S/T7) 100 100 I would feel more uncomfortable about having to discipline a disabled employee than I would about another employee. (S/T7) It would be harder for me to dismiss a disabled employee for poor performance than it would to dismiss another employee. (S/T3) SMEs: -A common feeling among supervisors. Breen: -Rewritten to provide a more clear example of the action required of the supervisor. -Retained. 38. (S/T8) 100 100 Supervisors may have to neglect other workers in order to meet the needs of disabled employees. (S/T8) Supervisors may have to spend less time with other workers in order to meet the needs of employees with disabilities. (S/T4) SMEs: -Reword to replace ‘neglect’ with ‘spend less time’ to generate more honest responses. Breen: -Rewritten as suggested to create a more objective, behavioral circumstance than the original item.. -Retained. 39. (S/T9) 89 100 Small performance failures on the part of disabled employees should be overlooked if they appear to be making an effort. (S/T9) Small performance failures by disabled employees should be overlooked if they appear to be making an effort. (S/T5) SMEs: -Represents the low expectations often attributed to PWDs. Breen: -Retained. 187 40 Item # 40 Item Relevance Score* 40 Item Clarity Score* Wording in 40-item form Final Wording in 20-item form Summary of SME Comments/Breen Comments (See Appendix I for complete list) 40. (S/T10) 89 100 The possible financial costs associated with providing accommodations for disabled employees could divert funds from other necessary expenditures. (S/T10) Costs to provide accommodations for disabled employees often divert funds from other necessary expenditures. (A5) SMEs: -Should be in ‘Accommodation’ section. Breen: -Rewritten for clarity and to provide a more active framework to the item. -Included in ‘Accommodation’ section. -Retained. 188 Key Points A determination of validity is based on the empirical and theoretical evidence available to support the specific inferences drawn from test scores. Evidence based on test content provides a structure within which the balance of this evidence may be considered. The conceptualization of the CADE Scale within the difference model of disability provides evidence of theoretical validity. The CADE Scale, as generated through a judgmental study, demonstrates evidence of empirical validity. 189 Chapter 5: Conclusions, Limitations and Future Research This dissertation contributes to the knowledge required to reduce the high unemployment rates within the disability community. I have identified that measures currently available to quantify the attitudes of co-workers toward PWDs are typically “without formal evaluation or detailed psychometric analyses” (Antonak, 1988, p. 110). Specifically, there is an absence of a scale, supported by theoretical and empirical validity evidence (AERA et al., 2014), which has been demonstrated to effectively measure co-workers’ acceptance of disabled employees. I have developed the CADE Scale to address this gap. As stated by Messick (1990), test validity is based on an “evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of interpretations and actions based on test scores or other modes of assessment” (p. 5). Through the mechanism of a content validity study (Lynn, 1986), I provide empirical validity evidence for the CADE Scale. At both the I-CVI and S-CVI levels, responses from SMEs meet or exceed the recommended levels of agreement to support an acceptable level of evidence of empirical validity within the CADE Scale. This development process is also supported through the explicit application of the difference model as a theoretical framework against which to screen whether individual items may be considered for inclusion within the scale. This process supports the requirement for a clear explanation of the item development process (Antonak, 1988). 190 I have also added to the theoretical validity evidence in support of the scale through a consideration of the newly described category of ontological validity evidence, which provides the means by which the application of the difference model may be considered within the development of the CADE Scale. In effect, ontological validity evidence is the procedural route which connects the CADE Scale to its theoretical underpinnings. This iterative process is an example of what has been described as the effect of pre-existing premises on theory formulation (Rioux, 1997; Thomas, 1999a). Each item included in the CADE Scale has been reviewed for its concurrence with the framework described within the difference model of disability. The development of the CADE Scale, as described within this dissertation, contributes to knowledge within the field of employment and PWDs at both the practical and academic levels. By providing a means to accurately measure co-worker attitudes toward PWDs, the CADE Scale is a tool intended to allow organizations to monitor both short-term and long-term changes to these attitudes, evaluate the benefit of various workplace interventions, and more effectively plan organizational change to meet their goals of an increased representation rate of PWDs. In addition, the articulation of the processes employed to generate both theoretical and empirical evidence in support of the CADE Scale will provide a basis for future academic considerations regarding, respectively, evidentiary implications within validity theory and scale development. Specifically, the application of the difference model to the development of the CADE Scale will serve to illustrate the application of ontological validity evidence as a new tool within the process of test development. 191 This dissertation is subject to a number of limitations. Although the population of SMEs participating in the content validity study was atypically large, geographically and professionally diverse, and included a significant percentage of PWDs, the validity evidence obtained may still be limited in its representation and not be applicable within certain contexts. In addition, the potential exists for an acquiescence bias on the part of SMEs. Responses provided by the SMEs were not anonymous and may have generated data based, at least in part, on an unwillingness to be critical. However, many of the comments and responses indicated a variety of recommendations for improvements, deletions and other modifications to the original materials. Finally, consideration must be given to the potential for a confirmation bias on my part as the developer of the scale (DeVellis, 2017; Furr & Bacharach, 2014; Kane, 2006). Future research, including initial evaluation studies (Rubio et al., 2003), will provide response data suitable for psychometric analyses and a continuation of the validation process of the CADE Scale. These next steps should also consider the several other streams of validity evidence, including response processes, internal structures, relations to other variables, and the consequences of testing (AERA et al., 2014). Data resulting from applications of the CADE Scale are hypothesized to provide empirical evidence in support of the difference model of disability; increased levels of acceptance of disabled employees are hypothesized to demonstrate a positive correlation with increased representation rates of PWDs within individual organizations and employment sectors. Future research will examine whether workplace interventions based on the difference model of disability bring about changes in these attitudes and, 192 subsequently, whether these anticipated changes in attitude lead to increased representation rates of PWDs. It is hypothesized that scores on the CADE Scale will allow for comparative analyses of acceptance levels of disabled employees within organizations and between organizations within various economic sectors. This research will contribute to a deeper understanding of the relationship between PWDs and others in workplace environments, allow for evaluations of training and other interventions, and measure changes in these levels over time. Text Box Commentary #5 Canadian Justice – One Size Fits Most (This article first appeared in the Yukon News in 2004. At the time, I was the Executive Director of the Yukon Council on Disability) Has our Justice system reduced itself to the same business principles as our hat manufacturers? If you have tried to buy a ball cap recently, you’ll know what I mean. The really cool ones don’t have the little plastic adjuster on the back. They just have a bit of elastic sewn in. If you consult any of the trendier “gear” catalogues, the copy under the picture showing ball caps proudly exclaims that “One size fits most!” What it doesn’t say is that if your head doesn’t fit within this group, you are going to be cold and wet. Recently, someone in the hat design world figured out that it is cheaper and more expedient to market one size of cap to most of the people and just not worry about the rest. The savings in inventory costs more than outweigh the few additional sales that could have been made to those of us on the edges of the head-size bell curve. This innovation reminds those of us who are a little longer in the tooth of a similar situation that occurred with shoes about 30 years ago. Remember when you could actually buy “widths” when you went shoe shopping? Maybe the shoe designers have branched out and are offering cost/benefit design courses to the hat guys. What, you say, has this got to do with the Justice System? You may have seen the 193 coverage by The Yukon News and the Whitehorse Star a few weeks ago about a man named Kingshott. He was convicted of sexually assaulting a local woman who was described by the Court as having “severe cognitive deficiencies”. This was the same man who, the day before, was convicted of an unrelated arson offense. Mr. Kingshott received a two-and-a-half year sentence for the arson conviction. The next day, he received a sentence of nine months for the sexual assault. However, the nine months wasn’t added on to the two-and-a-half year term. It is to be served at the same time as the original sentence for the arson. The term for this is a “concurrent sentence”. It’s what those of us who struggle with the rationale for this might call a “two-fer”. Or, in my more cynical moments, what I might call a “freebee”. After my first reading of the media accounts of this situation, I spent a week or so asking people for their perceptions of this case. Without exception, the responses ranged from “Appalling” to “How can they do this?” to “Guess it’s obvious who to pick on”. The overriding theme was that there does not appear to be any justice available to people with “severe cognitive deficiencies”. Of course, the dilemma in a case like this is that we, the community, rarely get to know what really went on behind the scenes. Was there a prior agreement between the Crown and the Defence lawyers? Did the media get the story right? What legal principles come into play? Was this a typical sentence for this type of crime? Since I, and the others whose comments I solicited, may have been reacting more to the media stories than to the facts of the case, I requested copies of the submissions made by the Crown and Defense lawyers to the Court. The Crown also provided me with the written reasons for the sentencing handed down by Judge Gower. After reviewing all of this material, and having a lengthy discussion with the Crown Counsel, I had a much better understanding of many of the legal principles that must be weighed prior to sentencing. Unfortunately, none of these principles seemed to be designed to offer very much protection to those whose heads don’t fit into the legal hat. In fact, they seemed to focus on ensuring that the man convicted of sexually assaulting a woman living with “severe cognitive deficiencies” wasn’t too harshly punished. Mr. Gouaillier, the Crown Counsel, when arguing for a concurrent sentence noted that, “the State, so to speak, got its pound of flesh yesterday (when Kingshott was sentenced for arson – JB) in terms of retribution and incapacitation.” 194 Judge Gower noted, during the proceedings, that although the Crown and Defense lawyers had agreed on what they believed was a fair punishment for Mr. Kingshott, he was not bound to accept that joint submission. He, and the lawyers, then went on to talk about all of the mitigating factors that could be considered. In paraphrase, here are several factors that everyone thought should be considered in Mr. Kingshott’s favour: He has no prior record of sexual assault; He has been crime-free since 1992 (although, admittedly, he had a very lengthy criminal record); He pleaded guilty (albeit at the last moment and following his arson conviction); He was not violent (if one can disregard the violence inherent in the act of sexual assault); “…the impact of this offense is not as serious as it could be on other individuals because she obviously has some difficulties sometimes relating by sequencing events” – Crown Counsel. This last comment, to my mind, showed an outstanding lack of understanding about the internal goings on within individuals with “severe cognitive deficiencies”. Finally, the Crown and the Court both noted that Mr. Kingshott’s plea of guilty had helped to avoid “…an epic battle where blood would have been shed.” This, as far as I can tell, referred to the stress and anxiety that a trial would have caused the victim. I guess Mr. Kingshott is really just a decent guy after all – if we can overlook the fact that he intimidated the victim’s husband, also a person with a “severe cognitive deficiency”, into agreeing to allow him to have sex with the victim. For all of these reasons, it appears that Judge Gower chose to accept the joint submission of the lawyers. In reviewing all of this material, several things became apparent. No one within this scenario had the tools necessary to offer a legitimate form of redress to the victim of this crime. The victim was not only the victim of a sexual assault, she was also the victim of our legal system. She had absolutely no chance for an opportunity to feel that justice was done and, if we think back to the several comments from community members, for justice to be seen to be done. Within our current system, this victim’s limitations relegate her to a second-class world within our community and within our country. And it is the legal system itself that should bear the shame produced from this example of its failure. It was quite clear that the players were making efforts to do the best they could with the available tools. What we need is a restructuring of our legal system that will allow people 195 such as this victim to fully participate within the justice system. To do that, we as a community must accept the fact that the days are over when those of us whose head didn’t fit the “one size fits most” hat were just removed from the community. And if we are to live in the community, we are all entitled to the same treatment by every system, including the justice system. As a final and particularly poignant note, no one during the whole of this convoluted and formal process, thought to make a comment about how frequently people with disabilities are victimized within our community. No one thought to point out that the community should not tolerate anyone taking advantage of its most vulnerable members. No one thought to state ‘for the record’ that people with disabilities should never be seen as “easy pickings”. Personally, I’ve given up wearing hats. 196 References Abberley, P. (1987). The concept of oppression and the development of a social theory of disability. Disability, Handicap and Society, 2(1), 5-19. Allport, G. (1935). Attitudes. In C. Murchison (Ed.), Handbook of social psychology (pp. 798-844). Worcester, MA: Clark University Press. Altman, B. (2001). Disability definitions, models, classification schemes, and applications. In G. Albrecht, K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 97-122). Thousand Oaks, CA: Sage. Alvesson, M., & Sköldberg, K. (2009). (Post-) positivism, social constructionism, critical realism: three reference points in the philosophy of science. In M. Alvesson & K. Skӧldberg (Eds.), Reflexive methodology: new vistas for qualitative research (pp. 15-52). London: Sage. American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. (U.S.). (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Americans with Disabilities Act of 1990, Pub. L. No. 101-336, 104 Stat. 328 (1990). Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37(1), 1-15. Anastasiou, D., & Kauffman, J. (2013). The social model of disability: Dichotomy between impairment and disability. Journal of Medicine and Philosophy, 38(4), 197 441-459. Antonak, R. (1982). Development and psychometric analysis of the Scale of Attitudes Toward Disabled Persons. Journal of Applied Rehabilitation Counseling, 13(2), 22-29. Antonak, R. (1988). Methods to measure attitudes toward people who are disabled. In H. Yuker (Ed.), Attitudes toward persons with disabilities (pp. 109-126). New York, NY: Springer. Antonak, R., & Livneh, H. (2000). Measurement of attitudes towards persons with disabilities. Disability and Rehabilitation, 22(5), 211-224. Aptowicz, C. (2014). Dr. Mutter’s marvels. New York, NY: Penguin Random House. Araten-Bergman, T. (2016). Managers’ hiring intentions and the actual hiring of qualified workers with disabilities. International Journal of Human Resource Management, 27(14), 1510-1530. Barnes, C. (1999). Disability studies: new or not so new directions? Disability and Society, 14(4), 577-580. Barnes, C. (2003). What a difference a decade makes: reflections on doing ‘emancipatory’ disability research. Disability and Society, 18(1), 3-17. Barnes, C. (2014). Reflections on doing emancipatory disability research. In J. Swain, S. French, C. Barnes., & C. Thomas (Eds.), Disabling barriers, enabling environments (3rd ed.). (pp. 37-44), London: Sage. Barnes, C., & Mercer, G. (2001). Disability culture – Assimilation or inclusion. In G. Albrecht, K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 198 515-534). Thousand Oaks, CA: Sage. Barnes, C., & Mercer, G. (2003). Disability. Cambridge: Polity Press. Barnes, C., & Mercer, G. (2010). Exploring disability: A sociological introduction. Cambridge, UK: Polity Press. Barnes, C., Mercer, G., & Shakespeare, T. (1999). Exploring disability: A sociological introduction. Cambridge, UK: Polity Press. Beaudry, J.-B. (2016). Beyond (models of) disability? Journal of Medicine and Philosophy, 41(3), 210-228. Beck, C., & Gable, R. (2001). Ensuring content validity: An illustration of the process. Journal of Nursing Measurement, 9(2), 201–215. Beckett, A., & Campbell, T. (2014). The social model of disability as an oppositional device. Disability and Society, 30(2), 270-283. Beckstead, J. (2009). Content validity is naught. International Journal of Nursing Studies, 46(9), 1274-1283. Bhaskar, R., & Danermark, B. (2006). Metatheory, interdisciplinarity and disability research: A critical realist perspective. Scandinavian Journal of Disability Research, 8(4), 278-297. Bickenbach, J., Jerome, E., Chatterji, S., Badley, E., & Ustun, T. (1999). Models of disablement, universalism and the international classification of impairments, disabilities and handicaps. Social Science and Medicine, 48(9), 1173-1187. Birch, S., & Longmore, P. (Eds.). (2009). Encyclopedia of American disability history. (Volumes 1-3). New York: Facts on File. 199 Bohner, G., & Dickel, N. (2011). Attitudes and attitude change. Annual Review of Psychology, 62, 391-417. Boland, A., Cherry, M., & Dickson, R. (Eds.). (2014). Doing a systematic review: A student’s guide. London: Sage. Boorse, C. (1977). Health as a theoretical concept. Philosophy of Science, 44(4), 542- 573. Braddock, D., & Parish, S. (2001). An institutional history of disability. In G. Albrecht., K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 11-68). Thousand Oaks, CA: Sage. Breen, J. (2015). The exoskeleton generation – disability redux. Disability and Society, 30(10), 1568-1572. Breen, J. (2017). Disability as difference – A fictional representation. The Qualitative Report, 22(10), 2722-2741. Breen, J. (in press). Attitudes toward employees with disabilities: A systematic review of self-report measures. Australian Journal of Rehabilitation Counselling. Breen, J., Havaei, N., & Pitassi, C. (2018). Employer attitudes toward hiring persons with disabilities in Armenia. Disability and Rehabilitation, doi: 10.1080/09638288.2018.1459882 Brennan, R. (2001). Some problems, pitfalls, and paradoxes in educational measurement. Educational Measurement: Issues and Practice, 20(4), 6-18. Brostrand, H. (2006). Tilting at windmills: Changing attitudes toward people with disabilities. Journal of Rehabilitation, 72(1), 4-9. 200 Bryson, S. (1998). Relationship between race and attitudes toward black men. Journal of Multicultural Counseling and Development, 26(4), 282-293. Buller, D. (2005). Adapting minds. Cambridge, MA: MIT Press. Burke, J., Bezyak, J., Fraser, R., Pete, J. Ditchman, N., & Chan, F. (2013). Employers’ attitudes towards hiring and retaining people with disabilities: A review of the literature. Australian Journal of Rehabilitative Counseling, 19(1), 21-38. Burkhauser, R., & Stapleton, D. (2004). The decline in the employment rate for people with disabilities: Bad data, bad health, or bad policy? Journal of Vocational Rehabilitation, 20(3), 185-201. Byrne, Z., Felker, S., Vacha-Haase, T., & Rickard, K. (2011). A comparison of responses on the attitudes toward women scale and attitudes toward feminism scale: Is there a difference between college-age and later-life adults with the original norms? Measurement and Evaluation in Counseling and Development, 44(4), 248-264. Campbell, F. (2005). Legislating disability. In S. Tremain (Ed.), Foucault and the government of disability (pp. 108-130). Ann Arbor, MI: The University of Michigan Press. Chan, F., Gelman, J., Ditchman, N., Kim, J., & Chiu, C. (2009). The world health organization ICF model as a conceptual framework of disability. In F. Chan, E. da Silva Cardoso, & J. Chronister (Eds.), A handbook for evidence-based practitioners in rehabilitation. (pp. 23-50). New York: Springer. Chan, F., Strausser, D., Maher, D., Lee, E-J., Jones, R., & Johnson, E. (2010). Demand- 201 side factors related to employment of people with disabilities: A survey of employers in the Midwest region of the United States. Journal of Occupational Rehabilitation, 20, 412-419. Charlton, J. (2006). The dimensions of disability oppression: An overview. In L. Davis (Ed.),The Disability Studies Reader (2nd ed., pp. 217-227). New York: Routledge. Charmaz, K. (1991). Turning points and fictional identities. In In D. Maines (Ed.), Social organization and social process (pp. 71-86). New York: Aldine de Gruyter. Chen, H-S, Horner, S., & Percy, M. (2003). Cross-cultural validation of the stages of the Tobacco Acquisition Questionnaire and the Decisional balance Scale. Research in Nursing and Health, 26(5), 233-243. Chien, W-T, & Norman, I. (2004). The validity and reliability of a Chinese version of the Family Burden Interview Schedule. Nursing Research, 53(5), 314-322. Chisholm, P., Smith, D., Groleau, G., & Mansour, V. (1988, September 12). Running their own show. MacLean’s Magazine, pp. 20-22. Clark, L., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319. http://doi.org/10.1037/1040-3518.104.22.1689 Clayton, S., Barr, B., Nylen, L., Burström, B., Thielen, K., Diderichson, F., Dahl, E., & Whitehead, M. (2011). Effectiveness of return-to-work interventions for disabled people: a systematic review of government initiatives focused on changing the behaviour of employers. European Journal of Public Health, 22(3), 434-439. Collier, A. (1994). Critical realism. An introduction to Roy Bhaskar’s philosophy. London: 202 Verso. Connolly, W. (2002). Identity/Difference: democratic negotiations of political paradox. Minneapolis, MN: University of Minnesota Press. Copeland, J., Chan, F., Bezyak, J., & Fraser, R. (2010). Assessing cognitive and affective reactions of employers toward people with disabilities in the workplace. Journal of Occupational Rehabilitation, 20(4), 427-434. Corker, M. (1999a). Differences, conflations and foundations: the limits to ‘accurate’ theoretical representation of disabled people’s experience? Disability and Society, 14(5), 627-642. Corker, M. (1999b). New disability discourse, the principle of optimization and social change. In M. Corker & S. French (Eds.), Disability discourse, (pp. 192-209). Buckingham, UK: Open University Press. Craig, D., & Bigby, C. (2015). Critical realism in social work research: Examining participation of people with intellectual disability. Australian Social Work, 68(3), 309-323. Creswell, J. (2013). Qualitative inquiry and research design. Los Angeles: Sage. Crisp, R. (2011). Person-centred rehabilitation counselling: Revisiting the legacy of Carl Rogers. Australian Journal of Rehabilitation Counselling, 17(1), 26-35. Cronbach, L., & Meehl, P. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302. Crow, L. (1996). Including all of our lives: Renewing the social model of disability. In C. Barnes & G. Mercer (Eds.), Exploring the divide (pp. 55-72). Leeds: The Disability 203 Press. Retrieved June 22, 2014 http://www.roaring-girl.com/wp- content/uploads/2013/07/Including-All-of-Our-Lives.pdf Damasio, A. (1999). The feeling of what happens: body and emotion in the making of consciousness. New York: Harcourt Brace. Damasio, A. (2010). Self comes to mind. New York: Pantheon Books. Danermark, B., Ekstrom, M., Jakobsen, L., & Karlsson, J. (2002). Explaining society: Critical realism in the social sciences. London, UK: Routledge. Danermark, B., & Gellerstedt, L. (2004). Social justice: redistribution and recognition – a non-reductionist perspective on disability. Disability and Society, 19(4), 340- 352. Darcy, S., Taylor, T., & Green, J. (2016). ‘But I can do the job’: examining disability employment practice through human rights complaint cases. Disability and Society, 31(9), 1242-1274. Darling, R. (2013). Disability and identity. Boulder, CO: Lynne Rienner Publishers. Davis, L. (1992). Instrument review: Getting the most from your panel of experts. Applied Nursing Research, 5(4), 194-197. Davis, L. (2000). Nude Venuses, Medusa’s body, and phantom limbs: Disability and visuality. In D. Mitchell & S. Snyder (Eds.), The body and physical difference (pp. 51-70). Ann Arbor, MI: The University of Michigan Press. Davis, L. (2001). Identity politics, disability, and culture. (pp. 535-545). In G. Albrecht, K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 535-545). Thousand Oaks, CA: Sage. 204 Davis, L. (2006a). Constructing normalcy: The bell curve, the novel, and the invention of the disabled body in the Nineteenth Century. In L. Davis, (Ed.), The disability studies reader (2nd ed., pp. 3-16). New York: Routledge. Davis, L. (2006b). The end of identity politics and the beginning of dismodernism: On disability as an unstable category. In Davis, L.J. (Ed.), The disability studies reader (2nd ed., pp. 231-242). New York: Routledge. Davis, L. (2013). The end of identity politics: On disability as an unstable category. In L. Davis (Ed.),The disability studies reader (4th ed., pp. 263-277), Oxford: Routledge. Davis, L., & Engel, R. (Eds.). (2011). Measuring race and ethnicity. New York: Springer. de Jonge, J. (2012). Rethinking rational choice theory: A companion on rational and moral action. London, UK: Palgrave MacMillan. DeMeuse, K., & Hostager, T. (2001). Developing an instrument for measuring attitudes toward and perceptions of workplace diversity: An initial report. Human Resource Development Quarterly, 12(1), 33-51. DeVellis, R. (2012). Scale development: Theory and applications (3rd ed.). Thousand Oaks, California: Sage. DeVellis, R. (2017). Scale development: Theory and applications (4th ed.). Los Angeles, CA: Sage. Devine, P. (1989). The role of stereotypes and personal beliefs. In A. Pratkanis, S. Breckler, & A. Greenwald (Eds.), Attitude, structure and function (p. 181-212). Hillsdale, NJ: Lawrence Erlbaum Associates. 205 Dhamoon, R. (2009). Identity/difference politics: How difference is produced and why it matters. Vancouver: UBC Press. Dirth, T., & Branscombe, N. (2017). Disability models affect disability policy support through awareness of structural discrimination. Journal of Social Issues, 73(2), 413-442. Domzal, C., Houtenville, A., & Sharma, R. (2008). Survey of Employer Perspectives on the Employment of People with Disabilities: Technical Report. (Prepared under contract to the Office of Disability and Employment Policy, U.S. Department of Labor). McLean, VA: CESSI. Donoghue, C. (2003). Challenging the authority of the medical definition of disability: an analysis of the resistance to the social constructionist paradigm. Disability and Society, 18(2), 199-208. Dutta, A., Gervey, R., Chan, F., Chou, C., & Ditchman, F. (2012). Understanding employers hiring intentions in relation to qualified workers with disabilities: A United States study. Journal of Occupational Rehabilitation, 18. 326-334. doi: 10.1007/s10926-008-9154-z. Eagly, A., & Chaiken, S. (1993). The psychology of attitudes. Fort Worth, TX: Harcourt, Brace and Jovanovich. Eagly, A., & Telaak, K. (1972). Width of the latitude of acceptance as a determinant of attitude change. Journal of Personality and Social Psychology, 23(3), 388-397. Edwards, M. (1997). Construction of physical disability in the ancient Greek world: The community concept. In D. Mitchell & S. Snyder (Eds.),The body and physical 206 difference: Discourses of disability (pp. 35-50). Ann Arbor, MI: University of Michigan Press. Findler, L., Vichinsky, N., & Werner, S. (2007). The multidimensional attitudes scale toward persons with disabilities (MAS): Construction and validation. Rehabilitation Counseling Bulletin, 50(3), 166-176. Fine, M., & Asch, A. (1988). Disability beyond stigma: Social interaction, discrimination, and activism. Journal of Social Issues, 44(1), 3-22. Fishbein, M. (1963). An investigation into the relationships between beliefs about an object and the attitude toward that object. Human Relations, 16(3), 233-239. [Reprinted in R. Fazio & R. Petty (Eds.), Attitudes: Their structure function and consequences. (pp. 137-142). New York: Psychology Press.] Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley. Fiske, S. (2004). Seek out the magician: Contrarian tricks of mere simplicity make affect appear and disappear from social psychology. In R. Zajonc (Ed.), The selected works of R.B. Zajonc, (pp. 1-10). Hoboken, NJ: Wiley. Fiske, S., & Neuberg, S. (1990). A continuum of impression-formation, from category- based to individuating processes: Influences of information and motivation on attention and motivation. Advances in Experimental Social Psychology, 23, 1-74. Forlin, C., & Fogarty, G. (1999). Validation of the factor structure of the interactions with disabled persons scale. Australian Journal of Psychology, 51(1), 50-55. Foucault, M. (1988). Madness and civilization: a history of insanity in the age of reason, 207 New York: Vintage-Random House. Fougeyrollas, P., & Beauregard, L. (2001). An interactive person-environment social creation. In In G. Albrecht., K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 171-194). Thousand Oaks, CA: Sage. French, S., & Swain, J. (2001). The relationship between disabled people and health and welfare professionals. In G. Albrecht, K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 734-753). Thousand Oaks, CA: Sage. Fujiura, G., & Rutkowski-Kmitta, V. (2001). Counting disability. In G. Albrecht, K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 69-95). Thousand Oaks, CA: Sage. Furr, R., & Bacharach, V. (2014). Psychometrics: An introduction (2nd ed.). Thousand Oaks, CA: Sage. Galvin, R. (2003). The paradox of disability culture: the need to combine versus the imperative to let go. Disability and Society, 18(5), 675-690. Gamson, J. (1995). Must identity movements self-destruct? A queer dilemma. Social Problems, 42(3), 390-407. Gamst, G., Liang, C., & Der-Karabetian, A. (2011). Racism- and prejudice-related measures. In Handbook of Multicultural Measures (pp. 251-334). Thousand Oaks, CA: Sage. Garland-Thomson, R. (2006). Integrating disability, transforming feminist theory. In L. Davis, (Ed.), The disability studies reader (2nd ed., pp. 257-272). New York: Routledge. 208 Gawronski, B., & Baudenhausen, G. (2006). Associative and propositional processes in evaluation: An integrative review of implicit and explicit attitude change. Psychological Bulletin, 132(5), 692-731. Gething, L., & Wheeler, B. (1992). The interaction with disabled persons scale: A new Australian instrument to measure attitudes towards people with disabilities. Australian Journal of Psychology, 44(2), 75-82. Gilbride, D., Vandergoot, D. Golden, K., & Stensrud, R. (2006). Development and validation of the Employer Openness Survey. Rehabilitation Counseling Bulletin, 49(2), 81-89. Ginsberg, F., & Rapp, R. (2013). Disability worlds. Annual Review of Anthropology, 42, 53-68. Goffman, E. (1963). Stigma: notes on the management of spoiled identity. Englewood Cliffs, NJ: Prentice Hall. Retrieved on August 8, 2014 from: file:///C:/Users/Jon/Desktop/Stigma%20%20Notes%20on%20the%20Managem ent%20of%20Spoiled%20Identity.htm Goodey, C. (2016). Learning disability and inclusion phobia. Oxon, UK: Routledge. Goodley, D. (2011). Disability studies: An interdisciplinary introduction. London: Sage. Gough, D. (2007). Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. In J. Furlong & A. Oancea (Eds.), Applied and Practice- based Research. Special Edition of Research Papers in Education, 22(2), 213- 228. Gough, D., Oliver, S., & Thomas, J. (2012). An introduction to systematic reviews. 209 London: Sage. Grant, J., & Davis, L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269–274. Greenwald, A., McGhee, D., & Schwartz, J. (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74(6), 1464-1480. Gregory, R. (2007). Psychological testing: History, principles, and applications. Boston, MA: Allyn & Bacon. Guidano, V. (1987). Complexity of the self: A developmental approach to psychopathology and therapy. New York: The Guilford Press. Guidano, V. (1995). Constructivist psychotherapy: A theoretical framework. In Constructivism in psychotherapy. (no author, pp. 93-108). Washington, DC: American Psychological Association. Gustavsson, A. (2004). The role of theory in disability research – springboard or straight-jacket? Scandinavian Journal of Disability Research, 6(1), 55-70. Hacking, I. (1999). The social construction of what? Cambridge, MA: Harvard University Press. Haegele, J., & Hodge, S. (2016). Disability discourse: Overview and critiques of the medical and social models. Quest, 68(2), 193-206. Hahn, H. (1993). The political implications of disability definitions and data. Journal of Disability Policy Studies, 4(2), 41-52. Halliday, V., Porock, D., Arthur, A., Manderson, C., & Wilcock, A. (2012). Development 210 and testing of a cancer appetite and symptom questionnaire. Journal of Human Nutrition and Dietetics, 25(3), 217–224. doi:10.1111/j.1365-277X.2012.01233.x Haynes, S., & Richard, D., & Kubany, E. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238-247. Hegarty, W., & Dalton, D. (2010). The organizational diversity scale. In L. Davis & R. Engel (Eds.), Measuring race and ethnicity, (p. 146). New York: Springer. Hernandez, B., Keys, C., & Balcazar, F. (2000). Employer attitudes toward workers with disabilities and their ADA employment rights: A literature review. Journal of Psychology, 66(4), 4-16. Hinkin, T. (1998). A brief tutorial on the development of measures for use in survey questionnaires [Electronic version]. Retrieved February 28, 2018], from Cornell University, School of Hotel Administration site: http://scholarship.sha.cornell.edu/articles/521 Houston, S. (2010). Prising open the black box: Critical realism, action research and social work. Qualitative Social Work, 9(1), 73-91. Hubley, A., & Palepu, A. (2007). Injection drug user quality of life scale (IDUQOL): Findings from a content validation survey. Health and Quality of Life Outcomes, 5, Article 47. Hubley, A., & Zumbo, B. (1996). A dialectic on validity: Where we have been and where we are going. The Journal of General Psychology, 123(3), 207–215. doi:10.1080/00221309.1996.9921273. 211 Hubley, A., & Zumbo, B. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103(2), 219-230. doi:10.1007/s11205-011- 9843-4 Hubley, A., & Zumbo, B. (2013). Psychometric characteristics of assessment procedures: An overview. In K. Geisinger et al., (Eds.), APA handbook of testing and assessment in psychology, (Vol. 1: Test theory and testing and assessment in industrial and organizational psychology). (pp. 3-19). Washington, DC: American Psychological Association. http://dx.doi.org.ezproxy.library.ubc.ca/10.1037/14047-000 Hughes, B. (2009). Wounded/monstrous/abject: a critique of the disabled body in the sociological imaginary. Disability and Society, 24(4), 399-410. Hughes, B., & Paterson, K. (1997). The social model of disability and the disappearing body: Towards a sociology of impairment. Disability & Society, 12(3), 325-340. Hunt, P. (1996). A critical condition. In P. Hunt, (Ed.), Stigma: The experience of disability (pp. 145-159). London, UK: Geoffrey Chapman. Hunt, C., & Hunt, B. (2004). Changing attitudes toward people with disabilities: Experimenting with an educational intervention. Journal of Managerial Issues, 16(2), 266-280. Iacono, T., Tracy, J., Keating, J., & Brown, T. (2009). The Interactions with Disabled Persons scale: Revisiting its internal consistency and factor structure, and examining item-level properties. Research in Developmental Disabilities, 30, 1490- 1501. 212 Imrie, R. (2004). Demystifying disability: a review of the international classification of functioning, disability and health. Sociology of Health and Illness, 26(3), 287- 305. Jasper, C., & Waldhart, P. (2013). Employer attitudes on hiring employees with disabilities in the leisure and hospitality industry. International Journal of Contemporary Hospitality Management, 25(4), 577-594. Kane, M. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319-342. Kane M. (2016). Explicating validity. Assessments in Education: Principles, Policies and Practice, 23(2), 198-211. Katz, I., Hass, R., & Bailey, J. (1988). In H. Yuker (Ed.), Attitudes toward persons with disabilities (pp. 46-57). New York, NY: Springer. Kidd, M., Sloane, P., & Ferko, I. (2000). Disability and the labour market: an analysis of British males. Journal of Health Economics, 19(6), 961-981. Kools, S., McCarthy, M., Durham, R., & Robrecht, L. (1996). Dimensional analysis: Broadening the conception of grounded theory. Qualitative Health Research, 6(3), 312-330. Kuhn, T. (2012). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press. (original publication date, 1962) Lam, W., Gunukula, S., McGuigan, D., Isaiah, N., Symons., & Akl, E. (2010). Validated instruments used to measure attitudes of healthcare students and professionals towards patients with physical disability: a systematic review. Journal of 213 NeuroEngineering and Rehabilitation, 7, 55-61. Lave, C., & March, J. (1975). An introduction to models in the social sciences. New York: Harper & Row. Lawrence, R., & Jette, A. (1996). Disentangling the disablement process. Journal of Gerontology, 51B(4), S173-S182. Link, B., & Phelan, J. (2001). Conceptualizing stigma. Annual Review of Sociology, 27, 363-385. Linton, S. (1998). Claiming disability: Knowledge and identity. NY: New York University Press. Littell, J., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis. New York: Oxford University Press. Livneh, H. (1988). A dimensional perspective on the origin of negative attitudes toward persons with disabilities. In H. Yuker (Ed.), Attitudes toward persons with disabilities (pp. 35-46). New York, NY: Springer. Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3(3), 635-694. Loo, R. (2001a). A psychometric re-analysis of the interaction with disabled persons scale. Canadian Journal of Behavioural Science, 33(4), 245-250. Loo, R. (2001b). Attitudes of management undergraduates toward persons with disabilities: A need for change. Rehabilitation Psychology, 46(3), 288-295. Loo, R. (2004). Attitudes toward employing persons with disabilities: A test of the sympathy-discomfort categories. Journal of Applied Social Psychology, 34(10), 214 2200-2214. Louvet, E. (2007). Social judgment toward job applicants with disabilities: Perception of personal qualities and competence. Rehabilitation Psychology, 52(3), 297-303. Lynn, M. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382-386. Martinez, M. (2007). Ian Hacking’s proposal for the distinction between natural and social sciences. Philosophy of the Social Sciences, 39(2), 212-234. Mason, J. (2002). Qualitative researching (2nd ed.). London: Sage. Maxwell, J. (2012). A realist approach for qualitative research. Thousand Oaks, CA: Sage. McCarthy, H. (1988). Attitudes that affect employment opportunities for persons with disabilities. (pp. 246-261). In H. Yuker (Ed.), Attitudes toward persons with disabilities. New York: Springer. McCaughey, T., & Strohmer, D. (2005). Prototypes as an indirect measure of attitudes toward disability groups. Rehabilitation Counseling Bulletin, 48(2), 89-99. McKee, D., & Chappel, J. (1992). Spirituality and medical practice. The Journal of Family Practice, 35(2), 205-208. McKenzie, J., Wood, M., Kotecki, J., Clark, J., & Brey, R. (1999). Establishing content validity: Using qualitative and quantitative steps. American Journal of Health Behavior, 23(4), 311-318. doi:10.5993/AJHB.23.4.9 McLaughlin, M., Bell, M., & Stringer, D. (2004). Stigma and acceptance of persons with disabilities: understudied aspects of workforce diversity. Group and Organization 215 Management, 29(3), 302-333. Mercer, G. (2002). Emancipatory disability research. In C. Barnes, M. Oliver & L. Barton (Eds.), Disability studies today (pp. 229-249). Cambridge, UK: Polity Press. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012-1027. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: American Council on Education & Macmillan Publishing. Messick, S. (1990). Validity of test interpretation and use. Research Rep. No. 90-11. Princeton, NJ: Educational Testing Service. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. doi:10.1037/0003-066X.50.9.741 Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research, 45, 35-44. Meyers, S., & Lester, D. (2016). An attempt to change college students’ attitudes toward individuals with disabilities. Comprehensive Psychology, 5, 1-7. doi:10.1177/2165222816648076 Mik-Meyer, N. (2016). Othering, ableism and disability: A discursive analysis of co- workers’ construction of colleagues with visible impairments. Human Relations, 69(6), 1341-1363. Miller, L., McIntire, S., & Lovler, R. (2011). How do we develop a test? (Chap. 11, pp. 311-333). Foundations of psychological testing: A practical approach (3rd ed.), 216 Los Angeles: Sage. Mineka, S., & Ohman, A. (2002). Phobias and preparedness: the selective, automatic, and encapsulated nature of fear. Biological Psychiatry, 52(10), 927-937. Nagi, S. (1976). An epidemiology of disability among adults in the United States. Milbank Memorial Fund Quarterly, 54(4), 439-467. Oliver, M. (1990). The politics of disablement. London: MacMillan Press. Oliver, M. (1992a). Changing the social relations of research production? Disability, Handicap and Society, 7(2), 101-114. Oliver, M. (1992b). Intellectual masturbation: a rejoinder to Söder and Booth. European Journal of Special Needs Education, 7(1), 20-28. Oliver, M. (1998). Theories of disability in health care practice and research. British Medical Journal, 317(7170), 1446-1449. Oliver, M. (2004). If I had a hammer: The social model in action. In J. Swain, S. French, C. Barnes, & C. Thomas (Eds.), Disabling barriers - enabling environments (2nd ed., pp. 7-12). London: Sage. Oliver, M. (2009). Understanding disability: From theory to practice. NY: Palgrave MacMillan. Oliver, M. (2013). The social model of disability: thirty years on. Disability and Society, 28(7), 1024-1026. Oliver, M., & Barnes, C. (2012).The new politics of disablement. NY: Palgrave Macmillan. Osipov, V. (2013). Employment of People with Disabilities in Armenia: Needs and 217 Barriers. International Organization for Migration Mission in Armenia. Oxford English Dictionary (2009). 2nd edition on CD-ROM (v. 4.0). Oxford: Oxford University Press. Owens, J. (2015). Exploring the critiques of the social model of disability: The transformative possibility of Arendt’s notion of power. Sociology of Health and Illness, 37(3), 385-403. Ozawa, A., & Yaeda, J. (2007). Employer attitudes toward employing persons with psychiatric disability in Japan. Journal of Vocational Rehabilitation, 26(2), 105- 113. Palad, Y., Barquia, R., Domingo, H., Flores, C., Padilla, L., & Ramel, J. (2016). Scoping review of instruments measuring attitudes toward disability. Disability and Health Journal, 9(3), 354-374. Parker, R., & Aggleton, P. (2003). HIV and AIDS-related stigma and discrimination: a conceptual framework and implications for action. Social Science and Medicine, 57(1), 13-24. Parsons, T. (1951). Illness and the role of the physician: A sociological perspective, Journal of Orthopsychiatry, 21(3), 452-460. Parsons, T. (1952). The social system. London: Tavistock Publications. (Original work published in 1951). Pernick, M. (1997). Eugenics and public health in American history. American Journal of Public Health, 87(11), 1767-1772. Peters, S. (1999). Transforming disability identity through critical literacy and the 218 cultural politics of language. In M. Corker & S. French (Eds.), Disability discourse, (pp. 103-115). Buckingham, UK: Open University Press. Polit, D., & Beck, C. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489-497. Polit, D., Beck, C., & Owen, S. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing and Health, 30(6), 459-467. Popovich, P., Scherbaum, C., Scherbaum, K., & Polinko, N. (2003). The assessment of attitudes toward individuals with disabilities in the workplace. The Journal of Psychology, 137(2), 163-177. Priestley, M. (1998). Constructions and creations; idealism, materialism and disability theory. Disability and Society, 13(1), 75-95. Priestley, M. (1999). Discourse and identity: disabled children in mainstream high schools. In M. Corker & S. French (Eds.), Disability discourse, (pp. 92-102). Buckingham, UK: Open University Press. Pruett, S., & Chan, F. (2006). The development and psychometric validation of the Disability Attitude Implicit Association Test. Rehabilitation Psychology, 51(3), 202-213. Reisman, C. (2008). Narrative methods for the human sciences. Los Angeles, CA: Sage. Rioux, M. (1997). Disability: The place of judgment in a world of fact. Journal of Intellectual Disability Research, 41(2), 102-111. 219 Rioux, M., & Patton, L. (2014). Employment equity and disability: Moving forward to achieve employment integration and fulfil promises of inclusion and participation. In C. Agócs (Ed.), Employment equity in Canada: The legacy of the Abella report (pp. 133-155). Toronto: University of Toronto Press. Rogers, T. (1995). The psychological testing enterprise: An introduction. Pacific Grove, CA: Brooks/Cole. Rose, M. (2006). Deaf and dumb in ancient Greece. In L. Davis, (Ed.), The disability studies reader (2nd ed., pp. 17-32). New York: Routledge. Rubio, D., Berg-Weger, M, Tebb, S., Lee, E., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94-104. Russell, L. (2013). Content validation of the quality of life for homeless and hard-to- house individuals (QoLHHI) health and living conditions impact sections: Implications for instrument use and content validation methodology (T). University of British Columbia. Retrieved from https://open.library.ubc.ca/cIRcle/collections/24/items/1.0165615 Russell, M. (2002). What disability civil rights cannot do: employment and political economy. Disability & Society, 17(2), 117-135. Schatzman, L. (1991). Dimensional analysis: Notes on an alternative approach to the grounding of theory in qualitative research. In D. Maines (Ed.), Social organization and social process (pp. 303-314). New York: Aldine de Gruyter. Schilling, L., Dixon, J., Knafl, K., Grey, M., Ives, B., & Lynn, M. (2007). Determining 220 content validity of a self-report instrument for adolescents using a heterogeneous expert panel. Nursing Research, 56(5), 361-366. Schur, L., Kruse, D., & Blanck, P. (2005). Corporate culture and the employment of persons with disabilities. Behavioral Sciences and the Law, 23(1), 3-20. Schur, L., Kruse, D., Blasi, J., & Blanck, P. (2008). Is disability disabling in all workplaces? Workplace disparities and corporate culture. Industrial Relations: A Journal of Economy and Society, 48(3), 381-410. Schwarz, N. (2007). Cognitive aspects of survey methodology. Applied Cognitive Psychology, 21(2), 277-287. Schwarz, N. (2008). Self-reports: How the questions shape the answers. In R. Fazio & R. Petty (Eds.), Attitudes: Their structure , function, and consequences. (pp. 49- 67). New York: Psychology Press. Seligman, M. (1971). Phobias and preparedness. Behavioral Therapy, 2(3), 307-320. Shah, P., & Mountain, D. (2007). The medical model is dead – long live the medical model. British Journal of Psychiatry, 191, 375-377. Shah, S., & Priestley, M. (2011). Disability and social change. Bristol, UK: Polity Press. Shakespeare, T. (1994). Cultural representation of disabled people: Dustbins for disavowal? Disability and Society, 9(3), 283-299. Shakespeare, T. (1996). Disability, identity and difference. In C. Barnes & G. Mercer, (Eds.), Exploring the divide (pp. 94-113). Leeds, UK: The Disability Press. Shakespeare, T. (2004). Social models of disability and other life strategies. Scandinavian Journal of Disability Research, 6(1), 8-21. 221 Shakespeare, T. (2006a). Disability rights and wrongs. New York: Routledge. Shakespeare, T. (2006b). The social model of disability. In L. Davis, (Ed.), The disability studies reader (2nd ed., pp. 197-204). New York: Routledge. Shakespeare, T. (2014). Disability rights and wrongs revisited (2nd ed.). London: Routledge. Shakespeare, T., & Watson, N. (2001). The social model of disability: an outdated ideology? In S. Barnartt & B. Altman (Eds.), Exploring theories and expanding methodologies: Where we are and where we need to go (Research in Social Science and Disability, Volume 2). (pp. 9–28), Bingley, UK: Emerald Group. Sherry, M. (2004). Overlaps and contradictions between queer theory and disability studies. Disability and Society, 19(7), 769-783. Shildrick, M. (2012). Dangerous discourses of disability, subjectivity and sexuality. Hampshire, UK: Palgrave MacMillan. Shilling, C. (2002). Culture, the ‘sick role’ and the consumption of health, British Journal of Sociology, 53(4), 621-638. Siebers, T. (2006). Disability in theory: From social constructionism to the new realism of the body. In L. Davis, (Ed.), The disability studies reader (2nd ed., pp. 173- 184). New York: Routledge. Siebers, T. (2008). Disability theory. Ann Arbor, MI: University of Michigan Press. Silvers, A. (2003). On the possibility and desirability of constructing a neutral conception of disability. Theoretical Medicine, 24(6), 471-487. Siminski, P. (2003). Patterns of disability and norms of participation through the life 222 course: empirical support for a social model of disability. Disability and Society, 18(6), 707-718. Sireci, S. (1998a). The construct of content validity. Social Indicators Research, 45(1-3), 83-117. Sireci, S. (1998b). Gathering and analyzing content validity data. Educational Assessment, 5(4), 299-321. Smart, J. (2009). The power of models of disability. Journal of Rehabilitation, 75(2), 3- 11. Snyder, S., & Mitchell, D. (2006). Cultural locations of disability. Chicago, IL: University of Chicago Press. Stainton, T. (2004). Reason’s other: the emergence of the disabled subject in the northern renaissance. Disability and Society, 19(3), 225-243. Stainton, T. (2008). Reason, grace and charity: Augustine and the impact of church doctrine on the construction of intellectual disability. Disability and Society, 23(5), 485-496. Stainton, T., & Swift, K. (1996). “Difference” and social work curriculum. Canadian Social Work Review, 13(1), 75-87. Statistics Canada. (2006). PALS Survey. http://www.statcan.gc.ca/daily- quotidien/080724/dq080724a-eng.htm (accessed January 7, 2015). Statistics Canada. (2013). Canadian Survey on Disability Data Tables. http://www.statcan.gc.ca/pub/89-654-x/89-654-x2013001-eng.pdf (accessed May 3, 2018). 223 Stark, S. (2001). Creating disability in the home: the role of environmental barriers in the United States. Disability and Society, 16(10), 37-49. Stewart, J., Lynn, M., & Mishel, M. (2005). Evaluating content validity for children’s self-report instruments using children as content experts. Nursing Research, 54(6), 414–418. Stiller, J. (1976). Attitudes toward disability. In H. Rusalem & D. Malikin (Eds.), Contemporary vocational rehabilitation (pp. 67-80). New York: New York University Press. Stone, D., & Colella, A. (1996). A model of factors affecting the treatment of disabled individuals in organizations. Academy of Management Review, 21(2), 352-401. Swain, J., & French, S. (2000). Towards an affirmation model of disability. Disability and Society, 15(4), 569-582. Taura, M. (2011). Evaluation of anthropomorphic status of Hausas of northern Nigeria. Bajopas, 4(2), 80-82. Taylor, S., & Bogdan, R. (1989). On accepting relationships between people with mental retardation and non-disabled people: Towards an understanding of acceptance. Disability, Handicap and Society, 4(1), 21-36. Thomas, C. (1999a). Narrative identity and the disabled self. In M. Corker & S. French (Eds.), Disability discourse, (pp. 47-55). Buckingham, UK: Open University Press. Thomas, C. (1999b). Female forms: Experiencing and understanding disability. Buckingham, UK: Open University Press. Thomas, C. (2004). How is disability understood? An examination of sociological 224 approaches. Disability and Society, 19(6), 569-583. Thomas, C. (2007). Sociologies of disability and illness. Hampshire, UK: Palgrave MacMillan. Thurstone, L. (1928). Attitudes can be measured. Journal of American Sociology, 33(4), 529-554. Tilden, V., Nelson, C., & May, B. (1990). Use of qualitative methods to enhance content validity. Nursing Research, 39(3), 172–175. Titchkosky, T. (2001). Disability: A rose by any other name? “People-first” language in Canadian society. Canadian Review of Sociology, 38(2), 125-140. Trieschmann, R. (1988). Spinal cord injuries – Psychological, social and vocational rehabilitation (2nd ed.). New York: Demos. Turcotte, M. (2014). Persons with disabilities and employment. (Catalogue No. 75-006- X). Statistics Canada. Retrieved April 23, 2015, from http://www.statcan.gc.ca/pub/75-006-x/2014001/article/14115-eng.pdf Turner, B. (2001). Disability and the sociology of the body. In G. Albrecht, K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 252-264). Thousand Oaks, CA: Sage. United Nations, 2006. Convention on the rights of persons with disabilities. Treaty Series, 2515, 3. UPIAS (1976). Fundamental principles of disability. London: Union of the Physically Impaired Against Segregation. Retrieved June 5, 2017 from http://disability- studies.leeds.ac.uk/files/library/UPIAS-fundamental-principles.pdf 225 Vehmas, S., & Watson, N. (2014). Moral wrongs, disadvantages, and disability: a critique of critical disability studies. Disability and Society, 29(4), 638-650. Vilchinsky, N., & Findler, L. (2010). Attitudes toward people with disabilities: The perspective of attachment theory. Rehabilitation Psychology, 55(3), 298-306. Vornholt, K., Uitdewilligen, S., & Nijhuis, J. (2013). Factors affecting the acceptance of people with disabilities at work: A literature review. Journal of Occupational Rehabilitation, 23, 463-475. Waldschmidt, A. (2005). Who is normal? Who is deviant? In S.L. Tremain (Ed.), Faucault and the government of disability (pp. 191-207). Ann Arbor, MI: University of Michigan Press. Waltz, C. & Bausell, R. (1981). Nursing research: Design, statistics and computer analysis. Philadelphia, PA: F.A. Davis. Waltz, C., Strickland, O., & Lenz, E. (2005). Measurement in nursing and health research (3rd ed.). New York: Springer. Waltz, C., Strickland, O., & Lenz, E. (2010). Measurement in nursing and health research (4th ed.). New York: Springer. Watson, N. (2012). Researching disablement. In N. Watson, A. Roulstone, & C. Thomas (Eds.), Routledge handbook of disability studies (pp. 93-105). London: Routledge, Taylor & Francis Group. Wiggett-Barnard, C., & Swartz, L. (2012). What facilitates the entry of persons with disabilities into South African companies? Disability and Rehabilitation, 34(12), 1016-1023. 226 Williams, G. (2001). Theorizing disability. In G. Albrecht, K. Seelman, & M. Bury (Eds.), The Handbook of Disability Studies (pp. 123-145). Thousand Oaks, CA: Sage. Williams, S. (1999). Is anybody there? Critical realism, chronic illness and the disability debate. Sociology of Health and Illness, 21(6), 797-819. Wilson, C., & Scior, K. (2014). Attitudes toward individuals with disabilities as measured by the Implicit Association Test: A literature review. Research in Developmental Disabilities, 35(2), 294-321. Wolbring, G. (2008). The politics of ableism. Development, 51(2), 252-258. Wolfensberger, W. (1972). The principle of normalization in human services. Toronto, Canada: Leonard Crainford. World Health Organization. (2001). International classification of functioning, disability and health – ICF Short version. Geneva: WHO. Young, M. (1990). Justice and the politics of difference. Princeton, NJ: Princeton University Press. Yuker, H., Block, J., & Younng, J. (1970). The measurement of attitudes toward disabled persons. Albertson, NY: Insurance Company of North America. Yun, S., & Weaver, R. (2010). Development and validation of the short form of the attitude toward poverty scale. Advances in Social Work, 11(2), 174-187. Zajonc, R. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35(2), 151-175. Zajonc, R. (2000). Closing the debate over the independence of affect. In J. Forgas (Ed.), Feeling and thinking: The role of affect in social cognition, (pp. 31-58). 227 Cambridge, UK: Cambridge University Press. Zajonc, R. (2004). Cognition and social cognition: A historical perspective. In R. Zajonc (Ed.), The selected works of R.B. Zajonc, (pp. 41-61). Hoboken, NJ: Wiley. Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H., & Nikanfar, A-R. (2015). Design and implementation Content Validity Study: Development of an instrument for measuring patient-centered communication. Journal of Caring Sciences, 4(2), 165-178. Zola, I. (1989). Towards the necessary universalizing of a disability policy. The Milbank Quarterly, 67 (suppl. 2, pt. 2), 401-428. Zola, I. (1993). Disability statistics, what we count and what it tells us. Journal of Disability Policy Studies, 4(2), 9-39. 228 Appendix A: Initial 40 Item Scale Original 40 Item Scale # Cat. Item 1. H It would be fair if disabled job seekers were required to disclose the likely progression of their disability as part of the hiring process. 2. H When hiring disabled employees, consideration should be given to how well they will be accepted by other members of the organization. 3. H Job candidates with disabilities are less likely than others to have all of the skills necessary for technical positions. 4. H A job seeker who indicates on an application form that he has a disability should include the medical diagnosis. 5. H Job candidates with disabilities are more likely than others to exaggerate their skills in an interview. 6. H An applicant for a job that is similar to yours has advised that she requires a disability-related accommodation for her interview. This could also indicate that she may not be able to satisfactorily perform all aspects of this job. 7. H Job candidates with disabilities are less likely than others to have all of the skills necessary for physically demanding positions. 8. H At a technology-oriented customer service desk, a “tech” with a visible disability would instill less confidence than one who was able-bodied. 9. H I would be comfortable introducing a new disabled employee to my work group. 10. H Job candidates with disabilities are less likely than others to have the emotional stability that is necessary to succeed at very stressful jobs. 11. P For purposes of receiving annual cash bonuses, employees with disabilities should be rated on the effort that they put in instead of how well they perform. 12. P I would be concerned that working with disabled colleagues as part of my team could negatively affect my own performance evaluation. 13. P Having a person with a disability as a member of my team would add extra burdens to my job. 14. P I would be less comfortable reporting to a supervisor with a visible disability than to someone without a disability. 15. P I would be less likely to become friends outside of our workplace with a disabled colleague than with others. 16. P Working with disabled co-workers would distract me from my own responsibilities. 17. P The inclusion of disabled employees can be detrimental to a project team. 18. P Compared to others, disabled employees are more likely to miss work due to illness. 19. P I would be more comfortable working with co-workers who did not have disabilities. 20. P I would be concerned about a disabled co-worker’s ability to perform my job while I was on an extended vacation. 21. A If necessary, disabled employees should be entitled to have peripheral responsibilities within their job descriptions modified, even if this requires corresponding changes in others’ duties. 22. A Employees with certain disabilities should be allowed to wear headphones to listen to music at work if this helps them concentrate. 229 23. A I would have to be careful about my language so as not to inadvertently offend a disabled co-worker. 24. A Disabled employees typically expect some degree of help in performing their day-to-day tasks. 25. A It would be fair for an employee who requires an ergonomic chair to go to the top of our workgroup waiting list, even if others have been waiting longer for new chairs. 26. A I would be unhappy about being requested by a supervisor to assist a disabled colleague with performing a brief, daily task. 27. A My employer should make training available to managers and other employees to help us understand disability-related challenges within the workplace. 28. A Supervisors should not have to be burdened with additional responsibilities associated with disabled employees. 29. A The need to include disabled employees in workplace-sponsored social events limits opportunities for others. 30. A If people with invisible disabilities require any special treatment in the workplace, they should explain their specific disability to co-workers. 31. S/D People with disabilities are entitled to have different rules about their working hours if they p medical reasons. (Included typographical error. “p” should be “provide” - JB) 32. S/D I would feel more uncomfortable dismissing a disabled employee for poor performance than I would about another employee. 33. S/D If criticized for poor performance, disabled employees are more likely than others to blame external circumstances. 34. S/D Disabled employees should always be given additional opportunities to correct inappropriate behaviour. 35. S/D The supervision of disabled employees, as compared to others, can be more difficult because of the potential for human rights claims. 36. S/D As a supervisor, I would not know how to respond to a disabled employee who blamed a poor performance review on her disability. 37. S/D I would feel more uncomfortable about having to discipline a disabled employee than I would about another employee. 38. S/D Supervisors may have to neglect other workers in order to meet the needs of disabled employees. 39. S/D Small performance failures on the part of disabled employees should be overlooked if they appear to be making an effort. 40. S/D The possible financial costs associated with providing accommodations for disabled employees could divert funds from other necessary expenditures. 230 Appendix B: Final 20 Item Scale CADE Scale (20 items) Hiring (1-5); Performance (6-10); Accommodations (11-15); Supervision/Termination (16-20) *signifies an item to be Reverse Scored. # Cat. Item 1. H Disabled applicants should provide job-related details of their disability during the interview process. 2. H When hiring disabled employees, consideration should be given to how well their disability will be accepted by co-workers. 3.* H Applicants with disabilities are as likely as other candidates to have all of the required job skills. 4. H I would be less comfortable welcoming a new employee into my work group if they had a disability. 5. H Job candidates with disabilities are less likely than others to have the emotional stability necessary to succeed at very stressful jobs. 6. P I would be concerned that having a disabled person as part of my work unit could negatively affect my own performance evaluation. 7. P The inclusion of disabled employees can be detrimental to a project team. 8. P Employees with disabilities are more likely than others to miss work. 9. P I would be more comfortable working with others who did not have disabilities. 10. P Employees with disabilities typically expect some help in performing their day-to-day tasks. 11.* A If medically necessary, employees with disabilities should be entitled to have their responsibilities modified, even if this means changing co-workers’ duties. 12. A I would need to be careful about my language so I did not accidentally offend a disabled co-worker. 13. A I would be unhappy about being requested by a supervisor to assist a disabled co-worker with performing a brief, daily task. 14. A Supervisors should not have to accept additional responsibilities to manage disabled employees. 15. A Costs to provide accommodations for disabled employees often divert funds from other necessary expenditures. 16. S/T As a supervisor, I would be particularly uncomfortable following up with an employee who blamed a poor performance review on their disability. 17. S/T The supervision of disabled employees can be more difficult because of possible claims of discrimination. 18. S/T It would be harder for me to dismiss a disabled employee for poor performance than it would to dismiss another employee. 19. S/T Supervisors may have to spend less time with other workers in order to meet the needs of employees with disabilities. 20. S/T Small performance failures by disabled employees should be overlooked if they appear to be making an effort. 231 Appendix C: Databases and search parameters The following databases were searched between April 7, 2015 and April 11, 2015 – Medline (OvidSP), PsycInfo, Cinahl, Google Scholar, Cochrane Collaboration. Details of each are provided below within each of the search descriptions. Thirteen searches were conducted as detailed below: (KW = key words; TI = title; AB = abstract) Search 1: Medline (OvidSP) (April 9, 2015) 1. Disabled person (exp) KW 2. Employ* KW 3. 1 + 2 4. Integrat* OT accept* OR attitude* TI/AB 5. 3 + 4 6. Measure* OR questionnaire* OR Empirical OR quantitative 7. 5 + 6 8. Total = 234 Search 2: Medline (OvidSP) (April 9, 2015) 1. Disab* TI 2. Disab* TI Limits: 2005-2015 (all months)/English/humans 3. Employ* OR work TI 4. Employ* OR work TI Limits: 2005-2015 (all months)/English/humans 5. Attitude* OR accept* OR integrat* TI 6. Attitude* OR accept* OR integrat* TI Limits: 2005-2015 (all months)/English/humans 7. Combine 2,4 & 6 TI 8. Total = 11 Search 3: Medline (OvidSP) (April 9, 2015) (all limited to 201=05-2015 (all months), English/humans) 1. Measure* OR questionnaire* OR tool OR evaluat* TI/AB 2. Disab* TI/AB 3. Empl* OR work TI/AB 4. Attitude* OR accept* OR integrat* TI/AB 5. 1+2+3+4 TI/AB 6. Total = 39 Search 4: Medline (OvidSP) (April 9, 2015) Limits: English/humans, 2005-2015 232 1. Disability AND attitude AND employment TI AND factors KW 2. Total = 105 Search 5: Medline (OvidSP) April 9, 2015 1. Disab* TI/AB 2. Disab* TI/AB Limits: 2005-2015 (all months)/English/humans 3. Employ* OR work TI/AB 4. Employ* OR work TI/AB Limits: 2005-2015 (all months)/English/humans 5. Attitude* OR accept* OR integrat* TI/AB 6. Attitude* OR accept* OR integrat* TI/AB Limits: 2005-2015 (all months)/English/humans 7. Combine 2,4 & 6 TI/AB 8. Total = 11 Search 6: Medline (OvidSP) (April 13, 2015) Limits: 1998-2015, English, humans, all in Title 1. valid* AND disab* AND (employ* OR work OR organiz*) 2. Total = 78 Search 7: PsycInfo (April 9, 2015) Limits: Jan. 2005 – Mar. 2015, “apply related words”, English, 18+ years, all in Title) 1. Disab* AND (attitude* OR accept* OR integrat*) AND (employ* OR work) Total = 18 Search 8: PsycInfo (April 9, 2015) Limits: Peer reviewed journals, 2005-2015 1. Disability AND attitude AND employment TI 2. Total = 2 Search 9: PsycInfo (April 11, 2015) Limits: TI/AB 1. Disab* AND (employ* OR work) AND (scale* OR factor* OR questionnaire* OR tool* OR measure*) AND (accept* OR integrat* OR attitude*) TI/AB 2. Total = 537 Search 10: Cochrane Collaboration (Apr 23, 2015) Limits: none 1. Disab* 2. Total = 25 Search 11: Google Scholar (April 9, 2015) Limits: 2005-2015 1. Disability AND employment AND TI (attitude OR acceptance OR integration) TI 2. Total = 9 233 Search 12: Google Scholar (April 9, 2015) 1. Disability AND attitude AND TI (attitude OR acceptance OR integration OR employment) TI 2. Total = 37 Search 13: Cinahl Science (April 11, 2015) Limits: 2005-2015, English, peer reviewed, abstract available, apply related words 1. Disab* AND (employ* OR work) AND (measure* OR questionnaire* OR factor* OR scale OR rat*) TI 2. Total = 73 234 Appendix D: SME Introductory Letter T H E U N I V E R S I T Y O F B R I T I S H C O L U M B I A May 21, 2016 RE: Request for your participation as a Subject Matter Expert (SME) in the development of the Co-worker Attitude Toward Disabled Employees (CADE) Scale My name is Jon Breen and I am a PhD candidate at the University of British Columbia. I am developing a questionnaire (scale) to measure co-workers’ attitudes toward disabled employees and am requesting your input into its content. Through an extensive review of the academic literature and by an evaluation of participants’ responses in many disability-related training sessions, I have generated an initial list of 40 items (questions) for inclusion in this proposed scale. My request, which I am submitting to you and a minimum of fourteen others, is that you will review each of these items and provide me with your rating and opinion of the value that each brings to the questionnaire. I will also ask you to provide me with any additional items that you believe would add to the sufficiency of the measure. I expect that this review will be completed within one hour. Please note that your review and critique of these initial items will provide the basis for the final version of this scale. This scale will ultimately be made available to a wide range of employers. I anticipate that three sets of benefits will accrue from the application of this scale. These include: providing a basis to gauge the quality of disability-related training that is delivered to various employers by measuring attitudes on a pre- and post-training basis; supporting comparative analyses between corporate attitudes and representation rates of disabled employees within specific organizations, and; providing a foundation for a more in-depth analysis of the relationship between co-workers’ attitudes and the success of disabled employees. School of Social Work 2080 West Mall Vancouver, B.C. Canada V6T 1Z2 Tel: (604) 822-2255 Fax: (604) 822-8656 www.swfs.ubc.ca 235 I have approached you specifically because I believe that this study will benefit by having your input. I have selected individuals who bring at least one of a variety of perspectives to issues of disability and employment. These include: your extensive interaction with employees through your provision of disability awareness training, or; your understanding of the major structural and functional components of the workplace, including their relationship to disability, either from an academic or practice perspective, or; your experience as a person with a disability who has participated in a variety of relationships with co-workers and who has observed the effects of disability on those relationships;. You will be asked to consider each of the items provided in terms of its relevance to a measure of acceptance by co-workers. Research has, for many years, assumed that a combination of three variables – legislative environment, corporate culture, and attitudinal/personal characteristics – have had the greatest influence on the success of disabled employees. However, most recently, data indicate that the degree of acceptance by others in the workplace is of the most significant influence. Although there are numerous scales which gauge attitudes of various groups toward people with various disabilities in various environments, there are few that have been intended for application specifically within the workplace. For this reason, I have directed my research to developing a measure of acceptance by co-workers as it applies specifically to an employment environment. Please note that I have located this measure of acceptance along a continuum of perceived difference. Therefore, for purposes of this study, I have described acceptance as “the degree of perceived difference attributed to disabled employees within a workplace environment”. In other words, I have predicated an inverse relationship between acceptance and perceived difference; the more perceived difference, the less acceptance. Each of the 40 items that has been drafted for your consideration is intended to be an example of a workplace-related circumstance which could be affected by degrees of perceived difference. Your input will ensure that the most appropriate of these items are included and that those that are unclear or unsuitable are rejected. When a final version of this questionnaire is compiled and administered, each item will be numerically rated by co-workers. By summing these scores, we will be able to calculate a range of acceptance levels. The Rating Form that you will receive categorizes each of the initial items within one of the four most common workplace areas of complaint by disabled employees – 236 hiring procedures, workplace performance, accommodations/adjustments, and supervision/discipline/dismissal. I have attempted to include items that address different aspects of each of these categories, and to provide clear and unambiguous statements. Your task will be to review these items and provide input from your particular perspective. Your role in this project will be to review approximately forty statements and indicate the degree to which you agree with that statement in terms of its relevance to the working definition of acceptance as described above. You will be asked to rate your level of agreement along a four-point scale (1=not relevant, 2=somewhat relevant, 3=quite relevant, to 4= extremely relevant). You will also be asked, where applicable, to provide brief narrative comments regarding your selections. Finally, you will be asked to consider whether any additional items could be included in order to provide a more complete representation of “the degree of perceived difference attributed to disabled employees within a workplace environment”. If you are able to participate in this project, or if you have immediate questions, please respond to my email address below at your earliest convenience. (Please include your complete mailing address.) I will forward you a copy of the proposed items and response forms, and an Informed Consent form at that time. (This project has been approved by the University of British Columbia Behavioural Research Ethics Board.) Finally, please note that I expect to be able to provide you with the above documentation and forms within two weeks of my receipt of your agreement to participate. I would request that the response forms are completed and returned to me within three weeks of receipt. I will include a return envelope with your package. Thank you very much for your consideration of this request. Sincerely, Att: Rating Form Informed Consent form Self-Identification form Return envelope 237 Appendix E: Informed Consent Form THE UNIVERSITY OF BRITISH COLUMBIA CADE SCALE CONSENT FORM Title of Study: The Co-Worker Acceptance of Disabled Employees (CADE) Scale: A Study to Gather Content Validity Evidence PRINCIPAL INVESTIGATOR: CO-INVESTIGATOR: School of Social Work and Family Studies School of Social Work and Family Studies University of British Columbia University of British Columbia 2080 West Mall 2080 West Mall Vancouver, BC V5N 2L1 Vancouver, BC V5N 2L1 Phone: 604-822-2255 Phone: 604-822-2255 Fax: 604-822-8656 Fax: 604-822-8656 Purpose Research suggests that the low rate of employment for people with disabilities is substantially due to the negative attitudes of co-workers toward this group. This study will gather statistical evidence to determine the questions to be included in a self-report questionnaire intended to measure those attitudes. This will be accomplished by providing an initial set of questions, generated through a prior literature review, to nine Subject Matter Experts (SMEs) who have experience with disability in the workplace, or with organizational structure, or with the provision of workplace, disability-related training. Each of the SMEs will rate each question for relevance and clarity on a form to be provided. These responses will be analyzed, with those exceeding predetermined levels of endorsement selected for the questionnaire. You are being invited to participate in this study because you fit within the inclusion criteria and definitions as outlined below. In addition, your expertise will add significantly to the value of this study. Inclusion Criteria You are over 19 years of age. School of Social Work 2080 West Mall Vancouver, B.C. Canada V6T 1Z2 Tel: (604) 822-2255 Fax: (604) 822-8656 www.swfs.ubc.ca 238 You can communicate in English. You are able to provide informed, voluntary consent. You meet the definition provided below for at least one of the three categories of Subject Matter Experts. Definitions Subject Matter Expert SMEs will have the level of skill and experience as described below in at least one of the following categories: Individuals who have developed and delivered organizational training programs related to disability-awareness in the workplace, disability management training, or related topics. Each will have delivered a minimum of 50 training sessions to employees of either public or private organizations. Individuals with expertise in organizational human resources, either from within academic or corporate environments. Each will have experience with research associated with the structure of the workplace environment, including as it pertains to equity concerns, or practitioner skills based on a minimum of five years employed at a Human Resources director level with private or public organizations. Individuals with disabilities who have been employed within the field of disability management, vocational rehabilitation, or disability-related vocational counselling for a minimum of five years. Each participant is requested to circle which of the above categories is most applicable. It should be noted that individual SMEs may fit within more than one of the above categories. If that is the case, those SMEs are requested to circle each category that applies. Disability A disability1 is a physical condition, a mental condition, or a health problem that restricts the performance of one or more of a person’s significant life activities2 for an extended period3. 1This would include examples such as: hearing, seeing, communicating, mobility, agility, pain, learning, confusion/memory, development, emotional/psychological, other. 2This would include examples such as: school, work, transportation, recreation, housing, family, relationships, other. 3This would be for peroiods longer than six months or on a recurring or intermittent basis. Study Procedures If you decide to participate in this study, please contact Mr. Breen at either the telephone number or email address provided above. You will be provided with any requested additional details of the study. You will also be advised when you should expect to receive additional background information regarding the study and a set of Response Forms that contain the initial questionnaire items for your evaluation. 239 Following your completion of these forms, you will be able to mail them back to Mr. Breen in the self-addressed, stamped envelope that will be provided with the background information described above. It is requested that you complete this task, and mail the completed forms, back to Mr. Breen within two weeks of receipt. Upon receipt of the completed forms, Mr. Breen will analyze the data received from all of the SMEs in order to develop a final structure for the Co-Worker Acceptance of Disabled Employees (CADE) Scale. Please note that this Informed Consent Form may be returned separately, following your determination and advice of whether you wish to participate in this study, or with the Response Forms. Potential Risks This study deals with a sensitive topic. Therefore, it has the potential to be emotionally distressing for you. However, this risk will be minimized by your professional experience with the topic and the researcher’s history of disability-related training, counselling and assessment. However, please note that you may choose to terminate your involvement at any time. Potential Benefits While the risk of emotional distress is acknowledged, you may benefit from this opportunity to share your knowledge and experience and to have this information contribute to this study. Your input into the development of the CADE Scale will be of future benefit to disabled employees and job seekers. Confidentiality As one of the participating Subject Matter Experts, your name and other identifying information may be included in subsequent reports or journal articles generated on this topic. However, all data related to your responses and comments provided on the Response Forms will be kept confidential. All response data will be stored in a secure facility within the School of Social Work at the University of British Columbia under the supervision of the Principal Researcher of this study. There is little potential risk associated with maintaining that confidentiality. Remuneration You will receive a gift certificate in the amount of $20.00 for the Starbucks Coffee Company to compensate you for your contribution of your time and expertise to this study. This honorarium will not be dependent upon the completion of your Response Forms. Contact for Further Information About the Study If you have any questions or would like any additional information about this study, please contact the Principal Researcher or the Co-Researcher at the contacts given above. Contact for Concerns About the Rights of Research Participants 240 If you have any concerns or complaints about your rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the UBC Office of Research Ethics at 604-822-8598 or if long distance email at RSIL@ors.ubc.ca or call toll free 1-877-822-8598. Consent Your participation in this study is entirely voluntary, and you may refuse to participate or withdraw from the study at any time. Your signature below indicates that you have received a copy of this consent form for your own records. Your signature indicates that you consent to participate in this study. _____________________________ _________________ Participant Signature Date ______________________________________ Printed name of the Participant signing above 241 Appendix F: SME Self-Identification Form THE UNIVERSITY OF BRITISH COLUMBIA Title of Study: The Co-Worker Acceptance of Disabled Employees (CADE) Scale: A Study to Gather Content Validity Evidence Subject Matter Expert Self-Identification Form Name: ____________________________________________ Organization: _______________________________________ (optional unless participation is on behalf of the organization) I do not require approval from my organization to participate in this study. I do require approval from my organization to participate in this study, and I have attached an approval letter from an authorized representative. Disclosure: I understand that my name may be included in a list of participating Subject Matter Experts with regard to this study in future journal articles or other publications. However, I also understand that no data that may reveal any of my responses or comments provided within the Subject Matter Expert rating forms, or through other verbal or written communications, will be made available to anyone other than to the two investigators of this study. Qualifications: As noted in my introductory letter, I am seeking input from Subject Matter Experts with experience and qualifications within at least one of the following categories. Please indicate below which of these apply to you. (You may select more than one): Extensive interaction with employees through your provision of disability awareness training sessions and workshops; School of Social Work 2080 West Mall Vancouver, B.C. Canada V6T 1Z2 Tel: (604) 822-2255 Fax: (604) 822-8656 www.swfs.ubc.ca 242 Experience as a person with a disability who has participated in a variety of relationships with co-workers and who has observed the effects of disability on those relationships; An in-depth understanding of the major structural and functional components of the workplace, including their relationship to disability, either as an academic or practitioner. Mailing Address & Contact Information Address: _____________________________________ Address: _____________________________________ City, Province/State: ____________________________ Country: ______________________________ Postal/Zip Code: ________________________ Telephone: ________________ Email: ___________________ _________________________ Date: _________________ Signature 243 Appendix G: SME Response Form 244 THE CO-WORKER ACCEPTANCE OF DISABLED EMPLOYEES (CADE) SCALE: A STUDY TO GATHER CONTENT VALIDITY EVIDENCE RATING FORM for Subject Matter Experts This document is a Rating Form that provides you with samples of items associated with four major dimensions of employment as these relate to disabled employees. These include - hiring, workplace performance, accommodations/adjustments, and supervision/discipline/termination. Each of these four sections lists ten items and asks you to provide a rating, from one to four, regarding the relevance of each of these items to that dimension of employment. You will also be asked to consider each of these items for clarity. That could include a consideration of ambiguity, colloquialism/jargon, or inappropriate or vague language. Again, you are asked to rate each item from one to four. Following each item, you are provided with a space to offer any comments and/or explanations regarding why you have rated each item as you did. There is also space at the end of each section for any comments that you may have regarding that specific dimension of employment. Of particular value would be whether you consider any aspects of that specific dimension to have been overlooked or underrepresented. If so, examples of items that may address these gaps would be helpful. A final section is also provided for you to comment about any experiences of disabled employees that you believe have not been sufficiently addressed within the framework provided. Any of these final comments or observations would also be greatly appreciated. REMEMBER: You are rating these items in terms of their applicability to these employment dimensions. Specifically, does each item describe a circumstance that may be affected by co-worker attitudes toward disabled employees. You are not responding in terms of whether you agree or disagree with each of these statements. [highlight in original] Upon completion, please mail this Rating Form, along with your signed Informed Consent Form and Identification Form, to Jon Breen in the envelope included with this package. 245 Section 1.1 THE HIRING PROCESS This section provides ten sample items that represent circumstances that may be influenced by co-workers’ attitudes toward disabled employees within the hiring process. (In this section, disabled job seekers are considered as equivalent to disabled employees.) There are three tasks associated with each item. First, please indicate the degree of relevance of each item to this dimension of employment by placing a checkmark in the appropriate box. Second, please indicate the degree of clarity of each item by checking the appropriate box in that section. Third, following each item, there is a space for any comments that you may have with regard to that item. These comments could be related to relevance, clarity, or any other considerations. Finally, at the end of this section is an opportunity for you to recommend any additional items that you believe will further illustrate the range of experiences of disabled employees within the hiring process. Relevance to “The Hiring Process” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 1 It would be fair if disabled job seekers were required to disclose the likely progression of their disability as part of the hiring process. Comments: 2 When hiring disabled employees, consideration should be given to how well they will be accepted by other members of the organization. Comments: 246 Relevance to “The Hiring Process” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 3 Job candidates with disabilities are less likely than others to have all of the skills necessary for technical positions. Comments: 4 A job seeker who indicates on an application form that he has a disability should include the medical diagnosis. Comments: 5 Job candidates with disabilities are more likely than others to exaggerate their skills in an interview. Comments: 6 An applicant for a job that is similar to yours has advised that she requires a disability-related accommodation for her interview. This could also indicate that she may not be able to satisfactorily perform all aspects of this job. Comments: 7 Job candidates with disabilities are less likely than others to have all of the skills necessary for physically demanding positions. Comments: 247 Relevance to “The Hiring Process” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 8 At a technology-oriented customer service desk, a “tech” with a visible disability would instill less confidence than one who was able-bodied. Comments: 9 I would be comfortable introducing a new disabled employee to my work group. Comments: 10 Job candidates with disabilities are less likely than others to have the emotional stability that is necessary to succeed at very stressful jobs. Comments: Section 1.2 RECOMMENDED ADDITIONAL ITEMS FOR “THE HIRING PROCESS” # Additional Items Comments/Explanations 1 2 248 Section 2.1 WORKPLACE PERFORMANCE This section provides ten sample items that represent circumstances that may be influenced by co-workers’ attitudes toward disabled employees within considerations of workplace performance. Please indicate the degree of relevance of each item to this dimension of employment by placing a checkmark in the appropriate box. In addition, please indicate the degree of clarity of each item by checking the appropriate box in that section. Again, at the end of this section is an opportunity for you to recommend any additional items that you believe will further illustrate the range of experiences of disabled employees within the hiring process. Relevance to “Workplace Performance” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 1 For purposes of receiving annual cash bonuses, employees with disabilities should be rated on the effort that they put in instead of how well they perform. Comments: 2 I would be concerned that working with disabled colleagues as part of my team could negatively affect my own performance evaluation. Comments: 3 Having a person with a disability as a member of my team would add extra burdens to my job. Comments: 249 Relevance to “Workplace Performance” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 4 I would be less comfortable reporting to a supervisor with a visible disability than to someone without a disability. Comments: 5 I would be less likely to become friends outside of our workplace with a disabled colleague than with others. Comments: 6 Working with disabled co-workers would distract me from my own responsibilities. Comments: 7 The inclusion of disabled employees can be detrimental to a project team. Comments: 8 Compared to others, disabled employees are more likely to miss work due to illness. Comments: 9 I would be more comfortable working with co-workers who did not have disabilities. Comments: 250 Relevance to “Workplace Performance” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 10 I would be concerned about a disabled co-worker’s ability to perform my job while I was on an extended vacation. Comments: Section 2.2 RECOMMENDED ADDITIONAL ITEMS FOR “WORKPLACE PERFORMANCE” # Additional Items Comments/Explanations 1 2 251 Section 3.1 ACCOMMODATIONS/ADJUSTMENTS This section provides ten sample items that represent circumstances that may be influenced by co-workers’ attitudes toward disabled employees regarding workplace accommodations/adjustments. Please indicate the degree of relevance of each item to this dimension of employment by placing a checkmark in the appropriate box. In addition, please indicate the degree of clarity of each item by checking the appropriate box in that section. Again, at the end of this section is an opportunity for you to recommend any additional items that you believe will further illustrate the range of experiences of disabled employees within the hiring process. Relevance to “Accommodations/Adjustments” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 1 If necessary, disabled employees should be entitled to have peripheral responsibilities within their job descriptions modified, even if this requires corresponding changes in others’ duties. Comments: 2 Employees with certain disabilities should be allowed to wear headphones to listen to music at work if this helps them concentrate. Comments: 3 I would have to be careful about my language so as not to inadvertently offend a disabled co-worker. Comments: 252 Relevance to “Accommodations/Adjustments” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 4 Disabled employees typically expect some degree of help in performing their day-to-day tasks. Comments: 5 It would be fair for an employee who requires an ergonomic chair to go to the top of our workgroup waiting list, even if others have been waiting longer for new chairs. Comments: 6 I would be unhappy about being requested by a supervisor to assist a disabled colleague with performing a brief, daily task. Comments: 7 My employer should make training available to managers and other employees to help us understand disability-related challenges within the workplace. Comments: 8 Supervisors should not have to be burdened with additional responsibilities associated with disabled employees. Comments: 253 Relevance to “Accommodations/Adjustments” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 9 The need to include disabled employees in workplace-sponsored social events limits opportunities for others. Comments: 10 If people with invisible disabilities require any special treatment in the workplace, they should explain their specific disability to co-workers. Comments: Section 3.2 RECOMMENDED ADDITIONAL ITEMS FOR “ACCOMMODATIONS/ADJUSTMENTS” # Additional Items Comments/Explanations 1 2 254 Section 4.1 SUPERVISION, DISCIPLINE & TERMINATION This section provides ten sample items that are intended to represent circumstances that are influenced by co-workers’ attitudes toward disabled employees within the supervision, discipline and termination process. Please indicate the degree of relevance of each item to this dimension of employment by placing a checkmark in the appropriate box. In addition, please indicate the degree of clarity of each item by checking the appropriate box in that section. Again, at the end of this section is an opportunity for you to recommend any additional items that you believe will further illustrate the range of experiences of disabled employees within the hiring process. Relevance to “Supervision, Discipline & Termination” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 1 People with disabilities are entitled to have different rules about their working hours if they p medical reasons. Comments: 2 I would feel more uncomfortable dismissing a disabled employee for poor performance than I would about another employee. Comments: 3 If criticized for poor performance, disabled employees are more likely than others to blame external circumstances. Comments: 255 Relevance to “Supervision, Discipline & Termination” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 4 Disabled employees should always be given additional opportunities to correct inappropriate behaviour. Comments: 5 The supervision of disabled employees, as compared to others, can be more difficult because of the potential for human rights claims. Comments: 6 As a supervisor, I would not know how to respond to a disabled employee who blamed a poor performance review on her disability. Comments: 7 I would feel more uncomfortable about having to discipline a disabled employee than I would about another employee. Comments: 8 Supervisors may have to neglect other workers in order to meet the needs of disabled employees. Comments: 256 Relevance to “Supervision, Discipline & Termination” Wording Clarity # Item (1) not relevant (2) somewhat relevant (3) quite relevant (4) very relevant (1) not clear (2) somewhat clear (3) quite clear (4) very clear 9 Small performance failures on the part of disabled employees should be overlooked if they appear to be making an effort. Comments: 10 The possible financial costs associated with providing accommodations for disabled employees could divert funds from other necessary expenditures. Comments: Section 4.2 RECOMMENDED ADDITIONAL ITEMS FOR “SUPERVISION, DISCIPLINE & TERMINATION” # Additional Items Comments/Explanations 1 2 257 SECTION 5.0 CATEGORICAL GAPS OR INSUFFICIENCIES This section provides space for your comments regarding how well you believe that the items/dimensions described above will measure the attitudes of co-workers across the broad range of employment circumstances experienced by disabled employees. Although each of the above sections provided opportunity for the inclusion of additional items to address possible gaps or under-representations within those sections, this final section is to allow for more broadly based comments. Specifically, I am seeking any input about whether you consider the above sections to have reasonably addressed the broad spectrum of employment-related experiences likely to be encountered by disabled employees. If not, please indicate any gaps or insufficiencies that require additional consideration. Topic/Category Comment Thank you very much for completing this questionnaire. Your participation is greatly appreciated. Regards, Name: ______________________________ Date completed: ______________________ 258 Appendix H: Review of full texts and additional references Results of Full Text Review (Database Searches & Additional References) ID# - Internal Search Code TM – Transferable Measure Reasons for Removal – (SE) Single Evaluation, (LitRev) Literature review, (-Dis) Not disability, (-Quan) Not quantitative, (-Dates) Outside of dates, (-Ages) Outside of age group, (-Emp) Not employment, (-Att) Not attitudinal # ID# Database TM Reason for Removal SE only LitRev -Dis -Quan -Dates -Ages -Emp -Att 1. R17 Add’l ref (-2205) TM 2. R12 Add’l Ref (-2005) TM 3. 346 Cinahl (2005+) Out x 4. 345 Cinahl (2005+) Out x 5. R18 Add’l Ref (2005+) Out x 6. R16 Add’l Ref (2005+) Out x 7. 296 Add’l Ref (2005+) Out x 8. R20 Add’l Ref (-2005) TM 9. R4 Add’l Ref (-2005) Out x 10. 311 Medline (2005+) Out x 11. R2 Add’l Ref (-2005) Out x 12. 360 PsycInfo (2005+) TM 13. 278 GoogleSch (2005+) TM 14. 331 PsycInfo (2005+) Out x 15. R19 Add’l Ref (2005+) TM 16. 359 PsycInfo (2005+) Out x 17. R16 Add’l Ref (-2205) TM 18. R5 Add’l Ref (-2005) Out x 19. R13 Add’l Ref (-2005) Out x 20. 333 PsycInfo (2005+) Out x 21. 354 PsycInfo (-2005) Out x 22. 362 PsycInfo (-2005) Out x 23. 332 PsycInfo (2005+) Out x 24. R22 Add’l Ref (2005+) Out x 25. R17 Add’l Ref (2005+) Out x 26. 300 Medline (2005+) Out x 27. R15a Add’l Ref (2005+) Out x 28. R12a Add’l Ref (2005+) Out x 29. 299 Medline (2005+) Out x 30. R1 Add’l Ref (-2005) Out x 31. 303 Medline (2005+) Out x 32. R3 Add’l Ref (-2005) Out x 33. 283 Medline (-2005) Out x 34. R10 Add’l Ref (2005+) Out x 259 35. R14 Add’l Ref (2005+) Out x 36. R15 Add’l Ref (2005+) Out x 37. R9 Add’l Ref (-2005) Out x 38. 357 PsycInfo (2005+) Out x 39. 327 PsycInfo (2005+) Out x 40. R11 Add’l Ref (2005+) Out x 41. 320 Medline (2005+) Out x 42. 361 PsycInfo (2005+) Out x 43. 330 PsycInfo (2005+) Out x 44. 304 Medline (2005+) Out x 45. 316 Medline (2005+) Out x 46. R12 Add’l Ref (-2005) TM 47. R6 Add’l Ref (2005+) Out x 48. 279 Medline (2005+) Out x 49. R21 Add’l Ref (-2005) TM 50. R23 Add’l Ref (-2005) Out x TOTAL 9 6 8 2 7 11 0 4 3 260 Appendix I: SME narrative responses Section One – The Hiring Process The comments provided by SMEs are reproduced in this appendix in their entirety, with two exceptions. First, multiple comments that are identical in meaning have not been repeated. Second, comments that are directed to general topics of disability that are not related to the scale items have been withheld. All comments have been edited for spelling and grammar. 1. (H1) - It would be fair if disabled job seekers were required to disclose the likely progression of their disability as part of the hiring process. An employer should not hire based on a likely progression of disability. Depends on potential restrictions and likelihood of PWD being able to complete bona fide job requirements. You are using ‘disabled job seeker’ as opposed to ‘job seeker with a disability’. Approximately half of the interested disability population cases strongly support ‘people first’ language and the other half doesn’t care at all. An employer should not hire based on a likely progression – speculation. Depends on the potential restrictions and limitations and likelihood of the candidate being able to complete bona fide job requirements due to said restrictions/limitations. E.g. progressive disease, obtaining/maintaining a driver’s licence, job as a bus driver, and immediacy of potential or definite progression. I don’t know if “progression” is the best word. The term “prognosis” might be more appropriate here in forecasting the probable course and outcome of a medical condition. Firstly, we should not be using terms such as “disabled job seeker”. We are persons, job seekers or employees first; who happen to have a disability. This is critical to the presentation of any document pertaining to persons with disabilities. I suggest the text read: jobseekers with a disability, employee with a disability. Although I feel that this is a very relevant question for the survey, it screams human rights. I think this question will be answered differently with the person disclosed a “particular/specific” disability versus a general disability. I wasn’t sure what was intended in this question. Equally important is the “how” this is accomplished. Will everyone grasp what “likely progression” means? Key issue. But the language is unclear. I might say “other employees” rather than “members…”. It is conceivable that an employer could present disclosure as an option. How might the applicant feel about this? “Progression of their disability” is somewhat ambiguous in this context. It could progress toward greater or lesser impairment; which way it progresses could influence the fairness of disclosure. This question could relate to any employee. 261 2. (H2) - When hiring disabled employees, consideration should be given to how well they will be accepted by other members of the organization. Could not do it for racial minorities, ethnicities, or others. Why disability? We can’t check the prejudices of co-workers to determine who they like. Great question to gather information. Very relevant, especially if this is the first time a person with a disability has been hired for a position with an employer. If this question is about “fit” to the organization, it could be a generic question asked of everyone. However, if asking about acceptance of the disability by members of the organization, then the question is not relevant. Employers need to be able to build a cohesive team but it needs to happen without discrimination. One must assume that we have progressed far enough that this will not be an explicit disclosure. The phrase “how well they will be accepted” provided some challenge for me. Organizational cultural/personality fit can be a consideration for all hires (irrespective of disability), so maybe the question could focus on their disability. 3. (H3) – Job candidates with disabilities are less likely than others to have all of the skills necessary for technical positions. ‘People first’ language used here and not in Q1 above. If this question were to be asked, I’d be highly offended by it. Not true. They are either qualified or not. A reasonable accommodation is not evidence of skills. This question or statement is often the core of beliefs or attitudes held by non-disabled employees. What is particularly relevant is that in most cases, they lack access to the adaptive equipment and software required to do the job. Perhaps not singling out technical positions, perhaps needs to be broader. Substitute “candidates without disabilities” for “others” to be very clear. I have never observed co-workers distinguishing technical positions from other positions, so I don’t think it could be considered a relevant indicator. There may be some ambiguity in what constitutes a “technical” position. It may hide prejudice concerning non-technical positions. A more apposite example than Item 1.2, but on similar grounds. 4. (H4) - A job seeker who indicates on an application form that he has a disability should include the medical diagnosis. In this case only the effects of the disability on the ability to do the job is important. The diagnosis in itself does not do it. Pre-employment disclosure is a choice of the applicant with a disability. Only when safety of the applicant or others with whom s/he would work is an absolute job requirement can this be required to be disclosed. 262 Relevant due to perceived beliefs that co-worker has a “right to know”, specifically in cases of mental illness. “Specific” medical diagnosis? The employer does not have the right to know diagnosis but is entitled to know about restrictions or limitations. However, this statement does open up in the area of question that non-disabled employees often have - which is “If X has a disability, why don’t I (we) get to know the diagnosis? If it is not disclosed, how do we know if the disabled employee is really disabled or playing the system?”, reinforcing the attitude of non-acceptance in the workplace. So if the latter is true, then this statement is relevant but unclear in wording. If the candidate is disclosing they have a disability, the nature of the disability is very relevant. And “be required” following the word “should”. Need to address gendered language. Using “he” and “she” is a trigger for some - so reword. A disability expert will see the relevance but a positive response may not indicate a negative attitude. 5. (H5) - Job candidates with disabilities are more likely than others to exaggerate their skills in an interview. No evidence of this. It’s without merit. Everyone is different. Everyone exaggerates, with or without a disability. Excellent. Definitely a relevant statement for any hiring process. But I am having difficulty with the wording of the statement. I am not sure I understand what this is getting at. Not something I have come across. You may be wiser than me. 6. (H6) - An applicant for a job that is similar to yours has advised that she requires a disability-related accommodation for her interview. This could also indicate that she may not be able to satisfactorily perform all aspects of this job. No. It means she can satisfactorily perform with the accommodation. It might mean she cannot perform without it, but not always. Do you want to be gender specific? Bias? Upon the second and third reading, the correlation between the request for accommodation and perceived ability to perform is clear. Very relevant and prevalent, particularly in work units that lack of awareness regarding persons with disabilities. Long item. Not clear how this relates to acceptance. Only the second sentence seems relevant. This question is tricky. Unless they are in human resources, not everyone would know what a disability-related accommodation may be. Be specific with accommodation requirements and the job. Reword to eliminate reference to gender. I find this slightly more difficult to read but it is relevant. 263 7. (H7) - Job candidates with disabilities are less likely than others to have all of the skills necessary for physically demanding positions. What about emotional disabilities, or intellectual disabilities? Nothing to do with physically demanding jobs. This situation often requires workplace accommodation or customization of the tasks that are required for a specific position. This very much depends on the nature of the disability, not simply the position. A very general question. Not sure what you will be able to get/infer from the answer. Eliminate the phrase “physically demanding”. 8. (H8) - At a technology-oriented customer service desk, a “tech” with a visible disability would instill less confidence than one who was able-bodied. In an early (1980s) federal discrimination case it was held customer preferences for certain appearances/ gender, etc. would not be used as job criteria. Too specific to one job context. Less confidence in who? Customers, co-workers? Technology has propelled many advances for PWDs. Less ‘confidence’ is an example of attitudes. Anyone or anything that appears outside the norm, especially with respect to one’s comfort zone, will generate a feeling of unease and could instill questions regarding someone’s ability to perform a specific function. Pick a visible disability or give examples to enhance the wording clarity of this question. I don’t much like the term “able-bodied”. 9. (H9) - I would be comfortable introducing a new disabled employee to my work group. You are not stating “newly” disabled, just a new employee with a disability. The introducer’s comfort should not be a consideration under law, but it matters. Yes, but not because of the disability, because of the person. In my experience, few employees or managers would be comfortable. They would not know what to say, especially with visible disabilities. Disabled employee versus employee with a disability? What about “I would be comfortable introducing a new employee, who happens to have a disability, to my work group”. Very relevant and great directions/approach. However, this statement lacks clarity for me. Is this new employee part of your work group? And Very broad. A mental health disability may generate a different answer than a physical disability. This can vary depending on the personality type of the individual. 10. (H10) - Job candidates with disabilities are less likely than others to have the emotional stability that is necessary to succeed at very stressful jobs. Blatant discrimination if this is stated, written or implied. 264 Stereotype and wrong. Many PWDs have had to use emotional stability just to become a job candidate. Strong link to perception/attitude/beliefs. The statement is relevant as it is an assumption based on myth but one which could certainly swing the decision of the hiring process and affect the perceptions of co-workers. This might penetrate the PC barrier, I suspect. Additional Recommended Items and Comments from SMEs for Section One: Item SME Comment There are good jobs for people based on their disabilities. A common but wrong belief. PWDs should not mention their disability in the interview process. The reason accommodations are not offered is because PWDs are afraid to ask. Candidates who require accommodations during the interview process are given an unfair advantage A widespread belief. Candidates with Generalized Anxiety Disorder should not be given accommodations because everyone is anxious and interviews. I have heard several labor relations staff make this statement. Is there any merit and having a statement that is something like “Unless the job ad indicates preference for the disabled worker, PWDs need not apply”? The disabled person’s disability will make it difficult to fit in at work. Applicants with disabilities require different accommodations than other applicants. An item on pre-employment testing… It would be good to have an item around this topic as testing is often Step One before an interview. An item on salary… It would be useful to have an item around perceptions of salary and accommodations. For example, attitudes toward lower salary based on a belief that the PWD cannot do the “full job”. Job candidates with disabilities have an advantage in the hiring process due to human rights. Substitute this item for number five above. This is a more common prejudice voiced in the workplace. Hiring a disabled employee will make my workplace more interesting and rewarding. This would explore employees’ experience with PWDs. It is more expensive to employ people with disabilities and this is not fair. Disabled people should tell us they are disabled on their application. A time gap in a resume or application (if no Getting at previous work history or experience. 265 work) probably means the applicant has a disability. Regarding disclosure of applicants’ disabilities…. Include a statement relating to when an applicant should disclose their disability. E.g. at the application stage, during the interview, after being offered a position. I ensure that my hiring process encourages a diversity of applications. Does the person have any concept/knowledge of hiring a PWD? Section Two – Workplace Performance 11. (P1) - For purposes of receiving annual cash bonuses, employees with disabilities should be rated on the effort that they put in instead of how well they perform. Insulting. Work is not Special Olympics. How would we react if we change ‘disabled’ in this sentence to race, gender or ethnicity? This statement is very clear as to its intent and relevant to those with misperceptions as to how anyone should be treated within a work unit. I see the relevance but the item relies on qualitatively different treatment rather than adjustment (or we reward everyone for effort). Not all incorporated entities provide annual cash bonuses. 12. (P2) - I would be concerned that working with disabled colleagues as part of my team could negatively affect my own performance evaluation. This is exactly where the heart of the concerns/where discrimination occurs. Not me. Are we afraid they would bring the team down? Is this just referring to evaluation or also workload? This is relevant as an example of how negativity can be fostered in the workplace when individuals have little or no awareness and sensitivity to specific populations. Change “working with” to “having” will produce a broader concept that is more relevant. 13. (P3) - Having a person with a disability as a member of my team would add extra burdens to my job. No. In some cases a reasonable accommodation might change some duties but the term ‘extra burdens’ is extremely loaded. Very relevant. A common misperception. This item is similar to Item 2.2 above. I would choose one of these two. The threshold is probably higher than in Items 2.1 and 2.2. 14. (P4) - I would be less comfortable reporting to a supervisor with a visible disability than to someone without a disability. What if I said ‘less comfortable with a Black supervisor or a woman? We can’t let the bigotry of employees dictate hiring. 266 Communicating? Areas of concern? Negative items? Could this statement be turned around? “I would prefer reporting to a non-disabled supervisor then to a supervisor with a visible disability”. Extremely relevant in work units that have not had any exposure to working with persons with disabilities. Emotionally, I have a problem with this one. Hopefully, no one is so stupid as to think this and/or agree to it when finding to a questionnaire. This addresses attitudes/feelings directly but is it so relevant to “performance”? You could say “I would be less confident in being supervised by…”. Back to surface discourse. Excellent. 15. (P5) - I would be less likely to become friends outside of our workplace with a disabled colleague than with others. There might be legitimate reasons. But how would this be important to the job? Very relevant as performance can often be affected by the type of relationships that staff have developed with each other both in and out of the workplace. Not related to performance. As above in Item 2.4, I don’t like to think there are people like this around. These two questions reflect pure prejudice rather than a concern linked to the job. Maybe that is why I had such an emotional reaction to them. Subjective, could get a false judgmental response. 16. (P6) - Working with disabled co-workers would distract me from my own responsibilities. Only if my prejudices and fears were not resolved. Not the PWD’s fault. What if I said that working with a pretty girl would distract me? I have heard many people make this comment. 17. (P7) - The inclusion of disabled employees can be detrimental to a project team. No. If anything, it would add a different point of view, which is why we have teams. Because you reference “project team” does this implies strict deadlines? This perception, going into new or existing project, could have a definite effect on project success. Potential strength in numbers for someone otherwise unwilling to sound prejudiced. I wonder if this is too blunt. People would likely just lie. This is very broad/open to interpretation. If more specificity will provide greater insight, could say “detrimental to team efficiency/team unity/ team…”. 18. (P8) - Compared to others, disabled employees are more likely to miss work due to illness. Common belief. Not relevant until it happens to the few. Data dispute this fear. This appears to be a widely held belief. Another common misconception that certainly would affect performance if true. 267 Delete “due to illness”. 19. (P9) - I would be more comfortable working with co-workers who did not have disabilities. We see this as speculation until a co-worker works with them. Relevant as persons with disabilities are not the norm. There is an unknown to this. I’d be concerned about lying. 20. (P10) - I would be concerned about a disabled co-worker’s ability to perform my job while I was on an extended vacation. This would cause anxiety. Members of the team need to build a supporting atmosphere and be comfortable with the skill set of their peers. This likely varies depending on the nature of the disability and the nature of the job. Substitute “away from work” for “on an extended vacation”. Substitute “absence” for “vacation”. Additional Recommended Items and Comments from SMEs for Section Two: Item SME Comment “Having a disabled colleague in an identical position to mine lessens the value of my work.” Questions asking specifically about non-disabled employee’s perceptions of work when a disabled employee is hired/performs in same/like position. Additional items could be based on awareness and sensitivity. Performance of PWDs can be affected if there is not a level playing field to support a fair comparison. I can only suppose that the data that would emerge from such questions would provide a very dark view of the workplace if answered honestly by participants – which would be immensely valuable as baseline data pre-workshop and professional development. Having a person with a disability on our team will help us understand different perspectives and perform better as a team. Developing questions that reflect employee experience working with PWDs, either positive or negative, could impact treatment of new employees and make it easier or harder for them to succeed. I would be concerned that disabled employees would get special treatment when it came to promotion. This gets to the idea that PWDs would be treated differently because the company was afraid of being taken to court or need to up their disabled employee numbers. Section Three – Accommodations/Adjustments 268 21. (A1) - If necessary, disabled employees should be entitled to have peripheral responsibilities within their job descriptions modified, even if this requires corresponding changes in others’ duties. The Americans with Disabilities Act allows non-essential functions to be accommodated. If they cannot, it requires them to be waived. I would remove “peripheral” and leave it at “responsibilities” or insert “some” responsibilities if you want to get at this issue that non-essential duties are okay to modify by the core duties are not. A well informed, supportive workplace will rise to the occasion in these situations. The level of language is too complicated. I don’t think all people would understand “peripheral responsibilities” nor have a clear vision of what modifications would mean. I wouldn’t link it to peripheral duties. Difficult to understand. 22. (A2) - Employees with certain disabilities should be allowed to wear headphones to listen to music at work if this helps them concentrate. Yes. PTSD and some impairments are accommodated by this. Too specific to one condition and one kind of job. Give examples of “certain disabilities”. When the answer be different if the workplace had a policy of no headphones at work? What about safety considerations? Seems a bit specific. Find a more general example of accommodation. May not be relevant to just PWDs. Other workplace policies may exclude a true answer. 23. (A3) - I would have to be careful about my language so as not to inadvertently offend a disabled co-worker. No one should offend. But being afraid of all language is silly. This would mean I would feel bad about asking a person who is blind if they saw that report. Several co-workers have been afraid to mention anything having to do with “hearing words” in my presence. Extremely relevant. An excellent example in support of sensitivity and awareness training. Not clear how this relates to accommodation/adjustment. 24. (A4) - Disabled employees typically expect some degree of help in performing their day-to-day tasks. Not true. But people who believe it may use their belief to discriminate. Very relevant. Colleagues will be uncertain as to how this person with the disability will perform their tasks. I would move this question to the section on performance. 25. (A5) - It would be fair for an employee who requires an ergonomic chair to go to the top of our workgroup waiting list, even if others have been waiting longer for new chairs. 269 Yes. There is no federal law that gives reasonable accommodation the same status as ‘would like’. Should “medically requires” be added to this statement? Too specific to a particular condition and job. It seems that most office chairs are marketed as “ergonomic”. You may want to say “specialized ergonomic” or something to indicate this is not a normal chair. 26. (A6) - I would be unhappy about being requested by a supervisor to assist a disabled colleague with performing a brief, daily task. We see this as the problem with the non-disabled employee, not the PWD. These are feelings that are going to crop up in the workplace, should colleagues be asked to do something that is outside of their job description as well as possibly, their comfort level. Change to “continually assist”. I like the concept of the question but think the “brief daily task” should be fleshed out to be relevant. 27. (A7) - My employer should make training available to managers and other employees to help us understand disability-related challenges within the workplace. Yes. It resolves much of the fear and misunderstanding. Available training and mandatory training tend to be received very differently. I struggle with the words “disability-related challenges”. Is this about duty to accommodate? Supporting employees with training to understand what? Is this contaminated by the implementation of the tool in such a training session? 28. (A8) - Supervisors should not have to be burdened with additional responsibilities associated with disabled employees. What other parts of their job are burdens? The use of the word itself screams bad attitudes. It is relevant, as the supervisor and the people who report to him/her will be unsure as to how the potential workload could change the person with a disability on staff. But not necessarily to me if I am not a supervisor. 29. (A9) - The need to include disabled employees in workplace-sponsored social events limits opportunities for others. No, it means the company finds accessible locations for everyone. Great item to get it attitudes. I do not understand this. What “opportunities”? What kind of opportunities would be limited (give examples)? “Opportunities for others” is quite open-ended and perhaps not totally clear in at least two ways. What does “opportunities” refer to? Who “others” are. 270 30. (A10) - If people with invisible disabilities require any special treatment in the workplace, they should explain their specific disability to co-workers. ‘What’ is important. ‘Why’ is not. Either to co-workers or supervisors. Another very good item. Clarification of “invisible disability” could be important. Give examples of common invisible disabilities. Is the emphasis on disclosure (i.e. hiring) rather than the adjustment? I am not sure if everyone will know what “invisible disabilities” means. i.e. does it mean a fake disability or one that is not readily apparent/obvious? Additional Recommended Items and Comments from SMEs for Section Three: Item SME Comment Accommodations always cost a significant amount of money. I should have the same availability of accommodations as my disabled colleague. This gets to feelings of jealousy and a lack of understanding. I think it is unfair if we can’t hold a social event just because a colleague with a disability has to be included. This gets to the resentment of colleagues with disabilities. An item related to the impact on the workplace if more than one employee with a disability is employed. This could address a quota system becoming a cumulative burden. Develop an item that addresses the cost of accommodations. It would be interesting to have people with disabilities in my workgroup to broaden our thinking. If someone needs to work flexible hours or start later in the day? I would consider one or two items that deal with types of adjustment. If the office needs to be re-organized to accommodate…. that would be okay. Employers should not have to spend money to accommodate disabled employees. A person who requires adjustments to my workplace or computer systems is too disruptive. Addresses managing required change. Section Four – Supervision and Termination 31. (S/T1) - People with disabilities are entitled to have different rules about their working hours if they p(sic) medical reasons. (Note: should be “provide” - JB) Yes, if it’s a reasonable accommodation, not an undue hardship to the employer. 271 I think you are seeing if they have a medical reason for an alternative work arrangement. There is a missing word in this item. This item should be in the section on accommodations. 32. (S/T2) - I would feel more uncomfortable dismissing a disabled employee for poor performance than I would about another employee. This happens all the time. It can cause supervisors not to effectively supervise. Letting a PWD do poor work will cause the PWD to fail. Great! Social desirability will be in issue here (who will say “yes”?). 33. (S/T3) - If criticized for poor performance, disabled employees are more likely than others to blame external circumstances. “My disability made me do it!”. I read this all the time and non-disabled people see it as an excuse for bad behaviour. No. Only if the company is afraid to ask how the disability is impacting on performance. This is the start of the interactive communication process. This could say that PWDs are “more than likely to cite or indicate that the poor performance is really about the supervisor’s discomfort with PWDs.” A strong rationale for ensuring that all accommodations have been put in place prior to the person starting work. This seems more about PWDs’ own attributions rather than attitude about supervision/discipline or termination. Rewrite this item to begin with “If poor performances discussed with disabled employees…”. 34. (S/T4) - Disabled employees should always be given additional opportunities to correct inappropriate behaviour. Only if the behaviour is associated with the disability. Then the additional opportunities should be reasonable accommodations. This is relevant in situations where the nature of the disability is cognitive. In these cases, there is often a job coach or other resource in place. This would be the same for any employee. Not sure what you learn from this one. 35. (S/T5) - The supervision of disabled employees, as compared to others, can be more difficult because of the potential for human rights claims. Not if it’s done correctly. And who are ‘others’? Any protected groups in that bunch? And takes more time and effort? “Claims” should be changed to “complaints”. Add “or legal action”. 36. (S/T6) - As a supervisor, I would not know how to respond to a disabled employee who blamed a poor performance review on her disability. 272 The reason that they do not know is because they have not been trained, a failure of the employer. ‘Their’ versus ‘her’? Gender bias? 37. (S/T7) - I would feel more uncomfortable about having to discipline a disabled employee than I would about another employee. Yes. Common. The problem of a failure to discipline is a failure to supervise. Insert “disciplining” after “then I would about…”. This item could be combined with item 4.2 above. Both are a form of discipline. This could reflect discomfort or less capability. Some supervisors are not good at discipline. 38. (S/T8) - Supervisors may have to neglect other workers in order to meet the needs of disabled employees. This is what supervisors think, but it’s wrong. Should not happen. There is training, job coaches, etc. I would say “spent less time with supervising” versus “neglect”. I think you will get more honest answers with the former. 39. (S/T9) - Small performance failures on the part of disabled employees should be overlooked if they appear to be making an effort. Sets low expectations and justifies discrimination. Excellent! 40. (S/T10) - The possible financial costs associated with providing accommodations for disabled employees could divert funds from other necessary expenditures. Only if it causes undue hardship. Not relevant. Supports are in place to provide funding for workplace accommodations for persons with disabilities. Seems context/job specific. Should be in the section on accommodation. Additional Recommended Items and Comments from SMEs for Section Four: Item SME Comment Employees with disabilities and accommodations cannot be disciplined or terminated A common perception. PWDs require special supervision because of their disability. A more general statement. Develop an item specific to “termination”. An item about covers it human rights with respect to supervision. You need one regarding termination and human rights. 273 As a supervisor, it may be difficult to ensure that other team members treat the disabled employee properly. Someone who waits to disclose a disability until they are having difficulty performing (or in a performance review) should be disciplined or terminated. Not sure if this is needed but it is an issue that we hear about that has not been addressed. SME Comments Regarding Categorical Gaps or Insufficiencies: You have done a good job identifying beliefs and attitudes that make it difficult to succeed in the workplace. We need to address each one of these. PWDs who are also members of visible minorities experience twofold prejudices and extra hurdles to employment based on co-worker perceptions. Additional topics could include employee training, team building and change management related to the inclusion of PWDs, organizational development, and providing accommodations in training circumstances. I particularly like how you have identified the four key areas of employment situation and then, within these, created the questions that directly apply to each of those areas. The diversity and degree of severity of specific disabilities, particularly the “invisible” disabilities, could be addressed. Psychological diagnoses can be just as disabling and/or exacerbate the effects of other more visible disabling conditions these are often misunderstood. The topic of interpersonal inclusion could be expanded to encompass the social aspects of disability. My only concern is because the items are so close to the bone, then efforts will need to be made to ensure anonymity of participants – otherwise respondents might not be willing to answer honestly. The issue of “rights” could be further explored. If people believe that disabled people should “be on benefits” and leave work for the non-disabled. In addition, to what extent are views framed by people’s experience with disability outside of the workplace? More on concerns relating to the perceived costs associated with workplace accommodations, as this is a commonly heard issue.
UBC Theses and Dissertations
The co-worker acceptance of disabled employees (CADE) scale : a study to gather evidence of content validity Breen, Jonathon Steven 2018
Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.
- 24-ubc_2019_february_breen_jonathon.pdf [ 4.28MB ]
- JSON: 24-1.0373472.json
- JSON-LD: 24-1.0373472-ld.json
- RDF/XML (Pretty): 24-1.0373472-rdf.xml
- RDF/JSON: 24-1.0373472-rdf.json
- Turtle: 24-1.0373472-turtle.txt
- N-Triples: 24-1.0373472-rdf-ntriples.txt
- Original Record: 24-1.0373472-source.json
- Full Text