Arts and Social Sciences, Irving K. Barber Faculty of (Okanagan)
Psychology, Department of (Okanagan)
DSpace
UBCO
Khattar, Nataasha
Master of Arts - MA
University of British Columbia
Background: Humans are social and cultural beings. We all have racial, ethnic and social identities. It is thus crucial to integrate these into the practice of therapy and research. With the increasing rates of migration (and consequently diversity) all over the globe, multicultural society is likely to be a rule rather than an exception. This diversity will also reflect in the clinician’s offices, necessitating future clinicians to be multiculturally competent. Multicultural competence comprises of clinicians’ knowledge of their own cultural background, perceived notions and biases; clinicians’ efforts to understand and validate the worldviews of clients from different backgrounds; and clinicians’ skills to engage in culturally appropriate and sensitive therapy and assessments. Objectives: This research aimed to investigate the dimensionality of multicultural competence as there is some confusion in the literature. This was followed by adapting a precise scale based on items from previous measures using item response theory for measurement precision. Method: A sample of 165 graduate students in clinical, counselling psychology and social work programs were recruited. The data was collected primarily from North America and India. A scale for checking socially desirable responding was also included. Dimensionality analysis and item response theory were used to assess the psychometric properties of the existing multicultural competence scales. Network Analysis and exploratory factor analysis were then used to identify item clusters in order to derive the items for the new brief scale. Item response theory was conducted on this new scale with three factors. Implications: This research helps further our understanding of the dimensionality of multicultural self-efficacy and suggests pools of items that provide more precise measurement of the construct based on modern psychometrics.
INVESTIGATION OF DIMENSIONALITY AND DEVELOPMENT OF A PRECISE MEASURE OF MULTICULTURAL SELF-EFFICACY FOR COUNSELLORS by Nataasha Khattar B.Sc. Hons, Christ University, 2019 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE COLLEGE OF GRADUATE STUDIES (Clinical Psychology) THE UNIVERSITY OF BRITISH COLUMBIA (Okanagan) August 2023 © Nataasha Khattar, 2023 ii The following individuals certify that they have read, and recommend to the College of Graduate Studies for acceptance, a thesis entitled: INVESTIGATION OF DIMENSIONALITY AND DEVELOPMENT OF A PRECISE MEASURE OF MULTICULTURAL SELF-EFFICACY FOR COUNSELLORS submitted by Nataasha Khattar in partial fulfillment of the requirements of the degree of Master of Arts. Dr. Brian O’Connor, Irving K. Barber School of Arts and Sciences Supervisor Dr. Maya Libben, Irving K. Barber School of Arts and Sciences Supervisory Committee Member Dr. Soma Ganesan, Faculty of Medicine, UBC Vancouver Supervisory Committee Member Additional Committee Members include: Dr. Vijayalaya Srinivas, Christ University, India Supervisory Committee Member Dr. Jannik Eikenaar, Faculty of Applied Science External Examiner iii Abstract Background: Humans are social and cultural beings. We all have racial, ethnic and social identities. It is thus crucial to integrate these into the practice of therapy and research. With the increasing rates of migration (and consequently diversity) all over the globe, multicultural society is likely to be a rule rather than an exception. This diversity will also reflect in the clinician’s offices, necessitating future clinicians to be multiculturally competent. Multicultural competence comprises of clinicians’ knowledge of their own cultural background, perceived notions and biases; clinicians’ efforts to understand and validate the worldviews of clients from different backgrounds; and clinicians’ skills to engage in culturally appropriate and sensitive therapy and assessments. Objectives: This research aimed to investigate the dimensionality of multicultural competence as there is some confusion in the literature. This was followed by adapting a precise scale based on items from previous measures using item response theory for measurement precision. Method: A sample of 165 graduate students in clinical, counselling psychology and social work programs were recruited. The data was collected primarily from North America and India. A scale for checking socially desirable responding was also included. Dimensionality analysis and item response theory were used to assess the psychometric properties of the existing multicultural competence scales. Network Analysis and exploratory factor analysis were then used to identify item clusters in order to derive the items for the new brief scale. Item response theory was conducted on this new scale with three factors. Implications: This research helps further our understanding of the dimensionality of multicultural self-efficacy and suggests pools of items that provide more precise measurement of the construct based on modern psychometrics. iv Lay Summary Clinicians are expected to be competent in counselling clients from different racial and cultural backgrounds due to the increasing diversity of societies. Multicultural competence thus becomes a necessity and is characterized by skills, attitudes and knowledge. Multicultural self-efficacy is the concept used to capture the self-perceived multicultural competence of a clinician. The present study aims to examine the different aspects or dimensions of this concept. The second aim is to adapt a more precise scale on those different dimensions using existing scales. The goal is to improve our understanding of this concept and provide researchers and clinicians a better way to assess it in future studies. By doing so, we hope to contribute valuable insights to the field. This adapted scale will be a measure to capture the multicultural self-efficacy of students in graduate training programs. v Preface The Behavioural Research Ethics Board of the University of British Columbia’s Okanagan Campus granted ethics approval for this research. The certificate approval numbers for the project are H22-02798 and PAA #: H22-02798-A001. The results of this study have not yet been published. vi Table of Contents Abstract ..................................................................................................................................................... iii Lay Summary ........................................................................................................................................... iv Preface ........................................................................................................................................................ v Table of Contents ..................................................................................................................................... vi List of Tables ............................................................................................................................................ ix List of Figures ............................................................................................................................................ x Acknowledgements .................................................................................................................................. xi Dedication ................................................................................................................................................ xii Chapter 1: Introduction ....................................................................................................................... 1 1.1 Multicultural Competence ............................................................................................................ 2 1.1.1 Cultural Encapsulation ........................................................................................................... 3 1.1.2 Cultural Responsiveness ........................................................................................................ 4 1.1.3 Cultural Humility ................................................................................................................... 5 1.2 Need for Multicultural Competence ............................................................................................. 6 1.3 Existing Measures of Multicultural Competence ......................................................................... 8 1.3.1 MAKSS-CE-R ....................................................................................................................... 9 1.3.2 MCI ...................................................................................................................................... 10 1.3.3 MCCTS ................................................................................................................................ 11 1.3.4 MCKAS ............................................................................................................................... 12 1.3.5 CBMCS ................................................................................................................................ 13 1.3.6 SPMCS ................................................................................................................................. 14 1.3.7 MCSHPS .............................................................................................................................. 15 1.4 Limitations of Existing MCC Assessments ................................................................................ 16 1.5 Dimensionality of MCC .............................................................................................................. 18 1.6 Item Response Theory ................................................................................................................ 19 1.7 Rationale for this Research ......................................................................................................... 20 1.8 Present Study .............................................................................................................................. 21 Chapter 2: Methods ............................................................................................................................ 22 vii 2.1 Sample......................................................................................................................................... 22 2.2 Procedure .................................................................................................................................... 23 2.2.1 Recruitment .......................................................................................................................... 23 2.3 Measures ..................................................................................................................................... 23 2.3.1 Demographics ...................................................................................................................... 23 2.3.2 Questionnaire on Multicultural Self-Efficacy ...................................................................... 24 2.3.3 Balanced Inventory of Desirable Responding (Paulhus, 1991) ........................................... 24 2.3.4 HEXACO-60 (Ashton & Lee, 2009) ................................................................................... 25 2.3.5 Self report survey on Multicultural training ........................................................................ 26 2.4 Ethical considerations ................................................................................................................. 27 2.5 Analytic Methods ........................................................................................................................ 27 2.5.1. Dimensionality Analyses .................................................................................................... 27 2.5.2 Factor Analysis .................................................................................................................... 28 2.5.2.1 Exploratory Factor Analysis ............................................................................................. 29 2.5.2.2 Confirmatory Factor Analysis........................................................................................... 30 2.5.3 Item Response Theory ............................................................................................................. 32 2.5.5 Network Analysis..................................................................................................................... 35 2.5.6 Multidimensional Scaling .................................................................................................... 35 2.7 Design ......................................................................................................................................... 36 2.8 Data Analysis Plan ...................................................................................................................... 37 Chapter 3: Results and Discussion ........................................................................................................ 38 3.1 Sample Characteristics ................................................................................................................ 38 3.2.1 Psychometrics of Existing Scales ........................................................................................ 39 3.2.1.1 MAKSS-CE-R .............................................................................................................. 39 3.2.1.2 MCCTS ......................................................................................................................... 42 3.2.1.3 MCKAS ........................................................................................................................ 43 3.2.1.4 CBMCS ......................................................................................................................... 44 3.2.1.5 SPMCS .......................................................................................................................... 46 3.2.1.6 MCSHPS ....................................................................................................................... 47 3.2.3 Development of the New Scale............................................................................................ 49 3.2.4 Multidimensional Scaling .................................................................................................... 51 viii 3.3 Discussion ................................................................................................................................... 52 3.3.1 Findings for the Existing Scales .......................................................................................... 52 3.3.2 Development of the New Scale............................................................................................ 54 3.3.3 Dimensions of the New Scale .............................................................................................. 57 3.3.3.1 Skills ............................................................................................................................. 57 3.3.3.2 Knowledge .................................................................................................................... 58 3.3.3.3 Acceptance of Cultural Relativism ............................................................................... 60 3.3.4 Correlations with Other Variables ....................................................................................... 62 3.5 Limitations and Future Directions .............................................................................................. 64 3.6 Implications................................................................................................................................. 67 References .............................................................................................................................................. 110 Appendices ............................................................................................................................................. 118 Appendix A – MAKSS- CE-R ........................................................................................................ 118 Appendix B - MCI .......................................................................................................................... 122 Appendix C - MCCTS .................................................................................................................... 123 Appendix D – MCKAS ................................................................................................................... 125 Appendix E – CBMCS.................................................................................................................... 130 Appendix F - SPMCS ..................................................................................................................... 132 Appendix G - MCSHPS .................................................................................................................. 134 Appendix H – BIDR ....................................................................................................................... 135 Appendix I – HEXACO – 60 (Honesty-Humility dimension) ....................................................... 138 Appendix J – Informed Consent for Participants ............................................................................ 139 ix List of Tables Table 1. Summary of Existing MCC Scales .................................................................................................. 69 Table 2. Participant Sociodemographic ....................................................................................................... 70 Table 3. Schooling Related Characteristics .................................................................................................. 74 Table 4. Scales' Psychometric Properties ..................................................................................................... 75 Table 5. Dimensionality Analyses for all Scales ........................................................................................... 76 Table 6. Fit Coefficients for the Original Factors of each scale .................................................................. 77 Table 7. Dimensionality Analyses for Factors in the New scale .................................................................. 78 Table 8. Items for the New Scale .................................................................................................................. 79 Table 9. Correlation among the factors in the new scale ............................................................................. 81 x List of Figures Figure 1. IRT plot for MAKSS-CE-R Awareness .......................................................................................... 82 Figure 2. IRT plot for MAKSS-CE-R Knowledge ......................................................................................... 83 Figure 3. IRT plot for MAKSS-CE-R Skills ................................................................................................... 84 Figure 4. IRT plot for MCCTS Knowledge ................................................................................................... 85 Figure 5. IRT plot for MCCTS Awareness .................................................................................................... 86 Figure 6. IRT plot for MCCTS Definition of Terms ...................................................................................... 87 Figure 7. IRT plot for MCCTS Racial Identity ............................................................................................. 88 Figure 8. IRT plot for MCCTS Skills ............................................................................................................ 89 Figure 9. IRT plot for MCKAS Knowledge ................................................................................................... 90 Figure 10. IRT plot for MCKAS Awareness .................................................................................................. 91 Figure 11. IRT plot for CBMCS Multicultural Knowledge ........................................................................... 92 Figure 12. IRT plot for CBMCS Awareness of Cultural Barriers ................................................................. 93 Figure 13. IRT plot for CBMCS Sensitivity to Consumers ........................................................................... 94 Figure 14. IRT plot for CBMCS Socio-cultural Diversities .......................................................................... 95 Figure 15. IRT plot for SPMCS Cultural Skills ............................................................................................ 96 Figure 16. IRT plot for SPMCS Cultural Knowledge ................................................................................... 97 Figure 17. IRT plot for SPMCS Cultural Appreciation ................................................................................ 98 Figure 18. IRT plot for SPMCS Cultural Awareness .................................................................................... 99 Figure 19. IRT plot for MCSHPS Knowledge ............................................................................................. 100 Figure 20. IRT plot for MCSHPS Understanding of Terms ....................................................................... 101 Figure 21. IRT plot for MCSHPS Activity................................................................................................... 102 Figure 22. IRT plot for MCSHPS Awareness .............................................................................................. 103 Figure 23. IRT plot for MCSHPS Communication Skills............................................................................ 104 Figure 24. Network Plot for all Items ......................................................................................................... 105 Figure 25. Multidimensional Scaling Graph .............................................................................................. 106 Figure 26. IRT plot for New Scale – Factor 1 (Skills) ................................................................................ 107 Figure 27. IRT plot for New Scale – Factor 2 (Knowledge) ....................................................................... 108 Figure 28. IRT plot for New Scale – Factor 3 (Acceptance of Cultural Relativism) .................................. 109 xi Acknowledgements First and foremost, I want to thank my supervisor and mentor, Dr. Brian O’Connor – I simply would not be here in this program and doing this research that fuels my soul, had it not been for your consistent support, guidance and mentorship. You showed unwavering faith in my abilities and this project, and I could not have hoped for a kinder and more supportive supervisor. Thank you for the mentor and supervisor you are to me. I further owe my sincerest thanks to Dr. Vijayalaya Srinivas, who ignited my passion about research and mentored me as a person and a student for me to be where I am today. To, Dr. Maya Libben and Dr. Soma Ganesan, for their valuable feedbacks and inputs and their support for this project. I am so grateful for the wonderful committee, who believes in this project and this topic. Thanks to my mentors in the psychology department, Dr. Harry Miller who genuinely cares about me excelling in the program and about my well-being as well. Special thanks to Dr. Tareq Yousef, who with his passion towards Equity, Diversity and Inclusion issues, is making a big change in our department. Thank you, Dr. Yousef for guiding me in this project and for your course, it helped me personally and with my thesis. Thank you, Dr. Eikenaar, for invaluable inputs during the defence and for helping me refine my thesis. Deepest thanks to my mother and sister, who put up with all my crying calls and encouraged me to finish this thesis and give my best. I love you both so much, and I could not imagine doing this, had it not been for you. Thank you, Dad, for always showering your blessings from heaven, not a day goes by, when I don’t think of you. Thanks to my silly goose for all your support, you make living life easier. To my friends and colleagues, who shared the journey with me, and shared my joys and sorrows of research. xii Dedication Mumma, I dedicate my thesis to you – thank you for all the sacrifices, all the hard work and all that you have done, just to ensure you can fulfil Papa and your dreams for Nics and me. All my achievements are forever yours. Thank you for always making our house, a home. Aap se badh kar kuch nahi – I love you forever, and always. 1 Chapter 1: Introduction Multicultural competence is often referred to as the “fourth force” in the field of counselling and psychology. It was developed in 1991 as a response to the president of the Association for Multicultural Counseling and Development (AMCD) proposing a document identifying the growing need for a multicultural perspective in counselling (Arredondo, 1999; Sue et al., 1992). Pedersen (1989) emphasized that the ethical principles and guidelines in the American Psychological Association (APA) are dominated by the Western cultural perspective, and specific changes in the ethical guidelines are necessary to ensure that the minority clients’ interests and needs are met along with that of the majority groups. One of the historical models for explaining the increased rates of mental illness and non-adherence to treatment among diverse clients was the inferiority or pathological model. Based on this model, the minorities were seen as less evolved and deficient in the desirable genes compared to their White counterparts and thus more susceptible to psychopathologies (Sue & Sue, 1990). Another historical model called the culturally deprived model essentially blamed the culture for this problem and rejected the genetically deficient model in favour of the view that ethnic and cultural minorities do not have the “right culture” (Sue et al., 1992). These models propagated the notion that minorities are inherently pathological and eventually gave the health care professionals an easy way to not address these social inequities in the system (Sue & Sue, 1990; Sue et al., 1992). One of the significant areas of focus should be on the training and education of the psychologists in graduate studies, who are often not sensitized to the significance of various cultural factors that influence the quality of care (Pedersen 1989, Sue et al., 1992). This research aims to investigate and understand the dimensionality of multicultural competence as there is confusion about the dimensions in the literature, as shown by the existing measures. Another goal of the research is to enhance the measurement precision for the identified dimensions and to adapt a briefer scale using item response theory by using items from the previous measures. 2 1.1 Multicultural Competence Multicultural competence (MCC) is often characterized by three higher-order factors – attitudes (or beliefs), knowledge, and skills (Sue et al., 1992). Sue and colleagues (1992) justified the need to develop multicultural counselling competencies to meet the increasing global diversity. When clinicians use culturally sensitive modalities and define treatment goals according to the client's life experiences and cultural values, the therapy is more likely to be effective (Sue et al., 1992). Kirmayer (2012) claimed that cultural diversity presents certain obstacles to mental health services as culture influences how mental health is experienced and expressed. It also impacts the help-seeking strategies and how an individual may respond to treatment intervention. Sue (1990) expanded on multicultural research by examining how various non-western cultures may conceptualize mental health problems. Numerous minority groups which have historically faced systematic discrimination and oppression, often perceive the root of the problem to be external to societal forces rather than originating within them. Western cultures, on the other hand, tend to focus more on internalizing the problem and seeking solutions through a sense of personal agency. To work better with racialized clients, the clinician should investigate adopting active systems approach instead, which aims to tackle the environmental stressors actively. The clinician plays a more active role by providing strategies and suggestions for coping with difficulties. The MCC framework, as conceptualized by Sue and colleagues (1992), is comprised of a hierarchical model with three characteristics, each containing three dimensions. The characteristics are – (a) counsellor’s awareness of their own assumptions, biases and values; (b) counsellor’s understanding of a culturally different client’s worldview; and (c) developing culturally sensitive techniques and strategies. These characteristics further each have three dimensions – (a) beliefs and attitudes, (b) knowledge, and (c) skills. Through this research, we expect these three dimensions to emerge as we 3 systematically review the existing measures on MCC. We hypothesize this, as all the previous measures were developed on this three-dimensional model by Sue and colleagues (1992). Previous measures attempted to map the items on these three higher-order dimensions; however, that did not happen. The present research tries to resolve this by using appropriate methods. It is also important to note here that although Sue and colleagues (1992) called the dimension - "attitude", which was the original term; however, most researchers in the future literature have adopted the term "awareness". This was done by acknowledging that attitude could be multifaceted and thus harder to capture in self-reports. In the context of this research, we shall adhere to the term “awareness”. To better understand the emergence of this movement towards multicultural competence, we must first understand some schools of thought that guided the movement toward culturally sensitive training and techniques. These include understanding cultural encapsulation, cultural responsiveness and cultural humility. These schools of thought laid the philosophical groundwork for the development of multicultural competence. 1.1.1 Cultural Encapsulation In 1962, C. Gilbert Wrenn coined the term cultural encapsulation to capture the idea that healthcare professionals are bound by culture and time that influences their worldview and practices. Wrenn also explained how counsellors tend to avoid changes in their perception of the world based on cross-cultural exposure and the development of society over time (Bergkamp & Ponsford, 2020). Cultural encapsulation comes from a place of ignorance of other's cultures and the pervasive impact of culture on people's lives (McQueen, 2022). A culturally encapsulated counsellor would lack knowledge of other's cultures and how that influences their perceptions of the world. They would often rely on stereotypes for making judgments about racially and ethnically different clients and often ignore cultural differences instead of embracing them (Pedersen, 1989). Wrenn explained that being encapsulated 4 referred to being in a cocoon of imagined reality that serves a self-preservative function of protecting one's attitudes, beliefs and assumptions. Counsellors often ground their truth in cultural and generational factors that are often immutable in the face of change. This leads to counsellors generalizing their beliefs about their client's experiences irrespective of the differences in culture and generation (Bergkamp & Ponsford, 2020; McQueen, 2022). It is believed that counsellors often rely on their own experiences and shared assumptions of human behaviour in counselling settings; however, these assumptions may not be generalizable to all settings (Bergkamp & Ponsford, 2020). McQueen (2022) elaborated that a culturally encapsulated counsellor would rely on culturally inappropriate references from mythologies or traditions that would often not be relevant or understood by a client from a different culture. Cultural encapsulation not only impacts verbal aspects of conversation but also impacts how a counsellor may perceive a client's nonverbal cues like the client's body language and eye contact. This impacts the process of rapport building and the effectiveness of therapy (McQueen, 2022). Sue (1990) underscored the importance of non-verbal cues in a multicultural setup. He stated that non-verbal cues are unique to a culture and thus not universally interpretable. A clinician needs to be cognizant of the non-verbal cues as they reflect an individual's unconscious biases and trigger stereotypes an individual may hold (Arredondo, 1999; Sue et al., 1992). 1.1.2 Cultural Responsiveness Cultural responsiveness recognizes and respects other cultures and integrates that into practice and research (Kirmayer et al., 2016; Kirmayer et al., 2020). A culturally responsive counsellor considers people's cultural diversity and shows compassion and understanding towards people's cultural and ethnic backgrounds. They encourage people to express their cultures and consider the implication of culture on their mental health and well-being. It incorporates people's unique social, cultural and political situations and addresses the use of these orientations in clinical practice and research (Kirmayer et al., 2020). Incorporating this in diverse societies is paramount as this viewpoint encourages people to express their 5 culture in therapy and education (McQueen, 2022). Research has also shown that when the clients perceive the clinicians to be more culturally responsive, it results in overall better outcomes for the clients, and the clients are more satisfied with the treatment (Sue et al., 1992). Kirmayer et al. (2016) stressed that engaging in cultural self-reflectiveness is essential to becoming culturally responsive. This includes reflecting on their own cultural values, their relationship with their self-identity, and how their clients perceive them (Sue et al., 1992). This will also lead to the clinician becoming more culturally sensitive to the needs of their clients. However, because of the various cultures in the world, it is almost impossible for a clinician to have knowledge of every client's culture. A clinician should thus take a stance of cultural humility (Tervalon & Murray-García, 1998). 1.1.3 Cultural Humility Cultural humility is an approach based on which a healthcare professional takes a lifelong learning perspective, acknowledging the limits to a clinician's knowledge of a client's cultural worldview. It aims to foster an open dialogue and learn from the clients to harbour safe feelings in a multicultural clinical setting. This model is seen as a more sustainable goal in multicultural medical education (Tervalon & Murray-García, 1998). As explained by Tervalon and Murray-García (1998) in the field of medicine, the cultural humility model has a component of self-reflection and lifelong learning. It emphasizes realistic and continuous self-appraisal along with a dedication to engaging in continuous learning. The component of self-reflection involves being humble and accommodating enough to let go of any firmly held stereotypes about a community and assessing new cultural dimensions with the clients. It also includes a commitment on the part of the clinician to make a change in behaviour and attitude when necessary (Tervalon & Murray-García, 1998). Another significant component in multicultural training of clinicians includes self-knowledge and awareness (Sue et al., 1992). Understanding one's own cultural practices and orientation and understanding the power 6 dynamics inherent in the clinician and client relationship is crucial for developing a relationship of trust (Arredondo, 1999; Sue et al., 1992; Tervalon & Murray-García, 1998). Tervalon and Murray-García (1998) claimed that cultural humility eliminates the need to have absolute mastery over every community's health belief. Instead, this model aims to reduce the power distance by creating an atmosphere of learning and growth where the clinician can potentially learn from the client's health and illness stories. As per their conceptualization, competence is illustrated best through a clinician's humility, as they learn to recognize, believe in and expand on the adaptive strengths and resources of the communities. 1.2 Need for Multicultural Competence With the increase in diversity and migration across the globe, there is a large influx of racially and culturally diverse clients in the clinician's offices (Carpenter-Song et al., 2007). The increase in the population of cultural and ethnic minorities in North America will push the clinicians – (a) to be more culturally sensitive, (b) to increase their knowledge of different cultures and ethnic groups, and (c) to develop multicultural counselling skills (Sue, 1991). Sue and Sue (1990) emphasized that race, culture and ethnicity are not just spheres within which minorities exist, but instead are functions of every individual and their identity. Lee (2011) looked at the clinical significance of cross-cultural competencies. They conducted a review to understand if the clinician's cultural competencies lead to an improved treatment process and outcomes for clients from culturally diverse backgrounds. Based on their review, they found that cross-culturally competent therapists achieve better results for culturally diverse clients which are based on three factors – 1) stronger therapeutic alliance, 2) having access to the client’s inner working models and 3) presence of corrective emotional and interpersonal experiences. This improvement is displayed as an increase in client satisfaction with treatment (Constantine, 2002) and self-disclosure (Burkard et al., 7 2006) and a reduction in attrition rates (Wade & Berntein,1999). Constantine (2002) found that clients rated the clinicians higher on MCC if they had previously rated them high on traditional counselling competence; thus, there was a high overlap between the two. Racialized clients with better and more favourable attitudes towards counselling had higher satisfaction with the overall counselling experience. After taking this into account, their results showed that the perception of the clinician’s general competence contributed to the satisfaction ratings more than the attitudes toward counselling. Thus, skills like empathy, active listening and rapport building played a significant role in making the counsellors seem more multiculturally competent. Their research also showed that the racialized clients’ perception of their counsellor’s MCC mediated the relationship between their clinicians' general counselling competence ratings and their satisfaction with therapy. The clinical experience of the client when accessing mental health services is filtered through a cultural lens (Kirmayer, 2012). This can include the differences between the patient and the clinician's social and power distance, language, religion, and other intertwined aspects of an individual's cultural identity (Kirmayer, 2012). It can also lead to certain racialized groups facing health disparities due to a lack of shared meaning with the clinicians concerning health outcomes (Kirmayer, 2012; Lee, 2011). In recent years there has been a surge in racialized population and clinicians are interacting with various ethnic groups of clients who may hold varied care-seeking behaviours and expectations and may have limited English proficiency. Culturally diverse populations often bear the burden of untrained clinicians. They also have a disproportionately higher incidence of mental illness, limited access to care, and are often underrepresented in mental health research (Carpenter-Song et al., 2007). The importance of MCC comes into the picture here as a culturally competent psychologist would be sensitized to the sociopolitical factors that contribute to mental illness, lack of adherence and increased attrition among culturally diverse clients rather than blaming it on their culture (Carpenter-Song et al., 2007). 8 In the next section, we will review some of the existing measures on MCC and go through some of the methods researchers used when developing these scales. 1.3 Existing Measures of Multicultural Competence There are various measures in the field of MCC, and all these different scales have come up with different factors for conceptualizing and understanding the construct. Researchers who developed these scales have also relied on the schools of thought discussed earlier and the three-factor higher-order conceptualization given by Sue and colleagues (1992) to guide their items for the measures. However, despite relying on the same theories for the foundational groundwork and measuring the same construct, these existing measures have come up with different factors, adding to the ambiguity in the field. Hays (2008) recommended that future multicultural conceptualization should aim to incorporate issues of power and the role of oppression and privilege when working with diverse clients. MCC assessment is a multidimensional process and involves a continuous evaluation of clinicians working with diverse clients. There are numerous MCC instruments to measure a professional's MCC. These include – the Multicultural Awareness-Knowledge-and-Skills Survey Counselor Edition- Revised (MAKSS-CE-R; Kim, Cartwright, Asay, & D'Andrea, 2003), the Multicultural Counseling Inventory (MCI; Sodowsky, Taffe, Gutkin, & Wise, 1994), the Multicultural Counseling Competence and Training Survey (MCCTS; Holcomb-McCoy & Myers, 1999), the Multicultural Counseling Knowledge and Awareness Scale (MCKAS; Ponterotto, Gretchen, Utsey, Rieger, & Austin, 2002), the California Brief Multicultural Competence Scale (CBMCS; Gamst, Dana, Der-Karabetian, Aragon, Arellano, Morrow & Martenson, 2004), the School Psychology Multicultural Competence Scale (SPMCS; Malone, Briggs, Ricks, Middleton, Fisher & Connell, 2015), the Multicultural Competence Scale in Helping-Profession Students (MCSHPS; Hladik, 2014). These scales measure the self-perceived MCC as they are all self-report measures. More broadly, most of these instruments assess the three broader factors of MCC based 9 on Sue and colleagues’ (1992) conceptualization. Thus, they try to assess the clinician's skills, the extent of knowledge of the clinician and the attitudes (or beliefs) of the clinicians when working with multicultural groups (Hays, 2008). Other instruments that measure a practising clinician's multicultural competence are – The Multicultural School Psychology Counseling Competency Scale (MSPCCS; Rogers & Ponterotto, 1997) and the Cross-Cultural Counseling Inventory-Revised (CCCI-R; LaFromboise, Coleman, & Hernandez, 1991). However, the MSPCCS and the CCCI-R are supervisor-rated rather than self-report scales. 1.3.1 MAKSS-CE-R The MAKSS-CE-R (Kim et al., 2003) is a counsellor edition which was later revised and now comprises of 33 items in total. MAKSS-CE-R is originally based on MAKSS which is 60-item scale measuring the multicultural awareness, knowledge, and skills of a clinician (D'Andrea et al., 1991). The 33 items in MAKSS-CE-R consists of 10 items each for Awareness and Skills subscales and 13 items for the Knowledge subscale. This scale has Likert-type responses, with higher scores corresponding to greater competence. An exploratory factor analysis (EFA) was conducted on the responses from 180 graduate students in counselling courses across the United States, while the confirmatory factor analysis (CFA) was conducted 158 cases for MAKSS-CE-R. Kim et al. (2003) used principal components factor analysis and used scree plot for factor extraction leading to a four-factor solution. However, they chose to retain three factors for maximum interpretability after examining the results of factor solutions ranging from three to five factors. They used oblique rotation (Direct Oblimin) after specifying a three-factor solution for the second principal factor analysis. Their results indicated that the three-factor solution accounted for 29.8% of the total variance. The CFA was done on a separate dataset of 158 cases to establish the stability and construct validity of the three-factor solution. This was done using the maximum likelihood estimation method. The hypothesized structure consisted of one second order latent variable which represented the broarder construct of MCC and three first order latent variables 10 representing the three factors extracted from the EFA with each having 10, 13 or 10 indicators. Their results yielded fit indices of Comparative Fit Index, CFI = .96, Tucker-Lewis Index, TLI = .96, and Incremental Fit Index = .96. Thus, confirming the construct validity of the factor structure for their sample. The reliability of the scale was established using Cronbach’s alpha which indicated adequate reliability for their sample i.e., .82 for the entire scale. The construct validity of the MAKSS-CE-R was established using the MCKAS scale scores that showed a moderate correlation between the two scales (Kim et al., 2003). For criterion related validity, the researchers conducted a multivariate analysis of variance (MANOVA) to compare those who had taken a course on multicultural counselling compared to those who had not, and the results showed a significant multivariate effect. 1.3.2 MCI The MCI (Sodowsky et al., 1994) is a 40-item scale measuring a clinician's MCC. This scale yielded a four-factor solution, resulting in four subscales are Awareness, Skills, Knowledge, and Relationship. In the initial phases of the study, the researchers conducted EFA on a sample of 604 participants and the CFA was conducted on a sample of 350 participants. These participants were students and long-standing mental health practitioners from the United States. They obtained four factors, out of which three which were consistent with the Sue and colleagues’ (1992) model – skills, knowledge, and attitudes. However, they also identified a fourth factor reflecting the interpersonal relationships in multicultural counselling context. The four-factor oblique model was confirmed by CFA which showed high correlations among the factors. The researchers also established validity by demonstrating high inter scale correlations with other scales like CCCI-R, MAKSS and MCAS. The researchers also collected state and national level data and the relationships between EFA factors from the state sample were generalizable to the national level as indicated by the coefficients of factor congruence ranging between .75 to .87. In the CFA the researchers hypothesized a second-order factor which represents the broader construct and four first order factors that had further indicators (items) 11 under it. Based on their index of fit for the model using the national data, they found that oblique four factor model had the best fit based on multiple confirmatory indices. Some of their indices were – goodness-of-fit index, GFI = .84, adjusted GFI = .81, root-mean-square residual, RMR = .024. Based on comparison of different models, they concluded that a four-factor oblique model was the best fit with a higher order factor which captures the general multicultural competence factor. The scale showed adequate internal consistency reliability values (Cronbach's alphas) – .83 for skills, .83 for awareness, .79 for knowledge, .65 for relationship and .88 for the full scale (Sodowsky et al., 1994). 1.3.3 MCCTS The MCCTS (Holcomb-McCoy & Myers, 1999) is a 32 items instrument that consists of five factors – knowledge, awareness, the definition of terms, racial identity, and skills. This scale measures the MCC of a counsellor along with their perceptions of the adequacy of their training. The instrument is a 4-point Likert-type scale. The researchers used stratified sampling to ensure that people of different ethnicities are represented in the sample. Their sample size was 151, which was based on the membership of the American Counseling Association and represented all the regions of the US. Principal components factor analysis with an Oblimin rotation was done to determine the underlying factors in the scale which led to the five factors solution. The five-factor solution explained 63% of the total variance. The internal consistency (Cronbach’s alphas) for the factors – knowledge, awareness, the definition of terms, racial identity and skills were .92, .92, .79, .66 and .91 respectively. They also assessed various other questions like self-perceived competence ratings of participants on each factor which revealed that on average people felt most competent on awareness and definition of terms dimensions, and the least competent on racial identity and knowledge dimensions. Another question they addressed was the relationship between various demographic factors and the self-perceived MCC, and found that among education level, ethnicity, gender, age, and work settings, only ethnicity had a statistically significantly influence on knowledge, awareness, racial identity and skills. They also found 12 that the participants who had taken a course on MCC, had significantly higher levels of self-perceived MCC on racial identity and knowledge dimensions. Criterion validity of the scale was established by comparing graduates from accredited and non-accredited programs (Holcomb-McCoy & Myers, 1999). 1.3.4 MCKAS The MCKAS (Ponterotto et al., 2002) comprises 32 items that are based on a two-factor solution – knowledge and awareness. This scale is based on the revision of a pre-existing scale called Multicultural Counseling Awareness Scale (MCAS; Ponterotto et al., 1996). The items for the original scale (MCAS) were developed based on the three dimensions given by Sue and colleagues (1992). However, they found that the knowledge and skills items loaded together while awareness items loaded separately, giving the two-factor solution. Their initial sample for EFA included 525 students and professionals with counselling backgrounds. They conducted a principal component analysis with both orthogonal and oblique rotations. Scree plot indicated a three-factor solution, yet they concluded that the two extractions had very similar factor structure and loadings. They found that orthogonal (Varimax) rotation provided the most interpretable extraction. A CFA was conducted to confirm the factor solution of the revised scale – MCKAS, the researchers recruited a sample of 199 counsellors in training from the United States. They used confirmatory maximum likelihood factor analysis and compared three models – a null model, an orthogonal model (with no correlation between the two factors) and an oblique model (where the two factors were assumed to correlated). The various indices of fit showed that the orthogonal and oblique models were nearly equivalent and other fit indices like Root Mean Square Residual (RMSR) was around .09, thus signifying a mediocre fit. They rationalised that the reason for poor fit is due to inadequate sample size and then created item parcels (i.e., creating aggregate variables) to improve the sample size-to-variable ratio and thus improve the reliability of the measure. Post this, the indices of fit improved, leading to satisfactory fit for the two-factor solution, and they also found that 13 the oblique and orthogonal fits were almost identical. This two-factor solution accounts for 32% of the total variance. Convergent validity of the MCKAS was established using the MCI subscale scores. The internal consistency of the subscales was .85 for knowledge and .85 for awareness (Ponterotto, et al., 2002). 1.3.5 CBMCS The CBMCS (Gamst et al., 2004) is a 21-item multidimensional scale that consists of four factors – Socio-cultural diversities, awareness of cultural barriers, multicultural knowledge, and sensitivity to consumers. The items are rated on a 4-point Likert scale, with higher numbers indicating more competence. The researchers used previous scales to compile a list of all items and eliminated some items which had high correlations with social desirability measures. They followed this up by conducting two EFA and one CFA. The first EFA was done on a sample of 415 participants with all the remaining items from the pool of items, the purpose of this was to reduce the items by imposing strict exclusion criteria, for conducting the second EFA. They conducted principal component analysis resulting in an initial 25-factor solution. Based on scree plot analysis, they chose to retain four factors. Varimax rotation four factor principal component analysis was then run. This provided the best fit for the data with 27 items which accounted for 44.2% of the total variance. The second EFA was done after seeking content validation from a regional expert panel of 20 professionals who had been active members of an advisory committee related to cultural competence. These professionals were asked to rate the appropriateness of the item and the items which yielded some ambiguity about the appropriateness by more than half the expert sample, they were then eliminated, leading to a four factor 21-item scale. The second EFA was done using principal component extraction method and a Varimax rotation. This yielded a four-factor solution based on the Kaiser-Guttman retention criteria which accounted for 59% of the total variance. The internal consistency of the subscales, as measured by Cronbach's alpha, ranged from .75 to .9. Criterion validity was established by comparing the CBMCS 14 subscale scores with the MCI subscale scores. A final confirmatory maximum likelihood factor analysis was conducted on the four-factor solution with a sample of 414 for construct validity. The researchers tested four different models. Based on the various comparison indices of fit, the four-factor correlated model showed the best fit for the sample. The TLI and CFI yielded high goodness of fit (i.e., >.97). The discriminant validity between the factors was demonstrated by conducting intercorrelations between the factors which were all less than .9. The researchers later conducted an additional CFA to overcome the confounding factor of exposing all respondents to all the items from different MCC inventories, which could have biased their responding. The sample consisted of 366 mental health providers, using the same analytical strategy as earlier. They found that the comparison indices showed that the four-factor correlated model was superior to the other models (Gamst et al., 2004). 1.3.6 SPMCS The SPMCS (Malone et al., 2015) is a 45-item measure to evaluate the MCC of school psychologists, out of which 28 items contributed to the final factor solution. The scale identified a four-factor solution for the items: cultural skills, cultural knowledge, cultural appreciation, and cultural awareness. The study had a total sample of 312 graduate students enrolled in school psychology graduate programs. The items were based on the three domains given by Sue and colleagues (1992). Content validity was established by seeking expert ratings from nine experts in the field and the items were modified on their suggestions. For EFA, principal axis factoring with Promax rotation was used over principal component analysis to examine the shared variance versus the unique variance. The researchers justified using Promax rotation as it is the most ideal method when factors are correlated. They used parallel analysis to determine the maximum number of factors to be extracted, which yielded nine factors. However, looking at the scree plot, the researchers decided to extract four factors which explained 42% of the variance. The researchers eliminated a total of 13 items which failed to meet the minimum criteria for adequate factor loading or cross-loading. Four other factors were excluded as their 15 removal lead to a significant increase in the scale’s internal consistency. Conducting principal axis factoring on the remaining 28 items with Promax rotation led to a four-factor solution that explained 50% of the variance. The Cronbach's alpha for the entire scale was .92, while the alpha for the subscales ranges between .75 to .86 showing moderate to strong internal consistency. Only EFA was done for the scale, and the CFA and other validity measures are yet to be conducted. The researchers also looked at the extent of MCC training on the perceived MCC using a factorial multivariate analysis of variance (MANOVA). Their results showed that there were significant differences on cultural skills and cultural knowledge for having completed MCC training or course. Post hoc analyses also revealed that taking at least one multicultural or diversity course led to a significantly higher mean score. Students who had completed practicums with culturally and linguistically diverse clients, showed a significantly higher mean score than those who had not (Malone et al., 2015). 1.3.7 MCSHPS The MCSHPS (Hladik, 2014) consists of 20 items comprising five factors – knowledge, understating of terms, activity, awareness and communication skills. The purpose of this scale construction was to make a scale for students in the helping professions in central Europe. The researcher took previous scales like CBMCS, MAKSS and MSPCCS and translated it into the Czech language and reworded certain items to fit it better to the students’ context who had limited exposure to clients and created a pool of 58 items. Content validation was done by two experts in the field of MCC and multicultural education, based on which 7 items were removed due to the low agreeability of the experts about the appropriateness of those items, and 2 more items were removed due to difficulty and duplication. The sample included 363 undergraduate students from a university, who completed the 49-item version now comprising of the various items left from the different scales along with some new items. The researcher conducted EFA on 46 items (after removing 3 items which showed corrected item total correlation value less than .2). Using the scree plot, the researchers chose a 5-factor structure. They 16 used principal components analysis with orthogonal factor rotation using the Varimax norm method. These five factors accounted for 45.63% of the total variance and the Cronbach's alpha for the overall scale was .92. Each subscale's internal consistency (Cronbach's alpha) was between .68 to .85. The researcher removed 24 items which had factor loadings below .55, leading to the 22-item version of MCSHPS. This was followed by a CFA to test the structure of a five-factor model using parameters like GFI, adjusted GFI, CFI etc. to assess the goodness of fit. This led to removal of 2 more items based on their high residual value. The researcher used the same sample of 363 students to conduct the CFA. Their results indicated a good fit for the 20-item model and a five-factor solution (Hladik, 2014). An overview of all these scales with their factors and methods used is summarised in Table 1. 1.4 Limitations of Existing MCC Assessments There is a recognition and acknowledgement that the future clinicians should be trained to be multiculturally competent. However, there are still some deficiencies in the field which can act as barriers to training graduate students towards this goal. O'Hara and colleagues (2021) explained that the cornerstone of the lack of good research in the field of MCC is due to the lack of adequate and standardized measures in this domain. Despite the numerous scales on MCC, there is a lack of consensus on the dimensionality of the construct. Thus, we must examine these measures simultaneously in our study to understand if they indeed tap into the same underlying construct. Consequently, this paves the way for our research to evaluate these measures comprehensively and systematically so as to identify which factors go together and whether they are really measuring the same construct. Even though all these measures as summarised in the previous section are grounded in the tripartite conceptualization (attitude, knowledge, and skills) of Sue and colleagues’ (1992) works, yet the factor structures of these measures present different findings (Rogers & Ponterotto, 1997). One of the explanations for this disparity is the use of different methods to map out the dimensions using factor analysis. All these scales are based on mathematical techniques like factor analysis to derive the underlying constructs from the 17 variables in the data, however one of the acknowledged limitations of this technique is that it relies heavily on the quality of the variables used (Gorsuch, 2015). Factor analysis involves many stages of decision making like choosing a method for factor extraction, retention, and rotation, thus adding to the endless combinations possible. Therefore, potentially leading to varied results, which makes people suspicious as such different results can be obtained using the same data (O’Connor, 2000). A lot of the scales (like MCKAS, CBMCS, MSCHPS etc.) have used PCA with orthogonal rotation (Varimax), which is not an ideal method for factor extraction or rotation for the MCC construct. This is because the construct is assumed to have latent dimensions with uncorrelated factors. In social sciences, however, correlation between factors is more common, and thus using oblique rotation (Promax) makes more sense. Another reason for the lack of consensus is the use of less-than-ideal methods like Kaiser’s method, i.e., eigen value > 1 for the number of factors to retain. Identifying these as the primary issues for the lack of structure in the dimensionality of the construct, we used PAF with oblique rotation (Promax) as it assumes correlation between the factors and assumes error to exist, which again are more likely to happen in social sciences related research. Some other potential factors a researcher may consider in using a measure are the cost, the number of items, standardized and well-researched psychometric properties and the population used for norming (O'Hara et al., 2021). One of the significant limitations of the many existing MCC assessment tools is that they often have limited data available to evaluate the psychometric properties of the measures (Hays, 2008; Ponterotto et al., 2002). Also, many of them are not based on empirical validation of the items and are instead based on a committee consensus or expert validation (Gamst et al., 2004). The meaning of certain constructs and concepts vary across cultures, and it becomes crucial to be mindful of that when conducting MCC research in different cultures (O'Hara et al., 2021). Most of these instruments used predominantly White sample or were normed just in North America (D'Andrea et al., 1991; Gamst et al., 2004; Holcomb-McCoy & Myers, 1999; Malone et al., 2015; Ponterotto et al., 18 2002; Sodowsky et al., 1994); except for MCHSPS which was developed in Central Europe (Hladik, 2014). Although, to the researchers’ credit, most of these samples had some representation from different cultures and ethnicity, and some even tried to over-recruit from minoritized cultures. Another limitation of these instruments is the use of classical test theory (CTT) for the development of all these scales. With more advanced measures of scale construction like item response theory (IRT), scales can now be made to measure the latent underlying construct and improve the overall quality of assessment (Morizot et al., 2007). 1.5 Dimensionality of MCC The dimensionality of a measure is assessed using factor analysis. Factor analysis (FA) is a technique used in the field of psychology to develop measures of constructs which are represented by some factors. The process starts with the researcher coming up with certain items that represent the construct well as per their best guess. This is followed by testing these items on a set of participants, leading to derivation of some factors based on what items go together. The next step involves adding and deleting the appropriate items based on the results. This process is iterative till the researcher has certain factors that represent the construct (Tabachnick & Fidell, 2014). After selecting the variables, the correlation matrix is generated for each pair of variables. This R-matrix is then examined to extract variables which go together in a meaningful way (i.e., which correlate highly with a set of variables and do not correlate with other variables outside this set), thereby reducing the data (Field et al., 2013; Tabachnick & Fidell, 2014). Sometimes these factors are also rotated in the multivariate space to increase the interpretability of the data (Tabachnick & Fidell, 2014). However, the innate problem with this is that there is no benchmark to compare the solutions. This leads to the issue of different researchers coming up with various factors when measuring the same construct, which is prevalent in the MCC literature, despite it being based on the three-factor model given by Sue and colleagues (1992) – attitude, knowledge, and skills. In the current research, this was rectified by using state-of-the-art 19 methods to derive dimensions using the items from the existing measure. The goal here is to understand how these derived dimensions fit better together compared to the previous studies. 1.6 Item Response Theory Item response theory (IRT) is a model-based technique for measuring a person’s ability on a given construct. Instead of just relying on the summation of the scores on the items for calculating the total score, IRT essentially estimates parameters for each item that then aims at tapping into the person’s underlying ability of that latent construct (Yang & Kao, 2014). The assumption on which IRT is built, is that every individual has an underlying trait that leads to their response on an item. In CTT, the raw scores derived are representative of the individual differences between people, whereas in IRT, that is gauged by the relative position of a person on that latent trait (Morizot et al., 2007). Interpretations derived from scales that are based on CTT are dependent on the person’s characteristics and the test characteristics, i.e., if the test items are difficult, the person’s ability would appear lower and if the test items are easy, the same person’s ability would appear higher. Tests based on CTT as thus used for testing individuals who share similar characteristics to the sample the test was originally developed with (Yang & Kao, 2014). Also, in CTT, the reliability of the test is a function of the sample on which the test is normed, thus leading to poorer precision of scores with a varied samples in other independent conditions. While in IRT, reliability is an inherent property of the item and is thus not sample dependent (Morizot et al., 2007). Also, in IRT, the steepness of the curve of an item (discrimination) is representative of how well that item can differentiate between individuals endorsing that trait around that range. In contrast, in CTT, this is based on the item-test correlation or the magnitude of the factor loading of the items (Morizot et al., 2007). Due to some of these advancements of IRT over CTT, this research aimed at adapting a measure using IRT, as all the existing scales have relied on CTT. 20 1.7 Rationale for this Research The purpose of this research is to build an improved measure of multicultural self-efficacy by focusing on understanding the dimensionality of MCC, then using IRT to adapt the existing items from previous measures. This was done by pooling the items from existing measures and refining the measurement precision of each of these newly identified dimensions. One of the acknowledged limitations of this type of assessment is that it is difficult to gauge the actual skills of the clinician, and the scale would thus aim to tap into the self-perceived multicultural competence (Gamst et al., 2004). This also leads to the issue of social desirability as the responses of the clinicians/students in the healthcare profession may be influenced by the desire to seem competent (D'Andrea et al., 1991; Holcomb-McCoy & Myers, 1999). Hence, drawing from the works of recent researchers who recognized and addressed this limitation, we assess self-efficacy beliefs rather than skills (Gamst et al., 2004). The research also incorporated some validity measures like the Balanced Inventory of Desirable Responding (Paulhus, 1991) along with a set of survey items on multicultural training and experience with culturally and linguistically diverse clients to check for validity of the scale. As noted earlier in the limitations, the methods used in the past are not the most ideal for evaluating social constructs because they assumed uncorrelated dimensions. The first goal of this research is to use appropriate methods and techniques to understand the dimensionality of the construct of MCC. This was achieved by using dimensionality analysis. Furthermore, we assessed the model structures proposed by the original researchers using CFA. After this all the items from the existing measures were pooled in, and we used network analysis, mokken scale analysis, EFA, and IRT to create a new and hopefully better measure. The second goal of this research is to improve the measurement fidelity of these identified dimensions and remove some problematic and redundant items using IRT. 21 1.8 Present Study The present study aimed to understand the dimensionality of MCC and construct a brief and precise scale on multicultural self-efficacy using IRT to tap into the underlying trait of an individual. The literature on multicultural competence is unstructured. Some works have identified three factors, others have identified four factors, while others have only studied it as a unidimensional model. This study incorporated this as a primary research question and looked at the structure of the construct using dimensionality analysis and network plot. For this study, we used the Multicultural Awareness-Knowledge-and-Skills Survey Counselor Edition- Revised (MAKSS-CE-R; Kim, Cartwright, Asay, & D'Andrea, 2003), the Multicultural Counseling Competence and Training Survey (MCCTS; Holcomb-McCoy & Myers, 1999), the Multicultural Counseling Knowledge and Awareness Scale (MCKAS; Ponterotto, Gretchen, Utsey, Rieger, & Austin, 2002), the California Brief Multicultural Competence Scale (CBMCS; Gamst, Dana, Der-Karabetian, Aragon, Arellano, Morrow & Martenson, 2004), the School Psychology Multicultural Competence Scale (SPMCS; Malone, Briggs, Ricks, Middleton, Fisher & Connell, 2015), the Multicultural Competence Scale in Helping-Profession Students (MCSHPS; Hladik, 2014). 1 1 An additional goal for the study was to use a confirmatory factor analysis (CFA) to assess the degree of measurement invariance across the North American and Indian samples (Brown & Moore, 2012). However, sufficient data for such analyses were not available and so the issue will not be addressed in this thesis. 22 Chapter 2: Methods Adaptation of a brief and precise multicultural self-efficacy measure was done in 2 stages. The first stage focused on conducting confirmatory factor analysis on the existing measures to see the model fit with the sample in our research. This was followed by looking at other psychometric properties of the existing measures and conducting dimensionality analysis to look at the number of dimensions for each of these measures. The second stage comprised of using visual analytical tools like network analysis to visualize the clustering of all the items from various measures. Item response theory was then used to adapt a more precise measure based on existing measures. 2.1 Sample The present study focused on a sample comprising graduate-level students enrolled in clinical, counselling, and social work programs. Participants were recruited online from Canadian universities following the requisite approval from the respective ethics board. Subsequently, a cover letter was sent via email to the department heads, requesting them to disseminate the initial contact letter and the questionnaire link to the graduate students within their departments. A similar process was adopted for the recruitment process in India, albeit limited to a single university. The rationale behind focussing on graduate students is that, while many programs include a component of multicultural counselling, it is often regarded as supplementary rather than a core requisite for course completion (Sue et al., 1992). Due to the low response rate from graduate students in both countries, the recruitment strategies were expanded to encompass online recruitment through social media platforms and expanded to worldwide recruitment. This approach aimed to diversify the participant pool and increase the chances of attaining the desired sample size. Furthermore, in order to enhance inclusivity and facilitate the attainment of the required sample size, professionals actively engaged in the field and recent graduates pursuing academic careers were also included in the recruitment process. 23 The participants were fluent in English, with the majority of the North American sample being Caucasian, and the Indian sample primarily consisting of Indo-Aryan people (see Table 2). The sample was predominately female, with 75.7% identifying as women. Other variables such as primary cultural and ethnic background are also reported. 2.2 Procedure 2.2.1 Recruitment The recruitment process involved getting ethical approval from different ethics boards from respective institutions. After getting the approval to recruit participants, the cover letter was emailed to the department heads/ assistants requesting them to distribute the same to their graduate students in the departments. This was done across 12 universities in Canada offering graduate level courses in clinical psychology, counselling psychology or social work. More participants were then recruited from online social media platforms like Reddit, Facebook and Twitter, using online poster, where participants clicked on the anonymous links to fill out the survey. Refer to Appendix J for the informed consent. The survey was developed using Qualtrics and the data was be stored on OneDrive to ensure that the data is stored in a safe manner. No identifying information was linked with the data. 2.3 Measures 2.3.1 Demographics Demographic information such as age, ethnic background, cultural background, racial background, province (or state) of residence (or continent), gender identity, sex, program enrolled in, program year, linguistic fluency etc. were collected from the participants. Additionally, other crucial variables were included, such as, whether the participants completed any coursework relevant to MCC, whether they received any training or attended seminars pertaining to MCC, and whether they have worked with clients from diverse ethno-racial backgrounds. The survey 24 also inquired about the program accreditation by a federal or national level body (like Canadian Psychological Association, American Psychological Association, or the Rehabilitation Council of India). The inclusion of these questions in the survey was driven by the relevance of these variables in the present research and their alignment with the thematic construct established in prior scales by researchers such as Malone and colleagues (2015) and Vereen and colleagues (2008). Vereen and colleagues (2008) found that some of the variables that impact the development of perceived multicultural competencies include factors like receiving clinical supervision pertaining to multicultural issues and conducting therapy sessions with racialized clients. Some scales have also concluded that with the increase in education and training with culturally diverse clients, the clinicians showed an increase in their self-perceived MCC scores, especially on subscales like skills and knowledge (Malone et al., 2015). Other researchers have also found the same results when they included validity checks like asking clinicians about their self-perceived MCC and self-perceived adequacy of their multicultural training (Holcomb-McCoy & Myers, 1999, Kim et al., 2003). 2.3.2 Questionnaire on Multicultural Self-Efficacy For the purpose of this research, we used all the relevant items from the various MCC scales. To avoid redundancy, the items which had the same content were removed, depending on the clarity of the items. The MCC scales that were used to create a pool of items for multicultural self-efficacy included – MAKSS-CE-R, MCCTS-R, MCKAS, CBMCS, SPMCS and MCSHPS. Overall, 25 items were deleted after carefully reviewing the repeated items. 2.3.3 Balanced Inventory of Desirable Responding (Paulhus, 1991) The Balanced Inventory of Desirable Responding (BIDR) is a 40-item self-report measure which acts as a validity scale to check for deceptive or socially desirable responding, which can particularly be an issue in self-report measures like this one (Hart et al., 2015). A large enough association between the 25 BIDR and the multicultural self-efficacy scale would suggest that the scale may not be entirely free from response bias, and therefore, the bias can be accounted for and controlled. Conversely, a small association would indicate that the multicultural self-efficacy scale is devoid of any response bias (Hart et al., 2015). The BIDR has two components: self-deceptive enhancement (SDE) and impression management (IM). SDE consists of 20 items and people that score higher on this believe they are answering honestly but are overestimating their abilities. IM also consists of 20 items, but this, on the other hand, measures deliberate attempts to present oneself in a more favourable way (Paulhus, 1991; Vispoel & Tao, 2013). The BIDR is a more advanced measure compared to other measures of social desirability responding as it captures both IM and SDR, while most scales capture one or the other (Vispoel & Tao, 2013). The BIDR uses a 7-point Likert scale ranging from 1 (not true) to 7 (very true), with higher scores indicating greater levels of socially desirable responding. 2.3.4 HEXACO-60 (Ashton & Lee, 2009) HEXACO framework represents the six broader dimensions that can be used to understand the broader structure of personality. HEXACO model has added the Honesty-Humility dimension to the famous big five-factor model of personality classification. The Honesty-Humility dimension is defined as characteristics like earnestness, sincerity, modesty etc. HEXACO model has been shown to have incremental validity over the classical five-factor model in several studies. HEXACO scale also showed higher criterion validity than the traditional five-factor model in many self-report measures like materialism, social adroitness, delinquency, and seductiveness (Ashton & Lee, 2008). The original scale, HEXACO-PI-R consisted of 100 items; however, a briefer version with 60 items called HEXACO-60 was developed. The internal consistency has been found to be high in both college and community samples ranging from .77 to .80 and .73 to .80 respectively. 26 For the purpose of this research, we used the Honesty-Humility dimension items from the HEXACO-60 scale, which comprises ten items focusing on facets like sincerity, fairness, greed-avoidance, and modesty. This dimension was added to this research because a significant characteristic of a culturally competent clinician is a sense of humility and earnestness to want to learn more about the client and their cultural and racial background and how it influences them and shapes their experiences. This humility measure was used in conjugation with the BIDR as it would be a way to tap into the social desirability responding from a personality lens. 2.3.5 Self report survey on Multicultural training The survey included some questions on multicultural training and experience with culturally and linguistically diverse clients. Previous scales have incorporated questions like “number of courses taken with a primary focus on multicultural and/or diversity issues”, “completion of practicum/training with culturally and linguistically diverse clients” etc. Vereen and colleagues (2008) found that some of the variables that impact the development of perceived multicultural competencies include factors like receiving clinical supervision pertaining to multicultural issues and conducting therapy sessions with racialized clients. Some scales have also concluded that with the increase in education and training with culturally diverse clients, the clinicians showed an increase in their self-perceived MCC scores, especially on subscales like skills and knowledge (Malone et al., 2015). Other researchers have also found the same results when they included validity checks like asking clinicians about their self-perceived MCC and self-perceived adequacy of their multicultural training (Holcomb-McCoy & Myers, 1999, Kim et al., 2003). 27 2.4 Ethical considerations Prior to data collection and commencement of the study, ethical approval of the proposed study protocols was sought from UBCO’s Behavioural Research Ethics Board. The certificate approval numbers for the project are H22-02798 and PAA #: H22-02798-A001. 2.5 Analytic Methods 2.5.1. Dimensionality Analyses Dimensionality analyses were conducted to identify the underlying latent factor structure by integrating converging evidence from multiple analytical procedures. Several techniques including parallel analysis (Horn, 1965), Velicer’s minimum average partial (MAP) test (1976), Hull’s method (Lorenzo-Seva et al., 2011), Kaiser’s method (1970), and Empirical Kaiser Criterion method were executed using the RAWPAR, MAP, HULL and NEVALSGT1 and EMPKC function respectively, from the EFA.dimensions package in the R programming language (O’Connor, 2018). Parallel analysis and MAP were selected as superior alternatives to traditional factor extraction methods, such as Kaiser’s (1970) eigenvalues-greater-than-1 rule and examination of Cattell’s scree plots. Both parallel analysis and MAP provide factor solutions that are statistically based rather than determined by mechanical rule of thumb (O’Connor, 2000). Hull’s method (Lorenzo-Seva et al., 2011) resembling nongraphical variants of Cattell’s scree plot, tries to identify an elbow point to determine the appropriate number of common factors. It relies on goodness-of-fit indices based on the model’s degrees of freedom for the new proposed model. Empirical Kaiser Criterion considers random sample variations of eigenvalues within Kaiser’s criterion and accounts for the distribution of eigenvalues in normally distributed data, for a sample level statistic (Auerswald and Moshagen, 2019). These estimates are then compared to the original number of dimensions reported by the developers of each scale. In the context of the current 28 study, this table will aid in determining the broader dimensions within the construct of multicultural competence for the new brief scale we are developing. 2.5.2 Factor Analysis Factor analysis (FA) is a technique used to identify groupings or clusters within a set of variables. It is a way of understanding the underlying patterns in the data based on the shared variance (Yong & Pearce, 2013). FA essentially serves three purposes – reducing data sets, understanding the structure of higher order constructs, and constructing a scale to measure a latent variable (Field et al., 2013). FA aims to reduce dimensionality in the observable data set based on the common variance and the purpose is to identify fewer latent or unobservable variables (Field et al., 2013; Yong & Pearce, 2013). Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) are two main FA techniques. EFA is used for exploring the dataset by uncovering the complex pattern inherent in it and is used for putting predictions to test. While CFA, is used for confirming a said hypothesis by representing the variables using path analysis (Child, 2006 as cited in Yong & Pearce, 2013). EFA is thus often used as the preliminary step when constructing a new scale and CFA is used to confirm the factor structure of the scale. When using EFA, researchers cannot extrapolate the results generated from one sample to another and thus the conclusions drawn about the factor structure are limited to the sample from which the original data is collected (Field et al., 2013). One of the limitations of FA is that naming of the factors can pose a problem as the cluster of variables is often named by the researchers based on what they think might reflect that common construct (Field et al., 2013; Yong & Pearce, 2013). This essentially causes a bigger problem when certain variables load on more than one factor (Yong & Pearce, 2013). Another limitation of FA is the use of various techniques for factor extraction, retention, and rotation, which may lead to varied factor 29 structures for the same underlying construct (O’Connor, 2000). This was consistent with our literature review of the various scales of MCC as well, as some scales yielded three or four factor solutions, while one scale even identified only one dimension to the scale. 2.5.2.1 Exploratory Factor Analysis Exploratory factor analysis (EFA) is a technique used to explore the data we have gathered. It is used to parsimoniously explain the covariation observed in the manifest variables (observed variables or items) by identifying the underlying hypothetical constructs (called as factors or dimensions; Goretzko et al., 2019; Watkins, 2018). EFA is done at the preliminary stages of the research when the researchers aim to generate a hypothesis and incorporate variables to represent a construct they are trying to measure (Tabachnick & Fidell, 2014). In the correlation matrix generated between the scores on the factors, the variability for any given measure can be divided into three components – common, unique and error variance. Common variance is the variance that an item shares with other items. Unique variance, as the name suggests, is the variance that is distinct to item and error variance is the variance that is also specific to one item, however this is not reliably attributed to just that one item. In FA, we are only interested in the common variance as we want to identify the common underlying dimensions within our data (Field et al., 2013). Three essential things to consider when conducting an EFA include – the extraction method, the factor retention criteria and the factor rotation method chosen to conduct the analysis. For the current study, we used principal axis factoring (PAF) for extraction of the factors. PAF is the preferred method when the measured variables are correlated and thus the assumption is that they are influenced by the same underlying construct. PAF tries to separate the total variance into common variance and unique variance (Watkins, 2018). Using PAF, the conclusions derived from the study will only be limited to the sample and in order to achieve generalization, the same factor structure should be revealed on a different 30 sample (Field et al., 2013). The next step deals with retaining factors that sufficiently account for the covariation between the measured variables, which requires a balance between parsimony and comprehensiveness. In the previous scales, this has been estimated by plotting scree plots of eigen values or using Kaiser’s method of eigen values greater than 1, as seen in Table 1. These methods are considered less than ideal because they are either subjective (like in the case of scree plot) or they are not accurate (like Kaiser’s method) and may not lead to the retention of the best factors, especially in the case of a construct like MCC. In research done by Auerswald and Moshagen (2019), they found that there is no single best method for factor retention. They investigated factors using Monte Carlo simulation with various conditions like one underlying factor, and multiple uncorrelated and correlated factors. Based on their study, they found that a combination of a few methods is superior in detecting the correct number of factors to retain over individual methods. Their general recommendation was to use a combination of methods like sequential 2 model tests (SMT) along with either traditional parallel analysis (PA), the empirical kaiser criterion (EKC) or the Hull method would be the most appropriate. Using these methods together allows for identifying the correct number of factors if these methods are converged. They also recommended using traditional PA, comparison data (CD) and EKC otherwise, in case the results from the combination rule are inconclusive and increasing the sample size to over 500 to allow easier detection of the correct number of factors to retain. For factor rotation, we used oblique factor rotation (Promax) as it allows for the correlation between factors. The purpose of rotation is to obtain factor structure where each variable is maximally loaded on single factor or as few factors as possible (Yong & Pearce, 2013). 2.5.2.2 Confirmatory Factor Analysis Confirmatory factor analysis (CFA) is a more nuanced technique to test a theory and is often used later in research to confirm a hypothesis. It is thus a tool used for validating an existing construct. 31 CFA is an advanced approach for construct validation as it adjusts for measurement error in the resulting estimates for convergent and discriminant validity (Brown & Moore, 2012). Just like EFA, CFA models are also specified by unique variances, factor loadings and factor variances. In CFA the model is declared with free and fixed parameters and a model is specified prior to the analysis. The specification of the model is based on the information that is known and the information that is to be estimated. There are three types of parameters – free, fixed, and constrained. Free parameter is the one which is unknown and free to vary, and the aim is to find its best possible value based on minimizing the difference between the observed and predicted variance-covariance matrix. Fixed parameter is one which is specified by the researcher prior to the research (often assigned a value of 1 or 0). Constrained parameter is like free parameter, in that it is unknown, however unlike the free parameter it is not free to vary but has some specific restrictions in place for the values it can assume (Brown & Moore, 2012). The original objective of our study was to employ CFA to assess measurement invariance between North American and Indian samples. The goal was thus to evaluate the measurement models across both samples and determine whether the factor structure differed between the two samples. However, due to the limited number of responses obtained on our survey, we adapted our approach and instead utilized CFA to calculate the fit coefficients. This allowed us to evaluate the different model structures proposed by the test developers for all the aforementioned scales. Our combined sample was then used to confirm these model structures and assess their alignment with the original proposed factor structures, proposed by the respective test developers for various MCC scales. Fit coefficients are used to assess the goodness-of-fit between the hypothesized model and the observed data. There are various fit indices that are commonly used to evaluate model fit in CFA. These included Comparative Fit index (CFI), Tucker-Lewis Index (TLI), Non-Normed Fit Index (NNFI), Incremental Fit Index (IFI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMSR). In general, for CFI, TLI, NNFI and IFI fit coefficients, the 32 relative improvement in fit by comparing the hypothesized model to a null model (Bentler 1990; Hu & Bentler, 1999; Tucker & Lewise, 1973). These values range from 0 to 1, with values closer to 1 indicating a better fit. A common guideline for good fit is often a value of .95 or higher. In general, for RMSEA and SRMR fit coefficients, the procedure estimates the discrepancy between hypothesized model and the population covariance structure. Smaller RMSEA and SRMR values indicate a better fit. Generally, a value of .08 or lower indicate a reasonable fit (Browne & Cudeck, 1993; Hu & Bentler, 1999). The lavaan package in the R programming language was used for computing the fit indices. 2.5.3 Item Response Theory Item Response Theory (IRT) is a model-based approach for identifying underlying latent traits estimated based on an individual’s responses to the items and the inherent property of the items (Embretson & Reise, 2000). Since multicultural self-efficacy can essentially be viewed as a latent construct that can be measured through manifested behaviour tendencies, thus an IRT scale may be a better measure of it, rather than using a scale constructed using CTT. IRT is based on mathematical models intended for psychometric analyses that offer improvement in the quality of measurement (Embretson & Reise, 2000; Morizot et al., 2007). The world of psychometrics has changed drastically since the introduction of IRT as the theoretical basis of measurement (Embretson & Reise, 2000). IRT is based on the probability of endorsing an item based on the level of latent trait present; this gives an estimate of the item response function (IRF). IRF is the fundamental unit to derive other psychometric properties. The IRF is essentially a curve which is estimated for each item of a scale and is specified by two parameters: location and discrimination (Morizot et al., 2007). The location reflects an item’s difficulty and indicates the level of latent trait required to have a 50% probability of endorsing an item in a keyed direction. In other words, the extent to which items differ in probabilities across trait levels is described by location. Thus, items with high positive locations need a greater amount of the underlying 33 trait to have a greater likelihood of endorsement. While items with negative locations, are even endorsed by people with a lower amount of that underlying trait. The location parameter of an item thus indicates where the information is maximally concentrated on the entire latent trait continuum (Morizot et al., 2007). The discrimination parameter, on the other hand, is the measure of an item’s significance to the underlying trait being measured, which is indicated by the steepness of an IRF slope (Morizot et al., 2007). Items with higher discrimination have a greater impact on the overall score as they are better at distinguishing people on the continuum at the point of inflection (Embretson & Reise, 2000). Therefore, a steeper IRF for an item indicates a greater significance of that item in measuring the underlying trait; thus, these are thus weighed heavier in determining an individual’s relative standing on the latent trait, and vice versa for flatter IRFs. These IRFs can then be transformed to give an item information function (IIF) which can be used to differentiate people at different trait levels (Morizot et al., 2007). The summation of these all IIFs, in turn, yields the scale information function (SIF), which indicates the accuracy of the measure at all levels of the latent trait (Morizot et al., 2007). These item parameters together (i.e., location and discrimination) together can then be used to produce information curves for each item. These are referred to as item information curve (IIC). The IICs are used to determine the relative importance of each item as it depicts the nature and contribution of each item to the overall latent trait being measured. Similar to IRFs, steeper IICs indicate items that are more precise in nature. To calculate the function for the entire scale, these IICs can then be added to yield test information function (TIF), which provides a measure of the precision across the entire scale. For the current study, IICs and TIFs were produced using the LTM package in R (Rizopoulos, 2013). These IICs graphs were plotted for all the subscales within each scale used in the study. The overall TIFs were generated as well, however since the scales were shown to be multidimensional, these graphs were thus, not deemed appropriate as there was no unidimensional underlying latent trait on the x-axis. 34 2.5.4 Mokken Scaling Analysis Mokken Scale Analysis (MSA) is a statistical method used to assess the scalability and the hierarchical structure of ordinal data, particularly for survey and questionnaire dataset. MSA helps to analyze the extent to which items in a scale relate to the hierarchical structure of a construct (van der Ark, 2012). MSA is a non-parametric technique considered to have evolved from Guttman scaling method, which is a scaling method that does not take random probability distribution into account. MSA, on the other hand, is a more evolved method, such that it takes these stochastic elements into account. MSA, thus then falls under the broad realm of IRT methods. MSA is an advanced version of Guttman scaling, as it incorporates more items from the pool of items into the hierarchical structure and provides greater insight into the function of an individual item (Watson et al., 2011). In MSA, the overall scores on the higher order scale, represent the extent to which they relate with the individual items in terms of probability (Sijtsma and Molenaar 2002 as cited in Watson et al., 2011), thus if a person scores higher on the latent trait, they are more likely to score higher on the item as well. However, it is important to acknowledge that the model also accounts for error and the possibility of the individuals responding in a counter-intuitive manner to certain items (Watson et al., 2011). Mokken analysis helps in evaluating the internal structure and the validity of a scale, helping the researcher by identifying poorly performing items, which then informs the researcher about the inclusion and the exclusion of items in the new scale (Sijtsma & van der Ark, 2007). Mokken analysis helps us answer the broader question if items form a single scale and whether the scale is unidimensional or multidimensional in nature. The Automatic item selection algorithm (aisp) is a statistical procedure that helps in partitioning items into different subscales based on the scalability coefficient, thereby increasing the homogeneity within each subscale. This is calculated based on the pairwise item correlations. Mokken analysis is touted as a more informative procedure when it comes to structural validity and conducting preliminary analysis as it takes into account the idiosyncratic response 35 patterns of participants (Dima, 2018). The scalability of the total scale, denoted by H, is evaluated using Loveinger’s H coefficients, while the scalability of individual item is demoted by Hi. The values for scalability coefficient (H) can range between 0 to 1, with higher values indicating higher scalability. Generally, H = .30 – .40 indicates weak scalability, H = .40 - .50, and H ≥ .50 indicates strong scalability as explained by Mokken (1971). 2.5.5 Network Analysis Network analysis is an analytical procedure has the ability to uncover the complex patterns among the variables. Network analysis was run using the qgraph package in R (Epskamp et al., 2020). Network analysis helps in visually representing the complex statistical relationships among the variables, without necessitating the initial data reduction (Epskamp et al., 2012). It is a psychometric tool that allows the evaluation and analyses of the patterns of complex relationships where the relationships are denoted by the edges and the nodes represent the variables (Hevey, 2018). The edges are representations of the strength of the correlation between the variables, thus the higher the correlation, the more saturated or darker is the edge (Epskamp eet al., 2012; Epskamp et al., 2018). In psychological research, the nodes could be various psychological constructs or variables, and the edge represents the strength and the magnitude of the relationships (e.g., correlations etc.) which can be visually plotted and examined (Hevey, 2018). We used network analysis in the current research to visualize the relationship between all the items and to understand the strength of the relationships between these items, pooled from all the existing scales that are used in the study. 2.5.6 Multidimensional Scaling Multidimensional Scaling (MDS) is a statistical technique used to analyze and visually represent the relationships between variables or entities based on their similarity or dissimilarity. It is a valuable tool for reducing complex multivariate data into a lower-dimensional space, making it easier to interpret 36 (Fitzgerald & Hubert, 1987). MDS can be applied to explore underlying structure or relationships (e.g., correlations, similarity judgements etc.) among various objects (e.g., people, places, things). In many ways, MDS shares similarities with factor analysis, as both methods aim to identify meaningful dimensions or patterns within the data and reduced bigger data sets into smaller dimensions. However, MDS unlike factor analysis is aimed at providing a spatial representation (Fitzgerald & Hubert, 1987). It thus allows researchers to observe and interpret these otherwise unnoticeable patterns within larger datasets by visually analyzing their distance from one another in the map (Steyvers, 2006). Overall, MDS serves as a valuable tool for data exploration and visualization, enabling researchers to gain insights into the underlying structure and relationships of their data. We used MDS in the current research to visually depict and explore the relationships between the three factors that were derived for our newly developed scale. Using this technique, we wanted to gain a better understanding of how these factors relate to one another and how they map onto the dimensions of existing scales. The MDS analysis was conducted using the smacof package in R (Mair et al., 2022). 2.7 Design We administered scales which hold the most relevance for our research purpose and use the Balanced Inventory of Desirable Responding (BIDR) to assess deliberate and non-deliberate socially desirable responding. The purpose of using BIDR is to ensure that the validity of the focal scale (multicultural self-efficacy scale) is not distorted due to self-favouring responses by the participants (Hart et al., 2015). After data collection, dimensionality analysis was done to understand the dimensionality of the different scales. CFA was used to assess the model fit for all the existing scales. This was followed by plotting item response functions for each item to look at how the probability of an individual endorsing an item, changes as a function of their level of latent trait (Morizot et al., 2007). All of this information was taken into consideration before developing the new scale. 37 2.8 Data Analysis Plan All data analyses were conducted in R, version 4.1.1 statistical software (R Core Team, 2021). All self-report measures were scored following the guidelines provided published articles or manualized protocols. In cases where duplicate items were identified (25 in total) and subsequently removed, the scores from the scales containing the duplicate items were added to calculate the total scaled scores. To address the issue where different scales had varying response option ranges, we utilized the RECODE function from the EFA.dimensions package to assign new range values to the items. For example, MCKAS had 7 response options, while most other scales had 4 response options. To circumvent this issue, we used the RECODE function for MCKAS items when they were added to various scales, wherein new values are assigned to all the response options, now ranging from 1-4, instead of the original 1-7. Our original goal for the research was to analyze the North American and Indian data separately. However, due to the limited responses we decided to combine the data and analyze them together, as most statistical procedures require a certain sample size to generate appropriate results. For the current study, we thus decided to report the demographics separately, as the demographic questions reflected the specifics of North American and Indian population. The participants completed the comprehensive online self-report survey items encompassing the aforementioned six scales on multicultural competence, in addition to demographic and validity indices. It is pertinent to note that, during the subsequent stages of data analyses, only complete responses were analyzed, owning to the intrinsic requirement of the specific analyses conducted. While there was a total of 165 respondents, there were numerous missing values for some participants. For the majority of the analyses necessitating complete cases, the effective sample sizes (Ns) notably deviated from the total count of 165. In certain cases, such as those involving responses to all items, e.g., in the context of IRT and other methods, the effective sample sizes were as low as 66. 38 Chapter 3: Results and Discussion This chapter presents the results and the discussion for the current study, which aimed to examine the dimensionality of multicultural competence. The chapter begins with a brief overview of the sample, research objectives and methodology to provide context for the findings. The results are then presented, followed by a comprehensive discussion and interpretation of the results. The chapter concludes with the limitations, future directions and the implications of the study. 3.1 Sample Characteristics The study included a total of 165 participants. Among them, 33 (20%) identified as men, 125 (75.7%) identified as women, 5 (3%) identified as non-binary, and 2 (1.2%) people preferred not to answer. The sociodemographic characteristics of the sample are presented in Table 2. The mean age of the sample was 28.79 years, and they had an average of 4.1 years of studies/training. In terms of racial background, the majority of the North American sample identified as Caucasian (63.3%), followed by 35.6% who identified as Asian. Regarding ethnic and cultural background, the majority participants identified as Canadians (61%), followed by South Asian (16%). Among the Indian sample, 62% identified with the Indo-Aryan race and 33.3% belonged to the Dravidian race. Indo-Aryan was the primary ethnic and cultural background for 57.7% of the participants, while 38.6 % identified with the Dravidian ethnic and cultural background. In the North American sample, 50% of the participants were monolingual, 33.9% were bilingual and 16.1% were multilingual. Among the Indian sample, 42.2% were bilingual and 57.8% were multilingual. Schooling-related characteristics are summarized in Table 3. Approximately 60.9% of the participants had completed some course work related to MCC, 62.6% had some attended some training or seminars related to it, and 81.1% had worked with clients from different ethnoracial backgrounds. 85% of the participants indicated that their graduate programs were affiliated by national or federal-level bodies. Although there were 165 respondents in total, there were 39 many missing values for some participants. The Ns were well below 165 for most of the analyses that required completed cases. The Ns were as low as 66 for analyses that required responses to all of the items. 3.2 Results One of the primary aims of this research was to explore the dimensionality of the construct of MCC by examining the existing scales. This was accomplished by exploring the psychometric properties of each of these scales which are presented below. 3.2.1 Psychometrics of Existing Scales 3.2.1.1 MAKSS-CE-R The MAKSS-CE-R (Kim et al., 2003) is a 33-item scale consisting of three dimensions: Awareness, Knowledge and Skills. Kim and colleagues (2003) utilized principal components factor analysis and a scree plot to extract factors, resulting in a three-factor solution at the end that maximized interpretability. In the present study, reliability tests were conducted using Cronbach’s alpha and McDonald’s omega, yielding values of .83 and .85 respectively, for the entire scale (see Table 4). The alpha values for the individual dimensions were .70 for Awareness, .84 for Knowledge and .80 for Skills, while the corresponding omega values were .68, .85 and .80, respectively. These findings suggest a slightly lower level of internal consistency within each factor compared to the overall scale. It is important to note that combining all items to create the entire scale can inflate alpha values, which may give a false impression of higher internal consistency. To assess the hierarchical and dimensional structure, Mokken analysis was performed, revealing an H value of .16 for the overall scale, indicating that the scale is broad in its scope and may not be unidimensional. Similarly, Awareness, Knowledge and Skills exhibited H values 40 of .24, .36 and .34, respectively. These values suggest that the Knowledge subscale demonstrates a general acceptable level of hierarchical structuring. However, considering that H values below .3 often indicate items that do not adhere to the hierarchical structure, it may be necessary to consider removing certain items during the final stage of developing the precise scale in the current study. This step along with IRT will help identify the most appropriate items for inclusion in the final version of the new scale. The dimensionality of the MAKSS-CE-R was further explored using Hull’s method, Empirical Kaiser Criterion method, comparison data, parallel analysis, MAP, and Kaiser’s (1970) eigenvalues-greater-than-1 rule (see Table 5). Across the sample in our study, these analyses consistently suggested a four to six factor solution, as indicated by methods like parallel analysis, MAP and Empirical Kaiser Criterion method and Hull’s method. Comparison data, however, resulted in a seven-factor solution. More traditional methods like Kaiser’s (1970) eigenvalues-greater-than-1 rule, proposed a ten-factor solution, which is generally considered a less reliable technique, as it simply returns the count of eigenvalues greater than 1. Parallel analysis, which indicated a five-factor solution, suggests that any factors extracted beyond this are likely to be as meaningful as factors emerging from random data, thereby making them statistically unreliable and unimpressive. Therefore, analyses suggesting more factors than the results obtained from parallel analysis, are more susceptible to over-extraction as they might be identifying relatively minor dimensions. CFA was conducted using the original model structure given by the test developers, Kim and colleagues (2003). The results indicated poor fit indices for MAKSS-CE-R for their three-factor solution. Table 6 provides the computed statistical fit coefficients for all the scales. The estimates for CFI, TFI, NNFI and IFI for MAKSS-CE-R were .90, .89, .89 and .90 respectively, indicating near-adequate model fit. These values fall below the general guideline of .95 or more, which indicates a good model fit. Additionally, the RMSEA and SRMR values were .11 and .13, respectively, further 41 confirming a less than adequate model fit. According to the general guideline, RMSEA and SRMR values less than .08 are considered indicative of a reasonable fit. The IRT analysis yielded several important statistics, including the item information curves (IIC), item discrimination parameter (slope) and item difficulty parameters (thresholds). In the interest of conserving space and focusing on the primary objectives of this thesis, detailed statistics from the IRT analysis are not reported in this study. Although these statistics provide us valuable information about the performance of individual items, their comprehensive presentation is beyond the scope of the current research. The results for each subscale (Awareness, Knowledge, and Skills) are depicted in the Figure 1, Figure 2 and Figure 3, respectively. The information curves in the figure illustrate the precision of the items along the latent trait continuum, Higher levels of information indicate items that are more precise and have better discriminatory power. In the case of MAKSS-Awareness, items 4 and 8 are informative almost all across the latent trait continuum compared to the other items within that subscale. Similarly, for MAKSS-Knowledge, item 1 shows greater precision, however at the lower to middle range of the latent trait continuum. Likewise, in the case of MAKSS-Skills, item 2 demonstrates higher precision, while other items are less informative. Conversely, flatter information curves suggest lower precision and limited ability to reliably discriminate individuals (with varying trait levels) along the latent continuum. Additionally, it is also important to consider the item information provided by each of these items (as indicated by the y-axis). The results of the IRT analysis, supported by the graphical representation, contribute to a deeper understanding of the measurement properties of the scale and its individual items. The most informative items, based on their precision across the latent trait continuum, in each subscale, were selected for the final stage of the precise scale development. 42 3.2.1.2 MCCTS The MCCTS (Holcomb-McCoy & Myers, 1999) is a 32 items instrument that consists of five factors – Knowledge, Awareness, Definition of terms, Racial identity, and Skills. Holcomb-McCoy and Myers (1999) utilized principal components factor analysis and a criterion for minimum eigen values to extract factors, resulting in a five-factor solution. Cronbach’s alpha and McDonald’s omega yielded values of .93 and .93 respectively, for the entire scale (see Table 4). These values indicate a high level of internal consistency across the entire scale. The alpha values for the were .89 for Knowledge, .82 for Awareness, .78 for Definition of terms, .73 for Racial identity, and .49 for Skills while the corresponding omega values were .89, .83, .79, .73 and .52 respectively. Mokken analysis revealed an H value of .37 for the overall scale. Similarly, Knowledge, Awareness, Definition of terms, Racial identity, and skills exhibited H values of .40, .49, .51, .65 and .29 respectively. The dimensionality analyses of the MCCTS scale consistently suggested a two-factor solution, as indicated by parallel analysis, comparison data and Empirical Kaiser Criterion method. Hull’s method, however, resulted in a one-factor solution, while MAP extracted a four-factor solution. More traditional methods like Kaiser’s (1970) eigenvalues-greater-than-1 rule, proposed an eight-factor solution. CFA was conducted to assess the fit of the original model structure given by the test developers, Holcomb-McCoy and Myers (1999). The results indicated good fit indices for MCCTS using the five-factor solution that was proposed by the test developers. The estimates for CFI, TFI, NNFI and IFI for MCCTS were .96, .95, .95, and .96 respectively, indicating a good model fit. Additionally, the RMSEA and SRMR values were .13 for both. 43 The results for the IRT analysis for each subscale (Knowledge, Awareness, Definition of terms, Racial identity, and Skills) are depicted in Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 respectively. The information curves in the figure illustrate the precision of the items along the latent trait continuum. In the case of MCCTS- Knowledge, item 6 exhibits higher precision at various points of the latent trait continuum compared to the other items within that subscale, while other items like 5,12 and 17 are also somewhat informative. Similarly, for MCCTS- Awareness, item 2 shows greater precision, however at the lower end of the latent trait continuum. Likewise, in the case of MCCTS- Definition of terms, item 4 demonstrates higher precision, at the middle range of the latent continuum, while item 3 is also somewhat informative. Item 2 exhibits higher precision at the middle to higher end of the latent trait continuum while item 1 does poorly within the MCCTS- Racial identity subscale. In the case of MCCTS- Skills, item 1 demonstrates higher precision at various points, while the other two items are less informative. 3.2.1.3 MCKAS The MCKAS (Ponterotto et al., 2002) consists of 32 items that are based on a two-factor solution – Knowledge and Awareness. Ponterotto and colleagues (2002) utilized principal components factor analysis and a scree plot, resulting in a two-factor solution. Cronbach’s alpha and McDonald’s omega yielded values of .87 and .88 respectively, for the entire scale (see Table 4). These values indicate a high level of internal consistency across the entire scale. The alpha values for the were .84 for Awareness and .92 for Knowledge, while the corresponding omega values were .85, and .92 respectively. Mokken analysis revealed an H value of .21 for the overall scale. Similarly, Knowledge and Awareness, exhibited H values of .23 and .56 respectively. 44 The dimensionality analyses of the MCKAS scale broadly suggested a two-factor solution, as indicated by comparison data, Hull’s method and Empirical Kaiser Criterion method. Parallel analysis, however, resulted in a three-factor solution, while MAP extracted a four-factor solution. Kaiser’s (1970) eigenvalues-greater-than-1 rule, proposed a nine-factor solution. CFA was conducted to assess the fit of the original model structure given by the test developers, Ponterotto and colleagues (2002). The results indicated adequate fit indices for MCKAS using the two-factor solution that was proposed by the test developers. The estimates for CFI, TFI, NNFI and IFI for MCKAS were .91, .90, .90, and .91 respectively, indicating an adequate model fit. Additionally, the RMSEA and SRMR values were .14 and .15, respectively. The results for the IRT analysis for both the subscales (Knowledge and Awareness) are depicted in Figure 9 and Figure 10, respectively The information curves in the figure illustrate the precision of the items along the latent trait continuum. In the case of MCKAS- Knowledge, item 2 shows greater precision, however at the lower end of the latent trait continuum, and items 1 and 3 are also somewhat informative around the lower end. Similarly, for MAKSS- Awareness, item 6 exhibits higher precision at various points across the latent trait continuum while it is not precise at the highest end. Other items like 5,12 and 17 are somewhat informative around the middle range across the continuum. 3.2.1.4 CBMCS The CBMCS (Gamst et al., 2004) is a 21 items instrument that consists of four factors –, Multicultural knowledge, Awareness of cultural barriers, Sensitivity to consumer and Socio-cultural diversities. Gamst and colleagues (2004) utilized principal components factor analysis and Kaiser-Guttman retention criterion and scree plot to extract factors, resulting in a five-factor solution. 45 Cronbach’s alpha and McDonald’s omega yielded values of .85 and .86 respectively, for the entire scale (see Table 4). These values indicate good internal consistency across the entire scale. The alpha values were .76 for Multicultural knowledge, .78 for Awareness of cultural barriers, .73 for Sensitivity to consumers, .79 for Socio-cultural diversities, while the corresponding omega values were .77, .75, .75, and .79 respectively. Mokken analysis revealed an H value of .28 for the overall scale. Similarly, Multicultural knowledge, Awareness of cultural barriers, Sensitivity to consumers, Socio-cultural diversities exhibited H values of .46, .43, .65 and .42 respectively. The dimensionality analyses of the CBMCS scale consistently suggested a two-to-four factor solution. MAP suggested a four-factor solution, while parallel analysis suggested a three-factor solution. Empirical Kaiser Criterion method, comparison data and Hull’s method, however, resulted in a two-factor solution. Kaiser’s (1970) eigenvalues-greater-than-1 rule, proposed a five-factor solution. CFA was conducted to assess the fit of the original model structure given by the test developers, Gamst and colleagues (2004). The results indicated adequate fit indices for MCKAS using the four-factor solution that was proposed by the test developers. The estimates for CFI, TFI, NNFI and IFI for MCKAS were .91, .90, .90, and .91 respectively, indicating an adequate model fit. Additionally, the RMSEA and SRMR values were .15 for both. The results for the IRT analysis for each subscale (Multicultural knowledge, Awareness of cultural barriers, Sensitivity to consumers, Socio-cultural diversities) are depicted in Figure 11, Figure 12, Figure 13, and Figure 14, respectively. The information curves in the figure illustrate the precision of the items along the latent trait continuum. In the case of CBMCS- Multicultural knowledge, items 2, 4 and 5 exhibit higher precision at various points in the latent trait continuum compared to the other items within that subscale. Similarly, for CBMCS- Awareness of cultural barriers, item 1 shows greatest 46 precision, however at the lower end of the latent trait continuum, while other items were barely informative. Likewise, in the case of CBMCS- Sensitivity to consumers, item 2 demonstrates higher precision at the lower to middle range of the continuum, while other items are less informative. Item 2 exhibits higher precision at middle range of the latent trait continuum, while item 5 is also somewhat informative compared to the other items within the CBMS- Socio-cultural diversities subscale. 3.2.1.5 SPMCS The SPMCS (Malone et al., 2015) is a 45-item measure to evaluate the MCC of school psychologists, out of which 28 items contributed to the final factor solution. The scale identified a four-factor solution for the items: Cultural skills, Cultural knowledge, Cultural appreciation, and Cultural awareness. Malone and colleagues (2015) utilized principal axis factoring with parallel analysis and scree plot to extract factors, resulting in a four-factor solution. Cronbach’s alpha and McDonald’s omega yielded values of .89 and .88 respectively, for the entire scale (see Table 4). These values indicate a high level of internal consistency across the entire scale. The alpha values for the were .84 for Cultural skills, .92 for Cultural knowledge, .80 for Cultural appreciation, .75 for Cultural awareness, while the corresponding omega values were .85, .92, .81, and .77 respectively. Mokken analysis revealed an H value of .30 for the overall scale. Similarly, Cultural skills, Cultural knowledge, Cultural appreciation, and Cultural awareness exhibited H values of .41, .72, .50 and .46 respectively. The dimensionality analyses of the SPMCS scale consistently suggested a two-factor solution consistently. Comparison data, MAP, Empirical Kaiser Criterion method, Hull’s method, and parallel analysis all suggested a two-factor solution. Kaiser’s (1970) eigenvalues-greater-than-1 rule, proposed a six-factor solution. 47 CFA was conducted to assess the fit of the original model structure given by the test developers, Malone and colleagues (2015). The results indicated good fit indices for SPMCS using the four-factor solution that was proposed by the test developers. The estimates for CFI, TFI, NNFI and IFI for SPMCS were .98, .97, .97, and .98 respectively, indicating a good model fit. Additionally, the RMSEA and SRMR values were .10 and .12, respectively. The results for the IRT analysis for each subscale (Cultural skills, Cultural knowledge, Cultural appreciation, and Cultural awareness) are depicted in Figure 15, Figure 16, Figure 17, and Figure 18, respectively. The information curves in the figure illustrate the precision of the items along the latent trait continuum. In the case of SPMCS- Cultural skills item 7 exhibits higher precision at the lower and higher end of the latent trait continuum compared to the other items within that subscale. Similarly, for SPMCS- Cultural knowledge item 5 shows greatest precision along the continuum, while items 3, 4 and 6 are also informative along the middle range. Likewise, in the case of SPMCS- Cultural appreciation, item 2 demonstrates highest precision at the lower end of the continuum, while other items are barely informative. Items 5 and 3 exhibit higher precision at the lower end of the latent trait and at a higher end of the continuum compared to the other items within the SPMCS- Cultural Awareness subscale. 3.2.1.6 MCSHPS The MCSHPS (Hladik, 2014) consists of 20 items comprising five factors – Knowledge, Understating of terms, Activity, Awareness and Communication skills. Hladik (2014) utilized principal components factor analysis and scree plot to extract factors, resulting in a five-factor solution. Cronbach’s alpha and McDonald’s omega yielded values of .86 and .86 respectively, for the entire scale (see Table 4). These values indicate good internal consistency across the entire scale. The alpha values for the were .80 for Knowledge, .61 for Understating of terms, .73 for Activity, .63 for 48 Awareness, and .66 for Communication skills while the corresponding omega values were .80, .65, .75, .73 and .66 respectively. Mokken analysis revealed an H value of .28 for the overall scale. Similarly, Knowledge, Understating of terms, Activity, Awareness and Communication skills exhibited H values of .44, .34, .46, .45 and .46 respectively. The dimensionality analyses of the MCSHPS scale consistently suggested a two-factor solution consistently. Methods like comparison data, MAP, Empirical Kaiser Criterion method, and parallel analysis all suggested a two-factor solution, while Hull’s method proposed a one-factor solution. Kaiser’s (1970) eigenvalues-greater-than-1 rule, proposed a seven-factor solution. CFA was conducted to assess the fit of the original model structure given by the test developer, Hladik (2014). The results indicated good fit indices for MCSHPS using the five-factor solution that was proposed by the test developers. The estimates for CFI, TFI, NNFI and IFI for MCSHPS were .94, .93, .93, and .94 respectively, indicating a good model fit. Additionally, the RMSEA and SRMR values were .11 and .12, respectively. The results for the IRT analysis for each subscale (Knowledge, Understating of terms, Activity, Awareness and Communication skills) are depicted in Figure 19, Figure 20, Figure 21, Figure 22 and Figure 23, respectively. The information curves in the figure illustrate the precision of the items along the latent trait continuum. In the case of MCSHPS - Knowledge, item 1 and 4 exhibits higher test information almost throughout the latent trait continuum compared to the other items within that subscale. Similarly, for MCSHPS - Understating of terms, item 4 shows some precision, however at the lower and the middle part of the latent trait continuum. Likewise, in the case of MCSHPS - Activity, only item 2 demonstrates some precision at the lower-middle range, while other items are less informative. Item 2 exhibits higher precision at the lower-middle range of the latent trait continuum 49 within the MCSHPS - Awareness subscale. In the case of MCSHPS - Communication skills, item 1 demonstrates higher precision however at the lower end, while other the other two items are less informative. Another goal of this research was to develop a new scale using modern psychometric tools. This was accomplished using visual methods like network plot, multidimensional scaling and item response theory. 3.2.2 Network plot Figure 24 depicts the results obtained from the network analysis. It shows the network plot for all the 141 items used in our survey, which were compiled from the various scales used in the study. In this case, nodes were represented by each item used in our survey and the edges were represented by the absolute correlation between the items. The green edges indicate a positive correlation between the items while the red edges indicate negative correlations. Examining the figure, it is evident that the items exhibit clustering, suggesting the presence of three larger dimensions and two minor dimensions. The higher the correlation between items, the darker the green lines are, that is the thicker and more saturated the edges (Epskamp et al., 2012; Empskamp et al., 2018). These three bigger clusters guided the selection of items for the final new scale. Items within each cluster shared similarities and common themes. 3.2.3 Development of the New Scale Items for the new scale were derived from the network plot analysis and EFA factor loadings with the Promax rotation, specifically related to the three identified factors. Subsequent dimensionality analyses were performed on these three factors to explore their underlying structure. Initially, a set of 20 items were derived from the first cluster in network plot analysis, forming factor 1. The preliminary 50 results of the dimensionality analysis, broadly suggested a one to two-factor solution, as indicated by comparison data, Hull’s method and Empirical Kaiser Criterion method and parallel analysis. However, MAP proposed a three-factor solution. To refine the factor structure, a preliminary IRT analysis revealed seven items which were not informative and were functioning poorly across the entire latent trait continuum. Following the removal of these seven items, another dimensionality test was conducted to assess the extent to which the first factor approached a unidimensional structure. Interestingly, all the aforementioned methods consistently indicated a one-factor solution (see Table 7). Moreover, the ratio of the first to second eigenvalue was approximately seven, indicating a dominant first factor. After thoroughly examining the items within this factor, factor 1 was named Skills. Furthermore, both Cronbach’s alpha and McDonald’s omega yielded high values of .94 for factor 1, indicating excellent internal consistency. Mokken analysis was employed to assess the scalability of the factor after the removal of the items. The analysis revealed a single strong factor with H values of .63, suggesting strong scalability and internal structure within the factor. Factor 2, derived from cluster 2 in the network plot analysis consisted of 25 items. The preliminary results of the dimensionality analysis broadly suggested a one factor solution, as indicated by comparison data, Hull’s method and Empirical Kaiser Criterion method and parallel analysis. However, MAP proposed a three-factor solution respectively. This was followed by a preliminary IRT analysis revealing three items which were not informative and were functioning poorly across the entire latent trait continuum. After dropping these items, another dimensionality test was conducted to assess whether the factor approached one-dimensionality (see Table 7). However, the results did not show improvement in the factor structure. Nonetheless, the three non-informative items were still excluded from the analysis based on the IRT analysis results. The ratio of the first to second eigenvalue was approximately five, confirming a dominant first factor. After thoroughly examining the items within this 51 factor, factor 2 was named Knowledge. Internal consistency reliability measures, including Cronbach’s alpha and McDonald’s omega indicated strong internal consistency for factor 2, with values of .93 and .94, respectively. Mokken analysis revealed one strong factor for factor 2 after dropping the three items, and yielded H values of .48, indicating medium to strong scalability. Factor 3 derived from cluster 3 in the network plot analysis comprised of 19 items. The preliminary results of the dimensionality analysis broadly suggested a one to two factor solution, as indicated by Hull’s method and Empirical Kaiser Criterion method and parallel analysis. However, MAP and comparison data suggested a three-factor solution. The preliminary IRT analysis revealed six items which were not informative and were functioning poorly across the entire latent trait continuum. After removing these six items, another dimensionality test was conducted to assess whether the remaining items formed a closer approximation to a unidimensional structure for factor 3. The results indicated a one to two-factor solution as revealed by all the aforementioned methods (see Table 7). Moreover, the ratio of the first to second eigenvalue was approximately five, confirming a dominant first factor. After thoroughly examining the items within this factor, factor 3 was named Acceptance of Cultural Relativism. The Cronbach’s alpha and McDonald’s omega for the third factor were .91 and .93, suggesting really strong internal consistency. Mokken analysis revealed one strong factor after dropping the three items, and yielded H values of .56, indicating strong scalability. Table 8 provides the factor loadings and the items for this newly proposed three factor scale. Correlations between these new factors are reported in Table 9. The final IRT plots for the new factors are shown in Figure 26, Figure 27, and Figure 28. 3.2.4 Multidimensional Scaling Multidimensional scaling was used to understand the spatial representation of these new factors with the scaled scores of different dimensions within the other scales used in our study. Preliminary 52 analyses indicated that the mspline estimation method was most appropriate for the data and that a two-dimensional mode provided reasonable fit. MDS helps reveal the variable relationships and the underlying structure of these new factors in relation to the scaled scores from existing scales. The MDS plot, depicted in Figure 25 aimed to reveal the distinct patterns. However, the MDS plot indicates a diffuse dispersal of the measures across the space, rather than distinct cluster or groupings of data points. For example, some supposedly similar scale (e.g., Knowledge) did not cluster together as closely as would be expected if the factors were measuring the same construct. 3.3 Discussion The primary aim of this study was to understand and work through the complex confusion in the dimensionality of the structure of multicultural competence, followed by developing a precise scale using modern psychometric methods like item response theory. 3.3.1 Findings for the Existing Scales The dimensionality analysis of the previous existing scales (see Table 5) provides valuable insights into the theoretical confusion surrounding the dimension structure of the broader construct of MCC in the literature. These findings reveal inconsistent factor solutions, ranging from the two to five factors, as evidenced by various methods such as parallel analysis, hull’s method, Velicer’s MAP, comparison data and Empirical Kaiser Criterion method, based on our sample. These inconsistencies and limitations as substantiated by the current study reflect the broader challenges in the field of MCC research, namely the lack of adequate and standardized measures in this domain (O'Hara et al., 2021). The reliance on outdated methods, such as eigen-values-less-than-one rule and scree plots, by the original test developers to determine the number of factors to extract further contributes to the theoretical confusion. Additionally, many of these tests were developed using principal component analysis with orthogonal rotation, which may not be ideal for social science constructs, as they do not adequately account for the multidimensional nature of the constructs typically found in the social 53 sciences. Furthermore, it is important to acknowledge that factor analysis involves multiple stages of decision-making, including choosing a method for factor extraction, retention, and rotation. The choice of specific methods within each stage can vary, leading to a multitude of possible combinations and potentially yielding different results using the same data (O’Connor, 2000). This variation adds to the complexity and challenges in establishing a consistent and reliable dimension structure for MCC. These discrepancies and limitations underscore the need for a more robust and comprehensive approach to capturing the multidimensional nature of MCC. In order to address this, we employed advanced statistical techniques and considered the theoretical foundations of the construct to gain a deeper understanding of its dimensions and develop a more valid and reliable measurement tool using the entire item pool. In the current study, we also conducted IRT analyses to assess the performance of the previous scales developed by the test developers on the latent trait continuum. The IRT graphs provided valuable insights into the precision and information of items within a dimension. The IRT analyses revealed that most items within a dimension exhibited poor precision and information. Higher information values indicate an increased ability to differentiate between individuals who fall within the specific latent trait regions where higher information is observed (O’Connor, 2017). In our findings, the IRT graphs indicated relatively low information values for the majority of the items. It is important to note that in research settings, information values (y-axis) of 4 or more are generally considered adequate. However, in clinical applications, where cut-off scores determine diagnoses of individuals, higher information values of 10 or more are typically required (O’Connor, 2017). The low information values observed in our analyses suggest that the previous scales may not be optimal for accurately assessing the individuals’ positions on the latent trait continuum. This suggests a potential limitation in the ability of these scales to provide precise and informative measurements. Additionally, the poor information and precision of these individual subscales may be influenced by the fact that these subscales are in fact 54 probably multidimensional in nature. It is important to note that achieving good IRT typically requires unidimensional subscales, which may explain the lower performance observed for various subscales in our analyses. 3.3.2 Development of the New Scale In addition to dimensionality analysis, we employed network analysis to visually conceptualize the factor clusters. The network plot analysis of the item pool used in our study provided valuable insights into the interconnectedness and relationships among the items from various scales examined in our study. The network plot exhibited a complex pattern of the relationships, revealing clusters or subgroups of items that were more closely connected to each other compared to items in the other clusters. This analysis helped us gain a deeper understanding of the multidimensional nature of the MCC construct. Initially, the network plot appeared to have five nodes, suggesting five dimensions. However, upon closer examination, we identified three major dimensions with two probable additional minor dimensions. This analysis allowed us to visually comprehend the interrelationships and the structure of the existing MCC item pool, beyond the unidimensional perspective. This approach helped us recognize that previous researchers may have focused on different clusters, with some emphasizing the major clusters and grouping the minor ones together, leading to the varied factor structures and confusion observed in the literature. The network plot analysis served as a comprehensive visual representation, helping us identify the initial clusters of related items and showcasing the strengths of associations between items. These identified clusters were then used to derive items for the three items for our new scale. After deriving the items from the clusters identified in the network plot analysis, we conducted dimensionality tests for each of the three factors. However, the results of the dimensionality analysis indicated that both factor 1 and factor 3 (also see Figure 24) still exhibited some degree of non- 55 unidimensionality, as supported by parallel analysis and Velicer's MAP. To refine the item pools within these new subfactors we looked at the item and test level information functions for these factors using IRT. This analysis is aimed to assess the precision and information provided by the items within each factor on the latent trait continuum. Consequently, we made the decision to remove these poorly performing items to ensure the newly adapted scale is precise and concise, in line with our primary aim for this research. Following the removal of these items, another round of dimensionality and IRT analyses were conducted to assess the functioning of the remaining items. This iterative process resulted in better performing IRT functions, and consequently better performing items, as the factors also became more unidimensional. By refining the item pools based on the results of the IRT analysis and eliminating items with poor performance, we aimed to enhance the precision and reliability of the newly adapted scale. This rigorous approach ensured that the new scale consisted of high-quality items that accurately measured the intended MCC dimensions within each subfactor. The MDS plot provided visual evidence of the lack of consistent structure within the literature, as indicated by the distances between apparently similar dimensions derived from different subtests. By plotting the scaled scores, we gained insights into the wide variation among items and the incongruence between dimensions across different scales. One notable observation was the positioning of the Knowledge dimensions, which is common to nearly all the scales examined in the study. The SPMCS_Knowledge and MCSHPS_Knowledge closely mapped to each other, suggesting that they measure an equivalent construct. Similarly, MAKSS_Knowledge, MCCTS_Knowledge and CBMCS_Knowledge also exhibited close proximity, suggesting they tap into the same underlying construct. However, MCKAS_Knowledge stood apart, separated by a considerable margin, and not aligning with the other knowledge scaled scores. Overall, this discrepancy supports the conjecture, that these subscales purporting to measure the same construct (i.e., MCC knowledge) may in fact be measuring different constructs. Another striking example is the Awareness dimension, where 56 MCKAS_Awareness, MAKSS_Awareness and MCCTS_Awareness showed little proximity in their spatial mapping. Only CBMCS_Awareness maps exhibited some degree of similarity with MCKAS_Awareness. This again raises the question of whether these dimensions, which are intended to measure the same aspect within a broader MCC construct, truly assess the same underlying construct. It is noteworthy that these dimensions were derived from the same tripartite conceptualization and informed by the same MCC literature, yet they exhibited significant disparities in their mapping. This highlights one of the shortcomings of the MCC literature, because despite the numerous scales on MCC, there is a lack of consensus on the dimensionality of the construct, which leads to less than adequate and unstandardized measures in this field (O'Hara et al., 2021). This also brings up the issue of whether the previous scales have really adhered to the tripartite conceptualisation (Attitude, Knowledge, and Skills) of MCC as given by Sue and colleagues (1992). The scaled scores on various subscales did not align well on the MDS plot, due to statistical shortcomings of the methods adopted by the previous researchers. As discussed, all the previous scales were developed using classical testing theory and factor analysis. Classical testing theory uses item-test correlations or the magnitude of the factor loadings of items which could lead to various problems as the items may not really be representative of the same underlying latent construct or dimension and may just correlate with one another. Using factor analysis can also exacerbate this problem as factor analysis involves multiple levels of decision making like choosing a method for factor extraction, retention, and rotation, thus adding to the endless combinations and permutations possible. This could in turn, lead to varied results (O’Connor, 2000). Also, most test developers of the existing MCC scales used PCA with orthogonal rotation (Varimax), which is not an ideal method for factor extraction or rotation especially for the MCC construct, as MCC is assumed to comprise of correlated underlying factors. Methods like PCA with orthogonal rotation (Varimax) do not allow for these correlations between factors which can lead to unreliable results. 57 3.3.3 Dimensions of the New Scale 3.3.3.1 Skills Based on our current findings, the new scale consists of 48 items that are grouped into three factors. The first factor, referred to as the Skills factor, demonstrated high factor loadings. This factor aligns with the tripartite model of MCC proposed by Sue and colleagues (1992). Additionally, this factor has also been identified in other existing MCC scales (MAKSS-CE-R; Kim, Cartwright, Asay, & D'Andrea, 2003), the Multicultural Counseling Competence and Training Survey (MCCTS; Holcomb-McCoy & Myers, 1999), the Multicultural Counseling Knowledge and Awareness Scale (MCKAS; Ponterotto, Gretchen, Utsey, Rieger, & Austin, 2002), the California Brief Multicultural Competence Scale (CBMCS; Gamst, Dana, Der-Karabetian, Aragon, Arellano, Morrow & Martenson, 2004), the School Psychology Multicultural Competence Scale (SPMCS; Malone, Briggs, Ricks, Middleton, Fisher & Connell, 2015), the Multicultural Competence Scale in Helping-Profession Students (MCSHPS; Hladik, 2014). Within the context of our research the skills factor encompasses the intervention or therapy related skills of a clinician, as well as the assessment skills of a clinician. (e.g., “I am knowledgeable of effective assessment strategies used with culturally and linguistically diverse students” and “I am skilled in terms of being able to provide appropriate intervention services to culturally diverse students”). Sue and Sue (1990) highlighted the significance of clinicians actively engaging in the ongoing process of developing and implementing culturally and clinically relevant intervention and assessment strategies. Culturally and clinically relevant intervention and assessment involves tailoring therapeutic approaches to meet the unique needs of clients from diverse backgrounds. Therapists adapt treatment approaches to align with clients’ cultural identities ensuring their interventions are sensitive respectful and meaningful to the clients and their culture. This may involve integrating cultural practices, rituals or beliefs into therapy or modifying traditional therapeutic techniques to be culturally appropriate (Sue & Sue, 1990; Sue et al., 1992) 58 A culturally competent approach to intervention and assessment involves the openness to explore a wider variety of assessment methods and moving beyond traditional assessment methods (Dana, 2003). It recognizes that standard assessment measures may not fully capture the diverse experiences, perspectives, and needs of individuals from diverse backgrounds. This was also reflected in the items for this factor (e.g., “I know how to use alternate assessment methods such as dynamic assessment and ecological assessment”) 3.3.3.2 Knowledge The Knowledge factor, which is consistent with the tripartite model proposed by Sue and colleagues (1992) was also supported by our study. However, in the study, items classified as both knowledge and awareness consistently fell under the factor. Many dimensions related to awareness focused on aspects of understanding and knowledge about a clinician’s own cultural and ethnic background and how it influences them. Indeed, self-reflection and self-awareness regarding cultural identity and their influence on clinical practice are essential components of MCC. This factor comprises items that encompass two distinct aspects: absolute knowledge about cultural concepts and self-knowledge. Items that fall under the aspect of absolute knowledge focus on acquisition of factual knowledge (e.g., “I can define stereotype” and “I can discuss how culture affects the health seeking behaviors of clients”), thus assessing clinician’s familiarity with established cultural concepts and their understanding of how it impacts client’s lived experiences. Items that fall under the aspect of self-knowledge, focus on clinician’s self-knowledge acquired through introspection and reflection (e.g., “I am aware of how my cultural background and experiences have influenced my attitudes about psychological processes”). Tervalon and Murray-García (1998) self-reflection and lifelong learning, which is based on the cultural humility model of MCC. Understanding one's own culture practices, orientations, and biases as well as recognizing the power dynamics inherent in client clinician relationships are critical for fostering 59 a trusting and effective therapeutic alliance (Arredondo, 1999; Sue et al., 1992; Tervalon & Murray-García, 1998). By engaging in self-reflection, continuously expanding their cultural knowledge and sensitivity clinicians can enhance their ability to provide culturally responsive and competent care to the diverse client populations. Sue and Sue (1990) conducted a comprehensive literature review on the characteristics of a culturally competent counsellor and emphasized the significance of clinicians actively engaging in self-reflection to enhance their cultural competence. They highlighted the importance of counselors becoming aware of their own assumptions about human behavior, biases, values and preconceived notions. By engaging in the process of becoming more aware, clinicians can develop an understanding of their own worldviews and how these perspectives are shaped by their cultural conditioning. This component of self-knowledge and reflexivity is crucial as it can impact the therapeutic alliance. The authors identified self-reflection and the development of cultural self-awareness as core competencies for culturally skilled clinicians (Sue et al., 1992). By critically examining their own cultural conditioning, clinicians can gain insight into how their beliefs and values may shape their interactions with clients and how to navigate the complexities of working with clients from diverse backgrounds. The Knowledge factor reflects the integration of cultural competence and cultural humility with the field of multicultural counselling and psychotherapy. Cultural competence is conceptualized as a process of acquiring knowledge pertaining to diverse cultures, while cultural humility involves a humble and receptive stance towards learning from clients. Cultural humility challenges the notion of absolute expertise in cultural understanding and instead emphasizes mindset characterized by curiosity, openness and respect when engaging with clients from diverse backgrounds (Kirmayer, 2012; Tervalon & Murray-García, 1998). 60 3.3.3.3 Acceptance of Cultural Relativism In the present study the Acceptance of Cultural Relativism factor emerged as distinct from the tripartite conceptualization of MCC. This factor comprised items that demonstrated high loadings and captured the essence of being nonjudgmental and accepting of ethnoracial clients’ worldviews perspectives and lived experiences. This aspect of acceptance is considered fundamental element of effective therapy. Respect and appreciation for clients with diverse background and worldviews are recognized as fundamental aspects of being a culturally competent counsellor (Sue & Sue, 1990). By approaching therapy from a non-judgmental stance, therapists can effectively address power dynamics and reduce social distance between themselves and the clients. This is crucial as the clinical experience of the client is coloured by a cultural lens, which becomes even more prominent in a vulnerable space like therapy (Kirmayer, 2012). The concept of unconditional positive regard as proposed by Carl Rogers (1957), and acceptance as emphasized in the framework of third and fourth waves of counselling psychology (e.g., Hayes, 2004), align with the core principles underlying the Acceptance of Cultural Relativism factor. These modalities emphasize the importance of creating a therapeutic environment that fosters genuine acceptance, empathy and understanding for clients, regardless of their backgrounds (Hayes, 2004; Rogers, 1957). By recognizing and embracing the uniqueness of clients, cultural identities and experiences, counselors can establish a trusting and supportive therapeutic relationship. This acceptance allows clients to feel safe in sharing their perspectives and challenges ultimately enhancing the effectiveness of therapy. This factor can be seen as the heart of effective therapy. Constantine (2002) found that clients rated the clinicians higher on MCC if they had previously rated them high on traditional counselling competence; thus, there was a high overlap between the two, indicating that skills like active listening, empathy and rapport building played a significant role in making the counsellors seem more multiculturally competent. 61 Cognitively based third wave-therapies like Cognitive Behavioural Therapy (CBT) focusses strongly on fostering personal independence, promoting rationality and reconstructing unhelpful cognitions. These may in fact be in conflict with the values of certain populations of the world where interdependence, passivity and spirituality are instead celebrated (Hays, 2009). Mindfulness-based therapies, such as Acceptance and Commitment Therapy (ACT; Hayes, Strosahl, & Wilson, 1999) offer an alternative perspective that aligns with the values of cultural diversity. ACT emphasizes the core processes of acceptance and mindfulness. Acceptance, in the context of ACT, involves a stance of willingness to experience events and emotions without attempting to control or alter them. In the context of MCC, this approach recognizes the value of accepting not only one’s thoughts and feelings without judgement, but also the client’s thoughts, values and worldviews, without imposing one’s value judgements. This allows clinicians to be more compassionate and adopt a non-judgmental stance towards their clients and their worldviews. Items within the dimension of Acceptance also reflect this philosophy of nonjudgement (e.g., “I think that being highly competitive and achievement oriented are traits that all clients should work towards (R)2”). In the larger context, these three factors seem to emulate the overarching components in psychotherapy which are essential components of therapeutic effectiveness. Research has consistently shown that therapeutic alliance is the single strongest predictor of positive therapeutic outcomes (Lambert, 1992; Horvath et al., 2011; Wampold, 2015). Lampert (1992) estimated that approximately 30% of the effectiveness of psychotherapy arise from common factors (i.e., elements that are universal across all therapeutic modalities, like acceptance, empathy, warmth etc.) Wampold (2001) found that overall specific therapeutic modalities contribute marginally to effectiveness of therapy, whereas it is the therapists’ competence that predominantly accounts for the variance. Enhancing a clinician’s skills, 2 R indicates reverse coded items 62 knowledge and acceptance of cultural relativism inherently contributes to an increase in a clinician’s multicultural competence, which would thereby augment a clinician’s overall competence. These higher order factors found in our research also broadly map onto the higher order factors that lead to psychotherapeutic effectiveness – namely, knowledge about specific therapeutic techniques, skills of tailoring and implementing those techniques and general acceptance of the client and their worldviews. 3.3.4 Correlations with Other Variables We also aimed to explore the relationship between various auxiliary variables and the scaled scores of the existing measures, as well as the new scale. These auxiliary variables included experience with ethnoracial clients, coursework related to MCC, training or seminars related to MCC, linguistic fluency, age, and measures of social desirability. Regarding the correlations between the auxiliary variables and the scaled scores of the existing measures, we found that training or seminars related to MCC showed weak positive correlations, ranging from .02 to .23 with the total scaled scores and the dimension scaled scores. These findings were consistent to previous literature, which suggested that increased education and training with culturally diverse clients, would result in higher self-perceived MCC scores, particularly in the areas of skills and knowledge (Malone et al., 2015). Consistent with this literature, Skills and Knowledge scaled scores did show relatively higher correlations with training compared to other dimensions. Experience with clients from diverse backgrounds, also exhibited weak positive correlations ranging from .01 to .15 with the scaled scores, although some scaled scores showed negative correlations with this variables. Interestingly, the scaled scores on the Awareness dimension of most scales demonstrated the lowest correlation with this variable. Linguistic fluency was analysed by combining bilingual and multilingual together and correlation was then conducted. Linguistic fluency showed variation in its correlations with different scaled scores, ranging from -.44 to +.27. It showed positive correlation with certain scaled scores such as MCCTS_Identity, SPMCS_Knowledge, SPMCS_Skills, and MCSHPS_Knowledge. Coursework showed weak negative correlations ranging 63 from -.23 to -.02. Age, on the other hand, consistently exhibited positive correlations from .03 to .26 with the scaled scores. Of note, scaled scores on Awareness dimension on most scales showed the highest correlation with age. Similar trends were observed when examining the correlations between these auxiliary variables and the scaled scores of the new scale. Age and training showed a weak positive correlation of .15 and .16, respectively. Scaled scores on Skills (.15) and Knowledge (.18) dimensions of the new scale also demonstrated positive correlations with training, although weak in magnitude. Notably, the Skills dimension on the new scale exhibited positive correlation with linguistic fluency (.27), which was contrasting to the findings for most existing scales. Coursework showed negative correlations with the scaled scores on the dimensions new scale, ranging between -.05 (for Acceptance) to -.19 (for Knowledge). Furthermore, we examined the correlations between the scaled scores of the existing measures with two measures of social desirability, BIDR (Paulhus, 1991) and HEXACO-60 (Honesty-Humility) (Ashton & Lee, 2009). In BIDR, there are two components: self-deceptive enhancement (SDE) and impression management (IM). Overall, BIDR showed weak correlation with the various measures in the study, which indicates that the scales are less susceptible to socially desirable responding. People that score higher on SDE believe they are answering honestly but are overestimating their abilities. On the other hand, IM measures deliberate attempt to present oneself in a more favourable way. Our findings indicated that the magnitude of correlation for SDE was more than the magnitude for IM. This is interesting because if people score higher on SDE it indicates that people are answering more honestly, even though they may be endorsing responding in a socially desirable way. On the other hand, IM represents a more serious threat to validity, as it represents purposeful distortion in order to present oneself in a favourable light and possible faking (Vispoel & Tao, 2013). These results were consistent with the scaled scores on the new scale as well, with scores on SDE showing higher magnitude of correlation than IM, except for the Skills dimension, where the magnitude of correlation for IM (.34) 64 was higher than the magnitude of SDE (.21). The HEXACO-60 Honesty-Humility scores demonstrated weak to moderate correlations ranging from -.14 (for MCSHPS_Knowledge) to .40 (for MCKAS total). The Awareness dimension consistently displayed higher correlations with HEXACO-60 Honesty-Humility. These findings suggest that the scales have a stronger correlation with traits such as earnestness, sincerity, and modesty, which are considered crucial characteristic of a culturally competent clinician. These findings were consistent for the new scale, with Acceptance showing the highest correlation (.38) with HEXACO-60 (honesty-humility), followed by Knowledge (.22). 3.5 Limitations and Future Directions Our study is subject to several limitations that should be acknowledged. First the sample size used in our study was jointly small considering the number of variables examined. This limitation stems from the constraints of our research funding, which restricted our ability to recruit a larger sample. Additionally, the use of a draw system for participant selection may introduce bias or limitations of the representativeness of our samples. As a result, the generalizability of our findings may be limited, and caution should be exercised when applying the results to a broader population. Although the sample size in our study was relatively small, it is important to note the diversity of the sample in terms of national and ethnic identities is a notable strength. Unlike the existing measures that were predominantly constructed in the United States, our study aimed to actively recruit participants from various backgrounds and cultural contexts. Another limitation is related to the survey length and the potential participant fatigue. Given the extensive number of scales and the items included in the survey, participants may have experienced survey fatigue, leading to incomplete or missing data. Moreover, the cognitive demands imposed by the survey may have caused attentional fatigue potentially affecting the accuracy and quality of the responses. 65 These limitations should be taken into account when interpreting the findings of the study. Particularly, the results from IRT should be thus considered very tentative. Future research with larger samples as well as measures to mitigate survey fatigue and improve participant selection process would help address these limitations and provide a comprehensive understanding of the examined constructs. Another notable limitation of our first study is the use of a student specific scale, namely the School Psychology Multicultural Competence Scale (SPMCS; Malone et al., 2015). This scale was originally designed for practicing school psychologists, focusing specifically on the knowledge and skills required to work with culturally and linguistically diverse students. The narrow focus of the scale on this particular population may have introduced some bias and restricted the generalizability of our findings, considering that our sample were clinical, counselling and social work students. To address this limitation, it would be beneficial to conduct validity studies by replacing the specific term "students" with a more generic term such as "clients." This modification would allow for a more inclusive assessment of multicultural competence that encompasses diverse clinical population beyond just students. By doing so researchers can explore the performance and validity of the scale in a wider range of clinical contexts uninsured but the scale adequately captures the broad construct of multicultural competence. It is worth noting that in our study, factor one which was identified as a broader "Skills" factor, also contain a minor dimension specific to student population. This confounding element should be taken into consideration when interpreting factor 1 for this scale. An additional unanticipated challenge in our study was the limited responses from participants in India. Despite expecting higher response rate from India, given the larger size of graduate-level cohorts in Indian universities, we received significantly fewer responses from India. This outcome was unexpected, especially considering the primary author’s affiliation with India and personal connections with graduate students there. One possible explanation for the lower response rate from Indian students is that the scales used in our study were primarily developed in North America and focused on concepts 66 such as white person privilege, which may be less familiar or relevant to participants in India. This cultural disparity could have hindered their engagement and willingness to participate and complete the survey. This particular challenge highlights the need for future research to address the question of how to develop skills on multicultural competence that can be universally applicable. It raises important considerations about the adaptability and cultural relevance of MCC tools in different populations across the world. Future research in the field of multicultural competence should strive to develop skills that are generic enough to be adaptable and inclusive of diverse cultural contexts. One of the other limitations of the study was the improper framing of the school-related demographic questions, specifically the question asking the participants about their current year of graduate school. Some participants may have misunderstood the question and responded thinking year of their graduate program. For example, first-year Ph.D. students should have ideally selected the option “3rd year” of the graduate program, but they incorrectly selected “1st year”. This issue was realized during the analysis stage, as there were no participants represented in the for 3rd and 4th year categories for the Indian sample, as well as the 4th and 5th year categories for the North America sample. Another limitation for the study was the use of various scales that were scored on different metrics. For instance, the MCKAS scale was scored on a scale of 1 to 7, while the MCSHPS was scored on a scale of 1to 4. In contrast, most other scales were scored on 1 to 4 scale. To address this issue the scales were recoded to the same metric while analyzing. However, this recoding process resulted in non-integer values. As a consequence, it is possible that the clustering of items from the same scale in the network plot may have artificially been influenced by this recoding procedure. Furthermore, in future directions researchers can conduct validity studies on the new scale developed in the research. This can involve examining the performance of the new scale in relation to other scales that measure specific domains of multicultural competence. For example, correlating the scale scores on the acceptance of cultural relativism factor with other scale that assessed non-judgmental 67 attitudes or acceptance of clients by clinicians would provide valuable insights into the validity and utility of the newly developed scale. Overall, addressing the limitations mentioned above and conducting further validity studies will contribute to the ongoing advancement of research in the field of multicultural competence, promoting its universal application and ensuring the development of robust measurement tools. 3.6 Implications This research addresses the confusion in the literature about the dimensionality of the construct and used modern psychometric tools to help in developing a more precise scale. Within the field of counselling psychology and psychotherapy, there is an increasing demand for developing valid and reliable measures that can assess meaningful changes in clinician’s cultural competence skills (Kirmayer et al., 2020). The confusion surrounding the dimensionality of multicultural competence has hindered the development of robust assessment tools. By employing advanced psychometric techniques, this study aimed to clarify the underlying structure of MCC and provide a more accurate and comprehensive assessment framework. Lee (2011) demonstrated that therapists who are culturally competent tend to achieve better outcomes when working with culturally diverse clients. This improvement is evidenced by higher levels of client satisfaction with treatment (Constantine, 2002), increased self-disclosure during therapy (Burkard et al., 2006) and a reduced rates of client attrition (Wade & Berntein, 1999). The significance of MCC becomes evident in these findings, as a culturally competent psychologist is more attuned to sociopolitical factors that contribute to mental illness, lack of treatment adherence, and increased attrition among culturally diverse clients. Rather than attributing these challenges solely to clients’ cultural background, a culturally competent psychologist recognizes the complex interplay of various factors and acknowledges the influence of sociopolitical contexts (Carpenter-Song et al., 2007). By highlighting the positive impact of MCC on client outcomes, these studies underscore the importance of training and developing cultural competence skills among therapists. These findings 68 support the notion that MCC is not only ethically imperative but also contributes to better therapeutic outcomes for culturally diverse individuals. By emphasizing the role of cultural sensitivity, sociopolitical awareness, and the ability to navigate cultural dynamics, this research highlights the significant impact that a culturally competent psychologist can have on the well-being of their diverse clientele. 69 Tables Table 1. Summary of Existing MCC Scales Scales Authors Factors Subscale names Sample size Method Factor Retention Region Items MAKSS-CE-R Kim et al., 2003 3 • Awareness • Knowledge • Skills EFA (180) and CFA (158) PCA with oblique rotation (Direct Oblimin) Scree plot United States 33 MCI3 Sodowsky et al., 1994 4 • Awareness • Knowledge • Skills • Relationship EFA (604) and CFA (350) PAF (oblique rotation used) Kaiser and Scree plot United States 40 MCCTS Holcomb-McCoy & Myers, 1999 5 • Awareness • Knowledge • Skills • Definition of terms • Racial identity EFA (151) PCA with Oblimin rotation Eigen values United States 32 MCKAS Ponterotto et al., 2002 2 • Awareness • Knowledge EFA (525- MCAS scale) and CFA (199) PCA with orthogonal (Varimax) rotation Scree plot United States 32 CBMCS Gamst et al., 2004 4 • Socio-cultural diversities • Awareness of cultural barriers • Multicultural knowledge • Sensitivity to consumers EFA (415), EFA-experts (20), CFA (414) and CFA-new sample (366) PCA with Varimax rotation Kaiser-Guttman retention criterion and Scree plot United States 21 SPMCS Malone et al., 2015 4 • Cultural skills • Cultural knowledge • Cultural appreciation • Cultural awareness EFA (312) PAF with Promax rotation Parallel analysis and Scree plot United States 28 MCSHPS Hladik, 2014 5 • Knowledge • Understating of terms • Activity • Awareness • Communication skills EFA and CFA (363) – on the same sample PCA (orthogonal) with Varimax norm method Scree plot Central Europe 20 3 This scale (MCI) was not used in the study due to limited permissions. 70 Table 2. Participant Sociodemographic Sociodemographic Characteristics Samples North American Indian Sex Male 20 Male 13 Female 97 Female 34 Intersex 0 Intersex 0 Prefer not to answer 1 Prefer not to answer 0 Gender Man 19 Man 13 Woman 91 Woman 34 Non-binary 5 Non-binary 0 Prefer not to answer 2 Prefer not to answer 0 Primary Racial Background First Nations 4 Indo-Aryan 28 Métis 1 Dravidian 15 Inuit 0 Mongoloid (Tibeti-Burmans) 2 Caucasian 64 Black 4 Asian 36 Latin American 9 Other 7 Ethnic Background Canadian 53 Dravidian (Kannadiga, Malayali, Tulu, Tamil, Telugu etc.) 19 (US) American 11 Indo - Aryan (Assamese, Bengali, Gujarati, Hindi, Kashmiri, Konkani, Marathi, Punjabi, Sindhi etc.) 25 English 13 Sino-Tibetan (Bodo, Manipuri etc.) 0 French 3 Austroasiatic (Santhali etc.) 1 Italian 3 Other 0 German 8 Prefer not to say 0 Scottish 5 Scandinavian (e.g., Finland, Denmark, Norway, Sweden) 2 Irish 4 71 Sociodemographic Characteristics Samples North American Indian Ukrainian 3 Dutch 1 Polish 3 Portuguese 1 Spain 1 Greek 1 Russian 0 West Indies/Caribbean (e.g., Jamaican, Cuban, Haiti, Puerto Rican, Trinidadian, etc.) 1 First Nations 1 Métis 1 Inuit 0 African 2 South/Central Latin American 6 Northeast Asian (e.g., Chinese, Japanese, Korean, etc.) 12 West Asian (e.g., Iranian, Iraqi, Afghan, Lebanese, Israeli, etc.) 5 South Asian (e.g., East Indian, Pakistani, Sri Lankan, etc.) 21 Southeast Asian (e.g., Vietnamese, Filipino, Cambodian, Malaysian, Laotian, etc.) 4 Other 10 Cultural Background (N = 113) Cultural Background (N = 44) Canadian 69 Dravidian (Kannadiga, Malayali, Tulu, Tamil, Telugu etc.) 17 (US) American 11 Indo - Aryan (Assamese, Bengali, Gujarati, Hindi, Kashmiri, Konkani, Marathi, Punjabi, Sindhi etc.) 26 English 7 Sino-Tibetan (Bodo, Manipuri etc.) 1 French 0 Austroasiatic (Santhali etc.) 0 Italian 1 Other 1 German 2 Prefer not to say 0 Scottish 2 72 Sociodemographic Characteristics Samples North American Indian Scandinavian (e.g., Finland, Denmark, Norway, Sweden) 1 Irish 0 Ukrainian 3 Dutch 1 Polish 1 Portuguese 0 Spain 1 Greek 0 Russian 0 West Indies/Caribbean (e.g., Jamaican, Cuban, Haiti, Puerto Rican, Trinidadian, etc.) 2 First Nations 3 Métis 1 African 3 South/Central Latin American 4 Northeast Asian (e.g., Chinese, Japanese, Korean, etc.) 7 West Asian (e.g., Iranian, Iraqi, Afghan, Lebanese, Israeli, etc.) 7 South Asian (e.g., East Indian, Pakistani, Sri Lankan, etc.) 18 Southeast Asian (e.g., Vietnamese, Filipino, Cambodian, Malaysian, Laotian, etc.) 5 Other 13 Linguistic ability (N = 118) Linguistic ability (N = 45) Monolingual 59 Monolingual 0 Bilingual 40 Bilingual 19 Multilingual 19 Multilingual 26 Geographic location (N = 115) Geographic location (N = 42) Alberta 7 Andhra Pradesh 1 British Columbia 44 Arunachal Pradesh 0 Manitoba 7 Assam 1 New Brunswick 1 Bihar 0 Northwest territories 0 Chhattisgarh 1 Newfoundland and Labrador 1 Goa 0 73 Sociodemographic Characteristics Samples North American Indian Ontario 27 Gujarat 0 Prince Edward Island 0 Haryana 0 Quebec 6 Himachal Pradesh 0 Saskatchewan 0 Jammu and Kashmir 0 Nunavut 0 Jharkhand 0 Nova Scotia 0 Karnataka 20 Yukon 0 Kerala 2 North America 12 Madhya Pradesh 0 South America 0 Maharashtra 5 Europe 3 Manipur 0 Asia 1 Meghalaya 0 Africa 0 Mizoram 0 Australia/ Oceania 6 Nagaland 0 Odisha 0 Punjab 0 Rajasthan 2 Sikkim 0 Tamil Nadu 1 Telangana 0 Tripura 0 Uttar Pradesh 4 Uttarakhand 0 West Bengal 0 Andaman and Nicobar Islands 0 Chandigarh 0 Dadra and Nagar Haveli 0 Daman and Diu 0 Delhi 5 Lakshadweep 0 Pondicherry 0 74 Table 3. Schooling Related Characteristics Schooling Related Characteristics North American Indian Program enrolled in 114 46 MA/MSc Clinical Psychology 20 12 MA/MSc/MEd Counselling Psychology 11 5 MA/MSc/MEd Social Work 30 1 MPhil Clinical psychology - 14 MPhil Counselling psychology - 0 MPhil Social Work - 2 PhD Clinical Psychology 36 1 PhD Social Work 2 0 PhD Counselling Psychology 0 1 Completed grad school, now practicing in field/ in academia 15 10 Year of school/ training 1st year 19 7 2nd year 11 15 3rd year 4 0 4th year 5 0 5th year 0 2 6th year 0 2 Internship/Placement 2 8 Other 17 10 Coursework on MCC Yes 76 24 No 42 22 Training on MCC Yes 84 18 No 34 27 Diverse Clients Yes 97 36 No 21 10 Accreditation of program by national/federal bodies Yes 109 28 No 8 16 75 Table 4. Scales' Psychometric Properties Scales Alpha Omega H value MAKSS .83 .85 .16 Awareness .70 .68 .24 Knowledge .84 .85 .36 Skills .80 .80 .34 MCCTS .93 .93 .37 Knowledge .89 .89 .40 Awareness .82 .83 .49 Definition of terms .78 .79 .51 Racial identity .73 NA .65 Skills .49 .52 .29 MCKAS .87 .88 .21 Awareness .84 .85 .23 Knowledge .92 .92 .56 CBMCS .85 .86 .28 Multicultural knowledge .76 .77 .46 Awareness of cultural barriers .78 .75 .43 Sensitivity to consumers .73 .75 .65 Socio-cultural diversities .79 .79 .42 SPMCS .89 .88 .30 Cultural skills .84 .85 .41 Cultural knowledge .92 .92 .72 Cultural appreciation .80 .81 .50 Cultural awareness .75 .77 .46 MCSHPS .86 .86 .28 Knowledge .80 .80 .44 Understating of terms .61 .65 .34 Activity .73 .75 .46 Awareness .63 .73 .45 Communication skills .66 .66 .46 76 Table 5. Dimensionality Analyses for all Scales Scales Original No. of Factors Different Tests for Number of Factors HULL EMPKC CD RAWPAR MAP NEVALSGT1 MAKSS 3 6 4 7 5 6 10 MCCTS 5 1 2 2 2 4 8 MCKAS 2 2 2 2 3 4 9 CBMCS 4 2 2 2 3 4 5 SPMCS 4 2 2 2 2 2 6 MCSHPS 5 1 2 2 2 2 7 77 Table 6. Fit Coefficients for the Original Factors of each scale Scales Fit Coefficients CFI TLI NNFI IFI RMSEA SRMR MAKSS .90 .89 .89 .90 .11 .13 MCCTS .96 .95 .95 .96 .13 .13 MCKAS .91 .90 .90 .91 .14 .15 CBMCS .91 .90 .90 .91 .15 .15 SPMCS .98 .97 .97 .98 .10 .12 MCSHPS .94 .93 .93 .94 .11 .12 78 Table 7. Dimensionality Analyses for Factors in the New scale Factors Different Tests for Number of Factors HULL EMPKC CD RAWPAR MAP NEVALSGT1 Factor 1 (Skills) 2 1 1 1 1 1 Factor 2 (Knowledge) 1 1 1 1 3 4 Factor 3 (Acceptance) 2 1 2 1 2 2 79 Table 8. Items for the New Scale Items SKILLS 1. I am skilled in implementing home-school collaboration programs and interventions 2. I have knowledge of research on assessing culturally and linguistically diverse children 3. I am knowledgeable of effective assessment strategies used with culturally and linguistically diverse students 4. I know how to use alternate assessment methods such as dynamic assessment and ecological assessment 5. I am knowledgeable of the most effective consultation strategies used with culturally and linguistically diverse students 6. I know how to adapt instruments to assess linguistically diverse students 7. I am knowledgeable of evidence-based intervention strategies used with culturally and linguistically diverse students 8. I can make culturally relevant curriculum and classroom management recommendations 9. I understand the process of second language acquisition and its impact on the acquisition of academic skills 10. I know the specifics of working with minorities 11. I am skilled in terms of being able to provide appropriate intervention services to culturally diverse students 12. I can discuss family therapy from a cultural /ethnic perspective 13. I can describe the degree to which a counseling approach is appropriate for a specific group of people KNOWLEDGE 14. I can give examples of how stereotypical beliefs about culturally different people impact the counseling relationship 15. I can discuss how culture affects our client’s vocational choices 16. At the present time, how would you rate your understanding of the term: "Transcultural" 17. I can discuss how the counseling process may be at conflict with cultural values of at least two ethnic groups 18. At this point in your life, how would you rate your understanding of the impact of the way you think and act when interacting with persons of different cultural backgrounds 19. I nonverbally communicate my acceptance of culturally different clients 20. I can identify my negative and positive emotional reaction towards people of other racial and ethnic groups 21. I can explain how factors suggest poverty, and powerlessness have influenced the current conditions of at least two ethnic groups 22. I can discuss how culture affects the health seeking behaviors of clients 23. I am aware of how my cultural background and experiences have influenced my attitudes about psychological processes 24. I am able to discuss how my culture has influenced the way I think 25. I can define stereotype 26. I can discuss my own ethnic/cultural heritage 80 Items 27. I can define discrimination 28. At the present time, how would you rate your understanding of the term: "Prejudice" 29. I understand the impact and operations of oppression and the racist concepts that have permeated the mental health professions. 30. I have an understanding of the role culture and racism play in the development of identity and worldviews among minority groups. 31. At the present time, how would you rate your understanding of the term: "Ethnicity" 32. At the present time, how would you rate your understanding of the term: "Culture" 33. I verbally communicate my acceptance of culturally different clients 34. I can discuss within group differences among ethnic groups (e.g., low social economic status [SES] client vs high SES client) 35. I can help clients determine whether a problem stems from racism or bias in others ACCEPTANCE OF CULTURAL RELATIVISM 36. I believe that it is important to emphasize objective and rational thinking in minority clients (R) 37. I believe that minority clients will benefit most from counseling with a majority who endorses White middle-class values and norm (R) 38. I think that my clients should exhibit some degree of psychological mindedness and sophistication (R) 39. I feel all the recent attention directed toward multicultural issues in counseling is overdone and not really warranted (R) 40. I believe that my clients should view a patriarchal structure as the ideal (R) 41. I think that being highly competitive and achievement oriented are traits that all clients should work towards (R) 42. I think that clients should perceive the nuclear family as the ideal social unit (R) 43. I am aware that being born a White person in this society carries with it certain advantages 44. I believe all clients should maintain direct eye contact during counseling (R) 45. I am aware that being born a minority in this society brings with it certain challenges that White people do not have to face 46. I think that clients who do not discuss intimate aspects of their lives are being resistant and defensive (R) 47. I believe that all clients must view themselves as their number one responsibility (R) 48. I have a sense of the values, strengths, and limitations of my own culture 81 Table 9. Correlation among the factors in the new scale Full scale Skills Knowledge Acceptance Full scale Skills .45 Knowledge .82 .37 Acceptance .62 -.26 .22 82 Figures Figure 1. IRT plot for MAKSS-CE-R Awareness 83 Figure 2. IRT plot for MAKSS-CE-R Knowledge 84 Figure 3. IRT plot for MAKSS-CE-R Skills 85 Figure 4. IRT plot for MCCTS Knowledge 86 Figure 5. IRT plot for MCCTS Awareness 87 Figure 6. IRT plot for MCCTS Definition of Terms 88 Figure 7. IRT plot for MCCTS Racial Identity 89 Figure 8. IRT plot for MCCTS Skills 90 Figure 9. IRT plot for MCKAS Knowledge 91 Figure 10. IRT plot for MCKAS Awareness 92 Figure 11. IRT plot for CBMCS Multicultural Knowledge 93 Figure 12. IRT plot for CBMCS Awareness of Cultural Barriers 94 Figure 13. IRT plot for CBMCS Sensitivity to Consumers 95 Figure 14. IRT plot for CBMCS Socio-cultural Diversities 96 Figure 15. IRT plot for SPMCS Cultural Skills 97 Figure 16. IRT plot for SPMCS Cultural Knowledge 98 Figure 17. IRT plot for SPMCS Cultural Appreciation 99 Figure 18. IRT plot for SPMCS Cultural Awareness 100 Figure 19. IRT plot for MCSHPS Knowledge 101 Figure 20. IRT plot for MCSHPS Understanding of Terms 102 Figure 21. IRT plot for MCSHPS Activity 103 Figure 22. IRT plot for MCSHPS Awareness 104 Figure 23. IRT plot for MCSHPS Communication Skills 105 Figure 24. Network Plot for all Items Note. Circle show the dimension clusters, which were then used for the new scale. 106 Figure 25. Multidimensional Scaling Graph Note. Blue dots represent the scaled scores of the dimensions of all the existing scales. Red dots represent the factors from the new scale. 107 Figure 26. IRT plot for New Scale – Factor 1 (Skills) 108 Figure 27. IRT plot for New Scale – Factor 2 (Knowledge) 109 Figure 28. IRT plot for New Scale – Factor 3 (Acceptance of Cultural Relativism) 110 References Alexandra L. Dima (2018) Scale validation in applied health research: tutorial for a 6-step R-based psychometrics protocol, Health Psychology and Behavioral Medicine, 6:1, 136-161, https://doi.org/10.1080/21642850.2018.1472602 Auerswald, M., & Moshagen, M. (2019). How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychological Methods, 24(4), 468-491. https://doi.org/10.1037/met0000200 Arredondo, P. (1999). Multicultural Counseling Competencies as Tools to Address Oppression and Racism. Journal Of Counseling & Development, 77(1), 102-108. https://doi.org/10.1002/j.1556-6676.1999.tb02427.x Ashton, M. C., & Lee, K. (2008). The prediction of honesty–humility-related criteria by the HEXACO and five-factor models of personality. Journal of Research in Personality, 42(5), 1216–1228. https://doi.org/10.1016/j.jrp.2008.03.006 Ashton, M., & Lee, K. (2009). The HEXACO-60: A short measure of the major dimensions of personality. Journal of Personality Assessment, 91(4), 340–345. https://doi.org/10.1080/00223890902935878 Bergkamp, J., & Ponsford, M. (2020). Cultural Encapsulation. The Wiley Encyclopedia Of Personality And Individual Differences, 239-241. https://doi.org/10.1002/9781119547181.ch304 Burkard, A., Knox, S., Groen, M., Perez, M., & Hess, S. (2006). European American therapist self-disclosure in cross-cultural counseling. Journal Of Counseling Psychology, 53(1), 15-25. https://doi.org/10.1037/0022-0167.53.1.15 Brown, T. A., & Moore, M. T. (2012). Confirmatory factor analysis. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 361–379). The Guilford Press. 111 Carpenter-Song, E., Schwallie, M., & Longhofer, J. (2007). Cultural Competence Reexamined: Critique and Directions for the Future. Psychiatric Services, 58(10), 1362-1365. https://doi.org/10.1176/ps.2007.58.10.1362 Constantine, M. (2002). Predictors of satisfaction with counseling: Racial and ethnic minority clients' attitudes toward counseling and ratings of their counselors' general and multicultural counseling competence. Journal Of Counseling Psychology, 49(2), 255-263. https://doi.org/10.1037/0022-0167.49.2.255 Dana, Richard H., "Multicultural Assessment: Research, Training, and Practice" (2003). Regional Research Institute. 35. https://pdxscholar.library.pdx.edu/rri_facpubs/35 Davison, M. L., & Sireci, S. G. (2000). Multidimensional scaling. Handbook of Applied Multivariate Statistics and Mathematical Modeling, 323–352. https://doi.org/10.1016/b978-012691360-6/50013-6 D'Andrea, M., Daniels, J., & Heck, R. (1991). Evaluating the Impact of Multicultural Counseling Training. Journal Of Counseling & Development, 70(1), 143-150. https://doi.org/10.1002/j.1556-6676.1991.tb01576.x Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates. Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195-212. https://doi.org/10.3758/s13428-017-0862-1 Epskamp, S., Costantini, G., Haslbeck, J., Isvoranu, A., Cramer, A. O. J., Waldorpp, L. J., Schmittmann, V. D., & Borsboom, D. (2020). qgraph: Graph Plotting Methods, Psychometric Data Visualization and Graphical Model Estimation. R package version 1.6.5 Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittamnn, V. D., Borsboom, D. (2012). qgraph: Network visualization of relationships in psychometric data. Journal of Statistical Software, 48(4), 1-18. https://doi.org/10.18637/jss.v048.i04 112 Field, A., Miles, J., & Field, Z. (2013). Discovering statistics using R (pp. 711-761). SAGE. Gamst, G., Dana, R. H., Der-Karabetian, A., Aragon, M., Arellano, L., Morrow, G., & Martenson, L. (2004). Cultural competency revised: The California Brief Multicultural Competence Scale. Measurement and Evaluation in Counseling and Development, 37(3), 163–183. https://doi.org/10.1080/07481756.2004.11909758 Goretzko, D., Pham, T., & Bühner, M. (2019). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40(7), 3510-3521. https://doi.org/10.1007/s12144-019-00300-2 Gorsuch, R. (2015). Factor analysis. Routledge Taylor & Francis Group. Hart, C., Ritchie, T., Hepper, E., & Gebauer, J. (2015). The Balanced Inventory of Desirable Responding Short Form (BIDR-16). SAGE Open, 5(4), 215824401562111. https://doi.org/10.1177/2158244015621113 Hayes, S. C. (2004). Acceptance and Commitment Therapy and the new behavior therapies: Mindfulness, acceptance and relationship. In S. C. Hayes, V. M. Follette, & M. Linehan (Eds.), Mindfulness and acceptance: Expanding the cognitive behavioral tradition (pp. 1-29). New York: Guilford. Hays, D. (2008). Assessing Multicultural Competence in Counselor Trainees: A Review of Instrumentation and Future Directions. Journal Of Counseling & Development, 86(1), 95-101. https://doi.org/10.1002/j.1556-6678.2008.tb00630.x Hevey, D. (2018). Network analysis: A brief overview and tutorial. Health Psychology and Behavioral Medicine, 6(1), 301–328. https://doi.org/10.1080/21642850.2018.1521283 Hladik, J. (2014). Design and Development of a Multicultural Competence Scale in Helping-Profession Students. Asian Social Science, 10(9). https://doi.org/10.5539/ass.v10n9p162 113 Holcomb-McCoy, C., & Myers, J. (1999). Multicultural Competence and Counselor Training: A National Survey. Journal Of Counseling & Development, 77(3), 294-302. https://doi.org/10.1002/j.1556-6676.1999.tb02452.x Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy, 48(1), 9–16. https://doi.org/10.1037/a0022186 Keyser, V., Gamst, G., Meyers, L., Der-Karabetian, A., & Morrow, G. (2014). Predictors of self-perceived cultural competence among children’s mental health providers. Cultural Diversity And Ethnic Minority Psychology, 20(3), 324-335. https://doi.org/10.1037/a0035762 Kim, B., Cartwright, B., Asay, P., & D’Andrea, M. (2003). A Revision of the Multicultural Awareness, Knowledge, and Skills Survey-Counselor Edition. Measurement And Evaluation In Counseling And Development, 36(3), 161-180. https://doi.org/10.1080/07481756.2003.11909740 Kirmayer, L. (2012). Rethinking cultural competence. Transcultural Psychiatry, 49(2), 149-164. https://doi.org/10.1177/1363461512444673 Kirmayer, L., Bennegadi, R., & Kastrup, M. (2016). Cultural Awareness and Responsiveness in Person-Centered Psychiatry. Person Centered Psychiatry, 77-95. https://doi.org/10.1007/978-3-319-39724-5_7 Kirmayer, L., Fung, K., Rousseau, C., Lo, H., Menzies, P., & Guzder, J. et al. (2020). Guidelines for Training in Cultural Psychiatry. The Canadian Journal Of Psychiatry, 66(2), 195-246. https://doi.org/10.1177/0706743720907505 Lambert, M. J. (1992). Implications of outcome research for psychotherapy integration. In J. C. Norcross, & M. R. Goldstein (Eds.), Handbook of psychotherapy integration (pp. 94129). New York: Basic Books. Lee, E. (2011). Clinical significance of cross-cultural competencies (CCC) in social work practice. Journal Of Social Work Practice, 25(02), 185-203. https://doi.org/10.1080/02650533.2011.573654 Mair, P., Groenen, P. J. F., & de Leeuw, J. (2022). More on multidimensional scaling and unfolding in R: Smacof version 2. Journal of Statistical Software, 102(10), 1-47. https://doi.org/10.18637/jss.v102.i10 114 Malone, C., Briggs, C., Ricks, E., Middleton, K., Fisher, S., & Connell, J. (2015). Development and Initial Examination of the School Psychology Multicultural Competence Scale. Contemporary School Psychology, 20(3), 230-239. https://doi.org/10.1007/s40688-015-0079-1 McQueen, K. (2022, March 24). Cultural Encapsulation. Study.com | Take Online Courses. Earn College Credit. Research Schools, Degrees & Careers. Retrieved May 14, 2022, from https://study.com/learn/lesson/cultural-encapsulation-counselor.html Mokken, R. J. (1971). A theory and procedure of scale analysis. De Gruyter Morizot, J., Ainsworth, A. T., & Reise, S. P. (2007). Toward modern psychometrics: Application of item response theory models in personality reseach. In R.W. Robins, R.C. Fraley, & R. F. Krueger (Eds.), Handbook of research methods in personality psychology (pp. 407-423). Guilford Press. O’Connor, B. P. (2017). An illustration of the effects of fluctuations in test information on measurement error, the attenuation of effect sizes, and diagnostic reliability. Psychological Assessment, 30(8), 991–1003. https://doi.org/10.1037/pas0000471 O’Connor, B. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396-402. https://doi.org/10.3758/bf03200807 O’Connor, B. P. (2020). EFA.dimesnions: Exploratory Factor Analysis Functions for Assessing Dimensionality. R package version 0.1.7.7 https://cran.r-project.org/web/packages/EFA.dimensions/ O'Hara, C., Chang, C., & Giordano, A. (2021). Multicultural Competence in Counseling Research: The Cornerstone of Scholarship. Journal Of Counseling & Development, 99(2), 200-209. https://doi.org/10.1002/jcad.12367 Paulhus, D. (1991). Measurement and Control of Response Bias. Measures Of Personality And Social Psychological Attitudes, 17-59. https://doi.org/10.1016/b978-0-12-590241-0.50006-x 115 Pedersen, P. (1989). Developing Multicultural Ethical Guidelines for Psychology. International Journal Of Psychology, 24(1-5), 643-652. https://doi.org/10.1080/00207594.1989.10600073 Ponterotto, J., Gretchen, D., Utsey, S., Rieger, B., & Austin, R. (2002). A Revision of the Multicultural Counseling Awareness Scale. Journal Of Multicultural Counseling And Development, 30(3), 153-180. https://doi.org/10.1002/j.2161-1912.2002.tb00489.x Putnick, D., & Bornstein, M. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71-90. https://doi.org/10.1016/j.dr.2016.06.004 R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/. Rizopoulos, D. (2013). Ltm: latent trait models under IRT. R package version 1.2-0. Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology, 21(2), 95–103. https://doi.org/10.1037/h0045357 Rogers, M., & Ponterotto, J. (1997). Development of the Multicultural School Psychology Counseling Competency Scale. Psychology In The Schools, 34(3), 211-217. https://doi.org/10.1002/(sici)1520-6807(199707)34:3<211::aid-pits3>3.0.co;2-r Sijtsma, K., & van der Ark, L. A. (2017). A tutorial on how to do a Mokken scale analysis on your test and questionnaire data. British Journal of Mathematical & Statistical Psychology, 70(1), 137-158. https://doi.org/10.1111/bmsp.12078 Sodowsky, G., Taffe, R., Gutkin, T., & Wise, S. (1994). Development of the Multicultural Counseling Inventory: A self-report measure of multicultural competencies. Journal Of Counseling Psychology, 41(2), 137-148. https://doi.org/10.1037/0022-0167.41.2.137 Steyvers, M. (2006). Multidimensional scaling. Encyclopedia of Cognitive Science. https://doi.org/10.1002/0470018860.s00585 116 Sue, D. (1990). Culture-specific strategies in counseling: A conceptual framework. Professional Psychology: Research And Practice, 21(6), 424-433. https://doi.org/10.1037/0735-7028.21.6.424 Sue, D. (1991). A Model for Cultural Diversity Training. Journal Of Counseling & Development, 70(1), 99-105. https://doi.org/10.1002/j.1556-6676.1991.tb01568.x Sue, D., & Sue, D. (2022). Counseling the Culturally Diverse: Theory and Practice (5th ed., pp. 29-52). John Wiley & Sons, Inc. Sue, D. W., Arredondo, P., & McDavis, R. J. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Counseling & Development, 70(4), 477–486. https://doi.org/10.1002/j.1556-6676.1992.tb01642.x Tabachnick, B., & Fidell, L. (2014). Using multivariate statistics (6th ed., pp. 660-721). Pearson Education. Tervalon, M., & Murray-García, J. (1998). Cultural Humility Versus Cultural Competence: A Critical Distinction in Defining Physician Training Outcomes in Multicultural Education. Journal Of Health Care For The Poor And Underserved, 9(2), 117-125. https://doi.org/10.1353/hpu.2010.0233 Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1-10. van der Ark, L. A. (2012). New Developments in Mokken Scale Analysis in R. Journal of Statistical Software, 48 (5), 1-27. https://doi.org/10.18637/jss.v048.i05 Vereen, L., Hill, N., & McNeal, D. (2008). Perceptions of Multicultural Counseling Competency: Integration of the Curricular and the Practical. Journal Of Mental Health Counseling, 30(3), 226-236. https://doi.org/10.17744/mehc.30.3.g34u122m16q64g44 Vispoel, W., & Tao, S. (2013). A generalizability analysis of score consistency for the Balanced Inventory of Desirable Responding. Psychological Assessment, 25(1), 94-104. https://doi.org/10.1037/a0029061 117 Wade, P., & Bernstein, B. (1991). Culture sensitivity training and counselor's race: Effects on Black female clients' perceptions and attrition. Journal Of Counseling Psychology, 38(1), 9-15. https://doi.org/10.1037/0022-0167.38.1.9 Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Lawrence Erlbaum Associates. Wampold, B. E. (2015). How important are the common factors in psychotherapy? an update. World Psychiatry, 14(3), 270–277. https://doi.org/10.1002/wps.20238 Watkins, M. (2018). Exploratory Factor Analysis: A Guide to Best Practice. Journal Of Black Psychology, 44(3), 219-246. https://doi.org/10.1177/0095798418771807 Watson, R., van der ARK, L.A., Lin, L.-C., Fieo, R., Deary, I.J., & J., & Meijer, R.R. (2011). Item response theory: How Mokken scaling can be used in clinical practice. Journal of Clinical Nuring, 21, 273-2746. https://doi.org/10.1111/j.1365-2702.2011.03893.x Yang, F. M., & Kao, S. T. (2014). Item response theory for measurement validity. Shanghai archives of psychiatry, 26(3), 171–177. https://doi.org/10.3969/j.issn.1002-0829.2014.03.010 Yong, A., & Pearce, S. (2013). A Beginner’s Guide to Factor Analysis: Focusing on Exploratory Factor Analysis. Tutorials In Quantitative Methods For Psychology, 9(2), 79-94. https://doi.org/10.20982/tqmp.09.2.p079 118 Appendices Appendix A – MAKSS- CE-R Multicultural Awareness, Knowledge, and Skills Survey – Counselor Edition - Revised (MAKSS-CE-R) Bryan S. K. Kim University of California, Santa Barbara Brenda Y. Cartwright University of Hawaii at Manoa Penelope A. Asay University of Maryland, College Park Michael J. D’Andrea University of Hawaii at Manoa Kim, B. S. K., Cartwright, B. Y., Asay, P. A., & D’Andrea, M. J. (2003). A revision of the Multicultural Awareness, Knowledge, and Skills Survey – Counselor Edition. Measurement and Evaluation in Counseling and Development, 36, 161-180. Before the MAKSS-CE-R is copied or distributed, permission must be obtained from one of these authors: Michael J. D’Andrea, Ed.D.: michael@hawaii.edu Bryan S. K. Kim, Ph.D.: bkim@education.ucsb.edu This survey is designed to obtain information on the educational needs of counselor trainees. It is not a test. No grade will be given as a result of completing this survey. Multicultural Awareness, Knowledge, and Skills Survey – Counselor Edition – REVISED (MAKSS-CE-R) 119 Please complete the demographic items listed below. Following the demographic section, you will find a list of statements and/or questions related to a variety of issues related to the field of multicultural counseling. Please read each statement/question carefully. From the available choices, circle the one that best fits your reaction to each statement/question. Thank you for your participation. 1. Gender: ______ MALE ______ FEMALE 2. Age _______ 3. Race ______ 4. Ethnic/Cultural Background _________________ 5. State of residence: _____________________ 6. Highest educational degree earned: __________ In the specialty area of (check one) ______College Student Personnel Counseling ______Community Counseling ______Counselor Education ______Counseling Psychology ______Rehabilitation Counseling ______School Counseling ______School Psychology Other: ______________________________ 7. If a current student, educational degree sought: _____________ In the specialty area of (check one) ______College Student Personnel Counseling ______Community Counseling ______Counselor Education ______Counseling Psychology ______Rehabilitation Counseling ______School Counseling ______School Psychology Other: ______________________________ 8. Are you currently enrolled in a course on multicultural counseling? ____YES ____ NO 9. Number of completed courses on multicultural counseling: ______ 10. Years of experience working with clients who were racially/ethnically different from you: _____ Less than 1 year 120 _____ 1-2 years _____ 3-4 years _____ 5 years or more 11. Number of past and current clients who were racially/ethnically different than you: ______ 12. Current occupation (if not a full-time student) _____________ 13. Annual Family Income (Check one): _____ $7,500 or less _____ $7,501 - 15,000 _____ $15,001 - 25,000 _____ $25,001 - 35,000 _____ $35,001 - 50,000 _____ $50,001 or more 1. Promoting a client's sense of psychological independence is usually a safe goal to strive for in most counseling situations. Strongly DisagreeDisagree AgreeStrongly Agree 2. Even in multicultural counseling situations, basic implicit concepts such as "fairness" and "health", are not difficult to understand. Strongly DisagreeDisagree AgreeStrongly Agree 3. How would you react to the following statement? In general, counseling services should be directed toward assisting clients to adjust to stressful environmental situations. Strongly DisagreeDisagree AgreeStrongly Agree 4. While a person's natural support system (i.e., family, friends, etc.) plays an important role during a period of personal crisis, formal counseling services tend to result in more constructive outcomes. Strongly DisagreeDisagree AgreeStrongly Agree 5. The human service professions, especially counseling and clinical psychology, have failed to meet the mental health needs of ethnic minorities. Strongly DisagreeDisagree AgreeStrongly Agree 6. The effectiveness and legitimacy of the counseling profession would be enhanced if counselors consciously supported universal definitions of normality. Strongly DisagreeDisagree AgreeStrongly Agree 7. Racial and ethnic persons are under-represented in clinical and counseling psychology. Strongly DisagreeDisagree AgreeStrongly Agree 8. In counseling, clients from different ethnic/cultural backgrounds should be given the 121 same treatment that White mainstream clients receive. Strongly DisagreeDisagree AgreeStrongly Agree 9. The criteria of self-awareness, self-fulfillment, and self-discovery are important measures in most counseling sessions. Strongly DisagreeDisagree AgreeStrongly Agree 10. The difficulty with the concept of "integration" is its implicit bias in favor of the dominant culture. Strongly DisagreeDisagree AgreeStrongly Agree At the present time, how would you rate your understanding of the following terms: 11. "Ethnicity" Very LimitedLimited Good Very Good 12. "Culture" Very LimitedLimited Good Very Good 13. "Multicultural" Very LimitedLimited Good Very Good 14. "Prejudice" Very LimitedLimited Good Very Good 15. "Racism" Very LimitedLimited Good Very Good 16. "Transcultural" Very LimitedLimited Good Very Good 17. "Pluralism" Very LimitedLimited Good Very Good 18. "Mainstreaming" Very LimitedLimited Good Very Good 19. "Cultural Encapsulation" Very LimitedLimited Good Very Good 20. "Contact Hypothesis" Very LimitedLimited Good Very Good 21. At this point in your life, how would you rate your understanding of the impact of the way you think and act when interacting with persons of different cultural backgrounds? Very LimitedLimitedFairly AwareVery Aware 22. At this time in your life, how would you rate yourself in terms of understanding how your cultural background has influenced the way you think and act? Very LimitedLimitedFairly AwareVery Aware 23. How well do you think you could distinguish "intentional" from "accidental" communication signals in a multicultural counseling situation? Very LimitedLimited Good Very Good 24. How would you rate your ability to effectively consult with another mental health professional concerning the mental health needs of a client whose cultural background is significantly different from your own? Very LimitedLimited Good Very Good 122 25. How well would you rate your ability to accurately assess the mental health needs of lesbian women? Very LimitedLimited Good Very Good 26. How well would you rate your ability to accurately assess the mental health needs of older adults? Very LimitedLimited Good Very Good 27. How well would you rate your ability to accurately assess the mental health needs of gay men? Very LimitedLimited Good Very Good 28. How well would you rate your ability to accurately assess the mental health needs of persons who come from very poor socioeconomic backgrounds? Very Limited Limited Good Very Good 29. How would you rate your ability to identify the strengths and weaknesses of psychological tests in terms of their use with persons from different cultural/racial/ethnic backgrounds? Very LimitedLimited Good Very Good 30. How would you rate your ability to accurately assess the mental health needs of men? Very LimitedLimited Good Very Good 31. How well would you rate your ability to accurately assess the mental health needs of individuals with disabilities? Very LimitedLimited Good Very Good 32. How would you rate your ability to effectively secure information and resources to better serve culturally different clients? Very LimitedLimited Good Very Good 33. How would you rate your ability to accurately assess the mental health needs of women? Very LimitedLimited Good Very GoodAppendix B - MCI 123 Appendix C - MCCTS Survey Items 1. I can discuss my own ethic/cultural heritage 2. I am aware of how my cultural background and experiences have influenced my attitudes about psychological processes 3. I am able to discuss how my culture has influences the way I think 4. I can recognize when my attitudes, beliefs, and values are interfering with providing the best services of my clients 5. I verbally communicate my acceptance of culturally different clients 6. I nonverbally communicate my acceptance of culturally different clients 7. I can discuss my family's perspectives regarding acceptable and non-acceptable codes of conduct 8. I can discuss models of white identity development 9. I can define racism 10. I can define prejudice 11. I can define discrimination 12. I can define stereotype 13. I can identify the cultural basis of my communication style 14. I can identify my negative and positive emotional reaction towards people of other racial and ethnic groups 15. I can identify my reactions that are based on stereotypical beliefs about different ethnic groups 16. I can give examples of how stereotypical beliefs about culturally different people impact the counseling relationship 124 17. I can articulate the possible differences between the nonverbal behaviors of the five major ethnic groups (i.e., African/Black, Hispanic/Latino, Asian, Native American, European/White) 18. I can articulate the possible differences between the verbal behaviors of the five major ethnic groups (i.e., African/Black, European/White, Hispanic, Asian, Native American) 19. I can discuss the counseling implications for at least two models of “Minority Identity Development” 20. I can discuss within group differences among ethnic groups (e.g., low social economic status [SES] Puerto Rican client vs high SES Puerto Rican client) 21. I can discuss how culture affects our client’s vocational choices 22. I can discuss how culture affects the health seeking behaviors of clients 23. I can discuss how culture affects the manifestation of psychological disorders 24. I can describe the degree to which a counseling approach is appropriate for a specific group of people 25. I can explain how factors suggest poverty, and powerlessness have influenced the current conditions of at least two ethnic groups 26. I can discuss research regarding mental health issues and culturally different populations work 27. I can discuss how the counseling process may be conflict with cultural values of at least two ethnic groups 28. I can list at least three barriers that prevent ethnic minorities from using mental services 29. I can discuss the potential bias of two assessment instruments frequently used in the counseling process 30. I can discuss family therapy from a cultural /ethnic perspective 31. I can anticipate when my helping style is inappropriate for a culturally different client 32. I can help clients determine whether a problem stems from racism or bias in others 125 Appendix D – MCKAS Multicultural Counseling Knowledge and Awareness Scale (MCKAS) Copyrighted by Joseph G. Ponterotto, 1997 A Revision of the Multicultural Counseling Awareness Scale (MCKAS) Copyrighted by Joseph G. Ponterotto, 1991 --------------------------------------------------------------------------------------------------------------------------------- Using the following scale, rate the truth of each item as it applies to you. 1 2 3 4 5 6 7 Not at Somewhat Totally All True True True ------------------------------------------------------------------------------------------------------------------------------ 1. I believe all clients should maintain direct eye contact during counseling. 1 2 3 4 5 6 7 2. I check up on my minority/cultural counseling skills by monitoring my functioning – via consultation, supervision, and continuing education. 1 2 3 4 5 6 7 3. I am aware some research indicates that minority clients receive “less preferred” forms of counseling treatment than majority clients. 1 2 3 4 5 6 7 4. I think that clients who do not discuss intimate aspects of their lives are being resistant and defensive. 1 2 3 4 5 6 7 5. I am aware of certain counseling skills, techniques, or approaches that are more likely to transcend culture and be effective with any clients. 1 2 3 4 5 6 7 6. I am familiar with the “culturally deficient” and “culturally deprived” depictions of minority mental health and understand how these labels serve to foster and perpetuate discrimination. 1 2 3 4 5 6 7 126 ------------------------------------------------------------------------------------------------------------------------------ Using the following scale, rate the truth of each item as it applies to you. 1 2 3 4 5 6 7 Not at Somewhat Totally All True True True ------------------------------------------------------------------------------------------------------------------------------ 7. I feel all the recent attention directed toward multicultural issues in counseling is overdone and not really warranted. 1 2 3 4 5 6 7 8. I am aware of individual differences that exist among members within a particular ethnic group based on values, beliefs, and level of acculturation. 1 2 3 4 5 6 7 9. I am aware some research indicates that minority clients are more likely to be diagnosed with mental illnesses than are majority clients. 1 2 3 4 5 6 7 10. I think that clients should perceive the nuclear family as the ideal social unit. 1 2 3 4 5 6 7 11. I think that being highly competitive and achievement oriented are traits that all clients should work towards. 1 2 3 4 5 6 7 12. I am aware of the differential interpretations of nonverbal communication (e.g., personal space, eye contact, handshakes) within various racial/ethnic groups. 1 2 3 4 5 6 7 13. I understand the impact and operations of oppression and the racist concepts that have permeated the mental health professions. 1 2 3 4 5 6 7 14. I realize that counselor-client incongruities in problem conceptualization and counseling goals may reduce counselor credibility. 1 2 3 4 5 6 7 127 --------------------------------------------------------------------------------------------------------------------------------- Using the following scale, rate the truth of each item as it applies to you. 1 2 3 4 5 6 7 Not at Somewhat Totally All True True True --------------------------------------------------------------------------------------------------------------------------------- 15. I am aware that some racial/ethnic minorities see the profession of psychology functioning to maintain and promote the status and power of the White Establishment. 1 2 3 4 5 6 7 16. I am knowledgeable of acculturation models for various ethnic minority groups. 1 2 3 4 5 6 7 17. I have an understanding of the role culture and racism play in the development of identity and worldviews among minority groups. 1 2 3 4 5 6 7 18. I believe that it is important to emphasize objective and rational thinking in minority clients. 1 2 3 4 5 6 7 19. I am aware of culture-specific, that is culturally indigenous, models of counseling for various racial/ethnic groups. 1 2 3 4 5 6 7 20. I believe that my clients should view a patriarchal structure as the ideal. 1 2 3 4 5 6 7 21. I am aware of both the initial barriers and benefits related to the cross-cultural counseling relationship. 1 2 3 4 5 6 7 22. I am comfortable with differences that exist between me and my clients in terms of race and beliefs. 1 2 3 4 5 6 7 128 ------------------------------------------------------------------------------------------------------------------------------ Using the following scale, rate the truth of each item as it applies to you. 1 2 3 4 5 6 7 Not at Somewhat Totally All True True True ------------------------------------------------------------------------------------------------------------------------------ 23. I am aware of institutional barriers which may inhibit minorities from using mental health services. 1 2 3 4 5 6 7 24. I think that my clients should exhibit some degree of psychological mindedness and sophistication. 1 2 3 4 5 6 7 25. I believe that minority clients will benefit most from counseling with a majority who endorses White middle-class values and norms. 1 2 3 4 5 6 7 26. I am aware that being born a White person in this society carries with it certain advantages. 1 2 3 4 5 6 7 27. I am aware of the value assumptions inherent in major schools of counseling and understand how these assumptions may conflict with values of culturally diverse clients. 1 2 3 4 5 6 7 28. I am aware that some minorities see the counseling process as contrary to their own life experiences and inappropriate or insufficient to their needs. 1 2 3 4 5 6 7 29. I am aware that being born a minority in this society brings with it certain challenges that White people do not have to face. 1 2 3 4 5 6 7 30. I believe that all clients must view themselves as their number one responsibility. 1 2 3 4 5 6 7 129 ------------------------------------------------------------------------------------------------------------------------------ Using the following scale, rate the truth of each item as it applies to you. 1 2 3 4 5 6 7 Not at Somewhat Totally All True True True --------------------------------------------------------------------------------------------------------------------------------- 31. I am sensitive to circumstances (personal biases, language dominance, stage of ethnic identity development) which may dictate referral of the minority client to a member of his/her own racial/ethnic group. 1 2 3 4 5 6 7 32. I am aware that some minorities believe counselors lead minority students into non-academic programs regardless of student potential, preferences, or ambitions. 1 2 3 4 5 6 7 130 Appendix E – CBMCS 131 132 Appendix F - SPMCS 133 134 Appendix G - MCSHPS Survey Items 1.I am familiar with the immigrant integration process in my country. 2. I know the number of foreigners in my country and their nationalities. 3. I am aware of the possibilities of implementing multicultural. education into practice 4. I know the specifics of working with minorities. 5. I know the cultural specifics of Roma, Vietnamese and Ukrainians living in my country 6. I understand literature dealing with multicultural issues. 7. I understand the concept of racism very well. 8. I understand the concept of prejudice very well. 9. I understand the concept of egocentrism very well. 10. I understand the concept of culture very well. 11. I work on my multicultural knowledge, competences and skills. 12. I am interested in multicultural education. 13.I think about multicultural education. 14. I observe and control my behaviour towards members of minorities. 15. I respect and accept other cultures. 16. I am aware that cooperation of all parties is necessary in order to achieve a conflict-free cultural contact. 17. I respect socio-cultural differences. 18. I can apply various communicative strategies in communication with minority group members. 19. I am able to contribute to a conflict-free and constructive minority problem solution. 20. I can communicate adequately with members of minorities. 135 Appendix H – BIDR BIDR Version 6 - Form 40A Using the scale below as a guide, write a number beside each statement to indicate how true it is. + + + + + + + 1 2 3 4 5 6 7 not true somewhat very true ____ 1. My first impressions of people usually turn out to be right. ____ 2. It would be hard for me to break any of my bad habits. ____ 3. I don't care to know what other people really think of me. ____ 4. I have not always been honest with myself. ____ 5. I always know why I like things. ____ 6. When my emotions are aroused, it biases my thinking. ____ 7. Once I've made up my mind, other people can seldom change my opinion. ____ 8. I am not a safe driver when I exceed the speed limit. ____ 9. I am fully in control of my own fate. ____ 10. It's hard for me to shut off a disturbing thought. ____ 11. I never regret my decisions. ____ 12. I sometimes lose out on things because I can't make up my mind soon enough. ____ 13. The reason I vote is because my vote can make a difference. ____ 14. My parents were not always fair when they punished me. ____ 15. I am a completely rational person. ____ 16. I rarely appreciate criticism. ____ 17. I am very confident of my judgments ____ 18. I have sometimes doubted my ability as a lover. ____ 19. It's all right with me if some people happen to dislike me. 136 ____ 20. I don't always know the reasons why I do the things I do. ____ 21. I sometimes tell lies if I have to. ____ 22. I never cover up my mistakes. ____ 23. There have been occasions when I have taken advantage of someone. ____ 24. I never swear. ____ 25. I sometimes try to get even rather than forgive and forget. ____ 26. I always obey laws, even if I'm unlikely to get caught. ____ 27. I have said something bad about a friend behind his/her back. ____ 28. When I hear people talking privately, I avoid listening. ____ 29. I have received too much change from a salesperson without telling him or her. ____ 30. I always declare everything at customs. ____ 31. When I was young I sometimes stole things. ____ 32. I have never dropped litter on the street. ____ 33. I sometimes drive faster than the speed limit. ____ 34. I never read sexy books or magazines. ____ 35. I have done things that I don't tell other people about. ____ 36. I never take things that don't belong to me. ____ 37. I have taken sick-leave from work or school even though I wasn't really sick. ____ 38. I have never damaged a library book or store merchandise without reporting it. ____ 39. I have some pretty awful habits. ____ 40. I don't gossip about other people's business. 137 Scoring key for BIDR Version 6 - Form 40A Self Deceptive Enhancement (SDE): Items 1 - 20 Reverse scored items: 2,4,6,8,10,12,14,16,18,20. Impression Management (IM): Items 21 - 40 Reverse scored items: 21,23,25,27,29,31,33,35,37,39. Dichotomous Scoring procedure 1. Reverse the Likert ratings for the items indicated above. 2. For each subscale, add one point for every '6' or '7'. (In the case of 5-point scales, add one point for every '5' on the SDE and one point for every '4' or '5' on the IM scale.) For each subscale, the minimum score is 0; the maximum is 20. Reliabilities: Typical alphas are .67-.77 (SDE) and .77-.85 (IM) Norms: Means and standard deviations for 177 UBC undergraduates under two scale formats and two instructional sets. 7-point scale 5-point scale Males (182) Females (251) Males (122) Females (248) Respond Honestly SDE 7.5 (3.2) 6.8 (3.1) 2.3 (2.3) 2.1 (2.0) IM 4.3 (3.1) 4.9 (3.2) 5.5 (3.5) 6.1 (3.6) Play Up Your Good Points SDE 9.0 (3.9) 7.8 (3.9) 4.1 (3.6) 4.2 (3.6) IM 10.5 (4.1) 10.9 (4.2) 11.8 (4.3) 12.5 (4.5) 138 Appendix I – HEXACO – 60 (Honesty-Humility dimension) On the following pages, you will find a series of statements about you. Please read each statement and decide how much you agree or disagree with that statement. Then indicate your response using the following scale: 5 = strongly agree 4 = agree 3 = neutral (neither agree nor disagree) 2 = disagree 1 = strongly disagree Please answer every statement, even if you are not completely sure of your response. 1. I wouldn’t use flattery to get a raise or promotion at work, even if I thought it would succeed. 2. If I knew that I could never get caught, I would be willing to steal a million dollars. 3. Having a lot of money is not especially important to me. 4. I think that I am entitled to more respect than the average person is. 5. If I want something from someone, I will laugh at that person’s worst jokes. 6. I would never accept a bribe, even if it were very large. 7. I would get a lot of pleasure from owning expensive luxury goods. 8. I want people to know that I am an important person of high status. 9. I wouldn’t pretend to like someone just to get that person to do favors for me 10. I’d be tempted to use counterfeit money, if I were sure I could get away with it. Honesty-Humility: Emotionality: 1, 2R, 3, 4R, 5R, 6, 7R, 8R, 9, 10R (R indicates reverse-scored item.) 139 Appendix J – Informed Consent for Participants Project Title: Investigation of Dimensionality and Development of a Precise Measure of Multicultural Self-Efficacy Who is Conducting the Study? • Principal Investigator: o Brian O’Connor, Ph.D. - Psychology Department, UBCO o Email: brian.oconnor@ubc.ca • Co-Investigator(s): o Nataasha Khattar, M.A. Student - Psychology Department, UBCO o Email: nataasha.khattar@ubc.ca You are being invited to participate in a research project that is part of a Graduate Student’s (Nataasha Khattar) research program. This consent form serves to provide an overview of what this project is about and what your participation would entail. Please read this consent form carefully. You should understand the accompanying information. If you have any questions, please ask for help. Funding: This study is not funded by any internal or external agency. There are no actual or potential conflicts of interest with respect to remuneration received from the funding agency for conducting or being involved with any part of the study. Purpose of the study: You are invited to take part in this research study because you are a graduate student in either clinical, counselling or social work programs. We are inviting graduate students like you to participate so we can understand the dimensionality of the multicultural self-efficacy and to develop a precise scale. Study Procedures: If you choose to participate in this study, you will be asked to complete a survey on UBC-hosted Qualtrics. It will take approximately 40-45 min to complete the survey form. If you have any questions before beginning this survey, you can reach out to the researcher at – nataasha.khattar@ubc.ca. Who Can Participate In This Study? To participate in this study, you must: • Be a graduate student • Be in clinical, counselling or social work programs • Be able to read and understand fluent English Study Results: The results of this study will be reported to aid future research in the area of multicultural competence and multicultural training. Results may also be published in journal articles and books. A summary of the findings will be available once the study is finished, and the data have been analyzed. This report will not include individual responses. Instead, only group findings will be described. The thesis will be publicly available on the internet. If you are interested in receiving a copy of this summary report, please email Nataasha Khattar (nataasha.khattar@ubc.ca). It will be sent to you as soon as it becomes available. 140 Potential Risks: If you participate in this study, there are no risks greater than what you would experience in your daily life. Although it is not anticipated, should you feel any distress while participating in this study, you may withdraw at any time without penalty or decline to answer questions. Potential Benefits: We do not think taking part in this study will help you. However, in the future, others may benefit from what we learn in this study, and it may help future graduate students. Renumeration: You will not be paid directly to participate in the study. However, you may choose to enter a draw for one of twenty $20 amazon gift cards (or receive an e-transfer for the same amount). Winners will be required to answer a skill-testing question. Confidentiality: We will gather your information using a secure online platform hosted by UBC Survey Tool (Qualtrics), a secure, Canadian survey system that meets all of the requirements of the BC Freedom of Information and Protection of Privacy Act. All data are stored and backed up in Canada. Once the data are collected, all information will be encrypted and stored in password-protected digital folders that are only accessible by members of the research team, including Nataasha Khattar and Dr. Brian O’Connor. Once the survey is closed, Nataasha Khattar will download the files from Qualtrics and save them on her private laptop. The files will then be deleted from the Qualtrics site. Only Nataasha will have access to the file with your name and contact information. This file will be destroyed once the draw has been conducted and all of the prizes awarded. At this point, data will be deleted from Qualtrics. All de-identified data will be kept in password-protected computer files for five years after the publication of any paper(s) based on the results of this study, as required by the University of British Columbia. Your participation will be confidential but not anonymous. We will not ask you to provide any identifying information on the survey itself. However, if you want to enter the draw, you will be automatically directed to a separate, independent survey site (also hosted by Qualtrics). Here, you will need to enter your name and contact information (email address and telephone number). Thus, the researchers will know who participated but we will not be able to connect your name to your answers. Because of this, it is not possible to withdraw from the study once you have submitted your answers. Contact for information about the study: If you have any questions or desire further information with respect to this study, you may contact Dr. Brian O’Connor at brian.oconnor@ubc.ca or Nataasha Khattar at nataasha.khattar@ubc.ca. Who can you contact if you have concerns or complaints about the study?: If you have any concerns about your treatment or rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the  The amount stated 350 INR for the Indian survey (based on the purchasing power parity conversion between the two currencies at the time of the study) 141 UBC Office of Research Services at 1-877-822-8598 or the UBC Okanagan Research Services Office at 250-807-8832. It is also possible to contact the Research Participant Complaint Line by email (RSIL@ors.ubc.ca). Please reference the study number H22-02798 when calling so the Complaint Line staff can better assist you. You have the right to refuse to participate in this study. If you decide to take part, you may choose not to complete the survey at any time by quitting your browser. If you would like to participate, please click on the arrow below to begin the survey. This will indicate that you have read and understood the above information and have consented to participate in this study. If you do not wish to participate, please exit this website.
Thesis/Dissertation
eng
Psychology
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
O'Connor, Brian
Investigation of dimensionality and development of a precise measure of multicultural self-efficacy for counsellors
Text