UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Does intercultural sensitivity cross cultures? : an investigation of validity issues involved in porting… Greenholtz, Joe 2005

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2005-104961.pdf [ 6.45MB ]
Metadata
JSON: 831-1.0092510.json
JSON-LD: 831-1.0092510-ld.json
RDF/XML (Pretty): 831-1.0092510-rdf.xml
RDF/JSON: 831-1.0092510-rdf.json
Turtle: 831-1.0092510-turtle.txt
N-Triples: 831-1.0092510-rdf-ntriples.txt
Original Record: 831-1.0092510-source.json
Full Text
831-1.0092510-fulltext.txt
Citation
831-1.0092510.ris

Full Text

DOES INTERCULTURAL SENSITIVITY CROSS CULTURES? AN INVESTIGATION OF VALIDITY ISSUES INVOLVED IN PORTING INSTRUMENTS ACROSS LANGUAGES AND CULTURES by JOE GREENHOLTZ B.A. University of Winnipeg, 1978 M.Ed. Temple University, 1986 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION in THE FACULTY OF GRADUATE STUDIES (Educational Leadership and Policy) THE UNIVERSITY OF BRITISH COLUMBIA July 2005 © Joe Greenholtz, 2005 ABSTRACT This study examined the v a l i d i t y of data obtained using a Japanese t r a n s l a t i o n of version one of the I n t e r c u l t u r a l Development Inventory (IDI) (Bennett & Hammer, 1998) and, by extension, some assumptions surrounding the c r o s s - c u l t u r a l t r a n s f e r a b i l i t y of the Developmental Model of I n t e r c u l t u r a l S e n s i t i v i t y (DMIS) upon which i t was based. F i r s t , the research was situated within the f i e l d of international exchanges i n education. I t explored what i n t e r n a t i o n a l i s a t i o n means i n an educational context, and how i t might be conceptualised and measured as an outcome. The case was made that an increase i n i n t e r c u l t u r a l s e n s i t i v i t y i s a desirable, enduring, and achievable legacy for students to take home with them. The difference between i n t e r c u l t u r a l s e n s i t i v i t y and i n t e r c u l t u r a l competence or the psychological t r a i t s posited to underlie i n t e r c u l t u r a l adaptability were discussed and a rationale given for using the Developmental Model of I n t e r c u l t u r a l S e n s i t i v i t y (DMIS) as the t h e o r e t i c a l framework that best represents the construct. The protocol for t r a n s l a t i n g the IDI was reported and issues related to t r a n s l a t i n g and adapting instruments were raised. The t r a n s l a t i o n protocol used for t h i s study was a five-step procedure, much more rigorous i n nature than the widely-used translation/back-translation protocol. The added step of having translators use the d r a f t t r a n s l a t i o n to perform a task outside the t r a n s l a t i o n context was a contribution to the art of t r a n s l a t i n g and adapting instruments. The v a l i d a t i o n procedure followed Messick's (1998) conception of v a l i d i t y as a multi-layered process incorporating both s t a t i s t i c a l and judgmental evidence. Examinations of content, concurrent, and consequential v a l i d i t y took place within Messick's (1998) framework po s i t i n g these aspects of v a l i d i t y to be contributors to an overarching judgment of construct v a l i d i t y . Results of the v a l i d a t i o n process raised questions about whether the concepts that comprise the IDI, and by extension the DMIS, are transferable across languages and cultures. These questions regarding t r a n s f e r a b i l i t y of the IDI and the DMIS were discussed i n d e t a i l and implications for p r a c t i t i o n e r s using e x i s t i n g instruments i n a c r o s s - c u l t u r a l context, and i n the context of v a l i d i t y issues, were elucidated. F i n a l l y , the l i m i t a t i o n s of the study and d i r e c t i o n s for future research were outlined. TABLE OF CONTENTS ABSTRACT i i TABLE OF CONTENTS i v L I S T OF TABLES v i L I S T OF FIGURES v i i ACKNOWLEDGEMENTS v i i i CHAPTER ONE OVERVIEW AND INTRODUCTION 1 1.1 OVERVIEW ••• 1 1.2 INTRODUCTION • • ^ 1.3 DEFINING THE FIELD 9 .1.4 A FRAMEWORK FOR INTERCULTURAL SENSITIVITY 14 1.5 MEASURING INTERCULTURAL SENSITIVITY 21 1.6 LINGUISTIC DETERMINISM 24 CHAPTER TWO METHODOLOGY 27 2.1 INTRODUCTION • • • • 27 2.2 CONTENT VALIDITY 29 2.3 PROCEDURES FOR TRANSLATING THE I D I 30 2.4 ASSESSING THE ADEQUACY OF CONSTRUCT REPRESENTATION 37 2.4.1 Participants .. • 37 2.4.2 Principal Components Analysis— • 38 CHAPTER THREE RESULTS 40 3.1 THE SEVEN-FACTOR MODEL 46 CHAPTER FOUR DISCUSSION 53 4.1 VALIDITY REVISITED 53 4.2 THE VALIDATION PROCESS 60 4.3 CONTENT VALIDITY 61 4.4 CRITERION-RELATED VALIDITY 67 4.5 SOCIAL DESIRABILITY 70 4.6 CONSTRUCT VALIDITY 74 4.7 THE TRANSLATION PROCESS ,82 CHAPTER F I V E CONCLUSION . . . . ; . 84 5.1 CONTRIBUTIONS TO THE FIELD 84 5.2 LIMITATIONS OF THE STUDY 85 5.3 DIRECTIONS FOR FURTHER RESEARCH 86 5.4 IMPLICATIONS FOR PRACTITIONERS 87 5.5 AFTERWARD 88 BIBLIOGRAPHY 89 LIST OF TABLES Table 2.1 Translation Protocol 36 Table 3.1 Component Correlation Matrix. 40 Table 3.2 KMO Measure of Sampling Adequacy and B a r t l e t t ' s , , Test of Sphericity 41 Table 3.3 Rotated Factor Matrix 43 Table 3.4 Cronbach's C o e f f i c i e n t Alpha 52 Table 4.1 Correlations between IDI and Worldmindedness and I n t e r c u l t u r a l Anxiety Scales 69 Table 4.2 Analysis for Gender E f f e c t 72 LIST OF FIGURES Figure 1.1 Continuum of I n t e r c u l t u r a l S e n s i t i v i t y 17 Figure 3.1 Scree Plot 42 ACKNOWLEDGEMENTS I would l i k e to thank Drs Milton Bennett and M i t c h e l l Hammer for granting me permission to translate the Int e r c u l t u r a l Development Inventory into Japanese and to use i t i n my research. I would also l i k e to thank my research assistants, Vickie Yau, who kept a l l of my data i n good order and was always there when I needed to be reminded how to f i n d them, and Michi Ohashi, whose insights into language and c u l t u r a l concepts helped shape my thinking about the issues involved i n porting an instrument from one culture into another. I am extremely grateful to my t r a n s l a t i o n team, Makoto Morise, Yumiko Morise, Naoko Robb, and Shizu Yamamoto who contributed t h e i r expertise and t h e i r time, and to the raters, Masaki Kobayashi and Mitsunori Takakuwa whose insights helped me to re f i n e and shape the f i n a l version. I would l i k e to thank the members of my committee, Dan Pratt, the chair, Bruno Zumbo, and Pierre Walter whose i n c i s i v e comments kept me focussed and who encouraged me to believe i n the value of the work I was doing. I am e s p e c i a l l y grateful to David Coulter, Tom Sork and the other champions of the Ed D cohort programme for creating t h i s opportunity to re-acquaint, at least t h i s semi-jaded p r a c t i t i o n e r , with the world of ideas and to see practice i n a new and invigorating l i g h t . F i n a l l y , I would l i k e to thank my wife Hiroko and my family for putting up with my di s t r a c t e d a i r , f o r being enthusiastic and supportive about the challenge I was undertaking, and for not r o l l i n g t h e i r eyes too much during my repeated attempts to answer the seemingly innocuous question, 'So, what's your research about?' CHAPTER ONE OVERVIEW AND INTRODUCTION 1.1 Overview The purpose of t h i s study was to examine the v a l i d i t y of data obtained using a Japanese t r a n s l a t i o n of version one of the I n t e r c u l t u r a l Development Inventory (IDI) (Bennett & Hammer, 1998) and, by extension, some assumptions surrounding the c r o s s - c u l t u r a l t r a n s f e r a b i l i t y of the Developmental Model of I n t e r c u l t u r a l S e n s i t i v i t y (DMIS) upon which i t i s based. This d i s s e r t a t i o n situated t h i s research within the f i e l d of international exchanges i n education. I t explored, b r i e f l y , what i n t e r n a t i o n a l i s a t i o n means i n an educational context, and how i t might be conceptualised and measured as an outcome. I examined the goals that students and i n s t i t u t i o n s have for a year abroad or other international exchange experiences and made the case that an increase i n i n t e r c u l t u r a l s e n s i t i v i t y i s a desirable, enduring, and achievable legacy for students to take home with them. The construct of i n t e r c u l t u r a l s e n s i t i v i t y i s rela t e d to, but fundamentally d i f f e r e n t from, i n t e r c u l t u r a l competence or the psychological t r a i t s posited to underlie i n t e r c u l t u r a l a daptability. There was a discussion of those differences i n order to demonstrate that the Developmental Model of I n t e r c u l t u r a l S e n s i t i v i t y (DMIS), the t h e o r e t i c a l framework that was chosen, best represents the construct. The I n t e r c u l t u r a l Development Inventory (IDI) i s the psychometric instrument that operationalises and measures the DMIS. Because the IDI was available only i n English when I embarked on t h i s study, i t was f i r s t necessary to translate i t into Japanese so that I could use i t with my intended student population. However, the t r a n s l a t i o n process raised many questions about whether the concepts that comprised the instrument, and by extension the DMIS, were transferable across language and culture. In t h e i r writings and i n IDI qu a l i f y i n g workshops, the instrument's authors, Bennett and Hammer, always characterise the instrument as r e l i a b l e and v a l i d . In fa c t , there was some surprise when I informed them I wanted to do a v a l i d a t i o n study of the instrument i n tr a n s l a t i o n , because ' i t ' s already been validated' (personal correspondence, 2003). The fact that v a l i d i t y i s not a property of an instrument, but rather of the inferences one can draw from the data i t y i e l d s i n a p a r t i c u l a r context, necessitated the empirical investigation that comprised t h i s study. I reported on the protocol used to translate the IDI, and on the v a l i d a t i o n process. After o u t l i n i n g the r e s u l t s , I also discussed the implications of t h i s study for v a l i d i t y assessment i n a c r o s s - c u l t u r a l context, i n terms of t r a n s l a t i o n protocols, i n terms of the c r o s s - c u l t u r a l t r a n s f e r a b i l i t y of the DMIS and the IDI, and as these issues a f f e c t p r a c t i t i o n e r s looking for research tools to use i n c r o s s - c u l t u r a l investigations. 1.2 Introduction This study was to comprise the f i r s t stage of a larger research project intended to examine one facet of my practice as Executive Director of a b i l a t e r a l academic programme, now i n i t s fourteenth year, between a major Canadian research unive r s i t y and a leading private Japanese u n i v e r s i t y . The aim of the larger research agenda was to assess how well the exchange programme was f u l f i l l i n g i t s mandate, as set out i n i t s Mission Statement, to 'develop i n t e r c u l t u r a l understanding among the students'. However, the re s u l t s obtained led me to temporarily c u r t a i l the rest of the research because of serious issues they raised regarding the c r o s s - c u l t u r a l t r a n s f e r a b i l i t y of the t h e o r e t i c a l framework and the v a l i d i t y of the research instrument I had used, i n t r a n s l a t i o n . The Mission Statement c a l l s on the academic exchange programme (AEP), "to contribute to the i n t e r n a t i o n a l i s a t i o n of the partner u n i v e r s i t i e s , and to develop i n t e r c u l t u r a l understanding among the students through an academic, l i n g u i s t i c , and s o c i o - c u l t u r a l programme, combining classroom work and community-outreach a c t i v i t i e s , within an integrated r e s i d e n t i a l environment." As the Mission Statement indicates, the AEP i s somewhat more ambitious than ordinary year-abroad programmes. I t brings a cohort of 100 students to spend a f u l l academic year i n Canada on a senate-approved, credit-bearing ( i . e . , non-ESL) curriculum (see Greenholtz, 2002 for a complete d e s c r i p t i o n ) . In addition to course work, the Japanese undergraduates who come to Canada l i v e i n campus residences with Canadian students, engage i n language exchanges with Canadian students who are studying Japanese, and p a r t i c i p a t e i n a v a r i e t y of community-outreach a c t i v i t i e s such as volunteer placements and internship assignments. However, the AEP shares with a l l i n t e r n a t i o n a l educational exchanges i n the assumption that t h i s type of experience w i l l y i e l d a number of tangible benefits to the part i c i p a n t s , and to t h e i r respective u n i v e r s i t i e s . Indeed, as noted above, the AEP i s mandated to contribute to the i n t e r n a t i o n a l i s a t i o n of both the students and the i n s t i t u t i o n s . The Mission Statement i s s i l e n t on what inte r n a t i o n a l understanding should look l i k e i n p r a c t i c e , although i t sets out the domains i n which i t should occur; academic, l i n g u i s t i c , and s o c i o - c u l t u r a l . Therefore,.it f a l l s to me as Executive Director to translate intent into s p e c i f i c s , and to assess how well the AEP f u l f i l s i t s mandate. Increased language proficiency i s i n v a r i a b l y f i r s t on the l i s t of things that programme participants (according to s e l f reports) most desire from t h e i r exchange. We t r y to help students achieve t h e i r l i n g u i s t i c goal both through a curriculum of content-based,language education based on Mohan's (1986) work and by providing them with opportunities to i n t e r a c t with the community at large, on and o f f campus. (For a detailed description of the curriculum and the t h e o r e t i c a l underpinnings of the academic exchange see Greenholtz, 1997, and Greenholtz, 2002). Students gain an understanding of the academic context by following a programme of Senate-approved, credit-bearing courses during t h e i r academic year i n Canada. These courses require them to perform within North American academic conventions and give them an opportunity to i n t e r a c t with t h e i r Canadian peers. Defining intercultural-understanding goals within the s o c i o - c u l t u r a l domain has proven to be the most d i f f i c u l t part of the puzzle. Fuzzy notions such as 'better' c r o s s - c u l t u r a l awareness, s e n s i t i v i t y , or understanding, or 'increased' i n t e r c u l t u r a l competence or effectiveness (see Bradford, A l l e n , & Beisser, 1998; also Chen, 1997 for a discussion of the differences between competence and effectiveness) usually turn up on s e l f reports as students' most important goal a f t e r language proficiency. (The terms c r o s s - c u l t u r a l and i n t e r c u l t u r a l w i l l be used interchangeably i n t h i s paper). The f i r s t set of terms (understanding, awareness, or s e n s i t i v i t y ) connotes changes i n worldview and appreciation of other cultures (Bennett, 1986) while the second set (competence, effectiveness) connotes changes within what Ward and Kennedy (1999) c a l l the s o c i o - c u l t u r a l (behavioural) domain. Despite the s i m i l a r i t y i n nomenclature, I believe that the term s o c i o - c u l t u r a l as i t appears i n the AEP's Mission Statement refers to more than behaviour. I have taken the s o c i o - c u l t u r a l to e x i s t within the realm of the s o c i a l l y constructed, and thus have chosen to look for changes i n worldview and attitudes towards those d i f f e r e n t from oneself, rather than simple behavioural adaptations. U n t i l my thinking about my practice began to evolve i n the context of t h i s research, the AEP operated on the unexamined and i m p l i c i t assumption that i n t e r n a t i o n a l exposure leads to increased awareness or s e n s i t i v i t y i n c r o s s - c u l t u r a l contexts. This was based primarily on the commonly-held notion that exposure, and i n the case of the AEP, prolonged, varied, and targeted exposure, i s s u f f i c i e n t to act as a c a t a l y s t for desired personal growth. To t h i s end, the AEP had consciously b u i l t i n elements that provide students with opportunities to in t e r a c t with Canadian society i n a va r i e t y of ways, both on and o f f campus. For example, as part of the requirement for LLED 206: Language F i e l d Experience, a compulsory course provided to the AEP by the Faculty of Education, every student must spend at least 30 hours i n a community placement as a volunteer or intern. Some past examples of community-outreach a c t i v i t i e s include: teaching swimming to children with d i s a b i l i t i e s , volunteering at seniors' centres, internships i n t r a v e l agencies, s t a f f i n g reception desks at community centres, helping to organise a trade show, working at the SPCA, volunteering with the Down Syndrome Research Society, creating news reports for the community-access cable channel, writing, for one of the Japanese community newspapers, working at a food bank, and research assistantships with a quasi-governmental agency. Unfortunately, there i s ample evidence to suggest that exposure alone i s i n s u f f i c i e n t for increasing i n t e r c u l t u r a l s e n s i t i v i t y . In contrast with our optimistic assumptions about the e f f e c t s of an international exchange experience, Bochner (1986) c i t e s a number of studies i n the s o c i a l psychology l i t e r a t u r e that indicate that: increased contact does not necessarily reduce i n t e r -group h o s t i l i t y , and under some conditions a c t u a l l y increases f r i c t i o n and animosity (Bloom, , 1971; M i t c h e l l , 1968; T a j f e l & Dawson, 1965). Even i n c u l t u r a l l y mixed (sic) r e s i d e n t i a l settings such as International Houses, where there are e x p l i c i t pressures to form c r o s s - c u l t u r a l friendships, studies i n the United States, England, and A u s t r a l i a have shown that the various groups prefer the company of t h e i r fellow nationals (Bochner, Buker, & McLeod, 1976; Bochner, Hutnik, & Furnham, 1985; Bochner, McLeod, & L i n , 1977; Bochner & Orr, 1979; Furnham & A l i b h a i , 1985; Furnham & Bochner, 1982). In many cases, the foreign students had not made a single host-country f r i e n d even a f t e r a lengthy sojourn (pp. 348-349). 1.3 Defining the F i e l d I t i s cl e a r that an empirical i n v e s t i g a t i o n of the AEP's impact was required since the quotation above c l e a r l y seems to contradict the very raison d'etre of exchange programmes. However, personal development i n the realm of c r o s s - c u l t u r a l s e n s i t i v i t y appears not to lend i t s e l f very e a s i l y to measurement, or even conceptualisation. The trend (see Adler, 1977; Gudykunst & Hammer, 1983, for example) had been to concentrate on observable t r a i t s that appeared to comprise i n t e r c u l t u r a l s e n s i t i v i t y . Bradford et a l . (1998) conducted a meta-analysis of intercultural-communication-competence research. While i n t e r c u l t u r a l communication i s not the subject of t h i s research, the studies incorporated i n the meta-analysis reinforced the impression that the main focus of the f i e l d i s on observable, performance-based measures or evidence of an a b i l i t y to perform successfully i n c r o s s - c u l t u r a l , environments. Bradford et a l . (1998) mention Bennett's (1986) model of development (see below), only as an example of a point of view outside of the mainstream. Most of the l i t e r a t u r e i n the f i e l d , as reviewed by Bradford et a l . (.1998) i s concerned with pperationalising overt and e x p l i c i t manifestations of i n t e r c u l t u r a l effectiveness and appropri atene s s. Cross-cultural learning i s addressed i n other l i t e r a t u r e s . Business journals, for example, examine the phenomenon from the perspective of maximising the adaptation and effectiveness of employees working i n c r o s s - c u l t u r a l environments. O'Sullivan (1999) looked at the impact of personality t r a i t s on t r a i n a b i l i t y i n the dynamic cross-c u l t u r a l s k i l l s that constrain the speed of adjustment. Selmer (1999) looked at the relat i o n s h i p between achieving career goals and international adjustment, as measured by subjective well-being. There i s also the ever-burgeoning f i e l d of cross-c u l t u r a l psychology which examines the impact of such c u l t u r a l constructs as individualism and c o l l e c t i v i s m on c r o s s - c u l t u r a l assimilation and t r a i n i n g (for examples, see Bhawuk, 1998, and Bhawuk & Triandis, 1996). . Ward and Kennedy (1999) have developed a measure of so c i o - c u l t u r a l adaptation that taps the behavioural, rather than the a t t i t u d i n a l domain. "[The s o c i o - c u l t u r a l domain] i s related to the a b i l i t y to ' f i t i n ' , to acquire c u l t u r a l l y appropriate s k i l l s and to negotiate i n t e r a c t i v e aspects of the host environment" (p. 660). I chose to p r i v i l e g e worldview over the behavioural domain i n my practice because behavioural accommodations can be learned without a f f e c t i n g the actor's attitudes towards other cultures. In other words, a person can learn a va r i e t y of behavioural adaptations that permit him or her to function i n a new culture without understanding why c e r t a i n behaviours are desirable or to be avoided. Thus, a sojourner can function e f f e c t i v e l y i n a foreign context while continuing to consider the other culture(s) s i l l y , i l l o g i c a l , quaint, or i r r e l e v a n t . In f a c t , as Bradford et a l . (1998) point out i n t h e i r discussion of the terms effective and competent, a sojourner can be e f f e c t i v e , from the point of view of accomplishing ; desired goals i n business negotiations for example, while acting l i k e the c u l t u r a l equivalent of a b u l l i n a china shop. A t t i t u d i n a l changes r e s u l t from the conscious, s e l f -r e f l e c t i v e processes that K e l l y (1963) c a l l e d construing and reconstruing one's experiences. Although t h i s does not always lead to a more favourable i n c l i n a t i o n towards the cultures i n question, (see discussion of Bennett's DMIS stages below) i t does engender a more complex and nuanced experience of l i f e . Outside the behavioural domain, attempts have been made to measure emotional/affective changes with psychometric tools such as the Zung S e l f - r a t i n g Depression Scale (Zung, 1965) or the P r o f i l e of Mood States (McNair, Lorr, & Droppleman, 1971). These instruments tap the processes of c u l t u r a l adaptation to a new environment, i . e . , culture-shock symptoms such as tension, depression, anger, fatigue, and confusion (Ward & Kennedy, 1999). Two of the better-known instruments i n the cross-culture domain, the Culture Shock Inventory (Reddin, 1994) and the Cross-cultural Adaptability Inventory (Kelly & Meyers, 1992) measure t r a i t s thought to be associated with c r o s s - c u l t u r a l adaptability or s e n s i t i v i t y ; also a r e f l e c t i o n of the t h e o r e t i c a l emphasis i n the f i e l d alluded to e a r l i e r . These t r a i t s include f l e x i b i l i t y , openness, and emotional strength. A newer entry to the f i e l d i s Matsumoto et a l . ' s (2001, as reported i n Matsumoto et a l . , 2003) I n t e r c u l t u r a l Adjustment Potential Scale (ICAPS). ICAPS i s s i g n i f i c a n t i n that i t was developed for Japanese sojourners and immigrants, instead of U.S. Americans, and recognises that t r a n s l a t i n g an instrument into another language i n e f f e c t creates a new instrument (more about that below). Its focus, however, i s s t i l l on adjustment, rather than c r o s s - c u l t u r a l s e n s i t i v i t y , and on the psychological t r a i t s that '• are posited to f a c i l i t a t e adjustment; emotion regulation, openness, f l e x i b i l i t y , and c r i t i c a l thinking. Chen and Starosta's (2000) I n t e r c u l t u r a l S e n s i t i v i t y Scale (ISS) i s another instrument that takes a s i m i l a r approach. The authors have posited that " i n d i v i d u a l s must possess s i x a f f e c t i v e elements to be i n t e r c u l t u r a l l y s e n s i t i v e : self-esteem, self-monitoring, open-mindedness, empathy, i n t e r a c t i o n involvement, and suspending judgment" (p. 6). They, therefore, attempted to validate the ISS by comparing correlations of scores on the instrument with e x i s t i n g scales believed to be related; i n t e r a c t i o n attentiveness, impression rewarding, self-esteem, s e l f -monitoring, and perspective taking. The c o r r e l a t i o n s obtained, while a l l s t a t i s t i c a l l y s i g n i f i c a n t , (ranging from r=.17 for the'self-esteem scale to r=.57 for the perspective-taking scale) are mixed. In s t a t i s t i c a l c i r c l e s a c o r r e l a t i o n of .57 i s considered to be outstanding. However, given that a c o r r e l a t i o n of .57 means that only 27% of the variance i s explained by the c o r r e l a t i o n , compounded by the f a c t that the r e l a t i o n s h i p among the variables i s speculative,. I believe the evidence i s at t h i s point s t i l l too tenuous to account for the construct. Some of these t r a i t s make i n t u i t i v e sense as p o t e n t i a l contributors to i n t e r c u l t u r a l s e n s i t i v i t y , but without an overarching t h e o r e t i c a l framework to explain the r e l a t i o n s h i p among them, a l o t of work remains to be done to demonstrate construct v a l i d i t y for the instrument. 1.4 A Framework for I n t e r c u l t u r a l S e n s i t i v i t y Both Matsumoto et a l . (2001) and Chen and Starosta (2000) made s i g n i f i c a n t contributions to the f i e l d i n exploring the possible constituent elements o f , i n t e r c u l t u r a l s e n s i t i v i t y . They are complementary to Bennett's (1986) Developmental Model of I n t e r c u l t u r a l S e n s i t i v i t y (DMIS). The DMIS describes the developmental stages through which one progresses on the journey to i n t e r c u l t u r a l s e n s i t i v i t y , but i s s i l e n t on how one makes that progress, or indeed on what the construct involves, except to echo Kelly's (1963) observation on the necessity to construe and re-construe experience. As grounded theory based on long years of f i e l d observation, the DMIS provides a t h e o r e t i c a l framework that sketches what increasing i n t e r c u l t u r a l s e n s i t i v i t y looks l i k e , i n a worldview rather than a behavioural sense. That makes i t possible to empirically observe and measure the process without necessarily measuring the constituent underlying components. Bennett's DMIS,. for example, accounts for the increase i n f r i c t i o n and animosity, p a r t i c u l a r l y for f i r s t - t i m e sojourners, described i n the quotation from Bochner (1986) above. The DMIS positions i t as the necessary and unavoidable intermediate step from t o t a l i n s u l a r i t y and lack of awareness of other cultures - the inevitable end r e s u l t of s o c i a l i s a t i o n into one's own culture (a stage that Bennett c a l l s 'Denial') -towards a broader appreciation of the r e a l i t y and worth of other c u l t u r a l perspectives. B r i e f l y , the DMIS describes the process of acquiring i n t e r c u l t u r a l s e n s i t i v i t y i n which people move through three ethnocentric stages to three, more i n c l u s i v e , ethnorelative stages. As mentioned above, the f i r s t stage, Denial, i s the default end-product of s o c i a l i s a t i o n into one's native culture. In Denial, a person i s insensible to the existence of other c u l t u r a l norms and worldviews, considering only his or her own culture or way of thinking to be the natural order of things. The la b e l Denial refers to the denial of the r e a l i t y of other c u l t u r a l perspectives or values, on any l e v e l worth , considering. When i t i s no longer possible to ignore the existence of other c u l t u r a l points of view, the natural reaction i s to f i r s t t r eat them with h o s t i l i t y and suspicion. This i s the stage that Bennett labels Defence. Defence can take the form . of a c t i v e l y denigrating other cultures or merely asserting the super i o r i t y of one's own. I t i s the worldview that underlies exasperated expatriate statements that begin with, 'Why can't these people learn how to . . .?' or 'Why do they do everything backwards here?' Further exposure to, and observation of, other c u l t u r a l points of view can lead to the t h i r d ethnocentric stage, Minimization. In Minimization, one views differences i n clothing, food, language, or customs as s u p e r f i c i a l , beneath which people are r e a l l y pretty much the same. This sounds l i k e a laudable, e g a l i t a r i a n point of view u n t i l one remembers that from an ethnocentric worldview, the standard for the sameness of people must be oneself. This worldview s t i l l does not allow for other cultures to have an independent r e a l i t y or v a l i d i t y . Moving beyond the ethnocentric stages, a person progresses to the DMIS stage of Acceptance, which, as the l a b e l implies, brings the r e a l i s a t i o n that other cultures do have an external r e a l i t y and are v a l i d on t h e i r own terms. Acceptance lays the foundation for Adaptation, both Cognitive and Behavioural, i n which other cultures' frames of reference and modes of acting are incorporated into one's own cognitive and behavioural repertoire. In the f i n a l stage, Integration, a person can act as a bridge between cultures, having absorbed and incorporated other c u l t u r a l worldviews. However, he or she operates at the margins of culture, having transcended the confines of a single worldview. The ambivalence contained i n the description of Integration, the use of the term 'margins', for example, i s i n t e n t i o n a l . In my understanding, at l e a s t , Integration i s not as easy to grasp as the other DMIS stages and not, as one might expect, simply the promised land at the end of the journey towards perfect i n t e r c u l t u r a l s e n s i t i v i t y . Bennett and Hammer (1998) explain that Integration "describes the e f f o r t at integrating disparate aspects of one's i d e n t i t y into a new whole while remaining c u l t u r a l l y marginal" (p. 16). Integration can manifest i t s e l f as "Constructive marginality ( i t a l i c s i n o r i g i n a l ) . . . the form of Integration that refers to the subjective experience of people who are attempting to integrate various c u l t u r a l i d e n t i t i e s into larger, more h o l i s t i c frames of reference" (p. 16) or Encapsulated Marginality, where one i s on the fringes, looking i n . Figure 1.1 Continuum of I n t e r c u l t u r a l S e n s i t i v i t y Continuum of Intercultural Sensitivity Mi l t on B e n n e t t - D e v e l o p m e n t a l M o d e l of In tercu l tura l S e n s i t i v i t y , 1 9 9 3 Defense S t r o n g d e f e n s e of o n e ' s o w n wo r l d v i e w Minimization T r i v i a l i z e s d i f f e r e n c e s ; f o c u s e s o n s im i l a r i t i es Denial D e n i e s that d i f f e r e n c e s ex i s t Adaptation C a p a b l e of t a k i n g t he o the r ' s po in t o f v i e w a n d c o m m u n i c a t i n g a c c o r d i n g l y • Acceptance R e c o g n i z e s a n d V a l u e s d i f f e r e n c e s Integration V a l u e s va r i e t y of c u l t u r e s i n t e g r a t e s that into b e h a v i o u r The DMIS i s a s o c i a l - c o n s t r u c t i v i s t model that posits that i t takes more than exposure, even of a prolonged nature, to a l t e r one's worldview. As George K e l l y (1963, o f t c i t e d by Bennett i n workshops and lectures) observed: A person can be witness to a tremendous parade of episodes and yet, i f he f a i l s to keep making something out of them . . . he gains l i t t l e i n the way of experience from having been around when they happened. I t i s not what happens around him that makes a man (sic) experienced; i t i s the successive construing and reconstruing of what happens, as i t happens, that enriches the experience of his l i f e (p. 73). In other words, i t i s not enough to be i n the v i c i n i t y of events as they occur, as Bennett l i k e s to remind us. One must interpret and make sense of those events i n order to learn from them, through a process of r e f l e c t i o n . In the context of i n t e r c u l t u r a l s e n s i t i v i t y , events must be observed i n order to y i e l d data about other cultures, but those data must then be interpreted ( r e f l e c t e d upon) to y i e l d information and then knowledge. Conscious r e f l e c t i o n upon the feelings and reactions the experience engenders, both towards other cultures and towards one's own, contributes to the growth and evolution of one's worldview. During the roughly f i f t e e n years I have spent l i v i n g i n Japan, some of i t as a ' c i v i l i a n ' and some as a member of Canada's diplomatic corps, and i n my continuing involvement with Japanese students, faculty, and s t a f f , I have had ample opportunity to see i n myself and others evidence of the stages that I have come to recognise from Bennett's DMIS model. I have also had the opportunity to learn to engage i n the types of s e l f - r e f l e c t i o n and s e l f - r e f l e c t i v e discussions that have permitted the construing and re-construing of events that K e l l y (1963) describes. A personal anecdote might serve to i l l u s t r a t e the concept. Part of my role as Executive Director i s to serve a l i a i s o n function between the two u n i v e r s i t i e s . In that capacity, I was acting as an intermediary between the Japanese university's administrative s t a f f and a professor v i s i t i n g from the Canadian university. The professor i s fluent i n Japanese and i s p e r f e c t l y capable of communicating with s t a f f d i r e c t l y , but the s i t u a t i o n a l context put me at the centre of these exchanges. I was given information concerning deadlines for submitting a syllabus, i t s length, format, etc. I passed the information on to the professor and she t o l d me she would take care of i t . I thought my duties had been f u l f i l l e d . Three weeks l a t e r , I received an angry c a l l from the administrative s t a f f e r t e l l i n g me that he hadn't received the syllabus and implying i t was my f a u l t . I explained to him that I had given the information to the professor and had received assurances that she would handle i t . I suggested to him that he contact her d i r e c t l y , since she was fluent i n Japanese, i f there was a problem. From a North American c u l t u r a l perspective, I had done as much as could be expected. I t would have been improper for me to check up on the professor to confirm she had done as she had promised. I would have been thought to be questioning her s i n c e r i t y or her competence. My f i r s t reaction as I hung up the phone was, 'What i s that guy's problem? I've done my b i t i n conveying the information to the professor. The res t i s between them.' Upon further r e f l e c t i o n , however, I r e a l i s e d (as I should have at the time of the phone conversation) that from a Japanese perspective the role of intermediary that I had accepted included r e s p o n s i b i l i t y f or seeing the task through to completion. There i s no opting out. After speaking to the professor, I found out she had been delayed by computer and other problems which, as a North American, she considered beyond her control, and therefore excusable. She had not.contacted the administrator to t e l l him the reason for the delay. I had to put aside my North American persona and c a l l the administrator back to apologise for f a i l i n g to f u l f i l my obligat i o n to see his request through. I also had to c a l l the professor and ask her to f u l f i l her obligations. I was able to do t h i s without embarrassment to eithe r of us by making my role as a c u l t u r a l bridge e x p l i c i t i . e . , that I was acting as way required by Japanese custom, i n our shared Japanese context. Without the r e f l e c t i v e stage, I probably would have wasted a considerable amount of psychic energy stewing about the absurdity of being asked to take r e s p o n s i b i l i t y f o r another competent adult's f a i l u r e to take action, and learned nothing about my ro l e as a c u l t u r a l intermediary as a r e s u l t . This i s something I have seen many expatriates do when faced with c r o s s - c u l t u r a l conundra. Extrapolating from K e l l y (1963), i t i s c l e a r that the question of whether students on educational exchange programmes become more i n t e r c u l t u r a l l y s e n s i t i v e and/or competent i s an empirical one. Answering that empirical question i s one way i n which I can evaluate whether the AEP i s providing participants with s u f f i c i e n t exposure and opportunity to provide s t i m u l i to s e l f r e f l e c t i o n and, by extension, s u f f i c i e n t opportunity for growth. 1.5 Measuring I n t e r c u l t u r a l S e n s i t i v i t y Bennett and Hammer (1994) attempted to operationalise the DMIS framework by creating the I n t e r c u l t u r a l Development Inventory (IDI), a psychometric instrument. The IDI i s an English-medium instrument that was developed and validated on English-speaking populations ( a l b e i t hot a l l of them U.S. Americans). In IDI t r a i n i n g workshops and other fora, the instrument's authors respond to questions about the t r a n s f e r a b i l i t y of the IDI to other than U.S. Americans or English speakers by saying that because the IDI was developed with the p a r t i c i p a t i o n of subjects from a large number of c u l t u r a l groups, i t can be administered to members of any culture, presumably i n any language. They also claim that the IDI has been 'validated' and i s therefore ready to p u l l o f f the shelf i n any context. In addition to the o r i g i n a l r e l i a b i l i t y and v a l i d i t y analyses by the authors (Bennett & Hammer, 1998), the IDI was the subject of a study by Paige, Jacobs-Cassuto et a l . (2003) that sought to "examine the empirical properties of the IDI" (p. 467), s p e c i f i c a l l y to see whether the IDI successfully tracked the underlying DMIS constructs, through a factor analysis. Paige et a l . , concluded that "the IDI i s a r e l i a b l e measure that has l i t t l e or no s o c i a l d e s i r a b i l i t y bias and reasonably, although not exactly, approximates the developmental model of i n t e r c u l t u r a l s e n s i t i v i t y " (p. 467). However, t h e i r study was also c a r r i e d out using U.S. American high school and college students, i n a u n i - c u l t u r a l , u ni-l i n g u i s t i c context. In order to compensate f o r the c u l t u r a l homogeneity of t h e i r subject pool, Paige et a l . chose t h e i r sample "for v a r i a b i l i t y on a set of personal c h a r a c t e r i s t i c s that could t h e o r e t i c a l l y be expected to c o r r e l a t e with i n t e r c u l t u r a l s e n s i t i v i t y (e.g., p r i o r i n t e r n a t i o n a l experience, amount of exposure to language education)" (p. 473). However, the c r o s s - c u l t u r a l and c r o s s - l i n g u i s t i c t r a n s f e r a b i l i t y and a p p l i c a b i l i t y of the DMIS and the IDI remain empirical questions. In v a l i d a t i n g the ICAPS, Matsumoto et a l . (2003) were also cognizant that the t r a n s f e r a b i l i t y of t h e i r instrument and the underlying psychological t r a i t s was subject to empirical v e r i f i c a t i o n . Although they argued that: . the same psychological s k i l l s may ( i t a l i c s added) be necessary for i n t e r c u l t u r a l adjustment of any i n d i v i d u a l s from any culture as they adjust to a d i f f e r e n t culture because the psychological s k i l l s underlying the process of managing i n t e r c u l t u r a l stress and c o n f l i c t may ( i t a l i c s added) be the same regardless of culture even though the manifest content of the c o n f l i c t i s c u l t u r a l l y s p e c i f i c " (p. 545) they acknowledge that, "We do not know ( i t a l i c s added) whether or not the ICAPS can predict i n t e r c u l t u r a l adjustment for individuals other than Japanese" (p. 545), and went on to report s i x studies that investigated the question. I took issue with t h e i r casual attitude to the t r a n s l a t i o n of the English o r i g i n a l into Japanese, " ( A ) l l measures were translated into Japanese and t h e i r accuracy was v e r i f i e d using back-translation with no problems" (p. 546) and proposed a rationale for a more comprehensive t r a n s l a t i o n protocol l a t e r i n t h i s paper. However, they acknowledge, by t r a n s l a t i n g the instrument, that language has the p o t e n t i a l to s i g n i f i c a n t l y a f f e c t the re s u l t s obtained. 1.6 L i n g u i s t i c Determinism In 1929, Edward Sapir f i r s t advanced the ideas that he and his student Benjamin Whorf would l a t e r propose as the theory of l i n g u i s t i c determinism. In a c l a s s i c passage, Sapir (1929) argued that: Human beings do not l i v e i n the objective world alone, nor alone i n the world of s o c i a l a c t i v i t y as o r d i n a r i l y understood, but are very much at the mercy of the p a r t i c u l a r language which has become the medium of expression for t h e i r society. I t i s quite an i l l u s i o n to imagine that one adjusts to r e a l i t y e s s e n t i a l l y without the use of language and that language i s merely an in c i d e n t a l means of solving s p e c i f i c problems of communication or r e f l e c t i o n . The fac t of the matter i s that the 'real world' i s to a large extent unconsciously b u i l t upon the language habits of the group. No two languages are ever s u f f i c i e n t l y s i m i l a r to be considered as representing the same s o c i a l r e a l i t y . The worlds i n which d i f f e r e n t s o c i e t i e s l i v e are d i s t i n c t worlds, not merely the same world with d i f f e r e n t labels attached. . . We see and hear and otherwise experience very large l y as we do because the language habits of our community predispose c e r t a i n choices of int e r p r e t a t i o n (p. 69). Sapir and Whorf were ahead of t h e i r time i n understanding that r e a l i t y i s constructed and that one must have categories for c l a s s i f y i n g experience, otherwise i t (the experience) i s meaningless. As Whorf (1956) put i t : We dissect nature along l i n e s l a i d down by our native languages. The categories and types that we i s o l a t e from the world of phenomena we do not f i n d there because they stare every observer i n the face; on the contrary, the world i s presented i n a kaleidoscopic flux of impressions which has to be organised by our minds. . . We cut nature up, organise i t into concepts, and ascribe significances as we do (pp. 213-214). The Sapir-Whorf Hypothesis states, i n i t s strong form, that language determines thought; that language and thought are i d e n t i c a l . L i n g u i s t i c determinism a t t r a c t s few followers today since there i s strong evidence against i t , including the p o s s i b i l i t y of t r a n s l a t i o n between languages. However, the weak version of the Sapir-Whorf Hypothesis, that language strongly influences thought by shaping our categories for c l a s s i f y i n g experience, i s much more widely accepted. In order to pursue a research agenda based on the DMIS as a t h e o r e t i c a l framework, and the IDI as an empirical instrument, my f i r s t task, therefore, was to examine the r e l i a b i l i t y of the IDI i n Japanese t r a n s l a t i o n , and the v a l i d i t y of inferences from data gathered from a population that i s c u l t u r a l l y and l i n g u i s t i c a l l y very d i f f e r e n t from the population on which the aforementioned s t a t i s t i c a l analyses were performed. The language above was chosen very d e l i b e r a t e l y . I t i s commonly believed that an instrument can be validated, and that once that has been accomplished the instrument i s ready for use i n any context. However, as Messick (1998) points out v a l i d i t y i s not the property of an instrument. I t i s a method for determining the appropriateness of inferences drawn from set of data. Messick (1998) describes v a l i d i t y thus: What needs to be v a l i d are the inferences made about score meaning, namely the score i n t e r p r e t a t i o n and i t s action implications for t e s t use. Because value implications both derive from and contribute to score meaning, d i f f e r e n t value perspectives may lead to d i f f e r e n t score implications and hence to. d i f f e r e n t v a l i d i t i e s of interpretation and use for the same scores. This suggests that v a l i d i t y might be indexed to values and perhaps contingent on d i f f e r e n t facts (or interpretations of facts) surrounding the conditions of use (p. 37). As we have seen from Sapir and Whorf above, language and culture are s i g n i f i c a n t determinants of values and perspective. I t was therefore c l e a r l y evident, to me at least that when an instrument i s translated into a d i f f e r e n t language and administered to a c u l t u r a l l y d i f f e r e n t group of subjects, v a l i d i t y must be examined and established empirically. CHAPTER TWO METHODOLOGY 2.1 Introduction The I n t e r c u l t u r a l Development Inventory (IDI) i s a s i x t y -item psychometric instrument designed to operationalise the Developmental Model of In t e r c u l t u r a l S e n s i t i v i t y (DMIS) (Bennett, 1986) by i d e n t i f y i n g test-takers' dominant positioning on a developmental continuum of i n t e r c u l t u r a l s e n s i t i v i t y , and worldview-related issues that remain to be resolved. The IDI i s a se l f - r e p o r t instrument using a seven-point L i k e r t scale with responses ranging from 'strongly disagree' through 'strongly agree'. The IDI i s a proprietary instrument under copyright to i t s authors. Anyone who wishes to work with the IDI must attend a q u a l i f y i n g workshop. C e r t i f i e d IDI administrators are required to sign an undertaking not to release the contents of the IDI i n whole or i n part ( i n d i v i d u a l items). As a r e s u l t , neither the o r i g i n a l instrument nor the Japanese t r a n s l a t i o n appear i n t h i s d i s s e r t a t i o n . Completing the IDI items requires a r e l a t i v e l y high l e v e l of English proficiency. The IDI's authors often state ( i n IDI q u a l i f y i n g workshops, for example) that the instrument i s impervious to the e f f e c t s of language and culture because the process of creating i t involved input from subjects from a v a r i e t y of c u l t u r a l backgrounds. A l l of that input, however, was i n English. In order to make the IDI accessible to my intended research pool, I led a team that translated the IDI into Japanese. That, i n turn, raised v a l i d i t y issues; the empirical question of whether inferences from data acquired from the IDI i n Japanese t r a n s l a t i o n would be equivalent to those acquired from the o r i g i n a l English version. The Standards for Educational and Psychological Testing (APA, AERA, & NCME, 1985) define v a l i d i t y as "the appropriateness, meaningfulness, and usefulness of the specific inferences made from test scores" ( i t a l i c s added) (p. 9). As the quotation above from Messick (1998) also makes clea r , instruments cannot be validated, only the s p e c i f i c inferences from the data (test scores) that they y i e l d . I t was important, then, to f i r s t e s t a b l i s h the v a l i d i t y of the inferences that might be made from the scores from the Japanese t r a n s l a t i o n of the IDI. Current v a l i d i t y theory, according to Messick (1998), t e l l s us that: A l l v a l i d i t y i s of one kind, namely, construct v a l i d i t y . Other so-called separate types of v a l i d i t y — whether l a b e l l e d content v a l i d i t y , c r i t e r i o n -related v a l i d i t y , consequential v a l i d i t y , or whatever — cannot stand alone i n v a l i d i t y arguments. Rather, these so-called v a l i d i t y types r e f e r to complementary forms of evidence to be integrated into an o v e r a l l judgment of construct v a l i d i t y (p. 37). In building t h i s evidentiary base, I f i r s t looked at the r e l i a b i l i t y of the scale scores, the extent to which they yielded consistent r e s u l t s , using Cronbach's c o e f f i c i e n t alpha and the Intraclass Correlation C o e f f i c i e n t because, as Hubley and Zumbo (1996) remind us, " r e l i a b i l i t y i s a necessary, but not s u f f i c i e n t condition for v a l i d i t y " (p. 208). Following the r e l i a b i l i t y analysis, I looked at the instrument's construct v a l i d i t y , the extent to which i t represented the construct, through an examination of content-validity evidence, c r i t e r i o n - r e l a t e d v a l i d i t y evidence, and a p r i n c i p a l components analysis. 2.2 Content V a l i d i t y The DMIS i s a product of grounded theory that evolved from a long period of observing a phenomenon; i n t h i s case, Milton Bennett's long and thoughtful observation of i n t e r c u l t u r a l development. The items that make up the IDI were selected from among statements made i n interviews by various people, U.S. Americans and others from various c u l t u r a l backgrounds l i v i n g i n the United States, about how they viewed and interacted with cultures other than t h e i r own. A large number of statements were whittled down, using t r a d i t i o n a l test-development methods (see the Discussion section below for a detailed description), to the s i x t y statements that comprise the f i r s t version of the IDI. Given the grounded nature of the development of the instrument, I thought that the f i d e l i t y of the t r a n s l a t i o n to the o r i g i n a l would be one piece of evidence f o r content v a l i d i t y . However, t h i s was not as straightforward as demonstrating l i n g u i s t i c equivalence with translation/back-t r a n s l a t i o n or other measures for assessing t r a n s l a t i o n q u a l i t y . The process of developing the o r i g i n a l IDI used input only from participants ( a l l i n the United States although not necessarily U.S. American), who were s u f f i c i e n t l y p r o f i c i e n t i n English to p a r t i c i p a t e i n interviews and to contribute statements that could be used verbatim i n the item pool. Given the r e l a t i o n s h i p between language and c u l t u r a l schemata, I knew that I could not take for granted the t r a n s f e r a b i l i t y of the concepts measured by the IDI or inferences from i t s r e s u l t s , to a c u l t u r a l l y and l i n g u i s t i c a l l y very d i f f e r e n t group of subjects. 2.3 Procedures for Translating the IDI Bennett and Hammer (1998), i n reporting t h e i r IDI development and v a l i d a t i o n procedures, stressed the cross-c u l t u r a l a p p l i c a b i l i t y of the instrument ( i . e . , that a s u f f i c i e n t l y diverse cross-section of d i f f e r e n t languages and cultures had been represented i n t h e i r subject pool to eliminate any possible e f f e c t s on the r e s u l t s from c u l t u r a l and native-language differences) so I thought i t was important to t e s t that assertion by p r i v i l e g i n g a t r a n s l a t i o n that was f a i t h f u l to the wording and concepts of the o r i g i n a l . However, a number of the translators commented on the 'foreignness' to the Japanese mind of some of the concepts appearing i n the instrument. These items were flagged for attention i n the s t a t i s t i c a l analyses (see Results chapter). There i s also a difference between t r a n s l a t i n g and adapting instruments. Ideally, the process of t r a n s f e r r i n g an instrument from one language to another should be a process of adaptation, i n which the p i t h of the t h e o r e t i c a l concepts i s given precedence. Test adaptation, includes such a c t i v i t i e s as (1) deciding whether or not a t e s t can measure the same construct i n a d i f f e r e n t language or culture, (2) sel e c t i n g t r a n s l a t o r s , (3) deciding on appropriate accommodations to be made i n preparing a t e s t f o r use i n a second language, and (4) adapting the t e s t and checking i t s equivalence i n the adapted form (Hambleton and Patsula, 1998, p. 155). Test adaptation also sometimes requires generating a new item where conceptual equivalence i s d i f f i c u l t to achieve (Kristjansson, Desrochers, & Zumbo, 2003). Although the process I followed incorporates many of the points enumerated by Hambleton and Patsula (1988) above, the Japanese-version IDI created for t h i s study was the product of a conscious process of t r a n s l a t i o n rather than adaptation. For example, I t r i e d to avoid making judgments on conceptual equivalence and accommodations because the c r o s s - c u l t u r a l t r a n s f e r a b i l i t y of the o r i g i n a l instrument and i t s concepts was the object of empirical v e r i f i c a t i o n i n t h i s study. The IDI was f i r s t translated into Japanese f o r an e a r l i e r study (Greenholtz, unpublished) that attempted to determine the l e v e l of English proficiency, as measured by the Test of English as a Foreign Language (TOEFL), necessary to meaningfully complete the IDI i n English. I t was the lack of any c l e a r c o r r e l a t i o n between TOEFL scores arid evidence of .. a b i l i t y to complete the IDI that convinced me to use a Japanese t r a n s l a t i o n for t h i s study. B r i e f l y , that o r i g i n a l study asked 100 native-Japanese-speaking u n i v e r s i t y undergraduates at various l e v e l s of English proficiency each to translate ten of the s i x t y IDI items into Japanese. They were also asked to state t h e i r l e v e l of confidence with t h e i r t r a n s l a t i o n . That was an e f f o r t to i n d i r e c t l y i d e n t i f y concepts i n the instrument that might pose e x t r a - l i n g u i s t i c (conceptual) d i f f i c u l t i e s f o r Japanese speakers. The translations were rated and s t a t i s t i c a l l y analysed with TOEFL scores as the independent v a r i a b l e . As noted e a r l i e r , no c l e a r c o r r e l a t i o n with TOEFL scores emerged with predictive value from those data. In order to rate the undergraduates' responses f o r that study, i t had been necessary to create a Japanese IDI 'master' t r a n s l a t i o n . Four Japanese-English b i l i n g u a l s (native Japanese speakers) were each asked to independently translate a l l s i x t y items into Japanese. They were judged to be b i l i n g u a l because they had each completed post-graduate work i n the United States (one at the doctoral l e v e l ) . One of the t r a n s l a t o r s , who had completed an MBA at an American u n i v e r s i t y , was working i n the United States for a large Japanese multinational corporation. Two others, one of whom was also a c e r t i f i e d IDI administrator, were experienced English teachers working i n Japan. The fourth t r a n s l a t o r had completed a PhD from Columbia University i n second-language a c q u i s i t i o n , and was also l i v i n g i n the United States. My q u a l i f i c a t i o n s for leading the t r a n s l a t i o n team were as follows. I am a native English speaker who has spent over half of my adult l i f e i n Japan. I passed the highest l e v e l of the Japanese Language Proficiency Test (Nihongo Noryoku Shiken, Japan Foundation) i n 1984 and have done a large number of Japanese-to-English translations and published an English-to- Japanese t r a n s l a t i o n of My Friend David (Edwards and Dawson, 1983) e n t i t l e d Mai Frendb Deibido (Greenholtz and Morita, 1988). I was c e r t i f i e d as a Japanese-English court interpreter by the government of Ontario i n 1991 and serve as a consecutive interpreter i n a v a r i e t y of contexts. I am also a c e r t i f i e d administrator of the IDI. I selected a f i n a l version from the pool of items (the four independent translations) and t h i s version was given to two raters, native-Japanese-speaking doctoral candidates at a Canadian university, to rate the undergraduates' t r a n s l a t i o n s . The r a t i n g exercise required the raters to consider the items much more deeply than simply comparing them to the English o r i g i n a l s or performing a back t r a n s l a t i o n would have, because they had to j u s t i f y the score they had assigned to each of the tr a n s l a t i o n s . Following the r a t i n g phase, I discussed each item with the two raters for f i d e l i t y to the language and concepts of the o r i g i n a l , within the constraints of natural Japanese. Several changes to wording of items were suggested and many of those changes were incorporated into what became the Japanese-language version used i n the present study. This procedure f a r exceeded the 'translate/back-translate' procedure that had long been considered the 'gold standard' for t r a n s l a t i n g material into other languages. The method employed exceeded even the 'new gold standard' described i n Kristjansson, Desrochers, and Zumbo (2003) who ( c i t i n g Behling & Law, 2000; Hambleton & Patsula, 1998) point out, " d i r e c t t r a n s l a t i o n and back t r a n s l a t i o n can deal with l i t e r a l meaning only", and "(B)ack t r a n s l a t i o n cannot detect differences i n conceptual understanding of the question, and so cannot ensure psychological equivalence of the items i n a scale or questionnaire" (p. 135). Although my inten t i o n was to use as l i t e r a l a t r a n s l a t i o n as possible, I also needed to be cognizant of conceptual inconsistencies or d i f f i c u l t i e s posed to Japanese speakers by the IDI because I wasn't interested only i n producing a t r a n s l a t i o n . I was also examining whether there was a s u f f i c i e n t l y strong evidentiary v a l i d i t y argument for using the t r a n s l a t i o n . As part of that process, the conceptual nuances helped me to make sense of the p r i n c i p a l components analysis. The t r a n s l a t i o n process i s summarised i n Table 2.1, below. Despite the multi-step process followed i n the tra n s l a t i o n , a few c l e r i c a l errors slipped i n that were discovered only a f t e r the tes t was administered. One was a typographical error i n which thie transformation from Roman characters to Japanese text was only p a r t i a l l y completed, leaving an English l e t t e r i n the middle of a Japanese word. None of the many participants questioned a f t e r t h i s error was discovered thought that the typographical error obscured the item's meaning i n any way. Table 2.1 Translation Protocol Step Procedure Notes One Whole instrument translated independently by each of four Japanese-English b i l i n g u a l s Create a r i c h pool from which to select items Two Four versions compared and working version synthesised Item s e l e c t i o n based on f a m i l i a r i t y with the instrument Three Two Japanese-English b i l i n g u a l s use a d r a f t version of the master t r a n s l a t i o n to rate English-Japanese translations by undergraduate students of varying l e v e l s of English proficiency Having raters use the working version rather than just examine i t adds richness to t h e i r comments Four Working version modified as a r e s u l t of comments by raters from Step Three Discussion to c l a r i f y comments Five Japanese-English b i l i n g u a l completes both o r i g i n a l and Japanese versions consecutively Item scores compared and discrepancies i n scores examined to see i f they were due to the language of the items In two of the items a modifier, 'some', was l e f t out, i n another 'often' was omitted, and i n a fourth, 'nearly'. These items were a l l flagged for attention during the analysis (see Results). F i n a l l y , there was an ambiguous choice of characters (between two characters with the same reading, and s i m i l a r meaning, that are, however, followed by d i f f e r e n t verb complements). This item was also flagged for attention during the analysis. These errors came to l i g h t when my research assistant (RA), a Japanese-English b i l i n g u a l undergraduate student, completed both the English and Japanese versions at one s i t t i n g . We compared her scores on an item-by-item basis and where they were d i f f e r e n t , t r i e d to determine why. Some of the differences were due to items that had concepts that d i d not transfer well to a Japanese c u l t u r a l framework (as flagged by the tr a n s l a t o r s , see discussion above), but were conceptually available to her in the original English version. The others revealed the errors discussed above. Based on t h i s experience, I would recommend that t h i s technique (having a b i l i n g u a l complete both versions of the instruments at the same s i t t i n g and comparing the r e s u l t s on an item-by-item basis) be added to the protocol f o r translating/adapting instruments. Although t h i s should have been done during the t r a n s l a t i o n phase, the information about these items was s t i l l useful i n the analysis. 2.4 Assessing the Adequacy of Construct Representation 2.4.1 Participants Four hundred students completed the Japanese version of the IDI, over a three-year period. A l l of the IDIs were completed before the students began t h e i r exchange experience. The students were a l l undergraduates at a large private uni v e r s i t y i n Japan. Two hundred eighty-seven (287) were female and 113 were male. Three hundred and seventy-one (371) of the students were i n second year, twenty-seven (27) i n t h i r d year, and two (2) were fourth-year students. Participants ranged i n age from nineteen (19) to forty-eight (48) years. Ninety-five (95) percent of the par t i c i p a n t s were between nineteen and twenty-one years of age. These IDIs provided the data for assessing the construct v a l i d i t y of the Japanese version. 2.4.2 Principal Components Analysis The 400 IDIs were analysed using p r i n c i p a l components analysis, f i r s t with Direct Quartimin r o t a t i o n (see Zumbo & Taylor, 1993) to determine whether there were any correlations among the factors. Once i t was confirmed that the i n t e r - f a c t o r correlations were small, Varimax rotation was used to determine whether the factor structure of the Japanese version matched that of the English o r i g i n a l , for t h i s population. The resu l t s were further confirmed with a factor analysis (Maximum Likelihood with Varimax r o t a t i o n ) . A hybrid exploratory/confirmatory approach was used for the p r i n c i p a l components analysis. I t was not e n t i r e l y exploratory because the res u l t s of the analyses of the o r i g i n a l English IDI guided the determination of the general parameters for t h i s analysis. However, since i t wasn't a certainty that the Japanese version would be i d e n t i c a l to the o r i g i n a l , the analysis was not wholly confirmatory e i t h e r . In addition, following Kelloway's (1998) guideline that confirmatory factor analysis requires a 10:1 subject to item r a t i o , the data set, with only 400 subjects for a sixty-item instrument was not large enough for a purely confirmatory factor analysis. P r i n c i p a l components analyses were performed for four-through seven-factor models to f i n d the best f i t . Knowing that there ought to be i n the neighbourhood of s i x factors permitted the s e l e c t i o n of i n i t i a l Eigenvalues greater than 1.5 (rather than the more t r a d i t i o n a l rule-of-thumb of 1.0), which greatly s i m p l i f i e d the analysis. The four-, f i v e - , s i x - , and seven-factor models were analysed and interpreted using the rotated factor matrix, the scree p l o t , and nonredundant residuals with absolute values greater than 0.05, to determine which made the most sense both s t a t i s t i c a l l y and i n t e r p r e t i v e l y . CHAPTER THREE RESULTS Following Zumbo and Taylor (1993) an analysis was f i r s t performed with Direct Quartimin rotation (Direct Oblimin with Delta set at zero) to determine whether there was any cor r e l a t i o n between the factors (see Table 3 .1 ) . Factors with i n i t i a l Eigenvalues of 1.0 or greater were retained. This yielded fourteen factors that accounted for 57.56% of the variance. The analysis and v a l i d a t i o n of the o r i g i n a l IDI (Bennett and Hammer, 1998) led me to expect somewhere i n the neighbourhood of six factors so factors with i n i t i a l Eigenvalues of greater than 1.5 were retained instead. This brought the number of factors down to seven, accounting for 42.296% of the variance. Table 3.1 Component Correlation Matrix Component 1 2 3 4 5 6 7 1 1.000 2 .093 1.000 3 -.042 -.088 1.000 4 .187 .224 -.116 1.000 5 -.205 -.015 -.052 -.319 1.000 6 -.087 .039 -.058 -.087 .145 1.000 7 .167 -.010 -.077 .022 -.012 .011 1.000 Extraction Method: P r i n c i p a l Component Analysis. Rotation Method: Oblimin with Kaiser Normalization. The Kaiser-Meyer-Olkin measure of sampling adequacy was used (Kaiser, 1970 as c i t e d i n Zumbo & Taylor, 1993) y i e l d i n g a respectable value of .85 (Table 3.2). Table 3.2 KMO Measure of Sampling Adequacy and B a r t l e t t ' s Test of Sphericity Kaiser-Meyer-Olkin Measure of Sampling Adequacy. .850 B a r t l e t t ' s Test of Sphericity Approx. Chi-Square 8031.29 df 1770 Sig. .000 Once i t was determined that the corr e l a t i o n s among the factors were very small, four-, f i v e - , s i x - , and seven-factor solutions were t r i e d with Varimax rot a t i o n . This decision was also based on foreknowledge from the va l i d a t i o n process for the English-language ( o r i g i n a l version) of the IDI (Bennett & Hammer,. 1998) and a second analysis by Paige et a l . (2003) in d i c a t i n g that the instrument measured t six discrete factors. The scree plo t (Figure 3.1), rotated factor matrix (Table 3.3), and nonredundant residuals were a l l considered i n determining which model yielded the best i n t e r p r e t a t i o n of the data. The scree plo t i d e n t i f i e d f i v e f a i r l y c l e a r factors. I t i s always d i f f i c u l t to decide where the scree begins and factors s i x and seven were i n a pos i t i o n that made i t d i f f i c u l t to judge whether they were s i g n i f i c a n t or not. The reason for t h i s ambiguity became cleare r i n the analysis of the rotated factor matrix. The progression from four- to seven-factor solution models brought the nonredundant residuals with absolute values greater than .05 down from 30% to 20%. Figure 3.1 Scree Plot Scree Plot 12 '-10 f] 1 7 13 19 25 31 37 43 49 55 4 10 16 22 28 34 40 46 52 58 Component Number Table 3.3 Rotated Factor Matrix Factor Item 1 2 3 4 6 24-(mo) Cognitive Adaptation-Bridge Builder .673 .024 -.025 .257 -.041 .015 -.015 18-Cognitive Adaptation-Bridge Builder .633 .030 .017 .085 -.062 .048 .033 53-Cognitive Adaptation-Bridge Builder .606 -.104 -.019 -.140 -.089 -.048 -.027 50-Behavioural Adaptation-Behavioural Shift .597 .044 .226 .134 -.012 .088 -.016 60-Cognitive Adaptation-Bridge Builder .597 .061 .225 .020 .047 -.075 .087 25-Cognitive Adaptation-Frame Shifting .589 .021 .185 -.006 .058 .011 .015 52-Cognitive Adaptation-Frame Shifting .573 -.079 -.086 -.058 -.071 -.152 .036 46-Cognitive Adaptation-Frame Shifting .547 -.066 -.025 -.023 .020 -.048 -.031 37-Denial-Disinterest .536 -.006 .014 .286 .039 -.005 .020 36-Behavioural Adaptation-Cultural Complexity .499 -.092 -.038 .085 .032 -.045 -.062 26-Behavioural Adaptation-Cultural Complexity .462 .050 .083 .092 -.031 .035 .076 13-Behavioural Adaptation-B ehavioural Shift .442 .093 .069 -.075 -.051 -.024 .023 35-Behavioural Adaptation-Cultural Complexity .440 .041 .103 .063 -.096 .008 .049 3-Cognitive Adaptation-Frame Shifting .405 .019 .070 -.006 .091 .014 .005 9-(nearly) Behavioural Adaptation-Cultural Complexity .385 -.093 .087 .247 -.105 .048 -.070 7-Behavioural Adaptation-Behavioural Shift .365 .038 .057 .066 .131 -.108 .002 58-Behavioural Adaptation-Behavioural Shift .331 -.041 .254 -.164 -.067 -.139 .047 17-Acceptance-Enjoying Difference .322 .126 .212 .165 .024 .094 -.036 54-Behavioural Adaptation-Behavioural Shift .268 -.054 .154 -.222 -.108 -.100 .035 Factor Item 1 2 3 4 20-Defence-Denigration .015 .732 .135 -.005 -.045 .115 28-Defence-Denigration .056 .708 .174 .051 .063 .052 .003 41 -Defence-Denigration .054 .681 .237 .169 .035 -.069 .033 16-Defence-Superiority -.004 .678 .125 .042 -.006 .100 .036 55-Defence-Superiority -.058 .653 .214 .216 -.046 -.031 .036 11-Defence-Superiority .010 .644 .042 .013 .077 .095 -.015 39-Defence-Superiority -.023 .582 .228 .095 .196 .018 -.073 10-Defence-Denigration -.034 .517 .123 .098 -.011 .087 .035 44-Defence-Denigration .049 .514 .137 .377 .045 -.021 .053 56-Defence-Superiority -.108 .458 .222 .035 .202 .060 -.034 14-Denial-Avoidance Separation .030 .240 .153 .141 -.092 -.024 -.098 31-Cognitive Adaptation-Multiple Perspective .057 .070 .566 .050 .009 -.068 -.035 29-Acceptance-Enjoying Difference .019 .119 .564 .196 -.106 -.022 .103 32-Acceptance-Describing Difference .025 .090 .524 -.052 .085 .047 .132 57-Denial-Dis interest .117 .262 .509 .147 .017 .012 -.058 27-Acceptance-Learning Difference .343 .112 .470 .190 -.045 -.014 -.032 34-Minimisation-Superficial Differences .001 -.139 -.460 -.143 .419 -.046 .067 21 -Acceptance-Value Relativity .069 .183 .448 .089 .008 .073 .047 15-Denial-Disinterest .108 .233 .423 .209 -.066 .004 -.054 19-Acceptance-Learning Difference .105 .099 .391 .071 -.068 .066 -.004 33-Acceptance-Describing Difference .040 .171 .377 .077 -.050 -.020 -.045 1 -Denial-Disinterest .059 .167 .359 .143 -.027 -.047 -.160 47-Acceptance-Enjoying Difference .073 .140 .323 .075 .028 .098 .052 45-Cognitive Adaptation-Multiple Perspective .245 .028 .283 -.035 -.119 .055 .128 42-Behavioural Adaptation-Behavioural Shift .201 .052 .224 -.030 -.160 .140 .057 Factor Item 1 2 3 4 • * 7 38-Denial-Avoidance Separation .096 .222 .277 .668 -.008 -.019 .028 40-Denial-Avoidance Separation .151 .216 .264 .655 .041 .046 -.091 43-Denial-Avoidance Separation .081 .333 .199 .640 .054 .027 -.033 49-Denial-Avoidance Separation .186 .241 .339 .566 -.068 .015 -.030 30-Denial-Avoidance Separation .103 .298 .266 .364 -.065 .052 -.007 6-Min im isation-Human Similarities -.111 .129 -.073 -.167 .126 .021 .115 22-Min im isation-Human Similarities -.004 -.005 -.071 -.056 .670 -.010 .173 4-Minimisation-Superficial Differences .023 .015 -.091 .045 .564 .025 -.008 51 -Min im isation-Human Similarities -.100 .187 .038 -.152 .498 .206 -.053 8-Minimisation-Superficial Differences .032 .048 .033 .054 .477 .083 -.046 5 -Min im isation-Uni versal Values -.022 .033 -.068 .022 .429 .091 .097 59-(sb) Minimisation-Universal Values -.075 .112 .054 .057 .101 -.041 12-(sb) Minimisation-Universal Values -.087 .149 .060 .023 .168 .695 .065 23-(sb) Minimisation-Universal Values -.052 .056 .040 .000 .194 .305 .156 2-(ms) Acceptance-Value Relativity .034 -.019 .039 -.028 .124 .092 .851 48-(ms) Acceptance-Value Relativity .123 .011 .038 -.054 .053 .024 .624 % of Variance 14.923 9.436 5.132 3.757 3.499 2.880 2.668 Cumulative % of Variance 14.923 24.359 29.491 33.248 36.748 39.628 42.296 Eigenvalues 8.954 5.661 3.079 2.254 2.100 1.728 1.601 In analysing the rotated factor matrix (Table 3.3), t r a d i t i o n a l rule-of-thumb indicators for factor loadings (.45 or more i s usually considered a strong loading, loadings between .30 and .45 are moderate, and loadings less than .30 are usually considered too weak) were used. The t r a n s i t i o n zones between factors contained items that loaded at nearly .30, but also loaded almost as highly across the matrix with other factors. These were reduced i n number with successive i t e r a t i o n s from four to seven factors. F i n a l l y , given that these items did not belong to the factor indicated by the rest of the items they were no t i o n a l l y excluded from the o v e r a l l judgment regarding what the factor mainly indicated. This process was greatly aided by choosing the SPSS option to have the factor loadings l i s t e d by s i z e . The seven-factor solution was the most s a t i s f a c t o r y . I t yielded f i v e f a i r l y coherent factors, the f i r s t of which appeared to be comprised of two subscales, and two additional factors that r e f l e c t e d issues raised i n the t r a n s l a t i o n process (see Discussion). 3.1 The Seven-Factor Model In the seven-factor model, the f i r s t f actor corresponded to the IDI's Adaptation dimension, although i t came out as one factor rather than the two, Cognitive Adaptation and Behavioural Adaptation, that the o r i g i n a l IDI contains. Having said that, while they could not be said to be separate factors, i t was generally true that Cognitive Adaptation items were clustered together with higher loadings than the Behavioural Adaptation items, so a convincing argument can be e a s i l y made for two subscales within a main fa c t o r . This corresponds with the IDI's structure. Overall, f i f t e e n of the twenty Adaptation items loaded on t h i s factor. The second factor was c l e a r l y the IDI's Defence dimension, with a l l ten items loading cleanly and exclusively (factor loadings ranging from .754 to .499) on t h i s factor. The t h i r d factor was s l i g h t l y less clean. I t contained seven of ten items from the IDI's Acceptance dimension, but the highest loading item on the factor (.649) was an orphaned Cognitive Adaptation item and there were also three items from the Denial dimension i n the mix. Overall, the factor was dominated by the Acceptance dimension, with loadings for Acceptance items ranging from .612 to .380. Given that Acceptance and Adaptation are adjacent dimensions on the scale, the fact that an Adaptation item would load highly on the Acceptance factor i s not beyond comprehension. In addressing the r e l a t i v e l y lower i n t e r n a l consistency of the Acceptance dimension, Paige et a l . (2003) noted the "complexity and multidimensionality of t h i s f i r s t ethnorelative stage, which i n operational terms, i s r e f l e c t e d i n a larger v a r i e t y of statement types used to measure t h i s construct" (p. 483). Four subscales, describing difference, enjoying difference, learning difference and value r e l a t i v i t y a l l co-exist within the ten-item scope of t h i s dimension. An alternative to Paige et a l . ' s i n t e r p r e t a t i o n w i l l be presented i n the Discussion chapter. The fourth factor (following a t r a n s i t i o n area containing two orphaned Adaptation items) contained f i v e of the items from the Denial scale. This i s only half of the Denial items, but they c l e a r l y dominated the factor with loadings ranging from .690 to .381. The f i f t h factor corresponded to the Minimization dimension of the IDI, containing s i x of the ten Minimization items, although the l a s t item loaded almost as strongly on the Acceptance factor. This i s somewhat surp r i s i n g because although Minimization and Acceptance are adjacent dimensions, they are on opposite sides of the Ethnocentric — Ethnorelative divide. The other two factors did not f i t the IDI structure, but made sense i n the context of conceptual d i f f i c u l t i e s that the Japanese speakers brought to l i g h t during the t r a n s l a t i o n process. The s i x t h factor was a neat c l u s t e r of the three items on the Universal Values subscale of the Minimisation dimension that contained references to a l l humans being 'children of a s p i r i t u a l being'. A l l of the native-Japanese-speaking translators found t h i s to be a t r u l y foreign concept. One feature of Japanese that makes i t so welcoming of foreign loan words i s a s y l l a b i c s c r i p t (katakana) used almost exclusively for rendering those loan words into the nearest Japanese phonetic equivalent. (How the meanings of those words di f f u s e through the Japanese l e x i c a l awareness should be the topic of what I am sure would be a fascinating s o c i o - l i n g u i s t i c study). Thus i t was possible to have the phrases, rendered phonetically i n katakana, marked with an a s t e r i s k leading to a glossary at the bottom of the form l i s t i n g a l t e r n a t i v e dictionary d e f i n i t i o n s . Although three of s i x t y items i s hot that s i g n i f i c a n t i n the larger scheme of things, and do not by themselves compromise the u t i l i t y of the instrument for a Japanese-speaking population, i t was cause for some r e f l e c t i o n within the context of a multi-dimensional search f o r v a l i d i t y evidence. This problem w i l l also be dealt with at greater length i n the Discussion section. The seventh factor contained two items that had been flagged from the t r a n s l a t i o n for two reasons. One was the omission of the modifier 'some' from the Japanese rendition. The second was that they were actually the same item. The o r i g i n a l English items are nearly i d e n t i c a l except that one asks whether i t i s appropriate for cultures to have d i f f e r e n t conceptions of 'right and wrong' and other, d i f f e r e n t conceptions of 'good and bad'. I hadn't noticed that the translators had used the same term i n both instances (and neither, apparently, had they) u n t i l my RA pointed i t out when we were examining why her scores for those items d i f f e r e d between the English and Japanese versions. The difference ( i n her case) was due to the omission of the word 'some' from the translations, (she also hadn't noticed that the same Japanese t r a n s l a t i o n had been used for both items) but that discussion brought to the l i g h t the fact that the items were i d e n t i c a l . Whether the c l u s t e r i n g on a c l e a r l y d i s t i n c t seventh factor (those items had the highest loading i n the e n t i r e matrix of .798 and .768 respectively) was due to the omission of 'some' (unlikely, i n my view, since the omission of the word 'often' from item 24 did not prevent i t from having the highest loading on the Adaptation factor) or the f a c t that the items were the same, (that again being no explanation as to why they should form a d i s t i n c t factor) or to the f a c t that the concept of 'good and bad' somehow represented a discrete factor to Japanese participants, i s unclear. Further discussion with other Japanese-English b i l i n g u a l s has revealed that i t i s d i f f i c u l t for them to provide c l e a r l y .distinct l e x i c a l alternatives ('right' and 'wrong' usually comes out f i r s t as some v a r i a t i o n of 'correct' and 'incorrect', i . e . , as adjectives rather than nouns). This i s not to say that l e x i c a l equivalents cannot be found, only that they must be a c t i v e l y searched f o r , or perhaps more context provided i n the sentence. One tr a n s l a t o r suggested that without a complement the concept would prove s i g n i f i c a n t syntactic d i f f i c u l t i e s i n Japanese. Because the true nature of the seventh factor was compromised i n a variety of ways and the s i x t h factor i s explainable i n terms of the foreignness of the concept of ' c h i l d of a s p i r i t u a l being', I judged that the Japanese version of the I'D I could be characterised as having f i v e c l e a r factors, with one, Adaptation, comprising two sub-factors. For confirmation I ran a factor analysis (Maximum Likelihood with Varimax rotation) on the seven-factor model. The r e s u l t s of t h i s analysis were e s s e n t i a l l y i d e n t i c a l with the p r i n c i p a l components analysis (some items within the factors were i n a d i f f e r e n t order) except that the nonredundant residuals with an absolute value greater than 0.05 were further reduced to 15%. Another piece of the evidentiary puzzle for v a l i d i t y i s r e l i a b i l i t y . To demonstrate the r e l i a b i l i t y of the Japanese IDI Cronbach's c o e f f i c i e n t alpha was computed. The r e s u l t s (see Table 3.4) yielded a c o e f f i c i e n t alpha of 0.854. Table 3.4 Cronbach's C o e f f i c i e n t Alpha R e l i a b i l i t y C o e f f i c i e n t s N of Cases = 400.0 N of Items = 60 Alpha = .8540 These analyses appeared to e s t a b l i s h the e s s e n t i a l equivalence of the Japanese t r a n s l a t i o n of the IDI with the o r i g i n a l English version. They r e f l e c t quite c l o s e l y r e s u l t s reported by Bennett and Hammer (1998) and Paige et a l . (2003). S t a t i s t i c a l analyses can be accepted u n c r i t i c a l l y at face value; factor loadings above .45 or correlations of .8 or greater t r o t t e d out to 'demonstrate' v a l i d i t y , but the va l i d a t i o n process r e a l l y requires active i n t e r p r e t a t i o n on the part of the researcher, whatever story the numbers might appear to t e l l . Looking beneath the numerical r e s u l t s , the data r a i s e questions about whether the IDI (and by extension, the DMIS that i t i s meant to operationalise) i s 'culture free' or applicable to any c u l t u r a l context. This w i l l be further explored i n the Discussion chapter. , . CHAPTER FOUR DISCUSSION 4.1 V a l i d i t y Revisited This research was not i n i t i a l l y intended to be about v a l i d i t y issues or the examination of v a l i d i t y i n a cross-c u l t u r a l context. When I learned about the IDI, I had been excited to f i n d out that a ' r e l i a b l e and v a l i d ' instrument existed to operationalise the DMIS, and was keen to use the IDI as a t o o l for gaining some empirical insights into the effectiveness of the academic exchange programme I was administering. Like most p r a c t i t i o n e r s , I was mostly unaware that there might be reasons to be cautious about using i t i n my p a r t i c u l a r research context. However, the English-language proficiency of my subject pool was not s u f f i c i e n t l y high to use the IDI i n i t s o r i g i n a l form and that necessitated a t r a n s l a t i o n into Japanese. I knew enough to suspect that changing the language of the instrument might a f f e c t i t s r e l i a b i l i t y and v a l i d i t y , although I was s t i l l not very conversant i n v a l i d i t y issues. I had hoped to quickly 'validate' the Japanese version of the IDI i n order to use the instrument i n the next stage of my research, not yet knowing that v a l i d i t y was a property of inferences drawn from data, not of the instrument i t s e l f . Further i n v e s t i g a t i o n into the f i e l d of v a l i d i t y made i t quickly c l e a r that differences i n language and culture posed p o t e n t i a l l y c r i t i c a l v a l i d i t y issues. This i s p a r t i c u l a r l y important for other p r a c t i t i o n e r s i n academic exchanges or others involved i n education i n cross-c u l t u r a l contexts who may not be conversant with v a l i d i t y theory. A case i n point was a recent study by Westrick (2004) who used the IDI i n a quantitative analysis of the e f f e c t of service learning on the i n t e r c u l t u r a l s e n s i t i v i t y of high school students at an international school i n Hong Kong. Westrick's (2004) rationale for using the IDI was that: Psychometric analysis shows that 'the IDI i s a highly r e l i a b l e measure which has l i t t l e or no s o c i a l d e s i r a b i l i t y bias and also reasonably approximates the developmental model of i n t e r c u l t u r a l s e n s i t i v i t y (Bennett, 1986, 1993) upon which i t i s based' (Paige et a l . , 1999). Now that there is a reliable instrument to measure these theoretical stages (Hammer et a l . , 2002) there i s a po t e n t i a l to gain insights into international school students and the di f f e r e n t programs and strategies that can increase t h e i r i n t e r c u l t u r a l s e n s i t i v i t y ( i t a l i c s added) (p. 282). This rationale, which represented my own s t a r t i n g point i n embarking on t h i s research, now rings serious,alarm b e l l s for me. F i r s t , i t mentioned only the r e l i a b i l i t y of the instrument, and not i t s v a l i d i t y . I t accepted without question that the instrument would be suitable for use i n the context of high school students, from a vari e t y of c u l t u r a l backgrounds, where "over a t h i r d , 38.7 percent of the students i n the sample claim n a t i o n a l i t i e s i n an Asian country" (Westrick, 2004, p. 286). In her conclusion, Westrick (2004) went on to state that, "(A)s a s t a t i s t i c a l l y valid and r e l i a b l e instrument, the IDI has been shown to be a valuable t o o l for evaluating students' l e v e l of i n t e r c u l t u r a l s e n s i t i v i t y " ( i t a l i c s added) (p. 296). However, she made no mention of having considered p o t e n t i a l v a l i d i t y issues i n any way. This was disturbing f o r two reasons. The f i r s t was the u n c r i t i c a l perpetuation of the notion that an instrument can be validated. The second involved an aspect of v a l i d i t y theory that l i e s beyond what have t r a d i t i o n a l l y been regarded as the elements of construct v a l i d i t y . Messick (1998) c a l l s i t consequential v a l i d i t y , "the r e l a t i o n between the ev i d e n t i a l and consequential bases of v a l i d i t y " (p. 38) (about which more w i l l be said below). The DMIS describes the process of increasing one's i n t e r c u l t u r a l s e n s i t i v i t y as a c o n s t r u c t i v i s t one. Westrick (2004) summarised the requirements as follows: F i n a l l y , Bennett a r t i c u l a t e s three assumptions about the development of i n t e r c u l t u r a l s e n s i t i v i t y , a l l of which have p a r t i c u l a r relevance to the experience of students involved i n service-learning. 1. The phenomenology of difference i s the key to i n t e r c u l t u r a l s e n s i t i v i t y . 2. The construing of difference necessary for i n t e r c u l t u r a l s e n s i t i v i t y i s that of ethnorelativism, whereby different cultures are perceived as variable and viable constructions of reality. 3. Ethical choices can and . must be made for i n t e r c u l t u r a l s e n s i t i v i t y to develop. However, these choices cannot be based on either absolute or universal p r i n c i p l e s . Rather, ethical behavior must be chosen with awareness that different viable actions are possible. (Bennett, 1993:6) ( i t a l i c s added) (p. 281). The l e v e l of r e f l e c t i o n and e t h i c a l development that the process requires i s extremely high. I t i s not inconceivable that the high school students involved i n t h i s study, some as young as grade nine, might have achieved the r e q u i s i t e l e v e l s of moral and cognitive development to consciously construe difference, to self-consciously r e f l e c t on t h e i r experiences, and to make e t h i c a l choices, but there was no evidence i n the a r t i c l e to suggest that these considerations entered into the choice of the IDI as an appropriate instrument for t h i s study, or the DMIS as an appropriate t h e o r e t i c a l framework. The issue of consequential v a l i d i t y i s important here i n two ways. One way to consider i t i s i n terms of the IDI's authors' assurances that t h e i r instrument i s r e l i a b l e and v a l i d . These assurances, from two very respected figures i n the f i e l d of c r o s s - c u l t u r a l research, are what prompted me (and Westrick, I suspect, since a researcher can use the IDI only a f t e r attending a qu a l i f y i n g workshop given by the authors) to reach for the IDI as the empirical sol u t i o n to my research problem. The second way to look at consequential v a l i d i t y i s i n terms of the user of the instrument. Although the decision to use the IDI may have originated i n the misconception that the 'instrument i s v a l i d ' , Hambleton and Patsula (1998) remind us that, (A) researcher using an adapted te s t s t i l l has the r e s p o n s i b i l i t y of producing evidence of v a l i d i t y i n the context where that adapted t e s t i s used" because "researchers r i s k imposing conclusions based on concepts which e x i s t i n t h e i r own cultures but which are foreign, or at least p a r t i a l l y i n c o r r e c t , when used i n another culture (p. 156). Hambleton and Patsula r e f e r to an adapted t e s t , but i n the case at hand, while i t was the subjects who were 'adapted' ( i . e . , d i f f e r e n t from the population upon which the o r i g i n a l data inferences were validated), the same p r i n c i p l e applies. I am not t r y i n g to imply that Westrick's 2004 study i s necessarily flawed, but i t i s deeply disturbing that v a l i d i t y issues were not even considered, neither i n the t r a d i t i o n a l , p r a c t i c a l sense (as a property of the inferences drawn from the data), nor i n terms of consequential v a l i d i t y ; the appropriateness of using the framework and the instrument i n the p a r t i c u l a r context. Most p r a c t i t i o n e r s do not question v a l i d i t y claims, and would be stupefied to discover they needed to address v a l i d i t y issues every time they wanted to use an instrument i n a novel context. Yet t h i s i s p r e c i s e l y what Hambleton and Patsula, and other experts i n v a l i d i t y (see Messick, 1995, 1998; Hubley & Zumbo, 1996), are c a l l i n g f o r . The only v a l i d a t i o n studies to date (Bennett & Hammer, 1998; Paige et a l . , 2003) have used data and inferences from respondents i n the United States, using the o r i g i n a l English version. This would not be a shortcoming i n and of i t s e l f , were i t not for the issue of consequential v a l i d i t y and pr a c t i t i o n e r s ' and researchers' propensity to believe that an instrument can be validated i f rigorous s t a t i s t i c a l protocols are followed. Messick (1995) l a i d out the v a l i d i t y issues that shaped\' t h i s research as follows: V a l i d i t y i s not a property of a te s t or assessment as such, but rather of the meaning of the te s t scores. These scores are a function not only of the items or stimulus conditions, but also of the persons responding as well as the context of the assessment. In p a r t i c u l a r , what needs to be v a l i d i s the meaning or i n t e r p r e t a t i o n of the scores; as well as any implications f or action that t h i s meaning e n t a i l s (Cronbach, 1971). The extent to which scores' meaning and action implications hold across persons or population groups and across settings or contexts i s a persistent and perennial empirical question. This i s the main reason that v a l i d i t y i s ah evolving property and v a l i d a t i o n a continuing process (p. 741). The r e s u l t s of t h i s study confirmed that v a l i d i t y i s i n part dependent on the persons responding to the instrument, that conclusions based on concepts from one culture may be at least p a r t i a l l y incorrect when used i n another culture, and that v a l i d a t i o n i s an ongoing process. The inferences that one must draw from analyses of data from the Japanese-language version of the IDI administered to Japanese subjects d i f f e r s i g n i f i c a n t l y from those drawn with the o r i g i n a l IDI and English-speaking subjects i n the United States. The discussion surrounding information from native-Japanese-speaking informants, i n the t r a n s l a t i o n process and elsewhere, w i l l highlight that. The ongoing process of v a l i d a t i n g the IDI i n t r a n s l a t i o n was also s i g n i f i c a n t because although the IDI i s positioned as a instrument for measuring i n t e r c u l t u r a l s e n s i t i v i t y i t , as an extension of the DMIS, can more broadly assess orientations towards difference of a l l kinds. However one defines the culture with which he or she i d e n t i f i e s , including sexual orientation, l i n g u i s t i c group, ethnic group, or age group, to name a few, the IDI p o t e n t i a l l y o f f e r s a window into a person's worldview v i s - a - v i s those d i f f e r e n t from him or herself. Thomas and Inksoh (2004) described culture i n t h e i r book on C u l t u r a l Intelligence i n the broadest terms. "'Culture' i n t h i s case i s not confined to national or ethnic culture, but, consistent with our d e f i n i t i o n i n t h i s book, can be any s o c i a l group, and subcultures i n a society . . . " (p. 73). The IDI's p o t e n t i a l value i n helping us to understand orientations toward and (in)tolerance of difference make i t worthwhile to rigorously examine i t s c r o s s - c u l t u r a l t r a n s f e r a b i l i t y and to work towards modifying i t , as necessary, to make i t work seamlessly across cultures. The DMIS i s not a l l that h e l p f u l i n explaining how one's worldview s h i f t s from one stage to the next, except to say that construing and reconstruing one's experience through c r i t i c a l r e f l e c t i o n t r i g g e r change. This does not, however, diminish i t s value i n providing a framework fo r describing the progression from an ethnocentric to ethnorelative worldview. The work of researchers such as Matsumoto et a l . (2003), Chen (1997) and Chen and Starosta (2000) may help us to understand the substructure or constituent components of i n t e r c u l t u r a l understanding. 4.2 The Vali d a t i o n Process According to Messick (1995) v a l i d i t y has t r a d i t i o n a l l y been seen to consist of three components: content, c r i t e r i o n -related, and construct. Angoff (1988, c i t e d i n Hubley & Zumbo, 1996), reported that although currently accepted thinking on v a l i d i t y i s that content, c r i t e r i o n - r e l a t e d , and construct v a l i d i t y are "three aspects of a unitary psychometric d i v i n i t y " (p. 211) Hubley and Zumbo (1996) commented on common practice i n the f i e l d as follows: An examination of studies reporting on the v a l i d i t y measure or observation indicates the presence of es s e n t i a l l y three assumptions re l a t e d to the t r a d i t i o n a l view of v a l i d i t y : (a) v a l i d i t y i s a property of the observation or measure ( i t a l i c s i n o r i g i n a l ) , (b) there are various types of v a l i d i t y , and (c) r e l i a b i l i t y and v a l i d i t y are presented as d i s t i n c t concepts with d i f f e r e n t purposes (p. 209). The v a l i d a t i o n work conducted by Bennett and Hammer (1998) and l a t e r by Paige et a l . (2003) were products of the three assumptions c i t e d above, with p a r t i c u l a r emphasis on the f i r s t two; that v a l i d i t y i s a property of the instrument and that there are various types of v a l i d i t y to be tested. The following discussion, while not accepting eit h e r the property-of-the-instrument, nor the discrete-elements assumptions about v a l i d i t y , was set out within the framework of the o r i g i n a l v a l i d a t i o n work. 4.3 Content V a l i d i t y Content v a l i d i t y i n the o r i g i n a l IDI was established following the guidelines for te s t construction set out i n DeVellis (1991, as c i t e d i n Hammer, Bennett, and Wiseman, 2003). I t was, to a l l appearances, a.thorough and rigorous process. While I am not c r i t i c i s i n g the procedures followed, that very rigour, set i n a context of binary decisions regarding v a l i d i t y ( i . e . , that an instrument i s or i s not valid) might have encouraged c e r t a i n conclusions to be drawn that appear to have been unwarranted. Ignoring the v a l i d i t y issues inherent i n language and culture adaptation, the IDI's authors c l e a r l y intended the instrument to be i n t e r c u l t u r a l l y transferable. They (Hammer, Bennett, & Wiseman, 2003): were i n i t i a l l y concerned that the empirical observations upon which the DMIS was based could be re-created i n systematic ways. This concern was addressed by examining discourse of people from a vari e t y of cultures i n order to determine i f observers could r e l i a b l y categorize the discourse i n ways i d e n t i f i e d i n the DMIS t h e o r e t i c a l framework (p. 7). Care was taken to have a broad c u l t u r a l and e x p e r i e n t i a l base. Hammer et a l . (2003) report that: While the p i l o t interviews were conducted with i n d i v i d u a l students from a v a r i e t y of cultures, i t was decided that the actual sample of interviewees would consist of people of varied c u l t u r a l backgrounds who also extended beyond the u n i v e r s i t y community. Therefore, the interview sample was selected from residents from such places as the International House i n Washington D.C. (where professionals from many d i f f e r e n t countries reside) as well as various places of employment i n and around the Washington D.C. area (p. 7). Forty people as described above were interviewed regarding t h e i r experience with, and attitudes toward, other cultures. Of the forty, there were twelve U.S. Americans of European o r i g i n , three more U.S. Americans of South-Asian o r i g i n (bringing the number of U.S. Americans to f i f t e e n of the f o r t y ) . There were also three people from each of B r i t a i n , Japan and France, two from each of Switzerland, Korea, Ireland, and Russia, and one person from each of China, Denmark, Spain, France, Germany, Estonia, India, Turkey, Ecuador, Guyana, and Ivory Coast. There were a number of ways to i n t e r p r e t these data besides the conclusion drawn by Hammer et a l . , that they had a s u f f i c i e n t l y c u l t u r a l l y - d i v e r s e mix to assure the cross-c u l t u r a l robustness of the IDI. Most importantly i n my view, a l l f o r t y subjects spoke English s u f f i c i e n t l y well to p a r t i c i p a t e i n a lengthy interview of some conceptual sophistication; a discussion of t h e i r experience of c u l t u r a l difference. Had the non-native speakers been interviewed i n t h e i r native languages and quotes from the t r a n s l a t i o n s of those t r a n s c r i p t s been used i n the analysis, a d i f f e r e n t set of items might well have emerged, i f not a DMIS of a d i f f e r e n t texture. This conclusion was not merely speculative. Yamamoto (1998) used the IDI interview protocol to interview Japanese univer s i t y students studying in,the United States, i n Japanese. F i r s t , she analysed the interview data against the DMIS stages. Then, she c l a s s i f i e d the data into the seven categories that she found emerging naturally from them: "Attention to Physical Difference; Physical Admiration for Caucasians; Attention to Physical S i m i l a r i t y ; Attention to Own Frame of Reference; Naturalness of Difference; I n e v i t a b i l i t y of Difference; and Suspension of Judgement" (p. 77). She states that "(T)hese emergent categories are c l o s e l y related to Japanese c u l t u r a l values and perceptions of r e a l i t y " (p. 77). Going on to compare the DMIS to her emergent categories, she found that IDI: statements such as 'I appreciate (enjoy or respect) c u l t u r a l differences' were hardly expressed by the Japanese students. . . Also, the students paid much attention to the differences and s i m i l a r i t i e s i n physical appearance, which they associated with the degree of discomfort or comfort (p. 77). Yamamoto concluded that: These r e s u l t s suggest that the d e f i n i t i o n s of each stage may need some modification i n order to understand i n t e r c u l t u r a l s e n s i t i v i t y i n the Japanese context. I t might be possible to say that what Japanese perceive as d i f f e r e n c e s / s i m i l a r i t i e s or how they deal with d i f f e r e n c e / s i m i l a r i t i e s are d i f f e r e n t from or not included i n the stages of the model. These aspects need to be considered and added to the model i n order to modify i t to apply i n the Japanese context (pp. 77-78). I was also reminded here that when I was going over my RA's responses to both the English and Japanese versions of the IDI, her responses d i f f e r e d i n places where items that were problematic i n Japanese were conceptually available to her i n English. Additionally, one of my t r a n s l a t o r s , commenting on a p a r t i c u l a r item and concepts d i f f e r e n t i a l l y available i n English and Japanese said, "Another mutsukashii ( d i f f i c u l t ) sentence, r e f e r r i n g to culture i n terms of superior and i n f e r i o r , although i t makes sense i n English" (personal communication). Returning to the subject pool, another way to look at i t i s to note that i t s members are overwhelmingly from Judeo-Ch r i s t i a n backgrounds, assuming that the twenty-eight subjects of European descent were members of the mainstream. S o c i a l i s a t i o n into a p a r t i c u l a r r e l i g i o - c u l t u r a l worldview has obvious implications i n the context of t o l e r a t i n g , appreciating, or enjoying difference. For the next step i n the instrument-building process, c u l l i n g utterances to include i n the IDI, four members of the research team each reviewed twenty-five randomly selected tr a n s c r i p t s from among the o r i g i n a l f o r t y . Hammer et a l . (2003) reported that they "rated the DMIS orientations the interviewees' (sic) most consistently expressed during the interview" (p. 8). These resulted i n an item pool of 239 IDI sample items for further p i l o t t e s t i n g . Although adequate i n t e r - r a t e r r e l i a b i l i t i e s were obtained i n t h i s process, ranging from .66 ( f a i r ) to .86 (excellent), we do not know how many utterances survived from non-U.S. Americans, for example, or non-native speakers of English. I t could very well be that the utterances that consistently r e f l e c t e d the DMIS orientations during the interview came from p a r t i c u l a r c u l t u r a l or l i n g u i s t i c subsets of subjects; a threat to v a l i d i t y that Messick (1995) c a l l s "construct underrepresentation" (p. 742). Another p o t e n t i a l shortcoming of the content-validation process was that although Hammer et a l . (2003) t e l l us that the pool of 239 quotes r e s u l t i n g from the process outlined above was twice tested i n a p i l o t version of the IDI, "with a c u l t u r a l l y diverse group of people" (p. 9) the p i l o t t e s t i n g was looking for " c l a r i t y of ins t r u c t i o n s , item c l a r i t y , response option a p p l i c a b i l i t y , and o v e r a l l amount of time taken to complete the instrument" (p. 9), but not conceptual t r a n s f e r a b i l i t y . The surviving items (assuming that some of the items f a i l e d the tes t of item c l a r i t y and response-option a p p l i c a b i l i t y ) were then given to another group of DMIS experts who further d i s t i l l e d them into the s i x t y items that comprise version one of the IDI. This methodology was not without redeeming features. I t had the twin merits of employing items that were not generated by experts s i t t i n g down to write what seemed to be reasonable items f o r f i e l d t e s t i n g , and a s t a t i s t i c a l l y - r i g o r o u s process involving agreement by experts on the strength of the r e l a t i o n s h i p of the items to the* DMIS. However, i t lacked rigour i n i t s f a i l u r e to confirm the cr o s s - c u l t u r a l robustness of the items... While the authors claim that the use of a c u l t u r a l l y -mixed pool of interviewees demonstrates pan-cultural content v a l i d i t y , i n addition to simple content v a l i d i t y , I believe that the arguments raised i n t h i s discussion were s u f f i c i e n t to bring that into question. Further research with non-Indo-European language versions and with subjects from other than Judeo-Christian backgrounds i s c e r t a i n l y warranted before the pan-cultural a p p l i c a b i l i t y of the IDI can be asserted with any confidence. 4.4 C r i t e r i o n - r e l a t e d V a l i d i t y In the o r i g i n a l v a l i d a t i o n work, c r i t e r i o n - r e l a t e d v a l i d i t y was tested by comparing the re s u l t s of the IDI with scores obtained on two other scales, the Worldmihdedness Scale and the I n t e r c u l t u r a l Anxiety Scale. A t h i r d scale was used to assess s o c i a l d e s i r a b i l i t y e f f e c t s (Bennett & Hammer, 1998). Bennett and Hammer (1998) report that "(T)he Worldmindedness Scale (Sampson & Smith, 1957) i s a measure used to assess " i n t e r n a t i o n a l i s t ^ " attitudes" (p. 59). A six-item version of the scale was used for the o r i g i n a l v a l i d a t i o n . Results obtained (see Table 4.1, reproduced from Bennett & Hammer, 1998, p. 81) showed that higher l e v e l s of Worldmindedness correlated negatively with ethnocentrism and p o s i t i v e l y with ethnorelativism. Half of the correlations (.40 and above) are considered strong i n a s t r i c t l y s t a t i s t i c a l sense (although they account only for between sixteen and twenty-four percent of the variance) the other half are quite weak and there i s no information provided on how well the Worldmindedness Scale 'travels' i n t e r c u l t u r a l l y . This might not have been s i g n i f i c a n t for the o r i g i n a l IDI with English-speaking subjects. However, any claim of c r o s s - c u l t u r a l t r a n s f e r a b i l i t y cannot be legitimately made without empirical evidence that there i s no language or culture e f f e c t on the instrument's performance. Since the criterion-referenced evidence i s co r r e l a t i o n a l that must include instruments that are introduced to e s t a b l i s h c r i t e r i o n - r e l a t e d v a l i d i t y through cor r e l a t i o n s . A modified version of Stephan and Stephan's (1985) So c i a l Anxiety Scale, focussing "on the degree of anxiety respondents experience when int e r a c t i n g with people from cultures other. than t h e i r own" (Bennett & Hammer, 1998, p. 81) was used as a second measure. Bennett and Hammer (1998) report that: A number of studies have found t h i s measure to maintain s a t i s f a c t o r y r e l i a b i l i t i e s across c u l t u r a l contexts (e.g., Gao & Gudykunst, 1990; Gudykunst, 1989; Hammer, Wiseman, Rasmussen, & Bruschke, 1998) (p. 81). However, as Table 4.1 shows, the corr e l a t i o n s between the Int e r c u l t u r a l Anxiety Scale and the IDI are not p a r t i c u l a r l y strong, even i n a purely s t a t i s t i c a l sense, and the rel a t i o n s h i p between the Minimization Scale and the Int e r c u l t u r a l Anxiety Scale i s obscure. Table 4.1 Correlations between IDI and Worldmindedness and Int e r c u l t u r a l Anxiety Scales IDI Scale Worldmindedness Scale I n t e r c u l t u r a l Anxiety Scale Denial -.40, p=.001 .28; p=.001 Defense -.18; p=.007 .13; p=.05 Minimization -.18; p=.007 -.11; p=.08 Acceptance .47; p=.001 -.17; p=.01 Cognitive Adaptation .47; p=.001 -.21; p=.002 Behavioral Adaptation .36; p=.001 -.15; p=.02 (reproduced from Bennett & Hammer, 1998, p. 81) Bennett and Hammer (1998) concluded that the r e s u l t s obtained from the Worldmindedness and I n t e r c u l t u r a l Anxiety scales " o f f e r compelling evidence of the construct v a l i d i t y of the IDI" (p. 82). I would beg to d i f f e r . As noted above, the c r i t e r i o n - r e l a t e d v a l i d i t y evidence produced for the o r i g i n a l IDI was not convincing, except to confirm a general pattern. Because I didn't have the resources to duplicate the process of t r a n s l a t i n g the other two scales as I had with the IDI, I was o r i g i n a l l y prepared (when I embarked on t h i s project) to accept the c r i t e r i o n - r e l a t e d v a l i d i t y evidence from Bennett and Hammer's (1998) o r i g i n a l work as s u f f i c i e n t to make an argument for the c r i t e r i o n - r e l a t e d component of the v a l i d i t y evidence. I t i s the only evidence available and i t c e r t a i n l y seemed to s a t i s f y the authors. Fortunately, the work I've done i n examining conceptions of v a l i d a t i o n i n t h i s study has precluded me from taking the easy way out. The wishy-jwashiness of the resu l t s i s compounded, for my purposes, by the lack of evidence that the Worldmindedness Scale has any cro s s - c u l t u r a l a p p l i c a b i l i t y . One could be excused for saying that i t ' s a seemingly impossible task to obtain convincing c r i t e r i o n - r e l a t e d v a l i d i t y evidence involving a number of instruments that have been shown to y i e l d consistent r e s u l t s across cultures, but that i s exactly the l e v e l of evidence that must be adduced before claims of c r o s s - c u l t u r a l t r a n s f e r a b i l i t y can be made. 4.5 So c i a l D e s i r a b i l i t y Bennett and Hammer's (1998) report on the v a l i d a t i o n process mentioned the addition of a scale to te s t whether s o c i a l d e s i r a b i l i t y was a factor i n IDI responses, but they did not report the resu l t s for s o c i a l d e s i r a b i l i t y . In IDI workshops and other fora, however, Bennett and Hammer state that s o c i a l d e s i r a b i l i t y i s indeed not a factor. They did present s t a t i s t i c a l evidence against any e f f e c t s from gender and socio-economic status, as measured by l e v e l of education, i n t h e i r v a l i d a t i o n r e s u l t s (Bennett & Hammer, 1998). The influence of gender i n Japanese s o c i a l roles has been widely noted. Hofstede (1980) described Japanese culture as being the most 'masculine' i n his study; the culture i n which gender roles and behaviour were most c l e a r l y and r i g i d l y d i f f e r e n t i a t e d . As a consequence, I could not assume that the res u l t s of analysis for gender e f f e c t s i n the o r i g i n a l v a l i d a t i o n study would necessarily hold for a Japanese sample so I undertook an analysis from my own data, based on mean IDI scores. There were an i n s u f f i c i e n t number of cases ( p a r t i c u l a r l y with an n of 106 for male respondents) to redo a p r i n c i p a l components analysis on the basis of gender. The analysis undertaken showed no gender e f f e c t (see Table 4.2). Heine and Lehman (1995) have done some work on the s o c i a l d e s i r a b i l i t y aspect of the v a l i d a t i o n procedure. They compared Canadian and Japanese students' responses on the Balanced Inventory of Desirable Responding (Paulhus, 1991), using a Japanese t r a n s l a t i o n for the Japanese subjects, and concluded, te n t a t i v e l y , that: In the context of an anonymous questionnaire, then, there was no evidence that the responses of the Japanese were more s o c i a l l y desirable than those of the Canadians. A tentative conclusion i s that comparisons of the responses of Japanese and North Americans— :at least for students—on anonymous questionnaires are not confounded by s o c i a l l y desirable response sets (p. 779). Table 4.2 Analysis for Gender E f f e c t by Mean Score Between-Subjects Factors N GENDER M 106 F 287 Tests of Between-Subjects Effects Dependent Variable: MEAN Source Type III Sum of Squares df Mean Square F Sig. Corrected Model 7.181E-03 3 1 7.181E-03 .010 .919 Intercept 8781.229 1 8781.229 12602.069 .000 GENDER 7.181E-03 1 7.181E-03 .010 .919 Error 272.452 391 .697 Total 11408.500 393 Corrected Total 272.459 392 a R Squared = .00 0 (Adjusted R Squared = -.003) This seemed to minimise concerns about possible confounding e f f e c t s from s o c i a l d e s i r a b i l i t y responses, but Heine and Lehman (1995) also c i t e d studies that concluded that: C u l t u r a l differences have been observed with respect to a moderacy response bias; answers given by Japanese respondents to a questionnaire tended to be closer than those of U.S. respondents to the midpoint on Likert-type scales (Stening & Everett, 1984; Zax & Takahashi, 1967) (p. 777) . This was s i g n i f i c a n t for two reasons. The f i r s t was that i t again reinforced the need to empirically v e r i f y the a p p l i c a b i l i t y of the c r i t e r i o n - r e l a t e d v a l i d i t y measures used i n the o r i g i n a l process (both of which employed L i k e r t scales). The second, and more important reason, was that the midpoint responses to the IDI are c r i t i c a l to in t e r p r e t i n g the data. After reverse coding of negatively worded items, each item i n the IDI has an id e a l score i n terms of the p e r f e c t l y i n t e r c u l t u r a l l y - s e n s i t i v e person of one, strongly disagree, for ethnocentric items and seven, strongly agree, f o r ethnorelative items. In analysing IDI data, scores of one or two, six or seven are non-problematic as they represent c l e a r l y - h e l d convictions. Scores i n what Hammer c a l l s 'the bucket', i n the three to f i v e range, i d e n t i f y where the respondent i s 'having issues'. Responses i n d i c a t i n g uncertainty about an item are key indicators about where the respondent i s working through ambivalence i n the corresponding DMIS stage. If Japanese respondents are act u a l l y more l i k e l y , as a c u l t u r a l artefact, to choose midpoint items on Li k e r t - s c a l e d instruments, that c u l t u r a l tendency could compromise the interpretations made from the data and represent a cl e a r threat to v a l i d i t y . This i s another area i n which further research i s c l e a r l y c a l l e d f o r . 4.6 Construct V a l i d i t y The p r i n c i p a l components analysis yielded important insights both into the IDI and also into the v a l i d a t i o n process i t s e l f . The number of subjects i n t h i s study was larger than the group used by Paige et a l . (2003) i n t h e i r analysis (n=330) and also larger than the o r i g i n a l IDI va l i d a t i o n study (n=226). I t was noteworthy that seventeen of the s i x t y items, 28.33% of the instrument, d i d not map onto t h e i r predicted IDI stages. That indicated a need for a deeper analysis of those items, but of a c u l t u r a l l y and q u a l i t a t i v e l y d i f f e r e n t kind than that which comprised the v a l i d a t i o n process for the o r i g i n a l or the r e p l i c a t i o n by Paige et a l . (2003). In addition to items that did not map onto the expected factor, there were a number of items that loaded with some strength on a number of factors across the matrix, i n d i c a t i n g that t h e i r r e l a t i o n s h i p with the underlying DMIS stages was ambiguous, further clouding the issue of the Japanese IDI's construct v a l i d i t y . Among the items that loaded above the threshold (minimum of .30) on more than one factor, were (see Table 3.3 — Rotated Factor Matrix) item 44 which loaded at .377 on Factor 4 (Denial) i n addition to i t s main (or strongest) loading of .514 on the Defence'factor. Item 27 loaded at .343 on Factor 1 (Adaptation) i n addition to i t s main loading of .470 on the Acceptance factor. Item 34, an orphaned Minimization item i n the middle of the Acceptance factor (with a main loading there of -.460) had a secondary loading of nearly equal strength of .419 on the Minimization factor. There were also a number of items that loaded on other factors with some strength, although the loadings were below the .30 threshold. Item 37, a Denial item marooned i n the Adaptation factor loaded secondarily on Denial, but only at .286. Item 58, a weak Adaptation item (loading of .331) loaded almost as strongly (.254) on Factor 3 (Acceptance). Item 30 loaded (somewhat weakly) on i t s 'home' Denial factor at .364, but had two other loadings of nearly the same strength (.298) on Factor 2 (Defence) and on Factor 3 (.266), the Acceptance factor. These ambiguous items bring the t o t a l to 40% of the IDI items i n the Japanese t r a n s l a t i o n that do not perform as expected v i s - a - v i s r e s u l t s from the analyses on the English o r i g i n a l . While the f a c t that some of the items d i d not p e r f e c t l y map onto t h e i r intended IDI stages was not a novel finding, i t was i n t e r e s t i n g to follow how the lack of f i t has been variously interpreted. Consider the Minimization stage of the IDI as example. In the Japanese t r a n s l a t i o n , the three Transcendent Universalism items clustered as a factor unto themselves (see Results). They had been flagged during the t r a n s l a t i o n process as being p o t e n t i a l l y problematic on a conceptual l e v e l and that prediction was borne out i n the p r i n c i p a l components analysis. We can trace s i m i l a r results through the o r i g i n a l v a l i d a t i o n process (Bennett & Hammer, 1998). Bennett and Hammer's (1998) o r i g i n a l factor analysis: resulted i n a ten-item scale with reasonably high corrected item-total c o r r e l a t i o n and a r e l i a b i l i t y estimate of .87 (n=212). These items r e f l e c t a mixture of both "Physical Universalism" and "Transcendent Universalism" within the Minimization stage description of the DMIS and therefore were l a b e l l e d with MINIMIZATION (p. 67). So, i n the o r i g i n a l analysis of the factor structure, the items i n Minimization were deemed to be s u f f i c i e n t l y correlated and coherent to form a single dimension with two subscales. When Paige et a l . (2003) performed t h e i r analysis, t h e i r conclusion was also that while: . . . the analyses of the i n t e r n a l structure of the IDI have shown i t to be a reasonable approximation of the t h e o r e t i c a l model of i n t e r c u l t u r a l development . . . Minimization items s p l i t along the t h e o r e t i c a l l i n e s of physical and transcendental universalism (Factors 3 and 4) (p. 483). They went on to say that, "(T)he Minimization s p l i t i s i n t e r e s t i n g because i t follows exactly the form structure of the DMIS" (p. 484) and speculated that t h e i r subjects had responded to the Transcendence items the way they had because " i t i s very l i k e l y that many individuals of the age groups represented i n our sample have not given any serious thought . to t h e i r p o s i t i o n on the issues of s p i r i t u a l i t y and r e l i g i o u s b e l i e f s " (p. 484). In reporting on the Paige study, however, Hammer, Bennett, and Wiseman (2003) interpreted the findings to mean that: the factors i d e n t i f i e d i n the 60-item IDI might hot be as stable as desired. Further, t h e i r findings suggested the p o s s i b i l i t y of three more fundamental dimensions: a factor composed of Denial and Defense items, a Minimization factor, and a factor that lar g e l y consists of Acceptance and Adaptation (pp. 11-12). This i s somewhat surprising given that Paige et a l . (2003) reported that, " ( O ) v e r a l l , the factor analyses provide strong empirical support for the broader two-factor (ethnocentric and ethnorelative) structure of the development model" (p. 483), but a l l r e s u l t s are open to i n t e r p r e t a t i o n . On the surface, the r e s u l t s from the p r i n c i p a l components analysis of the Japanese IDI and those obtained by Bennett and Hammer (1998) and Paige et a l . (2003) were i n agreement. A l l three showed a s p l i t structure for Minimization with Transcendent Universalism c l u s t e r i n g separately from Physical Universalism. However, my Japanese informants (the t r a n s l a t o r s , my RA, and the raters who helped me to refin e the items), unanimously noted that the Transcendent Universalism items were conceptually nonsensical i n Japanese culture, despite the fact that the words could be translated. As one of the translators put i t i n private e-mail correspondence "I don't understand t h i s whole sentence . . . M [another t r a n s l a t o r ] avoided t r a n s l a t i n g a l l 'the children of a s p i r i t u a l being' [items]. I don't know what that i s either. It's possible to put Japanese words together as a tr a n s l a t i o n without understanding what that means, demo hen" [but i t ' s weird — author's t r a n s l a t i o n ] . Yamamoto and Tanno (2002) c i t e d a s i m i l a r d i f f i c u l t y . They said, ( i n t r a n s l a t i o n from the o r i g i n a l Japanese by t h i s author) that: I made an e f f o r t to be as f a i t h f u l to the o r i g i n a l wording of the IDI as possible, but some expressions came out as d i f f i c u l t to understand i n t r a n s l a t i o n . However, i t wasn't a simple t r a n s l a t i o n problem. I t i s possible that some of the items were d i f f i c u l t to understand c u l t u r a l l y . For,example, i n the Universal Values subscale of the Minimization stage items 12, 23 and 59 are phrased as 'At our root i s a supernatural holy being (choshizende shinseina sonzai), but the English o r i g i n a l f o r 'supernatural holy being' was ' s p i r i t u a l being'. I thought about t r a n s l a t i n g i t [in a s y l l a b i c katakana rendition] as 'supirichuaru bi'ingu', but tr y i n g to be f a i t h f u l to the o r i g i n a l , I checked with Bennett and as a r e s u l t I used 'supernatural holy being'. However, some doubt remains as to whether the value held by those who believe we are united under a supernatural being is' an appropriate expression of what the Minimization stage of i n t e r c u l t u r a l s e n s i t i v i t y means to Japanese (p. 40). Although the most glaring examples were the items that comprised the Transcendent Universalism subscale of Minimization, c l e a r l y products of a Judeo-Christian mainstream culture, t h i s was not a quibble over three items out of s i x t y . I t was fundamental to the way the instrument was constructed; out of quotes from actual interviews, not fabricated items. Bennett and Hammer (1998) and Paige et a l . (2003) looked at the data through t h e i r own c u l t u r a l and l i n g u i s t i c lenses and concluded that they were an accurate r e f l e c t i o n of the DMIS. If that conclusion had been lim i t e d to the English version of the IDI, administered to h i g h l y - p r o f i c i e n t speakers of English, there would be no problem. However, I believe Bennett and Hammer were being premature at best i n pronouncing the IDI to be v a l i d , i n any language version and with any c u l t u r a l group, because they had used a c u l t u r a l l y and l i n g u i s t i c a l l y diverse pool of subjects i n the item-building phase. From the resu l t s I obtained and the conclusions drawn by Yamamoto (1998) i t i s cle a r that had the o r i g i n a l interviews been conducted i n Japanese, for example, the items appearing on the IDI would have been at le a s t somewhat, perhaps su b s t a n t i a l l y , d i f f e r e n t . Other issues came to l i g h t during consultations on the tr a n s l a t i o n . One recurring issue was the IDI's references to 'being a member' of one's own culture as juxtaposed with people from other cultures. Although some of the phrases sound (to my ear) a b i t laboured even i n English, e.g., 'Although I f e e l I am a member of my own culture . . .' they are otherwise unremarkable i n English. To the Japanese, references to one's. Own and other people's cultures have an unnatural clang. As one t r a n s l a t o r put i t , "I don't think Japanese people di s t i n g u i s h people by culture, but by country or n a t i o n a l i t y " (personal correspondence). In the Japanese worldview there are two types of people i n the world, nihonjin (Japanese) and gaijin or gaikokujin ( l i t e r a l l y 'outside' people or 'foreign-country' people), i . e . , non-Japanese. This i s not d i s s i m i l a r , i n my personal experience, to the Jewish view of the world as consisting of Jews and goyim (non-Jews). Finer c u l t u r a l d i s t i n c t i o n s than that are generally not germane. I t i s d i f f i c u l t to say what e f f e c t the s p e c i f i c insistence on the word 'culture' (to which I remained f a i t h f u l i n the Japanese version of the IDI) might have had on respondents. One trans l a t o r had to consciously make ah e f f o r t not to substitute gaikokujin for the phrase 'people from other cultures' and was at times overcome by the unnaturalness of the construction to her Japanese s e n s i b i l i t i e s . I t c e r t a i n l y made the items stand out, compelling the Japanese respondents to e i t h e r r e f l e c t on the r e l a t i o n s h i p i n t h e i r minds between n a t i o n a l i t y and culture or to automatically substitute the more natural gaikokujin i n t h e i r minds for 'people from other cultures'. Another unintended consequence might have been to merely annoy respondents and weaken the 'face v a l i d i t y ' of the instrument. It might be t h i s p r o c l i v i t y for dichotomising the world that led three of the Denial — D i s i n t e r e s t subscale items and one Minimization — S u p e r f i c i a l Differences item to f a l l into factor four, which was primarily an Acceptance (seven out of twelve items) factor, i n my analysis. There are four Acceptance subscales; Value R e l a t i v i t y , and Enjoying Difference, Describing Difference, and Learning Difference. I t seems that for Japanese respondents, there might have been a 'Difference' factor rather than an Acceptance f a c t o r . This notion was reinforced by Yamamoto's (1998) l i s t of dimensions that spontaneously emerged from her interviews i n Japanese with Japanese subjects. In her analysis, three of the seven dimensions, Attention to Physical Difference, Naturalness of Difference, and I n e v i t a b i l i t y of Difference c i t e d difference s p e c i f i c a l l y , with another, Attention to Physical S i m i l a r i t y , invoking difference as i t s opposite. This was p a r t i c u l a r l y contentious for construct v a l i d i t y with t h i s population because Denial and Minimization l i e on the.opposite side of the ethnocentric-ethnorelative divide from Acceptance and so, unlike items from adjacent dimensions crossing factors, are t h e o r e t i c a l l y incompatible within the same factor. Had I r e l i e d e n t i r e l y on formulaic v a l i d a t i o n protocols consisting of i n t e r - r a t e r r e l i a b i l i t y s t a t i s t i c s and the numerical rules of thumb for factor analyses I might have drawn the same conclusions as previous v a l i d a t i o n studies; p a r t i c u l a r l y i f I had been working from the premise that the instrument, rather than the inferences made with i t , could be validated. However, interpreting the r e s u l t s of the p r i n c i p a l components analysis through the lens of information from the t r a n s l a t i o n processes made that impossible. 4.7 The Translation Process My understanding of the data from the Japanese IDI would have been much poorer had i t not been for the t r a n s l a t i o n protocols employed. The contribution this, study makes to the f i e l d of t r a n s l a t i n g questionnaires and instruments i s to suggest two additional steps. The f i r s t was to use the t r a n s l a t i o n outside of the t r a n s l a t i o n framework. The most thoughtful approaches to t r a n s l a t i o n (and adaptation) involve intensive discussions over the accuracy, naturalness, and a c c e p t a b i l i t y of the tra n s l a t i o n s . These, however, take place within the t r a n s l a t i o n process i t s e l f , l i m i t i n g the focus and scope of the discussion. The step that I added was outside of the t r a n s l a t i o n cocoon; using the d r a f t t r a n s l a t i o n as a r a t i n g t o o l for 'amateur' attempts to translate the items (step three of Table 2.1, above). The d i s t i n c t i o n might be subtle, but analysing t r a n s l a t i o n options, word choice, syntax, etc., i s a s e l f - r e f e r e n t i a l exercise. I t exists within the context of the t r a n s l a t i o n . Taking a step outside of the t r a n s l a t i o n process to see how the items worked i n the context of interpretations by the instrument's intended f i n a l users led to d i f f e r e n t insights i n terms of how non-translators might i n t e r p r e t wording or, more importantly, concepts. The second contribution t h i s study made was to introduce the step (step f i v e i n Table 2.1) of having a b i l i n g u a l complete both versions of the instrument consecutively. Ideally, i f the instrument performs equally well i n both cultures and the t r a n s l a t i o n i s 'perfect' the score f o r each item would be i d e n t i c a l . Any discrepant scores would point to a problem eith e r with the instrument or the t r a n s l a t i o n . This procedure uncovered some minor errors i n the t r a n s l a t i o n . More importantly, however, score discrepancies confirmed the existence of conceptual issues, as noted above. CHAPTER FIVE CONCLUSION 5.1 Contributions to the F i e l d This research has contributed to the f i e l d i n a number of ways. The v a l i d i t y issues raised i n t h i s paper should have r e a l resonance for p r a c t i t i o n e r s , and even for academics i n the f i e l d who reach for an instrument because they believe ' i t i s v a l i d ' . I t was the f i r s t study to examine the v a l i d i t y of inferences made with the IDI i n t r a n s l a t i o n , with a c u l t u r a l l y - d i f f e r e n t population. In doing so i t raised strong doubts about the cr o s s - c u l t u r a l t r a n s f e r a b i l i t y of version one of the IDI and raised some questions about the DMIS as a model for understanding worldviews with respect to difference, i n cultures other than U.S. American, and perhaps even within U.S American culture i t s e l f , given i t s breadth and d i v e r s i t y . By introducing a culturally-based alternative analysis to what appeared to be si m i l a r data sets, the study highlighted the importance of current thinking i n v a l i d i t y theory that instruments cannot be validated, only s p e c i f i c inferences made from data. Issues related to consequential v a l i d i t y i n the use of an instrument or a th e o r e t i c a l framework i n contexts other than the ones for which they were s p e c i f i c a l l y developed were also raised. The study has also contributed to the f i e l d of t r a n s l a t i o n by introducing additional steps to enrich t r a n s l a t i o n protocols. 5.2 Limitations of the Study The study was limited primarily by i t s subject pool. A l l of the subjects were Japanese unive r s i t y undergraduates, within a f a i r l y narrow range of age and c r o s s - c u l t u r a l experience. Given the role of c r i t i c a l s e l f - r e f l e c t i o n i n advancing i n t e r c u l t u r a l s e n s i t i v i t y , subjects with a broader l i f e experience might have responded d i f f e r e n t l y to the IDI items. The study was also limited i n a deliberate way by i t s self-imposed l i t e r a l faithfulness to the IDI. This seemed, and seems, necessary i n terms of t e s t i n g the claimed cross-c u l t u r a l t r a n s f e r a b i l i t y of the instrument, but further research with a t r u l y adapted, as opposed to a translated version of the IDI, would speak more c l e a r l y to i t s actual p o t e n t i a l for g e n e r a l i s a b i l i t y . 5.3 Directions for Further Research A number of research projects are suggested by t h i s study. The f i r s t would be to r e p l i c a t e the study with an adapted, rather than a translated version of the IDI, as. suggested above. The adapted version could p r o f i t from an expanded subject pool, encompassing a greater range of age and experience, both l i f e experience and c r o s s - c u l t u r a l experience. Research of t h i s nature on an adaptation of version two of the IDI would be of even greater u t i l i t y . Further research i n the d i r e c t i o n suggested by Yamamoto (1998), looking at what Japanese respondents say i n r e l a t i o n to t h e i r experience of c u l t u r a l difference and comparing that to the DMIS, i s also warranted. If Japanese speakers d i d not reproduce the range of stages suggested by the DMIS, or c l e a r l y produced a stage or stages not presently encompassed by the model, the DMIS would have to be rethought, at least i n terms of i t s p o t e n t i a l to more generally illuminate the human condition. Another, related, area to pursue i s a more systematic examination of whether a 'Difference' dimension a c t u a l l y exists for Japanese respondents and the implications that a dimension encompassing both ethnocentric and ethnorelative elements might hold. 5.4 Implications for Practice Given that t h i s d i s s e r t a t i o n was written from a p r a c t i t i o n e r ' s perspective, I hope that the implications for practice have been s a l i e n t throughout. To make them e x p l i c i t , t h i s study w i l l serve as a caution to other p r a c t i t i o n e r s who might be tempted to reach for a psychometric or other instrument i n the b e l i e f that i t has been validated and i s therefore suitable to the purpose they have i n mind. Pract i t i o n e r s need to be cognizant of t h e i r r e s p o n s i b i l i t y to ensure that whatever instrument they choose, or research design they formulate, s a t i s f i e s the c r i t e r i a f o r construct v a l i d i t y or other standards of appropriateness for t h e i r p a r t i c u l a r research contexts. This i s p a r t i c u l a r l y true for practice because those re s u l t s are l i k e l y to be translated into programmatic action. Those working across languages ( i n the case of translated or adapted instruments) or cultures (even i f using a research instrument i n i t s o r i g i n a l form) face a p a r t i c u l a r l y onerous burden of proof since they are, i n e f f e c t , navigating uncharted waters. 5.5 Afterward This research sprang from an attempt to define and quantify a tangible facet of i n t e r n a t i o n a l i s a t i o n for students to take home with them from an exchange experience. Although i n t e r c u l t u r a l s e n s i t i v i t y remains a worthy goal we appear to be only s l i g h t l y closer to operationalising and quantifying i t . In the process of conducting t h i s research, I have learned a l o t about the notion of v a l i d i t y , i n c r o s s - l i n g u i s t i c and c r o s s - c u l t u r a l contexts i n p a r t i c u l a r , and I have come to understand the DMIS i n a more profound way. I hope and believe that both of these deeper understandings w i l l return to enrich my practice, bringing the exercise f u l l c i r c l e . BIBLIOGRAPHY Adler, P. (1977). Beyond c u l t u r a l i d e n t i t y : Reflections on c u l t u r a l and m u l t i c u l t u r a l man. In Richard B r i s l i n (Ed.), Culture learning: Concepts/ application and research. Honolulu: University of Hawaii•Press. American Psychological Association, A. E. R. A., and National Council on Measurement i n Education (1985). Standards for educational and psychological t e s t i n g . Washington, DC,: American Psychological Association. Behling, 0., & Law, K. S. (2000). Translating questionnaires and other research instruments: Problems and solutions. Thousand Oaks, CA: Sage. Bennett, M. (1986). Towards ethnorelativism: a developmental model of i n t e r c u l t u r a l s e n s i t i v i t y . In Michael Paige (Ed.), Education for the Intercultural Experience (2nd ed.) Yarmouth, ME: I n t e r c u l t u r a l Press. Bennett, M., & Hammer, M. (1998). The Intercultural Development Inventory (IDI) Manual. Portland, Oregon: I n t e r c u l t u r a l Communication I n s t i t u t e . Bhawuk, D. (1998). The role of culture theory i n cross-c u l t u r a l t r a i n i n g . Journal of Cross-cultural Psychology, 29, 630-656. Bhawuk, D., & Triandis, H. (1996). The r o l e of culture theory i n the study of culture and i n t e r c u l t u r a l t r a i n i n g . In D. Landis and R. Bhagat (Eds.), Handbook of Intercultural Training. Newbury Park, CA: Sage. Bochner, S. (1986). Coping with unfamiliar cultures: Adjustment or culture learning? Australian Journal of Psychology, 38 (3), 347-358. Bradford, L., A l l e n , M., & Beisser, K. (1998). An evaluation and meta-analysis of i n t e r c u l t u r a l communication competence research. Milwaukee, WI: University of Wisconsin-Milwaukee, Department of Communication. Retrieved from ERIC Document Reproduction Service. Chen, G-M. (1997). A review of the concept of i n t e r c u l t u r a l s e n s i t i v i t y . Biennial convention of the Pacific and Asian Communication Association. Honolulu, HI. Retrieved from ERIC Document Reproduction Service. Chen, G-M., & Starosta, W. J . (2000). The development and va l i d a t i o n of the i n t e r c u l t u r a l s e n s i t i v i t y scale. Annual meeting of the National Communication Association. Seattle, WA. November 8-12, 2000. Retrieved from ERIC Document Reproduction Service. DeVellis, R. F. (1991). Scale development. Thousand Oaks, CA: Sage. Edwards, J . , & Dawson, D. (1983). My friend David: a sourcebook about Down's Syndrome and a personal story about friendship. Portland, OR: Ednick Communications. Greenholtz, J . (2003). Socratic teachers and Confucian learners: Examining the benefits and p i t f a l l s of a year abroad. Language and Intercultural Communication, 3 (2), 122-130. Greenholtz, J . (2002). Escape from Alcatraz: Breaking away from the ESL isl a n d model. In E. Papatzikou Cochran (Ed.), Mainstreaming (pp. 103-113). Alexandria, V i r g i n i a : TESOL. Greenholtz, J . (1997). Content-based i n s t r u c t i o n i n the UBC-Ritsumeikan Programme. Ritsumeikan Kyouiku Kagaku Kenkyuu. 10, 193-198. Greenholtz, J . , & Morita, Y. (1988). Mai Frendo Deibido. Kyoto, Japan: Dohosha. Gudykunst, W. B., & Hammer, M. R. (1983). Dimensions of i n t e r c u l t u r a l effectiveness: An exploratory study. International Journal of Intercultural Relations, 8, 1-10. Hambleton, R. K., & Patsula, L. (1998). Adapting tests for use i n multiple languages and cultures. Social Indicators Research, 45, 153-171. Hammer, M. R., Bennett, M. J . , & Wiseman, R. (2003). Measuring i n t e r c u l t u r a l s e n s i t i v i t y : The I n t e r c u l t u r a l Development Inventory. In R. M. Paige (Guest Ed.), International Journal of Intercultural Relations, 27 (4), 421—443. Heine, S. J . , & Lehman, D. R. (1995). S o c i a l d e s i r a b i l i t y among Canadian and Japanese students. Journal of Social Psychology, 135 (6), 111-119. Hofstede, G. (1980). Culture's consequences: International differences in work-related values. Newbury Park, CA: Sage. Hubley, A., & Zumbo, B. D. (1996). A d i a l e c t i c on v a l i d i t y : Where we have been and where we are going. The Journal of General Psychology, 123 (3), 207-215. Kelley, C , & Meyers, J. (1992). The Cross cultural adaptability inventory. Yarmouth, ME: I n t e r c u l t u r a l Press. Kelly, G. (1963). A theory of personality. New York: Norton. Kelloway, E. K. (1998). Using LISREL for structural equation modelling: A researcher's guide. Thousand Oaks, CA: Sage. Kristjansson, E. E., Desrochers, A., & Zumbo, B. D. (2003). Translating and adapting measurement instruments for c r o s s - l i n g u i s t i c and c r o s s - c u l t u r a l research: A guide for pr a c t i t i o n e r s . Canadian Journal of Nursing Research, 35 (2), 127-142. Matsumoto, D., LeRoux, J . , Iwamoto, M., Choi, J . W., Rogers, D., Tatani, H., & Uchida, H. (2003). The robustness of the I n t e r c u l t u r a l Adjustment Potential Scale (ICAPS): The search for a universal psychological engine of adjustment. International Journal of Intercultural Relations, 27, 543-562. Messick, S. (1998). Test v a l i d i t y : A matter of consequence. Social Indicators Research, 45, 35-44. Messick, S. (1995). V a l i d i t y of psychological assessment: Val i d a t i o n of inferences from persons' responses and performances as s c i e n t i f i c enquiry into score meaning. American Psychologist, 50, 741-749. McNair, D., Lorr, M., & Dropplemah, F. (1971). Profile of mood stages. San Diego: Educational and I n d u s t r i a l Testing Service. Mohan, B. (1986). Language and content. Reading, MA: Addison Wesley. O'Sullivan, S. (1999). The d i s t i n c t i o n between stable and dynamic c r o s s - c u l t u r a l competencies: Implications for expatriate t r a i n a b i l i t y . Journal of International Business Studies, 30, 709-736. Paige, R. M., Jacobs-Cassuto, M., Yershova, Y., & DeJaeghere, J. (2003). Assessing i n t e r c u l t u r a l s e n s i t i v i t y : An empirical analysis of the I n t e r c u l t u r a l Development Inventory. In R. M. Paige (Guest Ed.), International Journal of Intercultural Relations, 27 (4), 467-486. Paulhus, D. L. (1991). Measurement and control of response bias. In J.P. Robinson, P. Shaver, and L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17-59). San Diego: Academic Press. Reddin, W. J . (1994). Using tests to improve training: A complete guide to selecting, developing and using training instruments. Inglewood C l i f f s , NJ: Prentice-H a l l . Sapir, E. (1929). The status of l i n g u i s t i c s as a science. Reprinted i n David Mandelbaum (Ed.), Selected writings of Edward Sapir. Berkeley: University of C a l i f o r n i a Press, 1949. Selmer, J . (1999). Career issues and in t e r n a t i o n a l adjustment of business expatriates. Career Development International, 4 (2), 77-87. Thomas, D. C , & Inkson, K. (2004). Cultural intelligence: people skills for global business. San Francisco: Berrett-Koehler Publications Inc. Ward, C , & Kennedy, A. (1999). The measurement of s o c i o l c u l t u r a l adaptation. International Journal of Intercultural Relations, 23, 659-677. Westrick, J . (2004). The influence of service-learning on i n t e r c u l t u r a l s e n s i t i v i t y : A quantitative study. Journal of Research in International Education, 3 (3), 277-299. Whorf, B. L. (1956). Language, thought and reality. ( J . B. C a r r o l l , Ed.), Cambridge, MA: MIT Press. Yamamoto, S. (1998). Ibunkasenshitivity modem wo nihonhin ni taioh suruniattate —saiteigi no hitsuyousei ni tsuite-Applying the developmental model of i n t e r c u l t u r a l s e n s i t i v i t y i n Japanese context. Journal of Intercultural Communication, 2, 77-100. Yamamoto, S., & Tanno, D. (2002). Ibunkakanjuseihattatsushakudb (The Intercultural Development Inventory) no nihonjin ni taisuru tekiyohsei no kentoh: Nihongo bansakusei wo shiya ni irete (Assessing the a p p l i c a b i l i t y of the I n t e r c u l t u r a l Development Inventory to Japanese: bringing i n the viewpoint from the creation of a Japanese version). Journal of the Aomori National University, 7 (2), 24-42. Zumbo, B. D., & Taylor, S. V. (1993). The construct v a l i d i t y of the extroversion subscales of the Myers-Briggs Type Indicator, Canadian Journal of Behavioral Science, 25, 590-604. Zung, W. (1965). A s e l f - r a t i n g depression scale. Archives of General Psychiatry, 12, 63-70. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0092510/manifest

Comment

Related Items