"Education, Faculty of"@en . "Curriculum and Pedagogy (EDCP), Department of"@en . "DSpace"@en . "UBCV"@en . "Crittenden, Rose Elizabeth"@en . "2009-02-11T20:34:25Z"@en . "1996"@en . "Master of Arts - MA"@en . "University of British Columbia"@en . "Some second language learners are more successful than others. Students in\r\nthe University of British Columbia/Ritsumeikan Joint Academic Exchange\r\nProgramme, a sheltered English-as-a-second-language (ESL) immersion program,\r\nhave in the past exhibited varying degrees of gain in English proficiency in their\r\nwriting, reading, speech, and academic achievement during their stay in Vancouver.\r\nThe explanation of why some learners become proficient in a second language may\r\nlie in our understanding the interactions of such individual attributes as the\r\nlearner's age, language aptitudes, autonomy and motivation, attitudes, personality,\r\ncognitive style, learning strategies, background in language and knowledge of other\r\nlanguages.\r\nIn this study the gain in English proficiency of all the students in the program\r\nwas examined first and then the gain of two different groups of students who were\r\ncategorized and \"low\" and \"high\" on the basis of their entry level scores on the Test\r\nof English as a Foreign Language (TOEFL). The standardized tests used in this\r\nprogram were supplemented with two additional tests to measure gain in\r\nproficiency. A language experience questionnaire was given to all students and\r\ninterviews of selected students from each group were conducted. The results of the\r\nstandardized tests were analyzed and a significant difference in the gain of English\r\nproficiency between these two groups was found. An evaluation of the individual\r\nlanguage learning histories and the interview data was conducted to further\r\nunderstand the language proficiency gains found from the psychometric measures.\r\nImplications for instruction and further research were reached."@en . "https://circle.library.ubc.ca/rest/handle/2429/4466?expand=metadata"@en . "8716853 bytes"@en . "application/pdf"@en . "A B E T W E E N G R O U P S C O M P A R I S O N OF G A I N S I N E N G L I S H P R O F I C I E N C Y I N A S H E L T E R E D E N G L I S H I M M E R S I O N P R O G R A M by ROSE E L I Z A B E T H C R I T T E N D E N B.A. , The University of Ottawa, 1963 A THESIS SUBMITTED I N P A R T I A L F U L F I L L M E N T OF T H E R E Q U I R E M E N T S FOR T H E D E G R E E OF M A S T E R OF A R T S i n T H E F A C U L T Y OF G R A D U A T E STUDIES (Centre for the Study of Curr iculum and Instruction) We accept this thesis as conforming to the required standard T H E UNIVERSITY OF BRITISH C O L U M B I A A p r i l 1996 \u00C2\u00A9 Rose Elizabeth Crittenden, 1996 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of The University of British Columbia Vancouver, Canada DE-6 (2/88) ABSTRACT Some second language learners are more successful than others. Students in the University of British Columbia/Ritsumeikan Joint Academic Exchange Programme, a sheltered English-as-a-second-language (ESL) immersion program, have in the past exhibited varying degrees of gain in English proficiency in their writing, reading, speech, and academic achievement during their stay in Vancouver. The explanation of why some learners become proficient in a second language may lie in our understanding the interactions of such individual attributes as the learner's age, language aptitudes, autonomy and motivation, attitudes, personality, cognitive style, learning strategies, background in language and knowledge of other languages. In this study the gain in English proficiency of all the students in the program was examined first and then the gain of two different groups of students who were categorized and \"low\" and \"high\" on the basis of their entry level scores on the Test of English as a Foreign Language (TOEFL). The standardized tests used in this program were supplemented with two additional tests to measure gain in proficiency. A language experience questionnaire was given to all students and interviews of selected students from each group were conducted. The results of the standardized tests were analyzed and a significant difference in the gain of English proficiency between these two groups was found. An evaluation of the individual language learning histories and the interview data was conducted to further understand the language proficiency gains found from the psychometric measures. Implications for instruction and further research were reached. ii TABLE OF CONTENTS Abstract i i Table of Contents iii List of Tables v List of Figures v i Acknowledgement vii INTRODUCTION 1 Chapter One Identification of the Problem Purpose of the Present Study 3 The Research Questions and Hypothesis 3 Significance of the Problem 4 Definition of Terms 4 Description of the University of British Columbia/ 7 Ritsumeikan Academic Exchange Programme Chapter Two Review of the Relevant Literature Introduction 15 Second Language Motivation and Bilingualism 15 The Immersion Environment 17 Study Abroad in the Target Language Environment 24 Discussion of Proficiency in English as a Second Language 26 Factors Affecting Individual Second Language Acquisition 30 Summary 34 Chapter Three Design of the Study 35 Research subjects 36 Measuring Instruments 38 Qualitative Measures 42 Data Collection 42 Data Analysis 43 Limitations of Design 44 iii Chapter Four Findings and Interpretations The Research Questions 45 Quantitative Findings 46 Qualitative Findings 68 Interpretation of Results 76 Synopsis 77 Chapter Five Discussion and Implications Introduction 78 Discussion 78 Implications 80 References 83 Appendix 1 91 Appendix 2 97 iv List of Tables Table 1 Assignment of Students to Classes Year 5 (1995/1996) 12 Table 2 Distribution of Students by Faculty, Year, and Gender for Rits 4 (1994/1995) 37 Table 3 Distribution of Students by Faculty, Year, and Gender for Rits 5 (1995/1996) 37 Table 4 Year 4 Test Results 46 Table 5 Year 5 Test Results 46 Table 6 Correlation of Tests for Year 4 65 Table 7 Correlation of Tests for Year 5 66 v List of Figures Figure 1 Progress of all students in Year 4 47 Figure 2 Progress for high and low groups in Year 4 48 Figure 3 Progress of all students in Year 5 53 Figure 4 Progress of high and low groups in Year 5 for TOEFL, Oral Interviews and Writing Sample 55 Figure 5 Progress of high and low groups in Year 5 for CELT and Cloze 56 Figure 6 Comparison of the progress of all students in Year 4 and Year 5 60 Figure 7 Comparison of progress in Year 4 and Year 5 by groups 61 vi ACKNOWLEDGEMENTS I sincerely thank Dr. Stephen Carey, my research supervisor, for his generous and patient guidance, help, support, and encouragement during all phases of this thesis, and Dr. Richard Berwick, member of my committee, for his time and expertise. I appreciated the privilege of working in the University of British Columbia/ Ritsumeikan Joint Academic Exchange Programme. I wish to thank Masahiko Chikakiyo, Joe Greenholtz, Jean Hamilton, Cheri MacLeod, William McMichael, Garold Murray, Kiyokatsu Ohashi, Akio Tanaka, Dwight Tkatschow, Sheri Wenman, and the two hundred and one students of Rits 4 and Rits 5 for making this study possible. I also thank my friend Sara who is always willing to listen and my family for understanding. vii CHAPTER ONE: INTRODUCTION In today's shrinking world English has become the lingua franca of international communication, publication, and commerce. To effectively cope with this reality \"some degree of bilingualism is essential for non-English speakers to communicate in the international communicative network\" (Carey, 1993). Consequently, Japan, the world's leading industrial nation, has up-dated its education system so that Japanese students now begin the study of English in elementary school. Until 1992 Japanese students began instruction in English in junior high school, then Monbusho (the Japanese Ministry of Education), instituted the change in response to the fact that Japanese students are consistently at the lowest level of the scale for the Test of English as a Foreign Language (TOEFL) examinees (ranking 22ndi out of 27 Asian countries). Also, in 1994 major revisions were made to the Japanese foreign language teaching policy to include a variety of foreign languages, not just English (Monbusho, 1992). Unfortunately, after studying English for six to eight years in Japan with native Japanese teachers and the traditional grammar translation method, few Japanese students are able to speak the language. One way to overcome this situation is for Japanese students to participate in exchange programs with institutions in English-speaking countries. The program under consideration in this study, the University of British Columbia/Ritsumeikan Joint Academic Exchange Programme (AEP) is an unique sheltered post-secondary immersion program,and it is the largest academic exchange program at the tertiary level of education in North America. The difference between the entrance and exit test scores of students in this program can be taken as evidence of the students' gain in English proficiency acquired during their stay abroad. In year-end program summaries, course administrators have determined the significance, in statistical terms, of any gains achieved by students 1 TOEFL Test and Score Data Summary 1995-1996 Edition 1 on the Test of English as a Foreign Language (TOEFL), the Oral Interview and the Writing Sample, measures of proficiency used in the program (Berwick & McMichael, 1993). These measures of absolute change during an academic year provide a gauge for assessing oral and written language development. 1.1. IDENTIFICATION OF THE PROBLEM While most children under normal circumstances master their mother tongue, not all language learners master a second language. Second language achievement can range from near native fluency to minimal success. The question of why second language learners meet with different degrees of success is one that has not been adequately answered because the research done in this area has yielded inconclusive or contradictory findings (Larson-Freeman & Long, 1991). The most common variables involved in learners becoming proficient in a second language are gender, intelligence, and individual attributes such as the learner's age, language aptitude, autonomy and motivation, attitude, personality, cognitive style, learning strategies, background in language, and knowledge of other languages (Ellis, 1994). The students in the AEP have in the past exhibited varying degrees of gain in their writing, reading, speech, and academic achievement in English during their stay in Vancouver. (Berwick & McMichael, 1993). The questions of which students make the most gain in proficiency (as measured by a battery of standardized and course tests) and why, are intriguing and should be explored so that this and other exchange programs may: (a) better select students, ( b) identify those measures that are superior at predicting students' success in an exchange program, and (c) determine how such programs can be improved. 2 1.2. PURPOSE OF THE PRESENT STUDY The purpose of this study was to try to determine if there were any differences in the gains in English proficiency between the \"low\" students, those who entered the program with a TOEFL score falling within the first quartile, and those \"high\" students who entered with a TOEFL score in the fourth quartile, during an academic year of study abroad in a sheltered immersion program in an English mil ieu. 1.3. THE RESEARCH QUESTIONS There are four specific questions to be addressed in this study. Question 1. How much improvement (gain) in the English writing, reading, speech, and academic achievement of students occurs during the 8-month stay of students in the University of British Columbia/Ritsumeikan Joint Academic Exchange Programme? Question 2. Is the improvement greater in \"low\" students than in \"high\" students? What is the difference in gain between groups? Question 3. Is it possible to isolate those students who show the most and the least gain and to determine the correlates of both groups? Question 4. Is it possible through quantitative and qualitative analyses to determine the factors and activities that would account for improvement in English as a Second Language? 1.4. NULL HYPOTHESIS Because of the exploratory nature of this study, the null hypothesis was tested, namely, that there is no significant difference in the gain of English proficiency between \" low\" and the \"high\" students in this sheltered immersion program. 3 1.5. SIGNIFICANCE OF THE PROBLEM A review of the literature indicates that no study has tried to predict the characteristics of a group of learners who would make the most gain in proficiency in a sheltered English immersion program at the college level, nor has any study looked at the dimensions of such gains as pertains to the University of British Columbia/ Ritsumeikan Joint Academic Exchange Programme. This study, based on the above research questions, was accomplished with a group of Japanese English-as-a-second-language (ESL) learners. The use of a between-group comparison design to determine the actual growth in English proficiency during the students' eight-month stay at the University of British Columbia has implications for (a) teaching and further research in the areas of second language acquisition, (b) English sheltered immersion programs, and (c) study abroad programs. Furthermore, there are implications for studying the criteria applied to the selection process of students for this and similar types of exchange programs. Predictors of which students would be the most successful in an exchange or immersion program and insights into gains in English proficiency would be valuable in improving this AEP, and in offering suggestions to administrators and program designers of other programs. 1.6. DEFINITION OF TERMS A brief glossary of some key terms used in this study follows. Where possible some comment on the history of the term and the manner in which it is generally used in second language acquisition research is given. LI refers to an individual's first language (mother tongue) and L 2 to a second or subsequent language. The common term for English as a second-language, ESL, is used in this thesis. Additive Bilingualism: the type of bilingualism where the learner adds a second or subsequent language to a well-developed first language without losing any competence in the first language. 4 Basic Interpersonal Communication Skills (BICS): the term used by Cummins (1981) to denote the kind of L2 oral proficiency required by learners to use the second language in undemanding, context-based, informal social communicative tasks. Cognitive Academic Language Proficiency (CALP): the kind of second language proficiency in reading and writing necessary for successful academic study (Cummins, 1981). BICS incorporates the language skills of listening and speaking but does not require literacy as does the CALP. Common Underlying Proficiency (CUP): the assumption that the L2 is mediated through a well-developed LI (Cummins, 1984). Communicative Competence: refers to the knowledge that language users have internalized that enables them to understand and produce messages in the language. Communicative competence may consist of four components: (a) grammatical or linguistic competence; (b) sociocultural competence; (c) discourse competence, the ability to sustain coherent discourse with another speaker; and (d) strategic competence, the means by which learners deal with potential breakdowns in communication (Canale & Swain, 1980). Comprehensible Input Hypothesis: advanced by Krashen to explain how language is acquired. Krashen (1981,1982,1985) claims that (a) language is acquired only when students understand language that is \"a little bit\" (i +1) beyond their current level, (b) students need to intake (\"let in\") what they comprehend, (c) this input is made comprehensible by extralinguistic and textual indicators; and (d) speaking contributes only indirectly to acquisition. Divisible Competence Hypothesis: claims that language proficiency can be divided into separate components (phonology, syntax, lexicon, etc.), skills (listening, speaking, reading and writing), or comprehension aspects of these, each of which can be measured by discrete point testing. \"High\" and \"Low\" Students: students in this study were differentiated on the basis of their in-coming TOEFL scores, as it is this score that is the main determinant of 5 their entry into the program. The incoming TOEFL scores for the academic years 1994/95 and 1995/1996 (see Table 1) were used to assign students with scores in the 1st quartile (Qi) as \"low\" and students in the 4th quartile ( Q 4 ) as \"high\". Interactional Ability: one of Bachman's (1991) approaches to defining language proficiency by modular abilities, such as skills, functions, or communication. Interdependency Principle: a way of explaining why students who have a high literacy in their first language (LI) may find it easier to develop cognitive academic language proficiency (CALP) in a second language than those lacking such literacy (Cummins, 1981). Proficiency: in general terms, the word \"proficiency\" from the Latin \"peritus\" means skilled. In terms of language acquisition, proficiency refers to a learner's skill in using a second language (L2), (Ellis, 1994). Ellis further distinguishes between competence (learning) or knowledge of the second language internalized by the learner and proficiency (performance) or the learner's ability to use this knowledge in different tasks. Ellis' (1990) earlier definition of proficiency as the ability to use language in a specific sociopragmatic context is the meaning given to the term proficiency in this study. Related concepts that have much in common are capability and capacity . A more complete discussion of the concept of proficiency will be undertaken in Chapter Two in the Review of Relevant Literature. Psychometric Measures: standard tests which are used to establish benchmarks of cognitive ability, namely, intelligence testing, gain in proficiency, or standards of capability. The psychometric measures used in this study to establish baseline norms were the Test of English as a Foreign Language (TOEFL), the Oral Interview, the Writing Sample, the Comprehensive English Language Test (CELT) and the Cloze. Sheltered Immersion: a program in which the second language (L2) is not taught as a separate subject but is the medium of instruction for teaching content. A sheltered immersion program usually separates a selected group of L2 learners from other 6 students at an institution and the L2 presented is for L2 learners. Related programs in which second language students are taught content in separate classes are also designated adjunct, supporting, and English for special purposes classes. The University of British Columbia/Ritsumeikan Joint Academic Exchange Programme takes place in a double-sheltered, immersion environment for not only are the classes separate but most students are housed in a separate building that includes some of their classrooms, a multimedia lab and other resources. The value of this double-sheltering has not been studied. Study Abroad: programs designed to permit students to participate in another culture by living and studying in that environment. There are numerous variations on this theme, ranging from short two-week \"culture\" tours to long-term graduate work in another country. What happens to students when they participate in study abroad programs for substantial lengths of time is an interesting question. Unitary Competence Hypothesis: the claim that the components of language competence function in similar ways in any language-based task and can be measured using an integrative approach where it is possible to apply Spearman's factor-analytic techniques for testing for the g (general factor) of language proficiency (Oiler, 1980). 1.7. T H E UNIVERSITY OF BRITISH COLUMBIA/RITSUMEIKAN A C A D E M I C EXCHANGE PROGRAMME 1.7.1. Introduction: The University of British Columbia/Ritsumeikan Academic Exchange Programme (AEP) is an eight-month residential academic exchange between the University of British Columbia in Vancouver, Canada, and Ritsumeikan University in Kyoto, Japan. Every September, approximately one hundred, second and third year students, from Ritsumeikan University participate in the program. These 7 students, from 20 to 23 years of age, study at the University of British Columbia from September to April.The program which started in 1991 is now in its fifth year. The purpose of the AEP 1 is to further intercultural understanding between students in Canada and students in Japan through academic, sociocultural, and linguistic development in a credit-bearing, residential academic exchange program. The instructional program provides training in English language in the three interrelated areas of academic development, sociocultural development, and linguistic development. Academic development is furthered by both universities working together to develop and implement a variety of cooperative learning activities that are not restricted solely to class or study times. The two institutions strive to ensure that opportunities are provided for the integration and sharing of knowledge and expertise not only between the two universities but also through interaction with the appropriate outside communities such as other institutions. Students are required to master the content of relevant areas of research. The two universities work together on sociocultural programs that empower students to become independent in their learning. Every effort is made to ensure that a positive learning environment where every participant has a voice, is maintained. Students are expected to take full advantage of opportunities to become involved in the atmosphere of a Canadian university. Linguistic development in English is supported through content-based instruction in such areas as environmental and social issues. Organizational context, technological support, and curriculum resource materials essential for the development of communication skills are provided. It is intended that students become autonomous in the development of their listening, reading, speaking, writing, grammar and vocabulary skills. The program participants are considered to be fully-fledged University of British Columbia students. They are required to take two courses offered by the see Programme Handbook 8 Department of Language Education, Faculty of Education, and two offered by the Faculty of Arts, for a maximum total of 18 transferable credits. These courses are outlined in section 1.7.3. Successful participants receive credits from both the University of British Columbia and Ritsumeikan University. 1.7.2. \"Rits House\" \"Rits House\" was completed in 1992. It consists of fifty 4-bedroom apartments, each with one and a half bathrooms, shared kitchen and living room. Accommodation assignment usually places two Japanese and two Canadian students to an apartment. The building also contains three seminar rooms, a tatami (Japanese style) room, a multi-purpose room, and the program offices. The program's multimedia lab is in the basement. 1.7.3. Required Courses for the Academic Year 1995/96 In Term 1 students take four compulsory courses, two in Language Education and two in Arts Studies. Each course includes a lecture, a session in the multimedia lab, and a tutorial for a total of four and a half hours of class time per week. Language Education 206: Language Field Experience in Second Language Education is a comprehensive introduction to Vancouver and its region with respect to the geography, history, the natural environment, political structure, and economics. Students read and discuss articles on Canadian culture, history, economics, politics, and ethnic groups. Language Education 226: An Introduction to Language Across the Curriculum is an introduction to a variety of topics related to Canadian second language education issues and matters relating to Canadian society as a whole are explored. In Term 1 students design a research project based on a visit to a local school and then make oral reports on what they learned. Topics selected for research by students are usually cultural specific, namely, food, dress, school regulations, in-class conduct, and attitudes. Students return from their school visits with opened-eyes about school culture and education in Canada. 9 Time is spent in class comparing their experiences in Canada with their experiences in Japan. In addition to this research project students write a major paper on a topic of interest to them, preferably, with a Canadian focus. Papers on environmental issues, First Nations people, the diversity between Canadian and Japanese culture, and feminist issues are popular examples. Papers are then presented orally, with appropriate media, during a mini-conference. Students with similar topics are encouraged to present their term papers jointly and can utilize multimedia (video, audio, computer generated graphics and animation) equipment made available in the multimedia lab, to present their research in many dimensions. All Ritsumeikan exchange students take two adjunct courses offered by the Faculty of Arts. These courses which are jointly developed and taught by University of British Columbia and Ritsumeikan faculty are: 1. Arts Studies 201. Canada, Japan and the Pacific: Cultural Studies: A sociopyschological and linguistic approach to experience across cultures, with particular reference to the exposure of Japanese students to North American sociocultural environments and Western values. The course is meant to create an open-minded attitude to cross-cultural experiences. 2. Arts Studies 202. Canada, Japan and the Pacific: Political, Economic, and Geographical Perspectives: An exploration of the political, economic and geographical interactions between Canada and Japan that explores the links, both historical and contemporary, between these countries and other Pacific Rim nations. In Term 2 students are allowed to take regular UBC courses in other Faculties if their TOEFL scores and academic index are high enough. 1.7.4. \"RitsLab\" One of the special features of this program is the combination language and computer lab (RitsLab) which allows students access to the network resources at UBC. Students are taught computer skills in ClarisWorks (word processing, 10 spreadsheets, drawing and painting), the World Wide Web (Internet), and electronic communication and then use these tools to pursue their individual interests. Students construct their own Home Page on the Internet and use other media to prepare and present research in a computer-based format rather than in the traditional bulletin type academic research paper. 1.7.5. Sociocultural Activities Ritsumeikan participants take an active role in campus life and a number of off campus activities. As part of their course curricula, participants visit local schools and colleges and conduct much of their required fieldwork off campus. Each year students publish a Yearbook and a Research Bulletin in addition to hosting a mini-conference on issues of intercultural communication. Many students act as volunteers in local nursery schools, hospitals and join the Buddy Program which matches Japanese students with Canadian students who have similar interests. Music plays an important role in extra-curricular activities and every year students have expressed their unique musical talents. In October 1995, the Ritsumeikan students took a very active role in UBC's Open House by planning and conducting a number of activities and demonstrations at \"Rits House\". Projects included a daily lunch of Japanese food prepared by the students, a Tea Ceremony, origami, ikebana, haiku, calligraphy and music. 1.7.6. Other Ritsumeikan English Language Exchange Programmes Exchanges between Ritsumeikan University in Japan take place with The University of Pittsburgh; The University of Oklahoma; The American University at Washington, D . C , which offers a double degree program for twenty to twenty-five exchange students; The University of Warwick (UK); and The University of Melbourne in Australia (Rits English Calendar, 1994-1995). 1.7.7. The Selection of Student for the AEP Students for the AEP are selected primarily on the basis of their interest and TOEFL scores. The administration prefers to have students in their second year of 11 university although third year students may join. Due to the current economic conditions in Japan students in their third and fourth year of university have become apprehensive about finding employment after graduation and need to devote more time job searching in Japan starting in their third year. Thus, third-year students are relatively under-represented in the AEP. Students who wish to come to Canada for this program are selected solely on the basis of their TOEFL scores with the top one hundred candidates accepted. If a student drops out of the program after the pre-departure orientation courses held in Japan have commenced they are not replaced, so that in some years the numbers may be below 100 and in others, like 1994/95, above. Variations in TOEFL scores from year to year give each set of students a different mean TOEFL score. Ritsumeikan University provides scholarships for the top ten students who require some financial assistance based on a family means test. 1.7.8. The Assignment of Students to Classes In Year 4 the students were randomly assigned to classes but in Year 5 TOEFL scores were the basis of initial class assignments: the one-fifth of the students with the highest TOEFL scores were assigned to one class; the one-fifth of the students with the lowest TOEFL scores, to another; and the remaining students were divided into three classes. Table 1. Assignment of Students to Classes in Year 5 (1995/1996) CLASS TOEFL RANGE MALES FEMALES TOTAL Class 1 493-533 9 10 19 Class 2 497-540 8 12 20 Class 3 540-607 3 17 20 Class 4 477-493 6 14 20 Class 5 493-540 9 9 18 TOTAL 35 62 97 12 Enrolment in the regular course electives offered by the Faculty of Arts and the Faculty of Science for Term 2 in Year 4 was dependent on the student's best TOEFL score in the Institutional Testing Program (LTP) TOEFL. Students wrote the FTP in September, October, and November. In Year 5, an indexing system consisting of an average of students' Term 1 marks in addition to their best TOEFL score, was initiated. It was found in the past that some Japanese students could not achieve the TOEFL score of 570 required by UBC to enter regular programs so arrangements were made to allow the participants in the AEP special status with regard to qualification for these courses. In Term 2 students were able to select their core Language Education classes by lottery. Each student was randomly assigned a computer-generated number that allowed that student a ranked position in the sign-up line for the class of their choice. 1.7.9. Students' Personal Evaluation of the Program Student evaluation of courses and discrete non-instructional components of the program were acquired by means of a 10-point Likert-type scale for all items. Students have stated that \"they were satisfied\" with such assorted elements as course content, intellectual challenge, difficulty of assignments, teaching quality, amount learned, and these approval ratings have, each year, increased in Term 2 compared to Term 1. 1.7.10. Other Studies Involving Students Previous research conducted using the Japanese students in the AEP were in the areas of: 1. Second language writer's processes, performance, and perceptions in English as a second language (ESL) composition; the effects of second language learning on the first language. 2. The language of requests made on the telephone by native English speakers, native Japanese speakers and Japanese ESL speakers. 13 3. The restructuring of linguistic knowledge among adult ESL students. 4. A longitudinal study of the effects of instruction on the development of noun phrase reference by adult Japanese ESL learners. 5. A study in the field of social research examining predictors of adjustment and satisfaction among the participants to their study abroad environment. At present there are no research projects being undertaken at the other Ritsumeikan University exchange sites listed in section 1.7.6. 14 CHAPTER TWO: REVIEW OF THE RELEVANT LITERATURE 2.1. INTRODUCTION Literature relevant to this thesis touches on the following subject areas: (a) bilingualism and bilingual education, (b) the context of sheltered post-secondary immersion programs, (c) study abroad in the target language environment, (d) the construct of proficiency in a second language, (e) the definition and measurement of proficiency, and (f) Japanese second language learners studying English as a second language. All of these subject areas will be related to Japanese college student studying English as a second language at the University of British Columbia/Ritsumeikan Joint Academic Exchange Programme. 2.2. SECOND L A N G U A G E MOTIVATION A N D BILINGUALISM People learn a second or additional language for diverse reasons including basic necessity, economic improvement, and entry into another culture. A fourth reason, often cited as a rationale for immersion language programs, is the belief that a bilingual may have greater cognitive flexibility, and superior ability in concept formation than does a monolingual (Peal & Lambert, 1962). Motives are never straightforward, nor mutually exclusive. Dunkel (1948) believed that motivation was an important element in language learning and that there were two facets to this motivation: ( a) \"kind\", referring to the purpose for studying the language; and (b) \"intensity\", the effort expended in learning the language. Gardener and Lambert (1972) also claimed there were two distinct types: (a) instrumental motivation, the learning of a language for the social and economic benefits to be gained; and (b) integrative motivation, the learning of a language 15 because of an interest in, liking for and the desire to become a member of a certain cultural group. Immersion programs designed to produce bilinguals have the same motivational foundations. In Canada the Official Languages Act of 1969 made the country officially bilingual and gave the public access to services in both French and English. This situation required a civil service trained in a second language and the federal government provided second language training for federal employees. No doubt the majority of people who took these programs, to become effectively bilingual, did so from necessity or to improve their chances for promotion. Yet there were those who had a genuine interest in and wish to enter the other official founding culture. The principle of additive bilingualism proposed by Lambert (1974) proposes that it is possible for a majority language student to add a second language to a well-developed first language through instruction in the second language and that this addition can be made at no cost either to the student's first language abilities or to their level of academic achievement. Studies of Canadian French immersion programs have led researchers such as Genesee (1984,1987a, 1991) and Swain and Lapkin (1982) to conclude that students in French immersion classes achieved high levels of L2 French proficiency while maintaining their normal English language proficiency. Additive bilingualism is typified by English speakers in Canada and Flemish speakers in Belgium learning French as a second language. Lambert (1974) further claimed that additive bilingualism was possible where students come from family backgrounds in which their first language is well established and the acquisition of a second language is deemed to be an enrichment. However, for those students the use of French was confined to the classroom and they continued to use English as their dominant language. This claim is also applicable to the Japanese speaking population of Japan where English enjoys the prestige of being considered a necessary component of 16 education (Monbusho, 1992). Japan's need to understand western cultures, import technology and goods, and to foster internationalism has led to the teaching of English at the high school level since the Meiji era. Now the increasing need for English, in a world of instant communication, has led to experimental teaching of English communication at the elementary school level (Koike, 1994). Second language motivation and additive bilingualism found in the Canadian studies are directly linked to the AEP and this study. The Japanese students, their parents, and Ritsumeikan university all view English proficiency as desirable and economically essential in present-day Japan. The principle of additive bilingualism applies since the students' first language of Japanese is already well-developed when they enter the AEP and the acquisition of English is considered prestigious. 2.3. THE IMMERSION ENVIRONMENT The environment in which second language learning takes place is thought to be important but its precise influence on the acquisition of second language has been the subject of a long-standing debate. Behaviourist theories of learning attach great importance to external factors; mentalist theories focus on the internal factors of the learner; and cognitive theories of language acquisition recognise the influence of both external and internal factors (Ellis, 1994). Learner (internal) and environmental (external) factors in language development are both multidimensional (compound) and interactive. Recent research has examined how and when these multidimensional factors arbitrate in second language development but the precise relationship has, so far, been elusive (Ellis, 1994; Long, 1990). Environmental, or external factors seem to have an indirect affect on second language (L2) learning since they are subject to mediation by the learner's attitudes and social context. Ellis (1994) points out that second language teaching and learning 17 occurs in both educational and non-educational contexts and raises the question of whether the learning that takes place in each is unconnected. There is a strong connection between the educational and non-educational elements of the learning environment in the AEP. A setting that is both educational and natural is provided through the sheltered immersion classes, the housing of most students in a separate building 2 with English speaking roommates, and regular classes with Canadian and other international students. Sheltered Post-Secondary Immersion A sheltered immersion program teaches the second language through content in the second language and separates a particular group of L2 learners from other students at an institution. Studies, such as Carey (1984,1991) and Dube (1991) found that to develop proficiency in a second language, it is important that language learners be segregated or sheltered in an attempt to provide both a natural and an educational social context for L2 learning. It is suggested that the gains in proficiency in the AEP Students are attributable in part to their sheltering as students are more likely to feel at ease, a necessary component of comprehensible input, in a classroom of their peers with a teacher who is both experienced and trained in the instruction of English as a second language. The teaching staff of the AEP has both expertise in the instruction of Japanese students, and knowledge of Japanese culture and language. Canada's French language immersion programs, are well known around the world. In Canada, the term immersion is used to denote the education of students belonging to the dominant language (English) group in the language of the minority (French). Such program types are usually divided into early total immersion, early partial immersion and later immersion. An immersion program may, however, be of any length and range from an entire elementary and secondary program, to partial immersion in elementary or secondary school, or a short intensive program. 2 see description in section 1.7.2. 18 Underlying Theories of Immersion The original intention of immersion programs was to promote bilingualism by educating students in a second language after their first language was well developed (Carey, 1991). The documented general effectiveness of immersion programs indicates that an approach in which second language instruction is integrated with academic or content instruction is a more effective way to teach language skills than an approach in which the second language is taught in isolation. (Genesee, 1991 p. 184) It is also argued that there may be a cognitive gain in immersion (Cummins, 1979, Lambert, 1974, Lambert & Tucker, 1972). Students in second language immersion programs make ordinary or even extraordinary progress in content subjects that are taught in the second language (Cohen & Swain, 1979; Genesee, 1987) when compared with similar groups of monolingual students. Findings by Bamford and Mizokawa (1991) imply that children who develop second language skills in supportive immersion environments show enhanced nonverbal problem-solving abilities as measured by the Coloured Progressive Matrices (CPM), as well as problem solving abilities in science (Roseberry, Warren, and Connant, 1992). However, the results of these and similar studies that use between group comparisons are as Carey (1996) so succinctly states: ...fraught with danger since we do not even have sufficient understanding of the second language acquisition process to know which variables it would be critical to equate in these two student populations, let alone equate for teachers, program, family background, parental encouragement or a host of other obvious motivational variables and their interactions with student variables, (p. 1) 19 Benefits of Immersion: Some of the positive benefits derived from learning a second language are outlined in the following studies: 1. Canadian French immersion programs have been successful in helping students achieve proficiency in a second language (Genesee and Lambert, 1983; Genesee, 1984 and Swain, 1985), although it is possible that such positive results are coloured by the reality for the need of continued funding. 2. Students in immersion programs benefit from high levels of comprehensible input (see Krashen, 1985 re: the \"comprehensible input hypothesis\"); and immersion programs encourage such comprehensible input. Sheltered content teaching, by its very nature, does lend itself to simplification, use of context, and negotiation, all factors that contribute to comprehension. 3. Students remain as a homogeneous LI group which does not threaten their identity, yet allows them to adjust to the setting easily (Swain and Lapkin, 1982); and 4. Students in immersion programs tend to have less rigid ethnocentric stereotypes of the target language culture and place more importance on intercultural contact (Ellis, 1994). Cohen (1993) suggests that immersion programs help create an internal language environment in which the learner becomes capable of conducting their cognitive processing in the foreign language. The more efficient immersion students are at thinking in the second language, the better they are able to process input and output, thus functioning better in that language. Essentially, the ability to conduct cognitive processes in a second language is the hallmark of knowing a second language. A capacity for rapid translation from one language to another is a major advantage, particularly in the area of problem solving. A Comparison of Similarities in French Immersion Programs and the AEP The benefits found in French immersion in Canada are also represented in the AEP. Since the program is elective, those students who enter, along with their 20 parents, take a positive supportive stance. Student satisfaction surveys on the programme from previous years give high levels of approval in key categories and interviews with the students in this study indicate that students feel they have attained a great deal of increase in proficiency, both oral and written, during the academic year (Berwick and McMichael, 1993). Disadvantages of Immersion Although they are highly touted as the ideal way to acquire a second language sheltered immersion or segregated classes are not without their drawbacks. The perception is that immersion programs are highly successful, particularly in promoting bilingualism in Canada, and researchers tend to shy away from critical or negative evaluations. By so doing, evidence of weakness that could be used to improve a program is overlooked or down-played (Carey, 1984). Furthermore, the majority of studies conducted on French immersion in Canada have used a between groups design where students in immersion were compared with those in the regular English stream. Carey (1993) stated that it was impossible to \"cleanly compare the progress of these students with those in the regular English stream with any objectivity, since immersion students are an enriched group with select treatment\". While researchers agree that the Canadian French immersion programs are very effective in promoting second language acquisition in an educational setting, bilingual minority immersion programs in the United States have been targets of criticism. The two arguments given for opposing immersion in that country are based on the what Cummins (1988) called the false premises of: (a) linguistic mismatch' i.e., minority children will be retarded academically if they are taught only in L2; and (b) 'maximum exposure' i.e. children are deprived of the L2 exposure necessary for successful acquisition. Later, Cummins (1992) stated that immersion programs exclusively for majority LI students could be designed to reflect the \"interdependency principle\" and the \"comprehensible input hypothesis\", 21 thus assuring success as demonstrated by the Canadian immersion programs. Perhaps the major drawback to language immersion programs is that they tend to leave gaps in the foreign language proficiency. In spite of having had a number of years of comprehensible input, students in French immersion classes in Canada were found to produce a second language in which both vocabulary and structure was reduced when compared with their LI. Genesee (1987) and Lapkin, Swain and Shapson (1990) found that immersion students' spoken and written French contained extensive morphological, syntactic, and lexical differences when compared with the French of a native speaker, and Hammerly (1987, 1989) found that a kind of \"classroom pidgin\" could develop. In other studies Long (1991) determined that students may use a form of negative language transfer called \"relexification\" wherein foreign language words are slotted into native language structures while communication transfers like code-mixing (the use of both LI and L2 in the construction of the same sentence), and code switching (the alternative use of LI and L2 in discourse) have been found by a number of researchers (see Kasper, 1984b for types of production transfer). Such gaps may be attributable to the fact that French immersion students have relatively little opportunity for extended discourse either in class or in the community (Carey, 1984; Cohen, 1993). Major Studies on Sheltered Immersion: French Immersion in Canada has produced in the past twenty years a number of studies on a variety of questions such as: 1. The development of literacy at a level appropriate to master academic content (Carey and Cummins, 1983,1984; Carey, 1984; Cummins, 1983,1987; Lambert, 1974; Swain and Lapkin, 1982). 2. The effect of immersion on first language skills: issues of additive or subtractive bilingualism (Lambert, 1974; Genesee, 1984), and 3. The social and psychological aspects of immersion (Lambert and Tucker, 1972; 22 Cziko, 1978). These studies have been done in elementary and secondary schools using between group designs which fail to take into consideration the fact that students not in immersion classes may not be a suitable comparison group (Reynolds, 1991). With the exception of studies done on French immersion by Carey (1984,1991) at the University of Alberta, Dube (1991) and (Edwards, Wesche, Krashen, Clement, and Kruidenier, 1984) at the University of Ottawa, no studies have been conducted with college students. A strong correlation between reading speed in the first language and the reading speed and comprehension in a second language was found to indicate the role that LI plays in L2 reading comprehension, thereby leading to the prediction of the level of comprehension in L2 from comprehension in LI (Carey, 1991, 1993a, 1993b and de Moissac, 1990). The higher the comprehension and speed of reading in LI, the higher the comprehension and speed of reading in L2. The principles of additivity and interdependence were found to be true for students who had well-developed reading skills as well as prior knowledge in their LI. Research done at the University of Ottawa in sheltered second language introductory psychology courses at the college level found that adult students who had volunteered for sheltered subject matter sections in psychology were able to make verifiable gains in second language proficiency, even during a single semester. Sheltered class groups showed clear and statistically significant gains on all second language proficiency measures with the exception of a multiple choice cloze test for which gains did not reach statistical significance, possibly due to a ceiling effect (Edwards, Wesche, Krashen, Clement, and Kruidenier 1984). Second language teaching within the setting of sheltered classes allows for authentic linguistic input in a linguistically rich environment through instructor input and opportunity to engage in authentic conversation (Dube, 1991). Unfortunately, the results of these studies cannot be generalized due to subject selection (volunteers) and the fact that the immersion classes had extra tutoring, and time to study. 23 Implications of Immersion for this Thesis The sheltered immersion environment of the AEP appears suited to the development of English proficiency, particularly in the area of functional oral skills. 2.4. STUDY A B R O A D IN T H E T A R G E T L A N G U A G E ENVIRONMENT Language immersion programs, which have existed in Canada and the United States for over 20 years assume that second language acquisition occurs most easily and rapidly when the learners are within the target language environment and culture. (Cohen, 1993 p. 89) For centuries young people have been sent \"abroad\" with their tutors to learn the language and presumably, even more importantly, to sample the culture of foreign countries. After World War n study abroad programs for Japanese students have proliferated in English-speaking countries. From short 3-week summer stays to full academic exchanges and graduate school programs, Japanese students have been important clients to English language teaching institutions around the world. The general assumption that students will make greater progress in acquiring a foreign language, supposedly due to greater and more varied exposure, when living in the target language milieu (Opper,Teichler, and Carlson, 1990) has been supported in a number of studies which showed more progress with greater response time on task. Veguez (1984) found that native English-speaking students who studied Spanish in Spain showed an increase in oral fluency (BICS) but that that increase did not carry over to writing skills or to grammatical knowledge (CALP's) in Spanish. Data collected over a period of 20 years by the American Council of Teachers of Russian (ACTR) was published in four papers by the A C T R / N F L C Project. Brecht, Davidson and Ginsberg (1991); Ginsberg (1992); Ginsberg and others (1992); Brecht and others (1993) analyzed test results of studies of gains in Russian language 24 competence by American college students who took part in one semester (4-month) language programs in Moscow and Leningrad and concluded that these results expressed significant gains in language proficiency in oral, listening, and reading skills. All students made positive gains with 53% of the students moving up one whole level on the rating scale e.g., from level 2 to 3; 21% gained two levels and 4% three levels in the standardized tests. Gains on all test forms were found to be positively correlated. Listening gains, reading gains, and oral proficiency appeared to be associated. Many students gained on one modality but not on others. Results indicated that certain student characteristics including gender, experience in learning other foreign languages, and a command of grammar and reading skills were predictive of language gains abroad. One of the interesting findings of this Russian study was that the gains in oral proficiency were significantly higher for males than for females. It was suggested that this fact may be due to the communication interactions specific to the country at that time where the males had more opportunities to meet and converse with native speakers. In his study of Japanese students, Ellis (1993) found a positive correlation between use of English outside the classroom and a high academic achievement in language and subject matter in English. Most students enjoyed and appreciated the chance to interact with an English-speaking community, although several students reported negative experiences caused by their inability to sustain a conversation in English which led to a communication breakdown. Another study by Kitao (1993 ) which used a questionnaire and self-assessment protocol to measure Japanese students' attitudes towards learning English during the first week of the program and again on returning to Japan, determined that those students found learning English functions in context very useful and that they personally felt their English had made significant improvement because they were able to put what they had learned into practice. 25 Findings from these studies are reflected in this study as the benefits of study abroad are confirmed by measurable progress in the standardized tests together with students' self-reported gains. Students in the AEP are faced with ongoing transformation processes in second language acculturation and adaptation in a host environment which is not without pitfalls, however, students reported positive experiences and achievement in student satisfaction surveys conducted by the program (Berwick and McMichael, 1993). It would, however, be ill-advised to assume that all students studying abroad make the growth in proficiency and experience the positive social interactions found in these studies. 2.5. DISCUSSION OF PROFICIENCY IN ENGLISH AS A SECOND L A N G U A G E Since the pioneer study of Gardner and Lambert (1965), on the role of intelligence in the prediction of second language learning proficiency, the issue of second language proficiency and how it best can be measured has triggered a corpus of empirical work. Ellis' (1990) explanation of proficiency is defined by performance, that is, the scope and skill of the learner's control of the foreign language in all kinds of settings and social interchanges. It is the outer demonstration of the knowledge of the second language internalized by the learner. Other definitions involve the construct level where proficiency is defined as the degree of competence in a foreign language demonstrated by an individual learner at a given point in time (Vollmer, 1983). Vollmer also makes a clear distinction between \"proficiency\" and language \"ability\" . The later notion will be used as a synonym for \"competence \" thus referring to a dynamic construct. It includes aspects of implicit knowledge of rules (defined as consolidations of prior mental experiences and information stored) just as much as basic cognitive operations being more or less consciously carried out. Skills , for examples, are considered to be acquired routines, forming a subsystem within the overall structure of human abilities. They are automatized processes, carried out subconsciously, cognitive or sensory-motor in nature, yet being a definite part of the domain-specific ability system, (p. 5) 26 What is Second Language Proficiency and H o w D o We Measure It? Historical Background: The debate, on one side, between believers that language proficiency can be divided into discrete components and skills that can be measured independently, and, on the other, by those who believe that language competence like Spearman's general factor of intelligence, is unitary and indivisible, has given rise to two mutually exclusive hypotheses both of which may be possible. The \"divisible competence hypothesis\" argues that language proficiency can be divided into separate components (phonology, syntax, lexicon, etc.), and skills (listening, speaking, reading, and writing) or aspects of these, each of which can be measured by discrete point testing. Thus, there are different kinds and degrees of proficiency (competencies) in such areas as BICS, CALP, and illiteracy. The \"unitary competence hypothesis\" asserts contrariwise that the elements of language competence function in similar ways in any language-based task and therefore can be measured using an integrative approach where it is possible to apply Spearman's factor-analytic techniques that test for general language proficiency. An understanding of the fundamental elements of second language proficiency is essential to its testing. Oiler (1978), who had originally claimed in 1976, from his work of language score analyses, that there was a single (general) factor in language proficiency which might be the same central factor that accounts for general intelligence, later stated (1980) that his earlier position had been undermined. Spolsky (1983) concluded that any individual's overall language proficiency might be made up of very different functional abilities and that the high correlations that show up over groups of subjects disguise the enormous potential for individual variation. 27 Present Situation: In what Bachman (1990) calls the \"dilemma of language testing\", language is both the object and the instrument of measurement. We need to understand the nature of language proficiency and the nature of methods used to test this proficiency so that the effects of the testing methods can be attenuated. Douglas and Selinker (1992) have extended their interlanguage work into the field of language testing where they have tried to benefit from the effect of the test method by producing tests which would be useful in both language testing situations and second language acquisition research. They hope to generate information on language ability in specific purpose environments. Bachman (1991) pointed out that we cannot experience language proficiency directly and must infer its existence by observing behaviour that is presumably influenced by this underlying ability. Bachman further extends his definition by distinguishing between two types of language proficiency. He calls one type of language proficiency the \"real-life\" approach, wherein actual language use is identified as being, or not being, characteristic of competent language users (Lowe, 1982 and ACTFL, 1986) and the other type, the \"interactional/ability\" approach, that separates language proficiency into modular abilities. Examples of the \"interactional/ability\" approach are found in: (a) the skills and component frameworks of Lado (1961), and Carroll (1961a); (b) the functional framework of Halliday (1973); (c) the communicative framework of Munby (1978), and Canale and Swain (1980); and (d) the research of Bachman and Palmer (1982a). Work in the area of the \"interactional/ability\" approach is presently focusing on identifying and experimentally verifying the multiple components of language. Canale and Swain (1980) and Canale (1983) used factor analysis and between group comparisons to theorize and concluded that grammatical, discourse, and sociolinguistic competence were distinct components of second language proficiency. With the present focus on communicative language teaching, the 28 search for measures of communicative competence have led to a renewed interest in language testing in the 1990's. BICS, CALP and CUP No examination of the construct of proficiency would be complete without mention of Cummins' (1981,1983) typology of competences: 1. Basic Interpersonal Communicative Proficiency (BICS), which is the mastery of context-embedded use of L2 in undemanding communicative tasks, e.g., social talk. 2. Cognitive Academic Language Proficiency (CALP), the kind of L2 proficiency necessary for successful academic study (e.g. reading this thesis), and 3. Common Underlying Proficiency (CUP), the assumption that the L2 is mitigated through a well-developed LI. Cummins' (1981) \"interdependency principle\" may explain why students who have a high literacy in the first language (LI) may find it easier to develop cognitive academic language proficiency (CALP) in a second language than those lacking such literacy. Cummins further contends that CALP is common across languages and is easily transferred from a first language to a second. The fact that ESL students may acquire basic skills in speaking (BICS) quite quickly can lead evaluators to falsely believe that this skill also exemplifies skills in reading, writing, and cognitive academic language proficiency (CALP), when in reality they may still be functionally illiterate. Even after being immersed in English for several years, both in social and academic situations, immigrant ESL students acquire only limited English proficiency (LEP). It takes many years for ESL students to acquire reading and writing skills in English that reflect the same proficiency they possess in their LI (Carey, 1993; Cummins, 1987). Communicative Competence Whatever the factors of proficiency are, they are intimately linked. The standard measures used by institutions in their attempts to classify and rank candidates for courses in second languages have established this bond. Tests designed to measure different traits such as grammar, textual, illocutionary, and 29 sociolinguistic competence (Bachman, 1991) have long relied on psychometric methods. Language proficiency, whether it is the ability to perform well on written grammar and vocabulary-based tests, or language proficiency measured in terms of oral competence or communicative proficiency is reduced to the assignment of a numerical value to performance. Fortunately, the whole area of language testing is now being studied with a view to designing new tests that more accurately reflect the test-takers communicative language ability. Work in the area of learner-designed tests and tests based on the Knowledge Framework (Mohan, 1986) look most promising. 2.6. FACTORS AFFECTING SECOND L A N G U A G E ACQUISITION \"There are no absolute barriers to second-language acquisition.\" (Bialystok and Hakuta, 1994 p. 207) It is generally assumed that age, gender, language aptitude, autonomy and motivation, attitude, personality, cognitive style, learning strategies, background in language and knowledge of other languages all play a role in second language acquisition. Although these factors do not play a vital part in this study it is important to note that English proficiency, as measured by test scores, is not mediated only by cognitive ability and performance. Gender Second language acquisition studies have reported gender differences in elementary school students, particularly in the domain of listening comprehension tests (Faraday, 1982) and dialect discrimination (Einstein, 1982). In these two areas it was found that female subjects significantly outperformed male subjects. Academic Achievement Reynolds (1991) discussed the question of whether or not there is a correlation 30 between bilingualism and cognitive development and to what extent bilinguality affected cognitive performance. Since bilingualism and standardized tests (both intelligence and academic) are linguistically based, there is an inherent correlation (Carey, 1984). The suggestion that bilinguals have a metacomponential advantage (Peal and Lambert, 1962) points to a causal link between bilingualism and academic (verbal, but not reading and writing) achievement because of greater mental flexibility, superiority in concept formation and more diversified mental abilities. Other studies have linked bilingualism to academic achievement (Albert and others, 1979; Cummins, 1976; Darcy, 1953; Lambert, 1975; Swain and Cummins, 1979). Some researchers have used standardized language measures as predictors of academic achievement but in this and previous studies such tests were found to be poor predictors of course marks (Berwick and McMichael,1993; Carey, 1984) The Age Factor It is presumed that young children learn a second language effortlessly, yet it should be remembered that the amount of \"language\" necessary to be fluent in the social situations of childhood is very limited, regardless of how native-like their accent is. It should also be remembered that children have ample time to learn and play whereas adults are restrained by the demands of daily life. Perhaps if adults could devote the same amount of time to language learning they too would achieve similar \"proficiency\". Adults simply have different strategies for learning language (Carey, 1984). Bialystock and Hakuta (1994) found that this so-called age restriction to second language learning, that is often used to explain adults' lack of achievement, is not supported by evidence and that changes in brain physiology at maturity do not prevent languages from being learned. Language Learning Aptitude Aptitude is one of the general factors that characterize individual learner differences (Ellis, 1994). It has been suggested that people possess a special ability, known as \"language learning ability\" for learning a second language. Such ability is 31 thought to be separate from \"intelligence\", the general ability to master academic skills. Carrol's (1981) definition of aptitude is that it is a concept corresponding to the notion that an individual approaching a particular learning task possess a current state of capacity for learning that task if the individual is motivated and has the opportunity for doing so. The capability is presumed to depend on some combination of more or less enduring characteristics of the individual. The standardized test used to determine language aptitude is the (MLAT) Modern Language Aptitude Test (Carrol and Sapon, 1959). Attitude Toward Target Language Attitudes the language learner has towards the target language and its speakers, as evidenced by seeking out L2 opportunities, impact on the learner's success (Kitao, 1991) by: (a) influencing the amount and type of input the learner receives, (b) altering any guarantee of success in learning the second language, (c) varying that degree of success, (d) accounting for a difference in goals, (e) fossilizing, (f) altering the role instruction plays in the learning process and, (g) modifying the strong influence of first language knowledge and general problem solving systems. Kitao (1991) also examined, with Japanese students, the perceived difference in social status between speakers of the target language and the learner which she felt accounted for a more positive attitude and thus led to success when the learner admired and wanted to become part of the target language culture. Background i n Language Language is a tool (Gardner, 1983) and as with any other tool humans use, greater practice leads to more competent use. The longer a second language is studied in any environment, natural or academic, the more it is practised, and the greater the variety of its use in a number of registers and sociolinguistic settings, the more proficient language learners will become. 32 Knowledge of Other Languages Studies of bilingual education in Canada point to language transfer and interference between first and second languages. French and English share many cognates thus giving learners a definite benefit when it comes to vocabulary learning. Similarly, Japanese learners of Chinese have the tremendous advantage of many commonalities in the two writing systems (i.e., through use of ideographs). Studies by Gass (1979; 1982) and Kellerman (1985a) pointed out the facultative effect of LI, which allows the L2 learner to draw on similar forms and grammatical rules. Contrastive analysis, according to Stockwell, Bowen and Martin (1965), may predict the degree of difficulty learners will have by documenting the similarities and differences between a learner's first language and the target language being studied. The United States Foreign Service Institute has rated the ease with which another language may be learned according to its family grouping. For speakers whose native language belong to the same family group as English, it is one of the easiest languages to learn, whereas, because of its distance from the English family grouping, Japanese is one of the most difficult for English speakers to acquire. Motivation In a series of studies, beginning in the late 1960's, Gardner and Lambert investigated the roles of instrumental and integrative motivation in the acquisition of French as a second language. Results of their work have important implications for the program in this study. At first Gardner and Lambert (1972, 1975) found evidence for the integrative position but this was replaced by other studies (Oiler, 1977; Spolsky, 1989) that demonstrated the superiority of the instrumental position. In a more recent study, Gardner and Maclntyre (1995) found that integrative motivation was not necessarily superior, but that it facilitated successful second language acquisition. 33 Conclusion Although many factors have roles in the acquisition of a second language as shown by the gain in English proficiency in this study, it is important to note that English proficiency, as measured by test scores, is not mediated only by cognitive ability, performance, and attitudes. It is the individual student who sets personal goals and conditions for acquiring the second language. 2.7. S U M M A R Y Many factors and their interactions influence second language acquisition and previous research has focused on one or two variables. What is needed is a comprehensive view which takes into account all these variables and their interactions in order to give a more complete picture of sheltered immersion post-secondary programs such as the AEP. Students learn language through culture, so by trying to increase awareness of Canadian culture, integrative motivation will increase the likelihood of mastering both the culture and the language awareness. By using quantitative and qualitative data and a comparison over two different years, this study attempts to achieve a formative evaluation of the success of the AEP in promoting ESL. In addition, based on the findings in these studies, it is hoped that recommendations can be made to further promote the success of both the cultural and language goals of the AEP. 34 CHAPTER THREE: DESIGN OF THE STUDY 3.1 INTRODUCTION This study used both quantitative and qualitative methods to answer the four specific research questions. Question 1: How much improvement (gain) in English writing, reading, speech and academic achievement occurs during the eight-month stay of students in the University of British Columbia/Ritsumeikan Joint Academic Exchange Programme? Question 2: Is this improvement greater in \"high\" or \"low\" students? What is the difference in gain between groups? Question 3. Is it possible to isolate those students who show the most and the least gain and to determine correlates of both groups? Question 4. Is it possible through quantitative and qualitative analyses to determine the factors and activities that would account for improvement in English as a Second Language? Questions 1 and 2 were examined using quantitative measures, Question 3 by qualitative methods, and Question 4 by a combination of both methods. Quantitative Methods: A quasi-experimental one group design was used with a pretest and posttest where the pretest was the measure used to establish a baseline norm in language proficiency. Psychometric tests employed in Year 4 and Year 5 were the Test of English as a Foreign Language (TOEFL), an Oral Interview (OI), and a Writing Sample. In Year 5 the Comprehensive English Language Test (CELT), and a Cloze test were added to the first three. 35 Qualitative Methods: Year 5 By means of the questionnaire (Appendix 1), students were surveyed to determine whether or not any of those in the \"high\" group had had additional English language training, had been abroad, had close family members who spoke English or had studied other foreign languages. A survey of present language use was given at the same time to establish the amount of English students were using in their daily lives outside the classroom. Three students were interviewed to determine if it was possible to isolate those students who showed the most and the least gain and to extract factors which contributed to their gain in English proficiency. 3.2. RESEARCH SUBJECTS All members of the student body for the two academic years 1994/1995 and 1995/96 in The University of British Columbia/Ritsumeikan Joint Academic Exchange Programme. In the following charts students are grouped by Faculty and year of study. The majority of students in the \"high\" group for both years were females and belonged to the Faculty of Literature which may account, in part, for their progress in English. In Year 4 the second largest group represented was from the Faculty of Social Science, followed by Law and International Relations. In Year 5, the second largest number of students came from the new faculty of Policy Science, followed by Social Science and International Relations. 36 Table 2. Distribution of Students by Faculty, Year, and Gender for Rits 4 (1994/1995) Faculty Total Females Year 2 Year 3 Males Year 2 Year 3 Business Adm. 11 5 5 0 6 2 4 Economics 11 2 1 1 9 6 3 Inf 1 Relations 17 9 9 0 8 7 1 Law 18 9 6 3 9 8 1 Literature/Letters 21 20 19 1 1 1 0 Science 5 1 1 0 4 4 0 Social Science 21 10 8 2 11 11 0 Total 104 56 females and 48 males. Table 3 Distribution of Students by Faculty, Year, and Gender for Rits 5 (1995/1996) Faculty Total Females Year 2 Year 3 Males Year 2 Year Business Adm. 11 3 2 1 8 2 6 Economics 7 3 3 0 4 1 3 Int'l Relations 13 11 11 0 2 1 1 Law 8 4 3 1 4 1 3 Literature 22 19 18 1 3 2 1 Other 5 3 3 0 2 0 2 Policy Science 17 7 7 0 10 10 0 Sc/Engineering 1 0 0 0 1 0 1 Social Science 13 12 10 2 1 0 1 Total 97 62 females and 35 males. \"Low\" and \"High\" Students At the beginning of this study students were categorized on the basis of their in-coming TOEFL scores. It is this score which is the main determinant of their entry into the program. In this study the 25 students whose entry TOEFL scores were the highest and the 25 students whose entry TOEFL scores were the lowest 37 were chosen and designated as \"high\" and \"low\" groups (see Table 1). In the one case where two students had the same TOEFL score, the higher entry oral interview score was used to determine which student was chosen for the high group. This arbitrary selection of groups closely parallels division by quartiles. Students below the 25th percentile (Qi) are \"low\" and those with scores above the 75th percentile (Q_4) were considered \"high\". In Japan, English language instruction begins at the junior high school level and since the questionnaire on the previous language learning experiences of students in the \"high\" group revealed that their instruction time (6 years) and methods (grammar translation) were similar to the \"low\" group, other factors must account for their higher scores on the TOEFL test. Auxiliary factors contributing to language proficiency such as the students' age, gender, language aptitude, background in language and knowledge of other languages will be taken into account in order to give a more complete profile of a language learner in this program. A third group of language learners: the below-average language learners, will not be considered in this study as such learners would be excluded from the program by the selection process. Additional Data With the inclusion of test results for the standardized tests (TOEFL, the Oral Interview and the Writing Sample) for the 104 students of Year 4 (1994/1995) who were also classified as \"high\" and \"low\" on the same basis as in Year 5, more depth and support for the research hypothesis is provided. 3.3. MEASURING INSTRUMENTS There are a number of ways to assess gain in language proficiency. The AEP has established three methods for evaluating students on entrance and exit of the program using standardized tests. In addition, this study employed the 38 Comprehensive English Language Test (CELT) and a Cloze test. Test of English as a Foreign Language The TOEFL is the most widely approved instrument for testing a student's knowledge of academic English, and the most common test for admission to many North American universities, including the University of British Columbia. It was developed in 1964 by Educational Testing Service (1995), (ETS), Princeton, N.J. The TOEFL is a long-standing, highly secure, international, standardized, multiple-choice test. It consists of three sections, each of which is separately timed, uses a test booklet which follows a multiple-choice, four-option format, and has a separate answer sheet. The three sections are Listening Comprehension, which measures the ability of examinees to understand English as spoken in the U.S.; Structure and Written Expression, which measures competence in the structure and grammar of written English; and Reading Comprehension and Vocabulary which tests the ability to understand the meanings and use of words in written English, as well as the ability to understand a variety of written materials. The format used in this study was the institutional TOEFL, which although administered by staff of the AEP, uses materials sent by ETS to the institution for the designated test date. All answer sheets are forwarded to ETS to be scored and the results returned to the institution. The instructions and stimuli for the Listening Comprehension section are on audio tape. This test is unique among commercial and institutional ESL/EFL tests in that it is the most widely researched and certainly the most widely used of all foreign language tests which results in ETS's information and data base being unparalleled. The content validity of TOEFL is the responsibility of a Committee of Examiners composed of linguists and English language pedagogy specialists whose membership is rotated. The validity of the TOEFL test is supported by the fact that it correlates well with other instruments claiming to measure similar abilities and less well with those that do not. There is moderate to high correlation with direct and integrative 39 measures such as cloze tests, oral interviews, and essays which fact was considered in the design of this study (see Table 4). Over the years TOEFL has itself become the criterion for validation of other tests and although it is not considered a predictor of academic achievement it can be included in predictive studies of English proficiency or readiness with positive results (Bachman, 1991). Construct validity studies of TOEFL are limited as the concept of language proficiency has not yet been adequately specified but factor analysis seems to support the belief that TOEFL does measure major interrelated language proficiency areas and has demonstrated that it does measure what it claims to. Any weaknesses found in the TOEFL is a reflection of the weaknesses existing in the general area of language testing as this test is the best of its breed. Oral Interview The Oral Interview is based on the more structured method developed by The American Council on the Teaching of Foreign Languages (ACTFL) for testing and rating the speaking ability of second language learners. Students took part in a twenty-minute oral interview in September and again in April by interviewers trained in the use of ETS's Oral Proficiency Interview Protocol and each performance was rated immediately following the interview using the 6-point ETS scale. The oral interview is, by nature, a criteria-referenced test which is scored holistically, and as such relies on the integrity of the interviewer for its validity. The oral interview, as a test, has a high correlation with the TOEFL and the writing task, so its validity has been presumed more than empirically demonstrated, although the literature on this type of test is both exhaustive and varied. Perhaps the most telling criticism of the oral interview is that it employs artificial level descriptions and forces the interviewee into a highly controlled situation. 40 Writing Sample Students produce a writing sample based on the same set of tasks and questions in September and April. The writing samples were obtained during a thirty-minute period under controlled conditions. Students were given a statement and asked to give their personal response. The essays were scored using a modified version of Jacob's Profile (Jacobs and others, 1981). Comprehensive English Language Test Form S-A Structure of this normed standardized test was used as it is highly comparable with the TOEFL section B. Like the TOEFL, the CELT uses a test booklet which follows a multiple-choice, four option format, and has a separate answer sheet but unlike the TOEFL the test is scored by the testing institution from a punched key. Further, unlike the TOEFL, the CELT is structured to permit test-takers to complete each section within the time allotted, on the assumption that it measures power and not speed. The test reliability coefficients shown in the manual ( Harris and Palmer, 1970) are high ranging from .82 to .97 using the Kuder-Richardson formula. Validity of the CELT test is presumed from its correlation to three other highly regarded tests of English as a second language, the Michigan series and the TOEFL. Correlation with the TOEFL is given as .91 in the CELT test manual. Again, like the TOEFL, the CELT is not considered a predictor of academic achievement but it can be included in predictive studies of English proficiency or readiness with positive results. Cloze Exercise The cloze procedure is both a language testing technique and an instructional activity that involves giving students reading material from which words have been deleted and the students must fill in the blanks with logically fitting words. As a testing devise the cloze exercise can point out areas in which students need more practice and instruction. The biggest advantage of the cloze exercise is that it is flexible enough to be used with an individual student, small groups, or a class. 41 The foremost reason for choosing to include a cloze test in this study was its high correlation with the TOEFL, the oral interview and the writing task (Table 6). 3.4. QUALITATIVE MEASURES Language History Questionnaire In November students answered a language history questionnaire (see Appendix 1) about their personal history of English study and use in Japan and abroad. Information on other languages they may have studied, the length of such study, and proficiency attained, was gathered. Language Use Survey A survey of the participants' language use in the AEP asked for information about the length of time they listened to, read, and spoke English was given the same day as the CELT and Language History Questionnaire. Data obtained from this measure was collated and displayed in graphs. (Appendix 2) Interviews with Selected Students In order to obtain a more complete profile of the two groups of language learners, three students were selected and asked to take part in an interview. Protocol of the interview (see Appendix 3) was to determine, if possible, what activities, motivations, tactics, and strategies the students used to improve and/or maintain their English abilities. Students from both groups were chosen from personal in-class observations of the researcher and these students were interviewed in mid-March when their program was almost over. The interviews were approximately 45 minutes in length and were audio taped. Students answered a group of core questions and reflected on their language learning experiences. 3.5. D A T A COLLECTION Scope In Year 4,103 students participated in both TOEFL tests, 102 in both oral 42 interviews and 78 in both writing samples. In Year 5, 82 students participated in both TOEFL tests, 86 the Personal History Questionnaire and the English Use Survey in November, 1995. The number of participants on whom there was complete data was limited to 81 since some students did not participate in all the outgoing tests. Protocol All standardized test were given under the mandated test conditions. Test instructions for the Oral Interview and the Cloze were considered self-evident and were not explained. Students took all tests either together or within a short period of time for the different groups to prevent disclosure of test items. The writing posttest required students to write for 30 minutes on the same topic as the pretest. Format Incoming TOEFL tests were taken in Japan prior to entering the program. The incoming TOEFL was the highest TOEFL score a student had achieved either in the March or subsequent tests. The pretest Writing Sample and the Oral Interview were done in September during the first week of the program in Orientation Week. Post-tests in Year 4 were administered during the last week of March. Additional data was collected in Year 5 over the eight month period of the academic year. The CELT, the Cloze, the Language History Questionnaire, and Personal Language Use were administered the first week of November after the students had been in the program about eight weeks. Posttests were administered during the months of March and April, 1996: the TOEFL in late March and the other tests just prior to the final exams in April, 1996. 3.6. D A T A ANALYSIS Statistics Packages Data from the progress reports was formatted in a ClarisWorks T M spreadsheet application for transfer to the Systat program (Systat Inc., 1991) for Macintosh. Data obtained from the questionnaires, surveys, and interviews were 43 also put into spreadsheet format and the summarized using ClarisWorks 4.0 (1995) Groups Analysis: Multiple correlations of scores between the two groups of students (\"low\" and \"high\") were done and the results displayed in charts and graphs. Gender Analysis The overall and between-groups comparisons were broken down to reflect the scores of females and males. 3.7. LIMITATIONS OF THE DESIGN By using students in Year 4 (1994-1995 ) and Year 5 (1995-1996) of the AEP, this study is generalizable only to those groups. All students did not participate in all of the tests and some students did not write the final TOEFL, thus data on 103 students was available for analysis in Year 4 and for 81 students in Year 5, which may be problematic as the test scores and characteristics of the missing students might have affected the result. The researcher recognizes that labelling some students as \"low\" and others as \"high\" second language learners, on the basis of their incoming TOEFL scores, may bias the interpretation of results. However, it was necessary to establish some baseline norm in order to be able to compare students' gains during the AEP. In addition, data from the questionnaires, the surveys, and the interviews are susceptible to the personal attitude of the individual student at the time they were completed and on the personal bias of the researcher who had been associated with the AEP for several years. The interpretation of the findings of this study in Chapter 4 were done with acknowledgement of the limitation of the design. 44 CHAPTER FOUR: FINDINGS AND INTERPRETATIONS 4.1. THE RESEARCH QUESTIONS RESTATED At the beginning of this study three research questions were posed. In addition to seeking answers to the research questions it was also the intention of this thesis to determine whether or not a portrait of an \"extraordinary\" language learner could be assembled. The first two questions may be answered by examining the quantitative data generated by the standardized tests used in this study. Question 1: How much improvement (gain) in English writing, reading, speech, and academic achievement occurs during the eight- month stay of students in the University of British Columbia/Ritsumeikan Joint Academic Exchange Programme? Question 2: Is this improvement greater in the \"high\" students? What is the difference in gain between groups? 4.2. QUANTITATIVE FINDINGS The first question to be answered is: How much improvement (gain) in English writing, reading, speech, and academic achievement occurs during the eight-month stay of students in the AEP? The following sections deal with the three standard tests used by the program. Results of the Test of English as a Foreign Language (TOEFL), the Oral Interview, and the Composition or Writing test all yield information on the students' abilities and progress. Statistical comparison of scores was conducted by paired-samples T-Tests. An alpha level of .01 was used for all statistical tests and the mean is shown as M . _A11 figures show mean differences in proportion. 45 Quantitative Findings for Year 4 - 1994/1995 Table 4 gives the results of tests for Year 4 and shows the minimum and maximum raw scores on entry and exit, together with range, mean, variance and standard deviation. Table 4 Table of Year 4 Test Results TOEFL 1 TOEFL 2 ORAL 1 ORAL 2 WRITING 1 WRITI1 N OF CASES 104 104 104 103 103 79 MINIMUM 4 8 0 . 0 0 0 4 7 7 . 0 0 0 0 . 9 0 0 1 . 0 0 0 1 . 0 0 0 1 . 0 0 0 MAXIMUM 5 9 0 . 0 0 0 6 0 3 . 0 0 0 3 . 9 0 0 4 . 0 0 0 4 . 0 0 0 5 . 0 0 0 RANGE 1 1 0 . 0 0 0 1 2 6 . 0 0 0 3 . 0 0 0 3 . 0 0 0 3 . 0 0 0 4 . 0 0 0 MEAN 5 2 7 . 8 9 4 5 3 7 . 3 7 5 1 . 7 0 4 2 . 9 2 8 2 . 3 1 1 3 . 2 7 8 VARIANCE 6 3 5 . 2 0 2 6 4 1 . 4 0 2 0 . 5 7 0 0 . 4 8 2 0 . 4 5 2 0 . 8 1 9 STANDARD DEV 2 5 . 2 0 3 2 5 . 3 2 6 0 . 7 5 5 0 . 6 9 5 0 . 6 7 2 0 . 9 0 5 TOTAL OBSERVATIONS:104 Figure 1 represents the overall progress by means for all students in the three standardized tests in Year 4. Results are shown in mean proportion and are separated by gender. All paired samples T-Tests are significant. Results for All Students The overall mean difference (n=103) for the TOEFL, the Oral Interview and the Writing test were: M = 10.419, (i= 5.686, p_ < .01); M = 1-227, (t = 15.424, p_ < .01); and U = 0.962, (t = 8.729, p_ < .01) respectively, with the greatest increase in the Oral Interview, the measure of speaking ability. When the results of all students are displayed by gender (Figure 1), the similarity in the TOEFL means for females and males, both on exit and entry, is apparent. The mean difference for all the females (n = 54) was M = 8.648, (t = 3.490, p_< .01) and for all males (n = 49) M = 11.490,{t= 3.863, p.< .01). 46 YEAR 4 - PROGRESS FOR ALL STUDENTS M 0 . 9 7 I Females (n=54) H Males (n=49) Figure 1. Progress of all Students in Year 4 (1994/1995) Results for the Oral Interview (Figure 1) show a significant difference which indicates real gain overall. The mean difference for females was M = 1.4, (t_= 11.864, p_ <.01) and for males, M = 1.041, (t = 10.395,_p_ <.01). In the Writing Sample (Figure 1) test, the results of the paired-samples T-Test showed that the females had a greater mean difference than the males. The females had M = 1-128, (Jt = 9.382, p_< .01) and males M = 0.710, (t = 3.493, p_ < .01). This difference between exit and entry means was significant and represents real gains for all students. \"High\" and \"Low\" Groups The progress of those students who were assigned to the \"low\" and \"high\" groups at the beginning of the study is shown in Figure 2. Not all paired samples T-Tests were significant. 47 The mean difference of the paired samples T-Test for the \"high\" group (n=25) for TOEFL scores was not significant; M = 0.120, (1 = -0.038, p_ = 0.970) while the mean difference for the low group (n=25) was significant, representing a very substantial gain at M =21.080 (t = 6.313, p_ < .01), when compared to the \"high\" group. Thus the group that started with the higher entry TOEFL scores, showed no gain on exit while the \"low\" group made significant gains. The paired samples T-Test of TOEFL scores for the two groups by gender indicated that the females in the \"low\" group (n=ll) had a significant mean difference, M = 16.909 (t_ = 3.032, p_ = .013), while the difference for females in the high group (n=12) was a not significant at M= -1.417 (t_ =-0.301, p_ = 0.769), indicating no real gains. Figure 2 graphs the slight gain in TOEFL scores for the \"low\" females and the loss for the \"high\" females. The males (n = 13) in the \"high\" group had a mean difference of M = 1-077 (t = 0.243, p_ = 0.812), not significant, while the males in the \"low\" group (n = 14) had a mean difference of M = 24.357 (L= 6.073, p_ <.01), significant. The \"low\" males increased their mean TOEFL scores and the \"high\" males did not. The gains of both genders in the \"low\" group, as shown in Figure 2, for the TOEFL are significantly more than those of the \"high\" group. Males in the \"low\" group started lower than the females in that group but did better on exit, while males in the high group started higher and remained at the same level compared to the females who showed a slight loss. Figures 2 shows real gains between the exit and entry Oral Interview scores for both groups. The paired samples T-Test mean difference was M =1.227 (t = 15.424, p_ <.01) for the \"high\" group and M = 1 448, (t = 8.674, p_ < .01) for the \"low\" group, both statistically significant. 49 Gender differences for the Oral Interview tests of both groups gave the advantage to the females, with females (n = 12) in the \"high\" group having a mean difference of M =1.408 (t =6.473, p_ <.01) and those in the \"low\" groups ( n = 11) a mean difference of M =1.882 (t = 6.852, p_ <.01), indicating real gains for females in both groups. The males in the \"high\" groups (n = 13) had a mean difference of M = 0.800 (t = 3.592, p_ = 0.004), not a statistically significant difference, and the \"low\" group ( n = 14) a mean significant difference of M =1.107 (t = 6.868, p_ <.01), which indicated real gains. Figure 2 shows that the \"low\" females started higher than the \"high\" females but while they made significant gains ended lower. Whereas, the \"low\" males started lower and ended lower. Results of the paired samples T-test for the Writing Sample by groups are a difference of M = 1.333,( t = 5.739, p_ < .01) for the \"high\" group and an M = 1.048, (t = 4.690, p_ < .01) for the \"low\", both statistically significant, indicating real gains for both groups. Gender differences were also apparent in the between-groups Writing Sample scores. Females in the \"high\" group had a mean difference of M =1333, (t = 4.000, p_ <.01) compared to the females in the \"low\" group at M = 1-4, (t = 5.250, p_ < .01), both mean differences were statistically significant and indicated real gains. Males in the \"high\" group had a mean difference of M = 1.333, (t = 4.000, p_ = .010), a significant difference but males in the \"low\" group had a mean difference of M =0.727, (t = 2.185, p_ = 0.054), not indicative of real gains. Figure 2 shows that the \"high\" females started higher and finished higher than the other three groups. Summary for Year 4 The paired samples T- Tests show significant mean differences between entry and exit means for all students in Year 4 (Figure 1) with all females showing greater progress in the Oral Interviews and Writing Samples than the males. The males showed slightly greater progress in the TOEFL. 50 The \"high\" group did not show significant mean differences for the TOEFL whereas, the \"low\" group did. The \"high\" group had statistically significant differences for the Oral Interview and the Writing Sample, indicative of real gains. Paired samples T-Tests for the \"low\" group were significant on all tests, proof of real gains over the course of the AEP. \"Low\" group females had significant mean differences on all tests, but \"low\" males had non-significant differences on the Writing Sample. Males in the \"low\" group gained more than the \"low\" females on the TOEFL but females in that group made greater gains in the Oral Interview and Writing Samples than did the males. Mean differences by gender in the \"high\" group were not significant for the TOEFL tests and indicated no real gains. The females in the \"high\" group lost ground in the TOEFL, although they made greater gains in the Oral Interviews and Writing Samples when compared to the males in the \"high\" group. The highest gains were noted in the Oral Interviews (BICS) which is consistent with the fact that students had been listening and speaking English in the interval between the entry and exit tests. The least gain was made in the Writing Sample and is probably due to the length of time it takes to acquire this academic skill (CALP). 51 Quantitative Findings for Year 5 - 1995/1996 Table 5 Summary of Test Scores for Year 5 TOEFL 1 TOEFL 2 ORAL 1 ORAL 2 WRITING 1 WRITING N OF CASES 97 84 97 85 97 81 MINIMUM 4 7 7 . 0 0 0 4 7 3 . 0 0 0 1 . 0 0 0 1 . 9 0 0 2 . 000 0 . 0 0 0 MAXIMUM 6 0 7 . 0 0 0 6 1 7 . 0 0 0 4 . 0 0 0 4 . 9 0 0 8 . 300 8 . 0 0 0 RANGE 1 3 0 . 0 0 0 1 4 4 . 0 0 0 3 . 0 0 0 3 . 0 0 0 6 . 300 8 . 0 0 0 MEAN 5 1 9 . 2 2 7 5 3 8 . 2 6 2 2 . 4 8 8 3 . 2 3 6 5 . 089 6 . 4 3 5 VARIANCE 6 9 3 . 6 7 7 7 6 7 . 5 9 3 0 . 3 5 1 0 . 3 5 6 1 . 916 1 . 3 6 5 STANDARD DEV 2 6 . 3 3 8 2 7 . 7 0 5 0 . 5 9 3 0 . 5 9 7 1 . 384 1 . 1 6 8 CELT 1 CELT 2 CLOZE 1 CLOZE 2 N OF CASES 86 83 86 93 MINIMUM 3 1 . 0 0 0 4 3 . 0 0 0 3 . 0 0 0 2 . 0 0 0 MAXIMUM 7 5 . 0 0 0 6 8 . 0 0 0 2 6 . 0 0 0 26 . 0 0 0 RANGE 4 4 . 0 0 0 2 5 . 0 0 0 2 3 . 0 0 0 24 . 0 0 0 MEAN 5 6 . 4 3 0 5 7 . 5 3 0 1 4 . 1 0 5 12 . 3 6 6 VARIANCE 4 1 . 6 3 6 3 1 . 5 9 4 1 7 . 9 7 7 20 . 8 6 5 STANDARD DEV 6 . 4 5 3 5 . 6 2 1 4 . 2 4 0 4 . 5 6 8 TOTAL OBSERVATIONS: 97 Results for All Students The paired-samples T-Tests of all students were significant for the TOEFL, M = 17.405, (t = 7.438, p_ < .01); the Oral Interview, M = .722, (t = 9.643, p_ < .01); the Writing Sample, M = 1.353, (t = 7.309, p_ < .01); and the Cloze, M = -1.345, (t = -2.651, p_ < .01) but not for the CELT, M = 1014, Q = 1.500, p_ = .138). The entry level raw TOEFL scores (Table 5) for all students ranged from 477 to 607 (TOEFL 1) and the from 473 to 617 (TOEFL 2). a drop in the minimum and an increase in the maximum. The range of Oral Interview scores increased one full level on the rating scale (minimum and maximum) between entry and exit. The Writing Sample ranges of 2.0 to 8.3 on entry fell on exit to 0.0 to 8.0. The CELT minimum exit range started higher but the maximum ended lower than the entry maximum. Entry and exit minimums and maximums for the Cloze were similar. 52 53 Figure 3 represents the overall progress by means for all students in the five standardized tests administered in Year 5. Results are shown in mean proportion and are separated by gender. Significant differences for the TOEFL, Oral Interview, Writing Sample and the Cloze are shown. The TOEFL paired samples T-Test for all students by gender, as illustrated in Figure 3, was M = 15.518, (t = 6.254, p_ < .01) for the females and M = 19.393, (t = 4.072, p_ < .01) for the males. Results for the Oral Interview for all students as shown Figure 3, reflect results of the paired samples T-Test which was M = 0.707, (t = 8.490, p_ < .01) for the females, and M = 0.664, (_t = 4.843, p_ < .01) for the males. Both differences are statistically significant indicating real gains. In the Writing Sample test, females had an M = 1-698, (t = 7.960, p. < .01), significant and males an M = 0.711, (t = 2.097, p_ = .046), not significant. This indicates real gain for the females but not for the males overall. Paired samples T- Tests for the CELT test were a M = 0.938, (t = 1.189, p_ = .240) for the females and M = 1.769, (t = 1.448, p_ = .160) for the males, with neither gender showing statistical significance. Figure 3 reflect a slight gain with the greater gain for the males. Paired-samples T- Test for the Cloze tests were not significant and did not reflect gains. In fact, as shown in Figure 3, both genders lost ground on Cloze 2. The females had an M_ = -1.245, (t = -1.805, p_ = .077) and the males an M = -2.308, (t = -2.316, p_ = .029). \"High\" and \"Low Groups The progress of those students who were assigned to the \"low\" and \"high\" groups is shown in Figures 4 and 5. Figure 4 shows the progress for the TOEFL, the Oral Interview and the Writing Samples and Figure 5 the two additional tests in Year 5, the CELT and the Cloze. 54 6\u00C2\u00A3 The findings of the paired samples T-Tests in the case of the \"high\" group were not significant at M = 6.000, (t = 1.531, p_ = .139) but those of the \"low group were at M = 23.60, (t = 5.697, p_ <.01), and an indication of real gain by the \"low\" group. The group that started high showed no gains, but the groups that started lower did. Paired-samples T-Tests on TOEFL by gender within groups show that the females in the \"low\" group had an M = 21.588, (t = 5.495, p_ < .01), a significant difference, while the females in the \"high\" groups had a non-significant difference at M = 3.941, (_t = .835, p_ = .416). The males in the \"low\" group had a n M = 27.875, (t_= 2.719, p_ = .03) while the males in the \"high\" group had an M = 10.875, Q = 1.744, p_ = .125), not statistically significant differences. Figure 4 shows real gains between entry and exit scores on the Oral Interviews for both groups. The paired samples T-Test mean difference was statistically significant at M = -849, (t = 4.806, p < .01) for the \"high group and M = -671, (t = 4.862, p_ < .01) for the \"low\". The paired-samples T-Tests for females in the \"high\" group was M = -740, (t = 3.821, p_ < .01) and for females in the \"low\" group was M = -588, (t = 3.932, p_ < .01), both significant indicating real gains. Males in the \"high\" group had a n M = 1.081, Q = 2.868, p_ = .029), and males in the \"low7' group M = .838, (_t = 2.855, p_ = .025), neither significant differences. Males in both groups did not have statistically significant gains but the practical significance are shown in Figure 4. The paired-samples T-Test for the two groups on the Writing Sample show significant differences, indicating real gains. The \"high\" group had an M_ = 1550, (t = 3.881, p_ <.01) and the \"low\" group an M = L082, (t = 3.719, p_ < .01). Gender differences for the Writing Sample test were similar to those on the Oral Interview. The females had statistically significant differences at M_ = 1-667, (t = 3.061, p_ < .01) for the \"high\" females and M = 1487, (t = 4.051, p_ < .01) for the \"low\" females. The paired samples T- Test for males in both groups were not significant at M = 1.317, (t = 2.379, p_ = .063) for the \"high\" males and M = -357, Q = 1.355, p_ = .224) for the \"low\" males. 57 Figure 4 shows that there were gains for the tests but when the paired samples T- Test was applied some of the mean differences were not significant. The results shown in Figure 5 indicate differences between the pretest and the posttests of the CELT and the Cloze. The differences in entry and exit means give an indication that there was overall gain for both groups for the CELT but not for the Cloze. Paired-samples T-Tests for the \"high\" group resulted in an M = 4.000, (1= 2.538, p_ = .023) and M = 1-810, (_t = 1.585, p_ = .129) for the \"low\" group, both not significant. Comparison of these groups by gender yielded the same non-significant findings. The Cloze tests showed similar findings, with the paired-samples T- Tests for the \"high\" group and at M = -0.438, (t = -0.389, p_ = .703) and M = -1-762, (t = -1.925, p_ = .69) for the \"low\" group, and no significant differences between genders. Summary for Year 5. Progress was made in all tests with the exception of the Cloze. The Oral Interview and the Writing Samples showed the most gains and overall there was very little difference in gains between the females and the males with the exception of the Writing Sample, where the females made greater gains than the males. The CELT gains were too small to be considered significant and the Cloze appears to be a loss, rather than a gain. 58 Comparison of Year 4 and Year 5 Figure 6 illustrates that there was very little difference in the TOEFL scores between Year 4 and Year 5 for all students. This similarity is also reflected when the comparison of TOEFL scores between groups was made as shown in Figure 7. Entry and exit means between gender for all students and for the \"low\" and \"high\" groups again points up the close proximity of the TOEFL scores for the two years. An investigation of what would account for the similarity in the TOEFL scores is worth consideration. One of the suggestions is that the test conditions for the ITP TOEFL given at UBC are very different from those the students are accustomed to in Japan. Other factors to be considered are the ceiling effect for those students with high entry level TOEFL and the practice effect of writing several ITP tests over a short period of time in Term 1 then having a 4-month lacuna until the final TOEFL in March. The entry level scores for all students in the Oral Interview are higher for Year 5 but are below the exit scores for Year 4. The exit scores in Year 5 do not show as much gain as Year 4 and this may be due to the higher entry scores. The greatest difference between years is in the Writing Samples where the entry mean of all students in Year 5 is higher than the exit mean for Year 4 and the exit mean for Year 5 shows that more gains were made in Year 5 by all students. 59 Between years comparison by groups is similar to the comparison of all students for the Oral Interview where the entry level in Year 5 of both groups is above that of Year 4 but is not above the exit level for Year 4. Again, the exit levels by groups show that the \"high\" group entered higher and remained higher, while the \"low\" group showed gains that were greater in comparison to the \"high\" group. The means for the Writing Samples show that the two groups have close mean scores for both years but that greater progress was made in Year 5 by both groups. 4.2.1. Reliability and validity of the test instruments Test of English as a Foreign Language (TOEFL) test: \"Reliability is the consistency of measures across times, test forms, raters, and other characteristics of the measurement context\" (Bachman, 1991). The reliability of the TOEFL depends on whether or not the difference is between two tests taken separately or one test taken on two different occasions. The Test and Score Manual published by ETS (1995), states that the difference between two test scores has a lower rate of reliability due to the accumulation of errors of measurement that occur in each of the tests and that the common aspects of language proficiency measured in the two tests are cancelled out in the difference score. Hence, the reliability of the different scores decreases as the correlation between pretest and posttest increases. Swinton (1983) found that TOEFL score gains decrease as a function of proficiency level at the time of initial testing. Validity by definition is according to Bachman, (1991) the most important quality of test interpretation. It is the extent to which any inferences or decisions which are made on the basis of test scores are meaningful, appropriate and useful. In other words does the test actually measure what it is supposed to measure or as Messick (1987) put it \"validity refers to the usefulness of inferences made from test scores\". Evidence of validation for the TOEFL includes: 1. Content-related evidence which demonstrates that the content of the test 62 displayed and the behaviour derived from the test constituted an appropriate sample of the content and behaviours of the subject matter being tested. ETS has identified aspects of English communication, ability, and proficiency that are to be tested and the appropriate techniques for testing them. One of the problems inherent in the TOEFL is the American cultural content that is present in the test, but Angoff (1980) established that there were no detected cultural advantages for those students who had resided in the United States for more than one year. 2. Criterion-related evidence of validity is the relationship between a score on one test and the score on some other test variable, a criterion. TOEFL has been compared with other measures of English proficiency such as the Michigan Test of English Language Proficiency, other language institute tests, and teacher ratings and studies show a high correlation between the two. Henning and Cascallar (1992) found evidence for the criterion-related validity of the TOEFL by relating performance to other independent ratings of oral and written communicative language ability in controlled academic settings. 3. Construct-related validity refers to the adequacy of the test to measure the constructs for the purposes of generalizing and replicating. Evidence for the construct validity of the TOEFL is found in a series of studies available from the ETS. In studies which focus on the relationship between the TOEFL and other popular aptitude tests, findings show the extent to which the test has integrity as a measure of proficiency in English. Oral Interview: In the present controversy over language proficiency testing one of the most challenging aspects is communicative testing and the construction of practical, reliable, and valid tests of oral proficiency. The oral interview used in the program is based on the U.S. Foreign Service Institute's Oral Proficiency Interview where performance is judged by the interviewer who uses a detailed checklist of criteria to rate the testee. The very nature of this form of test which is an artificial closed 63 system with many possible differences in setting and interviewers, simply points out that the reliability and validity of the oral interview cannot be demonstrated because it confounds abilities with elicitation procedures in its design, and provides only a single subjective rating (Bachman, 1988). Since July, 1995 ETS has been administering a revised edition of the Test of Spoken English (TSE) which is a more semi-direct test using audio-recording equipment and prerecorded prompts and printed test books. Instead of speaking to an interviewer, test takers for the TSE are speaking to a recording device which may be a little disconcerting for some students. Current research is outlining theoretical assumptions for speaking ability and providing guidelines for test construction so that new ways of testing communicative competence may appear shortly. Writing Sample: As stated above the reliability of a test is the extent to which is gives consistent results across different forms, different administrations and different scorers. The reliability of the Writing Sample test can be evidenced by interrater reliability which is the extent to which different readers agree on the rating given to each essay. Validity of the Writing Sample may be construct-related or content-related. Construct validity is evidenced by the relationship between this test and the TOEFL (see Table 5 and 6). Content-related validity is evidenced by the types of writing tasks required. By testing the ability to compose in standard written English using writing tasks similar to those required by colleges and universities, the Writing Sample test evidences content-related validity. Comprehensive English Language Test This test developed by Harris and Palmer (1970) closely resembles the original TOEFL test (a normed, multiple-choice, three-part test which measures listening, structure and vocabulary) and was designed for high school, college and adult 64 programs. The test manual gives reliability coefficients and standard errors of measurement in the .90+ range. The test authors further state that the test correlates with the comparable TOEFL section at .83 although this was not found in the correlations shown in Table 6. The same version of the test was administered in both the pretest and posttest as it was felt that no practice effect would interfere and in fact examination of scores showed that those who did well on the pretest did not do as well on the posttest. Cloze Bachman (1982, 1985) has demonstrated construct validity of the Cloze as a test in specific contexts and he further found that personal attributes affected this test. It was found that there is a high correlation between cognitive styles and performance on cloze tests. In Table 6 the best correlation is between the Cloze is with TOEFL 2 and the CELT test. The passage used for this test was given for both the pretest and the posttest again because it was felt that the length of time between the test was sufficient to diminish any practice effects. As with the CELT it was found that those who did well on the pretest did not necessarily do well on the posttest. Table 6 The correlation of tests used in this study for Year 4 PEARSON CORRELATION MATRIX TOEFL 1 TOEFL 2 ORAL 1 ORAL 2 WRITING 1 WRITING 2 TOEFL 1 TOEFL 2 ORAL 1 ORAL 2 1 . 0 0 0 0 . 6 6 8 0 . 3 1 1 0 . 2 5 6 0 . 2 7 1 0 . 2 6 6 1 . 0 0 0 0 . 3 2 7 0 . 1 7 6 0 . 2 6 0 0 . 3 0 6 1 . 0 0 0 0 . 2 9 6 0 . 0 9 0 0 . 1 1 7 1 . 0 0 0 0 . 1 5 4 0 . 1 7 4 WRITING 1 WRITING 2 1 . 0 0 0 0 . 2 4 5 1 . 0 0 0 NUMBER OF OBSERVATIONS: 77 65 Table 7 The correlation of tests used in this study for Year 5 PEARSON CORRELATION MATRIX TOEFL 1 TOEFL 2 ORAL 1 ORAL 2 WRITING 1 WRITING TOEFL 1 1 . 0 0 0 TOEFL 2 0 . 6 6 1 1 . 0 0 0 ORAL 1 0 . 1 4 4 0 . 1 7 3 1 . 0 0 0 ORAL 2 0 . 1 9 5 0 . 1 9 2 0 . 3 4 7 1 . 0 0 0 WRITING 1 0 . 1 0 1 0 . 0 8 2 0 . 1 9 6 0 . 1 2 7 1 . 0 0 0 WRITING 2 0 . 2 7 5 0 . 2 1 2 0 . 1 5 0 0 . 2 5 3 0 . 2 62 1 . 0 0 0 CELT 1 0 . 4 3 2 0 . 3 6 7 0 . 0 3 3 0 . 0 0 0 0 . 0 4 2 0 . 2 9 0 CELT 2 0 . 5 3 6 0 . 3 6 5 - 0 . 0 4 7 0 . 1 4 8 0 . 0 4 0 0 . 1 9 7 CLOZE 1 0 . 2 1 9 0 . 2 7 2 0 . 2 5 2 0 . 1 0 4 0 . 2 2 3 0 . 2 2 6 CLOZE 2 0 . 2 8 8 0 . 4 9 4 0 . 2 0 7 0 . 0 5 9 0 . 0 3 8 0 . 1 5 9 CELT 1 CELT 2 CLOZE 1 CLOZE 2 CELT 1 1 . 0 0 0 CELT 2 0 . 4 8 9 1 . 0 0 0 CLOZE 1 0 . 3 1 8 0 . 2 1 6 1 . 0 0 0 CLOZE 2 0 . 1 2 3 0 . 1 8 8 0 . 2 7 4 1 . 0 0 0 NUMBER OF OBSERVATIONS: 58 4.2.2. Interpretation of Findings There seems to be no ready explanation as to why the \"high\" group failed to gain on the TOEFL test in Year 4. It may be possible that the length of time between the last two TOEFL's during which students did not participate in TOEFL preparation has some influence on the scores. Other factors to be considered is the ceiling effect for the \"high\" group and the physical conditions of the test itself which differ from conditions students are accustomed to in Japan. The are noticeable differences between the two year for the entry and exit means of the Oral Interviews. Year 5 is clearly higher than Year 4 for all students and for both groups. 66 The difference between Year 4 and 5 for the Writing Sample (Figure 6) is striking, particularly the increase in the entry means between Year 4 and 5. One of the reasons that would account for this is the pre-departure orientation given in Japan to students coming into Year 5. Students attended an academic writing class and computer orientation class for six weeks prior to arriving in Canada. Results for the two additional test in Year 5, the CELT and the CLOZE seem to indicate a strong correlation between CELT and TOEFL, probably because of the similar construction of the two tests. What is unusual about these two tests is the almost identical scores of many students on the pretest and the posttest. 4.2.3. Summary of Quantitative Findings In reviewing the tests results it was noted: Year 4. The \"high\" group who had entered the program with the higher TOEFL scores remained highest overall but the \"low\" group made greater progress over the course of the academic year and the females in each group consistently made greater progress than the males. The fact that the \"high\" group lost ground on the TOEFL in Year 4 may be an anomaly and certainly bears further investigation. The Oral Interview means showed significant improvement as did the Writing Samples. Year 5. The \"high\" group remained higher but the \"low\" group made greater gains on all tests. TOEFL scores remained alike with the females starting higher and ending higher than the males. The same sharp rise in the Oral Interview and the Writing Samples as noted for Year 4 also occurred in Year 5 but the gains for the Writing Sample were dramatically greater on exit. The CELT and Cloze pretest and posttests were remarkably similar. 67 Comparison of Year 4 and Year 5. TOEFL levels for both years remained fairly constant with no significant gains. The Oral Interviews entry means for all students in Year 5 were significantly greater than those of Year 4. The results of the Writing Sample were dramatically different between Year 4 and Year 5 as the entry means in Year 5 were even higher than the exit mean for Year 4. 4.3 QUALITATIVE FINDINGS Question 3. Is it possible to isolate those students who show the most and the least gain and to determine correlates of both groups? Question 4. Is it possible through quantitative and qualitative analyses to determine the factors and activities that would account for improvement in English as a Second Language? Personal Language History Questionnaire: In November students in Year 5 were asked to complete the Personal Language History Questionnaire (Appendix 1). The purpose of the questionnaire was twofold: (a) to see if the students in the two groups had divergent language histories, and (b) to determine if possible what attributes the more proficient language learners may share. A greater percentage (30%) of the \"high\" group began the study of English in elementary school compared to the \"low\" group at 19.7%. The percentage of students in the \"high\" group who had studied English for more than 8 years was 55% compared to the \"low\" group at 40.9%. A higher percentage ( 50%) of students in the \"high\" group had taken private English lessons for an average of 2.4 years compared to the \"low\" group at 37.9% and 1.5 years. Thirty percent of the \"high\" group had family members who spoke English compared to 24.2 % of the \"low\" 68 group. Twenty percent of the \"high\" group also spoke English at home to friends, visitors or other family members compared to 16.7 % of the \"low\" group. More (55%) of the \"high\" group had been abroad and 25% of them had studied English abroad compared to 51.5% of the \"low\" group of whom 10.6% had studied English abroad. Results indicated that the majority of students (80%) had studied a second foreign language, some even several, and that they rated their proficiency in the second foreign language as either poor or fair. Present Language Use: The results of the Present Language Use Survey also conducted during the second week of November are shown in Appendix 2. Both groups were alike in their use of TV, radio and magazines. The \"low\" group spent more time listening to music and reading newspapers. At the time of the survey the \"high\" group spent an average of 7 hours per week talking with people whose first language was English compared to 1 hour for the \"low\" group. Case Histories: In March interviews were conducted with three of the students in the program, two from the \"low\" group ( one male and one female) and one (female) from the \"high\" group. Prior to the interviews the three students completed language charts to indicate native or other languages, age, use and time spent learning English. The interview lasted approximately 30 minutes and was conducted very informally and audio-taped. Case 1. Chiaki was a 20 year old, second year female student majoring in Literature, her entry level TOEFL score was 483 which placed her in the \"low\" group. Chiaki began to study English at age 12 when she entered junior high. She took 3-50 minute classes of English per week at age 12, increasing to 4 - 50 minute class per 69 week at age 13 and 14. In senior high school at ages 15 and 16 she took 6-50 minute classes of English per week and at age 17 in her last year of high school took 8-50 minute classes per week. After entering university at age 18, Chiaki took 3-90 minute classes of English per week for the first year which increased to 5 classes per week, at age 19. Also at age 18, Chiaki began to study French for 2-90 minute classes per week and felt that English helped her study French because of the similarity of words. If one considers the average school year to be approximately 45 weeks in Japan, then Chiaki had, by the time she entered the exchange program, about 1600 hours of English study. She did not take private English lessons, study English abroad and no other member of her family speaks English. After entering the program she spent an average of 5 hours per week conversing in English with native speakers and she became very active in the social and extra curricular activities of the program. In the interview Chiaki talked about the English lesson she took. She stated that in junior and senior high school the teachers were Japanese and that the teaching was done from a textbook (New Horizons). Students were required to read the text and memorize words. In senior high school, the lessons were separated into grammar and literature components. In university, Chiaki had one Japanese teacher and two native English speaking teachers. The Japanese teacher gave the students an English text which they read and translated into Japanese and then made class presentations on the different sections. Thus, as she stated she \"studied English in Japanese\". The classes Chiaki took with native English speakers were more communicative, including the exchange of personal histories and daily events together with poetry readings. Chiaki felt that the easiest way to learn English was to go to an English-speaking country, like Canada and take part in immersion courses. When asked about her reasons for coming to Canada she replied that when she was in her third year of high school she had gone to Okinawa to take part in a week-long home stay 70 program with an American family stationed at the US base there and because she couldn't communicate well and felt so frustrated, she decided that she really wanted to learn English. She believes that English will be useful to her when she looks for a job and she thinks that because English is \"so common in business\" that it is necessary. Because conversation was not part of her high school training in English, Chiaki found the greatest challenge in the program to be the discussion classes. But she realized the necessity of joining in even if she knew her English was not good because she saw it as one way to improve. She found the \"lecture classes\" the least useful as they were \"so hard\" especially when she had to take notes. She stated that some instructors combined discussions with their lectures which she liked. However, in classes with instructors who only lectured, she found that she could not understand what was taking place and that it took five or six lectures before she had any idea of what was going on. Even after she had became accustomed to the instructor, she still wondered if she was understood everything. Chiaki found that having to think critically was challenging because she was unaccustomed to do so because of the transmission style of teaching in Japan. Because the assignments were so interesting, she enjoyed doing them even if she had trouble keyboarding on the computer. She found that her listening skills improved to the point where she did not have to translate what she heard into Japanese in order to understand and that she could reply in the same manner. However, when she was writing assignments she thought and worked in Japanese and then translated her work into English. Although, most of her friends were Japanese, Chiaki indicated that she frequently spoke English with her Canadian roommates and attended many activities and events where she spoke English with native speakers. Over the course of the program as her English improved she found it easier to \"get out\" of her room and participate in activities with her roommates. 71 When asked what her experience learning English had taught her about learning another language she replied that the best way to learn another language was to talk, \"just find someone and talk to them\". She had attended the fluency workshops held in the first term and felt that pronunciation practice by computer software programs would be helpful as she could work on her own. Test scores for this student showed dramatic increases for the TOEFL (483 to 540) and the Writing Sample (4.3 to 6.7) with a lesser increase in the Oral Interview (2 to 2.9) and the CELT (57 to 59). There was a drop in her Cloze (16 to 14). Case 2. Aki was a 21 year old, third year student who started to study English at age 13 in junior high school, taking approximately 6 hours of classes per week during junior high and 9 hours of classes in senior high. In addition he went to an English conversation school for another 3 hours per week. In university he began to study French for 2 hours per week and learned Chinese and Korean from his friends in conversation only. In his first year of university Aki took English classes for 3 hours per week, in second year, 5 hours per week, and in third year ten hours per week. He also attended an English conversation school (Eikaiwa) \"during long vacation\" for about 6 hours per day. It would appear that Aki had accumulated some 2000 hours of English by the time he arrived in Vancouver. But as he put it \"could not speak\". When he entered university, he actively sought out native English speakers and volunteered to help with foreign visitors on campus in order to talk to them. This is how he also practised Chinese and Korean. In high school he was not interested in languages and it was not until his experienced communication problems with foreign students that he realized that while he could manage to read and write English, he couldn't \"speak and listen\" so he started going to an English conversation school and began to speak English every day. 72 Aki's main reason for coming to Canada was not to study English but to study, research, and experience associations in an intercultural milieu. He approached his study of comparative cultures by designing and administering questionnaires to his friends and associates to learn all he could about multiculturalism in Canada. He also learned how to do library and internet search for material on his topic and spent his vacation time in the lab working. When he returns to Japan he intends to take \"unofficial lectures\" (audit) in English. His problem with English, as he sees it, is his lack of listening skills. When asked what he felt he could do to improve he said just practice. He is still unable to take notes at English lectures but he can remember what he hears and sometimes underlines things in an article or text. During the first term he did not have the confidence to speak English and had several bad experiences with native speakers not wanting to talk to him in his dorm. When he realized that people lost interest in what he was saying if he spoke slowly or hesitated, Aid developed some strategies for making his conversation more fluid. Aki did not like to work on group projects because he felt that he inability to speak fluently in English caused other group members to switch to Japanese for expediency. When he arrived in Vancouver he had resolved to speak only English but the stress became too much for him, so that he now speaks Japanese to his cohorts outside of class and English to his other friends. The teaching approach he felt benefited him the most was the lab work where he could listen to a video and then write his opinion. He stated that this gave him listening practice as well as practice in formulating and writing his personal opinions. The reading load for his four classes was too much and took too much of his time because he felt his language skills were worse than other people. He used a Japanese-English dictionary when reading which he realized slowed him down but he felt he could not guess words in context because of his lack of vocabulary. 73 Aki's entry level TOEFL was 493 and since he did not write the exit TOEFL there is no indication of his progress for this test. His entry and exit Oral Interview scores (2.9 to 3.0), Writing Sample (6.3 to 6.7) CELT (56 to 57) and the Cloze (12 to 12). Because he was so busy socializing and conducting his own research for his graduating paper in Japan, Aki missed many classes and failed the core courses in the program. The question of whether or not this student was successful despite failing to obtain credits at UBC could be debated in view of his approach and obvious success in improving his listening and speaking abilities. Cage 3 Naomi was a third year student in Law who belonged to the \"high\" group by virtue of her entry level TOEFL score of 587, the second highest entry score for Year 5. Like the other two students in the case studies Naomi started the study of English in junior high school at age 12 with 3 hours of classes per week which she supplemented by listening to the radio (US Armed Forces radio in Japan) for one and a half hours at home. At age 15 she started going to an English cram school (\"juku\") for 2 hours per week in addition to her regular 3 hours of English lessons at school and listening to English radio. At age 16 when she entered high school her English lessons increased to 5 hours per week. At 17 she went to the US as an exchange student where she had the good fortune of being in Home Stay with retired teachers who tutored her for several hours each day. (She was also the only Japanese student in her classes.) The benefit of these extra hours and attention is obvious, for Naomi speaks with hardly a trace of an accent and has an excellent vocabulary. When she was 17 she returned to Japan and experienced some difficulty \"after not speaking Japanese for several years\". Her English skills were sufficient to get her a job teaching at an English language school in Japan where she used English for about 25 hours per week in addition to her regular habit of listening to English radio for 20 minutes a day. When she started university at age 21 she continued to 74 work for the English school for 10 hours per week in addition to taking 3-1 hour English classes. She stated that she spent half her day speaking English in the AEP. With a total estimated time of three to four thousand hours spent acquiring English, Naomi's test scores reflect her interest and motivation. Her entry level TOEFL was 587 well above the level of 570 required by UBC for undergraduate study and her exit score was 607, a substantial increase at this high level. Her Oral Interviews (4 to 4.9), Writing Sample (6.7 to 8), CELT (62 to 65) and the Cloze (19 to 22) all reflect her dedication to learning English as it is difficult in these high ranges to increase scores due to the ceiling effect. Naomi stated that she chose to study at Ritsumeikan University because of this exchange program and that she feels she was one of those students who really benefited from the AEP. 4.3.1. Interpretation of Qualitative Findings The Personal Language History Questionnaire (see Appendix 1) gives an indication that the members of the \"high\" group started the study of English at an earlier age, studied longer, took private English lessons, had a higher percentage of family members who spoke English, and spoke English at home. This group also had more students who had been abroad, lived abroad and had studied English abroad. It may then be presumed that this group had more exposure to English which may have accounted for their higher entry TOEFL scores. The Personal Language Use Survey ( see Appendix 2) for the \"high\" group also showed that they interacted more with native English speakers when they arrived in Vancouver. This may reflect the fact that they were able to converse competently and felt comfortable speaking in English. 4.3.2. Summary of Qualitative Findings The \"high\" group entered their year abroad with higher scores in the standardized tests and better English skills which reflected the amount of time and the circumstances under which they had studied or been exposed to English prior to 75 entering the program. The \"low\" group may have spent less time studying English prior to coming to Canada but once they were here may, like Aki and Chiaki, have become more involved in social activities and made a more determined effort to immerse themselves in English by interacting with their Canadian roommates, going to movies and social events on campus. One unique strategy, revealed in an Oral Interview, by a \"low\" student was that she had made a pact with her Japanese roommate to speak only English to each other while in the AEP. Perhaps it is this kind of motivation that accounts for the gains of the \"low\" group. Members of the \"high\" group seemed to be the kind of students, who like Naomi, focused on the academic side of the program and participated in courses with Canadian students. 4.4. INTERPRETATION OF RESULTS While it would appear that the \"low\" group made significantly higher gains in proficiency than the \"high\" group, the results should be interpreted with caution. It cannot be clearly determined whether the gain reflect increased language proficiency, a practice effect, regression toward the mean, or some combination . It should also be kept in mind that the lower the entry score, the greater the gain over a period of instruction. Factors which also affect gains are the type of instruction, the instructors, the time on instruction, the test conditions, and factors affecting the studenf s performance, such a homesickness, stress and test anxiety. The difference between results for year 4 and year 5 are very striking. Clearly the students coming into Year 5 were better prepared academically by the introduction of the pre-departure orientation in Japan prior to Year 5. Other possible explanation for the differences between Year 4 and 5 would be the different dynamics of the two cohorts, a higher percentage of students from the Faculties of Literature/Letters and International Relations, and the natural selection process 76 which encourages the best candidates to apply. Perhaps these findings also answer the question of how to define an \"extraordinary\" language learner. 4.5. SYNOPSIS The original hypothesis for this thesis was that there would be no significant difference in the gain of English proficiency between \"low\" and \"high\" students in this sheltered immersion program. Results of the standardized tests show statistical differences between the two groups of students. The \"low\" group made more overall gain than did the \"high\" group so the null hypothesis must be rejected. 77 CHAPTER FIVE: DISCUSSION AND IMPLICATIONS 5.1. INTRODUCTION The broad range of data presented in this study was intended to give as complete a report on the gain in English proficiency as possible during two academic years of the University of British Columbia/Ritsumeikan Joint Academic Exchange Programme. By using both quantitative and qualitative methods to answer the research questions it was determined that there was no significant difference in the gain in English proficiency, as evidenced by gain scores, between students who were designated \"low\" and those who were designated \"high\" during their study abroad for the period of an academic year in a sheltered immersion program in an English milieu. Students who had scored high or low on the pretests showed statistical regression to the mean on the posttests. 5.2. DISCUSSION One of the conclusions to arise from this study was that all female students generally did significantly better on the standardized tests and achieved higher overall ratings than did their male counterparts. However, results for the two groups in this study did not continue to support the findings. A number of other studies had found that females were better at learning a second language than males but this would not appear to be the case here. Of further consideration is the fact that the majority of students in both Years 4 and 5 belonged to those faculties in Japan that either support or require the use of English. The TOEFL mean scores for all students remained very similar in Years 4 and 5 and this stability of the TOEFL scores appears to be statistically true also for Year 1 and Year 2 as shown in the Program Summary Reports (1991/1991; 1992/1993). One of the inconsistencies noted was that while the TOEFL scores for Year 4 were comparable with others years for all students, the breakdown by groups and gender 78 produced a very different picture. Both the \"high\" females and males in Year 4 dropped in their mean scores, with the greatest drop experienced by the females. However, the rest of the students made sufficient gains to produce overall gains . In Year 5 both groups made statistically significant gains in the TOEFL with more gains by the \"low\" group. It could be speculated that a contributing factor to this was the listening practice the \"low\" students experienced during the course of the program. Another special feature of Year 5 was the gender mix of the students (67 females and 35 males) a ratio of almost two to one. Overall gender differences were almost non-existent in the TOEFL scores with the males leading slightly in Year 4 and the females leading in Year 5 by a slight margin. When the progress of the two groups is compared, the TOEFL scores remained similar for Year 4 and Year 5. The \"high\" males did slightly better than the \"high\" females in Year 4 with the reverse in Year 5. The \"low\" males did better in both Year 4 and Year 5 than the \"low\" females. There were significant differences between years for the entry and exit means of the Oral Interview, this is not surprising when one considers that Japanese students have had very little practice \"speaking\" English during their lessons in Japan. Means for all female students indicated that while they started slightly lower they finished higher in the Oral Interviews in Year 4, but in Year 5 they started higher and finished at the same mean as the males, despite a smaller number of males in the program. Females in the \"high\" group started lower but did better than their male counterparts in the Oral Interview in Year 4 with the situation reversed in Year 5. The \"low\" females started higher and did significantly better than the \"low\" males in Year 4 and similar to the \"high\" group the situation was reversed in Year 5. One possible explanation may be that the females in Year 4 were more sociable and sought out opportunities to speak with native English speakers. By receiving comprehensible input in a sheltered environment through content taught only in English it is not surprising that students showed significant 79 overall gains in their oral English proficiency over the course of an academic year. Writing Sample results were very different for the two years, due to higher entry and exit means in Year 5 than in Year 4. Writing Samples results for Year 4 showed that the \"high\" females started higher than the \"high\" males, yet both genders ended the same, whereas, in Year 5 the \"low\" males had the highest entry mean with the \"high\" females the highest exit mean. Males in the \"low\" group did better than the females in that group on all tests in Year 4. In Year 5 the \"low\" males again did significantly better than the \"low\" females. In Year 5, the \"low\" group did better on the entry CELT than the \"high\" group and the \"low\" males had the highest exit mean, although significant gains were made by both groups. Both the \"high\" and \"low\" females showed a loss on the exit Cloze test whereas, the \"high\" and \"low\" males showed gains. The gain in the Writing Sample in Year 5 may be attributable to the introduction of a pre-departure program in writing in Japan for students coming to the AEP. It is also possible that the increase in the use of computers by students influenced the writing scores. In fact, a number of students came with their own PowerBooks and were accustomed to keyboarding and using editing tools. The lack of significant gain in the TOEFL and Writing Samples by all groups, probably corresponds with the length of time it takes to acquire academic skills (CALP). It is possible that the \"low\" students were able to improve their communicative skills through the listening and speaking practice provided by the program and environment, while the \"high\" oral students who entered with good BICS were able to make some gains in their CALP. 5.3. IMPLICATIONS As the analysis of the quantitative data indicated, the \"low\" students made the most gains and the \"high\" group the least. It should be pointed out that what is 80 being perceived as the \"most\" and the \"least\" gain are in fact two different aspects of second language learning. The \"low\" group improved their communicative skills (BICS) while the \"high\" group made progress in their cognitive academic language proficiency (CALP), however, BICS are easier to acquire and usually precede CALP. If the \"low\" students were able to make such progress in their BICS then what could be done to help them make similar progress in their academic skills in English? There is no doubt that the students in the program are, for the most part, very motivated to acquire as much English as possible during their stay and have discovered their own unique methods for such improvement. Question 3. Is it possible to isolate those students who show the most and the least gain and to determine correlates of both groups?This question suggests an examination of the screening procedure used to select students for the program. By selecting students on the basis of their TOEFL scores alone, which has not proved to be a good predictor of a student's success in the AEP, the patterns of the previous years are being repeated. Psychometric measures are not good predictors of second language potential. In fact, as Gardner (1983,1992) suggests, language is just one facet of multiple intelligences and the tests presently being used to measure second language ability or proficiency are measures of performance, not knowledge or potential. Other screening and selection procedures done in Japan, such as interviews and self-reported degrees of international awareness and motivation, might be more appropriate for selecting students who would do well in this program. 5.3.1. Implications for Instruction Question 4. Is it possible through quantitative and qualitative analyses to determine the factors and activities that would account for improvement in English as a Second Language? Results have shown that the \"low\" TOEFL students in the program have trouble with their academic work due to the lack of oral and written comprehension 81 and oral and written production. They seem to translate what they read, appear to lack vocabulary, and the confidence to \"guess\" at the meaning of words from context. Any support for the improvement of CALP's, in addition to the specifically designed program courses, that would help students read faster and better would be beneficial. The full potential of computers in second language acquisition has not been realized in the Rits multimedia lab. Software can provide the drill and practice necessary for students to increase their vocabulary, reading comprehension and speed, and critical on-line reading courses would enable students to become autonomous in the self-involvement which is essential for any growth in their CALFs . 5.3.2. Implications for Further Research In Year 5 the students entered the program better prepared than those of previous years and most likely will continue to do so in the future. Research into what causes this improved preparation would shed new light on student selection and pre-program preparation. The in-house measures of students' language proficiency such as the Oral Interview and the Writing Sample would appear to be more discriminating measures of proficiency than the TOEFL and may be better predictors of success in the academic components of the AEP. Further research in this area, together with a norming of these tests would be useful so that computation and analysis of the data available from previous years could be consolidated to formulate a complete profile of the program. It is suggested that the formative evaluation in this study be continued as a longitudinal study in order to provide a monitoring of the progress and development of the AEP in which the RitsLab is a critical component. 82 References American Council on the Teaching of Foreign Languages (1986). AC'l'FL oral proficiency interview. WA: ACTFL. Albert, M . L. & Obler, L. L. (1979). The bilingual brain . New York: Academic Press. Angoff, W. A. (1980). Context bias in the test of English as a foreign language. (TOEFL Research Report 29). Princeton, NJ: Educational Testing Service. Bachman, L. (1991). Fundamental considerations in language testing. Oxford University Press. Bachman, L. F. & Palmer, A. S. (1982a). The construct validation of the FSI oral interview. Language Learning, 31,1: 67-86. Bamford, K. & Mizokawa, D. T. (1991). Additive-bilingual (immersion) education: cognitive and language development. Language Learning 41 (3), 413-429 Berwick, R. & McMichael, M . (1992). Findings related to course marks, proficiency test results and student questionnaires for the 1991-1992 LANE-Ritsumeikan Program. Language Education Department, University of British Columbia. Berwick, R. & McMichael, M . (1993). Findings related to course marks, proficiency test results and student questionnaires for the 1992-1993 LANE-Ritsumeikan Program. Language Education Department, University of British Columbia. Berwick, R. & Ross, S. (1993). Cross-cultural pragmatics in Oral Proficiency Interview strategies. Paper presented at the Annual Language Testing Research Colloquium, Cambridge, England, and Arnhem, The Netherlands. Bialystok, E. & Hakuta, K. (1994). In other words: the science and psychology of second language acquisition. BasicBooks: New York. Brecht, R. D . , Davidson, D. E. & Ginsberg, R. (1991). On evaluating language proficiency gain in study abroad environments (A preliminary analysis of data. In Zita D. Dabars (Ed.) Selected Papers Delivered at the N E H Symposium in Russian Language and Culture (Bryn Mawr, PA, May 1990). Brecht, R. D. and Others. (1993). Predictors of foreign language gain during study abroad. National Foreign Language Center Occasional Papers. 1993. Canale, M . (1983). From comunicative competence to language pedagogy. In J. Richards & J. Schmidt (Eds.) Language and communication. London: Longman. 8 3 Canale, M . & Swain, M. (1980). Theoretical basis of communicative approaches to second langauge teaching and testing. Applied Linguistics, 7,133-46. Carey, S.T. & Cummins, J. (1983). Achievement, behavioral correlates and teachers' perception of Francophone and Anglophone immersion students. Alberta Tournal of Educational Research. 29 (3), 159-67. Carey, S. T. & Cummins, J. (1984). Communication skills in immersion programs. The Alberta Tournal of Educational Research, 30, (4), 270-283. Carey, S. T. (1984). Reflections on a decade of French immersion. The Canadian Modern Language Review, 41 (2), 246-259. Carey, S.T. (1991). The culture of literacy in majority and minority language schools. The Canadian Modern Language Review 47 5, 950-975. Carey, S.T. (1993). Only in Canada you say - pity!: On the generalizability of French immersion results to English immersion in other countries. In L. Stewin (Ed.), Contemporary educational issues in Canada. Toronto: Copp Clark. Carey, S.T. (1993). The threshold and interdependence concepts in minority and second language immersion programs. Tournal of Canadian Association of Applied Linguistics, 15 (2), 23-36. Carey, S. T. (1995). English immersion on the playground and French immersion in the class: which produces more rapid second langauge acquisition? Proceedings from the conference on second language acquisiton York Univeristy, Glendon Campus. June. Carroll, J. B. (1961a). Fundamental considerations in testing language proficiency of foreign students. In Testing the English proficiency of foreign students. Washington, DC: Center for Applied Lingusitics: 30-40. Carroll, J. (1981). Twenty-five years of research on foreign language aptitude. In Diller, K. (Ed.) Individual differences and universals in language learning aptitude. Rowley, MA. Newbury House. Carrol J. & Sapon, S. (1959). Modern Language Aptitude Test. New York, NY: The Psychological Corporation. Cohen, A. D. (1993). SLA theory and pedagogy: some research issues. In Second Language Acquisiton Theory and Pedagogy. Eckman, F.R., Highland, D . , Lee, P. W . , Milcham, J . , Weber, R.R. (Eds.). Lawrence Erlbaum Associates, Publishers. N.J. 84 Cohen, A.D. & Swain, M . (1979). Bilingual education: the iummersion model in the North American context. In Sociolinguistic Aspects of Language Learning and Teaching. Price, J.B. (Ed.). Oxford: Oxford University Press. Cummins, J. (1976). The influence of bilingualism on cognitive growth: a synthesis of research findings and explanatory hypotheses. Working Papers on Bilingualism, 9, 1-43. Cummins, J. (1979). Linguistic interdependence and the eductional development of bilingual children. Review of Educational Research 49, 222-51. Cummins, J. (1981). Bilingualism and minority children. Ontario: Ontario Institute for Studies in Education. Cummins, J. (1983). Language proficiency and academic acheivement. In Oiler, J. W. (Ed.) Issues in Langauge Testing Research. Rowley, Mass.: Newbury House. Cummins, J. (1988). Second langauge acquisition within bilingual education programs. In Beebe, L. (ed.). Issues in Second Language Acquisition: Multiple Perspective. New York: Newbury House. Cummins, J. (1991). Conversational and academic language proficiency in bilingual contexts. AILA Review 8, 75-89. Cziko, G.A. (1978). Differences in first and second langauge readings: the use of syntactic, semantic, and discourse constraints. Canadian Modern Language Review 39, 473-89. Cziko, G. A. (1982). Improving the psychometric, criterion-references, and practical qualities of integrative language tests. TESOL Quarterly, 16 Darcy, N . T. (1953). A review of the literature on the effects of bilingualism upon the measure of intelligence. Tournal of Genetic Psychology , 118, 235-44. de Moissac, J. (1990) Reading Comprehension, Stress, Motivation in University Bilingual Students. Unpublished Master's Thesis. University of Alberta, Edmonton, Alberta. Douglas, D. & Selinker, L. (1992). Analyzing Oral Proficiency Test performance in general and specific purpose contexts. System, 20, 317-328. Dube, L.L. (1991). Les cours encadres a l'Universite d'Ottawa: Quelles lecpns peut-on entirer pour l'immersion? Canadian Modern Language Review, 28, 64-76. 85 Educational Testing Service, (1995). TOEFL Test and score manual. Eductional Testing Service. Princeton: NJ. Educational Testing Service Edwards, H . , Wesche, M . , Krashen, S., Clement, R. & Kruidenier, B. (1984). Second language acquisition through subject-matter learning: a study of sheltered psychology classes at the University of Ottawa. The Canadian Modern Language Review, 41, 268-282. Eisenstein, M . (1982). A study of social variation in adult second language acquisition. Language Learning , 32, 367-92. Ellis, J. (1993). Japanese students abroad: relating language ability in class and in the community. Thought-Currents in English Literature, 66 , 45-82. Ellis, R. (1990). Individual learning styles in classroom second langauge development. In de Jong and Stevenson (Eds.) Individualizing the assessment of language ability. Clevedon, Avon: Multilingual Matters. Ellis, R. (1994). The study of second language acquisition. Oxford University Press. Farhady, H . (1982). Measure of language proficiency from the learner's perspective. TESOL Quarterly, 16, 43-59. Gardner, H . (1983). Frames of Mind: The Theory of Multiple Intelligences. Basic Books, Inc. New York. Gardner, R. C. & Lambert, W. E. (1965). Language aptitude, intelligence, and second language learning. Tournal of Educational Psychology 56, 191-199. Gardner, R. C. & Lambert, W. E. (1972). Attitudes and motivations in second language learning. Rowley, Mass.: Newbury House. Gardner, R. C , & Maclntyre, P. D. (1995). An instrumental motivation in language study: who says it isn't effective. In H. D. Brown and S. Gonzo (Eds.) Readings on second language acquisition. Prentice Hall Regents. Gass, S. M . (1979). Language transfer and universal grammatical relations. Langauge Learning 29, 327-44. Gass, S. M . (1982). From theory to practice, In M . Hines & W. Rutherford (Eds.), On TESOL '81 pp. 129-39. Washington, DC: Teac hers of English to Speakers of Other Languages. Gass, S. M . & Selinker, L. (1994). Second language acquisition: an introductory course. Hillsdale, NJ: Lawrence Erlbaum Associates. 86 Genesee, F. (1984). French immersion programs. In S. Shapson & V. D'Oyley (Eds.) Bilingual and Multicultural Education: Canadian Perspectives. Clevedon: Multilingual Matters. Genesee, F. (1987). Learning Through Two Languages. Rowley, MA: Newbury House. Genesee, F. (1991). Second language learning in school settings: lessons from immersion. In Reynolds, A.G. (Ed.). (1991). Bilingualism, Multiculturalism and Second Language Learning. The McGill Conference in Honour of Wallace E. Lambert. Montreal: Lawrence Earlbaum Associates, Inc. Genesee, F. & Lambert, W. E. (1983). Trilingual education for majority language children. Child Development, 54, 105-14. Ginsberg, R. B. (1992). Language gains during study abroad: analysis of the ACTR data. National Foreign Language Center Working Papers. Ginsberg, R. B. and Others. (1992). Listening comprehension before and after study abroad. National Foreign Language Center Report. Halliday, M . (1976). Explorations in the functions of langauge. London: Edward Arnold. Hammerly, H . (1987). The immerison approach: litmus test of second langauge acquisition through classroom communication. Modern Language Tournal 71:395-401. Hammerly, H . (1989). French immersion: myths and reality. Calgary, Alberta: Detselig Enterprises. Harris, D.P. & Palmer, L.A. (1970). A comprehensive English language test for speakers of English as a second language. New York: McGraw-Hill Book Company. Henning, G. (1992). The ACTFL Oral Proficiency Interview: validity evidence. System, 20, 365-372. Henning, G. and Cascallar, E.A. (1992). A preliminary study of the nature of communicative competence. (TOEFL Research Report 36). Princeton, NJ: Educational Testing Service. Jacobs, H.L. & Others, (1981). Testing ESI composition: a practical approach. Rowley, Mass: Newbury House. Kellerman, E. (1985a). If a first you do succeed... In S. Gass & C. Madden (Eds.) Input in second language acquisition. Rowley, Mass.: Newbury House. 87 Kitao, Kathleen. (1991). Variations in success in acquiring a second language: two approaches. Annual Report of Studies, 42, 245-60. Kitao, Kathleen. (1993). Preparation for and result of short-term overseas study program in The United States. Bulletin of the Institute for Interdisciplinary Studies of Culture, 10, 107-118. Koike, I. (1994). New Directions of the Governmental Policy of Foreign Language Teaching in Japan. Paper presented at the International Conference on Applied Linguistics. 1994. Krashen, S. (1981). Second langauge acquisition and second language learning. Oxford: Pergamon. Krashen, S. (1985). The input hypothesis: issues and implications. London: Longman. Lado, R. (1961). Langauge testing: the construction and use of foreign language tests: a teacher's book. London: Longmans. Lambert, W. E. (1974). Culture and language as factors in learning and education. In F.E. Aboud & R.D. Meade (Eds.) Cultural factors in learning. Bellingham: Western Washington State College. Lambert, W. E. (1975). Culture and language as factors in learning and education. In A. Wolfgang (Ed.) Education and immigrant students. Toronto; Ontario Institute for Studies in Education. Lambert, W.E. & Tucker, G.R. (1972). Bilingual education of children: the St. Lambert experiment. Rowley, Mass: Newbury House. Larsen-Freeman, D. & Long, M . (1991). An introduction to second language acquisition research. (Applied linguistics and language study). London and New York: Longman. Long, M . H . (1990). The least a second language acquisition theory needs to explain. TESOL Quarterly, 24, 4, 649-666. Lowe, P. (1982). ILR (Handbook of Oral Interview Testing. Washington, DC: DLI/LS Oral Interview Project. Ministry of Education, Science and Culture (1992). Government of Japan. Monbusho 1992 . Tokyo, Japan. Mohan, B.A. (1986). Language and content. Reading, MA: Addison-Wesley Publishing Company. 88 Munby, J. (1978). Communicative syllabus design. Cambridge: Cambridge University Press. Oiler, J. W. & Perkins, K. (1978). Intelligence and language proficiency as sources of varients in self-reported affective variables . Language Learning , 28, 85-97. Oiler, J.W. & Perkins, K. (Eds.) (1978). Language in education: testing the tests. Rowley, Mass.: Newbury House. Oiler, J. & Hinofotis, F.B. (1980). Two mutually exclusive hypotheses about second langauge ability: indivisible and partly divisible competence. In J.w. Oiler & K. Perkins, (Eds.) Research in language testing. Rowley, Mass.: Newbury House. Opper, S., Teichler, U . , Carlson, J. (1990). Impacts of study abroad programmes on students and graduates. Higher education policy series: 11, v 2. European Cultural Foundation. London. Peal, E. & Lambert, W.E. (1962). The relation of bilingualism to intelligence. Psychological Monographs, 76. Reynolds, A. G. (1991). The cognitive consequences of bilingualism. In Reynolds, A.G. (Ed.). (1991). Bilingualism, Multiculturalism and Second Language Learning. The McGill Conference in Honour of Wallace E. Lambert. Montreal: Lawrence Earlbaum Associates, Inc. Ritsumeikan University, Kyoto, Japan English Calendar. Rits 1994-1995. Roseberry, A. S., Warren, B., & Connant, F.R. (1992). Appropriating scientific discourse: findings from language minority classrooms. Tournal of the Learning Sciences 2 (1), 61-94. Shapson, S. 1984). Bilingual and multicultural education in Canada. In Shapson, S. & D'Oyley, (eds.) Bilingual and multicultural education: Canadian perspectives. Multilingual Matters 15. Stockwell, R., Bowen, J . , and Martin, J. (1965). The grammatical structures of English and Spanish. Chicago: Chicago University Press. Swain, M . (1985). Communicative competence: some roles of comprehensible input and comprhensible output in its development. In S. Gass & C. Madden (Eds.) Input in second language acquisition. Rowley, Mass: Newbury House. Swain, M . & Cummins, J. (1979). Bilingualism, cognitive functioning and education. Language Teaching and Linguistics Abstracts , 12, 4-18. Swain, M . & Lapkin, S. (1982). Evaluating bilingual education: a Canadian case study. Clevedon: Multilingual Matters. 89 Swinton, S.S. (1983). A manul for assessing language growth in instrucitonal settings. TOEFL Research Report 14. Princeton, NJ: Educational Testing Service. UBC/Ritsumeikan Academic Exchange Programme Student Handbook (1994). Revised ed. Programme Office. Veguez, R. (1984). The Oral Proficiency Interview and the junior year abroad: some unexpected results. Paper presented at the Annual Northeast Conference on the Teaching of Foreign Languages, New York, NY. Volmer, H . (1983). The structure of foreign language competence. In Hughes, A. & Porter, D. (Eds.). Current developments in language testing. London: Academic Press. 90 Appendix 1 English Language Personal History Questionnaire 1. When did you start to study English? a) elementary school b) junior high school c) other please specify . Began to Study English Low Females (n=13) High Females (n=15) Low Males (n=9) High Males (n=8) Elementary School B Junior High Figure 1.1 91 2. How many years have you studied English? a) 6 or less b) 6 to 8 years c) more than 8 years Number of Years English Studied H 6 to 8 I more than 8 Figure 1.2 3. Have you ever taken private English lessons? If so, for how long? (months) Extra English Lessons % 100-9 0 -o F 8 0 -7 0 -S 6 0 -T 5 0 -U 4 0 -D 3 0 -E 2 0 -N 1 0 -T 0 -S I Low Females (n=13)High Females (n=15) Low Males (n=9) High Males (n=8) Yes No EH No Response Figure 1.3 92 4. Does anyone else in your family speak English? a) yes b) no If yes please specify Family Members Speak English % 90 o 8 0 F 7 0 S T U D E N T S 40 30 0 + 87.5 73.3 61 .5 55.6 30 7 2 6 | 33.3 12.5 1 M . ~ ] Low Females (n=13) High Females (n=15) Low Males (n=9) High Males (n=8) Yes No [ No Response Figure 1.4 5. Do you ever speak English at home? a)yes b) no If yes, when, on what occassion? % 90 o 8 0 F 7 0 60 S 50 U D E N T S 30H 20 1 0 0 Speaks English in Japan 87.5 80 77.8 69.2 23.2 20 I 1 11.1; 12.5 Low Females (n=13) High Females (n=15) Low Males (n=9) High Males (n=8) I Yes No I No Response Figure 1.5 93 6. Have you ever been abroad before coming to the UBC/Rits Programme?? a) yes b) no If yes for how long? (months) and to what country (countries)? Have Been Abroad % 7 0 n O 60 F 50H S 40-] J j 3 C H D 2 0 E N 10 T 0 + 38.5 53.9 B \u00E2\u0080\u00A2 \u00E2\u0080\u00A2 \u00E2\u0080\u00A2 \u00E2\u0080\u00A2 \u00E2\u0080\u00A2 Hi 44.4 45.5 40 g||ff 37.5 62.5 W&L HHI \u00E2\u0080\u00A2 H H I :X::;:;::::X :: ::;:;:;: Low Females (n=13)High Females (n=15) Low Males (n=9) High Males (n=8) Yes No HI No Response Figure 1.6 7. Have you ever studied English abroad? a) yes b) no If yes for how long? (months). Studied English Abroad o f S t u d e n t s 100 90 80 70 60 50 40 10 0 100 84.8 73.3 66.7 26.7 6 I, Low Females (n=13)High Females (n=15) Low Males (n=9) High Males (n=8) Yes No I No Response Figure 1.7 94 8. Apart from Japanese and English have you studied another language(s) ? a) yes b) no If yes please specify Studied Another Language % 100-9 0 -o F 8 0 -7 0 -S 6 0 -T 5 0 -U 4 0 -D 3 0 -E 2 0 -N 1 0 -T o - -S I MHHMH BHHI \u00E2\u0080\u00A2\u00E2\u0080\u00A2iii \u00E2\u0080\u00A2Bll IHMI Low Females (n=13)High Females (n=15) Low Males (n=9) High Males (n=8) U Yes I No H No Response Figure 1.8 N U M B E R O F S T U D E N T S Studied More Than 2 Other Languages 1 5 1 4 13 12 1 10 9 8 7 6 5 4 3 2 1 0 13 1 2 9 7 2 z \u00E2\u0080\u0094 Low Females (n=13) High Females (n=15) Low Males (n=9) High Males (n=8) Yes No Figure 1.9 95 Other Languages Studied Chinese Korean Tibetan Low Females (n=13) 1 High Females (n=15) L\"H Low Males (n=9) High Males (n=8) Figure 1.10 9. If you answered yes to # 8 please provide details: a) how long have you studied this language? b) how would you describe your proficiency? 1 2 3 4 5 (with 5 as very good) 96 Appendix 2 Present Language Use: 1. Approximately how many hours do you spend watching English language television each week? 2. Approximately how many hours per week do you spend listening to English language radio? 3. How many hours per week do you listen to English music? 4. How many hours to you spend each week reading English language magazines? 5. How many hours to you spend each week reading English language newspapers? 6. How many hours do you spend each week speaking English with people whose first language is English? 7. Do you ever speak English with other Japanese students outside of class? a) yes b) no 97 "@en . "Thesis/Dissertation"@en . "1996-05"@en . "10.14288/1.0054899"@en . "eng"@en . "Curriculum Studies"@en . "Vancouver : University of British Columbia Library"@en . "University of British Columbia"@en . "For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use."@en . "Graduate"@en . "A between groups comparison of gains in English proficiency in a sheltered English immersion program"@en . "Text"@en . "http://hdl.handle.net/2429/4466"@en .