"Education, Faculty of"@en . "Language and Literacy Education (LLED), Department of"@en . "DSpace"@en . "UBCV"@en . "Rothschild, Denise Terry"@en . "2011-02-17T21:56:03Z"@en . "1991"@en . "Master of Arts - MA"@en . "University of British Columbia"@en . "Research in written composition in first language (L1) has undergone a major paradigm shift from interest in product to interest in processes experienced by writers as they compose. Changes in instructional approaches have begun to follow: in many L1 classrooms a variety of process or workshop approaches to the teaching of writing have been implemented. Second language (12) composing research and instruction are also undergoing a similar paradigm shift\u00E2\u0080\u0094 with some reservations about the value of implementing a process or workshop approach in the second language classroom. The question now being asked is, \"How effective are the various process/workshop approaches in the 12 classroom situation?\"\r\nThe current study, building upon mother-tongue research as well as the mainly case study research which provided the foundation of the English as a second language (ESL) literature on composing, examines the effects of a process or workshop approach on the writing performance of adult English as a second language learners. In addition, the study investigates certain biodemographic variables such as first language, and an affective variable, attitudes toward writing, all of which were hypothesized to interact with the treatment.\r\nThis study is a controlled experiment in which the treatment consisted of instruction in writing using a workshop format. Two pre- and posttest measures-informal (classroom conditions) and formal (test conditions) writing tests-were used to ascertain writing growth. On each test overall scores were analysed as well as two sub scores, one for content and organization, and one for structure and mechanics. In addition, a pre-instruction background survey was given to elicit information on seven biodemographic variables, and a post-instruction survey on attitudes toward writing was administered. \r\nResults were mixed. For writing quality, only results obtained on the formal (test-like) measure were significant or near significant in favor of the treatment, the workshop approach. Of the biodemographic variables, only length of time in an English-speaking environment could be interpreted because of a cell distribution problem: it may be that those students with less than two years in a second language environment benefit more from the workshop approach than students with more time and experience in their adopted culture. Regarding attitudes toward writing, the workshop group showed significantly more positive attitudes than the product group. In addition, the content of responses to an open-ended question about writing revealed differences between the two conditions. The workshop students' comments showed awareness of (1) writing as communication and (2) writing as a process requiring time for the development, revision, and editing of ideas and language. \r\nThese findings indicate that this variety of workshop approach may offer a viable alternative to product-oriented instruction. The formal (test conditions) measure suggests that the workshop may be of benefit in helping students improve their writing, particularly the content and organization aspects. Results from the attitude survey imply that students in the writing workshop are receptive to this approach and that they exhibit more positive attitudes toward writing than do students in the product group. If attitude is indeed the key to improved motivation and performance, as many suggest, these results have important implications for the L2 classroom."@en . "https://circle.library.ubc.ca/rest/handle/2429/31462?expand=metadata"@en . "THE ADULT ENGLISH AS A SECOND LANGUAGE WRITER AND THE WRITING WORKSHOP APPROACH: PERFORMANCE, BIODEMOGRAPHIC VARIABLES, AND ATTITUDES by Denise Terry Rothschild Baccalaureat es arts specialise en frangais, Universite Laval, 1973 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS ~- ' n THE FACULTY OF GRADUATE STUDIES Department of Language Education We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA March, 1991 \u00C2\u00A9 Denise Terry Rothschild, 1991 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of Lan c\ u^^e. Educcj-i&n. i The University of British Columbia Vancouver, Canada Date Arpr 17 H I DE-6 (2/88) ii A B S T R A C T Research in written composition in first language (L1) has undergone a major paradigm shift from interest in product to interest in processes experienced by writers as they compose. Changes in instructional approaches have begun to follow: in many L1 classrooms a variety of process or workshop approaches to the teaching of writing have been implemented. Second language (12) composing research and instruction are also undergoing a similar paradigm shift\u00E2\u0080\u0094 with some reservations about the value of implementing a process or workshop approach in the second language classroom. The question now being asked is, \"How effective are the various process/workshop approaches in the 12 classroom situation?\" The current study, building upon mother-tongue research as well as the mainly case study research which provided the foundation of the English as a second language (ESL) literature on composing, examines the effects of a process or workshop approach on the writing performance of adult English as a second language learners. In addition, the study investigates certain biodemographic variables such as first language, and an affective variable, attitudes toward writing, all of which were hypothesized to interact with the treatment. This study is a controlled experiment in which the treatment consisted of instruction in writing using a workshop format. Two pre- and posttest measures-informal (classroom conditions) and formal (test conditions) writing tests-were used to ascertain writing growth. On each test overall scores were analysed as well as two sub scores, one for content and organization, and one for structure and mechanics. In addition, a pre-instruction background survey was given to elicit information on seven biodemographic variables, and a post-instruction survey on attitudes toward writing was administered. iii Results were mixed. For writing quality, only results obtained on the formal (test-like) measure were significant or near significant in favor of the treatment, the workshop approach. Of the biodemographic variables, only length of time in an English-speaking environment could be interpreted because of a cell distribution problem: it may be that those students with less than two years in a second language environment benefit more from the workshop approach than students with more time and experience in their adopted culture. Regarding attitudes toward writing, the workshop group showed significantly more positive attitudes than the product group. In addition, the content of responses to an open-ended question about writing revealed differences between the two conditions. The workshop students' comments showed awareness of (1) writing as communication and (2) writing as a process requiring time for the development, revision, and editing of ideas and language. These findings indicate that this variety of workshop approach may offer a viable alternative to product-oriented instruction. The formal (test conditions) measure suggests that the workshop may be of benefit in helping students improve their writing, particularly the content and organization aspects. Results from the attitude survey imply that students in the writing workshop are receptive to this approach and that they exhibit more positive attitudes toward writing than do students in the product group. If attitude is indeed the key to improved motivation and performance, as many suggest, these results have important implications for the L2 classroom. iv. TABLE OF CONTENTS ABSTRACT \u00C2\u00BB TABLE OF CONTENTS iv LIST OF TABLES , vii ACKNOWLEDGEMENTS viii CHAPTER ONE: INTRODUCTION AND IDENTIFICATION OF THE PROBLEM 1 Introduction of the research problem 1 Purpose of and rationale for the present study 2 Research questions 4 Operational statement of hypotheses 5 Definition of terms 6 Limitations 7 CHAPTER TWO: THE RELATED RESEARCH 9 The paradigm shift .-. 9 Concerns with real-world demands of writing: meeting needs 11 Concerns with research 11 Concerns with implementation 12 Instructional approach and writing performance: quality of the product 13 L1 research 13 ESL research 14 Instructional approach and biodemographic variables 16 First language and first culture variables 17 Educational variables > 17 Environmental variables 18 Instructional approach and attitude toward writing 18 L1 research 19 ESL research 20 Context of the present study 21 CHAPTER THREE: PROCEDURES 22 Design of the study 22 Participants 22 Treatments 23 i) the process approach 24 ii) the product approach 27 Teacher C1 28 Teacher C2 29 Instruments used 30 The ESL composition profile and the Sager scale (adapted) 30 The background survey 31 The survey on attitude toward writing 32 Collection of data 32 Part A: The writing samples 32 i) composition topics 33 ii) schedule 33 PartB: Completion of the background survey 34 Part C: Completion of the survey on attitude toward writing 34 Scoring and coding of data 34 Part A: The writing samples 34 i) data preparation 34 ii) training sessions 35 iii) scoring of data 36 PartB: The background survey 36 Part C: The survey on attitude toward writing 37 i) the ten statements 37 ii) the open-ended question 38 Analysis of data 40 Descriptive statistics 40 Correlational statistics 40 Inferential statistics: t-tests 40 Inferential statistics: ANCOVA 41 Inferential statistics: standardized mean differences 41 CHAPTER FOUR: FINDINGS 42 Part A: The writing sample and the effect of treatment 42 1. overall growth 43 i) changes in final total scores 43 ii) discussion 44 test 1: the informal writing task 44 test 2: the formal writing task 45 2. growth on sub scores 45 i) changes in final sub scores 46 content and organization 46 structure and mechanics 46 ii) discussion 47 test 1: the informal writing task 47 test 2: the formal writing task 47 Part B: Background survey: main effects & interactions 49 vi PartC: The survey on attitude toward writing 52 1. the ten statements: results between conditions 53 2. the ten statements: results within conditions 56 3. the open-ended question: results 57 i) category 1: the process of writing 60 ii) category 2: general attitudes toward and comments on writing 61 iii) the teacher's role 62 iv) general attitude to class 62 v) topics for writing 63 Summary 64 CHAPTER FIVE: SUMMARY, DISCUSSION, IMPLICATIONS AND LIMITATIONS 66 Summary 66 Discussion 68 Implications 73 Limitations 75 Suggestions for future research 76 REFERENCES 78 APPENDICES 83 Appendix A: Peer revision focus sheet 83 Appendix B: Peer editing focus sheet 84 Appendix C: The ESL composition profile/ Sager scale (adapted) 85 Appendix D: The background survey 87 Appendix E: The survey on attitude toward writing 88 Appendix F: Topics for the writing tests 89 Appendix G: Teachers' instructions for the writing tests 93 Appendix H: Mark sheet for composition rating 97 Appendix I: Students' unedited comments (by treatment and class) elicited by the open-ended question of the writing attitude survey ... 98 Appendix J: Table of interactions (background survey) 102 Appendix K: Population distribution table 103 vii LIST OF TABLES Table 1 ANCOVA explanation of variance for comparisons per informal (testl) and formal (test 2) situations of treatment conditions 43 Table 2 Means, standard deviations, pretest/posttest differences, and standardized mean differences on final scores for each treatment and measure 44 Table 3 ANCOVA explanation of variance for comparisons per informal (test 1) and formal (test 2) situations of treatment conditions broken down by sub scores a) content and organization, and b) structure and mechanics 46 Table 4 Means, standard deviations, pretest/posttest differences, and standardized mean differences on final sub scores for each treatment and measure 47 Table 5 ANCOVA explanation of interaction between treatment condition and length of time in Canada variable for both measures 49 Table 6 The informal measure: means, standard deviations and pretest/posttest differences on scores for each treatment on the length of time in Canada variable (low, high) 50 Table 7 The formal measure: means, standard deviations and pretest/posttest differences on scores for each treatment on the length of time in Canada variable (low, high) 51 Table 8 The survey on attitudes toward writing: percentage agreeing, difference and rank 54 Table 9 Comparison of attitudes to progress in writing per cluster 54 Table 10 The survey on attitudes toward writing: within-condition ranking from high consensus (rank 1) to low consensus (rank 10) 56 Table 11 (1) Total and (2) average number of words and comments per group in response to the open-ended question 58 Table 12 Comments categorized by total number (TN) and percentages (P) per class and per treatment 59 viii ACKNOWLEDGEMENTS Throughout this project, questions and comments from my advisor,0 Dr. Joe Belanger, have served as prompts and guidance on a journey of understanding my own composing processes while undertaking to research the writing outcomes and attitudes of others. For this dual learning experience, and for his invaluable assistance and encouragement, I offer Dr. Belanger my sincere gratitude. To the other committee members, Dr. Alister Cumming and Dr. Charles Curtis, I am also grateful for their advice and different perspectives. I would also like to acknowledge Dr. Harold Ratzlaff for help and comments on the design of this study, and Ross Barbour for assistance in the statistical analysis of the data. Particular thanks go to Dr. Patricia Groves, Dean of Instruction at Vancouver Community College, for her support of the project, and to the students and their teachers in the English Language Skills Department of V.C.C. who participated in the study. Special thanks are extended to my colleagues at V.C.C. and at Columbia College who endured long hours scoring and coding the data. Finally, I wish to thank Wayne Dechka for his patience, support and encouragement throughout the completion of this thesis. CHAPTER ONE INTRODUCTION AND IDENTIFICATION OF THE PROBLEM Introduction of the research problem Current English as a Second Language (ESL) instruction in writing generally consists of two orientations, commonly labelled 'product1 and 'process', terms popularized by Murray in a 1972 article entitled Teach Writing as Process, Not Product\" (in Hairston, 1982, 84). The labels imply that these approaches are mutually exclusive; in fact, that is not the case. However, the labels do capture the essence of the major differences between the two orientations. Product-oriented approaches to the teaching of writing, deriving from such methods as grammar transformation or modelling, focus primarily on syntactic accuracy. The mode of instruction is often presentational (Hillocks, 1986). Its pattern-practice philosophy is rooted in behavioral psychology. In contrast, process-based writing instruction focuses on what writers go through as they struggle to make meaning. The subprocesses, commonly called prewriting, writing and rewriting, allow for individual and/or group problem-solving efforts and focus on thinking skills as well as language skills. This approach owes its origins to cognitive psychology (Bereiter, 1980; Rower and Hayes, 1980; Collins and Gentner, 1980). The product orientation has been the traditional way of teaching written composition to English as a second language (ESL) learners. This approach remains entrenched in the ESL classroom (Zamel, 1987) in spite of a growing body of research, mainly in first language (L1) composition, suggesting alternatives which appear to be more effective. Promising areas of investigation noted in Hillocks' (1986) epochal work Research on Written Composition are focus of instruction (scales, heuristics, and inquiry), mode of instruction (environmental~a problem-solving, structured, small group approach), feedback, and their interaction. It is interesting to note that while second language research followed the L1 lead in examining and describing the processes ESL writers undergo, little second language research in I 2 writing has been done to confirm or disconfirm the L1 findings regarding effectiveness of a process approach to writing; i.e., whether such an approach actually improves the product. Certain ESL researchers (Hamp-Lyons, 1986; Piper, 1989) believe the time has come to answer this question. Consequently, this study examines whether a 'process' or 'workshop' approach to writing in the ESL classroom might be more effective than a 'producf approach. As stated above, certain procedures of a problem-solving nature involving the use of heuristics and peer collaboration (Clifford, 1981; Hillocks, 1986) seem to be effective in improving the writing of first language (L1) students. Few controlled investigations of this nature exist in the ESL literature. Earlier ESL research focused on studies of an ethnographic nature1 and has helped the ESL field assemble a body of literature on the processes of composing separate from the L1 works. (Silva examines the findings of this body of research~twenty-two case study reports dating from 1982, focusing primarily on the composing processes of ESL college student writers, and addressing a variety of issues such as general accounts of L2 composing processes, revision, monitor use, and planning-in his 1989 critical review.) The earlier case study data has laid a foundation for this experimental study. Purpose of and rationale for the present study The present study is a controlled experiment in which the experimental group receives a treatment in writing instruction posited to result in different outcomes between the experimental and control groups. The purposes of this research are threefold: 1) to investigate the effects of instruction~the writing workshop treatment~on writing performance, 2) to examine independent biodemographic variables such as first language (L1) which may interact with the treatment and with students' second language (L2) composing ability, and 1 Borg and Gall (1983) note that some researchers consider case study, participant observation, and ethnography as essentially synonymous (p. 488). 3 3) to determine whether students in the workshop treatment have more positive attitudes toward writing than the control group and what specific differences in attitude are displayed. Specific aspects of the process or workshop approach to writing instruction under investigation are the use of heuristic focus sheets, and the environmental mode of instruction with peer groups and feedback to help students generate ideas, rework and/or recast drafts, and refine the language of their writing. As noted above, this area of inquiry has shown promise in L1 research, but has not been pursued in a controlled experiment in ESL research on composing. The independent biodemographic variables under examination are first language (L1), level of education, length of time in an English-speaking environment, and self-reports of writing \u00E2\u0080\u00A2 ability in L1. The existence of these and other confounding factors influencing composing ability is generally acknowledged. Many studies have attempted to 'tease out' these variables. For example, studies have examined or included some examination of first language-first culture factors (Kaplan, 1966; Mohan and Lo, 1985), second language proficiency factors (Cumming, 1988), and length of time in the target language environment (Cummins, 1980, 1984; Cumming, 1988). This study attempts to pursue this direction of inquiry. Data on attitudes toward writing are based on students' responses to a ten-statement survey followed by one open-ended question on what they liked best about writing in their class. Previous L1 research has shown connections between instruction (especially the peer group aspect of the workshop approach) and attitude toward writing (Katstra, 1987; Clifford, 1981; King, 1979; Beaven, 1977). ESL research, however, while acknowledging the value of positive attitudes toward writing (Piper, 1989; Zamel, 1987), has not yet examined the interactions among instruction, attitude and performance. This study aims to examine this relationship more closely. By looking at learner attitudes and comments between groups (workshop versus product) as well as within group, it should be possible to determine whether there might be interactions with the treatment. 4 Research questions The dependent variable in this research is the writing quality of the compositions, as determined by the ratings of judges. The ratings are based on two (informal and formal) pre- and posttest measures. The writing quality variable consists of overall scores comprised of two sub scores: a) content and organization, and b) structure and mechanics. The two measures are informal and formal writing tests. The informal measure is a classroom-like task with a paired-interactive prewriting component as preparation for the writing test. The formal measure is a typical test-like situation with no verbal interaction allowed. Because writers are working independently during the formal test, this measure is likely to be more representative of individual effort than the informal measure. This is Part A of the study. The two central questions here are: 1. Overall Growth Do students who have been taught to use the process or workshop approach (the experimental group) show greater gains in writing skill as measured in posttests 1 (informal) and 2 (formal) than students trained in the product approach (the control group)? 2. Growth in Sub Scores Do students taught the process or workshop approach (the experimental group) show greater gains in a) the content and organization aspects of writing and b) the structure and mechanics components of writing as measured in posttests 1 (informal) and 2 (formal) than students trained in the product approach (the control group)? There are seven independent biodemographic variables in this research: first language (L1), number of years of education, length of time in an English-speaking environment, and self-reports of writing ability in L1 for informal letter writing, formal business letter writing, short reports and long reports. This is Part B of the study. The central question here is: 3. Will and how will these variables interact with treatment and composing ability? Part C of the study, the survey on attitudes toward writing, consists of ten statements about writing followed by one open-ended question. The ten statements are comprised of three 5 clusters: attitudes toward content and organizational aspects, attitudes toward structure and mechanics components, and general attitudes toward writing. The open-ended question asks students what they like best about writing in their class. Part C of the study is informed by the following research questions. 4. Do students who have been taught to use the process or workshop approach as defined in this study (the experimental group) show more positive attitudes to writing as measured by an end-of-term survey than students in the control group? 5. Do students from the experimental group respond differently to the survey's open-ended question, \"What did you like best about writing in your class?\", than the control students according to 1) number of words written 2) number of comments written 3) categories of comments 4) content of the comments? Operational statement of hypotheses 1. Hi Students who have been taught to use the process or workshop approach (the experimental group) show greater gains in writing skill as measured in posttests 1 (informal, classroom-like) and 2 (formal, test-like) than students trained in the product approach (the control group). HQ Students who have been taught to use the process or workshop approach (the experimental group) do not show greater gains in writing skill as measured in posttests 1 (informal, classroom-like) and 2 (formal, test-like) than students trained in the product approach (the control group). 2. H-| Students taught the process or workshop approach (the experimental group) show greater gains in a) the content and organization aspects of writing and b) the structure and mechanics components of writing as measured in posttests 1 (informal) and 2 (formal) than students trained in the product approach (the control group). 6 Ho : Students taught the process or workshop approach (the experimental group) do not show greater gains in a) the content and organization aspects of writing and b) the structure and mechanics components of writing as measured in posttests 1 (informal) and 2 (formal) than students trained in the product approach (the control group). 3. H-j Students who have been taught to use the process or workshop approach as defined in this study (the experimental group) show more positive attitudes to writing as measured by an end-of-term survey than students in the control group. H 0 Students who have been taught to use the process or workshop approach as defined in this study (the experimental group) do not show more positive attitudes to writing as measured by an end-of-term survey than students in the control group. 4. H-| : Students from the experimental group respond differently to the survey's open-ended question, \"What did you like best about writing in your class?\", than the control students according to 1) number of words written 2) number of comments written 3) categories of comments 4) content of the comments. HQ Students from the experimental group do not respond differently to the survey's open-ended question, \"What did you like best about writing in your class?\", than the control students according to 1) number of words written 2) number of comments written 3) categories of comments 4) content of the comments. Definition of terms For the purpose of this study certain key terms are defined as follows: a) 'Process' approach refers to both the problem-solving strategies learners are taught to use as they compose as well as to the workshop or peer group environment in which they compose. Strategies generally include idea generation techniques and focus sheets to aid groups of learners as they revise for content and organization and edit for structure and 7 mechanics. This approach implies that much time is devoted to the above activities in class, that the teacher takes on more of a facilitator role and that multiple drafting will occur. The final draft is not error-free, but represents the best the learner can do at delivering his or her message, given the learner's level of ability and expertise. b) 'Product1 approach refers to any method focusing mainly on an error-free final draft. The problem-solution role of the learner is much reduced; i.e., the teacher may be the one who presents the (usually grammatical) solutions. This approach does not exclude activities commonly labelled 'process'. These activities most often concern the generation of ideas and editing. While the former (idea generation) is most likely to occur in groups, the latter (editing) is often done by the teacher marking either outside of class or in a conference with the student. The teacher usually marks the first or second draft, and may ask the student to copy out a perfect final draft. This approach takes less class time with the teacher taking on a more 'presentational' role. Multiple drafting may still occur, but its function is different in that the goal is an error-free product. Limitations The conclusions that can be drawn from this study are limited by the following considerations: 1. Due to administrative constraints on registration procedures, intact classes were used instead of individuals randomly assigned to treatment and control sections. Therefore, this study could not control for equal cell distribution over the seven biodemographic variables. As well, the quasi-experimental design and the population sampled limit the extent to which findings can be generalized to the larger population with certainty. 2. Only two writing measures were used. As well, they were dissimilar: one was an informal, classroom-like writing task, while the other was a formal test. 3. Only students enrolled in high intermediate level English as a second language (ESL) courses at a single community college in British Columbia were studied. 8 4. Without the use of pre- and post-instruction writing attitude surveys, no direct causal relationship between the results obtained and the treatment administered could be established. Instead, a comparison between the experimental and control groups was observed and commented upon. 9 CHAPTER TWO THE RELATED RESEARCH Because the current study pursues three strands of investigation-effects of instruction, and possible interactions with both biodemographic as well as affective variables-the review of relevant research incorporates findings from these areas. As well, this review distinguishes between mother-tongue (L1) research, which initiated the paradigm shift in the literature on composing from a focus on product to one on process, and English as a second language (ESL) research, which for the most part has followed the L1 lead. Because the paradigm shift in composing research from product to process has given rise to questions and concerns fundamental to any major shift in perspective, an overview attempts first to place this study in this historical context and second to outline some of the concerns stemming from the change and which have given this study direction. Once this context is established, the review of the research focuses on the more specific components-the effects of instructional approach on writing performance, and potential interactions between approach and cultural, educational, environmental as well as attitudinal variables-related to this study. The paradigm shift According to Kuhn (1963, in Hairston, 1982), revolutions in a field occur \"only when the number of unsolved problems in a discipline reaches crisis proportions and some major figures in the field begin to focus on those unsolved problems\" (p. 77). Such was the case with the traditional product paradigm when certain researchers in mother-tongue (L1) written composition began to note shortcomings of earlier pedagogy and redirect their research efforts (Hairston cites, for example, Shaughnessy, 1977; Young, 1978; Murray, 1976; Flower and Hayes, 1980). 10 Where did the product paradigm go wrong? In Hairston's view, among other problems, it didn't take into account 1) content over form 2) the recursive nature of composing in lieu of the linear approach 3) the fact that instruction in writing is more than instruction in editing. (For a fuller description of features of the product paradigm versus the process paradigm, see Hairston, 1982.) From Shaughnessy's perspective, it didn't enable us to \"understand what goes on during the internal act of writing . . . and to intervene during the act of writing if we want to affect its outcome\" (in Hairston, p. 84). And so this decades-old approach broke down. Text-based product research began to be replaced by writer-based process inquiry. Since then, almost three generations of mother-tongue (L1) composing research (Faigley et al., 1985) have moved the field considerable distance from the early first generation 'product' concerns of testing which instructional methods 'workecT to improve overall writing quality. The movement to examine \"process, not product\" was felt also in English as a second language (ESL) composing research. However, with the widespread examination of composing behaviors and adoption of (or tentative steps toward the adoption of) the process approach(es) to writing, concerns began to emerge. This result is to be expected because \"the new paradigm is sketchy and leaves many problems about the teaching of writing unresolved\" (Hairston, p. 88). The concerns about writing instruction center around three points: the real-world (school and work) demands of writing, i.e., meeting needs (Horowitz, 1986), research questions (answering questions that now need to be addressed) (Piper, 1989), and problems of implementation (Zamel, 1987). All of these concerns seem to indicate that it is time once again to go back to a first generation question, \"How does a process method of instruction affect writing performance and the quality of the product?\" Although this question has been asked by L1 researchers such as Clifford (1981), Hillocks (1982), and Carroll (1984), who found significant positive effects for a process approach over a product approach, it remains virtually unasked in the field of ESL writing research, where the focus throughout the eighties was on case study investigations addressing a variety of issues such as general accounts of the composing processes of ESL writers, revision, and planning (Silva, 1989). 11 Concerns with reai-world demands of writing: meeting needs Researchers, teachers, and administrators query whether a process approach prepares students for real-world writing demands. Horowitz (1986b) argues that the process approach has failed ESL students in academic writing situations (i.e., essay type examinations). Piper (1989, p. 215) notes that while the thrust of many investigations has led to information on various aspects of the composing process, little has been said \"explicitly about how far, if at all, the learners' writing actually improved during the study. Either improvements have not come about or improvement was not a criterion for success.\" The latter position that improvement was not a criterion for success was taken, for example, by Bereiter and Scardamalia-\"it is what they are doing differently that counts, not how well they are doing it compared to how well they previously did something of a different sort\" (1982, p. 51, cited by Piper, p. 216). This position, commendable as it may be, may not be greeted as enthusiastically by the learners themselves for the same reason that teachers and educational administrators might also balk at this position: they recognize fully the real-world (school and work) demands that a product meet a standard deemed acceptable for the given context. Eventually \"the quality of the product to which the process gives rise is absolutely crucial\" (Piper, p. 216) and claims regarding the merits of process approach(es) will have to be justified. Concerns with research Many researchers (Piper, 1989; Hamp-Lyons, 1986; Horowitz, 1986b) express concern that claims about process leading to a better product are simply that: claims. Piper (1989, p. 215) states that beliefs about the effectiveness of the process approach are based on experience and intuition rather than on evidence from research: \"Such beliefs may be perfectly valid sources of classroom principles, but they do not tell us directly whether any particular approach 'works' in an instrumental sense.\" Hamp-Lyons (1986, p. 790-1), while acknowledging the \"superiority [of 12 the process approach] in terms of student involvement and interaction and- therefore of motivation\" makes the same complaint and laments the fact that \"we do not yet have research evidence that emphasis on process leads to a better product in 12 classes . . . and until the results are in, the process approach will remain vulnerable to assault.\" Horowitz (1986b) also, admitting (somewhat reluctantly) that process seems to have some merit, believes that too many questions remain unanswered. Deficiencies in some of the earlier, product research into the effectiveness of different approaches to the teaching of writing have been acknowledged. Zamel's view is that these studies provided us with few significant findings and were often contradictory (1976). However, earlier, product-oriented research does not appear to be the only orientation to suffer deficiencies. Silva (1989), in a critical review of ESL composing process research covering twenty-two reports between 1982 and 1989, notes that, \"while these studies' findings are interesting, provocative, and potentially very useful, their credibility is not beyond question\" (p. 1). Concerns with implementation The process approach to the teaching of composition has been accepted as the new norm in LI theory and, to a lesser degree, in L1 classrooms. Faigley et al. (1985) describe this phenomenon best: \" . . . no development has been more influential than the trend toward understanding and teaching writing as a process. Nearly every current article and book on writing claims to be concerned with \"process\" in one form or another. Old texts are refurbished with new titles, and old lessons are recast in new language\" (p. 3). However, behind the facade of 'process' textbooks lie the \"static and insular\" ways (Rose, 1981, p. 65) of previous mechanistic approaches to the teaching of writing (see Rose, 1981, for a review of L1 texts, and Raimes,1986, for a recent survey of ESL writing texts). As for ESL classroom practices, researchers such as Zamel (1987) note that \"recent surveys of writing instruction indicate that what we have learned from process research is not informing pedagogy . .. perhaps because of the problematic nature of incorporating change in the classroom (p. 697).\" 13 . It appears that teachers may be hesitant to implement change for several reasons. Among the various possibilities are three factors cited by Courtland and Welsh (1990, p. 63-4): \" . . . firstly, teachers' assumptions about curriculum (Miller & Seller, 1985: Zais, 1976) and language teaching and learning (Harste, Woodward, & Burke, 1984); secondly, the expectations which teachers hold about their roles and the role of students as learners (Fullan, 1982); and thirdly, the practices which teachers employ and value (Sarason, 1982). It may be that, in the absence of evidence to the contrary, teachers may be reluctant to abandon dear routines, well-defined roles and accepted practices (regardless of whatever doubts and hesitations they may have) which are seen to guarantee an acceptable product in favour of one where the outcome is in doubt. The perceived risks-by teachers, their students, and administrators-may simply be too great. (See Sola & Bennett (1985) reviewed in Zamel (1987) for one example of curricular constraints overriding a teacher's efforts to establish writing communities in the classroom.) That a paradigm shift is \"apt to be marked by insecurity and conflict within the discipline\" (Hairston, 1982, p. 77) is to be expected. It would seem, then, that some research evidence which might help establish \"whether any approach 'works' in an instrumental sense\" (Piper, 1989, p. 215) may be needed as one step toward implementing change in the classroom. Instructional approach and writing performance: quality of the product L1 Research As noted above, L1 studies by Carroll (1984), Hillocks (1982), and Clifford (1981) indicate that a process approach toward instruction leads to an improved product. As reviewed in Hillocks (1986), promising areas of investigation are focus of instruction (scales, heuristics, and inquiry), mode of instruction (environmental: a problem-solving, structured, small group approach), feedback, and their interaction. In Research on Written Composition Hillocks (1986) examined studies (most notably Sager*s) showing the significant impact of a focus of instruction (scales) on changing the quality 14 of student writing. The Sager study (1973) is of particular interest: her sixth graders' sketchy narratives parallel those written by ESL adult writers at the Intermediate level. Following instruction in the use of a set of questions as a prompt to add detail, her students made large gains in quality over the control students. However, one problem with the Sager study is that only a very few students displayed large gains. D.R. Coleman replicated the study in 1982 with the same effect. Stein (1984) suggested that the environmental approach may owe its success to increased opportunities for feedback. Zoellner (1969) also advocated feedback, albeit at the pre-writing/invention stage only, because it provided for audience input. Other studies using focused feedback sheets with instruction in their use as well as collaboration show increased writing quality over more diffuse treatment groups. Clifford's study (1981), in which college students collaborated with their peers in seven structured stages in addition to using two feedback sheets, one general and the other highly focused, showed positive gains, as did the Benson study (1979). More recently, Graner (1985) found two methods resulting in improved student writing: one group collaborated using peer critique and feedback; the other group of students revised on their own after evaluating sample essays using an editorial checklist followed by a teacher-led group discussion on the merits of the samples. Not all process-oriented studies have reported improvement in writing quality (see for example, Bereiter and Scardamalia, 1982). However, enough evidence on certain aspects of a process methodology in the L1 literature has been gathered to indicate that process instruction can lead to improved writing performance. ESL Research ESL research of a pre- and posttest design using control groups on the relationship between writing performance and instruction in process-oriented strategies appears to be non-existent. However, many studies do investigate the use of peer collaboration, heuristic procedures, and peer or teacher feedback in prewriting, revising or editing tasks . These studies 15 appear to focus on descriptive elements (Davies and Omberg, 1987; Ammon, 1985) or use between-draft evaluation (Chaudron, 1983; Partridge, 1981). In addition to the research studies are teacher-practitioner articles extolling the virtues of collaborative learning (Grimm, 1986). Amnion's (1985) year-long study of four successful ESL third graders generated the hypothesis that \"success in helping children learn to write in English as a second language hinges primarily on the use of instructional activities that are rich in opportunities for exposure to, production of, and reflection on English discourse, and that such activities must include frequent writing, with guidance and feedback, on topics of personal interest\" (p. 82). His study does not answer why the chosen four responded well to instruction using guidance and feedback while others did not. In this sense, the Ammon study suffers from the same shortcoming as the Sager (1973) L1 study where large gains were made, but only by a few students. The Davies and Omberg (1987) study describes the activities used by the peer groups in the prewriting, writing and responding stages, and investigates the learners' attitudes to peer group work. Although the authors state, There is no question that there has been a great improvement in the quality of essays submitted since this system was introduced\" (p. 322), they do not explain how they arrived at that conclusion. The ESL studies which focused on improvement in revisions of drafts (Partridge, 1981; Chaudron, 1983) rather than on pre- and posttests produced contradictory findings: the Partridge study had a more positive effect for teacher feedback, whereas Chaudron found no overall difference between improvement based on teacher or peer feedback. Chaudron speculates that the between-draft design allowing students immediate feedback for revision rather than a pre- and posttest design permitting more developmental learning may account for Partridge's stronger teacher effect. However, his rationale does not explain why his own research, based on a similar design, resulted in more neutral findings. Perhaps the large amount of attrition, too few and unequal N in each cell, and slightly different procedures by each teacher at different ESL proficiency levels interfered with Chaudron's results. It is interesting to note Chaudron's methodological reason for not conducting a pre- and posttest study with control 16 group. He felt that \"the uncertain status of peer evaluation a s an a i d in 12 writing limits the possibilities, out of concern for the learners' potential disenchantment with a solely peer treatment\" (p. 6). At the time of Chaudron's study, peer work did not have the widespread appeal or acceptance that it now does. Two of the few studies documenting writing improvement in terms of pre- and posttest results are Cumming (1986) and Parks (1987). In neither study was a control group against which to compare effects of instruction used. The Cumming study asked engineering students to focus on setting and achieving long-term writing goals. Self-analysis and discussion were cycled throughout the semester. The majority of the group displayed improvement in those areas on which each had focused, but their overall writing improvement as measured by posttests revealed only a slightly higher level of overall performance than was the norm for such students. The Parks study asked ESL pre-college entry adults to use a set of revision criteria during the semester. Their pre- to posttest expository essays showed significant improvement, especially at the text or meaning-based level. As for classroom advice to teachers, Grimm (1986) wrote a teacher-practitioner article dealing with peer collaboration, but from a classroom management point of view. She noted the importance of guiding students through the peer response process, advocating a general framework of guidelines for writing groups rather than a blueprint of specific, teacher-imposed questions. Instructional approach and biodemographic (first language/first culture, educational and environmental! variables Some of the variables often discussed in the literature as interacting with either writing instruction, performance or attitude or some combination of the three include cultural variables (first language and first culture's rhetorical differences), educational variables (L1 composing ability, L2 language proficiency, previous experiences in instruction, level of literacy in L1), and environmental variables such as length of time in the adopted country. A review of the literature in these areas follows. 17 First Language and First Culture Variables The cross-cultural 'interference' factor, with a twenty-five year history of contrastive rhetoric theory, has a long tradition by ESL standards. However, there is no consensus concerning culturally-biased composing patterns interacting with the composing process and product. Kaplan initiated the debate in 1966 upon publication of \"Cultural thought patterns in intercultural education\", but the findings have been disputed (Mohan & Lo, 1985). Interest in first language influence only does not appear to be a concern in the ESL research. L1 is usually linked to level of literacy in L1 (Hudelson, 1984), bilingual writing development (Edelsky, 1982) or first culture, as previously noted. Educational Variables Research evidence has indicated that L2 writers enact their L1 composing strategies (Zamel, 1983; Raimes, 1985). Comparisons of learners writing in their first and second languages suggest that composing behaviors of each individual writer are consistent across languages (Arndt, 1987; Jones & Tetroe, 1987; Edelsky, 1986). However, it is the idiosyncratic nature of composing behaviors that is most notable in many of these studies (reviewed in Cumming, 1988). Second language proficiency is seen to play a certain role, but is labelled by Cumming (1988) as \"an additive factor, enhancing the overall quality of writing produced\" (p. 2). Both L1 and ESL research indicate that prior learning experiences interact with approach to instruction (Witte, 1985 in Zamel, 1987; Blanton, 1987). Blanton notes ESL students' relief when \"at least one of their assignments fits their traditional notions of a writing class\" (p. 116). She believes this security helps students \"tolerate\" radical (to the students) activities such as free writing in journals and learning logs. Previous education, or level of L1 literacy, has generated some interest in ESL composing research. Both attitudes toward literacy in L1 as well as its level of development have been shown to impact upon literacy in L2 (Cummins and Swain, 1986; Edelsky, 1982; Hudelson, 1984). 18 Environmental Variables Cumming (1988) found that measures of the second-language proficiency of the participants in his study \"proved to correlate, almost perfectly with length of residence in an English-speaking milieu\" (p. 49). This finding is similar to conclusions by other researchers (Carroll, 1975; Spada, 1985; Stem, 1985 cited in Cumming, 1988). Cummins (1980, 1984) distinguished between the more social communication skills and cognitive-academic language proficiency (CALP), a domain where writing expertise seems to belong. He noted the longer development period necessary for the development of cognitive-academic skills over social communication skills, in both L1 and 12. The time element required for cognitive-academic skills to develop is inevitably linked to the educational variable above. Instructional approach and attitude toward writing The following review first examines the relevant L1 literature and the instruments commonly used to measure attitude before turning attention specifically to ESL studies. It shows that instruction affects how learners write not just in terms of text performance and writing behaviors, but also in terms of learners' attitudes to writing. The pedagogical approach most often examined is a process/workshop one whereby pairs or small groups collaborate in some organized (often feedback/focus sheets) way to 'evaluate' (in the formative sense) each other's writing-in-progress. The discussion prompts learners to deal with their writing as developing pieces, providing them with some in-class time necessary to begin to work through the drafting and crafting process. The peer groupings also build in an awareness of audience/reader. Implicit in this approach is the teacher's change of role from sole knower and evaluator to one where responsibility for learning is shared. Although the importance of the teacher's attitudes to writing has been documented in both L1 research ( Daly et al.,1988; Rose, 1980, among others) and ESL studies (Zamel, 1987; Piper, 1989), this review will focus solely on learner attitudes. 19 L1 Research Early interest in the link between production problems and attitude led to the creation of the Daly-Miller Writing Apprehension (WA) instrument (1975b). Since then, other instruments have evolved to include a broader range of attitudes than simply the apprehension component. For example, the Emig-King Writing Attitude Scale for Students (WASS.1979) contained three clusters: preference for writing, perception of writing, and process of writing. However, the Daly Miller WA instrument, or an adaptation of it (for example, the English as a Second Language Writing Apprehension Test developed by Gungle & Taylor, in D. Johnson & D. Roen, 1989), remains the instrument of choice in research studies related to apprehension. Of interest is the fact that certain researchers, in addition to using one of the standard writing attitude instruments, also develop an additional questionnaire tailored to the needs of the study. Such is the case in the Katstra et al. report (1987) and the Wolcott & Buhr (1987) study. For the former, an instrument to measure attitudes toward peer evaluation as a process could not be found and therefore had to be developed. This was used in addition to the WASS. In the case of the latter, a questionnaire containing three clusters similar to the WASS was used. Here, however, cluster one on apprehension was similar to the Daly-Miller WA; subset two on students' perceptions of the usefulness of writing resembled items from the \"Writing Attitude Scale\" by Thomas Reigstaad and Donald McAndrew; subset three on students' understanding and use of prewriting and revising techniques appeared to be developed by the researchers themselves for the purpose of their study. Such tailoring of instruments, although enhancing the individual study, make generalizations across studies difficult. The most common aspect of the process/workshop classroom often thought or shown to lead to improvement in attitude toward writing is the use of peer groups (King, 1979; Beaven, 1977; Clifford, 1981; Katstra, 1987, for example). Peer work is often credited with building trust, support, confidence, and motivation, and reducing stress and fear of writing. It is assumed that improved attitude to writing may lead to improved quality of writing. Katstra (1987) calls for 20 further research in this area using both attitude measures and measures of writing quality to see whether, in fact, peer evaluation procedures do improve the quality of writing. Few studies were found which examined the relationship between attitude to writing and writing performance. Fox's study (1980, cited in Gungle and Taylor, 1989) examined whether student-centered (Hillock's 'environmental' mode) methods of teaching writing measurably reduced writing apprehension more than conventional, teacher-centered (Hillock's 'presentational' mode) methods. He concluded that not only was apprehension reduced more in the group using a collaborative workshop approach, but the group also produced writing of a quality at least as high as the control group. A study by Wolcott and Buhr (1987) did not specify whether a peer group component was part of the instructional approach. Their study concluded that for their developmental students, with the exception of the usefulness subset, \"overall writing attitude, as well as apprehension of writing and comprehension of the process involved, seems related to the gain they made in their writing skills\" (p. 7). Two of their recommendations included the clarification of the writing process a s an essential first step in modifying students' attitudes toward writing, and collaborative learning through the peer review of papers. Modifying of attitudes is seen a s critical because, a s John Daly observes, \"A positive attitude about writing is associated with, and may even be a critical precursor of, the successful development and maintenance of writing skills\" (p. 44). ESL Research Daly's perceptions of the value of positive attitudes toward writing are repeated in the ESL literature (Diaz, 1985; Cummins and Swain, 1986; Edelsky, 1982; Hudelson, 1984, all cited in Piper, 1989; Zamel, 1987). As Blanton (1987) notes, \"ESL students bring to their courses perceptions about writing that work against their becoming proficient writers\" (p. 112). Although there are teacher-practitioner articles on how to change students' perceptions about writing, lower their anxiety, and increase their writing proficiency by using journals, peer groups, feedback sheets, discussion, and self and peer evaluation (see for example, Blanton, 1987 and Mittan, 21 1989), little ESL research measuring the relationship between attitudes toward and improvement of writing exists. Context of the present study A review of the literature has shown that the paradigm shift from product to process has been accompanied by questions, concerns, insecurities, and conflict within the discipline. This transition is a part of the emergence of a new norm. Concurrent with the shift from product-based research to a process-oriented inquiry has been a broadened scope of inquiry. Both L1 and ESL researchers are examining various aspects of a process- or workshop-based instructional approach. Peer groups, feedback/evaluation (self and peer), and heuristics such as focus sheets, all of which encourage idea generation, discovery, multiple drafting and crafting, are being investigated for effectiveness. Motivation and attitude toward writing are also seen to be important avenues of research, providing insight into students' willingness to 'perform'. The present study extends the understanding of the process/workshop instructional approach by examining quality of text produced and learners' attitudes toward writing. The investigation does so by comparing an experimental group with a control group. As well, the study pursues inquiry into interactions between instructional approach and biodemographic variables such as L1 culture, L1 level of education and L2 environment, factors which have only begun to be sketched out. 22 CHAPTER THREE PROCEDURES Design of the study The purpose of the present study, as outlined in Chapter One, is threefold. The primary objective is to investigate whether English as a Second Language (ESL) students who have been taught writing using the process approach show greater improvement in writing ability than students trained in the product approach. The second objective is to examine whether a number of relevant background variables might interact with writing skill and treatment. The final objective is to establish whether students from the 'process' treatment show more positive attitudes toward writing than students from the 'product* treatment and what differences, if any, their comments reveal. The experiment was conducted as the writing portion of the Upper Intermediate (Ul) level curriculum for adult ESL learners at a local community college. The methodology in this study included gaining access to four High Intermediate ESL classes for a four-month term, delivering the treatment, collecting a variety of data based on the three objectives outlined above, selecting instruments to be used in the scoring and coding of the data, and lastly, analysing the results. Participants Four intact Upper Intermediate level classes - an 8:30 a.m. section, a noon section and two 6:45 p.m. sections - at Vancouver Community College were chosen for this study. The adult students represented a variety of first language and educational backgrounds. Most worked part or full time and chose one of the four possible time slots which best suited their work schedule. Those students who worked shifts or had changing routines were transferred to appropriate sections as necessary. Students transferring in or out of the experiment's four classes throughout the term were not included in the data base. Students were registered in the Ul level 23 classes based on a battery of tests covering the traditional skill areas of listening, speaking, reading, writing and grammar. Returning students took the departmental battery, whereas new intake students were admitted on the basis of either the intermediate level desk test or, if their marks were high enough on the desk test, the more complete English Language Assessment (ELA). Students registered in Ul classes would expect to follow the 'general skills' curriculum comprised of oral and written communication as well as reading and grammar study. The researcher and another teacher each taught an experimental class (E1 and E2) using the process approach. Two others taught a control class each (C1 and C2) using the product approach. All teachers had experience teaching at the Ul level. As well, all four teachers were scheduled to teach the four-month term without interruption; i.e., no holidays or professional development leaves were to be taken. The teachers agreed to participate in the study after having been given (scant) information about 'a writing project\". The college normally registers twenty to twenty-two students per ESL class expecting approximately sixteen students to complete the term. An attrition rate of roughly twenty to twenty-five per cent is considered typical. Attrition occurs for several reasons, usually work and family demands. The experimental classes began with forty-four students, finishing the term with thirty-four for an attrition rate of 22.7 percent. The control classes originally had forty-two students registered with thirty-one completing for an attrition rate of 26.2 percent. The overall rate of attrition by the end of the experiment was 24.4 percent, indicating that participants in the experiment withdrew at a rate similar to that of the general ESL population in the department. Treatments The terms 'process' and 'product* as used in this study have previously been defined in Chapter One. This section will elaborate by specifying the methods and materials used in the two treatments, process and product. The two teachers (teachers E1 and E2) using the process approach below worked as a team, planning and discussing all aspects of the class, not just the writing component. The two control teachers (teachers C1 and C2) worked independently 24 following their usual classroom routines. They had agreed to hand in a general plan of the assigned writing done in class. It is important to remember that the 'product1 label has been assigned partly as a matter of convenience. It is an umbrella label encompassing many methods and variations thereof, i) The Process Approach The process or workshop approach as implemented in the study used various focus sheets (also called heuristic devices or methods of inquiry) and the environmental mode of instruction (Hillocks, 1986) with the teacher facilitating small groups of learners and intervening as appropriate. It endeavored to incorporate Zoellner's (1969) principles of the writing process, principles which he contends are characteristic of natural human communication (p. 38): 1: to allow the student to see the writing act in process, 2. to make writing a social event, and 3. to provide an audience for the communication. Zoellner would have done well to add a fourth principle, also characteristic of natural human communication: 4. to allow learners to use the language of decision-making to solve (writing) problems. The approach also aimed to take the burden off short term memory by breaking the tasks down into manageable 'chunks' such as idea generation and rehearsing, drafting, revising, editing, publishing and evaluating. A more detailed description of these 'stages' follows. The idea generation/rehearsing stage, often called pre-writing, initially involved the class as a whole, but quickly led to small group (or pair) work. It was essentially an oral/note-taking activity engaging the groups in a variety of invention strategies such as listing, clustering, and \"WH\" (who, what, where, when, and why) questions which helped the group explore ideas, search and find necessary language, and develop and expand the content. During the discussion the groups were encouraged to take notes, jot, scribble, sketch, and so on. This stage led to the writing of a rough, first or discovery draft. 25 In the revision stage each learner in a group of four took a turn at leading the group through an analysis of his or her draft. They followed a focus sheet from the text Oral and Written Composing by Ling and Rothschild (Appendix A). This procedure required learners first to identify possible problems and then attempt to solve them collaboratively. The sheet focused discussion on three aspects of writing: main idea, supporting detail and extraneous or misplaced information. Each writer read aloud to the group stopping after each 'paragraph' (or 'chunk') to initiate discussion on the three points above. The group, forced to listen instead of read, attended to what they heard (i.e., the content or the message), not what they saw. (We theorize that most ESL students have been trained to 'see' surface level errors such as punctuation, spelling, and grammar infelicities, at the expense of the higher level issues of purpose, audience and content/message.) The paragraph by paragraph (or chunk by chunk) procedure had the added advantage of allowing the group in consultation with the writer to explore and/or develop the piece of writing in a way perhaps not originally envisioned. In other words, the writer was free to make content-based changes, either major or minor. This technique offered the writer a way to avoid the draft-two-is-just-a-pretty-draft-one syndrome so common in ESL writing instruction (Zamel, 1985). It also helped writers see that ideas are not pre-formed and ready to be fitted into one of the rhetorical patterns commonly taught. Writers had the opportunity of experiencing ideas triggering other possibilities. The entire revision stage as outlined here was tape-recorded for later reference by the writer who kept in mind the revision experience as he or she redrafted the piece of writing. Although the writer received input from the group in their peer-as-audience role, the writer was still the one responsible for making final decisions. The revision procedure above is what most separates this variety of a process or workshop approach from standard ESL classroom practice, that is, the product approach. While learners in product-oriented classrooms may participate in a few peer-editing activities, rarely are they asked to revise-either solo or in collaboration-for content. Rarer still in product-oriented ESL classrooms is the implication that revising the message is of a higher priority than editing the surface features. 26 The editing stage, also collaborative, followed procedures outlined on a focus sheet from the same Ling and Rothschild text used in the revision stage (Appendix B). Groups of four used a class-devised editing key to signal errors. Each learner/editor had a photocopy of the writer's (chief editor's) draft two and collectively the group went over the draft sentence by sentence. The writer (chief editor) marked the symbol for the kind of error on his master copy. The discussion of errors w a s very much a problem-solving one; unsolved problems were later referred to the teacher. This stage ended with the writer redrafting, filling out an error chart and making an appointment for a conference with the teacher. While the focus here w a s on language use, questions of content and organization recurred. The error chart, based on each learner's struggles with a new language, allowed both the learner and the teacher to track progress in dealing with key problem areas. Taking place after peer revision and editing, the 'full' conference offered an opportunity for both the learner and the teacher to discuss and question the piece's strengths and weaknesses. The same progression of concerns as outlined above was followed; i.e., priority to content/message and organization of the writing before editing language, unless the structures interfered with meaning. This kind of conference normally consisted of one or two meetings. At this point students usually had a choice: to continue with the piece of writing or to file the draft in their writing folders and begin a new theme, keeping in mind comments from the previous writing cycle. For the sake of this study, students were encouraged to redraft on the basis of the conference discussion. In addition to 'full' conferences, very brief 'spot' conferences also occurred if the teacher was not already engaged in the kind of conference described above and if peer groups summoned the teacher. After the conference, the writer wrote a final draft, filled out another column on the error chart and handed the best draft to two peers for evaluation. A fuller explanation of peer evaluation as done in this study is reported in the article \"Self and Peer Evaluation of Writing in the Interactive ESL Classroom: An Exploratory Study\" (Rothschild and Klingenberg, 1990). After 27 this procedure, the writer handed in the entire portfolio to the teacher for evaluation. Finally, the writer chose to share his or her best draft by adding it to the bulletin board or reading it aloud. The workshop format outlined above applied to the three major writing assignments throughout the term. The broad focus of the assignments was provided by either the teacher (e.g., \"Places\") or the class in consultation with the teacher (e.g., The Ben Johnson Affair\"). Each assignment used a variety of invention strategies to help students find their own particular focus. For example, for assignment one students went on a tour of the campus in order to select and draw a diagram of the place they had chosen. For assignment two groups of four worked on a jigsaw reading activity (Brubacher and Payne, 1985), brainstormed/listed possible points of view and roleplayed the various key players involved in the Ben Johnson scandal before choosing the point of view they wished to express. In addition to the major writing assignments, learners wrote weekly in a dialogue journal with their teacher, this kind of journal stresses writing as communication (Kreeft-Peyton, 1984). ii) The Product Approach As previously noted, the product approach covers a variety of methods of writing instruction. Some general assumptions and procedures of the product approach follow before specific descriptions of the two product-oriented classes which participated in the current study are detailed. Generally, the primary focus of the product orientation is the production of an error-free piece of writing. Teachers most likely do not give as much time as is spent in workshop classes asking learners to discover their ideas as they write, or to identify and solve problems in their writing. Teachers may bypass the processes writers undergo by providing models, rhetorical patterns, key words, and necessary structures. This kind of teacher intervention is what Hillocks (1986) labels the presentational mode of instruction. In intermediate level general skills ESL classes, such as those found in this study, much traditional instruction tends to focus on structure. It may be grammar-based: sentence 28 transformation, expansion and combining; and grammar exercises. Or it may be based on non-student generated models. Their use may require manipulation of sentences as outlined above. Sometimes students have more control: they may put their own ideas into a given format. Many teachers test rather than teach writing. Themes are assigned and marked as if they were final drafts (Zamel, 1985). Teacher response is usually written on the first draft (or second if a pre-writing component has been included) where the teacher usually corrects or comments on the learner's errors. A rewrite incorporating the suggested changes is often required. Ways of commenting on students' papers vary (Zamel, 1985). Some teachers correct the students' grammar errors whereas others use symbols to denote kind of error. Where symbols are used, they are often teacher-chosen and students may not comprehend their meaning or purpose. Some teachers comment on grammar and content simultaneously, but seldom is the message or purpose of the writing task given priority. Proponents of the product approach suggest that focusing on more global issues of content may be beyond the capabilities of many ESL students, especially those at the intermediate level. They argue that a more effective and efficient pedagogy is to focus on language use. Evaluation also plays a different role in the product approach: its primary purpose is often to fulfill institutional obligations regarding student assessment. As a result, in assigning marks, few teachers share the criteria with the students. In this situation where the criteria are not intended to help students diagnose and learn from problems, evaluation is incidental to the teaching and learning of writing. The above overview is a general description of variations commonly found in the product approach. More detailed information specific to teacher C1 and C2 follows. Teacher C1 This teacher focused mainly on narrative writing. The exception was a theme asking students to compare Canada and their native country. For this theme students brainstormed five 29 possible areas of comparison: education, transportation, sports, food, leisure. As preparation for the narrative tasks, students engaged in pre-writing activities based on picture sequences or cartoons. These strip stories helped students produce one composition per week. Two compositions, \"My Home\" and \"My Job\" were based on a model paragraph, \"Julia's Story\" from an ESL writing text. Wh questions (who, what, where, why) were used to help students generate ideas. The correction and evaluation of compositions followed standard procedures. The teacher corrected (using symbols) draft one. The students referred to the class handout of symbols with examples to correct draft one. Peer correction was attempted once. Each composition was marked using either an A,B,C or pass/fail scheme. The departmental writing scale used at end-of-term testing was used once before the final writing test. Teacher C2 This teacher also focused mainly on narrative writing. As well, some descriptive tasks were assigned. Students were given various strip stories to model. The teacher also developed a model based on a personal story. Some prewriting was done, usually in the form of a discussion of a strip story. For example, partners decided upon a title, point of view, new vocabulary and introductory sentences together before separating to write draft one alone. Some topics were \"A Short Trip\" and \"A Terrible Day\" (strip stories), \"A child you know\" and \"Someone you admire\" (from the teacher's model). From the stories, the teacher taught various points: referents, transitions, the use of articles, organization, the use of time expressions and time clauses, past tense, compounding, subject and object adjective clauses, prepositions, and reason, result and purpose clauses. Grammar review and instruction were related to writing difficulties or the Ul curriculum. The computer laboratory was also used for additional grammar practice. This teacher corrected spelling and punctuation but occasionally left grammar errors for students to find and correct either in pairs or for homework. These errors were put on the blackboard, discussed and marked 30 by the teacher. As well, she gave a sentence combining exercise using sets of sentences from a previous draft three, worked on various spelling rules, and did error correction on selected sentences from a previous draft one. To use as models for writing, the teacher selected four 'good sentences' from certain students' writing: 'good sentences' meant they contained complex constructions such as adjective and adverb clauses and different kinds of coordination. This teacher also discussed ways to organize writing and writing versus conversational style. She sometimes wrote questions about and made minimal comments on content. When this was done, students checked the meaning with their original partners and worked together on revisions. Draft two was marked according to the departmental writing scale. After a conference with the teacher, students wrote draft three which was again corrected by the teacher. One topic was given in exam-like conditions, draft one was marked and a rewrite was assigned as homework. The students selected their best composition and rewrote it for display. Instruments used The ESL Composition Profile and the Sager Scale (adapted) Part A of this study required an instrument to measure writing quality. Holistic scoring, often used in studies of writing quality, would not yield the information necessary to answer the research questions posed in Chapter One and relating to Part A: The writing sample and the effect of treatment. What was needed was a (criterion-based) scale which would allow for not just overall ratings but also sub scores. A further complicating factor was the intention to have students use the same scale for instructional and diagnostic purposes, as reported in Rothschild and Klingenberg, 1990. The solution was to adapt the widely-used ESL Composition Profile (Jacobs et al., 1981) and the Sager scale (1973), shown to have produced positive results in students' writing growth (as reviewed in Hillocks, 1986), for use by both teacher-judges and the learners themselves. 31 The adapted evaluation scale (Appendix C) was tailored in the following ways to suit the requirements of the project. The scale is most similar to the ESL Profile in its language use and mechanics components, whereas it is most similar to the Sager scale in its content and organization categories. Sager*s focus on reader awareness and elaboration of detail is more appropriate for the expressive/reflective kinds of writing done in the program. Because of the focus of this study (content and organization, structure and mechanics) and plan for student use of the scale, it was felt that a vocabulary component could be a complicating factor. Therefore, it is not included, as it is in the ESL Profile and the Sager scale. The weighting of the categories parallels that of the ESL Composition Profile; i.e., content is weighted most heavily (35 points), followed by language use (30 points), organization (25 points) and mechanics (10 points) for a total of 100 points. Each category contains four mastery levels (except for the mechanics category with only three levels). As Jacobs et al., the creators of the ESL Profile, explain in Testing ESL Composition: A Practical Approach (1981), the purpose of the descriptors in each level is \"to focus attention on significant aspects of a composition which affect the degree to which a writer succeeds (or fails) in the communicative effort\" (p. 91). The Background Survey The review of literature in Chapter Two suggested certain factors might interact with writing ability. Certain of these factors became the seven biodemographic variables of the background survey (Appendix D) developed for Part B of this report. The survey requested information (following Cumming, 1988, where these items were field-tested) from students about 1. language spoken at home (recoded as 1. Asian 2. Spanish 3. Persian 4. Other), 2. number of years of education (recoded as 1. low 2. mid 3. high), and 3. number of years living in an English-speaking country (recoded as 1. low 2. high). As well, the survey asked students about their self-reported ability in their first language (L1) to 4. write letters to their friends and family, 5. write letters of a business or work-related nature to others, 6. write short reports at school or work, and 7. write long reports at school or work. 32 Their responses to their L1 writing ability, originally measured on a one-to-five-point scale, were receded into 1. low, 2. mid, and 3. high categories. The Survey on Attitude Toward Writing The survey on attitude toward writing (Appendix E) was developed by the Assistant Department Head of the department in which the study was run in consultation with the researcher. It was agreed that the four classes involved in the project should not feel singled out from the other nine Ul classes. It was also felt that the department would benefit from having information from the entire Ul level of thirteen classes for curriculum review purposes. Therefore, the survey was administered to all UI classes in the department. The criteria were that the survey be brief, clear and easy to administer. It asked students to agree or disagree with ten statements about their writing. All statements were positive and compared end-of-term writing performance and attitude with beginning-of-term. The statements reflected the major aspects of the treatment received (content and organization, language use and mechanics) as well as general attitude to writing. An open-ended question at the end of the survey gave students the opportunity to write about what they liked best about writing in their class. Collection of d a t a Three kinds of data were collected: writing samples, a background survey of seven biodemographic variables, and a survey on attitudes toward writing. Part A: The Writing Samples Two aspects of the writing samples are explained below: the selection and description of the composition topics, and the writing schedule for the topics. 33 i) Composition Topics Four topics were selected for use in the current study (Appendix F). Topic 1, used as the informal pretest (pretest 1), was a common, first week teaching activity in which teachers ask students to interview a partner and then write a story about that partner. Topics 2 and 3, used as the formal measure in both pre and posttest situations, were selected from a departmental item bank of suitable topics. They met the department's criteria for a Ul level writing test; i.e., they tended to elicit writing of a narrative, descriptive or mixed narrative-descriptive nature. To control for topic difficulty, these two topics were randomly distributed within each of the classes as the formal pretest (pretest 2). Then, in the formal posttest situation (posttest 2), each student received the alternate topic. Topic 4 was an informal measure parallel to topic 1 and was used as posttest 1. Because there could be no control for topic difficulty on the informal measure, both of these topics were chosen for their proven 'track records'. The informal measure (topics 1 and 4) reflected typical, informal classroom writing tasks with paired-interactive pre-writing activities included. The formal measure (topics 2 and 3) was a true test situation with no collaboration permitted. The informal measure was intended to measure the kinds of writing commonly done in the classroom, whereas the formal measure represented controlled testing situations and was thought to be more indicative of individual effort. Both measures were weighted equally for the purpose of the current study. Students were allowed one hour for all tests except for the informal pretest where forty-five minutes was felt to be more appropriate for a first-week in-class activity of this nature. The instructions the four teachers received are included in Appendix G. ii) Schedule The informal pretest was administered in the first week of class (September 1988) with the formal pretest in week two. An early informal posttest was given in week 11 in November followed by the formal posttest in week 13. 34 Part B: Completion of the Background Survey Teachers asked their students to fill out the background survey at the beginning of the term. The teachers asked the students to write down answers for numbers one to four. They also explained the one-to-five-point scale in question five and asked students to circle the numbers representing their answers. If any questions arose, the teachers answered them. The students were given the time needed to complete the form, five to ten minutes. Part C: Completion of the Survey on Attitude Toward Writing At the end of the term each teacher explained all Ul classes were being asked to complete the survey because the information would be helpful to the department, but individual's names were not necessary. There was no space for either the teacher's name or the section. Students were told to check whether they agreed or disagreed with the ten statements, and to answer the question, \"What did you like best about writing in your class?\" Students were free to ask for help if they did not understand. They were given the necessary time to complete the form, approximately fifteen minutes. Scoring and coding of data Three components of the study were scored and/or coded: the writing samples, the background survey, and the survey on attitude toward writing. Part A: The Writing Samples The scoring of the writing samples is explained in three sections: data preparation, training sessions, and the scoring of the data. i) Data Preparation The treatment sessions produced sets of four compositions per student. To ensure anonymity of the subjects, after the data gathering sessions each student was randomly assigned 35 a number which was put in the upper left comer of the original composition as well as on the class key. The writer's name was then clipped from the top. The handwritten originals from each test were then sorted in numerical order in groups of ten in folders coded either Sample A, B, C or D depending on treatment session. The informal pretests were Sample A, formal pretests Sample B, informal posttests Sample C and formal posttests Sample D. Each folder was clearly marked; e.g., Sample A, # 1 -10. To the outside of each folder were stapled three identical mark sheets for composition rating (Appendix H) with the composition numbers in the left-hand column, spaces for the rater's name and date at the top, and in the other five columns spaces for the four sub scores as well as the total mark. The first rater was to use the top mark sheet. When all compositions in a folder had been rated by the first rater, this rater returned the entire folder to the researcher who then took off the top sheet, wrote rater two's name on the second sheet and handed the folder to the second rater. The third sheet was used only in cases where a third rater was needed, that is, when raters one and two disagreed by ten or more marks out of one hundred. Only complete data sets were used. In other words, a student had to be present for all four tests throughout the term to be included in the study. Incomplete data sets were used to train the raters. ii) Training Sessions Nine raters participated in the study. For reasons beyond the researcher's control, one group of four marked Samples A and B, while two weeks later another group of five marked Samples C and D. Training procedures for both groups were identical. Although the raters were familiar with various writing scales, none had used the adapted scale before. Therefore, it was distributed a week before the training and scoring sessions so that raters could begin the training session with questions or comments concerning the scale. In preparation for the training sessions with the raters, the researcher was assisted by the colleague who had originally helped adapt the scale and who had previously trained ESL 36 students to use it. This team selected a variety of compositions to be used throughout the training sessions, rated papers individually and then compared and discussed results. The initial training session with the raters began with a general discussion of questions concerning the scale distributed the previous week. That discussion completed, training on the sample compositions began. The group scored one paper and followed the scoring by discussing decisions about the first paper. This cycle was repeated three more times until the group demonstrated they were interpreting the scale with greater consensus. At this point the raters felt ready to begin scoring papers. It had been agreed that regular retraining sessions would take place after each break so that raters would be able to recalibrate their interpretation of the scale, if necessary. iii) Scoring of Data Each rater took one folder of approximately ten compositions. Upon completion, the rater returned the folder to the researcher or assistant, and was given another folder to mark. Raters took a break about every one and a half to two hours. While the raters were marking, the researcher and assistant checked the mark sheets. When a third rater was needed, one was identified and his or her name was written on the third mark sheet. Any totals on marks sheets one and two with a discrepancy of ten or more marks were circled. As had been agreed, after each break, a retraining session occurred. Normally, only one sample was needed before raters felt ready to return to scoring. On both the informal and formal pretests, a reliability coefficient of .83 (adjusted for third rater) was calculated. On the informal posttest the adjusted inter-rater reliability figure was .90 while it was .86 on the formal posttest. Part B: The Background Survey Raw data from the seven biodemographic variables were recoded as follows: 1. language spoken at home (recoded as 1. Asian 2. Spanish 3. Persian 4. Other), 2. number of years of education (recoded as 1. low = eight years or fewer 2. mid = nine to twelve years 3. high = thirteen or more years), and 37 3. number of years living in an English-speaking country (recoded as 1. low = fewer than two years 2. high = two to twenty-five years). As well, the survey asked students about their self-reported ability in their first language (L1) to 4. write letters to their friends and family, 5. write letters of a business or work-related nature, 6. write short reports at school or work, and 7. write long reports at school or work. Their responses to their L1 writing ability, originally measured on a one-to-five-point scale, were recoded into the following categories: 1. low = points one and two, 2. mid = point three, and 3. high = points four and five. An analysis of covariance was run to test for interactions between the treatment conditions and the seven independent biodemographic variables. Part C: The Survey on Attitude Toward Writing The scoring of the first part of the survey, consisting of ten statements, is described first. A description of the scoring and coding of the second part of the survey, the open-ended question, follows. i) The Ten Statements Responses were tallied for each statement per condition. In the few cases where a student did not respond to one of the statements, that student was not included in the tally for that statement. In the rare event that a student placed a check between the agree and disagree columns, the check was counted half for each column. Percentages based on the number of agreements per statement (out of the total number who answered each statement) were calculated for each statement and each treatment condition. The difference between percentages per condition was calculated for each statement and those differences were then rank ordered with one indicating the largest difference and ten the smallest between the two groups. The tally procedure was repeated when the ten statements were divided into the content and organization, structure and mechanics and other categories. The within-condition ranking simply used the percentages already calculated for each statement and each condition to rank each statement within each treatment condition. The 38 statement with the highest percentage of agreements was ranked first, and the statement with the lowest percentage was ranked tenth. A t-test was used to compare differences between the mean number of agreements on all ten statements per condition. Three additional t-tests were done when the ten statements were divided into the three clusters of content and organization, structure and mechanics, and general attitude. ii) The Open-Ended Question Responses to the open-ended question were analyzed according to three surface measures: number of words written per treatment condition, number of comments written per treatment condition, and categorization of the comments. For the first surface measure, the researcher counted the number of words per treatment condition. For the second surface measure, the researcher and the colleague who assisted during the composition quality training session separately counted the number of comments based on the rule that a change in 'topic' signalled a new comment. Agreement on the total number of comments was almost unanimous: a discrepancy of two comments out of a total number of ninety-eight. The following unedited examples from the four classes illustrate how the comments were counted. Counted as two comments: Example A Topics were short and things I could write about easily. / Examples were given in class. Example B The way how we doing our writting. You can learn a lot / and you can see what kind of mistakes we usually make. After couple corections I never make the same mistake. .39 Counted as three comments: Example C Since I started my class I never miss class even one day. / And when ever my teacher gave me a topic to write a composition I was happy to write / and I was got good marks. Counted as four comments: Example D The teacher is excellent and she helps us a lot. / She explains the grammar very well. / If s very good to writte composition because you can know what kinds of mistake you have. / Before I came to KEC I didn't know how to writte a composition but now I don't have many problems. T-tests were performed on the two surface measures above (number of words written and number of comments written) to see if significant differences existed between the means of the experimental and control conditions for each measure. As for the third surface measure, categorization of the comments, initially, nine categories were conceived after a preliminary look at the data and based on the various aspects of the treatment After a pilot attempt at categorizing the comments, the researcher revised the categories, collapsing comments from the earlier pre-writing, revising/editing/drafting, group work and time categories (some of which were empty) into one category called The Process of Writing'. Similarly, comments from the needs/wants and improvement categories were moved into the 'Attitude to Writing' category. There were enough comments on The Teacher's Role' to warrant separating them from those in the 'General Attitude to Class' category. To the latter category were transferred comments pertaining to marks and assessment. Finally, the old time/topic category was retained as simply Topics'. After a training session in which the researcher and colleague categorized the comments from approximately ten per cent of the surveys, a final classification was done. Agreement was high with a reliability coefficient of 0.96. The raw scores of the number of comments per category per class in each treatment were also 40 calculated as percentages. Appendix I presents the students' unedited comments divided by treatment into the categories used in the study. Analysis of data All of the data collected in the study except for data requiring analysis by t-tests were coded and prepared for statistical analysis using the SPSSX package at the University of British Columbia Computing Centre. T-tests were calculated using the program Statistics for Researchers V2.0 SFRP. Four techniques were employed for the statistical analysis of the data. Descriptive Statistics In order to compare the anticipated differences of central tendency and variability between the data collected for each group of subjects, the means and standard deviations of the raw scores (overall scores and the two sub scores of content and organization, and structure and mechanics) for writing quality were calculated per treatment condition. Percentages and rankings were also calculated for the survey on attitudes toward writing in Part C. Correlational Statistics Reliability coefficients were used to determine rater reliability in scoring the compositions as well as in coding the comments. Inferential Statistics: t-tests To determine the statistical significance level of differences between the means of each treatment group, t-tests for group means between independent samples (on the total sample of ten statements as well as on the three sub samples created when the ten statements were divided into categories) were calculated in Part C, the ten-statement survey on attitudes toward writing. Because the direction of the difference was predicted, one-tailed tests were used with the alpha level of significance set at 0.05. In addition, t-tests were performed on two surface 41 measures-number of words written and number of comments written~of the open-ended question component of the survey on attitude toward writing to see if significant differences existed between the means of the experimental and control conditions for each measure. Inferential Statistics: ANCOVA In Part A of the study in order to determine if significant differences existed between the mean scores of each treatment group on variables of overall and sub score writing quality ratings when the influences of the mean scores for earlier tests were controlled, analysis of covariance (ANCOVA) was used. Analyses on two posttests were computed using scores on two pretests as covariates. Analysis of covariance instead of analysis of variance was used to compensate for pre-instruction differences on scores between the experimental and control conditions. The experimental group scored slightly lower than the control group on the informal pretest, while the reverse occurred on the formal pretest. As with the analyses of covariance outlined above, the statistical significance of interactions was measured between treatment conditions and the seven independent variables from the background survey in Part B. The level of significance was set as 0.05. Inferential Statistics: Standardized Mean Differences (SMD) Standardized mean differences, previously known as effect size (Glass, McGaw and Smith, 1981), were also calculated. To compute the SMD, the difference between the experimental mean and control group mean is divided by a standard deviation which is free of treatment effects. Because there are no formal conventions for judging SMD magnitudes, criteria suggested by Cohen (1977) for small (.2), medium (.5), and large (.8) standardized mean differences were used. 42 CHAPTER FOUR FINDINGS The findings of the study are presented in this chapter, categorized separately by the research questions that were presented earlier in Chapter One. Part A looks at whether students taught the process approach showed greater gains in writing quality on overall scores as well as on sub scores than did the control group. Two measures were used: informal and formal tests. Part B examines whether significant interactions occurred between writing skill and the seven independent variables from the background survey. Part C investigates whether students in the process writing treatment showed more positive attitudes to writing than did the control group. It also asks how their comments differed. Tables that summarize the statistical analysis of the data are provided and interpretations of the findings are offered in order to allow a clear picture to emerge of how the subjects responded to the treatments. Ancillary tables of the statistical results have been included in the appendices. Part A: The writing sample and the effect of treatment The discussion of the findings for Part A is divided into two components representative of the first two research questions: overall growth in total scores and growth on the sub scores. Both of these components are further divided into three areas. First, the analysis of covariance (ANCOVA) results are reported. Second, the changes in the raw scores are reported. Finally, the results from the ANCOVA, the changes in the raw scores, and the magnitude of the standardized mean differences (SDM) are discussed for each measure, the informal, classroom-like writing task and the formal, test-like task. 43 1. Overall growth The first question asked in the study was, \"Do students who have been taught to use the process approach (the experimental group) show greater gains in writing skill as measured in informal and formal posttests than students trained in the product approach (the control group)?\" The study examined differences in the overall quality of the compositions written by the experimental and control groups. An analysis of covariance (ANCOVA) was performed to control the influence of initial differences between treatment conditions in both the informal and formal pretest situations. The ANCOVA in Table 1 below shows no statistically significant differences between the experimental and control groups in either the posttest 1 (informal) or posttest 2 (formal) situations, although the results based on the formal writing test approach significance (p = .06). Table 1: ANCOVA explanation of variance for comparisons per informal (test 1) and formal (test 2) situations of treatment conditions Measure MeanSquare DF F Prob Post 1/Pre 1 28.21 1 .18 .67 Post 2/Pre 2 307.22 1 3.76 .06 i) Changes in final total scores (Both the control and the experimental groups gained over the twelve-week experiment on both the informal (test 1) and formal (test 2) measures. The changes in writing quality during the experiment, as Table 2 shows, favor the control group by a small margin (.94 points out of 100 or almost one percent) on the pre-post test 1 comparison, and the experimental group by a larger margin (4.23 points or more than four percent) on the pre-post test 2 comparison. 44 Table 2: Means, standard deviations, pretest/posttest differences, and standardized mean differences (SMD) on final scores for each treatment and measure Treatment/ measure Exp/test 1 Con/test 1 Exp/test 2 Con/test 2 34 31 34 31 Pretest x1 44.06 44.60 50.47 50.29 S 7.33 8.62 12.30 7.30 Posttest x2 46.12 47.60 56.44 52.03 s 13.11 12.05 8.92 10.14 Diff. x?-x1 +2.06 +3.00 +5.97 +1.74 7' SMD -.10 +.42 Note: The total scores are out of 100. These marks represent the Ul range. ii) Discussion The hypothesized gains in writing quality for the treatment group in the experiment were not confirmed, although results on the formal (second) measure approached both statistical significance (p = .06) and a standardized mean difference of medium magnitude (.42). Test 1: The Informal Writing Task The high probability figure (.67) on the first measure, the informal test, suggests that the difference between the experimental and control groups on this measure was small. The standardized mean difference (SMD of -.10) failed to reach the criteria for a small SMD, reinforcing the fact that little difference between groups was found on the informal measure. Both groups showed small gains on the pre-post test 1 comparison, with the control group gaining almost one percent (.94 points) over the experimental group. An interaction with the informal test may have influenced this outcome: one of the control teachers reported to the researcher that posttest 1, an informal test based on the pedagogy of the writing workshop approach (i.e., a prewriting \"trigger\" as an idea-generating technique and small group interaction) caused much stimulation among the students. The control students did not normally use this approach during the term, whereas the experimental group did. It may be that the inclusion of what may be regarded as a process writing strategy, the prewriting \"trigger\", used in the informal task 'contaminated1 the results, causing the control students to perform better on this measure and 'watering down' the outcome. 45 Test 2: The Formal Writing Task The second measure was the more standard or traditional one commonly found in studies on composing. It consisted of a formal test (i.e., no prewriting \"triggers\" or other idea generators used in small group interaction), and might therefore be considered more indicative of the individual writer's ability in a test situation. On this measure, the two topics had been equally but randomly distributed within each of the classes for pretest 2 with the alternate used in the posttest 2 situation. This formal test elicited results approaching significance (p = .06). Because the 0.05 level of significance was not reached, we are unable to reject the null hypothesis and cannot conclude that the thirteen-week experimental treatment had a positive effect upon overall writing growth. However, the fact that the results (p=.06) approached statistical significance suggests further research is warranted. The standardized mean difference for the pretest-posttest mean gains was .42, which approached a medium SMD. Although the level of statistical significance was not reached, the SMD approached the criteria for a medium effect, indicating the treatment was successful. Thus, when sample size is not a factor, the results are quite promising. 2. Growth on sub scores The second research question asked was, \"Do students taught the process approach (the experimental group) show greater gains in a) the content and organization (C&O) aspects of writing and b) the structure and mechanics (S&M) components of writing as measured in posttests 1 (informal) and 2 (formal) than students trained in the product approach (the control group)? Because the results above show no statistically significant differences between the experimental and control groups in either test situation, further statistical inquiry would normally be suspended. However, given the nearness to statistical significance in the test 2 situation (p = .06) and in light of the second research question, the researcher felt that an analysis of the two 46 sub scores (content and organization, structure and mechanics) was warranted. Table 3 below shows mid- to high-probability figures on three of the four measures, indicating little difference between the experimental and control groups in their content and organization scores on the informal first test as well as in their structure and mechanics scores (informal test 1 and formal test 2). Only the content and organization scores on the formal test 2 measure demonstrate a statistically significant difference (p - .03) between the experimental and control groups. Table 3: ANCOVA explanation of variance for comparisons per informal (test 1) and formal (test 2) situations of treatment conditions broken down by sub scores a) content and organization (C&O) and b) structure and mechanics (S&M) Measure MeanSquare D.F. F Prob a) C&O Post 1/Pre 1 11.35 1 .15 .70 Post 2/Pre 2 216.24 1 5.10 .03* b)S&M Post1/Pre 1 4.81 1 .22 .64 Post 2/Pre 2 7.78 1 .75 .39 *p<.05 i) Changes in final sub scores Content and Organization Changes in the content and organization sub scores show positive growth in both informal and formal test situations for both the experimental and control groups (Table 4). The change on the informal measure slightly favors the control group (by .29 points out of 60 or less than a half percent). The larger margin (3.6 points out of 60 or 6 percent) on the formal measure favors the experimental group. Structure and Mechanics The structure and mechanics sub scores on the informal measure show a slight loss (less than one point out of 40) for both groups, the experimental group displaying the marginally 47 larger loss. On the formal measure both groups improve, the experimental group gaining .63 points out of 40 or slightly more than one and a half percent over the control group. Table 4: Means, standard deviations, pretest/posttest differences, and standardized mean differences (SMD) on final sub scores for each treatment and measure n Pretest Posttest Diff. SMD a)C&0 x1 s x2 s x2- x1 Exp/test 1 34 24.82 5.00 27.59 9.33 + 2.77 \ Con/test 1 31 25.50 5.65 28.56 7.96 + 3.06 >- -.04 Exp/test 2 34 31.31 8.59 34.94 5.93 + 3.63 Con/test 2 31 31.24 5.43 31.27 7.39 + 0.03 \u00E2\u0080\u0094 ^ - . 5 0 b)S&M Exp/test 1 34 19.24 3.64 18.53 4.81 - 0.71 Con/test 1 31 19.10 4.02 19.03 4.66 - 0.07 =^-.16 Exp/test 2 34 19.16 4.90 21.50 3.79 + 2.34^. Con/test 2 31 19.05 3.12 20.76 3.51 + 1.71 .16 Note: The content and organization (C&O) score is out of 60, the structure and mechanics (S&M) score out of 40. ii) Discussion Test 1: The Informal Writing Task An interaction with the informal test, as explained above, is most likely responsible for the slightly better performance favoring the control group on the informal measure for both sub scores. The loss in the structure and mechanics sub score for both the experimental and control groups may be attributable to the informal nature of the pre- and posttest 1 writing tasks, which may have led both groups to focus on content over form. The standardized mean differences on the sub scores (-.04 and -.16) did not reach the criteria (.20) for a small SMD. Test 2: The Formal Writing Task On the formal measure the experimental group outperformed the control group on both sub scores, the largest difference (6 percent) reflected in the content and organization sub score. This large difference, which is statistically significant (p = .03), appears to be the result of the 48 treatment: the content and organization component (i.e., the peer revision stage) is where the experimental group's treatment departs most from the control group's treatment of the teaching of writing in the English as a Second Language (ESL) classroom. (In fact, control teacher 1 did not teach students to revise for content and organization; control teacher 2 did some teaching of the need to revise organization, but did not deal with revision of content; both teachers focused on teaching the structural aspects of language.) The standardized mean difference for the pretest-posttest mean gains was .50, which is a medium SMD and indicates that the treatment was successful. The structure and mechanics sub score on the formal measure shows the experimental group gaining just more than one and a half percent over the control group. The difference is not statistically significant. The standardized mean difference of .16 approached the criteria for a small SMD, indicating the treatment had a slight positive effect. It is possible that the treatment had less effect on the experimental group's structure and mechanics sub score than was the case with their content and organization sub score because this aspect of the treatment did not differ as radically between the experimental and control groups as did the revision (for content and organization) component of the treatment; i.e., although the experimental group used peer editing to focus on form while the control group relied on mainly teacher feedback and marking for errors, both groups did attend to form. 49 Part B: The background survey - main effects and interactions It was hypothesized that a number of relevant biodemographic variables might interact with writing ability. Consequently, analysis of covariance was computed in order to test for the existence of any significant interactions between writing skill and background variables. The independent variables under investigation were 1. first language (Asian, Spanish, Persian, other) 2. years of education (low, mid, high) 3. length of time living in Canada (low, high) 4-7. serf-reports of L1 writing ability (low, mid, high) of informal letters, formal letters, short reports and long reports. Because of the unequal cell distribution on six of the seven variables, the interaction effects are uninterpretable (Glass and Hopkins, 1984). Appendices J and K present the table of interactions as well as the population table of cell sizes for all seven variables. Only one of the variables, length of time living in an English-speaking environment in Canada, was thought to have acceptable cell distribution. While both high groups are equal in size (n=16), the low groups are not (Exp n=18, Con n=11), perhaps calling into question the acceptability of cell distribution (Marascuilo and Seriin, 1988). Table 5 shows a statistically significant interaction between the treatment and the length of time in an English-speaking environment on the pre-post test 1 (informal) comparison only. That interaction was statistically significant on the Structure and Mechanics sub score (p=.O07) as well as on the overall total score (p=.05). Table 5: ANCOVA explanation of Interaction between treatment condition and length of time in Canada variable for both measures Measure Cell Size C&O S&M Total C&O + S&M Exp Con P P DF MSq F p Testl high 16 16 .16 .007* 1 582.19 4.13 .05\" Informal low 18 11 Test 2 high 16 16 .25 .19 1 132.98 1.57 .22 Formal low 18 11 *p<.05 \"p=.05 50 The informal test produced results (Table 6) which show a contrasting pattern regarding both treatments' interaction on the performance times length of time in an English-speaking environment variable. Whereas the low (less than two years) Experimental group's scores were lower than those of the high (two or more years) Experimental group for all three scores (content and organization, structure and mechanics and total) and their gain smaller (+0.70 versus +3.59), the low Control group began with higher scores than the high Control group and also outperformed them (a gain of 7.27 versus one of 0.91). In other words, not only did the low Control group gain more than any other group, they also started higher. Before discussing why the eleven students in the low Control treatment, those having spent less than two years in Canada, outperformed to such a degree the students in the high Control treatment as well as those in the Experimental treatment on the informal measure, examination of the formal test (Table 7) is warranted to see if the pattern is repeated. Table 6: The Informal Measure: means, standard deviations and pretest/posttest differences on scores for each treatment on the length of time in Canada variable (low, high) Posttest C & O Posttest S & M Pretest Total Posttest Total Post -Pre n & group X s x s x s x s diff. 18 Exp low 27.22 9.50 17.42 4.30 43.94 6.74 44.64 12.66 +0.70 16 Exp high 28.00 9.43 19.78 5.18 44.19 8.16 47.78 13.82 +3.59 11 Con low 32.23 6.11 21.64 2.69 46.59 8.95 53.86 7.81 +7.27 16 Con high 26.50 7.07 17.41 4.38 43.00 8.00 43.91 10.79 +0.91 The formal measure is similar to the informal test in one respect: the low Control group again started off higher than any other group. However, the gain pattern is different. On the 51 formal test the low Experimental group showed considerable improvement (+7.39), the high Experimental group moderate growth again (+4.37), the low Control group some improvement (+2.63) and the high Control group almost no growth again (+0.35). Only the high Experimental and Control groups appear to be consistent; that is, their gain on the informal test (moderate for high Experimental, negligible for high Control) was similar to their gain on the formal test. The low Experimental and Control groups, on the other hand, appear to have reversed their positions on the formal, second measure, with the low Experimental group now gaining the most. Table 7: The Formal Measure: means, standa scores for each treatment on the leng rd deviations and pretest/posttest differences on th of time in Canada variable (low, high) Posttest C & O Posttest S & M Pretest Total Posttest Total Post -Pre n& group x s X s x s X s diff. 18 Exp low 34.44 5.07 21.03 4.33 48.08 9.58 55.47 8.77 +7.39 16 Exp high 35.50 6.90 22.03 3.13 53.1614.64 57.53 9.24 . +4.37 11 Con low 33.91 8.61 22.18 3.66 53.46 4.64 56.0911.87 +2.63 16 Con high 29.44 6.80 19.69 3.56 48.78 8.78 49.13 9.23 +0.35 Three explanations may account for the differences observed. First, the informal nature of the first measure may have allowed the low Control group, those having spent less than two years in an English-speaking environment, to 'catch up' to, and surpass, the high Control group. Recall that the informal measure engaged the students in paired pre-writing activities, enabling all students to search for and find the language necessary to complete the (oral) task with their partner before moving on to the writing stage. In the discussion of the informal writing task in Part A it was noted that the novelty of this approach for the control students and resulting interaction with the informal test may have caused them to perform better than the experimental 52 group on this measure. It now seems that this interaction may have been felt mainly by the low Control students, and not by the entire Control group. Second, other variables may have interacted with the environment variable, confounding the outcome. Common sense dictates possible interactions with educational background as well as with other factors occurring in the learners' environment. For example, it would be important to know whether the language used at a student's workplace is English or some other language. Those who work in English, regardless of length of time in this country, may well have an advantage over those who do not. These confounding factors may also account for the lack of a statistically significant difference between the two treatments on overall writing quality in Part A of the study. The final explanation is that the significance of the differences between conditions may be spurious given the cell distribution. In their discussion of cell sizes in two-factor analysis, Marascuilo and Seriin (1988) state unequal cell sizes \"make it impossible to determine the unique contributions of each family in a single analysis\" (p. 684). A fuller analysis, perhaps via interviews and more detailed surveys of the English-as-a-second-language learners involved in this study as well as other statistical procedures such as multiple regression, might answer the question. Unfortunately, such analysis is outside the scope of this report. Part C: The survey on attitude toward writing Part C of the study was informed by the following research questions. 1) Do students who have been taught to use the process approach as defined in this study (the experimental group) show more positive attitudes to writing as measured by an end-of-term survey than students in the control group? 2) Do students from the experimental group respond differently to the survey's open-ended question, \"What did you like best about writing in your class?\", than the control students according to 1) number of words written 2) number of comments written 3) categories of comments 4) content of the comments. 53 At the end of the term ail students were asked to respond to a survey on their attitudes toward writing. They read ten statements comparing end-of-term with beginning-of-term attitudes concerning specific aspects of their progress and writing in general, and checked whether they agreed or disagreed. The ten statements were followed by an open-ended question, \"What did you like best about writing in your class?\" 1. The ten statements: results between conditions It was hypothesized that the experimental group would display more positive attitudes toward writing than would the control group. To see whether the experimental group's responses to the ten statements differed significantly from those in the control group in the predicted direction, a one-tailed t-test was done. The comparison between the mean number of agreements on all ten statements reveals significant differences between conditions (t:1.82, d.f.:18, p = .04, one-tailed), where the experimental condition had the larger mean score (25.10, 21.70) indicating greater consensus. table 8 shows the experimental group responding more positively (i.e., agreeing more) than the control group to all ten statements regarding their progress and their attitudes. The largest differences in agreement between the two conditions concern the areas of organization (ranked first with a 19.35 point difference), vocabulary development (second, 13.61), ideas (third, 13.01), and details (fourth, 12.48). These four areas fall into the content and organization category. In the middle ranks are grammar improvement (ranked fifth with a 10.54 point difference, comfort in writing (sixth, 10.00) and comprehension of language problems (seventh, 9.25). The smallest differences in agreement between the two conditions are ease of writing (eighth, 4.95), writing enjoyment (ninth, 4.60) and writing improvement (tenth, 0.39). In these last three areas, the general attitude category, the two conditions appear to be similar. In fact, both groups are almost unanimous in the degree to which they agree that their writing is better than in September. 54 Table 8: The Survey on Attitudes Toward Writing: percentage agreeing, difference and rank Statement percenfge agreeing Exp n = 33 percenfge agreeing Control n = 31 Diff. Rank of the diff. 1. My writing is better than in September 1988 93.94 93.55 + 0.39 10 2. I have more ideas for writing now 96.88 83.87 + 13.01 3 3. Writing is easier for me 56.67 51.72 + 4.95 8 4. My writing is better organized 83.87 64.52 + 19.35 1 5. I feel more comfortable about my writing now 76.67 66.67 + 10.00 6 6. My writing has more details now 86.67 74.19 + 12.48 4 7. I understand the kind of mistakes I make now 86.67 77.42 + 9.25 7 8. My grammar has improved 83.87 73.33 + 10.54 5 9. I have more vocabulary now 78.13 64.52 + 13.61 2 10.1 enjoy writing now 66.67 62.07 + 4.60 9 To investigate further which components of the treatment may have affected learners' attitudes to perceived progress in writing to a statistically significant degree between conditions, the ten statements were divided into three categories and t-tests were done (Table 9). Two of the three clusters were specific to the two major components of the treatment; i.e., content and organization (statements 2,4,6,9), and structure and mechanics (statements 7,8). The third cluster focused on general attitudes to writing (statements 1,3 5,10). Table 9: Comparison of attitudes to progress in writing per cluster Exp x s Control x s DF T Prob. Content and Organization 27.75 2.33 22.25 2.87 6 2.98 0.01* Structure and Mechanics 27.25 0.35 23.25 1.06 2 5.06 0.02* General 23.75 5.52 21.00 5.64 6 0.70 0.26 * p<.05, one-tailed Table 9 shows a statistically significant difference (p = .01) between the two conditions regarding students' attitude to progress in the content and organizational aspects of writing. The experimental group reached high levels of agreement on their perceived progress in this area (a 55 mean of 27.75 students out of a possible thirty-three agreed with the four statements, s=2.33). Agreement in the control group was lower (a mean of 22.25 out of thirty-one) with a slightly greater standard deviation (2.87). Note that greater progress in this category as perceived by the experimental group parallels observed writing improvement in the content and organization sub score on the formal second measure, where there is a statistically significant difference between the experimental and control groups (p=.03). Attitude to progress in the structural and mechanical aspects of writing also reveals a statistically significant difference between the two treatments (p=.02). The standard deviation is negligible (0.35), indicating a high degree of consensus among students in the experimental or workshop group. Although the experimental students perceived greater progress in this aspect of writing, in fact the quality ratings show no statistically significant difference between the two groups on the structure and mechanics sub scores. Three explanations are possible. First, perhaps students' perceptions of their ability to progress in this area outstrips their ability to acquire language at this level (Ul). A second possibility is the peer editing groups have given students a false sense of their ability to monitor their own language use in independent situations; i.e., a test. A final reason might be the lack of time in a test situation to check language use as thoroughly as is done in class. The remaining cluster (general attitudes) does not reach statistical significance, although the experimental group reached a higher level of agreement. Out of the four statements in this category, the only one showing a reasonable spread (10 percent) between the two groups is statement five, \"I feel more comfortable about my writing now.\" It is likely that the workshop approach with peer groupings as provided in the experimental treatment helped raise the comfort level for this group. It appears then that the experimental group's responses to the survey on attitudes toward writing have been influenced by the process treatment, an approach which, with its emphasis on peer collaboration and problem-solving, leads students to perceive improved performance and motivation. 56 2. The ten statements: results within conditions Upon examination of the within-condition rankings of the ten statements (Table 10), a similar pattern appears in both groups; i.e., highest agreement within both groups concerns ideas and overall improvement (ranks 1 or 2), followed by details, comprehension of mistakes, and improvement in grammar, organization and comfort (ranks 3 to 8). Students within both conditions agree least with the statements concerning enjoyment and ease of writing (ranked 9 and 10). It is interesting that, while both groups differed significantly on their overall level of agreement regarding the ten statements, their pattern of consensus was remarkably similar. That both conditions agreed to a very high degree (experimental = 93.94 percent, ranked 2; control = 93.55 percent, ranked 1) with statement 1 may be attributed to that fact that all learners had invested a considerable amount of time and money in the form of registration fees in their learning opportunities. That consensus was low concerning enjoyment and ease of writing may have more to do with the angst of writing in one's second language (Raimes, 1984), or the fact that they have improved and now realize how much more difficult writing is. Table 10: The Survey on Attitudes Toward Writing: within-condition ranking from high consensus (rank 1) to low consensus (rank 10) Statement percenfge agreeing Exp rank percenfge agreeing Control rank 1. My writing is better than in September 1988. 93.94 2 93.55 1 2. I have more ideas for writing now. 96.88 1 83.87 2 3. Writing is easier for me. 56.67 10 51.72 10 4. My writing is better organized. 83.87 5.5 64.52 7.5 5. I feel more comfortable about my writing now. 76.67 8 66.67 6 6. My writing has more details now. 86.67 3.5 74.19 4 7. I understand the kind of mistakes I make now. 86.67 3.5 77.42 3 8. My grammar has improved. 83.87 5.5 73.33 5 9. I have more vocabulary now. 78.13 7 64.52 7.5 10. I enjoy writing now. 66.67 9 62.07 9 57 3. The open-ended question: results it was hypothesized that the students in the experimental group would differ from those in the control group in the following ways: they would generate more written language as measured by 1) the number of words and 2) the number of comments. A different profile would emerge based on 3) the pattern of the categories into which the comments fit and 4) the content of the comments. In the survey, the students were also asked to respond to the question, \"What did you like best about writing in your class?\" Three students out of 33 in the experimental group chose not to respond to this section of the survey. In the control group, two out of 31 did not respond. Appendix I presents the students' unedited comments divided by treatment into the categories used in the study. The results were analyzed according to 1. number of words 2. comments written 3. categories of comments 4. content of comments The researcher alone did the word count. As explained in Chapter Three in the section on the scoring and coding of data, the researcher and a colleague performed the analysis on point 2, reaching near unanimity. On point 3, where nine categories were initially conceived before being reduced to five, agreement between the two coders on the categorization of the comments was high (reliability coefficient of 0.96). Table 11 shows both groups responded similarly to the first two criteria, with the experimental group writing just slightly more words and comments overall and per student than the control group. No significant difference between the two groups was found. 58 Table 11: (1) Total and (2) Average number of words and comments per group in response to the open-ended question n total # of words average # of words per st. total # of comments average # of comments per st. Exp 30 595 19.83 56 1.87 Con 29 535 18.45 42 1.45 The five categories into which the students' comments were divided are as follows: 1. the process of writing (idea generation, revision, editing, conferences, time, the overall process) 2. general attitudes to and comments on writing 3. the teacher's role 4. general attitudes to the class (not specific to writing) 5. topics Table 12 shows the majority of comments for both the experimental and control groups fell into the two writing categories, the process of writing and general attitudes to and comments on writing (Exp = 66.1 percent, Con = 61.9 percent). For both groups, category 2, general attitudes to and comments on writing, was most popular (Exp = 33.9 percent, Con = 35.7 percent), followed by category 1, the process of writing (Exp = 32.1 percent, Con = 26.2 percent). The third most popular category was 3, the teacher's role, for the experimental group (12.5 percent), and 4, general attitudes to the class, for the control group (21.4 percent). Tied for fourth for the experimental group were categories 4, general attitudes to the class, and 5, topics, at 10.7 percent each. For the control group, comments fell least often into categories 3, the teacher's role, at 9.5 percent, and 5, topics, at 7.1 percent. The only large difference between the groups occurred in category 4, general attitudes to the class, where the percentage of comments by the control group was double that of the experimental group. A t-test of the overall categorization of comments indicates no significant difference between the two groups. That no overall difference was found on this surface feature of categorization is, upon reflection, not surprising. Given the nature of the question \"What did you like best about writing 59 in your class?\", one would expect categories one and two, both concerning writing, to contain the majority of the comments. Where differences might be expected are the other three categories. In category 3 it was anticipated that the less traditional role of teacher as facilitator in the process writing treatment would elicit more comments from the experimental group. Perhaps that did not occur because the students, even though they had never before experienced writing groups with a teacher/facilitator, were used to teacher-as-facilitator in the oral communication component of current and previous classes. It was also projected that category 4 might elicit more comments concerning general attitudes to class from the workshop group because of the heavier emphasis on peer groupings and knowledge sharing. In fact, most of the comments in this category came from the control group. Finally, category 5 was also expected to produce more comments because the experimental group, unlike the control group, shared some control over the themes and generated a range of topics within a theme. However, only slightly more comments by the experimental group fell into this category. Table 12: Comments Categorized by Total Number (TN) and Percentages (P) per class and per treatment class treat-ment 1 process of writing 2 attitudes to writing 3 teacher's role 4 attitudes to class 5 topics TN P TN P TN P TN P TN P TN Exp1 9 37.5 8 33.3 3 12.5 1 4.2 3 12.5 24 Exp2 9 28.1 11 34.4 4 12.5 5 15.6 3 9.4 32 Total Exp 18 32.1 19 33.9 7 12.5 6 10.7 6 10.7 56 Con1 7 26.9 11 42.3 2 7.7 4 15.4 2 7.7 26 Con2 4 25.0 4 25.0 2 12.5 5 31.3 1 6.3 16 Total Con 11 26.2 15 35.7 4 9.5 9 21.4 3 7.1 42 Although the hypothesized differences regarding 1) the number of words 2) the number of comments and 3) the pattern of categories were not found, examination of 4) the content of 60 the comments indicated some differences between the experimental and control groups. A discussion of the differences follows. i) Category 1: The Process of Writing Students' unedited comments elicited by the open-ended question of the writing attitude survey (Appendix I) reveal that the experimental group's comments covered a wider range-idea generation (3 comments), revision for content and organization (3), editing for structure and mechanics (7), writing conferences (1), time for the process (1), and how to write (3)-than the control group which commented mainly on idea generation (7 comments), editing (3), and the way of writing (1). This difference reflects the wider range of activities performed by learners in the experimental treatment. Appendix I also reveals that the experimental group's comments were fairly uniformly distributed between both E1 and E2 classes. This was not the case for the control group. All seven comments regarding idea generation came from the same control class (C1) and all of those comments referred to the use of pictures to help generate ideas (e.g., \"Frequently teacher gave us a few pictures and we explained these pictures, which gave us more idea and it was usefull for me.\"). Two students added that group discussion about the pictures made it easier to write. All three comments on editing were generated by the other control class (C2) and referred to finding and fixing mistakes (e.g. \"I liked noticing in puntuation very much and also about spelling because those are were my reall weakness.\"). The differences between C1 and C2 appear attributable to each teacher's approach, described in detail in Chapter Three. A further difference between the two groups was that all (11) comments by students in the control group were positive, whereas two of the 18 comments by students in the experimental group were negative. The complaint concerning perceived lack of fellow students' proficiency in peer editing appears to be a fairly common one (Chaudron, 1983; Davies and Omberg, 1987). The other negative comment came from a student wanting to learn more English before learning how to write. 61 In addition to the comments generated by both groups concerning idea generation and editing, the experimental group also mentioned liking organization and revision tasks, conferences, the time for the writing process and the way writing was taught. These comments reflect favorably upon the process writing treatment that the experimental group received. ii) Category 2: General Attitudes toward and Comments on Writing The most frequent comment from students in both groups was that they liked writing. Those students who explained why recognized the value of writing: they found composing a useful way to learn and improve their English. Control students referred to specific areas of improvement such as vocabulary, grammar and structure while experimental students did not mention these aspects of writing. Instead, they wrote about the communicative aspect of writing, not mentioned by the control group. This is to be expected because much of the time spent by the experimental students was in peer group situations where their first priority was to get their intended message across to their audience. Other differences between the groups concerned comments on journal writing (done exclusively by the experimental group) and models (used only in the control group). Three students from the experimental group liked journal writing best, while one student from the control group liked model paragraphs. Only one student from each group did not like writing. Interestingly, one student from each group mentioned the best thing was short compositions. The differences in the students' comments appear to reflect the different focus and methods of the experimental and control groups. What is most interesting, however, is students' generally high level of awareness and acceptance of the method being used, regardless of the method. Acceptance by the control students is perhaps to be expected because their treatment reflected standard classroom practice. It is the ease of acceptance by the experimental students of a 'new1 approach that is somewhat surprising. It must be assumed that the benefits of the 62 process approach were seen to outweigh any disadvantages, and that these students, similar to those in the Davies and Omberg study (1987), were therefore more inclined to comment on the advantages. iii) Category 3: The Teacher's Role Students from both groups commented positively on their teachers' ability (4 comments) and perceived the teacher's role similarly; i.e., to explain and to correct (7 comments). Correction for both groups appears to mean fixing grammar mistakes, not rhetorical problems or audience concerns. It is surprising that students from the experimental group commented only on the teacher's \"mistake-correcting\" ability because the teacher-student conference followed a hierarchy of writing concerns, with attention to error last and if need be in a later conference after higher-level problems of content and organization, if neglected by the peer groups, had been addressed. One student from this group might have been indicating his dislike of the teacher's initial questions on content (and avoidance of error correction) when he commented, \"Should the teacher talking about what kind the mistake the student made. I think so\". Only one student, from the control group, referred to the teacher's non-grammatical help (...and our teacher explain about it and give the some idea\"). It would have been interesting to interview the students from the experimental group further on their perception of the teacher's role to explore why they didn't 'notice' 1) other aspects of the conference (both 'on-the spot' group conferences during and fuller conferences at the end of a writing cycle) and 2) the general teacher-as-facilitator approach. It may be that students have been so inured to the role of teacher-as-corrector-of-mistakes that they simply do not notice behaviour outside of the expected or known. iv) Category 4: General Attitude to Class Most comments from both groups referred to other skill areas of language learning such as reading, listening and grammar, and of those the majority stressed the grammar component (4 out of 6 for the experimental group, 4 out of 9 for the control group). 63 The two other comments from the experimental group mentioned a feeling of confidence with the teacher and classmates, and reading and listening as favourite subjects. Confidence with the teacher may be due to the teacher/facilitator role which \"encourages students to take the kinds of risks necessary for the development of writing and helped make them more confident about their own intuitions about writing\" (Zamel, 1987, p. 703). Confidence with peers may owe its origins to the informal but secure workshop approach offered by the process writing treatment. It is difficult to ascertain what is meant by \"reading and listening as favourite subjects\". Only an interview could help determine whether the student was referring to the various (jigsaw) reading and listening activities that were part of the writing workshop, or to activities external to it. The five remaining comments from the control group concerned marks (2), attendance (1), improvement of writing and speaking together (1) and stories every day (1). What is most evident regarding students' comments about general attitude to class is both groups' overriding concern about the importance of grammar instruction in language learning regardless of method of teaching writing used in this experiment. The question why this is so remains to be answered. Here again, interviews with students might help clarify the matter. One wonders to what degree the students have 'noticed the hidden curriculum manifested by the department's rigorous standards and extensive battery of end-of-term tests as well as by teachers' (sub)conscious comments and references. v) Category 5: Topics for Writing Students from the experimental group tended to name specific topics they liked (4 out of the 6 comments) whereas the control group made more general comments about the kinds of topics they preferred (all three comments). The experimental group enjoyed writing about \"a real story which happened\" because it was \"useful for students\": two students cited the Ben Johnson affair, a theme selected by the class, as an example. (For this theme students did a jigsaw-reading activity, listed the various characters, and roleplayed their relationship to Ben before writing from the perspective of one of 64 the key players.) They also liked writing about their country and about Canada. All of the topics or themes were personally relevant for these students. The writing done by the experimental group tended to go beyond the easier picture-story routines produced by the control students. Two students from the experimental group commented negatively on the kinds of topics. One found writing about a restaurant or a cafeteria boring because he didn't have much experience in that situation. (The student, having the choice to visit and describe any place on campus that interested him, was responsible for selecting the place to be described.) The other student generalized that some topics were \"hard to talk about\". Several factors seem to be at work here: lack of experience in choosing a topic, lack of involvement in the topic, lack of background knowledge, and possibly mode of discourse. Perl, as reviewed in Zamel (1982, p. 197) noted that \"students wrote more and with greater fluency and satisfaction when their writing involved them personally, while they wrote with less facility when the writing was more objectified (1980a, p. 30-31). It is also known that certain kinds of writing tend to be easier for the ESL student, with narrative being easiest, followed by descriptive and expository modes (Sinclair, 1983). Students in the control group mentioned liking writing about cartoons, stories and short topics because they were \"things I could write about easily\". Teacher C1 relied heavily upon the use of pictures to produce storylines, causing her students to generate some seven comments on this pre-writing technique in category 1 above. The students' comments about the ease of this kind of writing tend to be in line with the findings cited above. Summary Chapter Four has presented the findings of the study categorized according to the research questions posed. In general, the hypothesized gains in overall writing quality for the treatment condition in the experiment were not confirmed, although results on the second measure, the formal writing test, approached statistical significance (p=.06). As for the two sub scores, (1) content and 65 organization and (2) structure and mechanics, only the content and organization sub score on the formal measure was found to be statistically significant (p=.03) in favor of the experimental condition. Also, on the second, formal measure, the standardized mean differences (SDM) approached the criteria for a small effect on the structure and mechanics sub score (.16) and a medium effect on the total score (.42), and reached the criteria for a medium effect on the content and organization sub score (.50). The background survey, consisting of seven biodemographic variables thought to interact with writing ability, proved impossible to interpret on all but one variable due to uneven distribution of the population in the cells. When intact classes are used in educational research, as is often the case, unforeseen problems such as this arise. The only variable to have shown reasonable cell distribution, the length of time in Canada in an English-speaking environment, was significant at the .05 level of confidence on the total score as well as on the structure and mechanics sub score (p=.007) on the first measure only, the informal writing test. Interpretation based on the existing data, deemed incomplete, was complex and beyond the scope of this study. Results based on the survey on attitudes toward writing reveal the experimental condition having significantly more positive attitudes overall to writing (p=.04). Of the three categories, two most affected the overall results: the content and organization cluster of attitudes and the structure and mechanics cluster were statistically significant at the .01 and .02 levels of confidence respectively. It was also noted that, while the overall level of agreement on the ten statements in the survey indicated the two conditions did not share the same attitudes to the same degree, both the experimental and control groups showed a remarkably similar pattern of response; i.e., they ranked the ten statements in roughly the same order. The open-ended question component of the survey showed no statistical differences between the two conditions on the surface measures; i.e., average number of words and comments written per student, and the categories into which these comments were classified. However, differences were found when the content of the comments was examined. 66 CHAPTER FIVE SUMMARY, DISCUSSION, IMPLICATIONS AND LIMITATIONS Summary The primary purpose of the study was to examine the effects of the treatment, a process or workshop instructional approach to writing, on high intermediate adult English as a second language (ESL) students' written output or product. Secondary and tertiary concerns asked questions about biodemographic variables such as language background, level of education, length of time in an English-speaking environment and L1 writing ability, and an affective variable, attitude toward writing. These questions were, \"What biodemographic factors interact with treatment and students' ability to compose in their second language?\" and \"Do students from the 'process' treatment display more positive attitudes toward writing than students from the 'product' treatment, and what differences, if any, do their comments reveal?\" The study was a controlled naturalistic experiment with four teachers instructing four intact classes divided into the experimental and the control groups. Over a four-month term several measures were administered: informal (classroom-like) and formal (test-like) pre- and posttest writing measures, a beginning-of-term background survey concerning biodemographic variables, and an end-of-term survey regarding attitudes toward writing. An analytic, criterion-based scale based on the ESL Composition Profile and the Sager scale was adapted for the study and used by teacher-judges to score the compositions. Two teacher-judges categorized the open-ended comments from the attitude survey. Correlations on the quality ratings ranged from .83 to .90. The correlation on the categorization of the comments from the attitude survey was .96. In answer to the five research questions asked in Chapter One, the following findings are provided. 67 1. Overall growth in writing Over the thirteen weeks of the treatment, students who had been taught to use the process or workshop approach did not show greater gains in overall writing skill than students taught by the product approach as measured by both an informal (classroom conditions) and a formal (test conditions) posttest. However, on the formal test, which emphasized individual performance (unlike the informal, classroom-like measure with its collaborative prewriting component), the difference approached statistical significance in favor of the workshop approach (p=.06). Although the .05 level of statistical significance was not reached, the standardized mean difference calculated on the second, formal measure approached the criteria for a medium effect (.42), indicating the treatment was successful. Thus, when sample size is not a factor, the results are quite promising. The inclusion of a collaborative prewriting component typical of a writing process methodology (in other words, the treatment) in the informal, classroom-like writing task appears to have 'contaminated' or weakened the results on this measure. 2. Growth in writing sub scores: Content and organization, structure and mechanics Of the two sub scores tested, only the content and organization sub score on the formal writing test was found to be statistically significant (p=.03) in favor of the workshop approach. As well, standardized mean differences showed a medium effect (.50) on the content and organization sub score, and a small effect (.16) on the structure and mechanics sub score, indicating the treatment had some success. 3. Interactions among the seven biodemographic variables, treatment and composing ability Of the seven variables tested, the environmental variable (length of time in Canada in an English-speaking environment) was significant at the .05 level of confidence on the total score as well as on the structure and mechanics sub score (p=.007) on the informal writing test only. The interaction with the informal task noted in point one above 'watered down' results. However, it 68 may be that the inclusion of a collaborative prewriting activity enabled the low Control group, those having spent less than two years in an English-speaking environment, to find the language necessary to catch up to, and surpass, the high Control group on the informal writing measure. Complications arising from uneven cell distribution made further interpretation of the six remaining variables unreliable. 4. Attitudes toward writing: responses to the ten statements Students in the workshop condition had significantly more positive attitudes overall to writing (p=.04). Of the three categories of attitudes toward writing, two most affected the overall results: the content and organization cluster of attitudes and the structure and mechanics cluster were statistically significant at the .01 and .02 levels of confidence respectively. While the overall level of agreement on the ten statements in the survey indicated the two conditions did not share the same attitudes to the same degree, both the workshop and the product groups showed a remarkably similar pattern of response; i.e., they ranked the ten statements in roughly the same order. 5. Attitudes toward writing: responses to the open-ended question No statistical differences between the two conditions on the surface measures, i.e., average number of words and comments written per student, and the categories into which these comments were classified, were revealed. However, differences were found when the content of the comments was examined. Discussion The following discussion addresses the research problem from three perspectives, that is, the three components of the study: performance outcomes (the effects of treatment instruction on students' writing), the interaction of the biodemographic variables, treatment and composing ability (i.e., the idiosyncratic nature of writing), and affective outcomes (learner attitudes toward 69 writing). The discussion attempts to draw the three strands into a more cohesive portrait. It is hoped that this synthesis may answer some questions (within the boundaries of this study) regarding a workshop approach to the teaching of writing to adult English a s a second language learners. That the treatment, i.e., the workshop approach, appears to have had some, albeit limited, positive effect on students' writing performance and attitudes is clear. That the effect on performance is not stronger is perhaps not surprising in view of acknowledged difficulties with this kind of research (Hillocks, 1986; Zamel, 1976). However, it is interesting to note that the strongest effect (a statistically significant difference of .03 on the content and organization sub score of the formal writing test and a standardized mean difference of medium magnitude, .50) was found on that aspect of the treatment-the peer revision groups using a focus sheet for feedback-which departs most from traditional, product-oriented writing instruction. Other researchers have also noted large gains made by students under similar peer group/feedback conditions (Parks, 1987; Clifford, 1981; Benson, 1979; Sager, 1973). It seems plausible that the peer revision activity, a cognitively demanding task, helps students focus enough to begin to \"trouble-shoot\", evaluate, and solve some of their own problems, start to see their writing as their peer-audience sees it, and begin to move from \"writer-based\" to \"reader-based prose\" (Rower, 1979). For ESL learners, the peer revision activity, has the added advantage of helping them collaborate to find the language they need, hone or adjust it, and expand their language base as they make decisions about writing. (Poliakoff, 1990, expands upon this aspect in \"Eliciting the language of decision making through collaborative revision of compositions\". She examined the use of composition-related activities such a s peer revision and editing versus the use of pre-constructed, somewhat artificial tasks used to stimulate talk in the adult ESL classroom. She found that the collaborative composition tasks not only offered a natural alternative to pre-constructed decision-making tasks, but also provided a context for conscious reinforcement of principles of grammar and rhetoric, thereby integrating oral and written language learning.) 70 Other impediments to a stronger result might include the 'unlearning' factor (Newkirk, 1984). Regarding the learners in his study, Newkirk points out: Students had to unlearn in order to learn in this new way, to discard approaches and expectations, to take on a new kind of student role and attitude (143-4). It seems evident that learners new to any approach may have to do some 'unlearning'. In this study, for example, simply demonstrating the use of the revision focus sheet and providing opportunities for peer groups to use it does not guarantee its successful implementation. The following incident involving the teacher of the second experimental class (teacher E2) and one of her students, 'Sylvia', may help clarify. Sylvia had been revising her untitled discovery draft (draft one) with her group. The piece, detailing parts of Ben Johnson's life from early days to the Olympic scandal, followed a direct chronological time line. The group felt information was missing, but if the missing information were supplied, the draft would be long and boring. The writer agreed and wanted to change direction. Animated, she caught the teacher's attention, and explained the problem. \"Now I know what I want to say,\" she exclaimed. When asked what it was she wanted to say, she continued, \"How he was here (gestures pointing high), but now he's here (more gestures aimed at the ground). When the teacher praised the idea, Sylvia added, \"But can I do that?\" She hadn't believed, in spite of input (with which she agreed) from her revision group, that she could abandon her original choice to pursue other options. When assured that she was free to do so, she went off happily and returned with draft two entitled From the top to the bottom which now began: My name is Ben Johnson. For a short time I was known as \"Big Ben\". But now I'm known as \"Big Cheater\". That episode provided Sylvia with the opportunity to understand and operationalize peer revision. Her group experience helped her learn that \"it is important to use plans that are detailed enough to test, but cheap enough to throw away\" (Flower and Hayes, 1980, 43). 71 Biodemographic variables may also interfere with the effects of instruction on students' writing performance. Although first language (L1) per se may not be a complicating factor, unless the L1 operates entirely within an oral tradition, first culture and related educational experiences within that culture may interact with the second language (12) classroom experience. Whether this potential interaction may be due to cultural differences (as Kaplan, 1966, implies), individual differences, or other factors is not clear. Jeroski, an L1 researcher, addressed the contribution of the individual in the instruction/performance relationship (called aptitude-treatment-interactions or ATl-the extent to which individual differences moderate the effects of instructional conditions on student achievement) in her 1982 doctoral dissertation, Competence in written expression, with inconclusive results. In any event, the L1 variable suffered from a cell distribution problem, as did five of the other variables examined in this study, level of education and self-reports of writing ability in L1 based on four task types. The unequal cell distribution made interpretation impossible. Further discussion of these factors based on the current information would be skewed by the unequal N in the cells, and is therefore not allowed. The only variable with a healthy cell distribution was length of time in an English-speaking environment. The correlation between time in the 12 environment and performance gains did not occur as expected. It appears that there was an interaction with the informal measure, a classroom-like writing situation with a paired-interactive prewriting component, which allowed the low (less than two years) control group to surpass all other groups in writing performance. On the formal measure, a typical test situation with no paired-interactive component, the results were reversed: the low experimental group appeared to benefit most as measured by writing gains. It must be noted that results were steady for both high (two or more years) control (negligible gains) and experimental (moderate growth) groups on both measures. These results invite the hypothesis that a treatment involving a prewriting component might be particularly helpful for learners having spent little time, in this case less than two years, in the 12 environment. The 72 prewriting component of the workshop might help these students 'catch up' to fellow students who have had more time and experience in the new culture and language. The process or workshop treatment appears to impact more upon affective outcomes, that is, attitudes toward writing, than on performance outcomes. This positive relationship between workshop approaches involving peer feedback and attitude toward writing has been documented by many researchers, including Davies and Omberg (1987) and Chaudron (1983). Others have commented on the high levels of involvement, interaction, and motivation often evident in process-oriented classrooms (Hamp-Lyons, 1986; Poliakoff, 1990). As well, it seems that the workshop treatment is related to the experimental group's positive attitudes toward writing to a higher degree than the product-oriented treatment is related to the control group's attitudes. That the relationship is stronger does not imply a causal relationship, however. Perhaps the most surprising aspect of the attitude section of this study, as revealed by the students' open-ended comments, is the high level of (1) awareness and (2) acceptance of instructional delivery method by students in the process-oriented classrooms. That the students in this group were so accepting of what is likely a novel approach seems to indicate their recognition that advantages of the workshop approach outweigh disadvantages. Certain aspects of the survey remain puzzling: both groups perceive (1) the teacher's role and (2) the teaching of grammar similarly. Their view of the teacher is somewhat narrow: corrector of language problems. One wonders, then, why the experimental students did not complain more (only one comment) about the experimental teachers' (teachers E1 and E2) focus on other aspects of writing such as rhetorical problems supplanting language concerns. As well, both groups commented frequently on the importance of grammar instruction. One could assume that the experimental students were aware that their language/grammar needs were being attended to in other ways, that is, via the editing groups and the conference. Regarding topic selection, it appears that students at this level, particularly those taught by product methods, prefer easy topics. Topics relating to real-world events and places, used mainly in the writing workshop classrooms, appealed to learners in the experimental group. It 73 may be that the writing workshop provides learners with the necessary support to tackle more cognitively demanding writing tasks, possibly enhancing their growth in both oral and written communication. Implications A review of the literature on effects of instruction on students' writing performance indicates that while there is a growing awareness in mother-tongue (L1) research that some connection exists between instruction-specifically instruction involving collaborative inquiry and feedback-and performance outcomes, this awareness stops short of a controlled investigation into what that connection might be in the English as a second language (ESL) realm. This study reports conflicting results for the two measures, the informal and formal writing tasks. On the first measure, an informal, classroom-like writing situation with a paired-interactive prewriting component, it has already been noted that the slightly better performance of the control students who were unused to this kind of stimulation as preparation for writing may be accounted for by an interaction effect. However, on the second measure, a formal test-like situation thought to be more indicative than the first measure of the individual writer's efforts, the experience gained in the writing workshop appeared to benefit the experimental students more than the product approach helped students in the control group. Specifically, the writing workshop seemed to enhance students' performance in the content and organization aspects of written communication, and did not hinder their performance in structure and mechanics. Thus, one implication is that the variety of process or workshop approach used in this study can offer a viable alternative to product-oriented, grammar-based approaches to writing instruction in the adult ESL classroom at the high intermediate level of proficiency. This implication is important for several reasons: as noted in Chapter Two, some researchers (Hamp-Lyons, 1986; Piper, 1989) feel that evidence on the efficacy of the process approach will lend credence to the discussion whether the process orientation to writing merits a place in the ESL classroom; related to the first point-the efficacy of the approach-is the reluctance, also noted in Chapter Two, to implement a 74 process approach to writing instruction. If indeed the traditional focus on error-free product makes teachers (as well as students and administrators) fearful of abandoning an accepted approach such as the product orientation (regardless of whatever doubts or hesitations they may have about it) which is seen to guarantee an acceptable product in favor of one such as the workshop approach where the outcome is in doubt, perhaps the results of this study will provide some encouragement toward the implementation of alternative approaches such as the workshop to writing instruction in the ESL classroom. Implications regarding biodemographic interactions with treatment on performance must remain limited because of shortcomings-unequal cell distribution-in this part of the analysis. Evidence was found which seems to imply that a treatment involving an interactive paired or group prewriting component such as was provided by the informal, classroom-like writing measure might be more helpful for those ESL learners having spent little time, in this case less than two years, in the 12 environment than for those with more time and experience in it. If that is the case, a workshop approach may help newcomers accommodate or adjust to second language and second culture classroom writing demands more efficiently and effectively than a product orientation. Research findings (Katstra, 1987; Clifford, 1981; King, 1979; Beaven, 1977) indicate a strong connection between a variety of workshop approaches such as the use of peer groups and attitude toward writing. Only a few studies (Fox, 1980; Wolcott and Buhr, 1987) link treatment and affective measures to actual performance. The current study suggests that there is an interaction between the workshop treatment and performance as well as affective outcomes. Students in the workshop treatment reacted strongly and favorably toward the approach; in addition, their writing performance showed greater gains overall on the measure most indicative of individual performance-the formal test-than those found in the control group. One implication is that the implementation of a workshop approach to writing instruction is somehow related to more positive attitudes toward writing, and hence to increased motivation to perform well. The positive interaction that the findings suggest might be welcomed by ESL 75 teachers, who are aware that their students, coming from diverse language and cultural backgrounds with varying levels of education and learning styles, may arrive with expectations of the teacher and the classroom based on their L1 experience. This awareness of the potential to risk students' disapproval-and perhaps ultimately administrator's admonitions-may padly account for some ESL teachers' reluctance to implement an approach such as the writing workshop, which is almost certain to be seen by some students as novel and perhaps unacceptable. The findings of this study may counterbalance teachers' hesitations. Limitations The fact that the experimental and control sections consisted of intact classes rather than classes of students randomly assigned to sections is the main limitation of the present study. A common problem in educational research, the use of intact classes limits the current study in at least two ways: findings cannot be formally generalized to other ESL populations; cell distribution cannot be controlled for. The latter problem, cell distribution, severely constrained discussion in Part B of the study, the biodemographic variables and their potential interactions with the treatment. Secondly, because two different kinds of writing measures were used (informal and formal) instead of two similar measures (for example, both formal) and because results from the informal measure conflicted (with good reason, in retrospect) with those based on the formal measure, results are not as clear as otherwise might have been the case. Thirdly, the experiment was carried out at only one level of ESL ability, high intermediate, and in only four classes. Replication studies at other levels such as advanced ESL and university-transfer (also known as academic preparation) ESL and in a larger number of classes are needed in order to corroborate the current findings. Finally, the use of a post-instruction survey on attitudes toward writing instead of both a pre- and post-instruction survey limits any discussion of a causative nature regarding treatment by performance by attitude interactions. The use of a pre- and post-instruction survey on 76 attitudes toward writing, preferably a standardized one such as the ESL Writing Attitude Test (WAT) adapted by Gungle and Taylor (1990) from the Daly-Miller WAT, in addition to the use of the survey designed for this study, would have enhanced the current study. Unfortunately, the current study cannot benefit from such hindsight. Suggestions for future research Future research would likely benefit from the use of randomly-assigned students instead of intact classes, as mentioned above, to avoid problems of interpretation of data. As well, future studies would do well to consider using two similar writing measures, which might help stabilize results. Another adaptation of the present study might include an exploration of similar questions with higher level (e.g., university transfer) students as recommended previously to see if the findings hold across ability levels. To counter the problem of cultural differences, it might be best to include students only from one culture (e.g., Japanese, or Qu6becois as in Cumming, 1988). Regarding the addition of classes, potential problems might outweigh advantages. For example, for the current study, both experimental teachers synchronized the entire term and communicated regularly. A larger study might preclude this kind of cooperation. If the study were extended over a longer period of time, for example, two semesters instead of one, problems similar to those which might be encountered by adding classes may occur, negating any value in having a larger sample. A more elaborate component on attitude toward writing, incorporating the suggestions in the preceding section regarding the use of a pre- and post-writing attitude test such as the ESL WAT in addition to the attitude survey developed for this project, would provide data which might help establish the cause of interactions among treatment, performance and attitude. As well, if an interview component were added to the survey section on attitudes toward writing, the resulting information might go a long way toward answering some of the unanswered questions that were raised by the students' responses to the open-ended question about writing. For 77 example, it would be informative to know why students from both the workshop and the product groups focused on certain aspects such as the importance of grammar. Concerning students in the workshop group, it would be useful to determine why there were extremely few comments-either positive or negative-on the role of teacher as facilitator. A further promising area of investigation might include how successful a) peer revision and b) peer editing groups differ from unsuccessful ones in a writing workshop. Finally, ft might be fruitful to examine the relationship between teachers' and students' attitudes toward writing. Further inquiry in any of the above directions could provide additional data which might help teachers and administrators in making routine decisions about instructional approaches. 78 REFERENCES Ammon, P. (1985). Helping children leam to write in English as a second language: some observations and some hypotheses. In S.W. Freedman (Ed.) The Acquisition of Written Language: Response and Revision. Norwood, NJAblex, 65-84. Arndt, V. (1987). Six writers in search of texts: a protocol-based study of L1 and 12 writing. English Language Teaching Journal. 41, 257-267. Beaven, M.H. (1977). Individualized goal setting, self evaluation, and peer evaluation. In C.R. Cooper & L. Odell (Eds.) Evaluating Writing. Champaign, III: National Council of Teachers of English, 135-153. Benson, N.L (1979). The effects of peer feedback during the writing process on writing performance, revision behavior, and attitude toward writing. DAI 40:1987-A. Bereiter, C. (1980). Development in writing. In L.W. Gregg & E.R. Steinberg (Eds.) Cognitive Processes in Writing. Hillsdale, NJ: Lawrence Erlbaum Associates, 73-93. Bereiter, C. & Scardamalia, M. (1982). From conversation to composition: the role of instruction in a developmental process. In R. Glaser (Ed.) Advances in Instructional Psychology. Vol. 2. Hillsdale, NJ: Lawrence Erlbaum Associates, 1-64. Blanton, Linda L. (1987). Reshaping ESL students' perceptions of writing. ELT Journal. 41,2,112-8. Borg, W.R. & Gall, M.D. (1983). Educational Research: An Introduction. New York: Longman. Brubacher, M. & Payne, R. (1985). The team approach to small group learning. In M. Maguire & A. Pare (Eds.) Patterns of Development. Ottawa: Canadian Council of Teachers of English, 140-149. Calkins, L.M. (1983). Lessons from a child. London: Exeter Heinemann Educational Books. Carroll, J.A. (1984). Process into product: teacher awareness of the writing process affects students' written products. In R. Beach & L. Bridwell (Eds.) New Directions in Composition Research. New York: Guilford Press, 315-333. Chaudron, C. (1983). Evaluating writing: effects of feedback on revision. Paper presented at the Seventeenth Annual TESOL Convention, Toronto, Canada. Clifford, J.P. (1981). Composing in stages: the effects of a collaborative pedagogy. Research in the Teaching of English. 15,1, 37-53. Cohen, A. (1987). Student processing of feedback on their compositions. In A. Wenden &J. Rubin (Eds.) Learner Strategies in Language Learning. Oxford: Pergamon Press. Cohen, J. (1977). Statistical Power Analysis for the Behavioral Sciences. New York:NY: Academic Press. 79 Coleman, D.R. (1982). The effects of pupil use of a creative writing scale as an evaluative and instructional tool by primary gifted students. DAI 42: 3409-A. Collins, A. & Gentner, D. (1980). A framework for a cognitive theory of writing. In L.W. Gregg & E.R. Steinberg (Eds.) Cognitive Processes in Writing. Hillsdale, NJ: Lawrence Erlbaum Associates, 51-72. CourtJand, M.C. & Welsh, R. (1990). A case study of a teacher's changing perceptions of the writing process: The second and third years. English Quarterly. 23,1 & 2, 62-79. Cumming, A. (1988). Writing expertise and second-language proficiency. Language Learning. 38,4,1-72. Cumming, A. (1986). Intentional learning as a principle for ESL writing instruction: A case study. TESL Canada Journal, special issue 1, 69-83. Cummins, J. (1980). The cross-lingual dimensions of language proficiency: Implications for bilingual education and the optimal age issue. TESOL Quarterly. 14,175-187. Cummins, J. (1984). Bilingualism and special education: Issues in assessment and pedagogy. Clevedon, Avon: Multilingual Matters. Daly, J.A. (1985). Writing apprehension. In M. Rose (Ed.) When a Writer Can't Write: Studies in Writer's Black and Other Composing Process Problems. New York: Guilford, 43-82. Daly, J.A. & Miller, M.D. (1975b). The empirical development of an instrument to measure writing apprehension. Research in the Teaching of English. 9, 242-249. Daly, J. A., Vangelisti, A. & Witte, S.P. (1988). Writing apprehension in the classroom context. In B.A. Rafoth & D.L Rubin (Eds.) The Social Construction of Written Communication. Norwood, NJ: Ablex Publishing Corporation, 147-171. Davies, N.F. & Omberg, M. (1987). Peer group teaching and the composition class. System. 15, 3, 313-323. Edelsky, C. (1986). Writing in a bilingual program. Norwood, NJ: Ablex. Emig, J. & King, B. (1979). Emig-King attitude scale for students. New Brunswick, NJ: Rutgers University/Douglass College. (ERIC Document Reproduction Service No. ED 236 630). Faigley, L., Cherry, R., Jolliffe, D., & Skinner, A. (1985). Assessing Writers' Knowledge and Processes of Composing. Norwood, NJ: Ablex. Rower, L. (1979). Writer-based prose: a cognitive basis for problems in writing. College English. 41,1,19-37. Rower, L. & Hayes, J. (1980). The dynamics of composing: Making plans and juggling constraints. In LW. Gregg & E.R. Steinberg (Eds.) Cognitive Processes in Writing. Hillsdale, NJ: Lawrence Erlbaum Associates, 31-50. Glass, G.V. & Hopkins, K.D. (1984). Statistical Methods i n Education and Psychology ( 2nd ed.). Englewood Cliffs, N.J.: Prentice Hall. 80 Glass, G.V., McGaw, B., & Smith, M.L (1981). Meta-analysis in Social Research. Beverly Hill, CA: Sage. . Graner, M. (1985). Revision Techniques: peer editing and the revision workshop. DAI 47:109-A. Grimm, N. (1986). Improving students' responses to their peers' essays. College Composition and Communication. 37, 91-94. Gungle, B.W. & Taylor, V. (1989). Writing apprehension and second language writers. In D.M. Johnson & D.H. Roen (Eds.) Richness In Writing: Empowering ESL Students. New York: Longman, 235-248. Hairston, M. (1982). The winds of change: Thomas Kuhn and the revolution in the teaching of writing. College Composition and Communication , 33, 76-88. Hamp-Lyons, L. (1986). No new lamps for old yet, please. Forum, TESOL Quarterly.20.790-6. Hillocks, G. (1986). Research on Written Composition. Urbana, IL: Eric Clearinghouse on Reading and Communication Skills, NCRE. Horowitz, D.M. (1986a). Process, not product: less than meets the eye. Forum, TESOL Quarterly. 20.141-4. Horowitz, D.M. (1986c). The author responds to Liebman-Kleine.... Forum, TESOL Quarterly. 20, 788-789. Horowitz, D.M. (1986d). The author responds to Hamp-Lyons.... Forum, TESOL Quarterly. 20, 796-797. Hudelson, S. (1984). Kan yu ret an rayt en ingles: children become literate in English as a second language. TESOL Quarterly , 18, 221-238. Jacobs, H.L., Zinkgraf, S.A., Wormuth, D.R., Hartfiel, V.F., & Hughey, J.B. (1981). Testing ESL Composition: A practical approach. Rowley, MA: Newbury House. Jeroski, S.F. (1982). Competence in written expression: Interactions between instruction and individual differences among junior high school students. Unpublished Doctoral dissertation, University of British Columbia, Vancouver, British Columbia. Jeroski, S.F. & Conry, R.F. (1981). Development and field application of the attitude toward writing scale. Paper presented at the annual conference of the American Educational Research Association, Los Angeles, USA. Jones, S. (1985). Problems with monitor use in second language composing. In M. Rose (Ed.) When a writer can't write. New York: Guilford Press, 96-118. Jones, S. & Tetroe, J. (1987). Composing in a second language. In A. Matsuhashi (Ed.) Writing in Real Time. Norwood, NJ: Ablex, 34-57. Kantor, K.J. (1984). Classroom contexts and the development of writing intuitions: an ethnographic case study. In R. Beach & L. Bridwell (Eds.) New Directions in Composition Research. New York: Guilford Press, 72-94. 81 Kaplan, R. (1967). Contrastive rhetoric and the teaching of composition. TESOL Quarteriy. 1,10-16. Katstra, Joyce, Tollefson, N. & Gilbert, E. (1987). The effects of peer evaluation on attitude toward writing and writing fluency of ninth grade students. Journal of Educational Research. 80, 3,168-172. King, B. (1979). Measuring attitudes toward writing: The King construct scale. Presented at the College Conference on Composition and Communication, Minneapolis, MN. (ERIC Document Reproduction Service No. ED 172 258). Kreeft, J., Shuy, R. W., Staton, J., Reed, L, & Morroy, R. (1984). Dialogue writing: analysis of student-teacher interactive writing in the learning of English as a second language. Final Report to the National Institute of Education, NIE-G-83-0030. Washington, DC: Center for Applied Linguistics. (ERIC DRS No. ED 252-097). Ling, S. & Rothschild, D. (I988). Oral and Written Composing. Vancouver Community College. Marascuilo, LA. & Seriin, R.C. (1988). Statistical Methods for the Social and Behavioural Sciences. New York, NY: W.H. Freeman and Company. Matsuhashi, A. (Ed.) (1987). Writing in Real Time: Modelling Production Processes. Norwood, NJ: Ablex Publishing Corporation. Mittan, R. (1989). The peer review process: harnessing students' communicative power. In D.M. Johnson & D.H. Roen (Eds.) Richness in Writing: Empowering ESL Students. NY: Longman, 207-219. Mohan, B. & Lo, W. (1985). Academic writing and Chinese students: Transfer and developmental Newkirk, T. (1984). Anatomy of a breakthrough. In R. Beach & L. Bridwell (Eds.) New Directions in Composition Research. New York: Guilford Press, 143-148. Parks, S. (1987). Effects of instruction on revision. Paper presented at the TESL Canada Conference, Vancouver, Canada. Partridge, K. L (1981). A comparison of the effectiveness of peer vs. teacher evaluation for helping students of English as a second language to improve the quality of their written compositions. M.A. in TESL thesis, University of Hawaii at Manoa. Piper, A. (1989). Writing instruction and the development of ESL writing skills: is there a relationship? System. 17, 2, 211-222. Poliakoff, N.K. (1990). Eliciting the language of decision making through collaborative revision of compositions. Master's thesis, University of British Columbia, Vancouver, Canada Raimes, A. (1984). Anguish as a second language? Remedies for composition teachers. In S. McKay (Ed.) Composing in a Second Language. Rowley, MA: Newbury House. Raimes, A.(1985). What unskilled ESL students do as they write. TESOL Quarterly.19. 229-258. Raimes, A. (1986). Teaching Writing: What we know and what we do. Paper presented at the 20th Annual TESOL Convention, Anaheim, CA. 82 Rose, M. (1980). Rigid rules, inflexible plans, and the stifling of language: a cognitivist analysis of writer's block. College Composition and Communication. 31, 389-401. Rose, M. (1981). Sophisticated, ineffective books-the dismantling of process in composition texts. College Composition and Communication. 32.1. 65-74. Rose, M. (Ed.) (1985). When a writer can't write. New York: Guilford Press. Rothschild, D. & Klingenberg, K. (1990). Self and peer evaluation of writing in the interactive ESL classroom: an exploratory stucty. TESL Canada Journal. 8,1, 52-65. Sager, C. (1973). Improving the quality of written composition through pupil use of rating scale. DAI 34:1496-A. Silva, A. (1989). A critical review of ESL composing process research. Paper presented at the 23rd annual TESOL Convention, San Antonio, USA. Sinclair, V. (1983). Mode and topic effects on complexity in advanced composition. TEAL Occasional Papers. No. 7. Sola, M. & Bennett, AT. (1985). The struggle for voice: narrative, literacy and consciousness in an East Harlem school. Boston University Journal of Education. 167. 88-110. Spack, R. (1984). Invention strategies and the ESL college composition student. TESOL Quarjgrjv. 20, 649-670. Stein, N.L (1984). Critical issues in the development of literacy education: toward a theory of learning and instruction. American Journal of Education. 93,171-199. Stein, N.L. (1986). Knowledge and process in the acquisition of writing skills. In E.Z. Rothkopf (Ed.) Review of Research in Education. No. 13. Washington: American Educational Research Association. Taylor, B. (1981). Content and written form: A two-way street. TESOL Quarterly. 15, 5-12. Wolcott, W. & Buhr, D. (1987). Attitude as it affects developmental writers' essays. Journal of Zamel, V. (1976). Teaching composition in the ESL classroom: What we can learn from research in the teaching of English. TESOL Quarterly. 10, 67-76. Zamel, V. (1983). The composing processes of advanced ESL students: Six case studies. Zamel, V. (1987). Recent research on writing pedagogy. TESOL Quarterly. 21, pp. 697-715. Zoellner, R. (1969). Talk-Write: a behavioral pedagogy for composition. College English. 30, 267-320. 83 Appendix A: Peer revision focus sheet T O R E V I S E D R A F T 1 S T U D E N T S P A G E Students sit in a group with a tape - recorder o n ( D E S C R I P T I V E ) 1. 2. R E P E A T E D W R I T E R R E A D S : T I T L E of compos i t ion I N T R O D U C T I O N (Paragraph 1) B O D Y of compos i t ion P A R A G R A P H B Y P A R A G R A P H R E P E A T S / E X P L A I N S , IF N E C E S S A R Y . C O N C L U S I O N (last paragraph) . L I S T E N E R S P R E D I C T : M a y b e this compos i t ion is about D I S C U S S : A . Su i tab le? B. C l e a r ? Y E S N O IF N O T , D E L E T E / A D D D I S C U S S : Pa rag raph 2, 3 , 4, 5 , etc. A . It is mostly about B. Information miss ing on C . Th ings to be taken out Q U E S T I O N W R I T E R , IF N E C E S S A R Y . D I S C U S S : A . Su i tab le? B. C l e a r ? Y E S N O IF N O T , D E L E T E / A D D D I S C U S S T O G E T H E R T H E C O M P O S I T I O N A S F O L L O W S : A . Is the title O K ? Why /why not? B. W h a t do you like best about the writing? W R I T E R , using tape, rewrites into draft 2. K e e p s it for editing. N O T E : Turn to p a g e 73 w h e n you are ready for EDIT ING. from Oral and Written Composing by Ling & Rothschild 84 Appendix B: Peer editing focus sheet S T U D E N T ' S P A G E T E C H N I Q U E T O EDIT: P E E R EDITING B E F O R E C L A S S : 1. A. G i v e your teacher draft 2 (the revised draft) so that he can make enough photocopies for your group. S T E P S IN C L A S S : 1. B. Sit in a group. M a k e sure you have i. a good \"grammarian\" and a good \"speller\" in your group; ii. a dictionary, a grammar reference book and the c l a s s editing key o n the table. 2. The W R I T E R (the chief editor) G i v e s a copy of his draft 2 to \u00E2\u0080\u00A2 G R O U P (called editors) e a c h editor in the group 3. A . R e a d s a loud one sentence. S tops . A s k s , \"Is it O K ? \" \u00E2\u0080\u00A2 L istens and s a y s \" Y e s / N o \" B. If not O K , writes the symbol in the margin on the line where the error is. If O K , cont inues with the next sentence . D o e s the s a m e as the chief editor. I I C. W h e n vou are not sure or w h e n you disagree. i. U s e your g rammar book and the dictionary; ii. Write down what the group thinks is best, for the t ime be ing ; O R iii. Put quest ion marks in the margin and ask the teacher later on , or at | Conferenc ing time. N O T E : Y o u may not be able to find all the mistakes , but you will find some . i 4. W h e n the writer f in ishes editing his draft 2, he b e c o m e s one of the editors in the group. 5. C h o o s e another student's draft 2. Fol low steps 2 - 4 above . N O T E : Turn to page 80 for P O S T EDIT ING T A S K S . from Oral and Written Composing by Ling & Rothschild 85 Appendix C: The ESL Composition Profile/Sager Scale (adapted) Rating Scale for composition no. CONTENT: ideas, information or message 27-35 The reader can see & feel what the writer sees & feels \u00E2\u0080\u00A2 ideas & details create an impression on the reader \u00E2\u0080\u00A2 all ideas are clear & fully developed \u00E2\u0080\u00A2 all ideas are nicely related to each other & to the title \u00E2\u0080\u00A2 details fill out the ideas & make people, places &/o r events come alive \u00E2\u0080\u00A2 all questions are answered (a sense of completeness) IX-26 the reader begins to see & feel what the writer does * ideas & details start to make an impression \u00E2\u0080\u00A2 most ideas are clear & well-developed \u00E2\u0080\u00A2 most ideas are related to each other & to the title but some details don't belong \u00E2\u0080\u00A2 details begin to Fdl out the ideas & make people, places &/or events come alive \u00E2\u0080\u00A2 a few questions are unanswered because some details are missing 9-17 the reader doesn't see & feel what the writer docs \u00E2\u0080\u00A2 ideas & details make little or no impression \u00E2\u0080\u00A2 some ideas are clear & developed \u00E2\u0080\u00A2 some ideas & detail are not related to each other & to the title \u00E2\u0080\u00A2 there aren't enough details to fdl out the ideas; as a result, people, places &/or events don't come alive \u00E2\u0080\u00A2 many questions are unanswered because of missing details 0-8 the ideas make no impression on the reader because they are confusing, hard to follow, unclear &/or undeveloped \u00E2\u0080\u00A2 ideas & details don't seem to be related to each other & to the title \u00E2\u0080\u00A2 important questions are unanswered because there are no or almost no details ORGANIZATION: the arrangement of ideas in order 19-25 the introduction is interesting: it makes the reader want to continue reading \u00E2\u0080\u00A2 the conclusion helps the reader understand the writer's point of view &/o r feelings \u00E2\u0080\u00A2 a main idea ties all story parts together in an obvious logical order \u00E2\u0080\u00A2 each paragraph has only 1 main idea or purpose & all details support that idea 12-18 there is an introduction but it doesn't grab the reader's attention \u00E2\u0080\u00A2 there is a conclusion but it doesn't help the reader understand the writer's point of view &/or feelings \u00E2\u0080\u00A2 a main idea ties all story parts together but some events are told out of order \u00E2\u0080\u00A2 some paragraphs have more than 1 main idea or purpose, or no obvious main idea or purpose 6-11 the introduction/conclusion is not useful or interesting \u00E2\u0080\u00A2 there is a main idea but many events are out of order \u00E2\u0080\u00A2 many paragraphs don't have 1 main idea or purpose 0-5 no introduction or conclusion \u00E2\u0080\u00A2 the reader doesn't see point to story because ideas are so disorganized 86 Appendix C continued STRUCTURE: the way language is used 23-30 clauses are joined in the most effective/meaningful way by connectors (e.g. if, so . . . that, although, because as. since, who, which, where, whose, that, when, etc.) \u00E2\u0080\u00A2 each sentence is complete (no run-ons or frag-ments) \u00E2\u0080\u00A2 there are almost no errors of agreement, tense, articles, word order, word form, prepositions, etc. 15-22 clauses are not always joined in the most effective way but the ideas are still easy to understand \u00E2\u0080\u00A2 almost all sentences arc complete (few run-ons or fragments \u00E2\u0080\u00A2 there are some errors of agreement, tense, articles, word order, etc. but we can easily understand the story 7-14 clauses are not joined or are poorly joined so that ideas are sometimes difficult to understand \u00E2\u0080\u00A2 some sentences are complete (some run-ons or fragments) \u00E2\u0080\u00A2 there arc many errors of agreement, tense, articles, word order, etc. and sometimes they make the story difficult to understand 0-6 ideas/meaning is unclear or difficult to understand \u00E2\u0080\u00A2 few sentences are complete (many run-ons or fragments) \u00E2\u0080\u00A2 dominated by errors of agreement, tense, articles, word order, etc. which make the story confusing MECHANICS: the way the writing looks 7-10 almost no errors of spelling, punctuation, capitalization or paragraphing \u00E2\u0080\u00A2 handwriting is legible 4-6 some errors of spelling, punctuation, capitalization or paragraphing but this does not usually interfere with understanding the story \u00E2\u0080\u00A2 handwriting is usually easy to read 0-3 many errors of spelling, etc. which may make the story difficult to under-stand \u00E2\u0080\u00A2 handwriting is hard to read WRITER STUDENT I STUDENT 2 TEACHER AVERAGE CONTENT ORGANIZATION STRUCTURE MECHANICS T O T A L Adapted from the \"ESL Composition Profile\" and the \"Sager Scale\" 87 Appendix D: The background survey (adapted with permission from Cumming, 1988) UPPER INTERMEDIATE QUESTIONNAIRE 1. Name 2. L a n g u a g e s p o k e n a t home 3. Number o f y e a r s o f e d u c a t i o n 4 . Number o f y e a r s l i v i n g I n an E n g l i s h - s p e a k i n g c o u n t r y 5. Can y o u , o r c o u l d y o u , i n y o u r n a t i v e l a n g u a g e a ) w r i t e l e t t e r s t o y o u r f r i e n d s and f a m i l y ? b) w r i t e l e t t e r s t o o t h e r s ( e . g . , a b u s i n e s s l e t t e r , a l e t t e r o f a p p l i c a t i o n , e t c . ) ? c ) w r i t e s h o r t r e p o r t s a t s c h o o l o r w o r k ' d) w r i t e l o n g r e p o r t s a t s c h o o l o r w o r k ? * I 3 A 1 2 3 4 1 2 3 4 1 2 3 4 88 Appendix E: The survey on attitude toward writing TO: A l l ELS__U.I. Students FROM: DATE: November 2 9, 1988 SUBJECT: EVALUATION OF WRITING AT THE U.I. LEVEL The E.L.S. Department i s i n t e r e s t e d i n what you t h i n k about the way we te a c h w r i t i n g a t the U.I. l e v e l . It would help us i f you could answer these q u e s t i o n s f o r us. Thank you very much. agree d i s a g r e e 1. My w r i t i n g i s b e t t e r than in September 2. I have more ideas f o r w r i t i n g now. 3. W r i t i n g i s e a s i e r f o r me. 4. My w r i t i n g i s b e t t e r o rganized. 5. I am more comfortable w r i t i n g now. 6. My w r i t i n g has more d e t a i l s now. 7. I understand the kinds of mistakes I make now. 8. My grammar has improved. 9. I have more vocabulary now. 10. I enjoy w r i t i n g now. What d i d you l i k e best about w r i t i n g in your c l a s s ? 89 STUDENT'S PAGE PREWRITING: Appendix F: Topics for the writing tests PRE-TEST 1 (Informal test: topic 1) INTERVIEWING .1. 2. 3. S i t with a partner. One at a time, interview each other. Ask each other the questions that your c l a s s and your teacher made up together. Make sure you understand e v e r y t h i n g about your p a r t n e r ' s s t o r y . If you are 'not sure, ask your partner to e x p l a i n . If you want to take notes to help you remember your partner's s t o r y , please do. Use the space below or the back of t h i s sheet for your notes. (job, WHAT educat ion) \u00C2\u00ABtc. WRITING: 1. W r i t e a s t o r y a b o u t your p a r t n e r so t h a t y o u r t e a c h e r has c l e a r b a c k g r o u n d i n f o r m a t i o n a b o u t him or h e r . 2. W r i t e i n COMPLETE SENTENCES, not p o i n t form. 3. Your TIME LIMIT i s 45 m i n u t e s . You w i l l n o t have t i m e to check y o u r d i c t i o n a r y o r grammar book, ask a b o u t s p e l l i n g , or t a l k t o y o u r p a r t n e r a g a i n . The most i m p o r t a n t t h i n g i s t o g e t t h e i n f o r m a t i o n down. 4. Your t e a c h e r w i l l t e l l you when 5 m i n u t e s r e m a i n so t h a t you can f i n i s h . 5. W r i t e YOUR NAME and YOUR TEACHER'S NAME a t t h e t o p of e a c h page of your s t o r y . Appendix F continued 90 STUDENT'S PAGE COMPOSITION: TOPIC 2 (Formal test) A. GENERAL INSTRUCTIONS P l e a s e put the f o l l o w i n g i n f o r m a t i o n at the top of each page: 1. your name 2 . your t e a c h e r ' s name 3. the date You w i l l have one hour to w r i t e a c o m p o s i t i o n on the t o p i c g i v e n below. Your t e a c h e r w i l l t e l l you when t e n minutes remain so t h a t you w i l l be a b l e to f i n i s h w i t h i n the time l i m i t . Because you o n l y have one hour, you w i l l not be a l l o w e d to check any grammar books or use a d i c t i o n a r y . I t i s more i m p o r t a n t to get your ideas down on paper b e f o r e you f o r g e t them. Your c o m p o s i t i o n does not have to be any s p e c i a l l e n g t h . W r i t e as much as you have to say about the t o p i c . P l e a s e w r i t e on every l i n e ( s i n g l e - s p a c e d ) . P l e a s e do your b e s t . T h i s c o m p o s i t i o n w i l l h e l p your t e a c h e r t e a c h you t h i s term. If you would l i k e to make notes b e f o r e w r i t i n g , p l e a s e use the bottom or back of t h i s sheet f o r your n o t e s . B. SPECIFIC INSTRUCTIONS P l e a s e w r i t e your c o m p o s i t i o n on the t o p i c \"My F a v o r i t e (or s p e c i a l ) P l a c e \" or \"A P l a c e That i s Important to Me\". You may choose any p l a c e you want, i n any c o u n t r y . You may w r i t e about t h i s p l a c e i n any way you wish. For example, you may d e s c r i b e i t or t e 1 1 what happens or happened to you t h e r e . You may i n c l u d e both r e a l and imaginary d e t a i l s . Appendix F continued 91 STUDENT'S PAGE COMPOSITION: TOPIC 3 (Formal test) A. GENERAL INSTRUCTIONS P l e a s e put the f o l l o w i n g i n f o r m a t i o n a t the top of each page: 1. your name 2 . your t e a c h e r ' s name 3. the date You w i l l have one hour to w r i t e a c o m p o s i t i o n on the t o p i c g i v e n below. Your t e a c h e r w i l l t e l l you when t e n minutes remain so t h a t you w i l l be a b l e to f i n i s h w i t h i n the time l i m i t . Because you o n l y have one hour, you w i l l not be a l l o w e d to check any grammar books or use a d i c t i o n a r y . It i s more i m p o r t a n t t o get your ideas down on paper b e f o r e you f o r g e t them. Your c o m p o s i t i o n does not have to be any s p e c i a l l e n g t h . W r i t e as much as you have to say about the t o p i c . P l e a s e w r i t e on every l i n e ( s i n g l e - s p a c e d ) . P l e a s e do your b e s t . T h i s c o m p o s i t i o n w i l l h e l p your t e a c h e r t e a c h you t h i s term. If you would l i k e t o make notes b e f o r e w r i t i n g , p l e a s e use the bottom or back of t h i s sheet f o r your n o t e s . B. SPECIFIC INSTRUCTIONS P l e a s e w r i t e your c o m p o s i t i o n on the t o p i c \"My F a v o r i t e ( o r s p e c i a l ) P e r s o n \" or \"A Person That i s Important to t i e \" . You may choose any person you want. You may w r i t e about t h i s person i n any way you wi s h . For example, you may d e s c r i b e him or her or w r i t e about your r e l a t i o n s h i p w i t h t h i s person. You may i n c l u d e both r e a l and ima g i n a r y d e t a i l s . 92 Appendix F continued STUDENT'S PAGE C O M P O S I T I O N : P O S T - T E S T 1 (Informal t e s t : topic 4) A. GENERAL I N S T R U C T I O N S P l e a s e put the f o l l o w i n g i n f o r m a t i o n at the top of each page: 1. your name 2 . your t e a c h e r ' s name 3. the date You w i l l have one hour t o w r i t e a c o m p o s i t i o n on the t o p i c g i v e n below. Your t e a c h e r w i l l t e l l you when ten minutes remain so t h a t you w i l l be a b l e to f i n i s h w i t h i n the t i m e . l i m i t . Because you o n l y have one hour, you w i l l not be a l l o w e d to check any grammar books or use a d i c t i o n a r y . I t i s more i m p o r t a n t t o get your i d e a s down on paper b e f o r e you f o r g e t them. Your c o m p o s i t i o n does not have to be any s p e c i a l l e n g t h . W r i t e as much as you have to say about the t o p i c . P l e a s e w r i t e on every l i n e ( s i n g 1 e - s p a c e d ) . P l e a s e do your b e s t . T h i s c o m p o s i t i o n w i l l h e l p your t e a c h e r f i n d out how much you l e a r n e d t h i s term. If you would l i k e to make notes b e f o r e w r i t i n g , p l e a s e use the bottom or back of t h i s sheet f o r your n o t e s . B. SPECIFIC INSTRUCTIONS Look a t your p e n c i l . P r e t e n d t h a t you are t h a t p e n c i l . W r i t e a c o m p o s i t i o n on the t o p i c \"My L i f e As A P e n c i l \" . You may w r i t e about your l i f e as a p e n c i l i n any way you wis h . Try to w r i t e a good and c o n v i n c i n g s t o r y so t h a t your reader can und e r s t a n d something about your l i f e as a p e n c i l , share your e x p e r i e n c e and see your s p e c i a l q u a l i t i e s . 93 Appendix G: Teachers' instructions for the writing tests TEACHER'S PAGE PRE-TEST 1 INTERVIEWING (Informal) MATERIALS: s t o r i e s b a s e d on s t u d e n t s ' i n t e r v i e w s CLASS ORGANIZATION: whole c l a s s ( e x p l a n a t i o n and d e m o n s t r a t i o n ) p a i r s ( i n t e r v i e w i n g ) s o l o ( w r i t i n g ) PROCEDURE: PREWRITING 1. T e l l t he s t u d e n t s t h a t p a i r s of s t u d e n t s w i l l i n t e r v i e w e ach o t h e r and th e n each s t u d e n t w i l l w r i t e a s t o r y a bout t h e o t h e r . 2. E x p l a i n t h e p u r p o s e of t h e t a s k : t h e t e a c h e r needs b a c k g r o u n d i n f o r m a t i o n a b o u t each s t u d e n t . 3. BRAINSTORM w i t h the s t u d e n t s f o r t y p e of i n f o r m a t i o n t o be i n c l u d e d ; e.g. i f t h e y t h i n k c o u n t r y of o r i g i n i s i m p o r t a n t , put a WHERE wheel on t h e b l a c k b o a r d and on one of t h e spokes w r i t e down t h e c o u n t r y . Ask them w h a t / i f o t h e r i n f o r m a t i o n c a n go on t h e WHERE s p o k e s . 4. C o n t i n u e u n t i l you enough i n f o r m a t i o n wheels on t h e bbd and t h e n ask s t u d e n t s t o cho o s e the 5 or 6 most i m p o r t a n t ones f o r t h i s t a s k . S u g g e s t i o n s : WHO WHEN WHAT ( j o b e x p e r i e n c e , e d u c a t i o n , t r a i n i n g ) WHY ( t a k e t h i s c l a s s , l e a v e y o u r c o u n t r y ) 5. You may want t o DEMONSTRATE an i n t e r v i e w by h a v i n g t h e whole c l a s s i n t e r v i e w a v o l u n t e e r s t u d e n t . 6. Then have t h e s t u d e n t s s i t i n p a i r s f o r t h e i n t e r v i e w s . L e t s t u d e n t s d e c i d e whether t o t a k e n o t e s . WRITING 7. A f t e r t h e i n t e r v i e w s , ask t h e s t u d e n t s t o w r i t e a s t o r y about t h e i r p a r t n e r . Remind them t o use COMPLETE SENTENCES and n o t p o i n t form. D e m o n s t r a t e t h i s on t h e bbd. 8. T e l l them t h e TIME LIMIT i s 45 m i n u t e s maximum and t h e y w i l l n o t have time t o c h e c k t h e d i c t i o n a r y o r t h e i r grammar books, ask a b o u t s p e l l i n g , t a l k t o t h e i r p a r t n e r , e t c . T e l l them t h e most i m p o r t a n t t h i n g i s t o get the i n f o r m a t i o n down. 9. T e l l them when 5 m i n u t e s r e m a i n so t h a t t h e y c a n f i n i s h w i t h i n t h e t i m e 1 i m i t . NOTE TO TEACHERS P l e a s e hand i n o r i g i n a l s . If you'd l i k e t h e i n f o r m a t i o n f o r yo u r f i l e s , p l e a s e make a p h o t o c o p y . 94 Appendix G continued TEACHERS' INSTRUCTIONS FOR PRE-TEST 2 (Formal) BEFORE you g i v e out the students' page, pl e a s e 1. go over P a r t A General I n s t r u c t i o n s o r a l l y 2. e x p l a i n t h a t P a r t B S p e c i f i c I n s t r u c t i o n s t e l l s them what to w r i t e about and gi v e s a l l the i n f o r m a t i o n they need. You w i l l not go over i t with them. There i s no need to e x p l a i n t h a t the t o p i c s are d i f f e r e n t or why. 3. e x p l a i n ( i f necessary) the d i f f e r e n c e between \" r e a l and imagi-nary\", and make sure they understand t h a t t h e i r s t o r i e s may be l e s s than 100% t r u t h f u l . 4. t e l l them to hand i n both the student's page (with notes) and the t e s t as soon as they are done. They may then continue w i t h other work, or go to the l i b r a r y , e t c . HAND OUT the student's page. RANDOMLY g i v e h a l f the c l a s s t o p i c 2 and the other h a l f t o p i c 3. For example, you might g i v e every 2nd person on your c l a s s l i s t t o p i c 2, e t c . Remind the students when 10 minutes are remaining. Please r e t u r n the t e s t s & students' pages to me ASAP. Don't d i s c u s s with the students. Don't encourage them to com-pare t o p i c s . 95 Appendix G continued TEACHER'S PAGE COMPOSITION: POST-TEST 1 (Informal) A. BEFORE/DURING THE TEST 1. Do the p r e w r i t i n g s t e p s from the t e a c h e r ' s page ( s t e p s 1, 2 and 3 o n l y ) of U n i t 2, 0 b s e r v i n g A P e n c i l , from O r a l and W r i t t e n Composing. 2. A f t e r the o r a l a c t i v i t y i n which a l l s t u d e n t s have i d e n t i f i e d t h e i r p a r t n e r ' s p e n c i l s and d i s c u s s e d / u n d e r l i n e d u s e f u l d e t a i l s , make sure t h a t each stud e n t keeps h i s p e n c i l . 3. E x p l a i n to s t u d e n t s t h a t the c o m p o s i t i o n t e s t w i l l b e g i n . Tel 1 them t h a t they wi 1 1 have one hour o n l y and t h a t you wi 1 1 warn them when 10 m i n u t e s remain so t h a t they can f i n i s h w i t h i n the time l i m i t . 4 . E x p l a i n t h a t they are not a l l o w e d to use d i c t i o n a r i e s , grammar books, or each o t h e r . 5. D i s t r i b u t e l i n e d t e s t paper. Hand out the t e s t . 6 . The t e s t t o p i c i s s e l f - e x p l a n a t o r y . Do not dev e l o p , e x p l a i n or d i s c u s s the t o p i c w i t h the s t u d e n t s . You may, however, c o n f i r m t h a t the s t u d e n t s are to pretend to be a p e n c i l and imagine l i f e as one. 7. C o l l e c t a l l c o m p o s i t i o n t e s t s a f t e r the hour i s up. Do not g i v e e x t r a t i me. B. AFTER THE TEST 1. Give the o r i g i n a l s to Dennie. 2. You may make p h o t o c o p i e s of the t e s t and use them w i t h the c l a s s i n any way you wish. For example, you may want to use the Departmental c r i t e r i a to g i v e a mark. Or you may want s t u d e n t s to do another d r a f t , an e r r o r a n a l y s i s , e t c . 3. T h i s t e s t ( p o s t - t e s t 1) and the one the f o l l o w i n g week ( p o s t -t e s t 2) w i l l be marked as soon as p o s s i b l e . They wi11 be marked u s i n g a d i f f e r e n t s e t of c r i t e r i a . The s c o r e w i l l be out of 100. 96 Appendix G continued J - E A C H E R S : I N S T J i y C J l l u j M S FOR P O S I z T E B J 2 (Formal) B E F O R E GI.VING__jDUT THE STUDENT1.S PAGE 1 ,, g o o v e r P a r t A G e n e r a l I n s t r u c t i o n s o r a l l y 2:\u00E2\u0080\u009E e x p l a i n t h a t P a r t B S p e c i f i c I n s t r u c t i o n s t e l l s t h e m w h a t t o w r i t e a b o u t a n d q i v e s a l l t h e i n f o r m a t i o n t h e y need., Y o u w i l l n o t g o o v e r i t w i t h t h e m . T h e r e i s no n e e d t o e x p l a i n t h a t t h e t o p i c s a r e d i f f e r e n t o r why 3\u00E2\u0080\u009E e x p l a i n ( i f n e c e s s a r y ! t h e d i f f e r e n c e b e t w e e n \" r e a l a n d i m a g i n a r y \" , a n d make s u r e t h e y u n d e r s t a n d t h a t t h e i r s t o r i e s may b e l e s s t h a n 1 0 0 % t r u t h f u l 4. t e l l t h e m t h e y w i l l h a n d i n b o t h . t h e s t u d e n t ' s p a g e ( w i t h n ame, #, a n d n o t e s ) a n d t h e c o m p o s i t i o n t e s t HANOI M B Q i J T T H E , T E S T 1. e a c h s t u d e n t . w i l l w r i t e on t h e a l t e r n a t e t o p i c he d i d n o t r e c e i v e i n September 2 . each s t u d e n t ' s name and # i s w r i t t e n a t t h e top o f t h e s t u d e n t ' s page each i s t o w r i t e a b o u t 3. a s you hand out the t e s t s t o t h e students,, remind them t h e y w i l l have to hand i n t h i s s h e e t a l o n g w i t h t h e i r c o m p o s i t i o n 4. i f t h e y ask you what t h e number means, t e l l them i t ' s j u s t a number and e v e r y s t u d e n t has one! DURING AMD AFTER THE TEST 1. d o n ' t d i s c u s s t h e t o p i c w i t h t h e s t u d e n t s 2 . r e m i n d them when 1 0 m i n u t e s a r e r e m a i n i n g 3. c o l l e c t a l l p a p e r s a f t e r t h e hour i s up 4. a t t a c h ( s t a p l e , c l i p ) t h e s t u d e n t ' s page t o t h e c o m p o s i t i o n t e s t _ 5. you may make p h o t o c o p i e s o f t h e t e s t and do any f o l l o w \u00E2\u0080\u0094 u p you wi s h 6. p l e a s e r e t u r n a t t a c h e d s t u d e n t ' s page ?< c o m p o s i t i o n t e s t s t o D e n n i e ASAP 97 Appendix H: Mark sheet for composition rating RATER: d a t e C O M P . # C O N T E N T O R G A N I Z ' N L A N G U A G E M E C H A N I C S T O T A L 98 Appendix I: Students' unedited comments (by treatment and by class) elicited by the open-ended question of the writing attitude survey Control Class 1 Category 1: Process of Writing 1) I get more detail from picture composition. It gives me information to imagine 2) Frequently teacher gave us a few pictures and we explained these pictures, which gave us more idea and it was usefull for me 3) I like to choose a kind of picture which we have to write down something about from the picture. From the picture we can imagine anything we want to explain 4) I liked to write something with a picture which showed us a few ideas 5) and we could discuss in a group then it made us easy to write 6) I like writing about seeing picture. Writing story whaf s happing in the picture 7) when we do in group we can write better Category 2: General Attitude to Writing I) and when ever my teacher gave me a topic to write a composition I was happy to write 2) I would like to get used to some vocabulary in every writing 3) It is worth to do in order to learn more English language 4) Has improved more grammar and composition 5) Secondly pratices more writing structure or esay 6) I don't like writing 7) I would like to say I like composition 8) I enjoy writing everything in the class 9) paragraph using vocabulary in paragraph that teacher show us 10) I like best the short paragraph composition which only spend about 20-30 minutes II) 1 think grammer the best thing to help for writing Category 3: The Teacher's Role 1) and my teacher X help me very much 2) and our teacher explain about it and give the some idea Category 4: General Attitude to Class 1) Since I started my class I never miss class even one day 2) I like improve my writing and speaking together 3) Grammar is very interested to me 4) and I was got good marks Category 5: Topic 1) I liked best about writing history in our class 2) I like writing about carton because it is easy for me to explain 99 Appendix I continued Control Class 2 Category 1: Process of Writing 1) I like the compositions which have written in different topics every week because I have some problems about the skills of writing, so I can learn or make sure which way is good 2) I liked noticing in punctuation very much and also about spelling because those are were my reall weakness 3) the best thing about writing that I like it is rewriting the same subjack tow or tree time and that's help me to find my mistakes 4) I liked best about writing in my class which was that we had three draft to practice Category 2: General Attitude to Writing 1) I like best in my class to write the compositions because I have to use a lot of vocabulary, grammar 2) I would like write about composition 3) I think my writing is getting better in this class 4) I liked the best about writing in my class was the grammer, 'Comp Comp' and sometimes the comp Category 3: The Teacher's Role 1) Examples were given in class 2) I like best in my class writing that first I write composition and after teacher makes corrections I can see my mistakes that is very helpful for me Category 4: General Attitude to Class 1) I like to learn a new word like Adverb Clauses, Noun Clauses, Adjective Clauses because I want to make a good composition 2) grammar, reading, speaking, sometimes comp 3) I liked that I had been learned more ideoms and grammer in this class 4) I really got good reviewed 5) storee everyday Category 5: Topic 1) topics were shod and things I could write about easily 100 Appendix I continued Experimental Class 1 Category 1: Process of Writing 1) I like first of all give us more idea and vocabulary and reading. We didn't more vocabulary when there isn't anything in my mind we can't be good wider 2) I didn't take time before when I writted about details 3) and you can see what kind of mistakes we usually make. After couple corections I never make the same mistake 4) and I had known the mistakes which I had made 5) To check gramar, spelling and puntuation. It helped a lot each one of the students 6) The way how we doing our writing. You can learn a lot 7) The best I liked about .writing in my class was we had enough time to finish the composition 8) This term we learn some metod about writing but I think learn some English before and after that writing 9) The conferencing can learn more English and I can know what I have mistake about English Category 2: General Attitude to Writing 1) The best thing about writing in my class was a short composition 2) journal every week 3) Journal is the best writing I like in my class 4) What I like the best about writing is compositions 5) composition 6) The compositions which we did in our class were useful to improve our writing 7) and now I'm sure, I'm writing better than in September 8) What I like was writing compositions because you comunicate with classmates Category 3: The Teacher's role 1) I like that my teacher helped me correct my writing when I finished my writing 2) I like best about the teacher help me to correct my composition 3) My teacher X is pretty good and she help me a lot Category 4: General Attitude to class 1) Is my first time in the school but I'm feeling confidence with teacher and my classmates Category 5: Topic 1) Composition The place is important to me\"; \"Ben Johnson's Story\" 2) The best one which I liked it was about Ben sprinter 3) Also writing about a real story which happened for students was useful 101 Appendix I continued Experimental Class 2 Category 1: Process of Writing 1) I like to writ composition in my class Because I get more vocabulary and idea 2) I guess everything was good for me because I have more ideas now 3) and (liked) organization of composition 4) (liked) revising 5) Didn't like editing with a group because some student's grammar are weak they just wasted my time 6) (liked) editing 7) I know how would find the mistakes and fixed 8) Before I came to KEC I didn't know how to wte a comp'ri but now I don't have many problem 9) K's very good to write comp because you can know what kinds of mistake you have Category 2: General Attitude to Writing 1) I hate writing 2) I liked to write a lots of meanful sentences which help us in speaking to other body 3) I liked to write composition in my class 4) I liked writing compositions in this class 5) I liked everything about writing 6) I liked writing about composition 7) I like to write some composition 8) I liked the best about writing composition 9) I think writing course is interesting but sometimes heavy 10) I recommend introduce some changes 11)1 liked best about jornal Category 3: The Teacher's role 1) Should the teacher talking about what kind the mistake the student made. I think so they are 2) The teacher is excellent and she helps us a lot 3) She explains the grammar very well 4) I'm grateful with my teacher because she taught me a lot of it (writing) Category 4: General Attitude to class 1) Reading and listening are my favour 2) Also (liked) the gramer work too 3) and (liked) grammar 4) and grammar tests because I can eliminate my mistakes 5) and (liked) grammar Category 5: Topics 1) and don't make the title to bored such as 'a restaurant\" or 'cafeteria' we don't eat that too often 2) I think that some topics are hard to talk about it 3) I liked write about my country and differents peoples Also write about Canada 102 Appendix J: Table of interactions from the background survey ANCOVA Table of interactions between the seven independent variables, two measures and treatment: probability levels T he Informal Test (test 1) The Formal Test [test 2) Variable C&O S&M Total C&O S&M Total 1. First Language .43 .76 .57 .08 .57 .14 2. Years of Education .40 .16 .26 .07 .19 .09 3. Time in Eng. Envirt .16 .007 .05 .25 .19 .22 4. Writing: Informal .95 .55 .79 .88 .61 .80 5. Writing: Formal .94 .87 .93 .57 .60 .55 6. Writing Short Reports .35 .18 .24 .09 .56 .24 7. Writing Long Reports .30 .64 38 .04 .82 .14 103 Appendix K: Population distribution table Population Table: Cell Sizes broken down by Variable, Treatment and Value Labels Variable Treatment Value 1. First Language Asian Spanish Persian Other Exp 15 9 3 7 Con 17 2 6 6 2. Years of Educ'n Low Mid High Exp 3 19 12 Con 3 13 12 3. Time in Canada Low High Exp 18 16 Con 11 16 Self-Report of L1 Writing Ability Low Mid High 4. Informal Exp 0 4 30 Con 0 2 25 5. Formal Exp 1 7 26 Con 6 4 17 6. Short Reports Exp 3 3 28 Con 5 4 18 7. Long Reports Exp 8 5 21 Con 5 10 12 "@en . "Thesis/Dissertation"@en . "10.14288/1.0078322"@en . "eng"@en . "Language and Literacy Education"@en . "Vancouver : University of British Columbia Library"@en . "University of British Columbia"@en . "For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use."@en . "Graduate"@en . "The adult English as a second language writer and the writing workshop approach : performance, biodemographic variables, and attitudes"@en . "Text"@en . "http://hdl.handle.net/2429/31462"@en .