UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The color-coded cloze procedure : a method to assist adult ESL students in searching for clues to fill… Labrum, Howat Alan 1992

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-ubc_1992_fall_labrum_howat_alan.pdf [ 8.34MB ]
JSON: 831-1.0078110.json
JSON-LD: 831-1.0078110-ld.json
RDF/XML (Pretty): 831-1.0078110-rdf.xml
RDF/JSON: 831-1.0078110-rdf.json
Turtle: 831-1.0078110-turtle.txt
N-Triples: 831-1.0078110-rdf-ntriples.txt
Original Record: 831-1.0078110-source.json
Full Text

Full Text

THE COLOR-CODED CLOZE PROCEDURE:A METHOD TO ASSIST ADULT ESL STUDENTSIN SEARCHING FOR CLUES TO FILL INCLOZE BLANKSHOWAT ALAN LABRUMB.A., The University of British Columbia, 1972Dip, Ad. Ed., The University of British Columbia, 1973A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF ARTSinTHE FACULTY OF GRADUATE STUDIES(Department of Language Education)We accept this thesis as conformingto the requistandardTHE UNIVERSITY OF BRITISH COLUMBIAOctober, 19920 Howat Alan Labrum, 1992In presenting this thesis in partial fulfilment of the requirements for an advanceddegree at the University of British Columbia, I agree that the Library shall make itfreely available for reference and study. I further agree that permission for extensivecopying of this thesis for scholarly purposes may be granted by the head of mydepartment or by his or her representatives. It is understood that copying orpublication of this thesis for financial gain shall not be allowed without my writtenpermission.Department of L &U11C. ELI) iCIi r,,vThe University of British ColumbiaVancouver, CanadaDate OCTo6.I. )1) 1992DE-6 (2/88)1:1.ABSTRACTResearch by Shanahan and Kamil (1982, 1983, 1984)suggested that students did not use inter-sententialinformation to improve their doze test scores. Chihara andOiler (1977) had discovered in their research with moreproficient ESL students the contrary. Building on the ideathat the issue of too-local reading depended on both theproficiency and motivation of the students along with theavailability of beyond—sentence clues, the present researchproject was designed using color-coded blanks and parts ofspeech to highlight inter-sentential clues, especially reiterative—word clues.The purpose of the color—coding was to see if, bydifferentiating blanks and words, students could focus moresharply on the necessary information, primarily re—iterativewords that occurred beyond the sentence of each doze blank,and use these words to fill the doze blanks. The studentsin the treatment group were given a key to the relationshipbetween the color and the parts of speech but were notexplicitly told to look for re-iterative-word clues. It wasexpected, however, that each colored blank would stimulatethe students to look through the passage to find a word ofthe same color to fill the said blank. It was also the goalto maximize the number of inter—sentential re—iterativeclues and see how c].oze scores would be affected when color—iiicoding was used as opposed to no color—coding (the majorcontrol).If the inter—sentential clues were present, then color—coding should have resulted in the treatment group havingbetter scores. Randomly—colored words were used as anothercontrol to determine whether any positive effect had comefrom the coding or just from color as a motivational factor.Lastly, the pre- and post—tests were non—colored in order tosee if the color—coding treatment had just a temporary meansof help which led to no lasting gains once removed..A pilot project was done with a class of twenty adultupper—intermediate ESL students using four forms of astandard doze test. Based on the results the colored—blankform was dropped so the three classes in the main projectcould each have one form. To suit the needs of the moreadvanced university students, new stories were chosen andprepared using a rationalized doze to maximize the numberof re-iterative—word clues. In the main research projectmost of the random-color group (the least proficient group)dropped out after the pre—test. The non—colored rationaldoze group received higher scores than the color—codedtreatment group on all of the tests. When the mean scoreswere graphed both these groups made steady progress frompractice test to practice test, the treatment groupappearing to almost catch up. Improvement was made frompre- to post—test by both groups but less by the treatmentivgroup, especially when only inter—sentential blanks werecounted. Generally speaking, t—tests and a very sensitivestatistical program (“One Between and One Repeated MeasuresFactor ANOVA”) confirmed this improvement but showed thatthe color—coded doze treatment group and the non-coloreddoze control group in most cases did not differsignificantly.The graphical analyses of the results were moreoptimistic in favor of the color-coded treatment than thestatistical analyses were but the small sample size (N = 13)made the statistical findings unclear at times. Improvementin some cases may have been because of the decliningreadability levels of subsequent passages.The number of blanks filled, the number filledcorrectly, and the relationship of these two were analyzedto determine the confidence and productive confidence levelsof each group. Results showed the color—coded treatmentgroup were less confident in filling blanks and made limitedgains in productive confidence over the control group.VTABLE OF CONTENTS2.BS’]?R..ACT iiTABLE OF CONTENTS VLIST OF TABLES ixLIST OF FIGURES XiiiACKNOWLEDGEMENTS XVCHAPTER ONE: AN OVERVIEW OF THE RESEARCH PROJECT 1Goal of the Present Research Using the ColorCoded doze Procedure 1Significance of Studying Color-Coded dozeProcedure 6Scope of the Study 8Definitions of Terms 12CHAPTER TWO: A REVIEW OF THE LITERATUREIntroduction to the Review of the Literature 20General Setting 22What is the scope of the research’ 24What is traditional doze procedure’ 26What are some of the controversial areas ofthe doze procedure 27What are some considerations about students’motivation and proficiency’ 32What are the recent education 41How does color—coding as a trend fitinto the present research’ 48Key Area of Investigation Underlying This ResearchPaper 55How do discourse and the nature of dozeoverlap’ 55Is doze procedure essentially global or5 6What is the chronology of the global\discretecontroversy 60Halliday and Hasan (1971) 60Caroll (1972), Chihara and Holler(1977) 60Yamada (1979) 62Thomas (1980) 64Shanahan and Karnil (1982) 68Bridge and Winograd (1982) 69Leys (1983) 72Shanahan and Kamil (1983) 73Shanaharz and Kainil (1984) 75Henk (1985) 77Backman (1985) 79viWhat is the nature of non—native speakersof Eng1ish . . . . . .83What is the possible role of color in theimprovement of non—natives’ doze?reading scores?..... . . . . . . . . . . . . . . .85What are the contributions of the key areasof investigations to the presentresearch?..... . . . . . . . . . . . .88Results of the Review of the Literature 89Introduction . . . . . . . . . . . . 89Overview of the method proposed in relationto the 1 iterature . . . . . . . 8 9goals of the present dozeresearch . . . . . . . . . . . . . . . . . . . . . . 89useofpreviousresearch ..........09Implications of review of literaturefor the present research paper. .. . . .. .. . . . .92research questions. 95hypotheses . . . . . . . . . . . . . . . . . 95CHAPTER THREE: RESEARCH DESIGN AND METHODOLOGY. . .. 99Introduction...... •......... . . . . . . . . . . . . . . . . . . . . . . . . .99The Subjects.... •....... . . . . . . . . . . . . . . . . . . . . .102The Treatment and Control Groups......... 103The Treatment and Control Passage Forms.............106Nature of the forms. . . . . . . . . . . . . . 106Method of coloring the passage...... . . . ...... . .108The Stories Used for the doze Passages..... ..108Introduction to the selectionofthepilotpassages.... 108Rationale for the type of textanalysis (pilot project)........ 110Presentation of the text analysis form(pilot project) . . . . . . . . . . . . . . . . . . . . . . . . . . .111Rationale for the type of textanalysis (main project).............. 112Presentation of the text analysis(main project) . . . . . . . . . . . . . . . . . . . . . . 113The Administration of the doze Passages. 115Overview of the distribution of thetext and practice doze passages(pilot project) . . . . . . . .115Length of time required (pilot project).. .116Overview of the distribution of thetext and practice doze passages(main proj ect) . . . . . . . . . . . . . . 116Length of time required (main project) 117Roles of the researcher, teachers,and the students. . . . . . . . . . . . . 118The Scoring of the doze Tests andData Analysis..... .. . . . . . .. . . .. . . . 121viiCHAPTERFOUR: ANALYSISANDDISCUSSION........ ...130Introduction. . . . . . . . . . . . . . . . . . . . . . . 130Changes Made to the doze Procedure after thePilot Project...... . . •......... . . . . . . . . . . . . . . . .131Problems and Solutions in Pilot and MainProjects . . . .133The Research Data . . . . . .135The Pilot Project . . . . . . . . . . . . 135The Main Project . . . . . . . . . . 139An overview’ 139Asumxnaryofthedata 140Anexplanationofthedata 175A closer look . . . . . . . . . 175A statistical look at the data 180Thehypothesesandresults 198The problems with the present researchdesign...... . . . . . . . . . . . . . . . .200Hope given by the questionnaire 201The conclusions drawn from theanalysis of the productiveconfidence scores........ . .. . . .. . . . . .204CHAPTER FIVE: IMPLICATIONS OF THE RESEARCH.. 208Introduction. . . . . . 208Future Research Designs 210BIBLIOGPHY. . . . . . . . . . . . . . . . . . . . . •....... 220APPENDIX A: PILOT PASSAGES . .. . .. . . . .230The Original Texts.... . . . . . . . . . . . . . . . . .230The Color—coded Fornis . . . . . . . . . . . . . . . . . . 243A Sample Answer Key...... . . . . . . 254The Distribution of Deleted Words . .. . . .. . .261An Analysis of Re—iterative Clues...... . 251APPENDIX B: PRODUCTION OF PILOT CLOZE PASSAGE FORMS.. . . .263APPENDIX C: DISTRIBUTION OF RESEARCH TASKS . .... . .269Role of the Researcher. . . . . . .269Instructions for Teachers . 272Instructions for Students. . . . 277APPENDIX D: SPREADSHEET CHART...... .. ... . .281For Collection of Answers andDetermination of Scores...... . . . .. . . . 281viiiAPPENDICES FOR MAIN PROJECTAPPENDIXE: MAINPROJECTPASSAGES..... 288The Original Texts. . . . . . . • . . . . .288The Color—coded Forms . . . . . . . . . . . . . 299A Sample Answer Key . . 309Sample of distribution of re-iterativewords . . . . . . . . . . . . . . . . . . . . 311APPENDIX F: PRODUCTION OF THE CLOZE PASSAGES .... . . 313APPENDIX G: COMPARISON OF EXACT- AND ACCEPTABLE-WORD I4EA}I SCORES . . . . . . . . . . . . . . . . . . . . . . . . . . 316APPENDIX H: QUESTIONNAIRES........ ... .. . ... . . . .. . . .320FormsforGroupsl,2,and3...... .. 320ixLIST OF TABLESTABLE 1 Overall Reading Comprehension Strategic 34TABLE 2 Quotes about Recent Education TrendsTABLE3 VariousKindsofCohesion.. .98TABLE 4 Proficiency Classification of Students 103TABLE 5 Configurations of the Standard dozeProcedure and Variations..... . .. 107TABLE 6 Number of Blanks/50 Deletions whose Fillersdo not Appear at Least Once Somewhere in thePassage; and the Number of Words in thePassage . ... . .... . .... .111TABLE 7 An Analysis and Ordering of the doze PassagesBased on a) the Number of Non—re—iterative Cluesfor the Blank Fillers and b) ReadabilityFou1as. . . . . . . . . . . . . . . . . 112TABLE 8 Number of Blanks/50 Deletions whose Fillers donot Appear at Least Once Somewhere in thePassage; and the Number of Words in thePassage. .. ... . .. ... .. .. 113TABLE 9 An Analysis and Ordering of the ClozePassages Based on a) the Number ofNon-re-iterative Clues for the Blank Fillersand b) Readability Formulas....................114TABLE 10 Number of Re—iterative Blanks..................1l4TABLE 11 Distribution of the Treatment, Variationsand Control Cloze Tests, with VariationsApplied to Standard (every nth word) ClozeProcedure Format . . . . . . . . 115TABLE 12 Distribution of the Treatment, Variationsand Control Cloze Tests, with VariationsApplied to Rational Cloze Procedure Formatin which Deletions were Chosen to MaximizeNumber of Intra- and Inter—sentential Clues....117TABLE 13 Answer Collection and Blank ScoreDetermination Chart. . . . . . 124TABLE 14 Criteria for Development of Weighted Scores....127TABLE 15 Calculation of Cloze Passage Scores............127xTABLE 16 Answer Collection and Blank ScoreDetermination Chart . . . . . . . . . 129TABLE 17 changes to Cloze Procedure after PilotProject. . . . . . . . . . . . . . . . . . . 131TABLE 18 Problems and Solutions in Pilot andMain Projects. . . . . . .133TABLE19 TestData:Exact—WordScores 143TABLE 20 Mean Scores for All 50 Cloze Blanks............147TABLE 2]. Productive Confidence Scores for All 50 dozeBlanks . . . . . . . . . . . . . . . . . .149TABLE 22 Mean Scores for Intra- and Inter—sententialBlanks ... .. . * . . .. . .154TABLE 23 Productive Confidence Scores for Intra- andInter—sentential Blanks . ... . . . .. . ..l56TABLE 24 Mean Scores for Intra-sentential Blanks........161TABLE 25 Productive Confidence Scores ForIntra—Sentential Blanks... .... . ...l63TABLE 26 Mean Scores for Inter—sentential Blanks ...168TABLE 27 Productive Confidence Scores forInter—sentential Blanks...... . . .. . .... . 170TABLE 28 T—tests for the Main Project.............. . . .183TABLE 29 Significant Differences...... ... . . . . .. .. .. . . . .186TABLE 30 Statistical Program: Trend Analysis ..J.87TABLE 31 Data Collection Table for ANOVA ...188TABLE 32 Summary Table: ANOVA.(All 50 Blanks) .189TABLE 33 Summary Table: ANOVA (Intra- and Intrasentential blanks) . . . . . . . . . . . . . 189TABLE 34 Summary Table: ANOVA (Inter-sententialblanks)..... •1•11••••••• .. . ... . • 190TABLE 35A Student-Newman-Keuls Test for VarianceScore (A) (All5oBlanks) ......19lxiTABLE 35B Student-Newman-Keuls Test for VarianceScore (B) (All5OBlanks) 191TABLE 36A Student-Newman-Keuls Test for VarianceScore (A) (Intra- and Inter-sententialBlanks) . . . . . . . . . . . . . . . . . . . . 192TABLE 36B Student-Newman-Keuls Test for VarianceScore (B) (Intra- and Inter-sententialBlanks) . . . . . . . . . . . . . . . . . . . . . . . . . 192TABLE 37A Student-Newman-Keuls Test for VarianceScore (A) (Inter—sentential Blanks) . . .. . .193TABLE 37B Student-Newman-Keuls Test for VarianceScore (B) (Inter—sentential Blanks).. 193TABLE 38 General Linear Models Procedure (All 50Blanks) . . . . . . . . . . . . . . . . . . . . . . . . . . . 194TABLE 39 General Linear Models Procedure (Intra— andInter—sentential Blanks)...... .... . . .. .. . ...... 194TABLE 40 General Linear Models Procedure (Inter-sentential Blanks) . . . . . . . . . . 194TABLE 41 Post—session Questionnaire......... . . .... .. . . . .202TABLE 42 Treatment Group’s Confidence Level. . . .. .. .. . . . .206TABLE 43 Suggestions for Possible Future ResearchDesigns. . . . . . . . . . . . . . . . . . . . 213APPENDIXTABLE A An Analysis of Re-iterative-Word Clues.........261TABLE B Method of Color-Coding and Printing Passages...263TABLE B.l Meanings of the Symbols Used for ColorCoding . . . . . . . . . . . . . . . 265TABLEC RoleofResearcher.............. ..........269TABLED InstructionsforTeachers......................272TABLEE InstructionsforStudents ..............277TABLE F Special Instructions for Students in theTreatment Group . . . . . . . . . . . . . . . . . . . . . . . . . . 2 8 0xiiTABLE G Answer Collection and Blank Score-Determination Chart (Pilot Project) . . .. . 281TABLEH SampleofPassageAnalyses.....................3llTABLE I Answer Collection and Blank Score-Determination Chart (Main Project) .. .. . . . . . . .. .315TABLE J Exact- and Acceptable-word Scores (AllBlanks . . . . . . . . . . . . . . . . . 316TABLE K Exact- and Acceptable-word Scores (Intra- andInter—sentential Blanks . . 317TABLE L Exact- and Acceptable-word Scores (Intrasentential Blanks . . . . . . . . . . . . . . . . . . . . . . . . . 318TABLE M Exact- and Acceptable-word Scores (Intersentential Blanks. . . . . . . . . . . . . . . . . .319xiiiLIST OF FIGURESPilot ProjectGRAPH 1 All 50 Blanks: Four Forms, EXACT-word scores(in %) . .137GRAPH 2 All 50 Blanks: Standard vs CCC, EXACT-wordscores (in %) . . . . . . .138Main ProjectGRAPH 3 All 50 Blanks: EXACT—word Scores (in %).. 148GRAPH 4 All 50 Blanks: Total no. of Blanks Filled......150GRAPH 5 All 50 Blanks: Percent of Blanks Filledto Possible . . . . . . . . . . . 151GRAPH 6 All 50 Blanks: Total No. of Blanks FilledCorrectly . . . . . . . . . . . . . . 152GRAPH 7 All 50 Blanks: Correct Blanks/No. of Filled(in %) . . . . . ..... 153GRAPH 8 Intra- and Inter-sentential Blanks: EXACT-word Scores (in %) . . . . . . .155GRAPH 9 Intra- and Inter-sentential Blanks:Totalno • of Blanks Filled . . . . . . . . . . . • . . . . • • . . . . • . . . . 157GRAPH 10 Intra- and Inter-sentential Blanks: Percentof Blanks Filled to Possible...................158GRAPH 11 Intra- and Inter-sentential Blanks: TotalNo. of Blanks Filled Correctly....... . . .... . . . .159GRAPH 12 Intra- and Inter-sentential Blanks: CorrectBlanks/No.ofFilled(in%)GRAPH 13 Intra-sentential Blanks: EXACT-word Scores(in %) . . . . . . . . . . .162GRAPH 14 Intra-sentential Blanks: Total no. of BlanksFilled...... . . . . . . . . . . • .164GRAPH 15 Intra-sentential Blanks: Percent of BlanksFii led to Pcs S ib1 e . . . . . . . . . . . 165xivGRAPH 16 Intra-sentential Blanks: Total No. of BlanksFilled Correctly...... . . . . . . .166GRAPH 17 Intra-sentential Blanks: Correct Blanks/No.of Filled (in %) . 167GRAPH 18 Inter-sentential Blanks: EXACT-word Scores(in %) . .. . . •....... ..... ... ... ..... . . . . .169GRAPH 19 Inter-sentential Blanks: Total no. of BlanksFilled.... . . .... . . . ... . .. .... ... .. •.. .171GRAPH 20 Inter-sentential Blanks: Percent of BlanksFilled to Possible . . . . . . . . . . . . . . . . . . . . . 17 2GRAPH 21 Inter-sentential Blanks: Total No. of BlanksFilled Correctly. . . . . . . . . . . . . . . . . . . . . . . . 173GRAPH 22 Inter-sentential Blanks: Correct Blanks/No.of Filled (in %) . . . . . . . . . . . . ........ . . . . . . . .174xvACKNOWLEDGEMENTSI would like to thank my advisor, Dr. Kenneth Slade,for his kind and patient guidance in my efforts to completethis thesis. Also, I wish to thank Dr. Walter Boldt, mystatistics advisor, for sharing his special knowledge in avery warm and enjoyable way. To Dr. Lee Gunderson, a thankyou goes for all his work on and insights into the dozeprocedure with ESL students. The teachers and students whoparticipated in my color—coded doze research deserve amedal for their time and effort.As is traditional in writing thesis acknowledgements,and now more deeply felt and appreciated, thanks go to myparents, wife, and children for supporting my efforts in somany ways. To my family and my friend, Omar, in SaudiaArabia, I owe a debt a gratitude for making me keenly awareof how important it is to persist in the opportunities oneis given.1CHAPTER ONEAN OVERVIEW OF THE RESEARCH PROJECTTHE GOAL OF THE PRESENT RESEARCH USING THE COLOR-CODED CLOZEPROCEDUREOne goal of teaching reading comprehension to adultlearners of English as a Second Language (ESL) is to helpthem process a passage in as large chunks as possible.Richards maintained this view (1980) and pointed out that“learners need to develop strategies for interacting with anative speaker so that they are able to extract totalmeaning from discourse rather than being limited to thepropositional content of single sentences” (p. 64).Halliday and Hasan (1976), Long (1981), and. Mohan (1986),supported the same position that coomunication goes beyondthe sentence level and involves interaction with a cohesivetext.To students the proposed advantage of discourseprocessing (as opposed to processing each sentence inisolation) in understanding of a passage is two-fold.First, having a broad purview could possibly allow them totake advantage of the built-in redundancy in the text tofill in gaps in information caused by unknown words,especially when a dictionary is not at hand, or if handy,would take the student away from the text for a long periodof time. Ashby-Davis (1984) pointed out that the normalreader:2must check overall understanding of atext or at least inter- and intrasentence meanings to make educatedguesses concerning the meanings ofunknown words. (p.87)Second, a wide purview could lead adult students to aclearer understanding of the needed words. Thus aproficient adult ESL reader can be defined as one who isable to find the most suitable words to fit the meaning ofthe unknown words in a written passage.It is the goal of this research paper to suggest andtest a method of helping adult ESL learners to become moreproficient readers. The method, to be explained further inthe review of the literature in Chapter Two and in theresearch design in Chapter Three, is a modified version ofthe doze procedure. Henceforward it is referred to as the“color—coded doze” (CCC). Also to be explained is arationalization of the CCC in which as many deletions aspossible are made to words which appear more than once inthe text.The CCC follows the standard doze procedure in that:a) every “nth” word, commonly the fifth, is deleted ina passage and replaced with a standard length blank;b) the first and last sentence of a selection are leftintact to give the students an idea as to thegeneral direction of the meaning of the text; andC) the passage is about 300 words in length allowingfor 50 blanks.3The CCC is different from the standard doze procedurein that every word and blank in the CCC passage was given aparticular color to designate its part of speech. Also theCCC procedure in this research counted proper nouns in thedeletion process. This is because some of the proper nouns,such as people’s names, were used more than once in thepassage and thus were open to the possibility of beingfound, especially when color—coding was applied to thepassage.The purpose of the color—coding was to draw thestudents’ attention beyond the sentence of the doze blankto look for clues which would help the students to determinethe suitable word to fill in the doze blank. This purposewas in response to researchers such as Cohen (1980), whopointed out that, indeed, there is a tendency for non—nativespeakers to look locally (within the sentence of the blank)for such clues. The purpose of usina tarts of speech to becolor—coded was to take advantage of the informationinherent in those parts of speech. including the rammatica1re1ationshis. Cohen stated that “doze can also be used tocheck for awareness of grammatical relationships” (p. 96).He illustrated grammatical relationships as “grammaticalagreement with the elements in the passage (tense, gender,number, person, or whatever)” (p. 96) as shown byinflections on the parts of speech, resulting in cohesion inthe discourse. In making the above statement and4clarification, Cohen gave his support to the importance ofgrammatical information for doing the doze tests.Notwithstanding the advantages of the parts of speechand their grammatical relationships for doing doze tests,it was not the purpose of this research project to teachgrammar as an end in itself. Nor was it the purpose of thecolor-coded doze to make the students dependent on thecolor—coding. Instead, it was the purpose to remove thecolor at some future point and to let the students completethe standard doze exercises, while still making use oftheir awareness of the parts of speech clues beyond thesentence to help them fill in the blanks.Given a) examples in the research literature on thedoze procedure of ESL students looking inter-sententiallyfor clues to fill doze blanks and b) observations of howbusiness people and students use colors to highlight keyideas, it was thought that the CCC method of teachingstudents to look for clues beyond the local (intrasentential) level to find information at the intersentential level seemed promising. However, the CCC methodneeded to be tested. As will be seen in Chapter Two, therewere a number of hypotheses that should be tested, but atthis point the prime null hypothesis to be tested was statedas follows:5Students trained with the color-coded doze will showno sianificant difference in scores on a standarddoze yost—test after treatment when comvared tostudents trained on standard doze vrocedures alone.In Chapter Four if higher mean scores on the post—test arereported for the treatment group the inference will be madethat CCC students were able to make more use of grammatical(syntactic) and lexical (semantic) clues than the studentswho received lower mean scores. On the other hand, it canbe argued that the clues might not have come from beyond thesentence but from within the sentence of the blank, albeitwith the help of the color-coding. Though this problem andthe resulting hypotheses are better left to Chapter Twoafter a review of the literature on doze procedure has beendone.Before proceeding, however, it should be noted herethat, because of the necessary limited scope of thisresearch project, there is an explicit focus on thefollowing question. Having trained using the CCC, dostudents become more aware of inter—sentential same—word reiterative word fillers that match the deleted words, thanstudents who have trained with non—color—coded parts ofspeech, color—coded blanks only, or randomly—colored blanksand words? Clearly, re-iterative words of the kind ‘ustmentioned above are the easiest type of cohesive words toquantify for purposes of identification in preparation of6the doze passages. This and other types of cohesive wordsare given and illustrated in Table 3 at the end of ChapterTwo. Their interaction in and effect on the CCC researchproject can be analyzed and possibly determined by ananalysis of the answers given by students on the pre—tests,practice tests, and post—tests.SIGNIFICANCE OF STUDYING COLOR-CODED CLOZE PROCEDUREAs will become evident in the review of a portion ofthe abundant literature about the doze procedure, the dozeprocedure has been seen to have a place in the ESLcurriculum, as well as in other language curricula. Forexample, Schoenfeld (1980) and Valmont (1983) suggesteddoze procedure could make use of students’ knowledge ofsyntax, phonics, and word meanings to improve the students’prediction and confirmation strategies.Other uses of doze procedure include:a) teaching (Soudek & Soudek, 1983 — 1984),b) testing ( Foley, 1983),c) readability (Rye, 1982),...but the doze procedure has deficiencies, attemptedsolutions to which have generated many of the studies toreplace it with modified versions. Shanahan and Kamilthrough a series of experiments (to be outlined in Chapter7Two) have even tried to disparage the doze procedure as ateaching device. This present study intends to build on thestrengths of the doze procedure by trying to overcome oneof its possible weaknesses — its questioned ability to helpstudents look beyond the sentence of the blank to find cluesto a filler for the blank (Shanahan & Kamil, 1982, 1983, and1984; Leys, 1983).The use of color—coding as a so.ution to this supposedinability of students to look beyond the sentence needed tobe tested, not dust as an intellectual exercise, but todetermine whether there was sufficient benefit to justifythe extra time and expense it takes to prepare color-codedpassages. Certainly it can be said that color in oursociety is playing a greater and greater role both inattracting and categorizing, and the means to producingcolor—coded products are becoming more efficient and cheaperall the time. Thus as the benefit-to—cost ratio continuesto increase, the use of color—coded c].oze procedure becomesmore feasible. This is all the more reason why the color—coded doze can and should be tested in a controlled way.Because color—coding has become so prevalent in oureveryday lives, even in a limited way in education, it seemsever more and more plausible to use color—coded parts ofspeech in the teaching of ESL. It should be noted thatGattegno was using color—coding to teach phonics and to alesser degree, grammar in 1957, and in lesser ways languagetextbook writers predate even this work (Dale, 1926, Hay &8Wingo, 1954). The problem with color-coding has been andmay continue to be with the necessary standardization of thecolor-code, which in itself is difficult to attain, and theconstraints this would put on teachers and textbooks if theydid not want to confuse the students.Scope of the StudyThe color—coded doze experiment described in thispaper was divided into two sections, each carried out in adifferent location in Greater Vancouver, a large Canadianurban setting (population 1.5 million). The first was apilot Drolect in which 18 young adult ESL students, largelyfrom the Pacific Rim and studying at a major private collegein Burnaby, B.C., took part. They were defined according totheir college’s categorization system as at the upper-intermediate level. The college students were in a programin which they studied five hours a day, five days a week in14—week terms. The second was the main prolect in which 60nineteen— to twenty—two—year old university level ESLlearners from Japan, studying in three classes in theRitsumeikan English program at the University of BritishColumbia in Vancouver, participated. The sixty students(chosen from a total of one hundred students) wereanthropology and political science majors mostly in theirsecond year (some in their third or fourth year) atRitsumeikan University in Japan, visiting Canada for a nine-9month English course. In Japan they had been studyingSocial Science and International Relations, Science andEngineering, Law, Literature, Economics and Businessadministration, so the ESL course content revolved aroundsuch topics. According to their TOEFL scores they were notquite up to the required standard for university entrance.More specifics on their English proficiency levels will begiven in Table 4 in Chapter Three.The language skill area being studied in the presentresearch was reading and learning to use context clues,particularly parts of speech with a special emphasis on reiterative words to help find clues that would enable thestudents to restore deleted words from passages of betweenapproximately 295 and 330 words each. These passages weretaken from two books. The book, CANread, about Canadianpersonalities, written for ESL students and edited byPatricia Raymond (1987), was used for the college class.Canadian Society, A Macro Analysis, a college text by HarryH. Hifler (1991) was used for the Ritsuiueikan students.Seven passages were selected (one each from separatestories) for the college students. Six passages were chosenfor the Ritsumeikan students.With regards to the reading level of the pilot projectstories, Little (1988) in his review of Raymond’s CANreadstories and two ESL schools known to this researcher allassessed her stories at the intermediate level for ESLstudents. According to a Plesch readability (1948) rating10of the parts of the stories used in the pilot study, most ofthe stories were mostly in the grade seven to eight range,although one was at the sixth grade level. A Fryreadability (1977) rating gave somewhat the same results.(For details please refer to Table 7 in Chapter Three onpage 102.) Ad hoc trials of two of the stories with ESLstudents had suggested that the doze forms of the CANReadstories would have to be given to upper-intermediatestudents or even higher if strategically possible.The passages for the Ritsumeikan students ranged from areadability level 12 to 15, with most being at level 15, themost difficult. The assessment was done using a readabilitysub—program in a shareware computer program called WordCount vl.2 by Chris B. Sakkas. An analysis of the passageswill be given in Table 4 in Chapter Three.The CCC study revolved around the questions of how toget students to look beyond the immediate sentence of theblank to find the suitable (preferably the deleted) word andto show whether or not the students were, in fact, lookingbeyond the sentence. The methodology was to divide thestudents into groups and give each group the same standarddoze passage with the same deletions, but the treatmentgroups were given passages which had various amounts orkinds of color clues as is explained forthwith. Thepassages of the main treatment group had all the words andblanks color—coded according to parts of speech. Thepassages of the secondary control group had dust the blanks11color—coded. (In order to allow for more students to beinvolved in the more important treatment cells, this blank-only colored treatment was not used in the main project.)The passages of the tertiary control group had all the wordsand blanks colored randomly. The passages of the primary(most important) control group had no color at all.In Chapter Two, a selective review of the literaturewill focus primarily on research which has a bearing onfinding an answer to the questions of how to help studentslook for inter—sentential clues and to show whether or notthe students are doing so. The review will also show thebasis of the research design. Chapter Three will presentthe resulting design of this two month study. (The studywas carried out over two periods of about a month each, oneperiod for the pilot and one for the main research project.)The results will be reported and commented upon in ChapterFour. In Chapter Five conclusions will be made about theeffectiveness of the doze procedure and what influence, ifany, color—coded parts of speech had on it, particularlywith adult ESL students. In addition, recommendations willbe given to improve both the color—coded doze procedure andthe method of analyzing the results.12DEFINITION OF TERMSdoze (color—coded) — the same as the standard doze withthe exception that in the color—coded doze all the parts ofspeech are color—coded. Also, for the purposes of thepresent research paper, proper nouns are counted in thedeletion process.doze (fixed-ratio) — a passage in which words are deletedin a fixed manner, e.g. every 5th word is deleted. It isalso called the “random doze”.doze (modified) — a doze passage which varies in one ormore aspects from the traditional doze passage.doze (random) — a passage in which words are deletedaccording to a set pattern, e.g. every fifth word. It israndom insofar as the deletor has no control over thespecific words which are deleted in a fixed manner.(Another name for this is the “fixed—ratio doze”.doze (rational) — a passage in which words are selectivelydeleted, e • g. all nouns and verbs only.; words that arefound elsewhere in the text at least once (theseiterative words were targeted for deletion in the mainproject of the present research)13doze (standard) — a passage in which every nth word isdeleted, with the exceptions that proper nouns are skippedover (not deleted or counted) and all words in the first andlast sentences are left intact. (see traditional doze)doze (traditional) — the same as the standard dozeclozentropy — the process of collecting native speakers’responses to doze blanks in order to determine possibleacceptable responses for scoring non-native speakers’answers.cohesive — the state of words having a relationship withother words.color—coded parts of speech — see Parts of Speech: (colorcoded)confidence — the amount of faith the students have inthemselves as manifested by the number of doze blanksfilled.confidence (productive) — the number of correct answers as aproportion of the number of doze blanks filled.14conjunctive- the state of having a relationship in whichwords are joined with conjunctions such as “and”, “but”,etc.constraints (discourse)— the limitations put on the meaningof the passage by each of the words within it. For example,if the passage is describing “dogs”, the pronoun referencecannot be “it”. (see textual constraints)constraints (textual)— the same as discourse constraintscontext — the sum of the inter-relationships involved in areading passage.cues (grapho—phonemic) - clues from the written form of thesound of word.cues (re—iterative)— clues that are the same word as orwhich contain the root of the deleted word.cues (semantic)— clues giving lexical meaningcues (syntactic)— clues giving syntactical (functional)meaningdeletion (fixed-ratio)- a pattern of deletion in the dozeprocedure where every nth word is removed.15deletion (rational) — a selective deletion of words based ona particular teaching need, rather than a random (every nthword) deletion.discourse— the chunk of passage larger than a sentence,i.e. the whole passage and all the inter-relationships.Erra — an abbreviation for a combination of intra- andinter-sentential blanks, the blanks left when re-iterativewords are deleted. (see Intra— and Inter—)filler - in a doze passage a word which is required to fillin a blank which has been left when a word has been deleted;ideally a filler would be re-iterative, i.e. the same wordas has been deleted; the filler may or may not be foundsomewhere in the text.gap filling exercise— any exercise which requires missinginformation to be found and inserted.guessing strategy — a systematic way - to determine thenecessary solution to a problem, such as filling in dozeblanks by carefully and logically considering theinformation available to the reader and deciding upon themost suitable word to use as a filler.16Inter—— a short form for inter-sentential blanks, the spaceleft when words are deleted from a doze passage. (see Erraand Intra-)Intra—— a short form for intra—sentential blanks, the spaceleft when words are deleted from a doze passage (see Erraand Inter).intermediate level - the level of ESL proficiency in whiöhthe students are familiar with all the English tenses in theactive voice, and are learning the passive voice,conditionals, modal auxiliaries and adjective clauses.inter—sentential clues: words that are located beyond thesentence of the doze blank.intra-sentential clues: words that are located within thesentence of the doze blankparts of speech (color-coded)— the parts of speech arecondensed to five categories, each designated with its owncolor a) blue: nouns and (subject and object) pronouns b)red: verbs C) green: adlectives and articles d) purple:adverbs e) black: remaining aroups including the followingconjunctions, prepositions, interj ections, relativepronouns, “not”17parts of speech (traditional)— a) nouns: people, places,ideas, things, e.g. “student”, “Vancouver”, “philosophy”.test b) pronouns: representatives of nouns, e.g. “I”, “me”;relative pronouns—-those pronouns which ‘oin ideas, e.g.”who”,” which” C) verbs: actions, e.g. “write”, “think d)adjectives: descriptors of nouns, e.g. “famous” e) adverbs:descriptors of verbs, e.g. “quickly”, “often” f)conjunctions: joining words, e.g. “and”, “thus” g)prepositions: words which relate location and time, e.g.“in”, “on” h) interiections: words that express feelings,e.g. “oh” i) articles: e.g. “a”, “the”referential - the state of a word having a relationship withanother.re—iterative- the state of being repeated; re—iterativewords are words that are found more than once in a passage.scoring method (acceptable)— a method of scoring dozepassages using any word which fits the sense of the passageregardless of whether it is a synonym or even grammaticallycorrect.scoring method (sensible)— a method of scoring dozepassages using synonyms as alternatives of the exact—word.18semantic— the state of expressing a meaning rather than afunction.syntactic- the state of the interaction of the parts ofspeech.syntax — the interaction of the parts of speech.test (discrete)— a test which measures the students’understanding at the sentence level, rather than relying oncontinuous textual understanding.test (global)— a test which measures the students’understanding of the whole passage. (see integrative test)test (integrative) - the same as the global test.vocabulary— all the words in a passage, be they havinglexical or functional meaning.words (function)— the words which have no specific meaningin themselves and serve to enhance lexical words by showingrelationships, e.g. “and”, “but”, “because”, “if”, “in”,“the”, etc.19words (lexical)— the words which carry meaning inthemselves and give understanding when they stand alone,e.g. “house”, “man”, “speak”, “big”, “quickly”20CHAPTER TWOREVIEW OF THE LITERATUREINTRODUCTION TO THE REVIEW OF THE LITERATUREThe review of the literature in this chapter will bepresented with the current research design in mind. Thedesign evolved from having read literature on and fromhaving had practical experience in a) the doze procedureand b) the use of color in education. Those points relevantto the design will be discussed in terms of what previousresearchers have said about them from their experiments.The purpose of this discussion is to show the genesis of andneed for the present research question and to show thereasoning behind and the organization of the research designin this paper. Thus it can be said that each topic examinedin this chapter has its source in the literature and is anintegral part in the development of the research design. Itshould be noted that certain topics are either part of thepilot project, the main project, future research based onthe present research, or a combination of these.In the next part of this chapter, to give the generalsetting for the research, the scope of the experiment willbe given and then the standard (traditional) doze procedurewill be described as it has been used by severalresearchers. The various aspects of this topic includedeletions, length of blanks and passages, and parts of a21passage left intact. Then, areas of investigation will belooked at, including deletion ratios and patterns, scoring,ways of improving students’ scores and the education trendstoward the use of discourse, context, and information gaps.In the subsequent part of this chapter the key areas ofinvestigation in this research paper will be presented.They are:a) the importance of discourse to the teaching ofEnglish as a second language,b) the definition of the terms “discrete” and “global”and opposing arguments as to which of them reflectsthe nature of the doze procedure,c) the linguistic nature of non—native speakers ofEnglish, andd) the implications of these key areas of research.In the last part of this chapter a bridge will be madefrom the review of the literature to the research design inwhich the goals and use of the previous research will bediscussed. It should be noted that an effort to link therationale of the design to the key points in the review willalso be made throughout the second chapter. This has beendone to firmly establish the bonds between the past research22and the present development of a new and somewhat complexresearch design. The original design used in the pilotproject and the more conservative revised design used in themain project were developed to discover if students could betaught with the color-coded parts of speech to look beyondsentence boundaries for clues to help them achieve higherscores on doze passages.THE GENERAL SETTINGThis section of the research paper will endeavor toexplore a number of questions about the doze procedure,recent trends in education, and the linguistic attitudes ofnon-native speakers of English. The purpose is to show howeach is related to the use of color—coding in the presentESL experiment with the doze procedure in order to justifythe effort of carrying out the research. The questions areas follows:a) What is the scope of the research?b) What is traditional doze procedure?C) What are some of the controversial areas of thedoze procedure as they relate to this researchpaper?23d) Can color—coding be a way of improving students’motivation for and their proficiency in doing thedoze procedure?e) What are recent education trends that have lead tothe use of color—coded doze procedure?f) How does color-coding as a trend fit into thepresent research?g) How do discourse and the nature of the dozeprocedure overlap?h) What is the history of the arguments in theglobal/discrete controversy as it relates to thedoze procedure?i) How do non-native speakers of English feel aboutdoing the doze procedure?j) What are the contributions of the key areas ofinvestigation to the present research?24What is the scope of the research?It should be noted here that the purpose of thefollowing literature review on the doze procedure andcolor—coding is to set the stage for the research model ofthis paper. Although there is a vast array of researchwhich has been undertaken since 1953 when Wilson Taylorfirst described the doze approach, only those areasrelevant to the present project have been selected forreview. Likewise, of the limited amount of work that hasbeen done on color—coding in education, only those relevantareas have been included in this paper.A selection of the areas or subtopics of doze researchwell summarized in Rankin’s overview (1974) of 1958 to 1974will be considered in this paper. Of Ran]cin’s subtopicswhich include “readability, reading comprehension, learning,information, redundancy, thinking, aptitude, readiness,listening, flexibility, and context clues” (1974, p. 2) thispaper will be looking at reading comprehension, learning,redundancy. thinking, and context clues in the followingway. This research deals with adult non—native speakers ofEnglish. These college and university ESL studentsparticipated in a study using the standard doze procedureand color-coded parts of speech. The urose of the study25was to determine if color-coded Darts of speech in dozepassages would be able to encourage and assist students inlookina beyond the immediate sentence of a doze blank tofind clues as to the filler that matched the word originallydeleted from the passage.The amount of research already done and which continuesto be done on the doze procedure is evidence of the greatcontribution Taylor has made to education with his easy tomake gap-filling exercise. At the same time the interest inso much research by so many researchers is indicative of theongoing controversies which surround the doze procedure.This paper is a reaction to the debate which seemed to be ofprimary importance in the late 1970’s and early 1980’sbetween the 011cr group and the Shanahan group. Thecontroversy was whether or not the doze procedure was“global” or “discrete” in nature. To put it another way,Chihara, 011cr et al (1977) appeared to show in theirexperiment that students did use inter-sentential cues todecide on what fillers to put into the doze blanks.Shanahan et al (1982, 1983, and 1984) claimed to demonstratesuch was not the case. Their reasons and arguments will begiven later.Before proceeding to this key controversy, however, itshould be noted that the doze procedure has been shown tobe effective for teaching reading to the type of targetgroup in this present research, i.e. ESL students. (Craker,1971; Oiler and Conrad, 1971; Oiler, 1972a; Anderson, 1973;26Hinofotis, 1977; Streiff, 1978; and Soudek and Soudek, 1983)and with adults (Peterson, Paradis, and Peters, 1973; andRankin, 1974). Also before proceeding, a look at thedefinition of the traditional doze procedure is in order.What is traditional doze procedure?As to the appearance of a typical traditional dozetest Ashby-Davis (1984) summed it up as being a) a passageof about 300 words with b) the first sentence left intact togive students a sense of the content and c) 50 deletionswith d) every fifth or seventh word deleted to ensurerandomness in the deletion of syntactical and lexical items.She said that answers that are e) either originally deletedwords or acceptable alternatives are given credit. Petersonet al (1973) and Valmont (1983) noted that the traditionaldoze procedure uses a standard blank length of 15 spaces.Giving credence to standardized blank length, Valmont wenton to elaborate on the use of standard versus non—standardlength doze blanks. He concluded that a standardized dozeblank is useful for teaching because it did not allow thelength of the blank to give a clue that would detract fromthe students’ need to make an effort to use the availablesyntactic and semantic clues.It was this traditional doze which was the startingpoint of the present research, i.e. as the basic format tobe used with the control and treatment groups. In the pilotproject for the treatment groups the traditional doze was27modified only by color-coding the words and/or blanksaccording to the appropriate part of speech represented. Inthe major project the color—coding was used in the same way,except the traditional doze was modified. The followinghistory of the doze procedure gives among other things thereasons for that modification.What are some of the controversial areas of the dozeprocedure as they relate to this research paper?Researchers have investigated various methods ofdeleting words and of scoring the students’ replacements.It was found by Foley (1983) using random patterns withdeletion rates of every fifth to twelfth word that there wasno difference in difficulty of replacement. However,several researchers (MacGinitie, 1961; Ramanauskis, 1972;O’Reilly and Streeter, 1977) preferred the deletion of everyfifth word because it sampled meaning and grammar morethoroughly and oblectively while keeping enough informationfor the students to be able to comDlete the doze blanks.Such evidence supporting the every fifth word deletionpattern added justification to the use of this traditionaldoze pattern in the pilot project in the present researchdesign. This evidence built on the benefits of thetraditional doze procedure which are as follows:a) The traditional doze design of approximately 300words had proved to be of a convenient length as it allows28for the passage to be typed double—spaced on no more thantwo pages.b) Furthermore, this length and the every 5th wordpattern allowed for 50 blanks which was convenient forcalculating scores.c) The traditional doze design also allowed for thefirst sentence and the last sentence to be left intact togive the reader a sense of the general meaning of thepassage.With regards to every nth word deletion as a randompattern (as opposed to a rational pattern in which there iscontrol in the selection of deleted items), there was somedisagreement by Taylor (1956), Rarikin (1974), and O’Reillyand Streeter (1977). In order to improve the validity ofthe doze procedure as it relates to locating informationfor filling the doze blanks they proposed rationallydeleting only lexical items, i.e. nouns, verbs, adjectives,and adverbs. Martin (1973), rather than use the random“every nth” word deletion pattern, decided to delete using arational doze form in which he chose to delete any one ofthe parts of speech from among the following categories:“(1) nouns and pronouns (2) verbs (3) adjectives and adverbs(4) prepositions (5) articles and conjunctions” (p. 103).Like Martin, Cohen (1980) argued that using“discretionary judgement modified doze) in determiningwhich words to delete rather than simple random deletion”29(p. 128) had merit. For the present research the random(fixed-ratio) blank design was chosen for the pilot projectbut was replaced by a rational doze in the main project.The change was made to take advantage of the large number ofre—iterative—word clues in the text chosen by theRitsumeikan teachers for their university students.However, whatever the deletion pattern, the contribution ofthe above researchers’ focus on parts of speech is thecredibility given to the focus on parts of speech in thepresent research design.Also having a bearing on this research paper was thematter of scoring. There was a controversy over the use ofexact—word and acceptable—word fillers to replace thedeleted words. In her study with bilingual Hopi students,Streiff (1978) considered both the strong research evidencein favor of the exact—word method for native speakers(Oller, l972a) arid the speculation of Oiler of the fairer,yet reliable, sensible scoring method, using synonyms asalternatives of the exact—word, for non—native speakers. By“fairer” Oiler meant that by accepting alternate answerswhich conveyed the same meaning as the original word,students had a chance to get higher doze scores than theymight if only exact—word answers were accepted.Consequently Streiff used both sensible and exact-wordmethods of scoring but found like, Anderson (1973), andStubbs and Tucker (1974), that language proficiency could be30confidently indicated by the more exact and less timeconsuming exact—word scoring method.On the other side of the controversy, Cohen (1980) andHinofotis (1977a) found respectively that acceptable-wordscoring was more reliable and seemed to discriminate amongstudent proficiency levels more than exact—word scoring.From the above evidence it would seem that thecontroversy between the exact—word and the acceptable—wordscoring methods is not over. This, then, is one reason whythe present research design followed the example of Streiffand proposed to use both scoring methods for comparison.Another reason for the inclusion of the acceptable—wordmethod was due to the separate, but overlapping,observations of Stump and Hinofotis. Stump (1980) foundthat some students’ answers violated local constraints(grammatical and semantic influences) and other answersviolated long range constraints. Hinofotis (l978b) notedthat some answers were acceptable within the context of thesentence, but violated long range constraints.The above observations of Stump and Hinofotis pointedto the need to consider long range constraints in the choiceof scoring methods and to thus choose the acceptable—wordscoring method. Indeed, it was the goal of this researchpaper to see whether or not the students were looking beyondthe sentence of the doze blank. In fact, to determinethis, it appeared a modification needed to be made to theacceptable—word scoring method to take advantage of the31observations of Stump and Hinofotis. Consequently thisresearcher decided that an answer should be scored moreprecisely than ‘ust receiving a point for being acceptableand breaking no constraints either at the local or longrange level. It was felt that if students were supposed tolook beyond the sentence for clues (a more difficult taskthan looking at the local level), then they should berewarded more for not breaking long range constraints thanfor not breaking local constraints.The observations of Stump and Hinofotis and my goal tohave students look beyond the sentence level lead me todesign a scoring procedure which could possibly make finerdistinctions than either the exact-word or the typicalacceptable—word scoring methods. The scoring procedure wasdesigned to include both exact-word scoring and aninnovative technique which gave weight to each answeraccording to whether it was a) exact, b) acceptable to themeaning of the passage, or C) acceptable only to theparticular sentence in which it was located as each of theserelated to the availability and types of clues. Thisscoring procedure will be described in more detail in Tables14 and 15 located in Chapter Three.With regards to the spelling of the fillers, Petersonet al (1973) decided to allow for mistakes in spelling aslong as the intended word was evident and otherwiseappropriate for the doze blank. Following the above ideasabout spelling, the present research design adopted the idea32of accepting misspellings as correct. However, when thespellings made the word fall into a different grammaticalcategory, the answers were to be scored as 0. For the mainproject when the filler was available somewhere in thepassage, misspellings were not to be accepted. In suchcases misspellings would indicate that the student had notbeen totally aware of the same—word re—iterative clues.What are some considerations about students’ motivation andproficiency as these relate to setting up the doze tests?Researchers have pointed out that the traditional dozeis “an extremely difficult and anxiety-invoking test”(Cranney, 1972; Rankin, 1974; and O’Reilly and Streeter,1977). Thus this section of the literature review will dealwith various solutions to these problems given by variousresearchers. It will also show how the CCC is a logicalextension of the research and explore the merits of the CCCas a possible solution.Solutions being considered in this section include a)explaining the nature of doze tests to students, b) givinga guessing strategy for students to follow, C) supplyingclues within the text itself, and, d) providing feedback.The first solution, explaining the nature of the dozeprocedure to relieve the anxiety of the students, arose outof the efforts of the researchers who determined that adoze score of approximately 43% would compare to a reading33comprehension multiple-choice score of (approximately) 75%(on a standardized test) (Bormuth, 1967, 1968; Peterson,3,, 1973; Streiff, 1977; and Ashby—Davis, 1984a,). Thisinformation allowed the teachers in the present research toinform the students that they did not have to worry aboutseemingly low scores on the doze tests and should besatisfied with scores around 40%.The second solution came from the work of Ashby-Davis(1984b) who, after determining what good readers do whenthey take doze tests, outlined and shared their guessingstrategy. This strategy can be used to help other studentsto improve their doze scores. In fact, it was used in thepresent research for both the treatment and control groupsto make the task easier and worthwhile for all groups,regardless of the inherent advantage or disadvantage of thecolor-coded or other treatments. A copy of the guessingstrategy, as given in Table 1 below, was attached to theinside of each student’s test passage folder to be used asneeded during each testing session.34TABLE 1OVERALL READING COMPREHENSION STRATEGIESSuccessful doze text takers oftenfollow these strategies.a) They read the first and lastsentences of uninutilated text todetermine the gist (main idea) ofthe passage.b) Then they skim the mutilated texttrying to get clues to the gist.c) They read from the beginning to theend of the text, trying to findmeanings for the omitted words bychecking context clues before orafter the omitted words or fromgeneral knowledge related to thetext.d) Finally, they reread the entiretext when they have guessed all ormost of the words, filling in wordspreviously not guessed orcorrecting words already guessed interms of the total text. (p. 587)Ashby—Davis (1985, p. 587)The third solution revolved around various kinds ofclues being given somewhere on the doze passage paper. Twotypes were evident in the literature: a) partial fillers andb) matching words.Partial fillers and equivalent meaning clues, including“grapho—phonic, syntactic and semantic information (Goodman,351970; Propst & Baldauf, 1981) plus graphic clues such as“one short underline. . . for each letter of the deleted words”(Valmont, 1983) were suggested as ways to assist indetermining replacements for missing words. As has alreadybeen mentioned, the present research design followed thestandard doze procedure which uses standard—length blanks,so Valmont’s short underlining could not be used.One grapho-phonic clue, i.e. the initial letter of aword, as mentioned by Henk (1981), Soudek and Soudek (1983),and Valmont (1983) could have been used in the presentresearch to understand the effect of initial letter clues inimproving doze scores as compared to the use of color—codedparts of speech. In fact, Valmont found that using initialletters of the deleted words as clues was effective but itappeared to me that this could cause students to increasetheir focus on the immediate sentence of the blank ratherthan encouraging them to look beyond the sentence level. Inlight of the supposed localizing effect of initial lettersand other graphic aids on the search for clues to fill inthe doze blanks, the pilot project tested one group ofstudents using color—coded blanks. The purpose was todetermine whether the color-coded blanks are used bystudents to find answers using local clues at the expense ofthe use of more distant clues. When the mean scores of eachof the control and treatment groups in the pilot project(see the bar graph in Figure 1 in the Appendices) werecompared, it appeared that dust giving color-coded blanks36only as clues had the least positive effect and possiblyeven a negative effect on doze scores. Keeping in mindthis result of the pilot project and because there were notenough students in the main project to allow for four typesof groups, the color—coded blank treatment was dropped fromthe research design.The idea of matching word clues came from severalresearchers who provided the intended fillers somewhere inthe passage. Some fillers were included as follows:a) at various places in the margins (Ashby-Davis, 1985),b) in a list of five words on the right side of the pagefor matching one to one with the appropriate blank(called “list”) (Propst & Baldauf, 1981),c) below the doze blanks three choices (called the“maze”) (Pikulski and Pikulski, 1977) andd) below the doze blanks four choices (called “multiplechoice doze”) (Aitken, 1978).Neville and Pugh (1971) explained the benefit ofmatching doze procedure to the student by stating that:matching doze parallels andmeasures the actual reading processin that it often forces observableregression...The simple act ofrereading, erasing, writing newanswers, and again rereadingprovides considerable evidence thatthe reader is editing. (p. 105)37Another benefit was pointed out by O’Reilly and Streeter(1977). They said that in using the multiple-choiceprocedure “excessive difficulty and ambiguity of theoriginal doze testing situation appears to have beenconsiderably reduced” (p. 48). (Logically, the samebenefit could be given by the maze procedure.)Upon reflection, it seemed to me that each of the aboveattempts to include clues somewhere on the page of the texthad the same problem. Looking for words in the margins orunder the lines of the text is a rather unnatural processfor a reader to undertake. A more natural process, whilemaintaining the benefit of a matching type doze, might beone in which the clues could be found in the text and alongthe line of the sentences instead of below. The idea ofcolor—coding the parts of speech and the doze blanksappeared to be a reasonable solution in as much as reiterative words (acting as matching words) would stand outmore and possibly be identified as the necessary fillers.It should be admitted here that it would be verydifficult for any unmodified doze passage to containmatching fillers for all of the deleted words. The colorcoded doze passages are rio exception, so the color—codedprocedure cannot be classified as a true matching exercise,nor claim all the benefits of matching—doze procedures.But the color-coded doze procedure might be able to claimthe benefits in proportion to the number of matching fillers(same—word or same-root re—iterative clues) that are38available in the passage. (Whether or not this was true inthe present research will be discussed in Chapter Four.)In the passages selected for the pilot projectresearch, for the 50 deletions per passage, as few as 26matching fillers and as many as 32 matching fillers werelocated somewhere in the text. In those passages selectedfor the main project there were as few as 30 matchingfillers and as many as 40. For the purpose of the presentresearch, i.e. trying to determine if students could betrained using color—coded parts of speech to look beyond thesentence of the doze blanks for clues to the requiredfillers, it was not necessary to have matches for all thefillers. It was hoped that at least the students would beable to find the available matching fillers. This wouldhave suggested that the color—code parts of speech had hadsome influence on the purpose of this research. Tables 21,23, 25, and 27 in Chapter Four show how many matching fillerblanks were filled and how many were filled correctly.It must be admitted here, notwithstanding the proposedadvantages of the CCC, that the rationalized color-codeddoze procedure was somewhat cumbersome to construct but ina different way from the other matching doze procedures.In their multiple-choice doze (matching) design Streeterand O’Reilly (1977) increased the amount of effort needed toprepare the doze tests. The increase was a result oftaking care to use distractors of the same part of speech toensure students would have to look at meaning instead of39‘ust syntactic clues to find the blank filler. Although therationalized color-coded doze skirted this extra high levelcognitive effort, it required a great deal of lower levelcognitive energy to locate the re—iterative—word clues.Another drawback in using the CCC was that the CCCrequired the maker to analyze the parts of speech and thencode them with the defined colors consistently andcorrectly. Sometimes it was not as easy as one might thinkto assign the appropriate color. For example, in thesentence, “He worked at a French Canadian Hospital”, theword “French” can be colored “green” to indicate that it isan adjective. But in the sentence “He worked at a NativeHospital” the word “Native” is not as easily coded if“native” is also used in the sentence “He is a native.” Thedilemma is whether “Native” should be colored “green” toshow it is an adjective or “blue” to show it is a noun.”Fortunately, however, in the passages used in the presentresearch there were not very many difficulties like this.On the surface, there was the additional difficulty ofphysically color-coding the doze passages. Each word hadto be given a computer code to color the word appropriately.For the CCC project this took a lot of time, especially forthe pilot passages. But a revised technique helpedtremendously in reducing the coding time for the mainproject passages. The techniques will be described inChapter Three and in more detail in Appendices B and F. Inthe future, with more advanced computer hardware and40software, the coding process should be even much less timeconsuming.A fourth solution to help students improve theirperformance on the doze procedure is to provide feedback.In fact, Rankin (1974) stressed the lack of improvement inreading comprehension when doze materials were used withoutfeedback. Ideally each doze test should be reviewed withthe students but time and security concerns because of theuse of several classes doing the same set of tests atdifferent times made this unmanageable in the present kindof research.In the CCC project two kinds of feedback were given.The first kind of feedback was suggested by the matchingdoze procedure, i.e. when there iS a set of blanks andanother set of words to fill the blanks such that each wordcan only be used once and each blank only has one correctfiller. It was felt that CCC, although it can not claimthis advantage, might be able to gain from the colored-part-of—speech clues, where the colored blank must be matchedwith a filler of the same color representing the same partof speech. The second, more traditional though partial,kind of feedback was given when all the students wereallowed to look at the originals of the practice passages.In the pilot project the students took about a five minutelook after the practice test in the subsequent testingsession while in the main project the students looked at the41answer key for about five minutes right after each practicetest.In Chapter Five the above CCC solutions to the problemof student doze test anxiety will be discussed in terms ofthe outcome of the results of the pilot and main CCCprojects.What are the recent education trends that have lead to theuse of color—coded doze procedure?So far in this chapter on the review of the literaturethe doze procedure has been defined and variousmodifications have been presented, some of which were chosenfor the present research. Those choices were made in lightof recent trends in education including a) passage analysis,I,) information gap, c) educated guessing, d) vocabulary andgrammar in context, and e) redundancy.One of the most important trends at this time and onewhich makes a great deal of sense is the use of passageanalysis rather than just sentence analysis. Long (1981)urged students to go beyond “knowing the lexical value ofindividual words, or of retrieving factual information fromindividual sentences” (p. 72). Richards (1980) stressedthat “language in any authentic communications event cannotbe understood entirely by considering sentences inisolation” (p. 64).42Mohan (1986) has been concerned with discourse andemphasizes that an “important part in competence in readingis the ability to draw inferences from written text.” Hesubsequently defines inference as “a relation betweensentences” (p. 129).Referring to doze and discourse, Foley (1983) notedthat Chihara and Oiler, (1977); Anderson, (1980); andBachinan, (1982) believed that doze tests are sensitive todiscourse constraints. Cohen (1980) thought that dozeprocedure could even encourage students to consider thepassage as a whole rather than viewing it as being composedof isolated sentences.A second important trend in education, related somewhatto passage analysis and certainly tocloze procedure, is theidea of reducing or filling an information gap. Morley(1972) contributed to ESL with her many kinds of gap fillingexercises which have motivated students to look for missing(needed) information and thus caused them to use English tocommunicate. Mohan (1986) re-iterated Morley’s idea andexpressed the need for students to increase their “abilityto communicate successfully across wider and widerinformation distances” (p. 110).In this research paper the goal was to determine if. infact, students were able with the doze to be trained how tofind information (clues) for fi1lin in blanks, at greaterdistances than just at the sentence level. Correct and/oracceptable responses to blanks which require the use of43beyond-sentence clues would indicate that the students havemade use of discourse clues and constraints.A third trend in education revolves around educatedguessing, as opposed to rote learning. Foley referred toone kind of guessing in language learning as “crrammarexpectancy” in which the student as decoder relies (orshould) rely on the highly redundant nature of a message tomake accurate restorations. These “accurate restorations”are a result of students processing information from withinthe passage. The relation of guessing to comprehension ofdiscourse can also be seen in Katz and Fodor’s (1965)definition of literal comprehension (as mentioned byO’Reilly and Streeter) which refers to the understanding ofgrammatical and semantic relations both within and acrosssentence boundaries.In going a little further in understanding Katz andFodor’s definition, a fourth trend in education becomesapparent. It can be referred to as learning vocabulary andgrammar in context. This fits in well with the idea that thediscourse level is more important than the sentence levelfor teaching vocabulary (Chastain, 1976; Rivers, 1968;Nilsen, 1976; and Judd, 1978). Also Judd commented thatlearning words in context helps retention and a wider andmore precise understanding of them.On the subject of grammar, Wilkins (1975) and Janssens(1977) took opposing positions. Janssens preferredcommunicating and accomplishing goals to learning about44grammatical rules. On the other hand, Wilkins, alsobelieving in communicating ideas, stressed the role ofgrammatical rules in the learning process as they are thefoundation which allows language functions to successfullycommunicate the intended meaning of speakers or writers.From looking carefully at these opposing argumentsabout grammar and communication, it is evident that there ispossible common ground, i.e. students must know how to usegrammar actively as a tool in context rather than just toknow about it passively.Still on the subject of grammar and its influence onthe doze procedure, Cohen (1980) explained an importantconnection between grammar and doze. In being able tocheck for awareness of grammatical relationships, andincluding inflectional clues to cohesion in discourse,ungrammatical doze answers can provide evidence for too—local reading, a trait common to non—native readers.Evident in this explanation for the purpose of the presentpaper is that students do (or if they do not, should) usegrammatical clues to be able to fill doze blanks, grammarbeing considered as a tool rather than as a nuisance in thenon—natives’ attempts to become proficient in English.Aitken (1977) continued the argument for the value ofgrammar as it refers to using redundancy to fill in dozeblanks. He argued that without knowing grammar the studentcould not take advantage of the redundancy in the languageand thus would have more difficulty with doze tests.45It is the students’ awareness and understanding of thisredundancy as it is expressed through the vocabulary and thegrammatical structure that the color—coding of parts ofspeech was aimed to improve. The present research. thus,attemDted to use color to make students more aware of thevocabulary and structures in the practice passaaes to beable to use them to auess the words to fill the dozeblanks.If Aitken is right that students’ awareness ofredundancy is an indicator of their overall languageproficiency, and, if the color—coded clues are shown toheighten this awareness, then it could be said that thecolor—coded doze procedure can lead to greater languageproficiency.Although the literature about the doze procedure thatI reviewed pointed to the advantage of students being awareof redundancy, I also found a negative reaction by Tuinman(1972) to an increased awareness of redundancy in a passage.He discovered that when students tried to sift through agreater number of non—target items, they lost speed andaccuracy in finding the correct response for the dozeblank. This lead me to question the effect of color-codingall the words, if all the coding does is increase the numberof words to be scrutinized. Color-coding parts of speechdoes not increase the context in terms of number of words,but hopefully in the amount of awareness of context cluesavailable. The result may be the problem Tuinman noted46above, i.e. that students will be slowed down by the extrainformation made aware to them. On the other hand, it canbe argued that color—coding parts of speech should lead tothe students focussing on clues given by the particularparts of speech rather than on all the parts of speech andtherefore bring some speed back into the searching processby reducing the number of words to be looked through.The immediate question thus would be whether or notcolor—coding all the words and blanks in a doze passagecauses an information overload which in turn decreases thestudents’ scores? This question will be discussed furtherin Chapter Five after the research data has been collected.Summary of language trendsWhether the recent trends in ESL deal with discourse,information gaps, guessing, redundancy, vocabulary, orgrammar, they all can lead toward language proficiency.Aitken (1977) with the help of Foley (1983) ties the recenttrends together. In Table 2 below the fourth quote wasfound in Foley’s article and the other quotes were given inAitken’s article.47TABLE 2QUOTES FROM *AITN (1977) AND **LEY (1983) ABOUT RECENTEDUCATIONAL TRENDS.*Language proficiency is more thanmastery of a specific nuniber of discretestructure points and lexical items. (p.62)*Reading comprehension in ESL depends ona knowledge of the meanings of words,phrases and sentences, and ofarrangements of words, phrases, andsentences according to the conventionsof written English. (p. 62); Thomas(1970, p. 164)*Three layers of language meanings thatmust be dealt with in readingcomprehension(1) Meaning carried by the lexicalitems.(2) Meanings carried by the grammaticalstructures.(3) Socio—cultural meanings.” (p. 62);(Fries, 1963)**Because the message is normally highlyredundant, the decoder should be able tomake accurate restorations of many ofthe blanks that appear in the passage.(p. 58)*The correspondence between the ESLstudent’s conception of Englishredundancy becomes an index of hisoverall language proficiency in English.(p. 65)It would seem possible that the doze procedure, in whichgaps need to be filled with information given by clues48embedded in one or more vocabulary items and grammaticalstructures located somewhere in a passage either within orbeyond the sentence of the blank, could also yield an indexof the non-native adults’ proficiency in English.How does color-coding as a trend fit into the presentresearch?There is no doubt that color is playing an everincreasing part in our lives. Black and white televisionand movies, for the most part, are out of style. Color iseverywhere in the entertainment field, in business, and ineducation. It is in the business field, however, wherecolor—coding has been exploited the most, such as fororganizing office files and for helping customersdiscriminate amongst product sizes. Perhaps because offinancial restrictions education has only been able to beginexploiting the use of color. There are now instructionalfilms in color, especially in the video format, and booksare illustrated in sophisticated four—color color printing.Some books are organized into chapters or topics by colorand some even have key words in a passage highlighted in onecolor. But there the color—coding stops. Multi—coloredprinting is expensive.Attempts to use color or color-coding in teachinglanguage have been made but subject to financial andtechnical restrictions. For example, by at least 1926, in49the tenth edition of her book called “On the Teaching ofEnglish Reading”, Nellie Dale used colors to differentiatethe graphemes of English. She used “red” for short vowels,“blue” for voiceless consonants, “black” for voicedconsonants, and orange for semi—vowels. However, other thaneight pages of colored plates (grapheme charts) and threepages of colored examples, Dale’s book is in the usual blackprint. In another language text, i.e. the 1954 edition of“Reading with Phonics” the authors, Hay and Wingo, tookadvantage of improved color reproduction techniques to haveseveral multi—colored pictures, and colored words on almostevery page. However, they only used one color, “red”, tosupplement the basic black print. Actually they used “red”only to highlight the graphemes being studied in theparticular lesson. Unlike Dale they did not color-code thegraphemes.The most well known user of color and color-coding forteaching languages, especially non—native languages, wasGaleb Gattegno. In his paper called “Words in Color” (1964)he explained how he used color to divide the graphemes ofEnglish into their appropriate phonetic categories. Inother words, his purpose was to help students match thespelling of words to the way they are pronounced. What Dalehad started, Gattegno made even more detailed and colorful,using not four colors including black, but 21 colors forvowel sound categories and 30 colors for consonant soundcategories. Like Dale, Gattegno used colored charts to50display the colored categories. He, too, was unable toprint a text in color. However, cost was not given as hisrationale. Rather, from an educational point of view, hebelieved that students should write (and thus read) in onecolor.Of the above authors, the most influential on thepresent research paper was Gattegno (1964). Although he wasprimarily working at the grapho-phonemic level, he didcolor—code parts of speech. He referred to this in his1964 article called “Words In Color” where he stated thatthere is a card for each word, the card being the color ofthe word’s part of speech. He further noted that if a wordhad the potential of being used as more than one part ofspeech, then the card would be of the appropriate number ofcolors. However, he did not mention which color meant whatpart of speech.Gattegno’s research and use of color—coding isextremely important for the present research paper.Concerning color blindness, even when it existed, he saw inhis teaching that there was no interference. He stated thatstudents could still differentiate the shades of the colors.Even more significant is what he said about his method beinganalytic and synthetic at the same time, while other methodswere either one or the other. He pointed out that color—coding could help students differentiate the graphemes fromeach other and put them into their appropriate pronunciationgroup simultaneously. In “Words in Color” Gattegno provided51some testimonials from teachers who obtained good resultsand Dodds (1966) stated that this was true with beginningreaders, remedial readers and illiterate adults. On theother hand, Cunliffe (1986) in her unpublished critique ofGattegno’s colored words, pointed out the concern of McHugh(1968) that the positive effects of Gattegno’s color-codinghad a lot to do with the dedication, motivation andcompetence of the teacher. It should be noted here thatwith the color—coded doze procedure the student has tointeract more with the passage than with the teacher.Therefore, it would appear that McHugh’s criticism of theintervening effect of the teacher should not detract fromthe value of the CCC.In the 1960’s Gattegno’s color-coding as part of his“silent way” method was popular in schools in the UnitedStates and was still being used in the late 1970’s to teachother languages to Peace Corps (American) and CUSO(Canadian) volunteers. Being one of the latter volunteers,the author of the present research started using color—codedparts of speech in teaching English to inmigrants in Canadaand to nationals in Saudi Arabia.In Upper Volta in 1975 Montety (1977) used the silentway and colored—coded lines with colored chalk on theblackboard to elicit past tense sentences from his studentsas part of their review. For use without color when it wasnot available he devised a set of various shaped lines to dothe equivalent. Although he coded words, it was not until52later that other teachers started to use color—coded partsof speech. However, at this time, there is only one otherteacher, known to the author, who is using color—coded partsof speech to teach English as a Second Language, and thedistribution of colors is a little different from the systemused by this researcher. This is partly because she hasdefined her categories of parts of speech slightlydifferently and partly because she has labelled some of hercategories with different colors from those used by thepresent researcher.From my review of literature and my experience withcolor-coding in teaching English it can be seen that usingcolor—coding is not a recent trend. Yet it has shownpotential but has been defeated, at least in part by thecost of the necessary technology, and even by the. lack oftechnology. However, with ever newer and cheaper technologyin color printing, the time may be right to revive the useof color-coding. It should be pointed out that, because ofsimilar reasoning to Gattegno’s that students should useblack-on-white when they write, the color-coding should bephased out. This means that color-coding might be used atthe. beginning of a course but by the end there should be nocolor—coding at all. The present research designincorporated this idea and thus the post-test was leftuncolored for the treatment groups as well as the controlgroups. Meanwhile it was felt that the advantage in colorcoding that Gattegno found in both differentiating and53categorizing for graphemes also could apply to theseanalytic and synthetic techniques in finding clues in thecolor—coded doze procedure. By using only five colors (todistinguish the major grammatical categories) instead ofGattegno’s 51, it was thought the load on the memory couldbe reduced and decoding made less confusing.To summarize how color—coding fits into the presenttrends and thus, into the doze procedure research, it washypothesized that by being given color-coded parts of speechstudents would be helped to differentiate the parts ofspeech from each other, and then to mentally group thesimilar parts of speech in order to look through the passagefor the words in the same group to fill the informationgaps. Sometimes (from 52% to 64% of the time in thepassages analyzed for the pilot research and 60% to 80% ofthe time in those for the main CCC research) the fillersfound in the text were exactly the same or had the same rootas the deleted word. Sometimes the key clue words in thepassage were synonyms or antonyms in the same part ofspeech. However, as it is obvious that all the key cluewords cannot always be in the same part of speech group asthe deleted word, so the strategy cannot be foolproof. Thusthis strategy at best can only be classified as one ofguessing but an educated rather than a wild one. This isbecause the color-coded guessing strategy allows thestudents an organized method of searching through the text.With the time and energy saved by looking through the same54part of speech, the student can look systematically throughother parts of speech for roots of words which could havethe required meaning and then adjust that word to thecorrect part of speech. Given grammatical relationshipsthat were highlighted by the color-coding, suchrelationships were expected to assist the students infinding words to fill the doze blanks also. Such use ofgrammatical relationships were hoped to be reflected inhigher doze scores for the CCC groups than for the controlgroups.To put the above briefly, the proposed advantage of thecolor—coded doze procedure was that by using it CCCstudents would be able to make a more organized andconsequently a more thorough if not faster search for cluesto fill in the missing words. If the proposal proved true,it would seem reasonable that it should be true for inter—sentential searching, not just for intra—sentential cluesearching. With the benefits of the guideposts provided bythe color-coded parts of speech, it was anticipated thatstudents would find it less intimidating and also easier tolook beyond the sentence of the blank for clues. Thereforethey would be able to fill in more blanks correctly, thusimproving their doze scores. It was also hoped thatstudents would be able to internalize the parts of speech sothat the color-coding could be removed and still allow thestudents to use the same parts of speech guessing strategyacross sentence boundaries.55WHAT IS THE KEY AREA OF INVESTIGATION UNDERLYING THISRESEARCH PAPER?To answer this question there are four subtopics to belooked at. They are:a) the link of discourse to the nature of dozeprocedure,b) the nature of doze procedure——global or discrete,c) the chronology of the global/discrete controversyd) the motivational nature of non—native speakers ofEnglish, ande) the role of color in assisting non—native studentsto locate more global clues with the aim ofimproving the students’ doze procedure scores.How do discourse and the nature of doze procedure overlap?Shanahan and Kamil (1982, 1983, and 1984) in theirresearch on doze procedure tried to show that students useintra—sentential rather than inter—sentential clues infinding fillers for doze blanks. The question theyconsidered was whether doze is global or discrete innature. Their starting point was opposite to thecontemporary opinion which was dominated by Halliday andHasan (1976), and Oiler et al (1978) who were convinced fromtheir own research that doze was global; that is, it didtest the students’ use of context clues across sentenceboundaries. Their research will be discussed briefly laterin Chapter Three.56Given the recent trends, especially about being able tocomprehend text at the discourse level, Shanahan and Kamil’sresearch had significant influence on this paper. In fact,it provided a starting point for the present researchquestion, i.e. can students be trained in the dozeprocedure such that they look beyond the immediate sentenceof the doze blank to find clues to help them fill theblank? An answer to this research question will add to thecontroversial answers supplied by the review of theliterature which follows immediately below.Is doze essentially global or discrete?On the global side of the arguauent according to Foley(1983, p. 59) were Halliday and Hasan who saw a need for theuse of both clues within and beyond the sentence. Oller(1975) and Chavez and Oller (1977) believed students couldbe trained with the doze procedure to use global readingcomprehension skills. Cohen (1980) even defined doze asbeing able to test the ability to read and write cohesiveEnglish.Foley (1983) gave credence to the global nature of thedoze by explaining the mechanism of this global languageproficiency in the following way:57In doze procedure all sorts ofdeletions, whether they be content wordsor connecting devices, carry with themconstraints which may range backwardsand forwards across several sentences.This places a strain on the short—termmemory which presses the student’sgrammar expectancy into operation, theaccuracy with which the student is ableto supply the correct or acceptableresponse can therefore be taken as anindex of the efficiency of the student’sdeveloping ‘grammatical’ system. (p.59)Ashby-Davis (1984) also gave the argument credence byexplaining that doze:...draws at once on the overallgrammatical, semantic, and rhetoricalknowledge of the language. . . . andstudents have to understand key ideasand perceive interrelationships within astretch of continuous discourse, andthey have to produce, rather than simplyrecognize, an appropriate word for eachblank. The focus of the task involvedis more communicative than formal innature, and it is therefore consideredto reflect a person’s ability tofunction in the language. (p. 99)Streiff (1978) listed supporters of the globalproficiency of doze. The list included Darnell (1970),Oller and Conrad (1971), and Oller (1973 and 1975) andHanzeli (1977) who apparently had shown in their researchthat doze could lead to students having global Englishproficiency.Stump (1978) tried to prove the global nature of dozeby reporting the ability of the doze procedure to “so58accurately predict scores on both the Lorge—ThorndikeIntelligence Tests and the Iowa Tasks of Basic Skills (inthat) all are essentially measuring the same thing - globallanguage proficiency” (p. 57). Ashby-Davis offered the samekind of proof when she reported “substantial concurrentvalidity (of doze) ...as an integrative test of overallproficiency in English as a second language (Oiler & Conrad,1971; Oiler, 1972; Irvine, Atai & Oiler, 1974; Stubbs &Tucker, 1974; Hinofotis, 1980)....(on tests) such as theUCLA English as a Second Language Placement Examination” (p.99)Although through the years a great deal of support wasgiven to the idea of doze being global in nature, evenproponents such as Streiff admitted some contrary evidence.Streiff admitted the observation of Klare et al (1976) thatdoze answers are most likely to use constraints that occurwithin a range of only four or five words before or afterthe blank in question. Likewise, Ashby-Davis includedAnderson’s reservation (1980) about the doze being a lowlevel skill, suggesting it was not integrative, butdiscrete, that is, working only at the sentence level withregards to finding clues to fill in the doze blanks.Foley (1983) came to a compromise between the globaland discrete position seeing the doze as an intermediateskill “able to assess the student’s knowledge of the syntax,lexis and rhetorical devices...(rather than) the extent the59student has understood the conceptual world of the writer”(p. 61).If many researchers favored the global argument andwere convinced that the doze was effective for teaching,the notion was to be vigorously attacked by Shanahan andKaiuil (1982, 1983, 1984) among others. In 1986, Reutzellooked back at the research of Shanahan, Kamil and Tobin(1982), and Leys (1983), plus the literature review ofJongsma (1980). He noted the researchers’ conclusion thatthe traditional doze procedure could not effectivelymeasure inter—sentential comprehension, and Jongsma’sconclusion that doze procedure was rather ineffective forteaching reading comprehension.The debate involving the global versus discrete natureof doze has been fervent. Although the maor points ofview have been identified and described in this paper, to dojustice to the main researchers, a chronology of the battlewill now be given. The chronology will also provide thebasis for the research questions in the present researchpaper and for the design of the methodology.60What is the chronology of the global/discrete controversy?1971: Halliday and HasanHalliday and Hasan (1971) wrote that “the dozeprocedure transcends the confines of a single sentence” (p.51).1972: Carroll; Chihara & Oiler (1977)Chihara, Oiler et al (1977) pointed to one of theoriginal key players in the global/discrete question asbeing Carroll in 1972 who wrote that the “doze procedure isone of the techniques that has been proposed as a possiblebasis for investigating discourse constraints” (Carroll,1972). They went on to present Carroll’s 1972 claim thatlinguistic clues are usually in the same sentence as theblank and that grammatical clues play a greater role thansemantic clues in doze scores.Chihara and Oiler (1977) using “two passages ofprose...selected from texts written for non—native speakersof English” (p. 94) tested native and non—native speakers.The later were either at the basic, intermediate, oradvanced proficiency level. They were given scrambled andregular forms of the passages. Chihara and Oiler concludedthat “the doze procedure is sensitive to discourse61constraints ranging across sentences” (p. 68) and that thehigher the proficiency level of the students, the morelikely they are to take advantage of the constraints.Chihara and Oiler put forward a strong case for theglobal nature of the doze. By testing students atdifferent proficiency levels, they were able to use theproficiency factor to explain that any lack of influence bydiscourse constraints is a result of the students, not thedoze procedure itself. They found that the more proficientthe ESL students were (as determined by their grade levels)the higher were their doze scores. This observation willbe pursued later as it has some bearing on the nature of thepresent research paper.If Chihara and Oller were correct, then the color—codeddoze procedure should be able to increase the students’proficiency as indicated by a) improved doze scores and byb) more correct fillers, fillers of the kind which depend onclues from other sentences beyond that of the doze blank.The increased proficiency should come as a result of theassistance given by the color—coded parts of speech. Thecolor—coding should be able to make the students more awareof the parts of speech both at the intra-sentential andinter-sentential levels. The parts of speech in turn shouldprovide clues and constraints as to the required blankfillers. For example, a blank requiring a noun to fill itas indicated by a “blue” line, would indicate to thestudents that they ought to look through the passage for62“blue” words, i.e. nouns and pronouns. Because in at leasthalf the instances in the present research’s passages, thematching filler (the same-word as the deleted word) islocated somewhere in the text, usually inter-sententially,students stand a good chance of taking advantage of thecolor-coded parts of speech and blanks to easily find thematching fillers. Such discoveries would improve students’doze scores. The ability to find matching fillers, atfirst through the use of color-coded parts of speech, andfinally through the use of non—colored parts of speech, iswhat was being explicitly tested in this research project.An analysis of the doze test scores (calculated using boththe exact—word and the acceptable—word scoring methods) alsomight show whether or not students could be trained to usecolor—coded and non—colored parts of speech to find fillerswhich were graphically different to any of the words in thepassage. The clues necessary to locate these kind offillers are shown in Table 3 at the end of Chapter 2. Thetable outlines the various types of cohesion in passages.1979: YamadaChihara and Oiler’s findings, however, were to bechallenged by Shanahan and Kamil, among others, includingJun Yamada in 1979. Yamada built on:63a) the work of Carver (1975 — 1976), who “stated thatsentence order would probably have little effect ondoze scores (p. 70);b) Chihara, Oiler et ai, who compared scrambled tounscrambled doze; andC) Halliday and Hasan who had developed a “theory ofinter—sentential cohesion” in 1976.Yamada tested the hypothesis that the location of cohesiveclues, either within or beyond the sentence of the blank,must cause differences in doze scores.Yamada’s findings agreed with those of Chihara’s andOiler’s that scrambling the sentence order does make thedoze significantly more difficult but he continued that:doze scores are highly correlated withdiscrete-point grammatical test scores(due to the nature of the cohesion inthe discourse). (p. 76).Yamada suggested that most of the time the sequential dozeprocedure tests “the subjects’ intra-sentential (withinsentence) ability” (p. 76) as opposed to “the subjects’inter-sentential (between-sentence) ability” (p. 76) becausethe number of intra—sentential cohesive items is far greaterthan the number of inter—sentential cohesive items. Thus toall intents and purposes the students were encouraged by thenature of the cohesion in the passages themselves to look64for cohesive clues primarily in the sentence of the blankfor both the scrambled and unscrambled doze tests just thesame as they would in a discrete-point grammar test.Yamada thus concluded that:a doze test is primarily a discretegrammar test rather than an integrativereading comprehension test. (p. 76)To restate the point in a slightly different way, it shouldbe noted here also that Yamada, unlike Chihara et al, foundthat the primary factor in the debate was the nature of thecohesion, i.e. on the availability of cohesive ties in thepassage. The primary factor, he thus believed, was not theproficiency of the students. This point, like the oppositemade by Chihara and Oller, had a bearing on the design ofthe present research. On one hand, Chihara and Oller’sideas led me to examine the nature of students as they haverelated to the doze procedure, i.e in terms of proficiencylevel. On the other hand, Yamada’s comments wereinstrumental in the analysis and subsequent selection ofdoze passages in terms of availability of matching fillersto put in the doze blanks.1980: ThomasSusan Thomas’ research, like Yamada’s, put the onus ofdoze results on the doze itself rather than on thestudents. But instead of looking at cohesion within andbetween sentences, she looked at content and function words65and their predictability in lesser and greater amounts ofcontext. She restated that function words are easier thancontent words to predict, but she found that:greater context facilitates dozeinferences for content words butinhibits doze inferences for functionwords. (p. 52)She carefully explained this by saying that:global predictions involve(d) generaland more long range estimations of whatis coming next in a readingpassage....In contrast, focalpredictions arise out of specificevents, deal with more localanticipations, and are dispensed withmore quickly... (and) a larger contextactually inhibits the correct predictionof function words...(as) the reader maynot be particularly attentive to (nearbysurrounding) clues because of otherredundant clues in the remainingcontext...Therefore, the availability ofa large amount of contextual informationmay simply provide more alternativeclues than are necessary...This kind of‘information overload’ is forced uponthe reader and inhibits. ..exact functionword predictions. (p. 52 - 53)Given the above ideas, Thomas was able to explain apossible reason why students tend to use such local clues(constraints) in order to guess what words should be put inthe blanks. The cause can likely be discovered in easymaterials which by their very nature contain relatively more66function words than more difficult materials do. Thomassupposed that:the greater the proportion of functionwords in a given message, the smallerthe influence of greater context for thetest as a whole should be. (p. 53)Thomas presented for further study a subsequenthypothesis to test her belief in the global side of thedebate. She hypothesized that:the use of materials with more difficultreadability levels and/or therestriction of deletions to contentwords will expand the boundaries ofcontextual constraints operating indoze tests. (p. 53)Thomas’ (1980) description of the tug of war betweenthe proponents of content and function words plus hersubsequent hypothesis called for a solution for which thecolor-coded doze (CCC) procedure seemed to fit. The CCCseemed appropriate because it distinguishes (most of) theparts of speech from each other (blue = nouns and pronouns,red = verbs, green = adjectives, purple = adverbs, black =conjunctions, prepositions, interjections, and “wh” words)and highlights the content words (blue, red, green, andpurple) as opposed to the function words (black). At thesame time the color—coded doze procedure still apparently67allowed for the benefits of the random every nth worddeletion pattern.In terms of difficulty of passages, the presentresearch pilot passages were identified at an intermediatelevel at two ESL schools and the type of vocabulary,structures, and length of sentences follow the curriculumsof those levels at those schools. An IBM computer programcalled “Readability Calculations, Version 1.00” used forrating readability, calculated the readability levels of theselected passages in the pilot study as being as low asgrade six and as high as grade eight on the Flesch scale andbetween grade six and nine on the Fry graph. A trial at oneschool in a mid—intermediate class with the easiest passageusing the traditional doze format showed that the passageswere not too easy nor too difficult and were able todistinguish the different language proficiencies of thestudents. The above data about the passages used in thisresearch plus conversations with teachers having used dozetests with students pointed to the need to find ESL studentswho were as proficient as possible. This meant that thestudents should be studying at as high a grade level aspossible. Due to the number of students needed and theavailability of such numbers, the upper-intermediate levelwas agreed upon for the pilot doze passages. Although itwould have been preferred if the students were at a higherproficiency level for the selected doze passages, Thomas’hypothesis calling for more difficult readability levels to68expand the boundaries of contextual constraints, presumablygave support to the selection of the particular proficiencylevels of the students in the pilot project. Her hypothesisalso seemed to give credence to the use of even moredifficult passages with the university level students in theRitsumeikan Program. Due to the requirements of theuniversity program the selection of the doze passages waslimited to two of the texts used in the regular program.Passages were thus chosen from the easier of the universitylevel texts in which the sentences still tended to becomplex with many technical terms. The seeming advantage ofthese passages was that many key semantic terms and severalfunction words were repeated once or more. This highly reiterative nature of the words in the passages, especiallyfor inter-sentential clues, in the main project (60% - 80%intra— and inter—sentential re—iterative clues, and 44% —62% for inter-sentential clues) appeared to bode well forstudents finding intra- and inter-sentential re-iterativewords to fill the doze blanks.1982: Shanahan and KamilIn 1982 Shanahan and Kamil used intact/doze!sequential/scrambled passages and “confirm(ed) earlierfindings that the doze test is insensitive to theintegration of information across sentence boundaries” (p.69207). However, they made some concession to their opponentsin the global/discrete debate but qualified this as follows:It is not that students fail tointegrate information across sentenceboundaries when completing a dozeexercise, but only that the informationdoes not aid in the sentence completionactivity. (p. 207)In 1983 Shanahan and Kamil explained their justificationfurther in that:sub)ects perform no better on typicalrandom nth word deletion doze teststhan they do on the same tests in whichthe sentences have been placed in randomorder. (p. 123)1982: Bridge & WinogradBridge and Winograd in 1982, using verbal (think-aloud)protocols from ninth graders (both good and poor), foundthat students did use inter-sentential clues but the amountof use depended on the ability of the readers and on thetype of cohesive ties. They found that conjunctive andreferential items were more likely to be determined by theuse of inter—sentential clues. Lexical items weredetermined by the use of intra—sentential clues. Based onthe result that good readers used within-sentence clues 54%of the time while poor readers 38% of the time, Bridge and70Winograd believed that deleted lexical items could bediscerned from within—sentence clues. Furthermore,believing that most ties in language are lexical ones, andthus also occur in the traditional every fifth word deletiondoze procedure, Bridge and Winograd concluded that thetendency is toward using intra—sentential clues. Theirsubsequent solution was to use doze passages “in which theresearcher selectively deleted items which require theintegration of information across sentences” (p. 310).The significance of the contribution of Bridge andWinograd’s research to the present research paper is that itexplained how the nature of the passage controls the way inwhich students try to find the solutions to the dozeblanks. It also explained in what direction, i.e. towardthe intra—sentential clues —— the lexical items, andsomewhat towards the inter—sentential clues—— thereferential and causative items. By setting out the typesof cohesive ties1 Bridge and Winograd prompted in thepresent research paper the analysis of a number of passagesto find the distribution of ties and subsequent selection ofthe passages with the blanks which had their replacementssomewhere in the text. (The details of this and other textanalyses will be presented in Chapter Three.) The presentanalysis chose to find same—word and same—root re—iterativeties2, i.e. lexical and syntactic clues, as they seemed tobe the ones most easily found, once highlighted, in order tobe used to fill in the doze blanks. Other types of71lexical, referential, and conjunctive ties were not analyzedfor passage selection purposes, but in the main treatmentthey were color—coded and anticipated to be even moreevident than Bridge and Winograd found them to be. Itshould be pointed out again here that in the analysis ofsame—word and same—root re-iterative clues in the pilotpassages and the main passages, the number of deleted itemsappeared at least once elsewhere in the text as high as 64%and as low as 52% of the time and 80% and 60%, respectively.(This means that if students were using the color—codedclues, most of them beyond the sentence of the blank, tofind the appropriate re-iterative items, the color-codedscores were expected to show as much as a similar proportionof improvement over the control (non—color—codedtraditional) doze test. Any other improvement would haveto have come from the influence of color—coding on the othertypes of clues.Because of Bridge and Winograds suggestion about usinga rational (non—random) deletion doze, for each of thechosen passages an attempt was made to increase the numberof blanks with re-iterative clues by deleting a differentset of words. However, for the pilot passages, as theincrease was not very great, and due to the large number oftreatments already and the limited number of studentsavailable, it was left for the main experiment. It shouldalso be noted that it is beyond the constraints of thisresearch to have included an analysis of all the types of72cohesion mentioned by both Halliday & Hasan, and Bridge &Winograd.1983: LeysThe following year (1983) Leys et al replicatedShanahan & Kamil’s 1982 study using junior high schoolstudents instead of adults. The Leys study asked threequestions:1. Does prior knowledge of the passagetopic aid doze performance?2. Does the sequential as opposed to therandom ordering of passage sentencesaid doze performance?3. Does sequential context as opposed torandom or no context aid performanceon a target doze sentence? (p. 112)Ley’s answer to each question was, “No”, and therefore,she seemed to confirm...Shanahan and Kamil’s (1982)findings...(that) the traditional dozeformat (i.e., every fifth word deletedand exact replacement criteria) does notappear to be sensitive to theintegration of information acrosssentence boundaries. (p. 113)To explain the results, Ley drew on the idea of Weaverand Kingston (1983) that the traditional doze proceduretends to measure the language skill of readers (rather thantheir understanding of the passage).73The first contribution of Shanahan et al and Leys’research to the present research paper was the pressure totake another, but closer look, at the discrete (intra—sentential) versus the global (inter-sentential) question.The second was the pressure to look again at the scrambleddoze method. Thus, a subsequent study is needed, based onthe present research paper, in which two secondarytreatments could be given, one using a scrambled dozemethod with color—coding and the other using the samesentence method without color—coding. The scores on both ofthese methods could be compared with scores on the colored—and uncolored—unscrambled forms of the passages. WhereasShanahan et al and Leys did not find higher scores forunscrambled passages over scrambled ones, and whereas it wasthus interpreted that students were only using within-sentence clues in both varieties, it can be hypothesizedthat if there are higher scores for the CCC for theunscrambled treatment when compared to the other threetreatments, then this would show that there is indeed someinfluence of the color-coding on the students to look forinter—sentential clues to fill in the doze blanks.1983: Shanahan and KamilIn 1983 Shanahan and Kamil took a retrospectivelook at their earlier study in light of the findings ofBridge and Winograd (1982) and Rankin (1982) about theglobal nature and the high level of comprehension of the74doze procedure, respectively. Shanahan and Kamil had toagree with their opponents that students did use inter-sentential information in doze tests, but would not acceptthe criticism about their own lack of proper attention tointer—sentential cohesive clues (Bridge and Winograd, p.310). Through argument and research, however, Shanahan andKamil tried to show that the students’ use of globalinformation was only minimal (infrequent) and trivial(processed at a low level).First, Shanahan and Kamil pointed out Bridge andWinograd’s admission that students used inter-sententialinformation minimally to effect doze scores. Second, theyapplied Rankin’s ideas of literal recall to their previousscrambled doze experiment to test what was happening intersententially. Their experiment was meant to address Bridgeand Winograd’s call to make use of inter—sentential cluesand refute Rankin’s idea that “doze tests measure highorder comprehension processes, such as inferencing, to theexclusion of lower level processes, such as literal recall”(Shanahan et al. 1983, p. 123). Shanahan and Kamil focussedon the level of importance and quantity of ideas recalled inboth a) unscrambled intact passages versus unscrambled dozepassages and b) scrambled doze passages versus unscrambleddoze passages.Their new study showed that students were able tobetter recall ideas, more of them, and in order with thernunscrambled doze ‘ust as with the unscrambled intact75passage. This suggested to Shanahan and Kamil that studentsdo “integrate information across sentences when completingdoze exercises” (p. 127). However, because scrambled andunscrambled doze scores were similar rather than different,they concluded that inferencing can take place unrelated toor despite the doze procedure. Thus it can be said thatShanahan and Kamil had shown the inverse to what Rankin hadsaid about inferencing and literal recall as these high andlow order comprehension processes pertain to the dozeprocedure. Shanahan and Kamil concluded that inter—sentential inferencing took place unrelated to or despitethe doze procedure and consequently that doze was a lowerorder process.1984: Shanahan and KaiuilThe following year Shanahan and Kamil continued to tryto challenge their critics by doing an archival study onBormuth’s data of 1962. They re-analyzed the data todetermine “the number of comprehension test items requiringwithin- and across-sentence information” (p. 254). Theyconcluded that:(the doze procedure) predictsperformance best in those situations inwhich within—sentence comprehensiondominates. doze is less useful insituations in which students areexpected to integrate information acrosssentence boundaries. (p. 255)76Although Shanahan and Kamil showed the lower ordernature of the doze procedure in their 1983 and 1984research, they did not really answer the criticism of Bridgeand Winograd. Shanahan should have looked at the lexicaland other cohesive items to respond properly to thechallenge. Thus it was the goal of the present researchstudy to look at cohesive items as clues to filling thedoze blanks, especially those that occurred inter—sententially.For reasons of ease with color—coding, the focus was onsame—word re—iterative clues. The researcher included reiteration of both lexical and syntactic words. It should benoted that this was different from Bridge and Winograd whosuggested the opposing influences of a) conjunctive andreferential items and b) lexical items, the latter favoringhigher intra—sentential scores. The present researchdivided the lexical items into i) the collocative, such asthe “United States” where one word usually is located besideanother one, and ii) the same—word re—iterative—word items.It is obvious that collocative words are intra—sententialand it was found by careful analysis of the passages used inthe present research that most of the same—word re—iterativeitems were inter—sentential. Furthermore, it must be addedhere that it was beyond the scope of the present research toconsider comprehension directly. Therefore, Shanahan andKamil’s conclusion about the lower order of the doze couldnot be debated but left for future research. However, even77having such a debate rested on whether or not students couldget beyond the first step of learning to look beyond thesentence of the doze blank for clues to find the filler.That step itself depended on the inter-sentential cluesbeing available.1985: HenkHenk joined the inter/intra-sentential debate with abias toward the within-sentence side of the debate. Heagreed with Shanahan et al that:readers use local redundancy whenanswering doze items and tend toneglect inter—sentence cues and priorknowledge as sources of information.If, indeed, doze tests fail to tap themultifaceted nature of text processing,their usefulness as measures of readingcomprehension ability becomes seriouslylimited. (p. 213)Like Bridge and Winograd, Henk used think-aloud protocolswith the standard (traditional) doze and found thatsubjects used five types of passage cues:1) within-sentence cues only,2) beyond sentence but within text,3) combinations within and beyondsentence in text,4) prior knowledge/beyond text,5) combinations within and beyond text.(Henk p. 215)78However, he found that for the most part students usedintra-sentential cues. He did find the use of inter-sentential clues, but very infrequently and thus concludedthat doze procedure was not “a dynamic and sensitivemeasure of reading comprehension” (p. 217).Yet Henk did leave the door open for more research inthe present debate. Admitting that inter-sententialsearching for clues for some content word deletions isnecessary, he emphatically suggested:examining various contextual factorsabout deletions such as position insentence, linguistic function andsupporting syntactic and semanticcontext, cohesive importance andrelevance to other text elements. (p.217)In other words, Henk proposed the idea that perhaps themotivating force for students to cross sentence boundarieswas dependent on the nature of the passage, as well as onthe passage’s readers. This idea is important for thepresent research because it is another reason for analyzingthe passages for cohesive clues. Certainly it would be awaste of time and energy to color—code a passage hoping tolead the students to cohesive clues that did not exist.791985: BachmanBachman, like Henk, reviewed proponents of both sidesof the inter/intra-sentential debate, but rather than take astand on one side of the other, he tried to explain theinconsistencies in results as partly caused by deletingwords in the fixed ratio (every-fifth-word deletion) manner.Thus Bachinan proposed to compare this fixed-ratio(random) selection of words with a rational selection. Forthe rational selection he tried to develop criteriaaccording to Halliday and Hasan’s framework. Theseincluded:1) syntactic, which depended only onclause—level content,2) cohesive, which depended onintercausal (intra-sentential) andinter—sentential cohesive content, asdescribed by Halliday and Hasan(1976); and3) strategic, which depended on ‘long-range patterns of parallel patterns(coherence). (p. 63)However, despite no problems with the rational deletionprocedure, Bachman found the identification process wasimpractical for teachers and test writers, thuscounteracting the desirable ease of construction of thestandard doze.To solve this problem Bachman designed his researchusing the following two forms. For the first form hedeleted words rationally and classified them into thefollowing four context levels:801) within clause;2) across clause, within sentence;3) across sentences, within text; and4) extra textual...(where type 2 and 4deletions were maximized). (p. 539)For the second form he deleted every 11th word, thendescribed them in terms of the above levels. For both formsanswer keys were made which included exact words andacceptable alternatives.Baclunan’s results for his rational doze tests showedthat:a) test developers, using the above fourlevels, were appreciably consistentin identifying deletions,b) his criteria were practical forteachers,c) (scores) were comparable in both inreliability and concurrent validityto those on the fixed—ratio test (p.549),d) difficulty of closure increase(d) asthe amount of context required forclosure increased (p. 549),e) the development and use of an answerkey make scoring entirely objectiveand greatly increase scoringefficiency (p. 550),f) the use of a rational deletionprocedure allows the test developermuch greater flexibility in revisingspecific items on the basis of boththe content specification of the testand the item statistics. (p. 550)Although Bachman did not, in fact, solve theintra/inter-sentential debate, he set the stage for the nextattempt to determine whether students could be trained to81find clues beyond the sentence of the doze blanks. Onewould expect that next step to be a rational doze where thetext was deleted to maximize the number of inter—sententialclues. At first it was thought for the present researchpaper it would be better to investigate the color-codeddoze procedure with the traditional every-nth-word deletionfirst in order to set a baseline for further research with arational deletion pattern. However, due to the decidedlydifficult task of locating enough suitable ESL students, itwas decided after the pilot project to move right away to arational form of the color—coded doze where as many reiterative clues as possible were used for deletions. Thisdecision was made to take advantage of the suspectedadvantage that an increase of re—iterative clues couldbring.Bachman’s influence on the present research design wassignificant and resulted in two contributions. The firstcontribution was the idea of content levels. It resulted inan attempt to identify various locations of the same—wordand same—root re—iterative clues and their distance from theblank. The second contribution was the inclusion of ananswer key which included exact words and acceptablealternatives. It inspired a two-style answer key, thedifference being that the present research key was to useweighted scores for acceptable responses. (An explanation ofthe weighting formula is to be given in more detail inChapter Three).82Bachman’s contributions were significant and thepresent project was over ambitious in trying to incorporatemany of them. The modified weighted scoring scheme whichhad been proposed for the pilot project, was suspected ofbeing biased, tried and found to be too time-consuming, anddropped during the pilot project in favor of a simplifiedmethod which was discovered unintentionally during thescoring process. The revised scoring methodology will bedescribed in Chapter Four. Simplified along with andpartially because of the revised scoring methodology was thelocation of clues indicating what word should fill each ofthe doze blanks. The exact location of each clue to thedoze blank was noted in preparation of the pilot and mainprojects’ doze passages but not considered quantitativelyto any great degree. This kind of information could beconsidered in future projects, but for the present researchthe essential details were whether the words needed to fillthe doze blanks were located a) outside of the passage, b)within the sentence of the doze blank, or c) somewhere inthe passage beyond the sentence of the doze blank.As a result of the ongoing discrete—versus—globaldebate as participated in by Chavez and Oiler, Yamada,Thomas, Shanahan and Kamil, Leys, Bridge and Winograd, Henk,and Bachman, the stage was set for the present CCC project.The key question which emerged dealt with a) theavailability of clues for students to be able to find in83order to fill the doze blanks and b) how much the studentswere paying attention to the inter—sentential clues.What is the nature of non-native speakers of English?In the review of the literature so far, the focus hasbeen primarily on the nature of the doze itself and onlysomewhat on the nature of the students but enough toindicate the need for a closer look at that nature.Following, then, is what various researchers have said aboutstudents and how they interact with the doze procedure.Some of the problems with the doze have something todo with the students themselves and their orientation towardthe doze procedure. Looking at the research literature thefollowing problems have been identified:a) non-linguistic interference (Streiff),b) reluctance to guess (Brutten and Tuimuan),c) inattention to redundancy, and shorter memory span(Kalivoda),d) lack of knowledge of rules of grammar and redundancy(Aitken),e) and reading ability of poor students compared togood students (Goodman, Neville & Pugh, Propst,and Schwartz & Stanovich).Streiff, talking about children doing doze tests, saidthat extra—lingual concerns, such as uncertainty about thetask, lack of attention, momentary forgetfulness, not iustlack of competence in their English prevents a child from84filling some doze blanks. It is conceivable that under thestress of the doze test, some of the above problems couldapply to adults. Talking about non-native speakers, Brutten(1981) mentioned that some ESL students are reluctant to usecontext clues to guess because of their limited vocabularyand resultant insecurity. Tuirunan (1972) pointed out thatthe students in his study seemed to make only one attempt on75% of the doze blanks, despite the type of deletions.Kalivoda (1980) talked about students “disregardingredundancy” (p. 2) and that a student’s “memory span isshorter in a second language” (p. 3). Aitken (1977) arguedthat the amount of redundancy has little effect “if thereceiver (student) does not know the structure (of thelanguage)” (p. 65).Looking at the differences between good and poorreaders and drawing on the ideas of Goodman (1970) as wellas research of Neville and Pugh (1976— 1977), Propst andBaldauf (1981) pointed out that one difference between goodand poor readers is that of editing, i.e. looking back andforth through the passage as they read. Goodman (1970)argued that good readers use context clues optimally.Neville and Pugh (1976 - 1977) found that poor readers onlyused context clues occurring before the blank as opposed tobefore and after by good readers. Schwartz and Stanovich(1981) stressed that poor readers tried to find clues butgot entangled in the details of a passage. Yet, Bridge andWinograd found that poor readers were able to orally explain85cohesive relationships, especially for referential andlexical examples, though to a limited extent conjunctiveones. Meanwhile, Bridge and Winograd found that goodreaders were able to explain all the types of cohesiverelationships. Likewise, Henk (1985) in his students’verbal protocols did find the use of inter—sentential clues,but only to a very limited extent.To summarize the literature on the nature of studentsand doze procedure it appears that there is a contradictionin the research over the amount of use of redundantinformation. But looked at from a different angle it couldbe argued in the following way. Poor readers have the mostdifficulty, in that they do not try very hard to look forclues, especially inter—sentential ones, to help them fillin the blanks. Or, if they do, they get lost in the detailsof the text. Good readers apparently know more vocabularyand grammar rules and are able to extract the necessaryinformation to be able to see the whole picture moreclearly.What is the possible role of color in the improvement ofnon—natives’ doze reading scores?A solution to the poor readers’ problem of becomingbogged down in reading a passage seemed to be to help themto find the appropriate amount of use of the context. Thiswas Gattegno’s solution to teaching phonics, by grouping thesound—spellings into color—coded groups of similar sounds,86helping the students analyze (discriminate) and synthesize(categorize> or vice versa at the same time, depending onwhether they were reading a sound-grapheme chart or readinga story. What Gattegno did at the sound-grapheme level,the color—coded doze tries to do at the morpheme andsentence level with the parts of speech. The purpose ofcolor—coding is to help the students focus on particularpart of speech details (analysis), thus eliminating someextraneous information, while helping them to relate otherclues to the isolated detail (synthesis) to find theappropriate fillers.To explain further, color—coding should counteractstudents’ reluctance to get involved in a search for cluesby a) making the parts of speech more readily evident to thestudents and by b) making it easier to pay attention toredundancy. But color—coding should also make the searchmore focussed on relevant redundant (part—of—speech)information, reduce the memory load, waste less time andleave more time for more attempts to find the appropriateclues involved in the cohesion of the passage. In addition,color-coding should make the syntactic (grammatical)relationships of individual sentences easier to see and usein the search. Both poor and good readers should gain anadvantage from color-coding, although there could be aceiling effect for the really good readers. Furthermorethere is no good reason foreseen why good readers should get87confused by the increased information, nor why the poorreaders should either.Intrinsically, color is usually more attractive thanblack and white. Color is more motivating, too. However,this research experiment is not primarily interested incolor for its own sake. It is interested in color as ameans to guiding students to an awareness of the parts ofspeech in the text which in turn may point toward the wordsto fill the blanks. In other words, the present researchneeds more than motivation to help improve the students’scores on the doze tests. To determine the effect of coloralone as compared to the effect of color plus the teachingpoint, the researcher provided a control where the passageswere randomly colored rather than according to thesystematic form of the treatment passages. It was proposedat the beginning of the present research project that if thecolored—treatment forms both result in higher mean scoresthan the standard-doze control form, then it could be saidthat color is a motivating factor. Furthermore, if the meanscores for the color—coding are higher than for therandomly-colored forms, this would suggest that the color-coding has a positive effect. This is to say that thestudents are making use of the intended assistance of thecolor—coded parts of speech. Furthermore, if the meanscores of the randomly—colored forms are lower than those ofthe standard—doze control group and even lower than thoseof the color-coded treatments, it could be said that the88random colors are motivating the students to use them butthis very use of wrong information is having a negativeeffect. Such a situation would indicate the positive valueof color—coding the parts of speech.What are the contributions of the key areas ofinvestigations to the present research?From Carroll, through Halliday and Hasan, Chihara andOiler, Shanahan and Kamil, Leys, Yaluada, Thomas, Bridge andWinograd, Henk, to Bachman the debate over theglobal/discrete debate raged. Yet no extreme positionresulted nor was a satisfactory solution forthcoming butsuggestions were made to get closer to the answer. Streiff,Brutten, Tuinman, Kalivoda, kitken, Goodman, Propst, Nevilleand Pugh, and Schwartz and Stanovich had all put forth ideasabout the nature of students and their reactions to dozeprocedure. The present research had its roots in theglobal/discrete debate and experiments and the presentresearch methodology evolved from these efforts, selectingand recombining various strategies of the above researchers,leaving some of the other strategies for future experiments.In Chapter Three the bits and pieces of the methodologicalstrategy accompanying the contributions of the researchersabove will be combined into a description of the researchdesign.89RESULTS OF THE REVIEW OF THE LITERATUREIntroductionThe research design in this paper came largely as aresult of a review of the literature on doze procedure.This design suggested and tested a method of helpingstudents to use cohesive cues that were located beyond theimmediate sentence of the doze blank.Overview of the method proposed in relation to theliteratureThe goals of the present doze researchThe original goals were the following but because ofnegative results in the pilot project and because there wereso few students to draw on the third goal below was omittedin the main project. The goals were:1) to train students using the color—coded doze methodto be aware of inter—sentential as well as intrasentential clues toward the point where they no longerneed color-coding to help them in the use of part ofspeech clues,2) to compare the major treatment group which hadpractised with several color—coded doze passages to a90control group which had practised the same number ofuncolored doze passages to determine if color—codinghelped students to look beyond the sentence of theblank,3) to compare the major treatment group to a controlgroup which used passages with only the doze blankscolored, to determine whether the students were usingthe color—coding of the blanks alone to find clues orwhether they were using the beyond—sentence color—coded words as clues,4) to compare the major treatment group to a controlgroup which had practised in the same way except thatthe words in the passages were colored randomly. Thiswas to determine whether or not students were justbeing motivated by the colored words to try harder inthe test rather than being motivated to take advantageof the instructional information provided by thecolor—coding.The use of previous researchMany of the experiments found in the review of theliterature used the standard doze and then made amodification to it which was then tested. For the pilotpart of the present experiment the standard doze was usedbut replaced by a rationalized doze in the main part. Both91the pilot and the main projects then modified the passagesin no other way except by coloring the parts of speech. Thepilot project used three modifications and the main projectonly two, the main project dropping the use of number 2below. The control and the modifications were as follows:1) In order to determine a base line to test for anyimprovement in scores as a result of using the all-color—coded doze, the standard doze form, withoutany color—coding, was used.2) In order to determine if color—coded blanks alone(not together with the color—coded words) wassufficient for students to find inter- as well asintra-sentential clues, only the blanks were color-coded.3) In order to determine if color and not the color—coding was effecting the scores of the students, allthe words and blanks were colored randomly to makesure the colors had no connection to grammaticalmeaning.4) In order to help students find clues beyond thesentence level, all the words and blanks of thestandard doze were color—coded.£492Implications of Review of Literature for the PresentResearch PaperIn summarizing the research on doze it can be seenthat there were two goals of importance to this researchpaper. The first goal was to improve doze scores; thesecond to see if students could extend their use of contextclues beyond the sentence to help determine blank fillers.The more recent studies seemed to look down on the firstgoal as dust improving lower reading comprehension grammarskills. It was expressed that, although the doze has someeducational value, it does not have as much as otherprocedures to help students in discourse comprehensioninferencing, i.e. higher level reading comprehension skills.However, given what Goodman and Tuinman said about readersusing the least amount of effort to determine doze blankfillers, it appeared to me that clues at the sentence level,such as blank length and initial letters of deleted wordscould short circuit the students’ need to look beyond thesentence for clues. The same could be true for the use ofcolor-coded blanks. If indeed this short circuiting didtake place, then it could be argued that the weakness in thedoze procedure was due to the short circuiting itselfrather than to the students’ inability to look beyond thesentence level. When there is no need to look for inter—93sentential clues because intra—sentential clues are readilyavailable, why should anyone waste time searching for inter—sentential ones? Sometimes, as was shown in an analysis ofthe passages for this research project, many clues as to theappropriate doze blank fillers, are to be found somewherein the text beyond the sentence of the doze blank but notwithin the boundaries of the same sentence of the blank.This appeared to be a good environment to test to determinewhether or not students would be able to locate the inter-sentential clues, especially the same—word or same—root reiterative ones.It was the goal of this research project to discoverwhether or not the procedure of color—coding all the partsof speech and all the doze blanks in a passage couldprovide the necessary stimulus and pathway for students tomake the effort to look beyond the sentence of the dozeblanks to find the appropriate blank fillers. I thoughtthat such an effort could lead to higher doze scores anddecrease the strength of the researchers opposed to thetraditional doze procedure.In the present research the goal was to improve thestudents’ doze scores using the color—coded dozeprocedure. The intention of the present paper was todetermine if color—coding, in fact, can help students toincrease their use of inter—sentential clues using thecolor—coded standard doze. Note that same—word and same—root re-iterative type clues were identified in the analysis94of the research passages, but other cohesive clues may haveassisted the students.The review of the doze literature suggested stronglythat it is important to know the nature of the doze passageand how much effect the passage has on the doze scores. Ifreading by ESL students tends to be too localized thequestion to be answered is whether increasing the number ofinter—sentential (re—iterative) clues can increase thescores, Obviously, if there are more local clues thandistant clues, then there is a strong bias toward usingintra—sentential clues. Therefore, the effect of the lattermust be neutralized. In a rational doze, words can bedeleted to increase the number of distant clues and decreasethe number of local clues. However, in a standard doze, itappears to be the proportion of local to distant clues is amatter of chance. To circumvent this problem, dozepassages can be and, in fact, were analyzed for location andproportion of within sentence clues to beyond sentence cluesand the most favorable passages were retained. For thepilot project only exact answers were accepted. Instead ofusing the complex weighted scoring as suggested earlier inthis paper, in addition to the scores out of 50 blanks, reiterative inter-sentential blank types were tallied. Forthe major project, these two types plus the intra— andinter—sentential type were used. These procedures will beexplained in Chapter Three when the research design isdescribed and the scores given in Chapter Four.95Research Question 1From questions arising out of the literature review,the overall question for this research paper would besubdivided as follows.1) Does color—coded doze lead students to look beyondthe immediate sentence for clues to be able to fillin the doze blanks? In other words, are meanscores higher on passages which are completelycolor—coded than on passages which have less color,random color, and no color at all?2) Are the students’ mean scores on completely—color-coded passages lower than those mean scores ofstudents who do the standard doze passages?Hypothesis 1Out of these questions the following hypothesis forthis paper was put forward:lh) On the practice tests there will be no significantdifference between the treatment group which hastrained using the color-code doze and the controlgroups which have trained on either the non—coloredcolored-doze, the colored-blanks only, or therandomly-colored doze practice tests.96Research Question 23) Can students transfer any acquired skill learnedfrom their practice with color—coding as it pertainsto using inter—sentential and intra—sententialclues, to the solving of uncolored rational dozepassages?Hypothesis 22h) On the post-test there will be no significantdifference between the treatment group which hastrained using the color—coded doze and the controlgroups which have trained on either the non—coloredcolored—doze, the colored—blanks only, or therandomly-colored doze practice tests.Research Question 33) Does the color—coded doze give students moreconfidence in filling doze blanks than the otherforms of the doze?Hypothesis 33h) On the post-test there will be no significantdifference on confidence scores between thetreatment group which has trained using the color—code doze and the control groups.Research Question 43) Does the color—coded doze lead to more productiveconfidence in filling doze blanks than the otherforms of the doze?97- Hypothesis 33h) on the post-test there will be no significantdifference in productive confidence scores betweenthe treatment group which has trained using thecolor—code doze and the control groups.Footnote 1: Same—word re—iterative clues were chosen foranalysis and subsequent choice of the passages because thoseclues were the most obvious. They fit into the color—codeddoze procedure because they are the same part of speech asthe corresponding blank. According to Bridge and Winograd,re-iterative clues fall into the lexical category and thecategory of clue which they said lead to intra-sententialsearches rather than beyond. However, in the analysis ofthe passages for the present research, by far most of thetime the same—word re—iterative clues were either before orsometimes even after the sentence of the blank rather thanbeing within. All other factors being equal, one wouldexpect the students, given the color—coding, to look beyondthe sentence.Footnote 2 It may be that the color-coded doze helpsimprove conjunctive relationships in as much as the color—coding makes them stand out. For a definition ofconjunctive and other types of cohesion, please see Table 3below. It is an adaptation and extension of ideas presentedby Halliday and Hasan (1976) and quoted by Bridge andWinograd (1982).98TABLE 3VARIOUS KINDS OF COHESIONo COHESION° TYPES KINDS SUBKINDS/ ° EXAMPLESo 0 DEFINITIONS 0o o 0 0 0°References °persona]. °pronouns, °I, you 0adj°My, your::demonstrative0adjectives :tI5, that:0•comparative °adjectives °bigger 0o°adverbs °fastero o 0 0 0°Lexical ore_iteration °same words °car/car0°synonyius °house/hoiue°o e esuperordinates0pear/fiteo°general words °home/: :that refer to: dwellinge0°collocation °words that eUnitedtogether° Statesoconjunctive °additive °logical °and••adversative 0 relationships °yet:causal :relationships :°cEllipsis 0 °word in paral—°He has a0 ° ° ].el structure° house,0 0 0 which is left° car, and° ° ° out to pre- ° boat.o 0 ° vent unwanted° 00 0 0 repetition 0 0Adapted from information cited by Bridge and Winograd(1982) and belonging to Halliday and Hasan (1976).99CHAPTER THREERESEARCH DESIGN AND METHODOLOGYINTRODUCTIONBased on a review of the literature concerning thedoze procedure and the use of color—coding, the presentresearch project involved training adult upper—intermediateand university level English as a Second Language (ESL)students. The research design was a blending of customaryand unique ways of training and testing with the dozeprocedure. The purpose of the design was to discover ifstudents could increase their purview of the text and makeuse of inter—sentential clues available to them in order togive them a more precise understanding of a whole passage(elsewhere referred to as a piece of discourse).The first innovation was the combination of thetraditional doze procedure and the coloring, and moreimportantly, the color-coding of the parts of speech in apiece of discourse of approximately 300 words in the pilotstudy and 330 in the main project. The purpose was todetermine whether or not systematically highlighting theparts of speech with color could lead to the greater use ofgrammatical and semantical clues across sentence boundariesto locate the correct fillers for the doze blanks. In thepresent research the principal focus was on the same—wordand same-root re-iterative clues, as they are explicit in100the text and thus are the most objectively assessed. Anyimprovement by the students would have been shown by theirhigher doze scores on the post—treatment doze test ascompared to their pre—treatment (same) doze test and ascompared to the scores of control groups who had practisedwith a randomly colored doze or a non—colored traditional(n the pilot project) or non-colored rational doze (in themain project).Being used along with the above training methods weretwo scoring methods. Both were used after the answers hadbeen inserted into a spreadsheet. The first method was thetraditional exact—word method where only the replacement ofthe original word received credit. This method was used forboth the pilot and the main projects. In this method all 50blanks were considered and scored. The second method wasintended to be a more sensitive measure and give more creditto the students answers where at least partial credit wasdue. But the second method was intended to do more than thetraditional acceptable—word scoring method in that it was toindicate also the kind of clues the students had used tofind the doze blank fillers. Of primary importance werethe inter—sentential re-iterative—word clues. For the pilotproject this concern was manifested by the weighted—scoremethod which looked good at first glance but was far tootime consuming and subjective. In scoring the practice testfor the pilot project the researcher discovered analternative way to focus on inter—sentential clues. It was101much more efficient to label each answer on the spreadsheetas being either of the external type, the inter-sententialtype, or the intra—sentential type, to assign each cell onthe spreadsheet a “1” for a right answer, a half point foran acceptable answer, or a “0” if the answer was unrelatedto the deleted word and then to delete the columnscontaining external type blanks and intra-sentential typeblanks. Table 16 at the end of Chapter Three illustratesthis scoring procedure.This revised method, an innovation in doze research,was used for the main project. It took advantage of theease of deleting columns on the spreadsheet, i.e. columnscontaining external word fillers, and columns containingintra—sentential word fillers. As each blan] had a columnand a cell for each student’s answer with .a score already 1given by the first method, it was easy to delete theunwanted columns and automatically get the total score foreach of the columns and rows plus the overall totals for therows and columns. The advantage of this method over theweighted—scores method was that with this method it waseasier to give exact—word points, delete columns, and makegraphs to show the scores given to inter—sentential blanktypes, all in an unbiased, straightforward way. Finetuning, that is to say giving points to acceptable-wordanswers, was done later. There was still the danger ofbiased scoring but the scoring was more transparent than theweighted scores could be. The only regret was that in102scoring no specific attention was paid to whether thecorrect acceptable—word answers violated inter-sententialrestraints or not. This awareness of possible beyond—sentence constraints and their violation can be built intofuture projects. Because the key expected answers and thestudents’ answers are on the spreadsheet the scorers will beable to check for violation of the sentence constraints andassign partial points if desired.THE SUBJECTSThe subjects (mostly female) were students from fourclasses of male and female adult (ESL) students. Thestudents represented a variety of nationalities (primarilyfrom Pacific Rim countries, some from South America). Thestudents in the pilot project class were enrolled atColumbia College in Burnaby, British Columbia. Their upper-intermediate program consisted of classroom work covering 14weeks, five hours/day, five days/week. The students in themain project, all from one university in Japan, were fromthree classes in the Ritsumeikan University English Programat the University of British Columbia in Vancouver, B.C.These students, between the ages of 19 and 22 years old,were studying in an ESL program based on their fields ofinterest, often business and culture. All the students haddone the TOEFL test and were short of the 550 which was thenormal standard for foreign students studying at UBC. The103class profiles given in Table 4 below show how the studentswere divided for their UBC studies. For the presentresearch the classes were each assigned one of three dozeformats. Class one was given the standard doze, class twothe randomly—colored doze, arid class three the color—codeddoze. Note that the assessments done by the Ritsumeikanstaff in order to assign the students to their appropriateproficiency levels showed that Class 1 was the mostproficient, Class 3 less proficient, and Class 2 the leastproficient.TABLE 4CLASSIFICATION OF STUDENTS BY TOEFL AND THE LANGUAGEPROFICIENCY INDEX AS DETERMINED BY THE RITSUMEIKAN STAFF FORPLACEMENT INTO THE RITSUMEIKAN PROGRAMo TOEFL • LANG. PROFICIENCY INDEX0 Range .. Avg. ° Range....... Avg.Class 1 53 (500 — 553) ° 527.80° .1 (1 9 — 2) ° 1 94°Class 3 34 (483— 517) 505 750 1 0 (1 0— 2) 1 80°Class 2 40 (483 — 523) ° 494.80° .5 ( .5 — 1) ° .86THE TREATMENT AND CONTROL GROUPSIn the pilot project 18 students from the ColumbiaCollege ESL class were divided into basically four groups offour students. The selection of students for each group wasbased on the pre-test scores such that the mean of eachgroup was the same on the pre—test. (Please refer to Figure1 in Chapter 4.)104It should be noted here that the process of dividingthe students into groups was changed from the intended plandue to the choice of some students not to participate in thestudy and due to the fact that the pre-test proved to bemore difficult for the students than had been anticipated.It had been intended to divide the 20 students in the pilotclass to be divided into four groups of five and to makesure there would be, as determined by the researcher rankingscores on the pre—test doze, a strong member, a fairlystrong member, an average member, somewhat weak member, anda weak member. The students were to have fallen basicallyinto three groups. The strong group was to be defined asgetting a score of at the independent reading level (43% andabove). The average group was to be defined as getting ascore at the instructional reading level (38% - 43%). Theweak group was to be defined as those students getting ascore at the frustration reading level (below 38 %). Infact, all the students in the pilot project gotfrustrational level scores on the pre—test.As a result of the pilot project, and keeping in mindOiler’s finding about the proficiency levels of non-nativespeakers, more proficient ESL students were looked for. TheUniversity of British Columbia provided three classes of 20Japanese students each. Based on the available numbers ofclasses plus the results of the pilot project (to be givenin Chapter Four) a new research design was chosen in whichonly three types of doze tests (two controls and one105treatment) were given. Each class was randomly given one ofthe three tests. In fact, at the time the test forms wereassigned, the researcher had not been given the students’group level profiles from the Ritsuiueikan project leader.As it turned out, the class with the highest rating asassessed by the Ritsuineikan teachers became the controlgroup, the second highest became the color—coded dozetreatment group, and the weakest group became the random-color control group. The matching of classes to test typeswas somewhat intuitive and perhaps a touch biased, butindeed it was a fortunate choice because all but six of thestudents in the weakest group dropped out of the mainproject after the pre—test. After the first practice testeven those six decided not to continue. If the weakestclass had been assigned the rational CCC treatment anddropped out, that would have devastated the researchproject. Fortunately the color—coded treatment group andthe main control group stayed in to test the questionrelated to the effectiveness of the color—coded rationaldoze procedure in helping students to look beyond thesentence of the doze blanks for color—coded words to fillthe blanks.106THE TREATMENT AND CONTROL PASSAGE FORMSThe nature of the formsThe purpose of the present research was to determinewhether or not color-coding parts of speech could help adultESL students learn to use inter-sentential clues that appearin the test passage. Therefore, in the pilot project thetreatment group was given Form 4 below and each controlgroup was given one of the three remaining forms of dozeprocedure:a) (Form 4) color—coded standard doze passages inwhich all the words and blanks were coded accordingto the following non-random procedure. Nouns andpronouns were blue; verbs: red; adjectives: green;adverbs: purple: and conjunctions, prepositions, andinterj ectioris: black.b) (Form 3) standard doze + randomly colored wordsand blanks.c) (Form 2) standard doze + color—coded blanks only,d) (Form 1) standard dozeNote that standard doze means the deletion of every fifthword, regardless of type of word, with standard lengthblanks. The first and last sentences of the approximately300 word passage are left intact. The standard ortraditional doze was the reference point for the pilotproject in the present experiment and thus was used for thecontrol group and for the basic design of the color—codeddoze and the other controlling variations in treatment.Each of these forms of doze is described in Table 5 below.107TABLE 5CONFIGURATIONS OF THE STANDARD CLOZE PROCEDURE ANDVARIATIONS (PILOT PROJECT)°TYPE OF CLOZE ° DELETION 0 BLANKS • WORDS 0 INTACT0 0 0 • 0 00 0 nth = 5 ° 50 300+/—6 1st ande 0 0 last0 0 0 0°sentence 0°Forml) 0 0 0 0 0°STANDARD doze ° Yes ° Yes ° Yes 0 Yes0 0 0 0 0 0°Form2) 0 0 0 0°Standard doze +0 0 o • 0°ALL BLANKS 0 Yes 0 Yes ° Yes • Yes 0°COLOR-CODED o 0 00 0 0 0 0 0°Foriu3) 0 o e 0°Standard doze +0 0 0 0 0°all blanks/words° 0 0 0 0°RANDOMLY ° Yes 0 Yes ° Yes ° Yes°colored 0 0 00 0 0 0 0 0°Form4) 0 0 0Standard doze +0 0 e 0°all blanks/words0 0 • e 0°COLOR-CODED Yes 0 Yes 0 Yes •0 0 0 0 0 0Note: In the main project form 2 was not used and the otherthree forms used a rational doze in which as many reiterative words were deleted as possible.108The method of coloring the passagesThe process of preparing the doze passages for colorprinting on an a printer compatible with the IBM personalcomputer will be given in the Appendices. Please refer toAppendix B.THE STORIES USED FOR THE CLOZE PASSAGESIntroduction to the selection of the passagesOriginally it had been intended to use the samepassages throughout the CCC research project. For the mainproject, however, due to circumstances beyond the control ofthe researcher, a new set of passages had to be chosen froma text provided by the institution involved.The doze passages for the pilot project were takenfrom a recent ESL text called CANread by Patricia Raymond(1987). It contains ten stories about Canada. After adatabase analysis of parts of speech plus the frequency andlocation of deleted words, seven stories were chosen. Basedon the number of same—word and same-root re—iterative cluesavailable, on the Flesch readability formula (1948), plus onthe Fry readability graph (1977), the stories were rankedand ordered according to their increasing difficulty. Thepre—test/post—test selection, “Terry Fox”, appeared to be109ranked in the middle of the difficulty range. See Tables 6and 7 for more details about this range.Used for the main project was a university introductorysociology text called Canadian Society, A Macro Analysis byHarry H. Hiller (1991). Several passages were chosen,usually one per chapter, usually from the introductorysection where more re—iterative words were available.Passages with the most number of sets of re—iterative wordswere selected and written down. Then a square was drawnaround all repeated words and one of each multiple markedfor possible deletion. Words to be deleted werehighlighted, keeping in mind that deleted words should notbe contiguous as two blanks in a row would be extremelydifficult for the students. Deletions were also identifiedin such a way that the remaining multiple would sometimes bebefore and sometimes after in order to determine if theposition of the clues would have an influence on theretrieval of the deleted word. To help in theidentification, all the words to be deleted were marked asto their location relative to their multiples elsewhere inthe passage. i.e. before or after, and within or beyond thesentence of the deletion. All these words were given asymbol to indicate where they were located in relation tothe undeleted multiples which were within/beyond orbefore/after the sentence of the deleted words. As nopassage had 50 sets of re-iterative words, other words fordeletion had to be selected. The researcher tried to choose110the easiest ones for the students to think of, while at thesame time making sure that there was a reasonable distancebetween deletions.The rationale for the type of text analysis performed on thepilot projectThe text analysis was done:a) to identify the parts of speech being deleted by thetraditional every nth word procedure; andb) to determine the proportion of the total number of50 deletions each speech part category involved.The purpose of this text analysis was primarily for futurereference in understanding the test results and inexplaining them in light of the previous research. In termsof more immediate goals, the process of determining theexistence and location of same—word and same—root reiterative clues was used to:C) select the passages with the greatest number ofsame—word and same—root re—iterative clues, thuseliminating the supposedly more difficult passages;andd) weight the scores according to the availability andlocation beyond or within the sentence of the same—word and same—root re—iterative clues.111The presentation of the text analysis for the pilot projectTable 6 and Table 7 below present a partial analysisof the stories. Appendix A lists all the words of eachstory passage, indicates which words have been deleted andgives the location of the same—word and same—root reiterative clues (within or beyond the sentence of thedeletion). Appendix A groups the deletions into part ofspeech categories, shows the number of deletions in eachcategory and gives the percentage of the total 50 deletiontotal per story. Appendix A also shows a suimiaarizeddistribution of the same-word and same-root re—iterativeclues, as well as indicating the possible values for answersbased on the location of the these re-iterative clues.TABLE 6NUMBER OF BLANKS/50 DELETIONS WHOSE FILLERS DO NOT APPEAR ATLEAST ONCE SOMEWHERE IN THE PASSAGE AND THE NUMBER OF WORDSIN THE PASSAGE. (STORIES ARE IN THE ORDER AS THEY APPEAR INCANREAD.)°BLANKS/50°WORDS IN°o°PASSAGE°STORY TITLE°NUMBERCOMMENTSC°Story 10 The Donnellys° 18/50 ° 303 (pre/post—test)°°Story 20 Louis Riel 18/50 ° 300 (no change)°Story 3° Sasauatch 24/50 ° 295 ° (2 changes)*°Story 40 Bethune ° 19/50 ° 305 ° (no changes)*°Story 50 Japanese 0 23/50 303 0 (no changes)°Story 6° Terry Fox 0 21/50 ° 297 (2 changes)*°Story 70 Insulin 24/50 0 300 0 (2 changes)**changes from original text made to simplify text112_______7—8midthe relative difficulty and thecompared with each other)The rationale for the type of text analysis performed on themain project passagesThe text analysis was done to:a) select the passages with the greatest number ofsame—word and same—root re—iterative clues, thuseliminating the supposedly more difficult passages;and,b) give as many intra- and inter-sentential reiterative clues as possible for the students tolocate with or without the benefit of color-coding.TABLE 7AN ANALYSIS AND ORDERING OF THE CLOZE PASSAGES BASED ONA) THE NUMBER OF NON-RE-ITERATIVE CLUES FOR THE BLANKFILLERS AND B) READABILITY FORMULAS.0 0 READABLILITY°STORY° TITLE ç BLANKS/50 FRY FLESCH0 0 0 (non-re-iterative) 0 Grade Grade01: °Terrv Fox 21/50 0 8 7 — 80 0 0 mid 0 mid mid2: °The Donnellys ° 18/50 0 6 — 7 60 0 0 low 0 low low 003: °Louis Riel 18/5 0 7 7 — 8 00 0 0 low ° low mid04: Bethune 19/50 0 8 7— 80 0 0 low 0 mid mid 0°5: °Sasquatch 14/50 0 7 60 0 0 high ° low low 006: °Japanese 0 23/50 0 8 7— 8 00 0 0 high 0 mid mid07: Olnsulin 0 24/50 0 9 7 — 8 00 0 high 0 high mid 0°8. °Terrv Fox 0 21/50 0 8 00 0 0 mid 0 mid 0(low, mid and high =passages, when113The presentation of the text analysis for the main projectTables B and 9 below are a sunmtary of the data gatheredabout the deletions in the passages from the text calledCanadian Society, A Macro Analysis.TABLE 8NUMBER OF BLANKS/50 DELETIONS WHOSE FILLERS DO NOT APPEARAT LEAST ONCE SOMEWHERE IN THE PASSAGE; AND THE NUMBEROF WORDS IN THE PASSAGE.‘STORY ‘ TITLE °BLANKS/50’ WORDS IN’ COMMENTS‘NUMBER * 0° PASSAGE 0‘Story 10 Perspectives’ 10/50 • 337 ‘2 words repeated’0 e o‘ bymistake‘Story 2’ Society ‘ 20/50 ° 333 °‘Story 30 Ethnicity ‘ 13/50 ° 328 ° 0°Story 40 Uniqueness 0 15/50 ° 335 ° 0°Story 50 Polities ‘ 13/50 ‘ 349 ‘ 0‘Story 6’ Identity ‘ 12/50 • 329 ‘‘Story 7’ Perspectives’ 10/50 337 ‘o00000000000000000rnocncnornozrncnocnocnrnZcnci-ci-ci-rtci-ci-cti-3(Ici-ci-ci-ci-ci-ci-C1-3o0000000000000010IIIiIiW1t.ititi‘1II‘1tiWJ::..1<‘<‘<N’•<1<1<<1<<<-O01UMI—’.JO01wHZ•0000000000000000011rIIN10rtJI’Il-iIIIcINIU)IJj0iPI10I::Ifrt10100IPi10I::Kt1010NCl)IIIi0I’’II’IIP110IF-’II-’-1Z’10P1Cl)HjiItIH1t1iIi-’-iii-’10)H-Cl)r1(tIctii’0‘i-i-ic’-iii-’liI’110t0I’—’•I’-’-1010ci-0101!;!-10IIr10NIt‘aIct0)1IZ1<rn10in-ictNtI”II’•0ii-’-in<Ii-’u)1<0100lIT)100)10)0Ill)10)o00000000ITj00000000OHNZNI-’F-’F-’HF-’)I-’OHzzZI301)U01U00Z1-3H‘Wi-3H‘Me.-l3WVIViVi01010101(1)ZINt-’0000000‘.UUUUU.b1-301HItlltO0-J01-00F-oN•%%0000000O000000UI0101UI01Ui01lJHOWI-IN WUt3iUL)UUUUUUUCfl=4111I-.J.D‘.0010)U‘Cl)0NCflI-NH Zo00000000Cl)0o•o00000N’0(1) (1)N00UMU.Jts3MUWI--’‘.o‘.-J,I—’ioe(1)I—’HHHHF-’H01010101010101N011.3UUi(3101‘WO0000004Z(1)1-3Nt-It’1ZI3‘-3o000ooaoaaa000115THE ADMINISTRATION OF THE CLOZE PASSAGES FOR FIVE CLASSESOverview of the distribution of the test and practice dozepassages in the pilot projectIn the pilot project each group of students was giventhe same standard doze passage as the pre—test and post—test. The passage was not modified in any way. A series ofsix more passages were given to each of the classes, onemore the first week, two the second week, and two the thirdweek, with the sixth and post-test coming during the fourthweek. In each class each group of four (sometimes five)students had a different variation of the passage. Thedistributions of doze tests were as follows in Table 11.a°(FORM--0 (FORM° (FORM° (FORM0TABLE 11DISTRIBUTION OF THE TREATMENT, VARIATIONS AND CONTROLCLOZE TESTS, WITH VARIATIONS APPLIED TO STANDARD(EVERY NTH WORD) CLOZE PROCEDURE FORMATPilot Class4) Color—coded °all blanks and words color-codedE) random °blanks and words colored randry2) blank colored °only blanks color-coded1) standard °no color givenNote: each cell had four students.00116Length of time required for pilot projectEach contact session with a class was a maximum of 50minutes. That allowed 30 minutes for the students to fillin the doze passage and 20 minutes to be divided up betweeninitial instructions and feedback for the previous session’sdoze passage. Given a total of eight sessions over a fourweek period that meant 6.6 hours of class time per class.In other words, a total of 33 hours spread over five classeswas required of the ESL institutions’ teaching time, 6.6hours at Columbia College. In fact, from the second sessiononward the students took less and less time. As a resultless time was given to the students in the main project todo the doze passages; (20 minutes for the test and fiveminutes for looking at the answer keys)Overview of the distribution of the test and practice dozepassages in the main projectIn the main project each group of students was giventhe same standard doze passage as the pre—test and post—test. The passage was not modified in any way. A series offive more passages were given to each of the classes, twothe second week, and two the third week, with the fifth andpost—test being administered during the fourth week. Eachof the three classes in the main project was made up of apotential of 20 students. Table 20 shows the number ofstudents in attendance for each session. Each class had a117different color variation of the passage. The distributionsof doze tests were as follows in Table 12.TABLE 12DISTRIBUTION OF THE TREATMENT, VARIATIONS AND CONTROL CLOZETESTS, WITH VARIATIONS APPLIED TO RATIONAL CLOZE PROCEDUREFORMAT IN WHICH DELETIONS WERE CHOSEN TO MAXIMIZE NUMBER OFINTRA- AND INTER-SENTENTIAL CLUESClass 1 Class 20Class30 0 0) Color—coded°(FORM 4o (FORM 3°(FORN 1) no colorNote: Each class had a maximum attendance of 20 students,but the scores for only 13 students each from class 2. andclass 3 were used for statistical purposes. Only the above26 students completed the pre-test, post-test, and most ofthe practice tests. The majority of class 2 dropped afterthe pre-test and the rest after the first practice test.) ° random color0-— 0 0Length of time required for the main projectFor the pre-test 25 minutes were given. For theremaining six tests 20 minutes were given for completingeach of the them and five minutes for reviewing the answerkey of the previous test. The time used for presenting theinstructions for the pre-test was much more than had beenplanned for: approximately 45 minutes instead of 15.However, subsequent sessions were on schedule.118The roles of the researcher, teachers and the studentsIn order to administer the doze practice and testpassages it was necessary for all participants to understandtheir roles correctly. Therefore, in this study the rolesof the researcher, the teacher, and the students aredescribed in detail. As it was necessary for the teachersto understand the roles of everyone and the students to knowtheir own roles, the necessary instructions for carrying outthe experiment were included in handout fori and were givento the appropriate participants. The originals of thesehandouts are included in the appendices of this thesis. Therole of the researcher is given in Table C, the role of theteacher in Table D, and the role of the students in Table Eand Table F. Table E contains general and specificinstructions for all the groups. The specific instructionswere read out by the teacher as the students read along intheir test folders. Table F adds special instructions forthe groups using color—coded passages. The specialinstructions were included in the folders of the studentsdoing the color-coded passages. In the pilot project thespecial instructions were on the inside left page of thefolder. For the main project, thanks to a suggestion byVera Wojna of the Language Institute of the University ofBritish Columbia, the special instructions were placed atthe top of the first page of the two page doze passagesheets. This adjustment was made because it was believedthat students would not have to look so far to find out the119meanings of color—coded words and blanks. For both thepilot and main projects the special instructions were notread out loud by the supervising researcher or teachers.The groups doing tests with randomly colored words or nocoloring were not provided with any special instructions.In order to give an idea of the proceedings, theteacher’s role and part of the researcher’s role aredescribed at this time. In the pilot project the teacherwas not required to take part, as the researcher carried outthe teacher’s role in administering of the tests. In themain project the teachers were in charge of their ownclasses.In general, each day followed the same routine, withsome exceptions. On the first day, the teacher handed outthe doze folders and pencils. In order to ensure thestudents’ anonymity while providing the researcher withinformation to allow the tracking of the students’ resultsfrom test to test, the students wrote a self—chosen secretcode on the paper on the outside of the folder. Then theywere asked to open the folder to the instructions on theinside left page of the folder. Next, the teacher read theinstructions with the students. These instructionscontained Ashby—Davis’ suggestions for doing doze passages.When finished this, the teacher told the students to begin.While the students were doing the doze, the teacher watchedto see there was no talking or other forms of compromisingthe test. Approximately every ten minutes the teacher120indicated on the board how much time was left. After thespecified time was up, the students were told to put theirpencils down and close their folders and wait while theteacher collected the folders and pencils. Then the teachercontinued with the regular classwork if there was time. Inthe pilot and main projects there was no time left beforethe end of the class.For the pilot project, during the same day theresearcher collected the folders and pencils, then markedthe tests. The pre-test scores were used to group thestudents by assigning the scores so that the sum totals ofeach group were equivalent. Then, each student’s folder wasgiven a number to identify the student in the class, toindicate the group, the form of the test, the rank of thestudent within the group, and the student’s score on thepre—test. These sets of numbers were typed onto anAppleworks’ spreadsheet and the scores recorded. Then thepassage two tests were stapled into a duplicate set ofidentified folders.On day two through the last day the procedure was thesame. The only exception was with regards to the answerkeys. On day three onward after the folders and pencilswere collected the teacher handed out the undeleted text ofthe previous day’s passage for the students to read andreview individually. After about five minutes the teachercollected these passages.121Day eight was the last session of the pilot project.Due to time constraints, session seven was the last one forthe main project. At the end the researcher or teacherthanked the students for their cooperation and a small giftwas given to each of the classes.The routine for the main project was similar except fortwo procedures. First, it was not necessary to enter thetest data on the spreadsheet and analyse it right away.This was because the assignment of test forms did not dependon the pre—test scores. This, in turn, was because theassignment had been quasi—randomly pre—determined alongclass lines. Second, the students in the main project weregiven the answer keys to the day’s test right after the testsession instead of waiting until after the following testday.THE SCORING OF THE CLOZE TESTS AND DATA ANALYSISSpreadsheets were used to collect the students’responses for each test. “Appleworks” and an Apple liccomputer were sufficient for the pilot project but morememory was needed for the major project. Thus, “Quattro”was used on an IBM 386 computer for the latter project.Using the Appleworks’ spreadsheet program the researchertyped in all the students’ answers into a formatted table(see Table 5 below). As planned the exact-answer marking122system was used. However, although a formula had beendeveloped to weight scores (see Table 6 and Table 7 belowand Table A in the appendix), this particular acceptable-word system was dropped. This was because of the insightthat there was a more valid and less cumbersome way toanalyze the answers in relation to the goal of finding outif students were looking beyond the immediate sentence ofthe doze blank rather than just within the blank.The new method of analysis was to delete from thespreadsheet the columns containing the types of fillerswhich either required external knowledge, followed by intrasentential re-iterative fillers.To analyze the data for each doze passage two methodswere used. First, to get a general idea, the mean score(exact—answer) was found for each group and graphed using“Quattro” bar graphs (see Figures 3, 8, 13, and 18).Second, two statistical programs were used. The first wasthe T—test to see if the two classes involved in the mainproject were significantly different from each other on thepre-test and on the post-test. (Note that class 2 hadcompletely dropped out by the end of the second test becausethe uncolored pre—test had proved to be too frustrating forthe majority and the randomly—colored second test toodifficult for the remaining six more determined students.)The second statistical analysis, called “One Between andRepeated Measures ANOVA” was then applied to give asensitive analysis and to check for trends across the series123of test passages. These statistical programs were used onlyfor the main project where the cells were larger than forthe pilot project.It should be pointed out here that bar graphs using“Quattro” were made for a) the mean scores for the full setof the students in a class who wrote the particular test andb) a subset of students (n=14 for each class) who hadwritten the pre- and post-test and most of the practicepassages. The graphs of the full set were made to give ageneral idea of what the trend was for the different formsof doze tests and the subset were to give a more validpicture of what had happened in the tests.To use the “One Between and Repeated Measures ANOVA”statistical program the score data for the five practicetests were typed into a data file and run on the Main FrameComputer at the University of British Columbia. The purposeof the statistical analysis was to determine whether or notthe treatment (completely—color—coded doze practice) groupshad made any significant improvement compared to the control(standard doze practice) groups and to the other colortreatment groups and also to see what effects the passageshad on the group scores.The data were analyzed and interpreted in Chapter Four,and recommendations and further research suggested inChapter Five.124TABLE 13ANSWER COLLECTION AND BLANK SCORE DETERMINATION CHARTAS IT LOOKED ON A SPREADSHEET(PILOT PROJECT)Page 1TOTAL PRETEST TEST 2 TOTALMEAN EXACT MTH MEAN EXACT MTHMEAN ACCEP MTH MEAN ACCEP MTHPage 1SCORES FOR BLANKSPOINTS FOR......... EXACT WORD Exact Mth = 1EXPLANATION. .. ..... CLUE LOCATION same wrd : diff wrdPOINTS FOR T1:not in passage T:1O/9/8TYPES OF CLUE...... T2: beyond senten T:7/6/5 : 8/7/6T3: within senten T:5/4/4 : 6/5/5Cloze Form 4 lA4a: Student 1 Class A Group 4 Level a(everything 2A4b: Student 2 Class A Group 4 Level bcolor—coded) 3A4c: Student 3 Class A Group 4 Level c4A4d: Student 4 Class A Group 4 Level d5A4e: Student 5 Class A Group 4 Level eMEAN EXACT MTHMEAN ACCEP MTH125doze Form 3 6A3a: Student 6 Class A Group 3 Level a(everything. .... . . . 7A3b: Student 7 Class A Group 3 Level bcolor-coded 8A3c: Student 8 Class A Group 3 Level crandomly) 9A3d: Student 9 Class A Group 3 Level dlOA3e: StudentlO Class A Group 3 Level eMEAN EXACT MTHMEAN ACCEP MTHCloze Form 2 llA2a(blanks l2A2bcolor-coded) l3A2cl4A2dl5A2eMEAN EXACT MTHMEAN ACCEP MTHdoze Form 1 l6Ala(Standard l7Albdoze) . . . . . .... . .l8Alcl9Ald2OAleMEAN EXACT MTHMEAN ACCEP MTH126Cloze Form 4 21B4a22B4b23B4c24B4d25B4eMEAN EXACT MTHMEAN ACCEP MTH(For more see Appendix D.)page 1BLANK1 B2 B3his 0004 son 000(1) He 10002same same sameT:10/9/8T:7/6/5 T:7/6/5127TABLE 14CRITERIA FOR DEVELOPMENT OF WEIGHTED SCORESFOR THE PILOT PROJECTA. Availability of clue1. clue is not in passage at all2. clue is beyond sentence of blank3. clue is within sentence of blankB. Relation of clue to deleted word1. clue is same as deleted word (re—iterative)2. clue is same part of speech (synonym)3. clue is same part of speech with same root(friend.. .friendship4. clue is different part of speech, but with same root(eq. except. . .exception)5. clue is a phrase equivalent to the deleted word (eq.in the world...universal)6. clue is a pronoun or antecedent (Bill...he)7. clue is a referent (eq. the boy...Bill)8. clue is a cohesive pair (eg. salt and pepper)C. Correctness of answer1. answer is exact and same, i.e. same word as blank2. answer is acceptable to meaning of passage3. answer is acceptable to meaning of only the sentenceof the blankTABLE 15CALCULATION OF CLOZE PASSAGE SCORESFOR THE PILOT PROJECTTo calculate the scores categories, use the followingprocedure.1. Start with category A: availability of clue.The highest possible score is 10.If clue is not in the passage, Al. = 10If clue is beyond sentence, A2. = 8If clue is within sentence, A3. = 61282. Then consider category B in conjunction with A. Thisgives three types of scores as follows.TYPE 1: (no same-word (re-iterative) clue withinpassage)pvAl, if Cl (answer is same as blank) = 10Al, if C2 (acceptable to passage) = 9Al, if C3 (acceptable to sentence) = 8TYPE 2: (clue beyond sentence)(clue same as deletion) (clue different)A2, if Bi A2, if B2 through B7andCl = 7 andCl = 8or C2 = 6 or C2 = 7or = 5 or C3 = 6TYPE 3: (clue within sentence)(clue same as deletion) (clue different)A3, if Bl A3, if B2 through B7andCl = 5 andCl = 6or C2 = 4 or C2 = 5or C3 = 4 or C3 = 5TABLE 16ANSWER COLLECTION AND BLANK SCORE DETERMINATION CHARTAS IT LOOKED ON A SPREADSHEET(MAIN PROJECT)°NAME OF PASSAGE° TYPE OF BLANK° (EXTERN) ° INTER- ° INTRA0 0 B1 0 B2 0 B30 0 0 0 00 KEY WORD 0 *** 0 0°STUDENT 0 0 0°IDENTITY 0 0 0 00 0 0 0 0Classl 0 0 00 0 0 0 0°JNK 0 1 0 0 0°DAV 0 AAAAA 0 1 0 (empty):APR : 1 : (empty) : 1 :0Total: 2 0 1 1°Mean 0 .66 0 33 0 33 0C e 0 0 a°Class3 0 00 0 0 0 0°TM ° (empty) ° 1 0 00 0 1 1 0: : (empty) : (empty) : (empty) :°Total: 0 1 • 2 • o°Mean: 0 •33 ° .66 0 0129130CHAPTER FOURANALYSES AND DISCUSSIONINTRODUCTIONIn Chapter One the reasons for the present researchproject were given. Chapter Two presented the historicalbackground which underpinned these reasons and suggested theframework for the research design. Chapter Three set outthe design of the pilot project and mentioned some of theresults which led to the design of the main project. It isnow the role of Chapter Four to summarize the changes indesign (please see Table 17 below), point out some of theproblems and solutions in the projects (located in Table18), and in light of all of these to present the researchdata. The data are of two kinds. One kind includes a lookat the scores for the right answers. The other looks at thesame information as it interacts with the confidence levelsof the students (the confidence levels being determined bythe number of doze blanks filled by the students).Finally, in Chapter Four some conclusions will be drawnabout the effectiveness of the color—coded doze procedurein helping students look for inter-sentential reiterativeword clues for use in filling the doze blanks. ChapterFive will discuss implications of the research and suggest avariety of possible future designs for research with thecolor—coded doze.131CHANGES MADE TO THE CLOZE PROCEDURE AFTER THE PILOT PROJECTAs a result of observations made in the pilot project anumber of changes were undertaken.Table 17 below.They are outlined inCHANGES TOTABLE 17CLOZE PROCEDURE AFTER PILOT PROJECT0 TYPES OF 0 PILOT PROJECT 0 MAIN PROJECT°CHANGES ° 00° Prof ici— • upper intermediate 0 0 upper advanced0 ency (as defined by the ° (as defined by° level of 0 participating ° TOEFL and Language• students college) 0 0 Proficiency Index,e 0 0 three levels mdi—0 0 ° ° cated) to make0 0 ° ° doze easierAssign— according to pre— ° • each of the three• ment of • test results: the 0 0 classes was given 0• students 0 students were placed° 0 a different form of 0to groups° so the groups’ mean ° the doze test0 0 scores were the same° 0 0No. of ° four 0 0 three• forms of 0 °° (to increase the 0• the doze0 0 no. of students 00 0 0 0 in each cell to0 0 0 0 lessen the factor 00 0 00 of chance)0 No. of • eight ° seven° tests 0 0 (to save time and0 0 0 0 reduce boredom) 0° Source of° “CANRead”— an ESL ° “Canadian Society”° passages e anthology about 0 0 an introductory 00 0 Canadian heroes 0 0 university text on 00 0° Canadian issues 0Length ° 300 words (approx.) ° ° 330 words (approx.) 0° of the ° (to keep the sense 0‘ passages • 0 of the passage) 0V -- 30 minutes for00000Length oftime todothedoze 0tests 0each test25 minutes for pre-°0 0 and post-tests; 0• 20 minutes others 0o 0 (in pilot only 15 00 minutes needed) 01320 Deletion 0 every fifth word 0 0 rational deletion 0° pattern 0 0 0 (to maximize no.0 0 0 0 of reiterative—0 0 0 0 word blanks) 00 Method of° typed passages 0 xeroxed passages;° deleting 0 onto spreadsheets 0 0 underlined and° words 0 and deleted every ° ° counted the 00 ° fifth word 0 reiterative-words; 00 0 0 wrote the passages 00 0 0 0 into a notebook; 00 0 0 0 highlighted 00 0 0 reiterative—words;0 0 0 deleted some from 00 0 ° before, others 00 0• from after, some 00 0 0 from between other 0e 0 0 0 members of the set.°0 0 0 0 Blanks were never 00 0 0 side by side. 00 Location 0 on the inside of 0 on top of page 1 of0° of color—0 the front cover • the passage (for 0ocoding key° ° easier reference 00 Feedback ° gave answer keys 0 0 gave answer key° ° after the following 0 immediately after0 0 test 0 its practice test0 Data 0 used Appleworks ° used Quattro (IBM) 0° Collect— • spreadsheet; used 0 0 spreadsheet; used 00 ion 0 rows for students, rows for students, 00 0 columns for key ° columns for key0 0 answers for each 0 0 answers for each 00 0 doze blank and each° ° doze blank and0 0 student’s answer; 0 0 each student’s 00 0 assigned each stud— 0 anonymous ID listed0 0 ent a ranking number° 0 within class in no 00 0 (Student 1. Class A, 0 particular order; 00 0 Group 1, Level a) to° 0 labelled each blank°0° attach to their own 0 as to type (intra-,°0 0 anonymous ID code; inter—sentential, 00 0 each answer weighted 0 external (nothing) 0o Analysis 0 graphed mean scores ° • graphed mean scores°0 0 for a) all 50 blanks ° for a) all 500 0 and b) beyond re- ° blanks, b) intra- +00° iterative Nouns, 0 0 inter—sentential,0 0 Verbs, Adjectives, • ° c) intra-sentential°0 0 Adverbs 0 0 and d) inter-sent- 00 0 0 0 ential blanks 0133PROBLEMS AND SOLUTIONS IN PILOT AND MAIN PROJECTSTable 18 which follows looks at the changes in terms ofthe problems faced in the pilot project. Changes are givenas intended solutions to those problems. Problems in themain project are presented as a guide to solutions insubsequent doze procedure research.TABLE 18PROBLEMS AND SOLUTIONS IN PILOT AND MAIN PROJECTSo ao amortality0000anumber insufficientfor statisticalpurposes (4/cell)number barelysufficient(20/cell max)(13/cell wrotemost tests and sowere suitable fordoing statistics)a few studentsmissed tests anddid not do themlater, but theirscores were based° on the mean of• PROBLEMS ° PILOT PROJECT 0 0 MAIN PROJECT 0° and 0 0 0 00 SOLUTIONS 0 0° Schools: 0 tight schedules ° ° tight schedules 00 0 to finish ° to finish their0 their own curr— ° ° own curriculums, 0° ° iculuius, but they ° ° but volunteered0 0 volunteered one ° ° three classes 0° °class 000 0° did not accept0 0 ° ° pilot project 00 0 0 passages, but 00 0 0 0 offered a text 00 0 00 fromtheir0 0°°program° Test 0 0 0 poorly lighted, 0° Location:° 0 0 but teacher of0 0 0 0 treatment groupa 0 0 put strong lamp0 0 00 onatablein 00 0middle of group0 00a00Students: 0quantity :a000a00aa00a00a000000ae000a few studentso missed tests, most° wrote them anotherday0000000a00a000oooeoooeoa••oe•00UOTSS3S.•S9.O.SAeoaet;uoepuze.xaz•SeTdODo;otd—oToD•vq‘AtToT;..xe•••apoo—otooo.••ubeq.oeço.xdet;jopuee;.xeu•..xe;u.xd—ioiooet.•::: pu-axdet.uo.ieioeqo.se.xoosaP9SrLD.xO.x.ze000UTdStT.fl.T000UOU(OUSTT)0ea(;uTod;o.000s;uapn.stie;c.eA) 0uTodeuos0Ope;deoosietisu000etqssodAu;nq00000‘se;—;sod/-e.xd000.9UTUOTSflUOD00a0ao.buTpetUT000•pesn.xet;oupU000•U3jeSt4ioie0.et;0;ieisuet;•..bUTdI;000000000e.tez4et;•0•00;t’s;uepn;s00.0pro;atesai•0;nq‘sp.xo,jo0000dnoietouneuo0bUT;UtId•.UOT.OflpOJd. 0..tflDTJTPLeA.tflDTTPLeA•ase00000:s;se;.000ezo.•dno.xb;ueiu;ei;oet;..zotox;uooaUTeti;;ouo;nq‘;rtopecIdopOSSTDetots;se;eoT;oxdet.tt;osueut0e0000aa000a0000‘Ct135THE RESEARCH DATAThe Pilot ProjectWhen the test means were graphed for the four groupsthe data appeared as on the graphs immediately below. It isapparent from these sample graphs that overall the CCCtreatment group did not do as well as the standard—dozecontrol group but did accelerate and slightly overtake themain control group on tests 5 and 6, losing the advantageonce again in the post-test when the color—coding wasremoved. Yet, because the number of subjects in each groupcell was so small (N = 4), it would have been difficult tostate any statistical conclusions with very much confidence.The results needed to be confirmed by research done withmore students. Even though the results of the pilot projectcould prove nothing, they did suggest that the color-blank-only control was the weakest and could be eliminated, andthus allow more students per group in the main project.Using the pilot graphs as a (very rough) guideline andgiven the limited opportunity to obtain students to do dozeresearch, it was decided to move on and try out ideasuncovered in the review of the literature. The researcherwanted to know what would happen when the students were at ahigher language proficiency level and when more intersentential reiterative words were available to be found tofill the doze blanks. As a result of these questions thesechanges were introduced into the main project for all thegroups. It was hoped that more proficient readers could136make better use of the color—coded parts of speech to findthe reiterative clues that were known to the researcher tobe somewhere in the passages. It was thought that these newconditions, applied to all the groups, just might improvethe treatment group’s scores over the control groups who hadeverything but the color—coded clues. To make the color—coded parts of speech easier for the treatment group tolearn, the color—coding keys were moved from the inside ofthe test folder to the top of page one of each of thepractice doze passages.rPILOT:Mi50Blanks(N=4)1. Standard/2. Color—Blank/S.Random/4. CCC407 CJJ.21Stq....CROLJP2r_________S2”a__20ICROUP315.JGROUP410W5...0L_1IITi1213141516Ti18tests(forgroup1—4)(-t7pIJI)sdnf:ijb.ioj)Sj.Sq.m ii.. If.’—.-.C;’ZO1[P3P01—J[0i0)/S(UI)S4LE1flEJ[IIVIO1Id91LL91caLLci[1-fr I.I-’ 4-.)0,139The Main ProjectAn overviewThe data for the main project are comprised of twokinds and pertain to groups (classes) 1 and 3, group 2having completely dropped out due to frustration by the endof the first practice test. The first data summarize thescores for the exact—word and acceptable—word answers givenby the students in the control group (Cli) and the treatmentgroup (C13). The summaries are in tabular, graphic, andstatistical forms for the exact-word scores and tabular formfor the acceptable—word scores. The second data summarizethe same information but as it relates to the confidencelevels of the students. (In finding confidence levels, itshould be noted that for absentee students the actual N wasused instead of including the mean of the means for thoseabsentee students as was done for the other kind ofsummaries.) Confidence levels were defined in terms of thenumber of doze blanks filled. The interaction of exact—word answers (only exact—word answers were focussed on) andconfidence levels yielded scores under the heading of“productive confidence”. This data was summarized intabular and graphic forms only. While summarizing andfinding trends in exact—word answer scores were the main andoriginal concerns in the present CCC study, productiveconfidence information emerged during the research as a wayto describe and discover what effects the CCC was having onthe students’ motivation to use the color-coding to find the140same—word re—iterative inter—sentential clues. Together,the outcome of these two kinds of data analyses along withthe answers from a post—session questionnaire was meant toindicate •the effectiveness and value of using the color-coded doze procedure with adult ESL students.Summary of the dataTo determine any improvement in the doze ability theresults of each doze test were subjected to four analysesfor answer scores as follows:a) all 50 doze blanks,b) intra— and inter—sentential (Erra) doze blanks,C) intra—sentential doze blanks, andd) inter—sentential doze blanks.Each of these analyses examined exact—word answer scores andacceptable—word answer scores. Raw—score tables areincluded for exact—word answers for all the tests. Withregards to acceptable—word answer scores, as a matter ofeconomy, raw—score tables are only included for the pre— andpost tests. In Appendix H mean—score tables summarize andcompare data for both the exact- and acceptable-word types.Then, to determine any improvements in motivation(confidence), four more analyses were made:e) the total number of blanks filled,f) the number of blanks filled to the possible numberof blank types,g) the total number of blanks filled correctly, and141h) the nwnber of correct blanks as a proportion of thenuier of blanks filled.Analysis (e) was meant to indicate the confidence level and(g) the level of productive confidence of each group foreach test. For the motivational analyses it was feltsufficient to present only the exact—word data. The reasonfor this was that the motivational data was determined byincluding the score data in their calculation. Because theacceptable—word scores increased the distance between thecontrol group and the treatment group scores in favor of thecontrol group, this meant that the resulting motivationalscores would be greater for the control group. As the goalof the motivational analysis was to discover if thetreatment group could close the gap between it and thecontrol group, there was no point in pursuing theacceptable—word analysis any further. However, the pursuitmay be worthwhile in subsequent research if the treatmentgroup does better than the control group.For the present research many tables and graphs arestill provided. In order for them to be conveniently viewedand understood, the data for answer scores and confidencescores are grouped together so that all summaries for (a)the 50 blanks are shown, all the summaries for b) the intraand inter-sentential (Erra) blanks are shown, etc. through(c) and (d). After these tables and graphs there is adiscussion section expressing the apparent findings.Following these are the T-test and ANOVA tables and an142explanation. Included next, the discussion of themotivational data intends to add information to the previousdiscussion to help decide whether or not the rationalizedcolor—coded doze procedure is worth all the extra effort toproduce the color—coded passages. The motivational data arepresented in that particular location in order to help inthe interpretation of the data and in the decision on whichto rely on more, the tabular/graphic or the statisticaldata. Finally, a sumary table of the questionnaire and adiscussion are included to further formulate the answerabout the value of color—coding to the doze procedure.143PRE- °(N = cell)TESTInter0 (Erra =Intra- +4.0° Inter—2.0° sentential4.0°word blanks)TABLE 19 FOR THE MAIN PROJECTTEST DATA: EXACT-WORD (AND SOME ACCEPTABLE-WORD) SCORES°CLASS 1°EXACT-WORD SCORES° N ° PRE- PRE°CL1° TEST TEST°° All Erra°CLASS 1°0° N ° PRE- PRE- PRE°CL1° TEST TEST TEST°° All Erra Inter0°JNK° 205155 120°°DAV° 17 0 12 0 8 0°APR° 150 110 90°°ABC° 7.0 6.0 3.0°JPN° 13.5 12.5 8.0°BOX° 14 5 10 5 7 0°MAT° 210 100 70°°USA° 18 0 14 0 10 0°MCN° 17.0 13.0 10 0°TAK° 17.0 15.0 12.0°YAS° 11.0 7.0 6.0°SSK° 15 0 11 0 8 0°ABC° 12.5 8.5 6.5°JNK° 105 75°DAV° 10 0 6 0°APR° 90 60°ABC° 6 0 5 0 3 0°°JPN° 12 5 10 5 6 0°°BOX° 12 5 9 5 7 0°°MAT° 10 0 8 0 6 00°USA° 11 0 10 0 6 0°°MCN° 13 0 10 0 8 00°TAK° 11 0 10 0 8 0°°YAS° 7 0 4 0 3 005(O 9 0 7 0 5 00°ABC° 7 5 5 5 3 50eTotol2g 0 99 0 65 50°Mn ° 9 9 7 6 5 00° 0 19.9 15.2 10.1°(Tot=Total) °Tot° 199 • 0(Mn = MeanScore)146.0 106.5°°Mn ° 153 112 820° ° 30.6 22.2 16.4°° N ° T2 T2 T2 ° T3 T3 T3 ° T4 T4 T4 °°CL].° All Erra Inter0 All Erra Inter ° All Erra Inter0°JNK° 12 6 10 7 6 70 13 0 11 0 8 0 ° 17 0 15 0 12.0°°DAV° 10 0 9 0 4 00 13 0 12 0 6 0 ° 15.0 13 0 10 0°°APR° 5 0 4 0 2 00 9 0 7 0 6 0 ° 16 0 14 0 11 0°°ABC° 7.0 6 0 2 0° 12 0 11.0 7 0 ° 12 0 9 0 6 00°JPN° 13.0 11 0 5 00 13 0 12 0 6 0 • 15 0 12 0 9 0°°BOX° 9 0 8 0 4 00 7 0 5 0 4 0 ° 8 0 6 0 4 00°MAT° 9 0 8 0 3 0° 15 0 14 0 10 0 ° 15 0 13 0 9 0°°USA° 3 0 3 0 2 0° 10 0 9 0 5 0 ° 12 0 9 0 7 0°°MCN° 11 0 10 0 6 0° 14 0 14.0 9.0 ° 12 0 10 0 8 0°°TAK° 8 0 6.0 3 0° 4 0 3.0 2 0 ° 11 0 8 0 5 0°°YAS° 8 0 7 0 3.0° 12 0 10 0 5 0 ° 16 0 14 0 11 00°SSK° 9 0 8 0 3 0° 12 0 10 0 7 0 0 15 0 13 0 10 0°°ABC° 12.6 10.7 67° 12.0 11.0 9.0 ° 14.0 10.0 7.0°°Tot°117.2 101.4 50.4°146.0 129.0 84.0 0178.0 146.0 109°Mn ° 9.0 7.8 3.9° 11.2 9.9 6.5 0 13.7 11.2 8.4°°° 18.0 15.6 7.8° 22.5 19.8 13.1 ° 27.4 22.5 16.8°144°CLASS 1 EXACT-WORD SCORES (cont’d)o N ° T5 T5 T5 T6 T6 T6°CL1° All Erra Inter All Erra Intere°JNK° 15 0 13 0 11 0° 14 0 11 0 10 0°DAV° 12 0 11 0 8 0° 13 0 11.0 8 0°APR° 12 0 10 0 7 0° 13 0 11 0 9 0°ABC° 12 0 10 0 7.0° 12 6 10 7 6 7°JPN° 15 0 15 0 12 0° 12 6 10 7 6 7°BOX° 16 0 14.0 11.0° 13.0 10.0 8.0 0°MAT° 15 0 13 0 10 0° 13 0 9 0 7 0°USA° 6 0 6 0 5 0° 12 6 10 7 6 7°MCN° 17 0 15 0 12 0° 27 0 23 0 17 0°TAK° 11 0 11 0 9.0° 12 6 10 7 6 7 0°YAS° 18 0 18 0 14 00 12 6 10 7 6 7°SSK° 13 0 12 0 10 0° 12 6 10 7 6 7°ABC° 17 0 14 0 11 0° 22 0 18 0 13 0°Tot°179.0 162.0 127.0°190.6 157.2112.2°e 13.8 12.5 9•70 14.7 12.1 8.60 0 27.6 24.9 19.4° 29.3 24.2 17.3VEXACT-WORD SCORES(Erra =intra- +6.0° inter-2.5° sentential6.0°word blanks)6 0°7 5°8.0°7.0°5 0°13.0°11 0°8 0°10.005.0°ACCEPTABLE-WORD SCORES° N °POST- POST- POST-°°CL1°TEST TEST TEST°°A11 Erra Inter0°JNK° 105 85°DAV° 7 5 5 5°APR° 12 0 10 0°ABC° 11 0 9 0°JPN° 11 5 9.5°BOX° 13 0 11 0°MAT° 90 70°USA° 110 75°MCN° 15 0 14 0°TAK° 17 0 15 0°YAS° 12 0 11 0°SSK° 15 0 13 0°ABC° 10.0 8.00 N °POST- POST- POST°CL1°TEST TEST TEST° °A11 Erra Inter°JNK° 21 5 17 5 13 0°DAV° 12 5 6 5 3 5°APR° 180 120 80°°ABC° 190 120 90°°JPN° 15 5 12 5 10 5°BOX° 18 0 15 0 11 0 0°MAT° 21 0 15 0 14 0°USA° 110 105 80°°MCN° 19 0 16 0 14 0°TAK° 23 0 20 0 16 0°YAS° 21 0 16 0 23 0°SSK° 21 0 17 0 13 0°ABC° 15.0 12.0 8.0°Tot°154.5 129.0 95.0°°Mn ° 11.9 9.9 7.3°°° 23.8 19.9 14.6°°Tot°235 5 182 0 151 0°Mn ° 18.1 14.0 11.60 0 36.2 28.0 23.2145(MN = MeanScore)0 N ‘ T2 T2 T2 ‘ T3 T3 T3 ‘ T4 T4 T4 0°CL3° All Erra Inter’ All Erra Inter ° All Erra Inter0°TMI° 0 0 0 ° 4 0 4 0 3 0 ‘ 4 0 2 0 1 0’‘NM].° 6 0 5 0 3 0’ 18 0 15 0 9 0 • 15 0 12 0 9 0°°TM2° 14 0 12 0 7 0° 7 0 6 0 5 0 ‘ 11 0 9 0 6 00‘HN ‘ 7 0 6 0 4 0° 10 0 10 0 6 0 ‘ 9 0 7 0 4 00° 4 0 3 0 0 0’ 9 0 7 0 5 0 0 15 0 12 0 9 0’°GST° 7 0 6 0 3 0° 8 0 7 0 5 0 0 9 7 8 3 5 7’‘YK 0 9 0 6 0 2 0’ 11 0 10 0 8 0 • 9 0 7 0 3 000MM2’ 10 0 7 0 3 0’ 12 0 11 0 7 0 14 0 12 0 8 0°‘HG ° 4 0 3 0 1 0° 7 0 5 0 2 0 • 12 0 10 0 7 0’‘HO ‘ 4 0 3 0 0 ° 2 0 2 0 2 0 ‘ 13 0 10 0 7 0°‘MI ° 7.0 6 0 3 0° 4 0 4 0 3 0 ‘ 9 7 8 3 5 70‘TI ‘ 6 0 6 0 4 00 9 0 8 0 5 0 ‘ 10 0 8 0 6 0’°NN ° 8 0 6 0 2 0° 9 0 9 0 5 0 0 16 0 13 0 9 0’‘Tot° 86.0 69.0 32.0°l10.0 98.0 65.0 ‘147.4 118.6 80.4°‘Mn ° 6.6 5.3 2.5° 8.5 7.5 5.0 11.3 9.1 6.2°° ° 13.2 10.6 4•90 16.9 15.1 10.0 ° 22.7 18.2 12.4°‘CLASS 3‘EXACT-WORD SCORES‘CLASS 3‘ACCEPTABLE-WORD RE’° N °PRE- PRE- PRE- ‘ ( N = cell) ° N °PRE- PRE- PRE-‘CL3’TEST TEST TEST ‘ °CL3°TEST TEST TEST° ‘All Erra Inter’ (Erra = ‘All Erra Inter000intra- +°TMI’ 7.5 4.5 3.0° inter— ‘TM].’ 11.5 6.5 5.0 0‘MMl° 9.5 7.0 5.0’ sentential ‘MM].’ 15.5 11.0 9.0‘TM2° 11.0 9.0 6.0°word blanks)’TM2° 14.0 9.0 6.0°HN 0 8 0 7 0 6 0° ‘uN ‘ 20 0 9 0 8 0‘MW 0 10 0 8 0 6 0’ ‘MW ‘ 14 0 10 0 8 0°GST° 6 0 4 0 3 0’ ‘GST° 9 0 5 0 4 0‘YK ° 8 5 6 5 1 0° ‘YK 0 13 5 8 5 3 0 0°MM2’ 20 0 0 ‘ ‘MM2’ 40 10 10’‘HG ° 13 5 10 5 6 0° ‘HG 0 20.5 16 5 12 0 0‘HO ° 14 0 10 0 7 0° ‘HO ‘ 21 0 15 0 12 0‘MI ‘ 14 0 13 0 9 0’ ‘MI ° 16 0 14 0 10 0‘TI 0 3 5 2 0 1 0° ‘TI ° 8 5 5 0 4 0‘NN ° 6.0 4.5 4.0° ‘NN ‘ 8.0 4.5 4.0‘Tot°1l3 5 86 0 57 00‘Mn° 87 62 44°‘ ‘ 17.5 13.2 8.8’(Tot=Total) ‘Tot’ 175.5 115.0 86.0 0‘Mn’ 135 88 66’‘ ‘ 27.0 17.6 13.2146°CLASS 3 EXACT WORD SCORES (continued)• N 0 T5 T5 T5 T6 T6 T6 °°CL3° All Erra Inter All Erra Inter0°TMI° 8 0 6 0 6 0° 4 0 4 0 2 0 00)]O 15 0 13 0 12 0° 9 7 8 3 5 7 0°TM2° 11 0 10 0 7 0° 19 0 15 0 11 0‘HN • 14 0 14 0 11 00 9 7 8 3 5 7°MW ° 9 0 9 0 7 0° 9 7 8 3 5 7 0°GST° 10.0 8.0 7.0° 13.0 10.0 6.0°YK ° 22.0 19.0 15.0° 19.0 17.0 12.0°1012° 14 0 13 0 10 0° 14 0 11 0 7 0 °°HG ° 16 0 16 0 12 0° 9 7 8 3 5 7 °°HO ° 11 0 9 0 7 00 17 0 15 0 12.0 0°MI 0 13 0 9 0 7 0° 15 0 11 0 7.0 0°TI ° 9 0 9 0 7.0° 16 0 15 0 11.0 0•° 17 0 16 0 13 0° 9 7 8 3 5 7 0°Tot°169 0 151 0 121 0°l65 5 139 5 96 500l3O 116 930 127 107 74° ° 26.0 23.4 18.6° 25.5 21.5 14.8Inter0 (Erra =intra- +4.0° inter—6.0° sentential7.0°word blanks)8.0°6.502 0010 0055°6 0°7.006.0°2.005.0°0 N °POST- POST- POST-°°CL3°TEST TEST TEST°A11 Erra°TMI° 6 5 4 5°MM1° 13 0 10 0°TM2° 12 0 11 0°HN° 105 95°MW° 95 75°GST° 5 5 3 5°YK ° 19 0 16.5°MM2° 90 70°HG° 110 70°HO ° 13 0 10 0°MI ° 12.0 10.0°TI 0 3.5 2.0°NN ° 11.0 9.0° N °POST- POST- POST°CL3°TEST TEST TEST° °A11 Erra Inter°TM1° 8 5 5 5 5 0°MM1° 21 0 13 0 8 0 0°TM2° 21 0 17 0 12 00 ° 22 5 17 5 15 0°MW° 165 125 95°°GST° 5 5 3 5 2 0°YK ° 27.0 25 5 15 0°!012° 120 100 85°°HG ° 19 0 14 0 12 0 °°HO ° 19 0 15 0 11 0°!4I° 140 100 60°°TI° 55 30 30°°NN 0 18.0 14.0 10.0°Tot°135 5 107 5 75 0°°Mn° 104 83 58°°° 20.8 16.5 11.5°°Tot°226 0 160 5 117 0°Mn° 174 123 90°° 34.8 24.6 18.0 0•00000000000aaaaaa>‘-3‘-3‘-3‘-3‘-3o0’Lii.W1)(tI-Cn(I)0It(DI1ooa000aa00aaaaaaa0owItHP’Jt’I-JI-1-’I-’I-’(tI-’‘-3WM0t!iUI13WUi01e0Ui‘101t1UTjftIIIoaaaaa00aa000aaaaaLa)La)La)La)La)La)0w0.)La)La)MLa)ICDtJ0Z-0)Uit.)t.JLa)-J•U)Cl)•°*0aaaaaa0oaoaaaaaaa>1>>>>>>II>1>‘ILi1OWIJM1%)t4.)1%.)ê-’(DWllaQIXPI0IW‘.0—1—It30)OI’.O0101•aa•a•••I•0IC)HwUi..00aiw‘dIrtHZT.1’1ii-ti00oo0000000*0000)JCflIItIIt’iO*01oWIM(43(43(43fl(43IHQIXPIO010’0wi-’--ii-.oi•I.•••••U)•I•010WWMtMLii0-J‘.0P.30101‘dIrtIi-i-I•000000‘1•a00000000OHHHHHHHHU-JLa)UUI—’‘dWI-i’MOIll01•I-IJ,-.‘i-J‘-s’-<’oWMLa)0)La)I-’UUI-’La)s.QWI-.Moaaoaaa•aaaaaoLTJ1-3t-1ZcCl)0‘-3tI-’HItHI1M‘.0-J0’0)‘.00Z0’HIt‘-30II-bOHHHHHHH0(4.)U0)—10)‘.00U•aaaaaaaaaa0000aoacoAll50BlanksN=1330-____---..--.--.3_rj015””-V--5....SV0’III•IIPRE—121.3141516POST—tests(fixclasses1and3)149TABLE 21 FOR THE MAIN PROJECTPRODUCTIVE CONFIDENCE SCORES FOR ALL 50 CLOZE BLANKSPCS°TEST°LEVEL NO NO TOTAL TOTAL°% OF TOTAL CORo C OF OF • OF ° NO C NO CBLANKSC NO • RECT°o°READ-°BLANKS°STUD-° OF C OF CFILLEDC OF °BLANKS°C°ABIL-° PER °ENTS .BLANKSeBLANKSC TO °BLANKS° TO CC CITY • TEST C IN e CFILDCPOSS_ FILLED° NO C°°EACH C C ° IBLE °COR- • OF CC 0°TEST 0 C C °RECTLY°FILLED°o 0 C 0 C C • In % C C In % CC C C C AxB= C C D/C= • ° F/D=C C°A C B • C CD CE • C G CC°Grl • C 13 ° 650 CA388 CA59 7 •A129 0A33 2 CCPre_C 15 C 50 C C C C V CC C3 C C 13 ° 650 • 353 0 543 C 113.5C 32.2 CA means that the score for this group is higher than forthe other group.C CGr1 C C 11 ° 550 C 344 CA62 5 C 92 C 26 7 CC T2 C 14 C 50 C C C o0 CGr3 C ° 13 C 650 C 309 C 47•5 C 86 0A27 8 CC°Grl 0 ° 13 C 650 CA496 CA76 3 CA146 C 29 4 •o T3 C 15 C 50 C C 0 C C CC°Gr3 C 0 13 C 650 C 367 C 56 5 C 110 CA3O 0 CC°Grl C 13 C 650 C 514 CA79 1 ° 178 CA34 6 Co T4 0 15 • 50 C C C C C C CC CGr3 C C 11 C 550 C 395 C 71 8 C 128 C 32 4C°Grl C 13 C 650 CA478 CA73 5 CA18O C 37 7 Co T5 ° 13 C 50 C C C C C C CC CGr3 C C 13 ° 650 C 397 C 61 1 C 169 CA42 6• °Grl • 0 7 C 350 C 275 CA78 6 C 115 41 8e T6 • 12 0 50 C C C C C C CC CGr3 C C 8 0 400 • 254 63 5 C 117 CA46 1 CC CGr1 C C 13 C 650 CA432 0A66 5 CA154 5C 35 9 CCpostC 15 50 C C C C C C0 CGr3 C C 13 C 650 C 379 C 58.3 C 135.5 35.80InHAll50BlanksN=13(T2N[=ii,T4N2=11,T6N1=?,N2=8)600.__________________CLASS1500a:’7-ib..jL)L)C)C200-•...PRE—T21.3141516POST—tests(forgroups1i:Jfld3)rIM[il.30BlanksN=13(T2NI=ii,r1l4N)Ziirp(3NI.7Np.8)—_____I-)J1ICLL\SS3I—•RSi___I-..JI.-...I-siR$LHIIjwL—[•1FPRE—T2T.314T5T6POST—tests(forclusses1and3)All50BlanksN=13(T2NI=11,T4N2=11,T6NI=7,N2=8)200-CLASS10•1L(’Cii0CL/SS3_____________—i00”0g50-----—0-0IIIIIPRE---T2T314T516POST—tests(forclasses1and3)If)‘-IAU50BlanksN=13(T2NI=ii,T4N2=11,T6NI=7N2—8)F..--“0CLASS140CLASS.3002O”j_JIiJ00__.IPRE—T2T.3’T4T5113POST—tests(forclasses1and3)oo0000000000000000>‘-3‘-3‘-3‘-31-3‘-3oO•Uiit.3(tw0‘-3r10WIoo00000000000000tn0(I)CFF-44ZHHHHHCtI—’UiWUIUi.(DUI<F-’xi<((3tnflCFIIt-’0o00000000000000000rFtnWWWWWObW0-0UIt3t.3Wd-J‘J0“iI...)WW3W$00Z•11)((3•00o000000000000000000>1>>>>>>F>1>‘IMCHI—It.3JHH0MIH0I<IiZ0ID.t’J‘.0UIMIU101WrFtTi.3op.•.•••I-b•I•OIOF-’IiOI’.OF)‘.0UI0)Oi0t3I)‘Ir1-HWI‘1I(F1oa00000000a000ICFIItxiØ*I-Ifl(3tJHHHHIH0IX0HW0)Ui0I-’-JIW01W(tJ•I••••••in•t•olowtiLlOIUIUII3HHOsOIFJ‘IrFWtI)H.,.I‘.QIrIPIo.()a•aaaaa.()ao.()a00001-3I—IoGHHHHHHHi’:z’Z0)W-.1WWWH‘tiL.30Lii(1)ID•GH1-3HHHHHti’<’0LiiW0)WWWI-’.Wm’nlwi-a-Lu1-3U)00)Hoaaaaaa00•0o00tElt-3Zct-’0HHHHHrIHIi[ujw-Os0).‘.00ZHIDHrFt-3‘(1)ZIDI-bOHCOHHHHH01%.)0)-.10)‘.001<w0000000000000000H VI]{ntra—andInt..ersententialBlanksN=13-2;fl_________i_iAC—(I••JiLU_________0:;-___________PRE—T21.3T415T6POST—tests(fc..irclasses1and3)156TABLE 23 FOR THE MAIN PROJECTPRODUCTIVE CONFIDENCE SCORES FOR INTRA- AND INTER-SENTENTIAL BLANKSPCs°TEST° LEVEL° NO 0 NO 0 TOTAL° TOTAL°% OF °TOTAL °COR- 0o 0 OF 0 OF OF 0 NO. ° NO °BLANKS° NO C RECT°0°READ-°INTRA-°STUD-° OF 0 OF °FILLED° OF °BLANKS°o°ABIL—°INTER-°ENTS °INTRA-°INTRA-° TO °BLANKS° TO °0 0ITY °SENT- 0 IN °INTER-°INTER-°POSS- °FILLED° NO 00 0°ENTIAL°EACH °BLANKS°BLANKS° IBLE °COR- ° OF C0°BLANKS°TEST 0 °FILLED° ‘RECTLY°FILLED°0 0 0 0 0 C 0 In % 0 In % •0 0 0 0 0 AxB= 0 D/C= 0 0 F/D= 00 C ° A B° C • D °E F • G 0°Gr]. ° 13 0 520 0A318 0A61 2 • A99 0 31 1 0°Pre— 15 0 40 0 0 0 0 0 0 00°Gr3 ° 0 13 ° 520 0 274 • 52.7 0 86 °“31.4 0A means that the score for this group is higher than forthe other group.0°Grl ° 11 0 330 0 220 0A66 7 0 80 0A36 4 0T2 0 14 0 30 C 0 C 0 0 0 00°Gr3 0 0 13 390 • 196 ° 50 3 • 69 0 35 20°Gr1 ° 0 13 ° 481 0A381 0A79 2 0A129 0 33 9 00 T3 0 15 0 37 0 0 0 0 0 0•°Gr3 • 13 • 481 • 282 58 6 • 98 0A34 8°°Grl ° 0 13 • 455 • 371 0A81 5 146 0A39 4 0• T4 C 15 0 35 0 0 0 0 C 0 0°°Gr3 ° 0 11 0 385 • 282 0 73.2 • 106 37 60°Grl 0 0 13 0 481 0A380 0A79 0 0A163 * 42 9 00 T5 • 13 c 37 0 0 0 0 0 0 0° CGr3 0 13 ° 481 0 316 65.7 ° 151 0A478 00°Grl ° 0 7 ° 273 0 213 0A78 1 ° 93 ° 43 7 0C T6 0 12 C ?39 0 0 0 0 0 0 00°Gr3 0 * 8 ° 312 ° 199 • 63 8 ° 98 CA49 2°Grl ° 13 520 0A350 0A673 CA129 •A36 9 0°Post° 15 0 40 0 0 0 0 0 C°°Gr3 ° 13 ° 520 C 308 • 59.2 107.5° 349I—InInt.ra-andInter—sententjaiBlanksN=13(T2NI=ii,T4N2=11,T6NI=7,N2=3)400—-________LLI-kcc1t1I_‘f.’)uUCLASS.3200--”---1IY)--“-.PRE—T3T4T5T6POST—tests(forclasses1and3):E:ZE.Ej(1IjF.i.’1ICLLiC:_i.._‘UI(j)(/‘CflUIcjoWIL’/IIII//I!Ii/!’/iIIIIJI//ifII/III/L///I -‘r•.•.II-:tjjN00II C\1zHHzzW?iifflJJiflZlZJI7ZlIIij-,7/Iflh7i’Vi/ii))1/ill/Z7Z’1ii 1-L1-=:f=;—HJLiLi)CH-0(i) jI\C)(J3LI.-.I,---fl77il//,’///////,’1jjJj/Jli//z1//,)//J/,llj,LU0C)C)C)C)C)(0 C)-i89t0•iInI-IIntra—andInt..er—slententialBlanksN=13(T2NI=ii,T4N2=11,T6N1=7,N2=8)1Q’_________________IL)L)16CCUSS•140IIU.IAC7-Dion‘LF43I)___________E100_s80”zzJzzzzz:zzzPRE—T213T4T516POST—tests(forclasses1and3)0I’DI-IIntra—andInter-sententialBlanksN=13(T2NI=i1,T4N2=11,T6N1=7,N2=8)‘UC1içcCLASS.33Q....C...)20””.C••:__....PRE—T21314T516POST—tests(farclasses1and3)oo00000000000000>‘-3‘-3‘-3‘-3‘-3‘tJti001UiU)CD‘-3o00000000000000000OWr1F-4P’-‘I-’HHHHHrtHI-3WLTiOLTjUi(%)WUiUi01liP)ttilr1IIt-oa0000000000000000lirtt#)Li)(A)(A)(A)(A)OLi)0t.j(A)(A)M(.)ØI’)dOZOUit%)t%)(A)xjo•U)Cfl0o00000000000000000III[ZjØHI>>>>CD>1>0IXIIITj(DIm0101Ui0101UiIDIP)o••••••I•CDl0I-li’I-Io0t.JJ(00)00)1(0IrIHP)Z0II(tiZIHoaa00000000000Cl)z>1>rt-IILG*Wflp31-3>IOIXliHO)IH(0(31UII-•DIP)•I.•••••U)•I•010WUlI(0(A)01H-.J011H‘CI)IIrtIU)oa0000000000000ZHoGHHHHHHljtj(A)W(A)(A)H(A)H”Z 0HOU)•HI-HHH‘1’dOWH(A)WI-’-WWJ’ijWCflti)oaaao0a0aaaa00Ijl_3G‘-3II-’HHHitHljt4(0..J01(00ZWHit‘-3 Co1bGHHHHHHH0t%)(A)0)-0)(0Ii0(A)0000000000000000H 01 I-’Intra—sent..en[ialiBlanksN=1312‘m1L)C’IACC7LLJ-JJ3rjB______________lbC.-)0)ELLLi1 :PRE—T2T.3T4T5T6POST—tests(farclasses1and3)163TABLE 25 FOR THE MAIN PROJECTPRODUCTIVE CONFIDENCE SCORES FOR INTRA-SENTENTIAL BLANKSPCS°TEST° LEVEL0 NO 0 NO 0 TOTAL° TOTAL°% OF °TOTAL COR- 00 0 OF 0 OF 0 OF ° NO ° NO 0BLANKS NO 0 RECT°0°READ-°INTRA-°STUD-° OF 0 OF °FILLED° OF °BLANKS°0°ABIL-° SENT- °ENTS °INTRA-°INTRA--° TO °BLANKS° TO0°ITY °ENTIAL° IN °BLANKS°BLANKS°POSS- 0FILLED NO 00 0°BLANKS°EACH 0 °FILLED° IBLE °COR- 0 OF 00 0 • PER °TEST 0 0 0 °RECTLY°FILLED0 0 0 TEST 0 0 0 0 0 0 00 0 0 0 0 0 0 0 In %0 0 0 0 0 AxB= e D/C= 0 0 F/D=o o 0 A 0 B 0 C ° D 0! 0F 0 G0°Grl ° 0 13 ° 117 0 A9j 0A77 8 0A32 5 0 35 7°Pre—° 15 0 9 0 0 0 0 0 0 00°Gr3 ° ° 13 0 117 0 70 ° 59.8 0 29.0 0A414A means that the score for this group is higher than forthe other group.°°Grl 0 011 0 88 ° 75 0A8520 43 0A57300 T2 0 14 0 8 0 0 0 0 0 0 0°°Gr3 ° 0 13 ° 104 ° 74 0 71 2 ° 37 0 50 5°°Grl 0 13 0 130 0 A95 0A73 1 ° A45 0A47 40 T3 0 15 0 10 0 0 0 0 0 00°Gr3 0 °13 °130 070 05380 33 04710°Grl °13 0 78 0 84 0A9550 37 °440°e T4 15 • 6 0 0 0 0 0 0 00°Gr3 0 11 0 66 0 44 06670 33 0A75000°Grl 0 0 13 0 91 ° ‘63 °‘69 2 ° 35 ° 55 6T5 13 7 0 0 0 0 0 0°°Gr3 ° 013 ° 91 0 54 05930 30 05560°°Grl ° ° 7 0 70 0 49 0A7780 21 °429°T6 0 12 0 10 e 0 0 0 0 0 0°°Gr3 • 8 ° 72 ° 48 667° 30 •A6250°Grl ° 0 13 ° 117 0 “90 076 9 0 “34 0 37 8°Post 15 ° 9 0 0 0 0 0 0 00°Gr3 ° 0 13 ° 117 0 79 ° 67.5 0 32.5°”41.1 0Ifntra—sententialBlanksN=U3(T2NI=1I,T4N2=11,T6NI=7,N2=E3)100CLASS1-o1)(‘IACCnV..Vn20””.0IIIIPRE—12T3141516POST—tests(forclasses1and3)(cPLJr:.sssL)I.:)Jcij)ssq-1: (t ‘I:’ () —I—.crU)CL0 () if;’ Cr00(9ZN’ZTNaL‘TTz:Nf’1J‘TT=TNEa)TNS)1Efl]lR.!1.U[-Y41U[ S_1EUI1TJ[1[—-ISC’.d91cifrir0£S9[10CrfrIr’4J1.’0’,LiC’9 ifl1.0Intrasententia1BlanksN=13(T2N1=I1,T4N2=i1,T6N1=7,N2=8)50—___oCLJ31a-)4C_____CLI6S3a-)30...(I)•S••‘-4-IC•b:PRE—T21.3141516POST—tests(forciass1and3)Int..ra—sententja1BlanksN=13(T2N1=ii,T4N2=11,T6NI=?,N2=E3)-—L)U,..C.(‘IACCt)1.J-....CLASS3‘4—_____[ilPRE—T2T.3T4T516POST—tests(forclasses1and3)oo0000000000000000>‘-31-3i-31-3)-3’-30oUi,.wr.i(1•jIiCl)rn00‘-3ft(Dp,Ioo0000000000000000owftHiHHH)iHHftH‘-3Wt40111Ui1%)W0101.O01I<Hxj<1:1ftIIt’o0000000000000000WWWWWOW0WWWfØXJOZ03UIw‘jITjoCl).o00000000000000000>1>>>>>Ii>1>‘IMØHt’JIH(‘0HHH>0HIHOIXliWI0‘.0ai-JOl0Qip,ftti•1.•••••H)•I•OIC)H(DGUi0)‘.0-.30IH‘dirtHli:I’xiIiIftlo0000000000000(1)ftIP’IMG)dPW()HIHHHHHHIOiXliHO(‘3oi-’-wiaip,•I.•••••U)•I•0(0W010100010‘.0(‘310)‘dIrt‘Cl)IIrtiWoo000e0•00e)00000ZH0ØHH,HHHHHFlW-JtOtOtOH‘dtoI-,.H’HOw1•HHHHHI-’li”’Ot’tltotoHtotoI-’-toU)Cl)Cl)o000000000000001:11-3‘1ZclG‘-3t)HHHHHftH1:1C.f)to-ja,toaHft‘-3 Cl)1I-bG)HHHHHHH0(‘3to0)-0)‘.0Ii0to0000000000000000I-’ 0 0)Inter—sententialBlanksN.=1325Ic-icç20•_______1)jçD*LiU)V1GV.PRE—1213141516POST—tests(forclasses1and3)170TABLE 27 FOR THE MAIN PROJECTPRODUCTIVE CONFIDENCE SCORES FOR INTER-SENTENTIAL BLANKSPCS°TEST°LEVEL° NO 0 NO ° TOTAL° TOTAL°% OF °TOTAL °COR- oo ° OF ° OF ° OF 0 NO ° NO °BLANKS° NO • RECT°•°READ-°INTER-°STUD-° OF 0 OF °FILLED° OF °BLANKS°o°ABIL-°SENT °ENTS °INTER-°INTER-° TO °BLANKS° TO •• °ITY °ENTIAL° IN °BLANKS°BLANKS°POSS- °FILLED° NO 0o°BLANKS°EACH ° °FILLED° IBLE °COR- ° OF 0o 0 PER °TEST ° 0 °RECTLY°FILLED°0 TEST 0 0 0 0 0 0 0o o 0 0 0 0 • In % ° ° In % •o o 0 0°AxB=° °D/C=°o a•A 0 B 0 c 0 D ° E °F G 0o°Grl 0 ° 13 • 403 0A227 °‘56 3 ‘65 5°’28 9°Pre—° 15 • 31 ° 0 0 0 0 0• °Gr3 ° 0 13 • 403 0 204 • 50.6 • 57 ° 27.9A means that the score for this group is higher than forthe other group.°°Grl 0 ° 11 0 242 • 145 0A59 9 ° 37 • 25 5T2 0 14 ° 22 0 0 0 0 0 0 0°°Gr3 0 13 0 286 122 0 42 7 0 32 0A26 2 00°Grl 0 ° 13 ° 351 0A286 0A81 5 0 A84 ° 29.4 0T3 0 15 • 27 0 0 0 0 0 0 00°Gr3 0 0 13 0 351 • 212 0 60 4 ° 65 0A30 7° °Grl ° ° 13 0 377 ° 297 0A78 8 0 109 0A36 70 T4 0 15 0 29 0 0 0 0 0 0 0° °Gr3 ° 0 319 238 0 74 6 0 69 0 29 0 0° °Grl ° 0 13 0 390 0A310 079 5 0A127 0 41 0 0T5 0 13 0 30 0 0 0 0 0 0 0°Gr3 ° 0 13 ° 390 0 291 0 74 6 • 121 0A41 6 0°°Grl 0 0 7 ° 203 0 164 0A80 8 0 72 0 43 9 00 T6 0 12 0 ?29 ° 0 0 0 0 00°Gr3 ° ° 8 ° 232 0 151 • 65 1 ° 68 0A45 0 00°Grl ° 0 13 0 403 0A260 064 5 ° A95 0A36 5°Post° 15 0 31 ° ° 0 0 0 0 0°°Gr3 0 ° 13 • 403 0 229 • 56.8 • 75 0 32.8 0I-Iinter—sententialBlanksN=13(T2NI=Ii,T4N2=i1,T6N1=7,N2=8)-3ft).ULAS125G.CLASS3QJ200””15C:luO5CCIIIIIPRE—t213T41516POST—tests(forc.Iasses1and3)pr-i-t of blanks fiiled to posble1720 S.-.—-- Cr)CD C) CD C) flL)‘ I1. .,L. ‘AL.&LL4J.LJ-L.LLA4..LI.A.LLL4-flEl(Dvj-H(.0o —1D C)]CJJ-aU/II//I/I/II,iIII/I/////h,i////Il/iiii,,’/IiifiiIii),,1ff!ul/IJ/J1ffJ//fItuIlIIJIiIullllffJ/47771177/,lfl)11I............TI!Jl!!II!JIlfJIIIII!!IlfJ!if!1A’J’J!JfIJf!1JJfJlIIIfiCDCO-4Fø44c4.r”zHHzHHzHII-YffL1I1AUn!I!IHInter-sententiaiBlanksN=13(T2N1=I1,T4N2=11,T6NI=?,N2=B)140IILL!100CSS380(•1J_iiPRE—T3141516POST—tests(fürclasses1and3)Inter—sententialBlanksN=13(T2NI=11,T4N2=11,T6N1=?,N2=8)‘roCLASS1400CLASS330........20””j.ci-U__.PRE—12131415113POST—tests(forclasses1and3)175An explanation of the tables and graphsThe above graphs are all very similar and show that thecontrol group did better on the tests than the treatmentgroup. If this were the only fact given, then it could beconcluded that the rationalized CCC had no significanteffect on the treatment group over and above the effect ofthe non—color-coded doze on the control group and thereforethe treatment would have to be considered a failure.However, another look at the graphs appears to indicate thatthere was a tendency for the treatment group to catch up tothe control group as long as the crutch of color—coding wasavailable. As the only difference in the tests was thecolor—coding, the acceleration can be seen to have beencaused by the color-coding, if not by the parts of speechthemselves. Therefore, it can be said that the color—codinghad reached the objective of the research, that is, to helDiluDrove doze scores and to increase the students’ awarenessof inter—sentential reiterative—word clues.A closer lookWhen the inter-sentential blanks were considered bythemselves it was seen that both groups had made progress.Interestingly, the increases approximately followed theincrease in nuiber of inter—sentential reiterative—wordclues. However, as long as the color—coding was present,the treatment group tended to improve more than the control176group in being able to locate and use reiterative—wordinter—sentential clues.Part of that greater improvement can be explained bythe fact that the treatment group seemed to doproportionally much worse than the control group on thefirst practice test, giving the treatment more room torebound. Actually, all else being equal, there should nothave been such a difference in test two, given that the pretest showed the two groups as being much more equal inproficiency, at least in terms of doing doze tests. It canbe assumed that the color—coding was the cause because itadded information that the students had to consider, thusadding an encumbrance, if not confusion. Then, as thestudents in the treatment group became more familiar withthe color-coding they were able to narrow the differencebetween themselves and those students in the control group.This suggests that those students in the treatment groupwere not only becoming less confused, but also moreconfident and able in the use of the color-coding todiscover the inter—sentential reiterative words to fill inthe doze blanks. Thus it was thought that another part ofthe greater improvement of the treatment group could beattributed to the color-coding of the rational doze.Four additional points should be made from examiningthe graphs. First, the main project showed that to someextent students’ were able to locate inter—sentential clues,with or without the color-coding as a crutch. This seemed177to validate Chihara and Oiler’s contention that proficientESL students do search globally for clues and do use them.Second, both groups tended to improve scores on all blank,intra—/inter—sentential, and inter-sentential analyses onboth out—of—50 mean scores and percentage—of— possible meanscores. This suggests that practice with the doze did anddoes lead to improvement. Third, there was a ceiling effectto that improvement. Fourth, despite the evidence for thevaluable role of color-coding, it is difficult to say thatthe students were using the parts of speech to help find theclues as to what word to put in the doze blank. Becausethe gap between the control and the treatment group appearedto widen in the post—test on inter—sentential blanks whencompared to the pre—test difference, it seems that thecolor—coding had not done as much to make the studentssensitive to parts of speech as had the practice by thecontrol group with no special color—coded clues. On theother hand, without the color—coding as a crutch, thetreatment group still did seem to improve. The improvementmust have come from the same source as did the improvementof the control group. As the pre—test and the practice testwere the same, part of the improvement could have come fromhaving done the test before. However, some of theimprovement for both groups could have come from the partsof speech, with or without the influence of the colorcoding. On the other hand, some of the improvement may havebeen an illusion as some of the tests had a lower178readability level. Table 9 shows that the readability oftests 2, 5, and 6 are easier than the rest and in some casesthe graphs rise in an apparent reflection of this. However,this conclusion is rather tenuous as others of the graphs donot behave in this fashion, possibly due to other influencessuch as boredom as in test 6, the passage with the easiestreadability level. See test 6 on the All 50 Blanks graphwhere both groups fall, the decrease occurring primarily onthe inter-sentential blanks.To summarize the above points it could be said thatboth forms of the rational doze procedure, color—coded ornot, were useful as a function of practice. It isinteresting and necessary to note, however, that despite allthe increases, the mean scores were still quite low, evenfor doze scores. No mean score ever went beyond thefrustration level. This, of course, could have been aresult of the difficulty of the passages.There is another observation that deserves somespecial attention. It was an afterthought in the researchanalysis to use the intra—sentential scores. This wasespecially so because the focus of the research had been oninter—sentential clues rather than on intra—sentential ones.It was also because it was easier to get total scores on thespreadsheet by first deleting the non-reiterative-wordblanks and then the intra-sentential blanks. It would havetaken another step to have gone back to the intra-/intersentential copy of each of the spreadsheets and deleted the179inter—sentential blanks. Instead, an arithmetic calculationwas done later by subtracting the inter—sentential meanscores from the intra—/inter—sentential mean scores, and agraph drawn.The intra-sentential graph showed that the mean scoreswere not consistent in improvement. On the first practicetest both groups improved on the intra-sentential blanks,but then declined. By test four (the third practice test)the treatment group started to improve while the controldeclined until test five when the mean scores were about thesame. On test six both made gains, but the treatment groupmade more. Both declined on the post—test, the controlgroup taking the lead once more.In terms of the number of available intra—sententialblanks the above can be explained in the following way. Fortest two both groups improved when there were less intrasentential clues and for test three declined when there weremore. After that the control group followed the downs andups in the number of intra-sentential blanks, while thetreatment group continued to improve until test six, thelast practice test. The treatment group was not affectedvery much by the decline in intra-sentential reiterativeword clues, but did, like the control group, reactpositively to the presence of the greater number of intrasentential reiterative-word clues, even more so. In thepost-test when no color—coding was present both groupsdeclined, with the treatment group falling the most and even180losing its advantage over the control group. It isimportant to note that on the post-test both groups stillhad made improvements when compared with what they had doneon the pre-test. From the above observations about theintra—sentential blanks, it can be concluded that color-coding appeared to have even a more powerful influence onfinding intra-sentential fillers than on finding inter-sentential fillers but there were not enough occurrences ofintra—sentential blanks in the passages to be certain, giventhe somewhat inconsistent nature of the intra—sententialmeans. It is important to emphasize here that despite theinfluence of the color—coding on the intra—sentential scoresthis does not negate its influence on inter—sententialscores.A statistical look at the dataThe above description of the graphs has a certainamount of merit and should be considered carefully.However, statistical methods have been designed andvalidated over the years. Without them any research can bemisleading, even though Dr. Boldt, the statistics advisor tothis research project, cautioned that statistics themselvesare only one of the tools in understanding the data. WhenT-tests (reported in Tables 28 and 29 below) and “OneBetween and One Repeated Measures Factor ANOVA” analyses(see Table 30 for the steps in the program and Tables 32 -40 for the results) were applied to the data, a different181picture from the graphs emerged. First of all thedependent-variable T-tests indicated that the control andtreatment groups were not significantly different on thepre—test. The difference shown On the graph was merely achance difference according to the T—test. Nor were theysignificantly different on any of the practice tests exceptfor Test 3, where the difference was in favor of the controlgroup. Even on the post—test there was no significantdifference indicated between the two groups. The conclusionis that the rationalized color—coded treatment led to nosignificant improvement for the treatment group over thecontrol group on the doze scores.T—tests (above) done on class 1 comparing the pre-testsand post-tests showed a significant difference indicatingthat the practice had led to improvement. Meanwhile, nosignificant differences were found by the T-tests (above)for class 3 from the pre-test to the post-test. This latterfinding is contrary to what was interpreted from the graphs.(Somewhat puzzling, however, is a seeming statisticalcontradiction. If there was no significant differencebetween class 1 and class 3 on both the pre— and post—tests,how could there be a significant difference between class 1from pre- to post—test, but not one for class 3? There musthave been some data error or a flaw in the interpretation ofthe statistical findings. At first the answer was thoughtto have been that dependent variable t—tests were donethroughout instead of using independent T-tests when the two182different groups were compared. To check this assumption astatistical program called “Minitab” was used and themistake was corrected. However, the previous results wereconfirmed.The problem was probably that the statistical testslacked the power to pick up the subtleties in the data. Thestatistical tests were too conservative given that thesample size available to the researcher was so small.Likely there was a significant difference from the pre- tothe post test for the treatment group. The graphs suggestedthat this was the case. Furthermore, the t-tests hadindicated that the groups were not significantly differenton the pre— and post—tests from each other and that therehad been significant improvement for the control group.Therefore, logically speaking, there must have been asignificant improvement for the treatment group from thepre- to post-test.183TABLE 28T-TESTS FOR THE MAIN PROJECT:Pre-test (Exact-Word Scores)1 vs Class 3 degrees of freedom = 120 Mean Diff SD Diff T-score P =• 0‘All 50 blanks 1.192 4.294 1.001 0.337 0• 0‘Intra—/inter— 1.000 4.619 0.781 0.450‘blanks 000‘Inter—blanks 0.654 3.555 0.663 0.520 0:Test 2 (Exact-Word Scores) :1 vs Class 3 degrees of freedom = 120 Mean Diff SD Diff T-score P =00‘All 50 blanks 2.394 5.846 1.477 0.166 0O0‘Intra—/inter— 2.494 4.891 1.838 0.91°blanks00‘Inter—blanks 1.408 3.330 1.524 0.153003 (Exact-Word Scores) :1 vs Class 3 degrees of freedom = 12• Mean Diff SD Diff T-score P =00‘All 50 blanks 2.769 3.940 2.534 0.026*•0‘Intra—/inter— 2.385 3.664 2.347 0.037*‘blanks•.‘Inter—blanks 1.462 2.757 1.911 0.080* Significant at the .05 level184:Test 4 (Exact-Word Scores) :1 vs Class 3 degrees of freedom = 120 Mean Diff SD Diff T-score P =•.All 50 blanks 2.348 4.533 1.867 0.086• 0lntra—/inter— 2.114 4.672 1.631 0.129 0°blanks0°Inter—blanks 2.208 3.884 2.050 0.0630 0T55t 5 (Exact-Word Scores) :1 vs Class 3 degrees of freedom = 120 Mean Diff SD Diff T-score P =0 0°All 50 blanks 0.769 4.850 0.572 0.578 00°Intra—/inter— 0.846 5.129 0.595 0.563°blanks a0 0°Inter—blanks 0.462 4.196 0.39.7 0.699O 06 (Exact—Word Scores) :1 vs Class 3 degrees of freedom = 12o Mean Diff SD Diff T-score P =o 0°All 50 blanks 1.897 7.300 0.937 0.367• 01ntra—/inter— 1.382 6.277 0.794 0.443 0°blanks 00 0lnter—blanks 1.204 5.134 0.845 0.4140185:Post_test (Exact-Word Scores)1 vs Class 3 degrees of freedom = 12• Mean Diff SD Diff T-score9°All 50 blanks 1.462 5.387 0.978Vlntra—/inter— 1.654 5.398 1.1050a:d1a 1 (Exact—Word Scores)vs Post—test degrees of freedom = 12• Mean Diff SD Diff T—Ratios—1.962 2.883 —2.453—2.308 2.983 —2.789°All 50 blanksVlntra—/inter°blanks0• Inter-blanks* Significant at the .05 level:d1a55 3 (Exact—Word Scores):Pre_test vs Post-test degrees of freedom = 12• Mean Diff SD Duff T-Ratios050 blanks —1.923 4,476 —1.549lntra—/inter— —1.654 3.815 —1.563• blanksInter—blanks0V000347V0291V90.1661.538 3.761 1.47500VVP=•0.030*00 016*000.001*—2.269 1.867 —4.383000V00.147a014400.118•2 .966V—1.385—1.6830186TABLE 29SIGNIFICANT DIFFERENCES(As determined by t-tests) P < .05 Ni 13 N3 = 13PRE-TT: Class 1 vs 0 ERRA TNTEIClass 3 .ALL 0°Exact—word • not sign. not sign. not sign. aCLASS 1: Pre-test vs ALL ERRA INTER° Post-test 0°Exact-word °significant significant significant°CLASS 3: Pre-test vs • ALL ERRA INTER. Post-test°Exact—word not sign. not sign. not sign.°POST-TEST: Class 1 vs ALL ERRA INTER0 Class3°Exact-word • not sign. not sign. not sign.For the practice tests a “One Between and One RepeatedMeasures Factor” ANOVA was used. This ANOVA is a moresensitive tool than the T—test and was warranted because theT-tests on the pre-tests showed the treatment and controlgroups as not significantly different. It was used todetermine what effects the subjects, the tests, and aninteraction of the two, if any, had on the doze scores. Itwas also used to analyze the trend across the five practicetests. This ANOVA was applied to the data which considereda) all the 50 doze blanks, b) a combination of the intraand inter-sentential blanks, and C) the inter-sentential187blanks. The ANOVA computer program is given in Table 30following and the ANOVA summary tables right after.TABLE 30STATISTICAL PROGRAM: TREND ANALYSIS -ONE BETWEEN AND ONE REPEATED MEASURES FACTOR ANOVA01 $run sas:sas sercom=—log sprint=—a 1=howdata• par=;size=4000k2 data;infile filel;04 input subjects a b score;5 proc sort;byab;07 proc means mean std n;byab;var score;10 title ‘rep. meas.’;°l1 proc anova;°12 class sujJects a b;°13 model score= a b subects(a) a*b b*subjects(a) /0ss4; 0014 means. b / snk e=b*subjects(a);015 test h=a e=subects(a); 0016 test h=b a*b e=b*sub ects(a);17 means a / snk e=subects(a); 0‘18 title ‘rep. meas’; 019 proc glm;20 class subjects a b; 021 model score = a b subjects(a) a*b b*subjects(a) /• ss4; 0°22 contrast ‘b linear’ b —2 —1 0 1 2 / e=subects (a);‘23 contrast ‘b quadratic’ b 2 -1 -2 —1 2 / e=subjects(a);’‘24 contrast ‘b cubic’ b -l 2 0 -2 1 / e=subjects(a);•25 title ‘trend anal.’;TABLE 31DATA COLLECTION TABLE FOR ANOVA°A(GROUPS)° B (TESTS)‘ °Subjects’Test 2 ‘Test 3 ‘Test 4 ‘Test 5 ‘Test 600 ‘ 1. JNK 0 0‘ 2. DAV ‘ ‘0 3. APR ‘ 0 0 00 ‘ 4. ABC 0 00 0 5• JPN ‘ ‘ ‘ 0‘ O 6. BOX 0 0 0 0°Classl°7.MkT°0 8. USA ‘ 0 0 0 00 9. MCN 0 0 0 0 0 0‘ 10. TAI( ‘ *0‘11. YAS 0 0 0 00‘12. 55K 0 0 0 0 00‘13. ABC 0 0 0 0 0 014 • TN 0 0 0 0 0‘ 015. TM 0 0 0 0 0 *0 016. MM 0 0 0 0 0 00 17 . TM 0 0 0 0 0‘‘18. HN 0 0 0 0 0‘19. MW 0 0 0 00 Class 3 °20. GST 0 0 0 0 0 0021. 1414 0 00 022. HG 0 0 0 0 00‘23. HO 0 0 0 0 0 00‘24. MI 0 0 0 0 0025. TI 0 0 00‘26. NN * 0 0 0Note: A Group, B = Test, variables used in ANOVA program.188189TABLE 32SUMMARY TABLE: ONE BETWEEN-GROUPS FACTOR AND ONE REPEATEDMEASURES FACTORFOR ALL 50 BLANKS°Source Sum of Degrees of Mean F• squares freedom square‘Between°subects• A 134.64 a—1= 1 134.64 4.72*’5(A) 684.69 a(n—1)= 24 28.53‘Within‘subjects• B 668.92 b—1= 4 18.5l*°• AB 15.51 (a—l)(b—l)= 4 0.43° B x 5(A) 867.35 a(n—l)(b—l)= 96 9.03 0‘TOTAL 2371.11 abn—1=129* Statistical significance at the .05 levelResult 1: There was a significant difference betweensubjects, i.e. between the control and treatmentgroups.Result 2: There was a significant diference withinsubjects, i.e. between some tests.TABLE 33SUNMARY TABLE: ONE BETWEEN-GROUPS FACTOR AND ONE REPEATEDMEASURES FACTORFOR ALL INTRA- AND INTER-SENTENTIAL BLANKS‘Source Sum of Degrees of Mean F: squares freedom square‘Between‘subjectsO A 110.51 a—l= 1 110.52 4.87*’• S(A) 545.12 a(n—l)= 24 22.71 :Within 0subjects ,00 B 504.51 b—l= 4 16.50*’• •AB 12.98 (a—1)(b—l)= 4 0.42B x S(A) 733.86 a(n—1)(b—l)= 96 7.64TOTAL 1906.98 abn—1=129 0* Statistical significance at te .05 levelResult 1: There was a significant difference betweensubjects, i.e. between the control and treatmentgroups.Result 2: There was a significant diference withinsubjects, i.e. between some tests.190TABLE 34SUMMARY TABLE: ONE BETWEEN-GROUPS FACTOR AND ONE REPEATEDMEASURES FACTORFOR INTER-SENTENTIAL BLANKS°Source-- FD• 0subjects •0• A 59.10 59.10 4.88*°290.77 12.12 :Withinsubjects• B 611.58AB 10.15• B x S(A) 464.64Sum ofsquaresMeansquareDegrees ofreedoma-1= 1a(n—1)= 24b-1= 4(a—i) (b—1)= 4a(n—1) (b—1)= 960.31.59*°0.520°TOTAL 1436.24 abn—1=1294.84* Statistiàai significance at the .05 levelResult 1: There was a significant difference betweensubjects, i.e. between the control and treatmentgroups.Result 2: There was a significant diference withinsubjects, i.e. between some tests.191TABLE 35ASTUDENT-NEWMAN-KEULS TEST FOR VARIABLE SCORE (A)FOR ALL 50 BLANKS• This test controls the type I experimentwise error rateo under the complete null hypothesis but not under partialnull hypothesesAlpha0.05 DF=96MSE=9.03 0°NumberofMeans 2 3 4 5 0• Critical Range 1.65 1.98 2.18 2.32 :GROUPING MEAN NUMBER B (TEST) DIFFERENCE0 A 13.69 26 6° A Not Signif.• A 13.38 26 5o A Not Signif.° A 12.52 26 40 Significant0 B 9.85 26 3• Significanto c 7.82 26 2* Means with the same letter are significantly different.Results: When the scores of class 1 and class 3 wereconsidered together, from test 2 to test three,and from test 3 to test 4 there was significantimprovement.TABLE 35BSTUDENT-NEWMAN-KEULS TEST FOR VARIABLE SCORE (B)FOR ALL 50 BLANKS• This test controls the type I experimentwise error rate• under the complete null hypothesis but not under partial: null hypotheses :° Alpha — 0.05 DF = 24 MSE = 28.53• Number of Means 2 CCritical Range 1.93 :GROUPING MEAN NUMBER A (Group) DIFFERENCE :• A 12.47 65• SignificantB 10.43 65 3* Means with the same letter are significantly different.Results: When class 1 and class 3 were compared on thetotal of all there score for the practice tests,there was a significant difference, with class1, the control group getting the higher scores.192TABLE 36ASTUDENT-NENMAN-KEULS TEST FOR VARIABLE SCORE (A)FOR INTRA- AND INTER-SENTENTIAL BLANKS• This test controls the type I experimentwise error rate° under the complete null hypothesis but not under partial: null hypotheses :° Alpha 0.05 DF 96 MSE = 7.644°NumberofMeans 2 3 4 5Critical Range 1.52 1.83 2.00 2.13: GROUPING MEAN NUMBER B (TEST) DIFFERENCE :O A 12.04 26 5• A Not Signif.• B A 11.41 26 6o B Not Signif.O B C 10.17 26 4• C Not Signif.0 C 8.73 26 3 0• Significant 0o D 6.55 26 2 0* Means with the same letter are significantly different.Results: When the scores of class 1 and class 3 wereconsidered together, from test 2 to test threethere wa& significant improvement.TABLE 36B STUDENT-NEWMAN-KEULS TEST FOR VARIABLE SCORE (B)FOR INTRA- AND INTER-SENTENTIAL BLANKS0 This test controls the type I experimentwise error rate• under the complete null hypothesis but not under partial: null hypotheses :°Alpha—O.05 DF=24 MSE=28.53• Number of Means 2• Critical Range 1.93• GROUPING MEAN NUMBER A (Group) DIFFERENCE :o A 10.70 65 1• Significant• B 8.86 65 3* Means with the same letter are significantly different.Results: When class 1 and class 3 were compared on thetotal of all there score for the practice tests,there was a significant difference, with class1, the control group getting the higher scores.193TABLE 37ASTUDENT-NEWMAN-KEULS TEST FOR VARIABLE SCORE (A)FOR INTER-SENTENTIAL BLANKS• This test controls the type I experimentwise error rate• under the complete null hypothesis but not under partial: null hypotheses. :‘Alpha—0.05 DF=96 !4SE=4.84NuinberofMeans 2 3 4 5Critical Range 1.21 1.45 1.60 1.70 ::GROUPING MEAN NUMBER B (TEST) DIFFERENCE :o A 9.54 26 5• Significant• B 8.01 26 6B Not Signif.• B 7.28 26 4• Significant• C 5.73 26 30 Significanto D 3.17 26 2* Means with the same letter are significantly different.Results: When the scores of class 1 and class 3 wereconsidered together, from test 2 to test threethere was significant improvement. From test5 to 6 there was a significant decline.TABLE 37BSTUDENT-NEWMAN-KEULS TEST FOR VARIABLE SCORE (B)FOR INTER-SENTENTIAL BLANKS• This test controls the type I experimentwise error rate• under the complete null hypothesis but not under partial: null hypotheses.Alpha=0.05 DF=24MSE=12.12• Number of Means 2: Critical Range 1.26: GROUPING MEAN NUMBER A (Group) DIFFERENCEA 7.42 65 1• Significant• B 6.07 65 3* Means with the same letter are significantly different.Results: When class 1 and class 3 were compared on thetotal of all there score for the practice tests,there was a significant difference, with class1, the control group getting the higher scores.194TABLE 38GENERAL LINEAR MODELS PROCEDURE FOR ALL 50 BLANKSTest of hypotheses using the type iv MS for subjects (A)’° as an error term.° CONTRAST DF SS F VALUE PR > F 0a.Linear 1 608.88 21.34 0.0001 Significant’Quadratic 1 51.26 1.80 0.1926 Not sign.° Cubic 1 3.71 0.13 0.7215 Not sign.Results: For the linear contrast the F Value is higherthan 2.0 and the Probality is less than .05, sothere is a linear tendency of improvement fromtest to test.TABLE 39GENERAL LINEAR MODELS PROCEDUREFOR INTRA- AND INTER-SENTENTIAL BLANKS• Test of hypotheses using the Type IV MS for subjects (A)°o as an error term.• CONTRAST DF SS F VALUE PR0: Linear 1 440.13 19.38 0.0002 Significant’Quadratic 1 50.13 2.21 0.1504 Not sign.o Cubic 1 8.09 0.36 0.5562 Not sign.Results: For he linear contrast the F Value is higherthan 2.0 and the Probality is less than .05, sothere is a linear tendency of improvement fromtest to test.TABLE 40GENERAL LINEAR MODELS PROCEDUREFOR INTER-SENTENTIAL BLANKS• Test of hypotheses using the type iv MS for subjects (A)°• as an error term. 0• CONTRAST DF SS F VALUE PR > F•Linear 1 473.04 39.04 0.0001 Significant°: Quadratic 1 104.14 8.60 0.0073 Significant’° Cubic 1 20.02 1.65 0.2109 Not sign.Results: For the linear contrast the F Value is higherthan 2.0 and the Probality is less than .05, sothere is a linear tendency of improvement fromtest to test. There is a slight quadratic trend.0>F000195To summarize the ANOVA results for the five practicetests it can be seen that there was, a) a significantdifference between the two classes, b) some significantdifferences between earlier tests for both classes whentheir scores were pooled, and that C) there were nointeractive effects of the students and tests. The ANOVAalso showed that, d) the trends of the tests scores werelinear, i.e there was generally steady improvement frompractice test to practice test with the exception of aquadratic tendency in the inter-sentential analysis results.This quadratic tendency by definition takes precedence overthe linear trend but the much higher significance of thelinear trend suggests that despite the quadratic downturn inthe inter—sentential results, the general overall directionof the scores was upwards.Point (a): A significant difference between classesPoint (a) above is interesting but not very helpful interms of the research question. The ANOVA does notilluminate the fact, like the graphs do, that the treatmentgroup did poorly at first and then improved. Most likelythe initial drop while the treatment group was getting usedto the color—coding is what caused the significantdifference between the two classes. When compared to the T—test, the ANOVA possibly overstates the significantdifference between the groups, being that the T-test statedthat only test 3 was significantly different.196Point (b): Some significant differences between testsPoint (b) about the differences between tests for allthe students taken together is not very helpful as thisresearch project is more interested in the groups comparedto each other. The graphs appear to clearly show whichgroups did better on which tests, the T-tests showing whichdifferences were real and which probably were by chance. Infact, after looking at the graphs, the T-tests and theANOVA, there was a lot of “noise” as Dr. Boldt referred tothe ambiguity in the information generated by the researchdesign.Point (c): No interactive effects of students and testsPoint (c) is the most interesting,, revealing, anddisappointing as far as those people who support color—codeddoze procedure are concerned. If the rationalized CCC hadbeen really effective, then there would have been somethingin the tests that interacted with something within thesubjects to create a new dynamic which was different fromboth the tests and the students. Perhaps the color—codingwould have sparked an understanding of the doze exercisewhich would have generated motivation to get even moreinvolved in the process of finding inter-sentential clues.to fill in the doze blanks.197Point (d): Linear trend given by the scores of the practicetestsPoint (d) is encouraging for the doze procedure inthat there was benefit gained from the practice. The scoreswere never above the frustration level, but the studentswere headed in the right direction. The exception was inthe last practice test when the inter—sentential resultsdeclined and the gap between the treatment group and thecontrol group increased although not significantly. For theinter—sentential analysis there was also a slight tendencyof the trend to be quadratic. This quadratic tendency isshown on the graphs and in the statistics as a decrease inthe scores from tests 5 to 6 as far as the inter—sententialblanks were concerned. The downturn suggests that theceiling of accomplishment had been reached for both groupsby test 5 in finding inter-sentential reiterative-wordclues. It should be pointed out here, however, that forpractice test 6 the attendance was very low for both thecontrol and the treatment group. Only seven students in thecontrol group and eight students in the treatment group tookthe test. The others had to go on field trips.So far in this discussion of the different ways ofanalyzing the data, exact—word scores have been concentratedon. Acceptable-word scores should be talked about becausethey were discussed in the review of the literature.Indeed, however, the discussion will be short because in198terms of the present research results they did not improvethe effects of the rationalized CCC. They only widened thedifferences and brought in more error from assigning pointsto words that may or may not have been acceptable.The HypothesesHypothesis 1On the practice tests there will be no significantdifference between the treatment group which hastrained using the color—code doze and the controlgroups which have trained on either the non-coloredcolored-doze, the colored-blanks only, or therandomly—colored doze practice tests.Result 1According to the data as analyzed by the T-tests andANOVA the above null hypothesis was not rejected.Hypothesis 2On the post-test there will be no significantdifference between the treatment group which hastrained using the color—coded doze and the controlgroups which have trained on either the non—coloredcolored-doze, the colored—blanks only, or therandomly—colored doze practice tests.Result 2According to t-tests there were no significantdifferences, so the null hypothesis was notrejected.199Hypothesis 3On the post-test there will be no significantdifference on confidence scores between thetreatment group which has trained using the color—code doze and the control groups.Result 3Raw data was kept and informally compared throughthe use of graphs. These graphs followed similarpatterns to the mean score data. No significancewas suggested, so again the null hypothesis couldnot be rejected.Hypothesis 4On the post-test there will be no significantdifference in productive confidence scores betweenthe treatment group which has trained using thecolor—code doze and the control groups.Result 4From the statistical data it appears that there wereno significant differences between the control groupwhich used the non-colored rationalized dozeprocedure and the treatment group which used thecolor-coded version. This was the case for thepractice tests and the post test. Therefore, thenull hypotheses could not be rejected.200As is typically the case when the null hypothesis isnot rejected it does not mean that the treatment failed. Itjust means that it was not proved to be successful, in whichcase more research was suggested. The fact that both groupsimproved from pre—test to post—test was encouraging andsupported a futher look within the test results and infuture research projects. Therefore, immediately followingis a look at a) the statistical design of the presentresearch, b) information taken from the post—sessionquestionnaire, and c) the confidence and productiveconfidence data. In Chapter Five there are some suggestionsfuture research designs that are intended to address some ofthe questions raised in the present research.Problems with the present research designHaving looked at a number of data analyses it can beseen that the results of the research are not very clear.There was a lot of noise created because the samplingtechnique did not choose the students randomly. Thedifferences on the graphs could have been chancedifferences, statistically different, or part of each.Certainly, given the weak effect or even the inconclusiveresults of the rationalized CCC treatment, and combiningthe burdens of the expense and time involved in color—codingthe parts of speech, there is no way one could justify goingto the trouble of color—coding doze passages for classroomuse.201Hope given by the questionnaireYet, there are some things about the present researchwhich should keep the rationalized CCC alive for at least ashort time. First, in the questionnaire (please refer toTable 41 below) given after all the tests had been writtenthe ma)ority of the students in the treatment groupindicated that although they did not like doze procedurethey would prefer the color—coded form. Also they reportedthat they felt that they had learned something from thedoze exercises. Furthermore, the treatment group increasedthe nuaiber of guesses they made and increased on theirproductive confidence scores as indicated in the graphsabove. Finally, the students reported that they looked forclues inter—sententially as the clues appeared before orafter the doze the doze blank in question.202TABLE 41POST-SESSION QUESTIONNAIRE0 Class 1 Class 3• (N=l4) (N=15)o Yes No Yes No01. doze experience in• a) Japan: 1 13 2 130 b) Canada: 1 13 1 14: Enjoyed? 3 0 3Any difficulties? 13 1 14 1 :°2.2 More difficult 0• project beginning: 7 11• project middle: 3 2: project end: 4 :°2.3 Easiesto project beginning: 2 3• project middle: 4 10: project end: 8 1°2.4 Gain confidence? 5 8 8 7o2.5 Problems seeing words? 7 7 102.6 Improved guessing? 6 8 8 7°2.7 Looked0• a) within sentence: 14 0 15 0° b) before sentence: 12 2 13 1• c) after sentence: 12 2 11 2• Guessed: 13 0 12 2Did something else: 3 10 1 13Want to do doze again: 3 11 5 10 :°3.l Used color—coded doze? 12 3••3.2 CCC parts of speech 13 2: helpful?03•3 CCC parts of speech 2 13• made passages more• difficult?•• a) passages 2 and 3?o b) passage 4? 1: c) passages 5 and 6? :0020303•4 parts of speech: made passages easier? 13 2o a) passages 2 and 3? 6• b) passage 4? 5: C) passages 5 and 6? 2 ::helped find clueso a) within sentence fl° b) before sentence 12: c) after sentence 10• of the blank?.• 0:3.6. CCC helped? 7 :3.7. Type of doze preferred° in the future:0 0° a) non—colored 1° b) colored doze 14 0Whether the students in the treatment group were helpedor hindered by the color-coding, in the short run or thelong, with intra—sentential or inter—sentential reiterative-word clues, it is exciting to know that it playedan active part in the present research. Fortunately, ithelped to show that students did look beyond the éentence ofthe doze blank for clues to find fillers. Even moreimportant is that the color-coding did aid in the globalprocess, at least as long as it was present. Even when thecolor—coding was not used in the post—test the treatmentgroup still made progress over the pre—test. Finally,despite the difficulties of the doze procedure (color—codedor not) and a certain dislike of it, the students in thetreatment group reported that they thought they had learned204something and, most satisfying of all, the majority of thestudents in the treatment group indicated on thequestionnaire that in the future they would prefer thecolor—coded doze form to the non—colored one. To rule outthat their answer was ‘ust by chance, when group two (onlysix of whom had experienced one randomly—colored doze test)answered the same question on the questionnaire, color-coding got very few votes. These votes were so few and hadsuch a limited experiential basis that they could beconsidered to be given by chance unlike the firm choice bygroup three.The conclusions drawn from the analysis of the productiveconfidence scoresA look at the questionnaires indicated that thetreatment group preferred the color-coded doze to the non-colored form but was this conclusion supported by thestudents efforts on the doze tests? To find out the testswere re—analyzed in terms of the number of blanks thestudents had filled out (the confidence level) and thenumber of right answers as a proportion of that confidence(the productive confidence level). It should be noted herethat unlike the previous analysis absentee student scoreswere not interpolated.The reasoning behind the productive confidence tables(Tables 21, 23, 25, and 27 above and Table 42, below) was tofind out whether or not the color-coded parts of speech205treatment was encouraging students to fill in more blanksand more blanks more accurately than the control group.A quick look at the tables will show that the controlgroup did better than the treatment group on most of themeasurements and most of the time. This was true for theexact—word answers for a) all the doze blanks, b) intraand inter—sentential blanks, and c) inter— blanks. It wasalso true for intra- blanks through test 5. In test 6 andthe post—test the treatment group earned higher mean scoresfor exact answers.With regards to the productive confidence scores thereare two columns in the tables to look at. The first isColumn E (% of possible blanks filled in) and the second,Column G, the correct blanks as a proportion of the numberof blanks filled in. Column E gives the confidence leveland G, the amount of success the group had in the number ofblanks filled out. Group 1 was ahead in Column F despitethe kind of blank used, i.e. the control group was moreconfident throughout the research project. However, on theproductive confidence scores, the treatment group usuallygot higher scores than the control group. This isunderstandable and predictable in light of the fact thatconservative risk takers make less mistakes. In addition,as the treatment scores were only slightly higher, theimprovement made by the treatment groups was morestatistical than real. Furthermore, if there was anyadvantage over the control group gained by the treatment206group on the practice tests for inter- blanks, it was lostin the post-test. For intra- blanks, the treatment groupbegan on the pre-test lower on the number of blanks filledbut were a little higher than the control group onproductive answers. During the practice sessions theproductive confidence scores of the treatment group weregenerally lower than those of the control group but thetreatment regained the lead on the post—test. Nevertheless,the treatment group did not do as well as on the pre—test,while the control group reduced the difference slightly.TABLE 42TREATMENT GROUP’S CONFIDENCE LEVEL AS A % OF CONTROL GROUP’SCONFIDENCE LEVELFOR THE MAIN PROJECT0 TEST ALL BLANKS INTRA+INTER INTRA INTERPre— • 91.0 % • 86.1 % • 76.9 % • 89.3 %T2 • 760% • 75.4% • 836% • 71.3%0 0 0 0 0 0T3 • 74.0 % • 74.0 % • 73.6 % • 74.1 %0 • . 0 0 0T4 • 90.1 % 0 89.8 % • 69 8 % • 94•7 %0 0 0 0 0 0° T5 • 83.1% ° 83.2% 0 85.7% • 938% 0. • 0 0 • 0T6 • 80.8 % • 81.7 % • 85.7 % • 80.6 %Post— • 87.7 % 0 88.0 % 0 87.8 % 0 88.1 %Table 42 above shows the students in the treatmentgroup did not surpass the confidence levels of the controlgroup on any of the levels. The only major improvement in207confidence from pre— to post test came on the intra—sentential measurement.It appears that color—coding helped improve confidencemore on the intra—sentential level than the targeted inter—sentential level. This is surprising given that thedeletions were made in order to maximize the number ofbeyond—sentence clues and given that there were only a lownumber of intra—sentential re—iterative-word clues.208CHAPTER FIVEIMPLICATIONS OF THE RESEARCHINTRODUCTIONI wonder if the inventor of the doze procedure, WilsonTaylor, had a sense of humour besides his genius. Whatcould be simpler than deleting every fifth word from a 300word passage and multiplying the number of correct answersout of 50 by two to get the students scores in percent. Iwonder if Taylor realized at first how difficult it would befor gaps in the doze procedure to be filled, not onlydifficult for ESL students, but also difficult forresearchers in the context of methodological questions. Areview of the literature showed a challenge for both. Thereare so many seeming possibilities on both levels to considerin dozing the gaps that there will be many more “dozeencounters” (to borrow the name of a series of bookscontaining graded cJ.oze exercises).The original enthusiasm and effort expended on thedoze procedure by students and researchers, both in termsof moral support and the great amount of research, has wanedover the years. It is curious to note that ‘ust as studentshave been criticized for not looking beyond the sentence ofthe doze blank for clues, researchers can be criticized fornot trying to develop the doze procedure beyond a discreteexercise. In fact, the case is somewhat the opposite asindicated in the present research and in a review of thedoze literature that showed that students and researchers209do look and act more globally, sometimes successfully.Chapter Four pointed out how much students looked intersententially. Chapter Two told of Chihara and Oiler whoclaimed to have shown that the more proficient ESL studentsthey tested were able to use inter—sentential clues. Alsoit told of others who devised ways to test for use ofbeyond—sentence clues. (See Bachman and Henk.) On theother hand, Shanahan and Kamil and others claimed to showthe opposite. Working on the discrete level someresearchers tried to improve c].oze scores by focussing onintra—sentential clues such as initial letters and blanklengths equal to that of the missing words. Meanwhile,others tried to bridge the seeming opposite point of view(discrete-versus-global) by dissecting the doze passages toaccount for what influenced the students to look beyond thesentences, i.e. to the presence of inter—sentential clues.Apparently, few existed. Still others tried multiple-choicefillers and various kinds of matching which sometimesrequired students to draw on inter—sentential information.The present research made use of color-coding in an attemptto help students find intra- and inter-sentential fillers(explicitly of the reiterative-word type). The choice ofcolor—coding was borrowed from the use of color—coding inbusiness and in teaching from Gattegno who taught ESL/EFLstudents to read by color—coding the sounds and theirvarious spellings.210At the heart of the present research, then, is the notso simple issue of “discreteness” versus “globality”.Regardless of the difficulty of the doze passages in thisresearch, students using the rational doze and studentsusing the color—coded rational doze improved in theirability to find intra- and inter-sentential clues of thereiterative—word type.FUTURE RESEARCH DESIGNSTable 43 below suggests many variations on the color-coded doze that can be tested to see under what conditionsthe CCC could possibly help students look beyond thesentence of the doze blank to find the appropriate filler.(Note the “MP”s ‘ust to the right of the table. Theseindicate that these characteristics were used in the “Main(Research) Project”.There are two ways to test the effects of the color—coded doze procedure. The first is against the non-coloreddoze forms and the second is the color—coded doze againstvarious forms of itself.In using the former, the groups should be keptconstant. Group I should use the non—colored (I) form;Group 2, the randomly-colored (#)farm; and Group 3 thecolor—coded (*) form of the doze procedure. If the niunberof students available is limited, then the random form couldbe omitted. (In later studies, the non-colored form couldbe left out, so the random-color form could be left in.211There is a danger that the students doing the random—coloreddoze would drop out as they did in the present researchproject.) If two classes are used, one being judged to bebetter than the other, then the non—colored doze and color—coded doze could be used, each half of the each classreceiving one or the other of the forms. This technique ofsplitting the classes should partially compensate for thebias caused by one class being more proficient than theother at the start. Complete compensation would requiremuch larger classes to allow for a random selection ofstudents in each class. However, such large classes arerarely available.In using the latter, the differences within the variouscategories (read down left column) could be compared witheach other.All the above variations could be repeated by comparingESL speakers to native speakers of English and by comparingyoung ESL students to adult ones.Even though the possibilities for doze pro)ects seemlimitless, the two questions so important in the presentresearch still apply. First, “Can and do students lookbeyond the sentence of the doze blank to find reiterative—word clues to fill in the doze blank?” Second, “Do thecolor—coded parts of speech help students in their inter—sentential clue search more than when the words were notcolor—coded?”212The second question also leads to another question andanother variation in the research. The question is, “Whathappens when the color—coding is removed?” The presentresearch appeared to show that the advantage of the crutchcalled “color—coding” disappeared when the color wasremoved. To counteract this result, it was suggested by myadvisor, Dr. Slade, that the color could be phased out tosee if diminishing the students’ reliance on the color-coding could help them focus on the parts of speechthemselves rather than on the color—coding. The phasingout of the color could be done by reducing the intensity ofthe color from the first practice test to the last.Actually this tended to happen in the present research asthe printer ribbon started to wear out. However, a newribbon had to be used before all the tests had been printed,so the colors became more intense again. Another way tophase out the color might be to make the first practice allin color, the second one one-fifth in color, the next one—fourth in color, and so on. An added benefit of this wouldbe that the test scores could be broken down into color—coded and non—color—coded parts and these compared to eachother. (With a good computer program and/or a lot ofpatience this could be done successfully. Otherwise, theprocess would be too time consuming and tedious.)213TABLE 43SUGGESTIONS FOR POSSIBLE FUTURE RESEARCH DESIGNS“HP” = done in________________________________________________Major Project 0 GROUP I • GROUP II 0 GROUP III •)_____________I • 0 *TYPES OF CLOZE non—colored random—colored color—coded°MP°1. Standard • • *. 1 • . *2 Rational• maximum intra-type intra—type intra—type• no. of a a• re-iteratives inter-type inter-type inter-typeo (all parts ‘: of speech) intra/inter’ intra/inter 0 intra/interMP• maximum__1___*• no • of intra—type intra—type 0 intra-type a0 re—iteratives 0• (nouns inter-type inter—type inter—typeverbs, a 00 adjectives ‘ intra/inter intra/inter 0 intra—inter°0 adverbs)°PASSAGES • 1 • *l. Readability’ independent’ indedendent ° indépeñdeñt’o Level by 0— formula • instructiono instruction 0 instruction°dozeO results frustration’ frustration • frustration °MP°2. Order increasing 0 increasing 0 increasing mp°(according to ‘ 0O no. of re— decreasing decreasing decreasingoiterative_word 0 0• blanks)0 same • same • same °MPa 0 0• different • different • different0 0 0 0lsame+ • lsame+‘1 different 1 different3. Pre-/Post• tests• (no—color)a• lsame+°l different4. Relatedness’ related ‘ related • related ‘HP• of subject• matter • unrelated • unrelated • unrelatedSTUDENTS • • *0Di. Proficiency° high high high °MP• Level 0 e 0medium • medium • medium• some 0 some • some MP.. 0 0none none noneSCORING • 0 • *0 0o exact exact exact °). 0 0 0O acceptable acceptable 0 acceptable •MP• incorrecto if not same°• as re—iter—°• ative blank°°wordincorrectif not sameas re-iterative blankword• incorrect •M?O if not same°• as re_iter_0O ative blank°• word02142. Experienceo with doze0:e0: Spelling000000• exact exact exact °MP0 0 0acceptab :•DATA (steps) 0• *0 Collection spreadsheet° spreadsheet 0 spreadsheetMP0 (1 student! (1 word! (1 word/ (1 word!° row) coiumn/ 0 column/ 0 coiumn/ 0: : student) : student) : student) :Assignment -intra-type ° -intra-type °-intra-type °MPof type of °-inter-type 0 -inter-type -inter-type °MPre-iterative 0 neither • -neither °—neither •MP°word 00 0 0 0 00 Evaluating •-right = 1 -right — 1 °-right 1 •)(0 answers—accept = .5 —accept = .5 °—accept = •5°MP: _wron — O -wrong = 0 -wrong = 0 :Isolation (delete (delete (delete mpof type of • columns ° columns • columnsre—iterative containing 0 containing 0 containingword • other two • other two • other twotypes) types) types)215Note: “MW’ means a similar version done in Major Project.The above table suggest many variations on what wasdone in the present research project. Many variations arewhat could and should have been done in the main research ifthere had been enough students and enough time. However,even a lot more could be done to find out if students arelooking inter-sententially for clues fill in doze blanks.Think-aloud protocols could be recycled to include the newidea of color—coding. Furthermore, to increase the numberof participants for statistical purposes, many studentscould use the language laboratory where their thoughts couldbe recorded to determine which and how many inter—sententialreiterative-word clues they are able to find..0 Summation ‘—across for • —across for ‘—across for MP‘ of points ‘ student’s • student’s • student’s• to find • score ‘ score • score‘ total scores ‘—down for —down for ‘—down for ‘HPfor each • blank’s • blank’s blank’sstudent, score/group’ score/group score/group’means for ‘—compare —compare ‘—compare ‘MPa each blank! sum of down’ sum of down • sum of down’° group, and • totals to • totals to • totals to‘ means for ‘ sum of • sum of • sum of‘ each group ‘ across • across • across0 ° totals • totals • totals‘ ‘—do averages’ —do averages ‘—do averages’MP‘ Graphs ‘—use bar a —use bar ‘—use bar ‘HP‘ 0 graphs in graphs in ‘ graphs in‘ spreadsheet’ spreadsheet spreadsheet’‘ program program • program‘ Statistics ‘—use “One ‘ —use “One ‘—use “One ‘HPProgram • Between and’ Between and ‘ Between and’‘ ‘One Repeated’ One Repeated ‘One Repeated’‘‘ Measure 0 Measure • Measure‘ ‘Factor ANOVA° Factor ANOVA ‘Factor AVOVA’216For those researchers with even more resourcesavailable, i.e. sophisticated color computers, a whole newera in doze research is awaiting. Needed would be acomputer laboratory with a minimum of 20 IBM 486 colorcomputers in a network. The doze passage would bepresented to each student on his/her computer. Half thestudents would have the color-coded version, the other halfthe non—colored form. The students could type in theanswers. A computer program (yet to be written) would keeptrack of every key stroke to determine where the student waslooking for answers, and record every answer on a groupspreadsheet, keeping statistics throughout. The researcherwould know at every instant how everyone was doing. Bothsets of information could be produced in hard copy to makethe flow of results easy to see. The research could bedesigned so that the students would get immediate feedbackafter the test.Once the methodology has been polished, teachers couldmake use of the instant feedback to do what researchers bydefinition cannot do. Teachers could make any changes thatthe feedback indicated. With the sophisticated computers,spreadsheet and analysis programs, and peripherals likescanners, teachers could easily create more doze passages.Teachers could choose easier passages, more interestingstories, scan them into a word processing/spreadsheetprogram, check to confirm the readability level, find thenumber and location of reiterative words, delete as many of217those words as possible, color—code the blanks and remainingwords and make a master copy for use over the computernetwork. Such computer technology could give the teacherthe insights of a researcher without restricting the teacherto the inflexibility of pure research. Students wouldbenefit because the teacher could quickly adapt thematerials to the needs and interests of the student. Theresult of ease in preparation and analysis plus therelevancy to students of the doze passages could increasethe motivation of both the students and teachers to takefull advantage of the intended value, i.e. the globality ofthe doze procedure.The above vision about computer technology, color—coding, and the doze procedure is a dream, indeed. Butgiven the capabilities of computer designers, computerprogramers, plus the insights from the present researchproject, the dream of a computerized color—coded dozeprocedure is a reachable one. Even though the color—codedtreatment group in the present research did not overtake thenon-colored control group, there was enough improvement inoverall and inter—sentential scores and in motivation tosupport the value of more research into the use of colorcoded doze passages. Taylor, would have been excited thatthe door has been opened wide once more on his dozeprocedure and his dream for a simple global teaching/testingmethod is closer than ever.218From discrete, to global, to the universe...One’simagination can run on endlessly and that’s exciting for theresearcher. The actual research is much more down to earth,with the research and the students buried in the nittygritty. Getting out of the discrete level with the dozeprocedure, color—coded or otherwise, is like breaking the Gforce of gravity in a rocket ship. It takes a specialstudent with lots of determination to take off with thedoze test and reach some distant destination. The dozeprocedure is not really for the masses who want only to bespoon-fed. Nor is it for those researchers who want toisolate one variable in a universe of interacting forces,while students dance around the candy shop of fancy packagedknowledge. The doze procedure and its color—coded crutchare better left to the classroom situation where they can bepulled out when a) the students are in the mood for them, b)when the teacher can discuss the guessing strategies withthe students, and C) when there is time for feedback.Unfortunately none of these conditions do coincide withtraditional research which by its very nature is discrete,not global. Perhaps, though as humankind has managed toanalyze the universe with billions of discrete questions,almost simultaneously using sophisticated computers, maybe,as suggested above, we can bring the power of computers tobear on researching the doze procedure as it relates togoing beyond the sentence level. Maybe in the process andunder more favorable circumstances we can see if the color-219coded doze procedure is as good as on the surface it wouldappear to be.220BIBLIOGRAPHYAitken, K. G. Using doze procedure as an overall languageproficiency test. TESOL Quarterly, 1977, j,, 59-67.Aitken, K. G. Constructing a multiple-choice test of ESL.RELC Journal, 1978, (2), 105—108.Alderson, J. C. The doze procedure and proficiency inEnglish as a foreign language. TESOL Quarterly, 1979,(2), 219—227.Alderson, 3. C. Native and non—native speaker performanceon doze tests. Lariauage Learning, 1980, (1), 59—76.Anderson, 3. The use of readability formulae in theTerritory. Papua and New Guinea Journal of Education,1969,., 17—22.Anderson, 3. The application of doze procedure to Englishlearned as a foreign language in Papua and New Guinea.Encrlish Lanquage Teaching, 1973, 22, 66—72.Anderson, D. C. Cohesion and the doze test. Unpublished M.A. thesis, University of Illinois, 1980Anderson, 3. The application of doze procedure to Englishlearned as a foreign language in Papua and New Guinea.En1ish Lariauage Teachinc, 1973, 21, 66-72. (supportedexact scoring over alternative response method p. 71)Ashby-Davis, C. A. Levels of certitude for educatedguessing in strict doze passage. Journal of Reading,1984, 22, 96—108.Ashby-Davis, C. A. doze and comprehension: A qualitativeanalysis and critique. Journal of ReadincT, 1985, ,585—589.Bachman, L. P. The trait structure of doze test scores.TESOL Quarterly, 1982, i(1), 61—70.221Baclmian, L. F. Performance on doze tests with fixed ratioand rational deletions. TESOL Quarterly, 1985, (3),535—556.Bormuth, J. R. Comparable doze and multiple choicecomprehension test scores. Journal of Reading, 1967,291—99.Bormuth, J. R. The doze readability procedure. ElementaryEnglish, 1968, 4., 429—436.Borrnuth, J. R., Carr, J., Manning, J., & Pearson, P. D.Children’ s comprehension of between-and—within—sentencesyntactic structures. Journal of Educational Psychology,1970, i, 349 —357.Bridge, C. A, Thomas, S., & Evers, L. The effects oftextual cohesion on doze comprehension scores.1980, , 184—190.Bridge, C. A., & Winograd, P. N. Readers’ awareness ofcohesive relationships during doze comprehension.Journal of Reading Behavior, 1982, j(3), 299—312.Brutten, S. An analysis of student and teacher indicationsof vocabulary difficulty. RELC Journal, 1981, .(i), 66-71.Carroll, J. B. Defining language comprehension: Somespeculations. In J. B. Carroll & R. 0. Freedie, (Eds.),Lanauae comprehension and the acguisition of knowledge.New York: Halsted, 1972.Carver, R. P. Measuring prose difficulty using the Raudingscale. Reading Research Quarterly, 1975-1976, U(4),660—685.Chastain, K. Developing second-language skills: Theory topractice, 2nd edition. Chicago: McNally, 1976.Chihara, T., Oiler, J., Weaver, K., & Chavez-Oiler, M.A.Are doze items sensitive to constraints acrosssentences? Lanauaae Learning, 1977, aZ(l), 68—73.222Cohen, Andrew D. Teaching language ability. Rowley:Newbury House, 1980.Craker, H. U. Clozentropy procedures as an instrument formeasuring oral English competencies of first gradechildren. (Doctoral dissertation, University of NewMexico, 1971) Dissertation Abstracts International,1972, fl, 6286A—6287A. (University Microfilm No. 72—13,781)Cranney, A. The construction of two types of reading testsfor college students. Journal of Reading Behavior, 1972,, 60—64.Cunliffe, M. Reading programs using colour cues.Unpublished manuscript, U.B.C., 1986.Dale, N. On the teaching of Enlish reading. Edinburgh:Turnbull & Spears, 1926.Darnell, D. K. The relationship between sentence order andcomprehension. Speech Monoarah, 1963, ,Q, 97—100.Darnell, D. K. The development of an English languageproficiency test of foreign students using a clozentropyprocedure. Speech Monographs, 1960, fl, 36-46. (ERIC ED024 039)Dodds, W. G. Jr. A comparison of words in color andtraditional basal reading.Fleech, R. A new readability yardstick. Journal of AliedPsycho1ocy, 1948, 221—233.Foley, J. A. More questions on assumptions about dozetesting. RELC Journal, 1983, li(l), 57—68.Fries, C. C. Linauistics and reading. New York: MoltRinehart and Winston, 1963.Fry, E. Fry’s readability graph: Clarification, validity,and extensions to level 17. Journal of Reading, 1977,249.223Gattegno, C. Words in color: Background and orinciples.Chicago: Encyclopedia Britannica, 1964.Gattegno, C. Teaching foreign lanauaae in schools thesilent way. N.Y.: Educational Solutions, 1972.Goodman, K. S. Psycholinguistic universals in the readingprocess. In P. Pimsteur and T. Quinn (Eds.), gsycholoav of second lanuae learnina. Cambridge:Cambridge University Press, 1971, 135-142.Halliday, M.A.K. and Hasan, R. Cohesion in English.London: Longman, 1976.Hanzeii, V. E. doze tests in French as a foreign languageerror analysis. In E. J. Briere and F. B. Hinofotis,(Eds.), Concepts in language testing: Some recentstudies. Washington, D.C: TESOL, 1979, 3—11.Hanze].li, V. E. The effectiveness of doze tests inmeasuring the competence of students of French in anacademic setting. French Review, 1977,.Q, 865-874.Hay, J. & Wingo, C. E. Reading with phonics. N.Y.: 3. B.Lippincott, 1954.Henk, W. A. Effects of modified deletion strategies andscoring procedures on doze test performance. Journal ofReading Behavior, 1981, (4), 347—357.Herik, W. A., Helfeldt, 3. P., & Rinehart, S. D. Ametacognitive approach to estimating intersententialintegration in doze tests. , 1985, , 213—218.Hiller, Harry H. Canadian Society, A Macro Analysis.Scarborough, Ontario, University of Calgary, Prentice-Hall Canada Inc., 1991.Hinofotis, F. B. doze as an alternativie method of ESLplacement and proficiency testing. In J. W. Oiler, Jr.and K. Perkins, (Eds.), Research in lanauaae testing.Rowley, Mass: Newbury House, 121-128.224Hinofotis, F. B. and Snow, G. B. An alternative dozetesting procedure: multiple—choice format. In 3. W.Oiler, Jr. and K. , (Eds.), Research in lanagetesting. Rowley, Mass: Newbury House, 129-133.Hymes, D. H. On communicative competence. In 3. B. Pride &3 • Holmes (Eds.), Sociolinquistics. Harmondsworth:Penguin, 1974.Irvine, P., Atai, P., and Oiler, 3. W. doze, dictation andthe Test of English as a Foreign Language. LanauaaeLearning, 1974, 2.4, 245—252.Janssens, R. Stimulating motivation through audio—visualaids based on ‘English by Radiovision’. Enalish LanguageTeaching Journal, 1977, fl(l), 43—49.Jongsma, E. A. doze instructor research: A second look.Newark, Delaware: IRA, 1980.Judd, E. . vocabulary teaching and TESOL: A need forreevaluation of existing assumptions. TESOL Quarterly,1978, 14(1), 71—76.Jupp, T. C. & Hodlin, C. Industrial English: An examDle oftheory and practice in functional lanuae teaching.London: Heinmann, 1975.Kalivoda, T. Learning to listen what can be done. EnalishTeachina Forum, 1980, , 2-7.Katz, 3. and Fodor, 3. The structure of semantic theory.Language Learning, 1963, , 170—210.Kibby, N. W. Intersentential processes in readingcomprehension. Journal of Reading Behavior, 1980, ,299—312.Kirby, C. L. Using the doze procedure as a testingtechnique, reading diagnosis and evaluationInternational Readina Association Conference Proceedings,1968, (4), 68-77. Newark: International ReadingAssociation, 1968.225Klare, G. R. A second look at the validity of readabilityformulas. Journal of Readina Behavior, 1976, , 129—151.Leys, N., Fielding, L., Herman, P., & Pearson, P. D. Doesdoze measure intersentence comprehension. A modifiedreplication of Shanahan, Kamil, and Tobin. , 1983,.3., 111—114.Little, J. Canadian ESL readers for adults: An annotatedbibliography. TESL Talk, 1988, .2(l), 104—111.Long, N. N. A new look at discourse in the teaching ofreading. RELC Journal, 1981, 12(1), 72-77.Lord, F. N. and M.R. Novick. Statistical theories of mentaltest scores. Reading, MA Addison-Wesley.MacGinitie, W. Contextual constraint in English proseparagraphs. Journal of Psycholoqy, 1961, , 121—130.Martin, C. J. Comprehension of telegraphic prose 11. Effectof a grammatical scheme. , 1973, 2.2., 101—111.McHugh, W. Reaction to using colour in early readinginstruction, Decade of Innovations— Aoroaches toBeginnin Readinc, Newark: International ReadingAssociation, 1968.Mohan, B. Lanquage and Content. Reading, Mass: Addison—Wesley, 1986.Montety, P. de Silent teacher, talking pupils. EnalishTeaching Forum, 1977, 2, 11—14.Morley, J. Improvina aural comprehension. Ann Arbor:University of Michigan Press, 1972.Neville, N. H., & Pugh, A. K. Context in reading andlistening: variations in approach to doze tasks.Readina Research Quarterly, 1976 — 1977, , 13-31.226Nilsen, D. L. F. Contrastive semantic vocabularyinstruction. TESOL Quarterly, 1976, j, 99-104.Oiler, John W., Jr. Contrastive analysis, difficulty, andpredictability. Foreign Lanuae Annals, 1972, , 95-106.Oiler, John W., Jr. Scoring methods and difficulty levelsfor doze tests of ESL proficiency. Modern languageJournal, 1972, , 151—158.Oiler, John W., Jr. doze tests of second languageproficiency and what they measure. Language Learning,1973, 21, 105—118.Oiler, John W., Jr. Research with doze procedure inmeasuring the proficiency of non—native speakers ofEnglish: an annotated bibliography. Arlington, Va.: ERICClearinghouse of Languages and Linguistics, Center forApplied Linguistics, 1975. (ERIC ED 104 154)Oiler, John W., Jr. doze, discourse, and approximations toEnglish. In M. K. Burt & H. C. Dulay (Eds.), Newdirections in second language learning, teaching, andbilingual education: Selected papers from the NinthAnnual TESOL Convention. Washington, D. C.: TESOL, 1975.Oiler, John W., Jr. and Conrad, C. A. The doze techniqueand ESLJ proficiency. Language Learning, 1971, 2J, 183-196.Oller, J. W., Jr. & Perkins, K. (Eds.). Lanquage ineducation: Testing the tests. Rowley, Mass.: NewburyHouse, 1978.Oller, J. W., Jr. & Perkins, K. (Eds.). Research inlanauage testina. Rowley, Mass.: Newbury House, 1980.O’Reilly, R. P. & Streeter, R. E. Report on the developmentand validation of a system measuring literalcomprehension in a multiple—choice doze format:Preliminary factor analysis results. Journal of ReadingBehavior, 1977, (1), 45—69.227Pack, A. C. doze testing and procedure. TESL ReDorter,1973, , 1—2.Peterson, J., Paradis, E, & Peters, N. Revalidation of thedoze procedure as a measure of the instructional levelfor high school students. , 1973, , 144—149.Peterson, J., Peters, N., & Paradis, E. Validation of dozeprocedure as a measure of readability with High school,trade school, and college populations. 1972, ,45—50.Pikuiski, J.J., & Pikulski, E,C. doze, maze, and teacherjudgment. The Readinc Teacher, 1977, Q, 766-770.Propst, I. K. Jr., & Baldauf, R. B. Jr., A psycholinguisticrationale for measuring beginning ESL reading andmatching doze tests. RELC Journal, 1981, (l), 85-89.Publication manual of the American PsychologicalAssociation. Baltimore, Maryland: Garamond/PridemarkPress, 1979.Ramanauskas, S. The responsiveness of doze readabilitymeasures to linguistic variables operating over segmentsof text longer than a sentence. Reading ResearchQuarterly, 1972, (l), 72—91.Ramanauskas, S. Contextual constraints beyond a sentence ondoze responses of mentally retarded children. AmericanJournal of Mental Deficiency, 1972, fl, 338—345.Rankin, E. F. The doze procedure revisited. In P. L.Nacke (Ed.), Interaction: Research and vractice forcol1ee—adu1t reading, , 1974, , 1—8.Rarzkin, E. F. Characteristics of the doze procedure as aresearch tool in the study of language. ç, 1978,27,149—153.Raymond, P. CANread: Reading skills in ESL. Scarborough,Ontario: Center for Second Language Learning, Universityof Ottawa, Prentice-Hall Canada Inc., 1987.228Reutzel, D. R. dozing in on comprehension: The doze storymap. The Reading Teacher, 1986, 6, 524—528.Richards, D. Communicative interaction in the languagelaboratory. RELC Journal, 1980, 11(2), 64-76.Rivers, W. K. Teaching foreiqn—lanquage skills. Chicago:The University of Chicago Press, 1968.Rye, J. doze procedure and the teaching of reading.London: Heinemann Educational Books, 1982.Schwartz, R. K. and Stanovich, K. E. Flexibility in the useof graphic and contextual information by good and poorreaders. Journal of Reading Behavior, 1981, (3), 263-269.Shanahan, T., & Kamil, N. L. The sensitivity of doze topassage Organization. Zj, 1982, fl, 204—208. (In J.Niles & L. Harris (Eds.), New Inquiries in ReadingResearch and Instruction. Thirty-First Yearbook of theNational Reading Conference, 1982, 204 - 208.)Shanahan, T., Kamil, N L., and Tobin, A. W. doze as ameasure of intersentential comprehension. ReadingResearch Quarterly, 1982, )j, 229-255.Shanahan, T •, & Kamil, K. L. A further comparison ofsensitivity of doze and recall to passage organization.1jQ, 1983, , 123—128.Shanahan, T., & Kamil, K. L. The relationship of concurrentand construct validity of doze. ZjR, 1984, , 252—256.Schoenfeld, F.G. Instructional uses of the doze procedure.The Reading Teacher, 1980, 4., 147-151.Soudek, M and Soudek, L. I. New uses in language teaching.ELT Journal, 1983,.3.Z, 4, 335—339.229Streiff, V. Relationships among oral and written doze andwritten doze scores and achievement test scores in abilingual setting In J. W. Oiler & K. Perkins (Eds.),Lanauacie in education. Rowley, Mass: Newbury House,1978. 65—102.Stubbs, J. B. and Tucker, G. R. The doze test as a measureof English proficiency. Modern Lanauaae Journal, 1974,, 239—241.Stump, T. A. doze and dictation tasks as predictors ofintelligence and achievement scores. In J. W. Oller & K.Perkins (Eds.), Lanuae in education. Rowley, Mass:Newbury House, 1980.Taylor, W. L. doze procedure: a new tool for measuringreadability. Journalism Quarterly, 1953, , 415-433.Taylor, W. L. Recent developments in the use of dozeprocedure. Journalism Quarterly, 1956, , 42—48, 99.Thomas, S. Contextual constraints and the constraintvalidity of the doze procedure. 1980, 2, 47—55.Tuinman, J. J. Speculations on doze as a search procedure.1972, , 74—84.Valmont, W. J. Cloze deletion patterns: How deletions mademakes a big difference. The Readina Teacher, 1983,fl(2), 172—175.Weaver, W. & Kingston, A. A factor analysis of the dozeprocedure and other measures of reading and languageability. Journal of Communication, 1963, j3., 252—261.Ya]nada, J. The relationship between scrambled sentenceorder and doze difficulty among EFL students. RELCJournal, 1979, (l), 70—80.230APPENDICES FOR PILOT PROJECTAPPENDIX A:PILOT PASSAGESThe Original Texts with Deletions IndicatedPre-test: Terry FoxTest 2: The Black DonnellysTest 3: The Doctor (Norman Bethune)Test 4: The Leader, (Louis Riel)Test 5: Creatures of the Wild (Sasguatch)Test 6 InsulinPost-Test: Terry FoxPre—test: A Young Man’s DreamTerry Fox, born in Winnipeg, Manitoba on July 28, 1958,was Betty and Rolly Fox’s second child. The family of sixl)moved to Port Coguitlam, near 2)Vancouver when Terry wasseven. 3)At school, Terry was always 4)involved in sportsand began 5)cross-country running as early 6)as the eighthgrade. His 7)keen interest in all sports 8)led him to SimonFraser 9)University where he began studying lO)kinesiology,the study of human ll)movement.In November of 1976, 12)he was involved in a 13)carcrash which injured his 14)right knee. Never a complainer,231l5)he one day collapsed in l6)front of his mother becausel7)his pain was so intense. 18)At hospital he was diagnosedl9)as having osteogenic sarcoma, a 20)rare, malignant tumourthat develops 21)mostly in human males between 22)the agesof ten and 23)25. It is a bone 24)cancer that begins at the25)knee where it renders the 26)bone soft and mushy.Eventually 27)it breaks through the bone 28)to thesurrounding muscles, sending 29)cancer cells into thebloodstream 30)to be carried all over 3l)the body. Itscause is 32)unknown. Terry’s leg would have 33)to beamputated dust above 34)the right knee in order 35)to stopthe cancer.All 36)through elementary school, Terry’s teachers37)spoke of his drive and 38)determination, hisperseverance, tenacity, and 39)mental toughness. A harddriving, 40)gutsy guy who never gave 4l)up, Terry decided tolook 42)upon the loss of his 43)leg as a new challenge.44)He decided he could be 45)just as positive with one46)leg as he had been 47)with two. The night before48)his amputation, he formulated his 49)idea to run acrossCanada 50)to raise money for cancer research.Later, he underwent a series of gruelling chemotherapytreatments to destroy the cancer cells in his blood. Terrydid not complain.232Test 2: The Black DonnellysArrival in CanadaJames Donnelly arrived in Lucan, Ontario from hisnative Ireland in 1487. He was accompanied by l)his wifeJohannah and his 2)sons, James Jr. and William. 3)He hadmarried Johannah in 4)Ireland. She was a strange-5)lookingwoman-—extremely masculine, with 6) large hands and broadshoulders 7)and certainly no beauty. In 8)later years shegrew a 9)beard and even smoked a l0)pipe. Both Jim andJohannah ll)loved to fight. Any dispute, 12)no matter howinsignificant, was 13)good reason for a brawl. 14)Theyhated guns of any 15)kind and preferred clubs and 16)theirown fists.The Beginning of TroubleArriving in 17)Canada, where land grants were 18)easilyobtained, they preferred to 19)settle on 40 hectares of20)privately owned land. They simply 2l)took it over. Theywere 22)thus called “squatters”. In eight 23)years Johannahbore five more 24)sons-—John, Patrick, Michael, Robert25)and Thomas --and a daughter, 26)Jenny. After ten yearsthe 27)land changed hands and the 28)new owner, whose namewas 29)John Farrell, took James Donnelly 30)to court. Jameshad to 3l)surrender nearly 20 hectares of 32)land to233Farrell. From then 33)on there was constant battling.34)Farrell found his cows poisoned 35)and his barn burned.Moreover, 36)one day while sitting in 37)his kitchen,Farrell felt a 38)bullet go by. It could 39)be none otherthan one 40)of the Donnelly boys. They 41)were as black insin 42)as their father. Thus they 43)were called the blackDonnellys.Prison44)James was a hard drinker. 45)At a public gatheringone 46)day, he had too much 47)to drink. John Farrellquarrelled 48)with him. James hit Farrell 49)on the headwith an 50)iron bar. It took Farrell three days to die.James ran into the forest surrounding his home and hid therefor two years. The people of Lucan thought that he hadescaped.234Test 3: An Exceptional LeaderThe MetisLouis Riel grew up on the east side of the Red River inSt. Boniface. He belonged to the l)Metis--a culturallydistinct people 2)of Native Indian and French 3)ancestry.The Metis were semi—nomadic, 4)buffalo each year in June5)traveled West for their annual 6)buffalo hunt. It was the7)high point of the year 8)for them because it provided9)meat for the upcoming winter. lO)The buffalo hunt had all)certain military precision. There were l2)appointedcaptains, soldiers, and guides. l3)Each had a job tol4)perform, and all participants were l5)required to obeythe rules l6)of the hunt. No buffalo 17)could be hunted onthe l8)Sabbath day. Any participant who l9)did not followorders, who 20)lagged behind or advanced or 21)huntedwithout permission, was punished 22)because food for theentire 23)year was at stake. Afterwards, 24)all meat fromthe hunt 25)was shared equally among the 26)participants.Louis’ family, like most 27)Metis families, were ferventlyCatholic. 28)Louis excelled in school and 29)was chosen forreligious studies. 30)In 1858, when he was 31)13, he wassent East 32)to a Jesuit college in 33)Montreal to study forthe 34)priesthood. He remained in school 35)for six years.Then he 36)left and took a job 37)as a clerk in a 38)law235office. A year later, 39)he was back in Red 40)River,actively participating in the 41)Metis community.Trouble in the Red RiverAt this time, 42)Red River belonged to the 43)Hudson’sBay Company. The Hudson’s 44)Bay Company had decided to45)sell Red River to Canada. 46)No one asked the Metis47)and white settlers there if 48)they wanted to ioinCanada. 49The) Metis feared that their 50)long-establishedfarms would be divided up because of new land surveys. TheCanadian government sent William McDougall as the newlieutenant governor of Red River.Test 4: The People’s DoctorNorman Bethune was born on March 3, 1890, inGravenhurst, Ontario. He enrolled at the l)University ofToronto Medical School 2)in 1909. He took a 3)two-yearbreak from medical 4)school to work by day 5)at lumber campsand railroad 6)construction sites. It was here 7)that hefirst encountered the 8)harsh life of the workers. 9)Hereturned to medical school l0)and was qualified as al1)surgeon in 1916. He married l2)and moved to Detroit,Michigan 13)where he set up his 14)first medical practice.236In Detroit, 15)Bethune became acquainted with the16)enormous gap between the level 17)of health careavailable to l8)the rich and to the 19)poor. Most of hispatients 20)were barely able to pay 21)him.As his medical practice 22)was becoming securelyestablished, Bethune 23)developed tuberculosis. In 1920people 24)feared tuberculosis as they fear 25)cancer today.He was forced 26)to stop working and spent 27)a year at asanitorium, 28)recuperating. At this time, he 29)beganpainting. His despair during 30)this year is reflected in31)his paintings, for he was 32)sure he was dying. To33)make matters worse, his wife 34)divorced him.Once fully recovered, 35)Bethune went to Montreal in36) 1928 where he worked as 37)a tuberculosis (TB) specialistin 38)an English hospital. He conducted 39)his operationsat high speed 40)in order to minimize operating 41)time anddesigned several new 42)surgical instruments to improvesurgical 43)procedures. Thus he brought both 44)criticalskill and creative style 45)to surgery.Bethune then took 46) his work at a 47)Quebecoishospital. At this time, 48)French Canadian hospitals werepoorer 49)and less well equipped than 50)those servingEnglish Canada. Just as he had noticed the glaringdisparity between the rich and the poor of Detroit, he now237saw the difference between health care in French and EnglishCanada. And he was determined to do something about it.Test 5: Creature of the WildThe earliest references to the Sasquatch are found onthe carved totem poles and masks of the coast Indians ofBritish Columbia. These Indians say that l)the Sasguatch(meaning “wild man 2)of the woods”) is the 3)remnant of anancient race 4)which has managed to avoid 5)capture.However, more than 750 6)sightings of the beast or 7)itsfoot prints have been 8)reported in the past 100 9)years.These sightings have given 10)us a good idea of ll)what theSasguatch might look l2)like.In some ways the l3)Sasquatch seems to resemble humanl4beings A Sasguatch is approximately 15)360 centimetrestall, weighing between 16)270 and 360 kilograms. It 17)iscompletely covered with short l8)hair except for the palms19)of the hands and the 20)soles of the feet. Similarly,21)humans have no hair on 22)their palms or soles. The23)hair on the Sasguatch head 24)is longer than the hair25)on the rest of the 26)body. The Sasquatch face is27)black, and the eyes are 28)larger than a human’s. The29)Sasquatch head is said to 30)be larger at the back2383l)than at the front. Its 32)ears are comparable to human33)ears, its nose, broad and 34)flat. Unlike a human, the35) Sasquatch has a very short 36) neck. The arms are longer37)than a human’s and reach 38)below the knees. The hands39)are massive. The females have 40)large pendulousbreasts.Like a 4l)huinan, the Sasguatch walks upright 42)on twolegs, places its 43)heel down first, and swings 44)its armsas it walks. 45)It is extremely agile and 46)mobile for itshuge size. 47)The Sasquatch foot, which is 48)about 37centimetres long, differs 49)from a human foot in 50)thatits ankle bones are enlarged to support its great weight.Yet there is much flexibility in the toes.239Test 6: A Different WarBefore 1941 there were 23000 Japanese people in Canada--22000 in British Columbia and 1000 elsewhere. AlthoughWorld War II did l)not really touch Canada, the 2)Japanesepeople living there were 3)in a personal war with 4)theCanadian government. There were 5)no bombs and no guns6)but, all the same, they 7)lost everything--homes, )obs,families, 8)pride —-because the Canadian government 9)tookit all away from 10)them.The Japanese had been ll)coming to British Columbiasince 12)1877. They found work as 13)fisherinen andlumberjacks on the 14)west coast of British Columbia,15)sending later for their families. 16)Many of them livedin 17)Vancouver, on Powell Street. Working 18)long, hardhours, these Japanese l9)immigrants, or Issei (meaning“first 20)generation” in Japanese), were good 21)citizenswho valued family honor. 22)If one person in the 23)familydid something wrong, the 24)vhole family suffered. Therewas 25)very little crime among Japanese 26)families for theywould not 27)stand for it.The Issei 28)did not like to be 29)in debt. Often, ifthey 30)did not have enough money 31)to buy something, theywaited 32)until they had enough money 33)saved up. Theytook good 34)care of their elderly, refusing 35)to place240them in old 36)people’s homes to die, poor 37)and alone.They considered their 38)old people to be the 39)mostimportant members of the 40)Japanese family who could teach41)the young a lot about 42)life. The Xssei also looked43)after their own poor, saying 44)that it was up to 45)themto ensure that they 46)had enough to eat and 47)a place tostay.The 48)Issei preferred to live in 49)groups with otherJapanese. They 50)did not want to change to be likeCanadians. They wanted to keep their Japanese way of life,so they cut themselves off from other people.Test 7: The Discovery of InsulinLooking back, it must have seemed like a miracle.Banting was ‘ust 29 in l)that summer of 1921, a 2)surgeonnot long out of 3)medical school. Best was a 4)boy of 22,only a 5)recent graduate of arts. Their 6) researchlaboratory, grudgingly loaned for 7)three months by theUniversity 8)of Toronto, was dark and 9)humid. They atepoorly. They lO)were not paid. Calculate the ll)odds: twoinexperienced young men, l2)badly equipped, with 90 days13)to change the face of l4)medical history.241As the world 15)knows well what Sir Frederick l6)G..Banting and Dr. Charles 17)Herbert Best discovered thatsummer——l8)a crude extract of precious 19) insulin, achemical derived from 20)the pancreas and capable of21)controlling diabetes meflitus, a killer 22)disease as oldas China.23)The existence of insulin had 24)been suspected formore than 25)a decade and the attempt 26)to extract andisolate it 27)occupied researchers around the world. 28)ButBanting and Best, experimenting 29)with diseased pancreasesremoved from 30)diabetic dogs, were the first. 31)They madeone mongrel famous—— 32)Marorie; a shot of unpurified33)insulin roused her from a 34)coma and she lived for35)years. Six months later, on 36)January 11, 1922——afterfirst 37)verifying its safety with large 38)doses onthemselves--the first 39)human diabetic was given insulin:40)14-year-old Leonard Thompson 41) at Toronto Generalhospital. With 42)insulin, Thompson went on to 43)liveanother 12 years before 44)dying of causes unrelated to45) diabetesThe world at large 46)proclaimed them, although onlyBanting 47)and Dr. J.J.R. l4acleod, the physiologist 48)whohad reluctantly made the 49)lab space available and then50)left for a summer holiday, were cited for the 1923 Nobel242Prize in medicine. Banting graciously shared his award withBest.If Best was slighted at the lack of recognitionaccorded him, and vexed by Macleod’s tendency to want allthe credit for his own, he nursed his bitterness privately... .. ........ •...o................ ..........................zq3The Black Dc3nnellysArrival in CanadaJames Donnelly arrived in Lucan, Ontario from hisnative Ireland in 1487. He was accompanied by1)________________wife Johannah and his 2)________________James Jr. and William. 3)_______________had marriedJohannah in 4)_____________. She was a strange—5)____woman——extremely masculine, with6) hands and broad shoulders 7)____certainly no beauty. In 8)__ yearsshe grewa 9)- and even smoked a10)_. Both Jim and Johannah 11)to fight. Any dispute, 12) matter howinsignificant, was 13)_____reason for abrawl. 114)_ __hated guns of any15) and preferred clubs and 16)own fists.The Beginning of TroubleArriving in 17)_, where land grants were18) obtained, they preferred to19) on ‘40 hectares of 20)owned land. They simply 21)__ _ __it over.They were 22)__called squatters’. In eight23) Johannah bore five more24)——John,Patrick, Michael, RobertThomas ——and a daughter,After ten years the 27)the 28)_______________owner, whose______Farrell, took James Donnellycourt. James had to 31)____________of 32) to Farrell.there was constant battling.- found hi8 COWS poisoned 35)_____ _____Moreover, 36)_________day whilekitchen, Farrell felt ago by. It could 39)the Donnelly___ __as black in sintheir father. Thus they 143)called the black DonnellysPr i son45)he had too much 47)quarrelled 48)_was a hard drinker.a public gathering one ‘46)__drink. John FarrellJames hit Farrell50)____ _25)26)changed hands andname was 29)______30)nearly 20 hectaresFrom then 33)34)his barn burned.sitting in 37)_38)none other than one 40)boys. They 141)42)‘44)him.49)_________the head with anbar. It took Farrell three days to die. James raninto the forest surrounding his home and hid therefor two years. The people of Lucan thought thathe had escaped.An Exceptional LeaderThe MetisLouis Riel yre up on the east side of theRed River in St. Bonifacs. He belonged to the1)_____________——aculturally distinct people2)______ __Native Indian and French 3)________________The Metis ere semi—nomadic, 4)_____________each yearin June 5)___________West for their annual6)_hunt. It as the 7)____ __point of the year 8)_______them because itprovided 9) for the upcoming winter.10)_buffalo hunt had a 11)_ _military precision. There were 12>__captains..soldiers, and guides. 13)_________had a job to14) , and all participants vere15) to obey the rules 16)_thehunt. No buffalo 17) be hunted on the18)_day. Any participant NHo19) not follow orders., ho 20)behind or advanced or 21)_without permission.as punished 22>_______food for the entire23) as at stake. Afterwards,24)— meat from the hunt 25)shared equally among the 26)__. Louis’ family,like most 27) families, were ferventlyCatholic. 28) excelled in school and29)_____________chosen for religious studies.30)__ ___ __1858, when he was 31)________he was sent East 32)— a Jesuit collegein 33)_ _ ____to study for the34) . He remained in school 35)_____six years. Then he 36)______________and took ajob 37)_ _a clerk in a 38)_ ___office. A year later, 39)___________was back inRed ‘40) , actively participating in the41)__community.Trouble in the Red RiverAt this time, 42)_River belonged tothe 43) Bay Company. The Hudson’s44) Company had decided to 45)_ _—Red River to Canada. 46)_one asked theMetis 47)_ _white settlers there if48) wanted to join Canada. 49)_________Metis feared that their 50)—establishedfarmswould be divided up because of new land surveys.The Canadian government sent William McDougall as thenew lieutenant governor of Red River.27The People’s DoctorNorman Bethune was born on March 3, 1890, inGravenhurst, Ontario He enrolled at the 1)__________of Toronto Medical School 2)________________1909. Hetook a 3)________________—yearbreak from medical‘4)________________to work by day 5)_____________ __lumber camps and railroad 6) sites. Itwas here 7)______he first encountered the8)_life of the workers. 9)______________returned to medical school 10)_ _was qualifiedas a 11)_in 1916. He married12) moved to Detroit, Michigan13)he set up his 114)_medical practice. In Detroit. 15)_becameacquainted with the 16) — gap between the level17)health care available to 18)___ ___rich and to the 19)_________Most of hispatients 20)_________ barely able to pay21)s his medical practice 22)__ -becomingsecurely established, E1ethune 23) tuberculosis. In1920 people 2’4)___tuberculosis as they fear25) today. He was forced 26)_ ___ _stop working and spent 27) year at asanitorium, 28)__. At this time, he29)_painting. His despair during30) . f rj j r 3_______________________ca intl nos tor he 32)_______________he es ciy i ncjmatttre. vJorse his -i eHi m1. v re c:c’ver e ci • 35 ie r 1: to Hontr so II ::; a a I••i a o r k a3.;7 )___________t. be i c::u I. cs is T:B ) a r.:ce ate .1 is t:. :t38)__________imi ish hosp :1 He cordtc..tei39)___—.c: atic:ns at h:i. gh speed .) )_or cler to ml rLi.L ze operati nq 41. ) end deal qnedsever a I new 42)___ _________natru.ments to imorc::ive sur q ice].43) Thus he brou.qrit both 44j____ __ __1 1. 3. onc: creat. 1 VS St\/ Is: 4j OLtlthen toc:d his47 52.5. S .L r- t this t. I.48) Cane di an hc:s p1 to I a e re poorer—]as cii E1LIJF4tdno E.Enq i. :i. sh node J as h o ci not : c:o ci tp1 ar I n p d : a r.:: [ t.y be tw n t ‘i.c:: and t. he pc::o rt;etr c: :1. t he nc::w saw the di ffe enc:e set h i Lb c:a ren French anci Enpl i sh Cenecie ndhs was determ :1 nedcic:sc:meth :1 ng about it.zLt1Creature of the WildThe earliest references to the Sasquatch are foundon the carved totem poles and masks of the coastIndians of British Columbia,. These Indians say that1)_______________Sasquatch ( meaning ‘wild man2)______the woods’) is the 3)________________of an ancient race 14)________________has managed toavoid 5)_________. However, more than 7506)__ _of the beast or 7)________footprints have been 8)__________in the past 1009). These sightings have given10)_a good idea of 11)____theSasquatch might look 12)In some ways the 13)_seems toresemble human l’4)___. 1 Sasquatch isapproximately 15)__centimetres tall, weighing between16) and 360 kilograms. It 17)__completely covered with short 18)___except forthe palms 19)_the hands and the20) of the feet. Similarly, 21)_have no hair on 22) palms or soles.The 23) on the Sasquatch head214) longer than the hair 25)the rest of the 26>_The Sasquatch faceis 27) • and the eyes are28) than a human’s. The250head is sa i d to 30larger at the baoi.:. 31)__________ aL the frontIts ‘—— compo ab) a humi ts roan broad and 24)Un ii k a a human the .3) nsa. a varyshort The arms arc: :I.c:nqer_____________________a hi..rransanc:she knees The hanis 29)mass i T he toma 1 as. have 40) pendu :L QuabreastsLike a 4.!. • the Oz:c!:;..I..ch walks nor ciht42) to ieqs pisces its 4%)dc:wn +i rat and swi nqs 44)————.— arms as it4%) is a::<trema I y aqi La and4é)_for its huqe siza 47)___________%as.qusl:c::h foc:t ii ci .s 18 •___________. 27 •••••t elDng d:i.fsrs 49)____a huiran foot ini ts an k 1. a bores are en L ar qed tasuqpc:r t I is qreat wc I riOt Yet there :1. a mucri f lexi bi I :in the _c.€.o;251t Different WarBefore 19141 there were 23000 Japanese people inCanada ——22000 in British Columbia and 1000 elsewhere.lthough World War II did 1)_______________really touchCanada, the 2)______________people living there were3)_____________a personal war with 14)______________Canadian government. There were 5)__ _bombs andno guns 6)____ _______, all the same, they7) everything——homes, Jobs, families,8)__-—because the Canadian government9) it all away from 10)_The Japanese had been 11)-to BritishColumbia since 12)_____. They found work as13) and lumberjacks on the 114)____coast of British Columbia, 15) later fortheir families. 16) of them lived in17)__, on Powell Street. Working18) , hard hours, these Japanese 19)_______or lsssi ( meaning “first 20)____inJapanese ) , were good 21)__who valuedfamily honour. 22) one person in the23) did something wrong, the 214)family suffered. There was 25) little crimeamong Japanese 26) for they would not27) for it.The Issei 28)_______not like to be29)_______________debt. Often, if they 30)____—_____not have enough money 31)________________buy something,they waited 32)_they had enough money33)___up. They took good 34)___ ___ __of their elderly, refusing 35)___place themin old 36)________homes to die, poor37)— alone. They considered their 38)___people to be the 39)__important members ofthe ‘40) family who could teach41)______young a lot about 42)_. TheIssei also looked 143)____their own poor,saying 44)_it was up to‘45)___to ensure that they 46)__ ___enough to eat and 47)_place to stay.The ‘48) preferred to live in149) with other Japanese. They50) not want to change to be likeCanadians. They wanted to keep their Japanese way oflife, so they cut themselves off from other people.23The Discovery of InsulinLooking back, it must have seemed like a miracle.Banting was just 29 in l)_______________ summer of1921, a 2)_____________—not long out of3)________school. Best was a 4)_____________of 22. only a 5)__________graduate of arts.Their 6)______laboratory, grudgingly loaned for7)______months by the University 8)__________Toronto, was dark and 9)_____________. They atepoorly. They 10)—not paid. Calculate the11) : two inexperienced young men,12)_equipped, with 90 days 13)change the face of 14)___history.As the world 15) well what SirFrederick 16)__. Banting and Dr. Charles17) Best discovered that summer——18)__crude extract of precious19), a chemical derived from 20)__pancreas and capable of 2l)_,_ diabetes mellitus,a killer 22) as old as China.23) existence of insulin had24)_suspected for more than 25>decade and the attempt 26)______ _____extract and2.53 Cs.:i sc3:L.c.e i : Z) —c:r :rr s .rs:’:1 1:h ur i. d_________Dflt.L1 C Snd d25t e;<1er inenti nqdiseased panc:reesec-:. removed -romdo cis.,rc the -F :i. r s t,do one no nn .L famous-— ..2)_________—a a hot ofunpur if-i ed rc..sod - aand aiio 1 ivacJ fr SSi::< munt:ho iatE•:r. on ... . .11. L*:S::.—t?cv rat.its i th iarqean t: riocna.o V55” t [35 t 1 r st.________ ________________di. as I a 3. VOO 3. nsu 1. i n 4(1)——\/e r I ciLoorsar ci 1 L3 c r 41____ _Toronto henoral riospi telWith 42) • Thom3scjn went on to43) . . . . another 1 year ._S::c____.i. S-:: causes unrelated to $ñrI—E. Rit!if ares 45)onl’’ Ianti rq Dr 33. P. Tiac:leod •c:: 1. 1 at 48.) had re.L uc tent :t made theava lable end then f:f)a. saiio r ho 1:1 iiN/ • we r a a i. j:j .f:c_... the INc:ib5 I P .i. ;:‘ei n mc cii na I3e n t i np qr a c: i ous.l. V SPr ad h : aSOSIf Dr-si: a. ii cjhter .: the I ask reren I Li oncc:c:r dad hi in end sexed [35 Meal sod a tenc:ienc:v to wa itall t hec:r ci f :.r hi a sw n he nor as c: c— 151:3 i 1: tar ness pr I vats 1.TheBlackDonnellys !HHH!!!rrival in CanadaJames Donnelly arrived in Lucan, Ontario from hisnative Ireland in 1467. He was accompanied by1) JL1i w i fe Joha nnah and hi s 2)__.5.Qfl5_James Jr. and William. 3)______ had marriedJohannah in 4)_Xff]4$ji_. She was a strange—5)Jo4cJ woman——extremely masculine, with6)j0Jb hands and broad shoulders 7)____fl_certainly no beauty. In 8)_________yearsshe grewa 9)b€&r__ and even smoked a. Both Jim and Johannahto fight. Pny dispute, l2)__fl______ matter howinsignificant, was 13)__—— reason for abrawl. 14) hated guns of any15)<Ij and preferred clubs and____own fists.The Beginning of Troublerriving in 17)_CO.flê&....., where land grani wereobtained, they preferred toon 40 hectares of 2O)ft1.VDiflowned land. They simply 21).hfIk — it over.They were 22)__j___ called “squatters’. In eight23)_Q_L___ Johannah bore five more24)_jê5___——John, Patrick, Michael, Robert25)____ Thomas ——and a daughter,26)_ny___. fter ten years thechanged hands and the 28)__fleIA____ owner, whosename was 29)_..Ah___ Farrell, took James Donnelly30)______ court. James had to 3l)_.$fV!eynearly 20 hectares of 32) IOF%ti to Farrell.From then 33___tL there was constant battling.34)1O!.tCLII__ found his cows poisoned 35)_fl.c[his barn burned. Moreover, 3S)On..e day whilesitting in 37)_j —— kitchen, Farrell felt a38)_L!?JJ€.±’ go by. I t cou 1 d __b..enone other than one 40)______ the Donnellyboys. They 4l)jAJ€f. as black in sin_their father. Thus theycalled the black Donnellys.Pr i son44)__.f____ as a hard drinker.45)_ a public gathering one 4b)__y____he had too much 47)___O — drink. John Farrellquarrelled 48)__J _ him. James hit Farrell49)_____ark.the head with anbar. It took Farrell three days to die. James raninto the -Forest surrounding hi home and hid therefor two years. The people of Lucan thought thathe had escaped.255The Cloze Passages and the Distribution of the Deleted Words.1. Terry Fox(ord = order of words in the passage)ord word location bef bef aft aft1A2 Young3 Man’s4 Dream5 Terry6 Fox,7 born8 in9 Winnipeg,10 Manitoba11 on12 July13 28,14 1958,15 was16 Betty17 and18 Rolly19 Fox’s20 second21 child.22 The23 family24 of25 six26moved 1: 000027 to28 Port29 Coquitlam,30 near31 Vancouver 1: o 0 0 032 when33 Terry34 was35 seven.36 At 3:: 144,(111) 0 0 0 1(137 school,38 Terry39 was40 always41 Involved 2: 83 0 0 0 142 in43 sports44 and25645 began46cross- 1: 000 047 country48 running49 as50 early51 as 6:: 49, 237,247,252,(1 16) 0 1 0 3(152 the53 eighth54 grade.55 His56keen 1: 000057 interest58 in59 all60 sports6lIed 1: oooo62 him63 to64 Simon65 Fraser66 University 1: o 0 0 067 where68 he69 began70 studying71 kinesiology, 1: 0 0 0 072 the73 study74 of75 human76 movement. 1: 0 0 0 077 In78 November79 of80 1976,81 he 9:::68, 113,243,253,263,278,(96,241 1 0 0 5(282 was83 involved84 in85a86car 1: oooo87 crash88 which89 injured90 his91 right 2: 192 0 0 0 192 knee.93 Never94a95 complainer,96 he 9:::68, 113,243,253,263,278,(96,241 1 0 0 5(297 one98 day25799 collapsed100 inlOifront 1: 0 0 0 0102 of103 his104 mother105 because106 his 10::55,90, 103, 208,212,235,265,292, (261 2 1 0 5(1107 pain108 was109 so110 intense.111 At 3:: 144,(36) (1) 0 0 1112 hospital113 he114 was115 diagnosedll6as 6::49, 237,247,252(51) 1(1 0 0 3117 having118 osteogenic119 sarcoma,120al2lrare, 1: 0 0 0 0122 malignant123 tumour124 that125 developsl26mostly 1: 0 0 0 0127 In128 human129 males130 between131 the 15:::22,52,72/145,150,159,162,169,198,232,25 3 0 0 9(2132 ages133 of134 ten135 and13625. 1: 0 0 0 0137 It138 is139a140 bone141 cancer 5:: 199,275,289,(166) 0 0 0 3(1142 that143 begins144 at145 the146 knee 3:92, 193 1 0 0 1147 where148 it149 renders150 the151 bone 3: 140, 160 0 1 0 1152 soft258153 and154 mushy.155 Eventually156it 3: 137,148 2 0 0 0157 breaks158 through159 the160 bonel6lto 3:27,63 2 0 0 0162 the163 surrounding164 muscles,165 sending166 cancer 5:: 199,275,289,(141) 0 0 0 3(1167 cells168 into169 the170 bloodstream171 to 7:::: 229,267,286(186,196,271) 0 0 0 3(3172 be173 carried174 all175 over176 the 1 5:::22,52,72, 145,150/159,162,169/198,232,25 5(1 3 0 4(1177 body.178 Its179 cause180 is181 unknown. 1: 0 0 0 0182 Terry’s183 leg184 would185 have186to 7:::: 229,267,286,(171,196,271) (1 0 1 3(1187 be188 amputated189 just190 above191 the 15:::22,52,72,145,150,159,162,169/198/232,25 8(2 0 1 3192 right193 knee194 in195 order196to 7:::: 229,267,286,(171,186,271) (1 (1 0 3(1197 stop198 the199 cancer.200M201 through 2: 158 1 0 0 0202 elementary203 school,204 Terry’s205 teachers2o6spoke 1: 0 0 0 0259207 of208 his209 drive210 and211 determination 1: 0 0 0 0212 his213 perseverance214 tenacity,215 and216 mental 1: 0 0 0 0217 toughness.218A219 hard-220 driving,22lgutsy 1: 0 0 0 0222 guy223 who224 never225 gave226up, 1: 0 0 0 0227 Terry228 decided229 to230 look23lupon 1: 0 00 0232 the233 loss234 of235 his236 leg 3:: 183, (251) 1 0 0 (1237 as238 a239 new240 challenge.241 He 9::: 68,113,243, 253,263,278,(81,96 2(2 0 1 3242 decided243 he244 could245 be246 just 2:189* 0 0 0 1247 as248 positive249 with250 one251 leg 3:: 183,(236) 1(1 0 0 0252 as253 he254 had255 been256with 2: 249 0 1 0 0257 two.258 The259 night260 before260261 his 10::55,90,103,208,212,235/265/292,(106) 6(1 1 0 1262 amputation,263 he264 formulated265 his2661dea 1: 0 0 0 0267 to268 run269 across270 Canada271 to 7::::229, 267, 286,(171,186,196) 1 1 0 1(3272 raise273 money274 for275 cancer276 research.277 Later,278 he279 underwent280 a281 series282 of283 grueling284 chemotherapy285 treatments286 to287 destroy288 the289 cancer290 cells291 in292 his293 blood.294 Terry295 did296 not297 complain.261An Analysis of Same-Word Re-Iterative CluesTABLE AAN ANALYSIS OF RE-ITERATIVE CLUES:LOCATION AND WEIGHTED VALUES FOR SCORING THE CLOZEPROCEDURE3. An Exceptional Leader: (50 deletions, 299 word passage)no. ofclue words...BEFORE= AFTER ....sentence of doze blankB W:W B nit: (clue word) not in passagee i:i e bey: (clue word) beyond sentencey t:t y bed: (different clue) beyond sentenceo h:h o win : (due word) within sentencen i:i n wid: (different clue) within sentenced n:n dSCORES FOR CLUE LOCATIONNOUNS and PRONOUNS: VERBS1) Metis 1 00 3(2)= 7/6/5 bey 5) traveled— = 10/9/8 nit3) ancestry — = 10/9/8 nit 14) perform — = 10/9/8 nit6) buffalo 0002 = 7/6/5 bey 15) required— = 10/9/8 nit9) meat 000 1 = 7/6/5 bey 17) could — = 10/9/8 nit13) Each— 10/9/8nit 19)did — = 10/9/8nit18) Sabbath — = 10/9/8 nit 20) lagged — = 10/9/8 nit23)year 2002 = 7/6/5bey2l)huntedldood = 7/6/Sbey26) participants2 00 d = 7/6/5 bey 25) was 3004(1) = 7/6/5 bey28) Louis 2d0 00 = 7/6/5 bey 29) was 4003(1) = 7/6/5 bey33) Montreal — = 10/9/8 nit 36) left — = 10/9/8 nit34) priesthood — = 10/9/8 nit 45) sell— 10/9/8 nit38) law — = 10/9/8 nit = = = = = = = = = = = = = = = = = = = = = =39) he 5000 = 7/6/5 bey 11 verbs/50 deletions40) River 1 004 = 7/6/5 bey = 22%42) Red 3002 = 7/6/5 bey43) Hudson’s 000 1 = 7/6/5 bey44) Bay 1 000 = 7/6/5 bey48) they — = 10/9/8 nit18 nouns/50 deletions= 36%262ADJECTh’ES ADVERBS7) high— = 10/9/8 nit10)the xOOx = 7/6/5bey11) certain — = 10/9/8 nit 0 adverbs/SO deletions12) appointed — = 10/9/8 nit = 0%24) all 1000 = 7/6/5bey27)Metis 2(1)002(1)= 7/6/5bey31)13— =10/9/8nit41)Metis 4002 = 7/6/5bey46) No 1 000 = 7/6/5 bey49) the x 001 = 7/6/5 bey50) long — = 10/9/8 nit11 adjectives/SO deletions= 22%NIT BEY(D) WIN(D)OTHER NOUNS +PRONS = 36%> 14% 22% 0%2) of 1 003(1) = 7/6/5 bey4) and 1 005 = 7/6/5 bey VERBS = 22%> 16% 6% 0%8)for 1013(1)= 7/6/5bey16) of 2(1>002 = 7/6/5 bey ADJS = 22%> 10% 12% 0%22) because 1 00 1 = 7/6/5 bey30) In 30 1 5 = 5/4/4 win ADVS = 0%> 0% 0% 0%32)to 1002 = 7/6/5bey35) for 5(1)000 = 7/6/5 bey OTHER = 20%> 0% 18% 2%37) as 000 1 = 7/6/5 bey = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =47) and 4(1)000 = 7/6/5 bey all = 100% ) 40% 58% 2%10 other/SO deletions=20%263APPENDIX B:PRODUCTION OF THE PILOT CLOZE PASSAGE FORMSThe following procedure sets out a reasonably efficient wayto make the various modifications of the doze passages inthe present research. As technology improves the processshould become even easier.TABLE BMETHOD OF COLOR-CODING AND PRINTING PASSAGES FOR FIVECLASSESHow to make the standard doze procedure Format for dozepassages 2. — 8.1. Create a file and call it by the name of the dozepassage, e.g. “Fox”.2. Type out the doze passage using a word processingprogram, entitling it the name of the passage, e.g.Fox Passage.3. Make a copy of the passage within the file and labelthe copied portion (Fox)*blank.4. Using (Fox)*blank,a) put an * before every word which is to be deleted;b) then go to the search and replace function andreplace the * with a 15 space blank;C) press “all” and all the *s will be replaced with a15 space line;d) save the amended file.2645. Make a copy of the completed (Fox)*blank and call thenew portion (Fox)Form—l.6. Using (Fox)Form-l,a) delete one by one the words immediately after theblank;b) then start at the first blank and number it twospaces before as “1)”;c) continue numbering the subsequent blanks “2)”,“3)”, etc. (This will give you the standard dozeprocedure Format);.d) save the amended file. ((Fox)Form-l is now ready tobe printed.)B. How to make Form 4 (the standard doze Format with color-coded doze blanks and words) for passages 2 — 7.1. Within the file, copy (Fox)Form-l. and call the copy(Fox)—Form 3/4.2. Using (Fox)Foriu-3/4,a) before each word or blank designate a coloraccording to the part of speech represented by thatword or blank;b) (Color designations made with a Star color-printerrequire a printer code to be placed before theintended color change.) using the appropriatecoding from Table B.l below, type the appropriateabbreviated symbol before each word and blank.265TABLE B.lMEANINGS OF THE SYMBOLS USED FOR CODINGPart of Speech Color Abbreviated Printer CodeSymbolNOUN blue ((C))2.....VERB red ((C))lADJECTIVE green $ ((C))6ADVERB violet ((C))3OTHER black A ((C))OC) save the file.3. To make Form 4, copy (Fox)Form-3/4 and call the copy(Fox) Form-4.4. Using (Fox)Form—4,a) use the search—and-replace function to replace eachabbreviated symbol with the printer code necessaryfor the printer to color the words and blanks;b) when all the printer codes are in place, save thefile. (Passage (Fox)Form-4 is now ready to beprinted on the Star color—printer.)D. How to make Form 3 (the standard doze procedure with allblanks and words colored randomly).1. Make another copy within the file of (Fox)Form-3/4,before each word or blank randomly designate a colorusing the codes for the Star color printer.2. Call the new copy (Fox)Form-3.E. How to make Form 2 (the standard doze with all blankscolor coded) for passages 2- 7.266i. copy (Fox)Form-3/4 and call the new copy (Fox)Form-22. In order to set up the printer codes to color theblanks,a) after each abbreviated symbol for the blanks inserta “b”;b) then go to the search—and—replace function andreplace each (symbol)b with the appropriate printercode as in the code table above;c) save the file.3. Next to remove the remaining unnecessary abbreviatedsymbols,a) go to the search-and-replace function and replaceeach symbol with a “b) save the file. (Passage (Fox)Form-2 is now readyfor the Star color—printer.)F. How to print the four variations of the standard dozepassages for passages 2 —7.1. doze passage Form 1 can be printed on any IBMcompatible printer.2. Forms 2, 3 and 4 can be printed on the Star NX1000R,NX1000CL, or LC1OCL color printers.U’)t268G. The number of copies of the Forms to be made.Form 4 = 25 copies/passage all color codedForm 3 = 25 copies/passage all randomly coloredForm 2 = 25 copies/passage blanks coloredForm 1 = 25 copies/passage nothing coloredSubtotal = 100 copies/passagesx 6 passages (passages 2 to 7)Subtotal = 600 copies+ 200 copieEd(lOO Form 1 pe1&.ett)Total = 800 copies for 100 students269APPENDIX C:DISTRIBUTION OF THE RESEARCH TASKSThe Role of the ResearcherTABLE CROLE OF THE RESEARCHER1. Before day 1— the researcher will prepare the tests, the the testbooklets, and the pencils. He will also orient theteachers as to the duties during the test sessions.2. Day 1:a) before the class the researcher will give the testbooklets and the pencils to the teachers.b) after the test the researcher will collect the testbooklets and pencils.c) then the researcher will type test answers into thedata base score sheet for each student, calculate thescores for each blank (both exact and weighted), addthe scores to get totals, rank the students accordingto the exact score, assign the students to theirappropriate group and rank within the group, andchoose the passage form accordingly.d) then the researcher will take out test one from thestudents’ booklets and replace with the assigned formof test 2.270Day 2:a) before the class the researcher will give the testbooklets and the pencils to the teachers.b) after the test the researcher will collect the testbooklets and pencils.C) then the researcher will type test answers into thedata base score sheet, calculate scores for eachstudent.d) next the researcher will take out test one from thestudents’ booklets and replace with the assigned formof test 3.4. Day 3 through day 7:a) the researcher will give to the teacher 20 copies ofthe undeleted passage of the previous testb) the researcher will repeat the same routine as for day2, except he will go to the subsequent test.5. Day 8:a) before the class the researcher will give the teacherthe test booklets.b) after the class the researcher will thank the studentsand teacher.C) then he will collect the booklets, questionnaires andpencils.2716. Finale:a) the researcher will take the data and calculate itall, analyze it, and draw conclusions.b) the researcher will meet with the teachers and discussthe data and conclusions.272Instructions for TeachersTABLE DINSTRUCTIONS FOR TEACHERSADMINISTERING THE CLOZE RESEARCHI • GENERAL STATEMENTThis research requires eight sessions, two per week.Each session should take about 45 minutes to administer theday’s doze passage, which includes 30 minutes for the testand 15 minutes for instructions, distribution, andcollection of the tests. The task of the teacher is to handout pencils and test booklets, to read the instructions tothe students given in their booklets, watch the students tosee they are doing their own work, to keep the studentsinformed of the remaining time (every five minutes), tocollect the booklets and pencils, and after the third dozetest onward to hand out the previous sessions undeletedpassage which after five minutes is to be collected again.II. DETAILED STATEMENTDay 1:a) Please explain to the students that they are going tohelp in an E.S.L. research project, that their workwill be anonymous, and that they must treat theresearch as an official test and act according to273standard test behavior, i.e. no collaboration withother students.b) Tell the students that there will be eight tests, foreach of which they will be given half an hour. Alsotell them that if they can get 20 blanks correct outof 50, then they are doing reasonably well.C) Hand out the pencils.d) Hand out the booklets, telling the students to keepthem closed until they are asked to turn to theinstructions and suggestions on page 1.e) When all the students have a test booklet tell thestudents to write a three-figure number plus theinitials of their mother or father on the front oftheir booklet. Tell them to keep a copy of the numberand initials in their purse, wallet, etc. for the nextsessions when the booklets will be returned to them.f) Read the instructions to the students as they silentlyread along and make sure they understand them. (Tryto remember their questions for subsequent recording.)g) Read the guessing strategy to them as they read alongsilently. Answer any questions they have about thestrategy. (Try to remember their questions forsubsequent recording.)h) Tell the students to open their booklets and to begin.Remind them they have 30 minutes to do the best theycan on the test and that 20 blanks correct out of 50is a reasonable score.274i) Watch the students to see they are doing their ownwork. Do not allow them to use their dictionaries.j) While they are doing the test,i) write down on the back of this sheet thequestions the students asked you about theinstructions and suggestions,ii) write down the names of the students that areabsent, (or their number-initial code), andiii) write on the board every five minutes the amountof time left for the test.k) After 30 minutes tell the students to put theirpencils down and to close their booklets.1) Collect the booklets and pencils and remind thestudents to remember their number—initial code.m) Thank the students for participating in the day’s testand continue with your regular lesson.n) Keep the pencils and booklets for the researcher whowill pick them up on the same day.0) When the researcher comes, ask any questions and giveany suggestions or concerns you might have about thetest and give him the booklets and pencils.Day 2:p) Please follow steps c, d, and f through o. Beforethis session the researcher will give you the275students’ booklets. Display them on a table and askthe students one by one to tell you quietly his/hernumber-initial code and then hand him/her thecorresponding booklet from the table.Day 3:q) Please follow steps c through 1.r) Hand out copies of the undeleted doze passage fromthe previous day.s) Let students look at the undeleted passage for aboutfive minutes. Remind them that if they got 20 blanksright out of 50, then they are doing reasonably well.t) Collect all the undeleted passages and keep them forthe researcher.U) Follow steps m through o.Day 4 through 7:v) Please follow the routine of day 3.Day 8w) Please follow steps c through 1. (Do not hand out theundeleted passage from day 7.)x) Hand out a questionnaire to each student.276y) Give the students about 10 minutes to fill them outand then collect them.z) Follow steps m through o to complete your part in theresearch project.Thank you very much for your co—operation. You will receivea summation of the results when this thesis is completed.277Instructions for StudentsTABLE EINSTRUCTIONS FOR STUDENTSThank you for participating in this research project.Today you are going to have one of eight doze tests thatwill be given to you in a period of four weeks. Your taskis to fill in the blanks with the best word you can thinkof. Do your best. If you can get 20 blanks correct out of50, then you will have done reasonably well.A. GENERAL INSTRUCTIONS:1. Please do not put your name on the booklet. Instead,think of any three figure number, eq. 123 and write itdown in the upper left hand corner on the front ofthis booklets cover. Before your number write theinitials of your father or mother, eg, FGS for FrankGeorge Smith. The example looks like this: FGS 123.This method will identify your booklet, but willprotect your identity.Please make a note of your identity code as youwill need it for the other tests in this researchproj edt.2. For this doze test you will have 30 minutes. Everyten minutes your teacher will write on the board howmuch time there is left for you to complete the test.2783. As this test is for research purposes it is importantthat you do your own work. Remember that the testwill not be counted for your own personal grade, so donot worry if you find some parts of the testdifficult.4. In this test and all the doze tests try to do yourbest. The harder you try, the more benefit you willget from your effort.B. SPECIFIC INSTRUCTIONS:1. With your pencil fill in as many blanks as you canwith the best answer you can think of.2. Follow these suggestions.a) read the first and last sentences of theuruuutilated text to determine the gist (main idea)of the passage.b) then skim the mutilated text trying to get clues tothe gist.C) read from the beginning to the end of the text,trying to find meanings for the omitted words bychecking context clues before or after the omitted279words or from general knowledge related to thetext.d) Finally, reread the entire text when you haveguessed all or most of the words, filling in wordspreviously not guessed or correcting words alreadyguessed in terms of the total text.280TABLE FSPECIAL INSTRUCTIONS FOR STUDENTS USING COMPLETELY-COLOR-CODED PASSAGES (FORM 4) AND COLOR-CODED BLANKS (FORM 2)SPECIAL INSTRUCTIONS:(Teacher cannot read any of the special instructions.)(This is for the students who are practicing with the colorcoding.)The following passage is color coded to help you tofind clues to be able to fill in the blanks. The colorcoding means as follows:BLUE = Nouns: student; Bob; Tokyo; school; science.... = Pronouns: I, he; me, her...RED = Verbs: go, study, think; went; will study...GREEN = Adjectives: big; my, his; this; a, the; no...PURPLE = Adverbs: quickly; today, then; there; very...BLACK = Other: and, or; in, on; where, how; not...281APPENDIX D:SPREADSHEET CHART FOR DATA COLLECTION AND DETERMINATION OFSCORESFor Collection of Answers an Determination of ScoresTABLE GANSWER COLLECTION AND BLANK SCORE DETERMINATION CHARTPage 1TOTAL PRETEST TEST 2 TOTALMEAN EXACT MTH MEAN EXACT MTHMEAN ACCEP 14TH MEAN ACCEP 14THPage 1SCORES FOR BLANKSPOINTS FOR.. ....... EXACT WORD Exact Mth =EXPLANATION. ....... CLUE LOCATION same wrd : diff wrdPOINTS FOR Tl:not in passage T:1O/9/8TYPES OF CLUE...... T2: beyond senten T:7/6/5 : 8/7/6T3: within senten T:5/4/4 : 6/5/5doze Form 4 lA4a: Student 1 Class A Group 4 Level a(everything 2A4b: Student 2 Class A Group 4 Level bcolor coded) 3A4c: Student 3 Class A Group 4 Level c4A4d: Student 4 Class A Group 4 Level d5A4e: Student 5 Class A Group 4 Level eMEAN EXACT 14TH282MEAN ACCEP MTHdoze Form 3 6A3a: Student 6 Class A Group 3 Level a(everything. ...... . 7A3b: Student 7 Class A Group 3 Level bcolor coded 8A3c: Student B Class A Group 3 Level crandomly) 9A3d: Student 9 Class A Group 3 Level dlOA3e: StudentlO Class A Group 3 Level eMEAN EXACT MTHMEAN ACCEP MTHdoze Form 2 llA2a(blanks l2A2bcolor coded) l3A2cl4A2dl5A2eMEAN EXACT MTHMEAN ACCEP MTHCloze Form 1 l6Ala(Standard l7Albdoze) . . ... . ... . .l8Alcl9AldV 2OAleMEAN EXACT MTHMEAN ACCEP MTH283Cloze Form 4 21B4a22 B4 b23B4c24B4d25B4eMEAN EXACT MTHMEAN ACCEP 14THdoze Form 3 26B3a27B3b28B3c29B3d3OB3eMEAN EXACT 14THMEAN ACCEP MTHCloze Form 2 31B2a32B2b33B2c34 B2 d35B2eMEAN EXACT 14THMEAN ACCEP MTH“Si285doze Form 1 36B1a37B1b38B1c39B1d4OBleMEAN EXACT MTHMEAN ACCEP MTHdoze Form 4 41C4a42C4b43C4c44C4d45C4eMEAN EXACT MTHMEAN ACCEP MTHdoze Form 3 46C3a47C3b48C3c49C3d5OC3eMEAN EXACT MTHMEAN ACCEP MTH286Cloze Form 2 51C2a52C2b53C2c54C2d55C2eMEAN EXACT MTHMEAN ACCEP MTHdoze Form 1 56C1a57 Clb5BClc59Cld6OCleMEAN EXACT MTHMEAN ACCEP MTHdoze Form 4 6OD4a61D4b62 D4 C63D4d64D4eMEAN EXACT MTHMEAN ACCEP MTH287doze Form 3 65D3a66D3b67D3c68D3d69D3eMEAN EXACT MTHMEAN ACCEP MTHCloze Form 2 7OD2a71D2b72 D2 c73D2d74 D2 eMEAN EXACT 14THMEAN ACCEP MTHand so on tolOOEleMEAN EXACT 14THMEAN ACCEP MTHpage 1BLANK1 B2 B3his 0004 son 000(1) He 10002same same sameT: 10/9/8T:7/6/5 T:7/6/5288APPENDIX E:THE MAIN PROJECT PASSAGESThe Main Project PassagesPre—test: PerspectivesTest 2: The Question of SocietyTest 3: EthnicityTest 4: UniquenessTest 5:... PolityTest 6: IdentityPost-test: PerspectivesThe Original TextsPre-test: PerspectivesThere are three dominant perspectives for understandinga society.The first perspective attempts to determine how asociety holds together as an entity. This perspective isknown as functionalism because whatever aspect of society isanalyzed, the object is to determine how that structuralfeature contributes to the integration of the society. Forexample, broadcasting can be assessed in terms of how itcontributes to national unity, or conversely, disunity. DoCanadian radio stations give priority to recordings byCanadian artists so that Canadian culture is developed, ordoes the American music industry dominate? Is hockeyCanada’s national sport, bringing Canada together, or,because there are more National Hockey League teams in theUnited States than in Canada, does hockey does hockey289contribute to North American continental integration? Howimportant are a national anthem and a flag to thedevelopment of societal unity? What is the role ofregionalism in creating sectional rather than societalunity? All of these questions focus on the function anaspect of society has in developing either cohesion orintegration within the society, or in preventing suchconcensus from occurring.The second perspective tends to see society less as anon-going equilibrium than as a struggle between conflictinggroups. This is known as the conflict’ perspective becauseits focus is on what groups have power within a society,what groups possess less power, and how the use of thatpower determines what a society is like. Terms likedominance and subordination, center and periphery, andwealth and poverty all express different relationshipsbetween people within a society. For example, the fact thatpersons of British descent have historically had more powerin Canadian society than persons of French descent, createda set of institutions in Canada expressing a Britishheritage and a form of communications in which English wasdominant. But evidence of power struggles are found amongboth language groups regarding whose definition of whatCanadian society should be like will dominate.Test 2: The Question of Society290It may seem ironic that even though the Canadian stateis over one hundred years old, the precise nature ofCanadian society and its existence as an entity is still inquestion. In fact, the stormy years after the centennialbirthday in 1967 suggested more than ever that the conceptof a Canadian society could not be taken for granted. WhileQuebec was contemplating what degree of distance from therest of Canadian society was most appropriate, The SymonsReport was concluding that Canadians knew little about theirown society, and a Federal Task Force on Canadian Unity wasscouring the country for clues about ways to create a moreintegrated and cohesive society. More recently, the debateprompted by the Meech Lake Accord whereby Quebec soughtrecognition for its status as a “distinct society” createdcontroversy that again suggested the fragility of nationalunity. What kind of society is this that has beenproblematic for so long?The use of the term “Canadian society” implies that itcan be differentiated from other societies and that it hassome measure of internal coherence. Yet there seems to beevidence to suggest that internal coherence in Canadiansociety has been in continual question. Repeated waves ofimmigration and emigration, British and American influences,French-English differences, a relatively sparse butclustered population in a vast territory, and uneveneconomic development are only some of the factors that havecontributed to fragmentation rather than societal unity.291It is therefore, by no means certain that there reallyis such a thing as a Canadian Society. Does the strength ofthe various small scale sub—societies in Canada preclude anymeaningful discussion about Canadian society as a whole? Dodifferences in the resident population overwhelm whatevermay be held in common.Canada exists as a nation by the political andlegislative degree of the British North America Act passedby the British parliament in 1867. This legislativedocument created an independent national unity.292Test 3: EthnicityPerhaps the most dominant feature of the “New World asopposed to the “Old” Word, as seen through European eyes,was the vast amount of sparsely settled territory in the“New” World. It is this basic fact of a relatively smallpopulation in an enormous area, compared to the crowded“Old” World, in combination with the expansionist ambitionsof European powers, that predestined ethnicity to be asalient issue in the building of Canadian society. First itwas the contact of European cultures with Native Peoples,then the intermingling of European peoples with each otherin the new land, and now more recently, immigration from newsource areas such as Asia and South America. Even afteryears of residence in CaTlada, the government policy ofmulticulturalism encourages members of the society toremember, rediscover, or retain their ethnic origins.Clearly, ethnicity has been, and continues to be, asignificant feature of Canadian societyEthnicity is an amalgam of objective factors relatingto place of birth, citizenship, mother tongue, and customsand traditions which are transmitted through a person’sheritage and characterize that individual. In the Canadianexperience, ethnicity is frequently rooted in reference toanother nation—state which provides a “foreign” dimension tothe concept. But ethnicity does not only involve theseobjective traits such as language and customs; it alsoinvolves a subjective element pertaining to how people view293themselves, i.e., their ethnic identity. There is adifference, then, not only in whether a person speaksItalian or English, or whether that person is a citizen ofCanada or Italy, but also whether the person embraces anidentity as “Italian,” “Italian—Canadian,” Canadian, or evenCanadian. Each alternative tells us something importantabout that person in relation to the wider society.The diversity of contemporary responses people give tothe objective and subjective facts of their background makesthe analysis of ethnicity both a dynamic and essential forthe understanding of Canadian society.Test 4: UniquenessOne of the best ways to learn about a society is tocompare it with other societies. Such comparisons areimportant because, while each society is unique and no twosocieties are identical, there is a tendency to assume thatthe society under review has few parallels elsewhere. Thisis particularly the case for people who live in Canada, whoare caught up in the internal struggles of Canada’s societalproblems, or who are trying to resolve them as if they hadnever been experienced anywhere before. A comparativeanalysis helps us to see that there are other societies withsimilar problems for which similar or different solutionshave been found. For those studying Canadian society fromthe outside, it is also important to see linkages to othersocieties with which they may be familiar through the media294or personal experience. In sum, a comparative focus enablesus to place a society in a global context which sharpens ourability to identify distinctives and similarities. Not onlyar our horizons broadened, but we are able to understand thedynamics of Canadian society in a new way.No attempt is made in this chapter it exhaustivelydetail the many ways in which Canadian society contrastswith other societies. In fact, some of the contrasts willremain implicit or weakly developed because socialscientists have not engaged in much comparative work as suchwork is both difficult and complex. Yet it is possible toidentify some of the characteristic features of Canadiansociety, to look for parallels in other societies, and tooutline some of their significance and meaning in thatsocial context.We have already seen that the nature of Canada’s landsurface, settlement history, population distribution, andregional differences has produced a society with its owncharacter. We have also noted that the variables such associal class, ethnicity, language, religion, and occupationadd a dynamic to these formative features. What has beenthe experience of other societies where similar factors arepresent?Test 5: PolityIf you want to create some controversy in a group ofCanadian people, there is no faster way to do it than to295bring up the issue of bilingualism. Everyone has an opinionabout bilingualism and everyone also has stories oranecdotes to tell from everyday life to justify theirposition. Whether it is hearing a bilingual version of “0Canada,” hearing two Beets of preflight instructions on anairplane, or seeing a simultaneous translation of a documentor instructional sign, bilingualism is an issue manyCanadians are still grappling with because it is foreign tothe unilingual manner in which most people live their lives.It is frequently assumed that most countries operatewith one dominant universal language. Many Canadians, forexample, are aware that the United States has taken as manyimmigrant peoples utilizing numerous languages, but they notthat one language (English) is still used as the primarymeans of communication. What is perhaps less well—known isthat the large influx of Spanish-speaking peoples into thesouthern United States in recent years has created a realbattle in some areas (e.g., California and Florida) overwhether two languages (English and Spanish) should beendorsed. The national English-speaking has so farsuccessfully argued that unilingualism should be theAmerican norm. The facts of Canadian history and polity,however, are somewhat different, though there is a tendencyfor people to assume that unilingualism ought to be thenormal pattern in Canada as well.But if unilingualism is the norm in many countries, arethere countries where more than one language is officially296recognized, or does Canada stand alone on this issue? WEhave already seen that regional ethnic majorities maysupport a language distinct from the official language. TheSoviet Union, for example, has several regions where ethnicgroups possess a distinct language (e.g., the Ukraine andMoldavia). It is possible for countries to officiallyrecognize the existence of more than one language as a basisfor communication within their society as a whole?297Test 6: IdentityIt is the commonality of sharing a territory (in spiteof its size) and participating in its polity (in spite ofits inequities) that makes the country’s residents Canadian.In other words, it is the collective interaction of peoplewho share the symbol “Canadian” that creates a nationalidentity. A social identity is the sum of the sentiments,cultural attributes, and structural arrangements peopleshare which gives them a feeling that they belong together.Individuals and groups create and contribute to thatidentity, but they can also internalize the nationalidentity into their personal definition of themselves(though to varying degrees). A societal identity, then, hasa collective component as well as an individual dimension.For this reason, it is possible to speak of the residents ofCanada forming a national society, and individual membersaccepting that collective identity as something that ispersonally meaningful.But there is a second aspect to being part of anational collectivity; that is, we learn what it means to bepart of Canadian society by distinguishing this society fromother societies. One study found that members of Canadiansociety became more aware of their national identity throughinteracting with foreigners. It could also be argued thatstate negotiations with other societies make Canadians moreaware of their national interests. The point is that anational identity can coagulate as a result of external298relationships, and as a consequence this external dialogue,a society may become more aware of its internalrelationships or the concerns its people share.There is also a third aspect to national identity. Oneperception of a national identity is merely that adistinctive identity emerges as things as they are(descriptive). For example, no society has quite the samerelationship between anglophones and francophones as Canadadoes, and for better or worse, that is what gives Canadiansociety its identity. But another perspective is far moreprescriptive, suggesting vision and ideals about whatCanadian society should be.299Color-”cc:di ng is used to hel p you fi d clue fill in the blanksElue Nouns:= Pronouns:Red = Yerbsreen AdjectivesPurple AdverhsEllack = Othersstudent; Ebb; Tokyo; school; science,..I he; me, her...go, study, think; went; will study...big; my his; this4 a the; flOMquickly; Loday then; there; very.and, or; in, on...CANADIAN SOCIETY i The Question of SocietyIt may seem ironic: that even though the Canadianstate is over one hundred years ci ci the cirec:i senature of Canadian society and its existence as anentity is still in question. En fact, the1) years after the centennial bi rthday2)_ 1967 suggested more than ever that theconc:ept of a 3)_._ society could not befor granted. While Quebec 5)__contemplating what degree of 6)________________from the7)_ of Canadian society was Lapproprate the Symons Report 9)__.Canadians 10)..Ii tt.le about their own11) and a Federal Task Force on Canadian12)_ was scouring the coutry fc;r 13)_,__,about ways to 14).,, a more i nteqrated and15)____ society. More recently the 16)__prompted by the Meech Lake Accord whereby 17)._sought recogni Lion for its status as a diet . nct16)_______created controversy that again 19)the fragility of 20) unity. What kind ofconc 1 LLCJ i rig that3oosociety 21)_for 50 1 on ci?The 23)_implies that it can be $5societies and 26)....to be evidence to 29)31)_differences,, a 3:3)in a vast 34)are only 35)37)______this that has been 22)_______________of the term “Canadian 24)_____________H—.from otherit has some measureinternal coherence N Vet there 26)that internal______in Canadian society has been in continualRepeated waves of immigration and emigration9and American influences FrencriEnplishsparsebut clustered populationand uneven economic developmentof the factc,rs that haveto fragmentation rather than societalIt is9 therefore by no means 36),_,._ thatthere really is 39)—a thing as a Canadiansoc iety 40) the strenqth of the var ious41. ),,,..scale sub-”societ ies in Canac:Ia preclude anym’ ninju1 4’?) —— dbouL C anach an 4 ) — as awhole? DC) 44),__ in the resident 45)overhelm whatever 46) be held in common?47)___,,, exists as a nation by the— and 49)_ - decree of the BritishNorth America Act passed by the 50) parliamenti n 1667 This 1 ep i slat i ye docume nt created an i ndependentnational unity03o1Color—coding is used to help you find clues to fill in the blanks.Blue = Nouns:= Pronouns:Red = Verbs:Green = djectivss:Purple = AdverbsBlack = Other:The Issue of Ethnicitystudent; Bob; Tokyo; school; science...I, he; me, her...go, study. think; went: will study...big; my, his; this; a, the; no.quickly; today, then; there; very...and, or; in, on...Perhaps the most dominant feature of the ‘New Worldas opposed to the “Old” World, as seen throughEuropean eyes, was the vast amount of sparsely settledterritory in the “New’ World. It is this 1)_________________fact of a relatively 2)________________enormous 3)________population in ancompared to the crowded ‘Old”in combination 5)_________the expansionisambitions of 6)___ _____powers, that predestined ethnicity7)_____________be a salient issue 8)__ __thebuilding of Canadian society. First it 9)_____thecontact of European 10) with Native Peoples,11))remember, rediscover, 21)_retain 22)ethnic origins. Clearly, ethnicity has been, andthesiato4)the intermingling ofEuropean 12)___with each other 13__ ___new land, and now 14)_ ___recently, immigrationfrom 15)____ __source areas such 16)and South 17). Even after years 18)residence in Canada, the government policy of19)____encourages members of the 20)_to be, a significant feature of24)__,_ societyEthnicity is an amalgam of 2.5)_ factorsrelating to place of birth, 26)____ mother tongue,and customs and 27)____ which are transmitteda 29L_ heritage and characterizethat individual In 3O)__.,. ___ Canadian exper ienceqis frequently rooted in reference3$)_,__ another nation—state 33)_• —— providesa “foreign” dimension to the 34)________?utethnicity does not only 35) _ _, these objectivetraits such as language and 36)_ ---j it alsoinvolves a 37) ____. element pertaining to how36)____view themselves9 iNel., their 39)__,,_______identity There is a difference, then, not 40)___________in whether a person speaks I tal ian or 41)or whether that 42)__ ___ is a citizen of43)_____ or Italy, hut also 44) __ theperson embraces an 45) as “Italian, “ “Italian—46)____ “ Canadian or even Canadian.. 47)_,_alternative tells us 4$)________ important about that49)______ ____in relation to the wider 5O)_The diversity of contemporary responses people give tothe objective and suhective facts of their backgroundmakes the analysis of ethnicity both a dynamic andessential fc:;r the understanding of Canadian society..3L3Color—coding is used to help you find clues to fill in the blanks.Blue = Nouns: student: Bob: Tokyo: school; science...= Pronouns: I, he; me, her...Red = Verbs: go, study, think; went: will study...Green = Adjectives: big; my, his; this: a, the; no..Purple = Adverbs: quickly; today, then; there: very...Black = Other: and, or; in, on...The fuestion of UniquenessOne of the best ways to learn about a societyis to compare it with other societies. Such comparisons1)_______________important because, while each 2)_______________1unique and no two 3)_______________are identical44)— is a tendency to 5)______________ ___particularly the___ _______Canada, 9)___________are caught10)______ __of Canada’s societal_______who are trying to resolve___as if they 13) never beenanywhere before. A comparative analysis helpsthat there are otherwith similar problems for whichor different solutions have been found.Canadian society from the outdside,is also 19)— to see linkagessocieties with which 21>_ _may___the media or personal_ _• In sum, a 24)_focus enablesto place a society in a globalsociety under review has 6)This 7)people who live inup in the internalproblems, 11)that theparallels elsewhere.case 8)12)i’4)us to 15)16)17)For those studying18)to 20)be familiar 22)23)25)3o’1context which 26) our ability to identifydistinctives and 27) • Not only are28) horizons broadened, 29) we areable to 30) the dynamics of Canadian societyinanew3i)___________No attempt 32) made in 33)chapter to exhaustively detail the many ways in whichCanadian society contrasts with 34) societies. Infact, some of the contrasts 38) remain implicitor weakly developed because social scientists 36)not engaged in much comparative work as such37) is both difficult and 38) • Yetit is possible to 39) some of thecharacteristic features of 40) society, to lookfor parallels in other societies, 41) tooutline some of 42) significance and meaning inthat 43) context.We have already 44) that the nature ofCanada’s land surface, settlement history, population distribution,and regional 45) has produced a 46)with its own character. 47) have also notedthat variables 48) as social class, ethnicity,49) , religion, and occupation add a dynamic to50) formative features. What has been theexperience of other societies where similar factors arepresent?;c 0•Color—coding is used to help you find clues to fill in the blanks.Blue = Nouns: student: Bob; Tokyo; school; science...= Pronouns: I, he; me, her...Red = Verbs: go, study, think; went: will study...Oreen Adjectives: big: my, his; this; a, the; no...Purple Adverbs: quickly; today, then: there; very...Black = Other: and, or; in, en...Polities With Mere Than One Official Language: anIntroductionIf you want to create some controversy in agroup of Canadian people, there is no faster way todo it than to bring up the issue of bilingualism.Everyone 1)_______________an opinion about 2)_______________3)___________also has stories or anecdotes to‘4)_from everyday 5)_____________to justify6) position. Whether it is hearing a bilingualversion of “0 Canada,” 7)_________two sets ofpreflight 8)______on an 9)_, or seeinga simultaneous translation 10) a document orinstructional sign, bilingualism is an 11)___________manyCanadians are still grappling 12)__because it isforeign to the unilingual 13) in which most14)____live their lives._________that most countries______dominant universal language. Many__for example, are 18) that theUnited States 19)_taken as many immigrantpeoples utilizing numerous languages, 20)that one 21) (English) is still 22)andit is frequently 15operate with 16)17)they note3oas the primary 23)_____ of communication, What is24)_____less well—known is 25)___________ the1 ar ge influx o1 Span i sh”i eak i nq 26) into thesouthern Un ted 27)__ in recent 26) hascreated a real battle in 29) __ areas (e0q11California and Florida) over 3O)_ to languages(English and Spanish) should 3l)__ _ endorsed. Thenational 3$)_,. __—speakxng majority has so farsuccessfully argued that uni lingual ism 33),_,, be theAmerican 34)_._ The facts of Canadian history35)__________polity however are somewhat different though36)______is a tendency for 37)__ - toassume that 36) ought to he the normalpattern in Canada as 39)____But if uni lingual ism is the norm in many4O)___,_ are there countries where more than oneis officially recognized or does42)__stand alone on this issue? 43)_—have already seen that 44)____ethnic majorities45)__• support a language distinct 46)_,__the official 47) The Soviet Linionq for46)__has several regions where 49)____groups possess a 5O) language (eugu, theUkraine and Mol davia) It is possi bie for countries toofficially recognize the existence of more than onelanguage as a basis for communi c:ation within theirsociety as a whole?3c7Color—coding is used to help you find clues to fill in the blanks.Blue = Nouns:= Pronouns:Red = Verbs:Green = Adjectives:Purple = Adverbs:Black = Other:student; Bob; Tokyo; school; science...I, he; me, her...go, study, think; went; will study...big: my, his; this: a, the: no...quickly: today, then; there: very...and, or; in, on...Aspects of Societal Identity14)15)16)17)18)a 20)It is the commonality of sharing a territory (inspite of its size) and participating in its polity(in spite of its inequities) that makes the country’sresidents Canadian. In other words, it 1)________________thecollective interaction of people 2)________________share thesymbol Canadian 3)______________creates a national identity.4)_______________eccial identity is the 5)___ofthe sentiments, cultural attributes, 6)____structuralarrangements 7)_________share which 8)_______them afeeling that they belong 9)_________. Individuals and10)_______create 11)_contribute to that12),but they can also internalize the nationalidentity 13)_their personal definition of_(though to varying degrees) . Aidentity, then, has a collective componentwell as an individual dimension.this reason, it is possible toof the residents of 19) forming__society, and individual 21)___3°facceptinq that 22),_.,,,,_ identity as something23)________________is personally meanincful.But 24)__ is a second aspect to being25)__,,,_—of a national collectivity; that is26)_ learn what it means to he part of27),,,___ society by distinguishing this 29)___from other societies. One study found 29)members of Canadian society became more aware of3O)_ national identity through their travels outsidethe country and 3i)_.._ interacting with foreigners, Itcould 32)_ be argued that state negotiations33) other societies make Canadians more34__,__ of their national interests. The point35)_ that a national identity 36)_coagulate as a result of external relationshipsq 37)_______________as a consequence 3E1)__ external diaiogue asociety may become more aware 39)_____ i internal4c)),_ or the concerns its people 41)_,__,There is also a th:ird 42) _ to nationalidentity. One perception oF a 43) identity ismerely that a distinctive identity emerges 44) thingas 45)_ are (descriptive). For 46)___ —,nsociety 47)_ quite the same relationship4a)_ anglophones and francophones as Canada does,and for 49). or worse, that is what givessocietyits identity. But another perspectiveis far more prescr,:pta ve suggesting vision and ideals aboutwhat Canadian society should be.3°’1Arswer kyAipue tf Beieta1 IdentityIt is the commonality of sharing a territory (inspite of its size) and participating in its polity(in spite of its inequities) that makes the country’sresidents Canadian. In other words, it 1)_._..4 C thecollective interaction of people 2) aij)aPk. share thesymbol “Canadian” 3) creates a national identity.4)__A. social identity is the 5) UrI ofthe sentiments, cultural attributes, 6), structuralarrangements share which B)•1.C them afeeling that they belong 9) • Individuals and10) create 11) contribute to thatbut they can also internalize the nationalidentity 13) their personal definition of(though to varying degrees). Aidentity, then, has a collective component16) well as an individual dimension.17) this reason, it i. possible to16) of the residents of 19)...ChflJ.L..... forminga 20) society, and individual 21)rn,hIkef,_3/0baccepting that 22) identity as something23),, is personally meaningful.But 24L+lhOt is a second aspect to being25)...fQ.r..f_._... of a national collectivity; that is,26) learn what it means to be part ofsociety by distinguishing this 28)...f.CJ..L_.from other societies. One study found 29)members of Canadian society became mor. aware of30) national identity through their travels outsidethe country and 31) interacting with foreigners. Itcould 32) be argued that state negotiationsether societies make Canadians moreof their national interests. The point35) IS that a national identity 36) COtlcoagulate as a result of external relationships, 37)as a consequence 3B) external dialogue, asociety may become more aware 39) its internal4o)fI±1DniL9pser the concerns its people 41) aThere is also a third 42) to national1identity. One perception of a 43) identity ismerely that a distinctive identity emerges 44)__#tflb1r%_.. thingsas 45)J he •r (descriptive). For 48) nosociety 47)ritk.c quite the sam. relationship48) anglophon.s and Francophon.s as Canada does,and for 49) or worse, that is what gives50) society its identity. But another perspectiveis Far mar. prescriptive, suggesting vision and ideals aboutwhat Canadian society should be.311Sample of distribution of reiterative wordsTABLE HA SAMPLE OF HOW THE PASSAGES WERE ANALYZED FOR SELECTION OFTHE MAXIMUM NUMBER OF REITERATIVE WORDS AND THEIR LOCATIONPerspectivesThere are three dominant perspectives for understandinga society.THE first °perspectiveattempts to determine asociety°ho1ds together AS an entity. °This°perspective is°known°AS functionalism°because°whatever aspect°° societyis analyzed, the object is to°determine°h that structuralfeature°contributes°to the integration of °society. Forexample, broadcasting°can°be assessed in terms OF°°itcontributes to national unity, conversely, disunity. DoCanadian radio stations ogiveepriority to recordings by°Canadian°artists so that_Canadian°culture°is developed, orDOES THE American music industry°dominate°? Is hockeyCanada’s national °, bringing 0 together°, I,because there are more National°Hockey°League teams THEUnited States than .jj Canada, °does°hockey does hockeycontribute to North Americanimportant are aonationale anthem and a°flag°to THEdevelopment of What is THE role OFregionalism°°creating sectional rather than societalunity? °°of these questions focus on THE°function°an°aspect°of society has cohesionintegration within THE society, iii preventing suchconcensus fromtoccurringo.312THE secondoperspectiveo tends to see society less°ranon-going equilibrium° ° as ° struggle °between conflictinggroups. This is known as becauseits°focus°is on what groups have power within a society,what groups possess less°ower°, and THE use OF thatpower determines what a society is LIKE. °Terms°likedominance and subordination, center °° periphery, and°like°wealth and poverty°fl°express different relationships°between°people within a society. For°example°, THE factthat persons OF British°descenthave historically had more°POWER° CANADIAN society than°persons°of French descent,created a set OF institutions jj Canada expressinga°British°heritage AND a°form°OF communications whichEnglish was°dominant°. But evidence OF POWER struggles arefound among both language groups regarding whose definitionOF what CANADIAN society should be like will dominate.313APPENDIX F:PRODUCTION OF THE CLOZE PASSAGESStep 1 Type out all the storiesStep 2 Code for parts of speech using one symbol for eachcorresponding color(verb = 1)(noun =(4 = adverb)= adjective)(*= other parts of speech)Using the word processor’s “search and replace”function, a word can be replaced by itself plusthe appropriate symbol in front. Using the “all”feature, all of the same word can be replaced in asecond or so. Starting at the beginning of thefirst story, replace each word, one after theother. After a paragraph or two have been done,many words all the stories will have been coded.As more and more words are replaced the taskbecomes exponentially easier. Care must be takenwith some words which can vary their part ofspeech by their position in the sentence. Wordslike “work” can change in part of speech in thisway. Either be sensitive to such cases whenreplacing words with words and symbols, or checkafter the passages have been coded.314Step 3 Choose the words to be deleted and replace themwith a question, number, and two short lines.Step 4 After all the words have been deleted, replace theshort blanks with the standard 15 space lines andmake all characters bold.Step 5 Replace the single symbols with the correspondingprinter codes as follows:1 = ((C))l = red = verb= ((C))2 = blue = noun= ((C)) 3 = purple = adverb& = ((C))6 = green = adjectiveStep 6 Use the “search and replace” function and replaceall the printer codes with “0” to give all blackprint for the standard doze for the controlgroup.Step 7 Use the “search and replace” function and replaceall the printer codes with “0”, “1”, “2”, 3”, or“6” in a random fashion to give make the randomfor of the doze procedure.Step 8 Print with the Star NX1000 (or NX1O2O) rainbowprinter using a colored ribbon.315TABLE ICOLLECTION AND BLANK SCORE DETERMINATION CHART(MAIN PROJECT)°NAME OF PASSAGE 0° TYPE OF BLANK° (EXTERN) INTER- 0 INTRA- 00 0 Bi 0 B2 ° B30 0 0 0 00 KEY WORD *** 0 0°STUDENT 0 0 0°IDENTITY 0 0 0 00 0 0 0 0°Classl 0 0 00 0 0 0 0°JNK ° ° 0°DAV 0 AAAAA 0 1 • (empty):APR : 1 : (empty) : 1 :°Total: 0 2 1 0 1°Mean .66 0 33 0 330 0 0 0 0°Class3 0 0 0 00 0 0 0 0TM 0 (empty) 0 1 0 0°NM 0 1 0 1 0 0:HN : (empty) : (empty) (empty)°Total: 1 2 0°Mean: 0 •33 ° .66 0 0316APPENDIX G:COMPARISONS OF EXACT- AND ACCEPTABLE-WORD MEAN SCORESTABLE JEXACT- AND ACCEPTABLE-WORD MEAN SCORESALL BLANKSClass 1 Class 3Pre-test19.9 17.5 °exact—word scores• C° 23.2 0 22.3 °acceptable—word scoresTest 2 Test 3 Test 4180°132 00 225169° 274 • 227o C 00 C 00 Co 23.2 • 17.0 °° 33.1 ° 24.6 39.2 0 344 0Test 5 Test 6027602600029302550o C 00 C 00 377 • 32.8 °° 39.7 0 32.7Post-test° 23 8 ° 20 8 °• 0 00 30.6 0 27.1317TABLE KEXACT- AND ACCEPTABLE-WORD MEAN SCORESINTRA-/ INTER-SENTENTIALClass 1 Class 3Pre-test15.2 13.2 °exact—word scores0 0 0• 191 ° 16.0 °acceptable-word________________scoresTest 2 Test 3 Test 40 1560 10600 19801510022501820o 0 00 0 00 0 0° 18.4 ° 12.2 00 28.0 ° 20.6 •° 30.3 0 26.0Test 5 Test 6024902340024202150• 0 00 0 0O 32.9 28.7 °° 32.3 0 27.8Post-test° 19 9 0 16 5 °0 0 0° 30.6 0 27.0 0318TABLE LEXACT- AND ACCEPTABLE-WORD MEAN SCORESINTRA-SENTENTIALClass 1 Class 3Pre-Test0 5.1 0 4•4 °exact—word scores0 00 5.8 0 5.0 °acceptable-word_________________scoresTest 2 Test 3 Test 40 780 57°° 67 5.100 570 580o 0 00 0 00 0 00 95 0 57 00 7.6 ° 6.4 00 6.0 0 6.5Test 5 Test 60 550 4800 690 6700 0 00 0 00 54 0 55 °° 9.8 9.1 0Post-test0 530 5000 0 00 12.5 0 12.9 0319TABLE MEXACT- AND ACCEPTABLE-WORD MEAN SCORESINTER-SENTENTIALPre-testo io.i • 8.8 °exact—word scores• 0o 13.3 ° 11.0 °acceptable—word________________scoresTest 2 Test 3 Test 40 780 49°° 1310100 ° 168012400 0 00 0 00 0 0o 8.9 0 6.5 00 20.4 0 14.2 •° 24.3 0 19.5Test 5 Test 60194018600173014800 C 00 0 0° 27.5 ° 23.2 00 22.5 0 18.7Post-test0 14 6 ° 11 5 0o 0 0° 18.1 0 14.1320APPENDIX H THE COLOR-CODED CLOZE PROCEDURE PROJECTQUESTIONNAIREThank you very much for participating in the Color-Codeddoze Procedure Project. Now you have had the chance topractise the doze procedure, you have the opportunity toshare your ideas and feelings about the project.1. Before this project had you ever done doze exercisesa) in Japan? Yes Nob) In Canada? Yes NoIf yes, did you enjoy the doze exercises?Yes No2.1 In the present project did you have any difficultieswith exercises?Yes No2.2 Please check one. Did you find the doze exercise(s)more difficult at the:a) beginning of the project,b) at the middle of the project, orc) at the end of the project?2.3 Please check one. Did you find the doze exercise(s)easiest at the:a) beginning of the project,b) at the middle of the project, orc) at the end of the project?3212.4 As you got more practice did you gain more confidence indoing the doze exercises?Yes No_2.5 Did you have any problems seeing the words on theexercise papers?Yes No_If yes, what problems?2.6 Do you think this project has helped you improve yourability to accurately guess missing words in a passage?Yes No2.7 When trying to find words to fill in the blanks did youever:a) look in the sentence of the blank?Yes Nob) look in the sentence before the blank?Yes Noc) look in the sentence after the blank?Yes Nod) use your knowledge of the topic?Yes Noe) guess?Yes Nof) do something else?Yes NoIf yes, what did you do?2.8 Would you like to do doze exercises in the future?Yes_ NoIf yes, what changes would you like to bemade?2.9 Please write down any comments you have about anythingin this project.3223233.1* Did you practise the doze exercises using color?Yes No3.2 Did the color help you fill in the blanks?Yes No3.3 Did the color make the exercises more difficult?Yes NoIf yes, please circle the most difficultexercise(s).a) the beginning exercises (#2 and #3)b) the middle exercise (#4)C) the end exercises (#5 and #6)3.4 Did the color make the exercises easier?Yes NoIf yes, please circle the easiest exercise(s)?a) the beginning exercises (#2 and #3)b) the middle exercise (#4)c) the end exercises (#5 and #6)3.5 Did the color help you to find words to fill in theblanks, words that were:a) within the sentence of the blank?Yes Nob) in a sentence before the sentence of the blank?Yes Noc) in a sentence after the sentence of the blank?Yes No3.6 Do you think the colored exercises helped you to do thelast exercise (uncolored)?Yes No3.7 Please circle the type of doze exercise you wouldprefer to do in the future?a) non-colored doze b) colored doze3243.l** Did you practise the doze exercises using the color—coded parts of speech?Yes No_3.2 Did the color—coded parts of speech help you fill in theblanks?Yes No3.3 Did the color—coded parts of speech make the exercisesmore difficult?Yes NoIf yes, please circle the exercise whichwas the most difficult.a) the beginning exercises (#2 and #3)b) the middle exercise (#4)c) the end exercises (#5 and #6)3.4 Did the color—coded parts of speech make the exerciseseasier?Yes NoIf yes, please circle the easiest exercise(s)?a) the beginning exercises (#2 and #3)b) the middle exercise (#4)C) the end exercises (#5 and #6)3.5 Did the color-coded parts of speech help you find wordsto fill in the blanks, words that were:a) within the sentence of the blank?Yes Nob) in a sentence before the sentence of the blank?Yes No_c) in a sentence after the sentence of the blank?Yes No3.6 Do you think the color-coded exercises helped you to dothe last exercise (uncolored)?Yes No3.7 Please circle the type of doze exercise you wouldprefer to do in the future?a) non—colored doze b) color—coded doze* This section was for Class 2.** This section was for Class 3325BIOGRAPHICAL INFORMATIONNAME: Howat Alan LabrumMAILING ADDRESS: #241-13612 67th AvenueSurrey, B.C.V3W 6X5PLACE AND DATE OF BIRTH: Vancouver, B.C., Canada24/02/50EDUCATION: (Colleges and Universities attended,dates, and degrees)The University of British Columbia Bachelor of Arts 1972The University of British Columbia Diploma in AdultEducation 1973POSITIONS HELD:Supervisor of E.S.L teachers, United Nations RefugeeProgramE.S.L. teacher in Canada, Thailand, Malaysia, andSaudi ArabiaPUBLICATIONS:AWARDS:


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items