UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Cross-cultural study of test-wiseness Man, Daisy Wailing 1990

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1990_A8 M36.pdf [ 6.77MB ]
Metadata
JSON: 831-1.0098029.json
JSON-LD: 831-1.0098029-ld.json
RDF/XML (Pretty): 831-1.0098029-rdf.xml
RDF/JSON: 831-1.0098029-rdf.json
Turtle: 831-1.0098029-turtle.txt
N-Triples: 831-1.0098029-rdf-ntriples.txt
Original Record: 831-1.0098029-source.json
Full Text
831-1.0098029-fulltext.txt
Citation
831-1.0098029.ris

Full Text

CROSS-CULTURAL STUDY OF TEST-WISENESS by DAISY WAILING MAN B.A.(Hons), U n i v e r s i t y of Winnipeg, 1987 B.Ed., U n i v e r s i t y of Winnipeg, 1988 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS i n THE FACULTY OF GRADUATE STUDIES DEPARTMENT OF EDUCATIONAL PSYCHOLOGY AND SPECIAL EDUCATION We accept t h i s t h e s i s as conforming to the r e q u i r e d s t a n d a r d THE UNIVERSITY OF BRITISH COLUMBIA J u l y 1990 © DAISY WAILING MAN In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of E d u c a t i o n a l P s y c h o l o g y a n d S p e c i a l E d u c a t i o n The University of British Columbia Vancouver, Canada Date A u g u s t 9, 1 990. DE-6 (2/88) ABSTRACT The o b j e c t i v e s of t h i s study were to i n v e s t i g a t e the c u l t u r a l d i f f e r e n c e s i n t e s t - w i s e n e s s between Chinese and Canadian students; to study the impact of t e s t - w i s e n e s s on B r i t i s h Columbia Grade 12 p r o v i n c i a l examination r e s u l t s among Chinese students i n B r i t i s h Columbia; and to examine the r e l a t i o n s h i p of t e s t - w i s e n e s s , and four c o r r e l a t e s ( v e r b a l a b i l i t y , achievement m o t i v a t i o n , l e n g t h of r e s i d e n c y i n Canada, m u l t i p l e choice experience) among the Chinese students. The Test of Test-wiseness and Student Survey q u e s t i o n n a i r e were admi n i s t e r e d to 41 Grade 12 f o r e i g n Chinese students who wrote the p r o v i n c i a l examinations i n January 1990. Completed t e s t data were compared with the same data from Canadian and Chinese immigrant students who wrote the p r o v i n c i a l examination i n June 1989. The data were analyzed by a l i n e a r r e g r e s s i o n model. The r e s u l t s i n d i c a t e d that e t h n i c i t y accounted f o r a s i g n i f i c a n t amount of v a r i a n c e i n t e s t - w i s e n e s s . F o r e i g n Chinese students were s i g n i f i c a n t l y l e s s t e s t - w i s e than both Canadian and Chinese immigrant s t u d e n t s . F o r e i g n Chinese students scored s i g n i f i c a n t l y lower than the other two groups i n a bsurd-options, d i f f e r e n t - o p t i o n s , stem-options l i n k and guessing s u b t e s t s i n the Test of Test-wiseness. The r e s u l t s a l s o showed that t e s t - w i s e n e s s d i d not account f o r a s i g n i f i c a n t amount of v a r i a n c e i n p r o v i n c i a l examination scores ( E n g l i s h 12 and Algebra 12). The four n c o r r e l a t e s together accounted f o r very l i t t l e v a r i a n c e i n t e s t - w i s e n e s s . None of them s i g n i f i c a n t l y c o r r e l a t e d with t e s t - w i s e n e s s , except that l e n g t h of r e s i d e n c y i n Canada c o r r e l a t e d p o s i t i v e l y with t e s t - w i s e n e s s . F i n a l l y , the three e t h n i c groups showed no s i g n i f i c a n t d i f f e r e n c e s i n achievement m o t i v a t i o n but f o r e i g n Chinese students scored s i g n i f i c a n t l y lower than Chinese immigrant students who scored i n t u r n lower than Canadian students i n v e r b a l a b i l i t y . The f i n d i n g s imply that language p l a y s a c r u c i a l p a r t i n t e s t - t a k i n g s i t u a t i o n s . With inadequate E n g l i s h language s k i l l s , f o r e i g n Chinese students were unable to s a t i s f a c t o r i l y comprehend the t e s t q u e s t i o n s , and hence t h e i r t e s t performance were very c l o s e to what would be expected from random guessing. I t i s suggested that these students r e q u i r e examination coaching s k i l l s as w e l l as improvement i n E n g l i s h language s k i l l s . in TABLE OF CONTENTS ABSTRACT i i LIST OF TABLES v i LIST OF FIGURES v i i ACKNOWLEDGEMENTS v i i i CHAPTER ONE - INTRODUCTION 1.1 The B r i t i s h Columbia P r o v i n c i a l Examinations 1 1.2 D e f i n i t i o n s Used i n The Study 2 1.3 Background of The Problem 6 1.4 Statement of The Problem . .. 9 1.5 Research Hypotheses of The Study ....10 1.51 Major Hypothesis 10 1.52 Hypothesis One 10 1.53 Hypothesis Two ( a ) . . 11 1.54 Hypothesis Two (b) 11 1.55 Hypothesis Three 11 1.6 R a t i o n a l e of The Hypotheses 12 1.7 S e l e c t i o n and Measurement of Independent V a r i a b l e s . . . 1 5 1.8 S e l e c t i o n of Dependent V a r i a b l e s 16 1.9 J u s t i f i c a t i o n of The Study 17 CHAPTER TWO - LITERATURE REVIEW 2.1 B r i t i s h Columbia P r o v i n c i a l Examinations 19 2.11 H i s t o r i c a l Context 19 2.12 The Examination-Development Process 22 2.13. Impact of The P r o v i n c i a l Examinations 26 2.2 Concept of Test-wiseness 26 2.21 D e f i n i t i o n of Test-wiseness 26 2.22 T h e o r e t i c a l Approaches t o Test-wiseness........32 2.23 Construct of Test-wiseness 34 2.24 Test-wiseness and Bia s 36 2.3 Test-wiseness Research 40 2.31 C o r r e l a t e s of Test-wiseness 40 2.32 Test-wiseness and S e l e c t i o n Item 45 2.33 Test-wiseness and V a r i a b i l i t y of Test Score....50 2.34 Test-wiseness and C u l t u r a l D i f f e r e n c e s 51 CHAPTER THREE - METHODOLOGY 3.1 Su b j e c t s 59 3.2 Instrumentations 61 3.21 Grade Twelve P r o v i n c i a l Examinations 61 3.22 Test of Test-wiseness 61 3.23 Student Survey 66 3.24 Language P r o f i c i e n c y Index.... 67 3.3 Procedure 68 3.4 S t a t i s t i c a l Hypotheses 69 3.41 Major Hypothesis 69 iv 3.42 Hypothesis One ..71 3.43 Hypothesis Two (a) 71 3.44 Hypothesis Two (b) 72 3.45 Hypothesis Three 73 3.5 Method of A n a l y s i s 74 3.51 Major Hypothesis 75 3.52 Hypothesis One 75 3.53 Hypotheses Two (a) S> (b) 76 3.54 Hypothesis Three 77 CHAPTER FOUR - ANALYSIS AND RESULTS 4.1 Background Information of the Chinese Sample 79 4.11 Demographic C h a r a c t e r i s t i c s 79 4.12 Experience i n M u l t i p l e - c h o i c e Examinations 80 4.13 Grade Twelve P r o v i n c i a l Examinations 82 4.2 T e s t s of Hypotheses 84 4.21 P r o v i n c i a l Examinations, Test-wiseness and Four C o r r e l a t e s 84 4.22 E t h n i c i t y and Test-wiseness 89 4.23 P r o v i n c i a l Examinations and Test-wiseness 90 4.24 C o r r e l a t e s of Test-wiseness 92 4.3 Comparison of F o r e i g n Chinese, Chinese Immigrant, and Canadian students 94 CHAPTER FIVE - DISCUSSION 5.1 I n t e r p r e t a t i o n of F i n d i n g s 100 5.11 E t h n i c i t y and Test-wiseness.. 100 5.12 B r i t i s h Columbia P r o v i n c i a l Examinations and Test-wiseness 104 5.13 C o r r e l a t e s of Test-wiseness.... 107 5.2 Summary of F i n d i n g s and C o n c l u s i o n s 111 5.3 L i m i t a t i o n s of the Study 113 5.4 I m p l i c a t i o n s and D i r e c t i o n s f o r F u r t h e r Research 115 REFERENCES 1 20 APPENDIX A. S t a n d a r d i z e d r e g r e s s i o n weights and a d j u s t e d R2 i n each s t e p of d e l e t i o n of the major hypothesis 126 B. R e s u l t s of item a n a l y s i s of s u b t e s t s ID1, ID2, ID3 and IIB4 i n the Test of Test-wiseness .127 C. Items used f o r the v a l i d a t i o n of the Test of T e s t -wiseness 130 D. Student Survey 133 E. Test of Test-wiseness 139 v LIST OF TABLES Table 1 - An O u t l i n e of Test-wiseness P r i n c i p l e s 4 Table 2 - L e v e l of Achievement and the E q u i v a l e n t Reported Percentages of the B r i t i s h Columbia P r o v i n c i a l Examinations 25 Table 3 - Composition of the Test of Test-wiseness 63 Table 4 Psychometric P r o p e r t i e s of the Test of Test-wiseness 64 Table 5 - Composition of the Test of Test-wiseness f o r V a l i d a t i o n 65 Table 6 - Experience i n M u l t i p l e - c h o i c e Examinations of Chinese Sample 81 Table 7 - Grade 12 P r o v i n c i a l Examination R e s u l t s i n the M u l t i p l e Choice S e c t i o n of the Chinese Students and the P r o v i n c i a l Average 83 Table 8 - Pearson C o r r e l a t i o n s of F i v e Independent V a r i a b l e s and Two Dependent V a r i a b l e s 85 Table 9 - Summary S t a t i s t i c s of Major Hypothesis 87 Table 10 - Summary S t a t i s t i c s of Hypothesis One 90 Table 11 - Summary S t a t i s t i c s of Hypothesis Two (a) 92 Table 12 - Summary S t a t i s t i c s of Hypothesis Three 93 Table 13 - Comparison of Three E t h n i c Groups on Test-wiseness, Language P r o f i c i e n c y Index and Achievement M o t i v a t i o n 96 Table 14 - Comparison of Three E t h n i c Groups on F i v e Subtests i n the Test of Test-wiseness 97 vi LIST OF FIGURES F i g u r e 1 - Smith's Model of T e s t - t a k i n g Behavior of S k i l l e d Test Takers 47 F i g u r e 2 - Graphic Comparison of Three E t h n i c Groups on Four Subtests i n the Test of Test-wiseness. . 9 9 vn ACKNOWLEDGEMENT S I would l i k e to express my deepest g r a t i t u d e to my t h e s i s s u p e r v i s e r , Dr. David Bateson, f o r h i s continued support, encouragement, and guidance throughout t h i s t h e s i s . I a l s o thank members of my t h e s i s committee, Dr. Walter B o l d t and Dr. David Whittaker f o r t h e i r h e l p f u l comments and p a t i e n c e i n e d i t i n g t h i s t h e s i s . As w e l l , an acknowledgement i s o f f e r e d to Dr. Todd Rogers and Dr. P a t r i c i a A r l i n f o r t h e i r guidance and i n s p i r a t i o n i n my program at UBC. vin 1 CHAPTER ONE INTRODUCTION 1.1 The B r i t i s h Columbia Pr o v i n c i a l Examinations The p r o v i n c i a l examinations of B r i t i s h Columbia were in t r o d u c e d by the B r i t i s h Columbia M i n i s t r y of Education i n 1983 f o r every Grade 12 student who wished to graduate. Comprised of m u l t i p l e - c h o i c e , completion and short essay q u e s t i o n s , performance of these t e s t s counted 50% toward the f i n a l grade of each student from 1983 to 1989, and now c o n t r i b u t e s 40% toward t h e i r f i n a l grade i n examinable Grade 12 s u b j e c t s . I n s t i t u t e d i n January of 1983, the B r i t i s h Columbia P r o v i n c i a l Examinations are c a r e f u l l y c o n s t r u c t e d and reviewed by committees of p r a c t i c i n g teachers i n each s u b j e c t area. These teachers are s e l e c t e d by the Examination Coordinator and the Manager of the Examinations from the Student Assessment Branch of the M i n i s t r y of Ed u c a t i o n . Each s u b j e c t area committee i s given the r e s p o n s i b i l i t y of developing t e s t items u s i n g the t a b l e of s p e c i f i c a t i o n s prepared by the sub j e c t area committee, f o r that s u b j e c t a r ea. Items from p r e v i o u s examinations and p i l o t e d item banks are reviewed f o r p o t e n t i a l i n c l u s i o n on each t e s t . In a d d i t i o n , new items are developed to i n s u r e coverage of the c u r r i c u l u m i n the su b j e c t area t e s t e d . The items are then reviewed by measurement s p e c i a l i s t s , and a l s o by a d i f f e r e n t panel of p r a c t i c i n g t e a c h e r s f o r the sub j e c t area before the t e s t i s d i s t r i b u t e d . Machine s c o r e a b l e , s e l e c t i o n - t y p e ( m u l t i p l e - c h o i c e ) items 2 c o n s t i t u t e 50% to 80% of the q u e s t i o n s i n c l u d e d i n each t e s t ; the remainder of the items are w r i t t e n (supply-type) q u e s t i o n s . F o l l o w i n g the a d m i n i s t r a t i o n of the t e s t s (January and June f o r semestered s c h o o l s ; June f o r non-semestered schools) to a l l students e n r o l l e d i n examinable courses, marking committees comprised of p r a c t i c i n g teachers i n each s u b j e c t area mark the supply items. The sco r e s f o r these items, along with the responses to each of the s e l e c t i o n items, are then entered i n t o a computer f o r s c o r i n g and item a n a l y s e s . The r e s u l t s of the item a n a l y s e s , together with the comments of the marking committee and the t e c h n i c a l s c o r i n g s t a f f , are then submitted to the P r o v i n c i a l Board of Examiners f o r i t s review and c o n s i d e r a t i o n . Items are r e t a i n e d or r e j e c t e d from the students' f i n a l scores on the b a s i s of the d e c i s i o n s made by the P r o v i n c i a l Board of Examiners. 1.2 D e f i n i t i o n s Used i n t h i s Study The main terms used i n t h i s study were d e f i n e d as f o l l o w s : 1. Test-Wiseness: ( a l s o c a l l e d t e s t - t a k i n g s t r a t e g i e s , t e s t -s o p h i s t i c a t i o n , t e s t - f a m i l i a r i z a t i o n , t e s t - t a k i n g o r i e n t a t i o n and test-wisdom). Millman, Bishop and Ebel (1965) f i r s t attempted to d e f i n e i t as "a s u b j e c t ' s c a p a c i t y to u t i l i z e c h a r a c t e r i s t i c s and formats of the t e s t and/ or t e s t t a k i n g s i t u a t i o n s to r e c e i v e a high s c o r e . Test-wiseness i s l o g i c a l l y independent of the 3 examinee's knowledge of the s u b j e c t matter f o r which the items are supposedly measure" (p. 707). The d e f i n i t i o n of t e s t - w i s e n e s s used i n t h i s study was l i m i t e d to the s t r a t e g i e s used i n the Test of Test-wiseness developed by Rogers and Bateson (1990b). Adapted from the work of Millman (1966) and Gibb (1964), t h i s instrument was comprised of 24 m u l t i p l e - c h o i c e items c o v e r i n g the t e s t -wiseness s t r a t e g i e s of Absurd-options (1D1), S i m i l a r - o p t i o n s (1D2), D i f f e r e n t - o p t i o n s (1D3), and Stem-options Link (1D4) (see Table 1 f o r a f u l l d e s c r i p t i o n of these four s t r a t e g i e s ) . Moreover, i t i n c l u d e s 10 items designed to assess students' p r o p e n s i t y to guess. T h i s measure was v a l i d a t e d from the responses from the students' i n t e r v i e w s d u r i n g t e s t - t a k i n g . 2 . Test - wi sex Students r e c e i v e d a score of '17 or above' on the Test of Test-wiseness (Rogers and Bateson, 1990b). The d e t e r m i n a t i o n of the c u t - o f f c r i t e r i a f o r t e s t - w i s e n e s s was s t a t e d i n Chapter Three, S e c t i o n 3.22. 3. Test-nai ve : Students r e c e i v e d a score of '11 or below' on the Test of Test-wiseness (Rogers and Bateson, 1990b). (See Chapter Three, S e c t i o n 3.22 f o r the d e t e r m i n a t i o n of the c u t - o f f c r i t e r i a ) 4 Table 1 An O u t l i n e of Test-wiseness P r i n c i p l e s I. Elements independent of t e s t c o n s t r u c t o r or t e s t purpose A. Time-using s t r a t e g y 1. Begin a work as r a p i d l y as p o s s i b l e with reasonable assurance of accuracy 2. Set up a schedule f o r progress through the t e s t 3. Omit or guess at items (see I.C. and II.B.) which r e s i s t a quick response 4. Mark omitted items, or items which c o u l d use f u r t h e r c o n s i d e r a t i o n , to assure easy r e l o c a t i o n 5. Use time remaining a f t e r completion of the t e s t to r e c o n s i d e r answers B. E r r o r - a v o i d a n c e s t r a t e g y 1. Pay c a r e f u l a t t e n t i o n to d i - r e c t i o n s , determining c l e a r l y the nature of the task and the intended b a s i s f o r response 2. Pay c a r e f u l a t t e n t i o n to the items, determining c l e a r l y the nature of the q u e s t i o n 3. Ask examiner f o r c l a r i f i c a t i o n when necessary, i f i t i s p e r m i t t e d 4. Check a l l answers C. Guessing s t r a t e g y 1. Always guess i f r i g h t answers only are scored 2. Always guess i f the c o r r e c t i o n f o r guessing i s l e s s severe than a " c o r r e c t i o n f o r guessing" formula that g i v e s an expected score of zero f o r random responding 3. Always guess even i f the usual c o r r e c t i o n or a more severe p e n a l t y f o r guessing i s employed, Whenever e l i m i n a t i o n of o p t i o n s p r o v i d e s s u f f i c i e n t chance of p r o f i t i n g D. Deductive reasoning s t r a t e g y 1. E l i m i n a t e o p t i o n s which are known to be i n c o r r e c t and choose from among the remaining o p t i o n s 2. Choose n e i t h e r or both of two o p t i o n s which imply the c o r r e c t n e s s of each other 3. Choose n e i t h e r or one (but not both) of two statements, one of which, i f c o r r e c t , would imply the i n c o r r e c t n e s s of other 4. R e s t r i c t c h o i c e to those o p t i o n s which encompass a l l two or more given statements known to be c o r r e c t 5. U t i l i z e r e l e v a n t content i n f o r m a t i o n i n other t e s t items and o p t i o n s I I . Elements dependent upon the t e s t c o n s t r u c t o r or purpose A. Intent c o n s i d e r a t i o n s t r a t e g y 1. I n t e r p r e t and answer q u e s t i o n s i n view of p r e v i o u s i d i o s y n c r a t i c emphases of the t e s t c o n s t r u c t o r of i n view of the t e s t purpose 5 Table 1 (continued) 2. Answer items as the t e s t c o n s t r u c t o r intended 3. Adopt the l e v e l of s o p h i s t i c a t i o n that i s expected 4. Consider the relevance of s p e c i f i c d e t a i l B. Cue-using s t r a t e g y 1. Recognize and make use of any c o n s i s t e n t i d i o s y n c r a s i e s of the t e s t c o n s t r u c t o r which d i s t i n g u i s h the c o r r e c t answer from i n c o r r e c t o p t i o n s a. He makes i t longer ( s h o r t e r ) than the i n c o r r e c t o p t i o n s b. He q u a l i f i e s i t more c a r e f u l l y , or makes i t represent a higher degree of g e n e r a l i z a t i o n c. He i n c l u d e s more f a l s e (true) statements d. He p l a c e s i t i n c e r t a i n p h y s i c a l p o s i t i o n among an ordered set of o p t i o n s (such as the middle of the sequence) e. He p l a c e s i t i n a c e r t a i n l o g i c a l p o s i t i o n among an ordered set of o p t i o n s (such as the middle of the sequence) f . He i n c l u d e s (does not i n c l u d e ) i t among s i m i l a r statements, or makes (does not make) i t one of a p a i r of d i a m e t r i c a l l y o p p o s i t e statements g. He composes (does not compose) i t of f a m i l i a r or st e r e o t y p e d phraseology h. He does not make i t grammatically i n c o n s i s t e n t with the stem 2. Consider the rele v a n c y of s p e c i f i c d e t a i l when answering a given item 3. Recognize and make use of s p e c i f i c determiners 4. Recognize and make use of resemblances between the o p t i o n s and an aspect of the stem 5. Consider the su b j e c t matter and d i f f i c u l t y of nei g h b o r i n g items when i n t e r p r e t i n g and answering a given item Source: Millman et a l . , 1965, p. 707 6 4. Selection Items'. ( a l s o c a l l e d m u l t i p l e - c h o i c e , o b j e c t i v e and s t r u c t u r e d response items) Thorndike and Hagen (1969) d e f i n e d t h i s type of items as "one i n which the examinees must choose h i s answer from the op t i o n s s u p p l i e d by the t e s t maker r a t h e r than producing i t h i m s e l f . T h i s type of item i s a l s o termed ' o b j e c t i v e item'. The form of the o b j e c t i v e items may be a l t e r n a t i v e - r e s p o n s e such as t r u e - f a l s e , m u l t i p l e - c h o i c e with three t o f i v e answer c h o i c e s , or matching" (p. 93). The terms ' m u l t i p l e - c h o i c e ' item and ' s e l e c t i o n - t y p e ' item were used interchangeably i n t h i s study. 5. Canadi an students: They were students who were white Caucasian of European o r i g i n , and who were born i n Canada. 6. Chinese students: They were students whose e t h n i c o r i g i n was Chinese, who were not born i n Canada and who came to study i n Canada on a student v i s a . They were a l s o c a l l e d f o r e i g n or i n t e r n a t i o n a l s t u d e n t s . T h e i r p l a c e of o r i g i n might be Hong Kong, or other Southeast Asian c o u n t r i e s . 1.3 Background of The Problem Dating back to Thorndike's (1951) o r i g i n a l work i n t e s t r e l i a b i l i t y , l a c k of c o n s i s t e n c y of t e s t scores was con s i d e r e d as one source of e r r o r v a r i a n c e i n t e s t s c o r e s . A seminal work i n test-wis e n e s s was the taxonomy developed by Millman et a l . (1965) (see Table 1). They d e f i n e d test-wiseness as "a 7 student's c a p a c i t y to u t i l i z e the c h a r a c t e r i s t i c s and formats of the t e s t and/or t e s t - t a k i n g s i t u a t i o n to r e c e i v e a high s c o r e " (p. 707). Sarnacki (1979) f u r t h e r p o i n t e d out that t e s t - w i s e n e s s c o u l d p o s s i b l y depress the v a l i d i t y of t e s t score because d i f f e r e n t students possess d i f f e r e n t l e v e l s of t e s t - w i s e n e s s . Test-wiseness may gi v e some students an u n f a i r advantage on examinations, p a r t i c u l a r l y i f the items i n the examinations are s u s c e p t i b l e to t e s t - w i s e n e s s . Since the development of the taxonomy of t e s t - w i s e n e s s , c o n s i d e r a b l e a t t e n t i o n has been given t o t h i s c o n s t r u c t . However, as Scruggs and L i f s o n (1985) p o i n t e d out, r e l a t i v e l y l i t t l e r e s e a r c h focused on i d e n t i f y i n g e t h n i c group d i f f e r e n c e s i n t e s t - w i s e n e s s . O r i g i n a l l y developed among the white Caucasian p o p u l a t i o n , i t has not been determined whether t h i s c o n s t r u c t c o u l d be a p p l i e d to other e t h n i c groups. Educators have provided c o n s i d e r a b l e documentation of the poor performance of m i n o r i t y groups on s t a n d a r d i z e d achievement t e s t s , and have suggested that t h i s f i n d i n g i s a f u n c t i o n of a la c k of test-wi s e n e s s ( A n a s t a s i & Cordova, 1953; K a l e c h s t e i n , K a l e c h s t e i n & Docter, 1981; G o s l i n , E p s t e i n & H a l l o c h , 1965). I t i s p o s s i b l e that the m i n o r i t y groups are l e s s t e s t - w i s e , and i t i s a l s o p o s s i b l e that the taxonomy cannot be g e n e r a l i z e d to m i n o r i t y p o p u l a t i o n . G e n e r a l l y , the a v a i l a b l e r e s e a r c h on c u l t u r a l d i f f e r e n c e i n tes t - w i s e n e s s was i n c o n c l u s i v e (e.g., Lo & S l a k t e r , 1973; Wu & S l a k t e r , 1978). F u r t h e r r e s e a r c h with m i n o r i t y p o p u l a t i o n s i s r e q u i r e d i n 8 order to shed l i g h t on the g e n e r a l i z a b i l i t y of tes t - w i s e n e s s to other p o p u l a t i o n s . Canada i s a p l u r a l i s t i c s o c i e t y that i s made up of a l a r g e p r o p o r t i o n of immigrants. From the census data i n 1986, Chinese immigrants from Hong Kong i n B r i t i s h Columbia ranked second i n number a f t e r East Indian immigrants. In F i s h e r and E c h o l s ' (1989) study on e t h n i c r e l a t i o n s i n the Vancouver School Board, the second l a r g e s t e t h n i c group i n Vancouver was Chinese . Apparently, one of the dominant m i n o r i t y groups i n the schools i n B r i t i s h Columbia, p a r t i c u l a r l y Vancouver, i s Chinese. Despite a l l the c r i t i c i s m s surrounding the p r o v i n c i a l examinations, i t s e x i s t e n c e i s a matter of r e a l i t y . Every Grade 12 student who wishes to graduate must w r i t e at l e a s t the E n g l i s h 12 or Communications 12 examination. I t i s t h e r e f o r e the r e s p o n s i b i l i t y of educators to ensure that the examination program i s not abused and i s f a i r to a l l students. A p r e l i m i n a r y study (Rogers and Bateson, 1990a) i n d i c a t e d t hat a s i g n i f i c a n t number of t e s t items i n the p r o v i n c i a l examinations were s u s c e p t i b l e to test-wi s e n e s s s t r a t e g i e s . T h i s leads t o concerns r e g a r d i n g the i n t e r p r e t a t i o n of the scores s i n c e d i f f e r e n c e s i n tes t - w i s e n e s s a b i l i t i e s may u n f a i r l y b i a s the scores f o r c e r t a i n groups of students. I f t e s t - w i s e students tended to make b e t t e r use of t e s t item flaws or secondary cues to choose t h e i r answers, t e s t - n a i v e students would be c o m p e t i t i v e l y disadvantaged. Such 9 d i f f e r e n c e s i n t e s t - t a k i n g behavior may l e a d to i n c o r r e c t assessment and d e c i s i o n s about the students' a b i l i t y . 1.4 Statement of The Problem Suggested by Rogers and Bateson (1990a), students brought d i f f e r e n t l e v e l s of t e s t - w i s e n e s s to the p r o v i n c i a l examinations. To examine t h i s c l a i m , t h i s study i n v e s t i g a t e d the r e l a t i o n s h i p of t e s t - w i s e n e s s and performance on m u l t i p l e c h o i c e q u e s t i o n s among Chinese students i n B r i t i s h Columbia. The problem addressed i n t h i s study was whether Chinese students s i g n i f i c a n t l y would d i f f e r from Canadian students on t e s t - w i s e n e s s s k i l l s . Since the p r o v i n c i a l examinations had an important impact on the students, t h i s study a l s o i n v e s t i g a t e d the degree to which test-wiseness i n f l u e n c e d the i n t e r p r e t a t i o n of t e s t scores r e c e i v e d by Chinese students on the Grade 12 p r o v i n c i a l examinations. A l s o , whether or not Chinese students d i f f e r e d from the Canadian students i n t e s t -wiseness s k i l l s was examined. F i n a l l y , t h i s study e x p l o r e d the r e l a t i o n s h i p between test-w i s e n e s s and the demographic c h a r a c t e r i s t i c s and the backgrounds of the Chinese students. More s p e c i f i c a l l y , the r e s e a r c h q u e s t i o n s i n v e s t i g a t e d were as f o l l o w s : 1. Do Canadian and Chinese students d i f f e r i n t h e i r scores on t e s t - w i s e n e s s ? 2. Do the t e s t scores of the Chinese students on m u l t i p l e -c h o i c e items i n the p r o v i n c i a l examinations i n P h y s i c s , 10 A l g e b r a , H i s t o r y and/or E n g l i s h r e l a t e to t e s t wiseness; and i f so, what i s the s t r e n g t h of the r e l a t i o n s h i p ? 3. Do the scores on tes t - w i s e n e s s among the Chinese students r e l a t e to t h e i r l e n g t h of re s i d e n c y i n Canada, experience with m u l t i p l e - c h o i c e items, v e r b a l a b i l i t y (measured by the Language P r o f i c i e n c y Index), and achievement m o t i v a t i o n ? 1.5 Research Hypotheses of The Study T h i s study hypothesized that s u b j e c t s ' t e s t scores on the s e l e c t i o n items i n P h y s i c s , A l g e b r a , H i s t o r y , and/or E n g l i s h examinations of the B r i t i s h Columbia P r o v i n c i a l Examinations would i n c r e a s e as t h e i r t e s t s cores i n the Test of T e s t -wiseness i n c r e a s e d . The n u l l - f o r m s t a t i s t i c a l hypotheses were d e s c r i b e d i n Chapter 3. The res e a r c h hypotheses are s t a t e d as f o l l o w s : 1.51 Major Hypothesis'. The performance of the f o r e i g n Chinese students on the p r o v i n c i a l examinations i n H i s t o r y , E n g l i s h , A l g e b r a , and/or P h y s i c s i s a f u n c t i o n of t e s t - w i s e n e s s , and four r e l a t e d v a r i a b l e s , namely the Language P r o f i c i e n c y Index, l e n g t h of res i d e n c y i n Canada, achievement m o t i v a t i o n and m u l t i p l e c h o i c e e x p e r i e n c e . 1.52 Hypothesis One F o r e i g n Chinese students are not s i g n i f i c a n t l y d i f f e r e n t from Canadian students i n t h e i r scores of te s t - w i s e n e s s , as 11 measured by the Test of Test-wiseness developed by Rogers and Bateson (1990b). 1.53 Hypothesis Two (a) The scores on the m u l t i p l e - c h o i c e items i n the P h y s i c s , A l g e b r a , H i s t o r y , and/or E n g l i s h examinations of the f o r e i g n Chinese students who possess t e s t - w i s e n e s s s k i l l s and a b i l i t i e s are s i g n i f i c a n t l y d i f f e r e n t from those of the f o r e i g n Chinese students who possess l e s s of the same a b i l i t i e s . 1.54 Hypothesis Two (b) The v a r i a b i l i t y of the scores on the m u l t i p l e - c h o i c e items i n the P h y s i c s , A l g e b r a , H i s t o r y , and/or E n g l i s h examinations of the f o r e i g n Chinese students who possess t e s t -wiseness s k i l l s and a b i l i t i e s i s s i g n i f i c a n t l y d i f f e r e n t from that of the f o r e i g n Chinese students who possess l e s s of the same a b i l i t i e s . 1.55 Hypothesis Three The scores on tes t - w i s e n e s s of the f o r e i g n Chinese students s i g n i f i c a n t l y i n c r e a s e as the f o l l o w i n g v a r i a b l e s i n c r e a s e , and the f o l l o w i n g v a r i a b l e s account f o r a s i g n i f i c a n t amount of v a r i a n c e on t h e i r scores of t e s t -wiseness : 1 . Language Proficiency Index (LPI); 2. Length of Residency in Canada (RES); 3. Multiple Choice Experience (MCE); and 4. Achievement Motivation (AM). 12 1.6 R a t i o n a l e of The Hypotheses Test-wiseness has been c o n s i d e r e d to be one source of e r r o r v a r i a n c e and b i a s on t e s t s (Benson, 1988; Crehan, Koehler & S l a k t e r , 1974; K a l e c h s t e i n et a l . , 1981). P r e l l and P r e l l (1986) p a r t i c u l a r l y s t a t e d that the presence of t e s t -wiseness cues on a t e s t might reward hi g h t e s t - w i s e students and p e n a l i z e low t e s t - w i s e students d i f f e r e n t i a l l y . Furthermore, c u l t u r a l l y d i f f e r e n t c h i l d r e n were o f t e n e v a l u a t e d by school p r o f e s s i o n a l s p r i m a r i l y on the b a s i s of s t a n d a r d i z e d t e s t score r e s u l t s ( K a l e c h s t e i n et a l . , 1981). I f these t e s t s were s u s c e p t i b l e to t e s t - w i s e n e s s , any e v a l u a t i o n s of these c h i l d r e n should be preceded by an understanding of the antecedent v a r i a b l e s that c o n t r i b u t e d to t e s t score v a r i a n c e . Since the p r o v i n c i a l examinations have p e r v a s i v e i n f l u e n c e on students' f u t u r e , i t i s imperative to f i n d out i f test-wiseness has a s i g n i f i c a n t impact on the r e s u l t s of the p r o v i n c i a l examinations. Apart from being one source of e r r o r v a r i a n c e and b i a s on t e s t s , s t u d i e s (Benson, 1988; Crehan et a l . , 1974; K a l e c h s t e i n et a l . , 1981; S a r n a c k i , 1979) a l s o suggested that i f the e f f e c t s of test-wiseness were somehow reduced or e l i m i n a t e d , v a r i a b i l i t y on t e s t s due to t h i s f a c t o r would be reduced or e l i m i n a t e d . However, Gibb (1964) and S l a k t e r , Koehler & Hampton (1970) seemed to h o l d an opposite view. Since arguments on i n t e r - g r o u p v a r i a b i l i t y of t e s t - w i s e n e s s had not been w e l l - e s t a b l i s h e d , t h i s study attempted to explore i f the score v a r i a b i l i t y of the students with higher l e v e l s of t e s t -13 wiseness would be d i f f e r e n t from those who possessed l e s s of t h i s s k i l l . Chinese students i n the U.S.A. were c o n s i s t e n t l y demonstrated i n re s e a r c h to be l e s s t e s t - w i s e than American students (Lo & S l a k t e r , 1973; Wu & S l a k t e r , 1978). T h i s study i n v e s t i g a t e d i f the same r e l a t i o n s h i p would h o l d t r u e between Chinese and Canadian s t u d e n t s . I f so, would the r e s u l t of the Chinese students on p r o v i n c i a l examinations be a f f e c t e d because they were l e s s t e s t - w i s e ? Furthermore, i n two s t u d i e s (Lo & S l a k t e r , 1973; Wu & S l a k t e r , 1978), Chinese students tended to adopt d i f f e r e n t t e s t - w i s e n e s s s t r a t e g i e s from the American students i n attempting q u e s t i o n s . T h i s study f u r t h e r e x p l o r e d i f the Chinese students would behave s i g n i f i c a n t l y d i f f e r e n t from the Canadian students i n t e s t - t a k i n g s i t u a t i o n s . Test-wiseness was found to be c o r r e l a t e d with s e v e r a l f a c t o r s . Based on some r e s e a r c h f i n d i n g s , four c o r r e l a t e s were s e l e c t e d i n t h i s study to f i n d out i f they were r e l a t e d to the l e v e l of te s t - w i s e n e s s among Chinese students. The c o r r e l a t i o n of te s t - w i s e n e s s and v e r b a l a b i l i t i e s has been documented i n s e v e r a l s t u d i e s , such as A r d i f f . ( 1 9 6 5 ) ; Benson (1988); Diamond and Evans (1972) and Sarnacki (1979). In t h i s study, E n g l i s h was not the f i r s t language of most of the r e c e n t l y immigrated Chinese. T h e r e f o r e , the r e l a t i o n s h i p between test-w i s e n e s s and v e r b a l a b i l i t y (as measured by the Language P r o f i c i e n c y Index) was examined. Furthermore, Samuda (1975) p o i n t e d out that s t a n d a r d i z e d t e s t s , with the exception 14 of performance or nonverbal t e s t s , were h e a v i l y v e r b a l l y loaded. T h e r e f o r e , f a m i l i a r i t y with the language of the t e s t would l i k e l y be a b e n e f i t to t e s t performance. In a d d i t i o n , A n a s t a s i and Cordova (1953) found that c h i l d r e n who were i n the process of l e a r n i n g E n g l i s h scored c o n s i s t e n t l y lower than those who had a l r e a d y mastered the language used on i n t e l l i g e n c e t e s t s . T h e r e f o r e , language i s an important component i n determining the t e s t performance of immigrant c h i l d r e n . Another f a c t o r that might a f f e c t the l e v e l and use of te s t - w i s e n e s s of the Chinese immigrant students i s the time s i n c e a r r i v a l i n Canada. T h i s f a c t o r was a l s o examined. The l e n g t h of r e s i d e n c y i n Canada would r e l a t e to t h e i r mastery of E n g l i s h and f a m i l i a r i t y with the school c u r r i c u l u m i n B r i t i s h Columbia which i n t u r n , would a f f e c t t h e i r t e s t performance. Research on t e s t - w i s e n e s s and m i n o r i t y p o p u l a t i o n s has demonstrated that t e s t scores were h i g h l y a f f e c t e d by t e s t -t a k i n g s k i l l s ( K a l e c h s t e i n et a l . , 1981). A m a j o r i t y of the Chinese students i n B r i t i s h Columbia were immigrants from Hong Kong. They a r r i v e d at Canada at d i f f e r e n t times and have had d i f f e r i n g e xperiences with examinations, p a r t i c u l a r l y s e l e c t i o n - t y p e examinations. P r i o r experience with s e l e c t i o n -type items has been found to be c o r r e l a t e d with t e s t - w i s e n e s s ( S a r n a c k i , 1979). Another f a c t o r that may be r e l a t e d to the l e v e l of t e s t -wiseness among Chinese students i s achievement m o t i v a t i o n . E e l s , David, Havighurst, H e r r i c k and T y l e r (1951) demonstrated t h a t C a u c a s i a n c h i l d r e n from low soc i o e c o n o m i c backgrounds d i s p l a y e d d i f f e r e n t problem s o l v i n g s t y l e s because the m i n o r i t y c h i l d r e n l a c k e d s k i l l s i n g u e s s i n g s t r a t e g i e s , a p p r o p r i a t e v o c a b u l a r y , and m o t i v a t i o n t o s u c c e s s . Samuda (1975) f u r t h e r p o i n t e d out t h a t a pe r s o n ' s need t o e x c e l a c a d e m i c a l l y was i t s e l f a f u n c t i o n of e t h n i c i t y , s o c i a l c l a s s and t e s t - t a k i n g e x p e r i e n c e . T h e r e f o r e , c u l t u r a l d i f f e r e n c e s i n m o t i v a t i o n cannot be o v e r l o o k e d . O f t e n i g n o r e d by contemporary p s y c h o m e t r i c i a n s , Dubois (1964) p o i n t e d out t h a t the C h i n e s e had p r o l o n g e d and i n t e n s i v e e x p e r i e n c e w i t h t e s t i n g . C hinese c u l t u r e has a l o n g t r a d i t i o n of heavy emphasis on e d u c a t i o n a l achievement. C i v i l s e r v i c e e x a m i n a t i o n s i n C h i n a were t r a d i t i o n a l l y the main means t o a c a r e e r and hence t o s o c i a l advancement. Ch i n e s e immigrants from Hong Kong a r e p a r t i c u l a r l y m o t i v a t e d f o r p u b l i c e x a m i n a t i o n s , which i s the major venue f o r advancement i n c a r e e r or h i g h e r e d u c a t i o n i n Hong Kong. 1.7 S e l e c t i o n and Measurement of Independent V a r i a b l e s T h i s study examined i f t e s t - w i s e n e s s and f o u r c o r r e l a t e s , namely v e r b a l a b i l i t y , e x p e r i e n c e w i t h w r i t i n g m u l t i p l e - c h o i c e e x a m i n a t i o n s , l e n g t h of r e s i d e n c y i n Canada, and achievement m o t i v a t i o n would s i g n i f i c a n t l y a f f e c t the s t u d e n t s ' s c o r e s on p r o v i n c i a l e x a m i n a t i o n s . T e s t - w i s e n e s s and the f o u r c o r r e l a t e s a r e the independent v a r i a b l e s i n the major h y p o t h e s i s . S i n c e t h i s s t u d y a l s o i n v e s t i g a t e d i f Canadian and Chin e s e s t u d e n t s would d i f f e r s i g n i f i c a n t l y on t e s t - w i s e n e s s , 16 t e s t - w i s e n e s s was a l s o used as the dependent v a r i a b l e and e t h n i c i t y became the independent v a r i a b l e i n Hypothesis One. S i m i l a r l y , to examine i f the four c o r r e l a t e s would c o n t r i b u t e a s i g n i f i c a n t amount of v a r i a n c e i n t e s t - w i s e n e s s , the four c o r r e l a t e s were used as the independent v a r i a b l e s i n Hypothesis Three. Students' l e v e l of t e s t - w i s e n e s s was measured by the Test of Test-wiseness developed by Rogers and Bateson (1990b). The scores of the four c o r r e l a t e s were obtained from the Student Survey q u e s t i o n n a i r e . The q u e s t i o n n a i r e was made up of four s e c t i o n s : (I) background i n f o r m a t i o n on the students' e t h n i c o r i g i n and p r i o r experience i n w r i t i n g m u l t i p l e - c h o i c e examinations; (II) an achievement m o t i v a t i o n t e s t measuring t h e i r d e s i r e to achieve and compete; ( I I I ) sentence s t r u c t u r e and (IV) E n g l i s h usage measuring t h e i r v e r b a l a b i l i t y . Based on the students' responses and i n f o r m a t i o n on the Test of Test-wiseness and the Student Survey, t h i s study i n v e s t i g a t e d the r e s e a r c h q u e s t i o n s addressed above. 1.8 S e l e c t i o n of Dependent V a r i a b l e s As d e s c r i b e d i n s e v e r a l s t u d i e s ( A r d i f f , 1965; Diamond & Evans, 1972), te s t - w i s e n e s s was found to be r e l a t e d to v e r b a l a b i l i t y . The E n g l i s h t e s t should t h e r e f o r e be s u s c e p t i b l e to s k i l l s i n t e s t - w i s e n e s s . The E n g l i s h t e s t , though not c o n t a i n i n g the l a r g e s t numbers of items with t e s t - w i s e n e s s cues i n the p r e v i o u s p r o v i n c i a l examinations, would i n s u r e a r e p r e s e n t a t i v e sample of students from the d i s t r i c t s sampled, 17-because a l l students i n B r i t i s h Columbia wishing to graduate from high school must w r i t e e i t h e r the E n g l i s h 12 or Communications 12 p r o v i n c i a l examination. Most students e l e c t to w r i t e E n g l i s h 12 r a t h e r than Communications 12. The H i s t o r y examination was i n v e s t i g a t e d because i t a l s o i n v o l v e d a s t r o n g v e r b a l component. The P h y s i c s and Algebra examinations were a l s o i n v e s t i g a t e d . These two s u b j e c t areas were found to c o n t a i n a c o n s i d e r a b l e number of items s u s c e p t i b l e to test-wiseness s k i l l s (Rogers and Bateson, 1990a). Moreover, a l a r g e p r o p o r t i o n of Chinese students p a r t i c i p a t e i n these two examinations, thus i n s u r i n g a l a r g e r sample of Chinese f o r e i g n students. 1.9 J u s t i f i c a t i o n of the Study F i n d i n g s i n t h i s study should shed l i g h t on the impact of test- w i s e n e s s among d i f f e r e n t e t h n i c groups on p r o v i n c i a l government examinations. T h i s impact i s important to understand, p a r t i c u l a r l y i n view of the important d e c i s i o n s that are made based on scores from these examinations. T h i s study may improve the understanding of t e s t - t a k i n g behavior and the i n f l u e n c e of t e s t - w i s e n e s s upon d i f f e r e n t e t h n i c groups, p a r t i c u l a r l y i n m u l t i p l e - c h o i c e items. A l s o , r e s u l t s w i l l p r o v i d e t e s t c o n s t r u c t o r s with u s e f u l i n f o r m a t i o n that c o u l d be used to minimize the flaws of the p r o v i n c i a l examinations, l e s s e n the p o s s i b l e b i a s f o r c e r t a i n e t h n i c groups, and to design m a t e r i a l s to i n s t r u c t students on t e s t -t a k i n g s k i l l s . 18 Furthermore, t h i s study e x p l o r e d whether test-w i s e n e s s or r e l a t e d t e s t - t a k i n g behavior would be c u l t u r a l l y r e l a t e d . Research on e t h n i c groups with a Chinese background seems p r e s s i n g because such students make up a l a r g e p r o p o r t i o n of the student p o p u l a t i o n i n B r i t i s h Columbia, p a r t i c u l a r l y i n Vancouver. Other p l a u s i b l e c o r r e l a t e s of t e s t - w i s e n e s s , such as achievement m o t i v a t i o n , v e r b a l a b i l i t y , p r i o r t e s t - t a k i n g experience, e t c . c o u l d be c u l t u r a l l y o r i e n t e d or i n f l u e n c e d and c o u l d hence i n f l u e n c e the students' t e s t - t a k i n g behavior or l e v e l of t e s t - w i s e n e s s . A c r o s s - c u l t u r a l study of t e s t -wiseness may c l a r i f y t h i s p o s t u l a t i o n . 19 CHAPTER TWO LITERATURE REVIEW 2.1 B r i t i s h Columbia P r o v i n c i a l Examinations 2.11 Hi st orical Cont ext : P r o v i n c i a l examinations i n B r i t i s h Columbia have e x i s t e d i n v a r i o u s forms s i n c e the opening of the f i r s t h igh school i n the p r o v i n c e . The f i r s t t e s t s were admi n i s t e r e d i n 1876 to i d e n t i f y those students who were a c a d e m i c a l l y ready f o r high s c h o o l . Four s u b j e c t s areas were t e s t e d : a r i t h m e t i c , grammar, s p e l l i n g and geography. In 1876-1924, government examinations were used to s e l e c t students f o r high s c h o o l , monitor t h e i r p rogress through high school and to assess teacher e f f i c i e n c y on the b a s i s of t e s t r e s u l t s (Calam, 1980). P r e v i o u s l y , h i g h school p r i n c i p a l s had the a u t h o r i t y to grant a High School Diploma, but s t a r t i n g i n 1883, students were r e q u i r e d to pass the annual Departmental Examinations i n order to graduate (Superintendent of Edu c a t i o n , 1898). Requested by the p r o v i n c i a l government i n 1924 to survey the B.C. school system, Putnam and Weir (1925) recommended that students should be abl e to enter high school s o l e l y on the b a s i s of t h e i r teachers' recommendations because t e a c h i n g standards and teacher q u a l i f i c a t i o n s had improved over the pr e v i o u s 25 y e a r s . F u r t h e r , they c r i t i c i z e d the misuses of the examinations, such as l e a d i n g teachers to prepare f o r f i n a l t e s t s r a t h e r than to teach the f u l l c u r r i c u l u m , overemphasizing examination s u b j e c t s while i g n o r i n g other 20 areas, c ausing undue grade r e t a r d a t i o n , and encouraging weaker students to withdraw from school before g r a d u a t i o n . In 1931, teachers were granted a u t h o r i t y to promote students up to Grade 11. Grade 12 students were s t i l l r e q u i r e d to w r i t e and pass f i n a l p r o v i n c i a l examinations i n order to graduate. In 1937, the requirement that a l l Grade 12 students wrote p r o v i n c i a l examinations was m o d i f i e d ; those students a t t e n d i n g an a c c r e d i t e d high s c h o o l , and who had obtained a C+ standing or b e t t e r , were recommended f o r graduation without w r i t i n g the p r o v i n c i a l examinations. However, non-recommended students and students a t t e n d i n g non-a c c r e d i t e d high schools were s t i l l r e q u i r e d to w r i t e and pass Departmental Examinations i n order to graduate. In 1946, the new D i v i s i o n of T e s t s , Standards and Measurement was c r e a t e d in the Education M i n i s t r y to oversee and administer the Grade 12 and p r o v i n c i a l achievement t e s t i n g programs. Support f o r p r o v i n c i a l examinations was seen again i n the 1960 Royal Commission. Report on Education (Chant, L i e r s c h & Walrod, 1960). The r e p o r t s t a t e d that examinations would help to maintain s a t i s f a c t o r y progress i n s c h o o l , p rovide i n c e n t i v e fo r a b l e students to work harder, and e s t a b l i s h a more uniform standard of i n s t r u c t i o n throughout the p u b l i c school system. The Commission Report paved the way f o r more changes to the examination program. The U n i v e r s i t y Programme Departmental Examinations at Grades 11 and 12 were r e p l a c e d by p r o v i n c i a l examinations i n Grade 12 courses d u r i n g 1965/66 and 1966/67. These examinations were set and marked by the same 21 procedures p r e v i o u s l y used f o r the U n i v e r s i t y Program Examinations. The marks from these examinations were combined with school marks so that each component accounted f o r 50% of the f i n a l mark. In 1969, the Departmental Examinations were e n t i r e l y made up of s e l e c t i o n - t y p e items so t h a t they c o u l d be e a s i l y scored by computer. During t h i s p e r i o d , s e r i o u s concerns were r a i s e d about the worth of school l e a v i n g , or Grade 12 examinations. In 1973, the r e g u l a r Departmental Examinations were d i s c o n t i n u e d and were r e p l a c e d by the S c h o l a r s h i p Examination Program, supplemented by the P r o v i n c i a l L e a r n i n g Assessment Program i n 1976/77 and the Classroom Achievement Test Program i n 1980. The S c h o l a r s h i p Examination Program was designed f o r students who wished to be c o n s i d e r e d f o r government s c h o l a r s h i p s . They would w r i t e the examinations set f o r the s e n i o r courses i n which they were e n r o l l e d . The P r o v i n c i a l L earning Assessment Program was e s t a b l i s h e d to a d v i s e the M i n i s t r y on the implementation of province-wide assessments, with p a r t i c u l a r r e f e r e n c e to school achievement data, a l l o c a t i o n of resources, c u r r i c u l u m r e v i s i o n , e t c . The Classroom Achievement Test Program was i n t r o d u c e d to a s s i s t teachers i n a s s i g n i n g f i n a l grades and d i a g n o s i n g s t r e n g t h and weaknesses of the students. Despite the e x i s t e n c e of the new a s s e s s i n g programs, the i s s u e of Grade 12 examinations s t i l l l i n g e r e d on. Concerned with what was p e r c e i v e d to be d e c l i n i n g standards, the Minster of Education r e - i n t r o d u c e d province-wide examinations at the end of the 1982/83 school year. Only a l i m i t e d v e r s i o n of a 22 f u l l examination program was implemented i n the f i r s t year. In September of the 1983/84 school year, the M i n i s t e r of Education announced that Grade 12 Departmental Examinations would be adm i n i s t e r e d a n n u a l l y on a province-wide b a s i s to count f o r 50% of a student's f i n a l s t a n d i n g . The reasons f o r t h i s d e c i s i o n were to maintain a c o n s i s t e n t standard of high school graduates, ensure an equal admission c r i t e r i a to a c o l l e g e or u n i v e r s i t y , and reassure the p u b l i c of the q u a l i t y of the p u b l i c s c hools ( H e i n r i c h , 1983). Since the 1983/84 school year, the p r o v i n c i a l examinations have been a d m i n i s t e r e d three times a year - i n January f o l l o w i n g f i r s t semester courses; i n June f o l l o w i n g the completion of second-semester and f u l l year courses; and i n August f o r students who wish to w r i t e a supplemental examination to improve a course s t a n d i n g and f o r students who were unable to w r i t e e i t h e r of the pr e v i o u s two due to extenuating circumstances. Examinations i n 15 courses are o f f e r e d : E n g l i s h 12, Communications 12, Algebra 12, Bi o l o g y 12, P h y s i c s 12, Chemistry 12, Geology 12, L i t e r a t u r e 12, Geography 12, H i s t o r y 12, French 12, F r a n c a i s - l a n g u e 12, German 12, Spanish 12, and L a t i n 12. To put more emphasis on teach e r s ' assessment, the weight of these examinations has been reduced to 40% i n the c a l c u l a t i o n of f i n a l course marks s i n c e the 1989/90 school year. 2.12 The Examination-Development Process: The c u r r e n t p r o v i n c i a l examinations are based on the p r o v i n c i a l c u r r i c u l u m guides and i n v o l v e hundreds of teachers 23 i n t r a n s l a t i n g c u r r i c u l a g o a l s , o b j e c t i v e s , and i n t e n t s i n t o a c t u a l examinations q u e s t i o n s . The f i r s t s tep i n the process i s the p r o d u c t i o n of a t a b l e of examination s p e c i f i c a t i o n s which p r o v i d e s teachers and students with an i n d i c a t i o n of the t o p i c s and c o g n i t i v e l e v e l s to be covered by each examination. Every year, about 10 t e a c h e r s h e l p review and e s t a b l i s h the t a b l e of s p e c i f i c a t i o n s f o r each course. A f t e r the t a b l e of s p e c i f i c a t i o n s has been e s t a b l i s h e d , the g u i d e l i n e s are transformed i n t o January, June, and August examinations by the p r e p a r a t i o n committees c o n s i s t i n g of experienced and q u a l i f i e d t e a c h e r s . The f i r s t d r a f t s of the examinations are e d i t e d by the s t a f f of the Student Assessment Branch of the M i n i s t r y of Education, and reviewed by examination c o - o r d i n a t o r s and measurement s p e c i a l i s t s . The review process i s repeated u n t i l consensus i s reached on the content of items and of the examination as a whole ( C a r b o l , 1987). The s t a n d a r d - s e t t i n g process of the p r o v i n c i a l examinations r e l i e s on the judgements of expert o p i n i o n , namely classroom t e a c h e r s . Teachers are employed to mark the examinations and to determine the standards or c u t - s c o r e s that w i l l represent student performance along the range of l e t t e r grades (A through F ) . For i n s t a n c e , markers may decide that 76/100 should be the a c c e p t a b l e standard f o r an A standing on the p a r t i c u l a r examination they are marking. Then, the markers proceed to score the w r i t t e n p o r t i o n of the 24 examinations. When they have almost completed, they are given a new set of i n f o r m a t i o n on the r e s u l t s of the i n i t i a l s t a n d a r d - s e t t i n g s e s s i o n . With the new i n f o r m a t i o n , some markers may change t h e i r o r i g i n a l standards f o r c u t - s c o r e s . A f t e r the adjustment, marker standards are compared to teacher-awarded mark d i s t r i b u t i o n s , and a midpoint between the two d i s t r i b u t i o n s i s found. The r e v i s e d standards become the b a s i s f o r the f i n a l d i s t r i b u t i o n of examination marks presented to the P r o v i n c i a l Board of Examiners. The board •reviews the"exam-mark and teacher-mark d i s t r i b u t i o n s and makes any necessary f i n a l adjustment to the examination d i s t r i b u t i o n . F i n a l l y , student examination marks are then s c a l e d to r e f l e c t a c c u r a t e l y the adopted d i s t r i b u t i o n s ( C a r b o l , 1987). The f i n a l s c hool marks i n courses that have p r o v i n c i a l examinations are transformed from raw scores and r e p o r t e d as percentages as shown i n Table 2. According to the B.C. M i n i s t r y of Education, t h e i r i n t e n t i s not to set pre-determined percentage that students must o b t a i n i n order to a t t a i n c e r t a i n l e t t e r grade. The standard s e t t i n g procedure i s intended to s t a n d a r d i z e the r e p o r t i n g of d i f f e r e n t l e v e l s of l e t t e r grades d u r i n g the school year a c c o r d i n g to the d i f f i c u l t y of t e s t s and other c o n s i d e r a t i o n s ( B r i t i s h Columbia M i n i s t r y of Education, 1989). Table 2 L e v e l of Achievement and the E q u i v a l e n t Reported Percentages of the B r i t i s h Columbia P r o v i n c i a l Examinations L e v e l of Achievement Reported Range of Scores A ( E x c e l l e n t ) 86 100 B (Very Good) 73 85 C+ ( S a t i s f a c t o r y ) 67 72 C ( S a t i s f a c t o r y ) 60 66 P (Passing) 50 59 F ( F a i l i n g ) below 50 Source: B r i t i s h Columbia M i n i s t r y of Education, i~989. G e n e r a l l y , the markers are experienced teachers who may a l s o s i t on s p e c i f i c a t i o n committees, s e t t i n g committees, and/or review committees f o r the examinations. To date, over 2,200 d i f f e r e n t teachers have been i n v o l v e d i n v a r i o u s aspects of the program, from developing t e s t items to marking examinations. Hence, the development of p r o v i n c i a l examinations can be c o n s i d e r e d as a t e a c h e r - c e n t e r e d process ( C a r b o l , 1989). These teachers make judgements about the l e t t e r - g r a d e , and t h e r e f o r e the s c a l i n g c u t - s c o r e s , based on the raw scores of the examinations. The raw scores are i n t e r p r e t e d to be r e f l e c t i v e of students' knowledge of the s u b j e c t under examination. However, to the extent that the raw scores may be r e f l e c t i v e of a b i l i t y to employ test-w i s e n e s s s t r a t e g i e s r a t h e r than 26 r e f l e c t i v e of knowledge of the su b j e c t matter, the v a l i d i t y of these i n t e r p r e t a t i o n s may be a f f e c t e d . 2.13 Impact of the Provincial Examinations'. Although the p r o v i n c i a l examinations are w e l l -c o n s t r u c t e d , there i s s t i l l a c o n t i n u a l debate over t h e i r worthwhile and adverse e f f e c t s . P r o v i n c i a l examination means have become the standard a g a i n s t which student, school and d i s t r i c t performance i s compared. F u r t h e r , the p r o v i n c i a l examinations t e s t only a s p e c i f i c p o r t i o n of in f o r m a t i o n b a s i c a l l y at a s u p e r f i c i a l l e v e l of understanding of the concepts (Anderson, Muir, Bateson, Blackmore, and Rogers, 1990). Development of the c u r r e n t forms of the p r o v i n c i a l examinations has c a r e f u l l y c o n s i d e r e d both the l e v e l of q u e s t i o n i n g and the format of the q u e s t i o n s . Yet, many of the qu e s t i o n s have been found to be s u s c e p t i b l e to test-wis e n e s s s t r a t e g i e s (Rogers and Bateson, 1990a) due to both flaws of item c o n s t r u c t i o n and the b a s i c nature of m u l t i p l e - c h o i c e q u e s t i o n s . Since the p r o v i n c i a l examinations are so c r u c i a l to the high school graduate, t h e i r p o s s i b l e r e l a t i o n s h i p with t e s t - w i s e n e s s deserves f u r t h e r a t t e n t i o n . 2.2 Concept of Test-wiseness 2.21 Definition of Test-wiseness The concept of test-wis e n e s s was f i r s t i n t r o d u c e d by Thorndike (1951). He c o n s i d e r e d t e s t - w i s e n e s s as "general a b i l i t y to comprehend i n s t r u c t i o n ... [and i s ] l i k e l y to enter i n t o any t e s t s c o r e " (p. 588), c o n t r i b u t i n g to one 27 source of v a r i a n c e i n t e s t r e s u l t s . Thorndike (1951) argued that t e s t score was a f u n c t i o n of the extent to which the i n d i v i d u a l was f a m i l i a r with t e s t s and had the degree of i n s i g h t with regards to t r i c k s of t a k i n g them. T h e r e f o r e , f a c t o r s , such as "freedom from emotional t e n s i o n , shrewdness with regard to when to guess, and a keen eye f o r secondary and extraneous cues" (p. 569) would be u s e f u l i n many t e s t s , p a r t i c u l a r l y p o o r l y c o n s t r u c t e d ones. In a d d i t i o n to the v a r i a n c e which was common i n a l l t e s t s , Thorndike (1951) p o i n t e d out that each t e s t had some v a r i a n c e which came from the p e r s i s t e n t c h a r a c t e r i s t i c s of the i n d i v i d u a l s , namely t e s t - w i s e n e s s . Such c h a r a c t e r i s t i c s cannot be ignored because they represent "systematic i n v a l i d v a r i a n c e s e r v i n g s y s t e m a t i c a l l y to reduce the v a l i d i t y of the t e s t " (Thorndike, 1951, p. 569). In Thorndike's view, t e s t -wiseness p r e s e n t s a problem of v a l i d i t y r a t h e r than r e l i a b i l i t y because i t r e p r e s e n t s q u a l i t i e s of the i n d i v i d u a l that w i l l a f f e c t performance t e s t s at a p a r t i c u l a r time. Moreover, i n s t e a d of being a source of systematic v a r i a n c e that i s s p e c i f i c to the abilities being t e s t e d , t e s t - w i s e n e s s can be c o n s i d e r e d as a source of e r r o r v a r i a n c e that i s s p e c i f i c to p a r t i c u l a r test items and i s l i k e l y to be s u s c e p t i b l e to a 'chance' f a c t o r . In one of the e a r l i e s t e m p i r i c a l s t u d i e s of t e s t -wiseness, Gibb (1964) d e f i n e d i t as "the a b i l i t y to r e a c t p r o f i t a b l y to the presence of secondary cues i n a t e s t " (p. 5). Furthermore, he c o n f i n e d h i s d e f i n i t i o n to i n c l u d e "the 28 a b i l i t y of a t e s t e e to r e a c t to the presence of secondary cues i n ways advantageous to him s e l f on a m u l t i p l e - c h o i c e t e s t of knowledge of f a c t u a l i n f o r m a t i o n " (p. 5). However, he d i d not imply that t e s t - w i s e n e s s was s e n s i t i v e only to secondary cues. He d i d recognize the e x i s t e n c e of other components or kinds of u n d e s i r a b l e cues to c o r r e c t answers. In f a c t , he agreed with Thorndike that some i n d i v i d u a l s tended to guess more j u d i c i o u s l y than other, even when secondary cues were absent. A seminal work of test - w i s e n e s s was done by Millman et a l . (1965). They d e f i n e d i t as "a s u b j e c t ' s c a p a c i t y to u t i l i z e the c h a r a c t e r i s t i c s and formats of the t e s t and/or the t e s t t a k i n g s i t u a t i o n to r e c e i v e a h i g h s c o r e . Test-wiseness i s l o g i c a l l y independent of the examinee's knowledge of the s u b j e c t matter f o r which the items supposedly measures" (p. 707). Based on an e m p i r i c a l study, they developed an ^ O u t l i n e of Test-wiseness P r i n c i p l e s ' (see Table 1, Chapter 1) which i s by f a r the most complete and c o n c i s e e x i s t i n g e x p l i c a t i o n of the c o n s t r u c t of t e s t - w i s e n e s s . The o u t l i n e was c r e a t e d by a d m i n i s t e r i n g an u n s t r u c t u r e d q u e s t i o n n a i r e to 240 h i g h - a c h i e v i n g suburban high school students who were asked to d e s c r i b e t h e i r t e s t t a k i n g b e h a v i o r s . The students were found to be capable of v e r b a l i z i n g some test-wi s e n e s s p r i n c i p l e s . A d d i t i o n a l elements i n c l u d e d i n the taxonomy were i d e n t i f i e d from reviewing l i t e r a t u r e r e l a t e d to t e s t c o n s t r u c t i o n or advice f o r t a k i n g examinations. 29 The o u t l i n e (taxonomy) of tes t - w i s e n e s s i n c l u d e s two main p a r t s - (I) "Elements independent of t e s t c o n s t r u c t o r or t e s t purpose" and (II) "Elements dependent upon the t e s t c o n s t r u c t o r or purpose". In Part I, s k i l l s are orga n i z e d i n terms of t e s t t a k i n g s t r a t e g i e s - time-using, e r r o r - a v o i d a n c e , guessing and deductive r e a s o n i n g . Time-using s t r a t e g i e s r e f e r to pacing of responding, o m i t t i n g items, and r e t u r n i n g to the omitted items i f time a l l o w s . E r r o r - a v o i d a n c e s t r a t e g i e s can be seen as comprehension of i n s t r u c t i o n s which i n c l u d e s paying a t t e n t i o n to i n f o r m a t i o n i n the d i r e c t i o n s or w i t h i n items and reviewing answers or r e q u e s t i n g c l a r i f i c a t i o n when necessary. Guessing s t r a t e g i e s enable the examinee to r e c e i v e c r e d i t f o r responses on a completely chance b a s i s . Deductive reasoning s t r a t e g i e s d e a l with methods of o b t a i n i n g the c o r r e c t answers i n d i r e c t l y , or c o r r e c t l y answering a q u e s t i o n with p a r t i a l knowledge o n l y . Part I I , elements dependent upon the t e s t c o n s t r u c t o r or purpose, i n c l u d e s s t r a t e g i e s that the examinee may use a f t e r knowing the t e s t c o n s t r u c t o r ' s views or the t e s t purpose, or a f t e r o b t a i n i n g feedback from s i m i l a r t e s t s . Millman et a l . (1965) o u t l i n e d two s t r a t e g i e s i n t h i s category - i n t e n t c o n s i d e r a t i o n and cue-using s t r a t e g i e s . In i n t e n t c o n s i d e r a t i o n s t r a t e g i e s , examinees c o n s i d e r the i n t e n t of the t e s t c o n s t r u c t o r and past experiences i n d e a l i n g with the same t e s t . Cue-using s t r a t e g i e s i n c l u d e the r e c o g n i t i o n of t e s t c o n s t r u c t o r ' s i d i o s y n c r a s i e s , such as use of longer (or s h o r t e r ) c o r r e c t a l t e r n a t i v e s , g r e a t e r q u a l i f i c a t i o n or 30 g e n e r a l i z a t i o n of c o r r e c t responses, i n c l u s i o n of more true (or f a l s e ) statements, p l a c i n g the c o r r e c t o p t i o n i n one p o s i t i o n more o f t e n , the use of s i m i l a r o p t i o n s , the use of s t e r e o t y p i n g phrases and the presence of grammatical cues. Millman et a l . ' s (1965) ^Taxonomy of Test-wiseness' p r o v i d e s the ground work f o r most subsequent s t u d i e s of t e s t -wiseness. In a study of the e f f e c t s of test-wiseness m a t e r i a l s on s t a n d a r d i z e d t e s t performance of p r e s c h o o l disadvantaged c h i l d r e n , Oakland (1972) d e f i n e d t e s t - w i s e n e s s as "the a b i l i t y to manifest t e s t - t a k i n g s k i l l s which u t i l i z e the c h a r a c t e r i s t i c and format of a t e s t and/or t e s t - t a k i n g s i t u a t i o n i n order to r e c e i v e a score commensurate with the a b i l i t i e s being measured" (p. 355). H i s d e f i n i t i o n i s s i m i l a r to Millman et a l . ' s , except that Oakland c o n s i d e r s that p o s s e s s i o n of the s k i l l s would only r e s u l t i n a commensurate t e s t s core; while Millman et a l . proposes that the s k i l l s would enhance t e s t s c o r e s . In an i n v e s t i g a t i o n of c o g n i t i v e c o r r e l a t e s of t e s t - w i s e n e s s , Diamond and Evans (1972) d e f i n e d t h i s c o n s t r u c t as "the a b i l i t y to respond advantageously to m u l t i p l e c h o i c e items, c o n t a i n i n g extraneous c l u e s and to ob t a i n c r e d i t on these items without knowledge of the subj e c t matter" (p. 135). Diamond and Evans' d e f i n i t i o n i s s i m i l a r to Gibb's (1964) and Millman et a l . ' s as they a l l b e l i e v e that t e s t - w i s e n e s s s k i l l s enable examinees to make use of extraneous or secondary cues of some flawed items and to ob t a i n c r e d i t with no, or only p a r t i a l knowledge of the content being measured. 31 In d e f i n i n g t e s t - w i s e n e s s , Sarnacki (1979) f i r s t p o i n t e d out that " n e i t h e r guessing nor response s e t s are synonyms of TW ( t e s t - w i s e n e s s ) " (p. 253). He then d e s c r i b e d t e s t - w i s e n e s s as a c o g n i t i v e a b i l i t y or set of s k i l l s t h a t a t e s t - t a k e r c o u l d use to improve h i s / h e r score no matter what the content area of a t e s t was. He f u r t h e r e l a b o r a t e d that " ( t e s t - t a k i n g ) s k i l l s i n g e n e r a l , and TW ( t e s t - w i s e n e s s ) i n p a r t i c u l a r , g e n e r a l i z e to a number of s i t u a t i o n s . These composite f a c u l t i e s a l l o w examinees not only to e x p l o i t s p e c i f i c t e s t flaws, but a l s o to experience a g e n e r a l sense of s e c u r i t y i n t a k i n g t e s t s " (p. 253). More important, he d i s a g r e e s with Millman et a l . (1965) that t e s t wiseness i s n e i t h e r c o n f i n e d to o b j e c t i v e t e s t or to the s p e c i f i c s t r a t e g i e s l i m i t e d by them. Furthermore, i n e x p l o r i n g the r e l a t i o n s h i p of t e s t -wiseness and item c o n s t r u c t i o n used to develop m u l t i p l e - c h o i c e items, Smith (1980, 1982) noted that t e s t makers tended to c r e a t e d i s t r a c t o r s using a 'convergence s t r a t e g y ' which was not o u t l i n e d i n Millman et a l . ' s taxonomy (1965). With t h i s s t r a t e g y , the item c o n s t r u c t o r f i r s t develops the q u e s t i o n and c o r r e c t answer, and then c o n s t r u c t s the d i s t r a c t o r s using v a r i o u s c o r r e c t a l t e r n a t i v e s . Smith p o i n t e d out that these d i s t r a c t o r s "bear a l o g i c a l r e l a t i o n s h i p to the r i g h t answer, but not to the wrong answers" (p. 214). In t h i s way, he i n t e r p r e t e d that "testwiseness might be viewed as i n t e r a c t i o n between c h a r a c t e r i s t i c s of a t e s t - t a k e r and c h a r a c t e r i s t i c s of t e s t format" (p. 214). 32 Some more recent i n v e s t i g a t i o n s tend to d e f i n e t e s t -wiseness along s i m i l a r l i n e s . For example, D o l l y and W i l l i a m s (1983) c o n s i d e r e d i t to be "the a b i l i t y of the t e s t - t a k e r to perform at a b e t t e r than chance l e v e l on a m u l t i p l e c h o i c e t e s t no matter what the content being t e s t e d " (p. 2 ) . B a s i c a l l y , t e s t - w i s e n e s s encompasses both the method of measurement ( t e s t - w i s e cues p r o v i d e d i n flawed t e s t items) and c h a r a c t e r i s t i c s of the t e s t - t a k e r (a c o g n i t i v e a b i l i t y or set of a b i l i t i e s t h a t an examinee can employ i n any t e s t i n g s i t u a t i o n no matter what area of a t e s t i s ) . Since Millman et a l . ' s (1965) d e f i n i t i o n e x e r t e d the g r e a t e s t impact i n subsequent r e s e a r c h and has been widely adopted, i t was used as the o p e r a t i o n a l d e f i n i t i o n of t h i s t h e s i s . 2.22 Theoretical Approaches to Test-wiseness Sarnacki (1979) suggested three approaches to i n t e r p r e t the nature of t e s t - w i s e n e s s . Supported by Thorndike (1951), S t a n l e y (1971) and Diamond and Evans (1972), the f i r s t i n t e r p r e t a t i o n a s s e r t s that t e s t - w i s e n e s s e x i s t s because of the e x i s t e n c e of some s p e c i f i c cues i n p o o r l y c o n s t r u c t e d items. Test-wiseness i s viewed, not as a t r a i t , but rat h e r as a measurement problem. A l s o , t e s t - w i s e n e s s w i l l c r e a t e problems of v a l i d i t y , r a t h e r than r e l i a b i l i t y because i t i s c o n s i s t e n t l y r e s p o n s i b l e f o r systematic i n v a l i d v a r i a n c e . N e v e r t h e l e s s , Nunnally (1967) argued that f a c t o r s such as guessing, c l e r i c a l e r r o r s i n responding, e t c . would a l s o a f f e c t i n t e r n a l c o n s i s t e n c y estimates of r e l i a b i l i t y . G e n e r a l l y , proponents of t h i s i n t e r p r e t a t i o n emphasize the 33 e l i m i n a t i o n of these item cues i n examinations as a c r i t i c a l s t ep to ac c u r a t e and v a l i d measurement. The second approach to tes t - w i s e n e s s was f i r s t suggested by Thorndike (1951). He d e s c r i b e d t e s t - w i s e n e s s as the po s s e s s i o n of d i f f e r e n t amounts of c e r t a i n g e n e r a l and p e r s i s t e n t t r a i t s of t e s t - t a k i n g a b i l i t i e s . In an e m p i r i c a l study, Gibb (1964) found r e l i a b l e i n d i v i d u a l d i f f e r e n c e s i n tes t - w i s e n e s s a b i l i t i e s . A l s o , Crehan, Koehler and S l a k t e r (1974) demonstrated a s t a b l e t e s t - w i s e n e s s c h a r a c t e r i s t i c over Grades f i v e through 11 students. G e n e r a l l y , proponents of t h i s i n t e r p r e t a t i o n maintain that t e s t - w i s e n e s s i s best understood as a b i l i t i e s , s t r a t e g i e s , or t r a i t s of i n d i v i d u a l examinees, and not by c h a r a c t e r i s t i c s of t e s t s . The t h i r d approach, proposed by Sarnacki (1979), i s a s y n t h e s i s of the pre v i o u s two approaches. Since n e i t h e r of the above two approaches alone can s u f f i c i e n t l y e x p l a i n the c o n s t r u c t , he proposed that "test-wiseness encompass both the method of measurement and the c h a r a c t e r i s t i c s of the t e s t -t a k e r " (p. 267). T h i s approach enjoys the advantage of d e l i m i t i n g a l l p o s s i b l e sources of v a r i a n c e i n the c o n s t r u c t , as w e l l as t r a c k i n g down sources that account f o r most v a r i a n c e i n tes t - w i s e n e s s performance ( S a r n a c k i , 1979). In f a c t , Smith (1982) a l s o shared a s i m i l a r view of an i n t e g r a t e d approach by sa y i n g , "when the i n t e r a c t i o n between the c h a r a c t e r i s t i c s of an item and a person i s viewed from the p e r s p e c t i v e of the item, i t i s g e n e r a l l y c o n s i d e r e d to be an is s u e of item c o n s t r u c t i o n ; when viewed from the p e r s p e c t i v e 34 of the person, i t i s co n s i d e r e d to be an iss u e of tes t w i s e n e s s " (p. 211). Such c o n j e c t u r e i s demonstrated i n D o l l y and W i l l i a m ' s (1986) study of using t e s t - t a k i n g s t r a t e g i e s to maximize m u l t i p l e - c h o i c e t e s t s c o r e s . They found that i f an examinee possessed t e s t - w i s e n e s s , and i f the examination c o n t a i n e d s u s c e p t i b l e items, the two combined circumstances would improve scores a f t e r t e s t - w i s e n e s s t r a i n i n g . Since t h i s approach p r o v i d e d the most comprehensive i n t e r p r e t a t i o n of te s t - w i s e n e s s , i t was adopted as the t h e o r e t i c a l approach of t h i s t h e s i s . 2.23 Construct of Test-wiseness One major component of the concept of te s t - w i s e n e s s i s to i n t e r p r e t i t as a c o g n i t i v e c o n s t r u c t or c h a r a c t e r i s t i c ( s ) of the examinee. Thorndike (1951) suggested that t e s t - w i s e n e s s was a l a s t i n g general t r a i t i n regard to an i n d i v i d u a l ' s a b i l i t y to guess and i d e n t i f y cues; and a l a s t i n g specific t r a i t which was r e l a t e d to c e r t a i n types of t e s t s or item formats. Supporting Thorndike's i d e a , S t a n l e y (1971) a l s o c o n s i d e r e d t e s t - w i s e n e s s to be a p e r s i s t e n t t r a i t that might be r e l a t e d to b a s i c i n t e l l e c t u a l a b i l i t y . In a more recent study, Weiten (1984) f a i l e d to f i n d any s i g n i f i c a n t d i f f e r e n c e s i n i n t e r n a l r e l i a b i l i t y of flawed t e s t items. He t h e r e f o r e argued that t e s t - w i s e n e s s seemed to be a general t r a i t r a t h e r than s p e c i f i c to the cues p r o v i d e d i n flawed t e s t items. With r e s p e c t to the g e n e r a l i t y or s p e c i f i c i t y of the t r a i t ( s ) of te s t - w i s e n e s s , Millman et a l . (1965) t r i e d to demonstrate t h i s idea using the taxonomy of te s t - w i s e n e s s . They c o n s i d e r e d the s k i l l s to be general i n terms of the academic s u b j e c t of the t e s t , but specific when answering a s p e c i f i c t e s t item. They f u r t h e r broke down the idea of s p e c i f i c i t y i n t o two components: ( 1 ) s t r a t e g i e s dependent upon the t e s t - t a k e r , and (2) s t r a t e g i e s dependent on the t e s t c o n s t r u c t o r and/or t e s t purpose. To v a l i d a t e h i s argument f o r s p e c i f i c i t y , Millman (1966) developed an instrument to study four t e s t - w i s e n e s s s t r a t e g i e s / s k i l l s (guessing, d e d u c t i v e reasoning, stem-option s i m i l a r i t y , and i n c o n s i s t e n c i e s i n the response o p t i o n s ) i n four academic areas ( E n g l i s h l i t e r a t u r e , h i s t o r y - p o l i t i c a l s c i e n c e , mathematics, and b i o l o g y ) . However, when he t r i e d to v a l i d a t e h i s measure, the subscales d i d not h o l d up. Diamond and Evans (1972) a l s o t r i e d to develop an instrument designed to measure d i f f e r e n t elements of secondary cue-using s t r a t e g i e s . Comparing the c o r r e l a t i o n s of the secondary cue-using s t r a t e g i e s among the f i v e s u b s c a l e s (longer s p e c i f i c determiners, grammatical cues, and ov e r l a p p i n g d i s t r a c t o r ) , Diamond and Evans r e p o r t e d low c o r r e l a t i o n s of .02 to .33 and r e s u l t s of f a c t o r a n a l y s i s showed separate l o a d i n g of the su b s c a l e s . T h e i r r e s u l t s suggest that use of secondary cues are a c t u a l l y a set of s e v e r a l s k i l l s i n s t e a d of one gene r a l s k i l l . More r e c e n t l y , Benson i n 1985 ( c i t e d i n Benson, 1988) surveyed f o u r t h - through s i x t h - g r a d e students on knowledge of tes t - w i s e n e s s using a 20-item L i k e r t - t y p e s c a l e o r i g i n a l l y 36 designed to measure of time, e r r o r avoidance, m o t i v a t i o n , use of cues, and guessing s t r a t e g i e s . An i n i t i a l item a n a l y s i s of the 20 items obtained an alpha of .40. F i v e items were d e l e t e d because of negative c o r r e l a t i o n with t o t a l t e s t s c o r e s , l e a v i n g a 15-item s c a l e (r=.54) which was submitted to e x p l o r a t o r y f a c t o r a n a l y t i c procedures. Both of these analyses and another c o n f i r m a t o r y f a c t o r a n a l y s i s produced a f o u r - f a c t o r s o l u t i o n . The f a c t o r s were t e n t a t i v e l y l a b e l l e d as thoroughness, p r e p a r a t i o n , achievement m o t i v a t i o n and perseverance. C o r r e l a t i o n s among these four f a c t o r s ranged from -.08 to .57, suggesting some s p e c i f i c i t y of tes t - w i s e n e s s s k i l l s . Moreover, Benson (1988) maintained that the alpha c o e f f i c i e n t was not high enough to i n d i c a t e a uni d i m e n s i o n a l t e s t - w i s e n e s s s k i l l . He f u r t h e r suggested that t e s t - w i s e n e s s seemed to i n c o r p o r a t e s e v e r a l t r a i t s and c o u l d be measured i n both c h i l d r e n and a d u l t s . Although r e s e a r c h d i d not s t r o n g l y support Benson's case, some s t u d i e s ( S l a k t e r et a l . , 1970; Crehan et a l . , 1974; Benson, Urman, and Hocevar, 1986) b a s i c a l l y viewed test-wiseness as a mu l t i d i m e n s i o n a l c o n s t r u c t , s p e c i f i c to item format, general i n r e l a t i o n to academic s u b j e c t , and l a s t i n g or p e r s i s t e n t i n nature. However, i t i s beyond the scope of t h i s t h e s i s to explo r e and argue the s p e c i f i c d e t a i l s of the u n d e r l y i n g c o n s t r u c t of te s t - w i s e n e s s . 2.24 Test-wiseness and Bias Another component of the concept of test-wiseness d e a l s with the iss u e of measurement problems. Test-wiseness has 37 been widely r e c o g n i z e d as one source of e r r o r v a r i a n c e and a t h r e a t to t e s t v a l i d i t y (Millman et a l . , 1965; Oakland, 1972; P r e l l and P r e l l , 1986; S a r n a c k i , 1979; S l a k t e r et a l . , 1970; Thorndike, 1951). Millman et a l . f u r t h e r advocated the t e a c h i n g of t e s t - w i s e n e s s to minimize inter-examinee t e s t -wiseness d i f f e r e n c e s , thereby reducing measurement e r r o r . Moreover, McPhail (1978) argued, "testwiseness operates as e r r o r v a r i a n c e and i t s e f f e c t i s to reduce v a l i d i t y and r e l i a b i l i t y of t e s t s " (p. 168). Furthermore, Smith (1982) suggested that t e s t - w i s e n e s s c o u l d be viewed from the p e r s p e c t i v e of both the t e s t items and the c h a r a c t e r i s t i c s of t e s t - t a k e r . However, no matter what p e r s p e c t i v e one a c c e p t s , t e s t - w i s e n e s s w i l l c o n t r i b u t e some unwanted v a r i a n c e or b i a s i n t e s t r e s u l t s . The f a c t t h a t t e s t - w i s e n e s s c o n t r i b u t e s at l e a s t some measurement e r r o r seems i n d i s p u t a b l e . Research on t r a i n i n g and coaching t e s t - w i s e n e s s s k i l l s g e n e r a l l y suggested t h a t t e s t performance was much enhanced a f t e r r e c e i v i n g i n s t r u c t i o n and p r a c t i c e on t e s t - w i s e n e s s p r i n c i p l e s . In two meta-analyses on the e f f e c t s of t r a i n i n g t e s t - t a k i n g s k i l l s , Samson (1985) examined 24 s t u d i e s and Bangert-Drowns, K u l i k and K u l i k (1983) reviewed 30 s t u d i e s of coaching programs. They both a r r i v e d at s i m i l a r c o n c l u s i o n that programs of t r a i n i n g i n t e s t - t a k i n g s k i l l s produced s i g n i f i c a n t improvements i n students' achievement t e s t s c o r e s . A student's score i s a composite of t h e i r knowledge of the s u b j e c t matter being t e s t e d (t r u e score) and an e r r o r 38 component. In t h i s case, the amount of t e s t - w i s e n e s s students possess w i l l p a r t l y determine how much the e r r o r component w i l l c o n t r i b u t e to the t e s t s c o r e , s i n c e t e s t - w i s e n e s s s k i l l s are independent of t e s t c o n t e n t . In other word, t e s t - w i s e n e s s c o u l d mask a student's a c t u a l knowledge of the s u b j e c t matter. As p o i n t e d out by Smith (1982), t e s t scores would be " i n f l a t e d or d e f l a t e d due to an a b i l i t y ( t est-wiseness) which i s l o g i c a l l y independent of the t r a i t being measured" (p. 211). I f the p o s s e s s i o n or l e a r n i n g of t e s t - w i s e n e s s can l e a d to an improvement of t e s t performance, those students who possess l e s s t e s t - w i s e n e s s or have not r e c e i v e d i n s t r u c t i o n on these s k i l l s w i l l be i n an u n f a i r p o s i t i o n . In f a c t , i t has been a s s e r t e d i n many s t u d i e s (Bangert-Drowns et a l . , 1983; Fueyo, 1977; Samson, 1985; S a r n a c k i , 1979) that t r a i n i n g i n t e s t - t a k i n g s k i l l s tended to decrease some of the measurement e r r o r r e s u l t i n g from the i n f l u e n c e of t e s t - w i s e n e s s on t e s t s c o r e s , and a l l e v i a t e d the handicap of some students on t h i s a s p e c t . Research (Benson, 1988, Fueyo, 1977, P r e l l and P r e l l , 1986, S a r n a c k i , 1979) a l s o demonstrated that t e s t - w i s e n e s s tended to r e l a t e to f a m i l i a r i t y with t e s t format, deductive reasoning s k i l l s , cue-using s t r a t e g i e s , amount of previous t e s t i n g , and language a b i l i t y . For students with the same l e v e l of knowledge on the t e s t content being measured, those with l e s s of these a b i l i t i e s or experiences w i l l be more l i k e l y to o b t a i n a lower t e s t score than those students who have more t e s t - w i s e n e s s . 39 The f a c t that test-wiseness accounts f o r a source of e r r o r v a r i a n c e seems unquestionable. The next q u e s t i o n i s whether te s t - w i s e n e s s c o n s t i t u t e s a l a r g e enough source of v a r i a n c e to be of concern. Scruggs and L i f s o n (1985) argued that the d i s s e r t a t i o n c i t e d by Sarnacki (1979) f a i l e d to demonstrate a l a r g e source of v a r i a n c e which i s a t t r i b u t a b l e to by t e s t - w i s e n e s s . Scruggs and L i f s o n f u r t h e r s u b s t a n t i a t e d t h e i r argument by p o i n t i n g out the f a l l a c i e s i n two s t u d i e s . F i r s t , low c o r r e l a t i o n s among t e s t - w i s e n e s s , r i s k - t a k i n g and mathematics t e s t performance in Gross's (1976) study was unable to i d e n t i f y a l a r g e source of v a r i a n c e . Second, M e t f e s s e l and Sax's (1958) argument f o r i n f l a t e d score and t e s t - w i s e n e s s c o u l d not be j u s t i f i e d because of a lack of q u a n t i t a t i v e d a t a . Despite Scruggs and L i f s o n ' s attempt, the m y t h i c a l r e l a t i o n s h i p between te s t - w i s e n e s s and y l a r g e ' source of v a r i a n c e remains unknown. Research i n t h i s area has a r r i v e d at mixed r e s u l t s i n regard to the amount of e r r o r v a r i a n c e c o n t r i b u t e d by t e s t - w i s e n e s s or the a c t u a l gain score obtained from having t r a i n e d i n t h i s s k i l l . However, what has been c l e a r l y demonstrated i s that t e s t - w i s e n e s s c o n t r i b u t e s some b i a s or e r r o r to t e s t performance. In a d d i t i o n , more measurement e r r o r " i s l i k e l y to o r i g i n a t e from students who have too l i t t l e , r a t h er than too much, s k i l l i n t a k i n g t e s t s " ( E b e l , 1965, p. 97). 40 2.3 Test-wiseness Research 2.31 Correlates of Te s t ~wi s e ne s s I t was suggested by Sarnacki (1979) and Millman et a l . (1965), that i f the r e l a t i o n s h i p between te s t - w i s e n e s s and other e s t a b l i s h e d c o n s t r u c t s was e s t a b l i s h e d through l o g i c a l or t h e o r e t i c a l c o n s i d e r a t i o n , the v a l i d i t y of the c o n s t r u c t has been e s t a b l i s h e d . I f not, f u r t h e r refinement of the c o n s t r u c t might be necessary. Test-wiseness has been found to be r e l a t e d p o s i t i v e l y to i n t e l l i g e n c e (Diamond and Evans, 1972), and v e r b a l achievement ( B a j t e l s m i t , 1977; Diamond and Evans, 1972; Rowley, 1974). However, some re s e a r c h has obtained low c o r r e l a t i o n s of t e s t -wiseness with i n t e l l i g e n c e ( A r d i f f , 1965; Dunn and G o l d s t e i n , 1959), and negative c o r r e l a t i o n s with t e s t a n x i e t y ( B a j t e l s m i t , 1977; Millman et a l . , 1965) and p e r s o n a l i t y ( B o r r e l l o and Thompson, 1985). Since t e s t - w i s e n e s s i s r e l a t e d to the c o g n i t i v e a b i l i t y of t e s t - t a k e r s , i t should r e l a t e p o s i t i v e l y to i n t e l l i g e n c e . In Diamond and Evans' study (1972) with s i x - g r a d e students, a p o s i t i v e r e l a t i o n s h i p was found between i n t e l l i g e n c e and three s p e c i f i c t e s t - w i s e n e s s cues (stem-cue, s p e c i f i c determiners and grammar). The Lorge-Thorndike I n t e l l i g e n c e Test (Form AA) and the Iowa Test of B a s i c S k i l l s (Form 2) were used as measures of c o g n i t i v e a b i l i t i e s . Although t o t a l t e s t - w i s e n e s s scores c o r r e l a t e d moderately with i n t e l l i g e n c e , the a b i l i t y to use two s p e c i f i c cues (longer a l t e r n a t i v e s and o v e r l a p p i n g d i s t r a c t o r s ) were not. With the v a r i a b i l i t y of the c o r r e l a t i o n between v a r i o u s t e s t - w i s e n e s s cues and i n t e l l i g e n c e , Diamond and Evans concluded that t e s t - w i s e n e s s was s p e c i f i c to c e r t a i n cues only, and not a general t r a i t . However, i n A r d i f f ' s (1965) study, i n t e l l i g e n c e , reading a b i l i t y and components of t e s t - w i s e n e s s ( c a r e f u l n e s s , guessing, and reasoning a b i l i t i e s ) were found to be r e l a t e d at the t h i r d - g r a d e l e v e l but not at the six-grade l e v e l . Perhaps, the d i s c r e p a n c y between A r d i f f ' s (1965) and Diamond and Evans' (1972) s t u d i e s suggests that the r e l a t i o n s h i p between i n t e l l i g e n c e and t e s t - w i s e n e s s i s not as s t r o n g as i t would be expected. However, the d i s s o c i a t i o n of t e s t - w i s e n e s s and s e v e r a l c o g n i t i v e a b i l i t i e s has not been v e r i f i e d because i t i s not l o g i c a l to a s s e r t that these deductive reasoning or guessing s t r a t e g i e s are not r e l a t e d to general mental a b i l i t y (Scruggs and L i f s o n , 1985). F u r t h e r examination of t e s t -wiseness and other c o n s t r u c t s i s necessary i n order to e s t a b l i s h i t s c o n s t r u c t v a l i d i t y ( S a r n a c k i , 1979). Another v a r i a b l e that has been found to c o r r e l a t e p o s i t i v e l y with t e s t - w i s e n e s s i s v e r b a l a b i l i t y . Since r e c o g n i t i o n of extraneous cues r e q u i r e s knowledge of grammar, vocabulary and sentence s t r u c t u r e , i t i s reasonable to expect a t e s t - w i s e person to a l s o have hi g h v e r b a l a b i l i t y . S e v e r a l s t u d i e s using s i m i l a r t e s t - w i s e n e s s cues (stem-options, s p e c i f i c determiner and s i m i l a r o p t i o n s ) but d i f f e r e n t p o p u l a t i o n s , a l l have a r r i v e d at s i m i l a r c o n c l u s i o n s , p r o v i d i n g e m p i r i c a l evidence f o r the t h e o r e t i c a l r e l a t i o n s h i p of t e s t - w i s e n e s s and v e r b a l a b i l i t y . Test-wiseness was found 42 to be c o r r e l a t e d p o s i t i v e l y with the v e r b a l s c a l e of the S c h o l a s t i c A p t i t u d e Test among a d u l t s ( B a j t e l s m i t , 1977); with V e r b a l Achievement Test of Iowa B a s i c S k i l l s B a t t e r y among grade s c h o o l e r s (Diamond and Evans, 1972); and with a m u l t i p l e - c h o i c e vocabulary t e s t among hig h school students (Rowley, 1974). As c o n s i s t e n t l y demonstrated i n the l i t e r a t u r e , t e s t - w i s e n e s s seems to be h i g h l y c o r r e l a t e d with v e r b a l a b i l i t y . T h e r e f o r e , t h i s c h a r a c t e r i s t i c , as measured by the Language P r o f i c i e n c y Index ( L P I ) , was used i n t h i s study to f i n d i f the same r e l a t i o n s h i p would hol d t r u e with other p o p u l a t i o n s , p a r t i c u l a r l y English-as-a-second-language students. The idea that t e s t - w i s e n e s s i s a c o g n i t i v e c o n s t r u c t i s f u r t h e r strengthened by the p o s i t i v e c o r r e l a t i o n of t e s t -wiseness with achievement, as demonstrated i n s e v e r a l s t u d i e s ( F e r r e l l , 1972; B o r r e l l o and Thompson, 1985). Supported by other e m p i r i c a l l i t e r a t u r e on c o r r e l a t e s of t e s t - w i s e n e s s and F e r r e l l ' s own study of high school students, a r e l a t i o n s h i p between te s t - w i s e n e s s and performance on s t a n d a r d i z e d achievement t e s t s was demonstrated. S i m i l a r l y , B o r r e l l o and Thompson s t u d i e d the c o r r e l a t e s of s e l e c t e d t e s t - w i s e n e s s s k i l l s among 162 u n i v e r s i t y students. The F e r r e l l Test of Test-wiseness was used to measure two elements of t e s t -wiseness (deductive reasoning s t r a t e g i e s and cue-using s t r a t e g i e s ) ; p e r s o n a l i t y v a r i a b l e s were measured by Myers Br i g g s Type Indicator-Form Z; and the American C o l l e g e Test was used to measure the students' c o g n i t i v e achievement. 43 R e s u l t s showed that c o g n i t i v e achievement alone shared s i g n i f i c a n t l y l a r g e r v a r i a n c e (R2=.5S) than p e r s o n a l i t y p r e d i c t o r s (iJ^=.l9) with t e s t - w i s e n e s s . B o r r e l l o and Thompson's study t h e r e f o r e suggested "test-wiseness s k i l l s may p r i m a r i l y be c o g n i t i v e i n nature, and may not be s e n s i t i v e to p e r s o n a l i t y - r e l a t e d d i f f e r e n c e s i n i n c l i n a t i o n t o develop and manifest these s k i l l s " (p. 128). The r e l a t i o n s h i p of t e s t - w i s e n e s s and c o g n i t i v e achievement was f u r t h e r demonstrated i n Rowley's study (1974). Working with 198 nine-grade students from a southern O n t a r i o h i g h s c h o o l , he f i r s t o b t ained i n f o r m a t i o n on students' achievement m o t i v a t i o n , t e s t a n x i e t y , t e s t - w i s e n e s s and r i s k -t a k i n g . He then a d m i n i s t e r e d both f r e e response and m u l t i p l e c h o i c e format t e s t s i n mathematics and vocabulary, with an i n t e r v a l of f i v e weeks between a d m i n i s t r a t i o n s . Using p a r t i a l c o r r e l a t i o n s , c o r r e c t e d f o r the u n r e l i a b i l i t y of the f r e e response score and t e s t e d f o r s i g n i f i c a n c e u sing Lord's s t a t i s t i c , he found that p a r t i a l c o r r e l a t i o n s of t e s t - w i s e n e s s and r i s k - t a k i n g were s i g n i f i c a n t with vocabulary but not with mathematics. Rowley t h e r e f o r e concluded that m u l t i p l e - c h o i c e format t e s t s favored c e r t a i n types of examinees and p e n a l i z e d o t h e r s f o r reasons which c o u l d not be e x p l a i n e d by the m a t e r i a l being t e s t e d . A l s o , the nature of the m a t e r i a l being t e s t e d i s c r u c i a l i n determining which type of t e s t format i s s u i t a b l e . Moreover, Rowley's study (1974) f a i l e d to demonstrate that the use of m u l t i p l e c h o i c e t e s t s would favor or p e n a l i z e 44 students i n terms of achievement m o t i v a t i o n and t e s t - a n x i e t y . However, h i s study does not r e f u t e the p o s s i b l e r e l a t i o n s h i p between t e s t - w i s e n e s s and achievement m o t i v a t i o n . Since t e s t -wiseness i s h i g h l y r e l a t e d to c o g n i t i v e achievement, i t i s a l s o p e r t i n e n t f o r a h i g h academic a c h i e v e r to be w e l l motivated i n school achievement. In f a c t , R u s s e l l (1969) obtained a c o r r e l a t i o n of .60 to .72 between scores on m o t i v a t i o n and achievement as measured by C a l i f o r n i a Achievement T e s t . A l s o , Benson (1988) d i d an e x p l o r a t o r y f a c t o r a n a l y s i s and came up with a f a c t o r t e n t a t i v e l y l a b e l l e d as 'achievement m o t i v a t i o n ' i n her study with s i x t h g r a d e r s . T h e r e f o r e , t h i s study attempted to f i n d out whether t e s t -wiseness would be r e l a t e d to the achievement m o t i v a t i o n of the stu d e n t s . One of the general f a c t o r s r e l a t e d to test- w i s e n e s s i s t e s t - t a k i n g e xperience. K r e i t (1968) found that t h i r d graders s i g n i f i c a n t l y i n c r e a s e d t h e i r i n t e l l i g e n c e t e s t scores a f t e r p r e v i o u s exposure to three d i f f e r e n t i n t e l l i g e n c e t e s t s . However, Sarnacki (1979) p o i n t e d out that t e s t - t a k i n g experience c o u l d be tempered by a number of f a c t o r s . F i r s t , mere experience c o u l d not guarantee f u t u r e success on t e s t s . The e f f e c t i v e n e s s of p r e v i o u s t e s t experience needed to be c o n s i d e r e d as w e l l . Second, B a j t e l s m i t (1977) s t u d i e d a d u l t s u b j e c t s who showed d e f i c i e n t t e s t - t a k i n g s k i l l s ; and c o n j e c t u r e d that t h e i r d e f i c i e n c y c o u l d have been due to a l a c k of recent exposure to t e s t s . T h i r d , the use of a l t e r n a t e t e s t i n g procedures or t e s t formats might modify t e s t - t a k i n g 45 experience. In t h i s study, the r e l a t i o n s h i p between p r i o r t e s t - t a k i n g experience t e s t - w i s e n e s s was examined, with the hope of c o n t r i b u t i n g to the e x i s t i n g body of r e s e a r c h . Other c o r r e l a t e s of te s t - w i s e n e s s that have been s t u d i e d i n c l u d e grade l e v e l , sex, t e s t a n x i e t y , c h a r a c t e r i s t i c s of the examiner and p e r s o n a l i t y . S l a k t e r et a l . (1970) found sex was not r e l a t e d to tes t - w i s e n e s s a b i l i t i e s but grade l e v e l was. Research on te s t - w i s e n e s s and t e s t a n x i e t y i s l i m i t e d and c o n f l i c t i n g . As p o i n t e d out by Benson (1988), Millman et a l . (1965) r e p o r t e d no r e l a t i o n s h i p between the two v a r i a b l e s , but B a j t e l s m i t (1977) observed a negative r e l a t i o n s h i p . B o r r e l l o and Thompson (1985) f a i l e d to demonstrate a s i g n i f i c a n t r e l a t i o n s h i p between test-w i s e n e s s and p e r s o n a l i t y v a r i a b l e s . Sarnacki (1979) demonstrated a r e l a t i o n s h i p of test-wis e n e s s with some areas of the a f f e c t i v e domain, such as c h a r a c t e r i s t i c s of the examiner which may serve as general cues to the examinees. The examinees may f e e l l e s s s t r e s s f u l and anxious when the t e s t i s a d m i n i s t e r e d by someone of t h e i r own race. However, such c o n j e c t u r e was not w e l l supported with e m p i r i c a l evidence. As a summary, tes t - w i s e n e s s appears to be moderately r e l a t e d to achievement and v e r b a l a b i l i t y ; not h i g h l y r e l a t e d to i n t e l l i g e n c e and p e r s o n a l i t y ; and p o s s i b l y n e g a t i v e l y r e l a t e d to t e s t a n x i e t y . 2.32 Test-wiseness and Selection item: M u l t i p l e - c h o i c e ( s e l e c t i o n ) items enjoy the advantage of being s t r a i g h t forward, easy to score and e a s i l y understood by 46 t e s t - t a k e r s ; but they a l s o have some shortcomings. D o l l y and V i c k (1986) p o i n t e d out t h a t guessing c o u l d i n c r e a s e the l i k e l i h o o d of marking a c o r r e c t answer among a l l the o p t i o n s i n a m u l t i p l e c h o i c e q u e s t i o n . In a d d i t i o n , m u l t i p l e c h o i c e q u e s t i o n s e a s i l y i n t e r a c t with the c h a r a c t e r i s t i c s of t e s t -t a k e r s , r e s u l t i n g i n an i n f l a t e d or d e f l a t e d scores (Smith, 1982). M u l t i p l e choice q u e s t i o n s are o f t e n more v u l n e r a b l e to t e s t - w i s e components as o u t l i n e d i n Millman et a l . ' s (1965) taxonomy. Sarnacki (1979) suggested two reasons f o r t h i s phenomena. F i r s t , s e l e c t i o n items c o n t a i n many components, such as a stem and f i v e o p t i o n s , where t e s t - w i s e n e s s cues may occur. Second, s e l e c t i o n items are more d i f f i c u l t to c o n s t r u c t than supply items. They may be more s u s c e p t i b l e to numerous c o n s t r u c t i o n flaws i n c l u d i n g flaws a l l o w i n g the a p p l i c a t i o n of t e s t - w i s e n e s s p r i n c i p l e s . How a s k i l l e d t e s t - t a k e r makes use of t e s t - w i s e n e s s i n answering a m u l t i p l e c h o i c e item i s best i l l u s t r a t e d i n the model of t e s t - t a k i n g behavior proposed by Smith (1980). (see F i g u r e 1) The model r e f l e c t s v a r i o u s routes a s k i l l e d examinee may take to determine what o p t i o n to s e l e c t on a m u l t i p l e c h o i c e item, and which routes are used. There are four components i n t h i s model: A. the c o g n i t i v e monitor which c o n t r o l s which s k i l l s are going to be used to answer the item ; B. the i n d i v i d u a l ' s a b i l i t i e s and s k i l l s that are r e l a t e d to the t r a i t being measured; 47 C. the set of test-w i s e n e s s s t r a t e g i e s which are r e l e v a n t to the t e s t ; and D. the response of the i n d i v i d u a l (the s e l e c t i o n and r e c o r d i n g of a c h o i c e ) . No s o l u t i o n Found C o g n i t i v e Monitor f o r Test Taking (A) D e f i n i t i o n of Ap p r o p r i a t e S t r a t e g i e s A b i l i t i e s Relevant to T r a i t Being Measured (B) <—1 Most l i k e l y C h o i c e ( s ) Determined Determination of Need f o r Cues Test-wiseness S t r a t e g i e s Relevant to Author(s) S t y l e (C) S o l u t i o n Found Best Guess Taken Response and Refinement of Relevant Test-wiseness S t r a t e g i e s (D) F i g u r e 1 ; Smith's Model of Test Taking Behavior of S k i l l e d Test Takers (Smith, 1980) Smith (1980) suggested that a s k i l l e d t e s t taker f i r s t read the stem of a m u l t i p l e - c h o i c e item and then attempted to rec o g n i z e , u s i n g knowledge about the p e r c e i v e d content being t e s t e d , what he or she b e l i e v e d to be the c o r r e c t answer from among the o p t i o n s l i s t e d . I f the answer was not found, he or she would apply the set of t e s t - w i s e n e s s s t r a t e g i e s that he or she possessed. When a match was made, a t e s t - w i s e response was recorded. I f there was no item t e s t - w i s e cues or because the t e s t - t a k e r has exhausted a l l of h i s or her test-wi s e n e s s s t r a t e g i e s , the s k i l l e d t e s t taker would probably respond randomly. T h e r e f o r e , Smith's model f u r t h e r suggests that t e s t - w i s e n e s s , i n a d d i t i o n to knowledge of the t e s t content being t e s t e d , w i l l c o n t r i b u t e to any student's t e s t s c o r e . The r e l a t i o n s h i p between s e l e c t i o n items and t e s t -wiseness i s supported by e m p i r i c a l r e s e a r c h . A l k e r , C a r l s o n , and Hermann (1969) found a s i g n i f i c a n t p o s i t i v e c o r r e l a t i o n between m u l t i p l e - c h o i c e t e s t performance and a b i l i t y to recog n i z e ambiguous items. However, they f a i l e d to demonstrate t h a t m u l t i p l e c h o i c e q u e s t i o n s p e n a l i z e d students who were f a v o r a b l e to supply items, s i n c e the students performed e q u a l l y w e l l i n both types of q u e s t i o n s . Rowley (1974) a l s o found high p o s i t i v e c o r r e l a t i o n s between te s t - w i s e n e s s and m u l t i p l e c h o i c e t e s t s c o r e s . However, l i t t l e e f f e c t was found on supply items. More s p e c i f i c a l l y , Rowley's study showed that the use of s e l e c t i o n items c o u l d favor c e r t a i n examinees who were h i g h l y t e s t - w i s e and high i n r i s k - t a k i n g p r o p e n s i t y . However, the extent to 49 which these examinees are favored depends on the nature of t e s t ; a v e r b a l t e s t seems more s u s c e p t i b l e than a numerical t e s t . Furthermore, m u l t i p l e - c h o i c e q u e s t i o n s are s u s c e p t i b l e a 'convergence s t r a t e g y ' (Smith, 1982). T h i s s t r a t e g y i s r e l a t e d to the procedure of developing s e l e c t i o n items in which the stem i s f i r s t w r i t t e n , then the c o r r e c t response i s formulated, f o l l o w e d by the g e n e r a t i o n of the d i s t r a c t o r s . The d i s t r a c t o r s are u s u a l l y w r i t t e n with v a r i o u s degrees of c o r r e c t n e s s with regard to the stem and the c o r r e c t answer. Smith a s s e r t e d that a t e s t - w i s e i n d i v i d u a l would s e l e c t the c o r r e c t o p t i o n because the c o r r e c t o p t i o n w i l l converge. Such a s s e r t i o n was demonstrated i n h i s study i n which the students were exposed only to the response o p t i o n s without being g i v i n g the stem of the q u e s t i o n s . Students' responses were found to converge on the c o r r e c t o p t i o n more f r e q u e n t l y than by chance al o n e . A l s o , high school students t r a i n e d i n convergence s t r a t e g y performed s i g n i f i c a n t l y b e t t e r on a S c h o l a s t i c A p t i t u d e Test (SAT) v e r b a l subtest than d i d those who r e c e i v e d i n s t r u c t i o n on other t e s t - t a k i n g s k i l l s . As Smith p o i n t e d out, s u s c e p t i b i l i t y to the convergence s t r a t e g y d i d not mean a major flaw i n m u l t i p l e c h o i c e items. More important, i t demonstrates a p e c u l i a r i t y of t h i s type of item and i t s i n t e r a c t i o n with the t r a i t being measured, which i s of g r e a t e r concern to t e s t c o n s t r u c t o r s . T h e r e f o r e , t h i s study examined p a r t i c u l a r l y the m u l t i p l e - c h o i c e q u e s t i o n s of the s e l e c t e d content areas i n the p r o v i n c i a l examinations. 50 2.33 Test-wiseness and Variability of Test Score: As d i s c u s s e d above, the f a c t that test-wiseness has c o n t r i b u t e d one source of v a r i a n c e to students' t e s t scores was unquestionable, and i t was w e l l demonstrated i n v a r i o u s s t u d i e s (Benson, 1988; Crehan et a l . , 1974; K a l e c h s t e i n et a l . , 1981; Oakland, 1972; P r e l l and P r e l l , 1986;. Sarnacki , 1979; S l a k t e r et a l . , 1970). The r e s e a r c h a l s o suggested that i f the d e l e t e r i o u s e f f e c t s of tes t - w i s e n e s s were somehow reduced or e l i m i n a t e d , v a r i a b i l i t y on t e s t s due to t h i s f a c t o r c o u l d be reduced or" eliminated." I f students possessed or had been t r a i n e d on t h i s s k i l l , i t would be l o g i c a l l y to assume that t h e i r t e s t scores would be l e s s v a r i a b l e , and they would be a more homogeneous group than those who d i d not pos s e s s or had not been t r a i n e d on t h i s s k i l l . N e v e r t h e l e s s , two s t u d i e s (Gibb, 1964; S l a k t e r et a l . , 1970) seemed to demonstrate f i n d i n g s c o n f l i c t e d with the above argument. In Gibb's study, r e l i a b l e i n d i v i d u a l d i f f e r e n c e s between t r a i n e d and u n t r a i n e d group i n test-wis e n e s s was demonstrated. He noted that t r a i n i n g of test-wis e n e s s d i d improve the performance of the t r a i n e d group r e l a t i v e to the u n t r a i n e d . More important, he p o i n t e d out that "the t r a i n i n g a l s o accentuated d i f f e r e n c e s between i n d i v i d u a l s , i n c r e a s i n g the d i s p e r s i o n of the d i s t r i b u t i o n of sc o r e s " (p. 52). In h i s study, an i n c r e a s e of tes t - w i s e n e s s seemed to i n f l a t e r a t h e r than reduce t e s t score v a r i a b i l i t y . S l a k t e r et a l . (1970) a l s o a r r i v e d at s i m i l a r f i n d i n g s . In t h e i r study, s e n i o r high school students were randomly 51 a s s i g n e d to two groups: one group was t r a i n e d on a programmed t e x t of s e l e c t e d aspects of te s t - w i s e n e s s ; and the other group r e c e i v e d a programmed t e x t designed to teach answering a l l items on an examination (R program). Each programmed t e x t was found to be e f f e c t i v e i n i n c r e a s i n g mean score but only the R program decreased the t e s t score v a r i a n c e . T h e r e f o r e , they concluded that t e a c h i n g t e s t - w i s e n e s s would not guarantee a re d u c t i o n of v a r i a b i l i t y i n t e s t s c o r e . Moreover, they claimed that i f students only mastered c e r t a i n aspects of te s t - w i s e n e s s , t h e i r attainment would s t i l l be low and one co u l d not expect the t o t a l v a r i a b i l i t y to be reduced. Test score v a r i a n c e has r a r e l y been d i r e c t l y addressed i n the l i t e r a t u r e on t e s t - w i s e n e s s . Though Gibb (1964) and S l a k t e r et a l . (1970) p r o v i d e d some evidence, more e m p i r i c a l study would be needed i n order t o v e r i f y t h e i r arguments. T h i s study examined i f the score v a r i a b i l i t y of t e s t - w i s e students would be s i g n i f i c a n t l y d i f f e r e n t from that of those students who possessed l e s s of the same s k i l l . 2.34 Test-wiseness and Cultural Differences: In Millman et a l . ' s (1965) seminal a r t i c l e of t e s t -wiseness, i t was po i n t e d out that t e s t - w i s e n e s s found i n o b j e c t i v e t e s t s was c u l t u r a l l y determined. However, re s e a r c h on c u l t u r a l d i f f e r e n c e s i n tes t - w i s e n e s s was s c a t t e r e d and i n s u f f i c i e n t to a r r i v e at any c o n c l u s i v e statements. Scruggs and L i f s o n (1985), i n h i s review of the c u r r e n t m y t h i c a l and r e a l c o nceptions of te s t - w i s e n e s s , p o i n t e d that d e f i c i e n c y on test - w i s e n e s s s k i l l s e x h i b i t e d by m i n o r i t y groups have simply 52 been assumed r a t h e r than documented, and r e l a t i v e l y l i t t l e r e s e a r c h has focused on i d e n t i f y i n g t h a t d e f i c i t . The e a r l i e s t comparative study with respect to t e s t f a c i l i t i e s was done by Vernon (1962) with B r i t i s h and American students on reading comprehension. H i s study suggested that f a m i l i a r i t y with p a r t i c u l a r item formats was not the reason fo r d i f f e r e n c e s found between the two n a t i o n a l groups i n t e s t performance. Other f a c t o r s , such as method of t e s t i n g and the s u b j e c t s ' f a c i l i t y i n h a n d l i n g the t e s t may a l s o a f f e c t the assessment process of d i f f e r e n t n a t i o n a l groups. Attempting to v a l i d a t e Vernon's (1962) statement, Millman and S e t i j a d i (1966) compared the performance of American and Indonesian students on three types of Algebra t e s t items -open ended q u e s t i o n s , good m u l t i p l e - c h o i c e items with p l a u s i b l e o p t i o n s and poor m u l t i p l e - c h o i c e items with i m p l a u s i b l e o p t i o n s . A l l students had completed the e q u i v a l e n t of one year of elementary a l g e b r a , but Indonesian students had no p r i o r experience with o b j e c t i v e t e s t items. T h e i r r e s u l t s showed that American students enjoyed an advantage on the o b j e c t i v e q u e s t i o n s , even a f t e r the Indonesian students were f a m i l i a r i z e d with the mechanics of choosing the c o r r e c t answer. T h e i r study a l s o suggested that students with l i t t l e experience with o b j e c t i v e items might be e v a l u a t e d a d v e r s e l y i n n a t i o n a l t e s t i n g programs. A m a j o r i t y of r e s e a r c h on test-wiseness among e t h n i c / c u l t u r a l groups i n the U n i t e d S t a t e s has focussed on lower achievement t e s t s or s t a n d a r d i z e d t e s t scores that were caused 53 by c u l t u r a l d i f f e r e n c e s , p a r t i c u l a r l y with black or H i s p a n i c students. Hence, much a t t e n t i o n has been drawn to re s e a r c h on t r a i n i n g American m i n o r i t y c h i l d r e n (e.g., K a l e c h s t e i n et a l . , 1981; Dreibach and Keogh, 1982) or a d u l t s (e.g., F r i e r s o n , 1976) on t e s t - w i s e n e s s s k i l l s . K a l e c h s t e i n et a l . (1981) i n v e s t i g a t e d the e f f e c t of i n s t r u c t i o n on t e s t - t a k i n g s k i l l s of 43 Black i n n e r - c i t y c h i l d r e n . R e s u l t s showed that the experimental group (those r e c e i v e d t e s t - t a k i n g s k i l l t r a i n i n g ) performed s i g n i f i c a n t l y b e t t e r on the SAT than the c o n t r o l group (those r e c e i v i n g no s i m i l a r t r a i n i n g ) . They concluded that Black, i n n e r - c i t y c h i l d r e n b e n e f i t t e d from i n s t r u c t i o n i n t e s t - t a k i n g s k i l l s . L i k e w i s e , Dreisbach and Keogh (1982) s u c c e s s f u l l y taught t e s t - w i s e n e s s s k i l l s to Mexican-American k i n d e r g a r t e n c h i l d r e n from a low socioeconomic l e v e l . T r a i n e d c h i l d r e n performed b e t t e r on a Readiness Test than u n t r a i n e d c h i l d r e n i n both E n g l i s h and Spanish a d m i n i s t r a t i o n of that t e s t . A l s o , performance was improved when the test.was f i r s t taken i n Spanish, then i n E n g l i s h ; but no i n f l u e n c e of E n g l i s h on subsequent performance was found when the language of a d m i n i s t r a t i o n was r e v e r s e d . They commented, "test-wiseness may be p a r t i c u l a r l y important when t e s t i n g c h i l d r e n from economically disadvantaged backgrounds and/or where the primary language of the home i s not standard E n g l i s h " (p. 228). The above s t u d i e s ( K a l e c h s t e i n et a l . , 1981; Dreibach and Keogh, 1982) eval u a t e d the impact of t e s t - t r a i n i n g on the subsequent scores on a s t a n d a r d i z e d t e s t , and concluded that these m i n o r i t y students l a c k e d t e s t - w i s e n e s s s k i l l s i n doing these s t a n d a r d i z e d t e s t s . However, t h e i r r e s u l t s have not v a l i d l y addressed the q u e s t i o n s of t h e i r i n v e s t i g a t i o n . F i r s t , without comparing to other groups, the performance of black second graders a f t e r exposure to a t e s t - w i s e n e s s treatment t o l d l i t t l e about c u l t u r a l d i f f e r e n c e s i n t e s t -wiseness. Second, without r e f e r r i n g to other comparable groups, the i s s u e of whether the t r a i n i n g was p a r t i c u l a r l y important f o r low socio-economic s t a t u s or m i n o r i t y p o p u l a t i o n s was l e f t unanswered i n D r e i s b a c h and Keogh's (1982) study. L i k e w i s e , Diamond, Ayres, Fishman and Green (1976) conducted a study to address d i r e c t l y the i s s u e of whether m i n o r i t y p o p u l a t i o n s l a c k e d t e s t - w i s e n e s s . However, black i n n e r - c i t y c h i l d r e n performed s i g n i f i c a n t l y above chance on a t e s t - w i s e n e s s instrument. A l s o , the scores on the t e s t -wiseness instrument d i d not p r e d i c t scores on the v e r b a l achievement subtest of the C a l i f o r n i a Achievement T e s t . T h e i r r e s u l t s f a i l e d to support the assumption that m i n o r i t y students l a c k e d t e s t - w i s e n e s s , nor that t e s t - w i s e n e s s was r e l a t e d to t h e i r achievement t e s t s c o r e s . Again, t h e i r study examined only one m i n o r i t y group on the l e v e l of t e s t -wiseness, t h e r e f o r e i t c o u l d not be used to p r e d i c t c u l t u r a l d i f f e r e n c e s i n t e s t - w i s e n e s s i n g e n e r a l . However, these s t u d i e s p r o v i d e d some e m p i r i c a l evidence s u p p o r t i n g McPhail's (1978) a s s e r t i o n that t e s t - w i s e n e s s 55 s t u d i e s on black and m i n o r i t y students were s t i l l i n c o n c l u s i v e . Perhaps, c r o s s - c u l t u r a l comparison would address some of the unanswered q u e s t i o n s . Benson et a l . (1986) i n v e s t i g a t e d the e f f e c t s of test-wiseness t r a i n i n g with t h i r d and f i f t h grade Black, White and Mexican-American students. Both experimental (test-wiseness t r a i n i n g ) and c o n t r o l (no t r a i n i n g ) groups were p r e - t e s t e d to i d e n t i f y i n i t i a l d i f f e r e n c e s among them. The experimental group then r e c e i v e d i n s t r u c t i o n on four t e s t - t a k i n g s t r a t e g i e s - f o l l o w i n g d i r e c t i o n s , time-using, guessing, and answer-changing. The two groups were p o s t - t e s t e d using the SAT Reading and Mathematics, and then t e s t e d again one month l a t e r . In terms of race, the t r a i n i n g e f f e c t s tended to favor the white students, except i n the delayed math t e s t i n which the white and H i s p a n i c students performed s i m i l a r l y a c r o s s the t h i r d and f i f t h grades. Although t e s t - w i s e n e s s was assumed to be more e f f e c t i v e f o r low-achieving m i n o r i t y students, t h e i r study showed no evidence f o r d i f f e r e n t i a l e f f e c t i v e n e s s by e t h n i c group. However, they p o i n t e d out that understanding why c e r t a i n student subgroups f a i l e d to apply t e s t s t r a t e g i e s on t e s t i n g s i t u a t i o n s or d i f f e r e n t t e s t formats would h e l p to develop remediation techniques and improve the e d u c a t i o n a l e q u i t y f o r a l l students. Apart from the t e s t - w i s e n e s s s t u d i e s done predominantly with Black and H i s p a n i c c u l t u r a l groups, Lo and S l a k t e r ' s (1973) and Wu and S l a k t e r ' s (1978) s t u d i e s with Chinese 56 students have a l s o been commonly used to demonstrate c u l t u r a l d i f f e r e n c e s i n t e s t - w i s e n e s s . Lo and S l a k t e r (1973) compared Grade 12 Chinese and American students on t h e i r r i s k - t a k i n g behavior and t e s t - w i s e n e s s . Drawn from two high schools i n a c i t y i n Northern Taiwan, 131 Chinese students were adm i n i s t e r e d a Chinese v e r s i o n of Standard E d u c a t i o n a l I n t e l l i g e n c e Test (SEIT) which c o n t a i n e d two p a r t s : (I) r i s k t a k i n g on O b j e c t i v e Examination (RTOOE) measured r i s k t a k i n g , and (II) c o n t e n t - f r e e m u l t i p l e - c h o i c e items measured four t e s t - w i s e n e s s behaviors (stem-options, absurd-options, s i m i l a r - o p t i o n s , and s p e c i f i c d e t e r m i n e r s ) . T h e i r r e s u l t s showed that Chinese students obtained s i g n i f i c a n t l y lower mean scor e s on a l l four test-wiseness subscales than t h e i r American c o u n t e r p a r t s , but no d i f f e r e n c e s were observed i n RTOOE. A l s o , they found that Chinese students tended to s e l e c t , r a t h e r than e l i m i n a t e , o p t i o n s with a s p e c i f i c determiner. They a t t r i b u t e d the d i f f e r e n c e s i n the four s u b s c a l e s to the i n a b i l i t y of Chinese to d e t e c t cues from a v a s t l y d i f f e r e n t w r i t i n g system and a d i f f e r e n t mode of e x p r e s s i o n i n the Chinese language. Likewise, Wu and S l a k t e r (1978) examined r i s k - t a k i n g and t e s t - w i s e n e s s of Grade f i v e , e i g h t , and eleven Chinese students from both r u r a l and urban schools i n Taiwan. S i m i l a r t o Lo and S l a k t e r ' s (1973) study, a Chinese t r a n s l a t i o n of SEIT with the same two s e c t i o n s were used. D i f f e r e n t m u l t i p l e c h o i c e items i n p a r t II and d i f f e r e n t forms of SEIT f o r each of the three grade l e v e l s were used. They found that the 57 r e l a t i o n s h i p of grade l e v e l with RTOOE was q u i t e d i f f e r e n t f o r Chinese students than f o r American students. For Chinese students, higher graders obtained higher r i s k scores than the lower g r a d e r s . However, the o p p o s i t e p a t t e r n was e x h i b i t e d among American students. The authors e x p l a i n e d that higher grade Chinese students had to pass h i g h l y c o m p e t i t i v e examinations f o r f u r t h e r e d u c a t i o n , hence were o f t e n encouraged by teacher to respond to a l l items. A l s o , the i n c r e a s e of t e s t - w i s e n e s s scores over grades may simply be a t t r i b u t e d to maturation or more experience with t e s t s . Wu and S l a k t e r (1978) a l s o p r o v i d e d f u r t h e r support to the o b s e r v a t i o n that Chinese students tended to s e l e c t r a t h e r than e l i m i n a t e o p t i o n s with s p e c i f i c d eterminers. The above two s t u d i e s with Chinese and American students seem to p r o v i d e some evidence f o r c u l t u r a l d i f f e r e n c e s i n t e s t - w i s e n e s s . Although t h e i r s t u d i e s d i d not f i r m l y support a lower t e s t - w i s e n e s s score f o r Chinese than f o r American students, d i f f e r e n t t e s t - t a k i n g s t r a t e g i e s seemed c o n s i s t e n t l y demonstrated between two c u l t u r a l groups. However, t h e i r r e s u l t s c o u l d not be j u s t i f i e d to address t h e i r r e s e a r c h q u e s t i o n s because of some methodological flaws. F i r s t , t r a n s l a t i o n of SEIT i n t o Chinese posed a t h r e a t to the v a l i d i t y of the instrument, and p r o h i b i t e d any c o n c l u s i v e statements about d i f f e r e n c e s between the two c u l t u r a l groups. I t was q u e s t i o n a b l e i f the s t r u c t u r a l cues of the two v e r s i o n s remained, or i f the same test-wi s e n e s s s t r a t e g i e s would s t i l l h o l d true when they had been t r a n s l a t e d i n t o another language. 58 In other words, t h e i r r e s u l t s c o u l d not address whether the d i f f e r e n c e s r e s u l t e d from language d i f f e r e n c e s , t r a n s l a t i o n problems, or r e a l d i f f e r e n c e s i n t e s t - w i s e n e s s . Second, although they demonstrated some c u l t u r a l d i f f e r e n c e s between Chinese and American students i n t e s t - w i s e n e s s , they d i d not show i f these d i f f e r e n c e s were s i g n i f i c a n t enough to a f f e c t t h e i r scores on some important examinations or any s t a n d a r d i z e d or achievement t e s t s . What i s of more concern to the educators i s not the d i f f e r e n c e f o r i t s own sake, but r a t h e r the d i f f e r e n c e s between the two groups that would favor one over the other i n t e s t r e s u l t s . A l l the s t u d i e s d e s c r i b e d above on c u l t u r a l d i f f e r e n c e s were based e n t i r e l y on American samples. S l a k t e r (1969) compared Canadian and American e i g h t h graders and showed that Canadian students scored lower on the RTOOE than t h e i r American c o u n t e r p a r t s . Other than t h i s study, very l i t t l e r e s e a r c h has been done i n Canada to examine te s t - w i s e n e s s behavior among d i f f e r e n t e t h n i c groups and the e d u c a t i o n a l i m p l i c a t i o n s of any d i f f e r e n c e s . T h i s study attempted to overcome the major flaws of Lo and S l a k t e r ' s (1973) and Wu and S l a k t e r ' s (1978) s t u d i e s and to provide some e m p i r i c a l work of the t e s t - t a k i n g behaviors of Chinese and Canadian students. 59 CHAPTER THREE METHODOLOGY The p r i n c i p a l o b j e c t i v e of t h i s study was to i n v e s t i g a t e the c u l t u r a l d i f f e r e n c e s i n te s t - w i s e n e s s between Chinese and Canadian students; and the impact of test-wi s e n e s s on B r i t i s h Columbia Grade 12 P r o v i n c i a l Examination r e s u l t s among Chinese students i n B r i t i s h Columbia. B r i e f l y , the Test of T e s t -wiseness and the Student Survey q u e s t i o n n a i r e were admin i s t e r e d to Grade 12 Chinese students who wrote the p r o v i n c i a l examinations i n January 1990. Completed t e s t data was checked, coded, scored, analyzed, and compared with the same data from Canadian students who wrote the p r o v i n c i a l examinations i n June 1989. S p e c i f i c i n f o r m a t i o n p e r t a i n i n g to the methods u t i l i z e d i s d e s c r i b e d under separate headings below. 3.1 Subjects The Chinese sample used i n t h i s study was obtained from Coquitlam C o l l e g e . I t i s an independent community c o l l e g e composed l a r g e l y of f o r e i g n students from A s i a , who came to Canada to pursue s e n i o r h i g h school and t e r t i a r y e d u c a t i o n . Forty-one Chinese s u b j e c t s were o b t a i n e d from those who wrote the Grade 12 E n g l i s h , A l g e b r a , P h y s i c s , and/or H i s t o r y p r o v i n c i a l examinations i n January 1990. Only s u b j e c t s whose e t h n i c background was Chinese were i n c l u d e d i n the study. A comparable Canadian sample was obtained from the data c o l l e c t e d by Rogers and Bateson (1990a) i n t h e i r study on the 60 impact of te s t - w i s e n e s s upon p r o v i n c i a l examination performance of Grade 12 stu d e n t s . Among the 936 students i n the 10 schools they i n v e s t i g a t e d , a random sample of Canadian s u b j e c t s was drawn from those students who wrote Grade 12 Algebra , E n g l i s h , P h y s i c s , and/or H i s t o r y i n June 1989. The 10 schools d i s t r i b u t e d a c r o s s the Lower Mainland, C e n t r a l , and Northern C o a s t a l area of B r i t i s h Columbia. T h e r e f o r e , the s e l e c t e d s c hools cover a sample that r e p r e s e n t s a good range of academic a b i l i t y . F u r t h e r , among a l l the students who wrote Grade 12 Algebra, E n g l i s h , P h y s i c s , and/or H i s t o r y i n June 1989 i n Rogers and Bateson's study (1990a), students of Chinese o r i g i n were s o r t e d out to compare with the Chinese sample i n t h i s study. Since the Chinese students i n t h i s study were a l l f o r e i g n students who r e c e n t l y came to study i n Canada, they may not represent other Chinese p o p u l a t i o n s i n Canada. Comparison of the Chinese students i n t h i s study with other Chinese students may h e l p to r e v e a l any r e l a t i o n s h i p between e t h n i c i t y and t e s t - w i s e n e s s . T h e r e f o r e , the Chinese students used i n t h i s study ( d i s t i n g u i s h e d as ' f o r e i g n Chinese' i n the subsequent s e c t i o n s ) were compared with those Chinese students in Rogers and Bateson's study (1990a). Since they have been i n Canada f o r a much longer p e r i o d and are l a r g e l y immigrants to Canada, they are t h e r e f o r e termed as 'Chinese immigrants' i n the subsequent a n a l y s e s . 61 3.2 I n s t r u m e n t a t i o n s 3.21 Grade Twelve Provincial Examinations Performance on the m u l t i p l e - c h o i c e items i n the June 1989 examinations ( f o r Canadian and Chinese immigrant students) and in the January 1990 examinations ( f o r f o r e i g n Chinese students) were the dependent or c r i t e r i o n v a r i a b l e f o r the major hypothesis and hypothesis two. The t e s t r e s u l t s f o r these s u b j e c t areas were pr o v i d e d by the M i n i s t r y of Educati o n . 3.22 Test of Test-wiseness (see Appendix E) The Test of Test-wiseness (TTW) used i n t h i s study was developed by Rogers and Bateson (1990b). I t was mo d i f i e d and adapted from the items developed by Gibb (1964), Millman (1966), and S l a k t e r et a l . (1970). T h i s t e s t was designed to be content f r e e . For some items, there was no c o r r e c t answer s i n c e the content was e n t i r e l y f i c t i t i o u s . The instrument was designed to assess four d i f f e r e n t a spects of t e s t - w i s e n e s s : 1. three deductive reasoning s t r a t e g i e s : ID1 - e l i m i n a t e o p t i o n s known to be i n c o r r e c t , ID2 - choose n e i t h e r or both of two op t i o n s which imply the c o r r e c t n e s s of each other, and ID3 - choose n e i t h e r or one of two o p t i o n s , one of which, i f c o r r e c t , would imply the i n c o r r e c t n e s s of the other; and 2. one cue-using s t r a t e g y : IIB4 - recognize and use s i m i l a r i t i e s between the stem and the o p t i o n s . 62 The s e l e c t i o n of these t e s t - w i s e n e s s elements was based on the most f r e q u e n t l y o c c u r r i n g item f a u l t s i d e n t i f i e d from the m u l t i p l e - c h o i c e items i n c l u d e d i n the 1988 p r o v i n c i a l examinations i n the s u b j e c t s of E n g l i s h , Geography, H i s t o r y , B i o l o g y , Chemistry, and A l g e b r a . A separate s e c t i o n on guessing was a l s o i n c l u d e d . To pro v i d e face v a l i d i t y , the content of the items was u n f a m i l i a r t o the students and was d i s t r i b u t e d evenly a c r o s s four content areas of four Grade 12 courses that have the l a r g e s t students enrollments - E n g l i s h , A l g e b r a , H i s t o r y , and B i o l o g y (Rogers and Bateson, 1990b). The t e s t was made up of two s e c t i o n s . S e c t i o n I was composed of 24 items i n which s u b j e c t s were asked to s e l e c t an answer from four p o s s i b l e o p t i o n s . S e c t i o n II was made up of 10 items - t r u e / f a l s e and f o u r - o p t i o n m u l t i p l e - c h o i c e . In S e c t i o n I I , s u b j e c t s were t o l d t h a t there i s a c o r r e c t i o n f o r guessing - "zero p o i n t s w i l l be given to each omitted q u e s t i o n but two p o i n t s w i l l be deducted f o r each q u e s t i o n answered i n c o r r e c t l y " (Rogers and Bateson, 1990b). Students recorded t h e i r responses on separate machine-scoreable answer sheets ( f o r Canadian and Chinese immigrant s u b j e c t s ) or on the t e s t booklet ( f o r f o r e i g n Chinese s u b j e c t s ) . A d m i n i s t r a t i o n time was t h i r t y minutes. A copy of the Test of Test-wiseness i s i n c l u d e d i n Appendix E. The composition of the t e s t i s shown in Table 3. The r e l i a b i l i t y and v a l i d i t y of t h i s t e s t were assessed i n v o l v i n g 936 students i n 10 p u b l i c s c h o o l s throughout B r i t i s h Columbia. A l l the students e n r o l l e d i n E n g l i s h 12 p r o v i n c i a l 63 examination i n June 1989. They covered a d i v e r s e s o c i o -economic background and a wide range of achievement, and represented gender and e t h n i c groups d i s t r i b u t i o n of the p r o v i n c e . Table 3 Composition of the Test of Test-wiseness Subject Areas Test-wiseness Elements Algebra " B i o l o g y H i s t o r y E n g l i s h S e c t i o n I Absurd 0ptions(ID1) 3* 7 13,17 9,22 S i m i l a r 0ptions(ID2) 6 12,18 21 4,15 D i f f e r e n t Options (ID3) 16,23 1 5 10,20 Stem-options Link (IIB4) 14 2,24 8,19 11 T o t a l No. of Items 5 6 6 7 S e c t i o n II Guessing 3 5,6 2 9,10 Non-guessing 8 1 7 4 T o t a l No. of Items 2 3" 2 3 * = item number (see Appendix E) The psychometric p r o p e r t i e s of t h i s instrument are presented i n Table 4. As seen i n Table 4, the r e l i a b i l i t y ( i n t e r n a l c o n s i s t e n c y ) i n d i c e s i n s u b t e s t s ID1, ID2, ID3, and 64 IIB4 are r e l a t i v e l y low. I t was due to the random responses of the students which i n turn c o n t r i b u t e d to measurement e r r o r and low i n t e r n a l c o n s i s t e n c y of the instrument (Rogers and Bateson, 1990a). Table 4 Psychometric P r o p e r t i e s of the Test of Test-wiseness Subtests Max. Score M SD R e l . SEM Absurd Options(IDI) 6 3. ,66 1 , .27 .22 1 , .02 S i m i l a r Options(ID2) 6 3. ,70 1 . .30 .36 i .95 D i f f e r e n t Options(ID3) 6 3. .06 1 , .23 .16 1 , .02 Stem-options L i n k ( l I B 4 ) 6 3. .47 1 , .23 .24 1 , .00 Test-wiseness T o t a l 24 14. .0 2, .79 .37 2, .17 Guessing 24 15, .70 4 .76 .50 3, .21 The v a l i d i t y of t h i s t e s t was assessed by i n d i v i d u a l i n t e r v i e w s . Two weeks a f t e r the data c o l l e c t i o n of the group a d m i n i s t r a t i o n of the Test of Test-wiseness, a subsample of the h i g h s c o r i n g (17 or above) (n = 36) and low s c o r i n g (11 or below) (n = 41) students were obtained from s i x out of the" 10 s c h o o l s i n the i n i t i a l sample. The students were asked by the i n t e r v i e w e r to "think aloud", to " d e s c r i b e and e x p l a i n what you are doing" or "how d i d you get your answer" f o r each of the 14 q u e s t i o n s . T h e i r comments were recorded on a d i f f e r e n t form i n order to determine the method by which the students 65 a r r i v e d at t h e i r answers. The composition of the TTW f o r v a l i d a t i o n i s presented i n Table 5 and the content of these items i s i n c l u d e d i n Appendix C. Table 5 Composition of the Test of Test-wiseness f o r V a l i d a t i o n Test-wiseness Subject Areas Elements Algebra B i o l o g y H i s t o r y E n g l i s h Absurd Options * 14 9 8 S i m i l a r Options - 4,7 -D i f f e r e n t Options - 2,5 1 Stem-options Link 6 10 3 Guessing 13 - 1 1 Non-guessing 12 T o t a l No. of Items 3 6 5 0 * = item number (see Appendix C) S t r a t e g i e s and any a d d i t i o n a l i n f o r m a t i o n provided by the students were a l s o recorded. R e s u l t s i n d i c a t e d that s e v e r a l t e s t - w i s e n e s s s t r a t e g i e s were being used by the students who scored 17 or more. Those who scored below 11 appeared to be guessing randomly r a t h e r than a p p l y i n g any of the t e s t -wiseness s k i l l s c o n s i s t e n t l y . The c u t - o f f c r i t e r i a of t e s t -wise (17 or above) and t e s t - n a i v e (11 or below) was approximately one standard d e v i a t i o n above and below the mean 66 i n the S e c t i o n I of the TTW of the i n i t i a l sample (M =13.9, SD = 2.8) (Rogers and Bateson, 1990b). However, the r e s u l t s of t h i s t e s t would be more meaningful i f they are i n t e r p r e t e d i n the l i g h t of a phenomenon proposed by Smith (1980) and a l s o observed by Rogers and Bateson (1990b). As they p o i n t e d out, students must have knowledge of the content t e s t e d before they can apply a t e s t - w i s e s t r a t e g y to answer a m u l t i p l e - c h o i c e item which i s flawed with t e s t - w i s e cues. I f students d i d not have knowledge or b e l i e v e d that they d i d not have knowledge r e l e v a n t to the content t e s t e d , they were unable to engage a te s t - w i s e s t r a t e g y i n an attempt to answer the item. 3.23 Student Survey (see Appendix D) T h i s instrument was made up of four s e c t i o n s . 1. Background Information Students were asked to i d e n t i f y t h e i r own e t h n i c or c u l t u r a l r o o t , and provide i n f o r m a t i o n about t h e i r p r e v i o u s experience i n w r i t i n g m u l t i p l e - c h o i c e examinations. 2. Achievement M o t i v a t i o n T h i s s e c t i o n i s composed of 16 items adapted from R u s s e l l ' s (1969) Test of Achievement M o t i v a t i o n . The o r i g i n a l s c a l e c o n s i s t e d of 30 items which measure s t u d y i n g , school m o t i v a t i o n , w i l l i n g n e s s to compete, and p e r c e i v e d achievement i n r e a d i n g , mathematics and language. The items were found to be r e l i a b l e and v a l i d f o r the p r e d i c t i o n of w i l l i n g n e s s to t r y and d e s i r e t o compete. Product moment c o r r e l a t i o n s of mo t i v a t i o n scores and achievement scores ( C a l i f o r n i a 67 Achievement Test) ranged from .60 to .72. The chosen 16 items a l l had a r e l i a b i l i t y c o e f f i c i e n t ranging from .75 to .95. 3. Sentence S t r u c t u r e and 4. E n g l i s h Usage Both of S e c t i o n s 3 and 4 of the Student Survey were adapted from the Language P r o f i c i e n c y Index (LPI) developed by E d u c a t i o n a l Measurement Research Group (EMRG) 3.24 Language Proficiency Index The Language P r o f i c i e n c y Index (LPI) i s a t e s t designed fo r the post secondary s c h o o l s to determine a student's l e v e l of competence i n E n g l i s h essay composition and r e c o g n i t i o n of common e r r o r s i n E n g l i s h usage and sentence c o n s t r u c t i o n ( E d u c a t i o n a l Research Measurement Group, EMRG, 1989). The LPI i s c u r r e n t l y used by 10 post-secondary i n s t i t u t i o n s i n B r i t i s h Columbia to p l a c e newly admitted students i n a s u i t a b l e f i r s t -year E n g l i s h course. The LPI i s composed of four s e c t i o n s - (1) Sentence S t r u c t u r e , (2) E n g l i s h Usage, (3) Paragraph Development, S t r u c t u r e and Content, and (4) Essay W r i t i n g . The f i r s t three s e c t i o n s are made up of 30 short answer q u e s t i o n s c o v e r i n g v a r i o u s t o p i c s of sentence s t r u c t u r e , E n g l i s h usage, and the development, s t r u c t u r e , and content of a paragraph. These items, are machine-scoreable, s e l e c t i o n - t y p e ( m u l t i p l e - c h o i c e ) items. The essay s e c t i o n i s marked out of 40, by two d i f f e r e n t markers. These marks are t r a n s l a t e d to a l e v e l from 0 (not w r i t t e n ) t o 6 (above-average w r i t i n g ) . The LPI i s admi n i s t e r e d through EMRG i n v a r i o u s s i t e s throughout B r i t i s h Columbia three times a year. In t h i s study, only Sentence 68 Structure (10 items) and English Usage (10 items) S e c t i o n s were used as a measure of the students' v e r b a l a b i l i t y . R e l i a b i l i t y ( i n t e r n a l c o n s i s t e n c y ) c o e f f i c i e n t s f o r the t o t a l score d e r i v e d from forms p a r a l l e l to the two s e c t i o n s used i n t h i s study ranged between .65 and .73 over the l a s t f i v e years ( c i t e d i n Rogers and Bateson, 1990a). 3.3 Pro c e d u r e For the f o r e i g n Chinese s u b j e c t s , t e s t i n g took p l a c e immediately a f t e r they wrote the January p r o v i n c i a l examinations. On the days of the examinations of Algebra 12, E n g l i s h 12, P h y s i c s 12, and H i s t o r y 12, immediately a f t e r the students completed the examinations, they were asked to v o l u n t a r i l y p a r t i c i p a t e i n the present study. They were informed about the nature of the study and asked to s i g n a consent form to permit the i n v e s t i g a t o r to o b t a i n t h e i r p r o v i n c i a l examination r e s u l t s from the M i n i s t r y of Education. Students were asked to answer a l l q u e s t i o n s as best they c o u l d f o r the S e c t i o n I of the Test of Test-wiseness and the Student Survey. For the S e c t i o n II of the Test of T e s t -wiseness, students were informed of a p e n a l t y f o r guessing formula to be used when s c o r i n g t h i s s e c t i o n . Students were asked to r e c o r d t h e i r answer on the t e s t - b o o k l e t ( f o r e i g n Chinese s u b j e c t s ) or on the answer sheets p r o v i d e d (Canadian and Chinese immigrant s u b j e c t s ) . F u r t h e r h e l p was not given d u r i n g the t e s t i n g s e s s i o n s . Students were encouraged to proceed as best they c o u l d . 69 For the f o r e i g n Chinese s t u d e n t s , the a d m i n i s t r a t i o n was done by the p r i n c i p a l i n v e s t i g a t o r . T e s t i n g r e q u i r e d about 50 minutes (30 minutes f o r the T e s t of T e s t - w i s e n e s s and 20 m inutes f o r the Student S u r v e y ) . S i n c e the two i n s t r u m e n t s measured d i f f e r e n t c o n s t r u c t s , the o r d e r of t e s t a d m i n i s t r a t i o n was l e f t t o the p r e f e r e n c e of the s t u d e n t s . For the Canadian and C h i n e s e immigrant s u b j e c t s , the t e s t i n g p r o c e d u r e was the same. Except t h a t , the a d m i n i s t r a t i o n was completed by the p r i n c i p a l i n v e s t i g a t o r s of Rogers and B a teson's study (1990a) or the c l a s s r o o m t e a c h e r s a c c o r d i n g t o the g u i d e l i n e s s e t out by the M i n i s t r y of E d u c a t i o n i n B r i t i s h Columbia. 3.4 S t a t i s t i c a l Hypotheses 3.41 Major Hypothesis The performance of the f o r e i g n C hinese s t u d e n t s on the p r o v i n c i a l e x a m i n a t i o n s i n H i s t o r y , E n g l i s h , A l g e b r a , and/or P h y s i c s i s a f u n c t i o n of t e s t - w i s e n e s s , and f o u r r e l a t e d v a r i a b l e s , namely Language P r o f i c i e n c y Index, l e n g t h of R e s i d e n c y i n Canada, Achievement M o t i v a t i o n , and M u l t i p l e C h o i c e E x p e r i e n c e . The f u l l model t e s t e d was: Y i = 00 + M l i + 0 2 x 2 i + 0 3 X 3 i + 0 4 X 4 i + 0 5 X 5 i + e i where = s c o r e s of the f o r e i g n C hinese s t u d e n t s on m u l t i p l e - c h o i c e items i n A l g e b r a , E n g l i s h , H i s t o r y , and/or P h y s i c s e x a m i n a t i o n s ; 70 = scores of the f o r e i g n Chinese students on Test-wi seness; = scores of the f o r e i g n Chinese students on the Language P r o f i c i e n c y Index; = number of months that the f o r e i g n Chinese students have been i n Canada; x 4 i = s c o r e s °f t n e f o r e i g n Chinese students on Achievement M o t i v a t i o n ; X 5 i = M u l t i p l e Choice Experience of the f o r e i g n Chinese students; and 01' 02' 03' 04' a n o - 05 = fc^e treatment e f f e c t s of the above f i v e v a r i a b l e s r e s p e c t i v e l y . The s t a t i s t i c a l h y p o t h e s i s was: H o : pop R 2y.12345 = 0 H l : pop R 2y.12345 * 0 where y = scores of the f o r e i g n Chinese students on m u l t i p l e - c h o i c e items i n H i s t o r y , E n g l i s h , A l g e b r a , and/or P h y s i c s examinations; 1 = scores of the f o r e i g n Chinese students on T e s t -wiseness; 2 = scores of the f o r e i g n Chinese students on the Language P r o f i c i e n c y Index; 3 = number of months that the f o r e i g n Chinese students have been i n Canada; 4 = scores of the f o r e i g n Chinese students on Achievement M o t i v a t i o n ; and 71 5 = scores of the f o r e i g n Chinese students on M u l t i p l e Choice Experience. The secondary hypotheses are as f o l l o w s : 3.42 Hypothesis One F o r e i g n Chinese students are not s i g n i f i c a n t l y d i f f e r e n t from Canadian students i n t h e i r scores of te s t - w i s e n e s s , as measured by the Test of Test-wiseness developed by Rogers and Bateson (1990b). The model t e s t e d was: Y i j = 0o + h x 1 i j + e i j where Y^j = scores of a l l the students on the Test of Test-wiseness; X ^ j = e t h n i c i t y of a p a r t i c u l a r group (Chinese or Canadian); and |31 = treatment e f f e c t of e t h n i c i t y . The s t a t i s t i c a l h ypothesis was Ho: 0, = 0 H1: 0 1 * 0 where = treatment e f f e c t of e t h n i c i t y 3.43 Hypothesis Two (a) The scores on the m u l t i p l e - c h o i c e items i n the P h y s i c s , A l g e b r a , H i s t o r y , and/or E n g l i s h examinations of the f o r e i g n Chinese students who possess t e s t - w i s e n e s s s k i l l s and a b i l i t i e s are s i g n i f i c a n t l y d i f f e r e n t from those of the f o r e i g n Chinese students who possess l e s s of the same a b i l i t i e s . 72 Four separate analyses f o r four s u b j e c t areas were performed. The model t e s t e d was Y i = 0o + M l i + e i where Y^ = scores of the f o r e i g n Chinese students on m u l t i p l e - c h o i c e items i n H i s t o r y , E n g l i s h , A l g e b r a , and/or P h y s i c s examinations; = scores of the f o r e i g n Chinese students on Test-wiseness; and j31 = treatment e f f e c t of Test-wiseness. The s t a t i s t i c a l h y p o thesis was: HO : 01 = 0 HI : 01 * 0 where $y = treatment e f f e c t of Test-wiseness. 3.44 Hypot hesi s Two (b) The v a r i a b i l i t y of the scores on the s e l e c t i o n items i n the P h y s i c s , Algebra, H i s t o r y , and/or E n g l i s h examinations of the f o r e i g n Chinese students who possess t e s t - w i s e n e s s s k i l l s and a b i l i t i e s i s s i g n i f i c a n t l y d i f f e r e n t from that of the f o r e i g n Chinese students who possess l e s s of the same a b i l i t i e s . The model t e s t e d was the same as hypothesis 2 (a) and the s t a t i s t i c a l h ypothesis was: . HO : o2] = a 2 2 HI : o 2! * a 2 2 where a 2 1 = t e s t score v a r i a n c e of t e s t - w i s e f o r e i g n Chinese students on m u l t i p l e - c h o i c e items 73 i n P h y s i c s , A l g e b r a , H i s t o r y , and/or E n g l i s h examinations; and a^2 = t e s t score v a r i a n c e of the t e s t - n a i v e f o r e i g n Chinese students on the same items i n the same examinations. 3.45 Hypothesis Three The scores on Test-wiseness of the f o r e i g n Chinese students s i g n i f i c a n t l y i n c r e a s e as the f o l l o w i n g v a r i a b l e s i n c r e a s e , and the f o l l o w i n g v a r i a b l e s account f o r a s i g n i f i c a n t amount of v a r i a n c e on t h e i r scores of t e s t -wiseness : 1. Language Proficiency Index (LPI), 2. Length of Residency in Canada (RES), 3. Multiple Choice Experience (MCE), and 4. Achievement Motivation (AM). The model t e s t e d was: Y i = 00 + 0 1 x 1 i + 0 2 x 2 i + 0 3 x 3 i + 0 4 x 4 i + e i where = scores of the f o r e i g n Chinese students on Test-wiseness; = scores of the f o r e i g n Chinese students on the Language P r o f i c i e n c y Index; x 2 i = n u m D e r °f months t h a t the f o r e i g n Chinese students have been i n Canada; X3j = scores of the f o r e i g n Chinese students on Achievement M o t i v a t i o n ; X4j = scores of the f o r e i g n Chinese students on M u l t i p l e Choice Experience; and 01' 02' 03' a n c ^ 04 = t * i e treatment e f f e c t s of the above four v a r i a b l e s r e s p e c t i v e l y . The s t a t i s t i c a l h ypothesis was: H 0 : pop R 2y-1234 = 0 H l : p o p R 2 y l 2 3 4 * 0 where y = scores of the f o r e i g n Chinese students on Test-wiseness; 1 = scores of the f o r e i g n Chinese students on the Language P r o f i c i e n c y Index; 2 = number of months that the f o r e i g n Chinese students have been in Canada; 3 = scores of the f o r e i g n Chinese students on Achievement M o t i v a t i o n ; and 4 = scores of the f o r e i g n Chinese students on M u l t i p l e Choice Experience. 3.5 Method of A n a l y s i s D e s c r i p t i v e s t a t i s t i c s were obtained on the performance and background i n f o r m a t i o n of the s u b j e c t s , such as age and p l a c e of o r i g i n . Before t e s t i n g the hypotheses, Pearson Product-Moment c o r r e l a t i o n s were obtained on the students' scores on the four p r o v i n c i a l examinations (Algebra, H i s t o r y , P h y s i c s , and E n g l i s h ) to determine t h e i r r e l a t i o n s h i p . T h i s would h e l p to determine i f i t was worthwhile to c o n s i d e r the four s u b j e c t areas as four separate dependent ( c r i t e r i o n ) v a r i a b l e s or not. The method of a n a l y s i s f o r each hypothesis i s as f o l l o w s : 75 3.51 Major Hypothesis T h i s h y p o t h e s i s was t e s t e d using m u l t i p l e r e g r e s s i o n to determine the amount of v a r i a n c e on each of the dependent v a r i a b l e ( s c o r e s on the H i s t o r y , E n g l i s h , A l g e b r a , and P h y s i c s examinations of the f o r e i g n Chinese students) accounted f o r by the f i v e independent v a r i a b l e s (Test-wiseness, Language P r o f i c i e n c y Index, Length of Residency i n Canada, Achievement M o t i v a t i o n , and M u l t i p l e Choice Experience of the f o r e i g n Chinese s t u d e n t s ) . The scores of the f i v e independent v a r i a b l e s were determined i n the f o l l o w i n g ways: 1. Test-Wiseness was the t o t a l scores that the students obtained from S e c t i o n I i n the Test of Test-wiseness (a t o t a l of 24 items); 2. Language P r o f i c i e n c y Index was the t o t a l score they obtained i n E n g l i s h Usage and Sentence S t r u c t u r e S e c t i o n s i n the Language P r o f i c i e n c y Index; 3. Length of Residency was the number of months they have been i n Canada; 4. Achievement M o t i v a t i o n was based on t h e i r t o t a l score i n Se c t i o n II i n the Student Survey; and 5. M u l t i p l e Choice Experience was based the t o t a l score they obtained i n qu e s t i o n s 5, 6a, 7a, and 8 i n the Student Survey, with 'yes, three or more times' coded as 3, 'yes, once or twice' coded as 2, and 'no, never' as 1. 3.52 Hypothesis One 76 Using e f f e c t coding with f o r e i g n Chinese as '1' and Canadian as t h i s hypothesis was t e s t e d using m u l t i p l e r e g r e s s i o n . With e t h n i c i t y as the c a t e g o r i c a l independent ( p r e d i c t o r ) v a r i a b l e and t e s t - w i s e n e s s as the dependent ( c r i t e r i o n ) v a r i a b l e , the t e s t determined whether e t h n i c i t y accounted f o r a s i g n i f i c a n t p r o p o r t i o n of v a r i a n c e on T e s t -wiseness. 3.53 Hypotheses Two (a) & (b) Part (a) of t h i s hypothesis was t e s t e d using m u l t i p l e r e g r e s s i o n . With Test-wiseness of t h e " f o r e i g n Chinese students as the independent v a r i a b l e and t h e i r scores on the P h y s i c s , A l g e b r a , H i s t o r y , and/or E n g l i s h examinations as the dependent v a r i a b l e , the t e s t determined whether the scores of Test-wiseness accounted f o r a s i g n i f i c a n t amount of v a r i a n c e on the p r o v i n c i a l examinations. Four separate analyses were performed i n four d i f f e r e n t s u b j e c t a r e a s . Part (b) of t h i s h y p o thesis was t e s t e d using B a r l e t t ' s t e s t of homogeneity of v a r i a n c e s . F o r e i g n Chinese students were d i v i d e d i n t o two groups, a c c o r d i n g to the c u t - o f f c r i t e r i a of t e s t - w i s e n e s s - '17 or above' f o r t e s t - w i s e and '11 or below' f o r t e s t - n a i v e . Based on the standard d e v i a t i o n s of the performance of the two groups on the p r o v i n c i a l examinations i n the four s u b j e c t s areas, the B a r l e t t ' s t e s t of homogeneity of v a r i a n c e was done to determine whether the v a r i a b i l i t y of the scores of the t e s t -wise f o r e i g n Chinese students was s i g n i f i c a n t l y d i f f e r e n t from that of the t e s t - n a i v e f o r e i g n Chinese students. 77 3.54 Hypothesis Three T h i s hypothesis was t e s t e d by m u l t i p l e r e g r e s s i o n u s i n g f o r c e d e n t r y and backward e l i m i n a t i o n . With t e s t - w i s e n e s s as the dependent ( c r i t e r i o n ) v a r i a b l e and the Language P r o f i c i e n c y Index, Achievement M o t i v a t i o n , Length of Residency i n Canada and M u l t i p l e Choice Experience as the independent ( p r e d i c t o r ) v a r i a b l e s , t h i s h y p othesis was t e s t e d to assess whether the amount of v a r i a n c e accounted f o r i n tes t - w i s e n e s s by the four v a r i a b l e s would be s i g n i f i c a n t . A l l the four independent v a r i a b l e s were entered as i n t e r v a l - s c a l e v a r i a b l e s . A l s o , by e l i m i n a t i n g the v a r i a b l e s backwardly one at a time and c a l c u l a t i n g the d i f f e r e n c e i n the increment of va r i a n c e accounted by each of them, i t was p o s s i b l e to determine the unique v a r i a n c e c o n t r i b u t e d by each v a r i a b l e to Test-wiseness. The alpha l e v e l set f o r the s i g n i f i c a n c e t e s t of v a r i a n c e accounted f o r by the independent v a r i a b l e s of the dependent v a r i a b l e s was 0.05. To f u r t h e r explore the nature of test-wis e n e s s among Chinese students, item a n a l y s i s was performed on the Test of Test-wiseness among the f o r e i g n Chinese students. A l s o , to a f f i r m the r e l a t i o n s h i p of tes t - w i s e n e s s and e t h n i c i t y , a comparison was made among f o r e i g n Chinese, Chinese immigrant and Canadian students on the Test of Test-wiseness, the Language P r o f i c i e n c y Index, and Achievement M o t i v a t i o n . Owing to the l i m i t a t i o n of the small sample s i z e f o r the f o r e i g n Chinese s u b j e c t s , no c r o s s - v a l i d a t i o n check c o u l d be 78 c a r r i e d out. T h e r e f o r e , the r e g r e s s i o n a n a l y s i s was only a b l e to e x p l o r e , r a t h e r than p r e d i c t the r e l a t i o n s h i p of the v a r i a b l e s i n v e s t i g a t e d . 79 CHAPTER FOUR ANALYSIS AND RESULTS T h i s c h a p t e r p r e s e n t s the r e s u l t s of t h i s s t u d y i n t h r e e p a r t s . The f i r s t p a r t i s demographic i n f o r m a t i o n r e g a r d i n g the f o r e i g n C h i n e s e sample. The t e s t s of hypotheses a r e p r e s e n t e d i n t h e second p a r t . The l a s t p a r t compares the Chin e s e sample i n t h i s s t u d y w i t h the Canadian and Chinese s t u d e n t s i n Rogers and Bateson's s t u d y (1990a). D e s c r i p t i v e s t a t i s t i c s were c a l c u l a t e d f o r responses on items i n the Student Survey. Items i n the T e s t of T e s t -w i s e n e s s and the f o u r Grade 12 p r o v i n c i a l e x a m i n a t i o n s ( A l g e b r a , E n g l i s h , H i s t o r y , and P h y s i c s ) were a n a l y z e d . The assumptions of n o r m a l i t y , independence, homogeneity of v a r i a n c e , o u t l i e r s , and l i n e a r i t y u n d e r l y i n g the s t a t i s t i c a l models were examined by a n a l y z i n g the r e s i d u a l s . Where m u l t i p l e r e g r e s s i o n was employed, the t r e a t m e n t e f f e c t of the independent v a r i a b l e s was e x p r e s s e d i n terms of s t a n d a r d i z e d r e g r e s s i o n c o e f f i c i e n t s (/3). S i n c e s t a n d a r d i z e d r e g r e s s i o n c o e f f i c i e n t s were based on s t a n d a r d s c o r e s , a comparison of the r e l a t i v e importance among the v a r i a b l e s was f a c i l i t a t e d . 4.1 Background I n f o r m a t i o n of the C h i n e s e Sample 4.11 Demographic Characteristics The C h i n e s e sample i n t h i s s tudy c o n s i s t e d of 41 s t u d e n t s , 21 males and 20 fema l e s . T w e n t y - f i v e s u b j e c t s d i d not r e p o r t t h e i r date of b i r t h . For the 26 s t u d e n t s who r e p o r t e d , t h e i r median age was 19.39 y e a r s which was s l i g h t l y 80 o l d e r than that of the Canadian students (Median = 18.98 y e a r s ) . A l l of the Chinese students were born o u t s i d e of Canada. They were a l l f u l l time students i n Coquitlam C o l l e g e which i s an independent school c a t e r i n g mainly to f o r e i g n s t u d e n t s . Among the 41 s u b j e c t s , there were 32 from Hong Kong, three from Indonesia, three from Macau, two from China, and one from Singapore. A l l i d e n t i f i e d Chinese as t h e i r e t h n i c or c u l t u r a l r o o t . T h e i r l e n g t h of Residency i n Canada v a r i e d from f i v e to 30 months, with M = 12.53 months and SD = 5.95. 4.12 Experience in Mul l i pi e-c hoi ce Examinations Questions 5 to 9 i n the Student Survey d e a l t with t h e i r p r e v i o u s experience i n w r i t i n g m u l t i p l e - c h o i c e examinations, both at home and i n Canada. D e t a i l e d r e s u l t s t o these q u e s t i o n s are d i s p l a y e d i n Table 6. In q u e s t i o n 5, 23 (56.1%) r e p o r t e d 'yes, three or more times' i n w r i t i n g m u l t i p l e - c h o i c e examinations i n t h e i r home countr y . When they were asked i f they had r e c e i v e d any formal coaching i n t e s t t a k i n g (question 6a), 23 (56.1%) s a i d 'no, never'. Among the 18 students who s a i d 'yes', 10 (55.6%) r e c e i v e d coaching i n t h e i r home c o u n t r i e s . S i m i l a r l y , i n qu e s t i o n 7a, 'having i n f o r m a l c o n s u l t a t i o n on t e s t - t a k i n g from t e a c h e r s , parents or f r i e n d s ' , 23 (56.1%) s a i d , 'no, never'; and among the 18 students who s a i d ' y e s 1 , 10 (55.6%) r e c e i v e d i t i n t h e i r home c o u n t r i e s . In 'using p r e v i o u s p r o v i n c i a l examination papers f o r p r a c t i c e ' (question 8), 25 (61%) s a i d , 'yes, once or twice' but those who s a i d 'yes, three or more 81 times', 4 (9.8%) were fewer than those s a i d 'no, never', 12 (29.3%). Twenty-two p o i n t f i v e percent i n d i c a t e d that they would w r i t e or had w r i t t e n p r o v i n c i a l s c h o l a r s h i p examinations but most of them (77.5%) denied i n t e r e s t i n the s c h o l a r s h i p program. Table 6 Experience i n M u l t i p l e - c h o i c e Examinations of Chinese Sample N (%) N (%) N (%) Questions 'No,never t 'Yes,once ' or twice' Yes, more three < times' 5 (MCE at home) 8 (19. 5) 10 (24.4) 23 (56.1) 6a (Formal Coaching) 23 (56. 1) 13 (31.7) ' 5 (12.2) 7a (Informal Consultati on) 23 (56. 1) 13 (31.7) 5 (12.2) 8 (Provincial Exam Practice) 12 (29. 3) 25 (61) 4 (9.8) Canada Home country Elsewhere 6b (Formal Coaching) 2 (11. 1) 10 (55.6) 6 (33.3) 7b (Informal Consul t at i on) 3 (16. 6) 10 (55.6) 5 (27.8) Yes No 9 (Schol ars hip Exami nation) 9 (22.5) 31 (77.5) Note: r e f e r to the Student Survey i n Appendix D f o r the s p e c i f i c wording of the q u e s t i o n s 82 In the subsequent t e s t s of hypotheses, a t o t a l score of experience i n m u l t i p l e - c h o i c e examinations was based on the sum of scores on qu e s t i o n s 5, 6a, 7a and 8. The t o t a l score was c o n s i d e r e d to represent the best q u a n t i t a t i v e measure of student's p r i o r experience i n t h i s type of examination. I t c o n s t i t u t e d the v a r i a b l e termed ' M u l t i p l e Choice Experience' (MCE) used i n the f o l l o w i n g a n a l y s e s . 4.13 Grade Twelve Provincial Examinations The f o r e i g n Chinese s u b j e c t s i n t h i s study were those who wrote any one or more of the f o l l o w i n g Grade 12 p r o v i n c i a l examinations i n January of 1990: Algebra , E n g l i s h , H i s t o r y , and/or P h y s i c s . Among the 41 f o r e i g n Chinese s u b j e c t s , 35 wrote Algeb r a , 16 wrote E n g l i s h , four wrote H i s t o r y and three wrote P h y s i c s . Since there were only a very small number of f o r e i g n Chinese students who wrote H i s t o r y and Ph y s i c s i n t h i s study, t h e i r performance may not represent the o v e r a l l Chinese p o p u l a t i o n . T h e r e f o r e , only the r e s u l t s i n Algebra and E n g l i s h were used i n the f o l l o w i n g t e s t s of hypotheses i n which p r o v i n c i a l examinations were used as the dependent v a r i a b l e s . The f o r e i g n Chinese students' examination r e s u l t s i n the m u l t i p l e - c h o i c e s e c t i o n along with the p r o v i n c i a l r e s u l t s are presented i n Table 7. Comparatively, Chinese students performed b e t t e r i n Algebra (58.2%) than i n E n g l i s h (33.8%), H i s t o r y (35%) or Ph y s i c s (33.3%). Compared to the p r o v i n c i a l examination r e s u l t s , the average raw score of the f o r e i g n Chinese students on the m u l t i p l e c h o i c e s e c t i o n was lower than that of the 83 p r o v i n c e i n both Algebra (66.6%) and E n g l i s h (77.0%). P a r t i c u l a r l y i n E n g l i s h 12, f o r e i g n Chinese students scored l e s s than h a l f of the scores obtained by average students i n the p r o v i n c e . Table 7 Grade 12 P r o v i n c i a l Examination R e s u l t s i n the M u l t i p l e Choice  S e c t i o n of the Chinese Students and the P r o v i n c i a l Average Algebra E n g l i s h Chinese Province Chinese Province Number of students 35 3251 16 6423 M 29. 1 4 33.3 9. 1 3 20.80 SD 8. 15 8.72 2. 78 3.35 Mean % C o r r e c t 58. 30 66.6 33. 8 77.0 T o t a l No. of Items 50 50 27 27 Note: The above p r o v i n c i a l examination r e s u l t s were s t a t e d i n raw s c o r e s . Please r e f e r to Chapter Two S e c t i o n 2.12, 'The Examination-Development Process' on the procedure of tra n s f o r m i n g raw scores to percentages and r e p o r t e d grades. 84 Furthermore, a c o r r e l a t i o n c o e f f i c i e n t was computed between the sco r e s of the f o r e i g n Chinese s u b j e c t s on the Algebra and E n g l i s h examinations. The r e s u l t i n g c o r r e l a t i o n (r = .27) was not s t a t i s t i c a l l y s i g n i f i c a n t . C o r r e l a t i o n c o e f f i c i e n t s f o r the other s u b j e c t areas c o u l d not be computed because of the small sample s i z e s . In the subsequent t e s t s of hypotheses, separate analyses were performed with Algebra and E n g l i s h scores as the dependent v a r i a b l e s (major hypothesis and hypothesis 2a). 4.2 T e s t s of Hypotheses The r e s e a r c h hypotheses presented i n Chapter Three were presented here again f o r easy r e f e r e n c e . The c r i t e r i o n f o r r e j e c t i o n of the hypotheses was the alpha l e v e l of .05. 4.21 Provincial Examinations, Test-wiseness and Four Cor r el ates The major h y p o t h e s i s s t a t e d that the performance of the f o r e i g n Chinese students on the p r o v i n c i a l examinations (Algebra and E n g l i s h ) would be a f u n c t i o n of test-wis e n e s s (TW), and four r e l a t e d v a r i a b l e s , namely v e r b a l a b i l i t y (as measured by the L P I ) , l e n g t h of Residency i n Canada (RES), Achievement M o t i v a t i o n (AM), and M u l t i p l e Choice Experience (MCE). Before proceeding with the a n a l y s i s , c o r r e l a t i o n s were computed among a l l the v a r i a b l e s to determine whether the independent v a r i a b l e s were c o r r e l a t e d . As seen i n Table 8, the c o r r e l a t i o n matrix f o r a l l the v a r i a b l e s , three 85 c o r r e l a t i o n c o e f f i c i e n t s were s t a t i s t i c a l l y s i g n i f i c a n t : Test-wiseness and E n g l i s h scores (r = .44, p<.0S), E n g l i s h scores and Achievement M o t i v a t i o n (r = .62, p<.0'\) and Residency and Test-wiseness (r = -.29, p<.05). The c o r r e l a t i o n s among other independent v a r i a b l e s were r e l a t i v e l y low, t h e r e f o r e , i t was concluded that the problem of m u l t i c o l l i n e a r i t y was not l i k e l y to s e r i o u s l y a f f e c t the r e s u l t s of the r e g r e s s i o n a n a l y s i s . Table 8 Pearson C o r r e l a t i o n s of F i v e Independent V a r i a b l e s and Two  Dependent V a r i a b l e s 1 2 3 4 5 6 ALG ENG TW LPI RES AM 1. ALG 1 2. ENG .27 (13) 1 3. TW -.075 (35) .44* (16) 1 4. LPI -.01 (35) .35 (16) .067 (41) 1 5. RES -.12 (35) -.41 (16) -.29* (41) -.22 (41) 1 6. AM -.26 (35) .62** (16) .024 (41) .073 (41) -.082 (41 ) 1 7. MCE -.12 (35) -.0075 (16) -.056 (41) -.025 (41). .092 (41) .18 (41) * /?<.05 ** JP<.01 ( ) = (number of cases) ALG=Algebra exam scores ENG=English exam scores TW =Test-wiseness LPI=Language P r o f i c i e n c y Index RES=Residency i n Canada AM =Achievement M o t i v a t i o n MCE=Multiple Choice Experience 7 MCE 86 M u l t i p l e r e g r e s s i o n with f o r c e d entry was performed to determine whether the amount of v a r i a n c e c o n t r i b u t e d by the independent v a r i a b l e s (Test-wiseness, Language P r o f i c i e n c y Index, Residency i n Canada, Achievement M o t i v a t i o n and M u l t i p l e Choice Experience) to the dependent v a r i a b l e s (Algebra or E n g l i s h scores) was s i g n i f i c a n t l y d i f f e r e n t from z e r o . The assumptions u n d e r l y i n g the model as i d e n t i f i e d at the beginning of t h i s Chapter were met. Two separate r e g r e s s i o n a n a l y s e s were performed with Algebra and E n g l i s h examination scores as dependent v a r i a b l e s r e s p e c t i v e l y . R e s u l t s of the r e g r e s s i o n analyses are presented i n Table 9. As seen i n Table 9, when a l l the independent v a r i a b l e s were f o r c e d i n , they c o n t r i b u t e d about 10% of the v a r i a n c e to Algebra scores or R2 = .099 and that amount of v a r i a n c e was not s t a t i s t i c a l l y s i g n i f i c a n t . Although the amount of v a r i a n c e c o n t r i b u t e d by a l l v a r i a b l e s to E n g l i s h scores was h i g h e r , the r e g r e s s i o n c o e f f i c i e n t {R2 = .66) was s t i l l not s t a t i s t i c a l l y s i g n i f i c a n t . T h e r e f o r e , the n u l l hypothesis was t e n a b l e . N e i t h e r Algebra nor E n g l i s h scores was a f u n c t i o n of Test-wiseness, Language P r o f i c i e n c y Index, l e n g t h of Residency i n Canada, Achievement M o t i v a t i o n and M u l t i p l e Choice Experience. Using backward e l i m i n a t i o n , the independent v a r i a b l e s were d e l e t e d from the equation one at a time i n order to observe the c o n t r i b u t i o n of each independent v a r i a b l e to the equation. The c r i t e r i o n f o r e l i m i n a t i o n was based on the d e f a u l t value (.1) of F-to-remove. Table 9 Summary S t a t i s t i c s of Major Hypothesis Steps R2 R2 Change F V a r i a b l e s Removed Dependent V a r i a b l e = ALG {N = 35) 0 .099 — .15 ( A l l in) 1 .095 --.0043 .21 LPI 2 .084 -.011 .28 MCE 3 .079 -.0057 .43 AM 4 .031 --.047 .35 RES 5 .00 -.031 .00 TW Dependent V a r i a b l e = ENG (N = 16) 0 .66 — 2.75 ( A l l in) 1 .66 -.0007 3.91* AM 2 .62 -.04 4.93* TW 3 .53 -.096 5.55* MCE 4 .46 -.063 9.49** LPI * p<.05 ** /K.01 ALG=Algebra exam scores ENG=English exam scores TW =Test-wiseness LPI=Language P r o f i c i e n c y Index RES=Residency i n Canada AM =Achievement m o t i v a t i o n MCE=Multiple-choice Experience 88 As seen i n Table 9, with Algebra as the dependent v a r i a b l e , F - t e s t s of R2 were not s t a t i s t i c a l l y s i g n i f i c a n t i n a l l the s t e p s . A l s o , the d e l e t i o n of each independent v a r i a b l e l e d to very l i t t l e l o s s of v a r i a n c e ( i ^ - c h a n g e ) , ranging from 5% to l e s s than 1%. None of the /^-changes w e r e s t a t i s t i c a l l y s i g n i f i c a n t . In the l a s t step, when a l l the independent v a r i a b l e s were d e l e t e d , the l o s s of R2 remained not s t a t i s t i c a l l y s i g n i f i c a n t . Furthermore, the f i r s t v a r i a b l e d e l e t e d was the Language P r o f i c i e n c y Index (LPI), a measure of the students' v e r b a l a b i l i t y , i n d i c a t i n g that i t c o n t r i b u t e d the l e a s t amount of v a r i a n c e to the Algebra examination s c o r e s . On the other hand, Test-wiseness was the l a s t v a r i a b l e to be removed from the equation, which suggested i t s g r e a t e r c o n t r i b u t i o n to the v a r i a n c e i n the Algebra examination s c o r e s . Since none of /^-changes were s t a t i s t i c a l l y s i g n i f i c a n t , no f i r m i m p l i c a t i o n c o u l d be drawn. I t should be noted that the l a c k of r e l a t i o n s h i p between the f i v e independent v a r i a b l e s and the Algebra examination scores was f u r t h e r demonstrated i n the small s t a n d a r d i z e d r e g r e s s i o n weights and the corresponding a d j u s t e d R2 (see Appendix A). When the E n g l i s h examination scores were used as the dependent v a r i a b l e , F - t e s t s of J?2 were s i g n i f i c a n t at the alpha = .05 l e v e l i n steps 1 (F[4, 8] = 3.91), 2 (F[3, 9] = 4.93), and 3 (F[2, 10] = 5.55) and at .01 l e v e l i n step 4 (F[1, 11] = 9.49) (see Table 9). T h i s r e s u l t suggested that the v a r i a b l e s remaining i n the equation c o n t r i b u t e d a s i g n i f i c a n t amount of v a r i a n c e to the E n g l i s h examination 89 s c o r e s . Moreover, the ^ - c h a n g e w a s s l i g h t l y l a r g e r than that noted i n Alg e b r a , ranging from l e s s than 1% to 10%. The f i r s t v a r i a b l e removed from the equation was Achievement M o t i v a t i o n and the l a s t one was len g t h of Residency i n Canada. As seen in Appendix A, The ad j u s t e d R2 (.42) was s l i g h t l y lower than the R2 (.66) and the s t a n d a r d i z e d r e g r e s s i o n weights were g e n e r a l l y higher than those when the Algebra examination scores were used as the dependent v a r i a b l e . However, s i n c e none of the .^-changes w e r e s t a t i s t i c a l l y s i g n i f i c a n t , c o n c l u s i v e statements about the c o n t r i b u t i o n of each v a r i a b l e c o u l d not be made. The s t a b i l i t y of the estimates was a f f e c t e d by the small sample s i z e i n t h i s a n a l y s i s . 4.22 Ethnicity and Test-wiseness Hypothesis one s t a t e d that f o r e i g n Chinese students would not d i f f e r s i g n i f i c a n t l y from Canadian students on t h e i r scores of t e s t - w i s e n e s s , as measured by the Test of T e s t -wiseness. Using e f f e c t coding with Chinese as '1' and Canadian as '-1', t h i s h y p o thesis was t e s t e d u s i n g the l i n e a r r e g r e s s i o n model. The Canadian sample was randomly drawn from the students i n Rogers and Bateson's study (1990a). The f o r e i g n Chinese students i n t h i s study were compared t o the Canadian st u d e n t s . The dependent v a r i a b l e was Test-wiseness and the independent v a r i a b l e was E t h n i c i t y . The assumptions u n d e r l y i n g the model as i d e n t i f i e d i n the beginning of t h i s 90 Chapter were met. Summary s t a t i s t i c s are presented i n Table 10. Table 10 Summary S t a t i s t i c s of Hypothesis One Et h n i c Groups Chinese Canadian (N = 41 ) (AT = 41) Mean" 10.49 14.37 Standard D e v i a t i o n 2.89 2.60 R2 .34 Adjusted R2 .33 F(1, 80) 40.85** Std. Regression Weights .58 ** /><.01 As seen i n Table 10, e t h n i c i t y accounted f o r 34% of the va r i a n c e on t e s t - w i s e n e s s , and t h i s f i n d i n g was s t a t i s t i c a l l y s i g n i f i c a n t at .01 l e v e l . In other words, the n u l l hypothesis was not t e n a b l e . Chinese students performed s i g n i f i c a n t l y poorer (M = 10.49, SD = 2.89) than the Canadian students (M = 14.37, SD = 2.6) i n the Test of Test-wiseness. 4.23 Provincial Examinations and Test-wiseness Hypothesis two (a) s t a t e d that the scores on the m u l t i p l e - c h o i c e items i n E n g l i s h and Algebra examinations of the f o r e i g n Chinese students who possessed t e s t - w i s e n e s s s k i l l s and a b i l i t i e s would be s i g n i f i c a n t l y d i f f e r e n t from 91 those of the f o r e i g n Chinese students who possessed l e s s of the same a b i l i t i e s . Again, t h i s h y p o thesis was t e s t e d by a l i n e a r r e g r e s s i o n model. Two separate analyses were performed with the two dependent v a r i a b l e s , Algebra or E n g l i s h Examination. Assumptions u n d e r l y i n g the model as i d e n t i f i e d at the beginning of t h i s Chapter were g e n e r a l l y met, with the e x c e p t i o n of some lack of homogeneity of v a r i a n c e . However, s i n c e the departure from homogeneity of v a r i a n c e was not l a r g e , the r e g r e s s i o n model was s t i l l apt f o r p r e d i c t i o n purpose. As seen i n Table 11, the summary s t a t i s t i c s f o r t h i s h y p o t h e s i s , the amount of v a r i a n c e i n the two p r o v i n c i a l examination scores accounted f o r by t e s t - w i s e n e s s was very s m a l l . Although the R2 and s t a n d a r d i z e d r e g r e s s i o n weights i n E n g l i s h were higher (R2 - .13, 0 = .36) than those i n Algebra (R2 = .0001, 0 = .0086), F - t e s t s f o r R2 i n both analyses were not s t a t i s t i c a l l y s i g n i f i c a n t . T h e r e f o r e , the n u l l hypothesis was t e n a b l e . The c o n c l u s i o n that a r e l a t i o n s h i p e x i s t e d between Test-wiseness and Algebra or E n g l i s h examination scores c o u l d not be j u s t i f i e d . Hypothesis two (b) s t a t e d that the v a r i a b i l i t y of the scores on the s e l e c t i o n items i n Algebra and E n g l i s h examinations of the f o r e i g n Chinese students who possessed t e s t - w i s e s k i l l s and a b i l i t i e s would be s i g n i f i c a n t l y d i f f e r e n t from that of the f o r e i g n Chinese students who possessed l e s s of the same a b i l i t i e s . 92 Table 11 Summary S t a t i s t i c s of Hypothesis Two (a) Subject N R2 Adjusted R2 F ( 1 , 10) Std Reg. Areas Weights Algebra 35 .0001 -.10 .001 .0086 E n g l i s h 16 .13 .04 1.47 .36 Th i s h y p o t h e s i s c o u l d not be t e s t e d because t e s t - n a i v e students (n = 28) very much outnumbered t e s t - w i s e students (n = 1). Even i f the c r i t e r i a of tes t - w i s e n e s s i s lowered to '14 or above', a median p o i n t between 11 (the upper bound f o r t e s t - n a i v e group) and 17 (the lower bound f o r t e s t - w i s e group), the t e s t - w i s e group would only i n c r e a s e from one to four s t u d e n t s . The two groups would s t i l l be so unbalanced that the comparison of the v a r i a b i l i t y of t h e i r scores would not be r e l i a b l e . Moreover, the newly c l a s s i f i e d t e s t - w i s e group would probably not be competently t e s t - w i s e so the va r i a n c e of t h e i r scores would probably not d i f f e r s i g n i f i c a n t l y from that of the t e s t - n a i v e group. 4.24 Correlates of Test-wiseness Hypothesis three s t a t e d that the scores of test-wis e n e s s of the f o r e i g n Chinese students would s i g n i f i c a n t l y i n c r e a s e as t h e i r scores i n the Language P r o f i c i e n c y Index, Length of Residency i n Canada, M u l t i p l e Choice Experience, and Achievement M o t i v a t i o n i n c r e a s e d ; and these v a r i a b l e s would account f o r a s i g n i f i c a n t amount of v a r i a n c e of t h e i r scores on t e s t - w i s e n e s s . 93 T h i s h y pothesis was a l s o t e s t e d using a m u l t i p l e l i n e a r r e g r e s s i o n model. Assumptions u n d e r l y i n g the model as i d e n t i f i e d i n the beginning of t h i s Chapter were met. R e s u l t s presented i n Table 8 demonstrated t h a t the c o r r e l a t i o n s among the four independent v a r i a b l e s (Language P r o f i c i e n c y Index, Residency i n Canada, M u l t i p l e Choice Experience, Achievement M o t i v a t i o n ) were g e n e r a l l y low to moderate, ranging from .006 to .48. With t h i s being the case, the problem of m u l t i c o l l i n e a r i t y was not l i k e l y to a f f e c t the r e s u l t s of t e s t i n g t h i s h y p o t h e s i s . Summary s t a t i s t i c s of t h i s h y p o t h e s i s are presented i n Table 12. Table 12 Summary S t a t i s t i c s of Hypothesis Three Steps R2 J? 2-Change F Std Regress Weights V a r i a b l e s removed MCE LPI AM RES 0 .083 .083 .81 -.031 .002 .0069 -.28 ( A l l 1 .083 .00 1.11 -.031 .007 -.28 LPI 2 .083 .00 1 .71 -.030 -.28 AM 3 .082 .001 3.47 -.29 MCE LPI=Language P r o f i c i e n c y Index RES=Residency i n Canada AM =Achievement M o t i v a t i o n MCE=Multiple Choice Experience As seen i n Table 12, the amount of v a r i a n c e accounted f o r by a l l four independent v a r i a b l e s to test-wi s e n e s s was 8.3%, but t h i s q u a n t i t y was not s t a t i s t i c a l l y s i g n i f i c a n t . The 94 v a r i a b l e s were removed from the equation one at a time a c c o r d i n g the d e f a u l t c r i t e r i a s t a t e d i n the major h y p o t h e s i s . When the Language P r o f i c i e n c y Index, Achievement M o t i v a t i o n , and M u l t i p l e Choice Experience were removed one at a time, there was very l i t t l e J? 2-change. The e l i m i n a t i o n stopped at step 4 when the d e l e t i o n of r e s i d e n c y i n Canada would produce s i g n i f i c a n t l o s s of R2. However, none of the F - t e s t s f o r R2 and /? 2-change were s t a t i s t i c a l l y s i g n i f i c a n t . T h e r e f o r e , c o n c l u s i v e statements c o u l d not be made about the r e l a t i o n s h i p of these v a r i a b l e s to t e s t - w i s e n e s s . Furthermore, the l a c k of r e l a t i o n s h i p , between test-w i s e n e s s and the four v a r i a b l e s was demonstrated i n the small r e g r e s s i o n weights. I t was t h e r e f o r e , concluded that the n u l l h ypothesis was t e n a b l e . Language P r o f i c i e n c y Index, Achievement M o t i v a t i o n , M u l t i p l e Choice Experience and Residency i n Canada d i d not account f o r s i g n i f i c a n t v a r i a n c e in Test-wiseness among the Chinese s t u d e n t s . 4.3 Comparison of foreign Chinese, Chinese immigrant, and Canadian students Since the Chinese students i n t h i s study were a l l f o r e i g n students who r e c e n t l y came to study i n Canada, they might not repre s e n t other Chinese p o p u l a t i o n s i n Canada. Comparison of the Chinese students in t h i s study with other Chinese students would h e l p to assess and c o n f i r m the r e l a t i o n s h i p between e t h n i c i t y and t e s t - w i s e n e s s . T h e r e f o r e , the Chinese students used i n t h i s study (termed as ' f o r e i g n Chinese') were compared 95 with some Chinese students who have been i n Canada f o r a r e l a t i v e l y longer p e r i o d of time (termed as 'Chinese immigrant'). A comparable sample of Chinese immigrant students was drawn from those students who p a r t i c i p a t e d i n Rogers and Bateson's study (1990a) and who a l s o wrote e i t h e r Grade 12 E n g l i s h , Algebra, P h y s i c s , or H i s t o r y P r o v i n c i a l Examinations i n June 1989. One-way A n a l y s i s of V a r i a n c e was performed to compare the three e t h n i c groups, f o r e i g n Chinese, Chinese immigrant, and Canadian, on t h e i r scores on Test-wiseness, Language P r o f i c i e n c y Index, and Achievement M o t i v a t i o n . Post hoc m u l t i p l e comparisons were done using the Student-Newman-Keul procedure. Since the sample s i z e s of the three e t h n i c groups were small and unbalanced, the harmonic mean (n = 38.78) was used to weight the three samples e q u a l l y d u r i n g the post hoc a n a l y s i s (Howell, 1982). Table 13 compares the performance of the three e t h n i c groups on t h e i r o v e r a l l scores i n Test-wiseness, Language P r o f i c i e n c y Index, and Achievement M o t i v a t i o n . O v e r a l l F-t e s t s showed t h a t the three groups were s i g n i f i c a n t l y d i f f e r e n t i n t h e i r o v e r a l l scores i n t e s t - w i s e n e s s , F(2, 114) = 27.11, /K.01 and v e r b a l a b i l i t y , F(2, 114) = 89.14, p<.0l. M u l t i p l e comparisons showed that the f o r e i g n Chinese students were s i g n i f i c a n t l y lower (M = 10.49) than both Canadian (M = 14.14) and Chinese immigrant (M = 14.37), p<.05 i n t e s t -wiseness but the l a t e r two groups were not s i g n i f i c a n t l y d i f f e r e n t . 96 Table 13 Comparison of Three E t h n i c Groups on Test-wiseness, Language  P r o f i c i e n c y Index and Achievement M o t i v a t i o n E t h n i c Groups Fo r e i g n Chinese Canadian Chinese Immigrant Dependent (A^  = 41) {N = 35) . (N = 41) V a r i a b l e M {SD) M {SD) M {SD) F{2, 114) TW 10.49 (2.89) 14.14 (2.4 ) LPI' 4.46 (2.17) 9.91 (4.27) AM 42.98 (7.64) 41.89 (6.13) 14.37 (2.6 ) 13.61 (2.73) 40.51 (5.8 ) 27.11** 89.14** 1 .43 * * /7<.01 TW=Test-wiseness LPI= Language P r o f i c i e n c y Index AM=Achievement M o t i v a t i o n On the Language P r o f i c i e n c y Index, f o r e i g n Chinese students performed the worst {M = 4.46), with Chinese immigrant second {M = 9.91), and Canadian the best {M = 13.61). These f i n d i n g s were a l l s i g n i f i c a n t a t the .01 l e v e l . In terms of Achievement M o t i v a t i o n , no s i g n i f i c a n t d i f f e r e n c e s were observed among the three e t h n i c groups. F u r t h e r comparisons were made among the three e t h n i c groups on the f i v e s u b t e s t s i n the Test of Test-wiseness. The r e s u l t s are presented i n Table 14. 97 Table 14 Comparison of Three E t h n i c Groups on F i v e Subtests i n the Test  of Test-wiseness E t h n i c Groups Fo r e i g n Chinese Canadian Subtests Chinese Immigrant M (SD) M (SD) M (SD) F(l, 1 14) ID1 2 .32(1.27) 3 .68(1.17) 3 .69(1.28) 16. 19** ID2 2 .98(1.23) 4 .5 (1.31) 4 .0 (1.16) 10. 63** ID3 3 .44(1.36) 2 .83(1.30) 3 .0 (1.61) 2. 0 IIB4 1 .76(1.09) 3 .71(1.25) 3 .46(1.34) 30. 37** Guess 1 1 .95( .5 ) 17 .78(5.02) 14 .83(4.53) 23. 14** ** p <.01 ID1=Absurd o p t i o n s ID2=Similar o p t i o n s Guess=Guessing ID3=Different o p t i o n s IIB4=Stem-options Link O v e r a l l F - t e s t s i n Absurd-options (F[1, 114] = 16.19), S i m i l a r - o p t i o n s (F[1, 114] = 10.63), D i f f e r e n t - o p t i o n s (F[1, 114] = 30.37), and guessing (F[1, 114] = 23.14) were a l l s i g n i f i c a n t at the .01 l e v e l , showing that the three e t h n i c groups performed s i g n i f i c a n t l y d i f f e r e n t on these s u b t e s t s . However, i t was concluded that the items i n the guessing subtest d i d not s a t i s f a c t o r i l y d i s c r i m i n a t e between t e s t - w i s e and t e s t - n a i v e students, and that might a f f e c t i t s r e l i a b i l i t y i n measuring the student's l e v e l of t e s t - w i s e n e s s . The guessing subtest was, t h e r e f o r e , dropped from the subsequent 98 analyses and d i s c u s s i o n . The c a l c u l a t i o n of the t o t a l scores of Test-wiseness was based only on four s u b t e s t s , ID1, ID2, ID3, and IIB4. M u l t i p l e comparisons u s i n g the Student-Newman-Keul procedure were performed on the three s i g n i f i c a n t s u b t e s t s , ID1, ID2, and IIB4. As seen i n F i g u r e 2, the g r a p h i c a l p r e s e n t a t i o n of the r e s u l t s of the post hoc m u l t i p l e comparisons, f o r e i g n Chinese students were s i g n i f i c a n t l y lower than both Canadian and Chinese immigrant students i n ID1, ID2, and IIB4. The d i f f e r e n c e between Canadian and Chinese immigrant was not s t a t i s t i c a l l y s i g n i f i c a n t i n these three s u b t e s t s . O v e r a l l , the f o r e i g n Chinese students performed s i g n i f i c a n t l y lower than the other two groups who performed more s i m i l a r l y . Among the four s u b t e s t s i n the Test of Test-wiseness, f o r e i g n Chinese performed the worst i n IIB4 ( D i f f e r e n t o p t i o n s ) {M = 1.76). However, i n ID3 ( S i m i l a r O p t i o n s ) , f o r e i g n Chinese students performed s l i g h t l y but not s i g n i f i c a n t b e t t e r than Canadian s t u d e n t s . The i m p l i c a t i o n of these d i f f e r e n c e s was d i s c u s s e d i n the subsequent chapter. 99 m Foreign Chinese • Canadian Chinese Immigrant ID1 ID2 ID3 IIB4 Subtests Note: * s i g n i f i c a n t l y d i f f e r e n t from the other two groups F i g u r e 2 - Graphic Comparison of Three E t h n i c Groups on Four Subtests in the T e s t of T e s t - w i s e n e s s CHAPTER FIVE DISCUSSION T h i s f i n a l c h a p t e r p r o v i d e s a r e v i e w of the f i n d i n g s and t h e i r i n t e r p r e t a t i o n i n r e l a t i o n t o t h e t h e o r i e s and i s s u e s i n r e s e a r c h c o n s i d e r e d i n e a r l i e r c h a p t e r s . The f i r s t p a r t i s the i n t e r p r e t a t i o n of the f i n d i n g s . A summary of the f i n d i n g s and the c o n c l u s i o n s a r e p r e s e n t e d i n t h e second p a r t . P a r t t h r e e i n c l u d e s the l i m i t a t i o n s of the s t u d y , f o l l o w e d by i m p l i c a t i o n s and d i r e c t i o n s f o r f u r t h e r r e s e a r c h i n p a r t f o u r . 5.1 I n t e r p r e t a t i o n of F i n d i n g s 5.11 Ethnicity and Test-wiseness I t was h y p o t h e s i z e d t h a t f o r e i g n C h i n e s e s t u d e n t s would be s i g n i f i c a n t l y d i f f e r e n t from Canadian s t u d e n t s i n T e s t -w i s e n e s s as measured by the Test of T e s t - w i s e n e s s (Rogers and Ba t e s o n , 1990b). The d a t a s u p p o r t e d t h a t f o r e i g n C h i n e s e s t u d e n t s were l e s s t e s t - w i s e , as compared t o b o t h Canadian and Ch i n e s e immigrant s t u d e n t s . Some p r e v i o u s s t u d i e s a l s o s u p p o r t e d the c l a i m t h a t C h i n e s e s t u d e n t s were l e s s t e s t - w i s e . S i m i l a r t o b o t h Lo and S l a k t e r ' s (1973) and Wu and S l a k t e r ' s (1978) s t u d i e s , f o r e i g n C h i n e s e s t u d e n t s s c o r e d s i g n i f i c a n t l y lower i n Stem-options L i n k , A b s u r d - o p t i o n s and S i m i l a r - o p t i o n s s u b t e s t s i n the Test of T e s t - w i s e n e s s . D i f f e r e n t from t h e i r s t u d i e s , the i n s t r u m e n t s used i n t h i s s t u d y f o r b o t h e t h n i c groups were i n E n g l i s h , the problem of t r a n s l a t i o n or the p e c u l i a r c h a r a c t e r i s t i c s of Chinese c h a r a c t e r s would not account f o r 101 the d i f f e r e n c e i n test-wiseness between Chinese and Canadian or American st u d e n t s . In other words, t h i s study d i d not support t h e i r e x p l a n a t i o n f o r d i f f e r e n t t e s t - w i s e n e s s behavior of Chinese s t u d e n t s . Since a l l the instruments used were i n E n g l i s h , the problem of E n g l i s h p r o f i c i e n c y among the f o r e i g n Chinese students became apparent. As Dreibach and Keogh (1982) s t a t e d , f o r c h i l d r e n who came from a background where the primary language of the home was not standard E n g l i s h , t e s t -wiseness became very important. Some pr e v i o u s s t u d i e s f u r t h e r p o i n t e d out t h a t s t a n d a r d i z e d t e s t s were h e a v i l y v e r b a l l y loaded. F a m i l i a r i t y of the language of the t e s t would l i k e l y be a b e n e f i t t o t e s t performance (Samuda, 1975). C h i l d r e n who had a l r e a d y mastered the language c o n s i s t e n t l y outperformed those who were i n the process of l e a r n i n g the language (Anastasi and Cordova, 1953). The f o r e i g n Chinese students i n t h i s study a l l have been i n Canada f o r not more than 30 months. A l s o , over 90% of them were English-as-a-second-language st u d e n t s . T h e i r low l e v e l of E n g l i s h competency was c l e a r l y demonstrated i n t h e i r s i g n i f i c a n t l y lower scores on the Language P r o f i c i e n c y Index than both Chinese immigrants and Canadians. Without a competent l e v e l of E n g l i s h p r o f i c i e n c y , t h e i r a b i l i t y to apply t e s t - w i s e n e s s s k i l l s , such as a b i l i t y to read i n s t r u c t i o n s and p i c k i n g up cues from the stems, are l i k e l y l i m i t e d . Although Test-wiseness and the Language P r o f i c i e n c y Index were not s i g n i f i c a n t l y c o r r e l a t e d , the impact of E n g l i s h upon 102 Test-wiseness among the f o r e i g n Chinese c o u l d be demonstrated' i n d i r e c t l y . Compared to other s u b t e s t s i n the Test of T e s t -wiseness, the IIB4 (Stem-options Link) s u b t e s t r e q u i r e s more E n g l i s h language s k i l l s i n order to d e t e c t the grammatic cues i n the stem that enable t e s t - w i s e examinees to choose the most probable o p t i o n . The f o r e i g n Chinese students performed the worst i n t h i s s u b t e s t . On some items i n t h i s subtest (e.g., items 2, 4, and 5), the p-values or the p r o p o r t i o n of examinees who answered the item c o r r e c t l y (Crocker and A l g i n a , 1986) were approximately 25%, at the random guessing l e v e l . F u r t h e r examination of these items r e v e a l s some p o i n t s worthy of d i s c u s s i o n . F i r s t , i n most items, the d i s c r i m i n a t i o n i n d i c e s or p o i n t b i s e r i a l c o r r e l a t i o n s were a c c e p t a b l e . Among the 24 items i n S e c t i o n I of the Test of Test-wiseness (see Appendix E ) , 16 of them (items number 1, 2, 3, 4, 7, 9, 11, 13, 14, 15, 17, 19, 20, 22, 23, and 24) had i n d i c e s ranged between .30 and .60, a s a t i s f a c t o r y range suggesting that no r e v i s i o n of the items was necessary (Crocker and A l g i n a , 1986). A l s o , p o s i t i v e v a l u e s were l a r g e l y found i n the c o r r e c t o p t i o n s and the c o r r e l a t i o n s f o r other f o i l s were mostly negative (see Appendix B). The item a n a l y s e s showed that the Test of Test-wiseness d i d p r o p e r l y d i s c r i m i n a t e between the more and the l e s s t e s t - w i s e students. However, the p-values of the f o r e i g n Chinese students were not much above that which would be expected from random guessing. 103 Second, the stems and o p t i o n s of these items are n e i t h e r p a r t i c u l a r l y lengthy nor complex i n sentence s t r u c t u r e or vocabulary. For example, in item 11, (see Appendix E) C h a r l e s Dicken's Hard Times d e a l s with ... a) the d i f f i c u l t l i f e of a f a c t o r y worker. b) the p o l i t i c s of the French Chateau county. c) the c o u r t of King Edward I I I . d) the l i m i t a t i o n s of European e x i s t e n t i a l i s m . Probably, i t i s not too demanding f o r a Grade 12 student to know that "Hard Times" means " d i f f i c u l t l i f e " , and o p t i o n a) would be the most t e s t - w i s e guess. However, fewer than 50% of the students chose t h i s o p t i o n . Given a t e s t item w r i t t e n i n o r d i n a r y academic language, such as the above example, f o r e i g n Chinese students were s t i l l unable to p i c k grammatic cues from the stem to make a t e s t -wise guess. T h i s c o u l d be e x p l a i n e d by two p o s s i b l e reasons. F i r s t , the students r e a l l y might not have known that "Hard Times" means " d i f f i c u l t l i f e " , so they c o u l d not see any resemblance between the stem and t h i s o p t i o n . In t h i s case, the reading l e v e l of the Test of Test-wiseness needs to be lowered i n order to s u i t the E n g l i s h standard of these students. Second, they might have la c k e d t e s t - w i s e n e s s s k i l l s t hat enabled them to look f o r cues to answer a q u e s t i o n about which they had no knowledge. The second e x p l a n a t i o n would be more p l a u s i b l e because the response p a t t e r n throughout the Test of Test-wiseness was very c l o s e to random guessing, r e g a r d l e s s of content areas or l i n g u i s t i c s t r u c t u r e . 104 Compared to the Canadian and Chinese immigrant students, f o r e i g n Chinese students were s i g n i f i c a n t l y t e s t - n a i v e . In the group of 41 s u b j e c t s used i n t h i s study, there was only one student who c o u l d be c l a s s i f i e d as 'te s t - w i s e ' as d e f i n e d by the c r i t e r i a of a score of 17 or above. Given t h i s extremely r e s t r i c t e d range of te s t - w i s e n e s s , t e s t - w i s e n e s s c o u l d h a r d l y be expected to r e l i a b l y p r e d i c t the t e s t performance of the f o r e i g n Chinese students. Although f o r e i g n Chinese students were found to be l e s s t e s t - w i s e than Canadian and Chinese immigrant students, t h i s study c o u l d only show that they c o u l d not, and d i d not apply t e s t - w i s e n e s s s k i l l s as w e l l as the other two groups. T h i s study cannot conclude that f o r e i g n Chinese students lack t e s t -wiseness s k i l l s . In f a c t , t h i s r a i s e s an important q u e s t i o n about whether e m p i r i c a l t e s t s of test-wis e n e s s a c t u a l l y measure students' l e v e l of test - w i s e n e s s or j u s t the a b i l i t y to apply the s k i l l s , or to read. 5.12 British Col umbi a Provincial Examinations and Test-wi s enes s R e s u l t s showed that the v a r i a n c e accounted f o r by T e s t -wiseness f o r both the Algebra and E n g l i s h examination scores was not s i g n i f i c a n t . A r e l a t i o n s h i p between te s t - w i s e n e s s and p r o v i n c i a l examination s c o r e s , t h e r e f o r e c o u l d not be e s t a b l i s h e d . However, test-wi s e n e s s was s i g n i f i c a n t l y c o r r e l a t e d with E n g l i s h examination s c o r e s . Furthermore, i n the t e s t of the major h y p o t h e s i s , when E n g l i s h scores were used as the 105 c r i t e r i o n v a r i a b l e , the Language P r o f i c i e n c y Index was the l a s t v a r i a b l e removed from the equation. A l s o , F - t e s t of the v a r i a n c e accounted by the Language P r o f i c i e n c y Index f o r E n g l i s h s cores was s t a t i s t i c a l l y s i g n i f i c a n t . T h i s r e s u l t demonstrates i n d i r e c t l y a s t r o n g v e r b a l component i n t e s t -wiseness. Students who performed w e l l i n the E n g l i s h Examination were more t e s t - w i s e and a l s o more competent i n the E n g l i s h language. The important r o l e of v e r b a l a b i l i t y i n t e s t - w i s e n e s s i s f u r t h e r r e v e a l e d i n the l a c k of a r e l a t i o n s h i p between Algebra scores and t e s t - w i s e n e s s and the Language P r o f i c i e n c y Index r e s p e c t i v e l y . The Algebra examination demands mainly mathematical computation and manipulation of symbols. Students t h e r e f o r e do not r e q u i r e as much E n g l i s h language s k i l l to perform w e l l i n Algebra as they do i n the Test of Test-wiseness or the E n g l i s h Examination. However, the r e l a t i o n s h i p between p r o v i n c i a l examinations and t e s t - w i s e n e s s was not f i r m l y e s t a b l i s h e d i n t h i s study. A p o s s i b l e reason i s r e l a t e d to the r e s t r i c t e d range of v a l u e s i n Test-wiseness f o r the s u b j e c t s used i n t h i s study. Among the f o r e i g n Chinese, only one out of 41 students was c l a s s i f i e d as t e s t - w i s e , with the m a j o r i t y of them l y i n g i n the lower end of the continuum of Test-wiseness. One assumption u n d e r l y i n g simple l i n e a r r e g r e s s i o n i s that the values of the p r e d i c t o r s are evenly spaced at i n t e r v a l s a long the continuum of i n t e r e s t (Pedhazur, 1982). However, given the r e s t r i c t e d range of v a l u e s i n test-wiseness of the 106 s u b j e c t s and the i n f l a t e d standard e r r o r of r e g r e s s i o n c o e f f i c i e n t s , t h i s assumption was v i o l a t e d i n the study. T h e r e f o r e , u s i n g test-wiseness to p r e d i c t p r o v i n c i a l examination i n t h i s study was prone to have c o n s i d e r a b l e e r r o r . There i s another probable reason that t e s t - w i s e n e s s d i d not account f o r a s i g n i f i c a n t amount of v a r i a n c e on the p r o v i n c i a l examinations. As Smith (1980) and Rogers and Bateson (1990a) p o i n t e d out, students who l a c k e d knowledge of the content of an item would guess randomly among a l l the o p t i o n s f o r the item. Students who had i n s u f f i c i e n t knowledge to answer the item d i r e c t l y would e l i m i n a t e some and guess from the remaining o p t i o n s . The students' random guessing behavior t h e r e f o r e c o n t r i b u t e d to the measurement e r r o r and low i n t e r n a l c o n s i s t e n c i e s i n the Test of Test-wiseness which attenuated the c o r r e l a t i o n s between Test-wiseness and the other measures i n t h i s study. As demonstrated i n the r e s u l t s , the t e s t of h y p o t h e s i s 2b c o u l d not be c a r r i e d out because of very unbalanced group s i z e s between t e s t - w i s e and t e s t - n a i v e students. T h e r e f o r e , the study f a i l e d to support or r e f u t e S l a k t e r et a l . ' s (1970) and Gibb's (1964) argument f o r l a r g e r v a r i a b i l i t y among the scores of t e s t - w i s e students than those of t e s t - n a i v e students. Another i n t e r e s t i n g f i n d i n g i s the s i g n i f i c a n t c o r r e l a t i o n between the E n g l i s h examination scores and Achievement M o t i v a t i o n . Students who were more achievement motivated performed b e t t e r on the E n g l i s h p r o v i n c i a l examination. According to an E n g l i s h teacher of the f o r e i g n Chinese students, E n g l i s h 12 i s the subj e c t that the f o r e i g n Chinese students fear most. Most of these students came to Canada with a r e l a t i v e l y s a t i s f a c t o r y standard of knowledge i n Alge b r a , so they u s u a l l y c o u l d complete both Grades 11 and 12 Algebra i n one or two semesters. However, most of them would not be competent t o w r i t e the E n g l i s h 12 examination u n t i l they had completed at l e a s t two or more semesters of E n g l i s h . Some have completed a l l other Grade 12 course requirements, but they s t i l l have not passed the E n g l i s h 12 p r o v i n c i a l examination. T h e r e f o r e , those who performed w e l l i n E n g l i s h are l i k e l y t o be those who are more a c a d e m i c a l l y competent, s e r i o u s , and hard-working students. 5.13 Correlates of Test-wiseness R e s u l t s showed that the amount of v a r i a n c e accounted f o r i n Test-wiseness by v e r b a l a b i l i t y (as measured by the Language P r o f i c i e n c y Index), Achievement M o t i v a t i o n , Length of Residency i n Canada, and M u l t i p l e Choice Experience was not s t a t i s t i c a l l y s i g n i f i c a n t . Test-wiseness d i d not c o r r e l a t e with any of these four v a r i a b l e s , except Length of Residency i n Canada. The lack of s i g n i f i c a n t c o r r e l a t i o n s might be r e l a t e d to the u n r e l i a b i l i t y of the Test of Test-wiseness as e x p l a i n e d above. A negative c o r r e l a t i o n was found between Test-wiseness and Length of Residency i n Canada, implying that the longer the f o r e i g n Chinese students stay i n Canada, the l e s s t e s t - w i s e they a r e . 108 Such a f i n d i n g i s most l i k e l y due to chance. F i r s t , the c o r r e l a t i o n c o e f f i c i e n t was not very strong (r = -.29). With a small f o r e i g n Chinese sample, i t i s p o s s i b l e to a r r i v e at such a c o r r e l a t i o n c o e f f i c i e n t by chance. Second, l o g i c a l l y , i t i s u n l i k e l y t h a t the f o r e i g n Chinese students become l e s s t e s t - w i s e as they stay longer i n Canada. T h e i r major goal of coming to Canada i s to a t t e n d h i g h s c h o o l , pass the p r o v i n c i a l examinations, and then pursue t e r t i a r y e d u c a t i o n . T h e r e f o r e , they should have been exposed to more and more t e s t i n g s i t u a t i o n s while they were p r e p a r i n g f o r the p r o v i n c i a l examinations. However, there i s another s i d e of the argument. Mere p h y s i c a l r e s i d e n c y i n Canada does not ensure that a student w i l l be more t e s t - w i s e . Many other f a c t o r s come i n t o p l a y : language a b i l i t y , s k i l l s i n w r i t i n g m u l t i p l e choice examinations, e t c . T h e r e f o r e , b e s i d e s the l e n g t h of Residency in Canada, the type of experience the students encounter s i n c e a r r i v a l a l s o need to be c o n s i d e r e d . F u r t h e r examination of the r e l a t i o n s h i p of Length of Residency and other v a r i a b l e s r e v e a l s some i n t e r e s t i n g f i n d i n g s . The c o r r e l a t i o n c o e f f i c i e n t s between Length of Residency and other v a r i a b l e s were a l l n e g a t i v e , except the one with M u l t i p l e Choice E x p e r i e n c e . Although the c o e f f i c i e n t s were very low and not s t a t i s t i c a l l y s i g n i f i c a n t , they i m p l i e d that the longer the students stayed i n Canada, the poorer t h e i r performance i n A l g e b r a , E n g l i s h , T e s t -wiseness, and Language P r o f i c i e n c y Index; and a l s o the lower t h e i r Achievement M o t i v a t i o n . However, f u r t h e r e x p l o r a t i o n of 109 the data seemed to suggest a c u r v i l i n e a r r e l a t i o n s h i p between Length of Residency and other v a r i a b l e s . T h i s i m p l i e s that those who have stayed i n Canada f o r the longest p e r i o d were those who scored the lowest i n most of the v a r i a b l e s . I f they were a c a d e m i c a l l y competent, they would have completed t h e i r courses w i t h i n a reasonable p e r i o d (between 18 to 24 months). Ther e f o r e , those who scored lowest i n the other v a r i a b l e s were those a c a d e m i c a l l y l e a s t capable s t u d e n t s . Although M u l t i p l e Choice Experience d i d not account f o r a s i g n i f i c a n t amount of v a r i a n c e i n t e s t - w i s e n e s s , i t was the l a s t v a r i a b l e removed from the equation (see Table 12). In terms of experience i n w r i t i n g m u l t i p l e c h o i c e examinations, the s u b j e c t s were not t o t a l l y n a i v e . Over h a l f of them had experience w r i t i n g t h i s type of examination i n t h e i r home c o u n t r i e s . However, over h a l f of them d i d not have any inf o r m a l or formal c o n s u l t a t i o n i n e x a m i n a t i o n - w r i t i n g s k i l l s . T h i s i s r e l a t e d to what Sarnacki (1979) p o i n t e d out, "mere experience i n t e s t i n g does not guarantee f u t u r e success on t e s t s , nor does i t q u a l i f y an examinee as a s k i l l e d t e s t -t a k e r " (p. 204). Most of the f o r e i g n Chinese students had experience i n w r i t i n g m u l t i p l e c h o i c e examinations, but very few of them had lea r n e d the s k i l l s of w r i t i n g them. Th e r e f o r e , they may r e p e a t e d l y use some i n a p p r o p r i a t e s k i l l s which may r e s u l t i n a performance that does not a c c u r a t e l y r e f l e c t what they know. In t h i s way, mere M u l t i p l e Choice Experience does not seem to s a t i s f a c t o r i l y p r e d i c t or e x p l a i n Test-wiseness. 110 Achievement M o t i v a t i o n a l s o accounted f o r very l i t t l e v a r i a n c e i n t e s t - w i s e n e s s . However, the f o r e i g n Chinese students' mean scores i n t h i s t e s t was high but the standard d e v i a t i o n was very low, r e s u l t i n g i n a very r e s t r i c t e d range of v a l u e s i n Achievement M o t i v a t i o n . Since both the p r e d i c t o r (Achievement M o t i v a t i o n ) and the c r i t e r i o n ( test-wiseness) l a y w i t h i n a r e s t r i c t e d range of v a l u e s , the p r e d i c t i o n was prone to be ambiguous because the u n d e r l y i n g assumption of simple l i n e a r r e g r e s s i o n was v i o l a t e d . As seen i n Table 12, the Language P r o f i c i e n c y Index was the f i r s t v a r i a b l e removed from the r e g r e s s i o n equation, showing that i t accounted f o r the l e a s t amount of v a r i a n c e i n t e s t - w i s e n e s s . T h e r e f o r e , t h i s study d i d not show the d i r e c t r e l a t i o n s h i p between v e r b a l a b i l i t y and Test-wiseness that had p r e v i o u s l y been demonstrated in other s t u d i e s ( B a j t e l s m i t , 1977; Diamond and Evan, 1972; Rowley, 1974). N e v e r t h e l e s s , t h i s study should not c a s t doubt on the r e l a t i o n s h i p between te s t - w i s e n e s s and v e r b a l a b i l i t y . F i r s t , the mean score of the f o r e i g n Chinese students was extremely low and r e s t r i c t e d i n range. The assumption of simple l i n e a r r e g r e s s i o n was v i o l a t e d and so p r e d i c t i o n c o u l d not be a c c u r a t e . Second, some i n d i r e c t evidence f o r the r e l a t i o n s h i p between v e r b a l a b i l i t y and t e s t - w i s e n e s s c o u l d be demonstrated i n t h i s study. The IIB4 (Stem-options Link) subtest i n the Test of T e s t -wiseness r e q u i r e s a r e l a t i v e l y higher l e v e l of E n g l i s h competence i n order to d e t e c t the grammatic cues in the stem that a i d choosing the o p t i o n s . T h i s was the subtest on which I l l the f o r e i g n Chinese students performed the worst. A l s o , a f t e r the t e s t a d m i n i s t r a t i o n , s e v e r a l students t o l d the examiner that they d i d not understand most of the qu e s t i o n s i n the Test of Test-wiseness and the two t e s t s of the Language P r o f i c i e n c y Index, so most of t h e i r answers were l a r g e l y based on guessing. 5.2 Summary of F i n d i n g s and C o n c l u s i o n s T h i s i n v e s t i g a t i o n began with the primary o b j e c t i v e s of examining c u l t u r a l d i f f e r e n c e s i n te s t - w i s e n e s s and the impact of t e s t - w i s e n e s s on B r i t i s h Columbia Grade 12 P r o v i n c i a l Examinations among f o r e i g n Chinese s t u d e n t s . Two e t h n i c groups were compared. The impact of Test-wiseness on two p r o v i n c i a l examination scores was s t u d i e d . The r e l a t i o n between t e s t - w i s e n e s s and four v a r i a b l e s was a l s o examined. The f i n d i n g s and the c o n c l u s i o n s that can be drawn from t h i s study are presented below. 1. E t h n i c i t y accounted f o r a s i g n i f i c a n t amount of v a r i a n c e i n t e s t - w i s e n e s s . F o r e i g n Chinese students were s i g n i f i c a n t l y l e s s t e s t - w i s e than both Canadian and Chinese immigrant students, who performed very s i m i l a r l y . Among the f i v e s u b t e s t s i n the Test of Test-wiseness, f o r e i g n Chinese students performed s i g n i f i c a n t l y lower i n ID1 (Absurd-o p t i o n s ) , ID2 ( D i f f e r e n t - o p t i o n s ) , IIB4 (Stem-options Link) and Guessing. No s i g n i f i c a n t d i f f e r e n c e s were found i n ID3 ( S i m i l a r - o p t i o n s ) among the three e t h n i c groups, and i n a l l f i v e s u b t e s t s between Canadian and Chinese immigrants. The 112 low scores of t e s t - w i s e n e s s among f o r e i g n Chinese students was probably i n p a r t due to inadequate E n g l i s h language s k i l l s as demonstrated i n t h e i r lowest score i n IIB4. 2. Test-wiseness d i d not account f o r a s i g n i f i c a n t amount of v a r i a n c e i n E n g l i s h or A l g e b r a examination s c o r e s . However, E n g l i s h scores d i d c o r r e l a t e s i g n i f i c a n t l y with t e s t -wiseness. The reason that t e s t - w i s e n e s s d i d not p r e d i c t p r o v i n c i a l examination scores might be r e l a t e d to a r e s t r i c t e d range of v a l u e s found i n t e s t - w i s e n e s s among the f o r e i g n Chinese s t u d e n t s . 3. Achievement M o t i v a t i o n , the Language P r o f i c i e n c y Index, Length of Residency i n Canada, and M u l t i p l e Choice Experience together accounted f o r very l i t t l e v a r i a n c e i n t e s t - w i s e n e s s . None of them s i g n i f i c a n t l y c o r r e l a t e d with t e s t - w i s e n e s s , except l e n g t h of Residency i n Canada. The longer the f o r e i g n Chinese students stayed i n Canada, the l e s s t e s t - w i s e they were. T h i s r e s u l t i s more l i k e l y due to chance. A l s o , i t i m p l i e s that a mere presence i n Canada does not guarantee an improvement i n t e s t i n g t a k i n g s k i l l s . The lack of r e l a t i o n s h i p between M u l t i p l e Choice Experience and Test-wiseness c a l l s f o r a q u a l i t a t i v e c o n s i d e r a t i o n of M u l t i p l e Choice Experience because mere experience i n w r i t i n g m u l t i p l e c h o i c e examination does not n e c e s s a r i l y enhance t e s t -wise s k i l l s . Owing to the r e s t r i c t e d range of v a l u e s obtained i n Achievement M o t i v a t i o n , and the Language P r o f i c i e n c y Index, the assumption of simple l i n e a r r e g r e s s i o n was v i o l a t e d . 113 Using these two v a r i a b l e s to p r e d i c t Test-wiseness was t h e r e f o r e u n r e l i a b l e . 4 . Comparison was made among f o r e i g n Chinese, Chinese immigrant, and Canadian students on Test-wiseness, the Language P r o f i c i e n c y Index, and Achievement M o t i v a t i o n . In Achievement M o t i v a t i o n , a l l three groups were high and not s i g n i f i c a n t l y d i f f e r e n t from each o t h e r . In terms of the Language P r o f i c i e n c y Index and Test-wiseness, f o r e i g n Chinese performed the worst, t h e i r mean scores were no b e t t e r than chance. The Chinese immigrants, although s i g n i f i c a n t l y lower than Canadians, were b e t t e r than f o r e i g n Chinese students on the Language P r o f i c i e n c y Index. With b e t t e r E n g l i s h language a b i l i t y , Chinese immigrants performed as w e l l as Canadians i n Test-wiseness. With poorer v e r b a l a b i l i t y and s h o r t e r r e s i d e n c y i n Canada, f o r e i g n Chinese students were l e s s t e s t -wise than Chinese immigrant students, d e s p i t e the f a c t that they were of the same e t h n i c o r i g i n . Even w i t h i n the same e t h n i c group, there i s d i v e r s i t y . T h i s i m p l i e s that e t h n i c i t y per se may not be the c r u c i a l f a c t o r i n t e s t - w i s e n e s s . Instead, the q u a l i t y and the amount of p r i o r experience i n t e s t - t a k i n g t hat the e t h n i c groups possess needs to be c o n s i d e r e d as w e l l . 5.3 Limitations of the Study L i k e a l l r e s e a r c h , the present i n v e s t i g a t i o n had l i m i t a t i o n s . The c o n c l u s i o n s drawn above should t h e r e f o r e be 114 c o n s i d e r e d i n the l i g h t of the l i m i t a t i o n s d i s c u s s e d i n t h i s p a r t . The major l i m i t a t i o n of t h i s study was the e x t e r n a l v a l i d i t y and g e n e r a l i z a b i l i t y of the r e s u l t s . The major t h r e a t to the e x t e r n a l v a l i d i t y was the sampling problem i n t h i s study. The f o r e i g n Chinese students were drawn e n t i r e l y from one s c h o o l . A school e f f e c t might confound some of the r e s u l t s about f o r e i g n Chinese s t u d e n t s . T h e r e f o r e , g e n e r a l i z i n g these r e s u l t s to other Chinese students cannot be j u s t i f i e d . T h i s i n v e s t i g a t i o n was mainly a e x p l o r a t o r y study that was intended to pr o v i d e some i n s i g h t s to the t h e o r e t i c a l f o r m u l a t i o n of t e s t - w i s e n e s s . Probably due to the sampling problem, the f o r e i g n Chinese students tended to behave very s i m i l a r l y . T h e i r response p a t t e r n s a l l l a y w i t h i n a very l i m i t e d range - i n the lower end along the continuum of Test-wiseness and the Language P r o f i c i e n c y Index, and i n the upper end along the continuum of Achievement M o t i v a t i o n . T h e r e f o r e , p r e d i c t i o n was very s u s c e p t i b l e t o e r r o r because the p r e d i c t o r s l a y i n a r e s t r i c t e d range of values and had l i t t l e v a r i a n c e among them. Test-wiseness, t h e r e f o r e , d i d not seem to p r e d i c t p r o v i n c i a l examination r e s u l t s , nor d i d i t c o r r e l a t e with the Language P r o f i c i e n c y Index, Achievement M o t i v a t i o n , M u l t i p l e Choice Experience and l e n g t h of Residency i n Canada. Another sampling r e l a t e d problem arose when p r o v i n c i a l examinations were used as the c r i t e r i o n v a r i a b l e . O r i g i n a l l y , four p r o v i n c i a l examination s u b j e c t areas were to be 115 i n v e s t i g a t e d . Owing to the small number of students who wrote P h y s i c s and H i s t o r y , these s u b j e c t s c o u l d not be s t u d i e d . A l s o , the numbers of students who wrote E n g l i s h were, i n f a c t , so small that the estimates might not be a c c u r a t e . F i n a l l y , the c o m p a r a b i l i t y of the Canadian and Chinese sample poses another l i m i t a t i o n to t h i s study. F o r e i g n Chinese and Canadian students wrote the p r o v i n c i a l examinations i n d i f f e r e n t years ( f o r e i g n Chinese students i n January 1990 and Chinese immigrant and Canadian students i n June 1989). T h e r e f o r e , d i r e c t comparison between the two groups on the p r o v i n c i a l examinations c o u l d not be made. 5.4 I m p l i c a t i o n s and D i r e c t i o n s f o r F u r t h e r R e s e a r c h The major i m p l i c a t i o n drawn i n t h i s study was that language p l a y e d a c r u c i a l p art i n t e s t - t a k i n g s i t u a t i o n s . Test-wiseness i s a s k i l l that can be a p p l i e d to t a k i n g t e s t s and examinations, but the a b i l i t y t o apply any t e s t - t a k i n g s k i l l s that a student possesses to performance on an examination i s contingent upon a c e r t a i n standard of language f a c i l i t i e s . With low scores on the Language P r o f i c i e n c y Index, the f o r e i g n Chinese students o b v i o u s l y d i d not a t t a i n a b a s i c standard that was necessary to comprehend the examination q u e s t i o n s . In f a c t , they admitted t h i s themselves. I f they d i d not s a t i s f a c t o r i l y comprehend the q u e s t i o n , t h e i r scores on the Test of Test-wiseness was not an i n d i c a t i o n of t h e i r test-wiseness a b i l i t y , but r a t h e r a p r o b a b i l i t y score of random guessing. That might be one of 116 the reasons why they scored the h i g h e s t on the Guessing s u b t e s t . T h e r e f o r e , f o r students who had not been i n Canada long enough and whose E n g l i s h f a c i l i t i e s were very l i m i t e d , i t may be more important to improve t h e i r E n g l i s h language s k i l l s , then to coach them i n techniques used to w r i t e m u l t i p l e c h o i c e examinations. Although the study showed that e t h n i c i t y accounted f o r s i g n i f i c a n t v a r i a n c e i n t e s t - w i s e n e s s , e t h n i c i t y per se probably may not be the primary f a c t o r i n determining t e s t - w i s e n e s s . The d i f f e r e n c e observed between the f o r e i g n Chinese and Chinese immigrant students i s a case i n p o i n t . With v e r b a l a b i l i t y being one type of p r i o r experience • a f f e c t i n g c u l t u r a l d i f f e r e n c e s i n t e s t - w i s e n e s s , t h i s study a l s o demonstrated that q u a l i t y r a t h e r than q u a n t i t y of p r i o r experience was probably more important. A mere p h y s i c a l presence i n Canada or mere experience i n w r i t i n g M u l t i p l e Choice examinations does not guarantee f u t u r e improvement or success i n s i m i l a r examinations. N e v e r t h e l e s s , formal coaching or guidance on how to w r i t e m u l t i p l e c h o i c e examination may be q u i t e important i n improving students' t e s t - w i s e n e s s s k i l l s . The random guessing behavior of the f o r e i g n Chinese students i s c o n s i s t e n t with the model of t e s t - t a k i n g behavior proposed by Smith (1980). As Smith claimed, a t e s t - t a k e r would respond randomly i f he/she had no knowledge of the p e r c e i v e d content being t e s t e d , and he/she had exhausted a l l h i s / h e r t e s t - w i s e s t r a t e g i e s i n making use of the item flaws to a r r i v e 117 at an answer. Th e r e f o r e , t h i s study r a i s e s doubt on Millman et a l . ' s c l a i m that t e s t - w i s e n e s s i s " l o g i c a l l y independent of the examinee's knowledge of the s u b j e c t matter f o r which the items are supposedly measure" (p. 707). Rogers and Bateson (1990a) a l s o came up with a s i m i l a r o b s e r v a t i o n that i f the students possessed no knowledge of the content being t e s t e d , or d i d not p e r c e i v e themselves as having the knowledge, they were very l i k e l y to guess randomly. In t h i s way, te s t - w i s e n e s s s t r a t e g i e s may not be totally independent of the examinee's knowledge of the sub j e c t matter being t e s t e d . Another important e d u c a t i o n a l i m p l i c a t i o n can be drawn from the hig h achievement m o t i v a t i o n among the f o r e i g n Chinese students. However, Achievement M o t i v a t i o n does not seem to c o r r e l a t e with Test-wiseness. The reason c o u l d be that although the students are h i g h l y motivated a c a d e m i c a l l y , they may not possess the a p p r o p r i a t e t e s t - t a k i n g s k i l l s to improve t h e i r academic performance d u r i n g examinations. That i s , i f they can improve t h e i r t e s t - t a k i n g s k i l l s , they may perform b e t t e r i n the examination. In q u e s t i o n s 13 to 15 on the Achievement M o t i v a t i o n t e s t (see Appendix D), over h a l f of them s a i d they almost always t r i e d hard to get the r i g h t answers i n the examination, s t u d i e d hard f o r the t e s t s they take i n s c h o o l , and that grades were almost always important to them. However, i n q u e s t i o n 16, over h a l f of the s u b j e c t s b e l i e v e d that t h e i r grades almost always r e a l l y showed what they knew. Apparently, they have t r i e d hard to study f o r the t e s t s but they d i d not r e a l i z e t h at there might be other 118 extraneous f a c t o r s , such as t e s t - t a k i n g s k i l l s , that would a f f e c t t h e i r t e s t performance. Educating them on the t e s t -t a k i n g s k i l l s t h e r e f o r e would probably h e l p improving t h e i r t e s t performance. To i n c r e a s e the understanding of whether test-wi s e n e s s would b e n e f i t the t e s t performance of the f o r e i g n Chinese students, f u r t h e r r e s e a r c h should examine the e f f e c t of coaching of t e s t - t a k i n g s k i l l s . Students c o u l d be f i r s t t e s t e d on t h e i r i n i t i a l l e v e l of t e s t - w i s e n e s s , f o l l o w e d by coaching l e s s o n s on t e s t - w i s e n e s s s k i l l s , and then a p o s t -t e s t of t h e i r l e v e l of t e s t - w i s e n e s s . T h i s would h e l p to i n c r e a s e t h e i r awareness of t e s t - t a k i n g s k i l l s , hence e n a b l i n g them to apply the s k i l l s to improve t h e i r t e s t performance. Since the sampling problem poses a major t h r e a t to the e x t e r n a l v a l i d i t y , f u r t h e r study can be improved by drawing a l a r g e r sample from d i f f e r e n t types of s c h o o l s . T h i s would probably ensure a wider range of student a b i l i t i e s . More important, to i n c r e a s e the understanding of t h e i r t e s t - t a k i n g behavior and the nature of t h e i r d e f i c i e n c y i n examination w r i t i n g s k i l l s , a c a r e f u l follow-up case study, and i n d i v i d u a l i n t e r v i e w and o b s e r v a t i o n s would be b e n e f i c i a l . T h i s would h e l p to e x p l a i n how Chinese students approach and s o l v e a problem, and f i n d out i f the methods they use r e a l l y d i f f e r from those of the Canadians or Chinese immigrants. T h i s would a l s o h e l p to assess the extent to which E n g l i s h language s k i l l s a f f e c t students' t e s t performance. In a d d i t i o n , the reading l e v e l of the t e s t i n g instruments should be lowered to a l e v e l a p p r o p r i a t e to the f o r e i g n Chinese students who are i n 119 the process of l e a r n i n g the E n g l i s h language. The t e s t r e s u l t s would then more a c c u r a t e l y measure the c o n s t r u c t of i n t e r e s t r a t h e r than E n g l i s h language a b i l i t y . In choosing the p r e d i c t o r s of t e s t - w i s e n e s s , both q u a l i t a t i v e as w e l l as q u a n t i t a t i v e components need to be c o n s i d e r e d . For example, a q u a l i t a t i v e measure of the students' p r i o r experience with m u l t i p l e c h o i c e examinations, such as experience i n t a k i n g examination coaching c l a s s e s , may be a more r e l i a b l e p r e d i c t o r of t e s t - w i s e n e s s . In terms of Length of Residency i n Canada, a more v a l i d i n d i c a t o r may be the experience and frequency of i n t e r a c t i o n with E n g l i s h speaking people and the type of a c t i v i t i e s i n which students have engaged s i n c e they came to Canada. To o b t a i n a more d i r e c t r e l a t i o n s h i p between v e r b a l a b i l i t y and t e s t - w i s e n e s s , a wider range of values i n the Language P r o f i c i e n c y Index need to be obtained among the Chinese students. A more evenly d i s t r i b u t e d range of values i n v e r b a l a b i l i t y and t e s t -wiseness would enable a more r e l i a b l e p r e d i c t i o n . F i n a l l y , to f i n d out i f te s t - w i s e n e s s a f f e c t s examination s c o r e s , the use of a g r e a t e r v a r i e t y of sample i n v a r i o u s examination areas i s necessary. However, f e a s i b i l i t y of o b t a i n i n g a r e p r e s e n t a t i v e sample i s l a r g e l y c o n t i n g e n t upon the c o o p e r a t i o n of p r i n c i p a l s and t e a c h e r s . 120 REFERENCES A l k e r , H.A., C a r l s o n , J.A., & Hermann, M.G. (1969). M u l t i p l e -c h o i c e q u e s t i o n s and student c h a r a c t e r i s t i c s . Journal of Educational Psychology, 60(3), 231-243. A n a s t a s i , A. & Cordova, F. (1953). Some e f f e c t s of b i l i n g u a l i s m upon the i n t e l l i g e n c e t e s t performance of Puerto Rican c h i l d r e n i n New York C i t y . Journal of Educational Psychology, 44, 1-19. Anderson, J.O., Muir, W., Bateson, D.T., Blackmore, D., & Rogers, W.T. (1990). The impact of pr ovi net al examinations on education in British Columbia: General Report. V i c t o r i a : B.C. M i n i s t r y of Education. A r d i f f , M.B. (1965). The relation of three aspects test-wiseness to intelligence and reading ability in grades three and six. Unpublished master t h e s i s , C o r n e l l U n i v e r s i t y , New York. B a j t e l s m i t , J.W. (1977). Test-wiseness and systematic d e n s i t i z a t i o n programs f o r i n c r e a s i n g a d u l t t e s t - t a k i n g s k i l l s . Journal of Educational Measurement, 74(4), 335-341 . Bangert-Drowns, R.L., K u l i k , J.A., & K u l i k , C C . (1983). H i g h l i g h t s from r e s e a r c h on the e f f e c t s of coaching f o r t e s t s . Educational Leadership, 47(4), 82. Benson, J . (1988). The psychometric and cognitive aspects of test-wiseness: A review of literature. Manuscript submitted f o r p u b l i c a t i o n , U n i v e r s i t y of Maryland. Benson, J . Urman, H., & Hocevar, D. (1986). E f f e c t s of t e s t -wiseness t r a i n i n g and e t h n i c i t y on achievement and t h i r d -and f i f t h - g r a d e s t u d e n t s . Measurement and Evaluation in Counseling and Devel opment , 75(4), 154-162. B o r r e l l o , G.M. & Thompson, B. (1985). C o r r e l a t e s of s e l e c t e d t e s t - w i s e n e s s s k i l l s . Journal of Experimental Education, 53(3), 124-128. B r i t i s h Columbia M i n i s t r y of Education (1989). Administrative handbook. V i c t o r i a : Queen's P r i n t e r s . Calam, J . (1980). C u l t u r e and c r e d e n t i a l s : A note on l a t e n i n e t e e n t h century teacher c e r t i f i c a t i o n i n B r i t i s h Columbia. B.C. Historical News, 74(1), 12-15. C a r b o l , B.C. (1987). B r i t i s h Columbia p r o v i n c i a l and s c h o l a r s h i p examinations: An overview of i s s u e s , Horizon, 25(3), 17-20. 1 2 1 C a r b o l , B.C. (1989). P r o v i n c i a l examinations: Answering some of the q u e s t i o n s . Research Forum, 4 (Winter), 20-23. Chant, S.N.F., L i e r s c h , J.E., & Walrod, R.R. (1960). Report of the Royal Commission on Education. V i c t o r i a , B.C.: Queen's P r i n t e r . Crehan, K.D., Koehler, R.A., & S l a k t e r , M.J. (1974). L o n g i t u d i n a l s t u d i e s of t e s t - w i s e n e s s . Journal of Educational Measurement, 11(2), 209-212. Crocker, L. & A l g i n a , J . (1986). Introduction to classical and modern test theory. New York: H o l t , R i n e h a r t , and Winston. Diamond, J . J . , Ayres, J . , Fishman, R., & Green, P. (1976). Are inner c i t y c h i l d r e n t e s t - w i s e ? Journal of Educational Measurement, 14('\), 39~45. Diamond, J . J . & Evans, W.J. (1972). An i n v e s t i g a t i o n of the c o g n i t i v e c o r r e l a t e s of t e s t - w i s e n e s s . Journal of Educational Measurement, 9(2), 145-150. D o l l y , J.P. & V i c k , D.S. (1986). An attempt to i d e n t i f y p r e d i c t o r s of t e s t - w i s e n e s s . Psychological Reports, 58, 663-672. D o l l y , J.P. & W i l l i a m s , K.S. (1983). Teaching testwiseness. Paper presented at the annual meeting of the Rocky Mountain E d u c a t i o n a l Research A s s o c i a t i o n , Jackson, Wyoming. D o l l y , J.P. & W i l l i a m s , K.S. (1986). Using t e s t - t a k i n g s t r a t e g i e s to maximize m u l t i p l e - c h o i c e t e s t s c o r e s . Educational and Psychological Measurement, 46, 619-625. Dreisbach, M. & Keogh, B.K. (1982). Testwiseness as a f a c t o r in r e a d i n e s s t e s t performance of young Mexican-American C h i l d r e n . Journal of Educational Psychology, 74(2), 224-229. Dubois, P.H. (1964). A test-dominated s o c i e t y : China 1115 B.C. - 1905 A.D.. In A. A n a s t a s i (Ed), Testing problems in perspective (pp.29~36). Washington: American C o u n c i l on Educat i o n . Dunn, T.F. & G o l d s t e i n , L.G. (1959). Test d i f f i c u l t y v a l i d i t y and r e l i a b i l i t y as f u n c t i o n s of s e l e c t e d m u l t i p l e - c h o i c e item c o n s t r u c t i o n p r i n c i p l e s . Educational and Psychological Measurement, 19(2), 171-179. E b e l , R.L. (1965). Measuring educational achievement. Englewood C l i f f s , N . J . : P r e n t i c e - H a l l . 122 E d u c a t i o n a l Measurement Research Group (EMRG) (1989). Preparing to write the language Proficiency Index. U n i v e r s i t y of B r i t i s h Columbia, F a c u l t y of Education, LPI Committee. E e l s , K., David, H., Havighurst, R., H e r r i c k , U., & T y l e r , R. (1951). Intelligence and cultural differences. Chicago: U n i v e r s i t y of Chicago P r e s s . F e r r e l l , G.M. (1972). The r e l a t i o n s h i p of scores on a measure of t e s t - w i s e n e s s to performance on teacher-made o b j e c t i v e achievement examinations and on s t a n d a r d i z e d a b i l i t y and achievement t e s t s t o grade-point average, and to sex f o r each of f i v e high s c h o o l samples. Doctoral Abstracts International, 33, 1510A. F i s h e r , D. & Ec h o l s , F. (1989). Evaluation report on the Vancouver School Board's race relations policy. Vancouver: U n i v e r s i t y of B r i t i s h Columbia. F r i e r s o n , H. T. (1976). E v a l u a t i o n of an i n t e r v e n t i o n program's impact on m i n o r i t y medical students' s t a n d a r d i z e d t e s t performance, Report ED 246097, 1-10. Fueyo, V. (1977). T r a i n i n g t e s t - t a k i n g s k i l l s : A c r i t i c a l a n a l y s i s . Psychol ogy In The Schools, 14(2), 180-185. Gibb, B.G. (1964). Test-wiseness as a secondary cue response. Unpublished d o c t o r a l d i s s e r t a t i o n , S t a n f o r d U n i v e r s i t y . G o s l i n , E., E p s t e i n , R.R., & H a l l o c h , A. (1965). The use of standardized tests in elementary schools. New York: R u s s e l l Sage Foundation. Gross, L . J . (1976). The e f f e c t s of three s e l e c t e d a spects of tes t - w i s e n e s s on the s t a n d a r d i z e d t e s t performance of e i g h t h grade students. Dissertation Abstracts International, 36, 6551A. H e i n r i c h , J . (1983). Why p r o v i n c i a l examinations? B.C. Teacher, 63(2), 59-60. Howell, D. (1982). S t a t i s t i c a l methods for psychology (2nd ed.). Boston: Durxbury P r e s s . K a l e c h s t e i n , P., K a l e c h s t e i n , M., & Docter, R. (1981). The e f f e c t s of i n s t r u c t i o n on t e s t - t a k i n g s k i l l s i n second-grade black c h i l d r e n . Measurement and Evaluation in Guidance, 73(4), 198-202. K r e i t , L.H. (1968). The e f f e c t s of t e s t - t a k i n g s k i l l s i n second-grade black c h i l d r e n . Measurement and Evaluation in Guidance, 73(4), 198-202. 123 Lo, M. & S l a k t e r , M.J.(1973). R i s k - t a k i n g and testwiseness of Chinese students. The Journal of Experimental Education, 42(2), 56-59. McPhail, I.P. (1978). A p s y c h o l i n g u i s t i c approach of t r a i n i n g urban h i g h schools students i n t e s t - t a k i n g s t r a t e g i e s . The Journal of Negro Education, 47(2), 168-176. M e t f e s s a l , N.S. & Sax, G. (1958). Systemic b i a s i n the keying of c o r r e c t responses on c e r t a i n s t a n d a r d i z e d t e s t s . Educational and Psychological Measurement , 18(4), 787-790. Millman, J . (1966). Test-wi seness in taking objective achievement and aptitude examinations: Its nature and importance. Unpublished manuscript, C o r n e l l U n i v e r s i t y . Millman, J . , Bishop, C.H., & E b e l , R. (1965). An a n a l y s i s of te s t - w i s e n e s s . Educational and Psychological Measurement, 25(3), 707-726. Millman, J . & S e t i j a d i (1966). A comparison of the performance of American and Indonesian students on three types of t e s t items. Journal of Educational Research, 59(6), 273-275. Nunnally, J.C. (1967). Psychometric theory. New York: McGraw H i l l . Oakland, T. (1972). The e f f e c t s of tes t - w i s e n e s s m a t e r i a l on st a n d a r d i z e d t e s t performance of pr e s c h o o l disadvantaged c h i l d r e n . Journal of School Psychology, 10(4), 355-360. Pedhazur, E . J . (1982). Multiple regression in behavioral research (2nd ed.). New York: H o l t , Rinehart, and Winston. Putnam, J.H. & Weir, G.M. (1925). Survey of the school system. V i c t o r i a , B.C.: King's P r i n t e r . P r e l l , J . M. & P r e l l , P.A. (1986). Improving t e s t scores -teaching t e s t - w i s e n e s s - A review of the l i t e r a t u r e . Phi Delta Kappa, 5, 2-5. Rogers, W.T. & Bateson, D.J. (1990a). The impact of test-wiseness upon year-end school leaving examinations. Manuscript submitted f o r p u b l i c a t i o n . Rogers, W.T. & Bateson, D.J. (1990b). Verification of a model of test-taking behavior of high school seniors. Manuscript submitted f o r p u b l i c a t i o n . 124 Rowley, G.L. (1974). Which examinees are most favoured by the use of m u l t i p l e c h o i c e t e s t ? Journal of Educational Measurement, 77(1), 15-23. R u s s e l l , I.L. (1969). M o t i v a t i o n of school achievement measurement and v a l i d a t i o n . The Journal of Educational Research, 62(6), 263-266. Samson, G.E. (1985). E f f e c t s of t r a i n i n g i n t e s t t a k i n g s k i l l s on achievement t e s t performance: A q u a n t i t a t i v e s y n t h e s i s . Journal of Educational Research, 78(5), 261-266. Samuda, R.J. (1975). Psychological testing of American minorities - Issues and consequences. New York: Dodd, Mead & Co. Sa r n a c k i , R.E. (1979). An examination of tes t - w i s e n e s s i n the c o g n i t i v e domain. Review of Educational Research, 49(2), 252-279. Scruggs, T.E. & L i f s o n , S.A. (1985). Current conceptions of te s t - w i s e n e s s : Myths and r e a l i t i e s . School Psychology Review, 14(3), 339-350. S l a k t e r , M.J. (1969). G e n e r a l i t y of r i s k t a k i n g on o b j e c t i v e examinations. Educational and Psychological Measurement, 29(1), 115-128. S l a k t e r , M.J., Koehler, R.A., & Hampton, S.H. (1970). Learning t e s t - w i s e n e s s by programmed t e x t s . Journal of Educational Measurement, 7, 247-254. Smith, J.K. (1980). The convergence strategy of test-wiseness. Paper presented at the annual meeting of the American Research A s s o c i a t i o n , Boston, M.A. Smith, J.K. (1982). Converging on c o r r e c t answers: A p e c u l i a r i t y of m u l t i p l e c h o i c e items. Journal of Educational Measurement, 19(3), 211-220. St a n l e y , J.C. (1971). R e l i a b i l i t y . In R.L. Thorndike (Ed.) Educational measurement (pp.363-365). Washington D.C.: American C o u n c i l on Ed u c a t i o n . Superintendent of Education (1898). Annual report of the public schools of the Province of British Columbia. V i c t o r i a , B.C.: Queen's P r i n t e r . Thorndike, R.L. (1951). R e l i a b i l i t y . In E.F. L i n d q u i s t (Ed.), Educational measurement (pp.560-620). Washington D.C.: American C o u n c i l on Ed u c a t i o n . 125 Thorndike, R.L. & Hagen, E. (1969). Measurement and evaluation in psychology and education. New York: John Wiley and Sons Inc. Vernon, P.E. (1962). The determinants of reading comprehension. Educational and Psychological Measurement, 22(2), 269-286. Weiten, W. (1984). V i o l a t i o n of s e l e c t e d item c o n s t r u c t i o n p r i n c i p l e s i n e d u c a t i o n a l measurement. Journal of Experimental Education, 52(3), 174-178. Wu, T.H. & S l a k t e r , M.J. (1978)..Risk-taking and test - w i s e n e s s of Chinese students by grade l e v e l and residen c e a r e a . Journal of Educational Research, 71(3), 167-170. Appendix A Standardized Regression Weights and Adjusted R2— i n Each Step of  D e l e t i o n of the Major Hypothesis Steps R2 Adj R2 Standardized V a r i a b l e s Regression Weights Removed MCE AM TW LPI RES Dependent V a r i a b l e = ALG (N = 35) 0 .099 -.55 .13 -.21 .63 .08 .51 ( A l l in) 1 .095 -.36 .13 -.15 -.57 -.46 LPI 2 .08 -.22 -.10 -.48 -.42 MCE 3 .078 -.11 -.41 -.32 AM 4 .031 -.06 -.18 RES 5 .00 .00 TW Dependent V a r i a b l e = ENG (N = 16) 0 .66 .42 -.28 .05 -.28 .35 -.86 ( A l l in) 1 .66* .49 -.27 -.32 .37 -.90 AM 2 .62* .50 -.34 .34 -.70 TW 3 .52* .43 .26 -.60 MCE 4 .46** .41 -.68 LPI * p<.05 ** /7<.01 ALG=Algebra exam scores ENG=English exam scores TW =Test-wiseness LPI=Language P r o f i c i e n c y Index RES=Residency i n Canada AM =Achievement M o t i v a t i o n MCE=Multiple Choice Experience 127 Appendix B R e s u l t s of Item A n a l y s i s of Subtests ID1. ID2, ID3 and IIB4 i n the Test of Test-wiseness of F o r e i g n Chinese Students Subtests IDi (Absurd Options) ID2 ( S i m i l a r Options) I tern Options P-values PBST PBTT 3 1 4.9 .21 .28 2 29.3 -.42 -.22 *3 53.7 .39 .19 4 12.2 -.15 -.16 7 1 12.2 -.21 -.00 *2 58.5 .57 .31 3 22.0 -.37 -.10 4 2.4 -.17 -.29 Others 4.9 -.15 -.30 9 1 12.2 -.03 -.06 *2 34. 1 .43 .47 3 19.5 -.32 -.12 4 26.8 -.11 -.15 Others 7.3 -.07 -.36 13 1 22.0 .01 .12 2 26.8 .07 .19 *3 29.3 .31 .05 4 17.1 -.37 -.18 Others 4.9 -.15 -.42 17 1 17.1- .09 .09 2 31.7 -.38 -.32 *3 29.3 .56 .49 4 19.5 -.17 -.12 Others 2.4 -.29 -.40 22 1 22.0 -.09 -. 12 *2 26.8 .42 .08 3 24.4 -.14 .01 4 24.4 -.10 .17 Others 2.4 -.29 -.40 4 1 22.0 -.23 -.02 2 26.8 .24 -.07 *3 12.2 .44 .33 *4 36.6 -.23 -.08 Others 2.4 -.26 -.19 6 *1 12.2 -.05 -.10 2 31.7 -.03 .04 *3 29.3 .28 .20 4 19.5 -.09 -.12 Others 7.3 -.22 -.36 12 1 19.5 -.04 -.22 *2 22.0 .25 .12 3 26.8 -.26 -.17 *4 24.4 .20 .48 Others 7.3 -.22 -.36 Subtests Item Options F-values PBST PBTT ID2 15 1 17.1 -.10 .04 ( S i m i l a r *2 31.7 .36 .18 Options) *3 24.4 .34 .21 4 22.0 -.57 -.30 Others 4.9 -.18 -.30 18 1 14.6 -.05 -.10 *2 29.3 .23 -.15 3 17.1 -.47 -.11 *4 36.6 .26 .28 Others 2.4 -.26 -.40 21 *1 22.0 . 1 1 -.04 2 24.4 .01 .32 3 24.4 -.22 -.10 *4 26.8 .19 -.03 Others 2.4 -.26 -.40 ID3 1 *1 39.0 .22 -.12 ( D i f f e r e n t *2 31 .7 .32 .26 Options) 3 29.3 -.57 -.13 4 0.0 .00 .00 5 1 26.8 -.03 -.07 2 22.0 -.26 -.12 *3 17.1 .24 .31 *4 26.8 .21 .13 Others 7.3 -.23 -.36 10 *1 26.8 .12 .21 *2 36.6 .21 .04 3 12.2 -.14 . 1 1 4 17.1 -.04 -.16 Others 7.3 -.29 -.40 16 *1 7.3 .12 .21 *2 31 .7 .21 .04 3 36.6 -.14 . 1 1 4 22.0 -.04 -.16 Others 2.4 -.29 -.40 20 1 12.2 -.23 -.08 *2 36.6 .39 .13 3 31 .7 -.42 -.26 *4 17.1 .33 .40 Others 2.4 -.29 -.40 23 1 17.1 -.44 -.33 *2 51 .2 .39 .48 3 7.3 .05 -.04 *4 22.0 .00 -.10 Others 2.4 -.29 -.40 Subtests Item Options P-values PBST PBTT IIB4 2 1 9.8 -.23 -.19 (Stem-option *2 36.6 .45 .02 Link) 3 24.4 -.24 .01 4 26.8 -.02 .15 Others 2.4 -.26 -.19 8 1 22.0 .07 .10 *2 22.0 .12 .12 3 24.4 -.03 -.01 4 24.4 -.08 .01 Others 7.3 -.11 -.36 11 *1 41 .5 .33 .18 2 17.1 -.14 -.27 3 29.3 -.25 .03 4 9.8 .07 .20 Others 2.4 -.11 -.40 14 1 22.0 -.26 -.10 2 17.1 -.14 -.00 *3 31.7 .54 .13 4 24.4 -.19 . 1 1 Others 4.9 -.05 -.30 19 *1 22.0 .50 .58 2 22.0 -.21 -.06 3 41 .5 -.04 -.18 4 12.2 -.26 -.21 Others 2.4 -.11 -.40 24 *1 22.0 .45 .32 2 7.3 .20 -.07 3 43.9 -.26 -.10 4 24.4 .02 -.01 Others 2.4 -.11 -.40 Subtests M SD Hoyt R e l . C o e f f . SEM ID1 2.32 1 .27 .19 1 .04 ID2 2.98 1 .23 .03 1.11 ID3 3.44 1 .36 .29 1 .05 IIB4 1 .76 1 .09 .00 1 .02 * = c o r r e c t o p t i o n ( s ) PBST = P o i n t - b i s e r i a l C o r r e l a t i o n (Subtest) PBTT = P o i n t - b i s e r i a l C o r r e l a t i o n ( T o t a l t e s t ) 130 Appendix C Items Used f o r V a l i d a t i o n of the Test of Test-wiseness 1. The compromise between Democrats and Republicans a f t e r the p o s t - c i v i l war e l e c t i o n of 1876 r e s u l t s i n ... A. f e d e r a l a i d to southern r a i l r o a d s . B. the entrance of Texas i n t o the Union as a s l a v e s t a t e . C. a t r e a t y with Joseph S t a l i n . D. the f u g i t i v e Slave Law. 2. I f r i b u l o s e biphosphate were removed from a c h l o r o p l a s t , which of the f o l l o w i n g statements would BEST d e s c r i b e the immediate r e s u l t ? A. CO2 c o u l d not enter the C a l v i n c y c l e . B. ATP c o u l d not be produced i n the t h y l a k o i d . C. c o u l d not enter the C a l v i n c y c l e . D. L i g h t energy c o u l d not be trapped i n the grana. 3. In 1911 the T o r i e s i n c o n t r o l of the House of Commons showed t h e i r p o l i t i c a l i n f l u e n c e and democratic s p i r i t when they A. enacted the A s q u i t h p r i c e - c o n t r o l b i l l . B. vetoed a b i l l to extend the s u f f r a g e p r i v i l e g e . C. f o r c e the House of Lords to accept a r e d u c t i o n i n i t s power. D. were def e a t e d i n an attempt to enact the N a t i o n a l Insurance Act. 4. The sigma e f f e c t of the F a h r a e u s - L i n d q v i s t phenomenon i s r e l a t e d t o ... A. the flow of l i q u i d through the kidney. B. the muscular c o n t r a c t i o n s of the kidney. C. the kidney's t r a n s m i s s i o n of f l u i d s . D. the diameter of the red c e l l s f l o w i n g through the kidney's blood v e s s e l s . 5. Which of the f o l l o w i n g would cause an oat s e e d l i n g to bend to the r i g h t ? A. Auxins p l a c e on the l e f t s i d e of the shoot t i p . B. G i b e r e l l i n s p l a c e on the l e f t s i d e of the shoot t i p . C. C y t o k i n i n s p l a c e d on the r i g h t s i d e of the shoot t i p . D. Auxins p l a c e on the r i g h t s i d e of the shoot t i p . 131 6. Why i s C a v a l i e r i ' s P r i n c i p l e important i n S o l i d Geometry? A. I t shows that the s u r f a c e area of a square of s i d e s i s s 2 . B. I t p r o v i d e s c o n t r a d i c t i o n s to the p r i n c i p l e s of E u c l i d and Gauss. C. I t i s used to prove that two polygons are congruent. D. I t p r o v i d e s the b a s i s f o r f i n d i n g the volume formulae f o r many s o l i d s . 7. A major c h a r a c t e r i s t i c of n a t u r a l resources i s that they are ... A. sources of energy. B. n o n - r e c y c l a b l e . C. non-renewable. D. unevenly d i s t r i b u t e d . 8. The emperor of the a n c i e n t Hsin Dynasty who r e s i g n e d to undertake r a d i c a l reform was ... A. Saigon. B. Wang Mang. C. Mao T'se Tung. D. Alexander I. 9. The nose ... A. records l i g h t s e n s a t i o n . B. develops d u r i n g g a s t r u l a t i o n . C. has two moveable j o i n t s . D. i s s t r u c t u r e d i n p a r t by the t u r b i n a l s . E. i s an organ of balance. 10. S t r i d u l a t i o n at times f a c i l i t a t e s ... A. c o o r d i n a t i o n . B. d i s p e r s a l . C. n u t r i t i o n . D. r e p r o d u c t i o n . E. e x c r e t i o n . 11. A f t e r the t r e a t y of R a s t a t t and Baden i n March-September 1714, A u s t r i a took p o s s e s s i o n of the Spanish Netherlands. A. True B. F a l s e . 132 12. The Zmstvo-Law proclaimed by Alexander II of Russia i n 1864 A. f r e e d m i l l i o n s of Russian s e r f s . B. a b o l i s h e d the o l d system of c l a s s c o u r t s . C. allowed j u s t i c e s of the peace to d e a l with minor c i v i l s u i t s . D. i n t r o d u c e d the p r i n c i p l e of u n i v e r s a l m i l i t a r y l i a b i l i t y . 13. Every bounded i n f i n i t e set has at l e a s t one c l u s t e r p o i n t . A. True B. F a l s e 14. " I f the group G has order n, the order of every subgroup H of G i s d i v i s o r of n." T h i s i s the theorem of ... A. Napoleon. B. Bach. C. Cauchy. D. Homer. E. Legrange. Note: Items were p r i n t e d one on a page and were presented to the students one at a time. 133 Appendix D - Student Survey NAME: SCHOOL: BIRTH DATE: / / SEX: M / F Yr /Mon /Day STUDENT SURVEY INSTRUCTIONS : 1. Print your NAME and the NAME OF YOUR SCHOOL at the top of this Test Booklet. 2. F i l l in your BIRTH DATE. 3. Circle the appropriate place marked SEX (M or F). SECTION I: BACKGROUND INFORMATION INSTRUCTIONS: Please read each question and circle the appropriate number below each item or write down your response on the space provided. ****************************************************** 1. To which ethnic or cultural root do you belong? 0 English 1 French 2 Native Indian 3 East Indian 4 Chinese 5 German 6 Italian 7 Japanese 8 Vietnamese 9 Others 2. Were you born in Canada? 0 No (continue on to question #3) 1 Yes (skip to question #6. Do not answer questions 3, 4 & 5) 3. What is your place of origin? (City) (Country) 4. How long have you been in Canada? (Years) (Months) 5. Have you ever practised writing any multiple choice examinations in your home country? 0 No, never 1 Yes, once or twice 2 Yes, three or more times 134 6. Have you ever had any formal coaching or specific lessons on how to take a test? 0 No, never 1 Yes, once or twice 2 Yes, three or more times If yes, where did you receive the coaching or lessons? (Check all that apply) 0 in Canada 1 in my home country other than Canada 2 somewhere else 7. Have you ever received any informal consultation on how to write multiple choice examinations (e.g. from teachers, parents, friends etc)? 0 No, never 1 Yes, once or twice 2 Yes, three or more times If yes, where did you have the consultation? (check all that apply) 0 in Canada 1 in my home country other than Canada 2 somewhere else 8. Have you practised writing provincial examinations using previous examinations or questions from these examinations? 0 No, never 1 Yes, once or twice 2 Yes, three or more times 9. Have you written or are you going to write any provincial scholarship examinations? 0 No 1 Yes SECTION II - ACHIEVEMENT MOTIVATION INSTRUCTIONS: Please read each statement or question and decide whether it Almost Never applies to YOU, Sometimes applies to YOU, Often applies to YOU, or Almost Always applies to YOU, and circle the appropriate response below each item. You are asked to state YOUR ACTUAL FEELINGS OR HABITS with regard to these items, not in accordance with what you think you should or should not do, or what you see others do. Please answer in accordance with what you yourself feel or are in the habit of doing. No one will ever know what you have personally said on this survey, so please be as honest as possible. The Scale of Agreement is: Almost Never, Sometimes, Often, Almost Always ********************************************* 1. Does it bother you if another student makes better grades than you do? Almost Never Sometimes Often Almost Always 2. Does failure discourage you from trying as hard the next time? Almost Never Sometimes Often Almost Always 135 The Scale of Agreement is: Almost Never, Sometimes, Often, Almost Always ********************************************* 3. Do you often compare your work with the work of others? Almost Never Sometimes Often Almost Always 4. Are you usually on time with school assignments? Almost Never Sometimes Often Almost Always 5. Do you try to make better grades than the other students in your class? Almost Never Sometimes Often Almost Always 6. Would you, or do you enjoy being one of the class leaders? Almost Never Sometimes Often Almost Always 7. Do you stick to an assignment until it is completed even though it is dull and boring to you? Almost Never Sometimes Often Almost Always 8. If you lost several times consecutively, would you quit trying? Almost Never Sometimes Often Almost Always 9. Would you prefer to enroll in a course in which no grades were to be given? Almost Never Sometimes Often Almost Always 10. If you had to choose between taking part in a contest or being one of the judges, would you choose to be a judge? Almost Never Sometimes Often Almost Always 11. Would you prefer to sit in the back of the classroom? Almost Never Sometimes Often Almost Always 12. Your friend stopped running when it was evident that he was losing the race. Would you have stopped running in this situation? Almost Never Sometimes Often Almost Always 13. Do you always try hard to get the right answer? Almost Never Sometimes Often Almost Always 14. Do you study hard for the tests you take in school? Almost Never Sometimes Often Almost Always 15. Are your grades important to you? Almost Never Sometimes Often Almost Always 16. Do your test grades in school really show what your know? Almost Never Sometimes Often Almost Always ************** * pigQQg Turn Pase ****** ******** * * * * * * * 136 GENERAL INSTRUCTIONS FOR SECTIONS III AND IV: Read each sentence completely. Circle the number (1,2,3, or 4) following the underlined portion of the sentence that you think contains an error. If you think the sentence is CORRECT as it appears, circle number 5 following the ' C at the end of the item. For example: Maria, Louisa, and (1) Clara wasn't (2) able to be (3) at (4) the party. C (5) Since the portion "wasn't" contains an error, you should circle "(2)" as your answer. SECTION III : SENTENCE STRUCTURE Some, though not all, of the ten items found below contain an error related to sentence structure, such as a misplaced modifier, a run-on sentence, a lack of subject-verb agreement, faulty parallelism, and so on. No item contains more than one underlined error. Now proceed with the following sentences. C=Correct Sentence 1. There were(l> several letters that he knew(2) he had to write, he toldP) himself that he would write them(4) on Wednesday afternoon. C (5) 2. Whenever I go(l) to a fashion show, I like to get an insight mto(2) the latest in(3) women's clothes. Perhaps even buy something^). C (5) 3. I really enjoy(1) sitting in front of(2) a fire blazing away with a special friendffl) on a cold winter's day(4). C (5) 4. The ability to pl&y(l) tennis well demands(2) a strong arm, running and(3j really excellentf4j co-ordination. C (5) 5. Despite the factfl j that it was a holiday w e e k e n d ^ , the line-ups at the ferry t e r m i n a l ^ were not all that long (4). C (5) 6. There is f l ) a real problem concerning(2) the amount of(3) garbage we produce which is increasing^). C (5) 7. Drivingd) in the downtown traffic during(2> the evening rush hour can sometimes(3j be very hard on one's(4) nerves. C (5) 137 C=Correct Sentence 8. The increasedfl) cost of food and clothing v?ere(2) primarily responsible for last month's rise/3) in the cost of living (4). C (5) 9. The noise made by(l ) the large jet planes flying over our part of the city(2) often becomes quite upsettingf3). Especially in the early morning(4). C (5) 10. Whenfl) a famous athlete retiresf.2), the number they(3) had worn as a player is often retired withf4) him or her.C (5) SECTION IV : ENGLISH USAGE INSTRUCTIONS: Some, though not all, of the ten items found below contain errors in standard usage, such as the wrong form of a verb, the wrong form of a noun, an incorrectly used preposition, and so on. Circle the number (1, 2, 3, or 4) following the underlined portion of the sentence that you think contains an error. If you think the sentence is CORRECT as it appears, circle number 5 following the ' C at the end of the item. No item contains more than one underlined error. ************************************************** C=Correct sentence 1. In recentfi) years, windsurfing has becomefi?) one of the most (3) popular summer sportf4) in Canada. C(5) 2. Capital punishments are notfl) justified whenfl?) the accused has not been(3) proven guilty by means of(4) absolute evidence. C(5) 3. The tornado sweeped over(l) the village withfi?) terrible force and/3,) left much destruction behind it(4). C(5) 4. Sonia still has not beenfl) able to get use to(2) the fact that in Canada(3) we drive on the right-hand side(4) of the road. C (5) 5. I believe thatfi) the very best way for/2) an individual to serve the humanityf3) is by becoming^) a teacher, a scientist or a farmer. C(5) 6. Incidents(1) that embarrass us when they happen(2) can sometimes be amused over(3) several years later/4). C (5) 138 C=Correct Sentence 7. Possibly(1) you will think quite different^) about your job after(3) you come back from your annual v a c a t i o n ^ . C (5) 8. Increased(l) crime in many suburban areas has causedft?) hundreds of families^) to install burglar alarm systems(4j. C (5) 9. Her optimistic attitudefl.) is the trait(2) that have always endearedffl her to her many friends^). C (5) 10. He complained to(1) the manager on the factfj?) that the saleperson at the ticket counter(3) had been unnecessarilyf4) rude. C (5) ******************** jg^d ******************** 139 Appendix E - T e s t o f T e s t - w i s e n e s s Test Booklet School: TEST OF TEST-WISENESS This is a test of test-wiseness which measures some of the abilities needed to do well on tests. Many of the questions are about things you may not have studied. However, there are test-taking strategies which can be usedto figure out what to do when faced with such questions. For example: The area test advantage of using slent In the manufacture of steel Is that slent makes steel a. transparent. b. stainless. c. heavy. d. rubbery. Using test-wiseness strategies, options 'a' and'd' can be eliminated since they are clearly not correct (steel is not transparent, nor is it rubbery). Therefore, either 'b' or 'c' is the correct answer. Now we stand a better chance of guessing the correct answer for we have narrowed the number of possible options down to two from four. Please be sure to follow the specific Instructions for each of the following sections. Section 1: Suggested Time: 20 minutes INSTRUCTIONS: For each question, select the B E S T answer and record your choice on the answer sheet provided. Each question is worth one mark. There will be no correction for guessing. 1. Compared to normal cells, bileuvial cells * a . divide more rapidly. * b . divide more slowly. c. have more cytoplasm. d. have more mitochondria. 2. The Flying Spider Is known for Its ability to a. blend in with its surroundings. *b . glide through the air. c. kill its prey with poison. d. make very large webs. 3. The square root of 1.1 can be best approximated by a. the cube root of 11056. b. the solution of x 2 -16 = 0. * c . using the binomial series. d. factoring the expression 9x 2 -18. 4. Mr. Adams, In Henry Fledllng's Joseph Andrews. a. learns his parents were of the nobility. *b. takes sick after fishing through the ice. *c . falls into the mud while reading. d. discovers he is of noble birth. 140 5. In 404 B.C. , the Athenian oligarchs, supported by Lysander and Theramenes, a. approved a plan to rebuild sections of the city. b. encouraged a military expedition against the Spartans. * c. rejected the constitution created by the committee of ten. * d. set up a commission of thirty to write a new constitution. 6. Which of the following would help to determine if D is the fourth harmonic of C with respect to A and B? * a. The relative size of angle ACB to angle ADB b. The length of line segment AB * c. The fact that A, B, C, and D lie on one straight line d. The straight line distance from A to B 7. A normal percentage of polymorphonuclear leukocytes found In the human peripheral blood is a. 53/260. * b. 70%. c. 115%. d. 035. 8. The mercantilists believed in a. freedom from all governmental interference in trade. * b. merchant control of colonies. c. an income tax. d. organized trade unions. 9. Wordsworth's "The Prelude" (1805) a. tells of a descent into Hell in a Model-T Ford. * b. makes use of a distinction between "the sublime" and "the beautiful". c. is concerned with the emerging African nations. d. was influenced by Hemmingway's The Sun A lso Rises. 10. The literature of the early eighteenth century Is * a. public in nature, relating to society's outlooks and values. * b. private in nature, relating to an individual's emotions and feelings. c. rough and irregular compared to the literature of the later eighteenth century. d. filled with despair over the apparent collapse of traditional values. 11. Charles Dickens' Hard Times deals with * a. the difficult life of a factory worker. b. the politics of the French chateau country. c. the court of King Edward III. d. the limitations of European existentialism. 12. Organisms of the Pavo genus a. change from masculine to feminine gender. * b. display their plumage for the female. c. become female after existing for a period of time as male. * d. possess an excess quantity of masculine hormone. 13. Which of the following most likely caused the War of 1693? a. Spain was building roads to connect her cities. b. France was going through great agricultural change. * c. France believed that Spain was increasing her troops. d. Spain had a series of earthquakes. Fount Coyrngh l»»9 EMKO UBC - L » M . , 01:3(11.,...) 141 14. A spherical triangle Is the triangle on the surface of a sphere. What name Is given to the number of degrees in a spherical triangle minus 180? a. The arc of the triangle. b. The size of the triangle. * c. The spherical excess of the triangle. d. The polar measurement of the triangle. 15. In Horace Walpole's The Castle of Otranto a. Manfred is the father of Hippolita. * b. Hippolita is the wife of Manfred. * c. Manfred is the uncle of Hippolita. d. Hippolita is the daughter of Manfred. 16. The ring F[x] / s(x) Is a field If and only If * a. sfx) is a prime polynomial over F. * b. s(xj is a rational polynomial over F. c. F[x] is a multiple of s(x). d. any element of F[x] contains an inverse. 17. The career of Marlus (155-86 B.C.), the opponent of Sulla, is significant in Roman history because a. he gave many outstanding dinners and entertainments for royalty. b. he succeeded in arming the gladiators. * c. he showed that the civil authority could be thrust aside by the military. d. he made it possible for the popular party to conduct party rallies outside the city of Rome. 18. A substance that, in Its pure form, is the best conductor of electricity is a. water. * b. deuterium. c. H O . * d. silver. 19. The august character of the work of Pericles In Athens frequently causes his work to be likened to that In Rome of * a. Augustus. b. Sulla. c. Pompey. d. Claudius. 20. "Lucifer In Starlight" Is a. a modem psychological story of World War II. * b. a Shakespearean sonnet by Gerard Manley Hopkins, c. a controversial French novel written by Resnais. * d. the title of a Petrarchan sonnet by George Meredith. 21. The treaty of Brest Lltovsk was ratified by Moscow because * a. Tsar Alexander I wanted to prevent Napoleon's invasion of Russia. b. Russia was unable to keep up with the armament manufacture of Austria. c. Russia could not keep pace with the military production of Austria. * d. Nicolai Lenin wanted to get the Soviet Union out of World War I. I W a Carrot" HM»0 UBC -L»:M«r 0 M : ( * » f t T . ) 142 22. How many Iambic feet (one Iambic foot - one unstressed syllable followed by one stressed syllable, as In "perFORM") are in each line of Robert Pack's poem "The Compact"? a. 1 * b . 5 . c. 16 d. 22 23. What Is the probability that a needle of length L < D, when dropped on a table ruled with equidistant parallel lines at distance D apart, will cross one of the lines? a - S (LD) 3 * b . 2L_ icD c. L + .001D * d . 2D id. 24. The Feulgen Nucleal Reaction demonstrates the presence of * a . desoxyribonucleoprotein. b. lysosomes. c. mitochondria. d. endoplasmic reticulum. Please Turn Page Note: * C o r r e c t Option . — — Section 2: Suggested Time: 10 minutes INSTRUCTIONS: For each question in this part of the test, select the B E S T answer and record your choice on the answer sheet provided. Each question answered correctly is worth 5 points. Zero points will be given for each omitted question. Two points will be deducted for each question answered incorrectly. 25. The transmitter Vagustoff was Identified a. after studies on choline and its esters. b. by streptococcal infusion of monkey tissue. c. with the aid of the electron microscope. d. by using the Fuchsin reaction. e. after careful dissection of the papillary duct. 26. The Second Servile War In Sicily, which lasted for the four year period 103-99 B.C. , was suppressed by the Roman consul a. Douglas MacArthur. b. Julius Britannicus. c. Dwight Eisenhower. d. M. Acquillius. e. Napoleon. 27. If every neighbourhood of p contains Infinitely many points of S, p is called a (an) a. ratio. b. cluster point. c. accumulation point. d. trapezoid. e. subtraction. 28. John Dunne's Pseudo-Martyr first appeared In a. 1606. b. 1607. c. 1608. d. 1609. e. 1610. 29. The labryrlnthlne righting reaction Is static. a. True b. False 30. The human skin a. possesses a Malpiglian layer. b. is always brown. c. cannot regenerate. d. is uncoloured. e. is composed of cuboidal epithelium. 31. By uniting most of Northern India, which Indian dynasty excluded the Moslems until the end of the tenth century? a. The Maitrakas dynasty. b. The Gurjara-Prathihara dynasty. c. The Kumarapala dynasty. d. The Tughluk dynasty. e. The Khaljii dynasty. 144 32. What Is the value of In sin 8°15' to five decimal p laces? a. .14321 b. .14332 c. .14349 d. .14352 e. .14361 33. Flrdausl 's tenth-century Persian epic, Shah Nameh or (Book of Kings), was translated Into French by a. William Shakespeare. b. Jules Mohl. c. Pope John XXIII. d. Louis Bonnerot. e. Ludwig van Beethoven. 34. Anne Rldler wrote "The Song of the G o o d Samaritan". a. True b. False END of TEST Pra ia C<VJ*I* IVf* 04X0 UBC - L» M . T 01 7 <*.r".| 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0098029/manifest

Comment

Related Items