Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Examinee control of item order effects on latent trait model and classical model test statistics Scales, Michael J. 1990

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1990_A8 S32.pdf [ 6.23MB ]
Metadata
JSON: 831-1.0098329.json
JSON-LD: 831-1.0098329-ld.json
RDF/XML (Pretty): 831-1.0098329-rdf.xml
RDF/JSON: 831-1.0098329-rdf.json
Turtle: 831-1.0098329-turtle.txt
N-Triples: 831-1.0098329-rdf-ntriples.txt
Original Record: 831-1.0098329-source.json
Full Text
831-1.0098329-fulltext.txt
Citation
831-1.0098329.ris

Full Text

EXAMINEE CONTROL OF ITEM ORDER EFFECTS ON LATENT TRAIT MODEL AND  CLASSICAL MODEL TEST S T A T I S T I C S by MICHAEL J . SCALES  B.Ed., U n i v e r s i t y  of V i c t o r i a ,  1978  A THESIS SUBMITTED IN PARTIAL FULFILLMENT THE REQUIREMENTS FOR  OF  THE DEGREE OF  MASTER OF ARTS  in THE FACULTY (Department  OF GRADUATE  STUDIES  of E d u c a t i o n a l Psychology  and S p e c i a l  We  conforming  accept to  this  thesis  the r e q u i r e d standard  THE UNIVERSITY  OF BRITISH COLUMBIA  July (c)  as  Michael  1990 Scales,  1990  Education)  In  presenting  degree freely  at  this  the  thesis  in  partial  fulfilment  University  of  British  Columbia, I agree  available for  copying  of  department publication  this or of  reference  thesis by  this  for  his thesis  and study. scholarly  or for  her  Department  of  The University of British C o l u m b i a Vancouver, Canada  DE-6  (2/88)  the  requirements that the  I further agree  purposes  may  representatives.  financial  permission.  of  gain  It  shall not  be  that  advanced  Library shall make  by  understood be  an  permission for  granted  is  for  allowed  the that  without  it  extensive  head  of  my  copying  or  my  written  ABSTRACT The changes trait  purpose  test  statistics.  questions  were t h e n  a s an a d d i t i o n a l  Four test  analyzed  different  were u s e d  with  formats  hardest.  The o t h e r  easiest.  In a d d i t i o n ,  had  special  i n the t e s t  of a f o r t y  item  sequenced  half  from  of the t e s t s  which prevented  altering  the given  teachers  provided a rating  The  ability  level  mathematics  i n grade e i g h t . easiest  b o o k l e t s were s e q u e n c e d  directions  booklet.  factor.  590 s t u d e n t s  t h e b o o k l e t s had t h e i t e m s  t o answer t h e  using the student's  independent  latent  t o answer t h e q u e s t i o n s i n  who were r e q u i r e d presented  effect  c o m p a r i s o n s were made  who were a l l o w e d  In t h e o r d e r  what  had on c l a s s i c a l and on  As w e l l ,  o r d e r , and s t u d e n t s  results  s t u d y was t o d e t e r m i n e  i n the item order  between s t u d e n t s any  of t h i s  item d i f f i c u l t y  from  of each  students  sequence.  of each s t u d e n t ' s  H a l f of to hardest to sequence from  The c l a s s r o o m ability in  mathematics.  The effect. lower  order  was f o u n d  T e s t s w h i c h were s e q u e n c e d  mean s c o r e .  restrictive not  of the items  directions  a statistically  significant  Although  students  had lower  t o have a s i g n i f i c a n t  from with  hard test  found.  11  had a  booklets  s c o r e s on a v e r a g e ,  significant difference.  interactions  to easy  with  i t was  T h e r e were no  C l a s s i c a l and l a t e n t  trait  item d i f f i c u l t y  statistics  showed a h i g h d e g r e e o f  correlation. It  was c o n c l u d e d t h a t  order  of the items c o u l d  trait  statistics.  be  effect  I t was a l s o  t a k e n when a s s u m p t i o n s  local  under  certain  both c l a s s i c a l recommended  a r e made a b o u t  independence.  iii  circumstances, the  that  and  latent  care should  parallel  forms o r  TABLE OF CONTENTS CHAPTER I.  II.  PAGE INTRODUCTION  1  Context  of the Research Problem  1  Purpose  of the Study  7  REVIEW OF LITERATURE Initial  Studies  Critical Initial  9  Re-examinations Studies: Conclusions  12 16  A n x i e t y and I t e m O r d e r  17  Achievement  28  Other  Simple  Research: Conclusions  Effect  Controlling Test  and Item O r d e r  Interactions  Interaction  44 46  Other F a c t o r s  52 61  T r a i t and C o n t e x t E f f e c t s  Context E f f e c t  and I t e m O r d e r  Summary and C o n c l u s i o n s PROBLEM Statement  32  Re-examined  Wiseness  Latent  III.  9  65 69 71 78  of the Problem  78  Rationale  79  Hypotheses  82  lv  IV.  V.  VI.  METHOD  85  Design  85  Subjects  86  Instrument and Tasks  86  Procedure  88  Analysis  88  RESULTS  90  Main E f f e c t s  91  Interactions  94  Item D i f f i c u l t i e s  96  SUMMARY AND CONCLUSIONS  102  Purpose of The Study  102  Sequence  103  Directions  104  Ability  105  Interactions  105  Latent  106  Trait  Limitations  107  Implications  113  Future Research  114  REFERENCES  116  APPENDIX I:  Teacher I n s t r u c t i o n s  121  APPENDIX I I :  Test  134  APPENDIX I I I :  P - l e v e l Item A n a l y s i s Data and B-value Item Parameter Estimates  Booklet Covers and End Pages  v  143  APPENDIX  IV:  Sample T e s t  Booklet,  AUTHOR INDEX  Format  3  146 1  vi  8  9  List  of Tables  Table  1:  L i s t o f Item O r d e r R e s e a r c h : R e s u l t s , T y p e s , and Samples  73  Table  2:  T e s t Means and Sample S i z e s o f Student A b i l i t y L e v e l s  93  Table  3:  Summary o f A n a l y s i s o f V a r i a n c e of T e s t S c o r e s by A b i l i t y ( A b ) , Item O r d e r ( O r ) , and T e s t Directions (Dir)  95  Table  4:  Summary o f M u l t i v a r i a t e A n a l y s i s of V a r i a n c e o f T e s t Item P - l e v e l s  97  Table  5:  Summary o f M u l t i v a r i a t e A n a l y s i s of V a r i a n c e o f T e s t Item B - v a l u e s  98  Table  6:  T e s t F o r m a t P - l e v e l Means, B - v a l u e Means, and S c o r e Means  99  Table  7:  Pearson C o r r e l a t i o n C o e f f i c i e n t s 100 o f P - l e v e l s and B - v a l u e s f o r E a s y t o Hard ( E H ) , Hard t o E a s y ( H E ) , R e s t r i c t e d ( R ) , and U n r e s t r i c t e d ( U ) , T e s t Formats  Table  8:  Summary o f A n a l y s i s o f V a r i a n c e of T h e t a V a l u e s b y A b i l i t y ( A b ) , I t e m O r d e r ( O r ) , and T e s t Directions (Dir)  vii  101  List  Figure  1:  Low  Ability  Figure  2: H i g h A b i l i t y  of  Figures  Students Students  viii  ACKNOWLEDGEMENTS I would l i k e t o thank the members of the t h e s i s committee f o r t h e i r a s s i s t a n c e . Donald A l l i s o n , and Dr. David much needed a d v i c e and support  Dr. Robert Conry, Dr.  Bateson provided me with under what I am sure were  c o n d i t i o n s of v e r y s h o r t n o t i c e .  I would a l s o l i k e to  thank the many teachers and students their  who provided me with  g r e a t l y appreciated cooperation.  Finally,  I would  l i k e t o bestow my deepest g r a t i t u d e to my w i f e , Margot Lakoduk, my son, Riordan, sacrifices  and my daughter, A l y s h a .  Their  on my behalf have been s i n c e r e l y a p p r e c i a t e d .  ix  Chapter  I  Introduction Context  of the Research  Ever  since  multiple  "new-type e x a m i n a t i o n s " descriptions Antes,  that  the e a s i e s t  choice tests (Ruch,  of computerized  1985),  authors  Problem  1929) t o t h e p r e s e n t  marking  a d v i c e h a s been multiple  were c o n s i d e r e d t h e  systems  forthcoming  choice tests  (Hopkins  f r o m many  &  textbook  s h o u l d be a r r a n g e d  with  questions a t the beginning to the hardest  q u e s t i o n s a t t h e e n d . T h i s a d v i c e h a s had a g r e a t d e a l o f intuitive author  a p p e a l and assumed c e r t a i n t y .  states:  The  level  of d i f f i c u l t y  as a b a s i s placing later,  of o b j e c t i v e  f o r a r r a n g i n g these  the easy  ones f i r s t ,  average  pupil.  testing  time  improved.  many p u p i l s  With  ones  last.  f o r the average kind  I f the d i f f i c u l t of average  of t e s t  t o answer e a s i e r  test  t o answer  test  (1)  i s used by  items  ones  Such an and b e l o w  he u s e s t h e and h i s m o r a l e  items appear  or low achievement  trying  items  in a test  a l l o w e d more e f f i c i e n t l y ,  g r e a t d e a l of time fail  this  items  test  t h e more d i f f i c u l t  and t h e most d i f f i c u l t  a r r a n g e m e n t has a d v a n t a g e s  is  F o r example one  them.  later  will  first,  waste a  T h e y may  i n the t e s t  (2) b e c a u s e so much t i m e M o r e o v e r , t h e y may hostile. are  On  listed  smooth p r o g r e s s  difficult attack  When t h e y  later  encounter  items, Even  i f they  as w i l l  very  will  be  However, d e s p i t e s u c h  this  same l a c k  fail  likely  items  the  by t h e  Items make  feel  more  conviction  resulting that  i n a manner 1963,  p.115)  o f what e x a m i n e e s  r e s e a r c h does not over  the y e a r s  that  to them  knowledge  (Ahmann & G l o c k ,  make a d i f f e r e n c e  even  have t i m e  happen, t h e  I t i s not a c e r t a i n t y  will  test  consequently  t o some i t e m s  Research  or  t o answer some o f  moderated  empirical  of doubt.  inconclusive. test  feel,  ones.  at f i r s t  t h e y no d o u b t w i l l  i s probably correct.  and  will  and  them.  d o u b t do  i f the e a s i e r  same p u p i l s  t h e y a l r e a d y have r e s p o n d e d  no  first  test,  disappointment  that  the  become d i s c o u r a g e d  hand,  these  on  i n the  test  correctly,  spent  quickly  the o t h e r  first,  encouraged.  was  will  support has  been  the arrangement  to the s c o r e of  of  the  examinee. In area  fact,  issue  of  item order  of r e s e a r c h f o r n e a r l y f o r t y  Oorans  (1985) p o i n t e d o u t , t h e  interests  and  the a c c u r a c y precise order  the  statistical of t e s t s  statistics  has  c o n t i n u e s t o be  years.  also  improved,  of the  due  As  has Leary  been  times.  an  and  reflected  t h e need  increased.  a concern  As  r e s e a r c h has  abilities  have  effects  the  So,  while  f o r more  a result,  to c o n f l i c t i n g  item  research  (3) results;  one r e s e a r c h e r w i l l  significant  effect  (Allison,  researcher  will  (Hambleton  & Traub,  and  (1987) r e p o r t e d  in  Newman their  second  first  conclude  first  classical  questions easy  initial  test  finding  the simple  statistics. should  first.  hard  MacNicol,  significant  Kundert,  order  effects  Melton,  Hambleton  Researchers  effects  in their  to easy,  & Meyers,  & Traub,  emotional anxiety,  state  wanted t o t e s t t h e the e a s i e s t  effect  1968; S i r o t n i k  difference  Some  (Mollenkopf,  1960; Sax & Cromack, 1966; & Wellington,  1964; Huck &  1970; K l o s n e r  1970; A l l i s o n ,  & Gellman,  1984).  about the  o£ t h e exam t a k e r s a n d t h e i r  l e v e l of  of the r e s e a r c h s h i f t e d In a d d i t i o n ,  1984).  item order d i d  (Brenner,  1960s t h e r e was a c o n c e r n  states.  such  random, a n d s p i r a l l i n g .  research reported that  so t h e emphasis  internal  o f i t e m o r d e r on  1974; K l e l n k e , 1980; Hodson,  Kestenbaum & Weiner,  the late  i n the e a r l y  o f a r r a n g e m e n t s were t r i e d  Bowers, 1972; Monk & S t a l l i n g s ,  In  began  main e f f e c t  1956; Sax & C a r r ,  make a s i g n i f i c a n t  these  is significant  Lane, B u l l ,  studies reported a s i g n i f i c a n t  However, some l a t e r  1973;  has no  another  be c o n s t r u c t e d w i t h  A variety  t o hard,  Flaugher,  not  In f a c t ,  r e s e a r c h on i t e m o r d e r  axiom t h a t t e s t s  1974;  that the e f f e c t  1974).  item order  1984) whereas  s t u d y and n o n - s i g n i f i c a n t  1950s and e x a m i n e d  1950;  that  study.  The  as  conclude  statistical  t o examine techniques  (4)  using  factor  a n a l y s i s were  i n more common u s a g e ,  r e s e a r c h e r s c o u l d examine t h e anxiety  level  variety  of  and  test  interaction  results.  significant  effects  Towle & M e r r i l l ,  1982)  whereas o t h e r  1975;  research  effects  t o be  1964;  Smouse & Munz 1968;  1969;  Marso,  1970;  1983;  R e c e n t r e s e a r c h has item order,  but  the  The  statistics  results  reported  the  of  this  Smouse & Munz,  t h e main e f f e c t s  & Lowry and  the  &  Berger,  Angelino,  Munz, Smouse, & 1971;  returned  r a t h e r than  Plake,  Melican, 1984;  Plake,  Carter, Kllmko,  to a concern  latent  classical  Greer,  1980;  1984).  about  have begun t o use  trait  test  effects  of  item order  & Dawis, 1976;  Yen,  on  1980;  simple  a more models  statistics.  number o f s t u d i e s t o d a t e  recent research raises  i f item order results  t h e y may  results  a  some  have  Item  Kingston  &  1984).  This one,  (Whitely  research,  n o n - s i g n l f l e a n t (French  Ansorge,  limited  reported  Ansorge, Parker,  analysis involving  significant  parameters Dorans,  of the  found  researchers  modern c o m p u t e r i z e d test  Plake,  Plake,  Plake,  of  some s t u d i e s r e p o r t i n g  Munz & J a c o b s ,  Thompson, & L o w r y , 1980; Shaughnessy,  The  (Munz & Smouse, 1968;  1969;  interaction  effect  in previous  i t e m a r r a n g e m e n t s were u s e d .  r e s e a r c h were a l s o mixed w i t h  of  As  and  has  a significant  of p r e v i o u s  have  some i m p o r t a n t  l a c k e d the  r e s e a r c h may power or  effect, be  Issues.  then  some o f  questionable  statistical  For  since  sophistication  (5)  to  d e t e c t an i t e m o r d e r  conclusions findings. trait  of  Previous  may have t o be r e - e x a m i n e d Of c o u r s e ,  model  findings  effect.  this  discrepancy  under  independent.  trait  I f item order  model  probability  o f g e t t i n g some o t h e r  assumption test  of l o c a l  independence  suitable  for using  parameters.  Hopefully,  can  small violations  tolerate  however, t h e e x t e n t  such  i s effected  effects trait  trait  by t h e then the  Therefore,  would n o t be model  item  m o d e l s a r e r o b u s t and  o£ some b a s i c  and e f f e c t  is locally  that the  is violated.  latent  latent  item  question right,  t h a t d i d show i t e m o r d e r  perfectly  question a basic  has an e f f e c t  o f g e t t i n g one i t e m r i g h t  of t h i s  assumptions;  source  o f e r r o r needs  research.  Research  into  growing concern.  reported  by l a t e n t  source  item order  classical  must  trait  o f t h e mixed  trait  areas  effect  model s t u d i e s be e x a m i n e d , b u t a l s o results  among t h e c l a s s i c a l  I t i s unclear  model s t u d i e s l a c k e d  the latent  investigate several  Not o n l y must t h e i t e m o r d e r  s t u d i e s must be c o n s i d e r e d .  of  between t h e l a t e n t  model t h a t e a c h  probability  the  o f new  study.  premise of the l a t e n t  of  and  may be due t o f u n d a m e n t a l d i f f e r e n c e s i n t h e t y p e s  factors  further  in light  f i n d i n g s and some o f t h e c l a s s i c a l  T h i s r e s e a r c h may a l s o b r i n g i n t o  any  concerns  model  i f perhaps the  t h e power o r t h e s e n s i t i v i t y  s t u d i e s o r i f some o f t h e c l a s s i c a l  (6) models d i d not their  s t u d i e s w h i c h may  obtained. past  properly control  Nevertheless,  research  is called  o n g o i n g measurement One  study,  significant studies  without  allowed then  control  f o r t o shed  test  Traub, as booklet  and  a result,  t o be  developed  f i n d i n g s of  of  their  In t h i s  t h a t may field.  a  previous  of  easy  ones  effect  are  first  of  item  item  orde  Hambleton test  with  a  within-subject results  do  question  research. have a  Traub's study  limits  the  However, t h  restricted the  findings.  Traub's  w i t h i n - s u b j e c t rearrangement  factor  the  a mathematics  previous  However, i f Hambleton and then  an  If subjects  t h a t the  group t h a t d i d not  i n H a m b l e t o n and  generalizability  do  that prevented  validity  format  on  in error since  insignificant.  Their s i g n i f i c a n t  of a c o n t r o l  been  that  t h e d i f f e r e n c e s between  rearrangement.  lack  had  & Traub, 1974).  questions  masked and  of the  and  that did find  suggested  results  Traub reasoned  format  between p r e s e n t further light  of  they  for w i t h i n - s u b j e c t rearrangement  a r r a n g e m e n t s would a p p e a r and  design  results  statistics,  Item o r d e r  (Hambleton  to s k i p hard  would be  i n the  problem.  from  H a m b l e t o n and  order  comparisons  significant  item order  factor  i n f l u e n c e d the  using c l a s s i c a l  effect  t h e y d i d not test  have  a  have been c a u s i n g  f i n d i n g s are c o r r e c t  Is p e r h a p s a random e r r o r the  inconsistent results  In a d d i t i o n , w i t h i n - s u b j e c t  rearrangement  (7) may be s u c h be  a significant  factor  made aware o f i t s p o t e n t i a l  when t h e y a r e t a k i n g a t e s t , of  research  i s a need  o f examinee c o n t r o l group, t o determine  control  of the order  the  latent  trait  both  types  i f item order  of concern  much o f t h e c o n c e r n interaction  effect  over with  studies  included this  Klosner  & Gellman,  results  were  Purpose  study  and e x a m i n e e  statistics  In a d d i t i o n ,  c o u l d be u s e d i n  are duplicated  i s the e f f e c t  that  Surprisingly,  item order  involved  with  of the  & Gellman,  only  four  1984).  The  was  1973).  Study  replicated  the procedures  o f H a m b l e t o n and T r a u b  examined  though  ( S a x & Cromack, 1966;  1973; Hodson, 1984; A l l i s o n ,  (Klosner  order  this  low a c h i e v i n g s t u d e n t s ,  as a f a c t o r  item  even  i n c o n c l u s i v e , but f u r t h e r r e s e a r c h  This study research  test.  but with a  factors.  i f the r e s u l t s  on low a c h i e v i n g s t u d e n t s .  recommended  a  of s t a t i s t i c s .  Another area has  design  Hambleton and T r a u b ' s  are s i g n i f i c a n t  a n a l y s i s t o determine  might use i t  of item order,  and c l a s s i c a l  should  a s e x a m i n e r s must be aware  when t h e y  to replicate  control  both  so t h a t they  just  i t s error causing a b i l i t i e s There  t h a t a l l examinees  the e f f e c t s  (1974).  of the item  In a d d i t i o n , t h i s  of w i t h i n - s u b j e c t  a s d e f i n e d by Hambleton and T r a u b ,  order  rearrangement,  by u s i n g a n a d d i t i o n a l  (8)  g r o u p of s t u d e n t s booklets  with  as  restrictive  Another a s p e c t performance students. believed  t o be  able  s t u d i e d was  to hard  of b e n e f i t  easily  arrangement. be  easy  On  to avoid  the  the  item order.  students  with  performance easy  t o hard  under  of  or h a r d  the  or  hard  by  b o o k l e t s do  not  statistics.  students to  test  are  students  prevent  may  comparable  of  levels  the  under  sequences  and  formats. of the  classical on  changes  t o compare  booklet  on  based  such  ability  a comparison  based  who  easy  levels  done t o d e t e r m i n e are  ability  been g e n e r a l l y  item d i f f i c u l t y  statistics  statistics  test  the  l a r g e sample  different  statistics  latent  based  with  s t u d i e d was  T h i s was  given  using within-subject  used a  unrestricted  aspect  low  has  ability  of  hand, h i g h a b i l i t y  to easy  statistics.  classical  by  This study  item d i f f i c u l t y  trait  low  frustration  of s t u d e n t s  item d i f f i c u l t y  with  to  a wide r a n g e o f a b i l i t y  restricted A final  with  arrangement  other  this  were not  a comparison  students  frustrated  r e a r r a n g e m e n t when t e s t to  g r o u p who  directions.  of h i g h a b i l i t y An  supposedly  a control  latent  results  test  theory  trait  i f studies to s t u d i e s  using using  chapter Review of Initial  r e s e a r c h d i d tend  of a t e s t  examinee. effects  item could  Mollenkopf  of changing  examined the  effect  grade  12  to  11 and  take  one  o f two  was  test  under  modified  hard, had  and  was,  one,  the  to study  the  In a d d i t i o n ,  his  limits.  were d i v i d e d  He  into  Then e a c h g r o u p was  assigned  o f two  each s e c t i o n  tests  arranged  areas  kept  changes.  The  however, a l l o w e d  together. time  the  to  finish  from  The  order  limits the the  test  by  easy  time  to  limits  were 1 hour  other  half  s h o r t time with  a  were limit  different  pencil.  Most o f M o l l e n k o p f  results  changing  o f whole s e c t i o n s d i d not  changes  groups  Item  internally  group with  to complete  382  were r e a r r a n g e d  the s t u d e n t s , while The  used  and  timing conditions. The  the  study  four  verbal  o n l y 35 m i n u t e s .  coloured  of  forms of a combined  more s u b s t a n t i a l  given  who  score  first  of r e d u c i n g time  one  for half  the  item order.  with content  45 m i n u t e s  the view t h a t  i n f l u e n c e the  only s l i g h t l y .  sections with  to support  (1950) was  students  m a t h e m a t i c s exam. their  Literature  Studies  Initial context  II  i n the  the  order  were a s  performance of the (9)  he  students  expected.  on  the  For  cause  any  (10) mathematics t e s t . caused  In a d d i t i o n , d e c r e a s i n g  a deterioration  i n performance.  recommends t h a t t o g e t u s e f u l t e s t should to  the t e s t .  only minimal re-arrangement was a s t a t i s t i c a l l y level the  statistics,  One u n e x p e c t e d  of those  test  items  this  possibly  related  result  change  limits  of the students was t h a t items,  with there  i n the d i f f i c u l t y  (p < . 0 5 ) . Items p l a c e d a t t h e end o f  had a lower  dismissed  half  time  of the v e r b a l t e s t  significant  limits  Mollenkopf  be l o n g enough t o a l l o w a t l e a s t  complete  t h e time  proportion of c o r r e c t responses.  finding  as a s m a l l  insignificant  He  error  t o f a t i g u e t h a t c o u l d be i g n o r e d  by t e s t  developers.  Another report  early  study  by K. M a c N i c o l  authors  (Flaugher,  into  item order  i n 1956.  Melton,  was a n  I t was c i t e d  & Myers,  Hambleton & T r a u b ,  Leary  & D o r a n s , 1985; L a n e , B u l l , K u n d e r t ,  1,500  high  analogies  to Leary  1974; P l a k e ,  and Dorans  school students test.  significantly  (1985),  lower  than  & Newman, 1 9 8 7 ) .  MacNicol  randomly  gave  forms o f a v e r b a l  to easy  the easy  Stallings,  1980; Hodson, 1984;  one o f t h r e e  The mean o f h a r d  by s e v e r a l  1968; Monk &  1970;  According  unpublished  a r r a n g e m e n t was  t o hard  arrangement  whereas t h e random a r r a n g e m e n t was n o t s i g n i f i c a n t l y different the  from t h e easy  30 m i n u t e  time  t o hard  arrangement.  l i m i t on a 50 i t e m t e s t  Unfortunately,  may have been a  (11) factor, finish  p a r t i c u l a r l y s i n c e some s t u d e n t s the t e s t .  Further who  used  Nelson  r e s e a r c h was c o n d u c t e d  325 c o l l e g e  Mental  of  and w i t h  arrangement  type  Test f o r College Students.  i n two f o r m s .  with the d i f f i c u l t y  One  levels  t h e time  40 m i n u t e s .  unaltered  easy,  medium o r  categories intermixed.  spiral-omnibus  A l l students  and C a r r t r i e d  increasing  among  form.  the items  test  This  type  The o t h e r into  their  t y p e s , v o c a b u l a r y , m a t h e m a t i c s , and s p a t i a l  relationships.  Sax  (1962),  The  had t h e t e s t  alternating  the content Is c a l l e d  form  o f a r r a n g e m e n t was t o r e g r o u p  three content  by Sax and C a r r  f r e s h m e n t a k i n g two forms o f t h e Henmon  Ability  was a r r a n g e d  hard,  r e p o r t e d l y d i d not  took  both  t o reduce  limit  from  forms.  the e f f e c t s  of speed  by  t h e recommended 30 m i n u t e s t o  U n f o r t u n a t e l y , many i t e m s were n o t c o m p l e t e d  the s t u d e n t s , so speed  was, u n f o r t u n a t e l y , a  by  confounding  factor.  The of  c o n c l u s i o n found  by Sax and C a r r was t h a t  t h e i t e m s d i d make a d i f f e r e n c e  more a n s w e r s c o r r e c t addition,  students  spiral-omnibus omissions based  occurred  test.  with the spiral-omnibus  omitted  form.  (p. < .001).  fewer  format.  In  number o f  i n the mathematics s e c t i o n that  Students got  i t e m s a t t h e end on t h e  The most s i g n i f i c a n t  They concluded  the order  the presence  of the c o n t e n t of  (12) i n c r e a s i n g l y complex  items tends  r e s p o n d i n g t o t h e more d i f f i c u l t  to discourage students  from  items.  C r i t i c a l Re-examinations It  was a c l a s s r o o m t e a c h e r who wanted t o s o l v e t h e  practical  problem  rearrange  test  forms test he  i n class  wanted  rearranged  pairs  of t e s t s  compared  were  on t h e f i r s t  first  easy  i n random o r d e r .  statistical  or j e o p a r d i z i n g  (1964) u s e d  several o£ t h e  On b o t h  tests  statistics four  he compared  forms,  the s e c u r i t y of h i s  t o compare t h e  t o hard arrangements  As w e l l ,  without  the r e s u l t s  a d m i n i s t e r e d over  t e n items w i t h a hard  t e n items.  the years  and d i f f i c u l t y  of  terms.  against  hard  an e a s y t o h a r d  t o e a s y o r d e r on  the l a s t  thirty  items  U n f o r t u n a t e l y , he d i d n o t r e p o r t t h e  number o f s u b j e c t s i n v o l v e d , the  over  407 m i d t e r m t e s t s  discrimination  easy arrangements.  the  t o have two forms  format  M. H. B r e n n e r  Psychology  Brenner  i t e m bank t o c r e a t e  He wanted  a l l new i t e m s e a c h y e a r  reliability,  randomly  t o p r e v e n t c h e a t i n g i n crowded s i t u a t i o n s , and  h i s items.  order  from a t e s t  t o change h i s t e s t  Educational  to  items  o f t h e same t e s t .  writing of  o f w h e t h e r o r n o t he c o u l d  that  n o r d i d he a d e q u a t e l y  he u s e d  describe  to analyze h i s multiple  comparisons.  Brenner differences  r e p o r t e d o n l y one s i g n i f i c a n t i n twelve  comparisons.  statistical  One p a i r  o f t e s t s had  (13) significantly  different  However, w i t h o u t t-test  used,  results  an a d e q u a t e  this  indicated  college  based  that  The fourth  of the type of Since h i s  of the item order d i d not  i n student performance, not bother  he  recommended  arranging test  items  difficulties.  generalizability  year  indexes.  be a c h a n c e e v e n t .  changing  instructors  on i t e m  description  could merely  make a d i f f e r e n c e that  (p < .05) d i s c r i m i n a t i o n  of t h i s  study  education students taking  a very motivated seems u n l i k e l y  and s o p h i s t i c a t e d  that  such  i s limited  a required  group  s t u d e n t s would  since  course are  of s t u d e n t s . become  It  discouraged  by a n y a r r a n g e m e n t .  Other  researchers also  made a d i f f e r e n c e ,  especially  (Sax a n d C a r r , 1962) f o u n d item arrangement. students w r i t i n g students  a spiral  arrangement.  These  commonly  view  arrangements  G. Sax whose  a significant  In h i s e a r l i e r  who had t h e more  held  wanted t o f i n d  forms:  easy  that  than  traditional  increasing  difficulty  were  i n contrast  recommended  hard  easy  t o the  to hard  1963).  Sax a n d Cromack  to hard,  from t h e  d i d better  Henmon-Nelson T e s t s o f M e n t a l four  effect  study  test  (Ahmann and G l o c k ,  As a r e s u l t ,  first  r e s e a r c h he had f o u n d  format  findings that  out i f item order  (1966) r e a r r a n g e d t h e  Ability  into  t o easy,  the following  spiral,  and random.  (14) The  four  college Half  forms were t h e n a d m i n i s t e r e d t o 467 f i r s t s t u d e n t s who  were a l l o w e d  t h e s t u d e n t s had a g e n e r o u s  one o f two t i m e 48 m i n u t e s  30 m i n u t e s .  In a d d i t i o n ,  o f a l l s t u d e n t s were used analysis ability  of the r e s u l t s  more t i m e  then  (p.  that  better  than  time  difference. point  time  t o hard  different.  limits,  limit  f o r m was t o easy  then  averages  the r e s u l t s  and low g r a d e  differences  f o r time  q u e s t i o n s on a h a r d  interaction  Despite achieving  was  their  imposed,  significantly  finding  was  i f t h e r e were  t h e a r r a n g e m e n t d i d n o t make a of students with  point  averages  to easy  significant  that  s t u d e n t s do i n f a c t  when  and were  format  test.  better.  a t t h e .05  under c e r t a i n  high  t h e r e were  or order except  high a c h i e v i n g s t u d e n t s performed  unusual  given  arrangement  s t u d e n t s were g i v e g r e a t e r amounts o f t i m e answering  students  In a d d i t i o n ,  On t h e o t h e r hand,  In comparing  no s i g n i f i c a n t  case,  high  The o t h e r two a r r a n g e m e n t s were n o t  significantly  grade  points  i n the  into  that  on t h e t e s t s .  t h e mean o f t h e h a r d  < .001).  found  i f a restrictive  t h e mean o f t h e e a s y  longer  factor  the group  Sax and Cromack  performed  found  higher  grade  and l o w a b i l i t y .  Predictably,  study  to divide  i s 18 more  had t h e  cumulative  as a c o v a r i a t e  limits.  which  t h a n t h e manual s u g g e s t s w h i l e t h e o t h e r h a l f suggested  year  In t h i s  This level.  circumstances  have more d i f f i c u l t y  low  with  one  (15)  order  than  gained  another,  in arranging  However, w h i l e that  one  higher  to  item order  actually  performed  positive  and  involved  used  1968,  performed  adverse C.  E.  5,000 c o l l e g e  Large  sequence  the  the  to  easy  that is  although  low  high achieving unusual  format  Item o r d e r  as a r e s u l t  Of  hard  course,  have  this  and  effects both  study  i s not  students  within a very limited,  scale  Examination  effects. E.  B.'s  of t h e  but  very  involved small  developers  needed  items  Scholastic  the easy  content  test  Flaugher,  a p p l i c a n t s with  five similar  item order  Board  s m a l l banks o f  arrangements v a r i e d of  study,  discouraged,  most s t u d i e s on  samples.  rearrange  blocks  the  in attitude  In t h i s  effects.  evidence  achievement.  Entrance  the  i s not  on  is  generous."  o f a wide p o p u l a t i o n s i n c e t h e c o l l e g e  Until  without  a difference  effects.  "...little  achieving students,  b e t t e r as a r e s u l t .  negative  range of  could  low  d i f f e r e n t l y , and  probably  classroom  l i m i t s are  have been c h a l l e n g e d by  people  indicative  help  seem t o be  may  affect  i f time  true that there  indicate  students  College  be  Cromack c o n c l u d e d  by h i g h a c h i e v i n g s t u d e n t s  a c h i e v e r s d i d not  high,  items  i t may  a r r a n g e m e n t may  may  and  arrangement w i l l  scores  related  Sax  such  Melton,  and  Aptitude Test  to hard  content  items,  based  blocks.  forms.  arrangement and  the  t o know i f t h e y  on d i f f e r e n t  4 different  as  varied  Students  tests  Myers to  (1968)  test The  within the had  30  (16) minutes to  to complete  complete  the  Although changes  40 v e r b a l t y p e  25 m a t h e m a t i c s  this  verbal  items  (p <.001).  involve very  found  speeded c o n d i t i o n , item order  t h a t under  their  They d i d not  find  t h a t s i n c e some o f t h e  differing  they  felt,  s m a l l e r than  Nonetheless, tests  relatively  were o m i t t e d  this  was  the  tests  factor  involved item  an  would  error  easy  by some  numbers o f u n a n s w e r e d q u e s t i o n s  Item o r d e r , factor  and  but  error  have t o be  with  They verbal  students,  were a  factor,  standard  on  a difference  concluded  last  somewhat  d i d make a d i f f e r e n c e  arrangements of mathematics q u e s t i o n s .  occurred  minutes  significant  differing  items  30  items.  s t u d y d i d not  i n item order, they  q u e s t i o n s and  factor. an  of  error  measurement.  considered i f  rearrangement.  I n i t i a l Studies; conclusions To the  summarize  late  the  1960s, t i m e  findings limits  impact.  Item s t a t i s t i c s  effects  as more q u e s t i o n s  effect  of t i m e  w h i c h were n o t chance  level  limits reached  e a r l y r e s e a r c h up  were shown t o have a  became more p r o n e t o are  omitted  by  i n some s t u d i e s was or o m i t t e d  of being c o r r e c t .  questions  t h a t were c o m p l e t e d  beginning  o f one  test  of the  t o be  the  definite  item  order  students.  that  One  questions  were g i v e n o n l y a  T h i s would c a u s e by a l l s t u d e n t s  to  at  random  easy the  r e p o r t e d as more d i f f i c u l t  if  (17)  placed  at  students or  the  end  taking  frustration  at  the  at  end  becomes, t h e  Anxiety  the are  easy questions hard  and  One stress 152  different  the  anxiety  researchers of  the by  first  F r e n c h and  versions  of  order.  the  to  for  be  test  develop.  the  The  General  r e s i s t a n c e on  variables.  Test  item  order  of  study  and  involved four  I n t e l l i g e n c e in  f o r m s were e i t h e r e a s y within one  the  Anxiety  for Children.  a polygraph  students  were g i v e n  took  P.T.I. B e h a v i o r a l  of the  students  four  their  students.  The  e a s y and  also  the  (1964).  Pictorial The  o f two  Scale  Anxiety  began t o t u r n  to consider  easy to hard  the  skin  seeming  more s p e e d e d  stress level  Greer  the  the  on  test  responses of and  grade s t u d e n t s .  Mental M a t u r i t y ,  rated  one  sixties  studies  P.T.I., students  Test  of  t o examine t h e s e  within subtests,  the  by  In a d d i t i o n , f a t i g u e  uncertainty  late  internal  random, or a s p i r a l to  reached  f a c t o r s t h a t might account  beginning  i n the  a counter-rotated hard  be  Item O r d e r  t o more  was  first  some o t h e r the  not  to easy t e s t .  more e r r o r and  Concerns over prompted  hard  would  o f a n o t h e r . The  Researchers attention  where t h e y  Rating  the  whole  hard.  The  Test  of  for Children,  and  c h i l d r e n were a l s o  Scale  recorder.  test,  In a d d i t i o n  California Scale  to  and  measured  for  (18) Despite  a  l a r g e amount o f a s s e s s m e n t  l a r g e amount o f d a t a French that  and  Greer  indicated  any  the  P . T . I , was be  r e p o r t e d t o be  The  of the  order.  used  with  sensitive  authors  a similar  to d i f f e r e n t  Unfortunately,  their  concluded  item  reported  scores  students  frustration  of  not  low a b i l i t y  i n c l u d e low  students find  may  tests  about  ability  not  stressful  or  anxious  study, about  enough, nor  addition French  their  students,  and  were t h e any  to  findings.  from  but  their  the  one, 100  the  to  first  125.  grade  school experience A l s o , the  item  possibly  most  to easy.  The  authors  to  did  have been s u f f i c i e n t l y  on  the  of  the  test  arousing  galvanometer.  the g e n e r a l i z a b i l i t y  one,  would  sample d i d  c o u l d have r e p o r t e d more For  the  i n v o l v e s the  pictures  reaction  Improving  Greer  ranged  i n v o l v e the  not  of  they  For  In a d d i t i o n ,  sufficient  sample may  enough t o r e g i s t e r  In  item order  arrangement of hard  mention t h a t t h i s  If  limits  study.  frustrating.  arrangements g i v e n d i d not frustrating  Greer  students.  have had  that  of students  and  concern  a result  were  arrangements.  of the F r e n c h  Most o f t h e  data  There  group of s t u d e n t s ,  sample  of the  as  a  first  lower.  readings  applicability I.Q.  was  not that  were t h e  o r d e r , the  four exposures  The  only results  significant  i n c r e a s e s i n galvanometer  changing  not  reported.  t h a t r e g a r d l e s s of the  performance out not  was  techniques,  t h e y d i d not  of  their  details  provide  the  (19) details  of t h e d i f f i c u l t y  Secondly,  level  of the  items  used.  the  results  of t h e a n x i e t y m e a s u r e s were  r e p o r t e d , and  finally  t h e r e was  determine  not a  factorial  i f s t u d e n t s w i t h h i g h a n x i e t y had  particular  order  that  were d i f f e r e n t  than  not  analysis  to  r e a c t i o n s to  students  with  any low  anxiety. Researchers anxiety F  test  i n the  not  late  statistics  interaction  o n l y began t o t a k e 1960s, b u t  As  high a n x i e t y with hard not  overlooked  113  college  psychology  by  final  t h e end  Smouse and  exam.  level  o r random  Adjective  a result,  to easy  either  (R).  Check L i s t ,  of the  The  test.  easy  on  The  to prevent  the M u l t i p l e  three d i f f e r e n t  directions neutral,  f o r the  test  non-arousing  Smouse and test  widespread  test  results  Munz  the  a  arranged to  easy  Affect  item groups  directions  were  groups.  information concerning  other group  their  just received  directions.  (1968) f o u n d  among any  was  included at  cheating along with  while  test  hard  of  study,  forms of  (E-H),  f o r a n x i e t y , was  different  sequence  t h e exam were  t o hard  group r e c e i v e d a n x i e t y p r o v o k i n g  steps  in  items  the  effect  In t h e i r  of t h r e e  the  and  interaction  item d i f f i c u l t y  In a d d i t i o n ,  r a n d o m l y a s s i g n e d t o two One  the  in  began t o use  Munz ( 1 9 6 8 ) .  a test  interest  t h e main e f f e c t s  f r e s h m e n were g i v e n one  by d i f f i c u l t y (H-E),  they a l s o  to look at both  effects.  an  of the  no  significant  groups.  The  differences  item  (20) a r r a n g e m e n t d i d n o t make a d i f f e r e n c e , group d i d not  make a d i f f e r e n c e ,  between t h o s e  f a c t o r s were n o t  with t h e i r could  be  highly anxious.  result  caused  and  of the The  concluded  t e s t s which can  effected  be  by  120  study,  item  i n the  the students  A.A.T. s c o r e s  While  The  with the d i f f i c u l t y  ( R ) , as  statistical  their  c o l l e g e freshmen.  three t e s t s  (E-H), or  into  results  of a  the  final  individual  exam.  r e a c t i o n s to  sequence. (1968) used  psychology students levels  the  final  had  exam t o  to take  However  one  (H-E),  arranged  f o u r groups of a n x i e t y l e v e l s  in  this their  for  analysis. Munz and  Smouse  (1968) d i d n o t  interaction  between a n x i e t y l e v e l  (p  One  on t h e  group  t h e r e was  labelled  and  arrangement.  best  on  Another group, the  random b u t  worst  on  a  show a  fairly  form  significant  complex  of the  "non-affecteds"  random a r r a n g e m e n t b u t  easy  the  that  differences that  were d i v i d e d a c c o r d i n g t o  f o r item order,  < .01).  of  were d i s a p p o i n t e d  previous study.  main e f f e c t  lowest  be  f o l l o w - u p s t u d y Munz & Smouse  A c h i e v e m e n t A n x i e t y T e s t and  of  authors  results  o n l y showed  situation  t h a t t h e r e may  compare  test  The  by a n x i e t y were p o s s i b l y masked by  They also- reasoned  a  of a n x i e t y  interactions  t h a t any  already h i g h l y anxiety producing  In  the  type  significant.  t h e a n x i e t y measure a t t h e end e v e r y o n e was  and  the  h i g h e s t on  test  performed the  hard  to  "high-affecteds", did  the easy  to hard.  The  (21) " d e b i l l t a t o r s " d i d p o o r l y on a l l forms w h i l e "facilitators" The  hard  did well  to easy  v a r i a n c e w i t h no  Munz and  form  was  increased,  some p e o p l e  poorly.  arousal that  their  types  tended  types,  s e r v e s as The  t o be  person the  i s an  the  be  Munz r e p l i c a t e d  their  the  by c e r t a i n  with a caution that  s e q u e n c e may  to easy  essentially  findings,  other  but  hard  to  the  had  to  minimize  Smouse  and  tempered  even though the  hard  to  their easy  may  not examined  by  1969).  f o l l o w - u p s t u d y by Munz and replicated  of  However,  the  t o use  of  different  arrangement  that  t a k i n g contaminants  s t u d i e s (Smouse & Munz,  will  anxiety  e l i m i n a t e some unwanted v a r i a n c e , i t  other t e s t  Another  by  significant.  format  are  level  levels  In a f o l l o w - u p s t u d y , s t u d y and  others  provided  different  hard  best  an  optimum  by d e c r e a s e s  not  groups.  l n v e r t e d - U graph  formats  i n performance out  on  while  peak o f an  was  form.  between t h e  well  v a r i a n c e , so t h e y recommended  conclusion  to easy  If arousal levels  there  different  that  personality variables.  their  respond  cancelled  s e q u e n c e would  introduce  differences  as a r e s u l t ,  Increases  Smouse n o t e d  least  easy  each  so t h e main e f f e c t  Munz and the  For  will  o f a r o u s a l and,  performance.  hard  t o have t h e s m a l l e s t  of a r o u s a l t h e o r y .  performance.  levels  the  Smouse's c o n c l u s i o n s were b a s e d  level  of  found  significant  inverted-U  respond  on a l l b u t  the  procedures  and  Jacobs the  (1971)  results  also of  the  (22) s t u d y by Munz and was  t o t r y and  by s u b j e c t i v e •p  1  level  address  the  difficulty items  asked  issue  on  .62.  results  The  with t h i s  reflected  correlation  result,  the  'p'  item d i f f i c u l t y  which  o f 133  of  an  of sequencing  the  be students  inter-observer and  They  to rate  college  was  of the group  on  agreement  r e p o r t e d as of  They  items  felt  was  their  that  the  more complex  than  was  (1971) d i d n o t r e p o r t  the  index.  difficulty  their  d i d use  the  'p'  in obtaining  This  ratings  findings  necessary i n examining  sequence.  to  demonstrated  In a d d i t i o n ,  Increased e f f o r t  d o e s n o t seem t o be  would  significantly.  Jacobs  from s t u d i e s  item  typical  difficulty  9 instructors  between t h e s u b j e c t i v e  index r a t i n g s .  the  n o t be a c t u a l l y  The  i t e m was  by t h e t y p i c a l  Munz and  group  procedure  of the t e s t  Oddly,  to d i f f e r  may  difficulty  new  than  then arrangements  high r e l a t i o n s h i p ,  average  of each  as the r e s e a r c h e r i n t e n d e d .  exam.  as not d i f f e r i n g  difficulty  1  level  to the second  psychology f i n a l  reported  •p  'p'  procedure  an a t t e m p t  of each q u e s t i o n which  showed a m o d e r a t e l y =  rather  i f the s u b j e c t i v e  p s y c h o l o g y s t u d e n t s and level  level  T h i s was  among i n d i v i d u a l s ,  administered  r  method.  that  change o f  the d i f f i c u l t y  f o r the s u b j e c t  142  One  judgement p r o c e d u r e s  based  difficulty  their  (1968).  determine  statistical  item d i f f e r s  the  Smouse  and  d i d not  index.  subjective  As  a  ratings  the e f f e c t s  is particularly  seem  of  the case  1£  (23)  there  is a strong correlation  subjective further  rating.  Of  r e s e a r c h on  difficulty  The  levels  interaction  with achievement  t o be  present  that  of  1969).  the  Berger  previous  the  three d i f f e r e n t  the  four d i f f e r e n t  Henmon-Nelson T e s t students  had  two  forms o f  2 levels  received  instructions  that  on  their  a  permanent  Not  o n l y d i d Berger but  found  t o be  significant  followed  by  One  e t a l . not  giving  from  "debilltators"  last.  had  identified used high  In t h i s  group of  the school  study  students  the  scores  find  any  changing test  would  o t h e r s were  for research  told  purposes.  interaction item order  and  instructions  were  However, a n x i e t y t y p e  "Facilitators"  Berger  to  c o n d i t i o n s to h o p e f u l l y  different  " n o n - a f f e c t e d s " then  found  They  and  330  exam.  o n l y used  non-significant.  factor.  with  record while  t h e main e f f e c t s from  final  not  similar  Munz.  t h a t the mental a b i l i t y  s c o r e s were t o be  t h e main e f f e c t also  Abilities  of a n x i e t y .  was  However, t h e y  instruction  the  effects,  format  item d i f f i c u l t y  of Mental  test  a  (1968)  Munz, Smouse, &  Smouse and  anxiety types.  generate  used  (Berger,  e t a l . used  s t u d i e s by  the  concluded,  Smouse  a n x i e t y types  tests  r a t h e r than a c o l l e g e  they a l s o  be  and  Jacobs  and  i s needed.  t h a t Munz and  tests  index  of s u b j e c t i v e  other v a r i a b l e s  with a b i l i t y  'p'  as Munz and  relationship  effect  found  Angelino,  course,  the  with  between t h e  scored  was  highest  "high-affecteds" with  et a l . concluded  t h a t the  item  (24) difficulty they  felt  sequence does not differential  this  stability is  that  variation taking  in  not  high  of  or  the  student  this  providing details  could  mask a n y  order  s i n c e those  of  an  of  with  first  possible  possible  the  as  the  intelligence  Such  year  to  item arrangements  difficulties  ranging  complete  questions  of  scores  the  since  that  information  might  l a c k of  most  in  variance the  noticeable  ability.  two  His  H-E, 100%.  Quick  remiss  and  of c h a n g i n g  c o l l e g e students  to  are  a result  lowest  f r o m 0%  school  A  supposedly  (E-H,  test  sample.  item sequence.  122  and  l a c k of v a r i a t i o n  (1970) r e p o r t e d  involved  greater  authors  were made t o r e p l i c a t e  Marso  a  for a l l high  unusual  their  explanation  has  sample e s p e c i a l l y  changes are  attempts  Munz.  139  the  changes caused  e x a m i n e d a n x i e t y and  three  true  does  et a l . for  taking a b i l i t y ,  s a m p l e , and  was  scores  students  Smouse and  be  Berger  population  reportedly gathered.  intelligence  Further  test  s t a t u s of the  i n d i c a t e i f there  among t h e  i s the  T h i s may  for just  i n f o r m a t i o n was also  one,  Another  However,  tests.  by  there  tests.  taking anxiety  were g i v e n  intelligence,  socioeconomic  aptitude  tests.  school  motivation.  students  For  of a p t i t u d e  the  on  explanations  l a c k of e f f e c t .  ability  r e a c t i o n to test  have a s i g n i f i c a n t e f f e c t  Several  effect  the  studies  studies  first  The  R)  assigned  with  students  Word T e s t .  that  study  randomly  and  of  item had  to  In a d d i t i o n ,  (25) each s t u d e n t completed prior  to taking  be g r o u p e d groups.  into  a series  t h e Q.W.T. s o t h a t  how  a n x i e t y measures  their  test  results  h i g h , a v e r a g e , o r low a n x i e t y  T h e r e were no t i m e  determine  of t e s t  limits,  test  but a r e c o r d  l o n g each s t u d e n t took  taking  was  t o complete  could  kept t o  h i s or her  test.  Marso's a n a l y s i s effect and  from  anxiety  level  level  level.  used  was  Also,  neither  significant  impact  t o complete  on t h e l e n g t h  their  (p < . 0 1 ) .  of item order  item order nor a n x i e t y  test.  f o u n d t o be a s i g n i f i c a n t  performance lowest  f o u n d no  i t e m o r d e r n o r f r o m an i n t e r a c t i o n  had a s i g n i f i c a n t  students  of time  However,  factor  anxiety  level  anxiety  In t h e l e v e l o f  The most a n x i o u s s t u d e n t s had t h e  was  s t u d y found s i m i l a r  results.  f o u n d t o be a s i g n i f i c a n t  of performance  156 c o l l e g e  (p < . 0 1 ) .  students writing  The s e c o n d  their  h i g h , a v e r a g e , and low a n x i e t y  of a n x i e t y t e s t s . different  i n the  study  involved  psychology f i n a l  levels  T h i s e x p e r i m e n t was,  i n the arrangement.  in class  was  used  based  item  on a  series  quite  difficulties  the t o p i c s  f o r the b a s i s  exam.  for analysis  however,  The a c t u a l  were n o t u s e d , b u t t h e o r d e r i n w h i c h presented  Only the  factor  As i n t h e p r e v i o u s s t u d y , s t u d e n t s were g r o u p e d into  that  scores.  Marso's second  level  of v a r i a n c e  of the  were  (26) arrangement.  \ One t e s t  that  their  test  was p r e s e n t e d  final  content  had t h e i t e m s  was p r e s e n t e d  order, similar first  content.  experiment  subjects  levels  s t u d y c o u l d have tests  of c l a s s  with  In  t o easy  involved high  levels  college  by d i f f i c u l t y  (1968),  levels.  educational psychology t o take  Test.  arranged  i n t h e t h r e e common d i f f i c u l t y  and  The s t u d e n t s were a l s o  of mathematics a b i l i t i e s .  anxiety  82 and community  the F l o r i d a  Mathematics Achievement of d i f f i c u l t i e s ,  used  courses  wide r a n g e  range  to r e p l i c a t e the  comparing d i f f e r e n t  reported  R).  In p a r t i c u l a r ,  i s t h e Towle and M e r r i l l  Towle a n d M e r r i l l  mathematics courses  Twelfth-Grade  H i s second  of anxiety, but the  w h i c h a l s o was a n a t t e m p t  from  typical  s e q u e n c e was n o t t e s t e d .  and i t e m o r d e r s .  volunteers  in difficulty  enough t o e x p e r i e n c e  o f a n x i e t y and f r u s t r a t i o n .  work o f Munz a n d Smouse levels  tests  Word T e s t may n o t have had  c o n t r a s t t o Marso's s t u d y  (1975) s t u d y ,  were t h a t  p r e s e n t a t i o n , or i n groups of  the Quick  were n o t a r r a n g e d  the hard  t h e two t e s t s  T h e s e c o n c l u s i o n s a r e made e v e n t h o u g h h i s  who were m o t i v a t e d  classroom  of p r e s e n t a t i o n , and a  l i m i t s do n o t have t o be a r r a n g e d  or i n order  another  i n random o r d e r .  Marso's c o n c l u s i o n s from time  i n the order  by t h e t e a c h e r ,  i n a reverse order  f o r m was a r r a n g e d  without  presented  Statewide  The t e s t  and t h e items patterns  had a  were  (E-H, H-E,  r e p o r t e d t o have a wide In a d d i t i o n  t o the  (27) mathematics Anxiety the  test,  Test  and  mathematics  test. and  unlike the  the  the  students  and  item  on  the  order exam.  a  every  increase  in anxiety  This  different increase theory  s u c h an  on  norms  norms.  As  different sample  Smouse  They  fact  that  the  anxiety i n the  an  level  time  limits  have t i m e result  was  related  the  orders  other.  While  felt  i n d i c a t e the  the  effect level  rather  than  could  may  post the  to  lnverted-U  supported  Merrill  the  to  seemed the  effect  to  not  not  of  placed  however  individual  study.  significant  S.T.A.I,  anxiety  group  order  the  would that  the  on  Towle and  i n each study  a result,  used  was  However,  .05).  significant  recorded  interactions,  that  d i d not  (1968) was  felt  the  slight  none o f  tendency that  the  sample  as  so  interaction,  masked by  the  more t h a t any  effect.  that  Another  increase  orders,  o f Munz and  showed a  (p <  students  level  anxiety  anxiety  significant data  problem.  item  significant  to  mathematics  factor.  find  of  the  significant  a significant  concluded  the  Achievement  (S.T.A.I.) p r i o r  did  result  Many o f  consider  test.  was  Merrill was  no  s t u d i e s , they  item d i f f i c u l t i e s  of  Anxiety  the  S.T.A.I, a f t e r  found  were n o t  previous  Towle and  the  Merrill  levels  completed  State-Trait  test  Towle and  anxiety  the  with  that  the  presence have  a  of  been  groups are  based  standardized be  In e a c h s t u d y  placed  in a  depending  on  the  (28) In summary, r e s e a r c h on a n x i e t y was For  one,  not  the  p r o v i n g t o be  1968;  found  of the  a significant  five  studies, including  find  an  interaction. of o r d e r  involved  a test  with  Achievement and An  theorized  time  area  levels  of  and  easy  arrangement  Cromack  the  (1966),  three  as  students  other  The  other  Munz, f a i l e d a  to  significant  studies, i t  research  and  or below  Students  forms of the  the e f f e c t  average  (1973) r a n k e d  m i d t e r m marks a s  54  The  1963).  factor.  reported to easy  Sax  that  test i f  time.  graduate  either  were r a n d o m l y a s s i g n e d exam.  efficiently  of t h i s  d i d b e t t e r on a h a r d  final  been  (Ahmann & G l o c k ,  p r e v i o u s l y reviewed,  Gellman  i s with  I t has  g i v e n a g e n e r o u s amount o f  b a s i s of t h e i r  achievers.  1971).  most c o n f i d e n t l y and  and  on  with  interaction  s t u d i e s have c o n s i d e r e d  Klosner  (Munz & Smouse,  Smouse and  with average  perform  t o hard  t h e y were a l s o  three  limits.  Few  high a b i l i t y  Further, only  item sequencing.  that students  a c h i e v e m e n t would w i t h an  the  however, d i d f i n d  but a s  research.  norms l i m i t e d  interaction  with  of  to  Item Order  important  achievement  two  its relation  line  Munz & J a c o b s ,  One,  alone,  and  fruitful  findings.  Smouse & Munz, 1969;  effect  a  l a c k of a n x i e t y l e v e l  generalizability studies  item order  students  h i g h or  to take  item format  was  one  low of  arranged  (29)  using test  item d i f f i c u l t y was  an  easy  in a typical  to hard  groupings.  A n o t h e r was  order.  third  The  arrangement.  order  classroom  within similar  random o r d e r  format  was  T h e r e were no  manner.  the  One  content  within similar  more common e a s y item  content  to  time  limits  or  Gellman d i d not  show a  significant  hard  difficulties  reported.  Klosner Interaction authors (p <  and  or main e f f e c t  felt  .15)  students content  t h a t the  and  small  seemed t o do  best  on  groups arrangement. proceed  sample s i z e . be  graduate  some o f  the  considered  One  was  low  However,  almost  easy  significant  The  low  to hard  Their suggestion  study  warranted lacked  sample  with  since there power due  of  this  study not  was  to  is the  high  probably  do  demonstrate  behaviour  patterns  of s t u d e n t s  characteristic  of  low  Although  achievers  normally  d i d not  examine s p e c i f i c a l l y v a r i o u s  levels,  t h e y d i d examine a c h i e v e m e n t m o t i v a t i o n . examined  a n x i e t y and  the  is  their  K e s t e n b a u m and  (1970),  test  the  that further  who  achievement m o t i v a t i o n .  they  achieving  achievers.  typical  addition,  the  I n a d d i t i o n , a more g e n e r a l i z a b l e s t u d y  students  typical  the  i s indeed  done s i n c e t h e  ability  low  interaction  to b e l i e v e that t h e i r  should  item order.  t h e r e f o r e showed a t r e n d .  research should reason  of  relationship  Weiner  achievement  between  In item  a c h i e v e m e n t m o t i v a t i o n . T h e y used  79  order,  (30) seventh test  and  eighth graders  in either  who  random o r e a s y  s t u d e n t s were a d m i n i s t e r e d C h i l d r e n and There  but  significant  t h e r e was  how  motivated  they d i d conclude anxiety  tend  have r e s u l t e d  levels  and  that  in this  Also,  a hard  of  correlation scores. i n the  h i g h l y motivated  different  between They d i d study,  students  study  t o see  the  i f i t had to  however, d i d use  to easy  sequence.  interactions i n c l u d e d the  He  compared  reported  t o be  students. different  Nonetheless,  exam r e s u l t s . arranged hard  to  high a b i l i t y  ability The  levels  who  based  on  t h e exam were motivated  were g r o u p e d  their  easy  possibly  competitive  highly  Into  three  previous O-level  were g i v e n one  formats:  may  were t a k i n g t h e  took  s t u d e n t s and  the s t u d e n t s  students  i n the t y p i c a l easy.  who  t o t h e h i g h l y a c a d e m i c and  o f t h e exams, t h e s t u d e n t s  that  157  exam i n c h e m i s t r y .  s c h o o l system's A - l e v e l  It  v a r i o u s achievement  British  due  low  easy.  t o 19  Unfortunately,  not  but  with  s t u d e n t s b e t w e e n t h e a g e s o f 16  nature  the  at endeavours d e s p i t e f a i l u r e .  sequence of hard  (1984),  as a r e s u l t  performance  interesting  frustrating  Hodson  effects  t h e s t u d e n t s were  to persist  would have been  most  sequence.  the T e s t A n x i e t y S c a l e f o r  a significant  m o t i v a t i o n , a n x i e t y , and report  to hard  reading  the C h i l d r e n ' s Achievement M o t i v a t i o n S c a l e .  were no  order,  were a d m i n i s t e r e d a  of t h r e e  to hard,  tests  random, or  (31)  The the  s t u d e n t s who took  lowest  addition, test  average  the hard  to easy  mean, 26.6 on t h e f i f t y  most o f t h e s t u d e n t s who  w i t h i n the time  limits  failed  had t a k e n  a r r a n g e m e n t had item t e s t .  In  t o complete  the  the hard  Item o r d e r  had an e f f e c t  difference  between t h e means o f a l l t h e t e s t s  The  mean o f t h e e a s y  s i n c e t h e r e was a  t o hard  t h e mean o f t h e random f o r m a t was a l s o but  test  significant  was 29.6.  main e f f e c t  interaction  also  b u t no s i g n i f i c a n t  interactions  were  Surprisingly, inflated value,  hard  slightly  not  i s also  receiving 26.6  finish.  t o support  mean s c o r e .  effects  from a  scores.  In an e a s y t o  show more t h a n a significant  The t h r e e t e s t s  From a n a c a d e m i c s t u d e n t ' s difference  a s c o r e o f 31.5 on t h e t e s t  on a c r u c i a l  are c l e a r l y  p o i n t of view between  a s compared  and p o s s i b l y n o t h a v i n g  Such a d i f f e r e n c e  slightly  the p r a c t i c e of  T h e y show a  a very s i g n i f i c a n t  on t h e t e s t  main  was  have some m o t i v a t i o n a l  However, h i s f i n d i n g s  between t e s t  equivalent.  there  "Apart  multiple choice chemistry questions  inflated  difference  Sex o f t h e s t u d e n t  Hodson c o n c l u d e d ,  t h e r e was no e v i d e n c e  sequence."  with the  found.  mean s c o r e , w h i c h m i g h t  presenting  level  (p < . 0 1 ) ,  effect  levels.  or  (p < . 0 1 ) .  The a b i l i t y  i t e m o r d e r and t h e a b i l i t y examined as a f a c t o r ,  test.  was t h e h i g h e s t , 31.5, and  r e p o r t e d as a s i g n i f i c a n t  t h e r e were no s i g n i f i c a n t  t o easy  with a  enough t i m e t o exam c o u l d have  very  (32)  significant the  effects  suggestions  spend very  to easy  f u t u r e as w e l l .  b y Hodson, c h e m i s t r y  the e f f o r t least  on a s t u d e n t ' s  t o sequence  avoids  items  the t h e o r e t i c a l  teachers  Despite  should  i n fact  i n a way w h i c h a t t h e discouragement  of the hard  sequence.  Other I n t e r a c t i o n s  The  interactions  of item order  unusual  factors  Plake.  Plake  factors  were e x a m i n e d , t h e p r i m a r y  studies  was t h e e f f e c t  of  were a l s o o f p a r t i c u l a r  was i n v o l v e d i n f i v e  that order,  with  with  area  other  interest  studies.  of item order  the student's  various  t o B. S.  Although  of i n t e r e s t  and s t u d e n t ' s  other  i n three knowledge  p e r f o r m a n c e and  perceptions.  Plake's nurses  first  taking three  hard,, s p i r a l , information strategies no  such  expected  o r random.  t o deal with  questionnaire  ( P l a k e , 1980) used  forms o f t h e i r  i n the t e s t  extra  perceptions  study  Half  of the nurses  the order.  about  A l l students  score.  order,  their  were  given  t h e o r d e r and  The o t h e r  a t t h e end o f t h e t e s t  of the t e s t  psychiatric  m i d t e r m exam, e a s y t o  instructions  Information.  104  half  received  had t o c o m p l e t e a  describing their p e r f o r m a n c e , and t h e i r  U n f o r t u n a t e l y , an adequate d e s c r i p t i o n of  his  measure o f s t u d e n t  are  some q u e s t i o n s  p e r c e p t i o n was n o t p r o v i d e d ,  about t h e v a l i d i t y  of t h i s  so there  measure.  (33)  In t h i s item order effected  first  the  perceptions  of the  examination  and  students' test.  the  generalizations  She  type  performance  offset  the  rise  the  Anxiety Test  Merrill before  was  next  first  and  test.  the  the  of the  about the system  order  receiving  instead.  who the  provide  for a control  scoring  or o r d e r .  The  in earlier  factor  this  group to not  and  student. of  that  research  Achievement  Item  TRAIT  different  i n the  receive  procedure  Anxiety  order,  were t h e  same  as  scoring Knowledge  study  with  information  i n f o r m a t i o n about  Unfortunately,  have  knowledge  or e l i m i n a t i o n .  d i d not  may  to drop  STATE and  I n a d d i t i o n , two  students  did  ( P l a k e , Thompson, &  perceptions  also a  of  factor.  anxious  the  post-tests.  student  s c o r i n g method was  half  the  type  the  order  They used the  exam and  reported  interacting  and  that  However, she  less  study  methods were u s e d , number r i g h t of  limited  t o hard  the  their  t h a t the  study.  measured as  (1975). the  or  h i g h l y anxious  item order  p r e - t e s t s and  knowledge o f o r d e r Plake's  the  of the  Anxiety  i n v e n t o r i e s as  in  used  have been an  of  with  focus  1980).  Towle and  scores  of s t u d e n t  find  significantly  d i d admit  i n p e r f o r m a n c e by  Anxiety along  by  test  t h a t knowledge o f e a s y  the  Lowry,  item order  p o s s i b l e from t h i s  caused  was  (1980) d i d not  Plake  t h a t a n x i e t y may  proposed  order  Plake  o r knowledge of t h e  either  theorize  study,  the does  scoring not  r e c e i v e i n f o r m a t i o n about  s u b j e c t s were 97  educational  (34)  psychology students  who  volunteered  to take  the  C o l l e g e Mathematics Placement Program t e s t . r e c e i v e d course  credit  Thompson e t a l . were q u i t e d i f f i c u l t  of order  be  with using The  the  authors  the knowledge of order non-significant.  The  t h a t was  For knowledge  may  i n t e r a c t i o n between a n x i e t y  main e f f e c t  almost s i g n i f i c a n t  (p <  other  findings  did  was  not  interact  with  to produce a  .10).  The  h i n t s at a trend which  t h a t some i t e m a r r a n g e m e n t s  and  be  of order  However, o r d e r  may  was  t h a t such a r e s u l t  post-test anxiety scores  a n x i e t y whereas o t h e r s authors  Plake,  interpret.  a l r e a d y shown t o  The  post-test anxiety scores indicate  to  i m p l i c a t i o n s of the  significant.  p r e - t e s t and  result  admitted  was  were a b i t more c l e a r .  the  r e s u l t s of  number r i g h t m a r k i n g s y s t e m  m e a n i n g f u l s i n c e the  f o u n d t o be  students  i n t e r a c t i o n of a n x i e t y c o n d i t i o n w i t h  significant. not  The  for volunteering.  W i t h s o many f a c t o r s , some o f t h e  example, the  A.C.T.  increased  lower  might pre-test  have c a u s e d a d e c r e a s e .  The  state:  Trends i n the data  do  possible effects.  Therefore,  results  of t h i s s t u d y  knowledge t h a t the been due  support  should  the  p r e s e n c e o f some  conclusions be  t e m p e r e d by  l a c k of s i g n i f i c a n t  i n part to i n s u f f i c i e n t  b a s e d on  the  the  e f f e c t may  power a n d / o r  have  (35) motivation  i n the  Lowry, 1980, It her  i s the  third  p.  research design.  on  with  the  more m o t i v a t e d  graduate  students  course.  The  previous  study  common given  the  was  arranged  Plake,  questionnaires studies  (E-H,  anxiety  the  Mathematics A n x i e t y  students  Rating  to motivate  that  of the  determine  results  mathematics c l a s s e s . the  tests  done t o examine t h e p e r f o r m a n c e and anxiety, of  An  were g r o u p e d  sex  arrangement.  S), half  and  by  manner as  the  the  were  students  3 were  same  perception  were a l s o g i v e n was  students,  on  a  only they  would  two  Revised one  scoring  were be  for extra  for factor  i n f l u e n c e of sex  and  statistics  a d d i t i o n a l change was  subject,  senior  In a d d i t i o n , t h e  would q u a l i f y  sex  a larger  d i f f e r e n c e s between t h e  the  t o examine t h e  of the  R,  mathematics t e s t  which s t u d e n t s  170  order  Lowry, t h e r e  Scale, there  s y s t e m u s e d , and the  used  get  introductory  order.  The  To  in a similar  tests,  were u s e d .  were t h a t  knowledge o f  Thompson, and  i n f o r m a t i o n about the test,  the  authors  e n r o l l e d i n an  item arrangements  mathematics  of  that characterizes  & Lowry, 1982).  sample,  study by  motivation  effects  (Plake, Ansorge, Parker, and  &  218)  concern  study  ( P l a k e , Thompson,  told  used  to  remedial  t h a t a l l of  analysis.  This  mathematics  test  interactive  effects  i t e m a r r a n g e m e n t , and  was  of knowledge  (36)  Even though the s t u d e n t s study  had  75  though the  minutes t o complete a  volunteers  Thompson, and time,  these  items.  two  i n the  Lowry e a s i l y  test.  previous  speeded well  As  The  there are  a planned  effect  established.  As  test  d i d not  even  Plake,  i n the  allotted  complete a l l  f a i l e d to f i n i s h  time  left  blank  power t e s t  of t i m e  previous  significant  by  and  the  t o some p r o b l e m s w i t h d i r e c t i o n s ,  a result,  test.  study  students  p e r c e p t i o n q u e s t i o n s were a l s o  students.  Ansorge, et a l .  item t e s t ,  the  students  of t h e  Due  Plake,  48  finished  h i g h l y motivated  Twenty p e r c e n t  mathematics  In t h e  limits  on  many  became a order  r e s e a r c h has  limits,  by  the  has  been  shown, when  item order  can  have  an  effect.  So, found  an  with  time  limits  item order  effect.  main e f f e c t  from  significant  interaction  males out (p did  <  best  while spiral  on  both  on  t h e r e was  t h e r e was  effect  females  (p <  the  easy  to hard  order  m a l e s and  females  performed  be  no  unusual Not  authors  showed  but  only did  and  combined  t h a t males  actually  the  order  random  e q u a l l y w e l l on  suggested  that  needed u s i n g n o n - m a t h e m a t i c s  limits.  significant  a l l mathematics t e s t s  analysis  The  an  .007).  factor  arrangement.  time  While  but  r e s e a r c h would longer  item order,  perform  .002),  i n v o l v e d , Plake, Ansorge et a l .  the  further tests  and  (37)  Item o r d e r self-reported  was  also a significant  Although  significant,  t h e random f o r m  One  fact  will  about  order  of s t u d e n t s limits  unusual  do n o t f i n i s h ,  and  item order  findings with was  1984).  then  effects  of Plake,  further  Both t e s t s  a psychology  final  write tests  each s e c t i o n  having  interaction  was  will who  Despite time  power  test, i f  the assumptions might  apply.  Ansorge e t a l . t h a t  e x a m i n e d i n two  used  exam.  item  follow-up  questions  the item order not found  college Plake,  1983;  students  Melican  Plake  who  were  e t a l . had  composed o f t h r e e s e c t i o n s w i t h t h a t would  t h e f o l l o w i n g t h r e e ways: e a s y While  test  t h e s e x o f t h e s u b j e c t on a  167 s t u d e n t s  random.  limits.  ( P l a k e , M e l i c a n , C a r t e r , & Shaughnessy,  & Ansorge,  of  be t i m e  and t h e r e f o r e u s i n g a s s u m p t i o n s a b o u t  mathematics t e s t  writing  t h a t a power  Sometimes, i t i s t h e s t u d e n t s  i f there w i l l  interacted  studies  the Plake,  what one m i g h t c o n s i d e r t o be g e n e r o u s  time The  t o have t h e  and t h e h i g h e s t  i s t o n o t assume  be a power t e s t .  providing  a number  rating  l e s s o n t h a t m i g h t be l e a r n e d f r o m  determine  limits,  showed a t e n d e n c y  were  rating.  Ansorge e t a l . study in  test  none o f t h e p a i r w i s e c o m p a r i s o n s  perceived d i f f i c u l t y  performance  i n the  p e r c e i v e d p e r f o r m a n c e and p e r c e i v e d  difficulty.  lowest  factor  be a r r a n g e d  to hard,  spiral,  i n one  or  and s e x o f t h e s u b j e c t  t o be s i g n i f i c a n t  a t t h e .05  (38) level, There  i t would have been s i g n i f i c a n t was a s i g n i f i c a n t  the s u b j e c t w i t h than  279 f e m a l e  Both  any such  and 73 male  of these  mathematics t e s t ,  non-quantitative difficult  tests.  hand, P l a k e & with  students.  that  the f i n d i n g s of  involved higher  I t was n o t e d  t o make s i n c e t h e t e s t s  equally d i f f i c u l t .  and  A n s o r g e had a d i f f i c u l t y  .32  scores  s c o r e s by m a l e s  were n o t a p p l i c a b l e t o  not  used  higher  the sex of  sex of the s u b j e c t e f f e c t  s t u d i e s concluded  P l a k e , Ansorge e t a l . , which on a  receiving  (p < . 0 5 ) . On t h e o t h e r  Ansorge d i d not f i n d  level.  i n connection with  t h e 128 f e m a l e s  t h e 39 m a l e s  their  effect  a t t h e .10  that  comparisons  were  i n a l l t h e s t u d i e s were  F o r example, rating  the t e s t  used  by P l a k e  o f .67, b u t t h e t e s t s  by P l a k e , M e l i c a n e t a l . had d i f f i c u l t y  ratings  between  a n d .48. Another  comparisons, either  a r e a which causes  w h i c h was n o t m e n t i o n e d  study,  Unfortunately, certain  their  courses.  reasons, females  who t a k e these  samples  i n making  by t h e a u t h o r s o f  was t h e non-random n a t u r e  m a l e s and f e m a l e s females  difficulty  were b a s e d  of t h e i r  on s t u d e n t s  who  took  I t d o e s n o t seem a p p r o p r i a t e t o e q u a t e who t a k e a s t a t i s t i c s a psychology  class.  class  course  might  w i t h males and  For a v a r i e t y of  two s a m p l e s may be d i f f e r e n t .  i n either  samples.  The m a l e s o r  n o t be t h e same a s t h e m a l e s  (39)  and  females  i n the  total  population.  g e n d e r d i f f e r e n c e s must be Another students Klimko  and  order  t a k i n g an  (1984).  anxiety level  item  level  was  study  and  new  assigned  or  s e t of  only  found  factor  and  achievement  with  of  specific  cognitive  (E-H,  H-E,  entry  skills  were  was  forty-five  T h e r e were  formats  were  l e a r n i n g of a  using a  who  .0001).  level was  93  randomly  c o n t a i n i n g the  three  R).  examination  relationship  Item o r d e r ,  were n o t  that  item  Unfortunately,  he  sex  significant order  performance.  a meaningful  He  provided  entry characteristics.  of c o g n i t i v e e n t r y c h a r a c t e r i s t i c s  to  the  of  the  influence  a l s o concluded  little  i n the  parameters  of  His  forty-five  item  no  that  achievement  the  had  was  factors.  d o e s not  p r e d i c t o r of  information d e t a i l i n g  by  entry  knowledge,  Cognitive  Klimko.  a significant  (p <  anxiety  e n t r y was  performance.  was  that cognitive entry c h a r a c t e r i s t i c s  H i s main c o n c l u s i o n s  cognitive  by  of  course  male c o l l e g e s t u d e n t s  performance score subject,  male  Cognitive  t o the  tasks.  of the  designed  common i t e m a r r a n g e m e n t s  the  essential  to midterm examination  Klimko  caution.  examination  level.  p r e r e q u i s i t e types  beginning  w h i c h was 18  psychology  cognitive entry  task  measured a t t h e  f e m a l e and  extreme  A d d i t i o n a l f a c t o r s t h a t were c o n s i d e r e d  competencies which are  item t e s t  with  about  i n v o l v i n g f e m a l e and  educational  d e f i n e d as  particular  taken  Conclusions  validity  way  test  data  (40) other  than  i t s results  were s i m i l a r  to a psychology  midterm  examination.  Another Bull,  factor  Kundert,  based  on c o g n i t i v e  and Newman  (1987).  T h e y used  taxonomy t o s u b j e c t i v e l y d e t e r m i n e difficulty"  of every t e s t  course examination.  The t e s t  different  formats.  cognitive  difficulty  statistical  item  Two  decreasing  statistical  studies.  college  formats.  significant  factor  highest  Although  were u s e d  statistical  items  first  was t h a t  Students  f o l l o w e d by q u e s t i o n s  and c o g n i t i v e  difficulty  The l o w e s t mean s c o r e s were  and i n c r e a s i n g  finding  i n two  different  (p < . 0 5 ) .  by s t u d e n t s who had exams w i t h d e c r e a s i n g  unusual  had t h e  s c o r e s , some s u b s c o r e s d i d  by t h e i t e m o r d e r  knowledge  mean s c o r e s .  difficulty  format  i t e m o r d e r was n o t a  In t h e t o t a l  seem t o be e f f e c t e d  increasing  i n c r e a s i n g or  A fifth  These t e s t s  with  s t u d y by Lane e t a l . , 59 male and 96  examination  of  increasing  forms had t h e i t e m s  s t u d e n t s w r o t e one o f t h e f i v e  who had t h e e a s y  in five  or d e c r e a s i n g  and e i t h e r  difficulty.  different  female  increasing  difficulty  i n random o r d e r .  the f i r s t  i t e m s were a r r a n g e d  Two o t h e r  questions  In  item education  forms had t h e i t e m s w i t h  difficulty. cognitive  Bloom's  the "cognitive  in a forty  and e i t h e r  decreasing  t h e o r y was by L a n e ,  had t h e received  cognitive  statistical  difficulty.  g e n d e r was a  significant  Another  (41) interactive  and main  factor  with  females  scoring  higher  than  males. In  the second  original  five  cognitive  form w i t h  cognitive  the t e s t s  had l a b e l s  difficulty.  education  results  item order  total  score  or  absence  increasing with  difficulty,  and t h e  In a d d i t i o n ,  half  item to i n d i c a t e i t s  The s i x forms were r a n d o m l y g i v e n t o college  s t u d e n t s a s an exam i n an  of the second  study d i f f e r e d  had no s i g n i f i c a n t  had h i g h e r  of l a b e l s  m a l e s and f e m a l e s  was a l s o  m a l e s and f e m a l e s  the f i r s t  with e i t h e r the  However, a s i n t h e f i r s t males.  a significant  The  presence  factor.  t h e r e was a n i n t e r a c t i o n  were  Although  Both were  effect  a l l students d i d  i n c l u d e d , t h e d i s c r e p a n c y between  decreased  Lane e t a l . c o n c l u d e d  when t h e l a b e l s  that  the presence  was b e n e f i c i a l  and c o u l d p o s s i b l y  ordering  on i t e m d i f f i c u l t y .  based  from  had h i g h e r s c o r e s when t h e l a b e l s  In a d d i t i o n ,  when l a b e l s  effect  s c o r e s than  between g e n d e r and l a b e l l i n g . better  the form  of d i f f i c u l t y .  or w i t h t h e s u b s c o r e s .  females  present.  t h e form w i t h  difficulty,  on e a c h  o n l y t h r e e of h i s  course.  since  study,  They used  and s t a t i s t i c a l  male and 169 f e m a l e  The  Lane e t a l . used  t h e random o r d e r  cognitive 78  formats.  and s t a t i s t i c a l  decreasing  of  study,  negate  were i n c l u d e d .  of l a b e l l i n g  the e f f e c t  However, t h e  of item  (42) generalizability  of t h i s  that  o n l y s u b j e c t s who  took  the t e s t .  research since  was  this  Ansorge  understood  needed  i n the area  studied  of item order (1980),  He had 484  an e a s y  just and  t o hard  under  An a d d i t i o n a l  left other  arrangement  s i d e of h a l f  compared these test.  and  on  was  o n l y t e n minutes,  thirty  spiral  of study  students  were  w h i c h q u e s t i o n t h e y had on e a c h s t u d e n t ' s  and drew t h e  by K l e i n k e was  of item responses  This response  line.  response  were p l a c e d  on t h e  s i d e of the  positioning  t o t h e h a n d e d n e s s and s e x o f t h e s t u d e n t to the student's  speed  Unfortunately,  t h e b o o k l e t s and on t h e r i g h t  related  a  presented in  or a u n i f o r m  followed the d i r e c t i o n s  area  i n c l u d e d as a  complete  v e r y speeded c o n d i t i o n s .  of the b o o k l e t s .  factors  females  were g i v e n t w e n t y m i n u t e s t o  to indicate  The p o s i t i o n  half  further  of Plake  was a l s o  The t e s t  This provided data  314 s t u d e n t s  location.  of females  but speed  but a f t e r  t o draw a l i n e  accuracy  fact  and s e x o f t h e s u b j e c t were  sciences test.  the t e s t ,  finished.  only  that  o f p e r f o r m a n c e by  f o u r t h grade s t u d e n t s  a r r a n g e m e n t . The s t u d e n t s complete  by t h e  tests.  by K l e i n k e  item s o c i a l  asked  suggested  r e s e a r c h c o n t r a d i c t e d the r e s u l t s  effects  factor.  be l i m i t e d  Bloom's c o g n i t i v e l e v e l s  (1984) on t h e p e r f o r m a n c e  The  either  may  Lane e t a l . a l s o  non-quantitative  six  study  performance  was  t o see on t h e  how  (43)  Kleinke had (p  a higher < .01).  found  t h a t the easy  mean under  the speeded  ample t i m e  matter  higher  minutes, also  t o complete a t e s t ,  and more q u e s t i o n s  had h i g h e r  interactions  It  scores after after  i s unfortunate  random s a m p l e .  that  some s t u d e n t s  will  after  questions  i n comparison with  items  difficult  test.  the l e s s ,  i f they  begin  He  handed  effects  and  t o draw a  Conclusions 314 a r e a  i t c a n be  the t e s t  some s t u d e n t s  correct with whose  However, t h i s  t h e i s s u e o f what e f f e c t s  questions  Other  that the remaining  of v a r y i n g d i f f i c u l t y .  address  ten minutes.  that l e f t  g e t more q u e s t i o n s  of a t e s t  not  scores after ten  t o be s i g n i f i c a n t .  None  beginning  no  F o r one, m a l e s  t h a t 170 s u b j e c t s f a i l e d  by t h e f a c t  than  persist  t e n m i n u t e s and more  a t the t e n minute p o i n t of t h e i r  less  will  n o t a b l e t o draw  t e n minutes.  were n o t f o u n d  must be l i m i t e d  with  complete  under t h e  i f an e x a m i n e e  findings.  mean t o t a l s c o r e s , h i g h e r  q u e s t i o n s complete  line  they  He was  on some o f h i s o t h e r  that  had no c o n c l u s i o n f o r h i s f i n d i n g  pupils  arrangement  t e n minute c o n d i t i o n  He c o n c l u d e d  what t h e a r r a n g e m e n t .  conclusions had  item  However, t h e t o t a l s c o r e s were e q u a l  twenty minute c o n d i t i o n . had  t o hard  concluded at the  easy test  study d i d  would a s e r i e s  have on a s t u d e n t ' s  total  begins  score.  of  (44)  I n t e r a c t i o n R e s e a r c h ; Conclusions The r e s e a r c h other For  into  the i n t e r a c t i o n  of item order  measures d i d not produce c l e a r l y  example  studies involving  success  (Munz & Smouse, 1968;  effects  were  Munz, 1968; Merrill,  not confirmed Berger  a n x i e t y showed  some  results. initial  Smouse & Munz, 1 9 6 9 ) , b u t t h e  by o t h e r  e t a l . , 1969;  researchers  Marso,  1970;  (Smouse  Towle &  a n x i e t y has been one o f t h e more common  included  i n item order  also  been s t u d i e d  most  have had r e s u l t s  research, other  i n conjunction with of l i m i t e d  factors  item order.  significance  and  limited  F o r one, knowledge  line  t h a t d i d not produce any s i g n i f i c a n t  of i n q u i r y  et  1980;  Plake,  a l . , 1982).  K l i m k o (1984)  characteristics order.  He  along  found  o f a r r a n g e m e n t was  Thompson e t a l . , 1980;  with  sex, t e s t  a n x i e t y , and  cognitive entry characteristics performance.  (1987) had v e r y mixed r e s u l t s  when t h e y  statistical  item d i f f i c u l t y  i t e m a r r a n g e m e n t and g e n d e r . the  test  showed  items  items  effect. were  results  Ansorge  along  Lane e t a l .  included cognitive  with  In one s t u d y ,  However,  labelled  with  item t o be t h e  knowledge  of t h e  the o r d e r i n g of  b a s e d on c o g n i t i v e and s t a t i s t i c a l  a significant  when t h e t e s t  Plake,  one  included cognitive entry  o n l y p r e d i c t o r of examination  and  have  However  applicability.  (Plake,  &  1975).  Although factors  significant  with  in their  methods  other  their cognitive  study,  (45)  level,  there  were no  were s i g n i f i c a n t labels  and  The other  the  sex  interaction subject. effect,  did  of the  item  would  indicate  Kleinke,  1984).  factors  tendency toward (1973) s t u d y , found On  order find  sex  an  of  interaction  main e f f e c t  1980;  the  (Plake,  Lane e t a l . , 1 9 8 7 ) .  a significant  factor  (Plake  level  mixed r e s u l t s  must be  as  an  significance  but  u n l i k e Sax  of these  cautious  factor &  any  reported  interacting i n the and  studies  hand, Hodson  Interaction  effect  with  significant  main e f f e c t  importance  of a b i l i t y  sequencing  questions area  ability  factor  for item order.  levels  and  t o the  from e a s i e s t  with  but  he  Gellman it the find  did  justification the  was sample an  find  Considering  to hardest,  is surprising.  d i d show a  (1966),  (1984) d i d n o t level,  be  observations.  Klosner  Cromack  Ansorge,  i n making  a non-significant factor  other  in this  the  and  an  g e n d e r d i f f e r e n c e s s i n c e t h e r e may  t o be  the  of  in several  subject  involved with  Achievement  factor  of the  The  there  presence  sex  t h a t one  c o n c l u s i o n s about  research  also a  a significant  s t u d i e s t h a t examined  Klimko,  used.  between  i t t o be  to the  but  A n s o r g e e t a l . (1982) found  e t a l . , 1983;  find  effects,  subject.  s u b j e c t was  did find  1984;  still  the  Plake,  effect  but  not  other  of  S e v e r a l s t u d i e s d i d not  Melican, Other  order  differences related  sex  studies.  significant  a  the  of lack  of  (46) Any  study  significant findings. effect  results; The  results  speeded  for item order  Hodson, 1984) student Simple  or an  tests  speeded  Effect  effects  which confirmed either  interaction  effect  interaction  of  of  of v a r i a b l e s later  item order.  with  researchers  The  simple  since textbooks  suggest  be  that  classroom  items  teachers  analysis  previous  an  outright  1975; with  main  Kleinke,  the  sex  be  main e f f e c t  1980;  of  used  easy  experiments  which allowed  use  the  of  t h a t compared  random a r r a n g e m e n t s on  to a psychology  students.  In t h e  second  final  study  random forms o f a p s y c h o l o g y  the  order  and  the  to  since  equivalency  multifactor analysis,  of the  simple  groups.  the  results  effect  of two  ten d i f f e r e n t  exam w i t h  they  used  midterm with  very  of s e v e r a l  item d i f f i c u l t y .  Bowers u s e d  variations  not  continued  interaction  (1972) r e p o r t e d t h e  Huck and  item  to hard  f o r m u l t i p l e comparisons  Bowers  experiment  of  of the day  the  was  investigating  q u e s t i o n s about  among s e v e r a l s i m i l a r  Huck and  different  had  item order  effect  from  In a d d i t i o n ,  procedures,  also  arranged  still  of m u l t i p l e f o r m s .  first  caused  (Towle & M e r r i l l ,  remained a concern  similar  found  Re-examined  only concern  could  tests  ( P l a k e , Ansorge e t a l . , 1982).  The the  using s l i g h t l y  120  In  the  random  psychology  six different 162  students.  The  (47)  difficulty s t u d y and  ratings .15  t o 1.00  difficulty  rating  study's  10  tests  tests.  Huck and  the  ranged  i n the  ranged and  They c o n c l u d e d  t h a t the  be  and  a valid  one  concerning compared  This  sample order and  i f low  sample,  purpose are  a college  However,  large,  based  from  sample  do  not  ability"  i s supposedly Glock,  result  1963).  would have a p p r o x i m a t e l y  difference test  forms. might  not  "comments  For  as  random  In t h e o r y , ability  have  one,  students  represent a  "low  item  students, any  ability"  i s i n r e f e r e n c e to a  a college students  Secondly,  sample  f o r whom the  o f a l a c k of  Therefore equal  in  statements."  would  ability  6  reference relevant to  p o p u l a t i o n , then  the  study's  somewhat q u a l i f i e d  of g e n e r a l i z a t i o n .  low  first  hypothesis  mention, c o l l e g e  order  and  of the  of  in i t s applicability.  m i g h t make a d i f f e r e n c e .  beginning  i n the  second  be  first  average  writers,  should  order  the  any  other  class  i f "low  i n c l u d e any  (Ahmann and  .61  i n the  significant  i s a category  students. age  to  most a p p l i c a b l e t o low  ability  then  does not  limited  i n a psychology  f o r the  The  sequence e f f e c t  Bowers b r i e f l y  effects  no  criticized  was  t o 1.00  i n the  presently appearing  study  as Huck and enrolled  found  a sequence e f f e c t  with  .54  t o 70  ratings  .00  second.  from  .66  Bowers  item d i f f i c u l t y  from  item  effect  of  item  success  v a r i o u s random  amounts o f e a s y  correspondingly equal  probably  orders  questions  amounts o f  at  success  (48) at  the  beginning  student.  Huck and  o n l y one ability  type  of  level,  Sirotnik try  f o r the  easily  Bowers' f i n d i n g s  t o have t h e i r  item order  study  and  they  random.  The  used  mathematics, s o c i a l  content  based  students  they  were  used  final  exam q u e s t i o n s  and  systematically divided four t e s t s  matrix  f o u r hour  hypothesized  would  be  introduce a s l i g h t  was  negligible  but  was  another  broad  and  practical  of  hand, d i d larger  arrangements  as  items  an  five  them i n t o  sample  item pool they basic subject  four t e s t s  equivalent.  hour The  effect  to the  them t o tests one  used  areas  This  they  hour t i m e  limit  t h a t the a u t h o r s  to a l l o w t h e i r  results  by  rearrange  which  s c h o o l system.  low  arranged  i n random s e q u e n c e .  speeding  or  by most d e f i n i t i o n s ,  For  e i g h t one  attempt  The  felt  time  t o have a  application.  Most o f S i r o t n i k their  other  a r r a n g e m e n t s had  design allowed  typical  limit  the  based  so,  the  arranged  into  did  supported  from  sampling  test  on  a l a r g e grade based  included.  the  multiple  type  s t u d i e s , s c i e n c e , language a r t s ,  In a d d i t i o n ,  or  one  to  basic subject a r e a s — e i t h e r  2,463 e i g h t h g r a d e s t u d e n t s ,  content  therefore limited  g e n e r a l i z e to a  a content  grouped a c c o r d i n g to t h e i r  ability  ability  high.  (1974),  reading.  low  i t e m a r r a n g e m e n t , random, and  Wellington  w e l l as  one  are  and  population,  of  frustrated  and  hypothesis  Wellington's  tests  t h a t t h e r e was  no  of  significance  difference  in  (49)  the  means, v a r i a n c e ,  for  most o f t h e i r  means of  the  item d i f f i c u l t i e s ,  tests.  reading based  However, t h e y  tests  The  content  reading  the  random a r r a n g e m e n t .  test  was  Although  s i g n i f i c a n t , i t was  Wellington  not  Once a g a i n , with  have an  effect  area.  course,  the  been  improved  such  as  with  use  m u l t i p l e comparisons.  the  mathematics t e s t s time  (Towle & M e r r i l l ,  difficult  t o make a c c u r a t e  because  Sirotnik  was  with  and  by  Sirotnik  the  and  play  results  real  is  likely  research,  i n one  interactions  that  or a c h i e v e m e n t  main e f f e c t  could  increased alpha The  and  very  Wellington's  None t h e  less,  subject were  have  error  study  show an  effect  1975).  of  Wellington  Of  been  level  could  item  have tests that  order  previous  Unfortunately,  comparisons  not  level.  i t i s notable  i s c o n t r a r y t o some  research  tests  result  than  of a p p r o p r i a t e s i g n i f i c a n c e  d i d not  l i m i t s which  higher  item order  of s i g n i f i c a n c e .  the  percent  t y p i c a l classrooms  the  sex  of t h e  for  under  and  have been o t h e r  tests  the  l i m i t s comes i n t o  a result,  a chance anomaly from the to repeated  that  significance.  must c o n t e n d As  significance  did find  dismissed  involving  to e f f e c t  in t h i s study  this  of t i m e  In S i r o t n i k  o n l y seemed  T h e r e may  analyzed  limits.  1.5  practical  Research  effect.  item order  just  of  numbers o f s t u d e n t s  problem of time  due  the  item order.  large  to  being  reliability  were s i g n i f i c a n t l y d i f f e r e n t .  statistically as  or KR-20  of t h e d i d not  i t is  mathematics provide  any  (50) details nor  as t o the d i f f i c u l t y  d i d they  A study is  the study  have t h e h a r d  similar by F e l d t  groups.  sampling  They used  Development  t o easy  and F o r s y t h with  (1974).  used  with  i n grades  forms and i n c l u d e d a s a s p e c i a l  test  form.  As a r e s u l t , and t e s t e d .  found  effects  caused  content  based  significant  questions. special  differences  section  than  test  have an e f f e c t .  enough t i m e rushed  to complete  regular  form  special  section.  other  They s a i d  to f i n i s h  the questions  Another  were  differences (p < .05) and  i n the  i n the r e g u l a r for their  s t u d i e s , time  section. mixed  limits  seemed  t h a t e v e n t h o u g h s t u d e n t s had  the t e s t ,  r a t h e r than  packages  t h e means o f t h e same  s e v e r a l p o s s i b l e reasons  F o r one, a s w i t h  groups of  i n the other  significant  q u e s t i o n s when t h e y were p r e s e n t e d  to  One g r o u p o f  between t h e means o f t h e l a n g u a g e  s e c t i o n s were h i g h e r  results.  item  one o f t h e two r e g u l a r  The means o f m a t h e m a t i c s q u e s t i o n s  They c i t e d  by  of E d u c a t i o n a l  between t h e means o f t h e m a t h e m a t i c s q u e s t i o n s no  order.  T h e y were,  9 t o 12.  eight different  They  item,  and W e l l i n g t o n  o r one g r o u p o f t h r e e d i f f e r e n t  test  and  of item  two forms o f t h e Iowa T e s t  m a t h e m a t i c s q u e s t i o n s were drawn f r o m  developed  tests  variation  the context  techniques  on 530 s t u d e n t s  language q u e s t i o n s  of t h e i r  t o t h e one by S i r o t n i k  however, o n l y c o n c e r n e d the matrix  rating  t h e y may have when t h e y  more  were g i v e n  when t h e y were g i v e n possibility  felt  i n the  i n the shorter  g i v e n was t h a t t h e  (51)  s t u d e n t s may  have been s l i g h t l y  more f a t i g u e d  questions  when t h e y were p r e s e n t e d  section.  These c o n c l u s i o n s  the  e f f e c t s were o b s e r v e d  but  not  with  the  may  reason suggested  that  questions order  i n the  f o r the  section. or  speeded  tests.  was  omission their was  not  not  possible  their  clearly  the A  final have  order  identical  did  item  address  the  t e s t s were power t e s t s or that  the  to  of  the  the  they a l s o s t a t e  i t e m s was that  common.  t i m e was  possible  that a  not  claim  a  that  This  by  effect.  problem r a t h e r  than  of  i f they of  were the  the  results in  t i m e was  time  t e s t s may  issue  f a c t o r when  sequence e f f e c t s can  students  sequence  that  item  speeded  not  non-completion  a significant  since  sections  arrangements of each s p e c i a l  was  orders  item  e f f e c t s of  when t h e r e  s i m i l a r to  that  Forsyth  and  what seems t o be  item  the  effect,  of  Results  not  to c l a i m  also  that  mathematics  not  F e l d t and  a problem, but  is  fact  s e q u e n c e e f f e c t s may  subtle  They s t a t e  able  the  They f e l t  form s i n c e  d i f f e r e n t item  a significant  factor  item  that  of mathematics  being  by  regular  form.  They a l s o d i d  whether  the  s p e c i a l s e c t i o n was  regular  administering  tests  was  i s unfortunate  control  with  longer  have been more r i g o r o u s .  i n some s u b t l e  i n the  It  only  tempered  language s e c t i o n .  mathematics s e c t i o n s  been p r e s e n t  are  i n the  when a n s w e r i n g  not  there a  However, i t be the  the  cause  effect.  occur with d i f f e r e n t  f r u s t r a t e d by  one  arrangement  may  (52) tend  to omit  students  or  to complete  Time  standardized  replicable  and  Controlling  limits  shortcomings  controlled  conducting  a fairly  w h i c h had  The well the  first  controlled  taken  t h a t an  intended  three  the  was  ones f i r s t and  of how high  controlled effects  more  the  t h a t s e v e r a l of effect  important  from  items  i n the  not  of  extensive school  this  felt  not  item order,  be  the that  i n the may  test  case  or  are  that  was case  of  t r y t o do mask t h e  Traub d i d not  they  had  was  to easy  within-subject  population  order  I t i s commonly  order  Particularly  Hambleton and  previous  Traub  therefore s t a t i s t i c a l l y  While  research,  item  or a h a r d  i t may  After  factors.  factor.  test  However, the  (1974).  of p r e v i o u s  examinee's c o n t r o l  to hard  do  r e s e a r c h were more  Traub  a r r a n g e m e n t s , many s t u d e n t s  item order.  i n the  item order  t h a t H a m b l e t o n and  researcher.  easy  was  fairly  easy  to easy  estimate  on  review  shown an  factor  in fact, by  more c a r e f u l l y  H a m b l e t o n and  careful  not  hard  of  by  i n that order.  students,  the  certain.  w i t h i n - s u b j e c t rearrangement  assumed  without  of p r e v i o u s  Traub concluded  controlled  than  Factors  adequately  not  must be  c o n c l u s i o n s more  Other  Hambleton and  order  t o make r e s e a r c h  Some o f t h e  studies  more q u e s t i o n s  experiencing a different  frustration. and  fail  the  effect  have  an  rearrangement  sampled,  they  (53)  attempted  to prevent  instructing  the  examinee c o n t r o l  students  presented  in their  specially  p r i n t e d with  booklets.  page t o d i s c o u r a g e questions  The studies  t o do  had  r a n g e of pointed  the out  t h a t no was  Traub's a r t i c l e item p o s i t i o n  from the  to  data  student of  the  on  perceived  to the  used  the  the  rank  the  study.  i n the  with  test  the  clear  have  To  to the  test.  limited  They of  item  Hambleton  correlation the  level  as  and  between  item  should  estimated  test  was  a  difficulty  Traub d e a l t with  was  Traub contended  lacked r e a l i s t i c  t o be.  o f how  or  effects,  importance  insure  out  important  Item o r d e r  to  t h a t some  They p o i n t e d  to the  attempt  other  manual.  subjects.  related  felt  variation  item  description  tests  last.  Traub due  each  easy  t o do  position  Hambleton and  by  for  However, t h e i r  test  order  p r i n t e d on  i n the  item d i f f i c u l t y  directly test.  was  i n the  by  were a l s o  Unfortunately,  and  s t u d i e s may  motivation  would be  attaches  effect  Hambleton and  most s t u d i e s gave no  felt,  an  published  factor  previous  student  questions  o n l y the  test  motivation.  effective  find  of t h e i r and  question  t h a t Hambleton and  reported  information available  A third  hard  item order  questions booklets  i n f o r m a t i o n on  i n the  the  from s e a r c h i n g  published.  i n , based  standardized  The  item d i f f i c u l t i e s  difficulties  have been  and  reason  failed  the  o n l y one  students  first  second  t o do  of  a  student  that the they  student  (54) motivation, students teacher  H a m b l e t o n and  that  to a r r i v e  significant volunteers Ansorge,  since and  With  the  reported that  would  be  final  used  grade.  found  no  significant  Lowry  by t h e i r  This  P l a k e , Thompson, and  they t o l d  point  Lowry.  effects,  (1980) u s e d  (1980)  when P l a k e ,  motivated  students in  Hambleton  that  easier  controlled,  i t e m o r d e r s h o u l d have an e f f e c t .  that  the e f f e c t  as s u g g e s t e d  was  caused  by two  by M o l l e n k o p f  q u e s t i o n s a t t h e end  to  personality  trait  s u c h as a n x i e t y .  As Munz and  (1968) s u g g e s t e d ,  some a r r a n g e m e n t s  s u c h as  anxiety level  performance higher  i s lowered  eleventh  test  their  grade  to easy  theories,  was  Achievement test  to determine  to  need  easy t o have a time by  the  arrangement.  Their  to average.  A n x i e t y T e s t two  Smouse  hard  Traub  mathematics s t u d e n t s e n r o l l e d  c o n s i d e r e d low  cause  was  are d e b i l i t a t e d  Hambleton and  m a t h e m a t i c s summer s c h o o l p r o g r a m . ability  cause  w h i l e a t t h e same  f o r t h o s e who  a n x i e t y of the hard  To  possible  o f t h o s e s t u d e n t s who  on t h e t e s t  For  easy  seem h a r d e r .  higher  They  factors.  arrangement  the performance  and  (1950), c o u l d  of a hard  Another  used  but  these three f a c t o r s  fatigue,  improve  is also  results.  felt  the  classroom  study, they d i d get s i g n i f i c a n t  hypothesized one,  at a  P a r k e r , and  a similar  Traub  the r e s u l t s  Traub  106  in a  mathematics  T h e y were g i v e n an  weeks p r i o r  the h i g h e s t s c o r i n g  used  25  to the  mathematics  p e r c e n t and  the  (55)  lowest  scoring  reactions.  25 p e r c e n t  On t h e d a y o f t h e t e s t  the C o o p e r a t i v e Educational assigned  two  forms,  were t o l d stress  not  that  to hard  t h e exam c o u l d be used students  had t h e i r  order.  was  as s l i g h t l y  due t o i t e m o r d e r  e a s y a r r a n g e m e n t were arrangement.  While  results  (p < . 0 5 ) .  lower  than  Scores  a  of the  b u t some be  significant  t o hard  some s t u d e n t s d i d n o t f i n i s h  f o r contingency.  types  of q u e s t i o n s not completed  would  n o t be s u b s t a n t i a l l y the t e s t .  10  on t h e h a r d t o  on t h e e a s y  test  was  must  indicated  chi  speediness  for only half  so the t e s t  i t was an i n s i g n i f i c a n t  finished  To measure  t h e p u l s e meter d i d  felt  had  students  p u l s e measured e v e r y  the a u t h o r s squared  The  speeded.  H a m b l e t o n and T r a u b ' s effect  to discourage  r e p o r t e d a s a power t e s t ,  the students d i d not f i n i s h ,  considered  to easy  f o r marks.  work, s o p u l s e r a t e s were c o l l e c t e d The t e s t  randomly  or hard  b o o k l e t s were d e s i g n e d the arrangement  given  I I p u b l i s h e d by t h e  The s t u d e n t s were  easy  trait  a l l s t u d e n t s were  U n f o r t u n a t e l y , i n one c l a s s  students. of  changing  levels  minutes.  either  The t e s t  from  of a n x i e t y  Mathematics Test Algebra  Testing Service.  arrangements. students  for analysis  number  They f u r t h e r and f e l t  that  enough d i f f e r e n t  As a r e s u l t ,  not a p l a u s i b l e  they  their based  tests on a  analyzed the the r e s u l t s  i f a l l students felt  that  e x p l a n a t i o n of the r e s u l t s .  (56) In  addition,  hard  to easy  with  the  result  item order  item order  p u l s e meter  i s marred  affected  produced  than  test  more s t r e s s  the easy  to hard  by  the  s m a l l numbers, and  difficult  s i n c e the  two  samples  with,  H a m b l e t o n and  but  the  order  the  two  does e f f e c t  show a s i g n i f i c a n t with of  item order.  significance  Munz and  Smouse  report not  an  find  students Plake,  analysis any  this based  significant  in this  related  to sex  and  a sex  Hambleton and a difference,  and  was  that  stress  level.  identified  .05  nor  on  .10;  by  the  one  of  sex.  Lowry  the  first  in a later  that  Two  with d i f f e r e n t  tests  p r o p e r t i e s and  was  from  what C r o n b a c h  Traub d i d  study  practice  c h a n c e of c h e a t i n g .  effect  s t u d i e s to  item order  in a class  They f e l t  of  by an  effect  to reduce  d o e s make of the  orders are  two  t h e r e f o r e make  t h a t the  (1946,  female  interaction.  they c a u t i o n a g a i n s t the  invalid.  level  results  (1980) t h e r e was  making s e v e r a l forms o f a t e s t  comparisons  a  between male and  item order  tests  not  interaction  however, t h e  H a m b l e t o n and  Traub concluded  with d i f f e r e n t  A.A.T. d i d  of  replicated.  difference  and  with  performance  d i d show a t r e n d w i t h  and  was  The  a significant  s t u d y even though  Ansorge, Parker,  the a n a l y s i s  conclude  (1968) were n o t  Interestingly,  This  Traub d i d t e n t a t i v e l y  results  between  item order.  begin  main e f f e c t The  measured  means t o  as  had  as  The  different  the  anxiety levels  anxiety.  cause  1950)  for  called  this response  (57)  set.  In a m u l t i p l e c h o i c e  format,  students  expected  an  easy  to hard  a r r a n g e m e n t , s o when a s t u d e n t  begins  hardest  items,  since his expectation  is the  he becomes more a n x i o u s  t o have e v e n h a r d e r  with the  q u e s t i o n s a s he p r o g r e s s e s  through  test. Monk and S t a l l i n g s  (1970) were c o n c e r n e d  about  Hambleton and T r a u b ' s r e c o m m e n d a t i o n a g a i n s t u s i n g m u l t i p l e forms  in a class  results  of twenty-two t e s t s  three year made f r o m pairs. by  to prevent  period in their items  chosen  Each p a i r  content  students typical  had a d m i n i s t e r e d  questions  but each p a i r  had t h e i r  to discourage  i t .  in large scale  The t e s t s  t-tests,  testing  and f o u n d  were n o t s i g n i f i c a n t l y equivalent However, One was  tests  The t e s t s  grouped  were  together  form  categories  Nearly  2000  had a p p r o x i m a t e l y  of t h e i r  that nine  c o u l d be p r o d u c e d  a t t h e .01 l e v e l  cheating.  on  100  speeded as i s  situations.  different.  they d i d concede  content  were s l i g h t l y  They t e s t e d the s i g n i f i c a n c e repeated  a  T h e r e were no a r r a n g e m e n t s based  i n v o l v e d s i n c e each  writing  over  an i t e m p o o l and a d m i n i s t e r e d i n  had i d e n t i c a l  randomly rearranged s u b j e c t s were  they  They examined t h e  geography course.  from  categories.  item d i f f i c u l t y ,  cheating.  result  by  out of e l e v e n  pairs  They c o n c l u d e d  that  by r e a r r a n g i n g  items.  t h a t two t e s t s of s i g n i f i c a n c e ,  were  significant.  and t h e o t h e r  was  (58) at  the  order  .001  level.  effects  may  As  a result,  in fact  be  they  present  cautioned  that  item  in large scale  testing  programs.  Monk and use  of  the  t-test from  Stallings*  t-test  used.  without  I f an  chance a l o n e  result  i n the  analysis  a description  t h e r e c o u l d be  Comparisons with  limited  by  absence  absence  of d a t a  possibility if  the  showed any  Allison  While  questions.  the  .05  level  may  be  type  of then,  significant of  s t u d i e s are  also  item d i f f i c u l t i e s questions  and  r a n g e was  then  evidence  the  that  fact  and  the  minimal, i f the  t h a t two  item order  the  in  These weaknesses h i d e  item d i f f i c u l t y  were u s e d ,  repeated  were used  i n i t e m s e q u e n c e were m i n i m a l ,  t-tests  the  the  one  other on  by  least  o r d e r i n g of easy  i f the  tests  is a  factor.  of c o n c e r n  o f H a m b l e t o n and  to A l l i s o n  (1984).  previous  involved students  who  Hambleton and  young s t u d e n t s .  T r a u b had  test He  format, felt  Traub  I t was  research  into  were p r o b a b l y  item rearranging s t r a t e g y c i t e d  with a r e s t r i c t e d of  the  t h a t most of t h e  w h i c h had use  of d a t a  research findings  were a l s o  to  that  effect  significant  The  on  to hard  changes  appropriate  at  ten comparisons a t the  the  of  inappropriate t-test  significance.  relationship  i s marred  tried  simply  that students  noted  by  item  order  mature  by Tuck  to c o n t r o l  Allison  (1974)  this  enough  (1978). factor  chose a  sample  i n g r a d e s i x would  (59)  generally one  n o t y e t use t h i s  strategy.  o f t h e few s t u d i e s t o examine  college  classroom,  Allison's  studies  t o examine t h e i n t e r a c t i o n  low a n d h i g h a b i l i t y  In  h i s study,  students orders,  Allison  hard  and  Traub  The i t e m s  .981 w i t h a mean o f .673.  90  minutes t o complete t h a t was t a k e n  High  a n d low a b i l i t y  scores did  were used Finally, the to  found  and random.  were  three d i f f e r e n t significant  test  part of t h e i r identified  .178  were  program.  from  I . Q.  Thirty-five  students  s o o n l y 327 s t u d e n t s  T h e i r mean I . Q. was 113.31. s t u d i e s have done, t h e s c o r e s o f with  the scores  has a n y i n t e r a c t i v e  difference formats.  main e f f e c t s  from  M o t i v a t i o n was a  c o n t r a s t t o Hambleton and T r a u b ' s  no s i g n i f i c a n t  by Hambleton  s i n c e the students  i n f o r m a t i o n on f i l e  a s s o many o t h e r  items  were g i v e n an ample  school records.  i n the a n a l y s i s .  common  The  in difficulty  Students  account  students  160 boys were compared  In  with  364 g r a d e s i x  a s recommended  was an i m p o r t a n t  see i f item order  of item order  t h e 64 i t e m t e s t . into  i n the students  n o t have s u c h  effect  i n one o f t h e t h r e e  ranged  to  that the t e s t  one o f t h e few  t o easy,  a wide r a n g e o f d i f f i c u l t y  told  was a l s o  r a n d o m l y gave  had  factor  outside of a  students.  t o hard,  (1974).  to being  item order  study  a s c i e n c e exam a r r a n g e d easy  In a d d i t i o n  o f t h e 167 g i r l s  effect  with  findings,  gender.  Allison  between t h e means o f t h e Nonetheless,  a s s o c i a t e d with  there  were  I . Q. and g e n d e r .  (60) Boys and this the  students  with  science test. two  significant  interactions,  high  Allison  Allison  reported  interaction  sex,  and  other  concluded before  recommending any  both  T r a u b were v e r y  factors  one,  study  these  two  group with  that  controlled  to a s e r i e s  of d i f f i c u l t  of d i f f i c u l t  statistically  difficult  for students.  difficult On  the  by  in their  d i d not  Although the  an his  technique  Another  may  be  Science be  the  control  have a  math q u e s t i o n s  other  open  Traub used a  not  the  results.  have a  Students  may  Hambleton  i t leaves  Traub.  science questions.  another.  many o f  have u s e d  H a m b l e t o n and  to  Allison  over  students.  still  used a s c i e n c e t e s t .  which are  may  Allison  Hambleton and  i s that while  Allison  sensitive  by  may  similar  study  Traub d i d not  restricted  were young, t h e y  difference  the  research,  students,  of  should h e s i t a t e  studies differed  subjects was  and  factors  speeded.  In c o n t r o l l i n g  where H a m b l e t o n and  experimental  series  Allison  no  are  Item a r r a n g e m e n t  thorough  group of u n r e s t r i c t e d  test,  were n o t  i n v o l v e d i n item order  q u e s t i o n why For  the  one  the  on  the  were  involving  means  regarding  In e x a m i n i n g  These r e s u l t s  tests  higher  conclusions  t h a t measurement s p e c i a l i s t s  Since and  item order.  had  that there  effects  s t u d i e s where t h e  scores  drew no  main e f f e c t .  significant I . Q.,  I . Q.  math more  than  to a  questions  subjectively  hand, math  have a b e t t e r match between s u b j e c t i v e and  questions  statistical  as  (61)  difficulty.  As a r e s u l t ,  produce  of an i t e m o r d e r  of  less  speeded  Although  test  both  difficult  a s compared  t h a t time  Test  Of c o u r s e  t o power t e s t  both  I f time  s t u d i e s merely  was a  suggest  was  i n the  confirm previous  s t u d i e s by Hambleton and T r a u b  t h a t s u b j e c t s may p o s s e s s  modifying  the e f f e c t s  research.  (1974) and A l l i s o n  questions  in  the order  of  s e v e r a l under  of s k i l l i n  They s u g g e s t e d  the item order  r a t h e r than  effects to  answering  by t h e r e s e a r c h e r .  questions  first  the t e s t ,  Test-wiseness  and f o r m a t s  the questions  This s k i l l  i s one  i s d e f i n e d by M i l l m a n and  of the t e s t  and/or  to receive a high score."  that  then  the  the t e s t  Hambleton and  i f t o o many s u b j e c t s d i d t h e e a s i e s t  by o m i t t i n g t h e h a r d the e f f e c t  who d i d n o t use t h i s those  the  the g e n e r a l c a t e g o r y d e s c r i b e d as  situation  Traub reasoned  that  by a n s w e r i n g  (1965) a s "a s u b j e c t ' s c a p a c i t y t o u t i l i z e  characteristics taking  first  presented  "Test-wiseness". Bishop  some s o r t  of item order.  examinees c o u l d c o n t r o l  easiest  of  there  significant  (1984) were t h e o n l y s t u d i e s o f i t e m o r d e r  of  factor.  Wiseness  The  the  may  the issue  may be a  may have been a f a c t o r  Hambleton and T r a u b s t u d y . then  effect.  s t u d i e s c l a i m t o be power t e s t s ,  some s u g g e s t i o n  factor,  science questions  questions  of item order  s t r a t e g y would  who d i d use t h e s t r a t e g y .  until  t h e end  on t h e s t u d e n t s  be masked  by t h e r e s u l t s  (62)  This in  light  ninety  c o n c l u s i o n o f Hambleton and of a l a t e r  psychology  s t u d i e s . For  students  multiple  choice tests.  reported  that they  and  He  would  leave the d i f f i c u l t  evidence Klimko  found  t h a t 69%  seek  out  with  s u b j e c t s took  the  other  the  hard  the  hand, 58 questions  addition,  questions  any  h i s 111  until  test  t o do  strictly  students  went  t o work on  5 students first,  end  skipped and  3 students  t o b e g i n a t a random p o i n t , and o f t h e methods l i s t e d In  on  gave a s h o r t q u e s t i o n n a i r e on f o r answering  students  from  grade  All  grades  the  three d i f f e r e n t  hard,  as  presented,  within-subject  the  four through  types  end  found  given.  of t h e  that On  skipped  over  test.  through  2 students  and  easy the  d i d not  use  Thomas  (1986)  student's preferred test  to  items  fifth  to easy.  to  year  who  of s t r a t e g i e s ,  rearrangement  He  order  flipped  some s t u d e n t s  or h a r d  midterm  l o o k i n g f o r any  Allison  achievement  r e p o r t e d l y had  by  the q u e s t i o n n a i r e .  a very extensive study,  strategy  first  included a  i n o r d e r , but  around  students  uncovered  students.  them a t t h e  on  Further  Klimko  i n the  used  questions  was  of a  questioned  they  of the  last.  study.  psychology  (1978)  easy  of w i t h i n - s u b j e c t rearrangement  (1984) i n h i s i t e m o r d e r  is significant  strategies  the  questions  examination  test  Tuck  the  q u e s t i o n n a i r e a t the  In  one,  about  self-report  43  Traub  415  university.  would  either  use  easy  one to  The  easy  to  hard  s t r a t e g y was  used  by  58.4%  of  of  (63) the  students  students senior to  in junior  the  year "as  h i g h s c h o o l , 49.6%  and  59.6%  strategy.  s t u d e n t s may  Thomas  conclude:  the  supersede examiner  s t u d e n t s ' own  the  of t h e  of  the  students  Although  use  item-difficulty  i s not c l e a r .  these  rearrangement  test-taking  There  in  in their  Most o f t h e r e m a i n i n g  that  Whether  62.7%  of the s t u d e n t s  of u n i v e r s i t y . presented"  are evidence Allison  intermediate grades,  h i g h s c h o o l , and  fifth  used  i n the  third  students  results strategies,  strategies  sequence  intended  does not  seem t o  by  the  be  enough e v i d e n c e  to doubt  t h e m a j o r i t y o f t h e s t u d i e s on  item-difficulty  sequence  s i m p l y because  sequence fact,  of r e s p o n d i n g  i t may  be  item-difficulty  that  the  to t y p i c a l  are allowed  t o use  It i s also  perceive  the q u e s t i o n s  be  testing  possible  items.  more  majority  of s t u d e n t s  say,  (Allison  they cannot  when  students  usually  to argue  students  that  answer  t o be  p.869)  the  easy  answer e a s i l y  l t may  be  quite reasonably,  answer a r e  & Thomas, 1986,  involving  s t r a t e g y they  words,  In  readily  t h e q u e s t i o n s t h e y c a n answer  questions  controlled.  situations  they can  In o t h e r  not  actual  of s t u d i e s  the q u e s t i o n s t h e y cannot  the hard  answer  was  results  whichever  choose.  and  items  sequence can  generalized  items  to  the  left  first until  that  the  that and  to  they  the  later.  be  (64)  It  i s c e r t a i n l y not c l e a r  s c o r e s by answering Rindler  (1980) had  i f students can  improve  the more d i f f i c u l t q u e s t i o n s 160  college volunteers write a  their  later. thirty  item v e r b a l a p t i t u d e t e s t with s p e c i a l s c o r i n g sheets designed  to i d e n t i f y  a l s o put  i n t o one  minutes,  25 minutes,  all  i f items were s k i p p e d .  Students  were  of three d i f f e r e n t timed c o n d i t i o n s , 20 or 65 minutes.  Grade p o i n t averages  students were a l s o obtained to d i v i d e the students  of  into  h i g h , medium, or low a b i l i t y rankings f o r a comparison of their  performance. The r e s u l t s i n d i c a t e d a complex i n t e r a c t i o n between  a b i l i t y groups and some students who  skipping questions.  While  there were  skipped i n every a b i l i t y group and  under  most t i m i n g c o n d i t i o n s , o n l y the high a b i l i t y students  who  skipped q u e s t i o n s had c o n s i s t e n t l y higher s c o r e s under every timing c o n d i t i o n . students who  On the other hand, middle  d i d not s k i p had the higher scores under  timing c o n d i t i o n .  In c o n t r a s t , low a b i l i t y students  skipped had higher scores o n l y when they had the t e s t .  Low  when they had  ability  R i n d l e r concluded  who  25 minutes f o r  a b i l i t y students d i d not s k i p any the ample 65 minute time  every  questions  limit.  t h a t some students may  be b e t t e r  advised to spend more time on the e a s i e r q u e s t i o n s which are u s u a l l y a t the beginning of the t e s t .  T h i s t a c t i c would  help to make sure t h a t those students would spend the most  (65) time She  on t h e q u e s t i o n s felt  their  i t was  time  difficult  fact  One  only  between  Latent  to suggest  likely  to get c o r r e c t .  t o a l l examinees t h a t  effectively  shown by t h i s  of procedures  i f they  research  skipped  the  questions  students  skipped  20% t o 98% o f t h e i r  and C o n t e x t  The r e s e a r c h  into  predict  skipped  question.  they  returned  High a b i l i t y  available,  item order  students,  would r e t u r n t o  questions.  began t o have an  another  statistical  has been p r o b l e m a t i c a l .  significant  or d i s m i s s  uncertainty new  skipping  questions,  skipped.  different  Effects  T h e r e a r e n o t enough c l e a r l y conclusively  l t w e l l a r e two  i s to r e t u r n t o the s k i p p e d  on t h e amount o f t i m e  Trait  i s that using a  a s s o c i a t e d with  5% o f t h e q u e s t i o n s  depending  most  s t r a t e g y and u s i n g  When low a b i l i t y to  were  questions.  test-wiseness  difficult  unfair  would be used  Another  skills.  they  results to  item order  impact  technique  effects.  i n the l a t e  gained  This  1970s a s  increasing  popularity.  Item r e s p o n s e several  advantages over  constructors an  assumption  In o t h e r  theory  found  or l a t e n t  classical  attractive,  t h a t each  test  trait  test  model t h e o r y  theory  that  but i t s s t a t i s t i c s  i t e m was  w o r d s , t o be s t a t i s t i c a l l y  locally  effective,  had  test involved  independent. the  context  (66) of  questions  b e f o r e and  probabilities Therefore, not  latent  assumption  and  of  Dawis  local  tests  students. fifteen one  of the  i n d e p e n d e n c e and  with  junior  this  The  was  does a d d r e s s  the  issue  that did  starts  indicate the  the  was  has  As  difference  for nine  of t h e i r  found  a significant  means o f t h e  core  items  on  two  a common c o r e  on  that  to  core  to t e s t  on  if  question  might  Dawis*  best with a  rasch  items  tests  item-parameter-invariant  test  results  item  difference  of t h e i r  unique  items.  W h i t e l y and  core  any  students, i t  one  do  of  provide  However, t h e i r  i n the  they  that  high.school  the  a result,  questions.  verbal  tests  effect  the e f f e c t  context  q u e s t i o n s were  around  an  of  this  same p o s i t i o n  seven  addition,  They c o n c l u d e d  effect  senior  i f some s t u d e n t s  easy  show a s i g n i f i c a n t  parameters obtained In  of  question.  with  item.  to t e s t  sixty  not d e s i g n e d  item order  not  the  i n the  contexts  the  do  the  item order  first  forty-five  f o r each of  specifically  results  and  item  other  study  have on a n o t h e r  that  f o r m s w h i c h had  each  totally different While  required  They a d m i n i s t e r e d  questions developed seven  change  to that  (1976) were t h e  T h e r e were s e v e n  test.  responded  theory  t o 1,568  questions  item c o u l d not  factor.  item parameters.  analogies  an  students  trait  a significant  Whitely  on  o f how  after  (p <  .05).  between  (p <  .05).  m o d e l s must  the  (67)  have t h e i r  assumptions t e s t e d before developing  equivalent  measures. U n f o r t u n a t e l y , rasch item parameters a r e u s u a l l y established  in relation  t o the other  items  on t h e t e s t .  S i n c e e a c h t e s t was d i f f e r e n t , t h e r a s c h p a r a m e t e r w o u l d be different had  f o r t h e c o r e g r o u p o f i t e m s . So W h i t e l y a n d Dawis  t o u s e a n uncommon s t a t i s t i c a l  method t o e s t a b l i s h a  common p o i n t o f r e f e r e n c e f o r t h e c o r e g r o u p o f i t e m s they could conclude  before  t h a t t h e r e was i n f a c t a s i g n i f i c a n t  difference. W h i t e l y and Dawis r a i s e d  some c o n c e r n s  c o n s t r u c t o r s who were u s i n g l a t e n t t r a i t  by t e s t  models t o p r e t e s t  q u e s t i o n s a t one t e s t a d m i n i s t r a t i o n a n d t h e n at  another  sitting  item parameters.  use them  u s i n g what were assumed t o be  later  invariant  As a r e s u l t , K i n g s t o n a n d D o r a n s  (1984)  tested context effects w i t h i n the pre-operational calibration  of Graduate Record Examinations  Their research design  d i f f e r e n t questions divided into content.  A fifth  Test.  E a c h t e s t had  four o p e r a t i o n a l s e c t i o n s  p r e - o p e r a t i o n a l s e c t i o n was  composed o f a random s e l e c t i o n o f q u e s t i o n s test  form.  different  S i x v e r s i o n s of the f i r s t questions  Test.  i n v o l v e d 1500 e x a m i n e e s who t o o k one  o f two f o r m s o f t h e G.R.E. G e n e r a l  of s i m i l a r  General  from the second  from the other  f o r m were made w i t h  form i n t h e f i f t h  p r e - o p e r a t i o n a l s e c t i o n , and s i x v e r s i o n s o f t h e second  form  (68)  were made a l s o w i t h d i f f e r e n t q u e s t i o n s As a r e s u l t ,  every  section.  estimates  form i n the p r e - o p e r a t i o n a l  R a s c h model i t e m d i f f i c u l t y  parameter  f o r v e r b a l , q u a n t i t a t i v e , and a n a l y t i c a l  c o u l d t h e n be c o m p a r e d b e t w e e n t h e i r pre-operational  items  o p e r a t i o n a l and t h e i r  placements.  K i n g s t o n and Dorans d i d f i n d  some i t e m s  by c o n t e x t w h i l e many were u n a f f e c t e d . types, items  form.  o p e r a t i o n a l q u e s t i o n o f e a c h f o r m was  u s e d i n one v e r s i o n o f t h e o t h e r fifth  from the other  t o be a f f e c t e d  Of v e r b a l  item  i n v o l v i n g antonyms d i s p l a y e d a s l i g h t  practice  e f f e c t when t h e y were p l a c e d a f t e r a s e r i e s o f s u c h questions, while fatigue effect Quantitative  " r e a d i n g c o m p r e h e n s i o n " showed a s l i g h t  i f located i n the f i n a l  i t e m s showed l i t t l e  change w i t h t h e e x c e p t i o n  t h a t one i t e m on one f o r m i n " d a t a s i g n i f i c a n t l y more d i f f i c u l t test  interpretation"  was  when p l a c e d a t t h e end o f t h e  ( p < . 0 1 ) . By c o m p a r i s o n ,  "analysis  s e c t i o n (p < . 0 5 ) .  i n the a n a l y t i c a l  section,  o f e x p l a n a t i o n " and " l o g i c a l d i a g r a m s " b o t h  significant effect "Analytical  from p r a c t i c e  reasoning"  on b o t h  showed  f o r m s (p < . 0 1 ) .  i n t h a t same s e c t i o n d i d n o t show a n y  significant differences. As w i t h t h e p r e v i o u s r e s e a r c h , K i n g s t o n a n d D o r a n s s t u d y d o e s n o t show t h e s p e c i f i c  e f f e c t s of item  However, i t d o e s i n d i c a t e t h a t some i t e m s placement a t e i t h e r  the beginning  order.  c a n be a f f e c t e d b y  o r t h e end o f t h e t e s t .  (69)  Of  course,  students  these  findings  who t a k e  are limited  by t h e f a c t  that  the Graduate Record  Examination  are not a  random c r o s s s e c t i o n , competent that,  students.  for their  effects  None t h e l e s s ,  purposes,  were s p e c i f i c  elimination,  b u t may be u n u s u a l l y m o t i v a t e d and Kingston  and D o r a n s  they could conclude  to certain  or the proper  types  placement  that  of items,  of those  felt  location  and t h e  types  o f items  would s o l v e t h e p r o b l e m .  contex.t  Effects and litem  One  study  order  using  latent  trait  examine  the s p e c i f i c  effect  of item order  general  context  students  effects.  complete  completed one  while  different  forms  combination  of f i v e  difficulties  purposes. the  fit  good model  and d i s c r i m i n a t i o n ,  Each person  used  the mathematics  was a d i f f e r e n t S i x of the  w h i c h had a r a n g e o f f i t w h i c h were  f o r item parameter  used  anchoring  i n t e r m i x e d were s e t X on f o u r o f  forms and s e t Y on t h e o t h e r interest  students  s e t s of questions.  of questions  A l s o randomly  were o f p r i m a r y  form  s e t A questions  and r e l a t i v e l y  t h e common c o r e  Each  different  forms c o n t a i n e d  for either  grade  Test  1,100 f o u r t h g r a d e  group or the r e a d i n g group.  as  Achievement  the a p p r o p r i a t e reading s e c t i o n .  of the seven  seven  a s w e l l a s more  Yen (1980) had 1,300 s i x t h  the C a l i f o r n i a  mathematics s e c t i o n  item parameters d i d  three  forms.  These  and were q u e s t i o n s w i t h b u t more l i m i t e d  in their  good  sets model  range of  (70) difficulty. with  The f i n a l  s e t s A and X a l s o  A and Y a l s o fit  variation  had a s e t V w h i l e  had a s e t W.  Sets  and e x t r e m e d i f f i c u l t i e s  items  from  questions  with  sets  V and W had r e l a t i v e l y  poor  i n c l u d e d i n the form  s o t h a t some forms c o n t a i n e d  but i n randomly d i f f e r e n t  had  many i d e n t i c a l  had  some a d d i t i o n a l  the  test Yen  questions  found  Item o r d e r  t h a t changing  the order  questions d i d s i g n i f i c a n t l y  effects  the c o r r e l a t i o n  considered  a factor  an  the l a s t  but a l s o  impatience  alter  the d i f f i c u l t y (p < . 0 5 ) .  d i d omit  analysis level  some  Speediness  items.  gave o n l y t h o s e of answering  i s one r e a s o n to f i n i s h  were p o s s i b l e c a u s e s  t h a t the  As a  students  Of result, who  omitted  the q u e s t i o n , but who d i d n o t r e a c h an  t h a t Yen c o n c l u d e d  r a t h e r than  was n o t  of the students d i d  the m i s s i n g answers of s t u d e n t s This  or i n c l u d i n g  q u e s t i o n o f one b o o k l e t e x a m i n e d .  item a chance  item.  forms  the c o n t e x t of  by t h e f a c t  estimates.  s i n c e 93 p e r c e n t  some s t u d e n t s  computer  ignored  other  of sequence the g r e a t e r the  of item parameter  their  order  parameters  were d e m o n s t r a t e d  similarity  course,  orders while  t h a t changed  p a r a m e t e r s and t h e d i s c r i m i n a t i n g  answer  were  a l l identical  in a different  questions  A l l  items.  additional  greater  one f o r m  o r low d i s c r i m i n a t i o n s .  t h e s e t s t h a t were  intermingled  o f forms was t h a t one t e s t  a computer  o f some q u e s t i o n s  near  t h a t f a t i g u e or s c o r i n g anomaly t h e end o f t h e  (71)  t e s t s e e m i n g t o be more d i f f i c u l t .  However, n o t a l l  q u e s t i o n s a t t h e end were more d i f f i c u l t , s o Yen f e l t other  f a c t o r s such  as the content  of the preceding  that  i t e m may  have c a u s e d t h e i n s t a b i l i t y . Although similar and  Yen's s t u d y d o e s i n d i c a t e a n e f f e c t ,  to other recent studies using latent t r a i t a n a l y s i s  does not address  sequence t e s t low a b i l i t y  the i s s u e of whether  from e a s i e s t t o hardest  students.  i t i s best to  f o r the b e n e f i t of the  Yen's s t u d y d i d u s e a l a r g e a n d  heterogeneous sample of e l e m e n t a r y altering  i t is  o f c o n t e x t added o t h e r  into her a n a l y s i s that l i m i t  s t u d e n t s , but her complex  f a c t o r s such  as t e s t  the c o n c l u s i o n s about  length  item  order. Summary a n d C o n c l u s i o n s T h i r t y - s e v e n s t u d i e s p u b l i s h e d over show t h e c o n t i n u e d  interest  demonstrate the d i f f i c u l t y drawing  conclusions.  i n item order  p u b l i s h e d , b u t t h e many a t t e m p t s  conflicting Table similar  40 y e a r s  r e s e a r c h and  t h a t a n y r e s e a r c h e r w o u l d have i n  The d e f i n i t i v e  c o n t r o v e r s y have s e r v e d  the last  s t u d y h a s n o t y e t been  to settle  the item  order  t o c r e a t e a l a r g e pool of  results. 1 presents  the research results  t o t h e a n a l y s i s by L e a r y and Dorans  Unfortunately, results  ina  format  (1985).  o f i t e m o r d e r e f f e c t s do n o t a l w a y s  (72) fall  simply  into  interaction  the c a t e g o r y  effects  (non) or t h e c a t e g o r y  main o r i n t e r a c t i o n show some f i n d i n g s extent  based  clearly as  i n Table  finished  some s t u d e n t s  power  one way  state  is listed  samples,  be  additional  according to  tests,  or timed  and  tests,  the t e s t .  of t h e i r  and  time  a study  1, a l t h o u g h  This also  limits  which  are  is a  no  indicated  involved a large tests for  f o r a few.  t h e number o f  Such a  by Monk and S t a l l i n g s 11 t i m e s  but o n l y the t o t a l  students  j u d g m e n t s t o make,  b r e v i t y and a c c u r a c y  repeated  reported  test.  of the e a s i e s t  between  reported  S t u d i e s w h i c h do n o t  but a speeded t e s t s  i n Table  was  divided  power  the t e s t ,  as a timed  of the study  experiment  will  c o u l d be e s s e n t i a l l y a power  test  i s one  also  were  In a d d i t i o n ,  of the s t u d e n t s  case  with  limited  t o make s i n c e some s t u d i e s gave  most  compromises  they  the e f f e c t s  tests.  Finally  1 were  or t h e o t h e r .  number o f s t u d e n t s  involved  which t o a  of item order  d i d not f i n i s h  judgement  indication  significant  T h o s e s t u d i e s t h a t do  non-significant  on whether  students  difficult  of  main or  (non +).  The t e s t s  that  (sig).  that report results  as b e i n g  information  all  effects  show some p o s s i b l e e f f e c t  indicated  type  of n o n - s i g n i f i c a n t  with  aggregate  were  made.  In t h e  (1970) t h e same  11 d i f f e r e n t number  small  is listed.  (73) Table 1 List  Results Non  Non  Sig  of Item Order R e s e a r c h : R e s u l t s .  Samples  Sample  n  P P P P P P P P P P P P P  elem. col. sec. col. col. sec. col. col. col. col. col. elem. col.  152 113 330 122 156 79 120 162 104 352 111 327 247  Brenner 1964 Munz & Smouse 1968 Smouse & Munz 1969 Monk & S t a l l i n g s 1970 Munz & J a c o b s 1971 Klosner & Gellman 1973 P l a k e , Thompson e t a l . 1980 Plake, Melican eta l . 1983  P P P P P P P P  col. col. col. col. col. col. col. col.  120 181 2 ,000 133 54 97 167  Mollenkopf MacNicol Sax & C a r r Sax & Cromack Flaugher e t a l . Sirotnik & Wellington Feldt & Forsyth Hambleton & T r a u b Towle & M e r r i l l W h i t e l y & Dawis Yen Kleinke P l a k e , Ansorge e t a l . K i n g s t o n & Dorans Hodson Lane e t a l . ( a )  t t t t t t t t t t t t t t t P  sec. sec. col. col. sec. sec. sec. sec. col. sec. elem. elem. col. col. sec. col.  382 1 , 500 335 467 5 ,000 2 ,463 530 106 82 1 ,568 2 , 400 484 170 4 ,000 157 155  Authors French & Greer Smouse & Munz Berger e t a l . Marso ( a ) Marso (b) Kestenbaum & Weiner Huck & Bowers ( a ) Huck & Bowers (b) Plake Plake & Ansorge Klimko Allison Lane e t a l . (b)  +  T y p e s , and  Year 1964 1968 1969 1970 1970 1970 1972 1972 1980 1984 1984 1984 1987  1950 1956 1962 1966 1968 1974 1974 1974 1975 1976 1980 1980 1982 1984 1984 1987  Type  -  Note non = n o t s i g n i f i c a n t ; non + = a d d i t i o n a l relevant e f f e c t s ; s i g = s i g n i f i c a n t ; p = power; t = t i m e d ; e l e m . = elementary; s e c . = secondary; c o l . = c o l l e g e ; - = not given  (74) The  results  significant reporting and  the  effects  related  was  a  w i t h i n the  a  whenever s p e e d  that students  more q u e s t i o n s  order  with  correct  i s supported  exception,  any  power t e s t ,  order  effect  Another  had  any  the  study  students  t h a t attempted  to  significant  to  effects  larger  numbers o f s t u d e n t s , a s w e l l as  finish  improve wider  the  the  exam.  the  effect  need  than  the  get  This  one item order  was  item  finish.  is that  statistical  larger  f o r time  power  using  numbers.  study,  typical  This resulted  With  limits  t h a t some s t u d e n t s attempts  of the  will  a significant  t o have  likelihood  generalizability  s c h o o l s y s t e m and  from  be  reason  of time.  coincidence  In a d d i t i o n ,  sample were u s e d .  first  or s t u d i e s t h a t were  the  finishing  seems t o  that, with  d i d not  needed  range of a b i l i t i e s  student public  statistics  not  suggested  out  increase their  trait  not  no  One  w h i c h had  who  latent  increased  fact  explanation for this  studies find  by  Leary  indicated  Item o r d e r  run  As  were s t u d i e s t h a t  questions  they  s t u d y w h i c h had and  to students  limits,  easiest  twenty-one  studies that  factor.  before  conclusion  a  due  and  item order.  effects.  is a  the  of  those  g i v e n time  significant  is  item order  effects  factor,  reported factor  to  (1985) c o n c l u d e d ,  speediness test  1 show s i x t e e n s t u d i e s r e p o r t i n g  non-significant  Dorans  that  of T a b l e  would  were made t o so  samples w i t h  first  year  i n samples  t h e r e f o r e samples w i t h a  a  college from  wider  the  (75)  variation  i n performance speed.  necessary  and t y p i c a l  enough  to f i n d  Table seem  an  factor  item order  with  significant probably  over  Eleven  effect  As w e l l ,  secondary  or e l e m e n t a r y  order a  effects,  factor  order  effects,  numbers,  a test  opposed  and t i m e ,  may  there  item order  students  not t o f i n i s h  students  t o be e f f e c t e d  but i t w i l l  a  involving  limits  significant  on t i m e  were  item  models r e q u i r e l a r g e as a f a c t o r  & Dorans,  or a power  test  is a correlation  r a t h e r than  by t h e i t e m o r d e r .  cause  time  of the  factors.  to c o r r e l a t i o n  in fact,  (Yen,  1984)  involve other  may,  item  three s t u d i e s  between t h e s p e e d i n e s s  effects  While  studies  limits  problem r e l a t e d  to causation.  item order  trait  i s c o n s i d e r e d a timed  i n v o l v e s a judgmental  effect,  a l a r g e number o f  indicated  1976; K i n g s t o n  and t h e i t e m o r d e r  time.  reported s i g n i f i c a n t  Finally,  relationship  size  have a  r e p o r t e d t h a t time  i n v o l v e d time  to  s t u d i e s show s u c h  students,  models  & Dawis,  The o b v i o u s  Whether  on s u c h  but s i n c e l a t e n t  they a l s o  1980; W h i t e l y  test  trait  related  probably  t e n out of f i f t e e n  but they a l s o  latent  and t h e sample  or c o n t e x t  limits  f o r some s t u d e n t s .  involving  will  out of f i f t e e n  pattern.  powerful  effect.  300 s t u d e n t s  have t i m e  became a  o f s t u d i e s t h a t were  but they a r e f a c t o r s  item order  also  subjects.  limits  1 shows t h a t t h e sample t y p e  t o be a f a c t o r s ,  A study  Time  as between some  causing  A strong  some  (76) correlation  c o u l d a l s o be  unexpectedly it  as  found  an  a side effect  reported  students  declared  that their  r e s e a r c h e r s who  who  test  found  related  test  t o be  s u b j e c t s d i d not to  i n v o l v e speeded  Clearly, factors  that  item order power.  So,  appropriate  effects  test,  size  may  Involve  Further,  controlling  such  the  have been shown by Finally,  effects.  400  would  the  who  their  their  is considered  of  item order  seem t o be  an  In a d d i t i o n , variance. wide also  v a r i a b l e s as  range.  t o be  r e s e a r c h must  a  r e s e a r c h may  gender, t e s t  research  the  statistical  from a p o p u l a t i o n with  item d i f f i c u l t y the  (1974)  I t seems t h a t  power.  confounding  rearrangement,  m o t i v a t i o n , and  Traub  the  1985).  also involve subject be  have  complete  study  a question  over  should  of a b i l i t i e s .  may  f o r f u r t h e r r e s e a r c h about  to ensure adequate  sampling  within-subject  their  hand,  considered  (Leary & Dorans,  a sample s i z e  Therefore,  error.  the  i s a need  may  and  they  and  other  order  d i d not  effect  test  the  item  dismiss  a result,  the  i s H a m b l e t o n and  i n f l u e n c e item order  effects  involve  from  as  On  who  chose to  However, s i n c e some o f  tests  there  item order  range  speeded.  item order  finish  and  limits;  that students  a power t e s t .  researchers  to complete  effect  example  f i n d an  of  effect  time  was  no  test.  to  their failed  to mention  expected  item order  of  chosen not One  a result  use  content, A l l of  potential a power  subject which  sources test  to  of avoid  problems w i t h t e s t scores that speed i n e s s .  are the r e s u l t s  of  chapter  III  Problem Statement  The whether to  of  Problem  main p r o b l e m t h a t or n o t  easiest  easiest  the  a test  with  was  addressed  the  items  i s more d i f f i c u l t  to h a r d e s t .  Two  than  tests  were u s e d .  i n the a s c e n d i n g  the  the  test  difficulty that  the  one,  addressed  directions  other  of these the by  used  A third ability  i n the  students  with  with  two  the  alter  other  the  had  size  the  sequence,  of t h e to  and  descending indicate  item  order  determine  variables.  i f the  factor,  different  r e s t r i c t i v e test  test  also  (1974)  group  whether  item order  booklet  (78)  are  in  with  d i f f e r e n t l y than  different  study.  used.  to determine  test  Traub  booklet  booklet directions  a control  was  performed two  the  Hambleton and  p r o b l e m was  the  from  test  i n the  p r o b l e m s were a d d r e s s e d  directions  students  hardest  However, s i n c e r e s e a r c h would  replicating  a significant  unrestrictive  One  was  problem of w i t h i n - s u b j e c t rearrangement  However, t o d e t e r m i n e fact  from  difficulty  questions arranged  c o n d i t i o n s can  several  influence For  was  sequence.  several  effect,  had  arranged  study  i f i t i s arranged  questions arranged other  in this  or n o t  high  low  ability  arrangements  formats.  A large  or  and  (79)  diverse  sample was  used  to ensure  an  ample  r a n g e of  ability  levels. Finally, latent  trait  classical  the  problem of whether  statistics  statistics  statistics  was  are  comparable  addressed  i n the a n a l y s i s  or n o t  of t h e  studies using  to s t u d i e s u s i n g  by u s i n g b o t h  types  of  data.  Rationale Forty years has  still  order  not  of r e s e a r c h  r e s o l v e d the  sequence w i l l  intended several  variables  factors  which might  item order One  the  the  taking  easy  test.  scores.  Allison  have been s u g g e s t e d  Traub  i n f l u e n c e the  able  not  the  This study  by  In  the  presence  is indicated  (1974).  to rearrange ones  first  In t h e i r  item  was  addition,  research  as  or a b s e n c e  of  the  order  students  T u c k , 1978;  & Thomas, 1 9 8 6 ) . when any  t o use  i t , then  the from  may  item order  research  they d i d  of the  R i n d l e r , 1980;  who  the  items  an  item's  test (Millman  Klimko,  1984;  Hambleton and  has  an  doing  location  using a  have t h i s  of  not  by  seem t o p o s s e s s  However, as  students  by  study  r e g a r d l e s s of  This prevented  1965;  concluded,  or  controversy.  s t r a t e g y t h a t some s t u d e n t s  & Bishop,  not  this  possible variable  allow students  in  test  arrangements  effect.  Hambleton and  only  item order  i s s u e o f whether  effect  to help c l a r i f y  into  Traub  s t r a t e g y are  effect  that is  (80) statistically their did,  significant.  T h e y may be p r e v e n t e d  o r t h e y may be h i n d e r e d  If  i n using their  l i m i t s which tend  back and f o r t h between  s t r a t e g y by t h e  to discourage  skipping  items.  t h i s w i t h i n - s u b j e c t rearrangement technique  significant  factor,  t h e n much o f t h e p r e v i o u s  i t e m o r d e r must be more s e r i o u s l y q u e s t i o n e d control this  order  using  s t r a t e g y by t h e t e s t b o o k l e t as Hambleton and Traub  presence of time  to  from  variable.  The v a l i d i t y  s t u d i e s c a n be q u e s t i o n e d  m e a s u r i n g an a d d i t i o n a l test-wiseness  isa  r e s e a r c h on for i t s failure  of previous  item  s i n c e t h e s t u d y may be  f a c t o r o f knowledge and usage o f a  strategy.  A s e c o n d v a r i a b l e i s t h e p o s s i b i l i t y t h a t when l o w achieving students  are involved i n the study,  these  students  may l a c k t h e s k i l l  t o e f f e c t i v e l y use t h e s t r a t e g y of  w i t h i n - s u b j e c t r e a r r a n g e m e n t a n d a r e t h e r e f o r e a f f e c t e d by the  item order.  historically items  held view that supports  Their  t h e need f o r a r r a n g i n g  frustrating  low a c h i e v i n g  f r u s t r a t i o n may be c a u s e d by a l a c k o f t h i s  some e x a m i n e e s c a n c o n t r o l t h e d e t r i m e n t a l  item order,  hard  to avoid  w i t h t h e more common  taking strategy. If  of  i s i n keeping  from easy t o hard  students. test  This  then  tests should  t o obtain the best r e s u l t s  effects  be a r r a n g e d  from easy to  from students  who a r e n o t  (81)  adept at are  omitting  unaware o f  the  hard q u e s t i o n s .  this strategy  within-subject  c o u l d be  rearrangement s t r a t e g y  Also,  students  t a u g h t how  to  who  use  the  to perform b e t t e r  on  tests. A third  variable  parameters are  c o u l d be  more s e n s i t i v e  classical  model s t a t i s t i c s .  classical  and  latent  item d i f f i c u l t y studies  i t e m o r d e r and  changes i n the  be  significant i f latent  If  latent  w i t h one latent  that  are  statistics.  with Dawis,  determined would  were u s e d  instead.  conclusions  inappropriate  trait  statistics  In a d d i t i o n , to  t r a i t model a s s u m p t i o n  i n d e p e n d e n c e must c e r t a i n l y be  the  statistics  particularly sensitive  latent  the  statistics  different  latent  to  (Whitely &  t h e n i t w o u l d be  used c l a s s i c a l  o r d e r changes then the local  indicate  both  r e a s o n why  I t n e e d s t o be  trait statistics  from a t e s t u s i n g  trait statistics  sensitive  trait  results with c l a s s i c a l  statistics,  compare r e s u l t s  one  item parameters  trait statistics  than c l a s s i c a l  (1980) found  t o be  latent  than  c o n t e x t were a s s o c i a t e d  K i n g s t o n & Dorans, 1984).  non-significant  item  item order e f f e c t s  trait statistics  if  trait  A l t h o u g h Yen  s i g n i f i c a n t changes i n the 1976;  to  latent  s e q u e n c e , t h i s c o u l d be  involving  reported that  that  questioned.  if item of  to  (82)  Hypotheses #1:  Hypothesis easy  t o hard  items  will  arranged  evident  from  A test  with t h e items  be e a s i e r  from  hard  the easy  than  mean, a h i g h e r a v e r a g e  average  parameter d i f f i c u l t y  indexes.  #2:  arrangement  A test  with  having  test  trait  w i t h i n - s u b j e c t rearrangement  difficult  tests  will and  such  classical  difficulty  indexes  Hypothesis  will  If3:  between t h e a b i l i t y test  lowest format,  format.  hard  There  will  classical  difficulty  parameter  on a n y h a r d  Figure  2 display  will  Students  index w i l l  students  test  this  item  directions  will  having  The l a t e n t  be more This  effect  a lower  trait  be an i n t e r a c t i o n  of the s t u d e n t ,  t o easy  difficulty  mean  model  be h i g h e r .  mean and c l a s s i c a l  difficulty ability  index.  a higher  restrictions.  be e v i d e n t w i t h t h e r e s t r i c t e d  difficulty  the  without  be  model  booklet  which r e s t r i c t than  will  item p - l e v e l  of l a t e n t  from  with the  This effect  of c l a s s i c a l  i n d e x e s , and a l o w e r  Hypothesis  t h e same t e s t  t o easy.  t o hard  arranged  with  the test  arrangement. be h i g h e s t . have t h e i r index,  index  lowest  have t h e  on t h e r e s t r i c t e d trait  On t h e o t h e r  model  hand, low  mean, t h e i r  arrangement.  hypothesis.  will  The l a t e n t  and t h e h i g h e s t  t o easy  a r r a n g e m e n t and  high a b i l i t y  difficulty  effect  lowest  b-value F i g u r e 1 and  (83) Figure Low A b i l i t y  Unrestricted  1 Students  Restricted  Easy to Hard  = high Mean = h i g h Mean p - l e v e l == h i g h p - l e v e l == h i g h b - v a l u e == low b - v a l u e == low  Hard to Easy  = low Mean = low Mean p - l e v e l == l o w p - l e v e l == low b - v a l u e == h i g h b - v a l u e == h i g h  Figure High A b i l i t y  Unrestricted  2 Students  Restricted  Easy to Hard  = h i g h Mean = high Mean p - l e v e l == h i g h p - l e v e l == h i g h b - v a l u e = low b - v a l u e = low  Hard to Easy  = low Mean = h i g h Mean p - l e v e l == h i g h p - l e v e l == low b - v a l u e == low b - v a l u e = h i g h  :  :  :  (84) Hypothesis statistics other. latent  will  #4:  indicate  Comparisons trait  correlation.  C l a s s i c a l and  b-value  similar  between  latent  patterns  classical  statistics  will  trait  test  of r e s u l t s  p-level  t o each  statistics  and  have a h i g h d e g r e e o f  Chapter  IV  Method Pesjtgn,  The Four  d e s i g n was a p o s t - t e s t  different  treatment  groups.  unrestrictive hard  test  b o o k l e t s were u s e d  were p l a c e d r a n d o m l y  other  easy  treatment  groups  mathematics t e s t  the  post-test.  and  ability  either  rated  level.  Item  restricted  with  next  for  the next  used  were  o r d e r was  of  one o f t h e booklets.  the easy  t o easy arrangement.  from  the highest mathematical  Ability  15%, a "3" was  used  format,  t o hard format  level  was  The 10% o f a  used f o r  25%, a "5" was  used  f o r t h e 10% o f s t u d e n t s  ability.  (85)  Test  "1" t o "6".  f o r the next  15%, and a "6" was  and a s  i n mathematics r e c e i v e d  f o r the next used  group  item order, t e s t either  or  students  s e r v e d as the treatment  or u n r e s t r i c t e d .  25%, a "4 was  The  the other t e s t  the lowest a b i l i t y  A "2" was  item order. the c o n t r o l  used  different  an e a s y t o  b o o k l e t s or i n t o  variables  or the hard  the  with  that  test  b y t h e t e a c h e r on a s c a l e  students "1".  either  itself  independent  arrangement was  into  t o hard  The  The  t o easy  design.  restrictive  and t h e y had e i t h e r  item order or a hard  group  as the four  The b o o k l e t s had e i t h e r  directions,  unrestricted,  only control  (86) The  dependent  variables  item d i f f i c u l t i e s . using  classical  using  latent  Item d i f f i c u l t y  p-level  trait  were t h e t e s t  statistics,  s c o r e s and t h e  was w i t h  comparisons  and w i t h  comparisons  b-value item parameters.  Subjects  Students classrooms suburban The of  i n grade  school d i s t r i c t s  the three  backgrounds  schools,  in a regular  excused absence participate  A grade  levels problems  i n two  B.C. were u s e d  eight  class  .  i n each  of socioeconomic  were  included  significant  the  sample.  enough t o n o t be  c l a s s r o o m , o r s t u d e n t s who  had an  was a d m i n i s t e r e d d i d n o t  Tasks  8 mathematics t e s t  chosen  1987).  There  was  was d e v e l o p e d u s i n g  i t e m s f r o m t h e 150  items  levels  (Robitaille &  had a r a n g e  w i t h t h e mean o f d i f f i c u l t y  no a t t e m p t  to r e f l e c t  i n the items chosen  since  40  i n t h e Second  Study of Mathematics  The d i f f i c u l t y  p = .13 t o p = .89  curriculum  so a v a r i e t y  on t h e d a y t h e t e s t  I . E . A. I n t e r n a t i o n a l  .492.  near Vancouver  different  i n the study.  I n s t r u m e n t and  Garden,  secondary schools  from e v e r y grade  and a b i l i t y  Students with l e a r n i n g  randomly  f r o m a p p r o x i m a t e l y 25  i n three d i f f e r e n t  590 s t u d e n t s were  enrolled  eight  the  from  levels at  provincial  individual  schools  (87) differed  In the t i m i n g of t h e i r  Instruction  of the  curriculum. Four  m a t h e m a t i c s t e s t b o o k l e t s were p r e p a r e d  40 r a n d o m l y c h o s e n q u e s t i o n s . had  the questions arranged  One  half  of the t e s t  of  In addition,  directed  One  which  f o r t h through printed  from hard t o  format  had  instructions  The o t h e r  format  a l l o w e d t h e s t u d e n t t o l o o k b a c k and  the t e s t b o o k l e t .  were c o l o u r c o d e d t o a l l o w t e s t  Both  booklet formats  instructions  were formats  t a k e r s and t e s t s u p e r v i s o r s  t o be s u r e t h a t t h e p r o p e r  unrestricted  i n the  had  w i t h one q u e s t i o n p e r p a g e , a n d b o t h b o o k l e t  the a b i l i t y  on one  t h e s t u d e n t s t o o n l y do t h e q u e s t i o n s  o r d e r t h a t t h e y were p r e s e n t e d . instructions  booklets  b o t h a r r a n g e m e n t s were p r e s e n t e d  two t e s t b o o k l e t f o r m a t s .  which  these  from easy t o hard  w h i l e t h e o t h e r two b o o k l e t s were a r r a n g e d easy.  using  r e s t r i c t e d or  were b e i n g f o l l o w e d .  I n o r d e r t o make a p r a c t i c a l a s s e s s m e n t o f s t u d e n t ' s ability,  t e a c h e r s were a s k e d  rating scale.  t o complete  T h i s s c a l e asked  c u r v e d i s t r i b u t i o n t o rank  a simple s i x point  t h e t e a c h e r t o use a normal  the s t u d e n t s ' mathematics  b a s e d on t h e t e a c h e r ' s p e r s o n a l j u d g e m e n t . A r a t i n g indicated a rating ability  the students with the lowest a b i l i t y of "6" i n d i c a t e d  i n math.  ability of "1"  i n m a t h , and  the students with the highest  (88) Procedure  All  four test  alternately assumed  to the  that this  essentially  students  the  test  i f going  students  who  sought  were t o l d  students remaining  used  grade.  fifty  with  since this forth  by  was  the  in a test  the  m i n u t e s of c l a s s  the  their  The  the were  experiment  b o o k l e t would the  to  help  students  experiment,  in determining  were t h e n  and  format  students  p a r t o f an  time  test  c o o p e r a t i o n of  In a d d i t i o n ,  teacher  Students  to put  Next,  restricted  p a r t o f an  i t was  an  Identification  the  tests.  t h a t even though  final  were a s k e d  i n the d i r e c t i o n s .  to take  m i g h t be  It is  produced  for completing  m o t i v a t i o n and  back and had  distributed  b o o k l e t answer s h e e t .  f o r those  to cooperate  results  Students  mathematics c l a s s  Both the was  were  i n each classroom.  were g i v e n d i r e c t i o n s  booklets.  see  on  restrictions  asked  students  random s a m p l e .  information  the  types  systematic d i s t r i b u t i o n  name, s c h o o l , and  students  booklet  the  the  allowed  the  to complete  the  test.  Analys i s The Anova that  analysis  format  of t h e  u s i n g the  test  and  with  S.P.S.S.-X p r o g r a m .  t h e means were n o r m a l l y  variance,  s c o r e s was  independent.  distributed, I t was  also  a 2 x 2 x 6  I t was  assumed  homogeneous i n  assumed  that  the  (89) factors  o f s c h o o l and c l a s s r o o m  factors  with  The repeated  equal  analysis  of the item d i f f i c u l t i e s  measures format  with each  test  means were n o r m a l l y independent.  school  item's  with  equal  The was  booklet  being  distributed,  program.  It  to equivalent  i t was assumed t h a t t h e  homogeneous  in variance,  assumed t h a t t h e f a c t o r s o f  were n o n - s i g n i f i c a n t  random  of  factors  means.  level  chosen  This  level  was c h o s e n  conditions  a Manova  was m e a s u r e d f o u r  presented  Further,  alpha  .05.  (1974).  difficulty  I t was a l s o  and c l a s s r o o m  was w i t h  u s i n g t h e S.P.S.S.-X  samples of the p o p u l a t i o n .  and  random  means.  was assumed t h a t e a c h times  were n o n - s i g n i f i c a n t  f o r the t e s t s  to r e p l i c a t e  significance the  statistical  e s t a b l i s h e d i n t h e s t u d y by H a m b l e t o n and  Traub  Chapter  V  Results  A mathematics t e s t to  590  grade e i g h t  presented  i n four d i f f e r e n t randomly g i v e n  booklets  had  other  hardest  the  two  items  i n two  directions  test  t o the  had  Further,  i t e m o r d e r , and four tests  the  other  booklets  administered  with Two  four  the  to hardest  arranged  each sequencing booklets.  booklet  were  the of  from e a s i e s t  subjects  were d e s i g n e d  was  questions  questions  test  which r e s t r i c t e d  forty  students.  the  different  questions  The  arranged  booklets  to e a s i e s t .  presented  The  forty  students.  booklets  the  of  from  format  One  was  booklet  from r e a r r a n g i n g  had  no  such  and  had the  restrictions.  t o answer t h e  following  of t e s t  result  questions:  1)  Does a l t e r i n g  i n means o f 2)  Do  the  effects 3)  order  on  the  the  Specifically,  test  rearrangement  an  the  low  s c o r e whenever t h e y  of  booklet test  changes  interaction  item order do  items  i n changes  scores?  a s s o c i a t e d with  Is t h e i r  ability,  test  directions  within-subject  the  effect  and  the  ability  items  (90)  result  i n item  test  booklet  in greater  students format?  have t h e i r  which begins  the  order?  between a  students  have a t e s t  which r e s t r i c t  with  lowest hard  (91) i t e m s , a n d do h i g h a b i l i t y when t h e t e s t b e g i n s allowed 4)  students  w i t h hard  i t e m s a n d when t h e y a r e n o t  t o a l t e r the item d i f f i c u l t y  Are any changes i n p - l e v e l  similar Main  o n l y have l o w s c o r e s  t o changes i n b-value  sequence?  difficulty  statistics  item parameter s t a t i s t i c s ?  Effects The  found  means o f t h e two d i f f e r e n t  i t e m s e q u e n c e s were  t o be s i g n i f i c a n t l y d i f f e r e n t  (p < . 0 0 1 ) .  s t u d e n t s who had a n exam w i t h a n e a s y t o h a r d  The 297  s e q u e n c e had  an a v e r a g e s c o r e on t h e 40 i t e m t e s t o f 18.56.  The 293  s t u d e n t s who had a t e s t w h i c h b e g a n w i t h h a r d q u e s t i o n s had  only  a n a v e r a g e s c o r e o f 15.90. The  found  means o f t h e two d i f f e r e n t t e s t  t o be s i g n i f i c a n t l y d i f f e r e n t .  the u n r e s t r i c t e d 17.58.  test  format  formats  were n o t  The 296 s t u d e n t s  with  b o o k l e t had an a v e r a g e s c o r e o f  On t h e o t h e r h a n d , t h e 294 s t u d e n t s w i t h t h e  restricted  s e q u e n c e d i d a b o u t t h e same w i t h a n a v e r a g e  score  o£ 16.89. The  four formats  reliability.  demonstrated a high l e v e l of  C r o n b a c h ' s c o e f f i c i e n t a l p h a was  for each format.  The f i r s t  sequence and u n r e s t r i c t e d .845.  The s e c o n d f o r m a t  format  calculated  w i t h an e a s y t o h a r d  directions  had a r e l i a b i l i t y o f  w i t h a hard  t o e a s y s e q u e n c e and  (92) unrestricted  directions  third  w i t h an  format  directions format  had  The were  had  found  to easy  Table  of  a reliability  of  t o be  2.  .829.  The  restricted the  fourth  restricted  .835. teacher-rated a b i l i t y  between t e a c h e r r a t i n g  of  .855.  Finally  significantly different  coefficient  of  s e q u e n c e and  s e q u e n c e and  s i g n i f i c a n t l y correlated  correlation in  t o hard  means o f t h e d i f f e r e n t  correlation was  easy  a reliability  a reliability  with a hard  directions  had  (p < .6340.  and .001) The  (p <  the  .001).  student's  with a results  levels The score  Pearson are  presented  (93)  Table Test  Level  Means and Sample S i z e s  Ability  Label  1  Lowest  2  Next  3  of Student A b i l i t y  Mean  Levels  n_  11.27  44  15%  11.76  90  Next  25%  14.96  178  4  Next  25%  18.92  112  5  Next  15%  21.63  115  6  Highest  26.41  51  17.37  590  Total  10%  2  10%  (94)  Interactions T h e r e were no s i g n i f i c a n t ability  students  receive  a lower s c o r e  questions  than  associated  interaction  Low  d i d n o t seem t o be a n y more l i k e l y t o on a t e s t  d i d high a b i l i t y  with  effects.  w h i c h began w i t h students.  changes of item order  hard  The e f f e c t s  effected  a l l ability  groups e q u a l l y .  As  well,  effects. overall  directions  So, i n a d d i t i o n main e f f e c t  directions, receive  test  to the fact  scores  interaction  t h a t t h e r e was no  associated with d i f f e r e n t  different  higher  d i d no have a n y  test  ability  g r o u p s were n o t more l i k e l y t o  i f they  had a d i f f e r e n t  type  of t e s t  d irect ion.  The presented  results  of the 2 x 2 x 6  i n Table  3.  analysis  of variance are  (95) Table  3  Summary o f A n a l y s i s o f V a r i a n c e Ability  ( A b ) , Item O r d e r  ( O r ) , and T e s t  Sum o f Source  of Test  S c o r e s by  Directions(Dir)  Mean  Squares  DF  Main  12743 .090  7  1820 . 441  65 .715  <  .001  Ability  11629 .609  5  2325 .922  83 .962  <  .001  694 .458  1  694 .458  25 .069  <  . 001  32 .034  1  32 .034  1 .156  .283  255 .123  5  51 .025  1 . 842  .103  Ab x D i r  80 .120  5  16 .024  . 578  . 717  Or x D i r  6 .453  1  6 .453  .233  .630  179 .829  5  35 .966  1 . 298  . 263  23  576 .065  20 .795  Order Directions Ab x Or  Ab x Or x D i r  Square  Explained  13249 . 491  Res i d u a l  15679 .289  566  27 .702  Total  28928 . 780  589  49 .155  F  Prob.  <  .001  (96) Item Qjf f jciUtiles Due t o t h e c l o s e m a t h e m a t i c a l r e l a t i o n s h i p b e t w e e n t h e mean a n d p - l e v e l ,  l t i s not s u r p r i s i n g  that a  significant  d i f f e r e n c e was a l s o f o u n d b e t w e e n t h e mean o f t h e p - l e v e l s of each t e s t . repeated  A m u l t i v a r i a t e a n a l y s i s of variance f o r  m e a s u r e s was u s e d t o d e t e r m i n e i f t h e r e was a  s i g n i f i c a n t d i f f e r e n c e between t h e i t e m d i f f i c u l t i e s test.  E a c h i t e m had p - l e v e l v a l u e s  s a m p l e s were assumed primarily a result  The  t o be e q u i v a l e n t w i t h t h e v a r i a n c e  of t h e e x p l a i n e d v a r i a n c e between t h e  t e s t s or a b i l i t y  l e v e l s as r e p o r t e d  results  3.  i n Table  from four t e s t s .  on e a c h  i n the previous  Anova  As e x p e c t e d , a s i g n i f i c a n t d i f f e r e n c e was f o u n d b e t w e e n the d i f f e r e n t T a b l e 4.  item d i f f i c u l t i e s  o f e a c h t e s t a s shown i n  (97)  Table 4 Summary o f M u l t i v a r i a t e A n a l y s i s  of Variance  of Test  Item  P-levels  Sum o f Source  Squares  Mean DF  P-levels  .10  3  Within  .18  57  Cells  Square  .03 .004  F  10.26  Prob.  < .001  (98) The  same t y p e  was  used.  The  the  Microcat  standardized. also  of a n a l y s i s w i t h  rasch  Table  item d i f f i c u l t i e s  item  parameters  i t e m p a r a m e t e r s were e s t i m a t e d  T e s t i n g System  have a s i m i l a r  rasch  (1988) w i t h  5 shows t h a t t h e level  i n Table  ability  rasch  of s i g n i f i c a n c e  item  as  the  using  levels parameters classical  4. Table  5  Summary o f M u l t i v a r i a t e A n a l y s i s of V a r i a n c e  of T e s t  Item  B-values  Sum  of  Mean  Source  Squares  DF  Square  F  B-values  3.14  3  1.05  9.09  Within  6.56  57  Cells  0.12  Prob.  <  .001  (99)  The  similarity  the  means  for  each  p-levels changes  of r e s u l t s  of the t e s t s c o r e s , t e s t as g i v e n  i s also  a p p a r e n t by  the p-values,  i n T a b l e 6.  and t h e b - l e v e l s  The c h a n g e s  and t h e t e s t means a r e i n v e r s e l y  comparing  i n the  related  to the  i n the b-values. Table 6  Test  Format  P-level  Means. B - v a l u e Means, and S c o r e  Format  Item  Test  Number  Order  Type  P-level Mean  B-value Mean  Means  Score Mean  1  E a s y - Hard  Unr.  .474  .128  18.946  3  E a s y - Hard  Res.  .454  .295  18.169  2  Hard  - Easy  Unr.  .405  .493  16.190  4  Hard  - Easy  Res.  .390  .652  15.603  Note U n r . = u n r e s t r i c t e d directions  directions;  Res. = r e s t r i c t e d  (100) The  Pearson  demonstrates p-levels used  c o r r e l a t i o n c o e f f i c i e n t s i n Table 7 also  a s t r o n g r e l a t i o n s h i p between an  and i t s b - v a l u e s r e g a r d l e s s  of which  item's test  format i s  with the item. Table  Pearson  Correlation  7  Coefficients  o f P - l e v e l s and  B-values  to Hard (EH), Hard to Easy (HE), R e s t r i c t e d  fQE E a s y  Unrestricted  P-level  (U) T e s t  Test  (R) and  Formats  Formats  B-value Test  Formats  EH, U  HE, U  EH, R  HE, R  EH,  U  -.9652  -.9385  -.8796  -.9125  HE,  U  -.9001  -.9050  -.8558  -.8820  EH,  R  -.9225  -.9350  -.8818  -.9099  HE,  R  -.8141  -.8294  -.7697  -.8181  Note A l l c o r r e l a t i o n s  a r e s i g n i f i c a n t a t p < .001.  (101)  Finally,  Table 8 further  between c l a s s i c a l theta  and l a t e n t  values calculated  demonstrates  the  similarities  trait statistics.  T a b l e 8 uses  from b - v a l u e s s t a n d a r d i z e d f o r  difficulty. Table 8 Summary o f A n a l y s i s Ability  o f V a r i a n c e o f T h e t a V a l u e s by  ( A b ) . Item O r d e r  ( O r ) , and T e s t  Sum o f  Directions (Dir)  Mean  Source  Squares  DF  Main  239, ,955  7  34 ,279 .  55., 127  <  .001  Ability  217. .250  5  43.,450  69..875  <  .001  14 .997 .  1  14 ,997 .  24 .117 ,  <  .001  .212  1  .212  Order Directions  Square  F  Prob.  , 341  . 559  Ab  x Or  5..052  5  1..010  1,.625  . 151  Ab  x Dir  2,.614  5  5..523  .841  .521  Or  x Dir  .002  1  .002  .004  .951  Ab  x Or x D i r  4 .160 ,  5  .832  1,.338  .247  Explained  251 .773  23  10 .947  17 .604  Residual  351,.954  566  .622  Total  603 .727  589  1 .025  <  .001  Chapter Summary and  Purpose of  The  This of  test  Further, were a b l e  was  has  this  an  an  study  effect tried  rearranging  the  in  the  booklet.  an  effect  on  test  years.  trait  statistics,  also  scores  the  the  i f the  performance  effects  by p e r s o n a l l y  as  presented  has with  issue  the  students.  such  or n o t been an the  by  the  the  area  can  of r e s e a r c h  i n c r e a s e d usage o f  of c o n t e x t  effect  researcher  item order  effects  become more o f a c o n c e r n .  examined  of  sequence  i f some e x a m i n e e s  Whether  Recently,  i n d e p e n d e n c e has  on  to determine  to determine  item order  forty  study  attempt  t o m i t i g a t e any  test  Conclusions  Study  study  items  VI  of  item order  As  and  have  for  latent local  a result  on  latent  this trait  stat1st ics.  Four d i f f e r e n t randomly to the  items  and  the  sequence booklet  590  booklets  grade e i g h t  arranged  other  test  two  f.rom h a r d  math s t u d e n t s .  i n sequence booklets  had  to easy.  which a l l o w e d  the  were used  arranged  items  students  was  d i d not  a l l o w such  booklet  in  the  one  order  between  of each sequence  within-subject  (102)  forth  had  questions,  o f s e q u e n c e s had  to rearrange  questions, that  booklets  the  Both types  one  Two  to hard  s k i p p i n g back and  there  given  from easy  i t e m p r e s e n t a t i o n by and  and  rearrangement.  type  of  (103)  The  results  of the study  hypotheses  about  the e f f e c t s  associated  with i t .  supported  several  of the  o f i t e m o r d e r and t h e f a c t o r s  Sequence The  sequence of the t e s t  performance  of the s t u d e n t s .  w i t h t h e items significantly to  easy  easy  other  taking  who had t h e h a r d  had a mean s c o r e o f 18.6 a s compared t o the hard  t o easy  test  and had a mean  The mean o f t h e s t u d e n t s who took  frustration  than  the hard  s t u d e n t s who t o o k t h e  scores  exercised  is  possible  that  measure o f t h e s t u d e n t s  and d i s c o u r a g e m e n t  t h e common c o n c e r n  t o o many h a r d  be  test  had a  The s t u d e n t s who t o o k t h e  t h e r e was no a c t u a l  the t e s t ,  i s clear  lower  t o hard  the t e s t  test.  theoretical  It  who took  the students  exam had s c o r e s 7% l o w e r  Although  with  easy  (p <.001).  who t o o k  o f 15.9.  easy  from  can a f f e c t the  Students  h i g h e r mean t h a n  tests  the s t u d e n t s  to  arranged  arrangement  t o hard  score  items  about  while  they  were  beginning a  test  q u e s t i o n s may have some  a hard  t o easy  f o r the students.  justification.  arrangement can r e s u l t i n As a r e s u l t ,  caution should  when d e v e l o p i n g two forms o f t h e same t e s t . to c r e a t e formats  characteristics  with  significantly  even though t h e items  It  different  are i d e n t i c a l .  (104) Directions  The not  difference  significant.  significance was lower  than  and  a comparable  In comparing  The s t u d e n t s  test  with with  the o v e r a l l  who were a l l o w e d  t o go back and f o r t h  17.6.  of d i r e c t i o n s  However, t h e r e was a t r e n d  s i n c e mean o f a t e s t  directions. students  between t h e two t y p e s  toward  restricted  directions  unrestricted test  scores,  t o do t h e q u e s t i o n s  who c o u l d n o t r e a r r a n g e  those  i n any order  between t h e q u e s t i o n s  exam had a mean s c o r e t h a t  was  had a mean o f  the order  of the  was n o t s i g n i f i c a n t l y lower a t  16.9. The  effort  t o prevent  students  from  r e a r r a n g i n g the  Item o r d e r  d o e s n o t seem t o g r e a t l y  improve  of  a significant  effect.  finding  for  the widespread  item order  and e f f e c t i v e  use o f t h i s  s t r a t e g y was n o t c l e a r l y d e m o n s t r a t e d . study that of  supports there  this  factor  trends  s t u d i e s would  evidence  test-wiseness  Therefore,  i n the data,  problematical  variable  Traub  suggested.  t o doubt t h a t  have had d i f f e r e n t  had been c o n t r o l l e d .  indicated  (1974)  The  this  t h e c o n c l u s i o n s o f A l l i s o n and Thomas  i s n o t enough e v i d e n c e  item order  the l i k e l i h o o d  this  the m a j o r i t y findings i f  However, i n l i g h t factor  i n some s i t u a t i o n s  (1986)  of the  may be a a s Hambleton and  (105)  Ability The  main e f f e c t  significant. well  The  associated with  fact  that teachers  their  students  will  unexpected  finding.  As  drawn f r o m t h i s  do  on  ability  levels  are able  to p r e d i c t  a mathematics t e s t  a result,  no  was how  i s not  conclusions w i l l  an  be  finding.  interactions A more s i g n i f i c a n t significant including  fact  interactions  ability.  between any  Students  limited  mathematical a b i l i t y  and  directions  the  were c o n s i d e r e d calls  into  concern appear helps  over  feelings  one  the  low  low  justification  to hard  based  Another interaction levels. the  The  on  the  The  easy  student So  f i n d i n g s of R i n d l e r  to  the  have  sequence  students  for  order  any  while  the  This the  does  a concern  not  test  for  there  items  the i s no  from  easy  study.  difference involves  directions  significant  who  more t h a n i t  i s admirable,  of t h i s  l a c k of s i g n i f i c a n t  l a c k o f any  the  to hard  for arranging  test  by  justifications  student.  between t h e  factors  mathematical a b i l i t y .  of the  results  no  were c o n s i d e r e d  achieving students  special  the  were e f f e c t e d  ability  high a b i l i t y of  who  of  same d e g r e e as  t o have h i g h  item order.  to help the  t o the  question  i s t h a t t h e r e were  and  the  the  ability  differences contradicts  (1980) t h a t a l l a b i l i t y  levels  (106) possess but  the t e s t - w i s e n e s s  they  use i t w i t h v a r y i n g d e g r e e s  demonstrated study.  s t r a t e g y of s k i p p i n g q u e s t i o n s ,  by t h e complex  interactions  However, t h e r e s u l t s  preclude  the value  Latent  of these  to learn  s t u d i e s do n o t s t r a t e g i e s to  t o use t h e i t e m  Trait  their  analysis  found  significant  parameters.  Only  one s t u d y  found  t h e same s i g n i f i c a n t  types  of s t a t i s t i c s .  this  effect  levels  study,  in their  based  were  and b o t h  difficulty  found  context  and found  would  demonstrated  classical  possible  that  based  have  difficulty both  levels  and l a t e n t  t h e same  statistics.  latent  effect  classical  trait  correlated  I t c a n be  trait changes i n  results  Further,  both  significant  from  similar  study  using  t o the item o r d e r .  found  i f some p r e v i o u s  latent  and t h a t  t o be s i g n i f i c a n t l y  a significant  probably  have a l l  of item order  t h a t t h e s t u d i e s which used  statistics  effects  statistics,  effect  a s s o c i a t e d w i t h changes  concluded  statistics in  ( Y e n , 1980) used  and c l a s s i c a l  Both c l a s s i c a l  trait  or c o n t e x t  differences  statistics  difficulty  latent  of item order  trait  used  in Rindler's  strategy effectively.  S t u d i e s w h i c h have used  in  found  of teaching test-wiseness  possibly help students rearrangement  of s u c c e s s as  had t h e y  i t i s also  based  s t u d i e s had  (107)  used  larger  similar  samples,  then  to the latent  The  assumption  supported  trait  difficulty  parameters  parameters  of preceding  since  a test  assumed easier  tests  results  based  of l o c a l  by t h e r e s u l t s  when c o m p a r i n g  their  may have  studies.  I n d e p e n d e n c e c a n n o t be  of t h i s  study.  were a f f e c t e d items.  Latent  t o be t h e p a r a l l e l  trait  by t h e d i f f i c u l t y  C a u t i o n must be e x e r c i s e d  by u s i n g t h e l a t e n t  which begins  been  with harder to a test  trait  statistics  q u e s t i o n s cannot which begins  be  with  questions.  Limitations The  c o n c l u s i o n s of t h i s  comparisons one  between a t e s t  arranged  from  random o r s p i r a l  hard  s t u d y must be l i m i t e d t o  arranged  t o easy.  from  Other  were n o t i n c l u d e d .  easy  t o h a r d and  formats  such as  I t i s open t o  conjecture  a n d f u t u r e r e s e a r c h i f a random a r r a n g e m e n t  began w i t h  primarily  lower  s c o r e s than  primarily  easy  A second  hard  q u e s t i o n s would  a random a r r a n g e m e n t  have  that  with  questions.  area  of l i m i t a t i o n  involves the content  T h i s s t u d y c o n f i r m s many o f t h e r e s u l t s  studies  that  Yen,  significantly  began  tests.  Traub,  that  used  1974; F e l d t  quantitative  type  tests  & F o r s y t h , 1974; Towle  of the  found  with  (Hambleton & & Merrill,  1980; P l a k e , A n s o r g e e t a l . 1982; K i n g s t o n  1975;  & Dorans,  (108) 1985).  However, t h i s  mathematics t e s t s . difficulty  of an  statistical actual  One  difficulty  encountering  the  question,  the  predictor  o f how  item.  be  statistically  it  incorrectly,  by  the  difficult effects  In t h e  the  but  other  due  to the  a result,  limitation  s t u d y were b a s e d  though  the  and  teacher with  of t h i s  measure o£  the  mathematical  rating  the study  student  individual  may  be  perceives  e f f e c t i v e n e s s of  not  of  ability  ability.  the  result  study.  of t e a c h e r  scale  had  differ  question  the  i n the  of t h e  on a t e a c h e r  with  may  statistically  result  The  same  questions.  definition ability  rating  system  behaviour,  a significant  .6340  from  a study  a more v a l i d  (p <  and  student  Even  scores  of  levels  judgement.  mathematics t e s t may  a good  b e c a u s e most s u b j e c t s answer  in this  errors  the  mathematics  level  s t r o n g l y i n f l u e n c e d by c l a s s r o o m  correlation results  used  of  hand, a s c i e n c e q u e s t i o n  i s the  used  personality,  The  estimate  of d i f f i c u l t m a t h e m a t i c s  ability  be  the  individual  a series  mathematical  would  the  i t i s p e r c e i v e d as a v e r y e a s y  as a s e r i e s  in this  an  of a  difficulty  s c i e n c e q u e s t i o n s may  A third  o n l y be  case  very d i f f i c u l t  respondents As  may  d i f f i c u l t each On  i s that  highly subjective.  i s p e r c e i v e d by  statistical  t o be.  distractors.  be  level  that  o n l y g e n e r a l i z e to  p o s s i b l e reason  i t e m may  difficulty  question  s t u d y may  .001),  the  which uses a criterion  of  (109)  The  issue of d i f f i c u l t y  Items used are  in this  identical  study  standard levels if  to other  a r e based  difficulty  out,  studies.  of items  of t h i s  compare w i t h  many s t u d i e s do n o t g i v e The d e g r e e  difficulty  levels  for  research.  future  A definite  secondary  of t h i s  i t i s not c l e a r the r e s u l t s of  enrolled  of student  changes  However, t h e r e s u l t s  students  effect.  item  i s a subject  i s that the r e s u l t s  i n the i n t e r m e d i a t e or  ability  levels.  i n the scores  of t h i s  to conclude  This study  where some  i n item order  study  do c a l l  that  into  previous  students. question  r e s e a r c h which  item order  may  do n o t r e s u l t  of c o l l e g e  some o f t h e g e n e r a l i z a t i o n s o f p r e v i o u s college  their  programs o f C a n a d i a n p u b l i c s c h o o l s w i t h a  has i n d i c a t e d , c h a n g e s  significant  (1974) p o i n t e d  effects  study  be a p p l i c a b l e t o a c o l l e g e s e t t i n g  research  of p  t h e mean and v a r i a t i o n o f  i n f l u e n c e item order  limitation  school  wide d i v e r s i t y  an  The r a n g e  i n f o r m a t i o n about  t o which  only generalize to c h i l d r e n  in  o f .49 w i t h a  s t u d i e s s i n c e , a s Hambleton and T r a u b  difficulty.  not  p value  Unfortunately  study  which  whose p r e - t e s t e d  t h a t mean o f .21.  .13 t o .89.  levels  The c o n c l u s i o n s o f t h i s  had an a v e r a g e  d e v i a t i o n around  was f r o m  limitation.  do n o t have d i f f i c u l t y  on a s e r i e s  levels  the r e s u l t s  other  study  i n v o l v e s another  used  d o e s n o t have  (110) Conclusions restricted  latent  by t h e s m a l l  590 s t u d e n t s taking  about  in total,  each t e s t .  sample there  The l a t e n t  that p a r t i c u l a r  test  Generalizability interaction study. of  may  of s e l e c t i o n  This  limitation  s t a t i s t i c s are  size.  Although  were o n l y a b o u t  f o r m a t were e s t a b l i s h e d j u s t given  trait  trait with  study  (1963).  While  147  parameters  the students  were  students  f o r each who  were  a l s o be l i m i t e d process  by t h e p o s s i b l e  and t h e t e s t s  used  in this  i s o u t l i n e d a s a p o s s i b l e weakness  i n the  a r e h o p e f u l l y r e p r e s e n t a t i v e of the t y p i c a l  junior  involved  the three  by C a m p b e l l and involved  secondary school,  o£ t h e f i v e  the  study.  did  so because  schools  they  felt  asked  to p a r t i c i p a t e i n  that the d i s t r i c t ' s  time.  participating  schools  Instructional  t i m e was a h i n d r a n c e  I t should  be n o t e d  d i d not f e e l  labour  shortened that  two o f t h e shortened  to t h e i r  participation. participation  seem t o be u n r e l a t e d  t o t h e f a c t o r s under s t u d y ,  concluded  probably  was  their  t h a t the  s i n c e the f a c t o r s i n v o l v e d with  there  three  which d e c l i n e d t o p a r t i c i p a t e  had a l r e a d y s i g n i f i c a n t l y  that  schools  were t h e o n l y  which agreed  instructional  Therefore,  the three  F o r one, t h e y  The two s c h o o l s  difficulties  schools  i t i s possible that  were a t y p i c a l .  out  test  booklet.  p o s t - t e s t o n l y c o n t r o l group d e s i g n s  Stanley  there  not any  i t c a n be  interaction  (Ill) between t h e s e l e c t i o n p r o c e s s and t h e t e s t s used  i nthe  study. C a m p b e l l and S t a n l e y a l s o p o i n t o u t t h a t t h e d e s i g n i s limited  by t h e p o s s i b l e e f f e c t  of r e a c t i v e elements.  c e r t a i n e x t e n t s t u d e n t s were a f f e c t e d of  the t e s t i n g  procedure.  the u n r e s t r i c t e d  by t h e unusual  t e s t b o o k l e t s may h a v e r e a c t e d more  pleasure a t having received the u n r e s t r i c t e d  not rearrange  the item order.  seemed t o be u n c o o p e r a t i v e  While  teacher attempts  d i r e c t i o n s a n d do t h e i r  best.  This  c o n c l u s i o n s about the e f f e c t  the m a j o r i t y of i n each s c h o o l  Those  students  t o have them f o l l o w t h e limits  the accuracy of  of the t e s t d i r e c t i o n s .  t h e o t h e r h a n d , t h e i t e m o r d e r e f f e c t s were l e s s be i n f l u e n c e d b y s u c h a f a c t o r  easy  I n f a c t , some t e a c h e r s s a i d  s e q u e n c e was more l i k e l y  l i k e l y to  s i n c e t h e s t u d e n t s were n o t  t h a t t h e t e s t s were a l s o p r e p a r e d  sequences.  limited  s i n c e t h e y assumed t h a t i t was  j u s t some t y p e o f a n e x p e r i m e n t .  passively resisted  told  booklet.  with the d i r e c t i o n s  s t u d e n t s were c o o p e r a t i v e , a f e w s t u d e n t s  any  test  t o d i r e c t i o n s s i n c e some  s t u d e n t s may n o t have f u l l y c o o p e r a t e d  really  expressed  t h e t e s t i n g s i t u a t i o n may have a l s o  the p o s s i b l e e f f e c t r e l a t e d  to  nature  F o r o n e , s t u d e n t s who r e c e i v e d  p o s i t i v e l y t o t h e t e s t i n g s i t u a t i o n s i n c e some  Unfortunately,  To a  t o cause  with different  item  t h a t the hard t o uncooperative  On  (112)  behaviour the  r a t h e r than  item order  the u n c o o p e r a t i v e  the c o n c l u s i o n s of t h i s  this 590  to speeded  test  be  students  easily  limits  test  were ample and d i d not  as  "not  w i t h i n the  complete  the  questions  were n o t  reached  distinction.  I t i s not  the  in this  trying  reached the  booklet  the  questions reached  on  rest  test,  of t h e  and  test  Most  of  students  allotted. that  and  have testing  The  the  time  there  are  there are  and  items  technically  r a t h e r than  only omitted  t o not For  hard  test.  As  a result  e r r o n e o u s l y be  omitted.  As  another  most  difficult  for tests  of  percentage  student  to easy  room and  the  is a  have a s m a l l one  because  because  possible  example,  a c r o s s the  can  test  by some s t u d e n t s  study  of the  his test  i s a power t e s t  i s a speeded  q u e s t i o n of the  threw h i s t e s t  complete  the  realistically  questions.  first  limitations  time  way  made t o  Nonetheless,  test  or whether a t e s t  not  i n any  was  stated  or n o t a t e s t  difficult  of  be  study,  reached".  students completed  used  test  generous.  However, w h e t h e r  type  of t h i s  s c h o o l system.  were o m i t t e d a t t h e end  classified  all  w i t h i n the  public  the  not  E v e r y attempt  a d m i n i s t e r e d the  s t u d e n t s who that  tests.  i n the  limitations  study should  a power t e s t  finished  t e a c h e r s who  limiting  effects.  N e v e r t h e l e s s , d e s p i t e the  limited  behaviour  after  sequence  refused to thirty-nine  s c o r e d as example,  not students  (113)  were o b s e r v e d  t o use a t e s t t a k i n g s t r a t e g y o f d o i n g t h e  q u e s t i o n s a t t h e b e g i n n i n g a n d t h e end f i r s t in the middle not  were l e f t  have enough t i m e  q u e s t i o n s would then between n o t reached since students  until  Students  be s c o r e d a s o m i t t e d . and o m i t t e d  who t o o k  questions  the easy t o hard  questions  students d i d not reached  The d i f f e r e n c e i s a l s o not c l e a r  format  had 1.7% o f  but the students w i t h t h e hard  had . 9 % o f t h e i r  w i t h hard  I f these  t o complete the t e s t t h e i r  t h e i r questions not reached, to easy format  last.  while  questions  not reached.  q u e s t i o n s a t t h e end o f t h e t e s t  omitted  more q u e s t i o n s a t t h e end o f t h e t e s t w h i c h i n c r e a s e s t h e number o f t e c h n i c a l l y n o t r e a c h e d questions  on a power t e s t  questions.  i s very d i f f e r e n t  reaching questions  on a s p e e d e d t e s t ,  make a s t a t i s t i c a l  distinction  c o n d i t i o n s of t h i s study. t e s t s used choosing  While from not  i t i s not accurate t o  b e t w e e n t h e two u n d e r t h e  F o r a l l i n t e n t s and p u r p o s e s t h e  i n t h i s s t u d y were power t e s t s w i t h some  t o omit  omitting  students  questions.  Implicat ions Caution should measurement f i e l d circumstances, to  influence  taken  still  about  be e x p r e s s e d  by w r i t e r s  item sequencing.  i t i s possible  the s t a t i s t i c s  Under  f o r the context  of the items.  i n the development of p a r a l l e l  i nthe  certain  of the items  C a r e must be  forms of a t e s t t o  (114) prevent  significant  differences Future  i n the item  sequencing.  Research  Many a r e a s one,  d i f f e r e n c e s i n s c o r e s as a r e s u l t of  remain as s u b j e c t s  the d i f f e r e n t i a t i o n  between  for future research.  the s i x d i f f e r e n t  g r o u p s c o u l d be t h e b a s i s o f f u r t h e r r e s e a r c h . could  be c l a s s i f i e d  pre-test  t h a t measures  achievement, used  into d i f f e r e n t  or both.  to identify  analysis  The s c o r e s  ability,  from those  between a b i l i t y  on a  mathematical tests  more o r fewer g r o u p s a s needed  of any i n t e r a c t i o n  ability  Students  groups based  intellectual  For  could  be  f o r the  level  and  other  factors.  A second area significant and  compared  if  item d i f f i c u l t y .  difficulty  with  correlation. compare  of t e s t s ,  statistical Different  the c o r r e l a t i o n  the types  statistical greatest  correlation  item order  Students and t h o s e  i f there  is a  difficulty  could r a t e the ratings could  be  r a t i n g s to determine the  subject areas  could  between c o n t e n t  of questions  Variations examined.  i s t o determine  d i f f e r e n c e between s u b j e c t i v e i t e m  statistical  subjective  of r e s e a r c h  with  areas  the h i g h e s t  are the content  be used t o to determine  s u b j e c t i v e and  areas  with the  effects.  i n i t e m and t e s t  F o r one, t h e number  difficulty  of d i f f i c u l t  c o u l d a l s o be items  a t the  (115) beginning  o f a t e s t c o u l d be v a r i e d t o d e t e r m i n e t h e maximum  number o f d i f f i c u l t students  without  items  t h a t c o u l d be t o l e r a t e d b y  resulting  i n lower  test scores.  o f t h e mean a n d r a n g e o f i t e m d i f f i c u l t y evidence  to the s e n s i t i v i t y  of students  Variations  would a l s o to item  give  difficulty.  (116)  References Ahmann, J . S., g r o w t h (2nd  & G l o c k , M. D. ( 1 9 6 3 ) . E v a l u a t i n g e d . ) . B o s t o n : A l l y n & Bacon.  pupil  A l l i s o n , D. E . ( 1 9 8 4 ) . The e f f e c t o f i t e m - d i f f i c u l t y s e q u e n c e , i n t e l l i g e n c e , and sex on t e s t p e r f o r m a n c e , r e l i a b i l i t y , and i t e m d i f f i c u l t y and d i s c r i m i n a t i o n . Measurement and E v a l u a t i o n i n G u i d a n c e , i 6 , 2 1 1 - 2 1 7 . A l l i s o n , D. E . , & Thomas, D. C. ( 1 9 8 6 ) . I t e m - d i f f i c u l t y sequence i n achievement e x a m i n a t i o n s : Examinees' p r e f e r e n c e s and t e s t - t a k i n g s t r a t e g i e s . P s y c h o l o g i c a l R e p o r t s , 59, 867-870. B e r g e r , V. (1969). anxiety Journal  F., Munz, D. C , Smouse, A. D., & A n g e l i n o , H. The e f f e c t s of i t e m d i f f i c u l t y s e q u e n c i n g and r e a c t i o n t y p e on a p t i t u d e t e s t p e r f o r m a n c e . o f P s y c h o l o g y . 71, 253-258.  B r e n n e r , M. H. ( 1 9 6 4 ) . T e s t d i f f i c u l t y , r e l i a b i l i t y , and d i s c r i m i n a t i o n as f u n c t i o n s of i t e m d i f f i c u l t y o r d e r . J o u r n a l o f A p p l i e d P s y c h o l o g y . 48, 98-100. C a m p b e l l , D. T., & S t a n l e y , J . C. and q u a s i - e x p e r i m e n t a l designs Houghton M i f f l i n .  (1963) E x p e r i m e n t a l for research. Boston:  C r o n b a c h , L. J . ( 1 9 4 6 ) . R e s p o n s e s e t s and t e s t validity. E d u c a t i o n a l and P s y c h o l o g i c a l Measurement. 6, 475-494. C r o n b a c h , L. J . ( 1 9 5 0 ) . F u r t h e r and t e s t d e s i g n . E d u c a t i o n a l Measurement, 10, 3-31.  e v i d e n c e on r e s p o n s e and P s y c h o l o g i c a l  sets  F e l d t , L. S., & F o r s y t h , R. A. ( 1 9 7 4 ) . An e x a m i n a t i o n of t h e c o n t e x t e f f e c t i n item s a m p l i n g . J o u r n a l of E d u c a t i o n a l Measurement, 1 1 , 73-82. F l a u g h e r , R. L., M e l t o n , R. S., & Myers, C. T. ( 1 9 6 8 ) . Item rearrangement under t y p i c a l t e s t c o n d i t i o n s . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 28, 813-824. F r e n c h , J . L., & G r e e r , D. ( 1 9 6 4 ) . E f f e c t o f t e s t - i t e m a r r a n g e m e n t on p h y s i o l o g i c a l and p s y c h o l o g i c a l b e h a v i o r i n p r i m a r y - s c h o o l c h i l d r e n . J o u r n a l of E d u c a t i o n a l Measurement. 1 , 151-153. Hambleton, R. K., & T r a u b , R. E . ( 1 9 7 4 ) . The e f f e c t s o r d e r on t e s t p e r f o r m a n c e and s t r e s s . J o u r n a l of E x p e r i m e n t a l E d u c a t i o n , 43, 40-46.  of  item  (117)  Hodson, D. ( 1 9 8 4 ) . The e f f e c t o f c h a n g e s i n i t e m s e q u e n c e on s t u d e n t p e r f o r m a n c e i n a m u l t i p l e - c h o i c e c h e m i s t r y t e s t . J o u r n a l of Research i n Science Teaching. 21, 489-495. H o p k i n s , C. D., & A n t e s , R. L. ( 1 9 8 5 ) . C l a s s r o o m measurement & e v a l u a t i o n ( 2 n d . ) . I t a s c a , I L : P e a c o c k . Huck, S. W., & Bowers, N. D. ( 1 9 7 2 ) . Item d i f f i c u l t y l e v e l and s e q u e n c e e f f e c t s i n m u l t i p l e - c h o i c e a c h i e v e m e n t t e s t s . J o u r n a l of E d u c a t i o n a l Measurement. 9, 105-111. Kestenbaum, J . M., & W e i n e r , B. ( 1 9 7 0 ) . A c h i e v e m e n t p e r f o r m a n c e r e l a t e d t o a c h i e v e m e n t m o t i v a t i o n and t e s t a n x i e t y . J o u r n a l o f C o n s u l t i n g and C l i n i c a l P s y c h o l o g y . 34, 343-344. K i n g s t o n , N. M., & D o r a n s , N. J . ( 1 9 8 4 ) . Item l o c a t i o n e f f e c t s and t h e i r i m p l i c a t i o n s f o r IRT e q u a t i n g and a d a p t i v e t e s t i n g . A p p l i e d P s y c h o l o g i c a l M e a s u r e m e n t. 147-154. K l e i n k e , D. J . ( 1 9 8 0 ) . Item o r d e r , r e s p o n s e l o c a t i o n examinee s e x and h a n d e d n e s s and p e r f o r m a n c e on a m u l t i p l e - c h o i c e t e s t . The J o u r n a l of E d u c a t i o n a l R e s e a r c h . 73, 225-229.  8,  and  K l i m k o , I . P. ( 1 9 8 4 ) . Item a r r a n g e m e n t , c o g n i t i v e e n t r y c h a r a c t e r i s t i c s , s e x , and t e s t a n x i e t y as p r e d i c t o r s of a c h i e v e m e n t e x a m i n a t i o n p e r f o r m a n c e . J o u r n a l of E x p e r i m e n t a l E d u c a t i o n . 52, 214-219. K l o s n e r , N. C , & G e l l m a n , E. K. ( 1 9 7 3 ) . The e f f e c t o f i t e m a r r a n g e m e n t on c l a s s r o o m t e s t p e r f o r m a n c e : I m p l i c a t i o n s f o r c o n t e n t v a l i d i t y . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 33, 413-418. L a n e , D. S., B u l l , K.-S., K u n d e r t , D. K., & Newman, D. L. ( 1 9 8 7 ) . The e f f e c t s of knowledge of i t e m a r r a n g e m e n t , g e n d e r , and s t a t i s t i c a l and c o g n i t i v e i t e m d i f f i c u l t y on t e s t p e r f o r m a n c e . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement. 47, 865-879. L e a r y , L. F., & D o r a n s , N. J . ( 1 9 8 5 ) . I m p l i c a t i o n s f o r a l t e r i n g the c o n t e x t i n which t e s t items appear: A h i s t o r i c a l p e r s p e c t i v e on an immediate c o n c e r n . Review of E d u c a t i o n a l R e s e a r c h . 55, 387-413. M a c N i c o l , K. ( 1 9 7 0 ) . E f f e c t s of v a r y i n g o r d e r of i t e m d i f f i c u l t y i n an u n s p e e d e d v e r b a l t e s t . U n p u b l i s h e d manuscript, Educational Testing Service, Princeton,  NJ.  (118)  M a r s o , R. N. ( 1 9 7 0 ) . T e s t i t e m a r r a n g e m e n t , t e s t i n g t i m e , and p e r f o r m a n c e . J o u r n a l of E d u c a t i o n a l Measurement, 7, 113-118. M i c r o c a t t e s t i n g system (3rd Ed.). A s s e s s m e n t Systems C o r p o r a t i o n .  (1988). S t . P a u l ,  M i l l m a n , J . , & B i s h o p , C. H. ( 1 9 6 5 ) . An a n a l y s i s t e s t - w i s e n e s s . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 25, 707-726.  MN:  of  M o l l e n k o p f , W. G. ( 1 9 5 0 ) . An e x p e r i m e n t a l s t u d y of the e f f e c t s on i t e m - a n a l y s i s d a t a of c h a n g i n g i t e m p l a c e m e n t and t e s t t i m e l i m i t . P s y c h o m e t r i k a , 15, 291-315. Monk, J . J . , & S t a l l i n g s , W. o r d e r on t e s t s c o r e s . The R e s e a r c h , 63, 463-465.  M. ( 1 9 7 0 ) . E f f e c t s o f i t e m J o u r n a l of. E d u c a t i o n a l ,  Munz, D. C , & J a c o b s , P. D. ( 1 9 7 1 ) . An e v a l u a t i o n of p e r c e i v e d i t e m - d i f f i c u l t y sequencing i n academic t e s t i n g , B r i t i s h J o u r n a l of E d u c a t i o n a l P s y c h o l o g y . 41, 195-205. Munz, D. C , & Smouse, A. D. ( 1 9 6 8 ) . I n t e r a c t i o n e f f e c t s of i t e m - d i f f i c u l t y s e q u e n c e and a c h i e v e m e n t - a n x i e t y r e a c t i o n on a c a d e m i c p e r f o r m a n c e . J o u r n a l o f E d u c a t i o n a l P s y c h o l o g y . 59, 370-374. P l a k e , B. S. ( 1 9 8 0 ) . Item a r r a n g e m e n t and knowledge of a r r a n g e m e n t on t e s t s c o r e s . J o u r n a l of Experimental E d u c a t i o n . 49, 56-58. P l a k e , B. S., & A n s o r g e , C. J . ( 1 9 8 4 ) . E f f e c t s of i t e m a r r a n g e m e n t , sex of t h e s u b j e c t , and t e s t a n x i e t y on c o g n i t i v e and s e l f - p e r c e p t i o n s c o r e s i n a n o n q u a n t i t a t i v e c o n t e n t a r e a . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement. 44, 423-430. P l a k e , B. S., A n s o r g e , C. J . , P a r k e r , C. S., & Lowry, S. ( 1 9 8 2 ) . E f f e c t s of i t e m a r r a n g e m e n t , knowledge of a r r a n g e m e n t t e s t a n x i e t y and s e x on t e s t p e r f o r m a n c e . J o u r n a l o f E d u c a t i o n a l Measurement. 19, 49-57.  R.  P l a k e , B. S., M e l i c a n , G. J . , C a r t e r , L., & S h a u g h n e s s y , L. C. ( 1 9 8 3 ) . D i f f e r e n t i a l p e r f o r m a n c e of m a l e s and f e m a l e s on e a s y t o h a r d i t e m a r r a n g e m e n t s : I n f l u e n c e of f e e d b a c k a t the i t e m l e v e l . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 43, 1067-1075.  (119) P l a k e , B. S., Thompson, P. A., & Lowry, S. ( 1 9 8 0 ) . E f f e c t of i t e m a r r a n g e m e n t , knowledge of a r r a n g e m e n t , and t e s t a n x i e t y on two s c o r i n g methods. J o u r n a l o f Experimental E d u c a t i o n , 49, 214-219. R i n d l e r , S. E. ( 1 9 8 0 ) . The e f f e c t s of s k i p p i n g o v e r more d i f f i c u l t i t e m s on t i m e - l i m i t e d t e s t s : I m p l i c a t i o n s f o r t e s t v a l i d i t y . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 40, 989-998. R o b i t a i l l e , D. F., & G a r d e n , R. A. ( E d s . ) ( 1 9 8 7 ) . The s e c o n d i n t e r n a t i o n a l m a t h e m a t i c s s t u d y : V o l . 2. C o n t e x t and outcomes o f s c h o o l m a t h e m a t i c s . A l b a n y , NY: I n t e r n a t i o n a l A s s o c i a t i o n f o r t h e E v a l u a t i o n of E d u c a t i o n a l Achievement. Ruch, G. M. ( 1 9 2 9 ) . The o b j e c t i v e or C h i c a g o : S c o t t Foresman.  new-type  examination.  Sax, G., & C a r r , A. ( 1 9 6 2 ) . An i n v e s t i g a t i o n of r e s p o n s e s e t s on a l t e r e d p a r a l l e l f o r m s . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement, 2 2 , 3 7 1 - 3 7 6 . Sax, G., & Cromack, T. R. ( 1 9 6 6 ) . The e f f e c t s o f v a r i o u s forms of i t e m a r r a n g e m e n t s on t e s t p e r f o r m a n c e . J o u r n a l of E d u c a t i o n a l Measurement. 3, 309-311. S i r o t n i k , K., & W e l l i n g t o n , R. ( 1 9 7 4 ) . S c r a m b l i n g content i n a c h i e v e m e n t t e s t i n g : An a p p l i c a t i o n o f m u l t i p l e m a t r i x s a m p l i n g i n e x p e r i m e n t a l d e s i g n . J o u r n a l of E d u c a t i o n a l Measurement. 11, 179-188. Smouse, A. D., & Munz, D. C. ( 1 9 6 8 ) . The e f f e c t s o f a n x i e t y and i t e m d i f f i c u l t y s e q u e n c e on a c h i e v e m e n t t e s t i n g s c o r e s . J o u r n a l o f Psychology,. 68, 181-184. Smouse, A. D., & Munz, D. C. ( 1 9 6 9 ) . Item d i f f i c u l t y s e q u e n c i n g and r e s p o n s e s t y l e : A f o l l o w - u p a n a l y s i s . E d u c a t i o n a l and P s y c h o l o g i c a l Measurement. 29, 469-472. SPSS-X u s e r ' s  guide.  ( 1 9 8 3 ) . New  Y o r k , NY:  McGraw-Hill.  T o w l e , N. J . , & M e r r i l l , P. F. ( 1 9 7 5 ) . E f f e c t s of a n x i e t y t y p e and i t e m - d i f f i c u l t y s e q u e n c i n g on m a t h e m a t i c s t e s t p e r f o r m a n c e . J o u r n a l o f E d u c a t i o n a l Measurement, 12, 241-249 . Tuck, P. J . ( 1 9 7 8 ) . E x a m i n e e s ' c o n t r o l of i t e m d i f f i c u l t y s e q u e n c e . P s y c h o l o g i c a l Reports,. 42, 1109-1110.  (120) W h i t e l y , S. E., & D a w i s , R. V. ( 1 9 7 6 ) . The i n f l u e n c e o f t e s t c o n t e x t on i t e m d i f f i c u l t y . E d u c a t i o n a l and P s y c h o l o g i c a l M e a s u r e m e n t . 36, 329-337. Yen, W. M. ( 1 9 8 0 ) . The e x t e n t , c a u s e s and i m p o r t a n c e o f c o n t e x t e f f e c t s on i t e m p a r a m e t e r s f o r two l a t e n t t r a i t m o d e l s . J o u r n a l o f E d u c a t i o n a l Measurement. 17, 297-311.  (121)  APPENDIX  I  (122)  TEACHER  INSTRUCTIONS  MATHEMATICS  8 EXAMINATION  THESIS PROJECT I.  General  II.  Thesis  III.  Detailed  IV.  OF M. J . SCALES  Directions  P r o j e c t Background Directions  a.  Start  b.  Booklets  c.  Answer  d.  Identification  e.  Name  f.  Gender  g.  Grade  h.  Birth  i.  Answer  j.  Start  k.  End  1.  Collect  m.  Math A b i l i t y  n.  Return Testing  (optional)  Explanations  Sheets Number  Section  Date Sheet  Usage  Examination  Examination Testing  Materials  Rating Materials  Appendix a.  Student  Identification  Sample  (123)  I. GENERAL DIRECTIONS M a t e r i a l s Required  by t h e E x a m i n e r  A.  A copy of these  B.  A c l a s s s e t o f mixed t e s t b o o k l e t s , c o m p l e t e w i t h a n answer s h e e t and some s c r a p p a p e r t o g i v e one b o o k l e t t o e a c h s t u d e n t . T e s t Format T e s t Format T e s t Format T e s t Format  instructions.  1 (orange) 2 (yellow) 3 (blue) 4 (green)  C.  A supply  of sharpened  soft-lead  pencils.  D.  An e x t r a s u p p l y o f b o o k l e t s , c o m p l e t e answer s h e e t s and s c r a t c h p a p e r .  with  1.  A class explain test.  p e r i o d o f one hour s h o u l d be s u f f i c i e n t t o (15 min.) and a d m i n i s t e r (45 min.) t h e  2.  E x p l a i n t o t h e s t u d e n t s t h a t t o d a y t h e y w i l l be t a k i n g a t e s t as p a r t of a s t u d y t o determine i f s k i p p i n g back and f o r t h between t e s t q u e s t i o n s w i l l h e l p s t u d e n t s t o do b e t t e r on t e s t s . Some s t u d e n t s w i l l r e c e i v e t e s t s w h i c h a l l o w them t o s k i p back and f o r t h . Other s t u d e n t s w i l l r e c e i v e b o o k l e t s w h i c h r e q u i r e t h a t t h e y do n o t s k i p ahead but must do t h e q u e s t i o n s i n t h e same o r d e r a s i n their booklet. Do n o t d i s c u s s t h e o r d e r o f t h e i t e m s i n t h e t e s t or the t e s t t a k i n g s t r a t e g y of s k i p p i n g the h a r d Q u e s t i o n s t o do t h e e a s y q u e s t i o n s f i r s t .  3.  Be s u r e  a l l students  have a p e n c i l  (No. 2 o r HB).  4.  C a u t i o n s t u d e n t s n o t t o open t h e i r t e s t u n t i l t i l l t h e y a r e t o l d t o do s o .  5.  D i s t r i b u t e one t e s t b o o k l e t w i t h a p p r o p r i a t e answer s h e e t and s c r a p p a p e r t o e a c h s t u d e n t . A l t e r n a t e e v e n l y between t h e f o u r d i f f e r e n t t y p e s of c o l o u r c o d e d t e s t b o o k l e t s .  6.  Have t h e s t u d e n t s c a r e f u l l y remove t h e answer s h e e t and t h e p i e c e o f s c r a p p a p e r f r o m t h e t e s t booklet. Have them c h e c k t o s e e i f box A i n t h e  booklet  (124) " I d e n t i f i c a t i o n No." s e c t i o n of t h e i r answer s h e e t has been marked w i t h t h e number t h a t c o r r e s p o n d s t o t h e t e s t f o r m a t number on t h e f r o n t c o v e r . 7.  Have t h e s t u d e n t s c o m p l e t e t h e name, s e x , b i r t h d a t e , and g r a d e s e c t i o n s o f t h e i r answer sheets. I f t h e c l a s s i s u n s u r e how t o c o m p l e t e t h e s e s e c t i o n s , read the a p p r o p r i a t e " D e t a i l e d D i r e c t i o n s " o f t h i s b o o k l e t t o t h e c l a s s or use the sample s h e e t i n t h e a p p e n d i x a s a g u i d e .  8.  To g e t t h e s t u d e n t s t o be r e a l i s t i c a l l y motivated, p l e a s e t e l l them t h a t t h e s e t e s t r e s u l t s may be u s e d t o c a l c u l a t e t h e i r f i n a l marks.  9.  Encourage the s t u d e n t s t o read the r e m a i n i n g d i r e c t i o n s f r o m number 5 t o t h e end o f t h e p a g e . I f n e c e s s a r y , r e a d them t o t h e whole c l a s s .  10.  Remind t h o s e s t u d e n t s w i t h " S p e c i a l I n s t r u c t i o n s " t h a t t h e y may n o t s k i p a h e a d t o new q u e s t i o n s or go back t o o l d o n e s . T h e y must do t h e q u e s t i o n s i n t h e same o r d e r as t h e y a r e p r e s e n t e d i n t h e test. I f t h e y c a n ' t answer a q u e s t i o n , t h e y may o m i t t h a t q u e s t i o n and go on t o t h e n e x t one. Nonetheless, they should at l e a s t t r y t h e i r best t o answer e v e r y q u e s t i o n on t h e t e s t .  11.  When you a r e s u r e d i r e c t i o n s , begin  12.  D u r i n g t h e t e s t i n g p e r i o d , s t u d e n t s m i g h t ask f o r help. E n c o u r a g e them t o r e a d and r e s p o n d t o e a c h item t o the best of t h e i r a b i l i t i e s . Do NOT change t h e w o r d i n g o f any i t e m s , or e x p l a i n s p e c i f i c t e r m s , or d i s c u s s t h e o r d e r i n g o f t h e questions. T r e a t t h i s t e s t i n g s i t u a t i o n as n o r m a l and as s e r i o u s as any o t h e r e x a m i n a t i o n .  13.  A f t e r 45 m i n u t e s , or s o o n e r 1£ a l l s t u d e n t s a r e f i n i s h e d , end t h e t e s t . C o l l e c t the t e s t b o o k l e t s i n c o l o u r coded groups. C o l l e c t t h e answer s h e e t s and c h e c k t o see t h a t a l l of t h e i d e n t i f i c a t i o n s e c t i o n s of t h e answer s h e e t s have been c o m p l e t e d correctly.  14.  On a c l a s s l i s t , r a t e e a c h s t u d e n t s a b i l i t y t o do mathematics. Using a s i x point s c a l e , record a number f r o m 1 t o 6 t h a t r e p r e s e n t s your b e s t e s t i m a t e of each s t u d e n t s mathematical abilities. T h i s r a t i n g s h o u l d be somewhat i n d e p e n d e n t from  that a l l students understand t h e t e s t (45 m i n u t e s ) .  the  (125) o v e r a l l i n t e l l i g e n c e or c l a s s r o o m b e h a v i o u r . Use a "1" f o r t h o s e s t u d e n t s w i t h t h e l o w e s t 10% of m a t h e m a t i c a l a b i l i t y , a "2" f o r t h e n e x t 15% o f s t u d e n t s w i t h h i g h e r m a t h e m a t i c a l a b i l i t y , a "3" f o r t h e n e x t 25%, a "4" f o r t h e n e x t 25%, a "5" f o r t h e n e x t 15%, and a "6" f o r t h e 10% o f s t u d e n t s w i t h the h i g h e s t mathematical ability.  (126)  THESIS PROJECT BACKGROUND E v e r s i n c e m u l t i p l e c h o i c e t e s t s f i r s t came o u t i n t h e e a r l y 1920s, most t e x t b o o k a u t h o r s have s u g g e s t e d t h a t t h e s e t e s t s s h o u l d be a r r a n g e d w i t h t h e e a s i e s t q u e s t i o n s a t t h e b e g i n n i n g and t h e h a r d e s t q u e s t i o n s a t the end. One j u s t i f i c a t i o n f o r s u c h an a r r a n g e m e n t i s t o h e l p low a b i l i t y s t u d e n t s a v o i d e a r l y f r u s t r a t i o n with the t e s t . However, much o f t h e r e s e a r c h o v e r t h e l a s t 40 y e a r s has g e n e r a l l y found t h a t t h e i t e m o r d e r does n o t make much of a d i f f e r e n c e t o t h e f i n a l r e s u l t s of t h e t e s t s . One p u r p o s e of t h i s s t u d y i s t o examine t h e d i s c r e p a n c y between what r e s e a r c h has s t a t i s t i c a l l y f o u n d and what t e a c h e r s and t e x t b o o k w r i t e r s have i n t u i t i v e l y found. S i n c e most o f t h e p a s t r e s e a r c h has used c o l l e g e s t u d e n t s , t h i s s t u d y w i l l i n v o l v e a y o u n g e r and more d i v e r s e sample of h i g h s c h o o l students. I t i s t h e h y p o t h e s i s of t h i s s t u d y t h a t s t u d e n t s who a r e i n f a c t a f f e c t e d by t h e i t e m o r d e r a r e more l i k e l y t o be f o u n d i n a t y p i c a l p u b l i c s c h o o l r a t h e r than i n a c o l l e g e c l a s s r o o m . I f s t u d e n t s o f low a b i l i t y a r e , i n f a c t , e a s i l y d i s c o u r a g e d by s t a r t i n g t e s t s w i t h the more d i f f i c u l t q u e s t i o n s , then another purpose of t h i s s t u d y i s t o examine one o f the s k i l l s t h a t h i g h a b i l i t y s t u d e n t s may use t o a v o i d t h a t d i s c o u r a g e m e n t . One p o s s i b l e s k i l l o f t h e more a b l e s t u d e n t s i s the t a c t i c of o m i t t i n g t h e h a r d q u e s t i o n s u n t i l t h e y have f i n i s h e d the easy q u e s t i o n s . T h i s may be a s k i l l t h a t t h e low a b i l i t y s t u d e n t s a r e e i t h e r unaware o f or j u s t f a i l t o use . A t h i r d p u r p o s e o f t h i s s t u d y i s a more e s o t e r i c one w h i c h i n v o l v e s e x a m i n i n g t h e r e s u l t s o f t h i s t e s t u s i n g two t y p e s o f t e s t s t a t i s t i c s . The s t u d i e s w h i c h have found no d i f f e r e n c e s a s a r e s u l t o f i t e m o r d e r have u s e d c l a s s i c a l s t a t i s t i c s t o examine t h e i r d a t a . However, r e c e n t s t u d i e s w h i c h have f o u n d some e f f e c t s of i t e m o r d e r , have used t h e more modern l a t e n t t r a i t statistics. T h i s s t u d y would compare t h e r e s u l t s o b t a i n e d f r o m e a c h t y p e of s t a t i s t i c a l method. A f i n a l r e a s o n , of c o u r s e , i s t o c o m p l e t e t h e r e q u i r e m e n t s t o o b t a i n a M a s t e r of A r t s d e g r e e i n t h e F a c u l t y of E d u c a t i o n a t the U n i v e r s i t y of B r i t i s h Columbia i n the department of E d u c a t i o n a l P s y c h o l o g y and S p e c i a l E d u c a t i o n w i t h a s p e c i a l i z a t i o n i n measurement, e v a l u a t i o n and r e s e a r c h m e t h o d o l o g y .  (127) III.  DETAILED DIRECTIONS ( o p t i o n a l ) A l l d i r e c t i o n s t h a t you can read t o t h e s t u d e n t s a r e i n d e n t e d s o t h a t t h e y s t a n d o u t . You may r e a d them e x a c t l y a s t h e y a r e w r i t t e n , u s i n g a n a t u r a l t o n e and manner. I f n e c e s s a r y , y o u may s u p p l e m e n t t h e d i r e c t i o n s w i t h y o u r own e x p l a n a t i o n s , b u t do n o t g i v e h e l p on s p e c i f i c t e s t q u e s t i o n s . T r y t o m a i n t a i n a n a t u r a l c l a s s r o o m atmosphere d u r i n g the t e s t a d m i n i s t r a t i o n . Encourage students to do t h e i r b e s t , and a d v i s e them n o t t o s p e n d t o o much t i m e on a n y one q u e s t i o n . Check p e r i o d i c a l l y t o make sure t h a t s t u d e n t s a r e r e c o r d i n g t h e i r answers p r o p e r l y , a r e f o l l o w i n g i n s t r u c t i o n s , and a r e w o r k i n g t o t h e end o f t h e t e s t , o r a s f a r a s t h e y c a n . The s c o r i n g m a c h i n e used t o p r o c e s s t h e answer s h e e t s i s c a p a b l e o f a l m o s t 100% a c c u r a c y i f t h e answer s h e e t s a r e marked c o r r e c t l y and k e p t i n good c o n d i t i o n . Remind t h e s t u d e n t s t o h a n d l e t h e s h e e t s w i t h c a r e ; t o r e c o r d t h e i r answer w i t h h e a v y , d a r k marks; and t o a v o i d m a k i n g s t r a y marks on t h e i r answer s h e e t s . Answer s h e e t s s h o u l d n e v e r be f o l d e d , c l i p p e d , o r t o r n . a.  Start  Explanations  (Have a l l d e s k s c l e a r e d , and s e e t h a t e a c h s t u d e n t has a s o f t - l e a d p e n c i l (No. 2 o r HB), and an e r a s e r . S a y : ) "You a r e g o i n g t o t a k e a s p e c i a l math t e s t t o d a y . Don't open y o u r t e s t book o r make a n y marks on i t u n t i l I t e l l y o u what t o d o . "  b_,  BooKlets  ( G i v e one t e s t b o o k l e t t o e a c h s t u d e n t . As y o u hand o u t t h e t e s t s , a l t e r n a t e between t h e f o u r d i f f e r e n t types of t e s t booklets t o evenly d i s t r i b u t e the four t y p e s among y o u r s t u d e n t s . P l a c e the b o o k l e t with the f r o n t c o v e r up. A l s o , make s u r e e a c h s t u d e n t has an answer s h e e t and a p i e c e o f s c r a t c h p a p e r i n h i s booklet. When t h e b o o k l e t s have been d i s t r i b u t e d , say: ) " P l e a s e d o n ' t open y o u r t o l d t o do s o by me."  booklets  until  you a r e  "Four d i f f e r e n t b o o k l e t s have been d i s t r i b u t e d a s p a r t of a s p e c i a l experiment to see i f students  (128) c a n do b e t t e r on t e s t s i f t h e y a r e a l l o w e d t o s k i p a r o u n d between t e s t q u e s t i o n s . Those o f you w i t h t h e y e l l o w o r o r a n g e t e s t s a r e a l l o w e d t o go back and f o r t h i n t h e t e s t b o o k l e t and do t h e q u e s t i o n s i n w h a t e v e r o r d e r you w i s h . Those o f you w i t h b l u e or g r e e n t e s t b o o k l e t s a r e r e q u e s t e d not t o s k i p a h e a d t o a new q u e s t i o n and t h e n go back t o an o l d one. You must answer t h e q u e s t i o n s i n t h e o r d e r t h e y a p p e a r on t h e t e s t . S t u d e n t s w i t h the b l u e or green t e s t b o o k l e t s w i l l a l s o f i n d t h a t t h e y have some s p e c i a l i n s t r u c t i o n s on t h e i r t e s t b o o k l e t s and on t h e i r t e s t q u e s t i o n s t o r e m i n d them o f t h e s e s p e c i a l rules."  (Pause and answer q u e s t i o n s . Do n o t d i s c u s s t h e s p e c i a l o r d e r of the items. T r y t o m a i n t a i n your normal t e s t i n g r o u t i n e . T r y to o b t a i n the c o o p e r a t i o n and m o t i v a t i o n of t h e s t u d e n t s . ) c.  Answer  Sheets  (Say:) " C a r e f u l l y remove t h e answer s h e e t f r o m t h e i n s i d e f r o n t cover of your t e s t b o o k l e t . Your answer s h e e t i s g o i n g t o be s c o r e d by m a c h i n e , so be c a r e f u l w i t h i t . Keep i t as c l e a n as p o s s i b l e , and d o n ' t bend i t o r f o l d t h e c o r n e r s . " d.  Identification  Number  "I may be u s i n g t h e r e s u l t s of t h i s exam t o h e l p me d e t e r m i n e y o u r f i n a l g r a d e a t t h e end of the year. I t i s t h e r e f o r e i m p o r t a n t t h a t you do y o u r best. I t i s a l s o i m p o r t a n t t h a t I know w h i c h t e s t you t o o k . I want e v e r y o n e t o f i n d t h e box marked ' I d e n t i f i c a t i o n No.' on t h e i r answer s h e e t and the t e s t f o r m a t number on t h e f r o n t of t h e i r t e s t book l e t s . " (Show t h e l o c a t i o n of t h e ' I d e n t i f i c a t i o n No.' section on t h e back of t h e answer s h e e t and t h e t e s t f o r m a t number on t h e f r o n t of t h e t e s t b o o k l e t . ) " I n t h e box l a b e l l e d 'A' i n t h e i d e n t i f i c a t i o n number s e c t i o n , make s u r e t h a t t h e number of t h e t e s t f o r m a t of y o u r t e s t i s i n t h a t box. The o r a n g e t e s t i s f o r m a t one. The y e l l o w t e s t i s f o r m a t two. The b l u e t e s t i s f o r m a t t h r e e . The green t e s t i s format f o u r .  (129) (Pause) Name  Section  "Find  the spaces  f o r your  name."  (Demonstrate) " F i r s t i n t h e b o x e s p r i n t a s many l e t t e r s o f y o u r l a s t name a s y o u c a n . Use one box f o r e a c h letter. Then, l e a v e one box a s a s p a c e . Next, p r i n t a s many l e t t e r s o f y o u r f i r s t name a s you can. Then, l e a v e a n o t h e r s p a c e . F i n a l l y , print your middle i n i t i a l . I f you cannot f i t your f u l l name i n t h e s p a c e p r o v i d e d , t r y t o p r i n t a t l e a s t most o f y o u r l a s t name, a s p a c e , y o u r first i n i t i a l , a s p a c e , and f i n a l l y y o u r m i d d l e initial." (Pause) "Now i n t h e c o l u m n b e l o w e a c h box, f i l l i n t h e c i r c l e t h a t has t h e same l e t t e r o r s p a c e a s t h e l e t t e r o r s p a c e i n t h e box above i t . Be s u r e t h a t you mark o n l y one c i r c l e i n e a c h c o l u m n . F i l l in the b l a n k c i r c l e a t t h e t o p o f e v e r y c o l u m n i n w h i c h you have l e f t a s p a c e . Be s u r e t o make h e a v y , s h i n y marks t h a t c o v e r t h e whole c i r c l e . I f y o u make a m i s t a k e , e r a s e y o u r mark c o m p l e t e l y . I f y o u have a n y q u e s t i o n s , r a i s e y o u r h a n d . " (Pause u n t i l a l l s t u d e n t s have name s e c t i o n . Then s a y : )  finished  filling  i n the  "You s h o u l d have 19 c i r c l e s f i l l e d i n under t h e name b o x e s . Count and make s u r e . " (Pause) f.  Gender  (After  s t u d e n t s have c h e c k e d  t h e name s e c t i o n , s a y : )  "Now l o o k a t t h e box b e l o w t h e c o l u m n s you i n f o r y o u r name."  filled  (Demonstrate) " F i l l i n t h e c i r c l e n e x t t o 'Male' i f y o u a r e a male o r n e x t t o 'Female' i f y o u a r e f e m a l e . "  (130)  (Pause)  "Now l o o k a t t h e box and c i r c l e d numbers t o t h e r i g h t l a b e l l e d 'Grade or E d u c a t i o n ' . Just f i l l in t h e c i r c l e w i t h an 8 s i n c e t h i s i s a g r a d e 8 course. (Pause) h.  Birth  Date  "Now l o o k a t t h e c o l u m n s l a b e l l e d ' B i r t h Date'."  underneath the  box  (Demonstrate) " F i l l i n t h e c i r c l e n e x t t o t h e month i n w h i c h were b o r n . "  you  (Pause) " F i l l i n t h e boxes l a b e l l e d 'Day' w i t h two numbers f o r the day of your b i r t h . F o r example, i f you were b o r n on t h e s e v e n t h o f t h e month, you would write zero seven." (Pause) " F i l l i n the c i r c l e s i n the columns u n d e r n e a t h the boxes l a b e l l e d d a y t o show t h e number i n t h e box above t h e c o l u m n . Be s u r e t o o n l y f i l l i n one c i r c l e i n each column." (Pause) "Now f i l l i n t h e b o x e s l a b e l l e d y e a r w i t h t h e two numbers f o r t h e y e a r you were b o r n i n , and f i l l i n t h e c i r c l e under e a c h box t o i n d i c a t e t h e number i n t h e box." (Pause) "Now c h e c k t o make s u r e t h a t you have c o r r e c t l y f i l l e d i n a l l the r e q u i r e d i n f o r m a t i o n . " (Pause)  (131) Li  Answer  Sheet  Usage  " B e f o r e I t e l l y o u t o open y o u r t e s t b o o k l e t and s t a r t , I am g o i n g t o t e l l you how t o p r o p e r l y mark y o u r answer s h e e t . L i s t e n c a r e f u l l y s o t h a t you w i l l know how t o mark y o u r a n s w e r s . You a r e t o mark a l l y o u r a n s w e r s on y o u r answer s h e e t . Don't make any s t r a y marks on i t and do n o t w r i t e i n y o u r b o o k l e t . You s h o u l d a l r e a d y have some s c r a t c h p a p e r f o r a n y f i g u r i n g t h a t you m i g h t have t o do. F o r each q u e s t i o n , choose the b e s t answer. Then, on y o u r answer s h e e t , f i n d t h e number f o r t h e q u e s t i o n , and mark t h e s p a c e f o r y o u r a n s w e r . Be s u r e t o mark t h e s p a c e f o r y o u r a n s w e r . Be s u r e t o mark o n l y one answer s p a c e f o r e a c h question. Make y o u r mark h e a v y and s h i n y , and s e e t h a t i t c o m p l e t e l y f i l l s t h e answer s p a c e . I f you change y o u r mind a f t e r you've marked an a n s w e r , e r a s e t h e wrong mark c o m p l e t e l y ; t h e n make y o u r new mark." (On t h e c h a l k b o a r d , show s t u d e n t s how t o f i l l answer s p a c e . Answer a l l q u e s t i o n s . )  i n an  "You w i l l have 45 m i n u t e s t o work on t h i s t e s t . I f y o u have a n y t r o u b l e r e a d i n g a q u e s t i o n , r a i s e y o u r hand and I w i l l h e l p y o u . Of c o u r s e , you may not use a c a l c u l a t o r . I f you're not sure about the answer t o a q u e s t i o n , do t h e b e s t you c a n , b u t d o n ' t s p e n d t o o much t i m e on any one q u e s t i o n . You may o m i t a q u e s t i o n i f y o u a r e s u r e t h a t you c a n n o t answer i t . " "Make s u r e t h a t y o u have t u r n e d y o u r answer s h e e t o v e r t o s i d e one, s o t h e name s e c t i o n i s f a c e down, s o t h e s i d e w i t h t h e p i c t u r e o f t h e p e n c i l i s f a c e up, and s o t h e answer s p a c e f o r q u e s t i o n one i s f a c e up." ( D e m o n s t r a t e and c h e c k s t a r t i n g on s i d e 1.)  t o make s u r e e v e r y o n e i s  (132)  i.  Start  Examination  (When y o u f e e l t h a t e v e r y o n e u n d e r s t a n d s d i r e c t i o n s , say:) "You  may s t a r t  working  the  now."  ( R e c o r d t h e s t a r t i n g and e n d i n g t i m e s on t h e chalkboard. W h i l e s t u d e n t s a r e w o r k i n g , walk a r o u n d t h e room t o make s u r e t h a t t h e s t u d e n t s a r e f o l l o w i n g directions. T r y y o u r b e s t t o make s u r e t h a t s t u d e n t s do n o t change t h e o r d e r o f t h e exam q u e s t i o n s i f t h e y are i n t h e blue or green b o o k l e t s with t h e s p e c i a l instructions. I f you see t h a t a s t u d e n t i s h a v i n g d i f f i c u l t y r e a d i n g a p r o b l e m , y o u may h e l p t h e s t u d e n t r e a d t h e p r o b l e m ; however, do n o t g i v e h e l p i n answering any of the q u e s t i o n s . ) k.  End  Examination  ( A f t e r 45 m i n u t e s , f i n i s h e d , say:) "Stop! booklet collect Collect  or sooner  i f a l l students  have  P u t y o u r p e n c i l down now, and c l o s e y o u r s o t h a t t h e f r o n t c o v e r i s up. I w i l l y o u r t e s t b o o k l e t s and answer s h e e t s . " Testing Materials  ( C o l l e c t t h e t e s t b o o k l e t s i n t o the f o u r c o l o u r coded groups. C o l l e c t t h e answer s h e e t s and c h e c k t o make s u r e t h a t t h e s t u d e n t i d e n t i f i c a t i o n s e c t i o n s have been c o r r e c t l y f i l l e d o u t . C o l l e c t t h e s c r a t c h p a p e r and d i s p o s e of i t . C o l l e c t any of the e x t r a p e n c i l s loaned to the s t u d e n t s . ) Sb  Math A b i l i t y  Rating  On a c l a s s l i s t , r a t e e a c h s t u d e n t s a b i l i t y t o do mathematics. U s i n g a s i x p o i n t s c a l e , r e c o r d a number from 1 t o 6 t h a t r e p r e s e n t s your b e s t e s t i m a t e of t h e i r mathematical a b i l i t i e s . T h i s r a t i n g s h o u l d be somewhat i n d e p e n d e n t o f o v e r a l l i n t e l l i g e n c e and g e n e r a l classroom behaviour. Use a " 1 " f o r t h o s e s t u d e n t s w i t h t h e l o w e s t 10% o f m a t h e m a t i c a l a b i l i t y , a "2" f o r t h e n e x t 15% o f s t u d e n t s w i t h h i g h e r m a t h e m a t i c a l ability, a "3" f o r t h e n e x t 25%, a "4" f o r t h e n e x t 25%, a "5" f o r t h e n e x t 15%, and a "6" f o r t h e 10% o f s t u d e n t s with the h i g h e s t mathematical ability.  (133) T h i s i n f o r m a t i o n w i l l be k e p t s t r i c t l y c o n f i d e n t i a l and used o n l y t o i d e n t i f y which s t u d e n t s , from a t e a c h e r ' s p o i n t o f v i e w , may be e i t h e r f r u s t r a t e d by t h e a r r a n g e m e n t o f t h e t e s t q u e s t i o n s o r h i n d e r e d by t h e d i r e c t i o n s of the t e s t b o o k l e t s . P l e a s e k e e p answer s h e e t s g r o u p e d i n c l a s s e s w i t h t h e i r class l i s t . The r e s u l t s f o r e a c h o f y o u r s t u d e n t s w i l l be s e n t t o y o u a t y o u r r e q u e s t .  n.  Return Testing Materials  P l e a s e r e t u r n t e s t i n g m a t e r i a l , t h e t e s t s , t h e answer s h e e t s , t h e p e n c i l s , and t h e r a t i n g l i s t s t o M i c h a e l Scales. The t e s t w i l l be s c o r e d and a n a l y z e d by M i c h a e l S c a l e s , graduate student a t the U n i v e r s i t y of B r i t i s h Columbia, and t e a c h e r a t A l d e r g r o v e S e c o n d a r y . The r e s u l t s w i l l be k e p t s t r i c t l y c o n f i d e n t i a l w i t h t h e r e s u l t s o f i n d i v i d u a l students only being sent t o that student's c l a s s r o o m t e a c h e r i f so r e q u e s t e d . I t i s not the i n t e n t i o n o f t h i s s t u d y t o make c o m p a r i s o n s between i n d i v i d u a l c l a s s e s , s c h o o l s , t e a c h e r s , or s t u d e n t s . Thank y o u f o r y o u r  c o o p e r a t i o n and y o u r  efforts.  (134)  APPENDIX  II  (135)  MATHEMATICS  8 EXAMINATION  TEST FORMAT 1  INSTRUCTIONS 1.  Do NOT open t h e t e s t b o o k l e t u n t i l you a r e t o l d t o do so. You w i l l have 45 m i n u t e s t o c o m p l e t e t h i s t e s t .  2.  C a r e f u l l y remove t h e answer s h e e t f r o m i n s i d e t h e f r o n t c o v e r and make s u r e t h e r e i s a 1 marked i n box A o f t h e I d e n t i f i c a t i o n No. s e c t i o n o f t h e answer s h e e t .  3.  Be s u r e paper.  4.  F i l l i n y o u r answer and b i r t h d a t e .  5.  Do NOT u s e a c a l c u l a t o r  6.  F o r each q u e s t i o n , s e l e c t t h e b e s t answer. Mark your c h o i c e on t h e answer s h e e t by f i l l i n g i n t h e b u b b l e under t h e c o r r e c t l e t t e r . Make s u r e t h e q u e s t i o n number i s t h e same a s t h e q u e s t i o n number i n t h e t e s t booklet.  7.  Do n o t s p e n d t o o l o n g on a n y one q u e s t i o n . b e s t p i c k a good answer t o e v e r y q u e s t i o n .  8.  I f y o u make a m i s t a k e , c o m p l e t e l y e r a s e y o u r f i r s t c h o i c e and f i l l i n t h e b u b b l e o f y o u r new c h o i c e .  9.  Do NOT w r i t e i n t h e t e s t b o o k l e t . Mark o n l y y o u r answer s h e e t . I f y o u r b o o k l e t a l r e a d y has a n y i n a p p r o p r i a t e marks, a s k f o r a c l e a n b o o k l e t .  y o u have a p e n c i l ,  sheet  an e r a s e r , and some s c r a t c h  with  your  name, s e x , g r a d e ,  or a p r o t r a c t o r .  T r y your  (136)  FINISHED? Close your t e s t  booklet.  Make s u r e y o u have f i l l e d i n y o u r a n s w e r s h e e t w i t h name, s e x , g r a d e , a n d b i r t h d a t e . Make s u r e t h a t t h e I d e n t i f i c a t i o n Turn  i n your t e s t  No. Box A h a s a 1 i n i t  b o o k l e t and answer s h e e t . THANK YOU  your  (137) MATHEMATICS  8 EXAMINATION  TEST FORMAT 2  INSTRUCTIONS 1.  Do NOT open t h e t e s t b o o k l e t u n t i l you a r e t o l d t o do so. You w i l l have 45 m i n u t e s t o c o m p l e t e t h i s t e s t .  2.  C a r e f u l l y remove t h e answer s h e e t f r o m i n s i d e t h e f r o n t c o v e r and make s u r e t h e r e i s a 2 marked i n box A o f t h e I d e n t i f i c a t i o n No. s e c t i o n o f t h e answer s h e e t .  3.  Be s u r e paper.  4.  F i l l i n y o u r answer and b i r t h d a t e .  5.  Do NOT u s e a c a l c u l a t o r  6.  F o r each q u e s t i o n , s e l e c t the best answer. Mark y o u r c h o i c e on t h e answer s h e e t by f i l l i n g i n t h e b u b b l e under t h e c o r r e c t l e t t e r . Make s u r e t h e q u e s t i o n number i s t h e same a s t h e q u e s t i o n number i n t h e t e s t booklet.  7.  Do n o t s p e n d t o o l o n g on a n y one q u e s t i o n . b e s t p i c k a good answer t o e v e r y q u e s t i o n .  8.  I f y o u make a m i s t a k e , c o m p l e t e l y e r a s e y o u r f i r s t c h o i c e and f i l l i n t h e b u b b l e o f y o u r new c h o i c e .  9.  Do NOT w r i t e i n t h e t e s t b o o k l e t . Mark o n l y y o u r answer s h e e t . I f y o u r b o o k l e t a l r e a d y has a n y i n a p p r o p r i a t e marks, a s k f o r a c l e a n b o o k l e t .  y o u have a p e n c i l ,  sheet  a n e r a s e r , and some s c r a t c h  with  your  name, s e x , g r a d e ,  or a p r o t r a c t o r .  T r y your  (138)  FINISHED? C l o s e your Make s u r e Make s u r e  test  booklet.  y o u have f i l l e d i n y o u r answer s h e e t name, s e x , g r a d e , and b i r t h d a t e . that  Turn  the I d e n t i f i c a t i o n  i n your  test  your  No. Box A has a 2 i n  b o o k l e t and answer THANK YOU  with  sheet.  it  MATHEMATICS  8 EXAMINATION  (139)  TEST FORMAT 3  INSTRUCTIONS 1.  Do NOT open t h e t e s t b o o k l e t u n t i l y o u a r e t o l d t o do so. You w i l l have 45 m i n u t e s t o c o m p l e t e t h i s t e s t .  2.  C a r e f u l l y remove t h e answer s h e e t f r o m i n s i d e t h e f r o n t c o v e r and make s u r e t h e r e i s a 3 marked i n box A o f t h e I d e n t i f i c a t i o n No. s e c t i o n o f t h e answer s h e e t .  3.  Be s u r e paper.  4.  F i l l i n y o u r answer s h e e t and b i r t h d a t e .  5.  Do NOT  6.  For each q u e s t i o n , s e l e c t the best answer. Mark your c h o i c e on t h e answer s h e e t by f i l l i n g i n t h e b u b b l e under t h e c o r r e c t l e t t e r . Make s u r e t h e q u e s t i o n number i s t h e same a s t h e q u e s t i o n number i n t h e t e s t booklet.  7.  Do n o t s p e n d t o o l o n g on a n y one q u e s t i o n . b e s t p i c k a good answer t o e v e r y q u e s t i o n .  8.  I f y o u make a m i s t a k e , c o m p l e t e l y e r a s e y o u r f i r s t c h o i c e and f i l l i n t h e b u b b l e o f y o u r new c h o i c e .  9.  Do NOT w r i t e i n t h e t e s t b o o k l e t . Mark o n l y y o u r answer s h e e t . I f y o u r b o o k l e t a l r e a d y has a n y i n a p p r o p r i a t e marks, ask f o r a c l e a n b o o k l e t .  you have a p e n c i l ,  use a c a l c u l a t o r  SPECIAL  an e r a s e r , and some  with  or a  your  scratch  name, s e x , g r a d e ,  protractor.  T r y your  INSTRUCTIONS  1.  You must b e g i n w i t h q u e s t i o n 1. When you have c h o s e n t h e b e s t answer and marked y o u r answer s h e e t , t h e n you must go t o q u e s t i o n 2. When you have f i n i s h e d q u e s t i o n 2, t h e n y o u must go on t o q u e s t i o n 3, t h e n q u e s t i o n 4, t h e n q u e s t i o n 5, and s o on t o t h e end o f t h e t e s t .  2.  T r y e a c h q u e s t i o n once and o n l y o n c e . I f you answer a q u e s t i o n , go on t o t h e n e x t one.  can't  Do NOT s k i p a h e a d t o new q u e s t i o n s or go back t o o l d o n e s . T r v t o answer e a c h Q u e s t i o n i n i t s p r o p e r o r d e r .  (140)  END OF TEST  Close  your  test  booklet.  Do n o t go back t o a n y o f t h e q u e s t i o n s . Make s u r e y o u have f i l l e d i n y o u r answer s h e e t name, s e x , g r a d e , and b i r t h d a t e . Make s u r e  that  Turn  the I d e n t i f i c a t i o n  i n your  test  your  No. Box A has a 3 i n  b o o k l e t and answer THANK YOU  with  sheet.  i t .  MATHEMATICS  8 EXAMINATION  (141)  TEST FORMAT 4  INSTRUCTIONS 1.  Do NOT open t h e t e s t b o o k l e t u n t i l you a r e t o l d t o do so. You w i l l have 45 m i n u t e s t o c o m p l e t e t h i s t e s t .  2.  C a r e f u l l y remove t h e answer s h e e t f r o m i n s i d e t h e f r o n t c o v e r and make s u r e t h e r e i s a 4 marked i n box A o f t h e I d e n t i f i c a t i o n No. s e c t i o n o f t h e answer s h e e t .  3.  Be s u r e paper.  4.  F i l l i n y o u r answer and b i r t h d a t e .  5.  Do NOT  6.  For each q u e s t i o n , s e l e c t the best answer. Mark your c h o i c e on t h e answer s h e e t by f i l l i n g i n t h e b u b b l e under t h e c o r r e c t l e t t e r . Make s u r e t h e q u e s t i o n number i s t h e same a s t h e q u e s t i o n number i n t h e t e s t booklet.  7.  Do n o t s p e n d t o o l o n g on a n y one q u e s t i o n . b e s t p i c k a good answer t o e v e r y q u e s t i o n .  8.  I f y o u make a m i s t a k e , c o m p l e t e l y e r a s e y o u r f i r s t c h o i c e and f i l l i n t h e b u b b l e o f y o u r new c h o i c e .  9.  Do NOT w r i t e i n t h e t e s t b o o k l e t . Mark o n l y y o u r answer s h e e t . I f y o u r b o o k l e t a l r e a d y has any i n a p p r o p r i a t e marks, a s k f o r a c l e a n b o o k l e t .  y o u have a p e n c i l ,  sheet  use a c a l c u l a t o r  SPECIAL  an e r a s e r , and some  with  or a  your  scratch  name, s e x , g r a d e ,  protractor.  T r y your  INSTRUCTIONS  1.  You must b e g i n w i t h q u e s t i o n 1. When you have c h o s e n t h e b e s t answer and marked y o u r answer s h e e t , t h e n you must go t o q u e s t i o n 2. When you have f i n i s h e d q u e s t i o n 2, t h e n y o u must go on t o q u e s t i o n 3, t h e n q u e s t i o n 4, t h e n q u e s t i o n 5, and s o on t o t h e end o f t h e t e s t .  2.  T r y e a c h q u e s t i o n once and o n l y o n c e . I f you answer a q u e s t i o n , go on t o t h e n e x t one.  can't  Do NOT s k i p ahead t o new q u e s t i o n s or go back t o o l d o n e s . T r y t o answer e a c h q u e s t i o n i n i t s p r o p e r o r d e r .  (142)  END OF TEST C l o s e your  test  booklet.  Do n o t go back t o a n y o f t h e q u e s t i o n s . Make s u r e y o u have f i l l e d i n y o u r answer s h e e t name, s e x , g r a d e , and b i r t h d a t e . Make s u r e  that  Turn  the I d e n t i f i c a t i o n  i n your  test  your  No. Box A has a 4 i n i t  b o o k l e t and answer THANK YOU  with  sheet.  (143)  APPENDIX III  P-level  Item A n a l y s i s  Data  E a s y t o Hard As G i v e n O r d e r  Item A B C D E F G H I J K L M N 0 P Q R S T U V W X Y Z AA BB CC DD EE FF GG HH II JJ KK LL MM NN  Seq. No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40  Form 1 Unr .  Hard t o E a s y Reversed Order  Form 3 Res.  .913 .906 .866 .832 .678 .765 .812 .624 .604 .705 .651 .470 .631 .617 . 503 .369 . 362 .416 . 490 .456 . 376 . 477 . 523 . 389 . 329 .456 .315 .356 .275 .201 . 362 . 369 .255 .275 . 242 .228 .242 .262 . 161 .181  Note U n r . = U n r e s t r i c t e d ;  (144)  .919 .838 .831 .723 .635 .723 .750 .696 .520 .676 .669 . 466 . 595 .541 . 493 .385 . 399 .351 . 527 .392 . 358 .419 . 527 .378 . 291 .459 .284 .351 . 324 .257 . 29 7 .439 .257 .209 . 223 .250 . 189 .182 .169 .176 Res. =  Seq. No. 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Restricted  Form 2 Unr . .728 .707 .762 .619 . 558 .599 . 565 .531 . 537 .660 .449 .435 . 483 .503 .442 . 374 .272 . 313 .435 . 361 . 367 .429 .435 . 367 . 374 . 306 .293 .354 .272 .204 . 313. . 367 .218 . 238 . 252 . 272 .238 .190 .122 . 245  Form 4 Res . .678 . 562 .651 .521 . 548 . 589 . 527 .445 . 527 . 575 .459 . 384 . 452 . 527 . 404 .288 .418 .342 .438 . 397 . 411 .30 8 . 500 . 329 .432 . 377 . 267 . 390 . 253 .267 . 281 . 370 .240 . 219 .151 .288 . 137 . 205 . 199 . 247  B-value  Item P a r a m e t e r  Estimates  E a s y t o Hard As G i v e n O r d e r  I tern A B C D E F G H I J K L  M N 0 P Q R S T U V  W  X Y Z AA BB CC DD EE FF GG HH II JJ KK LL  MM NN  (145)  Hard t o E a s y Reversed Order  Seq. No.  Form 1 Unr.  Form 3 Res.  Seq. No.  Form 2 Unr.  Form 4 Res.  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40  -2.743 -2.653 -2.206 -1.909 -0.915 -1.427 -1.752 -0.633 -0.531 -1.063 -0.772 0.126 -0.667 -0.599 -0.037 0.633 0.669 0. 392 0.028 0.192 0.598 0.093 -0.135 0 . 529 0. 8 50 0.192 0.925 0.704 1.162 1.657 0.669 0.633 1. 287 1.162 1.374 1. 465 1.374 1.245 1.981 1.812  -3.028 -2.096 -2.037 -1.255 -0.741 -1.255 -1.429 -1.089 -0.128 -0.970 -0.931 0.155 -0.520 -0.234 0.014 0.592 0.517 0.783 -0.163 0.555 0.744 0.407 -0.163 0.630 1.148 0.191 1.191 0.783 0.941 1. 369 1.106 0 . 299 1.369 1.710 1.608 1. 415 1.872 1.929 2.047 1.987  40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1  -1.257 -1.134 -1.476 -0.641 -0.325 -0.535 -0.360 -0.188 -0.222 -0.861 0.233 0.292 0.051 -0.051 0.257 0.611 1.195 0.950 0.292 0.684 0.647 0.327 0.292 0 .647 0.611 0.990 1.070 0 .721 1.195 1.658 0.950 0.647 1.558 1.416 1.325 1.195 1.416 1.762 2.389 1.370  -1.004 -0.357 -0.845 -0.139 -0.284 -0.504 -0.175 0.262 -0.175 -0.430 0.189 0 .602 0.226 -0.175 0.487 1.178 0.411 0 .839 0.299 0.525 0 .449 1.047 -0.030 0.921 0.337 0.641 1.314 0.563 1.409 1.314 1. 223 0 .680 1. 506 1.660 2.259 1.178 2.403 1.768 1.824 1.457  Note U n r . = U n r e s t r i c t e d ;  Res. = R e s t r i c t e d  (146)  APPENDIX  IV  MATHEMATICS 8 EXAMINATION  (147)  TEST FORMAT 3  INSTRUCTIONS 1.  Do NOT open t h e t e s t b o o k l e t u n t i l you a r e t o l d t o do so. You w i l l have 45 m i n u t e s t o c o m p l e t e t h i s t e s t .  2.  C a r e f u l l y remove t h e answer s h e e t f r o m i n s i d e t h e f r o n t c o v e r and make s u r e t h e r e i s a 3 marked i n box A of t h e I d e n t i f i c a t i o n No. s e c t i o n o f t h e answer s h e e t .  3.  Be s u r e paper.  4.  F i l l i n y o u r answer s h e e t and b i r t h d a t e .  5.  Do  6.  For each q u e s t i o n , s e l e c t the best answer. Mark your c h o i c e on t h e answer s h e e t by f i l l i n g i n t h e b u b b l e under t h e c o r r e c t l e t t e r . Make s u r e t h e q u e s t i o n number i s t h e same as t h e q u e s t i o n number i n t h e t e s t booklet.  7.  Do n o t s p e n d t o o l o n g on a n y one b e s t p i c k a good answer t o e v e r y  8.  I f you choice  9.  Do NOT w r i t e i n t h e t e s t b o o k l e t . Mark o n l y y o u r answer s h e e t . I f y o u r b o o k l e t a l r e a d y has any i n a p p r o p r i a t e marks, ask f o r a c l e a n b o o k l e t .  NOT  you  use  have a p e n c i l ,  a calculator  make a m i s t a k e , and f i l l i n t h e  SPECIAL  an  e r a s e r , and  with  or a  scratch  name, s e x ,  grade,  protractor.  question. question.  Try  your  c o m p l e t e l y erase your f i r s t b u b b l e o f y o u r new c h o i c e .  INSTRUCTIONS  1.  You must b e g i n w i t h t h e b e s t answer and must go t o q u e s t i o n 2, t h e n you must go t h e n q u e s t i o n 5, and  2.  T r y e a c h q u e s t i o n once and o n l y answer a q u e s t i o n , go on t o t h e  Do  your  some  q u e s t i o n 1. When you have c h o s e n marked y o u r answer s h e e t , t h e n you 2. When you have f i n i s h e d q u e s t i o n on t o q u e s t i o n 3, t h e n q u e s t i o n 4, so on t o t h e end o f t h e t e s t . once. I f you next one.  can't  NOT s k i p a h e a d t o new q u e s t i o n s or go back t o o l d o n e s . T r v t o answer e a c h q u e s t i o n i n i t s p r o p e r o r d e r .  ITEM  QUESTION  A:  (148)  1  The c i r c l e graph shows the proportions of various grain crops produced by a country. Which of the following statements i s TRUE? •  A  More oats than rye 1s produced.  8  The largest crop is barley.  C  Equal quantities of wheat and barley are produced.  0  The smallest crop 1s oats.  E  Wheat and oats together make up less than half the t o t a l grain crop.  P L E A S E DO NOT TURN BACK TO T H I S  PAGE.  ITEM  B:  QUESTION  (149)  2  162 x 45 is equal to  •  A  1378  B  1458  C  5890  D  6290  E  7290  PLEASE DO NOT TURN BACK TO THIS  PAGE  ITEM  QUESTION  C:  3  (150)  A team scores an average of 3 points per game over 5 games. How many points altogether were scored in the 5 games.  •  C  3  0  5  E  15  PLEASE DO NOT TURN BACK TO THIS  PAGE  ITEM  QUESTION  D:  4  In a discus-throwing competition, the winning throw was 61.60 metres. The second place throw was 59.72 metres. How much longer was the winning throw than the second place throw?  «  A.  1.12 metres  B.  1.88 metres  C.  1.92 metres  D-  2.12 metres  E.  121.32 metres  PLEASE DO NOT TURN BACK TO THIS PAGE.  (151)  I  T  E  M  E  QUESTION  :  If. 10  «  2  x 10  A  4  B  5  C  6  0  8  E  9  3  = 10  n  5  then n is equal to  PLEASE DO NOT TURN  BACK TO THIS PAGE  (152)  QUESTION 6  A group o f c h i l d r e n was d i v i d e d i n t o 7 teams w i t h nine i n each team. Later, the same group o f c h i l d r e n was d i v i d e d i n t o teams w i t h seven i n each team. How many teams were t h e r e then?  A  7  B  8  C  9  D  16  E  63  PLEASE DO NOT  TURN BACK TO THIS PAGE  ITEM  QUESTION  G:  Here 1s a table that shows the number of trees planted along a highway In a week. Days of the Week  Mon  Tues  Wed  Thurs  Tu«a  V«d  thurs  A  P  B  Q  C  R  0  S  E  T  Frl  PLEASE DO NOT  ( 154 )  If the graph were completed, which point would indicate the top of the bar on Thursday?  FH  Number of Trees 75 50 90 60 Planted 80 On the diagram below, the graph for the f i r s t two days' plantings has been'  Hon  7  TURN BACK TO THIS PAGE  ITEM  QUESTION  H:  8  What is the volume of a rectangular box with interior dimensions 10 cm long, 10 cm wide, and 7 cm high?  0  A  21 cm  B  70 cm  C  140 cm  0  280 cm  E  700 cm  3  3  3  3  3  PLEASE DO NOT TURN  BACK TO THIS PAGE  (155)  ITEM  I:  QUESTION  9  If the ratio of 2 to 5 equals the ratio of n to 100, then n is equal to  •  A  10  B  20  C  40  0  150  E  250  PLEASE DO NOT TURN BACK TO THIS PAGE.  (156)  ITEM  J:  QUESTION  (157)  10  A  20  B  40  C  50  0  80  E  100  5x / 4r B  » I f AB i s a s t r a i g h t l i n e , what i s the measure i n degrees o f a n g l e BCD?  PLEASE DO NOT TURN BACK TO THIS  PAGE  ITEM  QUESTION  K:  (158)  11  In a school of 800 pupils, 300 are boys The ratio of the number of boys to the number of girls is  A.  3:8  B.  5:8  C.  3 : 11  D.  5:3  E.  3:5  PLEASE  DO  NOT  TURN BACK TO THIS  PAGE  ITEM  L:  QUESTION  12  Which of the following equals 7 x (3 + 9)?  •  A  (7 x 3) + (7 x 9)  B  (7 x 9) + (3 x 9)  C  (7 x 3) + (3 x 9)  0  7 x 27  E  21+9  PLEASE DO NOT TURN BACK TO THIS PAGE  (159)  ITEM  M:  QUESTION  13  (160)  0.40 x 6.38 i s equal to  A.  ,  .2552  B.  2.452  C.  2.552  D.  24.52  E.  25.52  PLEASE DO NOT TURN BACK TO THIS  PAGE  ITEM  N:  QUESTION  When x = 2,  14  * \ = is equal to  11  B  C  Ii 5  9 5 7  PLEASE DO NOT TURN BACK TO THIS PAGE.  (161)  ITEM  0:  QUESTION  15  (162)  20 m  15 a  A square i s removed from the r e c t a n g l e as shown. What i s tne area o f the remaining part?  PLEASE  DO  NOT  TURN  BACK  TO  A.  316 ni*  e.  300 m*  c.  284-  0.  80 m  E.  16 m  THIS  2  2  PAGE  ITEM  P:  QUESTION  16  (163)  If segment 'RJ' were drawn for each figure shown below, 1t would divide one of the figures into two congruent triangles. Which figure?  • • '/zu  * ^  Q  B  C  PLEASE  DO  NOT  TURN  BACK  TQ  THIS  P A O E  ITEM  Q:  QUESTION  7 ^jj  •  17  (164)  is equal to  A.  7.03  B.  7.15  C  7.23  0.  7.3  E.  7.6  PLEASE DO NOT TURN  BACK TO THIS  PAGE  ITEM  QUESTION  R:  (165)  18  What is the square root of 12 x 75? A  6.25  B  30  C  87  D  625  E  900  PLEASE  DO NOT TURN  BACK TO THIS  PAGE  ITEM  QUESTION  S:  (166)  19  If x = - 3, the value of -3r is  «  PLEASE  D  1  E  9  DO  NOT TURN BACK TO THIS  PAGE  ITEM  T:  QUESTION  20  How many pieces of pipe each 20 metres long would be required to construct a pipeline 1 kilometre in length? '  •  A  5  B  50  C  500  D  5000  E  50,000  PLEASE DO NOT TURN BACK TO THIS PAGE  (167)  ITEM  QUESTION  U:  (168)  21  2 metres + 3 millimetres i s equal to  A.  2.0003 metres  B.  2.003 metres  C.  2.03 metres  D.  2.3  E.  5 metres  metres  PLEASE DO NOT  TURN BACK  TO THIS PAGE  ITEM  V:  QUESTION  22  8.S n  (169)  A.  48 m  B.  54 m2  C.  56 m  D.  63 m.2  E.  72 m*  2  5.9 m  Which of the following is the closest approximation to the area of the rectangle with measurements given?  PLEASE DO NOT TURN  BACK TO THIS PAGE  2  (170)  Three hours after starting, car A is how many kilometres ahead of car B?  PLEASE  DO NOT  TURN  BACK  TO T H I S  PAGP.  QUESTION  24  The arithmetic mean (average) 1.50, 2.40, 3.75 is equal to A  2.40  B  2.55  C  3.75  0  7.65  E  None of these  PLEASE DO NOT  of:  TURN BACK TO THIS  PAGE.  ITEM  Y:  QUESTION  25  PLEASE DO NOT TURN BACK TO.THIS PAGE  (172)  ITEM  Z:  QUESTION  x - y  I f  then  •  J  "  =  2  A  - 2  B  -1  z  =1  26  ,  i s equal  to  C  0  PLEASE DO NOT  TURN BACK TO THIS PAGE.  (173)  ITEM  2 ca  AA:  QUESTION  27  (174)  2 cm  2 aa  The area in square centimetres of this figure is  2 ca  8 cm  2  The rectangle shown above is cut along the dotted lines and the three parts put together, without overlapping, to give the figure shown below. 0  B  10 cm  C  12 cn  2  2  14 cm  2  16 cm  2  PLEASE  DO  NOT  TURN  BACK  TO THIS  PAHE-  ITEM  QUESTION  BB:  28  10 ca  "  What 1s the area of the above parallelogram?  •  A  30 cm  8  36 cm  C  48 on  2  0  60 cm  2  E  80 cm  2  2  2  PLEASE DO NOT TURN BACK TO THIS PAGE  (175)  ITEM  CC:  QUESTION  29  (176)  Suppose you start at point M(-l,-1), move a distance of one unit to N(-l,-2), then turn left and move one unit to the point P10.-2). If you again turn left and move one unit, you will now be at the point with coordinates  0. -2)  *  PLEASE  8  (0, -3)  C  (0, -1)  0  (-1.  E  None of the above  DO NOT TURN BACK  -2)  TO THTS  Panp  ITEM  DD:  QUESTION  30  0.00046 is equal to  A.  46 x TO"  B.  4.6 x 10" *  C  0.46 x 10  D.  4.6 x 10*  E.  46 x 10*  3  1  3  PLEASE DO NOT TURN BACK TO THIS PAGE.  (177)  ITEM  QUESTION  EE:  31  If, in the given figure PQ and RS are intersecting straight lines, then x + y is equal to  •  PLEASE  A  15  B  30  C  60  0  180  E  300  DO NOT TURN BACK TO T H I S P A G E .  (178)  ITEM  FF:  QUESTION  32  The table below compares the height from which a ball Is dropped (d) and the height to which i t bounces (b).  d  50  80  100  130  b  25  40  50  75  Which formula describes this relation?  *  A  b  -  d  B  b  -  2d  C  b- |  D  b- d *  25  E  b  25  -  l  d -  PLEASE DO NOT  TURN BACK TO THIS PAGE  (179 )  QUESTION  ITEM GG:  33  (180)  3  Si  NO  8 ca  8  ca  The total area of the two triangles is A  6 x 8 cm  B  1*1  C  2  2  10 x 6  0 E  —r~  rmZ  cm  2  cm 16 x 12 2~ 20 x 12 cm  2  2  PLEASE DO N O T T U R N  BACK T O THIS PAGE  ITEM  HH:  QUESTION  34  (181)  PQRS 1s a rectangle. Its Image after a transformation Is the rectangle P ' Q ' R ' S ' , as shown above. The transformation used could have been •  H*  Q'  PLEASE  DO N O T T U R N  A  a rotation about the o r i g i n .  6  a reflection in the y-axis  C  a translation parallel the z-axis  0  a reflection in the z-axis  E  a translation parallel the y - a x i s .  BACK  TO T H I S  PAfiF  to  to  ITEM  QUESTION  II:  (182)  35  One o f t h e f o l l o w i n g p o i n t s can be j o i n e d to t h e p o i n t (-3,4) by a l i n e segment which cuts NEITHER t h e * NOR t h e y a x i s . Which one? •  A  (-2,3)  B  (2,-3)  C  (2,3)  D  (-2,-3.)  E  (4,-3)  PLEASE DO NOT TURN BACK  TO THIS PAGE.  ITEM  QUESTION  JJ :  36  In a quadrilateral, two of the angles each have measure of 1 1 0 ° , and the measure of a third angle is 9 0 ° . What is the measure of the remaining angle? •  A  50°  B  90°  C  130°  D  140°  E  None of the above  PLEASE DO NOT TURN BACK TO THIS PAGE.  ( 1 8 3 )  ITEM  KK:  QUESTION  The symbol P ^ q represents the intersection of sets P and Q and the symbol P u Q represents the union of sets P and Q. Which of the following represents the shaded portion of the diagram below?  37  ,  (184)  to nn\ ^ "  w ;  ,  g  w  u a *  PU(qnR)  P A (Q U R) (P n 0) n R (P u Q ) n R  PLEASE  DO N O T T U R N  BACK  TO T H I S  PAGE  ITEM  QUESTION  LL:  38  There are 7,000,000 girls under the age of 21 in a country with a total population of 36,000,000. If a c i r c l e graph were drawn showing the d i s t r i bution of the population, the angle in the sector representing girls under the age of 21 would have measure  •  A  7°  B  20°  C  21°  0  70°  E  72°  PLEASE  DO NOT TURN BACK TO THIS PAGE  (185)  ITEM  MM:  QUESTION  If  5a: + 4 « 4x - 31 , then  x is equal  '  (186)  39  A.  -35  B.  -27  C.  3  D.  27  E.  35  PLEASE  to  DO NOT TURN BACK TO THIS  PAGE  ITEM  G i v e n v and is  DB,  NN:  w  QUESTION  as shown i n the  the v e c t o r  figure  40  above, what  from 0 to B  PLEASE DO  NOT  TURN BACK TO THIS PAGE  (188)  END OF TEST Close your t e s t b o o k l e t . Do not go back to any of the q u e s t i o n s . Make sure you have f i l l e d i n your answer sheet with your name, sex, grade, and b i r t h date. Make sure t h a t the I d e n t i f i c a t i o n Turn  No. Box A has a 3 i n i t .  i n your t e s t booklet and answer sheet. THANK YOU  AUTHOR INDEX  (190) Author Ahmann & G l o c k (1963) A l l i s o n (1984) A l l i s o n & Thomas (1986) B e r g e r e t a l . (1969) B r e n n e r (1964) C a m p b e l l & S t a n l e y (1963) C r o n b a c h (1946) C r o n b a c h (1950) F e l d t & F o r s y t h (1974) F l a u g h e r e t a l . (1968) F r e n c h & G r e e r (1964) H a m b l e t o n & T r a u b ( 1974)  Index  1-2,13,28,48 3,7,58-61,73 62-63,79,104 4,23-24, 44,73 3,12-13,73 110,111 57 57 50-52,73,107 3,10, 15-16,73 4,17-19,73 3,6-7,10,52-57,58, 59,60,61,62 73,76,78,79,88,104,107,109 Hodson (1984) 3,7,10, 30-32,45,46,73 H o p k i n s & A n t e s (1985) 1 Huck & Bowers (1972) 3,46-48,73 K e s t e n b a u m & W e i n e r (1970 )....3,29-30,73 K i n g s t o n & D o r a n s (1984) 4,68-69, 73,75,81,107 K l e i n k e (1980) 3,42-44,45,46,73 K l i m k o (1984) 4,39-40,44,45,62,73,79 K l o s n e r & G e l l m a n (1973) 3,7,28-29,45,73 Lane e t a l . (1987) 3, 10, 40-42,44, 45,73 L e a r y & D o r a n s (1985) 2,10,71,74,76 M a c N i c o l (1956) 3,10-11,73 Marso (1970) 4,25-26,44,73 M i l l m a n & B i s h o p ( 1965) 61,79 M o l l e n k o p f (1950) 3,9-10,54,73 Monk & S t a l l i n g s ( 1970) 3, 10,57-58,72,73 Munz & J a c o b s (1971) 4,22-23,28,73 Munz & Smouse (1968) 4,20-21,22,23,26,27,28, 44,54, 56,73 P l a k e (1980) 4,10, 32-33, 44,73 P l a k e & A n s o r g e (1984) 4,37-38,45,46,73 P l a k e , A n s o r g e e t a l . (1982)..4,35-37,44,45,54,56,73,107 P l a k e , M e l i c a n e t a l . (1983)..4,37-38,45,73 P l a k e , Thompson e t a l . ( 1 9 8 0 ) . 4, 33-35, 36,44, 54,73 R i n d l e r (1980) 64-65,79,105 R o b i t a i l l e & Garden (1987)....85 Ruch (1929 ) 1 Sax & C a r r (1962) 3,11-12,13,73 Sax & Cromack (1966) 3,7,13-15,28,45,73 S i r o t n i k & W e l l i n g t o n ( 1974 ) . . 3, 48-50, 73 Smouse & Munz (1968) 4,19-20, 44,73 Smouse & Munz ( 1969) 4,21-22,28,44,73 Towle & M e r r i l l (1975) 4,26-27, 33,44,46,60,73,107 Tuck (1978) 58,62,79 W h i t e l y & Dawis (1976) 4,66-68,73, 75,81 Yen (1980) 4,69-71,7 3,7 5,81,106,107  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0098329/manifest

Comment

Related Items